OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	121.55
►Process 304350+		1	121.55
○Thread 304350			121.55	123.35	98.54	0.00	0.00	123.34

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	61.87
►Process 304484+		2	61.87
○Thread 304484			61.87	63.71	97.10	0.00	0.00	61.45
○Thread 304551			61.11	61.46	99.43	2.25	0.00	61.19

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	32.01
►Process 304622+		4	32.01
○Thread 304622			32.01	33.82	94.66	0.00	0.00	31.54
○Thread 304689			31.14	31.55	98.73	2.28	0.00	31.27
○Thread 304690			31.13	31.55	98.70	2.28	0.00	31.27
○Thread 304691			31.12	31.55	98.67	2.28	0.00	31.27

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	17.15
►Process 304718+		8	17.15
○Thread 304718			17.15	18.92	90.64	0.00	0.00	16.64
○Thread 304785			16.20	16.65	97.30	2.27	0.00	16.38
○Thread 304786			16.12	16.65	96.78	2.27	0.00	16.38
○Thread 304787			16.13	16.65	96.87	2.27	0.00	16.38
○Thread 304788			16.16	16.65	97.02	2.27	0.00	16.38
○Thread 304789			16.17	16.65	97.08	2.27	0.00	16.38
○Thread 304790			16.15	16.65	96.99	2.27	0.00	16.36
○Thread 304791			16.21	16.65	97.35	2.27	0.00	16.38

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	9.85
►Process 304860+		16	9.85
○Thread 304860			9.85	11.82	83.32	0.00	0.00	9.54
○Thread 304927			8.91	9.55	93.29	2.27	0.00	9.28
○Thread 304928			8.97	9.55	93.92	2.27	0.00	9.28
○Thread 304929			8.93	9.55	93.50	2.27	0.00	9.28
○Thread 304930			8.93	9.55	93.55	2.27	0.00	9.28
○Thread 304931			8.95	9.55	93.76	2.27	0.00	9.28
○Thread 304932			8.97	9.55	93.97	2.27	0.00	9.28
○Thread 304933			8.95	9.55	93.70	2.27	0.00	9.27
○Thread 304934			8.88	9.55	93.02	2.27	0.00	9.28
○Thread 304935			8.87	9.55	92.92	2.27	0.00	9.28
○Thread 304936			8.90	9.55	93.18	2.27	0.00	9.28
○Thread 304937			8.89	9.55	93.07	2.27	0.00	9.27
○Thread 304938			8.90	9.55	93.22	2.27	0.00	9.27
○Thread 304939			8.91	9.55	93.33	2.27	0.00	9.28
○Thread 304940			8.93	9.55	93.54	2.27	0.00	9.27
○Thread 304941			8.94	9.55	93.59	2.27	0.00	9.28

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	8.09
►Process 304961+		24	8.09
○Thread 304961			8.09	10.15	79.69	0.00	0.00	7.86
○Thread 305028			7.15	7.88	90.71	2.27	0.01	7.60
○Thread 305029			7.26	7.88	92.17	2.27	0.01	7.61
○Thread 305030			7.28	7.88	92.42	2.27	0.01	7.61
○Thread 305031			7.12	7.88	90.39	2.27	0.01	7.61
○Thread 305032			7.21	7.88	91.53	2.27	0.01	7.60
○Thread 305033			7.15	7.88	90.71	2.27	0.01	7.60
○Thread 305034			7.08	7.88	89.82	2.27	0.01	7.61
○Thread 305035			7.11	7.88	90.26	2.27	0.00	7.60
○Thread 305036			7.13	7.88	90.51	2.27	0.00	7.60
○Thread 305037			7.17	7.88	91.02	2.27	0.00	7.61
○Thread 305038			7.10	7.88	90.06	2.27	0.00	7.61
○Thread 305039			7.24	7.88	91.84	2.27	0.00	7.60
○Thread 305040			7.15	7.88	90.69	2.27	0.00	7.59
○Thread 305041			7.11	7.88	90.25	2.27	0.00	7.60
○Thread 305042			7.21	7.88	91.45	2.27	0.00	7.60
○Thread 305043			7.19	7.88	91.26	2.27	0.00	7.60
○Thread 305044			7.08	7.88	89.86	2.27	0.00	7.60
○Thread 305045			7.16	7.88	90.87	2.27	0.00	7.61
○Thread 305046			7.17	7.88	91.00	2.27	0.00	7.60
○Thread 305047			7.22	7.88	91.63	2.27	0.00	7.61
○Thread 305048			7.12	7.88	90.30	2.27	0.00	7.60
○Thread 305049			7.21	7.88	91.44	2.27	0.00	7.61
○Thread 305050			7.03	7.88	89.15	2.27	0.00	7.61

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	7.23
►Process 305070+		32	7.23
○Thread 305070			7.23	9.31	77.65	0.00	0.00	7.02
○Thread 305137			6.32	7.03	89.87	2.27	0.01	6.76
○Thread 305138			6.32	7.03	89.79	2.27	0.01	6.76
○Thread 305139			6.36	7.03	90.36	2.27	0.01	6.76
○Thread 305140			6.29	7.03	89.44	2.27	0.01	6.76
○Thread 305141			6.26	7.03	89.01	2.27	0.01	6.76
○Thread 305142			6.31	7.03	89.64	2.27	0.01	6.76
○Thread 305143			6.28	7.03	89.22	2.27	0.01	6.76
○Thread 305144			6.37	7.03	90.49	2.27	0.01	6.76
○Thread 305145			6.21	7.03	88.29	2.27	0.01	6.76
○Thread 305146			6.34	7.03	90.06	2.27	0.01	6.76
○Thread 305147			6.28	7.03	89.28	2.27	0.01	6.76
○Thread 305148			6.33	7.03	89.92	2.27	0.01	6.76
○Thread 305149			6.20	7.03	88.07	2.27	0.01	6.76
○Thread 305150			6.20	7.03	88.06	2.27	0.01	6.76
○Thread 305151			6.29	7.03	89.41	2.27	0.01	6.76
○Thread 305152			6.39	7.03	90.76	2.27	0.00	6.76
○Thread 305153			6.25	7.04	88.84	2.27	0.00	6.75
○Thread 305154			6.32	7.04	89.76	2.27	0.00	6.76
○Thread 305155			6.28	7.04	89.26	2.27	0.00	6.75
○Thread 305156			6.24	7.04	88.69	2.27	0.00	6.76
○Thread 305157			6.21	7.04	88.26	2.27	0.00	6.76
○Thread 305158			6.20	7.04	88.12	2.27	0.00	6.76
○Thread 305159			6.28	7.04	89.25	2.27	0.00	6.76
○Thread 305160			6.24	7.04	88.61	2.27	0.00	6.75
○Thread 305161			6.13	7.04	87.12	2.27	0.00	6.76
○Thread 305162			6.23	7.04	88.54	2.27	0.00	6.75
○Thread 305163			6.31	7.04	89.67	2.27	0.00	6.75
○Thread 305164			6.28	7.04	89.17	2.27	0.00	6.76
○Thread 305165			6.26	7.04	88.96	2.27	0.00	6.76
○Thread 305166			6.23	7.04	88.46	2.27	0.00	6.75
○Thread 305167			6.11	7.04	86.82	2.27	0.00	6.76

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	6.58
►Process 305237+		40	6.58
○Thread 305237			6.58	8.84	74.41	0.00	0.00	6.54
○Thread 305304			5.75	6.55	87.85	2.28	0.01	6.27
○Thread 305305			5.60	6.55	85.55	2.28	0.01	6.26
○Thread 305306			5.72	6.55	87.38	2.28	0.01	6.27
○Thread 305307			5.66	6.55	86.47	2.28	0.01	6.27
○Thread 305308			5.67	6.55	86.62	2.28	0.01	6.27
○Thread 305309			5.72	6.55	87.30	2.28	0.01	6.27
○Thread 305310			5.64	6.55	86.15	2.28	0.01	6.26
○Thread 305311			5.61	6.55	85.62	2.28	0.01	6.26
○Thread 305312			5.71	6.55	87.14	2.28	0.01	6.26
○Thread 305313			5.66	6.55	86.38	2.28	0.01	6.27
○Thread 305314			5.68	6.55	86.76	2.28	0.01	6.27
○Thread 305315			5.69	6.55	86.91	2.28	0.01	6.28
○Thread 305316			5.70	6.55	87.06	2.28	0.01	6.27
○Thread 305317			5.63	6.55	85.99	2.28	0.01	6.27
○Thread 305318			5.71	6.55	87.21	2.28	0.01	6.27
○Thread 305319			5.75	6.55	87.82	2.28	0.01	6.26
○Thread 305320			5.59	6.55	85.30	2.28	0.01	6.27
○Thread 305321			5.67	6.55	86.59	2.28	0.01	6.26
○Thread 305322			5.70	6.55	87.05	2.28	0.01	6.27
○Thread 305323			5.57	6.55	85.06	2.28	0.01	6.27
○Thread 305324			5.73	6.55	87.43	2.28	0.01	6.28
○Thread 305325			5.64	6.55	86.13	2.28	0.01	6.27
○Thread 305326			5.68	6.55	86.66	2.28	0.00	6.27
○Thread 305327			5.64	6.55	86.12	2.28	0.00	6.27
○Thread 305328			5.66	6.55	86.35	2.28	0.00	6.28
○Thread 305329			5.63	6.55	85.97	2.28	0.00	6.28
○Thread 305330			5.57	6.55	85.05	2.28	0.00	6.28
○Thread 305331			5.64	6.55	86.04	2.28	0.00	6.27
○Thread 305332			5.69	6.55	86.88	2.28	0.00	6.27
○Thread 305333			5.63	6.55	85.88	2.28	0.00	6.26
○Thread 305334			5.56	6.55	84.89	2.28	0.00	6.28
○Thread 305335			5.87	6.55	89.62	2.28	0.00	6.27
○Thread 305336			5.66	6.55	86.41	2.28	0.00	6.27
○Thread 305337			5.66	6.55	86.41	2.28	0.00	6.27
○Thread 305338			5.75	6.55	87.78	2.28	0.00	6.27
○Thread 305339			5.73	6.55	87.40	2.28	0.00	6.27
○Thread 305340			5.61	6.55	85.56	2.28	0.00	6.27
○Thread 305341			5.50	6.55	83.96	2.28	0.00	6.26
○Thread 305342			5.51	6.55	84.03	2.28	0.00	6.28

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	6.34
►Process 305362+		48	6.34
○Thread 305362			6.34	8.51	74.50	0.00	0.00	6.28
○Thread 305429			5.34	6.28	85.06	2.22	0.01	6.00
○Thread 305430			5.40	6.28	86.02	2.22	0.01	6.00
○Thread 305431			5.45	6.28	86.73	2.22	0.01	6.01
○Thread 305432			5.37	6.28	85.45	2.22	0.01	6.00
○Thread 305433			5.33	6.28	84.89	2.22	0.01	6.01
○Thread 305434			5.33	6.28	84.89	2.22	0.01	6.00
○Thread 305435			5.30	6.28	84.33	2.22	0.01	6.00
○Thread 305436			5.31	6.28	84.49	2.22	0.01	6.00
○Thread 305437			5.28	6.28	84.09	2.22	0.01	6.00
○Thread 305438			5.33	6.28	84.81	2.22	0.01	6.00
○Thread 305439			5.30	6.28	84.33	2.22	0.01	6.00
○Thread 305440			5.55	6.28	88.30	2.22	0.01	6.00
○Thread 305441			5.33	6.28	84.80	2.22	0.01	6.00
○Thread 305442			5.32	6.28	84.72	2.22	0.01	6.00
○Thread 305443			5.32	6.28	84.72	2.22	0.01	6.00
○Thread 305444			5.35	6.28	85.19	2.22	0.01	5.99
○Thread 305445			5.32	6.28	84.63	2.22	0.01	6.00
○Thread 305446			5.22	6.28	83.04	2.22	0.01	5.99
○Thread 305447			5.28	6.28	84.07	2.22	0.01	6.00
○Thread 305448			5.21	6.28	82.95	2.22	0.01	6.00
○Thread 305449			5.26	6.28	83.75	2.22	0.01	6.00
○Thread 305450			5.29	6.28	84.22	2.22	0.01	6.00
○Thread 305451			5.27	6.28	83.82	2.22	0.01	5.99
○Thread 305452			5.32	6.28	84.70	2.22	0.01	6.00
○Thread 305453			5.33	6.28	84.85	2.22	0.01	6.00
○Thread 305454			5.28	6.28	84.06	2.22	0.01	5.99
○Thread 305455			5.28	6.28	83.98	2.22	0.01	6.00
○Thread 305456			5.41	6.28	86.04	2.22	0.01	5.99
○Thread 305457			5.27	6.28	83.81	2.22	0.01	6.01
○Thread 305458			5.26	6.28	83.73	2.22	0.01	6.00
○Thread 305459			5.24	6.28	83.33	2.22	0.00	6.00
○Thread 305460			5.42	6.28	86.19	2.22	0.00	6.00
○Thread 305461			5.29	6.28	84.12	2.22	0.00	6.00
○Thread 305462			5.22	6.28	83.09	2.22	0.00	5.99
○Thread 305463			5.42	6.28	86.19	2.22	0.00	6.00
○Thread 305464			5.29	6.28	84.12	2.22	0.00	6.00
○Thread 305465			5.22	6.28	83.00	2.22	0.00	6.00
○Thread 305466			5.27	6.28	83.79	2.22	0.00	6.00
○Thread 305467			5.32	6.28	84.59	2.22	0.00	6.00
○Thread 305468			5.32	6.28	84.67	2.22	0.00	6.00
○Thread 305469			5.15	6.28	81.96	2.22	0.00	5.99
○Thread 305470			5.25	6.28	83.55	2.22	0.00	6.00
○Thread 305471			5.21	6.28	82.91	2.22	0.00	6.00
○Thread 305472			5.33	6.28	84.74	2.22	0.00	6.00
○Thread 305473			5.17	6.28	82.27	2.22	0.00	6.00
○Thread 305474			5.12	6.28	81.39	2.22	0.00	6.00
○Thread 305475			4.88	6.28	77.57	2.22	0.00	6.00

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	5.90
►Process 305495+		56	5.90
○Thread 305495			5.90	8.36	70.52	0.00	0.00	6.10
○Thread 305562			4.96	6.11	81.09	2.23	0.02	5.83
○Thread 305563			5.07	6.11	82.88	2.23	0.02	5.82
○Thread 305564			5.10	6.11	83.37	2.23	0.01	5.83
○Thread 305565			5.19	6.11	84.93	2.23	0.01	5.83
○Thread 305566			5.01	6.11	81.90	2.23	0.01	5.81
○Thread 305567			5.14	6.11	84.02	2.23	0.01	5.83
○Thread 305568			5.10	6.11	83.36	2.23	0.01	5.83
○Thread 305569			5.02	6.11	82.14	2.23	0.01	5.83
○Thread 305570			5.00	6.11	81.72	2.23	0.01	5.83
○Thread 305571			4.96	6.11	81.15	2.23	0.01	5.83
○Thread 305572			5.02	6.11	82.13	2.23	0.01	5.82
○Thread 305573			5.12	6.11	83.68	2.23	0.01	5.82
○Thread 305574			5.01	6.11	81.88	2.23	0.01	5.83
○Thread 305575			5.09	6.11	83.19	2.23	0.01	5.83
○Thread 305576			4.91	6.11	80.24	2.23	0.01	5.82
○Thread 305577			5.01	6.11	81.96	2.23	0.01	5.83
○Thread 305578			4.93	6.11	80.64	2.23	0.01	5.83
○Thread 305579			4.90	6.11	80.07	2.23	0.01	5.83
○Thread 305580			4.96	6.11	81.13	2.23	0.01	5.83
○Thread 305581			4.97	6.11	81.29	2.23	0.01	5.83
○Thread 305582			5.00	6.11	81.70	2.24	0.01	5.80
○Thread 305583			4.93	6.11	80.64	2.24	0.01	5.81
○Thread 305584			5.02	6.11	82.02	2.24	0.01	5.81
○Thread 305585			4.94	6.11	80.79	2.24	0.01	5.83
○Thread 305586			4.84	6.11	79.16	2.24	0.01	5.82
○Thread 305587			4.85	6.11	79.32	2.24	0.01	5.81
○Thread 305588			4.87	6.11	79.64	2.24	0.01	5.83
○Thread 305589			5.04	6.11	82.34	2.24	0.01	5.83
○Thread 305590			5.08	6.11	83.07	2.24	0.01	5.82
○Thread 305591			4.94	6.12	80.70	2.24	0.01	5.81
○Thread 305592			5.01	6.12	81.84	2.24	0.01	5.83
○Thread 305593			4.97	6.12	81.19	2.24	0.01	5.82
○Thread 305594			4.96	6.12	81.10	2.24	0.01	5.83
○Thread 305595			4.96	6.12	81.10	2.24	0.01	5.82
○Thread 305596			4.99	6.12	81.51	2.24	0.01	5.81
○Thread 305597			5.05	6.12	82.65	2.24	0.01	5.81
○Thread 305598			4.93	6.12	80.60	2.24	0.01	5.81
○Thread 305599			4.90	6.12	80.11	2.24	0.01	5.81
○Thread 305600			4.86	6.12	79.46	2.24	0.01	5.83
○Thread 305601			5.08	6.12	82.97	2.24	0.00	5.83
○Thread 305602			4.90	6.12	80.02	2.24	0.00	5.82
○Thread 305603			4.95	6.12	80.84	2.24	0.00	5.82
○Thread 305604			4.86	6.12	79.37	2.24	0.00	5.83
○Thread 305605			5.01	6.12	81.82	2.24	0.00	5.82
○Thread 305606			4.97	6.12	81.24	2.24	0.00	5.83
○Thread 305607			4.91	6.12	80.26	2.24	0.00	5.83
○Thread 305608			4.99	6.12	81.48	2.24	0.00	5.83
○Thread 305609			5.31	6.12	86.79	2.24	0.00	5.82
○Thread 305610			5.00	6.12	81.72	2.24	0.00	5.82
○Thread 305611			5.00	6.12	81.72	2.24	0.00	5.83
○Thread 305612			5.13	6.12	83.84	2.24	0.00	5.82
○Thread 305613			5.00	6.12	81.64	2.24	0.00	5.82
○Thread 305614			4.88	6.12	79.67	2.24	0.00	5.83
○Thread 305615			4.64	6.12	75.83	2.24	0.00	5.81
○Thread 305616			4.71	6.12	76.89	2.24	0.00	5.82

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	5.77
►Process 305636+		64	5.77
○Thread 305636			5.77	8.35	69.07	0.00	0.00	6.09
○Thread 305703			4.90	6.11	80.17	2.23	0.02	5.82
○Thread 305704			4.98	6.11	81.47	2.23	0.02	5.83
○Thread 305705			4.86	6.11	79.51	2.23	0.02	5.83
○Thread 305706			4.86	6.11	79.51	2.23	0.02	5.83
○Thread 305707			4.93	6.11	80.73	2.23	0.02	5.82
○Thread 305708			4.92	6.11	80.57	2.23	0.02	5.83
○Thread 305709			4.87	6.11	79.66	2.23	0.02	5.80
○Thread 305710			4.87	6.11	79.66	2.23	0.02	5.80
○Thread 305711			4.84	6.11	79.17	2.23	0.02	5.82
○Thread 305712			4.81	6.11	78.67	2.23	0.02	5.81
○Thread 305713			4.89	6.11	79.98	2.23	0.01	5.82
○Thread 305714			4.85	6.11	79.49	2.23	0.01	5.82
○Thread 305715			4.82	6.11	78.91	2.23	0.01	5.83
○Thread 305716			4.83	6.11	79.08	2.23	0.01	5.82
○Thread 305717			4.82	6.11	78.91	2.23	0.01	5.82
○Thread 305718			4.85	6.11	79.32	2.23	0.01	5.83
○Thread 305719			4.75	6.11	77.84	2.23	0.01	5.83
○Thread 305720			4.83	6.11	79.07	2.23	0.01	5.82
○Thread 305721			4.83	6.11	79.07	2.23	0.01	5.81
○Thread 305722			4.85	6.11	79.39	2.23	0.01	5.82
○Thread 305723			4.87	6.11	79.63	2.23	0.01	5.82
○Thread 305724			4.69	6.11	76.77	2.23	0.01	5.82
○Thread 305725			4.83	6.11	78.98	2.23	0.01	5.82
○Thread 305726			4.72	6.11	77.26	2.23	0.01	5.82
○Thread 305727			4.90	6.11	80.20	2.23	0.01	5.81
○Thread 305728			4.90	6.11	80.12	2.23	0.01	5.82
○Thread 305729			4.83	6.11	79.05	2.23	0.01	5.82
○Thread 305730			4.91	6.11	80.28	2.23	0.01	5.82
○Thread 305731			4.76	6.11	77.90	2.23	0.01	5.82
○Thread 305732			4.76	6.11	77.90	2.23	0.01	5.83
○Thread 305733			4.80	6.11	78.47	2.23	0.01	5.83
○Thread 305734			4.98	6.11	81.41	2.23	0.01	5.83
○Thread 305735			4.87	6.11	79.61	2.23	0.01	5.82
○Thread 305736			4.93	6.11	80.59	2.23	0.01	5.83
○Thread 305737			4.94	6.11	80.75	2.23	0.01	5.83
○Thread 305738			4.95	6.11	80.99	2.23	0.01	5.82
○Thread 305739			4.76	6.11	77.80	2.23	0.01	5.82
○Thread 305740			4.77	6.11	78.04	2.23	0.01	5.82
○Thread 305741			4.76	6.11	77.80	2.23	0.01	5.82
○Thread 305742			4.82	6.11	78.78	2.23	0.01	5.81
○Thread 305743			4.81	6.11	78.69	2.23	0.01	5.82
○Thread 305744			4.83	6.11	78.94	2.23	0.01	5.83
○Thread 305745			4.75	6.11	77.71	2.23	0.01	5.82
○Thread 305746			5.00	6.11	81.80	2.24	0.01	5.83
○Thread 305747			4.94	6.11	80.73	2.24	0.01	5.82
○Thread 305748			4.88	6.11	79.83	2.24	0.01	5.82
○Thread 305749			4.79	6.11	78.35	2.24	0.01	5.82
○Thread 305750			5.25	6.11	85.88	2.24	0.00	5.83
○Thread 305751			4.93	6.11	80.56	2.24	0.00	5.83
○Thread 305752			4.85	6.11	79.25	2.24	0.00	5.83
○Thread 305753			4.98	6.11	81.45	2.24	0.00	5.83
○Thread 305754			4.95	6.11	80.88	2.24	0.00	5.83
○Thread 305755			4.85	6.11	79.24	2.24	0.00	5.83
○Thread 305756			4.90	6.11	80.14	2.24	0.00	5.82
○Thread 305757			4.91	6.11	80.22	2.24	0.00	5.82
○Thread 305758			4.86	6.11	79.56	2.24	0.00	5.83
○Thread 305759			4.94	6.11	80.70	2.24	0.00	5.82
○Thread 305760			4.85	6.12	79.31	2.24	0.00	5.83
○Thread 305761			4.84	6.12	79.15	2.24	0.00	5.82
○Thread 305762			4.98	6.12	81.43	2.24	0.00	5.83
○Thread 305763			4.90	6.12	80.04	2.24	0.00	5.82
○Thread 305764			4.81	6.12	78.65	2.24	0.00	5.82
○Thread 305765			4.88	6.12	79.79	2.24	0.00	5.82

Report Configuration