OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	346.69
►Process 3919359+		1	346.69
○Thread 3919359			346.69	347.73	99.70	0.00	0.00	347.69

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	171.97
►Process 3919629+		2	171.97
○Thread 3919629			171.92	178.13	96.52	0.00	0.00	178.10
○Thread 3919729			171.97	177.24	97.03	0.88	-0.00	176.91

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	86.97
►Process 3920774+		4	86.97
○Thread 3920774			86.29	93.55	92.24	0.00	0.00	93.52
○Thread 3920873			86.97	92.67	93.86	0.88	-0.00	92.33
○Thread 3920874			86.74	92.67	93.61	0.88	-0.00	92.33
○Thread 3920875			86.35	92.67	93.19	0.88	-0.00	92.33

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	48.50
►Process 3923626+		8	48.50
○Thread 3923626			46.50	56.41	82.43	0.00	0.00	56.37
○Thread 3923725			48.42	55.52	87.20	0.88	-0.00	55.19
○Thread 3923726			48.50	55.52	87.34	0.88	-0.00	55.18
○Thread 3923727			48.27	55.52	86.93	0.88	-0.00	55.19
○Thread 3923728			48.33	55.52	87.03	0.88	0.00	55.19
○Thread 3923729			48.28	55.52	86.94	0.88	0.00	55.19
○Thread 3923730			47.87	55.52	86.21	0.88	0.00	55.19
○Thread 3923731			47.68	55.52	85.87	0.88	-0.00	55.19

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	30.02
►Process 3929993+		16	30.02
○Thread 3929993			28.59	40.67	70.30	0.00	0.00	40.64
○Thread 3930092			30.02	39.79	75.44	0.88	-0.00	39.45
○Thread 3930093			30.00	39.79	75.40	0.88	-0.00	39.45
○Thread 3930094			29.92	39.79	75.19	0.88	-0.00	39.45
○Thread 3930095			29.90	39.79	75.14	0.88	0.00	39.45
○Thread 3930096			29.92	39.79	75.20	0.88	0.00	39.45
○Thread 3930097			29.57	39.79	74.31	0.88	0.00	39.45
○Thread 3930098			29.52	39.79	74.19	0.88	0.00	39.45
○Thread 3930099			29.74	39.79	74.74	0.88	0.00	39.45
○Thread 3930100			29.75	39.79	74.77	0.88	0.00	39.45
○Thread 3930101			29.74	39.79	74.75	0.88	0.00	39.45
○Thread 3930102			29.66	39.79	74.55	0.88	0.00	39.45
○Thread 3930103			29.66	39.79	74.54	0.88	0.00	39.45
○Thread 3930104			29.66	39.79	74.55	0.88	0.00	39.45
○Thread 3930105			29.57	39.79	74.31	0.88	0.00	39.45
○Thread 3930106			29.34	39.79	73.75	0.88	-0.00	39.45

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	21.68
►Process 3943393+		24	21.68
○Thread 3943393			19.42	35.15	55.25	0.00	0.00	35.11
○Thread 3943492			21.68	34.26	63.30	0.89	-0.00	33.92
○Thread 3943493			21.66	34.26	63.22	0.89	-0.00	33.92
○Thread 3943494			21.67	34.26	63.27	0.89	-0.00	33.92
○Thread 3943495			21.66	34.26	63.24	0.89	-0.00	33.92
○Thread 3943496			21.65	34.26	63.21	0.89	-0.00	33.90
○Thread 3943497			21.41	34.26	62.49	0.89	-0.00	33.91
○Thread 3943498			21.35	34.26	62.32	0.89	-0.00	33.92
○Thread 3943499			21.33	34.26	62.26	0.89	-0.00	33.92
○Thread 3943500			21.33	34.26	62.26	0.89	-0.00	33.91
○Thread 3943501			21.33	34.26	62.26	0.89	-0.00	33.91
○Thread 3943502			21.26	34.26	62.06	0.89	-0.00	33.91
○Thread 3943503			21.51	34.26	62.79	0.89	-0.00	33.92
○Thread 3943504			21.52	34.26	62.82	0.89	-0.00	33.92
○Thread 3943505			21.50	34.26	62.77	0.89	-0.00	33.91
○Thread 3943506			21.50	34.26	62.77	0.89	-0.00	33.91
○Thread 3943507			21.50	34.26	62.77	0.89	-0.00	33.92
○Thread 3943508			21.50	34.26	62.76	0.89	-0.00	33.91
○Thread 3943509			21.40	34.26	62.48	0.89	-0.00	33.92
○Thread 3943510			21.34	34.26	62.31	0.89	0.00	33.92
○Thread 3943511			21.32	34.26	62.25	0.89	0.00	33.92
○Thread 3943512			21.31	34.26	62.21	0.89	0.00	33.91
○Thread 3943513			21.33	34.26	62.26	0.89	0.00	33.92
○Thread 3943514			20.90	34.26	61.02	0.89	-0.00	33.91

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	17.23
►Process 3963916+		32	17.23
○Thread 3963916			15.81	33.68	46.94	0.00	0.00	33.63
○Thread 3964016			17.21	32.79	52.48	0.89	-0.00	32.44
○Thread 3964017			17.21	32.79	52.47	0.89	-0.00	32.44
○Thread 3964018			17.23	32.79	52.54	0.89	-0.00	32.43
○Thread 3964019			17.22	32.79	52.53	0.89	-0.00	32.44
○Thread 3964020			17.22	32.79	52.53	0.89	-0.00	32.44
○Thread 3964021			17.14	32.79	52.25	0.89	-0.00	32.44
○Thread 3964022			17.14	32.79	52.27	0.89	0.00	32.44
○Thread 3964023			17.14	32.79	52.25	0.89	-0.00	32.44
○Thread 3964024			17.14	32.79	52.25	0.89	-0.00	32.43
○Thread 3964025			17.14	32.79	52.27	0.89	-0.00	32.43
○Thread 3964026			17.05	32.79	52.00	0.89	-0.00	32.43
○Thread 3964027			17.04	32.79	51.95	0.89	-0.00	32.45
○Thread 3964028			17.04	32.79	51.97	0.89	0.00	32.44
○Thread 3964029			17.03	32.79	51.92	0.89	0.00	32.44
○Thread 3964030			16.97	32.79	51.75	0.89	-0.00	32.44
○Thread 3964031			17.22	32.79	52.50	0.89	0.00	32.44
○Thread 3964032			17.22	32.79	52.52	0.89	0.00	32.44
○Thread 3964033			17.23	32.79	52.55	0.89	0.00	32.44
○Thread 3964034			17.23	32.79	52.55	0.89	-0.00	32.44
○Thread 3964035			17.21	32.79	52.49	0.89	0.00	32.44
○Thread 3964036			17.21	32.79	52.50	0.89	0.00	32.44
○Thread 3964037			16.70	32.79	50.93	0.89	0.00	32.44
○Thread 3964038			16.72	32.79	50.99	0.89	0.00	32.44
○Thread 3964039			16.71	32.79	50.95	0.89	0.00	32.44
○Thread 3964040			16.71	32.79	50.96	0.89	0.00	32.44
○Thread 3964041			16.73	32.79	51.01	0.89	0.00	32.44
○Thread 3964042			16.62	32.79	50.67	0.89	0.00	32.44
○Thread 3964043			16.62	32.79	50.69	0.89	0.00	32.44
○Thread 3964044			16.63	32.79	50.72	0.89	0.00	32.44
○Thread 3964045			16.62	32.79	50.69	0.89	0.00	32.45
○Thread 3964046			16.34	32.79	49.82	0.89	-0.00	32.44

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	14.52
►Process 3991474+		40	14.52
○Thread 3991474			11.60	33.93	34.19	0.00	0.00	33.87
○Thread 3991573			14.52	33.04	43.94	0.89	-0.00	32.68
○Thread 3991574			14.47	33.04	43.80	0.89	-0.00	32.69
○Thread 3991575			14.45	33.04	43.74	0.89	-0.00	32.69
○Thread 3991576			14.44	33.04	43.71	0.89	-0.00	32.68
○Thread 3991577			14.45	33.04	43.74	0.89	-0.00	32.69
○Thread 3991578			14.47	33.04	43.79	0.89	-0.00	32.69
○Thread 3991579			14.47	33.04	43.79	0.89	-0.00	32.69
○Thread 3991580			14.43	33.04	43.68	0.89	-0.00	32.69
○Thread 3991581			14.43	33.04	43.67	0.89	0.00	32.69
○Thread 3991582			14.45	33.04	43.74	0.89	-0.00	32.69
○Thread 3991583			14.47	33.04	43.79	0.89	-0.00	32.69
○Thread 3991584			14.44	33.04	43.71	0.89	-0.00	32.69
○Thread 3991585			14.45	33.04	43.73	0.89	-0.00	32.69
○Thread 3991586			14.45	33.03	43.73	0.89	-0.00	32.69
○Thread 3991587			14.33	33.03	43.36	0.89	-0.00	32.69
○Thread 3991588			14.34	33.03	43.39	0.89	-0.00	32.69
○Thread 3991589			14.32	33.03	43.35	0.89	-0.00	32.69
○Thread 3991590			14.34	33.03	43.41	0.89	-0.00	32.69
○Thread 3991591			14.26	33.03	43.15	0.89	-0.00	32.69
○Thread 3991592			14.50	33.03	43.88	0.89	-0.00	32.68
○Thread 3991593			14.51	33.03	43.92	0.89	-0.00	32.69
○Thread 3991594			14.08	33.03	42.61	0.89	-0.00	32.69
○Thread 3991595			14.06	33.03	42.55	0.89	-0.00	32.69
○Thread 3991596			14.08	33.03	42.61	0.89	-0.00	32.68
○Thread 3991597			14.09	33.03	42.64	0.89	-0.00	32.69
○Thread 3991598			14.09	33.03	42.64	0.89	-0.00	32.69
○Thread 3991599			14.07	33.03	42.61	0.89	-0.00	32.69
○Thread 3991600			14.05	33.03	42.53	0.89	-0.00	32.69
○Thread 3991601			14.06	33.03	42.56	0.89	-0.00	32.69
○Thread 3991602			14.08	33.03	42.62	0.89	-0.00	32.68
○Thread 3991603			14.06	33.03	42.56	0.89	0.00	32.69
○Thread 3991604			14.07	33.03	42.58	0.89	-0.00	32.69
○Thread 3991605			14.06	33.03	42.55	0.89	0.00	32.69
○Thread 3991606			14.05	33.03	42.52	0.89	0.00	32.69
○Thread 3991607			13.95	33.03	42.22	0.89	0.00	32.68
○Thread 3991608			13.95	33.03	42.23	0.89	0.00	32.68
○Thread 3991609			13.93	33.03	42.16	0.89	0.00	32.69
○Thread 3991610			13.95	33.03	42.22	0.89	0.00	32.69
○Thread 3991611			13.57	33.03	41.08	0.89	-0.00	32.68

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	12.71
►Process 4026157+		48	12.71
○Thread 4026157			11.12	35.02	31.75	0.00	0.00	34.97
○Thread 4026256			12.71	34.14	37.21	0.88	-0.00	33.79
○Thread 4026257			12.70	34.14	37.19	0.88	-0.00	33.78
○Thread 4026258			12.70	34.14	37.18	0.88	-0.00	33.78
○Thread 4026259			12.71	34.14	37.22	0.88	-0.00	33.78
○Thread 4026260			12.71	34.14	37.22	0.88	-0.00	33.78
○Thread 4026261			12.49	34.14	36.57	0.88	-0.00	33.79
○Thread 4026262			12.49	34.14	36.58	0.88	-0.00	33.79
○Thread 4026263			12.48	34.14	36.54	0.88	-0.00	33.79
○Thread 4026264			12.47	34.14	36.52	0.88	-0.00	33.79
○Thread 4026265			12.48	34.14	36.54	0.88	-0.00	33.78
○Thread 4026266			12.42	34.14	36.36	0.88	-0.00	33.79
○Thread 4026267			12.40	34.14	36.32	0.88	-0.00	33.79
○Thread 4026268			12.42	34.14	36.38	0.88	-0.00	33.79
○Thread 4026269			12.43	34.14	36.41	0.88	-0.00	33.79
○Thread 4026270			12.37	34.14	36.23	0.88	-0.00	33.78
○Thread 4026271			12.28	34.14	35.95	0.88	-0.00	33.79
○Thread 4026272			12.31	34.14	36.05	0.88	-0.00	33.79
○Thread 4026273			12.32	34.14	36.07	0.88	-0.00	33.79
○Thread 4026274			12.29	34.14	36.00	0.88	-0.00	33.78
○Thread 4026275			12.30	34.14	36.01	0.88	-0.00	33.79
○Thread 4026276			12.29	34.14	36.00	0.88	-0.00	33.79
○Thread 4026277			12.26	34.14	35.91	0.88	-0.00	33.79
○Thread 4026278			12.23	34.14	35.82	0.88	-0.00	33.79
○Thread 4026279			12.25	34.14	35.86	0.88	-0.00	33.78
○Thread 4026280			12.23	34.14	35.81	0.88	-0.00	33.79
○Thread 4026281			12.24	34.14	35.85	0.88	-0.00	33.78
○Thread 4026282			12.17	34.14	35.63	0.88	-0.00	33.79
○Thread 4026283			12.18	34.14	35.67	0.88	-0.00	33.78
○Thread 4026284			12.17	34.14	35.65	0.88	-0.00	33.79
○Thread 4026285			12.16	34.14	35.60	0.88	-0.00	33.78
○Thread 4026286			12.11	34.14	35.46	0.88	-0.00	33.78
○Thread 4026287			12.29	34.14	36.00	0.88	-0.00	33.78
○Thread 4026288			12.29	34.14	35.98	0.88	-0.00	33.80
○Thread 4026289			12.29	34.14	35.98	0.88	-0.00	33.78
○Thread 4026290			12.29	34.14	35.98	0.88	-0.00	33.80
○Thread 4026291			12.27	34.14	35.92	0.88	-0.00	33.79
○Thread 4026292			12.27	34.14	35.94	0.88	-0.00	33.78
○Thread 4026293			12.21	34.14	35.76	0.88	-0.00	33.78
○Thread 4026294			12.24	34.14	35.85	0.88	-0.00	33.78
○Thread 4026295			12.23	34.14	35.82	0.88	0.00	33.79
○Thread 4026296			12.24	34.14	35.84	0.88	0.00	33.79
○Thread 4026297			12.22	34.14	35.79	0.88	0.00	33.79
○Thread 4026298			12.13	34.14	35.53	0.88	0.00	33.78
○Thread 4026299			12.14	34.14	35.56	0.88	0.00	33.79
○Thread 4026300			12.14	34.14	35.55	0.88	0.00	33.78
○Thread 4026301			12.15	34.14	35.57	0.88	0.00	33.78
○Thread 4026302			11.77	34.14	34.46	0.88	-0.00	33.78

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	10.74
►Process 4067872+		56	10.74
○Thread 4067872			8.97	35.38	25.36	0.00	0.00	35.33
○Thread 4067971			10.74	34.50	31.13	0.88	-0.00	34.14
○Thread 4067972			10.74	34.50	31.12	0.88	-0.00	34.14
○Thread 4067973			10.74	34.50	31.13	0.88	-0.00	34.13
○Thread 4067974			10.74	34.50	31.13	0.88	-0.00	34.13
○Thread 4067975			10.69	34.50	30.99	0.88	-0.00	34.13
○Thread 4067976			10.53	34.50	30.51	0.88	-0.00	34.14
○Thread 4067977			10.55	34.50	30.58	0.88	-0.00	34.15
○Thread 4067978			10.54	34.50	30.55	0.88	-0.00	34.14
○Thread 4067979			10.52	34.50	30.48	0.88	-0.00	34.13
○Thread 4067980			10.49	34.50	30.41	0.88	-0.00	34.13
○Thread 4067981			10.51	34.50	30.45	0.88	-0.00	34.14
○Thread 4067982			10.49	34.50	30.41	0.88	-0.00	34.13
○Thread 4067983			10.49	34.50	30.41	0.88	-0.00	34.14
○Thread 4067984			10.61	34.50	30.76	0.88	-0.00	34.14
○Thread 4067985			10.59	34.49	30.70	0.88	-0.00	34.13
○Thread 4067986			10.60	34.49	30.71	0.88	-0.00	34.13
○Thread 4067987			10.60	34.49	30.73	0.88	-0.00	34.13
○Thread 4067988			10.61	34.49	30.76	0.88	-0.00	34.13
○Thread 4067989			10.58	34.49	30.67	0.88	-0.00	34.13
○Thread 4067990			10.53	34.49	30.51	0.88	-0.00	34.14
○Thread 4067991			10.50	34.49	30.44	0.88	-0.00	34.13
○Thread 4067992			10.52	34.49	30.50	0.88	-0.00	34.13
○Thread 4067993			10.49	34.49	30.41	0.88	-0.00	34.15
○Thread 4067994			10.48	34.49	30.37	0.88	-0.00	34.13
○Thread 4067995			10.51	34.49	30.47	0.88	-0.00	34.13
○Thread 4067996			10.51	34.49	30.45	0.88	-0.00	34.13
○Thread 4067997			10.33	34.49	29.95	0.88	-0.00	34.13
○Thread 4067998			10.74	34.49	31.14	0.88	-0.00	34.13
○Thread 4067999			10.70	34.49	31.02	0.88	-0.00	34.13
○Thread 4068000			10.74	34.49	31.14	0.88	-0.00	34.14
○Thread 4068001			10.72	34.49	31.08	0.88	-0.00	34.13
○Thread 4068002			10.73	34.49	31.11	0.88	-0.00	34.14
○Thread 4068003			10.67	34.49	30.93	0.88	-0.00	34.13
○Thread 4068004			10.50	34.49	30.44	0.88	-0.00	34.13
○Thread 4068005			10.52	34.49	30.50	0.88	-0.00	34.13
○Thread 4068006			10.52	34.49	30.50	0.88	-0.00	34.13
○Thread 4068007			10.53	34.49	30.51	0.88	-0.00	34.14
○Thread 4068008			10.52	34.49	30.50	0.88	-0.00	34.12
○Thread 4068009			10.51	34.49	30.47	0.88	-0.00	34.13
○Thread 4068010			10.51	34.49	30.47	0.88	-0.00	34.14
○Thread 4068011			10.48	34.49	30.38	0.88	-0.00	34.13
○Thread 4068012			10.60	34.49	30.73	0.88	-0.00	34.13
○Thread 4068013			10.59	34.49	30.69	0.88	-0.00	34.13
○Thread 4068014			10.60	34.49	30.73	0.88	-0.00	34.13
○Thread 4068015			10.60	34.49	30.73	0.88	-0.00	34.13
○Thread 4068016			10.59	34.49	30.69	0.88	-0.00	34.13
○Thread 4068017			10.56	34.49	30.60	0.88	-0.00	34.13
○Thread 4068018			10.49	34.49	30.40	0.88	-0.00	34.13
○Thread 4068019			10.51	34.49	30.47	0.88	-0.00	34.13
○Thread 4068020			10.54	34.49	30.54	0.88	-0.00	34.12
○Thread 4068021			10.50	34.49	30.44	0.88	-0.00	34.12
○Thread 4068022			10.51	34.49	30.47	0.89	-0.00	34.13
○Thread 4068023			10.47	34.49	30.36	0.89	-0.00	34.13
○Thread 4068024			10.50	34.49	30.44	0.89	-0.00	34.13
○Thread 4068025			10.32	34.49	29.92	0.89	-0.00	34.13

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	9.62
►Process 4116718+		64	9.62
○Thread 4116718			8.13	36.83	22.06	0.00	0.00	36.79
○Thread 4116817			9.60	35.94	26.71	0.89	-0.00	35.57
○Thread 4116818			9.59	35.94	26.67	0.89	-0.00	35.57
○Thread 4116819			9.58	35.94	26.65	0.89	-0.00	35.57
○Thread 4116820			9.58	35.94	26.65	0.89	-0.00	35.58
○Thread 4116821			9.59	35.94	26.68	0.89	-0.00	35.58
○Thread 4116822			9.57	35.94	26.61	0.89	-0.00	35.57
○Thread 4116823			9.53	35.94	26.50	0.89	-0.00	35.57
○Thread 4116824			9.53	35.94	26.50	0.89	-0.00	35.57
○Thread 4116825			9.54	35.94	26.53	0.89	-0.00	35.56
○Thread 4116826			9.54	35.94	26.53	0.89	-0.00	35.56
○Thread 4116827			9.48	35.94	26.38	0.89	-0.00	35.57
○Thread 4116828			9.45	35.94	26.29	0.89	-0.00	35.56
○Thread 4116829			9.46	35.94	26.32	0.89	-0.00	35.56
○Thread 4116830			9.45	35.94	26.29	0.89	-0.00	35.57
○Thread 4116831			9.44	35.94	26.27	0.89	-0.00	35.58
○Thread 4116832			9.56	35.94	26.60	0.89	-0.00	35.56
○Thread 4116833			9.56	35.94	26.60	0.89	-0.00	35.56
○Thread 4116834			9.57	35.94	26.63	0.89	-0.00	35.58
○Thread 4116835			9.56	35.94	26.59	0.89	-0.00	35.58
○Thread 4116836			9.57	35.94	26.63	0.89	-0.00	35.58
○Thread 4116837			9.56	35.94	26.60	0.89	-0.00	35.58
○Thread 4116838			9.31	35.94	25.89	0.89	-0.00	35.57
○Thread 4116839			9.30	35.94	25.88	0.89	-0.00	35.56
○Thread 4116840			9.35	35.94	26.00	0.89	-0.00	35.57
○Thread 4116841			9.32	35.94	25.92	0.89	-0.00	35.57
○Thread 4116842			9.32	35.94	25.92	0.89	-0.00	35.58
○Thread 4116843			9.25	35.94	25.72	0.89	-0.00	35.57
○Thread 4116844			9.23	35.94	25.68	0.89	-0.00	35.57
○Thread 4116845			9.27	35.94	25.79	0.89	-0.00	35.57
○Thread 4116846			9.25	35.94	25.74	0.89	-0.00	35.57
○Thread 4116847			9.09	35.94	25.28	0.89	0.00	35.58
○Thread 4116848			9.62	35.94	26.75	0.89	-0.00	35.57
○Thread 4116849			9.59	35.94	26.67	0.89	-0.00	35.56
○Thread 4116850			9.59	35.94	26.69	0.89	-0.00	35.57
○Thread 4116851			9.59	35.94	26.67	0.89	-0.00	35.57
○Thread 4116852			9.59	35.94	26.67	0.89	-0.00	35.58
○Thread 4116853			9.58	35.94	26.64	0.89	0.00	35.58
○Thread 4116854			9.51	35.94	26.46	0.89	-0.00	35.58
○Thread 4116855			9.53	35.94	26.50	0.89	-0.00	35.56
○Thread 4116856			9.53	35.94	26.52	0.89	-0.00	35.57
○Thread 4116857			9.55	35.94	26.59	0.89	-0.00	35.58
○Thread 4116858			9.56	35.94	26.60	0.89	-0.00	35.57
○Thread 4116859			9.49	35.94	26.41	0.89	0.00	35.58
○Thread 4116860			9.48	35.94	26.38	0.89	-0.00	35.58
○Thread 4116861			9.47	35.94	26.34	0.89	-0.00	35.56
○Thread 4116862			9.48	35.94	26.37	0.89	-0.00	35.58
○Thread 4116863			9.46	35.94	26.31	0.89	-0.00	35.58
○Thread 4116864			9.59	35.94	26.69	0.89	-0.00	35.57
○Thread 4116865			9.59	35.94	26.69	0.89	-0.00	35.58
○Thread 4116866			9.60	35.94	26.71	0.89	-0.00	35.58
○Thread 4116867			9.60	35.94	26.71	0.89	-0.00	35.57
○Thread 4116868			9.59	35.94	26.67	0.89	-0.00	35.56
○Thread 4116869			9.58	35.94	26.64	0.89	-0.00	35.57
○Thread 4116870			9.31	35.94	25.91	0.89	-0.00	35.57
○Thread 4116871			9.32	35.94	25.92	0.89	0.00	35.56
○Thread 4116872			9.27	35.94	25.80	0.89	0.00	35.56
○Thread 4116873			9.32	35.94	25.92	0.89	-0.00	35.57
○Thread 4116874			9.34	35.94	25.98	0.89	0.00	35.56
○Thread 4116875			9.27	35.94	25.78	0.89	0.00	35.57
○Thread 4116876			9.27	35.93	25.80	0.89	0.00	35.57
○Thread 4116877			9.24	35.93	25.71	0.89	0.00	35.57
○Thread 4116878			9.26	35.93	25.78	0.89	0.00	35.58
○Thread 4116879			9.09	35.93	25.28	0.89	-0.00	35.56

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	8.84
►Process 4172641+		72	8.84
○Thread 4172641			7.21	38.86	18.54	0.00	0.00	38.81
○Thread 4172740			8.82	37.98	23.21	0.88	-0.00	37.61
○Thread 4172741			8.84	37.98	23.26	0.88	-0.00	37.60
○Thread 4172742			8.81	37.98	23.20	0.88	-0.00	37.62
○Thread 4172743			8.79	37.98	23.14	0.88	-0.00	37.61
○Thread 4172744			8.81	37.98	23.18	0.88	-0.00	37.60
○Thread 4172745			8.77	37.98	23.09	0.88	-0.00	37.62
○Thread 4172746			8.78	37.98	23.10	0.88	-0.00	37.63
○Thread 4172747			8.75	37.98	23.03	0.88	-0.00	37.59
○Thread 4172748			8.78	37.98	23.10	0.88	-0.00	37.61
○Thread 4172749			8.75	37.98	23.04	0.88	-0.00	37.62
○Thread 4172750			8.77	37.98	23.08	0.88	-0.00	37.62
○Thread 4172751			8.75	37.98	23.03	0.88	-0.00	37.60
○Thread 4172752			8.76	37.98	23.07	0.88	-0.00	37.62
○Thread 4172753			8.74	37.98	23.01	0.88	-0.00	37.62
○Thread 4172754			8.73	37.98	22.97	0.88	-0.00	37.62
○Thread 4172755			8.74	37.98	23.00	0.88	-0.00	37.61
○Thread 4172756			8.77	37.98	23.08	0.88	-0.00	37.63
○Thread 4172757			8.56	37.98	22.53	0.88	-0.00	37.60
○Thread 4172758			8.49	37.98	22.34	0.88	-0.00	37.62
○Thread 4172759			8.49	37.98	22.36	0.88	-0.00	37.61
○Thread 4172760			8.48	37.98	22.33	0.88	-0.00	37.62
○Thread 4172761			8.48	37.98	22.32	0.88	-0.00	37.62
○Thread 4172762			8.40	37.98	22.11	0.88	-0.00	37.62
○Thread 4172763			8.45	37.98	22.24	0.88	-0.00	37.62
○Thread 4172764			8.47	37.98	22.29	0.88	-0.00	37.62
○Thread 4172765			8.43	37.98	22.18	0.88	-0.00	37.61
○Thread 4172766			8.44	37.98	22.21	0.88	-0.00	37.61
○Thread 4172767			8.45	37.98	22.24	0.88	-0.00	37.60
○Thread 4172768			8.45	37.98	22.24	0.88	-0.00	37.61
○Thread 4172769			8.36	37.98	22.01	0.88	-0.00	37.62
○Thread 4172770			8.37	37.98	22.04	0.88	-0.00	37.61
○Thread 4172771			8.38	37.98	22.07	0.88	-0.00	37.62
○Thread 4172772			8.39	37.98	22.09	0.88	-0.00	37.63
○Thread 4172773			8.36	37.98	22.01	0.88	-0.00	37.63
○Thread 4172774			8.24	37.98	21.68	0.88	-0.00	37.61
○Thread 4172775			8.59	37.98	22.62	0.88	-0.00	37.62
○Thread 4172776			8.61	37.98	22.67	0.88	-0.00	37.63
○Thread 4172777			8.59	37.98	22.62	0.88	-0.00	37.62
○Thread 4172778			8.57	37.98	22.57	0.88	-0.00	37.62
○Thread 4172779			8.57	37.98	22.57	0.88	-0.00	37.62
○Thread 4172780			8.59	37.98	22.62	0.88	-0.00	37.60
○Thread 4172781			8.58	37.98	22.59	0.88	-0.00	37.62
○Thread 4172782			8.47	37.98	22.29	0.88	-0.00	37.61
○Thread 4172783			8.50	37.98	22.37	0.88	-0.00	37.60
○Thread 4172784			8.50	37.97	22.38	0.88	-0.00	37.62
○Thread 4172785			8.49	37.97	22.36	0.88	-0.00	37.60
○Thread 4172786			8.44	37.97	22.23	0.88	-0.00	37.62
○Thread 4172787			8.64	37.97	22.74	0.88	-0.00	37.62
○Thread 4172788			8.63	37.97	22.71	0.88	-0.00	37.60
○Thread 4172789			8.65	37.97	22.77	0.88	-0.00	37.62
○Thread 4172790			8.63	37.97	22.73	0.88	-0.00	37.60
○Thread 4172791			8.65	37.97	22.77	0.88	-0.00	37.62
○Thread 4172792			8.64	37.97	22.75	0.88	-0.00	37.62
○Thread 4172793			8.39	37.97	22.08	0.88	-0.00	37.62
○Thread 4172794			8.40	37.97	22.11	0.88	-0.00	37.62
○Thread 4172795			8.39	37.97	22.08	0.88	-0.00	37.62
○Thread 4172796			8.41	37.97	22.13	0.88	-0.00	37.62
○Thread 4172797			8.37	37.97	22.04	0.88	-0.00	37.62
○Thread 4172798			8.40	37.97	22.11	0.89	-0.00	37.62
○Thread 4172799			8.38	37.97	22.05	0.89	-0.00	37.62
○Thread 4172800			8.40	37.97	22.11	0.89	-0.00	37.62
○Thread 4172801			8.39	37.97	22.09	0.89	-0.00	37.62
○Thread 4172802			8.38	37.97	22.07	0.89	-0.00	37.62
○Thread 4172803			8.39	37.97	22.08	0.89	-0.00	37.62
○Thread 4172804			8.40	37.97	22.11	0.89	-0.00	37.62
○Thread 4172805			8.38	37.97	22.07	0.89	-0.00	37.62
○Thread 4172806			8.27	37.97	21.78	0.89	-0.00	37.62
○Thread 4172807			8.29	37.97	21.82	0.89	-0.00	37.62
○Thread 4172808			8.31	37.97	21.87	0.89	-0.00	37.62
○Thread 4172809			8.31	37.97	21.87	0.89	-0.00	37.62
○Thread 4172810			8.11	37.97	21.36	0.89	-0.00	37.62

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	8.04
►Process 42174+		80	8.04
○Thread 42174			7.15	40.70	17.55	0.00	0.00	40.65
○Thread 42273			8.02	39.81	20.14	0.89	0.00	39.44
○Thread 42274			7.98	39.81	20.04	0.89	-0.00	39.45
○Thread 42275			7.98	39.81	20.03	0.89	-0.00	39.44
○Thread 42276			8.02	39.81	20.13	0.89	-0.00	39.44
○Thread 42277			8.01	39.81	20.12	0.89	-0.00	39.44
○Thread 42278			7.98	39.81	20.03	0.89	-0.00	39.45
○Thread 42279			7.99	39.81	20.06	0.89	-0.00	39.44
○Thread 42280			7.98	39.81	20.04	0.89	-0.00	39.44
○Thread 42281			7.99	39.81	20.07	0.89	-0.00	39.44
○Thread 42282			7.98	39.81	20.03	0.89	-0.00	39.44
○Thread 42283			7.98	39.81	20.04	0.89	-0.00	39.44
○Thread 42284			7.96	39.81	19.99	0.89	-0.00	39.44
○Thread 42285			7.99	39.81	20.06	0.89	-0.00	39.44
○Thread 42286			7.98	39.81	20.04	0.89	-0.00	39.46
○Thread 42287			7.92	39.81	19.89	0.89	-0.00	39.44
○Thread 42288			7.92	39.81	19.88	0.89	0.00	39.44
○Thread 42289			7.92	39.81	19.88	0.89	-0.00	39.45
○Thread 42290			7.93	39.81	19.92	0.89	-0.00	39.44
○Thread 42291			7.87	39.81	19.77	0.89	-0.00	39.44
○Thread 42292			8.03	39.81	20.16	0.89	-0.00	39.45
○Thread 42293			7.99	39.81	20.07	0.89	-0.00	39.43
○Thread 42294			7.76	39.81	19.48	0.89	-0.00	39.45
○Thread 42295			7.78	39.81	19.54	0.89	-0.00	39.45
○Thread 42296			7.77	39.81	19.52	0.89	-0.00	39.43
○Thread 42297			7.76	39.81	19.48	0.89	-0.00	39.44
○Thread 42298			7.78	39.81	19.53	0.89	-0.00	39.45
○Thread 42299			7.78	39.81	19.53	0.89	-0.00	39.44
○Thread 42300			7.76	39.81	19.48	0.89	-0.00	39.44
○Thread 42301			7.78	39.81	19.53	0.89	-0.00	39.43
○Thread 42302			7.78	39.81	19.53	0.89	-0.00	39.44
○Thread 42303			7.76	39.81	19.49	0.89	-0.00	39.45
○Thread 42304			7.78	39.81	19.53	0.89	-0.00	39.43
○Thread 42305			7.74	39.81	19.44	0.89	-0.00	39.44
○Thread 42306			7.75	39.81	19.47	0.89	-0.00	39.43
○Thread 42307			7.69	39.81	19.32	0.89	-0.00	39.43
○Thread 42308			7.71	39.81	19.37	0.89	-0.00	39.44
○Thread 42309			7.70	39.81	19.33	0.89	-0.00	39.43
○Thread 42310			7.69	39.81	19.32	0.89	-0.00	39.44
○Thread 42311			7.50	39.81	18.84	0.89	-0.00	39.43
○Thread 42312			8.04	39.81	20.18	0.89	-0.00	39.44
○Thread 42313			8.00	39.81	20.08	0.89	-0.00	39.44
○Thread 42314			7.96	39.81	19.98	0.89	-0.00	39.45
○Thread 42315			7.98	39.81	20.05	0.89	-0.00	39.43
○Thread 42316			7.99	39.81	20.06	0.89	-0.00	39.43
○Thread 42317			7.97	39.81	20.01	0.89	-0.00	39.44
○Thread 42318			7.98	39.81	20.03	0.89	-0.00	39.44
○Thread 42319			7.98	39.81	20.05	0.89	-0.00	39.43
○Thread 42320			7.99	39.81	20.06	0.89	-0.00	39.44
○Thread 42321			7.93	39.81	19.91	0.90	-0.00	39.43
○Thread 42322			7.95	39.81	19.97	0.90	-0.00	39.43
○Thread 42323			7.99	39.81	20.06	0.90	-0.00	39.44
○Thread 42324			8.01	39.81	20.11	0.90	-0.00	39.44
○Thread 42325			7.99	39.81	20.06	0.90	-0.00	39.44
○Thread 42326			8.00	39.81	20.08	0.90	-0.00	39.44
○Thread 42327			7.93	39.81	19.91	0.90	-0.00	39.45
○Thread 42328			7.93	39.81	19.92	0.90	-0.00	39.44
○Thread 42329			7.95	39.81	19.97	0.90	-0.00	39.44
○Thread 42330			7.94	39.81	19.93	0.90	-0.00	39.44
○Thread 42331			7.89	39.81	19.81	0.90	-0.00	39.44
○Thread 42332			8.00	39.81	20.08	0.90	-0.00	39.44
○Thread 42333			7.99	39.81	20.07	0.90	-0.00	39.44
○Thread 42334			7.79	39.81	19.56	0.90	-0.00	39.44
○Thread 42335			7.80	39.81	19.58	0.90	-0.00	39.44
○Thread 42336			7.78	39.81	19.53	0.90	-0.00	39.44
○Thread 42337			7.76	39.81	19.49	0.90	-0.00	39.44
○Thread 42338			7.79	39.81	19.56	0.90	-0.00	39.44
○Thread 42339			7.78	39.81	19.54	0.90	-0.00	39.44
○Thread 42340			7.79	39.81	19.57	0.90	-0.00	39.45
○Thread 42341			7.76	39.81	19.49	0.90	-0.00	39.44
○Thread 42342			7.78	39.81	19.53	0.90	-0.00	39.45
○Thread 42343			7.81	39.81	19.62	0.90	-0.00	39.45
○Thread 42344			7.79	39.81	19.57	0.90	-0.00	39.45
○Thread 42345			7.78	39.80	19.53	0.90	-0.00	39.44
○Thread 42346			7.80	39.80	19.58	0.90	-0.00	39.44
○Thread 42347			7.72	39.80	19.38	0.90	-0.00	39.45
○Thread 42348			7.71	39.80	19.36	0.90	-0.00	39.44
○Thread 42349			7.70	39.80	19.34	0.90	-0.00	39.44
○Thread 42350			7.73	39.80	19.41	0.90	-0.00	39.44
○Thread 42351			7.53	39.80	18.91	0.90	-0.00	39.44

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	7.45
►Process 112257+		88	7.45
○Thread 112257			6.32	42.71	14.79	0.00	0.00	42.65
○Thread 112356			7.43	41.81	17.77	0.89	-0.00	41.44
○Thread 112357			7.45	41.81	17.82	0.89	-0.00	41.44
○Thread 112358			7.42	41.81	17.75	0.89	-0.00	41.44
○Thread 112359			7.45	41.81	17.81	0.90	-0.00	41.44
○Thread 112360			7.43	41.81	17.77	0.90	-0.00	41.44
○Thread 112361			7.42	41.81	17.75	0.90	-0.00	41.44
○Thread 112362			7.40	41.81	17.69	0.90	-0.00	41.44
○Thread 112363			7.39	41.81	17.67	0.90	-0.00	41.45
○Thread 112364			7.37	41.81	17.61	0.90	-0.00	41.44
○Thread 112365			7.40	41.81	17.69	0.90	-0.00	41.44
○Thread 112366			7.38	41.81	17.64	0.90	-0.00	41.44
○Thread 112367			7.33	41.81	17.52	0.90	-0.00	41.44
○Thread 112368			7.34	41.81	17.54	0.90	-0.00	41.44
○Thread 112369			7.32	41.81	17.51	0.90	-0.00	41.44
○Thread 112370			7.31	41.81	17.48	0.90	-0.00	41.44
○Thread 112371			7.31	41.81	17.48	0.90	-0.00	41.44
○Thread 112372			7.29	41.81	17.44	0.90	-0.00	41.45
○Thread 112373			7.31	41.81	17.48	0.90	-0.00	41.44
○Thread 112374			7.28	41.81	17.41	0.90	-0.00	41.43
○Thread 112375			7.30	41.81	17.46	0.90	-0.00	41.44
○Thread 112376			7.26	41.81	17.35	0.90	-0.00	41.44
○Thread 112377			7.40	41.81	17.69	0.90	-0.00	41.44
○Thread 112378			7.42	41.81	17.73	0.90	-0.00	41.44
○Thread 112379			7.43	41.81	17.77	0.90	-0.00	41.45
○Thread 112380			7.39	41.81	17.67	0.90	-0.00	41.44
○Thread 112381			7.40	41.81	17.69	0.90	-0.00	41.44
○Thread 112382			7.41	41.81	17.72	0.90	-0.00	41.44
○Thread 112383			7.40	41.81	17.70	0.90	-0.00	41.44
○Thread 112384			7.33	41.81	17.52	0.90	-0.00	41.44
○Thread 112385			7.37	41.81	17.63	0.90	-0.00	41.44
○Thread 112386			7.36	41.81	17.59	0.90	-0.00	41.44
○Thread 112387			7.37	41.81	17.63	0.90	-0.00	41.44
○Thread 112388			7.34	41.81	17.56	0.90	-0.00	41.44
○Thread 112389			7.09	41.81	16.96	0.90	-0.00	41.44
○Thread 112390			7.08	41.81	16.92	0.90	-0.00	41.44
○Thread 112391			7.06	41.81	16.89	0.90	-0.00	41.44
○Thread 112392			7.11	41.81	17.01	0.90	-0.00	41.44
○Thread 112393			7.07	41.81	16.91	0.90	-0.00	41.44
○Thread 112394			7.07	41.81	16.91	0.90	-0.00	41.44
○Thread 112395			7.07	41.81	16.90	0.90	-0.00	41.45
○Thread 112396			7.09	41.81	16.95	0.90	-0.00	41.44
○Thread 112397			7.06	41.81	16.89	0.90	-0.00	41.44
○Thread 112398			6.91	41.81	16.52	0.90	-0.00	41.44
○Thread 112399			7.42	41.81	17.74	0.90	-0.00	41.44
○Thread 112400			7.41	41.81	17.71	0.90	-0.00	41.44
○Thread 112401			7.37	41.81	17.63	0.90	-0.00	41.44
○Thread 112402			7.40	41.81	17.70	0.90	-0.00	41.44
○Thread 112403			7.40	41.81	17.69	0.90	-0.00	41.44
○Thread 112404			7.39	41.81	17.66	0.90	-0.00	41.44
○Thread 112405			7.38	41.81	17.65	0.90	-0.00	41.44
○Thread 112406			7.34	41.81	17.56	0.90	-0.00	41.45
○Thread 112407			7.36	41.81	17.59	0.90	-0.00	41.44
○Thread 112408			7.39	41.81	17.66	0.90	-0.00	41.44
○Thread 112409			7.36	41.81	17.60	0.90	-0.00	41.44
○Thread 112410			7.36	41.81	17.60	0.90	-0.00	41.44
○Thread 112411			7.30	41.81	17.46	0.90	-0.00	41.45
○Thread 112412			7.32	41.81	17.50	0.90	-0.00	41.44
○Thread 112413			7.33	41.81	17.53	0.90	-0.00	41.44
○Thread 112414			7.31	41.81	17.47	0.90	-0.00	41.44
○Thread 112415			7.32	41.81	17.51	0.90	-0.00	41.44
○Thread 112416			7.32	41.81	17.50	0.90	-0.00	41.44
○Thread 112417			7.25	41.81	17.34	0.90	-0.00	41.44
○Thread 112418			7.28	41.81	17.40	0.90	-0.00	41.44
○Thread 112419			7.34	41.81	17.56	0.90	-0.00	41.44
○Thread 112420			7.28	41.81	17.40	0.90	-0.00	41.44
○Thread 112421			7.42	41.81	17.74	0.90	-0.00	41.44
○Thread 112422			7.40	41.81	17.70	0.90	-0.00	41.44
○Thread 112423			7.43	41.81	17.76	0.90	-0.00	41.44
○Thread 112424			7.42	41.81	17.75	0.90	-0.00	41.44
○Thread 112425			7.41	41.81	17.72	0.90	-0.00	41.44
○Thread 112426			7.44	41.81	17.80	0.90	-0.00	41.44
○Thread 112427			7.37	41.81	17.63	0.90	-0.00	41.44
○Thread 112428			7.37	41.81	17.62	0.90	-0.00	41.44
○Thread 112429			7.36	41.81	17.61	0.90	-0.00	41.44
○Thread 112430			7.39	41.81	17.67	0.90	-0.00	41.44
○Thread 112431			7.31	41.81	17.47	0.90	-0.00	41.44
○Thread 112432			7.37	41.81	17.63	0.90	-0.00	41.45
○Thread 112433			7.08	41.81	16.94	0.90	-0.00	41.44
○Thread 112434			7.10	41.81	16.97	0.90	-0.00	41.44
○Thread 112435			7.11	41.81	17.00	0.90	-0.00	41.44
○Thread 112436			7.08	41.80	16.94	0.90	-0.00	41.44
○Thread 112437			7.10	41.80	16.97	0.90	-0.00	41.44
○Thread 112438			7.09	41.80	16.95	0.90	-0.00	41.44
○Thread 112439			7.11	41.80	17.00	0.90	-0.00	41.45
○Thread 112440			7.07	41.80	16.91	0.90	-0.00	41.44
○Thread 112441			7.07	41.80	16.91	0.90	-0.00	41.44
○Thread 112442			6.84	41.80	16.36	0.90	-0.00	41.44

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	7.07
►Process 189370+		96	7.07
○Thread 189370			5.34	47.34	11.27	0.00	0.00	47.30
○Thread 189469			7.06	46.46	15.20	0.88	-0.00	46.08
○Thread 189470			7.07	46.46	15.21	0.88	-0.00	46.07
○Thread 189471			7.02	46.46	15.11	0.88	-0.00	46.08
○Thread 189472			7.02	46.46	15.11	0.88	-0.00	46.07
○Thread 189473			7.03	46.46	15.12	0.88	-0.00	46.08
○Thread 189474			6.87	46.46	14.79	0.88	-0.00	46.10
○Thread 189475			6.91	46.46	14.86	0.88	-0.00	46.08
○Thread 189476			6.88	46.46	14.81	0.88	-0.00	46.08
○Thread 189477			6.90	46.46	14.84	0.88	-0.00	46.08
○Thread 189478			6.89	46.46	14.83	0.88	-0.00	46.08
○Thread 189479			6.91	46.46	14.86	0.88	-0.00	46.09
○Thread 189480			6.85	46.46	14.74	0.88	-0.00	46.07
○Thread 189481			6.85	46.46	14.74	0.88	-0.00	46.08
○Thread 189482			6.87	46.46	14.78	0.88	-0.00	46.08
○Thread 189483			6.86	46.46	14.75	0.88	-0.00	46.08
○Thread 189484			6.83	46.46	14.69	0.88	-0.00	46.08
○Thread 189485			6.84	46.46	14.72	0.88	-0.00	46.08
○Thread 189486			6.84	46.46	14.72	0.88	-0.00	46.08
○Thread 189487			6.79	46.46	14.62	0.88	-0.00	46.08
○Thread 189488			6.79	46.46	14.60	0.88	-0.00	46.08
○Thread 189489			6.79	46.46	14.62	0.88	-0.00	46.08
○Thread 189490			6.82	46.46	14.67	0.88	-0.00	46.08
○Thread 189491			6.78	46.46	14.59	0.88	-0.00	46.08
○Thread 189492			6.92	46.46	14.88	0.88	-0.00	46.08
○Thread 189493			6.90	46.46	14.84	0.88	-0.00	46.08
○Thread 189494			6.91	46.46	14.86	0.88	-0.00	46.09
○Thread 189495			6.90	46.46	14.84	0.88	-0.00	46.08
○Thread 189496			6.88	46.46	14.81	0.88	-0.00	46.06
○Thread 189497			6.90	46.46	14.84	0.88	-0.00	46.08
○Thread 189498			6.87	46.46	14.78	0.88	-0.00	46.08
○Thread 189499			6.88	46.46	14.80	0.88	-0.00	46.08
○Thread 189500			6.86	46.46	14.76	0.88	-0.00	46.08
○Thread 189501			6.87	46.46	14.78	0.88	-0.00	46.08
○Thread 189502			6.85	46.46	14.74	0.88	-0.00	46.07
○Thread 189503			6.89	46.46	14.82	0.88	-0.00	46.07
○Thread 189504			6.84	46.46	14.72	0.88	-0.00	46.08
○Thread 189505			6.81	46.46	14.65	0.88	-0.00	46.07
○Thread 189506			6.86	46.46	14.76	0.88	-0.00	46.08
○Thread 189507			6.85	46.46	14.75	0.88	-0.00	46.08
○Thread 189508			6.87	46.46	14.78	0.88	-0.00	46.08
○Thread 189509			6.86	46.46	14.76	0.88	-0.00	46.08
○Thread 189510			6.86	46.46	14.76	0.88	-0.00	46.07
○Thread 189511			6.80	46.46	14.63	0.88	-0.00	46.08
○Thread 189512			6.81	46.46	14.66	0.88	-0.00	46.08
○Thread 189513			6.79	46.46	14.62	0.88	-0.00	46.08
○Thread 189514			6.77	46.46	14.57	0.88	-0.00	46.06
○Thread 189515			6.57	46.46	14.14	0.88	-0.00	46.08
○Thread 189516			7.01	46.46	15.08	0.88	-0.00	46.07
○Thread 189517			7.02	46.46	15.10	0.88	-0.00	46.08
○Thread 189518			7.00	46.46	15.07	0.88	-0.00	46.07
○Thread 189519			7.05	46.45	15.17	0.88	-0.00	46.08
○Thread 189520			7.00	46.45	15.06	0.88	-0.00	46.08
○Thread 189521			7.02	46.45	15.10	0.88	-0.00	46.08
○Thread 189522			6.83	46.45	14.69	0.88	-0.00	46.08
○Thread 189523			6.88	46.45	14.80	0.88	-0.00	46.08
○Thread 189524			6.85	46.45	14.75	0.88	-0.00	46.07
○Thread 189525			6.88	46.45	14.81	0.88	-0.00	46.07
○Thread 189526			6.87	46.45	14.79	0.88	-0.00	46.08
○Thread 189527			6.88	46.45	14.81	0.88	-0.00	46.08
○Thread 189528			6.88	46.45	14.81	0.88	-0.00	46.08
○Thread 189529			6.85	46.45	14.74	0.88	-0.00	46.08
○Thread 189530			6.85	46.45	14.75	0.88	-0.00	46.08
○Thread 189531			6.82	46.45	14.67	0.88	-0.00	46.08
○Thread 189532			6.82	46.45	14.68	0.88	-0.00	46.08
○Thread 189533			6.84	46.45	14.71	0.88	-0.00	46.08
○Thread 189534			6.84	46.45	14.72	0.88	-0.00	46.08
○Thread 189535			6.80	46.45	14.64	0.88	-0.00	46.08
○Thread 189536			6.78	46.45	14.60	0.89	-0.00	46.08
○Thread 189537			6.81	46.45	14.66	0.89	-0.00	46.10
○Thread 189538			6.81	46.45	14.65	0.89	-0.00	46.07
○Thread 189539			6.75	46.45	14.53	0.89	-0.00	46.07
○Thread 189540			6.90	46.45	14.84	0.89	-0.00	46.09
○Thread 189541			6.91	46.45	14.86	0.89	-0.00	46.08
○Thread 189542			6.92	46.45	14.89	0.89	-0.00	46.08
○Thread 189543			6.90	46.45	14.84	0.89	-0.00	46.08
○Thread 189544			6.87	46.45	14.78	0.89	-0.00	46.08
○Thread 189545			6.91	46.45	14.88	0.89	-0.00	46.08
○Thread 189546			6.83	46.45	14.70	0.89	-0.00	46.07
○Thread 189547			6.88	46.45	14.80	0.89	-0.00	46.08
○Thread 189548			6.84	46.45	14.72	0.89	-0.00	46.08
○Thread 189549			6.86	46.45	14.77	0.89	-0.00	46.08
○Thread 189550			6.89	46.45	14.82	0.89	-0.00	46.08
○Thread 189551			6.84	46.45	14.73	0.89	-0.00	46.08
○Thread 189552			6.87	46.45	14.79	0.89	-0.00	46.08
○Thread 189553			6.82	46.45	14.68	0.89	-0.00	46.08
○Thread 189554			6.84	46.45	14.73	0.89	-0.00	46.08
○Thread 189555			6.84	46.45	14.73	0.89	-0.00	46.08
○Thread 189556			6.81	46.45	14.66	0.89	-0.00	46.08
○Thread 189557			6.83	46.45	14.70	0.89	-0.00	46.06
○Thread 189558			6.85	46.45	14.75	0.89	-0.00	46.08
○Thread 189559			6.78	46.45	14.60	0.89	-0.00	46.08
○Thread 189560			6.78	46.45	14.60	0.89	-0.00	46.08
○Thread 189561			6.78	46.45	14.59	0.89	-0.00	46.07
○Thread 189562			6.78	46.45	14.59	0.89	-0.00	46.07
○Thread 189563			6.60	46.45	14.20	0.89	-0.00	46.08

Report Configuration