OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	50.77
►Process 290913+		1	50.77
○Thread 290913			50.77	51.69	98.22	0.00	0.00	51.63

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	27.33
►Process 291082+		2	27.33
○Thread 291082			27.33	28.22	96.88	0.00	0.00	28.16
○Thread 291181			26.98	27.33	98.74	0.89	-0.00	27.01

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	14.23
►Process 291201+		4	14.23
○Thread 291201			14.23	15.14	94.01	0.00	0.00	15.09
○Thread 291300			13.91	14.24	97.66	0.89	-0.00	13.92
○Thread 291301			13.92	14.24	97.73	0.89	-0.00	13.92
○Thread 291302			13.92	14.24	97.73	0.89	-0.00	13.92

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	8.02
►Process 291371+		8	8.02
○Thread 291371			8.02	8.97	89.31	0.00	0.00	8.92
○Thread 291470			7.72	8.08	95.54	0.90	-0.00	7.75
○Thread 291471			7.68	8.08	95.11	0.90	-0.00	7.74
○Thread 291472			7.69	8.08	95.17	0.90	-0.00	7.75
○Thread 291473			7.72	8.08	95.60	0.90	-0.00	7.75
○Thread 291474			7.72	8.07	95.54	0.90	-0.00	7.75
○Thread 291475			7.71	8.07	95.48	0.90	0.00	7.74
○Thread 291476			7.72	8.07	95.55	0.90	-0.00	7.75

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	4.89
►Process 291497+		16	4.89
○Thread 291497			4.89	5.91	82.71	0.00	0.00	5.87
○Thread 291596			4.66	5.01	92.92	0.90	-0.00	4.69
○Thread 291597			4.58	5.01	91.33	0.90	0.00	4.69
○Thread 291598			4.60	5.01	91.63	0.90	-0.00	4.69
○Thread 291599			4.56	5.01	90.94	0.90	-0.00	4.69
○Thread 291600			4.62	5.01	92.04	0.90	-0.00	4.69
○Thread 291601			4.59	5.01	91.54	0.90	-0.00	4.69
○Thread 291602			4.61	5.01	91.94	0.90	-0.00	4.69
○Thread 291603			4.57	5.01	91.14	0.90	-0.00	4.69
○Thread 291604			4.60	5.01	91.74	0.90	0.00	4.69
○Thread 291605			4.57	5.01	91.15	0.90	0.00	4.69
○Thread 291606			4.62	5.01	92.05	0.90	0.00	4.69
○Thread 291607			4.57	5.01	91.05	0.90	0.00	4.69
○Thread 291608			4.58	5.01	91.25	0.90	0.00	4.69
○Thread 291609			4.62	5.01	92.05	0.90	0.00	4.69
○Thread 291610			4.54	5.01	90.46	0.90	-0.00	4.69

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	3.99
►Process 291632+		24	3.99
○Thread 291632			3.99	5.07	78.82	0.00	0.00	5.03
○Thread 291731			3.67	4.17	88.04	0.90	-0.00	3.85
○Thread 291732			3.59	4.17	86.24	0.90	-0.00	3.85
○Thread 291733			3.63	4.17	87.08	0.90	-0.00	3.84
○Thread 291734			3.66	4.17	87.81	0.90	-0.00	3.84
○Thread 291735			3.63	4.17	87.09	0.90	-0.00	3.84
○Thread 291736			3.64	4.17	87.33	0.90	-0.00	3.85
○Thread 291737			3.64	4.17	87.45	0.90	-0.00	3.83
○Thread 291738			3.65	4.17	87.69	0.90	-0.00	3.83
○Thread 291739			3.67	4.17	88.18	0.90	-0.00	3.84
○Thread 291740			3.64	4.17	87.46	0.90	-0.00	3.84
○Thread 291741			3.69	4.17	88.54	0.90	-0.00	3.84
○Thread 291742			3.68	4.17	88.42	0.90	-0.00	3.84
○Thread 291743			3.70	4.17	88.79	0.90	-0.00	3.84
○Thread 291744			3.74	4.17	89.75	0.90	-0.00	3.84
○Thread 291745			3.66	4.17	87.95	0.90	-0.00	3.84
○Thread 291746			3.66	4.17	87.95	0.90	0.00	3.84
○Thread 291747			3.81	4.17	91.56	0.90	0.00	3.84
○Thread 291748			3.67	4.17	88.20	0.90	0.00	3.84
○Thread 291749			3.71	4.17	89.16	0.90	0.00	3.84
○Thread 291750			3.73	4.17	89.52	0.90	0.00	3.83
○Thread 291751			3.66	4.17	87.97	0.90	0.00	3.84
○Thread 291752			3.62	4.17	87.01	0.90	0.00	3.84
○Thread 291753			3.70	4.17	88.81	0.90	-0.00	3.84

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	3.88
►Process 291773+		32	3.88
○Thread 291773			3.88	5.04	76.97	0.00	0.00	5.00
○Thread 291872			3.57	4.14	86.23	0.90	-0.00	3.81
○Thread 291873			3.69	4.14	89.13	0.90	-0.00	3.81
○Thread 291874			3.70	4.14	89.49	0.90	-0.00	3.81
○Thread 291875			3.51	4.14	84.79	0.90	-0.00	3.81
○Thread 291876			3.52	4.14	85.15	0.90	-0.00	3.81
○Thread 291877			3.48	4.14	84.19	0.90	-0.00	3.81
○Thread 291878			3.66	4.14	88.42	0.90	-0.00	3.81
○Thread 291879			3.56	4.14	86.12	0.90	-0.00	3.81
○Thread 291880			3.48	4.14	84.07	0.90	-0.00	3.81
○Thread 291881			3.52	4.14	85.16	0.90	-0.00	3.81
○Thread 291882			3.52	4.14	85.04	0.90	-0.00	3.81
○Thread 291883			3.44	4.14	83.24	0.90	-0.00	3.81
○Thread 291884			3.53	4.14	85.41	0.90	-0.00	3.81
○Thread 291885			3.56	4.14	86.02	0.90	-0.00	3.81
○Thread 291886			3.55	4.14	85.78	0.90	-0.00	3.81
○Thread 291887			3.48	4.14	84.09	0.90	-0.00	3.81
○Thread 291888			3.51	4.14	84.94	0.90	-0.00	3.81
○Thread 291889			3.54	4.14	85.54	0.90	-0.00	3.81
○Thread 291890			3.52	4.14	85.18	0.90	-0.00	3.81
○Thread 291891			3.51	4.14	84.94	0.90	-0.00	3.81
○Thread 291892			3.58	4.14	86.64	0.90	-0.00	3.81
○Thread 291893			3.51	4.14	84.95	0.90	-0.00	3.81
○Thread 291894			3.55	4.14	85.92	0.90	-0.00	3.81
○Thread 291895			3.51	4.14	84.83	0.90	-0.00	3.81
○Thread 291896			3.53	4.14	85.44	0.90	-0.00	3.81
○Thread 291897			3.57	4.14	86.29	0.90	-0.00	3.81
○Thread 291898			3.50	4.14	84.72	0.90	-0.00	3.81
○Thread 291899			3.54	4.14	85.57	0.90	-0.00	3.81
○Thread 291900			3.63	4.14	87.75	0.90	-0.00	3.81
○Thread 291901			3.55	4.14	85.94	0.90	-0.00	3.81
○Thread 291902			3.54	4.14	85.70	0.90	-0.00	3.81

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	3.73
►Process 291970+		40	3.73
○Thread 291970			3.73	4.95	75.30	0.00	0.00	4.92
○Thread 292069			3.42	4.07	84.13	0.89	-0.00	3.72
○Thread 292070			3.47	4.07	85.48	0.89	-0.00	3.73
○Thread 292071			3.47	4.07	85.49	0.89	-0.00	3.73
○Thread 292072			3.39	4.06	83.52	0.89	-0.00	3.73
○Thread 292073			3.34	4.06	82.17	0.89	-0.00	3.73
○Thread 292074			3.55	4.06	87.34	0.89	-0.00	3.73
○Thread 292075			3.41	4.06	83.90	0.89	-0.00	3.73
○Thread 292076			3.31	4.06	81.56	0.89	-0.00	3.73
○Thread 292077			3.40	4.06	83.65	0.89	-0.00	3.73
○Thread 292078			3.54	4.06	87.10	0.89	-0.00	3.73
○Thread 292079			3.42	4.06	84.15	0.89	-0.00	3.73
○Thread 292080			3.39	4.06	83.42	0.89	-0.00	3.73
○Thread 292081			3.43	4.06	84.53	0.89	-0.00	3.73
○Thread 292082			3.40	4.06	83.67	0.89	-0.00	3.72
○Thread 292083			3.34	4.06	82.19	0.89	-0.00	3.73
○Thread 292084			3.40	4.06	83.79	0.89	-0.00	3.73
○Thread 292085			3.25	4.06	79.98	0.89	-0.00	3.73
○Thread 292086			3.44	4.06	84.66	0.89	-0.00	3.73
○Thread 292087			3.41	4.06	83.92	0.89	-0.00	3.73
○Thread 292088			3.30	4.06	81.34	0.89	-0.00	3.72
○Thread 292089			3.34	4.06	82.33	0.89	-0.00	3.73
○Thread 292090			3.46	4.06	85.16	0.89	-0.00	3.73
○Thread 292091			3.41	4.06	84.06	0.89	-0.00	3.72
○Thread 292092			3.42	4.06	84.18	0.89	-0.00	3.73
○Thread 292093			3.46	4.06	85.17	0.89	-0.00	3.73
○Thread 292094			3.45	4.06	84.93	0.89	-0.00	3.73
○Thread 292095			3.36	4.06	82.71	0.89	-0.00	3.73
○Thread 292096			3.35	4.06	82.59	0.89	-0.00	3.73
○Thread 292097			3.44	4.06	84.69	0.89	-0.00	3.73
○Thread 292098			3.35	4.06	82.60	0.89	-0.00	3.73
○Thread 292099			3.46	4.06	85.18	0.89	-0.00	3.73
○Thread 292100			3.40	4.06	83.71	0.89	-0.00	3.73
○Thread 292101			3.37	4.06	83.10	0.89	-0.00	3.73
○Thread 292102			3.49	4.06	85.93	0.89	-0.00	3.73
○Thread 292103			3.43	4.06	84.58	0.89	-0.00	3.73
○Thread 292104			3.49	4.06	85.93	0.89	-0.00	3.72
○Thread 292105			3.38	4.06	83.23	0.89	0.00	3.73
○Thread 292106			3.41	4.06	83.97	0.89	0.00	3.72
○Thread 292107			3.30	4.06	81.39	0.89	-0.00	3.73

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	3.67
►Process 292128+		48	3.67
○Thread 292128			3.67	4.99	73.61	0.00	0.00	4.95
○Thread 292227			3.36	4.09	82.10	0.90	-0.00	3.75
○Thread 292228			3.48	4.09	85.16	0.90	-0.00	3.75
○Thread 292229			3.54	4.09	86.51	0.90	-0.00	3.76
○Thread 292230			3.35	4.09	81.99	0.90	-0.00	3.75
○Thread 292231			3.34	4.09	81.75	0.90	-0.00	3.75
○Thread 292232			3.35	4.09	81.99	0.90	-0.00	3.75
○Thread 292233			3.34	4.09	81.63	0.90	-0.00	3.75
○Thread 292234			3.31	4.09	81.02	0.90	-0.00	3.75
○Thread 292235			3.43	4.09	83.96	0.90	-0.00	3.75
○Thread 292236			3.27	4.09	79.93	0.90	-0.00	3.75
○Thread 292237			3.37	4.09	82.37	0.90	-0.00	3.75
○Thread 292238			3.34	4.09	81.76	0.90	-0.00	3.76
○Thread 292239			3.29	4.09	80.42	0.90	-0.00	3.75
○Thread 292240			3.27	4.09	80.06	0.90	-0.00	3.75
○Thread 292241			3.30	4.09	80.79	0.90	-0.00	3.75
○Thread 292242			3.35	4.09	81.89	0.90	-0.00	3.75
○Thread 292243			3.35	4.09	81.90	0.90	-0.00	3.75
○Thread 292244			3.33	4.09	81.41	0.90	-0.00	3.75
○Thread 292245			3.33	4.09	81.53	0.90	-0.00	3.75
○Thread 292246			3.31	4.09	81.05	0.90	-0.00	3.75
○Thread 292247			3.34	4.09	81.66	0.90	-0.00	3.76
○Thread 292248			3.33	4.09	81.54	0.90	-0.00	3.75
○Thread 292249			3.35	4.09	81.91	0.90	-0.00	3.75
○Thread 292250			3.36	4.09	82.28	0.90	-0.00	3.75
○Thread 292251			3.30	4.09	80.81	0.90	-0.00	3.75
○Thread 292252			3.32	4.09	81.18	0.90	-0.00	3.75
○Thread 292253			3.38	4.09	82.65	0.90	-0.00	3.76
○Thread 292254			3.33	4.09	81.55	0.90	-0.00	3.75
○Thread 292255			3.28	4.09	80.21	0.90	-0.00	3.74
○Thread 292256			3.33	4.09	81.56	0.90	-0.00	3.75
○Thread 292257			3.35	4.09	81.93	0.90	-0.00	3.75
○Thread 292258			3.30	4.09	80.83	0.90	-0.00	3.75
○Thread 292259			3.36	4.09	82.18	0.90	-0.00	3.75
○Thread 292260			3.40	4.09	83.16	0.90	-0.00	3.76
○Thread 292261			3.33	4.09	81.57	0.90	-0.00	3.75
○Thread 292262			3.19	4.09	78.15	0.90	-0.00	3.75
○Thread 292263			3.35	4.09	82.06	0.90	-0.00	3.74
○Thread 292264			3.36	4.09	82.19	0.90	-0.00	3.75
○Thread 292265			3.36	4.09	82.31	0.90	-0.00	3.75
○Thread 292266			3.30	4.09	80.85	0.90	-0.00	3.75
○Thread 292267			3.33	4.09	81.58	0.90	-0.00	3.76
○Thread 292268			3.48	4.09	85.26	0.90	-0.00	3.75
○Thread 292269			3.38	4.09	82.81	0.90	-0.00	3.75
○Thread 292270			3.33	4.09	81.59	0.90	-0.00	3.75
○Thread 292271			3.30	4.09	80.74	0.90	-0.00	3.75
○Thread 292272			3.36	4.09	82.21	0.91	-0.00	3.76
○Thread 292273			3.34	4.09	81.84	0.91	-0.00	3.74

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	3.49
►Process 292294+		56	3.49
○Thread 292294			3.49	4.92	71.00	0.00	0.00	4.86
○Thread 292393			3.21	4.01	80.09	0.90	-0.00	3.67
○Thread 292394			3.31	4.01	82.46	0.90	-0.00	3.67
○Thread 292395			3.18	4.01	79.22	0.90	-0.00	3.67
○Thread 292396			3.12	4.01	77.73	0.90	-0.00	3.66
○Thread 292397			3.22	4.01	80.35	0.90	-0.00	3.67
○Thread 292398			3.21	4.01	80.10	0.90	-0.00	3.66
○Thread 292399			3.21	4.01	80.10	0.90	-0.00	3.68
○Thread 292400			3.26	4.01	81.35	0.90	-0.00	3.67
○Thread 292401			3.27	4.01	81.60	0.90	-0.00	3.66
○Thread 292402			3.16	4.01	78.74	0.90	-0.00	3.67
○Thread 292403			3.21	4.01	79.99	0.90	-0.00	3.66
○Thread 292404			3.07	4.01	76.63	0.90	-0.00	3.66
○Thread 292405			3.19	4.01	79.49	0.90	-0.00	3.67
○Thread 292406			3.24	4.01	80.87	0.90	-0.00	3.66
○Thread 292407			3.13	4.01	78.13	0.90	-0.00	3.66
○Thread 292408			3.23	4.01	80.50	0.90	-0.00	3.67
○Thread 292409			3.19	4.01	79.63	0.90	-0.00	3.67
○Thread 292410			3.19	4.01	79.63	0.90	-0.00	3.67
○Thread 292411			3.21	4.01	80.13	0.90	-0.00	3.66
○Thread 292412			3.36	4.01	83.75	0.90	-0.00	3.66
○Thread 292413			3.23	4.01	80.51	0.90	-0.00	3.67
○Thread 292414			3.25	4.01	81.01	0.90	-0.00	3.67
○Thread 292415			3.21	4.01	80.02	0.90	-0.00	3.67
○Thread 292416			3.34	4.01	83.26	0.90	-0.00	3.67
○Thread 292417			3.25	4.01	81.02	0.90	-0.00	3.67
○Thread 292418			3.33	4.01	83.01	0.90	-0.00	3.66
○Thread 292419			3.21	4.01	80.15	0.90	-0.00	3.68
○Thread 292420			3.21	4.01	80.03	0.90	-0.00	3.67
○Thread 292421			3.15	4.01	78.66	0.90	-0.00	3.68
○Thread 292422			3.22	4.01	80.28	0.90	-0.00	3.67
○Thread 292423			3.18	4.01	79.41	0.90	-0.00	3.67
○Thread 292424			3.27	4.01	81.53	0.90	-0.00	3.68
○Thread 292425			3.31	4.01	82.66	0.90	-0.00	3.67
○Thread 292426			3.23	4.01	80.66	0.90	-0.00	3.67
○Thread 292427			3.24	4.01	80.92	0.90	-0.00	3.66
○Thread 292428			3.28	4.01	81.79	0.90	-0.00	3.68
○Thread 292429			3.19	4.01	79.55	0.91	-0.00	3.67
○Thread 292430			3.19	4.01	79.55	0.91	-0.00	3.67
○Thread 292431			3.29	4.01	82.17	0.91	-0.00	3.67
○Thread 292432			3.34	4.01	83.42	0.91	-0.00	3.67
○Thread 292433			3.25	4.01	81.18	0.91	-0.00	3.68
○Thread 292434			3.22	4.01	80.43	0.91	-0.00	3.67
○Thread 292435			3.29	4.01	82.06	0.91	0.00	3.68
○Thread 292436			3.23	4.01	80.69	0.91	-0.00	3.67
○Thread 292437			3.21	4.01	80.07	0.91	-0.00	3.66
○Thread 292438			3.21	4.01	80.19	0.91	-0.00	3.67
○Thread 292439			3.17	4.01	79.07	0.91	-0.00	3.68
○Thread 292440			3.17	4.01	79.20	0.91	-0.00	3.67
○Thread 292441			3.15	4.01	78.70	0.91	-0.00	3.67
○Thread 292442			3.33	4.01	83.07	0.91	-0.00	3.67
○Thread 292443			3.23	4.01	80.70	0.91	-0.00	3.68
○Thread 292444			3.29	4.01	82.20	0.91	-0.00	3.67
○Thread 292445			3.16	4.01	78.96	0.91	-0.00	3.67
○Thread 292446			3.31	4.01	82.71	0.91	-0.00	3.67
○Thread 292447			3.21	4.01	80.09	0.91	-0.00	3.67

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	3.44
►Process 292467+		64	3.44
○Thread 292467			3.44	4.89	70.28	0.00	0.00	4.83
○Thread 292566			3.07	3.99	76.89	0.90	-0.00	3.64
○Thread 292567			3.06	3.99	76.64	0.90	-0.00	3.65
○Thread 292568			3.12	3.99	78.14	0.90	-0.00	3.64
○Thread 292569			3.16	3.99	79.27	0.90	-0.00	3.63
○Thread 292570			3.31	3.99	82.91	0.90	-0.00	3.64
○Thread 292571			3.11	3.99	77.90	0.90	-0.00	3.64
○Thread 292572			2.99	3.99	74.90	0.90	-0.00	3.63
○Thread 292573			3.01	3.99	75.53	0.90	-0.00	3.64
○Thread 292574			3.07	3.99	76.91	0.90	-0.00	3.64
○Thread 292575			3.16	3.99	79.29	0.90	-0.00	3.64
○Thread 292576			3.18	3.99	79.67	0.90	-0.00	3.64
○Thread 292577			3.09	3.99	77.41	0.90	-0.00	3.63
○Thread 292578			3.05	3.99	76.54	0.90	-0.00	3.64
○Thread 292579			3.11	3.99	78.04	0.90	-0.00	3.64
○Thread 292580			3.10	3.99	77.80	0.90	-0.00	3.64
○Thread 292581			3.09	3.99	77.55	0.90	-0.00	3.64
○Thread 292582			3.14	3.99	78.80	0.90	-0.00	3.64
○Thread 292583			3.01	3.99	75.42	0.90	-0.00	3.64
○Thread 292584			3.07	3.99	76.93	0.90	-0.00	3.64
○Thread 292585			3.06	3.99	76.68	0.90	-0.00	3.64
○Thread 292586			3.18	3.99	79.81	0.90	-0.00	3.64
○Thread 292587			3.16	3.99	79.32	0.90	-0.00	3.64
○Thread 292588			3.22	3.99	80.70	0.90	-0.00	3.64
○Thread 292589			3.15	3.99	78.94	0.90	-0.00	3.63
○Thread 292590			3.14	3.99	78.70	0.90	-0.00	3.64
○Thread 292591			3.07	3.99	76.94	0.90	-0.00	3.65
○Thread 292592			3.08	3.99	77.32	0.90	-0.00	3.64
○Thread 292593			3.12	3.99	78.33	0.90	-0.00	3.64
○Thread 292594			3.23	3.99	81.09	0.90	-0.00	3.64
○Thread 292595			3.14	3.99	78.83	0.90	-0.00	3.64
○Thread 292596			3.22	3.99	80.71	0.91	-0.00	3.64
○Thread 292597			3.07	3.99	77.08	0.91	-0.00	3.64
○Thread 292598			3.00	3.99	75.33	0.91	-0.00	3.64
○Thread 292599			3.06	3.99	76.71	0.91	-0.00	3.64
○Thread 292600			3.12	3.99	78.22	0.91	-0.00	3.64
○Thread 292601			3.08	3.99	77.34	0.91	-0.00	3.64
○Thread 292602			3.20	3.99	80.23	0.91	-0.00	3.64
○Thread 292603			3.19	3.99	80.10	0.91	-0.00	3.64
○Thread 292604			3.09	3.99	77.47	0.91	-0.00	3.64
○Thread 292605			3.12	3.99	78.35	0.91	-0.00	3.64
○Thread 292606			3.15	3.99	78.98	0.91	-0.00	3.64
○Thread 292607			3.11	3.99	77.98	0.91	-0.00	3.64
○Thread 292608			3.18	3.99	79.86	0.91	-0.00	3.64
○Thread 292609			3.16	3.99	79.24	0.91	-0.00	3.64
○Thread 292610			3.21	3.99	80.50	0.91	-0.00	3.64
○Thread 292611			3.20	3.99	80.37	0.91	-0.00	3.64
○Thread 292612			3.23	3.99	81.00	0.91	-0.00	3.64
○Thread 292613			3.21	3.99	80.50	0.91	-0.00	3.64
○Thread 292614			3.08	3.99	77.24	0.91	-0.00	3.64
○Thread 292615			3.31	3.99	83.02	0.91	-0.00	3.64
○Thread 292616			3.19	3.99	80.13	0.91	-0.00	3.64
○Thread 292617			3.12	3.99	78.25	0.91	-0.00	3.64
○Thread 292618			3.06	3.99	76.75	0.91	-0.00	3.64
○Thread 292619			3.08	3.99	77.26	0.91	-0.00	3.64
○Thread 292620			3.09	3.99	77.63	0.91	-0.00	3.64
○Thread 292621			3.02	3.99	75.88	0.91	-0.00	3.64
○Thread 292622			3.12	3.99	78.27	0.91	-0.00	3.64
○Thread 292623			3.24	3.99	81.40	0.91	-0.00	3.64
○Thread 292624			3.14	3.99	78.90	0.91	-0.00	3.63
○Thread 292625			3.22	3.99	80.78	0.91	-0.00	3.64
○Thread 292626			3.13	3.99	78.53	0.91	-0.00	3.64
○Thread 292627			3.11	3.99	78.15	0.91	-0.00	3.64
○Thread 292628			3.14	3.99	78.91	0.91	-0.00	3.64

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	3.55
►Process 292697+		72	3.55
○Thread 292697			3.55	4.95	71.85	0.00	0.00	4.89
○Thread 292796			3.11	4.04	77.12	0.91	-0.00	3.68
○Thread 292797			3.22	4.04	79.72	0.91	-0.00	3.69
○Thread 292798			3.37	4.04	83.44	0.91	-0.00	3.69
○Thread 292799			3.33	4.04	82.57	0.91	-0.00	3.69
○Thread 292800			3.37	4.04	83.56	0.91	-0.00	3.69
○Thread 292801			3.10	4.04	76.88	0.91	-0.00	3.69
○Thread 292802			3.07	4.04	76.14	0.91	-0.00	3.69
○Thread 292803			3.17	4.04	78.50	0.91	-0.00	3.68
○Thread 292804			3.13	4.04	77.63	0.91	-0.00	3.68
○Thread 292805			3.12	4.04	77.14	0.91	-0.00	3.69
○Thread 292806			3.24	4.04	80.36	0.91	-0.00	3.69
○Thread 292807			3.10	4.04	76.90	0.91	-0.00	3.68
○Thread 292808			3.15	4.04	78.14	0.91	-0.00	3.68
○Thread 292809			3.19	4.04	79.13	0.91	-0.00	3.69
○Thread 292810			3.08	4.04	76.41	0.91	-0.00	3.69
○Thread 292811			3.11	4.04	77.03	0.91	-0.00	3.69
○Thread 292812			3.12	4.04	77.40	0.91	-0.00	3.68
○Thread 292813			3.13	4.04	77.65	0.91	-0.00	3.69
○Thread 292814			3.01	4.04	74.68	0.91	-0.00	3.69
○Thread 292815			3.13	4.04	77.53	0.91	-0.00	3.68
○Thread 292816			3.11	4.04	77.04	0.91	-0.00	3.69
○Thread 292817			3.13	4.04	77.66	0.91	-0.00	3.68
○Thread 292818			3.19	4.04	79.02	0.91	-0.00	3.69
○Thread 292819			3.11	4.04	77.17	0.91	-0.00	3.69
○Thread 292820			3.00	4.04	74.32	0.91	-0.00	3.68
○Thread 292821			3.05	4.04	75.69	0.91	-0.00	3.69
○Thread 292822			3.16	4.04	78.41	0.91	-0.00	3.69
○Thread 292823			3.04	4.04	75.44	0.91	-0.00	3.69
○Thread 292824			3.04	4.04	75.32	0.91	-0.00	3.68
○Thread 292825			3.03	4.04	75.20	0.91	-0.00	3.68
○Thread 292826			2.96	4.04	73.47	0.91	-0.00	3.68
○Thread 292827			3.06	4.04	75.95	0.91	-0.00	3.68
○Thread 292828			3.01	4.04	74.59	0.91	-0.00	3.69
○Thread 292829			3.11	4.04	77.19	0.91	-0.00	3.68
○Thread 292830			3.07	4.04	76.08	0.91	-0.00	3.69
○Thread 292831			3.01	4.04	74.59	0.91	-0.00	3.69
○Thread 292832			3.09	4.04	76.58	0.91	-0.00	3.69
○Thread 292833			2.96	4.04	73.48	0.91	-0.00	3.69
○Thread 292834			3.07	4.03	76.21	0.91	-0.00	3.69
○Thread 292835			3.04	4.03	75.47	0.91	-0.00	3.69
○Thread 292836			2.95	4.03	73.24	0.91	-0.00	3.69
○Thread 292837			3.08	4.03	76.34	0.91	-0.00	3.68
○Thread 292838			3.07	4.03	76.22	0.91	-0.00	3.69
○Thread 292839			3.06	4.03	75.85	0.91	-0.00	3.68
○Thread 292840			3.01	4.03	74.74	0.91	-0.00	3.69
○Thread 292841			2.99	4.03	74.24	0.91	-0.00	3.68
○Thread 292842			3.05	4.03	75.61	0.91	-0.00	3.69
○Thread 292843			3.10	4.03	76.97	0.91	-0.00	3.68
○Thread 292844			3.06	4.03	75.86	0.91	-0.00	3.69
○Thread 292845			3.09	4.03	76.73	0.91	-0.00	3.69
○Thread 292846			3.10	4.03	76.86	0.91	-0.00	3.68
○Thread 292847			3.01	4.03	74.63	0.91	-0.00	3.68
○Thread 292848			3.09	4.03	76.74	0.91	-0.00	3.69
○Thread 292849			3.06	4.03	75.87	0.91	-0.00	3.68
○Thread 292850			3.07	4.03	76.24	0.91	-0.00	3.69
○Thread 292851			3.05	4.03	75.63	0.91	-0.00	3.68
○Thread 292852			3.04	4.03	75.38	0.91	-0.00	3.68
○Thread 292853			3.09	4.03	76.75	0.92	-0.00	3.68
○Thread 292854			3.08	4.03	76.25	0.92	-0.00	3.69
○Thread 292855			3.20	4.03	79.35	0.92	-0.00	3.67
○Thread 292856			3.15	4.03	78.24	0.92	-0.00	3.67
○Thread 292857			3.19	4.03	79.11	0.92	-0.00	3.68
○Thread 292858			3.26	4.03	80.97	0.92	-0.00	3.69
○Thread 292859			3.42	4.03	84.94	0.92	-0.00	3.69
○Thread 292860			3.20	4.03	79.49	0.92	-0.00	3.69
○Thread 292861			3.02	4.03	74.90	0.92	-0.00	3.69
○Thread 292862			3.08	4.03	76.39	0.92	-0.00	3.69
○Thread 292863			3.04	4.03	75.53	0.92	-0.00	3.67
○Thread 292864			3.04	4.03	75.53	0.92	-0.00	3.69
○Thread 292865			3.08	4.03	76.52	0.92	-0.00	3.68
○Thread 292866			3.10	4.03	76.90	0.92	-0.00	3.68

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	3.65
►Process 292886+		80	3.65
○Thread 292886			3.65	5.09	71.80	0.00	0.00	5.05
○Thread 292985			3.21	4.19	76.75	0.90	-0.00	3.84
○Thread 292986			3.14	4.19	74.96	0.90	-0.00	3.84
○Thread 292987			3.18	4.19	75.92	0.90	-0.00	3.84
○Thread 292988			3.10	4.19	74.13	0.90	-0.00	3.83
○Thread 292989			3.15	4.19	75.33	0.90	-0.00	3.84
○Thread 292990			3.31	4.19	79.03	0.90	-0.00	3.83
○Thread 292991			3.09	4.19	73.90	0.90	-0.00	3.84
○Thread 292992			3.12	4.19	74.62	0.90	-0.00	3.83
○Thread 292993			3.18	4.19	75.94	0.90	-0.00	3.84
○Thread 292994			3.07	4.19	73.31	0.90	-0.00	3.84
○Thread 292995			3.14	4.19	75.10	0.90	-0.00	3.84
○Thread 292996			3.17	4.19	75.70	0.90	-0.00	3.84
○Thread 292997			3.15	4.19	75.23	0.90	-0.00	3.84
○Thread 292998			3.14	4.19	75.11	0.90	-0.00	3.83
○Thread 292999			3.10	4.19	74.04	0.90	-0.00	3.84
○Thread 293000			3.16	4.19	75.59	0.90	-0.00	3.84
○Thread 293001			3.19	4.19	76.31	0.90	-0.00	3.84
○Thread 293002			3.12	4.19	74.52	0.90	-0.00	3.84
○Thread 293003			3.18	4.19	75.96	0.90	-0.00	3.84
○Thread 293004			3.19	4.19	76.20	0.90	-0.00	3.84
○Thread 293005			3.18	4.19	75.96	0.90	-0.00	3.84
○Thread 293006			3.10	4.19	74.05	0.90	-0.00	3.84
○Thread 293007			3.14	4.19	75.13	0.90	-0.00	3.84
○Thread 293008			3.05	4.19	72.86	0.90	-0.00	3.84
○Thread 293009			3.09	4.19	73.82	0.90	-0.00	3.83
○Thread 293010			3.09	4.19	73.82	0.90	-0.00	3.84
○Thread 293011			3.15	4.19	75.26	0.91	-0.00	3.84
○Thread 293012			3.10	4.19	74.18	0.91	-0.00	3.84
○Thread 293013			3.11	4.19	74.30	0.91	-0.00	3.84
○Thread 293014			3.03	4.19	72.39	0.91	-0.00	3.84
○Thread 293015			3.11	4.19	74.31	0.91	-0.00	3.84
○Thread 293016			3.04	4.19	72.64	0.91	-0.00	3.83
○Thread 293017			3.17	4.19	75.75	0.91	-0.00	3.84
○Thread 293018			3.14	4.18	75.15	0.91	-0.00	3.84
○Thread 293019			2.99	4.18	71.57	0.91	-0.00	3.84
○Thread 293020			3.05	4.18	72.88	0.91	-0.00	3.83
○Thread 293021			3.00	4.18	71.81	0.91	-0.00	3.84
○Thread 293022			3.17	4.18	75.88	0.91	-0.00	3.84
○Thread 293023			3.12	4.18	74.56	0.91	-0.00	3.83
○Thread 293024			3.05	4.18	73.01	0.91	-0.00	3.84
○Thread 293025			3.13	4.18	74.93	0.91	-0.00	3.84
○Thread 293026			3.09	4.18	73.97	0.91	-0.00	3.84
○Thread 293027			3.09	4.18	73.97	0.91	-0.00	3.85
○Thread 293028			3.07	4.18	73.50	0.91	-0.00	3.84
○Thread 293029			3.13	4.18	74.93	0.91	-0.00	3.83
○Thread 293030			3.10	4.18	74.10	0.91	-0.00	3.84
○Thread 293031			2.99	4.18	71.59	0.91	-0.00	3.84
○Thread 293032			3.06	4.18	73.27	0.91	-0.00	3.84
○Thread 293033			3.07	4.18	73.51	0.91	-0.00	3.84
○Thread 293034			3.08	4.18	73.75	0.91	-0.00	3.84
○Thread 293035			3.06	4.18	73.27	0.91	-0.00	3.84
○Thread 293036			3.12	4.18	74.71	0.91	-0.00	3.84
○Thread 293037			3.14	4.18	75.07	0.91	-0.00	3.84
○Thread 293038			3.01	4.18	72.08	0.91	-0.00	3.84
○Thread 293039			3.15	4.18	75.43	0.91	-0.00	3.84
○Thread 293040			3.20	4.18	76.51	0.91	-0.00	3.84
○Thread 293041			3.06	4.18	73.17	0.91	-0.00	3.83
○Thread 293042			3.09	4.18	74.00	0.91	-0.00	3.84
○Thread 293043			3.06	4.18	73.29	0.91	-0.00	3.84
○Thread 293044			3.18	4.18	76.04	0.91	-0.00	3.84
○Thread 293045			3.05	4.18	73.05	0.91	-0.00	3.84
○Thread 293046			3.03	4.18	72.46	0.91	-0.00	3.84
○Thread 293047			3.05	4.18	72.94	0.91	-0.00	3.84
○Thread 293048			3.06	4.18	73.18	0.91	-0.00	3.84
○Thread 293049			3.10	4.18	74.26	0.91	-0.00	3.84
○Thread 293050			3.13	4.18	74.98	0.91	-0.00	3.84
○Thread 293051			3.10	4.18	74.14	0.91	-0.00	3.84
○Thread 293052			3.06	4.18	73.31	0.91	-0.00	3.84
○Thread 293053			3.00	4.18	71.75	0.91	-0.00	3.84
○Thread 293054			3.14	4.18	75.10	0.91	-0.00	3.84
○Thread 293055			3.10	4.18	74.27	0.91	-0.00	3.84
○Thread 293056			3.02	4.18	72.36	0.91	-0.00	3.83
○Thread 293057			3.10	4.18	74.27	0.91	-0.00	3.84
○Thread 293058			3.08	4.18	73.68	0.91	-0.00	3.84
○Thread 293059			3.13	4.18	75.00	0.91	-0.00	3.83
○Thread 293060			3.00	4.18	71.77	0.91	-0.00	3.84
○Thread 293061			3.12	4.18	74.76	0.91	-0.00	3.83
○Thread 293062			3.05	4.18	73.09	0.91	-0.00	3.83
○Thread 293063			2.97	4.18	71.06	0.91	-0.00	3.83

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	3.51
►Process 293083+		88	3.51
○Thread 293083			3.51	5.17	68.02	0.00	0.00	5.13
○Thread 293182			3.29	4.26	77.29	0.90	-0.00	3.91
○Thread 293183			3.12	4.26	73.19	0.91	-0.00	3.91
○Thread 293184			3.15	4.26	74.01	0.91	-0.00	3.91
○Thread 293185			3.16	4.26	74.13	0.91	-0.00	3.92
○Thread 293186			3.07	4.26	72.14	0.91	-0.00	3.91
○Thread 293187			3.33	4.26	78.13	0.91	-0.00	3.91
○Thread 293188			3.25	4.26	76.25	0.91	-0.00	3.91
○Thread 293189			3.06	4.26	71.80	0.91	-0.00	3.91
○Thread 293190			3.17	4.26	74.50	0.91	-0.00	3.91
○Thread 293191			3.17	4.26	74.50	0.91	-0.00	3.91
○Thread 293192			3.04	4.26	71.33	0.91	-0.00	3.91
○Thread 293193			3.06	4.26	71.80	0.91	-0.00	3.91
○Thread 293194			3.13	4.26	73.57	0.91	-0.00	3.91
○Thread 293195			3.12	4.26	73.22	0.91	-0.00	3.91
○Thread 293196			3.07	4.26	72.16	0.91	-0.00	3.91
○Thread 293197			3.25	4.26	76.27	0.91	-0.00	3.91
○Thread 293198			3.18	4.26	74.75	0.91	-0.00	3.91
○Thread 293199			3.13	4.26	73.46	0.91	-0.00	3.91
○Thread 293200			3.18	4.26	74.63	0.91	-0.00	3.92
○Thread 293201			3.20	4.26	75.22	0.91	-0.00	3.91
○Thread 293202			3.12	4.26	73.35	0.91	-0.00	3.91
○Thread 293203			3.23	4.26	75.93	0.91	-0.00	3.91
○Thread 293204			3.05	4.26	71.59	0.91	-0.00	3.91
○Thread 293205			3.01	4.26	70.65	0.91	-0.00	3.91
○Thread 293206			3.13	4.26	73.59	0.91	-0.00	3.92
○Thread 293207			3.08	4.26	72.42	0.91	-0.00	3.91
○Thread 293208			3.07	4.26	72.07	0.91	-0.00	3.91
○Thread 293209			3.19	4.26	75.00	0.91	-0.00	3.90
○Thread 293210			3.20	4.26	75.24	0.91	-0.00	3.91
○Thread 293211			3.26	4.26	76.53	0.91	-0.00	3.91
○Thread 293212			3.12	4.26	73.25	0.91	-0.00	3.91
○Thread 293213			3.17	4.26	74.54	0.91	-0.00	3.91
○Thread 293214			3.12	4.26	73.25	0.91	-0.00	3.91
○Thread 293215			3.16	4.26	74.31	0.91	-0.00	3.91
○Thread 293216			3.12	4.26	73.26	0.91	-0.00	3.91
○Thread 293217			3.11	4.26	73.02	0.91	-0.00	3.90
○Thread 293218			3.16	4.26	74.20	0.91	-0.00	3.91
○Thread 293219			3.11	4.26	73.15	0.91	-0.00	3.91
○Thread 293220			3.05	4.26	71.74	0.91	-0.00	3.92
○Thread 293221			3.14	4.26	73.85	0.91	-0.00	3.91
○Thread 293222			3.18	4.26	74.68	0.91	-0.00	3.91
○Thread 293223			3.15	4.26	73.98	0.91	-0.00	3.92
○Thread 293224			3.15	4.26	74.09	0.91	-0.00	3.92
○Thread 293225			3.13	4.26	73.63	0.91	-0.00	3.91
○Thread 293226			3.13	4.26	73.63	0.91	-0.00	3.91
○Thread 293227			3.20	4.26	75.16	0.91	-0.00	3.91
○Thread 293228			3.06	4.26	71.99	0.91	-0.00	3.91
○Thread 293229			3.20	4.26	75.16	0.91	-0.00	3.90
○Thread 293230			3.02	4.26	71.05	0.91	-0.00	3.90
○Thread 293231			3.20	4.26	75.28	0.91	-0.00	3.92
○Thread 293232			3.18	4.26	74.70	0.91	-0.00	3.91
○Thread 293233			3.32	4.26	78.11	0.91	-0.00	3.90
○Thread 293234			3.08	4.26	72.47	0.91	-0.00	3.91
○Thread 293235			3.22	4.26	75.76	0.91	-0.00	3.89
○Thread 293236			3.13	4.26	73.65	0.91	-0.00	3.92
○Thread 293237			3.08	4.26	72.48	0.91	-0.00	3.89
○Thread 293238			3.19	4.26	74.94	0.91	-0.00	3.91
○Thread 293239			3.05	4.26	71.66	0.91	-0.00	3.91
○Thread 293240			3.09	4.26	72.72	0.91	-0.00	3.91
○Thread 293241			3.11	4.26	73.19	0.91	-0.00	3.91
○Thread 293242			3.11	4.26	73.19	0.91	-0.00	3.91
○Thread 293243			3.15	4.26	74.13	0.91	-0.00	3.91
○Thread 293244			3.16	4.26	74.37	0.91	-0.00	3.91
○Thread 293245			3.13	4.26	73.55	0.91	-0.00	3.92
○Thread 293246			3.19	4.26	75.08	0.91	-0.00	3.91
○Thread 293247			3.12	4.26	73.32	0.91	-0.00	3.90
○Thread 293248			3.12	4.26	73.44	0.91	-0.00	3.91
○Thread 293249			3.09	4.26	72.62	0.91	-0.00	3.91
○Thread 293250			3.15	4.26	74.03	0.91	-0.00	3.91
○Thread 293251			3.15	4.25	74.15	0.91	-0.00	3.91
○Thread 293252			3.18	4.25	74.86	0.91	-0.00	3.91
○Thread 293253			3.20	4.25	75.21	0.91	-0.00	3.91
○Thread 293254			3.16	4.25	74.27	0.91	-0.00	3.91
○Thread 293255			3.12	4.25	73.45	0.91	-0.00	3.91
○Thread 293256			3.20	4.25	75.33	0.91	-0.00	3.90
○Thread 293257			3.20	4.25	75.22	0.91	-0.00	3.91
○Thread 293258			3.16	4.25	74.40	0.91	-0.00	3.91
○Thread 293259			3.15	4.25	74.16	0.91	-0.00	3.91
○Thread 293260			3.21	4.25	75.46	0.91	-0.00	3.91
○Thread 293261			3.20	4.25	75.23	0.91	-0.00	3.91
○Thread 293262			3.13	4.25	73.58	0.91	-0.00	3.91
○Thread 293263			3.20	4.25	75.35	0.91	-0.00	3.91
○Thread 293264			3.25	4.25	76.52	0.91	-0.00	3.91
○Thread 293265			3.32	4.25	78.17	0.91	-0.00	3.91
○Thread 293266			3.17	4.25	74.65	0.91	-0.00	3.91
○Thread 293267			3.28	4.25	77.12	0.91	-0.00	3.91
○Thread 293268			3.12	4.25	73.36	0.91	-0.00	3.91

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	3.77
►Process 293288+		96	3.77
○Thread 293288			3.77	5.30	71.09	0.00	0.00	5.26
○Thread 293387			3.34	4.40	75.87	0.90	-0.00	4.05
○Thread 293388			3.51	4.40	79.73	0.90	-0.00	4.06
○Thread 293389			3.38	4.40	76.78	0.90	-0.00	4.05
○Thread 293390			3.40	4.40	77.35	0.90	-0.00	4.05
○Thread 293391			3.52	4.40	80.08	0.90	-0.00	4.05
○Thread 293392			3.55	4.40	80.65	0.90	-0.00	4.05
○Thread 293393			3.54	4.40	80.43	0.90	-0.00	4.05
○Thread 293394			3.55	4.40	80.65	0.90	-0.00	4.04
○Thread 293395			3.37	4.40	76.68	0.90	-0.00	4.04
○Thread 293396			3.43	4.40	77.93	0.90	-0.00	4.05
○Thread 293397			3.36	4.40	76.46	0.90	-0.00	4.03
○Thread 293398			3.43	4.40	77.94	0.90	-0.00	4.05
○Thread 293399			3.27	4.40	74.42	0.90	-0.00	4.05
○Thread 293400			3.53	4.40	80.21	0.90	-0.00	4.04
○Thread 293401			3.36	4.40	76.35	0.90	-0.00	4.05
○Thread 293402			3.44	4.40	78.29	0.90	-0.00	4.04
○Thread 293403			3.40	4.40	77.26	0.90	-0.00	4.04
○Thread 293404			3.42	4.40	77.83	0.90	-0.00	4.05
○Thread 293405			3.45	4.40	78.52	0.90	-0.00	4.03
○Thread 293406			3.43	4.40	78.07	0.90	-0.00	4.05
○Thread 293407			3.39	4.40	77.05	0.90	-0.00	4.05
○Thread 293408			3.26	4.40	74.21	0.90	-0.00	4.05
○Thread 293409			3.11	4.40	70.80	0.90	-0.00	4.05
○Thread 293410			3.33	4.40	75.80	0.90	-0.00	4.05
○Thread 293411			3.20	4.40	72.85	0.90	-0.00	4.04
○Thread 293412			3.41	4.40	77.62	0.90	-0.00	4.03
○Thread 293413			3.25	4.40	73.99	0.90	-0.00	4.05
○Thread 293414			3.42	4.40	77.74	0.90	-0.00	4.04
○Thread 293415			3.36	4.40	76.49	0.90	-0.00	4.05
○Thread 293416			3.49	4.40	79.34	0.90	-0.00	4.05
○Thread 293417			3.31	4.40	75.36	0.90	-0.00	4.04
○Thread 293418			3.25	4.40	74.00	0.90	-0.00	4.05
○Thread 293419			3.22	4.40	73.20	0.90	-0.00	4.05
○Thread 293420			3.17	4.40	72.18	0.90	-0.00	4.05
○Thread 293421			3.32	4.40	75.48	0.91	-0.00	4.05
○Thread 293422			3.19	4.40	72.53	0.91	-0.00	4.05
○Thread 293423			3.19	4.40	72.53	0.91	-0.00	4.05
○Thread 293424			3.30	4.40	75.03	0.91	-0.00	4.05
○Thread 293425			3.27	4.40	74.35	0.91	-0.00	4.04
○Thread 293426			3.32	4.40	75.49	0.91	-0.00	4.05
○Thread 293427			3.21	4.40	72.99	0.91	-0.00	4.04
○Thread 293428			3.31	4.40	75.38	0.91	-0.00	4.05
○Thread 293429			3.32	4.40	75.50	0.91	-0.00	4.05
○Thread 293430			3.17	4.40	72.20	0.91	-0.00	4.05
○Thread 293431			3.23	4.40	73.45	0.91	-0.00	4.05
○Thread 293432			3.07	4.40	69.93	0.91	-0.00	4.05
○Thread 293433			3.19	4.40	72.55	0.91	-0.00	4.03
○Thread 293434			3.17	4.40	72.21	0.91	-0.00	4.05
○Thread 293435			3.19	4.40	72.67	0.91	-0.00	4.02
○Thread 293436			3.28	4.40	74.71	0.91	-0.00	4.05
○Thread 293437			3.26	4.40	74.15	0.91	-0.00	4.04
○Thread 293438			3.30	4.40	75.17	0.91	-0.00	4.04
○Thread 293439			3.07	4.40	69.94	0.91	-0.00	4.05
○Thread 293440			3.20	4.40	72.90	0.91	-0.00	4.05
○Thread 293441			3.15	4.40	71.65	0.91	-0.00	4.05
○Thread 293442			3.21	4.40	73.13	0.91	-0.00	4.04
○Thread 293443			3.22	4.40	73.36	0.91	-0.00	4.01
○Thread 293444			3.24	4.40	73.82	0.91	-0.00	4.05
○Thread 293445			3.19	4.40	72.68	0.91	-0.00	4.05
○Thread 293446			3.40	4.40	77.46	0.91	-0.00	4.04
○Thread 293447			3.20	4.40	72.80	0.91	-0.00	4.05
○Thread 293448			3.21	4.40	73.14	0.91	-0.00	4.05
○Thread 293449			3.18	4.40	72.46	0.91	-0.00	4.05
○Thread 293450			3.18	4.40	72.35	0.91	-0.00	4.03
○Thread 293451			3.29	4.40	74.86	0.91	-0.00	4.05
○Thread 293452			3.29	4.39	74.86	0.91	-0.00	4.04
○Thread 293453			3.21	4.39	73.04	0.91	-0.00	4.03
○Thread 293454			3.23	4.39	73.61	0.91	-0.00	4.05
○Thread 293455			3.13	4.39	71.22	0.91	-0.00	4.05
○Thread 293456			3.15	4.39	71.79	0.91	-0.00	4.05
○Thread 293457			3.16	4.39	71.91	0.91	-0.00	4.05
○Thread 293458			3.26	4.39	74.19	0.91	-0.00	4.05
○Thread 293459			3.11	4.39	70.78	0.91	-0.00	4.00
○Thread 293460			3.22	4.39	73.39	0.91	-0.00	4.04
○Thread 293461			3.24	4.39	73.74	0.91	-0.00	4.02
○Thread 293462			3.25	4.39	74.08	0.91	-0.00	4.05
○Thread 293463			3.19	4.39	72.60	0.91	-0.00	4.04
○Thread 293464			3.28	4.39	74.77	0.91	-0.00	4.04
○Thread 293465			3.27	4.39	74.43	0.91	-0.00	4.05
○Thread 293466			3.14	4.39	71.47	0.91	-0.00	4.05
○Thread 293467			3.13	4.39	71.36	0.91	-0.00	4.05
○Thread 293468			3.21	4.39	73.18	0.91	-0.00	4.03
○Thread 293469			3.30	4.39	75.23	0.91	-0.00	4.04
○Thread 293470			3.23	4.39	73.64	0.91	-0.00	4.05
○Thread 293471			3.28	4.39	74.67	0.91	-0.00	4.05
○Thread 293472			3.28	4.39	74.67	0.91	-0.00	4.05
○Thread 293473			3.44	4.39	78.43	0.91	-0.00	4.05
○Thread 293474			3.36	4.39	76.49	0.91	-0.00	4.06
○Thread 293475			3.38	4.39	76.95	0.91	-0.00	4.05
○Thread 293476			3.26	4.39	74.34	0.91	-0.00	4.05
○Thread 293477			3.25	4.39	74.00	0.91	-0.00	4.05
○Thread 293478			3.15	4.39	71.83	0.91	-0.00	4.05
○Thread 293479			3.21	4.39	73.09	0.91	-0.00	4.05
○Thread 293480			3.22	4.39	73.32	0.91	-0.00	4.05
○Thread 293481			3.14	4.39	71.61	0.91	-0.00	4.05

Report Configuration