OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	230.83
►Process 298885+		1	230.83
○Thread 298885			230.83	231.77	99.60	0.00	0.00	231.72

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	116.26
►Process 299154+		2	116.26
○Thread 299154			116.26	117.19	99.21	0.00	0.00	117.16
○Thread 299253			115.93	116.31	99.68	0.89	-0.00	115.99

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	58.97
►Process 299325+		4	58.97
○Thread 299325			58.97	59.87	98.50	0.00	0.00	59.83
○Thread 299424			58.61	58.98	99.38	0.89	-0.00	58.66
○Thread 299425			58.65	58.98	99.43	0.89	-0.00	58.66
○Thread 299426			58.64	58.98	99.42	0.89	-0.00	58.66

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	30.25
►Process 299499+		8	30.25
○Thread 299499			30.25	31.24	96.84	0.00	0.00	31.20
○Thread 299598			29.94	30.35	98.67	0.89	-0.00	30.03
○Thread 299599			29.97	30.35	98.75	0.89	-0.00	30.03
○Thread 299600			29.94	30.35	98.65	0.89	-0.00	30.03
○Thread 299601			29.94	30.35	98.67	0.89	0.00	30.02
○Thread 299602			29.96	30.35	98.72	0.89	0.00	30.03
○Thread 299603			29.96	30.35	98.70	0.89	0.00	30.03
○Thread 299604			29.99	30.35	98.82	0.89	-0.00	30.03

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	16.12
►Process 299674+		16	16.12
○Thread 299674			16.12	17.16	93.91	0.00	0.00	17.11
○Thread 299773			15.77	16.27	96.90	0.89	-0.00	15.96
○Thread 299774			15.84	16.27	97.34	0.89	-0.00	15.96
○Thread 299775			15.81	16.27	97.15	0.89	-0.00	15.96
○Thread 299776			15.84	16.27	97.34	0.89	-0.00	15.96
○Thread 299777			15.83	16.27	97.25	0.89	-0.00	15.96
○Thread 299778			15.77	16.27	96.91	0.89	-0.00	15.96
○Thread 299779			15.78	16.27	97.00	0.89	-0.00	15.96
○Thread 299780			15.83	16.27	97.28	0.89	-0.00	15.96
○Thread 299781			15.85	16.27	97.43	0.89	-0.00	15.95
○Thread 299782			15.81	16.27	97.13	0.89	-0.00	15.95
○Thread 299783			15.85	16.27	97.43	0.89	-0.00	15.96
○Thread 299784			15.77	16.27	96.91	0.89	0.00	15.96
○Thread 299785			15.86	16.27	97.50	0.89	0.00	15.96
○Thread 299786			15.79	16.27	97.04	0.89	0.00	15.96
○Thread 299787			15.75	16.27	96.82	0.89	-0.00	15.96

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	12.46
►Process 299807+		24	12.46
○Thread 299807			12.46	13.59	91.70	0.00	0.00	13.55
○Thread 299906			12.17	12.71	95.75	0.88	-0.00	12.39
○Thread 299907			12.09	12.71	95.12	0.88	-0.00	12.39
○Thread 299908			12.16	12.71	95.71	0.88	-0.00	12.38
○Thread 299909			12.15	12.71	95.63	0.88	-0.00	12.39
○Thread 299910			12.13	12.71	95.48	0.88	-0.00	12.39
○Thread 299911			12.18	12.71	95.83	0.88	-0.00	12.39
○Thread 299912			12.12	12.71	95.40	0.88	-0.00	12.38
○Thread 299913			12.08	12.71	95.05	0.88	-0.00	12.39
○Thread 299914			12.18	12.71	95.84	0.88	-0.00	12.39
○Thread 299915			12.10	12.71	95.17	0.88	-0.00	12.38
○Thread 299916			12.22	12.71	96.15	0.88	-0.00	12.39
○Thread 299917			12.22	12.71	96.19	0.88	-0.00	12.38
○Thread 299918			12.13	12.71	95.41	0.88	-0.00	12.39
○Thread 299919			12.13	12.71	95.41	0.88	-0.00	12.38
○Thread 299920			12.30	12.71	96.79	0.88	-0.00	12.39
○Thread 299921			12.15	12.71	95.57	0.88	0.00	12.39
○Thread 299922			12.10	12.71	95.25	0.88	0.00	12.39
○Thread 299923			12.15	12.71	95.65	0.88	0.00	12.39
○Thread 299924			12.14	12.71	95.57	0.88	0.00	12.39
○Thread 299925			12.13	12.71	95.49	0.89	0.00	12.39
○Thread 299926			12.19	12.71	95.96	0.89	0.00	12.39
○Thread 299927			12.18	12.71	95.81	0.89	0.00	12.39
○Thread 299928			12.12	12.71	95.38	0.89	-0.00	12.39

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	10.09
►Process 299948+		32	10.09
○Thread 299948			10.09	11.32	89.06	0.00	0.00	11.26
○Thread 300047			9.78	10.44	93.70	0.89	-0.00	10.12
○Thread 300048			9.77	10.44	93.60	0.89	-0.00	10.12
○Thread 300049			9.77	10.44	93.60	0.89	-0.00	10.12
○Thread 300050			9.75	10.44	93.41	0.89	-0.00	10.12
○Thread 300051			9.83	10.44	94.18	0.89	-0.00	10.12
○Thread 300052			9.82	10.44	94.13	0.89	-0.00	10.12
○Thread 300053			9.81	10.44	93.99	0.89	-0.00	10.12
○Thread 300054			9.75	10.44	93.42	0.89	-0.00	10.12
○Thread 300055			9.74	10.44	93.32	0.89	-0.00	10.12
○Thread 300056			9.77	10.44	93.61	0.89	-0.00	10.12
○Thread 300057			9.79	10.44	93.80	0.89	-0.00	10.12
○Thread 300058			9.77	10.44	93.66	0.89	-0.00	10.12
○Thread 300059			9.78	10.44	93.71	0.89	-0.00	10.12
○Thread 300060			9.85	10.44	94.38	0.89	-0.00	10.12
○Thread 300061			9.84	10.44	94.33	0.89	-0.00	10.12
○Thread 300062			9.76	10.44	93.52	0.89	-0.00	10.12
○Thread 300063			9.87	10.44	94.53	0.89	-0.00	10.12
○Thread 300064			9.80	10.44	93.91	0.89	-0.00	10.12
○Thread 300065			9.80	10.44	93.96	0.89	-0.00	10.12
○Thread 300066			9.86	10.44	94.53	0.89	-0.00	10.12
○Thread 300067			9.95	10.44	95.35	0.89	-0.00	10.12
○Thread 300068			9.92	10.44	95.06	0.89	-0.00	10.12
○Thread 300069			9.82	10.44	94.15	0.89	-0.00	10.12
○Thread 300070			9.83	10.44	94.15	0.89	-0.00	10.12
○Thread 300071			9.84	10.44	94.30	0.89	-0.00	10.12
○Thread 300072			9.78	10.44	93.72	0.89	-0.00	10.12
○Thread 300073			9.83	10.43	94.25	0.89	-0.00	10.12
○Thread 300074			9.76	10.43	93.49	0.89	-0.00	10.12
○Thread 300075			9.78	10.43	93.73	0.89	-0.00	10.12
○Thread 300076			9.81	10.43	93.97	0.89	0.00	10.12
○Thread 300077			9.85	10.43	94.40	0.89	-0.00	10.12

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	8.87
►Process 300146+		40	8.87
○Thread 300146			8.87	10.15	87.40	0.00	0.00	10.10
○Thread 300245			8.60	9.26	92.89	0.90	-0.00	8.94
○Thread 300246			8.54	9.26	92.24	0.90	-0.00	8.94
○Thread 300247			8.53	9.26	92.08	0.90	-0.00	8.94
○Thread 300248			8.58	9.26	92.68	0.90	-0.00	8.93
○Thread 300249			8.55	9.26	92.30	0.90	-0.00	8.93
○Thread 300250			8.51	9.26	91.98	0.90	-0.00	8.94
○Thread 300251			8.59	9.26	92.79	0.90	-0.00	8.93
○Thread 300252			8.58	9.26	92.74	0.90	-0.00	8.94
○Thread 300253			8.53	9.26	92.20	0.90	-0.00	8.94
○Thread 300254			8.55	9.26	92.31	0.90	-0.00	8.94
○Thread 300255			8.54	9.26	92.25	0.90	-0.00	8.94
○Thread 300256			8.64	9.26	93.34	0.90	-0.00	8.92
○Thread 300257			8.57	9.26	92.58	0.90	-0.00	8.93
○Thread 300258			8.54	9.26	92.20	0.90	-0.00	8.94
○Thread 300259			8.51	9.26	91.88	0.90	-0.00	8.94
○Thread 300260			8.66	9.26	93.56	0.90	-0.00	8.94
○Thread 300261			8.57	9.26	92.64	0.90	-0.00	8.94
○Thread 300262			8.58	9.26	92.64	0.90	-0.00	8.93
○Thread 300263			8.55	9.26	92.43	0.90	-0.00	8.94
○Thread 300264			8.56	9.26	92.43	0.90	-0.00	8.94
○Thread 300265			8.53	9.26	92.10	0.90	-0.00	8.93
○Thread 300266			8.61	9.26	93.02	0.90	-0.00	8.94
○Thread 300267			8.54	9.26	92.21	0.90	-0.00	8.94
○Thread 300268			8.63	9.26	93.30	0.90	-0.00	8.94
○Thread 300269			8.53	9.26	92.16	0.90	-0.00	8.93
○Thread 300270			8.53	9.26	92.11	0.90	-0.00	8.94
○Thread 300271			8.52	9.26	92.11	0.90	-0.00	8.93
○Thread 300272			8.66	9.26	93.62	0.90	-0.00	8.93
○Thread 300273			8.60	9.25	92.87	0.90	-0.00	8.93
○Thread 300274			8.49	9.25	91.79	0.90	-0.00	8.94
○Thread 300275			8.50	9.25	91.84	0.90	-0.00	8.94
○Thread 300276			8.53	9.25	92.22	0.90	-0.00	8.93
○Thread 300277			8.49	9.25	91.74	0.90	-0.00	8.93
○Thread 300278			8.58	9.25	92.71	0.90	-0.00	8.94
○Thread 300279			8.55	9.25	92.39	0.90	-0.00	8.94
○Thread 300280			8.56	9.25	92.50	0.90	-0.00	8.93
○Thread 300281			8.58	9.25	92.72	0.90	-0.00	8.93
○Thread 300282			8.56	9.25	92.50	0.90	-0.00	8.94
○Thread 300283			8.47	9.25	91.53	0.90	-0.00	8.93

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	7.89
►Process 300303+		48	7.89
○Thread 300303			7.89	9.33	84.55	0.00	0.00	9.28
○Thread 300402			7.57	8.43	89.80	0.90	-0.00	8.10
○Thread 300403			7.64	8.43	90.70	0.90	-0.00	8.10
○Thread 300404			7.55	8.43	89.57	0.90	-0.00	8.11
○Thread 300405			7.54	8.43	89.45	0.90	-0.00	8.09
○Thread 300406			7.62	8.43	90.40	0.90	-0.00	8.11
○Thread 300407			7.66	8.43	90.88	0.90	-0.00	8.10
○Thread 300408			7.68	8.43	91.12	0.90	-0.00	8.11
○Thread 300409			7.68	8.43	91.12	0.90	-0.00	8.09
○Thread 300410			7.55	8.43	89.58	0.90	-0.00	8.11
○Thread 300411			7.59	8.43	90.05	0.90	-0.00	8.09
○Thread 300412			7.54	8.43	89.52	0.90	-0.00	8.09
○Thread 300413			7.58	8.43	89.88	0.90	-0.00	8.10
○Thread 300414			7.59	8.43	90.00	0.90	-0.00	8.11
○Thread 300415			7.68	8.43	91.18	0.90	-0.00	8.10
○Thread 300416			7.59	8.43	90.00	0.90	-0.00	8.10
○Thread 300417			7.60	8.43	90.12	0.90	-0.00	8.09
○Thread 300418			7.60	8.43	90.18	0.90	-0.00	8.11
○Thread 300419			7.65	8.43	90.77	0.90	-0.00	8.10
○Thread 300420			7.62	8.43	90.36	0.90	-0.00	8.10
○Thread 300421			7.68	8.43	91.13	0.90	-0.00	8.10
○Thread 300422			7.64	8.43	90.66	0.91	-0.00	8.10
○Thread 300423			7.67	8.43	91.02	0.91	-0.00	8.11
○Thread 300424			7.64	8.43	90.72	0.91	-0.00	8.11
○Thread 300425			7.57	8.43	89.77	0.91	-0.00	8.10
○Thread 300426			7.57	8.43	89.89	0.91	-0.00	8.10
○Thread 300427			7.65	8.43	90.78	0.91	-0.00	8.10
○Thread 300428			7.65	8.43	90.79	0.91	-0.00	8.10
○Thread 300429			7.76	8.43	92.09	0.91	-0.00	8.10
○Thread 300430			7.57	8.43	89.84	0.91	-0.00	8.10
○Thread 300431			7.58	8.43	89.90	0.91	-0.00	8.09
○Thread 300432			7.60	8.43	90.14	0.91	-0.00	8.10
○Thread 300433			7.64	8.43	90.67	0.91	-0.00	8.11
○Thread 300434			7.69	8.43	91.21	0.91	-0.00	8.10
○Thread 300435			7.62	8.43	90.44	0.91	-0.00	8.10
○Thread 300436			7.54	8.43	89.49	0.91	-0.00	8.11
○Thread 300437			7.52	8.43	89.19	0.91	-0.00	8.10
○Thread 300438			7.53	8.43	89.31	0.91	-0.00	8.09
○Thread 300439			7.63	8.43	90.62	0.91	-0.00	8.10
○Thread 300440			7.67	8.43	91.10	0.91	-0.00	8.10
○Thread 300441			7.54	8.43	89.50	0.91	-0.00	8.11
○Thread 300442			7.53	8.42	89.38	0.91	-0.00	8.10
○Thread 300443			7.52	8.42	89.20	0.91	-0.00	8.10
○Thread 300444			7.59	8.42	90.15	0.91	-0.00	8.11
○Thread 300445			7.62	8.42	90.51	0.91	-0.00	8.10
○Thread 300446			7.59	8.42	90.09	0.91	-0.00	8.10
○Thread 300447			7.53	8.42	89.38	0.91	-0.00	8.10
○Thread 300448			7.62	8.42	90.45	0.91	-0.00	8.11

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	7.44
►Process 300469+		56	7.44
○Thread 300469			7.44	8.82	84.34	0.00	0.00	8.78
○Thread 300568			7.06	7.92	89.18	0.90	-0.00	7.58
○Thread 300569			7.10	7.92	89.63	0.90	-0.00	7.57
○Thread 300570			7.11	7.92	89.75	0.90	-0.00	7.57
○Thread 300571			7.20	7.92	90.95	0.90	-0.00	7.58
○Thread 300572			7.29	7.92	91.97	0.90	-0.00	7.58
○Thread 300573			7.03	7.92	88.81	0.90	-0.00	7.57
○Thread 300574			7.12	7.92	89.89	0.90	-0.00	7.57
○Thread 300575			7.13	7.92	90.08	0.90	-0.00	7.59
○Thread 300576			7.17	7.92	90.52	0.90	-0.00	7.57
○Thread 300577			7.31	7.92	92.23	0.90	-0.00	7.57
○Thread 300578			7.36	7.92	92.86	0.90	-0.00	7.57
○Thread 300579			6.98	7.92	88.12	0.90	-0.00	7.58
○Thread 300580			6.99	7.92	88.32	0.90	-0.00	7.58
○Thread 300581			7.03	7.92	88.76	0.90	-0.00	7.58
○Thread 300582			7.09	7.92	89.58	0.90	-0.00	7.57
○Thread 300583			7.22	7.92	91.10	0.90	-0.00	7.57
○Thread 300584			7.32	7.92	92.49	0.90	-0.00	7.57
○Thread 300585			6.99	7.92	88.26	0.90	-0.00	7.57
○Thread 300586			7.00	7.92	88.39	0.90	-0.00	7.58
○Thread 300587			6.98	7.92	88.20	0.90	-0.00	7.57
○Thread 300588			7.17	7.92	90.53	0.90	-0.00	7.57
○Thread 300589			7.20	7.92	90.98	0.90	-0.00	7.57
○Thread 300590			7.25	7.92	91.61	0.90	-0.00	7.57
○Thread 300591			7.07	7.92	89.28	0.90	-0.00	7.57
○Thread 300592			7.08	7.92	89.40	0.90	-0.00	7.57
○Thread 300593			7.09	7.92	89.59	0.90	-0.00	7.57
○Thread 300594			7.11	7.92	89.85	0.90	-0.00	7.57
○Thread 300595			7.11	7.92	89.79	0.90	-0.00	7.57
○Thread 300596			7.30	7.92	92.25	0.90	-0.00	7.59
○Thread 300597			6.95	7.92	87.77	0.90	-0.00	7.57
○Thread 300598			7.04	7.92	88.97	0.90	-0.00	7.57
○Thread 300599			7.09	7.92	89.60	0.90	-0.00	7.57
○Thread 300600			7.04	7.92	88.91	0.90	-0.00	7.57
○Thread 300601			7.08	7.92	89.41	0.90	-0.00	7.57
○Thread 300602			7.11	7.92	89.80	0.90	-0.00	7.57
○Thread 300603			7.07	7.92	89.23	0.90	-0.00	7.57
○Thread 300604			6.94	7.92	87.65	0.90	-0.00	7.57
○Thread 300605			7.01	7.92	88.60	0.90	-0.00	7.58
○Thread 300606			6.99	7.92	88.28	0.90	-0.00	7.58
○Thread 300607			6.99	7.92	88.29	0.90	-0.00	7.57
○Thread 300608			7.02	7.92	88.67	0.90	-0.00	7.58
○Thread 300609			7.12	7.92	89.99	0.90	-0.00	7.57
○Thread 300610			7.07	7.92	89.24	0.90	-0.00	7.59
○Thread 300611			6.94	7.92	87.66	0.90	-0.00	7.58
○Thread 300612			7.04	7.92	88.92	0.90	-0.00	7.57
○Thread 300613			7.02	7.92	88.74	0.90	-0.00	7.57
○Thread 300614			7.10	7.92	89.75	0.90	-0.00	7.58
○Thread 300615			7.08	7.92	89.43	0.90	-0.00	7.57
○Thread 300616			7.09	7.92	89.56	0.90	-0.00	7.57
○Thread 300617			7.02	7.92	88.68	0.90	-0.00	7.57
○Thread 300618			7.07	7.92	89.25	0.90	-0.00	7.57
○Thread 300619			7.08	7.92	89.44	0.90	-0.00	7.57
○Thread 300620			7.07	7.92	89.38	0.91	-0.00	7.57
○Thread 300621			7.14	7.92	90.20	0.91	-0.00	7.57
○Thread 300622			7.01	7.92	88.56	0.91	-0.00	7.58

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	6.74
►Process 300690+		64	6.74
○Thread 300690			6.74	8.28	81.39	0.00	0.00	8.23
○Thread 300789			6.48	7.39	87.73	0.89	-0.00	7.04
○Thread 300790			6.59	7.39	89.29	0.89	-0.00	7.03
○Thread 300791			6.46	7.39	87.46	0.89	-0.00	7.05
○Thread 300792			6.37	7.39	86.18	0.89	-0.00	7.04
○Thread 300793			6.43	7.39	87.06	0.89	-0.00	7.05
○Thread 300794			6.49	7.39	87.87	0.89	-0.00	7.04
○Thread 300795			6.59	7.39	89.23	0.89	-0.00	7.05
○Thread 300796			6.59	7.39	89.23	0.89	-0.00	7.04
○Thread 300797			6.50	7.39	88.08	0.89	-0.00	7.04
○Thread 300798			6.37	7.39	86.32	0.89	-0.00	7.04
○Thread 300799			6.44	7.39	87.27	0.89	-0.00	7.05
○Thread 300800			6.34	7.39	85.92	0.89	-0.00	7.04
○Thread 300801			6.47	7.39	87.61	0.89	-0.00	7.04
○Thread 300802			6.46	7.38	87.48	0.89	-0.00	7.04
○Thread 300803			6.42	7.38	86.93	0.89	-0.00	7.04
○Thread 300804			6.41	7.38	86.80	0.89	-0.00	7.05
○Thread 300805			6.45	7.38	87.28	0.89	-0.00	7.03
○Thread 300806			6.36	7.38	86.13	0.89	-0.00	7.03
○Thread 300807			6.43	7.38	87.08	0.89	-0.00	7.04
○Thread 300808			6.48	7.38	87.82	0.89	-0.00	7.06
○Thread 300809			6.44	7.38	87.21	0.89	-0.00	7.05
○Thread 300810			6.34	7.38	85.86	0.89	-0.00	7.05
○Thread 300811			6.39	7.38	86.54	0.89	-0.00	7.04
○Thread 300812			6.46	7.38	87.42	0.89	-0.00	7.05
○Thread 300813			6.33	7.38	85.73	0.89	-0.00	7.04
○Thread 300814			6.41	7.38	86.81	0.89	-0.00	7.04
○Thread 300815			6.36	7.38	86.21	0.89	-0.00	7.04
○Thread 300816			6.44	7.38	87.29	0.89	-0.00	7.04
○Thread 300817			6.45	7.38	87.36	0.89	-0.00	7.04
○Thread 300818			6.38	7.38	86.48	0.89	-0.00	7.02
○Thread 300819			6.38	7.38	86.48	0.89	-0.00	7.04
○Thread 300820			6.44	7.38	87.23	0.89	-0.00	7.04
○Thread 300821			6.38	7.38	86.48	0.89	-0.00	7.04
○Thread 300822			6.40	7.38	86.69	0.89	-0.00	7.04
○Thread 300823			6.32	7.38	85.61	0.89	-0.00	7.05
○Thread 300824			6.31	7.38	85.47	0.89	-0.00	7.04
○Thread 300825			6.43	7.38	87.03	0.89	-0.00	7.03
○Thread 300826			6.45	7.38	87.37	0.89	-0.00	7.04
○Thread 300827			6.44	7.38	87.24	0.89	-0.00	7.04
○Thread 300828			6.35	7.38	85.95	0.89	-0.00	7.04
○Thread 300829			6.34	7.38	85.89	0.89	-0.00	7.04
○Thread 300830			6.32	7.38	85.62	0.89	-0.00	7.04
○Thread 300831			6.38	7.38	86.43	0.89	-0.00	7.04
○Thread 300832			6.40	7.38	86.70	0.89	-0.00	7.04
○Thread 300833			6.25	7.38	84.60	0.89	-0.00	7.04
○Thread 300834			6.19	7.38	83.86	0.89	-0.00	7.04
○Thread 300835			6.35	7.38	86.03	0.89	-0.00	7.04
○Thread 300836			6.39	7.38	86.57	0.89	-0.00	7.03
○Thread 300837			6.39	7.38	86.64	0.89	-0.00	7.05
○Thread 300838			6.40	7.38	86.64	0.89	-0.00	7.03
○Thread 300839			6.32	7.38	85.70	0.89	-0.00	7.03
○Thread 300840			6.54	7.38	88.61	0.89	-0.00	7.04
○Thread 300841			6.40	7.38	86.65	0.89	-0.00	7.04
○Thread 300842			6.48	7.38	87.87	0.89	-0.00	7.04
○Thread 300843			6.45	7.38	87.46	0.90	-0.00	7.04
○Thread 300844			6.45	7.38	87.40	0.90	-0.00	7.04
○Thread 300845			6.42	7.38	86.99	0.90	-0.00	7.05
○Thread 300846			6.32	7.38	85.71	0.90	-0.00	7.04
○Thread 300847			6.41	7.38	86.86	0.90	-0.00	7.04
○Thread 300848			6.28	7.38	85.10	0.90	-0.00	7.04
○Thread 300849			6.43	7.38	87.20	0.90	-0.00	7.05
○Thread 300850			6.34	7.38	85.91	0.90	-0.00	7.04
○Thread 300851			6.28	7.38	85.17	0.90	-0.00	7.04

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	7.28
►Process 300872+		72	7.28
○Thread 300872			7.28	9.00	80.84	0.00	0.00	8.96
○Thread 300971			6.96	8.10	85.91	0.90	-0.00	7.77
○Thread 300972			6.93	8.10	85.54	0.90	-0.00	7.76
○Thread 300973			7.02	8.10	86.65	0.90	-0.00	7.77
○Thread 300974			7.05	8.10	87.02	0.90	-0.00	7.77
○Thread 300975			6.93	8.10	85.61	0.90	-0.00	7.77
○Thread 300976			6.93	8.10	85.48	0.90	-0.00	7.77
○Thread 300977			7.10	8.10	87.65	0.90	-0.00	7.77
○Thread 300978			6.94	8.10	85.67	0.90	-0.00	7.75
○Thread 300979			7.00	8.10	86.41	0.90	-0.00	7.77
○Thread 300980			7.05	8.10	87.03	0.90	-0.00	7.77
○Thread 300981			7.02	8.10	86.66	0.90	-0.00	7.77
○Thread 300982			6.96	8.10	85.86	0.90	-0.00	7.77
○Thread 300983			6.98	8.10	86.23	0.90	-0.00	7.77
○Thread 300984			6.91	8.10	85.37	0.90	-0.00	7.77
○Thread 300985			6.97	8.10	86.11	0.90	-0.00	7.77
○Thread 300986			7.06	8.10	87.10	0.90	-0.00	7.77
○Thread 300987			6.97	8.10	86.11	0.90	-0.00	7.77
○Thread 300988			6.95	8.10	85.81	0.90	-0.00	7.77
○Thread 300989			7.08	8.10	87.35	0.90	-0.00	7.77
○Thread 300990			6.97	8.10	86.12	0.90	-0.00	7.77
○Thread 300991			7.01	8.10	86.49	0.90	-0.00	7.77
○Thread 300992			7.00	8.10	86.37	0.90	-0.00	7.77
○Thread 300993			6.99	8.10	86.37	0.90	-0.00	7.77
○Thread 300994			6.87	8.10	84.83	0.90	-0.00	7.77
○Thread 300995			6.94	8.10	85.69	0.90	-0.00	7.76
○Thread 300996			7.04	8.10	86.93	0.90	-0.00	7.77
○Thread 300997			6.94	8.10	85.63	0.90	-0.00	7.77
○Thread 300998			6.92	8.10	85.45	0.90	-0.00	7.77
○Thread 300999			6.94	8.10	85.70	0.90	-0.00	7.77
○Thread 301000			7.04	8.10	86.93	0.90	-0.00	7.77
○Thread 301001			7.02	8.10	86.75	0.90	-0.00	7.77
○Thread 301002			7.00	8.10	86.38	0.90	-0.00	7.77
○Thread 301003			7.06	8.10	87.18	0.90	-0.00	7.77
○Thread 301004			7.01	8.10	86.51	0.90	-0.00	7.77
○Thread 301005			7.02	8.10	86.75	0.90	-0.00	7.77
○Thread 301006			6.91	8.10	85.27	0.90	-0.00	7.77
○Thread 301007			7.11	8.10	87.74	0.90	-0.00	7.77
○Thread 301008			7.08	8.10	87.44	0.90	-0.00	7.77
○Thread 301009			6.93	8.10	85.52	0.90	-0.00	7.77
○Thread 301010			7.00	8.10	86.45	0.90	-0.00	7.77
○Thread 301011			6.97	8.10	86.08	0.90	-0.00	7.77
○Thread 301012			7.03	8.10	86.82	0.90	-0.00	7.77
○Thread 301013			7.04	8.10	86.89	0.90	-0.00	7.77
○Thread 301014			6.94	8.10	85.72	0.90	-0.00	7.77
○Thread 301015			6.91	8.10	85.41	0.90	-0.00	7.77
○Thread 301016			6.98	8.10	86.27	0.90	-0.00	7.77
○Thread 301017			7.08	8.10	87.39	0.90	-0.00	7.77
○Thread 301018			7.05	8.10	87.14	0.90	-0.00	7.77
○Thread 301019			6.94	8.10	85.72	0.90	-0.00	7.77
○Thread 301020			7.01	8.10	86.65	0.90	-0.00	7.77
○Thread 301021			6.99	8.10	86.34	0.90	-0.00	7.77
○Thread 301022			7.05	8.10	87.08	0.90	-0.00	7.77
○Thread 301023			6.81	8.10	84.18	0.90	-0.00	7.75
○Thread 301024			6.97	8.10	86.10	0.90	-0.00	7.77
○Thread 301025			6.86	8.10	84.74	0.90	-0.00	7.75
○Thread 301026			6.96	8.10	85.98	0.90	-0.00	7.77
○Thread 301027			6.93	8.10	85.55	0.90	-0.00	7.77
○Thread 301028			7.04	8.09	86.97	0.90	-0.00	7.76
○Thread 301029			6.99	8.09	86.29	0.90	-0.00	7.77
○Thread 301030			7.06	8.09	87.22	0.90	-0.00	7.77
○Thread 301031			6.92	8.09	85.49	0.90	-0.00	7.77
○Thread 301032			6.96	8.09	85.98	0.90	-0.00	7.75
○Thread 301033			6.98	8.09	86.17	0.91	-0.00	7.77
○Thread 301034			6.95	8.09	85.86	0.91	-0.00	7.77
○Thread 301035			6.87	8.09	84.88	0.91	-0.00	7.77
○Thread 301036			6.98	8.09	86.17	0.91	-0.00	7.77
○Thread 301037			7.01	8.09	86.55	0.91	-0.00	7.77
○Thread 301038			7.03	8.09	86.92	0.91	-0.00	7.76
○Thread 301039			7.01	8.09	86.61	0.91	-0.00	7.77
○Thread 301040			7.01	8.09	86.67	0.91	-0.00	7.76
○Thread 301041			6.88	8.09	85.07	0.91	-0.00	7.77

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	6.14
►Process 301061+		80	6.14
○Thread 301061			6.14	7.92	77.63	0.00	0.00	7.88
○Thread 301160			5.82	7.03	82.83	0.89	-0.00	6.69
○Thread 301161			5.69	7.03	80.98	0.89	-0.00	6.68
○Thread 301162			5.89	7.03	83.83	0.89	-0.00	6.69
○Thread 301163			5.95	7.03	84.68	0.89	-0.00	6.68
○Thread 301164			5.85	7.03	83.33	0.89	-0.00	6.68
○Thread 301165			5.81	7.03	82.62	0.89	-0.00	6.68
○Thread 301166			5.79	7.03	82.48	0.89	-0.00	6.68
○Thread 301167			5.79	7.03	82.48	0.89	-0.00	6.67
○Thread 301168			5.84	7.03	83.12	0.89	-0.00	6.68
○Thread 301169			5.95	7.03	84.69	0.89	-0.00	6.67
○Thread 301170			5.86	7.03	83.34	0.89	-0.00	6.68
○Thread 301171			5.77	7.03	82.20	0.89	-0.00	6.68
○Thread 301172			5.79	7.03	82.42	0.89	-0.00	6.68
○Thread 301173			5.92	7.03	84.34	0.89	-0.00	6.68
○Thread 301174			5.96	7.02	84.91	0.89	-0.00	6.68
○Thread 301175			5.95	7.02	84.70	0.89	-0.00	6.67
○Thread 301176			5.82	7.02	82.85	0.89	-0.00	6.67
○Thread 301177			5.79	7.02	82.50	0.89	-0.00	6.68
○Thread 301178			5.74	7.02	81.64	0.89	-0.00	6.68
○Thread 301179			5.85	7.02	83.35	0.89	-0.00	6.68
○Thread 301180			5.77	7.02	82.14	0.89	-0.00	6.67
○Thread 301181			5.70	7.02	81.08	0.89	-0.00	6.69
○Thread 301182			5.69	7.02	81.08	0.89	-0.00	6.67
○Thread 301183			5.81	7.02	82.72	0.89	-0.00	6.68
○Thread 301184			5.66	7.02	80.65	0.89	-0.00	6.68
○Thread 301185			5.70	7.02	81.15	0.89	-0.00	6.68
○Thread 301186			5.82	7.02	82.94	0.89	-0.00	6.68
○Thread 301187			5.88	7.02	83.72	0.89	-0.00	6.68
○Thread 301188			5.83	7.02	83.01	0.89	-0.00	6.68
○Thread 301189			5.84	7.02	83.15	0.89	-0.00	6.68
○Thread 301190			5.78	7.02	82.30	0.89	-0.00	6.67
○Thread 301191			5.75	7.02	81.80	0.89	-0.00	6.68
○Thread 301192			5.75	7.02	81.88	0.89	-0.00	6.68
○Thread 301193			5.79	7.02	82.52	0.89	-0.00	6.68
○Thread 301194			5.82	7.02	82.87	0.89	-0.00	6.68
○Thread 301195			5.73	7.02	81.59	0.89	-0.00	6.67
○Thread 301196			5.78	7.02	82.38	0.89	-0.00	6.68
○Thread 301197			5.82	7.02	82.95	0.89	-0.00	6.68
○Thread 301198			5.85	7.02	83.31	0.89	-0.00	6.68
○Thread 301199			5.80	7.02	82.53	0.89	-0.00	6.68
○Thread 301200			5.76	7.02	82.10	0.89	-0.00	6.69
○Thread 301201			5.82	7.02	82.88	0.89	-0.00	6.68
○Thread 301202			5.84	7.02	83.24	0.89	-0.00	6.68
○Thread 301203			5.83	7.02	83.03	0.89	-0.00	6.68
○Thread 301204			5.74	7.02	81.75	0.89	-0.00	6.68
○Thread 301205			5.79	7.02	82.46	0.89	-0.00	6.68
○Thread 301206			5.90	7.02	84.10	0.89	-0.00	6.67
○Thread 301207			5.74	7.02	81.68	0.89	-0.00	6.68
○Thread 301208			5.72	7.02	81.54	0.89	-0.00	6.68
○Thread 301209			5.86	7.02	83.54	0.89	-0.00	6.68
○Thread 301210			5.79	7.02	82.47	0.89	-0.00	6.68
○Thread 301211			5.77	7.02	82.26	0.89	-0.00	6.68
○Thread 301212			5.78	7.02	82.33	0.90	-0.00	6.68
○Thread 301213			5.85	7.02	83.26	0.90	-0.00	6.68
○Thread 301214			5.79	7.02	82.47	0.90	-0.00	6.68
○Thread 301215			5.82	7.02	82.90	0.90	-0.00	6.67
○Thread 301216			5.74	7.02	81.76	0.90	-0.00	6.67
○Thread 301217			5.77	7.02	82.19	0.90	-0.00	6.67
○Thread 301218			5.59	7.02	79.70	0.90	-0.00	6.67
○Thread 301219			5.74	7.02	81.84	0.90	-0.00	6.68
○Thread 301220			5.77	7.02	82.27	0.90	-0.00	6.67
○Thread 301221			5.78	7.02	82.41	0.90	-0.00	6.68
○Thread 301222			5.77	7.02	82.27	0.90	-0.00	6.67
○Thread 301223			5.73	7.02	81.70	0.90	-0.00	6.68
○Thread 301224			5.75	7.02	81.99	0.90	-0.00	6.68
○Thread 301225			5.90	7.02	84.13	0.90	-0.00	6.68
○Thread 301226			5.79	7.02	82.49	0.90	-0.00	6.68
○Thread 301227			5.83	7.02	83.06	0.90	-0.00	6.68
○Thread 301228			5.69	7.02	81.14	0.90	-0.00	6.67
○Thread 301229			5.87	7.02	83.56	0.90	-0.00	6.67
○Thread 301230			5.83	7.02	82.99	0.90	-0.00	6.68
○Thread 301231			5.79	7.02	82.50	0.90	-0.00	6.68
○Thread 301232			5.74	7.02	81.79	0.90	-0.00	6.67
○Thread 301233			5.83	7.02	83.07	0.90	-0.00	6.68
○Thread 301234			5.75	7.02	82.00	0.90	-0.00	6.68
○Thread 301235			5.79	7.02	82.50	0.90	-0.00	6.68
○Thread 301236			5.80	7.02	82.72	0.90	-0.00	6.69
○Thread 301237			5.83	7.02	83.07	0.90	-0.00	6.68
○Thread 301238			4.93	7.02	70.25	0.90	-0.00	6.68

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	6.73
►Process 301259+		88	6.73
○Thread 301259			6.73	8.54	78.75	0.00	0.00	8.50
○Thread 301358			6.33	7.64	82.88	0.90	-0.00	7.29
○Thread 301359			6.31	7.64	82.68	0.90	-0.00	7.29
○Thread 301360			6.27	7.64	82.16	0.90	-0.00	7.29
○Thread 301361			6.29	7.64	82.36	0.90	-0.00	7.28
○Thread 301362			6.33	7.64	82.89	0.90	-0.00	7.28
○Thread 301363			6.40	7.64	83.80	0.90	-0.00	7.29
○Thread 301364			6.48	7.64	84.92	0.90	-0.00	7.29
○Thread 301365			6.41	7.64	84.00	0.90	-0.00	7.29
○Thread 301366			6.46	7.64	84.59	0.90	-0.00	7.30
○Thread 301367			6.42	7.64	84.14	0.90	-0.00	7.29
○Thread 301368			6.38	7.64	83.61	0.90	-0.00	7.29
○Thread 301369			6.30	7.64	82.44	0.90	-0.00	7.29
○Thread 301370			6.35	7.64	83.16	0.90	-0.00	7.29
○Thread 301371			6.33	7.64	82.90	0.90	-0.00	7.29
○Thread 301372			6.40	7.64	83.81	0.90	-0.00	7.29
○Thread 301373			6.42	7.64	84.08	0.90	-0.00	7.30
○Thread 301374			6.32	7.64	82.70	0.90	-0.00	7.29
○Thread 301375			6.46	7.64	84.67	0.90	-0.00	7.29
○Thread 301376			6.38	7.64	83.62	0.90	-0.00	7.28
○Thread 301377			6.41	7.64	83.95	0.90	-0.00	7.28
○Thread 301378			6.36	7.64	83.36	0.90	-0.00	7.29
○Thread 301379			6.42	7.64	84.15	0.90	-0.00	7.29
○Thread 301380			6.35	7.64	83.17	0.90	-0.00	7.30
○Thread 301381			6.21	7.63	81.34	0.90	-0.00	7.29
○Thread 301382			6.38	7.63	83.56	0.91	-0.00	7.29
○Thread 301383			6.34	7.63	83.11	0.91	-0.00	7.29
○Thread 301384			6.50	7.63	85.14	0.91	-0.00	7.29
○Thread 301385			6.36	7.63	83.31	0.91	-0.00	7.28
○Thread 301386			6.40	7.63	83.83	0.91	-0.00	7.30
○Thread 301387			6.43	7.63	84.23	0.91	-0.00	7.29
○Thread 301388			6.39	7.63	83.70	0.91	-0.00	7.29
○Thread 301389			6.39	7.63	83.70	0.91	-0.00	7.29
○Thread 301390			6.27	7.63	82.13	0.91	-0.00	7.29
○Thread 301391			6.22	7.63	81.48	0.91	-0.00	7.29
○Thread 301392			6.36	7.63	83.38	0.91	-0.00	7.29
○Thread 301393			6.34	7.63	82.99	0.91	-0.00	7.29
○Thread 301394			6.33	7.63	82.99	0.91	-0.00	7.29
○Thread 301395			6.40	7.63	83.91	0.91	-0.00	7.30
○Thread 301396			6.38	7.63	83.65	0.91	-0.00	7.29
○Thread 301397			6.28	7.63	82.34	0.91	-0.00	7.29
○Thread 301398			6.29	7.63	82.34	0.91	-0.00	7.29
○Thread 301399			6.42	7.63	84.18	0.91	-0.00	7.29
○Thread 301400			6.32	7.63	82.87	0.91	-0.00	7.28
○Thread 301401			6.28	7.63	82.21	0.91	-0.00	7.29
○Thread 301402			6.28	7.63	82.35	0.91	-0.00	7.29
○Thread 301403			6.30	7.63	82.61	0.91	-0.00	7.29
○Thread 301404			6.35	7.63	83.27	0.91	-0.00	7.29
○Thread 301405			6.30	7.63	82.61	0.91	-0.00	7.29
○Thread 301406			6.30	7.63	82.55	0.91	-0.00	7.29
○Thread 301407			6.26	7.63	82.02	0.91	-0.00	7.29
○Thread 301408			6.32	7.63	82.81	0.91	-0.00	7.29
○Thread 301409			6.31	7.63	82.68	0.91	-0.00	7.29
○Thread 301410			6.35	7.63	83.21	0.91	-0.00	7.29
○Thread 301411			6.38	7.63	83.60	0.91	-0.00	7.29
○Thread 301412			6.23	7.63	81.70	0.91	-0.00	7.29
○Thread 301413			6.28	7.63	82.29	0.91	-0.00	7.29
○Thread 301414			6.20	7.63	81.25	0.91	-0.00	7.29
○Thread 301415			6.26	7.63	82.10	0.91	-0.00	7.29
○Thread 301416			6.30	7.63	82.56	0.91	-0.00	7.29
○Thread 301417			6.33	7.63	82.95	0.91	-0.00	7.29
○Thread 301418			6.40	7.63	83.81	0.91	-0.00	7.29
○Thread 301419			6.30	7.63	82.56	0.91	-0.00	7.28
○Thread 301420			6.08	7.63	79.62	0.91	-0.00	7.29
○Thread 301421			6.28	7.63	82.37	0.91	-0.00	7.29
○Thread 301422			6.28	7.63	82.30	0.91	-0.00	7.29
○Thread 301423			6.22	7.63	81.52	0.91	-0.00	7.29
○Thread 301424			6.31	7.63	82.63	0.91	-0.00	7.29
○Thread 301425			6.33	7.63	82.96	0.91	-0.00	7.29
○Thread 301426			6.30	7.63	82.57	0.91	-0.00	7.29
○Thread 301427			6.28	7.63	82.38	0.91	-0.00	7.29
○Thread 301428			6.41	7.63	84.02	0.91	-0.00	7.29
○Thread 301429			6.30	7.63	82.58	0.91	-0.00	7.29
○Thread 301430			6.19	7.63	81.13	0.91	-0.00	7.29
○Thread 301431			6.38	7.63	83.63	0.91	-0.00	7.29
○Thread 301432			6.36	7.63	83.30	0.91	-0.00	7.29
○Thread 301433			6.30	7.63	82.58	0.91	-0.00	7.29
○Thread 301434			6.24	7.63	81.86	0.91	-0.00	7.29
○Thread 301435			6.29	7.63	82.45	0.91	-0.00	7.29
○Thread 301436			6.20	7.63	81.34	0.91	-0.00	7.29
○Thread 301437			6.28	7.63	82.32	0.91	-0.00	7.29
○Thread 301438			6.27	7.63	82.26	0.91	-0.00	7.29
○Thread 301439			6.30	7.63	82.65	0.91	-0.00	7.29
○Thread 301440			6.34	7.63	83.18	0.91	-0.00	7.27
○Thread 301441			6.14	7.63	80.56	0.91	-0.00	7.28
○Thread 301442			6.30	7.63	82.66	0.91	-0.00	7.29
○Thread 301443			6.33	7.63	83.05	0.91	-0.00	7.29
○Thread 301444			5.38	7.63	70.60	0.91	-0.00	7.29

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	6.51
►Process 301515+		96	6.51
○Thread 301515			6.51	8.44	77.15	0.00	0.00	8.39
○Thread 301614			6.18	7.55	81.82	0.89	-0.00	7.20
○Thread 301615			6.13	7.55	81.22	0.89	-0.00	7.20
○Thread 301616			6.23	7.55	82.55	0.89	-0.00	7.21
○Thread 301617			6.08	7.55	80.57	0.89	-0.00	7.21
○Thread 301618			6.26	7.55	82.88	0.89	-0.00	7.21
○Thread 301619			6.21	7.55	82.22	0.89	-0.00	7.19
○Thread 301620			6.15	7.55	81.50	0.89	-0.00	7.20
○Thread 301621			6.42	7.55	85.01	0.89	-0.00	7.20
○Thread 301622			6.23	7.55	82.49	0.89	-0.00	7.20
○Thread 301623			6.26	7.55	82.96	0.89	-0.00	7.20
○Thread 301624			6.16	7.55	81.63	0.89	-0.00	7.20
○Thread 301625			6.24	7.55	82.69	0.89	-0.00	7.21
○Thread 301626			6.10	7.55	80.84	0.89	-0.00	7.21
○Thread 301627			6.18	7.55	81.84	0.89	-0.00	7.20
○Thread 301628			6.17	7.55	81.70	0.89	-0.00	7.20
○Thread 301629			6.21	7.55	82.24	0.89	-0.00	7.20
○Thread 301630			6.21	7.55	82.17	0.89	-0.00	7.20
○Thread 301631			6.18	7.55	81.91	0.89	-0.00	7.19
○Thread 301632			6.10	7.55	80.85	0.89	-0.00	7.20
○Thread 301633			6.22	7.55	82.37	0.89	-0.00	7.20
○Thread 301634			6.18	7.55	81.78	0.89	-0.00	7.21
○Thread 301635			6.21	7.55	82.31	0.89	-0.00	7.20
○Thread 301636			6.15	7.55	81.52	0.89	-0.00	7.20
○Thread 301637			6.14	7.55	81.25	0.89	-0.00	7.20
○Thread 301638			6.04	7.55	80.00	0.89	-0.00	7.20
○Thread 301639			6.21	7.55	82.31	0.89	-0.00	7.20
○Thread 301640			5.91	7.55	78.28	0.89	-0.00	7.19
○Thread 301641			6.23	7.55	82.58	0.89	-0.00	7.21
○Thread 301642			6.15	7.55	81.52	0.89	-0.00	7.20
○Thread 301643			6.21	7.55	82.25	0.89	-0.00	7.18
○Thread 301644			6.20	7.55	82.19	0.89	-0.00	7.21
○Thread 301645			6.24	7.55	82.65	0.89	-0.00	7.20
○Thread 301646			6.14	7.55	81.40	0.89	-0.00	7.21
○Thread 301647			6.22	7.55	82.46	0.90	-0.00	7.20
○Thread 301648			6.05	7.55	80.14	0.90	-0.00	7.20
○Thread 301649			6.20	7.55	82.13	0.90	-0.00	7.20
○Thread 301650			6.03	7.55	79.94	0.90	-0.00	7.21
○Thread 301651			6.21	7.55	82.26	0.90	-0.00	7.20
○Thread 301652			6.08	7.55	80.61	0.90	-0.00	7.20
○Thread 301653			6.26	7.55	82.93	0.90	-0.00	7.21
○Thread 301654			6.14	7.55	81.41	0.90	-0.00	7.20
○Thread 301655			6.19	7.55	82.00	0.90	-0.00	7.20
○Thread 301656			6.16	7.55	81.61	0.90	-0.00	7.21
○Thread 301657			6.18	7.55	81.87	0.90	-0.00	7.20
○Thread 301658			6.18	7.55	81.87	0.90	-0.00	7.20
○Thread 301659			6.22	7.55	82.47	0.90	-0.00	7.21
○Thread 301660			6.23	7.55	82.60	0.90	-0.00	7.20
○Thread 301661			6.25	7.55	82.80	0.90	-0.00	7.20
○Thread 301662			6.08	7.55	80.55	0.90	-0.00	7.20
○Thread 301663			6.15	7.55	81.48	0.90	-0.00	7.20
○Thread 301664			6.26	7.55	82.87	0.90	-0.00	7.21
○Thread 301665			6.24	7.55	82.68	0.90	-0.00	7.21
○Thread 301666			6.25	7.55	82.88	0.90	-0.00	7.20
○Thread 301667			6.28	7.55	83.21	0.90	-0.00	7.20
○Thread 301668			6.27	7.55	83.08	0.90	-0.00	7.20
○Thread 301669			6.25	7.55	82.81	0.90	-0.00	7.21
○Thread 301670			6.27	7.55	83.15	0.90	-0.00	7.20
○Thread 301671			6.19	7.55	82.09	0.90	-0.00	7.21
○Thread 301672			6.21	7.55	82.35	0.90	-0.00	7.20
○Thread 301673			6.23	7.55	82.55	0.90	-0.00	7.21
○Thread 301674			6.28	7.55	83.28	0.90	-0.00	7.18
○Thread 301675			6.15	7.55	81.50	0.90	-0.00	7.20
○Thread 301676			6.23	7.55	82.62	0.90	-0.00	7.21
○Thread 301677			6.35	7.55	84.22	0.90	-0.00	7.21
○Thread 301678			6.11	7.55	80.97	0.90	-0.00	7.20
○Thread 301679			6.10	7.55	80.91	0.90	-0.00	7.21
○Thread 301680			6.20	7.55	82.23	0.90	-0.00	7.21
○Thread 301681			6.28	7.55	83.29	0.90	-0.00	7.20
○Thread 301682			6.21	7.55	82.30	0.90	-0.00	7.20
○Thread 301683			6.16	7.55	81.71	0.90	-0.00	7.21
○Thread 301684			6.18	7.55	81.97	0.90	-0.00	7.20
○Thread 301685			6.18	7.55	81.91	0.90	-0.00	7.20
○Thread 301686			6.00	7.55	79.46	0.90	-0.00	7.21
○Thread 301687			6.24	7.54	82.70	0.90	-0.00	7.21
○Thread 301688			6.11	7.54	80.98	0.90	-0.00	7.20
○Thread 301689			6.06	7.54	80.39	0.90	-0.00	7.21
○Thread 301690			6.20	7.54	82.18	0.90	-0.00	7.21
○Thread 301691			6.12	7.54	81.19	0.90	-0.00	7.20
○Thread 301692			6.17	7.54	81.78	0.90	-0.00	7.21
○Thread 301693			6.27	7.54	83.18	0.90	-0.00	7.20
○Thread 301694			6.19	7.54	82.12	0.90	-0.00	7.21
○Thread 301695			6.34	7.54	83.97	0.90	-0.00	7.21
○Thread 301696			6.13	7.54	81.26	0.90	-0.00	7.20
○Thread 301697			6.08	7.54	80.60	0.90	-0.00	7.20
○Thread 301698			6.05	7.54	80.27	0.90	-0.00	7.20
○Thread 301699			6.05	7.54	80.20	0.90	-0.00	7.21
○Thread 301700			6.17	7.54	81.86	0.90	-0.00	7.21
○Thread 301701			6.12	7.54	81.06	0.90	-0.00	7.19
○Thread 301702			6.09	7.54	80.73	0.90	-0.00	7.21
○Thread 301703			6.20	7.54	82.19	0.90	-0.00	7.20
○Thread 301704			6.05	7.54	80.21	0.90	-0.00	7.21
○Thread 301705			6.21	7.54	82.33	0.90	-0.00	7.21
○Thread 301706			6.19	7.54	82.13	0.90	-0.00	7.19
○Thread 301707			6.18	7.54	81.93	0.90	-0.00	7.21
○Thread 301708			5.38	7.54	71.33	0.90	-0.00	7.20

Report Configuration