OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	175.08
►Process 3644011+		1	175.08
○Thread 3644011			175.08	176.10	99.42	0.00	0.00	176.06

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	86.80
►Process 3644231+		2	86.80
○Thread 3644231			86.80	90.47	95.95	0.00	0.00	90.44
○Thread 3644330			86.74	89.59	96.82	0.88	-0.00	89.25

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	43.89
►Process 3644845+		4	43.89
○Thread 3644845			43.72	47.82	91.43	0.00	0.00	47.79
○Thread 3644944			43.89	46.94	93.50	0.88	-0.00	46.60
○Thread 3644945			43.75	46.94	93.21	0.88	-0.00	46.60
○Thread 3644946			43.55	46.94	92.77	0.88	-0.00	46.60

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	24.44
►Process 3646343+		8	24.44
○Thread 3646343			23.50	29.04	80.94	0.00	0.00	29.00
○Thread 3646442			24.40	28.16	86.65	0.88	-0.00	27.82
○Thread 3646443			24.44	28.16	86.80	0.88	-0.00	27.82
○Thread 3646444			24.35	28.16	86.48	0.88	-0.00	27.82
○Thread 3646445			24.35	28.16	86.49	0.88	-0.00	27.82
○Thread 3646446			24.32	28.16	86.39	0.88	0.00	27.82
○Thread 3646447			24.15	28.16	85.78	0.88	0.00	27.82
○Thread 3646448			24.01	28.16	85.29	0.88	-0.00	27.81

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	15.10
►Process 3649563+		16	15.10
○Thread 3649563			14.57	21.10	69.06	0.00	0.00	21.06
○Thread 3649662			15.10	20.21	74.68	0.89	-0.00	19.87
○Thread 3649663			15.10	20.21	74.70	0.89	-0.00	19.87
○Thread 3649664			15.05	20.21	74.45	0.89	-0.00	19.87
○Thread 3649665			15.05	20.21	74.43	0.89	-0.00	19.87
○Thread 3649666			15.06	20.21	74.48	0.89	-0.00	19.87
○Thread 3649667			14.87	20.21	73.57	0.89	-0.00	19.87
○Thread 3649668			14.84	20.21	73.42	0.89	-0.00	19.87
○Thread 3649669			14.97	20.21	74.04	0.89	-0.00	19.87
○Thread 3649670			14.97	20.21	74.04	0.89	-0.00	19.87
○Thread 3649671			14.97	20.21	74.04	0.89	-0.00	19.87
○Thread 3649672			14.93	20.21	73.86	0.89	-0.00	19.87
○Thread 3649673			14.91	20.21	73.77	0.89	0.00	19.87
○Thread 3649674			14.92	20.21	73.82	0.89	0.00	19.87
○Thread 3649675			14.88	20.21	73.59	0.89	0.00	19.87
○Thread 3649676			14.77	20.21	73.05	0.89	-0.00	19.87

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	10.93
►Process 3656374+		24	10.93
○Thread 3656374			9.96	18.41	54.09	0.00	0.00	18.37
○Thread 3656473			10.93	17.53	62.33	0.89	-0.00	17.18
○Thread 3656474			10.92	17.53	62.27	0.89	-0.00	17.18
○Thread 3656475			10.93	17.53	62.33	0.89	-0.00	17.18
○Thread 3656476			10.92	17.53	62.27	0.89	-0.00	17.18
○Thread 3656477			10.92	17.53	62.30	0.89	-0.00	17.18
○Thread 3656478			10.79	17.53	61.53	0.89	-0.00	17.18
○Thread 3656479			10.75	17.53	61.33	0.89	0.00	17.18
○Thread 3656480			10.75	17.53	61.33	0.89	0.00	17.18
○Thread 3656481			10.75	17.53	61.31	0.89	0.00	17.18
○Thread 3656482			10.75	17.53	61.36	0.89	0.00	17.18
○Thread 3656483			10.72	17.53	61.14	0.89	0.00	17.18
○Thread 3656484			10.85	17.53	61.91	0.89	0.00	17.17
○Thread 3656485			10.86	17.53	61.94	0.89	0.00	17.18
○Thread 3656486			10.85	17.53	61.88	0.89	0.00	17.18
○Thread 3656487			10.85	17.53	61.88	0.89	0.00	17.18
○Thread 3656488			10.85	17.53	61.91	0.89	0.00	17.18
○Thread 3656489			10.85	17.53	61.88	0.89	0.00	17.18
○Thread 3656490			10.79	17.53	61.57	0.89	0.00	17.18
○Thread 3656491			10.76	17.53	61.40	0.89	0.00	17.18
○Thread 3656492			10.75	17.53	61.34	0.89	0.00	17.18
○Thread 3656493			10.73	17.53	61.23	0.89	0.00	17.18
○Thread 3656494			10.75	17.53	61.34	0.89	0.00	17.18
○Thread 3656495			10.54	17.53	60.11	0.89	-0.00	17.18

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	8.68
►Process 3666681+		32	8.68
○Thread 3666681			8.06	17.72	45.48	0.00	0.00	17.69
○Thread 3666780			8.68	16.84	51.55	0.89	-0.00	16.49
○Thread 3666781			8.67	16.84	51.47	0.89	-0.00	16.48
○Thread 3666782			8.68	16.84	51.56	0.89	-0.00	16.49
○Thread 3666783			8.67	16.84	51.50	0.89	-0.00	16.49
○Thread 3666784			8.67	16.84	51.50	0.89	-0.00	16.49
○Thread 3666785			8.63	16.84	51.26	0.89	-0.00	16.49
○Thread 3666786			8.64	16.84	51.32	0.89	-0.00	16.49
○Thread 3666787			8.64	16.84	51.32	0.89	-0.00	16.49
○Thread 3666788			8.63	16.84	51.26	0.89	-0.00	16.49
○Thread 3666789			8.64	16.84	51.32	0.89	-0.00	16.49
○Thread 3666790			8.57	16.84	50.90	0.89	-0.00	16.49
○Thread 3666791			8.57	16.84	50.93	0.89	-0.00	16.49
○Thread 3666792			8.58	16.84	50.96	0.89	-0.00	16.49
○Thread 3666793			8.56	16.84	50.82	0.89	-0.00	16.48
○Thread 3666794			8.56	16.84	50.82	0.89	-0.00	16.49
○Thread 3666795			8.67	16.83	51.50	0.89	-0.00	16.49
○Thread 3666796			8.68	16.83	51.56	0.89	-0.00	16.49
○Thread 3666797			8.68	16.83	51.53	0.89	-0.00	16.49
○Thread 3666798			8.68	16.83	51.53	0.89	-0.00	16.49
○Thread 3666799			8.68	16.83	51.53	0.89	-0.00	16.49
○Thread 3666800			8.68	16.83	51.53	0.89	-0.00	16.49
○Thread 3666801			8.42	16.83	50.02	0.89	-0.00	16.49
○Thread 3666802			8.43	16.83	50.05	0.89	-0.00	16.49
○Thread 3666803			8.42	16.83	50.02	0.89	0.00	16.49
○Thread 3666804			8.42	16.83	49.99	0.89	0.00	16.49
○Thread 3666805			8.43	16.83	50.08	0.89	0.00	16.49
○Thread 3666806			8.36	16.83	49.66	0.89	0.00	16.49
○Thread 3666807			8.36	16.83	49.66	0.89	0.00	16.49
○Thread 3666808			8.38	16.83	49.75	0.89	0.00	16.49
○Thread 3666809			8.36	16.83	49.66	0.89	0.00	16.49
○Thread 3666810			8.26	16.83	49.04	0.89	-0.00	16.49

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	7.32
►Process 3680534+		40	7.32
○Thread 3680534			6.05	17.72	34.11	0.00	0.00	17.67
○Thread 3680633			7.32	16.84	43.44	0.88	-0.00	16.49
○Thread 3680634			7.29	16.84	43.29	0.88	-0.00	16.49
○Thread 3680635			7.27	16.84	43.17	0.88	-0.00	16.48
○Thread 3680636			7.28	16.84	43.23	0.88	-0.00	16.48
○Thread 3680637			7.29	16.84	43.26	0.88	-0.00	16.49
○Thread 3680638			7.28	16.84	43.20	0.88	-0.00	16.49
○Thread 3680639			7.28	16.84	43.23	0.88	-0.00	16.48
○Thread 3680640			7.27	16.84	43.17	0.88	-0.00	16.48
○Thread 3680641			7.29	16.84	43.29	0.88	-0.00	16.49
○Thread 3680642			7.29	16.84	43.26	0.88	-0.00	16.49
○Thread 3680643			7.28	16.84	43.23	0.88	-0.00	16.49
○Thread 3680644			7.28	16.84	43.23	0.88	-0.00	16.49
○Thread 3680645			7.29	16.84	43.26	0.88	-0.00	16.49
○Thread 3680646			7.28	16.84	43.23	0.88	-0.00	16.49
○Thread 3680647			7.23	16.84	42.94	0.88	-0.00	16.49
○Thread 3680648			7.23	16.84	42.91	0.88	-0.00	16.49
○Thread 3680649			7.22	16.84	42.88	0.88	-0.00	16.49
○Thread 3680650			7.22	16.84	42.85	0.88	-0.00	16.49
○Thread 3680651			7.19	16.84	42.67	0.88	-0.00	16.49
○Thread 3680652			7.32	16.84	43.47	0.88	-0.00	16.49
○Thread 3680653			7.31	16.84	43.41	0.88	-0.00	16.49
○Thread 3680654			7.10	16.84	42.17	0.88	-0.00	16.49
○Thread 3680655			7.09	16.84	42.11	0.88	-0.00	16.48
○Thread 3680656			7.11	16.84	42.20	0.88	-0.00	16.49
○Thread 3680657			7.09	16.84	42.08	0.88	-0.00	16.48
○Thread 3680658			7.11	16.84	42.20	0.88	-0.00	16.49
○Thread 3680659			7.11	16.84	42.20	0.88	-0.00	16.49
○Thread 3680660			7.07	16.84	41.99	0.88	-0.00	16.49
○Thread 3680661			7.09	16.84	42.08	0.88	-0.00	16.49
○Thread 3680662			7.09	16.84	42.11	0.88	-0.00	16.49
○Thread 3680663			7.10	16.84	42.17	0.88	0.00	16.49
○Thread 3680664			7.09	16.84	42.11	0.88	0.00	16.49
○Thread 3680665			7.08	16.84	42.05	0.88	-0.00	16.49
○Thread 3680666			7.09	16.84	42.08	0.88	0.00	16.49
○Thread 3680667			7.05	16.84	41.87	0.88	0.00	16.49
○Thread 3680668			7.05	16.84	41.87	0.88	0.00	16.49
○Thread 3680669			7.03	16.84	41.73	0.88	0.00	16.49
○Thread 3680670			7.02	16.84	41.67	0.88	0.00	16.49
○Thread 3680671			6.84	16.84	40.60	0.88	-0.00	16.48

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	6.41
►Process 3697979+		48	6.41
○Thread 3697979			5.77	18.36	31.43	0.00	0.00	18.30
○Thread 3698078			6.40	17.47	36.61	0.89	-0.00	17.11
○Thread 3698079			6.39	17.47	36.58	0.89	-0.00	17.11
○Thread 3698080			6.40	17.47	36.64	0.89	-0.00	17.11
○Thread 3698081			6.39	17.47	36.55	0.89	-0.00	17.11
○Thread 3698082			6.41	17.47	36.67	0.89	-0.00	17.11
○Thread 3698083			6.30	17.47	36.07	0.89	-0.00	17.12
○Thread 3698084			6.30	17.47	36.07	0.89	-0.00	17.11
○Thread 3698085			6.30	17.47	36.07	0.89	-0.00	17.11
○Thread 3698086			6.30	17.47	36.04	0.89	-0.00	17.11
○Thread 3698087			6.29	17.47	35.98	0.89	-0.00	17.11
○Thread 3698088			6.25	17.47	35.75	0.89	-0.00	17.11
○Thread 3698089			6.26	17.47	35.84	0.89	-0.00	17.11
○Thread 3698090			6.28	17.47	35.93	0.89	-0.00	17.11
○Thread 3698091			6.27	17.47	35.87	0.89	-0.00	17.11
○Thread 3698092			6.24	17.47	35.73	0.89	-0.00	17.11
○Thread 3698093			6.20	17.47	35.47	0.89	-0.00	17.11
○Thread 3698094			6.18	17.47	35.38	0.89	-0.00	17.11
○Thread 3698095			6.21	17.47	35.53	0.89	-0.00	17.11
○Thread 3698096			6.21	17.47	35.53	0.89	-0.00	17.11
○Thread 3698097			6.22	17.47	35.59	0.89	-0.00	17.12
○Thread 3698098			6.20	17.46	35.50	0.89	-0.00	17.11
○Thread 3698099			6.18	17.46	35.39	0.89	-0.00	17.12
○Thread 3698100			6.14	17.46	35.16	0.89	-0.00	17.11
○Thread 3698101			6.17	17.46	35.30	0.89	-0.00	17.11
○Thread 3698102			6.17	17.46	35.33	0.89	-0.00	17.10
○Thread 3698103			6.16	17.46	35.27	0.89	-0.00	17.11
○Thread 3698104			6.13	17.46	35.07	0.89	-0.00	17.11
○Thread 3698105			6.12	17.46	35.01	0.89	-0.00	17.11
○Thread 3698106			6.14	17.46	35.13	0.89	-0.00	17.11
○Thread 3698107			6.14	17.46	35.16	0.89	-0.00	17.11
○Thread 3698108			6.10	17.46	34.93	0.89	-0.00	17.11
○Thread 3698109			6.19	17.46	35.42	0.89	-0.00	17.11
○Thread 3698110			6.19	17.46	35.42	0.89	-0.00	17.11
○Thread 3698111			6.18	17.46	35.39	0.89	-0.00	17.11
○Thread 3698112			6.20	17.46	35.47	0.89	-0.00	17.11
○Thread 3698113			6.20	17.46	35.47	0.89	-0.00	17.11
○Thread 3698114			6.18	17.46	35.39	0.89	-0.00	17.11
○Thread 3698115			6.17	17.46	35.33	0.89	-0.00	17.11
○Thread 3698116			6.16	17.46	35.27	0.89	-0.00	17.11
○Thread 3698117			6.16	17.46	35.27	0.89	-0.00	17.11
○Thread 3698118			6.16	17.46	35.27	0.89	-0.00	17.11
○Thread 3698119			6.17	17.46	35.30	0.89	-0.00	17.11
○Thread 3698120			6.13	17.46	35.10	0.89	-0.00	17.11
○Thread 3698121			6.11	17.46	34.99	0.89	-0.00	17.11
○Thread 3698122			6.12	17.46	35.02	0.90	-0.00	17.11
○Thread 3698123			6.12	17.46	35.05	0.90	-0.00	17.11
○Thread 3698124			5.93	17.46	33.93	0.90	-0.00	17.11

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	5.44
►Process 3718922+		56	5.44
○Thread 3718922			4.68	18.60	25.16	0.00	0.00	18.56
○Thread 3719021			5.42	17.71	30.61	0.89	-0.00	17.33
○Thread 3719022			5.42	17.71	30.61	0.89	-0.00	17.32
○Thread 3719023			5.44	17.71	30.69	0.89	-0.00	17.34
○Thread 3719024			5.43	17.71	30.64	0.89	-0.00	17.33
○Thread 3719025			5.39	17.71	30.44	0.89	-0.00	17.33
○Thread 3719026			5.30	17.71	29.93	0.89	-0.00	17.34
○Thread 3719027			5.32	17.71	30.02	0.89	-0.00	17.32
○Thread 3719028			5.31	17.71	29.96	0.89	-0.00	17.35
○Thread 3719029			5.31	17.71	29.99	0.89	-0.00	17.33
○Thread 3719030			5.30	17.71	29.93	0.89	-0.00	17.34
○Thread 3719031			5.31	17.71	29.99	0.89	-0.00	17.34
○Thread 3719032			5.30	17.71	29.93	0.89	-0.00	17.34
○Thread 3719033			5.30	17.71	29.91	0.89	-0.00	17.33
○Thread 3719034			5.36	17.71	30.25	0.89	-0.00	17.33
○Thread 3719035			5.35	17.71	30.19	0.89	-0.00	17.33
○Thread 3719036			5.36	17.70	30.25	0.89	-0.00	17.34
○Thread 3719037			5.35	17.70	30.22	0.89	-0.00	17.33
○Thread 3719038			5.36	17.70	30.27	0.89	-0.00	17.33
○Thread 3719039			5.34	17.70	30.16	0.89	-0.00	17.35
○Thread 3719040			5.29	17.70	29.88	0.89	-0.00	17.34
○Thread 3719041			5.30	17.70	29.91	0.89	-0.00	17.33
○Thread 3719042			5.29	17.70	29.88	0.89	-0.00	17.33
○Thread 3719043			5.30	17.70	29.94	0.89	-0.00	17.33
○Thread 3719044			5.31	17.70	29.96	0.89	-0.00	17.34
○Thread 3719045			5.32	17.70	30.02	0.89	-0.00	17.33
○Thread 3719046			5.32	17.70	30.02	0.89	-0.00	17.34
○Thread 3719047			5.23	17.70	29.51	0.89	-0.00	17.33
○Thread 3719048			5.40	17.70	30.47	0.89	-0.00	17.33
○Thread 3719049			5.40	17.70	30.47	0.89	-0.00	17.33
○Thread 3719050			5.41	17.70	30.56	0.89	-0.00	17.33
○Thread 3719051			5.42	17.70	30.62	0.89	-0.00	17.34
○Thread 3719052			5.41	17.70	30.53	0.90	-0.00	17.33
○Thread 3719053			5.40	17.70	30.47	0.90	-0.00	17.34
○Thread 3719054			5.31	17.70	29.99	0.90	-0.00	17.34
○Thread 3719055			5.30	17.70	29.94	0.90	-0.00	17.33
○Thread 3719056			5.32	17.70	30.02	0.90	-0.00	17.34
○Thread 3719057			5.34	17.70	30.16	0.90	-0.00	17.36
○Thread 3719058			5.33	17.70	30.11	0.90	-0.00	17.34
○Thread 3719059			5.32	17.70	30.05	0.90	-0.00	17.34
○Thread 3719060			5.31	17.70	30.00	0.90	-0.00	17.35
○Thread 3719061			5.30	17.70	29.94	0.90	-0.00	17.34
○Thread 3719062			5.36	17.70	30.28	0.90	-0.00	17.34
○Thread 3719063			5.36	17.70	30.28	0.90	-0.00	17.33
○Thread 3719064			5.35	17.70	30.25	0.90	-0.00	17.34
○Thread 3719065			5.36	17.70	30.28	0.90	-0.00	17.34
○Thread 3719066			5.36	17.70	30.25	0.90	-0.00	17.33
○Thread 3719067			5.34	17.70	30.17	0.90	-0.00	17.33
○Thread 3719068			5.31	17.70	29.97	0.90	-0.00	17.34
○Thread 3719069			5.31	17.70	29.97	0.90	-0.00	17.33
○Thread 3719070			5.33	17.70	30.08	0.90	-0.00	17.33
○Thread 3719071			5.30	17.70	29.94	0.90	-0.00	17.32
○Thread 3719072			5.29	17.70	29.86	0.90	-0.00	17.33
○Thread 3719073			5.29	17.70	29.89	0.90	-0.00	17.33
○Thread 3719074			5.33	17.70	30.08	0.90	-0.00	17.34
○Thread 3719075			5.23	17.70	29.52	0.90	-0.00	17.33

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	4.86
►Process 3743405+		64	4.86
○Thread 3743405			4.28	19.46	22.00	0.00	0.00	19.41
○Thread 3743504			4.85	18.57	26.12	0.89	-0.00	18.20
○Thread 3743505			4.82	18.57	25.93	0.89	-0.00	18.19
○Thread 3743506			4.83	18.57	26.01	0.89	0.00	18.20
○Thread 3743507			4.83	18.57	26.01	0.89	0.00	18.19
○Thread 3743508			4.86	18.57	26.15	0.89	0.00	18.19
○Thread 3743509			4.83	18.57	25.98	0.89	-0.00	18.19
○Thread 3743510			4.82	18.57	25.98	0.89	-0.00	18.19
○Thread 3743511			4.78	18.57	25.74	0.89	0.00	18.19
○Thread 3743512			4.81	18.57	25.88	0.89	-0.00	18.19
○Thread 3743513			4.82	18.57	25.96	0.89	-0.00	18.19
○Thread 3743514			4.78	18.57	25.72	0.89	-0.00	18.21
○Thread 3743515			4.78	18.57	25.72	0.89	-0.00	18.18
○Thread 3743516			4.78	18.57	25.72	0.89	-0.00	18.19
○Thread 3743517			4.76	18.57	25.64	0.89	-0.00	18.19
○Thread 3743518			4.74	18.57	25.53	0.89	-0.00	18.19
○Thread 3743519			4.82	18.57	25.96	0.89	-0.00	18.21
○Thread 3743520			4.80	18.57	25.85	0.89	0.00	18.19
○Thread 3743521			4.82	18.57	25.93	0.89	0.00	18.19
○Thread 3743522			4.81	18.57	25.91	0.89	0.00	18.19
○Thread 3743523			4.82	18.57	25.93	0.89	0.00	18.19
○Thread 3743524			4.81	18.57	25.88	0.89	-0.00	18.19
○Thread 3743525			4.68	18.57	25.21	0.89	0.00	18.20
○Thread 3743526			4.68	18.57	25.21	0.89	0.00	18.19
○Thread 3743527			4.70	18.57	25.31	0.89	0.00	18.21
○Thread 3743528			4.67	18.57	25.15	0.89	0.00	18.19
○Thread 3743529			4.68	18.57	25.23	0.89	0.00	18.19
○Thread 3743530			4.66	18.57	25.10	0.89	0.00	18.21
○Thread 3743531			4.66	18.57	25.10	0.89	-0.00	18.18
○Thread 3743532			4.67	18.57	25.15	0.89	0.00	18.19
○Thread 3743533			4.63	18.57	24.96	0.89	-0.00	18.19
○Thread 3743534			4.57	18.57	24.64	0.89	0.00	18.21
○Thread 3743535			4.80	18.57	25.85	0.89	0.00	18.19
○Thread 3743536			4.83	18.57	26.02	0.89	-0.00	18.19
○Thread 3743537			4.82	18.57	25.96	0.89	0.00	18.19
○Thread 3743538			4.81	18.57	25.91	0.89	0.00	18.18
○Thread 3743539			4.83	18.57	25.99	0.89	0.00	18.18
○Thread 3743540			4.83	18.57	25.99	0.89	0.00	18.21
○Thread 3743541			4.80	18.57	25.83	0.89	0.00	18.19
○Thread 3743542			4.82	18.57	25.94	0.89	-0.00	18.21
○Thread 3743543			4.82	18.56	25.96	0.89	0.00	18.19
○Thread 3743544			4.81	18.56	25.91	0.89	0.00	18.19
○Thread 3743545			4.81	18.56	25.88	0.89	0.00	18.19
○Thread 3743546			4.76	18.56	25.64	0.89	0.00	18.19
○Thread 3743547			4.77	18.56	25.69	0.89	0.00	18.19
○Thread 3743548			4.77	18.56	25.69	0.89	0.00	18.19
○Thread 3743549			4.76	18.56	25.64	0.89	0.00	18.19
○Thread 3743550			4.74	18.56	25.51	0.89	0.00	18.19
○Thread 3743551			4.82	18.56	25.96	0.89	0.00	18.19
○Thread 3743552			4.82	18.56	25.96	0.89	0.00	18.19
○Thread 3743553			4.81	18.56	25.88	0.89	0.00	18.19
○Thread 3743554			4.79	18.56	25.78	0.89	0.00	18.19
○Thread 3743555			4.85	18.56	26.13	0.89	0.00	18.22
○Thread 3743556			4.82	18.56	25.96	0.89	0.00	18.19
○Thread 3743557			4.71	18.56	25.35	0.89	0.00	18.19
○Thread 3743558			4.68	18.56	25.21	0.89	0.00	18.19
○Thread 3743559			4.70	18.56	25.29	0.89	0.00	18.21
○Thread 3743560			4.70	18.56	25.32	0.89	0.00	18.20
○Thread 3743561			4.71	18.56	25.37	0.89	0.00	18.19
○Thread 3743562			4.66	18.56	25.10	0.89	0.00	18.19
○Thread 3743563			4.67	18.56	25.13	0.89	0.00	18.19
○Thread 3743564			4.66	18.56	25.10	0.89	0.00	18.19
○Thread 3743565			4.65	18.56	25.08	0.90	0.00	18.19
○Thread 3743566			4.57	18.56	24.62	0.90	0.00	18.19

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	4.44
►Process 3771483+		72	4.44
○Thread 3771483			3.87	20.35	19.02	0.00	0.00	20.30
○Thread 3771582			4.44	19.46	22.82	0.89	-0.00	19.10
○Thread 3771583			4.44	19.46	22.84	0.89	-0.00	19.09
○Thread 3771584			4.44	19.46	22.82	0.89	-0.00	19.09
○Thread 3771585			4.43	19.46	22.74	0.89	-0.00	19.08
○Thread 3771586			4.44	19.46	22.85	0.89	-0.00	19.09
○Thread 3771587			4.44	19.46	22.85	0.89	-0.00	19.08
○Thread 3771588			4.42	19.46	22.72	0.89	-0.00	19.09
○Thread 3771589			4.43	19.46	22.79	0.89	-0.00	19.10
○Thread 3771590			4.43	19.46	22.77	0.89	-0.00	19.09
○Thread 3771591			4.41	19.46	22.69	0.89	-0.00	19.09
○Thread 3771592			4.43	19.46	22.77	0.89	-0.00	19.09
○Thread 3771593			4.42	19.46	22.72	0.89	-0.00	19.08
○Thread 3771594			4.43	19.46	22.79	0.89	-0.00	19.09
○Thread 3771595			4.41	19.46	22.69	0.89	-0.00	19.09
○Thread 3771596			4.43	19.46	22.74	0.89	-0.00	19.09
○Thread 3771597			4.43	19.46	22.80	0.89	-0.00	19.10
○Thread 3771598			4.40	19.46	22.62	0.89	-0.00	19.09
○Thread 3771599			4.34	19.46	22.31	0.89	-0.00	19.09
○Thread 3771600			4.30	19.46	22.10	0.89	-0.00	19.09
○Thread 3771601			4.24	19.46	21.82	0.89	-0.00	19.08
○Thread 3771602			4.25	19.46	21.84	0.89	-0.00	19.09
○Thread 3771603			4.27	19.46	21.97	0.89	-0.00	19.08
○Thread 3771604			4.27	19.46	21.95	0.89	-0.00	19.10
○Thread 3771605			4.25	19.45	21.85	0.89	-0.00	19.07
○Thread 3771606			4.27	19.45	21.95	0.89	-0.00	19.08
○Thread 3771607			4.28	19.45	22.00	0.89	-0.00	19.10
○Thread 3771608			4.27	19.45	21.97	0.89	-0.00	19.08
○Thread 3771609			4.28	19.45	22.03	0.89	-0.00	19.10
○Thread 3771610			4.25	19.45	21.87	0.90	-0.00	19.09
○Thread 3771611			4.23	19.45	21.77	0.90	-0.00	19.09
○Thread 3771612			4.26	19.45	21.90	0.90	-0.00	19.09
○Thread 3771613			4.24	19.45	21.79	0.90	-0.00	19.08
○Thread 3771614			4.25	19.45	21.85	0.90	-0.00	19.10
○Thread 3771615			4.20	19.45	21.59	0.90	-0.00	19.09
○Thread 3771616			4.19	19.45	21.54	0.90	-0.00	19.10
○Thread 3771617			4.33	19.45	22.28	0.90	-0.00	19.09
○Thread 3771618			4.35	19.45	22.39	0.90	-0.00	19.10
○Thread 3771619			4.35	19.45	22.36	0.90	-0.00	19.09
○Thread 3771620			4.36	19.45	22.41	0.90	-0.00	19.09
○Thread 3771621			4.37	19.45	22.44	0.90	-0.00	19.09
○Thread 3771622			4.35	19.45	22.39	0.90	-0.00	19.09
○Thread 3771623			4.33	19.45	22.26	0.90	-0.00	19.09
○Thread 3771624			4.29	19.45	22.05	0.90	-0.00	19.09
○Thread 3771625			4.29	19.45	22.05	0.90	-0.00	19.08
○Thread 3771626			4.26	19.45	21.90	0.90	-0.00	19.09
○Thread 3771627			4.28	19.45	22.03	0.90	-0.00	19.08
○Thread 3771628			4.26	19.45	21.90	0.90	-0.00	19.08
○Thread 3771629			4.36	19.45	22.39	0.90	-0.00	19.08
○Thread 3771630			4.38	19.45	22.52	0.90	-0.00	19.09
○Thread 3771631			4.39	19.45	22.57	0.90	-0.00	19.09
○Thread 3771632			4.36	19.45	22.44	0.90	-0.00	19.08
○Thread 3771633			4.38	19.45	22.52	0.90	-0.00	19.10
○Thread 3771634			4.35	19.45	22.39	0.90	-0.00	19.09
○Thread 3771635			4.24	19.45	21.80	0.90	-0.00	19.09
○Thread 3771636			4.23	19.45	21.75	0.90	-0.00	19.08
○Thread 3771637			4.24	19.45	21.82	0.90	-0.00	19.09
○Thread 3771638			4.22	19.45	21.72	0.90	-0.00	19.09
○Thread 3771639			4.23	19.45	21.75	0.90	-0.00	19.09
○Thread 3771640			4.24	19.45	21.80	0.90	-0.00	19.09
○Thread 3771641			4.24	19.45	21.82	0.90	-0.00	19.09
○Thread 3771642			4.24	19.45	21.80	0.90	-0.00	19.09
○Thread 3771643			4.25	19.45	21.85	0.90	-0.00	19.09
○Thread 3771644			4.26	19.45	21.90	0.90	-0.00	19.09
○Thread 3771645			4.24	19.45	21.80	0.90	-0.00	19.09
○Thread 3771646			4.24	19.45	21.80	0.90	-0.00	19.09
○Thread 3771647			4.24	19.45	21.80	0.90	-0.00	19.08
○Thread 3771648			4.18	19.45	21.52	0.90	-0.00	19.08
○Thread 3771649			4.19	19.45	21.57	0.90	-0.00	19.09
○Thread 3771650			4.20	19.45	21.62	0.90	-0.00	19.09
○Thread 3771651			4.22	19.45	21.70	0.90	-0.00	19.09
○Thread 3771652			4.12	19.45	21.21	0.90	-0.00	19.10

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	4.06
►Process 3803054+		80	4.06
○Thread 3803054			3.77	21.22	17.77	0.00	0.00	21.17
○Thread 3803153			4.05	20.33	19.92	0.89	-0.00	19.96
○Thread 3803154			4.03	20.33	19.82	0.89	-0.00	19.93
○Thread 3803155			4.03	20.33	19.85	0.89	-0.00	19.96
○Thread 3803156			4.05	20.33	19.92	0.89	-0.00	19.96
○Thread 3803157			4.03	20.33	19.85	0.89	-0.00	19.96
○Thread 3803158			4.05	20.33	19.92	0.89	-0.00	19.96
○Thread 3803159			4.04	20.33	19.87	0.89	-0.00	19.96
○Thread 3803160			4.04	20.33	19.90	0.89	-0.00	19.96
○Thread 3803161			4.04	20.33	19.87	0.89	-0.00	19.96
○Thread 3803162			4.02	20.33	19.78	0.89	-0.00	19.96
○Thread 3803163			4.01	20.33	19.75	0.89	-0.00	19.96
○Thread 3803164			4.02	20.33	19.78	0.89	-0.00	19.96
○Thread 3803165			4.04	20.33	19.87	0.89	-0.00	19.96
○Thread 3803166			4.04	20.33	19.87	0.89	-0.00	19.96
○Thread 3803167			4.01	20.33	19.73	0.89	-0.00	19.97
○Thread 3803168			4.00	20.33	19.68	0.89	-0.00	19.97
○Thread 3803169			3.98	20.33	19.58	0.89	-0.00	19.97
○Thread 3803170			4.00	20.33	19.68	0.89	-0.00	19.96
○Thread 3803171			3.98	20.33	19.58	0.89	-0.00	19.96
○Thread 3803172			4.05	20.33	19.95	0.89	-0.00	19.96
○Thread 3803173			4.05	20.33	19.92	0.89	-0.00	19.96
○Thread 3803174			3.93	20.33	19.33	0.89	-0.00	19.96
○Thread 3803175			3.92	20.33	19.28	0.89	-0.00	19.96
○Thread 3803176			3.93	20.33	19.33	0.89	-0.00	19.97
○Thread 3803177			3.91	20.33	19.24	0.89	-0.00	19.97
○Thread 3803178			3.91	20.33	19.24	0.89	-0.00	19.92
○Thread 3803179			3.91	20.33	19.24	0.89	-0.00	19.96
○Thread 3803180			3.91	20.33	19.26	0.89	-0.00	19.96
○Thread 3803181			3.90	20.33	19.19	0.89	-0.00	19.96
○Thread 3803182			3.93	20.33	19.36	0.89	-0.00	19.96
○Thread 3803183			3.92	20.33	19.29	0.89	-0.00	19.96
○Thread 3803184			3.91	20.33	19.26	0.89	-0.00	19.95
○Thread 3803185			3.93	20.33	19.36	0.89	-0.00	19.96
○Thread 3803186			3.91	20.33	19.26	0.89	-0.00	19.95
○Thread 3803187			3.89	20.33	19.16	0.89	-0.00	19.96
○Thread 3803188			3.88	20.33	19.11	0.89	-0.00	19.95
○Thread 3803189			3.90	20.33	19.21	0.89	-0.00	19.96
○Thread 3803190			3.89	20.33	19.16	0.89	-0.00	19.96
○Thread 3803191			3.79	20.33	18.67	0.89	-0.00	19.97
○Thread 3803192			4.04	20.33	19.90	0.89	-0.00	19.95
○Thread 3803193			4.06	20.33	20.00	0.89	-0.00	19.97
○Thread 3803194			4.02	20.32	19.80	0.89	-0.00	19.96
○Thread 3803195			4.02	20.32	19.78	0.89	-0.00	19.96
○Thread 3803196			4.04	20.32	19.88	0.89	-0.00	19.96
○Thread 3803197			4.03	20.32	19.83	0.89	-0.00	19.96
○Thread 3803198			4.04	20.32	19.88	0.89	-0.00	19.95
○Thread 3803199			4.01	20.32	19.75	0.90	-0.00	19.96
○Thread 3803200			4.03	20.32	19.83	0.90	-0.00	19.95
○Thread 3803201			4.03	20.32	19.85	0.90	-0.00	19.96
○Thread 3803202			4.03	20.32	19.83	0.90	-0.00	19.96
○Thread 3803203			4.03	20.32	19.83	0.90	-0.00	19.96
○Thread 3803204			4.01	20.32	19.76	0.90	-0.00	19.96
○Thread 3803205			4.04	20.32	19.88	0.90	-0.00	19.96
○Thread 3803206			4.04	20.32	19.88	0.90	-0.00	19.96
○Thread 3803207			4.00	20.32	19.68	0.90	-0.00	19.96
○Thread 3803208			4.00	20.32	19.68	0.90	-0.00	19.96
○Thread 3803209			4.00	20.32	19.68	0.90	-0.00	19.96
○Thread 3803210			4.00	20.32	19.68	0.90	-0.00	19.95
○Thread 3803211			3.99	20.32	19.66	0.90	-0.00	19.98
○Thread 3803212			4.02	20.32	19.81	0.90	-0.00	19.96
○Thread 3803213			4.03	20.32	19.85	0.90	-0.00	19.96
○Thread 3803214			3.92	20.32	19.31	0.90	-0.00	19.96
○Thread 3803215			3.93	20.32	19.36	0.90	-0.00	19.96
○Thread 3803216			3.94	20.32	19.39	0.90	-0.00	19.96
○Thread 3803217			3.94	20.32	19.39	0.90	-0.00	19.96
○Thread 3803218			3.91	20.32	19.24	0.90	-0.00	19.96
○Thread 3803219			3.94	20.32	19.41	0.90	-0.00	19.96
○Thread 3803220			3.95	20.32	19.44	0.90	-0.00	19.96
○Thread 3803221			3.91	20.32	19.26	0.90	-0.00	19.96
○Thread 3803222			3.93	20.32	19.34	0.90	-0.00	19.96
○Thread 3803223			3.92	20.32	19.31	0.90	-0.00	19.96
○Thread 3803224			3.93	20.32	19.36	0.90	-0.00	19.96
○Thread 3803225			3.92	20.32	19.31	0.90	-0.00	19.95
○Thread 3803226			3.92	20.32	19.31	0.90	-0.00	19.96
○Thread 3803227			3.89	20.32	19.17	0.90	-0.00	19.96
○Thread 3803228			3.91	20.32	19.24	0.90	-0.00	19.97
○Thread 3803229			3.90	20.32	19.19	0.90	-0.00	19.98
○Thread 3803230			3.89	20.32	19.14	0.90	-0.00	19.96
○Thread 3803231			3.79	20.32	18.65	0.90	-0.00	19.93

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	3.76
►Process 3838220+		88	3.76
○Thread 3838220			3.42	22.25	15.37	0.00	0.00	22.20
○Thread 3838319			3.75	21.36	17.58	0.89	-0.00	20.98
○Thread 3838320			3.76	21.36	17.63	0.89	-0.00	20.98
○Thread 3838321			3.76	21.36	17.60	0.89	-0.00	20.98
○Thread 3838322			3.72	21.36	17.42	0.89	-0.00	20.94
○Thread 3838323			3.76	21.36	17.63	0.89	-0.00	20.98
○Thread 3838324			3.75	21.36	17.56	0.89	-0.00	20.98
○Thread 3838325			3.73	21.36	17.46	0.89	-0.00	20.99
○Thread 3838326			3.72	21.36	17.44	0.89	-0.00	20.99
○Thread 3838327			3.74	21.36	17.51	0.89	-0.00	20.98
○Thread 3838328			3.72	21.36	17.44	0.89	-0.00	20.98
○Thread 3838329			3.73	21.36	17.46	0.89	-0.00	20.99
○Thread 3838330			3.69	21.36	17.30	0.89	-0.00	20.98
○Thread 3838331			3.69	21.36	17.30	0.89	-0.00	20.98
○Thread 3838332			3.69	21.36	17.28	0.89	-0.00	20.95
○Thread 3838333			3.70	21.36	17.32	0.89	-0.00	20.98
○Thread 3838334			3.70	21.36	17.32	0.89	-0.00	20.98
○Thread 3838335			3.69	21.36	17.30	0.89	-0.00	20.98
○Thread 3838336			3.68	21.36	17.25	0.89	-0.00	20.98
○Thread 3838337			3.69	21.36	17.28	0.89	-0.00	20.98
○Thread 3838338			3.71	21.36	17.37	0.89	-0.00	20.98
○Thread 3838339			3.67	21.36	17.18	0.89	-0.00	20.98
○Thread 3838340			3.76	21.36	17.63	0.89	-0.00	21.00
○Thread 3838341			3.74	21.36	17.53	0.89	-0.00	20.94
○Thread 3838342			3.73	21.36	17.49	0.89	-0.00	20.98
○Thread 3838343			3.74	21.36	17.51	0.89	-0.00	20.98
○Thread 3838344			3.73	21.36	17.49	0.90	-0.00	20.98
○Thread 3838345			3.73	21.36	17.49	0.90	-0.00	20.98
○Thread 3838346			3.73	21.36	17.46	0.90	-0.00	20.94
○Thread 3838347			3.72	21.36	17.44	0.90	-0.00	20.99
○Thread 3838348			3.73	21.36	17.49	0.90	-0.00	20.98
○Thread 3838349			3.73	21.36	17.47	0.90	-0.00	20.99
○Thread 3838350			3.72	21.36	17.42	0.90	-0.00	20.99
○Thread 3838351			3.71	21.36	17.40	0.90	-0.00	20.98
○Thread 3838352			3.59	21.36	16.81	0.90	-0.00	20.98
○Thread 3838353			3.56	21.36	16.67	0.90	-0.00	20.98
○Thread 3838354			3.58	21.36	16.76	0.90	-0.00	20.98
○Thread 3838355			3.58	21.36	16.76	0.90	-0.00	20.98
○Thread 3838356			3.58	21.36	16.76	0.90	-0.00	20.98
○Thread 3838357			3.57	21.36	16.72	0.90	-0.00	20.98
○Thread 3838358			3.57	21.36	16.74	0.90	-0.00	20.97
○Thread 3838359			3.57	21.36	16.74	0.90	-0.00	20.98
○Thread 3838360			3.58	21.36	16.76	0.90	-0.00	20.99
○Thread 3838361			3.49	21.36	16.37	0.90	-0.00	20.98
○Thread 3838362			3.74	21.36	17.54	0.90	-0.00	20.98
○Thread 3838363			3.73	21.36	17.49	0.90	-0.00	20.98
○Thread 3838364			3.73	21.36	17.47	0.90	-0.00	20.98
○Thread 3838365			3.73	21.36	17.47	0.90	-0.00	20.99
○Thread 3838366			3.75	21.35	17.56	0.90	-0.00	20.99
○Thread 3838367			3.74	21.35	17.54	0.90	-0.00	20.99
○Thread 3838368			3.74	21.35	17.51	0.90	-0.00	20.98
○Thread 3838369			3.71	21.35	17.37	0.90	-0.00	20.99
○Thread 3838370			3.72	21.35	17.42	0.90	-0.00	20.98
○Thread 3838371			3.71	21.35	17.37	0.90	-0.00	20.98
○Thread 3838372			3.72	21.35	17.44	0.90	-0.00	20.98
○Thread 3838373			3.72	21.35	17.42	0.90	-0.00	20.98
○Thread 3838374			3.70	21.35	17.35	0.90	-0.00	20.98
○Thread 3838375			3.70	21.35	17.33	0.90	-0.00	20.98
○Thread 3838376			3.69	21.35	17.30	0.90	-0.00	20.98
○Thread 3838377			3.70	21.35	17.35	0.90	-0.00	20.99
○Thread 3838378			3.71	21.35	17.37	0.90	-0.00	20.99
○Thread 3838379			3.71	21.35	17.37	0.90	-0.00	20.99
○Thread 3838380			3.69	21.35	17.30	0.90	-0.00	20.98
○Thread 3838381			3.69	21.35	17.30	0.90	-0.00	20.98
○Thread 3838382			3.69	21.35	17.28	0.90	-0.00	20.98
○Thread 3838383			3.69	21.35	17.28	0.90	-0.00	20.98
○Thread 3838384			3.74	21.35	17.52	0.90	-0.00	20.99
○Thread 3838385			3.73	21.35	17.47	0.90	-0.00	20.98
○Thread 3838386			3.73	21.35	17.47	0.90	-0.00	20.98
○Thread 3838387			3.76	21.35	17.61	0.90	-0.00	20.98
○Thread 3838388			3.75	21.35	17.56	0.90	-0.00	20.98
○Thread 3838389			3.75	21.35	17.56	0.90	-0.00	20.99
○Thread 3838390			3.73	21.35	17.47	0.90	-0.00	20.98
○Thread 3838391			3.73	21.35	17.49	0.90	-0.00	20.98
○Thread 3838392			3.72	21.35	17.42	0.90	-0.00	20.98
○Thread 3838393			3.73	21.35	17.47	0.90	-0.00	20.98
○Thread 3838394			3.72	21.35	17.45	0.90	-0.00	20.98
○Thread 3838395			3.73	21.35	17.47	0.90	-0.00	20.98
○Thread 3838396			3.58	21.35	16.79	0.90	-0.00	20.98
○Thread 3838397			3.58	21.35	16.79	0.90	-0.00	20.99
○Thread 3838398			3.59	21.35	16.81	0.90	-0.00	21.00
○Thread 3838399			3.58	21.35	16.77	0.90	-0.00	20.98
○Thread 3838400			3.59	21.35	16.81	0.90	-0.00	20.98
○Thread 3838401			3.58	21.35	16.77	0.90	-0.00	20.98
○Thread 3838402			3.57	21.35	16.74	0.90	-0.00	20.98
○Thread 3838403			3.59	21.35	16.81	0.90	-0.00	20.98
○Thread 3838404			3.58	21.35	16.79	0.90	-0.00	20.99
○Thread 3838405			3.47	21.35	16.25	0.90	-0.00	20.98

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	3.57
►Process 3876885+		96	3.57
○Thread 3876885			2.88	24.35	11.83	0.00	0.00	24.30
○Thread 3876984			3.56	23.45	15.20	0.89	-0.00	23.07
○Thread 3876985			3.57	23.45	15.22	0.89	-0.00	23.08
○Thread 3876986			3.56	23.45	15.20	0.89	-0.00	23.07
○Thread 3876987			3.56	23.45	15.20	0.89	-0.00	23.07
○Thread 3876988			3.54	23.45	15.12	0.89	-0.00	23.07
○Thread 3876989			3.47	23.45	14.80	0.89	-0.00	23.07
○Thread 3876990			3.47	23.45	14.82	0.89	-0.00	23.07
○Thread 3876991			3.46	23.45	14.78	0.89	-0.00	23.05
○Thread 3876992			3.46	23.45	14.75	0.89	-0.00	23.07
○Thread 3876993			3.49	23.45	14.88	0.89	-0.00	23.08
○Thread 3876994			3.51	23.45	14.97	0.89	-0.00	23.07
○Thread 3876995			3.47	23.45	14.82	0.89	-0.00	23.07
○Thread 3876996			3.47	23.45	14.80	0.89	-0.00	23.07
○Thread 3876997			3.47	23.45	14.80	0.89	-0.00	23.07
○Thread 3876998			3.47	23.45	14.82	0.89	-0.00	23.07
○Thread 3876999			3.47	23.45	14.82	0.89	-0.00	23.08
○Thread 3877000			3.49	23.45	14.88	0.89	-0.00	23.07
○Thread 3877001			3.46	23.45	14.75	0.89	-0.00	23.07
○Thread 3877002			3.45	23.45	14.73	0.89	-0.00	23.07
○Thread 3877003			3.45	23.45	14.71	0.89	-0.00	23.07
○Thread 3877004			3.44	23.45	14.67	0.89	-0.00	23.07
○Thread 3877005			3.45	23.45	14.71	0.90	-0.00	23.07
○Thread 3877006			3.44	23.45	14.67	0.90	-0.00	23.07
○Thread 3877007			3.52	23.45	15.03	0.90	-0.00	23.07
○Thread 3877008			3.50	23.45	14.93	0.90	-0.00	23.07
○Thread 3877009			3.49	23.45	14.88	0.90	-0.00	23.06
○Thread 3877010			3.47	23.45	14.80	0.90	-0.00	23.07
○Thread 3877011			3.46	23.45	14.78	0.90	-0.00	23.06
○Thread 3877012			3.49	23.45	14.88	0.90	-0.00	23.07
○Thread 3877013			3.45	23.45	14.71	0.90	-0.00	23.06
○Thread 3877014			3.47	23.45	14.82	0.90	-0.00	23.07
○Thread 3877015			3.47	23.45	14.80	0.90	-0.00	23.07
○Thread 3877016			3.46	23.45	14.78	0.90	-0.00	23.07
○Thread 3877017			3.47	23.45	14.82	0.90	-0.00	23.07
○Thread 3877018			3.46	23.45	14.78	0.90	-0.00	23.02
○Thread 3877019			3.46	23.45	14.78	0.90	-0.00	23.07
○Thread 3877020			3.45	23.45	14.73	0.90	-0.00	23.07
○Thread 3877021			3.48	23.45	14.84	0.90	-0.00	23.07
○Thread 3877022			3.46	23.45	14.78	0.90	-0.00	23.07
○Thread 3877023			3.48	23.45	14.84	0.90	-0.00	23.07
○Thread 3877024			3.49	23.45	14.88	0.90	-0.00	23.07
○Thread 3877025			3.47	23.45	14.80	0.90	-0.00	23.07
○Thread 3877026			3.43	23.45	14.63	0.90	-0.00	23.07
○Thread 3877027			3.42	23.45	14.61	0.90	-0.00	23.06
○Thread 3877028			3.43	23.45	14.63	0.90	-0.00	23.07
○Thread 3877029			3.43	23.45	14.63	0.90	-0.00	23.07
○Thread 3877030			3.35	23.45	14.29	0.90	-0.00	23.07
○Thread 3877031			3.55	23.45	15.16	0.90	-0.00	23.07
○Thread 3877032			3.55	23.45	15.16	0.90	-0.00	23.07
○Thread 3877033			3.53	23.45	15.08	0.90	-0.00	23.07
○Thread 3877034			3.56	23.45	15.20	0.90	-0.00	23.06
○Thread 3877035			3.54	23.45	15.12	0.90	-0.00	23.05
○Thread 3877036			3.54	23.45	15.12	0.90	-0.00	23.07
○Thread 3877037			3.45	23.45	14.74	0.90	-0.00	23.08
○Thread 3877038			3.46	23.45	14.78	0.90	-0.00	23.07
○Thread 3877039			3.46	23.45	14.78	0.90	-0.00	23.06
○Thread 3877040			3.46	23.45	14.76	0.90	-0.00	23.07
○Thread 3877041			3.47	23.45	14.82	0.90	-0.00	23.07
○Thread 3877042			3.49	23.45	14.89	0.90	-0.00	23.07
○Thread 3877043			3.44	23.45	14.67	0.90	-0.00	23.07
○Thread 3877044			3.47	23.45	14.80	0.90	-0.00	23.07
○Thread 3877045			3.47	23.45	14.82	0.90	-0.00	23.07
○Thread 3877046			3.47	23.45	14.80	0.90	-0.00	23.07
○Thread 3877047			3.47	23.45	14.80	0.90	-0.00	23.07
○Thread 3877048			3.49	23.45	14.89	0.90	-0.00	23.07
○Thread 3877049			3.45	23.45	14.74	0.90	-0.00	23.07
○Thread 3877050			3.41	23.45	14.57	0.90	-0.00	23.02
○Thread 3877051			3.43	23.44	14.63	0.90	-0.00	23.07
○Thread 3877052			3.44	23.44	14.67	0.90	-0.00	23.07
○Thread 3877053			3.42	23.44	14.61	0.90	-0.00	23.07
○Thread 3877054			3.37	23.44	14.37	0.90	-0.00	23.07
○Thread 3877055			3.49	23.44	14.91	0.90	-0.00	23.07
○Thread 3877056			3.50	23.44	14.95	0.90	-0.00	23.07
○Thread 3877057			3.50	23.44	14.95	0.90	-0.00	23.07
○Thread 3877058			3.51	23.44	14.97	0.90	-0.00	23.07
○Thread 3877059			3.48	23.44	14.87	0.90	-0.00	23.06
○Thread 3877060			3.48	23.44	14.84	0.90	-0.00	23.07
○Thread 3877061			3.45	23.44	14.74	0.90	-0.00	23.07
○Thread 3877062			3.46	23.44	14.78	0.90	-0.00	23.07
○Thread 3877063			3.46	23.44	14.78	0.90	-0.00	23.06
○Thread 3877064			3.45	23.44	14.74	0.90	-0.00	23.06
○Thread 3877065			3.45	23.44	14.74	0.90	-0.00	23.07
○Thread 3877066			3.49	23.44	14.89	0.90	-0.00	23.07
○Thread 3877067			3.47	23.44	14.82	0.90	-0.00	23.07
○Thread 3877068			3.45	23.44	14.74	0.90	-0.00	23.07
○Thread 3877069			3.47	23.44	14.80	0.90	-0.00	23.07
○Thread 3877070			3.47	23.44	14.80	0.90	-0.00	23.07
○Thread 3877071			3.46	23.44	14.76	0.90	-0.00	23.07
○Thread 3877072			3.47	23.44	14.80	0.90	-0.00	23.07
○Thread 3877073			3.47	23.44	14.82	0.90	-0.00	23.07
○Thread 3877074			3.42	23.44	14.61	0.90	-0.00	23.06
○Thread 3877075			3.41	23.44	14.57	0.90	-0.00	23.07
○Thread 3877076			3.43	23.44	14.63	0.90	-0.00	23.06
○Thread 3877077			3.44	23.44	14.67	0.90	-0.00	23.06
○Thread 3877078			3.34	23.44	14.27	0.90	-0.00	23.07

Report Configuration