OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	89.17
►Process 3505048+		1	89.17
○Thread 3505048			89.17	90.20	98.86	0.00	0.00	90.16

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	44.30
►Process 3505227+		2	44.30
○Thread 3505227			44.30	46.66	94.95	0.00	0.00	46.61
○Thread 3505328			44.14	45.78	96.42	0.88	-0.00	45.44

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	22.50
►Process 3505593+		4	22.50
○Thread 3505593			22.50	24.95	90.21	0.00	0.00	24.92
○Thread 3505693			22.34	24.07	92.82	0.88	0.00	23.73
○Thread 3505694			22.27	24.07	92.55	0.88	0.00	23.73
○Thread 3505695			22.18	24.07	92.18	0.88	-0.00	23.73

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	12.43
►Process 3506429+		8	12.43
○Thread 3506429			12.13	15.35	78.98	0.00	0.00	15.32
○Thread 3506528			12.41	14.47	85.72	0.88	0.00	14.13
○Thread 3506529			12.43	14.47	85.86	0.88	0.00	14.13
○Thread 3506530			12.38	14.47	85.52	0.88	0.00	14.13
○Thread 3506531			12.39	14.47	85.59	0.88	0.00	14.13
○Thread 3506532			12.37	14.47	85.48	0.88	0.00	14.13
○Thread 3506533			12.28	14.47	84.83	0.88	0.00	14.13
○Thread 3506534			12.23	14.47	84.48	0.88	-0.00	14.13

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	7.66
►Process 3508101+		16	7.66
○Thread 3508101			7.51	11.30	66.42	0.00	0.00	11.27
○Thread 3508200			7.66	10.43	73.41	0.87	-0.00	10.08
○Thread 3508201			7.64	10.43	73.26	0.87	-0.00	10.08
○Thread 3508202			7.62	10.43	73.07	0.87	-0.00	10.08
○Thread 3508203			7.61	10.43	72.98	0.87	-0.00	10.08
○Thread 3508204			7.63	10.43	73.12	0.87	-0.00	10.08
○Thread 3508205			7.55	10.43	72.40	0.87	-0.00	10.09
○Thread 3508206			7.52	10.43	72.12	0.87	-0.00	10.08
○Thread 3508207			7.58	10.43	72.64	0.87	-0.00	10.09
○Thread 3508208			7.59	10.43	72.79	0.87	-0.00	10.09
○Thread 3508209			7.58	10.43	72.69	0.87	-0.00	10.08
○Thread 3508210			7.56	10.43	72.50	0.87	-0.00	10.08
○Thread 3508211			7.55	10.43	72.41	0.87	0.00	10.09
○Thread 3508212			7.58	10.43	72.65	0.87	0.00	10.09
○Thread 3508213			7.55	10.43	72.41	0.87	-0.00	10.09
○Thread 3508214			7.49	10.43	71.83	0.87	0.00	10.09

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	5.54
►Process 3511597+		24	5.54
○Thread 3511597			5.29	9.93	53.25	0.00	0.00	9.89
○Thread 3511696			5.53	9.04	61.17	0.88	-0.00	8.70
○Thread 3511697			5.53	9.04	61.17	0.88	-0.00	8.70
○Thread 3511698			5.52	9.04	61.06	0.88	-0.00	8.69
○Thread 3511699			5.54	9.04	61.28	0.88	-0.00	8.69
○Thread 3511700			5.53	9.04	61.17	0.88	-0.00	8.70
○Thread 3511701			5.48	9.04	60.56	0.88	-0.00	8.70
○Thread 3511702			5.45	9.04	60.29	0.88	-0.00	8.70
○Thread 3511703			5.45	9.04	60.29	0.89	-0.00	8.69
○Thread 3511704			5.46	9.04	60.40	0.89	-0.00	8.70
○Thread 3511705			5.45	9.04	60.29	0.89	-0.00	8.70
○Thread 3511706			5.43	9.04	60.07	0.89	-0.00	8.70
○Thread 3511707			5.49	9.04	60.68	0.89	-0.00	8.69
○Thread 3511708			5.49	9.04	60.73	0.89	-0.00	8.70
○Thread 3511709			5.49	9.04	60.68	0.89	-0.00	8.70
○Thread 3511710			5.50	9.04	60.85	0.89	-0.00	8.70
○Thread 3511711			5.50	9.04	60.85	0.89	-0.00	8.70
○Thread 3511712			5.49	9.04	60.68	0.89	-0.00	8.70
○Thread 3511713			5.47	9.04	60.52	0.89	-0.00	8.70
○Thread 3511714			5.44	9.04	60.19	0.89	-0.00	8.69
○Thread 3511715			5.44	9.04	60.19	0.89	0.00	8.70
○Thread 3511716			5.45	9.04	60.30	0.89	0.00	8.69
○Thread 3511717			5.45	9.04	60.30	0.89	0.00	8.70
○Thread 3511718			5.35	9.04	59.19	0.89	-0.00	8.70

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	4.41
►Process 3516824+		32	4.41
○Thread 3516824			4.20	9.52	44.12	0.00	0.00	9.47
○Thread 3516923			4.41	8.64	51.03	0.88	-0.00	8.29
○Thread 3516924			4.38	8.64	50.74	0.88	-0.00	8.30
○Thread 3516925			4.41	8.64	51.09	0.88	-0.00	8.30
○Thread 3516926			4.39	8.64	50.86	0.88	-0.00	8.29
○Thread 3516927			4.40	8.64	50.92	0.88	-0.00	8.29
○Thread 3516928			4.38	8.64	50.68	0.88	-0.00	8.29
○Thread 3516929			4.38	8.64	50.69	0.88	-0.00	8.29
○Thread 3516930			4.38	8.64	50.69	0.88	-0.00	8.29
○Thread 3516931			4.38	8.64	50.69	0.88	-0.00	8.29
○Thread 3516932			4.38	8.64	50.63	0.88	-0.00	8.29
○Thread 3516933			4.34	8.64	50.28	0.88	-0.00	8.29
○Thread 3516934			4.36	8.64	50.40	0.88	-0.00	8.29
○Thread 3516935			4.34	8.64	50.17	0.88	-0.00	8.29
○Thread 3516936			4.34	8.64	50.29	0.88	-0.00	8.30
○Thread 3516937			4.32	8.64	50.05	0.88	-0.00	8.29
○Thread 3516938			4.40	8.64	50.92	0.88	-0.00	8.29
○Thread 3516939			4.39	8.64	50.81	0.88	-0.00	8.29
○Thread 3516940			4.40	8.64	50.87	0.88	-0.00	8.29
○Thread 3516941			4.39	8.64	50.87	0.88	-0.00	8.29
○Thread 3516942			4.38	8.64	50.69	0.88	-0.00	8.29
○Thread 3516943			4.40	8.64	50.93	0.88	-0.00	8.29
○Thread 3516944			4.27	8.64	49.48	0.88	-0.00	8.29
○Thread 3516945			4.27	8.64	49.42	0.88	-0.00	8.29
○Thread 3516946			4.26	8.64	49.37	0.88	-0.00	8.29
○Thread 3516947			4.27	8.64	49.48	0.88	-0.00	8.29
○Thread 3516948			4.27	8.64	49.42	0.88	-0.00	8.29
○Thread 3516949			4.24	8.64	49.08	0.88	0.00	8.29
○Thread 3516950			4.23	8.64	48.96	0.88	0.00	8.29
○Thread 3516951			4.23	8.64	48.96	0.88	0.00	8.29
○Thread 3516952			4.25	8.64	49.25	0.88	0.00	8.29
○Thread 3516953			4.16	8.64	48.21	0.88	-0.00	8.29

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	3.71
►Process 3523824+		40	3.71
○Thread 3523824			3.23	9.54	33.89	0.00	0.00	9.51
○Thread 3523923			3.71	8.66	42.92	0.89	-0.00	8.31
○Thread 3523924			3.67	8.66	42.45	0.89	-0.00	8.30
○Thread 3523925			3.68	8.66	42.57	0.89	-0.00	8.30
○Thread 3523926			3.70	8.66	42.74	0.89	-0.00	8.31
○Thread 3523927			3.69	8.66	42.63	0.89	-0.00	8.30
○Thread 3523928			3.70	8.66	42.75	0.89	-0.00	8.31
○Thread 3523929			3.68	8.66	42.51	0.89	-0.00	8.30
○Thread 3523930			3.69	8.66	42.63	0.89	-0.00	8.31
○Thread 3523931			3.68	8.66	42.52	0.89	0.00	8.31
○Thread 3523932			3.68	8.66	42.57	0.89	-0.00	8.30
○Thread 3523933			3.69	8.66	42.69	0.89	-0.00	8.31
○Thread 3523934			3.68	8.66	42.58	0.89	-0.00	8.31
○Thread 3523935			3.68	8.66	42.58	0.89	-0.00	8.31
○Thread 3523936			3.68	8.65	42.58	0.89	-0.00	8.30
○Thread 3523937			3.65	8.65	42.23	0.89	-0.00	8.30
○Thread 3523938			3.66	8.65	42.35	0.89	0.00	8.30
○Thread 3523939			3.65	8.65	42.23	0.89	0.00	8.30
○Thread 3523940			3.65	8.65	42.23	0.89	0.00	8.31
○Thread 3523941			3.64	8.65	42.06	0.89	0.00	8.31
○Thread 3523942			3.71	8.65	42.93	0.89	0.00	8.31
○Thread 3523943			3.70	8.65	42.81	0.89	0.00	8.31
○Thread 3523944			3.61	8.65	41.77	0.89	0.00	8.31
○Thread 3523945			3.61	8.65	41.71	0.89	0.00	8.31
○Thread 3523946			3.60	8.65	41.66	0.89	0.00	8.30
○Thread 3523947			3.60	8.65	41.60	0.89	0.00	8.31
○Thread 3523948			3.61	8.65	41.77	0.89	0.00	8.31
○Thread 3523949			3.60	8.65	41.66	0.89	0.00	8.31
○Thread 3523950			3.61	8.65	41.72	0.89	0.00	8.30
○Thread 3523951			3.61	8.65	41.72	0.89	0.00	8.31
○Thread 3523952			3.60	8.65	41.60	0.89	0.00	8.30
○Thread 3523953			3.60	8.65	41.66	0.89	0.00	8.30
○Thread 3523954			3.60	8.65	41.60	0.89	0.00	8.31
○Thread 3523955			3.59	8.65	41.55	0.89	0.00	8.31
○Thread 3523956			3.60	8.65	41.66	0.89	0.00	8.31
○Thread 3523957			3.57	8.65	41.32	0.89	0.00	8.30
○Thread 3523958			3.57	8.65	41.26	0.89	0.00	8.30
○Thread 3523959			3.57	8.65	41.32	0.89	0.00	8.30
○Thread 3523960			3.57	8.65	41.26	0.89	0.00	8.30
○Thread 3523961			3.48	8.65	40.22	0.89	-0.00	8.31

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	3.26
►Process 3532648+		48	3.26
○Thread 3532648			3.08	9.91	31.14	0.00	0.00	9.85
○Thread 3532747			3.26	9.03	36.10	0.88	-0.00	8.67
○Thread 3532748			3.25	9.03	35.99	0.88	-0.00	8.68
○Thread 3532749			3.24	9.03	35.88	0.88	-0.00	8.68
○Thread 3532750			3.25	9.03	36.05	0.88	-0.00	8.68
○Thread 3532751			3.25	9.03	36.05	0.88	-0.00	8.68
○Thread 3532752			3.21	9.03	35.61	0.88	-0.00	8.68
○Thread 3532753			3.19	9.03	35.33	0.88	-0.00	8.67
○Thread 3532754			3.19	9.03	35.38	0.88	-0.00	8.67
○Thread 3532755			3.18	9.03	35.22	0.88	-0.00	8.68
○Thread 3532756			3.19	9.03	35.39	0.88	0.00	8.67
○Thread 3532757			3.17	9.03	35.11	0.88	-0.00	8.67
○Thread 3532758			3.18	9.03	35.22	0.88	-0.00	8.67
○Thread 3532759			3.17	9.03	35.11	0.88	-0.00	8.67
○Thread 3532760			3.18	9.03	35.22	0.88	-0.00	8.67
○Thread 3532761			3.17	9.03	35.11	0.88	-0.00	8.68
○Thread 3532762			3.15	9.03	34.89	0.88	-0.00	8.68
○Thread 3532763			3.14	9.03	34.83	0.88	-0.00	8.68
○Thread 3532764			3.14	9.03	34.83	0.88	-0.00	8.66
○Thread 3532765			3.13	9.03	34.72	0.88	-0.00	8.67
○Thread 3532766			3.14	9.03	34.78	0.88	0.00	8.67
○Thread 3532767			3.14	9.03	34.84	0.88	-0.00	8.67
○Thread 3532768			3.12	9.03	34.62	0.88	-0.00	8.67
○Thread 3532769			3.12	9.03	34.62	0.88	-0.00	8.67
○Thread 3532770			3.13	9.03	34.73	0.88	-0.00	8.68
○Thread 3532771			3.13	9.03	34.67	0.88	-0.00	8.68
○Thread 3532772			3.13	9.03	34.67	0.88	-0.00	8.68
○Thread 3532773			3.12	9.03	34.56	0.88	-0.00	8.68
○Thread 3532774			3.10	9.03	34.40	0.88	-0.00	8.67
○Thread 3532775			3.11	9.03	34.45	0.88	0.00	8.68
○Thread 3532776			3.11	9.03	34.45	0.88	-0.00	8.67
○Thread 3532777			3.09	9.03	34.29	0.88	-0.00	8.68
○Thread 3532778			3.14	9.03	34.79	0.88	-0.00	8.68
○Thread 3532779			3.14	9.03	34.79	0.88	-0.00	8.68
○Thread 3532780			3.14	9.03	34.84	0.88	-0.00	8.68
○Thread 3532781			3.13	9.03	34.73	0.88	-0.00	8.68
○Thread 3532782			3.14	9.03	34.79	0.88	-0.00	8.68
○Thread 3532783			3.15	9.03	34.90	0.88	-0.00	8.68
○Thread 3532784			3.13	9.03	34.68	0.88	0.00	8.68
○Thread 3532785			3.13	9.03	34.68	0.88	-0.00	8.68
○Thread 3532786			3.12	9.03	34.57	0.88	0.00	8.68
○Thread 3532787			3.12	9.03	34.57	0.88	0.00	8.67
○Thread 3532788			3.11	9.03	34.51	0.88	0.00	8.67
○Thread 3532789			3.10	9.03	34.35	0.88	0.00	8.68
○Thread 3532790			3.10	9.03	34.40	0.88	0.00	8.67
○Thread 3532791			3.11	9.03	34.46	0.88	0.00	8.68
○Thread 3532792			3.11	9.03	34.46	0.88	0.00	8.67
○Thread 3532793			3.02	9.03	33.46	0.88	-0.00	8.68

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	2.76
►Process 3543200+		56	2.76
○Thread 3543200			2.59	10.01	25.89	0.00	0.00	9.97
○Thread 3543299			2.76	9.12	30.27	0.89	-0.00	8.74
○Thread 3543300			2.75	9.12	30.16	0.89	-0.00	8.74
○Thread 3543301			2.76	9.12	30.27	0.89	-0.00	8.73
○Thread 3543302			2.76	9.12	30.33	0.89	-0.00	8.74
○Thread 3543303			2.76	9.12	30.27	0.89	-0.00	8.76
○Thread 3543304			2.69	9.12	29.56	0.89	-0.00	8.74
○Thread 3543305			2.70	9.12	29.67	0.89	-0.00	8.74
○Thread 3543306			2.69	9.12	29.51	0.89	-0.00	8.74
○Thread 3543307			2.70	9.12	29.62	0.89	-0.00	8.75
○Thread 3543308			2.71	9.12	29.73	0.89	-0.00	8.76
○Thread 3543309			2.70	9.12	29.62	0.89	-0.00	8.74
○Thread 3543310			2.68	9.12	29.45	0.89	-0.00	8.75
○Thread 3543311			2.69	9.12	29.51	0.89	-0.00	8.74
○Thread 3543312			2.72	9.12	29.89	0.89	-0.00	8.74
○Thread 3543313			2.72	9.12	29.84	0.89	-0.00	8.74
○Thread 3543314			2.71	9.12	29.78	0.89	-0.00	8.74
○Thread 3543315			2.73	9.12	29.95	0.89	-0.00	8.76
○Thread 3543316			2.72	9.12	29.89	0.89	-0.00	8.74
○Thread 3543317			2.70	9.12	29.68	0.89	-0.00	8.74
○Thread 3543318			2.70	9.12	29.62	0.89	-0.00	8.74
○Thread 3543319			2.71	9.12	29.79	0.89	-0.00	8.76
○Thread 3543320			2.71	9.11	29.73	0.89	-0.00	8.76
○Thread 3543321			2.70	9.11	29.62	0.89	-0.00	8.74
○Thread 3543322			2.69	9.11	29.57	0.89	-0.00	8.75
○Thread 3543323			2.70	9.11	29.62	0.89	-0.00	8.74
○Thread 3543324			2.69	9.11	29.57	0.89	-0.00	8.76
○Thread 3543325			2.66	9.11	29.18	0.89	-0.00	8.75
○Thread 3543326			2.76	9.11	30.28	0.89	-0.00	8.75
○Thread 3543327			2.75	9.11	30.23	0.89	-0.00	8.74
○Thread 3543328			2.75	9.11	30.17	0.89	-0.00	8.75
○Thread 3543329			2.76	9.11	30.28	0.89	-0.00	8.74
○Thread 3543330			2.76	9.11	30.34	0.89	-0.00	8.74
○Thread 3543331			2.75	9.11	30.23	0.89	-0.00	8.74
○Thread 3543332			2.69	9.11	29.57	0.89	-0.00	8.74
○Thread 3543333			2.71	9.11	29.74	0.89	-0.00	8.76
○Thread 3543334			2.69	9.11	29.52	0.89	-0.00	8.74
○Thread 3543335			2.71	9.11	29.79	0.89	-0.00	8.76
○Thread 3543336			2.70	9.11	29.63	0.89	-0.00	8.76
○Thread 3543337			2.71	9.11	29.74	0.89	-0.00	8.75
○Thread 3543338			2.71	9.11	29.74	0.89	-0.00	8.75
○Thread 3543339			2.68	9.11	29.41	0.89	-0.00	8.74
○Thread 3543340			2.73	9.11	29.96	0.89	-0.00	8.74
○Thread 3543341			2.73	9.11	30.01	0.89	-0.00	8.76
○Thread 3543342			2.73	9.11	29.96	0.89	-0.00	8.74
○Thread 3543343			2.71	9.11	29.79	0.89	-0.00	8.74
○Thread 3543344			2.73	9.11	29.96	0.89	-0.00	8.74
○Thread 3543345			2.71	9.11	29.74	0.89	-0.00	8.74
○Thread 3543346			2.70	9.11	29.69	0.89	-0.00	8.74
○Thread 3543347			2.70	9.11	29.63	0.89	-0.00	8.74
○Thread 3543348			2.71	9.11	29.74	0.89	-0.00	8.74
○Thread 3543349			2.70	9.11	29.63	0.89	-0.00	8.74
○Thread 3543350			2.70	9.11	29.63	0.89	-0.00	8.76
○Thread 3543351			2.71	9.11	29.74	0.89	-0.00	8.74
○Thread 3543352			2.70	9.11	29.63	0.89	-0.00	8.75
○Thread 3543353			2.64	9.11	29.03	0.89	-0.00	8.74

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	2.46
►Process 3555528+		64	2.46
○Thread 3555528			2.35	10.42	22.56	0.00	0.00	10.36
○Thread 3555627			2.45	9.53	25.72	0.89	-0.00	9.15
○Thread 3555628			2.45	9.52	25.72	0.89	-0.00	9.14
○Thread 3555629			2.45	9.52	25.77	0.89	-0.00	9.15
○Thread 3555630			2.45	9.52	25.78	0.89	-0.00	9.15
○Thread 3555631			2.45	9.52	25.78	0.89	-0.00	9.14
○Thread 3555632			2.43	9.52	25.51	0.89	-0.00	9.15
○Thread 3555633			2.45	9.52	25.78	0.89	-0.00	9.15
○Thread 3555634			2.44	9.52	25.67	0.89	-0.00	9.15
○Thread 3555635			2.44	9.52	25.67	0.89	-0.00	9.15
○Thread 3555636			2.44	9.52	25.67	0.89	-0.00	9.15
○Thread 3555637			2.43	9.52	25.52	0.89	-0.00	9.14
○Thread 3555638			2.44	9.52	25.62	0.89	-0.00	9.15
○Thread 3555639			2.43	9.52	25.57	0.89	-0.00	9.15
○Thread 3555640			2.42	9.52	25.46	0.89	-0.00	9.14
○Thread 3555641			2.41	9.52	25.36	0.89	-0.00	9.15
○Thread 3555642			2.46	9.52	25.88	0.89	-0.00	9.15
○Thread 3555643			2.44	9.52	25.67	0.89	-0.00	9.14
○Thread 3555644			2.44	9.52	25.62	0.89	-0.00	9.15
○Thread 3555645			2.44	9.52	25.62	0.89	-0.00	9.14
○Thread 3555646			2.44	9.52	25.62	0.89	-0.00	9.15
○Thread 3555647			2.45	9.52	25.78	0.89	-0.00	9.15
○Thread 3555648			2.38	9.52	24.99	0.89	-0.00	9.14
○Thread 3555649			2.38	9.52	24.99	0.89	-0.00	9.14
○Thread 3555650			2.39	9.52	25.15	0.89	-0.00	9.15
○Thread 3555651			2.39	9.52	25.15	0.89	-0.00	9.14
○Thread 3555652			2.38	9.52	24.99	0.89	-0.00	9.14
○Thread 3555653			2.36	9.52	24.84	0.89	-0.00	9.15
○Thread 3555654			2.36	9.52	24.79	0.89	-0.00	9.14
○Thread 3555655			2.36	9.52	24.84	0.89	-0.00	9.15
○Thread 3555656			2.37	9.52	24.94	0.89	-0.00	9.15
○Thread 3555657			2.34	9.52	24.58	0.89	-0.00	9.15
○Thread 3555658			2.44	9.52	25.63	0.89	-0.00	9.14
○Thread 3555659			2.46	9.52	25.89	0.89	-0.00	9.15
○Thread 3555660			2.45	9.52	25.73	0.89	-0.00	9.15
○Thread 3555661			2.45	9.52	25.73	0.89	-0.00	9.15
○Thread 3555662			2.46	9.52	25.89	0.89	-0.00	9.15
○Thread 3555663			2.45	9.52	25.73	0.89	-0.00	9.15
○Thread 3555664			2.45	9.52	25.73	0.89	-0.00	9.16
○Thread 3555665			2.43	9.52	25.58	0.89	-0.00	9.14
○Thread 3555666			2.45	9.52	25.79	0.90	-0.00	9.15
○Thread 3555667			2.42	9.52	25.47	0.90	-0.00	9.15
○Thread 3555668			2.45	9.52	25.79	0.90	-0.00	9.15
○Thread 3555669			2.43	9.52	25.52	0.90	-0.00	9.15
○Thread 3555670			2.42	9.52	25.47	0.90	-0.00	9.14
○Thread 3555671			2.42	9.52	25.42	0.90	-0.00	9.15
○Thread 3555672			2.42	9.52	25.42	0.90	-0.00	9.15
○Thread 3555673			2.42	9.52	25.42	0.90	-0.00	9.15
○Thread 3555674			2.43	9.52	25.58	0.90	-0.00	9.15
○Thread 3555675			2.45	9.52	25.79	0.90	-0.00	9.15
○Thread 3555676			2.45	9.52	25.79	0.90	-0.00	9.15
○Thread 3555677			2.45	9.52	25.74	0.90	-0.00	9.15
○Thread 3555678			2.44	9.52	25.69	0.90	-0.00	9.15
○Thread 3555679			2.45	9.52	25.74	0.90	-0.00	9.15
○Thread 3555680			2.39	9.52	25.16	0.90	-0.00	9.15
○Thread 3555681			2.40	9.52	25.21	0.90	-0.00	9.15
○Thread 3555682			2.37	9.52	24.95	0.90	-0.00	9.14
○Thread 3555683			2.39	9.52	25.11	0.90	-0.00	9.15
○Thread 3555684			2.38	9.52	25.06	0.90	-0.00	9.15
○Thread 3555685			2.38	9.52	25.00	0.90	-0.00	9.15
○Thread 3555686			2.38	9.52	25.00	0.90	-0.00	9.15
○Thread 3555687			2.38	9.52	25.06	0.90	-0.00	9.15
○Thread 3555688			2.38	9.52	25.06	0.90	-0.00	9.15
○Thread 3555689			2.33	9.52	24.53	0.90	-0.00	9.15

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	2.29
►Process 3569681+		72	2.29
○Thread 3569681			2.15	10.99	19.61	0.00	0.00	10.93
○Thread 3569780			2.27	10.11	22.45	0.88	-0.00	9.73
○Thread 3569781			2.26	10.11	22.41	0.88	-0.00	9.74
○Thread 3569782			2.29	10.11	22.65	0.88	-0.00	9.75
○Thread 3569783			2.27	10.11	22.50	0.88	-0.00	9.73
○Thread 3569784			2.26	10.11	22.41	0.88	-0.00	9.73
○Thread 3569785			2.23	10.11	22.11	0.88	-0.00	9.73
○Thread 3569786			2.27	10.11	22.46	0.88	-0.00	9.74
○Thread 3569787			2.25	10.11	22.31	0.88	-0.00	9.74
○Thread 3569788			2.28	10.11	22.56	0.88	-0.00	9.76
○Thread 3569789			2.24	10.11	22.21	0.88	-0.00	9.75
○Thread 3569790			2.27	10.11	22.51	0.88	-0.00	9.75
○Thread 3569791			2.29	10.11	22.66	0.88	-0.00	9.76
○Thread 3569792			2.26	10.11	22.36	0.88	-0.00	9.75
○Thread 3569793			2.26	10.11	22.36	0.88	-0.00	9.75
○Thread 3569794			2.26	10.11	22.36	0.88	-0.00	9.74
○Thread 3569795			2.24	10.11	22.16	0.88	-0.00	9.74
○Thread 3569796			2.27	10.11	22.51	0.88	-0.00	9.75
○Thread 3569797			2.21	10.11	21.91	0.88	-0.00	9.74
○Thread 3569798			2.18	10.11	21.62	0.88	-0.00	9.74
○Thread 3569799			2.19	10.11	21.67	0.88	-0.00	9.76
○Thread 3569800			2.18	10.11	21.57	0.88	-0.00	9.74
○Thread 3569801			2.18	10.11	21.62	0.88	-0.00	9.74
○Thread 3569802			2.19	10.11	21.67	0.88	-0.00	9.75
○Thread 3569803			2.17	10.11	21.47	0.88	-0.00	9.74
○Thread 3569804			2.17	10.11	21.52	0.88	-0.00	9.74
○Thread 3569805			2.17	10.11	21.52	0.88	-0.00	9.74
○Thread 3569806			2.17	10.11	21.47	0.88	-0.00	9.74
○Thread 3569807			2.18	10.11	21.62	0.88	-0.00	9.74
○Thread 3569808			2.16	10.11	21.37	0.88	-0.00	9.74
○Thread 3569809			2.15	10.11	21.27	0.88	-0.00	9.74
○Thread 3569810			2.16	10.11	21.42	0.88	-0.00	9.74
○Thread 3569811			2.17	10.11	21.52	0.88	-0.00	9.75
○Thread 3569812			2.15	10.11	21.32	0.88	-0.00	9.74
○Thread 3569813			2.17	10.11	21.47	0.88	-0.00	9.74
○Thread 3569814			2.13	10.11	21.08	0.88	-0.00	9.74
○Thread 3569815			2.20	10.11	21.77	0.88	-0.00	9.73
○Thread 3569816			2.20	10.11	21.77	0.88	-0.00	9.74
○Thread 3569817			2.22	10.11	22.02	0.88	-0.00	9.74
○Thread 3569818			2.23	10.11	22.12	0.88	-0.00	9.73
○Thread 3569819			2.20	10.10	21.82	0.88	-0.00	9.73
○Thread 3569820			2.22	10.10	21.97	0.88	-0.00	9.75
○Thread 3569821			2.20	10.10	21.82	0.88	-0.00	9.74
○Thread 3569822			2.20	10.10	21.77	0.88	-0.00	9.76
○Thread 3569823			2.19	10.10	21.72	0.88	-0.00	9.75
○Thread 3569824			2.20	10.10	21.77	0.88	-0.00	9.73
○Thread 3569825			2.20	10.10	21.77	0.88	-0.00	9.75
○Thread 3569826			2.20	10.10	21.77	0.88	-0.00	9.75
○Thread 3569827			2.23	10.10	22.07	0.89	-0.00	9.74
○Thread 3569828			2.25	10.10	22.32	0.89	-0.00	9.75
○Thread 3569829			2.23	10.10	22.12	0.89	-0.00	9.74
○Thread 3569830			2.23	10.10	22.07	0.89	-0.00	9.74
○Thread 3569831			2.25	10.10	22.32	0.89	-0.00	9.74
○Thread 3569832			2.24	10.10	22.17	0.89	-0.00	9.74
○Thread 3569833			2.17	10.10	21.48	0.89	-0.00	9.74
○Thread 3569834			2.17	10.10	21.53	0.89	-0.00	9.74
○Thread 3569835			2.17	10.10	21.53	0.89	-0.00	9.74
○Thread 3569836			2.16	10.10	21.43	0.89	-0.00	9.74
○Thread 3569837			2.15	10.10	21.33	0.89	-0.00	9.74
○Thread 3569838			2.18	10.10	21.58	0.89	-0.00	9.74
○Thread 3569839			2.16	10.10	21.43	0.89	-0.00	9.74
○Thread 3569840			2.16	10.10	21.43	0.89	-0.00	9.74
○Thread 3569841			2.17	10.10	21.48	0.89	-0.00	9.74
○Thread 3569842			2.15	10.10	21.28	0.89	-0.00	9.74
○Thread 3569843			2.15	10.10	21.33	0.89	-0.00	9.74
○Thread 3569844			2.15	10.10	21.33	0.89	-0.00	9.73
○Thread 3569845			2.15	10.10	21.33	0.89	-0.00	9.75
○Thread 3569846			2.15	10.10	21.28	0.89	-0.00	9.73
○Thread 3569847			2.15	10.10	21.33	0.89	-0.00	9.75
○Thread 3569848			2.14	10.10	21.18	0.89	-0.00	9.73
○Thread 3569849			2.15	10.10	21.33	0.89	-0.00	9.73
○Thread 3569850			2.09	10.10	20.74	0.89	-0.00	9.74

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	2.07
►Process 3585561+		80	2.07
○Thread 3585561			2.07	11.26	18.43	0.00	0.00	11.19
○Thread 3585660			2.06	10.38	19.85	0.88	-0.00	10.01
○Thread 3585661			2.05	10.38	19.81	0.88	0.00	10.01
○Thread 3585662			2.04	10.38	19.71	0.88	0.00	10.02
○Thread 3585663			2.04	10.38	19.66	0.88	-0.00	9.99
○Thread 3585664			2.06	10.38	19.85	0.88	0.00	10.01
○Thread 3585665			2.04	10.38	19.71	0.88	-0.00	9.99
○Thread 3585666			2.04	10.38	19.66	0.88	0.00	9.99
○Thread 3585667			2.05	10.37	19.76	0.88	0.00	9.99
○Thread 3585668			2.06	10.37	19.86	0.88	-0.00	10.02
○Thread 3585669			2.05	10.37	19.81	0.88	-0.00	10.02
○Thread 3585670			2.04	10.37	19.71	0.88	0.00	10.00
○Thread 3585671			2.05	10.37	19.76	0.88	0.00	10.00
○Thread 3585672			2.05	10.37	19.81	0.88	0.00	10.00
○Thread 3585673			2.04	10.37	19.66	0.88	0.00	10.00
○Thread 3585674			2.03	10.37	19.57	0.88	-0.00	10.01
○Thread 3585675			2.04	10.37	19.71	0.88	-0.00	10.00
○Thread 3585676			2.03	10.37	19.57	0.88	0.00	10.00
○Thread 3585677			2.03	10.37	19.62	0.88	-0.00	10.01
○Thread 3585678			2.04	10.37	19.67	0.88	-0.00	10.01
○Thread 3585679			2.05	10.37	19.81	0.88	-0.00	10.00
○Thread 3585680			2.06	10.37	19.86	0.88	0.00	10.00
○Thread 3585681			1.99	10.37	19.23	0.88	-0.00	10.01
○Thread 3585682			1.99	10.37	19.23	0.89	-0.00	10.01
○Thread 3585683			1.99	10.37	19.23	0.89	0.00	10.02
○Thread 3585684			1.99	10.37	19.23	0.89	-0.00	10.01
○Thread 3585685			1.99	10.37	19.23	0.89	0.00	10.02
○Thread 3585686			2.00	10.37	19.28	0.89	-0.00	10.00
○Thread 3585687			1.99	10.37	19.23	0.89	-0.00	10.01
○Thread 3585688			1.99	10.37	19.19	0.89	0.00	9.99
○Thread 3585689			2.00	10.37	19.33	0.89	-0.00	10.02
○Thread 3585690			1.99	10.37	19.23	0.89	-0.00	10.00
○Thread 3585691			1.97	10.37	18.99	0.89	-0.00	9.99
○Thread 3585692			2.00	10.37	19.28	0.89	-0.00	10.00
○Thread 3585693			1.98	10.37	19.14	0.89	-0.00	10.02
○Thread 3585694			2.00	10.37	19.28	0.89	-0.00	10.02
○Thread 3585695			1.98	10.37	19.14	0.89	-0.00	10.01
○Thread 3585696			1.98	10.37	19.14	0.89	-0.00	10.00
○Thread 3585697			1.98	10.37	19.14	0.89	-0.00	10.00
○Thread 3585698			1.93	10.37	18.66	0.89	-0.00	10.00
○Thread 3585699			2.05	10.37	19.77	0.89	-0.00	10.01
○Thread 3585700			2.05	10.37	19.81	0.89	-0.00	10.02
○Thread 3585701			2.04	10.37	19.67	0.89	-0.00	10.00
○Thread 3585702			2.05	10.37	19.77	0.89	-0.00	10.00
○Thread 3585703			2.06	10.37	19.86	0.89	-0.00	10.00
○Thread 3585704			2.04	10.37	19.72	0.89	-0.00	9.99
○Thread 3585705			2.05	10.37	19.82	0.89	-0.00	9.99
○Thread 3585706			2.05	10.37	19.82	0.89	-0.00	10.00
○Thread 3585707			2.05	10.37	19.77	0.89	-0.00	10.01
○Thread 3585708			2.06	10.37	19.86	0.89	-0.00	10.00
○Thread 3585709			2.04	10.37	19.67	0.89	-0.00	10.02
○Thread 3585710			2.03	10.37	19.58	0.89	-0.00	10.02
○Thread 3585711			2.04	10.37	19.72	0.89	-0.00	10.00
○Thread 3585712			2.03	10.37	19.62	0.89	-0.00	10.00
○Thread 3585713			2.05	10.37	19.77	0.89	-0.00	10.01
○Thread 3585714			2.03	10.37	19.62	0.89	-0.00	10.00
○Thread 3585715			2.03	10.37	19.62	0.89	-0.00	10.00
○Thread 3585716			2.05	10.37	19.77	0.89	-0.00	10.01
○Thread 3585717			2.03	10.37	19.62	0.89	-0.00	10.01
○Thread 3585718			2.03	10.37	19.63	0.89	-0.00	10.01
○Thread 3585719			2.04	10.37	19.72	0.89	-0.00	10.01
○Thread 3585720			2.05	10.37	19.82	0.89	-0.00	10.01
○Thread 3585721			2.00	10.37	19.29	0.89	-0.00	10.00
○Thread 3585722			2.00	10.37	19.34	0.89	-0.00	10.02
○Thread 3585723			2.00	10.37	19.29	0.89	-0.00	10.01
○Thread 3585724			2.01	10.37	19.39	0.89	-0.00	10.00
○Thread 3585725			1.99	10.37	19.24	0.89	-0.00	9.96
○Thread 3585726			2.00	10.37	19.29	0.89	-0.00	10.00
○Thread 3585727			2.00	10.37	19.29	0.89	-0.00	10.01
○Thread 3585728			2.00	10.37	19.34	0.89	-0.00	10.00
○Thread 3585729			1.99	10.37	19.24	0.89	-0.00	9.99
○Thread 3585730			1.99	10.37	19.24	0.89	-0.00	10.00
○Thread 3585731			1.99	10.37	19.19	0.89	-0.00	9.99
○Thread 3585732			1.99	10.37	19.24	0.89	-0.00	10.00
○Thread 3585733			1.99	10.37	19.24	0.89	-0.00	10.00
○Thread 3585734			1.99	10.37	19.19	0.89	-0.00	9.96
○Thread 3585735			1.98	10.37	19.10	0.89	-0.00	10.00
○Thread 3585736			1.99	10.37	19.19	0.89	-0.00	10.00
○Thread 3585737			1.98	10.37	19.15	0.89	-0.00	10.01
○Thread 3585738			1.92	10.37	18.57	0.89	-0.00	10.00

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	1.92
►Process 3603217+		88	1.92
○Thread 3603217			1.87	12.01	15.61	0.00	0.00	11.96
○Thread 3603316			1.92	11.13	17.25	0.88	-0.00	10.74
○Thread 3603317			1.90	11.13	17.12	0.88	-0.00	10.73
○Thread 3603318			1.92	11.13	17.30	0.88	-0.00	10.75
○Thread 3603319			1.92	11.13	17.25	0.88	-0.00	10.74
○Thread 3603320			1.91	11.13	17.21	0.88	-0.00	10.74
○Thread 3603321			1.91	11.13	17.21	0.88	-0.00	10.74
○Thread 3603322			1.90	11.13	17.12	0.88	-0.00	10.75
○Thread 3603323			1.90	11.13	17.07	0.88	-0.00	10.74
○Thread 3603324			1.89	11.13	17.03	0.88	-0.00	10.75
○Thread 3603325			1.91	11.13	17.21	0.88	-0.00	10.75
○Thread 3603326			1.89	11.13	17.03	0.88	-0.00	10.74
○Thread 3603327			1.89	11.13	17.03	0.88	-0.00	10.74
○Thread 3603328			1.88	11.13	16.90	0.88	-0.00	10.73
○Thread 3603329			1.90	11.13	17.12	0.88	-0.00	10.74
○Thread 3603330			1.90	11.13	17.08	0.88	-0.00	10.74
○Thread 3603331			1.90	11.13	17.08	0.88	-0.00	10.75
○Thread 3603332			1.88	11.13	16.94	0.88	-0.00	10.75
○Thread 3603333			1.90	11.13	17.12	0.88	-0.00	10.75
○Thread 3603334			1.89	11.13	16.99	0.88	-0.00	10.76
○Thread 3603335			1.90	11.13	17.12	0.88	-0.00	10.75
○Thread 3603336			1.89	11.13	17.03	0.88	-0.00	10.75
○Thread 3603337			1.90	11.13	17.12	0.88	-0.00	10.74
○Thread 3603338			1.92	11.13	17.30	0.88	-0.00	10.75
○Thread 3603339			1.92	11.13	17.26	0.88	-0.00	10.75
○Thread 3603340			1.91	11.13	17.21	0.88	-0.00	10.74
○Thread 3603341			1.92	11.13	17.26	0.88	-0.00	10.75
○Thread 3603342			1.91	11.13	17.21	0.88	-0.00	10.74
○Thread 3603343			1.92	11.13	17.26	0.88	-0.00	10.75
○Thread 3603344			1.91	11.13	17.17	0.88	-0.00	10.75
○Thread 3603345			1.91	11.13	17.17	0.89	-0.00	10.74
○Thread 3603346			1.91	11.12	17.17	0.89	-0.00	10.74
○Thread 3603347			1.89	11.12	17.03	0.89	-0.00	10.74
○Thread 3603348			1.91	11.12	17.17	0.89	-0.00	10.74
○Thread 3603349			1.84	11.12	16.54	0.89	-0.00	10.74
○Thread 3603350			1.84	11.12	16.59	0.89	-0.00	10.75
○Thread 3603351			1.84	11.12	16.54	0.89	-0.00	10.74
○Thread 3603352			1.84	11.12	16.54	0.89	-0.00	10.74
○Thread 3603353			1.84	11.12	16.59	0.89	-0.00	10.75
○Thread 3603354			1.84	11.12	16.54	0.89	-0.00	10.74
○Thread 3603355			1.83	11.12	16.50	0.89	-0.00	10.73
○Thread 3603356			1.83	11.12	16.50	0.89	-0.00	10.74
○Thread 3603357			1.84	11.12	16.59	0.89	-0.00	10.74
○Thread 3603358			1.79	11.12	16.14	0.89	-0.00	10.75
○Thread 3603359			1.91	11.12	17.22	0.89	-0.00	10.75
○Thread 3603360			1.91	11.12	17.22	0.89	-0.00	10.75
○Thread 3603361			1.91	11.12	17.22	0.89	-0.00	10.74
○Thread 3603362			1.92	11.12	17.31	0.89	-0.00	10.75
○Thread 3603363			1.90	11.12	17.08	0.89	-0.00	10.74
○Thread 3603364			1.90	11.12	17.08	0.89	-0.00	10.74
○Thread 3603365			1.92	11.12	17.31	0.89	-0.00	10.75
○Thread 3603366			1.91	11.12	17.17	0.89	-0.00	10.74
○Thread 3603367			1.89	11.12	17.04	0.89	-0.00	10.75
○Thread 3603368			1.89	11.12	17.04	0.89	-0.00	10.72
○Thread 3603369			1.90	11.12	17.08	0.89	-0.00	10.74
○Thread 3603370			1.88	11.12	16.90	0.89	-0.00	10.74
○Thread 3603371			1.89	11.12	16.99	0.89	-0.00	10.73
○Thread 3603372			1.88	11.12	16.90	0.89	-0.00	10.74
○Thread 3603373			1.89	11.12	17.04	0.89	-0.00	10.74
○Thread 3603374			1.89	11.12	16.99	0.89	-0.00	10.74
○Thread 3603375			1.89	11.12	16.99	0.89	-0.00	10.74
○Thread 3603376			1.88	11.12	16.90	0.89	-0.00	10.74
○Thread 3603377			1.89	11.12	17.04	0.89	-0.00	10.75
○Thread 3603378			1.89	11.12	17.04	0.89	-0.00	10.73
○Thread 3603379			1.90	11.12	17.08	0.89	-0.00	10.73
○Thread 3603380			1.86	11.12	16.77	0.89	-0.00	10.75
○Thread 3603381			1.92	11.12	17.31	0.89	-0.00	10.75
○Thread 3603382			1.92	11.12	17.31	0.89	-0.00	10.74
○Thread 3603383			1.91	11.12	17.17	0.89	-0.00	10.73
○Thread 3603384			1.91	11.12	17.18	0.89	-0.00	10.74
○Thread 3603385			1.92	11.12	17.27	0.89	-0.00	10.73
○Thread 3603386			1.91	11.12	17.18	0.89	-0.00	10.72
○Thread 3603387			1.91	11.12	17.22	0.89	-0.00	10.70
○Thread 3603388			1.92	11.12	17.27	0.89	-0.00	10.74
○Thread 3603389			1.90	11.12	17.13	0.89	-0.00	10.73
○Thread 3603390			1.90	11.12	17.13	0.89	-0.00	10.74
○Thread 3603391			1.90	11.12	17.09	0.89	-0.00	10.74
○Thread 3603392			1.91	11.12	17.22	0.89	-0.00	10.75
○Thread 3603393			1.85	11.12	16.68	0.89	-0.00	10.75
○Thread 3603394			1.84	11.12	16.59	0.89	-0.00	10.75
○Thread 3603395			1.85	11.12	16.64	0.89	-0.00	10.74
○Thread 3603396			1.85	11.12	16.64	0.89	-0.00	10.74
○Thread 3603397			1.85	11.12	16.64	0.89	-0.00	10.75
○Thread 3603398			1.84	11.12	16.59	0.89	-0.00	10.75
○Thread 3603399			1.85	11.12	16.64	0.89	-0.00	10.75
○Thread 3603400			1.85	11.12	16.68	0.89	-0.00	10.75
○Thread 3603401			1.82	11.12	16.37	0.89	-0.00	10.74
○Thread 3603402			1.77	11.12	15.96	0.89	-0.00	10.75

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	1.83
►Process 3622700+		96	1.83
○Thread 3622700			1.64	12.88	12.77	0.00	0.00	12.85
○Thread 3622799			1.82	11.99	15.22	0.89	-0.00	11.61
○Thread 3622800			1.81	11.99	15.13	0.89	-0.00	11.61
○Thread 3622801			1.83	11.99	15.26	0.89	-0.00	11.59
○Thread 3622802			1.83	11.99	15.30	0.89	-0.00	11.61
○Thread 3622803			1.81	11.99	15.09	0.89	-0.00	11.61
○Thread 3622804			1.78	11.99	14.88	0.89	-0.00	11.62
○Thread 3622805			1.79	11.99	14.97	0.89	-0.00	11.62
○Thread 3622806			1.75	11.99	14.59	0.89	-0.00	11.58
○Thread 3622807			1.77	11.99	14.80	0.89	-0.00	11.61
○Thread 3622808			1.79	11.99	14.93	0.89	-0.00	11.61
○Thread 3622809			1.79	11.99	14.97	0.89	-0.00	11.61
○Thread 3622810			1.79	11.99	14.97	0.89	-0.00	11.62
○Thread 3622811			1.78	11.99	14.88	0.89	-0.00	11.61
○Thread 3622812			1.77	11.99	14.76	0.89	-0.00	11.61
○Thread 3622813			1.76	11.99	14.72	0.89	-0.00	11.61
○Thread 3622814			1.79	11.99	14.97	0.89	-0.00	11.62
○Thread 3622815			1.79	11.99	14.97	0.89	-0.00	11.62
○Thread 3622816			1.77	11.99	14.80	0.89	-0.00	11.59
○Thread 3622817			1.79	11.99	14.93	0.89	-0.00	11.62
○Thread 3622818			1.77	11.99	14.76	0.89	-0.00	11.61
○Thread 3622819			1.76	11.99	14.68	0.89	-0.00	11.61
○Thread 3622820			1.77	11.99	14.76	0.89	-0.00	11.61
○Thread 3622821			1.77	11.99	14.76	0.89	-0.00	11.61
○Thread 3622822			1.75	11.99	14.64	0.89	-0.00	11.61
○Thread 3622823			1.79	11.99	14.97	0.89	-0.00	11.62
○Thread 3622824			1.78	11.99	14.89	0.89	-0.00	11.61
○Thread 3622825			1.80	11.99	15.01	0.89	-0.00	11.61
○Thread 3622826			1.78	11.99	14.89	0.89	-0.00	11.60
○Thread 3622827			1.81	11.99	15.09	0.89	-0.00	11.62
○Thread 3622828			1.78	11.99	14.84	0.89	-0.00	11.62
○Thread 3622829			1.80	11.99	15.01	0.89	-0.00	11.62
○Thread 3622830			1.78	11.99	14.84	0.89	-0.00	11.61
○Thread 3622831			1.76	11.99	14.68	0.89	-0.00	11.60
○Thread 3622832			1.76	11.99	14.72	0.89	-0.00	11.61
○Thread 3622833			1.78	11.99	14.85	0.89	-0.00	11.61
○Thread 3622834			1.78	11.99	14.89	0.89	-0.00	11.61
○Thread 3622835			1.76	11.99	14.72	0.89	-0.00	11.61
○Thread 3622836			1.79	11.99	14.93	0.89	-0.00	11.61
○Thread 3622837			1.78	11.99	14.89	0.89	-0.00	11.62
○Thread 3622838			1.77	11.99	14.80	0.90	-0.00	11.61
○Thread 3622839			1.77	11.99	14.80	0.90	-0.00	11.61
○Thread 3622840			1.78	11.99	14.89	0.90	-0.00	11.62
○Thread 3622841			1.78	11.99	14.85	0.90	-0.00	11.62
○Thread 3622842			1.76	11.99	14.72	0.90	-0.00	11.61
○Thread 3622843			1.77	11.99	14.80	0.90	-0.00	11.60
○Thread 3622844			1.76	11.99	14.68	0.90	-0.00	11.59
○Thread 3622845			1.71	11.99	14.30	0.90	-0.00	11.61
○Thread 3622846			1.82	11.99	15.22	0.90	-0.00	11.61
○Thread 3622847			1.82	11.99	15.18	0.90	-0.00	11.61
○Thread 3622848			1.82	11.99	15.18	0.90	-0.00	11.61
○Thread 3622849			1.82	11.99	15.18	0.90	-0.00	11.61
○Thread 3622850			1.78	11.99	14.85	0.90	-0.00	11.58
○Thread 3622851			1.81	11.99	15.14	0.90	-0.00	11.61
○Thread 3622852			1.78	11.99	14.89	0.90	-0.00	11.61
○Thread 3622853			1.77	11.99	14.76	0.90	-0.00	11.61
○Thread 3622854			1.73	11.99	14.47	0.90	-0.00	11.60
○Thread 3622855			1.76	11.99	14.72	0.90	-0.00	11.61
○Thread 3622856			1.79	11.99	14.93	0.90	-0.00	11.61
○Thread 3622857			1.79	11.99	14.93	0.90	-0.00	11.63
○Thread 3622858			1.77	11.99	14.77	0.90	-0.00	11.61
○Thread 3622859			1.78	11.99	14.85	0.90	-0.00	11.62
○Thread 3622860			1.74	11.99	14.52	0.90	-0.00	11.61
○Thread 3622861			1.77	11.99	14.81	0.90	-0.00	11.61
○Thread 3622862			1.77	11.99	14.81	0.90	-0.00	11.61
○Thread 3622863			1.77	11.99	14.81	0.90	-0.00	11.61
○Thread 3622864			1.79	11.99	14.93	0.90	-0.00	11.61
○Thread 3622865			1.76	11.99	14.68	0.90	-0.00	11.62
○Thread 3622866			1.75	11.99	14.60	0.90	-0.00	11.60
○Thread 3622867			1.76	11.99	14.68	0.90	-0.00	11.61
○Thread 3622868			1.74	11.99	14.56	0.90	-0.00	11.60
○Thread 3622869			1.76	11.99	14.68	0.90	-0.00	11.61
○Thread 3622870			1.80	11.99	15.06	0.90	-0.00	11.61
○Thread 3622871			1.80	11.99	15.06	0.90	-0.00	11.61
○Thread 3622872			1.78	11.99	14.85	0.90	-0.00	11.61
○Thread 3622873			1.80	11.99	15.02	0.90	-0.00	11.61
○Thread 3622874			1.78	11.99	14.89	0.90	-0.00	11.62
○Thread 3622875			1.78	11.99	14.85	0.90	-0.00	11.60
○Thread 3622876			1.76	11.99	14.73	0.90	-0.00	11.62
○Thread 3622877			1.79	11.99	14.93	0.90	-0.00	11.61
○Thread 3622878			1.79	11.99	14.93	0.90	-0.00	11.61
○Thread 3622879			1.76	11.99	14.73	0.90	-0.00	11.60
○Thread 3622880			1.75	11.99	14.64	0.90	-0.00	11.62
○Thread 3622881			1.76	11.99	14.73	0.90	-0.00	11.61
○Thread 3622882			1.79	11.98	14.94	0.90	-0.00	11.61
○Thread 3622883			1.79	11.98	14.94	0.90	-0.00	11.62
○Thread 3622884			1.79	11.98	14.94	0.90	-0.00	11.62
○Thread 3622885			1.79	11.98	14.98	0.90	-0.00	11.62
○Thread 3622886			1.80	11.98	15.02	0.90	-0.00	11.61
○Thread 3622887			1.79	11.98	14.94	0.90	-0.00	11.61
○Thread 3622888			1.79	11.98	14.98	0.90	-0.00	11.62
○Thread 3622889			1.77	11.98	14.81	0.90	-0.00	11.62
○Thread 3622890			1.74	11.98	14.56	0.90	-0.00	11.61
○Thread 3622891			1.77	11.98	14.77	0.90	-0.00	11.61
○Thread 3622892			1.77	11.98	14.81	0.90	-0.00	11.62
○Thread 3622893			1.70	11.98	14.19	0.90	-0.00	11.63

Report Configuration