OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	31.05
►Process 3923376+		1	31.05
○Thread 3923376			31.05	32.19	96.47	0.00	0.00	27.67

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	16.02
►Process 3923474+		2	16.02
○Thread 3923474			15.04	17.46	86.15	0.00	0.00	16.46
○Thread 3923541			16.02	16.46	97.32	1.00	0.00	16.28

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	8.72
►Process 3923832+		4	8.72
○Thread 3923832			7.31	10.17	71.85	0.00	0.00	9.18
○Thread 3923899			8.72	9.19	94.81	0.98	0.00	9.01
○Thread 3923900			8.71	9.19	94.70	0.98	0.00	9.02
○Thread 3923901			8.67	9.19	94.26	0.98	0.00	9.02

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	5.53
►Process 3924585+		8	5.53
○Thread 3924585			3.78	7.03	53.81	0.00	0.00	6.03
○Thread 3924652			5.49	6.04	91.02	0.99	0.00	5.86
○Thread 3924653			5.49	6.04	91.02	0.99	0.00	5.86
○Thread 3924654			5.52	6.04	91.51	0.99	0.00	5.86
○Thread 3924655			5.50	6.04	91.01	0.99	0.00	5.86
○Thread 3924656			5.53	6.04	91.67	0.99	0.00	5.86
○Thread 3924657			5.52	6.04	91.34	0.99	0.00	5.86
○Thread 3924658			5.51	6.04	91.26	0.99	0.00	5.86

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	4.44
►Process 3926227+		16	4.44
○Thread 3926227			2.43	6.05	40.19	0.00	0.00	5.03
○Thread 3926294			4.36	5.04	86.54	1.00	0.00	4.85
○Thread 3926295			4.31	5.04	85.64	1.00	0.00	4.85
○Thread 3926296			4.43	5.04	87.92	1.00	0.00	4.85
○Thread 3926297			4.39	5.04	87.23	1.00	0.00	4.85
○Thread 3926298			4.42	5.04	87.82	1.00	0.00	4.85
○Thread 3926299			4.36	5.04	86.63	1.00	0.00	4.85
○Thread 3926300			4.34	5.04	86.23	1.00	0.00	4.85
○Thread 3926301			4.34	5.04	86.12	1.00	0.00	4.85
○Thread 3926302			4.44	5.04	88.01	1.00	0.00	4.85
○Thread 3926303			4.35	5.04	86.32	1.00	0.00	4.85
○Thread 3926304			4.34	5.04	86.21	1.00	0.00	4.85
○Thread 3926305			4.42	5.04	87.70	1.00	0.00	4.85
○Thread 3926306			4.39	5.04	87.20	1.00	0.00	4.86
○Thread 3926307			4.39	5.04	87.20	1.00	0.00	4.84
○Thread 3926308			4.34	5.04	86.11	1.00	0.00	4.85

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	4.37
►Process 3929643+		24	4.37
○Thread 3929643			1.51	6.00	25.15	0.00	0.00	4.99
○Thread 3929710			4.18	4.99	83.83	1.00	0.01	4.81
○Thread 3929711			4.16	4.99	83.43	1.00	0.01	4.81
○Thread 3929712			4.20	4.99	84.13	1.00	0.01	4.81
○Thread 3929713			4.20	4.99	84.13	1.00	0.01	4.81
○Thread 3929714			4.21	4.99	84.32	1.00	0.01	4.81
○Thread 3929715			4.15	4.99	83.12	1.00	0.01	4.81
○Thread 3929716			4.19	4.99	84.02	1.00	0.01	4.81
○Thread 3929717			4.19	4.99	83.92	1.00	0.00	4.81
○Thread 3929718			4.13	4.99	82.71	1.01	0.00	4.81
○Thread 3929719			4.14	4.99	83.01	1.01	0.00	4.81
○Thread 3929720			4.25	4.99	85.11	1.01	0.00	4.81
○Thread 3929721			4.19	4.99	83.91	1.01	0.00	4.81
○Thread 3929722			4.17	4.99	83.60	1.01	0.00	4.81
○Thread 3929723			4.20	4.99	84.10	1.01	0.00	4.81
○Thread 3929724			4.19	4.99	83.90	1.01	0.00	4.81
○Thread 3929725			4.13	4.99	82.69	1.01	0.00	4.81
○Thread 3929726			4.23	4.99	84.79	1.01	0.00	4.81
○Thread 3929727			4.20	4.99	84.19	1.01	0.00	4.81
○Thread 3929728			4.37	4.99	87.59	1.01	0.00	4.81
○Thread 3929729			4.22	4.99	84.59	1.01	0.00	4.81
○Thread 3929730			4.19	4.99	83.99	1.01	0.00	4.81
○Thread 3929731			4.22	5.00	84.48	1.01	0.00	4.81
○Thread 3929732			4.26	5.00	85.38	1.01	0.00	4.81

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	4.41
►Process 3934890+		32	4.41
○Thread 3934890			1.36	6.15	22.11	0.00	0.00	5.15
○Thread 3934957			4.24	5.15	82.37	0.99	0.01	4.97
○Thread 3934958			4.27	5.15	82.95	0.99	0.01	4.97
○Thread 3934959			4.28	5.15	83.14	0.99	0.01	4.97
○Thread 3934960			4.26	5.15	82.56	0.99	0.01	4.97
○Thread 3934961			4.35	5.15	84.40	0.99	0.01	4.97
○Thread 3934962			4.22	5.15	81.97	0.99	0.01	4.97
○Thread 3934963			4.36	5.15	84.59	0.99	0.01	4.97
○Thread 3934964			4.28	5.15	83.13	0.99	0.01	4.97
○Thread 3934965			4.29	5.15	83.22	0.99	0.01	4.97
○Thread 3934966			4.27	5.15	82.83	0.99	0.01	4.97
○Thread 3934967			4.32	5.16	83.90	0.99	0.01	4.97
○Thread 3934968			4.21	5.16	81.67	0.99	0.01	4.97
○Thread 3934969			4.30	5.16	83.51	0.99	0.01	4.97
○Thread 3934970			4.25	5.16	82.44	0.99	0.01	4.97
○Thread 3934971			4.27	5.16	82.92	0.99	0.01	4.97
○Thread 3934972			4.20	5.16	81.56	0.99	0.00	4.97
○Thread 3934973			4.32	5.16	83.69	0.99	0.00	4.97
○Thread 3934974			4.29	5.16	83.20	0.99	0.00	4.97
○Thread 3934975			4.24	5.16	82.33	0.99	0.00	4.97
○Thread 3934976			4.23	5.16	82.03	0.99	0.00	4.97
○Thread 3934977			4.29	5.16	83.19	0.99	0.00	4.97
○Thread 3934978			4.38	5.16	84.84	0.99	0.00	4.97
○Thread 3934979			4.35	5.16	84.45	0.99	0.00	4.97
○Thread 3934980			4.32	5.16	83.87	0.99	0.00	4.97
○Thread 3934981			4.22	5.16	81.83	0.99	0.00	4.96
○Thread 3934982			4.24	5.16	82.31	0.99	0.00	4.96
○Thread 3934983			4.34	5.16	84.15	0.99	0.00	4.97
○Thread 3934984			4.31	5.16	83.57	0.99	0.00	4.97
○Thread 3934985			4.41	5.16	85.50	0.99	0.00	4.97
○Thread 3934986			4.24	5.16	82.30	0.99	0.00	4.97
○Thread 3934987			4.28	5.16	83.08	0.99	0.00	4.97

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	4.69
►Process 3941878+		40	4.69
○Thread 3941878			1.00	6.61	15.12	0.00	0.00	5.60
○Thread 3941945			4.64	5.61	82.71	0.99	0.01	5.43
○Thread 3941946			4.56	5.61	81.28	0.99	0.01	5.43
○Thread 3941947			4.59	5.61	81.82	0.99	0.01	5.43
○Thread 3941948			4.55	5.61	81.10	0.99	0.01	5.42
○Thread 3941949			4.59	5.61	81.81	0.99	0.01	5.43
○Thread 3941950			4.67	5.61	83.23	0.99	0.01	5.43
○Thread 3941951			4.61	5.61	82.07	0.99	0.01	5.41
○Thread 3941952			4.63	5.61	82.43	0.99	0.01	5.41
○Thread 3941953			4.53	5.61	80.64	0.99	0.01	5.42
○Thread 3941954			4.57	5.61	81.44	0.99	0.01	5.42
○Thread 3941955			4.54	5.61	80.91	0.99	0.01	5.42
○Thread 3941956			4.58	5.61	81.62	0.99	0.01	5.42
○Thread 3941957			4.57	5.61	81.35	0.99	0.01	5.42
○Thread 3941958			4.59	5.61	81.70	0.99	0.01	5.42
○Thread 3941959			4.59	5.61	81.79	0.99	0.01	5.42
○Thread 3941960			4.51	5.61	80.45	0.99	0.01	5.42
○Thread 3941961			4.55	5.61	81.07	0.99	0.01	5.43
○Thread 3941962			4.56	5.61	81.16	0.99	0.01	5.42
○Thread 3941963			4.58	5.61	81.51	0.99	0.01	5.42
○Thread 3941964			4.61	5.61	82.04	0.99	0.01	5.42
○Thread 3941965			4.65	5.61	82.84	0.99	0.01	5.42
○Thread 3941966			4.49	5.61	79.99	0.99	0.01	5.42
○Thread 3941967			4.61	5.61	82.04	0.99	0.01	5.43
○Thread 3941968			4.44	5.61	79.01	0.99	0.00	5.41
○Thread 3941969			4.62	5.61	82.30	0.99	0.00	5.42
○Thread 3941970			4.69	5.61	83.64	0.99	0.00	5.42
○Thread 3941971			4.60	5.61	81.85	0.99	0.00	5.41
○Thread 3941972			4.53	5.61	80.69	0.99	0.00	5.41
○Thread 3941973			4.49	5.61	79.98	0.99	0.00	5.41
○Thread 3941974			4.52	5.61	80.42	0.99	0.00	5.41
○Thread 3941975			4.64	5.61	82.73	0.99	0.00	5.41
○Thread 3941976			4.60	5.61	81.84	0.99	0.00	5.42
○Thread 3941977			4.56	5.61	81.22	0.99	0.00	5.41
○Thread 3941978			4.51	5.61	80.23	1.00	0.00	5.42
○Thread 3941979			4.54	5.61	80.77	1.00	0.00	5.42
○Thread 3941980			4.59	5.62	81.74	1.00	0.00	5.43
○Thread 3941981			4.54	5.62	80.85	1.00	0.00	5.43
○Thread 3941982			4.64	5.62	82.63	1.00	0.00	5.43
○Thread 3941983			4.58	5.62	81.56	1.00	0.00	5.43

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	5.14
►Process 3950623+		48	5.14
○Thread 3950623			0.93	7.07	13.08	0.00	0.00	6.05
○Thread 3950690			4.96	6.06	81.88	1.00	0.01	5.87
○Thread 3950691			4.93	6.06	81.38	1.00	0.01	5.86
○Thread 3950692			5.08	6.06	83.77	1.00	0.01	5.87
○Thread 3950693			4.93	6.06	81.38	1.00	0.01	5.87
○Thread 3950694			4.98	6.06	82.12	1.00	0.01	5.87
○Thread 3950695			4.97	6.06	82.04	1.00	0.01	5.87
○Thread 3950696			4.90	6.06	80.88	1.00	0.01	5.87
○Thread 3950697			5.08	6.06	83.76	1.00	0.01	5.87
○Thread 3950698			4.86	6.06	80.13	1.00	0.01	5.86
○Thread 3950699			4.93	6.06	81.37	1.00	0.01	5.87
○Thread 3950700			4.90	6.06	80.87	1.00	0.01	5.87
○Thread 3950701			4.90	6.06	80.79	1.00	0.01	5.87
○Thread 3950702			4.91	6.06	81.03	1.00	0.01	5.87
○Thread 3950703			4.90	6.06	80.78	1.00	0.01	5.87
○Thread 3950704			4.98	6.06	82.18	1.00	0.01	5.87
○Thread 3950705			4.92	6.06	81.11	1.00	0.01	5.87
○Thread 3950706			4.91	6.06	81.02	1.00	0.01	5.86
○Thread 3950707			4.96	6.06	81.85	1.00	0.01	5.87
○Thread 3950708			4.90	6.06	80.85	1.00	0.01	5.87
○Thread 3950709			4.92	6.06	81.18	1.00	0.01	5.86
○Thread 3950710			4.95	6.06	81.59	1.00	0.01	5.87
○Thread 3950711			4.92	6.06	81.10	1.00	0.01	5.87
○Thread 3950712			4.94	6.06	81.51	1.00	0.01	5.87
○Thread 3950713			4.94	6.06	81.51	1.00	0.01	5.86
○Thread 3950714			4.95	6.06	81.67	1.00	0.01	5.87
○Thread 3950715			5.01	6.06	82.57	1.00	0.01	5.87
○Thread 3950716			4.99	6.06	82.41	1.00	0.01	5.87
○Thread 3950717			5.00	6.06	82.49	1.00	0.01	5.87
○Thread 3950718			4.91	6.06	81.08	1.01	0.01	5.87
○Thread 3950719			5.14	6.06	84.71	1.01	0.01	5.86
○Thread 3950720			4.99	6.06	82.23	1.01	0.01	5.87
○Thread 3950721			4.94	6.06	81.41	1.01	0.00	5.87
○Thread 3950722			4.94	6.06	81.41	1.01	0.00	5.87
○Thread 3950723			4.84	6.06	79.84	1.01	0.00	5.87
○Thread 3950724			4.91	6.06	80.99	1.01	0.00	5.86
○Thread 3950725			4.98	6.06	82.06	1.01	0.00	5.87
○Thread 3950726			4.92	6.06	81.07	1.01	0.00	5.87
○Thread 3950727			4.84	6.06	79.75	1.01	0.00	5.87
○Thread 3950728			4.89	6.06	80.65	1.01	0.00	5.87
○Thread 3950729			5.08	6.06	83.70	1.01	0.00	5.87
○Thread 3950730			4.85	6.06	79.91	1.01	0.00	5.87
○Thread 3950731			4.96	6.06	81.88	1.01	0.00	5.87
○Thread 3950732			4.93	6.06	81.30	1.01	0.00	5.87
○Thread 3950733			4.91	6.06	80.97	1.01	0.00	5.87
○Thread 3950734			4.94	6.06	81.38	1.01	0.00	5.87
○Thread 3950735			4.84	6.06	79.73	1.01	0.00	5.87
○Thread 3950736			4.98	6.06	82.04	1.01	0.00	5.87

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	5.47
►Process 3961148+		56	5.47
○Thread 3961148			0.92	7.60	12.10	0.00	0.00	6.53
○Thread 3961215			5.24	6.55	79.97	1.03	0.02	6.37
○Thread 3961216			5.34	6.55	81.50	1.03	0.02	6.37
○Thread 3961217			5.40	6.55	82.33	1.03	0.01	6.37
○Thread 3961218			5.47	6.55	83.48	1.03	0.01	6.37
○Thread 3961219			5.32	6.55	81.11	1.03	0.01	6.37
○Thread 3961220			5.36	6.55	81.72	1.03	0.01	6.37
○Thread 3961221			5.29	6.55	80.72	1.03	0.01	6.37
○Thread 3961222			5.35	6.55	81.56	1.03	0.01	6.37
○Thread 3961223			5.31	6.55	81.02	1.04	0.01	6.36
○Thread 3961224			5.26	6.55	80.18	1.04	0.01	6.36
○Thread 3961225			5.30	6.55	80.87	1.04	0.01	6.37
○Thread 3961226			5.23	6.55	79.80	1.04	0.01	6.36
○Thread 3961227			5.34	6.55	81.47	1.04	0.01	6.37
○Thread 3961228			5.35	6.55	81.55	1.04	0.01	6.37
○Thread 3961229			5.38	6.55	82.00	1.04	0.01	6.37
○Thread 3961230			5.39	6.55	82.16	1.04	0.01	6.37
○Thread 3961231			5.36	6.55	81.77	1.04	0.01	6.37
○Thread 3961232			5.36	6.55	81.77	1.04	0.01	6.37
○Thread 3961233			5.23	6.56	79.79	1.04	0.01	6.37
○Thread 3961234			5.37	6.56	81.92	1.04	0.01	6.37
○Thread 3961235			5.37	6.56	81.92	1.04	0.01	6.37
○Thread 3961236			5.26	6.56	80.24	1.04	0.01	6.37
○Thread 3961237			5.31	6.56	81.00	1.04	0.01	6.36
○Thread 3961238			5.26	6.56	80.16	1.04	0.01	6.37
○Thread 3961239			5.36	6.56	81.76	1.04	0.01	6.37
○Thread 3961240			5.32	6.56	81.07	1.04	0.01	6.37
○Thread 3961241			5.32	6.56	81.07	1.04	0.01	6.37
○Thread 3961242			5.32	6.56	81.14	1.04	0.01	6.37
○Thread 3961243			5.28	6.56	80.53	1.04	0.01	6.37
○Thread 3961244			5.41	6.56	82.59	1.04	0.01	6.37
○Thread 3961245			5.21	6.56	79.46	1.04	0.01	6.37
○Thread 3961246			5.24	6.56	79.84	1.04	0.01	6.36
○Thread 3961247			5.24	6.56	79.84	1.04	0.01	6.37
○Thread 3961248			5.22	6.56	79.53	1.04	0.01	6.37
○Thread 3961249			5.25	6.56	79.99	1.04	0.01	6.37
○Thread 3961250			5.28	6.56	80.52	1.04	0.01	6.36
○Thread 3961251			5.41	6.56	82.50	1.04	0.01	6.37
○Thread 3961252			5.27	6.56	80.29	1.04	0.01	6.35
○Thread 3961253			5.35	6.56	81.58	1.04	0.01	6.37
○Thread 3961254			5.31	6.56	80.89	1.04	0.00	6.37
○Thread 3961255			5.42	6.56	82.64	1.04	0.00	6.37
○Thread 3961256			5.17	6.56	78.83	1.04	0.00	6.37
○Thread 3961257			5.22	6.56	79.59	1.04	0.00	6.37
○Thread 3961258			5.25	6.56	79.97	1.04	0.00	6.37
○Thread 3961259			5.42	6.56	82.64	1.04	0.00	6.37
○Thread 3961260			5.28	6.56	80.50	1.04	0.00	6.37
○Thread 3961261			5.36	6.56	81.72	1.04	0.00	6.37
○Thread 3961262			5.30	6.56	80.72	1.04	0.00	6.37
○Thread 3961263			5.24	6.56	79.88	1.04	0.00	6.36
○Thread 3961264			5.39	6.56	82.25	1.04	0.00	6.37
○Thread 3961265			5.33	6.56	81.25	1.04	0.00	6.36
○Thread 3961266			5.32	6.56	81.10	1.04	0.00	6.37
○Thread 3961267			5.35	6.56	81.55	1.04	0.00	6.37
○Thread 3961268			5.14	6.56	78.35	1.04	0.00	6.36
○Thread 3961269			5.33	6.56	81.24	1.04	0.00	6.37

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	7.54
►Process 3973496+		64	7.54
○Thread 3973496			0.99	10.26	9.60	0.00	0.00	9.20
○Thread 3973563			7.35	9.23	79.65	1.01	0.02	9.02
○Thread 3973564			7.36	9.23	79.76	1.01	0.02	9.02
○Thread 3973565			7.49	9.23	81.17	1.01	0.02	9.03
○Thread 3973566			7.43	9.23	80.46	1.01	0.02	9.03
○Thread 3973567			7.41	9.23	80.30	1.01	0.02	9.03
○Thread 3973568			7.40	9.23	80.19	1.01	0.02	9.03
○Thread 3973569			7.49	9.23	81.11	1.01	0.02	9.03
○Thread 3973570			7.42	9.23	80.35	1.01	0.02	9.01
○Thread 3973571			7.44	9.23	80.56	1.01	0.02	9.02
○Thread 3973572			7.45	9.23	80.72	1.01	0.02	9.03
○Thread 3973573			7.44	9.23	80.56	1.01	0.02	9.03
○Thread 3973574			7.39	9.23	80.07	1.01	0.02	9.02
○Thread 3973575			7.54	9.23	81.64	1.01	0.01	9.03
○Thread 3973576			7.39	9.23	80.12	1.01	0.01	9.03
○Thread 3973577			7.46	9.23	80.77	1.01	0.01	9.03
○Thread 3973578			7.50	9.23	81.26	1.01	0.01	9.03
○Thread 3973579			7.49	9.23	81.09	1.01	0.01	9.03
○Thread 3973580			7.25	9.23	78.54	1.01	0.01	9.02
○Thread 3973581			7.47	9.23	80.93	1.01	0.01	9.03
○Thread 3973582			7.36	9.23	79.73	1.01	0.01	9.04
○Thread 3973583			7.47	9.23	80.87	1.01	0.01	9.03
○Thread 3973584			7.36	9.23	79.68	1.01	0.01	9.02
○Thread 3973585			7.26	9.23	78.65	1.01	0.01	9.04
○Thread 3973586			7.37	9.23	79.84	1.01	0.01	9.03
○Thread 3973587			7.49	9.23	81.13	1.01	0.01	9.03
○Thread 3973588			7.07	9.23	76.64	1.01	0.01	9.03
○Thread 3973589			7.34	9.23	79.51	1.01	0.01	9.03
○Thread 3973590			7.39	9.23	80.10	1.01	0.01	9.03
○Thread 3973591			7.47	9.23	80.86	1.01	0.01	9.02
○Thread 3973592			7.31	9.23	79.12	1.01	0.01	9.02
○Thread 3973593			7.42	9.23	80.37	1.01	0.01	9.02
○Thread 3973594			7.49	9.23	81.07	1.01	0.01	9.03
○Thread 3973595			7.40	9.23	80.15	1.01	0.01	9.03
○Thread 3973596			7.27	9.23	78.69	1.01	0.01	9.02
○Thread 3973597			7.47	9.23	80.90	1.01	0.01	9.04
○Thread 3973598			7.42	9.23	80.36	1.01	0.01	9.02
○Thread 3973599			7.41	9.23	80.20	1.01	0.01	9.02
○Thread 3973600			7.34	9.23	79.55	1.01	0.01	9.03
○Thread 3973601			7.38	9.23	79.87	1.01	0.01	9.03
○Thread 3973602			7.22	9.23	78.19	1.01	0.01	9.02
○Thread 3973603			7.46	9.23	80.79	1.01	0.01	9.03
○Thread 3973604			7.43	9.23	80.52	1.01	0.01	9.02
○Thread 3973605			7.47	9.23	80.89	1.01	0.01	9.03
○Thread 3973606			7.39	9.23	79.97	1.01	0.01	9.03
○Thread 3973607			7.13	9.23	77.15	1.01	0.01	9.03
○Thread 3973608			7.29	9.23	78.94	1.02	0.01	9.03
○Thread 3973609			7.36	9.24	79.70	1.02	0.01	9.03
○Thread 3973610			7.33	9.24	79.32	1.02	0.01	9.03
○Thread 3973611			7.32	9.24	79.21	1.02	0.00	9.03
○Thread 3973612			7.30	9.24	78.99	1.02	0.00	9.03
○Thread 3973613			7.37	9.24	79.80	1.02	0.00	9.03
○Thread 3973614			7.45	9.24	80.66	1.02	0.00	9.03
○Thread 3973615			7.32	9.24	79.20	1.02	0.00	9.03
○Thread 3973616			7.43	9.24	80.44	1.02	0.00	9.03
○Thread 3973617			7.31	9.24	79.14	1.02	0.00	9.03
○Thread 3973618			7.30	9.24	79.03	1.02	0.00	9.03
○Thread 3973619			7.29	9.24	78.92	1.02	0.00	9.03
○Thread 3973620			7.26	9.24	78.54	1.02	0.00	9.04
○Thread 3973621			7.44	9.24	80.55	1.02	0.00	9.03
○Thread 3973622			7.29	9.24	78.87	1.02	0.00	9.03
○Thread 3973623			7.25	9.24	78.49	1.02	0.00	9.03
○Thread 3973624			7.27	9.24	78.70	1.02	0.00	9.03
○Thread 3973625			7.37	9.24	79.84	1.02	0.00	9.02

Report Configuration