OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	82.89
►Process 52744+		1	82.89
○Thread 52744			82.89	83.97	98.71	0.00	0.00	83.94

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	44.51
►Process 52879+		2	44.51
○Thread 52879			44.51	45.57	97.66	0.00	0.00	45.54
○Thread 52946			44.18	44.54	99.20	1.04	-0.00	44.19

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	23.28
►Process 53014+		4	23.28
○Thread 53014			23.28	24.36	95.58	0.00	0.00	24.32
○Thread 53081			22.94	23.30	98.47	1.05	0.00	22.95
○Thread 53082			22.95	23.30	98.49	1.05	0.00	22.95
○Thread 53083			22.95	23.30	98.49	1.05	-0.00	22.95

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	12.48
►Process 53105+		8	12.48
○Thread 53105			12.48	13.63	91.55	0.00	0.00	13.58
○Thread 53172			12.16	12.57	96.69	1.05	0.00	12.22
○Thread 53173			12.17	12.57	96.77	1.05	0.00	12.22
○Thread 53174			12.22	12.57	97.17	1.05	0.00	12.22
○Thread 53175			12.18	12.57	96.85	1.05	0.00	12.22
○Thread 53176			12.18	12.57	96.89	1.06	0.00	12.22
○Thread 53177			12.21	12.57	97.09	1.06	0.00	12.22
○Thread 53178			12.17	12.57	96.81	1.06	-0.00	12.22

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	7.47
►Process 53200+		16	7.47
○Thread 53200			7.47	8.60	86.79	0.00	0.00	8.57
○Thread 53267			7.05	7.54	93.51	1.06	-0.00	7.19
○Thread 53268			7.03	7.54	93.18	1.06	0.00	7.18
○Thread 53269			7.05	7.54	93.45	1.06	0.00	7.19
○Thread 53270			7.07	7.54	93.71	1.06	0.00	7.18
○Thread 53271			7.06	7.54	93.58	1.06	0.00	7.18
○Thread 53272			7.10	7.54	94.11	1.06	0.00	7.19
○Thread 53273			7.05	7.54	93.52	1.06	0.00	7.19
○Thread 53274			7.06	7.54	93.65	1.06	0.00	7.18
○Thread 53275			7.01	7.54	92.99	1.06	0.00	7.19
○Thread 53276			7.07	7.54	93.72	1.06	0.00	7.19
○Thread 53277			7.07	7.54	93.72	1.06	0.00	7.18
○Thread 53278			7.03	7.54	93.26	1.06	0.00	7.19
○Thread 53279			7.05	7.54	93.46	1.06	0.00	7.19
○Thread 53280			7.05	7.54	93.46	1.06	0.00	7.19
○Thread 53281			7.07	7.54	93.73	1.06	-0.00	7.19

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	6.63
►Process 53349+		24	6.63
○Thread 53349			6.63	7.92	83.61	0.00	0.00	7.89
○Thread 53417			6.26	6.86	91.15	1.06	0.00	6.51
○Thread 53418			6.32	6.86	92.03	1.06	0.00	6.51
○Thread 53419			6.34	6.86	92.32	1.06	0.00	6.51
○Thread 53420			6.14	6.86	89.48	1.06	0.00	6.51
○Thread 53421			6.25	6.86	91.01	1.06	0.00	6.51
○Thread 53422			6.22	6.86	90.65	1.06	0.00	6.51
○Thread 53423			6.20	6.86	90.36	1.06	0.00	6.51
○Thread 53424			6.17	6.86	89.92	1.06	0.00	6.51
○Thread 53425			6.28	6.86	91.45	1.06	0.00	6.51
○Thread 53426			6.27	6.86	91.31	1.06	0.00	6.51
○Thread 53427			6.28	6.86	91.45	1.06	0.00	6.51
○Thread 53428			6.34	6.86	92.33	1.06	0.00	6.51
○Thread 53429			6.34	6.86	92.40	1.06	0.00	6.51
○Thread 53430			6.19	6.86	90.22	1.06	0.00	6.51
○Thread 53431			6.28	6.86	91.46	1.06	0.00	6.51
○Thread 53432			6.28	6.86	91.53	1.06	0.00	6.51
○Thread 53433			6.22	6.86	90.66	1.06	0.00	6.51
○Thread 53434			6.24	6.86	90.88	1.06	0.00	6.51
○Thread 53435			6.31	6.86	91.90	1.06	0.00	6.51
○Thread 53436			6.26	6.86	91.25	1.06	0.00	6.50
○Thread 53437			6.30	6.86	91.76	1.06	0.00	6.51
○Thread 53438			6.18	6.86	90.01	1.06	0.00	6.51
○Thread 53439			6.30	6.86	91.83	1.06	-0.00	6.51

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	6.29
►Process 53459+		32	6.29
○Thread 53459			6.29	7.61	82.63	0.00	0.00	7.58
○Thread 53526			5.85	6.55	89.36	1.07	0.00	6.18
○Thread 53527			5.96	6.55	91.05	1.07	0.00	6.19
○Thread 53528			5.84	6.55	89.14	1.07	0.00	6.19
○Thread 53529			5.92	6.55	90.36	1.07	0.00	6.18
○Thread 53530			5.92	6.55	90.36	1.07	-0.00	6.18
○Thread 53531			6.04	6.55	92.27	1.07	0.00	6.19
○Thread 53532			5.86	6.55	89.53	1.07	0.00	6.18
○Thread 53533			5.85	6.55	89.37	1.07	0.00	6.19
○Thread 53534			5.91	6.55	90.29	1.07	0.00	6.19
○Thread 53535			5.88	6.55	89.84	1.07	0.00	6.19
○Thread 53536			5.93	6.55	90.52	1.07	0.00	6.19
○Thread 53537			5.94	6.55	90.68	1.07	0.00	6.19
○Thread 53538			5.84	6.55	89.23	1.07	0.00	6.19
○Thread 53539			5.87	6.54	89.69	1.07	-0.00	6.19
○Thread 53540			5.83	6.54	89.08	1.07	0.00	6.19
○Thread 53541			5.83	6.54	89.00	1.07	0.00	6.19
○Thread 53542			5.91	6.54	90.23	1.07	0.00	6.18
○Thread 53543			5.89	6.54	90.00	1.07	0.00	6.18
○Thread 53544			5.84	6.54	89.24	1.07	0.00	6.19
○Thread 53545			5.85	6.54	89.31	1.07	0.00	6.19
○Thread 53546			5.91	6.54	90.23	1.07	0.00	6.19
○Thread 53547			5.85	6.54	89.39	1.07	0.00	6.18
○Thread 53548			5.84	6.54	89.24	1.07	0.00	6.18
○Thread 53549			5.95	6.54	90.85	1.07	0.00	6.18
○Thread 53550			5.81	6.54	88.71	1.07	0.00	6.18
○Thread 53551			5.84	6.54	89.17	1.07	0.00	6.19
○Thread 53552			5.93	6.54	90.62	1.07	0.00	6.19
○Thread 53553			5.86	6.54	89.55	1.07	0.00	6.19
○Thread 53554			5.84	6.54	89.25	1.07	0.00	6.19
○Thread 53555			5.88	6.54	89.94	1.07	0.00	6.18
○Thread 53556			5.91	6.54	90.40	1.07	-0.00	6.19

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	6.03
►Process 53576+		40	6.03
○Thread 53576			6.03	7.52	80.11	0.00	0.00	7.47
○Thread 53643			5.84	6.46	90.42	1.06	0.00	6.10
○Thread 53644			5.88	6.46	91.05	1.06	0.00	6.10
○Thread 53645			5.74	6.46	88.88	1.06	0.00	6.10
○Thread 53646			5.87	6.46	90.82	1.06	0.00	6.10
○Thread 53647			5.65	6.46	87.49	1.06	0.00	6.10
○Thread 53648			5.75	6.46	89.04	1.06	0.00	6.10
○Thread 53649			5.74	6.46	88.89	1.06	0.00	6.10
○Thread 53650			5.70	6.46	88.27	1.06	0.00	6.10
○Thread 53651			5.74	6.46	88.81	1.06	-0.00	6.10
○Thread 53652			5.69	6.46	88.04	1.06	0.00	6.10
○Thread 53653			5.62	6.46	86.96	1.06	0.00	6.10
○Thread 53654			5.74	6.46	88.82	1.06	0.00	6.10
○Thread 53655			5.62	6.46	87.04	1.06	0.00	6.10
○Thread 53656			5.71	6.46	88.43	1.06	0.00	6.10
○Thread 53657			5.65	6.46	87.50	1.06	0.00	6.10
○Thread 53658			5.71	6.46	88.43	1.06	0.00	6.10
○Thread 53659			5.64	6.46	87.35	1.06	0.00	6.10
○Thread 53660			5.67	6.46	87.74	1.06	0.00	6.10
○Thread 53661			5.83	6.46	90.22	1.06	0.00	6.10
○Thread 53662			5.62	6.46	86.97	1.06	0.00	6.10
○Thread 53663			5.63	6.46	87.20	1.06	0.00	6.10
○Thread 53664			5.77	6.46	89.29	1.07	0.00	6.10
○Thread 53665			5.70	6.46	88.29	1.07	0.00	6.10
○Thread 53666			5.66	6.46	87.67	1.07	0.00	6.10
○Thread 53667			5.64	6.46	87.28	1.07	0.00	6.10
○Thread 53668			5.64	6.46	87.36	1.07	0.00	6.10
○Thread 53669			5.69	6.46	88.06	1.07	0.00	6.10
○Thread 53670			5.71	6.46	88.37	1.07	0.00	6.10
○Thread 53671			5.60	6.46	86.75	1.07	0.00	6.10
○Thread 53672			5.65	6.46	87.45	1.07	0.00	6.10
○Thread 53673			5.66	6.46	87.68	1.07	0.00	6.10
○Thread 53674			5.64	6.46	87.29	1.07	0.00	6.10
○Thread 53675			5.62	6.46	86.98	1.07	0.00	6.10
○Thread 53676			5.66	6.46	87.61	1.07	0.00	6.10
○Thread 53677			5.62	6.45	86.99	1.07	0.00	6.09
○Thread 53678			5.68	6.45	88.00	1.07	0.00	6.10
○Thread 53679			5.63	6.45	87.14	1.07	0.00	6.10
○Thread 53680			5.66	6.45	87.61	1.07	0.00	6.10
○Thread 53681			5.63	6.45	87.22	1.07	-0.00	6.10

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	5.85
►Process 53750+		48	5.85
○Thread 53750			5.85	7.24	80.74	0.00	0.00	7.20
○Thread 53817			5.38	6.19	86.83	1.05	0.00	5.83
○Thread 53818			5.26	6.19	84.97	1.05	0.00	5.82
○Thread 53819			5.31	6.19	85.70	1.05	0.00	5.83
○Thread 53820			5.27	6.19	85.14	1.05	0.00	5.83
○Thread 53821			5.37	6.19	86.76	1.05	-0.00	5.83
○Thread 53822			5.34	6.19	86.27	1.05	-0.00	5.82
○Thread 53823			5.29	6.19	85.47	1.05	0.00	5.83
○Thread 53824			5.34	6.19	86.19	1.05	0.00	5.83
○Thread 53825			5.30	6.19	85.63	1.05	0.00	5.83
○Thread 53826			5.27	6.19	85.23	1.05	0.00	5.83
○Thread 53827			5.38	6.19	86.84	1.05	0.00	5.82
○Thread 53828			5.37	6.19	86.76	1.05	0.00	5.83
○Thread 53829			5.23	6.19	84.50	1.05	0.00	5.82
○Thread 53830			5.26	6.19	84.91	1.05	0.00	5.83
○Thread 53831			5.39	6.19	87.01	1.05	0.00	5.83
○Thread 53832			5.31	6.19	85.80	1.05	0.00	5.83
○Thread 53833			5.44	6.19	87.82	1.05	-0.00	5.82
○Thread 53834			5.35	6.19	86.45	1.05	0.00	5.82
○Thread 53835			5.40	6.19	87.26	1.05	0.00	5.83
○Thread 53836			5.30	6.19	85.56	1.05	0.00	5.83
○Thread 53837			5.31	6.19	85.81	1.05	0.00	5.82
○Thread 53838			5.31	6.19	85.73	1.05	0.00	5.82
○Thread 53839			5.35	6.19	86.38	1.05	0.00	5.82
○Thread 53840			5.41	6.19	87.43	1.05	0.00	5.82
○Thread 53841			5.35	6.19	86.38	1.05	0.00	5.83
○Thread 53842			5.32	6.19	85.90	1.05	0.00	5.82
○Thread 53843			5.34	6.19	86.22	1.05	0.00	5.83
○Thread 53844			5.29	6.19	85.49	1.05	0.00	5.83
○Thread 53845			5.28	6.19	85.25	1.05	0.00	5.82
○Thread 53846			5.27	6.19	85.17	1.05	0.00	5.83
○Thread 53847			5.25	6.19	84.77	1.05	0.00	5.82
○Thread 53848			5.34	6.19	86.31	1.05	0.00	5.83
○Thread 53849			5.27	6.19	85.10	1.05	0.00	5.83
○Thread 53850			5.38	6.19	87.04	1.05	0.00	5.82
○Thread 53851			5.31	6.19	85.75	1.05	0.00	5.82
○Thread 53852			5.26	6.19	85.10	1.05	0.00	5.83
○Thread 53853			5.36	6.19	86.56	1.05	0.00	5.82
○Thread 53854			5.26	6.19	84.94	1.05	0.00	5.83
○Thread 53855			5.41	6.19	87.45	1.05	0.00	5.83
○Thread 53856			5.31	6.19	85.83	1.05	0.00	5.82
○Thread 53857			5.26	6.19	85.03	1.05	0.00	5.83
○Thread 53858			5.30	6.19	85.59	1.05	0.00	5.82
○Thread 53859			5.31	6.19	85.84	1.05	0.00	5.83
○Thread 53860			5.33	6.19	86.16	1.05	0.00	5.82
○Thread 53861			5.36	6.19	86.73	1.05	0.00	5.83
○Thread 53862			5.28	6.19	85.36	1.05	0.00	5.83
○Thread 53863			5.33	6.19	86.17	1.05	-0.00	5.83

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	5.55
►Process 53883+		56	5.55
○Thread 53883			5.55	7.13	77.82	0.00	0.00	7.10
○Thread 53950			5.17	6.07	85.08	1.06	0.00	5.71
○Thread 53951			5.16	6.07	85.00	1.06	0.00	5.70
○Thread 53952			5.10	6.07	83.93	1.06	-0.00	5.70
○Thread 53953			5.07	6.07	83.44	1.06	-0.00	5.70
○Thread 53954			5.05	6.07	83.11	1.06	-0.00	5.71
○Thread 53955			5.13	6.07	84.43	1.06	-0.00	5.71
○Thread 53956			5.16	6.07	85.01	1.06	0.00	5.71
○Thread 53957			5.08	6.07	83.61	1.06	0.00	5.70
○Thread 53958			5.17	6.07	85.09	1.06	0.00	5.71
○Thread 53959			5.13	6.07	84.43	1.06	0.00	5.70
○Thread 53960			5.12	6.07	84.35	1.06	0.00	5.71
○Thread 53961			5.17	6.07	85.09	1.06	0.00	5.71
○Thread 53962			5.16	6.07	85.01	1.06	0.00	5.71
○Thread 53963			5.17	6.07	85.18	1.06	0.00	5.71
○Thread 53964			5.06	6.07	83.37	1.06	0.00	5.70
○Thread 53965			5.19	6.07	85.51	1.06	0.00	5.71
○Thread 53966			5.11	6.07	84.28	1.06	0.00	5.71
○Thread 53967			5.14	6.07	84.69	1.06	0.00	5.71
○Thread 53968			5.15	6.07	84.78	1.06	-0.00	5.71
○Thread 53969			5.11	6.07	84.20	1.06	0.00	5.70
○Thread 53970			5.16	6.07	84.94	1.06	0.00	5.70
○Thread 53971			5.17	6.07	85.11	1.06	0.00	5.70
○Thread 53972			5.14	6.07	84.70	1.06	-0.00	5.70
○Thread 53973			5.11	6.07	84.12	1.06	0.00	5.70
○Thread 53974			5.24	6.07	86.35	1.06	-0.00	5.70
○Thread 53975			5.13	6.07	84.54	1.06	-0.00	5.71
○Thread 53976			5.31	6.07	87.42	1.06	-0.00	5.70
○Thread 53977			5.09	6.07	83.80	1.06	-0.00	5.71
○Thread 53978			5.09	6.07	83.88	1.06	-0.00	5.70
○Thread 53979			5.23	6.07	86.11	1.06	-0.00	5.71
○Thread 53980			5.19	6.07	85.45	1.06	-0.00	5.71
○Thread 53981			5.22	6.07	86.03	1.06	-0.00	5.70
○Thread 53982			5.09	6.07	83.89	1.06	-0.00	5.71
○Thread 53983			5.13	6.07	84.55	1.06	-0.00	5.70
○Thread 53984			5.26	6.07	86.61	1.06	-0.00	5.71
○Thread 53985			5.10	6.07	83.97	1.06	-0.00	5.70
○Thread 53986			5.28	6.07	87.02	1.06	-0.00	5.71
○Thread 53987			5.17	6.07	85.13	1.06	-0.00	5.70
○Thread 53988			5.04	6.07	82.99	1.06	0.00	5.71
○Thread 53989			5.17	6.07	85.22	1.06	-0.00	5.71
○Thread 53990			5.09	6.07	83.82	1.06	-0.00	5.71
○Thread 53991			5.24	6.07	86.37	1.06	0.00	5.71
○Thread 53992			5.18	6.07	85.30	1.06	-0.00	5.71
○Thread 53993			5.13	6.07	84.56	1.07	-0.00	5.71
○Thread 53994			5.18	6.07	85.39	1.07	0.00	5.70
○Thread 53995			5.11	6.07	84.24	1.07	0.00	5.71
○Thread 53996			5.18	6.07	85.31	1.07	-0.00	5.70
○Thread 53997			5.15	6.07	84.90	1.07	0.00	5.71
○Thread 53998			5.17	6.07	85.15	1.07	0.00	5.70
○Thread 53999			5.24	6.07	86.38	1.07	0.00	5.70
○Thread 54000			5.10	6.07	83.99	1.07	0.00	5.71
○Thread 54001			5.24	6.07	86.30	1.07	0.00	5.71
○Thread 54002			5.14	6.07	84.66	1.07	0.00	5.70
○Thread 54003			5.19	6.07	85.48	1.07	0.00	5.71
○Thread 54004			5.29	6.07	87.13	1.07	-0.00	5.71

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	5.46
►Process 54025+		64	5.46
○Thread 54025			5.46	7.05	77.50	0.00	0.00	7.01
○Thread 54092			5.17	5.98	86.31	1.06	-0.00	5.61
○Thread 54093			5.10	5.98	85.14	1.06	0.00	5.61
○Thread 54094			5.12	5.98	85.47	1.06	-0.00	5.61
○Thread 54095			5.26	5.98	87.98	1.06	-0.00	5.61
○Thread 54096			5.01	5.98	83.64	1.06	-0.00	5.61
○Thread 54097			5.02	5.98	83.81	1.06	-0.00	5.61
○Thread 54098			5.14	5.98	85.81	1.06	-0.00	5.61
○Thread 54099			5.04	5.98	84.23	1.06	-0.00	5.62
○Thread 54100			5.15	5.98	85.98	1.06	-0.00	5.61
○Thread 54101			5.04	5.98	84.23	1.06	-0.00	5.61
○Thread 54102			4.96	5.98	82.90	1.06	-0.00	5.61
○Thread 54103			5.08	5.98	84.90	1.06	-0.00	5.61
○Thread 54104			4.88	5.98	81.56	1.06	-0.00	5.61
○Thread 54105			5.10	5.98	85.24	1.06	-0.00	5.62
○Thread 54106			4.99	5.98	83.40	1.06	-0.00	5.60
○Thread 54107			4.96	5.98	82.82	1.06	-0.00	5.61
○Thread 54108			4.98	5.98	83.24	1.06	-0.00	5.61
○Thread 54109			5.01	5.98	83.66	1.06	-0.00	5.61
○Thread 54110			4.97	5.98	83.07	1.06	-0.00	5.61
○Thread 54111			4.96	5.98	82.91	1.06	-0.00	5.61
○Thread 54112			4.94	5.98	82.57	1.06	-0.00	5.61
○Thread 54113			4.95	5.98	82.74	1.06	-0.00	5.61
○Thread 54114			5.04	5.98	84.25	1.06	-0.00	5.62
○Thread 54115			5.15	5.98	86.01	1.06	-0.00	5.61
○Thread 54116			4.96	5.98	82.91	1.06	-0.00	5.61
○Thread 54117			4.90	5.98	82.00	1.06	-0.00	5.61
○Thread 54118			4.86	5.98	81.25	1.06	-0.00	5.60
○Thread 54119			4.93	5.98	82.42	1.06	-0.00	5.61
○Thread 54120			5.00	5.98	83.59	1.06	-0.00	5.60
○Thread 54121			5.05	5.98	84.34	1.06	-0.00	5.61
○Thread 54122			4.96	5.98	82.92	1.06	-0.00	5.61
○Thread 54123			5.14	5.98	85.93	1.06	-0.00	5.61
○Thread 54124			5.02	5.98	83.84	1.06	-0.00	5.60
○Thread 54125			4.90	5.98	81.92	1.06	-0.00	5.61
○Thread 54126			4.93	5.98	82.51	1.06	-0.00	5.61
○Thread 54127			4.96	5.98	82.85	1.06	-0.00	5.61
○Thread 54128			4.97	5.98	83.01	1.06	-0.00	5.61
○Thread 54129			4.96	5.98	82.93	1.06	-0.00	5.60
○Thread 54130			5.00	5.98	83.52	1.06	-0.00	5.61
○Thread 54131			4.94	5.98	82.52	1.06	-0.00	5.61
○Thread 54132			4.95	5.98	82.77	1.06	-0.00	5.61
○Thread 54133			5.01	5.98	83.69	1.06	-0.00	5.61
○Thread 54134			5.07	5.98	84.69	1.06	-0.00	5.60
○Thread 54135			4.97	5.98	83.11	1.06	-0.00	5.61
○Thread 54136			5.01	5.98	83.78	1.07	-0.00	5.62
○Thread 54137			5.09	5.98	85.12	1.07	-0.00	5.61
○Thread 54138			5.11	5.98	85.45	1.07	-0.00	5.61
○Thread 54139			5.02	5.98	83.86	1.07	-0.00	5.61
○Thread 54140			5.16	5.98	86.29	1.07	-0.00	5.61
○Thread 54141			4.91	5.98	82.20	1.07	-0.00	5.61
○Thread 54142			4.98	5.98	83.28	1.07	-0.00	5.60
○Thread 54143			4.98	5.98	83.20	1.07	-0.00	5.60
○Thread 54144			4.89	5.98	81.70	1.07	-0.00	5.61
○Thread 54145			4.99	5.98	83.37	1.07	-0.00	5.61
○Thread 54146			5.07	5.98	84.79	1.07	-0.00	5.61
○Thread 54147			4.97	5.98	83.12	1.07	-0.00	5.61
○Thread 54148			4.98	5.98	83.29	1.07	-0.00	5.61
○Thread 54149			5.01	5.98	83.79	1.07	-0.00	5.61
○Thread 54150			4.84	5.98	80.87	1.07	-0.00	5.61
○Thread 54151			4.99	5.98	83.46	1.07	-0.00	5.61
○Thread 54152			4.93	5.98	82.54	1.07	-0.00	5.61
○Thread 54153			5.17	5.98	86.39	1.07	-0.00	5.61
○Thread 54154			4.91	5.98	82.05	1.07	-0.00	5.61

Report Configuration