OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	62.36
►Process 293136+		1	62.36
○Thread 293136			62.36	64.12	97.25	0.00	0.00	64.11

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	32.06
►Process 293323+		2	32.06
○Thread 293323			32.06	33.93	94.50	0.00	0.00	31.66
○Thread 293390			31.25	31.67	98.70	2.26	0.00	31.39

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	16.84
►Process 293410+		4	16.84
○Thread 293410			16.84	18.76	89.75	0.00	0.00	16.48
○Thread 293477			16.07	16.49	97.43	2.27	0.00	16.22
○Thread 293478			16.11	16.49	97.67	2.27	0.00	16.22
○Thread 293479			16.05	16.49	97.34	2.27	0.00	16.22

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	9.50
►Process 293547+		8	9.50
○Thread 293547			9.50	11.30	84.01	0.00	0.00	9.03
○Thread 293614			8.60	9.03	95.19	2.27	0.00	8.76
○Thread 293615			8.62	9.03	95.41	2.27	0.00	8.77
○Thread 293616			8.57	9.03	94.80	2.27	0.00	8.76
○Thread 293617			8.58	9.03	94.91	2.27	0.00	8.77
○Thread 293618			8.60	9.04	95.19	2.27	0.00	8.76
○Thread 293619			8.55	9.04	94.63	2.27	0.00	8.76
○Thread 293620			8.58	9.04	94.91	2.27	0.00	8.76

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	5.74
►Process 293640+		16	5.74
○Thread 293640			5.74	7.66	74.89	0.00	0.00	5.40
○Thread 293707			4.92	5.41	90.88	2.25	0.00	5.14
○Thread 293708			4.85	5.41	89.58	2.25	0.00	5.14
○Thread 293709			4.93	5.41	90.96	2.25	0.00	5.15
○Thread 293710			4.91	5.41	90.68	2.25	0.00	5.14
○Thread 293711			4.85	5.41	89.48	2.25	0.00	5.14
○Thread 293712			4.89	5.41	90.22	2.25	0.00	5.14
○Thread 293713			4.86	5.41	89.66	2.25	0.00	5.14
○Thread 293714			4.91	5.41	90.58	2.25	0.00	5.14
○Thread 293715			4.92	5.42	90.86	2.25	0.00	5.15
○Thread 293716			4.92	5.42	90.76	2.25	0.00	5.14
○Thread 293717			4.85	5.42	89.47	2.25	0.00	5.14
○Thread 293718			4.87	5.42	89.93	2.25	0.00	5.14
○Thread 293719			4.87	5.42	89.83	2.25	0.00	5.14
○Thread 293720			4.84	5.42	89.37	2.25	0.00	5.14
○Thread 293721			4.91	5.42	90.56	2.25	0.00	5.15

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	5.08
►Process 293742+		24	5.08
○Thread 293742			5.08	7.05	72.09	0.00	0.00	4.77
○Thread 293809			4.15	4.78	86.79	2.26	0.01	4.50
○Thread 293810			4.14	4.78	86.58	2.26	0.01	4.49
○Thread 293811			4.16	4.78	87.00	2.26	0.01	4.51
○Thread 293812			4.09	4.78	85.53	2.26	0.01	4.50
○Thread 293813			4.07	4.78	85.21	2.26	0.01	4.50
○Thread 293814			4.13	4.78	86.36	2.26	0.01	4.51
○Thread 293815			4.09	4.78	85.63	2.26	0.01	4.51
○Thread 293816			4.11	4.78	86.04	2.26	0.00	4.49
○Thread 293817			4.08	4.78	85.41	2.26	0.00	4.50
○Thread 293818			4.11	4.78	85.93	2.26	0.00	4.51
○Thread 293819			4.10	4.78	85.72	2.26	0.00	4.51
○Thread 293820			4.11	4.78	85.93	2.26	0.00	4.50
○Thread 293821			4.10	4.78	85.72	2.26	0.00	4.51
○Thread 293822			4.12	4.78	86.23	2.26	0.00	4.50
○Thread 293823			4.18	4.78	87.38	2.26	0.00	4.51
○Thread 293824			4.18	4.78	87.38	2.26	0.00	4.51
○Thread 293825			4.05	4.78	84.76	2.26	0.00	4.50
○Thread 293826			4.09	4.78	85.49	2.26	0.00	4.51
○Thread 293827			4.19	4.78	87.68	2.26	0.00	4.50
○Thread 293828			4.11	4.78	85.91	2.26	0.00	4.51
○Thread 293829			4.14	4.78	86.53	2.26	0.00	4.51
○Thread 293830			4.10	4.78	85.69	2.26	0.00	4.51
○Thread 293831			4.06	4.78	84.85	2.26	0.00	4.51

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	4.82
►Process 293851+		32	4.82
○Thread 293851			4.82	6.84	70.43	0.00	0.00	4.55
○Thread 293918			3.84	4.56	84.12	2.27	0.01	4.29
○Thread 293919			3.88	4.57	84.99	2.27	0.01	4.29
○Thread 293920			3.89	4.57	85.21	2.27	0.01	4.30
○Thread 293921			3.88	4.57	85.10	2.27	0.01	4.29
○Thread 293922			3.90	4.57	85.42	2.27	0.01	4.29
○Thread 293923			3.77	4.57	82.68	2.27	0.01	4.29
○Thread 293924			3.76	4.57	82.46	2.27	0.01	4.29
○Thread 293925			3.85	4.57	84.32	2.27	0.01	4.28
○Thread 293926			3.78	4.57	82.78	2.27	0.01	4.29
○Thread 293927			3.82	4.57	83.66	2.27	0.01	4.29
○Thread 293928			3.75	4.57	82.23	2.27	0.01	4.29
○Thread 293929			3.83	4.57	83.87	2.27	0.01	4.29
○Thread 293930			3.79	4.57	82.99	2.27	0.01	4.29
○Thread 293931			3.76	4.57	82.33	2.27	0.01	4.29
○Thread 293932			3.83	4.57	83.97	2.27	0.01	4.29
○Thread 293933			4.03	4.57	88.35	2.27	0.00	4.29
○Thread 293934			3.80	4.57	83.20	2.27	0.00	4.29
○Thread 293935			3.79	4.57	83.08	2.27	0.00	4.29
○Thread 293936			3.79	4.57	83.08	2.27	0.00	4.29
○Thread 293937			3.86	4.57	84.50	2.27	0.00	4.29
○Thread 293938			3.74	4.57	81.98	2.27	0.00	4.29
○Thread 293939			3.70	4.57	81.10	2.27	0.00	4.29
○Thread 293940			3.76	4.57	82.30	2.27	0.00	4.29
○Thread 293941			3.88	4.57	85.04	2.27	0.00	4.29
○Thread 293942			3.81	4.57	83.39	2.27	0.00	4.30
○Thread 293943			3.80	4.57	83.28	2.27	0.00	4.29
○Thread 293944			3.83	4.57	83.93	2.27	0.00	4.28
○Thread 293945			3.83	4.57	83.93	2.27	0.00	4.29
○Thread 293946			3.75	4.57	82.07	2.27	0.00	4.29
○Thread 293947			3.88	4.57	85.02	2.27	0.00	4.30
○Thread 293948			3.76	4.57	82.39	2.27	0.00	4.30

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	4.55
►Process 294016+		40	4.55
○Thread 294016			4.55	6.64	68.56	0.00	0.00	4.37
○Thread 294083			3.63	4.39	82.77	2.25	0.01	4.11
○Thread 294084			3.53	4.39	80.49	2.25	0.01	4.08
○Thread 294085			3.65	4.39	83.22	2.25	0.01	4.12
○Thread 294086			3.55	4.39	80.94	2.25	0.01	4.11
○Thread 294087			3.60	4.39	82.19	2.25	0.01	4.11
○Thread 294088			3.52	4.39	80.37	2.25	0.01	4.11
○Thread 294089			3.59	4.39	81.84	2.25	0.01	4.12
○Thread 294090			3.52	4.39	80.36	2.25	0.01	4.10
○Thread 294091			3.53	4.39	80.58	2.25	0.01	4.11
○Thread 294092			3.53	4.39	80.58	2.25	0.01	4.12
○Thread 294093			3.55	4.39	81.03	2.25	0.01	4.11
○Thread 294094			3.53	4.39	80.46	2.25	0.01	4.12
○Thread 294095			3.62	4.39	82.51	2.25	0.01	4.11
○Thread 294096			3.58	4.39	81.60	2.25	0.01	4.12
○Thread 294097			3.47	4.39	79.20	2.25	0.01	4.11
○Thread 294098			3.60	4.39	82.16	2.25	0.01	4.12
○Thread 294099			3.58	4.39	81.70	2.25	0.01	4.11
○Thread 294100			3.60	4.39	82.15	2.25	0.01	4.11
○Thread 294101			3.60	4.39	82.04	2.25	0.01	4.11
○Thread 294102			3.56	4.39	81.12	2.25	0.01	4.10
○Thread 294103			3.57	4.39	81.35	2.25	0.01	4.11
○Thread 294104			3.49	4.39	79.52	2.25	0.01	4.11
○Thread 294105			3.54	4.39	80.77	2.25	0.01	4.11
○Thread 294106			3.58	4.39	81.68	2.25	0.00	4.11
○Thread 294107			3.57	4.39	81.45	2.25	0.00	4.11
○Thread 294108			3.46	4.39	78.94	2.25	0.00	4.11
○Thread 294109			3.57	4.39	81.44	2.25	0.00	4.11
○Thread 294110			3.54	4.39	80.64	2.25	0.00	4.11
○Thread 294111			3.55	4.39	80.98	2.25	0.00	4.12
○Thread 294112			3.53	4.39	80.41	2.25	0.00	4.11
○Thread 294113			3.58	4.39	81.66	2.25	0.00	4.11
○Thread 294114			3.79	4.39	86.33	2.25	0.00	4.11
○Thread 294115			3.55	4.39	80.97	2.25	0.00	4.11
○Thread 294116			3.55	4.39	80.86	2.25	0.00	4.11
○Thread 294117			3.62	4.39	82.56	2.25	0.00	4.11
○Thread 294118			3.56	4.39	81.19	2.25	0.00	4.12
○Thread 294119			3.57	4.39	81.42	2.25	0.00	4.11
○Thread 294120			3.47	4.39	79.14	2.25	0.00	4.11
○Thread 294121			3.36	4.39	76.63	2.25	0.00	4.11

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	4.40
►Process 294141+		48	4.40
○Thread 294141			4.40	6.63	66.33	0.00	0.00	4.33
○Thread 294208			3.45	4.35	79.43	2.27	0.01	4.08
○Thread 294209			3.35	4.35	77.13	2.27	0.01	4.06
○Thread 294210			3.44	4.35	79.19	2.27	0.01	4.07
○Thread 294211			3.43	4.35	78.85	2.27	0.01	4.08
○Thread 294212			3.42	4.35	78.73	2.27	0.01	4.08
○Thread 294213			3.40	4.35	78.27	2.27	0.01	4.07
○Thread 294214			3.32	4.35	76.31	2.27	0.01	4.07
○Thread 294215			3.47	4.35	79.87	2.27	0.01	4.08
○Thread 294216			3.43	4.35	78.83	2.27	0.01	4.08
○Thread 294217			3.43	4.35	78.94	2.27	0.01	4.08
○Thread 294218			3.35	4.35	76.99	2.27	0.01	4.08
○Thread 294219			3.53	4.35	81.24	2.27	0.01	4.07
○Thread 294220			3.37	4.35	77.56	2.27	0.01	4.07
○Thread 294221			3.44	4.35	79.05	2.27	0.01	4.07
○Thread 294222			3.39	4.35	78.01	2.27	0.01	4.06
○Thread 294223			3.44	4.35	79.16	2.27	0.01	4.06
○Thread 294224			3.43	4.35	78.81	2.27	0.01	4.08
○Thread 294225			3.43	4.35	78.81	2.27	0.01	4.07
○Thread 294226			3.36	4.35	77.31	2.27	0.01	4.07
○Thread 294227			3.45	4.35	79.26	2.27	0.01	4.07
○Thread 294228			3.42	4.35	78.68	2.27	0.01	4.07
○Thread 294229			3.40	4.35	78.11	2.27	0.01	4.07
○Thread 294230			3.41	4.35	78.45	2.27	0.01	4.07
○Thread 294231			3.39	4.35	77.87	2.27	0.01	4.07
○Thread 294232			3.48	4.35	80.05	2.27	0.01	4.04
○Thread 294233			3.38	4.35	77.75	2.27	0.01	4.07
○Thread 294234			3.53	4.35	81.19	2.27	0.01	4.06
○Thread 294235			3.48	4.35	80.04	2.27	0.01	4.07
○Thread 294236			3.45	4.35	79.35	2.27	0.01	4.05
○Thread 294237			3.46	4.35	79.58	2.27	0.01	4.07
○Thread 294238			3.37	4.35	77.51	2.27	0.00	4.07
○Thread 294239			3.52	4.35	80.83	2.27	0.00	4.07
○Thread 294240			3.43	4.35	78.88	2.27	0.00	4.07
○Thread 294241			3.43	4.35	78.76	2.27	0.00	4.07
○Thread 294242			3.45	4.35	79.22	2.27	0.00	4.07
○Thread 294243			3.51	4.36	80.59	2.27	0.00	4.06
○Thread 294244			3.37	4.36	77.49	2.27	0.00	4.07
○Thread 294245			3.34	4.36	76.69	2.27	0.00	4.08
○Thread 294246			3.52	4.36	80.82	2.27	0.00	4.08
○Thread 294247			3.54	4.36	81.39	2.27	0.00	4.07
○Thread 294248			3.44	4.36	78.97	2.27	0.00	4.07
○Thread 294249			3.32	4.36	76.33	2.27	0.00	4.07
○Thread 294250			3.42	4.36	78.62	2.28	0.00	4.07
○Thread 294251			3.49	4.36	80.11	2.28	0.00	4.08
○Thread 294252			3.41	4.36	78.39	2.28	0.00	4.08
○Thread 294253			3.30	4.36	75.75	2.28	0.00	4.08
○Thread 294254			3.09	4.36	71.04	2.28	0.00	4.07

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	4.39
►Process 294275+		56	4.39
○Thread 294275			4.39	6.68	65.80	0.00	0.00	4.39
○Thread 294343			3.39	4.40	76.96	2.26	0.02	4.13
○Thread 294344			3.38	4.40	76.73	2.26	0.01	4.13
○Thread 294345			3.40	4.41	77.18	2.26	0.01	4.13
○Thread 294346			3.36	4.41	76.27	2.26	0.01	4.13
○Thread 294347			3.44	4.41	78.09	2.26	0.01	4.13
○Thread 294348			3.32	4.41	75.36	2.26	0.01	4.13
○Thread 294349			3.31	4.41	75.13	2.26	0.01	4.13
○Thread 294350			3.42	4.41	77.62	2.26	0.01	4.13
○Thread 294351			3.43	4.41	77.96	2.26	0.01	4.13
○Thread 294352			3.26	4.41	74.10	2.26	0.01	4.13
○Thread 294353			3.33	4.41	75.69	2.26	0.01	4.12
○Thread 294354			3.46	4.41	78.63	2.26	0.01	4.13
○Thread 294355			3.43	4.41	77.84	2.26	0.01	4.13
○Thread 294356			3.39	4.41	76.93	2.26	0.01	4.13
○Thread 294357			3.35	4.41	76.02	2.26	0.01	4.12
○Thread 294358			3.45	4.41	78.28	2.26	0.01	4.13
○Thread 294359			3.36	4.41	76.24	2.26	0.01	4.13
○Thread 294360			3.43	4.41	77.94	2.26	0.01	4.13
○Thread 294361			3.29	4.41	74.64	2.26	0.01	4.12
○Thread 294362			3.43	4.41	77.82	2.26	0.01	4.13
○Thread 294363			3.36	4.41	76.34	2.26	0.01	4.12
○Thread 294364			3.42	4.41	77.58	2.26	0.01	4.13
○Thread 294365			3.40	4.41	77.13	2.26	0.01	4.13
○Thread 294366			3.41	4.41	77.47	2.26	0.01	4.13
○Thread 294367			3.39	4.41	76.90	2.26	0.01	4.13
○Thread 294368			3.31	4.41	75.19	2.26	0.01	4.13
○Thread 294369			3.29	4.41	74.74	2.26	0.01	4.13
○Thread 294370			3.39	4.41	76.89	2.26	0.01	4.12
○Thread 294371			3.41	4.41	77.34	2.26	0.01	4.13
○Thread 294372			3.34	4.41	75.86	2.26	0.01	4.13
○Thread 294373			3.42	4.41	77.67	2.26	0.01	4.13
○Thread 294374			3.42	4.41	77.56	2.26	0.01	4.13
○Thread 294375			3.37	4.41	76.42	2.26	0.01	4.13
○Thread 294376			3.27	4.41	74.26	2.26	0.01	4.13
○Thread 294377			3.45	4.41	78.23	2.26	0.01	4.13
○Thread 294378			3.42	4.41	77.66	2.26	0.01	4.13
○Thread 294379			3.34	4.41	75.84	2.26	0.01	4.13
○Thread 294380			3.46	4.41	78.45	2.26	0.01	4.13
○Thread 294381			3.35	4.41	75.95	2.26	0.01	4.12
○Thread 294382			3.32	4.41	75.38	2.26	0.00	4.13
○Thread 294383			3.30	4.41	74.93	2.26	0.00	4.12
○Thread 294384			3.31	4.41	75.04	2.26	0.00	4.13
○Thread 294385			3.34	4.41	75.71	2.26	0.00	4.13
○Thread 294386			3.41	4.41	77.30	2.26	0.00	4.13
○Thread 294387			3.36	4.41	76.28	2.26	0.00	4.13
○Thread 294388			3.40	4.41	77.18	2.26	0.00	4.13
○Thread 294389			3.41	4.41	77.29	2.26	0.00	4.13
○Thread 294390			3.74	4.41	84.88	2.26	0.00	4.13
○Thread 294391			3.28	4.41	74.34	2.26	0.00	4.13
○Thread 294392			3.40	4.41	77.17	2.26	0.00	4.13
○Thread 294393			3.46	4.41	78.41	2.26	0.00	4.13
○Thread 294394			3.35	4.41	75.92	2.26	0.00	4.13
○Thread 294395			3.35	4.41	76.03	2.26	0.00	4.13
○Thread 294396			3.21	4.41	72.74	2.26	0.00	4.12
○Thread 294397			3.25	4.41	73.64	2.27	0.00	4.13

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	4.47
►Process 294418+		64	4.47
○Thread 294418			4.47	6.80	65.74	0.00	0.00	4.50
○Thread 294485			3.49	4.53	77.10	2.25	0.02	4.25
○Thread 294486			3.44	4.53	75.99	2.25	0.02	4.25
○Thread 294487			3.52	4.53	77.64	2.25	0.02	4.26
○Thread 294488			3.44	4.53	75.88	2.25	0.02	4.25
○Thread 294489			3.40	4.53	74.99	2.25	0.02	4.24
○Thread 294490			3.41	4.53	75.32	2.25	0.02	4.24
○Thread 294491			3.46	4.53	76.42	2.25	0.02	4.24
○Thread 294492			3.61	4.53	79.72	2.25	0.02	4.25
○Thread 294493			3.32	4.53	73.22	2.25	0.02	4.25
○Thread 294494			3.43	4.53	75.75	2.25	0.02	4.26
○Thread 294495			3.47	4.53	76.52	2.25	0.02	4.24
○Thread 294496			3.52	4.53	77.62	2.25	0.01	4.26
○Thread 294497			3.46	4.54	76.40	2.25	0.01	4.24
○Thread 294498			3.35	4.54	73.98	2.25	0.01	4.26
○Thread 294499			3.39	4.54	74.74	2.25	0.01	4.26
○Thread 294500			3.47	4.54	76.51	2.25	0.01	4.24
○Thread 294501			3.26	4.54	71.87	2.25	0.01	4.26
○Thread 294502			3.35	4.54	73.85	2.25	0.01	4.26
○Thread 294503			3.36	4.54	74.18	2.25	0.01	4.24
○Thread 294504			3.40	4.54	75.06	2.25	0.01	4.26
○Thread 294505			3.25	4.54	71.75	2.25	0.01	4.26
○Thread 294506			3.40	4.54	74.95	2.25	0.01	4.25
○Thread 294507			3.47	4.54	76.49	2.25	0.01	4.25
○Thread 294508			3.43	4.54	75.71	2.25	0.01	4.26
○Thread 294509			3.26	4.54	71.96	2.25	0.01	4.25
○Thread 294510			3.33	4.54	73.39	2.25	0.01	4.26
○Thread 294511			3.49	4.54	77.03	2.25	0.01	4.25
○Thread 294512			3.46	4.54	76.36	2.25	0.01	4.26
○Thread 294513			3.38	4.54	74.60	2.25	0.01	4.24
○Thread 294514			3.36	4.54	74.04	2.25	0.01	4.25
○Thread 294515			3.43	4.54	75.69	2.25	0.01	4.25
○Thread 294516			3.50	4.54	77.23	2.25	0.01	4.26
○Thread 294517			3.38	4.54	74.48	2.25	0.01	4.24
○Thread 294518			3.41	4.54	75.24	2.25	0.01	4.24
○Thread 294519			3.38	4.54	74.47	2.25	0.01	4.25
○Thread 294520			3.37	4.54	74.25	2.25	0.01	4.25
○Thread 294521			3.36	4.54	74.02	2.25	0.01	4.26
○Thread 294522			3.40	4.54	74.90	2.25	0.01	4.24
○Thread 294523			3.37	4.54	74.24	2.25	0.01	4.26
○Thread 294524			3.43	4.54	75.67	2.25	0.01	4.25
○Thread 294525			3.31	4.54	73.02	2.25	0.01	4.25
○Thread 294526			3.46	4.54	76.21	2.25	0.01	4.26
○Thread 294527			3.31	4.54	73.02	2.25	0.01	4.26
○Thread 294528			3.45	4.54	76.10	2.25	0.01	4.25
○Thread 294529			3.41	4.54	75.21	2.25	0.01	4.24
○Thread 294530			3.43	4.54	75.54	2.25	0.01	4.25
○Thread 294531			3.34	4.54	73.67	2.25	0.01	4.26
○Thread 294532			3.62	4.54	79.83	2.25	0.00	4.24
○Thread 294533			3.37	4.54	74.32	2.25	0.00	4.24
○Thread 294534			3.43	4.54	75.64	2.25	0.00	4.26
○Thread 294535			3.43	4.54	75.64	2.25	0.00	4.26
○Thread 294536			3.40	4.54	74.86	2.25	0.00	4.24
○Thread 294537			3.41	4.54	75.08	2.25	0.00	4.25
○Thread 294538			3.42	4.54	75.30	2.25	0.00	4.25
○Thread 294539			3.33	4.54	73.31	2.25	0.00	4.24
○Thread 294540			3.44	4.54	75.84	2.25	0.00	4.26
○Thread 294541			3.41	4.54	75.07	2.25	0.00	4.24
○Thread 294542			3.38	4.54	74.52	2.26	0.00	4.24
○Thread 294543			3.41	4.54	75.18	2.26	0.00	4.24
○Thread 294544			3.51	4.54	77.26	2.26	0.00	4.26
○Thread 294545			3.36	4.54	73.96	2.26	0.00	4.26
○Thread 294546			3.42	4.54	75.39	2.26	0.00	4.25
○Thread 294547			3.42	4.54	75.39	2.26	0.00	4.24

Report Configuration