OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	36.06
►Process 271205+		1	36.06
○Thread 271205			36.06	37.76	95.51	0.00	0.00	37.74

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	19.30
►Process 271342+		2	19.30
○Thread 271342			19.30	21.10	91.48	0.00	0.00	18.84
○Thread 271409			18.50	18.85	98.16	2.25	0.00	18.58

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	10.70
►Process 271429+		4	10.70
○Thread 271429			10.70	12.45	85.95	0.00	0.00	10.20
○Thread 271496			9.78	10.21	95.83	2.24	0.00	9.94
○Thread 271497			9.75	10.21	95.48	2.24	0.00	9.94
○Thread 271498			9.80	10.21	96.02	2.24	0.00	9.94

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	6.46
►Process 271518+		8	6.46
○Thread 271518			6.46	8.26	78.20	0.00	0.00	6.02
○Thread 271586			5.59	6.02	92.80	2.23	0.00	5.76
○Thread 271587			5.55	6.02	92.13	2.23	0.00	5.76
○Thread 271588			5.54	6.02	91.96	2.23	0.00	5.76
○Thread 271589			5.57	6.02	92.46	2.23	0.00	5.76
○Thread 271590			5.48	6.02	90.88	2.23	0.00	5.76
○Thread 271591			5.57	6.02	92.45	2.23	0.00	5.76
○Thread 271592			5.49	6.02	91.12	2.23	0.00	5.75

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	4.75
►Process 271663+		16	4.75
○Thread 271663			4.75	6.66	71.24	0.00	0.00	4.41
○Thread 271730			3.77	4.42	85.40	2.24	0.00	4.15
○Thread 271731			3.82	4.42	86.42	2.24	0.00	4.15
○Thread 271732			3.83	4.42	86.76	2.24	0.00	4.15
○Thread 271733			3.86	4.42	87.43	2.24	0.00	4.15
○Thread 271734			3.81	4.42	86.18	2.24	0.00	4.15
○Thread 271735			3.80	4.42	86.07	2.24	0.00	4.15
○Thread 271736			3.83	4.42	86.74	2.24	0.00	4.15
○Thread 271737			3.89	4.42	87.98	2.24	0.00	4.15
○Thread 271738			3.78	4.42	85.61	2.24	0.00	4.15
○Thread 271739			3.88	4.42	87.86	2.24	0.00	4.15
○Thread 271740			3.76	4.42	85.03	2.24	0.00	4.15
○Thread 271741			3.87	4.42	87.52	2.24	0.00	4.15
○Thread 271742			3.81	4.42	86.16	2.24	0.00	4.15
○Thread 271743			3.83	4.42	86.61	2.24	0.00	4.16
○Thread 271744			3.83	4.42	86.72	2.24	0.00	4.15

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	4.48
►Process 271765+		24	4.48
○Thread 271765			4.48	6.47	69.22	0.00	0.00	4.21
○Thread 271833			3.53	4.22	83.76	2.25	0.01	3.95
○Thread 271834			3.55	4.22	84.12	2.25	0.01	3.95
○Thread 271835			3.55	4.22	84.23	2.25	0.01	3.95
○Thread 271836			3.60	4.22	85.41	2.25	0.01	3.95
○Thread 271837			3.60	4.22	85.29	2.25	0.01	3.95
○Thread 271838			3.54	4.22	83.99	2.25	0.01	3.95
○Thread 271839			3.51	4.22	83.15	2.25	0.00	3.95
○Thread 271840			3.56	4.22	84.33	2.25	0.00	3.95
○Thread 271841			3.52	4.22	83.38	2.25	0.00	3.95
○Thread 271842			3.52	4.22	83.50	2.25	0.00	3.95
○Thread 271843			3.58	4.22	84.92	2.25	0.00	3.95
○Thread 271844			3.63	4.22	86.10	2.25	0.00	3.95
○Thread 271845			3.48	4.22	82.42	2.25	0.00	3.95
○Thread 271846			3.60	4.22	85.38	2.25	0.00	3.95
○Thread 271847			3.50	4.22	83.01	2.25	0.00	3.95
○Thread 271848			3.64	4.22	86.20	2.25	0.00	3.95
○Thread 271849			3.58	4.22	84.78	2.25	0.00	3.95
○Thread 271850			3.56	4.22	84.42	2.25	0.00	3.95
○Thread 271851			3.54	4.22	83.83	2.25	0.00	3.95
○Thread 271852			3.59	4.22	85.01	2.25	0.00	3.95
○Thread 271853			3.56	4.22	84.41	2.25	0.00	3.95
○Thread 271854			3.55	4.22	84.05	2.25	0.00	3.95
○Thread 271855			3.53	4.22	83.70	2.25	0.00	3.95

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	4.29
►Process 271875+		32	4.29
○Thread 271875			4.29	6.32	67.97	0.00	0.00	4.08
○Thread 271942			3.29	4.09	80.48	2.22	0.01	3.82
○Thread 271943			3.38	4.09	82.56	2.22	0.01	3.83
○Thread 271944			3.35	4.09	81.94	2.22	0.01	3.83
○Thread 271945			3.33	4.09	81.45	2.22	0.01	3.82
○Thread 271946			3.36	4.09	82.18	2.22	0.01	3.82
○Thread 271947			3.37	4.09	82.42	2.22	0.01	3.82
○Thread 271948			3.37	4.09	82.42	2.22	0.01	3.82
○Thread 271949			3.37	4.10	82.29	2.22	0.01	3.82
○Thread 271950			3.30	4.10	80.58	2.22	0.01	3.83
○Thread 271951			3.38	4.10	82.53	2.22	0.01	3.82
○Thread 271952			3.34	4.10	81.67	2.22	0.01	3.83
○Thread 271953			3.41	4.10	83.38	2.22	0.01	3.82
○Thread 271954			3.36	4.10	82.03	2.22	0.01	3.82
○Thread 271955			3.31	4.10	80.81	2.22	0.01	3.82
○Thread 271956			3.43	4.10	83.74	2.22	0.00	3.82
○Thread 271957			3.54	4.10	86.54	2.22	0.00	3.82
○Thread 271958			3.30	4.10	80.56	2.22	0.00	3.82
○Thread 271959			3.44	4.10	84.09	2.22	0.00	3.82
○Thread 271960			3.37	4.10	82.26	2.22	0.00	3.82
○Thread 271961			3.32	4.10	81.16	2.22	0.00	3.82
○Thread 271962			3.32	4.10	81.03	2.22	0.00	3.82
○Thread 271963			3.37	4.10	82.37	2.22	0.00	3.83
○Thread 271964			3.37	4.10	82.25	2.22	0.00	3.82
○Thread 271965			3.32	4.10	81.02	2.22	0.00	3.82
○Thread 271966			3.34	4.10	81.63	2.22	0.00	3.82
○Thread 271967			3.28	4.10	80.17	2.22	0.00	3.82
○Thread 271968			3.39	4.10	82.72	2.22	0.00	3.83
○Thread 271969			3.38	4.10	82.48	2.22	0.00	3.82
○Thread 271970			3.26	4.10	79.67	2.22	0.00	3.83
○Thread 271971			3.32	4.10	81.13	2.22	0.00	3.83
○Thread 271972			3.38	4.10	82.59	2.22	0.00	3.83

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	4.17
►Process 271992+		40	4.17
○Thread 271992			4.17	6.26	66.63	0.00	0.00	4.01
○Thread 272059			3.25	4.02	80.76	2.22	0.01	3.74
○Thread 272060			3.25	4.02	80.76	2.22	0.01	3.75
○Thread 272061			3.24	4.02	80.51	2.22	0.01	3.75
○Thread 272062			3.20	4.02	79.64	2.22	0.01	3.74
○Thread 272063			3.30	4.02	82.12	2.22	0.01	3.75
○Thread 272064			3.11	4.02	77.27	2.22	0.01	3.75
○Thread 272065			3.18	4.03	79.13	2.22	0.01	3.75
○Thread 272066			3.22	4.03	80.00	2.22	0.01	3.75
○Thread 272067			3.20	4.03	79.62	2.22	0.01	3.75
○Thread 272068			3.22	4.03	80.11	2.22	0.01	3.75
○Thread 272069			3.25	4.03	80.86	2.22	0.01	3.75
○Thread 272070			3.37	4.03	83.84	2.22	0.01	3.75
○Thread 272071			3.22	4.03	79.98	2.22	0.01	3.75
○Thread 272072			3.17	4.03	78.86	2.23	0.01	3.75
○Thread 272073			3.19	4.03	79.36	2.23	0.01	3.74
○Thread 272074			3.24	4.03	80.47	2.23	0.01	3.75
○Thread 272075			3.25	4.03	80.71	2.23	0.01	3.75
○Thread 272076			3.21	4.03	79.72	2.23	0.01	3.74
○Thread 272077			3.15	4.03	78.22	2.23	0.01	3.74
○Thread 272078			3.24	4.03	80.46	2.23	0.01	3.75
○Thread 272079			3.23	4.03	80.33	2.23	0.01	3.74
○Thread 272080			3.21	4.03	79.83	2.23	0.01	3.75
○Thread 272081			3.23	4.03	80.20	2.23	0.00	3.74
○Thread 272082			3.24	4.03	80.57	2.23	0.00	3.75
○Thread 272083			3.12	4.03	77.46	2.23	0.00	3.75
○Thread 272084			3.15	4.03	78.21	2.23	0.00	3.75
○Thread 272085			3.17	4.03	78.70	2.23	0.00	3.74
○Thread 272086			3.16	4.03	78.57	2.23	0.00	3.75
○Thread 272087			3.17	4.03	78.82	2.23	0.00	3.74
○Thread 272088			3.19	4.03	79.31	2.23	0.00	3.75
○Thread 272089			3.28	4.03	81.54	2.23	0.00	3.74
○Thread 272090			3.47	4.03	86.13	2.23	0.00	3.74
○Thread 272091			3.17	4.03	78.68	2.23	0.00	3.75
○Thread 272092			3.22	4.03	79.92	2.23	0.00	3.74
○Thread 272093			3.35	4.03	83.14	2.23	0.00	3.74
○Thread 272094			3.32	4.03	82.40	2.23	0.00	3.75
○Thread 272095			3.20	4.03	79.41	2.23	0.00	3.75
○Thread 272096			3.16	4.03	78.54	2.23	0.00	3.75
○Thread 272097			3.03	4.03	75.19	2.23	0.00	3.74

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	4.12
►Process 272167+		48	4.12
○Thread 272167			4.12	6.21	66.46	0.00	0.00	3.95
○Thread 272234			3.11	3.97	78.53	2.23	0.01	3.67
○Thread 272235			3.11	3.97	78.40	2.23	0.01	3.69
○Thread 272236			3.08	3.97	77.77	2.23	0.01	3.70
○Thread 272237			3.05	3.97	76.88	2.23	0.01	3.70
○Thread 272238			3.03	3.97	76.50	2.23	0.01	3.69
○Thread 272239			3.05	3.97	77.00	2.23	0.01	3.69
○Thread 272240			3.07	3.97	77.50	2.23	0.01	3.70
○Thread 272241			3.08	3.97	77.63	2.23	0.01	3.69
○Thread 272242			3.06	3.97	77.24	2.23	0.01	3.69
○Thread 272243			3.15	3.97	79.51	2.23	0.01	3.69
○Thread 272244			3.09	3.97	77.87	2.23	0.01	3.69
○Thread 272245			3.17	3.97	79.88	2.23	0.01	3.69
○Thread 272246			2.98	3.97	75.09	2.23	0.01	3.69
○Thread 272247			3.03	3.97	76.47	2.23	0.01	3.69
○Thread 272248			3.07	3.97	77.35	2.23	0.01	3.70
○Thread 272249			3.23	3.97	81.51	2.23	0.01	3.69
○Thread 272250			3.05	3.97	76.97	2.23	0.01	3.69
○Thread 272251			3.12	3.97	78.73	2.23	0.01	3.69
○Thread 272252			3.04	3.97	76.58	2.23	0.01	3.70
○Thread 272253			3.17	3.97	79.86	2.23	0.01	3.70
○Thread 272254			3.02	3.97	76.07	2.23	0.01	3.70
○Thread 272255			3.07	3.97	77.46	2.23	0.01	3.69
○Thread 272256			3.03	3.97	76.32	2.23	0.01	3.69
○Thread 272257			3.19	3.97	80.35	2.23	0.01	3.70
○Thread 272258			3.01	3.97	75.94	2.23	0.01	3.69
○Thread 272259			2.99	3.97	75.30	2.23	0.01	3.69
○Thread 272260			3.04	3.97	76.69	2.23	0.01	3.69
○Thread 272261			3.02	3.97	76.18	2.23	0.01	3.69
○Thread 272262			3.05	3.97	76.81	2.23	0.01	3.69
○Thread 272263			3.04	3.97	76.55	2.23	0.01	3.69
○Thread 272264			3.07	3.97	77.43	2.23	0.01	3.69
○Thread 272265			3.12	3.97	78.56	2.23	0.01	3.70
○Thread 272266			2.98	3.97	75.03	2.23	0.01	3.69
○Thread 272267			3.09	3.97	77.92	2.23	0.00	3.69
○Thread 272268			3.09	3.97	77.79	2.23	0.00	3.70
○Thread 272269			3.11	3.97	78.42	2.23	0.00	3.69
○Thread 272270			3.14	3.97	79.05	2.23	0.00	3.69
○Thread 272271			3.05	3.97	76.90	2.23	0.00	3.69
○Thread 272272			3.04	3.97	76.65	2.23	0.00	3.69
○Thread 272273			3.14	3.97	79.16	2.23	0.00	3.70
○Thread 272274			3.14	3.97	79.03	2.23	0.00	3.68
○Thread 272275			3.04	3.97	76.64	2.23	0.00	3.69
○Thread 272276			3.14	3.97	79.15	2.23	0.00	3.69
○Thread 272277			3.06	3.97	77.14	2.23	0.00	3.69
○Thread 272278			3.10	3.97	78.01	2.23	0.00	3.70
○Thread 272279			3.05	3.97	76.88	2.23	0.00	3.69
○Thread 272280			2.79	3.97	70.33	2.23	0.00	3.69

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	4.11
►Process 272300+		56	4.11
○Thread 272300			4.11	6.33	65.00	0.00	0.00	4.05
○Thread 272367			3.12	4.08	76.50	2.24	0.02	3.81
○Thread 272368			3.10	4.08	76.13	2.24	0.02	3.81
○Thread 272369			3.02	4.08	74.16	2.24	0.02	3.80
○Thread 272370			3.14	4.08	77.10	2.24	0.02	3.81
○Thread 272371			3.05	4.08	74.76	2.24	0.01	3.80
○Thread 272372			3.09	4.08	75.86	2.24	0.01	3.80
○Thread 272373			3.14	4.08	76.96	2.24	0.01	3.79
○Thread 272374			3.21	4.08	78.67	2.24	0.01	3.80
○Thread 272375			3.02	4.08	74.14	2.24	0.01	3.79
○Thread 272376			3.08	4.08	75.60	2.24	0.01	3.81
○Thread 272377			3.11	4.08	76.21	2.24	0.01	3.80
○Thread 272378			3.12	4.08	76.45	2.24	0.01	3.80
○Thread 272379			3.09	4.08	75.84	2.24	0.01	3.80
○Thread 272380			3.11	4.08	76.33	2.24	0.01	3.80
○Thread 272381			2.99	4.08	73.26	2.24	0.01	3.81
○Thread 272382			3.13	4.08	76.69	2.24	0.01	3.81
○Thread 272383			3.02	4.08	74.11	2.24	0.01	3.78
○Thread 272384			3.04	4.08	74.60	2.24	0.01	3.80
○Thread 272385			3.18	4.08	77.90	2.24	0.01	3.79
○Thread 272386			3.03	4.08	74.23	2.24	0.01	3.81
○Thread 272387			3.11	4.08	76.18	2.24	0.01	3.80
○Thread 272388			3.08	4.08	75.45	2.24	0.01	3.81
○Thread 272389			3.13	4.08	76.67	2.24	0.01	3.79
○Thread 272390			3.14	4.08	77.03	2.24	0.01	3.80
○Thread 272391			3.04	4.08	74.46	2.24	0.01	3.81
○Thread 272392			3.17	4.08	77.76	2.24	0.01	3.79
○Thread 272393			3.10	4.08	76.04	2.24	0.01	3.81
○Thread 272394			3.05	4.08	74.69	2.24	0.01	3.79
○Thread 272395			3.12	4.08	76.41	2.24	0.01	3.81
○Thread 272396			2.97	4.08	72.73	2.24	0.01	3.81
○Thread 272397			3.13	4.08	76.65	2.24	0.01	3.78
○Thread 272398			3.12	4.08	76.40	2.24	0.01	3.80
○Thread 272399			3.06	4.08	75.05	2.24	0.01	3.79
○Thread 272400			3.02	4.08	74.07	2.24	0.01	3.80
○Thread 272401			3.07	4.08	75.16	2.24	0.01	3.79
○Thread 272402			3.17	4.08	77.61	2.24	0.01	3.81
○Thread 272403			3.14	4.08	76.87	2.24	0.01	3.81
○Thread 272404			3.09	4.08	75.64	2.24	0.01	3.79
○Thread 272405			3.04	4.09	74.42	2.24	0.01	3.80
○Thread 272406			3.11	4.09	76.13	2.24	0.00	3.81
○Thread 272407			3.06	4.09	75.02	2.24	0.00	3.79
○Thread 272408			3.20	4.09	78.33	2.24	0.00	3.80
○Thread 272409			3.16	4.09	77.34	2.24	0.00	3.80
○Thread 272410			3.15	4.09	77.22	2.24	0.00	3.81
○Thread 272411			3.09	4.09	75.62	2.24	0.00	3.81
○Thread 272412			2.93	4.09	71.83	2.24	0.00	3.80
○Thread 272413			2.95	4.09	72.19	2.24	0.00	3.80
○Thread 272414			3.40	4.09	83.20	2.24	0.00	3.79
○Thread 272415			3.19	4.09	78.18	2.24	0.00	3.79
○Thread 272416			3.22	4.09	78.91	2.24	0.00	3.81
○Thread 272417			3.23	4.09	79.15	2.24	0.00	3.80
○Thread 272418			3.14	4.09	76.95	2.24	0.00	3.80
○Thread 272419			3.09	4.09	75.72	2.24	0.00	3.81
○Thread 272420			2.92	4.09	71.56	2.24	0.00	3.79
○Thread 272421			2.98	4.09	72.90	2.24	0.00	3.81

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	4.28
►Process 272441+		64	4.28
○Thread 272441			4.28	6.51	65.70	0.00	0.00	4.25
○Thread 272508			3.15	4.26	73.89	2.23	0.02	3.99
○Thread 272509			3.18	4.26	74.59	2.23	0.02	3.98
○Thread 272510			3.13	4.26	73.42	2.23	0.02	3.99
○Thread 272511			3.18	4.26	74.71	2.23	0.02	3.98
○Thread 272512			3.03	4.26	71.07	2.23	0.02	3.99
○Thread 272513			3.22	4.26	75.64	2.23	0.02	3.99
○Thread 272514			3.16	4.26	74.23	2.23	0.02	3.99
○Thread 272515			3.22	4.26	75.52	2.23	0.02	3.99
○Thread 272516			3.21	4.26	75.40	2.24	0.02	3.99
○Thread 272517			3.16	4.26	74.10	2.24	0.01	3.99
○Thread 272518			3.15	4.26	73.87	2.24	0.01	3.98
○Thread 272519			3.33	4.26	78.08	2.24	0.01	3.99
○Thread 272520			3.16	4.26	74.21	2.24	0.01	3.99
○Thread 272521			3.14	4.26	73.74	2.24	0.01	3.99
○Thread 272522			3.17	4.27	74.33	2.24	0.01	3.99
○Thread 272523			3.24	4.27	75.96	2.24	0.01	3.98
○Thread 272524			3.09	4.27	72.44	2.24	0.01	3.99
○Thread 272525			3.18	4.27	74.67	2.24	0.01	3.99
○Thread 272526			3.20	4.27	75.02	2.24	0.01	3.98
○Thread 272527			3.12	4.27	73.26	2.24	0.01	3.99
○Thread 272528			3.15	4.27	73.84	2.24	0.01	3.98
○Thread 272529			3.20	4.27	75.13	2.24	0.01	3.99
○Thread 272530			3.24	4.27	75.95	2.24	0.01	3.99
○Thread 272531			3.18	4.27	74.65	2.24	0.01	3.99
○Thread 272532			3.16	4.27	74.06	2.24	0.01	3.98
○Thread 272533			3.13	4.27	73.36	2.24	0.01	3.98
○Thread 272534			3.12	4.27	73.12	2.24	0.01	3.99
○Thread 272535			3.14	4.27	73.59	2.24	0.01	3.98
○Thread 272536			3.10	4.27	72.65	2.24	0.01	3.99
○Thread 272537			3.19	4.27	74.76	2.24	0.01	3.99
○Thread 272538			3.17	4.27	74.40	2.24	0.01	3.98
○Thread 272539			3.27	4.27	76.74	2.24	0.01	3.98
○Thread 272540			3.16	4.27	74.04	2.24	0.01	3.99
○Thread 272541			3.14	4.27	73.57	2.24	0.01	3.98
○Thread 272542			3.12	4.27	73.10	2.24	0.01	3.97
○Thread 272543			3.28	4.27	76.85	2.24	0.01	3.98
○Thread 272544			3.07	4.27	71.93	2.24	0.01	3.99
○Thread 272545			3.13	4.27	73.33	2.24	0.01	3.98
○Thread 272546			3.27	4.27	76.61	2.24	0.01	3.98
○Thread 272547			3.23	4.27	75.67	2.24	0.01	3.99
○Thread 272548			3.15	4.27	73.79	2.24	0.01	3.99
○Thread 272549			3.02	4.27	70.74	2.24	0.01	3.98
○Thread 272550			3.17	4.27	74.25	2.24	0.01	3.99
○Thread 272551			3.22	4.27	75.54	2.24	0.01	3.98
○Thread 272552			3.09	4.27	72.37	2.24	0.01	3.98
○Thread 272553			3.13	4.27	73.42	2.24	0.01	3.96
○Thread 272554			3.08	4.27	72.13	2.24	0.01	3.98
○Thread 272555			3.41	4.27	79.86	2.24	0.00	3.98
○Thread 272556			3.23	4.27	75.76	2.24	0.00	3.98
○Thread 272557			3.21	4.27	75.29	2.24	0.00	3.98
○Thread 272558			3.30	4.27	77.27	2.24	0.00	3.98
○Thread 272559			3.13	4.27	73.29	2.24	0.00	3.99
○Thread 272560			3.08	4.27	72.12	2.24	0.00	3.98
○Thread 272561			3.06	4.27	71.76	2.24	0.00	3.99
○Thread 272562			3.10	4.27	72.58	2.24	0.00	3.99
○Thread 272563			3.31	4.27	77.49	2.24	0.00	3.99
○Thread 272564			3.05	4.27	71.41	2.24	0.00	3.99
○Thread 272565			3.19	4.27	74.80	2.24	0.00	3.99
○Thread 272566			3.20	4.27	74.91	2.24	0.00	3.98
○Thread 272567			3.35	4.27	78.54	2.24	0.00	3.99
○Thread 272568			3.17	4.27	74.21	2.24	0.00	3.98
○Thread 272569			3.19	4.27	74.67	2.24	0.00	3.98
○Thread 272570			3.36	4.27	78.65	2.24	0.00	3.99

Report Configuration