OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	64.17
►Process 841667+		1	64.17
○Thread 841667			64.17	65.64	97.76	0.00	0.00	65.63

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	32.29
►Process 841834+		2	32.29
○Thread 841834			32.29	33.86	95.39	0.00	0.00	32.04
○Thread 841934			31.75	32.04	99.09	1.81	0.00	31.80

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	16.59
►Process 842004+		4	16.59
○Thread 842004			16.59	18.07	91.82	0.00	0.00	16.26
○Thread 842103			15.99	16.26	98.31	1.81	0.00	16.03
○Thread 842104			16.00	16.26	98.40	1.81	0.00	16.03
○Thread 842105			15.98	16.26	98.24	1.81	0.00	16.02

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	8.67
►Process 842127+		8	8.67
○Thread 842127			8.67	10.20	84.98	0.00	0.00	8.37
○Thread 842226			8.13	8.38	97.04	1.82	0.00	8.14
○Thread 842227			8.12	8.38	96.91	1.82	0.00	8.14
○Thread 842228			8.10	8.38	96.67	1.82	0.00	8.14
○Thread 842229			8.10	8.38	96.67	1.82	0.00	8.14
○Thread 842230			8.08	8.38	96.43	1.82	0.00	8.14
○Thread 842231			8.10	8.38	96.61	1.82	0.00	8.14
○Thread 842232			8.12	8.38	96.84	1.82	0.00	8.14

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	4.83
►Process 842253+		16	4.83
○Thread 842253			4.83	6.30	76.62	0.00	0.00	4.46
○Thread 842352			4.17	4.47	93.41	1.83	0.01	4.23
○Thread 842353			4.16	4.47	93.07	1.83	0.01	4.23
○Thread 842354			4.17	4.47	93.40	1.83	0.01	4.23
○Thread 842355			4.18	4.47	93.51	1.83	0.00	4.22
○Thread 842356			4.21	4.47	94.18	1.83	0.00	4.23
○Thread 842357			4.20	4.47	93.95	1.83	0.00	4.23
○Thread 842358			4.19	4.47	93.83	1.83	0.00	4.23
○Thread 842359			4.17	4.47	93.38	1.83	0.00	4.23
○Thread 842360			4.19	4.47	93.82	1.83	0.00	4.23
○Thread 842361			4.19	4.47	93.82	1.83	0.00	4.23
○Thread 842362			4.17	4.47	93.25	1.83	0.00	4.21
○Thread 842363			4.20	4.47	93.92	1.83	0.00	4.23
○Thread 842364			4.17	4.47	93.36	1.83	0.00	4.22
○Thread 842365			4.16	4.47	93.02	1.83	0.00	4.23
○Thread 842366			4.18	4.47	93.57	1.83	0.00	4.23

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	3.70
►Process 842434+		24	3.70
○Thread 842434			3.70	5.22	70.91	0.00	0.00	3.38
○Thread 842533			3.08	3.39	90.92	1.82	0.01	3.14
○Thread 842534			3.11	3.39	91.80	1.82	0.01	3.15
○Thread 842535			3.06	3.39	90.31	1.82	0.01	3.15
○Thread 842536			3.12	3.39	92.08	1.82	0.01	3.15
○Thread 842537			3.12	3.39	92.22	1.82	0.01	3.15
○Thread 842538			3.09	3.39	91.18	1.82	0.01	3.15
○Thread 842539			3.12	3.39	92.06	1.82	0.01	3.15
○Thread 842540			3.10	3.39	91.61	1.82	0.01	3.15
○Thread 842541			3.12	3.39	92.05	1.82	0.01	3.15
○Thread 842542			3.12	3.39	92.04	1.82	0.01	3.15
○Thread 842543			3.12	3.39	92.18	1.82	0.01	3.15
○Thread 842544			3.10	3.39	91.59	1.82	0.00	3.15
○Thread 842545			3.10	3.39	91.43	1.82	0.00	3.15
○Thread 842546			3.11	3.39	91.72	1.82	0.00	3.15
○Thread 842547			3.09	3.39	91.13	1.82	0.00	3.14
○Thread 842548			3.11	3.39	91.86	1.82	0.00	3.15
○Thread 842549			3.10	3.39	91.56	1.82	0.00	3.15
○Thread 842550			3.11	3.39	91.70	1.82	0.00	3.15
○Thread 842551			3.10	3.39	91.40	1.82	0.00	3.15
○Thread 842552			3.09	3.39	91.24	1.82	0.00	3.14
○Thread 842553			3.09	3.39	91.09	1.82	0.00	3.15
○Thread 842554			3.10	3.39	91.38	1.82	0.00	3.15
○Thread 842555			3.05	3.39	89.90	1.82	0.00	3.15

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	3.00
►Process 842575+		32	3.00
○Thread 842575			3.00	4.61	65.19	0.00	0.00	2.78
○Thread 842674			2.46	2.78	88.61	1.82	0.01	2.53
○Thread 842675			2.50	2.78	90.04	1.82	0.01	2.54
○Thread 842676			2.49	2.78	89.49	1.82	0.01	2.54
○Thread 842677			2.49	2.78	89.66	1.82	0.01	2.53
○Thread 842678			2.48	2.78	89.12	1.82	0.01	2.53
○Thread 842679			2.51	2.78	90.19	1.82	0.01	2.54
○Thread 842680			2.49	2.78	89.64	1.82	0.01	2.54
○Thread 842681			2.50	2.78	89.81	1.82	0.01	2.54
○Thread 842682			2.48	2.78	89.27	1.82	0.01	2.54
○Thread 842683			2.50	2.78	89.98	1.82	0.01	2.54
○Thread 842684			2.50	2.78	89.79	1.82	0.01	2.54
○Thread 842685			2.48	2.78	89.07	1.82	0.01	2.53
○Thread 842686			2.52	2.78	90.50	1.82	0.01	2.54
○Thread 842687			2.51	2.78	90.13	1.82	0.01	2.54
○Thread 842688			2.50	2.79	89.94	1.82	0.01	2.54
○Thread 842689			2.51	2.79	90.30	1.82	0.01	2.54
○Thread 842690			2.48	2.79	89.21	1.82	0.01	2.54
○Thread 842691			2.50	2.79	89.92	1.82	0.01	2.54
○Thread 842692			2.49	2.79	89.38	1.82	0.00	2.54
○Thread 842693			2.51	2.79	90.09	1.82	0.00	2.54
○Thread 842694			2.48	2.79	89.00	1.82	0.00	2.54
○Thread 842695			2.51	2.79	90.07	1.82	0.00	2.54
○Thread 842696			2.47	2.79	88.81	1.82	0.00	2.54
○Thread 842697			2.50	2.79	89.70	1.82	0.00	2.53
○Thread 842698			2.49	2.79	89.51	1.82	0.00	2.54
○Thread 842699			2.51	2.79	90.04	1.82	0.00	2.54
○Thread 842700			2.50	2.79	89.86	1.82	0.00	2.54
○Thread 842701			2.50	2.79	89.67	1.82	0.00	2.54
○Thread 842702			2.47	2.79	88.59	1.82	0.00	2.54
○Thread 842703			2.48	2.79	89.12	1.82	0.00	2.54
○Thread 842704			2.48	2.79	88.93	1.82	0.00	2.54

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	2.65
►Process 842724+		40	2.65
○Thread 842724			2.65	4.25	62.42	0.00	0.00	2.40
○Thread 842823			2.13	2.41	88.20	1.82	0.01	2.17
○Thread 842824			2.12	2.42	87.78	1.82	0.01	2.17
○Thread 842825			2.09	2.42	86.74	1.82	0.01	2.17
○Thread 842826			2.13	2.42	88.39	1.82	0.01	2.18
○Thread 842827			2.09	2.42	86.72	1.82	0.01	2.18
○Thread 842828			2.14	2.42	88.58	1.82	0.01	2.18
○Thread 842829			2.14	2.42	88.57	1.82	0.01	2.17
○Thread 842830			2.13	2.42	88.35	1.82	0.01	2.17
○Thread 842831			2.13	2.42	88.35	1.82	0.01	2.17
○Thread 842832			2.13	2.42	88.13	1.82	0.01	2.18
○Thread 842833			2.11	2.42	87.50	1.82	0.01	2.17
○Thread 842834			2.11	2.42	87.50	1.82	0.01	2.17
○Thread 842835			2.11	2.42	87.49	1.82	0.01	2.17
○Thread 842836			2.13	2.42	88.29	1.82	0.01	2.18
○Thread 842837			2.12	2.42	87.66	1.82	0.01	2.18
○Thread 842838			2.14	2.42	88.48	1.82	0.01	2.18
○Thread 842839			2.15	2.42	88.88	1.82	0.01	2.17
○Thread 842840			2.11	2.42	87.43	1.82	0.01	2.17
○Thread 842841			2.12	2.42	87.83	1.82	0.01	2.17
○Thread 842842			2.14	2.42	88.65	1.82	0.01	2.18
○Thread 842843			2.12	2.42	87.82	1.82	0.01	2.18
○Thread 842844			2.13	2.42	88.02	1.82	0.01	2.17
○Thread 842845			2.15	2.42	88.83	1.82	0.01	2.18
○Thread 842846			2.15	2.42	88.83	1.82	0.01	2.18
○Thread 842847			2.12	2.42	87.58	1.82	0.01	2.18
○Thread 842848			2.12	2.42	87.78	1.82	0.01	2.18
○Thread 842849			2.15	2.42	88.80	1.82	0.01	2.17
○Thread 842850			2.12	2.42	87.56	1.82	0.00	2.18
○Thread 842851			2.12	2.42	87.76	1.82	0.00	2.17
○Thread 842852			2.13	2.42	88.16	1.82	0.00	2.17
○Thread 842853			2.12	2.42	87.74	1.82	0.00	2.18
○Thread 842854			2.16	2.42	89.38	1.82	0.00	2.18
○Thread 842855			2.11	2.42	87.10	1.82	0.00	2.18
○Thread 842856			2.14	2.42	88.54	1.82	0.00	2.18
○Thread 842857			2.12	2.42	87.50	1.82	0.00	2.18
○Thread 842858			2.13	2.42	88.11	1.82	0.00	2.17
○Thread 842859			2.13	2.42	87.90	1.82	0.00	2.18
○Thread 842860			2.13	2.42	88.10	1.82	0.00	2.18
○Thread 842861			2.04	2.42	84.38	1.82	0.00	2.18

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	2.38
►Process 842881+		48	2.38
○Thread 842881			2.38	4.00	59.59	0.00	0.00	2.14
○Thread 842980			1.85	2.16	85.57	1.82	0.02	1.91
○Thread 842981			1.85	2.16	85.79	1.82	0.02	1.92
○Thread 842982			1.84	2.16	85.32	1.82	0.02	1.92
○Thread 842983			1.84	2.16	85.08	1.82	0.02	1.91
○Thread 842984			1.86	2.16	86.23	1.82	0.02	1.92
○Thread 842985			1.82	2.16	84.14	1.82	0.02	1.91
○Thread 842986			1.84	2.16	85.28	1.82	0.02	1.91
○Thread 842987			1.88	2.16	87.13	1.82	0.01	1.91
○Thread 842988			1.85	2.16	85.73	1.82	0.01	1.91
○Thread 842989			1.86	2.16	85.95	1.82	0.01	1.92
○Thread 842990			1.87	2.16	86.40	1.82	0.01	1.91
○Thread 842991			1.84	2.16	85.01	1.82	0.01	1.91
○Thread 842992			1.86	2.16	85.93	1.82	0.01	1.91
○Thread 842993			1.86	2.16	85.92	1.83	0.01	1.92
○Thread 842994			1.86	2.17	86.14	1.83	0.01	1.92
○Thread 842995			1.83	2.17	84.51	1.83	0.01	1.92
○Thread 842996			1.88	2.17	86.81	1.83	0.01	1.92
○Thread 842997			1.87	2.17	86.34	1.83	0.01	1.91
○Thread 842998			1.90	2.17	87.95	1.83	0.01	1.92
○Thread 842999			1.88	2.17	86.79	1.83	0.01	1.92
○Thread 843000			1.87	2.17	86.31	1.83	0.01	1.92
○Thread 843001			1.85	2.17	85.38	1.83	0.01	1.91
○Thread 843002			1.85	2.17	85.37	1.83	0.01	1.90
○Thread 843003			1.86	2.17	85.83	1.83	0.01	1.91
○Thread 843004			1.85	2.17	85.59	1.83	0.01	1.92
○Thread 843005			1.85	2.17	85.58	1.83	0.01	1.92
○Thread 843006			1.82	2.17	83.96	1.83	0.01	1.92
○Thread 843007			1.88	2.17	86.71	1.83	0.01	1.92
○Thread 843008			1.85	2.17	85.55	1.83	0.01	1.92
○Thread 843009			1.85	2.17	85.54	1.83	0.01	1.92
○Thread 843010			1.87	2.17	86.46	1.83	0.01	1.92
○Thread 843011			1.87	2.17	86.45	1.83	0.01	1.92
○Thread 843012			1.86	2.17	85.75	1.83	0.01	1.92
○Thread 843013			1.83	2.17	84.35	1.83	0.01	1.91
○Thread 843014			1.89	2.17	87.11	1.83	0.01	1.92
○Thread 843015			1.86	2.17	85.95	1.83	0.00	1.91
○Thread 843016			1.85	2.17	85.25	1.83	0.00	1.91
○Thread 843017			1.88	2.17	86.85	1.83	0.00	1.92
○Thread 843018			1.85	2.17	85.23	1.83	0.00	1.92
○Thread 843019			1.87	2.17	86.37	1.83	0.00	1.91
○Thread 843020			1.87	2.17	86.13	1.83	0.00	1.91
○Thread 843021			1.84	2.17	84.97	1.83	0.00	1.92
○Thread 843022			1.85	2.17	85.43	1.83	0.00	1.92
○Thread 843023			1.85	2.17	85.42	1.83	0.00	1.90
○Thread 843024			1.85	2.17	85.41	1.83	0.00	1.92
○Thread 843025			1.82	2.17	84.02	1.83	0.00	1.92
○Thread 843026			1.76	2.17	81.24	1.83	0.00	1.91

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	2.29
►Process 843046+		56	2.29
○Thread 843046			2.29	3.81	60.05	0.00	0.00	1.95
○Thread 843145			1.65	1.97	84.07	1.82	0.02	1.72
○Thread 843146			1.65	1.97	84.07	1.82	0.02	1.72
○Thread 843147			1.70	1.97	86.34	1.83	0.02	1.73
○Thread 843148			1.67	1.97	84.81	1.83	0.02	1.73
○Thread 843149			1.70	1.97	86.32	1.83	0.02	1.73
○Thread 843150			1.68	1.97	85.30	1.83	0.02	1.73
○Thread 843151			1.65	1.97	84.00	1.83	0.02	1.72
○Thread 843152			1.66	1.97	84.50	1.83	0.02	1.73
○Thread 843153			1.66	1.97	84.24	1.83	0.02	1.72
○Thread 843154			1.66	1.97	84.23	1.83	0.02	1.72
○Thread 843155			1.64	1.97	83.46	1.83	0.02	1.72
○Thread 843156			1.68	1.97	85.22	1.83	0.02	1.72
○Thread 843157			1.66	1.97	84.45	1.83	0.02	1.73
○Thread 843158			1.66	1.97	84.44	1.83	0.01	1.72
○Thread 843159			1.67	1.97	84.94	1.83	0.01	1.73
○Thread 843160			1.69	1.97	85.94	1.83	0.01	1.73
○Thread 843161			1.68	1.97	85.43	1.83	0.01	1.72
○Thread 843162			1.67	1.97	84.66	1.83	0.01	1.73
○Thread 843163			1.67	1.97	84.90	1.83	0.01	1.73
○Thread 843164			1.68	1.97	85.15	1.83	0.01	1.72
○Thread 843165			1.66	1.97	84.12	1.83	0.01	1.73
○Thread 843166			1.65	1.97	83.86	1.83	0.01	1.72
○Thread 843167			1.64	1.97	83.09	1.83	0.01	1.73
○Thread 843168			1.68	1.97	85.11	1.83	0.01	1.72
○Thread 843169			1.66	1.97	84.34	1.83	0.01	1.73
○Thread 843170			1.64	1.97	83.06	1.83	0.01	1.73
○Thread 843171			1.66	1.97	84.32	1.83	0.01	1.73
○Thread 843172			1.68	1.97	85.07	1.83	0.01	1.72
○Thread 843173			1.68	1.97	85.06	1.83	0.01	1.72
○Thread 843174			1.70	1.98	86.32	1.83	0.01	1.72
○Thread 843175			1.66	1.98	84.03	1.83	0.01	1.73
○Thread 843176			1.70	1.98	86.05	1.83	0.01	1.73
○Thread 843177			1.68	1.98	85.03	1.83	0.01	1.73
○Thread 843178			1.66	1.98	84.01	1.83	0.01	1.73
○Thread 843179			1.66	1.98	84.25	1.83	0.01	1.71
○Thread 843180			1.69	1.98	85.50	1.83	0.01	1.73
○Thread 843181			1.68	1.98	85.24	1.83	0.01	1.73
○Thread 843182			1.65	1.98	83.46	1.83	0.01	1.72
○Thread 843183			1.66	1.98	84.21	1.83	0.01	1.73
○Thread 843184			1.65	1.98	83.44	1.83	0.01	1.71
○Thread 843185			1.64	1.98	82.93	1.83	0.01	1.71
○Thread 843186			1.66	1.98	84.18	1.83	0.01	1.72
○Thread 843187			1.67	1.98	84.68	1.83	0.00	1.72
○Thread 843188			1.65	1.98	83.66	1.83	0.00	1.72
○Thread 843189			1.68	1.98	84.91	1.83	0.00	1.73
○Thread 843190			1.69	1.98	85.66	1.83	0.00	1.73
○Thread 843191			1.68	1.98	85.15	1.83	0.00	1.73
○Thread 843192			1.68	1.98	85.14	1.83	0.00	1.73
○Thread 843193			1.66	1.98	84.12	1.83	0.00	1.72
○Thread 843194			1.67	1.98	84.62	1.83	0.00	1.73
○Thread 843195			1.66	1.98	84.10	1.83	0.00	1.73
○Thread 843196			1.65	1.98	83.34	1.83	0.00	1.73
○Thread 843197			1.62	1.98	82.06	1.83	0.00	1.72
○Thread 843198			1.57	1.98	79.02	1.83	0.00	1.72
○Thread 843199			1.58	1.98	80.02	1.83	0.00	1.72

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	2.08
►Process 843268+		64	2.08
○Thread 843268			2.08	3.65	57.05	0.00	0.00	1.82
○Thread 843367			1.48	1.81	81.96	1.82	0.02	1.56
○Thread 843368			1.50	1.81	83.06	1.82	0.02	1.56
○Thread 843369			1.46	1.81	80.83	1.82	0.02	1.55
○Thread 843370			1.49	1.81	82.48	1.82	0.02	1.55
○Thread 843371			1.50	1.81	83.03	1.82	0.02	1.56
○Thread 843372			1.51	1.81	83.85	1.82	0.02	1.56
○Thread 843373			1.48	1.81	81.90	1.82	0.02	1.56
○Thread 843374			1.46	1.81	80.78	1.82	0.02	1.56
○Thread 843375			1.49	1.81	82.71	1.82	0.02	1.55
○Thread 843376			1.50	1.81	83.25	1.82	0.02	1.56
○Thread 843377			1.49	1.81	82.69	1.82	0.02	1.56
○Thread 843378			1.48	1.81	81.85	1.82	0.02	1.56
○Thread 843379			1.47	1.81	81.29	1.82	0.02	1.55
○Thread 843380			1.47	1.81	81.55	1.82	0.02	1.56
○Thread 843381			1.48	1.81	81.82	1.82	0.02	1.56
○Thread 843382			1.51	1.81	83.47	1.82	0.02	1.56
○Thread 843383			1.47	1.81	81.52	1.82	0.02	1.56
○Thread 843384			1.47	1.81	81.24	1.82	0.02	1.55
○Thread 843385			1.41	1.81	77.91	1.82	0.02	1.54
○Thread 843386			1.47	1.81	81.22	1.82	0.02	1.56
○Thread 843387			1.47	1.81	81.48	1.82	0.02	1.56
○Thread 843388			1.48	1.81	81.75	1.82	0.02	1.55
○Thread 843389			1.49	1.81	82.57	1.82	0.01	1.56
○Thread 843390			1.48	1.81	82.01	1.82	0.01	1.56
○Thread 843391			1.48	1.81	82.00	1.82	0.01	1.55
○Thread 843392			1.48	1.81	81.71	1.82	0.01	1.56
○Thread 843393			1.49	1.81	82.53	1.82	0.01	1.56
○Thread 843394			1.48	1.81	81.97	1.82	0.01	1.55
○Thread 843395			1.48	1.81	81.96	1.82	0.01	1.56
○Thread 843396			1.50	1.81	82.78	1.82	0.01	1.56
○Thread 843397			1.50	1.81	82.77	1.82	0.01	1.56
○Thread 843398			1.47	1.81	81.10	1.82	0.01	1.56
○Thread 843399			1.48	1.81	81.64	1.82	0.01	1.56
○Thread 843400			1.51	1.81	83.29	1.82	0.01	1.55
○Thread 843401			1.47	1.81	81.07	1.82	0.01	1.50
○Thread 843402			1.50	1.81	82.72	1.82	0.01	1.56
○Thread 843403			1.49	1.81	82.16	1.82	0.01	1.56
○Thread 843404			1.48	1.81	81.87	1.82	0.01	1.55
○Thread 843405			1.48	1.81	81.59	1.82	0.01	1.56
○Thread 843406			1.48	1.81	81.58	1.82	0.01	1.55
○Thread 843407			1.48	1.81	81.84	1.82	0.01	1.56
○Thread 843408			1.46	1.81	80.45	1.82	0.01	1.56
○Thread 843409			1.48	1.81	81.55	1.82	0.01	1.56
○Thread 843410			1.45	1.82	79.88	1.82	0.01	1.55
○Thread 843411			1.47	1.82	81.25	1.82	0.01	1.56
○Thread 843412			1.51	1.82	83.17	1.82	0.01	1.56
○Thread 843413			1.50	1.82	82.88	1.82	0.01	1.55
○Thread 843414			1.50	1.82	82.60	1.82	0.01	1.55
○Thread 843415			1.47	1.82	81.21	1.82	0.01	1.56
○Thread 843416			1.47	1.82	81.20	1.82	0.01	1.56
○Thread 843417			1.48	1.82	81.47	1.82	0.00	1.56
○Thread 843418			1.48	1.82	81.46	1.82	0.00	1.54
○Thread 843419			1.48	1.82	81.45	1.82	0.00	1.56
○Thread 843420			1.49	1.82	82.26	1.82	0.00	1.56
○Thread 843421			1.48	1.82	81.70	1.82	0.00	1.56
○Thread 843422			1.47	1.82	81.14	1.82	0.00	1.55
○Thread 843423			1.46	1.82	80.58	1.82	0.00	1.56
○Thread 843424			1.49	1.82	81.95	1.83	0.00	1.54
○Thread 843425			1.47	1.82	80.84	1.83	0.00	1.55
○Thread 843426			1.50	1.82	82.48	1.83	0.00	1.56
○Thread 843427			1.46	1.82	80.54	1.83	0.00	1.54
○Thread 843428			1.49	1.82	82.19	1.83	0.00	1.56
○Thread 843429			1.46	1.82	80.53	1.83	0.00	1.55

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	2.02
►Process 843449+		72	2.02
○Thread 843449			2.02	3.57	56.55	0.00	0.00	1.80
○Thread 843548			1.39	1.72	80.59	1.82	0.03	1.48
○Thread 843549			1.39	1.73	80.87	1.82	0.03	1.48
○Thread 843550			1.40	1.73	81.15	1.82	0.03	1.48
○Thread 843551			1.38	1.73	79.98	1.82	0.02	1.48
○Thread 843552			1.40	1.73	81.42	1.82	0.02	1.47
○Thread 843553			1.39	1.73	80.54	1.82	0.02	1.48
○Thread 843554			1.37	1.73	79.37	1.82	0.02	1.48
○Thread 843555			1.36	1.73	78.78	1.82	0.02	1.46
○Thread 843556			1.36	1.73	78.77	1.82	0.02	1.48
○Thread 843557			1.41	1.73	81.94	1.82	0.02	1.48
○Thread 843558			1.37	1.73	79.33	1.82	0.02	1.47
○Thread 843559			1.41	1.73	81.92	1.82	0.02	1.48
○Thread 843560			1.38	1.73	79.88	1.82	0.02	1.48
○Thread 843561			1.39	1.73	80.45	1.82	0.02	1.48
○Thread 843562			1.37	1.73	79.57	1.82	0.02	1.48
○Thread 843563			1.40	1.73	81.30	1.82	0.02	1.48
○Thread 843564			1.37	1.73	79.55	1.82	0.02	1.47
○Thread 843565			1.39	1.73	80.70	1.82	0.02	1.48
○Thread 843566			1.35	1.73	78.37	1.82	0.02	1.48
○Thread 843567			1.38	1.73	79.81	1.82	0.02	1.48
○Thread 843568			1.41	1.73	81.53	1.82	0.02	1.48
○Thread 843569			1.34	1.73	77.77	1.82	0.02	1.46
○Thread 843570			1.38	1.73	80.07	1.82	0.02	1.47
○Thread 843571			1.40	1.73	80.92	1.82	0.02	1.48
○Thread 843572			1.37	1.73	79.47	1.82	0.02	1.45
○Thread 843573			1.35	1.73	78.30	1.82	0.02	1.48
○Thread 843574			1.37	1.73	79.13	1.82	0.02	1.48
○Thread 843575			1.37	1.73	79.12	1.82	0.02	1.48
○Thread 843576			1.39	1.73	80.56	1.82	0.02	1.48
○Thread 843577			1.35	1.73	78.24	1.83	0.02	1.48
○Thread 843578			1.37	1.73	79.09	1.83	0.01	1.48
○Thread 843579			1.37	1.73	79.08	1.83	0.01	1.48
○Thread 843580			1.38	1.73	79.94	1.83	0.01	1.48
○Thread 843581			1.40	1.73	81.08	1.83	0.01	1.47
○Thread 843582			1.39	1.73	80.21	1.83	0.01	1.48
○Thread 843583			1.36	1.73	78.75	1.83	0.01	1.48
○Thread 843584			1.38	1.73	79.61	1.83	0.01	1.48
○Thread 843585			1.39	1.73	80.46	1.83	0.01	1.48
○Thread 843586			1.40	1.73	81.03	1.83	0.01	1.47
○Thread 843587			1.40	1.73	81.02	1.83	0.01	1.48
○Thread 843588			1.37	1.73	79.28	1.83	0.01	1.48
○Thread 843589			1.37	1.73	79.27	1.83	0.01	1.48
○Thread 843590			1.37	1.73	78.97	1.83	0.01	1.47
○Thread 843591			1.41	1.74	81.56	1.83	0.01	1.48
○Thread 843592			1.37	1.74	79.24	1.83	0.01	1.48
○Thread 843593			1.37	1.74	78.94	1.83	0.01	1.48
○Thread 843594			1.39	1.74	80.37	1.83	0.01	1.48
○Thread 843595			1.43	1.74	82.38	1.83	0.01	1.48
○Thread 843596			1.39	1.74	80.06	1.83	0.01	1.47
○Thread 843597			1.38	1.74	79.77	1.83	0.01	1.47
○Thread 843598			1.38	1.74	79.47	1.83	0.01	1.47
○Thread 843599			1.35	1.74	77.73	1.83	0.01	1.47
○Thread 843600			1.33	1.74	76.86	1.83	0.01	1.48
○Thread 843601			1.40	1.74	80.59	1.83	0.01	1.47
○Thread 843602			1.40	1.74	80.58	1.83	0.01	1.48
○Thread 843603			1.40	1.74	80.56	1.83	0.01	1.48
○Thread 843604			1.41	1.74	81.13	1.83	0.01	1.48
○Thread 843605			1.37	1.74	79.11	1.83	0.01	1.48
○Thread 843606			1.38	1.74	79.38	1.83	0.01	1.47
○Thread 843607			1.41	1.74	81.39	1.83	0.00	1.47
○Thread 843608			1.37	1.74	79.08	1.83	0.00	1.48
○Thread 843609			1.40	1.74	80.50	1.83	0.00	1.48
○Thread 843610			1.39	1.74	80.21	1.83	0.00	1.48
○Thread 843611			1.39	1.74	80.19	1.83	0.00	1.48
○Thread 843612			1.39	1.74	80.19	1.83	0.00	1.47
○Thread 843613			1.43	1.74	82.47	1.83	0.00	1.48
○Thread 843614			1.43	1.74	82.18	1.83	0.00	1.48
○Thread 843615			1.35	1.74	77.57	1.83	0.00	1.47
○Thread 843616			1.34	1.74	76.98	1.83	0.00	1.46
○Thread 843617			1.34	1.74	76.97	1.83	0.00	1.46
○Thread 843618			1.30	1.74	74.95	1.83	0.00	1.47

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	1.78
►Process 843638+		80	1.78
○Thread 843638			1.78	3.48	51.31	0.00	0.00	1.80
○Thread 843737			1.31	1.62	80.80	1.83	0.03	1.38
○Thread 843738			1.29	1.62	79.86	1.83	0.03	1.38
○Thread 843739			1.31	1.62	80.77	1.83	0.03	1.37
○Thread 843740			1.27	1.62	78.30	1.83	0.03	1.34
○Thread 843741			1.26	1.62	77.98	1.83	0.03	1.37
○Thread 843742			1.28	1.62	78.89	1.83	0.03	1.38
○Thread 843743			1.26	1.62	77.65	1.83	0.03	1.37
○Thread 843744			1.30	1.62	80.41	1.83	0.03	1.38
○Thread 843745			1.30	1.62	80.40	1.83	0.03	1.37
○Thread 843746			1.27	1.62	78.23	1.83	0.03	1.36
○Thread 843747			1.29	1.62	79.76	1.83	0.03	1.36
○Thread 843748			1.27	1.62	78.21	1.83	0.02	1.38
○Thread 843749			1.27	1.62	78.51	1.83	0.02	1.37
○Thread 843750			1.28	1.62	78.80	1.83	0.02	1.38
○Thread 843751			1.27	1.62	78.18	1.83	0.02	1.37
○Thread 843752			1.28	1.62	79.09	1.83	0.02	1.38
○Thread 843753			1.27	1.63	78.44	1.83	0.02	1.38
○Thread 843754			1.30	1.63	80.28	1.83	0.02	1.38
○Thread 843755			1.32	1.63	81.19	1.83	0.02	1.38
○Thread 843756			1.29	1.63	79.33	1.83	0.02	1.36
○Thread 843757			1.27	1.63	78.40	1.83	0.02	1.37
○Thread 843758			1.28	1.63	79.00	1.83	0.02	1.37
○Thread 843759			1.28	1.63	78.99	1.83	0.02	1.37
○Thread 843760			1.27	1.63	78.06	1.83	0.02	1.36
○Thread 843761			1.32	1.63	81.12	1.83	0.02	1.38
○Thread 843762			1.32	1.63	81.11	1.83	0.02	1.38
○Thread 843763			1.29	1.63	79.26	1.83	0.02	1.38
○Thread 843764			1.30	1.63	79.86	1.83	0.02	1.38
○Thread 843765			1.26	1.63	77.70	1.83	0.02	1.36
○Thread 843766			1.30	1.63	80.15	1.83	0.02	1.37
○Thread 843767			1.26	1.63	77.68	1.83	0.02	1.38
○Thread 843768			1.32	1.63	81.05	1.83	0.02	1.38
○Thread 843769			1.28	1.63	78.89	1.83	0.02	1.38
○Thread 843770			1.29	1.63	79.18	1.83	0.02	1.37
○Thread 843771			1.28	1.63	78.56	1.83	0.02	1.38
○Thread 843772			1.29	1.63	79.47	1.83	0.02	1.36
○Thread 843773			1.28	1.63	78.54	1.83	0.02	1.37
○Thread 843774			1.25	1.63	76.99	1.83	0.02	1.37
○Thread 843775			1.28	1.63	78.52	1.83	0.02	1.37
○Thread 843776			1.31	1.63	80.65	1.83	0.01	1.38
○Thread 843777			1.29	1.63	79.42	1.83	0.01	1.38
○Thread 843778			1.31	1.63	80.63	1.83	0.01	1.37
○Thread 843779			1.27	1.63	78.17	1.83	0.01	1.38
○Thread 843780			1.31	1.63	80.30	1.83	0.01	1.36
○Thread 843781			1.32	1.63	80.91	1.83	0.01	1.37
○Thread 843782			1.31	1.63	80.59	1.83	0.01	1.38
○Thread 843783			1.25	1.63	76.59	1.83	0.01	1.37
○Thread 843784			1.31	1.63	80.26	1.83	0.01	1.37
○Thread 843785			1.29	1.63	79.02	1.83	0.01	1.37
○Thread 843786			1.29	1.63	79.32	1.83	0.01	1.37
○Thread 843787			1.25	1.63	76.86	1.84	0.01	1.37
○Thread 843788			1.28	1.63	78.68	1.84	0.01	1.37
○Thread 843789			1.28	1.63	78.37	1.84	0.01	1.37
○Thread 843790			1.30	1.63	79.58	1.84	0.01	1.36
○Thread 843791			1.26	1.63	77.43	1.84	0.01	1.38
○Thread 843792			1.30	1.63	79.56	1.84	0.01	1.37
○Thread 843793			1.28	1.63	78.33	1.84	0.01	1.35
○Thread 843794			1.29	1.63	79.23	1.84	0.01	1.38
○Thread 843795			1.29	1.63	78.92	1.84	0.01	1.37
○Thread 843796			1.28	1.63	78.60	1.84	0.01	1.38
○Thread 843797			1.27	1.64	77.98	1.84	0.01	1.36
○Thread 843798			1.28	1.64	78.27	1.84	0.01	1.37
○Thread 843799			1.31	1.64	80.40	1.84	0.01	1.37
○Thread 843800			1.30	1.64	79.47	1.84	0.01	1.38
○Thread 843801			1.30	1.64	79.77	1.84	0.01	1.36
○Thread 843802			1.27	1.64	77.62	1.84	0.01	1.37
○Thread 843803			1.29	1.64	79.13	1.84	0.01	1.37
○Thread 843804			1.25	1.64	76.37	1.84	0.00	1.36
○Thread 843805			1.29	1.64	79.11	1.84	0.00	1.38
○Thread 843806			1.28	1.64	78.18	1.84	0.00	1.37
○Thread 843807			1.30	1.64	79.39	1.84	0.00	1.36
○Thread 843808			1.31	1.64	80.30	1.84	0.00	1.38
○Thread 843809			1.30	1.64	79.68	1.84	0.00	1.38
○Thread 843810			1.26	1.64	77.22	1.84	0.00	1.36
○Thread 843811			1.28	1.64	78.13	1.84	0.00	1.36
○Thread 843812			1.32	1.64	80.56	1.84	0.00	1.38
○Thread 843813			1.29	1.64	78.71	1.84	0.00	1.38
○Thread 843814			1.31	1.64	79.92	1.84	0.00	1.38
○Thread 843815			1.23	1.64	75.03	1.84	0.00	1.36

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	1.85
►Process 843835+		88	1.85
○Thread 843835			1.85	3.41	54.42	0.00	0.00	1.80
○Thread 843935			1.24	1.56	79.64	1.82	0.03	1.30
○Thread 843936			1.22	1.56	78.35	1.82	0.03	1.30
○Thread 843937			1.22	1.56	78.34	1.82	0.03	1.30
○Thread 843938			1.21	1.56	78.00	1.82	0.03	1.31
○Thread 843939			1.26	1.56	80.88	1.82	0.03	1.30
○Thread 843940			1.22	1.56	78.30	1.82	0.03	1.30
○Thread 843941			1.22	1.56	78.29	1.82	0.03	1.30
○Thread 843942			1.19	1.56	76.35	1.82	0.03	1.29
○Thread 843943			1.24	1.56	79.55	1.82	0.03	1.30
○Thread 843944			1.21	1.56	77.94	1.82	0.03	1.29
○Thread 843945			1.23	1.56	78.89	1.82	0.03	1.29
○Thread 843946			1.22	1.56	78.55	1.82	0.03	1.30
○Thread 843947			1.23	1.56	79.18	1.82	0.03	1.30
○Thread 843948			1.21	1.56	77.89	1.82	0.03	1.29
○Thread 843949			1.21	1.56	77.88	1.82	0.03	1.29
○Thread 843950			1.23	1.56	78.83	1.82	0.03	1.29
○Thread 843951			1.23	1.56	79.14	1.82	0.03	1.30
○Thread 843952			1.22	1.56	78.49	1.82	0.03	1.30
○Thread 843953			1.23	1.56	78.80	1.82	0.02	1.29
○Thread 843954			1.20	1.56	77.18	1.82	0.02	1.30
○Thread 843955			1.22	1.56	78.13	1.82	0.02	1.30
○Thread 843956			1.23	1.56	78.76	1.82	0.02	1.28
○Thread 843957			1.21	1.56	77.79	1.82	0.02	1.30
○Thread 843958			1.22	1.56	78.42	1.82	0.02	1.30
○Thread 843959			1.22	1.56	78.41	1.82	0.02	1.29
○Thread 843960			1.21	1.56	77.44	1.82	0.02	1.53
○Thread 843961			1.24	1.56	79.35	1.82	0.02	1.30
○Thread 843962			1.23	1.56	78.70	1.82	0.02	1.30
○Thread 843963			1.19	1.56	76.45	1.82	0.02	1.29
○Thread 843964			1.22	1.56	78.04	1.82	0.02	1.30
○Thread 843965			1.22	1.56	78.03	1.82	0.02	1.30
○Thread 843966			1.23	1.56	78.66	1.82	0.02	1.30
○Thread 843967			1.24	1.56	79.28	1.82	0.02	1.29
○Thread 843968			1.20	1.56	77.04	1.82	0.02	1.29
○Thread 843969			1.24	1.56	79.27	1.82	0.02	1.29
○Thread 843970			1.21	1.56	77.34	1.82	0.02	1.31
○Thread 843971			1.22	1.56	77.96	1.82	0.02	1.29
○Thread 843972			1.24	1.57	79.23	1.83	0.02	1.30
○Thread 843973			1.23	1.57	78.58	1.83	0.02	1.31
○Thread 843974			1.19	1.57	76.34	1.83	0.02	1.30
○Thread 843975			1.20	1.57	76.96	1.83	0.02	1.30
○Thread 843976			1.22	1.57	77.91	1.83	0.02	1.30
○Thread 843977			1.20	1.57	76.94	1.83	0.02	1.29
○Thread 843978			1.21	1.57	77.57	1.83	0.02	1.29
○Thread 843979			1.20	1.57	76.92	1.83	0.02	1.30
○Thread 843980			1.22	1.57	78.18	1.83	0.02	1.31
○Thread 843981			1.20	1.57	76.58	1.83	0.02	1.30
○Thread 843982			1.20	1.57	76.57	1.83	0.01	1.29
○Thread 843983			1.23	1.57	78.79	1.83	0.01	1.30
○Thread 843984			1.20	1.57	76.86	1.83	0.01	1.30
○Thread 843985			1.21	1.57	77.17	1.83	0.01	1.30
○Thread 843986			1.21	1.57	77.48	1.83	0.01	1.30
○Thread 843987			1.22	1.57	78.11	1.83	0.01	1.30
○Thread 843988			1.22	1.57	77.78	1.83	0.01	1.30
○Thread 843989			1.21	1.57	77.13	1.83	0.01	1.30
○Thread 843990			1.20	1.57	76.48	1.83	0.01	1.30
○Thread 843991			1.23	1.57	78.38	1.83	0.01	1.29
○Thread 843992			1.23	1.57	78.69	1.83	0.01	1.30
○Thread 843993			1.18	1.57	75.17	1.83	0.01	1.30
○Thread 843994			1.15	1.57	73.57	1.83	0.01	1.30
○Thread 843995			1.23	1.57	78.66	1.83	0.01	1.56
○Thread 843996			1.21	1.57	77.05	1.83	0.01	1.30
○Thread 843997			1.21	1.57	77.36	1.83	0.01	1.30
○Thread 843998			1.23	1.57	78.62	1.83	0.01	1.29
○Thread 843999			1.21	1.57	77.02	1.83	0.01	1.30
○Thread 844000			1.22	1.57	77.65	1.83	0.01	1.29
○Thread 844001			1.20	1.57	76.68	1.83	0.01	1.30
○Thread 844002			1.19	1.57	76.03	1.83	0.01	1.30
○Thread 844003			1.19	1.57	75.70	1.83	0.01	1.30
○Thread 844004			1.22	1.57	77.92	1.83	0.01	1.30
○Thread 844005			1.22	1.57	77.59	1.83	0.01	1.30
○Thread 844006			1.21	1.57	77.26	1.83	0.01	1.30
○Thread 844007			1.20	1.57	76.61	1.83	0.01	1.30
○Thread 844008			1.18	1.57	75.01	1.83	0.01	1.29
○Thread 844009			1.17	1.57	74.68	1.83	0.01	1.30
○Thread 844010			1.19	1.57	75.94	1.83	0.00	1.30
○Thread 844011			1.20	1.57	76.25	1.83	0.00	1.30
○Thread 844012			1.23	1.57	78.46	1.83	0.00	1.30
○Thread 844013			1.22	1.57	77.50	1.83	0.00	1.30
○Thread 844014			1.22	1.57	77.80	1.83	0.00	1.30
○Thread 844015			1.22	1.57	77.47	1.83	0.00	1.30
○Thread 844016			1.20	1.57	76.51	1.83	0.00	1.29
○Thread 844017			1.24	1.58	78.72	1.83	0.00	1.30
○Thread 844018			1.22	1.58	77.44	1.83	0.00	1.29
○Thread 844019			1.19	1.58	75.84	1.83	0.00	1.30
○Thread 844020			1.18	1.58	75.19	1.83	0.00	1.30
○Thread 844021			1.19	1.58	75.18	1.83	0.00	1.30

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	1.85
►Process 844041+		96	1.85
○Thread 844041			1.85	3.38	54.75	0.00	0.00	1.71
○Thread 844140			1.18	1.51	78.37	1.84	0.03	1.26
○Thread 844141			1.17	1.51	78.03	1.84	0.03	1.25
○Thread 844142			1.19	1.51	79.01	1.84	0.03	1.26
○Thread 844143			1.16	1.51	77.31	1.84	0.03	1.25
○Thread 844144			1.13	1.51	75.30	1.84	0.03	1.26
○Thread 844145			1.20	1.51	79.94	1.84	0.03	1.26
○Thread 844146			1.14	1.51	75.62	1.84	0.03	1.24
○Thread 844147			1.15	1.51	76.27	1.84	0.03	1.26
○Thread 844148			1.21	1.51	80.24	1.84	0.03	1.26
○Thread 844149			1.18	1.51	78.57	1.84	0.03	1.26
○Thread 844150			1.15	1.51	76.24	1.84	0.03	1.24
○Thread 844151			1.16	1.51	76.89	1.84	0.03	1.25
○Thread 844152			1.14	1.51	75.88	1.84	0.03	1.25
○Thread 844153			1.18	1.51	78.52	1.84	0.03	1.26
○Thread 844154			1.16	1.51	76.53	1.84	0.03	1.26
○Thread 844155			1.16	1.51	77.18	1.84	0.03	1.26
○Thread 844156			1.17	1.51	77.50	1.84	0.03	1.26
○Thread 844157			1.18	1.51	78.48	1.84	0.03	1.26
○Thread 844158			1.18	1.51	78.14	1.84	0.03	1.26
○Thread 844159			1.17	1.51	77.80	1.84	0.03	1.25
○Thread 844160			1.16	1.51	77.12	1.84	0.03	1.26
○Thread 844161			1.15	1.51	76.12	1.84	0.03	1.26
○Thread 844162			1.17	1.51	77.43	1.84	0.03	1.26
○Thread 844163			1.19	1.51	78.74	1.84	0.03	1.26
○Thread 844164			1.18	1.51	78.40	1.84	0.03	1.26
○Thread 844165			1.19	1.51	79.05	1.84	0.02	1.26
○Thread 844166			1.15	1.51	76.40	1.84	0.02	1.26
○Thread 844167			1.20	1.51	79.69	1.84	0.02	1.26
○Thread 844168			1.18	1.51	78.36	1.84	0.02	1.26
○Thread 844169			1.18	1.51	78.02	1.84	0.02	1.26
○Thread 844170			1.17	1.51	77.34	1.84	0.02	1.26
○Thread 844171			1.21	1.51	79.98	1.84	0.02	1.26
○Thread 844172			1.16	1.51	76.99	1.84	0.02	1.26
○Thread 844173			1.18	1.51	77.97	1.84	0.02	1.26
○Thread 844174			1.14	1.51	75.65	1.84	0.02	1.26
○Thread 844175			1.18	1.51	77.95	1.84	0.02	1.26
○Thread 844176			1.19	1.51	78.60	1.84	0.02	1.25
○Thread 844177			1.17	1.51	77.27	1.84	0.02	1.25
○Thread 844178			1.16	1.51	76.93	1.84	0.02	1.25
○Thread 844179			1.13	1.51	74.94	1.84	0.02	1.26
○Thread 844180			1.15	1.51	75.92	1.84	0.02	1.26
○Thread 844181			1.17	1.52	77.23	1.84	0.02	1.26
○Thread 844182			1.18	1.52	77.54	1.84	0.02	1.26
○Thread 844183			1.16	1.52	76.54	1.84	0.02	1.26
○Thread 844184			1.18	1.52	78.18	1.84	0.02	1.26
○Thread 844185			1.17	1.52	77.51	1.84	0.02	1.25
○Thread 844186			1.17	1.52	77.17	1.85	0.02	1.26
○Thread 844187			1.16	1.52	76.50	1.85	0.02	1.25
○Thread 844188			1.17	1.52	77.15	1.85	0.02	1.26
○Thread 844189			1.16	1.52	76.15	1.85	0.02	1.25
○Thread 844190			1.13	1.52	74.82	1.85	0.02	1.26
○Thread 844191			1.18	1.52	78.10	1.85	0.02	1.26
○Thread 844192			1.18	1.52	78.09	1.85	0.02	1.26
○Thread 844193			1.17	1.52	77.42	1.85	0.02	1.25
○Thread 844194			1.16	1.52	76.75	1.85	0.01	1.26
○Thread 844195			1.14	1.52	75.10	1.85	0.01	1.26
○Thread 844196			1.18	1.52	78.05	1.85	0.01	1.26
○Thread 844197			1.17	1.52	77.38	1.85	0.01	1.25
○Thread 844198			1.18	1.52	78.03	1.85	0.01	1.26
○Thread 844199			1.17	1.52	77.36	1.85	0.01	1.24
○Thread 844200			1.17	1.52	77.02	1.85	0.01	1.26
○Thread 844201			1.18	1.52	77.66	1.85	0.01	1.26
○Thread 844202			1.17	1.52	76.99	1.85	0.01	1.26
○Thread 844203			1.17	1.52	76.98	1.85	0.01	1.25
○Thread 844204			1.16	1.52	76.64	1.85	0.01	1.26
○Thread 844205			1.13	1.52	74.66	1.85	0.01	1.26
○Thread 844206			1.15	1.52	75.96	1.85	0.01	1.26
○Thread 844207			1.13	1.52	74.64	1.85	0.01	1.26
○Thread 844208			1.18	1.52	77.59	1.85	0.01	1.26
○Thread 844209			1.17	1.52	77.25	1.85	0.01	1.26
○Thread 844210			1.18	1.52	77.89	1.85	0.01	1.26
○Thread 844211			1.19	1.52	78.54	1.85	0.01	1.26
○Thread 844212			1.13	1.52	74.59	1.85	0.01	1.25
○Thread 844213			1.14	1.52	75.23	1.85	0.01	1.24
○Thread 844214			1.13	1.52	74.23	1.85	0.01	1.25
○Thread 844215			1.16	1.52	76.19	1.85	0.01	1.25
○Thread 844216			1.15	1.52	75.53	1.85	0.01	1.25
○Thread 844217			1.18	1.52	77.49	1.85	0.01	1.26
○Thread 844218			1.16	1.52	76.16	1.85	0.01	1.26
○Thread 844219			1.15	1.52	75.82	1.85	0.01	1.25
○Thread 844220			1.11	1.52	72.86	1.85	0.01	1.25
○Thread 844221			1.17	1.52	76.79	1.85	0.01	1.25
○Thread 844222			1.13	1.52	74.48	1.85	0.01	1.25
○Thread 844223			1.17	1.52	77.09	1.85	0.00	1.26
○Thread 844224			1.17	1.52	76.75	1.85	0.00	1.26
○Thread 844225			1.20	1.52	78.71	1.85	0.00	1.26
○Thread 844226			1.18	1.52	77.71	1.85	0.00	1.25
○Thread 844227			1.14	1.53	75.08	1.85	0.00	1.26
○Thread 844228			1.18	1.53	77.36	1.85	0.00	1.25
○Thread 844229			1.18	1.53	77.35	1.85	0.00	1.25
○Thread 844230			1.18	1.53	77.66	1.85	0.00	1.25
○Thread 844231			1.18	1.53	77.65	1.85	0.00	1.25
○Thread 844232			1.15	1.53	75.02	1.85	0.00	1.25
○Thread 844233			1.11	1.53	72.72	1.85	0.00	1.26
○Thread 844234			1.05	1.53	68.78	1.85	0.00	1.24

Report Configuration