OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	31.72
►Process 569405+		1	31.72
○Thread 569405			31.72	33.20	95.55	0.00	0.00	33.18

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	16.90
►Process 569525+		2	16.90
○Thread 569525			16.90	18.38	91.96	0.00	0.00	16.57
○Thread 569624			16.18	16.58	97.59	1.80	0.00	16.35

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	9.34
►Process 569693+		4	9.34
○Thread 569693			9.34	10.86	85.92	0.00	0.00	9.07
○Thread 569792			8.70	9.07	95.88	1.79	0.00	8.84
○Thread 569793			8.66	9.07	95.38	1.79	0.00	8.84
○Thread 569794			8.64	9.07	95.27	1.79	0.00	8.84

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	5.78
►Process 569815+		8	5.78
○Thread 569815			5.78	7.30	79.17	0.00	0.00	5.48
○Thread 569914			5.09	5.49	92.70	1.81	0.00	5.25
○Thread 569915			5.08	5.49	92.60	1.81	0.00	5.25
○Thread 569916			5.08	5.49	92.60	1.81	0.00	5.25
○Thread 569917			5.04	5.49	91.87	1.81	0.00	5.25
○Thread 569918			5.08	5.49	92.50	1.81	0.00	5.25
○Thread 569919			5.12	5.49	93.32	1.81	0.00	5.25
○Thread 569920			5.05	5.49	92.04	1.81	0.00	5.25

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	3.97
►Process 569940+		16	3.97
○Thread 569940			3.97	5.59	71.15	0.00	0.00	3.77
○Thread 570039			3.26	3.78	86.38	1.80	0.01	3.55
○Thread 570040			3.23	3.78	85.58	1.80	0.01	3.55
○Thread 570041			3.31	3.78	87.69	1.80	0.01	3.54
○Thread 570042			3.37	3.78	89.27	1.80	0.01	3.55
○Thread 570043			3.26	3.78	86.23	1.80	0.00	3.55
○Thread 570044			3.28	3.78	86.88	1.80	0.00	3.55
○Thread 570045			3.29	3.78	87.14	1.80	0.00	3.54
○Thread 570046			3.33	3.78	87.93	1.80	0.00	3.54
○Thread 570047			3.33	3.78	87.93	1.80	0.00	3.55
○Thread 570048			3.26	3.78	86.32	1.80	0.00	3.55
○Thread 570049			3.30	3.78	87.24	1.80	0.00	3.55
○Thread 570050			3.40	3.78	89.88	1.80	0.00	3.54
○Thread 570051			3.30	3.78	87.23	1.80	0.00	3.55
○Thread 570052			3.32	3.78	87.75	1.80	0.00	3.54
○Thread 570053			3.35	3.78	88.54	1.80	0.00	3.54

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	3.48
►Process 570073+		24	3.48
○Thread 570073			3.48	5.07	68.59	0.00	0.00	3.25
○Thread 570172			2.63	3.25	80.98	1.81	0.01	3.02
○Thread 570173			2.66	3.25	81.74	1.81	0.01	3.02
○Thread 570174			2.72	3.25	83.74	1.81	0.01	3.02
○Thread 570175			2.64	3.25	81.12	1.81	0.01	3.02
○Thread 570176			2.60	3.25	79.89	1.81	0.01	3.02
○Thread 570177			2.71	3.25	83.41	1.81	0.01	3.02
○Thread 570178			2.70	3.26	83.10	1.81	0.01	3.02
○Thread 570179			2.76	3.26	84.94	1.81	0.01	3.02
○Thread 570180			2.64	3.26	81.25	1.81	0.01	3.02
○Thread 570181			2.67	3.26	82.01	1.81	0.01	3.01
○Thread 570182			2.71	3.26	83.23	1.81	0.00	3.01
○Thread 570183			2.74	3.26	84.30	1.81	0.00	3.00
○Thread 570184			2.71	3.26	83.07	1.81	0.00	3.02
○Thread 570185			2.73	3.26	83.83	1.81	0.00	3.01
○Thread 570186			2.73	3.26	83.83	1.81	0.00	3.01
○Thread 570187			2.78	3.26	85.51	1.81	0.00	3.02
○Thread 570188			2.69	3.26	82.74	1.81	0.00	3.02
○Thread 570189			2.67	3.26	82.12	1.81	0.00	3.02
○Thread 570190			2.67	3.26	81.96	1.81	0.00	3.01
○Thread 570191			2.71	3.26	83.34	1.81	0.00	3.02
○Thread 570192			2.70	3.26	82.87	1.81	0.00	3.02
○Thread 570193			2.65	3.26	81.33	1.81	0.00	3.02
○Thread 570194			2.63	3.26	80.87	1.81	0.00	3.02

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	3.40
►Process 570263+		32	3.40
○Thread 570263			3.40	5.10	66.76	0.00	0.00	3.27
○Thread 570362			2.58	3.28	78.65	1.81	0.01	3.01
○Thread 570363			2.64	3.28	80.63	1.81	0.01	3.04
○Thread 570364			2.65	3.28	80.93	1.81	0.01	3.03
○Thread 570365			2.64	3.28	80.62	1.81	0.01	3.04
○Thread 570366			2.62	3.28	80.01	1.81	0.01	3.04
○Thread 570367			2.61	3.28	79.70	1.81	0.01	3.04
○Thread 570368			2.64	3.28	80.45	1.81	0.01	3.04
○Thread 570369			2.63	3.28	80.14	1.81	0.01	3.04
○Thread 570370			2.62	3.28	79.99	1.81	0.01	3.04
○Thread 570371			2.67	3.28	81.51	1.81	0.01	3.04
○Thread 570372			2.65	3.28	80.89	1.81	0.01	3.04
○Thread 570373			2.82	3.28	85.91	1.81	0.01	3.03
○Thread 570374			2.60	3.28	79.36	1.81	0.01	3.04
○Thread 570375			2.65	3.28	80.88	1.81	0.01	3.04
○Thread 570376			2.69	3.28	82.09	1.81	0.01	3.04
○Thread 570377			2.62	3.28	79.95	1.81	0.01	3.04
○Thread 570378			2.63	3.28	80.10	1.81	0.01	3.04
○Thread 570379			2.58	3.28	78.72	1.81	0.01	3.03
○Thread 570380			2.60	3.28	79.33	1.81	0.00	3.04
○Thread 570381			2.58	3.28	78.56	1.81	0.00	3.04
○Thread 570382			2.59	3.28	79.01	1.81	0.00	3.04
○Thread 570383			2.52	3.28	76.88	1.81	0.00	3.04
○Thread 570384			2.65	3.28	80.68	1.81	0.00	3.04
○Thread 570385			2.64	3.28	80.37	1.81	0.00	3.04
○Thread 570386			2.61	3.29	79.60	1.81	0.00	3.04
○Thread 570387			2.54	3.29	77.31	1.81	0.00	3.04
○Thread 570388			2.60	3.29	79.28	1.81	0.00	3.04
○Thread 570389			2.68	3.29	81.72	1.81	0.00	3.04
○Thread 570390			2.56	3.29	78.06	1.81	0.00	3.04
○Thread 570391			2.59	3.29	78.97	1.81	0.00	3.04
○Thread 570392			2.59	3.29	78.96	1.81	0.00	3.04

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	3.30
►Process 570412+		40	3.30
○Thread 570412			3.30	5.18	63.83	0.00	0.00	3.33
○Thread 570511			2.52	3.35	75.31	1.82	0.01	3.11
○Thread 570512			2.60	3.35	77.84	1.82	0.01	3.11
○Thread 570513			2.66	3.35	79.48	1.82	0.01	3.11
○Thread 570514			2.64	3.35	79.03	1.82	0.01	3.11
○Thread 570515			2.66	3.35	79.47	1.82	0.01	3.09
○Thread 570516			2.60	3.35	77.68	1.82	0.01	3.11
○Thread 570517			2.59	3.35	77.52	1.82	0.01	3.11
○Thread 570518			2.62	3.35	78.26	1.82	0.01	3.11
○Thread 570519			2.64	3.35	78.85	1.82	0.01	3.11
○Thread 570520			2.52	3.35	75.26	1.82	0.01	3.11
○Thread 570521			2.53	3.35	75.71	1.82	0.01	3.11
○Thread 570522			2.70	3.35	80.78	1.82	0.01	3.11
○Thread 570523			2.66	3.35	79.43	1.82	0.01	3.11
○Thread 570524			2.67	3.35	79.72	1.82	0.01	3.11
○Thread 570525			2.61	3.35	77.93	1.82	0.01	3.10
○Thread 570526			2.59	3.35	77.32	1.82	0.01	3.09
○Thread 570527			2.65	3.35	79.26	1.82	0.01	3.11
○Thread 570528			2.59	3.35	77.46	1.82	0.01	3.11
○Thread 570529			2.64	3.35	78.95	1.82	0.01	3.10
○Thread 570530			2.56	3.35	76.56	1.82	0.01	3.11
○Thread 570531			2.64	3.35	78.94	1.82	0.01	3.11
○Thread 570532			2.56	3.35	76.40	1.82	0.01	3.11
○Thread 570533			2.56	3.35	76.39	1.82	0.01	3.11
○Thread 570534			2.62	3.35	78.33	1.82	0.01	3.10
○Thread 570535			2.60	3.35	77.73	1.82	0.01	3.11
○Thread 570536			2.64	3.35	78.92	1.82	0.01	3.10
○Thread 570537			2.61	3.35	78.02	1.82	0.00	3.10
○Thread 570538			2.59	3.35	77.42	1.82	0.00	3.11
○Thread 570539			2.64	3.35	78.75	1.82	0.00	3.11
○Thread 570540			2.59	3.35	77.26	1.82	0.00	3.11
○Thread 570541			2.62	3.35	78.15	1.82	0.00	3.11
○Thread 570542			2.83	3.35	84.40	1.82	0.00	3.11
○Thread 570543			2.53	3.35	75.45	1.82	0.00	3.11
○Thread 570544			2.63	3.35	78.58	1.82	0.00	3.11
○Thread 570545			2.86	3.35	85.28	1.82	0.00	3.11
○Thread 570546			2.63	3.35	78.27	1.82	0.00	3.11
○Thread 570547			2.61	3.35	77.82	1.82	0.00	3.11
○Thread 570548			2.56	3.35	76.32	1.82	0.00	3.10
○Thread 570549			2.57	3.35	76.62	1.82	0.00	3.10

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	3.17
►Process 570570+		48	3.17
○Thread 570570			3.17	5.13	61.85	0.00	0.00	3.30
○Thread 570669			2.56	3.31	77.41	1.80	0.02	3.07
○Thread 570670			2.55	3.31	76.96	1.80	0.02	3.07
○Thread 570671			2.49	3.31	75.29	1.80	0.02	3.07
○Thread 570672			2.51	3.31	75.89	1.80	0.02	3.07
○Thread 570673			2.46	3.31	74.23	1.80	0.02	3.07
○Thread 570674			2.49	3.31	75.13	1.80	0.02	3.07
○Thread 570675			2.53	3.31	76.33	1.80	0.01	3.07
○Thread 570676			2.53	3.31	76.33	1.80	0.01	3.06
○Thread 570677			2.56	3.31	77.38	1.80	0.01	3.07
○Thread 570678			2.50	3.32	75.41	1.80	0.01	3.07
○Thread 570679			2.45	3.32	73.90	1.80	0.01	3.07
○Thread 570680			2.67	3.32	80.68	1.81	0.01	3.07
○Thread 570681			2.52	3.32	76.15	1.81	0.01	3.07
○Thread 570682			2.52	3.32	75.99	1.81	0.01	3.07
○Thread 570683			2.47	3.32	74.63	1.81	0.01	3.07
○Thread 570684			2.56	3.32	77.19	1.81	0.01	3.06
○Thread 570685			2.49	3.32	75.07	1.81	0.01	3.07
○Thread 570686			2.44	3.32	73.71	1.81	0.01	3.07
○Thread 570687			2.50	3.32	75.51	1.81	0.01	3.07
○Thread 570688			2.57	3.32	77.62	1.81	0.01	3.07
○Thread 570689			2.58	3.32	77.77	1.81	0.01	3.07
○Thread 570690			2.53	3.32	76.25	1.81	0.01	3.07
○Thread 570691			2.54	3.32	76.55	1.81	0.01	3.07
○Thread 570692			2.46	3.32	74.29	1.81	0.01	3.07
○Thread 570693			2.46	3.32	74.28	1.81	0.01	3.07
○Thread 570694			2.48	3.32	74.73	1.81	0.01	3.07
○Thread 570695			2.57	3.32	77.59	1.81	0.01	3.07
○Thread 570696			2.47	3.32	74.42	1.81	0.01	3.07
○Thread 570697			2.51	3.32	75.77	1.81	0.01	3.07
○Thread 570698			2.48	3.32	74.86	1.81	0.01	3.06
○Thread 570699			2.40	3.32	72.44	1.81	0.01	3.07
○Thread 570700			2.52	3.32	75.90	1.81	0.01	3.06
○Thread 570701			2.52	3.32	75.90	1.81	0.01	3.07
○Thread 570702			2.53	3.32	76.20	1.81	0.01	3.06
○Thread 570703			2.52	3.32	75.89	1.81	0.00	3.07
○Thread 570704			2.50	3.32	75.43	1.81	0.00	3.07
○Thread 570705			2.48	3.32	74.83	1.81	0.00	3.07
○Thread 570706			2.49	3.32	75.12	1.81	0.00	3.07
○Thread 570707			2.53	3.32	76.17	1.81	0.00	3.07
○Thread 570708			2.53	3.32	76.32	1.81	0.00	3.07
○Thread 570709			2.44	3.32	73.45	1.81	0.00	3.07
○Thread 570710			2.52	3.32	76.01	1.81	0.00	3.07
○Thread 570711			2.57	3.32	77.51	1.81	0.00	3.07
○Thread 570712			2.56	3.32	77.05	1.81	0.00	3.07
○Thread 570713			2.42	3.32	72.98	1.81	0.00	3.06
○Thread 570714			2.44	3.32	73.43	1.81	0.00	3.06
○Thread 570715			2.35	3.32	70.87	1.81	0.00	3.07

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	3.23
►Process 570736+		56	3.23
○Thread 570736			3.23	5.16	62.68	0.00	0.00	3.31
○Thread 570835			2.49	3.32	75.17	1.82	0.02	3.06
○Thread 570836			2.51	3.32	75.77	1.82	0.02	3.07
○Thread 570837			2.56	3.32	77.27	1.82	0.02	3.07
○Thread 570838			2.50	3.32	75.46	1.82	0.02	3.07
○Thread 570839			2.43	3.32	73.20	1.82	0.02	3.08
○Thread 570840			2.43	3.32	73.34	1.82	0.02	3.08
○Thread 570841			2.39	3.32	72.13	1.82	0.02	3.08
○Thread 570842			2.47	3.32	74.39	1.82	0.02	3.06
○Thread 570843			2.42	3.32	72.88	1.82	0.02	3.08
○Thread 570844			2.37	3.32	71.37	1.82	0.02	3.08
○Thread 570845			2.44	3.32	73.61	1.82	0.02	3.08
○Thread 570846			2.45	3.32	73.75	1.82	0.02	3.08
○Thread 570847			2.48	3.32	74.80	1.82	0.01	3.07
○Thread 570848			2.47	3.32	74.35	1.82	0.01	3.04
○Thread 570849			2.43	3.32	73.14	1.82	0.01	3.07
○Thread 570850			2.56	3.32	77.05	1.82	0.01	3.08
○Thread 570851			2.46	3.32	74.18	1.82	0.01	3.06
○Thread 570852			2.49	3.32	75.08	1.82	0.01	3.08
○Thread 570853			2.38	3.32	71.77	1.82	0.01	3.07
○Thread 570854			2.31	3.32	69.66	1.82	0.01	3.08
○Thread 570855			2.47	3.32	74.47	1.82	0.01	3.08
○Thread 570856			2.43	3.32	73.11	1.83	0.01	3.08
○Thread 570857			2.37	3.32	71.45	1.83	0.01	3.07
○Thread 570858			2.41	3.32	72.65	1.83	0.01	3.07
○Thread 570859			2.45	3.32	73.85	1.83	0.01	3.08
○Thread 570860			2.43	3.32	73.24	1.83	0.01	3.08
○Thread 570861			2.50	3.32	75.34	1.83	0.01	3.08
○Thread 570862			2.51	3.33	75.64	1.83	0.01	3.08
○Thread 570863			2.47	3.33	74.28	1.83	0.01	3.08
○Thread 570864			2.45	3.33	73.67	1.83	0.01	3.08
○Thread 570865			2.38	3.33	71.57	1.83	0.01	3.08
○Thread 570866			2.48	3.33	74.57	1.83	0.01	3.07
○Thread 570867			2.43	3.33	73.06	1.83	0.01	3.07
○Thread 570868			2.49	3.33	74.86	1.83	0.01	3.07
○Thread 570869			2.46	3.33	73.95	1.83	0.01	3.07
○Thread 570870			2.44	3.33	73.50	1.83	0.01	3.08
○Thread 570871			2.51	3.33	75.60	1.83	0.01	3.08
○Thread 570872			2.49	3.33	74.99	1.83	0.01	3.08
○Thread 570873			2.43	3.33	73.03	1.83	0.01	3.07
○Thread 570874			2.54	3.33	76.48	1.83	0.01	3.07
○Thread 570875			2.49	3.33	74.83	1.83	0.01	3.08
○Thread 570876			2.47	3.33	74.22	1.83	0.01	3.08
○Thread 570877			2.44	3.33	73.31	1.83	0.00	3.07
○Thread 570878			2.39	3.33	71.81	1.83	0.00	3.07
○Thread 570879			2.46	3.33	73.91	1.83	0.00	3.08
○Thread 570880			2.43	3.33	73.00	1.83	0.00	3.07
○Thread 570881			2.42	3.33	72.69	1.83	0.00	3.07
○Thread 570882			2.76	3.33	82.90	1.83	0.00	3.08
○Thread 570883			2.48	3.33	74.64	1.83	0.00	3.07
○Thread 570884			2.55	3.33	76.58	1.83	0.00	3.08
○Thread 570885			2.73	3.33	81.99	1.83	0.00	3.07
○Thread 570886			2.49	3.33	74.92	1.83	0.00	3.07
○Thread 570887			2.38	3.33	71.62	1.83	0.00	3.07
○Thread 570888			2.40	3.33	72.21	1.83	0.00	3.06
○Thread 570889			2.50	3.33	75.06	1.83	0.00	3.08

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	3.23
►Process 570912+		64	3.23
○Thread 570912			3.23	5.17	62.52	0.00	0.00	3.29
○Thread 571011			2.49	3.32	75.02	1.82	0.02	3.07
○Thread 571012			2.40	3.32	72.30	1.82	0.02	3.07
○Thread 571013			2.42	3.32	72.90	1.82	0.02	3.06
○Thread 571014			2.34	3.32	70.64	1.82	0.02	3.07
○Thread 571015			2.37	3.32	71.37	1.82	0.02	3.08
○Thread 571016			2.42	3.32	73.02	1.82	0.02	3.07
○Thread 571017			2.47	3.32	74.52	1.83	0.02	3.08
○Thread 571018			2.40	3.32	72.41	1.83	0.02	3.07
○Thread 571019			2.33	3.32	70.15	1.83	0.02	3.08
○Thread 571020			2.35	3.32	70.75	1.83	0.02	3.07
○Thread 571021			2.35	3.32	70.89	1.83	0.02	3.07
○Thread 571022			2.45	3.32	73.90	1.83	0.02	3.07
○Thread 571023			2.34	3.32	70.58	1.83	0.02	3.08
○Thread 571024			2.35	3.32	70.88	1.83	0.02	3.07
○Thread 571025			2.39	3.32	71.93	1.83	0.02	3.07
○Thread 571026			2.29	3.32	68.91	1.83	0.02	3.07
○Thread 571027			2.31	3.32	69.66	1.83	0.02	3.08
○Thread 571028			2.36	3.32	71.16	1.83	0.02	3.08
○Thread 571029			2.37	3.32	71.30	1.83	0.02	3.08
○Thread 571030			2.44	3.32	73.41	1.83	0.02	3.08
○Thread 571031			2.30	3.32	69.19	1.83	0.02	3.08
○Thread 571032			2.35	3.32	70.69	1.83	0.02	3.08
○Thread 571033			2.37	3.32	71.28	1.83	0.01	3.08
○Thread 571034			2.39	3.32	71.88	1.83	0.01	3.08
○Thread 571035			2.39	3.33	72.03	1.83	0.01	3.08
○Thread 571036			2.33	3.33	70.07	1.83	0.01	3.08
○Thread 571037			2.38	3.33	71.57	1.83	0.01	3.08
○Thread 571038			2.45	3.33	73.67	1.83	0.01	3.06
○Thread 571039			2.40	3.33	72.16	1.83	0.01	3.07
○Thread 571040			2.35	3.33	70.65	1.83	0.01	3.06
○Thread 571041			2.28	3.33	68.54	1.83	0.01	3.08
○Thread 571042			2.56	3.33	76.96	1.83	0.01	3.06
○Thread 571043			2.33	3.33	70.19	1.83	0.01	3.07
○Thread 571044			2.35	3.33	70.78	1.83	0.01	3.08
○Thread 571045			2.37	3.33	71.38	1.83	0.01	3.07
○Thread 571046			2.47	3.33	74.23	1.83	0.01	3.08
○Thread 571047			2.34	3.33	70.32	1.83	0.01	3.07
○Thread 571048			2.46	3.33	74.07	1.83	0.01	3.08
○Thread 571049			2.44	3.33	73.32	1.83	0.01	3.08
○Thread 571050			2.45	3.33	73.61	1.83	0.01	3.07
○Thread 571051			2.40	3.33	72.10	1.83	0.01	3.08
○Thread 571052			2.39	3.33	71.80	1.83	0.01	3.07
○Thread 571053			2.37	3.33	71.34	1.83	0.01	3.08
○Thread 571054			2.42	3.33	72.84	1.83	0.01	3.08
○Thread 571055			2.36	3.33	70.88	1.83	0.01	3.07
○Thread 571056			2.37	3.33	71.33	1.83	0.01	3.06
○Thread 571057			2.39	3.33	71.78	1.83	0.01	3.07
○Thread 571058			2.71	3.33	81.53	1.83	0.01	3.07
○Thread 571059			2.40	3.33	72.07	1.83	0.01	3.07
○Thread 571060			2.43	3.33	73.11	1.83	0.01	3.07
○Thread 571061			2.42	3.33	72.81	1.83	0.01	3.07
○Thread 571062			2.40	3.33	72.05	1.83	0.00	3.07
○Thread 571063			2.37	3.33	71.30	1.83	0.00	3.08
○Thread 571064			2.29	3.33	68.89	1.83	0.00	3.08
○Thread 571065			2.35	3.33	70.54	1.83	0.00	3.07
○Thread 571066			2.43	3.33	72.93	1.83	0.00	3.07
○Thread 571067			2.40	3.33	72.03	1.83	0.00	3.07
○Thread 571068			2.37	3.33	71.12	1.83	0.00	3.07
○Thread 571069			2.45	3.33	73.67	1.83	0.00	3.07
○Thread 571070			2.43	3.33	73.06	1.83	0.00	3.07
○Thread 571071			2.39	3.33	71.86	1.83	0.00	3.07
○Thread 571072			2.29	3.33	68.85	1.83	0.00	3.08
○Thread 571073			2.44	3.33	73.20	1.83	0.00	3.06

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	3.14
►Process 571141+		72	3.14
○Thread 571141			3.14	5.25	59.86	0.00	0.00	3.38
○Thread 571240			2.46	3.41	72.25	1.81	0.03	3.16
○Thread 571241			2.35	3.41	68.87	1.81	0.02	3.16
○Thread 571242			2.41	3.41	70.63	1.81	0.02	3.16
○Thread 571243			2.44	3.41	71.50	1.81	0.02	3.16
○Thread 571244			2.40	3.41	70.47	1.81	0.02	3.17
○Thread 571245			2.43	3.41	71.35	1.81	0.02	3.16
○Thread 571246			2.37	3.41	69.44	1.81	0.02	3.16
○Thread 571247			2.43	3.41	71.19	1.81	0.02	3.15
○Thread 571248			2.39	3.41	70.01	1.81	0.02	3.16
○Thread 571249			2.43	3.41	71.33	1.81	0.02	3.16
○Thread 571250			2.40	3.41	70.30	1.81	0.02	3.16
○Thread 571251			2.38	3.41	69.86	1.81	0.02	3.17
○Thread 571252			2.28	3.41	66.78	1.81	0.02	3.16
○Thread 571253			2.35	3.41	68.82	1.81	0.02	3.16
○Thread 571254			2.42	3.41	71.01	1.81	0.02	3.16
○Thread 571255			2.43	3.42	71.30	1.81	0.02	3.16
○Thread 571256			2.38	3.42	69.69	1.81	0.02	3.15
○Thread 571257			2.34	3.42	68.51	1.81	0.02	3.16
○Thread 571258			2.39	3.42	70.12	1.81	0.02	3.15
○Thread 571259			2.43	3.42	71.14	1.81	0.02	3.17
○Thread 571260			2.39	3.42	69.96	1.81	0.02	3.13
○Thread 571261			2.44	3.42	71.57	1.81	0.02	3.17
○Thread 571262			2.39	3.42	69.95	1.81	0.02	3.16
○Thread 571263			2.34	3.42	68.63	1.81	0.02	3.16
○Thread 571264			2.32	3.42	67.89	1.81	0.02	3.15
○Thread 571265			2.48	3.42	72.57	1.81	0.02	3.17
○Thread 571266			2.45	3.42	71.84	1.81	0.02	3.17
○Thread 571267			2.38	3.42	69.64	1.81	0.02	3.16
○Thread 571268			2.28	3.42	66.71	1.81	0.02	3.17
○Thread 571269			2.40	3.42	70.21	1.81	0.02	3.13
○Thread 571270			2.31	3.42	67.58	1.81	0.01	3.17
○Thread 571271			2.48	3.42	72.69	1.81	0.01	3.16
○Thread 571272			2.38	3.42	69.62	1.81	0.01	3.15
○Thread 571273			2.44	3.42	71.51	1.81	0.01	3.16
○Thread 571274			2.36	3.42	69.02	1.81	0.01	3.16
○Thread 571275			2.47	3.42	72.23	1.81	0.01	3.16
○Thread 571276			2.32	3.42	67.99	1.81	0.01	3.16
○Thread 571277			2.33	3.42	68.13	1.81	0.01	3.16
○Thread 571278			2.45	3.42	71.78	1.81	0.01	3.17
○Thread 571279			2.43	3.42	71.05	1.81	0.01	3.16
○Thread 571280			2.37	3.42	69.29	1.81	0.01	3.15
○Thread 571281			2.38	3.42	69.72	1.81	0.01	3.16
○Thread 571282			2.42	3.42	70.89	1.81	0.01	3.16
○Thread 571283			2.51	3.42	73.37	1.81	0.01	3.15
○Thread 571284			2.37	3.42	69.27	1.81	0.01	3.16
○Thread 571285			2.32	3.42	67.95	1.81	0.01	3.16
○Thread 571286			2.30	3.42	67.36	1.81	0.01	3.12
○Thread 571287			2.42	3.42	70.86	1.81	0.01	3.16
○Thread 571288			2.41	3.42	70.42	1.81	0.01	3.17
○Thread 571289			2.38	3.42	69.68	1.81	0.01	3.16
○Thread 571290			2.42	3.42	70.85	1.81	0.01	3.16
○Thread 571291			2.48	3.42	72.45	1.81	0.01	3.16
○Thread 571292			2.36	3.42	68.94	1.81	0.01	3.16
○Thread 571293			2.37	3.42	69.37	1.82	0.01	3.17
○Thread 571294			2.42	3.42	70.68	1.82	0.01	3.16
○Thread 571295			2.51	3.42	73.31	1.82	0.01	3.16
○Thread 571296			2.39	3.42	69.95	1.82	0.01	3.15
○Thread 571297			2.37	3.42	69.21	1.82	0.01	3.16
○Thread 571298			2.48	3.42	72.56	1.82	0.01	3.15
○Thread 571299			2.55	3.42	74.60	1.82	0.00	3.15
○Thread 571300			2.42	3.42	70.80	1.82	0.00	3.16
○Thread 571301			2.33	3.43	68.03	1.82	0.00	3.16
○Thread 571302			2.40	3.43	70.06	1.82	0.00	3.16
○Thread 571303			2.81	3.43	82.03	1.82	0.00	3.17
○Thread 571304			2.49	3.43	72.68	1.82	0.00	3.16
○Thread 571305			2.64	3.43	77.20	1.82	0.00	3.17
○Thread 571306			2.73	3.43	79.68	1.82	0.00	3.16
○Thread 571307			2.53	3.43	73.83	1.82	0.00	3.16
○Thread 571308			2.39	3.43	69.89	1.82	0.00	3.12
○Thread 571309			2.43	3.43	70.91	1.82	0.00	3.16
○Thread 571310			2.37	3.43	69.15	1.82	0.00	3.17

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	3.17
►Process 571330+		80	3.17
○Thread 571330			3.17	5.33	59.59	0.00	0.00	3.44
○Thread 571429			2.39	3.46	69.14	1.83	0.03	3.21
○Thread 571430			2.42	3.46	70.00	1.84	0.03	3.21
○Thread 571431			2.41	3.46	69.57	1.84	0.03	3.21
○Thread 571432			2.35	3.46	67.83	1.84	0.03	3.21
○Thread 571433			2.38	3.46	68.69	1.84	0.03	3.21
○Thread 571434			2.34	3.46	67.68	1.84	0.03	3.20
○Thread 571435			2.31	3.47	66.81	1.84	0.03	3.20
○Thread 571436			2.38	3.47	68.82	1.84	0.03	3.21
○Thread 571437			2.30	3.47	66.37	1.84	0.03	3.21
○Thread 571438			2.36	3.47	68.24	1.84	0.03	3.21
○Thread 571439			2.35	3.47	67.80	1.84	0.03	3.21
○Thread 571440			2.34	3.47	67.65	1.84	0.03	3.21
○Thread 571441			2.32	3.47	67.07	1.84	0.03	3.21
○Thread 571442			2.26	3.47	65.19	1.84	0.02	3.21
○Thread 571443			2.37	3.47	68.50	1.84	0.02	3.21
○Thread 571444			2.35	3.47	67.92	1.84	0.02	3.21
○Thread 571445			2.33	3.47	67.05	1.84	0.02	3.21
○Thread 571446			2.32	3.47	66.90	1.84	0.02	3.20
○Thread 571447			2.29	3.47	66.18	1.84	0.02	3.22
○Thread 571448			2.32	3.47	66.89	1.84	0.02	3.21
○Thread 571449			2.25	3.47	64.87	1.84	0.02	3.21
○Thread 571450			2.37	3.47	68.33	1.84	0.02	3.21
○Thread 571451			2.40	3.47	69.33	1.84	0.02	3.21
○Thread 571452			2.43	3.47	70.19	1.84	0.02	3.21
○Thread 571453			2.26	3.47	65.14	1.84	0.02	3.22
○Thread 571454			2.37	3.47	68.16	1.84	0.02	3.20
○Thread 571455			2.39	3.47	69.02	1.84	0.02	3.20
○Thread 571456			2.33	3.47	67.29	1.84	0.02	3.21
○Thread 571457			2.32	3.47	67.00	1.84	0.02	3.21
○Thread 571458			2.24	3.47	64.54	1.84	0.02	3.20
○Thread 571459			2.32	3.47	66.99	1.84	0.02	3.21
○Thread 571460			2.38	3.47	68.71	1.84	0.02	3.21
○Thread 571461			2.32	3.47	66.98	1.84	0.02	3.21
○Thread 571462			2.23	3.47	64.38	1.84	0.02	3.21
○Thread 571463			2.34	3.47	67.40	1.84	0.02	3.21
○Thread 571464			2.34	3.47	67.54	1.84	0.02	3.21
○Thread 571465			2.37	3.47	68.40	1.84	0.02	3.21
○Thread 571466			2.30	3.47	66.38	1.84	0.02	3.21
○Thread 571467			2.28	3.47	65.80	1.84	0.02	3.21
○Thread 571468			2.35	3.47	67.67	1.84	0.02	3.21
○Thread 571469			2.27	3.47	65.50	1.84	0.01	3.21
○Thread 571470			2.33	3.47	66.94	1.84	0.01	3.20
○Thread 571471			2.33	3.47	67.22	1.84	0.01	3.21
○Thread 571472			2.42	3.47	69.66	1.84	0.01	3.21
○Thread 571473			2.32	3.47	66.78	1.84	0.01	3.21
○Thread 571474			2.40	3.47	69.22	1.84	0.01	3.21
○Thread 571475			2.28	3.47	65.62	1.84	0.01	3.21
○Thread 571476			2.54	3.47	73.10	1.84	0.01	3.21
○Thread 571477			2.31	3.48	66.62	1.84	0.01	3.20
○Thread 571478			2.42	3.48	69.63	1.84	0.01	3.21
○Thread 571479			2.42	3.48	69.77	1.84	0.01	3.21
○Thread 571480			2.39	3.48	68.91	1.84	0.01	3.21
○Thread 571481			2.27	3.48	65.30	1.84	0.01	3.21
○Thread 571482			2.35	3.48	67.60	1.84	0.01	3.21
○Thread 571483			2.41	3.48	69.47	1.84	0.01	3.20
○Thread 571484			2.43	3.48	70.04	1.84	0.01	3.21
○Thread 571485			2.39	3.48	68.74	1.84	0.01	3.20
○Thread 571486			2.31	3.48	66.58	1.84	0.01	3.20
○Thread 571487			2.37	3.48	68.30	1.84	0.01	3.21
○Thread 571488			2.51	3.48	72.32	1.84	0.01	3.21
○Thread 571489			2.47	3.48	71.16	1.84	0.01	3.21
○Thread 571490			2.34	3.48	67.28	1.84	0.01	3.21
○Thread 571491			2.31	3.48	66.41	1.84	0.01	3.21
○Thread 571492			2.70	3.48	77.61	1.84	0.01	3.21
○Thread 571493			2.42	3.48	69.70	1.84	0.01	3.21
○Thread 571494			2.40	3.48	68.98	1.84	0.01	3.22
○Thread 571495			2.43	3.48	69.98	1.84	0.01	3.22
○Thread 571496			2.53	3.48	72.85	1.84	0.00	3.22
○Thread 571497			2.32	3.48	66.81	1.84	0.00	3.21
○Thread 571498			2.39	3.48	68.82	1.84	0.00	3.22
○Thread 571499			2.46	3.48	70.68	1.84	0.00	3.22
○Thread 571500			2.49	3.48	71.68	1.84	0.00	3.21
○Thread 571501			2.42	3.48	69.52	1.84	0.00	3.22
○Thread 571502			2.34	3.48	67.36	1.84	0.00	3.20
○Thread 571503			2.40	3.48	69.08	1.84	0.00	3.22
○Thread 571504			2.65	3.48	76.11	1.84	0.00	3.21
○Thread 571505			2.46	3.48	70.80	1.84	0.00	3.21
○Thread 571506			2.40	3.48	69.07	1.84	0.00	3.22
○Thread 571507			2.40	3.48	68.92	1.84	0.00	3.22

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	3.32
►Process 571527+		88	3.32
○Thread 571527			3.32	5.48	60.64	0.00	0.00	3.60
○Thread 571626			2.32	3.63	64.12	1.82	0.03	3.37
○Thread 571627			2.57	3.63	70.87	1.82	0.03	3.38
○Thread 571628			2.45	3.63	67.70	1.82	0.03	3.38
○Thread 571629			2.55	3.63	70.31	1.82	0.03	3.36
○Thread 571630			2.42	3.63	66.86	1.82	0.03	3.36
○Thread 571631			2.49	3.63	68.79	1.82	0.03	3.38
○Thread 571632			2.39	3.63	65.89	1.82	0.03	3.38
○Thread 571633			2.46	3.63	67.95	1.82	0.03	3.37
○Thread 571634			2.41	3.63	66.57	1.82	0.03	3.38
○Thread 571635			2.30	3.63	63.39	1.82	0.03	3.38
○Thread 571636			2.34	3.63	64.63	1.82	0.03	3.37
○Thread 571637			2.55	3.63	70.41	1.82	0.03	3.37
○Thread 571638			2.48	3.63	68.34	1.82	0.03	3.38
○Thread 571639			2.54	3.63	70.13	1.82	0.03	3.36
○Thread 571640			2.50	3.63	69.02	1.82	0.03	3.37
○Thread 571641			2.45	3.63	67.64	1.82	0.03	3.37
○Thread 571642			2.42	3.63	66.67	1.82	0.03	3.37
○Thread 571643			2.47	3.63	68.05	1.82	0.03	3.34
○Thread 571644			2.40	3.63	66.11	1.82	0.03	3.37
○Thread 571645			2.44	3.63	67.35	1.82	0.03	3.36
○Thread 571646			2.33	3.63	64.04	1.82	0.03	3.38
○Thread 571647			2.31	3.63	63.76	1.82	0.03	3.38
○Thread 571648			2.39	3.63	65.82	1.82	0.03	3.38
○Thread 571649			2.39	3.63	65.96	1.82	0.02	3.38
○Thread 571650			2.43	3.63	67.05	1.82	0.02	3.38
○Thread 571651			2.43	3.63	67.05	1.82	0.02	3.38
○Thread 571652			2.36	3.63	65.12	1.82	0.02	3.38
○Thread 571653			2.41	3.63	66.49	1.82	0.02	3.38
○Thread 571654			2.46	3.63	67.86	1.82	0.02	3.38
○Thread 571655			2.27	3.63	62.49	1.82	0.02	3.38
○Thread 571656			2.39	3.63	65.93	1.82	0.02	3.36
○Thread 571657			2.33	3.63	64.27	1.82	0.02	3.38
○Thread 571658			2.28	3.63	62.89	1.82	0.02	3.38
○Thread 571659			2.39	3.63	65.78	1.82	0.02	3.37
○Thread 571660			2.38	3.63	65.64	1.82	0.02	3.37
○Thread 571661			2.36	3.63	65.08	1.82	0.02	3.36
○Thread 571662			2.45	3.63	67.55	1.82	0.02	3.38
○Thread 571663			2.28	3.63	62.87	1.82	0.02	3.37
○Thread 571664			2.47	3.63	68.10	1.82	0.02	3.38
○Thread 571665			2.56	3.63	70.43	1.82	0.02	3.38
○Thread 571666			2.41	3.64	66.30	1.82	0.02	3.38
○Thread 571667			2.40	3.64	66.02	1.82	0.02	3.38
○Thread 571668			2.40	3.64	66.15	1.82	0.02	3.38
○Thread 571669			2.39	3.64	65.87	1.82	0.02	3.37
○Thread 571670			2.49	3.64	68.62	1.82	0.02	3.38
○Thread 571671			2.46	3.64	67.79	1.82	0.02	3.38
○Thread 571672			2.40	3.64	66.00	1.82	0.02	3.37
○Thread 571673			2.47	3.64	68.06	1.82	0.02	3.37
○Thread 571674			2.37	3.64	65.17	1.82	0.02	3.36
○Thread 571675			2.39	3.64	65.71	1.82	0.02	3.35
○Thread 571676			2.53	3.64	69.69	1.82	0.02	3.37
○Thread 571677			2.33	3.64	64.19	1.82	0.01	3.37
○Thread 571678			2.37	3.64	65.15	1.82	0.01	3.38
○Thread 571679			2.32	3.64	63.77	1.82	0.01	3.38
○Thread 571680			2.40	3.64	65.96	1.82	0.01	3.38
○Thread 571681			2.46	3.64	67.75	1.82	0.01	3.37
○Thread 571682			2.43	3.64	66.78	1.82	0.01	3.38
○Thread 571683			2.47	3.64	67.88	1.82	0.01	3.38
○Thread 571684			2.41	3.64	66.36	1.82	0.01	3.37
○Thread 571685			2.32	3.64	63.75	1.82	0.01	3.37
○Thread 571686			2.37	3.64	65.25	1.82	0.01	3.37
○Thread 571687			2.40	3.64	66.07	1.82	0.01	3.38
○Thread 571688			2.44	3.64	67.03	1.82	0.01	3.38
○Thread 571689			2.61	3.64	71.70	1.82	0.01	3.38
○Thread 571690			2.44	3.64	67.14	1.82	0.01	3.36
○Thread 571691			2.34	3.64	64.25	1.82	0.01	3.38
○Thread 571692			2.37	3.64	65.21	1.82	0.01	3.36
○Thread 571693			2.44	3.64	67.13	1.82	0.01	3.37
○Thread 571694			2.48	3.64	68.22	1.82	0.01	3.38
○Thread 571695			2.33	3.64	64.10	1.83	0.01	3.38
○Thread 571696			2.32	3.64	63.68	1.83	0.01	3.37
○Thread 571697			2.44	3.64	67.11	1.83	0.01	3.37
○Thread 571698			2.39	3.64	65.59	1.83	0.01	3.38
○Thread 571699			2.37	3.64	65.04	1.83	0.01	3.38
○Thread 571700			2.25	3.64	61.88	1.83	0.01	3.37
○Thread 571701			2.76	3.64	75.87	1.83	0.00	3.37
○Thread 571702			2.37	3.64	65.03	1.83	0.00	3.38
○Thread 571703			2.46	3.64	67.49	1.83	0.00	3.38
○Thread 571704			2.51	3.65	69.00	1.83	0.00	3.38
○Thread 571705			2.59	3.65	70.91	1.83	0.00	3.37
○Thread 571706			2.42	3.65	66.38	1.83	0.00	3.38
○Thread 571707			2.40	3.65	65.97	1.83	0.00	3.38
○Thread 571708			2.42	3.65	66.37	1.83	0.00	3.36
○Thread 571709			2.44	3.65	67.05	1.83	0.00	3.37
○Thread 571710			2.37	3.65	65.13	1.83	0.00	3.38
○Thread 571711			2.31	3.65	63.48	1.83	0.00	3.38
○Thread 571712			2.19	3.65	60.19	1.83	0.00	3.38

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	3.31
►Process 571732+		96	3.31
○Thread 571732			3.31	5.58	59.45	0.00	0.00	3.71
○Thread 571831			2.48	3.73	66.52	1.81	0.03	3.47
○Thread 571832			2.54	3.73	68.26	1.81	0.03	3.48
○Thread 571833			2.46	3.73	66.11	1.81	0.03	3.48
○Thread 571834			2.52	3.73	67.71	1.81	0.03	3.47
○Thread 571835			2.49	3.73	66.90	1.81	0.03	3.48
○Thread 571836			2.55	3.73	68.51	1.81	0.03	3.48
○Thread 571837			2.48	3.73	66.63	1.81	0.03	3.48
○Thread 571838			2.64	3.73	70.78	1.81	0.03	3.47
○Thread 571839			2.44	3.73	65.55	1.81	0.03	3.48
○Thread 571840			2.52	3.73	67.69	1.81	0.03	3.46
○Thread 571841			2.62	3.73	70.23	1.81	0.03	3.47
○Thread 571842			2.53	3.73	67.81	1.81	0.03	3.47
○Thread 571843			2.40	3.73	64.33	1.81	0.03	3.47
○Thread 571844			2.50	3.73	67.00	1.81	0.03	3.47
○Thread 571845			2.52	3.73	67.53	1.81	0.03	3.47
○Thread 571846			2.46	3.73	66.05	1.81	0.03	3.48
○Thread 571847			2.39	3.73	64.18	1.81	0.03	3.48
○Thread 571848			2.47	3.73	66.31	1.81	0.03	3.47
○Thread 571849			2.45	3.73	65.78	1.81	0.03	3.48
○Thread 571850			2.35	3.73	62.82	1.81	0.03	3.48
○Thread 571851			2.47	3.73	66.30	1.82	0.03	3.47
○Thread 571852			2.39	3.73	64.02	1.82	0.03	3.48
○Thread 571853			2.43	3.73	65.22	1.82	0.03	3.47
○Thread 571854			2.46	3.73	66.02	1.82	0.03	3.48
○Thread 571855			2.44	3.73	65.35	1.82	0.03	3.48
○Thread 571856			2.46	3.73	65.88	1.82	0.03	3.48
○Thread 571857			2.50	3.73	66.95	1.82	0.03	3.46
○Thread 571858			2.45	3.73	65.74	1.82	0.03	3.48
○Thread 571859			2.38	3.73	63.86	1.82	0.03	3.48
○Thread 571860			2.39	3.73	63.99	1.82	0.02	3.48
○Thread 571861			2.39	3.74	63.99	1.82	0.02	3.48
○Thread 571862			2.65	3.74	70.94	1.82	0.02	3.48
○Thread 571863			2.38	3.74	63.71	1.82	0.02	3.46
○Thread 571864			2.36	3.74	63.31	1.82	0.02	3.48
○Thread 571865			2.64	3.74	70.66	1.82	0.02	3.48
○Thread 571866			2.44	3.74	65.44	1.82	0.02	3.48
○Thread 571867			2.42	3.74	64.90	1.82	0.02	3.48
○Thread 571868			2.46	3.74	65.83	1.82	0.02	3.48
○Thread 571869			2.55	3.74	68.24	1.82	0.02	3.47
○Thread 571870			2.55	3.74	68.37	1.82	0.02	3.48
○Thread 571871			2.51	3.74	67.15	1.82	0.02	3.72
○Thread 571872			2.52	3.74	67.55	1.82	0.02	3.48
○Thread 571873			2.50	3.74	67.01	1.82	0.02	3.48
○Thread 571874			2.52	3.74	67.40	1.82	0.02	3.48
○Thread 571875			2.54	3.74	68.07	1.82	0.02	3.48
○Thread 571876			2.39	3.74	64.05	1.82	0.02	3.48
○Thread 571877			2.40	3.74	64.18	1.82	0.02	3.48
○Thread 571878			2.76	3.74	73.94	1.82	0.02	3.48
○Thread 571879			2.49	3.74	66.58	1.82	0.02	3.48
○Thread 571880			2.52	3.74	67.51	1.82	0.02	3.45
○Thread 571881			2.34	3.74	62.56	1.82	0.02	3.47
○Thread 571882			2.56	3.74	68.44	1.82	0.02	3.48
○Thread 571883			2.55	3.74	68.17	1.82	0.02	3.48
○Thread 571884			2.51	3.74	67.10	1.82	0.02	3.48
○Thread 571885			2.45	3.74	65.62	1.82	0.02	3.48
○Thread 571886			2.42	3.74	64.68	1.82	0.01	3.46
○Thread 571887			2.52	3.74	67.49	1.82	0.01	3.48
○Thread 571888			2.35	3.74	62.94	1.82	0.01	3.48
○Thread 571889			2.49	3.74	66.54	1.82	0.01	3.48
○Thread 571890			2.42	3.74	64.67	1.82	0.01	3.48
○Thread 571891			2.39	3.74	63.86	1.82	0.01	3.46
○Thread 571892			2.42	3.74	64.79	1.82	0.01	3.48
○Thread 571893			2.43	3.74	64.92	1.82	0.01	3.47
○Thread 571894			2.49	3.74	66.52	1.82	0.01	3.48
○Thread 571895			2.46	3.74	65.85	1.82	0.01	3.48
○Thread 571896			2.39	3.74	63.84	1.82	0.01	3.48
○Thread 571897			2.47	3.74	66.11	1.82	0.01	3.48
○Thread 571898			2.57	3.74	68.64	1.82	0.01	3.48
○Thread 571899			2.43	3.74	64.77	1.82	0.01	3.48
○Thread 571900			2.36	3.74	63.16	1.82	0.01	3.48
○Thread 571901			2.45	3.74	65.43	1.82	0.01	3.48
○Thread 571902			2.60	3.74	69.56	1.82	0.01	3.48
○Thread 571903			2.52	3.75	67.42	1.82	0.01	3.46
○Thread 571904			2.35	3.75	62.88	1.82	0.01	3.47
○Thread 571905			2.39	3.75	63.94	1.82	0.01	3.48
○Thread 571906			2.51	3.75	67.14	1.82	0.01	3.48
○Thread 571907			2.48	3.75	66.34	1.82	0.01	3.48
○Thread 571908			2.43	3.75	64.86	1.82	0.01	3.47
○Thread 571909			2.46	3.75	65.66	1.82	0.01	3.48
○Thread 571910			2.54	3.75	67.79	1.82	0.01	3.47
○Thread 571911			2.48	3.75	66.32	1.82	0.01	3.45
○Thread 571912			2.52	3.75	67.25	1.82	0.01	3.47
○Thread 571913			2.42	3.75	64.58	1.82	0.01	3.48
○Thread 571914			2.54	3.75	67.77	1.82	0.00	3.48
○Thread 571915			2.45	3.75	65.50	1.82	0.00	3.48
○Thread 571916			2.49	3.75	66.56	1.82	0.00	3.48
○Thread 571917			2.44	3.75	65.23	1.82	0.00	3.48
○Thread 571918			2.45	3.75	65.36	1.82	0.00	3.48
○Thread 571919			2.45	3.75	65.35	1.82	0.00	3.47
○Thread 571920			2.36	3.75	63.08	1.82	0.00	3.48
○Thread 571921			2.38	3.75	63.61	1.82	0.00	3.48
○Thread 571922			2.50	3.75	66.81	1.82	0.00	3.48
○Thread 571923			2.30	3.75	61.33	1.82	0.00	3.48
○Thread 571924			2.34	3.75	62.53	1.82	0.00	3.47
○Thread 571925			2.23	3.75	59.59	1.82	0.00	3.47

Report Configuration