OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	54.22
►Process 595403+		1	54.22
○Thread 595403			54.22	55.74	97.27	0.00	0.00	55.72

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	27.57
►Process 595571+		2	27.57
○Thread 595571			27.57	29.17	94.53	0.00	0.00	27.36
○Thread 595670			27.04	27.36	98.83	1.81	0.00	27.13

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	14.48
►Process 595740+		4	14.48
○Thread 595740			14.48	15.96	90.68	0.00	0.00	14.13
○Thread 595839			13.83	14.14	97.80	1.83	0.00	13.90
○Thread 595840			13.82	14.14	97.73	1.83	0.00	13.90
○Thread 595841			13.79	14.14	97.55	1.83	0.00	13.90

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	8.13
►Process 595863+		8	8.13
○Thread 595863			8.13	9.64	84.32	0.00	0.00	7.81
○Thread 595962			7.40	7.81	94.65	1.82	0.00	7.58
○Thread 595963			7.48	7.81	95.67	1.82	0.00	7.58
○Thread 595964			7.47	7.81	95.54	1.82	0.00	7.58
○Thread 595965			7.43	7.81	95.09	1.82	0.00	7.57
○Thread 595966			7.48	7.81	95.72	1.82	0.00	7.58
○Thread 595967			7.42	7.81	94.95	1.82	0.00	7.56
○Thread 595968			7.40	7.81	94.69	1.82	0.00	7.57

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	5.01
►Process 595988+		16	5.01
○Thread 595988			5.01	6.64	75.34	0.00	0.00	4.80
○Thread 596087			4.39	4.81	91.42	1.83	0.01	4.57
○Thread 596088			4.40	4.81	91.62	1.83	0.01	4.57
○Thread 596089			4.45	4.81	92.56	1.83	0.00	4.57
○Thread 596090			4.33	4.81	90.06	1.83	0.00	4.57
○Thread 596091			4.26	4.81	88.70	1.83	0.00	4.57
○Thread 596092			4.35	4.81	90.46	1.83	0.00	4.57
○Thread 596093			4.31	4.81	89.63	1.83	0.00	4.56
○Thread 596094			4.32	4.81	89.94	1.83	0.00	4.57
○Thread 596095			4.32	4.81	89.93	1.83	0.00	4.57
○Thread 596096			4.27	4.81	88.78	1.83	0.00	4.56
○Thread 596097			4.33	4.81	90.03	1.83	0.00	4.57
○Thread 596098			4.42	4.81	91.89	1.83	0.00	4.57
○Thread 596099			4.32	4.81	89.92	1.83	0.00	4.57
○Thread 596100			4.28	4.81	88.98	1.83	0.00	4.57
○Thread 596101			4.35	4.81	90.43	1.83	0.00	4.57

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	4.29
►Process 596121+		24	4.29
○Thread 596121			4.29	5.91	72.63	0.00	0.00	4.07
○Thread 596221			3.51	4.09	86.03	1.81	0.01	3.85
○Thread 596222			3.57	4.09	87.37	1.81	0.01	3.84
○Thread 596223			3.57	4.09	87.37	1.81	0.01	3.84
○Thread 596224			3.54	4.09	86.75	1.81	0.01	3.84
○Thread 596225			3.53	4.09	86.50	1.81	0.01	3.85
○Thread 596226			3.56	4.09	87.23	1.81	0.01	3.85
○Thread 596227			3.44	4.09	84.29	1.81	0.01	3.85
○Thread 596228			3.49	4.09	85.51	1.81	0.01	3.84
○Thread 596229			3.46	4.09	84.65	1.81	0.01	3.85
○Thread 596230			3.44	4.09	84.28	1.81	0.01	3.85
○Thread 596231			3.55	4.09	86.97	1.81	0.01	3.84
○Thread 596232			3.60	4.09	88.18	1.81	0.00	3.85
○Thread 596233			3.47	4.09	85.00	1.81	0.00	3.84
○Thread 596234			3.57	4.09	87.44	1.81	0.00	3.85
○Thread 596235			3.47	4.09	84.99	1.81	0.00	3.84
○Thread 596236			3.57	4.09	87.31	1.81	0.00	3.85
○Thread 596237			3.50	4.09	85.59	1.81	0.00	3.84
○Thread 596238			3.45	4.09	84.49	1.81	0.00	3.85
○Thread 596239			3.59	4.09	87.91	1.81	0.00	3.85
○Thread 596240			3.55	4.09	86.80	1.81	0.00	3.85
○Thread 596241			3.56	4.09	87.04	1.81	0.00	3.85
○Thread 596242			3.46	4.09	84.71	1.81	0.00	3.85
○Thread 596243			3.44	4.09	84.22	1.82	0.00	3.83

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	3.80
►Process 596311+		32	3.80
○Thread 596311			3.80	5.54	68.65	0.00	0.00	3.70
○Thread 596410			3.13	3.71	84.36	1.82	0.01	3.47
○Thread 596411			3.08	3.71	83.14	1.82	0.01	3.47
○Thread 596412			3.04	3.71	82.06	1.82	0.01	3.47
○Thread 596413			2.99	3.71	80.57	1.82	0.01	3.47
○Thread 596414			3.00	3.71	80.84	1.82	0.01	3.47
○Thread 596415			3.11	3.71	83.93	1.82	0.01	3.47
○Thread 596416			3.03	3.71	81.64	1.82	0.01	3.47
○Thread 596417			3.04	3.71	82.04	1.82	0.01	3.47
○Thread 596418			3.09	3.71	83.24	1.82	0.01	3.47
○Thread 596419			3.04	3.71	82.03	1.82	0.01	3.45
○Thread 596420			2.98	3.71	80.26	1.82	0.01	3.47
○Thread 596421			3.13	3.71	84.43	1.82	0.01	3.46
○Thread 596422			2.97	3.71	79.98	1.82	0.01	3.46
○Thread 596423			2.96	3.71	79.71	1.82	0.01	3.46
○Thread 596424			3.13	3.71	84.41	1.82	0.01	3.47
○Thread 596425			3.01	3.71	81.18	1.82	0.01	3.46
○Thread 596426			3.05	3.71	82.25	1.82	0.01	3.47
○Thread 596427			3.04	3.71	81.98	1.82	0.01	3.47
○Thread 596428			3.03	3.71	81.70	1.82	0.00	3.47
○Thread 596429			3.08	3.71	82.91	1.82	0.00	3.47
○Thread 596430			3.14	3.72	84.65	1.82	0.00	3.47
○Thread 596431			3.07	3.72	82.76	1.82	0.00	3.47
○Thread 596432			3.07	3.72	82.76	1.82	0.00	3.47
○Thread 596433			3.11	3.72	83.83	1.82	0.00	3.47
○Thread 596434			3.03	3.72	81.54	1.82	0.00	3.47
○Thread 596435			3.03	3.72	81.53	1.82	0.00	3.47
○Thread 596436			3.05	3.72	82.20	1.82	0.00	3.47
○Thread 596437			3.08	3.72	83.00	1.82	0.00	3.47
○Thread 596438			3.04	3.72	81.92	1.82	0.00	3.47
○Thread 596439			2.96	3.72	79.77	1.82	0.00	3.47
○Thread 596440			3.01	3.72	80.97	1.82	0.00	3.47

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	3.57
►Process 596461+		40	3.57
○Thread 596461			3.57	5.42	65.93	0.00	0.00	3.57
○Thread 596560			2.83	3.58	79.14	1.83	0.01	3.33
○Thread 596561			2.87	3.58	80.25	1.83	0.01	3.34
○Thread 596562			2.87	3.58	80.25	1.83	0.01	3.33
○Thread 596563			2.77	3.58	77.31	1.83	0.01	3.34
○Thread 596564			2.84	3.58	79.26	1.83	0.01	3.34
○Thread 596565			2.89	3.58	80.65	1.83	0.01	3.34
○Thread 596566			2.83	3.58	79.11	1.83	0.01	3.34
○Thread 596567			2.82	3.58	78.69	1.83	0.01	3.34
○Thread 596568			2.72	3.58	76.03	1.83	0.01	3.34
○Thread 596569			2.84	3.58	79.24	1.83	0.01	3.33
○Thread 596570			2.94	3.58	82.02	1.83	0.01	3.32
○Thread 596571			2.99	3.58	83.55	1.83	0.01	3.34
○Thread 596572			2.77	3.58	77.27	1.83	0.01	3.34
○Thread 596573			2.86	3.58	79.78	1.83	0.01	3.33
○Thread 596574			2.82	3.59	78.80	1.83	0.01	3.34
○Thread 596575			2.86	3.59	79.77	1.83	0.01	3.34
○Thread 596576			2.77	3.59	77.39	1.83	0.01	3.34
○Thread 596577			2.78	3.59	77.67	1.83	0.01	3.33
○Thread 596578			2.82	3.59	78.78	1.83	0.01	3.34
○Thread 596579			2.80	3.59	78.21	1.83	0.01	3.34
○Thread 596580			2.88	3.59	80.30	1.83	0.01	3.33
○Thread 596581			2.76	3.59	77.09	1.83	0.01	3.33
○Thread 596582			2.89	3.59	80.57	1.83	0.01	3.34
○Thread 596583			2.86	3.59	79.87	1.83	0.01	3.34
○Thread 596584			2.80	3.59	78.19	1.83	0.01	3.34
○Thread 596585			2.81	3.59	78.32	1.83	0.01	3.34
○Thread 596586			2.87	3.59	79.99	1.83	0.00	3.34
○Thread 596587			2.87	3.59	79.99	1.83	0.00	3.34
○Thread 596588			2.87	3.59	80.12	1.83	0.00	3.34
○Thread 596589			2.76	3.59	77.05	1.83	0.00	3.34
○Thread 596590			2.77	3.59	77.33	1.83	0.00	3.34
○Thread 596591			2.88	3.59	80.25	1.83	0.00	3.34
○Thread 596592			2.92	3.59	81.36	1.83	0.00	3.34
○Thread 596593			2.85	3.59	79.40	1.83	0.00	3.34
○Thread 596594			2.94	3.59	81.90	1.83	0.00	3.34
○Thread 596595			2.88	3.59	80.23	1.83	0.00	3.34
○Thread 596596			2.81	3.59	78.27	1.83	0.00	3.34
○Thread 596597			2.78	3.59	77.43	1.83	0.00	3.33
○Thread 596598			2.71	3.59	75.62	1.83	0.00	3.34

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	3.46
►Process 596618+		48	3.46
○Thread 596618			3.46	5.40	64.04	0.00	0.00	3.54
○Thread 596717			2.79	3.56	78.50	1.83	0.02	3.32
○Thread 596718			2.78	3.56	78.07	1.83	0.02	3.32
○Thread 596719			2.68	3.56	75.26	1.83	0.02	3.32
○Thread 596720			2.81	3.56	78.91	1.83	0.02	3.32
○Thread 596721			2.73	3.56	76.80	1.83	0.02	3.32
○Thread 596722			2.80	3.56	78.62	1.83	0.02	3.32
○Thread 596723			2.71	3.56	76.08	1.83	0.01	3.32
○Thread 596724			2.77	3.56	77.76	1.83	0.01	3.31
○Thread 596725			2.77	3.56	77.90	1.83	0.01	3.32
○Thread 596726			2.68	3.56	75.37	1.83	0.01	3.31
○Thread 596727			2.76	3.56	77.61	1.83	0.01	3.32
○Thread 596728			2.94	3.56	82.66	1.83	0.01	3.31
○Thread 596729			2.68	3.56	75.35	1.83	0.01	3.32
○Thread 596730			2.72	3.56	76.33	1.83	0.01	3.31
○Thread 596731			2.72	3.56	76.33	1.83	0.01	3.31
○Thread 596732			2.78	3.56	78.01	1.83	0.01	3.32
○Thread 596733			2.73	3.56	76.60	1.83	0.01	3.31
○Thread 596734			2.69	3.56	75.47	1.83	0.01	3.32
○Thread 596735			2.69	3.56	75.47	1.83	0.01	3.32
○Thread 596736			2.77	3.56	77.85	1.83	0.01	3.32
○Thread 596737			2.71	3.56	76.02	1.83	0.01	3.32
○Thread 596738			2.73	3.57	76.57	1.83	0.01	3.32
○Thread 596739			2.80	3.57	78.53	1.83	0.01	3.32
○Thread 596740			2.76	3.57	77.41	1.83	0.01	3.32
○Thread 596741			2.72	3.57	76.28	1.83	0.01	3.32
○Thread 596742			2.70	3.57	75.85	1.83	0.01	3.31
○Thread 596743			2.75	3.57	77.25	1.83	0.01	3.32
○Thread 596744			2.73	3.57	76.68	1.83	0.01	3.32
○Thread 596745			2.71	3.57	76.12	1.83	0.01	3.32
○Thread 596746			2.76	3.57	77.38	1.83	0.01	3.32
○Thread 596747			2.70	3.57	75.69	1.83	0.01	3.32
○Thread 596748			2.75	3.57	77.09	1.83	0.01	3.32
○Thread 596749			2.81	3.57	78.76	1.83	0.01	3.32
○Thread 596750			2.75	3.57	77.08	1.83	0.01	3.32
○Thread 596751			2.67	3.57	74.97	1.83	0.00	3.32
○Thread 596752			2.80	3.57	78.61	1.83	0.00	3.32
○Thread 596753			2.70	3.57	75.80	1.83	0.00	3.32
○Thread 596754			2.72	3.57	76.36	1.83	0.00	3.32
○Thread 596755			2.76	3.57	77.33	1.83	0.00	3.32
○Thread 596756			2.76	3.57	77.47	1.83	0.00	3.32
○Thread 596757			2.69	3.57	75.36	1.83	0.00	3.32
○Thread 596758			2.78	3.57	78.02	1.83	0.00	3.32
○Thread 596759			2.70	3.57	75.77	1.83	0.00	3.32
○Thread 596760			2.84	3.57	79.69	1.83	0.00	3.32
○Thread 596761			2.77	3.57	77.73	1.83	0.00	3.31
○Thread 596762			2.70	3.57	75.76	1.83	0.00	3.32
○Thread 596763			2.63	3.57	73.80	1.83	0.00	3.32

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	3.33
►Process 596783+		56	3.33
○Thread 596783			3.33	5.40	61.71	0.00	0.00	3.58
○Thread 596883			2.79	3.58	78.04	1.81	0.02	3.33
○Thread 596884			2.67	3.58	74.82	1.81	0.02	3.33
○Thread 596885			2.63	3.58	73.55	1.81	0.02	3.33
○Thread 596886			2.74	3.58	76.63	1.81	0.02	3.32
○Thread 596887			2.68	3.58	75.08	1.81	0.02	3.33
○Thread 596888			2.62	3.58	73.40	1.81	0.02	3.33
○Thread 596889			2.68	3.58	75.07	1.81	0.02	3.33
○Thread 596890			2.77	3.58	77.44	1.81	0.02	3.32
○Thread 596891			2.64	3.58	73.81	1.81	0.02	3.33
○Thread 596892			2.61	3.58	73.10	1.81	0.02	3.32
○Thread 596893			2.70	3.58	75.47	1.81	0.02	3.33
○Thread 596894			2.71	3.58	75.89	1.81	0.02	3.33
○Thread 596895			2.69	3.58	75.32	1.81	0.02	3.33
○Thread 596896			2.70	3.58	75.60	1.81	0.02	3.33
○Thread 596897			2.67	3.58	74.76	1.81	0.01	3.33
○Thread 596898			2.75	3.58	76.99	1.81	0.01	3.33
○Thread 596899			2.72	3.58	76.00	1.81	0.01	3.33
○Thread 596900			2.67	3.58	74.74	1.81	0.01	3.33
○Thread 596901			2.72	3.58	75.99	1.81	0.01	3.33
○Thread 596902			2.66	3.58	74.31	1.81	0.01	3.33
○Thread 596903			2.64	3.58	73.89	1.81	0.01	3.32
○Thread 596904			2.60	3.58	72.63	1.81	0.01	3.32
○Thread 596905			2.65	3.58	74.16	1.81	0.01	3.30
○Thread 596906			2.71	3.58	75.69	1.81	0.01	3.29
○Thread 596907			2.62	3.58	73.31	1.81	0.01	3.33
○Thread 596908			2.63	3.58	73.59	1.81	0.01	3.32
○Thread 596909			2.61	3.58	73.02	1.81	0.01	3.32
○Thread 596910			2.72	3.58	75.95	1.81	0.01	3.33
○Thread 596911			2.67	3.58	74.55	1.81	0.01	3.33
○Thread 596912			2.67	3.58	74.69	1.81	0.01	3.33
○Thread 596913			2.66	3.58	74.26	1.81	0.01	3.32
○Thread 596914			2.79	3.58	78.03	1.81	0.01	3.33
○Thread 596915			2.71	3.58	75.65	1.81	0.01	3.33
○Thread 596916			2.59	3.58	72.29	1.81	0.01	3.33
○Thread 596917			2.68	3.58	74.80	1.81	0.01	3.33
○Thread 596918			2.70	3.58	75.49	1.81	0.01	3.32
○Thread 596919			2.68	3.58	74.79	1.81	0.01	3.33
○Thread 596920			2.69	3.58	75.07	1.81	0.01	3.33
○Thread 596921			2.63	3.58	73.39	1.81	0.01	3.33
○Thread 596922			2.71	3.58	75.62	1.81	0.01	3.31
○Thread 596923			2.65	3.58	73.94	1.81	0.01	3.33
○Thread 596924			2.66	3.58	74.35	1.81	0.01	3.32
○Thread 596925			2.64	3.58	73.65	1.81	0.01	3.32
○Thread 596926			2.70	3.58	75.32	1.82	0.00	3.33
○Thread 596927			2.62	3.59	73.08	1.82	0.00	3.33
○Thread 596928			2.72	3.59	76.00	1.82	0.00	3.33
○Thread 596929			2.56	3.59	71.54	1.82	0.00	3.33
○Thread 596930			3.08	3.59	86.03	1.82	0.00	3.33
○Thread 596931			2.72	3.59	75.85	1.82	0.00	3.33
○Thread 596932			2.70	3.59	75.43	1.82	0.00	3.33
○Thread 596933			2.95	3.59	82.25	1.82	0.00	3.32
○Thread 596934			2.74	3.59	76.53	1.82	0.00	3.33
○Thread 596935			2.62	3.59	73.04	1.82	0.00	3.33
○Thread 596936			2.59	3.59	72.20	1.82	0.00	3.33
○Thread 596937			2.65	3.59	73.73	1.82	0.00	3.32

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	3.30
►Process 596957+		64	3.30
○Thread 596957			3.30	5.42	60.93	0.00	0.00	3.57
○Thread 597056			2.71	3.59	75.55	1.82	0.02	3.33
○Thread 597057			2.59	3.59	72.20	1.82	0.02	3.34
○Thread 597058			2.59	3.59	72.34	1.82	0.02	3.34
○Thread 597059			2.63	3.59	73.45	1.82	0.02	3.34
○Thread 597060			2.58	3.59	72.05	1.82	0.02	3.34
○Thread 597061			2.60	3.59	72.60	1.82	0.02	3.34
○Thread 597062			2.60	3.59	72.46	1.82	0.02	3.34
○Thread 597063			2.67	3.59	74.55	1.82	0.02	3.33
○Thread 597064			2.50	3.59	69.81	1.82	0.02	3.34
○Thread 597065			2.69	3.59	75.10	1.82	0.02	3.34
○Thread 597066			2.65	3.59	73.84	1.82	0.02	3.34
○Thread 597067			2.64	3.59	73.69	1.82	0.02	3.34
○Thread 597068			2.56	3.59	71.46	1.82	0.02	3.34
○Thread 597069			2.60	3.59	72.57	1.82	0.02	3.34
○Thread 597070			2.64	3.59	73.54	1.82	0.02	3.34
○Thread 597071			2.62	3.59	72.98	1.82	0.02	3.34
○Thread 597072			2.59	3.59	72.28	1.82	0.02	3.34
○Thread 597073			2.57	3.59	71.58	1.82	0.02	3.33
○Thread 597074			2.58	3.59	71.85	1.82	0.02	3.33
○Thread 597075			2.56	3.59	71.29	1.82	0.02	3.33
○Thread 597076			2.66	3.59	74.21	1.82	0.02	3.34
○Thread 597077			2.56	3.59	71.42	1.82	0.02	3.34
○Thread 597078			2.65	3.59	73.78	1.82	0.01	3.34
○Thread 597079			2.66	3.59	74.20	1.82	0.01	3.34
○Thread 597080			2.61	3.59	72.66	1.82	0.01	3.32
○Thread 597081			2.58	3.59	71.96	1.82	0.01	3.33
○Thread 597082			2.58	3.59	71.96	1.82	0.01	3.33
○Thread 597083			2.61	3.59	72.79	1.82	0.01	3.34
○Thread 597084			2.61	3.59	72.64	1.82	0.01	3.34
○Thread 597085			2.49	3.59	69.44	1.82	0.01	3.33
○Thread 597086			2.66	3.59	74.03	1.82	0.01	3.34
○Thread 597087			2.73	3.59	75.97	1.82	0.01	3.34
○Thread 597088			2.55	3.59	71.10	1.82	0.01	3.33
○Thread 597089			2.57	3.59	71.65	1.82	0.01	3.33
○Thread 597090			2.59	3.59	72.20	1.82	0.01	3.33
○Thread 597091			2.73	3.59	76.09	1.82	0.01	3.34
○Thread 597092			2.66	3.59	74.14	1.82	0.01	3.34
○Thread 597093			2.51	3.59	69.82	1.82	0.01	3.33
○Thread 597094			2.59	3.60	72.18	1.82	0.01	3.34
○Thread 597095			2.62	3.60	72.87	1.82	0.01	3.34
○Thread 597096			2.57	3.60	71.62	1.82	0.01	3.33
○Thread 597097			2.58	3.60	71.75	1.82	0.01	3.34
○Thread 597098			2.63	3.60	73.14	1.82	0.01	3.34
○Thread 597099			2.64	3.60	73.55	1.82	0.01	3.34
○Thread 597100			2.54	3.60	70.77	1.82	0.01	3.34
○Thread 597101			2.61	3.60	72.71	1.82	0.01	3.34
○Thread 597102			2.60	3.60	72.43	1.82	0.01	3.33
○Thread 597103			2.76	3.60	76.73	1.82	0.01	3.34
○Thread 597104			2.67	3.60	74.22	1.82	0.01	3.34
○Thread 597105			2.57	3.60	71.44	1.82	0.01	3.33
○Thread 597106			2.64	3.60	73.52	1.82	0.01	3.34
○Thread 597107			2.66	3.60	73.93	1.82	0.00	3.34
○Thread 597108			2.67	3.60	74.21	1.82	0.00	3.34
○Thread 597109			2.65	3.60	73.64	1.82	0.00	3.33
○Thread 597110			2.58	3.60	71.83	1.82	0.00	3.34
○Thread 597111			2.86	3.60	79.47	1.82	0.00	3.34
○Thread 597112			2.51	3.60	69.88	1.82	0.00	3.34
○Thread 597113			2.62	3.60	72.93	1.82	0.00	3.33
○Thread 597114			2.64	3.60	73.48	1.82	0.00	3.33
○Thread 597115			2.75	3.60	76.53	1.82	0.00	3.34
○Thread 597116			2.58	3.60	71.67	1.82	0.00	3.32
○Thread 597117			2.67	3.60	74.16	1.82	0.00	3.34
○Thread 597118			2.64	3.60	73.33	1.82	0.00	3.34

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	3.13
►Process 597187+		72	3.13
○Thread 597187			3.13	5.49	57.15	0.00	0.00	3.62
○Thread 597286			2.55	3.64	70.01	1.82	0.03	3.39
○Thread 597287			2.52	3.64	69.19	1.82	0.03	3.39
○Thread 597288			2.75	3.64	75.50	1.82	0.02	3.39
○Thread 597289			2.63	3.64	72.20	1.82	0.02	3.38
○Thread 597290			2.53	3.64	69.59	1.82	0.02	3.39
○Thread 597291			2.66	3.64	73.15	1.82	0.02	3.38
○Thread 597292			2.53	3.64	69.44	1.82	0.02	3.39
○Thread 597293			2.57	3.64	70.53	1.82	0.02	3.39
○Thread 597294			2.46	3.64	67.65	1.82	0.02	3.39
○Thread 597295			2.51	3.64	68.88	1.82	0.02	3.39
○Thread 597296			2.59	3.64	71.21	1.82	0.02	3.39
○Thread 597297			2.66	3.64	72.99	1.82	0.02	3.40
○Thread 597298			2.59	3.64	71.06	1.82	0.02	3.39
○Thread 597299			2.62	3.65	71.74	1.82	0.02	3.38
○Thread 597300			2.65	3.65	72.70	1.82	0.02	3.37
○Thread 597301			2.57	3.65	70.50	1.82	0.02	3.39
○Thread 597302			2.63	3.65	72.14	1.82	0.02	3.39
○Thread 597303			2.50	3.65	68.57	1.82	0.02	3.39
○Thread 597304			2.53	3.65	69.52	1.82	0.02	3.39
○Thread 597305			2.44	3.65	67.05	1.82	0.02	3.39
○Thread 597306			2.49	3.65	68.42	1.82	0.02	3.39
○Thread 597307			2.63	3.65	72.25	1.82	0.02	3.38
○Thread 597308			2.47	3.65	67.86	1.82	0.02	3.39
○Thread 597309			2.55	3.65	70.05	1.82	0.02	3.38
○Thread 597310			2.61	3.65	71.69	1.82	0.02	3.38
○Thread 597311			2.56	3.65	70.18	1.82	0.02	3.39
○Thread 597312			2.58	3.65	70.86	1.82	0.02	3.39
○Thread 597313			2.56	3.65	70.17	1.82	0.02	3.37
○Thread 597314			2.61	3.65	71.67	1.82	0.02	3.38
○Thread 597315			2.53	3.65	69.34	1.82	0.02	3.39
○Thread 597316			2.46	3.65	67.42	1.82	0.02	3.38
○Thread 597317			2.62	3.65	71.80	1.82	0.01	3.39
○Thread 597318			2.59	3.65	70.97	1.82	0.01	3.36
○Thread 597319			2.54	3.65	69.60	1.82	0.01	3.39
○Thread 597320			2.57	3.65	70.55	1.82	0.01	3.38
○Thread 597321			2.44	3.65	66.99	1.82	0.01	3.39
○Thread 597322			2.50	3.65	68.49	1.82	0.01	3.39
○Thread 597323			2.65	3.65	72.59	1.82	0.01	3.39
○Thread 597324			2.55	3.65	69.85	1.82	0.01	3.36
○Thread 597325			2.61	3.65	71.63	1.82	0.01	3.39
○Thread 597326			2.54	3.65	69.70	1.82	0.01	3.39
○Thread 597327			2.61	3.65	71.48	1.82	0.01	3.39
○Thread 597328			2.55	3.65	69.83	1.82	0.01	3.36
○Thread 597329			2.64	3.65	72.29	1.82	0.01	3.39
○Thread 597330			2.54	3.65	69.55	1.82	0.01	3.39
○Thread 597331			2.56	3.65	70.23	1.82	0.01	3.39
○Thread 597332			2.45	3.65	67.08	1.82	0.01	3.39
○Thread 597333			2.71	3.65	74.19	1.82	0.01	3.38
○Thread 597334			2.64	3.65	72.41	1.82	0.01	3.39
○Thread 597335			2.52	3.65	68.98	1.82	0.01	3.35
○Thread 597336			2.62	3.65	71.71	1.82	0.01	3.39
○Thread 597337			2.59	3.65	70.89	1.82	0.01	3.38
○Thread 597338			2.58	3.65	70.75	1.82	0.01	3.39
○Thread 597339			2.59	3.65	70.88	1.82	0.01	3.39
○Thread 597340			2.64	3.65	72.24	1.82	0.01	3.39
○Thread 597341			2.56	3.65	70.05	1.82	0.01	3.39
○Thread 597342			2.65	3.65	72.65	1.82	0.01	3.39
○Thread 597343			2.63	3.65	71.96	1.83	0.01	3.38
○Thread 597344			2.65	3.66	72.50	1.83	0.01	3.39
○Thread 597345			2.59	3.66	70.85	1.83	0.00	3.38
○Thread 597346			2.51	3.66	68.80	1.83	0.00	3.38
○Thread 597347			2.55	3.66	69.89	1.83	0.00	3.39
○Thread 597348			2.56	3.66	70.16	1.83	0.00	3.39
○Thread 597349			3.11	3.66	85.19	1.83	0.00	3.39
○Thread 597350			2.70	3.66	73.84	1.83	0.00	3.38
○Thread 597351			2.75	3.66	75.34	1.83	0.00	3.39
○Thread 597352			2.87	3.66	78.48	1.83	0.00	3.39
○Thread 597353			2.62	3.66	71.64	1.83	0.00	3.39
○Thread 597354			2.58	3.66	70.54	1.83	0.00	3.38
○Thread 597355			2.53	3.66	69.30	1.83	0.00	3.39
○Thread 597356			2.42	3.66	66.16	1.83	0.00	3.38

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	3.23
►Process 597376+		80	3.23
○Thread 597376			3.23	5.58	58.01	0.00	0.00	3.71
○Thread 597475			2.52	3.73	67.73	1.82	0.03	3.48
○Thread 597476			2.60	3.73	69.74	1.82	0.03	3.48
○Thread 597477			2.62	3.73	70.41	1.82	0.03	3.47
○Thread 597478			2.65	3.73	71.21	1.82	0.03	3.48
○Thread 597479			2.58	3.73	69.20	1.82	0.03	3.48
○Thread 597480			2.57	3.73	68.92	1.82	0.03	3.47
○Thread 597481			2.58	3.73	69.32	1.82	0.03	3.47
○Thread 597482			2.68	3.73	71.87	1.82	0.03	3.48
○Thread 597483			2.49	3.73	66.77	1.82	0.03	3.48
○Thread 597484			2.55	3.73	68.51	1.82	0.03	3.48
○Thread 597485			2.60	3.73	69.84	1.82	0.03	3.48
○Thread 597486			2.68	3.73	71.85	1.82	0.02	3.48
○Thread 597487			2.53	3.73	67.82	1.82	0.02	3.48
○Thread 597488			2.67	3.73	71.71	1.82	0.02	3.48
○Thread 597489			2.63	3.73	70.50	1.82	0.02	3.48
○Thread 597490			2.69	3.73	72.10	1.82	0.02	3.48
○Thread 597491			2.55	3.73	68.48	1.82	0.02	3.48
○Thread 597492			2.53	3.73	67.80	1.82	0.02	3.48
○Thread 597493			2.65	3.73	71.02	1.82	0.02	3.48
○Thread 597494			2.55	3.73	68.33	1.82	0.02	3.48
○Thread 597495			2.57	3.73	69.00	1.82	0.02	3.45
○Thread 597496			2.50	3.73	67.12	1.82	0.02	3.41
○Thread 597497			2.59	3.73	69.39	1.82	0.02	3.48
○Thread 597498			2.53	3.73	67.91	1.82	0.02	3.47
○Thread 597499			2.45	3.73	65.63	1.82	0.02	3.47
○Thread 597500			2.53	3.73	67.77	1.82	0.02	3.48
○Thread 597501			2.60	3.73	69.64	1.82	0.02	3.48
○Thread 597502			2.68	3.73	71.92	1.82	0.02	3.48
○Thread 597503			2.61	3.73	70.04	1.82	0.02	3.48
○Thread 597504			2.55	3.73	68.43	1.82	0.02	3.48
○Thread 597505			2.57	3.73	68.82	1.82	0.02	3.48
○Thread 597506			2.69	3.73	72.17	1.82	0.02	3.47
○Thread 597507			2.55	3.73	68.42	1.82	0.02	3.48
○Thread 597508			2.55	3.73	68.41	1.82	0.02	3.48
○Thread 597509			2.50	3.73	66.94	1.82	0.02	3.48
○Thread 597510			2.61	3.74	70.01	1.82	0.02	3.46
○Thread 597511			2.57	3.74	68.80	1.82	0.02	3.48
○Thread 597512			2.59	3.74	69.47	1.83	0.02	3.47
○Thread 597513			2.51	3.74	67.19	1.83	0.02	3.48
○Thread 597514			2.58	3.74	69.06	1.83	0.02	3.48
○Thread 597515			2.50	3.74	66.91	1.83	0.02	3.47
○Thread 597516			2.50	3.74	67.04	1.83	0.01	3.47
○Thread 597517			2.47	3.74	66.10	1.83	0.01	3.48
○Thread 597518			2.71	3.74	72.52	1.83	0.01	3.49
○Thread 597519			2.62	3.74	70.24	1.83	0.01	3.48
○Thread 597520			2.53	3.74	67.83	1.83	0.01	3.48
○Thread 597521			2.63	3.74	70.37	1.83	0.01	3.48
○Thread 597522			2.75	3.74	73.71	1.83	0.01	3.48
○Thread 597523			2.58	3.74	69.02	1.83	0.01	3.47
○Thread 597524			2.66	3.74	71.16	1.83	0.01	3.49
○Thread 597525			2.64	3.74	70.73	1.83	0.01	3.48
○Thread 597526			2.64	3.74	70.73	1.83	0.01	3.48
○Thread 597527			2.59	3.74	69.39	1.83	0.01	3.48
○Thread 597528			2.64	3.74	70.72	1.83	0.01	3.48
○Thread 597529			2.55	3.74	68.31	1.83	0.01	3.47
○Thread 597530			2.74	3.74	73.25	1.83	0.01	3.48
○Thread 597531			2.67	3.74	71.51	1.83	0.01	3.47
○Thread 597532			2.64	3.74	70.70	1.83	0.01	3.48
○Thread 597533			2.59	3.74	69.23	1.83	0.01	3.48
○Thread 597534			2.66	3.74	71.10	1.83	0.01	3.48
○Thread 597535			2.56	3.74	68.42	1.83	0.01	3.49
○Thread 597536			2.60	3.74	69.49	1.83	0.01	3.48
○Thread 597537			2.64	3.74	70.55	1.83	0.01	3.47
○Thread 597538			2.96	3.74	79.10	1.83	0.01	3.48
○Thread 597539			2.70	3.74	72.28	1.83	0.01	3.45
○Thread 597540			2.68	3.74	71.74	1.83	0.01	3.47
○Thread 597541			2.79	3.74	74.68	1.83	0.01	3.48
○Thread 597542			2.82	3.74	75.34	1.83	0.00	3.47
○Thread 597543			2.56	3.74	68.52	1.83	0.00	3.48
○Thread 597544			2.66	3.74	71.19	1.83	0.00	3.48
○Thread 597545			2.59	3.74	69.18	1.83	0.00	3.48
○Thread 597546			2.71	3.74	72.52	1.83	0.00	3.45
○Thread 597547			2.65	3.74	70.91	1.83	0.00	3.48
○Thread 597548			2.61	3.74	69.84	1.83	0.00	3.49
○Thread 597549			2.59	3.74	69.30	1.83	0.00	3.49
○Thread 597550			2.72	3.74	72.63	1.83	0.00	3.48
○Thread 597551			2.58	3.75	68.89	1.83	0.00	3.46
○Thread 597552			2.57	3.75	68.62	1.83	0.00	3.47
○Thread 597553			2.57	3.75	68.62	1.83	0.00	3.47

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	3.23
►Process 597573+		88	3.23
○Thread 597573			3.23	5.69	56.72	0.00	0.00	3.83
○Thread 597672			2.66	3.85	69.09	1.81	0.03	3.59
○Thread 597673			2.65	3.85	68.96	1.81	0.03	3.60
○Thread 597674			2.70	3.85	70.25	1.81	0.03	3.60
○Thread 597675			2.68	3.85	69.60	1.81	0.03	3.60
○Thread 597676			2.69	3.85	69.86	1.81	0.03	3.59
○Thread 597677			2.69	3.85	69.98	1.81	0.03	3.58
○Thread 597678			2.59	3.85	67.25	1.81	0.03	3.60
○Thread 597679			2.70	3.85	70.23	1.81	0.03	3.60
○Thread 597680			2.76	3.85	71.66	1.81	0.03	3.60
○Thread 597681			2.69	3.85	69.96	1.81	0.03	3.60
○Thread 597682			2.64	3.85	68.53	1.81	0.03	3.59
○Thread 597683			2.65	3.85	68.79	1.81	0.03	3.60
○Thread 597684			2.53	3.85	65.67	1.81	0.03	3.60
○Thread 597685			2.64	3.85	68.52	1.81	0.03	3.60
○Thread 597686			2.71	3.85	70.33	1.81	0.03	3.60
○Thread 597687			2.68	3.85	69.55	1.81	0.03	3.60
○Thread 597688			2.70	3.85	70.19	1.81	0.03	3.60
○Thread 597689			2.57	3.85	66.82	1.81	0.03	3.58
○Thread 597690			2.66	3.85	69.02	1.81	0.03	3.60
○Thread 597691			2.63	3.85	68.24	1.81	0.03	3.59
○Thread 597692			2.53	3.85	65.77	1.81	0.03	3.60
○Thread 597693			2.66	3.85	69.14	1.81	0.03	3.59
○Thread 597694			2.66	3.85	69.00	1.82	0.02	3.60
○Thread 597695			2.65	3.86	68.87	1.82	0.02	3.60
○Thread 597696			2.59	3.86	67.18	1.82	0.02	3.60
○Thread 597697			2.60	3.86	67.56	1.82	0.02	3.59
○Thread 597698			2.57	3.86	66.78	1.82	0.02	3.60
○Thread 597699			2.69	3.86	69.89	1.82	0.02	3.61
○Thread 597700			2.43	3.86	63.14	1.82	0.02	3.60
○Thread 597701			2.62	3.86	67.94	1.82	0.02	3.60
○Thread 597702			2.63	3.86	68.32	1.82	0.02	3.59
○Thread 597703			2.58	3.86	67.02	1.82	0.02	3.61
○Thread 597704			2.71	3.86	70.39	1.82	0.02	3.60
○Thread 597705			2.63	3.86	68.31	1.82	0.02	3.60
○Thread 597706			2.69	3.86	69.86	1.82	0.02	3.60
○Thread 597707			2.62	3.86	68.04	1.82	0.02	3.60
○Thread 597708			2.66	3.86	69.06	1.82	0.02	3.60
○Thread 597709			2.59	3.86	67.12	1.82	0.02	3.60
○Thread 597710			2.59	3.86	67.24	1.82	0.02	3.60
○Thread 597711			2.66	3.86	69.05	1.82	0.02	3.59
○Thread 597712			2.64	3.86	68.53	1.82	0.02	3.60
○Thread 597713			2.60	3.86	67.23	1.82	0.02	3.60
○Thread 597714			2.53	3.86	65.67	1.82	0.02	3.60
○Thread 597715			2.66	3.86	68.91	1.82	0.02	3.59
○Thread 597716			2.66	3.86	68.90	1.82	0.02	3.58
○Thread 597717			2.63	3.86	68.12	1.82	0.02	3.59
○Thread 597718			2.74	3.86	70.97	1.82	0.02	3.60
○Thread 597719			2.74	3.86	71.09	1.82	0.02	3.61
○Thread 597720			2.65	3.86	68.76	1.82	0.01	3.60
○Thread 597721			2.56	3.86	66.42	1.82	0.01	3.60
○Thread 597722			2.61	3.86	67.58	1.82	0.01	3.60
○Thread 597723			2.58	3.86	66.80	1.82	0.01	3.60
○Thread 597724			2.61	3.86	67.70	1.82	0.01	3.60
○Thread 597725			2.64	3.86	68.35	1.82	0.01	3.59
○Thread 597726			2.66	3.86	68.86	1.82	0.01	3.60
○Thread 597727			2.68	3.86	69.37	1.82	0.01	3.58
○Thread 597728			2.59	3.86	67.17	1.82	0.01	3.60
○Thread 597729			2.61	3.86	67.68	1.82	0.01	3.60
○Thread 597730			2.74	3.86	70.91	1.82	0.01	3.59
○Thread 597731			2.77	3.86	71.69	1.82	0.01	3.60
○Thread 597732			2.69	3.86	69.61	1.82	0.01	3.60
○Thread 597733			2.68	3.86	69.35	1.82	0.01	3.59
○Thread 597734			2.59	3.86	67.15	1.82	0.01	3.60
○Thread 597735			2.73	3.86	70.76	1.82	0.01	3.61
○Thread 597736			2.65	3.87	68.69	1.82	0.01	3.60
○Thread 597737			2.65	3.87	68.69	1.82	0.01	3.60
○Thread 597738			2.61	3.87	67.65	1.82	0.01	3.60
○Thread 597739			2.73	3.87	70.75	1.82	0.01	3.60
○Thread 597740			2.56	3.87	66.22	1.82	0.01	3.59
○Thread 597741			2.71	3.87	70.22	1.82	0.01	3.59
○Thread 597742			2.67	3.87	69.05	1.82	0.01	3.60
○Thread 597743			2.71	3.87	70.21	1.82	0.01	3.61
○Thread 597744			2.66	3.87	68.78	1.82	0.01	3.60
○Thread 597745			2.60	3.87	67.23	1.82	0.01	3.60
○Thread 597746			2.62	3.87	67.74	1.82	0.01	3.60
○Thread 597747			3.09	3.87	79.89	1.82	0.01	3.59
○Thread 597748			2.61	3.87	67.60	1.82	0.00	3.60
○Thread 597749			2.64	3.87	68.25	1.82	0.00	3.60
○Thread 597750			2.61	3.87	67.47	1.82	0.00	3.61
○Thread 597751			3.22	3.87	83.23	1.82	0.00	3.60
○Thread 597752			2.72	3.87	70.30	1.82	0.00	3.58
○Thread 597753			2.70	3.87	69.78	1.82	0.00	3.59
○Thread 597754			2.74	3.87	70.94	1.82	0.00	3.60
○Thread 597755			2.61	3.87	67.44	1.82	0.00	3.58
○Thread 597756			2.52	3.87	65.24	1.82	0.00	3.60
○Thread 597757			2.59	3.87	67.05	1.82	0.00	3.60
○Thread 597758			2.47	3.87	63.94	1.82	0.00	3.59

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	3.32
►Process 597778+		96	3.32
○Thread 597778			3.32	5.79	57.47	0.00	0.00	3.89
○Thread 597877			2.75	3.92	70.14	1.83	0.04	3.67
○Thread 597878			2.70	3.92	68.87	1.83	0.04	3.66
○Thread 597879			2.64	3.92	67.33	1.83	0.03	3.65
○Thread 597880			2.64	3.92	67.46	1.83	0.03	3.66
○Thread 597881			2.61	3.92	66.69	1.83	0.03	3.64
○Thread 597882			2.70	3.92	68.85	1.83	0.03	3.66
○Thread 597883			2.63	3.92	67.19	1.83	0.03	3.65
○Thread 597884			2.65	3.92	67.57	1.83	0.03	3.66
○Thread 597885			2.60	3.92	66.29	1.83	0.03	3.67
○Thread 597886			2.61	3.92	66.67	1.83	0.03	3.66
○Thread 597887			2.65	3.92	67.56	1.83	0.03	3.66
○Thread 597888			2.83	3.92	72.14	1.83	0.03	3.66
○Thread 597889			2.67	3.92	68.18	1.83	0.03	3.66
○Thread 597890			2.74	3.92	69.84	1.83	0.03	3.66
○Thread 597891			2.71	3.92	69.07	1.83	0.03	3.65
○Thread 597892			2.64	3.92	67.41	1.83	0.03	3.65
○Thread 597893			2.65	3.92	67.66	1.83	0.03	3.66
○Thread 597894			2.56	3.92	65.23	1.83	0.03	3.66
○Thread 597895			2.63	3.92	67.14	1.83	0.03	3.66
○Thread 597896			2.57	3.92	65.61	1.83	0.03	3.66
○Thread 597897			2.56	3.93	65.22	1.83	0.03	3.66
○Thread 597898			2.63	3.93	67.00	1.83	0.03	3.66
○Thread 597899			2.62	3.93	66.74	1.83	0.03	3.64
○Thread 597900			2.75	3.93	70.18	1.83	0.03	3.64
○Thread 597901			2.58	3.93	65.84	1.83	0.03	3.66
○Thread 597902			2.68	3.93	68.26	1.83	0.03	3.66
○Thread 597903			2.63	3.93	66.98	1.83	0.03	3.66
○Thread 597904			2.70	3.93	68.76	1.83	0.03	3.66
○Thread 597905			2.66	3.93	67.87	1.83	0.03	3.66
○Thread 597906			2.53	3.93	64.42	1.83	0.02	3.66
○Thread 597907			2.73	3.93	69.51	1.83	0.02	3.66
○Thread 597908			2.63	3.93	66.96	1.83	0.02	3.66
○Thread 597909			2.62	3.93	66.71	1.83	0.02	3.66
○Thread 597910			2.73	3.93	69.50	1.83	0.02	3.65
○Thread 597911			2.63	3.93	67.08	1.83	0.02	3.66
○Thread 597912			2.71	3.93	68.99	1.83	0.02	3.66
○Thread 597913			2.49	3.93	63.51	1.83	0.02	3.66
○Thread 597914			2.77	3.93	70.63	1.83	0.02	3.66
○Thread 597915			2.62	3.93	66.68	1.83	0.02	3.66
○Thread 597916			2.68	3.93	68.21	1.83	0.02	3.66
○Thread 597917			2.55	3.93	64.89	1.84	0.02	3.63
○Thread 597918			2.70	3.93	68.71	1.84	0.02	3.65
○Thread 597919			2.67	3.93	68.07	1.84	0.02	3.66
○Thread 597920			2.66	3.93	67.81	1.84	0.02	3.65
○Thread 597921			2.65	3.93	67.42	1.84	0.02	3.66
○Thread 597922			2.63	3.93	66.91	1.84	0.02	3.66
○Thread 597923			2.66	3.93	67.67	1.84	0.02	3.66
○Thread 597924			3.06	3.93	77.84	1.84	0.02	3.67
○Thread 597925			2.62	3.93	66.64	1.84	0.02	3.67
○Thread 597926			2.60	3.93	66.13	1.84	0.02	3.66
○Thread 597927			2.69	3.93	68.42	1.84	0.02	3.66
○Thread 597928			2.71	3.93	69.05	1.84	0.02	3.64
○Thread 597929			2.56	3.93	65.10	1.84	0.02	3.66
○Thread 597930			2.55	3.93	64.97	1.84	0.02	3.66
○Thread 597931			2.56	3.93	65.22	1.84	0.02	3.66
○Thread 597932			3.00	3.93	76.28	1.84	0.02	3.66
○Thread 597933			2.59	3.93	65.85	1.84	0.02	3.66
○Thread 597934			2.61	3.93	66.48	1.84	0.02	3.66
○Thread 597935			2.77	3.93	70.42	1.84	0.01	3.66
○Thread 597936			2.80	3.93	71.18	1.84	0.01	3.66
○Thread 597937			2.69	3.93	68.38	1.84	0.01	3.66
○Thread 597938			2.80	3.93	71.30	1.84	0.01	3.66
○Thread 597939			2.71	3.93	69.01	1.84	0.01	3.66
○Thread 597940			3.06	3.93	77.90	1.84	0.01	3.65
○Thread 597941			2.63	3.93	66.96	1.84	0.01	3.66
○Thread 597942			2.71	3.94	68.99	1.84	0.01	3.66
○Thread 597943			2.74	3.94	69.75	1.84	0.01	3.66
○Thread 597944			2.68	3.94	68.10	1.84	0.01	3.67
○Thread 597945			2.63	3.94	66.95	1.84	0.01	3.64
○Thread 597946			2.57	3.94	65.41	1.84	0.01	3.65
○Thread 597947			2.68	3.94	68.07	1.84	0.01	3.66
○Thread 597948			2.82	3.94	71.75	1.84	0.01	3.67
○Thread 597949			2.73	3.94	69.46	1.84	0.01	3.65
○Thread 597950			2.73	3.94	69.33	1.84	0.01	3.67
○Thread 597951			2.65	3.94	67.42	1.84	0.01	3.67
○Thread 597952			3.16	3.94	80.37	1.84	0.01	3.66
○Thread 597953			2.78	3.94	70.59	1.84	0.01	3.67
○Thread 597954			2.71	3.94	68.93	1.84	0.01	3.67
○Thread 597955			2.86	3.94	72.61	1.84	0.01	3.66
○Thread 597956			2.80	3.94	71.21	1.84	0.01	3.63
○Thread 597957			2.64	3.94	67.02	1.84	0.01	3.66
○Thread 597958			2.61	3.94	66.25	1.84	0.01	3.66
○Thread 597959			2.59	3.94	65.87	1.84	0.01	3.65
○Thread 597960			2.66	3.94	67.51	1.84	0.00	3.62
○Thread 597961			2.69	3.94	68.40	1.84	0.00	3.66
○Thread 597962			2.64	3.94	67.00	1.84	0.00	3.65
○Thread 597963			2.57	3.94	65.34	1.84	0.00	3.66
○Thread 597964			2.71	3.94	68.89	1.84	0.00	3.66
○Thread 597965			2.75	3.94	69.90	1.84	0.00	3.67
○Thread 597966			2.62	3.94	66.47	1.84	0.00	3.66
○Thread 597967			2.68	3.94	68.12	1.84	0.00	3.66
○Thread 597968			2.67	3.94	67.73	1.84	0.00	3.66
○Thread 597969			2.52	3.94	64.05	1.84	0.00	3.66
○Thread 597970			2.53	3.94	64.30	1.84	0.00	3.66
○Thread 597971			2.41	3.94	61.25	1.84	0.00	3.67

Report Configuration