OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	47.16
►Process 582440+		1	47.16
○Thread 582440			47.16	48.59	97.05	0.00	0.00	48.57

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	24.19
►Process 582609+		2	24.19
○Thread 582609			24.19	25.75	93.94	0.00	0.00	23.94
○Thread 582708			23.60	23.94	98.58	1.81	0.00	23.71

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	12.84
►Process 582729+		4	12.84
○Thread 582729			12.84	14.36	89.39	0.00	0.00	12.56
○Thread 582828			12.25	12.57	97.43	1.79	0.00	12.33
○Thread 582829			12.18	12.57	96.91	1.79	0.00	12.33
○Thread 582830			12.18	12.57	96.91	1.79	0.00	12.33

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	7.23
►Process 582899+		8	7.23
○Thread 582899			7.23	8.74	82.67	0.00	0.00	6.94
○Thread 582998			6.66	6.94	95.83	1.79	0.00	6.70
○Thread 582999			6.61	6.94	95.18	1.79	0.00	6.71
○Thread 583000			6.55	6.94	94.32	1.79	0.00	6.70
○Thread 583001			6.58	6.94	94.74	1.79	0.00	6.70
○Thread 583002			6.56	6.95	94.38	1.79	0.00	6.70
○Thread 583003			6.55	6.95	94.31	1.79	0.00	6.71
○Thread 583004			6.63	6.95	95.46	1.79	0.00	6.71

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	4.59
►Process 583024+		16	4.59
○Thread 583024			4.59	6.20	74.06	0.00	0.00	4.37
○Thread 583123			3.99	4.38	90.99	1.81	0.01	4.15
○Thread 583124			3.81	4.39	87.00	1.81	0.01	4.14
○Thread 583125			3.92	4.39	89.50	1.81	0.01	4.15
○Thread 583126			3.97	4.39	90.64	1.81	0.01	4.14
○Thread 583127			3.96	4.39	90.41	1.81	0.00	4.15
○Thread 583128			3.94	4.39	89.95	1.81	0.00	4.15
○Thread 583129			3.88	4.39	88.46	1.81	0.00	4.15
○Thread 583130			3.94	4.39	89.93	1.81	0.00	4.15
○Thread 583131			3.90	4.39	89.01	1.81	0.00	4.15
○Thread 583132			3.90	4.39	89.01	1.81	0.00	4.15
○Thread 583133			3.89	4.39	88.77	1.81	0.00	4.15
○Thread 583134			3.99	4.39	91.05	1.81	0.00	4.14
○Thread 583135			3.94	4.39	89.90	1.81	0.00	4.15
○Thread 583136			3.88	4.39	88.53	1.81	0.00	4.14
○Thread 583137			3.91	4.39	89.21	1.81	0.00	4.14

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	3.86
►Process 583157+		24	3.86
○Thread 583157			3.86	5.57	69.44	0.00	0.00	3.74
○Thread 583256			3.22	3.75	86.01	1.81	0.01	3.51
○Thread 583257			3.19	3.75	85.20	1.81	0.01	3.50
○Thread 583258			3.15	3.75	84.13	1.81	0.01	3.51
○Thread 583259			3.27	3.75	87.33	1.81	0.01	3.51
○Thread 583260			3.20	3.75	85.46	1.81	0.01	3.51
○Thread 583261			3.14	3.75	83.85	1.81	0.01	3.51
○Thread 583262			3.15	3.75	83.98	1.81	0.01	3.51
○Thread 583263			3.19	3.75	85.04	1.81	0.01	3.51
○Thread 583264			3.25	3.75	86.64	1.81	0.01	3.51
○Thread 583265			3.17	3.75	84.50	1.81	0.01	3.50
○Thread 583266			3.16	3.75	84.36	1.81	0.01	3.50
○Thread 583267			3.27	3.75	87.15	1.81	0.00	3.51
○Thread 583268			3.21	3.75	85.55	1.81	0.00	3.51
○Thread 583269			3.19	3.75	85.14	1.81	0.00	3.51
○Thread 583270			3.25	3.75	86.74	1.81	0.00	3.50
○Thread 583271			3.34	3.75	89.00	1.81	0.00	3.51
○Thread 583272			3.20	3.75	85.40	1.81	0.00	3.51
○Thread 583273			3.22	3.75	85.92	1.81	0.00	3.50
○Thread 583274			3.24	3.75	86.45	1.81	0.00	3.51
○Thread 583275			3.27	3.75	87.11	1.81	0.00	3.48
○Thread 583276			3.17	3.75	84.44	1.81	0.00	3.51
○Thread 583277			3.20	3.75	85.24	1.81	0.00	3.51
○Thread 583278			3.08	3.75	82.03	1.81	0.00	3.51

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	3.60
►Process 583298+		32	3.60
○Thread 583298			3.60	5.29	68.09	0.00	0.00	3.46
○Thread 583397			2.83	3.47	81.66	1.80	0.01	3.23
○Thread 583398			2.84	3.47	81.80	1.80	0.01	3.23
○Thread 583399			2.82	3.47	81.22	1.80	0.01	3.23
○Thread 583400			2.83	3.47	81.64	1.80	0.01	3.23
○Thread 583401			2.88	3.47	82.93	1.80	0.01	3.23
○Thread 583402			2.87	3.47	82.64	1.80	0.01	3.23
○Thread 583403			2.81	3.47	81.05	1.80	0.01	3.23
○Thread 583404			2.89	3.47	83.21	1.81	0.01	3.23
○Thread 583405			2.83	3.47	81.62	1.81	0.01	3.23
○Thread 583406			2.79	3.47	80.46	1.81	0.01	3.23
○Thread 583407			2.79	3.47	80.17	1.81	0.01	3.23
○Thread 583408			2.91	3.47	83.76	1.81	0.01	3.23
○Thread 583409			2.82	3.47	81.17	1.81	0.01	3.23
○Thread 583410			2.81	3.47	80.87	1.81	0.01	3.23
○Thread 583411			2.87	3.47	82.60	1.81	0.01	3.23
○Thread 583412			2.89	3.47	83.31	1.81	0.01	3.23
○Thread 583413			2.83	3.48	81.43	1.81	0.01	3.23
○Thread 583414			2.76	3.48	79.42	1.81	0.01	3.23
○Thread 583415			2.77	3.48	79.84	1.81	0.00	3.23
○Thread 583416			2.76	3.48	79.55	1.81	0.00	3.23
○Thread 583417			2.84	3.48	81.85	1.81	0.00	3.23
○Thread 583418			2.74	3.48	78.96	1.81	0.00	3.23
○Thread 583419			2.73	3.48	78.53	1.81	0.00	3.23
○Thread 583420			2.84	3.48	81.83	1.81	0.00	3.22
○Thread 583421			2.82	3.48	81.11	1.81	0.00	3.23
○Thread 583422			2.83	3.48	81.39	1.81	0.00	3.23
○Thread 583423			2.77	3.48	79.80	1.81	0.00	3.22
○Thread 583424			2.88	3.48	82.96	1.81	0.00	3.23
○Thread 583425			2.85	3.48	81.95	1.81	0.00	3.23
○Thread 583426			2.77	3.48	79.79	1.81	0.00	3.23
○Thread 583427			2.81	3.48	80.93	1.81	0.00	3.23

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	3.27
►Process 583447+		40	3.27
○Thread 583447			3.27	5.19	63.10	0.00	0.00	3.35
○Thread 583546			2.62	3.36	78.06	1.81	0.01	3.12
○Thread 583547			2.57	3.36	76.42	1.81	0.01	3.12
○Thread 583548			2.53	3.36	75.37	1.81	0.01	3.12
○Thread 583549			2.63	3.36	78.34	1.81	0.01	3.12
○Thread 583550			2.68	3.36	79.82	1.81	0.01	3.12
○Thread 583551			2.57	3.36	76.40	1.81	0.01	3.11
○Thread 583552			2.68	3.36	79.81	1.81	0.01	3.12
○Thread 583553			2.57	3.36	76.39	1.81	0.01	3.12
○Thread 583554			2.58	3.36	76.83	1.81	0.01	3.12
○Thread 583555			2.64	3.36	78.61	1.81	0.01	3.12
○Thread 583556			2.60	3.37	77.27	1.81	0.01	3.12
○Thread 583557			2.67	3.37	79.34	1.81	0.01	3.12
○Thread 583558			2.62	3.37	78.00	1.81	0.01	3.12
○Thread 583559			2.66	3.37	79.18	1.81	0.01	3.12
○Thread 583560			2.63	3.37	78.14	1.82	0.01	3.11
○Thread 583561			2.62	3.37	77.83	1.82	0.01	3.11
○Thread 583562			2.60	3.37	77.38	1.82	0.01	3.12
○Thread 583563			2.61	3.37	77.53	1.82	0.01	3.12
○Thread 583564			2.60	3.37	77.37	1.82	0.01	3.12
○Thread 583565			2.65	3.37	78.85	1.82	0.01	3.12
○Thread 583566			2.58	3.37	76.77	1.82	0.01	3.12
○Thread 583567			2.61	3.37	77.51	1.82	0.01	3.12
○Thread 583568			2.63	3.37	78.24	1.82	0.01	3.12
○Thread 583569			2.61	3.37	77.65	1.82	0.01	3.12
○Thread 583570			2.65	3.37	78.68	1.82	0.01	3.12
○Thread 583571			2.65	3.37	78.82	1.82	0.01	3.12
○Thread 583572			2.59	3.37	77.04	1.82	0.00	3.12
○Thread 583573			2.65	3.37	78.66	1.82	0.00	3.12
○Thread 583574			2.65	3.37	78.66	1.82	0.00	3.12
○Thread 583575			2.64	3.37	78.36	1.82	0.00	3.12
○Thread 583576			2.62	3.37	77.91	1.82	0.00	3.11
○Thread 583577			2.88	3.37	85.62	1.82	0.00	3.12
○Thread 583578			2.69	3.37	79.98	1.82	0.00	3.12
○Thread 583579			2.71	3.37	80.41	1.82	0.00	3.12
○Thread 583580			2.78	3.37	82.64	1.82	0.00	3.11
○Thread 583581			2.74	3.37	81.44	1.82	0.00	3.12
○Thread 583582			2.61	3.37	77.43	1.82	0.00	3.12
○Thread 583583			2.61	3.37	77.43	1.82	0.00	3.12
○Thread 583584			2.53	3.37	75.20	1.82	0.00	3.12

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	3.34
►Process 583652+		48	3.34
○Thread 583652			3.34	5.13	65.25	0.00	0.00	3.30
○Thread 583751			2.62	3.31	79.22	1.80	0.02	3.07
○Thread 583752			2.57	3.31	77.71	1.80	0.02	3.06
○Thread 583753			2.48	3.31	74.99	1.80	0.02	3.06
○Thread 583754			2.44	3.31	73.63	1.80	0.02	3.06
○Thread 583755			2.57	3.31	77.54	1.80	0.02	3.07
○Thread 583756			2.52	3.31	76.03	1.80	0.02	3.07
○Thread 583757			2.50	3.31	75.42	1.80	0.01	3.07
○Thread 583758			2.64	3.31	79.64	1.80	0.01	3.07
○Thread 583759			2.43	3.32	73.45	1.80	0.01	3.07
○Thread 583760			2.56	3.32	77.22	1.80	0.01	3.07
○Thread 583761			2.54	3.32	76.76	1.80	0.01	3.06
○Thread 583762			2.59	3.32	78.11	1.80	0.01	3.07
○Thread 583763			2.51	3.32	75.69	1.80	0.01	3.07
○Thread 583764			2.45	3.32	73.88	1.80	0.01	3.07
○Thread 583765			2.47	3.32	74.63	1.80	0.01	3.06
○Thread 583766			2.51	3.32	75.68	1.80	0.01	3.06
○Thread 583767			2.46	3.32	74.17	1.80	0.01	3.07
○Thread 583768			2.46	3.32	74.16	1.80	0.01	3.06
○Thread 583769			2.52	3.32	76.12	1.80	0.01	3.07
○Thread 583770			2.47	3.32	74.45	1.80	0.01	3.07
○Thread 583771			2.52	3.32	76.11	1.80	0.01	3.07
○Thread 583772			2.65	3.32	79.87	1.80	0.01	3.07
○Thread 583773			2.54	3.32	76.55	1.80	0.01	3.07
○Thread 583774			2.61	3.32	78.65	1.80	0.01	3.06
○Thread 583775			2.54	3.32	76.69	1.80	0.01	3.07
○Thread 583776			2.53	3.32	76.23	1.80	0.01	3.07
○Thread 583777			2.52	3.32	75.92	1.80	0.01	3.07
○Thread 583778			2.59	3.32	78.18	1.80	0.01	3.07
○Thread 583779			2.54	3.32	76.52	1.80	0.01	3.07
○Thread 583780			2.51	3.32	75.61	1.80	0.01	3.07
○Thread 583781			2.58	3.32	77.86	1.80	0.01	3.07
○Thread 583782			2.56	3.32	77.26	1.80	0.01	3.07
○Thread 583783			2.50	3.32	75.45	1.80	0.01	3.07
○Thread 583784			2.57	3.32	77.55	1.80	0.01	3.06
○Thread 583785			2.53	3.32	76.19	1.80	0.00	3.07
○Thread 583786			2.56	3.32	77.24	1.80	0.00	3.06
○Thread 583787			2.56	3.32	77.08	1.80	0.00	3.07
○Thread 583788			2.57	3.32	77.38	1.80	0.00	3.07
○Thread 583789			2.47	3.32	74.51	1.80	0.00	3.07
○Thread 583790			2.58	3.32	77.82	1.80	0.00	3.07
○Thread 583791			2.54	3.32	76.61	1.80	0.00	3.07
○Thread 583792			2.62	3.32	78.86	1.80	0.00	3.04
○Thread 583793			2.55	3.32	76.90	1.80	0.00	3.07
○Thread 583794			2.53	3.32	76.29	1.80	0.00	3.07
○Thread 583795			2.49	3.32	75.08	1.80	0.00	3.07
○Thread 583796			2.44	3.32	73.42	1.80	0.00	3.07
○Thread 583797			2.38	3.32	71.61	1.80	0.00	3.07

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	3.27
►Process 583817+		56	3.27
○Thread 583817			3.27	5.21	62.90	0.00	0.00	3.35
○Thread 583916			2.59	3.37	76.97	1.82	0.02	3.12
○Thread 583917			2.57	3.37	76.37	1.82	0.02	3.12
○Thread 583918			2.56	3.37	76.07	1.82	0.02	3.12
○Thread 583919			2.51	3.37	74.58	1.82	0.02	3.10
○Thread 583920			2.47	3.37	73.53	1.82	0.02	3.12
○Thread 583921			2.50	3.37	74.27	1.82	0.02	3.12
○Thread 583922			2.46	3.37	73.22	1.82	0.02	3.11
○Thread 583923			2.56	3.37	76.04	1.82	0.02	3.12
○Thread 583924			2.47	3.37	73.51	1.82	0.02	3.12
○Thread 583925			2.48	3.37	73.80	1.82	0.02	3.12
○Thread 583926			2.43	3.37	72.17	1.82	0.02	3.12
○Thread 583927			2.63	3.37	78.10	1.82	0.02	3.12
○Thread 583928			2.54	3.37	75.42	1.82	0.02	3.12
○Thread 583929			2.47	3.37	73.34	1.82	0.02	3.11
○Thread 583930			2.51	3.37	74.67	1.82	0.01	3.11
○Thread 583931			2.56	3.37	76.15	1.82	0.01	3.12
○Thread 583932			2.51	3.37	74.51	1.82	0.01	3.12
○Thread 583933			2.52	3.37	74.95	1.82	0.01	3.12
○Thread 583934			2.53	3.37	75.09	1.82	0.01	3.11
○Thread 583935			2.42	3.37	71.97	1.82	0.01	3.12
○Thread 583936			2.44	3.37	72.56	1.82	0.01	3.12
○Thread 583937			2.54	3.37	75.38	1.83	0.01	3.11
○Thread 583938			2.48	3.37	73.59	1.83	0.01	3.11
○Thread 583939			2.53	3.37	75.07	1.83	0.01	3.12
○Thread 583940			2.47	3.37	73.29	1.83	0.01	3.12
○Thread 583941			2.39	3.37	70.91	1.83	0.01	3.11
○Thread 583942			2.54	3.37	75.50	1.83	0.01	3.12
○Thread 583943			2.57	3.37	76.24	1.83	0.01	3.12
○Thread 583944			2.36	3.37	70.15	1.83	0.01	3.12
○Thread 583945			2.45	3.37	72.82	1.83	0.01	3.12
○Thread 583946			2.45	3.37	72.81	1.83	0.01	3.12
○Thread 583947			2.56	3.37	75.92	1.83	0.01	3.12
○Thread 583948			2.51	3.37	74.58	1.83	0.01	3.12
○Thread 583949			2.54	3.37	75.32	1.83	0.01	3.12
○Thread 583950			2.52	3.37	74.87	1.83	0.01	3.12
○Thread 583951			2.47	3.37	73.23	1.83	0.01	3.12
○Thread 583952			2.43	3.37	72.04	1.83	0.01	3.12
○Thread 583953			2.47	3.37	73.37	1.83	0.01	3.12
○Thread 583954			2.41	3.37	71.44	1.83	0.01	3.12
○Thread 583955			2.52	3.37	74.85	1.83	0.01	3.11
○Thread 583956			2.44	3.37	72.32	1.83	0.01	3.12
○Thread 583957			2.46	3.37	73.06	1.83	0.01	3.11
○Thread 583958			2.57	3.37	76.17	1.83	0.00	3.12
○Thread 583959			2.64	3.37	78.23	1.83	0.00	3.12
○Thread 583960			2.50	3.37	74.23	1.83	0.00	3.11
○Thread 583961			2.54	3.37	75.41	1.83	0.00	3.12
○Thread 583962			2.51	3.38	74.37	1.83	0.00	3.11
○Thread 583963			2.86	3.38	84.88	1.83	0.00	3.12
○Thread 583964			2.53	3.38	75.10	1.83	0.00	3.12
○Thread 583965			2.66	3.38	78.80	1.83	0.00	3.12
○Thread 583966			2.71	3.38	80.27	1.83	0.00	3.12
○Thread 583967			2.63	3.38	78.05	1.83	0.00	3.12
○Thread 583968			2.42	3.38	71.67	1.83	0.00	3.12
○Thread 583969			2.42	3.38	71.82	1.83	0.00	3.12
○Thread 583970			2.36	3.38	70.04	1.83	0.00	3.10

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	3.14
►Process 583990+		64	3.14
○Thread 583990			3.14	5.28	59.61	0.00	0.00	3.42
○Thread 584089			2.43	3.44	70.65	1.81	0.02	3.15
○Thread 584090			2.40	3.44	69.92	1.81	0.02	3.19
○Thread 584091			2.41	3.44	70.21	1.81	0.02	3.19
○Thread 584092			2.43	3.44	70.64	1.81	0.02	3.19
○Thread 584093			2.46	3.44	71.65	1.81	0.02	3.20
○Thread 584094			2.43	3.44	70.63	1.81	0.02	3.15
○Thread 584095			2.44	3.44	71.04	1.81	0.02	3.19
○Thread 584096			2.57	3.44	74.81	1.81	0.02	3.20
○Thread 584097			2.51	3.44	73.06	1.81	0.02	3.14
○Thread 584098			2.42	3.44	70.44	1.81	0.02	3.19
○Thread 584099			2.43	3.44	70.73	1.81	0.02	3.19
○Thread 584100			2.54	3.44	73.92	1.81	0.02	3.19
○Thread 584101			2.40	3.44	69.70	1.81	0.02	3.17
○Thread 584102			2.47	3.44	71.73	1.81	0.02	3.19
○Thread 584103			2.45	3.44	71.29	1.82	0.02	3.20
○Thread 584104			2.48	3.44	72.16	1.82	0.02	3.18
○Thread 584105			2.47	3.44	71.72	1.82	0.02	3.20
○Thread 584106			2.43	3.44	70.70	1.82	0.02	3.19
○Thread 584107			2.45	3.44	71.27	1.82	0.02	3.20
○Thread 584108			2.51	3.44	72.87	1.82	0.02	3.19
○Thread 584109			2.44	3.44	70.97	1.82	0.02	3.20
○Thread 584110			2.41	3.45	70.10	1.82	0.02	3.20
○Thread 584111			2.50	3.45	72.56	1.82	0.01	3.20
○Thread 584112			2.55	3.45	74.01	1.82	0.01	3.19
○Thread 584113			2.40	3.45	69.65	1.82	0.01	3.20
○Thread 584114			2.44	3.45	70.95	1.82	0.01	3.20
○Thread 584115			2.46	3.45	71.53	1.82	0.01	3.20
○Thread 584116			2.42	3.45	70.36	1.82	0.01	3.17
○Thread 584117			2.52	3.45	73.26	1.82	0.01	3.20
○Thread 584118			2.52	3.45	73.26	1.82	0.01	3.19
○Thread 584119			2.46	3.45	71.37	1.82	0.01	3.19
○Thread 584120			2.47	3.45	71.80	1.82	0.01	3.19
○Thread 584121			2.48	3.45	71.94	1.82	0.01	3.19
○Thread 584122			2.46	3.45	71.35	1.82	0.01	3.19
○Thread 584123			2.42	3.45	70.19	1.82	0.01	3.16
○Thread 584124			2.56	3.45	74.24	1.82	0.01	3.19
○Thread 584125			2.46	3.45	71.48	1.82	0.01	3.20
○Thread 584126			2.38	3.45	69.02	1.82	0.01	3.19
○Thread 584127			2.41	3.45	69.88	1.82	0.01	3.20
○Thread 584128			2.50	3.45	72.63	1.82	0.01	3.18
○Thread 584129			2.34	3.45	67.99	1.82	0.01	3.19
○Thread 584130			2.44	3.45	70.88	1.82	0.01	3.19
○Thread 584131			2.43	3.45	70.44	1.82	0.01	3.19
○Thread 584132			2.48	3.45	72.03	1.82	0.01	3.19
○Thread 584133			2.43	3.45	70.43	1.82	0.01	3.19
○Thread 584134			2.47	3.45	71.59	1.82	0.01	3.19
○Thread 584135			2.45	3.45	71.15	1.82	0.01	3.19
○Thread 584136			2.86	3.45	82.88	1.82	0.01	3.20
○Thread 584137			2.50	3.45	72.44	1.82	0.01	3.18
○Thread 584138			2.56	3.45	74.18	1.82	0.01	3.18
○Thread 584139			2.54	3.45	73.74	1.82	0.01	3.18
○Thread 584140			2.48	3.45	71.85	1.82	0.00	3.18
○Thread 584141			2.49	3.45	72.28	1.82	0.00	3.19
○Thread 584142			2.50	3.45	72.42	1.82	0.00	3.19
○Thread 584143			2.42	3.45	70.24	1.82	0.00	3.20
○Thread 584144			2.60	3.45	75.31	1.82	0.00	3.18
○Thread 584145			2.42	3.45	70.09	1.82	0.00	3.19
○Thread 584146			2.46	3.45	71.39	1.82	0.00	3.18
○Thread 584147			2.47	3.45	71.67	1.82	0.00	3.20
○Thread 584148			2.70	3.45	78.18	1.82	0.00	3.19
○Thread 584149			2.51	3.45	72.68	1.82	0.00	3.20
○Thread 584150			2.57	3.45	74.55	1.82	0.00	3.19
○Thread 584151			2.51	3.45	72.81	1.82	0.00	3.19

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	3.18
►Process 584172+		72	3.18
○Thread 584172			3.18	5.33	59.77	0.00	0.00	3.46
○Thread 584271			2.58	3.48	74.08	1.82	0.03	3.23
○Thread 584272			2.57	3.48	73.79	1.82	0.03	3.24
○Thread 584273			2.59	3.48	74.36	1.82	0.03	3.24
○Thread 584274			2.51	3.48	72.06	1.82	0.03	3.24
○Thread 584275			2.38	3.48	68.47	1.82	0.03	3.23
○Thread 584276			2.47	3.48	70.90	1.82	0.02	3.23
○Thread 584277			2.42	3.48	69.61	1.82	0.02	3.24
○Thread 584278			2.56	3.49	73.45	1.82	0.02	3.23
○Thread 584279			2.54	3.49	72.87	1.82	0.02	3.24
○Thread 584280			2.50	3.49	71.72	1.82	0.02	3.23
○Thread 584281			2.37	3.49	68.13	1.82	0.02	3.21
○Thread 584282			2.58	3.49	74.15	1.82	0.02	3.23
○Thread 584283			2.48	3.49	71.13	1.82	0.02	3.23
○Thread 584284			2.41	3.49	69.12	1.82	0.02	3.24
○Thread 584285			2.42	3.49	69.40	1.82	0.02	3.24
○Thread 584286			2.44	3.49	69.97	1.82	0.02	3.23
○Thread 584287			2.48	3.49	71.11	1.82	0.02	3.24
○Thread 584288			2.50	3.49	71.68	1.82	0.02	3.24
○Thread 584289			2.38	3.49	68.38	1.82	0.02	3.22
○Thread 584290			2.39	3.49	68.66	1.82	0.02	3.23
○Thread 584291			2.39	3.49	68.66	1.82	0.02	3.24
○Thread 584292			2.45	3.49	70.38	1.82	0.02	3.24
○Thread 584293			2.50	3.49	71.66	1.82	0.02	3.24
○Thread 584294			2.41	3.49	69.22	1.82	0.02	3.24
○Thread 584295			2.44	3.49	69.93	1.82	0.02	3.23
○Thread 584296			2.39	3.49	68.64	1.82	0.02	3.24
○Thread 584297			2.47	3.49	70.93	1.82	0.02	3.24
○Thread 584298			2.41	3.49	69.20	1.82	0.02	3.23
○Thread 584299			2.35	3.49	67.34	1.82	0.02	3.23
○Thread 584300			2.41	3.49	69.05	1.82	0.02	3.24
○Thread 584301			2.38	3.49	68.33	1.82	0.01	3.24
○Thread 584302			2.53	3.49	72.62	1.82	0.01	3.24
○Thread 584303			2.38	3.49	68.32	1.82	0.01	3.24
○Thread 584304			2.42	3.49	69.32	1.82	0.01	3.24
○Thread 584305			2.48	3.49	71.03	1.82	0.01	3.23
○Thread 584306			2.41	3.49	69.17	1.82	0.01	3.24
○Thread 584307			2.40	3.49	68.88	1.82	0.01	3.23
○Thread 584308			2.34	3.49	67.01	1.82	0.01	3.23
○Thread 584309			2.39	3.49	68.58	1.82	0.01	3.23
○Thread 584310			2.42	3.49	69.29	1.82	0.01	3.24
○Thread 584311			2.40	3.49	68.86	1.83	0.01	3.23
○Thread 584312			2.40	3.49	68.86	1.83	0.01	3.24
○Thread 584313			2.49	3.49	71.29	1.83	0.01	3.23
○Thread 584314			2.34	3.49	66.99	1.83	0.01	3.24
○Thread 584315			2.45	3.49	70.27	1.83	0.01	3.24
○Thread 584316			2.42	3.49	69.27	1.83	0.01	3.24
○Thread 584317			2.42	3.49	69.26	1.83	0.01	3.24
○Thread 584318			2.51	3.49	71.84	1.83	0.01	3.24
○Thread 584319			2.40	3.49	68.83	1.83	0.01	3.24
○Thread 584320			2.38	3.49	68.25	1.83	0.01	3.21
○Thread 584321			2.38	3.49	68.10	1.83	0.01	3.24
○Thread 584322			2.47	3.49	70.82	1.83	0.01	3.24
○Thread 584323			2.50	3.50	71.53	1.83	0.01	3.24
○Thread 584324			2.40	3.50	68.66	1.83	0.01	3.24
○Thread 584325			2.36	3.50	67.51	1.83	0.01	3.24
○Thread 584326			2.46	3.50	70.51	1.83	0.01	3.24
○Thread 584327			2.35	3.50	67.36	1.83	0.01	3.24
○Thread 584328			2.44	3.50	69.79	1.83	0.01	3.23
○Thread 584329			2.40	3.50	68.64	1.83	0.01	3.23
○Thread 584330			2.45	3.50	70.21	1.83	0.00	3.23
○Thread 584331			2.44	3.50	69.92	1.83	0.00	3.23
○Thread 584332			2.43	3.50	69.63	1.83	0.00	3.23
○Thread 584333			2.50	3.50	71.48	1.83	0.00	3.24
○Thread 584334			2.95	3.50	84.48	1.83	0.00	3.24
○Thread 584335			2.45	3.50	70.18	1.83	0.00	3.22
○Thread 584336			2.60	3.50	74.32	1.83	0.00	3.24
○Thread 584337			2.93	3.50	83.75	1.83	0.00	3.23
○Thread 584338			2.79	3.50	79.75	1.83	0.00	3.24
○Thread 584339			2.53	3.50	72.31	1.83	0.00	3.23
○Thread 584340			2.47	3.50	70.73	1.83	0.00	3.24
○Thread 584341			2.37	3.50	67.73	1.83	0.00	3.24

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	3.12
►Process 584361+		80	3.12
○Thread 584361			3.12	5.38	58.01	0.00	0.00	3.52
○Thread 584460			2.56	3.54	72.23	1.81	0.03	3.29
○Thread 584461			2.48	3.54	69.97	1.81	0.03	3.29
○Thread 584462			2.44	3.54	68.98	1.81	0.03	3.29
○Thread 584463			2.43	3.54	68.55	1.81	0.03	3.29
○Thread 584464			2.48	3.55	70.10	1.81	0.03	3.30
○Thread 584465			2.34	3.55	66.14	1.81	0.03	3.28
○Thread 584466			2.37	3.55	66.98	1.81	0.03	3.29
○Thread 584467			2.48	3.55	70.08	1.81	0.03	3.29
○Thread 584468			2.31	3.55	65.14	1.81	0.03	3.28
○Thread 584469			2.42	3.55	68.38	1.81	0.03	3.28
○Thread 584470			2.40	3.55	67.67	1.81	0.02	3.29
○Thread 584471			2.43	3.55	68.52	1.81	0.02	3.29
○Thread 584472			2.40	3.55	67.81	1.81	0.02	3.28
○Thread 584473			2.40	3.55	67.80	1.81	0.02	3.30
○Thread 584474			2.47	3.55	69.77	1.81	0.02	3.30
○Thread 584475			2.42	3.55	68.36	1.81	0.02	3.29
○Thread 584476			2.45	3.55	69.06	1.81	0.02	3.29
○Thread 584477			2.53	3.55	71.45	1.81	0.02	3.28
○Thread 584478			2.35	3.55	66.23	1.81	0.02	3.28
○Thread 584479			2.35	3.55	66.37	1.81	0.02	3.29
○Thread 584480			2.37	3.55	66.93	1.81	0.02	3.29
○Thread 584481			2.38	3.55	67.20	1.81	0.02	3.29
○Thread 584482			2.41	3.55	67.90	1.81	0.02	3.29
○Thread 584483			2.44	3.55	68.74	1.81	0.02	3.28
○Thread 584484			2.40	3.55	67.61	1.81	0.02	3.28
○Thread 584485			2.36	3.55	66.62	1.81	0.02	3.28
○Thread 584486			2.37	3.55	66.90	1.81	0.02	3.29
○Thread 584487			2.42	3.55	68.31	1.81	0.02	3.30
○Thread 584488			2.55	3.55	71.82	1.81	0.02	3.29
○Thread 584489			2.39	3.55	67.31	1.81	0.02	3.29
○Thread 584490			2.44	3.55	68.86	1.81	0.02	3.29
○Thread 584491			2.67	3.55	75.19	1.81	0.02	3.30
○Thread 584492			2.37	3.55	66.74	1.81	0.02	3.29
○Thread 584493			2.41	3.55	68.00	1.81	0.02	3.28
○Thread 584494			2.42	3.55	68.14	1.81	0.02	3.30
○Thread 584495			2.42	3.55	68.13	1.81	0.02	3.30
○Thread 584496			2.41	3.55	67.85	1.81	0.02	3.29
○Thread 584497			2.38	3.55	67.14	1.81	0.02	3.29
○Thread 584498			2.39	3.55	67.42	1.81	0.01	3.30
○Thread 584499			2.49	3.55	70.09	1.81	0.01	3.29
○Thread 584500			2.40	3.55	67.69	1.81	0.01	3.30
○Thread 584501			2.35	3.55	66.28	1.81	0.01	3.28
○Thread 584502			2.41	3.55	67.96	1.81	0.01	3.28
○Thread 584503			2.52	3.55	70.91	1.81	0.01	3.29
○Thread 584504			2.52	3.55	71.05	1.81	0.01	3.30
○Thread 584505			2.47	3.55	69.50	1.81	0.01	3.27
○Thread 584506			2.40	3.55	67.66	1.81	0.01	3.30
○Thread 584507			2.57	3.55	72.30	1.81	0.01	3.28
○Thread 584508			2.50	3.55	70.33	1.81	0.01	3.30
○Thread 584509			2.52	3.56	70.88	1.81	0.01	3.28
○Thread 584510			2.51	3.56	70.74	1.81	0.01	3.28
○Thread 584511			2.43	3.56	68.34	1.81	0.01	3.29
○Thread 584512			2.42	3.56	68.20	1.81	0.01	3.30
○Thread 584513			2.52	3.56	71.01	1.81	0.01	3.30
○Thread 584514			2.48	3.56	69.74	1.81	0.01	3.30
○Thread 584515			2.54	3.56	71.56	1.81	0.01	3.28
○Thread 584516			2.46	3.56	69.17	1.81	0.01	3.28
○Thread 584517			2.26	3.56	63.54	1.81	0.01	3.28
○Thread 584518			2.41	3.56	67.75	1.81	0.01	3.30
○Thread 584519			2.51	3.56	70.70	1.81	0.01	3.28
○Thread 584520			2.41	3.56	67.88	1.81	0.01	3.29
○Thread 584521			2.48	3.56	69.85	1.81	0.01	3.29
○Thread 584522			2.41	3.56	67.74	1.81	0.01	3.29
○Thread 584523			2.98	3.56	83.75	1.81	0.01	3.29
○Thread 584524			2.43	3.56	68.43	1.81	0.01	3.29
○Thread 584525			2.43	3.56	68.43	1.81	0.01	3.30
○Thread 584526			2.62	3.56	73.76	1.81	0.01	3.30
○Thread 584527			2.57	3.56	72.21	1.81	0.00	3.30
○Thread 584528			2.50	3.56	70.38	1.82	0.00	3.28
○Thread 584529			2.46	3.56	69.11	1.82	0.00	3.29
○Thread 584530			2.50	3.56	70.37	1.82	0.00	3.30
○Thread 584531			2.51	3.56	70.65	1.82	0.00	3.29
○Thread 584532			2.47	3.56	69.52	1.82	0.00	3.30
○Thread 584533			2.41	3.56	67.83	1.82	0.00	3.29
○Thread 584534			2.48	3.56	69.65	1.82	0.00	3.29
○Thread 584535			2.72	3.56	76.53	1.82	0.00	3.30
○Thread 584536			2.51	3.56	70.48	1.82	0.00	3.30
○Thread 584537			2.46	3.56	68.93	1.82	0.00	3.29
○Thread 584538			2.31	3.56	65.00	1.82	0.00	3.29

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	3.32
►Process 584607+		88	3.32
○Thread 584607			3.32	5.62	59.13	0.00	0.00	3.75
○Thread 584706			2.54	3.77	67.51	1.82	0.03	3.52
○Thread 584707			2.51	3.77	66.58	1.82	0.03	3.51
○Thread 584708			2.58	3.77	68.56	1.82	0.03	3.52
○Thread 584709			2.60	3.77	69.09	1.82	0.03	3.52
○Thread 584710			2.50	3.77	66.30	1.82	0.03	3.51
○Thread 584711			2.52	3.77	66.96	1.82	0.03	3.51
○Thread 584712			2.54	3.77	67.49	1.82	0.03	3.51
○Thread 584713			2.54	3.77	67.35	1.82	0.03	3.51
○Thread 584714			2.46	3.77	65.22	1.82	0.03	3.51
○Thread 584715			2.48	3.77	65.75	1.82	0.03	3.52
○Thread 584716			2.60	3.77	68.93	1.82	0.03	3.50
○Thread 584717			2.67	3.77	70.76	1.82	0.03	3.52
○Thread 584718			2.56	3.77	67.98	1.82	0.03	3.50
○Thread 584719			2.59	3.77	68.77	1.82	0.03	3.52
○Thread 584720			2.43	3.77	64.52	1.82	0.03	3.49
○Thread 584721			2.62	3.77	69.55	1.82	0.03	3.51
○Thread 584722			2.53	3.77	67.03	1.82	0.03	3.51
○Thread 584723			2.50	3.77	66.37	1.82	0.03	3.52
○Thread 584724			2.35	3.77	62.26	1.82	0.02	3.50
○Thread 584725			2.50	3.77	66.36	1.82	0.02	3.51
○Thread 584726			2.51	3.78	66.49	1.82	0.02	3.52
○Thread 584727			2.48	3.78	65.69	1.82	0.02	3.52
○Thread 584728			2.50	3.78	66.21	1.82	0.02	3.51
○Thread 584729			2.44	3.78	64.62	1.82	0.02	3.52
○Thread 584730			2.51	3.78	66.47	1.82	0.02	3.52
○Thread 584731			2.55	3.78	67.66	1.82	0.02	3.47
○Thread 584732			2.39	3.78	63.29	1.82	0.02	3.52
○Thread 584733			2.59	3.78	68.58	1.82	0.02	3.52
○Thread 584734			2.56	3.78	67.91	1.82	0.02	3.51
○Thread 584735			2.42	3.78	64.07	1.82	0.02	3.51
○Thread 584736			2.49	3.78	66.05	1.82	0.02	3.52
○Thread 584737			2.55	3.78	67.64	1.83	0.02	3.51
○Thread 584738			2.54	3.78	67.24	1.83	0.02	3.51
○Thread 584739			2.51	3.78	66.57	1.83	0.02	3.50
○Thread 584740			2.54	3.78	67.23	1.83	0.02	3.48
○Thread 584741			2.44	3.78	64.71	1.83	0.02	3.49
○Thread 584742			2.50	3.78	66.16	1.83	0.02	3.51
○Thread 584743			2.46	3.78	65.10	1.83	0.02	3.51
○Thread 584744			2.44	3.78	64.70	1.83	0.02	3.52
○Thread 584745			2.41	3.78	63.77	1.83	0.02	3.51
○Thread 584746			2.56	3.78	67.74	1.83	0.02	3.51
○Thread 584747			2.42	3.78	64.03	1.83	0.02	3.52
○Thread 584748			2.53	3.78	66.93	1.83	0.02	3.51
○Thread 584749			2.49	3.78	65.87	1.83	0.02	3.51
○Thread 584750			2.64	3.78	69.97	1.83	0.02	3.52
○Thread 584751			2.66	3.78	70.36	1.83	0.02	3.52
○Thread 584752			2.55	3.78	67.45	1.83	0.02	3.51
○Thread 584753			2.55	3.78	67.44	1.83	0.01	3.51
○Thread 584754			2.48	3.78	65.72	1.83	0.01	3.51
○Thread 584755			2.54	3.78	67.17	1.83	0.01	3.50
○Thread 584756			2.48	3.78	65.71	1.83	0.01	3.52
○Thread 584757			2.55	3.78	67.56	1.83	0.01	3.50
○Thread 584758			2.51	3.78	66.50	1.83	0.01	3.51
○Thread 584759			2.42	3.78	64.11	1.83	0.01	3.52
○Thread 584760			2.51	3.78	66.36	1.83	0.01	3.51
○Thread 584761			2.46	3.78	65.03	1.83	0.01	3.51
○Thread 584762			2.62	3.78	69.26	1.83	0.01	3.52
○Thread 584763			2.51	3.78	66.48	1.83	0.01	3.52
○Thread 584764			2.61	3.78	68.98	1.83	0.01	3.50
○Thread 584765			2.56	3.78	67.79	1.83	0.01	3.50
○Thread 584766			2.50	3.78	66.20	1.83	0.01	3.51
○Thread 584767			2.61	3.78	69.10	1.83	0.01	3.52
○Thread 584768			2.52	3.78	66.59	1.83	0.01	3.51
○Thread 584769			2.59	3.78	68.57	1.83	0.01	3.51
○Thread 584770			2.37	3.78	62.62	1.83	0.01	3.51
○Thread 584771			2.50	3.79	66.18	1.83	0.01	3.51
○Thread 584772			2.47	3.79	65.38	1.83	0.01	3.52
○Thread 584773			2.57	3.79	67.89	1.83	0.01	3.51
○Thread 584774			2.46	3.79	64.98	1.83	0.01	3.50
○Thread 584775			2.51	3.79	66.30	1.83	0.01	3.52
○Thread 584776			2.57	3.79	67.88	1.83	0.01	3.51
○Thread 584777			2.49	3.79	65.89	1.83	0.01	3.52
○Thread 584778			2.52	3.79	66.68	1.83	0.01	3.52
○Thread 584779			2.43	3.79	64.30	1.83	0.01	3.52
○Thread 584780			2.62	3.79	69.18	1.83	0.01	3.51
○Thread 584781			2.68	3.79	70.76	1.83	0.00	3.51
○Thread 584782			2.53	3.79	66.93	1.83	0.00	3.51
○Thread 584783			2.40	3.79	63.36	1.83	0.00	3.52
○Thread 584784			2.41	3.79	63.62	1.83	0.00	3.51
○Thread 584785			2.78	3.79	73.52	1.83	0.00	3.52
○Thread 584786			2.52	3.79	66.65	1.83	0.00	3.51
○Thread 584787			2.52	3.79	66.64	1.83	0.00	3.52
○Thread 584788			2.63	3.79	69.41	1.83	0.00	3.52
○Thread 584789			2.60	3.79	68.75	1.83	0.00	3.50
○Thread 584790			2.57	3.79	67.82	1.83	0.00	3.52
○Thread 584791			2.44	3.79	64.52	1.83	0.00	3.51
○Thread 584792			2.44	3.79	64.51	1.83	0.00	3.52

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	3.49
►Process 584812+		96	3.49
○Thread 584812			3.49	5.78	60.37	0.00	0.00	3.90
○Thread 584912			2.61	3.93	66.43	1.82	0.03	3.68
○Thread 584913			2.48	3.93	63.12	1.82	0.03	3.66
○Thread 584914			2.52	3.93	64.13	1.82	0.03	3.66
○Thread 584915			2.58	3.93	65.66	1.82	0.03	3.65
○Thread 584916			2.60	3.93	66.29	1.82	0.03	3.66
○Thread 584917			2.49	3.93	63.49	1.82	0.03	3.66
○Thread 584918			2.46	3.93	62.59	1.82	0.03	3.65
○Thread 584919			2.60	3.93	66.28	1.82	0.03	3.64
○Thread 584920			2.51	3.93	63.86	1.82	0.03	3.68
○Thread 584921			2.54	3.93	64.74	1.82	0.03	3.64
○Thread 584922			2.55	3.93	64.87	1.82	0.03	3.68
○Thread 584923			2.76	3.93	70.33	1.82	0.03	3.66
○Thread 584924			2.61	3.93	66.51	1.82	0.03	3.67
○Thread 584925			2.52	3.93	64.09	1.82	0.03	3.66
○Thread 584926			2.61	3.93	66.38	1.82	0.03	3.67
○Thread 584927			2.67	3.93	68.03	1.82	0.03	3.67
○Thread 584928			2.50	3.93	63.57	1.82	0.03	3.67
○Thread 584929			2.52	3.93	64.08	1.82	0.03	3.68
○Thread 584930			2.58	3.93	65.73	1.82	0.03	3.66
○Thread 584931			2.45	3.93	62.42	1.82	0.03	3.67
○Thread 584932			2.56	3.93	65.08	1.82	0.03	3.62
○Thread 584933			2.58	3.93	65.59	1.82	0.03	3.66
○Thread 584934			2.48	3.93	63.17	1.82	0.03	3.66
○Thread 584935			2.48	3.93	63.17	1.82	0.03	3.67
○Thread 584936			2.54	3.93	64.69	1.82	0.03	3.67
○Thread 584937			2.46	3.93	62.65	1.82	0.03	3.66
○Thread 584938			2.66	3.93	67.73	1.82	0.03	3.67
○Thread 584939			2.59	3.93	65.82	1.82	0.03	3.67
○Thread 584940			2.58	3.94	65.69	1.82	0.02	3.67
○Thread 584941			2.52	3.94	64.03	1.82	0.02	3.67
○Thread 584942			2.54	3.94	64.67	1.82	0.02	3.65
○Thread 584943			2.59	3.94	65.93	1.82	0.02	3.66
○Thread 584944			2.50	3.94	63.64	1.82	0.02	3.63
○Thread 584945			2.61	3.94	66.43	1.82	0.02	3.65
○Thread 584946			2.51	3.94	63.89	1.82	0.02	3.67
○Thread 584947			2.43	3.94	61.73	1.82	0.02	3.65
○Thread 584948			2.58	3.94	65.53	1.82	0.02	3.64
○Thread 584949			2.46	3.94	62.48	1.82	0.02	3.60
○Thread 584950			2.53	3.94	64.25	1.82	0.02	3.66
○Thread 584951			2.54	3.94	64.63	1.82	0.02	3.67
○Thread 584952			2.39	3.94	60.82	1.82	0.02	3.67
○Thread 584953			2.55	3.94	64.88	1.82	0.02	3.67
○Thread 584954			2.56	3.94	65.13	1.82	0.02	3.66
○Thread 584955			2.53	3.94	64.36	1.82	0.02	3.66
○Thread 584956			2.56	3.94	65.12	1.82	0.02	3.66
○Thread 584957			2.54	3.94	64.61	1.82	0.02	3.67
○Thread 584958			2.54	3.94	64.61	1.82	0.02	3.67
○Thread 584959			2.65	3.94	67.39	1.82	0.02	3.66
○Thread 584960			2.54	3.94	64.47	1.82	0.02	3.65
○Thread 584961			2.50	3.94	63.58	1.82	0.02	3.64
○Thread 584962			2.63	3.94	66.75	1.82	0.02	3.66
○Thread 584963			2.60	3.94	66.11	1.82	0.02	3.66
○Thread 584964			2.48	3.94	63.06	1.82	0.02	3.66
○Thread 584965			2.56	3.94	65.09	1.82	0.02	3.67
○Thread 584966			2.49	3.94	63.18	1.82	0.02	3.66
○Thread 584967			2.58	3.94	65.59	1.82	0.02	3.66
○Thread 584968			2.54	3.94	64.57	1.82	0.01	3.66
○Thread 584969			2.54	3.94	64.44	1.82	0.01	3.66
○Thread 584970			2.44	3.94	62.02	1.83	0.01	3.67
○Thread 584971			2.73	3.94	69.25	1.83	0.01	3.65
○Thread 584972			2.63	3.94	66.71	1.83	0.01	3.67
○Thread 584973			2.46	3.94	62.39	1.83	0.01	3.65
○Thread 584974			2.53	3.94	64.17	1.83	0.01	3.64
○Thread 584975			2.89	3.94	73.29	1.83	0.01	3.67
○Thread 584976			2.51	3.94	63.65	1.83	0.01	3.64
○Thread 584977			2.60	3.94	65.93	1.83	0.01	3.65
○Thread 584978			2.61	3.94	66.18	1.83	0.01	3.66
○Thread 584979			2.55	3.94	64.78	1.83	0.01	3.67
○Thread 584980			2.51	3.94	63.64	1.83	0.01	3.66
○Thread 584981			2.49	3.94	63.13	1.83	0.01	3.62
○Thread 584982			2.48	3.94	62.99	1.83	0.01	3.67
○Thread 584983			2.62	3.95	66.54	1.83	0.01	3.67
○Thread 584984			2.46	3.95	62.35	1.83	0.01	3.66
○Thread 584985			2.46	3.95	62.35	1.83	0.01	3.65
○Thread 584986			2.48	3.95	62.98	1.83	0.01	3.65
○Thread 584987			3.07	3.95	77.80	1.83	0.01	3.67
○Thread 584988			2.60	3.95	66.01	1.83	0.01	3.66
○Thread 584989			2.65	3.95	67.15	1.83	0.01	3.66
○Thread 584990			2.63	3.95	66.76	1.83	0.01	3.66
○Thread 584991			2.75	3.95	69.80	1.83	0.01	3.67
○Thread 584992			2.63	3.95	66.76	1.83	0.01	3.66
○Thread 584993			2.53	3.95	64.22	1.83	0.01	3.58
○Thread 584994			2.51	3.95	63.71	1.83	0.01	3.67
○Thread 584995			2.58	3.95	65.48	1.83	0.00	3.67
○Thread 584996			2.53	3.95	64.08	1.83	0.00	3.65
○Thread 584997			2.56	3.95	64.84	1.83	0.00	3.67
○Thread 584998			2.63	3.95	66.60	1.83	0.00	3.66
○Thread 584999			2.60	3.95	65.84	1.83	0.00	3.65
○Thread 585000			2.46	3.95	62.42	1.83	0.00	3.65
○Thread 585001			2.54	3.95	64.44	1.83	0.00	3.65
○Thread 585002			2.56	3.95	64.82	1.83	0.00	3.65
○Thread 585003			2.63	3.95	66.71	1.83	0.00	3.65
○Thread 585004			2.44	3.95	61.77	1.83	0.00	3.67
○Thread 585005			2.43	3.95	61.64	1.83	0.00	3.68
○Thread 585006			2.41	3.95	61.00	1.83	0.00	3.66

Report Configuration