OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	27.32
►Process 3226931+		1	27.32
○Thread 3226931			27.32	28.27	96.64	0.00	0.00	28.24

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	13.27
►Process 3227100+		2	13.27
○Thread 3227100			13.27	15.48	85.72	0.00	0.00	15.45
○Thread 3227199			13.13	14.61	89.88	0.87	-0.00	14.27

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	6.62
►Process 3227441+		4	6.62
○Thread 3227441			6.54	9.15	71.42	0.00	0.00	9.12
○Thread 3227541			6.62	8.26	80.08	0.89	-0.00	7.92
○Thread 3227542			6.61	8.26	79.96	0.89	-0.00	7.92
○Thread 3227543			6.60	8.26	79.90	0.89	-0.00	7.92

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	3.62
►Process 3228276+		8	3.62
○Thread 3228276			3.33	6.50	51.30	0.00	0.00	6.46
○Thread 3228375			3.62	5.61	64.48	0.89	-0.00	5.27
○Thread 3228376			3.61	5.61	64.39	0.89	-0.00	5.27
○Thread 3228377			3.61	5.61	64.39	0.89	-0.00	5.28
○Thread 3228378			3.61	5.61	64.30	0.89	-0.00	5.27
○Thread 3228379			3.61	5.61	64.39	0.89	-0.00	5.27
○Thread 3228380			3.60	5.61	64.22	0.89	-0.00	5.27
○Thread 3228381			3.61	5.61	64.31	0.89	-0.00	5.27

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	2.20
►Process 3229949+		16	2.20
○Thread 3229949			1.95	5.79	33.75	0.00	0.00	5.76
○Thread 3230048			2.17	4.90	44.34	0.89	-0.00	4.56
○Thread 3230049			2.18	4.90	44.45	0.89	-0.00	4.56
○Thread 3230050			2.18	4.90	44.55	0.89	-0.00	4.56
○Thread 3230051			2.20	4.90	44.86	0.89	-0.00	4.57
○Thread 3230052			2.19	4.90	44.65	0.89	-0.00	4.56
○Thread 3230053			2.18	4.90	44.55	0.89	-0.00	4.56
○Thread 3230054			2.18	4.90	44.45	0.89	-0.00	4.57
○Thread 3230055			2.18	4.90	44.45	0.89	-0.00	4.57
○Thread 3230056			2.17	4.90	44.35	0.89	-0.00	4.56
○Thread 3230057			2.17	4.90	44.35	0.89	-0.00	4.56
○Thread 3230058			2.18	4.90	44.56	0.89	-0.00	4.56
○Thread 3230059			2.18	4.90	44.56	0.89	0.00	4.56
○Thread 3230060			2.18	4.90	44.46	0.89	0.00	4.56
○Thread 3230061			2.18	4.90	44.46	0.89	0.00	4.56
○Thread 3230062			2.18	4.90	44.56	0.89	-0.00	4.56

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	1.62
►Process 3233397+		24	1.62
○Thread 3233397			1.47	5.93	24.89	0.00	0.00	5.89
○Thread 3233496			1.61	5.04	31.98	0.89	-0.00	4.69
○Thread 3233497			1.61	5.03	31.98	0.89	-0.00	4.69
○Thread 3233498			1.62	5.03	32.27	0.89	-0.00	4.69
○Thread 3233499			1.62	5.03	32.18	0.89	-0.00	4.69
○Thread 3233500			1.61	5.03	32.08	0.89	-0.00	4.67
○Thread 3233501			1.62	5.03	32.18	0.89	-0.00	4.69
○Thread 3233502			1.62	5.03	32.18	0.89	-0.00	4.69
○Thread 3233503			1.62	5.03	32.18	0.89	-0.00	4.69
○Thread 3233504			1.61	5.03	32.08	0.89	-0.00	4.69
○Thread 3233505			1.62	5.03	32.18	0.89	-0.00	4.69
○Thread 3233506			1.62	5.03	32.28	0.89	0.00	4.69
○Thread 3233507			1.62	5.03	32.28	0.89	-0.00	4.69
○Thread 3233508			1.62	5.03	32.28	0.89	0.00	4.69
○Thread 3233509			1.60	5.03	31.89	0.89	-0.00	4.69
○Thread 3233510			1.61	5.03	32.08	0.89	0.00	4.69
○Thread 3233511			1.62	5.03	32.18	0.89	0.00	4.69
○Thread 3233512			1.62	5.03	32.19	0.89	0.00	4.69
○Thread 3233513			1.61	5.03	32.09	0.89	0.00	4.69
○Thread 3233514			1.61	5.03	31.99	0.89	0.00	4.69
○Thread 3233515			1.62	5.03	32.19	0.89	0.00	4.69
○Thread 3233516			1.60	5.03	31.89	0.89	0.00	4.69
○Thread 3233517			1.62	5.03	32.19	0.89	0.00	4.69
○Thread 3233518			1.61	5.03	31.99	0.89	-0.00	4.69

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	1.28
►Process 3238621+		32	1.28
○Thread 3238621			0.99	6.36	15.49	0.00	0.00	6.33
○Thread 3238720			1.27	5.46	23.33	0.89	0.00	5.12
○Thread 3238721			1.27	5.46	23.33	0.89	0.00	5.12
○Thread 3238722			1.27	5.46	23.24	0.89	0.00	5.12
○Thread 3238723			1.28	5.46	23.43	0.89	0.00	5.12
○Thread 3238724			1.27	5.46	23.34	0.89	0.00	5.12
○Thread 3238725			1.27	5.46	23.34	0.89	-0.00	5.12
○Thread 3238726			1.27	5.46	23.34	0.89	0.00	5.11
○Thread 3238727			1.27	5.46	23.34	0.89	0.00	5.08
○Thread 3238728			1.28	5.46	23.43	0.89	0.00	5.12
○Thread 3238729			1.28	5.46	23.43	0.89	0.00	5.12
○Thread 3238730			1.28	5.46	23.43	0.89	0.00	5.12
○Thread 3238731			1.27	5.46	23.34	0.89	0.00	5.12
○Thread 3238732			1.28	5.46	23.43	0.90	0.00	5.11
○Thread 3238733			1.27	5.46	23.34	0.90	0.00	5.11
○Thread 3238734			1.28	5.46	23.52	0.90	0.00	5.12
○Thread 3238735			1.27	5.46	23.34	0.90	0.00	5.12
○Thread 3238736			1.28	5.46	23.43	0.90	0.00	5.12
○Thread 3238737			1.28	5.46	23.43	0.90	0.00	5.12
○Thread 3238738			1.27	5.46	23.34	0.90	0.00	5.12
○Thread 3238739			1.27	5.46	23.34	0.90	0.00	5.12
○Thread 3238740			1.26	5.46	23.16	0.90	0.00	5.12
○Thread 3238741			1.27	5.46	23.34	0.90	0.00	5.12
○Thread 3238742			1.28	5.46	23.44	0.90	0.00	5.12
○Thread 3238743			1.28	5.46	23.44	0.90	0.00	5.12
○Thread 3238744			1.27	5.46	23.34	0.90	0.00	5.12
○Thread 3238745			1.28	5.46	23.44	0.90	0.00	5.12
○Thread 3238746			1.27	5.46	23.35	0.90	0.00	5.12
○Thread 3238747			1.27	5.46	23.25	0.90	0.00	5.12
○Thread 3238748			1.28	5.46	23.44	0.90	0.00	5.12
○Thread 3238749			1.27	5.46	23.35	0.90	0.00	5.12
○Thread 3238750			1.27	5.46	23.35	0.90	-0.00	5.11

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	1.10
►Process 3245670+		40	1.10
○Thread 3245670			0.90	6.91	13.09	0.00	0.00	6.87
○Thread 3245769			1.09	6.02	18.10	0.89	-0.00	5.67
○Thread 3245770			1.10	6.02	18.35	0.89	-0.00	5.67
○Thread 3245771			1.09	6.02	18.19	0.89	-0.00	5.67
○Thread 3245772			1.10	6.02	18.35	0.89	-0.00	5.67
○Thread 3245773			1.10	6.02	18.27	0.89	-0.00	5.67
○Thread 3245774			1.10	6.02	18.19	0.89	-0.00	5.67
○Thread 3245775			1.09	6.02	18.11	0.89	-0.00	5.67
○Thread 3245776			1.09	6.02	18.11	0.89	-0.00	5.67
○Thread 3245777			1.10	6.02	18.27	0.89	-0.00	5.67
○Thread 3245778			1.09	6.02	18.11	0.89	-0.00	5.67
○Thread 3245779			1.09	6.02	18.11	0.89	-0.00	5.67
○Thread 3245780			1.09	6.02	18.19	0.89	-0.00	5.67
○Thread 3245781			1.10	6.02	18.27	0.89	-0.00	5.67
○Thread 3245782			1.08	6.02	17.94	0.89	-0.00	5.67
○Thread 3245783			1.10	6.02	18.19	0.89	-0.00	5.66
○Thread 3245784			1.10	6.02	18.28	0.89	-0.00	5.67
○Thread 3245785			1.08	6.02	18.03	0.90	-0.00	5.66
○Thread 3245786			1.09	6.02	18.19	0.90	-0.00	5.66
○Thread 3245787			1.10	6.02	18.28	0.90	-0.00	5.67
○Thread 3245788			1.10	6.02	18.19	0.90	-0.00	5.67
○Thread 3245789			1.09	6.02	18.19	0.90	-0.00	5.66
○Thread 3245790			1.09	6.02	18.11	0.90	-0.00	5.66
○Thread 3245791			1.09	6.02	18.11	0.90	-0.00	5.66
○Thread 3245792			1.10	6.02	18.20	0.90	-0.00	5.67
○Thread 3245793			1.08	6.02	18.03	0.90	-0.00	5.67
○Thread 3245794			1.09	6.02	18.20	0.90	-0.00	5.66
○Thread 3245795			1.09	6.02	18.11	0.90	-0.00	5.67
○Thread 3245796			1.09	6.02	18.20	0.90	-0.00	5.67
○Thread 3245797			1.08	6.02	17.95	0.90	-0.00	5.66
○Thread 3245798			1.09	6.02	18.11	0.90	-0.00	5.66
○Thread 3245799			1.09	6.02	18.20	0.90	-0.00	5.67
○Thread 3245800			1.08	6.02	17.95	0.90	-0.00	5.67
○Thread 3245801			1.08	6.02	17.95	0.90	-0.00	5.67
○Thread 3245802			1.09	6.02	18.20	0.90	-0.00	5.66
○Thread 3245803			1.08	6.02	18.03	0.90	-0.00	5.62
○Thread 3245804			1.09	6.02	18.12	0.90	-0.00	5.67
○Thread 3245805			1.09	6.02	18.20	0.90	-0.00	5.66
○Thread 3245806			1.08	6.02	18.03	0.90	-0.00	5.67
○Thread 3245807			1.09	6.02	18.12	0.90	-0.00	5.67

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	0.97
►Process 3254447+		48	0.97
○Thread 3254447			0.89	7.49	11.88	0.00	0.00	7.46
○Thread 3254547			0.95	6.60	14.39	0.89	-0.00	6.24
○Thread 3254548			0.97	6.60	14.70	0.89	-0.00	6.25
○Thread 3254549			0.97	6.60	14.70	0.89	-0.00	6.25
○Thread 3254550			0.96	6.60	14.54	0.89	-0.00	6.25
○Thread 3254551			0.96	6.60	14.62	0.89	-0.00	6.25
○Thread 3254552			0.96	6.60	14.54	0.89	-0.00	6.25
○Thread 3254553			0.97	6.60	14.70	0.89	-0.00	6.25
○Thread 3254554			0.96	6.60	14.55	0.89	-0.00	6.24
○Thread 3254555			0.95	6.60	14.39	0.89	-0.00	6.25
○Thread 3254556			0.97	6.60	14.70	0.89	-0.00	6.25
○Thread 3254557			0.96	6.60	14.55	0.89	-0.00	6.20
○Thread 3254558			0.95	6.60	14.39	0.89	-0.00	6.24
○Thread 3254559			0.95	6.60	14.47	0.89	-0.00	6.24
○Thread 3254560			0.95	6.60	14.47	0.89	-0.00	6.25
○Thread 3254561			0.95	6.60	14.47	0.89	-0.00	6.24
○Thread 3254562			0.96	6.60	14.55	0.89	-0.00	6.25
○Thread 3254563			0.95	6.60	14.40	0.89	-0.00	6.24
○Thread 3254564			0.95	6.60	14.40	0.89	-0.00	6.24
○Thread 3254565			0.94	6.60	14.32	0.89	-0.00	6.24
○Thread 3254566			0.96	6.60	14.55	0.89	-0.00	6.25
○Thread 3254567			0.95	6.60	14.47	0.89	-0.00	6.24
○Thread 3254568			0.97	6.60	14.70	0.89	-0.00	6.25
○Thread 3254569			0.96	6.60	14.55	0.89	-0.00	6.25
○Thread 3254570			0.94	6.60	14.32	0.89	-0.00	6.23
○Thread 3254571			0.95	6.60	14.40	0.89	-0.00	6.24
○Thread 3254572			0.96	6.60	14.55	0.89	-0.00	6.24
○Thread 3254573			0.95	6.60	14.47	0.89	-0.00	6.24
○Thread 3254574			0.96	6.60	14.63	0.89	-0.00	6.24
○Thread 3254575			0.96	6.60	14.63	0.89	-0.00	6.24
○Thread 3254576			0.95	6.60	14.40	0.89	-0.00	6.23
○Thread 3254577			0.94	6.60	14.25	0.89	-0.00	6.25
○Thread 3254578			0.95	6.60	14.48	0.90	-0.00	6.24
○Thread 3254579			0.95	6.60	14.40	0.90	-0.00	6.24
○Thread 3254580			0.95	6.60	14.48	0.90	-0.00	6.24
○Thread 3254581			0.96	6.60	14.55	0.90	-0.00	6.24
○Thread 3254582			0.95	6.60	14.48	0.90	-0.00	6.24
○Thread 3254583			0.97	6.60	14.70	0.90	-0.00	6.25
○Thread 3254584			0.96	6.60	14.55	0.90	-0.00	6.25
○Thread 3254585			0.95	6.60	14.40	0.90	-0.00	6.25
○Thread 3254586			0.96	6.60	14.55	0.90	-0.00	6.24
○Thread 3254587			0.94	6.60	14.33	0.90	-0.00	6.25
○Thread 3254588			0.95	6.60	14.40	0.90	-0.00	6.25
○Thread 3254589			0.95	6.60	14.48	0.90	-0.00	6.24
○Thread 3254590			0.95	6.60	14.40	0.90	-0.00	6.25
○Thread 3254591			0.95	6.60	14.40	0.90	-0.00	6.25
○Thread 3254592			0.95	6.60	14.40	0.90	-0.00	6.24
○Thread 3254593			0.95	6.60	14.40	0.90	-0.00	6.24

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	0.90
►Process 3265000+		56	0.90
○Thread 3265000			0.86	8.17	10.47	0.00	0.00	8.13
○Thread 3265099			0.90	7.27	12.37	0.89	-0.00	6.92
○Thread 3265100			0.87	7.27	11.96	0.89	-0.00	6.89
○Thread 3265101			0.86	7.27	11.82	0.89	-0.00	6.91
○Thread 3265102			0.88	7.27	12.03	0.89	-0.00	6.91
○Thread 3265103			0.87	7.27	11.96	0.89	-0.00	6.91
○Thread 3265104			0.88	7.27	12.17	0.89	-0.00	6.92
○Thread 3265105			0.87	7.27	11.89	0.89	-0.00	6.92
○Thread 3265106			0.87	7.27	11.96	0.89	-0.00	6.89
○Thread 3265107			0.88	7.27	12.03	0.89	-0.00	6.89
○Thread 3265108			0.88	7.27	12.17	0.89	-0.00	6.91
○Thread 3265109			0.87	7.27	11.89	0.89	-0.00	6.91
○Thread 3265110			0.87	7.27	11.96	0.89	-0.00	6.92
○Thread 3265111			0.88	7.27	12.17	0.89	-0.00	6.91
○Thread 3265112			0.89	7.27	12.31	0.89	-0.00	6.92
○Thread 3265113			0.86	7.27	11.76	0.89	-0.00	6.88
○Thread 3265114			0.88	7.27	12.17	0.89	-0.00	6.89
○Thread 3265115			0.88	7.27	12.10	0.89	-0.00	6.91
○Thread 3265116			0.88	7.27	12.10	0.89	-0.00	6.91
○Thread 3265117			0.88	7.27	12.03	0.89	-0.00	6.84
○Thread 3265118			0.89	7.27	12.24	0.89	-0.00	6.92
○Thread 3265119			0.86	7.27	11.83	0.89	-0.00	6.84
○Thread 3265120			0.88	7.27	12.03	0.89	-0.00	6.89
○Thread 3265121			0.89	7.27	12.24	0.90	-0.00	6.92
○Thread 3265122			0.88	7.27	12.10	0.90	-0.00	6.84
○Thread 3265123			0.88	7.27	12.03	0.90	-0.00	6.92
○Thread 3265124			0.89	7.27	12.24	0.90	-0.00	6.91
○Thread 3265125			0.87	7.27	11.96	0.90	-0.00	6.91
○Thread 3265126			0.88	7.27	12.03	0.90	-0.00	6.91
○Thread 3265127			0.88	7.27	12.03	0.90	-0.00	6.89
○Thread 3265128			0.87	7.27	11.90	0.90	-0.00	6.92
○Thread 3265129			0.88	7.27	12.03	0.90	-0.00	6.92
○Thread 3265130			0.88	7.27	12.03	0.90	-0.00	6.91
○Thread 3265131			0.85	7.27	11.69	0.90	-0.00	6.91
○Thread 3265132			0.87	7.27	11.90	0.90	-0.00	6.91
○Thread 3265133			0.87	7.27	11.97	0.90	-0.00	6.91
○Thread 3265134			0.88	7.27	12.04	0.90	-0.00	6.92
○Thread 3265135			0.86	7.27	11.83	0.90	-0.00	6.92
○Thread 3265136			0.88	7.27	12.17	0.90	-0.00	6.90
○Thread 3265137			0.87	7.27	11.97	0.90	-0.00	6.91
○Thread 3265138			0.87	7.27	11.90	0.90	-0.00	6.92
○Thread 3265139			0.87	7.27	11.97	0.90	-0.00	6.84
○Thread 3265140			0.87	7.27	11.97	0.90	-0.00	6.91
○Thread 3265141			0.87	7.27	11.90	0.90	-0.00	6.92
○Thread 3265142			0.88	7.27	12.04	0.90	-0.00	6.93
○Thread 3265143			0.86	7.27	11.76	0.90	-0.00	6.89
○Thread 3265144			0.88	7.27	12.04	0.90	-0.00	6.93
○Thread 3265145			0.87	7.27	11.97	0.90	-0.00	6.92
○Thread 3265146			0.87	7.27	11.90	0.90	-0.00	6.92
○Thread 3265147			0.85	7.27	11.69	0.90	-0.00	6.92
○Thread 3265148			0.87	7.27	11.97	0.90	-0.00	6.92
○Thread 3265149			0.88	7.27	12.11	0.90	-0.00	6.93
○Thread 3265150			0.87	7.27	11.97	0.90	-0.00	6.92
○Thread 3265151			0.87	7.27	11.97	0.90	-0.00	6.92
○Thread 3265152			0.88	7.27	12.04	0.90	-0.00	6.92
○Thread 3265153			0.87	7.27	11.97	0.90	-0.00	6.92

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	0.88
►Process 3277329+		64	0.88
○Thread 3277329			0.88	8.82	9.92	0.00	0.00	8.79
○Thread 3277428			0.81	7.93	10.22	0.89	0.00	7.56
○Thread 3277429			0.81	7.93	10.16	0.89	0.00	7.56
○Thread 3277430			0.81	7.93	10.16	0.89	0.00	7.56
○Thread 3277431			0.81	7.93	10.16	0.89	-0.00	7.53
○Thread 3277432			0.81	7.93	10.16	0.89	0.00	7.56
○Thread 3277433			0.80	7.93	10.03	0.89	0.00	7.53
○Thread 3277434			0.81	7.93	10.16	0.89	-0.00	7.56
○Thread 3277435			0.81	7.93	10.22	0.89	-0.00	7.57
○Thread 3277436			0.80	7.93	10.03	0.89	0.00	7.57
○Thread 3277437			0.80	7.93	10.03	0.89	0.00	7.56
○Thread 3277438			0.81	7.93	10.28	0.89	0.00	7.53
○Thread 3277439			0.80	7.93	10.09	0.90	0.00	7.56
○Thread 3277440			0.81	7.93	10.16	0.90	0.00	7.56
○Thread 3277441			0.80	7.93	10.09	0.90	0.00	7.56
○Thread 3277442			0.79	7.93	9.97	0.90	-0.00	7.56
○Thread 3277443			0.80	7.92	10.09	0.90	0.00	7.56
○Thread 3277444			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277445			0.77	7.92	9.78	0.90	0.00	7.56
○Thread 3277446			0.78	7.92	9.84	0.90	0.00	7.56
○Thread 3277447			0.81	7.92	10.16	0.90	0.00	7.53
○Thread 3277448			0.77	7.92	9.72	0.90	0.00	7.56
○Thread 3277449			0.80	7.92	10.10	0.90	0.00	7.56
○Thread 3277450			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277451			0.79	7.92	9.91	0.90	-0.00	7.56
○Thread 3277452			0.80	7.92	10.03	0.90	0.00	7.56
○Thread 3277453			0.79	7.92	9.91	0.90	0.00	7.48
○Thread 3277454			0.79	7.92	9.91	0.90	0.00	7.56
○Thread 3277455			0.78	7.92	9.84	0.90	0.00	7.56
○Thread 3277456			0.80	7.92	10.10	0.90	0.00	7.56
○Thread 3277457			0.79	7.92	9.91	0.90	0.00	7.56
○Thread 3277458			0.77	7.92	9.78	0.90	0.00	7.53
○Thread 3277459			0.79	7.92	9.91	0.90	0.00	7.56
○Thread 3277460			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277461			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277462			0.80	7.92	10.03	0.90	0.00	7.56
○Thread 3277463			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277464			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277465			0.77	7.92	9.78	0.90	0.00	7.56
○Thread 3277466			0.80	7.92	10.10	0.90	0.00	7.53
○Thread 3277467			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277468			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277469			0.79	7.92	9.97	0.90	0.00	7.56
○Thread 3277470			0.77	7.92	9.78	0.90	0.00	7.56
○Thread 3277471			0.80	7.92	10.04	0.90	0.00	7.56
○Thread 3277472			0.80	7.92	10.10	0.90	0.00	7.53
○Thread 3277473			0.81	7.92	10.16	0.90	0.00	7.56
○Thread 3277474			0.79	7.92	9.91	0.90	0.00	7.56
○Thread 3277475			0.79	7.92	9.91	0.90	0.00	7.56
○Thread 3277476			0.79	7.92	9.91	0.90	0.00	7.45
○Thread 3277477			0.80	7.92	10.04	0.90	0.00	7.57
○Thread 3277478			0.78	7.92	9.85	0.90	0.00	7.56
○Thread 3277479			0.81	7.92	10.16	0.90	0.00	7.56
○Thread 3277480			0.80	7.92	10.04	0.90	0.00	7.56
○Thread 3277481			0.77	7.92	9.78	0.90	0.00	7.56
○Thread 3277482			0.76	7.92	9.60	0.90	0.00	7.56
○Thread 3277483			0.76	7.92	9.66	0.90	0.00	7.56
○Thread 3277484			0.80	7.92	10.10	0.90	0.00	7.57
○Thread 3277485			0.79	7.92	9.91	0.90	0.00	7.56
○Thread 3277486			0.78	7.92	9.85	0.90	0.00	7.48
○Thread 3277487			0.79	7.92	9.91	0.90	0.00	7.56
○Thread 3277488			0.78	7.92	9.85	0.90	0.00	7.56
○Thread 3277489			0.76	7.92	9.66	0.90	0.00	7.56
○Thread 3277490			0.77	7.92	9.79	0.90	0.00	7.53

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	0.86
►Process 3291482+		72	0.86
○Thread 3291482			0.86	9.51	9.04	0.00	0.00	9.46
○Thread 3291581			0.74	8.63	8.52	0.88	-0.00	8.26
○Thread 3291582			0.74	8.63	8.52	0.88	-0.00	8.27
○Thread 3291583			0.75	8.63	8.69	0.88	-0.00	8.26
○Thread 3291584			0.75	8.63	8.75	0.88	-0.00	8.27
○Thread 3291585			0.75	8.63	8.75	0.88	-0.00	8.23
○Thread 3291586			0.75	8.63	8.64	0.88	-0.00	8.18
○Thread 3291587			0.75	8.63	8.69	0.88	-0.00	8.27
○Thread 3291588			0.74	8.63	8.58	0.88	-0.00	8.24
○Thread 3291589			0.75	8.63	8.69	0.88	-0.00	8.20
○Thread 3291590			0.76	8.63	8.81	0.88	-0.00	8.27
○Thread 3291591			0.75	8.63	8.64	0.89	-0.00	8.26
○Thread 3291592			0.75	8.63	8.75	0.89	-0.00	8.24
○Thread 3291593			0.75	8.63	8.69	0.89	-0.00	8.27
○Thread 3291594			0.73	8.63	8.46	0.89	-0.00	8.26
○Thread 3291595			0.74	8.63	8.52	0.89	-0.00	8.26
○Thread 3291596			0.75	8.63	8.75	0.89	-0.00	8.23
○Thread 3291597			0.75	8.63	8.64	0.89	-0.00	8.27
○Thread 3291598			0.75	8.63	8.75	0.89	-0.00	8.27
○Thread 3291599			0.75	8.63	8.69	0.89	-0.00	8.26
○Thread 3291600			0.75	8.63	8.64	0.89	-0.00	8.20
○Thread 3291601			0.75	8.63	8.75	0.89	-0.00	8.23
○Thread 3291602			0.74	8.63	8.58	0.89	-0.00	8.23
○Thread 3291603			0.76	8.63	8.81	0.89	-0.00	8.27
○Thread 3291604			0.74	8.63	8.58	0.89	-0.00	8.26
○Thread 3291605			0.74	8.63	8.52	0.89	-0.00	8.27
○Thread 3291606			0.74	8.63	8.52	0.89	-0.00	8.24
○Thread 3291607			0.73	8.62	8.46	0.89	-0.00	8.26
○Thread 3291608			0.74	8.62	8.58	0.89	-0.00	8.26
○Thread 3291609			0.74	8.62	8.52	0.89	-0.00	8.26
○Thread 3291610			0.75	8.62	8.70	0.89	-0.00	8.26
○Thread 3291611			0.73	8.62	8.41	0.89	-0.00	8.26
○Thread 3291612			0.75	8.62	8.64	0.89	-0.00	8.27
○Thread 3291613			0.75	8.62	8.64	0.89	-0.00	8.27
○Thread 3291614			0.75	8.62	8.75	0.89	-0.00	8.27
○Thread 3291615			0.75	8.62	8.70	0.89	-0.00	8.26
○Thread 3291616			0.76	8.62	8.81	0.89	-0.00	8.27
○Thread 3291617			0.74	8.62	8.58	0.89	-0.00	8.27
○Thread 3291618			0.75	8.62	8.64	0.89	-0.00	8.27
○Thread 3291619			0.75	8.62	8.70	0.89	-0.00	8.27
○Thread 3291620			0.75	8.62	8.64	0.89	-0.00	8.27
○Thread 3291621			0.75	8.62	8.70	0.89	-0.00	8.27
○Thread 3291622			0.75	8.62	8.76	0.89	-0.00	8.27
○Thread 3291623			0.75	8.62	8.70	0.89	-0.00	8.26
○Thread 3291624			0.75	8.62	8.76	0.89	-0.00	8.26
○Thread 3291625			0.74	8.62	8.58	0.89	-0.00	8.20
○Thread 3291626			0.75	8.62	8.70	0.89	-0.00	8.26
○Thread 3291627			0.75	8.62	8.70	0.89	-0.00	8.26
○Thread 3291628			0.74	8.62	8.52	0.89	-0.00	8.26
○Thread 3291629			0.74	8.62	8.58	0.89	-0.00	8.23
○Thread 3291630			0.71	8.62	8.29	0.89	-0.00	8.26
○Thread 3291631			0.73	8.62	8.41	0.89	-0.00	8.26
○Thread 3291632			0.74	8.62	8.58	0.89	-0.00	8.24
○Thread 3291633			0.71	8.62	8.23	0.89	-0.00	8.17
○Thread 3291634			0.72	8.62	8.35	0.89	-0.00	8.26
○Thread 3291635			0.74	8.62	8.52	0.89	-0.00	8.26
○Thread 3291636			0.75	8.62	8.64	0.89	-0.00	8.26
○Thread 3291637			0.73	8.62	8.47	0.89	-0.00	8.26
○Thread 3291638			0.75	8.62	8.70	0.89	-0.00	8.27
○Thread 3291639			0.74	8.62	8.53	0.89	-0.00	8.20
○Thread 3291640			0.71	8.62	8.29	0.89	-0.00	8.27
○Thread 3291641			0.72	8.62	8.35	0.89	-0.00	8.26
○Thread 3291642			0.74	8.62	8.58	0.89	-0.00	8.27
○Thread 3291643			0.75	8.62	8.70	0.89	-0.00	8.26
○Thread 3291644			0.74	8.62	8.58	0.89	-0.00	8.26
○Thread 3291645			0.76	8.62	8.82	0.89	-0.00	8.27
○Thread 3291646			0.74	8.62	8.58	0.89	-0.00	8.27
○Thread 3291647			0.75	8.62	8.70	0.89	-0.00	8.27
○Thread 3291648			0.74	8.62	8.58	0.89	-0.00	8.27
○Thread 3291649			0.75	8.62	8.76	0.89	-0.00	8.27
○Thread 3291650			0.74	8.62	8.58	0.89	-0.00	8.27
○Thread 3291651			0.73	8.62	8.47	0.89	-0.00	8.24

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	0.70
►Process 3307362+		80	0.70
○Thread 3307362			0.63	10.19	6.18	0.00	0.00	10.14
○Thread 3307461			0.69	9.31	7.47	0.88	0.00	8.95
○Thread 3307462			0.69	9.31	7.47	0.88	-0.00	8.95
○Thread 3307463			0.69	9.31	7.47	0.89	0.00	8.95
○Thread 3307464			0.69	9.31	7.42	0.89	0.00	8.91
○Thread 3307465			0.70	9.31	7.52	0.89	0.00	8.94
○Thread 3307466			0.69	9.30	7.47	0.89	0.00	8.88
○Thread 3307467			0.68	9.30	7.31	0.89	0.00	8.94
○Thread 3307468			0.67	9.30	7.15	0.89	0.00	8.95
○Thread 3307469			0.69	9.30	7.42	0.89	0.00	8.94
○Thread 3307470			0.69	9.30	7.36	0.89	0.00	8.95
○Thread 3307471			0.68	9.30	7.25	0.89	0.00	8.91
○Thread 3307472			0.69	9.30	7.36	0.89	0.00	8.94
○Thread 3307473			0.70	9.30	7.52	0.89	0.00	8.94
○Thread 3307474			0.69	9.30	7.42	0.89	-0.00	8.95
○Thread 3307475			0.68	9.30	7.31	0.89	0.00	8.81
○Thread 3307476			0.69	9.30	7.36	0.89	0.00	8.80
○Thread 3307477			0.68	9.30	7.26	0.89	-0.00	8.94
○Thread 3307478			0.69	9.30	7.42	0.89	0.00	8.94
○Thread 3307479			0.68	9.30	7.31	0.89	0.00	8.94
○Thread 3307480			0.69	9.30	7.42	0.89	0.00	8.94
○Thread 3307481			0.69	9.30	7.42	0.89	0.00	8.94
○Thread 3307482			0.68	9.30	7.26	0.89	0.00	8.94
○Thread 3307483			0.69	9.30	7.47	0.89	0.00	8.93
○Thread 3307484			0.69	9.30	7.47	0.89	-0.00	8.94
○Thread 3307485			0.69	9.30	7.47	0.89	0.00	8.95
○Thread 3307486			0.68	9.30	7.26	0.89	0.00	8.93
○Thread 3307487			0.68	9.30	7.31	0.89	-0.00	8.90
○Thread 3307488			0.69	9.30	7.42	0.89	-0.00	8.95
○Thread 3307489			0.68	9.30	7.26	0.89	-0.00	8.94
○Thread 3307490			0.69	9.30	7.47	0.89	-0.00	8.95
○Thread 3307491			0.69	9.30	7.42	0.89	-0.00	8.87
○Thread 3307492			0.67	9.30	7.20	0.89	0.00	8.87
○Thread 3307493			0.69	9.30	7.42	0.89	-0.00	8.95
○Thread 3307494			0.69	9.30	7.36	0.89	-0.00	8.93
○Thread 3307495			0.69	9.30	7.36	0.89	-0.00	8.93
○Thread 3307496			0.70	9.30	7.53	0.89	-0.00	8.93
○Thread 3307497			0.69	9.30	7.47	0.89	-0.00	8.94
○Thread 3307498			0.69	9.30	7.42	0.89	0.00	8.83
○Thread 3307499			0.68	9.30	7.31	0.89	-0.00	8.93
○Thread 3307500			0.69	9.30	7.36	0.89	-0.00	8.94
○Thread 3307501			0.67	9.30	7.20	0.89	-0.00	8.90
○Thread 3307502			0.67	9.30	7.20	0.89	-0.00	8.91
○Thread 3307503			0.68	9.30	7.31	0.89	-0.00	8.90
○Thread 3307504			0.70	9.30	7.53	0.89	-0.00	8.94
○Thread 3307505			0.70	9.30	7.53	0.89	-0.00	8.95
○Thread 3307506			0.67	9.30	7.15	0.89	-0.00	8.94
○Thread 3307507			0.67	9.30	7.20	0.89	-0.00	8.94
○Thread 3307508			0.69	9.30	7.37	0.89	-0.00	8.88
○Thread 3307509			0.69	9.30	7.37	0.89	-0.00	8.94
○Thread 3307510			0.69	9.30	7.42	0.89	-0.00	8.95
○Thread 3307511			0.67	9.30	7.20	0.89	-0.00	8.87
○Thread 3307512			0.67	9.30	7.15	0.89	-0.00	8.84
○Thread 3307513			0.68	9.30	7.26	0.89	-0.00	8.94
○Thread 3307514			0.68	9.30	7.31	0.89	-0.00	8.95
○Thread 3307515			0.68	9.30	7.26	0.89	-0.00	8.91
○Thread 3307516			0.68	9.30	7.31	0.89	-0.00	8.94
○Thread 3307517			0.69	9.30	7.42	0.89	-0.00	8.95
○Thread 3307518			0.69	9.30	7.42	0.89	-0.00	8.95
○Thread 3307519			0.68	9.30	7.26	0.89	-0.00	8.91
○Thread 3307520			0.69	9.30	7.47	0.89	-0.00	8.95
○Thread 3307521			0.68	9.30	7.31	0.89	-0.00	8.95
○Thread 3307522			0.69	9.30	7.42	0.89	-0.00	8.95
○Thread 3307523			0.67	9.30	7.21	0.89	-0.00	8.91
○Thread 3307524			0.69	9.30	7.42	0.89	-0.00	8.94
○Thread 3307525			0.69	9.30	7.37	0.89	-0.00	8.95
○Thread 3307526			0.68	9.30	7.31	0.89	-0.00	8.80
○Thread 3307527			0.68	9.30	7.31	0.89	-0.00	8.93
○Thread 3307528			0.66	9.30	7.10	0.89	-0.00	8.83
○Thread 3307529			0.69	9.30	7.37	0.89	-0.00	8.95
○Thread 3307530			0.68	9.30	7.26	0.89	-0.00	8.90
○Thread 3307531			0.69	9.30	7.37	0.89	-0.00	8.87
○Thread 3307532			0.67	9.30	7.21	0.89	-0.00	8.94
○Thread 3307533			0.66	9.30	7.10	0.89	-0.00	8.91
○Thread 3307534			0.69	9.30	7.37	0.89	-0.00	8.88
○Thread 3307535			0.69	9.30	7.37	0.89	-0.00	8.91
○Thread 3307536			0.67	9.30	7.15	0.89	-0.00	8.87
○Thread 3307537			0.67	9.30	7.15	0.89	-0.00	8.94
○Thread 3307538			0.67	9.30	7.21	0.89	-0.00	8.84
○Thread 3307539			0.64	9.30	6.94	0.89	-0.00	8.94

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	0.68
►Process 3325018+		88	0.68
○Thread 3325018			0.64	10.90	5.87	0.00	0.00	10.87
○Thread 3325117			0.65	10.01	6.54	0.89	-0.00	9.61
○Thread 3325118			0.66	10.01	6.59	0.89	-0.00	9.57
○Thread 3325119			0.67	10.01	6.69	0.89	-0.00	9.65
○Thread 3325120			0.65	10.01	6.55	0.89	-0.00	9.57
○Thread 3325121			0.65	10.01	6.55	0.89	-0.00	9.64
○Thread 3325122			0.66	10.01	6.60	0.90	-0.00	9.64
○Thread 3325123			0.63	10.01	6.35	0.90	-0.00	9.60
○Thread 3325124			0.67	10.01	6.70	0.90	-0.00	9.64
○Thread 3325125			0.67	10.01	6.65	0.90	-0.00	9.57
○Thread 3325126			0.67	10.01	6.65	0.90	-0.00	9.65
○Thread 3325127			0.67	10.01	6.70	0.90	-0.00	9.60
○Thread 3325128			0.66	10.01	6.60	0.90	-0.00	9.60
○Thread 3325129			0.67	10.01	6.70	0.90	-0.00	9.65
○Thread 3325130			0.65	10.01	6.50	0.90	-0.00	9.60
○Thread 3325131			0.65	10.01	6.50	0.90	-0.00	9.65
○Thread 3325132			0.65	10.01	6.55	0.90	-0.00	9.60
○Thread 3325133			0.64	10.01	6.45	0.90	-0.00	9.64
○Thread 3325134			0.66	10.01	6.60	0.90	-0.00	9.64
○Thread 3325135			0.65	10.01	6.55	0.90	-0.00	9.64
○Thread 3325136			0.67	10.01	6.65	0.90	-0.00	9.61
○Thread 3325137			0.67	10.01	6.65	0.90	-0.00	9.60
○Thread 3325138			0.67	10.01	6.65	0.90	-0.00	9.60
○Thread 3325139			0.66	10.01	6.60	0.90	-0.00	9.65
○Thread 3325140			0.66	10.01	6.60	0.90	-0.00	9.60
○Thread 3325141			0.67	10.01	6.65	0.90	-0.00	9.60
○Thread 3325142			0.65	10.01	6.55	0.90	-0.00	9.64
○Thread 3325143			0.66	10.00	6.60	0.90	-0.00	9.60
○Thread 3325144			0.65	10.00	6.55	0.90	-0.00	9.60
○Thread 3325145			0.67	10.00	6.65	0.90	-0.00	9.60
○Thread 3325146			0.66	10.00	6.60	0.90	-0.00	9.60
○Thread 3325147			0.66	10.00	6.60	0.90	-0.00	9.65
○Thread 3325148			0.65	10.00	6.50	0.90	-0.00	9.57
○Thread 3325149			0.67	10.00	6.65	0.90	-0.00	9.65
○Thread 3325150			0.67	10.00	6.70	0.90	-0.00	9.64
○Thread 3325151			0.65	10.00	6.50	0.90	-0.00	9.64
○Thread 3325152			0.65	10.00	6.55	0.90	-0.00	9.57
○Thread 3325153			0.68	10.00	6.80	0.90	-0.00	9.65
○Thread 3325154			0.66	10.00	6.60	0.90	-0.00	9.65
○Thread 3325155			0.67	10.00	6.70	0.90	-0.00	9.65
○Thread 3325156			0.67	10.00	6.65	0.90	-0.00	9.65
○Thread 3325157			0.64	10.00	6.45	0.90	-0.00	9.65
○Thread 3325158			0.65	10.00	6.55	0.90	-0.00	9.60
○Thread 3325159			0.64	10.00	6.40	0.90	-0.00	9.60
○Thread 3325160			0.64	10.00	6.40	0.90	-0.00	9.65
○Thread 3325161			0.64	10.00	6.40	0.90	-0.00	9.60
○Thread 3325162			0.66	10.00	6.60	0.90	-0.00	9.65
○Thread 3325163			0.64	10.00	6.40	0.90	-0.00	9.64
○Thread 3325164			0.63	10.00	6.25	0.90	-0.00	9.60
○Thread 3325165			0.65	10.00	6.50	0.90	-0.00	9.60
○Thread 3325166			0.63	10.00	6.30	0.90	-0.00	9.61
○Thread 3325167			0.63	10.00	6.35	0.90	-0.00	9.57
○Thread 3325168			0.62	10.00	6.20	0.90	-0.00	9.64
○Thread 3325169			0.63	10.00	6.35	0.90	-0.00	9.64
○Thread 3325170			0.63	10.00	6.35	0.90	-0.00	9.64
○Thread 3325171			0.63	10.00	6.25	0.90	-0.00	9.60
○Thread 3325172			0.64	10.00	6.40	0.90	-0.00	9.65
○Thread 3325173			0.63	10.00	6.30	0.90	-0.00	9.61
○Thread 3325174			0.66	10.00	6.60	0.90	-0.00	9.57
○Thread 3325175			0.63	10.00	6.25	0.90	-0.00	9.64
○Thread 3325176			0.65	10.00	6.55	0.90	-0.00	9.61
○Thread 3325177			0.64	10.00	6.45	0.90	-0.00	9.64
○Thread 3325178			0.64	10.00	6.40	0.90	-0.00	9.64
○Thread 3325179			0.65	10.00	6.50	0.90	-0.00	9.65
○Thread 3325180			0.65	10.00	6.50	0.90	-0.00	9.64
○Thread 3325181			0.65	10.00	6.50	0.90	-0.00	9.65
○Thread 3325182			0.65	10.00	6.50	0.90	-0.00	9.64
○Thread 3325183			0.65	10.00	6.55	0.90	-0.00	9.65
○Thread 3325184			0.65	10.00	6.50	0.90	-0.00	9.64
○Thread 3325185			0.64	10.00	6.40	0.90	-0.00	9.64
○Thread 3325186			0.65	10.00	6.50	0.90	-0.00	9.65
○Thread 3325187			0.64	10.00	6.40	0.90	-0.00	9.64
○Thread 3325188			0.65	10.00	6.50	0.90	-0.00	9.64
○Thread 3325189			0.65	10.00	6.50	0.90	-0.00	9.65
○Thread 3325190			0.64	10.00	6.45	0.90	-0.00	9.65
○Thread 3325191			0.64	10.00	6.45	0.90	-0.00	9.65
○Thread 3325192			0.64	10.00	6.45	0.90	-0.00	9.64
○Thread 3325193			0.64	10.00	6.45	0.90	-0.00	9.64
○Thread 3325194			0.67	10.00	6.70	0.90	-0.00	9.65
○Thread 3325195			0.64	10.00	6.45	0.90	-0.00	9.64
○Thread 3325196			0.64	10.00	6.45	0.90	-0.00	9.64
○Thread 3325197			0.63	10.00	6.35	0.90	-0.00	9.64
○Thread 3325198			0.65	10.00	6.50	0.90	-0.00	9.60
○Thread 3325199			0.63	10.00	6.35	0.90	-0.00	9.64
○Thread 3325200			0.64	10.00	6.40	0.90	-0.00	9.57
○Thread 3325201			0.64	10.00	6.45	0.90	-0.00	9.65
○Thread 3325202			0.65	10.00	6.55	0.90	-0.00	9.65
○Thread 3325203			0.63	10.00	6.25	0.90	-0.00	9.64

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	0.67
►Process 3344459+		96	0.67
○Thread 3344459			0.51	11.66	4.37	0.00	0.00	11.61
○Thread 3344558			0.63	10.78	5.89	0.88	-0.00	10.41
○Thread 3344559			0.65	10.78	6.03	0.88	-0.00	10.42
○Thread 3344560			0.64	10.78	5.94	0.88	-0.00	10.37
○Thread 3344561			0.64	10.78	5.94	0.88	-0.00	10.41
○Thread 3344562			0.64	10.78	5.98	0.88	-0.00	10.42
○Thread 3344563			0.63	10.78	5.85	0.88	-0.00	10.33
○Thread 3344564			0.64	10.78	5.98	0.88	-0.00	10.42
○Thread 3344565			0.65	10.78	6.03	0.89	-0.00	10.42
○Thread 3344566			0.62	10.78	5.75	0.89	-0.00	10.25
○Thread 3344567			0.64	10.78	5.98	0.89	-0.00	10.41
○Thread 3344568			0.64	10.78	5.94	0.89	-0.00	10.33
○Thread 3344569			0.63	10.78	5.80	0.89	-0.00	10.41
○Thread 3344570			0.65	10.78	6.03	0.89	-0.00	10.42
○Thread 3344571			0.65	10.78	6.08	0.89	-0.00	10.41
○Thread 3344572			0.65	10.78	6.03	0.89	-0.00	10.42
○Thread 3344573			0.64	10.78	5.99	0.89	-0.00	10.41
○Thread 3344574			0.64	10.78	5.99	0.89	-0.00	10.42
○Thread 3344575			0.67	10.78	6.17	0.89	-0.00	10.41
○Thread 3344576			0.64	10.78	5.94	0.89	-0.00	10.37
○Thread 3344577			0.65	10.78	6.03	0.89	-0.00	10.37
○Thread 3344578			0.65	10.78	6.03	0.89	-0.00	10.41
○Thread 3344579			0.65	10.78	6.08	0.89	-0.00	10.41
○Thread 3344580			0.64	10.78	5.99	0.89	-0.00	10.42
○Thread 3344581			0.63	10.78	5.85	0.89	-0.00	10.42
○Thread 3344582			0.65	10.78	6.08	0.89	-0.00	10.42
○Thread 3344583			0.65	10.78	6.03	0.89	-0.00	10.42
○Thread 3344584			0.66	10.78	6.13	0.89	-0.00	10.42
○Thread 3344585			0.63	10.78	5.89	0.89	-0.00	10.41
○Thread 3344586			0.64	10.78	5.99	0.89	-0.00	10.42
○Thread 3344587			0.64	10.78	5.99	0.89	-0.00	10.29
○Thread 3344588			0.63	10.77	5.85	0.89	-0.00	10.42
○Thread 3344589			0.65	10.77	6.08	0.89	-0.00	10.42
○Thread 3344590			0.63	10.77	5.89	0.89	-0.00	10.28
○Thread 3344591			0.63	10.77	5.85	0.89	-0.00	10.41
○Thread 3344592			0.63	10.77	5.85	0.89	-0.00	10.41
○Thread 3344593			0.66	10.77	6.13	0.89	-0.00	10.42
○Thread 3344594			0.64	10.77	5.99	0.89	-0.00	10.33
○Thread 3344595			0.62	10.77	5.71	0.89	-0.00	10.42
○Thread 3344596			0.64	10.77	5.99	0.89	-0.00	10.42
○Thread 3344597			0.65	10.77	6.03	0.89	-0.00	10.38
○Thread 3344598			0.63	10.77	5.89	0.89	-0.00	10.41
○Thread 3344599			0.64	10.77	5.99	0.89	-0.00	10.37
○Thread 3344600			0.63	10.77	5.89	0.89	-0.00	10.37
○Thread 3344601			0.64	10.77	5.99	0.89	-0.00	10.41
○Thread 3344602			0.63	10.77	5.80	0.89	-0.00	10.41
○Thread 3344603			0.64	10.77	5.94	0.89	-0.00	10.32
○Thread 3344604			0.63	10.77	5.89	0.89	-0.00	10.42
○Thread 3344605			0.62	10.77	5.71	0.89	-0.00	10.37
○Thread 3344606			0.63	10.77	5.80	0.89	-0.00	10.42
○Thread 3344607			0.64	10.77	5.99	0.89	-0.00	10.37
○Thread 3344608			0.63	10.77	5.89	0.89	-0.00	10.37
○Thread 3344609			0.63	10.77	5.85	0.89	-0.00	10.33
○Thread 3344610			0.63	10.77	5.89	0.89	-0.00	10.42
○Thread 3344611			0.64	10.77	5.99	0.89	-0.00	10.38
○Thread 3344612			0.62	10.77	5.76	0.89	-0.00	10.41
○Thread 3344613			0.64	10.77	5.99	0.89	-0.00	10.37
○Thread 3344614			0.63	10.77	5.85	0.89	-0.00	10.38
○Thread 3344615			0.63	10.77	5.80	0.89	-0.00	10.33
○Thread 3344616			0.63	10.77	5.89	0.89	-0.00	10.25
○Thread 3344617			0.64	10.77	5.94	0.89	-0.00	10.41
○Thread 3344618			0.63	10.77	5.80	0.89	-0.00	10.41
○Thread 3344619			0.63	10.77	5.90	0.89	-0.00	10.41
○Thread 3344620			0.64	10.77	5.94	0.89	-0.00	10.37
○Thread 3344621			0.64	10.77	5.94	0.89	-0.00	10.37
○Thread 3344622			0.65	10.77	6.03	0.89	-0.00	10.42
○Thread 3344623			0.64	10.77	5.99	0.89	-0.00	10.37
○Thread 3344624			0.63	10.77	5.90	0.89	-0.00	10.37
○Thread 3344625			0.63	10.77	5.90	0.89	-0.00	10.37
○Thread 3344626			0.63	10.77	5.90	0.89	-0.00	10.32
○Thread 3344627			0.63	10.77	5.85	0.89	-0.00	10.41
○Thread 3344628			0.63	10.77	5.90	0.89	-0.00	10.42
○Thread 3344629			0.65	10.77	6.04	0.89	-0.00	10.37
○Thread 3344630			0.64	10.77	5.99	0.89	-0.00	10.42
○Thread 3344631			0.64	10.77	5.99	0.89	-0.00	10.33
○Thread 3344632			0.63	10.77	5.80	0.89	-0.00	10.33
○Thread 3344633			0.63	10.77	5.85	0.89	-0.00	10.41
○Thread 3344634			0.64	10.77	5.99	0.89	-0.00	10.41
○Thread 3344635			0.62	10.77	5.71	0.89	-0.00	10.41
○Thread 3344636			0.64	10.77	5.94	0.89	-0.00	10.41
○Thread 3344637			0.63	10.77	5.80	0.89	-0.00	10.41
○Thread 3344638			0.61	10.77	5.62	0.89	-0.00	10.42
○Thread 3344639			0.65	10.77	6.04	0.89	-0.00	10.41
○Thread 3344640			0.63	10.77	5.90	0.89	-0.00	10.41
○Thread 3344641			0.63	10.77	5.90	0.89	-0.00	10.37
○Thread 3344642			0.63	10.77	5.90	0.89	-0.00	10.37
○Thread 3344643			0.64	10.77	5.99	0.89	-0.00	10.42
○Thread 3344644			0.61	10.77	5.66	0.89	-0.00	10.42
○Thread 3344645			0.64	10.77	5.94	0.89	-0.00	10.33
○Thread 3344646			0.64	10.77	5.99	0.89	-0.00	10.42
○Thread 3344647			0.62	10.77	5.76	0.89	-0.00	10.33
○Thread 3344648			0.63	10.77	5.80	0.89	-0.00	10.41
○Thread 3344649			0.63	10.77	5.80	0.89	-0.00	10.36
○Thread 3344650			0.64	10.77	5.94	0.89	-0.00	10.42
○Thread 3344651			0.63	10.77	5.90	0.89	-0.00	10.41
○Thread 3344652			0.61	10.77	5.62	0.89	-0.00	10.42

Report Configuration