OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	75.05
►Process 1128650+		1	75.05
○Thread 1128650			75.05	76.82	97.70	0.00	0.00	37.34

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	38.01
►Process 1128787+		2	38.01
○Thread 1128787			38.01	39.74	95.63	0.00	0.00	37.50
○Thread 1128854			37.20	37.51	99.18	2.23	0.00	37.25

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	19.61
►Process 1128876+		4	19.61
○Thread 1128876			19.61	21.24	92.31	0.00	0.00	19.01
○Thread 1128943			18.70	19.02	98.34	2.22	0.00	18.77
○Thread 1128944			18.68	19.02	98.23	2.22	0.00	18.77
○Thread 1128945			18.72	19.02	98.42	2.22	0.00	18.76

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	10.29
►Process 1129015+		8	10.29
○Thread 1129015			10.29	11.98	85.90	0.00	0.00	9.77
○Thread 1129082			9.45	9.77	96.69	2.20	0.00	9.52
○Thread 1129083			9.48	9.77	96.94	2.20	0.00	9.52
○Thread 1129084			9.46	9.77	96.73	2.20	0.00	9.52
○Thread 1129085			9.48	9.77	96.94	2.20	0.00	9.52
○Thread 1129086			9.47	9.77	96.83	2.20	0.00	9.52
○Thread 1129087			9.49	9.77	97.04	2.20	0.00	9.52
○Thread 1129088			9.48	9.77	96.93	2.20	0.00	9.52

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	5.64
►Process 1129108+		16	5.64
○Thread 1129108			5.64	7.39	76.24	0.00	0.00	5.16
○Thread 1129175			4.85	5.17	93.84	2.22	0.00	4.91
○Thread 1129176			4.87	5.17	94.23	2.22	0.00	4.91
○Thread 1129177			4.84	5.17	93.64	2.22	0.00	4.91
○Thread 1129178			4.85	5.17	93.74	2.22	0.00	4.91
○Thread 1129179			4.88	5.17	94.32	2.22	0.00	4.91
○Thread 1129180			4.85	5.17	93.83	2.22	0.00	4.91
○Thread 1129181			4.86	5.17	94.02	2.22	0.00	4.91
○Thread 1129182			4.84	5.17	93.63	2.22	0.00	4.90
○Thread 1129183			4.82	5.17	93.24	2.22	0.00	4.91
○Thread 1129184			4.88	5.17	94.30	2.22	0.00	4.91
○Thread 1129185			4.84	5.17	93.62	2.22	0.00	4.91
○Thread 1129186			4.85	5.17	93.72	2.22	0.00	4.91
○Thread 1129187			4.88	5.17	94.39	2.22	0.00	4.91
○Thread 1129188			4.85	5.17	93.81	2.22	0.00	4.91
○Thread 1129189			4.86	5.17	94.00	2.22	0.00	4.91

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	4.50
►Process 1129210+		24	4.50
○Thread 1129210			4.50	6.23	72.25	0.00	0.00	3.99
○Thread 1129277			3.67	4.00	91.93	2.22	0.01	3.74
○Thread 1129278			3.67	4.00	91.92	2.22	0.01	3.73
○Thread 1129279			3.69	4.00	92.42	2.22	0.01	3.74
○Thread 1129280			3.69	4.00	92.29	2.22	0.01	3.74
○Thread 1129281			3.68	4.00	92.16	2.22	0.01	3.74
○Thread 1129282			3.66	4.00	91.66	2.22	0.01	3.74
○Thread 1129283			3.65	4.00	91.41	2.22	0.01	3.74
○Thread 1129284			3.70	4.00	92.53	2.22	0.00	3.74
○Thread 1129285			3.68	4.00	92.02	2.22	0.00	3.74
○Thread 1129286			3.68	4.00	92.15	2.22	0.00	3.74
○Thread 1129287			3.69	4.00	92.39	2.22	0.00	3.74
○Thread 1129288			3.72	4.00	93.14	2.23	0.00	3.74
○Thread 1129289			3.69	4.00	92.38	2.23	0.00	3.74
○Thread 1129290			3.68	4.00	92.00	2.23	0.00	3.74
○Thread 1129291			3.68	4.00	92.00	2.23	0.00	3.74
○Thread 1129292			3.69	4.00	92.25	2.23	0.00	3.74
○Thread 1129293			3.69	4.00	92.24	2.23	0.00	3.74
○Thread 1129294			3.68	4.00	91.99	2.23	0.00	3.74
○Thread 1129295			3.69	4.00	92.36	2.23	0.00	3.74
○Thread 1129296			3.68	4.00	92.11	2.23	0.00	3.74
○Thread 1129297			3.70	4.00	92.48	2.23	0.00	3.74
○Thread 1129298			3.68	4.00	92.10	2.23	0.00	3.74
○Thread 1129299			3.57	4.00	89.35	2.23	0.00	3.74

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	3.77
►Process 1129319+		32	3.77
○Thread 1129319			3.77	5.53	68.12	0.00	0.00	3.29
○Thread 1129386			2.96	3.30	89.98	2.23	0.01	3.03
○Thread 1129387			2.92	3.30	88.61	2.23	0.01	3.02
○Thread 1129388			2.97	3.30	90.28	2.23	0.01	3.03
○Thread 1129389			2.97	3.30	90.27	2.23	0.01	3.03
○Thread 1129390			2.96	3.30	89.96	2.23	0.01	3.03
○Thread 1129391			2.96	3.30	89.94	2.23	0.01	3.03
○Thread 1129392			2.96	3.30	89.79	2.23	0.01	3.03
○Thread 1129393			2.99	3.30	90.85	2.23	0.01	3.03
○Thread 1129394			2.94	3.30	89.33	2.23	0.01	3.03
○Thread 1129395			2.94	3.30	89.17	2.23	0.01	3.03
○Thread 1129396			2.94	3.30	89.32	2.23	0.01	3.03
○Thread 1129397			2.98	3.30	90.38	2.23	0.01	3.03
○Thread 1129398			2.96	3.30	89.77	2.23	0.01	3.03
○Thread 1129399			2.94	3.30	89.31	2.23	0.01	3.03
○Thread 1129400			2.98	3.30	90.51	2.23	0.01	3.03
○Thread 1129401			2.98	3.30	90.36	2.23	0.00	3.03
○Thread 1129402			2.97	3.30	90.05	2.23	0.00	3.03
○Thread 1129403			2.98	3.30	90.50	2.23	0.00	3.03
○Thread 1129404			2.98	3.30	90.35	2.23	0.00	3.03
○Thread 1129405			2.97	3.30	90.19	2.23	0.00	3.03
○Thread 1129406			2.97	3.30	90.03	2.23	0.00	3.03
○Thread 1129407			2.98	3.30	90.33	2.23	0.00	3.03
○Thread 1129408			2.95	3.30	89.42	2.23	0.00	3.03
○Thread 1129409			2.93	3.30	88.96	2.23	0.00	3.03
○Thread 1129410			2.97	3.30	90.02	2.23	0.00	3.03
○Thread 1129411			2.95	3.30	89.56	2.23	0.00	3.03
○Thread 1129412			2.98	3.30	90.46	2.23	0.00	3.03
○Thread 1129413			2.97	3.30	90.01	2.23	0.00	3.03
○Thread 1129414			2.95	3.30	89.55	2.23	0.00	3.03
○Thread 1129415			2.97	3.30	90.15	2.23	0.00	3.03
○Thread 1129416			2.96	3.30	89.69	2.23	0.00	3.03

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	3.37
►Process 1129485+		40	3.37
○Thread 1129485			3.37	5.09	66.32	0.00	0.00	2.86
○Thread 1129552			2.52	2.86	88.18	2.21	0.01	2.59
○Thread 1129553			2.53	2.86	88.35	2.21	0.01	2.60
○Thread 1129554			2.51	2.86	87.65	2.21	0.01	2.60
○Thread 1129555			2.51	2.86	87.82	2.21	0.01	2.60
○Thread 1129556			2.51	2.86	87.81	2.22	0.01	2.60
○Thread 1129557			2.53	2.86	88.51	2.22	0.01	2.60
○Thread 1129558			2.52	2.86	88.15	2.22	0.01	2.59
○Thread 1129559			2.52	2.86	88.15	2.22	0.01	2.60
○Thread 1129560			2.52	2.86	88.14	2.22	0.01	2.60
○Thread 1129561			2.52	2.86	88.14	2.22	0.01	2.59
○Thread 1129562			2.54	2.87	88.83	2.22	0.01	2.60
○Thread 1129563			2.53	2.87	88.48	2.22	0.01	2.60
○Thread 1129564			2.52	2.87	87.95	2.22	0.01	2.60
○Thread 1129565			2.52	2.87	87.94	2.22	0.01	2.60
○Thread 1129566			2.54	2.87	88.63	2.22	0.01	2.60
○Thread 1129567			2.54	2.87	88.63	2.22	0.01	2.59
○Thread 1129568			2.53	2.87	88.45	2.22	0.01	2.60
○Thread 1129569			2.49	2.87	87.06	2.22	0.01	2.60
○Thread 1129570			2.54	2.87	88.62	2.22	0.01	2.60
○Thread 1129571			2.52	2.87	88.09	2.22	0.01	2.60
○Thread 1129572			2.53	2.87	88.44	2.22	0.01	2.60
○Thread 1129573			2.51	2.87	87.74	2.22	0.01	2.60
○Thread 1129574			2.52	2.87	88.08	2.22	0.01	2.60
○Thread 1129575			2.53	2.87	88.42	2.22	0.00	2.60
○Thread 1129576			2.53	2.87	88.24	2.22	0.00	2.60
○Thread 1129577			2.49	2.87	87.02	2.22	0.00	2.59
○Thread 1129578			2.52	2.87	88.06	2.22	0.00	2.60
○Thread 1129579			2.53	2.87	88.23	2.22	0.00	2.59
○Thread 1129580			2.50	2.87	87.35	2.22	0.00	2.60
○Thread 1129581			2.52	2.87	87.87	2.22	0.00	2.60
○Thread 1129582			2.51	2.87	87.69	2.22	0.00	2.60
○Thread 1129583			2.52	2.87	88.03	2.22	0.00	2.60
○Thread 1129584			2.51	2.87	87.51	2.22	0.00	2.60
○Thread 1129585			2.54	2.87	88.55	2.22	0.00	2.60
○Thread 1129586			2.54	2.87	88.54	2.22	0.00	2.60
○Thread 1129587			2.50	2.87	87.14	2.22	0.00	2.60
○Thread 1129588			2.50	2.87	87.31	2.22	0.00	2.60
○Thread 1129589			2.49	2.87	86.96	2.22	0.00	2.60
○Thread 1129590			2.32	2.87	80.86	2.22	0.00	2.58

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	3.15
►Process 1129610+		48	3.15
○Thread 1129610			3.15	4.79	65.73	0.00	0.00	2.54
○Thread 1129677			2.20	2.55	86.34	2.23	0.01	2.28
○Thread 1129678			2.19	2.55	85.75	2.23	0.01	2.29
○Thread 1129679			2.20	2.55	86.33	2.23	0.01	2.29
○Thread 1129680			2.22	2.55	87.11	2.23	0.01	2.29
○Thread 1129681			2.22	2.55	87.10	2.23	0.01	2.28
○Thread 1129682			2.21	2.55	86.51	2.23	0.01	2.29
○Thread 1129683			2.23	2.55	87.49	2.23	0.01	2.29
○Thread 1129684			2.22	2.55	86.89	2.23	0.01	2.29
○Thread 1129685			2.19	2.55	85.72	2.23	0.01	2.28
○Thread 1129686			2.21	2.56	86.49	2.23	0.01	2.28
○Thread 1129687			2.21	2.56	86.49	2.23	0.01	2.29
○Thread 1129688			2.19	2.56	85.70	2.23	0.01	2.29
○Thread 1129689			2.19	2.56	85.69	2.23	0.01	2.29
○Thread 1129690			2.23	2.56	87.26	2.23	0.01	2.28
○Thread 1129691			2.21	2.56	86.66	2.23	0.01	2.29
○Thread 1129692			2.24	2.56	87.83	2.23	0.01	2.29
○Thread 1129693			2.22	2.56	86.85	2.23	0.01	2.29
○Thread 1129694			2.23	2.56	87.24	2.23	0.01	2.29
○Thread 1129695			2.21	2.56	86.64	2.23	0.01	2.28
○Thread 1129696			2.22	2.56	86.83	2.23	0.01	2.29
○Thread 1129697			2.19	2.56	85.66	2.23	0.01	2.28
○Thread 1129698			2.23	2.56	87.22	2.23	0.01	2.28
○Thread 1129699			2.17	2.56	84.86	2.23	0.01	2.29
○Thread 1129700			2.19	2.56	85.64	2.23	0.01	2.29
○Thread 1129701			2.19	2.56	85.83	2.23	0.01	2.28
○Thread 1129702			2.22	2.56	87.00	2.23	0.01	2.29
○Thread 1129703			2.22	2.56	86.80	2.23	0.01	2.29
○Thread 1129704			2.23	2.56	87.19	2.23	0.01	2.29
○Thread 1129705			2.22	2.56	86.98	2.23	0.01	2.29
○Thread 1129706			2.18	2.56	85.22	2.23	0.01	2.28
○Thread 1129707			2.22	2.56	86.78	2.23	0.01	2.28
○Thread 1129708			2.21	2.56	86.58	2.23	0.00	2.28
○Thread 1129709			2.21	2.56	86.38	2.23	0.00	2.28
○Thread 1129710			2.23	2.56	87.35	2.23	0.00	2.29
○Thread 1129711			2.24	2.56	87.54	2.23	0.00	2.29
○Thread 1129712			2.21	2.56	86.36	2.23	0.00	2.29
○Thread 1129713			2.22	2.56	86.94	2.23	0.00	2.28
○Thread 1129714			2.20	2.56	86.15	2.23	0.00	2.28
○Thread 1129715			2.21	2.56	86.54	2.23	0.00	2.28
○Thread 1129716			2.22	2.56	86.73	2.23	0.00	2.28
○Thread 1129717			2.23	2.56	87.12	2.23	0.00	2.29
○Thread 1129718			2.20	2.56	85.94	2.23	0.00	2.29
○Thread 1129719			2.20	2.56	85.93	2.23	0.00	2.29
○Thread 1129720			2.18	2.56	85.34	2.23	0.00	2.29
○Thread 1129721			2.19	2.56	85.53	2.23	0.00	2.28
○Thread 1129722			2.07	2.56	80.84	2.23	0.00	2.29
○Thread 1129723			1.99	2.56	77.91	2.23	0.00	2.29

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	2.75
►Process 1129743+		56	2.75
○Thread 1129743			2.75	4.57	60.34	0.00	0.00	2.32
○Thread 1129810			1.99	2.31	86.13	2.24	0.02	2.05
○Thread 1129811			2.00	2.31	86.55	2.24	0.01	2.04
○Thread 1129812			1.99	2.31	86.12	2.24	0.01	2.03
○Thread 1129813			1.96	2.31	85.03	2.24	0.01	2.04
○Thread 1129814			1.98	2.31	85.67	2.24	0.01	2.04
○Thread 1129815			2.00	2.31	86.53	2.24	0.01	2.05
○Thread 1129816			1.98	2.31	85.88	2.24	0.01	2.05
○Thread 1129817			1.99	2.31	86.09	2.24	0.01	2.05
○Thread 1129818			1.98	2.31	85.87	2.24	0.01	2.04
○Thread 1129819			1.98	2.31	85.65	2.24	0.01	2.05
○Thread 1129820			1.97	2.31	85.42	2.24	0.01	2.05
○Thread 1129821			1.98	2.31	85.85	2.24	0.01	2.03
○Thread 1129822			1.98	2.31	85.85	2.24	0.01	2.05
○Thread 1129823			1.96	2.31	84.76	2.24	0.01	2.04
○Thread 1129824			2.01	2.31	86.92	2.24	0.01	2.05
○Thread 1129825			1.97	2.31	85.18	2.24	0.01	2.05
○Thread 1129826			1.94	2.31	84.10	2.24	0.01	2.04
○Thread 1129827			1.98	2.31	85.82	2.24	0.01	2.04
○Thread 1129828			1.98	2.31	85.60	2.24	0.01	2.05
○Thread 1129829			1.96	2.31	84.95	2.24	0.01	2.05
○Thread 1129830			1.96	2.31	84.94	2.24	0.01	2.04
○Thread 1129831			1.96	2.31	84.72	2.24	0.01	2.05
○Thread 1129832			1.95	2.31	84.50	2.24	0.01	2.05
○Thread 1129833			1.92	2.31	83.19	2.24	0.01	2.04
○Thread 1129834			1.95	2.31	84.27	2.24	0.01	2.05
○Thread 1129835			1.97	2.31	85.13	2.24	0.01	2.05
○Thread 1129836			1.99	2.31	85.99	2.24	0.01	2.05
○Thread 1129837			1.96	2.31	84.90	2.24	0.01	2.05
○Thread 1129838			2.00	2.31	86.41	2.24	0.01	2.05
○Thread 1129839			1.98	2.31	85.54	2.24	0.01	2.05
○Thread 1129840			1.97	2.31	85.32	2.24	0.01	2.05
○Thread 1129841			2.01	2.32	86.82	2.24	0.01	2.04
○Thread 1129842			1.99	2.32	85.95	2.24	0.01	2.05
○Thread 1129843			1.99	2.32	85.95	2.24	0.01	2.05
○Thread 1129844			1.96	2.32	84.65	2.24	0.01	2.04
○Thread 1129845			1.95	2.32	84.21	2.24	0.01	2.03
○Thread 1129846			1.95	2.32	84.20	2.24	0.01	2.04
○Thread 1129847			1.99	2.32	86.14	2.24	0.01	2.04
○Thread 1129848			1.97	2.32	85.06	2.24	0.00	2.05
○Thread 1129849			1.94	2.32	83.97	2.24	0.00	2.05
○Thread 1129850			1.98	2.32	85.48	2.25	0.00	2.05
○Thread 1129851			1.97	2.32	85.26	2.25	0.00	2.05
○Thread 1129852			1.96	2.32	84.82	2.25	0.00	2.04
○Thread 1129853			1.94	2.32	83.95	2.25	0.00	2.05
○Thread 1129854			1.94	2.32	83.95	2.25	0.00	2.04
○Thread 1129855			1.94	2.32	83.94	2.25	0.00	2.04
○Thread 1129856			1.98	2.32	85.66	2.25	0.00	2.05
○Thread 1129857			1.95	2.32	84.15	2.25	0.00	2.04
○Thread 1129858			1.97	2.32	85.22	2.25	0.00	2.05
○Thread 1129859			1.98	2.32	85.65	2.25	0.00	2.05
○Thread 1129860			1.97	2.32	84.99	2.25	0.00	2.04
○Thread 1129861			1.94	2.32	83.70	2.25	0.00	2.05
○Thread 1129862			1.95	2.32	84.12	2.25	0.00	2.05
○Thread 1129863			1.88	2.32	81.10	2.25	0.00	2.05
○Thread 1129864			1.85	2.32	79.80	2.25	0.00	2.05

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	2.57
►Process 1129884+		64	2.57
○Thread 1129884			2.57	4.36	58.97	0.00	0.00	2.13
○Thread 1129951			1.78	2.12	84.24	2.22	0.02	1.85
○Thread 1129952			1.76	2.12	83.06	2.22	0.02	1.85
○Thread 1129953			1.75	2.12	82.58	2.22	0.02	1.85
○Thread 1129954			1.77	2.12	83.75	2.22	0.02	1.84
○Thread 1129955			1.71	2.12	80.92	2.22	0.02	1.85
○Thread 1129956			1.74	2.12	82.09	2.22	0.02	1.85
○Thread 1129957			1.74	2.12	82.08	2.22	0.02	1.85
○Thread 1129958			1.75	2.12	82.78	2.22	0.02	1.85
○Thread 1129959			1.77	2.12	83.72	2.22	0.02	1.85
○Thread 1129960			1.77	2.12	83.72	2.22	0.01	1.85
○Thread 1129961			1.78	2.12	83.95	2.22	0.01	1.84
○Thread 1129962			1.77	2.12	83.70	2.22	0.01	1.85
○Thread 1129963			1.72	2.12	81.11	2.22	0.01	1.85
○Thread 1129964			1.74	2.12	82.28	2.22	0.01	1.84
○Thread 1129965			1.73	2.12	81.80	2.22	0.01	1.85
○Thread 1129966			1.75	2.12	82.50	2.22	0.01	1.85
○Thread 1129967			1.74	2.12	82.26	2.22	0.01	1.85
○Thread 1129968			1.75	2.12	82.49	2.22	0.01	1.85
○Thread 1129969			1.72	2.12	81.31	2.22	0.01	1.85
○Thread 1129970			1.76	2.12	82.95	2.22	0.01	1.85
○Thread 1129971			1.74	2.12	82.24	2.22	0.01	1.85
○Thread 1129972			1.75	2.12	82.70	2.22	0.01	1.85
○Thread 1129973			1.77	2.12	83.40	2.22	0.01	1.85
○Thread 1129974			1.75	2.12	82.46	2.22	0.01	1.86
○Thread 1129975			1.77	2.12	83.39	2.22	0.01	1.85
○Thread 1129976			1.75	2.12	82.68	2.22	0.01	1.85
○Thread 1129977			1.74	2.12	81.97	2.22	0.01	1.84
○Thread 1129978			1.77	2.12	83.37	2.23	0.01	1.85
○Thread 1129979			1.74	2.12	82.19	2.23	0.01	1.85
○Thread 1129980			1.72	2.12	81.24	2.23	0.01	1.85
○Thread 1129981			1.73	2.12	81.71	2.23	0.01	1.84
○Thread 1129982			1.75	2.12	82.41	2.23	0.01	1.85
○Thread 1129983			1.74	2.12	82.16	2.23	0.01	1.84
○Thread 1129984			1.76	2.12	83.10	2.23	0.01	1.85
○Thread 1129985			1.73	2.12	81.45	2.23	0.01	1.85
○Thread 1129986			1.79	2.12	84.50	2.23	0.01	1.86
○Thread 1129987			1.73	2.12	81.67	2.23	0.01	1.85
○Thread 1129988			1.75	2.12	82.60	2.23	0.01	1.84
○Thread 1129989			1.75	2.12	82.36	2.23	0.01	1.85
○Thread 1129990			1.75	2.12	82.59	2.23	0.01	1.85
○Thread 1129991			1.75	2.13	82.35	2.23	0.01	1.84
○Thread 1129992			1.73	2.13	81.64	2.23	0.01	1.85
○Thread 1129993			1.76	2.13	83.05	2.23	0.01	1.85
○Thread 1129994			1.75	2.13	82.57	2.23	0.01	1.84
○Thread 1129995			1.76	2.13	82.80	2.23	0.01	1.84
○Thread 1129996			1.73	2.13	81.62	2.23	0.01	1.85
○Thread 1129997			1.72	2.13	80.91	2.23	0.00	1.84
○Thread 1129998			1.78	2.13	83.96	2.23	0.00	1.85
○Thread 1129999			1.76	2.13	82.78	2.23	0.00	1.85
○Thread 1130000			1.77	2.13	83.48	2.23	0.00	1.84
○Thread 1130001			1.75	2.13	82.30	2.23	0.00	1.84
○Thread 1130002			1.72	2.13	81.12	2.23	0.00	1.85
○Thread 1130003			1.74	2.13	82.05	2.23	0.00	1.85
○Thread 1130004			1.72	2.13	81.10	2.23	0.00	1.85
○Thread 1130005			1.79	2.13	84.15	2.23	0.00	1.85
○Thread 1130006			1.75	2.13	82.50	2.23	0.00	1.85
○Thread 1130007			1.73	2.13	81.32	2.23	0.00	1.85
○Thread 1130008			1.74	2.13	82.02	2.23	0.00	1.85
○Thread 1130009			1.75	2.13	82.25	2.23	0.00	1.85
○Thread 1130010			1.77	2.13	83.42	2.23	0.00	1.85
○Thread 1130011			1.75	2.13	82.47	2.23	0.00	1.85
○Thread 1130012			1.76	2.13	82.70	2.23	0.00	1.84
○Thread 1130013			1.72	2.13	81.05	2.23	0.00	1.85

Report Configuration