OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	38.61
►Process 1161772+		1	38.61
○Thread 1161772			38.61	40.39	95.59	0.00	0.00	16.55

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	19.72
►Process 1161909+		2	19.72
○Thread 1161909			19.72	21.61	91.24	0.00	0.00	19.34
○Thread 1161976			18.99	19.34	98.21	2.27	0.00	19.05

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	10.38
►Process 1161996+		4	10.38
○Thread 1161996			10.38	12.22	84.91	0.00	0.00	9.93
○Thread 1162063			9.61	9.94	96.71	2.28	0.00	9.64
○Thread 1162064			9.61	9.94	96.71	2.28	0.00	9.64
○Thread 1162065			9.60	9.94	96.55	2.28	0.00	9.64

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	5.71
►Process 1162133+		8	5.71
○Thread 1162133			5.71	7.50	76.06	0.00	0.00	5.22
○Thread 1162200			4.91	5.23	93.77	2.27	0.00	4.94
○Thread 1162201			4.90	5.23	93.58	2.27	0.00	4.94
○Thread 1162202			4.91	5.23	93.76	2.27	0.00	4.94
○Thread 1162203			4.89	5.23	93.38	2.27	0.00	4.93
○Thread 1162204			4.92	5.23	93.95	2.27	0.00	4.94
○Thread 1162205			4.89	5.23	93.47	2.27	0.00	4.94
○Thread 1162206			4.90	5.23	93.56	2.27	0.00	4.94

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	3.28
►Process 1162227+		16	3.28
○Thread 1162227			3.28	5.17	63.59	0.00	0.00	2.88
○Thread 1162294			2.57	2.89	88.90	2.27	0.00	2.60
○Thread 1162295			2.55	2.89	88.38	2.27	0.00	2.60
○Thread 1162296			2.57	2.89	89.07	2.27	0.00	2.60
○Thread 1162297			2.55	2.89	88.37	2.27	0.00	2.60
○Thread 1162298			2.54	2.89	88.02	2.27	0.00	2.60
○Thread 1162299			2.57	2.89	88.88	2.27	0.00	2.60
○Thread 1162300			2.56	2.89	88.53	2.27	0.00	2.60
○Thread 1162301			2.56	2.89	88.70	2.27	0.00	2.60
○Thread 1162302			2.55	2.89	88.18	2.27	0.00	2.59
○Thread 1162303			2.57	2.89	88.86	2.27	0.00	2.60
○Thread 1162304			2.54	2.89	87.99	2.27	0.00	2.60
○Thread 1162305			2.56	2.89	88.68	2.27	0.00	2.60
○Thread 1162306			2.57	2.89	88.85	2.27	0.00	2.60
○Thread 1162307			2.54	2.89	87.81	2.27	0.00	2.60
○Thread 1162308			2.56	2.89	88.49	2.27	0.00	2.60

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	2.73
►Process 1162328+		24	2.73
○Thread 1162328			2.73	4.59	59.49	0.00	0.00	2.30
○Thread 1162396			1.97	2.31	85.44	2.28	0.01	2.01
○Thread 1162397			1.95	2.31	84.78	2.28	0.01	2.00
○Thread 1162398			1.94	2.31	84.34	2.28	0.01	2.00
○Thread 1162399			1.96	2.31	84.99	2.28	0.01	2.00
○Thread 1162400			1.96	2.31	85.20	2.28	0.01	2.00
○Thread 1162401			1.97	2.31	85.41	2.28	0.01	2.00
○Thread 1162402			1.93	2.31	83.89	2.28	0.01	2.00
○Thread 1162403			1.97	2.31	85.62	2.28	0.01	2.00
○Thread 1162404			1.93	2.31	83.88	2.28	0.01	2.00
○Thread 1162405			1.94	2.31	84.31	2.28	0.01	2.00
○Thread 1162406			1.98	2.31	85.82	2.28	0.00	2.00
○Thread 1162407			1.93	2.31	83.86	2.28	0.00	2.00
○Thread 1162408			1.96	2.31	84.94	2.28	0.00	2.00
○Thread 1162409			1.96	2.31	85.15	2.28	0.00	2.00
○Thread 1162410			1.96	2.31	84.92	2.28	0.00	2.00
○Thread 1162411			1.96	2.31	84.92	2.28	0.00	2.00
○Thread 1162412			1.97	2.31	85.56	2.28	0.00	2.00
○Thread 1162413			1.94	2.31	84.26	2.28	0.00	2.00
○Thread 1162414			1.94	2.31	84.22	2.28	0.00	2.00
○Thread 1162415			1.95	2.31	84.64	2.28	0.00	2.00
○Thread 1162416			1.98	2.31	85.72	2.28	0.00	2.00
○Thread 1162417			1.96	2.31	85.06	2.28	0.00	2.00
○Thread 1162418			1.94	2.31	83.98	2.28	0.00	2.00

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	2.33
►Process 1162438+		32	2.33
○Thread 1162438			2.33	4.23	55.05	0.00	0.00	2.25
○Thread 1162505			1.60	1.94	82.68	2.28	0.01	1.64
○Thread 1162506			1.61	1.94	82.93	2.28	0.01	1.65
○Thread 1162507			1.60	1.94	82.67	2.28	0.01	1.65
○Thread 1162508			1.60	1.94	82.40	2.28	0.01	1.65
○Thread 1162509			1.60	1.94	82.65	2.28	0.01	1.65
○Thread 1162510			1.59	1.94	82.13	2.28	0.01	1.65
○Thread 1162511			1.59	1.94	82.13	2.28	0.01	1.65
○Thread 1162512			1.60	1.94	82.63	2.28	0.01	1.65
○Thread 1162513			1.59	1.94	82.11	2.28	0.01	1.65
○Thread 1162514			1.60	1.94	82.36	2.28	0.01	1.65
○Thread 1162515			1.60	1.94	82.36	2.28	0.01	1.65
○Thread 1162516			1.58	1.94	81.58	2.28	0.01	1.65
○Thread 1162517			1.61	1.94	82.86	2.28	0.01	1.65
○Thread 1162518			1.58	1.94	81.57	2.28	0.01	1.65
○Thread 1162519			1.56	1.94	80.53	2.28	0.01	1.65
○Thread 1162520			1.57	1.94	80.78	2.28	0.00	1.65
○Thread 1162521			1.60	1.94	82.32	2.28	0.00	1.64
○Thread 1162522			1.62	1.94	83.60	2.28	0.00	1.64
○Thread 1162523			1.59	1.94	81.79	2.28	0.00	1.65
○Thread 1162524			1.60	1.94	82.55	2.28	0.00	1.65
○Thread 1162525			1.61	1.94	82.80	2.28	0.00	1.65
○Thread 1162526			1.59	1.94	82.03	2.28	0.00	1.65
○Thread 1162527			1.60	1.94	82.54	2.29	0.00	1.64
○Thread 1162528			1.58	1.94	81.24	2.29	0.00	1.65
○Thread 1162529			1.60	1.94	82.53	2.29	0.00	1.65
○Thread 1162530			1.60	1.95	82.52	2.29	0.00	1.65
○Thread 1162531			1.57	1.95	80.97	2.29	0.00	1.64
○Thread 1162532			1.57	1.95	80.96	2.29	0.00	1.63
○Thread 1162533			1.59	1.95	81.99	2.29	0.00	1.65
○Thread 1162534			1.58	1.95	81.47	2.29	0.00	1.64
○Thread 1162535			1.59	1.95	81.72	2.29	0.00	1.64

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	2.12
►Process 1162605+		40	2.12
○Thread 1162605			2.12	3.99	53.10	0.00	0.00	2.23
○Thread 1162673			1.39	1.72	80.80	2.26	0.01	1.42
○Thread 1162674			1.38	1.72	80.50	2.26	0.01	1.42
○Thread 1162675			1.36	1.72	79.33	2.26	0.01	1.42
○Thread 1162676			1.36	1.72	79.03	2.26	0.01	1.42
○Thread 1162677			1.35	1.72	78.73	2.26	0.01	1.42
○Thread 1162678			1.37	1.72	79.89	2.26	0.01	1.41
○Thread 1162679			1.36	1.72	79.01	2.26	0.01	1.42
○Thread 1162680			1.36	1.72	79.29	2.26	0.01	1.42
○Thread 1162681			1.37	1.72	79.58	2.26	0.01	1.42
○Thread 1162682			1.39	1.72	80.73	2.26	0.01	1.42
○Thread 1162683			1.37	1.72	79.85	2.26	0.01	1.42
○Thread 1162684			1.34	1.72	78.10	2.26	0.01	1.42
○Thread 1162685			1.32	1.72	76.93	2.26	0.01	1.42
○Thread 1162686			1.37	1.72	79.54	2.26	0.01	1.42
○Thread 1162687			1.36	1.72	79.24	2.26	0.01	1.42
○Thread 1162688			1.35	1.72	78.36	2.26	0.01	1.42
○Thread 1162689			1.34	1.72	77.77	2.26	0.01	1.42
○Thread 1162690			1.37	1.72	79.80	2.26	0.01	1.42
○Thread 1162691			1.37	1.72	79.50	2.26	0.01	1.42
○Thread 1162692			1.33	1.72	77.46	2.26	0.01	1.42
○Thread 1162693			1.35	1.72	78.32	2.26	0.01	1.42
○Thread 1162694			1.37	1.72	79.48	2.26	0.01	1.42
○Thread 1162695			1.36	1.72	78.89	2.26	0.01	1.42
○Thread 1162696			1.38	1.72	80.04	2.26	0.00	1.42
○Thread 1162697			1.34	1.72	78.00	2.26	0.00	1.42
○Thread 1162698			1.36	1.72	78.86	2.26	0.00	1.42
○Thread 1162699			1.36	1.72	79.15	2.26	0.00	1.42
○Thread 1162700			1.38	1.72	80.01	2.26	0.00	1.42
○Thread 1162701			1.34	1.72	77.68	2.26	0.00	1.42
○Thread 1162702			1.38	1.73	80.00	2.26	0.00	1.42
○Thread 1162703			1.37	1.73	79.41	2.26	0.00	1.42
○Thread 1162704			1.39	1.73	80.56	2.26	0.00	1.41
○Thread 1162705			1.35	1.73	78.23	2.26	0.00	1.42
○Thread 1162706			1.36	1.73	78.80	2.26	0.00	1.42
○Thread 1162707			1.35	1.73	78.22	2.26	0.00	1.42
○Thread 1162708			1.32	1.73	76.76	2.26	0.00	1.42
○Thread 1162709			1.35	1.73	78.20	2.26	0.00	1.42
○Thread 1162710			1.36	1.73	79.06	2.26	0.00	1.42
○Thread 1162711			1.27	1.73	73.55	2.26	0.00	1.42

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	1.94
►Process 1162731+		48	1.94
○Thread 1162731			1.94	3.85	50.36	0.00	0.00	2.25
○Thread 1162798			1.21	1.57	77.37	2.27	0.01	1.27
○Thread 1162799			1.23	1.57	78.64	2.27	0.01	1.27
○Thread 1162800			1.22	1.57	78.00	2.27	0.01	1.27
○Thread 1162801			1.21	1.57	77.03	2.27	0.01	1.26
○Thread 1162802			1.21	1.57	77.03	2.27	0.01	1.27
○Thread 1162803			1.18	1.57	75.43	2.27	0.01	1.25
○Thread 1162804			1.20	1.57	76.69	2.27	0.01	1.27
○Thread 1162805			1.21	1.57	77.32	2.27	0.01	1.27
○Thread 1162806			1.21	1.57	77.00	2.27	0.01	1.27
○Thread 1162807			1.21	1.57	77.31	2.27	0.01	1.27
○Thread 1162808			1.22	1.57	77.62	2.27	0.01	1.27
○Thread 1162809			1.21	1.57	76.98	2.27	0.01	1.27
○Thread 1162810			1.19	1.57	75.70	2.27	0.01	1.27
○Thread 1162811			1.22	1.57	77.91	2.27	0.01	1.27
○Thread 1162812			1.23	1.57	78.54	2.27	0.01	1.26
○Thread 1162813			1.19	1.57	75.67	2.27	0.01	1.27
○Thread 1162814			1.21	1.57	77.26	2.27	0.01	1.27
○Thread 1162815			1.20	1.57	76.61	2.27	0.01	1.27
○Thread 1162816			1.21	1.57	76.92	2.27	0.01	1.27
○Thread 1162817			1.20	1.57	76.60	2.27	0.01	1.26
○Thread 1162818			1.21	1.57	77.22	2.27	0.01	1.27
○Thread 1162819			1.25	1.57	79.44	2.27	0.01	1.27
○Thread 1162820			1.23	1.57	78.48	2.27	0.01	1.27
○Thread 1162821			1.20	1.57	76.56	2.27	0.01	1.27
○Thread 1162822			1.19	1.57	75.60	2.27	0.01	1.25
○Thread 1162823			1.23	1.57	78.45	2.27	0.01	1.27
○Thread 1162824			1.23	1.57	78.45	2.27	0.01	1.27
○Thread 1162825			1.20	1.57	76.53	2.27	0.01	1.26
○Thread 1162826			1.22	1.57	77.48	2.27	0.01	1.27
○Thread 1162827			1.23	1.57	78.11	2.27	0.01	1.27
○Thread 1162828			1.18	1.57	74.61	2.27	0.01	1.26
○Thread 1162829			1.19	1.58	75.55	2.27	0.00	1.27
○Thread 1162830			1.22	1.58	77.76	2.27	0.00	1.27
○Thread 1162831			1.19	1.58	75.85	2.27	0.00	1.27
○Thread 1162832			1.22	1.58	77.43	2.27	0.00	1.26
○Thread 1162833			1.22	1.58	77.74	2.27	0.00	1.27
○Thread 1162834			1.21	1.58	76.78	2.27	0.00	1.27
○Thread 1162835			1.23	1.58	78.04	2.27	0.00	1.27
○Thread 1162836			1.23	1.58	78.35	2.27	0.00	1.27
○Thread 1162837			1.21	1.58	77.08	2.27	0.00	1.27
○Thread 1162838			1.22	1.58	77.39	2.27	0.00	1.26
○Thread 1162839			1.22	1.58	77.38	2.27	0.00	1.26
○Thread 1162840			1.20	1.58	76.42	2.27	0.00	1.26
○Thread 1162841			1.22	1.58	77.36	2.27	0.00	1.26
○Thread 1162842			1.21	1.58	76.72	2.27	0.00	1.27
○Thread 1162843			1.16	1.58	73.54	2.27	0.00	1.27
○Thread 1162844			1.12	1.58	70.68	2.27	0.00	1.25

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	1.80
►Process 1162865+		56	1.80
○Thread 1162865			1.80	3.75	48.07	0.00	0.00	2.25
○Thread 1162932			1.11	1.46	76.42	2.28	0.01	1.15
○Thread 1162933			1.10	1.46	75.04	2.28	0.01	1.15
○Thread 1162934			1.10	1.46	75.38	2.28	0.01	1.15
○Thread 1162935			1.09	1.46	75.03	2.28	0.01	1.15
○Thread 1162936			1.07	1.46	73.31	2.28	0.01	1.15
○Thread 1162937			1.10	1.46	75.36	2.28	0.01	1.15
○Thread 1162938			1.09	1.46	74.66	2.28	0.01	1.15
○Thread 1162939			1.11	1.46	76.03	2.28	0.01	1.15
○Thread 1162940			1.10	1.46	75.68	2.28	0.01	1.15
○Thread 1162941			1.09	1.46	74.64	2.28	0.01	1.15
○Thread 1162942			1.07	1.46	73.26	2.28	0.01	1.15
○Thread 1162943			1.10	1.46	75.31	2.28	0.01	1.15
○Thread 1162944			1.07	1.46	73.25	2.28	0.01	1.15
○Thread 1162945			1.08	1.46	73.92	2.28	0.01	1.14
○Thread 1162946			1.08	1.46	73.57	2.28	0.01	1.15
○Thread 1162947			1.12	1.46	76.65	2.28	0.01	1.15
○Thread 1162948			1.10	1.46	74.93	2.28	0.01	1.15
○Thread 1162949			1.07	1.46	73.21	2.28	0.01	1.14
○Thread 1162950			1.10	1.46	75.60	2.28	0.01	1.15
○Thread 1162951			1.08	1.46	73.54	2.28	0.01	1.15
○Thread 1162952			1.07	1.46	73.19	2.28	0.01	1.15
○Thread 1162953			1.09	1.46	74.55	2.28	0.01	1.15
○Thread 1162954			1.12	1.46	76.59	2.28	0.01	1.15
○Thread 1162955			1.10	1.46	74.87	2.28	0.01	1.15
○Thread 1162956			1.08	1.46	73.50	2.28	0.01	1.15
○Thread 1162957			1.07	1.46	72.81	2.28	0.01	1.15
○Thread 1162958			1.08	1.46	73.49	2.28	0.01	1.15
○Thread 1162959			1.11	1.46	76.21	2.28	0.01	1.15
○Thread 1162960			1.08	1.46	73.81	2.28	0.01	1.15
○Thread 1162961			1.08	1.46	73.47	2.28	0.01	1.15
○Thread 1162962			1.08	1.46	74.14	2.28	0.01	1.15
○Thread 1162963			1.07	1.46	73.11	2.28	0.01	1.14
○Thread 1162964			1.08	1.46	73.44	2.28	0.01	1.15
○Thread 1162965			1.10	1.46	75.14	2.28	0.01	1.15
○Thread 1162966			1.08	1.46	74.11	2.28	0.01	1.15
○Thread 1162967			1.08	1.46	74.10	2.28	0.01	1.15
○Thread 1162968			1.07	1.46	73.07	2.29	0.01	1.14
○Thread 1162969			1.08	1.46	73.41	2.29	0.01	1.15
○Thread 1162970			1.09	1.46	74.76	2.29	0.01	1.15
○Thread 1162971			1.09	1.46	74.76	2.29	0.00	1.15
○Thread 1162972			1.08	1.46	73.38	2.29	0.00	1.15
○Thread 1162973			1.08	1.47	74.06	2.29	0.00	1.15
○Thread 1162974			1.11	1.47	75.76	2.29	0.00	1.15
○Thread 1162975			1.09	1.47	74.73	2.29	0.00	1.15
○Thread 1162976			1.10	1.47	75.40	2.29	0.00	1.15
○Thread 1162977			1.10	1.47	75.05	2.29	0.00	1.15
○Thread 1162978			1.10	1.47	75.39	2.29	0.00	1.15
○Thread 1162979			1.09	1.47	74.70	2.29	0.00	1.15
○Thread 1162980			1.09	1.47	74.69	2.29	0.00	1.15
○Thread 1162981			1.09	1.47	74.34	2.29	0.00	1.15
○Thread 1162982			1.07	1.47	72.63	2.29	0.00	1.15
○Thread 1162983			1.08	1.47	73.98	2.29	0.00	1.15
○Thread 1162984			1.05	1.47	71.93	2.29	0.00	1.15
○Thread 1162985			0.99	1.47	67.49	2.29	0.00	1.14
○Thread 1162986			1.02	1.47	69.53	2.29	0.00	1.15

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	1.77
►Process 1163006+		64	1.77
○Thread 1163006			1.77	3.62	48.85	0.00	0.00	2.21
○Thread 1163073			0.98	1.36	71.93	2.24	0.02	1.05
○Thread 1163074			1.00	1.36	73.02	2.24	0.02	1.06
○Thread 1163075			1.00	1.36	73.75	2.24	0.02	1.05
○Thread 1163076			0.98	1.36	71.54	2.24	0.02	1.06
○Thread 1163077			1.02	1.36	74.83	2.24	0.02	1.06
○Thread 1163078			1.00	1.36	72.99	2.24	0.02	1.06
○Thread 1163079			0.99	1.36	72.25	2.24	0.02	1.06
○Thread 1163080			0.97	1.36	71.14	2.24	0.02	1.05
○Thread 1163081			0.99	1.36	72.60	2.24	0.02	1.06
○Thread 1163082			1.00	1.36	73.32	2.24	0.01	1.04
○Thread 1163083			0.99	1.36	72.21	2.24	0.01	1.05
○Thread 1163084			0.99	1.36	72.57	2.24	0.01	1.05
○Thread 1163085			0.98	1.36	71.83	2.24	0.01	1.06
○Thread 1163086			1.01	1.36	74.39	2.24	0.01	1.06
○Thread 1163087			1.00	1.36	73.28	2.24	0.01	1.06
○Thread 1163088			0.99	1.36	72.17	2.25	0.01	1.06
○Thread 1163089			1.01	1.36	74.00	2.25	0.01	1.06
○Thread 1163090			1.00	1.37	73.26	2.25	0.01	1.06
○Thread 1163091			0.99	1.37	72.15	2.25	0.01	1.05
○Thread 1163092			0.98	1.37	71.41	2.25	0.01	1.06
○Thread 1163093			0.98	1.37	71.40	2.25	0.01	1.06
○Thread 1163094			0.97	1.37	71.03	2.25	0.01	1.03
○Thread 1163095			0.99	1.37	72.49	2.25	0.01	1.06
○Thread 1163096			1.00	1.37	73.21	2.25	0.01	1.06
○Thread 1163097			0.96	1.37	70.28	2.25	0.01	1.05
○Thread 1163098			0.97	1.37	71.00	2.25	0.01	1.05
○Thread 1163099			0.98	1.37	71.36	2.25	0.01	1.05
○Thread 1163100			0.98	1.37	71.35	2.25	0.01	1.06
○Thread 1163101			0.96	1.37	70.25	2.25	0.01	1.06
○Thread 1163102			1.00	1.37	72.80	2.25	0.01	1.06
○Thread 1163103			0.99	1.37	72.06	2.25	0.01	1.06
○Thread 1163104			0.99	1.37	72.42	2.25	0.01	1.06
○Thread 1163105			0.99	1.37	72.41	2.25	0.01	1.06
○Thread 1163106			0.96	1.37	70.58	2.25	0.01	1.06
○Thread 1163107			0.98	1.37	71.67	2.25	0.01	1.05
○Thread 1163108			0.96	1.37	70.56	2.25	0.01	1.04
○Thread 1163109			1.00	1.37	73.48	2.25	0.01	1.06
○Thread 1163110			0.98	1.37	71.28	2.25	0.01	1.05
○Thread 1163111			0.99	1.37	72.36	2.25	0.01	1.06
○Thread 1163112			0.99	1.37	71.99	2.25	0.01	1.06
○Thread 1163113			0.97	1.37	70.89	2.25	0.01	1.06
○Thread 1163114			0.98	1.37	71.61	2.25	0.01	1.06
○Thread 1163115			0.99	1.37	72.33	2.25	0.01	1.05
○Thread 1163116			1.00	1.37	72.69	2.25	0.01	1.05
○Thread 1163117			1.00	1.37	73.05	2.25	0.01	1.06
○Thread 1163118			0.98	1.37	71.58	2.25	0.01	1.06
○Thread 1163119			0.99	1.37	72.30	2.25	0.01	1.06
○Thread 1163120			1.00	1.37	73.38	2.25	0.00	1.05
○Thread 1163121			0.99	1.37	72.28	2.25	0.00	1.05
○Thread 1163122			0.98	1.37	71.18	2.25	0.00	1.06
○Thread 1163123			0.99	1.37	72.26	2.25	0.00	1.06
○Thread 1163124			0.98	1.37	71.53	2.25	0.00	1.06
○Thread 1163125			0.99	1.37	71.88	2.25	0.00	1.06
○Thread 1163126			0.99	1.37	71.88	2.25	0.00	1.05
○Thread 1163127			1.00	1.37	72.96	2.25	0.00	1.06
○Thread 1163128			1.01	1.37	73.68	2.25	0.00	1.05
○Thread 1163129			0.99	1.37	71.85	2.25	0.00	1.05
○Thread 1163130			1.01	1.37	74.03	2.25	0.00	1.06
○Thread 1163131			1.00	1.37	72.93	2.25	0.00	1.06
○Thread 1163132			0.98	1.37	71.46	2.25	0.00	1.05
○Thread 1163133			0.99	1.37	72.18	2.25	0.00	1.06
○Thread 1163134			0.95	1.37	69.63	2.25	0.00	1.06
○Thread 1163135			0.98	1.37	71.08	2.25	0.00	1.04

Report Configuration