OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	147.73
►Process 5382+		1	147.73
○Thread 5382			147.73	184.23	80.19	0.00	0.00	131.09

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	74.31
►Process 5575+		2	74.31
○Thread 5575			74.31	76.07	97.70	0.00	0.00	73.85
○Thread 5642			73.53	73.86	99.56	2.21	0.00	73.63

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	37.82
►Process 5713+		4	37.82
○Thread 5713			37.82	39.51	95.72	0.00	0.00	37.28
○Thread 5780			37.00	37.29	99.22	2.22	0.00	37.06
○Thread 5781			37.01	37.29	99.25	2.22	0.00	37.07
○Thread 5782			37.02	37.29	99.28	2.22	0.00	37.07

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	19.36
►Process 5802+		8	19.36
○Thread 5802			19.36	21.11	91.72	0.00	0.00	18.89
○Thread 5869			18.61	18.90	98.49	2.21	0.00	18.68
○Thread 5870			18.64	18.90	98.65	2.21	0.00	18.68
○Thread 5871			18.62	18.90	98.54	2.21	0.00	18.67
○Thread 5872			18.60	18.90	98.43	2.21	0.00	18.67
○Thread 5873			18.63	18.90	98.59	2.21	0.00	18.68
○Thread 5874			18.60	18.90	98.41	2.21	0.00	18.67
○Thread 5875			18.62	18.90	98.51	2.21	0.00	18.68

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	10.13
►Process 5944+		16	10.13
○Thread 5944			10.13	12.00	84.40	0.00	0.00	9.77
○Thread 6011			9.47	9.77	96.91	2.22	0.00	9.53
○Thread 6012			9.44	9.77	96.55	2.22	0.00	9.53
○Thread 6013			9.47	9.77	96.90	2.22	0.00	9.53
○Thread 6014			9.48	9.77	97.01	2.22	0.00	9.53
○Thread 6015			9.50	9.77	97.21	2.22	0.00	9.53
○Thread 6016			9.47	9.77	96.90	2.22	0.00	9.53
○Thread 6017			9.52	9.77	97.41	2.22	0.00	9.53
○Thread 6018			9.51	9.77	97.31	2.22	0.00	9.53
○Thread 6019			9.45	9.77	96.64	2.22	0.00	9.53
○Thread 6020			9.47	9.77	96.89	2.22	0.00	9.53
○Thread 6021			9.50	9.77	97.15	2.22	0.00	9.53
○Thread 6022			9.50	9.77	97.20	2.22	0.00	9.53
○Thread 6023			9.49	9.77	97.04	2.22	0.00	9.53
○Thread 6024			9.49	9.77	97.09	2.22	0.00	9.53
○Thread 6025			9.47	9.77	96.88	2.22	0.00	9.53

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	8.01
►Process 6045+		24	8.01
○Thread 6045			8.01	9.66	82.95	0.00	0.00	7.44
○Thread 6112			7.14	7.44	95.92	2.21	0.01	7.22
○Thread 6113			7.11	7.44	95.52	2.21	0.01	7.22
○Thread 6114			7.15	7.44	96.05	2.21	0.01	7.20
○Thread 6115			7.13	7.44	95.71	2.21	0.01	7.22
○Thread 6116			7.15	7.44	95.98	2.21	0.01	7.22
○Thread 6117			7.14	7.44	95.91	2.21	0.01	7.22
○Thread 6118			7.12	7.44	95.57	2.21	0.01	7.22
○Thread 6119			7.16	7.44	96.18	2.21	0.00	7.21
○Thread 6120			7.15	7.44	95.97	2.21	0.00	7.22
○Thread 6121			7.14	7.45	95.84	2.21	0.00	7.22
○Thread 6122			7.15	7.45	95.97	2.21	0.00	7.21
○Thread 6123			7.16	7.45	96.17	2.21	0.00	7.21
○Thread 6124			7.15	7.45	95.96	2.21	0.00	7.22
○Thread 6125			7.13	7.45	95.69	2.21	0.00	7.22
○Thread 6126			7.13	7.45	95.76	2.21	0.00	7.21
○Thread 6127			7.16	7.45	96.16	2.21	0.00	7.22
○Thread 6128			7.16	7.45	96.09	2.21	0.00	7.22
○Thread 6129			7.15	7.45	95.95	2.21	0.00	7.21
○Thread 6130			7.15	7.45	95.95	2.21	0.00	7.21
○Thread 6131			7.15	7.45	96.02	2.21	0.00	7.21
○Thread 6132			7.12	7.45	95.54	2.21	0.00	7.22
○Thread 6133			7.17	7.45	96.21	2.21	0.00	7.22
○Thread 6134			6.98	7.45	93.73	2.21	0.00	7.22

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	6.47
►Process 6156+		32	6.47
○Thread 6156			6.47	8.26	78.27	0.00	0.00	6.03
○Thread 6223			5.72	6.03	94.80	2.22	0.01	5.80
○Thread 6224			5.70	6.03	94.47	2.22	0.01	5.80
○Thread 6225			5.76	6.03	95.46	2.22	0.01	5.80
○Thread 6226			5.74	6.03	95.13	2.22	0.01	5.79
○Thread 6227			5.71	6.03	94.63	2.22	0.01	5.79
○Thread 6228			5.74	6.03	95.12	2.22	0.01	5.80
○Thread 6229			5.72	6.03	94.79	2.22	0.01	5.80
○Thread 6230			5.73	6.03	94.87	2.22	0.01	5.79
○Thread 6231			5.76	6.03	95.44	2.22	0.01	5.80
○Thread 6232			5.75	6.04	95.19	2.22	0.01	5.80
○Thread 6233			5.72	6.04	94.78	2.22	0.01	5.80
○Thread 6234			5.74	6.04	95.02	2.22	0.01	5.79
○Thread 6235			5.71	6.04	94.61	2.22	0.01	5.80
○Thread 6236			5.73	6.04	94.85	2.22	0.01	5.80
○Thread 6237			5.69	6.04	94.27	2.22	0.01	5.79
○Thread 6238			5.74	6.04	95.09	2.22	0.01	5.80
○Thread 6239			5.75	6.04	95.17	2.22	0.00	5.80
○Thread 6240			5.72	6.04	94.68	2.22	0.00	5.80
○Thread 6241			5.75	6.04	95.17	2.22	0.00	5.80
○Thread 6242			5.73	6.04	94.83	2.22	0.00	5.80
○Thread 6243			5.69	6.04	94.25	2.22	0.00	5.80
○Thread 6244			5.71	6.04	94.50	2.22	0.00	5.79
○Thread 6245			5.73	6.04	94.83	2.22	0.00	5.80
○Thread 6246			5.73	6.04	94.91	2.22	0.00	5.80
○Thread 6247			5.70	6.04	94.32	2.22	0.00	5.80
○Thread 6248			5.73	6.04	94.90	2.22	0.00	5.80
○Thread 6249			5.74	6.04	95.06	2.22	0.00	5.80
○Thread 6250			5.74	6.04	94.98	2.22	0.00	5.80
○Thread 6251			5.69	6.04	94.15	2.22	0.00	5.79
○Thread 6252			5.77	6.04	95.47	2.22	0.00	5.79
○Thread 6253			5.71	6.04	94.48	2.22	0.00	5.79

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	5.75
►Process 6364+		40	5.75
○Thread 6364			5.75	7.41	77.53	0.00	0.00	5.16
○Thread 6431			4.89	5.17	94.45	2.23	0.01	4.93
○Thread 6432			4.88	5.17	94.26	2.23	0.01	4.94
○Thread 6433			4.84	5.17	93.58	2.23	0.01	4.93
○Thread 6434			4.83	5.17	93.38	2.23	0.01	4.94
○Thread 6435			4.87	5.17	94.15	2.23	0.01	4.94
○Thread 6436			4.82	5.17	93.18	2.23	0.01	4.92
○Thread 6437			4.83	5.17	93.28	2.23	0.01	4.93
○Thread 6438			4.87	5.17	94.14	2.23	0.01	4.94
○Thread 6439			4.84	5.17	93.56	2.23	0.01	4.94
○Thread 6440			4.87	5.17	94.14	2.23	0.01	4.93
○Thread 6441			4.85	5.17	93.65	2.23	0.01	4.93
○Thread 6442			4.88	5.17	94.23	2.23	0.01	4.93
○Thread 6443			4.84	5.17	93.55	2.23	0.01	4.94
○Thread 6444			4.87	5.17	94.13	2.23	0.01	4.94
○Thread 6445			4.84	5.17	93.45	2.23	0.01	4.93
○Thread 6446			4.83	5.17	93.25	2.23	0.01	4.94
○Thread 6447			4.87	5.17	94.02	2.23	0.01	4.93
○Thread 6448			4.86	5.17	93.82	2.23	0.01	4.93
○Thread 6449			4.89	5.17	94.40	2.23	0.01	4.93
○Thread 6450			4.86	5.17	93.82	2.23	0.01	4.94
○Thread 6451			4.81	5.18	92.85	2.23	0.01	4.93
○Thread 6452			4.87	5.18	94.10	2.23	0.01	4.93
○Thread 6453			4.86	5.18	93.81	2.23	0.01	4.94
○Thread 6454			4.88	5.18	94.29	2.23	0.00	4.93
○Thread 6455			4.87	5.18	94.09	2.23	0.00	4.94
○Thread 6456			4.86	5.18	93.90	2.23	0.00	4.94
○Thread 6457			4.87	5.18	94.09	2.23	0.00	4.94
○Thread 6458			4.86	5.18	93.79	2.23	0.00	4.93
○Thread 6459			4.85	5.18	93.60	2.23	0.00	4.93
○Thread 6460			4.84	5.18	93.40	2.23	0.00	4.93
○Thread 6461			4.86	5.18	93.79	2.23	0.00	4.93
○Thread 6462			4.85	5.18	93.59	2.23	0.00	4.94
○Thread 6463			4.86	5.18	93.88	2.23	0.00	4.94
○Thread 6464			4.83	5.18	93.20	2.23	0.00	4.94
○Thread 6465			4.85	5.18	93.68	2.23	0.00	4.93
○Thread 6466			4.86	5.18	93.87	2.23	0.00	4.93
○Thread 6467			4.81	5.18	92.90	2.23	0.00	4.93
○Thread 6468			4.84	5.18	93.48	2.23	0.00	4.94
○Thread 6469			4.53	5.18	87.39	2.23	0.00	4.94

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	5.02
►Process 6493+		48	5.02
○Thread 6493			5.02	6.78	74.10	0.00	0.00	4.54
○Thread 6565			4.24	4.56	93.19	2.21	0.01	4.32
○Thread 6566			4.23	4.56	92.97	2.21	0.01	4.32
○Thread 6567			4.25	4.56	93.40	2.21	0.01	4.32
○Thread 6568			4.24	4.56	93.07	2.21	0.01	4.32
○Thread 6569			4.23	4.56	92.96	2.21	0.01	4.31
○Thread 6570			4.25	4.56	93.29	2.21	0.01	4.32
○Thread 6571			4.20	4.56	92.19	2.21	0.01	4.32
○Thread 6572			4.23	4.56	92.95	2.21	0.01	4.32
○Thread 6573			4.23	4.56	92.95	2.21	0.01	4.32
○Thread 6574			4.23	4.56	92.94	2.21	0.01	4.32
○Thread 6575			4.21	4.56	92.39	2.21	0.01	4.32
○Thread 6576			4.25	4.56	93.38	2.21	0.01	4.32
○Thread 6577			4.21	4.56	92.50	2.21	0.01	4.32
○Thread 6578			4.21	4.56	92.49	2.21	0.01	4.32
○Thread 6579			4.24	4.56	93.04	2.21	0.01	4.32
○Thread 6580			4.21	4.56	92.38	2.21	0.01	4.29
○Thread 6581			4.22	4.56	92.70	2.21	0.01	4.32
○Thread 6582			4.22	4.56	92.70	2.21	0.01	4.32
○Thread 6583			4.21	4.56	92.48	2.21	0.01	4.31
○Thread 6584			4.21	4.56	92.47	2.21	0.01	4.32
○Thread 6585			4.22	4.56	92.58	2.21	0.01	4.32
○Thread 6586			4.21	4.56	92.36	2.21	0.01	4.32
○Thread 6587			4.23	4.56	92.79	2.21	0.01	4.32
○Thread 6588			4.22	4.56	92.57	2.21	0.01	4.31
○Thread 6589			4.22	4.56	92.68	2.21	0.01	4.31
○Thread 6590			4.23	4.56	92.89	2.21	0.01	4.31
○Thread 6591			4.22	4.56	92.67	2.21	0.01	4.32
○Thread 6592			4.25	4.56	93.22	2.21	0.01	4.32
○Thread 6593			4.25	4.56	93.32	2.21	0.01	4.32
○Thread 6594			4.26	4.56	93.43	2.21	0.01	4.32
○Thread 6595			4.23	4.56	92.77	2.21	0.00	4.31
○Thread 6596			4.24	4.56	93.10	2.21	0.00	4.32
○Thread 6597			4.21	4.56	92.33	2.21	0.00	4.32
○Thread 6598			4.23	4.56	92.87	2.21	0.00	4.32
○Thread 6599			4.25	4.56	93.20	2.21	0.00	4.31
○Thread 6600			4.26	4.56	93.41	2.21	0.00	4.32
○Thread 6601			4.19	4.56	91.99	2.21	0.00	4.32
○Thread 6602			4.23	4.56	92.75	2.21	0.00	4.32
○Thread 6603			4.23	4.56	92.86	2.21	0.00	4.31
○Thread 6604			4.24	4.56	93.07	2.21	0.00	4.31
○Thread 6605			4.27	4.56	93.62	2.21	0.00	4.31
○Thread 6606			4.22	4.56	92.52	2.21	0.00	4.32
○Thread 6607			4.19	4.56	91.97	2.21	0.00	4.32
○Thread 6608			4.22	4.56	92.62	2.21	0.00	4.31
○Thread 6609			4.17	4.56	91.52	2.21	0.00	4.32
○Thread 6610			4.06	4.56	89.11	2.21	0.00	4.31
○Thread 6611			3.81	4.56	83.52	2.21	0.00	4.31

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	4.53
►Process 6637+		56	4.53
○Thread 6637			4.53	6.31	71.91	0.00	0.00	4.05
○Thread 6704			3.73	4.07	91.80	2.22	0.01	3.83
○Thread 6705			3.74	4.07	92.05	2.22	0.01	3.83
○Thread 6706			3.75	4.07	92.17	2.22	0.01	3.83
○Thread 6707			3.77	4.07	92.78	2.22	0.01	3.83
○Thread 6708			3.76	4.07	92.40	2.22	0.01	3.83
○Thread 6709			3.73	4.07	91.66	2.22	0.01	3.83
○Thread 6710			3.75	4.07	92.15	2.22	0.01	3.83
○Thread 6711			3.77	4.07	92.76	2.22	0.01	3.83
○Thread 6712			3.75	4.07	92.27	2.22	0.01	3.83
○Thread 6713			3.74	4.07	92.02	2.22	0.01	3.83
○Thread 6714			3.74	4.07	92.02	2.22	0.01	3.83
○Thread 6715			3.76	4.07	92.50	2.22	0.01	3.83
○Thread 6716			3.75	4.07	92.13	2.22	0.01	3.83
○Thread 6717			3.76	4.07	92.37	2.22	0.01	3.82
○Thread 6718			3.77	4.07	92.62	2.22	0.01	3.83
○Thread 6719			3.72	4.07	91.39	2.22	0.01	3.83
○Thread 6720			3.77	4.07	92.61	2.22	0.01	3.83
○Thread 6721			3.72	4.07	91.50	2.22	0.01	3.83
○Thread 6722			3.75	4.07	92.11	2.23	0.01	3.83
○Thread 6723			3.75	4.07	92.23	2.23	0.01	3.83
○Thread 6724			3.75	4.07	92.11	2.23	0.01	3.83
○Thread 6725			3.76	4.07	92.47	2.23	0.01	3.83
○Thread 6726			3.74	4.07	91.85	2.23	0.01	3.83
○Thread 6727			3.76	4.07	92.34	2.23	0.01	3.83
○Thread 6728			3.77	4.07	92.58	2.23	0.01	3.83
○Thread 6729			3.74	4.07	91.84	2.23	0.01	3.83
○Thread 6730			3.73	4.07	91.72	2.23	0.01	3.82
○Thread 6731			3.76	4.07	92.33	2.23	0.01	3.83
○Thread 6732			3.75	4.07	92.20	2.23	0.01	3.83
○Thread 6733			3.73	4.07	91.58	2.23	0.01	3.83
○Thread 6734			3.75	4.07	92.19	2.23	0.01	3.83
○Thread 6735			3.75	4.07	92.07	2.23	0.01	3.83
○Thread 6736			3.74	4.07	91.94	2.23	0.01	3.83
○Thread 6737			3.72	4.07	91.45	2.23	0.01	3.83
○Thread 6738			3.74	4.07	91.82	2.23	0.01	3.83
○Thread 6739			3.75	4.07	92.18	2.23	0.01	3.83
○Thread 6740			3.74	4.07	91.81	2.23	0.01	3.82
○Thread 6741			3.74	4.07	91.81	2.23	0.01	3.83
○Thread 6742			3.75	4.07	92.05	2.23	0.01	3.83
○Thread 6743			3.74	4.07	91.92	2.23	0.00	3.83
○Thread 6744			3.71	4.07	91.06	2.23	0.00	3.83
○Thread 6745			3.72	4.07	91.30	2.23	0.00	3.83
○Thread 6746			3.73	4.07	91.54	2.23	0.00	3.83
○Thread 6747			3.73	4.07	91.54	2.23	0.00	3.83
○Thread 6748			3.75	4.07	92.15	2.23	0.00	3.83
○Thread 6749			3.73	4.07	91.66	2.23	0.00	3.83
○Thread 6750			3.75	4.08	92.14	2.23	0.00	3.83
○Thread 6751			3.78	4.08	92.75	2.23	0.00	3.83
○Thread 6752			3.67	4.08	90.05	2.23	0.00	3.82
○Thread 6753			3.75	4.08	92.01	2.23	0.00	3.83
○Thread 6754			3.75	4.08	92.13	2.23	0.00	3.81
○Thread 6755			3.72	4.08	91.27	2.23	0.00	3.83
○Thread 6756			3.68	4.08	90.41	2.23	0.00	3.83
○Thread 6757			3.47	4.08	85.13	2.23	0.00	3.83
○Thread 6758			3.36	4.08	82.55	2.23	0.00	3.83

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	4.20
►Process 10864+		64	4.20
○Thread 10864			4.20	5.90	71.21	0.00	0.00	3.65
○Thread 11452			3.32	3.67	90.53	2.22	0.02	3.42
○Thread 11453			3.28	3.67	89.44	2.22	0.02	3.43
○Thread 11454			3.30	3.67	90.12	2.22	0.02	3.42
○Thread 11455			3.33	3.67	90.93	2.22	0.02	3.43
○Thread 11456			3.33	3.67	90.93	2.22	0.02	3.43
○Thread 11457			3.27	3.67	89.29	2.22	0.02	3.43
○Thread 11458			3.33	3.67	90.78	2.22	0.02	3.43
○Thread 11459			3.34	3.67	91.05	2.22	0.02	3.42
○Thread 11460			3.31	3.67	90.23	2.22	0.02	3.43
○Thread 11461			3.29	3.67	89.82	2.22	0.01	3.43
○Thread 11462			3.30	3.67	90.08	2.22	0.01	3.43
○Thread 11463			3.30	3.67	89.94	2.22	0.01	3.42
○Thread 11464			3.32	3.67	90.49	2.22	0.01	3.42
○Thread 11465			3.30	3.67	89.94	2.22	0.01	3.42
○Thread 11466			3.32	3.67	90.61	2.22	0.01	3.43
○Thread 11467			3.31	3.67	90.34	2.22	0.01	3.42
○Thread 11468			3.31	3.67	90.20	2.22	0.01	3.42
○Thread 11469			3.32	3.67	90.60	2.22	0.01	3.43
○Thread 11470			3.32	3.67	90.46	2.22	0.01	3.43
○Thread 11471			3.31	3.67	90.32	2.22	0.01	3.43
○Thread 11472			3.30	3.67	89.91	2.22	0.01	3.42
○Thread 11473			3.30	3.67	90.04	2.22	0.01	3.43
○Thread 11474			3.35	3.67	91.40	2.22	0.01	3.43
○Thread 11475			3.30	3.67	89.90	2.22	0.01	3.43
○Thread 11476			3.28	3.67	89.49	2.22	0.01	3.41
○Thread 11477			3.26	3.67	88.94	2.22	0.01	3.42
○Thread 11478			3.26	3.67	88.80	2.22	0.01	3.43
○Thread 11479			3.30	3.67	89.88	2.22	0.01	3.43
○Thread 11480			3.29	3.67	89.75	2.22	0.01	3.43
○Thread 11481			3.28	3.67	89.47	2.22	0.01	3.43
○Thread 11482			3.32	3.67	90.55	2.22	0.01	3.43
○Thread 11483			3.31	3.67	90.14	2.22	0.01	3.43
○Thread 11484			3.32	3.67	90.41	2.22	0.01	3.43
○Thread 11485			3.30	3.67	89.86	2.22	0.01	3.43
○Thread 11486			3.33	3.67	90.67	2.22	0.01	3.43
○Thread 11487			3.32	3.67	90.40	2.22	0.01	3.42
○Thread 11488			3.30	3.67	89.98	2.22	0.01	3.42
○Thread 11489			3.30	3.67	89.98	2.22	0.01	3.43
○Thread 11490			3.31	3.67	90.11	2.22	0.01	3.42
○Thread 11491			3.30	3.67	89.84	2.22	0.01	3.43
○Thread 11492			3.30	3.67	89.83	2.22	0.01	3.43
○Thread 11493			3.34	3.67	91.05	2.22	0.01	3.43
○Thread 11494			3.27	3.67	89.14	2.22	0.01	3.43
○Thread 11495			3.33	3.67	90.77	2.22	0.01	3.43
○Thread 11496			3.29	3.67	89.54	2.22	0.01	3.42
○Thread 11497			3.28	3.67	89.27	2.22	0.01	3.41
○Thread 11498			3.27	3.67	88.99	2.23	0.01	3.42
○Thread 11499			3.33	3.67	90.62	2.23	0.00	3.42
○Thread 11500			3.32	3.67	90.48	2.23	0.00	3.43
○Thread 11501			3.28	3.67	89.25	2.23	0.00	3.43
○Thread 11502			3.30	3.68	89.79	2.23	0.00	3.43
○Thread 11503			3.35	3.68	91.28	2.23	0.00	3.41
○Thread 11504			3.30	3.68	89.92	2.23	0.00	3.42
○Thread 11505			3.31	3.68	90.05	2.23	0.00	3.42
○Thread 11506			3.33	3.68	90.73	2.23	0.00	3.41
○Thread 11507			3.33	3.68	90.59	2.23	0.00	3.42
○Thread 11508			3.31	3.68	90.04	2.23	0.00	3.43
○Thread 11509			3.31	3.68	90.04	2.23	0.00	3.43
○Thread 11510			3.32	3.68	90.44	2.23	0.00	3.42
○Thread 11511			3.36	3.68	91.52	2.23	0.00	3.42
○Thread 11512			3.31	3.68	90.16	2.23	0.00	3.43
○Thread 11513			3.30	3.68	89.75	2.23	0.00	3.43
○Thread 11514			3.33	3.68	90.70	2.23	0.00	3.43

Report Configuration