OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	173.67
►Process 622199+		1	173.67
○Thread 622199			173.67	175.23	99.11	0.00	0.00	175.20

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	87.56
►Process 622419+		2	87.56
○Thread 622419			87.56	89.13	98.25	0.00	0.00	87.30
○Thread 622518			86.97	87.31	99.61	1.82	0.00	87.07

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	44.30
►Process 622590+		4	44.30
○Thread 622590			44.30	45.89	96.53	0.00	0.00	44.06
○Thread 622689			43.78	44.07	99.33	1.82	0.00	43.83
○Thread 622690			43.75	44.07	99.26	1.82	0.00	43.83
○Thread 622691			43.73	44.07	99.23	1.82	0.00	43.84

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	22.89
►Process 622759+		8	22.89
○Thread 622759			22.89	24.44	93.67	0.00	0.00	22.61
○Thread 622859			22.29	22.62	98.54	1.82	0.00	22.38
○Thread 622860			22.28	22.62	98.52	1.82	0.00	22.38
○Thread 622861			22.32	22.62	98.67	1.82	0.00	22.38
○Thread 622862			22.26	22.62	98.41	1.82	0.00	22.38
○Thread 622863			22.24	22.62	98.32	1.82	0.00	22.38
○Thread 622864			22.23	22.62	98.27	1.82	0.00	22.38
○Thread 622865			22.31	22.62	98.65	1.82	0.00	22.38

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	12.45
►Process 622885+		16	12.45
○Thread 622885			12.45	14.00	88.89	0.00	0.00	12.19
○Thread 622984			11.76	12.19	96.45	1.80	0.01	11.96
○Thread 622985			11.71	12.19	96.03	1.80	0.01	11.95
○Thread 622986			11.73	12.19	96.15	1.80	0.01	11.95
○Thread 622987			11.72	12.19	96.07	1.80	0.00	11.96
○Thread 622988			11.77	12.19	96.48	1.80	0.00	11.96
○Thread 622989			11.68	12.19	95.74	1.80	0.00	11.95
○Thread 622990			11.69	12.19	95.86	1.80	0.00	11.95
○Thread 622991			11.72	12.20	96.10	1.80	0.00	11.95
○Thread 622992			11.69	12.20	95.82	1.80	0.00	11.95
○Thread 622993			11.72	12.20	96.06	1.80	0.00	11.96
○Thread 622994			11.71	12.20	96.02	1.80	0.00	11.96
○Thread 622995			11.81	12.20	96.83	1.80	0.00	11.95
○Thread 622996			11.76	12.20	96.38	1.80	0.00	11.96
○Thread 622997			11.72	12.20	96.09	1.80	0.00	11.95
○Thread 622998			11.81	12.20	96.83	1.80	0.00	11.95

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	9.61
►Process 623018+		24	9.61
○Thread 623018			9.61	11.38	84.37	0.00	0.00	9.56
○Thread 623117			9.03	9.57	94.34	1.81	0.01	9.33
○Thread 623118			9.02	9.57	94.24	1.81	0.01	9.33
○Thread 623119			9.02	9.57	94.29	1.81	0.01	9.33
○Thread 623120			9.11	9.57	95.17	1.81	0.01	9.33
○Thread 623121			9.01	9.57	94.12	1.81	0.01	9.31
○Thread 623122			8.98	9.57	93.81	1.81	0.01	9.32
○Thread 623123			9.02	9.57	94.22	1.81	0.01	9.33
○Thread 623124			9.02	9.57	94.27	1.81	0.01	9.32
○Thread 623125			8.98	9.57	93.85	1.81	0.01	9.32
○Thread 623126			9.01	9.57	94.17	1.81	0.01	9.32
○Thread 623127			8.98	9.57	93.80	1.81	0.01	9.33
○Thread 623128			9.10	9.57	95.10	1.81	0.00	9.32
○Thread 623129			9.07	9.57	94.73	1.81	0.00	9.33
○Thread 623130			9.00	9.57	94.00	1.81	0.00	9.33
○Thread 623131			9.00	9.57	94.05	1.81	0.00	9.33
○Thread 623132			8.99	9.57	93.89	1.81	0.00	9.33
○Thread 623133			8.99	9.57	93.94	1.81	0.00	9.33
○Thread 623134			9.00	9.57	94.04	1.81	0.00	9.33
○Thread 623135			8.99	9.57	93.88	1.81	0.00	9.31
○Thread 623136			8.99	9.57	93.88	1.81	0.00	9.33
○Thread 623137			8.98	9.57	93.83	1.81	0.00	9.33
○Thread 623138			9.04	9.57	94.45	1.81	0.00	9.33
○Thread 623139			8.80	9.57	91.94	1.81	0.00	9.32

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	8.06
►Process 623208+		32	8.06
○Thread 623208			8.06	9.96	80.89	0.00	0.00	8.10
○Thread 623307			7.48	8.13	92.04	1.82	0.01	7.88
○Thread 623308			7.55	8.13	92.90	1.82	0.01	7.88
○Thread 623309			7.51	8.13	92.34	1.82	0.01	7.89
○Thread 623310			7.49	8.13	92.16	1.82	0.01	7.89
○Thread 623311			7.47	8.13	91.85	1.82	0.01	7.89
○Thread 623312			7.47	8.13	91.84	1.82	0.01	7.89
○Thread 623313			7.47	8.13	91.90	1.82	0.01	7.89
○Thread 623314			7.58	8.13	93.25	1.82	0.01	7.88
○Thread 623315			7.50	8.13	92.27	1.82	0.01	7.88
○Thread 623316			7.41	8.13	91.16	1.82	0.01	7.88
○Thread 623317			7.41	8.13	91.09	1.82	0.01	7.88
○Thread 623318			7.58	8.13	93.18	1.82	0.01	7.88
○Thread 623319			7.46	8.13	91.70	1.82	0.01	7.88
○Thread 623320			7.41	8.13	91.08	1.82	0.01	7.89
○Thread 623321			7.58	8.13	93.23	1.82	0.01	7.89
○Thread 623322			7.39	8.13	90.89	1.82	0.01	7.88
○Thread 623323			7.45	8.13	91.63	1.82	0.01	7.89
○Thread 623324			7.37	8.13	90.64	1.82	0.01	7.86
○Thread 623325			7.52	8.13	92.49	1.82	0.01	7.89
○Thread 623326			7.42	8.13	91.19	1.82	0.00	7.89
○Thread 623327			7.40	8.13	91.01	1.82	0.00	7.88
○Thread 623328			7.34	8.13	90.27	1.82	0.00	7.87
○Thread 623329			7.42	8.13	91.18	1.82	0.00	7.88
○Thread 623330			7.45	8.13	91.61	1.82	0.00	7.89
○Thread 623331			7.39	8.13	90.81	1.82	0.00	7.88
○Thread 623332			7.37	8.13	90.56	1.82	0.00	7.88
○Thread 623333			7.44	8.13	91.42	1.82	0.00	7.88
○Thread 623334			7.35	8.13	90.31	1.82	0.00	7.89
○Thread 623335			7.40	8.13	90.92	1.82	0.00	7.88
○Thread 623336			7.39	8.13	90.86	1.82	0.00	7.89
○Thread 623337			7.37	8.13	90.61	1.82	0.00	7.88

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	7.10
►Process 623357+		40	7.10
○Thread 623357			7.10	9.06	78.34	0.00	0.00	7.22
○Thread 623456			6.51	7.23	90.03	1.81	0.01	6.99
○Thread 623457			6.44	7.23	89.06	1.81	0.01	6.99
○Thread 623458			6.52	7.23	90.16	1.81	0.01	6.99
○Thread 623459			6.39	7.23	88.29	1.81	0.01	6.99
○Thread 623460			6.52	7.23	90.15	1.81	0.01	6.99
○Thread 623461			6.50	7.23	89.81	1.81	0.01	6.99
○Thread 623462			6.51	7.23	89.94	1.81	0.01	6.99
○Thread 623463			6.46	7.23	89.25	1.81	0.01	6.99
○Thread 623464			6.45	7.23	89.17	1.81	0.01	6.99
○Thread 623465			6.44	7.23	89.03	1.81	0.01	6.99
○Thread 623466			6.50	7.23	89.78	1.81	0.01	6.99
○Thread 623467			6.49	7.23	89.71	1.81	0.01	6.99
○Thread 623468			6.45	7.23	89.09	1.81	0.01	6.99
○Thread 623469			6.42	7.23	88.67	1.81	0.01	6.99
○Thread 623470			6.50	7.23	89.77	1.81	0.01	6.99
○Thread 623471			6.48	7.24	89.56	1.81	0.01	6.99
○Thread 623472			6.44	7.24	88.94	1.81	0.01	6.99
○Thread 623473			6.48	7.24	89.49	1.81	0.01	6.99
○Thread 623474			6.47	7.24	89.42	1.81	0.01	6.98
○Thread 623475			6.47	7.24	89.35	1.81	0.01	6.99
○Thread 623476			6.40	7.24	88.44	1.81	0.01	6.99
○Thread 623477			6.38	7.24	88.17	1.81	0.01	6.99
○Thread 623478			6.37	7.24	87.96	1.81	0.01	6.99
○Thread 623479			6.45	7.24	89.06	1.81	0.01	6.99
○Thread 623480			6.41	7.24	88.57	1.81	0.01	6.99
○Thread 623481			6.32	7.24	87.33	1.81	0.01	6.99
○Thread 623482			6.52	7.24	90.02	1.81	0.01	6.99
○Thread 623483			6.49	7.24	89.60	1.81	0.00	6.99
○Thread 623484			6.36	7.24	87.80	1.81	0.00	6.99
○Thread 623485			6.48	7.24	89.53	1.81	0.00	6.99
○Thread 623486			6.45	7.24	89.11	1.81	0.00	6.98
○Thread 623487			6.73	7.24	92.97	1.81	0.00	6.99
○Thread 623488			6.42	7.24	88.62	1.81	0.00	6.98
○Thread 623489			6.53	7.24	90.21	1.81	0.00	6.99
○Thread 623490			6.68	7.24	92.28	1.81	0.00	6.98
○Thread 623491			6.38	7.24	88.13	1.81	0.00	6.98
○Thread 623492			6.41	7.24	88.54	1.81	0.00	6.99
○Thread 623493			6.42	7.24	88.61	1.82	0.00	6.98
○Thread 623494			6.20	7.24	85.63	1.82	0.00	6.99

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	6.56
►Process 623514+		48	6.56
○Thread 623514			6.56	8.54	76.74	0.00	0.00	6.69
○Thread 623613			5.95	6.70	88.77	1.82	0.02	6.46
○Thread 623614			5.89	6.70	87.80	1.82	0.02	6.46
○Thread 623615			5.94	6.70	88.54	1.82	0.02	6.46
○Thread 623616			5.74	6.70	85.55	1.82	0.02	6.46
○Thread 623617			5.84	6.70	87.04	1.82	0.02	6.46
○Thread 623618			5.85	6.70	87.26	1.82	0.02	6.46
○Thread 623619			5.72	6.70	85.32	1.82	0.01	6.46
○Thread 623620			5.98	6.70	89.20	1.82	0.01	6.46
○Thread 623621			5.68	6.70	84.72	1.82	0.01	6.46
○Thread 623622			5.84	6.70	87.10	1.82	0.01	6.46
○Thread 623623			5.79	6.70	86.35	1.82	0.01	6.46
○Thread 623624			6.03	6.71	89.86	1.82	0.01	6.46
○Thread 623625			5.89	6.71	87.84	1.82	0.01	6.46
○Thread 623626			5.75	6.71	85.75	1.82	0.01	6.46
○Thread 623627			5.94	6.71	88.58	1.82	0.01	6.45
○Thread 623628			5.82	6.71	86.79	1.82	0.01	6.46
○Thread 623629			5.71	6.71	85.07	1.82	0.01	6.46
○Thread 623630			5.69	6.71	84.84	1.82	0.01	6.46
○Thread 623631			5.77	6.71	86.03	1.82	0.01	6.46
○Thread 623632			5.85	6.71	87.15	1.82	0.01	6.46
○Thread 623633			5.78	6.71	86.10	1.82	0.01	6.45
○Thread 623634			5.76	6.71	85.80	1.82	0.01	6.46
○Thread 623635			5.81	6.71	86.62	1.82	0.01	6.46
○Thread 623636			5.70	6.71	84.90	1.82	0.01	6.46
○Thread 623637			5.76	6.71	85.87	1.82	0.01	6.46
○Thread 623638			5.74	6.71	85.49	1.82	0.01	6.46
○Thread 623639			5.68	6.71	84.67	1.83	0.01	6.46
○Thread 623640			5.83	6.71	86.83	1.83	0.01	6.46
○Thread 623641			5.75	6.71	85.71	1.83	0.01	6.46
○Thread 623642			5.78	6.71	86.08	1.83	0.01	6.46
○Thread 623643			5.77	6.71	86.00	1.83	0.01	6.46
○Thread 623644			5.97	6.71	88.98	1.83	0.01	6.46
○Thread 623645			5.74	6.71	85.55	1.83	0.01	6.46
○Thread 623646			5.70	6.71	84.87	1.83	0.01	6.45
○Thread 623647			5.81	6.71	86.51	1.83	0.01	6.46
○Thread 623648			5.82	6.71	86.66	1.83	0.00	6.46
○Thread 623649			5.76	6.71	85.76	1.83	0.00	6.46
○Thread 623650			5.84	6.71	86.95	1.83	0.00	6.46
○Thread 623651			5.69	6.71	84.71	1.83	0.00	6.46
○Thread 623652			5.90	6.71	87.84	1.83	0.00	6.46
○Thread 623653			5.77	6.71	85.90	1.83	0.00	6.46
○Thread 623654			5.76	6.71	85.74	1.83	0.00	6.46
○Thread 623655			5.78	6.71	86.11	1.83	0.00	6.46
○Thread 623656			5.91	6.71	88.05	1.83	0.00	6.46
○Thread 623657			5.78	6.71	86.03	1.83	0.00	6.44
○Thread 623658			5.60	6.71	83.50	1.83	0.00	6.46
○Thread 623659			5.48	6.71	81.63	1.83	0.00	6.46

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	6.17
►Process 623679+		56	6.17
○Thread 623679			6.17	8.13	75.85	0.00	0.00	6.26
○Thread 623778			5.45	6.28	86.75	1.83	0.02	6.04
○Thread 623779			5.39	6.28	85.79	1.83	0.02	6.03
○Thread 623780			5.43	6.28	86.35	1.83	0.02	6.04
○Thread 623781			5.41	6.28	86.11	1.83	0.02	6.03
○Thread 623782			5.29	6.28	84.19	1.83	0.02	6.04
○Thread 623783			5.49	6.28	87.37	1.83	0.02	6.03
○Thread 623784			5.36	6.28	85.30	1.83	0.02	6.03
○Thread 623785			5.40	6.28	85.86	1.83	0.02	6.03
○Thread 623786			5.33	6.28	84.74	1.83	0.02	6.04
○Thread 623787			5.39	6.28	85.69	1.83	0.02	6.04
○Thread 623788			5.43	6.28	86.40	1.83	0.02	6.04
○Thread 623789			5.49	6.28	87.28	1.83	0.02	6.04
○Thread 623790			5.45	6.28	86.72	1.83	0.02	6.04
○Thread 623791			5.38	6.29	85.60	1.83	0.02	6.03
○Thread 623792			5.41	6.29	86.07	1.83	0.02	6.04
○Thread 623793			5.38	6.29	85.67	1.83	0.01	6.03
○Thread 623794			5.32	6.29	84.63	1.83	0.01	6.04
○Thread 623795			5.33	6.29	84.71	1.83	0.01	6.04
○Thread 623796			5.36	6.29	85.27	1.83	0.01	6.03
○Thread 623797			5.37	6.29	85.42	1.83	0.01	6.04
○Thread 623798			5.36	6.29	85.26	1.83	0.01	6.04
○Thread 623799			5.25	6.29	83.51	1.83	0.01	6.04
○Thread 623800			5.24	6.29	83.34	1.83	0.01	6.04
○Thread 623801			5.23	6.29	83.10	1.83	0.01	6.04
○Thread 623802			5.27	6.29	83.74	1.83	0.01	6.04
○Thread 623803			5.39	6.29	85.72	1.83	0.01	6.03
○Thread 623804			5.41	6.29	86.11	1.83	0.01	6.04
○Thread 623805			5.34	6.29	84.92	1.83	0.01	6.04
○Thread 623806			5.36	6.29	85.23	1.83	0.01	6.28
○Thread 623807			5.29	6.29	84.20	1.83	0.01	6.04
○Thread 623808			5.27	6.29	83.88	1.83	0.01	6.04
○Thread 623809			5.48	6.29	87.13	1.83	0.01	6.04
○Thread 623810			5.33	6.29	84.75	1.83	0.01	6.04
○Thread 623811			5.28	6.29	83.95	1.83	0.01	6.03
○Thread 623812			5.31	6.29	84.42	1.83	0.01	6.04
○Thread 623813			5.28	6.29	83.94	1.83	0.01	6.03
○Thread 623814			5.32	6.29	84.57	1.83	0.01	6.04
○Thread 623815			5.32	6.29	84.65	1.83	0.01	6.04
○Thread 623816			5.29	6.29	84.01	1.83	0.01	6.04
○Thread 623817			5.26	6.29	83.61	1.83	0.01	6.02
○Thread 623818			5.38	6.29	85.59	1.83	0.01	6.04
○Thread 623819			5.38	6.29	85.51	1.83	0.01	6.04
○Thread 623820			5.36	6.29	85.27	1.83	0.01	6.02
○Thread 623821			5.39	6.29	85.66	1.83	0.00	6.03
○Thread 623822			5.29	6.29	84.07	1.83	0.00	6.04
○Thread 623823			5.38	6.29	85.58	1.83	0.00	6.04
○Thread 623824			5.38	6.29	85.50	1.83	0.00	6.04
○Thread 623825			5.79	6.29	91.93	1.83	0.00	6.04
○Thread 623826			5.64	6.29	89.62	1.83	0.00	6.04
○Thread 623827			5.73	6.29	90.97	1.83	0.00	6.04
○Thread 623828			5.60	6.29	88.98	1.83	0.00	6.04
○Thread 623829			5.32	6.29	84.53	1.83	0.00	6.04
○Thread 623830			5.36	6.29	85.08	1.83	0.00	6.03
○Thread 623831			5.15	6.29	81.82	1.83	0.00	6.04
○Thread 623832			5.11	6.29	81.10	1.83	0.00	6.04

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	5.58
►Process 623901+		64	5.58
○Thread 623901			5.58	7.83	71.16	0.00	0.00	5.98
○Thread 624000			5.01	6.00	83.44	1.81	0.02	5.74
○Thread 624001			5.09	6.00	84.85	1.81	0.02	5.75
○Thread 624002			5.06	6.00	84.35	1.81	0.02	5.74
○Thread 624003			5.07	6.00	84.43	1.81	0.02	5.75
○Thread 624004			5.12	6.00	85.43	1.81	0.02	5.75
○Thread 624005			5.01	6.00	83.59	1.81	0.02	5.74
○Thread 624006			5.03	6.00	83.84	1.81	0.02	5.75
○Thread 624007			5.02	6.00	83.58	1.81	0.02	5.75
○Thread 624008			5.04	6.00	84.08	1.81	0.02	5.75
○Thread 624009			4.96	6.00	82.66	1.81	0.02	5.74
○Thread 624010			5.00	6.00	83.24	1.81	0.02	5.75
○Thread 624011			5.17	6.00	86.24	1.81	0.02	5.75
○Thread 624012			4.94	6.00	82.39	1.81	0.02	5.73
○Thread 624013			4.82	6.00	80.31	1.81	0.02	5.75
○Thread 624014			5.01	6.00	83.47	1.81	0.02	5.75
○Thread 624015			4.98	6.00	83.05	1.81	0.02	5.75
○Thread 624016			4.99	6.00	83.13	1.81	0.02	5.75
○Thread 624017			4.92	6.00	81.96	1.81	0.02	5.75
○Thread 624018			4.92	6.00	81.88	1.81	0.02	5.75
○Thread 624019			4.87	6.00	81.12	1.82	0.02	5.74
○Thread 624020			4.97	6.00	82.79	1.82	0.02	5.75
○Thread 624021			5.00	6.00	83.28	1.82	0.02	5.74
○Thread 624022			4.97	6.00	82.70	1.82	0.01	5.75
○Thread 624023			4.97	6.00	82.78	1.82	0.01	5.75
○Thread 624024			4.92	6.00	81.86	1.82	0.01	5.74
○Thread 624025			4.89	6.00	81.52	1.82	0.01	5.75
○Thread 624026			4.91	6.00	81.85	1.82	0.01	5.73
○Thread 624027			4.98	6.00	82.93	1.82	0.01	5.75
○Thread 624028			4.88	6.01	81.26	1.82	0.01	5.75
○Thread 624029			4.98	6.01	83.01	1.82	0.01	5.73
○Thread 624030			4.89	6.01	81.43	1.82	0.01	5.75
○Thread 624031			5.07	6.01	84.50	1.82	0.01	5.74
○Thread 624032			4.91	6.01	81.75	1.82	0.01	5.75
○Thread 624033			4.92	6.01	81.83	1.82	0.01	5.75
○Thread 624034			4.95	6.01	82.50	1.82	0.01	5.74
○Thread 624035			4.95	6.01	82.41	1.82	0.01	5.75
○Thread 624036			4.99	6.01	82.99	1.82	0.01	5.75
○Thread 624037			4.92	6.01	81.90	1.82	0.01	5.74
○Thread 624038			5.00	6.01	83.23	1.82	0.01	5.74
○Thread 624039			4.95	6.01	82.48	1.82	0.01	5.74
○Thread 624040			4.99	6.01	82.98	1.82	0.01	5.75
○Thread 624041			4.94	6.01	82.31	1.82	0.01	5.73
○Thread 624042			4.99	6.01	82.97	1.82	0.01	5.74
○Thread 624043			4.93	6.01	82.05	1.82	0.01	5.75
○Thread 624044			4.94	6.01	82.13	1.82	0.01	5.74
○Thread 624045			4.93	6.01	82.05	1.82	0.01	5.75
○Thread 624046			4.94	6.01	82.21	1.82	0.01	5.75
○Thread 624047			5.27	6.01	87.61	1.82	0.01	5.75
○Thread 624048			4.99	6.01	83.12	1.82	0.01	5.75
○Thread 624049			4.90	6.01	81.62	1.82	0.01	5.75
○Thread 624050			5.03	6.01	83.69	1.82	0.01	5.74
○Thread 624051			4.92	6.01	81.94	1.82	0.00	5.75
○Thread 624052			4.90	6.01	81.52	1.82	0.00	5.75
○Thread 624053			5.03	6.01	83.68	1.82	0.00	5.75
○Thread 624054			4.90	6.01	81.52	1.82	0.00	5.74
○Thread 624055			4.93	6.01	82.10	1.82	0.00	5.75
○Thread 624056			4.95	6.01	82.34	1.82	0.00	5.75
○Thread 624057			4.83	6.01	80.35	1.82	0.00	5.75
○Thread 624058			4.95	6.01	82.34	1.82	0.00	5.75
○Thread 624059			5.10	6.01	84.83	1.82	0.00	5.75
○Thread 624060			5.03	6.01	83.66	1.82	0.00	5.75
○Thread 624061			5.01	6.01	83.33	1.82	0.00	5.75
○Thread 624062			4.99	6.01	83.07	1.82	0.00	5.75

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	5.48
►Process 624082+		72	5.48
○Thread 624082			5.48	7.68	71.32	0.00	0.00	5.82
○Thread 624181			4.77	5.83	81.79	1.83	0.03	5.56
○Thread 624182			4.86	5.83	83.33	1.83	0.03	5.58
○Thread 624183			4.74	5.83	81.27	1.83	0.03	5.58
○Thread 624184			4.72	5.83	80.92	1.83	0.02	5.57
○Thread 624185			4.79	5.83	82.04	1.83	0.02	5.58
○Thread 624186			4.76	5.83	81.60	1.83	0.02	5.58
○Thread 624187			4.75	5.83	81.43	1.83	0.02	5.58
○Thread 624188			4.69	5.83	80.40	1.83	0.02	5.58
○Thread 624189			4.68	5.83	80.31	1.83	0.02	5.58
○Thread 624190			4.73	5.83	80.99	1.83	0.02	5.58
○Thread 624191			4.76	5.83	81.59	1.83	0.02	5.55
○Thread 624192			4.76	5.83	81.58	1.83	0.02	5.58
○Thread 624193			4.80	5.83	82.27	1.83	0.02	5.58
○Thread 624194			4.68	5.83	80.21	1.83	0.02	5.58
○Thread 624195			4.62	5.84	79.18	1.83	0.02	5.57
○Thread 624196			4.74	5.84	81.31	1.83	0.02	5.58
○Thread 624197			4.67	5.84	80.03	1.83	0.02	5.58
○Thread 624198			4.65	5.84	79.68	1.83	0.02	5.58
○Thread 624199			4.68	5.84	80.19	1.83	0.02	5.58
○Thread 624200			4.61	5.84	78.99	1.83	0.02	5.58
○Thread 624201			4.53	5.84	77.61	1.83	0.02	5.58
○Thread 624202			4.58	5.84	78.47	1.83	0.02	5.58
○Thread 624203			4.70	5.84	80.44	1.83	0.02	5.58
○Thread 624204			4.65	5.84	79.75	1.83	0.02	5.58
○Thread 624205			4.65	5.84	79.66	1.83	0.02	5.58
○Thread 624206			4.69	5.84	80.43	1.83	0.02	5.58
○Thread 624207			4.67	5.84	80.08	1.83	0.02	5.57
○Thread 624208			4.70	5.84	80.51	1.83	0.02	5.58
○Thread 624209			4.70	5.84	80.50	1.83	0.02	5.57
○Thread 624210			4.73	5.84	81.01	1.83	0.02	5.58
○Thread 624211			4.59	5.84	78.70	1.83	0.02	5.56
○Thread 624212			4.83	5.84	82.72	1.83	0.01	5.58
○Thread 624213			4.74	5.84	81.18	1.83	0.01	5.58
○Thread 624214			4.65	5.84	79.63	1.83	0.01	5.58
○Thread 624215			4.71	5.84	80.74	1.83	0.01	5.57
○Thread 624216			4.85	5.84	83.05	1.83	0.01	5.58
○Thread 624217			4.70	5.84	80.56	1.83	0.01	5.58
○Thread 624218			4.73	5.84	81.07	1.83	0.01	5.58
○Thread 624219			4.72	5.84	80.90	1.83	0.01	5.58
○Thread 624220			4.65	5.84	79.70	1.83	0.01	5.58
○Thread 624221			4.75	5.84	81.32	1.83	0.01	5.58
○Thread 624222			4.57	5.84	78.24	1.83	0.01	5.58
○Thread 624223			4.65	5.84	79.69	1.83	0.01	5.58
○Thread 624224			4.64	5.84	79.52	1.83	0.01	5.57
○Thread 624225			4.70	5.84	80.54	1.83	0.01	5.57
○Thread 624226			4.57	5.84	78.31	1.83	0.01	5.57
○Thread 624227			4.60	5.84	78.82	1.83	0.01	5.58
○Thread 624228			4.68	5.84	80.10	1.83	0.01	5.55
○Thread 624229			4.64	5.84	79.50	1.83	0.01	5.58
○Thread 624230			4.63	5.84	79.24	1.83	0.01	5.58
○Thread 624231			4.74	5.84	81.12	1.83	0.01	5.58
○Thread 624232			4.93	5.84	84.45	1.83	0.01	5.58
○Thread 624233			4.68	5.84	80.17	1.83	0.01	5.56
○Thread 624234			4.66	5.84	79.83	1.83	0.01	5.58
○Thread 624235			4.76	5.84	81.54	1.83	0.01	5.58
○Thread 624236			4.72	5.84	80.76	1.83	0.01	5.57
○Thread 624237			4.67	5.84	79.99	1.83	0.01	5.58
○Thread 624238			4.72	5.84	80.76	1.83	0.01	5.58
○Thread 624239			4.68	5.84	80.15	1.83	0.01	5.57
○Thread 624240			4.69	5.85	80.24	1.83	0.00	5.58
○Thread 624241			4.71	5.85	80.58	1.83	0.00	5.58
○Thread 624242			4.70	5.85	80.40	1.83	0.00	5.57
○Thread 624243			4.83	5.85	82.62	1.83	0.00	5.58
○Thread 624244			5.19	5.85	88.78	1.83	0.00	5.58
○Thread 624245			4.63	5.85	79.28	1.83	0.00	5.57
○Thread 624246			4.96	5.85	84.92	1.83	0.00	5.58
○Thread 624247			5.08	5.85	86.88	1.83	0.00	5.58
○Thread 624248			4.82	5.85	82.43	1.83	0.00	5.58
○Thread 624249			4.52	5.85	77.30	1.83	0.00	5.58
○Thread 624250			4.53	5.85	77.55	1.83	0.00	5.58
○Thread 624251			4.45	5.85	76.18	1.84	0.00	5.57

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	5.27
►Process 624271+		80	5.27
○Thread 624271			5.27	7.54	69.84	0.00	0.00	5.67
○Thread 624370			4.51	5.69	79.21	1.82	0.03	5.44
○Thread 624371			4.62	5.69	81.23	1.82	0.03	5.45
○Thread 624372			4.55	5.69	79.90	1.82	0.03	5.44
○Thread 624373			4.42	5.69	77.62	1.82	0.03	5.45
○Thread 624374			4.56	5.69	80.07	1.82	0.03	5.45
○Thread 624375			4.49	5.69	78.93	1.82	0.03	5.44
○Thread 624376			4.47	5.70	78.49	1.82	0.03	5.43
○Thread 624377			4.58	5.70	80.50	1.82	0.03	5.44
○Thread 624378			4.61	5.70	81.03	1.82	0.03	5.41
○Thread 624379			4.47	5.70	78.48	1.82	0.03	5.45
○Thread 624380			4.52	5.70	79.35	1.82	0.03	5.44
○Thread 624381			4.61	5.70	80.93	1.82	0.03	5.44
○Thread 624382			4.52	5.70	79.35	1.82	0.03	5.44
○Thread 624383			4.48	5.70	78.64	1.82	0.02	5.44
○Thread 624384			4.52	5.70	79.34	1.82	0.02	5.43
○Thread 624385			4.44	5.70	78.02	1.82	0.02	5.44
○Thread 624386			4.42	5.70	77.67	1.82	0.02	5.45
○Thread 624387			4.47	5.70	78.45	1.82	0.02	5.43
○Thread 624388			4.43	5.70	77.75	1.82	0.02	5.44
○Thread 624389			4.42	5.70	77.66	1.82	0.02	5.44
○Thread 624390			4.35	5.70	76.34	1.82	0.02	5.44
○Thread 624391			4.34	5.70	76.16	1.82	0.02	5.44
○Thread 624392			4.48	5.70	78.70	1.82	0.02	5.43
○Thread 624393			4.42	5.70	77.64	1.82	0.02	5.43
○Thread 624394			4.47	5.70	78.43	1.82	0.02	5.44
○Thread 624395			4.46	5.70	78.34	1.82	0.02	5.44
○Thread 624396			4.50	5.70	78.95	1.82	0.02	5.44
○Thread 624397			4.47	5.70	78.51	1.82	0.02	5.44
○Thread 624398			4.44	5.70	77.98	1.82	0.02	5.44
○Thread 624399			4.48	5.70	78.59	1.82	0.02	5.44
○Thread 624400			4.67	5.70	82.01	1.82	0.02	5.44
○Thread 624401			4.45	5.70	78.15	1.82	0.02	5.44
○Thread 624402			4.44	5.70	77.88	1.82	0.02	5.44
○Thread 624403			4.47	5.70	78.40	1.82	0.02	5.44
○Thread 624404			4.40	5.70	77.17	1.82	0.02	5.43
○Thread 624405			4.34	5.70	76.21	1.82	0.02	5.41
○Thread 624406			4.33	5.70	75.94	1.82	0.02	5.43
○Thread 624407			4.51	5.70	79.09	1.82	0.02	5.44
○Thread 624408			4.38	5.70	76.90	1.82	0.02	5.44
○Thread 624409			4.44	5.70	77.86	1.82	0.02	5.42
○Thread 624410			4.32	5.70	75.84	1.82	0.02	5.43
○Thread 624411			4.43	5.70	77.68	1.82	0.01	5.44
○Thread 624412			4.37	5.70	76.62	1.82	0.01	5.44
○Thread 624413			4.38	5.70	76.79	1.82	0.01	5.38
○Thread 624414			4.41	5.70	77.41	1.82	0.01	5.43
○Thread 624415			4.48	5.70	78.63	1.82	0.01	5.44
○Thread 624416			4.39	5.70	77.05	1.82	0.01	5.44
○Thread 624417			4.62	5.70	81.08	1.82	0.01	5.44
○Thread 624418			4.49	5.71	78.70	1.82	0.01	5.40
○Thread 624419			4.43	5.71	77.65	1.82	0.01	5.44
○Thread 624420			4.42	5.71	77.55	1.82	0.01	5.43
○Thread 624421			4.29	5.71	75.27	1.82	0.01	5.44
○Thread 624422			4.48	5.71	78.60	1.82	0.01	5.44
○Thread 624423			4.50	5.71	78.95	1.82	0.01	5.43
○Thread 624424			4.31	5.71	75.53	1.82	0.01	5.40
○Thread 624425			4.41	5.71	77.28	1.82	0.01	5.44
○Thread 624426			4.49	5.71	78.76	1.82	0.01	5.44
○Thread 624427			4.48	5.71	78.50	1.82	0.01	5.44
○Thread 624428			4.39	5.71	77.00	1.82	0.01	5.43
○Thread 624429			4.58	5.71	80.33	1.82	0.01	5.40
○Thread 624430			4.42	5.71	77.44	1.82	0.01	5.43
○Thread 624431			4.55	5.71	79.62	1.82	0.01	5.44
○Thread 624432			4.50	5.71	78.83	1.82	0.01	5.44
○Thread 624433			5.01	5.71	87.76	1.82	0.01	5.43
○Thread 624434			4.55	5.71	79.79	1.82	0.01	5.44
○Thread 624435			4.56	5.71	79.87	1.82	0.01	5.44
○Thread 624436			4.78	5.71	83.72	1.82	0.01	5.42
○Thread 624437			4.46	5.71	78.11	1.82	0.00	5.43
○Thread 624438			4.41	5.71	77.24	1.82	0.00	5.44
○Thread 624439			4.47	5.71	78.28	1.82	0.00	5.43
○Thread 624440			4.52	5.71	79.24	1.82	0.00	5.43
○Thread 624441			4.54	5.71	79.50	1.82	0.00	5.44
○Thread 624442			4.48	5.71	78.54	1.82	0.00	5.40
○Thread 624443			4.41	5.71	77.22	1.82	0.00	5.42
○Thread 624444			4.49	5.71	78.62	1.83	0.00	5.44
○Thread 624445			4.55	5.71	79.66	1.83	0.00	5.43
○Thread 624446			4.48	5.71	78.52	1.83	0.00	5.44
○Thread 624447			4.56	5.71	79.83	1.83	0.00	5.44
○Thread 624448			4.24	5.71	74.23	1.83	0.00	5.45

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	5.25
►Process 624469+		88	5.25
○Thread 624469			5.25	7.56	69.34	0.00	0.00	5.71
○Thread 624568			4.56	5.71	79.85	1.82	0.03	5.46
○Thread 624569			4.47	5.71	78.27	1.82	0.03	5.46
○Thread 624570			4.40	5.71	77.04	1.82	0.03	5.46
○Thread 624571			4.48	5.71	78.53	1.82	0.03	5.45
○Thread 624572			4.49	5.71	78.70	1.82	0.03	5.46
○Thread 624573			4.45	5.71	77.91	1.82	0.03	5.44
○Thread 624574			4.38	5.71	76.77	1.82	0.03	5.46
○Thread 624575			4.54	5.71	79.56	1.82	0.03	5.46
○Thread 624576			4.48	5.71	78.51	1.82	0.03	5.44
○Thread 624577			4.39	5.71	76.84	1.82	0.03	5.46
○Thread 624578			4.37	5.71	76.49	1.82	0.03	5.45
○Thread 624579			4.47	5.71	78.33	1.82	0.03	5.46
○Thread 624580			4.39	5.71	76.92	1.82	0.03	5.46
○Thread 624581			4.50	5.71	78.84	1.82	0.03	5.46
○Thread 624582			4.42	5.71	77.44	1.82	0.03	5.46
○Thread 624583			4.43	5.71	77.61	1.82	0.03	5.45
○Thread 624584			4.44	5.71	77.78	1.82	0.03	5.45
○Thread 624585			4.48	5.71	78.48	1.82	0.03	5.46
○Thread 624586			4.43	5.71	77.52	1.82	0.03	5.44
○Thread 624587			4.60	5.72	80.57	1.82	0.03	5.45
○Thread 624588			4.46	5.72	78.03	1.82	0.02	5.45
○Thread 624589			4.48	5.72	78.47	1.82	0.02	5.45
○Thread 624590			4.29	5.72	75.05	1.82	0.02	5.45
○Thread 624591			4.36	5.72	76.36	1.82	0.02	5.45
○Thread 624592			4.37	5.72	76.45	1.82	0.02	5.45
○Thread 624593			4.40	5.72	77.06	1.82	0.02	5.46
○Thread 624594			4.33	5.72	75.74	1.82	0.02	5.45
○Thread 624595			4.39	5.72	76.79	1.82	0.02	5.46
○Thread 624596			4.38	5.72	76.70	1.82	0.02	5.45
○Thread 624597			4.46	5.72	78.09	1.82	0.02	5.46
○Thread 624598			4.39	5.72	76.78	1.82	0.02	5.46
○Thread 624599			4.55	5.72	79.66	1.83	0.02	5.46
○Thread 624600			4.41	5.72	77.21	1.83	0.02	5.46
○Thread 624601			4.39	5.72	76.77	1.83	0.02	5.46
○Thread 624602			4.38	5.72	76.59	1.83	0.02	5.46
○Thread 624603			4.47	5.72	78.25	1.83	0.02	5.45
○Thread 624604			4.38	5.72	76.67	1.83	0.02	5.46
○Thread 624605			4.38	5.72	76.58	1.83	0.02	5.46
○Thread 624606			4.39	5.72	76.75	1.83	0.02	5.46
○Thread 624607			4.50	5.72	78.76	1.83	0.02	5.46
○Thread 624608			4.32	5.72	75.61	1.83	0.02	5.46
○Thread 624609			4.49	5.72	78.58	1.83	0.02	5.46
○Thread 624610			4.33	5.72	75.69	1.83	0.02	5.46
○Thread 624611			4.37	5.72	76.39	1.83	0.02	5.46
○Thread 624612			4.45	5.72	77.87	1.83	0.02	5.46
○Thread 624613			4.34	5.72	75.86	1.83	0.02	5.46
○Thread 624614			4.42	5.72	77.34	1.83	0.02	5.45
○Thread 624615			4.49	5.72	78.47	1.83	0.02	5.46
○Thread 624616			4.43	5.72	77.51	1.83	0.01	5.46
○Thread 624617			4.34	5.72	75.84	1.83	0.01	5.46
○Thread 624618			4.36	5.72	76.28	1.83	0.01	5.45
○Thread 624619			4.43	5.72	77.50	1.83	0.01	5.46
○Thread 624620			4.39	5.72	76.71	1.83	0.01	5.45
○Thread 624621			4.42	5.72	77.23	1.83	0.01	5.46
○Thread 624622			4.61	5.72	80.63	1.83	0.01	5.46
○Thread 624623			4.58	5.72	80.11	1.83	0.01	5.46
○Thread 624624			4.42	5.72	77.31	1.83	0.01	5.46
○Thread 624625			4.43	5.72	77.48	1.83	0.01	5.45
○Thread 624626			4.43	5.72	77.48	1.83	0.01	5.46
○Thread 624627			4.30	5.72	75.20	1.83	0.01	5.46
○Thread 624628			4.41	5.72	77.12	1.83	0.01	5.46
○Thread 624629			4.45	5.72	77.73	1.83	0.01	5.46
○Thread 624630			4.33	5.73	75.63	1.83	0.01	5.46
○Thread 624631			4.63	5.73	80.87	1.83	0.01	5.45
○Thread 624632			4.33	5.73	75.62	1.83	0.01	5.45
○Thread 624633			4.36	5.73	76.14	1.83	0.01	5.46
○Thread 624634			4.44	5.73	77.54	1.83	0.01	5.46
○Thread 624635			4.40	5.73	76.84	1.83	0.01	5.46
○Thread 624636			4.31	5.73	75.26	1.83	0.01	5.46
○Thread 624637			4.46	5.73	77.88	1.83	0.01	5.46
○Thread 624638			4.35	5.73	76.04	1.83	0.01	5.46
○Thread 624639			4.45	5.73	77.78	1.83	0.01	5.44
○Thread 624640			4.28	5.73	74.73	1.83	0.01	5.45
○Thread 624641			4.40	5.73	76.82	1.83	0.01	5.46
○Thread 624642			4.37	5.73	76.38	1.83	0.01	5.46
○Thread 624643			4.82	5.73	84.14	1.83	0.00	5.46
○Thread 624644			4.43	5.73	77.33	1.83	0.00	5.45
○Thread 624645			4.49	5.73	78.38	1.83	0.00	5.46
○Thread 624646			4.46	5.73	77.85	1.83	0.00	5.46
○Thread 624647			4.56	5.73	79.59	1.83	0.00	5.45
○Thread 624648			4.31	5.73	75.22	1.83	0.00	5.46
○Thread 624649			4.37	5.73	76.27	1.83	0.00	5.46
○Thread 624650			4.46	5.73	77.83	1.83	0.00	5.46
○Thread 624651			4.42	5.73	77.13	1.83	0.00	5.46
○Thread 624652			4.34	5.73	75.82	1.83	0.00	5.46
○Thread 624653			4.19	5.73	73.11	1.83	0.00	5.46
○Thread 624654			4.17	5.73	72.76	1.83	0.00	5.47

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	5.08
►Process 624724+		96	5.08
○Thread 624724			5.08	7.60	66.79	0.00	0.00	5.71
○Thread 624824			4.49	5.73	78.50	1.84	0.04	5.48
○Thread 624825			4.33	5.73	75.61	1.84	0.04	5.48
○Thread 624826			4.41	5.73	77.01	1.84	0.04	5.48
○Thread 624827			4.46	5.73	77.97	1.84	0.03	5.47
○Thread 624828			4.38	5.73	76.57	1.84	0.03	5.48
○Thread 624829			4.38	5.73	76.48	1.84	0.03	5.48
○Thread 624830			4.34	5.73	75.77	1.84	0.03	5.46
○Thread 624831			4.33	5.73	75.68	1.84	0.03	5.48
○Thread 624832			4.42	5.73	77.24	1.84	0.03	5.47
○Thread 624833			4.28	5.73	74.80	1.84	0.03	5.48
○Thread 624834			4.35	5.73	75.93	1.84	0.03	5.48
○Thread 624835			4.43	5.73	77.41	1.84	0.03	5.48
○Thread 624836			4.39	5.73	76.62	1.84	0.03	5.48
○Thread 624837			4.31	5.73	75.22	1.84	0.03	5.47
○Thread 624838			4.31	5.73	75.22	1.84	0.03	5.47
○Thread 624839			4.28	5.73	74.69	1.84	0.03	5.48
○Thread 624840			4.38	5.73	76.43	1.84	0.03	5.48
○Thread 624841			4.23	5.73	73.81	1.84	0.03	5.48
○Thread 624842			4.24	5.73	73.98	1.84	0.03	5.48
○Thread 624843			4.24	5.73	74.07	1.84	0.03	5.48
○Thread 624844			4.34	5.73	75.81	1.84	0.03	5.48
○Thread 624845			4.21	5.73	73.54	1.84	0.03	5.48
○Thread 624846			4.39	5.73	76.67	1.84	0.03	5.47
○Thread 624847			4.21	5.73	73.53	1.84	0.03	5.47
○Thread 624848			4.39	5.73	76.67	1.84	0.03	5.48
○Thread 624849			4.30	5.73	75.10	1.84	0.03	5.47
○Thread 624850			4.37	5.73	76.23	1.84	0.03	5.48
○Thread 624851			4.31	5.73	75.18	1.84	0.03	5.48
○Thread 624852			4.32	5.73	75.35	1.84	0.03	5.48
○Thread 624853			4.22	5.73	73.60	1.84	0.02	5.48
○Thread 624854			4.26	5.73	74.30	1.84	0.02	5.46
○Thread 624855			4.28	5.73	74.64	1.84	0.02	5.48
○Thread 624856			4.28	5.73	74.72	1.84	0.02	5.47
○Thread 624857			4.22	5.73	73.68	1.84	0.02	5.48
○Thread 624858			4.37	5.73	76.29	1.84	0.02	5.47
○Thread 624859			4.21	5.74	73.50	1.84	0.02	5.48
○Thread 624860			4.34	5.74	75.76	1.84	0.02	5.48
○Thread 624861			4.24	5.74	74.01	1.84	0.02	5.47
○Thread 624862			4.38	5.74	76.36	1.84	0.02	5.48
○Thread 624863			4.34	5.74	75.75	1.84	0.02	5.47
○Thread 624864			4.42	5.74	77.05	1.84	0.02	5.48
○Thread 624865			4.29	5.74	74.87	1.84	0.02	5.48
○Thread 624866			4.20	5.74	73.30	1.84	0.02	5.47
○Thread 624867			4.48	5.74	78.18	1.84	0.02	5.48
○Thread 624868			4.32	5.74	75.39	1.84	0.02	5.47
○Thread 624869			4.33	5.74	75.56	1.84	0.02	5.48
○Thread 624870			4.30	5.74	74.94	1.84	0.02	5.48
○Thread 624871			4.83	5.74	84.26	1.84	0.02	5.48
○Thread 624872			4.34	5.74	75.63	1.84	0.02	5.48
○Thread 624873			4.42	5.74	77.11	1.84	0.02	5.48
○Thread 624874			4.27	5.74	74.50	1.84	0.02	5.47
○Thread 624875			4.36	5.74	75.97	1.84	0.02	5.48
○Thread 624876			4.31	5.74	75.19	1.84	0.02	5.48
○Thread 624877			4.23	5.74	73.79	1.84	0.02	5.48
○Thread 624878			4.29	5.74	74.74	1.84	0.02	5.46
○Thread 624879			4.52	5.74	78.75	1.84	0.02	5.47
○Thread 624880			4.53	5.74	78.92	1.84	0.01	5.48
○Thread 624881			4.31	5.74	75.17	1.84	0.01	5.48
○Thread 624882			4.20	5.74	73.16	1.84	0.01	5.47
○Thread 624883			4.60	5.74	80.13	1.84	0.01	5.47
○Thread 624884			4.33	5.74	75.51	1.84	0.01	5.48
○Thread 624885			4.32	5.74	75.33	1.84	0.01	5.48
○Thread 624886			4.34	5.74	75.59	1.84	0.01	5.48
○Thread 624887			4.34	5.74	75.59	1.84	0.01	5.48
○Thread 624888			4.31	5.74	75.15	1.84	0.01	5.48
○Thread 624889			4.23	5.74	73.75	1.84	0.01	5.47
○Thread 624890			4.21	5.74	73.40	1.85	0.01	5.47
○Thread 624891			4.30	5.74	74.88	1.85	0.01	5.48
○Thread 624892			4.36	5.74	76.01	1.85	0.01	5.48
○Thread 624893			4.38	5.74	76.27	1.85	0.01	5.48
○Thread 624894			4.35	5.74	75.74	1.85	0.01	5.48
○Thread 624895			4.45	5.74	77.48	1.85	0.01	5.48
○Thread 624896			4.30	5.74	74.95	1.85	0.01	5.48
○Thread 624897			4.33	5.74	75.38	1.85	0.01	5.48
○Thread 624898			4.24	5.74	73.90	1.85	0.01	5.48
○Thread 624899			4.45	5.74	77.47	1.85	0.01	5.46
○Thread 624900			4.34	5.74	75.55	1.85	0.01	5.47
○Thread 624901			4.31	5.74	75.11	1.85	0.01	5.48
○Thread 624902			4.34	5.75	75.63	1.85	0.01	5.48
○Thread 624903			4.48	5.75	77.98	1.85	0.01	5.48
○Thread 624904			4.31	5.75	75.10	1.85	0.01	5.48
○Thread 624905			4.29	5.75	74.75	1.85	0.01	5.48
○Thread 624906			4.22	5.75	73.44	1.85	0.01	5.47
○Thread 624907			4.28	5.75	74.57	1.85	0.01	5.48
○Thread 624908			4.30	5.75	74.83	1.85	0.00	5.48
○Thread 624909			4.30	5.75	74.91	1.85	0.00	5.48
○Thread 624910			4.19	5.75	72.91	1.85	0.00	5.48
○Thread 624911			4.40	5.75	76.64	1.85	0.00	5.46
○Thread 624912			4.34	5.75	75.51	1.85	0.00	5.48
○Thread 624913			4.25	5.75	74.03	1.85	0.00	5.48
○Thread 624914			4.12	5.75	71.76	1.85	0.00	5.48
○Thread 624915			4.25	5.75	74.02	1.85	0.00	5.48
○Thread 624916			4.15	5.75	72.28	1.85	0.00	5.48
○Thread 624917			3.98	5.75	69.23	1.85	0.00	5.48
○Thread 624918			4.13	5.75	71.92	1.85	0.00	5.47

Report Configuration