OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	209.44
►Process 316061+		1	209.44
○Thread 316061			209.44	211.25	99.15	0.00	0.00	211.24

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	105.57
►Process 316251+		2	105.57
○Thread 316251			105.57	107.44	98.26	0.00	0.00	105.12
○Thread 316318			104.70	105.12	99.60	2.32	0.00	104.85

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	53.48
►Process 316389+		4	53.48
○Thread 316389			53.48	55.37	96.57	0.00	0.00	53.07
○Thread 316457			52.66	53.08	99.20	2.29	0.00	52.80
○Thread 316458			52.69	53.08	99.26	2.29	0.00	52.81
○Thread 316459			52.66	53.08	99.20	2.29	0.00	52.81

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	27.69
►Process 316529+		8	27.69
○Thread 316529			27.69	29.63	93.45	0.00	0.00	27.31
○Thread 316596			26.85	27.31	98.30	2.32	0.00	27.04
○Thread 316597			26.89	27.31	98.46	2.32	0.00	27.04
○Thread 316598			26.81	27.31	98.17	2.32	0.00	27.04
○Thread 316599			26.85	27.32	98.30	2.32	0.00	27.04
○Thread 316600			26.87	27.32	98.37	2.32	0.00	27.04
○Thread 316601			26.90	27.32	98.48	2.32	0.00	27.04
○Thread 316602			26.87	27.32	98.37	2.32	0.00	27.04

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	15.01
►Process 316623+		16	15.01
○Thread 316623			15.01	16.97	88.42	0.00	0.00	14.65
○Thread 316690			14.05	14.65	95.88	2.31	0.00	14.38
○Thread 316691			14.13	14.65	96.39	2.31	0.00	14.38
○Thread 316692			14.11	14.65	96.29	2.31	0.00	14.39
○Thread 316693			14.08	14.65	96.05	2.31	0.00	14.38
○Thread 316694			14.06	14.65	95.94	2.31	0.00	14.38
○Thread 316695			14.08	14.65	96.08	2.31	0.00	14.39
○Thread 316696			14.08	14.65	96.08	2.31	0.00	14.38
○Thread 316697			14.01	14.65	95.60	2.31	0.00	14.38
○Thread 316698			14.06	14.65	95.94	2.31	0.00	14.39
○Thread 316699			14.06	14.66	95.90	2.31	0.00	14.38
○Thread 316700			14.16	14.66	96.62	2.31	0.00	14.38
○Thread 316701			14.10	14.66	96.18	2.31	0.00	14.38
○Thread 316702			14.06	14.66	95.94	2.31	0.00	14.38
○Thread 316703			14.17	14.66	96.65	2.31	0.00	14.38
○Thread 316704			13.97	14.66	95.29	2.31	0.00	14.39

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	11.61
►Process 316772+		24	11.61
○Thread 316772			11.61	13.71	84.69	0.00	0.00	11.39
○Thread 316839			10.74	11.40	94.18	2.30	0.01	11.12
○Thread 316840			10.77	11.40	94.48	2.30	0.01	11.12
○Thread 316841			10.78	11.40	94.57	2.30	0.01	11.12
○Thread 316842			10.72	11.40	94.00	2.30	0.01	11.12
○Thread 316843			10.70	11.40	93.82	2.30	0.01	11.12
○Thread 316844			10.63	11.40	93.21	2.30	0.01	11.12
○Thread 316845			10.74	11.40	94.21	2.30	0.01	11.13
○Thread 316846			10.72	11.40	93.99	2.30	0.00	11.12
○Thread 316847			10.72	11.40	94.04	2.30	0.00	11.12
○Thread 316848			10.75	11.40	94.30	2.30	0.00	11.12
○Thread 316849			10.68	11.40	93.64	2.30	0.00	11.12
○Thread 316850			10.82	11.40	94.91	2.30	0.00	11.13
○Thread 316851			10.75	11.40	94.25	2.30	0.00	11.12
○Thread 316852			10.71	11.40	93.90	2.30	0.00	11.13
○Thread 316853			10.76	11.40	94.33	2.30	0.00	11.13
○Thread 316854			10.80	11.40	94.68	2.31	0.00	11.13
○Thread 316855			10.66	11.40	93.45	2.31	0.00	11.12
○Thread 316856			10.72	11.40	93.98	2.31	0.00	11.12
○Thread 316857			10.71	11.40	93.93	2.31	0.00	11.13
○Thread 316858			10.73	11.40	94.11	2.31	0.00	11.12
○Thread 316859			10.64	11.40	93.27	2.31	0.00	11.12
○Thread 316860			10.72	11.40	93.97	2.31	0.00	11.13
○Thread 316861			10.37	11.40	90.90	2.31	0.00	11.11

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	9.57
►Process 316882+		32	9.57
○Thread 316882			9.57	11.74	81.52	0.00	0.00	9.47
○Thread 316949			8.78	9.48	92.56	2.25	0.01	9.21
○Thread 316950			8.77	9.48	92.50	2.25	0.01	9.20
○Thread 316951			8.78	9.48	92.61	2.25	0.01	9.21
○Thread 316952			8.81	9.48	92.87	2.25	0.01	9.21
○Thread 316953			8.71	9.48	91.86	2.25	0.01	9.21
○Thread 316954			8.69	9.48	91.65	2.25	0.01	9.20
○Thread 316955			8.74	9.48	92.12	2.25	0.01	9.20
○Thread 316956			8.83	9.48	93.12	2.25	0.01	9.21
○Thread 316957			8.67	9.48	91.44	2.25	0.01	9.21
○Thread 316958			8.74	9.48	92.17	2.25	0.01	9.20
○Thread 316959			8.76	9.48	92.33	2.25	0.01	9.20
○Thread 316960			8.78	9.48	92.59	2.25	0.01	9.21
○Thread 316961			8.77	9.48	92.48	2.25	0.01	9.20
○Thread 316962			8.73	9.48	92.06	2.25	0.01	9.21
○Thread 316963			8.76	9.48	92.32	2.25	0.01	9.21
○Thread 316964			8.86	9.48	93.37	2.25	0.00	9.21
○Thread 316965			8.70	9.48	91.74	2.25	0.00	9.20
○Thread 316966			8.74	9.48	92.11	2.25	0.00	9.21
○Thread 316967			8.70	9.48	91.74	2.25	0.00	9.21
○Thread 316968			8.71	9.48	91.79	2.25	0.00	9.20
○Thread 316969			8.69	9.48	91.57	2.25	0.00	9.20
○Thread 316970			8.66	9.48	91.31	2.25	0.00	9.20
○Thread 316971			8.64	9.48	91.10	2.25	0.00	9.20
○Thread 316972			8.75	9.48	92.25	2.25	0.00	9.20
○Thread 316973			8.79	9.48	92.67	2.25	0.00	9.20
○Thread 316974			8.82	9.48	92.94	2.25	0.00	9.20
○Thread 316975			8.68	9.49	91.51	2.25	0.00	9.21
○Thread 316976			8.76	9.49	92.35	2.25	0.00	9.21
○Thread 316977			8.74	9.49	92.14	2.25	0.00	9.20
○Thread 316978			8.70	9.49	91.67	2.25	0.00	9.21
○Thread 316979			8.66	9.49	91.24	2.25	0.00	9.21

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	8.60
►Process 316999+		40	8.60
○Thread 316999			8.60	10.70	80.33	0.00	0.00	8.42
○Thread 317066			7.55	8.44	89.42	2.25	0.01	8.16
○Thread 317067			7.66	8.44	90.72	2.25	0.01	8.16
○Thread 317068			7.70	8.44	91.25	2.25	0.01	8.16
○Thread 317069			7.67	8.44	90.84	2.25	0.01	8.15
○Thread 317070			7.53	8.44	89.24	2.25	0.01	8.16
○Thread 317071			7.62	8.44	90.30	2.25	0.01	8.16
○Thread 317072			7.66	8.44	90.71	2.25	0.01	8.15
○Thread 317073			7.59	8.44	89.88	2.25	0.01	8.16
○Thread 317074			7.58	8.44	89.82	2.25	0.01	8.16
○Thread 317075			7.52	8.44	89.11	2.25	0.01	8.14
○Thread 317076			7.55	8.44	89.46	2.25	0.01	8.16
○Thread 317077			7.68	8.44	91.00	2.25	0.01	8.16
○Thread 317078			7.62	8.44	90.23	2.25	0.01	8.17
○Thread 317079			7.59	8.44	89.93	2.25	0.01	8.16
○Thread 317080			7.61	8.44	90.17	2.25	0.01	8.15
○Thread 317081			7.61	8.44	90.17	2.25	0.01	8.17
○Thread 317082			7.52	8.44	89.04	2.25	0.01	8.16
○Thread 317083			7.56	8.44	89.51	2.25	0.01	8.15
○Thread 317084			7.46	8.44	88.32	2.25	0.01	8.16
○Thread 317085			7.64	8.44	90.45	2.25	0.01	8.17
○Thread 317086			7.58	8.44	89.74	2.25	0.01	8.16
○Thread 317087			7.48	8.44	88.56	2.25	0.01	8.16
○Thread 317088			7.48	8.44	88.55	2.25	0.01	8.15
○Thread 317089			7.47	8.44	88.43	2.25	0.00	8.16
○Thread 317090			7.63	8.44	90.33	2.25	0.00	8.16
○Thread 317091			7.63	8.44	90.33	2.25	0.00	8.16
○Thread 317092			7.54	8.44	89.32	2.25	0.00	8.16
○Thread 317093			7.61	8.44	90.09	2.25	0.00	8.15
○Thread 317094			7.63	8.44	90.38	2.25	0.00	8.15
○Thread 317095			7.47	8.44	88.48	2.25	0.00	8.16
○Thread 317096			7.47	8.44	88.42	2.25	0.00	8.16
○Thread 317097			7.67	8.44	90.85	2.26	0.00	8.16
○Thread 317098			7.58	8.44	89.72	2.26	0.00	8.17
○Thread 317099			7.65	8.44	90.61	2.26	0.00	8.16
○Thread 317100			7.69	8.44	91.02	2.26	0.00	8.16
○Thread 317101			7.64	8.44	90.43	2.26	0.00	8.17
○Thread 317102			7.51	8.44	88.95	2.26	0.00	8.15
○Thread 317103			7.61	8.44	90.07	2.26	0.00	8.16
○Thread 317104			7.30	8.44	86.40	2.26	0.00	8.16

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	7.63
►Process 317174+		48	7.63
○Thread 317174			7.63	10.04	75.94	0.00	0.00	7.77
○Thread 317241			6.78	7.77	87.19	2.26	0.01	7.49
○Thread 317242			6.78	7.77	87.19	2.26	0.01	7.49
○Thread 317243			6.83	7.77	87.83	2.26	0.01	7.49
○Thread 317244			6.74	7.77	86.67	2.26	0.01	7.49
○Thread 317245			6.82	7.77	87.76	2.26	0.01	7.49
○Thread 317246			6.76	7.77	86.99	2.26	0.01	7.49
○Thread 317247			6.91	7.77	88.92	2.26	0.01	7.49
○Thread 317248			6.88	7.77	88.53	2.26	0.01	7.49
○Thread 317249			6.82	7.77	87.69	2.26	0.01	7.49
○Thread 317250			6.79	7.77	87.37	2.26	0.01	7.49
○Thread 317251			6.73	7.77	86.59	2.26	0.01	7.49
○Thread 317252			6.92	7.77	89.04	2.26	0.01	7.49
○Thread 317253			6.83	7.77	87.88	2.26	0.01	7.49
○Thread 317254			6.80	7.77	87.42	2.26	0.01	7.49
○Thread 317255			6.78	7.77	87.17	2.26	0.01	7.49
○Thread 317256			6.87	7.77	88.32	2.26	0.01	7.48
○Thread 317257			6.72	7.77	86.45	2.26	0.01	7.47
○Thread 317258			6.77	7.77	87.10	2.26	0.01	7.49
○Thread 317259			6.78	7.77	87.16	2.26	0.01	7.49
○Thread 317260			6.78	7.77	87.16	2.26	0.01	7.48
○Thread 317261			6.68	7.77	85.87	2.26	0.01	7.48
○Thread 317262			6.75	7.77	86.77	2.26	0.01	7.48
○Thread 317263			6.71	7.77	86.32	2.26	0.01	7.49
○Thread 317264			6.79	7.77	87.28	2.26	0.01	7.49
○Thread 317265			6.67	7.77	85.80	2.26	0.01	7.49
○Thread 317266			6.71	7.77	86.25	2.26	0.01	7.49
○Thread 317267			6.63	7.77	85.21	2.26	0.01	7.49
○Thread 317268			6.72	7.77	86.37	2.26	0.01	7.49
○Thread 317269			6.73	7.77	86.50	2.26	0.01	7.49
○Thread 317270			6.71	7.77	86.30	2.26	0.01	7.49
○Thread 317271			6.75	7.78	86.75	2.26	0.01	7.47
○Thread 317272			6.85	7.78	88.10	2.26	0.00	7.48
○Thread 317273			6.79	7.78	87.26	2.26	0.00	7.49
○Thread 317274			6.79	7.78	87.32	2.26	0.00	7.49
○Thread 317275			6.78	7.78	87.13	2.26	0.00	7.49
○Thread 317276			6.84	7.78	87.96	2.26	0.00	7.49
○Thread 317277			6.78	7.78	87.19	2.26	0.00	7.49
○Thread 317278			6.76	7.78	86.93	2.26	0.00	7.49
○Thread 317279			6.85	7.78	88.09	2.26	0.00	7.49
○Thread 317280			6.68	7.78	85.90	2.26	0.00	7.49
○Thread 317281			6.72	7.78	86.35	2.26	0.00	7.49
○Thread 317282			6.77	7.78	86.99	2.26	0.00	7.49
○Thread 317283			6.80	7.78	87.37	2.26	0.00	7.49
○Thread 317284			6.98	7.78	89.75	2.26	0.00	7.49
○Thread 317285			6.81	7.78	87.56	2.26	0.00	7.49
○Thread 317286			6.57	7.78	84.47	2.26	0.00	7.49
○Thread 317287			6.34	7.78	81.45	2.26	0.00	7.49

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	7.31
►Process 317308+		56	7.31
○Thread 317308			7.31	9.55	76.52	0.00	0.00	7.28
○Thread 317375			6.40	7.29	87.77	2.25	0.02	7.01
○Thread 317376			6.27	7.29	85.92	2.25	0.02	7.01
○Thread 317377			6.36	7.29	87.15	2.25	0.01	7.00
○Thread 317378			6.22	7.29	85.23	2.25	0.01	7.01
○Thread 317379			6.19	7.29	84.89	2.25	0.01	7.00
○Thread 317380			6.28	7.29	86.12	2.25	0.01	7.01
○Thread 317381			6.31	7.29	86.53	2.25	0.01	7.01
○Thread 317382			6.26	7.29	85.84	2.25	0.01	7.00
○Thread 317383			6.29	7.29	86.18	2.25	0.01	7.01
○Thread 317384			6.26	7.29	85.84	2.25	0.01	7.01
○Thread 317385			6.37	7.29	87.34	2.25	0.01	7.01
○Thread 317386			6.24	7.29	85.49	2.25	0.01	7.01
○Thread 317387			6.15	7.29	84.25	2.25	0.01	7.01
○Thread 317388			6.26	7.29	85.76	2.25	0.01	7.01
○Thread 317389			6.29	7.29	86.17	2.25	0.01	7.01
○Thread 317390			6.27	7.29	85.96	2.25	0.01	7.01
○Thread 317391			6.19	7.29	84.86	2.25	0.01	7.01
○Thread 317392			6.17	7.29	84.52	2.25	0.01	7.01
○Thread 317393			6.10	7.29	83.63	2.25	0.01	7.01
○Thread 317394			6.18	7.29	84.72	2.25	0.01	7.01
○Thread 317395			6.18	7.29	84.65	2.25	0.01	7.02
○Thread 317396			6.34	7.29	86.91	2.25	0.01	7.01
○Thread 317397			6.31	7.30	86.43	2.25	0.01	7.00
○Thread 317398			6.32	7.30	86.56	2.25	0.01	7.01
○Thread 317399			6.05	7.30	82.93	2.25	0.01	7.01
○Thread 317400			6.24	7.30	85.53	2.25	0.01	7.02
○Thread 317401			6.13	7.30	84.02	2.25	0.01	7.02
○Thread 317402			6.30	7.30	86.35	2.25	0.01	7.02
○Thread 317403			6.23	7.30	85.39	2.25	0.01	7.01
○Thread 317404			6.32	7.30	86.62	2.25	0.01	7.01
○Thread 317405			6.25	7.30	85.66	2.25	0.01	7.01
○Thread 317406			6.35	7.30	86.96	2.25	0.01	7.02
○Thread 317407			6.21	7.30	85.04	2.25	0.01	7.02
○Thread 317408			6.20	7.30	84.97	2.25	0.01	7.01
○Thread 317409			6.23	7.30	85.31	2.25	0.01	7.01
○Thread 317410			6.19	7.30	84.83	2.25	0.01	7.00
○Thread 317411			6.23	7.30	85.37	2.25	0.01	7.01
○Thread 317412			6.14	7.30	84.14	2.25	0.01	7.01
○Thread 317413			6.13	7.30	83.93	2.25	0.01	7.01
○Thread 317414			6.22	7.30	85.16	2.25	0.00	7.02
○Thread 317415			6.17	7.30	84.48	2.25	0.00	7.01
○Thread 317416			6.18	7.30	84.61	2.25	0.00	7.01
○Thread 317417			6.19	7.30	84.82	2.25	0.00	7.01
○Thread 317418			6.28	7.30	85.98	2.25	0.00	7.01
○Thread 317419			6.25	7.30	85.63	2.25	0.00	7.01
○Thread 317420			6.30	7.30	86.32	2.25	0.00	6.99
○Thread 317421			6.22	7.30	85.15	2.25	0.00	7.01
○Thread 317422			6.48	7.30	88.71	2.25	0.00	7.01
○Thread 317423			6.25	7.30	85.56	2.25	0.00	7.01
○Thread 317424			6.26	7.30	85.69	2.25	0.00	7.01
○Thread 317425			6.25	7.30	85.55	2.25	0.00	7.01
○Thread 317426			6.31	7.30	86.37	2.25	0.00	7.01
○Thread 317427			6.24	7.30	85.48	2.25	0.00	6.99
○Thread 317428			5.83	7.30	79.86	2.25	0.00	7.01
○Thread 317429			5.93	7.30	81.16	2.25	0.00	7.00

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	6.97
►Process 317449+		64	6.97
○Thread 317449			6.97	9.26	75.23	0.00	0.00	6.96
○Thread 317516			5.73	6.98	82.01	2.26	0.02	6.70
○Thread 317517			5.81	6.98	83.23	2.26	0.02	6.69
○Thread 317518			5.87	6.98	84.09	2.26	0.02	6.70
○Thread 317519			5.91	6.98	84.66	2.26	0.02	6.70
○Thread 317520			5.86	6.98	83.87	2.26	0.02	6.70
○Thread 317521			5.77	6.98	82.65	2.26	0.02	6.70
○Thread 317522			5.82	6.98	83.29	2.26	0.02	6.70
○Thread 317523			5.90	6.98	84.44	2.26	0.02	6.70
○Thread 317524			5.79	6.98	82.93	2.26	0.02	6.70
○Thread 317525			5.78	6.98	82.79	2.26	0.02	6.70
○Thread 317526			5.92	6.98	84.72	2.26	0.02	6.70
○Thread 317527			5.82	6.98	83.36	2.26	0.01	6.69
○Thread 317528			5.75	6.98	82.35	2.26	0.01	6.71
○Thread 317529			5.84	6.98	83.57	2.26	0.01	6.71
○Thread 317530			5.87	6.98	84.14	2.26	0.01	6.70
○Thread 317531			6.05	6.98	86.57	2.26	0.01	6.70
○Thread 317532			5.69	6.98	81.41	2.26	0.01	6.71
○Thread 317533			5.70	6.98	81.63	2.26	0.01	6.70
○Thread 317534			5.84	6.98	83.63	2.26	0.01	6.70
○Thread 317535			5.81	6.98	83.13	2.26	0.01	6.70
○Thread 317536			5.80	6.98	82.98	2.26	0.01	6.71
○Thread 317537			5.82	6.98	83.34	2.26	0.01	6.70
○Thread 317538			5.70	6.98	81.55	2.26	0.01	6.70
○Thread 317539			5.89	6.98	84.34	2.26	0.01	6.69
○Thread 317540			5.72	6.98	81.83	2.26	0.01	6.70
○Thread 317541			5.77	6.98	82.61	2.26	0.01	6.70
○Thread 317542			5.67	6.98	81.11	2.26	0.01	6.71
○Thread 317543			5.83	6.98	83.40	2.26	0.01	6.71
○Thread 317544			5.84	6.98	83.54	2.26	0.01	6.71
○Thread 317545			5.78	6.98	82.68	2.26	0.01	6.70
○Thread 317546			5.71	6.99	81.67	2.26	0.01	6.70
○Thread 317547			5.94	6.99	84.96	2.26	0.01	6.71
○Thread 317548			5.86	6.99	83.89	2.26	0.01	6.70
○Thread 317549			5.77	6.99	82.60	2.26	0.01	6.70
○Thread 317550			5.78	6.99	82.74	2.26	0.01	6.71
○Thread 317551			5.89	6.99	84.24	2.26	0.01	6.70
○Thread 317552			5.79	6.99	82.88	2.26	0.01	6.71
○Thread 317553			5.77	6.99	82.59	2.26	0.01	6.70
○Thread 317554			5.78	6.99	82.73	2.26	0.01	6.71
○Thread 317555			5.82	6.99	83.23	2.26	0.01	6.69
○Thread 317556			5.80	6.99	83.01	2.26	0.01	6.70
○Thread 317557			5.80	6.99	83.01	2.26	0.01	6.71
○Thread 317558			5.71	6.99	81.72	2.26	0.01	6.70
○Thread 317559			5.77	6.99	82.51	2.26	0.01	6.70
○Thread 317560			5.78	6.99	82.72	2.26	0.01	6.71
○Thread 317561			5.81	6.99	83.07	2.26	0.01	6.71
○Thread 317562			5.79	6.99	82.86	2.27	0.01	6.70
○Thread 317563			5.92	6.99	84.64	2.27	0.00	6.70
○Thread 317564			5.76	6.99	82.35	2.27	0.00	6.70
○Thread 317565			5.78	6.99	82.71	2.27	0.00	6.70
○Thread 317566			5.87	6.99	83.92	2.27	0.00	6.70
○Thread 317567			5.82	6.99	83.35	2.27	0.00	6.70
○Thread 317568			5.74	6.99	82.13	2.27	0.00	6.70
○Thread 317569			5.72	6.99	81.84	2.27	0.00	6.71
○Thread 317570			5.81	6.99	83.13	2.27	0.00	6.70
○Thread 317571			5.79	6.99	82.77	2.27	0.00	6.70
○Thread 317572			5.74	6.99	82.12	2.27	0.00	6.70
○Thread 317573			5.75	6.99	82.26	2.27	0.00	6.70
○Thread 317574			5.84	6.99	83.48	2.27	0.00	6.70
○Thread 317575			5.94	6.99	84.98	2.27	0.00	6.70
○Thread 317576			5.80	6.99	82.90	2.27	0.00	6.69
○Thread 317577			5.75	6.99	82.26	2.27	0.00	6.71
○Thread 317578			5.75	6.99	82.25	2.27	0.00	6.70

Report Configuration