OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	53.20
►Process 3987570+		1	53.20
○Thread 3987570			53.20	54.34	97.90	0.00	0.00	49.81

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	27.14
►Process 3987710+		2	27.14
○Thread 3987710			26.26	28.61	91.78	0.00	0.00	27.61
○Thread 3987777			27.14	27.62	98.27	1.00	0.00	27.43

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	14.36
►Process 3988018+		4	14.36
○Thread 3988018			13.10	15.85	82.63	0.00	0.00	14.84
○Thread 3988085			14.35	14.85	96.63	1.00	0.00	14.68
○Thread 3988086			14.34	14.85	96.57	1.00	0.00	14.68
○Thread 3988087			14.36	14.85	96.67	1.00	0.00	14.68

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	8.74
►Process 3988822+		8	8.74
○Thread 3988822			6.99	10.24	68.20	0.00	0.00	9.23
○Thread 3988889			8.74	9.24	94.49	1.00	0.00	9.07
○Thread 3988890			8.66	9.24	93.68	1.00	0.00	9.07
○Thread 3988891			8.73	9.24	94.38	1.00	0.00	9.07
○Thread 3988892			8.64	9.24	93.46	1.00	0.00	9.07
○Thread 3988893			8.66	9.24	93.62	1.00	0.00	9.07
○Thread 3988894			8.70	9.24	94.11	1.00	0.00	9.07
○Thread 3988895			8.74	9.25	94.48	1.00	0.00	9.07

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	6.41
►Process 3990464+		16	6.41
○Thread 3990464			4.37	8.01	54.65	0.00	0.00	7.00
○Thread 3990531			6.31	7.01	90.04	0.99	0.00	6.83
○Thread 3990532			6.39	7.01	91.25	0.99	0.00	6.82
○Thread 3990533			6.39	7.01	91.18	0.99	0.00	6.83
○Thread 3990534			6.33	7.01	90.32	0.99	0.00	6.83
○Thread 3990535			6.28	7.01	89.61	0.99	0.00	6.83
○Thread 3990536			6.34	7.01	90.39	0.99	0.00	6.83
○Thread 3990537			6.32	7.01	90.17	0.99	0.00	6.83
○Thread 3990538			6.41	7.01	91.45	0.99	0.00	6.82
○Thread 3990539			6.36	7.01	90.74	0.99	0.00	6.83
○Thread 3990540			6.41	7.01	91.38	0.99	0.00	6.83
○Thread 3990541			6.34	7.01	90.52	0.99	0.00	6.83
○Thread 3990542			6.35	7.01	90.59	0.99	0.00	6.83
○Thread 3990543			6.30	7.01	89.80	0.99	0.00	6.82
○Thread 3990544			6.34	7.01	90.37	0.99	0.00	6.83
○Thread 3990545			6.38	7.01	91.01	0.99	0.00	6.83

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	5.77
►Process 3993881+		24	5.77
○Thread 3993881			3.22	7.48	43.05	0.00	0.00	6.47
○Thread 3993948			5.70	6.49	87.79	0.98	0.01	6.31
○Thread 3993949			5.68	6.49	87.48	0.98	0.01	6.31
○Thread 3993950			5.70	6.49	87.78	0.98	0.01	6.31
○Thread 3993951			5.70	6.49	87.78	0.98	0.01	6.31
○Thread 3993952			5.75	6.49	88.55	0.98	0.01	6.31
○Thread 3993953			5.71	6.49	87.93	0.98	0.01	6.31
○Thread 3993954			5.72	6.49	88.01	0.98	0.01	6.31
○Thread 3993955			5.72	6.49	88.08	0.98	0.00	6.31
○Thread 3993956			5.77	6.49	88.77	0.98	0.00	6.31
○Thread 3993957			5.70	6.49	87.85	0.98	0.00	6.31
○Thread 3993958			5.66	6.49	87.15	0.98	0.00	6.31
○Thread 3993959			5.67	6.49	87.30	0.98	0.00	6.31
○Thread 3993960			5.76	6.49	88.69	0.98	0.00	6.31
○Thread 3993961			5.63	6.49	86.68	0.98	0.00	6.31
○Thread 3993962			5.64	6.50	86.91	0.98	0.00	6.31
○Thread 3993963			5.70	6.50	87.83	0.98	0.00	6.31
○Thread 3993964			5.70	6.50	87.68	0.98	0.00	6.31
○Thread 3993965			5.66	6.50	87.14	0.98	0.00	6.31
○Thread 3993966			5.73	6.50	88.21	0.98	0.00	6.31
○Thread 3993967			5.62	6.50	86.52	0.98	0.00	6.31
○Thread 3993968			5.69	6.50	87.67	0.98	0.00	6.31
○Thread 3993969			5.76	6.50	88.59	0.98	0.00	6.31
○Thread 3993970			5.72	6.50	88.05	0.98	0.00	6.31

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	5.60
►Process 3999073+		32	5.60
○Thread 3999073			2.68	7.36	36.40	0.00	0.00	6.34
○Thread 3999140			5.52	6.35	86.89	1.00	0.01	6.17
○Thread 3999141			5.60	6.35	88.07	1.00	0.01	6.17
○Thread 3999142			5.54	6.35	87.20	1.00	0.01	6.16
○Thread 3999143			5.43	6.35	85.39	1.00	0.01	6.17
○Thread 3999144			5.52	6.35	86.89	1.00	0.01	6.17
○Thread 3999145			5.47	6.35	86.02	1.00	0.01	6.17
○Thread 3999146			5.44	6.35	85.62	1.00	0.01	6.17
○Thread 3999147			5.39	6.35	84.76	1.00	0.01	6.16
○Thread 3999148			5.57	6.35	87.59	1.00	0.01	6.17
○Thread 3999149			5.43	6.35	85.46	1.00	0.01	6.16
○Thread 3999150			5.48	6.35	86.24	1.00	0.01	6.16
○Thread 3999151			5.45	6.35	85.69	1.00	0.01	6.16
○Thread 3999152			5.52	6.35	86.79	1.00	0.01	6.17
○Thread 3999153			5.50	6.35	86.55	1.00	0.01	6.17
○Thread 3999154			5.55	6.35	87.26	1.00	0.00	6.17
○Thread 3999155			5.51	6.35	86.63	1.00	0.00	6.17
○Thread 3999156			5.46	6.35	85.92	1.00	0.00	6.17
○Thread 3999157			5.50	6.36	86.47	1.00	0.00	6.17
○Thread 3999158			5.48	6.36	86.31	1.00	0.00	6.17
○Thread 3999159			5.43	6.36	85.44	1.00	0.00	6.16
○Thread 3999160			5.44	6.36	85.60	1.00	0.00	6.16
○Thread 3999161			5.42	6.36	85.20	1.00	0.00	6.17
○Thread 3999162			5.49	6.36	86.30	1.00	0.00	6.17
○Thread 3999163			5.46	6.36	85.90	1.00	0.00	6.17
○Thread 3999164			5.45	6.36	85.75	1.00	0.00	6.17
○Thread 3999165			5.52	6.36	86.85	1.00	0.00	6.17
○Thread 3999166			5.43	6.36	85.35	1.00	0.00	6.17
○Thread 3999167			5.36	6.36	84.25	1.00	0.00	6.17
○Thread 3999168			5.51	6.36	86.68	1.00	0.00	6.16
○Thread 3999169			5.49	6.36	86.37	1.00	0.00	6.17
○Thread 3999170			5.53	6.36	86.91	1.00	0.00	6.17

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	5.74
►Process 4006090+		40	5.74
○Thread 4006090			2.23	7.61	29.29	0.00	0.00	6.60
○Thread 4006157			5.57	6.61	84.21	0.99	0.01	6.43
○Thread 4006158			5.58	6.61	84.29	0.99	0.01	6.43
○Thread 4006159			5.71	6.61	86.25	0.99	0.01	6.43
○Thread 4006160			5.59	6.61	84.43	0.99	0.01	6.43
○Thread 4006161			5.53	6.61	83.53	0.99	0.01	6.43
○Thread 4006162			5.65	6.61	85.34	0.99	0.01	6.43
○Thread 4006163			5.67	6.61	85.64	0.99	0.01	6.43
○Thread 4006164			5.61	6.62	84.73	0.99	0.01	6.43
○Thread 4006165			5.59	6.62	84.43	0.99	0.01	6.43
○Thread 4006166			5.56	6.62	84.05	0.99	0.01	6.43
○Thread 4006167			5.56	6.62	84.04	0.99	0.01	6.43
○Thread 4006168			5.63	6.62	85.02	0.99	0.01	6.43
○Thread 4006169			5.68	6.62	85.85	0.99	0.01	6.43
○Thread 4006170			5.65	6.62	85.32	0.99	0.01	6.43
○Thread 4006171			5.64	6.62	85.17	0.99	0.01	6.43
○Thread 4006172			5.50	6.62	83.05	0.99	0.01	6.43
○Thread 4006173			5.62	6.62	84.94	0.99	0.01	6.42
○Thread 4006174			5.65	6.62	85.32	0.99	0.01	6.43
○Thread 4006175			5.64	6.62	85.16	0.99	0.01	6.43
○Thread 4006176			5.54	6.62	83.73	0.99	0.01	6.43
○Thread 4006177			5.62	6.62	84.86	0.99	0.01	6.43
○Thread 4006178			5.61	6.62	84.70	0.99	0.01	6.43
○Thread 4006179			5.49	6.62	82.96	0.99	0.01	6.43
○Thread 4006180			5.66	6.62	85.45	0.99	0.00	6.43
○Thread 4006181			5.64	6.62	85.15	0.99	0.00	6.43
○Thread 4006182			5.62	6.62	84.92	0.99	0.00	6.43
○Thread 4006183			5.62	6.62	84.92	0.99	0.00	6.43
○Thread 4006184			5.60	6.62	84.62	0.99	0.00	6.43
○Thread 4006185			5.53	6.62	83.56	0.99	0.00	6.43
○Thread 4006186			5.61	6.62	84.76	0.99	0.00	6.43
○Thread 4006187			5.74	6.62	86.65	0.99	0.00	6.43
○Thread 4006188			5.70	6.62	86.20	0.99	0.00	6.43
○Thread 4006189			5.51	6.62	83.25	0.99	0.00	6.43
○Thread 4006190			5.55	6.62	83.85	0.99	0.00	6.43
○Thread 4006191			5.56	6.62	84.00	0.99	0.00	6.43
○Thread 4006192			5.61	6.62	84.68	0.99	0.00	6.43
○Thread 4006193			5.55	6.62	83.84	0.99	0.00	6.43
○Thread 4006194			5.60	6.62	84.60	0.99	0.00	6.43
○Thread 4006195			5.65	6.62	85.35	0.99	0.00	6.43

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	5.97
►Process 4014834+		48	5.97
○Thread 4014834			1.84	8.00	23.00	0.00	0.00	6.97
○Thread 4014901			5.71	7.00	81.69	0.99	0.01	6.80
○Thread 4014902			5.85	7.00	83.62	0.99	0.01	6.81
○Thread 4014903			5.87	7.00	83.97	0.99	0.01	6.81
○Thread 4014904			5.81	7.00	83.04	0.99	0.01	6.81
○Thread 4014905			5.87	7.00	83.83	0.99	0.01	6.81
○Thread 4014906			5.85	7.00	83.68	0.99	0.01	6.80
○Thread 4014907			5.92	7.00	84.68	0.99	0.01	6.81
○Thread 4014908			5.97	7.00	85.32	0.99	0.01	6.80
○Thread 4014909			5.89	7.00	84.18	0.99	0.01	6.81
○Thread 4014910			5.82	7.00	83.17	0.99	0.01	6.81
○Thread 4014911			5.82	7.00	83.17	0.99	0.01	6.81
○Thread 4014912			5.85	7.00	83.60	0.99	0.01	6.81
○Thread 4014913			5.90	7.00	84.31	0.99	0.01	6.81
○Thread 4014914			5.89	7.00	84.17	0.99	0.01	6.81
○Thread 4014915			5.80	7.00	82.81	0.99	0.01	6.81
○Thread 4014916			5.86	7.00	83.73	0.99	0.01	6.81
○Thread 4014917			5.89	7.00	84.09	0.99	0.01	6.81
○Thread 4014918			5.78	7.00	82.59	0.99	0.01	6.81
○Thread 4014919			5.82	7.00	83.16	0.99	0.01	6.81
○Thread 4014920			5.91	7.00	84.44	0.99	0.01	6.81
○Thread 4014921			5.88	7.00	84.01	0.99	0.01	6.81
○Thread 4014922			5.80	7.00	82.79	0.99	0.01	6.80
○Thread 4014923			5.84	7.00	83.44	0.99	0.01	6.81
○Thread 4014924			5.88	7.00	83.93	0.99	0.01	6.81
○Thread 4014925			5.84	7.00	83.43	0.99	0.01	6.81
○Thread 4014926			5.85	7.00	83.50	0.99	0.01	6.81
○Thread 4014927			5.86	7.00	83.64	0.99	0.01	6.81
○Thread 4014928			5.89	7.00	84.14	0.99	0.01	6.81
○Thread 4014929			5.94	7.00	84.78	0.99	0.01	6.81
○Thread 4014930			5.75	7.00	82.14	0.99	0.01	6.81
○Thread 4014931			5.83	7.00	83.28	0.99	0.01	6.81
○Thread 4014932			5.85	7.00	83.56	0.99	0.00	6.80
○Thread 4014933			5.96	7.00	85.06	0.99	0.00	6.81
○Thread 4014934			5.85	7.00	83.56	0.99	0.00	6.81
○Thread 4014935			5.81	7.00	82.91	0.99	0.00	6.80
○Thread 4014936			5.79	7.00	82.63	0.99	0.00	6.81
○Thread 4014937			5.79	7.00	82.70	0.99	0.00	6.81
○Thread 4014938			5.89	7.00	84.05	0.99	0.00	6.81
○Thread 4014939			5.79	7.00	82.69	0.99	0.00	6.80
○Thread 4014940			5.84	7.00	83.33	0.99	0.00	6.81
○Thread 4014941			5.80	7.00	82.76	0.99	0.00	6.81
○Thread 4014942			5.86	7.00	83.62	1.00	0.00	6.81
○Thread 4014943			5.88	7.00	83.90	1.00	0.00	6.81
○Thread 4014944			5.88	7.00	83.90	1.00	0.00	6.81
○Thread 4014945			5.78	7.00	82.47	1.00	0.00	6.80
○Thread 4014946			5.79	7.00	82.68	1.00	0.00	6.81
○Thread 4014947			5.84	7.00	83.32	1.00	0.00	6.80

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	6.26
►Process 4025354+		56	6.26
○Thread 4025354			1.58	8.38	18.84	0.00	0.00	7.37
○Thread 4025421			6.06	7.39	81.99	0.98	0.02	7.18
○Thread 4025422			6.02	7.39	81.45	0.98	0.01	7.19
○Thread 4025423			6.03	7.39	81.51	0.98	0.01	7.19
○Thread 4025424			6.09	7.39	82.39	0.98	0.01	7.19
○Thread 4025425			6.06	7.39	81.98	0.98	0.01	7.19
○Thread 4025426			6.11	7.39	82.66	0.98	0.01	7.19
○Thread 4025427			6.14	7.39	82.99	0.98	0.01	7.19
○Thread 4025428			6.03	7.39	81.64	0.98	0.01	7.19
○Thread 4025429			6.16	7.39	83.33	0.98	0.01	7.19
○Thread 4025430			6.13	7.39	82.92	0.98	0.01	7.19
○Thread 4025431			6.10	7.39	82.51	0.98	0.01	7.19
○Thread 4025432			6.10	7.39	82.44	0.98	0.01	7.20
○Thread 4025433			6.08	7.39	82.24	0.98	0.01	7.19
○Thread 4025434			6.12	7.39	82.71	0.98	0.01	7.19
○Thread 4025435			6.17	7.39	83.45	0.98	0.01	7.19
○Thread 4025436			6.12	7.39	82.78	0.98	0.01	7.19
○Thread 4025437			6.17	7.39	83.45	0.98	0.01	7.19
○Thread 4025438			6.16	7.39	83.31	0.98	0.01	7.19
○Thread 4025439			6.26	7.39	84.66	0.98	0.01	7.19
○Thread 4025440			6.06	7.39	81.96	0.98	0.01	7.19
○Thread 4025441			6.05	7.39	81.75	0.98	0.01	7.19
○Thread 4025442			6.10	7.39	82.50	0.98	0.01	7.19
○Thread 4025443			6.16	7.39	83.30	0.98	0.01	7.19
○Thread 4025444			6.08	7.39	82.22	0.98	0.01	7.19
○Thread 4025445			6.03	7.39	81.48	0.98	0.01	7.18
○Thread 4025446			6.23	7.39	84.25	0.98	0.01	7.19
○Thread 4025447			6.16	7.40	83.30	0.98	0.01	7.19
○Thread 4025448			6.07	7.40	82.08	0.98	0.01	7.19
○Thread 4025449			6.21	7.40	83.90	0.98	0.01	7.19
○Thread 4025450			6.08	7.40	82.14	0.98	0.01	7.19
○Thread 4025451			6.11	7.40	82.55	0.98	0.01	7.19
○Thread 4025452			6.11	7.40	82.61	0.98	0.01	7.19
○Thread 4025453			6.10	7.40	82.41	0.98	0.01	7.19
○Thread 4025454			6.06	7.40	81.93	0.98	0.01	7.19
○Thread 4025455			6.08	7.40	82.14	0.98	0.01	7.20
○Thread 4025456			6.14	7.40	83.01	0.98	0.01	7.19
○Thread 4025457			6.08	7.40	82.13	0.98	0.01	7.19
○Thread 4025458			6.01	7.40	81.32	0.98	0.01	7.19
○Thread 4025459			6.06	7.40	81.93	0.98	0.01	7.19
○Thread 4025460			5.95	7.40	80.37	0.98	0.00	7.19
○Thread 4025461			6.18	7.40	83.54	0.98	0.00	7.19
○Thread 4025462			6.11	7.40	82.53	0.98	0.00	7.19
○Thread 4025463			6.08	7.40	82.19	0.98	0.00	7.19
○Thread 4025464			6.00	7.40	81.11	0.98	0.00	7.19
○Thread 4025465			6.12	7.40	82.66	0.98	0.00	7.19
○Thread 4025466			6.02	7.40	81.37	0.98	0.00	7.17
○Thread 4025467			6.11	7.40	82.59	0.98	0.00	7.19
○Thread 4025468			6.09	7.40	82.32	0.98	0.00	7.19
○Thread 4025469			6.09	7.40	82.32	0.98	0.00	7.19
○Thread 4025470			6.05	7.40	81.77	0.98	0.00	7.19
○Thread 4025471			6.07	7.40	82.04	0.98	0.00	7.19
○Thread 4025472			6.04	7.40	81.57	0.98	0.00	7.19
○Thread 4025473			6.16	7.40	83.25	0.98	0.00	7.19
○Thread 4025474			5.97	7.40	80.62	0.98	0.00	7.19
○Thread 4025475			6.09	7.40	82.30	0.98	0.00	7.19

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	8.44
►Process 4037650+		64	8.44
○Thread 4037650			1.66	11.24	14.76	0.00	0.00	10.17
○Thread 4037718			8.26	10.20	80.99	1.03	0.02	9.99
○Thread 4037719			8.34	10.20	81.77	1.03	0.02	10.00
○Thread 4037720			8.38	10.20	82.11	1.03	0.02	10.00
○Thread 4037721			8.31	10.20	81.43	1.03	0.02	10.00
○Thread 4037722			8.42	10.20	82.55	1.03	0.02	10.00
○Thread 4037723			8.39	10.20	82.26	1.03	0.02	10.00
○Thread 4037724			8.21	10.20	80.44	1.03	0.02	10.00
○Thread 4037725			8.38	10.20	82.16	1.03	0.02	9.99
○Thread 4037726			8.18	10.20	80.19	1.03	0.02	9.99
○Thread 4037727			8.31	10.20	81.47	1.03	0.02	10.00
○Thread 4037728			8.29	10.20	81.27	1.03	0.02	10.00
○Thread 4037729			8.26	10.20	80.93	1.03	0.01	10.00
○Thread 4037730			8.30	10.20	81.32	1.03	0.01	10.00
○Thread 4037731			8.20	10.20	80.38	1.03	0.01	10.00
○Thread 4037732			8.29	10.20	81.22	1.03	0.01	9.98
○Thread 4037733			8.28	10.20	81.12	1.03	0.01	9.99
○Thread 4037734			8.26	10.20	80.97	1.03	0.01	9.99
○Thread 4037735			8.28	10.20	81.16	1.03	0.01	10.00
○Thread 4037736			8.35	10.20	81.80	1.03	0.01	10.00
○Thread 4037737			8.33	10.20	81.60	1.03	0.01	10.00
○Thread 4037738			8.33	10.20	81.65	1.03	0.01	10.00
○Thread 4037739			8.24	10.20	80.72	1.03	0.01	10.00
○Thread 4037740			8.34	10.20	81.70	1.03	0.01	10.00
○Thread 4037741			8.33	10.20	81.65	1.03	0.01	10.00
○Thread 4037742			8.24	10.20	80.76	1.03	0.01	10.00
○Thread 4037743			8.37	10.20	82.04	1.03	0.01	9.99
○Thread 4037744			8.19	10.20	80.27	1.03	0.01	10.00
○Thread 4037745			8.30	10.20	81.30	1.03	0.01	10.00
○Thread 4037746			8.34	10.20	81.74	1.03	0.01	10.00
○Thread 4037747			8.28	10.20	81.15	1.03	0.01	10.00
○Thread 4037748			8.34	10.20	81.73	1.03	0.01	9.99
○Thread 4037749			8.31	10.20	81.39	1.03	0.01	10.00
○Thread 4037750			8.25	10.20	80.80	1.03	0.01	9.99
○Thread 4037751			8.21	10.20	80.46	1.03	0.01	10.00
○Thread 4037752			8.33	10.20	81.63	1.03	0.01	10.00
○Thread 4037753			8.36	10.20	81.92	1.03	0.01	10.00
○Thread 4037754			8.25	10.20	80.85	1.03	0.01	10.01
○Thread 4037755			8.44	10.20	82.71	1.03	0.01	10.00
○Thread 4037756			8.28	10.20	81.14	1.03	0.01	9.99
○Thread 4037757			8.19	10.21	80.25	1.03	0.01	10.00
○Thread 4037758			8.27	10.21	81.04	1.03	0.01	10.00
○Thread 4037759			8.36	10.21	81.87	1.03	0.01	10.01
○Thread 4037760			8.27	10.21	81.03	1.03	0.01	10.00
○Thread 4037761			8.31	10.21	81.42	1.03	0.01	9.99
○Thread 4037762			8.30	10.21	81.28	1.03	0.01	9.99
○Thread 4037763			8.36	10.21	81.86	1.03	0.01	9.99
○Thread 4037764			8.36	10.21	81.86	1.03	0.01	9.99
○Thread 4037765			8.16	10.21	79.90	1.03	0.00	10.00
○Thread 4037766			8.35	10.21	81.81	1.03	0.00	9.99
○Thread 4037767			8.23	10.21	80.58	1.03	0.00	10.00
○Thread 4037768			8.20	10.21	80.34	1.03	0.00	9.99
○Thread 4037769			8.35	10.21	81.81	1.03	0.00	10.00
○Thread 4037770			8.36	10.21	81.90	1.03	0.00	10.00
○Thread 4037771			8.18	10.21	80.09	1.03	0.00	10.01
○Thread 4037772			8.28	10.21	81.07	1.03	0.00	10.00
○Thread 4037773			8.25	10.21	80.77	1.03	0.00	10.00
○Thread 4037774			8.26	10.21	80.92	1.03	0.00	10.00
○Thread 4037775			8.29	10.21	81.16	1.03	0.00	10.00
○Thread 4037776			8.26	10.21	80.87	1.03	0.00	10.00
○Thread 4037777			8.25	10.21	80.82	1.03	0.00	9.98
○Thread 4037778			8.36	10.21	81.89	1.03	0.00	9.98
○Thread 4037779			8.06	10.21	78.95	1.03	0.00	9.97
○Thread 4037780			8.19	10.21	80.23	1.03	0.00	9.99

Report Configuration