OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	63.32
►Process 49824+		1	63.32
○Thread 49824			63.32	64.36	98.38	0.00	0.00	64.32

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	33.96
►Process 49961+		2	33.96
○Thread 49961			33.96	35.04	96.93	0.00	0.00	35.01
○Thread 50028			33.61	33.98	98.90	1.05	-0.00	33.63

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	17.48
►Process 50097+		4	17.48
○Thread 50097			17.48	18.53	94.34	0.00	0.00	18.49
○Thread 50164			17.12	17.48	97.95	1.06	0.00	17.13
○Thread 50165			17.13	17.48	97.98	1.06	0.00	17.13
○Thread 50166			17.13	17.48	98.01	1.06	-0.00	17.13

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	9.41
►Process 50186+		8	9.41
○Thread 50186			9.41	10.48	89.79	0.00	0.00	10.45
○Thread 50253			9.06	9.42	96.10	1.06	0.00	9.07
○Thread 50254			9.03	9.42	95.78	1.06	0.00	9.07
○Thread 50255			8.96	9.42	95.09	1.06	0.00	9.07
○Thread 50256			9.02	9.42	95.73	1.06	0.00	9.07
○Thread 50257			9.01	9.42	95.62	1.06	0.00	9.07
○Thread 50258			9.02	9.42	95.73	1.06	0.00	9.07
○Thread 50259			9.03	9.42	95.84	1.06	-0.00	9.07

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	5.96
►Process 50279+		16	5.96
○Thread 50279			5.96	7.19	82.85	0.00	0.00	7.16
○Thread 50346			5.62	6.14	91.50	1.06	-0.00	5.77
○Thread 50347			5.65	6.14	92.08	1.06	0.00	5.77
○Thread 50348			5.63	6.14	91.75	1.06	0.00	5.77
○Thread 50349			5.64	6.14	91.92	1.06	0.00	5.77
○Thread 50350			5.66	6.14	92.24	1.06	0.00	5.77
○Thread 50351			5.67	6.14	92.33	1.06	0.00	5.77
○Thread 50352			5.67	6.14	92.41	1.06	0.00	5.77
○Thread 50353			5.61	6.14	91.35	1.06	0.00	5.77
○Thread 50354			5.61	6.14	91.35	1.06	0.00	5.77
○Thread 50355			5.65	6.14	92.09	1.06	0.00	5.77
○Thread 50356			5.61	6.14	91.36	1.06	0.00	5.77
○Thread 50357			5.64	6.14	91.93	1.06	0.00	5.77
○Thread 50358			5.65	6.14	92.09	1.06	0.00	5.77
○Thread 50359			5.64	6.14	91.85	1.06	0.00	5.77
○Thread 50360			5.60	6.13	91.28	1.06	-0.00	5.77

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	5.76
►Process 50430+		24	5.76
○Thread 50430			5.76	6.99	82.41	0.00	0.00	6.96
○Thread 50497			5.47	5.93	92.24	1.06	0.00	5.58
○Thread 50498			5.34	5.93	90.05	1.06	0.00	5.57
○Thread 50499			5.42	5.93	91.40	1.06	-0.00	5.57
○Thread 50500			5.36	5.93	90.31	1.06	0.00	5.56
○Thread 50501			5.27	5.93	88.79	1.06	0.00	5.57
○Thread 50502			5.31	5.93	89.55	1.06	0.00	5.57
○Thread 50503			5.32	5.93	89.64	1.06	0.00	5.57
○Thread 50504			5.30	5.93	89.30	1.06	0.00	5.57
○Thread 50505			5.35	5.93	90.15	1.06	0.00	5.57
○Thread 50506			5.34	5.93	90.07	1.06	0.00	5.57
○Thread 50507			5.56	5.93	93.78	1.06	0.00	5.58
○Thread 50508			5.56	5.93	93.78	1.06	0.00	5.57
○Thread 50509			5.51	5.93	92.94	1.06	0.00	5.58
○Thread 50510			5.38	5.93	90.75	1.06	0.00	5.57
○Thread 50511			5.46	5.93	92.10	1.06	0.00	5.57
○Thread 50512			5.39	5.93	90.83	1.06	0.00	5.57
○Thread 50513			5.45	5.93	92.02	1.06	0.00	5.57
○Thread 50514			5.33	5.93	89.91	1.06	0.00	5.57
○Thread 50515			5.32	5.93	89.66	1.06	0.00	5.57
○Thread 50516			5.40	5.93	91.09	1.06	0.00	5.58
○Thread 50517			5.36	5.93	90.42	1.06	0.00	5.57
○Thread 50518			5.37	5.93	90.51	1.06	0.00	5.57
○Thread 50519			5.28	5.93	89.07	1.06	-0.00	5.57

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	5.40
►Process 50539+		32	5.40
○Thread 50539			5.40	6.64	81.20	0.00	0.00	6.61
○Thread 50606			5.08	5.58	90.89	1.06	0.00	5.22
○Thread 50607			5.00	5.58	89.55	1.06	0.00	5.22
○Thread 50608			5.14	5.58	91.97	1.06	0.00	5.22
○Thread 50609			5.08	5.58	90.98	1.06	0.00	5.23
○Thread 50610			5.05	5.58	90.45	1.06	-0.00	5.22
○Thread 50611			5.01	5.58	89.73	1.06	-0.00	5.22
○Thread 50612			4.92	5.58	88.03	1.06	0.00	5.23
○Thread 50613			5.06	5.58	90.54	1.06	0.00	5.22
○Thread 50614			4.93	5.58	88.31	1.06	0.00	5.22
○Thread 50615			4.98	5.58	89.11	1.06	0.00	5.22
○Thread 50616			4.93	5.58	88.31	1.06	0.00	5.22
○Thread 50617			4.96	5.58	88.76	1.06	0.00	5.22
○Thread 50618			5.03	5.58	90.10	1.06	0.00	5.22
○Thread 50619			4.94	5.58	88.40	1.06	0.00	5.22
○Thread 50620			4.95	5.58	88.67	1.06	0.00	5.22
○Thread 50621			4.99	5.58	89.30	1.06	0.00	5.22
○Thread 50622			4.96	5.58	88.85	1.06	0.00	5.22
○Thread 50623			4.96	5.58	88.86	1.06	0.00	5.23
○Thread 50624			4.95	5.58	88.59	1.06	0.00	5.22
○Thread 50625			4.93	5.58	88.32	1.06	0.00	5.22
○Thread 50626			4.85	5.58	86.89	1.06	0.00	5.22
○Thread 50627			4.96	5.58	88.86	1.06	0.00	5.22
○Thread 50628			4.86	5.58	87.07	1.06	0.00	5.22
○Thread 50629			4.90	5.58	87.70	1.06	0.00	5.21
○Thread 50630			4.87	5.58	87.25	1.06	0.00	5.22
○Thread 50631			4.93	5.58	88.33	1.06	0.00	5.22
○Thread 50632			4.88	5.58	87.44	1.06	0.00	5.22
○Thread 50633			4.96	5.58	88.87	1.06	0.00	5.21
○Thread 50634			4.91	5.58	87.98	1.06	0.00	5.22
○Thread 50635			4.95	5.58	88.70	1.06	0.00	5.22
○Thread 50636			4.89	5.58	87.71	1.06	-0.00	5.23

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	5.10
►Process 50656+		40	5.10
○Thread 50656			5.10	6.47	78.88	0.00	0.00	6.43
○Thread 50723			4.83	5.40	89.35	1.07	0.00	5.04
○Thread 50724			4.97	5.40	91.95	1.07	-0.00	5.04
○Thread 50725			4.74	5.40	87.78	1.07	0.00	5.04
○Thread 50726			4.74	5.40	87.69	1.07	0.00	5.04
○Thread 50727			4.67	5.40	86.49	1.07	0.00	5.04
○Thread 50728			4.73	5.40	87.51	1.07	0.00	5.04
○Thread 50729			4.66	5.40	86.31	1.07	0.00	5.04
○Thread 50730			4.66	5.40	86.22	1.07	0.00	5.04
○Thread 50731			4.79	5.40	88.63	1.07	0.00	5.04
○Thread 50732			4.72	5.40	87.43	1.07	-0.00	5.04
○Thread 50733			4.66	5.40	86.32	1.07	0.00	5.04
○Thread 50734			4.68	5.40	86.60	1.07	0.00	5.04
○Thread 50735			4.69	5.40	86.87	1.07	0.00	5.04
○Thread 50736			4.61	5.40	85.39	1.07	0.00	5.04
○Thread 50737			4.65	5.40	86.14	1.07	0.00	5.04
○Thread 50738			4.90	5.40	90.77	1.07	0.00	5.04
○Thread 50739			4.78	5.40	88.55	1.07	0.00	5.04
○Thread 50740			4.70	5.40	87.07	1.07	0.00	5.04
○Thread 50741			4.73	5.40	87.62	1.07	0.00	5.04
○Thread 50742			4.70	5.40	87.07	1.07	0.00	5.04
○Thread 50743			4.66	5.40	86.33	1.07	0.00	5.03
○Thread 50744			4.70	5.40	87.07	1.07	0.00	5.04
○Thread 50745			4.67	5.40	86.43	1.07	0.00	5.04
○Thread 50746			4.64	5.40	86.06	1.07	0.00	5.03
○Thread 50747			4.70	5.40	86.99	1.07	0.00	5.04
○Thread 50748			4.67	5.40	86.52	1.07	0.00	5.04
○Thread 50749			4.65	5.40	86.06	1.07	0.00	5.04
○Thread 50750			4.62	5.40	85.51	1.07	0.00	5.04
○Thread 50751			4.58	5.40	84.77	1.07	0.00	5.04
○Thread 50752			4.65	5.40	86.07	1.07	0.00	5.04
○Thread 50753			4.63	5.40	85.70	1.07	0.00	5.03
○Thread 50754			4.59	5.40	85.05	1.07	0.00	5.04
○Thread 50755			4.67	5.40	86.44	1.07	0.00	5.04
○Thread 50756			4.62	5.40	85.52	1.07	0.00	5.04
○Thread 50757			4.64	5.40	86.08	1.07	0.00	5.04
○Thread 50758			4.61	5.40	85.34	1.07	0.00	5.04
○Thread 50759			4.69	5.40	86.82	1.07	0.00	5.04
○Thread 50760			4.62	5.40	85.62	1.07	0.00	5.04
○Thread 50761			4.68	5.40	86.82	1.07	-0.00	5.04

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	4.96
►Process 50781+		48	4.96
○Thread 50781			4.96	6.48	76.41	0.00	0.00	6.44
○Thread 50848			4.64	5.42	85.56	1.06	-0.00	5.05
○Thread 50849			4.61	5.42	85.01	1.06	0.00	5.06
○Thread 50850			4.57	5.42	84.28	1.06	0.00	5.05
○Thread 50851			4.61	5.42	85.02	1.06	-0.00	5.05
○Thread 50852			4.57	5.42	84.19	1.06	0.00	5.06
○Thread 50853			4.60	5.42	84.74	1.06	-0.00	5.05
○Thread 50854			4.59	5.42	84.65	1.06	-0.00	5.05
○Thread 50855			4.62	5.42	85.21	1.06	0.00	5.05
○Thread 50856			4.61	5.42	85.02	1.06	-0.00	5.05
○Thread 50857			4.65	5.42	85.67	1.06	0.00	5.05
○Thread 50858			4.68	5.42	86.32	1.06	0.00	5.05
○Thread 50859			4.67	5.42	86.04	1.06	0.00	5.06
○Thread 50860			4.77	5.42	87.89	1.06	0.00	5.06
○Thread 50861			4.89	5.42	90.20	1.06	0.00	5.06
○Thread 50862			4.58	5.42	84.39	1.06	0.00	5.05
○Thread 50863			4.60	5.42	84.85	1.06	0.00	5.06
○Thread 50864			4.55	5.42	83.84	1.06	0.00	5.06
○Thread 50865			4.54	5.42	83.65	1.06	0.00	5.05
○Thread 50866			4.47	5.42	82.36	1.06	0.00	5.05
○Thread 50867			4.51	5.42	83.29	1.06	-0.00	5.05
○Thread 50868			4.53	5.42	83.57	1.06	0.00	5.05
○Thread 50869			4.44	5.42	81.91	1.06	0.00	5.06
○Thread 50870			4.46	5.42	82.19	1.06	0.00	5.06
○Thread 50871			4.57	5.42	84.22	1.06	0.00	5.05
○Thread 50872			4.55	5.42	83.94	1.06	0.00	5.06
○Thread 50873			4.64	5.42	85.60	1.06	0.00	5.06
○Thread 50874			4.49	5.42	82.75	1.06	0.00	5.05
○Thread 50875			4.53	5.42	83.48	1.06	0.00	5.05
○Thread 50876			4.49	5.42	82.84	1.06	0.00	5.05
○Thread 50877			4.52	5.42	83.40	1.06	0.00	5.05
○Thread 50878			4.62	5.42	85.15	1.06	0.00	5.06
○Thread 50879			4.60	5.42	84.87	1.06	0.00	5.05
○Thread 50880			4.55	5.42	83.95	1.07	0.00	5.06
○Thread 50881			4.63	5.42	85.43	1.07	0.00	5.05
○Thread 50882			4.53	5.42	83.68	1.07	0.00	5.06
○Thread 50883			4.63	5.42	85.43	1.07	0.00	5.05
○Thread 50884			4.60	5.42	84.79	1.07	0.00	5.05
○Thread 50885			4.61	5.42	85.07	1.07	0.00	5.05
○Thread 50886			4.64	5.42	85.62	1.07	0.00	5.06
○Thread 50887			4.54	5.42	83.69	1.07	0.00	5.06
○Thread 50888			4.70	5.42	86.64	1.07	0.00	5.06
○Thread 50889			4.59	5.42	84.61	1.07	0.00	5.05
○Thread 50890			4.57	5.42	84.34	1.07	0.00	5.06
○Thread 50891			4.65	5.42	85.72	1.07	0.00	5.06
○Thread 50892			4.57	5.42	84.34	1.07	0.00	5.04
○Thread 50893			4.55	5.42	83.97	1.07	0.00	5.05
○Thread 50894			4.50	5.42	83.05	1.07	-0.00	5.05

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	5.00
►Process 50964+		56	5.00
○Thread 50964			5.00	6.60	75.71	0.00	0.00	6.57
○Thread 51031			4.57	5.53	82.60	1.07	0.00	5.14
○Thread 51032			4.57	5.53	82.52	1.07	0.00	5.15
○Thread 51033			4.55	5.53	82.25	1.07	0.00	5.16
○Thread 51034			4.58	5.53	82.79	1.07	0.00	5.17
○Thread 51035			4.54	5.53	82.07	1.07	-0.00	5.15
○Thread 51036			4.41	5.53	79.72	1.07	-0.00	5.16
○Thread 51037			4.48	5.53	80.99	1.07	0.00	5.16
○Thread 51038			4.46	5.53	80.63	1.07	0.00	5.16
○Thread 51039			4.62	5.53	83.43	1.07	0.00	5.17
○Thread 51040			4.60	5.53	83.16	1.07	0.00	5.17
○Thread 51041			4.67	5.53	84.34	1.07	0.00	5.17
○Thread 51042			4.45	5.53	80.54	1.07	0.00	5.16
○Thread 51043			4.57	5.53	82.62	1.07	0.00	5.16
○Thread 51044			4.63	5.53	83.71	1.07	0.00	5.16
○Thread 51045			4.50	5.53	81.36	1.07	0.00	5.16
○Thread 51046			4.51	5.53	81.45	1.07	0.00	5.16
○Thread 51047			4.55	5.53	82.18	1.07	-0.00	5.16
○Thread 51048			4.52	5.53	81.73	1.07	-0.00	5.16
○Thread 51049			4.58	5.53	82.81	1.07	-0.00	5.16
○Thread 51050			4.55	5.53	82.27	1.07	-0.00	5.16
○Thread 51051			4.57	5.53	82.54	1.07	-0.00	5.16
○Thread 51052			4.55	5.53	82.27	1.07	0.00	5.16
○Thread 51053			4.54	5.53	82.00	1.07	0.00	5.16
○Thread 51054			4.57	5.53	82.55	1.07	-0.00	5.16
○Thread 51055			4.65	5.53	84.09	1.07	0.00	5.16
○Thread 51056			4.64	5.53	83.91	1.07	-0.00	5.16
○Thread 51057			4.58	5.53	82.82	1.07	-0.00	5.16
○Thread 51058			4.54	5.53	82.10	1.07	-0.00	5.15
○Thread 51059			4.58	5.53	82.74	1.07	-0.00	5.16
○Thread 51060			4.64	5.53	83.91	1.07	-0.00	5.16
○Thread 51061			4.42	5.53	79.94	1.07	-0.00	5.15
○Thread 51062			4.57	5.53	82.56	1.07	-0.00	5.16
○Thread 51063			4.56	5.53	82.38	1.07	-0.00	5.16
○Thread 51064			4.56	5.53	82.38	1.07	-0.00	5.16
○Thread 51065			4.57	5.53	82.56	1.07	-0.00	5.16
○Thread 51066			4.61	5.53	83.47	1.07	-0.00	5.16
○Thread 51067			4.63	5.53	83.83	1.07	-0.00	5.16
○Thread 51068			4.58	5.53	82.84	1.07	-0.00	5.16
○Thread 51069			4.57	5.53	82.66	1.07	0.00	5.16
○Thread 51070			4.67	5.53	84.38	1.07	-0.00	5.16
○Thread 51071			4.63	5.53	83.75	1.07	0.00	5.16
○Thread 51072			4.57	5.53	82.57	1.07	-0.00	5.16
○Thread 51073			4.64	5.53	83.93	1.07	-0.00	5.16
○Thread 51074			4.54	5.53	82.03	1.07	-0.00	5.16
○Thread 51075			4.74	5.53	85.65	1.07	0.00	5.15
○Thread 51076			4.50	5.53	81.40	1.07	-0.00	5.16
○Thread 51077			4.55	5.53	82.31	1.07	0.00	5.16
○Thread 51078			4.53	5.53	81.86	1.07	0.00	5.15
○Thread 51079			4.67	5.53	84.48	1.07	0.00	5.15
○Thread 51080			4.65	5.53	84.03	1.07	0.00	5.15
○Thread 51081			4.61	5.53	83.31	1.07	0.00	5.15
○Thread 51082			4.48	5.53	80.96	1.07	0.00	5.16
○Thread 51083			4.59	5.53	83.04	1.07	-0.00	5.16
○Thread 51084			4.56	5.53	82.59	1.07	0.00	5.16
○Thread 51085			4.38	5.53	79.24	1.07	-0.00	5.16

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	5.06
►Process 51105+		64	5.06
○Thread 51105			5.06	6.68	75.78	0.00	0.00	6.65
○Thread 51172			4.85	5.61	86.38	1.07	-0.00	5.24
○Thread 51173			4.67	5.61	83.17	1.07	0.00	5.23
○Thread 51174			4.61	5.61	82.10	1.07	-0.00	5.23
○Thread 51175			4.57	5.61	81.48	1.07	-0.00	5.23
○Thread 51176			4.94	5.61	87.99	1.07	-0.00	5.23
○Thread 51177			4.65	5.61	82.91	1.07	-0.00	5.22
○Thread 51178			4.71	5.61	83.98	1.07	-0.00	5.23
○Thread 51179			4.56	5.61	81.22	1.07	-0.00	5.23
○Thread 51180			4.59	5.61	81.84	1.07	-0.00	5.23
○Thread 51181			4.84	5.61	86.30	1.07	-0.00	5.23
○Thread 51182			4.76	5.61	84.79	1.07	-0.00	5.23
○Thread 51183			4.73	5.61	84.34	1.07	-0.00	5.23
○Thread 51184			4.67	5.61	83.19	1.07	-0.00	5.23
○Thread 51185			4.70	5.61	83.72	1.07	-0.00	5.23
○Thread 51186			4.67	5.61	83.28	1.07	-0.00	5.23
○Thread 51187			4.66	5.61	83.01	1.07	-0.00	5.23
○Thread 51188			4.58	5.61	81.68	1.07	-0.00	5.23
○Thread 51189			4.60	5.61	81.95	1.07	-0.00	5.23
○Thread 51190			4.61	5.61	82.13	1.07	-0.00	5.23
○Thread 51191			4.59	5.61	81.86	1.07	-0.00	5.23
○Thread 51192			4.62	5.61	82.40	1.07	-0.00	5.23
○Thread 51193			4.70	5.61	83.73	1.07	-0.00	5.23
○Thread 51194			4.64	5.61	82.84	1.07	-0.00	5.23
○Thread 51195			4.57	5.61	81.42	1.07	-0.00	5.24
○Thread 51196			4.61	5.61	82.13	1.07	-0.00	5.23
○Thread 51197			4.53	5.61	80.71	1.07	-0.00	5.23
○Thread 51198			4.64	5.61	82.67	1.07	-0.00	5.23
○Thread 51199			4.51	5.61	80.44	1.07	-0.00	5.23
○Thread 51200			4.62	5.61	82.32	1.07	-0.00	5.24
○Thread 51201			4.76	5.61	84.91	1.07	-0.00	5.23
○Thread 51202			4.60	5.61	81.96	1.07	-0.00	5.23
○Thread 51203			4.62	5.61	82.32	1.07	-0.00	5.24
○Thread 51204			4.60	5.61	81.97	1.07	-0.00	5.23
○Thread 51205			4.66	5.61	83.04	1.07	-0.00	5.23
○Thread 51206			4.58	5.61	81.61	1.07	-0.00	5.23
○Thread 51207			4.59	5.61	81.88	1.07	-0.00	5.23
○Thread 51208			4.61	5.61	82.15	1.07	-0.00	5.23
○Thread 51209			4.61	5.61	82.24	1.07	-0.00	5.23
○Thread 51210			4.66	5.61	83.13	1.07	-0.00	5.23
○Thread 51211			4.63	5.61	82.51	1.07	-0.00	5.23
○Thread 51212			4.59	5.61	81.89	1.07	-0.00	5.23
○Thread 51213			4.60	5.61	81.98	1.07	-0.00	5.23
○Thread 51214			4.55	5.60	81.09	1.07	-0.00	5.23
○Thread 51215			4.58	5.60	81.72	1.07	-0.00	5.23
○Thread 51216			4.55	5.60	81.09	1.07	-0.00	5.23
○Thread 51217			4.68	5.60	83.50	1.07	-0.00	5.23
○Thread 51218			4.56	5.60	81.36	1.07	-0.00	5.23
○Thread 51219			4.59	5.60	81.99	1.07	-0.00	5.23
○Thread 51220			4.51	5.60	80.38	1.07	-0.00	5.23
○Thread 51221			4.59	5.60	81.81	1.07	-0.00	5.23
○Thread 51222			4.53	5.60	80.74	1.07	-0.00	5.23
○Thread 51223			4.56	5.60	81.28	1.07	-0.00	5.23
○Thread 51224			4.47	5.60	79.76	1.07	-0.00	5.23
○Thread 51225			4.49	5.60	80.03	1.07	-0.00	5.23
○Thread 51226			4.50	5.60	80.21	1.07	-0.00	5.23
○Thread 51227			4.53	5.60	80.93	1.07	-0.00	5.23
○Thread 51228			4.59	5.60	81.91	1.07	-0.00	5.23
○Thread 51229			4.50	5.60	80.31	1.07	-0.00	5.23
○Thread 51230			4.63	5.60	82.54	1.07	-0.00	5.23
○Thread 51231			4.61	5.60	82.18	1.07	-0.00	5.23
○Thread 51232			4.59	5.60	81.83	1.07	-0.00	5.23
○Thread 51233			4.50	5.60	80.31	1.07	-0.00	5.23
○Thread 51234			4.58	5.60	81.65	1.07	-0.00	5.23

Report Configuration