OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	155.99
►Process 54555+		1	155.99
○Thread 54555			155.99	157.05	99.32	0.00	0.00	157.01

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	78.27
►Process 54783+		2	78.27
○Thread 54783			78.27	79.37	98.61	0.00	0.00	79.32
○Thread 54850			77.91	78.31	99.49	1.06	-0.00	77.96

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	39.79
►Process 54930+		4	39.79
○Thread 54930			39.79	40.84	97.43	0.00	0.00	40.79
○Thread 54997			39.39	39.78	99.02	1.06	0.00	39.42
○Thread 54998			39.40	39.78	99.04	1.06	0.00	39.43
○Thread 54999			39.40	39.78	99.03	1.06	-0.00	39.43

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	20.84
►Process 55031+		8	20.84
○Thread 55031			20.84	21.92	95.08	0.00	0.00	21.88
○Thread 55098			20.41	20.86	97.89	1.06	0.00	20.50
○Thread 55099			20.44	20.86	98.03	1.06	0.00	20.51
○Thread 55100			20.41	20.86	97.89	1.06	0.00	20.51
○Thread 55101			20.48	20.86	98.20	1.06	0.00	20.51
○Thread 55102			20.44	20.86	98.01	1.06	0.00	20.51
○Thread 55103			20.44	20.86	98.01	1.06	0.00	20.51
○Thread 55104			20.42	20.86	97.91	1.06	-0.00	20.50

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	11.70
►Process 55173+		16	11.70
○Thread 55173			11.70	12.93	90.46	0.00	0.00	12.87
○Thread 55240			11.38	11.88	95.78	1.05	0.00	11.52
○Thread 55241			11.31	11.88	95.19	1.05	0.00	11.53
○Thread 55242			11.31	11.88	95.23	1.05	0.00	11.52
○Thread 55243			11.26	11.88	94.81	1.05	0.00	11.52
○Thread 55244			11.35	11.88	95.53	1.05	0.00	11.52
○Thread 55245			11.34	11.88	95.49	1.05	0.00	11.52
○Thread 55246			11.36	11.88	95.61	1.05	0.00	11.52
○Thread 55247			11.30	11.88	95.15	1.05	0.00	11.52
○Thread 55248			11.31	11.88	95.24	1.05	0.00	11.53
○Thread 55249			11.35	11.88	95.53	1.05	0.00	11.53
○Thread 55250			11.31	11.88	95.24	1.05	0.00	11.53
○Thread 55251			11.31	11.88	95.20	1.05	0.00	11.52
○Thread 55252			11.31	11.88	95.24	1.05	0.00	11.53
○Thread 55253			11.40	11.88	96.00	1.05	0.00	11.53
○Thread 55254			11.44	11.88	96.29	1.05	-0.00	11.53

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	9.13
►Process 55287+		24	9.13
○Thread 55287			9.13	10.42	87.55	0.00	0.00	10.37
○Thread 55354			8.70	9.37	92.89	1.06	-0.00	9.01
○Thread 55355			8.70	9.37	92.89	1.06	0.00	9.02
○Thread 55356			8.73	9.37	93.21	1.06	0.00	9.02
○Thread 55357			8.70	9.37	92.89	1.06	0.00	9.01
○Thread 55358			8.81	9.37	94.01	1.06	0.00	9.01
○Thread 55359			8.70	9.37	92.84	1.06	0.00	9.01
○Thread 55360			8.66	9.37	92.47	1.06	0.00	9.01
○Thread 55361			8.81	9.37	94.01	1.06	0.00	9.01
○Thread 55362			8.81	9.37	94.02	1.06	0.00	9.01
○Thread 55363			8.69	9.37	92.79	1.06	0.00	9.01
○Thread 55364			8.65	9.37	92.31	1.06	0.00	9.00
○Thread 55365			8.76	9.37	93.48	1.06	0.00	9.00
○Thread 55366			8.72	9.37	93.11	1.06	0.00	9.01
○Thread 55367			8.75	9.36	93.43	1.06	0.00	9.01
○Thread 55368			8.73	9.36	93.22	1.06	0.00	9.01
○Thread 55369			8.75	9.36	93.44	1.06	0.00	9.01
○Thread 55370			8.75	9.36	93.44	1.06	0.00	9.01
○Thread 55371			8.81	9.36	94.08	1.06	0.00	9.01
○Thread 55372			8.77	9.36	93.65	1.06	0.00	9.01
○Thread 55373			8.65	9.36	92.32	1.06	0.00	9.00
○Thread 55374			8.76	9.36	93.49	1.06	0.00	9.01
○Thread 55375			8.79	9.36	93.82	1.06	0.00	9.02
○Thread 55376			8.72	9.36	93.07	1.06	-0.00	9.00

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	7.71
►Process 55405+		32	7.71
○Thread 55405			7.71	9.15	84.23	0.00	0.00	9.10
○Thread 55472			7.32	8.09	90.45	1.07	0.00	7.73
○Thread 55473			7.38	8.09	91.26	1.07	0.00	7.72
○Thread 55474			7.34	8.09	90.71	1.07	0.00	7.72
○Thread 55475			7.29	8.09	90.09	1.07	0.00	7.72
○Thread 55476			7.34	8.09	90.71	1.07	0.00	7.72
○Thread 55477			7.31	8.09	90.40	1.07	0.00	7.73
○Thread 55478			7.34	8.09	90.77	1.07	0.00	7.73
○Thread 55479			7.33	8.09	90.59	1.07	-0.00	7.73
○Thread 55480			7.39	8.09	91.39	1.07	0.00	7.72
○Thread 55481			7.33	8.09	90.59	1.07	0.00	7.73
○Thread 55482			7.36	8.09	91.02	1.07	0.00	7.73
○Thread 55483			7.34	8.09	90.78	1.07	0.00	7.73
○Thread 55484			7.37	8.09	91.15	1.07	0.00	7.73
○Thread 55485			7.40	8.09	91.52	1.07	0.00	7.73
○Thread 55486			7.46	8.09	92.20	1.07	0.00	7.73
○Thread 55487			7.38	8.09	91.28	1.07	0.00	7.72
○Thread 55488			7.38	8.09	91.22	1.07	0.00	7.73
○Thread 55489			7.34	8.09	90.78	1.07	0.00	7.73
○Thread 55490			7.36	8.09	91.03	1.07	0.00	7.72
○Thread 55491			7.41	8.08	91.59	1.07	0.00	7.72
○Thread 55492			7.39	8.08	91.34	1.07	0.00	7.72
○Thread 55493			7.43	8.08	91.90	1.07	0.00	7.73
○Thread 55494			7.34	8.08	90.73	1.07	0.00	7.73
○Thread 55495			7.43	8.08	91.84	1.07	0.00	7.72
○Thread 55496			7.43	8.08	91.90	1.07	0.00	7.73
○Thread 55497			7.43	8.08	91.97	1.07	0.00	7.73
○Thread 55498			7.53	8.08	93.14	1.07	0.00	7.73
○Thread 55499			7.39	8.08	91.41	1.07	0.00	7.72
○Thread 55500			7.37	8.08	91.11	1.07	0.00	7.73
○Thread 55501			7.34	8.08	90.80	1.07	0.00	7.73
○Thread 55502			7.32	8.08	90.55	1.07	-0.00	7.73

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	7.27
►Process 55574+		40	7.27
○Thread 55574			7.27	8.74	83.10	0.00	0.00	8.70
○Thread 55643			6.74	7.67	87.76	1.07	0.00	7.31
○Thread 55644			6.82	7.67	88.87	1.07	-0.00	7.31
○Thread 55645			6.80	7.67	88.61	1.07	-0.00	7.32
○Thread 55646			6.83	7.67	88.93	1.07	-0.00	7.32
○Thread 55647			6.84	7.67	89.13	1.07	-0.00	7.32
○Thread 55648			6.77	7.67	88.22	1.07	0.00	7.32
○Thread 55649			6.92	7.67	90.11	1.07	0.00	7.32
○Thread 55650			6.74	7.67	87.77	1.07	0.00	7.32
○Thread 55651			6.73	7.67	87.64	1.07	0.00	7.32
○Thread 55652			6.77	7.67	88.16	1.07	0.00	7.31
○Thread 55653			6.83	7.67	88.94	1.07	0.00	7.31
○Thread 55654			6.89	7.67	89.79	1.07	0.00	7.32
○Thread 55655			6.84	7.67	89.14	1.07	0.00	7.32
○Thread 55656			6.85	7.67	89.21	1.07	0.00	7.32
○Thread 55657			6.79	7.67	88.43	1.07	0.00	7.32
○Thread 55658			6.84	7.67	89.08	1.07	0.00	7.32
○Thread 55659			6.91	7.67	89.99	1.07	0.00	7.31
○Thread 55660			6.71	7.67	87.45	1.07	0.00	7.31
○Thread 55661			6.86	7.67	89.41	1.07	0.00	7.31
○Thread 55662			6.82	7.67	88.89	1.07	0.00	7.31
○Thread 55663			6.81	7.67	88.69	1.07	0.00	7.31
○Thread 55664			6.89	7.67	89.87	1.07	0.00	7.32
○Thread 55665			6.82	7.67	88.83	1.07	0.00	7.31
○Thread 55666			6.91	7.67	90.06	1.07	0.00	7.32
○Thread 55667			6.81	7.67	88.70	1.07	0.00	7.32
○Thread 55668			6.81	7.67	88.70	1.07	0.00	7.31
○Thread 55669			6.82	7.67	88.90	1.07	0.00	7.31
○Thread 55670			6.75	7.67	87.98	1.07	0.00	7.31
○Thread 55671			6.90	7.67	90.01	1.07	0.00	7.31
○Thread 55672			6.91	7.67	90.07	1.07	0.00	7.31
○Thread 55673			6.82	7.67	88.90	1.07	0.00	7.31
○Thread 55674			6.83	7.67	89.03	1.07	0.00	7.32
○Thread 55675			6.85	7.67	89.29	1.07	0.00	7.31
○Thread 55676			6.92	7.67	90.27	1.07	0.00	7.32
○Thread 55677			6.90	7.67	90.01	1.07	0.00	7.31
○Thread 55678			6.82	7.67	88.91	1.07	0.00	7.31
○Thread 55679			6.77	7.67	88.26	1.07	0.00	7.31
○Thread 55680			6.88	7.67	89.69	1.07	0.00	7.31
○Thread 55681			6.83	7.67	88.97	1.07	-0.00	7.31

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	7.08
►Process 55711+		48	7.08
○Thread 55711			7.08	8.68	81.54	0.00	0.00	8.62
○Thread 55778			6.65	7.61	87.34	1.07	0.00	7.23
○Thread 55779			6.68	7.61	87.80	1.07	0.00	7.25
○Thread 55780			6.64	7.61	87.28	1.07	0.00	7.23
○Thread 55781			6.68	7.61	87.80	1.07	0.00	7.24
○Thread 55782			6.71	7.61	88.20	1.07	0.00	7.25
○Thread 55783			6.64	7.61	87.21	1.07	0.00	7.24
○Thread 55784			6.68	7.61	87.81	1.07	0.00	7.24
○Thread 55785			6.65	7.61	87.35	1.07	0.00	7.23
○Thread 55786			6.72	7.61	88.27	1.07	0.00	7.23
○Thread 55787			6.71	7.61	88.14	1.07	0.00	7.24
○Thread 55788			6.65	7.61	87.35	1.07	0.00	7.23
○Thread 55789			6.73	7.61	88.40	1.07	0.00	7.24
○Thread 55790			6.71	7.61	88.21	1.07	0.00	7.24
○Thread 55791			6.54	7.61	85.91	1.07	0.00	7.23
○Thread 55792			6.69	7.61	87.95	1.07	0.00	7.24
○Thread 55793			6.56	7.61	86.24	1.07	0.00	7.24
○Thread 55794			6.69	7.61	87.95	1.07	0.00	7.24
○Thread 55795			6.80	7.61	89.33	1.07	0.00	7.24
○Thread 55796			6.74	7.61	88.67	1.07	0.00	7.24
○Thread 55797			6.61	7.61	86.84	1.07	0.00	7.24
○Thread 55798			6.62	7.61	86.97	1.07	0.00	7.25
○Thread 55799			6.62	7.61	87.04	1.07	0.00	7.24
○Thread 55800			6.66	7.61	87.50	1.07	0.00	7.22
○Thread 55801			6.71	7.61	88.22	1.07	0.00	7.22
○Thread 55802			6.68	7.61	87.89	1.07	0.00	7.23
○Thread 55803			6.69	7.61	87.89	1.07	0.00	7.24
○Thread 55804			6.66	7.61	87.57	1.07	0.00	7.24
○Thread 55805			6.58	7.61	86.52	1.07	0.00	7.25
○Thread 55806			6.71	7.61	88.16	1.07	0.00	7.24
○Thread 55807			6.60	7.61	86.78	1.07	0.00	7.24
○Thread 55808			6.68	7.61	87.83	1.07	0.00	7.24
○Thread 55809			6.72	7.61	88.43	1.07	0.00	7.24
○Thread 55810			6.67	7.61	87.64	1.07	0.00	7.24
○Thread 55811			6.82	7.60	89.74	1.07	0.00	7.24
○Thread 55812			6.73	7.60	88.50	1.07	0.00	7.24
○Thread 55813			6.68	7.60	87.77	1.07	0.00	7.24
○Thread 55814			6.74	7.60	88.56	1.07	0.00	7.25
○Thread 55815			6.74	7.60	88.57	1.07	0.00	7.24
○Thread 55816			6.76	7.60	88.83	1.07	0.00	7.24
○Thread 55817			6.67	7.60	87.65	1.07	0.00	7.24
○Thread 55818			6.75	7.60	88.70	1.07	0.00	7.24
○Thread 55819			6.69	7.60	87.91	1.07	0.00	7.24
○Thread 55820			6.73	7.60	88.51	1.07	0.00	7.24
○Thread 55821			6.81	7.60	89.56	1.07	0.00	7.24
○Thread 55822			6.79	7.60	89.30	1.07	0.00	7.24
○Thread 55823			6.78	7.60	89.17	1.07	0.00	7.24
○Thread 55824			6.94	7.60	91.21	1.07	-0.00	7.24

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	6.82
►Process 55846+		56	6.82
○Thread 55846			6.82	8.51	80.10	0.00	0.00	8.47
○Thread 55913			6.69	7.46	89.67	1.06	0.00	7.08
○Thread 55914			6.53	7.45	87.53	1.06	0.00	7.08
○Thread 55915			6.51	7.45	87.33	1.06	0.00	7.08
○Thread 55916			6.46	7.45	86.59	1.06	0.00	7.07
○Thread 55917			6.40	7.45	85.79	1.06	0.00	7.07
○Thread 55918			6.43	7.45	86.19	1.06	0.00	7.08
○Thread 55919			6.49	7.45	87.06	1.06	0.00	7.07
○Thread 55920			6.52	7.45	87.40	1.06	0.00	7.08
○Thread 55921			6.55	7.45	87.80	1.06	0.00	7.08
○Thread 55922			6.48	7.45	86.87	1.06	0.00	7.07
○Thread 55923			6.45	7.45	86.53	1.06	0.00	7.08
○Thread 55924			6.40	7.45	85.79	1.06	0.00	7.08
○Thread 55925			6.40	7.45	85.86	1.06	0.00	7.08
○Thread 55926			6.52	7.45	87.47	1.06	0.00	7.08
○Thread 55927			6.47	7.45	86.80	1.06	0.00	7.07
○Thread 55928			6.40	7.45	85.80	1.06	0.00	7.08
○Thread 55929			6.50	7.45	87.21	1.06	0.00	7.07
○Thread 55930			6.40	7.45	85.87	1.06	0.00	7.08
○Thread 55931			6.42	7.45	86.14	1.06	0.00	7.07
○Thread 55932			6.48	7.45	86.94	1.06	-0.00	7.08
○Thread 55933			6.36	7.45	85.27	1.06	0.00	7.08
○Thread 55934			6.56	7.45	88.02	1.06	0.00	7.08
○Thread 55935			6.46	7.45	86.68	1.06	0.00	7.07
○Thread 55936			6.49	7.45	87.08	1.06	0.00	7.08
○Thread 55937			6.45	7.45	86.55	1.06	-0.00	7.07
○Thread 55938			6.46	7.45	86.62	1.06	0.00	7.08
○Thread 55939			6.48	7.45	86.89	1.06	0.00	7.08
○Thread 55940			6.41	7.45	86.01	1.06	-0.00	7.07
○Thread 55941			6.46	7.45	86.62	1.06	-0.00	7.08
○Thread 55942			6.45	7.45	86.55	1.06	0.00	7.08
○Thread 55943			6.39	7.45	85.68	1.06	-0.00	7.08
○Thread 55944			6.52	7.45	87.43	1.06	0.00	7.07
○Thread 55945			6.36	7.45	85.35	1.06	0.00	7.08
○Thread 55946			6.51	7.45	87.30	1.06	0.00	7.07
○Thread 55947			6.55	7.45	87.83	1.06	0.00	7.08
○Thread 55948			6.43	7.45	86.22	1.06	0.00	7.08
○Thread 55949			6.43	7.45	86.29	1.06	0.00	7.08
○Thread 55950			6.47	7.45	86.76	1.06	0.00	7.08
○Thread 55951			6.50	7.45	87.17	1.06	0.00	7.08
○Thread 55952			6.44	7.45	86.43	1.06	0.00	7.08
○Thread 55953			6.38	7.45	85.56	1.06	0.00	7.08
○Thread 55954			6.54	7.45	87.71	1.06	0.00	7.07
○Thread 55955			6.41	7.45	86.03	1.06	0.00	7.08
○Thread 55956			6.51	7.45	87.38	1.06	0.00	7.08
○Thread 55957			6.43	7.45	86.24	1.06	0.00	7.07
○Thread 55958			6.44	7.45	86.44	1.06	0.00	7.08
○Thread 55959			6.47	7.45	86.77	1.06	0.00	7.08
○Thread 55960			6.37	7.45	85.43	1.06	0.00	7.09
○Thread 55961			6.42	7.45	86.11	1.06	0.00	7.09
○Thread 55962			6.48	7.45	86.91	1.06	0.00	7.08
○Thread 55963			6.34	7.45	85.10	1.06	0.00	7.08
○Thread 55964			6.46	7.45	86.71	1.07	0.00	7.08
○Thread 55965			6.50	7.45	87.18	1.07	0.00	7.08
○Thread 55966			6.54	7.45	87.79	1.07	0.00	7.08
○Thread 55967			6.38	7.45	85.64	1.07	-0.00	7.07

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	6.51
►Process 56037+		64	6.51
○Thread 56037			6.51	8.33	78.18	0.00	0.00	8.29
○Thread 56104			6.13	7.26	84.44	1.07	-0.00	6.88
○Thread 56105			6.12	7.26	84.30	1.07	0.00	6.87
○Thread 56106			6.16	7.26	84.78	1.07	0.00	6.89
○Thread 56107			6.24	7.26	85.95	1.07	0.00	6.87
○Thread 56108			6.10	7.26	83.96	1.07	0.00	6.87
○Thread 56109			6.10	7.26	84.10	1.07	-0.00	6.87
○Thread 56110			6.23	7.26	85.75	1.07	-0.00	6.88
○Thread 56111			6.06	7.26	83.41	1.07	-0.00	6.88
○Thread 56112			6.19	7.26	85.20	1.07	-0.00	6.87
○Thread 56113			6.16	7.26	84.86	1.07	0.00	6.88
○Thread 56114			6.18	7.26	85.14	1.07	-0.00	6.88
○Thread 56115			6.16	7.26	84.86	1.07	-0.00	6.87
○Thread 56116			6.13	7.26	84.45	1.07	0.00	6.88
○Thread 56117			6.18	7.26	85.07	1.07	-0.00	6.88
○Thread 56118			6.06	7.26	83.56	1.08	0.00	6.88
○Thread 56119			6.14	7.26	84.52	1.08	-0.00	6.87
○Thread 56120			6.07	7.26	83.56	1.08	-0.00	6.88
○Thread 56121			6.08	7.26	83.70	1.08	-0.00	6.87
○Thread 56122			6.16	7.26	84.87	1.08	-0.00	6.88
○Thread 56123			6.14	7.26	84.60	1.08	-0.00	6.87
○Thread 56124			6.06	7.26	83.56	1.08	-0.00	6.87
○Thread 56125			6.16	7.26	84.94	1.08	-0.00	6.87
○Thread 56126			6.04	7.26	83.15	1.08	-0.00	6.88
○Thread 56127			6.05	7.26	83.36	1.08	0.00	6.87
○Thread 56128			6.16	7.26	84.81	1.08	-0.00	6.88
○Thread 56129			5.98	7.26	82.33	1.08	-0.00	6.87
○Thread 56130			6.05	7.26	83.30	1.08	-0.00	6.85
○Thread 56131			6.10	7.26	84.05	1.08	-0.00	6.86
○Thread 56132			5.96	7.26	82.13	1.08	-0.00	6.87
○Thread 56133			6.03	7.26	83.02	1.08	-0.00	6.86
○Thread 56134			6.11	7.26	84.20	1.08	-0.00	6.87
○Thread 56135			6.21	7.26	85.64	1.08	-0.00	6.87
○Thread 56136			6.21	7.26	85.58	1.08	-0.00	6.87
○Thread 56137			6.15	7.26	84.68	1.08	-0.00	6.87
○Thread 56138			6.38	7.26	87.85	1.08	-0.00	6.87
○Thread 56139			6.27	7.26	86.34	1.08	-0.00	6.87
○Thread 56140			6.08	7.26	83.72	1.08	-0.00	6.87
○Thread 56141			6.13	7.26	84.41	1.08	-0.00	6.87
○Thread 56142			6.04	7.26	83.17	1.08	-0.00	6.88
○Thread 56143			6.20	7.26	85.38	1.08	-0.00	6.89
○Thread 56144			6.19	7.26	85.31	1.08	-0.00	6.88
○Thread 56145			6.16	7.26	84.90	1.08	-0.00	6.87
○Thread 56146			6.09	7.26	83.87	1.08	-0.00	6.86
○Thread 56147			6.16	7.26	84.90	1.08	-0.00	6.88
○Thread 56148			6.13	7.26	84.49	1.08	-0.00	6.86
○Thread 56149			6.02	7.26	82.97	1.08	-0.00	6.87
○Thread 56150			6.16	7.26	84.90	1.08	-0.00	6.87
○Thread 56151			6.08	7.26	83.80	1.08	-0.00	6.87
○Thread 56152			6.09	7.26	83.94	1.08	-0.00	6.87
○Thread 56153			6.07	7.25	83.74	1.08	-0.00	6.87
○Thread 56154			6.11	7.25	84.22	1.08	0.00	6.87
○Thread 56155			6.10	7.25	84.01	1.08	0.00	6.87
○Thread 56156			6.22	7.25	85.67	1.08	0.00	6.87
○Thread 56157			6.08	7.25	83.81	1.08	0.00	6.88
○Thread 56158			6.23	7.25	85.95	1.08	0.00	6.88
○Thread 56159			6.14	7.25	84.57	1.08	0.00	6.88
○Thread 56160			6.14	7.25	84.64	1.08	0.00	6.88
○Thread 56161			6.07	7.25	83.61	1.08	0.00	6.87
○Thread 56162			6.04	7.25	83.19	1.08	0.00	6.87
○Thread 56163			6.11	7.25	84.23	1.08	0.00	6.87
○Thread 56164			6.06	7.25	83.47	1.08	0.00	6.88
○Thread 56165			6.11	7.25	84.30	1.08	0.00	6.87
○Thread 56166			6.18	7.25	85.20	1.08	-0.00	6.88

Report Configuration