OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	74.50
►Process 293552+		1	74.50
○Thread 293552			74.50	75.48	98.70	0.00	0.00	75.43

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	39.13
►Process 293721+		2	39.13
○Thread 293721			39.13	40.07	97.65	0.00	0.00	40.03
○Thread 293820			38.83	39.17	99.13	0.90	-0.00	38.85

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	20.33
►Process 293891+		4	20.33
○Thread 293891			20.33	21.26	95.65	0.00	0.00	21.21
○Thread 293990			20.02	20.36	98.34	0.90	-0.00	20.04
○Thread 293991			20.04	20.36	98.42	0.90	-0.00	20.04
○Thread 293992			20.03	20.36	98.37	0.90	-0.00	20.04

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	11.09
►Process 294013+		8	11.09
○Thread 294013			11.09	11.99	92.47	0.00	0.00	11.94
○Thread 294112			10.74	11.09	96.83	0.90	-0.00	10.77
○Thread 294113			10.74	11.09	96.88	0.90	-0.00	10.77
○Thread 294114			10.73	11.09	96.70	0.90	-0.00	10.77
○Thread 294115			10.74	11.09	96.84	0.90	-0.00	10.77
○Thread 294116			10.73	11.09	96.70	0.90	-0.00	10.77
○Thread 294117			10.72	11.09	96.66	0.90	-0.00	10.77
○Thread 294118			10.77	11.09	97.07	0.90	-0.00	10.76

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	6.44
►Process 294187+		16	6.44
○Thread 294187			6.44	7.49	85.95	0.00	0.00	7.45
○Thread 294286			6.17	6.60	93.52	0.90	-0.00	6.27
○Thread 294287			6.12	6.60	92.76	0.90	-0.00	6.28
○Thread 294288			6.18	6.60	93.60	0.90	-0.00	6.27
○Thread 294289			6.15	6.60	93.15	0.90	-0.00	6.27
○Thread 294290			6.24	6.60	94.51	0.90	-0.00	6.27
○Thread 294291			6.11	6.60	92.69	0.90	-0.00	6.27
○Thread 294292			6.17	6.60	93.61	0.90	-0.00	6.28
○Thread 294293			6.07	6.60	92.02	0.90	-0.00	6.27
○Thread 294294			6.15	6.60	93.23	0.90	-0.00	6.27
○Thread 294295			6.11	6.60	92.55	0.90	-0.00	6.27
○Thread 294296			6.15	6.60	93.16	0.90	0.00	6.27
○Thread 294297			6.14	6.60	93.01	0.90	0.00	6.27
○Thread 294298			6.07	6.60	92.02	0.90	0.00	6.27
○Thread 294299			6.13	6.60	92.93	0.90	0.00	6.28
○Thread 294300			6.13	6.60	92.94	0.90	-0.00	6.27

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	4.96
►Process 294320+		24	4.96
○Thread 294320			4.96	6.06	81.71	0.00	0.00	6.02
○Thread 294419			4.61	5.17	89.02	0.89	-0.00	4.85
○Thread 294420			4.64	5.17	89.79	0.89	-0.00	4.85
○Thread 294421			4.61	5.17	89.12	0.89	-0.00	4.85
○Thread 294422			4.64	5.17	89.70	0.89	-0.00	4.85
○Thread 294423			4.66	5.17	89.99	0.89	-0.00	4.85
○Thread 294424			4.61	5.17	89.03	0.89	-0.00	4.85
○Thread 294425			4.65	5.17	90.00	0.89	-0.00	4.85
○Thread 294426			4.60	5.17	88.93	0.89	-0.00	4.85
○Thread 294427			4.59	5.17	88.74	0.89	-0.00	4.85
○Thread 294428			4.66	5.17	90.20	0.89	-0.00	4.85
○Thread 294429			4.64	5.17	89.71	0.89	-0.00	4.85
○Thread 294430			4.60	5.17	88.94	0.89	-0.00	4.85
○Thread 294431			4.68	5.17	90.59	0.89	-0.00	4.85
○Thread 294432			4.68	5.17	90.59	0.89	-0.00	4.85
○Thread 294433			4.66	5.17	90.01	0.89	-0.00	4.85
○Thread 294434			4.75	5.17	91.85	0.89	-0.00	4.85
○Thread 294435			4.65	5.17	89.92	0.89	-0.00	4.85
○Thread 294436			4.68	5.17	90.60	0.89	0.00	4.85
○Thread 294437			4.81	5.17	93.02	0.89	-0.00	4.85
○Thread 294438			4.70	5.17	90.80	0.89	0.00	4.85
○Thread 294439			4.55	5.17	87.99	0.89	0.00	4.85
○Thread 294440			4.67	5.17	90.32	0.89	0.00	4.85
○Thread 294441			4.70	5.17	90.99	0.89	-0.00	4.85

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	4.25
►Process 294461+		32	4.25
○Thread 294461			4.25	5.46	77.86	0.00	0.00	5.42
○Thread 294560			3.98	4.56	87.27	0.90	-0.00	4.23
○Thread 294561			3.91	4.56	85.74	0.90	-0.00	4.23
○Thread 294562			3.97	4.56	87.16	0.90	-0.00	4.24
○Thread 294563			3.85	4.56	84.53	0.90	-0.00	4.23
○Thread 294564			3.97	4.56	87.06	0.90	-0.00	4.24
○Thread 294565			3.99	4.56	87.50	0.90	-0.00	4.23
○Thread 294566			4.05	4.56	88.71	0.90	-0.00	4.23
○Thread 294567			3.92	4.56	85.97	0.90	-0.00	4.24
○Thread 294568			3.99	4.56	87.50	0.90	-0.00	4.24
○Thread 294569			3.98	4.56	87.18	0.90	-0.00	4.23
○Thread 294570			4.03	4.56	88.28	0.90	-0.00	4.23
○Thread 294571			4.01	4.56	87.95	0.90	-0.00	4.24
○Thread 294572			3.91	4.56	85.76	0.90	-0.00	4.24
○Thread 294573			3.93	4.56	86.31	0.90	-0.00	4.23
○Thread 294574			4.03	4.56	88.50	0.90	-0.00	4.23
○Thread 294575			3.92	4.56	86.09	0.90	-0.00	4.23
○Thread 294576			3.96	4.56	86.86	0.90	-0.00	4.24
○Thread 294577			4.05	4.56	88.84	0.90	-0.00	4.23
○Thread 294578			4.14	4.56	90.71	0.90	-0.00	4.23
○Thread 294579			3.98	4.56	87.31	0.90	-0.00	4.23
○Thread 294580			4.01	4.56	87.86	0.90	-0.00	4.23
○Thread 294581			3.98	4.56	87.42	0.90	-0.00	4.23
○Thread 294582			4.01	4.56	88.08	0.90	-0.00	4.24
○Thread 294583			3.91	4.56	85.78	0.90	-0.00	4.23
○Thread 294584			3.92	4.56	86.00	0.90	-0.00	4.22
○Thread 294585			3.93	4.56	86.34	0.90	-0.00	4.23
○Thread 294586			4.02	4.56	88.20	0.90	-0.00	4.23
○Thread 294587			3.96	4.56	86.89	0.90	-0.00	4.23
○Thread 294588			3.96	4.56	86.89	0.90	-0.00	4.22
○Thread 294589			3.97	4.56	87.00	0.90	-0.00	4.23
○Thread 294590			3.98	4.56	87.33	0.90	-0.00	4.23

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	4.03
►Process 294610+		40	4.03
○Thread 294610			4.03	5.18	77.95	0.00	0.00	5.12
○Thread 294709			3.66	4.29	85.28	0.89	-0.00	3.95
○Thread 294710			3.63	4.29	84.58	0.89	-0.00	3.96
○Thread 294711			3.60	4.29	84.00	0.89	-0.00	3.96
○Thread 294712			3.68	4.29	85.99	0.89	-0.00	3.96
○Thread 294713			3.63	4.29	84.82	0.89	-0.00	3.96
○Thread 294714			3.56	4.29	83.07	0.89	-0.00	3.96
○Thread 294715			3.63	4.29	84.83	0.89	-0.00	3.96
○Thread 294716			3.73	4.29	87.05	0.89	-0.00	3.96
○Thread 294717			3.67	4.28	85.65	0.89	-0.00	3.96
○Thread 294718			3.67	4.28	85.77	0.89	-0.00	3.96
○Thread 294719			3.64	4.28	84.95	0.89	-0.00	3.96
○Thread 294720			3.71	4.28	86.59	0.89	-0.00	3.95
○Thread 294721			3.64	4.28	84.96	0.89	-0.00	3.96
○Thread 294722			3.57	4.28	83.44	0.89	-0.00	3.96
○Thread 294723			3.56	4.28	83.09	0.89	-0.00	3.96
○Thread 294724			3.70	4.28	86.25	0.89	-0.00	3.96
○Thread 294725			3.58	4.28	83.45	0.89	-0.00	3.96
○Thread 294726			3.58	4.28	83.57	0.89	-0.00	3.96
○Thread 294727			3.56	4.28	83.22	0.89	-0.00	3.96
○Thread 294728			3.58	4.28	83.57	0.89	-0.00	3.96
○Thread 294729			3.57	4.28	83.34	0.89	-0.00	3.96
○Thread 294730			3.78	4.28	88.25	0.89	-0.00	3.95
○Thread 294731			3.57	4.28	83.46	0.89	-0.00	3.95
○Thread 294732			3.66	4.28	85.45	0.89	-0.00	3.96
○Thread 294733			3.61	4.28	84.40	0.89	-0.00	3.96
○Thread 294734			3.57	4.28	83.35	0.89	-0.00	3.96
○Thread 294735			3.59	4.28	83.82	0.89	-0.00	3.96
○Thread 294736			3.69	4.28	86.04	0.89	-0.00	3.96
○Thread 294737			3.55	4.28	82.89	0.89	-0.00	3.96
○Thread 294738			3.60	4.28	84.18	0.89	-0.00	3.96
○Thread 294739			3.66	4.28	85.46	0.89	-0.00	3.96
○Thread 294740			3.63	4.28	84.77	0.89	-0.00	3.96
○Thread 294741			3.59	4.28	83.83	0.89	-0.00	3.96
○Thread 294742			3.70	4.28	86.52	0.89	-0.00	3.96
○Thread 294743			3.58	4.28	83.72	0.89	-0.00	3.96
○Thread 294744			3.74	4.28	87.46	0.89	-0.00	3.96
○Thread 294745			3.65	4.28	85.36	0.89	-0.00	3.95
○Thread 294746			3.72	4.28	87.00	0.89	-0.00	3.96
○Thread 294747			3.67	4.28	85.83	0.89	-0.00	3.96

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	4.00
►Process 294816+		48	4.00
○Thread 294816			4.00	5.20	76.89	0.00	0.00	5.15
○Thread 294915			3.68	4.31	85.29	0.89	-0.00	3.99
○Thread 294916			3.68	4.31	85.41	0.89	-0.00	3.99
○Thread 294917			3.55	4.31	82.40	0.89	-0.00	3.99
○Thread 294918			3.55	4.31	82.40	0.89	-0.00	3.98
○Thread 294919			3.50	4.31	81.13	0.89	-0.00	3.98
○Thread 294920			3.63	4.31	84.14	0.89	-0.00	3.98
○Thread 294921			3.61	4.31	83.80	0.89	-0.00	3.98
○Thread 294922			3.69	4.31	85.54	0.89	-0.00	3.97
○Thread 294923			3.55	4.31	82.18	0.89	-0.00	3.99
○Thread 294924			3.55	4.31	82.18	0.89	-0.00	3.99
○Thread 294925			3.52	4.31	81.61	0.89	-0.00	3.98
○Thread 294926			3.60	4.31	83.46	0.89	-0.00	3.98
○Thread 294927			3.61	4.31	83.70	0.89	-0.00	3.98
○Thread 294928			3.68	4.31	85.21	0.89	-0.00	3.98
○Thread 294929			3.58	4.31	82.89	0.89	-0.00	3.97
○Thread 294930			3.55	4.31	82.31	0.89	-0.00	3.99
○Thread 294931			3.56	4.31	82.66	0.89	-0.00	3.99
○Thread 294932			3.59	4.31	83.36	0.89	-0.00	3.99
○Thread 294933			3.60	4.31	83.59	0.89	-0.00	3.97
○Thread 294934			3.59	4.31	83.13	0.89	-0.00	3.99
○Thread 294935			3.61	4.31	83.71	0.89	-0.00	3.98
○Thread 294936			3.59	4.31	83.14	0.89	-0.00	3.98
○Thread 294937			3.62	4.31	84.07	0.89	-0.00	3.99
○Thread 294938			3.64	4.31	84.42	0.89	-0.00	3.98
○Thread 294939			3.60	4.31	83.49	0.89	-0.00	3.99
○Thread 294940			3.59	4.31	83.26	0.89	-0.00	3.99
○Thread 294941			3.57	4.31	82.68	0.89	-0.00	3.99
○Thread 294942			3.61	4.31	83.73	0.89	-0.00	3.98
○Thread 294943			3.64	4.31	84.54	0.89	-0.00	3.97
○Thread 294944			3.57	4.31	82.92	0.89	-0.00	3.97
○Thread 294945			3.67	4.31	85.13	0.89	-0.00	3.98
○Thread 294946			3.74	4.31	86.87	0.89	-0.00	3.98
○Thread 294947			3.60	4.31	83.51	0.89	-0.00	3.98
○Thread 294948			3.67	4.31	85.13	0.89	-0.00	3.98
○Thread 294949			3.62	4.31	83.98	0.89	-0.00	3.99
○Thread 294950			3.63	4.31	84.33	0.89	-0.00	3.98
○Thread 294951			3.63	4.31	84.10	0.89	-0.00	3.98
○Thread 294952			3.63	4.31	84.33	0.89	-0.00	3.97
○Thread 294953			3.66	4.31	84.91	0.89	-0.00	3.99
○Thread 294954			3.57	4.31	82.83	0.89	-0.00	3.98
○Thread 294955			3.57	4.31	82.83	0.89	-0.00	3.98
○Thread 294956			3.64	4.31	84.46	0.89	-0.00	3.99
○Thread 294957			3.51	4.31	81.44	0.89	-0.00	3.98
○Thread 294958			3.63	4.31	84.34	0.89	-0.00	3.99
○Thread 294959			3.63	4.31	84.23	0.89	-0.00	3.98
○Thread 294960			3.60	4.31	83.54	0.89	-0.00	3.99
○Thread 294961			3.60	4.31	83.42	0.89	-0.00	3.98

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	3.93
►Process 294982+		56	3.93
○Thread 294982			3.93	5.25	74.89	0.00	0.00	5.22
○Thread 295081			3.56	4.35	81.79	0.90	-0.00	4.02
○Thread 295082			3.51	4.35	80.76	0.90	-0.00	4.00
○Thread 295083			3.60	4.35	82.60	0.90	-0.00	4.02
○Thread 295084			3.47	4.35	79.61	0.90	-0.00	4.02
○Thread 295085			3.51	4.35	80.65	0.90	-0.00	4.02
○Thread 295086			3.50	4.35	80.42	0.90	-0.00	4.02
○Thread 295087			3.40	4.35	78.24	0.90	-0.00	4.01
○Thread 295088			3.55	4.35	81.58	0.90	-0.00	4.01
○Thread 295089			3.60	4.35	82.73	0.90	-0.00	4.01
○Thread 295090			3.58	4.35	82.16	0.90	-0.00	4.01
○Thread 295091			3.54	4.35	81.24	0.90	-0.00	4.01
○Thread 295092			3.48	4.35	80.09	0.90	-0.00	4.02
○Thread 295093			3.51	4.35	80.67	0.90	-0.00	4.01
○Thread 295094			3.63	4.35	83.54	0.90	-0.00	4.01
○Thread 295095			3.61	4.35	83.08	0.90	-0.00	4.01
○Thread 295096			3.56	4.35	81.82	0.90	-0.00	4.01
○Thread 295097			3.51	4.35	80.68	0.90	-0.00	4.02
○Thread 295098			3.56	4.35	81.71	0.90	-0.00	4.02
○Thread 295099			3.37	4.35	77.46	0.90	-0.00	4.02
○Thread 295100			3.50	4.35	80.45	0.90	-0.00	4.01
○Thread 295101			3.50	4.35	80.57	0.90	-0.00	4.01
○Thread 295102			3.49	4.35	80.11	0.90	-0.00	4.01
○Thread 295103			3.56	4.35	81.84	0.90	-0.00	4.01
○Thread 295104			3.60	4.35	82.87	0.90	-0.00	4.01
○Thread 295105			3.47	4.35	79.77	0.90	-0.00	4.00
○Thread 295106			3.53	4.35	81.15	0.90	-0.00	4.01
○Thread 295107			3.58	4.35	82.19	0.90	-0.00	4.00
○Thread 295108			3.50	4.35	80.47	0.91	-0.00	4.01
○Thread 295109			3.47	4.35	79.90	0.91	-0.00	4.00
○Thread 295110			3.59	4.35	82.66	0.91	-0.00	4.01
○Thread 295111			3.50	4.35	80.59	0.91	-0.00	4.01
○Thread 295112			3.55	4.35	81.74	0.91	-0.00	4.01
○Thread 295113			3.56	4.35	81.97	0.91	-0.00	4.02
○Thread 295114			3.54	4.35	81.52	0.91	-0.00	4.02
○Thread 295115			3.52	4.35	81.06	0.91	-0.00	4.01
○Thread 295116			3.60	4.35	82.79	0.91	-0.00	4.01
○Thread 295117			3.54	4.35	81.41	0.91	-0.00	4.01
○Thread 295118			3.63	4.35	83.59	0.91	-0.00	4.01
○Thread 295119			3.57	4.35	82.22	0.91	-0.00	4.00
○Thread 295120			3.63	4.35	83.48	0.91	-0.00	4.01
○Thread 295121			3.50	4.35	80.50	0.91	-0.00	4.01
○Thread 295122			3.50	4.35	80.61	0.91	-0.00	4.02
○Thread 295123			3.50	4.35	80.50	0.91	-0.00	4.02
○Thread 295124			3.57	4.35	82.23	0.91	-0.00	4.01
○Thread 295125			3.48	4.35	80.16	0.91	-0.00	4.01
○Thread 295126			3.43	4.35	78.90	0.91	-0.00	4.01
○Thread 295127			3.44	4.35	79.24	0.91	-0.00	4.01
○Thread 295128			3.52	4.35	81.09	0.91	-0.00	4.01
○Thread 295129			3.50	4.35	80.63	0.91	-0.00	4.01
○Thread 295130			3.51	4.35	80.75	0.91	-0.00	4.01
○Thread 295131			3.59	4.35	82.70	0.91	-0.00	4.01
○Thread 295132			3.48	4.35	80.06	0.91	-0.00	4.02
○Thread 295133			3.59	4.35	82.71	0.91	-0.00	4.01
○Thread 295134			3.51	4.35	80.64	0.91	-0.00	4.01
○Thread 295135			3.53	4.35	81.22	0.91	-0.00	4.00

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	3.74
►Process 295155+		64	3.74
○Thread 295155			3.74	5.22	71.65	0.00	0.00	5.18
○Thread 295254			3.47	4.32	80.48	0.90	-0.00	3.97
○Thread 295255			3.45	4.32	79.90	0.90	-0.00	3.96
○Thread 295256			3.34	4.32	77.36	0.90	-0.00	3.97
○Thread 295257			3.32	4.32	76.90	0.90	-0.00	3.97
○Thread 295258			3.42	4.32	79.33	0.90	-0.00	3.96
○Thread 295259			3.30	4.32	76.55	0.90	-0.00	3.97
○Thread 295260			3.39	4.32	78.52	0.90	-0.00	3.97
○Thread 295261			3.47	4.32	80.50	0.90	-0.00	3.97
○Thread 295262			3.34	4.32	77.37	0.90	-0.00	3.97
○Thread 295263			3.38	4.32	78.30	0.90	-0.00	3.96
○Thread 295264			3.39	4.32	78.53	0.90	-0.00	3.97
○Thread 295265			3.39	4.32	78.65	0.90	-0.00	3.96
○Thread 295266			3.37	4.32	78.07	0.90	-0.00	3.97
○Thread 295267			3.45	4.32	79.93	0.90	-0.00	3.97
○Thread 295268			3.36	4.32	77.96	0.90	-0.00	3.97
○Thread 295269			3.28	4.32	76.11	0.90	-0.00	3.97
○Thread 295270			3.40	4.32	78.89	0.90	-0.00	3.97
○Thread 295271			3.30	4.32	76.35	0.90	-0.00	3.96
○Thread 295272			3.36	4.32	77.97	0.90	-0.00	3.97
○Thread 295273			3.43	4.32	79.48	0.90	-0.00	3.97
○Thread 295274			3.23	4.32	74.96	0.90	-0.00	3.97
○Thread 295275			3.33	4.32	77.28	0.90	-0.00	3.97
○Thread 295276			3.30	4.32	76.59	0.90	-0.00	3.96
○Thread 295277			3.35	4.32	77.63	0.90	-0.00	3.97
○Thread 295278			3.38	4.32	78.33	0.90	-0.00	3.96
○Thread 295279			3.45	4.31	79.95	0.90	-0.00	3.97
○Thread 295280			3.25	4.31	75.44	0.90	-0.00	3.96
○Thread 295281			3.28	4.31	76.13	0.90	-0.00	3.97
○Thread 295282			3.35	4.31	77.64	0.91	-0.00	3.96
○Thread 295283			3.42	4.31	79.27	0.91	-0.00	3.97
○Thread 295284			3.39	4.31	78.57	0.91	-0.00	3.96
○Thread 295285			3.43	4.31	79.39	0.91	-0.00	3.96
○Thread 295286			3.43	4.31	79.62	0.91	-0.00	3.97
○Thread 295287			3.44	4.31	79.86	0.91	-0.00	3.97
○Thread 295288			3.32	4.31	76.96	0.91	-0.00	3.97
○Thread 295289			3.33	4.31	77.19	0.91	-0.00	3.97
○Thread 295290			3.44	4.31	79.86	0.91	-0.00	3.97
○Thread 295291			3.22	4.31	74.65	0.91	-0.00	3.97
○Thread 295292			3.36	4.31	78.01	0.91	-0.00	3.97
○Thread 295293			3.37	4.31	78.13	0.91	-0.00	3.97
○Thread 295294			3.29	4.31	76.16	0.91	-0.00	3.97
○Thread 295295			3.52	4.31	81.73	0.91	-0.00	3.97
○Thread 295296			3.33	4.31	77.21	0.91	-0.00	3.96
○Thread 295297			3.48	4.31	80.80	0.91	-0.00	3.97
○Thread 295298			3.50	4.31	81.27	0.91	-0.00	3.97
○Thread 295299			3.41	4.31	79.07	0.91	-0.00	3.97
○Thread 295300			3.34	4.31	77.45	0.91	-0.00	3.96
○Thread 295301			3.37	4.31	78.26	0.91	-0.00	3.97
○Thread 295302			3.32	4.31	77.10	0.91	-0.00	3.97
○Thread 295303			3.33	4.31	77.34	0.91	-0.00	3.96
○Thread 295304			3.34	4.31	77.46	0.91	-0.00	3.97
○Thread 295305			3.39	4.31	78.73	0.91	-0.00	3.97
○Thread 295306			3.30	4.31	76.42	0.91	-0.00	3.97
○Thread 295307			3.37	4.31	78.27	0.91	-0.00	3.97
○Thread 295308			3.32	4.31	77.12	0.91	-0.00	3.96
○Thread 295309			3.39	4.31	78.63	0.91	-0.00	3.97
○Thread 295310			3.40	4.31	78.86	0.91	-0.00	3.96
○Thread 295311			3.35	4.31	77.82	0.91	-0.00	3.97
○Thread 295312			3.42	4.31	79.44	0.91	-0.00	3.97
○Thread 295313			3.34	4.31	77.47	0.91	-0.00	3.97
○Thread 295314			3.30	4.31	76.55	0.91	-0.00	3.97
○Thread 295315			3.39	4.31	78.75	0.91	-0.00	3.96
○Thread 295316			3.30	4.31	76.67	0.91	-0.00	3.97

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	3.96
►Process 295336+		72	3.96
○Thread 295336			3.91	5.49	71.19	0.00	0.00	5.44
○Thread 295435			3.68	4.59	80.18	0.90	-0.00	4.25
○Thread 295436			3.74	4.59	81.60	0.90	-0.00	4.26
○Thread 295437			3.96	4.59	86.29	0.90	-0.00	4.26
○Thread 295438			3.65	4.59	79.53	0.90	-0.00	4.24
○Thread 295439			3.65	4.59	79.54	0.90	-0.00	4.26
○Thread 295440			3.66	4.59	79.76	0.90	-0.00	4.26
○Thread 295441			3.72	4.59	81.17	0.90	-0.00	4.26
○Thread 295442			3.74	4.59	81.61	0.90	-0.00	4.25
○Thread 295443			3.86	4.59	84.12	0.90	-0.00	4.25
○Thread 295444			3.65	4.59	79.66	0.90	-0.00	4.24
○Thread 295445			3.70	4.59	80.64	0.90	-0.00	4.26
○Thread 295446			3.57	4.59	77.81	0.90	-0.00	4.26
○Thread 295447			3.60	4.59	78.57	0.90	-0.00	4.26
○Thread 295448			3.70	4.59	80.64	0.90	-0.00	4.25
○Thread 295449			3.86	4.59	84.24	0.90	-0.00	4.25
○Thread 295450			3.61	4.59	78.80	0.90	-0.00	4.25
○Thread 295451			3.63	4.59	79.12	0.90	-0.00	4.25
○Thread 295452			3.63	4.59	79.13	0.90	-0.00	4.26
○Thread 295453			3.55	4.59	77.49	0.90	-0.00	4.25
○Thread 295454			3.63	4.59	79.13	0.90	-0.00	4.25
○Thread 295455			3.56	4.59	77.72	0.91	-0.00	4.25
○Thread 295456			3.60	4.59	78.48	0.91	-0.00	4.26
○Thread 295457			3.51	4.59	76.52	0.91	-0.00	4.25
○Thread 295458			3.55	4.59	77.39	0.91	-0.00	4.26
○Thread 295459			3.53	4.59	77.07	0.91	-0.00	4.25
○Thread 295460			3.56	4.59	77.62	0.91	-0.00	4.25
○Thread 295461			3.58	4.59	78.05	0.91	-0.00	4.26
○Thread 295462			3.49	4.59	76.20	0.91	-0.00	4.26
○Thread 295463			3.60	4.59	78.49	0.91	-0.00	4.25
○Thread 295464			3.63	4.59	79.26	0.91	-0.00	4.25
○Thread 295465			3.55	4.59	77.52	0.91	-0.00	4.26
○Thread 295466			3.59	4.59	78.39	0.91	-0.00	4.26
○Thread 295467			3.54	4.59	77.19	0.91	-0.00	4.25
○Thread 295468			3.59	4.59	78.29	0.91	-0.00	4.25
○Thread 295469			3.64	4.59	79.38	0.91	-0.00	4.26
○Thread 295470			3.52	4.59	76.76	0.91	-0.00	4.25
○Thread 295471			3.57	4.59	77.86	0.91	-0.00	4.25
○Thread 295472			3.49	4.59	76.22	0.91	-0.00	4.25
○Thread 295473			3.66	4.59	79.93	0.91	-0.00	4.25
○Thread 295474			3.63	4.59	79.28	0.91	-0.00	4.26
○Thread 295475			3.54	4.58	77.32	0.91	-0.00	4.25
○Thread 295476			3.67	4.58	80.15	0.91	-0.00	4.26
○Thread 295477			3.48	4.58	75.90	0.91	-0.00	4.26
○Thread 295478			3.55	4.58	77.54	0.91	-0.00	4.25
○Thread 295479			3.54	4.58	77.33	0.91	-0.00	4.25
○Thread 295480			3.51	4.58	76.67	0.91	-0.00	4.25
○Thread 295481			3.57	4.58	77.87	0.91	-0.00	4.25
○Thread 295482			3.61	4.58	78.86	0.91	-0.00	4.26
○Thread 295483			3.56	4.58	77.66	0.91	-0.00	4.25
○Thread 295484			3.55	4.58	77.44	0.91	-0.00	4.24
○Thread 295485			3.58	4.58	78.10	0.91	-0.00	4.26
○Thread 295486			3.40	4.58	74.28	0.91	-0.00	4.26
○Thread 295487			3.58	4.58	78.10	0.91	-0.00	4.24
○Thread 295488			3.49	4.58	76.25	0.91	-0.00	4.26
○Thread 295489			3.55	4.58	77.56	0.91	-0.00	4.26
○Thread 295490			3.52	4.58	76.91	0.91	-0.00	4.26
○Thread 295491			3.52	4.58	76.91	0.91	-0.00	4.25
○Thread 295492			3.47	4.58	75.82	0.91	-0.00	4.26
○Thread 295493			3.46	4.58	75.61	0.91	-0.00	4.25
○Thread 295494			3.61	4.58	78.77	0.91	-0.00	4.25
○Thread 295495			3.55	4.58	77.57	0.91	-0.00	4.26
○Thread 295496			3.64	4.58	79.54	0.91	-0.00	4.25
○Thread 295497			3.62	4.58	79.11	0.91	-0.00	4.25
○Thread 295498			3.56	4.58	77.69	0.91	-0.00	4.26
○Thread 295499			3.57	4.58	77.91	0.91	-0.00	4.25
○Thread 295500			3.60	4.58	78.57	0.91	-0.00	4.25
○Thread 295501			3.49	4.58	76.28	0.91	-0.00	4.25
○Thread 295502			3.50	4.58	76.50	0.91	-0.00	4.25
○Thread 295503			3.56	4.58	77.81	0.91	-0.00	4.25
○Thread 295504			3.57	4.58	78.03	0.91	-0.00	4.25
○Thread 295505			3.56	4.58	77.81	0.91	-0.00	4.25

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	3.68
►Process 295574+		80	3.68
○Thread 295574			3.68	5.31	69.30	0.00	0.00	5.27
○Thread 295673			3.40	4.41	77.06	0.90	-0.00	4.07
○Thread 295674			3.52	4.41	79.78	0.90	-0.00	4.07
○Thread 295675			3.57	4.41	81.03	0.90	-0.00	4.07
○Thread 295676			3.49	4.41	79.11	0.90	-0.00	4.07
○Thread 295677			3.29	4.41	74.69	0.90	-0.00	4.06
○Thread 295678			3.31	4.41	75.14	0.90	-0.00	4.08
○Thread 295679			3.38	4.41	76.62	0.90	-0.00	4.07
○Thread 295680			3.30	4.41	74.92	0.90	-0.00	4.07
○Thread 295681			3.36	4.41	76.28	0.90	-0.00	4.07
○Thread 295682			3.23	4.41	73.34	0.90	-0.00	4.06
○Thread 295683			3.36	4.41	76.17	0.90	-0.00	4.07
○Thread 295684			3.34	4.41	75.84	0.90	-0.00	4.06
○Thread 295685			3.31	4.41	74.93	0.90	-0.00	4.07
○Thread 295686			3.35	4.41	76.07	0.90	-0.00	4.04
○Thread 295687			3.39	4.41	76.98	0.90	-0.00	4.07
○Thread 295688			3.37	4.41	76.41	0.90	-0.00	4.07
○Thread 295689			3.23	4.41	73.24	0.90	-0.00	4.07
○Thread 295690			3.31	4.41	75.05	0.90	-0.00	4.05
○Thread 295691			3.27	4.41	74.15	0.90	-0.00	4.07
○Thread 295692			3.37	4.41	76.53	0.90	-0.00	4.06
○Thread 295693			3.32	4.41	75.40	0.90	-0.00	4.07
○Thread 295694			3.34	4.41	75.86	0.90	-0.00	4.07
○Thread 295695			3.23	4.41	73.36	0.90	-0.00	4.08
○Thread 295696			3.24	4.41	73.48	0.90	-0.00	4.07
○Thread 295697			3.31	4.41	75.18	0.90	-0.00	4.07
○Thread 295698			3.29	4.41	74.62	0.90	-0.00	4.06
○Thread 295699			3.36	4.41	76.32	0.90	-0.00	4.06
○Thread 295700			3.38	4.41	76.66	0.90	-0.00	4.07
○Thread 295701			3.25	4.41	73.83	0.90	-0.00	4.06
○Thread 295702			3.27	4.41	74.28	0.90	-0.00	4.07
○Thread 295703			3.33	4.41	75.53	0.90	-0.00	4.07
○Thread 295704			3.33	4.41	75.54	0.90	-0.00	4.07
○Thread 295705			3.38	4.41	76.78	0.90	-0.00	4.07
○Thread 295706			3.37	4.41	76.56	0.90	-0.00	4.08
○Thread 295707			3.31	4.41	75.09	0.90	-0.00	4.07
○Thread 295708			3.36	4.41	76.22	0.90	-0.00	4.07
○Thread 295709			3.25	4.41	73.84	0.90	-0.00	4.07
○Thread 295710			3.28	4.41	74.53	0.90	-0.00	4.07
○Thread 295711			3.27	4.41	74.19	0.90	-0.00	4.07
○Thread 295712			3.31	4.41	75.21	0.90	-0.00	4.07
○Thread 295713			3.36	4.41	76.23	0.90	-0.00	4.07
○Thread 295714			3.30	4.41	74.99	0.90	-0.00	4.07
○Thread 295715			3.31	4.41	75.22	0.90	-0.00	4.07
○Thread 295716			3.30	4.41	74.76	0.90	-0.00	4.07
○Thread 295717			3.24	4.41	73.63	0.90	-0.00	4.06
○Thread 295718			3.31	4.41	75.22	0.90	-0.00	4.07
○Thread 295719			3.35	4.41	76.13	0.90	-0.00	4.08
○Thread 295720			3.32	4.41	75.34	0.90	-0.00	4.08
○Thread 295721			3.33	4.41	75.68	0.90	-0.00	4.06
○Thread 295722			3.21	4.41	72.85	0.90	-0.00	4.07
○Thread 295723			3.33	4.41	75.69	0.90	-0.00	4.05
○Thread 295724			3.32	4.41	75.46	0.90	-0.00	4.06
○Thread 295725			3.28	4.41	74.44	0.90	-0.00	4.07
○Thread 295726			3.23	4.41	73.31	0.90	-0.00	4.07
○Thread 295727			3.12	4.41	70.81	0.90	-0.00	4.07
○Thread 295728			3.28	4.41	74.56	0.90	-0.00	4.07
○Thread 295729			3.29	4.41	74.79	0.90	-0.00	4.07
○Thread 295730			3.35	4.41	76.04	0.90	-0.00	4.06
○Thread 295731			3.31	4.41	75.25	0.90	-0.00	4.07
○Thread 295732			3.32	4.41	75.48	0.90	-0.00	4.07
○Thread 295733			3.39	4.41	76.95	0.91	-0.00	4.06
○Thread 295734			3.25	4.41	73.78	0.91	-0.00	4.07
○Thread 295735			3.20	4.41	72.64	0.91	-0.00	4.07
○Thread 295736			3.33	4.40	75.48	0.91	-0.00	4.07
○Thread 295737			3.35	4.40	76.17	0.91	-0.00	4.07
○Thread 295738			3.21	4.40	72.99	0.91	-0.00	4.07
○Thread 295739			3.22	4.40	73.22	0.91	-0.00	4.08
○Thread 295740			3.38	4.40	76.85	0.91	-0.00	4.07
○Thread 295741			3.31	4.40	75.15	0.91	-0.00	4.06
○Thread 295742			3.40	4.40	77.20	0.91	-0.00	4.07
○Thread 295743			3.28	4.40	74.59	0.91	-0.00	4.07
○Thread 295744			3.26	4.40	73.91	0.91	-0.00	4.06
○Thread 295745			3.22	4.40	73.12	0.91	-0.00	4.07
○Thread 295746			3.34	4.40	75.96	0.91	-0.00	4.07
○Thread 295747			3.31	4.40	75.16	0.91	-0.00	4.07
○Thread 295748			3.35	4.40	76.07	0.91	-0.00	4.07
○Thread 295749			3.31	4.40	75.28	0.91	-0.00	4.07
○Thread 295750			3.36	4.40	76.42	0.91	-0.00	4.07
○Thread 295751			3.21	4.40	72.90	0.91	-0.00	4.08

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	4.18
►Process 295771+		88	4.18
○Thread 295771			4.18	5.54	75.58	0.00	0.00	5.50
○Thread 295870			3.80	4.64	81.97	0.90	-0.00	4.30
○Thread 295871			3.82	4.64	82.51	0.90	-0.00	4.30
○Thread 295872			3.65	4.64	78.84	0.90	-0.00	4.30
○Thread 295873			3.57	4.64	77.01	0.90	-0.00	4.30
○Thread 295874			3.59	4.64	77.55	0.90	-0.00	4.30
○Thread 295875			3.77	4.64	81.44	0.90	-0.00	4.30
○Thread 295876			3.63	4.64	78.42	0.90	-0.00	4.29
○Thread 295877			3.57	4.64	77.13	0.90	-0.00	4.30
○Thread 295878			3.54	4.64	76.37	0.90	-0.00	4.29
○Thread 295879			3.51	4.63	75.84	0.90	-0.00	4.29
○Thread 295880			3.51	4.63	75.73	0.90	-0.00	4.30
○Thread 295881			3.50	4.63	75.62	0.90	-0.00	4.30
○Thread 295882			3.53	4.63	76.27	0.90	-0.00	4.29
○Thread 295883			3.56	4.63	76.82	0.90	-0.00	4.30
○Thread 295884			3.63	4.63	78.44	0.90	-0.00	4.29
○Thread 295885			3.56	4.63	76.93	0.90	-0.00	4.30
○Thread 295886			3.61	4.63	78.01	0.90	-0.00	4.30
○Thread 295887			3.56	4.63	76.82	0.90	-0.00	4.30
○Thread 295888			3.53	4.63	76.29	0.90	-0.00	4.30
○Thread 295889			3.52	4.63	76.07	0.90	-0.00	4.30
○Thread 295890			3.57	4.63	77.04	0.90	-0.00	4.29
○Thread 295891			3.59	4.63	77.59	0.90	-0.00	4.30
○Thread 295892			3.52	4.63	76.08	0.90	-0.00	4.29
○Thread 295893			3.54	4.63	76.51	0.90	-0.00	4.30
○Thread 295894			3.47	4.63	75.00	0.90	-0.00	4.29
○Thread 295895			3.45	4.63	74.57	0.90	-0.00	4.30
○Thread 295896			3.65	4.63	78.78	0.90	-0.00	4.29
○Thread 295897			3.49	4.63	75.44	0.90	-0.00	4.31
○Thread 295898			3.53	4.63	76.30	0.90	-0.00	4.30
○Thread 295899			3.48	4.63	75.12	0.90	-0.00	4.30
○Thread 295900			3.58	4.63	77.28	0.90	-0.00	4.30
○Thread 295901			3.59	4.63	77.50	0.90	-0.00	4.30
○Thread 295902			3.56	4.63	76.85	0.90	-0.00	4.29
○Thread 295903			3.52	4.63	76.10	0.90	-0.00	4.30
○Thread 295904			3.48	4.63	75.24	0.90	-0.00	4.30
○Thread 295905			3.53	4.63	76.32	0.90	-0.00	4.30
○Thread 295906			3.49	4.63	75.46	0.91	-0.00	4.30
○Thread 295907			3.49	4.63	75.46	0.91	-0.00	4.29
○Thread 295908			3.45	4.63	74.60	0.91	-0.00	4.30
○Thread 295909			3.57	4.63	77.08	0.91	-0.00	4.30
○Thread 295910			3.55	4.63	76.65	0.91	-0.00	4.30
○Thread 295911			3.54	4.63	76.44	0.91	-0.00	4.30
○Thread 295912			3.50	4.63	75.57	0.91	-0.00	4.30
○Thread 295913			3.36	4.63	72.55	0.91	-0.00	4.31
○Thread 295914			3.45	4.63	74.50	0.91	-0.00	4.30
○Thread 295915			3.41	4.63	73.74	0.91	-0.00	4.30
○Thread 295916			3.55	4.63	76.77	0.91	-0.00	4.30
○Thread 295917			3.53	4.63	76.23	0.91	-0.00	4.30
○Thread 295918			3.60	4.63	77.85	0.91	-0.00	4.30
○Thread 295919			3.62	4.63	78.29	0.91	-0.00	4.30
○Thread 295920			3.54	4.63	76.56	0.91	-0.00	4.30
○Thread 295921			3.54	4.63	76.56	0.91	-0.00	4.31
○Thread 295922			3.52	4.63	76.03	0.91	-0.00	4.30
○Thread 295923			3.44	4.63	74.30	0.91	-0.00	4.29
○Thread 295924			3.43	4.63	74.19	0.91	-0.00	4.29
○Thread 295925			3.54	4.63	76.57	0.91	-0.00	4.30
○Thread 295926			3.50	4.63	75.71	0.91	-0.00	4.30
○Thread 295927			3.53	4.63	76.36	0.91	-0.00	4.30
○Thread 295928			3.45	4.63	74.52	0.91	-0.00	4.30
○Thread 295929			3.42	4.63	73.88	0.91	-0.00	4.29
○Thread 295930			3.55	4.63	76.69	0.91	-0.00	4.29
○Thread 295931			3.51	4.63	75.83	0.91	-0.00	4.30
○Thread 295932			3.45	4.63	74.53	0.91	-0.00	4.30
○Thread 295933			3.57	4.63	77.13	0.91	-0.00	4.30
○Thread 295934			3.53	4.63	76.26	0.91	-0.00	4.31
○Thread 295935			3.49	4.63	75.51	0.91	-0.00	4.29
○Thread 295936			3.50	4.63	75.62	0.91	-0.00	4.30
○Thread 295937			3.47	4.63	75.08	0.91	-0.00	4.29
○Thread 295938			3.54	4.63	76.59	0.91	-0.00	4.30
○Thread 295939			3.60	4.63	77.89	0.91	-0.00	4.30
○Thread 295940			3.52	4.63	76.17	0.91	-0.00	4.29
○Thread 295941			3.47	4.63	74.98	0.91	-0.00	4.30
○Thread 295942			3.54	4.63	76.49	0.91	-0.00	4.30
○Thread 295943			3.52	4.63	76.17	0.91	-0.00	4.31
○Thread 295944			3.51	4.63	75.85	0.91	-0.00	4.30
○Thread 295945			3.65	4.63	78.99	0.91	-0.00	4.30
○Thread 295946			3.60	4.63	77.80	0.91	-0.00	4.30
○Thread 295947			3.54	4.63	76.50	0.91	-0.00	4.30
○Thread 295948			3.57	4.63	77.15	0.91	-0.00	4.30
○Thread 295949			3.55	4.63	76.72	0.91	-0.00	4.30
○Thread 295950			3.59	4.63	77.70	0.91	-0.00	4.30
○Thread 295951			3.45	4.63	74.67	0.91	-0.00	4.30
○Thread 295952			3.46	4.63	74.78	0.91	-0.00	4.30
○Thread 295953			3.54	4.63	76.52	0.91	-0.00	4.30
○Thread 295954			3.61	4.63	78.14	0.91	-0.00	4.30
○Thread 295955			3.45	4.63	74.68	0.91	-0.00	4.30
○Thread 295956			3.42	4.63	73.93	0.91	-0.00	4.30

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	3.94
►Process 295977+		96	3.94
○Thread 295977			3.86	5.78	66.88	0.00	0.00	5.72
○Thread 296076			3.80	4.88	77.99	0.90	-0.00	4.52
○Thread 296077			3.94	4.88	80.76	0.90	-0.00	4.53
○Thread 296078			3.70	4.88	75.84	0.90	-0.00	4.54
○Thread 296079			3.74	4.88	76.77	0.90	-0.00	4.54
○Thread 296080			3.55	4.88	72.87	0.90	-0.00	4.54
○Thread 296081			3.55	4.88	72.88	0.90	-0.00	4.54
○Thread 296082			3.60	4.88	73.80	0.90	-0.00	4.54
○Thread 296083			3.66	4.88	75.03	0.90	-0.00	4.54
○Thread 296084			3.53	4.88	72.47	0.90	-0.00	4.53
○Thread 296085			3.51	4.88	71.96	0.90	-0.00	4.54
○Thread 296086			3.59	4.88	73.71	0.90	-0.00	4.53
○Thread 296087			3.58	4.88	73.50	0.90	-0.00	4.54
○Thread 296088			3.63	4.88	74.43	0.90	-0.00	4.53
○Thread 296089			3.79	4.88	77.81	0.90	-0.00	4.54
○Thread 296090			3.66	4.88	75.05	0.90	-0.00	4.53
○Thread 296091			3.58	4.88	73.41	0.90	-0.00	4.54
○Thread 296092			3.60	4.88	73.82	0.90	-0.00	4.54
○Thread 296093			3.60	4.88	73.92	0.90	-0.00	4.54
○Thread 296094			3.52	4.88	72.18	0.90	-0.00	4.54
○Thread 296095			3.48	4.88	71.36	0.90	-0.00	4.52
○Thread 296096			3.56	4.88	73.11	0.90	-0.00	4.54
○Thread 296097			3.61	4.88	74.03	0.90	-0.00	4.54
○Thread 296098			3.61	4.88	74.03	0.90	-0.00	4.54
○Thread 296099			3.64	4.88	74.75	0.90	-0.00	4.53
○Thread 296100			3.70	4.88	75.88	0.90	-0.00	4.54
○Thread 296101			3.67	4.88	75.27	0.90	-0.00	4.53
○Thread 296102			3.50	4.88	71.89	0.90	-0.00	4.52
○Thread 296103			3.60	4.88	73.84	0.90	-0.00	4.53
○Thread 296104			3.63	4.88	74.45	0.90	-0.00	4.54
○Thread 296105			3.62	4.88	74.35	0.90	-0.00	4.54
○Thread 296106			3.62	4.88	74.25	0.90	-0.00	4.54
○Thread 296107			3.64	4.88	74.66	0.90	-0.00	4.54
○Thread 296108			3.56	4.88	73.13	0.90	-0.00	4.53
○Thread 296109			3.42	4.87	70.15	0.90	-0.00	4.54
○Thread 296110			3.62	4.87	74.36	0.90	-0.00	4.54
○Thread 296111			3.64	4.87	74.77	0.90	-0.00	4.54
○Thread 296112			3.58	4.87	73.44	0.90	-0.00	4.53
○Thread 296113			3.57	4.87	73.24	0.90	-0.00	4.53
○Thread 296114			3.56	4.87	73.03	0.90	-0.00	4.53
○Thread 296115			3.65	4.87	74.99	0.90	-0.00	4.54
○Thread 296116			3.50	4.87	71.91	0.91	-0.00	4.54
○Thread 296117			3.52	4.87	72.22	0.91	-0.00	4.53
○Thread 296118			3.56	4.87	73.14	0.91	-0.00	4.54
○Thread 296119			3.50	4.87	71.91	0.91	-0.00	4.54
○Thread 296120			3.47	4.87	71.30	0.91	-0.00	4.54
○Thread 296121			3.67	4.87	75.41	0.91	-0.00	4.54
○Thread 296122			3.55	4.87	72.95	0.91	-0.00	4.53
○Thread 296123			3.52	4.87	72.33	0.91	-0.00	4.54
○Thread 296124			3.65	4.87	75.00	0.91	-0.00	4.54
○Thread 296125			3.67	4.87	75.41	0.91	-0.00	4.54
○Thread 296126			3.65	4.87	74.90	0.91	-0.00	4.53
○Thread 296127			3.61	4.87	74.19	0.91	-0.00	4.52
○Thread 296128			3.54	4.87	72.75	0.91	-0.00	4.52
○Thread 296129			3.53	4.87	72.55	0.91	-0.00	4.53
○Thread 296130			3.50	4.87	71.83	0.91	-0.00	4.53
○Thread 296131			3.57	4.87	73.37	0.91	-0.00	4.54
○Thread 296132			3.61	4.87	74.09	0.91	-0.00	4.53
○Thread 296133			3.53	4.87	72.55	0.91	-0.00	4.52
○Thread 296134			3.58	4.87	73.48	0.91	-0.00	4.53
○Thread 296135			3.53	4.87	72.56	0.91	-0.00	4.54
○Thread 296136			3.60	4.87	73.89	0.91	-0.00	4.54
○Thread 296137			3.63	4.87	74.51	0.91	-0.00	4.54
○Thread 296138			3.49	4.87	71.74	0.91	-0.00	4.54
○Thread 296139			3.75	4.87	76.98	0.91	-0.00	4.54
○Thread 296140			3.66	4.87	75.13	0.91	-0.00	4.54
○Thread 296141			3.59	4.87	73.70	0.91	-0.00	4.54
○Thread 296142			3.52	4.87	72.36	0.91	-0.00	4.54
○Thread 296143			3.61	4.87	74.11	0.91	-0.00	4.54
○Thread 296144			3.49	4.87	71.65	0.91	-0.00	4.53
○Thread 296145			3.56	4.87	73.09	0.91	-0.00	4.54
○Thread 296146			3.54	4.87	72.68	0.91	-0.00	4.53
○Thread 296147			3.47	4.87	71.24	0.91	-0.00	4.53
○Thread 296148			3.47	4.87	71.35	0.91	-0.00	4.53
○Thread 296149			3.66	4.87	75.15	0.91	-0.00	4.54
○Thread 296150			3.41	4.87	70.02	0.91	-0.00	4.54
○Thread 296151			3.63	4.87	74.53	0.91	-0.00	4.54
○Thread 296152			3.73	4.87	76.69	0.91	-0.00	4.54
○Thread 296153			3.56	4.87	73.20	0.91	-0.00	4.52
○Thread 296154			3.60	4.87	73.92	0.91	-0.00	4.54
○Thread 296155			3.56	4.87	73.21	0.91	-0.00	4.54
○Thread 296156			3.58	4.87	73.62	0.91	-0.00	4.54
○Thread 296157			3.60	4.87	73.93	0.91	-0.00	4.54
○Thread 296158			3.50	4.87	71.88	0.91	-0.00	4.54
○Thread 296159			3.60	4.87	73.93	0.91	-0.00	4.53
○Thread 296160			3.55	4.87	73.01	0.91	-0.00	4.54
○Thread 296161			3.46	4.87	71.06	0.91	-0.00	4.54
○Thread 296162			3.55	4.87	73.01	0.91	-0.00	4.54
○Thread 296163			3.57	4.87	73.43	0.91	-0.00	4.54
○Thread 296164			3.69	4.87	75.89	0.91	-0.00	4.54
○Thread 296165			3.58	4.87	73.63	0.91	-0.00	4.54
○Thread 296166			3.53	4.87	72.51	0.91	-0.00	4.54
○Thread 296167			3.58	4.87	73.54	0.91	-0.00	4.54
○Thread 296168			3.50	4.87	72.00	0.91	-0.00	4.53
○Thread 296169			3.48	4.87	71.59	0.91	-0.00	4.54
○Thread 296170			3.56	4.87	73.23	0.91	-0.00	4.54

Report Configuration