OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	134.86
►Process 296196+		1	134.86
○Thread 296196			134.86	135.78	99.33	0.00	0.00	135.74

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	67.01
►Process 296412+		2	67.01
○Thread 296412			67.01	67.94	98.63	0.00	0.00	67.91
○Thread 296511			66.72	67.05	99.50	0.89	-0.00	66.73

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	34.33
►Process 296582+		4	34.33
○Thread 296582			34.33	35.24	97.42	0.00	0.00	35.20
○Thread 296681			34.02	34.36	99.04	0.89	-0.00	34.04
○Thread 296682			34.02	34.36	99.02	0.89	-0.00	34.03
○Thread 296683			34.01	34.36	99.01	0.89	-0.00	34.04

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	17.91
►Process 296704+		8	17.91
○Thread 296704			17.91	18.89	94.80	0.00	0.00	18.84
○Thread 296803			17.64	18.00	98.03	0.89	-0.00	17.68
○Thread 296804			17.65	18.00	98.06	0.89	-0.00	17.67
○Thread 296805			17.65	18.00	98.09	0.89	-0.00	17.67
○Thread 296806			17.66	18.00	98.12	0.89	0.00	17.67
○Thread 296807			17.65	18.00	98.09	0.89	0.00	17.67
○Thread 296808			17.65	18.00	98.09	0.89	0.00	17.67
○Thread 296809			17.65	18.00	98.09	0.89	-0.00	17.68

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	9.97
►Process 296878+		16	9.97
○Thread 296878			9.97	10.98	90.83	0.00	0.00	10.93
○Thread 296977			9.64	10.09	95.54	0.89	-0.00	9.76
○Thread 296978			9.69	10.09	95.99	0.89	-0.00	9.77
○Thread 296979			9.65	10.09	95.59	0.89	-0.00	9.77
○Thread 296980			9.60	10.09	95.15	0.89	-0.00	9.77
○Thread 296981			9.61	10.09	95.25	0.89	-0.00	9.77
○Thread 296982			9.61	10.09	95.25	0.89	-0.00	9.77
○Thread 296983			9.58	10.09	94.95	0.89	-0.00	9.76
○Thread 296984			9.64	10.09	95.50	0.89	-0.00	9.77
○Thread 296985			9.61	10.09	95.20	0.89	0.00	9.76
○Thread 296986			9.62	10.09	95.30	0.89	0.00	9.77
○Thread 296987			9.63	10.09	95.40	0.89	0.00	9.77
○Thread 296988			9.67	10.09	95.85	0.89	0.00	9.77
○Thread 296989			9.60	10.09	95.11	0.89	0.00	9.77
○Thread 296990			9.63	10.09	95.45	0.89	0.00	9.76
○Thread 296991			9.64	10.09	95.55	0.89	-0.00	9.77

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	7.70
►Process 297014+		24	7.70
○Thread 297014			7.70	8.81	87.31	0.00	0.00	8.76
○Thread 297113			7.41	7.93	93.43	0.89	-0.00	7.60
○Thread 297114			7.32	7.93	92.36	0.89	-0.00	7.61
○Thread 297115			7.42	7.93	93.63	0.89	-0.00	7.60
○Thread 297116			7.34	7.93	92.62	0.89	-0.00	7.61
○Thread 297117			7.34	7.92	92.56	0.89	-0.00	7.60
○Thread 297118			7.38	7.92	93.06	0.89	-0.00	7.61
○Thread 297119			7.39	7.92	93.19	0.89	-0.00	7.60
○Thread 297120			7.31	7.92	92.18	0.89	-0.00	7.61
○Thread 297121			7.40	7.92	93.32	0.89	-0.00	7.60
○Thread 297122			7.33	7.92	92.50	0.89	-0.00	7.61
○Thread 297123			7.36	7.92	92.82	0.89	-0.00	7.61
○Thread 297124			7.38	7.92	93.13	0.89	-0.00	7.61
○Thread 297125			7.41	7.92	93.51	0.89	-0.00	7.60
○Thread 297126			7.37	7.92	93.01	0.89	-0.00	7.60
○Thread 297127			7.38	7.92	93.07	0.89	-0.00	7.61
○Thread 297128			7.36	7.92	92.89	0.89	-0.00	7.61
○Thread 297129			7.35	7.92	92.76	0.89	-0.00	7.61
○Thread 297130			7.44	7.92	93.84	0.89	-0.00	7.60
○Thread 297131			7.33	7.92	92.51	0.89	-0.00	7.61
○Thread 297132			7.34	7.92	92.64	0.89	-0.00	7.61
○Thread 297133			7.28	7.92	91.88	0.89	0.00	7.61
○Thread 297134			7.31	7.92	92.26	0.89	0.00	7.60
○Thread 297135			7.36	7.92	92.89	0.89	-0.00	7.60

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	6.45
►Process 297155+		32	6.45
○Thread 297155			6.45	7.56	85.28	0.00	0.00	7.51
○Thread 297254			6.14	6.67	91.98	0.89	-0.00	6.35
○Thread 297255			6.14	6.67	92.06	0.89	-0.00	6.35
○Thread 297256			6.10	6.67	91.46	0.89	-0.00	6.35
○Thread 297257			6.07	6.67	91.01	0.89	-0.00	6.35
○Thread 297258			6.10	6.67	91.46	0.89	-0.00	6.35
○Thread 297259			6.03	6.67	90.34	0.89	-0.00	6.35
○Thread 297260			5.98	6.67	89.67	0.89	-0.00	6.35
○Thread 297261			6.01	6.67	90.04	0.89	-0.00	6.35
○Thread 297262			6.00	6.67	89.97	0.89	-0.00	6.35
○Thread 297263			6.05	6.67	90.72	0.89	-0.00	6.35
○Thread 297264			6.09	6.67	91.32	0.89	-0.00	6.35
○Thread 297265			5.99	6.67	89.75	0.89	-0.00	6.35
○Thread 297266			5.96	6.67	89.30	0.89	-0.00	6.35
○Thread 297267			5.92	6.67	88.78	0.89	-0.00	6.35
○Thread 297268			6.02	6.67	90.28	0.89	-0.00	6.35
○Thread 297269			5.99	6.67	89.83	0.89	-0.00	6.35
○Thread 297270			6.07	6.67	91.03	0.89	-0.00	6.35
○Thread 297271			5.99	6.67	89.83	0.89	-0.00	6.35
○Thread 297272			6.00	6.67	89.98	0.89	-0.00	6.35
○Thread 297273			5.97	6.67	89.53	0.89	-0.00	6.35
○Thread 297274			6.08	6.67	91.19	0.89	-0.00	6.35
○Thread 297275			5.96	6.67	89.39	0.89	-0.00	6.35
○Thread 297276			6.03	6.67	90.44	0.89	-0.00	6.35
○Thread 297277			6.03	6.67	90.44	0.89	-0.00	6.35
○Thread 297278			6.00	6.67	89.99	0.89	-0.00	6.35
○Thread 297279			6.00	6.67	89.92	0.89	-0.00	6.35
○Thread 297280			6.08	6.67	91.19	0.89	-0.00	6.35
○Thread 297281			6.10	6.67	91.42	0.89	0.00	6.35
○Thread 297282			6.07	6.67	90.97	0.89	0.00	6.35
○Thread 297283			5.97	6.67	89.55	0.89	0.00	6.35
○Thread 297284			6.02	6.67	90.23	0.89	-0.00	6.35

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	5.66
►Process 297304+		40	5.66
○Thread 297304			5.66	6.91	81.82	0.00	0.00	6.88
○Thread 297403			5.40	6.02	89.56	0.89	-0.00	5.70
○Thread 297404			5.39	6.02	89.39	0.89	-0.00	5.70
○Thread 297405			5.36	6.02	88.90	0.89	0.00	5.70
○Thread 297406			5.40	6.02	89.56	0.89	-0.00	5.70
○Thread 297407			5.41	6.02	89.81	0.89	-0.00	5.70
○Thread 297408			5.34	6.02	88.57	0.89	-0.00	5.70
○Thread 297409			5.32	6.02	88.32	0.89	-0.00	5.70
○Thread 297410			5.42	6.02	89.99	0.89	0.00	5.70
○Thread 297411			5.45	6.02	90.48	0.89	-0.00	5.70
○Thread 297412			5.37	6.02	89.16	0.89	0.00	5.70
○Thread 297413			5.34	6.02	88.66	0.89	0.00	5.70
○Thread 297414			5.41	6.02	89.74	0.89	-0.00	5.70
○Thread 297415			5.54	6.02	91.90	0.89	-0.00	5.70
○Thread 297416			5.29	6.02	87.84	0.89	-0.00	5.70
○Thread 297417			5.39	6.02	89.50	0.89	0.00	5.70
○Thread 297418			5.33	6.02	88.42	0.89	0.00	5.70
○Thread 297419			5.35	6.02	88.84	0.89	0.00	5.70
○Thread 297420			5.36	6.02	88.92	0.89	0.00	5.70
○Thread 297421			5.32	6.02	88.26	0.89	0.00	5.70
○Thread 297422			5.32	6.02	88.34	0.89	0.00	5.70
○Thread 297423			5.24	6.02	87.02	0.89	0.00	5.70
○Thread 297424			5.35	6.02	88.85	0.89	0.00	5.70
○Thread 297425			5.37	6.02	89.10	0.89	0.00	5.70
○Thread 297426			5.26	6.02	87.27	0.89	0.00	5.70
○Thread 297427			5.39	6.02	89.43	0.89	0.00	5.70
○Thread 297428			5.36	6.02	89.02	0.89	0.00	5.70
○Thread 297429			5.30	6.02	88.02	0.89	0.00	5.70
○Thread 297430			5.30	6.02	87.94	0.89	0.00	5.70
○Thread 297431			5.36	6.02	89.02	0.89	0.00	5.70
○Thread 297432			5.42	6.02	90.02	0.89	0.00	5.70
○Thread 297433			5.33	6.02	88.53	0.89	0.00	5.70
○Thread 297434			5.30	6.02	88.03	0.89	0.00	5.70
○Thread 297435			5.27	6.02	87.45	0.89	0.00	5.70
○Thread 297436			5.26	6.02	87.45	0.89	0.00	5.69
○Thread 297437			5.28	6.02	87.70	0.89	0.00	5.70
○Thread 297438			5.32	6.02	88.37	0.89	0.00	5.69
○Thread 297439			5.39	6.02	89.45	0.89	0.00	5.70
○Thread 297440			5.27	6.02	87.63	0.89	0.00	5.70
○Thread 297441			5.34	6.02	88.71	0.89	-0.00	5.70

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	5.25
►Process 297509+		48	5.25
○Thread 297509			5.25	6.59	79.56	0.00	0.00	6.55
○Thread 297608			4.88	5.70	85.46	0.89	-0.00	5.37
○Thread 297609			4.90	5.70	85.90	0.89	-0.00	5.38
○Thread 297610			4.81	5.70	84.33	0.89	-0.00	5.37
○Thread 297611			4.85	5.70	84.94	0.89	-0.00	5.37
○Thread 297612			4.83	5.70	84.60	0.89	-0.00	5.38
○Thread 297613			4.85	5.70	84.95	0.89	-0.00	5.37
○Thread 297614			4.81	5.70	84.25	0.89	-0.00	5.38
○Thread 297615			5.00	5.70	87.58	0.89	-0.00	5.38
○Thread 297616			4.84	5.70	84.78	0.89	-0.00	5.37
○Thread 297617			4.88	5.70	85.48	0.89	-0.00	5.38
○Thread 297618			4.89	5.70	85.74	0.89	-0.00	5.38
○Thread 297619			4.82	5.70	84.43	0.89	-0.00	5.37
○Thread 297620			4.83	5.70	84.70	0.89	-0.00	5.38
○Thread 297621			4.86	5.70	85.14	0.89	-0.00	5.37
○Thread 297622			4.83	5.70	84.70	0.89	-0.00	5.38
○Thread 297623			4.78	5.70	83.74	0.89	-0.00	5.37
○Thread 297624			4.81	5.70	84.35	0.89	-0.00	5.37
○Thread 297625			4.88	5.70	85.49	0.89	-0.00	5.37
○Thread 297626			4.89	5.70	85.76	0.89	-0.00	5.37
○Thread 297627			4.90	5.70	85.93	0.89	-0.00	5.37
○Thread 297628			4.85	5.70	84.97	0.89	-0.00	5.37
○Thread 297629			4.75	5.70	83.22	0.89	-0.00	5.37
○Thread 297630			4.83	5.70	84.71	0.89	-0.00	5.38
○Thread 297631			4.87	5.70	85.33	0.89	-0.00	5.38
○Thread 297632			4.85	5.70	84.98	0.89	-0.00	5.38
○Thread 297633			4.93	5.70	86.38	0.89	-0.00	5.37
○Thread 297634			4.80	5.70	84.10	0.89	-0.00	5.37
○Thread 297635			4.81	5.70	84.28	0.89	-0.00	5.37
○Thread 297636			4.79	5.70	84.02	0.89	-0.00	5.38
○Thread 297637			4.87	5.70	85.42	0.89	-0.00	5.38
○Thread 297638			4.90	5.70	85.95	0.89	-0.00	5.38
○Thread 297639			4.94	5.70	86.57	0.89	-0.00	5.37
○Thread 297640			4.92	5.70	86.31	0.89	-0.00	5.38
○Thread 297641			4.90	5.70	85.96	0.89	-0.00	5.37
○Thread 297642			4.82	5.70	84.56	0.89	-0.00	5.37
○Thread 297643			5.01	5.70	87.89	0.89	-0.00	5.37
○Thread 297644			4.85	5.70	85.00	0.89	-0.00	5.37
○Thread 297645			4.97	5.70	87.19	0.89	-0.00	5.38
○Thread 297646			4.93	5.70	86.40	0.89	-0.00	5.37
○Thread 297647			4.78	5.70	83.86	0.89	-0.00	5.37
○Thread 297648			4.87	5.70	85.35	0.89	-0.00	5.36
○Thread 297649			4.83	5.70	84.74	0.89	-0.00	5.37
○Thread 297650			4.87	5.70	85.45	0.89	-0.00	5.38
○Thread 297651			4.84	5.70	84.92	0.89	-0.00	5.37
○Thread 297652			4.90	5.70	85.98	0.89	-0.00	5.37
○Thread 297653			4.94	5.70	86.68	0.89	-0.00	5.38
○Thread 297654			4.94	5.70	86.59	0.89	-0.00	5.38

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	4.94
►Process 297674+		56	4.94
○Thread 297674			4.94	6.37	77.44	0.00	0.00	6.34
○Thread 297773			4.60	5.50	83.69	0.88	-0.00	5.15
○Thread 297774			4.59	5.50	83.51	0.88	-0.00	5.15
○Thread 297775			4.70	5.50	85.42	0.88	-0.00	5.16
○Thread 297776			4.51	5.50	81.96	0.88	-0.00	5.14
○Thread 297777			4.62	5.50	84.06	0.88	-0.00	5.15
○Thread 297778			4.65	5.50	84.52	0.88	-0.00	5.15
○Thread 297779			4.72	5.50	85.79	0.88	-0.00	5.15
○Thread 297780			4.52	5.50	82.15	0.88	-0.00	5.16
○Thread 297781			4.47	5.50	81.34	0.88	-0.00	5.16
○Thread 297782			4.63	5.50	84.16	0.88	-0.00	5.16
○Thread 297783			4.58	5.50	83.25	0.88	-0.00	5.15
○Thread 297784			4.65	5.50	84.62	0.88	-0.00	5.16
○Thread 297785			4.61	5.50	83.80	0.88	-0.00	5.16
○Thread 297786			4.56	5.50	82.98	0.88	-0.00	5.15
○Thread 297787			4.59	5.50	83.53	0.88	-0.00	5.15
○Thread 297788			4.54	5.49	82.62	0.88	-0.00	5.14
○Thread 297789			4.57	5.49	83.08	0.88	-0.00	5.16
○Thread 297790			4.61	5.49	83.81	0.88	-0.00	5.15
○Thread 297791			4.61	5.49	83.90	0.88	-0.00	5.16
○Thread 297792			4.60	5.49	83.72	0.88	-0.00	5.15
○Thread 297793			4.57	5.49	83.08	0.88	-0.00	5.16
○Thread 297794			4.52	5.49	82.27	0.88	-0.00	5.15
○Thread 297795			4.52	5.49	82.18	0.88	-0.00	5.16
○Thread 297796			4.59	5.49	83.54	0.88	-0.00	5.14
○Thread 297797			4.70	5.49	85.55	0.88	-0.00	5.15
○Thread 297798			4.56	5.49	83.00	0.88	-0.00	5.16
○Thread 297799			4.53	5.49	82.37	0.88	-0.00	5.16
○Thread 297800			4.60	5.49	83.64	0.88	-0.00	5.15
○Thread 297801			4.60	5.49	83.64	0.88	-0.00	5.15
○Thread 297802			4.64	5.49	84.46	0.88	-0.00	5.15
○Thread 297803			4.63	5.49	84.19	0.88	-0.00	5.15
○Thread 297804			4.60	5.49	83.65	0.88	-0.00	5.15
○Thread 297805			4.66	5.49	84.74	0.88	-0.00	5.15
○Thread 297806			4.62	5.49	84.02	0.88	-0.00	5.16
○Thread 297807			4.60	5.49	83.74	0.88	-0.00	5.16
○Thread 297808			4.61	5.49	83.84	0.88	-0.00	5.14
○Thread 297809			4.66	5.49	84.75	0.88	-0.00	5.15
○Thread 297810			4.61	5.49	83.84	0.88	-0.00	5.15
○Thread 297811			4.60	5.49	83.75	0.88	-0.00	5.14
○Thread 297812			4.56	5.49	83.02	0.88	-0.00	5.15
○Thread 297813			4.61	5.49	83.85	0.88	-0.00	5.15
○Thread 297814			4.61	5.49	83.94	0.88	-0.00	5.15
○Thread 297815			4.48	5.49	81.48	0.88	-0.00	5.15
○Thread 297816			4.57	5.49	83.12	0.88	-0.00	5.16
○Thread 297817			4.57	5.49	83.22	0.88	-0.00	5.15
○Thread 297818			4.51	5.49	82.03	0.88	-0.00	5.16
○Thread 297819			4.60	5.49	83.76	0.88	-0.00	5.16
○Thread 297820			4.62	5.49	84.04	0.88	-0.00	5.15
○Thread 297821			4.67	5.49	84.95	0.88	-0.00	5.16
○Thread 297822			4.64	5.49	84.50	0.88	-0.00	5.15
○Thread 297823			4.54	5.49	82.59	0.88	-0.00	5.15
○Thread 297824			4.66	5.49	84.77	0.88	-0.00	5.16
○Thread 297825			4.58	5.49	83.32	0.88	-0.00	5.15
○Thread 297826			4.61	5.49	84.05	0.88	-0.00	5.15
○Thread 297827			4.54	5.49	82.69	0.88	-0.00	5.14

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	4.63
►Process 297847+		64	4.63
○Thread 297847			4.63	6.29	73.59	0.00	0.00	6.25
○Thread 297946			4.47	5.40	82.64	0.89	-0.00	5.06
○Thread 297947			4.45	5.40	82.37	0.89	-0.00	5.06
○Thread 297948			4.41	5.40	81.63	0.89	-0.00	5.05
○Thread 297949			4.45	5.40	82.37	0.89	-0.00	5.06
○Thread 297950			4.45	5.40	82.38	0.89	-0.00	5.05
○Thread 297951			4.42	5.40	81.91	0.89	-0.00	5.08
○Thread 297952			4.32	5.40	79.97	0.89	-0.00	5.05
○Thread 297953			4.35	5.40	80.53	0.89	-0.00	5.06
○Thread 297954			4.37	5.40	80.90	0.89	-0.00	5.05
○Thread 297955			4.33	5.40	80.16	0.89	-0.00	5.05
○Thread 297956			4.50	5.40	83.22	0.89	-0.00	5.05
○Thread 297957			4.37	5.40	80.91	0.89	-0.00	5.05
○Thread 297958			4.38	5.40	81.09	0.89	-0.00	5.06
○Thread 297959			4.39	5.40	81.19	0.89	-0.00	5.05
○Thread 297960			4.30	5.40	79.71	0.89	-0.00	5.07
○Thread 297961			4.30	5.40	79.62	0.89	-0.00	5.05
○Thread 297962			4.46	5.40	82.49	0.89	-0.00	5.05
○Thread 297963			4.31	5.40	79.71	0.89	-0.00	5.05
○Thread 297964			4.32	5.40	79.99	0.89	-0.00	5.05
○Thread 297965			4.29	5.40	79.53	0.89	-0.00	5.05
○Thread 297966			4.45	5.40	82.40	0.89	-0.00	5.06
○Thread 297967			4.37	5.40	80.92	0.89	-0.00	5.06
○Thread 297968			4.52	5.40	83.70	0.89	-0.00	5.06
○Thread 297969			4.36	5.40	80.74	0.89	-0.00	5.06
○Thread 297970			4.39	5.40	81.39	0.89	-0.00	5.08
○Thread 297971			4.42	5.40	81.76	0.89	-0.00	5.05
○Thread 297972			4.44	5.40	82.13	0.89	-0.00	5.06
○Thread 297973			4.43	5.40	82.04	0.89	-0.00	5.06
○Thread 297974			4.48	5.40	82.97	0.89	-0.00	5.04
○Thread 297975			4.34	5.40	80.38	0.89	-0.00	5.04
○Thread 297976			4.28	5.40	79.27	0.89	-0.00	5.05
○Thread 297977			4.39	5.40	81.22	0.89	-0.00	5.06
○Thread 297978			4.38	5.40	81.03	0.89	-0.00	5.05
○Thread 297979			4.46	5.40	82.52	0.89	-0.00	5.06
○Thread 297980			4.40	5.40	81.41	0.89	-0.00	5.06
○Thread 297981			4.43	5.40	81.96	0.89	-0.00	5.05
○Thread 297982			4.31	5.40	79.84	0.89	-0.00	5.06
○Thread 297983			4.42	5.40	81.78	0.89	-0.00	5.05
○Thread 297984			4.38	5.40	81.14	0.89	-0.00	5.05
○Thread 297985			4.41	5.40	81.60	0.89	-0.00	5.06
○Thread 297986			4.43	5.40	82.07	0.89	-0.00	5.05
○Thread 297987			4.39	5.40	81.23	0.89	-0.00	5.06
○Thread 297988			4.36	5.40	80.68	0.89	-0.00	5.06
○Thread 297989			4.40	5.40	81.42	0.89	-0.00	5.06
○Thread 297990			4.43	5.40	81.98	0.89	-0.00	5.06
○Thread 297991			4.42	5.40	81.89	0.89	-0.00	5.06
○Thread 297992			4.49	5.40	83.19	0.89	-0.00	5.05
○Thread 297993			4.38	5.40	81.06	0.89	-0.00	5.05
○Thread 297994			4.37	5.40	80.97	0.89	-0.00	5.05
○Thread 297995			4.49	5.40	83.10	0.89	-0.00	5.06
○Thread 297996			4.41	5.40	81.80	0.89	-0.00	5.07
○Thread 297997			4.50	5.40	83.38	0.89	-0.00	5.05
○Thread 297998			4.26	5.40	79.03	0.89	-0.00	5.05
○Thread 297999			4.42	5.40	81.99	0.89	-0.00	5.06
○Thread 298000			4.37	5.40	80.98	0.90	-0.00	5.05
○Thread 298001			4.46	5.40	82.55	0.90	-0.00	5.05
○Thread 298002			4.39	5.40	81.35	0.90	-0.00	5.04
○Thread 298003			4.45	5.40	82.46	0.90	-0.00	5.06
○Thread 298004			4.32	5.40	80.15	0.90	-0.00	5.06
○Thread 298005			4.40	5.40	81.45	0.90	-0.00	5.05
○Thread 298006			4.43	5.40	82.19	0.90	-0.00	5.05
○Thread 298007			4.39	5.40	81.27	0.90	-0.00	5.05
○Thread 298008			4.38	5.40	81.08	0.90	-0.00	5.05

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	4.97
►Process 298028+		72	4.97
○Thread 298028			4.97	6.67	74.50	0.00	0.00	6.62
○Thread 298127			4.77	5.78	82.51	0.89	-0.00	5.44
○Thread 298128			4.77	5.78	82.51	0.89	-0.00	5.44
○Thread 298129			4.84	5.78	83.64	0.89	-0.00	5.45
○Thread 298130			4.79	5.78	82.86	0.89	-0.00	5.45
○Thread 298131			4.88	5.78	84.34	0.89	-0.00	5.44
○Thread 298132			4.81	5.78	83.13	0.89	-0.00	5.45
○Thread 298133			4.72	5.78	81.57	0.89	-0.00	5.44
○Thread 298134			4.66	5.78	80.62	0.89	-0.00	5.44
○Thread 298135			4.89	5.78	84.60	0.89	-0.00	5.44
○Thread 298136			4.74	5.78	82.01	0.89	-0.00	5.45
○Thread 298137			4.78	5.78	82.62	0.89	-0.00	5.44
○Thread 298138			4.73	5.78	81.84	0.89	-0.00	5.45
○Thread 298139			4.78	5.78	82.62	0.89	-0.00	5.45
○Thread 298140			4.80	5.78	82.97	0.89	-0.00	5.45
○Thread 298141			4.79	5.78	82.88	0.89	-0.00	5.44
○Thread 298142			4.73	5.78	81.84	0.89	-0.00	5.44
○Thread 298143			4.84	5.78	83.75	0.89	-0.00	5.44
○Thread 298144			4.78	5.78	82.71	0.89	-0.00	5.45
○Thread 298145			4.80	5.78	83.06	0.89	-0.00	5.45
○Thread 298146			4.74	5.78	82.02	0.89	-0.00	5.45
○Thread 298147			4.64	5.78	80.29	0.89	-0.00	5.44
○Thread 298148			4.75	5.78	82.11	0.89	-0.00	5.44
○Thread 298149			4.79	5.78	82.89	0.89	-0.00	5.44
○Thread 298150			4.68	5.78	80.99	0.89	-0.00	5.44
○Thread 298151			4.82	5.78	83.33	0.89	-0.00	5.44
○Thread 298152			4.70	5.78	81.34	0.89	-0.00	5.44
○Thread 298153			4.79	5.78	82.90	0.89	-0.00	5.45
○Thread 298154			4.75	5.78	82.12	0.89	-0.00	5.44
○Thread 298155			4.67	5.78	80.74	0.89	-0.00	5.44
○Thread 298156			4.72	5.78	81.69	0.89	-0.00	5.44
○Thread 298157			4.76	5.78	82.30	0.89	-0.00	5.44
○Thread 298158			4.74	5.78	81.96	0.89	-0.00	5.44
○Thread 298159			4.73	5.78	81.87	0.89	-0.00	5.45
○Thread 298160			4.78	5.78	82.74	0.89	-0.00	5.44
○Thread 298161			4.87	5.78	84.21	0.89	-0.00	5.44
○Thread 298162			4.77	5.78	82.48	0.89	-0.00	5.45
○Thread 298163			4.75	5.78	82.14	0.89	-0.00	5.45
○Thread 298164			4.78	5.78	82.74	0.89	-0.00	5.44
○Thread 298165			4.74	5.78	81.97	0.89	-0.00	5.44
○Thread 298166			4.70	5.78	81.45	0.89	-0.00	5.44
○Thread 298167			4.78	5.78	82.84	0.89	-0.00	5.44
○Thread 298168			4.79	5.78	82.92	0.89	-0.00	5.44
○Thread 298169			4.81	5.78	83.18	0.89	-0.00	5.43
○Thread 298170			4.77	5.78	82.58	0.89	-0.00	5.44
○Thread 298171			4.77	5.78	82.50	0.89	-0.00	5.45
○Thread 298172			4.90	5.78	84.75	0.89	-0.00	5.45
○Thread 298173			4.77	5.78	82.50	0.89	-0.00	5.45
○Thread 298174			4.92	5.78	85.18	0.90	-0.00	5.45
○Thread 298175			4.72	5.78	81.72	0.90	-0.00	5.44
○Thread 298176			4.82	5.78	83.46	0.90	-0.00	5.45
○Thread 298177			4.74	5.78	82.07	0.90	-0.00	5.45
○Thread 298178			4.85	5.78	83.98	0.90	-0.00	5.45
○Thread 298179			4.78	5.78	82.77	0.90	-0.00	5.44
○Thread 298180			4.88	5.78	84.50	0.90	-0.00	5.45
○Thread 298181			4.80	5.77	83.03	0.90	-0.00	5.44
○Thread 298182			4.83	5.77	83.55	0.90	-0.00	5.43
○Thread 298183			4.89	5.77	84.59	0.90	-0.00	5.44
○Thread 298184			4.81	5.77	83.21	0.90	-0.00	5.45
○Thread 298185			4.84	5.77	83.82	0.90	-0.00	5.44
○Thread 298186			4.74	5.77	82.09	0.90	-0.00	5.45
○Thread 298187			4.68	5.77	81.05	0.90	-0.00	5.44
○Thread 298188			4.72	5.77	81.74	0.90	-0.00	5.45
○Thread 298189			4.77	5.77	82.61	0.90	-0.00	5.45
○Thread 298190			4.86	5.77	84.08	0.90	-0.00	5.45
○Thread 298191			4.80	5.77	83.22	0.90	-0.00	5.43
○Thread 298192			4.80	5.77	83.05	0.90	-0.00	5.44
○Thread 298193			4.74	5.77	82.10	0.90	-0.00	5.44
○Thread 298194			4.76	5.77	82.36	0.90	-0.00	5.44
○Thread 298195			4.73	5.77	81.84	0.90	-0.00	5.44
○Thread 298196			4.74	5.77	82.10	0.90	-0.00	5.43
○Thread 298197			4.80	5.77	83.14	0.90	-0.00	5.45

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	4.52
►Process 298267+		80	4.52
○Thread 298267			4.52	6.23	72.51	0.00	0.00	6.18
○Thread 298366			4.26	5.34	79.79	0.89	-0.00	4.99
○Thread 298367			4.27	5.34	80.07	0.89	0.00	4.99
○Thread 298368			4.32	5.34	80.91	0.89	-0.00	5.00
○Thread 298369			4.13	5.34	77.36	0.89	-0.00	5.00
○Thread 298370			4.17	5.34	78.20	0.89	0.00	5.00
○Thread 298371			4.24	5.34	79.51	0.89	-0.00	5.00
○Thread 298372			4.23	5.34	79.33	0.89	-0.00	5.00
○Thread 298373			4.37	5.34	81.77	0.89	-0.00	5.00
○Thread 298374			4.17	5.34	78.11	0.89	-0.00	4.98
○Thread 298375			4.22	5.34	79.05	0.89	-0.00	4.99
○Thread 298376			4.11	5.34	77.09	0.89	-0.00	5.00
○Thread 298377			4.18	5.34	78.40	0.89	-0.00	4.99
○Thread 298378			4.16	5.34	78.03	0.89	-0.00	5.00
○Thread 298379			4.35	5.34	81.49	0.89	-0.00	5.00
○Thread 298380			4.25	5.34	79.72	0.89	-0.00	5.00
○Thread 298381			4.14	5.34	77.56	0.89	-0.00	4.99
○Thread 298382			4.15	5.34	77.85	0.89	-0.00	4.99
○Thread 298383			4.26	5.34	79.82	0.89	-0.00	4.98
○Thread 298384			4.06	5.34	76.16	0.89	-0.00	4.99
○Thread 298385			4.20	5.34	78.69	0.89	-0.00	5.00
○Thread 298386			4.18	5.34	78.32	0.89	-0.00	4.99
○Thread 298387			4.19	5.34	78.51	0.89	-0.00	5.00
○Thread 298388			4.17	5.34	78.14	0.89	-0.00	4.99
○Thread 298389			4.22	5.34	79.08	0.89	-0.00	5.00
○Thread 298390			4.15	5.34	77.77	0.89	-0.00	5.00
○Thread 298391			4.17	5.34	78.24	0.89	-0.00	5.00
○Thread 298392			4.09	5.34	76.64	0.89	-0.00	4.98
○Thread 298393			4.14	5.34	77.68	0.89	-0.00	4.98
○Thread 298394			4.21	5.34	78.90	0.89	-0.00	4.99
○Thread 298395			4.20	5.34	78.80	0.89	-0.00	4.98
○Thread 298396			4.20	5.34	78.71	0.89	-0.00	4.98
○Thread 298397			4.20	5.34	78.71	0.89	-0.00	5.00
○Thread 298398			4.22	5.34	79.09	0.89	-0.00	5.00
○Thread 298399			4.20	5.34	78.81	0.89	-0.00	4.98
○Thread 298400			4.20	5.34	78.72	0.89	-0.00	5.00
○Thread 298401			4.27	5.34	80.03	0.89	-0.00	5.00
○Thread 298402			4.19	5.34	78.63	0.89	-0.00	4.96
○Thread 298403			4.24	5.34	79.57	0.89	-0.00	5.00
○Thread 298404			4.12	5.34	77.23	0.89	-0.00	4.99
○Thread 298405			4.14	5.33	77.70	0.89	-0.00	4.98
○Thread 298406			4.08	5.33	76.57	0.89	-0.00	5.00
○Thread 298407			4.16	5.33	77.98	0.89	-0.00	4.98
○Thread 298408			4.16	5.33	77.98	0.89	-0.00	5.00
○Thread 298409			4.11	5.33	77.14	0.89	-0.00	4.98
○Thread 298410			4.18	5.33	78.36	0.89	-0.00	5.00
○Thread 298411			4.08	5.33	76.49	0.89	-0.00	4.99
○Thread 298412			4.11	5.33	77.05	0.89	-0.00	5.00
○Thread 298413			4.26	5.33	79.86	0.89	-0.00	5.00
○Thread 298414			4.20	5.33	78.83	0.89	-0.00	4.99
○Thread 298415			4.18	5.33	78.46	0.89	-0.00	5.00
○Thread 298416			4.22	5.33	79.12	0.89	-0.00	4.99
○Thread 298417			4.17	5.33	78.28	0.89	-0.00	5.00
○Thread 298418			4.19	5.33	78.56	0.89	-0.00	4.99
○Thread 298419			4.13	5.33	77.53	0.89	-0.00	4.99
○Thread 298420			4.06	5.33	76.22	0.89	-0.00	4.99
○Thread 298421			4.18	5.33	78.47	0.89	-0.00	5.00
○Thread 298422			4.19	5.33	78.57	0.89	-0.00	5.00
○Thread 298423			4.16	5.33	78.01	0.89	-0.00	4.99
○Thread 298424			4.14	5.33	77.63	0.89	-0.00	5.00
○Thread 298425			4.16	5.33	78.01	0.89	-0.00	5.00
○Thread 298426			4.13	5.33	77.45	0.89	-0.00	4.98
○Thread 298427			4.17	5.33	78.20	0.89	-0.00	5.00
○Thread 298428			4.19	5.33	78.67	0.89	-0.00	5.00
○Thread 298429			4.16	5.33	78.11	0.89	-0.00	4.99
○Thread 298430			4.28	5.33	80.36	0.89	-0.00	5.00
○Thread 298431			4.29	5.33	80.46	0.89	-0.00	5.00
○Thread 298432			4.17	5.33	78.30	0.89	-0.00	5.00
○Thread 298433			4.31	5.33	80.93	0.90	-0.00	5.00
○Thread 298434			4.20	5.33	78.87	0.90	-0.00	4.98
○Thread 298435			4.26	5.33	79.90	0.90	-0.00	5.00
○Thread 298436			4.09	5.33	76.71	0.90	-0.00	5.00
○Thread 298437			4.18	5.33	78.40	0.90	-0.00	4.99
○Thread 298438			4.19	5.33	78.69	0.90	-0.00	5.00
○Thread 298439			4.15	5.33	77.84	0.90	-0.00	5.00
○Thread 298440			4.23	5.33	79.35	0.90	-0.00	5.00
○Thread 298441			4.13	5.33	77.57	0.90	-0.00	4.99
○Thread 298442			4.22	5.33	79.16	0.90	-0.00	5.00
○Thread 298443			4.27	5.33	80.10	0.90	-0.00	4.99
○Thread 298444			3.80	5.33	71.29	0.90	-0.00	4.99

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	4.76
►Process 298464+		88	4.76
○Thread 298464			4.76	6.63	71.77	0.00	0.00	6.60
○Thread 298563			4.61	5.74	80.38	0.89	-0.00	5.38
○Thread 298564			4.58	5.73	79.86	0.89	-0.00	5.37
○Thread 298565			4.67	5.73	81.35	0.89	-0.00	5.39
○Thread 298566			4.52	5.73	78.91	0.89	-0.00	5.38
○Thread 298567			4.49	5.73	78.21	0.89	-0.00	5.39
○Thread 298568			4.58	5.73	79.87	0.89	-0.00	5.38
○Thread 298569			4.59	5.73	79.96	0.89	-0.00	5.38
○Thread 298570			4.56	5.73	79.52	0.89	-0.00	5.39
○Thread 298571			4.47	5.73	77.95	0.89	-0.00	5.38
○Thread 298572			4.58	5.73	79.87	0.89	-0.00	5.39
○Thread 298573			4.52	5.73	78.74	0.89	-0.00	5.39
○Thread 298574			4.55	5.73	79.35	0.89	-0.00	5.37
○Thread 298575			4.47	5.73	77.96	0.89	0.00	5.39
○Thread 298576			4.43	5.73	77.26	0.89	-0.00	5.39
○Thread 298577			4.44	5.73	77.53	0.89	-0.00	5.38
○Thread 298578			4.35	5.73	75.87	0.89	-0.00	5.38
○Thread 298579			4.35	5.73	75.96	0.89	-0.00	5.38
○Thread 298580			4.61	5.73	80.32	0.89	-0.00	5.38
○Thread 298581			4.40	5.73	76.84	0.89	-0.00	5.38
○Thread 298582			4.51	5.73	78.67	0.89	-0.00	5.38
○Thread 298583			4.52	5.73	78.85	0.89	-0.00	5.38
○Thread 298584			4.56	5.73	79.54	0.89	-0.00	5.39
○Thread 298585			4.48	5.73	78.06	0.89	-0.00	5.39
○Thread 298586			4.45	5.73	77.72	0.89	-0.00	5.39
○Thread 298587			4.50	5.73	78.50	0.89	-0.00	5.39
○Thread 298588			4.53	5.73	79.03	0.89	-0.00	5.38
○Thread 298589			4.53	5.73	79.03	0.89	-0.00	5.39
○Thread 298590			4.53	5.73	79.03	0.89	-0.00	5.38
○Thread 298591			4.60	5.73	80.25	0.89	-0.00	5.39
○Thread 298592			4.42	5.73	77.03	0.89	-0.00	5.38
○Thread 298593			4.53	5.73	79.03	0.89	-0.00	5.38
○Thread 298594			4.51	5.73	78.69	0.89	-0.00	5.39
○Thread 298595			4.50	5.73	78.51	0.89	-0.00	5.38
○Thread 298596			4.43	5.73	77.38	0.89	-0.00	5.38
○Thread 298597			4.54	5.73	79.22	0.89	-0.00	5.39
○Thread 298598			4.49	5.73	78.26	0.89	-0.00	5.37
○Thread 298599			4.64	5.73	80.88	0.89	-0.00	5.38
○Thread 298600			4.46	5.73	77.74	0.89	-0.00	5.38
○Thread 298601			4.59	5.73	80.09	0.89	-0.00	5.38
○Thread 298602			4.51	5.73	78.70	0.89	-0.00	5.39
○Thread 298603			4.45	5.73	77.57	0.90	-0.00	5.39
○Thread 298604			4.62	5.73	80.62	0.90	-0.00	5.39
○Thread 298605			4.59	5.73	80.10	0.90	-0.00	5.38
○Thread 298606			4.62	5.73	80.63	0.90	-0.00	5.38
○Thread 298607			4.62	5.73	80.54	0.90	-0.00	5.40
○Thread 298608			4.62	5.73	80.54	0.90	-0.00	5.39
○Thread 298609			4.48	5.73	78.27	0.90	-0.00	5.39
○Thread 298610			4.58	5.73	79.93	0.90	-0.00	5.39
○Thread 298611			4.48	5.73	78.28	0.90	-0.00	5.39
○Thread 298612			4.50	5.73	78.54	0.90	-0.00	5.38
○Thread 298613			4.49	5.73	78.28	0.90	-0.00	5.39
○Thread 298614			4.61	5.73	80.38	0.90	-0.00	5.39
○Thread 298615			4.54	5.73	79.24	0.90	-0.00	5.38
○Thread 298616			4.51	5.73	78.81	0.90	-0.00	5.39
○Thread 298617			4.58	5.73	79.94	0.90	-0.00	5.38
○Thread 298618			4.55	5.73	79.42	0.90	-0.00	5.37
○Thread 298619			4.61	5.73	80.47	0.90	-0.00	5.39
○Thread 298620			4.54	5.73	79.16	0.90	-0.00	5.39
○Thread 298621			4.49	5.73	78.29	0.90	-0.00	5.38
○Thread 298622			4.61	5.73	80.48	0.90	-0.00	5.39
○Thread 298623			4.52	5.73	78.91	0.90	-0.00	5.38
○Thread 298624			4.51	5.73	78.73	0.90	-0.00	5.39
○Thread 298625			4.58	5.73	79.96	0.90	-0.00	5.38
○Thread 298626			4.47	5.73	78.04	0.90	-0.00	5.39
○Thread 298627			4.44	5.73	77.43	0.90	-0.00	5.39
○Thread 298628			4.52	5.73	78.91	0.90	-0.00	5.38
○Thread 298629			4.59	5.73	80.05	0.90	-0.00	5.39
○Thread 298630			4.64	5.73	81.01	0.90	-0.00	5.38
○Thread 298631			4.50	5.73	78.57	0.90	-0.00	5.38
○Thread 298632			4.60	5.73	80.32	0.90	-0.00	5.39
○Thread 298633			4.52	5.73	78.92	0.90	-0.00	5.38
○Thread 298634			4.51	5.73	78.83	0.90	-0.00	5.40
○Thread 298635			4.52	5.73	78.84	0.90	-0.00	5.39
○Thread 298636			4.44	5.73	77.53	0.90	-0.00	5.39
○Thread 298637			4.47	5.73	78.05	0.90	-0.00	5.39
○Thread 298638			4.53	5.73	79.02	0.90	-0.00	5.39
○Thread 298639			4.50	5.73	78.49	0.90	-0.00	5.38
○Thread 298640			4.43	5.73	77.45	0.90	-0.00	5.38
○Thread 298641			4.48	5.73	78.32	0.90	-0.00	5.40
○Thread 298642			4.43	5.73	77.36	0.90	-0.00	5.37
○Thread 298643			4.46	5.73	77.98	0.90	-0.00	5.37
○Thread 298644			4.51	5.73	78.76	0.90	-0.00	5.39
○Thread 298645			4.48	5.73	78.15	0.90	-0.00	5.39
○Thread 298646			4.56	5.73	79.64	0.90	-0.00	5.38
○Thread 298647			4.50	5.73	78.59	0.90	-0.00	5.39
○Thread 298648			4.44	5.73	77.55	0.90	-0.00	5.38
○Thread 298649			4.08	5.73	71.35	0.90	-0.00	5.39

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	4.80
►Process 298669+		96	4.80
○Thread 298669			4.80	6.69	71.71	0.00	0.00	6.66
○Thread 298768			4.49	5.82	77.21	0.88	-0.00	5.46
○Thread 298769			4.45	5.82	76.44	0.88	-0.00	5.46
○Thread 298770			4.54	5.82	78.07	0.88	-0.00	5.45
○Thread 298771			4.37	5.82	75.06	0.88	-0.00	5.46
○Thread 298772			4.47	5.81	76.78	0.88	-0.00	5.46
○Thread 298773			4.43	5.81	76.18	0.88	-0.00	5.46
○Thread 298774			4.53	5.81	77.91	0.88	-0.00	5.46
○Thread 298775			4.50	5.81	77.31	0.88	-0.00	5.46
○Thread 298776			4.46	5.81	76.62	0.88	-0.00	5.47
○Thread 298777			4.55	5.81	78.17	0.88	-0.00	5.46
○Thread 298778			4.46	5.81	76.62	0.88	-0.00	5.47
○Thread 298779			4.41	5.81	75.94	0.88	-0.00	5.46
○Thread 298780			4.44	5.81	76.37	0.88	-0.00	5.46
○Thread 298781			4.42	5.81	76.02	0.88	-0.00	5.46
○Thread 298782			4.44	5.81	76.37	0.88	-0.00	5.46
○Thread 298783			4.35	5.81	74.82	0.88	-0.00	5.47
○Thread 298784			4.35	5.81	74.82	0.88	-0.00	5.46
○Thread 298785			4.45	5.81	76.55	0.88	-0.00	5.47
○Thread 298786			4.29	5.81	73.88	0.88	-0.00	5.46
○Thread 298787			4.45	5.81	76.55	0.88	-0.00	5.47
○Thread 298788			4.42	5.81	75.95	0.88	-0.00	5.47
○Thread 298789			4.39	5.81	75.52	0.88	-0.00	5.46
○Thread 298790			4.51	5.81	77.59	0.88	-0.00	5.46
○Thread 298791			4.43	5.81	76.21	0.88	-0.00	5.47
○Thread 298792			4.35	5.81	74.84	0.88	-0.00	5.46
○Thread 298793			4.35	5.81	74.84	0.88	-0.00	5.46
○Thread 298794			4.29	5.81	73.89	0.88	-0.00	5.46
○Thread 298795			4.42	5.81	76.04	0.88	-0.00	5.47
○Thread 298796			4.37	5.81	75.19	0.88	-0.00	5.47
○Thread 298797			4.46	5.81	76.74	0.88	-0.00	5.46
○Thread 298798			4.51	5.81	77.68	0.88	-0.00	5.46
○Thread 298799			4.44	5.81	76.31	0.88	-0.00	5.47
○Thread 298800			4.47	5.81	76.83	0.88	-0.00	5.46
○Thread 298801			4.46	5.81	76.74	0.88	-0.00	5.47
○Thread 298802			4.40	5.81	75.80	0.88	-0.00	5.46
○Thread 298803			4.40	5.81	75.80	0.88	-0.00	5.47
○Thread 298804			4.49	5.81	77.26	0.88	-0.00	5.47
○Thread 298805			4.42	5.81	76.06	0.88	-0.00	5.47
○Thread 298806			4.42	5.81	76.06	0.88	-0.00	5.45
○Thread 298807			4.53	5.81	77.87	0.88	-0.00	5.47
○Thread 298808			4.41	5.81	75.89	0.88	-0.00	5.46
○Thread 298809			4.42	5.81	76.15	0.88	-0.00	5.46
○Thread 298810			4.38	5.81	75.38	0.88	-0.00	5.46
○Thread 298811			4.38	5.81	75.38	0.88	-0.00	5.46
○Thread 298812			4.40	5.81	75.72	0.88	-0.00	5.46
○Thread 298813			4.58	5.81	78.82	0.88	-0.00	5.46
○Thread 298814			4.48	5.81	77.02	0.88	-0.00	5.47
○Thread 298815			4.54	5.81	78.14	0.88	-0.00	5.47
○Thread 298816			4.53	5.81	77.88	0.88	-0.00	5.47
○Thread 298817			4.38	5.81	75.30	0.88	-0.00	5.46
○Thread 298818			4.44	5.81	76.51	0.88	-0.00	5.47
○Thread 298819			4.59	5.81	79.00	0.88	-0.00	5.47
○Thread 298820			4.45	5.81	76.51	0.88	-0.00	5.46
○Thread 298821			4.43	5.81	76.25	0.88	-0.00	5.46
○Thread 298822			4.36	5.81	75.05	0.88	-0.00	5.46
○Thread 298823			4.45	5.81	76.51	0.88	-0.00	5.45
○Thread 298824			4.53	5.81	77.89	0.88	-0.00	5.46
○Thread 298825			4.51	5.81	77.55	0.88	-0.00	5.47
○Thread 298826			4.33	5.81	74.54	0.88	-0.00	5.46
○Thread 298827			4.54	5.81	78.07	0.88	-0.00	5.48
○Thread 298828			4.43	5.81	76.26	0.88	-0.00	5.46
○Thread 298829			4.42	5.81	76.09	0.88	-0.00	5.47
○Thread 298830			4.48	5.81	77.13	0.88	-0.00	5.46
○Thread 298831			4.54	5.81	78.16	0.88	-0.00	5.46
○Thread 298832			4.51	5.81	77.56	0.88	-0.00	5.47
○Thread 298833			4.47	5.81	76.87	0.89	-0.00	5.47
○Thread 298834			4.40	5.81	75.76	0.89	-0.00	5.46
○Thread 298835			4.33	5.81	74.55	0.89	-0.00	5.46
○Thread 298836			4.38	5.81	75.50	0.89	-0.00	5.47
○Thread 298837			4.43	5.81	76.19	0.89	-0.00	5.46
○Thread 298838			4.37	5.81	75.25	0.89	-0.00	5.47
○Thread 298839			4.50	5.81	77.49	0.89	-0.00	5.47
○Thread 298840			4.47	5.81	77.06	0.89	-0.00	5.47
○Thread 298841			4.49	5.81	77.32	0.89	-0.00	5.47
○Thread 298842			4.52	5.81	77.83	0.89	-0.00	5.47
○Thread 298843			4.42	5.81	76.20	0.89	-0.00	5.46
○Thread 298844			4.48	5.81	77.23	0.89	-0.00	5.46
○Thread 298845			4.53	5.81	77.92	0.89	-0.00	5.47
○Thread 298846			4.37	5.81	75.34	0.89	-0.00	5.46
○Thread 298847			4.48	5.81	77.15	0.89	-0.00	5.47
○Thread 298848			4.53	5.81	78.02	0.89	-0.00	5.46
○Thread 298849			4.48	5.81	77.07	0.89	-0.00	5.47
○Thread 298850			4.47	5.81	76.90	0.89	-0.00	5.45
○Thread 298851			4.45	5.81	76.64	0.89	-0.00	5.46
○Thread 298852			4.39	5.81	75.61	0.89	-0.00	5.46
○Thread 298853			4.31	5.81	74.23	0.89	-0.00	5.46
○Thread 298854			4.48	5.81	77.25	0.89	-0.00	5.46
○Thread 298855			4.57	5.81	78.71	0.89	-0.00	5.47
○Thread 298856			4.46	5.81	76.82	0.89	-0.00	5.47
○Thread 298857			4.45	5.81	76.65	0.89	-0.00	5.45
○Thread 298858			4.44	5.81	76.57	0.89	-0.00	5.47
○Thread 298859			4.34	5.81	74.76	0.89	-0.00	5.47
○Thread 298860			4.38	5.81	75.54	0.89	-0.00	5.46
○Thread 298861			4.48	5.81	77.17	0.89	-0.00	5.47
○Thread 298862			4.07	5.81	70.20	0.89	-0.00	5.45

Report Configuration