OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	105.93
►Process 608632+		1	105.93
○Thread 608632			105.93	107.37	98.66	0.00	0.00	107.36

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	53.57
►Process 608800+		2	53.57
○Thread 608800			53.57	55.09	97.25	0.00	0.00	53.28
○Thread 608899			52.99	53.29	99.43	1.80	0.00	53.05

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	27.52
►Process 608971+		4	27.52
○Thread 608971			27.52	29.13	94.47	0.00	0.00	27.31
○Thread 609070			27.02	27.31	98.92	1.81	0.00	27.08
○Thread 609071			26.92	27.31	98.58	1.81	0.00	27.07
○Thread 609072			26.93	27.31	98.59	1.81	0.00	27.08

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	14.82
►Process 609092+		8	14.82
○Thread 609092			14.82	16.40	90.32	0.00	0.00	14.58
○Thread 609191			14.22	14.58	97.48	1.82	0.00	14.35
○Thread 609192			14.13	14.58	96.86	1.82	0.00	14.35
○Thread 609193			14.22	14.58	97.51	1.82	0.00	14.35
○Thread 609194			14.08	14.58	96.52	1.82	0.00	14.34
○Thread 609195			14.21	14.58	97.44	1.82	0.00	14.35
○Thread 609196			14.20	14.58	97.34	1.82	0.00	14.34
○Thread 609197			14.20	14.58	97.34	1.82	0.00	14.35

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	8.71
►Process 609267+		16	8.71
○Thread 609267			8.71	10.28	84.70	0.00	0.00	8.44
○Thread 609366			8.02	8.45	94.92	1.83	0.01	8.21
○Thread 609367			7.99	8.45	94.56	1.83	0.01	8.21
○Thread 609368			7.91	8.45	93.55	1.83	0.01	8.21
○Thread 609369			7.97	8.45	94.26	1.83	0.00	8.21
○Thread 609370			7.98	8.45	94.38	1.83	0.00	8.21
○Thread 609371			7.99	8.45	94.55	1.83	0.00	8.21
○Thread 609372			7.91	8.45	93.54	1.83	0.00	8.22
○Thread 609373			7.97	8.45	94.31	1.83	0.00	8.21
○Thread 609374			7.94	8.45	93.95	1.83	0.00	8.21
○Thread 609375			7.92	8.45	93.66	1.83	0.00	8.21
○Thread 609376			7.91	8.45	93.59	1.83	0.00	8.21
○Thread 609377			7.92	8.45	93.71	1.83	0.00	8.21
○Thread 609378			7.86	8.45	93.00	1.83	0.00	8.21
○Thread 609379			7.91	8.45	93.53	1.83	0.00	8.21
○Thread 609380			7.96	8.45	94.12	1.83	0.00	8.21

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	6.77
►Process 609401+		24	6.77
○Thread 609401			6.77	8.65	78.26	0.00	0.00	6.82
○Thread 609500			6.30	6.82	92.32	1.82	0.01	6.58
○Thread 609501			6.23	6.82	91.29	1.82	0.01	6.58
○Thread 609502			6.33	6.82	92.76	1.82	0.01	6.57
○Thread 609503			6.22	6.82	91.14	1.82	0.01	6.58
○Thread 609504			6.25	6.82	91.65	1.82	0.01	6.58
○Thread 609505			6.23	6.82	91.28	1.82	0.01	6.58
○Thread 609506			6.21	6.82	90.98	1.82	0.01	6.58
○Thread 609507			6.25	6.82	91.63	1.82	0.01	6.58
○Thread 609508			6.20	6.82	90.82	1.82	0.01	6.58
○Thread 609509			6.17	6.82	90.38	1.82	0.01	6.57
○Thread 609510			6.15	6.82	90.09	1.82	0.00	6.58
○Thread 609511			6.28	6.82	92.06	1.83	0.00	6.58
○Thread 609512			6.15	6.82	90.15	1.83	0.00	6.58
○Thread 609513			6.23	6.82	91.25	1.83	0.00	6.58
○Thread 609514			6.21	6.82	91.03	1.83	0.00	6.58
○Thread 609515			6.22	6.82	91.10	1.83	0.00	6.58
○Thread 609516			6.08	6.82	89.04	1.83	0.00	6.58
○Thread 609517			6.19	6.82	90.72	1.83	0.00	6.58
○Thread 609518			6.18	6.82	90.50	1.83	0.00	6.58
○Thread 609519			6.14	6.82	89.91	1.83	0.00	6.58
○Thread 609520			6.26	6.82	91.67	1.83	0.00	6.57
○Thread 609521			6.24	6.82	91.37	1.83	0.00	6.56
○Thread 609522			6.11	6.82	89.46	1.83	0.00	6.58

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	5.84
►Process 609542+		32	5.84
○Thread 609542			5.84	7.73	75.47	0.00	0.00	5.89
○Thread 609641			5.33	5.90	90.25	1.82	0.01	5.66
○Thread 609642			5.24	5.90	88.81	1.82	0.01	5.66
○Thread 609643			5.29	5.90	89.65	1.82	0.01	5.66
○Thread 609644			5.29	5.90	89.65	1.82	0.01	5.66
○Thread 609645			5.22	5.90	88.38	1.82	0.01	5.66
○Thread 609646			5.29	5.90	89.64	1.82	0.01	5.66
○Thread 609647			5.17	5.90	87.61	1.82	0.01	5.66
○Thread 609648			5.26	5.90	89.21	1.82	0.01	5.66
○Thread 609649			5.25	5.90	88.96	1.82	0.01	5.66
○Thread 609650			5.26	5.90	89.12	1.82	0.01	5.65
○Thread 609651			5.23	5.90	88.53	1.82	0.01	5.66
○Thread 609652			5.38	5.90	91.06	1.82	0.01	5.66
○Thread 609653			5.28	5.90	89.45	1.82	0.01	5.66
○Thread 609654			5.22	5.90	88.43	1.82	0.01	5.66
○Thread 609655			5.24	5.90	88.68	1.82	0.01	5.66
○Thread 609656			5.22	5.90	88.43	1.82	0.01	5.66
○Thread 609657			5.26	5.90	89.01	1.82	0.01	5.65
○Thread 609658			5.20	5.90	87.99	1.82	0.01	5.66
○Thread 609659			5.17	5.90	87.48	1.82	0.00	5.66
○Thread 609660			5.14	5.90	87.14	1.82	0.00	5.66
○Thread 609661			5.15	5.90	87.14	1.82	0.00	5.65
○Thread 609662			5.15	5.90	87.13	1.82	0.00	5.66
○Thread 609663			5.23	5.90	88.49	1.82	0.00	5.66
○Thread 609664			5.24	5.91	88.65	1.82	0.00	5.66
○Thread 609665			5.21	5.91	88.14	1.82	0.00	5.66
○Thread 609666			5.21	5.91	88.22	1.82	0.00	5.66
○Thread 609667			5.13	5.91	86.95	1.82	0.00	5.66
○Thread 609668			5.16	5.91	87.29	1.82	0.00	5.66
○Thread 609669			5.16	5.91	87.37	1.82	0.00	5.66
○Thread 609670			5.22	5.91	88.38	1.82	0.00	5.66
○Thread 609671			5.25	5.91	88.80	1.82	0.00	5.66

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	5.36
►Process 609691+		40	5.36
○Thread 609691			5.36	7.25	73.91	0.00	0.00	5.42
○Thread 609790			4.74	5.43	87.26	1.81	0.01	5.19
○Thread 609791			4.69	5.43	86.43	1.81	0.01	5.19
○Thread 609792			4.68	5.43	86.24	1.81	0.01	5.18
○Thread 609793			4.69	5.43	86.33	1.81	0.01	5.18
○Thread 609794			4.64	5.43	85.59	1.81	0.01	5.19
○Thread 609795			4.62	5.43	85.13	1.81	0.01	5.19
○Thread 609796			4.66	5.43	85.95	1.81	0.01	5.19
○Thread 609797			4.66	5.43	85.86	1.81	0.01	5.18
○Thread 609798			4.62	5.43	85.03	1.81	0.01	5.19
○Thread 609799			4.60	5.43	84.75	1.81	0.01	5.19
○Thread 609800			4.62	5.43	85.02	1.81	0.01	5.19
○Thread 609801			4.66	5.43	85.84	1.81	0.01	5.19
○Thread 609802			4.58	5.43	84.37	1.81	0.01	5.19
○Thread 609803			4.66	5.43	85.84	1.81	0.01	5.18
○Thread 609804			4.57	5.43	84.27	1.81	0.01	5.19
○Thread 609805			4.61	5.43	84.91	1.81	0.01	5.19
○Thread 609806			4.63	5.43	85.18	1.81	0.01	5.19
○Thread 609807			4.62	5.43	85.09	1.81	0.01	5.18
○Thread 609808			4.64	5.43	85.36	1.81	0.01	5.18
○Thread 609809			4.65	5.43	85.73	1.81	0.01	5.19
○Thread 609810			4.53	5.43	83.42	1.81	0.01	5.19
○Thread 609811			4.67	5.43	86.00	1.81	0.01	5.19
○Thread 609812			4.66	5.43	85.90	1.81	0.01	5.19
○Thread 609813			4.63	5.43	85.16	1.81	0.01	5.18
○Thread 609814			4.62	5.43	85.06	1.81	0.01	5.19
○Thread 609815			4.63	5.43	85.15	1.82	0.01	5.19
○Thread 609816			4.62	5.43	85.06	1.82	0.00	5.19
○Thread 609817			4.56	5.43	84.04	1.82	0.00	5.19
○Thread 609818			4.73	5.43	86.98	1.82	0.00	5.19
○Thread 609819			4.57	5.43	84.22	1.82	0.00	5.18
○Thread 609820			4.56	5.43	83.94	1.82	0.00	5.18
○Thread 609821			4.98	5.43	91.58	1.82	0.00	5.18
○Thread 609822			4.70	5.43	86.42	1.82	0.00	5.19
○Thread 609823			4.66	5.43	85.68	1.82	0.00	5.18
○Thread 609824			4.88	5.43	89.82	1.82	0.00	5.19
○Thread 609825			4.69	5.43	86.41	1.82	0.00	5.18
○Thread 609826			4.52	5.43	83.18	1.82	0.00	5.19
○Thread 609827			4.67	5.43	85.94	1.82	0.00	5.18
○Thread 609828			4.58	5.43	84.19	1.82	0.00	5.17

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	5.07
►Process 609896+		48	5.07
○Thread 609896			5.07	6.99	72.42	0.00	0.00	5.14
○Thread 609996			4.31	5.17	83.43	1.81	0.02	4.93
○Thread 609997			4.36	5.17	84.50	1.81	0.02	4.93
○Thread 609998			4.36	5.17	84.49	1.81	0.02	4.92
○Thread 609999			4.41	5.17	85.36	1.81	0.02	4.92
○Thread 610000			4.27	5.17	82.65	1.81	0.02	4.92
○Thread 610001			4.40	5.17	85.26	1.81	0.02	4.92
○Thread 610002			4.28	5.17	82.83	1.81	0.01	4.92
○Thread 610003			4.30	5.17	83.22	1.81	0.01	4.93
○Thread 610004			4.22	5.17	81.66	1.81	0.01	4.92
○Thread 610005			4.34	5.17	83.98	1.81	0.01	4.92
○Thread 610006			4.27	5.17	82.72	1.81	0.01	4.91
○Thread 610007			4.60	5.17	89.01	1.81	0.01	4.92
○Thread 610008			4.30	5.17	83.20	1.81	0.01	4.92
○Thread 610009			4.36	5.17	84.35	1.81	0.01	4.92
○Thread 610010			4.36	5.17	84.35	1.81	0.01	4.91
○Thread 610011			4.28	5.17	82.90	1.81	0.01	4.93
○Thread 610012			4.24	5.17	82.12	1.81	0.01	4.92
○Thread 610013			4.28	5.17	82.89	1.81	0.01	4.92
○Thread 610014			4.39	5.17	85.01	1.81	0.01	4.92
○Thread 610015			4.24	5.17	82.11	1.81	0.01	4.92
○Thread 610016			4.26	5.17	82.39	1.81	0.01	4.92
○Thread 610017			4.23	5.17	81.91	1.81	0.01	4.92
○Thread 610018			4.27	5.17	82.58	1.81	0.01	4.93
○Thread 610019			4.42	5.17	85.48	1.81	0.01	4.93
○Thread 610020			4.29	5.17	82.96	1.81	0.01	4.92
○Thread 610021			4.30	5.17	83.15	1.81	0.01	4.93
○Thread 610022			4.41	5.17	85.18	1.81	0.01	4.92
○Thread 610023			4.43	5.17	85.66	1.81	0.01	4.93
○Thread 610024			4.28	5.17	82.85	1.81	0.01	4.92
○Thread 610025			4.36	5.17	84.20	1.81	0.01	4.91
○Thread 610026			4.25	5.17	82.26	1.81	0.01	4.92
○Thread 610027			4.44	5.17	85.93	1.82	0.01	4.92
○Thread 610028			4.28	5.17	82.84	1.82	0.01	4.92
○Thread 610029			4.33	5.17	83.70	1.82	0.01	4.92
○Thread 610030			4.39	5.17	84.96	1.82	0.00	4.92
○Thread 610031			4.31	5.17	83.41	1.82	0.00	4.92
○Thread 610032			4.33	5.17	83.69	1.82	0.00	4.93
○Thread 610033			4.22	5.17	81.66	1.82	0.00	4.93
○Thread 610034			4.19	5.17	80.98	1.82	0.00	4.93
○Thread 610035			4.32	5.17	83.49	1.82	0.00	4.91
○Thread 610036			4.25	5.17	82.23	1.82	0.00	4.91
○Thread 610037			4.32	5.17	83.48	1.82	0.00	4.92
○Thread 610038			4.26	5.17	82.32	1.82	0.00	4.93
○Thread 610039			4.49	5.18	86.76	1.82	0.00	4.93
○Thread 610040			4.28	5.18	82.70	1.82	0.00	4.92
○Thread 610041			4.20	5.18	81.25	1.82	0.00	4.92
○Thread 610042			4.14	5.18	80.08	1.82	0.00	4.90

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	4.81
►Process 610062+		56	4.81
○Thread 610062			4.81	6.86	70.14	0.00	0.00	4.99
○Thread 610161			4.09	5.01	81.71	1.83	0.02	4.75
○Thread 610162			4.15	5.01	82.80	1.83	0.02	4.75
○Thread 610163			4.13	5.01	82.40	1.83	0.02	4.77
○Thread 610164			4.18	5.01	83.48	1.83	0.02	4.76
○Thread 610165			4.13	5.01	82.48	1.83	0.02	4.76
○Thread 610166			4.20	5.01	83.87	1.83	0.02	4.76
○Thread 610167			4.07	5.01	81.18	1.83	0.02	4.77
○Thread 610168			3.98	5.01	79.38	1.83	0.02	4.77
○Thread 610169			4.12	5.01	82.27	1.83	0.02	4.76
○Thread 610170			4.11	5.01	81.96	1.83	0.02	4.77
○Thread 610171			4.00	5.01	79.87	1.83	0.02	4.77
○Thread 610172			4.09	5.01	81.56	1.83	0.02	4.77
○Thread 610173			4.00	5.02	79.86	1.83	0.02	4.77
○Thread 610174			4.19	5.02	83.64	1.83	0.02	4.76
○Thread 610175			4.12	5.02	82.15	1.83	0.01	4.77
○Thread 610176			4.04	5.02	80.65	1.83	0.01	4.77
○Thread 610177			3.94	5.02	78.55	1.83	0.01	4.77
○Thread 610178			4.07	5.02	81.24	1.83	0.01	4.75
○Thread 610179			3.92	5.02	78.24	1.83	0.01	4.75
○Thread 610180			4.04	5.02	80.63	1.83	0.01	4.77
○Thread 610181			4.13	5.02	82.42	1.83	0.01	4.77
○Thread 610182			4.03	5.02	80.33	1.83	0.01	4.77
○Thread 610183			3.98	5.02	79.33	1.84	0.01	4.77
○Thread 610184			4.13	5.02	82.41	1.84	0.01	4.77
○Thread 610185			4.06	5.02	81.01	1.84	0.01	4.76
○Thread 610186			4.03	5.02	80.41	1.84	0.01	4.77
○Thread 610187			4.15	5.02	82.80	1.84	0.01	4.76
○Thread 610188			4.21	5.02	83.89	1.84	0.01	4.77
○Thread 610189			4.03	5.02	80.40	1.84	0.01	4.77
○Thread 610190			4.03	5.02	80.40	1.84	0.01	4.77
○Thread 610191			3.89	5.02	77.51	1.84	0.01	4.77
○Thread 610192			4.25	5.02	84.77	1.84	0.01	4.77
○Thread 610193			4.09	5.02	81.58	1.84	0.01	4.77
○Thread 610194			4.11	5.02	81.88	1.84	0.01	4.76
○Thread 610195			4.12	5.02	82.07	1.84	0.01	4.77
○Thread 610196			4.08	5.02	81.37	1.84	0.01	4.77
○Thread 610197			3.92	5.02	78.18	1.84	0.01	4.77
○Thread 610198			4.02	5.02	80.17	1.84	0.01	4.76
○Thread 610199			4.01	5.02	79.87	1.84	0.01	4.77
○Thread 610200			4.07	5.02	81.06	1.84	0.01	4.77
○Thread 610201			4.11	5.02	81.95	1.84	0.01	4.77
○Thread 610202			4.07	5.02	81.05	1.84	0.01	4.77
○Thread 610203			4.05	5.02	80.65	1.84	0.01	4.77
○Thread 610204			4.17	5.02	83.03	1.84	0.00	4.77
○Thread 610205			4.01	5.02	79.94	1.84	0.00	4.75
○Thread 610206			4.20	5.02	83.62	1.84	0.00	4.77
○Thread 610207			4.05	5.02	80.63	1.84	0.00	4.76
○Thread 610208			4.42	5.02	88.10	1.84	0.00	4.77
○Thread 610209			4.15	5.02	82.72	1.84	0.00	4.77
○Thread 610210			4.27	5.02	85.10	1.84	0.00	4.77
○Thread 610211			4.49	5.02	89.38	1.84	0.00	4.77
○Thread 610212			4.22	5.02	84.10	1.84	0.00	4.77
○Thread 610213			4.14	5.02	82.50	1.84	0.00	4.77
○Thread 610214			3.86	5.02	76.83	1.84	0.00	4.77
○Thread 610215			3.97	5.02	79.11	1.84	0.00	4.77

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	4.63
►Process 610236+		64	4.63
○Thread 610236			4.63	6.74	68.73	0.00	0.00	4.90
○Thread 610335			3.87	4.89	79.17	1.83	0.02	4.62
○Thread 610336			3.88	4.89	79.37	1.83	0.02	4.62
○Thread 610337			4.00	4.89	81.82	1.83	0.02	4.64
○Thread 610338			3.90	4.89	79.87	1.83	0.02	4.63
○Thread 610339			3.90	4.89	79.76	1.83	0.02	4.63
○Thread 610340			3.96	4.89	81.08	1.83	0.02	4.64
○Thread 610341			3.84	4.89	78.62	1.83	0.02	4.64
○Thread 610342			3.83	4.89	78.31	1.83	0.02	4.64
○Thread 610343			3.89	4.89	79.54	1.83	0.02	4.64
○Thread 610344			3.84	4.89	78.61	1.83	0.02	4.65
○Thread 610345			3.85	4.89	78.81	1.83	0.02	4.64
○Thread 610346			3.91	4.89	80.04	1.83	0.02	4.64
○Thread 610347			3.89	4.89	79.52	1.83	0.02	4.63
○Thread 610348			3.85	4.89	78.80	1.83	0.02	4.63
○Thread 610349			3.90	4.89	79.82	1.83	0.02	4.63
○Thread 610350			3.92	4.89	80.13	1.83	0.02	4.64
○Thread 610351			3.93	4.89	80.33	1.83	0.02	4.63
○Thread 610352			3.87	4.89	79.20	1.83	0.02	4.64
○Thread 610353			3.87	4.89	79.09	1.83	0.02	4.64
○Thread 610354			3.87	4.89	79.19	1.83	0.02	4.64
○Thread 610355			3.84	4.89	78.58	1.83	0.02	4.64
○Thread 610356			3.82	4.89	78.06	1.83	0.01	4.64
○Thread 610357			3.85	4.89	78.77	1.83	0.01	4.64
○Thread 610358			4.00	4.89	81.73	1.83	0.01	4.64
○Thread 610359			3.86	4.89	78.97	1.83	0.01	4.64
○Thread 610360			3.83	4.89	78.35	1.83	0.01	4.64
○Thread 610361			3.88	4.89	79.27	1.83	0.01	4.64
○Thread 610362			4.03	4.89	82.43	1.83	0.01	4.65
○Thread 610363			3.82	4.90	78.04	1.83	0.01	4.64
○Thread 610364			3.79	4.90	77.42	1.83	0.01	4.65
○Thread 610365			3.81	4.90	77.83	1.83	0.01	4.65
○Thread 610366			4.17	4.90	85.28	1.83	0.01	4.63
○Thread 610367			3.86	4.90	78.84	1.83	0.01	4.65
○Thread 610368			3.96	4.90	80.98	1.83	0.01	4.64
○Thread 610369			3.80	4.90	77.71	1.83	0.01	4.64
○Thread 610370			3.85	4.90	78.63	1.83	0.01	4.65
○Thread 610371			3.78	4.90	77.19	1.83	0.01	4.64
○Thread 610372			3.82	4.90	78.11	1.83	0.01	4.63
○Thread 610373			3.92	4.90	80.05	1.83	0.01	4.65
○Thread 610374			3.97	4.90	81.06	1.83	0.01	4.64
○Thread 610375			3.89	4.90	79.53	1.83	0.01	4.64
○Thread 610376			3.97	4.90	81.06	1.83	0.01	4.64
○Thread 610377			3.86	4.90	78.91	1.83	0.01	4.65
○Thread 610378			3.99	4.90	81.46	1.83	0.01	4.64
○Thread 610379			3.82	4.90	78.09	1.83	0.01	4.64
○Thread 610380			3.82	4.90	78.08	1.83	0.01	4.64
○Thread 610381			3.97	4.90	81.04	1.83	0.01	4.65
○Thread 610382			4.06	4.90	82.87	1.83	0.01	4.64
○Thread 610383			3.85	4.90	78.69	1.83	0.01	4.64
○Thread 610384			3.82	4.90	78.07	1.83	0.01	4.64
○Thread 610385			3.89	4.90	79.49	1.83	0.01	4.65
○Thread 610386			3.93	4.90	80.31	1.83	0.00	4.64
○Thread 610387			3.86	4.90	78.87	1.83	0.00	4.65
○Thread 610388			3.87	4.90	79.08	1.83	0.00	4.64
○Thread 610389			3.82	4.90	77.95	1.83	0.00	4.65
○Thread 610390			3.90	4.90	79.68	1.83	0.00	4.62
○Thread 610391			3.96	4.90	80.80	1.83	0.00	4.65
○Thread 610392			3.87	4.90	79.06	1.83	0.00	4.64
○Thread 610393			3.87	4.90	79.06	1.83	0.00	4.64
○Thread 610394			4.17	4.90	85.17	1.83	0.00	4.64
○Thread 610395			3.93	4.90	80.27	1.83	0.00	4.63
○Thread 610396			3.90	4.90	79.66	1.83	0.00	4.63
○Thread 610397			3.91	4.90	79.86	1.83	0.00	4.64

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	4.52
►Process 610417+		72	4.52
○Thread 610417			4.52	6.76	66.91	0.00	0.00	4.86
○Thread 610516			3.76	4.89	77.06	1.84	0.03	4.64
○Thread 610517			3.89	4.89	79.71	1.84	0.03	4.64
○Thread 610518			3.79	4.89	77.56	1.84	0.02	4.64
○Thread 610519			3.82	4.89	78.28	1.84	0.02	4.64
○Thread 610520			3.85	4.89	78.78	1.84	0.02	4.63
○Thread 610521			3.70	4.89	75.81	1.84	0.02	4.62
○Thread 610522			3.81	4.89	78.06	1.85	0.02	4.63
○Thread 610523			3.84	4.89	78.67	1.85	0.02	4.64
○Thread 610524			3.82	4.89	78.26	1.85	0.02	4.64
○Thread 610525			3.73	4.89	76.31	1.85	0.02	4.62
○Thread 610526			3.81	4.89	77.94	1.85	0.02	4.63
○Thread 610527			3.81	4.89	77.94	1.85	0.02	4.64
○Thread 610528			3.80	4.89	77.83	1.85	0.02	4.63
○Thread 610529			3.87	4.89	79.26	1.85	0.02	4.64
○Thread 610530			3.93	4.89	80.49	1.85	0.02	4.64
○Thread 610531			3.78	4.89	77.31	1.85	0.02	4.63
○Thread 610532			3.77	4.89	77.21	1.85	0.02	4.63
○Thread 610533			3.90	4.89	79.86	1.85	0.02	4.64
○Thread 610534			3.79	4.89	77.61	1.85	0.02	4.63
○Thread 610535			3.74	4.89	76.58	1.85	0.02	4.64
○Thread 610536			3.73	4.89	76.37	1.85	0.02	4.63
○Thread 610537			3.65	4.89	74.73	1.85	0.02	4.64
○Thread 610538			3.75	4.89	76.77	1.85	0.02	4.64
○Thread 610539			3.71	4.89	75.85	1.85	0.02	4.62
○Thread 610540			3.82	4.89	78.10	1.85	0.02	4.63
○Thread 610541			3.80	4.89	77.68	1.85	0.02	4.64
○Thread 610542			3.65	4.89	74.61	1.85	0.02	4.63
○Thread 610543			3.78	4.89	77.37	1.85	0.02	4.64
○Thread 610544			3.64	4.89	74.51	1.85	0.02	4.63
○Thread 610545			3.78	4.89	77.26	1.85	0.02	4.63
○Thread 610546			3.84	4.89	78.48	1.85	0.01	4.64
○Thread 610547			3.81	4.89	77.97	1.85	0.01	4.64
○Thread 610548			3.68	4.89	75.21	1.85	0.01	4.64
○Thread 610549			3.77	4.89	77.15	1.85	0.01	4.63
○Thread 610550			3.70	4.89	75.71	1.85	0.01	4.64
○Thread 610551			3.78	4.89	77.34	1.85	0.01	4.63
○Thread 610552			3.74	4.89	76.52	1.85	0.01	4.63
○Thread 610553			3.64	4.89	74.37	1.85	0.01	4.64
○Thread 610554			3.67	4.89	75.09	1.85	0.01	4.64
○Thread 610555			3.81	4.89	77.94	1.85	0.01	4.64
○Thread 610556			3.78	4.89	77.33	1.85	0.01	4.64
○Thread 610557			3.66	4.90	74.87	1.85	0.01	4.63
○Thread 610558			3.71	4.90	75.89	1.85	0.01	4.62
○Thread 610559			3.81	4.90	77.93	1.85	0.01	4.63
○Thread 610560			3.79	4.90	77.52	1.85	0.01	4.63
○Thread 610561			3.69	4.90	75.37	1.85	0.01	4.63
○Thread 610562			3.59	4.90	73.42	1.85	0.01	4.63
○Thread 610563			3.88	4.90	79.34	1.85	0.01	4.64
○Thread 610564			3.72	4.90	75.97	1.85	0.01	4.64
○Thread 610565			3.76	4.90	76.78	1.85	0.01	4.63
○Thread 610566			3.85	4.90	78.62	1.85	0.01	4.64
○Thread 610567			3.89	4.90	79.53	1.85	0.01	4.63
○Thread 610568			3.78	4.90	77.28	1.85	0.01	4.64
○Thread 610569			3.76	4.90	76.87	1.85	0.01	4.64
○Thread 610570			3.69	4.90	75.44	1.85	0.01	4.61
○Thread 610571			3.83	4.90	78.29	1.85	0.01	4.62
○Thread 610572			3.67	4.90	75.02	1.85	0.01	4.64
○Thread 610573			3.83	4.90	78.18	1.85	0.01	4.64
○Thread 610574			3.74	4.90	76.34	1.85	0.01	4.63
○Thread 610575			3.79	4.90	77.36	1.85	0.00	4.63
○Thread 610576			3.68	4.90	75.21	1.85	0.00	4.63
○Thread 610577			3.79	4.90	77.35	1.85	0.00	4.63
○Thread 610578			3.75	4.90	76.53	1.85	0.00	4.63
○Thread 610579			4.33	4.90	88.37	1.85	0.00	4.64
○Thread 610580			3.74	4.90	76.42	1.85	0.00	4.63
○Thread 610581			3.88	4.90	79.28	1.85	0.00	4.63
○Thread 610582			4.01	4.90	81.93	1.85	0.00	4.64
○Thread 610583			3.79	4.90	77.43	1.85	0.00	4.64
○Thread 610584			3.56	4.90	72.64	1.85	0.00	4.64
○Thread 610585			3.66	4.90	74.67	1.85	0.00	4.64
○Thread 610586			3.56	4.90	72.63	1.85	0.00	4.64

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	4.35
►Process 610656+		80	4.35
○Thread 610656			4.35	6.74	64.54	0.00	0.00	4.86
○Thread 610755			3.76	4.89	77.00	1.82	0.03	4.64
○Thread 610756			3.66	4.89	74.85	1.82	0.03	4.64
○Thread 610757			3.67	4.89	75.16	1.82	0.03	4.64
○Thread 610758			3.76	4.89	76.89	1.82	0.03	4.64
○Thread 610759			3.74	4.89	76.58	1.82	0.03	4.63
○Thread 610760			3.76	4.89	76.99	1.82	0.03	4.64
○Thread 610761			3.66	4.89	74.84	1.82	0.03	4.64
○Thread 610762			3.75	4.89	76.78	1.82	0.03	4.64
○Thread 610763			3.78	4.89	77.28	1.82	0.03	4.64
○Thread 610764			3.67	4.89	75.13	1.82	0.03	4.63
○Thread 610765			3.69	4.89	75.54	1.82	0.03	4.64
○Thread 610766			3.80	4.89	77.68	1.82	0.02	4.64
○Thread 610767			3.62	4.89	74.10	1.82	0.02	4.64
○Thread 610768			3.67	4.89	75.12	1.82	0.02	4.64
○Thread 610769			3.64	4.89	74.50	1.82	0.02	4.64
○Thread 610770			3.78	4.89	77.26	1.82	0.02	4.64
○Thread 610771			3.63	4.89	74.29	1.82	0.02	4.64
○Thread 610772			3.63	4.89	74.29	1.82	0.02	4.64
○Thread 610773			3.70	4.89	75.61	1.82	0.02	4.63
○Thread 610774			3.68	4.89	75.30	1.82	0.02	4.64
○Thread 610775			3.64	4.89	74.48	1.82	0.02	4.64
○Thread 610776			3.68	4.89	75.19	1.82	0.02	4.63
○Thread 610777			3.64	4.89	74.37	1.82	0.02	4.63
○Thread 610778			3.57	4.89	73.04	1.82	0.02	4.63
○Thread 610779			3.75	4.89	76.61	1.82	0.02	4.64
○Thread 610780			3.60	4.90	73.54	1.82	0.02	4.63
○Thread 610781			3.61	4.90	73.74	1.82	0.02	4.64
○Thread 610782			3.72	4.90	75.99	1.83	0.02	4.64
○Thread 610783			3.71	4.90	75.88	1.83	0.02	4.64
○Thread 610784			3.71	4.90	75.88	1.83	0.02	4.64
○Thread 610785			3.60	4.90	73.63	1.83	0.02	4.62
○Thread 610786			3.73	4.90	76.18	1.83	0.02	4.64
○Thread 610787			3.72	4.90	76.07	1.83	0.02	4.64
○Thread 610788			3.62	4.90	73.92	1.83	0.02	4.64
○Thread 610789			3.55	4.90	72.49	1.83	0.02	4.64
○Thread 610790			3.66	4.90	74.84	1.83	0.02	4.64
○Thread 610791			3.63	4.90	74.12	1.83	0.02	4.64
○Thread 610792			3.57	4.90	72.89	1.83	0.02	4.64
○Thread 610793			3.64	4.90	74.42	1.83	0.02	4.63
○Thread 610794			3.64	4.90	74.42	1.83	0.01	4.64
○Thread 610795			3.71	4.90	75.84	1.83	0.01	4.64
○Thread 610796			3.67	4.90	74.92	1.83	0.01	4.64
○Thread 610797			3.56	4.90	72.77	1.83	0.01	4.64
○Thread 610798			3.66	4.90	74.71	1.83	0.01	4.64
○Thread 610799			3.63	4.90	74.09	1.83	0.01	4.64
○Thread 610800			3.60	4.90	73.48	1.83	0.01	4.64
○Thread 610801			3.66	4.90	74.80	1.83	0.01	4.64
○Thread 610802			3.82	4.90	78.06	1.83	0.01	4.64
○Thread 610803			3.52	4.90	71.94	1.83	0.01	4.64
○Thread 610804			3.70	4.90	75.50	1.83	0.01	4.64
○Thread 610805			3.72	4.90	76.01	1.83	0.01	4.64
○Thread 610806			3.87	4.90	78.96	1.83	0.01	4.64
○Thread 610807			3.66	4.90	74.68	1.83	0.01	4.64
○Thread 610808			3.67	4.90	74.88	1.83	0.01	4.64
○Thread 610809			3.65	4.90	74.47	1.83	0.01	4.63
○Thread 610810			3.78	4.90	77.22	1.83	0.01	4.64
○Thread 610811			3.61	4.90	73.74	1.83	0.01	4.64
○Thread 610812			3.68	4.90	75.07	1.83	0.01	4.63
○Thread 610813			3.65	4.90	74.55	1.83	0.01	4.64
○Thread 610814			3.78	4.90	77.20	1.83	0.01	4.64
○Thread 610815			3.74	4.90	76.28	1.83	0.01	4.63
○Thread 610816			3.71	4.90	75.77	1.83	0.01	4.64
○Thread 610817			3.64	4.90	74.23	1.83	0.01	4.64
○Thread 610818			4.01	4.90	81.77	1.83	0.01	4.64
○Thread 610819			3.59	4.90	73.31	1.83	0.01	4.64
○Thread 610820			3.75	4.90	76.47	1.83	0.01	4.64
○Thread 610821			3.81	4.90	77.79	1.83	0.01	4.63
○Thread 610822			3.76	4.90	76.76	1.83	0.00	4.64
○Thread 610823			3.49	4.90	71.26	1.83	0.00	4.64
○Thread 610824			3.64	4.91	74.21	1.83	0.00	4.63
○Thread 610825			3.63	4.91	74.10	1.83	0.00	4.64
○Thread 610826			3.80	4.91	77.46	1.83	0.00	4.64
○Thread 610827			3.64	4.91	74.20	1.83	0.00	4.64
○Thread 610828			3.68	4.91	75.01	1.83	0.00	4.64
○Thread 610829			3.69	4.91	75.21	1.83	0.00	4.64
○Thread 610830			3.72	4.91	75.82	1.83	0.00	4.63
○Thread 610831			3.74	4.91	76.22	1.83	0.00	4.64
○Thread 610832			3.63	4.91	73.98	1.83	0.00	4.64
○Thread 610833			3.47	4.91	70.81	1.83	0.00	4.63

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	4.33
►Process 610853+		88	4.33
○Thread 610853			4.33	6.75	64.15	0.00	0.00	4.88
○Thread 610952			3.76	4.90	76.70	1.82	0.03	4.66
○Thread 610953			3.69	4.90	75.27	1.82	0.03	4.63
○Thread 610954			3.76	4.90	76.70	1.82	0.03	4.65
○Thread 610955			3.56	4.90	72.72	1.82	0.03	4.65
○Thread 610956			3.68	4.90	75.16	1.82	0.03	4.65
○Thread 610957			3.64	4.90	74.24	1.82	0.03	4.64
○Thread 610958			3.68	4.90	75.05	1.82	0.03	4.65
○Thread 610959			3.68	4.90	75.15	1.82	0.03	4.65
○Thread 610960			3.51	4.90	71.68	1.82	0.03	4.65
○Thread 610961			3.68	4.90	75.04	1.82	0.03	4.66
○Thread 610962			3.56	4.90	72.59	1.82	0.03	4.65
○Thread 610963			3.88	4.90	79.11	1.82	0.03	4.65
○Thread 610964			3.68	4.90	75.03	1.82	0.03	4.65
○Thread 610965			3.63	4.90	74.01	1.82	0.03	4.65
○Thread 610966			3.63	4.91	74.11	1.82	0.03	4.62
○Thread 610967			3.62	4.91	73.90	1.82	0.03	4.66
○Thread 610968			3.67	4.91	74.91	1.82	0.03	4.65
○Thread 610969			3.61	4.91	73.58	1.82	0.03	4.66
○Thread 610970			3.56	4.91	72.56	1.82	0.03	4.66
○Thread 610971			3.65	4.91	74.39	1.82	0.03	4.66
○Thread 610972			3.71	4.91	75.61	1.82	0.03	4.66
○Thread 610973			3.63	4.91	73.98	1.82	0.02	4.66
○Thread 610974			3.66	4.91	74.69	1.82	0.02	4.66
○Thread 610975			3.63	4.91	73.97	1.82	0.02	4.65
○Thread 610976			3.48	4.91	71.01	1.82	0.02	4.65
○Thread 610977			3.61	4.91	73.66	1.82	0.02	4.65
○Thread 610978			3.63	4.91	73.96	1.82	0.02	4.64
○Thread 610979			3.57	4.91	72.74	1.82	0.02	4.66
○Thread 610980			3.63	4.91	73.96	1.82	0.02	4.65
○Thread 610981			3.55	4.91	72.43	1.82	0.02	4.65
○Thread 610982			3.63	4.91	74.05	1.82	0.02	4.65
○Thread 610983			3.66	4.91	74.66	1.82	0.02	4.65
○Thread 610984			3.66	4.91	74.55	1.82	0.02	4.66
○Thread 610985			3.60	4.91	73.33	1.82	0.02	4.65
○Thread 610986			3.79	4.91	77.30	1.82	0.02	4.65
○Thread 610987			3.69	4.91	75.26	1.82	0.02	4.66
○Thread 610988			3.57	4.91	72.81	1.82	0.02	4.65
○Thread 610989			3.68	4.91	74.94	1.82	0.02	4.65
○Thread 610990			3.55	4.91	72.29	1.82	0.02	4.65
○Thread 610991			3.68	4.91	74.94	1.82	0.02	4.64
○Thread 610992			3.53	4.91	71.98	1.82	0.02	4.64
○Thread 610993			3.58	4.91	73.00	1.82	0.02	4.64
○Thread 610994			3.59	4.91	73.20	1.82	0.02	4.65
○Thread 610995			3.66	4.91	74.52	1.82	0.02	4.65
○Thread 610996			3.62	4.91	73.70	1.82	0.02	4.66
○Thread 610997			3.71	4.91	75.53	1.82	0.02	4.66
○Thread 610998			3.62	4.91	73.79	1.82	0.02	4.65
○Thread 610999			3.93	4.91	80.00	1.82	0.02	4.65
○Thread 611000			3.60	4.91	73.38	1.82	0.02	4.64
○Thread 611001			3.61	4.91	73.57	1.82	0.01	4.65
○Thread 611002			3.69	4.91	75.09	1.82	0.01	4.65
○Thread 611003			3.75	4.91	76.41	1.82	0.01	4.66
○Thread 611004			3.57	4.91	72.74	1.82	0.01	4.65
○Thread 611005			3.60	4.91	73.25	1.82	0.01	4.65
○Thread 611006			3.66	4.91	74.57	1.82	0.01	4.64
○Thread 611007			3.82	4.92	77.82	1.82	0.01	4.65
○Thread 611008			3.63	4.92	73.95	1.82	0.01	4.65
○Thread 611009			3.73	4.92	75.98	1.82	0.01	4.64
○Thread 611010			3.66	4.92	74.55	1.82	0.01	4.64
○Thread 611011			3.88	4.92	78.92	1.82	0.01	4.65
○Thread 611012			3.68	4.92	74.95	1.82	0.01	4.66
○Thread 611013			3.69	4.92	75.15	1.82	0.01	4.63
○Thread 611014			3.76	4.92	76.57	1.82	0.01	4.65
○Thread 611015			3.83	4.92	77.99	1.82	0.01	4.65
○Thread 611016			3.57	4.92	72.60	1.82	0.01	4.65
○Thread 611017			3.67	4.92	74.63	1.82	0.01	4.66
○Thread 611018			3.71	4.92	75.44	1.82	0.01	4.65
○Thread 611019			3.81	4.92	77.57	1.82	0.01	4.65
○Thread 611020			3.60	4.92	73.20	1.82	0.01	4.65
○Thread 611021			3.63	4.92	73.90	1.82	0.01	4.65
○Thread 611022			3.67	4.92	74.61	1.82	0.01	4.66
○Thread 611023			3.70	4.92	75.22	1.82	0.01	4.65
○Thread 611024			3.76	4.92	76.54	1.82	0.01	4.64
○Thread 611025			3.59	4.92	73.08	1.83	0.01	4.65
○Thread 611026			3.71	4.92	75.51	1.83	0.01	4.65
○Thread 611027			3.65	4.92	74.19	1.83	0.00	4.65
○Thread 611028			3.60	4.92	73.27	1.83	0.00	4.66
○Thread 611029			3.61	4.92	73.37	1.83	0.00	4.66
○Thread 611030			3.70	4.92	75.30	1.83	0.00	4.66
○Thread 611031			4.09	4.92	83.12	1.83	0.00	4.65
○Thread 611032			3.58	4.92	72.75	1.83	0.00	4.65
○Thread 611033			3.71	4.92	75.49	1.83	0.00	4.66
○Thread 611034			3.71	4.92	75.38	1.83	0.00	4.66
○Thread 611035			3.70	4.92	75.28	1.83	0.00	4.66
○Thread 611036			3.62	4.92	73.55	1.83	0.00	4.64
○Thread 611037			3.45	4.92	70.09	1.83	0.00	4.65
○Thread 611038			3.44	4.92	69.88	1.83	0.00	4.65

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	4.46
►Process 611058+		96	4.46
○Thread 611058			4.46	6.88	64.88	0.00	0.00	4.99
○Thread 611157			3.73	5.01	74.54	1.84	0.04	4.75
○Thread 611158			3.71	5.01	74.03	1.84	0.03	4.76
○Thread 611159			3.65	5.01	72.83	1.84	0.03	4.76
○Thread 611160			3.71	5.01	74.03	1.84	0.03	4.75
○Thread 611161			3.68	5.01	73.52	1.84	0.03	4.75
○Thread 611162			3.58	5.01	71.43	1.84	0.03	4.76
○Thread 611163			3.57	5.01	71.22	1.84	0.03	4.76
○Thread 611164			3.68	5.01	73.51	1.84	0.03	4.76
○Thread 611165			3.62	5.01	72.21	1.84	0.03	4.76
○Thread 611166			3.68	5.01	73.41	1.84	0.03	4.75
○Thread 611167			3.73	5.01	74.50	1.84	0.03	4.74
○Thread 611168			3.75	5.01	74.80	1.84	0.03	4.76
○Thread 611169			3.62	5.01	72.30	1.84	0.03	4.76
○Thread 611170			3.78	5.01	75.39	1.84	0.03	4.74
○Thread 611171			3.56	5.01	71.00	1.84	0.03	4.76
○Thread 611172			3.69	5.01	73.69	1.84	0.03	4.76
○Thread 611173			3.66	5.01	72.99	1.84	0.03	4.76
○Thread 611174			3.60	5.01	71.79	1.84	0.03	4.76
○Thread 611175			3.55	5.02	70.88	1.84	0.03	4.75
○Thread 611176			3.61	5.02	71.98	1.84	0.03	4.74
○Thread 611177			3.69	5.02	73.57	1.84	0.03	4.76
○Thread 611178			3.56	5.02	70.97	1.84	0.03	4.76
○Thread 611179			3.48	5.02	69.28	1.84	0.03	4.76
○Thread 611180			3.66	5.02	72.96	1.84	0.03	4.74
○Thread 611181			3.61	5.02	72.06	1.84	0.03	4.76
○Thread 611182			3.75	5.02	74.85	1.84	0.03	4.75
○Thread 611183			3.69	5.02	73.55	1.84	0.03	4.76
○Thread 611184			3.65	5.02	72.85	1.84	0.03	4.75
○Thread 611185			3.71	5.02	73.94	1.84	0.02	4.75
○Thread 611186			3.50	5.02	69.85	1.84	0.02	4.72
○Thread 611187			3.57	5.02	71.14	1.84	0.02	4.76
○Thread 611188			3.76	5.02	74.93	1.84	0.02	4.72
○Thread 611189			3.66	5.02	72.93	1.84	0.02	4.73
○Thread 611190			3.74	5.02	74.62	1.84	0.02	4.74
○Thread 611191			3.49	5.02	69.54	1.84	0.02	4.75
○Thread 611192			3.67	5.02	73.12	1.84	0.02	4.76
○Thread 611193			3.54	5.02	70.53	1.84	0.02	4.76
○Thread 611194			3.63	5.02	72.42	1.84	0.02	4.75
○Thread 611195			3.57	5.02	71.22	1.84	0.02	4.76
○Thread 611196			3.63	5.02	72.41	1.84	0.02	4.76
○Thread 611197			3.61	5.02	72.01	1.84	0.02	4.76
○Thread 611198			3.60	5.02	71.70	1.84	0.02	4.76
○Thread 611199			3.48	5.02	69.31	1.84	0.02	4.76
○Thread 611200			3.68	5.02	73.39	1.84	0.02	4.75
○Thread 611201			3.61	5.02	71.89	1.84	0.02	4.75
○Thread 611202			3.78	5.02	75.38	1.84	0.02	4.75
○Thread 611203			3.57	5.02	71.19	1.84	0.02	4.75
○Thread 611204			3.75	5.02	74.77	1.84	0.02	4.75
○Thread 611205			3.69	5.02	73.57	1.84	0.02	4.76
○Thread 611206			3.57	5.02	71.18	1.84	0.02	4.76
○Thread 611207			3.60	5.02	71.77	1.84	0.02	4.76
○Thread 611208			3.65	5.02	72.67	1.84	0.02	4.76
○Thread 611209			3.62	5.02	71.97	1.84	0.02	4.74
○Thread 611210			3.67	5.02	73.06	1.84	0.02	4.76
○Thread 611211			3.58	5.02	71.26	1.84	0.02	4.75
○Thread 611212			3.70	5.02	73.65	1.84	0.02	4.76
○Thread 611213			3.63	5.02	72.25	1.84	0.01	4.76
○Thread 611214			3.60	5.02	71.65	1.84	0.01	4.76
○Thread 611215			3.62	5.02	72.05	1.84	0.01	4.76
○Thread 611216			3.55	5.02	70.65	1.84	0.01	4.75
○Thread 611217			3.61	5.03	71.84	1.84	0.01	4.76
○Thread 611218			3.47	5.03	69.15	1.84	0.01	4.74
○Thread 611219			3.62	5.03	72.03	1.84	0.01	4.73
○Thread 611220			4.07	5.03	80.98	1.84	0.01	4.75
○Thread 611221			3.60	5.03	71.63	1.84	0.01	4.74
○Thread 611222			3.60	5.03	71.62	1.84	0.01	4.75
○Thread 611223			3.76	5.03	74.80	1.84	0.01	4.76
○Thread 611224			3.70	5.03	73.61	1.84	0.01	4.73
○Thread 611225			3.61	5.03	71.81	1.84	0.01	4.76
○Thread 611226			3.85	5.03	76.58	1.84	0.01	4.76
○Thread 611227			3.69	5.03	73.40	1.84	0.01	4.76
○Thread 611228			3.78	5.03	75.18	1.84	0.01	4.75
○Thread 611229			3.67	5.03	72.99	1.84	0.01	4.76
○Thread 611230			3.73	5.03	74.18	1.84	0.01	4.74
○Thread 611231			3.74	5.03	74.38	1.85	0.01	4.76
○Thread 611232			4.15	5.03	82.63	1.85	0.01	4.76
○Thread 611233			3.84	5.03	76.36	1.85	0.01	4.75
○Thread 611234			3.69	5.03	73.37	1.85	0.01	4.76
○Thread 611235			3.81	5.03	75.76	1.85	0.01	4.75
○Thread 611236			3.84	5.03	76.35	1.85	0.01	4.76
○Thread 611237			3.73	5.03	74.16	1.85	0.01	4.75
○Thread 611238			3.69	5.03	73.46	1.85	0.01	4.76
○Thread 611239			3.65	5.03	72.66	1.85	0.01	4.76
○Thread 611240			3.68	5.03	73.25	1.85	0.00	4.74
○Thread 611241			3.51	5.03	69.87	1.85	0.00	4.75
○Thread 611242			3.65	5.03	72.55	1.85	0.00	4.75
○Thread 611243			3.52	5.03	70.06	1.85	0.00	4.75
○Thread 611244			3.64	5.03	72.34	1.85	0.00	4.75
○Thread 611245			3.66	5.03	72.84	1.85	0.00	4.76
○Thread 611246			3.60	5.03	71.54	1.85	0.00	4.75
○Thread 611247			3.68	5.03	73.23	1.85	0.00	4.74
○Thread 611248			3.69	5.03	73.42	1.85	0.00	4.76
○Thread 611249			3.36	5.03	66.86	1.85	0.00	4.73
○Thread 611250			3.32	5.03	65.96	1.85	0.00	4.76
○Thread 611251			3.43	5.03	68.15	1.85	0.00	4.73

Report Configuration