OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	126.18
►Process 26831+		1	126.18
○Thread 26831			126.18	127.63	98.87	0.00	0.00	127.62

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	63.63
►Process 27052+		2	63.63
○Thread 27052			63.63	65.18	97.62	0.00	0.00	63.36
○Thread 27151			63.07	63.36	99.53	1.82	0.00	63.12

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	32.12
►Process 27220+		4	32.12
○Thread 27220			32.12	33.63	95.51	0.00	0.00	31.82
○Thread 27319			31.53	31.82	99.09	1.81	0.00	31.58
○Thread 27320			31.53	31.82	99.08	1.81	0.00	31.57
○Thread 27321			31.54	31.82	99.11	1.81	0.00	31.58

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	16.53
►Process 27341+		8	16.53
○Thread 27341			16.53	18.02	91.72	0.00	0.00	16.20
○Thread 27440			15.92	16.21	98.20	1.81	0.00	15.96
○Thread 27441			15.94	16.21	98.32	1.81	0.00	15.96
○Thread 27442			15.92	16.21	98.23	1.81	0.00	15.96
○Thread 27443			15.92	16.21	98.23	1.81	0.00	15.96
○Thread 27444			15.93	16.21	98.25	1.81	0.00	15.96
○Thread 27445			15.93	16.21	98.28	1.81	0.00	15.96
○Thread 27446			15.89	16.21	98.03	1.81	0.00	15.96

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	8.80
►Process 27515+		16	8.80
○Thread 27515			8.80	10.23	86.00	0.00	0.00	8.41
○Thread 27614			8.14	8.42	96.61	1.80	0.01	8.18
○Thread 27615			8.14	8.42	96.67	1.80	0.01	8.18
○Thread 27616			8.14	8.42	96.61	1.80	0.01	8.18
○Thread 27617			8.15	8.42	96.72	1.80	0.01	8.18
○Thread 27618			8.12	8.42	96.42	1.80	0.01	8.18
○Thread 27619			8.14	8.42	96.66	1.80	0.00	8.18
○Thread 27620			8.13	8.42	96.48	1.80	0.00	8.18
○Thread 27621			8.14	8.42	96.65	1.80	0.00	8.18
○Thread 27622			8.15	8.42	96.71	1.80	0.00	8.18
○Thread 27623			8.11	8.42	96.23	1.80	0.00	8.18
○Thread 27624			8.12	8.42	96.41	1.80	0.00	8.18
○Thread 27625			8.10	8.42	96.10	1.80	0.00	8.18
○Thread 27626			8.11	8.42	96.28	1.80	0.00	8.18
○Thread 27627			8.14	8.42	96.63	1.80	0.00	8.18
○Thread 27628			8.14	8.42	96.57	1.80	0.00	8.18

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	6.58
►Process 27648+		24	6.58
○Thread 27648			6.58	8.11	81.13	0.00	0.00	6.29
○Thread 27747			6.00	6.29	95.26	1.81	0.01	6.04
○Thread 27748			6.02	6.29	95.58	1.81	0.01	6.05
○Thread 27749			6.03	6.29	95.81	1.81	0.01	6.05
○Thread 27750			6.00	6.29	95.25	1.81	0.01	6.05
○Thread 27751			6.00	6.29	95.25	1.81	0.01	6.05
○Thread 27752			5.97	6.29	94.85	1.81	0.01	6.05
○Thread 27753			5.99	6.29	95.08	1.81	0.01	6.05
○Thread 27754			5.96	6.29	94.68	1.81	0.01	6.04
○Thread 27755			6.00	6.29	95.24	1.81	0.01	6.05
○Thread 27756			5.99	6.30	95.07	1.81	0.01	6.04
○Thread 27757			6.01	6.30	95.39	1.81	0.01	6.05
○Thread 27758			6.00	6.30	95.23	1.81	0.00	6.04
○Thread 27759			6.00	6.30	95.30	1.81	0.00	6.05
○Thread 27760			6.01	6.30	95.38	1.81	0.00	6.04
○Thread 27761			6.00	6.30	95.29	1.81	0.00	6.04
○Thread 27762			6.02	6.30	95.53	1.81	0.00	6.05
○Thread 27763			6.02	6.30	95.61	1.81	0.00	6.05
○Thread 27764			6.00	6.30	95.28	1.81	0.00	6.05
○Thread 27765			5.98	6.30	94.96	1.81	0.00	6.05
○Thread 27766			5.99	6.30	95.04	1.81	0.00	6.04
○Thread 27767			6.00	6.30	95.19	1.81	0.00	6.05
○Thread 27768			5.98	6.30	94.95	1.81	0.00	6.05
○Thread 27769			5.82	6.30	92.41	1.81	0.00	6.04

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	5.34
►Process 27791+		32	5.34
○Thread 27791			5.34	6.89	77.40	0.00	0.00	5.07
○Thread 27890			4.78	5.08	93.95	1.80	0.01	4.83
○Thread 27891			4.79	5.08	94.24	1.80	0.01	4.83
○Thread 27892			4.78	5.08	94.04	1.80	0.01	4.83
○Thread 27893			4.77	5.08	93.74	1.80	0.01	4.83
○Thread 27894			4.78	5.08	94.03	1.80	0.01	4.83
○Thread 27895			4.77	5.08	93.74	1.80	0.01	4.83
○Thread 27896			4.75	5.08	93.34	1.80	0.01	4.83
○Thread 27897			4.79	5.08	94.12	1.80	0.01	4.83
○Thread 27898			4.78	5.08	93.92	1.80	0.01	4.83
○Thread 27899			4.77	5.08	93.72	1.80	0.01	4.83
○Thread 27900			4.78	5.08	93.91	1.80	0.01	4.83
○Thread 27901			4.79	5.08	94.10	1.80	0.01	4.83
○Thread 27902			4.79	5.08	94.20	1.80	0.01	4.83
○Thread 27903			4.80	5.09	94.39	1.80	0.01	4.81
○Thread 27904			4.74	5.09	93.11	1.80	0.01	4.83
○Thread 27905			4.80	5.09	94.29	1.80	0.01	4.83
○Thread 27906			4.77	5.09	93.69	1.80	0.01	4.83
○Thread 27907			4.79	5.09	94.18	1.80	0.01	4.83
○Thread 27908			4.78	5.09	93.98	1.80	0.01	4.83
○Thread 27909			4.76	5.09	93.58	1.80	0.00	4.83
○Thread 27910			4.77	5.09	93.67	1.80	0.00	4.81
○Thread 27911			4.77	5.09	93.67	1.80	0.00	4.83
○Thread 27912			4.77	5.09	93.76	1.80	0.00	4.83
○Thread 27913			4.78	5.09	93.96	1.80	0.00	4.82
○Thread 27914			4.78	5.09	93.85	1.80	0.00	4.83
○Thread 27915			4.79	5.09	94.15	1.80	0.00	4.83
○Thread 27916			4.77	5.09	93.65	1.80	0.00	4.83
○Thread 27917			4.80	5.09	94.24	1.80	0.00	4.83
○Thread 27918			4.76	5.09	93.44	1.80	0.00	4.83
○Thread 27919			4.74	5.09	93.15	1.80	0.00	4.83
○Thread 27920			4.77	5.09	93.63	1.80	0.00	4.83

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	4.65
►Process 27942+		40	4.65
○Thread 27942			4.65	6.19	75.13	0.00	0.00	4.34
○Thread 28041			4.06	4.35	93.36	1.83	0.01	4.10
○Thread 28042			4.08	4.35	93.81	1.83	0.01	4.10
○Thread 28043			4.05	4.35	93.12	1.83	0.01	4.10
○Thread 28044			4.03	4.35	92.76	1.83	0.01	4.09
○Thread 28045			4.05	4.35	93.10	1.83	0.01	4.09
○Thread 28046			4.04	4.35	92.98	1.83	0.01	4.10
○Thread 28047			4.04	4.35	92.86	1.83	0.01	4.10
○Thread 28048			4.05	4.35	93.08	1.83	0.01	4.09
○Thread 28049			4.06	4.35	93.31	1.83	0.01	4.10
○Thread 28050			4.03	4.35	92.73	1.83	0.01	4.10
○Thread 28051			4.05	4.35	93.18	1.83	0.01	4.10
○Thread 28052			4.02	4.35	92.49	1.83	0.01	4.09
○Thread 28053			4.04	4.35	92.94	1.83	0.01	4.09
○Thread 28054			4.02	4.35	92.48	1.83	0.01	4.09
○Thread 28055			4.04	4.35	92.93	1.83	0.01	4.10
○Thread 28056			4.02	4.35	92.47	1.83	0.01	4.09
○Thread 28057			4.02	4.35	92.35	1.83	0.01	4.10
○Thread 28058			4.06	4.35	93.26	1.83	0.01	4.10
○Thread 28059			4.04	4.35	92.80	1.83	0.01	4.10
○Thread 28060			4.01	4.35	92.22	1.83	0.01	4.09
○Thread 28061			4.05	4.35	93.02	1.83	0.01	4.10
○Thread 28062			4.02	4.35	92.44	1.83	0.01	4.10
○Thread 28063			4.04	4.35	92.89	1.83	0.01	4.10
○Thread 28064			4.03	4.35	92.66	1.83	0.01	4.10
○Thread 28065			4.03	4.35	92.66	1.83	0.01	4.10
○Thread 28066			4.01	4.36	92.19	1.83	0.01	4.10
○Thread 28067			4.03	4.36	92.65	1.83	0.00	4.10
○Thread 28068			4.00	4.36	91.95	1.83	0.00	4.10
○Thread 28069			4.04	4.36	92.75	1.83	0.00	4.09
○Thread 28070			4.04	4.36	92.75	1.83	0.00	4.10
○Thread 28071			4.06	4.36	93.32	1.83	0.00	4.10
○Thread 28072			4.05	4.36	93.08	1.83	0.00	4.10
○Thread 28073			4.00	4.36	91.81	1.83	0.00	4.09
○Thread 28074			4.05	4.36	92.96	1.83	0.00	4.10
○Thread 28075			4.08	4.36	93.64	1.83	0.00	4.10
○Thread 28076			4.04	4.36	92.83	1.83	0.00	4.09
○Thread 28077			4.00	4.36	91.80	1.83	0.00	4.10
○Thread 28078			4.04	4.36	92.71	1.83	0.00	4.10
○Thread 28079			3.85	4.36	88.34	1.83	0.00	4.08

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	4.04
►Process 28152+		48	4.04
○Thread 28152			4.04	5.67	71.23	0.00	0.00	3.81
○Thread 28251			3.52	3.84	91.91	1.82	0.02	3.58
○Thread 28252			3.49	3.84	91.12	1.82	0.02	3.59
○Thread 28253			3.54	3.84	92.29	1.82	0.02	3.59
○Thread 28254			3.54	3.84	92.42	1.82	0.02	3.59
○Thread 28255			3.51	3.84	91.50	1.82	0.02	3.59
○Thread 28256			3.50	3.84	91.23	1.82	0.02	3.59
○Thread 28257			3.53	3.84	92.14	1.82	0.02	3.59
○Thread 28258			3.52	3.84	91.87	1.82	0.01	3.58
○Thread 28259			3.50	3.84	91.22	1.82	0.01	3.59
○Thread 28260			3.51	3.84	91.47	1.82	0.01	3.59
○Thread 28261			3.54	3.84	92.25	1.82	0.01	3.59
○Thread 28262			3.52	3.84	91.72	1.82	0.01	3.58
○Thread 28263			3.50	3.84	91.19	1.82	0.01	3.58
○Thread 28264			3.54	3.84	92.23	1.82	0.01	3.59
○Thread 28265			3.53	3.84	92.10	1.82	0.01	3.58
○Thread 28266			3.52	3.84	91.70	1.82	0.01	3.58
○Thread 28267			3.54	3.84	92.35	1.82	0.01	3.59
○Thread 28268			3.52	3.84	91.69	1.82	0.01	3.59
○Thread 28269			3.51	3.84	91.55	1.82	0.01	3.59
○Thread 28270			3.55	3.84	92.46	1.82	0.01	3.59
○Thread 28271			3.54	3.84	92.19	1.82	0.01	3.59
○Thread 28272			3.55	3.84	92.58	1.82	0.01	3.58
○Thread 28273			3.51	3.84	91.53	1.82	0.01	3.59
○Thread 28274			3.50	3.84	91.26	1.82	0.01	3.58
○Thread 28275			3.52	3.84	91.78	1.82	0.01	3.59
○Thread 28276			3.52	3.84	91.64	1.82	0.01	3.59
○Thread 28277			3.51	3.84	91.51	1.82	0.01	3.59
○Thread 28278			3.51	3.84	91.50	1.82	0.01	3.57
○Thread 28279			3.51	3.84	91.50	1.82	0.01	3.59
○Thread 28280			3.49	3.84	90.97	1.82	0.01	3.59
○Thread 28281			3.55	3.84	92.40	1.82	0.01	3.59
○Thread 28282			3.52	3.84	91.61	1.82	0.01	3.58
○Thread 28283			3.53	3.84	92.00	1.82	0.01	3.59
○Thread 28284			3.51	3.84	91.47	1.82	0.01	3.59
○Thread 28285			3.48	3.84	90.56	1.82	0.01	3.57
○Thread 28286			3.55	3.84	92.37	1.82	0.00	3.59
○Thread 28287			3.48	3.84	90.68	1.82	0.00	3.59
○Thread 28288			3.53	3.84	91.84	1.82	0.00	3.59
○Thread 28289			3.54	3.84	92.23	1.82	0.00	3.58
○Thread 28290			3.53	3.84	91.83	1.82	0.00	3.58
○Thread 28291			3.55	3.84	92.48	1.82	0.00	3.59
○Thread 28292			3.50	3.84	91.04	1.82	0.00	3.58
○Thread 28293			3.50	3.84	91.16	1.83	0.00	3.58
○Thread 28294			3.53	3.84	91.81	1.83	0.00	3.58
○Thread 28295			3.49	3.85	90.89	1.83	0.00	3.58
○Thread 28296			3.40	3.85	88.55	1.83	0.00	3.58
○Thread 28297			3.29	3.85	85.55	1.83	0.00	3.58

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	3.63
►Process 28317+		56	3.63
○Thread 28317			3.63	5.28	68.83	0.00	0.00	3.43
○Thread 28416			3.12	3.45	90.37	1.81	0.02	3.20
○Thread 28417			3.12	3.45	90.51	1.81	0.02	3.20
○Thread 28418			3.11	3.45	90.07	1.81	0.02	3.20
○Thread 28419			3.15	3.45	91.22	1.81	0.02	3.20
○Thread 28420			3.14	3.45	90.93	1.81	0.02	3.20
○Thread 28421			3.11	3.45	90.20	1.81	0.02	3.20
○Thread 28422			3.12	3.45	90.48	1.81	0.02	3.20
○Thread 28423			3.12	3.45	90.33	1.81	0.02	3.20
○Thread 28424			3.15	3.45	91.34	1.81	0.02	3.20
○Thread 28425			3.12	3.45	90.32	1.81	0.02	3.20
○Thread 28426			3.12	3.45	90.32	1.81	0.02	3.20
○Thread 28427			3.13	3.45	90.74	1.81	0.02	3.20
○Thread 28428			3.14	3.45	90.88	1.81	0.02	3.19
○Thread 28429			3.13	3.46	90.73	1.81	0.02	3.19
○Thread 28430			3.14	3.46	91.02	1.81	0.02	3.20
○Thread 28431			3.15	3.46	91.30	1.81	0.02	3.19
○Thread 28432			3.12	3.46	90.28	1.81	0.02	3.20
○Thread 28433			3.08	3.46	89.26	1.81	0.02	3.18
○Thread 28434			3.12	3.46	90.41	1.81	0.01	3.20
○Thread 28435			3.13	3.46	90.70	1.81	0.01	3.20
○Thread 28436			3.08	3.46	89.10	1.81	0.01	3.19
○Thread 28437			3.12	3.46	90.40	1.81	0.01	3.20
○Thread 28438			3.15	3.46	91.11	1.81	0.01	3.20
○Thread 28439			3.07	3.46	88.79	1.81	0.01	3.20
○Thread 28440			3.14	3.46	90.81	1.81	0.01	3.20
○Thread 28441			3.12	3.46	90.23	1.81	0.01	3.20
○Thread 28442			3.12	3.46	90.37	1.81	0.01	3.20
○Thread 28443			3.14	3.46	90.79	1.81	0.01	3.20
○Thread 28444			3.14	3.46	90.93	1.81	0.01	3.20
○Thread 28445			3.14	3.46	90.93	1.81	0.01	3.18
○Thread 28446			3.10	3.46	89.76	1.81	0.01	3.20
○Thread 28447			3.12	3.46	90.19	1.81	0.01	3.20
○Thread 28448			3.14	3.46	90.76	1.81	0.01	3.20
○Thread 28449			3.12	3.46	90.33	1.81	0.01	3.20
○Thread 28450			3.10	3.46	89.74	1.81	0.01	3.20
○Thread 28451			3.14	3.46	90.75	1.81	0.01	3.20
○Thread 28452			3.13	3.46	90.45	1.81	0.01	3.20
○Thread 28453			3.09	3.46	89.43	1.81	0.01	3.20
○Thread 28454			3.11	3.46	90.01	1.81	0.01	3.19
○Thread 28455			3.12	3.46	90.15	1.81	0.01	3.20
○Thread 28456			3.12	3.46	90.14	1.81	0.01	3.20
○Thread 28457			3.08	3.46	88.98	1.81	0.01	3.19
○Thread 28458			3.09	3.46	89.41	1.81	0.01	3.20
○Thread 28459			3.07	3.46	88.82	1.81	0.01	3.20
○Thread 28460			3.11	3.46	89.97	1.81	0.01	3.20
○Thread 28461			3.13	3.46	90.51	1.81	0.00	3.18
○Thread 28462			3.11	3.46	89.93	1.81	0.00	3.18
○Thread 28463			3.11	3.46	89.78	1.81	0.00	3.20
○Thread 28464			3.12	3.46	90.06	1.81	0.00	3.20
○Thread 28465			3.13	3.46	90.34	1.81	0.00	3.20
○Thread 28466			3.10	3.46	89.61	1.81	0.00	3.20
○Thread 28467			3.07	3.47	88.74	1.81	0.00	3.20
○Thread 28468			3.06	3.47	88.30	1.81	0.00	3.20
○Thread 28469			2.95	3.47	85.27	1.81	0.00	3.19
○Thread 28470			2.93	3.47	84.68	1.81	0.00	3.20

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	3.36
►Process 28490+		64	3.36
○Thread 28490			3.36	4.97	67.66	0.00	0.00	3.11
○Thread 28589			2.78	3.12	89.01	1.82	0.02	2.87
○Thread 28590			2.79	3.12	89.49	1.82	0.02	2.87
○Thread 28591			2.80	3.12	89.64	1.82	0.02	2.87
○Thread 28592			2.78	3.12	89.16	1.82	0.02	2.85
○Thread 28593			2.76	3.12	88.35	1.82	0.02	2.86
○Thread 28594			2.77	3.12	88.82	1.82	0.02	2.87
○Thread 28595			2.78	3.12	88.98	1.82	0.02	2.87
○Thread 28596			2.81	3.12	89.93	1.82	0.02	2.86
○Thread 28597			2.77	3.13	88.79	1.82	0.02	2.87
○Thread 28598			2.77	3.13	88.62	1.82	0.02	2.87
○Thread 28599			2.77	3.13	88.61	1.82	0.02	2.84
○Thread 28600			2.78	3.13	89.09	1.82	0.02	2.86
○Thread 28601			2.76	3.13	88.44	1.82	0.02	2.87
○Thread 28602			2.78	3.13	88.91	1.82	0.02	2.86
○Thread 28603			2.77	3.13	88.75	1.82	0.02	2.86
○Thread 28604			2.77	3.13	88.58	1.82	0.02	2.87
○Thread 28605			2.80	3.13	89.69	1.82	0.02	2.87
○Thread 28606			2.78	3.13	88.89	1.82	0.02	2.87
○Thread 28607			2.75	3.13	88.08	1.82	0.02	2.85
○Thread 28608			2.77	3.13	88.55	1.82	0.02	2.87
○Thread 28609			2.72	3.13	86.95	1.82	0.02	2.87
○Thread 28610			2.77	3.13	88.54	1.82	0.02	2.84
○Thread 28611			2.78	3.13	89.01	1.82	0.01	2.87
○Thread 28612			2.76	3.13	88.21	1.82	0.01	2.86
○Thread 28613			2.78	3.13	89.00	1.82	0.01	2.87
○Thread 28614			2.79	3.13	89.16	1.82	0.01	2.85
○Thread 28615			2.74	3.13	87.71	1.82	0.01	2.85
○Thread 28616			2.77	3.13	88.66	1.82	0.01	2.85
○Thread 28617			2.77	3.13	88.66	1.82	0.01	2.85
○Thread 28618			2.75	3.13	88.01	1.82	0.01	2.86
○Thread 28619			2.76	3.13	88.33	1.82	0.01	2.86
○Thread 28620			2.77	3.13	88.64	1.82	0.01	2.86
○Thread 28621			2.79	3.13	89.11	1.82	0.01	2.87
○Thread 28622			2.79	3.13	89.11	1.82	0.01	2.87
○Thread 28623			2.75	3.13	87.98	1.82	0.01	2.85
○Thread 28624			2.81	3.13	89.73	1.82	0.01	2.87
○Thread 28625			2.72	3.13	87.01	1.82	0.01	2.86
○Thread 28626			2.76	3.13	88.12	1.82	0.01	2.87
○Thread 28627			2.78	3.13	88.91	1.82	0.01	2.87
○Thread 28628			2.79	3.13	89.23	1.82	0.01	2.87
○Thread 28629			2.76	3.13	88.10	1.82	0.01	2.85
○Thread 28630			2.81	3.13	89.69	1.83	0.01	2.87
○Thread 28631			2.76	3.13	88.25	1.83	0.01	2.86
○Thread 28632			2.78	3.13	88.88	1.83	0.01	2.86
○Thread 28633			2.78	3.13	88.88	1.83	0.01	2.87
○Thread 28634			2.80	3.13	89.35	1.83	0.01	2.86
○Thread 28635			2.77	3.13	88.54	1.83	0.01	2.87
○Thread 28636			2.79	3.13	89.18	1.83	0.01	2.86
○Thread 28637			2.75	3.13	87.73	1.83	0.01	2.87
○Thread 28638			2.77	3.13	88.37	1.83	0.01	2.85
○Thread 28639			2.77	3.13	88.52	1.83	0.01	2.85
○Thread 28640			2.79	3.14	89.15	1.83	0.00	2.85
○Thread 28641			2.80	3.14	89.46	1.83	0.00	2.87
○Thread 28642			2.79	3.14	89.14	1.83	0.00	2.87
○Thread 28643			2.77	3.14	88.33	1.83	0.00	2.87
○Thread 28644			2.78	3.14	88.65	1.83	0.00	2.86
○Thread 28645			2.76	3.14	88.00	1.83	0.00	2.85
○Thread 28646			2.78	3.14	88.79	1.83	0.00	2.85
○Thread 28647			2.75	3.14	87.67	1.83	0.00	2.85
○Thread 28648			2.77	3.14	88.46	1.83	0.00	2.86
○Thread 28649			2.77	3.14	88.29	1.83	0.00	2.87
○Thread 28650			2.76	3.14	87.97	1.83	0.00	2.87
○Thread 28651			2.80	3.14	89.40	1.83	0.00	2.86

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	3.20
►Process 28671+		72	3.20
○Thread 28671			3.20	4.80	66.74	0.00	0.00	2.92
○Thread 28770			2.58	2.94	87.76	1.83	0.03	2.67
○Thread 28771			2.59	2.94	88.26	1.84	0.03	2.68
○Thread 28772			2.57	2.94	87.57	1.84	0.03	2.68
○Thread 28773			2.59	2.94	88.04	1.84	0.02	2.68
○Thread 28774			2.58	2.94	87.86	1.84	0.02	2.68
○Thread 28775			2.58	2.94	87.86	1.84	0.02	2.64
○Thread 28776			2.59	2.94	88.02	1.84	0.02	2.68
○Thread 28777			2.56	2.94	86.99	1.84	0.02	2.66
○Thread 28778			2.60	2.94	88.52	1.84	0.02	2.68
○Thread 28779			2.63	2.94	89.36	1.84	0.02	2.68
○Thread 28780			2.58	2.94	87.82	1.84	0.02	2.68
○Thread 28781			2.59	2.94	87.99	1.84	0.02	2.68
○Thread 28782			2.56	2.94	87.13	1.84	0.02	2.68
○Thread 28783			2.57	2.94	87.46	1.84	0.02	2.67
○Thread 28784			2.61	2.94	88.65	1.84	0.02	2.67
○Thread 28785			2.57	2.94	87.45	1.84	0.02	2.66
○Thread 28786			2.57	2.94	87.27	1.84	0.02	2.68
○Thread 28787			2.55	2.94	86.76	1.84	0.02	2.68
○Thread 28788			2.60	2.95	88.28	1.84	0.02	2.68
○Thread 28789			2.55	2.95	86.57	1.84	0.02	2.68
○Thread 28790			2.59	2.95	87.92	1.84	0.02	2.67
○Thread 28791			2.58	2.95	87.75	1.84	0.02	2.68
○Thread 28792			2.53	2.95	86.04	1.84	0.02	2.68
○Thread 28793			2.60	2.95	88.41	1.84	0.02	2.68
○Thread 28794			2.55	2.95	86.54	1.84	0.02	2.68
○Thread 28795			2.54	2.95	86.36	1.84	0.02	2.68
○Thread 28796			2.56	2.95	86.87	1.84	0.02	2.68
○Thread 28797			2.59	2.95	87.88	1.84	0.02	2.68
○Thread 28798			2.59	2.95	88.04	1.84	0.02	2.69
○Thread 28799			2.58	2.95	87.52	1.84	0.02	2.68
○Thread 28800			2.56	2.95	86.84	1.84	0.02	2.68
○Thread 28801			2.56	2.95	87.00	1.84	0.01	2.68
○Thread 28802			2.57	2.95	87.16	1.84	0.01	2.68
○Thread 28803			2.57	2.95	87.33	1.84	0.01	2.68
○Thread 28804			2.58	2.95	87.49	1.84	0.01	2.67
○Thread 28805			2.58	2.95	87.65	1.84	0.01	2.68
○Thread 28806			2.60	2.95	88.15	1.84	0.01	2.68
○Thread 28807			2.58	2.95	87.47	1.84	0.01	2.68
○Thread 28808			2.56	2.95	86.95	1.84	0.01	2.65
○Thread 28809			2.58	2.95	87.46	1.84	0.01	2.66
○Thread 28810			2.56	2.95	86.77	1.84	0.01	2.68
○Thread 28811			2.56	2.95	86.93	1.84	0.01	2.68
○Thread 28812			2.57	2.95	87.27	1.84	0.01	2.68
○Thread 28813			2.57	2.95	87.09	1.84	0.01	2.68
○Thread 28814			2.58	2.95	87.42	1.84	0.01	2.68
○Thread 28815			2.54	2.95	86.06	1.84	0.01	2.68
○Thread 28816			2.57	2.95	87.07	1.84	0.01	2.69
○Thread 28817			2.59	2.95	87.74	1.84	0.01	2.68
○Thread 28818			2.54	2.95	86.21	1.84	0.01	2.68
○Thread 28819			2.59	2.95	87.73	1.84	0.01	2.69
○Thread 28820			2.60	2.95	88.06	1.84	0.01	2.68
○Thread 28821			2.56	2.95	86.70	1.84	0.01	2.68
○Thread 28822			2.55	2.95	86.35	1.84	0.01	2.68
○Thread 28823			2.59	2.95	87.70	1.84	0.01	2.66
○Thread 28824			2.55	2.95	86.51	1.84	0.01	2.68
○Thread 28825			2.57	2.95	87.18	1.84	0.01	2.67
○Thread 28826			2.57	2.95	87.17	1.84	0.01	2.68
○Thread 28827			2.60	2.95	88.01	1.84	0.01	2.68
○Thread 28828			2.59	2.95	87.83	1.84	0.01	2.68
○Thread 28829			2.58	2.95	87.32	1.84	0.00	2.68
○Thread 28830			2.59	2.95	87.82	1.84	0.00	2.68
○Thread 28831			2.58	2.96	87.30	1.84	0.00	2.65
○Thread 28832			2.57	2.96	87.13	1.84	0.00	2.68
○Thread 28833			2.59	2.96	87.63	1.84	0.00	2.69
○Thread 28834			2.58	2.96	87.28	1.84	0.00	2.69
○Thread 28835			2.61	2.96	88.29	1.84	0.00	2.69
○Thread 28836			2.61	2.96	88.28	1.84	0.00	2.69
○Thread 28837			2.46	2.96	83.20	1.84	0.00	2.69
○Thread 28838			2.45	2.96	82.86	1.84	0.00	2.68
○Thread 28839			2.44	2.96	82.68	1.84	0.00	2.68
○Thread 28840			2.41	2.96	81.49	1.84	0.00	2.68

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	2.89
►Process 28860+		80	2.89
○Thread 28860			2.89	4.59	62.94	0.00	0.00	2.69
○Thread 28959			2.36	2.73	86.49	1.83	0.03	2.48
○Thread 28960			2.33	2.73	85.20	1.83	0.03	2.47
○Thread 28961			2.37	2.73	86.84	1.83	0.03	2.48
○Thread 28962			2.37	2.74	86.83	1.83	0.03	2.48
○Thread 28963			2.37	2.74	86.82	1.83	0.03	2.47
○Thread 28964			2.38	2.74	87.00	1.83	0.03	2.48
○Thread 28965			2.37	2.74	86.81	1.83	0.03	2.48
○Thread 28966			2.38	2.74	86.98	1.83	0.03	2.48
○Thread 28967			2.34	2.74	85.70	1.83	0.03	2.47
○Thread 28968			2.37	2.74	86.60	1.83	0.03	2.47
○Thread 28969			2.39	2.74	87.51	1.83	0.03	2.48
○Thread 28970			2.35	2.74	86.04	1.83	0.03	2.46
○Thread 28971			2.36	2.74	86.40	1.83	0.03	2.48
○Thread 28972			2.38	2.74	87.12	1.83	0.03	2.48
○Thread 28973			2.38	2.74	87.11	1.83	0.02	2.47
○Thread 28974			2.38	2.74	86.92	1.83	0.02	2.48
○Thread 28975			2.38	2.74	86.92	1.83	0.02	2.48
○Thread 28976			2.34	2.74	85.63	1.83	0.02	2.47
○Thread 28977			2.36	2.74	86.17	1.83	0.02	2.44
○Thread 28978			2.36	2.74	86.16	1.83	0.02	2.47
○Thread 28979			2.38	2.74	87.07	1.83	0.02	2.48
○Thread 28980			2.35	2.74	85.96	1.83	0.02	2.48
○Thread 28981			2.35	2.74	85.77	1.83	0.02	2.47
○Thread 28982			2.34	2.74	85.58	1.83	0.02	2.48
○Thread 28983			2.34	2.74	85.39	1.83	0.02	2.47
○Thread 28984			2.36	2.74	86.12	1.83	0.02	2.48
○Thread 28985			2.39	2.74	87.21	1.83	0.02	2.48
○Thread 28986			2.37	2.74	86.65	1.83	0.02	2.47
○Thread 28987			2.38	2.74	86.83	1.83	0.02	2.48
○Thread 28988			2.36	2.74	86.27	1.83	0.02	2.48
○Thread 28989			2.39	2.74	87.18	1.83	0.02	2.47
○Thread 28990			2.39	2.74	87.17	1.83	0.02	2.47
○Thread 28991			2.37	2.74	86.43	1.83	0.02	2.48
○Thread 28992			2.36	2.74	86.24	1.83	0.02	2.47
○Thread 28993			2.39	2.74	87.15	1.83	0.02	2.48
○Thread 28994			2.36	2.74	86.23	1.83	0.02	2.46
○Thread 28995			2.37	2.74	86.40	1.83	0.02	2.46
○Thread 28996			2.37	2.74	86.39	1.83	0.02	2.48
○Thread 28997			2.38	2.74	86.93	1.83	0.02	2.47
○Thread 28998			2.33	2.74	85.10	1.83	0.02	2.47
○Thread 28999			2.40	2.74	87.47	1.83	0.02	2.47
○Thread 29000			2.38	2.74	86.91	1.83	0.01	2.47
○Thread 29001			2.38	2.74	86.72	1.83	0.01	2.48
○Thread 29002			2.39	2.74	87.26	1.83	0.01	2.47
○Thread 29003			2.37	2.74	86.34	1.83	0.01	2.46
○Thread 29004			2.40	2.75	87.43	1.83	0.01	2.48
○Thread 29005			2.38	2.75	86.69	1.83	0.01	2.47
○Thread 29006			2.39	2.75	87.23	1.83	0.01	2.48
○Thread 29007			2.37	2.75	86.49	1.83	0.01	2.48
○Thread 29008			2.37	2.75	86.49	1.83	0.01	2.47
○Thread 29009			2.35	2.75	85.75	1.83	0.01	2.47
○Thread 29010			2.39	2.75	87.20	1.83	0.01	2.46
○Thread 29011			2.37	2.75	86.46	1.83	0.01	2.47
○Thread 29012			2.39	2.75	87.00	1.83	0.01	2.47
○Thread 29013			2.37	2.75	86.27	1.83	0.01	2.47
○Thread 29014			2.36	2.75	85.89	1.83	0.01	2.47
○Thread 29015			2.35	2.75	85.52	1.83	0.01	2.48
○Thread 29016			2.34	2.75	85.15	1.83	0.01	2.47
○Thread 29017			2.37	2.75	86.23	1.84	0.01	2.48
○Thread 29018			2.38	2.75	86.77	1.84	0.01	2.48
○Thread 29019			2.36	2.75	86.04	1.84	0.01	2.46
○Thread 29020			2.34	2.75	85.30	1.84	0.01	2.48
○Thread 29021			2.35	2.75	85.66	1.84	0.01	2.48
○Thread 29022			2.39	2.75	86.92	1.84	0.01	2.47
○Thread 29023			2.35	2.75	85.64	1.84	0.01	2.48
○Thread 29024			2.35	2.75	85.64	1.84	0.01	2.47
○Thread 29025			2.37	2.75	86.17	1.84	0.01	2.48
○Thread 29026			2.37	2.75	86.35	1.84	0.01	2.48
○Thread 29027			2.37	2.75	86.34	1.84	0.00	2.48
○Thread 29028			2.36	2.75	85.79	1.84	0.00	2.47
○Thread 29029			2.35	2.75	85.42	1.84	0.00	2.47
○Thread 29030			2.34	2.75	85.04	1.84	0.00	2.48
○Thread 29031			2.35	2.75	85.40	1.84	0.00	2.48
○Thread 29032			2.35	2.75	85.57	1.84	0.00	2.48
○Thread 29033			2.34	2.75	85.20	1.84	0.00	2.47
○Thread 29034			2.40	2.75	87.19	1.84	0.00	2.48
○Thread 29035			2.36	2.75	85.73	1.84	0.00	2.48
○Thread 29036			2.39	2.75	86.81	1.84	0.00	2.48
○Thread 29037			2.14	2.75	77.91	1.84	0.00	2.47

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	2.85
►Process 29106+		88	2.85
○Thread 29106			2.85	4.46	64.05	0.00	0.00	2.55
○Thread 29205			2.25	2.59	87.23	1.84	0.03	2.33
○Thread 29206			2.21	2.59	85.68	1.84	0.03	2.31
○Thread 29207			2.23	2.59	86.25	1.84	0.03	2.31
○Thread 29208			2.23	2.59	86.24	1.84	0.03	2.32
○Thread 29209			2.25	2.59	87.20	1.84	0.03	2.31
○Thread 29210			2.23	2.59	86.23	1.84	0.03	2.33
○Thread 29211			2.24	2.59	86.61	1.84	0.03	2.33
○Thread 29212			2.23	2.59	86.21	1.84	0.03	2.33
○Thread 29213			2.18	2.59	84.27	1.84	0.03	2.32
○Thread 29214			2.20	2.59	85.04	1.84	0.03	2.32
○Thread 29215			2.24	2.59	86.58	1.84	0.03	2.33
○Thread 29216			2.23	2.59	86.38	1.84	0.03	2.32
○Thread 29217			2.21	2.59	85.60	1.84	0.03	2.32
○Thread 29218			2.23	2.59	86.17	1.84	0.03	2.33
○Thread 29219			2.23	2.59	86.16	1.84	0.03	2.33
○Thread 29220			2.22	2.59	85.96	1.84	0.03	2.33
○Thread 29221			2.21	2.59	85.57	1.84	0.03	2.33
○Thread 29222			2.21	2.59	85.56	1.84	0.03	2.32
○Thread 29223			2.26	2.59	87.29	1.84	0.03	2.33
○Thread 29224			2.23	2.59	86.12	1.84	0.03	2.32
○Thread 29225			2.22	2.59	85.73	1.84	0.03	2.33
○Thread 29226			2.21	2.59	85.34	1.84	0.03	2.32
○Thread 29227			2.22	2.59	85.91	1.84	0.02	2.33
○Thread 29228			2.22	2.59	85.71	1.84	0.02	2.31
○Thread 29229			2.23	2.59	86.09	1.84	0.02	2.32
○Thread 29230			2.24	2.59	86.47	1.84	0.02	2.32
○Thread 29231			2.17	2.59	83.95	1.84	0.02	2.32
○Thread 29232			2.25	2.59	86.84	1.84	0.02	2.33
○Thread 29233			2.17	2.59	83.94	1.84	0.02	2.32
○Thread 29234			2.23	2.59	86.24	1.84	0.02	2.32
○Thread 29235			2.20	2.59	85.08	1.84	0.02	2.31
○Thread 29236			2.25	2.59	86.81	1.84	0.02	2.33
○Thread 29237			2.25	2.59	86.99	1.84	0.02	2.33
○Thread 29238			2.20	2.59	84.86	1.84	0.02	2.32
○Thread 29239			2.25	2.59	86.79	1.84	0.02	2.32
○Thread 29240			2.21	2.59	85.43	1.84	0.02	2.32
○Thread 29241			2.22	2.59	85.61	1.84	0.02	2.33
○Thread 29242			2.24	2.59	86.57	1.84	0.02	2.33
○Thread 29243			2.20	2.59	85.02	1.84	0.02	2.33
○Thread 29244			2.24	2.59	86.56	1.84	0.02	2.32
○Thread 29245			2.24	2.59	86.36	1.84	0.02	2.31
○Thread 29246			2.22	2.59	85.77	1.85	0.02	2.32
○Thread 29247			2.21	2.59	85.18	1.85	0.02	2.33
○Thread 29248			2.21	2.59	85.37	1.85	0.02	2.31
○Thread 29249			2.21	2.59	85.36	1.85	0.02	2.31
○Thread 29250			2.20	2.60	84.78	1.85	0.02	2.32
○Thread 29251			2.22	2.60	85.73	1.85	0.02	2.32
○Thread 29252			2.24	2.60	86.30	1.85	0.02	2.32
○Thread 29253			2.18	2.60	84.18	1.85	0.02	2.30
○Thread 29254			2.21	2.60	85.32	1.85	0.02	2.33
○Thread 29255			2.24	2.60	86.47	1.85	0.01	2.33
○Thread 29256			2.23	2.60	85.89	1.85	0.01	2.31
○Thread 29257			2.20	2.60	84.92	1.85	0.01	2.32
○Thread 29258			2.20	2.60	84.91	1.85	0.01	2.32
○Thread 29259			2.21	2.60	85.29	1.85	0.01	2.32
○Thread 29260			2.21	2.60	85.09	1.85	0.01	2.31
○Thread 29261			2.26	2.60	87.20	1.85	0.01	2.33
○Thread 29262			2.20	2.60	84.88	1.85	0.01	2.32
○Thread 29263			2.22	2.60	85.45	1.85	0.01	2.32
○Thread 29264			2.21	2.60	85.22	1.85	0.01	2.31
○Thread 29265			2.24	2.60	86.18	1.85	0.01	2.32
○Thread 29266			2.22	2.60	85.40	1.85	0.01	2.32
○Thread 29267			2.20	2.60	84.81	1.85	0.01	2.33
○Thread 29268			2.24	2.60	86.15	1.85	0.01	2.32
○Thread 29269			2.24	2.60	86.14	1.85	0.01	2.32
○Thread 29270			2.23	2.60	85.75	1.85	0.01	2.32
○Thread 29271			2.22	2.60	85.36	1.85	0.01	2.33
○Thread 29272			2.21	2.60	84.97	1.85	0.01	2.32
○Thread 29273			2.23	2.60	85.92	1.85	0.01	2.32
○Thread 29274			2.22	2.60	85.34	1.85	0.01	2.32
○Thread 29275			2.23	2.60	85.71	1.85	0.01	2.32
○Thread 29276			2.23	2.60	85.70	1.85	0.01	2.31
○Thread 29277			2.18	2.60	83.97	1.85	0.01	2.32
○Thread 29278			2.21	2.60	84.92	1.85	0.01	2.32
○Thread 29279			2.26	2.60	86.83	1.85	0.01	2.33
○Thread 29280			2.24	2.60	86.06	1.85	0.00	2.33
○Thread 29281			2.22	2.60	85.47	1.85	0.00	2.33
○Thread 29282			2.23	2.60	85.66	1.85	0.00	2.33
○Thread 29283			2.27	2.60	87.19	1.85	0.00	2.32
○Thread 29284			2.26	2.60	86.80	1.85	0.00	2.33
○Thread 29285			2.23	2.60	85.83	1.85	0.00	2.33
○Thread 29286			2.23	2.60	85.63	1.85	0.00	2.33
○Thread 29287			2.23	2.60	85.81	1.85	0.00	2.32
○Thread 29288			2.22	2.60	85.42	1.85	0.00	2.33
○Thread 29289			2.14	2.61	82.34	1.85	0.00	2.32
○Thread 29290			2.11	2.61	80.99	1.85	0.00	2.33
○Thread 29291			2.02	2.61	77.72	1.85	0.00	2.33

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	2.67
►Process 29311+		96	2.67
○Thread 29311			2.67	4.36	61.30	0.00	0.00	2.46
○Thread 29410			2.09	2.49	84.01	1.83	0.03	2.22
○Thread 29411			2.11	2.49	84.80	1.83	0.03	2.23
○Thread 29412			2.14	2.49	86.00	1.83	0.03	2.22
○Thread 29413			2.12	2.49	85.19	1.83	0.03	2.23
○Thread 29414			2.08	2.49	83.38	1.83	0.03	2.23
○Thread 29415			2.13	2.49	85.58	1.83	0.03	2.22
○Thread 29416			2.11	2.50	84.77	1.83	0.03	2.23
○Thread 29417			2.12	2.50	84.96	1.83	0.03	2.22
○Thread 29418			2.13	2.50	85.35	1.83	0.03	2.23
○Thread 29419			2.11	2.50	84.74	1.83	0.03	2.23
○Thread 29420			2.12	2.50	84.94	1.83	0.03	2.23
○Thread 29421			2.16	2.50	86.53	1.83	0.03	2.23
○Thread 29422			2.10	2.50	84.32	1.83	0.03	2.22
○Thread 29423			2.10	2.50	84.11	1.83	0.03	2.22
○Thread 29424			2.09	2.50	83.90	1.83	0.03	2.23
○Thread 29425			2.12	2.50	84.90	1.83	0.03	2.23
○Thread 29426			2.10	2.50	84.09	1.83	0.03	2.23
○Thread 29427			2.15	2.50	86.28	1.83	0.03	2.23
○Thread 29428			2.13	2.50	85.48	1.83	0.03	2.20
○Thread 29429			2.10	2.50	84.27	1.83	0.03	2.23
○Thread 29430			2.10	2.50	84.06	1.83	0.03	2.23
○Thread 29431			2.12	2.50	85.05	1.83	0.03	2.22
○Thread 29432			2.10	2.50	84.24	1.83	0.03	2.22
○Thread 29433			2.14	2.50	85.64	1.83	0.03	2.23
○Thread 29434			2.05	2.50	82.03	1.83	0.03	2.23
○Thread 29435			2.09	2.50	83.82	1.83	0.03	2.23
○Thread 29436			2.07	2.50	83.01	1.83	0.02	2.23
○Thread 29437			2.11	2.50	84.61	1.83	0.02	2.22
○Thread 29438			2.12	2.50	84.80	1.83	0.02	2.23
○Thread 29439			2.08	2.50	83.19	1.83	0.02	2.23
○Thread 29440			2.14	2.50	85.59	1.83	0.02	2.22
○Thread 29441			2.09	2.50	83.78	1.83	0.02	2.15
○Thread 29442			2.09	2.50	83.77	1.83	0.02	2.23
○Thread 29443			2.09	2.50	83.56	1.83	0.02	2.22
○Thread 29444			2.13	2.50	85.15	1.83	0.02	2.23
○Thread 29445			2.14	2.50	85.75	1.83	0.02	2.23
○Thread 29446			2.08	2.50	83.14	1.83	0.02	2.23
○Thread 29447			2.09	2.50	83.73	1.83	0.02	2.23
○Thread 29448			2.12	2.50	84.92	1.83	0.02	2.24
○Thread 29449			2.07	2.50	82.92	1.83	0.02	2.22
○Thread 29450			2.08	2.50	83.11	1.83	0.02	2.22
○Thread 29451			2.10	2.50	83.90	1.83	0.02	2.23
○Thread 29452			2.13	2.50	85.29	1.83	0.02	2.23
○Thread 29453			2.13	2.50	85.09	1.83	0.02	2.23
○Thread 29454			2.11	2.50	84.48	1.83	0.02	2.22
○Thread 29455			2.12	2.50	84.87	1.83	0.02	2.24
○Thread 29456			2.08	2.50	83.07	1.83	0.02	2.16
○Thread 29457			2.10	2.50	84.06	1.83	0.02	2.23
○Thread 29458			2.12	2.50	84.85	1.83	0.02	2.23
○Thread 29459			2.11	2.50	84.44	1.83	0.02	2.23
○Thread 29460			2.12	2.50	84.83	1.83	0.02	2.23
○Thread 29461			2.12	2.51	84.82	1.83	0.02	2.23
○Thread 29462			2.11	2.51	84.42	1.83	0.02	2.23
○Thread 29463			2.09	2.51	83.61	1.83	0.02	2.22
○Thread 29464			2.13	2.51	85.00	1.83	0.02	2.23
○Thread 29465			2.09	2.51	83.40	1.83	0.01	2.22
○Thread 29466			2.14	2.51	85.58	1.83	0.01	2.23
○Thread 29467			2.10	2.51	83.78	1.83	0.01	2.22
○Thread 29468			2.14	2.51	85.37	1.83	0.01	2.22
○Thread 29469			2.11	2.51	84.36	1.84	0.01	2.22
○Thread 29470			2.15	2.51	85.95	1.84	0.01	2.22
○Thread 29471			2.10	2.51	83.95	1.84	0.01	2.23
○Thread 29472			2.10	2.51	83.74	1.84	0.01	2.21
○Thread 29473			2.16	2.51	86.13	1.84	0.01	2.23
○Thread 29474			2.11	2.51	84.13	1.84	0.01	2.23
○Thread 29475			2.10	2.51	83.92	1.84	0.01	2.22
○Thread 29476			2.07	2.51	82.72	1.84	0.01	2.22
○Thread 29477			2.12	2.51	84.51	1.84	0.01	2.22
○Thread 29478			2.13	2.51	84.90	1.84	0.01	2.22
○Thread 29479			2.10	2.51	83.89	1.84	0.01	2.22
○Thread 29480			2.10	2.51	83.88	1.84	0.01	2.23
○Thread 29481			2.08	2.51	82.88	1.84	0.01	2.23
○Thread 29482			2.12	2.51	84.67	1.84	0.01	2.23
○Thread 29483			2.08	2.51	83.06	1.84	0.01	2.22
○Thread 29484			2.11	2.51	84.25	1.84	0.01	2.23
○Thread 29485			2.12	2.51	84.44	1.84	0.01	2.23
○Thread 29486			2.14	2.51	85.23	1.84	0.01	2.23
○Thread 29487			2.12	2.51	84.43	1.84	0.01	2.23
○Thread 29488			2.14	2.51	85.22	1.84	0.01	2.22
○Thread 29489			2.12	2.51	84.41	1.84	0.01	2.22
○Thread 29490			2.08	2.51	82.81	1.84	0.01	2.23
○Thread 29491			2.15	2.51	85.59	1.84	0.01	2.23
○Thread 29492			2.14	2.51	85.39	1.84	0.01	2.23
○Thread 29493			2.11	2.51	83.98	1.84	0.00	2.21
○Thread 29494			2.12	2.51	84.57	1.84	0.00	2.22
○Thread 29495			2.11	2.51	83.97	1.84	0.00	2.24
○Thread 29496			2.07	2.51	82.37	1.84	0.00	2.20
○Thread 29497			2.09	2.51	83.36	1.84	0.00	2.22
○Thread 29498			2.09	2.51	83.35	1.84	0.00	2.23
○Thread 29499			2.11	2.51	84.14	1.84	0.00	2.23
○Thread 29500			2.11	2.51	84.13	1.84	0.00	2.23
○Thread 29501			2.09	2.51	83.13	1.84	0.00	2.22
○Thread 29502			2.04	2.51	80.93	1.84	0.00	2.22
○Thread 29503			1.95	2.51	77.54	1.84	0.00	2.22
○Thread 29504			1.96	2.51	78.13	1.84	0.00	2.23

Report Configuration