OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	17.29
►Process 867930+		1	17.29
○Thread 867930			17.29	18.79	92.03	0.00	0.00	18.78

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	9.08
►Process 868100+		2	9.08
○Thread 868100			9.08	10.53	86.23	0.00	0.00	8.70
○Thread 868199			8.44	8.71	96.93	1.82	0.00	8.47

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	4.88
►Process 868221+		4	4.88
○Thread 868221			4.88	6.39	76.28	0.00	0.00	4.57
○Thread 868320			4.29	4.58	93.75	1.81	0.00	4.34
○Thread 868321			4.31	4.58	94.30	1.81	0.00	4.34
○Thread 868322			4.31	4.58	94.29	1.81	0.00	4.34

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	2.79
►Process 868342+		8	2.79
○Thread 868342			2.79	4.33	64.42	0.00	0.00	2.52
○Thread 868441			2.25	2.52	89.23	1.81	0.00	2.28
○Thread 868442			2.26	2.52	89.61	1.81	0.00	2.28
○Thread 868443			2.25	2.52	89.41	1.81	0.00	2.28
○Thread 868444			2.23	2.52	88.41	1.81	0.00	2.28
○Thread 868445			2.24	2.52	88.80	1.81	0.00	2.28
○Thread 868446			2.25	2.52	89.39	1.81	0.00	2.28
○Thread 868447			2.27	2.52	89.97	1.81	0.00	2.28

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	1.85
►Process 868467+		16	1.85
○Thread 868467			1.85	3.32	55.87	0.00	0.00	1.80
○Thread 868566			1.25	1.50	83.30	1.81	0.01	1.26
○Thread 868567			1.24	1.50	82.27	1.81	0.01	1.26
○Thread 868568			1.23	1.50	82.25	1.81	0.01	1.26
○Thread 868569			1.21	1.50	80.91	1.81	0.00	1.26
○Thread 868570			1.21	1.50	80.90	1.81	0.00	1.26
○Thread 868571			1.22	1.50	81.55	1.81	0.00	1.26
○Thread 868572			1.21	1.50	80.54	1.81	0.00	1.26
○Thread 868573			1.21	1.50	80.86	1.81	0.00	1.26
○Thread 868574			1.23	1.50	81.85	1.81	0.00	1.26
○Thread 868575			1.22	1.50	81.50	1.81	0.00	1.26
○Thread 868576			1.21	1.50	80.83	1.81	0.00	1.26
○Thread 868577			1.20	1.50	80.15	1.81	0.00	1.26
○Thread 868578			1.23	1.50	81.80	1.81	0.00	1.26
○Thread 868579			1.22	1.50	81.45	1.82	0.00	1.25
○Thread 868580			1.22	1.50	81.11	1.82	0.00	1.26

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	1.58
►Process 868600+		24	1.58
○Thread 868600			1.58	3.06	51.77	0.00	0.00	1.80
○Thread 868699			0.94	1.23	76.90	1.82	0.01	0.98
○Thread 868700			0.96	1.23	78.52	1.82	0.01	0.99
○Thread 868701			0.98	1.23	79.32	1.82	0.01	0.99
○Thread 868702			0.94	1.23	76.86	1.82	0.01	0.97
○Thread 868703			0.93	1.23	75.63	1.82	0.01	0.98
○Thread 868704			0.93	1.23	75.21	1.82	0.01	0.99
○Thread 868705			0.90	1.23	73.57	1.82	0.01	0.99
○Thread 868706			0.94	1.23	76.81	1.83	0.01	0.97
○Thread 868707			0.94	1.23	76.80	1.83	0.01	0.97
○Thread 868708			0.93	1.23	75.57	1.83	0.01	0.98
○Thread 868709			0.94	1.23	75.96	1.83	0.01	0.99
○Thread 868710			0.94	1.23	76.35	1.83	0.00	0.98
○Thread 868711			0.95	1.23	77.15	1.83	0.00	0.99
○Thread 868712			0.94	1.23	75.92	1.83	0.00	0.99
○Thread 868713			0.93	1.23	75.09	1.83	0.00	0.97
○Thread 868714			0.96	1.23	77.92	1.83	0.00	0.98
○Thread 868715			0.93	1.23	75.47	1.83	0.00	0.98
○Thread 868716			0.94	1.23	76.67	1.83	0.00	0.99
○Thread 868717			0.95	1.23	77.07	1.83	0.00	0.99
○Thread 868718			0.92	1.23	74.21	1.83	0.00	0.99
○Thread 868719			0.94	1.23	76.23	1.83	0.00	0.99
○Thread 868720			0.95	1.23	77.43	1.83	0.00	0.99
○Thread 868721			0.93	1.23	74.98	1.83	0.00	0.98

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	1.31
►Process 868790+		32	1.31
○Thread 868790			1.31	2.90	45.14	0.00	0.00	1.79
○Thread 868889			0.76	1.07	71.08	1.82	0.01	0.82
○Thread 868890			0.77	1.07	72.00	1.82	0.01	0.83
○Thread 868891			0.77	1.07	72.46	1.82	0.01	0.83
○Thread 868892			0.77	1.07	72.44	1.82	0.01	0.83
○Thread 868893			0.79	1.07	73.36	1.82	0.01	0.82
○Thread 868894			0.77	1.07	72.41	1.82	0.01	0.83
○Thread 868895			0.78	1.07	72.87	1.82	0.01	0.81
○Thread 868896			0.81	1.07	75.66	1.82	0.01	0.83
○Thread 868897			0.79	1.07	73.77	1.82	0.01	0.83
○Thread 868898			0.80	1.07	74.69	1.82	0.01	0.83
○Thread 868899			0.74	1.07	69.08	1.82	0.01	0.82
○Thread 868900			0.79	1.07	73.26	1.82	0.01	0.82
○Thread 868901			0.78	1.07	72.78	1.82	0.01	0.83
○Thread 868902			0.78	1.07	72.77	1.82	0.01	0.83
○Thread 868903			0.81	1.07	75.08	1.82	0.01	0.83
○Thread 868904			0.80	1.07	74.14	1.82	0.01	0.83
○Thread 868905			0.76	1.07	70.86	1.82	0.01	0.83
○Thread 868906			0.81	1.07	75.50	1.82	0.01	0.83
○Thread 868907			0.77	1.07	72.23	1.82	0.00	0.83
○Thread 868908			0.80	1.07	74.54	1.82	0.00	0.83
○Thread 868909			0.77	1.07	71.73	1.82	0.00	0.83
○Thread 868910			0.79	1.07	73.12	1.82	0.00	0.83
○Thread 868911			0.77	1.07	72.17	1.82	0.00	0.83
○Thread 868912			0.77	1.07	71.69	1.82	0.00	0.83
○Thread 868913			0.79	1.07	73.07	1.82	0.00	0.82
○Thread 868914			0.77	1.07	72.13	1.82	0.00	0.83
○Thread 868915			0.76	1.07	71.18	1.83	0.00	0.82
○Thread 868916			0.77	1.07	72.10	1.83	0.00	0.82
○Thread 868917			0.77	1.08	72.08	1.83	0.00	0.83
○Thread 868918			0.76	1.08	71.14	1.83	0.00	0.83
○Thread 868919			0.77	1.08	71.59	1.83	0.00	0.83

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	1.24
►Process 868939+		40	1.24
○Thread 868939			1.24	2.81	44.28	0.00	0.00	1.77
○Thread 869038			0.69	0.98	70.40	1.82	0.01	0.73
○Thread 869039			0.71	0.98	72.94	1.82	0.01	0.74
○Thread 869040			0.70	0.98	71.90	1.82	0.01	0.73
○Thread 869041			0.67	0.98	68.31	1.82	0.01	0.74
○Thread 869042			0.69	0.98	70.34	1.82	0.01	0.74
○Thread 869043			0.68	0.98	69.30	1.82	0.01	0.73
○Thread 869044			0.70	0.98	71.32	1.82	0.01	0.73
○Thread 869045			0.68	0.98	68.76	1.82	0.01	0.72
○Thread 869046			0.69	0.98	70.28	1.82	0.01	0.73
○Thread 869047			0.71	0.98	72.30	1.82	0.01	0.74
○Thread 869048			0.70	0.98	71.26	1.82	0.01	0.74
○Thread 869049			0.70	0.98	71.25	1.82	0.01	0.73
○Thread 869050			0.68	0.98	69.20	1.82	0.01	0.74
○Thread 869051			0.70	0.98	71.21	1.82	0.01	0.74
○Thread 869052			0.68	0.98	69.16	1.82	0.01	0.74
○Thread 869053			0.68	0.98	68.64	1.82	0.01	0.74
○Thread 869054			0.69	0.98	69.64	1.82	0.01	0.74
○Thread 869055			0.69	0.98	70.13	1.82	0.01	0.74
○Thread 869056			0.68	0.98	69.10	1.82	0.01	0.74
○Thread 869057			0.70	0.98	71.11	1.82	0.01	0.74
○Thread 869058			0.69	0.98	70.08	1.82	0.01	0.73
○Thread 869059			0.69	0.98	69.56	1.82	0.01	0.73
○Thread 869060			0.69	0.99	69.54	1.82	0.01	0.74
○Thread 869061			0.70	0.99	71.05	1.82	0.01	0.73
○Thread 869062			0.69	0.99	69.51	1.82	0.01	0.74
○Thread 869063			0.69	0.99	69.50	1.82	0.01	0.74
○Thread 869064			0.68	0.99	68.47	1.82	0.01	0.73
○Thread 869065			0.67	0.99	67.44	1.82	0.00	0.72
○Thread 869066			0.69	0.99	70.47	1.82	0.00	0.73
○Thread 869067			0.69	0.99	70.45	1.82	0.00	0.74
○Thread 869068			0.69	0.99	69.93	1.82	0.00	0.74
○Thread 869069			0.69	0.99	69.91	1.82	0.00	0.74
○Thread 869070			0.69	0.99	69.90	1.82	0.00	0.74
○Thread 869071			0.69	0.99	69.88	1.82	0.00	0.73
○Thread 869072			0.71	0.99	71.89	1.82	0.00	0.74
○Thread 869073			0.70	0.99	71.37	1.82	0.00	0.72
○Thread 869074			0.67	0.99	67.81	1.82	0.00	0.73
○Thread 869075			0.71	0.99	71.84	1.82	0.00	0.73
○Thread 869076			0.69	0.99	69.80	1.82	0.00	0.74

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	1.22
►Process 869096+		48	1.22
○Thread 869096			1.22	2.75	44.48	0.00	0.00	1.80
○Thread 869195			0.63	0.92	68.18	1.82	0.02	0.67
○Thread 869196			0.64	0.92	70.35	1.82	0.02	0.67
○Thread 869197			0.64	0.92	69.79	1.82	0.02	0.67
○Thread 869198			0.64	0.92	69.77	1.82	0.02	0.67
○Thread 869199			0.62	0.92	67.58	1.82	0.02	0.66
○Thread 869200			0.60	0.92	64.83	1.82	0.01	0.67
○Thread 869201			0.62	0.92	67.54	1.82	0.01	0.67
○Thread 869202			0.63	0.92	69.16	1.82	0.01	0.67
○Thread 869203			0.63	0.92	68.05	1.82	0.01	0.91
○Thread 869204			0.60	0.92	65.32	1.82	0.01	0.67
○Thread 869205			0.59	0.92	64.21	1.82	0.01	0.67
○Thread 869206			0.62	0.92	67.46	1.82	0.01	0.66
○Thread 869207			0.63	0.92	67.99	1.82	0.01	0.66
○Thread 869208			0.61	0.92	66.34	1.82	0.01	0.67
○Thread 869209			0.63	0.92	68.50	1.82	0.01	0.67
○Thread 869210			0.64	0.92	70.11	1.82	0.01	0.67
○Thread 869211			0.62	0.92	67.38	1.82	0.01	0.67
○Thread 869212			0.60	0.92	65.19	1.82	0.01	0.65
○Thread 869213			0.63	0.92	68.43	1.82	0.01	0.66
○Thread 869214			0.62	0.92	67.33	1.82	0.01	0.66
○Thread 869215			0.62	0.92	66.77	1.82	0.01	0.67
○Thread 869216			0.61	0.92	66.21	1.82	0.01	0.66
○Thread 869217			0.64	0.92	69.99	1.82	0.01	0.67
○Thread 869218			0.64	0.92	69.43	1.82	0.01	0.67
○Thread 869219			0.62	0.92	67.25	1.82	0.01	0.66
○Thread 869220			0.60	0.92	64.52	1.82	0.01	0.66
○Thread 869221			0.63	0.92	68.85	1.82	0.01	0.67
○Thread 869222			0.62	0.92	66.66	1.82	0.01	0.67
○Thread 869223			0.63	0.92	67.73	1.82	0.01	0.66
○Thread 869224			0.58	0.92	62.84	1.82	0.01	0.67
○Thread 869225			0.63	0.92	68.24	1.82	0.01	0.68
○Thread 869226			0.63	0.92	68.23	1.82	0.01	0.67
○Thread 869227			0.61	0.92	65.50	1.83	0.01	0.67
○Thread 869228			0.63	0.92	68.19	1.83	0.01	0.67
○Thread 869229			0.63	0.92	68.18	1.83	0.00	0.67
○Thread 869230			0.64	0.92	69.79	1.83	0.00	0.67
○Thread 869231			0.63	0.92	68.15	1.83	0.00	0.67
○Thread 869232			0.63	0.92	67.59	1.83	0.00	0.66
○Thread 869233			0.63	0.92	67.57	1.83	0.00	0.67
○Thread 869234			0.62	0.93	67.02	1.83	0.00	0.66
○Thread 869235			0.64	0.93	69.70	1.83	0.00	0.66
○Thread 869236			0.63	0.93	68.07	1.83	0.00	0.67
○Thread 869237			0.63	0.93	68.05	1.83	0.00	0.67
○Thread 869238			0.62	0.93	66.96	1.83	0.00	0.67
○Thread 869239			0.62	0.93	66.94	1.83	0.00	0.66
○Thread 869240			0.62	0.93	66.92	1.83	0.00	0.66
○Thread 869241			0.61	0.93	65.29	1.83	0.00	0.67

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	1.18
►Process 869261+		56	1.18
○Thread 869261			1.18	2.73	43.37	0.00	0.00	1.78
○Thread 869360			0.57	0.87	65.30	1.84	0.02	0.63
○Thread 869361			0.60	0.87	68.15	1.84	0.02	0.62
○Thread 869362			0.57	0.87	65.84	1.84	0.02	0.63
○Thread 869363			0.59	0.87	67.54	1.84	0.02	0.62
○Thread 869364			0.57	0.87	65.24	1.84	0.02	0.63
○Thread 869365			0.57	0.87	65.22	1.84	0.02	0.62
○Thread 869366			0.56	0.87	63.49	1.84	0.02	0.62
○Thread 869367			0.57	0.87	65.76	1.84	0.02	0.63
○Thread 869368			0.56	0.88	63.97	1.84	0.02	0.63
○Thread 869369			0.59	0.88	67.38	1.84	0.02	0.60
○Thread 869370			0.56	0.88	64.51	1.84	0.02	0.62
○Thread 869371			0.57	0.88	65.07	1.84	0.02	0.63
○Thread 869372			0.58	0.88	66.76	1.84	0.02	0.62
○Thread 869373			0.58	0.88	66.17	1.84	0.01	0.62
○Thread 869374			0.59	0.88	67.30	1.84	0.01	0.63
○Thread 869375			0.58	0.88	66.71	1.84	0.01	0.63
○Thread 869376			0.57	0.88	64.98	1.84	0.01	0.63
○Thread 869377			0.58	0.88	66.11	1.84	0.01	0.63
○Thread 869378			0.57	0.88	64.95	1.84	0.01	0.62
○Thread 869379			0.57	0.88	64.93	1.84	0.01	0.62
○Thread 869380			0.58	0.88	66.63	1.84	0.01	0.62
○Thread 869381			0.58	0.88	66.61	1.84	0.01	0.62
○Thread 869382			0.57	0.88	64.89	1.84	0.01	0.62
○Thread 869383			0.55	0.88	62.02	1.84	0.01	0.61
○Thread 869384			0.56	0.88	64.29	1.84	0.01	0.63
○Thread 869385			0.57	0.88	65.41	1.84	0.01	0.63
○Thread 869386			0.56	0.88	63.69	1.84	0.01	0.63
○Thread 869387			0.56	0.88	63.67	1.84	0.01	0.61
○Thread 869388			0.60	0.88	68.20	1.84	0.01	0.63
○Thread 869389			0.57	0.88	64.78	1.84	0.01	0.63
○Thread 869390			0.56	0.88	63.06	1.84	0.01	0.62
○Thread 869391			0.57	0.88	64.75	1.84	0.01	0.63
○Thread 869392			0.57	0.88	65.30	1.84	0.01	0.63
○Thread 869393			0.56	0.88	63.58	1.84	0.01	0.60
○Thread 869394			0.57	0.88	65.27	1.84	0.01	0.63
○Thread 869395			0.58	0.88	66.39	1.84	0.01	0.63
○Thread 869396			0.56	0.88	64.10	1.84	0.01	0.63
○Thread 869397			0.60	0.88	67.49	1.84	0.01	0.63
○Thread 869398			0.57	0.88	65.20	1.84	0.01	0.62
○Thread 869399			0.57	0.88	64.62	1.84	0.01	0.62
○Thread 869400			0.55	0.88	61.77	1.84	0.01	0.63
○Thread 869401			0.60	0.88	67.42	1.84	0.01	0.63
○Thread 869402			0.54	0.88	61.17	1.84	0.00	0.61
○Thread 869403			0.57	0.88	64.55	1.84	0.00	0.62
○Thread 869404			0.56	0.88	63.97	1.84	0.00	0.63
○Thread 869405			0.57	0.88	64.52	1.85	0.00	0.62
○Thread 869406			0.56	0.88	63.94	1.85	0.00	0.61
○Thread 869407			0.58	0.88	65.62	1.85	0.00	0.62
○Thread 869408			0.56	0.88	62.78	1.85	0.00	0.63
○Thread 869409			0.57	0.88	65.02	1.85	0.00	0.63
○Thread 869410			0.59	0.88	66.70	1.85	0.00	0.63
○Thread 869411			0.56	0.88	63.30	1.85	0.00	0.63
○Thread 869412			0.56	0.88	63.28	1.85	0.00	0.63
○Thread 869413			0.57	0.89	64.39	1.85	0.00	0.63
○Thread 869414			0.56	0.89	63.25	1.85	0.00	0.62

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	1.08
►Process 869434+		64	1.08
○Thread 869434			1.08	2.68	40.48	0.00	0.00	1.80
○Thread 869533			0.52	0.84	62.73	1.82	0.02	0.59
○Thread 869534			0.54	0.84	64.50	1.82	0.02	0.59
○Thread 869535			0.54	0.84	64.49	1.82	0.02	0.59
○Thread 869536			0.53	0.84	63.28	1.82	0.02	0.59
○Thread 869537			0.54	0.84	63.86	1.82	0.02	0.59
○Thread 869538			0.49	0.84	58.47	1.82	0.02	0.59
○Thread 869539			0.53	0.84	63.23	1.82	0.02	0.58
○Thread 869540			0.55	0.84	65.00	1.82	0.02	0.59
○Thread 869541			0.52	0.84	62.00	1.82	0.02	0.59
○Thread 869542			0.50	0.84	59.60	1.82	0.02	0.57
○Thread 869543			0.52	0.84	61.97	1.82	0.02	0.59
○Thread 869544			0.52	0.84	62.55	1.82	0.02	0.59
○Thread 869545			0.51	0.84	61.34	1.82	0.02	0.58
○Thread 869546			0.54	0.84	63.71	1.82	0.02	0.58
○Thread 869547			0.51	0.84	61.31	1.82	0.02	0.57
○Thread 869548			0.53	0.84	63.07	1.82	0.02	0.59
○Thread 869549			0.50	0.84	60.08	1.82	0.02	0.58
○Thread 869550			0.51	0.84	60.66	1.82	0.02	0.58
○Thread 869551			0.56	0.84	66.59	1.82	0.02	0.59
○Thread 869552			0.52	0.84	62.41	1.82	0.02	0.59
○Thread 869553			0.52	0.84	62.39	1.82	0.02	0.59
○Thread 869554			0.52	0.84	62.38	1.82	0.02	0.59
○Thread 869555			0.53	0.84	62.96	1.82	0.01	0.59
○Thread 869556			0.51	0.84	60.57	1.82	0.01	0.59
○Thread 869557			0.51	0.84	61.15	1.82	0.01	0.58
○Thread 869558			0.51	0.84	60.54	1.82	0.01	0.59
○Thread 869559			0.51	0.84	60.52	1.82	0.01	0.59
○Thread 869560			0.52	0.84	62.28	1.82	0.01	0.58
○Thread 869561			0.54	0.84	64.05	1.82	0.01	0.59
○Thread 869562			0.56	0.84	66.40	1.83	0.01	0.59
○Thread 869563			0.55	0.84	64.61	1.83	0.01	0.59
○Thread 869564			0.55	0.84	65.19	1.83	0.01	0.59
○Thread 869565			0.54	0.84	63.98	1.83	0.01	0.59
○Thread 869566			0.52	0.84	61.60	1.83	0.01	0.57
○Thread 869567			0.54	0.84	63.36	1.83	0.01	0.59
○Thread 869568			0.52	0.84	61.57	1.83	0.01	0.59
○Thread 869569			0.55	0.84	64.51	1.83	0.01	0.59
○Thread 869570			0.52	0.85	61.54	1.83	0.01	0.59
○Thread 869571			0.54	0.85	63.29	1.83	0.01	0.59
○Thread 869572			0.53	0.85	62.68	1.83	0.01	0.59
○Thread 869573			0.51	0.85	60.90	1.83	0.01	0.59
○Thread 869574			0.55	0.85	65.02	1.83	0.01	0.59
○Thread 869575			0.55	0.85	65.00	1.83	0.01	0.59
○Thread 869576			0.50	0.85	59.67	1.83	0.01	0.58
○Thread 869577			0.51	0.85	60.25	1.83	0.01	0.58
○Thread 869578			0.55	0.85	64.95	1.83	0.01	0.58
○Thread 869579			0.52	0.85	61.99	1.83	0.01	0.59
○Thread 869580			0.53	0.85	62.56	1.83	0.01	0.59
○Thread 869581			0.52	0.85	61.95	1.83	0.01	0.59
○Thread 869582			0.54	0.85	63.71	1.83	0.01	0.59
○Thread 869583			0.53	0.85	62.51	1.83	0.00	0.59
○Thread 869584			0.53	0.85	62.49	1.83	0.00	0.58
○Thread 869585			0.54	0.85	63.65	1.83	0.00	0.59
○Thread 869586			0.52	0.85	61.87	1.83	0.00	0.59
○Thread 869587			0.53	0.85	62.44	1.83	0.00	0.59
○Thread 869588			0.55	0.85	64.78	1.83	0.00	0.58
○Thread 869589			0.51	0.85	60.64	1.83	0.00	0.59
○Thread 869590			0.51	0.85	60.04	1.83	0.00	0.59
○Thread 869591			0.55	0.85	64.14	1.83	0.00	0.59
○Thread 869592			0.53	0.85	62.36	1.83	0.00	0.58
○Thread 869593			0.52	0.85	61.76	1.83	0.00	0.59
○Thread 869594			0.55	0.85	64.68	1.83	0.00	0.59
○Thread 869595			0.54	0.85	63.49	1.83	0.00	0.59

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	1.12
►Process 869615+		72	1.12
○Thread 869615			1.12	2.65	42.44	0.00	0.00	1.79
○Thread 869714			0.53	0.81	65.30	1.81	0.03	0.56
○Thread 869715			0.52	0.81	64.66	1.81	0.03	0.57
○Thread 869716			0.51	0.81	63.36	1.81	0.02	0.57
○Thread 869717			0.51	0.81	62.73	1.81	0.02	0.57
○Thread 869718			0.50	0.81	60.87	1.81	0.02	0.56
○Thread 869719			0.51	0.81	62.69	1.81	0.02	0.56
○Thread 869720			0.50	0.81	61.45	1.81	0.02	0.57
○Thread 869721			0.51	0.81	63.27	1.81	0.02	0.57
○Thread 869722			0.50	0.81	60.80	1.81	0.02	0.56
○Thread 869723			0.51	0.81	62.62	1.81	0.02	0.56
○Thread 869724			0.52	0.81	63.83	1.81	0.02	0.56
○Thread 869725			0.53	0.81	65.04	1.81	0.02	0.57
○Thread 869726			0.52	0.82	63.80	1.81	0.02	0.57
○Thread 869727			0.51	0.82	62.55	1.81	0.02	0.57
○Thread 869728			0.52	0.82	64.38	1.81	0.02	0.56
○Thread 869729			0.50	0.82	61.29	1.81	0.02	0.56
○Thread 869730			0.51	0.82	63.11	1.81	0.02	0.57
○Thread 869731			0.51	0.82	63.10	1.81	0.02	0.57
○Thread 869732			0.51	0.82	62.46	1.82	0.02	0.56
○Thread 869733			0.50	0.82	61.83	1.82	0.02	0.55
○Thread 869734			0.51	0.82	63.04	1.82	0.02	0.57
○Thread 869735			0.48	0.82	58.74	1.82	0.02	0.57
○Thread 869736			0.52	0.82	63.62	1.82	0.02	0.57
○Thread 869737			0.51	0.82	62.38	1.82	0.02	0.57
○Thread 869738			0.50	0.82	60.53	1.82	0.02	0.57
○Thread 869739			0.52	0.82	64.18	1.82	0.02	0.57
○Thread 869740			0.51	0.82	62.94	1.82	0.02	0.56
○Thread 869741			0.50	0.82	61.70	1.82	0.02	0.55
○Thread 869742			0.49	0.82	59.85	1.82	0.02	0.57
○Thread 869743			0.49	0.82	59.23	1.82	0.02	0.55
○Thread 869744			0.51	0.82	62.88	1.82	0.01	0.56
○Thread 869745			0.49	0.82	59.81	1.82	0.01	0.57
○Thread 869746			0.51	0.82	62.23	1.82	0.01	0.57
○Thread 869747			0.51	0.82	62.21	1.82	0.01	0.52
○Thread 869748			0.47	0.82	57.93	1.82	0.01	0.57
○Thread 869749			0.52	0.82	64.01	1.82	0.01	0.57
○Thread 869750			0.51	0.82	62.16	1.82	0.01	0.55
○Thread 869751			0.52	0.82	63.97	1.82	0.01	0.56
○Thread 869752			0.50	0.82	60.30	1.82	0.01	0.57
○Thread 869753			0.50	0.82	60.29	1.82	0.01	0.57
○Thread 869754			0.51	0.82	62.10	1.82	0.01	0.56
○Thread 869755			0.51	0.82	62.69	1.82	0.01	0.56
○Thread 869756			0.53	0.82	64.50	1.82	0.01	0.57
○Thread 869757			0.48	0.82	58.40	1.82	0.01	0.56
○Thread 869758			0.49	0.82	58.99	1.82	0.01	0.57
○Thread 869759			0.50	0.82	60.19	1.82	0.01	0.57
○Thread 869760			0.50	0.82	60.17	1.82	0.01	0.57
○Thread 869761			0.55	0.82	66.23	1.82	0.01	0.57
○Thread 869762			0.49	0.82	58.93	1.82	0.01	0.57
○Thread 869763			0.50	0.82	60.73	1.82	0.01	0.54
○Thread 869764			0.50	0.82	60.11	1.82	0.01	0.57
○Thread 869765			0.49	0.82	59.49	1.82	0.01	0.57
○Thread 869766			0.51	0.82	61.90	1.82	0.01	0.56
○Thread 869767			0.49	0.82	59.45	1.82	0.01	0.56
○Thread 869768			0.51	0.82	62.47	1.82	0.01	0.56
○Thread 869769			0.50	0.82	61.24	1.82	0.01	0.57
○Thread 869770			0.49	0.82	58.80	1.82	0.01	0.57
○Thread 869771			0.49	0.83	58.78	1.82	0.01	0.54
○Thread 869772			0.49	0.83	58.77	1.82	0.01	0.56
○Thread 869773			0.54	0.83	64.81	1.82	0.00	0.57
○Thread 869774			0.47	0.83	57.53	1.82	0.00	0.54
○Thread 869775			0.46	0.83	55.09	1.82	0.00	0.52
○Thread 869776			0.52	0.83	62.94	1.82	0.00	0.57
○Thread 869777			0.50	0.83	61.11	1.82	0.00	0.57
○Thread 869778			0.54	0.83	64.72	1.82	0.00	0.57
○Thread 869779			0.52	0.83	63.50	1.82	0.00	0.57
○Thread 869780			0.54	0.83	64.69	1.82	0.00	0.57
○Thread 869781			0.53	0.83	64.06	1.82	0.00	0.57
○Thread 869782			0.49	0.83	59.21	1.82	0.00	0.56
○Thread 869783			0.47	0.83	56.78	1.82	0.00	0.56
○Thread 869784			0.49	0.83	58.58	1.82	0.00	0.53

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	1.05
►Process 869852+		80	1.05
○Thread 869852			1.05	2.64	39.91	0.00	0.00	1.77
○Thread 869951			0.46	0.80	57.17	1.82	0.03	0.54
○Thread 869952			0.50	0.80	62.80	1.82	0.03	0.54
○Thread 869953			0.49	0.80	60.90	1.82	0.03	0.54
○Thread 869954			0.46	0.80	57.12	1.82	0.03	0.55
○Thread 869955			0.48	0.80	60.24	1.82	0.03	0.55
○Thread 869956			0.46	0.80	57.62	1.82	0.03	0.55
○Thread 869957			0.47	0.80	59.48	1.82	0.03	0.54
○Thread 869958			0.47	0.80	58.84	1.82	0.03	0.55
○Thread 869959			0.48	0.80	60.07	1.82	0.03	0.54
○Thread 869960			0.46	0.80	56.93	1.82	0.03	0.55
○Thread 869961			0.48	0.80	60.04	1.82	0.02	0.55
○Thread 869962			0.47	0.80	59.39	1.82	0.02	0.55
○Thread 869963			0.47	0.80	58.13	1.82	0.02	0.55
○Thread 869964			0.46	0.80	56.86	1.82	0.02	0.53
○Thread 869965			0.49	0.80	60.59	1.82	0.02	0.55
○Thread 869966			0.47	0.80	58.70	1.82	0.02	0.55
○Thread 869967			0.47	0.80	58.68	1.82	0.02	0.53
○Thread 869968			0.48	0.80	59.92	1.82	0.02	0.53
○Thread 869969			0.49	0.80	60.52	1.82	0.02	0.50
○Thread 869970			0.47	0.80	58.01	1.82	0.02	0.55
○Thread 869971			0.46	0.80	57.37	1.82	0.02	0.55
○Thread 869972			0.49	0.80	60.47	1.82	0.02	0.55
○Thread 869973			0.47	0.80	57.96	1.82	0.02	0.54
○Thread 869974			0.47	0.80	59.19	1.82	0.02	0.55
○Thread 869975			0.46	0.80	57.30	1.82	0.02	0.55
○Thread 869976			0.47	0.80	57.91	1.82	0.02	0.55
○Thread 869977			0.47	0.80	59.14	1.82	0.02	0.55
○Thread 869978			0.47	0.80	58.50	1.82	0.02	0.54
○Thread 869979			0.46	0.80	56.61	1.82	0.02	0.54
○Thread 869980			0.47	0.80	57.84	1.82	0.02	0.47
○Thread 869981			0.47	0.80	57.83	1.82	0.02	0.53
○Thread 869982			0.48	0.80	59.67	1.82	0.02	0.55
○Thread 869983			0.49	0.80	60.90	1.82	0.02	0.54
○Thread 869984			0.47	0.80	58.40	1.82	0.02	0.54
○Thread 869985			0.46	0.81	56.52	1.82	0.02	0.48
○Thread 869986			0.47	0.81	58.99	1.82	0.02	0.48
○Thread 869987			0.47	0.81	58.35	1.82	0.02	0.55
○Thread 869988			0.47	0.81	58.95	1.82	0.02	0.55
○Thread 869989			0.46	0.81	57.08	1.82	0.02	0.55
○Thread 869990			0.49	0.81	60.78	1.82	0.01	0.54
○Thread 869991			0.49	0.81	60.76	1.82	0.01	0.55
○Thread 869992			0.46	0.81	57.03	1.82	0.01	0.55
○Thread 869993			0.46	0.81	57.01	1.82	0.01	0.55
○Thread 869994			0.46	0.81	57.00	1.82	0.01	0.55
○Thread 869995			0.46	0.81	56.98	1.82	0.01	0.55
○Thread 869996			0.49	0.81	60.06	1.82	0.01	0.55
○Thread 869997			0.46	0.81	56.95	1.82	0.01	0.54
○Thread 869998			0.50	0.81	61.27	1.82	0.01	0.54
○Thread 869999			0.47	0.81	57.54	1.82	0.01	0.54
○Thread 870000			0.49	0.81	60.00	1.82	0.01	0.55
○Thread 870001			0.49	0.81	59.98	1.82	0.01	0.55
○Thread 870002			0.48	0.81	59.35	1.82	0.01	0.55
○Thread 870003			0.46	0.81	56.24	1.82	0.01	0.50
○Thread 870004			0.45	0.81	55.61	1.82	0.01	0.54
○Thread 870005			0.45	0.81	55.59	1.82	0.01	0.53
○Thread 870006			0.47	0.81	57.43	1.82	0.01	0.54
○Thread 870007			0.47	0.81	57.41	1.82	0.01	0.54
○Thread 870008			0.49	0.81	59.87	1.82	0.01	0.55
○Thread 870009			0.47	0.81	58.62	1.83	0.01	0.53
○Thread 870010			0.50	0.81	61.07	1.83	0.01	0.55
○Thread 870011			0.46	0.81	56.73	1.83	0.01	0.53
○Thread 870012			0.47	0.81	57.95	1.83	0.01	0.55
○Thread 870013			0.46	0.81	56.70	1.83	0.01	0.55
○Thread 870014			0.48	0.81	59.15	1.83	0.01	0.55
○Thread 870015			0.44	0.81	54.82	1.83	0.01	0.55
○Thread 870016			0.46	0.81	56.65	1.83	0.01	0.54
○Thread 870017			0.47	0.81	58.49	1.83	0.01	0.55
○Thread 870018			0.46	0.81	56.62	1.83	0.00	0.55
○Thread 870019			0.48	0.81	59.07	1.83	0.00	0.55
○Thread 870020			0.49	0.81	60.28	1.83	0.00	0.55
○Thread 870021			0.49	0.81	59.65	1.83	0.00	0.54
○Thread 870022			0.45	0.81	55.33	1.83	0.00	0.55
○Thread 870023			0.47	0.81	58.39	1.83	0.00	0.55
○Thread 870024			0.49	0.81	60.21	1.83	0.00	0.55
○Thread 870025			0.46	0.81	56.51	1.83	0.00	0.55
○Thread 870026			0.46	0.81	56.49	1.83	0.00	0.53
○Thread 870027			0.48	0.81	58.93	1.83	0.00	0.54
○Thread 870028			0.49	0.81	60.14	1.83	0.00	0.55
○Thread 870029			0.47	0.81	58.29	1.83	0.00	0.52

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	1.11
►Process 870049+		88	1.11
○Thread 870049			1.11	2.65	41.95	0.00	0.00	1.78
○Thread 870148			0.47	0.78	60.93	1.83	0.03	0.51
○Thread 870149			0.47	0.78	59.63	1.83	0.03	0.53
○Thread 870150			0.48	0.78	61.53	1.83	0.03	0.53
○Thread 870151			0.47	0.78	60.23	1.83	0.03	0.53
○Thread 870152			0.47	0.78	60.86	1.83	0.03	0.53
○Thread 870153			0.47	0.78	60.78	1.83	0.03	0.53
○Thread 870154			0.47	0.78	60.76	1.83	0.03	0.53
○Thread 870155			0.49	0.78	62.66	1.83	0.03	0.53
○Thread 870156			0.46	0.78	58.17	1.83	0.03	0.51
○Thread 870157			0.46	0.78	58.79	1.83	0.03	0.53
○Thread 870158			0.47	0.78	60.69	1.84	0.03	0.53
○Thread 870159			0.50	0.78	63.87	1.84	0.03	0.53
○Thread 870160			0.48	0.78	61.30	1.84	0.03	0.53
○Thread 870161			0.47	0.78	60.64	1.84	0.03	0.53
○Thread 870162			0.49	0.78	61.90	1.84	0.03	0.53
○Thread 870163			0.49	0.78	62.52	1.84	0.03	0.53
○Thread 870164			0.44	0.78	56.12	1.84	0.03	0.53
○Thread 870165			0.48	0.78	61.21	1.84	0.03	0.53
○Thread 870166			0.45	0.78	57.36	1.84	0.03	0.53
○Thread 870167			0.48	0.78	61.17	1.84	0.02	0.53
○Thread 870168			0.46	0.78	58.60	1.84	0.02	0.53
○Thread 870169			0.47	0.79	60.49	1.84	0.02	0.52
○Thread 870170			0.49	0.79	61.75	1.84	0.02	0.53
○Thread 870171			0.46	0.79	58.55	1.84	0.02	0.53
○Thread 870172			0.47	0.79	59.81	1.84	0.02	0.52
○Thread 870173			0.44	0.79	55.97	1.84	0.02	0.52
○Thread 870174			0.46	0.79	57.86	1.84	0.02	0.53
○Thread 870175			0.47	0.79	59.76	1.84	0.02	0.53
○Thread 870176			0.46	0.79	58.47	1.84	0.02	0.53
○Thread 870177			0.47	0.79	59.09	1.84	0.02	0.53
○Thread 870178			0.47	0.79	59.71	1.84	0.02	0.53
○Thread 870179			0.48	0.79	60.96	1.84	0.02	0.52
○Thread 870180			0.44	0.79	56.50	1.84	0.02	0.51
○Thread 870181			0.47	0.79	60.29	1.84	0.02	0.53
○Thread 870182			0.46	0.79	58.37	1.84	0.02	0.53
○Thread 870183			0.45	0.79	57.08	1.84	0.02	0.53
○Thread 870184			0.47	0.79	59.60	1.84	0.02	0.52
○Thread 870185			0.44	0.79	55.78	1.84	0.02	0.53
○Thread 870186			0.44	0.79	55.13	1.84	0.02	0.53
○Thread 870187			0.46	0.79	58.29	1.84	0.02	0.53
○Thread 870188			0.46	0.79	58.27	1.84	0.02	0.53
○Thread 870189			0.47	0.79	60.15	1.84	0.02	0.53
○Thread 870190			0.46	0.79	58.23	1.84	0.02	0.53
○Thread 870191			0.47	0.79	58.85	1.84	0.02	0.52
○Thread 870192			0.44	0.79	56.30	1.84	0.02	0.52
○Thread 870193			0.47	0.79	58.82	1.84	0.02	0.53
○Thread 870194			0.46	0.79	58.17	1.84	0.02	0.52
○Thread 870195			0.48	0.79	60.68	1.84	0.01	0.53
○Thread 870196			0.49	0.79	61.29	1.84	0.01	0.53
○Thread 870197			0.45	0.79	56.85	1.84	0.01	0.51
○Thread 870198			0.46	0.79	57.47	1.84	0.01	0.52
○Thread 870199			0.47	0.79	59.34	1.84	0.01	0.53
○Thread 870200			0.44	0.79	55.54	1.84	0.01	0.53
○Thread 870201			0.47	0.79	58.68	1.84	0.01	0.52
○Thread 870202			0.46	0.79	58.03	1.84	0.01	0.53
○Thread 870203			0.46	0.79	58.01	1.84	0.01	0.53
○Thread 870204			0.46	0.79	57.99	1.84	0.01	0.53
○Thread 870205			0.45	0.79	56.72	1.84	0.01	0.51
○Thread 870206			0.44	0.79	56.07	1.84	0.01	0.53
○Thread 870207			0.47	0.79	59.83	1.84	0.01	0.53
○Thread 870208			0.47	0.79	58.56	1.84	0.01	0.51
○Thread 870209			0.44	0.79	54.76	1.84	0.01	0.53
○Thread 870210			0.46	0.79	57.90	1.84	0.01	0.52
○Thread 870211			0.48	0.79	60.40	1.84	0.01	0.53
○Thread 870212			0.47	0.79	59.75	1.84	0.01	0.53
○Thread 870213			0.45	0.80	56.59	1.84	0.01	0.53
○Thread 870214			0.44	0.80	54.69	1.84	0.01	0.51
○Thread 870215			0.47	0.80	59.07	1.84	0.01	0.53
○Thread 870216			0.45	0.80	56.57	1.84	0.01	0.53
○Thread 870217			0.44	0.80	55.29	1.84	0.01	0.53
○Thread 870218			0.47	0.80	58.42	1.84	0.01	0.53
○Thread 870219			0.45	0.80	56.52	1.84	0.01	0.53
○Thread 870220			0.47	0.80	59.64	1.84	0.01	0.52
○Thread 870221			0.46	0.80	57.74	1.84	0.01	0.53
○Thread 870222			0.46	0.80	57.10	1.84	0.01	0.53
○Thread 870223			0.45	0.80	56.45	1.84	0.00	0.53
○Thread 870224			0.49	0.80	60.83	1.84	0.00	0.53
○Thread 870225			0.47	0.80	58.93	1.84	0.00	0.53
○Thread 870226			0.44	0.80	55.78	1.84	0.00	0.53
○Thread 870227			0.48	0.80	60.15	1.84	0.00	0.53
○Thread 870228			0.47	0.80	58.88	1.84	0.00	0.53
○Thread 870229			0.47	0.80	58.24	1.84	0.00	0.53
○Thread 870230			0.47	0.80	58.22	1.84	0.00	0.53
○Thread 870231			0.47	0.80	59.46	1.85	0.00	0.53
○Thread 870232			0.42	0.80	52.56	1.85	0.00	0.52
○Thread 870233			0.47	0.80	58.17	1.85	0.00	0.51
○Thread 870234			0.46	0.80	57.53	1.85	0.00	0.53

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	1.00
►Process 870254+		96	1.00
○Thread 870254			1.00	2.65	37.88	0.00	0.00	1.81
○Thread 870353			0.49	0.78	62.77	1.84	0.04	0.53
○Thread 870354			0.46	0.78	58.27	1.84	0.03	0.50
○Thread 870355			0.47	0.78	60.81	1.84	0.03	0.52
○Thread 870356			0.47	0.78	59.51	1.84	0.03	0.52
○Thread 870357			0.44	0.78	56.30	1.84	0.03	0.52
○Thread 870358			0.47	0.78	60.12	1.84	0.03	0.52
○Thread 870359			0.46	0.78	58.18	1.84	0.03	0.50
○Thread 870360			0.46	0.78	58.80	1.84	0.03	0.51
○Thread 870361			0.50	0.78	63.90	1.84	0.03	0.51
○Thread 870362			0.47	0.78	60.68	1.84	0.03	0.52
○Thread 870363			0.44	0.78	56.19	1.84	0.03	0.52
○Thread 870364			0.48	0.78	61.28	1.84	0.03	0.53
○Thread 870365			0.50	0.78	63.18	1.84	0.03	0.52
○Thread 870366			0.47	0.78	60.61	1.84	0.03	0.52
○Thread 870367			0.45	0.78	57.40	1.84	0.03	0.52
○Thread 870368			0.44	0.78	56.11	1.84	0.03	0.51
○Thread 870369			0.48	0.78	61.19	1.84	0.03	0.53
○Thread 870370			0.46	0.78	57.99	1.84	0.03	0.52
○Thread 870371			0.49	0.78	62.43	1.84	0.03	0.52
○Thread 870372			0.44	0.79	55.40	1.84	0.03	0.52
○Thread 870373			0.47	0.79	59.84	1.84	0.03	0.52
○Thread 870374			0.45	0.79	57.28	1.84	0.03	0.51
○Thread 870375			0.44	0.79	55.99	1.84	0.03	0.52
○Thread 870376			0.47	0.79	60.42	1.84	0.03	0.52
○Thread 870377			0.46	0.79	58.50	1.84	0.03	0.52
○Thread 870378			0.47	0.79	59.12	1.84	0.03	0.52
○Thread 870379			0.46	0.79	57.83	1.84	0.03	0.50
○Thread 870380			0.49	0.79	61.63	1.84	0.03	0.52
○Thread 870381			0.44	0.79	55.90	1.84	0.02	0.52
○Thread 870382			0.46	0.79	57.79	1.84	0.02	0.48
○Thread 870383			0.43	0.79	53.96	1.84	0.02	0.51
○Thread 870384			0.48	0.79	60.93	1.84	0.02	0.52
○Thread 870385			0.45	0.79	57.10	1.84	0.02	0.50
○Thread 870386			0.47	0.79	58.99	1.84	0.02	0.52
○Thread 870387			0.47	0.79	58.97	1.84	0.02	0.52
○Thread 870388			0.44	0.79	55.15	1.84	0.02	0.52
○Thread 870389			0.45	0.79	57.03	1.84	0.02	0.52
○Thread 870390			0.47	0.79	59.55	1.84	0.02	0.52
○Thread 870391			0.47	0.79	59.53	1.84	0.02	0.53
○Thread 870392			0.47	0.79	59.52	1.84	0.02	0.53
○Thread 870393			0.46	0.79	57.60	1.84	0.02	0.50
○Thread 870394			0.44	0.79	56.32	1.84	0.02	0.52
○Thread 870395			0.47	0.79	58.83	1.84	0.02	0.52
○Thread 870396			0.47	0.79	58.82	1.84	0.02	0.51
○Thread 870397			0.46	0.79	58.17	1.84	0.02	0.52
○Thread 870398			0.45	0.79	56.88	1.84	0.02	0.52
○Thread 870399			0.44	0.79	54.97	1.84	0.02	0.52
○Thread 870400			0.46	0.79	57.48	1.84	0.02	0.52
○Thread 870401			0.47	0.79	58.73	1.84	0.02	0.52
○Thread 870402			0.46	0.79	57.45	1.84	0.02	0.52
○Thread 870403			0.45	0.79	56.80	1.84	0.02	0.52
○Thread 870404			0.46	0.79	57.42	1.84	0.02	0.52
○Thread 870405			0.46	0.79	57.40	1.84	0.02	0.52
○Thread 870406			0.47	0.79	59.27	1.84	0.02	0.51
○Thread 870407			0.49	0.79	61.15	1.84	0.02	0.53
○Thread 870408			0.46	0.79	57.35	1.84	0.02	0.51
○Thread 870409			0.48	0.79	60.48	1.84	0.01	0.52
○Thread 870410			0.46	0.79	57.32	1.85	0.01	0.52
○Thread 870411			0.46	0.79	57.93	1.85	0.01	0.52
○Thread 870412			0.47	0.79	59.17	1.85	0.01	0.52
○Thread 870413			0.46	0.79	57.89	1.85	0.01	0.51
○Thread 870414			0.45	0.79	56.62	1.85	0.01	0.52
○Thread 870415			0.45	0.79	56.60	1.85	0.01	0.52
○Thread 870416			0.46	0.80	57.22	1.85	0.01	0.52
○Thread 870417			0.45	0.80	56.57	1.85	0.01	0.51
○Thread 870418			0.46	0.80	57.19	1.85	0.01	0.52
○Thread 870419			0.44	0.80	55.91	1.85	0.01	0.52
○Thread 870420			0.42	0.80	52.76	1.85	0.01	0.49
○Thread 870421			0.46	0.80	57.76	1.85	0.01	0.51
○Thread 870422			0.45	0.80	56.49	1.85	0.01	0.52
○Thread 870423			0.45	0.80	56.48	1.85	0.01	0.52
○Thread 870424			0.44	0.80	55.83	1.85	0.01	0.52
○Thread 870425			0.46	0.80	57.07	1.85	0.01	0.52
○Thread 870426			0.45	0.80	56.42	1.85	0.01	0.52
○Thread 870427			0.47	0.80	58.29	1.85	0.01	0.52
○Thread 870428			0.46	0.80	57.64	1.85	0.01	0.52
○Thread 870429			0.46	0.80	57.63	1.85	0.01	0.52
○Thread 870430			0.46	0.80	56.98	1.85	0.01	0.52
○Thread 870431			0.47	0.80	58.84	1.85	0.01	0.52
○Thread 870432			0.46	0.80	56.95	1.85	0.01	0.53
○Thread 870433			0.45	0.80	56.31	1.85	0.01	0.52
○Thread 870434			0.45	0.80	56.29	1.85	0.01	0.52
○Thread 870435			0.44	0.80	55.65	1.85	0.01	0.52
○Thread 870436			0.46	0.80	56.88	1.85	0.00	0.52
○Thread 870437			0.46	0.80	57.49	1.85	0.00	0.51
○Thread 870438			0.45	0.80	56.22	1.85	0.00	0.52
○Thread 870439			0.46	0.80	57.46	1.85	0.00	0.52
○Thread 870440			0.46	0.80	57.44	1.85	0.00	0.52
○Thread 870441			0.47	0.80	58.05	1.85	0.00	0.51
○Thread 870442			0.44	0.80	55.53	1.85	0.00	0.52
○Thread 870443			0.46	0.80	57.39	1.85	0.00	0.51
○Thread 870444			0.47	0.80	58.62	1.85	0.00	0.52
○Thread 870445			0.46	0.80	56.73	1.85	0.00	0.52
○Thread 870446			0.46	0.80	57.33	1.85	0.00	0.53
○Thread 870447			0.44	0.80	54.20	1.85	0.00	0.52

Report Configuration