OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	9.49
►Process 854937+		1	9.49
○Thread 854937			9.49	11.02	86.10	0.00	0.00	11.00

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	5.14
►Process 855055+		2	5.14
○Thread 855055			5.14	6.64	77.46	0.00	0.00	4.81
○Thread 855154			4.56	4.81	94.60	1.82	0.00	4.58

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	2.90
►Process 855174+		4	2.90
○Thread 855174			2.90	4.46	65.16	0.00	0.00	2.64
○Thread 855273			2.38	2.65	89.87	1.81	0.00	2.41
○Thread 855274			2.36	2.65	89.11	1.81	0.00	2.41
○Thread 855275			2.38	2.65	90.05	1.81	0.00	2.41

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	1.88
►Process 855298+		8	1.88
○Thread 855298			1.88	3.36	55.88	0.00	0.00	1.81
○Thread 855397			1.28	1.54	82.89	1.82	0.00	1.31
○Thread 855398			1.27	1.54	82.56	1.82	0.00	1.31
○Thread 855399			1.30	1.54	84.49	1.82	0.00	1.31
○Thread 855400			1.28	1.54	82.86	1.82	0.00	1.31
○Thread 855401			1.26	1.54	81.55	1.82	0.00	1.31
○Thread 855402			1.26	1.55	81.87	1.82	0.00	1.31
○Thread 855403			1.25	1.55	81.21	1.82	0.00	1.31

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	1.29
►Process 855472+		16	1.29
○Thread 855472			1.29	2.83	45.63	0.00	0.00	1.76
○Thread 855571			0.75	1.01	73.91	1.81	0.01	0.77
○Thread 855572			0.69	1.01	68.94	1.81	0.01	0.76
○Thread 855573			0.73	1.01	72.39	1.81	0.01	0.76
○Thread 855574			0.74	1.01	73.37	1.81	0.01	0.77
○Thread 855575			0.73	1.01	72.36	1.81	0.01	0.77
○Thread 855576			0.75	1.01	74.33	1.81	0.01	0.77
○Thread 855577			0.73	1.01	72.33	1.81	0.00	0.77
○Thread 855578			0.73	1.01	72.32	1.81	0.00	0.77
○Thread 855579			0.75	1.01	73.79	1.81	0.00	0.77
○Thread 855580			0.74	1.01	73.28	1.81	0.00	0.77
○Thread 855581			0.75	1.01	74.17	1.81	0.00	0.77
○Thread 855582			0.74	1.01	73.16	1.81	0.00	0.77
○Thread 855583			0.74	1.01	73.15	1.81	0.00	0.77
○Thread 855584			0.75	1.01	74.61	1.81	0.00	0.77
○Thread 855585			0.73	1.01	72.13	1.81	0.00	0.77

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	1.14
►Process 855605+		24	1.14
○Thread 855605			1.14	2.72	41.97	0.00	0.00	1.82
○Thread 855706			0.60	0.86	68.88	1.84	0.01	0.62
○Thread 855707			0.60	0.86	69.45	1.84	0.01	0.63
○Thread 855708			0.61	0.86	70.01	1.84	0.01	0.63
○Thread 855709			0.58	0.86	67.68	1.84	0.01	0.62
○Thread 855710			0.59	0.86	68.24	1.84	0.01	0.63
○Thread 855711			0.60	0.86	68.80	1.84	0.01	0.63
○Thread 855712			0.58	0.87	67.05	1.85	0.01	0.63
○Thread 855713			0.60	0.87	69.34	1.85	0.01	0.62
○Thread 855714			0.58	0.87	67.01	1.85	0.01	0.63
○Thread 855715			0.57	0.87	66.42	1.85	0.01	0.63
○Thread 855716			0.60	0.87	68.71	1.85	0.00	0.62
○Thread 855717			0.58	0.87	66.96	1.85	0.00	0.62
○Thread 855718			0.60	0.87	69.25	1.85	0.00	0.63
○Thread 855719			0.59	0.87	68.08	1.85	0.00	0.62
○Thread 855720			0.58	0.87	66.91	1.85	0.00	0.63
○Thread 855721			0.58	0.87	66.90	1.85	0.00	0.63
○Thread 855722			0.59	0.87	68.03	1.85	0.00	0.63
○Thread 855723			0.61	0.87	69.74	1.85	0.00	0.63
○Thread 855724			0.56	0.87	65.12	1.85	0.00	0.62
○Thread 855725			0.61	0.87	69.71	1.85	0.00	0.62
○Thread 855726			0.59	0.87	67.96	1.85	0.00	0.63
○Thread 855727			0.59	0.87	67.94	1.85	0.00	0.63
○Thread 855728			0.55	0.87	63.32	1.85	0.00	0.63

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	1.02
►Process 855748+		32	1.02
○Thread 855748			1.02	2.61	39.22	0.00	0.00	1.79
○Thread 855847			0.49	0.78	61.93	1.82	0.01	0.54
○Thread 855848			0.49	0.78	61.92	1.82	0.01	0.54
○Thread 855849			0.47	0.78	60.63	1.82	0.01	0.52
○Thread 855850			0.50	0.78	64.44	1.82	0.01	0.54
○Thread 855851			0.47	0.78	60.59	1.82	0.01	0.54
○Thread 855852			0.49	0.78	62.49	1.82	0.01	0.54
○Thread 855853			0.49	0.78	62.47	1.82	0.01	0.54
○Thread 855854			0.50	0.78	63.09	1.82	0.01	0.54
○Thread 855855			0.49	0.79	62.38	1.82	0.01	0.54
○Thread 855856			0.51	0.79	65.55	1.82	0.01	0.54
○Thread 855857			0.47	0.79	59.80	1.82	0.01	0.53
○Thread 855858			0.47	0.79	59.78	1.82	0.01	0.54
○Thread 855859			0.51	0.79	65.49	1.82	0.01	0.54
○Thread 855860			0.49	0.79	62.29	1.82	0.01	0.54
○Thread 855861			0.50	0.79	64.18	1.82	0.01	0.54
○Thread 855862			0.50	0.79	64.17	1.82	0.01	0.54
○Thread 855863			0.50	0.79	62.88	1.82	0.01	0.54
○Thread 855864			0.51	0.79	64.77	1.82	0.01	0.54
○Thread 855865			0.47	0.79	60.30	1.82	0.01	0.54
○Thread 855866			0.49	0.79	61.56	1.82	0.00	0.54
○Thread 855867			0.48	0.79	60.90	1.82	0.00	0.54
○Thread 855868			0.48	0.79	60.89	1.82	0.00	0.54
○Thread 855869			0.49	0.79	62.14	1.82	0.00	0.54
○Thread 855870			0.51	0.79	64.65	1.82	0.00	0.54
○Thread 855871			0.50	0.79	64.00	1.82	0.00	0.54
○Thread 855872			0.50	0.79	63.35	1.82	0.00	0.53
○Thread 855873			0.47	0.79	60.17	1.82	0.00	0.54
○Thread 855874			0.50	0.79	62.69	1.82	0.00	0.54
○Thread 855875			0.49	0.79	61.40	1.82	0.00	0.54
○Thread 855876			0.50	0.79	62.65	1.82	0.00	0.54
○Thread 855877			0.49	0.79	62.00	1.82	0.00	0.54

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	1.07
►Process 855897+		40	1.07
○Thread 855897			1.07	2.57	41.70	0.00	0.00	1.80
○Thread 855996			0.46	0.74	61.80	1.82	0.01	0.50
○Thread 855997			0.48	0.74	65.18	1.82	0.01	0.50
○Thread 855998			0.46	0.74	62.45	1.82	0.01	0.49
○Thread 855999			0.47	0.74	64.47	1.82	0.01	0.50
○Thread 856000			0.44	0.74	60.38	1.82	0.01	0.50
○Thread 856001			0.44	0.74	59.68	1.82	0.01	0.49
○Thread 856002			0.44	0.74	60.34	1.82	0.01	0.50
○Thread 856003			0.45	0.74	61.00	1.82	0.01	0.50
○Thread 856004			0.45	0.74	60.98	1.82	0.01	0.49
○Thread 856005			0.47	0.74	62.99	1.82	0.01	0.50
○Thread 856006			0.44	0.74	60.26	1.82	0.01	0.49
○Thread 856007			0.46	0.74	62.28	1.82	0.01	0.49
○Thread 856008			0.47	0.74	62.93	1.82	0.01	0.49
○Thread 856009			0.46	0.74	62.24	1.82	0.01	0.50
○Thread 856010			0.47	0.74	63.57	1.82	0.01	0.49
○Thread 856011			0.46	0.74	61.52	1.82	0.01	0.49
○Thread 856012			0.47	0.74	63.54	1.82	0.01	0.50
○Thread 856013			0.46	0.74	61.49	1.82	0.01	0.49
○Thread 856014			0.46	0.74	61.47	1.82	0.01	0.50
○Thread 856015			0.49	0.74	65.51	1.82	0.01	0.50
○Thread 856016			0.47	0.74	62.79	1.82	0.01	0.50
○Thread 856017			0.45	0.74	60.74	1.82	0.01	0.48
○Thread 856018			0.46	0.74	61.40	1.82	0.01	0.49
○Thread 856019			0.45	0.74	60.71	1.82	0.01	0.50
○Thread 856020			0.47	0.74	62.71	1.82	0.01	0.49
○Thread 856021			0.47	0.74	62.69	1.82	0.01	0.50
○Thread 856022			0.47	0.74	63.35	1.82	0.01	0.49
○Thread 856023			0.46	0.74	61.98	1.82	0.00	0.50
○Thread 856024			0.47	0.74	63.99	1.82	0.00	0.50
○Thread 856025			0.50	0.74	66.66	1.82	0.00	0.50
○Thread 856026			0.45	0.74	60.59	1.82	0.00	0.49
○Thread 856027			0.46	0.74	61.91	1.82	0.00	0.49
○Thread 856028			0.46	0.74	61.90	1.82	0.00	0.50
○Thread 856029			0.45	0.74	60.53	1.82	0.00	0.50
○Thread 856030			0.47	0.74	62.53	1.82	0.00	0.50
○Thread 856031			0.44	0.74	58.48	1.82	0.00	0.49
○Thread 856032			0.47	0.74	63.17	1.82	0.00	0.50
○Thread 856033			0.46	0.74	61.81	1.82	0.00	0.50
○Thread 856034			0.44	0.74	58.43	1.82	0.00	0.49

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	0.95
►Process 856054+		48	0.95
○Thread 856054			0.95	2.55	37.49	0.00	0.00	1.80
○Thread 856153			0.43	0.71	60.86	1.82	0.02	0.46
○Thread 856154			0.40	0.71	55.88	1.82	0.02	0.46
○Thread 856155			0.43	0.71	60.82	1.82	0.02	0.46
○Thread 856156			0.43	0.71	60.09	1.82	0.02	0.46
○Thread 856157			0.42	0.71	59.36	1.82	0.02	0.46
○Thread 856158			0.41	0.71	58.64	1.82	0.02	0.46
○Thread 856159			0.43	0.71	60.03	1.82	0.01	0.46
○Thread 856160			0.43	0.71	60.01	1.82	0.01	0.46
○Thread 856161			0.43	0.71	60.70	1.82	0.01	0.46
○Thread 856162			0.41	0.71	57.86	1.83	0.01	0.46
○Thread 856163			0.43	0.71	60.66	1.83	0.01	0.46
○Thread 856164			0.43	0.71	60.64	1.83	0.01	0.46
○Thread 856165			0.38	0.71	53.57	1.83	0.01	0.46
○Thread 856166			0.40	0.71	56.37	1.83	0.01	0.46
○Thread 856167			0.41	0.71	57.06	1.83	0.01	0.46
○Thread 856168			0.41	0.71	57.04	1.83	0.01	0.46
○Thread 856169			0.40	0.71	55.61	1.83	0.01	0.46
○Thread 856170			0.38	0.71	52.78	1.83	0.01	0.45
○Thread 856171			0.41	0.71	58.39	1.83	0.01	0.46
○Thread 856172			0.41	0.71	58.37	1.83	0.01	0.46
○Thread 856173			0.41	0.71	57.65	1.83	0.01	0.45
○Thread 856174			0.40	0.71	56.23	1.83	0.01	0.46
○Thread 856175			0.43	0.71	59.72	1.83	0.01	0.46
○Thread 856176			0.43	0.71	59.70	1.83	0.01	0.46
○Thread 856177			0.43	0.71	60.38	1.83	0.01	0.46
○Thread 856178			0.43	0.71	60.37	1.83	0.01	0.46
○Thread 856179			0.42	0.71	58.95	1.83	0.01	0.46
○Thread 856180			0.39	0.71	54.72	1.83	0.01	0.44
○Thread 856181			0.41	0.71	57.51	1.83	0.01	0.46
○Thread 856182			0.43	0.71	59.59	1.83	0.01	0.46
○Thread 856183			0.41	0.71	58.17	1.83	0.01	0.46
○Thread 856184			0.44	0.71	62.36	1.83	0.01	0.46
○Thread 856185			0.43	0.71	59.54	1.83	0.01	0.45
○Thread 856186			0.43	0.71	59.52	1.83	0.01	0.46
○Thread 856187			0.39	0.71	54.60	1.83	0.01	0.45
○Thread 856188			0.44	0.71	61.58	1.83	0.00	0.46
○Thread 856189			0.40	0.71	55.97	1.83	0.00	0.46
○Thread 856190			0.44	0.71	60.84	1.83	0.00	0.46
○Thread 856191			0.43	0.72	59.43	1.83	0.00	0.46
○Thread 856192			0.41	0.72	57.31	1.83	0.00	0.46
○Thread 856193			0.41	0.72	57.30	1.83	0.00	0.46
○Thread 856194			0.41	0.72	57.28	1.83	0.00	0.44
○Thread 856195			0.43	0.72	60.05	1.83	0.00	0.46
○Thread 856196			0.41	0.72	57.94	1.83	0.00	0.45
○Thread 856197			0.41	0.72	57.22	1.83	0.00	0.46
○Thread 856198			0.43	0.72	59.30	1.83	0.00	0.45
○Thread 856199			0.39	0.72	54.40	1.83	0.00	0.45

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	0.93
►Process 856219+		56	0.93
○Thread 856219			0.93	2.53	36.82	0.00	0.00	1.75
○Thread 856318			0.40	0.68	58.70	1.82	0.02	0.44
○Thread 856319			0.40	0.68	58.57	1.82	0.02	0.44
○Thread 856320			0.40	0.68	57.82	1.82	0.02	0.44
○Thread 856321			0.40	0.68	58.53	1.82	0.02	0.44
○Thread 856322			0.40	0.68	57.78	1.82	0.02	0.44
○Thread 856323			0.40	0.68	58.50	1.82	0.02	0.44
○Thread 856324			0.40	0.68	58.48	1.82	0.02	0.44
○Thread 856325			0.41	0.68	60.65	1.82	0.02	0.44
○Thread 856326			0.38	0.68	55.52	1.82	0.02	0.44
○Thread 856327			0.38	0.68	55.50	1.82	0.02	0.44
○Thread 856328			0.38	0.68	56.21	1.82	0.02	0.44
○Thread 856329			0.38	0.69	56.19	1.82	0.02	0.44
○Thread 856330			0.40	0.69	58.36	1.82	0.02	0.44
○Thread 856331			0.40	0.69	58.34	1.82	0.02	0.44
○Thread 856332			0.40	0.69	57.59	1.82	0.01	0.44
○Thread 856333			0.39	0.69	56.85	1.83	0.01	0.42
○Thread 856334			0.41	0.69	59.01	1.83	0.01	0.44
○Thread 856335			0.40	0.69	58.26	1.83	0.01	0.44
○Thread 856336			0.40	0.69	58.24	1.83	0.01	0.44
○Thread 856337			0.42	0.69	61.14	1.83	0.01	0.43
○Thread 856338			0.38	0.69	56.02	1.83	0.01	0.44
○Thread 856339			0.42	0.69	61.10	1.83	0.01	0.43
○Thread 856340			0.38	0.69	55.99	1.83	0.01	0.44
○Thread 856341			0.38	0.69	55.24	1.83	0.01	0.44
○Thread 856342			0.40	0.69	58.13	1.83	0.01	0.44
○Thread 856343			0.38	0.69	55.21	1.83	0.01	0.44
○Thread 856344			0.40	0.69	57.37	1.83	0.01	0.44
○Thread 856345			0.38	0.69	55.90	1.83	0.01	0.44
○Thread 856346			0.40	0.69	57.34	1.83	0.01	0.44
○Thread 856347			0.38	0.69	54.42	1.83	0.01	0.44
○Thread 856348			0.41	0.69	58.75	1.83	0.01	0.43
○Thread 856349			0.38	0.69	55.11	1.83	0.01	0.43
○Thread 856350			0.37	0.69	53.64	1.83	0.01	0.42
○Thread 856351			0.40	0.69	57.25	1.83	0.01	0.43
○Thread 856352			0.40	0.69	57.95	1.83	0.01	0.43
○Thread 856353			0.41	0.69	59.38	1.83	0.01	0.44
○Thread 856354			0.39	0.69	56.46	1.83	0.01	0.44
○Thread 856355			0.39	0.69	56.45	1.83	0.01	0.44
○Thread 856356			0.40	0.69	57.15	1.83	0.01	0.44
○Thread 856357			0.40	0.69	57.14	1.83	0.01	0.44
○Thread 856358			0.40	0.69	57.12	1.83	0.01	0.44
○Thread 856359			0.38	0.69	55.65	1.83	0.01	0.42
○Thread 856360			0.38	0.69	54.19	1.83	0.01	0.44
○Thread 856361			0.40	0.69	57.78	1.83	0.00	0.44
○Thread 856362			0.39	0.69	56.32	1.83	0.00	0.44
○Thread 856363			0.38	0.69	54.86	1.83	0.00	0.44
○Thread 856364			0.37	0.69	53.40	1.83	0.00	0.44
○Thread 856365			0.38	0.69	55.54	1.83	0.00	0.44
○Thread 856366			0.39	0.69	56.25	1.83	0.00	0.44
○Thread 856367			0.40	0.69	57.67	1.83	0.00	0.44
○Thread 856368			0.40	0.69	56.93	1.83	0.00	0.44
○Thread 856369			0.38	0.69	55.47	1.83	0.00	0.44
○Thread 856370			0.38	0.69	54.73	1.83	0.00	0.44
○Thread 856371			0.38	0.69	55.43	1.83	0.00	0.43
○Thread 856372			0.38	0.69	55.42	1.83	0.00	0.44

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	0.92
►Process 856441+		64	0.92
○Thread 856441			0.92	2.53	36.34	0.00	0.00	1.77
○Thread 856540			0.40	0.67	59.61	1.84	0.02	0.43
○Thread 856541			0.38	0.67	57.35	1.84	0.02	0.43
○Thread 856542			0.40	0.67	59.57	1.84	0.02	0.43
○Thread 856543			0.38	0.67	56.57	1.84	0.02	0.43
○Thread 856544			0.39	0.67	58.04	1.84	0.02	0.43
○Thread 856545			0.38	0.67	57.27	1.84	0.02	0.43
○Thread 856546			0.37	0.67	54.28	1.84	0.02	0.42
○Thread 856547			0.36	0.67	53.52	1.84	0.02	0.42
○Thread 856548			0.35	0.67	52.76	1.84	0.02	0.42
○Thread 856549			0.39	0.67	57.94	1.84	0.02	0.42
○Thread 856550			0.36	0.67	53.47	1.84	0.02	0.42
○Thread 856551			0.37	0.67	54.94	1.84	0.02	0.42
○Thread 856552			0.37	0.67	54.92	1.84	0.02	0.42
○Thread 856553			0.38	0.67	57.12	1.84	0.02	0.42
○Thread 856554			0.39	0.67	57.84	1.84	0.02	0.42
○Thread 856555			0.38	0.67	55.60	1.84	0.02	0.43
○Thread 856556			0.40	0.67	58.54	1.84	0.02	0.41
○Thread 856557			0.38	0.67	55.56	1.84	0.02	0.43
○Thread 856558			0.38	0.68	55.54	1.84	0.02	0.43
○Thread 856559			0.39	0.68	57.74	1.84	0.02	0.43
○Thread 856560			0.37	0.68	54.02	1.84	0.02	0.42
○Thread 856561			0.38	0.68	56.97	1.84	0.02	0.43
○Thread 856562			0.39	0.68	57.69	1.84	0.01	0.42
○Thread 856563			0.38	0.68	55.45	1.84	0.01	0.42
○Thread 856564			0.38	0.68	56.91	1.84	0.01	0.43
○Thread 856565			0.38	0.68	56.89	1.84	0.01	0.43
○Thread 856566			0.37	0.68	53.92	1.84	0.01	0.43
○Thread 856567			0.38	0.68	56.85	1.84	0.01	0.42
○Thread 856568			0.36	0.68	53.14	1.84	0.01	0.41
○Thread 856569			0.36	0.68	53.13	1.84	0.01	0.43
○Thread 856570			0.38	0.68	56.06	1.84	0.01	0.42
○Thread 856571			0.39	0.68	57.52	1.84	0.01	0.42
○Thread 856572			0.39	0.68	57.50	1.84	0.01	0.43
○Thread 856573			0.34	0.68	50.85	1.84	0.01	0.42
○Thread 856574			0.36	0.68	53.04	1.84	0.01	0.42
○Thread 856575			0.41	0.68	61.12	1.84	0.01	0.43
○Thread 856576			0.38	0.68	55.21	1.84	0.01	0.42
○Thread 856577			0.37	0.68	53.72	1.84	0.01	0.41
○Thread 856578			0.36	0.68	52.97	1.84	0.01	0.43
○Thread 856579			0.39	0.68	57.36	1.84	0.01	0.43
○Thread 856580			0.37	0.68	54.40	1.84	0.01	0.43
○Thread 856581			0.37	0.68	54.39	1.84	0.01	0.42
○Thread 856582			0.38	0.68	55.84	1.84	0.01	0.43
○Thread 856583			0.38	0.68	56.55	1.84	0.01	0.43
○Thread 856584			0.37	0.68	54.33	1.84	0.01	0.43
○Thread 856585			0.38	0.68	55.05	1.84	0.01	0.42
○Thread 856586			0.40	0.68	58.70	1.84	0.01	0.43
○Thread 856587			0.38	0.68	55.74	1.84	0.01	0.43
○Thread 856588			0.37	0.68	54.26	1.84	0.01	0.43
○Thread 856589			0.38	0.68	56.44	1.84	0.01	0.41
○Thread 856590			0.37	0.68	53.49	1.84	0.01	0.42
○Thread 856591			0.41	0.68	59.34	1.84	0.00	0.42
○Thread 856592			0.39	0.68	57.12	1.84	0.00	0.43
○Thread 856593			0.38	0.68	54.91	1.84	0.00	0.43
○Thread 856594			0.39	0.68	57.08	1.84	0.00	0.43
○Thread 856595			0.36	0.68	52.68	1.84	0.00	0.39
○Thread 856596			0.37	0.68	54.13	1.85	0.00	0.42
○Thread 856597			0.37	0.68	53.38	1.85	0.00	0.43
○Thread 856598			0.37	0.68	54.09	1.85	0.00	0.43
○Thread 856599			0.37	0.68	54.08	1.85	0.00	0.42
○Thread 856600			0.37	0.68	53.33	1.85	0.00	0.42
○Thread 856601			0.37	0.68	54.04	1.85	0.00	0.43
○Thread 856602			0.38	0.68	54.75	1.85	0.00	0.43

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	0.89
►Process 856622+		72	0.89
○Thread 856622			0.89	2.51	35.70	0.00	0.00	1.81
○Thread 856721			0.38	0.66	57.13	1.82	0.03	0.41
○Thread 856722			0.38	0.66	57.87	1.82	0.03	0.41
○Thread 856723			0.38	0.66	57.10	1.82	0.03	0.41
○Thread 856724			0.35	0.66	53.27	1.83	0.02	0.38
○Thread 856725			0.38	0.66	57.06	1.83	0.02	0.41
○Thread 856726			0.36	0.66	54.75	1.83	0.02	0.41
○Thread 856727			0.37	0.66	55.49	1.83	0.02	0.41
○Thread 856728			0.37	0.66	56.23	1.83	0.02	0.41
○Thread 856729			0.35	0.66	53.93	1.83	0.02	0.40
○Thread 856730			0.37	0.66	56.19	1.83	0.02	0.40
○Thread 856731			0.38	0.66	57.69	1.83	0.02	0.41
○Thread 856732			0.34	0.66	51.60	1.83	0.02	0.39
○Thread 856733			0.35	0.66	53.86	1.83	0.02	0.41
○Thread 856734			0.37	0.66	55.36	1.83	0.02	0.40
○Thread 856735			0.34	0.66	52.31	1.83	0.02	0.40
○Thread 856736			0.35	0.66	53.80	1.83	0.02	0.35
○Thread 856737			0.36	0.66	54.54	1.83	0.02	0.41
○Thread 856738			0.37	0.66	55.28	1.83	0.02	0.41
○Thread 856739			0.36	0.66	54.50	1.83	0.02	0.41
○Thread 856740			0.35	0.66	52.97	1.83	0.02	0.41
○Thread 856741			0.37	0.66	55.98	1.83	0.02	0.41
○Thread 856742			0.34	0.66	51.42	1.83	0.02	0.41
○Thread 856743			0.37	0.66	55.94	1.83	0.02	0.41
○Thread 856744			0.35	0.66	52.90	1.83	0.02	0.41
○Thread 856745			0.35	0.66	52.88	1.83	0.02	0.41
○Thread 856746			0.38	0.66	58.15	1.83	0.02	0.40
○Thread 856747			0.38	0.66	56.62	1.83	0.02	0.41
○Thread 856748			0.37	0.66	55.84	1.83	0.02	0.41
○Thread 856749			0.34	0.66	52.05	1.83	0.02	0.41
○Thread 856750			0.34	0.66	51.28	1.83	0.02	0.41
○Thread 856751			0.33	0.66	49.75	1.83	0.02	0.41
○Thread 856752			0.36	0.66	54.26	1.83	0.01	0.41
○Thread 856753			0.34	0.66	50.47	1.83	0.01	0.41
○Thread 856754			0.36	0.66	54.22	1.83	0.01	0.41
○Thread 856755			0.38	0.66	57.21	1.83	0.01	0.41
○Thread 856756			0.35	0.66	52.68	1.83	0.01	0.41
○Thread 856757			0.35	0.66	53.41	1.83	0.01	0.41
○Thread 856758			0.37	0.66	55.65	1.83	0.01	0.41
○Thread 856759			0.34	0.67	51.87	1.83	0.01	0.41
○Thread 856760			0.35	0.67	52.60	1.83	0.01	0.40
○Thread 856761			0.38	0.67	57.85	1.83	0.01	0.41
○Thread 856762			0.32	0.67	48.81	1.83	0.01	0.41
○Thread 856763			0.35	0.67	53.30	1.83	0.01	0.41
○Thread 856764			0.37	0.67	55.54	1.83	0.01	0.41
○Thread 856765			0.37	0.67	54.77	1.83	0.01	0.41
○Thread 856766			0.35	0.67	53.25	1.83	0.01	0.40
○Thread 856767			0.36	0.67	53.98	1.83	0.01	0.41
○Thread 856768			0.34	0.67	51.72	1.83	0.01	0.41
○Thread 856769			0.37	0.67	54.70	1.83	0.01	0.41
○Thread 856770			0.37	0.67	55.43	1.83	0.01	0.41
○Thread 856771			0.38	0.67	57.65	1.83	0.01	0.41
○Thread 856772			0.35	0.67	52.39	1.83	0.01	0.39
○Thread 856773			0.35	0.67	53.12	1.83	0.01	0.40
○Thread 856774			0.38	0.67	57.59	1.83	0.01	0.41
○Thread 856775			0.33	0.67	49.35	1.83	0.01	0.35
○Thread 856776			0.37	0.67	55.31	1.83	0.01	0.41
○Thread 856777			0.37	0.67	55.29	1.83	0.01	0.41
○Thread 856778			0.34	0.67	50.79	1.83	0.01	0.41
○Thread 856779			0.35	0.67	53.02	1.83	0.01	0.41
○Thread 856780			0.35	0.67	52.25	1.83	0.00	0.41
○Thread 856781			0.37	0.67	55.22	1.83	0.00	0.40
○Thread 856782			0.37	0.67	54.46	1.83	0.00	0.41
○Thread 856783			0.33	0.67	49.22	1.83	0.00	0.41
○Thread 856784			0.38	0.67	56.66	1.83	0.00	0.40
○Thread 856785			0.37	0.67	54.40	1.83	0.00	0.41
○Thread 856786			0.35	0.67	52.90	1.83	0.00	0.41
○Thread 856787			0.38	0.67	55.86	1.83	0.00	0.40
○Thread 856788			0.33	0.67	49.13	1.83	0.00	0.40
○Thread 856789			0.34	0.67	51.35	1.83	0.00	0.41
○Thread 856790			0.37	0.67	55.05	1.83	0.00	0.40
○Thread 856791			0.35	0.67	52.06	1.83	0.00	0.40

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	0.88
►Process 856811+		80	0.88
○Thread 856811			0.88	2.52	34.79	0.00	0.00	1.80
○Thread 856910			0.40	0.65	60.48	1.83	0.03	0.40
○Thread 856911			0.35	0.65	53.57	1.83	0.03	0.40
○Thread 856912			0.34	0.65	51.26	1.83	0.03	0.39
○Thread 856913			0.31	0.65	47.42	1.83	0.03	0.41
○Thread 856914			0.36	0.65	55.04	1.83	0.03	0.38
○Thread 856915			0.37	0.65	56.55	1.83	0.03	0.40
○Thread 856916			0.34	0.65	51.95	1.83	0.03	0.40
○Thread 856917			0.35	0.65	53.46	1.83	0.03	0.39
○Thread 856918			0.35	0.65	54.20	1.83	0.03	0.40
○Thread 856919			0.28	0.66	42.74	1.83	0.03	0.39
○Thread 856920			0.26	0.66	39.67	1.83	0.03	0.41
○Thread 856921			0.34	0.66	51.86	1.83	0.02	0.40
○Thread 856922			0.33	0.66	50.31	1.84	0.02	0.40
○Thread 856923			0.31	0.66	46.49	1.84	0.02	0.40
○Thread 856924			0.37	0.66	55.61	1.84	0.02	0.40
○Thread 856925			0.35	0.66	53.31	1.84	0.02	0.41
○Thread 856926			0.37	0.66	56.33	1.84	0.02	0.40
○Thread 856927			0.38	0.66	57.84	1.84	0.02	0.40
○Thread 856928			0.26	0.66	39.56	1.84	0.02	0.40
○Thread 856929			0.34	0.66	50.95	1.84	0.02	0.41
○Thread 856930			0.34	0.66	50.94	1.84	0.02	0.39
○Thread 856931			0.36	0.66	54.72	1.84	0.02	0.40
○Thread 856932			0.37	0.66	55.46	1.84	0.02	0.40
○Thread 856933			0.37	0.66	55.44	1.84	0.02	0.40
○Thread 856934			0.35	0.66	53.90	1.84	0.02	0.40
○Thread 856935			0.34	0.66	50.85	1.84	0.02	0.41
○Thread 856936			0.37	0.66	55.38	1.84	0.02	0.40
○Thread 856937			0.37	0.66	55.36	1.84	0.02	0.40
○Thread 856938			0.34	0.66	52.31	1.84	0.02	0.36
○Thread 856939			0.36	0.66	54.57	1.84	0.02	0.40
○Thread 856940			0.34	0.66	51.52	1.84	0.02	0.40
○Thread 856941			0.33	0.66	49.99	1.84	0.02	0.40
○Thread 856942			0.37	0.66	55.27	1.84	0.02	0.40
○Thread 856943			0.34	0.66	52.22	1.84	0.02	0.40
○Thread 856944			0.34	0.66	50.69	1.84	0.02	0.40
○Thread 856945			0.33	0.66	49.92	1.84	0.02	0.39
○Thread 856946			0.35	0.66	52.92	1.84	0.02	0.40
○Thread 856947			0.37	0.66	55.93	1.84	0.02	0.40
○Thread 856948			0.36	0.66	54.40	1.84	0.02	0.40
○Thread 856949			0.29	0.66	43.81	1.84	0.01	0.40
○Thread 856950			0.34	0.66	50.59	1.84	0.01	0.39
○Thread 856951			0.32	0.66	48.31	1.84	0.01	0.39
○Thread 856952			0.33	0.66	49.80	1.84	0.01	0.40
○Thread 856953			0.35	0.66	53.55	1.84	0.01	0.40
○Thread 856954			0.34	0.66	51.27	1.84	0.01	0.35
○Thread 856955			0.35	0.66	52.76	1.84	0.01	0.40
○Thread 856956			0.32	0.66	48.22	1.84	0.01	0.40
○Thread 856957			0.35	0.66	52.73	1.84	0.01	0.40
○Thread 856958			0.38	0.66	57.23	1.84	0.01	0.40
○Thread 856959			0.35	0.66	52.69	1.84	0.01	0.40
○Thread 856960			0.30	0.66	45.15	1.84	0.01	0.39
○Thread 856961			0.31	0.66	46.64	1.84	0.01	0.40
○Thread 856962			0.37	0.66	55.64	1.84	0.01	0.40
○Thread 856963			0.33	0.67	49.61	1.84	0.01	0.40
○Thread 856964			0.35	0.67	52.60	1.84	0.01	0.40
○Thread 856965			0.35	0.67	52.58	1.84	0.01	0.40
○Thread 856966			0.27	0.67	40.55	1.84	0.01	0.40
○Thread 856967			0.33	0.67	49.55	1.84	0.01	0.35
○Thread 856968			0.35	0.67	52.53	1.84	0.01	0.40
○Thread 856969			0.36	0.67	54.02	1.84	0.01	0.40
○Thread 856970			0.34	0.67	51.00	1.84	0.01	0.39
○Thread 856971			0.34	0.67	51.73	1.84	0.01	0.39
○Thread 856972			0.35	0.67	52.46	1.84	0.01	0.40
○Thread 856973			0.34	0.67	50.95	1.84	0.01	0.40
○Thread 856974			0.35	0.67	52.43	1.84	0.01	0.41
○Thread 856975			0.31	0.67	47.17	1.84	0.01	0.40
○Thread 856976			0.36	0.67	53.89	1.84	0.01	0.41
○Thread 856977			0.37	0.67	54.62	1.84	0.00	0.41
○Thread 856978			0.37	0.67	54.60	1.84	0.00	0.40
○Thread 856979			0.31	0.67	45.61	1.84	0.00	0.41
○Thread 856980			0.35	0.67	52.32	1.84	0.00	0.41
○Thread 856981			0.37	0.67	55.29	1.84	0.00	0.40
○Thread 856982			0.32	0.67	48.55	1.84	0.00	0.41
○Thread 856983			0.32	0.67	47.79	1.84	0.00	0.39
○Thread 856984			0.37	0.67	54.49	1.84	0.00	0.41
○Thread 856985			0.38	0.67	55.96	1.84	0.00	0.41
○Thread 856986			0.35	0.67	52.96	1.84	0.00	0.41
○Thread 856987			0.33	0.67	49.21	1.84	0.00	0.41
○Thread 856988			0.34	0.67	49.94	1.84	0.00	0.41

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	1.02
►Process 857008+		88	1.02
○Thread 857008			1.02	2.49	41.01	0.00	0.00	1.78
○Thread 857107			0.33	0.64	51.25	1.81	0.03	0.39
○Thread 857108			0.34	0.64	52.01	1.81	0.03	0.39
○Thread 857109			0.32	0.64	49.66	1.81	0.03	0.39
○Thread 857110			0.34	0.64	53.53	1.81	0.03	0.40
○Thread 857111			0.32	0.64	50.41	1.81	0.03	0.39
○Thread 857112			0.34	0.64	53.49	1.81	0.03	0.39
○Thread 857113			0.35	0.65	54.25	1.81	0.03	0.39
○Thread 857114			0.35	0.65	54.23	1.81	0.03	0.37
○Thread 857115			0.35	0.65	54.21	1.81	0.03	0.39
○Thread 857116			0.35	0.65	54.19	1.81	0.03	0.39
○Thread 857117			0.35	0.65	54.17	1.81	0.03	0.40
○Thread 857118			0.34	0.65	52.60	1.81	0.03	0.39
○Thread 857119			0.34	0.65	53.36	1.81	0.03	0.35
○Thread 857120			0.34	0.65	51.79	1.81	0.03	0.38
○Thread 857121			0.34	0.65	51.78	1.81	0.03	0.39
○Thread 857122			0.35	0.65	54.08	1.81	0.03	0.40
○Thread 857123			0.33	0.65	50.97	1.81	0.03	0.39
○Thread 857124			0.32	0.65	49.41	1.81	0.03	0.38
○Thread 857125			0.32	0.65	50.17	1.81	0.03	0.39
○Thread 857126			0.36	0.65	55.55	1.81	0.02	0.39
○Thread 857127			0.33	0.65	50.91	1.81	0.02	0.39
○Thread 857128			0.34	0.65	51.66	1.81	0.02	0.38
○Thread 857129			0.32	0.65	49.33	1.82	0.02	0.37
○Thread 857130			0.32	0.65	50.08	1.82	0.02	0.39
○Thread 857131			0.32	0.65	49.30	1.82	0.02	0.39
○Thread 857132			0.32	0.65	50.05	1.82	0.02	0.39
○Thread 857133			0.34	0.65	52.34	1.82	0.02	0.38
○Thread 857134			0.33	0.65	50.79	1.82	0.02	0.38
○Thread 857135			0.32	0.65	50.01	1.82	0.02	0.39
○Thread 857136			0.35	0.65	54.60	1.82	0.02	0.39
○Thread 857137			0.34	0.65	51.51	1.82	0.02	0.39
○Thread 857138			0.31	0.65	48.42	1.82	0.02	0.35
○Thread 857139			0.36	0.65	55.31	1.82	0.02	0.39
○Thread 857140			0.33	0.65	50.69	1.82	0.02	0.39
○Thread 857141			0.34	0.65	51.44	1.82	0.02	0.38
○Thread 857142			0.33	0.65	50.65	1.82	0.02	0.38
○Thread 857143			0.34	0.65	51.40	1.82	0.02	0.39
○Thread 857144			0.34	0.65	51.38	1.82	0.02	0.39
○Thread 857145			0.31	0.65	48.30	1.82	0.02	0.39
○Thread 857146			0.35	0.65	53.65	1.82	0.02	0.39
○Thread 857147			0.34	0.65	52.10	1.82	0.02	0.40
○Thread 857148			0.31	0.65	47.49	1.82	0.02	0.39
○Thread 857149			0.34	0.65	52.06	1.82	0.02	0.39
○Thread 857150			0.33	0.65	50.52	1.82	0.02	0.39
○Thread 857151			0.32	0.65	48.97	1.82	0.02	0.38
○Thread 857152			0.34	0.65	51.25	1.82	0.02	0.39
○Thread 857153			0.34	0.65	51.99	1.82	0.02	0.38
○Thread 857154			0.34	0.65	51.21	1.82	0.01	0.39
○Thread 857155			0.33	0.65	50.43	1.82	0.01	0.38
○Thread 857156			0.35	0.65	53.46	1.82	0.01	0.39
○Thread 857157			0.32	0.65	48.87	1.82	0.01	0.34
○Thread 857158			0.32	0.66	49.61	1.82	0.01	0.37
○Thread 857159			0.35	0.66	54.17	1.82	0.01	0.39
○Thread 857160			0.35	0.66	54.15	1.82	0.01	0.40
○Thread 857161			0.32	0.66	49.56	1.82	0.01	0.39
○Thread 857162			0.35	0.66	53.35	1.82	0.01	0.39
○Thread 857163			0.31	0.66	47.24	1.82	0.01	0.39
○Thread 857164			0.34	0.66	52.55	1.82	0.01	0.39
○Thread 857165			0.30	0.66	45.68	1.82	0.01	0.39
○Thread 857166			0.33	0.66	50.24	1.82	0.01	0.39
○Thread 857167			0.32	0.66	48.70	1.82	0.01	0.37
○Thread 857168			0.34	0.66	51.73	1.82	0.01	0.39
○Thread 857169			0.32	0.66	49.43	1.82	0.01	0.39
○Thread 857170			0.32	0.66	49.41	1.82	0.01	0.39
○Thread 857171			0.34	0.66	50.92	1.82	0.01	0.39
○Thread 857172			0.32	0.66	49.38	1.82	0.01	0.39
○Thread 857173			0.34	0.66	50.88	1.82	0.01	0.39
○Thread 857174			0.32	0.66	49.34	1.82	0.01	0.39
○Thread 857175			0.32	0.66	48.57	1.82	0.01	0.39
○Thread 857176			0.35	0.66	53.10	1.82	0.01	0.39
○Thread 857177			0.35	0.66	53.84	1.82	0.01	0.39
○Thread 857178			0.35	0.66	53.07	1.82	0.01	0.39
○Thread 857179			0.34	0.66	50.77	1.82	0.01	0.35
○Thread 857180			0.31	0.66	46.97	1.82	0.01	0.39
○Thread 857181			0.29	0.66	44.68	1.82	0.01	0.38
○Thread 857182			0.33	0.66	49.96	1.82	0.00	0.39
○Thread 857183			0.34	0.66	50.70	1.82	0.00	0.39
○Thread 857184			0.31	0.66	47.66	1.82	0.00	0.38
○Thread 857185			0.35	0.66	53.69	1.82	0.00	0.39
○Thread 857186			0.34	0.66	52.16	1.82	0.00	0.39
○Thread 857187			0.34	0.66	50.63	1.82	0.00	0.39
○Thread 857188			0.32	0.66	49.11	1.82	0.00	0.39
○Thread 857189			0.32	0.66	49.09	1.82	0.00	0.39
○Thread 857190			0.35	0.66	52.85	1.82	0.00	0.38
○Thread 857191			0.32	0.66	49.05	1.82	0.00	0.39
○Thread 857192			0.34	0.66	50.55	1.82	0.00	0.40
○Thread 857193			0.32	0.66	49.02	1.82	0.00	0.39

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	0.94
►Process 857213+		96	0.94
○Thread 857213			0.94	2.52	37.32	0.00	0.00	1.82
○Thread 857312			0.37	0.65	56.57	1.84	0.04	0.40
○Thread 857313			0.32	0.65	49.58	1.84	0.03	0.39
○Thread 857314			0.35	0.65	54.21	1.84	0.03	0.39
○Thread 857315			0.32	0.65	49.55	1.84	0.03	0.39
○Thread 857316			0.33	0.65	51.08	1.84	0.03	0.39
○Thread 857317			0.36	0.65	55.63	1.84	0.03	0.39
○Thread 857318			0.32	0.65	50.21	1.84	0.03	0.39
○Thread 857319			0.35	0.65	54.82	1.84	0.03	0.39
○Thread 857320			0.33	0.65	50.94	1.84	0.03	0.37
○Thread 857321			0.31	0.65	48.61	1.84	0.03	0.36
○Thread 857322			0.33	0.65	50.91	1.84	0.03	0.39
○Thread 857323			0.32	0.65	49.35	1.84	0.03	0.38
○Thread 857324			0.33	0.65	50.87	1.84	0.03	0.39
○Thread 857325			0.33	0.65	50.85	1.84	0.03	0.39
○Thread 857326			0.37	0.65	57.00	1.84	0.03	0.39
○Thread 857327			0.34	0.65	52.36	1.84	0.03	0.39
○Thread 857328			0.32	0.65	49.26	1.84	0.03	0.39
○Thread 857329			0.35	0.65	54.63	1.84	0.03	0.39
○Thread 857330			0.36	0.65	55.38	1.84	0.03	0.39
○Thread 857331			0.34	0.65	52.28	1.84	0.03	0.39
○Thread 857332			0.35	0.65	53.80	1.84	0.03	0.39
○Thread 857333			0.35	0.65	54.55	1.84	0.03	0.40
○Thread 857334			0.36	0.65	55.30	1.84	0.03	0.39
○Thread 857335			0.32	0.65	49.90	1.84	0.03	0.39
○Thread 857336			0.34	0.65	52.19	1.84	0.03	0.39
○Thread 857337			0.31	0.65	48.34	1.84	0.03	0.36
○Thread 857338			0.31	0.65	48.32	1.84	0.03	0.38
○Thread 857339			0.34	0.65	52.14	1.84	0.02	0.38
○Thread 857340			0.30	0.65	45.99	1.84	0.02	0.36
○Thread 857341			0.34	0.65	51.34	1.84	0.02	0.39
○Thread 857342			0.33	0.65	50.56	1.84	0.02	0.39
○Thread 857343			0.34	0.65	52.07	1.84	0.02	0.39
○Thread 857344			0.34	0.65	52.82	1.84	0.02	0.39
○Thread 857345			0.32	0.65	49.74	1.84	0.02	0.37
○Thread 857346			0.29	0.65	45.13	1.84	0.02	0.37
○Thread 857347			0.34	0.65	51.23	1.84	0.02	0.38
○Thread 857348			0.30	0.65	45.86	1.84	0.02	0.34
○Thread 857349			0.34	0.65	51.20	1.84	0.02	0.38
○Thread 857350			0.32	0.65	49.65	1.84	0.02	0.39
○Thread 857351			0.34	0.65	52.69	1.84	0.02	0.39
○Thread 857352			0.31	0.65	46.57	1.84	0.02	0.39
○Thread 857353			0.33	0.66	50.37	1.84	0.02	0.39
○Thread 857354			0.32	0.66	48.83	1.84	0.02	0.39
○Thread 857355			0.34	0.66	51.86	1.84	0.02	0.38
○Thread 857356			0.32	0.66	48.79	1.84	0.02	0.39
○Thread 857357			0.33	0.66	50.30	1.84	0.02	0.37
○Thread 857358			0.35	0.66	53.33	1.84	0.02	0.38
○Thread 857359			0.33	0.66	50.27	1.84	0.02	0.38
○Thread 857360			0.34	0.66	51.01	1.84	0.02	0.40
○Thread 857361			0.32	0.66	48.71	1.84	0.02	0.38
○Thread 857362			0.34	0.66	51.74	1.84	0.02	0.39
○Thread 857363			0.32	0.66	48.68	1.84	0.02	0.39
○Thread 857364			0.32	0.66	49.42	1.84	0.02	0.39
○Thread 857365			0.32	0.66	49.40	1.85	0.02	0.39
○Thread 857366			0.34	0.66	52.43	1.85	0.02	0.39
○Thread 857367			0.35	0.66	53.17	1.85	0.01	0.39
○Thread 857368			0.35	0.66	53.15	1.85	0.01	0.39
○Thread 857369			0.34	0.66	51.61	1.85	0.01	0.39
○Thread 857370			0.31	0.66	47.04	1.85	0.01	0.39
○Thread 857371			0.34	0.66	51.58	1.85	0.01	0.38
○Thread 857372			0.34	0.66	51.56	1.85	0.01	0.34
○Thread 857373			0.35	0.66	53.06	1.85	0.01	0.37
○Thread 857374			0.35	0.66	53.04	1.85	0.01	0.39
○Thread 857375			0.34	0.66	50.75	1.85	0.01	0.39
○Thread 857376			0.32	0.66	49.22	1.85	0.01	0.39
○Thread 857377			0.31	0.66	47.69	1.85	0.01	0.37
○Thread 857378			0.31	0.66	47.67	1.85	0.01	0.39
○Thread 857379			0.32	0.66	49.17	1.85	0.01	0.39
○Thread 857380			0.34	0.66	51.42	1.85	0.01	0.39
○Thread 857381			0.35	0.66	53.67	1.85	0.01	0.39
○Thread 857382			0.30	0.66	45.34	1.85	0.01	0.39
○Thread 857383			0.33	0.66	49.85	1.85	0.01	0.39
○Thread 857384			0.34	0.66	51.35	1.85	0.01	0.39
○Thread 857385			0.34	0.66	52.08	1.85	0.01	0.39
○Thread 857386			0.31	0.66	47.54	1.85	0.01	0.39
○Thread 857387			0.33	0.66	49.79	1.85	0.01	0.38
○Thread 857388			0.34	0.66	51.28	1.85	0.01	0.39
○Thread 857389			0.32	0.66	48.24	1.85	0.01	0.39
○Thread 857390			0.33	0.66	49.73	1.85	0.01	0.39
○Thread 857391			0.33	0.66	49.72	1.85	0.01	0.39
○Thread 857392			0.34	0.66	51.20	1.85	0.01	0.39
○Thread 857393			0.32	0.66	48.93	1.85	0.01	0.39
○Thread 857394			0.31	0.66	47.41	1.85	0.01	0.39
○Thread 857395			0.34	0.66	50.40	1.85	0.00	0.39
○Thread 857396			0.32	0.66	48.13	1.85	0.00	0.37
○Thread 857397			0.35	0.67	53.37	1.85	0.00	0.39
○Thread 857398			0.32	0.67	48.84	1.85	0.00	0.39
○Thread 857399			0.32	0.67	48.07	1.85	0.00	0.39
○Thread 857400			0.31	0.67	47.31	1.85	0.00	0.38
○Thread 857401			0.33	0.67	49.54	1.85	0.00	0.37
○Thread 857402			0.34	0.67	50.27	1.85	0.00	0.38
○Thread 857403			0.35	0.67	53.25	1.85	0.00	0.39
○Thread 857404			0.36	0.67	53.98	1.85	0.00	0.39
○Thread 857405			0.31	0.67	45.72	1.85	0.00	0.39
○Thread 857406			0.29	0.67	44.21	1.85	0.00	0.35

Report Configuration