OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	557.43
►Process 57985+		1	557.43
○Thread 57985			557.43	558.55	99.80	0.00	0.00	558.51

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	277.65
►Process 58374+		2	277.65
○Thread 58374			277.65	278.76	99.60	0.00	0.00	278.70
○Thread 58441			277.27	277.69	99.85	1.07	-0.00	277.34

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	139.54
►Process 58613+		4	139.54
○Thread 58613			139.54	140.60	99.24	0.00	0.00	140.56
○Thread 58680			139.12	139.54	99.70	1.06	0.00	139.19
○Thread 58681			139.12	139.54	99.70	1.06	0.00	139.19
○Thread 58682			139.12	139.54	99.70	1.06	-0.00	139.19

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	70.65
►Process 58756+		8	70.65
○Thread 58756			70.65	71.75	98.47	0.00	0.00	71.70
○Thread 58823			70.24	70.69	99.37	1.06	0.00	70.35
○Thread 58824			70.25	70.69	99.38	1.06	0.00	70.34
○Thread 58825			70.23	70.69	99.35	1.06	0.00	70.34
○Thread 58826			70.23	70.69	99.35	1.06	0.00	70.34
○Thread 58827			70.22	70.69	99.34	1.06	0.00	70.34
○Thread 58828			70.22	70.69	99.33	1.06	0.00	70.34
○Thread 58829			70.23	70.69	99.35	1.06	-0.00	70.34

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	36.56
►Process 58897+		16	36.56
○Thread 58897			36.56	37.78	96.76	0.00	0.00	37.73
○Thread 58964			36.15	36.72	98.46	1.07	0.00	36.37
○Thread 58965			36.12	36.72	98.38	1.07	-0.00	36.37
○Thread 58966			36.14	36.72	98.42	1.07	0.00	36.36
○Thread 58967			36.18	36.72	98.54	1.07	0.00	36.37
○Thread 58968			36.15	36.72	98.45	1.07	0.00	36.37
○Thread 58969			36.09	36.72	98.30	1.07	0.00	36.36
○Thread 58970			36.16	36.72	98.49	1.07	0.00	36.36
○Thread 58971			36.08	36.72	98.26	1.07	0.00	36.36
○Thread 58972			36.09	36.72	98.28	1.07	0.00	36.36
○Thread 58973			36.09	36.72	98.30	1.07	0.00	36.37
○Thread 58974			36.12	36.72	98.37	1.07	0.00	36.37
○Thread 58975			36.14	36.72	98.43	1.07	0.00	36.37
○Thread 58976			36.11	36.71	98.35	1.07	0.00	36.37
○Thread 58977			36.09	36.71	98.28	1.07	0.00	36.37
○Thread 58978			36.11	36.71	98.34	1.07	-0.00	36.36

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	26.40
►Process 59047+		24	26.40
○Thread 59047			26.40	27.79	94.98	0.00	0.00	27.74
○Thread 59114			26.07	26.73	97.55	1.07	0.00	26.38
○Thread 59115			26.00	26.73	97.30	1.07	0.00	26.38
○Thread 59116			26.07	26.73	97.57	1.07	0.00	26.38
○Thread 59117			26.05	26.73	97.49	1.07	0.00	26.38
○Thread 59118			25.97	26.73	97.19	1.07	0.00	26.38
○Thread 59119			26.02	26.73	97.38	1.07	0.00	26.38
○Thread 59120			25.99	26.73	97.27	1.07	0.00	26.38
○Thread 59121			26.07	26.72	97.57	1.07	0.00	26.37
○Thread 59122			25.95	26.72	97.10	1.07	0.00	26.38
○Thread 59123			26.02	26.72	97.36	1.07	0.00	26.38
○Thread 59124			25.97	26.72	97.19	1.07	0.00	26.38
○Thread 59125			26.00	26.72	97.31	1.07	0.00	26.38
○Thread 59126			26.02	26.72	97.36	1.07	0.00	26.37
○Thread 59127			26.04	26.72	97.46	1.07	0.00	26.38
○Thread 59128			26.08	26.72	97.61	1.07	0.00	26.37
○Thread 59129			26.03	26.72	97.40	1.07	0.00	26.37
○Thread 59130			25.96	26.72	97.14	1.07	0.00	26.37
○Thread 59131			25.98	26.72	97.23	1.07	0.00	26.38
○Thread 59132			26.04	26.72	97.44	1.07	0.00	26.37
○Thread 59133			25.95	26.72	97.10	1.07	0.00	26.37
○Thread 59134			25.99	26.72	97.27	1.07	0.00	26.38
○Thread 59135			26.01	26.72	97.35	1.07	0.00	26.38
○Thread 59136			25.98	26.72	97.22	1.07	-0.00	26.38

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	19.98
►Process 59159+		32	19.98
○Thread 59159			19.98	21.42	93.31	0.00	0.00	21.37
○Thread 59226			19.62	20.35	96.42	1.07	0.00	19.99
○Thread 59227			19.53	20.35	96.00	1.07	0.00	19.98
○Thread 59228			19.54	20.35	96.05	1.07	0.00	19.99
○Thread 59229			19.58	20.35	96.25	1.07	0.00	19.99
○Thread 59230			19.49	20.35	95.78	1.07	0.00	19.98
○Thread 59231			19.54	20.35	96.05	1.07	-0.00	19.99
○Thread 59232			19.57	20.35	96.20	1.07	0.00	19.99
○Thread 59233			19.57	20.35	96.17	1.07	0.00	19.99
○Thread 59234			19.53	20.35	95.98	1.07	0.00	19.98
○Thread 59235			19.62	20.35	96.42	1.07	0.00	20.00
○Thread 59236			19.59	20.35	96.27	1.07	0.00	19.99
○Thread 59237			19.55	20.35	96.10	1.07	0.00	19.99
○Thread 59238			19.64	20.35	96.52	1.07	0.00	20.00
○Thread 59239			19.58	20.35	96.23	1.07	0.00	19.99
○Thread 59240			19.58	20.35	96.25	1.07	0.00	19.99
○Thread 59241			19.52	20.35	95.93	1.07	0.00	20.00
○Thread 59242			19.60	20.35	96.33	1.07	0.00	20.00
○Thread 59243			19.60	20.35	96.35	1.07	0.00	19.97
○Thread 59244			19.54	20.35	96.06	1.07	0.00	20.00
○Thread 59245			19.59	20.35	96.28	1.07	0.00	20.00
○Thread 59246			19.62	20.35	96.43	1.07	0.00	20.00
○Thread 59247			19.55	20.35	96.08	1.07	0.00	19.98
○Thread 59248			19.54	20.35	96.03	1.07	0.00	19.99
○Thread 59249			19.62	20.35	96.45	1.07	0.00	19.99
○Thread 59250			19.57	20.35	96.18	1.07	0.00	19.99
○Thread 59251			19.64	20.35	96.53	1.07	0.00	19.99
○Thread 59252			19.59	20.35	96.31	1.07	0.00	19.99
○Thread 59253			19.59	20.35	96.28	1.07	0.00	19.98
○Thread 59254			19.75	20.35	97.07	1.07	0.00	20.00
○Thread 59255			19.56	20.35	96.14	1.07	0.00	19.98
○Thread 59256			19.62	20.35	96.46	1.07	-0.00	19.98

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	17.27
►Process 59328+		40	17.27
○Thread 59328			17.27	18.92	91.27	0.00	0.00	18.88
○Thread 59395			16.91	17.85	94.71	1.06	0.00	17.49
○Thread 59396			16.92	17.85	94.77	1.06	-0.00	17.50
○Thread 59397			16.92	17.85	94.77	1.06	-0.00	17.49
○Thread 59398			16.84	17.85	94.30	1.06	-0.00	17.49
○Thread 59399			16.81	17.85	94.16	1.06	-0.00	17.49
○Thread 59400			17.00	17.85	95.19	1.06	-0.00	17.49
○Thread 59401			17.04	17.85	95.45	1.06	0.00	17.49
○Thread 59402			16.94	17.85	94.89	1.06	0.00	17.49
○Thread 59403			16.98	17.85	95.08	1.06	-0.00	17.50
○Thread 59404			16.89	17.85	94.61	1.06	-0.00	17.50
○Thread 59405			16.93	17.85	94.83	1.06	0.00	17.50
○Thread 59406			16.96	17.85	95.00	1.06	-0.00	17.49
○Thread 59407			16.87	17.85	94.50	1.06	0.00	17.50
○Thread 59408			16.93	17.85	94.81	1.06	0.00	17.49
○Thread 59409			16.79	17.85	94.05	1.06	0.00	17.49
○Thread 59410			16.88	17.85	94.55	1.06	0.00	17.50
○Thread 59411			16.88	17.85	94.53	1.06	0.00	17.50
○Thread 59412			16.81	17.85	94.16	1.06	0.00	17.49
○Thread 59413			16.84	17.85	94.30	1.06	-0.00	17.50
○Thread 59414			16.91	17.85	94.70	1.06	0.00	17.50
○Thread 59415			16.91	17.85	94.73	1.06	0.00	17.50
○Thread 59416			16.88	17.85	94.56	1.06	0.00	17.49
○Thread 59417			16.89	17.85	94.61	1.06	0.00	17.50
○Thread 59418			16.99	17.85	95.15	1.06	0.00	17.50
○Thread 59419			16.89	17.85	94.59	1.06	0.00	17.50
○Thread 59420			17.01	17.85	95.29	1.06	0.00	17.50
○Thread 59421			16.91	17.85	94.73	1.06	0.00	17.50
○Thread 59422			16.95	17.85	94.95	1.06	0.00	17.50
○Thread 59423			16.99	17.85	95.18	1.07	0.00	17.50
○Thread 59424			16.87	17.85	94.48	1.07	0.00	17.50
○Thread 59425			16.95	17.85	94.95	1.07	0.00	17.50
○Thread 59426			16.85	17.85	94.39	1.07	0.00	17.50
○Thread 59427			16.97	17.85	95.07	1.07	0.00	17.50
○Thread 59428			17.00	17.85	95.21	1.07	0.00	17.49
○Thread 59429			16.89	17.85	94.62	1.07	0.00	17.50
○Thread 59430			16.82	17.85	94.23	1.07	0.00	17.50
○Thread 59431			16.79	17.85	94.03	1.07	0.00	17.50
○Thread 59432			16.87	17.85	94.48	1.07	0.00	17.49
○Thread 59433			16.84	17.85	94.34	1.07	-0.00	17.50

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	15.24
►Process 59453+		48	15.24
○Thread 59453			15.24	16.99	89.66	0.00	0.00	16.95
○Thread 59521			14.85	15.92	93.24	1.07	0.00	15.55
○Thread 59522			14.89	15.92	93.49	1.07	0.00	15.55
○Thread 59523			14.86	15.92	93.33	1.07	0.00	15.55
○Thread 59524			14.84	15.92	93.18	1.07	0.00	15.55
○Thread 59525			14.85	15.92	93.27	1.07	0.00	15.55
○Thread 59526			14.86	15.92	93.33	1.07	0.00	15.55
○Thread 59527			14.93	15.92	93.74	1.07	0.00	15.55
○Thread 59528			14.78	15.92	92.83	1.07	0.00	15.55
○Thread 59529			14.88	15.92	93.43	1.07	-0.00	15.54
○Thread 59530			14.76	15.92	92.68	1.07	0.00	15.55
○Thread 59531			14.83	15.92	93.12	1.07	0.00	15.55
○Thread 59532			14.83	15.92	93.12	1.07	0.00	15.55
○Thread 59533			14.81	15.92	93.02	1.07	0.00	15.55
○Thread 59534			14.85	15.92	93.28	1.07	0.00	15.55
○Thread 59535			14.83	15.92	93.15	1.07	0.00	15.55
○Thread 59536			14.88	15.92	93.43	1.07	0.00	15.55
○Thread 59537			14.80	15.92	92.93	1.07	0.00	15.55
○Thread 59538			14.83	15.92	93.15	1.07	0.00	15.55
○Thread 59539			14.82	15.92	93.06	1.07	0.00	15.55
○Thread 59540			14.76	15.92	92.68	1.07	0.00	15.55
○Thread 59541			14.84	15.92	93.19	1.07	0.00	15.55
○Thread 59542			14.95	15.92	93.91	1.07	0.00	15.55
○Thread 59543			14.89	15.92	93.50	1.07	0.00	15.55
○Thread 59544			14.82	15.92	93.09	1.07	0.00	15.55
○Thread 59545			14.82	15.92	93.09	1.07	0.00	15.55
○Thread 59546			14.85	15.92	93.25	1.07	0.00	15.55
○Thread 59547			14.97	15.92	94.04	1.07	0.00	15.55
○Thread 59548			14.93	15.92	93.79	1.07	0.00	15.56
○Thread 59549			14.88	15.92	93.47	1.07	0.00	15.55
○Thread 59550			14.80	15.92	92.97	1.07	0.00	15.55
○Thread 59551			14.97	15.92	94.04	1.07	0.00	15.55
○Thread 59552			14.98	15.92	94.10	1.07	0.00	15.55
○Thread 59553			14.90	15.92	93.60	1.07	0.00	15.55
○Thread 59554			14.91	15.92	93.66	1.07	0.00	15.56
○Thread 59555			14.95	15.92	93.92	1.07	0.00	15.55
○Thread 59556			14.90	15.92	93.60	1.07	0.00	15.55
○Thread 59557			14.98	15.92	94.07	1.07	0.00	15.55
○Thread 59558			14.92	15.92	93.70	1.07	0.00	15.55
○Thread 59559			14.94	15.92	93.82	1.07	0.00	15.55
○Thread 59560			14.82	15.92	93.07	1.07	0.00	15.56
○Thread 59561			14.85	15.92	93.26	1.07	0.00	15.55
○Thread 59562			14.76	15.92	92.70	1.07	0.00	15.55
○Thread 59563			14.88	15.92	93.48	1.07	0.00	15.55
○Thread 59564			14.81	15.92	93.01	1.08	0.00	15.55
○Thread 59565			14.82	15.92	93.11	1.08	0.00	15.55
○Thread 59566			14.74	15.92	92.60	1.08	0.00	15.55
○Thread 59567			14.83	15.92	93.17	1.08	-0.00	15.55

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	14.04
►Process 59638+		56	14.04
○Thread 59638			14.04	15.95	87.98	0.00	0.00	15.89
○Thread 59705			13.66	14.88	91.77	1.07	0.00	14.50
○Thread 59706			13.67	14.88	91.88	1.07	0.00	14.49
○Thread 59707			13.66	14.88	91.78	1.07	0.00	14.50
○Thread 59708			13.68	14.88	91.91	1.07	0.00	14.50
○Thread 59709			13.65	14.88	91.71	1.07	0.00	14.51
○Thread 59710			13.62	14.88	91.54	1.07	0.00	14.50
○Thread 59711			13.81	14.88	92.82	1.07	0.00	14.52
○Thread 59712			13.67	14.88	91.85	1.07	0.00	14.50
○Thread 59713			13.67	14.88	91.85	1.07	0.00	14.50
○Thread 59714			13.62	14.88	91.51	1.07	0.00	14.50
○Thread 59715			13.71	14.88	92.12	1.07	0.00	14.51
○Thread 59716			13.64	14.88	91.68	1.07	0.00	14.50
○Thread 59717			13.76	14.88	92.46	1.07	0.00	14.50
○Thread 59718			13.75	14.88	92.42	1.07	0.00	14.51
○Thread 59719			13.65	14.88	91.75	1.07	0.00	14.50
○Thread 59720			13.74	14.88	92.32	1.07	0.00	14.50
○Thread 59721			13.57	14.88	91.18	1.07	0.00	14.49
○Thread 59722			13.44	14.88	90.31	1.07	0.00	14.50
○Thread 59723			13.57	14.88	91.22	1.07	0.00	14.50
○Thread 59724			13.51	14.88	90.78	1.08	0.00	14.49
○Thread 59725			13.62	14.88	91.55	1.08	0.00	14.50
○Thread 59726			13.47	14.88	90.51	1.08	0.00	14.50
○Thread 59727			13.46	14.88	90.48	1.08	0.00	14.50
○Thread 59728			13.65	14.88	91.72	1.08	0.00	14.50
○Thread 59729			13.72	14.88	92.19	1.08	0.00	14.50
○Thread 59730			13.70	14.88	92.09	1.08	0.00	14.52
○Thread 59731			13.52	14.88	90.88	1.08	0.00	14.50
○Thread 59732			13.65	14.88	91.76	1.08	0.00	14.49
○Thread 59733			13.76	14.88	92.50	1.08	0.00	14.50
○Thread 59734			13.82	14.88	92.87	1.08	0.00	14.50
○Thread 59735			13.73	14.88	92.30	1.08	0.00	14.50
○Thread 59736			13.68	14.88	91.96	1.08	0.00	14.50
○Thread 59737			13.71	14.88	92.13	1.08	0.00	14.50
○Thread 59738			13.71	14.88	92.13	1.08	0.00	14.50
○Thread 59739			13.81	14.88	92.80	1.08	0.00	14.52
○Thread 59740			13.65	14.88	91.73	1.08	0.00	14.50
○Thread 59741			13.82	14.88	92.87	1.08	0.00	14.50
○Thread 59742			13.67	14.87	91.87	1.08	0.00	14.50
○Thread 59743			13.64	14.87	91.70	1.08	0.00	14.50
○Thread 59744			13.70	14.87	92.10	1.08	0.00	14.50
○Thread 59745			13.75	14.87	92.41	1.08	0.00	14.51
○Thread 59746			13.59	14.87	91.33	1.08	0.00	14.50
○Thread 59747			13.75	14.87	92.41	1.08	0.00	14.50
○Thread 59748			13.69	14.87	92.00	1.08	0.00	14.50
○Thread 59749			13.65	14.87	91.77	1.08	0.00	14.50
○Thread 59750			13.64	14.87	91.67	1.08	0.00	14.50
○Thread 59751			13.57	14.87	91.20	1.08	0.00	14.50
○Thread 59752			13.75	14.87	92.41	1.08	0.00	14.51
○Thread 59753			13.56	14.87	91.17	1.08	0.00	14.50
○Thread 59754			13.63	14.87	91.60	1.08	0.00	14.50
○Thread 59755			13.51	14.87	90.83	1.08	0.00	14.50
○Thread 59756			13.48	14.87	90.60	1.08	0.00	14.50
○Thread 59757			13.51	14.87	90.83	1.08	0.00	14.50
○Thread 59758			13.52	14.87	90.90	1.08	0.00	14.50
○Thread 59759			13.59	14.87	91.37	1.08	0.00	14.50

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	12.55
►Process 59779+		64	12.55
○Thread 59779			12.55	14.35	87.40	0.00	0.00	14.31
○Thread 59846			12.03	13.28	90.61	1.08	0.00	12.91
○Thread 59847			12.02	13.28	90.53	1.08	-0.00	12.89
○Thread 59848			12.03	13.28	90.61	1.08	-0.00	12.89
○Thread 59849			12.24	13.28	92.19	1.08	-0.00	12.89
○Thread 59850			11.99	13.28	90.27	1.08	-0.00	12.88
○Thread 59851			12.08	13.28	90.99	1.08	-0.00	12.88
○Thread 59852			12.04	13.28	90.65	1.08	-0.00	12.91
○Thread 59853			12.07	13.28	90.88	1.08	-0.00	12.89
○Thread 59854			12.07	13.28	90.92	1.08	-0.00	12.89
○Thread 59855			12.01	13.28	90.47	1.08	-0.00	12.89
○Thread 59856			12.00	13.28	90.35	1.08	-0.00	12.89
○Thread 59857			11.98	13.28	90.24	1.08	-0.00	12.89
○Thread 59858			12.02	13.28	90.54	1.08	-0.00	12.89
○Thread 59859			12.12	13.28	91.26	1.08	-0.00	12.91
○Thread 59860			12.02	13.28	90.54	1.08	-0.00	12.89
○Thread 59861			11.98	13.28	90.24	1.08	-0.00	12.91
○Thread 59862			12.01	13.28	90.43	1.08	-0.00	12.88
○Thread 59863			12.08	13.27	90.96	1.08	-0.00	12.89
○Thread 59864			12.08	13.27	91.00	1.08	-0.00	12.89
○Thread 59865			11.94	13.27	89.95	1.08	-0.00	12.89
○Thread 59866			11.97	13.27	90.17	1.08	-0.00	12.90
○Thread 59867			11.94	13.27	89.95	1.08	-0.00	12.89
○Thread 59868			12.09	13.27	91.04	1.08	-0.00	12.89
○Thread 59869			11.98	13.27	90.25	1.08	-0.00	12.89
○Thread 59870			11.90	13.27	89.65	1.08	-0.00	12.89
○Thread 59871			12.01	13.27	90.44	1.08	-0.00	12.89
○Thread 59872			12.03	13.27	90.59	1.08	-0.00	12.88
○Thread 59873			11.96	13.27	90.10	1.08	-0.00	12.91
○Thread 59874			11.92	13.27	89.80	1.08	-0.00	12.91
○Thread 59875			12.00	13.27	90.37	1.08	-0.00	12.91
○Thread 59876			11.93	13.27	89.88	1.08	-0.00	12.88
○Thread 59877			12.07	13.27	90.93	1.08	-0.00	12.89
○Thread 59878			12.16	13.27	91.57	1.08	-0.00	12.89
○Thread 59879			12.03	13.27	90.63	1.08	-0.00	12.89
○Thread 59880			12.19	13.27	91.84	1.08	-0.00	12.91
○Thread 59881			12.08	13.27	91.01	1.08	-0.00	12.91
○Thread 59882			12.04	13.27	90.71	1.08	-0.00	12.89
○Thread 59883			12.20	13.27	91.88	1.08	-0.00	12.89
○Thread 59884			12.04	13.27	90.67	1.08	-0.00	12.88
○Thread 59885			12.02	13.27	90.52	1.08	-0.00	12.91
○Thread 59886			12.15	13.27	91.54	1.08	-0.00	12.91
○Thread 59887			12.04	13.27	90.68	1.08	-0.00	12.90
○Thread 59888			12.14	13.27	91.47	1.08	-0.00	12.88
○Thread 59889			11.86	13.27	89.36	1.08	-0.00	12.89
○Thread 59890			12.10	13.27	91.13	1.08	-0.00	12.89
○Thread 59891			12.02	13.27	90.53	1.08	-0.00	12.89
○Thread 59892			11.97	13.27	90.19	1.08	-0.00	12.89
○Thread 59893			12.02	13.27	90.57	1.08	-0.00	12.89
○Thread 59894			12.02	13.27	90.57	1.08	-0.00	12.90
○Thread 59895			11.96	13.27	90.08	1.08	0.00	12.89
○Thread 59896			12.10	13.27	91.13	1.08	-0.00	12.88
○Thread 59897			12.10	13.27	91.13	1.08	0.00	12.89
○Thread 59898			12.07	13.27	90.91	1.08	0.00	12.91
○Thread 59899			12.09	13.27	91.10	1.08	0.00	12.89
○Thread 59900			11.98	13.27	90.23	1.08	0.00	12.89
○Thread 59901			12.12	13.27	91.29	1.08	0.00	12.89
○Thread 59902			11.97	13.27	90.20	1.08	0.00	12.91
○Thread 59903			11.91	13.27	89.74	1.08	0.00	12.89
○Thread 59904			12.00	13.27	90.42	1.08	0.00	12.88
○Thread 59905			12.03	13.27	90.65	1.08	0.00	12.89
○Thread 59906			11.88	13.27	89.52	1.08	0.00	12.91
○Thread 59907			11.98	13.27	90.24	1.08	0.00	12.89
○Thread 59908			12.00	13.27	90.43	1.08	-0.00	12.91

Report Configuration