OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	189.32
►Process 4116022+		1	189.32
○Thread 4116022			189.32	190.46	99.40	0.00	0.00	185.95

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	95.84
►Process 4116209+		2	95.84
○Thread 4116209			94.23	97.57	96.58	0.00	0.00	96.58
○Thread 4116276			95.84	96.58	99.23	0.99	0.00	96.40

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	49.64
►Process 4116789+		4	49.64
○Thread 4116789			47.41	51.36	92.31	0.00	0.00	50.35
○Thread 4116856			49.64	50.36	98.56	1.00	0.00	50.19
○Thread 4116857			49.64	50.36	98.56	1.00	0.00	50.18
○Thread 4116858			49.61	50.36	98.51	1.00	0.00	50.19

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	29.60
►Process 4118255+		8	29.60
○Thread 4118255			25.79	31.44	82.04	0.00	0.00	30.43
○Thread 4118322			29.53	30.44	97.00	0.99	0.00	30.26
○Thread 4118323			29.60	30.44	97.25	0.99	0.00	30.27
○Thread 4118324			29.46	30.44	96.79	0.99	0.00	30.27
○Thread 4118325			29.34	30.44	96.38	0.99	0.00	30.26
○Thread 4118326			29.51	30.44	96.94	0.99	0.00	30.26
○Thread 4118327			29.50	30.44	96.92	0.99	0.00	30.27
○Thread 4118328			29.57	30.44	97.15	0.99	0.00	30.27

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	20.47
►Process 4121444+		16	20.47
○Thread 4121444			16.55	22.47	73.62	0.00	0.00	21.48
○Thread 4121511			20.32	21.48	94.61	0.99	0.00	21.30
○Thread 4121512			20.31	21.48	94.59	0.99	0.00	21.30
○Thread 4121513			20.44	21.48	95.19	0.99	0.00	21.30
○Thread 4121514			20.40	21.48	95.01	0.99	0.00	21.30
○Thread 4121515			20.39	21.48	94.93	0.99	0.00	21.30
○Thread 4121516			20.46	21.48	95.28	0.99	0.00	21.30
○Thread 4121517			20.47	21.48	95.33	0.99	0.00	21.29
○Thread 4121518			20.40	21.48	94.98	0.99	0.00	21.30
○Thread 4121519			20.15	21.48	93.81	0.99	0.00	21.29
○Thread 4121520			20.35	21.48	94.77	0.99	0.00	21.30
○Thread 4121521			20.33	21.48	94.67	0.99	0.00	21.30
○Thread 4121522			20.40	21.48	94.98	0.99	0.00	21.30
○Thread 4121523			20.27	21.48	94.37	0.99	0.00	21.30
○Thread 4121524			20.35	21.48	94.74	0.99	0.00	21.30
○Thread 4121525			20.23	21.48	94.21	0.99	0.00	21.30

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	17.07
►Process 4128225+		24	17.07
○Thread 4128225			11.67	19.32	60.38	0.00	0.00	18.30
○Thread 4128292			17.05	18.32	93.05	0.99	0.01	18.14
○Thread 4128293			16.97	18.32	92.64	0.99	0.01	18.14
○Thread 4128294			17.02	18.32	92.89	0.99	0.01	18.13
○Thread 4128295			16.96	18.32	92.59	0.99	0.01	18.13
○Thread 4128296			16.86	18.32	92.04	0.99	0.01	18.14
○Thread 4128297			16.96	18.32	92.59	0.99	0.01	18.13
○Thread 4128298			16.93	18.32	92.39	0.99	0.01	18.12
○Thread 4128299			16.90	18.32	92.23	0.99	0.00	18.14
○Thread 4128300			17.07	18.32	93.18	0.99	0.00	18.13
○Thread 4128301			16.92	18.32	92.36	0.99	0.00	18.13
○Thread 4128302			16.99	18.32	92.72	0.99	0.00	18.13
○Thread 4128303			16.93	18.32	92.39	0.99	0.00	18.13
○Thread 4128304			16.91	18.32	92.31	1.00	0.00	18.13
○Thread 4128305			16.98	18.32	92.66	1.00	0.00	18.13
○Thread 4128306			16.80	18.32	91.70	1.00	0.00	18.14
○Thread 4128307			16.89	18.32	92.20	1.00	0.00	18.14
○Thread 4128308			16.83	18.32	91.87	1.00	0.00	18.14
○Thread 4128309			16.76	18.32	91.48	1.00	0.00	18.14
○Thread 4128310			16.83	18.32	91.87	1.00	0.00	18.14
○Thread 4128311			16.77	18.32	91.51	1.00	0.00	18.14
○Thread 4128312			16.66	18.32	90.94	1.00	0.00	18.14
○Thread 4128313			16.87	18.32	92.08	1.00	0.00	18.14
○Thread 4128314			16.79	18.32	91.64	1.00	0.00	18.14

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	15.57
►Process 4138501+		32	15.57
○Thread 4138501			8.91	18.04	49.39	0.00	0.00	17.02
○Thread 4138568			15.48	17.04	90.83	1.00	0.01	16.85
○Thread 4138569			15.45	17.04	90.66	1.00	0.01	16.86
○Thread 4138570			15.49	17.04	90.92	1.00	0.01	16.84
○Thread 4138571			15.45	17.04	90.66	1.00	0.01	16.85
○Thread 4138572			15.47	17.04	90.80	1.00	0.01	16.85
○Thread 4138573			15.47	17.04	90.77	1.00	0.01	16.84
○Thread 4138574			15.54	17.04	91.21	1.00	0.01	16.84
○Thread 4138575			15.37	17.04	90.21	1.00	0.01	16.84
○Thread 4138576			15.44	17.04	90.62	1.00	0.01	16.84
○Thread 4138577			15.37	17.04	90.21	1.00	0.01	16.84
○Thread 4138578			15.46	17.04	90.74	1.00	0.01	16.85
○Thread 4138579			15.45	17.04	90.68	1.00	0.01	16.86
○Thread 4138580			15.57	17.04	91.35	1.00	0.01	16.85
○Thread 4138581			15.46	17.04	90.71	1.00	0.01	16.85
○Thread 4138582			15.40	17.04	90.38	1.00	0.01	16.85
○Thread 4138583			15.36	17.04	90.15	1.00	0.00	16.84
○Thread 4138584			15.36	17.04	90.15	1.00	0.00	16.85
○Thread 4138585			15.41	17.04	90.44	1.00	0.00	16.85
○Thread 4138586			15.30	17.04	89.76	1.00	0.00	16.84
○Thread 4138587			15.43	17.04	90.52	1.00	0.00	16.86
○Thread 4138588			15.43	17.04	90.55	1.00	0.00	16.85
○Thread 4138589			15.51	17.04	90.99	1.00	0.00	16.85
○Thread 4138590			15.51	17.04	91.02	1.00	0.00	16.85
○Thread 4138591			15.54	17.04	91.17	1.00	0.00	16.86
○Thread 4138592			15.43	17.04	90.55	1.00	0.00	16.85
○Thread 4138593			15.43	17.04	90.52	1.00	0.00	16.86
○Thread 4138594			15.40	17.04	90.37	1.00	0.00	16.84
○Thread 4138595			15.41	17.04	90.43	1.00	0.00	16.85
○Thread 4138596			15.33	17.04	89.96	1.00	0.00	16.85
○Thread 4138597			15.37	17.04	90.19	1.00	0.00	16.85
○Thread 4138598			15.37	17.04	90.19	1.00	0.00	16.85

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	15.24
►Process 4152320+		40	15.24
○Thread 4152320			7.86	17.92	43.85	0.00	0.00	16.90
○Thread 4152387			15.13	16.92	89.39	0.99	0.01	16.73
○Thread 4152388			15.07	16.92	89.06	0.99	0.01	16.73
○Thread 4152389			15.11	16.92	89.27	0.99	0.01	16.73
○Thread 4152390			15.12	16.92	89.32	0.99	0.01	16.73
○Thread 4152391			15.10	16.92	89.20	0.99	0.01	16.73
○Thread 4152392			15.00	16.92	88.61	0.99	0.01	16.73
○Thread 4152393			15.11	16.92	89.29	0.99	0.01	16.73
○Thread 4152394			15.12	16.92	89.32	0.99	0.01	16.73
○Thread 4152395			15.23	16.92	89.97	0.99	0.01	16.73
○Thread 4152396			15.09	16.92	89.14	0.99	0.01	16.73
○Thread 4152397			15.01	16.92	88.70	0.99	0.01	16.73
○Thread 4152398			15.11	16.92	89.29	0.99	0.01	16.73
○Thread 4152399			15.10	16.92	89.20	0.99	0.01	16.73
○Thread 4152400			15.11	16.92	89.26	0.99	0.01	16.73
○Thread 4152401			15.11	16.92	89.26	0.99	0.01	16.73
○Thread 4152402			14.94	16.92	88.28	0.99	0.01	16.73
○Thread 4152403			15.04	16.92	88.84	0.99	0.01	16.73
○Thread 4152404			15.24	16.92	90.05	0.99	0.01	16.73
○Thread 4152405			15.03	16.92	88.78	0.99	0.01	16.73
○Thread 4152406			14.99	16.92	88.54	0.99	0.01	16.73
○Thread 4152407			14.98	16.92	88.48	0.99	0.01	16.73
○Thread 4152408			15.08	16.92	89.10	0.99	0.01	16.73
○Thread 4152409			15.19	16.92	89.75	0.99	0.01	16.73
○Thread 4152410			15.08	16.92	89.07	0.99	0.00	16.73
○Thread 4152411			15.03	16.92	88.80	0.99	0.00	16.73
○Thread 4152412			15.16	16.93	89.57	0.99	0.00	16.73
○Thread 4152413			15.13	16.93	89.36	0.99	0.00	16.73
○Thread 4152414			14.91	16.93	88.06	0.99	0.00	16.73
○Thread 4152415			15.01	16.93	88.65	0.99	0.00	16.73
○Thread 4152416			15.02	16.93	88.74	0.99	0.00	16.73
○Thread 4152417			15.08	16.93	89.07	0.99	0.00	16.73
○Thread 4152418			15.02	16.93	88.74	0.99	0.00	16.73
○Thread 4152419			15.02	16.93	88.74	0.99	0.00	16.73
○Thread 4152420			15.00	16.93	88.62	0.99	0.00	16.73
○Thread 4152421			15.10	16.93	89.21	0.99	0.00	16.73
○Thread 4152422			15.05	16.93	88.91	0.99	0.00	16.73
○Thread 4152423			15.06	16.93	88.94	0.99	0.00	16.73
○Thread 4152424			15.07	16.93	89.00	0.99	0.00	16.73
○Thread 4152425			14.95	16.93	88.32	0.99	0.00	16.73

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	15.07
►Process 4169732+		48	15.07
○Thread 4169732			6.26	17.98	34.79	0.00	0.00	16.95
○Thread 4169799			14.95	16.98	88.00	0.98	0.01	16.78
○Thread 4169800			14.80	16.98	87.11	0.98	0.01	16.78
○Thread 4169801			14.90	16.98	87.70	0.98	0.01	16.78
○Thread 4169802			14.95	16.98	87.99	0.98	0.01	16.78
○Thread 4169803			15.01	16.98	88.38	0.98	0.01	16.77
○Thread 4169804			15.03	16.98	88.49	0.98	0.01	16.78
○Thread 4169805			15.04	16.98	88.52	0.98	0.01	16.78
○Thread 4169806			14.92	16.98	87.81	0.98	0.01	16.78
○Thread 4169807			14.90	16.98	87.72	0.98	0.01	16.79
○Thread 4169808			14.82	16.99	87.22	0.98	0.01	16.78
○Thread 4169809			14.92	16.99	87.84	0.98	0.01	16.79
○Thread 4169810			14.82	16.99	87.25	0.98	0.01	16.78
○Thread 4169811			14.89	16.99	87.66	0.98	0.01	16.78
○Thread 4169812			15.03	16.99	88.46	0.98	0.01	16.78
○Thread 4169813			15.07	16.99	88.72	0.98	0.01	16.76
○Thread 4169814			14.79	16.99	87.04	0.98	0.01	16.79
○Thread 4169815			14.94	16.99	87.95	0.98	0.01	16.78
○Thread 4169816			14.82	16.99	87.22	0.98	0.01	16.77
○Thread 4169817			14.77	16.99	86.95	0.99	0.01	16.79
○Thread 4169818			14.87	16.99	87.54	0.99	0.01	16.78
○Thread 4169819			14.80	16.99	87.10	0.99	0.01	16.78
○Thread 4169820			14.82	16.99	87.21	0.99	0.01	16.79
○Thread 4169821			14.93	16.99	87.89	0.99	0.01	16.79
○Thread 4169822			14.81	16.99	87.15	0.99	0.01	16.79
○Thread 4169823			14.90	16.99	87.68	0.99	0.01	16.79
○Thread 4169824			14.91	16.99	87.74	0.99	0.01	16.78
○Thread 4169825			14.96	16.99	88.03	0.99	0.01	16.78
○Thread 4169826			15.01	16.99	88.36	0.99	0.01	16.78
○Thread 4169827			15.01	16.99	88.36	0.99	0.01	16.79
○Thread 4169828			14.99	16.99	88.24	0.99	0.01	16.77
○Thread 4169829			14.99	16.99	88.21	0.99	0.01	16.78
○Thread 4169830			14.86	16.99	87.44	0.99	0.00	16.79
○Thread 4169831			14.72	16.99	86.62	0.99	0.00	16.78
○Thread 4169832			14.77	16.99	86.91	0.99	0.00	16.78
○Thread 4169833			14.77	16.99	86.94	0.99	0.00	16.78
○Thread 4169834			14.92	16.99	87.79	0.99	0.00	16.79
○Thread 4169835			14.83	16.99	87.29	0.99	0.00	16.77
○Thread 4169836			14.76	16.99	86.88	0.99	0.00	16.76
○Thread 4169837			14.89	16.99	87.64	0.99	0.00	16.79
○Thread 4169838			14.79	16.99	87.02	0.99	0.00	16.79
○Thread 4169839			14.88	16.99	87.58	0.99	0.00	16.78
○Thread 4169840			14.68	16.99	86.37	0.99	0.00	16.79
○Thread 4169841			14.96	16.99	88.05	0.99	0.00	16.77
○Thread 4169842			14.91	16.99	87.73	0.99	0.00	16.79
○Thread 4169843			14.87	16.99	87.49	0.99	0.00	16.78
○Thread 4169844			14.83	16.99	87.25	0.99	0.00	16.76
○Thread 4169845			14.93	16.99	87.84	0.99	0.00	16.78

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	14.85
►Process 4190639+		56	14.85
○Thread 4190639			5.72	18.07	31.62	0.00	0.00	17.02
○Thread 4190706			14.78	17.06	86.59	0.99	0.02	16.86
○Thread 4190707			14.71	17.06	86.18	1.00	0.01	16.86
○Thread 4190708			14.70	17.06	86.15	1.00	0.01	16.86
○Thread 4190709			14.68	17.06	86.00	1.00	0.01	16.85
○Thread 4190710			14.71	17.06	86.18	1.00	0.01	16.88
○Thread 4190711			14.77	17.06	86.56	1.00	0.01	16.86
○Thread 4190712			14.59	17.06	85.50	1.00	0.01	16.85
○Thread 4190713			14.65	17.06	85.82	1.00	0.01	16.86
○Thread 4190714			14.73	17.06	86.29	1.00	0.01	16.87
○Thread 4190715			14.73	17.06	86.29	1.00	0.01	16.86
○Thread 4190716			14.83	17.06	86.87	1.00	0.01	16.86
○Thread 4190717			14.65	17.07	85.85	1.00	0.01	16.88
○Thread 4190718			14.73	17.07	86.32	1.00	0.01	16.86
○Thread 4190719			14.79	17.07	86.64	1.00	0.01	16.86
○Thread 4190720			14.78	17.07	86.58	1.00	0.01	16.87
○Thread 4190721			14.66	17.07	85.90	1.00	0.01	16.86
○Thread 4190722			14.70	17.07	86.14	1.00	0.01	16.85
○Thread 4190723			14.75	17.07	86.43	1.00	0.01	16.85
○Thread 4190724			14.71	17.07	86.19	1.00	0.01	16.86
○Thread 4190725			14.61	17.07	85.61	1.00	0.01	16.85
○Thread 4190726			14.75	17.07	86.40	1.00	0.01	16.87
○Thread 4190727			14.70	17.07	86.13	1.00	0.01	16.86
○Thread 4190728			14.80	17.07	86.72	1.00	0.01	16.86
○Thread 4190729			14.76	17.07	86.48	1.00	0.01	16.86
○Thread 4190730			14.59	17.07	85.49	1.00	0.01	16.86
○Thread 4190731			14.66	17.07	85.90	1.00	0.01	16.85
○Thread 4190732			14.81	17.07	86.78	1.00	0.01	16.85
○Thread 4190733			14.85	17.07	87.01	1.00	0.01	16.86
○Thread 4190734			14.53	17.07	85.13	1.00	0.01	16.86
○Thread 4190735			14.53	17.07	85.10	1.00	0.01	16.84
○Thread 4190736			14.70	17.07	86.13	1.00	0.01	16.86
○Thread 4190737			14.60	17.07	85.51	1.00	0.01	16.85
○Thread 4190738			14.61	17.07	85.60	1.00	0.01	16.86
○Thread 4190739			14.65	17.07	85.83	1.00	0.01	16.86
○Thread 4190740			14.76	17.07	86.48	1.00	0.01	16.86
○Thread 4190741			14.66	17.07	85.89	1.00	0.01	16.85
○Thread 4190742			14.58	17.07	85.39	1.00	0.01	16.86
○Thread 4190743			14.68	17.07	86.01	1.00	0.01	16.86
○Thread 4190744			14.79	17.07	86.65	1.00	0.01	16.86
○Thread 4190745			14.63	17.07	85.68	1.00	0.00	16.86
○Thread 4190746			14.72	17.07	86.24	1.00	0.00	16.85
○Thread 4190747			14.82	17.07	86.79	1.00	0.00	16.86
○Thread 4190748			14.67	17.07	85.94	1.00	0.00	16.86
○Thread 4190749			14.78	17.07	86.56	1.00	0.00	16.86
○Thread 4190750			14.75	17.07	86.41	1.00	0.00	16.86
○Thread 4190751			14.61	17.07	85.59	1.00	0.00	16.84
○Thread 4190752			14.48	17.07	84.83	1.00	0.00	16.86
○Thread 4190753			14.68	17.07	85.97	1.00	0.00	16.87
○Thread 4190754			14.53	17.07	85.09	1.00	0.00	16.85
○Thread 4190755			14.54	17.07	85.18	1.00	0.00	16.84
○Thread 4190756			14.68	17.07	85.97	1.00	0.00	16.86
○Thread 4190757			14.52	17.07	85.06	1.00	0.00	16.85
○Thread 4190758			14.77	17.07	86.52	1.00	0.00	16.86
○Thread 4190759			14.60	17.07	85.52	1.00	0.00	16.85
○Thread 4190760			14.56	17.07	85.29	1.00	0.00	16.84

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	19.67
►Process 21368+		64	19.67
○Thread 21368			5.10	24.15	21.10	0.00	0.00	23.08
○Thread 21435			19.41	23.12	83.97	1.01	0.02	22.90
○Thread 21436			19.50	23.12	84.34	1.01	0.02	22.90
○Thread 21437			19.45	23.12	84.12	1.01	0.02	22.89
○Thread 21438			19.52	23.12	84.43	1.01	0.02	22.90
○Thread 21439			19.47	23.12	84.23	1.01	0.02	22.89
○Thread 21440			19.44	23.12	84.08	1.01	0.02	22.90
○Thread 21441			19.44	23.12	84.08	1.01	0.02	22.89
○Thread 21442			19.46	23.12	84.19	1.01	0.02	22.89
○Thread 21443			19.31	23.12	83.51	1.01	0.02	22.90
○Thread 21444			19.59	23.12	84.75	1.01	0.02	22.89
○Thread 21445			19.24	23.12	83.23	1.01	0.01	22.90
○Thread 21446			19.33	23.12	83.62	1.01	0.01	22.90
○Thread 21447			19.38	23.12	83.84	1.01	0.01	22.90
○Thread 21448			19.61	23.12	84.81	1.01	0.01	22.89
○Thread 21449			19.33	23.12	83.62	1.01	0.01	22.90
○Thread 21450			19.19	23.12	83.01	1.01	0.01	22.89
○Thread 21451			19.29	23.12	83.42	1.01	0.01	22.89
○Thread 21452			19.30	23.12	83.47	1.01	0.01	22.91
○Thread 21453			19.34	23.12	83.64	1.01	0.01	22.90
○Thread 21454			19.27	23.12	83.34	1.01	0.01	22.89
○Thread 21455			19.25	23.12	83.27	1.01	0.01	22.89
○Thread 21456			19.43	23.12	84.05	1.01	0.01	22.90
○Thread 21457			19.46	23.12	84.16	1.01	0.01	22.89
○Thread 21458			19.67	23.12	85.06	1.01	0.01	22.89
○Thread 21459			19.44	23.12	84.09	1.01	0.01	22.90
○Thread 21460			19.44	23.12	84.09	1.01	0.01	22.89
○Thread 21461			19.30	23.12	83.46	1.01	0.01	22.90
○Thread 21462			19.40	23.12	83.91	1.01	0.01	22.89
○Thread 21463			19.42	23.12	83.98	1.01	0.01	22.90
○Thread 21464			19.31	23.12	83.52	1.01	0.01	22.90
○Thread 21465			19.49	23.13	84.28	1.01	0.01	22.90
○Thread 21466			19.35	23.13	83.67	1.01	0.01	22.90
○Thread 21467			19.55	23.13	84.56	1.01	0.01	22.90
○Thread 21468			19.55	23.13	84.54	1.01	0.01	22.90
○Thread 21469			19.50	23.13	84.32	1.01	0.01	22.90
○Thread 21470			19.36	23.13	83.72	1.01	0.01	22.89
○Thread 21471			19.33	23.13	83.59	1.01	0.01	22.90
○Thread 21472			19.25	23.13	83.26	1.01	0.01	22.89
○Thread 21473			19.55	23.13	84.56	1.01	0.01	22.89
○Thread 21474			19.13	23.13	82.72	1.01	0.01	22.90
○Thread 21475			19.27	23.13	83.32	1.01	0.01	22.90
○Thread 21476			19.47	23.13	84.21	1.01	0.01	22.89
○Thread 21477			19.42	23.13	83.97	1.01	0.01	22.90
○Thread 21478			19.41	23.13	83.93	1.01	0.01	22.89
○Thread 21479			19.33	23.13	83.60	1.01	0.01	22.90
○Thread 21480			19.29	23.13	83.41	1.01	0.01	22.90
○Thread 21481			19.11	23.13	82.63	1.01	0.01	22.90
○Thread 21482			19.51	23.13	84.38	1.01	0.00	22.89
○Thread 21483			19.34	23.13	83.64	1.01	0.00	22.89
○Thread 21484			19.35	23.13	83.69	1.01	0.00	22.90
○Thread 21485			19.30	23.13	83.45	1.01	0.00	22.90
○Thread 21486			19.28	23.13	83.38	1.01	0.00	22.90
○Thread 21487			19.30	23.13	83.47	1.01	0.00	22.90
○Thread 21488			19.32	23.13	83.55	1.02	0.00	22.89
○Thread 21489			19.40	23.13	83.88	1.02	0.00	22.89
○Thread 21490			19.41	23.13	83.94	1.02	0.00	22.89
○Thread 21491			19.18	23.13	82.93	1.02	0.00	22.89
○Thread 21492			19.36	23.13	83.70	1.02	0.00	22.90
○Thread 21493			19.40	23.13	83.88	1.02	0.00	22.90
○Thread 21494			19.37	23.13	83.75	1.02	0.00	22.89
○Thread 21495			19.32	23.13	83.53	1.02	0.00	22.91
○Thread 21496			19.42	23.13	83.98	1.02	0.00	22.88
○Thread 21497			19.35	23.13	83.68	1.02	0.00	22.90

Report Configuration