OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	96.65
►Process 4051774+		1	96.65
○Thread 4051774			96.65	97.75	98.87	0.00	0.00	93.24

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	48.92
►Process 4051910+		2	48.92
○Thread 4051910			48.10	50.39	95.46	0.00	0.00	49.40
○Thread 4051977			48.92	49.41	99.01	0.98	0.00	49.23

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	25.39
►Process 4052270+		4	25.39
○Thread 4052270			24.17	26.88	89.93	0.00	0.00	25.87
○Thread 4052337			25.39	25.88	98.11	0.99	0.00	25.70
○Thread 4052338			25.35	25.88	97.95	0.99	0.00	25.70
○Thread 4052339			25.39	25.88	98.10	0.99	0.00	25.70

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	15.22
►Process 4053023+		8	15.22
○Thread 4053023			13.44	16.71	80.38	0.00	0.00	15.72
○Thread 4053090			15.15	15.73	96.27	0.98	0.00	15.56
○Thread 4053091			15.22	15.73	96.71	0.98	0.00	15.56
○Thread 4053092			15.11	15.73	96.04	0.98	0.00	15.53
○Thread 4053093			15.16	15.73	96.36	0.98	0.00	15.56
○Thread 4053094			15.12	15.73	96.07	0.98	0.00	15.55
○Thread 4053095			15.19	15.73	96.52	0.98	0.00	15.56
○Thread 4053096			15.12	15.73	96.07	0.98	0.00	15.56

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	10.44
►Process 4054714+		16	10.44
○Thread 4054714			8.51	12.08	70.42	0.00	0.00	11.06
○Thread 4054781			10.42	11.08	94.00	0.99	0.00	10.90
○Thread 4054782			10.25	11.08	92.46	0.99	0.00	10.90
○Thread 4054783			10.40	11.08	93.86	0.99	0.00	10.90
○Thread 4054784			10.42	11.08	93.99	0.99	0.00	10.90
○Thread 4054785			10.39	11.08	93.76	0.99	0.00	10.90
○Thread 4054786			10.43	11.08	94.08	0.99	0.00	10.90
○Thread 4054787			10.43	11.08	94.12	0.99	0.00	10.90
○Thread 4054788			10.38	11.08	93.63	0.99	0.00	10.90
○Thread 4054789			10.41	11.08	93.89	0.99	0.00	10.90
○Thread 4054790			10.35	11.08	93.35	0.99	0.00	10.90
○Thread 4054791			10.39	11.08	93.71	0.99	0.00	10.90
○Thread 4054792			10.37	11.08	93.53	0.99	0.00	10.90
○Thread 4054793			10.42	11.08	93.98	0.99	0.00	10.90
○Thread 4054794			10.36	11.08	93.44	0.99	0.00	10.88
○Thread 4054795			10.44	11.08	94.16	1.00	0.00	10.90

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	8.78
►Process 4058130+		24	8.78
○Thread 4058130			6.15	10.49	58.60	0.00	0.00	9.47
○Thread 4058197			8.65	9.48	91.18	1.00	0.01	9.30
○Thread 4058198			8.72	9.48	91.92	1.00	0.01	9.30
○Thread 4058199			8.56	9.48	90.28	1.00	0.01	9.30
○Thread 4058200			8.69	9.48	91.65	1.00	0.01	9.30
○Thread 4058201			8.69	9.48	91.60	1.00	0.01	9.30
○Thread 4058202			8.77	9.48	92.49	1.00	0.01	9.30
○Thread 4058203			8.70	9.48	91.75	1.00	0.01	9.30
○Thread 4058204			8.78	9.48	92.54	1.00	0.00	9.27
○Thread 4058205			8.73	9.48	92.07	1.00	0.00	9.30
○Thread 4058206			8.71	9.48	91.80	1.00	0.00	9.30
○Thread 4058207			8.68	9.48	91.48	1.00	0.00	9.29
○Thread 4058208			8.61	9.48	90.80	1.00	0.00	9.30
○Thread 4058209			8.68	9.48	91.48	1.00	0.00	9.30
○Thread 4058210			8.71	9.48	91.79	1.00	0.00	9.30
○Thread 4058211			8.70	9.48	91.69	1.00	0.00	9.29
○Thread 4058212			8.61	9.48	90.74	1.00	0.00	9.30
○Thread 4058213			8.65	9.48	91.21	1.00	0.00	9.30
○Thread 4058214			8.65	9.48	91.16	1.00	0.00	9.30
○Thread 4058215			8.75	9.48	92.26	1.00	0.00	9.30
○Thread 4058216			8.72	9.48	91.94	1.00	0.00	9.30
○Thread 4058217			8.60	9.48	90.63	1.00	0.00	9.30
○Thread 4058218			8.66	9.48	91.26	1.00	0.00	9.30
○Thread 4058219			8.56	9.48	90.25	1.00	0.00	9.30

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	7.95
►Process 4063322+		32	7.95
○Thread 4063322			4.69	9.80	47.85	0.00	0.00	8.78
○Thread 4063389			7.93	8.79	90.13	1.00	0.01	8.61
○Thread 4063390			7.95	8.79	90.42	1.00	0.01	8.60
○Thread 4063391			7.92	8.79	90.07	1.00	0.01	8.61
○Thread 4063392			7.92	8.79	90.07	1.00	0.01	8.61
○Thread 4063393			7.92	8.79	90.07	1.00	0.01	8.60
○Thread 4063394			7.90	8.79	89.84	1.00	0.01	8.60
○Thread 4063395			7.93	8.79	90.18	1.00	0.01	8.61
○Thread 4063396			7.84	8.79	89.16	1.00	0.01	8.60
○Thread 4063397			7.80	8.79	88.64	1.00	0.01	8.60
○Thread 4063398			7.78	8.79	88.47	1.00	0.01	8.60
○Thread 4063399			7.78	8.79	88.47	1.00	0.01	8.60
○Thread 4063400			7.88	8.79	89.55	1.00	0.01	8.59
○Thread 4063401			7.92	8.79	90.00	1.00	0.01	8.59
○Thread 4063402			7.84	8.79	89.15	1.00	0.01	8.60
○Thread 4063403			7.92	8.79	90.06	1.00	0.01	8.61
○Thread 4063404			7.84	8.79	89.20	1.00	0.00	8.60
○Thread 4063405			7.80	8.79	88.63	1.00	0.00	8.61
○Thread 4063406			7.90	8.79	89.77	1.00	0.00	8.60
○Thread 4063407			7.90	8.80	89.77	1.00	0.00	8.60
○Thread 4063408			7.85	8.80	89.25	1.00	0.00	8.60
○Thread 4063409			7.86	8.80	89.37	1.00	0.00	8.60
○Thread 4063410			7.80	8.80	88.62	1.00	0.00	8.58
○Thread 4063411			7.94	8.80	90.27	1.00	0.00	8.61
○Thread 4063412			7.89	8.80	89.65	1.00	0.00	8.60
○Thread 4063413			7.82	8.80	88.85	1.00	0.00	8.60
○Thread 4063414			7.88	8.80	89.59	1.00	0.00	8.60
○Thread 4063415			7.95	8.80	90.38	1.00	0.00	8.60
○Thread 4063416			7.75	8.80	88.10	1.00	0.00	8.60
○Thread 4063417			7.79	8.80	88.50	1.00	0.00	8.60
○Thread 4063418			7.80	8.80	88.61	1.00	0.00	8.61
○Thread 4063419			7.93	8.80	90.09	1.00	0.00	8.60

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	7.84
►Process 4070339+		40	7.84
○Thread 4070339			4.14	9.73	42.53	0.00	0.00	8.73
○Thread 4070406			7.80	8.74	89.22	0.98	0.01	8.55
○Thread 4070407			7.70	8.74	88.07	0.98	0.01	8.54
○Thread 4070408			7.73	8.74	88.41	0.98	0.01	8.54
○Thread 4070409			7.73	8.74	88.41	0.98	0.01	8.55
○Thread 4070410			7.81	8.74	89.32	0.98	0.01	8.55
○Thread 4070411			7.77	8.74	88.81	0.98	0.01	8.54
○Thread 4070412			7.84	8.74	89.66	0.98	0.01	8.54
○Thread 4070413			7.74	8.74	88.52	0.98	0.01	8.54
○Thread 4070414			7.71	8.74	88.17	0.98	0.01	8.55
○Thread 4070415			7.69	8.74	87.89	0.98	0.01	8.73
○Thread 4070416			7.62	8.74	87.08	0.98	0.01	8.56
○Thread 4070417			7.62	8.74	87.08	0.98	0.01	8.54
○Thread 4070418			7.75	8.74	88.57	0.98	0.01	8.54
○Thread 4070419			7.77	8.74	88.85	0.98	0.01	8.55
○Thread 4070420			7.66	8.75	87.59	0.98	0.01	8.54
○Thread 4070421			7.79	8.75	89.08	0.98	0.01	8.56
○Thread 4070422			7.65	8.75	87.47	0.98	0.01	8.55
○Thread 4070423			7.81	8.75	89.25	0.98	0.01	8.56
○Thread 4070424			7.66	8.75	87.59	0.98	0.01	8.55
○Thread 4070425			7.74	8.75	88.50	0.98	0.01	8.55
○Thread 4070426			7.69	8.75	87.93	0.98	0.01	8.54
○Thread 4070427			7.69	8.75	87.93	0.98	0.01	8.55
○Thread 4070428			7.70	8.75	87.98	0.98	0.01	8.56
○Thread 4070429			7.67	8.75	87.64	0.98	0.00	8.55
○Thread 4070430			7.62	8.75	87.06	0.98	0.00	8.56
○Thread 4070431			7.73	8.75	88.38	0.98	0.00	8.55
○Thread 4070432			7.71	8.75	88.09	0.98	0.00	8.56
○Thread 4070433			7.59	8.75	86.77	0.98	0.00	8.54
○Thread 4070434			7.73	8.75	88.31	0.98	0.00	8.54
○Thread 4070435			7.61	8.75	87.00	0.98	0.00	8.55
○Thread 4070436			7.65	8.75	87.45	0.98	0.00	8.55
○Thread 4070437			7.56	8.75	86.42	0.98	0.00	8.54
○Thread 4070438			7.69	8.75	87.91	0.98	0.00	8.56
○Thread 4070439			7.73	8.75	88.36	0.98	0.00	8.55
○Thread 4070440			7.68	8.75	87.73	0.98	0.00	8.55
○Thread 4070441			7.62	8.75	87.05	0.98	0.00	8.55
○Thread 4070442			7.72	8.75	88.19	0.98	0.00	8.56
○Thread 4070443			7.68	8.75	87.79	0.98	0.00	8.55
○Thread 4070444			7.65	8.75	87.44	0.98	0.00	8.56

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	7.78
►Process 4079083+		48	7.78
○Thread 4079083			3.32	9.84	33.74	0.00	0.00	8.78
○Thread 4079150			7.69	8.83	87.09	1.00	0.01	8.82
○Thread 4079151			7.78	8.83	88.05	1.00	0.01	8.64
○Thread 4079152			7.68	8.83	86.97	1.00	0.01	8.62
○Thread 4079153			7.54	8.83	85.38	1.00	0.01	8.64
○Thread 4079154			7.64	8.83	86.52	1.00	0.01	8.64
○Thread 4079155			7.72	8.83	87.42	1.00	0.01	8.64
○Thread 4079156			7.74	8.83	87.64	1.00	0.01	8.62
○Thread 4079157			7.66	8.83	86.74	1.00	0.01	8.63
○Thread 4079158			7.71	8.83	87.30	1.00	0.01	8.62
○Thread 4079159			7.68	8.83	86.90	1.00	0.01	8.63
○Thread 4079160			7.67	8.83	86.79	1.00	0.01	8.61
○Thread 4079161			7.72	8.83	87.35	1.00	0.01	8.62
○Thread 4079162			7.68	8.83	86.90	1.00	0.01	8.63
○Thread 4079163			7.65	8.83	86.62	1.00	0.01	8.64
○Thread 4079164			7.67	8.83	86.84	1.00	0.01	8.63
○Thread 4079165			7.70	8.83	87.18	1.00	0.01	8.64
○Thread 4079166			7.68	8.83	86.95	1.00	0.01	8.61
○Thread 4079167			7.60	8.83	86.04	1.00	0.01	8.62
○Thread 4079168			7.70	8.83	87.17	1.00	0.01	8.63
○Thread 4079169			7.74	8.83	87.57	1.00	0.01	8.62
○Thread 4079170			7.61	8.83	86.15	1.00	0.01	8.62
○Thread 4079171			7.61	8.83	86.21	1.00	0.01	8.62
○Thread 4079172			7.54	8.83	85.36	1.00	0.01	8.63
○Thread 4079173			7.65	8.83	86.60	1.00	0.01	8.61
○Thread 4079174			7.63	8.83	86.32	1.00	0.01	8.62
○Thread 4079175			7.63	8.83	86.31	1.00	0.01	8.62
○Thread 4079176			7.58	8.83	85.80	1.00	0.01	8.62
○Thread 4079177			7.71	8.83	87.22	1.00	0.01	8.64
○Thread 4079178			7.68	8.83	86.88	1.00	0.01	8.64
○Thread 4079179			7.75	8.83	87.67	1.00	0.01	8.62
○Thread 4079180			7.58	8.83	85.80	1.00	0.01	8.61
○Thread 4079181			7.61	8.83	86.08	1.00	0.00	8.63
○Thread 4079182			7.57	8.83	85.74	1.00	0.00	8.62
○Thread 4079183			7.64	8.84	86.53	1.00	0.00	8.63
○Thread 4079184			7.58	8.84	85.79	1.00	0.00	8.64
○Thread 4079185			7.59	8.84	85.85	1.00	0.00	8.62
○Thread 4079186			7.72	8.84	87.37	1.00	0.00	8.62
○Thread 4079187			7.57	8.84	85.68	1.00	0.00	8.61
○Thread 4079188			7.53	8.84	85.22	1.00	0.00	8.62
○Thread 4079189			7.58	8.84	85.73	1.00	0.00	8.63
○Thread 4079190			7.69	8.84	87.09	1.00	0.00	8.64
○Thread 4079191			7.65	8.84	86.58	1.00	0.00	8.62
○Thread 4079192			7.58	8.84	85.78	1.00	0.00	8.63
○Thread 4079193			7.66	8.84	86.69	1.00	0.00	8.62
○Thread 4079194			7.68	8.84	86.91	1.00	0.00	8.62
○Thread 4079195			7.63	8.84	86.34	1.00	0.00	8.64
○Thread 4079196			7.68	8.84	86.85	1.00	0.00	8.63

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	7.70
►Process 4089603+		56	7.70
○Thread 4089603			3.10	9.89	31.40	0.00	0.00	8.82
○Thread 4089670			7.50	8.85	84.75	1.02	0.02	8.65
○Thread 4089671			7.52	8.85	84.97	1.02	0.02	8.64
○Thread 4089672			7.60	8.85	85.82	1.02	0.01	8.65
○Thread 4089673			7.57	8.85	85.54	1.02	0.01	8.65
○Thread 4089674			7.61	8.85	86.04	1.02	0.01	8.65
○Thread 4089675			7.60	8.85	85.87	1.02	0.01	8.65
○Thread 4089676			7.54	8.85	85.25	1.02	0.01	8.65
○Thread 4089677			7.52	8.85	84.97	1.02	0.01	8.65
○Thread 4089678			7.55	8.85	85.30	1.02	0.01	8.65
○Thread 4089679			7.49	8.85	84.68	1.02	0.01	8.65
○Thread 4089680			7.65	8.85	86.37	1.02	0.01	8.65
○Thread 4089681			7.63	8.85	86.15	1.02	0.01	8.65
○Thread 4089682			7.52	8.85	84.90	1.02	0.01	8.65
○Thread 4089683			7.54	8.85	85.18	1.02	0.01	8.64
○Thread 4089684			7.62	8.85	86.03	1.02	0.01	8.65
○Thread 4089685			7.47	8.85	84.45	1.02	0.01	8.65
○Thread 4089686			7.61	8.85	85.91	1.02	0.01	8.64
○Thread 4089687			7.53	8.85	85.01	1.02	0.01	8.65
○Thread 4089688			7.58	8.85	85.63	1.02	0.01	8.64
○Thread 4089689			7.48	8.85	84.44	1.03	0.01	8.65
○Thread 4089690			7.66	8.85	86.53	1.03	0.01	8.65
○Thread 4089691			7.66	8.85	86.47	1.03	0.01	8.65
○Thread 4089692			7.70	8.85	86.92	1.03	0.01	8.64
○Thread 4089693			7.47	8.85	84.43	1.03	0.01	8.65
○Thread 4089694			7.48	8.85	84.55	1.03	0.01	8.65
○Thread 4089695			7.37	8.85	83.25	1.03	0.01	8.64
○Thread 4089696			7.58	8.85	85.62	1.03	0.01	8.65
○Thread 4089697			7.40	8.85	83.53	1.03	0.01	8.63
○Thread 4089698			7.63	8.85	86.18	1.03	0.01	8.65
○Thread 4089699			7.50	8.85	84.71	1.03	0.01	8.64
○Thread 4089700			7.63	8.85	86.12	1.03	0.01	8.64
○Thread 4089701			7.61	8.85	85.89	1.03	0.01	8.65
○Thread 4089702			7.46	8.85	84.25	1.03	0.01	8.64
○Thread 4089703			7.56	8.85	85.32	1.03	0.01	8.65
○Thread 4089704			7.57	8.85	85.44	1.03	0.01	8.64
○Thread 4089705			7.54	8.85	85.10	1.03	0.01	8.65
○Thread 4089706			7.47	8.85	84.36	1.03	0.01	8.65
○Thread 4089707			7.45	8.86	84.13	1.03	0.01	8.65
○Thread 4089708			7.54	8.86	85.20	1.03	0.01	8.65
○Thread 4089709			7.55	8.86	85.20	1.03	0.00	8.65
○Thread 4089710			7.43	8.86	83.90	1.03	0.00	8.65
○Thread 4089711			7.60	8.86	85.82	1.03	0.00	8.65
○Thread 4089712			7.62	8.86	86.05	1.03	0.00	8.65
○Thread 4089713			7.47	8.86	84.35	1.03	0.00	8.65
○Thread 4089714			7.51	8.86	84.75	1.03	0.00	8.65
○Thread 4089715			7.50	8.86	84.69	1.03	0.00	8.64
○Thread 4089716			7.55	8.86	85.25	1.03	0.00	8.65
○Thread 4089717			7.41	8.86	83.72	1.03	0.00	8.65
○Thread 4089718			7.55	8.86	85.30	1.03	0.00	8.65
○Thread 4089719			7.61	8.86	85.87	1.03	0.00	8.65
○Thread 4089720			7.46	8.86	84.23	1.03	0.00	8.65
○Thread 4089721			7.48	8.86	84.45	1.03	0.00	8.65
○Thread 4089722			7.59	8.86	85.64	1.03	0.00	8.65
○Thread 4089723			7.50	8.86	84.62	1.03	0.00	8.65
○Thread 4089724			7.59	8.86	85.69	1.03	0.00	8.65

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	9.94
►Process 4101947+		64	9.94
○Thread 4101947			2.83	12.76	22.18	0.00	0.00	11.72
○Thread 4102014			9.75	11.74	83.03	1.00	0.02	11.51
○Thread 4102015			9.85	11.74	83.92	1.00	0.02	11.53
○Thread 4102016			9.82	11.74	83.66	1.00	0.02	11.50
○Thread 4102017			9.70	11.74	82.64	1.00	0.02	11.52
○Thread 4102018			9.69	11.74	82.51	1.00	0.02	11.51
○Thread 4102019			9.85	11.74	83.87	1.00	0.02	11.51
○Thread 4102020			9.72	11.74	82.76	1.00	0.02	11.54
○Thread 4102021			9.86	11.74	83.95	1.00	0.02	11.52
○Thread 4102022			9.84	11.74	83.83	1.00	0.02	11.51
○Thread 4102023			9.93	11.74	84.59	1.00	0.02	11.52
○Thread 4102024			9.74	11.74	82.97	1.00	0.02	11.51
○Thread 4102025			9.93	11.74	84.55	1.00	0.02	11.52
○Thread 4102026			9.61	11.74	81.86	1.00	0.01	11.51
○Thread 4102027			9.76	11.74	83.14	1.00	0.01	11.51
○Thread 4102028			9.76	11.74	83.14	1.01	0.01	11.50
○Thread 4102029			9.67	11.74	82.33	1.01	0.01	11.52
○Thread 4102030			9.74	11.74	82.96	1.01	0.01	11.55
○Thread 4102031			9.68	11.74	82.45	1.01	0.01	11.52
○Thread 4102032			9.84	11.74	83.81	1.01	0.01	11.51
○Thread 4102033			9.77	11.74	83.17	1.01	0.01	11.51
○Thread 4102034			9.87	11.74	84.02	1.01	0.01	11.52
○Thread 4102035			9.74	11.74	82.96	1.01	0.01	11.51
○Thread 4102036			9.64	11.74	82.11	1.01	0.01	11.51
○Thread 4102037			9.74	11.74	82.91	1.01	0.01	11.52
○Thread 4102038			9.69	11.74	82.53	1.01	0.01	11.50
○Thread 4102039			9.74	11.74	82.91	1.01	0.01	11.51
○Thread 4102040			9.59	11.74	81.68	1.01	0.01	11.51
○Thread 4102041			9.89	11.74	84.19	1.01	0.01	11.52
○Thread 4102042			9.69	11.74	82.48	1.01	0.01	11.52
○Thread 4102043			9.83	11.74	83.67	1.01	0.01	11.52
○Thread 4102044			9.79	11.74	83.33	1.01	0.01	11.51
○Thread 4102045			9.81	11.74	83.54	1.01	0.01	11.51
○Thread 4102046			9.76	11.74	83.12	1.01	0.01	11.51
○Thread 4102047			9.87	11.74	84.01	1.01	0.01	11.52
○Thread 4102048			9.83	11.74	83.71	1.01	0.01	11.51
○Thread 4102049			9.80	11.74	83.41	1.01	0.01	11.51
○Thread 4102050			9.56	11.74	81.37	1.01	0.01	11.51
○Thread 4102051			9.69	11.74	82.47	1.01	0.01	11.51
○Thread 4102052			9.74	11.74	82.90	1.01	0.01	11.52
○Thread 4102053			9.85	11.74	83.88	1.01	0.01	11.53
○Thread 4102054			9.70	11.74	82.60	1.01	0.01	11.51
○Thread 4102055			9.72	11.74	82.77	1.01	0.01	11.55
○Thread 4102056			9.70	11.74	82.55	1.01	0.01	11.52
○Thread 4102057			9.76	11.74	83.10	1.01	0.01	11.52
○Thread 4102058			9.76	11.74	83.06	1.01	0.01	11.51
○Thread 4102059			9.86	11.74	83.91	1.01	0.01	11.51
○Thread 4102060			9.90	11.74	84.29	1.01	0.01	11.52
○Thread 4102061			9.71	11.74	82.72	1.01	0.01	11.51
○Thread 4102062			9.68	11.75	82.42	1.01	0.00	11.55
○Thread 4102063			9.73	11.75	82.84	1.01	0.00	11.52
○Thread 4102064			9.77	11.75	83.14	1.01	0.00	11.50
○Thread 4102065			9.71	11.75	82.71	1.01	0.00	11.52
○Thread 4102066			9.70	11.75	82.58	1.01	0.00	11.55
○Thread 4102067			9.61	11.75	81.82	1.01	0.00	11.52
○Thread 4102068			9.79	11.75	83.35	1.01	0.00	11.52
○Thread 4102069			9.69	11.75	82.45	1.01	0.00	11.53
○Thread 4102070			9.70	11.75	82.54	1.01	0.00	11.51
○Thread 4102071			9.70	11.75	82.58	1.01	0.00	11.52
○Thread 4102072			9.73	11.75	82.79	1.01	0.00	11.50
○Thread 4102073			9.72	11.75	82.75	1.01	0.00	11.52
○Thread 4102074			9.82	11.75	83.55	1.01	0.00	11.53
○Thread 4102075			9.94	11.75	84.57	1.01	0.00	11.51
○Thread 4102076			9.67	11.75	82.27	1.01	0.00	11.52

Report Configuration