OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	11.24
►Process 1139842+		1	11.24
○Thread 1139842			11.24	12.98	86.57	0.00	0.00	11.05

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	6.04
►Process 1139977+		2	6.04
○Thread 1139977			6.04	7.92	76.17	0.00	0.00	5.64
○Thread 1140044			5.31	5.64	94.03	2.28	0.00	5.35

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	3.53
►Process 1140064+		4	3.53
○Thread 1140064			3.53	5.33	66.32	0.00	0.00	3.06
○Thread 1140132			2.74	3.07	89.50	2.26	0.00	2.77
○Thread 1140133			2.71	3.07	88.35	2.26	0.00	2.76
○Thread 1140134			2.72	3.07	88.84	2.26	0.00	2.77

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	2.29
►Process 1140154+		8	2.29
○Thread 1140154			2.29	4.06	56.54	0.00	0.00	2.26
○Thread 1140221			1.43	1.79	80.37	2.27	0.00	1.50
○Thread 1140222			1.46	1.79	82.04	2.27	0.00	1.50
○Thread 1140223			1.47	1.79	82.60	2.27	0.00	1.48
○Thread 1140224			1.44	1.79	80.91	2.27	0.00	1.50
○Thread 1140225			1.44	1.79	80.62	2.27	0.00	1.49
○Thread 1140226			1.46	1.79	82.01	2.27	0.00	1.50
○Thread 1140227			1.47	1.79	82.29	2.27	0.00	1.50

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	1.70
►Process 1140248+		16	1.70
○Thread 1140248			1.70	3.43	49.49	0.00	0.00	2.26
○Thread 1140315			0.83	1.16	71.48	2.27	0.00	0.87
○Thread 1140316			0.80	1.16	68.89	2.27	0.00	0.87
○Thread 1140317			0.80	1.16	68.88	2.27	0.00	0.87
○Thread 1140318			0.86	1.16	73.61	2.27	0.00	0.87
○Thread 1140319			0.85	1.16	72.74	2.27	0.00	0.87
○Thread 1140320			0.82	1.16	71.01	2.27	0.00	0.87
○Thread 1140321			0.81	1.16	69.28	2.27	0.00	0.87
○Thread 1140322			0.82	1.16	70.99	2.27	0.00	0.87
○Thread 1140323			0.83	1.16	71.41	2.27	0.00	0.87
○Thread 1140324			0.83	1.16	71.83	2.27	0.00	0.87
○Thread 1140325			0.82	1.16	70.96	2.27	0.00	0.87
○Thread 1140326			0.81	1.16	70.09	2.27	0.00	0.86
○Thread 1140327			0.80	1.16	68.79	2.27	0.00	0.87
○Thread 1140328			0.85	1.16	72.65	2.27	0.00	0.87
○Thread 1140329			0.82	1.16	70.92	2.27	0.00	0.86

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	1.46
►Process 1140397+		24	1.46
○Thread 1140397			1.46	3.22	45.44	0.00	0.00	2.21
○Thread 1140464			0.67	1.00	67.13	2.22	0.01	0.71
○Thread 1140465			0.66	1.00	66.12	2.22	0.01	0.70
○Thread 1140466			0.68	1.00	67.61	2.22	0.01	0.70
○Thread 1140467			0.67	1.00	66.60	2.22	0.01	0.71
○Thread 1140468			0.69	1.00	68.59	2.22	0.01	0.71
○Thread 1140469			0.68	1.00	67.58	2.22	0.01	0.71
○Thread 1140470			0.68	1.00	67.57	2.22	0.01	0.71
○Thread 1140471			0.67	1.00	66.56	2.22	0.00	0.71
○Thread 1140472			0.64	1.00	64.55	2.22	0.00	0.71
○Thread 1140473			0.69	1.00	69.04	2.22	0.00	0.70
○Thread 1140474			0.69	1.00	69.02	2.22	0.00	0.71
○Thread 1140475			0.66	1.00	66.01	2.22	0.00	0.71
○Thread 1140476			0.67	1.00	66.50	2.22	0.00	0.71
○Thread 1140477			0.68	1.00	67.49	2.22	0.00	0.71
○Thread 1140478			0.67	1.00	66.48	2.22	0.00	0.71
○Thread 1140479			0.67	1.00	66.47	2.22	0.00	0.71
○Thread 1140480			0.65	1.00	64.96	2.22	0.00	0.70
○Thread 1140481			0.63	1.00	62.96	2.22	0.00	0.71
○Thread 1140482			0.68	1.00	67.44	2.22	0.00	0.71
○Thread 1140483			0.68	1.00	67.43	2.22	0.00	0.71
○Thread 1140484			0.66	1.00	65.92	2.22	0.00	0.71
○Thread 1140485			0.67	1.00	66.91	2.22	0.00	0.71
○Thread 1140486			0.63	1.00	62.91	2.22	0.00	0.70

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	1.36
►Process 1140507+		32	1.36
○Thread 1140507			1.36	3.18	42.96	0.00	0.00	2.25
○Thread 1140574			0.56	0.90	62.03	2.27	0.01	0.61
○Thread 1140575			0.56	0.90	62.02	2.27	0.01	0.60
○Thread 1140576			0.56	0.90	61.46	2.27	0.01	0.61
○Thread 1140577			0.58	0.90	64.22	2.27	0.01	0.61
○Thread 1140578			0.53	0.90	58.67	2.27	0.01	0.61
○Thread 1140579			0.59	0.90	65.30	2.27	0.01	0.61
○Thread 1140580			0.54	0.90	59.21	2.27	0.01	0.61
○Thread 1140581			0.56	0.90	62.52	2.27	0.01	0.61
○Thread 1140582			0.55	0.90	60.85	2.27	0.01	0.61
○Thread 1140583			0.55	0.90	60.28	2.27	0.01	0.61
○Thread 1140584			0.54	0.90	59.72	2.27	0.01	0.61
○Thread 1140585			0.56	0.90	62.47	2.27	0.01	0.61
○Thread 1140586			0.57	0.90	63.57	2.27	0.01	0.61
○Thread 1140587			0.56	0.90	61.90	2.27	0.01	0.61
○Thread 1140588			0.55	0.90	60.78	2.27	0.01	0.61
○Thread 1140589			0.58	0.91	64.08	2.27	0.00	0.61
○Thread 1140590			0.54	0.91	59.10	2.27	0.00	0.61
○Thread 1140591			0.56	0.91	61.85	2.27	0.00	0.61
○Thread 1140592			0.58	0.91	64.05	2.27	0.00	0.61
○Thread 1140593			0.56	0.91	62.38	2.27	0.00	0.61
○Thread 1140594			0.56	0.91	61.82	2.27	0.00	0.61
○Thread 1140595			0.58	0.91	64.02	2.27	0.00	0.61
○Thread 1140596			0.55	0.91	60.70	2.27	0.00	0.60
○Thread 1140597			0.56	0.91	62.34	2.27	0.00	0.61
○Thread 1140598			0.55	0.91	60.12	2.27	0.00	0.61
○Thread 1140599			0.58	0.91	63.97	2.27	0.00	0.61
○Thread 1140600			0.55	0.91	60.65	2.27	0.00	0.60
○Thread 1140601			0.56	0.91	61.75	2.27	0.00	0.61
○Thread 1140602			0.58	0.91	64.49	2.27	0.00	0.61
○Thread 1140603			0.56	0.91	62.28	2.27	0.00	0.61
○Thread 1140604			0.55	0.91	60.61	2.27	0.00	0.59

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	1.21
►Process 1140624+		40	1.21
○Thread 1140624			1.21	3.13	38.86	0.00	0.00	2.18
○Thread 1140691			0.51	0.85	60.69	2.27	0.01	0.56
○Thread 1140692			0.52	0.85	61.27	2.27	0.01	0.56
○Thread 1140693			0.50	0.85	58.91	2.27	0.01	0.56
○Thread 1140694			0.50	0.85	59.49	2.27	0.01	0.56
○Thread 1140695			0.50	0.85	59.48	2.27	0.01	0.55
○Thread 1140696			0.49	0.85	57.11	2.27	0.01	0.56
○Thread 1140697			0.50	0.85	58.28	2.27	0.01	0.56
○Thread 1140698			0.50	0.85	59.45	2.27	0.01	0.56
○Thread 1140699			0.50	0.85	59.44	2.27	0.01	0.55
○Thread 1140700			0.49	0.85	57.07	2.27	0.01	0.55
○Thread 1140701			0.52	0.85	61.18	2.27	0.01	0.56
○Thread 1140702			0.54	0.85	63.52	2.27	0.01	0.56
○Thread 1140703			0.52	0.85	61.74	2.27	0.01	0.56
○Thread 1140704			0.49	0.85	57.62	2.27	0.01	0.55
○Thread 1140705			0.52	0.85	61.13	2.27	0.01	0.56
○Thread 1140706			0.49	0.85	57.59	2.27	0.01	0.56
○Thread 1140707			0.50	0.85	58.17	2.27	0.01	0.55
○Thread 1140708			0.52	0.85	61.69	2.27	0.01	0.55
○Thread 1140709			0.51	0.85	60.50	2.27	0.01	0.56
○Thread 1140710			0.49	0.85	57.55	2.27	0.01	0.55
○Thread 1140711			0.52	0.85	61.07	2.27	0.01	0.56
○Thread 1140712			0.51	0.85	60.47	2.27	0.01	0.56
○Thread 1140713			0.52	0.85	61.04	2.27	0.01	0.56
○Thread 1140714			0.51	0.85	59.86	2.27	0.00	0.56
○Thread 1140715			0.51	0.85	60.43	2.27	0.00	0.54
○Thread 1140716			0.52	0.85	61.60	2.27	0.00	0.55
○Thread 1140717			0.50	0.85	58.65	2.27	0.00	0.55
○Thread 1140718			0.53	0.85	62.16	2.27	0.00	0.56
○Thread 1140719			0.50	0.85	58.64	2.27	0.00	0.56
○Thread 1140720			0.50	0.85	58.04	2.27	0.00	0.55
○Thread 1140721			0.55	0.85	63.89	2.27	0.00	0.56
○Thread 1140722			0.55	0.85	64.47	2.27	0.00	0.56
○Thread 1140723			0.51	0.85	60.35	2.27	0.00	0.56
○Thread 1140724			0.51	0.85	59.76	2.27	0.00	0.56
○Thread 1140725			0.50	0.85	58.57	2.27	0.00	0.55
○Thread 1140726			0.50	0.85	59.15	2.27	0.00	0.56
○Thread 1140727			0.54	0.85	62.65	2.27	0.00	0.54
○Thread 1140728			0.51	0.85	60.29	2.27	0.00	0.55
○Thread 1140729			0.50	0.85	57.94	2.27	0.00	0.56

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	1.24
►Process 1140750+		48	1.24
○Thread 1140750			1.24	3.10	40.21	0.00	0.00	2.23
○Thread 1140817			0.46	0.81	56.78	2.27	0.01	0.51
○Thread 1140818			0.48	0.81	59.24	2.27	0.01	0.49
○Thread 1140819			0.47	0.81	58.00	2.27	0.01	0.51
○Thread 1140820			0.47	0.81	58.60	2.27	0.01	0.51
○Thread 1140821			0.48	0.81	59.21	2.27	0.01	0.52
○Thread 1140822			0.49	0.81	59.81	2.27	0.01	0.52
○Thread 1140823			0.44	0.81	53.64	2.27	0.01	0.52
○Thread 1140824			0.47	0.81	57.32	2.27	0.01	0.52
○Thread 1140825			0.48	0.81	59.16	2.27	0.01	0.51
○Thread 1140826			0.44	0.81	54.84	2.27	0.01	0.52
○Thread 1140827			0.47	0.81	57.29	2.27	0.01	0.52
○Thread 1140828			0.46	0.81	56.66	2.27	0.01	0.51
○Thread 1140829			0.46	0.81	56.65	2.27	0.01	0.52
○Thread 1140830			0.47	0.81	57.26	2.27	0.01	0.51
○Thread 1140831			0.46	0.81	56.63	2.27	0.01	0.51
○Thread 1140832			0.49	0.81	60.31	2.27	0.01	0.52
○Thread 1140833			0.46	0.81	56.61	2.27	0.01	0.51
○Thread 1140834			0.48	0.81	59.06	2.27	0.01	0.52
○Thread 1140835			0.47	0.81	57.82	2.28	0.01	0.52
○Thread 1140836			0.49	0.81	59.65	2.28	0.01	0.51
○Thread 1140837			0.46	0.81	56.57	2.28	0.01	0.51
○Thread 1140838			0.44	0.81	54.71	2.28	0.01	0.51
○Thread 1140839			0.45	0.81	55.32	2.28	0.01	0.51
○Thread 1140840			0.44	0.81	53.46	2.28	0.01	0.51
○Thread 1140841			0.46	0.81	55.91	2.28	0.01	0.51
○Thread 1140842			0.47	0.81	57.74	2.28	0.01	0.52
○Thread 1140843			0.50	0.81	60.80	2.28	0.01	0.51
○Thread 1140844			0.47	0.81	57.72	2.28	0.01	0.52
○Thread 1140845			0.47	0.81	58.32	2.28	0.01	0.52
○Thread 1140846			0.46	0.81	55.86	2.28	0.01	0.52
○Thread 1140847			0.47	0.81	57.08	2.28	0.01	0.52
○Thread 1140848			0.47	0.81	58.29	2.28	0.00	0.52
○Thread 1140849			0.46	0.82	56.44	2.28	0.00	0.52
○Thread 1140850			0.44	0.82	53.98	2.28	0.00	0.51
○Thread 1140851			0.46	0.82	56.42	2.28	0.00	0.52
○Thread 1140852			0.44	0.82	53.96	2.28	0.00	0.48
○Thread 1140853			0.47	0.82	57.62	2.28	0.00	0.52
○Thread 1140854			0.46	0.82	55.77	2.28	0.00	0.51
○Thread 1140855			0.46	0.82	55.76	2.28	0.00	0.51
○Thread 1140856			0.48	0.82	58.82	2.28	0.00	0.51
○Thread 1140857			0.49	0.82	60.03	2.28	0.00	0.51
○Thread 1140858			0.43	0.82	52.67	2.28	0.00	0.52
○Thread 1140859			0.48	0.82	58.78	2.28	0.00	0.51
○Thread 1140860			0.46	0.82	56.32	2.28	0.00	0.52
○Thread 1140861			0.46	0.82	56.31	2.28	0.00	0.51
○Thread 1140862			0.44	0.82	53.85	2.28	0.00	0.52
○Thread 1140863			0.43	0.82	52.62	2.28	0.00	0.51

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	1.25
►Process 1140883+		56	1.25
○Thread 1140883			1.25	3.07	40.69	0.00	0.00	2.26
○Thread 1140950			0.43	0.78	54.97	2.27	0.02	0.49
○Thread 1140951			0.42	0.78	53.68	2.27	0.02	0.49
○Thread 1140952			0.45	0.78	57.51	2.27	0.01	0.49
○Thread 1140953			0.43	0.78	54.94	2.27	0.01	0.49
○Thread 1140954			0.44	0.78	55.57	2.27	0.01	0.49
○Thread 1140955			0.43	0.78	54.92	2.27	0.01	0.47
○Thread 1140956			0.44	0.78	56.82	2.28	0.01	0.49
○Thread 1140957			0.46	0.78	58.09	2.28	0.01	0.49
○Thread 1140958			0.44	0.78	56.17	2.28	0.01	0.49
○Thread 1140959			0.40	0.78	51.05	2.28	0.01	0.48
○Thread 1140960			0.44	0.78	56.15	2.28	0.01	0.49
○Thread 1140961			0.43	0.78	54.86	2.28	0.01	0.48
○Thread 1140962			0.44	0.78	56.12	2.28	0.01	0.48
○Thread 1140963			0.42	0.78	53.56	2.28	0.01	0.46
○Thread 1140964			0.43	0.78	54.83	2.28	0.01	0.49
○Thread 1140965			0.39	0.78	49.72	2.28	0.01	0.49
○Thread 1140966			0.42	0.78	53.53	2.28	0.01	0.48
○Thread 1140967			0.44	0.78	55.44	2.28	0.01	0.49
○Thread 1140968			0.45	0.78	57.34	2.28	0.01	0.49
○Thread 1140969			0.41	0.78	52.87	2.28	0.01	0.49
○Thread 1140970			0.41	0.79	52.86	2.28	0.01	0.48
○Thread 1140971			0.44	0.79	55.40	2.28	0.01	0.49
○Thread 1140972			0.46	0.79	57.93	2.28	0.01	0.49
○Thread 1140973			0.41	0.79	52.19	2.28	0.01	0.48
○Thread 1140974			0.43	0.79	54.73	2.28	0.01	0.49
○Thread 1140975			0.40	0.79	50.27	2.28	0.01	0.47
○Thread 1140976			0.42	0.79	53.44	2.28	0.01	0.48
○Thread 1140977			0.41	0.79	52.15	2.28	0.01	0.47
○Thread 1140978			0.41	0.79	52.78	2.28	0.01	0.48
○Thread 1140979			0.41	0.79	52.13	2.28	0.01	0.47
○Thread 1140980			0.44	0.79	55.30	2.28	0.01	0.49
○Thread 1140981			0.44	0.79	55.93	2.28	0.01	0.49
○Thread 1140982			0.40	0.79	50.83	2.28	0.01	0.47
○Thread 1140983			0.42	0.79	53.36	2.28	0.01	0.48
○Thread 1140984			0.41	0.79	52.72	2.28	0.01	0.49
○Thread 1140985			0.40	0.79	50.17	2.28	0.01	0.48
○Thread 1140986			0.41	0.79	52.70	2.28	0.01	0.49
○Thread 1140987			0.44	0.79	55.23	2.28	0.01	0.49
○Thread 1140988			0.43	0.79	54.58	2.28	0.01	0.49
○Thread 1140989			0.45	0.79	57.11	2.28	0.00	0.49
○Thread 1140990			0.44	0.79	55.83	2.28	0.00	0.48
○Thread 1140991			0.42	0.79	53.28	2.28	0.00	0.47
○Thread 1140992			0.46	0.79	57.71	2.28	0.00	0.49
○Thread 1140993			0.43	0.79	54.53	2.28	0.00	0.49
○Thread 1140994			0.43	0.79	53.89	2.28	0.00	0.48
○Thread 1140995			0.41	0.79	52.61	2.28	0.00	0.48
○Thread 1140996			0.45	0.79	57.03	2.28	0.00	0.48
○Thread 1140997			0.42	0.79	53.22	2.28	0.00	0.48
○Thread 1140998			0.42	0.79	53.21	2.28	0.00	0.49
○Thread 1140999			0.45	0.79	57.00	2.28	0.00	0.49
○Thread 1141000			0.43	0.79	53.82	2.28	0.00	0.49
○Thread 1141001			0.43	0.79	53.81	2.28	0.00	0.48
○Thread 1141002			0.41	0.79	51.27	2.28	0.00	0.49
○Thread 1141003			0.39	0.79	49.36	2.28	0.00	0.48
○Thread 1141004			0.44	0.79	55.05	2.28	0.00	0.49

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 1 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	1.17
►Process 1141073+		64	1.17
○Thread 1141073			1.17	3.04	38.55	0.00	0.00	2.24
○Thread 1141140			0.39	0.76	51.07	2.25	0.02	0.46
○Thread 1141141			0.40	0.76	51.71	2.25	0.02	0.46
○Thread 1141142			0.40	0.76	52.36	2.25	0.02	0.46
○Thread 1141143			0.42	0.76	54.96	2.25	0.02	0.46
○Thread 1141144			0.43	0.76	56.26	2.25	0.02	0.46
○Thread 1141145			0.40	0.76	52.33	2.25	0.02	0.46
○Thread 1141146			0.40	0.76	52.32	2.25	0.02	0.46
○Thread 1141147			0.41	0.76	53.62	2.26	0.02	0.46
○Thread 1141148			0.38	0.76	50.34	2.26	0.02	0.46
○Thread 1141149			0.42	0.76	54.90	2.26	0.02	0.47
○Thread 1141150			0.41	0.77	54.24	2.26	0.01	0.46
○Thread 1141151			0.38	0.77	49.00	2.26	0.01	0.47
○Thread 1141152			0.41	0.77	53.56	2.26	0.01	0.47
○Thread 1141153			0.38	0.77	49.63	2.26	0.01	0.47
○Thread 1141154			0.41	0.77	53.54	2.26	0.01	0.47
○Thread 1141155			0.38	0.77	50.27	2.26	0.01	0.44
○Thread 1141156			0.41	0.77	53.52	2.26	0.01	0.45
○Thread 1141157			0.41	0.77	52.86	2.26	0.01	0.46
○Thread 1141158			0.41	0.77	52.85	2.26	0.01	0.47
○Thread 1141159			0.40	0.77	51.54	2.26	0.01	0.46
○Thread 1141160			0.41	0.77	52.83	2.26	0.01	0.47
○Thread 1141161			0.41	0.77	53.47	2.26	0.01	0.47
○Thread 1141162			0.42	0.77	54.77	2.26	0.01	0.46
○Thread 1141163			0.41	0.77	54.10	2.26	0.01	0.46
○Thread 1141164			0.38	0.77	50.18	2.26	0.01	0.46
○Thread 1141165			0.43	0.77	55.38	2.26	0.01	0.46
○Thread 1141166			0.40	0.77	52.11	2.26	0.01	0.46
○Thread 1141167			0.41	0.77	53.41	2.26	0.01	0.46
○Thread 1141168			0.41	0.77	54.05	2.26	0.01	0.46
○Thread 1141169			0.38	0.77	49.48	2.26	0.01	0.46
○Thread 1141170			0.40	0.77	52.08	2.26	0.01	0.46
○Thread 1141171			0.38	0.77	50.12	2.26	0.01	0.46
○Thread 1141172			0.35	0.77	46.20	2.26	0.01	0.45
○Thread 1141173			0.38	0.77	50.09	2.26	0.01	0.47
○Thread 1141174			0.40	0.77	52.03	2.26	0.01	0.46
○Thread 1141175			0.38	0.77	48.77	2.26	0.01	0.46
○Thread 1141176			0.41	0.77	53.31	2.26	0.01	0.46
○Thread 1141177			0.40	0.77	51.35	2.26	0.01	0.45
○Thread 1141178			0.37	0.77	48.09	2.26	0.01	0.46
○Thread 1141179			0.40	0.77	51.33	2.26	0.01	0.46
○Thread 1141180			0.41	0.77	52.62	2.26	0.01	0.46
○Thread 1141181			0.39	0.77	50.66	2.26	0.01	0.46
○Thread 1141182			0.38	0.77	50.00	2.26	0.01	0.46
○Thread 1141183			0.39	0.77	50.64	2.26	0.01	0.46
○Thread 1141184			0.40	0.77	51.92	2.26	0.01	0.46
○Thread 1141185			0.38	0.77	49.97	2.26	0.01	0.46
○Thread 1141186			0.41	0.77	53.20	2.26	0.01	0.46
○Thread 1141187			0.40	0.77	51.89	2.26	0.00	0.46
○Thread 1141188			0.41	0.77	52.53	2.26	0.00	0.44
○Thread 1141189			0.40	0.77	51.22	2.26	0.00	0.46
○Thread 1141190			0.39	0.77	50.56	2.26	0.00	0.46
○Thread 1141191			0.38	0.77	49.26	2.26	0.00	0.46
○Thread 1141192			0.40	0.77	51.84	2.26	0.00	0.46
○Thread 1141193			0.38	0.77	49.88	2.26	0.00	0.46
○Thread 1141194			0.38	0.77	49.88	2.26	0.00	0.46
○Thread 1141195			0.40	0.77	51.81	2.26	0.00	0.46
○Thread 1141196			0.37	0.77	47.91	2.26	0.00	0.45
○Thread 1141197			0.40	0.77	51.78	2.26	0.00	0.46
○Thread 1141198			0.36	0.77	46.60	2.26	0.00	0.45
○Thread 1141199			0.39	0.77	50.47	2.26	0.00	0.45
○Thread 1141200			0.38	0.77	49.17	2.26	0.00	0.46
○Thread 1141201			0.40	0.77	51.10	2.26	0.00	0.46
○Thread 1141202			0.41	0.77	52.38	2.26	0.00	0.46

Report Configuration