OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	20.20
►Process 1150691+		1	20.20
○Thread 1150691			20.20	22.13	91.29	0.00	0.00	13.69

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	10.64
►Process 1150778+		2	10.64
○Thread 1150778			10.64	12.47	85.34	0.00	0.00	10.19
○Thread 1150845			9.84	10.19	96.53	2.27	0.00	9.90

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	5.77
►Process 1150865+		4	5.77
○Thread 1150865			5.77	7.62	75.67	0.00	0.00	5.35
○Thread 1150932			5.01	5.36	93.45	2.26	0.00	5.07
○Thread 1150933			5.02	5.36	93.64	2.26	0.00	5.07
○Thread 1150934			5.04	5.36	94.01	2.26	0.00	5.07

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	3.43
►Process 1151004+		8	3.43
○Thread 1151004			3.43	5.20	65.94	0.00	0.00	2.93
○Thread 1151071			2.60	2.93	88.79	2.27	0.00	2.64
○Thread 1151072			2.59	2.93	88.45	2.27	0.00	2.64
○Thread 1151073			2.62	2.93	89.46	2.27	0.00	2.64
○Thread 1151074			2.62	2.93	89.29	2.27	0.00	2.64
○Thread 1151075			2.62	2.93	89.45	2.27	0.00	2.64
○Thread 1151076			2.61	2.93	89.11	2.27	0.00	2.64
○Thread 1151077			2.60	2.93	88.76	2.27	0.00	2.64

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	2.23
►Process 1151097+		16	2.23
○Thread 1151097			2.23	4.01	55.79	0.00	0.00	2.24
○Thread 1151165			1.41	1.73	81.39	2.27	0.00	1.44
○Thread 1151166			1.38	1.73	79.94	2.27	0.00	1.44
○Thread 1151167			1.39	1.73	80.22	2.27	0.00	1.44
○Thread 1151168			1.38	1.73	79.63	2.27	0.00	1.44
○Thread 1151169			1.41	1.73	81.36	2.27	0.00	1.44
○Thread 1151170			1.43	1.73	82.79	2.27	0.00	1.44
○Thread 1151171			1.37	1.73	79.32	2.27	0.00	1.44
○Thread 1151172			1.38	1.73	79.60	2.27	0.00	1.44
○Thread 1151173			1.37	1.73	79.31	2.27	0.00	1.44
○Thread 1151174			1.38	1.73	79.59	2.27	0.00	1.44
○Thread 1151175			1.39	1.73	80.45	2.27	0.00	1.44
○Thread 1151176			1.39	1.73	80.44	2.27	0.00	1.44
○Thread 1151177			1.35	1.73	77.84	2.27	0.00	1.44
○Thread 1151178			1.41	1.73	81.29	2.27	0.00	1.44
○Thread 1151179			1.39	1.73	80.42	2.27	0.00	1.44

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	1.90
►Process 1151199+		24	1.90
○Thread 1151199			1.90	3.71	51.35	0.00	0.00	2.24
○Thread 1151266			1.10	1.43	76.87	2.27	0.01	1.14
○Thread 1151267			1.08	1.43	75.81	2.27	0.01	1.13
○Thread 1151268			1.09	1.43	76.50	2.27	0.01	1.14
○Thread 1151269			1.12	1.43	78.24	2.27	0.01	1.14
○Thread 1151270			1.09	1.43	76.49	2.27	0.01	1.14
○Thread 1151271			1.08	1.43	75.08	2.27	0.01	1.13
○Thread 1151272			1.08	1.43	75.42	2.27	0.01	1.13
○Thread 1151273			1.09	1.43	76.11	2.27	0.00	1.13
○Thread 1151274			1.08	1.43	75.06	2.27	0.00	1.13
○Thread 1151275			1.08	1.43	75.40	2.27	0.00	1.14
○Thread 1151276			1.09	1.43	76.09	2.27	0.00	1.14
○Thread 1151277			1.11	1.43	77.47	2.27	0.00	1.14
○Thread 1151278			1.08	1.43	75.72	2.27	0.00	1.13
○Thread 1151279			1.10	1.43	76.76	2.27	0.00	1.14
○Thread 1151280			1.09	1.43	76.05	2.27	0.00	1.13
○Thread 1151281			1.08	1.43	75.35	2.27	0.00	1.14
○Thread 1151282			1.08	1.43	75.69	2.27	0.00	1.14
○Thread 1151283			1.09	1.43	76.38	2.27	0.00	1.14
○Thread 1151284			1.10	1.43	76.72	2.27	0.00	1.14
○Thread 1151285			1.08	1.43	75.66	2.27	0.00	1.13
○Thread 1151286			1.08	1.43	75.30	2.27	0.00	1.13
○Thread 1151287			1.09	1.43	76.34	2.27	0.00	1.14
○Thread 1151288			1.08	1.43	74.94	2.27	0.00	1.13

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	1.65
►Process 1151308+		32	1.65
○Thread 1151308			1.65	3.53	46.93	0.00	0.00	2.22
○Thread 1151375			0.92	1.25	73.24	2.27	0.01	0.96
○Thread 1151376			0.88	1.25	70.83	2.27	0.01	0.96
○Thread 1151377			0.94	1.25	75.62	2.27	0.01	0.96
○Thread 1151378			0.91	1.25	72.81	2.27	0.01	0.96
○Thread 1151379			0.92	1.25	73.20	2.27	0.01	0.95
○Thread 1151380			0.89	1.25	71.19	2.27	0.01	0.96
○Thread 1151381			0.94	1.25	75.18	2.27	0.01	0.95
○Thread 1151382			0.90	1.25	71.97	2.27	0.01	0.96
○Thread 1151383			0.92	1.25	73.16	2.27	0.01	0.95
○Thread 1151384			0.91	1.25	72.75	2.27	0.01	0.96
○Thread 1151385			0.92	1.25	73.55	2.27	0.01	0.96
○Thread 1151386			0.93	1.25	74.34	2.27	0.01	0.96
○Thread 1151387			0.90	1.25	72.33	2.27	0.01	0.95
○Thread 1151388			0.92	1.25	73.12	2.27	0.01	0.96
○Thread 1151389			0.93	1.25	73.91	2.27	0.01	0.95
○Thread 1151390			0.89	1.25	71.50	2.27	0.00	0.96
○Thread 1151391			0.92	1.25	73.49	2.27	0.00	0.96
○Thread 1151392			0.90	1.25	72.28	2.27	0.00	0.96
○Thread 1151393			0.90	1.25	72.27	2.27	0.00	0.96
○Thread 1151394			0.89	1.25	71.06	2.27	0.00	0.96
○Thread 1151395			0.94	1.25	75.44	2.27	0.00	0.96
○Thread 1151396			0.91	1.25	72.64	2.27	0.00	0.96
○Thread 1151397			0.93	1.25	73.83	2.27	0.00	0.96
○Thread 1151398			0.92	1.25	73.02	2.27	0.00	0.96
○Thread 1151399			0.91	1.25	72.61	2.27	0.00	0.94
○Thread 1151400			0.90	1.25	71.81	2.27	0.00	0.96
○Thread 1151401			0.93	1.25	73.79	2.27	0.00	0.96
○Thread 1151402			0.93	1.25	73.78	2.27	0.00	0.96
○Thread 1151403			0.89	1.25	71.38	2.27	0.00	0.95
○Thread 1151404			0.88	1.25	69.77	2.27	0.00	0.94
○Thread 1151405			0.93	1.25	73.75	2.27	0.00	0.96

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	1.59
►Process 1151425+		40	1.59
○Thread 1151425			1.59	3.44	46.27	0.00	0.00	2.27
○Thread 1151492			0.81	1.14	70.77	2.28	0.01	0.85
○Thread 1151493			0.81	1.14	70.76	2.28	0.01	0.85
○Thread 1151494			0.81	1.14	70.75	2.28	0.01	0.85
○Thread 1151495			0.78	1.14	68.12	2.28	0.01	0.85
○Thread 1151496			0.81	1.15	70.73	2.28	0.01	0.85
○Thread 1151497			0.81	1.15	70.72	2.28	0.01	0.84
○Thread 1151498			0.83	1.15	72.46	2.28	0.01	0.85
○Thread 1151499			0.81	1.15	70.70	2.28	0.01	0.85
○Thread 1151500			0.81	1.15	70.26	2.28	0.01	0.85
○Thread 1151501			0.79	1.15	68.94	2.28	0.01	0.85
○Thread 1151502			0.81	1.15	70.68	2.28	0.01	0.85
○Thread 1151503			0.80	1.15	69.36	2.28	0.01	0.85
○Thread 1151504			0.78	1.15	68.04	2.28	0.01	0.83
○Thread 1151505			0.82	1.15	71.95	2.28	0.01	0.85
○Thread 1151506			0.80	1.15	69.76	2.28	0.01	0.85
○Thread 1151507			0.81	1.15	71.06	2.28	0.01	0.84
○Thread 1151508			0.80	1.15	69.31	2.28	0.01	0.85
○Thread 1151509			0.81	1.15	70.17	2.28	0.01	0.85
○Thread 1151510			0.82	1.15	71.47	2.28	0.01	0.85
○Thread 1151511			0.81	1.15	70.15	2.28	0.01	0.85
○Thread 1151512			0.82	1.15	71.45	2.28	0.01	0.85
○Thread 1151513			0.80	1.15	69.70	2.28	0.01	0.85
○Thread 1151514			0.81	1.15	70.56	2.28	0.01	0.84
○Thread 1151515			0.80	1.15	69.24	2.28	0.00	0.85
○Thread 1151516			0.81	1.15	70.54	2.28	0.00	0.84
○Thread 1151517			0.79	1.15	68.79	2.28	0.00	0.85
○Thread 1151518			0.81	1.15	70.09	2.28	0.00	0.85
○Thread 1151519			0.82	1.15	71.39	2.28	0.00	0.85
○Thread 1151520			0.80	1.15	69.64	2.28	0.00	0.85
○Thread 1151521			0.80	1.15	69.63	2.28	0.00	0.85
○Thread 1151522			0.81	1.15	70.05	2.28	0.00	0.85
○Thread 1151523			0.80	1.15	69.17	2.28	0.00	0.85
○Thread 1151524			0.80	1.15	69.16	2.28	0.00	0.84
○Thread 1151525			0.82	1.15	71.33	2.28	0.00	0.85
○Thread 1151526			0.79	1.15	68.71	2.28	0.00	0.84
○Thread 1151527			0.80	1.15	69.14	2.28	0.00	0.85
○Thread 1151528			0.81	1.15	70.43	2.28	0.00	0.85
○Thread 1151529			0.80	1.15	69.12	2.29	0.00	0.83
○Thread 1151530			0.73	1.15	63.46	2.29	0.00	0.85

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	1.52
►Process 1151598+		48	1.52
○Thread 1151598			1.52	3.36	45.17	0.00	0.00	2.21
○Thread 1151665			0.70	1.07	65.43	2.28	0.01	0.77
○Thread 1151666			0.71	1.07	66.82	2.28	0.01	0.77
○Thread 1151667			0.70	1.07	65.88	2.28	0.01	0.77
○Thread 1151668			0.74	1.07	68.67	2.28	0.01	0.76
○Thread 1151669			0.73	1.07	67.73	2.28	0.01	0.77
○Thread 1151670			0.73	1.07	67.72	2.28	0.01	0.77
○Thread 1151671			0.70	1.07	65.84	2.28	0.01	0.77
○Thread 1151672			0.71	1.07	66.77	2.28	0.01	0.77
○Thread 1151673			0.71	1.07	66.76	2.28	0.01	0.77
○Thread 1151674			0.68	1.07	63.48	2.28	0.01	0.77
○Thread 1151675			0.73	1.07	67.67	2.28	0.01	0.77
○Thread 1151676			0.70	1.07	65.80	2.28	0.01	0.77
○Thread 1151677			0.68	1.07	63.45	2.28	0.01	0.76
○Thread 1151678			0.73	1.07	67.64	2.28	0.01	0.77
○Thread 1151679			0.70	1.07	65.77	2.28	0.01	0.76
○Thread 1151680			0.73	1.07	67.62	2.28	0.01	0.77
○Thread 1151681			0.71	1.07	66.68	2.28	0.01	0.77
○Thread 1151682			0.70	1.07	65.27	2.28	0.01	0.77
○Thread 1151683			0.69	1.07	64.80	2.28	0.01	0.77
○Thread 1151684			0.68	1.07	63.39	2.28	0.01	0.77
○Thread 1151685			0.71	1.07	66.64	2.28	0.01	0.77
○Thread 1151686			0.73	1.07	68.03	2.28	0.01	0.77
○Thread 1151687			0.74	1.07	68.49	2.28	0.01	0.77
○Thread 1151688			0.73	1.07	68.01	2.28	0.01	0.77
○Thread 1151689			0.72	1.07	67.07	2.28	0.01	0.77
○Thread 1151690			0.69	1.07	64.27	2.28	0.01	0.77
○Thread 1151691			0.72	1.07	67.05	2.28	0.01	0.77
○Thread 1151692			0.71	1.07	66.58	2.28	0.01	0.76
○Thread 1151693			0.73	1.07	67.50	2.29	0.01	0.77
○Thread 1151694			0.70	1.07	65.16	2.29	0.01	0.77
○Thread 1151695			0.72	1.07	67.01	2.29	0.01	0.77
○Thread 1151696			0.72	1.07	67.00	2.29	0.00	0.77
○Thread 1151697			0.71	1.07	66.53	2.29	0.00	0.77
○Thread 1151698			0.70	1.07	65.59	2.29	0.00	0.77
○Thread 1151699			0.71	1.08	66.04	2.29	0.00	0.76
○Thread 1151700			0.73	1.08	67.89	2.29	0.00	0.77
○Thread 1151701			0.73	1.08	67.42	2.29	0.00	0.77
○Thread 1151702			0.70	1.08	65.09	2.29	0.00	0.77
○Thread 1151703			0.71	1.08	66.01	2.29	0.00	0.77
○Thread 1151704			0.70	1.08	65.53	2.29	0.00	0.77
○Thread 1151705			0.71	1.08	66.45	2.29	0.00	0.77
○Thread 1151706			0.74	1.08	68.31	2.29	0.00	0.77
○Thread 1151707			0.69	1.08	64.12	2.29	0.00	0.77
○Thread 1151708			0.69	1.08	64.57	2.29	0.00	0.76
○Thread 1151709			0.71	1.08	66.42	2.29	0.00	0.77
○Thread 1151710			0.71	1.08	65.95	2.29	0.00	0.76
○Thread 1151711			0.63	1.08	58.51	2.29	0.00	0.75

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	1.49
►Process 1151731+		56	1.49
○Thread 1151731			1.49	3.30	45.25	0.00	0.00	2.26
○Thread 1151798			0.66	1.01	65.05	2.27	0.02	0.72
○Thread 1151799			0.63	1.01	62.58	2.27	0.02	0.70
○Thread 1151800			0.65	1.01	64.54	2.27	0.01	0.71
○Thread 1151801			0.63	1.02	62.07	2.27	0.01	0.72
○Thread 1151802			0.63	1.02	62.06	2.27	0.01	0.72
○Thread 1151803			0.65	1.02	64.02	2.27	0.01	0.70
○Thread 1151804			0.64	1.02	63.02	2.27	0.01	0.72
○Thread 1151805			0.64	1.02	63.51	2.27	0.01	0.71
○Thread 1151806			0.65	1.02	63.99	2.27	0.01	0.72
○Thread 1151807			0.65	1.02	63.98	2.27	0.01	0.71
○Thread 1151808			0.64	1.02	62.98	2.28	0.01	0.71
○Thread 1151809			0.64	1.02	62.97	2.28	0.01	0.71
○Thread 1151810			0.68	1.02	66.41	2.28	0.01	0.72
○Thread 1151811			0.61	1.02	60.00	2.28	0.01	0.71
○Thread 1151812			0.66	1.02	64.91	2.28	0.01	0.72
○Thread 1151813			0.68	1.02	66.38	2.28	0.01	0.72
○Thread 1151814			0.65	1.02	64.40	2.28	0.01	0.72
○Thread 1151815			0.66	1.02	64.88	2.28	0.01	0.71
○Thread 1151816			0.66	1.02	64.87	2.28	0.01	0.72
○Thread 1151817			0.65	1.02	63.88	2.28	0.01	0.72
○Thread 1151818			0.66	1.02	64.85	2.28	0.01	0.70
○Thread 1151819			0.67	1.02	65.33	2.28	0.01	0.72
○Thread 1151820			0.63	1.02	62.37	2.28	0.01	0.71
○Thread 1151821			0.67	1.02	65.31	2.28	0.01	0.71
○Thread 1151822			0.61	1.02	59.41	2.28	0.01	0.72
○Thread 1151823			0.67	1.02	65.78	2.28	0.01	0.72
○Thread 1151824			0.63	1.02	61.84	2.28	0.01	0.72
○Thread 1151825			0.65	1.02	63.80	2.28	0.01	0.72
○Thread 1151826			0.65	1.02	64.28	2.28	0.01	0.71
○Thread 1151827			0.67	1.02	65.25	2.28	0.01	0.72
○Thread 1151828			0.63	1.02	61.80	2.28	0.01	0.70
○Thread 1151829			0.64	1.02	62.77	2.28	0.01	0.72
○Thread 1151830			0.63	1.02	61.78	2.28	0.01	0.71
○Thread 1151831			0.67	1.02	65.69	2.28	0.01	0.72
○Thread 1151832			0.63	1.02	61.76	2.28	0.01	0.71
○Thread 1151833			0.67	1.02	65.18	2.28	0.01	0.71
○Thread 1151834			0.63	1.02	62.23	2.28	0.01	0.70
○Thread 1151835			0.65	1.02	64.18	2.28	0.01	0.71
○Thread 1151836			0.65	1.02	63.68	2.28	0.01	0.71
○Thread 1151837			0.66	1.02	64.65	2.28	0.00	0.72
○Thread 1151838			0.63	1.02	62.19	2.28	0.00	0.71
○Thread 1151839			0.63	1.02	61.69	2.28	0.00	0.71
○Thread 1151840			0.63	1.02	62.17	2.28	0.00	0.71
○Thread 1151841			0.68	1.02	66.08	2.28	0.00	0.71
○Thread 1151842			0.63	1.02	61.67	2.28	0.00	0.71
○Thread 1151843			0.65	1.02	63.61	2.28	0.00	0.71
○Thread 1151844			0.63	1.02	62.14	2.28	0.00	0.71
○Thread 1151845			0.63	1.02	61.15	2.28	0.00	0.72
○Thread 1151846			0.63	1.02	61.63	2.28	0.00	0.71
○Thread 1151847			0.68	1.02	66.51	2.28	0.00	0.72
○Thread 1151848			0.65	1.02	64.06	2.28	0.00	0.72
○Thread 1151849			0.64	1.02	62.58	2.28	0.00	0.71
○Thread 1151850			0.64	1.02	63.06	2.28	0.00	0.71
○Thread 1151851			0.63	1.02	61.58	2.28	0.00	0.70
○Thread 1151852			0.58	1.02	57.17	2.28	0.00	0.71

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	1.45
►Process 1151872+		64	1.45
○Thread 1151872			1.45	3.30	44.00	0.00	0.00	3.28
○Thread 1151939			0.60	0.97	61.89	2.31	0.02	0.66
○Thread 1151940			0.58	0.97	60.34	2.31	0.02	0.67
○Thread 1151941			0.61	0.97	62.91	2.31	0.02	0.66
○Thread 1151942			0.61	0.97	62.39	2.31	0.02	0.66
○Thread 1151943			0.58	0.97	60.31	2.31	0.02	0.67
○Thread 1151944			0.63	0.97	64.43	2.31	0.02	0.67
○Thread 1151945			0.58	0.97	59.78	2.31	0.02	0.67
○Thread 1151946			0.62	0.97	63.37	2.31	0.02	0.67
○Thread 1151947			0.62	0.97	63.36	2.31	0.02	0.67
○Thread 1151948			0.60	0.97	61.29	2.31	0.02	0.67
○Thread 1151949			0.61	0.97	62.83	2.31	0.01	0.67
○Thread 1151950			0.59	0.97	60.76	2.31	0.01	0.67
○Thread 1151951			0.63	0.97	65.38	2.31	0.01	0.67
○Thread 1151952			0.59	0.97	60.74	2.31	0.01	0.67
○Thread 1151953			0.63	0.97	64.33	2.31	0.01	0.67
○Thread 1151954			0.59	0.97	60.72	2.31	0.01	0.67
○Thread 1151955			0.60	0.97	61.74	2.31	0.01	0.66
○Thread 1151956			0.62	0.97	63.27	2.31	0.01	0.67
○Thread 1151957			0.64	0.97	65.83	2.31	0.01	0.67
○Thread 1151958			0.62	0.97	63.76	2.31	0.01	0.66
○Thread 1151959			0.60	0.97	61.69	2.31	0.01	0.67
○Thread 1151960			0.58	0.97	60.14	2.31	0.01	0.67
○Thread 1151961			0.60	0.97	61.16	2.31	0.01	0.67
○Thread 1151962			0.63	0.97	65.26	2.31	0.01	0.67
○Thread 1151963			0.58	0.97	59.60	2.31	0.01	0.67
○Thread 1151964			0.61	0.97	62.67	2.31	0.01	0.67
○Thread 1151965			0.62	0.97	63.69	2.31	0.01	0.67
○Thread 1151966			0.58	0.97	60.09	2.31	0.01	0.65
○Thread 1151967			0.56	0.97	57.00	2.31	0.01	0.66
○Thread 1151968			0.62	0.97	63.66	2.31	0.01	0.67
○Thread 1151969			0.61	0.97	62.63	2.31	0.01	0.67
○Thread 1151970			0.58	0.97	60.05	2.31	0.01	0.67
○Thread 1151971			0.61	0.97	62.61	2.31	0.01	0.67
○Thread 1151972			0.62	0.97	63.11	2.31	0.01	0.67
○Thread 1151973			0.61	0.97	62.07	2.31	0.01	0.67
○Thread 1151974			0.63	0.97	64.11	2.31	0.01	0.67
○Thread 1151975			0.63	0.98	64.10	2.31	0.01	0.66
○Thread 1151976			0.61	0.98	62.04	2.31	0.01	0.67
○Thread 1151977			0.58	0.98	59.47	2.31	0.01	0.65
○Thread 1151978			0.60	0.98	61.51	2.31	0.01	0.67
○Thread 1151979			0.63	0.98	64.57	2.31	0.01	0.67
○Thread 1151980			0.62	0.98	63.54	2.31	0.01	0.67
○Thread 1151981			0.58	0.98	59.43	2.31	0.01	0.65
○Thread 1151982			0.62	0.98	63.00	2.31	0.01	0.67
○Thread 1151983			0.61	0.98	62.48	2.31	0.01	0.67
○Thread 1151984			0.61	0.98	61.96	2.31	0.01	0.67
○Thread 1151985			0.60	0.98	60.92	2.31	0.01	0.67
○Thread 1151986			0.60	0.98	60.91	2.31	0.00	0.67
○Thread 1151987			0.57	0.98	58.86	2.31	0.00	0.65
○Thread 1151988			0.63	0.98	64.48	2.31	0.00	0.67
○Thread 1151989			0.61	0.98	62.42	2.31	0.00	0.66
○Thread 1151990			0.59	0.98	60.36	2.31	0.00	0.67
○Thread 1151991			0.60	0.98	60.86	2.31	0.00	0.67
○Thread 1151992			0.57	0.98	58.81	2.31	0.00	0.65
○Thread 1151993			0.60	0.98	60.84	2.31	0.00	0.66
○Thread 1151994			0.61	0.98	62.37	2.31	0.00	0.67
○Thread 1151995			0.62	0.98	63.38	2.32	0.00	0.67
○Thread 1151996			0.58	0.98	59.28	2.32	0.00	0.67
○Thread 1151997			0.62	0.98	62.85	2.32	0.00	0.67
○Thread 1151998			0.62	0.98	63.35	2.32	0.00	0.67
○Thread 1151999			0.58	0.98	59.25	2.32	0.00	0.66
○Thread 1152000			0.57	0.98	58.73	2.32	0.00	0.66
○Thread 1152001			0.61	0.98	62.30	2.32	0.00	0.67

Report Configuration