OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	437.10
►Process 287912+		1	437.10
○Thread 287912			437.10	438.11	99.77	0.00	0.00	438.07

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	219.10
►Process 288237+		2	219.10
○Thread 288237			219.10	220.03	99.58	0.00	0.00	219.99
○Thread 288336			218.73	219.13	99.82	0.90	-0.00	218.81

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	110.34
►Process 288458+		4	110.34
○Thread 288458			110.34	111.28	99.15	0.00	0.00	111.23
○Thread 288557			110.02	110.38	99.67	0.90	-0.00	110.07
○Thread 288558			110.00	110.38	99.65	0.90	-0.00	110.07
○Thread 288559			109.99	110.38	99.64	0.90	-0.00	110.07

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	56.07
►Process 288632+		8	56.07
○Thread 288632			56.07	57.00	98.37	0.00	0.00	56.96
○Thread 288731			55.73	56.11	99.32	0.89	-0.00	55.79
○Thread 288732			55.73	56.11	99.32	0.89	-0.00	55.79
○Thread 288733			55.74	56.11	99.34	0.89	-0.00	55.79
○Thread 288734			55.73	56.11	99.31	0.89	-0.00	55.79
○Thread 288735			55.75	56.11	99.35	0.89	0.00	55.79
○Thread 288736			55.73	56.11	99.32	0.89	0.00	55.79
○Thread 288737			55.70	56.11	99.26	0.89	-0.00	55.79

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	29.11
►Process 288805+		16	29.11
○Thread 288805			29.11	30.10	96.73	0.00	0.00	30.05
○Thread 288904			28.80	29.20	98.63	0.90	-0.00	28.88
○Thread 288905			28.67	29.20	98.17	0.90	-0.00	28.88
○Thread 288906			28.75	29.20	98.45	0.90	-0.00	28.88
○Thread 288907			28.65	29.20	98.12	0.90	-0.00	28.87
○Thread 288908			28.74	29.20	98.41	0.90	-0.00	28.88
○Thread 288909			28.64	29.20	98.09	0.90	-0.00	28.88
○Thread 288910			28.74	29.20	98.41	0.90	-0.00	28.88
○Thread 288911			28.69	29.20	98.24	0.90	-0.00	28.88
○Thread 288912			28.68	29.20	98.23	0.90	-0.00	28.88
○Thread 288913			28.67	29.20	98.19	0.90	-0.00	28.88
○Thread 288914			28.65	29.20	98.12	0.90	-0.00	28.88
○Thread 288915			28.66	29.20	98.16	0.90	-0.00	28.88
○Thread 288916			28.69	29.20	98.24	0.90	0.00	28.88
○Thread 288917			28.78	29.20	98.55	0.90	0.00	28.88
○Thread 288918			28.72	29.20	98.36	0.90	-0.00	28.88

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	22.54
►Process 288938+		24	22.54
○Thread 288938			22.54	23.68	95.22	0.00	0.00	23.62
○Thread 289037			22.30	22.78	97.91	0.90	-0.00	22.46
○Thread 289038			22.12	22.78	97.14	0.90	-0.00	22.45
○Thread 289039			22.15	22.78	97.25	0.90	-0.00	22.45
○Thread 289040			22.20	22.78	97.49	0.90	-0.00	22.46
○Thread 289041			22.20	22.78	97.49	0.90	-0.00	22.45
○Thread 289042			22.17	22.78	97.36	0.90	-0.00	22.45
○Thread 289043			22.19	22.78	97.45	0.90	-0.00	22.45
○Thread 289044			22.21	22.78	97.54	0.90	-0.00	22.45
○Thread 289045			22.22	22.78	97.58	0.90	-0.00	22.45
○Thread 289046			22.18	22.78	97.38	0.90	-0.00	22.45
○Thread 289047			22.18	22.78	97.38	0.90	-0.00	22.45
○Thread 289048			22.22	22.78	97.58	0.90	-0.00	22.45
○Thread 289049			22.17	22.78	97.34	0.90	-0.00	22.45
○Thread 289050			22.16	22.78	97.30	0.90	-0.00	22.46
○Thread 289051			22.13	22.78	97.19	0.90	-0.00	22.45
○Thread 289052			22.08	22.78	96.97	0.90	-0.00	22.46
○Thread 289053			22.14	22.78	97.23	0.90	-0.00	22.45
○Thread 289054			22.16	22.77	97.30	0.90	-0.00	22.45
○Thread 289055			22.18	22.77	97.41	0.90	-0.00	22.45
○Thread 289056			22.16	22.77	97.32	0.90	-0.00	22.45
○Thread 289057			22.23	22.77	97.63	0.90	-0.00	22.46
○Thread 289058			22.13	22.77	97.19	0.90	0.00	22.45
○Thread 289059			22.15	22.77	97.28	0.90	-0.00	22.45

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	17.97
►Process 289129+		32	17.97
○Thread 289129			17.97	19.19	93.67	0.00	0.00	19.15
○Thread 289228			17.58	18.29	96.12	0.90	-0.00	17.96
○Thread 289229			17.65	18.29	96.53	0.90	-0.00	17.96
○Thread 289230			17.67	18.29	96.61	0.90	-0.00	17.97
○Thread 289231			17.59	18.29	96.17	0.90	-0.00	17.96
○Thread 289232			17.62	18.29	96.34	0.90	-0.00	17.96
○Thread 289233			17.68	18.29	96.67	0.90	-0.00	17.96
○Thread 289234			17.66	18.29	96.59	0.90	-0.00	17.97
○Thread 289235			17.55	18.29	95.99	0.90	-0.00	17.97
○Thread 289236			17.68	18.29	96.70	0.90	-0.00	17.97
○Thread 289237			17.69	18.29	96.72	0.90	-0.00	17.97
○Thread 289238			17.75	18.29	97.08	0.90	-0.00	17.96
○Thread 289239			17.65	18.29	96.51	0.90	-0.00	17.96
○Thread 289240			17.57	18.29	96.07	0.90	-0.00	17.96
○Thread 289241			17.68	18.29	96.70	0.90	-0.00	17.96
○Thread 289242			17.66	18.29	96.59	0.90	-0.00	17.96
○Thread 289243			17.57	18.29	96.10	0.90	-0.00	17.97
○Thread 289244			17.55	18.29	95.96	0.90	-0.00	17.96
○Thread 289245			17.62	18.29	96.35	0.90	-0.00	17.97
○Thread 289246			17.64	18.29	96.46	0.90	-0.00	17.96
○Thread 289247			17.60	18.29	96.27	0.90	-0.00	17.96
○Thread 289248			17.64	18.29	96.46	0.90	-0.00	17.97
○Thread 289249			17.65	18.29	96.54	0.90	-0.00	17.97
○Thread 289250			17.62	18.29	96.35	0.90	-0.00	17.96
○Thread 289251			17.59	18.29	96.19	0.90	-0.00	17.96
○Thread 289252			17.60	18.29	96.24	0.90	-0.00	17.96
○Thread 289253			17.59	18.29	96.21	0.90	0.00	17.96
○Thread 289254			17.64	18.29	96.46	0.90	0.00	17.97
○Thread 289255			17.68	18.29	96.71	0.90	0.00	17.96
○Thread 289256			17.69	18.29	96.76	0.90	0.00	17.97
○Thread 289257			17.65	18.29	96.52	0.90	0.00	17.96
○Thread 289258			17.65	18.29	96.52	0.90	-0.00	17.96

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	15.77
►Process 289278+		40	15.77
○Thread 289278			15.77	17.03	92.61	0.00	0.00	16.99
○Thread 289377			15.39	16.14	95.33	0.89	0.00	15.81
○Thread 289378			15.41	16.14	95.46	0.89	0.00	15.80
○Thread 289379			15.47	16.14	95.86	0.89	-0.00	15.81
○Thread 289380			15.37	16.14	95.24	0.89	0.00	15.81
○Thread 289381			15.33	16.14	94.97	0.89	0.00	15.81
○Thread 289382			15.43	16.14	95.59	0.89	0.00	15.81
○Thread 289383			15.39	16.14	95.34	0.89	0.00	15.81
○Thread 289384			15.32	16.14	94.91	0.89	0.00	15.81
○Thread 289385			15.38	16.14	95.31	0.89	0.00	15.81
○Thread 289386			15.50	16.14	96.08	0.89	0.00	15.81
○Thread 289387			15.38	16.14	95.31	0.89	0.00	15.81
○Thread 289388			15.36	16.14	95.16	0.89	0.00	15.81
○Thread 289389			15.34	16.14	95.09	0.89	0.00	15.81
○Thread 289390			15.45	16.14	95.72	0.89	0.00	15.81
○Thread 289391			15.40	16.14	95.41	0.89	0.00	15.81
○Thread 289392			15.40	16.14	95.44	0.89	0.00	15.81
○Thread 289393			15.35	16.14	95.13	0.89	0.00	15.81
○Thread 289394			15.36	16.14	95.16	0.89	0.00	15.81
○Thread 289395			15.49	16.14	95.97	0.89	0.00	15.80
○Thread 289396			15.26	16.14	94.57	0.89	0.00	15.79
○Thread 289397			15.28	16.14	94.70	0.89	0.00	15.81
○Thread 289398			15.30	16.14	94.82	0.89	0.00	15.81
○Thread 289399			15.40	16.14	95.41	0.89	0.00	15.81
○Thread 289400			15.44	16.14	95.66	0.89	0.00	15.81
○Thread 289401			15.50	16.14	96.03	0.89	0.00	15.81
○Thread 289402			15.37	16.14	95.26	0.89	0.00	15.80
○Thread 289403			15.39	16.13	95.38	0.89	0.00	15.80
○Thread 289404			15.34	16.13	95.07	0.89	0.00	15.81
○Thread 289405			15.33	16.13	94.98	0.89	0.00	15.81
○Thread 289406			15.44	16.13	95.66	0.89	0.00	15.81
○Thread 289407			15.46	16.13	95.82	0.89	0.00	15.81
○Thread 289408			15.30	16.13	94.80	0.89	0.00	15.81
○Thread 289409			15.39	16.13	95.39	0.89	0.00	15.81
○Thread 289410			15.28	16.13	94.71	0.89	0.00	15.81
○Thread 289411			15.31	16.13	94.86	0.89	0.00	15.81
○Thread 289412			15.44	16.13	95.73	0.89	0.00	15.81
○Thread 289413			15.38	16.13	95.30	0.89	0.00	15.81
○Thread 289414			15.32	16.13	94.96	0.89	0.00	15.81
○Thread 289415			15.48	16.13	95.95	0.89	-0.00	15.81

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	14.00
►Process 289486+		48	14.00
○Thread 289486			14.00	15.36	91.12	0.00	0.00	15.30
○Thread 289585			13.52	14.46	93.51	0.90	-0.00	14.12
○Thread 289586			13.64	14.46	94.34	0.90	-0.00	14.13
○Thread 289587			13.64	14.46	94.37	0.90	-0.00	14.13
○Thread 289588			13.69	14.46	94.69	0.90	-0.00	14.13
○Thread 289589			13.71	14.46	94.83	0.90	-0.00	14.12
○Thread 289590			13.58	14.46	93.93	0.90	-0.00	14.12
○Thread 289591			13.66	14.46	94.48	0.90	-0.00	14.13
○Thread 289592			13.55	14.46	93.69	0.90	-0.00	14.12
○Thread 289593			13.63	14.46	94.24	0.90	-0.00	14.12
○Thread 289594			13.67	14.46	94.55	0.90	-0.00	14.12
○Thread 289595			13.66	14.46	94.48	0.90	-0.00	14.12
○Thread 289596			13.69	14.46	94.66	0.90	-0.00	14.13
○Thread 289597			13.62	14.46	94.17	0.90	-0.00	14.12
○Thread 289598			13.74	14.46	95.04	0.90	-0.00	14.12
○Thread 289599			13.57	14.46	93.83	0.90	-0.00	14.12
○Thread 289600			13.58	14.46	93.93	0.90	-0.00	14.13
○Thread 289601			13.72	14.46	94.87	0.90	-0.00	14.13
○Thread 289602			13.65	14.46	94.46	0.90	-0.00	14.12
○Thread 289603			13.62	14.46	94.18	0.90	-0.00	14.13
○Thread 289604			13.68	14.46	94.63	0.90	-0.00	14.13
○Thread 289605			13.65	14.46	94.46	0.90	-0.00	14.13
○Thread 289606			13.63	14.46	94.25	0.90	-0.00	14.13
○Thread 289607			13.74	14.46	95.05	0.90	-0.00	14.12
○Thread 289608			13.60	14.46	94.08	0.90	-0.00	14.12
○Thread 289609			13.66	14.46	94.53	0.90	-0.00	14.12
○Thread 289610			13.59	14.46	94.01	0.90	-0.00	14.12
○Thread 289611			13.58	14.46	93.94	0.90	-0.00	14.12
○Thread 289612			13.62	14.46	94.22	0.90	-0.00	14.13
○Thread 289613			13.63	14.46	94.32	0.90	-0.00	14.12
○Thread 289614			13.58	14.46	93.91	0.90	-0.00	14.12
○Thread 289615			13.51	14.46	93.50	0.90	-0.00	14.12
○Thread 289616			13.61	14.46	94.15	0.90	-0.00	14.12
○Thread 289617			13.63	14.45	94.26	0.90	-0.00	14.12
○Thread 289618			13.62	14.45	94.26	0.90	-0.00	14.12
○Thread 289619			13.70	14.45	94.81	0.90	-0.00	14.12
○Thread 289620			13.67	14.45	94.54	0.90	-0.00	14.12
○Thread 289621			13.63	14.45	94.30	0.90	-0.00	14.12
○Thread 289622			13.65	14.45	94.40	0.90	-0.00	14.12
○Thread 289623			13.59	14.45	94.02	0.91	-0.00	14.12
○Thread 289624			13.59	14.45	94.02	0.91	0.00	14.12
○Thread 289625			13.64	14.45	94.33	0.91	0.00	14.13
○Thread 289626			13.63	14.45	94.30	0.91	0.00	14.12
○Thread 289627			13.55	14.45	93.71	0.91	0.00	14.13
○Thread 289628			13.57	14.45	93.89	0.91	0.00	14.12
○Thread 289629			13.62	14.45	94.20	0.91	0.00	14.12
○Thread 289630			13.66	14.45	94.51	0.91	0.00	14.13
○Thread 289631			13.63	14.45	94.30	0.91	-0.00	14.13

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	12.87
►Process 289651+		56	12.87
○Thread 289651			12.87	14.38	89.49	0.00	0.00	14.33
○Thread 289750			12.50	13.48	92.72	0.90	-0.00	13.13
○Thread 289751			12.41	13.48	92.05	0.90	-0.00	13.13
○Thread 289752			12.54	13.48	93.02	0.90	-0.00	13.13
○Thread 289753			12.53	13.48	92.94	0.90	-0.00	13.13
○Thread 289754			12.40	13.48	91.98	0.90	-0.00	13.13
○Thread 289755			12.43	13.48	92.24	0.90	-0.00	13.13
○Thread 289756			12.58	13.48	93.28	0.90	-0.00	13.13
○Thread 289757			12.51	13.48	92.76	0.90	-0.00	13.13
○Thread 289758			12.60	13.48	93.47	0.90	-0.00	13.13
○Thread 289759			12.46	13.48	92.39	0.90	-0.00	13.13
○Thread 289760			12.45	13.48	92.32	0.90	-0.00	13.13
○Thread 289761			12.41	13.48	92.06	0.90	-0.00	13.13
○Thread 289762			12.49	13.48	92.65	0.90	-0.00	13.13
○Thread 289763			12.54	13.48	93.03	0.90	-0.00	13.13
○Thread 289764			12.58	13.48	93.29	0.90	-0.00	13.12
○Thread 289765			12.49	13.48	92.66	0.90	-0.00	13.13
○Thread 289766			12.44	13.48	92.29	0.90	-0.00	13.13
○Thread 289767			12.40	13.48	91.95	0.90	-0.00	13.13
○Thread 289768			12.55	13.48	93.10	0.90	-0.00	13.13
○Thread 289769			12.45	13.48	92.36	0.90	-0.00	13.13
○Thread 289770			12.57	13.48	93.25	0.90	-0.00	13.13
○Thread 289771			12.47	13.48	92.48	0.90	-0.00	13.13
○Thread 289772			12.47	13.48	92.48	0.90	-0.00	13.13
○Thread 289773			12.47	13.48	92.48	0.90	-0.00	13.13
○Thread 289774			12.52	13.48	92.85	0.90	-0.00	13.13
○Thread 289775			12.46	13.48	92.44	0.90	-0.00	13.13
○Thread 289776			12.59	13.48	93.37	0.90	-0.00	13.13
○Thread 289777			12.46	13.48	92.41	0.90	-0.00	13.13
○Thread 289778			12.47	13.48	92.52	0.90	-0.00	13.13
○Thread 289779			12.47	13.48	92.52	0.90	-0.00	13.13
○Thread 289780			12.43	13.48	92.22	0.90	-0.00	13.13
○Thread 289781			12.41	13.48	92.08	0.90	-0.00	13.13
○Thread 289782			12.50	13.48	92.71	0.90	-0.00	13.12
○Thread 289783			12.42	13.48	92.11	0.90	-0.00	13.12
○Thread 289784			12.49	13.48	92.67	0.90	-0.00	13.13
○Thread 289785			12.30	13.48	91.23	0.90	-0.00	13.13
○Thread 289786			12.47	13.48	92.52	0.90	-0.00	13.13
○Thread 289787			12.46	13.48	92.41	0.90	-0.00	13.13
○Thread 289788			12.49	13.48	92.67	0.90	-0.00	13.13
○Thread 289789			12.50	13.48	92.71	0.90	-0.00	13.13
○Thread 289790			12.30	13.48	91.23	0.90	-0.00	13.13
○Thread 289791			12.45	13.48	92.38	0.90	-0.00	13.14
○Thread 289792			12.35	13.48	91.64	0.90	-0.00	13.13
○Thread 289793			12.43	13.48	92.20	0.90	-0.00	13.13
○Thread 289794			12.44	13.48	92.31	0.91	-0.00	13.13
○Thread 289795			12.45	13.48	92.38	0.91	-0.00	13.13
○Thread 289796			12.51	13.48	92.83	0.91	-0.00	13.13
○Thread 289797			12.32	13.48	91.38	0.91	-0.00	13.13
○Thread 289798			12.43	13.48	92.24	0.91	-0.00	13.14
○Thread 289799			12.45	13.48	92.42	0.91	-0.00	13.13
○Thread 289800			12.35	13.48	91.61	0.91	-0.00	13.13
○Thread 289801			12.30	13.48	91.27	0.91	-0.00	13.14
○Thread 289802			12.45	13.48	92.35	0.91	-0.00	13.13
○Thread 289803			12.46	13.48	92.46	0.91	-0.00	13.13
○Thread 289804			12.47	13.48	92.58	0.91	-0.00	13.13

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	11.21
►Process 289825+		64	11.21
○Thread 289825			11.21	12.79	87.67	0.00	0.00	12.75
○Thread 289924			10.95	11.89	92.12	0.90	-0.00	11.54
○Thread 289925			10.79	11.89	90.81	0.90	-0.00	11.55
○Thread 289926			10.80	11.89	90.86	0.90	-0.00	11.55
○Thread 289927			10.90	11.89	91.66	0.90	-0.00	11.54
○Thread 289928			10.85	11.89	91.24	0.90	-0.00	11.54
○Thread 289929			10.87	11.89	91.41	0.90	-0.00	11.55
○Thread 289930			10.88	11.89	91.49	0.90	-0.00	11.55
○Thread 289931			10.90	11.89	91.66	0.90	-0.00	11.55
○Thread 289932			10.94	11.89	92.04	0.90	0.00	11.55
○Thread 289933			10.93	11.89	91.96	0.90	-0.00	11.54
○Thread 289934			10.95	11.89	92.08	0.90	-0.00	11.54
○Thread 289935			10.84	11.89	91.20	0.90	-0.00	11.56
○Thread 289936			10.83	11.89	91.12	0.90	-0.00	11.55
○Thread 289937			10.81	11.89	90.91	0.90	-0.00	11.54
○Thread 289938			10.83	11.89	91.12	0.90	-0.00	11.54
○Thread 289939			10.90	11.89	91.75	0.90	-0.00	11.55
○Thread 289940			10.80	11.89	90.87	0.90	-0.00	11.54
○Thread 289941			10.92	11.88	91.92	0.90	-0.00	11.55
○Thread 289942			10.95	11.88	92.09	0.90	-0.00	11.55
○Thread 289943			10.89	11.88	91.67	0.90	-0.00	11.55
○Thread 289944			10.91	11.88	91.80	0.90	-0.00	11.55
○Thread 289945			11.03	11.88	92.81	0.90	-0.00	11.54
○Thread 289946			10.84	11.88	91.17	0.90	-0.00	11.55
○Thread 289947			10.87	11.88	91.42	0.90	-0.00	11.54
○Thread 289948			10.92	11.88	91.89	0.90	-0.00	11.55
○Thread 289949			10.89	11.88	91.64	0.90	-0.00	11.55
○Thread 289950			10.87	11.88	91.47	0.90	-0.00	11.55
○Thread 289951			11.03	11.88	92.86	0.90	-0.00	11.55
○Thread 289952			10.92	11.88	91.89	0.90	-0.00	11.55
○Thread 289953			10.75	11.88	90.46	0.90	-0.00	11.54
○Thread 289954			10.93	11.88	91.98	0.90	-0.00	11.55
○Thread 289955			10.99	11.88	92.48	0.90	-0.00	11.55
○Thread 289956			10.90	11.88	91.68	0.90	-0.00	11.55
○Thread 289957			10.95	11.88	92.15	0.90	-0.00	11.55
○Thread 289958			10.94	11.88	92.06	0.90	-0.00	11.55
○Thread 289959			10.94	11.88	92.02	0.90	-0.00	11.55
○Thread 289960			11.00	11.88	92.57	0.90	-0.00	11.55
○Thread 289961			10.91	11.88	91.81	0.90	-0.00	11.55
○Thread 289962			10.98	11.88	92.36	0.90	-0.00	11.55
○Thread 289963			10.89	11.88	91.65	0.90	-0.00	11.55
○Thread 289964			10.97	11.88	92.36	0.90	-0.00	11.55
○Thread 289965			10.95	11.88	92.11	0.90	-0.00	11.54
○Thread 289966			10.95	11.88	92.11	0.90	-0.00	11.55
○Thread 289967			10.95	11.88	92.16	0.90	-0.00	11.55
○Thread 289968			10.82	11.88	91.10	0.90	-0.00	11.54
○Thread 289969			10.93	11.88	91.99	0.90	-0.00	11.53
○Thread 289970			10.91	11.88	91.82	0.90	-0.00	11.55
○Thread 289971			10.92	11.88	91.91	0.90	-0.00	11.55
○Thread 289972			10.97	11.88	92.29	0.90	-0.00	11.55
○Thread 289973			10.97	11.88	92.33	0.90	-0.00	11.55
○Thread 289974			10.85	11.88	91.32	0.90	-0.00	11.55
○Thread 289975			10.86	11.88	91.41	0.90	-0.00	11.55
○Thread 289976			10.89	11.88	91.70	0.91	-0.00	11.53
○Thread 289977			10.93	11.88	91.95	0.91	-0.00	11.54
○Thread 289978			10.78	11.88	90.69	0.91	-0.00	11.55
○Thread 289979			10.84	11.88	91.20	0.91	-0.00	11.55
○Thread 289980			10.91	11.88	91.83	0.91	-0.00	11.54
○Thread 289981			10.98	11.88	92.42	0.91	-0.00	11.55
○Thread 289982			10.88	11.88	91.58	0.91	-0.00	11.54
○Thread 289983			10.91	11.88	91.88	0.91	-0.00	11.54
○Thread 289984			10.89	11.88	91.71	0.91	-0.00	11.55
○Thread 289985			10.93	11.88	91.96	0.91	-0.00	11.54
○Thread 289986			10.91	11.88	91.79	0.91	-0.00	11.55

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	12.26
►Process 290058+		72	12.26
○Thread 290058			12.26	13.95	87.85	0.00	0.00	13.89
○Thread 290157			11.90	13.06	91.07	0.89	0.00	12.73
○Thread 290158			11.96	13.06	91.53	0.89	-0.00	12.72
○Thread 290159			12.06	13.06	92.30	0.89	0.00	12.72
○Thread 290160			11.95	13.06	91.54	0.89	-0.00	12.71
○Thread 290161			11.92	13.06	91.27	0.89	-0.00	12.73
○Thread 290162			11.88	13.06	90.92	0.89	-0.00	12.72
○Thread 290163			11.86	13.06	90.77	0.89	-0.00	12.71
○Thread 290164			11.93	13.06	91.31	0.89	0.00	12.73
○Thread 290165			12.09	13.06	92.54	0.89	-0.00	12.71
○Thread 290166			12.08	13.06	92.50	0.89	0.00	12.72
○Thread 290167			11.96	13.06	91.54	0.89	-0.00	12.73
○Thread 290168			11.95	13.06	91.50	0.89	-0.00	12.72
○Thread 290169			11.97	13.06	91.66	0.89	-0.00	12.72
○Thread 290170			12.02	13.06	92.04	0.89	-0.00	12.71
○Thread 290171			12.24	13.06	93.69	0.89	-0.00	12.73
○Thread 290172			12.05	13.06	92.23	0.89	0.00	12.71
○Thread 290173			11.92	13.06	91.24	0.89	-0.00	12.72
○Thread 290174			11.99	13.06	91.81	0.89	0.00	12.72
○Thread 290175			11.95	13.06	91.47	0.89	-0.00	12.73
○Thread 290176			12.10	13.06	92.66	0.89	-0.00	12.71
○Thread 290177			12.17	13.06	93.20	0.89	-0.00	12.73
○Thread 290178			11.93	13.06	91.32	0.89	-0.00	12.73
○Thread 290179			11.97	13.06	91.67	0.89	0.00	12.72
○Thread 290180			12.00	13.06	91.86	0.89	-0.00	12.72
○Thread 290181			11.89	13.06	91.02	0.89	-0.00	12.71
○Thread 290182			11.85	13.06	90.71	0.89	-0.00	12.71
○Thread 290183			11.90	13.06	91.09	0.89	-0.00	12.73
○Thread 290184			11.94	13.06	91.44	0.89	-0.00	12.71
○Thread 290185			11.97	13.06	91.67	0.89	-0.00	12.71
○Thread 290186			12.02	13.06	92.02	0.89	-0.00	12.72
○Thread 290187			11.90	13.06	91.10	0.89	-0.00	12.72
○Thread 290188			12.00	13.06	91.86	0.89	-0.00	12.71
○Thread 290189			11.90	13.06	91.14	0.89	-0.00	12.73
○Thread 290190			11.98	13.06	91.79	0.89	-0.00	12.71
○Thread 290191			11.94	13.06	91.41	0.89	-0.00	12.72
○Thread 290192			11.92	13.06	91.25	0.89	-0.00	12.71
○Thread 290193			11.87	13.06	90.91	0.89	-0.00	12.72
○Thread 290194			11.93	13.06	91.37	0.89	-0.00	12.72
○Thread 290195			11.98	13.06	91.72	0.89	-0.00	12.73
○Thread 290196			12.02	13.06	92.06	0.89	-0.00	12.72
○Thread 290197			11.95	13.06	91.53	0.89	-0.00	12.71
○Thread 290198			11.97	13.06	91.64	0.89	-0.00	12.71
○Thread 290199			11.97	13.06	91.64	0.89	-0.00	12.71
○Thread 290200			11.92	13.06	91.26	0.89	-0.00	12.71
○Thread 290201			11.96	13.06	91.61	0.89	-0.00	12.72
○Thread 290202			11.90	13.06	91.11	0.89	-0.00	12.72
○Thread 290203			11.90	13.06	91.11	0.89	-0.00	12.73
○Thread 290204			11.96	13.06	91.57	0.89	-0.00	12.71
○Thread 290205			11.94	13.06	91.42	0.89	-0.00	12.72
○Thread 290206			11.87	13.06	90.88	0.89	-0.00	12.72
○Thread 290207			11.92	13.06	91.27	0.89	-0.00	12.72
○Thread 290208			11.88	13.06	90.96	0.89	-0.00	12.72
○Thread 290209			11.84	13.05	90.66	0.89	-0.00	12.71
○Thread 290210			12.01	13.05	91.96	0.89	0.00	12.72
○Thread 290211			12.06	13.05	92.38	0.89	-0.00	12.73
○Thread 290212			12.02	13.05	92.04	0.89	-0.00	12.72
○Thread 290213			11.89	13.05	91.08	0.89	-0.00	12.73
○Thread 290214			11.90	13.05	91.16	0.89	-0.00	12.73
○Thread 290215			11.83	13.05	90.58	0.89	-0.00	12.72
○Thread 290216			11.97	13.05	91.66	0.90	-0.00	12.71
○Thread 290217			11.98	13.05	91.73	0.90	-0.00	12.73
○Thread 290218			11.95	13.05	91.50	0.90	-0.00	12.73
○Thread 290219			11.92	13.05	91.28	0.90	0.00	12.71
○Thread 290220			11.99	13.05	91.81	0.90	-0.00	12.72
○Thread 290221			11.87	13.05	90.93	0.90	0.00	12.72
○Thread 290222			11.87	13.05	90.93	0.90	-0.00	12.71
○Thread 290223			11.80	13.05	90.40	0.90	0.00	12.71
○Thread 290224			12.01	13.05	91.97	0.90	0.00	12.71
○Thread 290225			11.97	13.05	91.66	0.90	0.00	12.72
○Thread 290226			11.86	13.05	90.86	0.90	0.00	12.72
○Thread 290227			11.79	13.05	90.32	0.90	-0.00	12.72

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	10.12
►Process 290248+		80	10.12
○Thread 290248			10.12	11.89	85.09	0.00	0.00	11.84
○Thread 290347			9.81	10.99	89.28	0.91	0.00	10.64
○Thread 290348			9.81	10.99	89.23	0.91	0.00	10.63
○Thread 290349			9.63	10.99	87.60	0.91	0.00	10.66
○Thread 290350			9.73	10.99	88.55	0.91	-0.00	10.63
○Thread 290351			9.84	10.99	89.56	0.91	-0.00	10.65
○Thread 290352			9.56	10.99	86.96	0.91	-0.00	10.63
○Thread 290353			9.74	10.99	88.60	0.91	0.00	10.64
○Thread 290354			9.72	10.99	88.42	0.91	-0.00	10.64
○Thread 290355			9.85	10.99	89.60	0.91	-0.00	10.64
○Thread 290356			9.98	10.99	90.83	0.91	-0.00	10.65
○Thread 290357			9.91	10.99	90.20	0.91	-0.00	10.63
○Thread 290358			9.71	10.99	88.38	0.91	-0.00	10.66
○Thread 290359			9.77	10.99	88.93	0.91	-0.00	10.64
○Thread 290360			9.81	10.99	89.29	0.91	0.00	10.64
○Thread 290361			9.98	10.99	90.79	0.91	-0.00	10.65
○Thread 290362			10.08	10.99	91.75	0.91	-0.00	10.64
○Thread 290363			9.89	10.99	90.02	0.91	-0.00	10.64
○Thread 290364			10.08	10.99	91.75	0.91	0.00	10.66
○Thread 290365			9.89	10.99	90.02	0.91	-0.00	10.64
○Thread 290366			9.71	10.99	88.34	0.91	-0.00	10.66
○Thread 290367			9.70	10.99	88.30	0.91	-0.00	10.64
○Thread 290368			9.72	10.99	88.48	0.91	-0.00	10.64
○Thread 290369			9.67	10.99	88.02	0.91	-0.00	10.64
○Thread 290370			9.81	10.99	89.25	0.91	-0.00	10.64
○Thread 290371			9.69	10.99	88.16	0.91	-0.00	10.64
○Thread 290372			9.89	10.99	90.03	0.91	-0.00	10.66
○Thread 290373			9.64	10.99	87.71	0.91	-0.00	10.64
○Thread 290374			9.87	10.99	89.85	0.91	-0.00	10.65
○Thread 290375			9.77	10.98	88.89	0.91	-0.00	10.64
○Thread 290376			10.06	10.98	91.54	0.91	-0.00	10.64
○Thread 290377			9.87	10.98	89.85	0.91	-0.00	10.64
○Thread 290378			9.76	10.98	88.81	0.91	-0.00	10.64
○Thread 290379			9.75	10.98	88.72	0.91	-0.00	10.64
○Thread 290380			9.63	10.98	87.62	0.91	-0.00	10.64
○Thread 290381			9.76	10.98	88.85	0.91	-0.00	10.63
○Thread 290382			9.75	10.98	88.72	0.91	-0.00	10.64
○Thread 290383			9.68	10.98	88.13	0.91	-0.00	10.63
○Thread 290384			9.79	10.98	89.08	0.91	-0.00	10.64
○Thread 290385			9.66	10.98	87.90	0.91	-0.00	10.64
○Thread 290386			9.82	10.98	89.41	0.91	-0.00	10.64
○Thread 290387			9.81	10.98	89.32	0.91	-0.00	10.64
○Thread 290388			9.84	10.98	89.54	0.91	-0.00	10.64
○Thread 290389			9.86	10.98	89.73	0.91	-0.00	10.64
○Thread 290390			9.70	10.98	88.27	0.91	-0.00	10.66
○Thread 290391			9.76	10.98	88.82	0.91	-0.00	10.64
○Thread 290392			9.63	10.98	87.68	0.91	-0.00	10.66
○Thread 290393			9.74	10.98	88.68	0.91	-0.00	10.65
○Thread 290394			9.60	10.98	87.41	0.91	-0.00	10.63
○Thread 290395			9.64	10.98	87.77	0.91	-0.00	10.63
○Thread 290396			9.75	10.98	88.73	0.91	-0.00	10.63
○Thread 290397			9.77	10.98	88.91	0.91	-0.00	10.64
○Thread 290398			9.78	10.98	89.05	0.91	-0.00	10.64
○Thread 290399			9.81	10.98	89.28	0.91	-0.00	10.64
○Thread 290400			9.74	10.98	88.64	0.91	-0.00	10.64
○Thread 290401			9.76	10.98	88.83	0.91	-0.00	10.64
○Thread 290402			9.73	10.98	88.60	0.91	-0.00	10.65
○Thread 290403			9.83	10.98	89.47	0.91	-0.00	10.64
○Thread 290404			9.70	10.98	88.28	0.91	-0.00	10.64
○Thread 290405			9.60	10.98	87.42	0.91	-0.00	10.64
○Thread 290406			9.75	10.98	88.79	0.91	-0.00	10.64
○Thread 290407			9.72	10.98	88.51	0.91	-0.00	10.64
○Thread 290408			9.83	10.98	89.47	0.91	-0.00	10.64
○Thread 290409			9.77	10.98	88.93	0.91	-0.00	10.64
○Thread 290410			9.85	10.98	89.70	0.91	-0.00	10.64
○Thread 290411			9.76	10.98	88.84	0.91	-0.00	10.64
○Thread 290412			9.73	10.98	88.56	0.91	-0.00	10.66
○Thread 290413			9.75	10.98	88.79	0.91	-0.00	10.64
○Thread 290414			9.77	10.98	88.93	0.91	-0.00	10.64
○Thread 290415			9.70	10.98	88.29	0.91	-0.00	10.64
○Thread 290416			9.74	10.98	88.70	0.91	0.00	10.65
○Thread 290417			9.77	10.98	88.93	0.91	-0.00	10.64
○Thread 290418			9.78	10.98	89.02	0.91	0.00	10.64
○Thread 290419			9.77	10.98	88.98	0.91	-0.00	10.64
○Thread 290420			9.73	10.98	88.57	0.91	0.00	10.64
○Thread 290421			9.77	10.98	88.94	0.91	-0.00	10.64
○Thread 290422			9.79	10.98	89.12	0.91	-0.00	10.65
○Thread 290423			9.73	10.98	88.62	0.91	0.00	10.64
○Thread 290424			9.70	10.98	88.35	0.91	0.00	10.64
○Thread 290425			7.52	10.98	68.45	0.91	0.00	10.64

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	10.68
►Process 290445+		88	10.68
○Thread 290445			10.68	12.66	84.35	0.00	0.00	12.62
○Thread 290544			10.40	11.76	88.44	0.90	-0.00	11.42
○Thread 290545			10.43	11.76	88.65	0.90	-0.00	11.42
○Thread 290546			10.46	11.76	88.99	0.90	-0.00	11.40
○Thread 290547			10.48	11.76	89.12	0.90	-0.00	11.40
○Thread 290548			10.43	11.76	88.70	0.90	-0.00	11.41
○Thread 290549			10.31	11.76	87.63	0.90	-0.00	11.41
○Thread 290550			10.36	11.76	88.06	0.90	-0.00	11.40
○Thread 290551			10.49	11.76	89.17	0.90	-0.00	11.40
○Thread 290552			10.33	11.76	87.85	0.90	-0.00	11.40
○Thread 290553			10.39	11.76	88.36	0.90	-0.00	11.41
○Thread 290554			10.35	11.76	87.98	0.90	-0.00	11.40
○Thread 290555			10.51	11.76	89.38	0.90	-0.00	11.41
○Thread 290556			10.51	11.76	89.34	0.90	-0.00	11.40
○Thread 290557			10.56	11.76	89.77	0.90	-0.00	11.40
○Thread 290558			10.43	11.76	88.75	0.90	-0.00	11.40
○Thread 290559			10.49	11.76	89.22	0.90	-0.00	11.40
○Thread 290560			10.26	11.76	87.26	0.90	-0.00	11.40
○Thread 290561			10.36	11.76	88.07	0.90	-0.00	11.40
○Thread 290562			10.36	11.76	88.07	0.90	-0.00	11.41
○Thread 290563			10.42	11.76	88.62	0.90	-0.00	11.40
○Thread 290564			10.38	11.76	88.33	0.90	-0.00	11.40
○Thread 290565			10.46	11.76	88.92	0.90	-0.00	11.40
○Thread 290566			10.46	11.76	88.92	0.90	-0.00	11.40
○Thread 290567			10.43	11.76	88.71	0.90	-0.00	11.40
○Thread 290568			10.47	11.76	89.05	0.90	-0.00	11.41
○Thread 290569			10.41	11.76	88.54	0.90	-0.00	11.40
○Thread 290570			10.44	11.76	88.80	0.90	-0.00	11.41
○Thread 290571			10.36	11.76	88.08	0.90	-0.00	11.39
○Thread 290572			10.30	11.76	87.61	0.90	-0.00	11.41
○Thread 290573			10.43	11.76	88.72	0.91	-0.00	11.40
○Thread 290574			10.33	11.76	87.83	0.91	-0.00	11.40
○Thread 290575			10.35	11.76	88.04	0.91	-0.00	11.41
○Thread 290576			10.26	11.76	87.27	0.91	-0.00	11.41
○Thread 290577			10.39	11.76	88.42	0.91	-0.00	11.40
○Thread 290578			10.45	11.76	88.93	0.91	-0.00	11.40
○Thread 290579			10.53	11.76	89.53	0.91	-0.00	11.41
○Thread 290580			10.56	11.76	89.79	0.91	-0.00	11.41
○Thread 290581			10.41	11.76	88.51	0.91	-0.00	11.40
○Thread 290582			10.53	11.76	89.53	0.91	-0.00	11.41
○Thread 290583			10.30	11.76	87.66	0.91	-0.00	11.40
○Thread 290584			10.41	11.76	88.51	0.91	-0.00	11.41
○Thread 290585			10.43	11.76	88.73	0.91	-0.00	11.41
○Thread 290586			10.30	11.75	87.62	0.91	-0.00	11.40
○Thread 290587			10.56	11.75	89.84	0.91	-0.00	11.40
○Thread 290588			10.42	11.75	88.65	0.91	-0.00	11.41
○Thread 290589			10.59	11.75	90.09	0.91	-0.00	11.40
○Thread 290590			10.42	11.75	88.65	0.91	-0.00	11.41
○Thread 290591			10.55	11.75	89.71	0.91	-0.00	11.41
○Thread 290592			10.40	11.75	88.44	0.91	-0.00	11.40
○Thread 290593			10.50	11.75	89.33	0.91	-0.00	11.41
○Thread 290594			10.46	11.75	88.99	0.91	-0.00	11.40
○Thread 290595			10.43	11.75	88.69	0.91	-0.00	11.41
○Thread 290596			10.40	11.75	88.44	0.91	-0.00	11.41
○Thread 290597			10.47	11.75	89.08	0.91	-0.00	11.40
○Thread 290598			10.37	11.75	88.23	0.91	-0.00	11.40
○Thread 290599			10.46	11.75	88.95	0.91	-0.00	11.41
○Thread 290600			10.39	11.75	88.40	0.91	-0.00	11.40
○Thread 290601			10.39	11.75	88.40	0.91	-0.00	11.40
○Thread 290602			10.45	11.75	88.87	0.91	-0.00	11.41
○Thread 290603			10.58	11.75	89.98	0.91	-0.00	11.41
○Thread 290604			10.42	11.75	88.66	0.91	-0.00	11.40
○Thread 290605			10.34	11.75	87.98	0.91	-0.00	11.41
○Thread 290606			10.25	11.75	87.21	0.91	-0.00	11.41
○Thread 290607			10.42	11.75	88.62	0.91	-0.00	11.40
○Thread 290608			10.41	11.75	88.58	0.91	-0.00	11.41
○Thread 290609			10.44	11.75	88.88	0.91	-0.00	11.41
○Thread 290610			10.46	11.75	88.96	0.91	-0.00	11.41
○Thread 290611			10.40	11.75	88.49	0.91	-0.00	11.41
○Thread 290612			10.35	11.75	88.07	0.91	-0.00	11.40
○Thread 290613			10.35	11.75	88.07	0.91	-0.00	11.40
○Thread 290614			10.45	11.75	88.88	0.91	-0.00	11.40
○Thread 290615			10.45	11.75	88.97	0.91	-0.00	11.41
○Thread 290616			10.38	11.75	88.33	0.91	-0.00	11.41
○Thread 290617			10.48	11.75	89.14	0.91	-0.00	11.41
○Thread 290618			10.43	11.75	88.71	0.91	-0.00	11.40
○Thread 290619			10.38	11.75	88.29	0.91	-0.00	11.40
○Thread 290620			10.51	11.75	89.40	0.91	-0.00	11.40
○Thread 290621			10.44	11.75	88.80	0.91	-0.00	11.40
○Thread 290622			10.47	11.75	89.14	0.91	-0.00	11.42
○Thread 290623			10.30	11.75	87.65	0.91	-0.00	11.41
○Thread 290624			10.46	11.75	89.02	0.91	-0.00	11.40
○Thread 290625			10.35	11.75	88.04	0.91	-0.00	11.40
○Thread 290626			10.41	11.75	88.59	0.91	-0.00	11.40
○Thread 290627			10.46	11.75	89.02	0.91	-0.00	11.41
○Thread 290628			10.43	11.75	88.76	0.91	-0.00	11.40
○Thread 290629			10.49	11.75	89.23	0.91	-0.00	11.40
○Thread 290630			7.75	11.75	65.96	0.91	-0.00	11.39

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	10.29
►Process 290698+		96	10.29
○Thread 290698			10.29	12.43	82.76	0.00	0.00	12.39
○Thread 290797			10.11	11.53	87.66	0.90	-0.00	11.18
○Thread 290798			10.03	11.53	87.01	0.90	-0.00	11.18
○Thread 290799			10.03	11.53	86.97	0.90	-0.00	11.18
○Thread 290800			10.08	11.53	87.40	0.90	-0.00	11.19
○Thread 290801			10.12	11.53	87.71	0.90	-0.00	11.18
○Thread 290802			10.08	11.53	87.45	0.90	-0.00	11.19
○Thread 290803			10.16	11.53	88.06	0.90	-0.00	11.18
○Thread 290804			10.14	11.53	87.93	0.90	-0.00	11.18
○Thread 290805			10.19	11.53	88.40	0.90	-0.00	11.18
○Thread 290806			10.20	11.53	88.40	0.90	-0.00	11.20
○Thread 290807			10.20	11.53	88.45	0.90	-0.00	11.18
○Thread 290808			10.05	11.53	87.11	0.90	-0.00	11.18
○Thread 290809			10.04	11.53	87.06	0.90	-0.00	11.17
○Thread 290810			10.15	11.53	88.02	0.90	-0.00	11.18
○Thread 290811			9.92	11.53	86.02	0.90	-0.00	11.18
○Thread 290812			10.03	11.53	87.02	0.90	-0.00	11.20
○Thread 290813			10.03	11.53	86.94	0.90	-0.00	11.18
○Thread 290814			10.14	11.53	87.89	0.90	-0.00	11.18
○Thread 290815			9.99	11.53	86.63	0.90	-0.00	11.18
○Thread 290816			10.08	11.53	87.37	0.90	-0.00	11.18
○Thread 290817			10.10	11.53	87.59	0.90	-0.00	11.18
○Thread 290818			10.27	11.53	89.07	0.90	-0.00	11.18
○Thread 290819			10.10	11.53	87.55	0.90	-0.00	11.18
○Thread 290820			9.94	11.53	86.16	0.90	-0.00	11.18
○Thread 290821			10.03	11.53	86.94	0.90	-0.00	11.18
○Thread 290822			9.98	11.53	86.55	0.90	-0.00	11.18
○Thread 290823			9.99	11.53	86.64	0.90	-0.00	11.18
○Thread 290824			10.04	11.53	87.08	0.90	-0.00	11.18
○Thread 290825			10.02	11.53	86.86	0.90	-0.00	11.18
○Thread 290826			10.04	11.53	87.08	0.90	-0.00	11.18
○Thread 290827			10.03	11.53	86.99	0.90	-0.00	11.18
○Thread 290828			10.15	11.53	88.03	0.90	-0.00	11.18
○Thread 290829			10.04	11.53	87.04	0.90	-0.00	11.18
○Thread 290830			10.14	11.53	87.91	0.90	-0.00	11.18
○Thread 290831			10.23	11.53	88.73	0.90	-0.00	11.18
○Thread 290832			9.99	11.53	86.61	0.90	-0.00	11.18
○Thread 290833			10.00	11.53	86.74	0.90	-0.00	11.18
○Thread 290834			10.04	11.53	87.08	0.90	-0.00	11.18
○Thread 290835			10.01	11.53	86.83	0.90	-0.00	11.18
○Thread 290836			10.11	11.53	87.69	0.90	-0.00	11.18
○Thread 290837			10.08	11.53	87.43	0.90	-0.00	11.18
○Thread 290838			10.20	11.53	88.43	0.90	-0.00	11.20
○Thread 290839			10.03	11.53	87.00	0.90	-0.00	11.18
○Thread 290840			10.23	11.53	88.74	0.91	-0.00	11.18
○Thread 290841			10.13	11.53	87.87	0.91	-0.00	11.18
○Thread 290842			10.23	11.53	88.78	0.91	-0.00	11.18
○Thread 290843			9.99	11.53	86.62	0.91	-0.00	11.18
○Thread 290844			9.99	11.53	86.66	0.91	-0.00	11.18
○Thread 290845			10.12	11.53	87.79	0.91	-0.00	11.18
○Thread 290846			10.10	11.53	87.62	0.91	-0.00	11.18
○Thread 290847			10.06	11.53	87.27	0.91	-0.00	11.17
○Thread 290848			10.07	11.53	87.36	0.91	-0.00	11.18
○Thread 290849			9.88	11.53	85.71	0.91	-0.00	11.18
○Thread 290850			10.11	11.53	87.66	0.91	-0.00	11.18
○Thread 290851			10.00	11.53	86.75	0.91	-0.00	11.18
○Thread 290852			10.17	11.53	88.18	0.91	-0.00	11.18
○Thread 290853			10.01	11.53	86.84	0.91	-0.00	11.18
○Thread 290854			10.05	11.53	87.15	0.91	-0.00	11.18
○Thread 290855			10.12	11.53	87.80	0.91	-0.00	11.18
○Thread 290856			9.91	11.53	85.98	0.91	-0.00	11.18
○Thread 290857			10.04	11.53	87.10	0.91	-0.00	11.18
○Thread 290858			10.12	11.53	87.80	0.91	-0.00	11.18
○Thread 290859			10.19	11.53	88.36	0.91	-0.00	11.19
○Thread 290860			10.13	11.53	87.89	0.91	-0.00	11.18
○Thread 290861			10.06	11.53	87.28	0.91	-0.00	11.18
○Thread 290862			10.14	11.53	88.02	0.91	-0.00	11.18
○Thread 290863			10.06	11.53	87.24	0.91	-0.00	11.18
○Thread 290864			10.04	11.53	87.15	0.91	-0.00	11.18
○Thread 290865			10.02	11.53	86.98	0.91	-0.00	11.18
○Thread 290866			10.02	11.53	86.94	0.91	-0.00	11.18
○Thread 290867			9.91	11.53	85.94	0.91	-0.00	11.18
○Thread 290868			10.06	11.53	87.24	0.91	-0.00	11.18
○Thread 290869			10.10	11.53	87.64	0.91	-0.00	11.18
○Thread 290870			10.14	11.52	87.94	0.91	-0.00	11.18
○Thread 290871			10.02	11.52	86.90	0.91	-0.00	11.18
○Thread 290872			9.97	11.52	86.47	0.91	-0.00	11.18
○Thread 290873			10.01	11.52	86.86	0.91	-0.00	11.18
○Thread 290874			9.95	11.52	86.34	0.91	-0.00	11.18
○Thread 290875			10.08	11.52	87.47	0.91	-0.00	11.18
○Thread 290876			10.07	11.52	87.38	0.91	-0.00	11.19
○Thread 290877			10.00	11.52	86.77	0.91	-0.00	11.18
○Thread 290878			10.06	11.52	87.30	0.91	-0.00	11.18
○Thread 290879			10.01	11.52	86.82	0.91	-0.00	11.19
○Thread 290880			10.04	11.52	87.08	0.91	-0.00	11.18
○Thread 290881			9.94	11.52	86.26	0.91	-0.00	11.18
○Thread 290882			10.03	11.52	87.08	0.91	-0.00	11.18
○Thread 290883			9.95	11.52	86.30	0.91	-0.00	11.18
○Thread 290884			10.04	11.52	87.08	0.91	-0.00	11.18
○Thread 290885			10.08	11.52	87.48	0.91	-0.00	11.18
○Thread 290886			10.15	11.52	88.04	0.91	-0.00	11.18
○Thread 290887			10.01	11.52	86.83	0.91	-0.00	11.18
○Thread 290888			10.13	11.52	87.87	0.91	-0.00	11.18
○Thread 290889			9.93	11.52	86.18	0.91	-0.00	11.18
○Thread 290890			10.17	11.52	88.22	0.91	-0.00	11.18
○Thread 290891			8.11	11.52	70.38	0.91	-0.00	11.18

Report Configuration