OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	32.81
►Process 881215+		1	32.81
○Thread 881215			32.81	34.35	95.54	0.00	0.00	27.13

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	16.70
►Process 881389+		2	16.70
○Thread 881389			16.70	18.27	91.41	0.00	0.00	16.47
○Thread 881488			16.20	16.47	98.30	1.79	0.00	16.23

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	8.75
►Process 881508+		4	8.75
○Thread 881508			8.75	10.29	85.06	0.00	0.00	8.47
○Thread 881607			8.19	8.48	96.63	1.81	0.00	8.24
○Thread 881608			8.20	8.48	96.68	1.81	0.00	8.24
○Thread 881609			8.22	8.48	96.98	1.81	0.00	8.23

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	4.76
►Process 881629+		8	4.76
○Thread 881629			4.76	6.29	75.64	0.00	0.00	4.47
○Thread 881728			4.18	4.47	93.48	1.81	0.00	4.23
○Thread 881729			4.20	4.47	93.93	1.81	0.00	4.23
○Thread 881730			4.20	4.47	93.92	1.81	0.00	4.23
○Thread 881731			4.20	4.47	94.02	1.81	0.00	4.23
○Thread 881732			4.19	4.47	93.68	1.81	0.00	4.23
○Thread 881733			4.21	4.47	94.12	1.81	0.00	4.23
○Thread 881734			4.20	4.47	94.00	1.81	0.00	4.23

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	2.81
►Process 881803+		16	2.81
○Thread 881803			2.81	4.30	65.37	0.00	0.00	2.48
○Thread 881902			2.22	2.48	89.48	1.81	0.01	2.24
○Thread 881903			2.20	2.48	88.66	1.81	0.01	2.24
○Thread 881904			2.21	2.48	89.26	1.81	0.01	2.24
○Thread 881905			2.22	2.48	89.65	1.81	0.01	2.24
○Thread 881906			2.17	2.48	87.63	1.81	0.00	2.24
○Thread 881907			2.20	2.48	88.63	1.81	0.00	2.24
○Thread 881908			2.22	2.48	89.41	1.81	0.00	2.24
○Thread 881909			2.19	2.48	88.20	1.81	0.00	2.24
○Thread 881910			2.24	2.48	90.20	1.81	0.00	2.24
○Thread 881911			2.20	2.48	88.59	1.81	0.00	2.24
○Thread 881912			2.20	2.48	88.58	1.81	0.00	2.24
○Thread 881913			2.22	2.48	89.37	1.81	0.00	2.24
○Thread 881914			2.19	2.48	88.36	1.81	0.00	2.24
○Thread 881915			2.23	2.48	89.76	1.81	0.00	2.24
○Thread 881916			2.20	2.48	88.75	1.81	0.00	2.24

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	2.16
►Process 881936+		24	2.16
○Thread 881936			2.16	3.75	57.63	0.00	0.00	1.93
○Thread 882035			1.64	1.94	84.69	1.80	0.01	1.70
○Thread 882036			1.66	1.94	85.45	1.80	0.01	1.70
○Thread 882037			1.66	1.94	85.44	1.80	0.01	1.70
○Thread 882038			1.67	1.94	85.94	1.80	0.01	1.70
○Thread 882039			1.66	1.94	85.68	1.80	0.01	1.70
○Thread 882040			1.67	1.94	86.18	1.80	0.01	1.70
○Thread 882041			1.66	1.94	85.40	1.80	0.01	1.70
○Thread 882042			1.67	1.94	86.16	1.80	0.01	1.70
○Thread 882043			1.64	1.94	84.61	1.80	0.01	1.70
○Thread 882044			1.66	1.94	85.37	1.80	0.01	1.70
○Thread 882045			1.66	1.94	85.62	1.80	0.01	1.70
○Thread 882046			1.65	1.94	85.10	1.80	0.00	1.70
○Thread 882047			1.66	1.95	85.60	1.80	0.00	1.70
○Thread 882048			1.67	1.95	85.85	1.80	0.00	1.70
○Thread 882049			1.68	1.95	86.61	1.80	0.00	1.70
○Thread 882050			1.65	1.95	85.06	1.80	0.00	1.70
○Thread 882051			1.67	1.95	86.07	1.80	0.00	1.70
○Thread 882052			1.65	1.95	84.78	1.80	0.00	1.70
○Thread 882053			1.66	1.95	85.54	1.80	0.00	1.70
○Thread 882054			1.66	1.95	85.53	1.80	0.00	1.70
○Thread 882055			1.64	1.95	84.49	1.80	0.00	1.70
○Thread 882056			1.66	1.95	85.25	1.80	0.00	1.70
○Thread 882057			1.58	1.95	81.14	1.80	0.00	1.69

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	1.87
►Process 882077+		32	1.87
○Thread 882077			1.87	3.45	54.18	0.00	0.00	1.78
○Thread 882176			1.35	1.64	82.41	1.80	0.01	1.40
○Thread 882177			1.38	1.64	84.22	1.80	0.01	1.40
○Thread 882178			1.35	1.64	82.09	1.80	0.01	1.40
○Thread 882179			1.37	1.64	83.29	1.80	0.01	1.40
○Thread 882180			1.37	1.65	83.28	1.80	0.01	1.40
○Thread 882181			1.39	1.65	84.48	1.80	0.01	1.40
○Thread 882182			1.35	1.65	82.35	1.80	0.01	1.40
○Thread 882183			1.38	1.65	84.16	1.80	0.01	1.40
○Thread 882184			1.36	1.65	82.63	1.80	0.01	1.40
○Thread 882185			1.36	1.65	82.92	1.80	0.01	1.40
○Thread 882186			1.36	1.65	82.61	1.80	0.01	1.39
○Thread 882187			1.38	1.65	84.11	1.80	0.01	1.40
○Thread 882188			1.36	1.65	82.89	1.80	0.01	1.40
○Thread 882189			1.32	1.65	80.45	1.80	0.01	1.39
○Thread 882190			1.37	1.65	83.47	1.80	0.01	1.40
○Thread 882191			1.36	1.65	82.55	1.80	0.01	1.40
○Thread 882192			1.36	1.65	82.84	1.80	0.01	1.40
○Thread 882193			1.37	1.65	83.44	1.80	0.01	1.40
○Thread 882194			1.39	1.65	84.34	1.80	0.00	1.40
○Thread 882195			1.37	1.65	83.11	1.80	0.00	1.40
○Thread 882196			1.37	1.65	83.41	1.80	0.00	1.39
○Thread 882197			1.37	1.65	83.39	1.80	0.00	1.40
○Thread 882198			1.34	1.65	81.56	1.80	0.00	1.40
○Thread 882199			1.38	1.65	83.98	1.80	0.00	1.40
○Thread 882200			1.35	1.65	82.15	1.80	0.00	1.40
○Thread 882201			1.38	1.65	83.65	1.80	0.00	1.40
○Thread 882202			1.37	1.65	83.04	1.80	0.00	1.40
○Thread 882203			1.36	1.65	82.72	1.80	0.00	1.40
○Thread 882204			1.35	1.65	82.10	1.80	0.00	1.40
○Thread 882205			1.34	1.65	81.19	1.80	0.00	1.40
○Thread 882206			1.38	1.65	83.90	1.80	0.00	1.40

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	1.74
►Process 882226+		40	1.74
○Thread 882226			1.74	3.28	53.03	0.00	0.00	1.79
○Thread 882325			1.17	1.45	80.42	1.81	0.01	1.21
○Thread 882326			1.17	1.46	80.75	1.81	0.01	1.21
○Thread 882327			1.16	1.46	80.06	1.81	0.01	1.21
○Thread 882328			1.16	1.46	80.04	1.81	0.01	1.21
○Thread 882329			1.16	1.46	79.69	1.81	0.01	1.21
○Thread 882330			1.17	1.46	80.71	1.81	0.01	1.21
○Thread 882331			1.15	1.46	78.98	1.81	0.01	1.21
○Thread 882332			1.16	1.46	80.00	1.81	0.01	1.21
○Thread 882333			1.17	1.46	80.33	1.81	0.01	1.21
○Thread 882334			1.17	1.46	80.66	1.81	0.01	1.21
○Thread 882335			1.17	1.46	80.65	1.81	0.01	1.21
○Thread 882336			1.16	1.46	79.61	1.81	0.01	1.21
○Thread 882337			1.16	1.46	79.59	1.81	0.01	1.21
○Thread 882338			1.17	1.46	80.61	1.81	0.01	1.21
○Thread 882339			1.15	1.46	79.23	1.81	0.01	1.20
○Thread 882340			1.15	1.46	79.21	1.81	0.01	1.21
○Thread 882341			1.15	1.46	79.20	1.81	0.01	1.21
○Thread 882342			1.16	1.46	79.53	1.81	0.01	1.21
○Thread 882343			1.16	1.46	79.52	1.81	0.01	1.21
○Thread 882344			1.16	1.46	79.51	1.81	0.01	1.20
○Thread 882345			1.17	1.46	80.18	1.82	0.01	1.21
○Thread 882346			1.16	1.46	79.83	1.82	0.01	1.21
○Thread 882347			1.15	1.46	78.80	1.82	0.01	1.20
○Thread 882348			1.15	1.46	79.12	1.82	0.01	1.21
○Thread 882349			1.17	1.46	80.14	1.82	0.01	1.21
○Thread 882350			1.15	1.46	79.10	1.82	0.01	1.21
○Thread 882351			1.15	1.46	78.75	1.82	0.01	1.21
○Thread 882352			1.18	1.46	80.79	1.82	0.00	1.21
○Thread 882353			1.19	1.46	81.46	1.82	0.00	1.21
○Thread 882354			1.19	1.46	81.79	1.82	0.00	1.21
○Thread 882355			1.18	1.46	80.75	1.82	0.00	1.21
○Thread 882356			1.16	1.46	79.71	1.82	0.00	1.21
○Thread 882357			1.18	1.46	80.73	1.82	0.00	1.21
○Thread 882358			1.17	1.46	80.38	1.82	0.00	1.21
○Thread 882359			1.14	1.46	77.97	1.82	0.00	1.21
○Thread 882360			1.16	1.46	79.67	1.82	0.00	1.20
○Thread 882361			1.17	1.46	79.99	1.82	0.00	1.21
○Thread 882362			1.17	1.46	79.98	1.82	0.00	1.21
○Thread 882363			1.10	1.46	75.19	1.82	0.00	1.21

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	1.68
►Process 882383+		48	1.68
○Thread 882383			1.68	3.17	52.93	0.00	0.00	1.80
○Thread 882482			1.04	1.33	78.23	1.83	0.02	1.08
○Thread 882483			1.03	1.33	77.47	1.83	0.02	1.08
○Thread 882484			1.04	1.33	78.59	1.83	0.02	1.08
○Thread 882485			1.01	1.33	76.32	1.83	0.02	1.08
○Thread 882486			1.02	1.33	77.06	1.83	0.02	1.08
○Thread 882487			1.05	1.33	79.30	1.83	0.02	1.08
○Thread 882488			1.06	1.33	79.66	1.83	0.01	1.08
○Thread 882489			1.05	1.33	78.90	1.83	0.01	1.08
○Thread 882490			1.04	1.33	78.51	1.83	0.01	1.08
○Thread 882491			1.02	1.33	76.62	1.83	0.01	1.08
○Thread 882492			1.05	1.33	79.24	1.83	0.01	1.08
○Thread 882493			1.06	1.33	79.60	1.83	0.01	1.08
○Thread 882494			1.02	1.33	76.58	1.83	0.01	1.08
○Thread 882495			1.04	1.33	78.45	1.83	0.01	1.08
○Thread 882496			1.04	1.33	78.06	1.83	0.01	1.08
○Thread 882497			1.05	1.33	78.80	1.83	0.01	1.08
○Thread 882498			1.00	1.33	74.66	1.83	0.01	1.07
○Thread 882499			1.03	1.33	77.65	1.83	0.01	1.08
○Thread 882500			1.01	1.33	76.13	1.83	0.01	1.08
○Thread 882501			1.03	1.33	77.62	1.83	0.01	1.08
○Thread 882502			1.04	1.33	77.98	1.83	0.01	1.08
○Thread 882503			1.02	1.33	76.47	1.83	0.01	1.08
○Thread 882504			1.03	1.33	77.21	1.83	0.01	1.08
○Thread 882505			1.03	1.33	77.57	1.83	0.01	1.07
○Thread 882506			1.01	1.33	75.68	1.83	0.01	1.08
○Thread 882507			1.02	1.33	76.42	1.83	0.01	1.08
○Thread 882508			1.03	1.33	77.53	1.83	0.01	1.08
○Thread 882509			1.02	1.34	76.77	1.83	0.01	1.07
○Thread 882510			1.03	1.34	77.13	1.83	0.01	1.07
○Thread 882511			1.04	1.34	77.87	1.83	0.01	1.08
○Thread 882512			1.02	1.34	76.74	1.83	0.01	1.08
○Thread 882513			1.03	1.34	77.47	1.83	0.01	1.08
○Thread 882514			1.00	1.34	75.21	1.83	0.01	1.06
○Thread 882515			1.01	1.34	75.95	1.83	0.01	1.08
○Thread 882516			1.02	1.34	76.68	1.83	0.01	1.07
○Thread 882517			1.05	1.34	78.54	1.83	0.00	1.08
○Thread 882518			1.03	1.34	77.03	1.83	0.00	1.07
○Thread 882519			1.04	1.34	78.14	1.83	0.00	1.08
○Thread 882520			1.04	1.34	77.75	1.83	0.00	1.08
○Thread 882521			1.03	1.34	76.99	1.83	0.00	1.08
○Thread 882522			1.03	1.34	77.35	1.83	0.00	1.08
○Thread 882523			1.03	1.34	77.34	1.83	0.00	1.08
○Thread 882524			1.04	1.34	78.08	1.83	0.00	1.08
○Thread 882525			1.01	1.34	75.45	1.83	0.00	1.08
○Thread 882526			1.03	1.34	76.93	1.83	0.00	1.08
○Thread 882527			1.01	1.34	75.80	1.83	0.00	1.08
○Thread 882528			0.99	1.34	73.54	1.83	0.00	1.08

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	1.48
►Process 882596+		56	1.48
○Thread 882596			1.48	3.06	48.29	0.00	0.00	1.77
○Thread 882695			0.94	1.24	75.68	1.81	0.02	0.98
○Thread 882696			0.90	1.24	72.83	1.81	0.02	0.98
○Thread 882697			0.93	1.24	74.84	1.81	0.02	0.97
○Thread 882698			0.95	1.24	76.85	1.81	0.02	0.98
○Thread 882699			0.94	1.24	76.03	1.81	0.02	0.98
○Thread 882700			0.93	1.24	74.80	1.81	0.02	0.98
○Thread 882701			0.93	1.24	75.20	1.81	0.02	0.98
○Thread 882702			0.92	1.24	74.37	1.81	0.02	0.98
○Thread 882703			0.94	1.24	75.98	1.81	0.02	0.98
○Thread 882704			0.94	1.24	75.56	1.81	0.02	0.99
○Thread 882705			0.94	1.24	75.95	1.81	0.02	0.97
○Thread 882706			0.91	1.24	73.51	1.81	0.02	0.98
○Thread 882707			0.92	1.24	74.31	1.81	0.02	0.97
○Thread 882708			0.93	1.24	75.10	1.81	0.02	0.98
○Thread 882709			0.92	1.24	74.28	1.81	0.01	0.97
○Thread 882710			0.95	1.24	76.69	1.81	0.01	0.98
○Thread 882711			0.90	1.24	73.04	1.81	0.01	0.97
○Thread 882712			0.91	1.24	73.43	1.81	0.01	0.98
○Thread 882713			0.93	1.24	75.03	1.81	0.01	0.98
○Thread 882714			0.94	1.24	75.42	1.81	0.01	0.97
○Thread 882715			0.93	1.24	75.01	1.81	0.01	0.98
○Thread 882716			0.95	1.24	76.61	1.81	0.01	0.97
○Thread 882717			0.93	1.24	74.98	1.81	0.01	0.98
○Thread 882718			0.93	1.24	74.97	1.81	0.01	0.97
○Thread 882719			0.93	1.24	74.95	1.81	0.01	0.97
○Thread 882720			0.95	1.24	76.55	1.81	0.01	0.99
○Thread 882721			0.95	1.24	76.94	1.81	0.01	0.98
○Thread 882722			0.90	1.24	72.90	1.81	0.01	0.98
○Thread 882723			0.92	1.24	73.69	1.81	0.01	0.98
○Thread 882724			0.93	1.24	74.48	1.81	0.01	0.98
○Thread 882725			0.91	1.24	73.26	1.81	0.01	0.98
○Thread 882726			0.94	1.24	75.26	1.81	0.01	0.98
○Thread 882727			0.94	1.24	75.25	1.81	0.01	0.98
○Thread 882728			0.93	1.24	74.43	1.81	0.01	0.98
○Thread 882729			0.93	1.24	74.42	1.81	0.01	0.98
○Thread 882730			0.92	1.24	73.60	1.81	0.01	0.97
○Thread 882731			0.94	1.24	75.60	1.81	0.01	0.99
○Thread 882732			0.95	1.24	76.39	1.81	0.01	0.98
○Thread 882733			0.92	1.24	73.96	1.81	0.01	0.98
○Thread 882734			0.92	1.24	73.95	1.81	0.01	0.98
○Thread 882735			0.94	1.24	75.55	1.81	0.01	0.98
○Thread 882736			0.93	1.24	74.73	1.81	0.01	0.99
○Thread 882737			0.93	1.24	74.31	1.81	0.01	0.98
○Thread 882738			0.93	1.24	74.30	1.81	0.00	0.96
○Thread 882739			0.94	1.25	75.09	1.82	0.00	0.99
○Thread 882740			0.96	1.25	77.08	1.82	0.00	0.98
○Thread 882741			0.95	1.25	76.67	1.82	0.00	0.98
○Thread 882742			0.95	1.25	76.65	1.82	0.00	0.98
○Thread 882743			0.92	1.25	73.83	1.82	0.00	0.97
○Thread 882744			0.90	1.25	72.61	1.82	0.00	0.94
○Thread 882745			0.94	1.25	75.81	1.82	0.00	0.98
○Thread 882746			0.92	1.25	73.39	1.82	0.00	0.98
○Thread 882747			0.90	1.25	72.17	1.82	0.00	0.98
○Thread 882748			0.85	1.25	67.75	1.82	0.00	0.97
○Thread 882749			0.91	1.25	72.95	1.82	0.00	0.98

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	1.40
►Process 882769+		64	1.40
○Thread 882769			1.40	2.99	46.98	0.00	0.00	1.79
○Thread 882868			0.83	1.15	72.46	1.82	0.02	0.89
○Thread 882869			0.85	1.15	73.32	1.82	0.02	0.90
○Thread 882870			0.86	1.15	74.61	1.82	0.02	0.90
○Thread 882871			0.84	1.15	72.86	1.82	0.02	0.90
○Thread 882872			0.85	1.15	73.71	1.82	0.02	0.90
○Thread 882873			0.83	1.15	72.40	1.82	0.02	0.90
○Thread 882874			0.83	1.15	72.38	1.82	0.02	0.88
○Thread 882875			0.88	1.15	75.84	1.82	0.02	0.90
○Thread 882876			0.83	1.15	71.92	1.82	0.02	0.89
○Thread 882877			0.84	1.15	72.78	1.82	0.02	0.90
○Thread 882878			0.87	1.15	74.93	1.82	0.02	0.90
○Thread 882879			0.85	1.15	73.18	1.82	0.02	0.90
○Thread 882880			0.83	1.15	71.87	1.82	0.02	0.89
○Thread 882881			0.84	1.16	72.72	1.82	0.02	0.88
○Thread 882882			0.86	1.16	74.44	1.82	0.02	0.89
○Thread 882883			0.85	1.16	73.56	1.82	0.02	0.90
○Thread 882884			0.83	1.16	71.81	1.82	0.02	0.89
○Thread 882885			0.83	1.16	71.80	1.82	0.02	0.90
○Thread 882886			0.85	1.16	73.08	1.82	0.02	0.90
○Thread 882887			0.87	1.16	75.23	1.82	0.02	0.90
○Thread 882888			0.82	1.16	71.32	1.82	0.02	0.90
○Thread 882889			0.82	1.16	71.31	1.82	0.02	0.89
○Thread 882890			0.86	1.16	74.32	1.82	0.01	0.90
○Thread 882891			0.82	1.16	71.28	1.82	0.01	0.90
○Thread 882892			0.84	1.16	72.56	1.82	0.01	0.90
○Thread 882893			0.86	1.16	73.84	1.82	0.01	0.89
○Thread 882894			0.82	1.16	70.81	1.82	0.01	0.90
○Thread 882895			0.85	1.16	72.95	1.82	0.01	0.90
○Thread 882896			0.80	1.16	69.05	1.82	0.01	0.90
○Thread 882897			0.83	1.16	71.63	1.82	0.01	0.89
○Thread 882898			0.85	1.16	72.91	1.82	0.01	0.90
○Thread 882899			0.83	1.16	71.60	1.82	0.01	0.90
○Thread 882900			0.87	1.16	74.61	1.82	0.01	0.90
○Thread 882901			0.83	1.16	72.01	1.82	0.01	0.90
○Thread 882902			0.83	1.16	72.00	1.82	0.01	0.90
○Thread 882903			0.86	1.16	73.71	1.82	0.01	0.90
○Thread 882904			0.78	1.16	67.23	1.82	0.01	0.89
○Thread 882905			0.82	1.16	70.66	1.82	0.01	0.89
○Thread 882906			0.83	1.16	71.94	1.82	0.01	0.90
○Thread 882907			0.82	1.16	70.64	1.82	0.01	0.89
○Thread 882908			0.85	1.16	73.21	1.82	0.01	0.90
○Thread 882909			0.82	1.16	71.04	1.82	0.01	0.90
○Thread 882910			0.85	1.16	73.18	1.82	0.01	0.90
○Thread 882911			0.85	1.16	72.73	1.82	0.01	0.89
○Thread 882912			0.81	1.16	69.71	1.82	0.01	0.90
○Thread 882913			0.85	1.16	73.14	1.82	0.01	0.90
○Thread 882914			0.83	1.16	71.83	1.82	0.01	0.90
○Thread 882915			0.81	1.16	70.10	1.82	0.01	0.88
○Thread 882916			0.81	1.16	70.09	1.82	0.01	0.90
○Thread 882917			0.85	1.16	73.08	1.82	0.01	0.90
○Thread 882918			0.82	1.16	70.92	1.82	0.01	0.90
○Thread 882919			0.83	1.16	71.76	1.82	0.00	0.90
○Thread 882920			0.84	1.16	72.18	1.82	0.00	0.90
○Thread 882921			0.83	1.16	71.31	1.82	0.00	0.90
○Thread 882922			0.83	1.16	71.29	1.82	0.00	0.89
○Thread 882923			0.87	1.16	74.28	1.82	0.00	0.90
○Thread 882924			0.84	1.16	72.12	1.82	0.00	0.89
○Thread 882925			0.87	1.16	74.26	1.82	0.00	0.88
○Thread 882926			0.83	1.17	71.24	1.82	0.00	0.89
○Thread 882927			0.83	1.17	71.22	1.82	0.00	0.90
○Thread 882928			0.86	1.17	73.35	1.82	0.00	0.90
○Thread 882929			0.82	1.17	70.34	1.82	0.00	0.88
○Thread 882930			0.83	1.17	71.61	1.82	0.00	0.90

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	1.35
►Process 882950+		72	1.35
○Thread 882950			1.35	2.97	45.42	0.00	0.00	1.81
○Thread 883049			0.80	1.11	71.57	1.84	0.03	0.86
○Thread 883050			0.81	1.11	73.36	1.84	0.03	0.86
○Thread 883051			0.80	1.11	71.99	1.84	0.03	0.85
○Thread 883052			0.81	1.11	72.43	1.84	0.02	0.86
○Thread 883053			0.81	1.11	72.42	1.84	0.02	0.86
○Thread 883054			0.79	1.11	71.05	1.84	0.02	0.86
○Thread 883055			0.81	1.11	72.39	1.84	0.02	0.86
○Thread 883056			0.81	1.11	72.82	1.84	0.02	0.86
○Thread 883057			0.79	1.11	71.01	1.84	0.02	0.86
○Thread 883058			0.81	1.11	72.79	1.84	0.02	0.86
○Thread 883059			0.82	1.11	74.13	1.84	0.02	0.86
○Thread 883060			0.79	1.11	70.52	1.84	0.02	0.86
○Thread 883061			0.80	1.11	71.40	1.84	0.02	0.86
○Thread 883062			0.75	1.11	67.80	1.84	0.02	0.86
○Thread 883063			0.82	1.11	74.07	1.84	0.02	0.86
○Thread 883064			0.80	1.11	71.81	1.84	0.02	0.85
○Thread 883065			0.82	1.11	73.59	1.84	0.02	0.85
○Thread 883066			0.83	1.11	74.47	1.84	0.02	0.85
○Thread 883067			0.76	1.11	68.63	1.84	0.02	0.86
○Thread 883068			0.80	1.11	71.30	1.84	0.02	0.85
○Thread 883069			0.81	1.12	72.19	1.84	0.02	0.85
○Thread 883070			0.80	1.12	71.28	1.84	0.02	0.86
○Thread 883071			0.79	1.12	70.81	1.84	0.02	0.86
○Thread 883072			0.80	1.12	71.25	1.84	0.02	0.86
○Thread 883073			0.80	1.12	71.68	1.84	0.02	0.86
○Thread 883074			0.77	1.12	69.43	1.84	0.02	0.84
○Thread 883075			0.81	1.12	72.55	1.84	0.02	0.86
○Thread 883076			0.79	1.12	70.75	1.84	0.02	0.85
○Thread 883077			0.80	1.12	71.63	1.84	0.02	0.86
○Thread 883078			0.76	1.12	68.48	1.84	0.02	0.85
○Thread 883079			0.79	1.12	70.70	1.84	0.02	0.86
○Thread 883080			0.80	1.12	71.14	1.84	0.01	0.86
○Thread 883081			0.76	1.12	68.44	1.84	0.01	0.86
○Thread 883082			0.80	1.12	71.11	1.84	0.01	0.85
○Thread 883083			0.77	1.12	69.31	1.84	0.01	0.86
○Thread 883084			0.77	1.12	68.85	1.84	0.01	0.86
○Thread 883085			0.80	1.12	71.51	1.84	0.01	0.86
○Thread 883086			0.79	1.12	70.16	1.84	0.01	0.86
○Thread 883087			0.80	1.12	71.04	1.84	0.01	0.86
○Thread 883088			0.80	1.12	71.47	1.84	0.01	0.86
○Thread 883089			0.79	1.12	70.12	1.84	0.01	0.85
○Thread 883090			0.78	1.12	69.66	1.84	0.01	0.86
○Thread 883091			0.80	1.12	70.98	1.84	0.01	0.85
○Thread 883092			0.80	1.12	71.41	1.84	0.01	0.86
○Thread 883093			0.81	1.12	72.74	1.84	0.01	0.86
○Thread 883094			0.80	1.12	70.94	1.84	0.01	0.85
○Thread 883095			0.78	1.12	69.59	1.84	0.01	0.85
○Thread 883096			0.80	1.12	71.36	1.84	0.01	0.86
○Thread 883097			0.80	1.12	70.90	1.84	0.01	0.85
○Thread 883098			0.80	1.12	71.33	1.84	0.01	0.85
○Thread 883099			0.82	1.12	73.54	1.84	0.01	0.85
○Thread 883100			0.79	1.12	70.41	1.84	0.01	0.85
○Thread 883101			0.81	1.12	71.73	1.84	0.01	0.86
○Thread 883102			0.81	1.12	71.71	1.84	0.01	0.86
○Thread 883103			0.80	1.12	71.25	1.84	0.01	0.85
○Thread 883104			0.80	1.12	70.79	1.84	0.01	0.86
○Thread 883105			0.80	1.12	70.78	1.84	0.01	0.85
○Thread 883106			0.83	1.12	74.32	1.84	0.01	0.86
○Thread 883107			0.81	1.12	72.08	1.84	0.01	0.86
○Thread 883108			0.81	1.12	72.07	1.84	0.00	0.86
○Thread 883109			0.81	1.12	72.50	1.84	0.00	0.85
○Thread 883110			0.81	1.12	72.04	1.84	0.00	0.86
○Thread 883111			0.81	1.12	72.47	1.84	0.00	0.86
○Thread 883112			0.81	1.12	71.57	1.84	0.00	0.86
○Thread 883113			0.79	1.13	70.22	1.84	0.00	0.85
○Thread 883114			0.81	1.13	72.43	1.84	0.00	0.85
○Thread 883115			0.81	1.13	71.53	1.84	0.00	0.85
○Thread 883116			0.76	1.13	67.51	1.84	0.00	0.85
○Thread 883117			0.75	1.13	67.05	1.84	0.00	0.84
○Thread 883118			0.75	1.13	67.04	1.84	0.00	0.84
○Thread 883119			0.75	1.13	67.03	1.84	0.00	0.86

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	1.37
►Process 883139+		80	1.37
○Thread 883139			1.37	2.91	47.12	0.00	0.00	1.73
○Thread 883238			0.75	1.06	71.31	1.82	0.03	0.81
○Thread 883239			0.75	1.06	71.30	1.82	0.03	0.80
○Thread 883240			0.75	1.06	71.28	1.82	0.03	0.81
○Thread 883241			0.75	1.06	70.32	1.82	0.03	0.79
○Thread 883242			0.75	1.06	70.78	1.82	0.03	0.81
○Thread 883243			0.75	1.06	71.24	1.82	0.03	0.81
○Thread 883244			0.76	1.06	72.16	1.82	0.03	0.81
○Thread 883245			0.75	1.06	71.21	1.82	0.03	0.81
○Thread 883246			0.76	1.06	71.66	1.82	0.03	0.81
○Thread 883247			0.73	1.06	68.82	1.82	0.03	0.81
○Thread 883248			0.75	1.06	70.21	1.82	0.03	0.81
○Thread 883249			0.75	1.06	71.14	1.82	0.03	0.81
○Thread 883250			0.75	1.06	70.66	1.82	0.02	0.81
○Thread 883251			0.74	1.06	69.70	1.82	0.02	0.79
○Thread 883252			0.75	1.06	70.62	1.82	0.02	0.81
○Thread 883253			0.74	1.06	69.19	1.82	0.02	0.79
○Thread 883254			0.76	1.06	71.53	1.82	0.02	0.80
○Thread 883255			0.75	1.06	70.58	1.82	0.02	0.80
○Thread 883256			0.75	1.06	70.56	1.82	0.02	0.80
○Thread 883257			0.75	1.06	70.07	1.82	0.02	0.80
○Thread 883258			0.74	1.06	69.12	1.82	0.02	0.80
○Thread 883259			0.75	1.06	70.04	1.82	0.02	0.80
○Thread 883260			0.75	1.06	70.03	1.82	0.02	0.81
○Thread 883261			0.75	1.06	70.95	1.82	0.02	0.81
○Thread 883262			0.75	1.06	70.47	1.82	0.02	0.80
○Thread 883263			0.75	1.06	70.46	1.82	0.02	0.80
○Thread 883264			0.75	1.06	69.97	1.82	0.02	0.81
○Thread 883265			0.79	1.06	74.18	1.82	0.02	0.81
○Thread 883266			0.70	1.07	65.72	1.82	0.02	0.79
○Thread 883267			0.73	1.07	68.52	1.82	0.02	0.80
○Thread 883268			0.76	1.07	71.32	1.82	0.02	0.81
○Thread 883269			0.74	1.07	69.43	1.82	0.02	0.81
○Thread 883270			0.75	1.07	69.88	1.82	0.02	0.81
○Thread 883271			0.76	1.07	71.74	1.82	0.02	0.81
○Thread 883272			0.76	1.07	71.73	1.82	0.02	0.80
○Thread 883273			0.75	1.07	69.84	1.82	0.02	0.80
○Thread 883274			0.76	1.07	71.70	1.82	0.02	0.80
○Thread 883275			0.73	1.07	68.41	1.82	0.02	0.80
○Thread 883276			0.74	1.07	68.86	1.82	0.02	0.80
○Thread 883277			0.74	1.07	68.84	1.82	0.02	0.80
○Thread 883278			0.73	1.07	68.36	1.82	0.01	0.79
○Thread 883279			0.74	1.07	68.81	1.82	0.01	0.80
○Thread 883280			0.75	1.07	69.73	1.82	0.01	0.80
○Thread 883281			0.74	1.07	68.78	1.82	0.01	0.80
○Thread 883282			0.74	1.07	68.77	1.83	0.01	0.79
○Thread 883283			0.75	1.07	69.69	1.83	0.01	0.79
○Thread 883284			0.70	1.07	65.93	1.83	0.01	0.81
○Thread 883285			0.75	1.07	70.12	1.83	0.01	0.80
○Thread 883286			0.75	1.07	70.11	1.83	0.01	0.81
○Thread 883287			0.75	1.07	70.56	1.83	0.01	0.80
○Thread 883288			0.75	1.07	70.08	1.83	0.01	0.79
○Thread 883289			0.73	1.07	68.19	1.83	0.01	0.79
○Thread 883290			0.71	1.07	66.31	1.83	0.01	0.79
○Thread 883291			0.75	1.07	69.56	1.83	0.01	0.81
○Thread 883292			0.75	1.07	70.01	1.83	0.01	0.79
○Thread 883293			0.73	1.07	67.67	1.83	0.01	0.80
○Thread 883294			0.75	1.07	69.99	1.83	0.01	0.80
○Thread 883295			0.74	1.07	68.57	1.83	0.01	0.80
○Thread 883296			0.74	1.07	68.55	1.83	0.01	0.80
○Thread 883297			0.75	1.07	69.94	1.83	0.01	0.80
○Thread 883298			0.73	1.07	67.59	1.83	0.01	0.80
○Thread 883299			0.74	1.07	68.51	1.83	0.01	0.81
○Thread 883300			0.72	1.07	67.10	1.83	0.01	0.80
○Thread 883301			0.75	1.07	69.41	1.83	0.01	0.80
○Thread 883302			0.76	1.07	70.79	1.83	0.01	0.81
○Thread 883303			0.75	1.07	70.31	1.83	0.01	0.76
○Thread 883304			0.75	1.07	70.29	1.83	0.01	0.81
○Thread 883305			0.72	1.07	67.02	1.83	0.01	0.79
○Thread 883306			0.77	1.07	71.66	1.83	0.00	0.81
○Thread 883307			0.75	1.07	69.78	1.83	0.00	0.79
○Thread 883308			0.75	1.08	69.76	1.83	0.00	0.81
○Thread 883309			0.75	1.08	69.75	1.83	0.00	0.81
○Thread 883310			0.74	1.08	68.34	1.83	0.00	0.77
○Thread 883311			0.72	1.08	66.93	1.83	0.00	0.81
○Thread 883312			0.73	1.08	67.84	1.83	0.00	0.78
○Thread 883313			0.75	1.08	70.14	1.83	0.00	0.81
○Thread 883314			0.75	1.08	70.13	1.83	0.00	0.80
○Thread 883315			0.73	1.08	67.79	1.83	0.00	0.81
○Thread 883316			0.69	1.08	64.52	1.83	0.00	0.80

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	1.30
►Process 883336+		88	1.30
○Thread 883336			1.30	2.88	45.21	0.00	0.00	1.79
○Thread 883435			0.75	1.03	72.40	1.81	0.03	0.77
○Thread 883436			0.75	1.03	72.88	1.82	0.03	0.77
○Thread 883437			0.72	1.03	69.95	1.82	0.03	0.78
○Thread 883438			0.74	1.03	71.39	1.82	0.03	0.78
○Thread 883439			0.73	1.03	70.41	1.82	0.03	0.76
○Thread 883440			0.73	1.03	70.39	1.82	0.03	0.78
○Thread 883441			0.71	1.03	69.41	1.82	0.03	0.77
○Thread 883442			0.71	1.03	69.39	1.82	0.03	0.77
○Thread 883443			0.71	1.03	69.38	1.82	0.03	0.76
○Thread 883444			0.72	1.03	69.85	1.82	0.03	0.78
○Thread 883445			0.72	1.03	69.83	1.82	0.03	0.78
○Thread 883446			0.72	1.03	69.82	1.82	0.03	0.78
○Thread 883447			0.70	1.03	68.35	1.82	0.03	0.77
○Thread 883448			0.71	1.03	68.82	1.82	0.03	0.76
○Thread 883449			0.74	1.03	71.22	1.82	0.03	0.78
○Thread 883450			0.69	1.03	66.36	1.82	0.03	0.76
○Thread 883451			0.75	1.03	72.64	1.82	0.03	0.78
○Thread 883452			0.70	1.03	67.79	1.82	0.03	0.76
○Thread 883453			0.72	1.03	69.71	1.82	0.03	0.76
○Thread 883454			0.70	1.03	68.24	1.82	0.02	0.78
○Thread 883455			0.70	1.03	67.74	1.82	0.02	0.78
○Thread 883456			0.70	1.03	67.73	1.82	0.02	0.77
○Thread 883457			0.71	1.03	68.68	1.82	0.02	0.78
○Thread 883458			0.74	1.03	71.09	1.82	0.02	0.77
○Thread 883459			0.71	1.03	68.66	1.82	0.02	0.78
○Thread 883460			0.69	1.03	66.23	1.82	0.02	0.75
○Thread 883461			0.71	1.03	68.63	1.82	0.02	0.78
○Thread 883462			0.73	1.03	70.55	1.82	0.02	0.78
○Thread 883463			0.74	1.03	71.50	1.82	0.02	0.78
○Thread 883464			0.70	1.04	68.11	1.82	0.02	0.78
○Thread 883465			0.69	1.04	66.16	1.82	0.02	0.78
○Thread 883466			0.73	1.04	70.01	1.82	0.02	0.76
○Thread 883467			0.72	1.04	69.51	1.82	0.02	0.76
○Thread 883468			0.71	1.04	68.53	1.82	0.02	0.78
○Thread 883469			0.70	1.04	68.03	1.82	0.02	0.78
○Thread 883470			0.71	1.04	68.50	1.82	0.02	0.77
○Thread 883471			0.72	1.04	69.45	1.82	0.02	0.77
○Thread 883472			0.68	1.04	65.10	1.82	0.02	0.77
○Thread 883473			0.72	1.04	69.42	1.82	0.02	0.78
○Thread 883474			0.72	1.04	69.41	1.82	0.02	0.78
○Thread 883475			0.69	1.04	66.02	1.82	0.02	0.77
○Thread 883476			0.71	1.04	68.42	1.82	0.02	0.78
○Thread 883477			0.72	1.04	69.37	1.82	0.02	0.78
○Thread 883478			0.70	1.04	67.43	1.82	0.02	0.78
○Thread 883479			0.69	1.04	66.45	1.82	0.02	0.76
○Thread 883480			0.73	1.04	69.81	1.82	0.02	0.78
○Thread 883481			0.71	1.04	68.83	1.82	0.02	0.77
○Thread 883482			0.71	1.04	68.82	1.82	0.02	0.78
○Thread 883483			0.71	1.04	68.80	1.82	0.01	0.78
○Thread 883484			0.73	1.04	70.23	1.82	0.01	0.78
○Thread 883485			0.70	1.04	67.81	1.82	0.01	0.77
○Thread 883486			0.71	1.04	68.28	1.82	0.01	0.77
○Thread 883487			0.70	1.04	67.31	1.82	0.01	0.77
○Thread 883488			0.71	1.04	68.26	1.82	0.01	0.77
○Thread 883489			0.71	1.04	68.72	1.82	0.01	0.78
○Thread 883490			0.68	1.04	65.34	1.82	0.01	0.78
○Thread 883491			0.69	1.04	65.81	1.82	0.01	0.77
○Thread 883492			0.69	1.04	66.76	1.82	0.01	0.78
○Thread 883493			0.69	1.04	66.75	1.82	0.01	0.76
○Thread 883494			0.73	1.04	70.09	1.82	0.01	0.78
○Thread 883495			0.74	1.04	70.56	1.82	0.01	0.77
○Thread 883496			0.70	1.04	67.67	1.82	0.01	0.76
○Thread 883497			0.68	1.04	65.20	1.82	0.01	0.77
○Thread 883498			0.73	1.04	69.50	1.82	0.01	0.77
○Thread 883499			0.70	1.04	67.56	1.82	0.01	0.78
○Thread 883500			0.74	1.04	70.42	1.82	0.01	0.78
○Thread 883501			0.73	1.04	69.45	1.82	0.01	0.78
○Thread 883502			0.71	1.04	68.00	1.82	0.01	0.78
○Thread 883503			0.68	1.04	65.11	1.82	0.01	0.76
○Thread 883504			0.70	1.04	67.01	1.82	0.01	0.78
○Thread 883505			0.71	1.04	67.96	1.82	0.01	0.77
○Thread 883506			0.71	1.04	68.42	1.82	0.01	0.76
○Thread 883507			0.69	1.05	66.49	1.82	0.01	0.77
○Thread 883508			0.73	1.05	69.35	1.82	0.01	0.77
○Thread 883509			0.69	1.05	66.47	1.82	0.00	0.76
○Thread 883510			0.74	1.05	70.28	1.82	0.00	0.77
○Thread 883511			0.69	1.05	65.96	1.82	0.00	0.77
○Thread 883512			0.75	1.05	71.20	1.83	0.00	0.77
○Thread 883513			0.75	1.05	71.19	1.83	0.00	0.78
○Thread 883514			0.72	1.05	68.79	1.83	0.00	0.77
○Thread 883515			0.71	1.05	68.30	1.83	0.00	0.78
○Thread 883516			0.71	1.05	68.28	1.83	0.00	0.75
○Thread 883517			0.71	1.05	67.79	1.83	0.00	0.76
○Thread 883518			0.74	1.05	70.16	1.83	0.00	0.77
○Thread 883519			0.68	1.05	64.90	1.83	0.00	0.77
○Thread 883520			0.68	1.05	64.41	1.83	0.00	0.78
○Thread 883521			0.66	1.05	62.96	1.83	0.00	0.75

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 4 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	1.34
►Process 883590+		96	1.34
○Thread 883590			1.34	2.85	47.17	0.00	0.00	1.65
○Thread 883689			0.68	1.00	67.91	1.81	0.04	0.75
○Thread 883690			0.66	1.00	65.90	1.81	0.04	0.74
○Thread 883691			0.68	1.00	67.88	1.81	0.04	0.72
○Thread 883692			0.70	1.00	69.87	1.81	0.04	0.74
○Thread 883693			0.66	1.00	65.86	1.81	0.04	0.74
○Thread 883694			0.71	1.00	70.84	1.81	0.03	0.75
○Thread 883695			0.67	1.00	66.33	1.81	0.03	0.75
○Thread 883696			0.69	1.00	68.81	1.81	0.03	0.75
○Thread 883697			0.70	1.00	69.79	1.82	0.03	0.74
○Thread 883698			0.67	1.00	66.78	1.82	0.03	0.75
○Thread 883699			0.69	1.00	69.26	1.82	0.03	0.74
○Thread 883700			0.69	1.00	68.24	1.82	0.03	0.73
○Thread 883701			0.65	1.00	65.24	1.82	0.03	0.75
○Thread 883702			0.69	1.00	68.71	1.82	0.03	0.75
○Thread 883703			0.69	1.00	68.20	1.82	0.03	0.75
○Thread 883704			0.67	1.00	66.69	1.82	0.03	0.73
○Thread 883705			0.69	1.00	68.16	1.82	0.03	0.75
○Thread 883706			0.66	1.01	65.66	1.82	0.03	0.75
○Thread 883707			0.67	1.01	66.64	1.82	0.03	0.74
○Thread 883708			0.68	1.01	67.12	1.82	0.03	0.75
○Thread 883709			0.69	1.01	68.60	1.82	0.03	0.74
○Thread 883710			0.70	1.01	69.58	1.82	0.03	0.75
○Thread 883711			0.68	1.01	67.57	1.82	0.03	0.74
○Thread 883712			0.69	1.01	68.55	1.82	0.03	0.75
○Thread 883713			0.68	1.01	67.04	1.82	0.03	0.74
○Thread 883714			0.66	1.01	65.54	1.82	0.03	0.74
○Thread 883715			0.66	1.01	65.52	1.82	0.03	0.73
○Thread 883716			0.70	1.01	69.97	1.82	0.03	0.75
○Thread 883717			0.68	1.01	66.98	1.82	0.03	0.74
○Thread 883718			0.70	1.01	69.94	1.82	0.03	0.74
○Thread 883719			0.70	1.01	69.43	1.82	0.03	0.74
○Thread 883720			0.67	1.01	66.44	1.82	0.03	0.74
○Thread 883721			0.68	1.01	66.92	1.82	0.02	0.75
○Thread 883722			0.70	1.01	69.88	1.82	0.02	0.75
○Thread 883723			0.69	1.01	68.38	1.82	0.02	0.73
○Thread 883724			0.72	1.01	71.33	1.82	0.02	0.74
○Thread 883725			0.69	1.01	67.85	1.82	0.02	0.75
○Thread 883726			0.69	1.01	68.83	1.82	0.02	0.74
○Thread 883727			0.69	1.01	67.82	1.82	0.02	0.75
○Thread 883728			0.69	1.01	68.80	1.82	0.02	0.75
○Thread 883729			0.71	1.01	70.27	1.82	0.02	0.75
○Thread 883730			0.69	1.01	68.27	1.82	0.02	0.75
○Thread 883731			0.67	1.01	66.28	1.82	0.02	0.75
○Thread 883732			0.67	1.01	66.26	1.82	0.02	0.74
○Thread 883733			0.69	1.01	67.73	1.82	0.02	0.75
○Thread 883734			0.69	1.01	67.72	1.82	0.02	0.72
○Thread 883735			0.67	1.01	65.72	1.82	0.02	0.73
○Thread 883736			0.68	1.01	66.70	1.82	0.02	0.72
○Thread 883737			0.67	1.01	66.19	1.82	0.02	0.75
○Thread 883738			0.65	1.01	64.20	1.82	0.02	0.75
○Thread 883739			0.69	1.01	68.13	1.82	0.02	0.74
○Thread 883740			0.70	1.01	69.10	1.82	0.02	0.75
○Thread 883741			0.69	1.01	67.61	1.82	0.02	0.74
○Thread 883742			0.68	1.01	66.60	1.82	0.02	0.75
○Thread 883743			0.66	1.01	65.11	1.82	0.02	0.74
○Thread 883744			0.69	1.01	67.56	1.82	0.02	0.74
○Thread 883745			0.70	1.02	68.92	1.82	0.01	0.75
○Thread 883746			0.67	1.02	65.46	1.82	0.01	0.74
○Thread 883747			0.69	1.02	67.42	1.82	0.01	0.74
○Thread 883748			0.69	1.02	67.90	1.82	0.01	0.74
○Thread 883749			0.69	1.02	67.88	1.82	0.01	0.74
○Thread 883750			0.70	1.02	69.34	1.82	0.01	0.75
○Thread 883751			0.68	1.02	66.87	1.82	0.01	0.74
○Thread 883752			0.67	1.02	65.38	1.82	0.01	0.74
○Thread 883753			0.65	1.02	64.38	1.82	0.01	0.75
○Thread 883754			0.65	1.02	64.36	1.82	0.01	0.74
○Thread 883755			0.68	1.02	66.81	1.82	0.01	0.73
○Thread 883756			0.67	1.02	65.81	1.82	0.01	0.73
○Thread 883757			0.65	1.02	64.32	1.82	0.01	0.75
○Thread 883758			0.68	1.02	66.27	1.82	0.01	0.75
○Thread 883759			0.67	1.02	65.76	1.82	0.01	0.75
○Thread 883760			0.70	1.02	68.69	1.82	0.01	0.73
○Thread 883761			0.69	1.02	68.18	1.82	0.01	0.74
○Thread 883762			0.67	1.02	65.72	1.82	0.01	0.72
○Thread 883763			0.68	1.02	66.19	1.82	0.01	0.75
○Thread 883764			0.67	1.02	65.69	1.82	0.01	0.74
○Thread 883765			0.69	1.02	67.14	1.82	0.01	0.74
○Thread 883766			0.68	1.02	66.64	1.82	0.01	0.74
○Thread 883767			0.67	1.02	65.15	1.82	0.01	0.74
○Thread 883768			0.70	1.02	68.57	1.82	0.01	0.74
○Thread 883769			0.67	1.02	65.61	1.82	0.01	0.74
○Thread 883770			0.70	1.02	69.03	1.82	0.01	0.74
○Thread 883771			0.69	1.02	67.05	1.82	0.01	0.74
○Thread 883772			0.68	1.02	66.06	1.82	0.00	0.73
○Thread 883773			0.69	1.02	67.51	1.83	0.00	0.75
○Thread 883774			0.67	1.02	65.05	1.83	0.00	0.75
○Thread 883775			0.68	1.02	66.01	1.83	0.00	0.75
○Thread 883776			0.69	1.02	66.97	1.83	0.00	0.75
○Thread 883777			0.69	1.02	67.93	1.83	0.00	0.74
○Thread 883778			0.66	1.02	64.50	1.83	0.00	0.72
○Thread 883779			0.70	1.02	68.39	1.83	0.00	0.74
○Thread 883780			0.68	1.02	65.93	1.83	0.00	0.75
○Thread 883781			0.67	1.02	65.43	1.83	0.00	0.75
○Thread 883782			0.69	1.02	66.88	1.83	0.00	0.75
○Thread 883783			0.61	1.02	59.05	1.83	0.00	0.75

Report Configuration