OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	1	47.82
►Process 3366092+		1	47.82
○Thread 3366092			47.82	48.79	98.00	0.00	0.00	48.75

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	2	23.63
►Process 3366264+		2	23.63
○Thread 3366264			23.63	25.78	91.70	0.00	0.00	25.75
○Thread 3366363			23.37	24.90	93.87	0.88	-0.00	24.56

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	4	11.79
►Process 3366604+		4	11.79
○Thread 3366604			11.75	14.34	81.92	0.00	0.00	14.31
○Thread 3366703			11.79	13.46	87.57	0.88	-0.00	13.12
○Thread 3366704			11.75	13.46	87.27	0.88	-0.00	13.12
○Thread 3366705			11.73	13.46	87.12	0.88	-0.00	13.12

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	8	6.48
►Process 3367437+		8	6.48
○Thread 3367437			6.26	9.39	66.67	0.00	0.00	9.36
○Thread 3367536			6.48	8.51	76.19	0.88	-0.00	8.17
○Thread 3367537			6.47	8.51	76.07	0.88	-0.00	8.16
○Thread 3367538			6.47	8.51	76.01	0.88	-0.00	8.16
○Thread 3367539			6.46	8.50	75.96	0.88	-0.00	8.17
○Thread 3367540			6.47	8.50	76.02	0.88	-0.00	8.17
○Thread 3367541			6.41	8.50	75.37	0.88	-0.00	8.16
○Thread 3367542			6.39	8.50	75.14	0.88	-0.00	8.17

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	16	3.97
►Process 3369109+		16	3.97
○Thread 3369109			3.63	7.63	47.67	0.00	0.00	7.59
○Thread 3369208			3.96	6.74	58.75	0.88	-0.00	6.40
○Thread 3369209			3.97	6.74	58.90	0.89	-0.00	6.40
○Thread 3369210			3.97	6.74	58.90	0.89	-0.00	6.40
○Thread 3369211			3.97	6.74	58.90	0.89	-0.00	6.40
○Thread 3369212			3.97	6.74	58.90	0.89	-0.00	6.40
○Thread 3369213			3.92	6.74	58.23	0.89	-0.00	6.40
○Thread 3369214			3.92	6.74	58.24	0.89	-0.00	6.40
○Thread 3369215			3.92	6.74	58.16	0.89	-0.00	6.40
○Thread 3369216			3.92	6.74	58.24	0.89	0.00	6.40
○Thread 3369217			3.93	6.74	58.31	0.89	-0.00	6.40
○Thread 3369218			3.90	6.74	57.94	0.89	-0.00	6.40
○Thread 3369219			3.89	6.74	57.79	0.89	0.00	6.40
○Thread 3369220			3.90	6.74	57.94	0.89	0.00	6.40
○Thread 3369221			3.90	6.74	57.87	0.89	0.00	6.40
○Thread 3369222			3.86	6.74	57.28	0.89	-0.00	6.40

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	24	2.89
►Process 3372557+		24	2.89
○Thread 3372557			2.66	7.28	36.52	0.00	0.00	7.24
○Thread 3372656			2.89	6.40	45.26	0.89	-0.00	6.05
○Thread 3372657			2.89	6.40	45.18	0.89	0.00	6.05
○Thread 3372658			2.89	6.40	45.26	0.89	0.00	6.05
○Thread 3372659			2.89	6.40	45.18	0.89	-0.00	6.05
○Thread 3372660			2.89	6.40	45.18	0.89	-0.00	6.05
○Thread 3372661			2.86	6.40	44.79	0.89	-0.00	6.05
○Thread 3372662			2.86	6.40	44.79	0.89	-0.00	6.05
○Thread 3372663			2.86	6.40	44.72	0.89	0.00	6.05
○Thread 3372664			2.87	6.40	44.87	0.89	-0.00	6.05
○Thread 3372665			2.87	6.40	44.88	0.89	0.00	6.05
○Thread 3372666			2.86	6.40	44.72	0.89	0.00	6.05
○Thread 3372667			2.86	6.40	44.80	0.89	0.00	6.05
○Thread 3372668			2.86	6.40	44.80	0.89	0.00	6.05
○Thread 3372669			2.83	6.40	44.33	0.89	0.00	6.05
○Thread 3372670			2.84	6.39	44.49	0.89	0.00	6.05
○Thread 3372671			2.84	6.39	44.41	0.89	0.00	6.05
○Thread 3372672			2.84	6.39	44.49	0.89	0.00	6.05
○Thread 3372673			2.84	6.39	44.49	0.89	0.00	6.05
○Thread 3372674			2.82	6.39	44.10	0.89	0.00	6.05
○Thread 3372675			2.82	6.39	44.10	0.89	0.00	6.05
○Thread 3372676			2.82	6.39	44.10	0.89	0.00	6.05
○Thread 3372677			2.82	6.39	44.10	0.89	0.00	6.05
○Thread 3372678			2.77	6.39	43.40	0.89	-0.00	6.05

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	32	2.33
►Process 3377783+		32	2.33
○Thread 3377783			2.11	7.42	28.44	0.00	0.00	7.39
○Thread 3377882			2.33	6.53	35.74	0.89	-0.00	6.19
○Thread 3377883			2.33	6.53	35.67	0.89	-0.00	6.19
○Thread 3377884			2.33	6.53	35.67	0.89	-0.00	6.19
○Thread 3377885			2.33	6.53	35.67	0.89	-0.00	6.19
○Thread 3377886			2.33	6.53	35.67	0.89	-0.00	6.19
○Thread 3377887			2.32	6.53	35.59	0.89	-0.00	6.19
○Thread 3377888			2.32	6.53	35.59	0.89	-0.00	6.19
○Thread 3377889			2.32	6.53	35.59	0.89	-0.00	6.19
○Thread 3377890			2.32	6.53	35.52	0.89	-0.00	6.19
○Thread 3377891			2.32	6.53	35.52	0.89	-0.00	6.19
○Thread 3377892			2.31	6.53	35.44	0.89	-0.00	6.19
○Thread 3377893			2.31	6.53	35.37	0.89	-0.00	6.18
○Thread 3377894			2.31	6.53	35.37	0.89	-0.00	6.19
○Thread 3377895			2.31	6.53	35.44	0.89	-0.00	6.19
○Thread 3377896			2.29	6.53	35.14	0.89	-0.00	6.19
○Thread 3377897			2.32	6.53	35.52	0.89	-0.00	6.19
○Thread 3377898			2.32	6.53	35.52	0.89	-0.00	6.19
○Thread 3377899			2.32	6.53	35.52	0.89	-0.00	6.19
○Thread 3377900			2.31	6.53	35.45	0.89	-0.00	6.19
○Thread 3377901			2.31	6.53	35.37	0.89	-0.00	6.18
○Thread 3377902			2.31	6.53	35.45	0.89	-0.00	6.19
○Thread 3377903			2.26	6.53	34.61	0.89	-0.00	6.19
○Thread 3377904			2.26	6.53	34.61	0.89	-0.00	6.19
○Thread 3377905			2.26	6.53	34.61	0.89	-0.00	6.19
○Thread 3377906			2.26	6.53	34.61	0.89	-0.00	6.19
○Thread 3377907			2.26	6.53	34.69	0.89	-0.00	6.19
○Thread 3377908			2.24	6.53	34.38	0.89	-0.00	6.19
○Thread 3377909			2.24	6.53	34.38	0.89	-0.00	6.19
○Thread 3377910			2.25	6.53	34.53	0.89	0.00	6.19
○Thread 3377911			2.24	6.53	34.31	0.89	0.00	6.19
○Thread 3377912			2.22	6.53	34.00	0.89	-0.00	6.19

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	40	1.96
►Process 3384833+		40	1.96
○Thread 3384833			1.96	7.79	25.22	0.00	0.00	7.74
○Thread 3384932			1.95	6.91	28.24	0.89	-0.00	6.55
○Thread 3384933			1.94	6.91	28.09	0.89	-0.00	6.55
○Thread 3384934			1.95	6.91	28.24	0.89	-0.00	6.55
○Thread 3384935			1.94	6.91	28.09	0.89	-0.00	6.55
○Thread 3384936			1.95	6.91	28.24	0.89	-0.00	6.56
○Thread 3384937			1.93	6.91	28.02	0.89	-0.00	6.55
○Thread 3384938			1.94	6.91	28.17	0.89	-0.00	6.55
○Thread 3384939			1.94	6.91	28.10	0.89	-0.00	6.55
○Thread 3384940			1.93	6.91	28.02	0.89	-0.00	6.53
○Thread 3384941			1.95	6.90	28.24	0.89	-0.00	6.55
○Thread 3384942			1.95	6.90	28.31	0.89	-0.00	6.56
○Thread 3384943			1.94	6.90	28.17	0.89	-0.00	6.53
○Thread 3384944			1.93	6.90	28.02	0.89	-0.00	6.55
○Thread 3384945			1.94	6.90	28.17	0.89	-0.00	6.56
○Thread 3384946			1.92	6.90	27.81	0.89	-0.00	6.55
○Thread 3384947			1.92	6.90	27.88	0.89	-0.00	6.55
○Thread 3384948			1.92	6.90	27.81	0.89	-0.00	6.55
○Thread 3384949			1.91	6.90	27.74	0.89	-0.00	6.55
○Thread 3384950			1.93	6.90	27.96	0.89	-0.00	6.55
○Thread 3384951			1.93	6.90	27.96	0.89	-0.00	6.55
○Thread 3384952			1.94	6.90	28.17	0.89	-0.00	6.56
○Thread 3384953			1.89	6.90	27.38	0.89	-0.00	6.56
○Thread 3384954			1.88	6.90	27.23	0.89	-0.00	6.55
○Thread 3384955			1.88	6.90	27.31	0.89	-0.00	6.55
○Thread 3384956			1.89	6.90	27.38	0.89	-0.00	6.55
○Thread 3384957			1.88	6.90	27.31	0.89	-0.00	6.55
○Thread 3384958			1.90	6.90	27.52	0.89	-0.00	6.55
○Thread 3384959			1.89	6.90	27.45	0.89	-0.00	6.56
○Thread 3384960			1.88	6.90	27.31	0.89	-0.00	6.56
○Thread 3384961			1.88	6.90	27.31	0.89	-0.00	6.55
○Thread 3384962			1.89	6.90	27.45	0.89	-0.00	6.55
○Thread 3384963			1.90	6.90	27.60	0.89	-0.00	6.56
○Thread 3384964			1.88	6.90	27.31	0.89	-0.00	6.55
○Thread 3384965			1.88	6.90	27.31	0.89	-0.00	6.55
○Thread 3384966			1.87	6.90	27.09	0.89	-0.00	6.55
○Thread 3384967			1.86	6.90	27.02	0.89	0.00	6.55
○Thread 3384968			1.87	6.90	27.09	0.89	0.00	6.55
○Thread 3384969			1.87	6.90	27.17	0.89	0.00	6.55
○Thread 3384970			1.84	6.90	26.73	0.89	-0.00	6.55

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	48	1.70
►Process 3393609+		48	1.70
○Thread 3393609			1.43	8.30	17.24	0.00	0.00	8.25
○Thread 3393708			1.70	7.40	23.03	0.89	-0.00	7.05
○Thread 3393709			1.69	7.40	22.89	0.89	-0.00	7.05
○Thread 3393710			1.69	7.40	22.89	0.89	-0.00	7.05
○Thread 3393711			1.69	7.40	22.83	0.89	-0.00	7.04
○Thread 3393712			1.69	7.40	22.83	0.89	-0.00	7.05
○Thread 3393713			1.68	7.40	22.69	0.89	-0.00	7.05
○Thread 3393714			1.67	7.40	22.63	0.89	-0.00	7.05
○Thread 3393715			1.68	7.40	22.69	0.89	-0.00	7.05
○Thread 3393716			1.66	7.40	22.49	0.89	-0.00	7.05
○Thread 3393717			1.67	7.40	22.63	0.89	-0.00	7.05
○Thread 3393718			1.67	7.40	22.63	0.89	-0.00	7.05
○Thread 3393719			1.68	7.40	22.69	0.89	-0.00	7.05
○Thread 3393720			1.68	7.40	22.76	0.89	-0.00	7.05
○Thread 3393721			1.67	7.40	22.56	0.89	-0.00	7.05
○Thread 3393722			1.65	7.40	22.36	0.89	-0.00	7.04
○Thread 3393723			1.66	7.40	22.49	0.89	-0.00	7.05
○Thread 3393724			1.65	7.40	22.36	0.89	-0.00	7.05
○Thread 3393725			1.67	7.40	22.63	0.89	-0.00	7.05
○Thread 3393726			1.65	7.40	22.36	0.89	-0.00	7.05
○Thread 3393727			1.64	7.40	22.22	0.89	-0.00	7.04
○Thread 3393728			1.61	7.40	21.75	0.89	-0.00	7.05
○Thread 3393729			1.63	7.40	22.02	0.89	-0.00	7.04
○Thread 3393730			1.63	7.40	22.02	0.89	-0.00	7.04
○Thread 3393731			1.66	7.40	22.50	0.89	-0.00	7.05
○Thread 3393732			1.66	7.40	22.50	0.89	-0.00	7.05
○Thread 3393733			1.67	7.40	22.56	0.89	-0.00	7.05
○Thread 3393734			1.66	7.40	22.43	0.89	-0.00	7.05
○Thread 3393735			1.67	7.40	22.56	0.89	-0.00	7.05
○Thread 3393736			1.66	7.40	22.50	0.89	-0.00	7.05
○Thread 3393737			1.66	7.40	22.50	0.89	-0.00	7.05
○Thread 3393738			1.66	7.40	22.50	0.89	-0.00	7.05
○Thread 3393739			1.65	7.40	22.36	0.89	-0.00	7.05
○Thread 3393740			1.67	7.40	22.57	0.89	-0.00	7.02
○Thread 3393741			1.67	7.40	22.57	0.89	-0.00	7.05
○Thread 3393742			1.65	7.40	22.36	0.90	-0.00	7.05
○Thread 3393743			1.67	7.40	22.57	0.90	-0.00	7.05
○Thread 3393744			1.66	7.40	22.43	0.90	-0.00	7.05
○Thread 3393745			1.65	7.40	22.30	0.90	-0.00	7.04
○Thread 3393746			1.64	7.40	22.16	0.90	-0.00	7.05
○Thread 3393747			1.64	7.40	22.23	0.90	-0.00	7.05
○Thread 3393748			1.65	7.40	22.30	0.90	-0.00	7.05
○Thread 3393749			1.65	7.40	22.37	0.90	-0.00	7.05
○Thread 3393750			1.63	7.40	22.03	0.90	-0.00	7.05
○Thread 3393751			1.64	7.40	22.16	0.90	-0.00	7.05
○Thread 3393752			1.64	7.40	22.16	0.90	-0.00	7.04
○Thread 3393753			1.64	7.40	22.17	0.90	-0.00	7.05
○Thread 3393754			1.60	7.40	21.62	0.90	-0.00	7.05

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	56	1.54
►Process 3404161+		56	1.54
○Thread 3404161			1.33	8.83	15.06	0.00	0.00	8.79
○Thread 3404260			1.54	7.94	19.39	0.89	-0.00	7.58
○Thread 3404261			1.52	7.94	19.14	0.89	-0.00	7.59
○Thread 3404262			1.51	7.94	19.01	0.89	-0.00	7.59
○Thread 3404263			1.51	7.94	19.01	0.89	-0.00	7.58
○Thread 3404264			1.52	7.94	19.20	0.89	-0.00	7.58
○Thread 3404265			1.49	7.94	18.76	0.89	-0.00	7.59
○Thread 3404266			1.49	7.94	18.76	0.89	-0.00	7.59
○Thread 3404267			1.49	7.94	18.82	0.89	-0.00	7.58
○Thread 3404268			1.49	7.94	18.82	0.89	-0.00	7.58
○Thread 3404269			1.48	7.94	18.70	0.89	-0.00	7.59
○Thread 3404270			1.48	7.94	18.70	0.89	-0.00	7.58
○Thread 3404271			1.48	7.94	18.64	0.89	-0.00	7.54
○Thread 3404272			1.48	7.94	18.70	0.89	-0.00	7.58
○Thread 3404273			1.50	7.94	18.95	0.89	-0.00	7.58
○Thread 3404274			1.49	7.94	18.76	0.89	-0.00	7.58
○Thread 3404275			1.49	7.94	18.76	0.89	-0.00	7.59
○Thread 3404276			1.49	7.94	18.76	0.89	-0.00	7.55
○Thread 3404277			1.50	7.94	18.89	0.89	-0.00	7.58
○Thread 3404278			1.48	7.94	18.70	0.89	-0.00	7.58
○Thread 3404279			1.49	7.94	18.76	0.89	-0.00	7.58
○Thread 3404280			1.47	7.94	18.58	0.89	-0.00	7.58
○Thread 3404281			1.47	7.94	18.58	0.89	-0.00	7.57
○Thread 3404282			1.48	7.94	18.70	0.89	-0.00	7.58
○Thread 3404283			1.48	7.94	18.64	0.89	-0.00	7.58
○Thread 3404284			1.48	7.94	18.70	0.89	-0.00	7.58
○Thread 3404285			1.46	7.94	18.45	0.89	-0.00	7.58
○Thread 3404286			1.48	7.94	18.64	0.89	-0.00	7.58
○Thread 3404287			1.47	7.94	18.58	0.89	-0.00	7.58
○Thread 3404288			1.51	7.94	19.02	0.89	-0.00	7.59
○Thread 3404289			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404290			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404291			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404292			1.48	7.94	18.64	0.89	-0.00	7.57
○Thread 3404293			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404294			1.48	7.94	18.71	0.89	-0.00	7.57
○Thread 3404295			1.49	7.94	18.83	0.89	-0.00	7.58
○Thread 3404296			1.49	7.94	18.83	0.89	-0.00	7.58
○Thread 3404297			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404298			1.48	7.94	18.64	0.89	-0.00	7.58
○Thread 3404299			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404300			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404301			1.50	7.94	18.90	0.89	-0.00	7.59
○Thread 3404302			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404303			1.49	7.94	18.83	0.89	-0.00	7.59
○Thread 3404304			1.47	7.94	18.52	0.89	-0.00	7.58
○Thread 3404305			1.47	7.94	18.58	0.89	-0.00	7.58
○Thread 3404306			1.49	7.94	18.77	0.89	-0.00	7.58
○Thread 3404307			1.47	7.94	18.58	0.89	-0.00	7.58
○Thread 3404308			1.47	7.94	18.52	0.89	-0.00	7.57
○Thread 3404309			1.47	7.94	18.58	0.89	-0.00	7.59
○Thread 3404310			1.45	7.94	18.27	0.89	-0.00	7.54
○Thread 3404311			1.47	7.94	18.58	0.89	-0.00	7.58
○Thread 3404312			1.46	7.94	18.39	0.89	-0.00	7.54
○Thread 3404313			1.48	7.94	18.65	0.89	-0.00	7.58
○Thread 3404314			1.41	7.94	17.77	0.89	-0.00	7.59

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	64	1.38
►Process 3416489+		64	1.38
○Thread 3416489			1.27	9.37	13.55	0.00	0.00	9.32
○Thread 3416588			1.38	8.48	16.28	0.89	-0.00	8.12
○Thread 3416589			1.36	8.48	16.10	0.89	-0.00	8.11
○Thread 3416590			1.36	8.48	16.04	0.89	-0.00	8.11
○Thread 3416591			1.35	8.48	15.98	0.89	-0.00	8.06
○Thread 3416592			1.37	8.48	16.22	0.89	-0.00	8.12
○Thread 3416593			1.36	8.48	16.10	0.89	-0.00	8.12
○Thread 3416594			1.36	8.48	16.04	0.89	-0.00	8.12
○Thread 3416595			1.36	8.48	16.04	0.89	-0.00	8.12
○Thread 3416596			1.37	8.48	16.16	0.89	-0.00	8.12
○Thread 3416597			1.37	8.48	16.22	0.89	-0.00	8.11
○Thread 3416598			1.36	8.48	16.04	0.89	-0.00	8.12
○Thread 3416599			1.35	8.48	15.92	0.89	-0.00	8.11
○Thread 3416600			1.35	8.48	15.98	0.89	-0.00	8.11
○Thread 3416601			1.35	8.48	15.93	0.89	-0.00	8.11
○Thread 3416602			1.35	8.48	15.98	0.89	-0.00	8.11
○Thread 3416603			1.35	8.48	15.93	0.89	-0.00	8.11
○Thread 3416604			1.36	8.48	16.10	0.89	-0.00	8.12
○Thread 3416605			1.35	8.48	15.93	0.89	-0.00	8.11
○Thread 3416606			1.34	8.48	15.87	0.89	-0.00	8.11
○Thread 3416607			1.36	8.48	16.10	0.89	-0.00	8.11
○Thread 3416608			1.35	8.48	15.93	0.89	-0.00	8.11
○Thread 3416609			1.28	8.48	15.16	0.89	-0.00	8.11
○Thread 3416610			1.29	8.48	15.28	0.89	-0.00	8.08
○Thread 3416611			1.31	8.48	15.46	0.89	-0.00	8.12
○Thread 3416612			1.30	8.48	15.40	0.89	-0.00	8.11
○Thread 3416613			1.32	8.48	15.57	0.89	-0.00	8.12
○Thread 3416614			1.30	8.48	15.34	0.89	-0.00	8.11
○Thread 3416615			1.30	8.48	15.40	0.89	-0.00	8.11
○Thread 3416616			1.32	8.48	15.57	0.89	-0.00	8.12
○Thread 3416617			1.30	8.48	15.34	0.89	-0.00	8.12
○Thread 3416618			1.29	8.48	15.28	0.89	-0.00	8.12
○Thread 3416619			1.31	8.48	15.52	0.89	-0.00	8.12
○Thread 3416620			1.32	8.47	15.58	0.90	-0.00	8.11
○Thread 3416621			1.31	8.47	15.52	0.90	-0.00	8.12
○Thread 3416622			1.31	8.47	15.52	0.90	-0.00	8.11
○Thread 3416623			1.31	8.47	15.46	0.90	-0.00	8.11
○Thread 3416624			1.31	8.47	15.46	0.90	-0.00	8.11
○Thread 3416625			1.28	8.47	15.10	0.90	-0.00	8.12
○Thread 3416626			1.31	8.47	15.46	0.90	-0.00	8.11
○Thread 3416627			1.31	8.47	15.52	0.90	-0.00	8.11
○Thread 3416628			1.30	8.47	15.40	0.90	-0.00	8.11
○Thread 3416629			1.32	8.47	15.58	0.90	-0.00	8.11
○Thread 3416630			1.30	8.47	15.40	0.90	-0.00	8.11
○Thread 3416631			1.31	8.47	15.46	0.90	-0.00	8.11
○Thread 3416632			1.31	8.47	15.46	0.90	-0.00	8.11
○Thread 3416633			1.30	8.47	15.34	0.90	-0.00	8.11
○Thread 3416634			1.31	8.47	15.46	0.90	-0.00	8.11
○Thread 3416635			1.31	8.47	15.52	0.90	-0.00	8.11
○Thread 3416636			1.32	8.47	15.58	0.90	-0.00	8.11
○Thread 3416637			1.31	8.47	15.52	0.90	-0.00	8.12
○Thread 3416638			1.30	8.47	15.40	0.90	-0.00	8.11
○Thread 3416639			1.32	8.47	15.58	0.90	-0.00	8.11
○Thread 3416640			1.30	8.47	15.40	0.90	-0.00	8.11
○Thread 3416641			1.30	8.47	15.40	0.90	-0.00	8.11
○Thread 3416642			1.30	8.47	15.34	0.90	-0.00	8.11
○Thread 3416643			1.30	8.47	15.34	0.90	-0.00	8.11
○Thread 3416644			1.30	8.47	15.40	0.90	-0.00	8.11
○Thread 3416645			1.30	8.47	15.34	0.90	-0.00	8.12
○Thread 3416646			1.28	8.47	15.11	0.90	-0.00	8.00
○Thread 3416647			1.28	8.47	15.17	0.90	-0.00	8.11
○Thread 3416648			1.29	8.47	15.23	0.90	-0.00	8.11
○Thread 3416649			1.29	8.47	15.29	0.90	-0.00	8.11
○Thread 3416650			1.24	8.47	14.64	0.90	-0.00	8.11

▶Scalability - 1x72

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 72OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	72	1.30
►Process 3430643+		72	1.30
○Thread 3430643			1.07	10.03	10.62	0.00	0.00	9.98
○Thread 3430742			1.28	9.13	14.07	0.90	0.00	8.77
○Thread 3430743			1.29	9.13	14.13	0.90	-0.00	8.77
○Thread 3430744			1.28	9.13	14.07	0.90	0.00	8.77
○Thread 3430745			1.28	9.13	14.02	0.90	-0.00	8.77
○Thread 3430746			1.30	9.13	14.29	0.90	-0.00	8.77
○Thread 3430747			1.30	9.13	14.29	0.90	0.00	8.78
○Thread 3430748			1.27	9.13	13.91	0.90	-0.00	8.77
○Thread 3430749			1.27	9.13	13.96	0.90	-0.00	8.77
○Thread 3430750			1.28	9.13	14.02	0.90	-0.00	8.77
○Thread 3430751			1.25	9.13	13.74	0.90	0.00	8.71
○Thread 3430752			1.28	9.13	14.02	0.90	-0.00	8.77
○Thread 3430753			1.28	9.13	14.07	0.90	0.00	8.78
○Thread 3430754			1.28	9.13	14.02	0.90	-0.00	8.77
○Thread 3430755			1.26	9.13	13.80	0.90	-0.00	8.77
○Thread 3430756			1.26	9.13	13.85	0.90	0.00	8.78
○Thread 3430757			1.26	9.13	13.80	0.90	-0.00	8.77
○Thread 3430758			1.27	9.13	13.96	0.90	-0.00	8.77
○Thread 3430759			1.27	9.13	13.91	0.90	-0.00	8.77
○Thread 3430760			1.26	9.13	13.80	0.90	-0.00	8.77
○Thread 3430761			1.26	9.13	13.80	0.90	-0.00	8.76
○Thread 3430762			1.26	9.13	13.80	0.90	-0.00	8.77
○Thread 3430763			1.25	9.13	13.75	0.90	-0.00	8.77
○Thread 3430764			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430765			1.27	9.13	13.91	0.90	0.00	8.77
○Thread 3430766			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430767			1.27	9.13	13.91	0.90	-0.00	8.77
○Thread 3430768			1.28	9.13	14.02	0.90	-0.00	8.78
○Thread 3430769			1.27	9.13	13.91	0.90	-0.00	8.78
○Thread 3430770			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430771			1.26	9.13	13.80	0.90	-0.00	8.77
○Thread 3430772			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430773			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430774			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430775			1.27	9.13	13.97	0.90	0.00	8.77
○Thread 3430776			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430777			1.28	9.13	14.08	0.90	-0.00	8.78
○Thread 3430778			1.27	9.13	13.97	0.90	-0.00	8.77
○Thread 3430779			1.28	9.13	14.02	0.90	-0.00	8.78
○Thread 3430780			1.27	9.13	13.97	0.90	-0.00	8.77
○Thread 3430781			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430782			1.27	9.13	13.97	0.90	-0.00	8.77
○Thread 3430783			1.28	9.13	14.02	0.90	-0.00	8.77
○Thread 3430784			1.27	9.13	13.97	0.90	-0.00	8.77
○Thread 3430785			1.27	9.13	13.97	0.90	-0.00	8.77
○Thread 3430786			1.27	9.13	13.91	0.90	-0.00	8.77
○Thread 3430787			1.27	9.13	13.91	0.90	-0.00	8.77
○Thread 3430788			1.27	9.13	13.97	0.90	-0.00	8.77
○Thread 3430789			1.27	9.13	13.91	0.90	-0.00	8.78
○Thread 3430790			1.27	9.13	13.91	0.90	-0.00	8.74
○Thread 3430791			1.26	9.13	13.81	0.90	-0.00	8.77
○Thread 3430792			1.25	9.13	13.75	0.90	-0.00	8.77
○Thread 3430793			1.26	9.13	13.86	0.90	-0.00	8.77
○Thread 3430794			1.25	9.13	13.75	0.90	-0.00	8.77
○Thread 3430795			1.22	9.13	13.42	0.90	-0.00	8.77
○Thread 3430796			1.22	9.13	13.42	0.90	-0.00	8.78
○Thread 3430797			1.21	9.13	13.26	0.90	-0.00	8.77
○Thread 3430798			1.22	9.13	13.37	0.90	-0.00	8.78
○Thread 3430799			1.21	9.13	13.26	0.90	-0.00	8.77
○Thread 3430800			1.20	9.13	13.20	0.90	-0.00	8.77
○Thread 3430801			1.21	9.13	13.31	0.90	-0.00	8.77
○Thread 3430802			1.21	9.13	13.31	0.90	-0.00	8.77
○Thread 3430803			1.22	9.13	13.37	0.90	-0.00	8.77
○Thread 3430804			1.21	9.13	13.26	0.90	-0.00	8.71
○Thread 3430805			1.22	9.13	13.37	0.90	-0.00	8.77
○Thread 3430806			1.21	9.13	13.26	0.90	-0.00	8.77
○Thread 3430807			1.21	9.13	13.26	0.90	-0.00	8.77
○Thread 3430808			1.20	9.13	13.21	0.90	-0.00	8.77
○Thread 3430809			1.20	9.12	13.21	0.90	-0.00	8.77
○Thread 3430810			1.21	9.12	13.26	0.90	-0.00	8.77
○Thread 3430811			1.20	9.12	13.21	0.90	-0.00	8.77
○Thread 3430812			1.17	9.12	12.82	0.90	-0.00	8.71

▶Scalability - 1x80

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 80OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	80	1.20
►Process 3446523+		80	1.20
○Thread 3446523			1.01	10.62	9.56	0.00	0.00	10.57
○Thread 3446622			1.19	9.73	12.24	0.89	-0.00	9.36
○Thread 3446623			1.18	9.73	12.19	0.89	-0.00	9.36
○Thread 3446624			1.19	9.72	12.29	0.89	-0.00	9.36
○Thread 3446625			1.19	9.72	12.24	0.89	-0.00	9.36
○Thread 3446626			1.20	9.72	12.34	0.89	-0.00	9.36
○Thread 3446627			1.18	9.72	12.19	0.89	-0.00	9.36
○Thread 3446628			1.18	9.72	12.19	0.89	-0.00	9.36
○Thread 3446629			1.19	9.72	12.24	0.89	-0.00	9.36
○Thread 3446630			1.18	9.72	12.19	0.89	-0.00	9.36
○Thread 3446631			1.17	9.72	12.03	0.89	-0.00	9.35
○Thread 3446632			1.19	9.72	12.29	0.89	-0.00	9.36
○Thread 3446633			1.19	9.72	12.24	0.89	-0.00	9.32
○Thread 3446634			1.20	9.72	12.34	0.89	-0.00	9.36
○Thread 3446635			1.18	9.72	12.14	0.89	-0.00	9.36
○Thread 3446636			1.19	9.72	12.29	0.89	-0.00	9.36
○Thread 3446637			1.19	9.72	12.29	0.89	-0.00	9.36
○Thread 3446638			1.20	9.72	12.34	0.89	-0.00	9.36
○Thread 3446639			1.18	9.72	12.19	0.89	-0.00	9.36
○Thread 3446640			1.19	9.72	12.24	0.89	-0.00	9.36
○Thread 3446641			1.20	9.72	12.39	0.89	-0.00	9.36
○Thread 3446642			1.20	9.72	12.34	0.89	-0.00	9.36
○Thread 3446643			1.14	9.72	11.78	0.90	-0.00	9.36
○Thread 3446644			1.15	9.72	11.88	0.90	-0.00	9.36
○Thread 3446645			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446646			1.13	9.72	11.67	0.90	-0.00	9.36
○Thread 3446647			1.13	9.72	11.62	0.90	-0.00	9.36
○Thread 3446648			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446649			1.15	9.72	11.88	0.90	-0.00	9.36
○Thread 3446650			1.15	9.72	11.83	0.90	-0.00	9.36
○Thread 3446651			1.15	9.72	11.88	0.90	-0.00	9.37
○Thread 3446652			1.13	9.72	11.67	0.90	-0.00	9.36
○Thread 3446653			1.15	9.72	11.83	0.90	-0.00	9.36
○Thread 3446654			1.16	9.72	11.93	0.90	-0.00	9.37
○Thread 3446655			1.14	9.72	11.78	0.90	-0.00	9.29
○Thread 3446656			1.14	9.72	11.78	0.90	-0.00	9.37
○Thread 3446657			1.14	9.72	11.68	0.90	-0.00	9.36
○Thread 3446658			1.14	9.72	11.68	0.90	-0.00	9.37
○Thread 3446659			1.12	9.72	11.52	0.90	-0.00	9.36
○Thread 3446660			1.12	9.72	11.52	0.90	-0.00	9.36
○Thread 3446661			1.15	9.72	11.83	0.90	-0.00	9.36
○Thread 3446662			1.16	9.72	11.93	0.90	-0.00	9.36
○Thread 3446663			1.13	9.72	11.62	0.90	-0.00	9.29
○Thread 3446664			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446665			1.15	9.72	11.83	0.90	-0.00	9.36
○Thread 3446666			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446667			1.13	9.72	11.63	0.90	-0.00	9.36
○Thread 3446668			1.13	9.72	11.68	0.90	-0.00	9.36
○Thread 3446669			1.16	9.72	11.93	0.90	-0.00	9.36
○Thread 3446670			1.13	9.72	11.63	0.90	-0.00	9.36
○Thread 3446671			1.12	9.72	11.52	0.90	-0.00	9.36
○Thread 3446672			1.15	9.72	11.88	0.90	-0.00	9.32
○Thread 3446673			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446674			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446675			1.14	9.72	11.68	0.90	-0.00	9.29
○Thread 3446676			1.12	9.72	11.52	0.90	-0.00	9.36
○Thread 3446677			1.14	9.72	11.78	0.90	-0.00	9.36
○Thread 3446678			1.15	9.72	11.88	0.90	-0.00	9.36
○Thread 3446679			1.13	9.72	11.68	0.90	-0.00	9.36
○Thread 3446680			1.15	9.72	11.88	0.90	-0.00	9.37
○Thread 3446681			1.14	9.72	11.78	0.90	-0.00	9.37
○Thread 3446682			1.14	9.72	11.78	0.90	-0.00	9.36
○Thread 3446683			1.14	9.72	11.78	0.90	-0.00	9.36
○Thread 3446684			1.14	9.72	11.78	0.90	-0.00	9.36
○Thread 3446685			1.15	9.72	11.83	0.90	-0.00	9.36
○Thread 3446686			1.14	9.72	11.78	0.90	-0.00	9.36
○Thread 3446687			1.14	9.72	11.78	0.90	-0.00	9.37
○Thread 3446688			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446689			1.15	9.72	11.89	0.90	-0.00	9.36
○Thread 3446690			1.14	9.72	11.78	0.90	-0.00	9.36
○Thread 3446691			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446692			1.16	9.72	11.94	0.90	-0.00	9.36
○Thread 3446693			1.15	9.72	11.89	0.90	-0.00	9.36
○Thread 3446694			1.14	9.72	11.78	0.90	-0.00	9.36
○Thread 3446695			1.15	9.72	11.83	0.90	-0.00	9.36
○Thread 3446696			1.14	9.72	11.73	0.90	-0.00	9.36
○Thread 3446697			1.14	9.72	11.68	0.90	-0.00	9.36
○Thread 3446698			1.12	9.72	11.58	0.90	-0.00	9.36
○Thread 3446699			1.12	9.72	11.53	0.90	-0.00	9.36
○Thread 3446700			1.05	9.72	10.81	0.90	-0.00	9.35

▶Scalability - 1x88

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 88OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	88	1.12
►Process 3464179+		88	1.12
○Thread 3464179			1.00	11.25	8.94	0.00	0.00	11.21
○Thread 3464278			1.12	10.37	10.80	0.88	-0.00	10.00
○Thread 3464279			1.12	10.37	10.80	0.88	-0.00	10.00
○Thread 3464280			1.11	10.37	10.71	0.88	-0.00	10.00
○Thread 3464281			1.10	10.37	10.66	0.88	-0.00	10.00
○Thread 3464282			1.11	10.37	10.76	0.88	-0.00	10.00
○Thread 3464283			1.12	10.37	10.80	0.88	-0.00	9.96
○Thread 3464284			1.11	10.37	10.76	0.88	-0.00	10.00
○Thread 3464285			1.10	10.37	10.66	0.88	-0.00	10.00
○Thread 3464286			1.10	10.37	10.66	0.88	-0.00	10.00
○Thread 3464287			1.11	10.37	10.71	0.88	-0.00	10.00
○Thread 3464288			1.10	10.37	10.61	0.88	-0.00	10.00
○Thread 3464289			1.09	10.37	10.52	0.88	-0.00	9.92
○Thread 3464290			1.10	10.37	10.66	0.88	-0.00	10.00
○Thread 3464291			1.12	10.36	10.85	0.88	-0.00	10.00
○Thread 3464292			1.10	10.36	10.61	0.88	-0.00	9.92
○Thread 3464293			1.09	10.36	10.56	0.88	-0.00	10.00
○Thread 3464294			1.09	10.36	10.56	0.88	-0.00	10.00
○Thread 3464295			1.08	10.36	10.37	0.88	-0.00	9.92
○Thread 3464296			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464297			1.11	10.36	10.71	0.88	-0.00	10.00
○Thread 3464298			1.10	10.36	10.61	0.88	-0.00	10.00
○Thread 3464299			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464300			1.11	10.36	10.71	0.88	-0.00	10.01
○Thread 3464301			1.09	10.36	10.52	0.88	-0.00	9.96
○Thread 3464302			1.10	10.36	10.61	0.88	-0.00	10.01
○Thread 3464303			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464304			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464305			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464306			1.12	10.36	10.81	0.88	-0.00	10.01
○Thread 3464307			1.10	10.36	10.61	0.88	-0.00	10.00
○Thread 3464308			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464309			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464310			1.10	10.36	10.61	0.88	-0.00	10.01
○Thread 3464311			1.10	10.36	10.61	0.88	-0.00	10.00
○Thread 3464312			1.12	10.36	10.76	0.88	-0.00	10.00
○Thread 3464313			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464314			1.11	10.36	10.71	0.88	-0.00	10.00
○Thread 3464315			1.11	10.36	10.76	0.88	-0.00	10.00
○Thread 3464316			1.09	10.36	10.57	0.88	-0.00	10.00
○Thread 3464317			1.10	10.36	10.62	0.88	-0.00	9.89
○Thread 3464318			1.08	10.36	10.47	0.88	-0.00	10.00
○Thread 3464319			1.08	10.36	10.37	0.88	-0.00	10.00
○Thread 3464320			1.08	10.36	10.42	0.88	-0.00	10.00
○Thread 3464321			1.10	10.36	10.66	0.88	-0.00	10.00
○Thread 3464322			1.09	10.36	10.57	0.88	-0.00	10.00
○Thread 3464323			1.10	10.36	10.62	0.88	-0.00	10.00
○Thread 3464324			1.11	10.36	10.71	0.88	-0.00	10.00
○Thread 3464325			1.12	10.36	10.81	0.88	-0.00	10.00
○Thread 3464326			1.08	10.36	10.47	0.88	-0.00	9.93
○Thread 3464327			1.11	10.36	10.71	0.88	-0.00	10.00
○Thread 3464328			1.08	10.36	10.47	0.88	-0.00	10.00
○Thread 3464329			1.10	10.36	10.62	0.88	-0.00	10.00
○Thread 3464330			1.10	10.36	10.67	0.89	-0.00	10.00
○Thread 3464331			1.09	10.36	10.52	0.89	-0.00	10.00
○Thread 3464332			1.11	10.36	10.71	0.89	-0.00	10.00
○Thread 3464333			1.10	10.36	10.67	0.89	-0.00	10.00
○Thread 3464334			1.11	10.36	10.71	0.89	-0.00	10.00
○Thread 3464335			1.09	10.36	10.52	0.89	-0.00	9.96
○Thread 3464336			1.10	10.36	10.57	0.89	-0.00	9.99
○Thread 3464337			1.10	10.36	10.67	0.89	-0.00	10.00
○Thread 3464338			1.10	10.36	10.67	0.89	-0.00	10.00
○Thread 3464339			1.12	10.36	10.81	0.89	-0.00	10.01
○Thread 3464340			1.09	10.36	10.52	0.89	-0.00	10.00
○Thread 3464341			1.11	10.36	10.71	0.89	-0.00	10.00
○Thread 3464342			1.12	10.36	10.81	0.89	-0.00	10.00
○Thread 3464343			1.10	10.36	10.67	0.89	-0.00	10.01
○Thread 3464344			1.10	10.36	10.62	0.89	-0.00	9.96
○Thread 3464345			1.11	10.36	10.72	0.89	-0.00	10.00
○Thread 3464346			1.11	10.36	10.76	0.89	-0.00	10.00
○Thread 3464347			1.09	10.36	10.52	0.89	-0.00	10.00
○Thread 3464348			1.10	10.36	10.57	0.89	-0.00	10.00
○Thread 3464349			1.09	10.36	10.57	0.89	-0.00	10.00
○Thread 3464350			1.09	10.36	10.57	0.89	-0.00	10.00
○Thread 3464351			1.11	10.36	10.72	0.89	-0.00	10.00
○Thread 3464352			1.09	10.36	10.52	0.89	-0.00	10.00
○Thread 3464353			1.10	10.36	10.57	0.89	-0.00	10.00
○Thread 3464354			1.09	10.36	10.57	0.89	-0.00	10.00
○Thread 3464355			1.03	10.36	9.94	0.89	-0.00	10.01
○Thread 3464356			1.03	10.36	9.99	0.89	-0.00	10.00
○Thread 3464357			1.04	10.36	10.04	0.89	-0.00	10.00
○Thread 3464358			1.04	10.36	10.09	0.89	-0.00	10.00
○Thread 3464359			1.04	10.36	10.09	0.89	-0.00	10.00
○Thread 3464360			1.04	10.36	10.04	0.89	-0.00	10.00
○Thread 3464361			1.02	10.36	9.85	0.89	-0.00	9.96
○Thread 3464362			1.01	10.36	9.80	0.89	-0.00	10.00
○Thread 3464363			1.03	10.36	9.94	0.89	-0.00	10.00
○Thread 3464364			0.95	10.36	9.22	0.89	-0.00	9.99

▶Scalability - 1x96

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-95Run Directory: .OMP_NUM_THREADS: 96OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-35-140.ec2.internal+	1	96	1.12
►Process 3483666+		96	1.12
○Thread 3483666			1.00	12.16	8.23	0.00	0.00	12.10
○Thread 3483765			1.11	11.28	9.89	0.88	-0.00	10.91
○Thread 3483766			1.09	11.28	9.67	0.88	-0.00	10.87
○Thread 3483767			1.09	11.28	9.71	0.88	-0.00	10.91
○Thread 3483768			1.12	11.28	9.93	0.88	-0.00	10.92
○Thread 3483769			1.11	11.28	9.84	0.88	-0.00	10.91
○Thread 3483770			1.11	11.28	9.89	0.88	-0.00	10.91
○Thread 3483771			1.10	11.28	9.80	0.88	-0.00	10.91
○Thread 3483772			1.10	11.28	9.76	0.88	-0.00	10.91
○Thread 3483773			1.10	11.28	9.80	0.88	-0.00	10.91
○Thread 3483774			1.10	11.28	9.76	0.88	-0.00	10.91
○Thread 3483775			1.10	11.28	9.80	0.88	-0.00	10.91
○Thread 3483776			1.10	11.28	9.80	0.88	-0.00	10.91
○Thread 3483777			1.10	11.28	9.76	0.88	-0.00	10.91
○Thread 3483778			1.10	11.28	9.71	0.88	-0.00	10.91
○Thread 3483779			1.09	11.28	9.67	0.88	-0.00	10.91
○Thread 3483780			1.08	11.28	9.62	0.88	-0.00	10.91
○Thread 3483781			1.08	11.27	9.62	0.88	-0.00	10.91
○Thread 3483782			1.09	11.27	9.71	0.88	-0.00	10.91
○Thread 3483783			1.09	11.27	9.67	0.88	-0.00	10.91
○Thread 3483784			1.09	11.27	9.67	0.88	-0.00	10.91
○Thread 3483785			1.09	11.27	9.67	0.88	-0.00	10.82
○Thread 3483786			1.09	11.27	9.67	0.88	-0.00	10.82
○Thread 3483787			1.09	11.27	9.67	0.88	-0.00	10.91
○Thread 3483788			1.08	11.27	9.58	0.88	-0.00	10.91
○Thread 3483789			1.08	11.27	9.58	0.88	-0.00	10.91
○Thread 3483790			1.09	11.27	9.67	0.88	-0.00	10.91
○Thread 3483791			1.07	11.27	9.49	0.88	-0.00	10.82
○Thread 3483792			1.09	11.27	9.67	0.88	-0.00	10.91
○Thread 3483793			1.09	11.27	9.67	0.88	-0.00	10.91
○Thread 3483794			1.04	11.27	9.27	0.88	-0.00	10.78
○Thread 3483795			1.07	11.27	9.45	0.88	-0.00	10.91
○Thread 3483796			1.07	11.27	9.49	0.88	-0.00	10.91
○Thread 3483797			1.08	11.27	9.62	0.88	-0.00	10.91
○Thread 3483798			1.09	11.27	9.67	0.88	-0.00	10.91
○Thread 3483799			1.08	11.27	9.62	0.88	-0.00	10.91
○Thread 3483800			1.08	11.27	9.62	0.88	-0.00	10.91
○Thread 3483801			1.08	11.27	9.54	0.89	-0.00	10.91
○Thread 3483802			1.05	11.27	9.31	0.89	-0.00	10.89
○Thread 3483803			1.08	11.27	9.58	0.89	-0.00	10.90
○Thread 3483804			1.10	11.27	9.80	0.89	-0.00	10.92
○Thread 3483805			1.08	11.27	9.54	0.89	-0.00	10.91
○Thread 3483806			1.07	11.27	9.45	0.89	-0.00	10.91
○Thread 3483807			1.08	11.27	9.63	0.89	-0.00	10.91
○Thread 3483808			1.08	11.27	9.63	0.89	-0.00	10.91
○Thread 3483809			1.07	11.27	9.45	0.89	-0.00	10.91
○Thread 3483810			1.04	11.27	9.23	0.89	-0.00	10.91
○Thread 3483811			1.05	11.27	9.36	0.89	-0.00	10.91
○Thread 3483812			1.07	11.27	9.45	0.89	-0.00	10.91
○Thread 3483813			1.07	11.27	9.45	0.89	-0.00	10.91
○Thread 3483814			1.06	11.27	9.40	0.89	-0.00	10.91
○Thread 3483815			1.05	11.27	9.32	0.89	-0.00	10.81
○Thread 3483816			1.07	11.27	9.49	0.89	-0.00	10.91
○Thread 3483817			1.04	11.27	9.23	0.89	-0.00	10.87
○Thread 3483818			1.00	11.27	8.87	0.89	-0.00	10.91
○Thread 3483819			0.99	11.27	8.78	0.89	-0.00	10.90
○Thread 3483820			1.00	11.27	8.92	0.89	-0.00	10.91
○Thread 3483821			1.00	11.27	8.83	0.89	-0.00	10.91
○Thread 3483822			1.00	11.27	8.92	0.89	-0.00	10.78
○Thread 3483823			1.00	11.27	8.83	0.89	-0.00	10.91
○Thread 3483824			0.99	11.27	8.74	0.89	-0.00	10.91
○Thread 3483825			1.00	11.27	8.87	0.89	-0.00	10.89
○Thread 3483826			1.01	11.27	9.01	0.89	-0.00	10.82
○Thread 3483827			1.00	11.27	8.83	0.89	-0.00	10.91
○Thread 3483828			1.01	11.27	8.96	0.89	-0.00	10.91
○Thread 3483829			1.01	11.27	9.01	0.89	-0.00	10.92
○Thread 3483830			1.00	11.27	8.92	0.89	-0.00	10.91
○Thread 3483831			0.99	11.27	8.78	0.89	-0.00	10.91
○Thread 3483832			0.98	11.27	8.65	0.89	-0.00	10.87
○Thread 3483833			0.99	11.27	8.74	0.89	-0.00	10.91
○Thread 3483834			1.01	11.27	9.01	0.89	-0.00	10.91
○Thread 3483835			0.98	11.27	8.65	0.89	-0.00	10.90
○Thread 3483836			1.00	11.27	8.87	0.89	-0.00	10.91
○Thread 3483837			0.97	11.27	8.61	0.89	-0.00	10.91
○Thread 3483838			1.02	11.27	9.05	0.89	-0.00	10.91
○Thread 3483839			1.00	11.27	8.92	0.89	-0.00	10.91
○Thread 3483840			0.95	11.27	8.43	0.89	-0.00	10.91
○Thread 3483841			0.98	11.27	8.65	0.89	-0.00	10.91
○Thread 3483842			0.99	11.27	8.79	0.89	-0.00	10.91
○Thread 3483843			0.99	11.27	8.79	0.89	-0.00	10.91
○Thread 3483844			0.98	11.27	8.65	0.89	-0.00	10.91
○Thread 3483845			1.00	11.27	8.87	0.89	-0.00	10.91
○Thread 3483846			1.00	11.27	8.87	0.89	-0.00	10.91
○Thread 3483847			0.99	11.27	8.74	0.89	-0.00	10.82
○Thread 3483848			1.00	11.27	8.92	0.89	-0.00	10.91
○Thread 3483849			1.00	11.27	8.92	0.89	-0.00	10.91
○Thread 3483850			0.98	11.27	8.65	0.89	-0.00	10.91
○Thread 3483851			1.00	11.27	8.83	0.89	-0.00	10.92
○Thread 3483852			1.00	11.27	8.83	0.89	-0.00	10.91
○Thread 3483853			0.98	11.27	8.65	0.89	-0.00	10.79
○Thread 3483854			0.99	11.27	8.79	0.89	-0.00	10.91
○Thread 3483855			0.98	11.27	8.65	0.89	-0.00	10.88
○Thread 3483856			0.98	11.27	8.65	0.89	-0.00	10.91
○Thread 3483857			0.99	11.27	8.79	0.89	-0.00	10.92
○Thread 3483858			1.00	11.27	8.83	0.89	-0.00	10.82
○Thread 3483859			0.91	11.27	8.08	0.89	-0.00	10.92

Report Configuration