OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	374.21
►Process 3670438+		1	374.21
○Thread 3670438			374.21	375.41	99.68	0.00	0.00	370.89

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	189.54
►Process 3670738+		2	189.54
○Thread 3670738			185.67	191.80	96.81	0.00	0.00	190.78
○Thread 3670805			189.54	190.79	99.35	1.01	0.00	190.61

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	98.26
►Process 3671809+		4	98.26
○Thread 3671809			93.20	100.48	92.76	0.00	0.00	99.47
○Thread 3671876			98.22	99.47	98.74	1.00	0.00	99.29
○Thread 3671877			98.10	99.47	98.62	1.00	0.00	99.29
○Thread 3671878			98.26	99.47	98.78	1.00	0.00	99.29

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	58.87
►Process 3674602+		8	58.87
○Thread 3674602			51.86	61.41	84.44	0.00	0.00	60.39
○Thread 3674669			58.76	60.40	97.28	1.01	0.00	60.21
○Thread 3674670			58.87	60.40	97.46	1.01	0.00	60.22
○Thread 3674671			58.74	60.40	97.25	1.01	0.00	60.22
○Thread 3674672			58.66	60.40	97.12	1.01	0.00	60.22
○Thread 3674673			58.68	60.40	97.15	1.01	0.00	60.22
○Thread 3674674			58.80	60.40	97.35	1.01	0.00	60.22
○Thread 3674675			58.68	60.40	97.14	1.01	0.00	60.22

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	40.47
►Process 3680934+		16	40.47
○Thread 3680934			32.31	43.40	74.45	0.00	0.00	42.37
○Thread 3681001			40.36	42.38	95.23	1.01	0.00	42.19
○Thread 3681002			40.42	42.38	95.38	1.01	0.00	42.18
○Thread 3681003			40.34	42.38	95.20	1.01	0.00	42.19
○Thread 3681004			40.32	42.38	95.14	1.01	0.00	42.19
○Thread 3681005			40.47	42.38	95.50	1.01	0.00	42.19
○Thread 3681006			40.41	42.38	95.36	1.01	0.00	42.19
○Thread 3681007			40.34	42.38	95.18	1.01	0.00	42.19
○Thread 3681008			40.35	42.38	95.21	1.01	0.00	42.19
○Thread 3681009			40.09	42.38	94.58	1.01	0.00	42.19
○Thread 3681010			40.28	42.38	95.05	1.01	0.00	42.19
○Thread 3681011			40.11	42.38	94.65	1.01	0.00	42.19
○Thread 3681012			40.09	42.38	94.59	1.01	0.00	42.19
○Thread 3681013			40.27	42.38	95.03	1.01	0.00	42.19
○Thread 3681014			40.31	42.38	95.11	1.01	0.00	42.18
○Thread 3681015			40.24	42.38	94.93	1.01	0.00	42.19

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	33.80
►Process 3694296+		24	33.80
○Thread 3694296			22.81	37.14	61.41	0.00	0.00	36.14
○Thread 3694363			33.80	36.15	93.51	0.99	0.01	35.95
○Thread 3694364			33.60	36.15	92.95	0.99	0.01	35.95
○Thread 3694365			33.75	36.15	93.37	0.99	0.01	35.95
○Thread 3694366			33.67	36.15	93.16	0.99	0.01	35.96
○Thread 3694367			33.72	36.15	93.28	0.99	0.01	35.96
○Thread 3694368			33.58	36.15	92.90	0.99	0.01	35.94
○Thread 3694369			33.69	36.15	93.21	0.99	0.01	35.96
○Thread 3694370			33.56	36.15	92.84	0.99	0.00	35.96
○Thread 3694371			33.55	36.15	92.83	0.99	0.00	35.96
○Thread 3694372			33.70	36.15	93.23	0.99	0.00	35.95
○Thread 3694373			33.57	36.15	92.88	0.99	0.00	35.96
○Thread 3694374			33.71	36.15	93.27	0.99	0.00	35.96
○Thread 3694375			33.57	36.15	92.88	0.99	0.00	35.96
○Thread 3694376			33.63	36.15	93.05	0.99	0.00	35.96
○Thread 3694377			33.73	36.15	93.32	0.99	0.00	35.96
○Thread 3694378			33.53	36.15	92.75	0.99	0.00	35.96
○Thread 3694379			33.57	36.15	92.86	0.99	0.00	35.95
○Thread 3694380			33.43	36.15	92.49	0.99	0.00	35.95
○Thread 3694381			33.38	36.15	92.34	0.99	0.00	35.96
○Thread 3694382			33.32	36.15	92.17	0.99	0.00	35.96
○Thread 3694383			33.34	36.15	92.23	0.99	0.00	35.96
○Thread 3694384			33.36	36.15	92.28	0.99	0.00	35.95
○Thread 3694385			33.21	36.15	91.88	0.99	0.00	35.96

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	30.79
►Process 3714790+		32	30.79
○Thread 3714790			17.44	34.57	50.45	0.00	0.00	33.55
○Thread 3714857			30.73	33.55	91.60	1.01	0.01	33.34
○Thread 3714858			30.52	33.55	90.99	1.01	0.01	33.35
○Thread 3714859			30.60	33.55	91.21	1.01	0.01	33.34
○Thread 3714860			30.59	33.55	91.18	1.01	0.01	33.34
○Thread 3714861			30.47	33.55	90.84	1.01	0.01	33.35
○Thread 3714862			30.57	33.55	91.14	1.01	0.01	33.35
○Thread 3714863			30.66	33.55	91.41	1.01	0.01	33.34
○Thread 3714864			30.50	33.55	90.93	1.01	0.01	33.34
○Thread 3714865			30.59	33.55	91.20	1.01	0.01	33.34
○Thread 3714866			30.79	33.55	91.78	1.01	0.01	33.34
○Thread 3714867			30.51	33.55	90.96	1.01	0.01	33.35
○Thread 3714868			30.71	33.55	91.55	1.01	0.01	33.35
○Thread 3714869			30.58	33.55	91.16	1.01	0.01	33.35
○Thread 3714870			30.54	33.55	91.05	1.01	0.01	33.35
○Thread 3714871			30.64	33.55	91.33	1.01	0.01	33.35
○Thread 3714872			30.58	33.55	91.16	1.01	0.00	33.35
○Thread 3714873			30.55	33.55	91.07	1.01	0.00	33.35
○Thread 3714874			30.63	33.55	91.31	1.01	0.00	33.33
○Thread 3714875			30.71	33.55	91.55	1.01	0.00	33.34
○Thread 3714876			30.67	33.55	91.43	1.01	0.00	33.35
○Thread 3714877			30.68	33.55	91.46	1.01	0.00	33.35
○Thread 3714878			30.36	33.55	90.49	1.01	0.00	33.33
○Thread 3714879			30.54	33.55	91.04	1.01	0.00	33.34
○Thread 3714880			30.49	33.55	90.89	1.01	0.00	33.35
○Thread 3714881			30.56	33.55	91.09	1.01	0.00	33.35
○Thread 3714882			30.45	33.55	90.77	1.01	0.00	33.35
○Thread 3714883			30.53	33.55	91.00	1.01	0.00	33.35
○Thread 3714884			30.51	33.55	90.95	1.01	0.00	33.35
○Thread 3714885			30.69	33.55	91.49	1.01	0.00	33.34
○Thread 3714886			30.48	33.55	90.86	1.01	0.00	33.35
○Thread 3714887			30.69	33.55	91.49	1.01	0.00	33.34

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	30.30
►Process 3742313+		40	30.30
○Thread 3742313			15.26	34.40	44.37	0.00	0.00	33.36
○Thread 3742380			30.03	33.38	89.98	1.01	0.01	33.17
○Thread 3742381			30.17	33.38	90.40	1.01	0.01	33.18
○Thread 3742382			30.30	33.38	90.78	1.01	0.01	33.18
○Thread 3742383			30.03	33.38	89.99	1.01	0.01	33.18
○Thread 3742384			29.95	33.38	89.75	1.01	0.01	33.18
○Thread 3742385			30.12	33.38	90.26	1.01	0.01	33.18
○Thread 3742386			29.96	33.38	89.78	1.01	0.01	33.18
○Thread 3742387			30.01	33.38	89.91	1.01	0.01	33.18
○Thread 3742388			30.03	33.38	89.97	1.01	0.01	33.17
○Thread 3742389			29.99	33.38	89.87	1.01	0.01	33.18
○Thread 3742390			29.79	33.38	89.27	1.01	0.01	33.17
○Thread 3742391			30.07	33.38	90.09	1.01	0.01	33.18
○Thread 3742392			29.81	33.38	89.31	1.01	0.01	33.18
○Thread 3742393			30.05	33.38	90.03	1.01	0.01	33.18
○Thread 3742394			30.07	33.38	90.09	1.01	0.01	33.17
○Thread 3742395			29.82	33.38	89.34	1.01	0.01	33.17
○Thread 3742396			29.94	33.38	89.72	1.01	0.01	33.18
○Thread 3742397			29.93	33.38	89.69	1.01	0.01	33.18
○Thread 3742398			29.81	33.38	89.31	1.01	0.01	33.18
○Thread 3742399			29.90	33.38	89.58	1.01	0.01	33.17
○Thread 3742400			29.87	33.38	89.50	1.01	0.01	33.18
○Thread 3742401			29.85	33.38	89.44	1.01	0.01	33.17
○Thread 3742402			29.70	33.38	88.99	1.01	0.01	33.18
○Thread 3742403			29.73	33.38	89.08	1.01	0.00	33.18
○Thread 3742404			29.89	33.38	89.55	1.01	0.00	33.18
○Thread 3742405			29.72	33.38	89.05	1.01	0.00	33.18
○Thread 3742406			29.70	33.38	88.98	1.01	0.00	33.18
○Thread 3742407			30.01	33.38	89.91	1.01	0.00	33.18
○Thread 3742408			29.79	33.38	89.25	1.01	0.00	33.18
○Thread 3742409			29.98	33.38	89.82	1.01	0.00	33.18
○Thread 3742410			29.99	33.38	89.86	1.01	0.00	33.18
○Thread 3742411			29.98	33.38	89.81	1.01	0.00	33.18
○Thread 3742412			29.84	33.38	89.39	1.01	0.00	33.18
○Thread 3742413			30.00	33.38	89.89	1.01	0.00	33.18
○Thread 3742414			29.59	33.38	88.66	1.01	0.00	33.18
○Thread 3742415			29.90	33.38	89.59	1.01	0.00	33.18
○Thread 3742416			29.80	33.38	89.29	1.01	0.00	33.18
○Thread 3742417			29.85	33.38	89.41	1.01	0.00	33.18
○Thread 3742418			29.76	33.38	89.15	1.01	0.00	33.18

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	30.07
►Process 3776966+		48	30.07
○Thread 3776966			12.11	34.75	34.85	0.00	0.00	33.67
○Thread 3777033			29.88	33.71	88.64	1.02	0.01	33.50
○Thread 3777034			29.83	33.71	88.51	1.02	0.01	33.50
○Thread 3777035			30.07	33.71	89.22	1.02	0.01	33.50
○Thread 3777036			29.86	33.71	88.59	1.02	0.01	33.50
○Thread 3777037			29.87	33.71	88.61	1.02	0.01	33.51
○Thread 3777038			29.56	33.71	87.69	1.02	0.01	33.50
○Thread 3777039			29.96	33.71	88.89	1.02	0.01	33.50
○Thread 3777040			29.81	33.71	88.44	1.02	0.01	33.51
○Thread 3777041			29.85	33.71	88.55	1.02	0.01	33.50
○Thread 3777042			29.82	33.71	88.47	1.02	0.01	33.50
○Thread 3777043			29.85	33.71	88.56	1.02	0.01	33.50
○Thread 3777044			29.77	33.71	88.32	1.02	0.01	33.50
○Thread 3777045			29.74	33.71	88.22	1.02	0.01	33.50
○Thread 3777046			29.49	33.71	87.48	1.02	0.01	33.50
○Thread 3777047			29.97	33.71	88.90	1.02	0.01	33.50
○Thread 3777048			29.57	33.71	87.73	1.02	0.01	33.49
○Thread 3777049			29.45	33.71	87.37	1.02	0.01	33.50
○Thread 3777050			29.68	33.71	88.06	1.02	0.01	33.50
○Thread 3777051			29.86	33.71	88.59	1.02	0.01	33.50
○Thread 3777052			29.70	33.71	88.11	1.02	0.01	33.50
○Thread 3777053			29.74	33.71	88.20	1.02	0.01	33.50
○Thread 3777054			29.68	33.71	88.04	1.03	0.01	33.50
○Thread 3777055			29.70	33.71	88.11	1.03	0.01	33.50
○Thread 3777056			29.65	33.71	87.96	1.03	0.01	33.50
○Thread 3777057			29.73	33.71	88.20	1.03	0.01	33.50
○Thread 3777058			29.79	33.71	88.36	1.03	0.01	33.50
○Thread 3777059			29.47	33.71	87.41	1.03	0.01	33.50
○Thread 3777060			29.68	33.71	88.04	1.03	0.01	33.50
○Thread 3777061			29.92	33.71	88.76	1.03	0.01	33.50
○Thread 3777062			29.64	33.71	87.92	1.03	0.01	33.50
○Thread 3777063			29.72	33.71	88.17	1.03	0.01	33.50
○Thread 3777064			29.63	33.71	87.89	1.03	0.00	33.50
○Thread 3777065			29.49	33.71	87.47	1.03	0.00	33.50
○Thread 3777066			29.44	33.71	87.34	1.03	0.00	33.50
○Thread 3777067			29.67	33.71	88.00	1.03	0.00	33.49
○Thread 3777068			29.59	33.71	87.78	1.03	0.00	33.49
○Thread 3777069			29.82	33.71	88.46	1.03	0.00	33.51
○Thread 3777070			29.67	33.71	88.00	1.03	0.00	33.50
○Thread 3777071			29.75	33.71	88.25	1.03	0.00	33.50
○Thread 3777072			29.94	33.71	88.80	1.03	0.00	33.50
○Thread 3777073			29.62	33.72	87.85	1.03	0.00	33.50
○Thread 3777074			29.82	33.72	88.45	1.03	0.00	33.50
○Thread 3777075			29.67	33.72	88.00	1.03	0.00	33.50
○Thread 3777076			29.54	33.72	87.63	1.03	0.00	33.50
○Thread 3777077			29.58	33.72	87.75	1.03	0.00	33.50
○Thread 3777078			30.03	33.72	89.07	1.03	0.00	33.50
○Thread 3777079			29.72	33.72	88.15	1.03	0.00	33.49

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	29.60
►Process 3818650+		56	29.60
○Thread 3818650			10.80	34.75	31.08	0.00	0.00	33.69
○Thread 3818717			29.44	33.73	87.29	1.00	0.02	33.52
○Thread 3818718			29.38	33.73	87.10	1.00	0.02	33.51
○Thread 3818719			29.10	33.73	86.27	1.00	0.01	33.51
○Thread 3818720			29.23	33.73	86.67	1.00	0.01	33.52
○Thread 3818721			29.31	33.73	86.91	1.00	0.01	33.52
○Thread 3818722			29.25	33.73	86.71	1.01	0.01	33.51
○Thread 3818723			29.20	33.73	86.58	1.01	0.01	33.52
○Thread 3818724			29.46	33.73	87.35	1.01	0.01	33.52
○Thread 3818725			29.23	33.73	86.67	1.01	0.01	33.52
○Thread 3818726			29.41	33.73	87.19	1.01	0.01	33.51
○Thread 3818727			29.14	33.73	86.38	1.01	0.01	33.51
○Thread 3818728			29.18	33.73	86.50	1.01	0.01	33.50
○Thread 3818729			29.25	33.73	86.71	1.01	0.01	33.52
○Thread 3818730			29.31	33.73	86.90	1.01	0.01	33.52
○Thread 3818731			29.37	33.73	87.07	1.01	0.01	33.52
○Thread 3818732			29.37	33.73	87.08	1.01	0.01	33.52
○Thread 3818733			29.41	33.73	87.18	1.01	0.01	33.51
○Thread 3818734			29.07	33.73	86.19	1.01	0.01	33.52
○Thread 3818735			29.42	33.73	87.21	1.01	0.01	33.52
○Thread 3818736			29.24	33.73	86.69	1.01	0.01	33.51
○Thread 3818737			29.38	33.73	87.09	1.01	0.01	33.53
○Thread 3818738			28.98	33.73	85.91	1.01	0.01	33.52
○Thread 3818739			29.06	33.73	86.16	1.01	0.01	33.51
○Thread 3818740			29.41	33.73	87.18	1.01	0.01	33.53
○Thread 3818741			29.46	33.73	87.34	1.01	0.01	33.53
○Thread 3818742			28.94	33.73	85.79	1.01	0.01	33.51
○Thread 3818743			29.26	33.73	86.75	1.01	0.01	33.52
○Thread 3818744			29.47	33.74	87.36	1.01	0.01	33.51
○Thread 3818745			29.08	33.74	86.22	1.01	0.01	33.51
○Thread 3818746			29.27	33.74	86.78	1.01	0.01	33.51
○Thread 3818747			29.48	33.74	87.40	1.01	0.01	33.52
○Thread 3818748			29.37	33.74	87.07	1.01	0.01	33.51
○Thread 3818749			29.60	33.74	87.76	1.01	0.01	33.50
○Thread 3818750			29.27	33.74	86.78	1.01	0.01	33.52
○Thread 3818751			29.27	33.74	86.78	1.01	0.01	33.52
○Thread 3818752			28.95	33.74	85.83	1.01	0.01	33.51
○Thread 3818753			29.51	33.74	87.49	1.01	0.01	33.52
○Thread 3818754			29.15	33.74	86.39	1.01	0.01	33.51
○Thread 3818755			29.35	33.74	87.01	1.01	0.01	33.53
○Thread 3818756			29.40	33.74	87.14	1.01	0.00	33.52
○Thread 3818757			29.42	33.74	87.22	1.01	0.00	33.52
○Thread 3818758			29.47	33.74	87.35	1.01	0.00	33.51
○Thread 3818759			29.01	33.74	85.99	1.01	0.00	33.51
○Thread 3818760			29.15	33.74	86.42	1.01	0.00	33.52
○Thread 3818761			29.42	33.74	87.20	1.01	0.00	33.52
○Thread 3818762			29.12	33.74	86.33	1.01	0.00	33.51
○Thread 3818763			29.12	33.74	86.33	1.01	0.00	33.51
○Thread 3818764			29.16	33.74	86.44	1.01	0.00	33.51
○Thread 3818765			28.89	33.74	85.64	1.01	0.00	33.51
○Thread 3818766			29.02	33.74	86.03	1.01	0.00	33.51
○Thread 3818767			29.35	33.74	86.99	1.01	0.00	33.52
○Thread 3818768			29.27	33.74	86.77	1.01	0.00	33.51
○Thread 3818769			29.10	33.74	86.25	1.01	0.00	33.51
○Thread 3818770			29.18	33.74	86.49	1.01	0.00	33.51
○Thread 3818771			29.36	33.74	87.04	1.01	0.00	33.52

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 128 -ntg 0 -npl 16 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	37.73
►Process 3867475+		64	37.73
○Thread 3867475			9.80	45.21	21.67	0.00	0.00	44.12
○Thread 3867542			37.24	44.16	84.32	1.04	0.02	43.95
○Thread 3867543			37.03	44.16	83.85	1.04	0.02	43.92
○Thread 3867544			37.10	44.16	84.01	1.04	0.02	43.92
○Thread 3867545			37.07	44.16	83.93	1.04	0.02	43.93
○Thread 3867546			37.22	44.16	84.27	1.04	0.02	43.93
○Thread 3867547			37.46	44.16	84.81	1.04	0.02	43.92
○Thread 3867548			37.31	44.16	84.48	1.04	0.02	43.94
○Thread 3867549			37.19	44.16	84.20	1.04	0.02	43.93
○Thread 3867550			37.26	44.16	84.37	1.04	0.02	43.93
○Thread 3867551			37.27	44.16	84.39	1.04	0.02	43.93
○Thread 3867552			37.29	44.16	84.43	1.04	0.02	43.93
○Thread 3867553			37.08	44.16	83.95	1.04	0.01	43.93
○Thread 3867554			36.92	44.16	83.59	1.04	0.01	43.93
○Thread 3867555			37.11	44.16	84.02	1.04	0.01	43.93
○Thread 3867556			37.54	44.16	85.00	1.04	0.01	43.93
○Thread 3867557			37.12	44.16	84.04	1.04	0.01	43.94
○Thread 3867558			37.38	44.16	84.63	1.04	0.01	43.93
○Thread 3867559			37.50	44.16	84.91	1.04	0.01	43.93
○Thread 3867560			37.61	44.16	85.15	1.04	0.01	43.93
○Thread 3867561			36.89	44.16	83.52	1.04	0.01	43.95
○Thread 3867562			37.44	44.16	84.77	1.04	0.01	43.92
○Thread 3867563			37.64	44.16	85.23	1.04	0.01	43.95
○Thread 3867564			37.16	44.16	84.14	1.04	0.01	43.93
○Thread 3867565			37.20	44.17	84.22	1.04	0.01	43.94
○Thread 3867566			37.03	44.17	83.83	1.04	0.01	43.93
○Thread 3867567			37.33	44.17	84.52	1.04	0.01	43.96
○Thread 3867568			37.10	44.17	84.00	1.04	0.01	43.94
○Thread 3867569			37.37	44.17	84.61	1.04	0.01	43.93
○Thread 3867570			37.54	44.17	84.99	1.04	0.01	43.94
○Thread 3867571			37.26	44.17	84.36	1.04	0.01	43.95
○Thread 3867572			37.44	44.17	84.76	1.04	0.01	43.93
○Thread 3867573			37.58	44.17	85.09	1.04	0.01	43.93
○Thread 3867574			37.73	44.17	85.43	1.04	0.01	43.94
○Thread 3867575			37.45	44.17	84.78	1.04	0.01	43.93
○Thread 3867576			37.21	44.17	84.24	1.04	0.01	43.93
○Thread 3867577			37.57	44.17	85.05	1.04	0.01	43.92
○Thread 3867578			37.31	44.17	84.47	1.04	0.01	43.94
○Thread 3867579			36.97	44.17	83.69	1.04	0.01	43.93
○Thread 3867580			37.19	44.17	84.20	1.04	0.01	43.93
○Thread 3867581			37.11	44.17	84.02	1.04	0.01	43.93
○Thread 3867582			36.89	44.17	83.52	1.04	0.01	43.93
○Thread 3867583			37.22	44.17	84.27	1.04	0.01	43.93
○Thread 3867584			37.26	44.17	84.36	1.04	0.01	43.93
○Thread 3867585			37.07	44.17	83.94	1.04	0.01	43.93
○Thread 3867586			37.14	44.17	84.08	1.04	0.01	43.96
○Thread 3867587			37.52	44.17	84.95	1.04	0.01	43.94
○Thread 3867588			37.05	44.17	83.87	1.04	0.01	43.94
○Thread 3867589			37.03	44.17	83.83	1.04	0.00	43.92
○Thread 3867590			37.09	44.17	83.97	1.04	0.00	43.96
○Thread 3867591			36.96	44.17	83.67	1.04	0.00	43.95
○Thread 3867592			37.32	44.17	84.49	1.04	0.00	43.93
○Thread 3867593			37.04	44.17	83.86	1.04	0.00	43.95
○Thread 3867594			36.83	44.17	83.37	1.04	0.00	43.93
○Thread 3867595			37.25	44.17	84.32	1.04	0.00	43.93
○Thread 3867596			37.29	44.17	84.42	1.04	0.00	43.93
○Thread 3867597			37.36	44.17	84.58	1.04	0.00	43.93
○Thread 3867598			37.08	44.17	83.94	1.04	0.00	43.94
○Thread 3867599			37.09	44.17	83.97	1.04	0.00	43.92
○Thread 3867600			37.17	44.17	84.14	1.04	0.00	43.92
○Thread 3867601			37.29	44.17	84.42	1.04	0.00	43.94
○Thread 3867602			36.96	44.17	83.66	1.04	0.00	43.92
○Thread 3867603			37.45	44.17	84.77	1.04	0.00	43.94
○Thread 3867604			36.94	44.17	83.62	1.04	0.00	43.94

Report Configuration