OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	54.15
►Process 282242+		1	54.15
○Thread 282242			54.15	55.90	96.86	0.00	0.00	55.89

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	28.00
►Process 282376+		2	28.00
○Thread 282376			28.00	29.81	93.93	0.00	0.00	27.54
○Thread 282443			27.15	27.55	98.55	2.26	0.00	27.28

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	14.96
►Process 282511+		4	14.96
○Thread 282511			14.96	16.75	89.32	0.00	0.00	14.47
○Thread 282578			14.08	14.48	97.27	2.27	0.00	14.21
○Thread 282579			14.06	14.48	97.13	2.27	0.00	14.21
○Thread 282580			14.01	14.48	96.78	2.27	0.00	14.21

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	8.39
►Process 282600+		8	8.39
○Thread 282600			8.39	10.31	81.36	0.00	0.00	8.01
○Thread 282667			7.55	8.01	94.22	2.29	0.00	7.74
○Thread 282668			7.57	8.01	94.40	2.29	0.00	7.74
○Thread 282669			7.58	8.01	94.59	2.29	0.00	7.74
○Thread 282670			7.56	8.01	94.28	2.29	0.00	7.74
○Thread 282671			7.53	8.01	93.90	2.29	0.00	7.74
○Thread 282672			7.54	8.01	94.02	2.29	0.00	7.74
○Thread 282673			7.53	8.01	93.90	2.29	0.00	7.74

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	5.30
►Process 282695+		16	5.30
○Thread 282695			5.30	7.20	73.55	0.00	0.00	4.93
○Thread 282762			4.41	4.94	89.38	2.26	0.00	4.67
○Thread 282763			4.37	4.94	88.47	2.26	0.00	4.67
○Thread 282764			4.39	4.94	88.87	2.26	0.00	4.67
○Thread 282765			4.43	4.94	89.78	2.26	0.00	4.66
○Thread 282766			4.37	4.94	88.57	2.26	0.00	4.67
○Thread 282767			4.34	4.94	87.96	2.26	0.00	4.67
○Thread 282768			4.29	4.94	86.94	2.26	0.00	4.67
○Thread 282769			4.42	4.94	89.47	2.26	0.00	4.67
○Thread 282770			4.41	4.94	89.36	2.26	0.00	4.67
○Thread 282771			4.38	4.94	88.65	2.26	0.00	4.67
○Thread 282772			4.41	4.94	89.26	2.26	0.00	4.66
○Thread 282773			4.41	4.94	89.25	2.26	0.00	4.67
○Thread 282774			4.38	4.94	88.64	2.26	0.00	4.67
○Thread 282775			4.36	4.94	88.34	2.26	0.00	4.67
○Thread 282776			4.45	4.94	90.06	2.26	0.00	4.67

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	4.72
►Process 282846+		24	4.72
○Thread 282846			4.72	6.63	71.11	0.00	0.00	4.35
○Thread 282913			3.75	4.36	86.00	2.26	0.01	4.09
○Thread 282914			3.71	4.36	85.20	2.26	0.01	4.09
○Thread 282915			3.68	4.36	84.51	2.26	0.01	4.09
○Thread 282916			3.79	4.36	86.91	2.26	0.01	4.09
○Thread 282917			3.67	4.36	84.16	2.26	0.01	4.09
○Thread 282918			3.69	4.36	84.73	2.26	0.01	4.09
○Thread 282919			3.68	4.36	84.38	2.26	0.01	4.09
○Thread 282920			3.76	4.36	86.33	2.26	0.00	4.09
○Thread 282921			3.74	4.36	85.86	2.26	0.00	4.09
○Thread 282922			3.63	4.36	83.34	2.26	0.00	4.09
○Thread 282923			3.72	4.36	85.29	2.26	0.00	4.08
○Thread 282924			3.75	4.36	85.97	2.26	0.00	4.08
○Thread 282925			3.70	4.36	84.82	2.26	0.00	4.08
○Thread 282926			3.76	4.36	86.19	2.26	0.00	4.09
○Thread 282927			3.73	4.36	85.50	2.27	0.00	4.09
○Thread 282928			3.72	4.36	85.27	2.27	0.00	4.09
○Thread 282929			3.69	4.36	84.69	2.27	0.00	4.09
○Thread 282930			3.73	4.36	85.49	2.27	0.00	4.08
○Thread 282931			3.65	4.36	83.66	2.27	0.00	4.09
○Thread 282932			3.77	4.36	86.40	2.27	0.00	4.09
○Thread 282933			3.60	4.36	82.62	2.27	0.00	4.09
○Thread 282934			3.69	4.36	84.56	2.27	0.00	4.09
○Thread 282935			3.51	4.36	80.44	2.27	0.00	4.09

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	4.40
►Process 282955+		32	4.40
○Thread 282955			4.40	6.42	68.56	0.00	0.00	4.16
○Thread 283022			3.49	4.17	83.83	2.25	0.01	3.89
○Thread 283023			3.52	4.17	84.42	2.25	0.01	3.90
○Thread 283024			3.55	4.17	85.26	2.25	0.01	3.89
○Thread 283025			3.45	4.17	82.86	2.25	0.01	3.90
○Thread 283026			3.47	4.17	83.22	2.25	0.01	3.89
○Thread 283027			3.41	4.17	81.77	2.25	0.01	3.90
○Thread 283028			3.45	4.17	82.85	2.25	0.01	3.90
○Thread 283029			3.53	4.17	84.65	2.25	0.01	3.89
○Thread 283030			3.48	4.17	83.44	2.25	0.01	3.89
○Thread 283031			3.42	4.17	82.12	2.25	0.01	3.90
○Thread 283032			3.59	4.17	86.19	2.25	0.01	3.90
○Thread 283033			3.58	4.17	85.95	2.25	0.01	3.90
○Thread 283034			3.43	4.17	82.35	2.25	0.01	3.90
○Thread 283035			3.43	4.17	82.23	2.25	0.01	3.90
○Thread 283036			3.41	4.17	81.86	2.25	0.01	3.90
○Thread 283037			3.51	4.17	84.26	2.25	0.00	3.89
○Thread 283038			3.42	4.17	82.10	2.25	0.00	3.90
○Thread 283039			3.47	4.17	83.29	2.25	0.00	3.90
○Thread 283040			3.37	4.17	80.89	2.25	0.00	3.90
○Thread 283041			3.44	4.17	82.57	2.25	0.00	3.90
○Thread 283042			3.41	4.17	81.73	2.25	0.00	3.89
○Thread 283043			3.36	4.17	80.64	2.25	0.00	3.90
○Thread 283044			3.44	4.17	82.44	2.25	0.00	3.90
○Thread 283045			3.46	4.17	83.03	2.25	0.00	3.89
○Thread 283046			3.48	4.17	83.39	2.25	0.00	3.90
○Thread 283047			3.46	4.17	82.91	2.25	0.00	3.90
○Thread 283048			3.43	4.17	82.31	2.25	0.00	3.90
○Thread 283049			3.47	4.17	83.26	2.25	0.00	3.90
○Thread 283050			3.42	4.17	81.94	2.25	0.00	3.89
○Thread 283051			3.44	4.17	82.42	2.25	0.00	3.90
○Thread 283052			3.45	4.17	82.77	2.25	0.00	3.89

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	4.14
►Process 283072+		40	4.14
○Thread 283072			4.14	6.38	64.94	0.00	0.00	4.10
○Thread 283139			3.38	4.11	82.21	2.26	0.01	3.83
○Thread 283140			3.30	4.11	80.26	2.26	0.01	3.84
○Thread 283141			3.30	4.11	80.26	2.26	0.01	3.83
○Thread 283142			3.34	4.11	81.35	2.26	0.01	3.84
○Thread 283143			3.27	4.11	79.52	2.26	0.01	3.84
○Thread 283144			3.15	4.11	76.72	2.26	0.01	3.84
○Thread 283145			3.21	4.11	78.18	2.26	0.01	3.84
○Thread 283146			3.33	4.11	80.97	2.26	0.01	3.84
○Thread 283147			3.32	4.11	80.73	2.26	0.01	3.84
○Thread 283148			3.29	4.11	79.99	2.26	0.01	3.84
○Thread 283149			3.30	4.11	80.23	2.26	0.01	3.84
○Thread 283150			3.36	4.11	81.81	2.26	0.01	3.84
○Thread 283151			3.24	4.11	78.89	2.26	0.01	3.84
○Thread 283152			3.28	4.11	79.86	2.26	0.01	3.83
○Thread 283153			3.21	4.11	78.15	2.26	0.01	3.84
○Thread 283154			3.35	4.11	81.43	2.26	0.01	3.84
○Thread 283155			3.32	4.11	80.70	2.26	0.01	3.84
○Thread 283156			3.36	4.11	81.67	2.26	0.01	3.84
○Thread 283157			3.42	4.11	83.25	2.26	0.01	3.84
○Thread 283158			3.28	4.11	79.72	2.26	0.01	3.84
○Thread 283159			3.34	4.11	81.29	2.26	0.01	3.84
○Thread 283160			3.22	4.11	78.25	2.26	0.01	3.82
○Thread 283161			3.32	4.11	80.80	2.26	0.01	3.84
○Thread 283162			3.33	4.12	80.92	2.26	0.00	3.84
○Thread 283163			3.35	4.12	81.53	2.26	0.00	3.84
○Thread 283164			3.28	4.12	79.82	2.26	0.00	3.83
○Thread 283165			3.36	4.12	81.76	2.26	0.00	3.84
○Thread 283166			3.39	4.12	82.37	2.26	0.00	3.84
○Thread 283167			3.24	4.12	78.84	2.26	0.00	3.84
○Thread 283168			3.30	4.12	80.30	2.26	0.00	3.83
○Thread 283169			3.27	4.12	79.44	2.26	0.00	3.84
○Thread 283170			3.57	4.12	86.73	2.26	0.00	3.82
○Thread 283171			3.37	4.12	81.87	2.26	0.00	3.84
○Thread 283172			3.34	4.12	81.26	2.26	0.00	3.84
○Thread 283173			3.42	4.12	83.20	2.26	0.00	3.81
○Thread 283174			3.39	4.12	82.34	2.26	0.00	3.84
○Thread 283175			3.33	4.12	80.88	2.26	0.00	3.84
○Thread 283176			3.32	4.12	80.64	2.26	0.00	3.84
○Thread 283177			3.22	4.12	78.20	2.26	0.00	3.84

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	4.27
►Process 283198+		48	4.27
○Thread 283198			4.27	6.45	66.24	0.00	0.00	4.16
○Thread 283265			3.25	4.15	78.25	2.29	0.01	3.88
○Thread 283266			3.23	4.15	77.77	2.29	0.01	3.88
○Thread 283267			3.21	4.15	77.41	2.29	0.01	3.88
○Thread 283268			3.29	4.15	79.21	2.29	0.01	3.87
○Thread 283269			3.31	4.15	79.69	2.29	0.01	3.86
○Thread 283270			3.20	4.15	77.04	2.29	0.01	3.88
○Thread 283271			3.16	4.15	76.19	2.29	0.01	3.88
○Thread 283272			3.29	4.15	79.32	2.29	0.01	3.88
○Thread 283273			3.11	4.15	74.86	2.29	0.01	3.88
○Thread 283274			3.16	4.15	76.18	2.29	0.01	3.88
○Thread 283275			3.31	4.15	79.67	2.29	0.01	3.88
○Thread 283276			3.40	4.15	81.95	2.29	0.01	3.87
○Thread 283277			3.16	4.16	76.05	2.29	0.01	3.88
○Thread 283278			3.23	4.16	77.85	2.29	0.01	3.88
○Thread 283279			3.28	4.16	78.93	2.29	0.01	3.87
○Thread 283280			3.36	4.16	80.98	2.29	0.01	3.88
○Thread 283281			3.20	4.16	77.00	2.29	0.01	3.88
○Thread 283282			3.13	4.16	75.32	2.29	0.01	3.87
○Thread 283283			3.22	4.16	77.48	2.29	0.01	3.86
○Thread 283284			3.22	4.16	77.60	2.29	0.01	3.87
○Thread 283285			3.26	4.16	78.43	2.29	0.01	3.87
○Thread 283286			3.16	4.16	76.15	2.29	0.01	3.88
○Thread 283287			3.17	4.16	76.38	2.29	0.01	3.88
○Thread 283288			3.24	4.16	78.06	2.29	0.01	3.87
○Thread 283289			3.29	4.16	79.15	2.29	0.01	3.88
○Thread 283290			3.22	4.16	77.46	2.29	0.01	3.87
○Thread 283291			3.20	4.16	77.09	2.29	0.01	3.88
○Thread 283292			3.25	4.16	78.29	2.29	0.01	3.87
○Thread 283293			3.18	4.16	76.49	2.29	0.01	3.88
○Thread 283294			3.22	4.16	77.45	2.29	0.01	3.87
○Thread 283295			3.23	4.16	77.68	2.29	0.01	3.87
○Thread 283296			3.26	4.16	78.40	2.29	0.00	4.15
○Thread 283297			3.25	4.16	78.28	2.29	0.00	3.88
○Thread 283298			3.28	4.16	79.00	2.29	0.00	3.88
○Thread 283299			3.24	4.16	78.03	2.29	0.00	3.87
○Thread 283300			3.30	4.16	79.47	2.29	0.00	3.87
○Thread 283301			3.18	4.16	76.58	2.29	0.00	3.88
○Thread 283302			3.27	4.16	78.63	2.29	0.00	3.88
○Thread 283303			3.22	4.16	77.42	2.29	0.00	3.88
○Thread 283304			3.26	4.16	78.38	2.29	0.00	3.87
○Thread 283305			3.21	4.16	77.17	2.29	0.00	3.88
○Thread 283306			3.21	4.16	77.29	2.29	0.00	3.88
○Thread 283307			3.12	4.16	75.13	2.29	0.00	3.88
○Thread 283308			3.19	4.16	76.80	2.29	0.00	3.87
○Thread 283309			3.18	4.16	76.56	2.29	0.00	3.88
○Thread 283310			3.16	4.16	76.08	2.29	0.00	3.88
○Thread 283311			3.10	4.16	74.39	2.29	0.00	3.87

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	4.37
►Process 283381+		56	4.37
○Thread 283381			4.37	6.54	66.88	0.00	0.00	4.26
○Thread 283448			3.32	4.28	77.76	2.25	0.02	4.00
○Thread 283449			3.30	4.28	77.28	2.25	0.02	4.00
○Thread 283450			3.32	4.28	77.75	2.25	0.01	3.98
○Thread 283451			3.37	4.28	78.80	2.25	0.01	3.99
○Thread 283452			3.26	4.28	76.34	2.25	0.01	4.00
○Thread 283453			3.12	4.28	73.07	2.25	0.01	4.00
○Thread 283454			3.28	4.28	76.80	2.25	0.01	4.00
○Thread 283455			3.21	4.28	75.17	2.25	0.01	4.00
○Thread 283456			3.26	4.28	76.33	2.25	0.01	3.99
○Thread 283457			3.31	4.28	77.38	2.25	0.01	4.00
○Thread 283458			3.16	4.28	73.99	2.25	0.01	3.99
○Thread 283459			3.43	4.28	80.30	2.25	0.01	3.99
○Thread 283460			3.31	4.28	77.37	2.25	0.01	3.99
○Thread 283461			3.21	4.28	75.15	2.25	0.01	3.99
○Thread 283462			3.30	4.28	77.25	2.25	0.01	4.00
○Thread 283463			3.40	4.28	79.47	2.25	0.01	4.00
○Thread 283464			3.27	4.28	76.42	2.25	0.01	3.99
○Thread 283465			3.20	4.28	74.90	2.25	0.01	3.98
○Thread 283466			3.33	4.28	77.82	2.25	0.01	4.00
○Thread 283467			3.30	4.28	77.12	2.25	0.01	4.00
○Thread 283468			3.14	4.28	73.49	2.25	0.01	3.99
○Thread 283469			3.29	4.28	76.88	2.25	0.01	4.00
○Thread 283470			3.19	4.28	74.54	2.25	0.01	4.00
○Thread 283471			3.25	4.28	75.94	2.25	0.01	4.00
○Thread 283472			3.24	4.28	75.70	2.25	0.01	4.00
○Thread 283473			3.17	4.28	74.06	2.25	0.01	4.00
○Thread 283474			3.33	4.28	77.80	2.25	0.01	4.00
○Thread 283475			3.34	4.28	78.03	2.25	0.01	4.00
○Thread 283476			3.22	4.28	75.34	2.25	0.01	4.00
○Thread 283477			3.34	4.28	78.14	2.25	0.01	3.99
○Thread 283478			3.15	4.28	73.58	2.25	0.01	4.00
○Thread 283479			3.36	4.28	78.48	2.25	0.01	4.00
○Thread 283480			3.25	4.28	76.03	2.25	0.01	4.00
○Thread 283481			3.32	4.28	77.54	2.25	0.01	4.00
○Thread 283482			3.24	4.28	75.67	2.25	0.01	3.99
○Thread 283483			3.26	4.28	76.13	2.25	0.01	4.00
○Thread 283484			3.15	4.28	73.68	2.25	0.01	3.98
○Thread 283485			3.22	4.28	75.31	2.25	0.01	3.99
○Thread 283486			3.31	4.28	77.41	2.25	0.01	4.00
○Thread 283487			3.20	4.28	74.84	2.25	0.00	4.00
○Thread 283488			3.14	4.28	73.43	2.25	0.00	4.00
○Thread 283489			3.25	4.28	75.88	2.25	0.00	4.00
○Thread 283490			3.23	4.28	75.53	2.25	0.00	4.00
○Thread 283491			3.27	4.28	76.34	2.25	0.00	3.99
○Thread 283492			3.23	4.28	75.52	2.25	0.00	4.00
○Thread 283493			3.27	4.28	76.45	2.25	0.00	4.00
○Thread 283494			3.22	4.28	75.17	2.25	0.00	4.00
○Thread 283495			3.74	4.28	87.30	2.25	0.00	4.00
○Thread 283496			3.32	4.28	77.61	2.25	0.00	4.00
○Thread 283497			3.42	4.28	79.83	2.25	0.00	4.00
○Thread 283498			3.34	4.28	77.96	2.25	0.00	4.00
○Thread 283499			3.24	4.28	75.62	2.26	0.00	4.00
○Thread 283500			3.30	4.28	77.14	2.26	0.00	4.00
○Thread 283501			3.07	4.28	71.65	2.26	0.00	4.00
○Thread 283502			3.18	4.28	74.21	2.26	0.00	4.00

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q4_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 2 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	4.38
►Process 283522+		64	4.38
○Thread 283522			4.38	6.77	64.69	0.00	0.00	4.42
○Thread 283589			3.33	4.43	75.22	2.32	0.02	4.13
○Thread 283590			3.35	4.43	75.56	2.32	0.02	4.16
○Thread 283591			3.29	4.43	74.32	2.32	0.02	4.15
○Thread 283592			3.31	4.43	74.65	2.32	0.02	4.16
○Thread 283593			3.24	4.43	73.18	2.32	0.02	4.16
○Thread 283594			3.23	4.43	72.84	2.32	0.02	4.16
○Thread 283595			3.24	4.43	73.18	2.32	0.02	4.13
○Thread 283596			3.40	4.43	76.67	2.32	0.02	4.15
○Thread 283597			3.25	4.43	73.29	2.32	0.02	4.16
○Thread 283598			3.23	4.43	72.83	2.32	0.02	4.15
○Thread 283599			3.25	4.43	73.39	2.32	0.01	4.15
○Thread 283600			3.35	4.44	75.53	2.32	0.01	4.15
○Thread 283601			3.22	4.44	72.60	2.32	0.01	4.16
○Thread 283602			3.24	4.44	73.16	2.32	0.01	4.15
○Thread 283603			3.23	4.44	72.82	2.32	0.01	4.14
○Thread 283604			3.31	4.44	74.62	2.32	0.01	4.16
○Thread 283605			3.33	4.44	75.18	2.32	0.01	4.15
○Thread 283606			3.25	4.44	73.37	2.32	0.01	4.16
○Thread 283607			3.31	4.44	74.72	2.32	0.01	4.16
○Thread 283608			3.30	4.44	74.50	2.32	0.01	4.16
○Thread 283609			3.25	4.44	73.37	2.32	0.01	4.14
○Thread 283610			3.34	4.44	75.39	2.32	0.01	4.16
○Thread 283611			3.35	4.44	75.50	2.32	0.01	4.14
○Thread 283612			3.27	4.44	73.81	2.32	0.01	4.13
○Thread 283613			3.21	4.44	72.45	2.32	0.01	4.16
○Thread 283614			3.22	4.44	72.68	2.32	0.01	4.16
○Thread 283615			3.35	4.44	75.60	2.32	0.01	4.15
○Thread 283616			3.28	4.44	73.91	2.32	0.01	4.15
○Thread 283617			3.26	4.44	73.57	2.32	0.01	4.16
○Thread 283618			3.26	4.44	73.45	2.32	0.01	4.15
○Thread 283619			3.21	4.44	72.32	2.32	0.01	4.16
○Thread 283620			3.36	4.44	75.70	2.32	0.01	4.15
○Thread 283621			3.17	4.44	71.42	2.32	0.01	4.15
○Thread 283622			3.32	4.44	74.91	2.32	0.01	4.14
○Thread 283623			3.27	4.44	73.78	2.32	0.01	4.15
○Thread 283624			3.29	4.44	74.23	2.32	0.01	4.14
○Thread 283625			3.21	4.44	72.31	2.32	0.01	4.14
○Thread 283626			3.24	4.44	73.10	2.32	0.01	4.16
○Thread 283627			3.24	4.44	73.09	2.32	0.01	4.15
○Thread 283628			3.25	4.44	73.20	2.32	0.01	4.16
○Thread 283629			3.28	4.44	73.99	2.32	0.01	4.15
○Thread 283630			3.36	4.44	75.67	2.32	0.01	4.15
○Thread 283631			3.29	4.44	74.21	2.32	0.01	4.16
○Thread 283632			3.27	4.44	73.76	2.32	0.01	4.15
○Thread 283633			3.24	4.44	72.96	2.32	0.01	4.16
○Thread 283634			3.36	4.44	75.66	2.32	0.01	4.15
○Thread 283635			3.25	4.44	73.30	2.32	0.01	4.16
○Thread 283636			3.62	4.44	81.63	2.32	0.00	4.16
○Thread 283637			3.32	4.44	74.87	2.32	0.00	4.15
○Thread 283638			3.28	4.44	73.96	2.32	0.00	4.16
○Thread 283639			3.30	4.44	74.30	2.32	0.00	4.15
○Thread 283640			3.27	4.44	73.73	2.32	0.00	4.16
○Thread 283641			3.07	4.44	69.23	2.33	0.00	4.14
○Thread 283642			3.20	4.44	72.04	2.33	0.00	4.11
○Thread 283643			3.27	4.44	73.61	2.33	0.00	4.16
○Thread 283644			3.26	4.44	73.38	2.33	0.00	4.15
○Thread 283645			3.34	4.44	75.30	2.33	0.00	4.15
○Thread 283646			3.22	4.44	72.48	2.33	0.00	4.15
○Thread 283647			3.16	4.44	71.24	2.33	0.00	4.15
○Thread 283648			3.54	4.44	79.68	2.33	0.00	4.15
○Thread 283649			3.30	4.44	74.38	2.33	0.00	4.15
○Thread 283650			3.35	4.44	75.51	2.33	0.00	4.14
○Thread 283651			3.38	4.44	76.18	2.33	0.00	4.16

Report Configuration