OV - exec - Topology

Software Topology

▼Scalability - 1x1

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threadsOMP_NUM_THREADS: 1

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	1	278.42
►Process 56201+		1	278.42
○Thread 56201			278.42	279.51	99.61	0.00	0.00	279.47

▶Scalability - 1x2

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 2OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	2	139.27
►Process 56577+		2	139.27
○Thread 56577			139.27	140.32	99.26	0.00	0.00	140.27
○Thread 56644			138.86	139.25	99.72	1.06	-0.00	138.91

▶Scalability - 1x4

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 4OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	4	70.42
►Process 56715+		4	70.42
○Thread 56715			70.42	71.46	98.56	0.00	0.00	71.42
○Thread 56784			69.99	70.39	99.43	1.07	0.00	70.04
○Thread 56785			70.00	70.39	99.45	1.07	0.00	70.04
○Thread 56786			70.00	70.39	99.45	1.07	-0.00	70.04

▶Scalability - 1x8

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 8OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	8	36.00
►Process 56860+		8	36.00
○Thread 56860			36.00	37.13	96.96	0.00	0.00	37.09
○Thread 56927			35.60	36.06	98.73	1.07	0.00	35.71
○Thread 56928			35.59	36.06	98.70	1.07	0.00	35.71
○Thread 56929			35.64	36.06	98.84	1.07	0.00	35.71
○Thread 56930			35.63	36.06	98.81	1.07	0.00	35.71
○Thread 56931			35.63	36.06	98.80	1.07	0.00	35.71
○Thread 56932			35.65	36.06	98.87	1.07	0.00	35.71
○Thread 56933			35.65	36.06	98.87	1.07	-0.00	35.71

▶Scalability - 1x16

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 16OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	16	19.27
►Process 57002+		16	19.27
○Thread 57002			19.27	20.45	94.22	0.00	0.00	20.41
○Thread 57069			18.83	19.39	97.13	1.07	0.00	19.04
○Thread 57070			18.82	19.39	97.08	1.07	0.00	19.04
○Thread 57071			18.88	19.39	97.39	1.07	0.00	19.04
○Thread 57072			18.81	19.39	97.06	1.07	0.00	19.03
○Thread 57073			18.85	19.39	97.24	1.07	0.00	19.03
○Thread 57074			18.86	19.39	97.32	1.07	0.00	19.04
○Thread 57075			18.80	19.38	96.98	1.07	0.00	19.03
○Thread 57076			18.85	19.38	97.24	1.07	0.00	19.04
○Thread 57077			18.84	19.38	97.19	1.07	0.00	19.03
○Thread 57078			18.90	19.38	97.50	1.07	0.00	19.03
○Thread 57079			18.79	19.38	96.96	1.07	0.00	19.03
○Thread 57080			18.80	19.38	97.01	1.07	0.00	19.04
○Thread 57081			18.87	19.38	97.35	1.07	0.00	19.03
○Thread 57082			18.79	19.38	96.93	1.07	0.00	19.04
○Thread 57083			18.80	19.38	96.99	1.07	-0.00	19.03

▶Scalability - 1x24

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 24OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	24	14.46
►Process 57103+		24	14.46
○Thread 57103			14.46	15.74	91.89	0.00	0.00	15.69
○Thread 57170			14.01	14.67	95.47	1.07	0.00	14.31
○Thread 57171			14.00	14.67	95.44	1.07	0.00	14.32
○Thread 57172			14.00	14.67	95.44	1.07	0.00	14.32
○Thread 57173			14.09	14.67	96.05	1.07	0.00	14.31
○Thread 57174			14.05	14.67	95.78	1.07	0.00	14.32
○Thread 57175			13.94	14.67	95.03	1.07	0.00	14.32
○Thread 57176			14.03	14.67	95.64	1.07	0.00	14.31
○Thread 57177			14.06	14.67	95.85	1.07	0.00	14.31
○Thread 57178			14.06	14.67	95.82	1.07	0.00	14.32
○Thread 57179			14.11	14.67	96.19	1.07	0.00	14.32
○Thread 57180			14.00	14.67	95.44	1.07	0.00	14.32
○Thread 57181			13.99	14.67	95.37	1.07	0.00	14.31
○Thread 57182			14.02	14.67	95.58	1.07	0.00	14.31
○Thread 57183			14.08	14.67	95.95	1.07	0.00	14.32
○Thread 57184			14.07	14.67	95.89	1.07	0.00	14.32
○Thread 57185			14.03	14.67	95.65	1.07	0.00	14.32
○Thread 57186			14.04	14.67	95.72	1.07	0.00	14.32
○Thread 57187			14.00	14.67	95.45	1.07	0.00	14.32
○Thread 57188			13.98	14.67	95.28	1.07	0.00	14.31
○Thread 57189			14.06	14.67	95.86	1.07	0.00	14.31
○Thread 57190			14.02	14.67	95.58	1.07	0.00	14.31
○Thread 57191			13.93	14.67	94.94	1.07	0.00	14.31
○Thread 57192			14.02	14.67	95.55	1.07	-0.00	14.31

▶Scalability - 1x32

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 32OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	32	11.50
►Process 57261+		32	11.50
○Thread 57261			11.50	12.97	88.69	0.00	0.00	12.93
○Thread 57328			11.08	11.90	93.08	1.07	0.00	11.54
○Thread 57329			11.17	11.90	93.84	1.07	0.00	11.55
○Thread 57330			11.16	11.90	93.80	1.07	0.00	11.54
○Thread 57331			11.19	11.90	94.01	1.07	0.00	11.53
○Thread 57332			11.13	11.90	93.51	1.07	0.00	11.53
○Thread 57333			11.22	11.90	94.27	1.07	0.00	11.53
○Thread 57334			11.20	11.90	94.14	1.07	0.00	11.55
○Thread 57335			11.23	11.90	94.39	1.07	0.00	11.53
○Thread 57336			11.19	11.90	94.06	1.07	0.00	11.55
○Thread 57337			11.15	11.90	93.68	1.07	0.00	11.53
○Thread 57338			11.15	11.90	93.72	1.07	0.00	11.54
○Thread 57339			11.13	11.90	93.51	1.07	0.00	11.54
○Thread 57340			11.20	11.90	94.10	1.07	0.00	11.54
○Thread 57341			11.12	11.90	93.43	1.07	0.00	11.55
○Thread 57342			11.15	11.90	93.68	1.07	0.00	11.54
○Thread 57343			11.22	11.90	94.32	1.07	0.00	11.54
○Thread 57344			11.20	11.90	94.15	1.07	0.00	11.54
○Thread 57345			11.16	11.90	93.77	1.07	0.00	11.54
○Thread 57346			11.15	11.90	93.69	1.07	0.00	11.54
○Thread 57347			11.14	11.90	93.65	1.07	0.00	11.53
○Thread 57348			11.13	11.90	93.52	1.07	0.00	11.54
○Thread 57349			11.17	11.90	93.86	1.07	0.00	11.54
○Thread 57350			11.19	11.90	94.03	1.07	0.00	11.54
○Thread 57351			11.18	11.90	93.99	1.07	0.00	11.54
○Thread 57352			11.13	11.90	93.57	1.07	0.00	11.54
○Thread 57353			11.16	11.90	93.82	1.07	0.00	11.54
○Thread 57354			11.14	11.90	93.65	1.07	0.00	11.54
○Thread 57355			11.09	11.90	93.19	1.07	0.00	11.54
○Thread 57356			11.14	11.90	93.61	1.07	0.00	11.54
○Thread 57357			11.11	11.89	93.40	1.07	0.00	11.54
○Thread 57358			11.04	11.89	92.81	1.07	-0.00	11.53

▶Scalability - 1x40

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 40OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	40	10.08
►Process 57378+		40	10.08
○Thread 57378			10.08	11.66	86.46	0.00	0.00	11.62
○Thread 57445			9.74	10.59	91.95	1.07	0.00	10.22
○Thread 57446			9.84	10.59	92.95	1.07	0.00	10.23
○Thread 57447			9.76	10.59	92.19	1.07	0.00	10.23
○Thread 57448			9.75	10.59	92.10	1.07	0.00	10.23
○Thread 57449			9.78	10.59	92.38	1.07	0.00	10.22
○Thread 57450			9.82	10.59	92.71	1.07	0.00	10.23
○Thread 57451			9.78	10.59	92.38	1.07	0.00	10.22
○Thread 57452			9.70	10.59	91.63	1.07	0.00	10.23
○Thread 57453			9.85	10.59	93.00	1.07	0.00	10.22
○Thread 57454			9.81	10.59	92.67	1.07	0.00	10.22
○Thread 57455			9.94	10.59	93.90	1.07	-0.00	10.23
○Thread 57456			9.77	10.59	92.25	1.07	0.00	10.22
○Thread 57457			9.76	10.59	92.15	1.07	0.00	10.23
○Thread 57458			9.80	10.59	92.58	1.07	0.00	10.22
○Thread 57459			9.72	10.59	91.78	1.07	0.00	10.22
○Thread 57460			9.74	10.59	92.01	1.07	-0.00	10.23
○Thread 57461			9.70	10.59	91.59	1.07	0.00	10.23
○Thread 57462			9.69	10.59	91.50	1.07	0.00	10.23
○Thread 57463			9.83	10.58	92.87	1.07	0.00	10.23
○Thread 57464			9.75	10.58	92.07	1.07	0.00	10.23
○Thread 57465			9.77	10.58	92.26	1.07	0.00	10.23
○Thread 57466			9.77	10.58	92.30	1.07	0.00	10.23
○Thread 57467			9.74	10.58	92.02	1.07	0.00	10.23
○Thread 57468			9.75	10.58	92.07	1.07	0.00	10.23
○Thread 57469			9.79	10.58	92.49	1.07	0.00	10.23
○Thread 57470			9.74	10.58	91.98	1.07	0.00	10.23
○Thread 57471			9.71	10.58	91.69	1.07	0.00	10.23
○Thread 57472			9.78	10.58	92.36	1.07	0.00	10.23
○Thread 57473			9.79	10.58	92.50	1.07	0.00	10.22
○Thread 57474			9.71	10.58	91.70	1.07	0.00	10.23
○Thread 57475			9.71	10.58	91.74	1.07	0.00	10.22
○Thread 57476			9.77	10.58	92.26	1.07	0.00	10.23
○Thread 57477			9.80	10.58	92.60	1.08	0.00	10.23
○Thread 57478			9.75	10.58	92.08	1.08	0.00	10.23
○Thread 57479			9.72	10.58	91.79	1.08	0.00	10.23
○Thread 57480			9.80	10.58	92.60	1.08	0.00	10.23
○Thread 57481			9.67	10.58	91.32	1.08	0.00	10.23
○Thread 57482			9.75	10.58	92.13	1.08	0.00	10.23
○Thread 57483			9.69	10.58	91.56	1.08	-0.00	10.23

▶Scalability - 1x48

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 48OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	48	9.12
►Process 57503+		48	9.12
○Thread 57503			9.12	10.77	84.70	0.00	0.00	10.72
○Thread 57570			8.84	9.71	91.07	1.06	-0.00	9.33
○Thread 57571			8.81	9.71	90.71	1.06	-0.00	9.34
○Thread 57572			8.77	9.71	90.30	1.06	-0.00	9.34
○Thread 57573			8.78	9.71	90.45	1.06	0.00	9.34
○Thread 57574			8.77	9.71	90.30	1.06	-0.00	9.34
○Thread 57575			8.85	9.71	91.12	1.06	-0.00	9.34
○Thread 57576			9.00	9.71	92.72	1.06	-0.00	9.34
○Thread 57577			8.84	9.71	91.07	1.06	0.00	9.34
○Thread 57578			8.78	9.71	90.41	1.06	0.00	9.34
○Thread 57579			8.78	9.71	90.46	1.06	0.00	9.34
○Thread 57580			8.77	9.71	90.30	1.06	0.00	9.34
○Thread 57581			8.74	9.71	90.05	1.06	-0.00	9.34
○Thread 57582			8.73	9.71	89.89	1.06	-0.00	9.33
○Thread 57583			8.76	9.71	90.20	1.06	0.00	9.34
○Thread 57584			8.79	9.71	90.51	1.06	0.00	9.33
○Thread 57585			8.89	9.71	91.60	1.06	0.00	9.33
○Thread 57586			8.95	9.71	92.16	1.06	0.00	9.34
○Thread 57587			8.81	9.71	90.72	1.06	0.00	9.33
○Thread 57588			8.72	9.71	89.80	1.06	0.00	9.34
○Thread 57589			8.68	9.71	89.39	1.06	0.00	9.34
○Thread 57590			8.84	9.71	91.09	1.06	0.00	9.34
○Thread 57591			8.81	9.71	90.73	1.06	0.00	9.34
○Thread 57592			8.79	9.70	90.52	1.06	-0.00	9.33
○Thread 57593			8.76	9.70	90.26	1.06	0.00	9.34
○Thread 57594			8.84	9.70	91.04	1.06	0.00	9.33
○Thread 57595			8.84	9.70	91.09	1.06	0.00	9.34
○Thread 57596			8.79	9.70	90.53	1.06	0.00	9.33
○Thread 57597			8.70	9.70	89.65	1.06	0.00	9.34
○Thread 57598			8.80	9.70	90.68	1.06	0.00	9.34
○Thread 57599			8.74	9.70	90.01	1.06	0.00	9.34
○Thread 57600			8.81	9.70	90.79	1.06	0.00	9.34
○Thread 57601			8.72	9.70	89.81	1.06	0.00	9.34
○Thread 57602			8.78	9.70	90.48	1.06	0.00	9.34
○Thread 57603			8.76	9.70	90.22	1.06	0.00	9.34
○Thread 57604			8.78	9.70	90.43	1.06	0.00	9.34
○Thread 57605			8.76	9.70	90.22	1.06	0.00	9.34
○Thread 57606			8.79	9.70	90.59	1.06	0.00	9.34
○Thread 57607			8.70	9.70	89.66	1.06	0.00	9.34
○Thread 57608			8.71	9.70	89.76	1.06	0.00	9.34
○Thread 57609			8.76	9.70	90.23	1.06	0.00	9.34
○Thread 57610			8.66	9.70	89.25	1.06	0.00	9.34
○Thread 57611			8.80	9.70	90.69	1.06	0.00	9.34
○Thread 57612			8.66	9.70	89.25	1.06	0.00	9.33
○Thread 57613			8.74	9.70	90.03	1.06	0.00	9.34
○Thread 57614			8.76	9.70	90.23	1.06	0.00	9.34
○Thread 57615			8.76	9.70	90.23	1.06	0.00	9.34
○Thread 57616			8.80	9.70	90.65	1.06	-0.00	9.34

▶Scalability - 1x56

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 56OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	56	9.12
►Process 57685+		56	9.12
○Thread 57685			9.12	10.83	84.22	0.00	0.00	10.79
○Thread 57752			8.69	9.75	89.09	1.08	0.00	9.38
○Thread 57753			8.69	9.75	89.09	1.08	0.00	9.38
○Thread 57754			8.70	9.75	89.15	1.08	0.00	9.37
○Thread 57755			8.60	9.75	88.12	1.08	0.00	9.37
○Thread 57756			8.69	9.75	89.05	1.08	0.00	9.39
○Thread 57757			8.70	9.75	89.15	1.08	0.00	9.38
○Thread 57758			8.69	9.75	89.10	1.08	0.00	9.37
○Thread 57759			8.60	9.75	88.18	1.08	0.00	9.38
○Thread 57760			8.71	9.75	89.26	1.08	0.00	9.36
○Thread 57761			8.69	9.75	89.10	1.08	0.00	9.36
○Thread 57762			8.60	9.75	88.13	1.08	0.00	9.38
○Thread 57763			8.59	9.75	88.08	1.08	0.00	9.38
○Thread 57764			8.58	9.75	87.98	1.08	0.00	9.37
○Thread 57765			8.68	9.75	88.95	1.08	0.00	9.37
○Thread 57766			8.70	9.75	89.21	1.08	0.00	9.38
○Thread 57767			8.60	9.75	88.13	1.08	0.00	9.37
○Thread 57768			8.54	9.75	87.52	1.08	0.00	9.38
○Thread 57769			8.69	9.75	89.11	1.08	0.00	9.37
○Thread 57770			8.72	9.75	89.37	1.08	0.00	9.37
○Thread 57771			8.65	9.75	88.65	1.08	0.00	9.37
○Thread 57772			8.59	9.75	88.04	1.08	0.00	9.38
○Thread 57773			8.60	9.75	88.14	1.08	0.00	9.37
○Thread 57774			8.68	9.75	89.01	1.08	0.00	9.38
○Thread 57775			8.61	9.75	88.24	1.08	0.00	9.38
○Thread 57776			8.81	9.75	90.35	1.08	0.00	9.38
○Thread 57777			8.69	9.75	89.12	1.08	0.00	9.37
○Thread 57778			8.61	9.75	88.25	1.08	0.00	9.37
○Thread 57779			8.69	9.75	89.12	1.08	0.00	9.37
○Thread 57780			8.68	9.75	88.97	1.08	0.00	9.37
○Thread 57781			8.88	9.75	91.02	1.08	0.00	9.38
○Thread 57782			8.77	9.75	89.94	1.08	0.00	9.38
○Thread 57783			8.74	9.75	89.58	1.08	0.00	9.37
○Thread 57784			8.60	9.75	88.20	1.08	0.00	9.38
○Thread 57785			8.67	9.75	88.92	1.08	0.00	9.38
○Thread 57786			8.76	9.75	89.84	1.08	0.00	9.37
○Thread 57787			8.71	9.75	89.28	1.08	0.00	9.38
○Thread 57788			8.71	9.75	89.33	1.08	0.00	9.37
○Thread 57789			8.72	9.75	89.44	1.08	0.00	9.38
○Thread 57790			8.65	9.75	88.67	1.08	0.00	9.37
○Thread 57791			8.78	9.75	90.00	1.08	0.00	9.38
○Thread 57792			8.63	9.75	88.46	1.08	0.00	9.38
○Thread 57793			8.67	9.75	88.88	1.08	0.00	9.37
○Thread 57794			8.73	9.75	89.54	1.08	0.00	9.38
○Thread 57795			8.69	9.75	89.13	1.08	0.00	9.37
○Thread 57796			8.56	9.75	87.75	1.08	0.00	9.38
○Thread 57797			8.63	9.75	88.47	1.08	0.00	9.38
○Thread 57798			8.65	9.75	88.68	1.08	0.00	9.39
○Thread 57799			8.65	9.75	88.73	1.08	0.00	9.37
○Thread 57800			8.57	9.75	87.86	1.08	0.00	9.38
○Thread 57801			8.69	9.75	89.09	1.08	0.00	9.38
○Thread 57802			8.72	9.75	89.45	1.08	0.00	9.38
○Thread 57803			8.62	9.75	88.37	1.08	0.00	9.37
○Thread 57804			8.55	9.75	87.71	1.08	0.00	9.37
○Thread 57805			8.65	9.75	88.68	1.08	0.00	9.37
○Thread 57806			8.68	9.75	88.99	1.08	-0.00	9.37

▶Scalability - 1x64

Number processes: 1 Number nodes: 1 Run Command: <executable> -m meta-llama-3.1-8b-instruct-Q8_0.gguf -t <OMP_NUM_THREADS> -b 2048 -ub 512 -npp 0 -ntg 128 -npl 8 -c 16384 --seed 0 --output-format jsonlMPI Command: mpirun -n <number_processes> --bind-to none --report-bindings /usr/bin/numactl -C 0-63Run Directory: .OMP_NUM_THREADS: 64OMP_PROC_BIND: spreadOMP_DISPLAY_AFFINITY: TRUEOMP_AFFINITY_FORMAT: 'OMP: pid %P tid %i thread %n bound to OS proc set {%A}'OMP_DISPLAY_ENV: TRUEOMP_PLACES: threads

ID	Observed Processes	Observed Threads	Time(s)	Elapsed Time (s)	Active Time (%)	Start (after process) (s)	End (before process) (s)	Maximum Time on the Same CPU (s)
►Node ip-172-31-46-37.ec2.internal+	1	64	8.53
►Process 57828+		64	8.53
○Thread 57828			8.53	10.31	82.75	0.00	0.00	10.25
○Thread 57895			8.11	9.23	87.82	1.07	0.00	8.85
○Thread 57896			8.15	9.23	88.26	1.07	-0.00	8.85
○Thread 57897			8.06	9.23	87.28	1.07	-0.00	8.84
○Thread 57898			8.10	9.23	87.72	1.07	0.00	8.87
○Thread 57899			8.05	9.23	87.18	1.07	-0.00	8.85
○Thread 57900			8.10	9.23	87.67	1.07	-0.00	8.85
○Thread 57901			8.19	9.23	88.70	1.07	-0.00	8.86
○Thread 57902			8.11	9.23	87.83	1.08	-0.00	8.85
○Thread 57903			8.04	9.23	87.07	1.08	-0.00	8.85
○Thread 57904			8.18	9.23	88.59	1.08	0.00	8.85
○Thread 57905			8.04	9.23	87.08	1.08	0.00	8.85
○Thread 57906			8.16	9.23	88.38	1.08	0.00	8.85
○Thread 57907			8.08	9.23	87.51	1.08	0.00	8.85
○Thread 57908			8.05	9.23	87.19	1.08	-0.00	8.85
○Thread 57909			8.14	9.23	88.11	1.08	-0.00	8.85
○Thread 57910			8.05	9.23	87.13	1.08	-0.00	8.85
○Thread 57911			8.09	9.23	87.57	1.08	-0.00	8.85
○Thread 57912			8.04	9.23	87.03	1.08	-0.00	8.85
○Thread 57913			8.00	9.23	86.60	1.08	-0.00	8.85
○Thread 57914			8.07	9.23	87.41	1.08	-0.00	8.85
○Thread 57915			8.07	9.23	87.41	1.08	-0.00	8.85
○Thread 57916			8.15	9.23	88.22	1.08	-0.00	8.85
○Thread 57917			8.12	9.23	87.90	1.08	-0.00	8.85
○Thread 57918			8.11	9.23	87.79	1.08	-0.00	8.85
○Thread 57919			8.12	9.23	87.96	1.08	-0.00	8.85
○Thread 57920			8.06	9.23	87.25	1.08	-0.00	8.84
○Thread 57921			8.17	9.23	88.44	1.08	-0.00	8.85
○Thread 57922			8.04	9.23	87.09	1.08	-0.00	8.85
○Thread 57923			8.07	9.23	87.42	1.08	-0.00	8.85
○Thread 57924			8.16	9.23	88.39	1.08	-0.00	8.85
○Thread 57925			8.16	9.23	88.39	1.08	-0.00	8.85
○Thread 57926			8.00	9.23	86.66	1.08	-0.00	8.85
○Thread 57927			8.01	9.23	86.77	1.08	-0.00	8.85
○Thread 57928			8.03	9.23	86.93	1.08	-0.00	8.85
○Thread 57929			8.00	9.23	86.61	1.08	-0.00	8.85
○Thread 57930			8.04	9.23	87.04	1.08	-0.00	8.85
○Thread 57931			8.06	9.23	87.32	1.08	-0.00	8.85
○Thread 57932			8.09	9.23	87.59	1.08	-0.00	8.85
○Thread 57933			8.12	9.23	87.91	1.08	-0.00	8.84
○Thread 57934			8.16	9.23	88.40	1.08	-0.00	8.85
○Thread 57935			8.06	9.23	87.27	1.08	-0.00	8.85
○Thread 57936			8.11	9.23	87.86	1.08	-0.00	8.84
○Thread 57937			8.04	9.23	87.05	1.08	0.00	8.86
○Thread 57938			8.04	9.23	87.11	1.08	-0.00	8.85
○Thread 57939			8.13	9.23	88.03	1.08	-0.00	8.85
○Thread 57940			8.08	9.23	87.54	1.08	-0.00	8.88
○Thread 57941			8.14	9.23	88.14	1.08	-0.00	8.85
○Thread 57942			8.01	9.23	86.73	1.08	-0.00	8.85
○Thread 57943			8.01	9.23	86.79	1.08	-0.00	8.83
○Thread 57944			8.00	9.23	86.62	1.08	0.00	8.85
○Thread 57945			7.89	9.23	85.49	1.08	0.00	8.85
○Thread 57946			8.00	9.23	86.63	1.08	0.00	8.85
○Thread 57947			8.05	9.23	87.17	1.08	0.00	8.85
○Thread 57948			8.07	9.23	87.39	1.08	0.00	8.85
○Thread 57949			8.09	9.23	87.60	1.08	0.00	8.85
○Thread 57950			8.03	9.23	86.96	1.08	0.00	8.85
○Thread 57951			8.07	9.23	87.39	1.08	0.00	8.85
○Thread 57952			8.02	9.23	86.90	1.08	0.00	8.85
○Thread 57953			7.95	9.23	86.15	1.08	0.00	8.85
○Thread 57954			8.08	9.23	87.50	1.08	0.00	8.85
○Thread 57955			8.02	9.23	86.91	1.08	0.00	8.85
○Thread 57956			8.13	9.23	88.10	1.08	0.00	8.85
○Thread 57957			8.08	9.23	87.56	1.08	-0.00	8.86

Report Configuration