OV - - Function _ZN16miniqmcreference17OneBodyJastrowRefIN11qmcplusplus14BsplineFunctorIdEEE8evalGradERNS1

/beegfs/hackathon/users/eoseret/qaas_runs/170-855-3059/intel/miniqmc/build/miniqmc/src/QMCWaveFunctions/Jastrow/OneBodyJastrowRef.h: 247 - 247

--------------------------------------------------------------------------------

247:   GradType evalGrad(ParticleSet& P, int iat) { return GradType(Grad[iat]); }

0x420240 MOV	0x190(%RSI),%RSI

0x420247 MOVSXD	%ECX,%RCX

0x42024a MOV	%RDI,%RAX

0x42024d LEA	(%RCX,%RCX,2),%RDX

0x420251 LEA	(%RSI,%RDX,8),%RDI

0x420255 VMOVDQU	(%RDI),%XMM0

0x420259 MOV	0x10(%RDI),%R8

0x42025d MOV	%R8,0x10(%RAX)

0x420261 VMOVDQU	%XMM0,(%RAX)

0x420265 RET

0x420266 NOPW	%CS:(%RAX,%RAX,1)

Path /

Average path: Display a virtual path defined by average values of all real paths

Source file and lines	OneBodyJastrowRef.h:247-247
Module	exec

The function is defined in /beegfs/hackathon/users/eoseret/qaas_runs/170-855-3059/intel/miniqmc/build/miniqmc/src/QMCWaveFunctions/Jastrow/OneBodyJastrowRef.h:247.

gain
potential
hint
expert

Code clean check

Detected a slowdown caused by scalar integer instructions (typically used for address computation). By removing them, you can lower the cost of an iteration from 1.67 to 0.67 cycles (2.50x speedup).

Workaround

Try to reorganize arrays of structures to structures of arrays
Consider to permute loops (see vectorization gain report)

Vectorization

Your function is poorly vectorized. Only 16% of vector register length is used (average across all SSE/AVX instructions). By fully vectorizing your function, you can lower the cost of an iteration from 1.67 to 0.46 cycles (3.64x speedup).

Details

28% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):

33% of SSE/AVX loads are used in vector version.
50% of SSE/AVX stores are used in vector version.
0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.

Since your execution units are vector units, only a fully vectorized function can use their full power.

Workaround

Try another compiler or update/tune your current one
Make array accesses unit-stride:
- If your function streams arrays of structures (AoS), try to use structures of arrays instead (SoA): for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)

Execution units bottlenecks

Found no such bottlenecks but see expert reports for more complex bottlenecks.

No data for this section

Type of elements and instruction set

No instructions are processing arithmetic or math operations on FP elements. This function is probably writing/copying data or processing integer elements.

Matching between your function (in the source code) and the binary function

The binary function does not contain any FP arithmetical operations. The binary function is loading 32 bytes. The binary function is storing 24 bytes.

General properties

nb instructions	10
nb uops	10
loop length	38
used x86 registers	6
used mmx registers	0
used xmm registers	1
used ymm registers	0
used zmm registers	0
nb stack references	0

Front-end

MACRO FUSION NOT POSSIBLE FIT IN UOP CACHE

micro-operation queue	1.67 cycles
front end	1.67 cycles

Back-end

	ALU0/BRU0	ALU1	ALU2	ALU3	BRU1	AGU0	AGU1	AGU2	FP0	FP1	FP2	FP3	FP4	FP5
uops	1.00	1.00	0.75	0.75	0.50	1.67	1.67	1.67	0.00	0.00	0.00	0.00	0.50	0.50
cycles	1.00	1.00	0.75	0.75	0.50	1.67	1.67	1.67	0.00	0.00	0.00	0.00	0.50	0.50

Execution ports to units layout:

ALU0/BRU0: ALU
ALU1: ALU
ALU2: ALU
ALU3: ALU
BRU1:
AGU0 (256 bits): store address, load
AGU1 (256 bits): store address, load
AGU2 (256 bits): store address, load
FP0 (256 bits): VPU, DIV/SQRT
FP1 (256 bits): VPU, DIV/SQRT
FP2 (256 bits): VPU
FP3 (256 bits): VPU
FP4 (256 bits): FP store data
FP5 (256 bits): FP store data

Cycles executing div or sqrt instructions

NA

Cycles summary

Front-end	1.67
Dispatch	1.67
Overall L1	1.67

Vectorization ratios

all	28%
load	33%
store	50%
mul	NA (no mul vectorizable/vectorized instructions)
add-sub	NA (no add-sub vectorizable/vectorized instructions)
fma	NA (no fma vectorizable/vectorized instructions)
div/sqrt	NA (no div/sqrt vectorizable/vectorized instructions)
other	0%

Vector efficiency ratios

all	16%
load	16%
store	18%
mul	NA (no mul vectorizable/vectorized instructions)
add-sub	NA (no add-sub vectorizable/vectorized instructions)
fma	NA (no fma vectorizable/vectorized instructions)
div/sqrt	NA (no div/sqrt vectorizable/vectorized instructions)
other	12%

Cycles and memory resources usage

Assuming all data fit into the L1 cache, each call to the function takes 1.67 cycles. At this rate:

20% of peak load performance is reached (19.20 out of 96.00 bytes loaded per cycle (GB/s @ 1GHz))
22% of peak store performance is reached (14.40 out of 64.00 bytes stored per cycle (GB/s @ 1GHz))

Front-end bottlenecks

Performance is limited by instruction throughput (loading/decoding program instructions to execution core) (front-end is a bottleneck). By removing all these bottlenecks, you can lower the cost of an iteration from 1.67 to 1.00 cycles (1.67x speedup).

ASM code

In the binary file, the address of the function is: 420240

Instruction	Nb FU	ALU0/BRU0	ALU1	ALU2	ALU3	BRU1	AGU0	AGU1	AGU2	FP4	FP5	Latency	Recip. throughput
MOV 0x190(%RSI),%RSI	1	0	0	0	0	0	0.33	0.33	0.33	0	0	3	0.33
MOVSXD %ECX,%RCX	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	1	0.25
MOV %RDI,%RAX	1	0	0	0	0	0	0	0	0	0	0	0	0.17
LEA (%RCX,%RCX,2),%RDX	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	1	0.25
LEA (%RSI,%RDX,8),%RDI	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	1	0.25
VMOVDQU (%RDI),%XMM0	1	0	0	0	0	0	0.33	0.33	0.33	0	0	3	0.50
MOV 0x10(%RDI),%R8	1	0	0	0	0	0	0.33	0.33	0.33	0	0	3	0.33
MOV %R8,0x10(%RAX)	1	0	0	0	0	0	0.33	0.33	0.33	0	0	4	0.50
VMOVDQU %XMM0,(%RAX)	1	0	0	0	0	0	0.33	0.33	0.33	0.50	0.50	4	1
RET	1	0.50	0	0	0	0.50	0	0	0	0	0	1	0.50

Source file and lines	OneBodyJastrowRef.h:247-247
Module	exec

The function is defined in /beegfs/hackathon/users/eoseret/qaas_runs/170-855-3059/intel/miniqmc/build/miniqmc/src/QMCWaveFunctions/Jastrow/OneBodyJastrowRef.h:247.

gain
potential
hint
expert

Code clean check

Detected a slowdown caused by scalar integer instructions (typically used for address computation). By removing them, you can lower the cost of an iteration from 1.67 to 0.67 cycles (2.50x speedup).

Workaround

Try to reorganize arrays of structures to structures of arrays
Consider to permute loops (see vectorization gain report)

Vectorization

Your function is poorly vectorized. Only 16% of vector register length is used (average across all SSE/AVX instructions). By fully vectorizing your function, you can lower the cost of an iteration from 1.67 to 0.46 cycles (3.64x speedup).

Details

28% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):

33% of SSE/AVX loads are used in vector version.
50% of SSE/AVX stores are used in vector version.
0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.

Since your execution units are vector units, only a fully vectorized function can use their full power.

Workaround

Try another compiler or update/tune your current one
Make array accesses unit-stride:
- If your function streams arrays of structures (AoS), try to use structures of arrays instead (SoA): for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)

Execution units bottlenecks

Found no such bottlenecks but see expert reports for more complex bottlenecks.

No data for this section

Type of elements and instruction set

No instructions are processing arithmetic or math operations on FP elements. This function is probably writing/copying data or processing integer elements.

Matching between your function (in the source code) and the binary function

The binary function does not contain any FP arithmetical operations. The binary function is loading 32 bytes. The binary function is storing 24 bytes.

General properties

nb instructions	10
nb uops	10
loop length	38
used x86 registers	6
used mmx registers	0
used xmm registers	1
used ymm registers	0
used zmm registers	0
nb stack references	0

Front-end

MACRO FUSION NOT POSSIBLE FIT IN UOP CACHE

micro-operation queue	1.67 cycles
front end	1.67 cycles

Back-end

	ALU0/BRU0	ALU1	ALU2	ALU3	BRU1	AGU0	AGU1	AGU2	FP0	FP1	FP2	FP3	FP4	FP5
uops	1.00	1.00	0.75	0.75	0.50	1.67	1.67	1.67	0.00	0.00	0.00	0.00	0.50	0.50
cycles	1.00	1.00	0.75	0.75	0.50	1.67	1.67	1.67	0.00	0.00	0.00	0.00	0.50	0.50

Execution ports to units layout:

ALU0/BRU0: ALU
ALU1: ALU
ALU2: ALU
ALU3: ALU
BRU1:
AGU0 (256 bits): store address, load
AGU1 (256 bits): store address, load
AGU2 (256 bits): store address, load
FP0 (256 bits): VPU, DIV/SQRT
FP1 (256 bits): VPU, DIV/SQRT
FP2 (256 bits): VPU
FP3 (256 bits): VPU
FP4 (256 bits): FP store data
FP5 (256 bits): FP store data

Cycles executing div or sqrt instructions

NA

Cycles summary

Front-end	1.67
Dispatch	1.67
Overall L1	1.67

Vectorization ratios

all	28%
load	33%
store	50%
mul	NA (no mul vectorizable/vectorized instructions)
add-sub	NA (no add-sub vectorizable/vectorized instructions)
fma	NA (no fma vectorizable/vectorized instructions)
div/sqrt	NA (no div/sqrt vectorizable/vectorized instructions)
other	0%

Vector efficiency ratios

all	16%
load	16%
store	18%
mul	NA (no mul vectorizable/vectorized instructions)
add-sub	NA (no add-sub vectorizable/vectorized instructions)
fma	NA (no fma vectorizable/vectorized instructions)
div/sqrt	NA (no div/sqrt vectorizable/vectorized instructions)
other	12%

Cycles and memory resources usage

Assuming all data fit into the L1 cache, each call to the function takes 1.67 cycles. At this rate:

20% of peak load performance is reached (19.20 out of 96.00 bytes loaded per cycle (GB/s @ 1GHz))
22% of peak store performance is reached (14.40 out of 64.00 bytes stored per cycle (GB/s @ 1GHz))

Front-end bottlenecks

Performance is limited by instruction throughput (loading/decoding program instructions to execution core) (front-end is a bottleneck). By removing all these bottlenecks, you can lower the cost of an iteration from 1.67 to 1.00 cycles (1.67x speedup).

ASM code

In the binary file, the address of the function is: 420240

Instruction	Nb FU	ALU0/BRU0	ALU1	ALU2	ALU3	BRU1	AGU0	AGU1	AGU2	FP4	FP5	Latency	Recip. throughput
MOV 0x190(%RSI),%RSI	1	0	0	0	0	0	0.33	0.33	0.33	0	0	3	0.33
MOVSXD %ECX,%RCX	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	1	0.25
MOV %RDI,%RAX	1	0	0	0	0	0	0	0	0	0	0	0	0.17
LEA (%RCX,%RCX,2),%RDX	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	1	0.25
LEA (%RSI,%RDX,8),%RDI	1	0.25	0.25	0.25	0.25	0	0	0	0	0	0	1	0.25
VMOVDQU (%RDI),%XMM0	1	0	0	0	0	0	0.33	0.33	0.33	0	0	3	0.50
MOV 0x10(%RDI),%R8	1	0	0	0	0	0	0.33	0.33	0.33	0	0	3	0.33
MOV %R8,0x10(%RAX)	1	0	0	0	0	0	0.33	0.33	0.33	0	0	4	0.50
VMOVDQU %XMM0,(%RAX)	1	0	0	0	0	0	0.33	0.33	0.33	0.50	0.50	4	1
RET	1	0.50	0	0	0	0.50	0	0	0	0	0	1	0.50

▶Scalability Runs Description

Run 2x1	Number processes: 2Number nodes: 1Number processes per node: 2Run Command: <executable> -g "4 2 2" -bMPI Command: mpirun -np <number_processes>Dataset: Run Directory: /beegfs/hackathon/users/eoseret/qaas_runs/170-855-3059/intel/miniqmc/run/oneview_runs/compilers/gcc_2/oneview_run_1708580749OMP_NUM_THREADS: 1
Run 2x2	OMP_NUM_THREADS: 2
Run 2x4	OMP_NUM_THREADS: 4
Run 2x8	OMP_NUM_THREADS: 8
Run 2x16	OMP_NUM_THREADS: 16
Run 2x32	OMP_NUM_THREADS: 32
Run 2x64	OMP_NUM_THREADS: 64
Run 2x96	OMP_NUM_THREADS: 96

(2x1) Efficiency	(2x1) Potential Speed-Up (%)	(2x2) Efficiency	(2x2) Potential Speed-Up (%)	(2x4) Efficiency	(2x4) Potential Speed-Up (%)	(2x8) Efficiency	(2x8) Potential Speed-Up (%)	(2x16) Efficiency	(2x16) Potential Speed-Up (%)	(2x32) Efficiency	(2x32) Potential Speed-Up (%)	(2x64) Efficiency	(2x64) Potential Speed-Up (%)	(2x96) Efficiency	(2x96) Potential Speed-Up (%)
		1	0	1	0	1	0	1	0	1	0	1	0	1	0

Speed-Up and Efficiency

▶Detailed Speed-Up and Efficiency

Run	Number of threads	Efficiency (ideal is 1)	Speedup	Ideal Speedup	Time (s)	Coverage (%)
2x1
2x2	3	1	1	1.5	0	0.01
2x4	3	1	1	2.25	0	0
2x8	6	1	1	4	0	0
2x16	16	1	1	8.25	0	0
2x32	43	1	1	16.5	0.01	0
2x64	110	1	1	32.5	0.03	0
2x96	186	1	1	48.5	0.04	0

Name	Coverage (%)	Time (s)
○_ZN16miniqmcreference17OneBodyJastrowRefIN11qmcplusplus14BsplineFunctorIdEEE8evalGradERNS1_11ParticleSetEi	0	0

Report Configuration

Code clean check

Workaround

Vectorization

Details

Workaround

Execution units bottlenecks

Type of elements and instruction set

Matching between your function (in the source code) and the binary function

General properties

Front-end

Back-end

Cycles summary

Vectorization ratios

Vector efficiency ratios

Cycles and memory resources usage

Front-end bottlenecks

ASM code

Code clean check

Workaround

Vectorization

Details

Workaround

Execution units bottlenecks

Type of elements and instruction set

Matching between your function (in the source code) and the binary function

General properties

Front-end

Back-end

Cycles summary

Vectorization ratios

Vector efficiency ratios

Cycles and memory resources usage

Front-end bottlenecks

ASM code

▶Scalability Runs Description

▼Colums Filter

Speed-Up and Efficiency

▶Detailed Speed-Up and Efficiency