	*************************************************
	*                                               *
	*          ONE-View report generation           *
	*                                               *
	*************************************************

[MAQAO] Info: Experiment configuration summary is available adding -dbg=1 in command line

* [MAQAO] Warning: Experiment directory /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/run/oneview_runs/defaults/aocc/oneview_results_1786630414 already exists and is reused.
           It can be replaced using --replace in the command line.
[MAQAO] Info: 
[MAQAO] Info: START THE APPLICATION PROFILING
[MAQAO] Info: -> RUNNING THE PROFILER...
[MAQAO] Info:   LPROF has already been run
[MAQAO] Info: STOP THE APPLICATION PROFILING
[MAQAO] Info: 
[MAQAO] Info: START FUNCTIONS AND LOOPS ANALYSIS ...
[MAQAO] Info: STOP FUNCTIONS AND LOOPS ANALYSIS ...
[MAQAO] Info: 
[MAQAO] Info: START THE REPORT GENERATION
[MAQAO] Info: -> ONE-VIEW EXPERIMENT DIRECTORY: /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/run/oneview_runs/defaults/aocc/oneview_results_1786630414


+====================================================================================================================+
+                                                    1  -  GLOBAL                                                    +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                             1.1  -  Experiment Summary                                             +
+--------------------------------------------------------------------------------------------------------------------+

  Application:			/beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/run/base_runs/defaults/aocc/exec
  Timestamp:			2026-08-13 16:13:34
  Universal Timestamp:		1786630414
  Experiment Type:		MPI; OpenMP; Throughput; 
  Machine:			isix06.benchmarkcenter.megware.com
  Architecture:			x86_64
  Micro Architecture:		GRANITE_RAPIDS
  Model Name:			Intel(R) Xeon(R) 6972P
  Cache Size:			491520 KB
  Number of Cores:		96
  OS Version:			Linux 5.14.0-687.31.1.el9_8.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Aug 1 05:38:01 EDT 2026
  Compilation Options:		
		exec: AMD clang version 17.0.6 (CLANG: AOCC_5.1.0-Build#1994 2025_12_23) /cluster/comp/aocc/5.1.0/bin/clang-17 --driver-mode=g++ -D USE_OMP -I /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/src/omp -I /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/aocc/generated -I /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/driver -g -fno-omit-frame-pointer -fcf-protection=none -nopie -grecord-command-line -D NDEBUG -std=c++17 -O3 -Wall -march=native -fopenmp=libomp -MD -MT CMakeFiles/tealeaf.dir/src/omp/cg.cpp.o -MF CMakeFiles/tealeaf.dir/src/omp/cg.cpp.o.d -o CMakeFiles/tealeaf.dir/src/omp/cg.cpp.o -c /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/src/omp/cg.cpp -I /cluster/hpcx/2.23/ompi5-aocc-mt/include -I /cluster/hpcx/2.23/ompi5-aocc-mt/include/openmpi 
  Number of processes observed:	6
  Number of threads observed:	192
  MAQAO version:		2026.1.0
  MAQAO build:			6d1be1d51c1e63266254997eb301734a7264775d::20260810-150026




+--------------------------------------------------------------------------------------------------------------------+
+                                               1.2  -  Global Metrics                                               +
+--------------------------------------------------------------------------------------------------------------------+

  Total Time:				64.91 s
  Max (Thread Active Time):		64.39 s
  Average Active Time:			64.11 s
  Activity Ratio:			99.9 %
  Average number of active threads:	189.616
  Affinity Stability:			99.9 %
  Time spent in analyzed loops:		83.5 %
  Time spent in analyzed innermost loops: 83.4 %
  Time spent in user code:		83.5 %
  Compilation Options Score:		100
  Array Access Efficiency:		100.0 %

   Potential Speedups
  ----------------------------------------------------
  Perfect Flow Complexity:		1.00
  Perfect OpenMP/MPI/Pthread/TBB:	1.07
  Perfect OpenMP/MPI/Pthread/TBB + Load Distribution:	1.20
  If No Scalar Integer:
      Potential Speedup:		1.00
      Nb Loops to get 80%:		1
  If FP Vectorized:
      Potential Speedup:		1.40
      Nb Loops to get 80%:		2
  If Fully Vectorized:
      Potential Speedup:		3.55
      Nb Loops to get 80%:		2
  If Only FP Arithmetic:
      Potential Speedup:		1.01
      Nb Loops to get 80%:		1




+--------------------------------------------------------------------------------------------------------------------+
+                                             1.3  -  Potential Speedups                                             +
+--------------------------------------------------------------------------------------------------------------------+

  If No Scalar Integer:
      Number of loops   | 1      | 1      | 3      | 4      | 6      | 
      Cumulated Speedup | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 1.0000 | 
  Top 5 loops:
    exec - 94:	1
    exec - 119:	1
    exec - 116:	1
    exec - 28:	1
    exec - 23:	1

  If FP Vectorized:
      Number of loops   | 1      | 1      | 3      | 4      | 6      | 
      Cumulated Speedup | 1.2461 | 1.2461 | 1.3972 | 1.3972 | 1.3972 | 
  Top 5 loops:
    exec - 20:	1.2461
    exec - 23:	1.3908
    exec - 28:	1.3972
    exec - 119:	1.3972
    exec - 116:	1.3972

  If Fully Vectorized:
      Number of loops   | 1      | 1      | 3      | 4      | 6      | 
      Cumulated Speedup | 1.6871 | 1.6871 | 3.5477 | 3.5500 | 3.5533 | 
  Top 5 loops:
    exec - 20:	1.6871
    exec - 23:	3.3722
    exec - 28:	3.5477
    exec - 119:	3.55
    exec - 116:	3.5521

  If Only FP Arithmetic:
      Number of loops   | 1      | 1      | 3      | 4      | 6      | 
      Cumulated Speedup | 1.0099 | 1.0099 | 1.0099 | 1.0099 | 1.0099 | 
  Top 5 loops:
    exec - 28:	1.0099
    exec - 94:	1.0099
    exec - 116:	1.0099
    exec - 23:	1.0099
    exec - 119:	1.0099



+====================================================================================================================+
+                                                   2  -  SUMMARY                                                    +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                             2.1  -  EXPERIMENT QUALITY                                             +
+--------------------------------------------------------------------------------------------------------------------+

  [4 / 4] Application profile is long enough (64.39 s)
To have good quality measurements, it is advised that the application profiling time is greater than 10 seconds.

  [3 / 3] Most of time spent in analyzed modules comes from functions with source/debug info
-g option gives access to debugging informations, such are source locations.

  [3 / 3] Most of time spent in analyzed modules comes from functions with compilation options informations and
-fno-omit-frame-pointer is present
-fno-omit-frame-pointer improves the accuracy of callchains found during the application profiling.

  [3 / 3] Host configuration allows retrieval of all necessary metrics.


  [2.9982533545527 / 3] Most of time spent in analyzed modules (99.94%) comes from functions compiled with architecture specialization option
-march=native


  [3 / 3] Optimization level option is correctly used


  [2 / 2] Application is correctly profiled ("Others" category represents 0.02 % of the execution time)
To have a representative profiling, it is advised that the category "Others" represents less than 20% of the execution
time in order to analyze as much as possible of the user code

  [1 / 1] Lstopo present. The Topology lstopo report will be generated.


  [0 / 0] Fastmath not used
Consider to add ffast-math to compilation flags (or replace -O3 with -Ofast) to unlock potential extra speedup by
relaxing floating-point computation consistency. Warning: floating-point accuracy may be reduced and the compliance
to IEEE/ISO rules/specifications for math functions will be relaxed, typically 'errno' will no longer be set after
calling some math functions.


+--------------------------------------------------------------------------------------------------------------------+
+                                                2.2  -  CODE QUALITY                                                +
+--------------------------------------------------------------------------------------------------------------------+

  [4 / 4] Enough time of the experiment time spent in analyzed loops (83.47%)
If the time spent in analyzed loops is less than 30%, standard loop optimizations will have a limited impact on
application performances.

  [4 / 4] Threads activity is good
On average, more than 98.76% of observed threads are actually active 

  [4 / 4] CPU activity is good
CPU cores are active 99.92% of time

  [4 / 4] Loop profile is not flat
At least one loop coverage is greater than 4% (46.55%), representing an hotspot for the application

  [4 / 4] Enough time of the experiment time spent in analyzed innermost loops (83.45%)
If the time spent in analyzed innermost loops is less than 15%, standard innermost loop optimizations such as
vectorisation will have a limited impact on application performances.

  [4 / 4] Affinity is good (99.91%)
Threads are not migrating to CPU cores: probably successfully pinned

  [3 / 3] Less than 10% (0.00%) is spend in BLAS1 operations
It could be more efficient to inline by hand BLAS1 operations

  [3 / 3] Functions mostly use all threads
Functions running on a reduced number of threads (typically sequential code) cover less than 10% of application
walltime (7.94%)

  [3 / 3] Cumulative Outermost/In between loops coverage (0.02%) lower than cumulative innermost loop coverage (83.45%)
Having cumulative Outermost/In between loops coverage greater than cumulative innermost loop coverage will make loop
optimization more complex

  [2 / 2] Less than 10% (0.00%) is spend in BLAS2 operations
BLAS2 calls usually could make a poor cache usage and could benefit from inlining.

  [2 / 2] Less than 10% (0.00%) is spend in Libm/SVML (special functions)



+--------------------------------------------------------------------------------------------------------------------+
+                                               2.3  -  LOOPS OVERVIEW                                               +
+--------------------------------------------------------------------------------------------------------------------+

  Top 5 loops:
   + exec - 20  :
     analysis: Execution Time: 46 % - Vectorization Ratio: 0.00 % - Vector Length Use: 12.50 %

   + exec - 23  :
     analysis: Execution Time: 33 % - Vectorization Ratio: 0.00 % - Vector Length Use: 12.50 %

   + exec - 28  :
     analysis: Execution Time: 2 % - Vectorization Ratio: 100.00 % - Vector Length Use: 50.00 %
     Data Access Issues: 0
        [0] [SA] Inefficient vectorization: more than 10% of the vector loads instructions are unaligned - When
            allocating arrays, don’t forget to align them. There are 0 issues ( = arrays) costing 2 points each

   + exec - 119 :
     analysis: Execution Time: 0 % - Vectorization Ratio: 0.00 % - Vector Length Use: 12.50 %
     Loop Computation Issues: 5
        [5] [SA] Peel/tail loop, considered having a low iteration count - Perform full unroll. Force compiler to use
            masked instructions. This issue costs 5 points.
     Control Flow Issues: 5
        [5] [SA] Peel/tail loop, considered having a low iteration count - Perform full unroll. Force compiler to use
            masked instructions. This issue costs 5 points.
     Data Access Issues: 4
        [4] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 1 issues ( = indirect data accesses) costing 4 point each.
     Vectorization Roadblocks: 4
        [4] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 1 issues ( = indirect data accesses) costing 4 point each.

   + exec - 116 :
     analysis: Execution Time: 0 % - Vectorization Ratio: 0.00 % - Vector Length Use: 12.50 %
     Loop Computation Issues: 5
        [5] [SA] Peel/tail loop, considered having a low iteration count - Perform full unroll. Force compiler to use
            masked instructions. This issue costs 5 points.
     Control Flow Issues: 5
        [5] [SA] Peel/tail loop, considered having a low iteration count - Perform full unroll. Force compiler to use
            masked instructions. This issue costs 5 points.
     Data Access Issues: 4
        [4] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 1 issues ( = indirect data accesses) costing 4 point each.
     Vectorization Roadblocks: 4
        [4] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 1 issues ( = indirect data accesses) costing 4 point each.



+====================================================================================================================+
+                                                 3  -  APPLICATION                                                  +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                               3.1  -  Categorization                                               +
+--------------------------------------------------------------------------------------------------------------------+

   Category | IO     | Exe    | System  | Others  | Memory | String | MPI   | TBB   | OMP   | Pthread | Math  |
  ----------+--------+--------+---------+---------+--------+--------+-------+-------+-------+---------+-------+
   Time (%) | 0.00   | 83.53  | 0.44    | 0.02    | 0.00   | 0.02   | 0.04  | 0.00  | 15.94 | 0.00    | 0.00  |




+--------------------------------------------------------------------------------------------------------------------+
+                                          3.2  -  Function Based Profiling                                          +
+--------------------------------------------------------------------------------------------------------------------+

   Buckets                    | Nb Functions              | Coverage                  | Cumulated Coverage        |
  ----------------------------+---------------------------+---------------------------+---------------------------+
   > 8%                       | 3                         | 91.33                     | 91.33                     |
   4% to 8%                   | 0                         | 0.00                      | 91.33                     |
   2% to 4%                   | 2                         | 5.82                      | 97.15                     |
   1% to 2%                   | 1                         | 1.96                      | 99.11                     |
   0.5% to 1%                 | 0                         | 0.00                      | 99.11                     |
   0.25% to 0.5%              | 1                         | 0.38                      | 99.49                     |
   0.125% to 0.25%            | 0                         | 0.00                      | 99.49                     |
   < 0.125%                   | 104                       | 0.45                      | 99.94                     |




+--------------------------------------------------------------------------------------------------------------------+
+                                            3.3  -  Loop Based Profiling                                            +
+--------------------------------------------------------------------------------------------------------------------+

   Buckets                    | Nb Loops                  | Coverage                  | Cumulated Coverage        |
  ----------------------------+---------------------------+---------------------------+---------------------------+
   > 8%                       | 2                         | 80.39                     | 80.39                     |
   4% to 8%                   | 0                         | 0.00                      | 80.39                     |
   2% to 4%                   | 1                         | 2.93                      | 83.33                     |
   1% to 2%                   | 0                         | 0.00                      | 83.33                     |
   0.5% to 1%                 | 0                         | 0.00                      | 83.33                     |
   0.25% to 0.5%              | 0                         | 0.00                      | 83.33                     |
   0.125% to 0.25%            | 0                         | 0.00                      | 83.33                     |
   < 0.125%                   | 25                        | 0.12                      | 83.45                     |


+====================================================================================================================+
+                                                  4  -  FUNCTIONS                                                   +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                              4.1  -  Top 10 Functions                                              +
+--------------------------------------------------------------------------------------------------------------------+

   Function                                               | Module              | Coverage (%)   | Time (s)       |
  --------------------------------------------------------+---------------------+----------------+----------------+
   cg_calc_w(int, int, int, double*, double const*, do... | exec                | 46.55          | 29.84          |
   cg_calc_ur(int, int, int, double, double*, double*,... | exec                | 33.85          | 21.70          |
   __kmp_hyper_barrier_release(barrier_type, kmp_info*... | libomp.so           | 10.92          | 7.00           |
   cg_calc_p(int, int, int, double, double*, double co... | exec                | 2.95           | 1.89           |
   __kmp_hardware_timestamp                               | libomp.so           | 2.87           | 1.84           |
   __kmp_hyper_barrier_gather(barrier_type, kmp_info*,... | libomp.so           | 1.96           | 1.26           |
   unknown_kernel_region                                  | kernel              | 0.38           | 0.24           |
   __GI___sched_yield                                     | libc.so.6           | 0.06           | 0.07           |
   __kmp_fork_barrier(int, int)                           | libomp.so           | 0.04           | 0.03           |
   __kmp_init_implicit_task                               | libomp.so           | 0.03           | 0.02           |


+====================================================================================================================+
+                                                    5  -  LOOPS                                                     +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                                5.1  -  Top 10 Loops                                                +
+--------------------------------------------------------------------------------------------------------------------+

   Loop Id        | Module              | Source Location                                        | Coverage (%)   |
  ----------------+---------------------+--------------------------------------------------------+----------------+
   20             | exec                | cg.cpp:86-90                                           | 46.55          |
   23             | exec                | cg.cpp:108-113                                         | 33.85          |
   28             | exec                | cg.cpp:128-131                                         | 2.93           |
   119            | exec                | pack_halos.cpp:103-105                                 | 0.02           |
   116            | exec                | pack_halos.cpp:86-88                                   | 0.02           |
   94             | exec                | local_halos.cpp:46-46                                  | 0.02           |
   16             | exec                | cg.cpp:62-68                                           | 0.01           |
   106            | exec                | pack_halos.cpp:33-35                                   | 0.01           |
   101            | exec                | pack_halos.cpp:15-17                                   | 0.01           |
   25             | exec                | cg.cpp:127-128,cg.cpp:131-131                          | 0.01           |





+====================================================================================================================+
+                                                     6  -  CQA                                                      +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                                   6.1  -  Loops                                                    +
+--------------------------------------------------------------------------------------------------------------------+





      6.1.1  -  Loop 20 from exec
  ==========================================================================================================

The loop is defined in /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/src/omp/cg.cpp:86-90.

The related source loop is not unrolled or unrolled with no peel/tail loop.

      6.1.1.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

8% of peak computational performance is used (2.73 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.1.1.1  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
8 data elements could be processed at once in vector registers.
<<image_1x64_512>>By vectorizing your loop, you can lower the cost of an iteration from 11.00 to 1.37 cycles (8.00x speedup).

Details
All SSE/AVX instructions are used in scalar version (process only one data element in vector registers).
Since your execution units are vector units, only a vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.1.1.2  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 11.00 to 8.00 cycles (1.38x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.1.1.3  -  FMA
  ----------------------------------------------------------------------------------------------------------

Detected 8 FMA (fused multiply-add) operations.
Presence of both ADD/SUB and MUL operations.

Workaround
Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
Estimated speedup by perfect pairing: 1.22x.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).




      6.1.1.1.4  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

22 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.1.1.5  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 30 FP arithmetical operations:
 - 18: addition or subtraction (8 inside FMA instructions)
 - 12: multiply (8 inside FMA instructions)
The binary loop is loading 152 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.1.1.6  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.18 FP operations per loaded or stored byte.


      6.1.1.1.7  -  Unroll opportunity
  ----------------------------------------------------------------------------------------------------------

Loop is potentially data access bound.

Workaround
Unroll your loop if trip count is significantly higher than target unroll factor and if some data references are common to consecutive iterations. This can be done manually. Or with the unroll (resp. unroll_and_jam) directive on top of the inner (resp. surrounding) loop. You can enforce an unroll factor: #pragma unroll_and_jam N, unroll_and_jam(N), unroll N or unroll(N)







      6.1.2  -  Loop 23 from exec
  ==========================================================================================================

The loop is defined in /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/src/omp/cg.cpp:108-113.

It is main loop of related source loop which is unrolled by 4 (including vectorization).

      6.1.2.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

4% of peak computational performance is used (1.50 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.2.1.1  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
8 data elements could be processed at once in vector registers.
<<image_1x64_512>>By vectorizing your loop, you can lower the cost of an iteration from 16.00 to 2.00 cycles (8.00x speedup).

Details
All SSE/AVX instructions are used in scalar version (process only one data element in vector registers).
Since your execution units are vector units, only a vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.2.1.2  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.2.1.3  -  FMA
  ----------------------------------------------------------------------------------------------------------

Detected 12 FMA (fused multiply-add) operations.




      6.1.2.1.4  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

12 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.2.1.5  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 24 FP arithmetical operations:
 - 12: addition or subtraction (all inside FMA instructions)
 - 12: multiply (all inside FMA instructions)
The binary loop is loading 192 bytes (24 double precision FP elements).
The binary loop is storing 64 bytes (8 double precision FP elements).


      6.1.2.1.6  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.09 FP operations per loaded or stored byte.







      6.1.3  -  Loop 28 from exec
  ==========================================================================================================

The loop is defined in /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/src/omp/cg.cpp:128-131.

It is main loop of related source loop which is unrolled by 16 (including vectorization).

      6.1.3.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

33% of peak computational performance is used (10.67 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.3.1.1  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is vectorized, but using only 256 out of 512 bits (AVX/AVX2 instructions on AVX-512 processors).
<<image_4x64_512>>

Details
All SSE/AVX instructions are used in vector version (process two or more data elements in vector registers).


Workaround
Read the "512-bits vectorization" report at "Potential" confidence level.


      6.1.3.1.2  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.3.1.3  -  512-bits vectorization
  ----------------------------------------------------------------------------------------------------------

On some x86 processors supporting 512-bits vectorization, compilers are often too conservative and limit vectorization to 256 bits. Performance can then be improved by enforcing 512-bits vectorization, especially with many vectorized and high trip count loops. 512-bits vectorization performance overhead (compared to 256-bits) is generally lower on newer processors.


Workaround
Recompile with -mprefer-vector-width=512


      6.1.3.1.4  -  FMA
  ----------------------------------------------------------------------------------------------------------

Detected 16 FMA (fused multiply-add) operations.




      6.1.3.1.5  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 8 optimal vector unaligned load/store instructions.


Details
 - VMOVUPD: 8 occurrences<<list_path_1_vec_align_1>>


Workaround
Use vector aligned instructions:
 1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
 2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.3.1.6  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (four at a time).



      6.1.3.1.7  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 32 FP arithmetical operations:
 - 16: addition or subtraction (all inside FMA instructions)
 - 16: multiply (all inside FMA instructions)
The binary loop is loading 256 bytes (32 double precision FP elements).
The binary loop is storing 128 bytes (16 double precision FP elements).


      6.1.3.1.8  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.08 FP operations per loaded or stored byte.







      6.1.4  -  Loop 119 from exec
  ==========================================================================================================

The loop is defined in /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/src/omp/pack_halos.cpp:103-105.

It is peel/tail loop of related source loop which is unrolled by 16 (including vectorization).

      6.1.4.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

0% of peak computational performance is used (0.00 out of 64.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.4.1.1  -  Unrolling/vectorization cost
  ----------------------------------------------------------------------------------------------------------

This loop is peel/tail of a unrolled/vectorized loop. If its cost is not negligible compared to the main (unrolled/vectorized) loop, unrolling/vectorization is counterproductive due to low trip count.

Details
The more iterations the main loop is processing, the higher the trip count must be to amortize peel/tail overhead.

Workaround
 - recompile with -fprofile-instr-generate, execute ,merge raw profiles with 'llvm-profdata merge -o default.profdata default.profraw' and recompile with -fprofile-instr-use (profile-guided optimization)
 - hardcode most frequent values of loop bounds by adding specialized paths.:
  *  For instance, replace for (i=0; i<n; i++) foo(i) with:
switch (n) {
  case (4): for (i=0; i<4; i++) foo(i); break;
  case (6): for (i=0; i<6; i++) foo(i); break;
  default : for (i=0; i<n; i++) foo(i); break;
}



      6.1.4.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
8 data elements could be processed at once in vector registers.


Details
All SSE/AVX instructions are used in scalar version (process only one data element in vector registers).



      6.1.4.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.



No data for this section



      6.1.4.1.4  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 1 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.4.1.5  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

No instructions are processing arithmetic or math operations on FP elements. This loop is probably writing/copying data or processing integer elements.


      6.1.4.1.6  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop does not contain any FP arithmetical operations.
The binary loop is loading 8 bytes.
The binary loop is storing 8 bytes.







      6.1.5  -  Loop 116 from exec
  ==========================================================================================================

The loop is defined in /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/src/omp/pack_halos.cpp:86-88.

It is peel/tail loop of related source loop which is unrolled by 16 (including vectorization).

      6.1.5.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

0% of peak computational performance is used (0.00 out of 64.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.5.1.1  -  Unrolling/vectorization cost
  ----------------------------------------------------------------------------------------------------------

This loop is peel/tail of a unrolled/vectorized loop. If its cost is not negligible compared to the main (unrolled/vectorized) loop, unrolling/vectorization is counterproductive due to low trip count.

Details
The more iterations the main loop is processing, the higher the trip count must be to amortize peel/tail overhead.

Workaround
 - recompile with -fprofile-instr-generate, execute ,merge raw profiles with 'llvm-profdata merge -o default.profdata default.profraw' and recompile with -fprofile-instr-use (profile-guided optimization)
 - hardcode most frequent values of loop bounds by adding specialized paths.:
  *  For instance, replace for (i=0; i<n; i++) foo(i) with:
switch (n) {
  case (4): for (i=0; i<4; i++) foo(i); break;
  case (6): for (i=0; i<6; i++) foo(i); break;
  default : for (i=0; i<n; i++) foo(i); break;
}



      6.1.5.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
8 data elements could be processed at once in vector registers.


Details
All SSE/AVX instructions are used in scalar version (process only one data element in vector registers).



      6.1.5.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.



No data for this section



      6.1.5.1.4  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 1 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.5.1.5  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

No instructions are processing arithmetic or math operations on FP elements. This loop is probably writing/copying data or processing integer elements.


      6.1.5.1.6  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop does not contain any FP arithmetical operations.
The binary loop is loading 8 bytes.
The binary loop is storing 8 bytes.







      6.1.6  -  Loop 94 from exec
  ==========================================================================================================

The loop is defined in /beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/build/TeaLeaf/src/omp/local_halos.cpp:46.

It is main loop of related source loop which is unrolled by 16 (including vectorization).

      6.1.6.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

0% of peak computational performance is used (0.00 out of 64.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.6.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 2.17 to 2.00 cycles (1.08x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.6.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is vectorized, but using only 256 out of 512 bits (AVX/AVX2 instructions on AVX-512 processors).


Details
All SSE/AVX instructions are used in vector version (process two or more data elements in vector registers).


Workaround
Read the "512-bits vectorization" report at "Potential" confidence level.


      6.1.6.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.6.1.4  -  512-bits vectorization
  ----------------------------------------------------------------------------------------------------------

On some x86 processors supporting 512-bits vectorization, compilers are often too conservative and limit vectorization to 256 bits. Performance can then be improved by enforcing 512-bits vectorization, especially with many vectorized and high trip count loops. 512-bits vectorization performance overhead (compared to 256-bits) is generally lower on newer processors.


Workaround
Recompile with -mprefer-vector-width=512




      6.1.6.1.5  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 1 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.6.1.6  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 8 optimal vector unaligned load/store instructions.


Details
 - VMOVUPS: 8 occurrences<<list_path_1_vec_align_1>>


Workaround
Use vector aligned instructions:
 1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
 2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.6.1.7  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

No instructions are processing arithmetic or math operations on FP elements. This loop is probably writing/copying data or processing integer elements.


      6.1.6.1.8  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop does not contain any FP arithmetical operations.
The binary loop is loading 128 bytes.
The binary loop is storing 128 bytes.





[MAQAO] Info: STOP THE REPORT GENERATION
[MAQAO] Info: 
[MAQAO] Info: If your application produces files, they can be found in directory "/beegfs/hackathon/users/eoseret/qaas_runs_test/178-663-0009/intel/TeaLeaf/run/oneview_runs/defaults/aocc/oneview_run_1786630414"
[MAQAO] Info: 
