	*************************************************
	*                                               *
	*          ONE-View report generation           *
	*                                               *
	*************************************************

[MAQAO] Info: Experiment configuration summary is available adding -dbg=1 in command line

* [MAQAO] Warning: Experiment directory /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/run/oneview_runs/defaults/aocc/oneview_results_1786614528 already exists and is reused.
           It can be replaced using --replace in the command line.
[MAQAO] Info: 
[MAQAO] Info: START THE APPLICATION PROFILING
[MAQAO] Info: -> RUNNING THE PROFILER...
[MAQAO] Info:   LPROF has already been run
[MAQAO] Info: STOP THE APPLICATION PROFILING
[MAQAO] Info: 
[MAQAO] Info: START FUNCTIONS AND LOOPS ANALYSIS ...
[MAQAO] Info: STOP FUNCTIONS AND LOOPS ANALYSIS ...
[MAQAO] Info: 
[MAQAO] Info: START THE REPORT GENERATION
[MAQAO] Info: -> ONE-VIEW EXPERIMENT DIRECTORY: /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/run/oneview_runs/defaults/aocc/oneview_results_1786614528


+====================================================================================================================+
+                                                    1  -  GLOBAL                                                    +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                             1.1  -  Experiment Summary                                             +
+--------------------------------------------------------------------------------------------------------------------+

  Application:			/beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/run/base_runs/defaults/aocc/exec
  Timestamp:			2026-08-13 11:48:48
  Universal Timestamp:		1786614528
  Experiment Type:		MPI; OpenMP; Throughput; 
  Machine:			isix07.benchmarkcenter.megware.com
  Architecture:			x86_64
  Micro Architecture:		GRANITE_RAPIDS
  Model Name:			Intel(R) Xeon(R) 6972P
  Cache Size:			491520 KB
  Number of Cores:		96
  OS Version:			Linux 5.14.0-687.31.1.el9_8.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Aug 1 05:38:01 EDT 2026
  Compilation Options:		
		exec: AMD clang version 17.0.6 (CLANG: AOCC_5.1.0-Build#1994 2025_12_23) /cluster/comp/aocc/5.1.0/bin/clang-17 --driver-mode=g++ -D USE_OMP -I /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/omp -I /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/aocc/generated -I /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/driver -I /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp -g -fno-omit-frame-pointer -fcf-protection=none -nopie -grecord-command-line -D NDEBUG -std=c++17 -Wall -Wno-unused-parameter -Wno-unused-function -Wno-unused-variable -O3 -fopenmp=libomp -MD -MT CMakeFiles/cloverleaf.dir/src/omp/advec_mom.cpp.o -MF CMakeFiles/cloverleaf.dir/src/omp/advec_mom.cpp.o.d -o CMakeFiles/cloverleaf.dir/src/omp/advec_mom.cpp.o -c /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/advec_mom.cpp -I /cluster/hpcx/2.23/ompi5-aocc-mt/include -I /cluster/hpcx/2.23/ompi5-aocc-mt/include/openmpi 
  Number of processes observed:	6
  Number of threads observed:	192
  MAQAO version:		2026.1.0
  MAQAO build:			6d1be1d51c1e63266254997eb301734a7264775d::20260810-150026




+--------------------------------------------------------------------------------------------------------------------+
+                                               1.2  -  Global Metrics                                               +
+--------------------------------------------------------------------------------------------------------------------+

  Total Time:				41.24 s
  Max (Thread Active Time):		40.72 s
  Average Active Time:			40.42 s
  Activity Ratio:			99.9 %
  Average number of active threads:	188.196
  Affinity Stability:			99.8 %
  Time spent in analyzed loops:		93.7 %
  Time spent in analyzed innermost loops: 93.7 %
  Time spent in user code:		93.7 %
  Compilation Options Score:		66.67
  Array Access Efficiency:		20.6 %

   Potential Speedups
  ----------------------------------------------------
  Perfect Flow Complexity:		2.30
  Perfect OpenMP/MPI/Pthread/TBB:	1.04
  Perfect OpenMP/MPI/Pthread/TBB + Load Distribution:	1.07
  If No Scalar Integer:
      Potential Speedup:		2.27
      Nb Loops to get 80%:		24
  If FP Vectorized:
      Potential Speedup:		1.29
      Nb Loops to get 80%:		14
  If Fully Vectorized:
      Potential Speedup:		2.31
      Nb Loops to get 80%:		25
  If Only FP Arithmetic:
      Potential Speedup:		3.34
      Nb Loops to get 80%:		26




+--------------------------------------------------------------------------------------------------------------------+
+                                             1.3  -  Potential Speedups                                             +
+--------------------------------------------------------------------------------------------------------------------+

  If No Scalar Integer:
      Number of loops   | 1      | 10     | 20     | 29     | 40     | 
      Cumulated Speedup | 1.0343 | 1.3649 | 1.8714 | 2.2293 | 2.2713 | 
  Top 5 loops:
    exec - 275:	1.0343
    exec - 273:	1.0686
    exec - 182:	1.1041
    exec - 174:	1.1412
    exec - 229:	1.1778

  If FP Vectorized:
      Number of loops   | 1      | 10     | 20     | 29     | 40     | 
      Cumulated Speedup | 1.0328 | 1.1996 | 1.2761 | 1.2888 | 1.2888 | 
  Top 5 loops:
    exec - 275:	1.0328
    exec - 220:	1.0568
    exec - 273:	1.0795
    exec - 141:	1.1012
    exec - 170:	1.119

  If Fully Vectorized:
      Number of loops   | 1      | 10     | 20     | 29     | 40     | 
      Cumulated Speedup | 1.0429 | 1.3901 | 1.8354 | 2.2489 | 2.3138 | 
  Top 5 loops:
    exec - 141:	1.0429
    exec - 275:	1.0886
    exec - 220:	1.128
    exec - 229:	1.1668
    exec - 273:	1.2079

  If Only FP Arithmetic:
      Number of loops   | 1      | 10     | 20     | 29     | 40     | 
      Cumulated Speedup | 1.0463 | 1.4706 | 2.2744 | 3.2038 | 3.3391 | 
  Top 5 loops:
    exec - 275:	1.0463
    exec - 141:	1.0856
    exec - 273:	1.126
    exec - 220:	1.169
    exec - 182:	1.2142



+====================================================================================================================+
+                                                   2  -  SUMMARY                                                    +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                             2.1  -  EXPERIMENT QUALITY                                             +
+--------------------------------------------------------------------------------------------------------------------+

  [4 / 4] Application profile is long enough (40.72 s)
To have good quality measurements, it is advised that the application profiling time is greater than 10 seconds.

  [3 / 3] Most of time spent in analyzed modules comes from functions with source/debug info
-g option gives access to debugging informations, such are source locations.

  [3 / 3] Most of time spent in analyzed modules comes from functions with compilation options informations and
-fno-omit-frame-pointer is present
-fno-omit-frame-pointer improves the accuracy of callchains found during the application profiling.

  [3 / 3] Host configuration allows retrieval of all necessary metrics.


  [0 / 3] Compilation of some functions is not optimized for the target processor
Architecture specific options are needed to produce efficient code for a specific processor ( -march=(target) ).

  [3 / 3] Optimization level option is correctly used


  [2 / 2] Application is correctly profiled ("Others" category represents 0.02 % of the execution time)
To have a representative profiling, it is advised that the category "Others" represents less than 20% of the execution
time in order to analyze as much as possible of the user code

  [1 / 1] Lstopo present. The Topology lstopo report will be generated.


  [0 / 0] Fastmath not used
Consider to add ffast-math to compilation flags (or replace -O3 with -Ofast) to unlock potential extra speedup by
relaxing floating-point computation consistency. Warning: floating-point accuracy may be reduced and the compliance
to IEEE/ISO rules/specifications for math functions will be relaxed, typically 'errno' will no longer be set after
calling some math functions.


+--------------------------------------------------------------------------------------------------------------------+
+                                                2.2  -  CODE QUALITY                                                +
+--------------------------------------------------------------------------------------------------------------------+

  [4 / 4] Enough time of the experiment time spent in analyzed loops (93.68%)
If the time spent in analyzed loops is less than 30%, standard loop optimizations will have a limited impact on
application performances.

  [4 / 4] Threads activity is good
On average, more than 98.02% of observed threads are actually active 

  [4 / 4] CPU activity is good
CPU cores are active 99.85% of time

  [4 / 4] Loop profile is not flat
At least one loop coverage is greater than 4% (6.02%), representing an hotspot for the application

  [4 / 4] Enough time of the experiment time spent in analyzed innermost loops (93.66%)
If the time spent in analyzed innermost loops is less than 15%, standard innermost loop optimizations such as
vectorisation will have a limited impact on application performances.

  [4 / 4] Affinity is good (99.84%)
Threads are not migrating to CPU cores: probably successfully pinned

  [3 / 3] Less than 10% (0.00%) is spend in BLAS1 operations
It could be more efficient to inline by hand BLAS1 operations

  [3 / 3] Functions mostly use all threads
Functions running on a reduced number of threads (typically sequential code) cover less than 10% of application
walltime (5.21%)

  [3 / 3] Cumulative Outermost/In between loops coverage (0.02%) lower than cumulative innermost loop coverage (93.66%)
Having cumulative Outermost/In between loops coverage greater than cumulative innermost loop coverage will make loop
optimization more complex

  [2 / 2] Less than 10% (0.00%) is spend in BLAS2 operations
BLAS2 calls usually could make a poor cache usage and could benefit from inlining.

  [2 / 2] Less than 10% (0.00%) is spend in Libm/SVML (special functions)



+--------------------------------------------------------------------------------------------------------------------+
+                                               2.3  -  LOOPS OVERVIEW                                               +
+--------------------------------------------------------------------------------------------------------------------+

  Top 5 loops:
   + exec - 275 :
     analysis: Execution Time: 6 % - Vectorization Ratio: 36.04 % - Vector Length Use: 16.95 %
     Loop Computation Issues: 22
        [16] [SA] Presence of expensive FP instructions - Perform hoisting, change algorithm, use SVML or proper
            numerical library or perform value profiling (count the number of distinct input values). There are 4
            issues (= instructions) costing 4 points each.
        [4] [SA] Less than 10% of the FP ADD/SUB/MUL arithmetic operations are performed using FMA - Reorganize
            arithmetic expressions to exhibit potential for FMA. This issue costs 4 points.
        [2] [SA] Presence of a large number of scalar integer instructions - Simplify loop structure, perform loop
            splitting or perform unroll and jam. This issue costs 2 points.
     Control Flow Issues: 4
        [4] [SA] Several paths (4 paths) - Simplify control structure or force the compiler to use masked
            instructions. There are 4 issues ( = paths) costing 1 point each.
     Data Access Issues: 83
        [4] [SA] Presence of constant non unit stride data access - Use array restructuring, perform loop interchange
            or use gather instructions to lower a bit the cost. There are 2 issues ( = data accesses) costing 2 point
            each.
        [48] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 12 issues ( = indirect data accesses) costing 4 point each.
        [29] [SA] Presence of special instructions executing on a single port (INSERT/EXTRACT, BLEND/MERGE) - Simplify
            data access and try to get stride 1 access. There are 29 issues (= instructions) costing 1 point each.
        [2] [SA] More than 20% of the loads are accessing the stack - Perform loop splitting to decrease pressure on
            registers. This issue costs 2 points.
     Vectorization Roadblocks: 56
        [4] [SA] Several paths (4 paths) - Simplify control structure or force the compiler to use masked
            instructions. There are 4 issues ( = paths) costing 1 point each.
        [4] [SA] Presence of constant non unit stride data access - Use array restructuring, perform loop interchange
            or use gather instructions to lower a bit the cost. There are 2 issues ( = data accesses) costing 2 point
            each.
        [48] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 12 issues ( = indirect data accesses) costing 4 point each.
     Inefficient Vectorization: 29
        [29] [SA] Presence of special instructions executing on a single port (INSERT/EXTRACT, BLEND/MERGE) - Simplify
            data access and try to get stride 1 access. There are 29 issues (= instructions) costing 1 point each.

   + exec - 229 :
     analysis: Execution Time: 5 % - Vectorization Ratio: 11.02 % - Vector Length Use: 13.61 %
     Loop Computation Issues: 14
        [8] [SA] Presence of expensive FP instructions - Perform hoisting, change algorithm, use SVML or proper
            numerical library or perform value profiling (count the number of distinct input values). There are 2
            issues (= instructions) costing 4 points each.
        [4] [SA] Less than 10% of the FP ADD/SUB/MUL arithmetic operations are performed using FMA - Reorganize
            arithmetic expressions to exhibit potential for FMA. This issue costs 4 points.
        [2] [SA] Presence of a large number of scalar integer instructions - Simplify loop structure, perform loop
            splitting or perform unroll and jam. This issue costs 2 points.
     Control Flow Issues: 4
        [4] [SA] Several paths (4 paths) - Simplify control structure or force the compiler to use masked
            instructions. There are 4 issues ( = paths) costing 1 point each.
     Data Access Issues: 26
        [8] [SA] Presence of constant non unit stride data access - Use array restructuring, perform loop interchange
            or use gather instructions to lower a bit the cost. There are 4 issues ( = data accesses) costing 2 point
            each.
        [16] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 4 issues ( = indirect data accesses) costing 4 point each.
        [2] [SA] More than 20% of the loads are accessing the stack - Perform loop splitting to decrease pressure on
            registers. This issue costs 2 points.
     Vectorization Roadblocks: 28
        [4] [SA] Several paths (4 paths) - Simplify control structure or force the compiler to use masked
            instructions. There are 4 issues ( = paths) costing 1 point each.
        [8] [SA] Presence of constant non unit stride data access - Use array restructuring, perform loop interchange
            or use gather instructions to lower a bit the cost. There are 4 issues ( = data accesses) costing 2 point
            each.
        [16] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 4 issues ( = indirect data accesses) costing 4 point each.

   + exec - 273 :
     analysis: Execution Time: 5 % - Vectorization Ratio: 48.46 % - Vector Length Use: 18.49 %
     Loop Computation Issues: 22
        [16] [SA] Presence of expensive FP instructions - Perform hoisting, change algorithm, use SVML or proper
            numerical library or perform value profiling (count the number of distinct input values). There are 4
            issues (= instructions) costing 4 points each.
        [4] [SA] Less than 10% of the FP ADD/SUB/MUL arithmetic operations are performed using FMA - Reorganize
            arithmetic expressions to exhibit potential for FMA. This issue costs 4 points.
        [2] [SA] Presence of a large number of scalar integer instructions - Simplify loop structure, perform loop
            splitting or perform unroll and jam. This issue costs 2 points.
     Control Flow Issues: 4
        [4] [SA] Several paths (4 paths) - Simplify control structure or force the compiler to use masked
            instructions. There are 4 issues ( = paths) costing 1 point each.
     Data Access Issues: 69
        [2] [SA] Presence of constant non unit stride data access - Use array restructuring, perform loop interchange
            or use gather instructions to lower a bit the cost. There are 1 issues ( = data accesses) costing 2 point
            each.
        [44] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 11 issues ( = indirect data accesses) costing 4 point each.
        [21] [SA] Presence of special instructions executing on a single port (INSERT/EXTRACT, BLEND/MERGE) - Simplify
            data access and try to get stride 1 access. There are 21 issues (= instructions) costing 1 point each.
        [2] [SA] More than 20% of the loads are accessing the stack - Perform loop splitting to decrease pressure on
            registers. This issue costs 2 points.
     Vectorization Roadblocks: 50
        [4] [SA] Several paths (4 paths) - Simplify control structure or force the compiler to use masked
            instructions. There are 4 issues ( = paths) costing 1 point each.
        [2] [SA] Presence of constant non unit stride data access - Use array restructuring, perform loop interchange
            or use gather instructions to lower a bit the cost. There are 1 issues ( = data accesses) costing 2 point
            each.
        [44] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 11 issues ( = indirect data accesses) costing 4 point each.
     Inefficient Vectorization: 21
        [21] [SA] Presence of special instructions executing on a single port (INSERT/EXTRACT, BLEND/MERGE) - Simplify
            data access and try to get stride 1 access. There are 21 issues (= instructions) costing 1 point each.

   + exec - 141 :
     analysis: Execution Time: 5 % - Vectorization Ratio: 25.17 % - Vector Length Use: 15.52 %
     Loop Computation Issues: 10
        [4] [SA] Presence of expensive FP instructions - Perform hoisting, change algorithm, use SVML or proper
            numerical library or perform value profiling (count the number of distinct input values). There are 1
            issues (= instructions) costing 4 points each.
        [4] [SA] Less than 10% of the FP ADD/SUB/MUL arithmetic operations are performed using FMA - Reorganize
            arithmetic expressions to exhibit potential for FMA. This issue costs 4 points.
        [2] [SA] Presence of a large number of scalar integer instructions - Simplify loop structure, perform loop
            splitting or perform unroll and jam. This issue costs 2 points.
     Control Flow Issues: 4
        [4] [SA] Several paths (4 paths) - Simplify control structure or force the compiler to use masked
            instructions. There are 4 issues ( = paths) costing 1 point each.
     Data Access Issues: 101
        [2] [SA] Presence of constant non unit stride data access - Use array restructuring, perform loop interchange
            or use gather instructions to lower a bit the cost. There are 1 issues ( = data accesses) costing 2 point
            each.
        [56] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 14 issues ( = indirect data accesses) costing 4 point each.
        [41] [SA] Presence of special instructions executing on a single port (INSERT/EXTRACT, BLEND/MERGE) - Simplify
            data access and try to get stride 1 access. There are 41 issues (= instructions) costing 1 point each.
        [2] [SA] More than 20% of the loads are accessing the stack - Perform loop splitting to decrease pressure on
            registers. This issue costs 2 points.
     Vectorization Roadblocks: 62
        [4] [SA] Several paths (4 paths) - Simplify control structure or force the compiler to use masked
            instructions. There are 4 issues ( = paths) costing 1 point each.
        [2] [SA] Presence of constant non unit stride data access - Use array restructuring, perform loop interchange
            or use gather instructions to lower a bit the cost. There are 1 issues ( = data accesses) costing 2 point
            each.
        [56] [SA] Presence of indirect accesses - Use array restructuring or gather instructions to lower the cost.
            There are 14 issues ( = indirect data accesses) costing 4 point each.
     Inefficient Vectorization: 41
        [41] [SA] Presence of special instructions executing on a single port (INSERT/EXTRACT, BLEND/MERGE) - Simplify
            data access and try to get stride 1 access. There are 41 issues (= instructions) costing 1 point each.

   + exec - 172 :
     analysis: Execution Time: 4 % - Vectorization Ratio: 70.63 % - Vector Length Use: 20.52 %
     Loop Computation Issues: 38
        [32] [SA] Presence of expensive FP instructions - Perform hoisting, change algorithm, use SVML or proper
            numerical library or perform value profiling (count the number of distinct input values). There are 8
            issues (= instructions) costing 4 points each.
        [4] [SA] Less than 10% of the FP ADD/SUB/MUL arithmetic operations are performed using FMA - Reorganize
            arithmetic expressions to exhibit potential for FMA. This issue costs 4 points.
        [2] [SA] Presence of a large number of scalar integer instructions - Simplify loop structure, perform loop
            splitting or perform unroll and jam. This issue costs 2 points.
     Control Flow Issues: 1000
        [1000] [SA] Too many paths (4096 paths) - Simplify control structure. There are 4096 issues ( = paths) costing 1
            point, limited to 1000.
     Data Access Issues: 36
        [0] [SA] Inefficient vectorization: more than 10% of the vector loads instructions are unaligned - When
            allocating arrays, don’t forget to align them. There are 0 issues ( = arrays) costing 2 points each
        [34] [SA] Presence of special instructions executing on a single port (INSERT/EXTRACT, BLEND/MERGE,
            SHUFFLE/PERM) - Simplify data access and try to get stride 1 access. There are 34 issues (= instructions)
            costing 1 point each.
        [2] [SA] More than 20% of the loads are accessing the stack - Perform loop splitting to decrease pressure on
            registers. This issue costs 2 points.
     Vectorization Roadblocks: 1000
        [1000] [SA] Too many paths (4096 paths) - Simplify control structure. There are 4096 issues ( = paths) costing 1
            point, limited to 1000.
     Inefficient Vectorization: 34
        [34] [SA] Presence of special instructions executing on a single port (INSERT/EXTRACT, BLEND/MERGE,
            SHUFFLE/PERM) - Simplify data access and try to get stride 1 access. There are 34 issues (= instructions)
            costing 1 point each.



+====================================================================================================================+
+                                                 3  -  APPLICATION                                                  +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                               3.1  -  Categorization                                               +
+--------------------------------------------------------------------------------------------------------------------+

   Category | IO     | Exe    | System  | Others  | Memory | String | MPI   | TBB   | OMP   | Pthread | Math  |
  ----------+--------+--------+---------+---------+--------+--------+-------+-------+-------+---------+-------+
   Time (%) | 0.00   | 93.68  | 0.39    | 0.02    | 0.00   | 0.01   | 0.04  | 0.00  | 5.87  | 0.00    | 0.00  |




+--------------------------------------------------------------------------------------------------------------------+
+                                          3.2  -  Function Based Profiling                                          +
+--------------------------------------------------------------------------------------------------------------------+

   Buckets                    | Nb Functions              | Coverage                  | Cumulated Coverage        |
  ----------------------------+---------------------------+---------------------------+---------------------------+
   > 8%                       | 0                         | 0.00                      | 0.00                      |
   4% to 8%                   | 7                         | 34.53                     | 34.53                     |
   2% to 4%                   | 18                        | 55.03                     | 89.56                     |
   1% to 2%                   | 7                         | 8.89                      | 98.44                     |
   0.5% to 1%                 | 0                         | 0.00                      | 98.44                     |
   0.25% to 0.5%              | 3                         | 1.06                      | 99.51                     |
   0.125% to 0.25%            | 0                         | 0.00                      | 99.51                     |
   < 0.125%                   | 162                       | 0.49                      | 100.00                    |




+--------------------------------------------------------------------------------------------------------------------+
+                                            3.3  -  Loop Based Profiling                                            +
+--------------------------------------------------------------------------------------------------------------------+

   Buckets                    | Nb Loops                  | Coverage                  | Cumulated Coverage        |
  ----------------------------+---------------------------+---------------------------+---------------------------+
   > 8%                       | 0                         | 0.00                      | 0.00                      |
   4% to 8%                   | 6                         | 30.19                     | 30.19                     |
   2% to 4%                   | 18                        | 55.03                     | 85.22                     |
   1% to 2%                   | 6                         | 7.81                      | 93.03                     |
   0.5% to 1%                 | 0                         | 0.00                      | 93.03                     |
   0.25% to 0.5%              | 1                         | 0.27                      | 93.29                     |
   0.125% to 0.25%            | 0                         | 0.00                      | 93.29                     |
   < 0.125%                   | 56                        | 0.37                      | 93.66                     |


+====================================================================================================================+
+                                                  4  -  FUNCTIONS                                                   +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                              4.1  -  Top 10 Functions                                              +
+--------------------------------------------------------------------------------------------------------------------+

   Function                                               | Module              | Coverage (%)   | Time (s)       |
  --------------------------------------------------------+---------------------+----------------+----------------+
   PdV_kernel(bool, int, int, int, int, double, clover... | exec                | 6.02           | 2.43           |
   ideal_gas_kernel(int, int, int, int, clover::Buffer... | exec                | 5.32           | 2.15           |
   PdV_kernel(bool, int, int, int, int, double, clover... | exec                | 5.11           | 2.06           |
   accelerate_kernel(int, int, int, int, double, clove... | exec                | 5.02           | 2.03           |
   advec_mom_kernel(int, int, int, int, clover::Buffer... | exec                | 4.37           | 1.77           |
   advec_mom_kernel(int, int, int, int, clover::Buffer... | exec                | 4.35           | 1.76           |
   __kmp_hyper_barrier_release(barrier_type, kmp_info*... | libomp.so           | 4.34           | 1.75           |
   flux_calc_kernel(int, int, int, int, double, clover... | exec                | 3.90           | 1.58           |
   advec_mom_kernel(int, int, int, int, clover::Buffer... | exec                | 3.89           | 1.57           |
   advec_mom_kernel(int, int, int, int, clover::Buffer... | exec                | 3.87           | 1.56           |


+====================================================================================================================+
+                                                    5  -  LOOPS                                                     +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                                5.1  -  Top 10 Loops                                                +
+--------------------------------------------------------------------------------------------------------------------+

   Loop Id        | Module              | Source Location                                        | Coverage (%)   |
  ----------------+---------------------+--------------------------------------------------------+----------------+
   275            | exec                | context.h:69-69,PdV.cpp:70-83                          | 6.02           |
   229            | exec                | ideal_gas.cpp:37-45,context.h:69-69                    | 5.32           |
   273            | exec                | context.h:69-69,PdV.cpp:49-63                          | 5.11           |
   141            | exec                | accelerate.cpp:41-53,context.h:69-69                   | 5.02           |
   172            | exec                | context.h:69-69,advec_mom.cpp:109-139                  | 4.37           |
   180            | exec                | context.h:69-69,advec_mom.cpp:181-211                  | 4.35           |
   220            | exec                | flux_calc.cpp:37-40,context.h:69-69                    | 3.90           |
   182            | exec                | context.h:69-69,advec_mom.cpp:219-221                  | 3.89           |
   174            | exec                | context.h:69-69,advec_mom.cpp:147-149                  | 3.87           |
   201            | exec                | context.h:46-46,context.h:69-69,calc_dt.cpp:49-75      | 3.36           |





+====================================================================================================================+
+                                                     6  -  CQA                                                      +
+====================================================================================================================+


+--------------------------------------------------------------------------------------------------------------------+
+                                                   6.1  -  Loops                                                    +
+--------------------------------------------------------------------------------------------------------------------+





      6.1.1  -  Loop 275 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/PdV.cpp:70-83


It is main loop of related source loop which is unrolled by 2 (including vectorization).
This loop has 4 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.1.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

3% of peak computational performance is used (1.09 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.1.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 67.77 to 30.50 cycles (2.22x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.1.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 16% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 67.77 to 22.50 cycles (3.01x speedup).

Details
36% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 3% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX divide and square root instructions are used in vector version.
 - 25% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.1.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 67.77 to 52.17 cycles (1.30x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.1.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 67.77 to 61.77 cycles (1.10x speedup).


      6.1.1.1.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.20x slowdown from special instructions executing on a single port.


Details
 - LEA: 19 occurrences<<list_path_1_single_port_special_1>>
 - MOVHPD: 26 occurrences<<list_path_1_single_port_special_2>>
 - UNPCKLPD: 1 occurrences<<list_path_1_single_port_special_3>>



      6.1.1.1.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.1.1.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_1_complex_1>>
 - IDIV: 1 occurrences<<list_path_1_complex_2>>



      6.1.1.1.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 2 occurrence(s)
 - Irregular (variable stride) or indirect: 12 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.1.1.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 28 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 28 occurrences<<list_path_1_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.1.1.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_1_cvt_1>>
 - CQTO: 1 occurrences<<list_path_1_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.1.1.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

37 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.1.1.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 74 FP arithmetical operations:
 - 36: addition or subtraction
 - 30: multiply
 - 8: divide
The binary loop is loading 796 bytes (99 double precision FP elements).
The binary loop is storing 104 bytes (13 double precision FP elements).


      6.1.1.1.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.08 FP operations per loaded or stored byte.




      6.1.1.2  -  Path 2
  ----------------------------------------------------------------------------------------------------------

3% of peak computational performance is used (1.09 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.1.2.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 67.77 to 30.50 cycles (2.22x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.1.2.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 16% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 67.77 to 26.00 cycles (2.61x speedup).

Details
35% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 3% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX divide and square root instructions are used in vector version.
 - 20% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.1.2.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 67.77 to 52.17 cycles (1.30x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.1.2.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 67.77 to 61.77 cycles (1.10x speedup).


      6.1.1.2.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.20x slowdown from special instructions executing on a single port.


Details
 - LEA: 19 occurrences<<list_path_2_single_port_special_1>>
 - MOVHPD: 26 occurrences<<list_path_2_single_port_special_2>>
 - UNPCKLPD: 1 occurrences<<list_path_2_single_port_special_3>>



      6.1.1.2.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.1.2.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 2 occurrences<<list_path_2_complex_1>>



      6.1.1.2.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 2 occurrence(s)
 - Irregular (variable stride) or indirect: 12 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.1.2.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 28 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 28 occurrences<<list_path_2_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.1.2.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_2_cvt_1>>
 - CQTO: 2 occurrences<<list_path_2_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.1.2.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

37 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.1.2.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 74 FP arithmetical operations:
 - 36: addition or subtraction
 - 30: multiply
 - 8: divide
The binary loop is loading 796 bytes (99 double precision FP elements).
The binary loop is storing 104 bytes (13 double precision FP elements).


      6.1.1.2.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.08 FP operations per loaded or stored byte.




      6.1.1.3  -  Path 3
  ----------------------------------------------------------------------------------------------------------

3% of peak computational performance is used (1.09 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.1.3.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 67.77 to 30.50 cycles (2.22x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.1.3.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 16% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 67.77 to 19.00 cycles (3.57x speedup).

Details
36% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 3% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX divide and square root instructions are used in vector version.
 - 33% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.1.3.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 67.77 to 51.83 cycles (1.31x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.1.3.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 67.77 to 61.77 cycles (1.10x speedup).


      6.1.1.3.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.21x slowdown from special instructions executing on a single port.


Details
 - LEA: 19 occurrences<<list_path_3_single_port_special_1>>
 - MOVHPD: 26 occurrences<<list_path_3_single_port_special_2>>
 - UNPCKLPD: 1 occurrences<<list_path_3_single_port_special_3>>



      6.1.1.3.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.1.3.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 2 occurrences<<list_path_3_complex_1>>



      6.1.1.3.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 2 occurrence(s)
 - Irregular (variable stride) or indirect: 12 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.1.3.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 28 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 28 occurrences<<list_path_3_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.1.3.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_3_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.1.3.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

37 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.1.3.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 74 FP arithmetical operations:
 - 36: addition or subtraction
 - 30: multiply
 - 8: divide
The binary loop is loading 796 bytes (99 double precision FP elements).
The binary loop is storing 104 bytes (13 double precision FP elements).


      6.1.1.3.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.08 FP operations per loaded or stored byte.




      6.1.1.4  -  Path 4
  ----------------------------------------------------------------------------------------------------------

3% of peak computational performance is used (1.09 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.1.4.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 67.77 to 30.50 cycles (2.22x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.1.4.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 16% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 67.77 to 22.50 cycles (3.01x speedup).

Details
36% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 3% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX divide and square root instructions are used in vector version.
 - 25% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.1.4.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 67.77 to 51.83 cycles (1.31x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.1.4.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 67.77 to 61.77 cycles (1.10x speedup).


      6.1.1.4.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.21x slowdown from special instructions executing on a single port.


Details
 - LEA: 19 occurrences<<list_path_4_single_port_special_1>>
 - MOVHPD: 26 occurrences<<list_path_4_single_port_special_2>>
 - UNPCKLPD: 1 occurrences<<list_path_4_single_port_special_3>>



      6.1.1.4.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.1.4.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_4_complex_1>>
 - IDIV: 1 occurrences<<list_path_4_complex_2>>



      6.1.1.4.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 2 occurrence(s)
 - Irregular (variable stride) or indirect: 12 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.1.4.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 28 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 28 occurrences<<list_path_4_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.1.4.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_4_cvt_1>>
 - CQTO: 1 occurrences<<list_path_4_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.1.4.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

37 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.1.4.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 74 FP arithmetical operations:
 - 36: addition or subtraction
 - 30: multiply
 - 8: divide
The binary loop is loading 796 bytes (99 double precision FP elements).
The binary loop is storing 104 bytes (13 double precision FP elements).


      6.1.1.4.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.08 FP operations per loaded or stored byte.







      6.1.2  -  Loop 229 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/ideal_gas.cpp:37-45
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69


The related source loop is not unrolled or unrolled with no peel/tail loop.
This loop has 4 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.2.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.54 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.2.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 18.50 to 8.50 cycles (2.18x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.2.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is probably not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By vectorizing your loop, you can lower the cost of an iteration from 18.50 to 9.25 cycles (2.00x speedup).

Details
Store and arithmetical SSE/AVX instructions are used in scalar version (process only one data element in vector registers).
Since your execution units are vector units, only a vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.2.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 18.50 to 9.07 cycles (2.04x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.2.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 18.50 to 8.17 cycles (2.27x speedup).


      6.1.2.1.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.2.1.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 1 occurrences<<list_path_1_complex_1>>



      6.1.2.1.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 4 occurrence(s)
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.2.1.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 1 occurrences<<list_path_1_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.2.1.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

11 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.2.1.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 10 FP arithmetical operations:
 - 1: addition or subtraction
 - 7: multiply
 - 1: divide
 - 1: square root
The binary loop is loading 100 bytes (12 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.2.1.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.09 FP operations per loaded or stored byte.




      6.1.2.2  -  Path 2
  ----------------------------------------------------------------------------------------------------------

Warnings:
Detected a function call instruction: ignoring called function instructions.
Rerun with --follow-calls=append to include them to analysis  or with --follow-calls=inline to simulate inlining.
2% of peak computational performance is used (0.64 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.2.2.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 14.00 to 5.50 cycles (2.55x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.2.2.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is probably not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By vectorizing your loop, you can lower the cost of an iteration from 14.00 to 7.00 cycles (2.00x speedup).

Details
Store and arithmetical SSE/AVX instructions are used in scalar version (process only one data element in vector registers).
Since your execution units are vector units, only a vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.2.2.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 14.00 to 10.67 cycles (1.31x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.2.2.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 14.00 to 10.00 cycles (1.40x speedup).


      6.1.2.2.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.2.2.6  -  CALL instructions
  ----------------------------------------------------------------------------------------------------------

Detected function call instructions.


Details
Calling (and then returning from) a function prevents many compiler optimizations (like vectorization), breaks control flow (which reduces pipeline performance) and executes extra instructions to save/restore the registers used inside it, which is very expensive (dozens of cycles). Consider to inline small functions.
 - unknown: 1 occurrences<<list_path_2_call_1>>



      6.1.2.2.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 1 occurrences<<list_path_2_complex_1>>



      6.1.2.2.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 4 occurrence(s)
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.2.2.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 1 occurrences<<list_path_2_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.2.2.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

10 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.2.2.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 9 FP arithmetical operations:
 - 1: addition or subtraction
 - 7: multiply
 - 1: divide
The binary loop is loading 144 bytes (18 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.2.2.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.




      6.1.2.3  -  Path 3
  ----------------------------------------------------------------------------------------------------------

2% of peak computational performance is used (0.69 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.2.3.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 14.50 to 8.50 cycles (1.71x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.2.3.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is probably not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By vectorizing your loop, you can lower the cost of an iteration from 14.50 to 5.75 cycles (2.52x speedup).

Details
Store and arithmetical SSE/AVX instructions are used in scalar version (process only one data element in vector registers).
Since your execution units are vector units, only a vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.2.3.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 14.50 to 9.07 cycles (1.60x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.2.3.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 14.50 to 8.00 cycles (1.81x speedup).


      6.1.2.3.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.2.3.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_3_complex_1>>



      6.1.2.3.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 4 occurrence(s)
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.2.3.8  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

11 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.2.3.9  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 10 FP arithmetical operations:
 - 1: addition or subtraction
 - 7: multiply
 - 1: divide
 - 1: square root
The binary loop is loading 100 bytes (12 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.2.3.10  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.09 FP operations per loaded or stored byte.




      6.1.2.4  -  Path 4
  ----------------------------------------------------------------------------------------------------------

Warnings:
Detected a function call instruction: ignoring called function instructions.
Rerun with --follow-calls=append to include them to analysis  or with --follow-calls=inline to simulate inlining.
2% of peak computational performance is used (0.87 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.2.4.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 10.33 to 5.50 cycles (1.88x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.2.4.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is probably not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By vectorizing your loop, you can lower the cost of an iteration from 10.33 to 3.50 cycles (2.95x speedup).

Details
Store and arithmetical SSE/AVX instructions are used in scalar version (process only one data element in vector registers).
Since your execution units are vector units, only a vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.2.4.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.2.4.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 10.33 to 9.83 cycles (1.05x speedup).


      6.1.2.4.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.2.4.6  -  CALL instructions
  ----------------------------------------------------------------------------------------------------------

Detected function call instructions.


Details
Calling (and then returning from) a function prevents many compiler optimizations (like vectorization), breaks control flow (which reduces pipeline performance) and executes extra instructions to save/restore the registers used inside it, which is very expensive (dozens of cycles). Consider to inline small functions.
 - unknown: 1 occurrences<<list_path_4_call_1>>



      6.1.2.4.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_4_complex_1>>



      6.1.2.4.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 4 occurrence(s)
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.2.4.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

10 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.2.4.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 9 FP arithmetical operations:
 - 1: addition or subtraction
 - 7: multiply
 - 1: divide
The binary loop is loading 144 bytes (18 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.2.4.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.


      6.1.2.4.12  -  Unroll opportunity
  ----------------------------------------------------------------------------------------------------------

Loop is potentially data access bound.

Workaround
Unroll your loop if trip count is significantly higher than target unroll factor and if some data references are common to consecutive iterations. This can be done manually. Or with the unroll (resp. unroll_and_jam) directive on top of the inner (resp. surrounding) loop. You can enforce an unroll factor: #pragma unroll_and_jam N, unroll_and_jam(N), unroll N or unroll(N)







      6.1.3  -  Loop 273 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/PdV.cpp:49-63


It is main loop of related source loop which is unrolled by 2 (including vectorization).
This loop has 4 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.3.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

4% of peak computational performance is used (1.56 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.3.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 52.40 to 20.50 cycles (2.56x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.3.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 18% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 52.40 to 22.50 cycles (2.33x speedup).

Details
48% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX divide and square root instructions are used in vector version.
 - 75% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.3.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 52.40 to 42.00 cycles (1.25x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.3.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 52.40 to 46.40 cycles (1.13x speedup).


      6.1.3.1.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.16x slowdown from special instructions executing on a single port.


Details
 - LEA: 12 occurrences<<list_path_1_single_port_special_1>>
 - MOVHPD: 18 occurrences<<list_path_1_single_port_special_2>>
 - UNPCKLPD: 1 occurrences<<list_path_1_single_port_special_3>>



      6.1.3.1.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.3.1.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_1_complex_1>>
 - IDIV: 1 occurrences<<list_path_1_complex_2>>



      6.1.3.1.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 1 occurrence(s)
 - Irregular (variable stride) or indirect: 11 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.3.1.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 20 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 20 occurrences<<list_path_1_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.3.1.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 1 occurrences<<list_path_1_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.3.1.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

41 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.3.1.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 82 FP arithmetical operations:
 - 36: addition or subtraction
 - 38: multiply
 - 8: divide
The binary loop is loading 548 bytes (68 double precision FP elements).
The binary loop is storing 72 bytes (9 double precision FP elements).


      6.1.3.1.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.13 FP operations per loaded or stored byte.




      6.1.3.2  -  Path 2
  ----------------------------------------------------------------------------------------------------------

4% of peak computational performance is used (1.56 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.3.2.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 52.40 to 20.50 cycles (2.56x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.3.2.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 18% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 52.40 to 26.00 cycles (2.02x speedup).

Details
47% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX divide and square root instructions are used in vector version.
 - 66% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.3.2.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 52.40 to 42.00 cycles (1.25x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.3.2.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 52.40 to 46.40 cycles (1.13x speedup).


      6.1.3.2.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.16x slowdown from special instructions executing on a single port.


Details
 - LEA: 12 occurrences<<list_path_2_single_port_special_1>>
 - MOVHPD: 18 occurrences<<list_path_2_single_port_special_2>>
 - UNPCKLPD: 1 occurrences<<list_path_2_single_port_special_3>>



      6.1.3.2.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.3.2.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 2 occurrences<<list_path_2_complex_1>>



      6.1.3.2.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 1 occurrence(s)
 - Irregular (variable stride) or indirect: 11 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.3.2.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 20 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 20 occurrences<<list_path_2_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.3.2.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 2 occurrences<<list_path_2_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.3.2.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

41 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.3.2.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 82 FP arithmetical operations:
 - 36: addition or subtraction
 - 38: multiply
 - 8: divide
The binary loop is loading 548 bytes (68 double precision FP elements).
The binary loop is storing 72 bytes (9 double precision FP elements).


      6.1.3.2.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.13 FP operations per loaded or stored byte.




      6.1.3.3  -  Path 3
  ----------------------------------------------------------------------------------------------------------

4% of peak computational performance is used (1.56 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.3.3.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 52.40 to 20.50 cycles (2.56x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.3.3.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 18% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 52.40 to 19.00 cycles (2.76x speedup).

Details
48% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX divide and square root instructions are used in vector version.
 - 85% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.3.3.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 52.40 to 41.67 cycles (1.26x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.3.3.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 52.40 to 46.40 cycles (1.13x speedup).


      6.1.3.3.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.17x slowdown from special instructions executing on a single port.


Details
 - LEA: 12 occurrences<<list_path_3_single_port_special_1>>
 - MOVHPD: 18 occurrences<<list_path_3_single_port_special_2>>
 - UNPCKLPD: 1 occurrences<<list_path_3_single_port_special_3>>



      6.1.3.3.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.3.3.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 2 occurrences<<list_path_3_complex_1>>



      6.1.3.3.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 1 occurrence(s)
 - Irregular (variable stride) or indirect: 11 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.3.3.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 20 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 20 occurrences<<list_path_3_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.3.3.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

41 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.3.3.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 82 FP arithmetical operations:
 - 36: addition or subtraction
 - 38: multiply
 - 8: divide
The binary loop is loading 548 bytes (68 double precision FP elements).
The binary loop is storing 72 bytes (9 double precision FP elements).


      6.1.3.3.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.13 FP operations per loaded or stored byte.




      6.1.3.4  -  Path 4
  ----------------------------------------------------------------------------------------------------------

4% of peak computational performance is used (1.56 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.3.4.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 52.40 to 20.50 cycles (2.56x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.3.4.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 18% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 52.40 to 22.50 cycles (2.33x speedup).

Details
48% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX divide and square root instructions are used in vector version.
 - 75% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.3.4.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 52.40 to 41.67 cycles (1.26x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.3.4.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 52.40 to 46.40 cycles (1.13x speedup).


      6.1.3.4.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.17x slowdown from special instructions executing on a single port.


Details
 - LEA: 12 occurrences<<list_path_4_single_port_special_1>>
 - MOVHPD: 18 occurrences<<list_path_4_single_port_special_2>>
 - UNPCKLPD: 1 occurrences<<list_path_4_single_port_special_3>>



      6.1.3.4.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.3.4.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_4_complex_1>>
 - IDIV: 1 occurrences<<list_path_4_complex_2>>



      6.1.3.4.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 1 occurrence(s)
 - Irregular (variable stride) or indirect: 11 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.3.4.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 20 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 20 occurrences<<list_path_4_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.3.4.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 1 occurrences<<list_path_4_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.3.4.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

41 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.3.4.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 82 FP arithmetical operations:
 - 36: addition or subtraction
 - 38: multiply
 - 8: divide
The binary loop is loading 548 bytes (68 double precision FP elements).
The binary loop is storing 72 bytes (9 double precision FP elements).


      6.1.3.4.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.13 FP operations per loaded or stored byte.







      6.1.4  -  Loop 141 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/accelerate.cpp:41-53
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69


It is main loop of related source loop which is unrolled by 2 (including vectorization).
This loop has 4 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.4.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

3% of peak computational performance is used (1.27 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.4.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 58.33 to 37.00 cycles (1.58x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.4.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 15% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 58.33 to 10.50 cycles (5.56x speedup).

Details
25% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.4.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.4.1.4  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.4.1.5  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_1_complex_1>>
 - IDIV: 1 occurrences<<list_path_1_complex_2>>



      6.1.4.1.6  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 1 occurrence(s)
 - Irregular (variable stride) or indirect: 14 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.4.1.7  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 40 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 40 occurrences<<list_path_1_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.4.1.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 1 occurrences<<list_path_1_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.4.1.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

37 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.4.1.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 74 FP arithmetical operations:
 - 38: addition or subtraction
 - 34: multiply
 - 2: divide
The binary loop is loading 1064 bytes (133 double precision FP elements).
The binary loop is storing 256 bytes (32 double precision FP elements).


      6.1.4.1.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.




      6.1.4.2  -  Path 2
  ----------------------------------------------------------------------------------------------------------

3% of peak computational performance is used (1.27 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.4.2.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 58.33 to 37.00 cycles (1.58x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.4.2.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 15% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 58.33 to 14.00 cycles (4.17x speedup).

Details
25% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.4.2.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.4.2.4  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.4.2.5  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 2 occurrences<<list_path_2_complex_1>>



      6.1.4.2.6  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 1 occurrence(s)
 - Irregular (variable stride) or indirect: 14 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.4.2.7  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 40 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 40 occurrences<<list_path_2_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.4.2.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 2 occurrences<<list_path_2_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.4.2.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

37 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.4.2.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 74 FP arithmetical operations:
 - 38: addition or subtraction
 - 34: multiply
 - 2: divide
The binary loop is loading 1064 bytes (133 double precision FP elements).
The binary loop is storing 256 bytes (32 double precision FP elements).


      6.1.4.2.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.




      6.1.4.3  -  Path 3
  ----------------------------------------------------------------------------------------------------------

3% of peak computational performance is used (1.28 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.4.3.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 58.00 to 37.00 cycles (1.57x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.4.3.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 15% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 58.00 to 7.00 cycles (8.29x speedup).

Details
25% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.4.3.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.4.3.4  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.4.3.5  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 2 occurrences<<list_path_3_complex_1>>



      6.1.4.3.6  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 1 occurrence(s)
 - Irregular (variable stride) or indirect: 14 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.4.3.7  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 40 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 40 occurrences<<list_path_3_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.4.3.8  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

37 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.4.3.9  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 74 FP arithmetical operations:
 - 38: addition or subtraction
 - 34: multiply
 - 2: divide
The binary loop is loading 1064 bytes (133 double precision FP elements).
The binary loop is storing 256 bytes (32 double precision FP elements).


      6.1.4.3.10  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.




      6.1.4.4  -  Path 4
  ----------------------------------------------------------------------------------------------------------

3% of peak computational performance is used (1.28 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.4.4.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 58.00 to 37.00 cycles (1.57x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.4.4.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 15% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 58.00 to 10.50 cycles (5.52x speedup).

Details
25% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.4.4.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.4.4.4  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.4.4.5  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_4_complex_1>>
 - IDIV: 1 occurrences<<list_path_4_complex_2>>



      6.1.4.4.6  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 1 occurrence(s)
 - Irregular (variable stride) or indirect: 14 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.4.4.7  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 40 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 40 occurrences<<list_path_4_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.4.4.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 1 occurrences<<list_path_4_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.4.4.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

37 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.4.4.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 74 FP arithmetical operations:
 - 38: addition or subtraction
 - 34: multiply
 - 2: divide
The binary loop is loading 1064 bytes (133 double precision FP elements).
The binary loop is storing 256 bytes (32 double precision FP elements).


      6.1.4.4.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.







      6.1.5  -  Loop 172 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/advec_mom.cpp:109-139


It is main loop of related source loop which is unrolled by 4 (including vectorization).
Warnings:
 - Ignoring paths for analysis
 - Too many paths. If you really need to analyze all of the 4096 paths individually, rerun with max-paths=4096
 - RecMII not computed since number of paths is unknown or > max_paths
 - Streams not analyzed since number of paths is unknown or > max_paths

Try to simplify control and/or increase the maximum number of paths per function/loop through the 'max-paths-nb' option.

This loop has 4096 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.5.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

Warnings:
The number of fused uops of the instruction [PCMPEQD	%XMM14,%XMM14] is unknown
2% of peak computational performance is used (0.71 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.5.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 96.00 to 56.17 cycles (1.71x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.5.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is partially vectorized.
Only 20% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 96.00 to 58.00 cycles (1.66x speedup).

Details
70% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 50% of SSE/AVX loads are used in vector version.
 - 50% of SSE/AVX stores are used in vector version.
 - 50% of SSE/AVX divide and square root instructions are used in vector version.
 - 76% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.5.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 96.00 to 75.33 cycles (1.27x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.5.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 96.00 to 70.50 cycles (1.36x speedup).


      6.1.5.1.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.5.1.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 4 occurrences<<list_path_1_complex_1>>
 - IDIV: 4 occurrences<<list_path_1_complex_2>>
 - PEXTRW: 2 occurrences<<list_path_1_complex_3>>



      6.1.5.1.7  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 16 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 12 occurrences<<list_path_1_vec_align_1>>
 - MOVHPS: 4 occurrences<<list_path_1_vec_align_2>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.5.1.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_1_cvt_1>>
 - CQTO: 4 occurrences<<list_path_1_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.5.1.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).
48 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.5.1.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 68 FP arithmetical operations:
 - 28: addition or subtraction
 - 24: multiply
 - 16: divide
The binary loop is loading 764 bytes (95 double precision FP elements).
The binary loop is storing 240 bytes (30 double precision FP elements).


      6.1.5.1.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.07 FP operations per loaded or stored byte.







      6.1.6  -  Loop 180 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/advec_mom.cpp:181-211


It is main loop of related source loop which is unrolled by 4 (including vectorization).
Warnings:
 - Ignoring paths for analysis
 - Too many paths. If you really need to analyze all of the 4096 paths individually, rerun with max-paths=4096
 - RecMII not computed since number of paths is unknown or > max_paths
 - Streams not analyzed since number of paths is unknown or > max_paths

Try to simplify control and/or increase the maximum number of paths per function/loop through the 'max-paths-nb' option.

This loop has 4096 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.6.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

Warnings:
The number of fused uops of the instruction [PCMPEQD	%XMM14,%XMM14] is unknown
2% of peak computational performance is used (0.71 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.6.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 96.00 to 56.17 cycles (1.71x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.6.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is partially vectorized.
Only 20% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 96.00 to 58.00 cycles (1.66x speedup).

Details
69% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 46% of SSE/AVX loads are used in vector version.
 - 52% of SSE/AVX stores are used in vector version.
 - 50% of SSE/AVX divide and square root instructions are used in vector version.
 - 74% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.6.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 96.00 to 78.17 cycles (1.23x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.6.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 96.00 to 73.33 cycles (1.31x speedup).


      6.1.6.1.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.6.1.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 4 occurrences<<list_path_1_complex_1>>
 - IDIV: 4 occurrences<<list_path_1_complex_2>>
 - PEXTRW: 2 occurrences<<list_path_1_complex_3>>
 - PSHUFD: 1 occurrences<<list_path_1_complex_4>>



      6.1.6.1.7  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 16 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 13 occurrences<<list_path_1_vec_align_1>>
 - MOVHPS: 3 occurrences<<list_path_1_vec_align_2>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.6.1.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CQTO: 4 occurrences<<list_path_1_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.6.1.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).
48 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.6.1.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 68 FP arithmetical operations:
 - 28: addition or subtraction
 - 24: multiply
 - 16: divide
The binary loop is loading 744 bytes (93 double precision FP elements).
The binary loop is storing 232 bytes (29 double precision FP elements).


      6.1.6.1.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.07 FP operations per loaded or stored byte.







      6.1.7  -  Loop 220 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/flux_calc.cpp:37-40
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69


It is main loop of related source loop which is unrolled by 2 (including vectorization).
This loop has 4 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.7.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.60 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.7.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 36.57 to 12.00 cycles (3.05x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.7.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 15% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 36.57 to 3.00 cycles (12.19x speedup).

Details
22% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX multiply instructions are used in vector version.
 - 0% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.7.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 36.57 to 25.83 cycles (1.42x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.7.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 36.57 to 30.57 cycles (1.20x speedup).


      6.1.7.1.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.06x slowdown from special instructions executing on a single port.


Details
 - LEA: 4 occurrences<<list_path_1_single_port_special_1>>
 - MOVHPD: 10 occurrences<<list_path_1_single_port_special_2>>
 - UNPCKLPD: 2 occurrences<<list_path_1_single_port_special_3>>



      6.1.7.1.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.7.1.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 2 occurrences<<list_path_1_complex_1>>



      6.1.7.1.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 2 occurrence(s)
 - Irregular (variable stride) or indirect: 8 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.7.1.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 12 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 12 occurrences<<list_path_1_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.7.1.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_1_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.7.1.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

2 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).
10 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.7.1.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 22 FP arithmetical operations:
 - 12: addition or subtraction
 - 10: multiply
The binary loop is loading 340 bytes (42 double precision FP elements).
The binary loop is storing 32 bytes (4 double precision FP elements).


      6.1.7.1.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.




      6.1.7.2  -  Path 2
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.60 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.7.2.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 36.57 to 12.00 cycles (3.05x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.7.2.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 15% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 36.57 to 6.50 cycles (5.63x speedup).

Details
22% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX multiply instructions are used in vector version.
 - 0% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.7.2.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 36.57 to 25.83 cycles (1.42x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.7.2.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 36.57 to 30.57 cycles (1.20x speedup).


      6.1.7.2.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.06x slowdown from special instructions executing on a single port.


Details
 - LEA: 4 occurrences<<list_path_2_single_port_special_1>>
 - MOVHPD: 10 occurrences<<list_path_2_single_port_special_2>>
 - UNPCKLPD: 2 occurrences<<list_path_2_single_port_special_3>>



      6.1.7.2.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.7.2.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_2_complex_1>>
 - IDIV: 1 occurrences<<list_path_2_complex_2>>



      6.1.7.2.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 2 occurrence(s)
 - Irregular (variable stride) or indirect: 8 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.7.2.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 12 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 12 occurrences<<list_path_2_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.7.2.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_2_cvt_1>>
 - CQTO: 1 occurrences<<list_path_2_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.7.2.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

2 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).
10 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.7.2.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 22 FP arithmetical operations:
 - 12: addition or subtraction
 - 10: multiply
The binary loop is loading 340 bytes (42 double precision FP elements).
The binary loop is storing 32 bytes (4 double precision FP elements).


      6.1.7.2.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.




      6.1.7.3  -  Path 3
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.60 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.7.3.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 36.57 to 12.00 cycles (3.05x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.7.3.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 15% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 36.57 to 6.50 cycles (5.63x speedup).

Details
22% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX multiply instructions are used in vector version.
 - 0% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.7.3.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 36.57 to 26.17 cycles (1.40x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.7.3.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 36.57 to 30.57 cycles (1.20x speedup).


      6.1.7.3.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.06x slowdown from special instructions executing on a single port.


Details
 - LEA: 4 occurrences<<list_path_3_single_port_special_1>>
 - MOVHPD: 10 occurrences<<list_path_3_single_port_special_2>>
 - UNPCKLPD: 2 occurrences<<list_path_3_single_port_special_3>>



      6.1.7.3.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.7.3.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_3_complex_1>>
 - IDIV: 1 occurrences<<list_path_3_complex_2>>



      6.1.7.3.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 2 occurrence(s)
 - Irregular (variable stride) or indirect: 8 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.7.3.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 12 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 12 occurrences<<list_path_3_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.7.3.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_3_cvt_1>>
 - CQTO: 1 occurrences<<list_path_3_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.7.3.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

2 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).
10 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.7.3.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 22 FP arithmetical operations:
 - 12: addition or subtraction
 - 10: multiply
The binary loop is loading 340 bytes (42 double precision FP elements).
The binary loop is storing 32 bytes (4 double precision FP elements).


      6.1.7.3.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.




      6.1.7.4  -  Path 4
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.60 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.7.4.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 36.57 to 12.00 cycles (3.05x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.7.4.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 15% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 36.57 to 10.00 cycles (3.66x speedup).

Details
21% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX multiply instructions are used in vector version.
 - 0% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.7.4.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 36.57 to 26.00 cycles (1.41x speedup).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.7.4.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 36.57 to 30.57 cycles (1.20x speedup).


      6.1.7.4.5  -  Special instructions executing on a single port
  ----------------------------------------------------------------------------------------------------------

1.06x slowdown from special instructions executing on a single port.


Details
 - LEA: 4 occurrences<<list_path_4_single_port_special_1>>
 - MOVHPD: 10 occurrences<<list_path_4_single_port_special_2>>
 - UNPCKLPD: 2 occurrences<<list_path_4_single_port_special_3>>



      6.1.7.4.6  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.7.4.7  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 2 occurrences<<list_path_4_complex_1>>



      6.1.7.4.8  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 2 occurrence(s)
 - Irregular (variable stride) or indirect: 8 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.7.4.9  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 12 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 12 occurrences<<list_path_4_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.7.4.10  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_4_cvt_1>>
 - CQTO: 2 occurrences<<list_path_4_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.7.4.11  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

2 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).
10 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.7.4.12  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 22 FP arithmetical operations:
 - 12: addition or subtraction
 - 10: multiply
The binary loop is loading 340 bytes (42 double precision FP elements).
The binary loop is storing 32 bytes (4 double precision FP elements).


      6.1.7.4.13  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.06 FP operations per loaded or stored byte.







      6.1.8  -  Loop 182 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/advec_mom.cpp:219-221


It is main loop of related source loop which is unrolled by 2 (including vectorization).
This loop has 4 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.8.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.37 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.8.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 21.37 to 5.00 cycles (4.27x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.8.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 21.37 to 10.50 cycles (2.03x speedup).

Details
16% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX addition or subtraction instructions are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.8.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.8.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 21.37 to 15.37 cycles (1.39x speedup).


      6.1.8.1.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.8.1.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_1_complex_1>>
 - IDIV: 1 occurrences<<list_path_1_complex_2>>



      6.1.8.1.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.8.1.8  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 6 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 6 occurrences<<list_path_1_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.8.1.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_1_cvt_1>>
 - CQTO: 1 occurrences<<list_path_1_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.8.1.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.8.1.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 8 FP arithmetical operations:
 - 4: addition or subtraction
 - 2: multiply
 - 2: divide
The binary loop is loading 156 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.8.1.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.




      6.1.8.2  -  Path 2
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.33 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.8.2.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 24.00 to 5.00 cycles (4.80x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.8.2.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
Only 14% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 24.00 to 14.00 cycles (1.71x speedup).

Details
16% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX addition or subtraction instructions are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.8.2.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 24.00 to 21.37 cycles (1.12x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.8.2.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 24.00 to 15.37 cycles (1.56x speedup).


      6.1.8.2.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.8.2.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 2 occurrences<<list_path_2_complex_1>>



      6.1.8.2.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.8.2.8  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 6 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 6 occurrences<<list_path_2_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.8.2.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_2_cvt_1>>
 - CQTO: 2 occurrences<<list_path_2_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.8.2.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.8.2.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 8 FP arithmetical operations:
 - 4: addition or subtraction
 - 2: multiply
 - 2: divide
The binary loop is loading 156 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.8.2.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.




      6.1.8.3  -  Path 3
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.37 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.8.3.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 21.37 to 5.00 cycles (4.27x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.8.3.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 21.37 to 7.00 cycles (3.05x speedup).

Details
16% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX addition or subtraction instructions are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.8.3.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.8.3.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 21.37 to 15.37 cycles (1.39x speedup).


      6.1.8.3.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.8.3.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 2 occurrences<<list_path_3_complex_1>>



      6.1.8.3.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.8.3.8  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 6 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 6 occurrences<<list_path_3_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.8.3.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_3_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.8.3.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.8.3.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 8 FP arithmetical operations:
 - 4: addition or subtraction
 - 2: multiply
 - 2: divide
The binary loop is loading 156 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.8.3.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.




      6.1.8.4  -  Path 4
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.37 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.8.4.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 21.37 to 5.00 cycles (4.27x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.8.4.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 21.37 to 10.50 cycles (2.03x speedup).

Details
16% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 66% of SSE/AVX addition or subtraction instructions are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.8.4.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.8.4.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 21.37 to 15.37 cycles (1.39x speedup).


      6.1.8.4.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.8.4.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_4_complex_1>>
 - IDIV: 1 occurrences<<list_path_4_complex_2>>



      6.1.8.4.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.8.4.8  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 6 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 6 occurrences<<list_path_4_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.8.4.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_4_cvt_1>>
 - CQTO: 1 occurrences<<list_path_4_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.8.4.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.8.4.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 8 FP arithmetical operations:
 - 4: addition or subtraction
 - 2: multiply
 - 2: divide
The binary loop is loading 156 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.8.4.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.







      6.1.9  -  Loop 174 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:69
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/advec_mom.cpp:147-149


It is main loop of related source loop which is unrolled by 2 (including vectorization).
This loop has 4 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.9.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.40 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.9.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 20.00 to 5.00 cycles (4.00x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.9.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 20.00 to 10.50 cycles (1.90x speedup).

Details
18% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.9.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.9.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 20.00 to 13.37 cycles (1.50x speedup).


      6.1.9.1.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.9.1.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_1_complex_1>>
 - IDIV: 1 occurrences<<list_path_1_complex_2>>



      6.1.9.1.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.9.1.8  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 6 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 6 occurrences<<list_path_1_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.9.1.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_1_cvt_1>>
 - CQTO: 1 occurrences<<list_path_1_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.9.1.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.9.1.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 8 FP arithmetical operations:
 - 4: addition or subtraction
 - 2: multiply
 - 2: divide
The binary loop is loading 156 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.9.1.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.




      6.1.9.2  -  Path 2
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.33 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.9.2.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 24.00 to 5.00 cycles (4.80x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.9.2.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
Only 14% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 24.00 to 14.00 cycles (1.71x speedup).

Details
17% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.9.2.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 24.00 to 19.37 cycles (1.24x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.9.2.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 24.00 to 13.37 cycles (1.80x speedup).


      6.1.9.2.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.9.2.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 2 occurrences<<list_path_2_complex_1>>



      6.1.9.2.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.9.2.8  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 6 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 6 occurrences<<list_path_2_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.9.2.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_2_cvt_1>>
 - CQTO: 2 occurrences<<list_path_2_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.9.2.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.9.2.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 8 FP arithmetical operations:
 - 4: addition or subtraction
 - 2: multiply
 - 2: divide
The binary loop is loading 156 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.9.2.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.




      6.1.9.3  -  Path 3
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.41 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.9.3.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 19.37 to 5.00 cycles (3.87x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.9.3.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 19.37 to 7.00 cycles (2.77x speedup).

Details
19% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.9.3.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of FP multiply or FMA (fused multiply-add) operations (the FP multiply/FMA unit is a bottleneck).


Workaround
Reduce the number of FP multiply/FMA instructions




      6.1.9.3.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 19.37 to 13.37 cycles (1.45x speedup).


      6.1.9.3.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.9.3.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 2 occurrences<<list_path_3_complex_1>>



      6.1.9.3.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.9.3.8  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 6 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 6 occurrences<<list_path_3_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.9.3.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_3_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.9.3.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.9.3.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 8 FP arithmetical operations:
 - 4: addition or subtraction
 - 2: multiply
 - 2: divide
The binary loop is loading 156 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.9.3.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.




      6.1.9.4  -  Path 4
  ----------------------------------------------------------------------------------------------------------

1% of peak computational performance is used (0.40 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.9.4.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 20.00 to 5.00 cycles (4.00x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.9.4.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is not vectorized.
Only 13% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 20.00 to 10.50 cycles (1.90x speedup).

Details
18% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 0% of SSE/AVX loads are used in vector version.
 - 0% of SSE/AVX stores are used in vector version.
 - 33% of SSE/AVX divide and square root instructions are used in vector version.
 - 0% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.9.4.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.9.4.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 20.00 to 13.37 cycles (1.50x speedup).


      6.1.9.4.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.9.4.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_4_complex_1>>
 - IDIV: 1 occurrences<<list_path_4_complex_2>>



      6.1.9.4.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Irregular (variable stride) or indirect: 4 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.


      6.1.9.4.8  -  Vector unaligned load/store instructions
  ----------------------------------------------------------------------------------------------------------

Detected 6 suboptimal vector unaligned load/store instructions.


Details
 - MOVHPD: 6 occurrences<<list_path_4_vec_align_1>>


Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Use vector aligned instructions:
  1) align your arrays on 64 bytes boundaries: replace { void *p = malloc (size); } with { void *p; posix_memalign (&p, 64, size); }.
  2) inform your compiler that your arrays are vector aligned: if array 'foo' is 64 bytes-aligned, define a pointer 'p_foo' as __builtin_assume_aligned (foo, 64) and use it instead of 'foo' in the loop.
<<image_vec_align>>


      6.1.9.4.9  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_4_cvt_1>>
 - CQTO: 1 occurrences<<list_path_4_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.9.4.10  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

4 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in vector mode (two at a time).



      6.1.9.4.11  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 8 FP arithmetical operations:
 - 4: addition or subtraction
 - 2: multiply
 - 2: divide
The binary loop is loading 156 bytes (19 double precision FP elements).
The binary loop is storing 16 bytes (2 double precision FP elements).


      6.1.9.4.12  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.







      6.1.10  -  Loop 201 from exec
  ==========================================================================================================

The loop is defined in:
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/context.h:46,69
 - /beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/build/CloverLeaf2.0-CXX/src/omp/calc_dt.cpp:49-75


The related source loop is not unrolled or unrolled with no peel/tail loop.
This loop has 4 execution paths.

The presence of multiple execution paths is typically the main/first bottleneck.
Try to simplify control inside loop: ideally, try to remove all conditional expressions, for example by (if applicable):
 - hoisting them (moving them outside the loop)
 - turning them into conditional moves, MIN or MAX


Ex: if (x<0) x=0 => x = (x<0 ? 0 : x) (or MAX(0,x) after defining the corresponding macro)


      6.1.10.1  -  Path 1
  ----------------------------------------------------------------------------------------------------------

2% of peak computational performance is used (0.78 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.10.1.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 38.50 to 28.50 cycles (1.35x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.10.1.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is probably not vectorized.
Only 16% of vector register length is used (average across all SSE/AVX instructions).
By vectorizing your loop, you can lower the cost of an iteration from 38.50 to 19.25 cycles (2.00x speedup).

Details
Store and arithmetical SSE/AVX instructions are used in scalar version (process only one data element in vector registers).
Since your execution units are vector units, only a vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.10.1.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 38.50 to 29.83 cycles (1.29x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.10.1.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 38.50 to 29.17 cycles (1.32x speedup).


      6.1.10.1.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.10.1.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 1 occurrences<<list_path_1_complex_1>>



      6.1.10.1.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 14 occurrence(s)
 - Irregular (variable stride) or indirect: 10 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.10.1.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_1_cvt_1>>
 - CQTO: 1 occurrences<<list_path_1_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.10.1.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

48 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.10.1.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 30 FP arithmetical operations:
 - 13: addition or subtraction
 - 10: multiply
 - 6: divide
 - 1: square root
The binary loop is loading 432 bytes (54 double precision FP elements).


      6.1.10.1.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.07 FP operations per loaded or stored byte.




      6.1.10.2  -  Path 2
  ----------------------------------------------------------------------------------------------------------

Warnings:
Detected a function call instruction: ignoring called function instructions.
Rerun with --follow-calls=append to include them to analysis  or with --follow-calls=inline to simulate inlining.
2% of peak computational performance is used (0.85 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.10.2.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 34.00 to 25.67 cycles (1.32x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.10.2.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 16% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 34.00 to 17.00 cycles (2.00x speedup).

Details
33% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 6% of SSE/AVX loads are used in vector version.
 - 25% of SSE/AVX stores are used in vector version.
 - 0% of SSE/AVX addition or subtraction instructions are used in vector version.
 - 0% of SSE/AVX multiply instructions are used in vector version.
 - 0% of SSE/AVX divide and square root instructions are used in vector version.
 - 59% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.10.2.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.10.2.4  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.10.2.5  -  CALL instructions
  ----------------------------------------------------------------------------------------------------------

Detected function call instructions.


Details
Calling (and then returning from) a function prevents many compiler optimizations (like vectorization), breaks control flow (which reduces pipeline performance) and executes extra instructions to save/restore the registers used inside it, which is very expensive (dozens of cycles). Consider to inline small functions.
 - unknown: 1 occurrences<<list_path_2_call_1>>



      6.1.10.2.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - IDIV: 1 occurrences<<list_path_2_complex_1>>



      6.1.10.2.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 14 occurrence(s)
 - Irregular (variable stride) or indirect: 10 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.10.2.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_2_cvt_1>>
 - CQTO: 1 occurrences<<list_path_2_cvt_2>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.10.2.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

47 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.10.2.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 29 FP arithmetical operations:
 - 13: addition or subtraction
 - 10: multiply
 - 6: divide
The binary loop is loading 528 bytes (66 double precision FP elements).
The binary loop is storing 40 bytes (5 double precision FP elements).


      6.1.10.2.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.


      6.1.10.2.12  -  Unroll opportunity
  ----------------------------------------------------------------------------------------------------------

Loop is potentially data access bound.

Workaround
Unroll your loop if trip count is significantly higher than target unroll factor and if some data references are common to consecutive iterations. This can be done manually. Or with the unroll (resp. unroll_and_jam) directive on top of the inner (resp. surrounding) loop. You can enforce an unroll factor: #pragma unroll_and_jam N, unroll_and_jam(N), unroll N or unroll(N)




      6.1.10.3  -  Path 3
  ----------------------------------------------------------------------------------------------------------

2% of peak computational performance is used (0.87 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.10.3.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 34.50 to 28.50 cycles (1.21x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.10.3.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is probably not vectorized.
Only 16% of vector register length is used (average across all SSE/AVX instructions).
By vectorizing your loop, you can lower the cost of an iteration from 34.50 to 15.75 cycles (2.19x speedup).

Details
Store and arithmetical SSE/AVX instructions are used in scalar version (process only one data element in vector registers).
Since your execution units are vector units, only a vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.10.3.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Performance is limited by execution of divide and square root operations (the divide/square root unit is a bottleneck).

By removing all these bottlenecks, you can lower the cost of an iteration from 34.50 to 29.50 cycles (1.17x speedup).


Workaround
 - Reduce the number of division or square root instructions:
  * If denominator is constant over iterations, use reciprocal (replace x/y with x*(1/y)). Check precision impact. This will be done by your compiler with ffast-math or Ofast
 - Check whether you really need double precision. If not, switch to single precision to speedup execution





      6.1.10.3.4  -  Expensive FP math instructions/calls
  ----------------------------------------------------------------------------------------------------------

Detected performance impact from expensive FP math instructions/calls.
By removing/reexpressing them, you can lower the cost of an iteration from 34.50 to 29.00 cycles (1.19x speedup).


      6.1.10.3.5  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.10.3.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_3_complex_1>>



      6.1.10.3.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 14 occurrence(s)
 - Irregular (variable stride) or indirect: 10 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.10.3.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_3_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.10.3.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

48 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.10.3.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 30 FP arithmetical operations:
 - 13: addition or subtraction
 - 10: multiply
 - 6: divide
 - 1: square root
The binary loop is loading 432 bytes (54 double precision FP elements).


      6.1.10.3.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.07 FP operations per loaded or stored byte.


      6.1.10.3.12  -  Unroll opportunity
  ----------------------------------------------------------------------------------------------------------

Loop is potentially data access bound.

Workaround
Unroll your loop if trip count is significantly higher than target unroll factor and if some data references are common to consecutive iterations. This can be done manually. Or with the unroll (resp. unroll_and_jam) directive on top of the inner (resp. surrounding) loop. You can enforce an unroll factor: #pragma unroll_and_jam N, unroll_and_jam(N), unroll N or unroll(N)




      6.1.10.4  -  Path 4
  ----------------------------------------------------------------------------------------------------------

Warnings:
Detected a function call instruction: ignoring called function instructions.
Rerun with --follow-calls=append to include them to analysis  or with --follow-calls=inline to simulate inlining.
2% of peak computational performance is used (0.88 out of 32.00 FLOP per cycle (GFLOPS @ 1GHz))

      6.1.10.4.1  -  Code clean check
  ----------------------------------------------------------------------------------------------------------

Detected a slowdown caused by scalar integer instructions (typically used for address computation).
By removing them, you can lower the cost of an iteration from 33.00 to 25.67 cycles (1.29x speedup).

Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)



      6.1.10.4.2  -  Vectorization
  ----------------------------------------------------------------------------------------------------------

Your loop is poorly vectorized.
Only 16% of vector register length is used (average across all SSE/AVX instructions).
By fully vectorizing your loop, you can lower the cost of an iteration from 33.00 to 13.50 cycles (2.44x speedup).

Details
34% of SSE/AVX instructions are used in vector version (process two or more data elements in vector registers):
 - 6% of SSE/AVX loads are used in vector version.
 - 25% of SSE/AVX stores are used in vector version.
 - 0% of SSE/AVX addition or subtraction instructions are used in vector version.
 - 0% of SSE/AVX multiply instructions are used in vector version.
 - 0% of SSE/AVX divide and square root instructions are used in vector version.
 - 60% of SSE/AVX instructions that are not load, store, addition, subtraction nor multiply instructions are used in vector version.
Since your execution units are vector units, only a fully vectorized loop can use their full power.


Workaround
 - Try another compiler or update/tune your current one:
  * recompile with ffast-math (included in Ofast) to extend loop vectorization to FP reductions.
 - Remove inter-iterations dependences from your loop and make it unit-stride:
  * If your arrays have 2 or more dimensions, check whether elements are accessed contiguously and, otherwise, try to permute loops accordingly:
C storage order is row-major: for(i) for(j) a[j][i] = b[j][i]; (slow, non stride 1) => for(i) for(j) a[i][j] = b[i][j]; (fast, stride 1)<<image_row_maj>>
  * If your loop streams arrays of structures (AoS), try to use structures of arrays instead (SoA):
for(i) a[i].x = b[i].x; (slow, non stride 1) => for(i) a.x[i] = b.x[i]; (fast, stride 1)



      6.1.10.4.3  -  Execution units bottlenecks
  ----------------------------------------------------------------------------------------------------------

Found no such bottlenecks but see expert reports for more complex bottlenecks.




      6.1.10.4.4  -  FMA
  ----------------------------------------------------------------------------------------------------------

Presence of both ADD/SUB and MUL operations.

Workaround
 - Pass to your compiler a micro-architecture specialization option:
  * use march=native
 - Try to change order in which elements are evaluated (using parentheses) in arithmetic expressions containing both ADD/SUB and MUL operations to enable your compiler to generate FMA instructions wherever possible.
For instance a + b*c is a valid FMA (MUL then ADD).
However (a+b)* c cannot be translated into an FMA (ADD then MUL).





      6.1.10.4.5  -  CALL instructions
  ----------------------------------------------------------------------------------------------------------

Detected function call instructions.


Details
Calling (and then returning from) a function prevents many compiler optimizations (like vectorization), breaks control flow (which reduces pipeline performance) and executes extra instructions to save/restore the registers used inside it, which is very expensive (dozens of cycles). Consider to inline small functions.
 - unknown: 1 occurrences<<list_path_4_call_1>>



      6.1.10.4.6  -  Complex instructions
  ----------------------------------------------------------------------------------------------------------

Detected COMPLEX INSTRUCTIONS.


Details
These instructions generate more than one micro-operation and only one of them can be decoded during a cycle and the extra micro-operations increase pressure on execution units.
 - DIV: 1 occurrences<<list_path_4_complex_1>>



      6.1.10.4.7  -  Slow data structures access
  ----------------------------------------------------------------------------------------------------------

Detected data structures (typically arrays) that cannot be efficiently read/written

Details
 - Constant unknown stride: 14 occurrence(s)
 - Irregular (variable stride) or indirect: 10 occurrence(s)
Non-unit stride (uncontiguous) accesses are not efficiently using data caches


Workaround
 - Try to reorganize arrays of structures to structures of arrays
 - Consider to permute loops (see vectorization gain report)
 - Try to remove indirect accesses. If applicable, precompute elements out of the innermost loop.



      6.1.10.4.8  -  Conversion instructions
  ----------------------------------------------------------------------------------------------------------

Detected expensive conversion instructions.

Details
 - CLTQ: 1 occurrences<<list_path_4_cvt_1>>


Workaround
Avoid mixing data with different types. In particular, check if the type of constants is the same as array elements.


      6.1.10.4.9  -  Type of elements and instruction set
  ----------------------------------------------------------------------------------------------------------

47 SSE or AVX instructions are processing arithmetic or math operations on double precision FP elements in scalar mode (one at a time).



      6.1.10.4.10  -  Matching between your loop (in the source code) and the binary loop
  ----------------------------------------------------------------------------------------------------------

The binary loop is composed of 29 FP arithmetical operations:
 - 13: addition or subtraction
 - 10: multiply
 - 6: divide
The binary loop is loading 528 bytes (66 double precision FP elements).
The binary loop is storing 40 bytes (5 double precision FP elements).


      6.1.10.4.11  -  Arithmetic intensity
  ----------------------------------------------------------------------------------------------------------

Arithmetic intensity is 0.05 FP operations per loaded or stored byte.


      6.1.10.4.12  -  Unroll opportunity
  ----------------------------------------------------------------------------------------------------------

Loop is potentially data access bound.

Workaround
Unroll your loop if trip count is significantly higher than target unroll factor and if some data references are common to consecutive iterations. This can be done manually. Or with the unroll (resp. unroll_and_jam) directive on top of the inner (resp. surrounding) loop. You can enforce an unroll factor: #pragma unroll_and_jam N, unroll_and_jam(N), unroll N or unroll(N)





[MAQAO] Info: STOP THE REPORT GENERATION
[MAQAO] Info: 
[MAQAO] Info: If your application produces files, they can be found in directory "/beegfs/hackathon/users/eoseret/qaas_runs_test/178-661-4073/intel/CloverLeaf2.0-CXX/run/oneview_runs/defaults/aocc/oneview_run_1786614528"
[MAQAO] Info: 
