* Info: Detected 2 Lprof instances in ins01.benchmarkcenter.megware.com: processes-per-node/ppn set accordingly.
If this is incorrect, rerun with an explicit value for this setting
* Info: Selecting the 'perf-high-ppn' engine for node ins01.benchmarkcenter.megware.com
* Info: "ref-cycles" not supported on ins01.benchmarkcenter.megware.com: fallback to "cpu-clock"
* Warning: Found no event able to derive walltime: prepending cpu-clock
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 123839)
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 123843)Wed Feb 21 16:46:52 2024: Starting Initialization
Mini-Application Name : CoMD-openmp-mpi
Mini-Application Version : 1.1
Platform:
hostname: gmz16.benchmarkcenter.megware.com
kernel name: 'Linux'
kernel release: '5.14.0-362.13.1.el9_3.x86_64'
processor: 'x86_64'
Build:
CC: '/cluster/intel/oneapi/2024.0.0/mpi/2021.11/bin/mpiicc'
compiler version: 'unknown'
CFLAGS: '-O3 -march=native -DDO_MPI -O3 -march=haswell -fno-tree-vectorize -fno-openmp-simd -funroll-loops -ffast-math -g -grecord-gcc-switches -fno-omit-frame-pointer -fcf-protection=none -no-pie -cc=gcc -fopenmp -funroll-loops'
LDFLAGS: ' '
using MPI: true
Threading: OpenMP (1 threads)
Double Precision: true
Run Date/Time: 2024-02-21, 16:46:52
Command Line Parameters:
doeam: 0
potDir: pots
potName: Cu_u6.eam
potType: funcfl
nx: 100
ny: 100
nz: 100
xproc: 2
yproc: 1
zproc: 1
Lattice constant: -1 Angstroms
nSteps: 100
printRate: 10
Time step: 1 fs
Initial Temperature: 600 K
Initial Delta: 0 Angstroms
Simulation data:
Total atoms : 4000000
Min global bounds : [ 0.0000000000, 0.0000000000, 0.0000000000 ]
Max global bounds : [ 361.5000000000, 361.5000000000, 361.5000000000 ]
Decomposition data:
Processors : 2, 1, 1
Local boxes : 31, 62, 62 = 119164
Box size : [ 5.8306451613, 5.8306451613, 5.8306451613 ]
Box factor : [ 1.0074548875, 1.0074548875, 1.0074548875 ]
Max Link Cell Occupancy: 32 of 64
Potential data:
Potential type : Lennard-Jones
Species name : Cu
Atomic number : 29
Mass : 63.55 amu
Lattice Type : FCC
Lattice spacing : 3.615 Angstroms
Cutoff : 5.7875 Angstroms
Epsilon : 0.167 eV
Sigma : 2.315 Angstroms
Initial energy : -1.166063303588, atom count : 4000000
Wed Feb 21 16:46:54 2024: Initialization Finished
Wed Feb 21 16:46:54 2024: Starting simulation
# Performance
# Loop Time(fs) Total Energy Potential Energy Kinetic Energy Temperature (us/atom) # Atoms
0 0.00 -1.166063303588 -1.243619295188 0.077555991600 600.0000 0.0000 4000000
10 10.00 -1.166059650499 -1.233157709948 0.067098059449 519.0938 0.9948 4000000
20 20.00 -1.166048438417 -1.208183014318 0.042134575902 325.9677 1.3394 4000000
30 30.00 -1.166037590737 -1.186586197151 0.020548606414 158.9711 1.4567 4000000
40 40.00 -1.166042093135 -1.183625399859 0.017583306724 136.0305 1.4931 4000000
50 50.00 -1.166051684893 -1.193713710258 0.027662025365 214.0030 1.5033 4000000
60 60.00 -1.166054646931 -1.202662201513 0.036607554582 283.2087 1.5027 4000000
70 70.00 -1.166052143011 -1.204911990844 0.038859847833 300.6332 1.4977 4000000
80 80.00 -1.166048803912 -1.203635015020 0.037586211108 290.7799 1.4893 4000000
90 90.00 -1.166048006780 -1.203820491598 0.037772484818 292.2210 1.4821 4000000
100 100.00 -1.166049793505 -1.206862845061 0.040813051556 315.7439 1.4749 4000000
Wed Feb 21 16:51:39 2024: Ending simulation
Simulation Validation:
Initial energy : -1.166063303588
Final energy : -1.166049793505
eFinal/eInitial : 0.999988
Final atom count : 4000000, no atoms lost
Timings for Rank 0
Timer # Calls Avg/Call (s) Total (s) % Loop
___________________________________________________________________
total 1 286.8737 286.8737 100.77
loop 1 284.6838 284.6838 100.00
timestep 10 28.4683 284.6832 100.00
position 100 0.0188 1.8794 0.66
velocity 200 0.0173 3.4692 1.22
redistribute 101 0.0913 9.2244 3.24
atomHalo 101 0.0096 0.9671 0.34
force 101 2.6900 271.6894 95.44
commHalo 303 0.0005 0.1487 0.05
commReduce 39 0.0001 0.0024 0.00
Timing Statistics Across 2 Ranks:
Timer Rank: Min(s) Rank: Max(s) Avg(s) Stdev(s)
_____________________________________________________________________________
total 0: 286.8737 1: 286.8739 286.8738 0.0001
loop 0: 284.6838 1: 284.6838 284.6838 0.0000
timestep 0: 284.6832 1: 284.6834 284.6833 0.0001
position 1: 1.8543 0: 1.8794 1.8669 0.0125
velocity 1: 3.4367 0: 3.4692 3.4530 0.0162
redistribute 0: 9.2244 1: 9.5460 9.3852 0.1608
atomHalo 0: 0.9671 1: 1.2915 1.1293 0.1622
force 1: 271.3855 0: 271.6894 271.5375 0.1520
commHalo 0: 0.1487 1: 0.4825 0.3156 0.1669
commReduce 0: 0.0024 1: 0.0454 0.0239 0.0215
---------------------------------------------------
Average atom update rate: 1.42 us/atom/task
---------------------------------------------------
---------------------------------------------------
Average all atom update rate: 0.71 us/atom
---------------------------------------------------
---------------------------------------------------
Average atom rate: 1.41 atoms/us
---------------------------------------------------
Wed Feb 21 16:51:39 2024: CoMD Ending
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 123839)
* Warning: (host ins01.benchmarkcenter.megware.com, process 123839) Observed more threads (2) than expected (1): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=2.
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 123843)
* Warning: (host ins01.benchmarkcenter.megware.com, process 123843) Observed more threads (2) than expected (1): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=2.
Your experiment path is /beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0
To display your profiling results:
##################################################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
##################################################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0 #
# Functions | Per-node | maqao lprof -df -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0 #
# Functions | Per-process | maqao lprof -df -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0 #
# Functions | Per-thread | maqao lprof -df -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0 #
# Loops | Cluster-wide | maqao lprof -dl xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0 #
# Loops | Per-node | maqao lprof -dl -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0 #
# Loops | Per-process | maqao lprof -dl -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_0 #
##################################################################################################################################################################################################
* Info: Detected 2 Lprof instances in ins01.benchmarkcenter.megware.com: processes-per-node/ppn set accordingly.
If this is incorrect, rerun with an explicit value for this setting
* Info: Selecting the 'perf-high-ppn' engine for node ins01.benchmarkcenter.megware.com
* Info: "ref-cycles" not supported on ins01.benchmarkcenter.megware.com: fallback to "cpu-clock"
* Warning: Found no event able to derive walltime: prepending cpu-clock
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 123985)
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 123986)Wed Feb 21 16:51:49 2024: Starting Initialization
Mini-Application Name : CoMD-openmp-mpi
Mini-Application Version : 1.1
Platform:
hostname: gmz16.benchmarkcenter.megware.com
kernel name: 'Linux'
kernel release: '5.14.0-362.13.1.el9_3.x86_64'
processor: 'x86_64'
Build:
CC: '/cluster/intel/oneapi/2024.0.0/mpi/2021.11/bin/mpiicc'
compiler version: 'unknown'
CFLAGS: '-O3 -march=native -DDO_MPI -O3 -march=haswell -fno-tree-vectorize -fno-openmp-simd -funroll-loops -ffast-math -g -grecord-gcc-switches -fno-omit-frame-pointer -fcf-protection=none -no-pie -cc=gcc -fopenmp -funroll-loops'
LDFLAGS: ' '
using MPI: true
Threading: OpenMP (2 threads)
Double Precision: true
Run Date/Time: 2024-02-21, 16:51:49
Command Line Parameters:
doeam: 0
potDir: pots
potName: Cu_u6.eam
potType: funcfl
nx: 100
ny: 100
nz: 100
xproc: 2
yproc: 1
zproc: 1
Lattice constant: -1 Angstroms
nSteps: 100
printRate: 10
Time step: 1 fs
Initial Temperature: 600 K
Initial Delta: 0 Angstroms
Simulation data:
Total atoms : 4000000
Min global bounds : [ 0.0000000000, 0.0000000000, 0.0000000000 ]
Max global bounds : [ 361.5000000000, 361.5000000000, 361.5000000000 ]
Decomposition data:
Processors : 2, 1, 1
Local boxes : 31, 62, 62 = 119164
Box size : [ 5.8306451613, 5.8306451613, 5.8306451613 ]
Box factor : [ 1.0074548875, 1.0074548875, 1.0074548875 ]
Max Link Cell Occupancy: 32 of 64
Potential data:
Potential type : Lennard-Jones
Species name : Cu
Atomic number : 29
Mass : 63.55 amu
Lattice Type : FCC
Lattice spacing : 3.615 Angstroms
Cutoff : 5.7875 Angstroms
Epsilon : 0.167 eV
Sigma : 2.315 Angstroms
Initial energy : -1.166063304092, atom count : 4000000
Wed Feb 21 16:51:50 2024: Initialization Finished
Wed Feb 21 16:51:50 2024: Starting simulation
# Performance
# Loop Time(fs) Total Energy Potential Energy Kinetic Energy Temperature (us/atom) # Atoms
0 0.00 -1.166063304092 -1.243619295692 0.077555991600 600.0000 0.0000 4000000
10 10.00 -1.166059650500 -1.233157709949 0.067098059449 519.0938 0.5077 4000000
20 20.00 -1.166048438416 -1.208183014318 0.042134575902 325.9677 0.6793 4000000
30 30.00 -1.166037590737 -1.186586197151 0.020548606414 158.9711 0.7385 4000000
40 40.00 -1.166042093134 -1.183625399859 0.017583306724 136.0305 0.7569 4000000
50 50.00 -1.166051684893 -1.193713710258 0.027662025365 214.0030 0.7617 4000000
60 60.00 -1.166054646931 -1.202662201513 0.036607554582 283.2087 0.7611 4000000
70 70.00 -1.166052143011 -1.204911990844 0.038859847833 300.6332 0.7589 4000000
80 80.00 -1.166048803912 -1.203635015020 0.037586211108 290.7799 0.7553 4000000
90 90.00 -1.166048006780 -1.203820491599 0.037772484818 292.2210 0.7518 4000000
100 100.00 -1.166049793504 -1.206862845060 0.040813051556 315.7439 0.7470 4000000
Wed Feb 21 16:54:15 2024: Ending simulation
Simulation Validation:
Initial energy : -1.166063304092
Final energy : -1.166049793504
eFinal/eInitial : 0.999988
Final atom count : 4000000, no atoms lost
Timings for Rank 0
Timer # Calls Avg/Call (s) Total (s) % Loop
___________________________________________________________________
total 1 145.5634 145.5634 100.83
loop 1 144.3647 144.3647 100.00
timestep 10 14.4364 144.3641 100.00
position 100 0.0090 0.9021 0.62
velocity 200 0.0085 1.7015 1.18
redistribute 101 0.0662 6.6827 4.63
atomHalo 101 0.0099 1.0016 0.69
force 101 1.3453 135.8771 94.12
commHalo 303 0.0006 0.1833 0.13
commReduce 39 0.0004 0.0149 0.01
Timing Statistics Across 2 Ranks:
Timer Rank: Min(s) Rank: Max(s) Avg(s) Stdev(s)
_____________________________________________________________________________
total 0: 145.5634 1: 145.5637 145.5636 0.0002
loop 0: 144.3647 1: 144.3647 144.3647 0.0000
timestep 0: 144.3641 1: 144.3643 144.3642 0.0001
position 1: 0.8960 0: 0.9021 0.8990 0.0031
velocity 0: 1.7015 1: 1.7040 1.7027 0.0013
redistribute 1: 6.6331 0: 6.6827 6.6579 0.0248
atomHalo 0: 1.0016 1: 1.0189 1.0103 0.0087
force 0: 135.8771 1: 135.9363 135.9067 0.0296
commHalo 0: 0.1833 1: 0.1984 0.1908 0.0076
commReduce 1: 0.0066 0: 0.0149 0.0107 0.0041
---------------------------------------------------
Average atom update rate: 0.72 us/atom/task
---------------------------------------------------
---------------------------------------------------
Average all atom update rate: 0.36 us/atom
---------------------------------------------------
---------------------------------------------------
Average atom rate: 2.77 atoms/us
---------------------------------------------------
Wed Feb 21 16:54:15 2024: CoMD Ending
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 123985)
* Warning: (host ins01.benchmarkcenter.megware.com, process 123985) Observed more threads (3) than expected (2): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=3.
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 123986)
* Warning: (host ins01.benchmarkcenter.megware.com, process 123986) Observed more threads (3) than expected (2): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=3.
Your experiment path is /beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1
To display your profiling results:
##################################################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
##################################################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1 #
# Functions | Per-node | maqao lprof -df -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1 #
# Functions | Per-process | maqao lprof -df -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1 #
# Functions | Per-thread | maqao lprof -df -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1 #
# Loops | Cluster-wide | maqao lprof -dl xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1 #
# Loops | Per-node | maqao lprof -dl -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1 #
# Loops | Per-process | maqao lprof -dl -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_1 #
##################################################################################################################################################################################################
* Info: Detected 2 Lprof instances in ins01.benchmarkcenter.megware.com: processes-per-node/ppn set accordingly.
If this is incorrect, rerun with an explicit value for this setting
* Info: Selecting the 'perf-high-ppn' engine for node ins01.benchmarkcenter.megware.com
* Info: "ref-cycles" not supported on ins01.benchmarkcenter.megware.com: fallback to "cpu-clock"
* Warning: Found no event able to derive walltime: prepending cpu-clock
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124102)
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124101)Wed Feb 21 16:54:26 2024: Starting Initialization
Mini-Application Name : CoMD-openmp-mpi
Mini-Application Version : 1.1
Platform:
hostname: gmz16.benchmarkcenter.megware.com
kernel name: 'Linux'
kernel release: '5.14.0-362.13.1.el9_3.x86_64'
processor: 'x86_64'
Build:
CC: '/cluster/intel/oneapi/2024.0.0/mpi/2021.11/bin/mpiicc'
compiler version: 'unknown'
CFLAGS: '-O3 -march=native -DDO_MPI -O3 -march=haswell -fno-tree-vectorize -fno-openmp-simd -funroll-loops -ffast-math -g -grecord-gcc-switches -fno-omit-frame-pointer -fcf-protection=none -no-pie -cc=gcc -fopenmp -funroll-loops'
LDFLAGS: ' '
using MPI: true
Threading: OpenMP (4 threads)
Double Precision: true
Run Date/Time: 2024-02-21, 16:54:26
Command Line Parameters:
doeam: 0
potDir: pots
potName: Cu_u6.eam
potType: funcfl
nx: 100
ny: 100
nz: 100
xproc: 2
yproc: 1
zproc: 1
Lattice constant: -1 Angstroms
nSteps: 100
printRate: 10
Time step: 1 fs
Initial Temperature: 600 K
Initial Delta: 0 Angstroms
Simulation data:
Total atoms : 4000000
Min global bounds : [ 0.0000000000, 0.0000000000, 0.0000000000 ]
Max global bounds : [ 361.5000000000, 361.5000000000, 361.5000000000 ]
Decomposition data:
Processors : 2, 1, 1
Local boxes : 31, 62, 62 = 119164
Box size : [ 5.8306451613, 5.8306451613, 5.8306451613 ]
Box factor : [ 1.0074548875, 1.0074548875, 1.0074548875 ]
Max Link Cell Occupancy: 32 of 64
Potential data:
Potential type : Lennard-Jones
Species name : Cu
Atomic number : 29
Mass : 63.55 amu
Lattice Type : FCC
Lattice spacing : 3.615 Angstroms
Cutoff : 5.7875 Angstroms
Epsilon : 0.167 eV
Sigma : 2.315 Angstroms
Initial energy : -1.166063303067, atom count : 4000000
Wed Feb 21 16:54:27 2024: Initialization Finished
Wed Feb 21 16:54:27 2024: Starting simulation
# Performance
# Loop Time(fs) Total Energy Potential Energy Kinetic Energy Temperature (us/atom) # Atoms
0 0.00 -1.166063303067 -1.243619294667 0.077555991600 600.0000 0.0000 4000000
10 10.00 -1.166059650499 -1.233157709949 0.067098059449 519.0938 0.2658 4000000
20 20.00 -1.166048438416 -1.208183014318 0.042134575902 325.9677 0.3507 4000000
30 30.00 -1.166037590737 -1.186586197151 0.020548606414 158.9711 0.3808 4000000
40 40.00 -1.166042093134 -1.183625399859 0.017583306724 136.0305 0.3895 4000000
50 50.00 -1.166051684893 -1.193713710257 0.027662025365 214.0030 0.3917 4000000
60 60.00 -1.166054646931 -1.202662201513 0.036607554582 283.2087 0.3915 4000000
70 70.00 -1.166052143011 -1.204911990845 0.038859847833 300.6332 0.3902 4000000
80 80.00 -1.166048803912 -1.203635015020 0.037586211108 290.7799 0.3883 4000000
90 90.00 -1.166048006780 -1.203820491599 0.037772484818 292.2210 0.3867 4000000
100 100.00 -1.166049793504 -1.206862845060 0.040813051556 315.7439 0.3844 4000000
Wed Feb 21 16:55:41 2024: Ending simulation
Simulation Validation:
Initial energy : -1.166063303067
Final energy : -1.166049793504
eFinal/eInitial : 0.999988
Final atom count : 4000000, no atoms lost
Timings for Rank 0
Timer # Calls Avg/Call (s) Total (s) % Loop
___________________________________________________________________
total 1 75.1050 75.1050 100.96
loop 1 74.3920 74.3920 100.00
timestep 10 7.4391 74.3914 100.00
position 100 0.0050 0.4979 0.67
velocity 200 0.0045 0.9075 1.22
redistribute 101 0.0509 5.1393 6.91
atomHalo 101 0.0096 0.9734 1.31
force 101 0.6760 68.2780 91.78
commHalo 303 0.0005 0.1556 0.21
commReduce 39 0.0002 0.0091 0.01
Timing Statistics Across 2 Ranks:
Timer Rank: Min(s) Rank: Max(s) Avg(s) Stdev(s)
_____________________________________________________________________________
total 0: 75.1050 1: 75.1054 75.1052 0.0002
loop 0: 74.3920 1: 74.3920 74.3920 0.0000
timestep 0: 74.3914 1: 74.3916 74.3915 0.0001
position 1: 0.4375 0: 0.4979 0.4677 0.0302
velocity 1: 0.8019 0: 0.9075 0.8547 0.0528
redistribute 0: 5.1393 1: 5.4131 5.2762 0.1369
atomHalo 0: 0.9734 1: 1.0962 1.0348 0.0614
force 1: 68.1677 0: 68.2780 68.2228 0.0551
commHalo 0: 0.1556 1: 0.2784 0.2170 0.0614
commReduce 0: 0.0091 1: 0.0166 0.0128 0.0038
---------------------------------------------------
Average atom update rate: 0.37 us/atom/task
---------------------------------------------------
---------------------------------------------------
Average all atom update rate: 0.19 us/atom
---------------------------------------------------
---------------------------------------------------
Average atom rate: 5.38 atoms/us
---------------------------------------------------
Wed Feb 21 16:55:41 2024: CoMD Ending
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124101)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124101) Observed more threads (5) than expected (4): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=5.
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124102)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124102) Observed more threads (5) than expected (4): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=5.
Your experiment path is /beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2
To display your profiling results:
##################################################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
##################################################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2 #
# Functions | Per-node | maqao lprof -df -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2 #
# Functions | Per-process | maqao lprof -df -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2 #
# Functions | Per-thread | maqao lprof -df -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2 #
# Loops | Cluster-wide | maqao lprof -dl xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2 #
# Loops | Per-node | maqao lprof -dl -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2 #
# Loops | Per-process | maqao lprof -dl -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_2 #
##################################################################################################################################################################################################
* Info: Detected 2 Lprof instances in ins01.benchmarkcenter.megware.com: processes-per-node/ppn set accordingly.
If this is incorrect, rerun with an explicit value for this setting
* Info: Selecting the 'perf-high-ppn' engine for node ins01.benchmarkcenter.megware.com
* Info: "ref-cycles" not supported on ins01.benchmarkcenter.megware.com: fallback to "cpu-clock"
* Warning: Found no event able to derive walltime: prepending cpu-clock
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124208)
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124209)Wed Feb 21 16:55:52 2024: Starting Initialization
Mini-Application Name : CoMD-openmp-mpi
Mini-Application Version : 1.1
Platform:
hostname: gmz16.benchmarkcenter.megware.com
kernel name: 'Linux'
kernel release: '5.14.0-362.13.1.el9_3.x86_64'
processor: 'x86_64'
Build:
CC: '/cluster/intel/oneapi/2024.0.0/mpi/2021.11/bin/mpiicc'
compiler version: 'unknown'
CFLAGS: '-O3 -march=native -DDO_MPI -O3 -march=haswell -fno-tree-vectorize -fno-openmp-simd -funroll-loops -ffast-math -g -grecord-gcc-switches -fno-omit-frame-pointer -fcf-protection=none -no-pie -cc=gcc -fopenmp -funroll-loops'
LDFLAGS: ' '
using MPI: true
Threading: OpenMP (8 threads)
Double Precision: true
Run Date/Time: 2024-02-21, 16:55:52
Command Line Parameters:
doeam: 0
potDir: pots
potName: Cu_u6.eam
potType: funcfl
nx: 100
ny: 100
nz: 100
xproc: 2
yproc: 1
zproc: 1
Lattice constant: -1 Angstroms
nSteps: 100
printRate: 10
Time step: 1 fs
Initial Temperature: 600 K
Initial Delta: 0 Angstroms
Simulation data:
Total atoms : 4000000
Min global bounds : [ 0.0000000000, 0.0000000000, 0.0000000000 ]
Max global bounds : [ 361.5000000000, 361.5000000000, 361.5000000000 ]
Decomposition data:
Processors : 2, 1, 1
Local boxes : 31, 62, 62 = 119164
Box size : [ 5.8306451613, 5.8306451613, 5.8306451613 ]
Box factor : [ 1.0074548875, 1.0074548875, 1.0074548875 ]
Max Link Cell Occupancy: 32 of 64
Potential data:
Potential type : Lennard-Jones
Species name : Cu
Atomic number : 29
Mass : 63.55 amu
Lattice Type : FCC
Lattice spacing : 3.615 Angstroms
Cutoff : 5.7875 Angstroms
Epsilon : 0.167 eV
Sigma : 2.315 Angstroms
Initial energy : -1.166063303663, atom count : 4000000
Wed Feb 21 16:55:52 2024: Initialization Finished
Wed Feb 21 16:55:52 2024: Starting simulation
# Performance
# Loop Time(fs) Total Energy Potential Energy Kinetic Energy Temperature (us/atom) # Atoms
0 0.00 -1.166063303663 -1.243619295263 0.077555991600 600.0000 0.0000 4000000
10 10.00 -1.166059650500 -1.233157709949 0.067098059449 519.0938 0.1503 4000000
20 20.00 -1.166048438416 -1.208183014318 0.042134575902 325.9677 0.1928 4000000
30 30.00 -1.166037590737 -1.186586197151 0.020548606414 158.9711 0.2060 4000000
40 40.00 -1.166042093134 -1.183625399859 0.017583306724 136.0305 0.2097 4000000
50 50.00 -1.166051684893 -1.193713710258 0.027662025365 214.0030 0.2115 4000000
60 60.00 -1.166054646931 -1.202662201513 0.036607554582 283.2087 0.2112 4000000
70 70.00 -1.166052143011 -1.204911990845 0.038859847833 300.6332 0.2102 4000000
80 80.00 -1.166048803912 -1.203635015020 0.037586211108 290.7799 0.2098 4000000
90 90.00 -1.166048006780 -1.203820491599 0.037772484818 292.2210 0.2087 4000000
100 100.00 -1.166049793504 -1.206862845060 0.040813051556 315.7439 0.2072 4000000
Wed Feb 21 16:56:33 2024: Ending simulation
Simulation Validation:
Initial energy : -1.166063303663
Final energy : -1.166049793504
eFinal/eInitial : 0.999988
Final atom count : 4000000, no atoms lost
Timings for Rank 0
Timer # Calls Avg/Call (s) Total (s) % Loop
___________________________________________________________________
total 1 40.8447 40.8447 101.23
loop 1 40.3479 40.3479 100.00
timestep 10 4.0347 40.3473 100.00
position 100 0.0032 0.3202 0.79
velocity 200 0.0030 0.5985 1.48
redistribute 101 0.0472 4.7711 11.82
atomHalo 101 0.0133 1.3479 3.34
force 101 0.3450 34.8429 86.36
commHalo 303 0.0017 0.5154 1.28
commReduce 39 0.0022 0.0840 0.21
Timing Statistics Across 2 Ranks:
Timer Rank: Min(s) Rank: Max(s) Avg(s) Stdev(s)
_____________________________________________________________________________
total 1: 40.8447 0: 40.8447 40.8447 0.0000
loop 0: 40.3479 1: 40.3479 40.3479 0.0000
timestep 0: 40.3473 1: 40.3475 40.3474 0.0001
position 1: 0.2376 0: 0.3202 0.2789 0.0413
velocity 1: 0.4418 0: 0.5985 0.5201 0.0783
redistribute 0: 4.7711 1: 4.9227 4.8469 0.0758
atomHalo 1: 1.0296 0: 1.3479 1.1888 0.1592
force 0: 34.8429 1: 34.9934 34.9182 0.0752
commHalo 1: 0.1729 0: 0.5154 0.3442 0.1713
commReduce 1: 0.0131 0: 0.0840 0.0485 0.0355
---------------------------------------------------
Average atom update rate: 0.20 us/atom/task
---------------------------------------------------
---------------------------------------------------
Average all atom update rate: 0.10 us/atom
---------------------------------------------------
---------------------------------------------------
Average atom rate: 9.91 atoms/us
---------------------------------------------------
Wed Feb 21 16:56:33 2024: CoMD Ending
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124208)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124208) Observed more threads (9) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=9.
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124209)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124209) Observed more threads (9) than expected (8): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=9.
Your experiment path is /beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3
To display your profiling results:
##################################################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
##################################################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3 #
# Functions | Per-node | maqao lprof -df -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3 #
# Functions | Per-process | maqao lprof -df -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3 #
# Functions | Per-thread | maqao lprof -df -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3 #
# Loops | Cluster-wide | maqao lprof -dl xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3 #
# Loops | Per-node | maqao lprof -dl -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3 #
# Loops | Per-process | maqao lprof -dl -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_3 #
##################################################################################################################################################################################################
* Info: Detected 2 Lprof instances in ins01.benchmarkcenter.megware.com: processes-per-node/ppn set accordingly.
If this is incorrect, rerun with an explicit value for this setting
* Info: Selecting the 'perf-high-ppn' engine for node ins01.benchmarkcenter.megware.com
* Info: "ref-cycles" not supported on ins01.benchmarkcenter.megware.com: fallback to "cpu-clock"
* Warning: Found no event able to derive walltime: prepending cpu-clock
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124300)
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124299)Wed Feb 21 16:56:44 2024: Starting Initialization
Mini-Application Name : CoMD-openmp-mpi
Mini-Application Version : 1.1
Platform:
hostname: gmz16.benchmarkcenter.megware.com
kernel name: 'Linux'
kernel release: '5.14.0-362.13.1.el9_3.x86_64'
processor: 'x86_64'
Build:
CC: '/cluster/intel/oneapi/2024.0.0/mpi/2021.11/bin/mpiicc'
compiler version: 'unknown'
CFLAGS: '-O3 -march=native -DDO_MPI -O3 -march=haswell -fno-tree-vectorize -fno-openmp-simd -funroll-loops -ffast-math -g -grecord-gcc-switches -fno-omit-frame-pointer -fcf-protection=none -no-pie -cc=gcc -fopenmp -funroll-loops'
LDFLAGS: ' '
using MPI: true
Threading: OpenMP (16 threads)
Double Precision: true
Run Date/Time: 2024-02-21, 16:56:44
Command Line Parameters:
doeam: 0
potDir: pots
potName: Cu_u6.eam
potType: funcfl
nx: 100
ny: 100
nz: 100
xproc: 2
yproc: 1
zproc: 1
Lattice constant: -1 Angstroms
nSteps: 100
printRate: 10
Time step: 1 fs
Initial Temperature: 600 K
Initial Delta: 0 Angstroms
Simulation data:
Total atoms : 4000000
Min global bounds : [ 0.0000000000, 0.0000000000, 0.0000000000 ]
Max global bounds : [ 361.5000000000, 361.5000000000, 361.5000000000 ]
Decomposition data:
Processors : 2, 1, 1
Local boxes : 31, 62, 62 = 119164
Box size : [ 5.8306451613, 5.8306451613, 5.8306451613 ]
Box factor : [ 1.0074548875, 1.0074548875, 1.0074548875 ]
Max Link Cell Occupancy: 32 of 64
Potential data:
Potential type : Lennard-Jones
Species name : Cu
Atomic number : 29
Mass : 63.55 amu
Lattice Type : FCC
Lattice spacing : 3.615 Angstroms
Cutoff : 5.7875 Angstroms
Epsilon : 0.167 eV
Sigma : 2.315 Angstroms
Initial energy : -1.166063303487, atom count : 4000000
Wed Feb 21 16:56:44 2024: Initialization Finished
Wed Feb 21 16:56:44 2024: Starting simulation
# Performance
# Loop Time(fs) Total Energy Potential Energy Kinetic Energy Temperature (us/atom) # Atoms
0 0.00 -1.166063303487 -1.243619295087 0.077555991600 600.0000 0.0000 4000000
10 10.00 -1.166059650500 -1.233157709949 0.067098059449 519.0938 0.0972 4000000
20 20.00 -1.166048438416 -1.208183014318 0.042134575902 325.9677 0.1147 4000000
30 30.00 -1.166037590737 -1.186586197151 0.020548606414 158.9711 0.1208 4000000
40 40.00 -1.166042093134 -1.183625399859 0.017583306724 136.0305 0.1231 4000000
50 50.00 -1.166051684893 -1.193713710257 0.027662025365 214.0030 0.1242 4000000
60 60.00 -1.166054646931 -1.202662201513 0.036607554582 283.2087 0.1234 4000000
70 70.00 -1.166052143011 -1.204911990844 0.038859847833 300.6332 0.1236 4000000
80 80.00 -1.166048803912 -1.203635015020 0.037586211108 290.7799 0.1230 4000000
90 90.00 -1.166048006780 -1.203820491599 0.037772484818 292.2210 0.1221 4000000
100 100.00 -1.166049793504 -1.206862845060 0.040813051556 315.7439 0.1217 4000000
Wed Feb 21 16:57:08 2024: Ending simulation
Simulation Validation:
Initial energy : -1.166063303487
Final energy : -1.166049793504
eFinal/eInitial : 0.999988
Final atom count : 4000000, no atoms lost
Timings for Rank 0
Timer # Calls Avg/Call (s) Total (s) % Loop
___________________________________________________________________
total 1 24.3152 24.3152 101.84
loop 1 23.8757 23.8757 100.00
timestep 10 2.3875 23.8750 100.00
position 100 0.0020 0.2041 0.85
velocity 200 0.0018 0.3600 1.51
redistribute 101 0.0470 4.7431 19.87
atomHalo 101 0.0099 1.0025 4.20
force 101 0.1860 18.7837 78.67
commHalo 303 0.0004 0.1152 0.48
commReduce 39 0.0000 0.0008 0.00
Timing Statistics Across 2 Ranks:
Timer Rank: Min(s) Rank: Max(s) Avg(s) Stdev(s)
_____________________________________________________________________________
total 0: 24.3152 1: 24.3152 24.3152 0.0000
loop 0: 23.8757 1: 23.8757 23.8757 0.0000
timestep 0: 23.8750 1: 23.8752 23.8751 0.0001
position 1: 0.1646 0: 0.2041 0.1844 0.0197
velocity 1: 0.3126 0: 0.3600 0.3363 0.0237
redistribute 0: 4.7431 1: 5.5652 5.1541 0.4110
atomHalo 0: 1.0025 1: 2.3530 1.6778 0.6752
force 1: 17.9064 0: 18.7837 18.3451 0.4386
commHalo 0: 0.1152 1: 1.5185 0.8168 0.7016
commReduce 0: 0.0008 1: 0.1529 0.0768 0.0760
---------------------------------------------------
Average atom update rate: 0.12 us/atom/task
---------------------------------------------------
---------------------------------------------------
Average all atom update rate: 0.06 us/atom
---------------------------------------------------
---------------------------------------------------
Average atom rate: 16.75 atoms/us
---------------------------------------------------
Wed Feb 21 16:57:08 2024: CoMD Ending
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124299)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124299) Observed more threads (17) than expected (16): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=17.
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124300)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124300) Observed more threads (17) than expected (16): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=17.
Your experiment path is /beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4
To display your profiling results:
##################################################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
##################################################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4 #
# Functions | Per-node | maqao lprof -df -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4 #
# Functions | Per-process | maqao lprof -df -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4 #
# Functions | Per-thread | maqao lprof -df -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4 #
# Loops | Cluster-wide | maqao lprof -dl xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4 #
# Loops | Per-node | maqao lprof -dl -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4 #
# Loops | Per-process | maqao lprof -dl -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_4 #
##################################################################################################################################################################################################
* Info: Detected 2 Lprof instances in ins01.benchmarkcenter.megware.com: processes-per-node/ppn set accordingly.
If this is incorrect, rerun with an explicit value for this setting
* Info: Selecting the 'perf-high-ppn' engine for node ins01.benchmarkcenter.megware.com
* Info: "ref-cycles" not supported on ins01.benchmarkcenter.megware.com: fallback to "cpu-clock"
* Warning: Found no event able to derive walltime: prepending cpu-clock
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124436)
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124437)Wed Feb 21 16:57:20 2024: Starting Initialization
Mini-Application Name : CoMD-openmp-mpi
Mini-Application Version : 1.1
Platform:
hostname: gmz16.benchmarkcenter.megware.com
kernel name: 'Linux'
kernel release: '5.14.0-362.13.1.el9_3.x86_64'
processor: 'x86_64'
Build:
CC: '/cluster/intel/oneapi/2024.0.0/mpi/2021.11/bin/mpiicc'
compiler version: 'unknown'
CFLAGS: '-O3 -march=native -DDO_MPI -O3 -march=haswell -fno-tree-vectorize -fno-openmp-simd -funroll-loops -ffast-math -g -grecord-gcc-switches -fno-omit-frame-pointer -fcf-protection=none -no-pie -cc=gcc -fopenmp -funroll-loops'
LDFLAGS: ' '
using MPI: true
Threading: OpenMP (32 threads)
Double Precision: true
Run Date/Time: 2024-02-21, 16:57:20
Command Line Parameters:
doeam: 0
potDir: pots
potName: Cu_u6.eam
potType: funcfl
nx: 100
ny: 100
nz: 100
xproc: 2
yproc: 1
zproc: 1
Lattice constant: -1 Angstroms
nSteps: 100
printRate: 10
Time step: 1 fs
Initial Temperature: 600 K
Initial Delta: 0 Angstroms
Simulation data:
Total atoms : 4000000
Min global bounds : [ 0.0000000000, 0.0000000000, 0.0000000000 ]
Max global bounds : [ 361.5000000000, 361.5000000000, 361.5000000000 ]
Decomposition data:
Processors : 2, 1, 1
Local boxes : 31, 62, 62 = 119164
Box size : [ 5.8306451613, 5.8306451613, 5.8306451613 ]
Box factor : [ 1.0074548875, 1.0074548875, 1.0074548875 ]
Max Link Cell Occupancy: 32 of 64
Potential data:
Potential type : Lennard-Jones
Species name : Cu
Atomic number : 29
Mass : 63.55 amu
Lattice Type : FCC
Lattice spacing : 3.615 Angstroms
Cutoff : 5.7875 Angstroms
Epsilon : 0.167 eV
Sigma : 2.315 Angstroms
Initial energy : -1.166063303522, atom count : 4000000
Wed Feb 21 16:57:21 2024: Initialization Finished
Wed Feb 21 16:57:21 2024: Starting simulation
# Performance
# Loop Time(fs) Total Energy Potential Energy Kinetic Energy Temperature (us/atom) # Atoms
0 0.00 -1.166063303522 -1.243619295122 0.077555991600 600.0000 0.0000 4000000
10 10.00 -1.166059650500 -1.233157709949 0.067098059449 519.0938 0.0796 4000000
20 20.00 -1.166048438416 -1.208183014318 0.042134575902 325.9677 0.0872 4000000
30 30.00 -1.166037590737 -1.186586197151 0.020548606414 158.9711 0.0900 4000000
40 40.00 -1.166042093134 -1.183625399859 0.017583306724 136.0305 0.0913 4000000
50 50.00 -1.166051684893 -1.193713710258 0.027662025365 214.0030 0.0915 4000000
60 60.00 -1.166054646931 -1.202662201513 0.036607554582 283.2087 0.0818 4000000
70 70.00 -1.166052143011 -1.204911990844 0.038859847833 300.6332 0.0795 4000000
80 80.00 -1.166048803912 -1.203635015020 0.037586211108 290.7799 0.0790 4000000
90 90.00 -1.166048006780 -1.203820491599 0.037772484818 292.2210 0.0790 4000000
100 100.00 -1.166049793504 -1.206862845060 0.040813051556 315.7439 0.0808 4000000
Wed Feb 21 16:57:37 2024: Ending simulation
Simulation Validation:
Initial energy : -1.166063303522
Final energy : -1.166049793504
eFinal/eInitial : 0.999988
Final atom count : 4000000, no atoms lost
Timings for Rank 0
Timer # Calls Avg/Call (s) Total (s) % Loop
___________________________________________________________________
total 1 17.1728 17.1728 102.25
loop 1 16.7942 16.7942 100.00
timestep 10 1.6793 16.7933 100.00
position 100 0.0014 0.1359 0.81
velocity 200 0.0013 0.2547 1.52
redistribute 101 0.0491 4.9624 29.55
atomHalo 101 0.0122 1.2341 7.35
force 101 0.1147 11.5889 69.01
commHalo 303 0.0004 0.1161 0.69
commReduce 39 0.0000 0.0015 0.01
Timing Statistics Across 2 Ranks:
Timer Rank: Min(s) Rank: Max(s) Avg(s) Stdev(s)
_____________________________________________________________________________
total 0: 17.1728 1: 17.1732 17.1730 0.0002
loop 1: 16.7942 0: 16.7942 16.7942 0.0000
timestep 0: 16.7933 1: 16.7935 16.7934 0.0001
position 1: 0.1104 0: 0.1359 0.1231 0.0128
velocity 1: 0.1986 0: 0.2547 0.2267 0.0280
redistribute 0: 4.9624 1: 6.0195 5.4909 0.5286
atomHalo 0: 1.2341 1: 3.0750 2.1545 0.9205
force 1: 10.4443 0: 11.5889 11.0166 0.5723
commHalo 0: 0.1161 1: 2.2107 1.1634 1.0473
commReduce 0: 0.0015 1: 0.1882 0.0949 0.0934
---------------------------------------------------
Average atom update rate: 0.08 us/atom/task
---------------------------------------------------
---------------------------------------------------
Average all atom update rate: 0.04 us/atom
---------------------------------------------------
---------------------------------------------------
Average atom rate: 23.82 atoms/us
---------------------------------------------------
Wed Feb 21 16:57:37 2024: CoMD Ending
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124436)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124436) Observed more threads (33) than expected (32): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=33.
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124437)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124437) Observed more threads (33) than expected (32): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=33.
Your experiment path is /beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5
To display your profiling results:
##################################################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
##################################################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5 #
# Functions | Per-node | maqao lprof -df -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5 #
# Functions | Per-process | maqao lprof -df -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5 #
# Functions | Per-thread | maqao lprof -df -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5 #
# Loops | Cluster-wide | maqao lprof -dl xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5 #
# Loops | Per-node | maqao lprof -dl -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5 #
# Loops | Per-process | maqao lprof -dl -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_5 #
##################################################################################################################################################################################################
* Info: Detected 2 Lprof instances in ins01.benchmarkcenter.megware.com: processes-per-node/ppn set accordingly.
If this is incorrect, rerun with an explicit value for this setting
* Info: Selecting the 'perf-high-ppn' engine for node ins01.benchmarkcenter.megware.com
* Info: "ref-cycles" not supported on ins01.benchmarkcenter.megware.com: fallback to "cpu-clock"
* Warning: Found no event able to derive walltime: prepending cpu-clock
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124612)
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124613)Wed Feb 21 16:57:48 2024: Starting Initialization
Mini-Application Name : CoMD-openmp-mpi
Mini-Application Version : 1.1
Platform:
hostname: gmz16.benchmarkcenter.megware.com
kernel name: 'Linux'
kernel release: '5.14.0-362.13.1.el9_3.x86_64'
processor: 'x86_64'
Build:
CC: '/cluster/intel/oneapi/2024.0.0/mpi/2021.11/bin/mpiicc'
compiler version: 'unknown'
CFLAGS: '-O3 -march=native -DDO_MPI -O3 -march=haswell -fno-tree-vectorize -fno-openmp-simd -funroll-loops -ffast-math -g -grecord-gcc-switches -fno-omit-frame-pointer -fcf-protection=none -no-pie -cc=gcc -fopenmp -funroll-loops'
LDFLAGS: ' '
using MPI: true
Threading: OpenMP (64 threads)
Double Precision: true
Run Date/Time: 2024-02-21, 16:57:48
Command Line Parameters:
doeam: 0
potDir: pots
potName: Cu_u6.eam
potType: funcfl
nx: 100
ny: 100
nz: 100
xproc: 2
yproc: 1
zproc: 1
Lattice constant: -1 Angstroms
nSteps: 100
printRate: 10
Time step: 1 fs
Initial Temperature: 600 K
Initial Delta: 0 Angstroms
Simulation data:
Total atoms : 4000000
Min global bounds : [ 0.0000000000, 0.0000000000, 0.0000000000 ]
Max global bounds : [ 361.5000000000, 361.5000000000, 361.5000000000 ]
Decomposition data:
Processors : 2, 1, 1
Local boxes : 31, 62, 62 = 119164
Box size : [ 5.8306451613, 5.8306451613, 5.8306451613 ]
Box factor : [ 1.0074548875, 1.0074548875, 1.0074548875 ]
Max Link Cell Occupancy: 32 of 64
Potential data:
Potential type : Lennard-Jones
Species name : Cu
Atomic number : 29
Mass : 63.55 amu
Lattice Type : FCC
Lattice spacing : 3.615 Angstroms
Cutoff : 5.7875 Angstroms
Epsilon : 0.167 eV
Sigma : 2.315 Angstroms
Initial energy : -1.166063303459, atom count : 4000000
Wed Feb 21 16:57:49 2024: Initialization Finished
Wed Feb 21 16:57:49 2024: Starting simulation
# Performance
# Loop Time(fs) Total Energy Potential Energy Kinetic Energy Temperature (us/atom) # Atoms
0 0.00 -1.166063303459 -1.243619295059 0.077555991600 600.0000 0.0000 4000000
10 10.00 -1.166059650500 -1.233157709949 0.067098059449 519.0938 0.0685 4000000
20 20.00 -1.166048438416 -1.208183014318 0.042134575902 325.9677 0.0655 4000000
30 30.00 -1.166037590737 -1.186586197151 0.020548606414 158.9711 0.0652 4000000
40 40.00 -1.166042093134 -1.183625399859 0.017583306724 136.0305 0.0659 4000000
50 50.00 -1.166051684893 -1.193713710258 0.027662025365 214.0030 0.0665 4000000
60 60.00 -1.166054646931 -1.202662201513 0.036607554582 283.2087 0.0665 4000000
70 70.00 -1.166052143011 -1.204911990844 0.038859847833 300.6332 0.0664 4000000
80 80.00 -1.166048803912 -1.203635015020 0.037586211108 290.7799 0.0663 4000000
90 90.00 -1.166048006780 -1.203820491599 0.037772484818 292.2210 0.0661 4000000
100 100.00 -1.166049793504 -1.206862845060 0.040813051556 315.7439 0.0660 4000000
Wed Feb 21 16:58:02 2024: Ending simulation
Simulation Validation:
Initial energy : -1.166063303459
Final energy : -1.166049793504
eFinal/eInitial : 0.999988
Final atom count : 4000000, no atoms lost
Timings for Rank 0
Timer # Calls Avg/Call (s) Total (s) % Loop
___________________________________________________________________
total 1 13.6531 13.6531 102.98
loop 1 13.2586 13.2586 100.00
timestep 10 1.3256 13.2565 99.98
position 100 0.0020 0.1995 1.50
velocity 200 0.0016 0.3283 2.48
redistribute 101 0.0526 5.3103 40.05
atomHalo 101 0.0118 1.1905 8.98
force 101 0.0746 7.5381 56.85
commHalo 303 0.0004 0.1363 1.03
commReduce 39 0.0000 0.0012 0.01
Timing Statistics Across 2 Ranks:
Timer Rank: Min(s) Rank: Max(s) Avg(s) Stdev(s)
_____________________________________________________________________________
total 0: 13.6531 1: 13.6534 13.6532 0.0001
loop 0: 13.2586 1: 13.2586 13.2586 0.0000
timestep 0: 13.2565 1: 13.2567 13.2566 0.0001
position 1: 0.1132 0: 0.1995 0.1563 0.0432
velocity 1: 0.2772 0: 0.3283 0.3027 0.0255
redistribute 0: 5.3103 1: 5.7273 5.5188 0.2085
atomHalo 0: 1.1905 1: 2.8077 1.9991 0.8086
force 1: 7.1949 0: 7.5381 7.3665 0.1716
commHalo 0: 0.1363 1: 1.9458 1.0411 0.9047
commReduce 0: 0.0012 1: 0.0818 0.0415 0.0403
---------------------------------------------------
Average atom update rate: 0.07 us/atom/task
---------------------------------------------------
---------------------------------------------------
Average all atom update rate: 0.03 us/atom
---------------------------------------------------
---------------------------------------------------
Average atom rate: 30.17 atoms/us
---------------------------------------------------
Wed Feb 21 16:58:02 2024: CoMD Ending
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124612)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124612) Observed more threads (65) than expected (64): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=65.
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124613)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124613) Observed more threads (65) than expected (64): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=65.
Your experiment path is /beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6
To display your profiling results:
##################################################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
##################################################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6 #
# Functions | Per-node | maqao lprof -df -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6 #
# Functions | Per-process | maqao lprof -df -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6 #
# Functions | Per-thread | maqao lprof -df -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6 #
# Loops | Cluster-wide | maqao lprof -dl xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6 #
# Loops | Per-node | maqao lprof -dl -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6 #
# Loops | Per-process | maqao lprof -dl -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_6 #
##################################################################################################################################################################################################
* Info: Detected 2 Lprof instances in ins01.benchmarkcenter.megware.com: processes-per-node/ppn set accordingly.
If this is incorrect, rerun with an explicit value for this setting
* Info: Selecting the 'perf-high-ppn' engine for node ins01.benchmarkcenter.megware.com
* Info: "ref-cycles" not supported on ins01.benchmarkcenter.megware.com: fallback to "cpu-clock"
* Warning: Found no event able to derive walltime: prepending cpu-clock
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124917)
* Info: Process launched (host ins01.benchmarkcenter.megware.com, process 124918)Wed Feb 21 16:58:14 2024: Starting Initialization
Mini-Application Name : CoMD-openmp-mpi
Mini-Application Version : 1.1
Platform:
hostname: gmz16.benchmarkcenter.megware.com
kernel name: 'Linux'
kernel release: '5.14.0-362.13.1.el9_3.x86_64'
processor: 'x86_64'
Build:
CC: '/cluster/intel/oneapi/2024.0.0/mpi/2021.11/bin/mpiicc'
compiler version: 'unknown'
CFLAGS: '-O3 -march=native -DDO_MPI -O3 -march=haswell -fno-tree-vectorize -fno-openmp-simd -funroll-loops -ffast-math -g -grecord-gcc-switches -fno-omit-frame-pointer -fcf-protection=none -no-pie -cc=gcc -fopenmp -funroll-loops'
LDFLAGS: ' '
using MPI: true
Threading: OpenMP (96 threads)
Double Precision: true
Run Date/Time: 2024-02-21, 16:58:14
Command Line Parameters:
doeam: 0
potDir: pots
potName: Cu_u6.eam
potType: funcfl
nx: 100
ny: 100
nz: 100
xproc: 2
yproc: 1
zproc: 1
Lattice constant: -1 Angstroms
nSteps: 100
printRate: 10
Time step: 1 fs
Initial Temperature: 600 K
Initial Delta: 0 Angstroms
Simulation data:
Total atoms : 4000000
Min global bounds : [ 0.0000000000, 0.0000000000, 0.0000000000 ]
Max global bounds : [ 361.5000000000, 361.5000000000, 361.5000000000 ]
Decomposition data:
Processors : 2, 1, 1
Local boxes : 31, 62, 62 = 119164
Box size : [ 5.8306451613, 5.8306451613, 5.8306451613 ]
Box factor : [ 1.0074548875, 1.0074548875, 1.0074548875 ]
Max Link Cell Occupancy: 32 of 64
Potential data:
Potential type : Lennard-Jones
Species name : Cu
Atomic number : 29
Mass : 63.55 amu
Lattice Type : FCC
Lattice spacing : 3.615 Angstroms
Cutoff : 5.7875 Angstroms
Epsilon : 0.167 eV
Sigma : 2.315 Angstroms
Initial energy : -1.166063303456, atom count : 4000000
Wed Feb 21 16:58:15 2024: Initialization Finished
Wed Feb 21 16:58:15 2024: Starting simulation
# Performance
# Loop Time(fs) Total Energy Potential Energy Kinetic Energy Temperature (us/atom) # Atoms
0 0.00 -1.166063303456 -1.243619295056 0.077555991600 600.0000 0.0000 4000000
10 10.00 -1.166059650500 -1.233157709949 0.067098059449 519.0938 0.0588 4000000
20 20.00 -1.166048438416 -1.208183014318 0.042134575902 325.9677 0.0572 4000000
30 30.00 -1.166037590737 -1.186586197151 0.020548606414 158.9711 0.0575 4000000
40 40.00 -1.166042093134 -1.183625399859 0.017583306724 136.0305 0.0587 4000000
50 50.00 -1.166051684893 -1.193713710258 0.027662025365 214.0030 0.0579 4000000
60 60.00 -1.166054646931 -1.202662201513 0.036607554582 283.2087 0.0587 4000000
70 70.00 -1.166052143011 -1.204911990844 0.038859847833 300.6332 0.0637 4000000
80 80.00 -1.166048803912 -1.203635015020 0.037586211108 290.7799 0.0621 4000000
90 90.00 -1.166048006780 -1.203820491599 0.037772484818 292.2210 0.0606 4000000
100 100.00 -1.166049793504 -1.206862845060 0.040813051556 315.7439 0.0634 4000000
Wed Feb 21 16:58:27 2024: Ending simulation
Simulation Validation:
Initial energy : -1.166063303456
Final energy : -1.166049793504
eFinal/eInitial : 0.999988
Final atom count : 4000000, no atoms lost
Timings for Rank 0
Timer # Calls Avg/Call (s) Total (s) % Loop
___________________________________________________________________
total 1 12.4030 12.4030 103.59
loop 1 11.9736 11.9736 100.00
timestep 10 1.1972 11.9716 99.98
position 100 0.0013 0.1350 1.13
velocity 200 0.0012 0.2307 1.93
redistribute 101 0.0601 6.0693 50.69
atomHalo 101 0.0161 1.6250 13.57
force 101 0.0544 5.4962 45.90
commHalo 303 0.0017 0.5221 4.36
commReduce 39 0.0037 0.1428 1.19
Timing Statistics Across 2 Ranks:
Timer Rank: Min(s) Rank: Max(s) Avg(s) Stdev(s)
_____________________________________________________________________________
total 0: 12.4030 1: 12.4032 12.4031 0.0001
loop 0: 11.9736 1: 11.9736 11.9736 0.0000
timestep 0: 11.9716 1: 11.9726 11.9721 0.0005
position 0: 0.1350 1: 0.2377 0.1863 0.0514
velocity 0: 0.2307 1: 0.6796 0.4551 0.2245
redistribute 1: 5.6048 0: 6.0693 5.8370 0.2323
atomHalo 0: 1.6250 1: 2.6720 2.1485 0.5235
force 1: 5.4676 0: 5.4962 5.4819 0.0143
commHalo 0: 0.5221 1: 1.7976 1.1599 0.6378
commReduce 1: 0.0637 0: 0.1428 0.1032 0.0396
---------------------------------------------------
Average atom update rate: 0.06 us/atom/task
---------------------------------------------------
---------------------------------------------------
Average all atom update rate: 0.03 us/atom
---------------------------------------------------
---------------------------------------------------
Average atom rate: 33.41 atoms/us
---------------------------------------------------
Wed Feb 21 16:58:27 2024: CoMD Ending
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124917)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124917) Observed more threads (97) than expected (96): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=97.
* Info: Process finished (host ins01.benchmarkcenter.megware.com, process 124918)
* Warning: (host ins01.benchmarkcenter.megware.com, process 124918) Observed more threads (97) than expected (96): in case of high IO overhead or suspicious profile, rerun with maximum-threads-per-process=97.
Your experiment path is /beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7
To display your profiling results:
##################################################################################################################################################################################################
# LEVEL | REPORT | COMMAND #
##################################################################################################################################################################################################
# Functions | Cluster-wide | maqao lprof -df xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7 #
# Functions | Per-node | maqao lprof -df -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7 #
# Functions | Per-process | maqao lprof -df -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7 #
# Functions | Per-thread | maqao lprof -df -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7 #
# Loops | Cluster-wide | maqao lprof -dl xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7 #
# Loops | Per-node | maqao lprof -dl -dn xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7 #
# Loops | Per-process | maqao lprof -dl -dp xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7 #
# Loops | Per-thread | maqao lprof -dl -dt xp=/beegfs/hackathon/users/eoseret/qaas_runs/CoMD/intel/CoMD/run/oneview_runs/compilers/gcc_4/oneview_results_scal/tools/lprof_npsu_run_7 #
##################################################################################################################################################################################################
