Hi,
I am trying to attempt a run of mr weather application with C96 resolution , 32 ranks and 4 threads per rank. here is the contents of env_mach_specific.xml -
<environment_variables>
<env name="OMP_STACKSIZE">256M</env>
<env name="OMP_NUM_THREADS">4</env>
</environment_variables>
This is what i get with ./pelayout -
Comp NTASKS NTHRDS ROOTPE
ATM : 32/ 4; 0
output of ./preview_run
CASE INFO:
nodes: 1
total tasks: 32
tasks per node: 32
thread count: 4
BATCH INFO:
FOR JOB: case.chgres
ENV:
Setting Environment OMP_STACKSIZE=256M
Setting Environment OMP_NUM_THREADS=4
Setting Environment OMP_NUM_THREADS=4
SUBMIT CMD:
sbatch .case.chgres
FOR JOB: case.run
ENV:
Setting Environment OMP_STACKSIZE=256M
Setting Environment OMP_NUM_THREADS=4
Setting Environment OMP_NUM_THREADS=4
SUBMIT CMD:
sbatch --dependency=afterok:0 .case.run --resubmit
MPIRUN (job=case.run):
mpirun -np 32 -prepend-rank /work/puneet/MyWork/Milan_UFStry01/1.1.0_intel/src3/run3/DORIAN_C96_GFSv15p2_2.2.6.8.1/bld/ufs.exe >> ufs.log.$LID 2>&1
FOR JOB: case.gfs_post
.....
had also added the export OMP_NUM_THREADS=4 in my .bashrc.
i monitored the compute node for the entire duration of run. I am able to see 32 MPI ranks, but however they did not spawn threads -
556144 user1 20 0 9499396 621120 42344 R 100.0 0.2 6:57.41 ufs.exe
556147 user1 20 0 9486968 622796 42424 R 100.0 0.2 6:57.55 ufs.exe
556134 user1 20 0 9505012 633764 41388 R 99.7 0.2 6:57.45 ufs.exe
556135 user1 20 0 9499528 621256 41420 R 99.7 0.2 6:57.41 ufs.exe
556136 user1 20 0 9487352 621628 41240 R 99.7 0.2 6:57.43 ufs.exe
556137 user1 20 0 9500096 619976 41708 R 99.7 0.2 6:57.44 ufs.exe
556138 user1 20 0 9611392 737904 42640 R 99.7 0.3 6:57.41 ufs.exe
556139 user1 20 0 9602168 723396 42452 R 99.7 0.3 6:57.47 ufs.exe
556140 user1 20 0 9602180 722400 41208 R 99.7 0.3 6:57.44 ufs.exe
556142 user1 20 0 9518612 636016 42804 R 99.7 0.2 6:57.39 ufs.exe
556143 user1 20 0 9500148 622756 42264 R 99.7 0.2 6:57.43 ufs.exe
556145 user1 20 0 9500692 622196 42320 R 99.7 0.2 6:57.44 ufs.exe
556148 user1 20 0 9496480 621540 43032 R 99.7 0.2 6:57.56 ufs.exe
556149 user1 20 0 9492664 620948 42792 R 99.7 0.2 6:57.58 ufs.exe
556150 user1 20 0 9621468 738080 43456 R 99.7 0.3 6:57.54 ufs.exe
556151 user1 20 0 9608276 725308 42400 R 99.7 0.3 6:57.59 ufs.exe
556152 user1 20 0 9607844 723220 42756 R 99.7 0.3 6:57.60 ufs.exe
556154 user1 20 0 9908108 960328 31276 R 99.7 0.4 6:58.31 ufs.exe
556155 user1 20 0 9893060 953720 30352 R 99.7 0.4 6:58.35 ufs.exe
556156 user1 20 0 9893060 953660 30108 R 99.7 0.4 6:58.32 ufs.exe
556157 user1 20 0 9884676 951744 28972 R 99.7 0.4 6:58.34 ufs.exe
556158 user1 20 0 9892040 951760 29216 R 99.7 0.4 6:58.50 ufs.exe
556159 user1 20 0 9887844 950892 28988 R 99.7 0.4 6:58.50 ufs.exe
556160 user1 20 0 9892044 951116 29208 R 99.7 0.4 6:58.50 ufs.exe
556161 user1 20 0 9887844 950340 29024 R 99.7 0.4 6:58.33 ufs.exe
556130 user1 20 0 9539632 637796 42152 R 99.3 0.2 6:57.45 ufs.exe
556131 user1 20 0 9511956 621072 41996 R 99.3 0.2 6:57.42 ufs.exe
556132 user1 20 0 9495644 620852 40912 R 99.3 0.2 6:57.39 ufs.exe
556133 user1 20 0 9521516 621592 41740 R 99.3 0.2 6:57.54 ufs.exe
556141 user1 20 0 9603692 728268 41904 R 99.3 0.3 6:56.86 ufs.exe
556146 user1 20 0 9506264 635568 43396 R 99.3 0.2 6:57.61 ufs.exe
556153 user1 20 0 9607984 724736 43104 R 99.3 0.3 6:57.57 ufs.exe
556280 user1 20 0 68852 5856 3728 R 0.7 0.0 0:02.65 top
Is there something which i need to set in config files to make this hybrid run to work?
please advice.
- 34 views