Hi,
I am able to run the C96 resolution with 16,32,64 and 128 MPI ranks with 8,4,2,1 threads respectively. I have 256 GB RAM on the servers.
with both C192 and C384 i get -
[0] Rayleigh friction E-folding time (days):
[0] 1 0.379150775374218 10.8096140887264
[0] 2 0.963871677296582 16.5818681310322
[0] 3 1.76542623475949 29.0560344329854
[0] 4 2.67225797307616 53.4602794273955
[0] 5 3.70625064534251 110.544757762893
[0] 6 4.88725381108638 293.610300159162
[0] 7 6.23670999273840 1568.14167700556
[1]
[1] FATAL from PE 1: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3118657) from all PEs.
[1]
[3]
[3] FATAL from PE 3: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3110209) from all PEs.
[3]
[5]
[5] FATAL from PE 5: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3118657) from all PEs.
[5]
[6]
[6] FATAL from PE 6: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3118657) from all PEs.
[6]
[7]
[7] FATAL from PE 7: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3110209) from all PEs.
[7]
[8]
[8] FATAL from PE 8: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3110209) from all PEs.
[8]
[9]
[9] FATAL from PE 9: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3118657) from all PEs.
[9]
[10]
[10] FATAL from PE 10: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3118657) from all PEs.
[10]
[11]
[11] FATAL from PE 11: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3110209) from all PEs.
[11]
[0]
[0] FATAL from PE 0: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3110209) from all PEs.
[0]
[0]
[0] FATAL from PE 0: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3110209) from all PEs.
[0]
[4]
[4] FATAL from PE 4: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3110209) from all PEs.
[4]
[6] Abort(1) on node 6 (rank 6 in comm 0): application called MPI_Abort(MPI_COMM_WORLD, 1) - process 6
for C192 -
...
[1] FATAL from PE 1: set_group_update: mpp_domains_stack overflow, call mpp_domains_set_stack_size( 3253249) from all PEs.
...
if increasing the domains_stack_size with mrweather application could resolve this, the how could i specify that ? and are there any recommendations for this value for C192, C384 and C768?
- 56 views