Numerical simulations of shallow clouds by SCALE‐LES3

Yousuke Sato, Seiya Nishizawa, Hisashi Yashiro, Yoshiaki Miyamoto, Hirofumi Tomita, and Team‐SCALE

Development of SCALE‐LES3 (SCALE‐LES ver. 3) and benchmark tests

SCALE（Scalable Computing for Advanced Library and Environment）Library for weather/climate research for peta‐scale or post peta‐scale computer

1:What is SCALE‐LES?

2

Co‐design with researchers of computational scicnce

Develop team in RIKEN AICS

System Software Research Team

System Software Research Team

Programming Environment Research Team

Programming Environment Research Team

Computational Climate Science Research Team

Computational Climate Science Research Team

HPC Programing Framework Research Team

HPC Programing Framework Research Team

HPC Usability Research TeamHPC Usability Research Team

Performanceabout 10% of peak FLOPS @K computer99.9% of weak scaling

SCALE memberH. Tomita S. Nishizawa H. Yashiro Y. Miyamoto Y. Sato

Weather/climate teamComputational science team

SCALE‐LES• Fluid system : 3D fully compressible • Numerical solving method

• Included component– SGS model: Smagorinsky‐Lilly– Cloud physics: 2‐moment bulk, 1‐moment Bin– Radiation: MXTRN‐X– Surface flux : Monin‐Obukhov’s similarity

2. Model: numerical method

Grid system Arakawa-C

Temporal descretization 3rd order Runge-Kutta (HE-VE)

Advection for momentum 4th order

Advection for ρ 4th order

Pressure gradient force 2nd order central

Numerical diffusion 4th orderSchematic illustration of grid system

Benchmark tests are mostly finished and first version will published soon

3. Benchmark test

• Cold bubble (S. Nishizawa)– Test of dynamical frame : only dynamics module

• Dry turbulence (S. Nishizawa and Y. Miyamoto)– Test of dynamical core and sub‐grid model : dynamics + SGS

• DYCOMS‐II RF01 (Stratocumulus) (Y.Sato)– Test of SGS and cloud physics without rain

• RICO (Shallow cumulus) (Y.Sato)– Test of cloud physics with rain

Cold‐bubble Experiment(Nishizawa et al. in preparation)

Experimental setup is based on Strakaet al. (1993)• Dynamical core only• Domain size: 51.2km x 6.4km (2D)• Resolution: 25m• Cold bubble: ‐15 K (max)

– location : x=25.6km, z=3km– size :4km x 2km

• No diffusion except for numerical diffusion 10‐3

Straka et al. (1993)

(Nishizawa et al. 2013 in preparation) 

Potential temperature at initial time

X

zDeviation of θ from mean value at t =900s 

SCALE‐LES3

Dry  turbulence  (Miyamoto  et  al.  2012)  

9.6  km  

2.4  km  5.0  m  s-‐1  

9.6  km  

300 304 308 3120

1

2

3

(a) and qv

z (k

m)

(K)

−1 0 1 2

0

1

2

3qv (g kg

−1)

qv

−4 −2 0 2 4 6 8 100

1

2

3 (b) u and v

velocity (m s−1)

z (k

m)

uv

Experimental  setup  is  Exactly  same  as  Tanaka  et  al.  (2008)  CalculaDon  Dme  :  6  hour  (dt=0.03s)  

x (km)

z (k

m)

(AR0001a) t=0021300(s)

−4−3−2−1 0 1 2 3 40

0.5

1

1.5

2

304

305

306

307

308

x (km)

y (k

m)

(AR0001a) t=0021300(s)

−4−3−2−1 0 1 2 3 4−4−3−2−1

01234

−1

−0.5

0

0.5

1

Horizontally  averaged  iniDal  profile DeviaDon  of  θ  from  horizontal  average  at  t=21300  [s]

dx=dy=dz=30mFlux  with  diurnal  cycle

0 0.5 10

0.5

1

1.5 (a) buoyancy flux

w / Fss

z/h

t=4ht=5ht=6h

0 0.1 0.2 0.3 0.40

0.5

1

1.5 (b) w variance

z/h

w w / w*

−0.5 0 0.5 1 1.5 20

0.5

1

1.5 (c) w skewness

z/h

w w w / (w w )3/2298 300 302 304 306 3080

0.5

1

1.5

2

2.5 (d)

z (k

m)

< > (K)

AR0001a [ z=30, U=5.0, Fsm=200]

0 0.5 10

0.5

1

1.5 (a) buoyancy flux

w / Fss

z/h

t=4ht=5ht=6h

0 0.1 0.2 0.3 0.40

0.5

1

1.5 (b) w variance

z/h

w w / w*

−0.5 0 0.5 1 1.5 20

0.5

1

1.5 (c) w skewness

z/h

w w w / (w w )3/2298 300 302 304 306 3080

0.5

1

1.5

2

2.5 (d) z

(km

)

< > (K)

AR0001a [ z=30, U=5.0, Fsm=200]

0 0.5 10

0.5

1

1.5 (a) buoyancy flux

w / Fss

z/h

t=4ht=5ht=6h

0 0.1 0.2 0.3 0.40

0.5

1

1.5 (b) w variance

z/h

w w / w*

−0.5 0 0.5 1 1.5 20

0.5

1

1.5 (c) w skewnessz/

h

w w w / (w w )3/2298 300 302 304 306 3080

0.5

1

1.5

2

2.5 (d)

z (k

m)

< > (K)

AR0001a [ z=30, U=5.0, Fsm=200]

buoyancy  flux

w’  variance

w’  skewness

SCALE Tanaka  et  al.  (2008)

DYCOMS‐II RF01 case (Stratocumulus without rain)

0

200

400

600

800

1000

1200

-20 0 20 40 60 80 100 120 140 16

He

igh

t[m

]

w’qt’[W /m2]

GCSS max minGCSS std

GCSS meanSCALE

0

200

400

600

800

1000

1200

-60 -50 -40 -30 -20 -10 0 10 20

He

ight

[m]

w’theta’[W /m2]

GCSS max minGCSS std

GCSS meanSCALE

0

200

400

600

800

1000

1200

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

He

ign

t[m

]

sgs TKE [m2/s2]

GCSS max minGCSS std

GCSS meanSCALE

0

200

400

600

800

1000

1200

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

He

igh

t[m

]

Grid resolved TKE [m2/s2]

GCSS max minGCSS meanGCSS mean

SCALE

0

200

400

600

800

1000

1200

-0.15 -0.1 -0.05 0 0.05 0.1 0.1

He

ign

t[m

]

Third moment of w’[m3/s3]

GCSS max minGCSS std

GCSS meanSCALE

0

200

400

600

800

1000

1200

0 0.1 0.2 0.3 0.4 0.5 0.6

He

igh

t[m

]

Variance of w’[m/s]

GCSS max minGCSS std

GCSS meanSCALE

0

200

400

600

800

1000

1200

0 0.05 0.1 0.15 0.2 0.25 0.3 0.35 0.4

He

ign

t[m

]

Liquid water mixing ratio [g/kg]

GCSS max minGCSS std

GCSS meanSCALE

0

200

400

600

800

1000

1200

1 2 3 4 5 6 7 8 9 10 11

He

ign

t[m

]

Total water mixing ratio [g/kg]

GCSS max minGCSS std

GCSS meanSCALE

GCSS(max-min)

GCSS(sigma)

GCSS(mean)

SCALE(SS)

Total watermixing ratio

Liquid watermixing ratio

Variance of w'

GR TKE SGS TKE w'θl' w'q'

Third moment of w'

Experimental setup is based on Stevens et al. (2005)• Domain size: 3.36km x 3.36km x 1.5 km(3D)• Resolution: dx=dy=35, dz=5m• Calculation time : 4 hour (dt=0.006s)• Cloud physics : 2‐moment bulk (Seiki and 

Nakajima, 2013) [without rain and sedimentation]• Radiation : Parameterization of Stevens et al. 

(2005)• Surface flux : Constant value• Numerical diffusion : 10‐6

810

820

830

840

850

860

870

880

890

0 50 100 150 200

Zi[m

]

Time [min]

GCSS max minGCSS std

GCSS meanSCALE

0

200

400

600

800

1000

1200

0 50 100 150 200

Ve

rtic

ally

inte

gra

ted

TK

E(S

GS

+G

R)[

m2

/s2

]

Time [min]

GCSS max minGCSS std

GCSS meanSCALE

0

10

20

30

40

50

60

70

80

0 50 100 150 200

Liq

uid

wa

ter

pa

th[g

/m2

]

Time[min]

GCSS max minGCSS std

GCSS meanSCALE

Tim

e ev

olut

ion

LWP Zi

Vertically integratedTKE

Cloud water mixing ratio at t=3.5h

© Tema‐SCALEvisualized by R. Yoshida

Time evolution 

Hourly averaged profile (last 1 hour)

Summary• SCALE‐LES ver. 3.1 is now developing at RIKEN AICS• Several benchmark tests indicate that SCALE‐LES 3.1 shows good performance

Announce• The SCALE‐LES ver. 3.1 will published from Web site of RIKEN soon.

• All of you can use the SCALE-LES by free (BSD 2 license)

Using K computer of Exa‐SCALE computer, experiments with extremely fine grid spacing will be conducted. ⇒ Sensitivity experiment of grid spacing is required before product run!!

4: Sensitivity of grid spacing and aspect ratio of grid

• Cold bubble• Dry Turbulence• RF01

From now the detailed description will be skipped because of time limitation….

If you have any question, please contact me after this session!! The detailed discussion will be shown at JMS Spring meeting by S. Nishizawa (D403), and Y.Sato (B460)

0 0.5 10

0.5

1

1.5 (a) buoyancy flux

/ Fss

z/h

AR0001aAR0002aAR0005aAR0010a

0 0.1 0.2 0.3 0.40

0.5

1

1.5 (b) w variance

z/h

/ w*2

0 1 20

0.5

1

1.5 (c) w skewness

z/h

/ 3/2298 300 302 304 306 3080

0.5

1

1.5

2

2.5 (d)

z (k

m)

(K)

t = 6 h [z=30, U=5.0, Fsm=200]

AR=5,10: w’w’ and w’w’w’ are different from those in AR=1,2=> the turbulent fields are different from that which should be simulated in the current setting

Dry turbulence

Stratocumulus (RF01)

Variance of w’ [m2/s2]

variance of w’

double mode ⇒decoupling

dz＼dx 70m 35m 15m 5m

5m 14 (H70V5) 7(H35V5) 3(H15V5) 1(H5V5) Red line : GCSS meanShade :GCSS range

Z [m

]

Decoupling should not occur from observation⇒ Aspect ratio = 3 (dx=15m, dz=5m) is required!

AR14 (H70V5)

AR7(H35V5)

AR3(H15V5)

AR1(H5V5)

Summary

• SCALE‐LES ver. 3.1 is now developing at RIKEN AICS• Several benchmark tests indicate that SCALE‐LES 3.1 

shows good performance• Aspect ratio should be set to 1〜3• The SCALE‐LES ver. 3.1 will published from Web site of 

RIKEN near future.• All of you can use the SCALE‐LES by free (BSD 2 license)• Effect of grid spacing is now investigating and detailed 

discussion will be published at JMS Spring meeting!