Validation of operational NWP forecasts for global, diffuse and direct

solar exposure over Australia

www.bom.gov.au

Lawrie Rikus, Paul Gregory, Zhian Sun,

Tomas Glowacki

Bureau of Meteorology Research Branch,

15 June 2015

Motivation: why am I here?

The Background: Model evaluation Need to compare model variables with observational data not included as

input to DA.

Surface solar radiation is an essential variable for the model forecast process

NWP solar radiation forecasts are potentially a basis for solar power forecasts

Solar power stations could be a source of additional validation data

The Question: How well do the raw NWP surface solar radiation

fields agree with the observations? Compare raw NWP fields with the Bureau’s surface solar measurements

Hourly accumulations available from all operational models

Limitations: Full radiation calculation is done at most each hour

Cloud fixed over the hour

Solar zenith angle corrected at each time step

ACCESS-NWP (APS1 - Domains)

APS0: • Operational 2Q2010 • N144 global (~80 km) • ACCESS-R 40 km • ACCESS-A 11 km • ACCESS-C 5 km • L60

The ACCESS NWP Systems

Australian Community Climate Earth-System Simulator

Based on MetOffice Unified Model and 4DVar data assimilation system

APS1: • Implemented 3Q2013 • N320 global (~40 km) • ACCESS-R 11 km • ACCESS-C 4 km • L70

APS2: • Implemented around now • N512 global (~40 km) • ACCESS-R 11 km • ACCESS-C 1.5 km • L70

http://www.bom.gov.au/climate/data/oneminsolar/stations.shtml

Station name Start End Years

Adelaide 1994 open 15

Alice Springs 1993 open 21

Broome 1996 open 18

Cairns 1997 2004 6

Cape Grim 1998 open 16

Cobar 2012 2014 1

Cocos Island 2004 open 9

Darwin 1993 open 20

Geraldton Airport 2012 2014 1

Geraldton Airport

Comparison 1996 2006 9

Kalgoorlie-Boulder 1998 2013 9

Learmonth 1996 2013 10

Longreach Aero 2012 open 1

Melbourne Airport 1999 open 15

Mildura 1996 2013 10

Mount Gambier 1993 2006 11

Rockhampton Aero 1996 open 19

Tennant Creek Airport 1996 2006 10

Townsville Aero 2012 2014 1

Wagga Wagga 1997 open 18

Woomera 2012 2013 1

The Validation sites

1 minute high quality data available

The relationship between the observations and

the model domains

4 domains have one long-term site

• DN

• BN

• AD

• SY

VT has 3 long-term sites

PH has no long-term sites

Documentation

o Legacy (pre-ACCESS) 12 km model o Gregory, P. A., L. J. Rikus, and J. D. Kepert, 2012: Testing and Diagnosing the Ability of the

Bureau of Meteorology’s Numerical Weather Prediction Systems to Support Prediction of Solar

Energy Production. J. Appl. Meteor. Climatol, 51, 1577–1601.

oAPS0 12km model (ACCESS-A) o Gregory, P. A. and L. J. Rikus: Validation of Bureau of Meteorology’s Global, Diffuse and Direct

Solar Exposure Forecasts using the ACCESS Numerical Weather Prediction Systems, submitted

to J. Appl. Meteor. Climatol

The 1-minute site data were aggregated into the relevant hour spanned by the

model’s forecasts

Hourly accumulated global, direct and diffuse solar irradiance at the surface

processed

Forecast metrics

•Solar variability is predominantly due to cloud cover and solar

position •Variation in solar position is completely deterministic

•Variation in cloud cover is mostly stochastic

•Need to de-couple these two factors, otherwise you inflate the skill of

the NWP model. •A clear sky model (Ineichen and Perez (2002)) was used to normalise forecast

and observed data.

•Standard statistical metrics used for validation •RMSE, correlation, multiplicative bias

•Metrics developed by Espinar et al. (2009) •Integrate the absolute difference between the observed and forecast empirical

cumulative distribution functions (CDFs)

Standard forecast metrics

Paul Gregory developed the scripts to implement the validation process for ACCESS We can now apply them easily to the model archive for any period (since late 2013).

Validation of ACCESS-A hourly data

Global exposure Diffuse exposure Direct exposure

Bias MAE RAE (%) Bias MAE RAE (%) Bias MAE RAE (%)

All sky 1.01 0.28 16.38 0.95 0.21 46.05 1.03 0.44 33.54

Clear sky 1.00 0.16 8.11 1.01 0.13 40.11 1.00 0.26 14.96

Low cloud 0.88 0.05 18.89 0.84 0.05 23.17 4.55 0.01 80.21

Hourly results for January 2012 at Adelaide

Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2

Global Bias 1.01 1.00 0.99 0.99 0.97 0.97 1.00 0.98 0.93 0.93 1.00 0.99 0.99 0.98 1.01 1.00

RMSE 0.23 0.25 0.28 0.25 0.28 0.28 0.34 0.35 0.39 0.39 0.33 0.29 0.24 0.27 0.27 0.27

Correlation 0.62 0.62 0.56 0.58 0.59 0.59 0.58 0.46 0.61 0.59 0.58 0.63 0.59 0.55 0.64 0.66

KSI 14.31 11.65 31.61 27.71 59.19 67.59 12.58 13.97 93.98 80.40 9.79 13.23 21.57 24.15 15.00 22.67

OVER 0.00 0.00 0.25 0.00 21.29 38.36 0.00 0.00 66.48 51.96 0.00 0.00 0.00 0.00 0.00 0.00

Direct Bias 1.02 1.01 1.02 1.02 0.97 0.96 0.93 0.94 0.87 0.88 0.97 0.91 1.00 0.97 1.05 1.02

RMSE 0.39 0.40 0.42 0.41 0.46 0.48 0.63 0.57 0.58 0.56 0.50 0.63 0.41 0.46 0.43 0.44

Correlation 0.64 0.65 0.59 0.59 0.57 0.56 0.53 0.51 0.64 0.62 0.56 0.50 0.60 0.56 0.65 0.63

KSI 23.58 24.60 34.28 31.97 47.91 62.14 35.86 35.30 150.10 125.10 15.35 49.43 26.51 31.95 40.25 24.84

OVER 0.00 0.00 8.18 9.67 16.54 23.85 0.00 0.00 135.50 106.90 0.00 0.00 0.00 1.92 0.00 0.00

Diffuse Bias 0.91 0.94 0.82 0.84 0.93 0.94 1.26 1.23 1.22 1.18 1.12 1.28 0.94 1.00 0.82 0.89

RMSE 0.21 0.21 0.20 0.21 0.19 0.19 0.34 0.29 0.23 0.22 0.30 0.42 0.20 0.23 0.20 0.21

Correlation 0.45 0.45 0.39 0.37 0.41 0.41 0.27 0.41 0.47 0.47 0.28 0.18 0.41 0.39 0.46 0.39

KSI 102.20 108.10 237.90 384.40 164.90 191.80 78.38 71.33 168.80 129.30 100.10 131.80 134.90 182.90 138.40 177.40

OVER 65.66 78.52 213.30 365.30 164.90 191.80 52.71 40.33 147.90 113.00 67.48 121.90 119.50 162.20 109.20 153.50

Wagga WaggaAdelaide Alice Springs Broome Cape Grim Darwin Melbourne Rockhampton

Annual ACCESS-A Clear Sky Results

Overall day 1 better than day 2 except for Darwin, Cape Grim

Correlation ~ 0.6 for global and direct

< 0.5 for diffuse

Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2 Day 1 Day 2

Global Bias 1.02 1.01 0.97 0.97 0.96 0.96 1.03 1.01 0.91 0.92 0.97 0.97 1.00 0.98 1.01 1.00

RMSE 0.50 0.50 0.47 0.48 0.52 0.49 0.50 0.55 0.60 0.62 0.48 0.54 0.44 0.49 0.43 0.47

Correlation 0.62 0.60 0.66 0.62 0.61 0.63 0.64 0.56 0.59 0.55 0.67 0.60 0.74 0.68 0.73 0.70

KSI 26.51 19.82 56.44 59.19 84.05 86.11 38.45 18.54 181.60 157.20 39.18 34.81 17.96 34.72 12.80 15.58

OVER 0.00 0.00 4.35 13.81 59.18 62.25 0.76 0.00 176.40 150.90 0.00 0.00 0.00 0.00 0.00 0.00

Direct Bias 0.99 0.98 0.98 0.97 0.94 0.93 0.90 0.88 0.79 0.83 0.84 0.84 0.94 0.91 1.02 1.01

RMSE 0.65 0.68 0.63 0.66 0.71 0.69 0.62 0.66 0.78 0.78 0.62 0.67 0.60 0.66 0.58 0.63

Correlation 0.63 0.60 0.68 0.63 0.63 0.65 0.61 0.56 0.61 0.58 0.64 0.58 0.72 0.67 0.71 0.67

KSI 87.32 83.29 65.25 71.38 106.90 110.50 82.71 103.60 328.90 262.30 148.90 152.20 79.25 113.70 70.88 57.25

OVER 0.00 46.38 38.87 42.01 82.82 85.45 55.80 79.13 326.20 258.60 136.30 137.10 55.49 93.46 24.04 21.28

Diffuse Bias 1.08 1.08 0.92 0.94 0.98 0.99 1.19 1.19 1.21 1.14 1.18 1.19 1.11 1.12 0.96 0.99

RMSE 0.34 0.33 0.30 0.30 0.30 0.29 0.33 0.33 0.33 0.34 0.33 0.34 0.32 0.33 0.29 0.31

Correlation 0.36 0.34 0.44 0.38 0.38 0.40 0.35 0.30 0.34 0.30 0.34 0.28 0.40 0.37 0.42 0.34

KSI 148.30 154.70 205.70 220.60 152.80 154.10 197.10 195.30 234.60 173.90 167.90 199.00 195.20 209.60 127.40 140.40

OVER 0.00 136.20 182.40 201.70 126.80 126.90 181.90 181.80 217.40 157.50 148.70 182.90 184.90 196.70 85.02 116.40

Rockhampton Wagga WaggaAdelaide Alice Springs Broome Cape Grim Darwin Melbourne

Annual ACCESS-A All Sky Results

Direct generally under-predicted

Diffuse generally over-predicted

But not always! The results are

site dependent.

Global exposure bias and RMS as function of CSI and SZA

Clear Sky Index created by dividing observed exposure by clear-sky model exposure

Discussion of direct and diffuse irradiance

Model tends to over-estimate direct and under-estimate diffuse Parameterization is tuned for global irradiance at the surface and TOA and

atmospheric heating rate.

Global and direct are calculated separately and differenced to produce

diffuse.

The two stream approach makes approximations for angular integration. Large number of different approximations in the literature

Optimised for different cloud properties

Can we try a different two stream scheme?

Schemes which give same global radiation should not effect

NWP forecast skill. Easier to implement in operational suite.

(Would possibly effect surface parameterization scheme)

GHI

DNI

DNI

Scaled

Unscaled

Unscaled direct two-stream approximation

Work by Zhian Sun

T2m 00Z + 24h Exp1 Exp2 Exp3

AD

89

Bias -0.6819 -0.6762 -0.7619

Err St Dev 1.4285 1.4248 1.8738

RMS Error 1.6964 1.6926 2.1304

BN

95

Bias -0.7933 -0.7956 -1.1621

Err St Dev 1.4054 1.4042 1.7572

RMS Error 1.7144 1.7138 2.1924

DN

35

Bias -0.3850 -0.3823 -0.3091

Err St Dev 1.3537 1.3495 2.4741

RMS Error 1.5807 1.5776 2.6110

PH

174

Bias -0.4630 -0.4575 0.0019

Err St Dev 1.6652 1.6623 2.3227

RMS Error 1.8254 1.8220 2.4602

SY

153

Bias -0.5869 -0.5915 -0.6702

Err St Dev 1.5021 1.4996 1.7590

RMS Error 1.7593 1.7584 2.0149

VT

266

Bias -0.6153 -0.6148 -0.5816

Err St Dev 1.5300 1.5328 1.8318

RMS Error 1.7759 1.7771 2.0474

D2m 00Z + 24h Exp1 Exp2 Exp3

AD

68

Bias -0.4438 -0.4571 -0.1234

Err St Dev 1.8676 1.8735 2.0035

RMS Error 2.2411 2.2509 2.3543

BN

62

Bias -0.0376 -0.0334 0.2782

Err St Dev 1.8704 1.8684 2.0874

RMS Error 2.0611 2.0580 2.3316

DN

32

Bias -0.5316 -0.5290 -0.1444

Err St Dev 2.1853 2.1920 2.7055

RMS Error 2.5135 2.5167 3.1528

PH

49

Bias 0.0634 0.0595 0.5265

Err St Dev 1.7709 1.7609 2.0151

RMS Error 1.9622 1.9488 2.2516

SY

69

Bias -0.2387 -0.2339 0.0161

Err St Dev 1.7616 1.7592 1.9189

RMS Error 1.9366 1.9351 2.0821

VT

153

Bias -0.4536 -0.4569 -0.2859

Err St Dev 1.6792 1.6778 1.8901

RMS Error 1.9321 1.9331 2.1025

ACCESS-C2 model experiments – 0UTC

Courtesy: Tomas Glowacki

Expt 1: Control

Expt 2: unscaled direct

Expt 3: PC2

Results for December 2014

ACCESS-C2 model experiments – Solar All Sky

Results for December 2014 - 0 and 12UTC runs

Direct increased/diffuse decreased in Exp 2

AD AdelaideBN RockhamptonDN Darwin SY Wagga_WaggaVT CapeGrimVT3 Melb_airportVT4 Wagga_Wagga5

Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2 Exp 1 Exp 2

Global Bias 1.03 1.03 0.99 0.99 0.93 0.93 1.02 1.02 1.00 1.00 1.06 1.06 1.03 1.03

RMSE 0.47 0.46 0.46 0.44 0.31 0.31 0.40 0.40 0.47 0.45 0.53 0.54 0.38 0.37

Correlation 0.64 0.64 0.76 0.77 0.80 0.81 0.73 0.73 0.64 0.65 0.60 0.60 0.75 0.76

KSI 21.14 21.81 15.94 16.16 47.16 44.26 19.07 74.79 16.78 15.00 28.22 30.22 21.51 20.09

OVER 0 0 0.00 0.00 16.29 15.54 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00

Direct Bias 0.98 1.13 0.98 1.11 0.87 0.95 1.04 1.18 0.88 1.07 1.00 1.18 1.07 1.21

RMSE 0.63 0.62 0.61 0.60 0.48 0.41 0.45 0.47 0.62 0.58 0.59 0.59 0.40 0.44

Correlation 0.64 0.65 0.73 0.74 0.77 0.78 0.79 0.80 0.58 0.58 0.63 0.63 0.83 0.83

KSI 21.41 55.25 29.54 61.14 81.15 32.44 24.71 74.79 39.03 29.20 37.42 68.25 34.65 87.32

OVER 0.00 11.11 0.00 8.02 50.05 0.97 0.00 29.97 0.90 0.00 4.24 8.91 0.00 47.27

Diffuse Bias 1.12 0.85 0.96 0.75 1.21 0.88 0.97 0.74 1.21 0.92 1.12 0.91 0.95 0.72

RMSE 0.32 0.33 0.28 0.34 0.20 0.19 0.25 0.31 0.30 0.32 0.31 0.35 0.24 0.32

Correlation 0.39 0.35 0.54 0.49 0.56 0.54 0.44 0.31 0.31 0.22 0.33 0.19 0.53 0.41

KSI 80.40 79.27 60.92 101.40 91.46 89.92 88.98 99.70 79.37 37.78 70.90 43.62 72.86 94.85

OVER 43.73 60.57 33.62 79.58 68.37 68.49 51.63 86.90 64.72 18.59 33.63 27.26 41.78 80.00

Count Global 283 283 318 318 328 328 313 313 290 290 319 319 313 313

Direct 283 283 318 318 328 328 313 313 290 290 319 319 313 313

Diffuse 283 283 318 318 328 328 313 313 290 290 319 319 313 313

Midl Cld Bias 80.78 74.32 155.10 151.30 0.00 0.00 105.70 105.50 0.00 0.00 109.50 108.10 98.94 100.80

Low Cld Bias 437.10 404.20 262.50 254.60 571.30 500.00 340.30 349.40 0.00 0.00 264.50 271.10 288.50 280.20

High Cld Bias 101.00 97.52 99.64 89.69 1829.00 1822.00 34.01 33.69 0.00 0.00 83.81 84.77 31.46 31.52

Little change in global

ACCESS-R model experiments – Solar Results

R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1 R2_CTL R2 R1

Global Bias 1.05 1.09 1.05 1.04 0.97 0.97 0.97 0.96 0.97 1.05 1.05 1.12 1.16 1.20 1.29 0.96 1.10 1.02 1.09 1.07 1.03 1.14 1.10 1.05 1.02 0.99 0.96

RMSE 0.60 0.57 0.65 0.58 0.59 0.67 0.50 0.51 0.50 0.65 0.62 0.73 0.65 0.71 0.87 0.83 0.84 0.78 0.55 0.57 0.65 0.63 0.68 0.69 0.52 0.53 0.58

Correlation 0.77 0.79 0.73 0.61 0.62 0.62 0.54 0.51 0.50 0.68 0.70 0.61 0.61 0.59 0.47 0.25 0.26 0.25 0.79 0.78 0.71 0.65 0.60 0.54 0.72 0.72 0.68

KSI 48.86 49.93 33.89 20.8 27.8 17.5 37.1 31.3 35.7 18.8 30.0 52.7 59.7 88.3 120.2 75.1 78.4 65.7 22.5 33.4 20.9 52.4 55.4 28.9 23.0 21.7 23.0

OVER 0.0 0.0 0.0 0.00 0.00 0.00 0.0 0.0 0.0 0.0 0.0 6.4 30.1 56.5 100.3 30.8 42.4 5.2 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Direct Bias 1.06 1.28 1.01 0.95 0.90 0.73 0.83 1.07 0.87 0.94 1.16 0.96 1.26 1.96 1.27 0.85 1.25 0.86 1.05 1.25 0.95 1.17 1.25 0.98 1.03 1.14 0.87

RMSE 0.80 0.82 0.82 1.03 0.94 1.06 0.76 0.79 0.76 0.73 0.65 0.76 0.82 0.97 0.95 0.82 0.82 0.89 0.62 0.67 0.73 0.76 0.81 0.87 0.67 0.66 0.77

Correlation 0.74 0.75 0.71 0.46 0.53 0.60 0.57 0.51 0.53 0.66 0.69 0.62 0.48 0.40 0.30 0.30 0.32 0.19 0.78 0.76 0.71 0.63 0.61 0.51 0.72 0.74 0.68

KSI 54.45 102.80 19.47 86.2 62.5 131.3 74.2 56.6 69.2 35.6 45.7 16.7 91.7 173.1 83.9 59.5 91.2 58.7 22.9 78.2 24.7 46.4 97.0 33.3 54.6 64.7 60.2

OVER 9.0 79.3 0.0 56.1 36.4 118.3 52.5 17.2 53.9 15.2 1.6 1.9 62.7 165.0 53.3 29.2 58.4 43.9 0.7 12.0 0.0 0.7 74.2 6.9 1.4 32.8 5.7

Diffuse Bias 1.02 0.75 1.18 0.90 0.84 1.58 1.32 0.77 1.32 1.19 0.94 1.32 1.03 0.83 1.29 1.16 0.96 1.27 1.15 0.86 1.19 1.09 0.89 1.18 1.00 0.75 1.27

RMSE 0.37 0.44 0.40 0.47 0.50 0.51 0.36 0.45 0.38 0.38 0.37 0.42 0.38 0.48 0.50 0.35 0.37 0.43 0.31 0.39 0.40 0.32 0.36 0.40 0.29 0.36 0.34

Correlation 0.47 0.38 0.43 0.20 0.23 0.38 0.46 0.40 0.43 0.51 0.52 0.48 0.20 0.12 0.09 0.12 -0.01 -0.02 0.49 0.35 0.35 0.43 0.30 0.23 0.52 0.51 0.50

KSI 54.28 101.30 46.30 85.7 99.0 137.5 86.4 105.7 118.6 85.3 34.1 125.1 48.1 80.6 106.7 118.2 39.2 115.3 54.3 62.4 73.7 65.7 61.8 74.4 63.4 99.4 81.0

OVER 23.9 83.6 24.1 49.5 71.2 126.4 50.2 91.5 92.3 68.0 19.1 110.7 8.4 64.3 81.6 103.8 0.0 96.1 4.5 35.1 33.1 31.5 42.7 29.3 33.7 75.3 43.6

Count Global 178 290 290 161 293 293 204 336 336 238 382 382 184 306 306 204 336 336 223 372 372 209 338 338 238 392 392

Direct 178 290 290 161 293 293 204 336 336 238 382 382 184 306 306 204 336 336 223 372 372 209 338 338 238 392 392

Diffuse 178 290 290 161 293 293 204 336 336 238 382 382 184 306 306 204 336 336 223 372 372 209 338 338 238 392 392

Midl Cld Bias 179.40 179.40 165.40 289.8 271.4 195.8 835.4 685.3 638.7 0.0 0.0 0.0 66.7 58.5 43.3 3078.0 507.2 602.6 90.2 85.4 91.5 163.9 171.3 246.3 115.5 95.5 98.4

Low Cld Bias 482.10 412.80 334.20 199.1 1125.0 362.7 145.3 135.5 169.7 0.0 0.0 0.0 286.8 236.1 278.7 291.9 269.3 305.6 297.2 271.9 263.8 226.8 268.6 255.8 250.0 318.6 257.8

High CldBias 100.10 75.60 66.80 45.6 28.1 30.8 445.3 417.2 397.9 0.0 0.0 0.0 123.7 106.4 139.1 619.4 449.1 609.9 36.6 44.2 50.7 136.5 150.9 161.8 79.9 54.2 61.9

Adelaide Broome Cape_Grim Cocos_Island DarwinAlice Springs Rockhampton Wagga_WaggaMelb_airport

Results for December 2014 - 0 and 12UTC runs

Wagga-wagga – all models

SY and VT same model

but different domains (1.5

km)

Wagga is close to the

boundary of SY

SY VT4 A R2 R1

Global Bias 1.02 1.03 1.01 1.02 0.96

RMSE 0.40 0.38 0.43 0.52 0.58

Correlation 0.73 0.75 0.73 0.72 0.68

KSI 19.07 21.51 12.80 23.0 23.0

OVER 0.00 0.00 0.00 0.0 0.0

Direct Bias 1.04 1.07 1.02 1.03 0.87

RMSE 0.45 0.40 0.58 0.67 0.77

Correlation 0.79 0.83 0.71 0.72 0.68

KSI 24.71 34.65 70.88 54.6 60.2

OVER 0.00 0.00 24.04 1.4 5.7

Diffuse Bias 0.97 0.95 0.96 1.00 1.27

RMSE 0.25 0.24 0.29 0.29 0.34

Correlation 0.44 0.53 0.42 0.52 0.50

KSI 88.98 72.86 127.40 63.4 81.0

OVER 51.63 41.78 85.02 33.7 43.6

Midl Cld Bias 105.70 98.94 115.5 98.4

Low Cld Bias 340.30 288.50 250.0 257.8

High Cld Bias 34.01 31.46 79.9 61.9

Melbourne – all models

VT A R2 R1

Global Bias 1.06 0.97 1.09 1.03

RMSE 0.53 0.48 0.55 0.65

Correlation 0.60 0.67 0.79 0.71

KSI 28.22 39.18 22.5 20.9

OVER 0.00 0.00 0.0 0.0

Direct Bias 1.00 0.84 1.05 0.95

RMSE 0.59 0.62 0.62 0.73

Correlation 0.63 0.64 0.78 0.71

KSI 37.42 148.90 22.9 24.7

OVER 4.24 136.30 0.7 0.0

Diffuse Bias 1.12 1.18 1.15 1.19

RMSE 0.31 0.33 0.31 0.40

Correlation 0.33 0.34 0.49 0.35

KSI 70.90 167.90 54.3 73.7

OVER 33.63 148.70 4.5 33.1

Midl Cld Bias 109.50 90.2 91.5

Low Cld Bias 264.50 297.2 263.8

High Cld Bias 83.81 36.6 50.7

Darwin – all models

DN is 1.5 km resolution and the

only model which is convection

permitting

DN A R2 R1

Global Bias 0.93 0.91 0.96 1.02

RMSE 0.31 0.60 0.83 0.78

Correlation 0.80 0.59 0.25 0.25

KSI 47.16 181.60 75.1 65.7

OVER 16.29 176.40 30.8 5.2

Direct Bias 0.87 0.79 0.85 0.86

RMSE 0.48 0.78 0.82 0.89

Correlation 0.77 0.61 0.30 0.19

KSI 81.15 328.90 59.5 58.7

OVER 50.05 326.20 29.2 43.9

Diffuse Bias 1.21 1.21 1.16 1.27

RMSE 0.20 0.33 0.35 0.43

Correlation 0.56 0.34 0.12 -0.02

KSI 91.46 234.60 118.2 115.3

OVER 68.37 217.40 103.8 96.1

Midl Cld Bias 0.00 3078.0 602.6

Low Cld Bias 571.30 291.9 305.6

High Cld Bias 1829.00 619.4 609.9

APS Upgrade Plans

APS1 APS2 APS3 (~2017/2018) APS4 (~2020)

G 40km L70, 4dVAR Mar-2012 (Op)

25km L70, 4dVAR (2 x 240FC + 2 x 78FC)

12km, L85?, 4dVAR / Hybrid (2 x 240FC + 2 x 78FC)

12km, L85?, 4dVAR / Hybrid (2 x 240FC + 2 x 78FC)

R 12km L70, 4dVAR Mar-2013 (Op)

12km L70, 4dVAR (4 x 72FC)

8km, L85?, 4dVAR / Hybrid? (4 x 72FC)

5km, L85?, 4dVAR / Hybrid? (4 x 72FC)

C 4km L70, FC-only Mar-2013 (Op)

1.5km L70, FC-only {6 X C1}

1.5km(V) L85? 4dVAR (Rad), LHN (4 x 36FC + 4 x 18FC + 16 x 9FC )

Unchanged

On Demand

1.5km L70, FC-only

1.5km(V) L85? DS + M * (3dVAR (Rad), LHN), 4 domains max (4 x 36FC + 4 x 18FC + 16 x 9FC )

Unchanged

En-G 60km L70, M24 30km L85?, M24 (2 x 240FC)

30km L85?, M32 (2 x 240FC)

En-C

2.2km(V) L85, M6 “En-C-1” (4 X 24FC, 4 X 36FC )

1.5km(V) L85?, M12? “En-C-1” (4 X 24FC, 4 X 36FC)

Rapid update cycle model FDP

The RUC and times

D0H23

D1H22

BA

SE

TIM

E

VALID TIME D0H23

D3H11

Daylight Daylight

Possible ensemble applications?

10 output for wind, screen variables, precip, etc

High frequency solar requires fast surface scheme (e.g.SUNFLUX)

SUNFLUX: A fast surface radiation parameterization

Zhian Sun's work

Radiative transfer is expensive

Hourly is 30% of model run

time

Clouds, SZA change but

assumed constant

SUNFLUX

Fast but accurate

calculation of surface

irradiance

Efficient enough to run

every time step

Accounts for cloud, SZA

changes

Could be implemented in

APS3

There is a scatter in the metrics with variations from site to site

• Different synoptics

• Cloud frequencies

• Cloud properties

• Accuracy of radiative transfer assumptions to different cloud

regimes

• Aerosol not accounted for in model

The comparisons all show a scatter in the metrics for individual sites

• Is that significant?

• If so which do we prefer?

Conclusions

Extend evaluation to all operational models for all archive times

Establish statistical significance for the different metrics

Partition hourly results in terms of solar zenith angle and time of year

(suggestion by John Boland)

Implement fast surface radiation scheme to produce 10 minute forecasts in

Model (SUNFLUX)

Find more data for validation

Global model

Satellite derived fields

Further work

The Centre for Australian Weather and Climate Research A partnership between CSIRO and the Bureau of Meteorology

Lawrie Rikus

Phone: 03 9669 4452

Email: l.rikus@bom.gov.au

Web: www.bom.gov.au

Thank you