Bias Adjusted Precipitation Scores

Fedor Mesinger

NOAA/Environmental Modeling Centerand

Earth System Science Interdisciplinary Center(ESSIC), Univ. Maryland, College Park, MD

VX-Intercompare MeetingBoulder, 20 February 2007

Most popular “traditional statistics”:

ETS, Bias

Problem: what does the ETS tell us ?

“The higher the value, the better the model skill is for the particular threshold”

(a recent MWR paper)

Example:Three models, ETS, Bias, 12 months, “Western Nest”

Is the green model loosing to red because of a bias penalty?

What can one do ?

BIAS NORMALIZED PRECIPITATION SCORES

Fedor Mesinger1 and Keith Brill2

1NCEP/EMC and UCAR, Camp Springs, MD2NCEP/HPC, Camp Springs, MD

J12.617th Prob. Stat. Atmos. Sci.; 20th WAF/16th NWP (Seattle AMS, Jan. ‘04)

Two methods of the adjustment for bias(“Normalized” not the best idea)

1. dHdF method: Assume incremental change in hits per incremental change in bias is proportional to the “unhit” area, O-

H

Objective: obtain ETS adjusted to unit bias, to show the model’s accuracy in placing

precipitation(The idea of the adjustment to unit bias to arrive at placement accuracy:

Shuman 1980, NOAA/NWS Office Note)

2. Odds Ratio method: different objective

O

H

a

b

c

d

F

Forecast, Hits, and Observed (F, H, O) area, or number of model grid boxes:

dHdF method, assumption:

can be solved;

a function H (F) obtained that satisfies the three

requirements:€

dHdF

= a (O − H ), a = const,

• Number of hits H -> 0 for F -> 0;

• The function H(F) satisfies the

known value of H for the model’s F,

the pair denoted by Fb, Hb, and,

• H(F) -> O as F increases

West

EtaGFS

NMM

Bias adjusted eq. threats

A downside: if Hb is close to Fb, or to O,it can happen that

dH/dF > 1 for F -> 0

Physically unrealistic !

Reasonableness requirement:

€

Hb ≤ O(1−e−Fb /O )

“dHdM”method:

O

H

a

b

c

d

F

Assume as F is increased by dF, ratio of the infinitesimal increase in H, dH, and that in

false alarms dM=dF-dH, is proportional to the yet unhit area:

€

dH

dM= b(O − H)

€

b = const

One obtains

( Lambertw, or ProductLog in Mathematica,is the inverse function of

€

z = wew )

€

H(F) = O −1

blambertw bOeb(O−F )

( )

H (F) now satisfies the additional requirement:

dH/dF never > 1

20 40 60 80 100 120 140

F

20

40

60

80

100

120H

H(F)

H = O

H = F

Fb , Hb

dHdF method

H(F)

H = O

H = F

Fb , Hb

20 40 60 80 100 120 140F

20

40

60

80

100

120H

dHdM method

Results for the two “focus cases”,dHdM method

(Acknowledgements: John Halley Gotway, data; Dušan Jović, code and

plots)

5/13 Case dHdM

wrf2capswrf4ncarwrf4ncep

6/01 Case

dHdM

wrf2capswrf4ncarwrf4ncep

Impact, in relative terms, for the two cases is small, because the biases of the three models

are so similar !

One more case, for good measure:

5/25 Case dHdM

wrf2capswrf4ncarwrf4ncep

Comment:

Scores would have generally been higher had the verification been done on grid squares

greater than ~4 km

This would have amounted to a poor-person’s version of “fuzzy” methods !