qpf issues in nwp william a. gallus, jr. dept. of geological & atmospheric science iowa state...
TRANSCRIPT
QPF ISSUES IN QPF ISSUES IN NWPNWP
William A. Gallus, Jr.
Dept. of Geological & Atmospheric Science
Iowa State University
ETAMM5
24HR PRECIPITATION 4/00-5/00
OBS PREC:3/12-4/12 OBS PREC:4/12-5/12
ETA MM5
48 HR PRECIPITATION (4/00-6/00)
OBS:3/12-4/12 OBS:4/12-5/12 OBS:5/12-6/12
What is “TRUTH” for QPF verification?
•Increased computer resources have allowed better parameterization schemes and model resolution
•2-day precipitation forecast today is now as accurate as 1-day forecast in 1974
•Each resolution improvement in NCEP Eta model improves skill scores
GOOD NEWS: QPF is GOOD NEWS: QPF is improving!!improving!!
MRF has some skill compared to persistence, even out to 7-8 days:
Roads et al. 1991 (WAF)
This skill is even more apparent for heavy rainfall cases
BAD NEWS: Problems aboundBAD NEWS: Problems abound
•Most improvement in QPF scores occurs during cold season - little improvement in warm season
•Flash flooding kills more people than any other convective-related event
•QPF problems have several potential sources
•Skill scores themselves may be misleading or of little “real” value
Roads and Maisel 1991 WAF:
MRF has regional biases in precipitation over long periods
Example of human improvements on numerical QPF (Olson et al. 1995, WAF)
NGM Manual OBS
Slow improvement in skill for human forecasters, but less skill for heavier amounts (Olson et al. 1995, WAF)
Annual bias has also improved slowly, but interestingly, is better for Day 2 than Day 1 (Olson et al. 1995, WAF)
QPF skill is better is winter than in summer, even when forecasters adjust the NWP guidance
What are sources of What are sources of QPF error?QPF error?
• Resolution inadequacies
• Parameterization errors
• Initialization deficiencies
• Observational errors in verification
If vertical motion is directly constrained by horizontal
resolution…..
•Shouldn’t forecasts for heavy rain events be greatly improved with finer resolution?
•Is there a “magic” resolution where model QPF will approach observed peak values
Gallus 1999 found QPF-horizontal resolution dependence is case-dependent and varies with convective parameterization
6/16/96 6/14/98 7/28/97
7/17/96 5/27/97
BMJ -shaded
KF - clear
Mx obs: 225 Mx obs: 330 Mx obs: 250
Mx obs: 300 Mx obs: 102
Extreme example of unexpected results and Conv. Param. Impacts: 7/17/96 00UTC
surface conditions
00 UTC 17 JUL 1996 - OMAHA
Betts-Miller- Janjic Reference T, Td profiles shown
Large MCS drops up to 300 mm of rain, causing record river crests and severe flash flooding in far eastern NE and western IA.
7/17/96 BMJ simulations with 78,39,22 and 12 km horizontal resolution
NOTE: actual reduction in peak QPF amounts as resolution improves
MX: 46 MX: 45
MX: 32 MX: 32
7/17/96 KF simulations:
NOTE: very strong QPF sensitivity to horizontal resolution. Precipitation area shifted much farther north than in BMJ runs, or observations
MX: 11 MX: 70
MX: 135MX: 186
Daytime precipitation (12-00 UTC 7/16-17/96)
BMJ produces much larger area and amounts
BMJ KF
Convective scheme influences cold pool strength, which in turn, affects evolution of events outside initial rain region
Impacts of convective schemes may be felt outside region of precipitation.
Here, stronger downdrafts in KF scheme result in greater northward transport of instability into Minnesota - leading to more intense subsequent development.
BMJ KF
Another case: Iowa flood of Another case: Iowa flood of June 1996June 1996
Large-scale region looked favorable for excessive rains
Heaviest rains (225 mm) fell in small area in warm sector
Impacts of horizontal resolution changes strongly depend on convective scheme used
Tropical-like soundings with very deep moisture
Td at 850 mb = 18 C
Td at 700 mb = 8 C
BMJ simulations:
Almost no horizontal resolution-QPF dependence
No hint of C IA maximum
12 UTC 6/16 cold pool affecting Iowa
12 UTC 6/16 Eta model 00 hr - initialization
NOTE: cold pool is missing: winds are southerly, without E component
21 UTC 6/16
Observed Surface Moisture Convergence
Flood-producing storms would form on C IA enhancement
Simulated Moisture Convergence -21 UTC - BMJ run with 12 km resolution
Despite poor initial wind field, model does show enhancement in W IA
BMJ simulation:
No general clearing into Iowa by 1 pm -
Less destabilization than actually occurred
KF simulations:
Strong horizontal resolution-QPF dependence
Some evidence of C IA enhancement with 22 and 12 km resolution
KF 6 hr forecast:
Some clearing into SW Iowa
more agreement with obs.
June case shows:June case shows:
• Moist low-mid troposphere allows BMJ scheme to be aggressive
• Even high resolution may not improve simulation of small QPF maxima if other simulated parameters are incorrect
• Generation of QPF upstream due to resolution changes may affect QPF downstream
Changes within a specific convective parameterization
can also have a very pronounced effect on QPF
Spencer and Stensrud (1998) show this using MM4 with KF scheme
Spencer & Stensrud variations in KF scheme
•Permit Precipitation Efficiency to remain at maximum (90%) instead of varying from 10-90%
•Neglect convective downdrafts
•Delay convective downdrafts
Max. Prec for 4 tests
Case OBS nomod mpe ndd DddAug86 170 53 61 107 66Jul87 254 48 53 188 79Sep89 150 76 94 170 97Jun90 127 48 36 142 52Nov92 236 51 61 132 80AVG 196 67 74 150 83
Maximum QPF in 4 KF MM4 runs
From Spencer and Stensrud 1998 - MWR
MicrophysicalMicrophysical schemes may be the schemes may be the next challenge - once resolution next challenge - once resolution
improves so that convective improves so that convective parameterization is no longer parameterization is no longer
necessarynecessary• Colle and Mass examine resolution-
orographic precipitation (1999) dependence
• Microphysical schemes influence results
OBS PRECIP IN PACIFIC NORTHWEST FLOOD EVENT (1996)
from Colle and Mass (1999; MWR)
Pronounced orographic effects
4 km MM5 run does well at crest but underestimates lee precipitation
Horizontal resolution affects precipitation patterns near mountain due to resolution of mountain wave effects. Model QPF performance in lee of mountain fluctuates - low bias is best in coarsest run, but heaviest precipitation just to lee of crest occurs with highest resolution
1.33
4
12
36
Although precipitation forecasts generally improved as resolution was refined from 36 to 4km, little additional improvement occurred with 1.3 km resolution (Colle & Mass)
Model QPF in relation to resolution of topography
Microphysical schemes may have significant influences at high resolution.
Colle and Mass (1999; MWR) found that lee-side precipitation was too small in high-res MM5 simulations, partly because snow fallspeeds were too large.
Best results may not occur with most sophisticated microphysical scheme
Microphysical scheme differences affect QPF in different areas
Mesoscale initialization may be Mesoscale initialization may be poor and affect QPFpoor and affect QPF
Stensrud and Fritsch (1994) have shown the impacts of improved cold
pool initialization
Typical initialization Typical initialization deficienciesdeficiencies
• Low-level jet characteristics
• Cloud boundaries
• Fronts and drylines
• Convective outflows
• Surface characteristics
Stensrud and Fritsch 1994 MWR:
Initialization of NE KS mesoscale boundary has important impact on QPF
MM4 -25KM
How do we verify QPF?How do we verify QPF?
• Bias scores (how many grid points have X amount of rain compared to observations)
• Threat Scores (area correct/(area forecast+ area observed - area correct))
• Probability of Detection
PRIMARY VERIFICATION PRIMARY VERIFICATION TOOLS TODAY USED BY TOOLS TODAY USED BY
NCEP FOR QPF ARE:NCEP FOR QPF ARE:
BIAS: Number of grid points having simulated rain of X amount divided by
number of observed points with X amount
EQUITABLE THREAT SCORE: Ratio of correct forecasts (hits) to total forecasts +
observations - hits (with correction for chance hits)
BIASBIAS•B=F/O
•Can vary from 0 to >> 1
•Bias > 1 means the model is generous with areal coverage of precipitation
•Bias < 1 means the model doesn’t generate enough areas with precipitation
•Many operational models have B>1 for small precipitation amounts, and B<1 for large amounts
ETSETS
•ETS=(H-C)/[F+O-(H+C)]
•0<ETS<1
•Similar to a Threat Score but takes into account that even “chance” forecasts will be correct some of the time (Schaefer 1990; Gandin and Murphy 1992)
1995-1997 ETS AT NCEP (Mesinger 1998)
.34 .36 .35 .30 .26 .22 .17 .12
.33 .355 .35 .30 .26 .22 .175 .12
.31 .34 .32 .27 .23 .20 .16 .10
.30 .315 .29 .23 .19 .16 .10 .08
.01 .10 .25 .50 .75 1.00 1.50 2.0029ETA
48ETA
MRF
NGM
How valuable are these How valuable are these verification methods?verification methods?
• Model A covers your state with 1inch of rain
•Model B simply produces 5 inches in the one county to your east
•A lone supercell drops 5 inches on your county
Which model had the Which model had the better forecast?better forecast?
What are our Bias and ETS?What are our Bias and ETS?
For measurable precip (or any category less than 1 inch):
Assume one grid point per county with 100 counties in state
Bias in model A: 100/1 = 100.0
Bias in model B: 1/1 = 1.00
ETS in A: 1/(100+1-1) = 0.01
ETS in B: 0/(1+1-0)= 0.0
Objective scores may not Objective scores may not agree with your answer!agree with your answer!
Improved mesoscale QPF verification may involve a phase shift of the simulated
precipitation field. Kalnay and others (1999) are studying such an approach
Concluding ThoughtsConcluding Thoughts
•QPF is probably the most difficult aspect of NWP - the hardest one to envision being solved in 25 years
•If convective parameterizations are used, behavior of these schemes exerts powerful impact (primary differences between different models are probably related to the Cu scheme)
•Thus, forecasters can benefit by understanding the specifics of how the schemes behave
Concluding Thoughts (Cont.)Concluding Thoughts (Cont.)
• At very high resolutions, microphysics will likewise complicate the picture
• Forecasters need to be aware of small-scale boundaries of importance, which will most likely be poorly depicted in initialization
• New methods of evaluating what is a “good” QPF will be needed