CLIMATE RESEARCHClim Res

Vol. 59: 135–147, 2014doi: 10.3354/cr01213

Published online March 25

1. INTRODUCTION

Unresolved subgrid physical processes are oftenparameterized in numerical weather and climatemodels to estimate the exchanges of mass, energy,and momentum with the resolved grids (e.g. Hack1994, Bechtold et al. 2001, Kain 2004, Qian et al.2010). Physics parameterizations generally use con-ceptual or empirical relationships to approximate the

impact of subgrid processes on the resolved-scale dy -namics and thermodynamics (e.g. Kain 2004, Becht -old et al. 2008). Consequently, the parameterizationschemes may contain some parameters that have nodirect physical equivalence in nature and are subjectto a certain degree of arbitrariness (Done et al. 2006,Berner et al. 2012). Due to nonlinear interactionsamong physical processes, large uncertainties couldbe introduced when these parameterizations are

© Inter-Research 2014 · www.int-res.com*Corresponding author: yun.qian@pnnl.gov

Parametric sensitivity and calibration for theKain-Fritsch convective parameterization scheme

in the WRF model

H. Yan1,2, Y. Qian2,*, G. Lin2, L. R. Leung2, B. Yang3, Q. Fu1,4

1School of Atmospheric Sciences, Lanzhou University, Lanzhou 730000, China2Pacific Northwest National Laboratory, Richland, Washington 99354, USA

3School of Atmospheric Sciences, Nanjing University, Nanjing 210000, China4Department of Atmospheric Sciences, University of Washington, Seattle 98105, USA

ABSTRACT: Convective parameterizations used in climate models display sensitivity to modelresolution and variable skill in different climatic regimes. Although parameters in convectiveschemes can be calibrated using observations to reduce model errors, it is not clear if the optimalparameters calibrated based on regional data can robustly improve model skill across differentmodel resolutions and climatic regimes. In this study, this issue is investigated using a regionalmodeling framework based on the Weather Research and Forecasting (WRF) model. To quantifythe response and sensitivity of model performance to model parameters, we identified 5 key inputparameters and specified their ranges in the Kain-Fritsch (KF) convection scheme in WRF, and cal-ibrated them across different spatial resolutions, climatic regimes, and radiation schemes usingobserved precipitation data. Results show that the optimal values for 5 input parameters in the KFscheme are similar, and model sensitivity and error exhibit similar dependence on the inputparameters for all experiments conducted in this study, despite differences in the precipitation climatology. We found that the model overall performances in simulating precipitation are rela-tively more sensitive to the coefficients of downdraft and entrainment mass flux, as well as to thestarting height of downdraft. However, we found that rainfall biases — which are probably morerelated to structural errors — still exist over some regions in the simulation, even with the optimalpara meters. This suggests that further studies are needed to identify the sources of uncertainties,as well as to reduce the model biases or structural errors, both of which are associated with missedor misrepresented physical processes and/or potential problems with the modeling.

KEY WORDS: Sensitivity · Convection scheme · Parameters · Calibration · Optimization · Regionalclimate model · WRF

Resale or republication not permitted without written consent of the publisher

Clim Res 59: 135–147, 2014

applied to numerical weather or climate models(Warren & Schneider 1979). Uncertainties associatedwith parameterizations and model parametersbecome a key challenge to skillful weather or climateprediction (Bowler et al. 2008, Hacker et al. 2011a,2011b, Reynolds et al. 2011, van Lier-Walqui et al.2012). Quantifying and reducing uncertainties ofparameterizations is crucial for improving model sim-ulations of present climate and obtaining more accu-rate projection of future climate.

Parameter tuning or perturbation can be applied tocalibrate climate models or investigate model sensi-tivity (Jackson et al. 2004, 2008). Yang et al. (2012)found that model performance is improved over theNorth American Monsoon (NAM) region when theoptimal parameters obtained from model calibrationover the Southern Great Plains (SGP) are used. Evenso, model parameter sensitivity could vary over dif-ferent regions with different climate regimes, e.g.SGP versus NAM, because the prevailing large-scalecirculation pattern, atmospheric moisture, and landsurface types can be very different (Zhu et al. 2009,Qian et al. 2013). Large-scale circulation couldimpose forcing on clouds by adjusting its radius andentrainment rate (Bechtold et al. 2001, Kain 2004,Berner et al. 2012), and land cover and land use typescould affect land–atmosphere interactions and, con-sequently, the intensity of convection and precipita-tion (e.g. Koster et al. 2004, Taylor & Ellis 2006). Fur-thermore, ambient moisture is a modulating factor inconvection activities (Redels perger et al. 2002, Bech-told et al. 2008). It remains unclear how parametertuning and calibration may lead to different results indifferent regions and climate regimes. Jackson et al.(2008) suggested that the values of optimal parame-ters might change from one region to anotherbecause of compensating errors. To determine therobustness of parameter tuning and calibration, it isimportant to compare their results over regions withdifferent climate regimes.

Currently, one of greatest challenges in climatemodeling is the scale dependence of model physicalparameterizations, which leads to sensitivity ofmodel simulations to model grid spacing (Arakawa etal. 2011, Qian et al. 2010). In addition, Yang et al.(2012) found that model-simulated precipitation issensitive to radiation schemes, with the model gener-ating wet bias when the Rapid Radiative TransferModel for General Circulation Models (RRTMG), isapplied but dry bias appears when the CommunityAtmosphere Model (CAM) scheme is used instead. Itis important to determine whether and how model-simulated precipitation may converge in the calibra-

tion process when the model is systematically wet ordry in control simulations owing to differences inmodel resolution and/or physics parameterizations.

In this study, we use the same Weather Researchand Forecasting (WRF) regional climate model as inYang et al. (2012) to examine the sensitivity anduncertainty of the parameter tuning process on 5convective parameters in the Kain-Fritsch (KF) con-vection parameterization scheme (CPS). We alsocalibrate the CPS by constraining simulated precipi-tation with the observations. We attempt to answerthe following questions: (1) How are the parametertuning and calibration processes sensitive to gridspacing, climate regimes, and initial model biases?(2) Are the optimal results transferable acrossspatial resolution and climate regimes? To addressthese questions, we conducted a series of simula-tions over SGP with the grid spacing of 12, 25, and50 km and using 2 radiation schemes (RRTMG ver-sus CAM). We also conducted parameter tuning andcalibration in the NAM region, a relative arid regioncompared to the humid SGP, where the processesinfluencing convection should differ (Zhu et al.2009). The simulated stochastic approximationannealing (SSAA) algorithm, a stochastic samplingapproach that has shown an advantage in efficiencyand lower proneness to local trap (Liang et al. 2013),was chosen to sample identified parameters in theCPS.

2. METHODOLOGY

2.1. Model setup and experimental design

In this study, the WRF model Version 3.2.1 (Skama -rock et al. 2008) was employed. Following the previ-ous parameter tuning study by Yang et al. (2012), theMorrison 2-moment cloud microphysics scheme(Morrison et al. 2005) and KF CPS (Kain 2004) wereused. The Noah land surface model (LSM) (Chen &Dudhia 2001) and Mellor-Yamada-Janjic (MYJ) (Janji 2002) planetary boundary layer (PBL) turbu-lence scheme were also applied.

We conducted a series of experiments over differ-ent regions including the SGP and NAM regions(Fig. 1) at different horizontal resolutions (12, 25, and50 km), which allowed us to investigate the depend-ence of the parameter tuning process on model spa-tial resolutions and climate regimes. To examine theimpact of different radiation schemes on parametertuning, we conducted the experiments with 2 differ-ent radiation schemes, i.e. RRTMG (Mlawer et al.

136

Yan et al.: Parametric sensitivity in the WRF model

1997, Barker et al. 2003, Pincus et al. 2003) and CAM3.0 (Collins et al. 2004). The experiments are summa-rized in Table 1.

The WRF simulations were driven by the 32 kmNorth American Regional Reanalysis (NARR), andlateral boundary conditions are updated every 3 h.For SGP, the first simulation was initialized at00:00 h UTC 1 May 2007 and run for 1 mo with thestandard KF scheme until 00:00 h UTC 1 June. Allen semble simulations were then run for anothermonth through June 2007, using identical initialland surface conditions from the first simulation at00:00 h UTC 1 June. The experiments over theNAM regime were similar, except they were initial-ized at 00:00 h UTC 1 June 1991 and the analysisperiod was July 1991 over the whole domain. To

minimize the potential effects of error in the simu-lated large-scale circulation and isolate the impactof convective parameterization scheme on precipita-tion, atmospheric conditions were reinitialized usingthe NARR data every 2 d in all experiments. Eachsimulation was run for 3 d, but the first day was dis-carded as model spin-up. Therefore our analysis re -present an average of 15 two-day ensembles (total-ing 1 mo), which can loosely be called ‘climate’. Soilmoisture and temperature were initialized usingdata from the last hour of the WRF simulation forthe prior 2 d. June 2007 was an ex tremely wetmonth over SGP, and abundant moisture in theatmospheric was mainly contributed by the low-level jet originated from the Gulf of Mexico.

2.2. Kain-Fritsch convective parameterizationscheme

The KF convective parameterization is a simple 1-dimensional mass flux cloud model (Kain & Fritsch1990, Kain 2004) specifically designed for mesoscalemodels. Convection is triggered when a mixture ofadjacent 60 hPa air generates positive buoyancy andascends. Then, updraft, downdraft, and entrainmentfluxes are calculated to redistribute air mass until90% of the convective available potential energy(CAPE) is eliminated. Entrainment between the envi-ronment and the cloud is proportional to the updraftflux mass and is explicitly calculated according to thecloud radius. Deep convection is activated when theupdraft flux rises upward over a certain cloud depth.Otherwise, shallow convection is formed based onturbulent kinetic energy (TKE) for the mass flux.Downdraft process occurs within a layer of 150 to200 hPa above the cloud base, and its mass is calcu-lated according to relative humidity and stability. There sidual condensate after updraft detainment anddown draft evaporation is considered as ‘convectiveprecipitation’. Different closure assumptions are ap-plied to shallow and deep convections. For shallowconvection, the initial updraft flux mass is propor-tional to the TKE in the sub-cloud layer, while the clo-sure assumption of deep convection depends on theCAPE removal for an entraining parcel. A de taileddescription of the KF CPS can be found in Kain &Fritsch (1990), Bechtold et al. (2001) and Kain (2004).

As in Yang et al. (2012), the 5 most critical parame-ters of the KF scheme were identified to calibratemodel precipitation. Brief descriptions and valueranges of the selected 5 parameters are shown inTable 2.

137

Experiment Resolution Region Radiation (km) scheme

SGP12 12 SGP RRTMGSGP25 25 SGP RRTMGSGP50 50 SGP RRTMGNAM25 25 NAM RRTMGCAM25 25 SGP CAM

Table 1. Experiments performed in this study. SGP: South-ern Great Plains; NAM: North American Monsoon; CAM:Community Atmosphere Model; RRTMG: Rapid Radiative

Transfer Model for General Circulation Models

Fig. 1. Two model domains used in this study: SouthernGreat Plains (SGP) outlined in red and North America Mon-soon (NAM) in black. Shades indicate the terrain (a.s.l.:above sea level). The yellow rectangle represents the sub-domain where statistics were conducted for this study

Clim Res 59: 135–147, 2014

2.3. Simulated stochastic approximation annealing(SSAA) calibration algorithm

This study employed the SSAA algorithm (Liang etal. 2013) to study the model sensitivity to the 5 convec-tive parameters and calibrate them. The SSAA is anapproach for global calibration developed on the basisof a very fast simulated annealing (VFSA) algorithm(Ingber 1989, Jackson et al. 2004) with an element ofthe stochastic approximate Monte Carlo (SAMC)algorithm (Liang 2011) blended in. Thus, it takesadvantage of both the annealing and SAMC tech-niques, which enables the algorithm to accelerate theconverging speed while the SAMC skill reduces theprobability of trapping into local maxima or minima.As the SSAA procedure progresses, the samplingrange of each parameter gradually narrows. As sug-gested in Yang et al. (2012), each experiment with aspecific configuration consisted of 150 simulations.More details of this algorithm can be found in Liang etal. (2013) and Yang et al. (2013). Model performancewas evaluated against the observational data aftereach simulation is completed. The University of Wash-ington 1/8 gridded daily precipitation data (Maurer etal. 2002) was used to calculate the model skill scoreaccording to the formula in Yang et al. (2012).

3. RESULTS

3.1. Evaluation for default simulation

Fig. 2 (top panel) shows the spatial distribution ofmonthly mean precipitation observed and simulatedby WRF at 25 km spatial resolution with the defaultconfiguration over SGP in June 2007. The most in -tense precipitation occurs in central Texas, Oklaho -ma, southeastern Kansas, and southwestern Mis-souri. The precipitation events during the extremewet period are initially generated by active synopticweather patterns, linked with moisture transport

from the Gulf of Mexico by the northward low-leveljet, and enhance the frequency of thunderstorms andtheir associated latent heat release (Qian et al. 2013).The control simulation with default convective para -meters can generally capture the observed intenseprecipitation band, but they significantly overesti-mate the magnitude of precipitation at the northeastcorner of the domain. Fig. 2 also shows the compari-son between skin temperature (Ts), latent heat (LH),sensible heat (SH) flux and surface net radiation flux(SWN) from the WRF control simulation and theNorth American Land Data Assimilation SystemPhase 2 (NLDAS-2) reanalysis dataset that weredriven by observed precipitation and NARR forcingfields during the same period. It can be seen that theWRF control simulation is able to capture the spatialpatterns and magnitude of the key land surface vari-ables reasonably well, especially for Ts and SH, com-pared to the NLDAS-2 reanalysis dataset. However,both LH (upward as positive) and SWN (downwardas positive) are significantly overestimated in thecontrol simulation with default parameters, espe-cially over the eastern part of domain. In the simula-tion with the optimal parameters, both LH and SWNare much improved (data not shown).

3.2. Dependence on model spatial resolution

Fig. 3 shows the responses of the cost function E tothe 5 convective parameters in the simulations withdifferent spatial resolutions at SGP. Among all thesimulations sampled using SSAA, E varies from 66 to282 in all simulations, with values of 165, 139, and148 for the default simulations at 12, 25, and 50-kmgrid spacing, respectively. The cost function E de -creases as the downdraft mass flux rate (Pd) and thestarting height of downdraft (Ph) increase or theentrainment mass flux rate (Pe) decreases. Similartrends of E can be found across the 3 spatial resolu-tions. The model performances are found to be more

138

Parameter Description Default value Range

Pd Coefficient related to downdraft mass flux rate 0 [−1, 1]Pe Coefficient related to entrainment mass flux rate 0 [−1, 1]Ph Starting height of downdraft above updraft source layer (USL) (hPa) 150 [50, 350]Pt Maximum turbulent kinetic energy (TKE) in sub-cloud layer (m2 s−2) 5 [3, 12]Pc Average consumption time of convective available potential energy (CAPE) 2700 [900, 7200]

Table 2. Abbreviations, descriptions, default values, and the ranges of the 5 input parameters in the Kain-Fritsch (KF) convective parameterization scheme in the Weather Research and Forecasting (WRF) model

Yan et al.: Parametric sensitivity in the WRF model

sensitive to Pd, Pe and Ph, with ap -proximately linear features. Theslopes of trends for the above 3 para -meters, as well as the maximum TKE(Pt), are smaller in coarse resolutionthan in fine resolution, suggesting amore sensitive model performance tothe KF parameters at higher resolu-tion. The response of E to CAPE con-sumption time (Pc) shows a de creasewhen Pc is <3700, then increases within creasing Pc.

Fig. S1 in the Supplement at www.int-res. com/ articles/ suppl/ c059 p135_supp. pdf shows the responses ofthe monthly mean explicit (non- convective), convective, and totalprecipitation (explicit + convective) tothe 5 parameters, averaged over thesubdomain highlighted by the yellowrectangle in Fig. 1. The mean explicitprecipitation is ~0.2 to 1.8 mm d−1,and the convective precipitation is~4.5 to 8.0 mm d−1. On average,>70% of the total precipitation is con-tributed by the convection schemeover SGP during the summer months.The total rain amount varies from 9.7to 4.6 mm d−1 in the parameter tuningranges, with more rainfall usuallyproduced in the 12 km simulation andless rainfall at 50 km resolution, andthis difference across model resolu-tion is more apparent for non-convec-tive precipitation. Indeed, Fig. S1shows that explicit precipitation in -creases with spatial resolution be -cause more clouds are resolved atfiner resolution, but less significantdifference can be found in the con-

139

Fig. 2. Comparison of climate variablesfrom the Weather Research and Forecast-ing (WRF) default simulation (left) and ob-servation/ reanalysis (right). Ts: surface skintemperature; LH: latent heat flux; SH: sen-sible heat flux; SWN: net short wave radia-tion flux at surface. Observation for precip-itation is from the University of Washington1/8 gridded daily precipitation dataset(UW) and for other variables from NorthAmerican Land Data Assimilation System

Phase 2 (NLDAS-2) reanalysis

Clim Res 59: 135–147, 2014

vective and total precipitation simulated at differentresolutions.

Similar changes and variability of the simulatedprecipitation to the 5 parameters can be found acrossall 3 spatial resolutions, which demonstrate the spe-cific roles of these parameters during the precipita-tion-related processes. For example, convective pre-cipitation amounts decrease with increases ofpara meters Pd, Ph, and Pc, and explicit precipitationamounts increase as Pe and Pc increase. Larger Pd andPh usually generate greater downdraft mass flux andhigher precipitation-driven downdraft starting levels.Thus, more condensate in the downdrafts is evapo-rated, and less precipitation is produced. The parame-ter Pe is the ratio of entrainment to the updraft flux,and measures how much ambient air is brought intothe updraft flux, which can lessen the difference be-tween the updraft parcel and the surrounding envi-ronment. A higher entrainment rate favors a more sta-ble atmosphere, so less frequent deep convection isactivated. As a larger portion of the initial instabilitycould be removed by shallow convection, more ex -plicit precipitation can potentially be produced. Theincrease of Pc generally results in decreased convec-tive precipitation and increased explicit precipitation(Done et al. 2006, Yang et al. (2012)). With a longer Pc

a deep convection event would weaken, and moregenerated shallow convections would reduce the in-stability in the atmosphere, which could potentially

provide additional moisture for the resolved scales(Kain 2004).

Many atmospheric variables (e.g. cloudiness, airtemperature, humidity, and surface energy budget)are expected to be sensitive to the perturbation of theconvective parameters (e.g. Bechtold et al. 2001,2008, Jackson et al. 2004, 2008, Berner et al. 2012)because latent heating from convective clouds caninfluence many atmospheric processes. For example,Fig. 4 shows the responses of several selected physi-cal variables to the Pd. Low-level clouds, air tempera-ture, humidity, and surface soil moisture and energyflux all depict a clear response to the parameter Pd.Larger Pd could result in more evaporation of con-densate generated in the updraft process, which willcool and moisten the atmosphere in layers of 900 to600 hPa. Consequently, more low-level clouds areformed, which could decrease the downward short-wave radiation flux. Decreased precipitation leads tolower soil moisture, larger SH, and lower LH. Thechanges of air humidity are opposite at low level(1000 to 900 hPa) and upper layers (900 to 600 hPa).And the low level air humidity, LH and soil moistureall show a decrease, consistent with the PBL humid-ity that is more influenced by mixing from the surfacerather than the upper layer atmosphere.

While the overall responses of physical variables toconvective parameters are consistent across differentresolutions, remarkable differences of magnitude in

140

Fig. 3. The response of model performance, quantified as the cost function E (see Section 2.2) to the 5 input parameters (seeTable 2) over the Southern Great Plains (SGP) region through the simulated stochastic approximation annealing (SSAA)

procedure based on 3 experiments (see Table 1) with spatial resolutions of 50, 25 and 12 km

Yan et al.: Parametric sensitivity in the WRF model 141

Fig. 4. The response of 14 model output variables (averaged over the subdomain [yellow box] shown in Fig. 1) to variations inthe input parameter Pd (see Table 2) over the Southern Great Plains (SGP) region, based on 3 experiments (see Table 1) withspatial resolutions of 50, 25 and 12 km. The 14 variables are: surface skin temperature; soil moisture; specific humidity and tem-perature at the boundary layer (1000−900 hPa), low troposphere (900−800 hPa) and middle troposphere (800− 600 hPa); cloudliquid water at 900−800 hPa; downward shortwave and longwave radiation flux at the surface; outward longwave radiation flux

at the top of atmosphere; and sensible and latent heat flux at the surface

Clim Res 59: 135–147, 2014

the response can be found among the experimentswith different horizontal resolutions. For example,the atmospheric humidity at all levels is lower in thesimulation with finer resolution than coarser resolu-tion (see Fig. 4), and the drier atmosphere may resultfrom more condensation (so more precipitation gen-erated) which consumes more moisture at the finerresolution

Fig. 5 shows the monthly mean precipitation differ-ences between default simulations and observationand between optimal and default simulations at 3spatial resolutions, respectively, over SGP in June2007. Using default parameters, all 3 simulationsoverall produce too much rain except for a few areasin central Texas, Oklahoma and southeastern Kan sas.With the optimal parameters, the overall overesti-mated precipitation is improved. Optimal simulationlargely removes the wet biases over the north easternpart of the domain and over western Texas, butstrong precipitation in the main precipitation bandalong central Texas, Oklahoma, and southeasternKansas is suppressed. The cost function E is reducedfrom 165 to 76, 139 to 70, and 148 to 66 for SGP12,SGP25, and SGP50, respectively (see Table 3).

Fig. 6 shows the correlation coefficients between

different simulated variables with each CPS para -meter, which, to a certain extent, can be used to rep-resent the sensitivity between the 5 convective para -meters and the physical variables at 3 spatialre so lu tions. As discussed in Section 3.1, parametersPd and Ph have a larger impact on convective precip-itation, while explicit rainfall is more sensitive to Pe

and Pc. Pe also has a large impact (with a correlationcoefficient >0.8) on surface and low-level air temper-ature and downward shortwave radiation flux. Themost sensitive parameter for SH is Pd while Pc has thelargest impact on downward longwave radiation fluxat the surface because longer Pc favors the formationof stratiform clouds especially with larger spatialscale. All variables are least sensitive to Pt belowclouds. Generally, almost all variables show similarsensitivities to the parameters across different spatialresolutions.

3.3. Dependence on climate regimes

To examine the dependence of parameter calibra-tion on different climate regimes, the calibration pro-cedure for the convective parameters was repeated

142

Fig. 5. Differences of precipitation between Weather Research and Forecasting (WRF) default configurations and observations(top) and between the optimal and default simulations (bottom) for 3 experiments (see Table 1) with spatial resolutions of 50,

25 and 12 km

Yan et al.: Parametric sensitivity in the WRF model

over the NAM region. Fig. S2 in the Supplement(www. int-res. com/ articles/ suppl/ c059 p135_ supp.pdf) shows the response of E to 5 convective parame-ters over NAM in July 1991. Among all the simula-tions in NAM25, E varies from 52.7 to 388. Similar toover SGP, E decreases as Pd and Ph increase or Pe de -creases. Fig. S2 suggests that larger Pd or Ph and

lower Pe favor a better performance ofthe simulation over NAM, which isconsistent with the results over SGP.Fig. S2 also shows the difference ofresponses of E to Pe over SGP andNAM: generally, E increases with Pe inSGP; however, in NAM, E in creaseswith Pe when Pe < 0.1 but de creaseswhen Pe > 0.1. In addition, theresponse of E to Pe scatters morewidely in NAM than SGP.

Fig. S3 in the Supplement depictsthe precipitation responses to the 5 pa-rameters over NAM. The total precipi-tation amount varies from 5 to 8 mmd−1, while convective and ex plicit pre-cipitation are ~2 to 5 mm d−1and 0.25to 2.5 mm d−1, respectively. The ratioof the explicit to total precipitation islarger over NAM than SGP. The totalprecipitation is most sensitive to Pd

and Ph, showing significant increasesin re sponse to greater values of the pa-rameters. Both ex plicit and convectiveprecipitation are especially sensitiveto Pc but with opposite responses.Thus, the total precipitation does notshow an obvious response to this para -meter, which is different from SGP,where the total precipitation de creaseswith in creased Pc, because the convec-tive rain dominates in the SGP case.

The precipitation response to E is also dif-ferent from SGP, i.e. neither explicit nor convectiveprecipitation shows a clear linear correlation with Pe

over NAM.Fig. 7 shows the spatial distributions of monthly

mean precipitation of July 1991, observed and simu-lated with default and optimal parameters. The simu-lation with default parameters captures the maxi-

143

Fig. 6. Relative sensitivities of the response of selected output variables (y-axis)to the 5 convection parameterization scheme (CPS) input parameters shown inTable 2 (x-axis) over the Southern Great Plains (SGP) region for 3 experiments(see Table 1) with spatial resolutions of 50, 25 and 12 km. For each parameter,corresponding columns (left to right) represent simulations with 12, 25 and50 km resolution. Sensitivity ranking is calculated based on the correlation coef-ficients of 150 simulations between output variables and input CPS parameters.‘+’ (‘–’): positive (negative) correlation between input parameters and outputvariables (i.e. that variables increase [decrease] with the parameters). TS: skintemperature; SM: soil moisture; Q(P): air specific humidity for 1000− 900 hPa;T(P): air temperature for 1000−900 hPa; QC: cloud liquid water content at layersfrom 900−800 hPa; Q(L): air specific humidity for 900−800 hPa; T(L): air temper-ature for 900− 800 hPa; Q(M): air humidity for 800−600 hPa; T(M): air tempera-ture for 800−600 hPa; SWD: downward shortwave radiation at surface; LWD:downward longwave radiation at surface; OLR: outward longwave radiation atthe top of atmosphere; SH: sensible heat flux at the surface; LH: latent heat flux

at the surface; EP: explicit precipitation; CP: convective precipitation

Fig. 7. Observed (left) and Weather Research and Forecasting (WRF)-simulated (25 km) monthly mean precipitation with default (middle) and optimal parameters (right) over the North American Monsoon (NAM) region for July 1991

Clim Res 59: 135–147, 2014

mum rainfall center over the monsoon core region inthe Sierra Madre Occidental (SMO), but evidentlyoverestimates the total precipitation over the north-east corner of the domain. When the optimal param-eters were applied, the spatial pattern of simulatedprecipitation is much improved, and the wet bias inthe northeast corner is significantly reduced. Theoverall E is reduced from 110 in the default simula-tion to 53 in the optimal one.

Fig. 8 compares the sensitivity of selected physicalvariables over SGP and NAM. Overall, the patternsof parameter sensitivity are similar between NAMand SGP, but there are a few differences: (1) theimpact of parameter Pc on explicit and convectiveprecipitation is greater in NAM than SGP; (2) thesign of correlation coefficient between low-levelhumidity and each para meter is opposite in NAMand SGP; and (3) the impact of Pe on the surface andlow-level air temp erature and the downward short-wave radiation flux is not as significant in NAM as inSGP. As in SGP, none of the variables in NAM is sen-sitive to the Pt.

3.4. Dependence on radiation parameterizationschemes

In the preceding experiments, the RRTMG radia-tion scheme is used, where precipitation is overesti-mated with the default parameter setting in both SGP

and NAM. Essentially, the model parameters in thoseexperiments (refer to Sections 3.1, 3.2 and 3.3) weretuned to reduce the overestimated precipitation. AsYang et al. (2012) pointed out, the precipitation overSGP is underestimated compared to observationwhen the CAM radiation scheme is used with KFwith the default convective parameters in the WRFmodel. It is interesting to examine the model sensitiv-ity and calibration based on the same model but witha different precipitation climatology that results fromthe use of a different radiation scheme. We per-formed an additional CAM25 experiment in whichthe CAM radiation package was applied at 25 kmgrid spacing over the SGP region (see Table 1).

Fig. S4 in the Supplement presents the response of E(left panels) and the simulated precipitation (right 3panels) to the 5 convective parameters. Among allthe simulations in CAM25, E varies from 390 to 69.As with SGP25, E decreases as Pd and Ph increase orPe decreases. Larger Pd or Ph and lower Pe favor a bet-ter performance of the simulation. The values of theoptimized parameters for Pd, Pe, Ph, and Pc are 0.92,−0.99, 306, and 3858, respectively (Table 3), similar tothe results of the simulations that used the RRTMGradiation scheme. The response of model perform-ance is generally consistent with that of SGP25, andmany common features can be found in the two setsof simulations.

The explicit, convective, and total precipitationsvary at ~0.2–0.8, 2.0–5, and 3.0–6.0 mm d−1, respec-

tively, and their ranges are smallerthan those of the simulations with theRRTMG scheme. Compared withother experiments, the convectiveprecipitation de creases with Pd at amuch larger rate. The precipitationresponses to the other 4 parameters,especially Pe, are less sensitive inCAM25 than in the simulations withRRTMG. Again, as in SGP25, the con-vective precipitation dominates thetotal precipitation amount in CAM25.

Fig. 9 shows the spatial distribu-tions of precipitation from the obser-vation and the simulations with de -fault and optimal parameters. Bothsimulations considerably underesti-mate the maximum intensity of pre-cipitation against observation. Theoptimized simulation improves thespatial pattern of precipitation by re -moving the anomalous rainy areas inthe eastern boundary. However, it

144

Fig. 8. Relative sensitivities of the response of selected output variables (y-axis) to the 5 convection parameterization scheme (CPS) input parametersshown in Table 2 (x-axis) over the Southern Great Plains (SGP) and NorthAmerican Monsoon (NAM) regions, showing results for the experimentSGP25 with 25 km grid spacing. See Fig. 6 for explanation of output variable

abbreviations and ‘+’, ‘–’

Yan et al.: Parametric sensitivity in the WRF model

decreases the total rainfall amount over the wholeregion from 3.1 mm d−1 in the default simulation to1.9 mm d−1 in the optimal one. The calibration pro-cess reduces values of E. Indeed, E is re duced from110 in the default simulation to 69 in the optimal one.Still, the maximum precipitation band is underesti-mated in the optimal simulation, which could belargely related to the structural error associated withthe radiation package. Further studies targeted toreduce model bias or structural errors associatedwith the missed or misrepresented physical pro-cesses and potential problems in the nested model-ing framework and initial and boundary conditionsare needed in the future.

4. SUMMARY AND DISCUSSION

In this study, several issues related to parametricsensitivity and calibration of physical parameteriza-tions in climate model are investigated. We focus onthe KF convection parameterization in the WRF re-gional climate model. Five key input parameters inthe KF convection scheme are identified to investi-gate the response and sensitivity of model perform-

ance. To obtain the optimal parameter set accordingto the cost function E constrained by observed pre-cipitation, the SSAA algorithm is employed to samplethe parameters at the 5-dimensional space itera -tively. The parametric sensitivity and calibration pro-cesses across different spatial scales, climatic re-gions/ regimes, and radiation schemes are compared.

The results show that E decreases (i.e. performanceimproves) as Pd and Ph increase or Pe decreases. Themodel performances are found to be more sensitiveto Pd, Pe and Ph. The responses of E to changes inthese 3 parameters, as well as Pt, are smaller in thecoarse resolution than fine resolution simulations.The explicit precipitation increases with spatial reso-lution because more clouds are resolved at finer res-olution. However, no significant difference can befound in the convective and total precipitation simu-lated at different resolutions.

Similar trends and variability of the simulated pre-cipitation in response to changes in the 5 parameterscan be found across 3 different spatial resolutions.While the overall trends of the response of physicalvariables to convective parameters are consistentacross different resolutions, remarkable differencesin the magnitude of the response can be foundamong the experiments at different horizontal reso-lutions. For example, the atmospheric humidity at alllevels is lower in the simulation at finer resolution.With the optimal parameters, the simulated spatialpattern and magnitude of precipitation are improvedat all 3 spatial resolutions, especially over the north-eastern boundary area. Generally, almost all physicalvariables show similar sensitivities to the parametersacross different spatial resolutions.

Overall, the patterns of parameter sensitivity aresimilar over NAM and SGP. Generally similar trendsand variability in model response are found at SGPversus NAM, although the response of E to Pe scat-ters more widely in NAM than in SGP. The ratio of

145

Fig. 9. Observed (left) monthly mean precipitation over Southern Great Plains (SGP) region, and simulated default (middle) and optimal values (right) for SGP with the CAM radiation scheme

Experiment Pd Pe Ph Pt Pc E E (default)

Default 0 0 150 5 2700 – –SGP12 0.98 −0.94 345 4.25 3914 76 165SGP25 0.97 −0.93 334 4.65 3585 70 139SGP50 0.94 −0.96 334 8.78 3867 66 148NAM25 0.93 −0.99 331 4.62 3386 53 110CAM25 0.92 −0.99 306 3.41 3858 69 110

Table 3. Optimal and default values of parameters shown inTable 2 and the cost function E. See Table 1 for details of

experiments

Clim Res 59: 135–147, 2014

explicit to total precipitation is larger over NAM thanSGP. The spatial pattern of simulated precipitation ismuch improved in the NAM simulation with the opti-mal parameters, and the model significantly reducesthe wet bias over the northeast areas.

We also examined the model sensitivity and cali-bration processes based on the same model but witha different precipitation climatology resulting fromthe use of a different radiation package when thedefault convective parameters are applied. The mo -del performance response was found to be generallyconsistent, regardless of the precipitation climatol-ogy (i.e. initial wet versus dry bias) and many com-mon features can be found in the 2 sets of simula-tions. However, the maximum precipitation band isstill underestimated in the optimal simulation, whichis likely related to structural error associated with theradiation package. The general reduction of precipi-tation in all experiments due to calibration is partlydriven by the large wet biases near the outflowboundary (i.e. northeast corner) and the use of equalweighting of model errors for all grid cells in definingE. Future studies are needed to target efforts on re -ducing model bias or structural errors associatedwith the missed or misrepresented physical processand addressing potential problems with the model-ing framework such as the larger systematic errorsdue to the nesting approach.

Another limitation in this study is that only ob -served precipitation data are used to evaluate modelperformance. Although Yang et al. (2013) found froma study where only convective precipitation is con-strained by observation that the improved convectionhas a positive impact on the general circulation andother aspects of global climate, the benefits of para -meter calibration on other variables at the processlevels (e.g. downdraft or entrainment mass flux) arestill not clear. While the tuning process may produceparameter settings that approximate the observedclimate, this outcome may not have been achievedthrough proper constraints of the balance of differentphysical processes. That is, the improved ultimatevariables, such as precipitation and temperature,could be a result of compensating errors, so suchoptimization may not guarantee better performancein projecting the future climate. In future studies, itmay be beneficial to calibrate model parameters byconstraining the behavior of physical processes (i.e.turbulence or shallow and deep convection pro-cesses) rather than simply reducing differencesbetween simulated and observed variables, such asprecipitation, that are the outcomes of delicate bal-ances among many processes.

Acknowledgements. This work was supported by theApplied Mathematics program of the US Department ofEnergy (DOE) Office of Science’s Advanced Scientific Com-puting Research program. The computations were per-formed using resources of the National Energy ResearchScientific Computing Center (NERSC) at Lawrence Berke-ley National Laboratory (LBNL). Pacific Northwest NationalLaboratory is operated by Battelle for the DOE under Con-tract DE-AC05-76RL01830.

LITERATURE CITED

Arakawa A, Jung J, Wu C (2011) Toward unification of themultiscale modeling of the atmosphere. Atmos ChemPhys 11: 3731−3742

Barker HW, Pincus R, Morcrette JJ (2003) The Monte-Carloindependent column approximation: application withinlarge-scale models. In: Proc GCSS/ARM Workshop onthe Representation of Cloud Systems in Large-ScaleModels. Kananaskis, Canada, 20−24 May 2002

Bechtold P, Bazile E, Guichard F, Mascart P, Richard E(2001) A mass-flux convection scheme for regional andglobal models. QJR Meteorol Soc 127: 869−886

Bechtold P, Kohler M, Jung T, Doblas-Reyes F and others(2008) Advances in simulating atmospheric variabilitywith the ECMWF model: from synoptic to decadal time-scales. QJR Meteorol Soc 134: 1337−1351

Berner J, Jung T, Palmer TN (2012) Systematic model error: the impact of increased horizontal resolution versusimproved stochastic and deterministic parameteriza-tions. J Clim 25: 4946−4962

Bowler NE, Arribas A, Mylne KR, Robertson KB, Beare SE(2008) The MOGREPS short-range ensemble predictionsystem. QJR Meteorol Soc 134: 703−722

Chen F, Dudhia J (2001) Coupling an advanced land sur-face-hydrology model with the Penn State-NCAR MM5modeling system. I: Model implementation and sensitiv-ity. Mon Weather Rev 129: 569−585

Collins WD, Rasch PJ, Boville BA, Hack JJ and others (2004)Description of the NCAR community atmosphere model(CAM 3.0). Technical Note TN-464+ STR, National Cen-ter for Atmospheric Research, Boulder, CO

Done JM, Craig GC, Gray SL, Clark PA, Gray MEB (2006)Mesoscale simulations of organized convection: impor-tance of convective equilibrium. QJR Meteorol Soc 132: 737−756

Hack JJ (1994) Parameterization of moist convection in theNational Center for Atmospheric Research communityclimate model (CCM2). J Geophys Res 99: 5551−5568

Hacker JP, Ha SY, Snyder C, Berner J and others (2011a) TheUS Air Force Weather Agency’s mesoscale ensemble: scientific description and performance results. Tellus A63: 625−641

Hacker JP, Snyder C, Ha SY, Pocernich M (2011b) Linearand non-linear response to parameter variations in ameso scale model. Tellus A 63: 429−444

Ingber L (1989) Very fast simulated re-annealing. MathComput Model 12: 967−973

Jackson C, Sen MK, Stoffa PL (2004) An efficient stochasticBayesian approach to optimal parameter and uncertaintyestimation for climate model predictions. J Clim 17: 2828−2841

Jackson CS, Sen MK, Huerta G, Deng Y, Bowman KP (2008)Error reduction and convergence in climate prediction.

146

Yan et al.: Parametric sensitivity in the WRF model 147

J Clim 21: 6698−6709Janji ZI (2002) Nonsingular implementation of the Mellor-

Yamada level 2.5 scheme in the NCEP Meso model.NCEP Office Note 437, National Centers for Environ-mental Prediction, Boulder, CO

Kain JS (2004) The Kain-Fritsch convective parameteriza-tion: an update. J Appl Meteorol 43: 170−181

Kain JS, Fritsch JM (1990) A one-dimensional entraining/detraining plume model and its application in convectiveparameterization. J Atmos Sci 47: 2784−2802

Koster RD, Dirmeyer PA, Guo ZC, Bonan G and others(2004) Regions of strong coupling between soil moistureand precipitation. Science 305: 1138−1140

Liang F (2011) Annealing evolutionary stochastic approxi-mation Monte Carlo for global optimization. Stat Comput21: 375−393

Liang F, Cheng Y, Lin G (2013) Simulated stochastic approx-imation annealing for global optimization with s square-root cooling schedule. J Am Stat Assoc

Maurer E, Wood A, Adam J, Lettenmaier D, Nijssen B (2002)A long-term hydrologically based dataset of land surfacefluxes and states for the conterminous United States.J Clim 15: 3237−3251

Mlawer EJ, Taubman SJ, Brown PD, Iacono MJ, Clough SA(1997) RRTM, a validated correlated-k model for thelongwave. J Geophys Res 102: 16663−16682

Morrison H, Curry J, Khvorostyanov V (2005) A new double-moment microphysics parameterization for application incloud and climate models. I. Description. J Atmos Sci 62: 1665−1677

Pincus R, Barker HW, Morcrette JJ (2003) A fast, flexible,approximate technique for computing radiative transferin inhomogeneous cloud fields. J Geophys Res 108: 4376

Qian Y, Gustafson WI Jr, Fast JD (2010) An investigation ofthe sub-grid variability of trace gases and aerosols forglobal climate modeling. Atmos Chem Phys 10: 6917−6946

Qian Y, Huang M, Yang B, Berg LK (in press) (2013) model-ing study of irrigation effects on surface fluxes and land-

air-cloud interactions in the Southern Great Plains.J Hydrometeor 14: 700−721

Redelsperger JL, Parsons DB, Guichard F (2002) Recoveryprocesses and factors limiting cloud-top height followingthe arrival of a dry intrusion observed during TOGACOARE. J Atmos Sci 59: 2438−2457

Reynolds CA, Ridout JA, McLay JG (2011) Examination ofparameter variations in the U S Navy Global Ensemble.Tellus A 63: 841−857

Skamarock W, Klemp J, Dudhia J, Gill D and others (2008)A description of the advanced research WRF version 3.Technical Note NCAR/TN−468+STR, National Centerfor Atmospheric Research, Boulder, CO

Taylor CM, Ellis RJ (2006) Satellite detection of soil moistureimpacts on convection at the mesoscale. Geophys ResLett 33: L03404, doi: 10.1029/2005GL025252

van Lier-Walqui M, Vukicevic T, Posselt DJ (2012) Quantifi-cation of cloud microphysical parameterization uncer-tainty using radar reflectivity. Mon Weather Rev 140: 3442−3466

Warren SG, Schneider SH (1979) Seasonal simulation as atest for uncertainties in the parameterizations of aBudyko-Sellers zonal climate model. J Atmos Sci 36: 1377−1391

Yang B, Qian Y, Lin G, Leung R, Zhang Y (2012) Some issuesin uncertainty quantification and parameter tuning: acase study of convective parameterization scheme in theWRF regional climate model. Atmos Chem Phys 12: 2409−2427

Yang B, Qian Y, Lin G, Leung LR and others (2013) Uncer-tainty quantification and parameter tuning in the CAM5Zhang-McFarlane convection scheme and impact ofimproved convection on the global circulation and cli-mate. J Geophys Res Atmos 118: 395−415

Zhu C, LR Leung, D Gochis, Y Qian, and DP Lettenmaier(2009) Evaluating the influence of antecedent soil mois-ture on variability of the North American Monsoon pre-cipitation in the coupled MM5/VIC modeling system.J Adv Model Earth Syst 1: 13,

Editorial responsibility: Filippo Giorgi, Trieste, Italy

Submitted: May 27, 2013; Accepted: December 4, 2013Proofs received from author(s):March 15, 2014