recent advances in da at ncep - bureau of meteorology advances in da at ncep ... globally for airs...
TRANSCRIPT
Recent advances in DA at NCEP
“Where America’s Climate, Weather, Ocean and Space Weather Services Begin”
December 8, 1016
Presented by John Derber
National Centers for Environmental Prediction
Recent Highlight
• Upgrade of global DA system (12 May 2016)
– 4D-hybrid en-var system
– Use of All-Sky AMSU-A radiances
– Upgrade of CRTM
– Bias correction of Aircraft
– Additional observations
– Some model changes
2
Future
• Improved forecast model– FV3 implementation in global/regional
• Coupled Data Assimilation• Improved techniques in assimilation system.
– Improved representation of model error using ensembles– Improved balance in initial state (especially with moisture variables)
• JEDI framework – long term direction• Improved use of observing system
– Addition of new data sources• GOES-R• JPSS-1• Additional aircraft observations• COSMIC-2• Etc.
– Observational handling and database– Forward models for observations
3
Talk focus
• Lots of presentations on techniques here. Fewer on
observations, so I will focus more on observations, but all
aspects important.
• All types of observational data can be used better.
• Details, Details, Details.
4
JEDI
• Joint Effort for Data assimilation Integration
(JEDI)
• Similar to idea of ECMWF OOPS system.
• Longer term direction – but parts will be included as they
are ready
• Phase I
– Unified forward operators
– Observational data base
5
Joint Effort for Data assimilation Integration(JEDI)
STRATEGY1. Collective path to unification, while allowing multiple levels of engagement
2. Modular, Object-Oriented code for flexibility, robustness and optimization
3. Mutualize model-agnostic components across
• Applications (atmosphere, ocean, land, aerosols, strongly coupled, etc.)
• Models & Grids (operational/research, regional/global models)
• Observations (past, current and future)
OBJECTIVES1. Facilitate innovative developments to address DA grand challenges
2. Increase R2O transition rate from community
3. Increase science productivity and code performance
JEDI + Academia
Obs. Pre-processor• Reading
• Data selection
• Basic QC
Solver• Variational/EnKF
• Hybrid
Background &
Background Error
Observations
Model
Unified
Forward
Operator
(UFO)
• Model Initial Conditions
DATA ASSIMILATION COMPONENTS for Atmosphere, Ocean, Waves, Sea-ice,
Land, Aerosols, Chemistry, Hydrology,
Ionosphere
DATA ASSIMILATION COMPONENTS for Atmosphere, Ocean, Waves, Sea-ice,
Land, Aerosols, Chemistry, Hydrology,
Ionosphere
Analysis Increments
Read
Model
Interpolate
ObserverCRTM, Bias Correction,
QC, Cloud Detection, etc.
Write
Obs. TypeObs.
Locations
(4D)
Model(s)
NEMS /
ESMF
Couple
r (model values @obs. locations)
[Jacobian, Revised QC, Obs. Error, Bias, …]H(xk)
Model
Options
Observer
Options
Model / Obs. Type
MatchingLook-up table
locstreams
Unified
Forward
Operator
NEMS/ESMF
Atm Dycore(TBD)
Wave(WW3/SWAN)
Sea Ice
(CICE/SIS2/KISS)
Aerosols(GOCART)
Ocean(HYCOM/MOM)
Land Surface(NOAH)
Atm Physics(GFS)
Atm DA(GSI)
(model equivalent)
(observations)
Observational database
• Preprocessing - everything that happens to data before it gets to the DA system
• Communications
– Volume issues with some data sources
• Metadata
• Station history for monitoring and quality control
• Restricted data
• Transition to BUFR for conventional observations
• Radiation correction with Radiosondes
• Specification of observation error (instrument and representativeness)
– Preliminary quality control – reject lists
– Situational dependence
– Correlated error
9
Transition to BUFR reporting
• Ingleby, B., P. Pauley, A. Kats, J. Ator, D. Keyser, A. Doerenbecher, E. Fucile, J. Hasegawa, E. Toyoda, T. Kleinert, W. Qu, J. St James, W. Tennant, and R. Weedon, 2016: Progress towards high-resolution, real-time radiosonde reports. Bull. Amer. Meteor. Soc. doi:10.1175/BAMS-D-15-00169.1, in press.
• In long term, more accurate – complete observations, but many difficulties.
– Inclusion of balloon drift, station location, all levels in one report.
– Change in paradigm from significant levels to frequent reports.
• NCEP behind some centers in using BUFR data.
10
Transition to BUFR reporting
12
Black – standard levels
Blue – significant levels
Red – BUFR levels
14 German stations
using Vaisala RS92
Rawinsonde bias correction
• More automation and bias corrections by the
instrument producers make the need for bias
correction less.
• However, many radiosondes still need bias correction.
• The NCEP bias correction tables have not been
updated for many years. Many new types.
• Project underway to use collocated GPS retrievals of
temperature (above 100hPa) to create new bias
correction.
13
Quality control & data monitoring.
• Improved non-linear (variational) quality control techniques in analysis can reduce impact of bad observations.
• Observations rejected by setting observation error to ∞, non-linear QC will down-weight observations
• Still need better monitoring (and feedback to source) to ensure that known bad observations are not used and eventually corrected.
• Improved reporting techniques can eventually reduce the number of bad observations.
• Have to do in way that allows input from desk meteorologist, does not add risk to system, and still allows needed flexibility.
15
Observational error specification
• Moving towards station by station and situation dependent observation error specification
• Separate instrument and representativeness error (including bias)
• Representativeness error will be modeled and dependent on forecast model/resolution/etc.
• Inclusion of correlated error
– Or using correlated error to chose what data to use.
– Impact on convergence is an issue.
– Situation dependent correlated error.
16
Improvements to observation error
specification and bias correction
• Increased granularity of observational errors
– Initially by instrument type
– Eventually observation by observation
• Separate modeling of instrument and
representativeness error
• Inclusion of correlated errors for satellite radiances
(GMAO, Bathmann and Collard)
– Reduction in specified observational error variance
– Regularization of covariance matrix
Observation error correlation matrices for AIRS over sea, before (left) and after reconditioning R (right). R is reconditioned by first setting the smallest eigenvalues equal to λmax/K1 and then inflating the diagonal. Here K1=150.
Observation error correlation Matrices for AIRS over land, before (left) and after reconditioning R (right). R is reconditioned by first setting the smallest eigenvalues equal to λmax/K1 and then inflating the diagonal. Here K1=150.
Observation errors for AIRS over sea, before and after reconditioning R. R is reconditioned by first setting the smallest eigenvalues equal to λmax/K1 and then inflating the diagonal. Here K1=150.
The cost function (left) and log of the gradient (right) during minimization. Green data points represent the result of using full R for AIRS and IASI globally, while black data points represent the result of using diagonal R.
Analysis increment RMS differences after using a full R globally for AIRS and IASI in a 2 month parallel GFS experiment
Temperature fit to observations (O minus F) for the 2 month experiment. Dotted lines indicate full R results,
while solid lines indicate the results of using a diagonal R.
Humidity fit to observations (O minus F) for the 2 month experiment. Dotted lines indicate full R
results, while solid lines indicate the results of using a diagonal R.
The 500 mb geopotential height anomaly correlation in the northern
hemisphere (left), southern hemisphere (right) after using a full R
globally for AIRS and IASI in a 2 month parallel GFS experiment.
Forward models
• Transforms analysis or model variables to the observations.
• Satellite observations
– Radiative transfer - RTTOV/CRTM – brightness temperatures/radiances
• Clouds and precipitation
• Surface emissivity
• Atmospheric composition (gases and aerosols)
• FOV size and path
– GPS-RO – bending angles
27
Surface emissivity issues under
scattering conditions – reflection of diffuse radiation and restricting to < 60 degrees
CRTM CRTM
RTTOV
Original Work-around
AMSU-A
Channel 3 Observation minus First-Guess
Forward model
• Conventional observations
– Wind components to radial winds
– Near surface effects on observations
– Model layers vs. point observations
• Improvements in the forward model should allow
reduction in representativeness error (σ and/or
bias)
29
Final comments
• All parts of a data assimilation system are important –all aspects can be improved.
• Our DA system is being redesigned based on JEDI concept to allow the increased use of information in the observations.
• Data handling, use of metadata, improved quality control and improved specification of observational errors in addition to development of better forward models for all observation types are essential to improve the extraction of information from observations.
30