4d-var for dummies - australia's official weather ... · 4d-var for dummies jeff kepert...
TRANSCRIPT
![Page 1: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/1.jpg)
4D-Var for Dummies
www.cawcr.gov.au
Jeff KepertBureau of Meteorology Research and DevelopmentCAWCR Student Workshop, December 1 2016
![Page 2: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/2.jpg)
![Page 3: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/3.jpg)
I 1854: Meteorological Dept of the British Board of Tradecreated
I “. . . in a few years . . . we might know in this metropolis thecondition of the weather 24 hours beforehand.” (M. J. BallMP, House of Commons, 30 June 1854.)
I Response from House: “Laughter”
![Page 4: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/4.jpg)
I 1854: Meteorological Dept of the British Board of Tradecreated
I “. . . in a few years . . . we might know in this metropolis thecondition of the weather 24 hours beforehand.” (M. J. BallMP, House of Commons, 30 June 1854.)
I Response from House: “Laughter”
![Page 5: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/5.jpg)
I 1854: Meteorological Dept of the British Board of Tradecreated
I “. . . in a few years . . . we might know in this metropolis thecondition of the weather 24 hours beforehand.” (M. J. BallMP, House of Commons, 30 June 1854.)
I Response from House: “Laughter”
![Page 6: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/6.jpg)
![Page 7: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/7.jpg)
![Page 8: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/8.jpg)
Why Data Assimilation is Important
I Numerical Weather Prediction (NWP) is (largely) an initialvalue problem.
I Has contributed to enormous forecast improvementsI Extracts the maximum value from expensive observations
I Accurate analyses are necessary for getting the most fromfield programs.
I Reanalyses of past data using modern methods are anessential resource for climate research.
![Page 9: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/9.jpg)
Why Data Assimilation is Important
I Numerical Weather Prediction (NWP) is (largely) an initialvalue problem.
I Has contributed to enormous forecast improvementsI Extracts the maximum value from expensive observations
I Accurate analyses are necessary for getting the most fromfield programs.
I Reanalyses of past data using modern methods are anessential resource for climate research.
![Page 10: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/10.jpg)
Why Data Assimilation is Important
I Numerical Weather Prediction (NWP) is (largely) an initialvalue problem.
I Has contributed to enormous forecast improvementsI Extracts the maximum value from expensive observations
I Accurate analyses are necessary for getting the most fromfield programs.
I Reanalyses of past data using modern methods are anessential resource for climate research.
![Page 11: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/11.jpg)
Best Linear Unbiased Estimate (BLUE)
I Observations y1 and y2 of a true state xt :
y1 =xt + ε1 y2 =xt + ε2
I The statistical properties of the errors are known:
〈ε1〉 = 0 〈ε21〉 = σ21 〈ε1ε2〉 = 0
〈ε2〉 = 0 〈ε22〉 = σ22
I Estimate xa of xt as a linear combination of theobservations such that 〈xa〉 = xt (unbiased) andσ2
a = 〈(xa − xt)2〉 is minimised (best).
I Then
xa =σ2
2y1 + σ21y2
σ21 + σ2
2
![Page 12: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/12.jpg)
Best Linear Unbiased Estimate (BLUE)
I Observations y1 and y2 of a true state xt :
y1 =xt + ε1 y2 =xt + ε2
I The statistical properties of the errors are known:
〈ε1〉 = 0 〈ε21〉 = σ21 〈ε1ε2〉 = 0
〈ε2〉 = 0 〈ε22〉 = σ22
I Estimate xa of xt as a linear combination of theobservations such that 〈xa〉 = xt (unbiased) andσ2
a = 〈(xa − xt)2〉 is minimised (best).
I Then
xa =σ2
2y1 + σ21y2
σ21 + σ2
2
![Page 13: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/13.jpg)
Best Linear Unbiased Estimate (cont’d)
I Same estimate found by minimising
J(xa) =(xa − y1)
2
σ21
+(xa − y2)
2
σ22
I Minimising J is the same as maximising exp(−J/2)I Hence for Gaussian errors the BLUE is the maximum
likelihood (or optimal) estimate.I For many pieces of data y = (y1, y2, . . . , yn)
T ,
J(xa) = (xa − y)T P−1(xa − y)
where P is the error covariance matrix of y.
![Page 14: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/14.jpg)
Best Linear Unbiased Estimate (cont’d)
I Same estimate found by minimising
J(xa) =(xa − y1)
2
σ21
+(xa − y2)
2
σ22
I Minimising J is the same as maximising exp(−J/2)I Hence for Gaussian errors the BLUE is the maximum
likelihood (or optimal) estimate.I For many pieces of data y = (y1, y2, . . . , yn)
T ,
J(xa) = (xa − y)T P−1(xa − y)
where P is the error covariance matrix of y.
![Page 15: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/15.jpg)
Best Linear Unbiased Estimate (cont’d)
I Same estimate found by minimising
J(xa) =(xa − y1)
2
σ21
+(xa − y2)
2
σ22
I Minimising J is the same as maximising exp(−J/2)I Hence for Gaussian errors the BLUE is the maximum
likelihood (or optimal) estimate.I For many pieces of data y = (y1, y2, . . . , yn)
T ,
J(xa) = (xa − y)T P−1(xa − y)
where P is the error covariance matrix of y.
![Page 16: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/16.jpg)
Assimilation: The Big BLUE
Assimilation combines a short-term numerical forecast withsome observations:
J(xa) = (xa − xf )T B−1(xa − xf ) + (H(xa)− y)T R−1(H(xa)− y)
I xa is the analysisI xf the short-term forecast (a.k.a. first-guess, background)I y are the observationsI H produces the analysis estimate of the observed valuesI R is the observation-error covarianceI B is the forecast-error covariance
![Page 17: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/17.jpg)
Atmospheric Infrared Transmission Spectrum
![Page 18: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/18.jpg)
HIRS Channel Weights
![Page 19: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/19.jpg)
Finding the minimum of J
J(xa) = (xa − xf )T B−1(xa − xf ) + (H(xa)− y)T R−1(H(xa)− y)
Solve directly ∇J = 0 (a.k.a. optimum interpolation).I Have to manipulate big matricesI Nonlinear H is very difficult (satellite radiances)
Iterative minimisation (a.k.a. variational assimilation).I Finds full 3-D structure of the atmosphere (3D-Var)I Other observations and background helps constrain the
poorly-conditioned and underdetermined inversion of thesatellite radiances
![Page 20: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/20.jpg)
Finding the minimum of J
J(xa) = (xa − xf )T B−1(xa − xf ) + (H(xa)− y)T R−1(H(xa)− y)
Solve directly ∇J = 0 (a.k.a. optimum interpolation).I Have to manipulate big matricesI Nonlinear H is very difficult (satellite radiances)
Iterative minimisation (a.k.a. variational assimilation).I Finds full 3-D structure of the atmosphere (3D-Var)I Other observations and background helps constrain the
poorly-conditioned and underdetermined inversion of thesatellite radiances
![Page 21: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/21.jpg)
Minimising J
J(xa) = (xa − xf )T B−1(xa − xf ) + (H(xa)− y)T R−1(H(xa)− y)
To minimise J, we need the gradient:
∇J(xa) = 2B−1(xa − xf ) + 2HT R−1(H(xa)− y)
H =[∂Hi∂xa,j
]is the Jacobian of H (a.k.a. the tangent linear)
HT is the adjoint of H
![Page 22: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/22.jpg)
Minimising J
J(xa) = (xa − xf )T B−1(xa − xf ) + (H(xa)− y)T R−1(H(xa)− y)
To minimise J, we need the gradient:
∇J(xa) = 2B−1(xa − xf ) + 2HT R−1(H(xa)− y)
H =[∂Hi∂xa,j
]is the Jacobian of H (a.k.a. the tangent linear)
HT is the adjoint of H
![Page 23: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/23.jpg)
The Importance of B
J(xa) = (xa − xf )T B−1(xa − xf ) + (H(xa)− y)T R−1(H(xa)− y)
B is important:I Conditioning and speed of convergenceI Getting the statistics rightI Describing atmospheric balanceI Spatial scale of analysis
B in the model variables fails miserably:I Rank deficientI Too large to store, let alone operate on
![Page 24: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/24.jpg)
The Importance of B
J(xa) = (xa − xf )T B−1(xa − xf ) + (H(xa)− y)T R−1(H(xa)− y)
B is important:I Conditioning and speed of convergenceI Getting the statistics rightI Describing atmospheric balanceI Spatial scale of analysis
B in the model variables fails miserably:I Rank deficientI Too large to store, let alone operate on
![Page 25: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/25.jpg)
B the best you can B
Representing B typically involves:I Transform to less-correlated variables.
I (u, v) =⇒ (ψ, χ)I u = −∂ψ/∂y + ∂χ/∂x , v = ∂ψ/∂x + ∂χ/∂yI Replace mass field by unbalanced mass:φunbal = φ− φbal(ψ)
I Transform to spectral space.I Rescale.
These make B diagonal =⇒ good conditioning andcomputational efficiency.
I Truncate the small scales. Forecast error spectrum is red,with little power at small scales. So truncate B.
![Page 26: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/26.jpg)
B the best you can B
Representing B typically involves:I Transform to less-correlated variables.
I (u, v) =⇒ (ψ, χ)I u = −∂ψ/∂y + ∂χ/∂x , v = ∂ψ/∂x + ∂χ/∂yI Replace mass field by unbalanced mass:φunbal = φ− φbal(ψ)
I Transform to spectral space.I Rescale.
These make B diagonal =⇒ good conditioning andcomputational efficiency.
I Truncate the small scales. Forecast error spectrum is red,with little power at small scales. So truncate B.
![Page 27: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/27.jpg)
B the best you can B
Representing B typically involves:I Transform to less-correlated variables.
I (u, v) =⇒ (ψ, χ)I u = −∂ψ/∂y + ∂χ/∂x , v = ∂ψ/∂x + ∂χ/∂yI Replace mass field by unbalanced mass:φunbal = φ− φbal(ψ)
I Transform to spectral space.I Rescale.
These make B diagonal =⇒ good conditioning andcomputational efficiency.
I Truncate the small scales. Forecast error spectrum is red,with little power at small scales. So truncate B.
![Page 28: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/28.jpg)
B the best you can B
Representing B typically involves:I Transform to less-correlated variables.
I (u, v) =⇒ (ψ, χ)I u = −∂ψ/∂y + ∂χ/∂x , v = ∂ψ/∂x + ∂χ/∂yI Replace mass field by unbalanced mass:φunbal = φ− φbal(ψ)
I Transform to spectral space.I Rescale.
These make B diagonal =⇒ good conditioning andcomputational efficiency.
I Truncate the small scales. Forecast error spectrum is red,with little power at small scales. So truncate B.
![Page 29: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/29.jpg)
B the best you can B
Representing B typically involves:I Transform to less-correlated variables.
I (u, v) =⇒ (ψ, χ)I u = −∂ψ/∂y + ∂χ/∂x , v = ∂ψ/∂x + ∂χ/∂yI Replace mass field by unbalanced mass:φunbal = φ− φbal(ψ)
I Transform to spectral space.I Rescale.
These make B diagonal =⇒ good conditioning andcomputational efficiency.
I Truncate the small scales. Forecast error spectrum is red,with little power at small scales. So truncate B.
![Page 30: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/30.jpg)
B the best you can B
Representing B typically involves:I Transform to less-correlated variables.
I (u, v) =⇒ (ψ, χ)I u = −∂ψ/∂y + ∂χ/∂x , v = ∂ψ/∂x + ∂χ/∂yI Replace mass field by unbalanced mass:φunbal = φ− φbal(ψ)
I Transform to spectral space.I Rescale.
These make B diagonal =⇒ good conditioning andcomputational efficiency.
I Truncate the small scales. Forecast error spectrum is red,with little power at small scales. So truncate B.
![Page 31: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/31.jpg)
B the best you can B
Representing B typically involves:I Transform to less-correlated variables.
I (u, v) =⇒ (ψ, χ)I u = −∂ψ/∂y + ∂χ/∂x , v = ∂ψ/∂x + ∂χ/∂yI Replace mass field by unbalanced mass:φunbal = φ− φbal(ψ)
I Transform to spectral space.I Rescale.
These make B diagonal =⇒ good conditioning andcomputational efficiency.
I Truncate the small scales. Forecast error spectrum is red,with little power at small scales. So truncate B.
![Page 32: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/32.jpg)
Incremental Formulation
Replace
J(xa) = (xa − xf )T B−1(xa − xf ) + (H(xa)− y)T R−1(H(xa)− y)
by
J(δx) = δxT B−1δx + (H(xf ) + Hδx− y)T R−1(H(xf ) + Hδx− y)
where δx = xa− xf and H is the Jacobian of H (tangent linear).I H(xa) becomes H(xf ) + HδxI Computational efficiency since
I δx now at reduced resolution of B,I Hδx maybe cheaper to compute than H(xa),I true quadratic form.
![Page 33: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/33.jpg)
A Matter of Time
So far all data assumed to be at the analysis time.I Assimilate e.g. four times a day.I All data in 6-hour window assumed to occur at the middle
of that window.I Introduces some errors =⇒ weather systems move and
develop!I Reduce errors by assimilating more frequently, but that has
its own problems.
A better way is to introduce the time dimension into theassimilation, 4-dimensional variational assimilation (4D-Var).
![Page 34: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/34.jpg)
A Matter of Time
So far all data assumed to be at the analysis time.I Assimilate e.g. four times a day.I All data in 6-hour window assumed to occur at the middle
of that window.I Introduces some errors =⇒ weather systems move and
develop!I Reduce errors by assimilating more frequently, but that has
its own problems.A better way is to introduce the time dimension into theassimilation, 4-dimensional variational assimilation (4D-Var).
![Page 35: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/35.jpg)
Observations at two times
Red: Observations. Blue: 3D-Var. Green: 4D-Var.
![Page 36: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/36.jpg)
4D-Var
Add a term for the later time:
J(xa) = . . .+ (H2(M(xa))− y2)T R−1
2 (H2(M(xa))− y2)
I M is the model forecast from t1 to t2I Subscripts 2 refer to the time t2.
The gradient becomes
∇J(xa) = . . .+ 2MT HT2 R−1
2 (H2(M(xa))− y2)
![Page 37: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/37.jpg)
4D-Var
Add a term for the later time:
J(xa) = . . .+ (H2(M(xa))− y2)T R−1
2 (H2(M(xa))− y2)
I M is the model forecast from t1 to t2I Subscripts 2 refer to the time t2.
The gradient becomes
∇J(xa) = . . .+ 2MT HT2 R−1
2 (H2(M(xa))− y2)
![Page 38: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/38.jpg)
4D-Var
Add a term for the later time:
J(xa) = . . .+ (H2(M(xa))− y2)T R−1
2 (H2(M(xa))− y2)
I M is the model forecast from t1 to t2I Subscripts 2 refer to the time t2.
The gradient becomes
∇J(xa) = . . .+ 2MT HT2 R−1
2 (H2(M(xa))− y2)
![Page 39: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/39.jpg)
4D-Var
J(xa) = . . .+ (H2(M(xa))− y2)T R−1
2 (H2(M(xa))− y2)
∇J(xa) = . . .+ 2MT HT2 R−1
2 (H2(M(xa))− y2)
I HT2 is the adjoint of the Jacobian of H, takes information
about the observation-analysis misfit from radiance spaceto analysis space
I MT is the adjoint of the Jacobian ofM and propagates thisgradient information back in time from t2 to t1.
![Page 40: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/40.jpg)
4D-Var
Minimising
J(xa) = (xa − xf )T B−1(xa − xf )
+ (H(xa)− y)T R−1(H(xa)− y)
+ (H2(M(xa))− y2)T R−1
2 (H2(M(xa))− y2)
gives an analysis xa at time t1 thatI is close to the background xf at t1I is close to the observations y at t1I initialises a (linearised) forecast that is close to the
observations y2 at time t2Adding additional time levels is straightforward, as is theincremental formulation (exercise).
![Page 41: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/41.jpg)
4D-Var analysis of a single pressure observation
One pressure observation at centre of low, 5 hPa belowbackground, at end of 6-hr assimilation window.
![Page 42: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/42.jpg)
4D-Var analysis of a single pressure observation
One pressure observation at centre of low, 5 hPa belowbackground, at end of 6-hr assimilation window.
MSLP analysisincrement at end of6-hr assimilationwindow. Gustaffson
(2007, Tellus)
![Page 43: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/43.jpg)
4D-Var analysis of a single pressure observation
One pressure observation at centre of low, 5 hPa belowbackground, at end of 6-hr assimilation window.
NW-SE section oftemperature and windincrements at start of6-hr assimilationwindow.
![Page 44: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/44.jpg)
In practice ...
This is not a small problem!I Atmospheric model has O(106 to 107) variablesI Millions of observations per dayI Limited time available under operational constraints
The model has several hundred thousand lines of code, 4D-Varrequires
I operations by the Jacobian of the modelI operations by the adjoint of the Jacobian
Good results require accurately estimating the necessarystatistics (R and B) and careful quality control of theobservations.
![Page 45: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/45.jpg)
In practice ...
This is not a small problem!I Atmospheric model has O(106 to 107) variablesI Millions of observations per dayI Limited time available under operational constraints
The model has several hundred thousand lines of code, 4D-Varrequires
I operations by the Jacobian of the modelI operations by the adjoint of the Jacobian
Good results require accurately estimating the necessarystatistics (R and B) and careful quality control of theobservations.
![Page 46: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/46.jpg)
In practice ...
This is not a small problem!I Atmospheric model has O(106 to 107) variablesI Millions of observations per dayI Limited time available under operational constraints
The model has several hundred thousand lines of code, 4D-Varrequires
I operations by the Jacobian of the modelI operations by the adjoint of the Jacobian
Good results require accurately estimating the necessarystatistics (R and B) and careful quality control of theobservations.
![Page 47: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/47.jpg)
Extensions
Multiple “Outer Loops”I Problem: Accuracy is limited by the linearisations of H andM.
I Solution: Update the nonlinear forecast (outer loop) severaltimes during the minimisation of the J(δx) (inner loop).
Multi-incremental 4D-VarI Problem: Balancing speed of convergence against need to
resolve small scales.I Solution: Begin minimising with δx at low resolution, and
increase resolution after each iteration of the outer loop.
![Page 48: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/48.jpg)
Extensions
Multiple “Outer Loops”I Problem: Accuracy is limited by the linearisations of H andM.
I Solution: Update the nonlinear forecast (outer loop) severaltimes during the minimisation of the J(δx) (inner loop).
Multi-incremental 4D-VarI Problem: Balancing speed of convergence against need to
resolve small scales.I Solution: Begin minimising with δx at low resolution, and
increase resolution after each iteration of the outer loop.
![Page 49: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/49.jpg)
Extensions: Weak Constraint 4D-Var
I Doesn’t assume that the model is perfectI Allows a longer window.
![Page 50: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/50.jpg)
Extensions: Weak Constraint 4D-Var
I Doesn’t assume that the model is perfectI Allows a longer window.
![Page 51: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/51.jpg)
Summary
Var is better than direct solution (a.k.a. Optimum Interpolation)because:
I Can handle lots of observationsI Can better cope with nonlinear observation operator HI Solves for the whole domain and all observations at once
4D-Var is better than 3D-Var because:I Uses observations at the correct timeI Calculates analysis at the correct timeI Implicitly generates flow-dependent BI Can extract tendency information from observations
![Page 52: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/52.jpg)
Summary
Var is better than direct solution (a.k.a. Optimum Interpolation)because:
I Can handle lots of observationsI Can better cope with nonlinear observation operator HI Solves for the whole domain and all observations at once
4D-Var is better than 3D-Var because:I Uses observations at the correct timeI Calculates analysis at the correct timeI Implicitly generates flow-dependent BI Can extract tendency information from observations
![Page 53: 4D-Var for Dummies - Australia's official weather ... · 4D-Var for Dummies Jeff Kepert Bureau of Meteorology Research and Development CAWCR Student Workshop, December 1 2016](https://reader034.vdocument.in/reader034/viewer/2022050313/5f75f010f9172e41b54de448/html5/thumbnails/53.jpg)
References
I Special issue of QJRMS from WMO DA workshop inPrague, 2005.
I Special issue of JMetSocJapan from WMO DA workshopin Tokyo, 1997. http://www.journalarchive.jst.go.jp/english/jnltoc_
en.php?cdjournal=jmsj1965&cdvol=75&noissue=1B
I Kepert, J.D., 2007: Maths at work in meteorology. Gazetteof the Australian Mathematical Society, 34, 150 – 155.http://www.austms.org.au/Publ/Gazette/2007/
Jul07/[email protected]
I Kalnay, E., 2002, Atmospheric Modeling, Data Assimilationand Predictability.