Upload File

Download - DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Transcript
Page 1: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

TheNa'onalCenterforAtmosphericResearchissponsoredbytheNa'onalScienceFounda'on.Anyopinions,findingsandconclusionsorrecommenda'onsexpressedinthispublica'onarethoseoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheNa'onalScienceFounda'on.

©UCAR

DARTTutorialSec'on10:RegressionandNon-linearEffects

Page 2: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Upda'ngaddi'onalpriorstatevariables

Twoprimaryerrorsources:1.Linearapproxima'onisinvalid.

Substan'alnonlinearityin‘true’rela'onoverrangeofprior.2.Samplingerrorduetonoise(we’vealreadylookedatthis).

Eveniflinearrela'on,sampleregressioncoefficientimprecise.Mayneedtoaddressbothissuesforgoodperformance.

DARTTutorialSec'on10:Slide2

Page 3: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.

DARTTutorialSec'on10:Slide3

Page 4: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Updateobservedsampleandcomputeincrements.

DARTTutorialSec'on10:Slide4

Page 5: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Regressionerrorvarieswithvalueofobservedvariable.

DARTTutorialSec'on10:Slide5

Page 6: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Regressionerrorvarieswithvalueofobservedvariable.Smallerincrementshavesmallerexpectederrors.

DARTTutorialSec'on10:Slide6

Page 7: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Pairingbetweenpriorandposteriorsampleofobservedvariablecanbeviewedasarbitrary.Posteriorissamesamplehoweverit’spaired.

DARTTutorialSec'on10:Slide7

Page 8: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Canminimizeincrementsbychangingpairing.Sor'ngpriorandposteriorandpairingsamplesminimizesonenormofincrementsize(coulddoothermethods)

DARTTutorialSec'on10:Slide8

Page 9: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Canminimizeincrementsbychangingpairing.Sor'ngpriorandposteriorandpairingsamplesminimizesonenormofincrementsize(coulddoothermethods)

DARTTutorialSec'on10:Slide9

Page 10: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Resul'ngregressionerrorisminimized.Impactofsor'ngcanbeverylargewhenposteriorselectedby‘random’algorithms.

DARTTutorialSec'on10:Slide10

Page 11: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Resul'ngregressionerrorisminimized.Impactofsor'ngcanbeverylargewhenposteriorselectedby‘random’algorithms.

DARTTutorialSec'on10:Slide11

Page 12: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

Canseethisimpactnicelyin9varmodel.Tryfilter_kind=2inassim_tools_nmlwith:

sort_obs_inc=.true.(incrementsminimized)andsort_obs_inc=.false.

Examinetheamountofnoiseindifferent'meseries.ImpactonRMSmaynotbewhatwasexpected.Theremaybesurprisesinotherlow-ordermodelswhentryingthis.

DARTTutorialSec'on10:Slide12

models/9var/work/

&assim_tools_nml filter_kind = 2 sort_obs_inc = .false. …

Page 13: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

AlsocanexamineinLorenz96.Tryfilter_kind=2inassim_tools_nmlwith:

sort_obs_inc=.true.(incrementsminimized)andsort_obs_inc=.false.

Tryacasewithnolocaliza'on(largecutoff)Varyinfla'onwithandwithoutthesor'ng.

DARTTutorialSec'on10:Slide13

models/lorenz_96/work/

&assim_tools_nml filter_kind = 2 sort_obs_inc = .true. cutoff = 1000000.0 …

Page 14: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Localregression

Priorsampleisnoisy.Un/observedrela'onisnon-linear.DoingglobalregressionwouldbeBADhere.Candoregressiononlyforpointsthatlieinrangeofupdateincrement.Couldalsopicklocalsetsinotherways.

DARTTutorialSec'on10:Slide14

Page 15: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Localregression

Priorsampleisnoisy.Un/observedrela'onisnon-linear.Forlargerensembles,localregressionscanworkwell.Errorislargestwheresignalisweakest(nearboeomofparabolahere).

DARTTutorialSec'on10:Slide15

Page 16: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Localregression

Priorsampleisnoisy.Un/observedrela'onisnon-linear.Assamplesizedecreases,errorgrows.(Exceptwhereitwasroeentostart).Applica'onswherelocalregressionisusefulareunknowntome.

DARTTutorialSec'on10:Slide16

Page 17: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Localregression

DARTdoesnotcurrentlysupportlocalregressionwithoutcodemodifica'on.

DARTTutorialSec'on10:Slide17

Page 18: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

1.   FilteringForaOneVariableSystem2.   TheDARTDirectoryTree3.   DARTRun>meControlandDocumenta>on4.   Howshouldobserva>onsofastatevariableimpactanunobservedstatevariable?

Mul>variateassimila>on.5.   ComprehensiveFilteringTheory:Non-Iden>tyObserva>onsandtheJointPhaseSpace6.   OtherUpdatesforAnObservedVariable7.   SomeAddi>onalLow-OrderModels8.   DealingwithSamplingError9.   MoreonDealingwithError;Infla>on10.   RegressionandNonlinearEffects11.   Crea>ngDARTExecutables12.   Adap>veInfla>on13.   HierarchicalGroupFiltersandLocaliza>on14.   QualityControl15.   DARTExperiments:ControlandDesign16.   Diagnos>cOutput17.   Crea>ngObserva>onSequences18.   LostinPhaseSpace:TheChallengeofNotKnowingtheTruth19.   DART-CompliantModelsandMakingModelsCompliant20.   ModelParameterEs>ma>on21.   Observa>onTypesandObservingSystemDesign22.   ParallelAlgorithmImplementa>on23.  Loca'onmoduledesign(notavailable)24.  Fixedlagsmoother(notavailable)25.   Asimple1Dadvec>onmodel:TracerDataAssimila>on

DARTTutorialIndextoSec'ons

DARTTutorialSec'on10:Slide18


Top Related

Dart Workshop
Dart Workshop
Testing Equipment for Construction Materials H-4114C.3F … · 18/02/2011  · 4" Dart— H-4114.4 6" Dart— H-4114.6 8" Dart— H-4114.8 10" Dart— H-4114.10 12" Dart— H-4114.12
Testing Equipment for Construction Materials H-4114C.3F … · 18/02/2011  · 4" Dart— H-4114.4 6" Dart— H-4114.6 8" Dart— H-4114.8 10" Dart— H-4114.10 12" Dart— H-4114.12
Intrinsic DART
Intrinsic DART
The DART SystemThe DART System - dallas.texite.orgdallas.texite.org/presentation-salin-dart.pdfThe DART SystemThe DART System Past, Present & Future Institute of Transportation Engineers
The DART SystemThe DART System - dallas.texite.orgdallas.texite.org/presentation-salin-dart.pdfThe DART SystemThe DART System Past, Present & Future Institute of Transportation Engineers
DART Tutorial Sec’on 7: Some Addi’onal Low-Order Models · DART Tutorial Sec’on 7: Slide 9 preassim.nc preassim.nc preassim.nc true_state.nc models/lorenz_96/work/ Lorenz 96
DART Tutorial Sec’on 7: Some Addi’onal Low-Order Models · DART Tutorial Sec’on 7: Slide 9 preassim.nc preassim.nc preassim.nc true_state.nc models/lorenz_96/work/ Lorenz 96
DART Tutorial Sec’on 15: DART Experiments: Control and Design · 1. Can just ‘restart’ from single ’me-level aer each assimilaon. This can lead to numerical instability if
DART Tutorial Sec’on 15: DART Experiments: Control and Design · 1. Can just ‘restart’ from single ’me-level aer each assimilaon. This can lead to numerical instability if
Google Dart
Google Dart
BLUE DART EXPRESS LIMITED BLUE DART AVIATION LIMITED
BLUE DART EXPRESS LIMITED BLUE DART AVIATION LIMITED