Upload File

dart tutorial sec’on 10: regression and non-linear effects · 2018-11-30 · dart-compliant...

  • Home
  • Documents
  • DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types
18
The Na’onal Center for Atmospheric Research is sponsored by the Na’onal Science Founda’on. Any opinions, findings and conclusions or recommenda’ons expressed in this publica’on are those of the author(s) and do not necessarily reflect the views of the Na’onal Science Founda’on. ©UCAR DART Tutorial Sec’on 10: Regression and Non-linear Effects

Upload: others

Post on 12-Jul-2020

2 views

Category:

Documents


0 download

Report

TRANSCRIPT

Page 1: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

TheNa'onalCenterforAtmosphericResearchissponsoredbytheNa'onalScienceFounda'on.Anyopinions,findingsandconclusionsorrecommenda'onsexpressedinthispublica'onarethoseoftheauthor(s)anddonotnecessarilyreflecttheviewsoftheNa'onalScienceFounda'on.

©UCAR

DARTTutorialSec'on10:RegressionandNon-linearEffects

Page 2: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Upda'ngaddi'onalpriorstatevariables

Twoprimaryerrorsources:1.Linearapproxima'onisinvalid.

Substan'alnonlinearityin‘true’rela'onoverrangeofprior.2.Samplingerrorduetonoise(we’vealreadylookedatthis).

Eveniflinearrela'on,sampleregressioncoefficientimprecise.Mayneedtoaddressbothissuesforgoodperformance.

DARTTutorialSec'on10:Slide2

Page 3: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.

DARTTutorialSec'on10:Slide3

Page 4: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Updateobservedsampleandcomputeincrements.

DARTTutorialSec'on10:Slide4

Page 5: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Regressionerrorvarieswithvalueofobservedvariable.

DARTTutorialSec'on10:Slide5

Page 6: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Regressionerrorvarieswithvalueofobservedvariable.Smallerincrementshavesmallerexpectederrors.

DARTTutorialSec'on10:Slide6

Page 7: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Pairingbetweenpriorandposteriorsampleofobservedvariablecanbeviewedasarbitrary.Posteriorissamesamplehoweverit’spaired.

DARTTutorialSec'on10:Slide7

Page 8: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Canminimizeincrementsbychangingpairing.Sor'ngpriorandposteriorandpairingsamplesminimizesonenormofincrementsize(coulddoothermethods)

DARTTutorialSec'on10:Slide8

Page 9: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Canminimizeincrementsbychangingpairing.Sor'ngpriorandposteriorandpairingsamplesminimizesonenormofincrementsize(coulddoothermethods)

DARTTutorialSec'on10:Slide9

Page 10: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Resul'ngregressionerrorisminimized.Impactofsor'ngcanbeverylargewhenposteriorselectedby‘random’algorithms.

DARTTutorialSec'on10:Slide10

Page 11: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

SupposepriorsamplehasNOnoise.But,rela'onbetweenun/observedvariablesisnon-linear.Resul'ngregressionerrorisminimized.Impactofsor'ngcanbeverylargewhenposteriorselectedby‘random’algorithms.

DARTTutorialSec'on10:Slide11

Page 12: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

Canseethisimpactnicelyin9varmodel.Tryfilter_kind=2inassim_tools_nmlwith:

sort_obs_inc=.true.(incrementsminimized)andsort_obs_inc=.false.

Examinetheamountofnoiseindifferent'meseries.ImpactonRMSmaynotbewhatwasexpected.Theremaybesurprisesinotherlow-ordermodelswhentryingthis.

DARTTutorialSec'on10:Slide12

models/9var/work/

&assim_tools_nml filter_kind = 2 sort_obs_inc = .false. …

Page 13: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Sor'ngincrements

AlsocanexamineinLorenz96.Tryfilter_kind=2inassim_tools_nmlwith:

sort_obs_inc=.true.(incrementsminimized)andsort_obs_inc=.false.

Tryacasewithnolocaliza'on(largecutoff)Varyinfla'onwithandwithoutthesor'ng.

DARTTutorialSec'on10:Slide13

models/lorenz_96/work/

&assim_tools_nml filter_kind = 2 sort_obs_inc = .true. cutoff = 1000000.0 …

Page 14: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Localregression

Priorsampleisnoisy.Un/observedrela'onisnon-linear.DoingglobalregressionwouldbeBADhere.Candoregressiononlyforpointsthatlieinrangeofupdateincrement.Couldalsopicklocalsetsinotherways.

DARTTutorialSec'on10:Slide14

Page 15: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Localregression

Priorsampleisnoisy.Un/observedrela'onisnon-linear.Forlargerensembles,localregressionscanworkwell.Errorislargestwheresignalisweakest(nearboeomofparabolahere).

DARTTutorialSec'on10:Slide15

Page 16: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Localregression

Priorsampleisnoisy.Un/observedrela'onisnon-linear.Assamplesizedecreases,errorgrows.(Exceptwhereitwasroeentostart).Applica'onswherelocalregressionisusefulareunknowntome.

DARTTutorialSec'on10:Slide16

Page 17: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

Nonlinearrela'onsbetweenvariables:Localregression

DARTdoesnotcurrentlysupportlocalregressionwithoutcodemodifica'on.

DARTTutorialSec'on10:Slide17

Page 18: DART Tutorial Sec’on 10: Regression and Non-linear Effects · 2018-11-30 · DART-Compliant Models and Making Models Compliant 20. Model Parameter Es>ma>on 21. Observaon Types

1.   FilteringForaOneVariableSystem2.   TheDARTDirectoryTree3.   DARTRun>meControlandDocumenta>on4.   Howshouldobserva>onsofastatevariableimpactanunobservedstatevariable?

Mul>variateassimila>on.5.   ComprehensiveFilteringTheory:Non-Iden>tyObserva>onsandtheJointPhaseSpace6.   OtherUpdatesforAnObservedVariable7.   SomeAddi>onalLow-OrderModels8.   DealingwithSamplingError9.   MoreonDealingwithError;Infla>on10.   RegressionandNonlinearEffects11.   Crea>ngDARTExecutables12.   Adap>veInfla>on13.   HierarchicalGroupFiltersandLocaliza>on14.   QualityControl15.   DARTExperiments:ControlandDesign16.   Diagnos>cOutput17.   Crea>ngObserva>onSequences18.   LostinPhaseSpace:TheChallengeofNotKnowingtheTruth19.   DART-CompliantModelsandMakingModelsCompliant20.   ModelParameterEs>ma>on21.   Observa>onTypesandObservingSystemDesign22.   ParallelAlgorithmImplementa>on23.  Loca'onmoduledesign(notavailable)24.  Fixedlagsmoother(notavailable)25.   Asimple1Dadvec>onmodel:TracerDataAssimila>on

DARTTutorialIndextoSec'ons

DARTTutorialSec'on10:Slide18


DaRT Overview

DART 2013 - oficina.exagono.netoficina.exagono.net/DC2/2012/Dodge/pdf/fichas-tecnicas/dart-2013.pdf · ESPECIFICACIONES TÉCNICAS Dodge Dart SXT Dodge Dart Limited Motor Potencia

DART Tutorial Sec’on 1: Filtering For a One Variable System · DART Tutorial Sec’on 1: Slide 2 Introduc’on This series of tutorial presentaons is designed to introduce both

Dart Engine

Intrinsic DART

Dart reliability

DART Presentation

Google Dart

Dart cover 2013 - performancewholesale.com › catalogs › dart › Dart-2013-Catalog.pdfCasting & Metallurgy Dart uses only virgin aluminum C355-T61 alloy for all of our cast aluminum

DArt project

Dart cover 2013 - Performance Wholesaleperformancewholesale.com.au/catalogs/dart/Dart-2013-Catalog.pdfmore powerful and more reliable parts for Dart ... respective trademark holders

DART Tutorial Sec’on 12: Adap’ve InflaonAdap’ve Observaon Space Inflaon in DART Try this in Lorenz 96 (verify other aspects of input.nml). Use 40 member ensemble. (set ens_size

BLUE DART EXPRESS LIMITED BLUE DART AVIATION LIMITED

DART 2040TransitPlan

DART Tutorial Sec’on 19: Making DART-Compliant Models · DART uses iden’cal assimilaon code for menagerie of models. Same namelists you’ve been using in low-order models s’ll

Introduction to the Dart - Dart Flight School Liberec Hackathon

DART Tutorial Sec’on 15: DART Experiments: Control and Design · 1. Can just ‘restart’ from single ’me-level aer each assimilaon. This can lead to numerical instability if

Dr. Dart Presentation Materials. Dart Presentation Materials

DART Tutorial Sec’on 13: Hierarchical Group Filters and Localizaon · 2020-07-06 · DART Tutorial Sec’on 13: Slide 9 models/lorenz_96/work/ 4C. Use hierarchical Monte Carlo:

DART Tutorial Sec’on 20: Model Parameter Es’maon

DART Tutorial Sec’on 4: How should observaons impact an ...Single observed variable, single unobserved variable. So far, have known observaon likelihood for single variable. Now,

DART Tutorial Sec’on 2: The DART Directory Tree · DART Tutorial Sec’on 2: The DART Directory Tree The DART Code Tree Much of DART is implemented as Fortran-90 modules and programs

- UNIGRAZ · in Accounting, Reporting, DART. and Taxation Doctoral Program in Accounting, Reporting, DART. and Taxation DART is funded by . About DART DART is a doctoral program in

DART Tutorial Sec’on 8: Dealing with Sampling ErrorDART Tutorial Sec’on 8: Slide 17 Experimen’ng with Lorenz 96 The lorenz_96 domain is mapped to a [0, 1] periodic range. Try

Dart Glazier

Dart Cagtfdrt

DART 16 Rigging Manual - Swords Sailing & Boating Clubswordssailing.ie/wp-content/uploads/2017/12/Dart-16-Rigging-Guide.… · DART 16 DART 16 RACE DART 16 Rigging Manual 1 Glossary

Evaluation of WRF-DART (ARW v.3.9.1.1 and DART manhattan

DART Tutorial Sec’on 7: Some Addi’onal Low-Order Models · DART Tutorial Sec’on 7: Slide 9 preassim.nc preassim.nc preassim.nc true_state.nc models/lorenz_96/work/ Lorenz 96

DART Risk Management Case Study - NASA Case/DART Text Case.pdf · DART Risk Management Case Study Human Exploration and Operations Mission Directorate Page 1 DART: Risk Management

DART Tutorial Sec’on 6: Other Updates for an Observed Variable · Obs. Likelihood Random Draws from Obs. Proceed to associate a random draw from obs. with each prior sample. Earliest

Dart Workshop

DART 16 DART 16 RACE - Sailboats.co.uk€¦ · DART 16 DART 16 RACE DART 16 Rigging Manual 1 Glossary 2 Useful Boat Terminology 3 Cordage Lengths 4 Sail Number Positioning 5 Building

DART Tutorial Sec’on 9: More on Dealing with Error: Inflaon · DART Tutorial Sec’on 9: Slide 20 1. Filtering For a One Variable System 2. The DART Directory Tree 3. DART Run>me

Annual MeetingAnnual Meeting - Dart Harbour & … - Safety Management System CURRENT - 3 Audits complete with very satisfactory results - Declared compliant with Port Marine Safety