april 29 – may 2, 20081space weather workshop 2008, boulder, co jpl/usc gaim: assimilating cosmic...

ANAS

Space Weather Workshop 2008, Boulder, COApril 29 – May 2, 2008 1

JPL/USC GAIM:Assimilating COSMIC occultations &

Sample Applications

Jet Propulsion Laboratory

California Institute of Technology

M/S 138-308

4800 Oak Grove Drive

Pasadena CA 91109

Email: [email protected]

Brian Wilson, Attila Komjathy, Vardan Akopian,Brian Wilson, Attila Komjathy, Vardan Akopian,Xioaqing Pi, Chi Ao, and Byron IijimaXioaqing Pi, Chi Ao, and Byron Iijima

ANAS


Outline

• Global Assimilative Ionosphere Model (JPL/USC GAIM)• First-principles physics model, with an adjoint• Assimilation by Kalman filter and 4DVAR

• Assimilating COSMIC TEC improves profile shapes• Three case studies: Arecibo, Jicamarca and Millstone ISR• Extensive validation using Jason VTEC, ISR profiles, ionosonde

• Storm Studies• TEC and density profile perturbations during August 7, 2006 storm

• Ray Tracing Capability Added to GAIM• Trace ray bending thru 3D electron density field• Application: Improving neutral atmosphere retrievals from COSMIC

• Short-Term Forecast (2-6 hours)• Adjust drivers using 4DVAR; feed improved drivers into Kalman filter• For longer-term, must forecast or measure upstream drivers/couplings

ANAS


Global Assimilative Ionospheric ModelGlobal Assimilative Ionospheric ModelData Assimilation ProcessData Assimilation Process

DrivingForces

DrivingForces

Mapping State To Measurements


PhysicsModel

PhysicsModel

Kalman FilterKalman Filter

State and covariance

Forecast

State andcovarianceAnalysis

AdjustmentOf Parameters

4DVAR4DVAR

Innovation Vector

• Kalman Filter• Recursive Filtering• Covariance estimation and state

correctionOptimal interpolationBand-Limited Kalman filter

• 4-Dimensional Variational Approach• Minimization of cost function by estimating

driving parameters • Non-linear least-square minimization• Adjoint method to efficiently compute the

gradient of cost function• Parameterization of model “drivers”

ANAS


Forward Model with an Adjoint• Driver Models

• NRL MSIS, HWM, Fejer-Scherliess ExB Drift (Fortran)

• Eulerian Solver• Variable-resolution, magnetic grid• Six ions: O+, H+, He+, N2

+, O2+, NO+

• Computational Efficiency• Adjoint computes all driver sensitivities in one pass

• Assim. cycle: Run forward model once & adjoint just once

• Multi-level physics cache:

– Find in memory, pre-computed file, or generate

• Optimized C++ (2nd generation code)• Object-oriented, Templated Matrix classes

• High-performance numerics

• Kudos to our C++ expert, Vardan Akopian

ANAS


Global Assimilative Ionospheric ModelGlobal Assimilative Ionospheric ModelData Assimilation ProcessData Assimilation Process

DrivingForces

DrivingForces



PhysicsModel

PhysicsModel

Kalman FilterKalman Filter

State and covariance

Forecast

State andcovarianceAnalysis

AdjustmentOf Parameters

4DVAR4DVAR

Innovation Vector

• Kalman Filter• Recursive Filtering• Covariance estimation and state

correctionOptimal interpolationBand-Limited Kalman filter

• 4-Dimensional Variational Approach• Minimization of cost function by estimating

driving parameters • Non-linear least-square minimization• Adjoint method to efficiently compute the

gradient of cost function• Parameterization of model “drivers”

ANAS


JPL/USC GAIM++JPL/USC GAIM++

Forward ModelWith Adjoint

4DVAR Kalman Filter

ANAS


Optimization Approach: 4DVAR

• Non-linear least squares minimization• Cost function to compute

model deviation from observations

• Adjoint method to compute gradient of cost function: computational efficiency

• Minimization: finding roots using Newton’s method by estimating driving parameters

• Parameterization of model drivers

Estimate ionospheric drivers & optimize state

2

0

2

0

1

2);();(

nn

tnHynJm

kkkk

αJ

0 αJ

N

kkkeqeq ttvtv

10, )()()(

7

17

1

0 )(),'(1

)()(i

ii

ii

fww

FF r'rr

ANAS


Better Drivers => Better Forecast

• Observation System Simulation Experiments (OSSE) to estimate “perturbed” drivers at low latitudes:

• Neutral winds

• EB vertical drift velocity

• Production terms

• Synthetic ground GPS TEC data

ANAS


JPL/USC Real-Time GAIM: RT TEC Map & Density SlicesThree Installations: JPL, AFRL, DoD

ANAS


Outline

• Global Assimilative Ionosphere Model (GAIM)• First-principles physics model, with an adjoint• Assimilation by Kalman filter and 4DVAR




• Short-Term Forecast (2-6 hours)• Adjust drivers using 4DVAR; feed improved drivers into Kalman filter• For longer-term, must forecast or measure upstream drivers/couplings

ANAS


GAIM Input Data TypesGAIM Input Data TypesGAIM Input Data TypesGAIM Input Data Types• Ground GPS Data (Absolute TEC) >150 5-min. to Hourly Global GPS Ground Stations Assimilate >300,000 TEC points per day (@ 5 min rate) per day

• Space GPS Data (Relative TEC)CHAMP (@ 440 km)SAC-C (@ 700 km)IOX (@ 800 km)GRACE (@ 350 km)Topex/Poseidon (@1330 km) (Upward looking only)Jason 1 (@1330 km) (Upward looking only)C/NOFS & COSMIC constellation

• UV Airglow: Limb & Nadir ScansLORAAS on ARGOS, GUVI on TIMEDSSUSI/SSULI on DMSP and NPOESS

• Other Data TypesTEC from TOPEX & JASON Ocean AltimetersIonosondeDMSP, CHAMP, C/NOFS in situ densityC/NOFS Electric fieldsGRACE Cross linksISR

ANAS


Six-satellite COSMIC constellationLaunched April 14, 2006

Low-Earth OrbiterGPS

ElectronDensityProfile

COSMIC coverage: 2500 profiles/day

COSMICCOSMICIonospheric Weather ConstellationIonospheric Weather Constellation

ANAS


Occultation TEC Links

ANAS


Kalman Assimilation Runs: Three Case Studies(Attila Komjathy)

• Three runs:• GAIM Climate (no data)• Ground GPS TEC (200 sites)• Ground + COSMIC links (upward & occultation)

• Medium and Low Resolution runs: 2.5 Vs. 5.0 Lat. In Deg. 10.0 Vs. 15.0 Lon. in Deg. 40 Vs. 80 Alt. in km 100,000 Vs. 18,000 voxels

• Sparse Kalman filter:• Update & propagate covariance• Truncate off-diagonal covariance that is beyond physical correlation lengths

Magnetic equatorEcc

en

tric

tilt

ed

dip

ole

Intersections of :- magnetic field lines, - magnetic geopotential lines- and magnetic longitudes

ANAS


Arecibo

FM2

FM5GPS15

20:0520:09

20:21

20:25

CRO1

JAMA

SCUB CRO1

UT 20:00

Case 1: Arecibo ISR Study for June 26, 2006

UT 20:00

UT 20:12 UT 20:24

ANAS


GAIM versus Abel ProfilesGAIM versus Abel Profiles

ANAS


UCAR and JPL Raw GPS Data Processing UCAR and JPL Raw GPS Data Processing Results for Sept 21, 2006Results for Sept 21, 2006

Comparison of UCAR and JPLcalibrated TEC near Jicamarca

Ground and COSMIC data availability near Jicamarca

Ground and COSMIC ground tracks near Jicamarca

ANAS


An Example of COSMIC Impact on Profile ShapeAn Example of COSMIC Impact on Profile Shape

2. UT 15:48

3. UT 16:36COSMIC UT 15:30

UT hours

Ele

vatio

n an

gle

1. UT 15:24

1. 2. 3.

1. UT 15:36

ANAS


Illustration for TEC data, GAIM Prefit and Postfit Illustration for TEC data, GAIM Prefit and Postfit ResidualsResiduals

ANAS


TEC Comparison with Jicamarca ISRTEC Comparison with Jicamarca ISR

GAIM Medium Resolution Assimilation Results for Sept 21, 2006

0

5

10

15

20

25

30

35

0 2 4 6 8 10 12 14 16 18 20 22 24 26UT in Hours

TEC in T

ECU

JRO: Jicamarca ISR

medium resolution climate

medium resolution ground-only assimilation

medium resolution ground+COSMIC assmilation

COSMIC Data

ANAS


NmF2 Comparison with Jicamarca ISRNmF2 Comparison with Jicamarca ISR

Medium resolution GAIM NmF2 with COSMIC data matches well during the dense data period 11-17 UT

GAIM Medium Resolution Assimilation Results for Sept 21, 2006

0

1E+11

2E+11

3E+11

4E+11

5E+11

6E+11

7E+11

8E+11

9E+11

1E+12

0 2 4 6 8 10 12 14 16 18 20 22 24

UT in Hours

Nm

F2

JRO: Jicamarca ISRmedium resolution climatemedium resolution ground-only assimilationmedium resolution ground+COSMIC assimilation

COSMIC Data

ANAS


HmF2 Comparison with Jicamarca ISRHmF2 Comparison with Jicamarca ISR

GAIM Medium Resolution Assimilation Results for HmF2 on Sept 21, 2006

0

50

100

150

200

250

300

350

400

450

0 2 4 6 8 10 12 14 16 18 20 22 24

UT in Hours

Hm

F2

JRO: Jicamarca ISRmedium resolution climatemedium resolution ground-only assimilationmedium resolution ground+COSMIC assimilation

COSMIC Data Available

Medium resolution GAIM HmF2 with COSMIC matches best with truth

ANAS


Summary of ResultsSummary of ResultsTEC Comparison of Low and Medium Resolution Runs for Sept 21, 2006

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

Climate Low GO Low G+C Medium GO Medium G+C

TE

C i

n T

EC

U

Abs Mean

Sigma

NmF2 Comparison of Low and Medium Resolution Runs for Sept 21, 2006

0.00E+00

1.00E+10

2.00E+10

3.00E+10

4.00E+10

5.00E+10

6.00E+10

7.00E+10

8.00E+10

9.00E+10

1.00E+11


Ele

ctr

on

De

nsit

y

Abs Mean

Sigma

TEC

NmF2

ANAS


Summary of ResultsSummary of Results

HmF2 Comparison of Low and Medium Resolution Runs for Sept 21, 2006

0.00

5.00

10.00

15.00

20.00

25.00

30.00

35.00


He

igh

t in

Km

Abs Mean

Sigma

GO = Ground-GPS only GD = Ground + down-looking COSMIC

ANAS


GAIM w/ COSMIC versus JASON VTECGAIM w/ COSMIC versus JASON VTEC

ANAS


Detail for JASON Track off AustraliaDetail for JASON Track off Australiaon 06/26/2006on 06/26/2006

Nearby COSMIC data crucial for achieving accuracy.

ANAS


Mean Sigma RMS

GIM -1.61 2.88 3.31

06/26/06 Ground -0.24 3.26 3.27

Ground+COSMIC -0.29 2.26 2.28

Mean Sigma RMS

GIM -2.01 3.48 4.02

09/21/06 Ground -1.08 3.45 3.62

Ground+COSMIC -0.31 2.66 2.67

Mean Sigma RMS

GIM -1.95 2.58 3.24

12/21/06 Ground -1.3 4.32 4.51

Ground+COSMIC 0.49 2.45 2.54

GAIM w/ COSMIC versus JASON VTECGAIM w/ COSMIC versus JASON VTEC

Statistics over Three ISR Periods

ANAS


GAIM Driven By Ground GPS OnlyGAIM Driven By Ground GPS Onlyversus JASON VTECversus JASON VTEC

June – Nov. 2004: 137 days

ANAS


Outline



• Storm Studies (Xiaoqing Pi)• TEC and density profile perturbations during August 7, 2006 storm


• Short-Term Forecast (2-6 hours)• Adjust drivers using 4DVAR; feed improved drivers into Kalman filter• For longer-term, must forecast or measure upstream drivers

ANAS


Drivers of the Ionosphere: Drivers of the Ionosphere: Coupling with Magnetosphere and ThermosphereCoupling with Magnetosphere and Thermosphere

• Solar EUV radiation Cause of the ionization Solar flares

• Auroral particle precipitation Cause of the ionization at high latitudes Significant variations during storms and substorms

• Thermospheric composition & temperature Gas to be ionized Loss of ionization due to chemical reactions Global thermospheric circulation changes during storms

• Dynamics Electric fields: originated from the magnetospheric and wind dynamo processes Thermospheric winds Controlled by the geomagnetic field Magnetospheric convection, penetration, and disturbance wind dynamo

• Ionospheric data assimilation combines first-principles physics-based modeling and global-scale observations.

• For long-term forecast of ionospheric weather, must deal with the drivers and magnetospheric & thermospheric couplings.

ANAS


Magnetic Storm: August 7, 2006Magnetic Storm: August 7, 2006

AUG 4 AUG 5 AUG6 AUG 7 AUG 8 AUG 9 AUG 10 AUG11

100 nT

-80 nT

ANAS


Global Ionospheric Maps 04:00 & 07:15 UTGlobal Ionospheric Maps 04:00 & 07:15 UT

ANAS


Global Ionospheric Maps: 10:30 & 23:00 UTGlobal Ionospheric Maps: 10:30 & 23:00 UT

ANAS


Validation of GAIM Validation of GAIM Using JASON Altimeter TEC MeasurementsUsing JASON Altimeter TEC Measurements

Storm Day

Tracks of COSMIC occul. tangent points

Jason orbit tracks

GPS ground stations

Statistics for the entire day

TEC along Jason orbit tracks

ANAS


COSMIC RO and JASON ComparisonCOSMIC RO and JASON Comparison

~ 1 hour

Comparisons between GAIM, IRI, and Jason

ANAS


Disturbed Ionosphere at 9 UTDisturbed Ionosphere at 9 UT

TEC perturbations in the west Pacific longitudes

COSMIC coverage in the regions of perturbations

ANAS


Ne Disturbance in the Pacific LongitudesNe Disturbance in the Pacific Longitudes

ANAS


TEC perturbations in the west Pacific longitudes

COSMIC coverage in the regions of perturbations

Disturbed Ionosphere at 10 UTDisturbed Ionosphere at 10 UT

ANAS


Ne Disturbance in the Pacific LongitudesNe Disturbance in the Pacific Longitudes

ANAS


Outline




• Ray Tracing Capability Added to GAIM (Chi Ao)• Trace ray bending thru 3D electron density field• Application: Improving neutral atmosphere retrievals from COSMIC

• Short-Term Forecast (2-6 hours)• Adjust drivers using 4DVAR; feed improved drivers into Kalman filter• For longer-term, must forecast or measure upstream drivers

ANAS


GAIM++ 3D Raytracing CapabilityGAIM++ 3D Raytracing Capability(Chi Ao)(Chi Ao)

• Adaptive Runge-Kutta solver

• Tricubic interpolation of electron density in (r,lat,lon) space

[Lekien et al., 2004]

• IGRF magnetic field

• Refractive index from Appleton-Hartree formula

• “Homing” using the Subplex algorithm (generalization of the

Nelder-Mead simplex method) [Rowan, 1990]

3D grid of Ne

Rx/Tx Loc. & Freq

Magnetic field

GAIM3D

RaytracerTEC,

Group path

Group pathIonosphere:

dsvg

dsn

ANAS


Elevation Angles and HomingElevation Angles and Homing

Geometric PathGeometric Path

True PathTrue Path

Non-Homing PathNon-Homing Path

STECNonhome > STECTrue > STECGeo

ANAS


Bending for Trans-Iono RaypathsBending for Trans-Iono Raypaths

200 Mhz30 Mhz50 Mhz

L1

See expected behavior:Much more bending at low frequencies

ANAS


Effect on Atmospheric Temperature RetrievalEffect on Atmospheric Temperature Retrieval

ANAS


Error Study: Large-Scale IonosphereError Study: Large-Scale Ionosphere

L1

L2

Transmitter Receiver11-year solar cycle

“Ionosphere-free” linear combination

Note: no assumptions made about ionospheric structure

ANAS


Magnitude of Different EffectsMagnitude of Different Effects

From Bassiri and Hajj, 1993

ANAS


Bending Along L1 & L2 Signal PathsBending Along L1 & L2 Signal Paths

Need accuracy to0.05 % in refractivityto get 0.1 degrees K.

ANAS


Effect on Atmospheric Temperature RetrievalEffect on Atmospheric Temperature Retrieval

ANAS


Short-Term Forecast Issues

• Start with accurate iono. nowcast from data assimilation• Exploit all global datasets, quantitative accuracy demonstrated• Lots of science can be done with accurate 3D density fields

• Estimate improved drivers using 4DVAR• Drivers: production, ExB drift, neutral winds, etc.• Feed improved drivers into Kalman filter• Forecast for 2-6 hours using physics propagator

• Continuous Forecast Validation Needed• Unknown drivers or couplings drive iono. physics away from reality• Simplest Benchmark: Must beat persistence• Community should start comparing Skill Scores

• Iono. Assim. Complementary to T-I-M Coupled Models• Couplings still incomplete, not quantitatively accurate, enough data?• Use 4DVAR to invert for coupled boundary conditions

ANAS


Summary

• Global Assimilative Ionosphere Model (JPL/USC GAIM)• First-principles physics model, with an adjoint• Assimilation by Kalman filter and 4DVAR

• Assimilating COSMIC TEC improves profile shapes• Extensive validation using Jason VTEC, ISR profiles, ionosonde

• Storm Studies using new level of global 3D accuracy• Quantitatively-accurate science can be done

• Ray Tracing Capability Added to GAIM• Looking for new iono. or atmo. applications for ray tracer

• Short-Term Forecast (2-6 hours)• Adjust drivers using 4DVAR; feed improved drivers into Kalman filter• High accuracy iono. forecast beyond 6 hours is HARD.• Many approaches to 12-72 hour forecast should be investigated.

april 29 – may 2, 20081space weather workshop 2008, boulder, co jpl/usc gaim: assimilating cosmic...

Documents

covariance forecast

forward model

measurements mapping

forces mapping state

principles physics model

space weather workshop

adjoint assimilation

driving parameters nonlinear