near-real-time hirdls processing for start08 vince dean hirdls science team meeting january 30, 2008...

Near-Real-Time HIRDLS Processing

for START08

Vince Dean

HIRDLS Science Team Meeting

January 30, 2008

Boulder, CO

START08 Timeline

Near Real Time Retrievals to support flight planning for START08.

• Forecast exercises• Feb 29, 2008• Mar 21, 2008

• Flight Windows – spring 2008• April 21 .. May 16 (maybe earlier)• June 16 .. 28

Observation Times

• START08 based in Jeffco. Covers Western US.

• Approximate times of passes of interest• Morning

• 06:30 .. 09:00 GMT• 23:30 .. 02:00 MST

• Evening• 20:45 .. 23:00 GMT• 13:45 .. 16:00 MST

• Flight Planning Meetings• 14:00 MST

Descending Passes22:00 ... 02:00 MST

Ascending Passes:13:00 ... 16:30 MST

START08 – timing requirements

23:30 MST 02:00 MST

Descending HIRDLS passes

over western US

START08 Flight planning meeting

14:00 MST

12 hours for data processing,

delivery to START08

Field Catalog, and analysis

Plots

Level 0 Science

Attitude

Ephemeris

L1PP

L1X

GEOS-5Product:

Temperature

L1C

L2PP

L2CLD

L2 HIRDLS2

Processing Flow

HIRDLS2

Plotting and Delivery

Plotting Code

Plot Files (.png)

EOL FieldCatalog

Rename and

deliver

http://catalog.eol.ucar.edu/trex/index.html

Rate-Buffered Science

Data

Standard Level 0

Science Data

L1PP

FileConversion

at GoddardDISC

HIRDLS Science Data

Attitude

Ephemeris

L1X

Rate-Buffered GBAD

“Direct Readout” Software

at GoddardDISC

Aura Attitude/Ephemeris

Geolocation Test

• L1X 7.4.1• Standard vs. sample attitude/ephemeris• One 2-hour contact session• Differences:

Min Max Mean Std Dev

Altitude(m)

7 20 12 3

Latitude(deg)

-0.002 0,001 -2.3e-5 1.2E-4

Longitude(deg)

-0.002 0.005 -5.4E-5 3.1E-4

L2CLD

L2CLD

Yesterday:24-hour productionjob.

Today:20 minute near-real-time job.

HIRRAD

HIRRADNC

L2CLD

HIRRAD

HIRRADNC

Daily Average Radiance

L2 HIRDLS2

GEOS-5 Forecast

Forecast Runs:Once or twice daily, up to five days in advance.

GEOS-5 Temperature forLine of sight gradient

GEOS-5 analysis product

D5HIRDL1

Strategy

• Outsource software development to DISC where possible:• Rate-buffered to PDS conversion• Attitude/ephemeris file conversion

• Relatively low risk:• Goddard’s plans seem attainable.• They seem confident.

• But “best effort”

Tasks

• Enhancements to science code• L2CLD• L2

• Software development at Goddard• Attitude/ephemeris• Rate-buffered to PDS conversion

• Data management• SIPS software enhancements

• Validation of approximations• Test runs needed

Extra Slides

Sources of HIRDLS science data

• Standard PDS files from Goddard– Typically arrives 3 hours after last date (quite variable!)

• Expedited PDS files from Goddard– Somewhat faster

• Rate buffered data from EDOS– Typically arrives ½ hour after last data (quite variable!)

• New shorter PDS files from Goddard

Level 0 Science Data Sources

Aura

Ground Station EDOS

Goddard DISC

HIRDLS SIPS

Rate-buffereddata.

PDS data

Expedited PDS data

PDS data

RBD?

PDS data

• HIRDLS Science Packets• In order• No duplicates

• Normally delivered in regular 2-hour files.• Additional latency to fill out complete 2-hour chunk—

maybe wait for next pass.

Rate-Buffered Data

• HIRDLS science packets with additional headers• Typically covers one contact session• About one orbit (100 min)• Size and timing are irregular• Packets may be out of order or duplicated• We understand file format, and we receive the data, but no

production experience.

RBD processing at HIRDLS

L1PP (modified)

Rate-buffered data

Standard HIRDLS Science

file (HIR0SCI_FIXED

)

We already receive RBD data routinely.L1PP already sorts packets, part of fixing clock problem.Includes library code to read RBD files.

Relatively small change to extend for RBD processing.Unknown operational issues—we have not used RBD routinely.

Goddard DISC to process rate-buffered data?

• Goddard DISC is planning to process rate-buffered data for a quick-delivery PDS product

• For MLS• For HIRDLS?

• 15 minutes granules instead of 2 hours• Same collecting and sorting as done by EDOS.

EDOS

RBD Goddard DISC

15-minute PDS science

granules

Attitude/Ephemeris

• We require files in Toolkit format• Normal delivery takes about 1 day• Other sources of forecasts/approximations:

• Use files from 16 days ago, advancing timestamps• Get a forecast from Flight Dynamics and convert file format• Use GBAD files and convert file format• Use orbsim program to generate forecasts

16-day old attitude & ephemeris

Rewrite times in

files

“Forecast” attitude and ephemeris

Small custom program. Not complex.

File formats are well documented.

Attitude/Ephemeris16-day cycle

Attitude/EphemerisFlight Dynamics Forecast

Forecasts of attitude/ephemeris

from Flight Dynamics

?

Forecasts of attitude/ephemeris in Toolkit Format

Custom code to convert file formats.

Unknown complexity.

Attitude/EphemerisGBAD to Toolkit

Aura GBAD file(s)

DirectReadout code

Attitude & Ephemeris in Toolkit format

MLS has provided leads to code whichperforms this conversion from NASA “DirectReadout” project.

This code has been used for AIRS and Goddard DISC is installingit.

What is GBAD data?(Ground-Based Attitude Determination)

Forecasts of attitude/ephemeris(Flight Dynamics?)

Aura

Converted to GBAD format and delivered to

HIRDLS SIPS, via EDOS and DISC.

GBAD files contain forecasts of attitude and ephemeris, delivered to the spacecraft and then back to earth. There is said to be software which will convert those files to toolkit format.

Attitude/EphemerisOrbsim

Attitude/Ephemeris in Toolkit Format

Orbsim

Testing tool provided with Toolkit. Intended only for software testing, not for production forecasts.

Orbital Parameters

GEOS-5D5HIRDL1 subset

GMAO Assimilation

Goddard DISC

HIRDLS SIPS

Files created 4 times daily.

•Delivered on regular 6-hour intervals?•Can they create forecasts as a special product?

Plots(Bill Hall’s IDL)

IDL Plot code Rename

and deliver to Field Catalog

EOLField

Catalog(Web site)

Plots

HIRDLS2

End to end test(Prototype)

• Early prototype of processing steps and data products.• Explore:

• Client requirements• System and configuration issues

• Test bed for refinement of processing steps.