Climate Data Records –A Maturity Model and
Research-Operation Transitions
John BatesNOAA’s National Climatic Data
Center
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Notional View of GPS-RO
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
GPS-RO Neumonic Device
Steve Martin demonstrates GPS-RO bending angle
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Outline
Climate Data Records (CDRs) and Principles of Scientific Data Stewardship (SDS)
CDR Example – Upper-Tropospheric Water Vapor (UTWV or UTH)
The NASA-NOAA model for research-operations transitions of CDRs
Assessing the maturity of COSMIC GPS-RO
CDRs
UTWV
R-O
COSMIC
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Principles of Scientific Data Stewardship
Develop archive stewardship to preserve information content of raw data stream,
Process large volumes of satellite data extending up to decades in length to account for systematic errors and to eliminate artifacts in the raw data (referred to as fundamental climate data records, FCDRs),
Generate retrieved geophysical parameters from the FCDRs (referred to as thematic climate data records TCDRs) including combining observations from all sources,
Conduct monitoring and research by analyzing data sets to conduct climate science and applied research – Toward a National Climate Service– Climate variability and change, global water, energy,
carbon cycles– Societal Applications – GEOSS benefit areas – Climate
Information Records (CIRs)
CDRs
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Sensor DataRecords (SDRs)
Data (Direct & Remotely Sensed)
Fundamental Climate Data
Records (FCDRs)
Thematic Climate Data Records
(TCDRs)
Climate Data Records or Homogenized Time Series
Homogenization and Calibration
Time-tagged Geo-Referenced
Converted to Bio-Geophysical
Variables
EnvironmentalData Records
(EDRs)
Converted to Bio-Geophysical
Variables
Defining CDRsClimate Data Records
CDRs
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Upper Tropospheric Water Vapor CDR
Upper tropospheric water vapor (UTWV or UTH) channel has flown operationally since 1979
UTWV was originally dismissed (until 1990)– Didn’t compare well with radiosondes– Spectroscopy of WV uncertain
Lindzen’s 1990 BAMS paper ‘Some coolness concerning global warming’ spurred new attention
Water vapor feedback accounts for ½ to 2/3 of total global warming in model projections
UTWV
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
UTWV Long-term Intercalibration
246
246.5
247
247.5
248
248.5
249
1980 1984 1988 1992 1996 2000 2004
30S - 30N
N06N07
N08N09
N10N11
N12N14
N15N16
N17
Year
238
240
242
244
246
248
1980 1984 1988 1992 1996 2000 2004
30S - 30N
N06N07N08N09N10N11N12N14N15N16N17
Year
UTWV
Before
After
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Becoming a CDR ‘Benchmark’
An objective benchmark may be determined by maturity assessment
A subjective benchmark is determined by independent application of a data set to climate monitoring, forcings, or feedbacks
Evidence for Strengthening of the Tropical General Circulation in the 1990s by Chen, Carlson and Del Genio
UTWV
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Thoughts on Benchmark Concept The term Benchmark (Goody, 2001) carries particular
importance in the context of long-term climate monitoring and with respect to testing the veracity of climate model predictions. The central elements in the definition of a climate Benchmark are:– Accuracy that extends over decades, or indefinitely;– Variable critical to defining long-term climate change that is
observed on the global scale;– A measurement that is tied to irrefutable standards, usually
with a broad laboratory base;– Observation strategy designed to reveal systematic errors
through independent cross-checks, open inspection, and continuous interrogation;
– Limited number of carefully selected observables, with highly confined objectives defining (a) climate forcings, (b) climate response.
Just as the concept of ‘truth’, as in ‘ground truth’ is perhaps more a religious concept, so may be the concept of benchmark
A better definition may be in the combined objective maturity measures (as above) combined with the quintessential hallmark of the scientific process – independent peer review
UTWV
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
EVOLUTION OF A CLIMATE DATA RECORD (CDR)
Develop Joint Model of CDR Process
Develop a "Generic" Budget
Develop a list of Candidate CDRs focusing on the NPOESS
C1 Mission
Assess the Maturity and Cost Complexity of each Candidate
CDR
Priortize the Candidate CDRs for order of implementation
Develop a schedule of implementation with a gradual
start
Develop the accompanying budget
Planning Development Path
R-O
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Core Activities in CDR Evolution
Sensor calibration and characterization Algorithm development and refinement
– Continuing incubation of algorithm alternatives that may eventual replace previous “standard”
Product (Re-)Processing– Research & Operational agencies co-generate
Maturity Level 3-4 products as part of transition
– Assures transition readiness Product validation and use-driven
evaluation Archive, Distribution, Documentation
R-O
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Notional Evolution of a CDR(From NASA-NOAA Climate Science Working
Group)
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Notional Evolution of GPS-RO(From Anthes et al.)
Need to map this into NASA-NOAA notional CDR evolution Gantt chart
Do we restore GPS-RO to NPOESS?
Activity Name Start Date Finish Date 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015
Launch of COSMIC 10/4/05
Operation of COSMIC 10/18/05 10/17/10
Planning for COSMIC II 10/1/04 9/30/06
Development of COSMIC II 10/1/06 10/18/09
Launch of COSMIC II 10/18/09
Operation of COSMIC II 10/18/09 10/17/14
Planning for COSMIC III 10/1/09 9/30/10
Development of COSMIC III 10/1/10 10/17/13
Launch of COSMIC III 10/18/13
Operation of COSMIC III 10/18/13 10/17/18
R-O
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Operational Climate Data Records –Prioritization, Production, & Productivity
R-O
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
CRD Maturity Research-Operations
NO
AA
LE
AD
R-O
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Cost Estimation Approach
1. Notional CDR lifecycle provides schedule and activity breakdown– NASA historical cost data estimates cost/activity– Separate Research and Operational cost profiles
2. Algorithm maturity* determines relative year in notional CDR lifecycle
3. Production complexity* determines multiplier of notional cost profile
4. CDR “ramp-up” rate treated as independent variable– Required CDR prioritization straw man
*Maturity and complexity estimates from joint agency sensor expert teams (names provided in April Panel brief)
R-O
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Complexity Serves As Multiplier of Notional Cost Profile
Factors:
1) Number, quality & diversity of input streams
2) Resolutions (vertical, horizontal, temporal, spectral)
3) Algorithm complexity4) Algorithm outputs (#CDRs)5) Cal/Val complexity and cost
R-O
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
NOAA GPS-RO Work (NOAA Industry Day; Courtesy J. Yo)
NWP– On-going 3-yr Joint Center for Satellite Data Assimilation
(JCSDA) development program to assimilate GPS-RO data into National Weather Service’s operational Global Forecast System (GFS)
– Developed, tested, and implemented the necessary components to assimilate GPS-RO observations (refractivity and bending angle) in GFS
Forward models to simulate the observations from analysis variables, and tangent linear and adjoint models
Quality control algorithms Error characterization models Data handling and decoding procedures
– Verification and impact evaluation procedures
Climate – Funded SDS proposal & AMS meetings– Ben Ho - UCAR Validation and Calibration of MSU/AMSU
Measurements and Radiosonde Observations using GPS RO Data for Improving Stratospheric and Tropospheric Temperature Trends Analysis
NOAA GPS-RO Industry Day – January 2008
COSMIC
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Meeting Future Needs (NOAA Industry Day; Courtesy J. Yo)
Continuity of COSMIC Mission after 2011– Maintain gain realized for global NWP– Maintain stable calibration/validation source for Climate
Data Records– Provide dense ionospheric soundings for Space WX– NWP Latency requirement drives downlink needs– Constellation Approach– Number/density of soundings ~ proportional to number of
receivers Multiple Global Navigation Satellite System Sources
– GALILEO, GLONASS – also increases number of soundings– Complementing radiometric satellite soundings
Recognize that GPR-RO is a KEY PART of the solution, not the whole
COSMIC
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Summary from Industry Day (NOAA Industry Day; Courtesy J. Yo)
GSP-RO becoming established as data source for– Numerical weather prediction– Climate– Ionospheric sounding– COSMIC demonstrating benefits of small-
satellite Constellation approach to GPS-RO
– Now is the time to consider how to maintain constellation capability for GPSRO after 2011
– NRC’s Decadal Survey published in early 2007; recommended follow-on GPS Radio Occultation satellite mission
New potential may exist for CDRs and climate sensors in NOAA
COSMIC
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Conclusions The foundations of research-
operations transition of satellite CDRs has begun within NOAA and NASA
NSF and NOAA should consider that work as a template for a potential GPS-RO transition
COSMIC should adopt a maturity model, assess progress on a regular basis, and encourage independent applications
How does GPS-RO ultimately fit in?
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
Backup Slides
Workshop on the Applications of GPS Radio Occultation to Climate17 March 2008
CDR Maturity MatrixMaturit
y Sensor UseAlgorith
m stability
Metadata & QA Documentation Validation Public
ReleaseScience &
Applications
1 Research MissionSignificant
changes likely
Incomplete Draft ATBD Minimal
Limited data availability to
develop familiarity
Little or none
2 Research MissionSome
changes expected
Research grade (extensive)
ATBD Version 1+Uncertainty estimated
for select locations/times
Data available but of unknown
accuracy; caveats
required for use.
Limited or ongoing
3 Research MissionsMinimal changes expected
Research grade (extensive);
Meets international
standards
Public ATBD; Peer-reviewed algorithm and
product descriptions
Uncertainty estimated over widely distribute
times/location by multiple investigators;
Differences understood.
Data available but of unknown
accuracy; caveats
required for use.
Provisionally used in
applications and assessments
demonstrating positive value.
4 Operational MissionMinimal changes expected
Stable, Allows provenance tracking and
reproducibility; Meets
international standards
Public ATBD; Draft Operational Algorithm
Description (OAD); Peer-reviewed algorithm and
product descriptions
Uncertainty estimated over widely distribute
times/location by multiple investigators;
Differences understood.
Data available but of unknown
accuracy; caveats
required for use.
Provisionally used in
applications and assessments
demonstrating positive value.
5
All relevant research and operational
missions; unified and coherent record
demonstrated across different sensors
Stable and reproducible
Stable, Allows provenance tracking and
reproducibility; Meeting
international standards
Public ATBD, Operational Algorithm Description (OAD) and Validation Plan; Peer-reviewed algorithm,
product and validation articles
Consistent uncertainties estimated over most
environmental conditions by multiple
investigators
Multi-mission record is publicly
available with associated uncertainty
estimate
Used in various published
applications and assessments by
different investigators
6
All relevant research and operational
missions; unified and coherent record over
complete series; record is considered
scientifically irrefutable following
extensive scrutiny
Stable and reproducible
; homogeneou
s and published
error budget
Stable, Allows provenance tracking and
reproducibility; Meeting
international standards
Product, algorithm, validation, processing and
metadata described in peer-reviewed literature
Observation strategy designed to reveal systematic errors
through independent cross-checks, open
inspection, and continuous
interrogation
Multi-mission record is publicly
available from Long-Term
archive
Used in various published
applications and assessments by
different investigators
R-O