versatile qa / validation system - earth online · belgisch instituut voor ruimte-aeronomie...

BELGISCH INSTITUUT VOOR RUIMTE-AERONOMIE INSTITUT D’AERONOMIE SPATIALE DE BELGIQUE BELGIAN INSTITUTE OF SPACE AERONOMY BELGISCH INSTITUUT VOOR RUIMTE-AERONOMIE INSTITUT D’AERONOMIE SPATIALE DE BELGIQUE BELGIAN INSTITUTE OF SPACE AERONOMY BELGISCH INSTITUUT VOOR RUIMTE-AERONOMIE INSTITUT D’AERO

[ ar

no.k

eppe

ns @

]

1

Versatile QA / validation system for the evaluation of atmospheric (ozone) data retrievals from GOME to the Copernicus Sentinels Era

A. Keppens, J. Granville, D. Hubert, T. Verhoelst, S. Compernolle, J.-C. Lambert Acknowledgements: ESA CCI Ozone, BELSPO/ProDEx TROVA ESA ACVE, 18-20 October 2016, ESA-ESRIN, Frascati (Rome), Italy


Royal Belgian Institute for Space Aeronomy

Institut royal d’Aéronomie spatiale de Belgique

Koninklijk Belgisch Instituut voor Ruimte-Aëronomie


[ ar

no.k

eppe

ns @

]

2

A. Keppens, J. Granville, D. Hubert, T. Verhoelst, S. Compernolle, J.-C. Lambert Acknowledgements: ESA ACVE, 18-20 October 2016, ESA-ESRIN, Frascati (Rome), Italy


Royal Belgian Institute for Space Aeronomy

Institut royal d’Aéronomie spatiale de Belgique

Koninklijk Belgisch Instituut voor Ruimte-Aëronomie

Versatile QA / validation system for the evaluation of atmospheric (ozone) data retrievals from GOME to the Copernicus Sentinels Era

custom/scientific operational

examples


[ ar

no.k

eppe

ns @

]

3

QA / validation system

mismatch error talk by Tijl Verhoelst

this presentation

terms and definitions poster by Steven

Compernolle

homogeneity talk by Daan Hubert

MIPAS L2 evolution talk by Daan Hubert

SCIAMACHY L2 evolution poster

AURORA poster by Ugo Cortesi

NDACC talk by Bavo Langerock

CCI validation poster by Andy Delcloo

EVDC talk by Ann Mari Fjaeraa


[ ar

no.k

eppe

ns @

]

4

All satellite & ground data in harmonised MAT / NC format (CF / GEOMS / HARP convention)

Overpass model & satellite data (or links) calculated/predicted and stored separately

Integrated outlier checks "Correlative-2" section: NDACC/SHADOZ/GAW Dobson,

Brewer, sonde, lidar, FTIR, MWR Harmonised in-house format Operational weekly sync

Correlative database + interface

Satellites

Networks

Campaigns

DA / CTM

CORRELATIVE DATABASE


[ ar

no.k

eppe

ns @

]

5

QA / validation chain: Needs

Validation is not straightforward, due to the complexity and variety of retrieval approaches, vertical smoothing, natural variability of (ozone) field, etc. Quality Indicators must enable users to assess the fitness-for-purpose of the data Many Quality Indicators cannot be accessed by classical comparisons with reference measurements

VALIDATION CHAIN MULTI-TASTE

Translation

User requirements

QA / VAL requirements

Quality Indicators

Analysis

Satellites

Networks

Campaigns

DA / CTM

CORRELATIVE DATABASE Selection

Co-locator

Harmonisation

Comparator

Reporting


[ ar

no.k

eppe

ns @

]

6

QA / validation chain: Specifications

1. Unbroken chain

2. Full traceability of data products, data manipulations, and associated uncertainties

3. Versatile in input and output

4. Tuneable expert system using switches (multiple paths)


Translation

User requirements


Quality Indicators

Analysis

Satellites

Networks

Campaigns

DA / CTM


Co-locator

Harmonisation

Comparator

Reporting


[ ar

no.k

eppe

ns @

]

7

QA / validation chain: 10-step

1. Translation of user requirements into validation requirements

2. Data selection / filtering 3. Data co-location 4. Data homogenisation:

representation conversion 5. Comparative studies 6. Reporting 7. Derivation of Quality Indicators 8. Analysis / compliance check Data content studies Information content studies


Translation

User requirements


Quality Indicators

Analysis

Satellites

Networks

Campaigns

DA / CTM


Co-locator

Harmonisation

Comparator

Reporting


[ ar

no.k

eppe

ns @

]

8

Processing: Data harmonisation

• Expected to reduce comparison errors due to differences in • prior (shape, weight, constraint) • vertical and horizontal resolution / sensitivity • air mass sampling (space-time mismatch)

• Yet introduces (hopefully smaller) data manipulation uncertainties related to • representation change: unit conversion,

regridding, integration… • smoothing approach and its technical

implementation (including unknown uncertainties on Averaging Kernels)

• model uncertainties Thus need for characterisation and propagation of uncertainties, as pioneered in EU FP7 QA4ECV and H2020 GAIA-CLIM


[ ar

no.k

eppe

ns @

]

9

Processing: Vertical matching

Distinction between: • Concentration (point) quantities • Column (integrated) quantities

Regridding methods: • Linear • Transpose linear • Pseudo-inverse linear • Pseudo-inverse linear over joint

super-grid (Calisesi, 2005) • Mass-conservation • Mass-conservation over super-grid

Smoothing using Averaging Kernel or predefined shape: Gaussian, box, triangle


[ ar

no.k

eppe

ns @

]

10

Processing: Practical implementation

Scientific/custom code tools E.g. Ozone-CCI / Envisat QWGs • create GND station/instrument list from val. criteria • select ‘best’ of multiple sources • read + filter GND data (only once) • limit space-time window to months/stations with good

GND data • read + filter SAT overpass data

(possibly several versions for delta-validation) • limit SAT data to common profiles of all versions

(for delta-validation) • co-locate measurements (possibly altitude-dependent) • extract coincidences (temporal) • keep closest pairs only (every profile used only once) • harmonise vertical coordinates and profile units • determine observables for regridding • regrid profile, VCM, AKM (several options) • smooth profile (several options) • determine vertical (range clipping / integration) • identify dependences • determine statistics • plotting tools

Operational NORS/HARP tools E.g. TROPOMI MPC VDAF Programming packages: • harp.ingest_product from "ingestion definitions" • harp.import_product Command line tools: • harpconvert • harpcheck • harpdump • harpfilter • harpcollocate

• collocation • obtaining collocation result • resampling collocation result

• harpprofile • resampling • smoothing

• harpmerge • online/offline combination • online visualisation S5P MPC talk by

Pepijn Veefkind

HARP talk and demo by Sander

Niemeijer


[ ar

no.k

eppe

ns @

]

11

QA / validation chain: 10-step

1. Translation of user requirements into validation requirements

2. Data selection / filtering 3. Data co-location 4. Data homogenisation:

representation conversion 5. Comparative studies 6. Reporting 7. Derivation of Quality Indicators 8. Analysis / compliance check Data content studies Information content studies


Translation

User requirements


Quality Indicators

Analysis

Satellites

Networks

Campaigns

DA / CTM


Co-locator

Harmonisation

Comparator

Reporting


[ ar

no.k

eppe

ns @

]

12

Goal: Broad view on data content and dependences, both before and after screening / co-location Typically histograms and scatter plots combining • Physical state quantities: time, latitude, longitude,

altitude/pressure, temperature, SZA, TPH, CF, SA, etc. • Retrieval properties: iterations, chi-square, cost function, flags • Retrieval outcomes: V/SCD, atmospheric state profile • Co-location settings: spatial distance, time difference

Data content studies


[ ar

no.k

eppe

ns @

]

13

Data content studies: Screening


[ ar

no.k

eppe

ns @

]

14

GOME-2A co-locs. with Uccle (Jan. 2008) IASI co-locs. with Uccle (Jan. 2008)

Data content studies: Co-location

Ozonesonde Lidar


[ ar

no.k

eppe

ns @

]

15

• Objective: Characterise sensitivity of retrieval to actual atmospheric profile: number of independent layers (DFS), Fisher information content, vertical sensitivity and resolution of the profile, height registration error (centroid offset)…

• Information content and properties are estimated from linear analysis of vertical Averaging Kernel matrix (AKM), product of Jacobians of inverse and forward models

• Dependence on SZA, SCD, clouds, instrumental degradation, etc. (if implemented in forward and/or inverse model)

• Fractional kernels required for unit-independent analysis!

Information content studies


[ ar

no.k

eppe

ns @

]

16


Backus-Gilbert spread Direct FWHM


[ ar

no.k

eppe

ns @

]

17


SCD [DU]


[ ar

no.k

eppe

ns @

]

18

• Comparison plots and tables • Bias and spread estimates • Statistical significance and chi-square testing • Difference error contributions (and EBC, cf. OSSSMOSE) • L2/3/4 assessments • Delta-validation for algorithm evolution • Validity in horizontal, vertical, temporal domains • Dependences on influence quantities • Trend and drift studies

Comparative studies


[ ar

no.k

eppe

ns @

]

19

Comparative studies IASI 2008 NH


[ ar

no.k

eppe

ns @

]

20

Comparative studies


[ ar

no.k

eppe

ns @

]

21

Summary


[ ar

no.k

eppe

ns @

]

22

A versatile QA/validation system is presented: • Containing a correlative database and an

unbroken and fully traceable validation chain • Including a suite of data manipulations and studies yielding

complementary perspectives on data quality: Detailed data and information content studies Careful / traceable treatment of representation, sampling, smoothing issues Comparisons with respect to ground-based reference measurements and

multivariate statistical analysis

• Providing quality indicators enabling air quality and climate data users to assess the fitness-for-purpose of the data

• Versatile & tuneable: Applicable to different sensors, retrieval approaches, reference data sources, etc.

• Currently operationalised for the Copernicus Sentinel-4/5(p)

Summary

versatile qa / validation system - earth online · belgisch instituut voor ruimte-aeronomie...

Documents