data qc & archiving: overview and status seth mcginnis

Data QC & Archiving:Overview and Status

Seth McGinnis

4th NARCCAP Users Workshop

NCAR Boulder, CO

April 10-11, 2012

Outline

• Current Archive Status

• QC Process Overview

• Derived Data Products

• Citations and Contributions

The Data Pipeline

• Simulation

• Post-processing

• Data transfer

• QC

• Publishing

Simulation Status

NCEP CCSM CGCM3 GFDL HadCM3

CRCM done done done

ECP2 done done setup

HRM3 done done done

MM5I done done running

RCM3 done done done

WRFG done done done

TMSL done done

Current Pipeline Status

• Ready to publish

CAM3 current 3D• QC in progress

HRM3-gfdl T5• Post-proc in progress

ECP2-ncep 2D• QC not started yet

MM5I-ccsm 2D fill-inMM5I-ccsm curr T5

• Needs post-process(in-house at NCAR)

ECP2 ncep, gfdl 3D ECP2 gfdl-future 2DCAM3 future timeslice

• Needs post-process(by modelers)MM5I ncep: misc. 2DMM5I ccsm-future: 3D

RCM3 all runs:

cli, clw, wa, misc. 2D

QC Overview

• Tally variables & files

• cf-checker

• Time coordinate continuity

• Temporal min/max/mean

• Time-space cross-sections

Example QC Plots

Fixing errors

• Metadata errors: fix

• Data gaps: fill with missing_value

• Bad values: replace with missing_value

• More serious errors: contact modelers

Derived Data Products

• Derived variables: tools on website

• Temporal averages & climatologies

• Regridded to 0.5-degree lat/lon grid

• Online GIS access via databasin.org

• New gridded daily obs: MICA

Bukovsky Regions

http://www.narccap.ucar.edu/contrib/bukovsky/index.html

• Descriptions

• Maps

• Mask Files

Citation & Contributions

• Data Citation: http://www.narccap.ucar.edu/about/citation.html

• Submit papers, presentations: http://www.narccap.ucar.edu/cgi/submit-paper.pl

• Sharing software: http://www.narccap.ucar.edu/contrib/tools/https://wiki.ucar.edu/display/narccaptopics/Tools

• DOI for the dataset: in progress

Questions?