research applications with click to edit master title ... · humidity profiles” – jessica...

Click to edit Master title style

Click to edit Master subtitle style

1

ResearchapplicationswithNOAA’sClimateDataRecord

Program

JessicaMatthewsNovember29,2016

2

NOAA’sCDRProgram

Mission:Todevelopandimplementarobust,sustainable,andscientificallydefensibleapproachtoproducingandpreservingclimaterecordsfromsatellitedata.

https://www.ncdc.noaa.gov/cdr

3

WhatareCDRs?

• TheNationalResearchCouncil(NRC)definesa CDR asatimeseriesofmeasurementsofsufficientlength,consistency,andcontinuitytodetermineclimatevariabilityandchange.

• NOAAisapplyingmoderndataanalysismethodstohistoricalglobalsatellitedatatoreturnneweconomicandscientificvaluefromtherecords.

4

WhyareCDRsimportant?

• Providetrustworthyinformationonhow,where,andtowhatextenttheland,oceans,atmosphereandicesheetsarechanging.

• Thisinformationwillbeusedbyenergy,waterresources,agriculture,humanhealth,nationalsecurity,coastalcommunityandotherinterestgroups.

5

FundamentalandthematicCDRs

• CDRs aregroupedbyFundamental CDRs andThematic(Atmospheric,Oceanic,andTerrestrial) CDRs.

• Fundamental CDRs aresensordata(e.g.calibrated radiances,brightnesstemperatures)thathavebeenimprovedandqualitycontrolledovertime,togetherwiththeancillarydatausedtocalibratethem.

• Thematic CDRs aregeophysicalvariablesderivedfromthe FCDRs.

6

CDRexamples

• Fundamental:– Brightnesstemperatures,Reflectances,Meanlayertemperatures

• Atmospheric:– Aerosolopticalthickness,Cloudproperties,Ozone,Precipitation

• Oceanic:– Seaiceconcentration,Seasurfacetemperature

• Terrestrial:– Snowcoverextent,LAI,NDVI,FAPAR

7

NOAACDROperationalSpiralDevelopment

6 – BenchmarkCDR

5 – FullOperationalCDR

4 – SustainedCDR

3 – InitialOperationalCDR

2 – ResearchCDR

1 - ConceptualCDR

Ope

ratio

ns

Research

8

ResearchtoOperations•Goal- TomovefromtheResearchandDevelopmentofaCDRtoInitialOperationalCapability(IOC).

• IOC- ACDRdevelopmentstateachievedwhenaCDR(includingthedataset,sourcecode anddocumentation)isscientificallydefensibleandisarchived,maintained,andmadepubliclyavailable byNOAA.Itsdatamayormaynothavebeenindependentlyreproduced.Itisnotregularlymaintainednorisitnecessarilyextendedasnewdatabecomeavailable.

• FOC- ACDRdevelopmentstatewhenaCDRwillbesystematicallyandoperationallymaintainedinacycleofcontinuousimprovement.Sourcecodeanddocumentationmeethighstandardsandresultsarereproducible.Therecordwillberoutinelyextendedasnewdatabecomeavailableortoreprocesstherecordifrecalibrateddatabecomeavailable.

R2O IOC FOC

9

CDRR20ProcessDiagram1)Assess

– AssembleIPT: PI,PIAssistant,SME,PM,Ops,Archive,Access– Assesssamplesofdata,documentation,sourcecode– Developtransitionschedule

2)Prepare– PIUpdatesSourceCode(CDRPcodingstandards,README)– PICreatesDocumentation(CATBD,FlowChart,MaturityMatrix)– PIPrepsDatafordelivery(netCDFFormat,Metadata,SubmissionAgreement)

3)Transfer– PItransfersacopyoftheSourceCode,Documentation,andDatatoNCDC

4)Verify– IPTverifiesSourceCode,Documentation,andData– Iterativeprocessuntilcode,documentation,anddatameetstandards

5)Archive– NCDCpreservesacopyofSourceCode,DocumentationandDatainthearchive

6)Access– ORR,thenprovidepublicaccesstoeverythingviatheCDRaccessportal

5/30/2012 OperationalReadinessReview

• ProductDescription• ORRChecklist• Websitedemo

Codemadeavailable

Documentationmadeavailable

Productmadeavailable

6)Access

ArchiveAncillaryData

ArchiveCodePackage

ArchiveDocumentation

ArchiveProductData

5)Archive

Verify‘README’

4)Verify

VerifyDataandMetadata

KDP

OperationsBranch ArchiveBranch AccessBranchPI

CDRAssessment

1)Assess

Entercodeintoversioncontrol

2)Prepare

Data

Documents

Code

VerifyFlowDiagram,

CATBD,andMaturityMatrix

3)Transfer

TransferDocuments

Verifysourcecodepackage

ConductSecurityReview

ITBranch

CreateDocuments

CDRP

TransferData

Iterativeuntilallpartshavebeenverified

CreateData

Updatesourcecode

Submissionagreement

TransferSourcecode

IOC

ORR

CDRIPT

Science Branch

10

IntegratedProductTeam

• R2OTransition– NotataskthePIdoesalone,partnershipwiththeIPT

• IPTconsistsof:– PIandanyAssistants– R2OProjectManager– InternalNCEIScientist– ArchiveBranchrepresentative– OperationsBranchrepresentative– AccessBranchrepresentative– AdditionalsupportfromIT,UserEngagement,andotherCDRPmembers

• NCEIpersonnelspend400-600hourstransitioningeachCDRtoIOC

• TypicalR2Otransitiontakestheteam8-12monthstocomplete

11

CICS-NCsupportofCDRs

Science

Scientificprogramming

Stewardship

12

Tropicalmonitoringandprediction

• TheMadden–JulianOscillation(MJO),equatorialRossby waves,andKelvinwavesaredominantsourcesofsynoptic-to-subseasonalvariabilityinthetropics

• Divergentcirculationsfromtheirconvectioncaninfluencetropicalcyclonesandotherweatherpatternsaroundtheglobe

• ForecastersintheenergyindustryharnesstheselongtimescalesandglobalimpactstoanticipateenergydemandintheUS

13

Tropicalmonitoringandprediction

• TropicalmonitoringderivedinpartfromOutgoingLongwaveRadiationCDRdata

• 18hr lag• ForecastsfromNCEPClimate

ForecastSystem• Tropicalcyclogenesistracking• Diagnosticsusedoperationally

byNOAACPCandAirForce14th WeatherSquadron

https://ncics.org/portfolio/monitor/mjo/

14

High-ResolutionNEXRADReanalysis

• Providesrainfallamountsat1-kmresolutioneveryfivemin

• Available from2000topresent• Processingstats:– 130millionradarfiles– 500kjobs– 3millionhrs CPU-time– ~1yr real-time

15


● Israinfallassociatedwithanincrease/decreaseinfatalaccidents?

● Howdoesthisvary...- byregion?- byseason?- bytimeofday?

● Comparefrequencyofcrasheswhichtakeplaceduringtheraintothefrequencyofrainitself

● Intheabsenceofacorrelation,thesefrequenciesshouldberoughlyequal

16


• Blue=climatologicalfrequencyofrainfall

• Red=frequencyofcrashesduringrain

• Allregionsshowahigherpercentageofrain-associatedcrashesthanwouldbeexpectedbasedonclimatology

17

SeaIceConcentrationCDR

• ThePassiveMicrowaveseaiceconcentrationsareanestimateofthefractionofoceanareacoveredbyseaicemean STD

18


• Verificationandvalidation(a)CDR-GSFC (b)CDR-BT

(c)CDR-NT (d)BT-NT

bias

biasbias

biasRMSD

RMSDRMSD

RMSD

19


WholeArctic

CentralArctic(lightgreen) Greenland(darkgreen) HudsonBay(darkbrown)

SpatialandTemporalVariabilityofSeaIceCoverage

20

Obs4MIPS

• TheObservationsforModelInter-ComparisonProject(obs4MIPs)isanefforttomakeobservationaldataavailabletotheclimatemodelingcommunityinthesameformatthattheyusetoshareclimatemodeloutputs.

• TosupportCDRdataaccess,CICS-NCcreatedatooltoconvertCDRdatatoobs4MIPscompliantformatandspecifications.

21

Obs4MIPS

https://ncics.org/data/obs4mips/

OptimalInterpolationSeaSurfaceTemperature(OISST)

HIRSOutgoingLongwaveRadiation(OLR)

PassiveMicrowaveSeaIceAreaFraction

22

Otherresearchhighlightedhere

• “ScientificStewardship- WhatIsItandWhatDoesItMeantoUs?”– GePeng

• “DiurnalCycleofLandSurfaceTemperaturesunderAll-skyConditions”– Anand Inamdar

• “Long-termHIRS-BasedTemperatureandHumidityProfiles”– JessicaMatthews

• “AnEvaluationofSatelliteQuantitativePrecipitationEstimates(QPEs)Products”– OlivierPrat

• “LinkingnetCDF DatawiththeSemanticWeb-EnhancingDataDiscoveryAcrossDomains”– JimBiard

23

Summary

• NOAA’sCDRsarerobust,sustainable,andscientificallydefensible.

• CICS-NCsupportstheeffortwithscience,scientificprogramming,andstewardshipexpertise.

• Tolearnmore:https://www.ncdc.noaa.gov/cdr