xbt fall rate workshop aoml, miami, fl march 10 – 12, 2008 1 managing xbt data in the global...

11XBT Fall Rate XBT Fall Rate WorkshopWorkshopAOML, Miami, FLAOML, Miami, FLMarch 10 – 12, 2008March 10 – 12, 2008

Managing XBT Data in the Managing XBT Data in the Global Temperature–Salinity Global Temperature–Salinity Profile Program (GTSPP) Data Profile Program (GTSPP) Data BaseBase

Charles Sun, Norman Hall and Melanie Charles Sun, Norman Hall and Melanie HamiltonHamilton

NOAA/National Oceanographic Data NOAA/National Oceanographic Data CenterCenter


OutlineOutline

What is GTSPP?What is GTSPP? GTSPP Best Copy Data and Data GTSPP Best Copy Data and Data

FormatsFormats Logic for the Depth Correction Logic for the Depth Correction

ProcessProcess XBT Probe Types StatusXBT Probe Types Status General DiscussionGeneral Discussion


What’s GTSPP? What’s GTSPP?

GTSPP is a joint WMO-GTSPP is a joint WMO-IOC program designed to IOC program designed to provide improved access provide improved access to the highest resolution, to the highest resolution, highest quality data as highest quality data as quickly as possible.quickly as possible.

GTSPP began as an GTSPP began as an official IODE pilot project official IODE pilot project in 1989.in 1989.

It went into operation in It went into operation in November 1990.November 1990.

Steering Group on the Global Temperature

Salinity Profile Program (GTSPP)

Committee on International Oceanographic Data

Exchange (IODE)

Intergovernmental Oceanographic Commission

(IOC)

Joint Commission on Oceanography and Marine

Meteorology(JCOMM)

World Meteorological Organization

(WMO)

GTSPP = GTSPP = GGlobal lobal TTemperature emperature SSalinity alinity PProfile rofile PProgramrogram


GTSPP PartnersGTSPP Partners

Major partners include Australia, Canada, France and USA.Major partners include Australia, Canada, France and USA.– Australia: Delay-Mode Data Assembly and QCAustralia: Delay-Mode Data Assembly and QC– Canada: Real-Time/Delay-Mode data assembly, QC and Canada: Real-Time/Delay-Mode data assembly, QC and

Duplicate managementDuplicate management– France: Delayed-Mode Data Assembly and QC and QC– USA/AOML: NOAA SEAS Real-time (delayed mode in preparation)– USA/NODC: Continuously managed database (CMD)USA/NODC: Continuously managed database (CMD)

Delayed mode data QCDelayed mode data QC Data Matching (Real-time / Delayed / Best Copy)Data Matching (Real-time / Delayed / Best Copy) Data available as monthly files Data available as monthly files Web Database AccessWeb Database Access Offline Data DeliveryOffline Data Delivery

New Partners include the National Marine Data and New Partners include the National Marine Data and Information Service (NMDIS) of China and the Federal Information Service (NMDIS) of China and the Federal Maritime, Hydrographic Agency (BSH) of Germany and Maritime, Hydrographic Agency (BSH) of Germany and Argentina INIDEP.Argentina INIDEP.

Contributing countries include those making at sea profile Contributing countries include those making at sea profile observations.observations.


GTSPP Best Copy Data GTSPP Best Copy Data and Data Formatsand Data Formats http://www.nodc.noaa.gov/GTSPP/http://www.nodc.noaa.gov/GTSPP/ http://data.nodc.noaa.gov/gtspp/http://data.nodc.noaa.gov/gtspp/ ftp://ftp.nodc.noaa.gov/pub/ftp://ftp.nodc.noaa.gov/pub/

data.nodc/gtsppdata.nodc/gtspp GTSPP Data FormatsGTSPP Data Formats

– (MEDS) ASCII Format(MEDS) ASCII Format– NetCDF FormatNetCDF Format


Logic for the depth Logic for the depth correction process - Icorrection process - I NODC worked with ISDM (formerly NODC worked with ISDM (formerly

MEDS) and CSIRO to develop logic for MEDS) and CSIRO to develop logic for the depth correction process of archived the depth correction process of archived XBT data.XBT data.

NODC does not make depth corrections NODC does not make depth corrections on the archived XBT data.on the archived XBT data.

Corrections to the depth would be Corrections to the depth would be applied only to the data that were placed applied only to the data that were placed on the WOCE DVDs (final version). on the WOCE DVDs (final version).


Logic for the depth Logic for the depth correction process - IIcorrection process - II If the data centers did make If the data centers did make

depth corrections, NODC depth corrections, NODC preserves the uncorrected depths preserves the uncorrected depths but loads the new corrected but loads the new corrected depths to a separate linked table depths to a separate linked table in the database and will allow in the database and will allow retrieval of those corrected retrieval of those corrected depths on special request. depths on special request.


Logic for the depth Logic for the depth correction process - IIIcorrection process - III Two new codes will be created to Two new codes will be created to

retain depth correction retain depth correction information in the surface codes information in the surface codes structure. structure.

The The "DPC$""DPC$" indicates the status indicates the status of depth correction and of depth correction and

The The "FRA$""FRA$" will retain the will retain the conversion factor of 1.0336. conversion factor of 1.0336.


Procedures for Setting Procedures for Setting DPC$ ValuesDPC$ Values

Inspect the values Inspect the values of Data Type = ‘BA’, of Data Type = ‘BA’, ‘XB’, or ‘DT’,‘XB’, or ‘DT’,

Check non-XBT PLAT Check non-XBT PLAT codes,codes,

Compare to tables Compare to tables of known of known correctable and correctable and non-correctable non-correctable types, andtypes, and

Set the values of Set the values of DPC$ accordingly.DPC$ accordingly.

Based on inspection of PLAT, PEQ$, PFR$, PRT$ and FRE$ Based on inspection of PLAT, PEQ$, PFR$, PRT$ and FRE$ valuesvalues


Depth Correction (DPC$) Depth Correction (DPC$) StatusStatus

The "DPC$" code will have the following states:The "DPC$" code will have the following states:– 01 = Known Probe Type, Needs Correction,01 = Known Probe Type, Needs Correction,– 02 = Known Probe Type, No Need to 02 = Known Probe Type, No Need to

Correct Correct – 03 = Unknown Probe Type, Not enough 03 = Unknown Probe Type, Not enough

information to know what to do, leave information to know what to do, leave alone,alone,

– 04 = Known XBT Probe Type, Correction 04 = Known XBT Probe Type, Correction was done, andwas done, and

– 05 = Unknown Probe Type, but a 05 = Unknown Probe Type, but a correction was done.correction was done.


Parameter Codes Surface Codes Parameter Codes Surface Codes GroupGroupParameter Codes History GroupsParameter Codes History Groups

Parameter Codes Surface Parameter Codes Surface Codes GroupCodes Group

DPC$DPC$ | XBT depth | XBT depth correction statuscorrection status

FRA$FRA$ | Fall rate (correction | Fall rate (correction factor for XBT probes)factor for XBT probes)

FRE$FRE$ | Code for fall rate | Code for fall rate equation used. See IGOSS equation used. See IGOSS code table.code table.

PEQ$PEQ$ | XBT fall rate | XBT fall rate equation (WMO code 1770)equation (WMO code 1770)

PFR$PFR$ | XBT probe type and | XBT probe type and fall rate equationfall rate equation

PRT$PRT$ | XBT probe type | XBT probe type (WMO code 4770)(WMO code 4770)

Processing Step (PRC_Code) ld08 | NODC program to build DPC$ for all stations

with datatype= XB, BA, or DT ld07 | NODC program to fix XBT depth corrected and ld07 | NODC program to fix XBT depth corrected and

original valuesoriginal values SQLP | NODC procedures to fix various database SQLP | NODC procedures to fix various database

problems using SQLproblems using SQL

Parameter Codes for Histories (Act_Parm in History group)Parameter Codes for Histories (Act_Parm in History group) ADDP | NODC code for adding DPC$ value.ADDP | NODC code for adding DPC$ value. ADPF | NODC code for adding PFR$ value.ADPF | NODC code for adding PFR$ value. ADPQ | NODC SQL procedure for Adding a PEQ$ value ADPQ | NODC SQL procedure for Adding a PEQ$ value

to the surfacecodes tableto the surfacecodes table ADPR| NODC SQL procedure for Adding a PRT$ value ADPR| NODC SQL procedure for Adding a PRT$ value

to the surfacecodes tableto the surfacecodes table CHDP | NODC code for updating DPC$ value.CHDP | NODC code for updating DPC$ value. CHPF | NODC code for updating the PFR$ value.CHPF | NODC code for updating the PFR$ value. CHPQ | NODC code for updating PEQ$ CHPQ | NODC code for updating PEQ$ DTP$ | NODC SQL Procedure indicating erroneous DTP$ | NODC SQL Procedure indicating erroneous

Datatype based on matching CTD dataDatatype based on matching CTD data FXPF | NODC SQL procedure to Fix a PFR$ value from FXPF | NODC SQL procedure to Fix a PFR$ value from

surfacecodes tablesurfacecodes table FXPQ | NODC SQL procedure to Fix a PEQ$ value from FXPQ | NODC SQL procedure to Fix a PEQ$ value from

surfacecodes tablesurfacecodes table FXPR | NODC SQL procedure to Fix a PRT$ value from FXPR | NODC SQL procedure to Fix a PRT$ value from

surfacecodes tablesurfacecodes table RMPQ | NODC SQL procedure to Remove a PEQ$ value RMPQ | NODC SQL procedure to Remove a PEQ$ value

from surfacecodes tablefrom surfacecodes table RMPR | NODC SQL procedure to Remove a PRT$ value RMPR | NODC SQL procedure to Remove a PRT$ value

from surfacecodes tablefrom surfacecodes table RMD$ = NODC SQL Procedure for removing DPC$RMD$ = NODC SQL Procedure for removing DPC$


Bathy Depth Corrected Bathy Depth Corrected by CSIROby CSIROExample 1Example 1: CSIRO Depth Corrected data --- WOCE : CSIRO Depth Corrected data --- WOCE

UOT CDUOT CD

Meta-data (Surface Codes Group)Meta-data (Surface Codes Group) FRA$ 1.0336FRA$ 1.0336 DPC$ 04DPC$ 04

History Group: History Group:

STATION SEQ ID PRCC VERS PRC_DATE AC ACTP AUXID STATION SEQ ID PRCC VERS PRC_DATE AC ACTP AUXID PREVVALPREVVAL

---------- ---------- ------ ------ ---- ------- -- ----- -------- -------------------- ---------- ------ ------ ---- ------- -- ----- -------- ----------

1777155 2 CS CSCB 1.0 20020709 DP DEPH 0.0 1777155 2 CS CSCB 1.0 20020709 DP DEPH 0.0 999.999999.999


Corrected Depths Corrected Depths Table Table vs.vs. Observation Observation TableTableSQL> desc corrected_depths; Name Null? Type -------------------------------- --------STATION NUMBER(38) PROFILE

NUMBER(38) SEQ NUMBER(38) DEPTHPRESS CHAR(6) DEPRESQ CHAR(1) PROFPARM CHAR(9) PROFQPARM CHAR(1))

SQL> desc observation;SQL> desc observation; Name Null? Type --------------------- -------- -------------- STATION NOT NULL NUMBER(38) PROFILE NOT NULL NUMBER(38) SEQ NOT NULL NUMBER(38) DEPTHPRESS CHAR(6) DEPRESQ CHAR(1) PROFPARM CHAR(9) PROFQPARM CHAR(1)

STATION PROFILE SEQ DEPTHP D PROFPARM P ---------- ---------- ---------- ------ - --------- -

1777155 0 0 0.0 1 29.000 2 1777155 0 1 2.1 1 29.000 2

….. Continued

1777155 0 59 122.0 1 22.100 2 1777155 0 60 124.0 1 21.500 2

STATION PROFILE SEQ DEPTHP D STATION PROFILE SEQ DEPTHP D PROFPARM PPROFPARM P ---------- ---------- ---------- ------ - ---------- ---------- ---------- ------ - --------- ---------- -1777155 0 0 0.0 1 29.000 1777155 0 0 0.0 1 29.000 2 21777155 0 1 2.0 1 29.000 1777155 0 1 2.0 1 29.000 2 2

…………continuedcontinued

1777155 0 59 118.0 1 22.100 1777155 0 59 118.0 1 22.100 2 21777155 0 60 120.0 1 21.500 1777155 0 60 120.0 1 21.500 2 2


Example 2: BATHY’s Example 2: BATHY’s matched to CTDmatched to CTD

Build meta-data designating CTD, generate audit trail of Build meta-data designating CTD, generate audit trail of changes to the database (Number found of this type: 1320)changes to the database (Number found of this type: 1320)

------------------------------------------------------------------------------------------------------------------------------DBID D DT A SC HIS DP PF PFVAL ACTIONDBID D DT A SC HIS DP PF PFVAL ACTION---------- - -- - -- --- -- -- ----- --------- -------------------- - -- - -- --- -- -- ----- --------- ---------- 2070 R BA Y 10 14 9 -1 xxxxx -DPC +PFR +HIS2070 R BA Y 10 14 9 -1 xxxxx -DPC +PFR +HIS 2085 R BA Y 10 14 9 -1 xxxxx -DPC +PFR +HIS2085 R BA Y 10 14 9 -1 xxxxx -DPC +PFR +HIS 2091 R BA Y 10 14 9 -1 xxxxx -DPC +PFR +HIS2091 R BA Y 10 14 9 -1 xxxxx -DPC +PFR +HIS2032802 R BA Y 9 8 8 0 ///99 mPFR -DPC +HIS2032802 R BA Y 9 8 8 0 ///99 mPFR -DPC +HIS

DBID -- The Station NumberDBID -- The Station NumberD -- dmode value (R or D) D -- dmode value (R or D) DT -- datatype (BA or XB)DT -- datatype (BA or XB)A -- Active (Y, N, or H) A -- Active (Y, N, or H) SC -- nsurfcSC -- nsurfcHIS – numhists HIS – numhists DP -- seq for DPC$ surface codeDP -- seq for DPC$ surface codePF -- seq for PFR$ surface code PF -- seq for PFR$ surface code PFVAL -- value of PFR$, if any (xxxxx for filler where PFVAL -- value of PFR$, if any (xxxxx for filler where none)none)

ACTION -- Recommended action to ACTION -- Recommended action to take take STUB -- stub station, take no actionSTUB -- stub station, take no actionISBT -- Is an XBT, according to ISBT -- Is an XBT, according to PFR$, PFR$, so take no actionso take no actionmPFR -- Modify the existing PFR$mPFR -- Modify the existing PFR$-DPC -- Remove the existing DPC$-DPC -- Remove the existing DPC$+PFR -- Add a PFR$ group+PFR -- Add a PFR$ group+HIS -- Add a history for the action+HIS -- Add a history for the action


Archived XBT Probe Archived XBT Probe States As of 4 March States As of 4 March 20082008

DPC$='01', 237,840,

22%

DPC$='02', 398,773,

37%

DPC$='03', 442,287,

41%

1,078,900 XBT stations (profiles) out of 4,212,422 active stations ~= 25.6% of total active stations are XBT.


Where we go from Where we go from here?here? Keep GTSPP informed regarding Keep GTSPP informed regarding

the XBT fall rate changes.the XBT fall rate changes. GTSPP would like to get XBT GTSPP would like to get XBT

metadata via its connections with metadata via its connections with JCOMM and IODE.JCOMM and IODE.


[email protected]

xbt fall rate workshop aoml, miami, fl march 10 – 12, 2008 1 managing xbt data in the global...

Documents