1 dapper and argo joe sirott pmel/noaa. 2 what is dapper? web server that provides distributed...
TRANSCRIPT
22
What is Dapper?What is Dapper?
Web server that provides distributed access to Web server that provides distributed access to in-situ data via OPeNDAP protocolin-situ data via OPeNDAP protocol
Clients include GrADS, Matlab, NcBrowse, Java Clients include GrADS, Matlab, NcBrowse, Java Ocean AtlasOcean Atlas
Available data:Available data:– PMEL EPIC databasePMEL EPIC database– ARGO profiles ARGO profiles – NODC World Ocean Database 2001NODC World Ocean Database 2001
All components are open-sourceAll components are open-source
33
What does Dapper do?What does Dapper do?
Efficiently aggregates thousands of in-situ data Efficiently aggregates thousands of in-situ data files (>2000000 for WODB2001)files (>2000000 for WODB2001)Handles several in-situ data formats:Handles several in-situ data formats:– Argo GDAC netCDFArgo GDAC netCDF– PMEL EPIC netCDFPMEL EPIC netCDF– COARDS compliant netCDFCOARDS compliant netCDF
Supports time series or profilesSupports time series or profilesSupports space/time queriesSupports space/time queriesBased on HPCC funded Climate Data Portal Based on HPCC funded Climate Data Portal (Soreide, Zhu, Denbo)(Soreide, Zhu, Denbo)
44
What is OPeNDAP?What is OPeNDAP?
OPeNDAP (Open source Project for a OPeNDAP (Open source Project for a Network Data Access Protocol) is a Network Data Access Protocol) is a protocol for remotely accessing scientific protocol for remotely accessing scientific datadata
Uses HTTP as transportUses HTTP as transport
55
Dapper OPeNDAP Example Dapper OPeNDAP Example (space/time query)(space/time query)
Client HTTP request:Client HTTP request:http://foo.org/.../dset.cdp.dods?lat,temp,pres&lat>0&lat<20http://foo.org/.../dset.cdp.dods?lat,temp,pres&lat>0&lat<20Dapper HTTP response:Dapper HTTP response:
Dataset {Dataset { Sequence {Sequence { Float32 lat;Float32 lat; Float64 _id;Float64 _id; Sequence {Sequence { Float32 pres,temp;Float32 pres,temp;
} profile;} profile; }} } location = { { -4.01,1, { {4.3,26.808}, {100,24.982}, …},} location = { { -4.01,1, { {4.3,26.808}, {100,24.982}, …}, { -52, 2, {{8.3,3.77},{100,3.743},…}{ -52, 2, {{8.3,3.77},{100,3.743},…}
66
Dapper OPeNDAP Example Dapper OPeNDAP Example (single profile)(single profile)
Client HTTP request:Client HTTP request:http://foo.org/.../dset.cdp.dods?lat,lon,time,profile&_id=20http://foo.org/.../dset.cdp.dods?lat,lon,time,profile&_id=20Dapper HTTP response:Dapper HTTP response:
Dataset {Dataset { Sequence {Sequence { Float64 time;Float64 time; Float32 lat,lon;Float32 lat,lon; Sequence {Sequence { Float32 pres,temp;Float32 pres,temp;
} profile;} profile; }} } location = { {1034140577999,-52.9,310.144},} location = { {1034140577999,-52.9,310.144},
{{8.3,3.77},{9.1,3.77},…}}; {{8.3,3.77},{9.1,3.77},…}};
77
Dapper architectural overviewDapper architectural overview
Uses relational database to store Uses relational database to store metadata and coordinate boundaries for metadata and coordinate boundaries for each profile/time series fileeach profile/time series file
Data maintained in original formatData maintained in original format
All data delivered as OPeNDAP All data delivered as OPeNDAP sequencessequences
88
Dapper architectureDapper architecture
CORBA (IIOP)
Climate Data Portal
MySQL
Java netCDFlibrary
netCDFprofile
netCDFprofile
Dapper HTTP Server
CDPservice
netCDFservice
OPeNDAP clientDChart
OPeNDAP Protocol(HTTP)
Database loader
99
DChartDChart
Browser based in-situ OPeNDAP clientBrowser based in-situ OPeNDAP client
Based on FY2004 HPCC funded project Based on FY2004 HPCC funded project for shipboard display of cruise datafor shipboard display of cruise data
AJAX based user interfaceAJAX based user interface
1010
DChart Design GoalsDChart Design Goals
Highly responsive browser based user Highly responsive browser based user interfaceinterface
Low learning curveLow learning curve
Rapid previewing and downloading of Rapid previewing and downloading of Dapper datasetsDapper datasets
Highly interactive map for region selectionHighly interactive map for region selection
No Java applets or proprietary plug-insNo Java applets or proprietary plug-ins
1111
Dapper architecture diagramDapper architecture diagram
CORBA (IIOP)
Climate Data Portal
MySQL
Java netCDFlibrary
netCDFprofile
netCDFprofile
Dapper HTTP Server
CDPservice
netCDFservice
OPeNDAP clientDChart
OPeNDAP Protocol(HTTP)
Database loader
1212
DChart architectureDChart architecture
Web Browser
JavaScript
DChart Servlet Dapper
OPeNDAP Protocol(HTTP)
XML/JavaScript(HTTP)
JFreeChartSGT
OPeNDAPClient
XMLEngine
1414
CDFsyncCDFsync
Rapid synchronization of netCDF in-situ files Rapid synchronization of netCDF in-situ files over a networkover a networkOnly compressed file differences transmitted Only compressed file differences transmitted across networkacross networkBased on rsync, but optimized for netCDF and Based on rsync, but optimized for netCDF and file lists consisting of thousands or millions of file lists consisting of thousands or millions of filesfilesFunded by HPCC 2004 (Charles Sun et al)Funded by HPCC 2004 (Charles Sun et al)Available at Available at http://www.epic.noaa.gov/epic/software/cdfsync/http://www.epic.noaa.gov/epic/software/cdfsync/
1515
Future plansFuture plans
Constraints (QC flags, attributes)Constraints (QC flags, attributes)
Metadata displayMetadata display
Trajectory and section plotsTrajectory and section plots
Multiple plot windowsMultiple plot windows
Save/restore Web app stateSave/restore Web app state
1717
More infoMore info
PMEL Dapper ServerPMEL Dapper Serverhttp://dapper.pmel.noaa.gov/dapperhttp://dapper.pmel.noaa.gov/dapper
PMEL Dapper UIPMEL Dapper UIhttp://dapper.pmel.noaa.gov/dcharthttp://dapper.pmel.noaa.gov/dchart
Downloads, propagandaDownloads, propagandahttp://http://www.epic.noaa.gov/epic/software/dapperwww.epic.noaa.gov/epic/software/dapper//
[email protected]@noaa.gov