framework for analyzing and visualizing met-ocean data
TRANSCRIPT
![Page 1: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/1.jpg)
![Page 2: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/2.jpg)
2
![Page 3: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/3.jpg)
3
![Page 4: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/4.jpg)
ROMS/COAWST NcML file
4
![Page 5: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/5.jpg)
5
![Page 6: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/6.jpg)
Exploiting IOOS: A Distributed, Standards-Based Framework and
Software Stack for Searching, Accessing, Analyzing and Visualizing Met-Ocean Data
Rich Signell (USGS-CMG) Filipe Fernandes (SECOORA)
Kyle Wilcox (Axiom Data Science) Andrew Yan (USGS-CIDA)
Regional IOOS DMAC Meeting: Silver Spring, 5/28/2015
![Page 7: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/7.jpg)
Objectives
• Set up a standards-based framework for easy and efficient access to insitu and ocean model data
• Provide a high-level search and browse web interface for program datasets, for scientists, end users and program managers
• Contribute to a growing standardized data search, access and use infrastructure that supports all geoscience
![Page 8: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/8.jpg)
Why not just use ERDDAP?
• Two reasons: • 1. Unstructured grid models • 2. Curvilinear grid models
![Page 9: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/9.jpg)
UGRID Conventions on GitHub
![Page 10: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/10.jpg)
SGRID Conventions: github/sgrid
![Page 11: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/11.jpg)
IOOS Model Data Interoperability Design
ROMS
ADCIRC
HYCOM
SELFE
NCOM NcML
NcML
NcML
NcML
NcML
Common Data Model
OPeNDAP+CF
WCS
NetCDF Subset
THREDDS Data Server
Standardized (CF-1.6, UGRID-0.9) Virtual Datasets
Nonstandard Model Output Data Files
Web Services Matlab
Panoply
IDV
Clients
NetCDF-Java
Library or Broker
WMS
ncISO
ArcGIS NetCDF4-Python
FVCOM
Python ERDDAP
NetCDF-Java
SOS
Geoportal Server
GeoNetwork
GI-CAT
Observed data (buoy, gauge, ADCP, glider)
Godiva2
pycsw-CKAN NcML
Grid Ugrid TimeSeries Profile Trajectory TimeSeriesProfile
Nonstandard Data Files
Catalog Services
![Page 12: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/12.jpg)
Interoperable Model Comparison in Matlab (using nctoolbox)
![Page 13: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/13.jpg)
compare_secoora_model_sections.m
![Page 14: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/14.jpg)
3D visualization of data with IDV
![Page 15: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/15.jpg)
NECOFS Access in ArcGIS (using the dap2arc python toolbox)
![Page 16: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/16.jpg)
USGS CMG Portal
![Page 17: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/17.jpg)
NetCDF Point Subset Service
![Page 18: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/18.jpg)
Iris Python tools from the UK Met Office
![Page 19: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/19.jpg)
Automated model comparison
![Page 20: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/20.jpg)
![Page 21: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/21.jpg)
Getting your model results connected
• Find someone with a THREDDS Data Server or install your own
• Drop your files in a directory, and add an NcML file that starts with “00_dir” (e.g. “00_dir_roms.ncml”) to aggregate, standardize and describe the dataset: Sample ROMS NcML file
• If you want your data to end up in the portal, add “CMG_Portal” to the “project” attribute: <attribute name=“project”value=“CMG_Portal”/>
• If you want your datasets to be discoverable, submit a PR on list of thredds catalogs being scanned on github
• Full instructions on the USGS-CMG Portal Github Wiki
![Page 22: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/22.jpg)
A few problems… Packaging
• Ipython notebooks are a great way to document model skill assessment workflows (Filipe will talk about this)
• But python environment uses a lot of tricky packages. How to make this easy for folks?
• Conda and binstar to the rescue! (Filipe will talk about this)
![Page 23: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/23.jpg)
A few problems… WMS
• ncWMS works great for CF compliant data • Unstructured grids are not CF compliant. • Staggered grids are not CF compliant. • ncWMS doesn’t work for unstructured grid
data (FVCOM, ADCIRC, SELFE), and doesn’t work for staggered grid velocities in models like ROMS, WRF and Delft3D
• sci-wms to the rescue, using UGRID conventions for unstructured grid (pyugrid), and SGRID conventions for staggered grid (pysgrid). (Kyle will talk about this)
![Page 24: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/24.jpg)
Key Infrastructure Components
• Common data models for “feature types” (structured, staggered and unstructured grids, time series, profiles, swaths) (Unidata CDM, UGRID, SGRID)
• Standard web data services for delivering these common data model “feature types” (OPeNDAP/CF/UGRID/SGRID, WMS, SOS, WFS, ERDDAP/tabledap, ERDDAP/griddap)
• Standard catalog services for the metadata (OGC CSW, OpenSearch)
• Tools for easy delivery of data in standard services
• Tools for easy search, access and use of data in standard services (in all major environments: Python, ArcGIS, R, Matlab, JavaScript)
![Page 25: Framework for Analyzing and Visualizing Met-Ocean Data](https://reader030.vdocument.in/reader030/viewer/2022012811/61c14883a5a5034edb472884/html5/thumbnails/25.jpg)
Infrastructure Benefits
• What are the benefits? – Less time wasted messing with data,
more time spent on science – More skill assessment of models – More usage and more appropriate
useage of model results – Faster feedback to modelers =>
improved models – Better science, better models =>better
world