David R. Maidment (PI) David G Tarboton

Ilya ZaslavskyMichael Piasecki

Jon Goodall

With support from collaborators, postdocs and graduate students: Rick Hooper, Jon Duncan, David Valentine, Tom Whitenack, Jeff Horsburgh, Bora Beran, Tim Whiteaker, Ernest To, Cedric David

http://www.cuahsi.org/his.html

CUAHSI Hydrologic Information System Update

Definition

The CUAHSI Hydrologic Information System (HIS) is a geographically distributed network of hydrologic data sources and functions that are integrated using web services so that they function as a connected whole.

CUAHSI HIS Update

• Current Status of the Project

• HIS 1.0 Components– WaterOneFlow web services– Observations data model

• Digital Watershed

CUAHSI HIS Update

• Current Status of the Project

• HIS 1.0 Components– WaterOneFlow web services– Observations data model

• Digital Watershed

HIS Progress• Phase 1 grant, 2.5 year grant completed• Phase 2 grant, 5-yr, $4.5M to Maidment (Texas)• Co-PI’s

– Ilya Zaslavsky (San Diego Supercomputing Center)– David Tarboton (Utah State)– Michael Piasecki (Drexel)– Jon Goodall (Duke)

• WATERS Testbeds serve as beta-testers for software

• First community-wide distribution in 12 – 18 months (dependant on establishing support services)

Project co-PI in Phase 2

Collaborator in Phase I

CUAHSI HIS Components(in development)

http://www.cuahsi.org/his.html

SuperComputerCenters:NCSA,TACC

Domain Sciences:

Unidata, NCARLTER, GEON

Government:USGS, EPA,

NCDC, USDA

Industry:ESRI, Kisters,

MicrosoftCUAHSI HIS

HIS Team and its Cyberinfrastructure Partners

HIS Team:Texas, SDSC,Utah, Drexel,

Duke

HISTeam

WATERSTestbed

WATERS Network Information System

SupercomputerCenters:NCSA,TACC

Domain Sciences:

Unidata, NCARLTER, GEON

Government:USGS, EPA,

NCDC, USDA

Industry:ESRI, Kisters,

OpenMI

HISTeam

WATERSTestbed

WATERS Network Information System

CUAHSI HIS

HIS, WATERS and the CUAHSI Community

SupercomputerCenters:NCSA,TACC

Domain Sciences:

Unidata, NCARLTER, GEON

Government:USGS, EPA,

NCDC, USDA

Industry:ESRI, Kisters,

OpenMI

HISTeam

WATERS Testbed

WATERS Network Information System

CUAHSI HIS

International Partners

CSIRO Land and Water ResourcesWater Resources Observations Network

(WRON)

European CommissionWater database design and model integration

(HarmonIT and OpenMI)

CUAHSI HIS Update

• Current Status of the Project

• HIS 1.0 Components– WaterOneFlow web services– Observations data model

• Digital Watershed

Rainfall & SnowWater quantity

and quality

Remote sensing

Water Data

Modeling Meteorology

Soil water

Water Data Web Sites

NWISWeb site output# agency_cd Agency Code# site_no USGS station number# dv_dt date of daily mean streamflow# dv_va daily mean streamflow value, in cubic-feet per-second# dv_cd daily mean streamflow value qualification code## Sites in this file include:# USGS 02087500 NEUSE RIVER NEAR CLAYTON, NC#agency_cd site_no dv_dt dv_va dv_cdUSGS 02087500 2003-09-01 1190USGS 02087500 2003-09-02 649USGS 02087500 2003-09-03 525USGS 02087500 2003-09-04 486USGS 02087500 2003-09-05 733USGS 02087500 2003-09-06 585USGS 02087500 2003-09-07 485USGS 02087500 2003-09-08 463USGS 02087500 2003-09-09 673USGS 02087500 2003-09-10 517USGS 02087500 2003-09-11 454

Time series of streamflow at a gaging station

USGS has committedto supporting CUAHSI’sGetValues function

Observation Stations

Ameriflux Towers (NASA & DOE) NOAA Automated Surface Observing System

USGS National Water Information System NOAA Climate Reference Network

Map for the US

Water Quality Measurement Sites in EPA Storet

Substantial variation in data availability from states

Data from Bora Beran, Drexel University

Water Quality Measurement Sites from Texas Commission for Environmental Quality (TCEQ)

Geographic Integration of Storet and TCEQ Data in HIS

Observations CatalogSpecifies what variables are measured at each site, over what time interval,

and how many observations of each variable are available

CUAHSI Hydrologic Data Access System

A common data window for accessing, viewing and downloading hydrologic information

USGSUSGS

NASANASANCDCNCDCEPAEPA NWSNWS

Observatory DataObservatory Data

(being built using HIS Server in collaboration with ESRI)

HIS Server

• Supports data discovery, delivery and publication– Data discovery – how do I

find the data I want?• Map interface and

observations catalogs• Metadata based Search

– Data delivery – how do I acquire the data I want?

• Use web services or retrieve from local database

– Data Publication – how do I publish my observation data?

• Use Observations Data Model

HIS Server and AnalystHIS Server

Implemented at San Diego

Supercomputer Center and at

academic departments and research

centers

Implemented by individual hydrologic

scientists using their own analysis

environments

HIS Analyst

Web Services

Sustainable – industrial strength technology

Flexible – any operating system, model, programming language or application

Details of HIS Analyst are here

http://www.cuahsi.org/his/webservices.html

Point Observations Information Model

Data Source

Network

Sites

Variables

Values

{Value, Time, Qualifier}

USGS

Streamflow gages

Neuse River near Clayton, NC

Discharge, stage (Daily or instantaneous)

206 cfs, 13 August 2006

• A data source operates an observation network• A network is a set of observation sites• A site is a point location where one or more variables are measured• A variable is a property describing the flow or quality of water• A value is an observation of a variable at a particular time• A qualifier is a symbol that provides additional information about the value

http://www.cuahsi.org/his/webservices.html

Example: Matlab use of CUAHSI Web Services

% create NWIS Class and an instance of the classcreateClassFromWsdl('http://water.sdsc.edu/wateroneflow/NWIS/DailyValues.asmx?WSDL');WS = NWISDailyValues;% Site Info for Site of Interestsiteid='NWIS:02087500';strSite=GetSiteInfoObject(WS,siteid,'');strSite.site.siteInfo.siteName ans =NEUSE RIVER NEAR CLAYTON, NC lat=strSite.site.siteInfo.geoLocation.geogLocation.latitudelong=strSite.site.siteInfo.geoLocation.geogLocation.longitude lat =35.6472222long =-78.4052778

Variable and variableTimeInterval

strSite.site.seriesCatalog(1).series(:).variable ans = variableCode: '00065' variableName: 'Gage height, feet' units: 'international foot'ans = variableCode: '00060' variableName: 'Discharge, cubic feet per second' units: 'cubic feet per second' strSite.site.seriesCatalog(1).series(:).variableTimeInterval ans = beginDateTime: '1927-08-01T00:00:00' endDateTime: '2006-10-16T00:00:00'ans = beginDateTime: '1927-08-01T00:00:00' endDateTime: '2006-10-16T00:00:00'

getVariableInfovarcode='NWIS:00060';varInfo=GetVariableInfoObject(WS,varcode,'') varInfo = variables: [1x1 struct] varInfo.variables.variable ans = variableCode: '00060' variableName: 'Discharge, cubic feet per

second' units: 'cubic feet per second'

GetValues% GetValues to get the datasiteid='NWIS:02087500';bdate='2002-09-30T00:00:00';edate='2006-10-16T00:00:00';variable='NWIS:00060';valuesxml=GetValues(WS,siteid,variable,bdate,edate,'');

Parse XML and Analyze% Parse the XML into a Matlab object to work withvaluesobj=xml_parseany(valuesxml);...plot(date,flowval);datetick;

1920 1930 1940 1950 1960 1970 1980 1990 2000 20100

0.5

1

1.5

2

2.5x 10

4

cfs

Daily Discharge NEUSE RIVER NEAR CLAYTON, NC

NWISNWIS

ArcGISArcGIS

ExcelExcel

NCARNCAR

UnidataUnidata

NASANASAStoretStoret

NCDCNCDC

AmerifluxAmeriflux

MatlabMatlab

AccessAccess JavaJava

FortranFortran

Visual BasicVisual Basic

C/C++C/C++

Some operational services

CUAHSI Web ServicesCUAHSI Web Services

Data SourcesData Sources

ApplicationsApplications

Extract

Transform

Load

http://www.cuahsi.org/his/

• Search multiple heterogeneous data sources simultaneously regardless of semantic or structural differences between them

Objective

NWIS

NARR

NAWQANAM-12

request

request

request

request

request

requestrequest

request

request

return

return

return

return

return

returnreturn

return

return

What we are doing now …..

Michael PiaseckiDrexel University

Semantic MediatorWhat we would like to do …..

NWIS

NAWQA

NARR

generic

request

GetValues

GetValues

GetValues

GetValues

GetValues

GetValuesGetValues

GetValues

GetValues HODM

Michael PiaseckiDrexel University

CUAHSI HIS Update

• Current Status of the Project

• HIS 1.0 Components– WaterOneFlow web services– Observations data model

• Digital Watershed

Hydrologic Science

Hydrologic conditions(Fluxes, flows, concentrations)

Hydrologic Process Science(Equations, simulation models, prediction)

Hydrologic Information Science(Observations, data models, visualization

Hydrologic environment(Dynamic earth)

Physical laws and principles(Mass, momentum, energy, chemistry)

It is as important to represent hydrologic environments precisely with

data as it is to represent hydrologic processes with equations

Data Cube

Space, L

Time, T

Variables, V

D

“What”

“Where”

“When”

A simple data model

Continuous Space-Time Model – NetCDF (Unidata)

Space, L

Time, T

Variables, V

D

Coordinate dimensions

{X}

Variable dimensions{Y}

Space, FeatureID

Time, TSDateTime

Variables, TSTypeID

TSValue

Discrete Space-Time Data ModelArcHydro

Terrain Data Models

Grid

Contour and flowline

TIN

CUAHSI Observations Data Model

• A relational database at the single observation level (atomic model)

• Stores observation data made at points

• Metadata for unambiguous interpretation

• Traceable heritage from raw measurements to usable information

Streamflow

Flux towerdata

Precipitation& Climate

Groundwaterlevels

Water Quality

Soil moisture

data

Ernest To

Center for Research in Water ResourcesUniversity of Texas at Austin

20061011

What are the basic attributes to be associated with each single observation and how can these best be organized?

A data source operates an observation network A network is a set of observation sites

Data Source and Network Sites Variables Values Metadata

Depth of snow pack

Streamflow

Landuse, Vegetation

Windspeed, Precipitation

Data Delivery

Controlled Vocabulary Tables

e.g. mg/kg, cfs

e.g. depth

e.g. Non-detect,Estimated,

A site is a point location where one or more variables are measured

Metadata provide information about the context of the observation.A variable is a property describing the flow or quality of water

A value is an observation of a variable at a particular time

Data Discovery

Hydrologic Observations Data Model

See http://www.cuahsi.org/his/documentation.html

Feature

Waterbody

HydroIDHydroCodeFTypeNameAreaSqKmJunctionID

HydroPoint

HydroIDHydroCodeFTypeNameJunctionID

Watershed

HydroIDHydroCodeDrainIDAreaSqKmJunctionIDNextDownID

ComplexEdgeFeature

EdgeType

Flowline

Shoreline

HydroEdge

HydroIDHydroCodeReachCodeNameLengthKmLengthDownFlowDirFTypeEdgeTypeEnabled

SimpleJunctionFeature

1HydroJunction

HydroIDHydroCodeNextDownIDLengthDownDrainAreaFTypeEnabledAncillaryRole

*

1

*

HydroNetwork

*

HydroJunction

HydroIDHydroCodeNextDownIDLengthDownDrainAreaFTypeEnabledAncillaryRole

HydroJunction

HydroIDHydroCodeNextDownIDLengthDownDrainAreaFTypeEnabledAncillaryRole

1

1

CouplingTable

SiteID (GUID)HydroID (Integer)

MonitoringPoint

SiteIDSiteCode

SiteNameLatitudeLongitude…

Hydrologic Observations Data Model

1

1

OR

Independent of, but coupled to Geographic Representation

HODM Arc Hydro

Variable attributes

VariableName, e.g. dischargeVariableCode, e.g. 0060SampleMedium, e.g. waterValuetype, e.g. field observation, laboratory sampleIsRegular, e.g. Yes for regular or No for intermittentTimeSupport (averaging interval for observation)DataType, e.g. Continuous, Instantaneous, CategoricalGeneralCategory, e.g. Climate, Water QualityNoDataValue, e.g. -9999

m3/sL3/TCubic meters per second

Stage and Streamflow Example

Daily Average Discharge ExampleDaily Average Discharge Derived from 15 Minute Discharge Data

Water Chemistry from a profile in a lake

ODM and HIS in an Observatory Setting

e.g. http://www.bearriverinfo.org

WaterOneFlow Web Services

Data access through web

services

Data storage through web

services

Dow

nlo

ads

Upl

oa

ds

Observatory data servers

CUAHSI HIS data servers

3rd party data servers

e.g. USGS, NCDC

GIS

Matlab

IDL

Splus, R

Excel

Programming (Fortran, C, VB)

Web services interface

HDAS Website Portal and Map Viewer

Information input, display, query and output services

Preliminary data exploration and discovery. See what is available and perform exploratory analyses

HTML -XML WS

DL

- SO

AP

CUAHSI HIS Update

• Current Status of the Project

• HIS 1.0 Components– WaterOneFlow web services– Observations data model

• Digital Watershed

Digital WatershedHow can hydrologists integrate observed and

modeled data from various sources into a single description of the environment?

Digital WatershedHydrologic Observation

Data

GeospatialData

Weather and ClimateData

Remote SensingData

(NetCDF)

(GIS)(Relational database)

(EOS-HDF)

Digital Watershed

A digital watershed is a synthesis of hydrologic observation data, geospatial data, remote sensing data and weather

and climate data into a connected database for a hydrologic region

NHDPlus for Region 17E

NHDPlus Reach Catchments ~ 3km2

About 1000 reach catchments in each 8-digit HUC

Average reach length = 2km 2.3 million reaches for continental US

Reach Attributes

• Slope• Elevation• Mean annual flow

– Corresponding velocity

• Drainage area• % of upstream

drainage area in different land uses

• Stream order

http://www.daymet.org/

• Project sponsored by the European Commission to promote integration of water models within the Water Framework Directive

• Software standards for model linking• Uses model core as an “engine”• http://www.openMI.org

OpenMI Conceptual Framework

VALUES

All values are referenced in a what-where-when framework, allowing different data resources or models to communicate data

Space, L

Time, T

Variables, V

D

An application of the data cube to integrate simulation modelsJon Goodall, Duke University

HIS as OpenMI Components

water balance model

NWISStreamflow

DaymetPrecipitation

To calculate storage, the model needs inflow and outflow

To calculate storage, the model needs precipitation

Goal: Link the National HIS web services with a simple water balance model using OpenMI as the mediator

Trigger:

Calculate storage

Watershed Hydrovolumes

Geovolume is theportion of a hydrovolumethat contains solidearth materials

USGS Gaging stations

Hydrovolume

Stream channel Hydrovolumes

Need the capacity to represent Acoustic Doppler Current Profiler (ADCP) data (Iowa)

Residence time distributions

Integration of surface water and groundwater data

• Describe the relationship between surface water features ( e.g. streams

and waterbodies) with groundwater features (aquifers, wells).

• Enable the connection with the surface water data model

Hydro network Aquifers

In the future go to 3D...

Water OneFlow• We need a “Water OneFlow” – a common

window for water data and models

• Advancement of water science is critically dependent on integration of water information

Federal

AcademicLocal

State

Model

Flow

Precip