A New Generation of Data Services for Earth System Science Education and

Research: Unidata’s Plans and Directions

AGU Fall MeetingSan Francisco, CA6 December 2005

Dr. Mohan RamamurthyDirector, Unidata Program Center

UCAR Office of ProgramsBoulder, CO

Science DriversScience Drivers Environmental problems like Environmental problems like

global change & water cycle global change & water cycle transcend disciplinary as well as transcend disciplinary as well as geographic boundaries, requiring geographic boundaries, requiring multidisciplinary approaches and multidisciplinary approaches and global teams for solving them;global teams for solving them;

Rapid advances in observational Rapid advances in observational technologies, especially in technologies, especially in remote sensing;remote sensing;

Increasing use of complex, Increasing use of complex, coupled modeling systems;coupled modeling systems;

Research studies on societal impact of hurricane-related flooding involve integrating data from atmospheric sciences, oceanography, hydrology, geology, geography, and social sciences.

Science Drivers: ExamplesScience Drivers: Examples

Emergency Response

Ensemble Predictions

Coastal Environments

GIS Integration

End to End Information Services

Need integrated services

Education DriversEducation Drivers

A “holistic” Earth-A “holistic” Earth-system science system science approach to educationapproach to education

Active, student-Active, student-centered learning. i.e., centered learning. i.e., learning science by learning science by doing sciencedoing science• Observations (data)Observations (data)• Tools (models, Tools (models,

visualization)visualization)• DiscoveryDiscovery

Technology Trends Enabling a Technology Trends Enabling a New Generation of Data ServicesNew Generation of Data Services

• Internet & the World Wide WebInternet & the World Wide Web• Commodity microprocessorsCommodity microprocessors• Object-oriented programmingObject-oriented programming• Open standardsOpen standards• Web services Web services • Extensible Markup Language Extensible Markup Language

(XML)(XML)• Global, high-bandwidth and Global, high-bandwidth and

wireless networkswireless networks• Digital librariesDigital libraries• CollaboratoriesCollaboratories• Grid Computing/e-ScienceGrid Computing/e-Science• Data Portals and Federated, Data Portals and Federated,

distributed Serversdistributed Servers• Geographic Information Systems Geographic Information Systems • Knowledge environmentsKnowledge environments• Ontologies and Semantic web Ontologies and Semantic web • Data mining and knowledge Data mining and knowledge

discoverydiscovery

Data Services: An Evolution

An evolution from proprietary data systems towards more open standards-based data services – i.e. web services.Data services should address the myriad applications and needs of the community: research, education, outreach, collaboration, etc.

This transition to web services poses many challenges.

Service-Oriented ScienceService-Oriented Science

Source: Ian Foster, Source: Ian Foster,

Science, 6 May 2005Science, 6 May 2005

Web Services are self-contained, self-describing, modular applications that can be published, located, and invoked across the Web.

XML based Web Services are emerging as tools for creating next generation distributed systems that facilitate program-to-program interaction without the user-to-program interaction.

Besides recognizing the heterogeneity as a fundamental ingredient, these web services, independent of platform and environment, can be packaged and published on the internet as they can communicate with other systems using the common protocols.

Emerging web services standards are enabling much easier system-to-system integration.

Google Maps and Personal Google Maps and Personal Weather DataWeather Data

How is this magic performed?

Answer: DHTML, JavaScript, CSS, XML, and XSLT

Google EarthGoogle Earth

A Partial List of Data ServicesA Partial List of Data Services

CollectionCollection TransportTransport NotificationNotification Cataloging and metadata generationCataloging and metadata generation Metadata submissionMetadata submission SubsettingSubsetting AggregationAggregation Decoders/format converterDecoders/format converter QueryingQuerying VisualizationVisualization CollaborationCollaboration OntologyOntology Data miningData mining Weblogging or blogging Weblogging or blogging ……

End–To–End Data Service End–To–End Data Service Development at UnidataDevelopment at Unidata

IDD Data LDM

THREDDSCatalog

Browse

Access

THREDDSData

Server(TDS)

THREDDSData

Repository(TDR)

Put Data

TDS Client

NotifyE-mail

Application(e.g. IDV)

Service

DataStorage

TDR Client

LocallyGenerated

Data

LocallyGenerated

Data

Internet Data DistributionInternet Data Distribution

Source

LDM

Source

Source

LDM LDM

LDMLDM

LDM LDM

LDM

LDM

Internet

RadarModel

Satellite

Fil

e F

orm

ats

Lo

cal/

Rem

ote

Ser

vice

sU

nd

erly

ing

In

terf

aces

Pri

mar

yIn

terf

aces

THREDDScatalog

TDS

(THREDDS Data Server Interface)

OGC WCS (Web Coverage Server)

NetCDF interfaceOPeNDAP

OpenDAP ADDE

GRID

Station

AREA

IOsp

GRIB

GININIDS

netCDF via HTTP

Jgoffs

DMSP

NetCDF NetCDF

TDS: A Collection of ServicesTDS: A Collection of Services

TDR: Another ViewTDR: Another View

PeoplePeople

DocumentsDocuments DataData

Catalog

Generation Tools

Analysis andVisualization Tools

Data Services

Discovery andPublication Tools

Discovery and Publication Services

Dat

a C

atal

ogS

ervi

ces

PeoplePeople

DocumentsDocuments DataData

Catalog

Generation Tools

Analysis andVisualization Tools

Data Services

Discovery andPublication Tools

Discovery and Publication Services

Dat

a C

atal

ogS

ervi

ces

Data Services for Education

THREDDSMiddleware

For effective incorporation of data into digital libraries, we need a range of data services tailored for education:

integrate data, models, viz. tools with learning objects and other curricular materials

NCSAOU

Unidata

Unidata

OU

User running local analysis and display tools

Data Service

Decoder Service

Assimilation Service Prediction

Service

User Orchestrates Web Services to Create Regional

Forecast

Product Generation & Mining Service

LEAD: Data Services for NWP

IU

Orchestration Service

UAH

Data Services for Field ProjectsData Services for Field Projects

GALEON IE: Data services for GALEON IE: Data services for GIS Integration GIS Integration

Concluding RemarksConcluding Remarks At Unidata, we are in the process of building many of At Unidata, we are in the process of building many of

these data-related services and technologies;these data-related services and technologies;

Despite the significant progress, much work still lies Despite the significant progress, much work still lies

ahead;ahead;

Other organizations and communities are engaged in Other organizations and communities are engaged in

similar exercises (e.g., NOAA, IOOS, and CUAHSI) ;similar exercises (e.g., NOAA, IOOS, and CUAHSI) ;

We will partner with those groups where necessary We will partner with those groups where necessary

and leverage each other’s work;and leverage each other’s work;

We are diligently building bridges with other We are diligently building bridges with other

communities for mutual benefit.communities for mutual benefit.