state geothermal data and the us geoscience information network

State Geothermal Data and the US Geoscience Information Network

State Geothermal Data and the US Geoscience Information Network System Design and Progress ReportStephen M Richard, AZGSCSIG 2011, SDSC

Funding from the National Science Foundation under award EAR-0753154 to the AZGS, acting on behalf of the Association of American State Geologists and by the U.S. Department of Energy under award DE-EE1002850 to the AZGs acting on behalf of the Association of American State Geologists

rOADMAPBackgroundApproachMetadataServicesInterchange formatsCurrent statusArchitecture and artifactsWay forward7/12/2011BackgroundUSGIN, Interoperability and the National Geothermal Data System7/12/2011February, 2007: Representatives of Association of American State Geologists (AASG) and U. S. Geological Survey (USGS) meet Explore making their data accessible and interoperable. Recommend that the geological surveys work together to create a distributed, national Geological Information Network (GIN) of digital data use of common standards and protocols respect ownership or control of data builds on existing data systems (AZGS Open-file Report 2008-01, 2008). US Geoscience Information NetworkPartnership between the Association of American State Geologists (AASG) and the US Geological Survey (USGS)Objective is to make geoscience information easier to find, distribute, and analyzeCustomers:Government agencies (regulatory, land management)Commercial users (mineral exploration, engineering, environmental)ResearcherEducators

What are the resourcesGeologic mapsGray literature (files, theses, unpublished reports)Core and cuttings repositoriesOil and Gas well records and logsWater well records and logsData compilations geochemical, geophysical, mineral resource, etc. etc

Arizona Geological Survey

Information is increasingly brought to us from disparate sources

Constantly improving search capabilities allow us to find all this informationBenefits of a distributed network:Keeps information in the hands of the data providersAllows for simpler update routinesAllows new information to be more rapidly conveyed to usersDistributed Network

Loose coupling between data provider and consumerKMLGeoSciMLWaterMLShift the Burden from the Userto the Provider

Peter Fox, RPI, 2010Effort to createEffort to maintainEffort to get dataWeb ServicesDynamic pages on the fly from databaseStatic HTML pagesMore complex data management8888In the heroic era of science, the provider of data plays a relatively passive role. The onus is on the user to identify each data source, accumulate the required data, and prepare the data for assimilation and analysis. A similar process applies for acquiring software for analysis, modeling, and visualisation. In many cases, the user and the provider are the same person. In an interoperable e-Science world, the provider has to put much more work into describing the structure and content of data and information, and someone has to provide and support Web Services. The user is relieved of these burdens and benefits accordingly. The overall reduction in work load is enormous, but the provider does not see that.Shift the Burden from the User to the Provider Server-side data integration standard interchange data schemaAccess to data from within user applications9999In the heroic era of science, the provider of data plays a relatively passive role. The onus is on the user to identify each data source, accumulate the required data, and prepare the data for assimilation and analysis. A similar process applies for acquiring software for analysis, modeling, and visualisation. In many cases, the user and the provider are the same person. In an interoperable e-Science world, the provider has to put much more work into describing the structure and content of data and information, and someone has to provide and support Web Services. The user is relieved of these burdens and benefits accordingly. The overall reduction in work load is enormous, but the provider does not see that.

USGIN System overviewThe National Geothermal Data SystemArizona Geological Survey

DOE & USGS Data

Boise State University

National Assessment

USGS University Data

Southern MethodistUniversityState Geological Survey Data

AASG - AZGS

DOE Geothermal Technologies Program-funded projects7/12/2011Arizona Geological Survey

7/12/2011Arizona Geological Survey


The Approach7/12/2011Arizona Geological SurveyInteroperabilityWe want mash-ups that workAnalyze: Provide data services in dataset-specific standardized schemaAccess: Provide resources themselves online using standard OGC protocolsFind: Provide standardized metadata for resources that may or may not be available online

in Three Easy StepsMetadata ImplementationUSGIN Profile and Custom Software7/12/2011Arizona Geological Survey

Development historyReview existing standards (FGDC, Dublin Core, ISO19115)Define user scenariosMetadata content requirements based on thoseExplore service-based metadata search tools (Deegree, GeoNetwork)Usage guidelines document for ISO19115/19139 on CSW 2.0.2Geonetwork prototypeSwitch to ESRI Geoportal after testing when it was openedPopulate catalog through various approaches7/12/2011Arizona Geological SurveyWhats it for?Find, evaluate, get resourcesBounding box, keywords, author, title, dateResource identifiable item of interestData vs metadata: Catalog is high levelIn catalogDocuments, physical samples, datasets, services, software, files, databasesNot in catalogIndividual database records--polygons in a GIS dataset, chemical analyses, temperature measurementsDomain specific description informationMetadata ContentTitleDescriptionExtentGeographicVerticalTemporalKeywordsOriginator(s)DateResource IDAccess instructionsAccess constraintsLanguageQualityLineageCitationDistribution contactMetadataDateContactSpecificationIdentifierOnline document: http://lab.usgin.org/profiles/doc/metadata-content-recommendations

ESRI ArcGIS CSW Client

22Display WMS in ArcMap

23USGIN Metadata Implementation7/12/2011Arizona Geological SurveyMetadata Creation and Inclusion in a CatalogFour Options:Write XML, upload to CatalogExcel Sheet + PythonFor bulk updates, ETLMetadata WizardFor offline resources or resources already onlineDocument RepositoryFor resources that need to be made available online

Document RepositoryUSGIN Catalog Implementation7/12/2011Arizona Geological SurveyWhy a Geoportal Catalog?Allow a variety of metadata creation methods to be aggregatedAllow distributed metadata records to be aggregated: harvesting between catalogsProvide a consistent interface for searching and retrieving metadata records: CSWWhy ESRIs Geoportal Server?Geonetwork was too hardThey made it open source!Geoportal CatalogMetadata WizardExcel TemplateXML in Web-Accessible FolderAnother CatalogAnother Catalog

Drupal-based metadata toolsHome pages look different, but metadata creation is the sameUser defines collectionsMetadata created in context of a collectionPublished records get pushed to WAF for harvestDrupal provides features for autocomplete, simple validationComplex web of dependencies between Drupal modules hard to maintain

http://mw.usgin.org AND http://repository.usgin.org Content groupsRequired content *AutocompletePrepopulate from collection

View in different encodingsMetadata from User PerspectiveSearch from within the users standard workspace (Excel, ArcGIS, ModFlow.)Natural language labels and instructionsStandardize interface components for creating, searching, browsing metadataWork against a content model not bound to a particular encoding or standardNagging questions7/12/2011Arizona Geological SurveyHow to leverage commercial search enginesWhat is sweet spot between structured metadata and text/link-based indexingImplementing ServicesOGC Services7/12/2011Arizona Geological Survey

The ServicesProtocolMessaging framework and interfaceWFS, WMS, CSWGeospatialServer implementationGeoServerESRI ArcGIS serverThe US Geoscience Information NetworkArizona Geological SurveyOGC ServicesWMS for providing a symbolized portrayal of vector dataWFS for full access to attributes and to download vector dataWCS for access to continuous raster data

-- Useful in Existing Applications Protocol: WMSSimple, widely used, many clientsAccess to georeferenced map imageServersMapServer, GeoServer, Deegree, ArcGIS

Data integration = agreeing on standard portrayal schemes (legends)WMS for well headers

Protocol - WFSOpen Geospatial Consortium Web Feature ServiceGetCapabilities, GetFeature, DescribeFeatureGML encoding of contentEach service defines feature content model (xml schema)Feature service designed for data access based on features (representation of geolocated entity with attributes)WFS example

Protocol: ArcGIS servicesIf using ArcGIS software clients, service is transparentOther client libraries are available (Flex, OpenLayers)ESRI has submitted Geoservices REST API for consideration as OGC standardAASG State Geothermal data at ArcGIS.com

Standardizing ContentThis is the hardest part.Arizona Geological Survey


Consistent Attributes Across State BoundariesFeature Service inventory: trade-offs7/12/2011Arizona Geological SurveyMany services, simple contentMore traffic on networkmore callsClient complexity: marshal related information from a variety of servicesFew services, complex contentLess network traffic, fewer callsClient complexity: unravel complex response documents

Interchange formats: complex FeaturesArizona Geological SurveyCOSTDifficult to validate and QAClient complexity mostly have to build your ownSteep learning curveBENEFITSGreat expressive capabilityaccurate representation of science informationComplex schemaOGC sensor web schema for observationsGeoSciML for Geologic informationISO19139 for metadataSimple Features7/12/2011Arizona Geological SurveySimple point, line, or polygon geometryAttributes are text, number, or dateEssentially a shape file encoded in GML (with fewer limitations)Service is easy to set upArcGIS client out of the boxEnables XML schema and rule-based (schematron) validationNamespaces indicate schema version to clientsData IntegrationArizona Geological Survey

Transforming data to another schema stinks. I only want to do it onceContent Models: interchangeArizona Geological SurveyActive FaultAlteration descriptionBorehole temperature dataCrustal Stress dataDeveloped geothermal system featureDirect use featureDrill stem testEarthquake hypocenterEnhanced geothermal system featureAqueous chemistry

Geologic map: contact, fault, outcropGeologic Unit geothermal characterizationGravity station observationHeat flow measurementHot spring descriptionMetadataPermeabilityProduction statistics recordRock chemistryThermal conductivity measurementWell header Volcanic vent description

7/12/2011U S Geoscience Information Network

Architecture conceptsArizona Geological SurveyLoose couplingFeature-level data integration by data providerUse http URI to identify resourcesServices registered in catalog systemNetwork is defined by interfaces and interchange formatsInterchange formats and profiles are reviewed, registered, versioned, and documented

ArtifactsTechnologicRepositorySpecificationsVocabulariesDocumentation of practiceResource registryHardware hostsTesting and validation resourcesUsage monitoring and reporting

ArtifactsSocialProcesses for developing, review, adopting artifactsMailing list/membershipOrganization procedures, transfer of leadershipStaff positionsBrandData source linkage to networkEnd user facing linkageConnections between community and software developers (vendor community)Feedback mechanisms/evaluation/reviewWith ultimate providers of information

ArtifactsEducationalTutorialsTraining materials, videos, etc.Demonstration softwareWiki, collaborative online toolsBlogs

The US Geoscience Information NetworkArizona Geological SurveyMoreDataMoreUsersMoreSoftwareBuilding the System: Social EngineeringStarting a bootstrapping processUse of existing protocols means lots of software is already availableStriving to provide simple transformation tools to ease the barrier to entryThe more the merrier!Making standards happen?7/12/2011Arizona Geological SurveySome party develops a specification for how to do somethingOther people see it working and want to do the same thingRequires documentation, outreach, trainingRequires commitment to reusing/extending existing work rather than reinventing.Darwinian processStandard implies stewardship: versioning, stability, orderly progress

A standard is a product that ships in volumeWhere to nowArizona Geological SurveyEvolving organization landscapeUSGIN CDIDataONEEarth Science Information Partners (ESIP)Evolving technology landscapeResource oriented approaches (REST)OpenSearchLighter weight encoding (JSON)Organic ProcessEmerging SystemMoving community forward7/12/2011Arizona Geological SurveyWhat kind of communityCommunity engagement takes time and effortHas to yield benefitsMore efficient workProfessional recognitionSense of identityBusiness value to individuals and organizationsAre we spread too thin?

USGIN SummaryLots of work on discoveryISO metadataCarefully considered profileESRI geoportalUse services that have out of the box clients and serversKeep content schema simple Use existing information as guideCurrent GIN DocumentsMetadata recommendations content model based on user scenariosProfile for ISO metadata detailed profile for interoperable metadata using ISO schema for geoscience resourcesURI scheme

http://lab.usgin.orgAASG Geothermal dataCatalog file-based resources in repositoriesSimple feature (flat file) schemaBasic borehole description (well header)Borehole temperature measurementsHot springsActive faultMore on the way geologic units, geochemical data, samples, heat flowhttp://stategeothermaldata.orgThe goal is simple, really.

turn this.Winona State U.

.into this NERC 2008606060Thank YouStephen RichardArizona Geological [email protected] Geological Surveyhttp://lab.usgin.org http://stategeothermaldata.org