building a national archive of standardized geologic …//ngmdb.usgs.gov building a national archive...

http://ngmdb.usgs.gov

Building a National

Archive of

standardized

geologic maps and

related geoscience

information

Dave Soller National Geologic Map Database


We stand on the shoulders of giants

“…the maps are designed not so much for the specialist as for the people, who justly look to the official geologist for a classification, nomenclature, and system of convention so simple and expressive as to render his work immediately [understandable]…”

– USGS Director J.W. Powell,

3RD IGC (Berlin, 1885)


Focus on content. That’s why users come to the Website.

Keep the technology simple to use. This isn’t a research project. Don’t build a site that’s beyond the interest, skill, and technologies of our users. Focus on clarity of presentation and navigation, and speed of delivery.

Customer Service. Listen to the user. Interact with them. Ensure they get the answers and data they need. As a result, we learn how to improve the system.

Plan for the long term. Focus on preserving information. Work with the Library community, to ensure NGMDB content is compatible and addresses generally-accepted, long-term objectives.

Our development strategy


85,000+ users make 270,000+ visits / month (~ 9,000 visits / day)


Focus on content. That’s why users come to the Website.

Keep the technology simple to use. This isn’t a research project. Don’t build a site that’s beyond the interest, skill, and technologies of our users. Focus on clarity of presentation and navigation, and speed of delivery.

Customer Service. Listen to the user. Interact with them. Ensure they get the answers and data they need. As a result, we learn how to improve the system.

Plan for the long term. Focus on preserving information. Work with the Library community, to ensure NGMDB content is compatible and addresses generally-accepted, long-term objectives.

Our development strategy


Maps

Reports

Unpublished notes, data

GIS files

Metadata

Images


Text here


1996 – NGMDB Website opened 1996-2010 – Content, content, content… 2010-2012 – Website redesign 2012- 2017 – Infrastructure overhaul 2017-2018 – Return to Website redesign, add functionality (and content ! ) 1997- present – Standards development

Timeline


NGMDB hardware USGS “Cloud” Red Hat CentOS Oracle PostgreSQL Older codebase Modern codebase Single Public/Dev environment Multiple dev servers with git repository …evolving IT security requirements…

Infrastructure overhaul


Public servers (USGS “Cloud”, in Denver) – 7 VMs with 40 TB (includes .tif files of State GS images shown in Catalog and mapView) Development servers (NGMDB hardware, in Reston) – includes 64 TB storage, for file backup (including topoView) Tertiary backup, and image-processing computer Offsite tape backup

Infrastructure


Adding images to the NGMDB Catalog and mapView


101,600+ publications

650+ publishers

66,000+ are “digital”

25,000 are geologic maps

37,000 are resource-related

11,000 are hazards-related

We provide bibliographies and links to where you can buy or download these publications.

Geoscience Map Catalog


mapView


“It’s online.”

Contract scan of USGS Monograph, for Internet Archive. Resolution is adequate only for text pages


Another group found a paper copy, and rescanned the plate. But it’s not linked with the Internet Archive site.


Foldouts weren’t included in the Google scanning contracts…


USGS PP-527 – No QC on this contract-scanned map. The error was found years later, by a user,

so the NGMDB

rescanned it.…


USGS B-1157 – The maps weren’t scanned…


For each ”part” of a publication, who has the best, or the authoritative, scan?


Each agency/publisher needs a scan-status list that includes Who, What, When for every “part” or object in a publication.

•What is the object? (e.g., text, plate, figure, foldout)

•Who scanned it? When? What specs? Where does the best version of each “part” reside?


Standards and guidelines…


http://ngmdb.usgs.gov/Info/standards

http://ngmdb.usgs.gov/Info/standards


Available in the following formats:

Paper

PDF

Postscript (EPS)

ESRI ArcGIS 10

Free copies available…

http://ngmdb.usgs.gov/fgdc_gds/

FGDC Geologic Map Symbolization Standard

http://ngmdb.usgs.gov/fgdc_gds/


How many geologic maps are there? (24-250k)

~24,000 listed in the NGMDB Map Catalog (plus an additional ~9000 tagged as having geologic map content)

550 USGS maps in GIS format

343 USGS GIS files are >10 yrs old

47 USGS are >20 yrs old


Born digital GIS files of geologic

maps (24-250k only):

In past 10 years, >2000 GIS maps by States and USGS

How many data formats were used?


Can the GIS file be opened, and migrated? If not, does a scale-stable plot exist? If there’s a plot, then re-digitize the map.

Many examples of this, resulting in multiple versions – which is authoritative?

If no plot, then permanent loss of a taxpayer-funded map publication


Surficial geology of Indiana (USGS, 1993)

Available only on floppy disk


Geology of Puerto Rico (USGS, 1998)

No useable geologic map graphic


Geology of Puerto Rico (USGS, 1998)

FMATN=MapUnitCode


Migration of mid-1990’s GIS files:

Technical issues, cost of migration…


A database that contains all aspects of a traditional geologic map

Geologic Map

Description of Map Units

Correlation of Map Units

Cross Sections

Discussion section

References

Geologic Terms, Symbols, Colors

http://ngmdb.usgs.gov/Info/standards/NCGMP09/

GeMS (NCGMP09 v.2) standard map schema

http://ngmdb.usgs.gov/Info/standards/NCGMP09/


Professional

users

Research

General

interest

NCGMP09

ESRI GMT

GeoSciML-Lite

NGMDB-FULL

(NADM)

NGMDB-LITE

“5 friendly fields”

Content

Publication

Corporate database

(archive)

Web portal

Variations on a single schema


Summary of AASG-USGS collaboration on geologic map database standards

• Geologic Data Model Working Group (formed August, 1996, at the meeting in St. Louis, Missouri). Published a database design (“v.4.3”) in 1999.

• North American Data Model Steering Committee (formed in 1998). Published the "NADM" multi-map, enterprise-level design in 2004.

http://ngmdb.usgs.gov/www-nadm/prd/Model43a.pdf

http://pubs.usgs.gov/of/2004/1334/




http://pubs.usgs.gov/of/2004/1334/


Summary of AASG-USGS collaboration on geologic map database standards, cont.

• In 2008, NCGMP requested development of a single-map database design. That design (NCGMP09) was presented at DMT'09, and published later that year.

• in 2013, based on discussions at DMT'13, a Progress Report on NCGMP09 implementation was released.

• in 2014-2016, revisions to NCGMP09 were systematically debated among a large group of AASG and USGS staff. Version 2 (renamed “GeMS”) will be released in 2017.

http://pubs.usgs.gov/of/2010/1335/pdf/usgs_of2010-1335_NCGMP09.pdf

http://ngmdb.usgs.gov/Info/standards/NCGMP09/docs/NCGMP09_ProgressReport_Sept2013.pdf




http://ngmdb.usgs.gov/Info/standards/NCGMP09/docs/NCGMP09_ProgressReport_Sept2013.pdf


Potential Obstacles to widespread adoption of NCGMP09, and some recommendations (from 2013 Progress Report):

1. Complexity of design

2. Legacy versus new mapping

3. Must we use NCGMP09 for a new map, from start to finish?

4. Software requirements

5. Standard terms and definitions

6. Software tools

7. Communication and collaboration

8. Agency commitment.


Participants in review process (late 2014 - 2016)

• Steering Committee (set the general course)

• State and USGS telecons (discussed each topic, made recommendations)

• Working Group (decided on revisions to NCGMP09 design and documentation)

• NCGMP09 Listserve (kept informed of revisions, to provide additional comment)


What aspects of design and documentation needed to be revised? We used the empirical approach – these issues were most commonly noted:

• Which fields should remain as mandatory, which should be optional, or deleted?

• Null values – how should they be encoded?

• GeneralLithology

• LocationConfidenceMeters

• Can NCGMP09 serve as an "Enterprise" or multi-map database?

• Stack-unit mapping

• Workflow


• Based on that review process, the schema and documentation were modified.

• NCGMP09 v. 2 (renamed as Geologic Map Schema, or GeMS) then was submitted to the Working Group for comment, in early 2017.


• Working Group and Peer Review comments have been addressed.

• Submitted in Sept., 2017, for USGS publication.

• GeMS will be posted to the GeMS (formerly NCGMP09) website, accompanied by various resources (e.g., tools, workflows).


• After publication, this schema will be formally proposed as the FGDC Standard for geologic map databases.

building a national archive of standardized geologic …//ngmdb.usgs.gov building a national archive...

Documents