1

Data Science for USGS Minerals Big Data

Dr. Brand NiemannDirector and Senior Data Scientist/Data Journalist

Semantic CommunitySemantic Community

Data ScienceData Science for USGS Minerals Big Data

June 15, 2015

2

Evolution at the USGS• Years ago when I served a detail to the USGS, they were organized into

the following divisions:– Geology– Water– Biology– Mapping

• Now they are organized into the following:– Ecosystems– Climate and Land-Use Change– Natural Hazards– Water– Energy and Minerals, and Environmental Health– Core Science Systems

• with a focus on Start with Science.

3

USGS Mineral Resources Program (MRP)

http://minerals.usgs.gov/minerals/pubs/mcs/

The USGS Mineral Resources Program (MRP) provides scientific information for objective resource assessments and unbiased research results on mineral potential, production, consumption, and environmental effects. The MRP is the sole Federal source for this information.

4

USGS Commodity Statistics and Information

http://minerals.usgs.gov/minerals/pubs/commodity/

Statistics and information on the worldwide supply of, demand for, and flow of minerals and materials essential to the U.S. economy, the national security, and protection of the environment.

5

Adobe Acrobat Convert From PDF Tool

https://cloud.acrobat.com/exportpdf

6

Adobe Acrobat Convert From PDF to Excel

http://semanticommunity.info/%40api/deki/files/33042/mcs-2015-abras.xlsx?origin=mt-web

My Note: This did not work!

7

Mineral Resources On-Line Spatial Data: National Geochemical Survey Database

http://mrdata.usgs.gov/geochem/

8

Mineral Resources On-Line Spatial Data: Mineral Resources Data System (MRDS)

http://mrdata.usgs.gov/mrds/

This is “Big Data”

9

Data Science for USGS Minerals Big Data: MindTouch Knowledge Base Index

Data Science for USGS Minerals Big Data

My Note: See well-defined URLsin Excel Spreadsheet.

10

Data Science for USGS Minerals Big Data: MindTouch Knowledge Base Attachments

Data Science for USGS Minerals Big Data

11

Data Science for USGS Minerals Big Data: MindTouch Knowledge Base Find

Google Chrome Find: AbrasiveMy Note: See highlighted hits and real tables!See tables in Excel in next slide!

Data Science for USGS Minerals Big Data

12

Data Science for USGS Minerals Big Data: Excel Knowledge Base Find

USGSMinerals.xlsx

13

Data Science for USGS Minerals Big Data: Spotfire Cover Page

Web Player

14

Data Science for USGS Minerals Big Data: Minerals Commodity Summaries 2015 Tables in Spotfire

Web Player

15

Data Science for USGS Minerals Big Data: MRDS-shp in Spotfire

Web Player

16

Some Conclusions and Recommendations

• The USGS has evolved to Starting with Science so I started with Data Science for USGS Minerals Big Data.

• I data mined the USGS Mineral Resources Program (MRP), the USGS Commodity Statistics and Information, and the Mineral Resources On-Line Spatial Data.

• I converted PDF to MindTouch and Excel Knowledge Bases and used Spotfire for analytics and visualizations.

• The result is a Data Science Data Publication. The USGS has many more PDF publications and data sets that could be Data Science Data Publications.