Kenneth Casey, PhD

NOAA’s National Centers for Environmental Information (NCEI)

June 19, 2019

Progress Toward Data Discovery, Machine Readability, and

Interoperability at NOAA: NOAA OneStop and the NESDIS Cloud

2National Centers for Environmental Information

National Centers for Environmental Information

(NODC+NGDC+NCDC)

3National Centers for Environmental Information

The NAO 212-15, 2010

● Environmental data will be visible, accessible and independently understandable to users, except where limited by law, regulation, policy

● Empowered the Environmental Data Management Committee (EDMC) to create Procedural Directives to ensure NOAA meets those responsibilities:○ Archive Appraisals, Data Access,

Data Citation, Data Documentation, Data Sharing, Data Management Planning

4National Centers for Environmental Information

The PARR, 2013-2015

● NOAA PARR plan (2015), in response to the White House OSTP Memorandum, Increasing Access to the Results of Federally Funded Scientific Research (2013), says NOAA Programs shall, within their own budgets...○ Allocate resources for data access

and archiving ○ Develop comprehensive data

management plans○ Document their data with metadata○ Make their data publicly accessible

PARR = Public Access to Research Results

5National Centers for Environmental Information

FEBPA and the OPEN Government Data Act, 2018

● Foundations for Evidence-Based Policymaking Act (FEBPA) of 2018

● Title II is the OPEN Government Data Act, which says that government data must be: ○ Open by default○ Cataloged○ Accessible○ Machine-readable

6National Centers for Environmental Information

The Data Steward’s Dream!

Metadata!

Data Management

Plans!

Machine Readability!

Image credit: Pixabay

7National Centers for Environmental Information

A Flurry of Activity

Big Data Project

NCEI netCDF templates

Data Management Plan Compliance

NOAA Environmental Data Management

Workshops

ISO 19115-2 Metadata

Data DOIs

… but will focus next on NOAA OneStop, and the NESDIS Cloud Project

8National Centers for Environmental Information

NOAA OneStop

To improve the discovery of, access to,

and usability of NOAA’s vast and diverse

collection of big data

Modern, user-friendly

discovery interface

and API

Open infrastructure,

standards, and best

practices

9National Centers for Environmental Information

NOAA OneStop

10National Centers for Environmental Information

NOAA OneStop: Two-tiered Discovery

https://data.noaa.gov/onestop/

Find the GHRSST Collections...

Examine the details for one...

List and interact with the granules...

11National Centers for Environmental Information

NOAA OneStop - Professional Usability Testing

● What OneStop is Doing Well○ Clean, modern and spacious design○ Mobile-friendly, responsive○ User-friendly UI for pre-search filtering○ Post-search results landing page○ Utilization of filters to drill down results○ Judicious use of icons and images○ Useful research tools○ Utilization of navigation○ Site speed

● Areas of Opportunity○ Personalized content / related searches○ Improving search engine optimization strategy

■ Cleaner headers and titles, schema.org, revise URL structures, deeply optimize top level landing pages, etc.

83% of surveyed users prefer OneStop over the previous NOAA Catalog!

12National Centers for Environmental Information

NESDIS Cloud Project

Experimentation Migration Transformation

Initiation &

Assessment

Proof of Concept

Implementation

Migrate to Cloud

Leverage Cloud

Optimize for Cloud

Initiate & Assess

Prototype Implement

• Develop an Implementation and Phasing Plan• Trades• Cloud orchestration

• Consolidate activities• Migrate services to cloud

• Cloud provisioning• Cloud management and monitoring

• Service, architecture, cost optimization

• Evolution of the service-based architecture

• NESDIS Cloud Strategy

• IPTs• Readiness Assessment

• Build and Evaluate Prototype

• ConOps

Where we are Today

• Cloud computing has already been demonstrated as a viable implementation for data system computing, storage, and dissemination

• Adoption of commercial cloud computing fundamentally changes many aspects of our organization… not just the technology

• NESDIS will leverage commercial cloud computing going forward

13National Centers for Environmental Information

Imagine a World Where NESDIS Data Systems are Completely in the Cloud

NESDIS is focused on the architectural level, not current system implementation:• Approach the cloud from

architectural functionality• Assume those functions are

done completely in the cloud• Address technical barriers to

cloud migration in the Cloud Pilot Project

• Use results to guide NESDIS cloud implementation decisions

14National Centers for Environmental Information

NESDIS Cloud ConOps

15National Centers for Environmental Information

Key Takeaways Operational Oceanography

● NOAA’s data are becoming increasingly discoverable,

machine-readable, and accessible via human portals

and machine APIs

● In the next few years, NOAA’s data will increasingly

become available via commercial cloud service

providers

Thank-you!!! Any Questions?