progress toward data discovery, machine readability, and ...€¦ · experimentation migration...

of 16/16
Kenneth Casey, PhD NOAA’s National Centers for Environmental Information (NCEI) June 19, 2019 Progress Toward Data Discovery, Machine Readability, and Interoperability at NOAA: NOAA OneStop and the NESDIS Cloud

Post on 22-Sep-2020

2 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Kenneth Casey, PhD

    NOAA’s National Centers for Environmental Information (NCEI)

    June 19, 2019

    Progress Toward Data Discovery, Machine Readability, and

    Interoperability at NOAA: NOAA OneStop and the NESDIS Cloud

  • 2National Centers for Environmental Information

    National Centers for Environmental Information

    (NODC+NGDC+NCDC)

  • 3National Centers for Environmental Information

    The NAO 212-15, 2010

    ● Environmental data will be visible, accessible and independently understandable to users, except where limited by law, regulation, policy

    ● Empowered the Environmental Data Management Committee (EDMC) to create Procedural Directives to ensure NOAA meets those responsibilities:○ Archive Appraisals, Data Access,

    Data Citation, Data Documentation, Data Sharing, Data Management Planning

  • 4National Centers for Environmental Information

    The PARR, 2013-2015

    ● NOAA PARR plan (2015), in response to the White House OSTP Memorandum, Increasing Access to the Results of Federally Funded Scientific Research (2013), says NOAA Programs shall, within their own budgets...○ Allocate resources for data access

    and archiving ○ Develop comprehensive data

    management plans○ Document their data with metadata○ Make their data publicly accessible

    PARR = Public Access to Research Results

  • 5National Centers for Environmental Information

    FEBPA and the OPEN Government Data Act, 2018

    ● Foundations for Evidence-Based Policymaking Act (FEBPA) of 2018

    ● Title II is the OPEN Government Data Act, which says that government data must be: ○ Open by default○ Cataloged○ Accessible○ Machine-readable

  • 6National Centers for Environmental Information

    The Data Steward’s Dream!

    Metadata!

    Data Management

    Plans!

    Machine Readability!

    Image credit: Pixabay

    https://pixabay.com/photos/baby-sleeping-sleeping-baby-dreaming-2073290/

  • 7National Centers for Environmental Information

    A Flurry of Activity

    Big Data Project

    NCEI netCDF templates

    Data Management Plan Compliance

    NOAA Environmental Data Management

    Workshops

    ISO 19115-2 Metadata

    Data DOIs

    … but will focus next on NOAA OneStop, and the NESDIS Cloud Project

  • 8National Centers for Environmental Information

    NOAA OneStop

    To improve the discovery of, access to,

    and usability of NOAA’s vast and diverse

    collection of big data

    Modern, user-friendly

    discovery interface

    and API

    Open infrastructure,

    standards, and best

    practices

  • 9National Centers for Environmental Information

    NOAA OneStop

  • 10National Centers for Environmental Information

    NOAA OneStop: Two-tiered Discovery

    https://data.noaa.gov/onestop/

    Find the GHRSST Collections...

    Examine the details for one...

    List and interact with the granules...

    https://data.noaa.gov/onestop/

  • 11National Centers for Environmental Information

    NOAA OneStop - Professional Usability Testing

    ● What OneStop is Doing Well○ Clean, modern and spacious design○ Mobile-friendly, responsive○ User-friendly UI for pre-search filtering○ Post-search results landing page○ Utilization of filters to drill down results○ Judicious use of icons and images○ Useful research tools○ Utilization of navigation○ Site speed

    ● Areas of Opportunity○ Personalized content / related searches○ Improving search engine optimization strategy

    ■ Cleaner headers and titles, schema.org, revise URL structures, deeply optimize top level landing pages, etc.

    83% of surveyed users prefer OneStop over the previous NOAA Catalog!

  • 12National Centers for Environmental Information

    NESDIS Cloud Project

    Experimentation Migration Transformation

    Initiation & Assessmen

    t

    Proof of Concept

    Implementation

    Migrate to Cloud

    Leverage Cloud

    Optimize for Cloud

    Initiate & Assess

    Prototype Implement

    • Develop an Implementation and Phasing Plan• Trades• Cloud orchestration

    • Consolidate activities• Migrate services to cloud

    • Cloud provisioning• Cloud management and monitoring

    • Service, architecture, cost optimization

    • Evolution of the service-based architecture

    • NESDIS Cloud Strategy

    • IPTs• Readiness Assessment

    • Build and Evaluate Prototype

    • ConOps

    Where we are Today

    • Cloud computing has already been demonstrated as a viable implementation for data system computing, storage, and dissemination

    • Adoption of commercial cloud computing fundamentally changes many aspects of our organization… not just the technology

    • NESDIS will leverage commercial cloud computing going forward

  • 13National Centers for Environmental Information

    Imagine a World Where NESDIS Data Systems are Completely in the Cloud

    NESDIS is focused on the architectural level, not current system implementation:• Approach the cloud from

    architectural functionality• Assume those functions are

    done completely in the cloud• Address technical barriers to

    cloud migration in the Cloud Pilot Project

    • Use results to guide NESDIS cloud implementation decisions

  • 14National Centers for Environmental Information

    NESDIS Cloud ConOps

  • 15National Centers for Environmental Information

    Key Takeaways Operational Oceanography

    ● NOAA’s data are becoming increasingly discoverable,

    machine-readable, and accessible via human portals

    and machine APIs

    ● In the next few years, NOAA’s data will increasingly

    become available via commercial cloud service

    providers

  • Thank-you!!! Any Questions?