u.s. department of the interior u.s. geological survey next generation data integration challenges...
TRANSCRIPT
![Page 1: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/1.jpg)
U.S. Department of the InteriorU.S. Geological Survey
Next Generation Data Integration Challenges
National Workshop on Large Landscape Conservation
Sean Finn, USFWS; Jim Strittholt, CBI; Tim Kern, USGSOctober 2014
![Page 2: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/2.jpg)
Data Integration Now and Tomorrow
Not just a technical concept
Interaction of People Policy Technology
Critical at every scale – local, regional, and national
![Page 3: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/3.jpg)
Data Integration Challenges
![Page 4: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/4.jpg)
Human Factors Challenges
Data collected for different missions may have custom and mission-specific vocabularies
Expensive to deal with varied data formats and types (extraction, transform, and load)
Inconsistent spatial representations can confuse visualization and analysis efforts
Complex schemas can make it hard to pull out project-appropriate data
Inadequate metadata and unclear provenance
![Page 5: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/5.jpg)
Human Factors Approaches
Provide support for efforts at all scales Establish best practices and standards for
cooperating data catalogs and repositories Establish connections across organizational
bounds Develop “share spaces” for tools and analysis
methods Provide examples of different integration
techniques and approaches (“successes”)
![Page 6: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/6.jpg)
Success Stories
![Page 7: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/7.jpg)
Policy and Economic Challenges
Metadata policy contradicts data.gov policy Data release policies inconsistent with
Project Open Data Skills mismatches Infrastructure needs , funding Resource plans rarely consider long-term
expenses due to new security policies User expectations
![Page 8: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/8.jpg)
Policy and Economic Approaches
Infrastructure Commercial and government clouds Open compute cluster access
Staff MOOCs Communities of interest
Resources Open source projects Free or lower cost collaboration sites
![Page 9: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/9.jpg)
Policy and Economic Approaches
User Expectations Agile project development involve scientists and not tech staff Infrastructure availability More ready-to-use toolkits available
Policy Current retirement rate Increased reliance on external sources
![Page 10: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/10.jpg)
Technical Challenges
Big Data Capture Analysis Presentation Access
Discovery Data lifecycle and long-term protection New sources and types of data
![Page 11: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/11.jpg)
Technical Approaches
Parallelization and federation
Bring tools to data, not data to tools
Design for cloud
New software frameworks
![Page 12: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/12.jpg)
Technical Approaches
Repo 1
Repo n
![Page 13: U.S. Department of the Interior U.S. Geological Survey Next Generation Data Integration Challenges National Workshop on Large Landscape Conservation Sean](https://reader035.vdocument.in/reader035/viewer/2022070409/56649e7a5503460f94b7a0d8/html5/thumbnails/13.jpg)
Integrated Data Management Network
Delivered a Roadmap to Interoperability, user support, and training
Provide project tracking and sharing options Established open-source projects Cataloged tools Demonstrated and shared data visualization,
processing, and access capabilities Built a community that can help address
these future challenges