climate data lake: empowering citizen scientists in acadia national park
TRANSCRIPT
![Page 1: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/1.jpg)
1© Copyright 2013 Pivotal. All rights reserved. 1© Copyright 2013 Pivotal. All rights reserved.
Climate Data Lake: Empowering Citizen Scientists in Acadia National Park
Srivatsan RamanujamPrincipal Data Scientist, Pivotal
@being_bayesian
John CardenteDistinguished Engineer/Lead Data Scientist, EMC
@johncardente
19-Feb-2015
![Page 2: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/2.jpg)
2© Copyright 2013 Pivotal. All rights reserved. 2© Copyright 2013 Pivotal. All rights reserved.
Climate Change is happening
Source: NASA
![Page 3: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/3.jpg)
3© Copyright 2013 Pivotal. All rights reserved.Source: National Geographic
At the Ice Caps…
From Chasing Ice
by James Balog
![Page 4: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/4.jpg)
4© Copyright 2013 Pivotal. All rights reserved.
And in the
Oceans…
![Page 5: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/5.jpg)
5© Copyright 2013 Pivotal. All rights reserved. 5© Copyright 2013 Pivotal. All rights reserved.
Weather is Nature’s clock
![Page 6: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/6.jpg)
6© Copyright 2013 Pivotal. All rights reserved. 6© Copyright 2013 Pivotal. All rights reserved.
And the clock is shifting
Source: EPA
![Page 7: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/7.jpg)
7© Copyright 2013 Pivotal. All rights reserved. 7© Copyright 2013 Pivotal. All rights reserved.
English Oak Winter Moth Pied Flycatcher
Both et al. 2006 Nature
EARLIER SAME TIME
EACH YEAR
Disrupting species interactions
![Page 8: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/8.jpg)
8© Copyright 2013 Pivotal. All rights reserved. 8© Copyright 2013 Pivotal. All rights reserved.
…the study of the timing of recurring plant and animal life-
cycle stages, or phenophases, and their relationship to
environmental conditions.
Phenology
Source: USA National Phenology Network
![Page 9: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/9.jpg)
9© Copyright 2013 Pivotal. All rights reserved. 9© Copyright 2013 Pivotal. All rights reserved.
Acadia National Park
Source: Wikipedia
A key phenology
observation point
![Page 10: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/10.jpg)
10© Copyright 2013 Pivotal. All rights reserved. 10© Copyright 2013 Pivotal. All rights reserved.
Citizen Science, the “Hadoop” of Phenology
![Page 11: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/11.jpg)
11© Copyright 2013 Pivotal. All rights reserved.
Challenges impeding scientific progress
A multitude of disparate data sources
Different approaches to data conditioning
Different analysis platforms and scripts
Inconsistent data treatment distracts the
scientific debate from the findings!
![Page 12: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/12.jpg)
12© Copyright 2013 Pivotal. All rights reserved.
Challenges to encouraging citizen science
Different infrastructures for data collection
Unable to see how contributions help science
Inability to analyze the data and collaborate with scientists
Missing a feedback loop to encourage and
engage citizen scientists
![Page 13: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/13.jpg)
13© Copyright 2013 Pivotal. All rights reserved.
EMC Federation teams up with Earthwatch
![Page 14: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/14.jpg)
14© Copyright 2013 Pivotal. All rights reserved.
Objectives
• Authoritative, curated data repository
• Shared analysis platform and software libraries
• Web portals to
– Show citizen scientists how contributions benefit science
– Present data visualization and exploration tools
– Provide access to data sets for volunteer analysis
– Offer common data collection APIs and interfaces
– Host forums to interact with leading scientists
![Page 15: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/15.jpg)
15© Copyright 2013 Pivotal. All rights reserved.
Climate Data Lake
PL/X
Hadoop++
Data
Data Lake Foundation
Explore
CloudFoundry
Collect
Isilon ECS
![Page 16: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/16.jpg)
16© Copyright 2013 Pivotal. All rights reserved.
The kick-off expedition at Acadia
![Page 17: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/17.jpg)
17© Copyright 2013 Pivotal. All rights reserved.
Post kick-off activities
• Established partnerships with data providers
• Stood up initial Data Lake platform
• Active collaboration across all participants on data
conditioning, analysis, feature selection, and visualization
• Launching an initial web presence in March
![Page 18: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/18.jpg)
18© Copyright 2013 Pivotal. All rights reserved.
What’s next
• Pursuing the “full vision” using Federation Technologies
• Talking with other potential corporate sponsors
• Exploring ways to expand support to other climate and
academic projects
![Page 19: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/19.jpg)
19© Copyright 2013 Pivotal. All rights reserved. 19© Copyright 2013 Pivotal. All rights reserved.
![Page 20: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/20.jpg)
20© Copyright 2013 Pivotal. All rights reserved.
Learn More
• White House Press Release:
Harnessing Climate Data to Boost
Ecosystem & Water Resilience
• Pivotal Blog: Pivotal Data Science
Looks at Climate Change @ Acadia
NP
• EMC Reflections Blog: Big Data vs.
Climate Change
BlogsEMC TV video
![Page 21: Climate Data Lake: Empowering Citizen Scientists in Acadia National Park](https://reader030.vdocument.in/reader030/viewer/2022032421/55a66a1a1a28abe67e8b468d/html5/thumbnails/21.jpg)
21© Copyright 2013 Pivotal. All rights reserved.
The Team
@alyssacaddle1
@stevendozen
@johncardente
@lilewite
@being_bayesian
@maryanngarwood
@emctv
@TheMatthewNolan
@johnnysuire
@sriauggie31
@earthwatch_org
@schoodicinst
@hwschoodic
+ many more (not on twitter ) !