The Devil is in the DataOpen Source Collaboration for Water Data Science

Dr Peter PrevosManager Data Science

Contents

● Data Science

● Code is Poetry

● Open source software

2WaterRA

Data Science

WaterRA 3

Source: Natalia Bilenko. Original by Drew Conway.

Good Data Science

Utility

Soundness

Aesthetics

4WaterRA

Utility

5

Knowledge

Information

Data

Reality

WaterRA

Soundness

ValidReliableReproducible

6WaterRA

Aesthetics

Data-to-Ink ratio

7WaterRA

Data Science Continuum

8WaterRA

Data Science Tools

9

scikit-learn.org

Hydroinformatics

10

● Flooded with data● Data deluge● Data lakes● Data flows● Sh*tloads of data

csgsolutions.com

Dark Data

11

Original: ecmconnection.com

SCADA

SCADAHISTORIAN

VISIBLE DATA

Lab data

Big Data versus Small Data

12

Data Science or Data Voodoo?

Data Science Mission

Create value through useful,

sound and aesthetic data products

14

Current water resource position.

Spreadsheets are Chaos

15

Spreadsheet Jungle

16

Code is Poetry

17

The Tower of Babel

18

Based on chdoig.github.io

Open Source

19

Free Software

● Use the software for any purpose

● Change the software to suit your needs

● Share the software with anyone

● Share the changes you make

20

auroralinux.net

Learn to Code

● R is ideal for engineers and scientists

● Tidyverse library

21

The Shadow IT Department

Gartner● By 2015, 35% of

enterprise IT expenditures will happen outside of the corporate IT budget

22

Water System Index

23

SCADA Alarm Annotation

24

Customer Observation Dashboard

25

My Vision

Collaborate to pool expertisePromote writing codeOpen Source code library

26