“Learn from details, act global”

Statistics Netherlands and Big Data

Marco Puts

Big Data Team

2

Martijn Tennekes

Piet Daas

Joep Burger

Marco Puts

From Survey to Big Data

Primairy data Secondary data

Our ‘own’ surveys Data from ‘others’ - Administrative sources

- Big Data

Statistics Netherlands

4

Big Data: Another Perspective

The internet

had 1800

exabytes of

data in 2011

exa = 10^18

NOW IDC/EMC white paper 2008

CES 2011 Geneva

5

Data Deluge: 50.000 EXAbytes in 2020

We live in exponential times

27 fold growth in the next 9 years

Stategic Aspects of Big Data

Statistics at Statistics Netherlands

• Known and good quality

• Slow

Statistics at commercial institutes

• Unknown Quality

• Fast

Bigger urge for Fast and Reliable Statistics

Data deluge

Attention for positioning Statistics

Netherlands w.r.t. Big Data

6

Big Data Organization: the Roadmap

Strategic importance of Big Data

Responsibility at DG level

Roadmap Big Data: o Mobile phone data

o Traffic loop data

o AIS

o Social Cohesion

Half-yearly update of Roadmap

Coordination Group Big Data

7

Isues

Used Methods

Stability of Sources

Representativity

Privacy

Usability

Infrastructure

Knowledge and Skills

Unknown or nonmatching Metadata

Quality of big Data

8

Big Data Processes Data driven vs. output driven

9

Stats Big Mess of Data

Quality of Big Data The Signal and the Data

10

Manually

Automatic

Q Q

Process parameters

Data Signal

Noise

Big Data Team Backgrounds

11

Martijn Tennekes

Piet Daas

Joep Burger

Marco Puts

Biology Bio-informatics

Computer Science Cognitive Science Psychophysics

Biology

Knowledge engineering

Secret of our Success

Experimental Skills

Explorative Research

Artificial Intelligence

Computer science

12

Some Data sources and Projects

13

Sentiment on social media o “Consumer Confidence”

o Safety indicator

Road Sensor Data o Traffic intensities

AIS data (Automatic Identification System for Vessels) o ESSnet

o Maritime traffic around the Netherlands

o Inland waterway traffic

o Emissions of vessels

Mobile phone meta data o Daytime population

o Tourism

Social Cohesion o Relationship between people on social media

The Future

The future

of statistics

looks

BIG 14