small sensors. big data. - iiea.com · small sensors. big data. prof. barry smyth insight centre...

54
Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin e: [email protected] t: @barrysmyth

Upload: others

Post on 26-May-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

Small Sensors. Big Data.

Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin !e: [email protected] t: @barrysmyth

Page 2: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

Data, data, everywhere…

Page 3: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

1 = 10bytes18exabyte

Page 4: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

1 = 10bytes18exabyte

1000,000,000,000,000,000

Page 5: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

1 = 10bytes18exabyte

20,000 x all of the printed material in the

US Library of Congress. Or all of the words spoken by humans. Ever!

Page 6: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

1 = 10bytes18exabyte

6 !hours

But, we now create this much information every

Page 7: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

A PARADIGM SHIFT

algorithm !

data

algorithm !

data

Page 8: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

computation

sensorsdev

data

Page 9: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

THE WORLD’S FIRST SUPERCOMPUTER

Brainchild of Seymour Cray, in 1964 the closet-sized CDC 6600 was the biggest, baddest computer of the age. !

5,500 kgs, 480 kb RAM, 3M FLOPs, $60m

Page 10: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

MOORE’S MAGICAL LAWS

Page 11: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

A SELF-FULFILLING PROPHECY?

CDC 6600

[1964]

IBMPC [1981]

iPHONE 5S [2013]

0.05FLOPs/$

200FLOPs/$

8MFLOPs/$

x4,000 x40,000

Page 12: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

IF MOORE’S LAW APPLIED TO CARS?

“If the auto industry had moved at the same speed ... ...your car today would cruise comfortably at a million miles an hour and probably get a half a million miles per gallon of gasoline. But it would be cheaper to throw your Rolls Royce away than to park it downtown for an evening.” (Carver Mead)

Page 13: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

THE RISE OF THE SENSOR WEB

Page 14: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

UNIVERSAL MOBILE SENSING PLATFORM

Page 15: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

UNIVERSAL MOBILE SENSING PLATFORM

C A M E R A

M ICROPHONES P E E D

L I G H T

O R I E N T A T I O N

HUM IDITY

T E M P E R A T U R E

L OCATION

TOUCH

MOTION

DIRECT IONF I N G E R P R I N T S

Page 16: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

Connectivity high-speed data

Mobility location-aware

Power always on

Page 17: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

THE DISRUPTION OF HEALTHCARE

Always-on personal sensing, 24/7/365 !

The Creative Disruption of Healthcare !

Activity and exercise, sleep and moods, food, blood glucose, heart rate, pulse ox, lung function, ... !

Page 18: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

THERE’S AN APP FOR THAT

Page 19: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

EXERCISE & FITNESS

Runkeeper iPhone/Android !

Running, Walking, Biking, ... !

Age, gender, weight, ... !

Location, pace, duration, climb, calories, heart rate,...

Page 20: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

TRACKING SLEEP

Basic ‘sleep tracking’ based on motion. !

Duration vs Movement !

Sleep Quality (≈ time/move) !

Sleep Notes / Wakeup Moods !

Comparative Analytics

Page 21: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

MOOD & FOCUS

The Melon Headband !

Uses EEG to track brain activity to assess ‘focus’. !

Tagging, location, and activity information helpsusers to better assess what impacts their focus.

Page 22: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

FOOD & NUTRITION

Meal logging and nutritionalanalysis. !

Manual vs Semi-Automatic. !

Calorie goals and diet plans. !

Integrated weight tracking.

Page 23: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

HEART RATE SENSING

Using smartphone camera with your finger. No external sensor required. !

Detecting colour changes due to capillary blood-flow. !

Tagging, comparative analytics etc

Page 24: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

MOBILE SPIROMETRY

Using a mobile phone microphone to evaluate lung function. !

FVC, FEV, PEF measures. !

Audio ⇒ Features ⇒ Machine Learning. !Mean 5.1% error wrt clinical spirometry ⇒ suitable for home-based monitoring.

Page 25: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

MOBILE SPIROMETRY

Using a mobile phone microphone to evaluate lung function. !

FVC, FEV, PEF measures. !

Audio ⇒ Features ⇒ Machine Learning. !Mean 5.1% error wrt clinical spirometry ⇒ suitable for home-based monitoring.

Page 26: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

CONSUMER-DRIVEN HEALTHCARE?

Towards preventative, sensor-based, data-driven healthcare. !

Sparse checkups ⇒ 24/7/365 Sensing !

The data is ours to share ... !

Apps vs Prescriptions?

Page 27: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

ALWAYS ON MOBILE SENSING

Page 28: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

SCALING

Page 29: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

vertical scaling

Page 30: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

horizontal scaling

Page 31: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

horizontal scaling

Page 32: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

PARTICIPATORYSENSING

Page 33: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

ASTHMOPOLIS SMART INHALER

Page 34: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

PARTICIPATORY SENSING

Page 35: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

HACKING YOUR COMMUTE

GPS & Navigation Assistants !

Map Apps Rule the World !

TomTom, Garmin, Google, Apple, Nokia, ... !

Page 36: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

CROWDSOURCED MAPPING (WAZE)

Free smartphone app. !

Real-time sensing of users’ location, time, speed etc. !

x millions of users !

= social mapping + traffic flow, alerts, hazards, ...

Page 37: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:
Page 38: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:
Page 39: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

TURNING PEOPLE INTO SENSORS

Participatory/Citizen Sensing !Big, messy data ⇒ real-time insights. !The smartphone as a mobile sensor platform... !... and the willingness of people to contribute to data to causes that matter to them.

Page 40: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

FROM REAL TO VIRTUAL SENSORS

Page 41: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

MINING THE DATA EXHAUST

From Real to Virtual Sensors !

Page Views, Read Times, Mouse Movements, Search Queries, Result Clicks, Social Connections, Share, Comments, Likes, Posts, Emails, IMs, ...

Page 42: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

THE ORIGINAL BIG DATA COMPANY

Mining relevance & reputation from links. !

Search logs as sensor data.

Page 43: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

SEARCH LOGS AS SENSOR DATA

“... Web search ... can be likened to a large-scale distributed network of sensors for identifying potential side effects of drugs. There is a potential public health benefit in listening to such signals, and integrating them with other sources of information.”

“Web-Scale Pharmacovigilance: Listening to Signals from the Crowd” J Am Med Inform Assoc. (2013)

Page 44: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

SENSING DRUG SIDE-EFFECTS

82M Queries

6M Users

Page 45: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

SENSING FLU TRENDS

Identified trigger terms correlated we known past outbreaks. Tracked real-time occurrence of these terms, location by location to deliver accurate* regional outbreak maps that correlated well with verified CDC data.

Page 46: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

MINING USER-GENERATED REVIEWS

Page 47: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

USER-GENERATED REVIEWS

+‘ves staff

location bed

service breakfast

-‘ves noise elevators carpet health club public transport

Chicago Hotels

Page 48: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

OPINION AMPLIFICATION

Twitter, FaceBook as a source of real-time opinions. !

Raw Text ⇒ Sentiment ⇒ Opinion !

These days Twitter data has been used to predict election outcomes, box office success, and musical talent ...

Page 49: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

DATA-DRIVEN EVERYTHING

Social Science, Linguistics, Anthropology, Cultural Studies, Journalism, Political Science, Humanities ... !

All impacted by Big Data Thinking...

Page 50: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

GOOGLE’S N-GRAM VIEWER

Acerbi A, Lampos V, Garnett P, Bentley RA (2013) The Expression of Emotions in 20th Century Books. PLoS ONE 8(3)

Page 51: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

DATA-DRIVEN EVERYTHING

Michel J-P, Shen YK, Aiden AP, Veres A, Gray MK, et al. (2011) Quantitative analysis of culture using millions of digitized books. Science 331: 176–182 !Lieberman E, Michel J-P, Jackson J, Tang T, Nowak MA (2007) Quantifying the evolutionary dynamics of language. Nature 449: 713–716 !Richards, Daniel Rex. "The content of historical books as an indicator of past interest in environmental issues." Biodiversity and Conservation (2013): 1-9. !Lampos, Vasileios, et al. "Analysing Mood Patterns in the United Kingdom through Twitter Content." arXiv preprint arXiv:1304.5507 (2013).

Page 52: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

POWER TO THE PEOPLE

The Big Data Revolution is here today! !

Transformation ⇒ Disruption ⇒ Opportunity !

People as Sensors ⇒ Collective Intelligence !

Personal Data, Privacy, Ethics …

Page 53: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

SMALL SENSORS. BIG DATA. FROM CLARITY TO INSIGHT IN THE WORLD OF THE SENSOR WEB

Creating a Data-Driven Society

Page 54: Small Sensors. Big Data. - iiea.com · Small Sensors. Big Data. Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin ! e: barry.smyth@insight-centre.org t:

Small Sensors. Big Data.

Prof. Barry Smyth Insight Centre for Data Analytics University College Dublin !e: [email protected] t: @barrysmyth