B I G D A T A L I T T L E

D E V I C E S

W H A T I T W I L L D O T O U S A N D F O R U S

W H A T I S B I G D A T A ?

0 - 2 0 0 3

5 exabytes

2 0 1 1

2.5 exabytes per day

P E R S P E C T I V E S

1MB 1GB 1TB 2PB 5EB

W H E R E ’ S I T C O M I N G F R O M ?

Source: domo.com 2012

W H A T D O E S I T L O O K L I K E ?

D E F I N I T I O N S

• Big Data: unstructured data, don’t know what questions are yet

• Business Intelligence: structured data, know what the questions

you want answered

• Statistics: structured data, not realtime, no action taken as a

result

• Machine Learning: creation of algorithms and applying them to

data sets in an attempt to learn from data

• Predictive Analytics: extracting existing data to predict trends

W H Y N O W ?

• 2003: Doug Cutting & Mike Cafarella, Nutch

• 2004:Google Labs: Map Reduce

• 2006:Doug Cutting moves to Yahoo and creates Hadoop

• 2008: Yahoo open sources Hadoop, Apache Software Foundation

• 2009: Matei Zaharia starts Spark at UC Berkley

• 2013: Spark open sourced under Apache

M A P R E D U C E

Traditional / Sequential

Map

Reduce

S P A R K

x 100

Map

Reduce

C A S E S

W H A T I T W I L L D O T O U S

S E C U R I T Y - P R I V A C YN S A P R I S M

P R O F I L I N G

V U L N E R A B I L I T

Y

• Target

• Home Depot

• Michaels

• Blue Cross Blue Shield

• Sony Entertainment

S O C I E T Y

C O M M E R C EA M A Z O N D A S H

C O M M E R C EA M A Z O N

C A S E S

W H A T I T W I L L D O F O R U S

S P O R T SS A B E R M E T R I C S ( M O N E Y B A L L )

95%

5%

P R O D U C T I V I T YG O O G L E N O W

P O L I T I C SO B A M A C A M P A I G N 2 0 1 2

S C I E N C EM O N T E R E Y B A Y A Q U A R I U M R E S E A R C H I N S T I T U T E

H E A L T HA P P L E R E S E A R C H K I T

for context, Stanford says that it would normally take a national year-long effort to get that kind of scale. The flood of data will theoretically improve the quality of the findings, especially since the automatic, ph

M O R E R E A D I N G

• http://www.domo.com/blog/2014/04/data-never-sleeps-2-0/

• http://www.redorbit.com/education/reference_library/general-2/history-of/1113190638/the-history-of-

mobile-phone-technology/

• http://www.forbes.com/sites/gilpress/2013/05/09/a-very-short-history-of-big-data/

• http://www.wired.com/2015/04/robots-roam-earths-imperiled-oceans/?mbid=nl_041315

• http://www.allbusiness.com/what-does-your-supermarket-know-about-you-15611312-1.html

• http://www.geekwire.com/2015/baseball-analytics-mystery-mlb-team-uses-a-cray-supercomputer-to-

crunch-data/

• http://www.geekwire.com/2015/this-big-data-startup-just-raised-cash-to-analyze-driver-behavior-creating-

safety-scores-for-individual-

motorists/?utm_source=GeekWire+Daily+Digest&utm_campaign=20eb1892b3-daily-digest-

email&utm_medium=email&utm_term=04e93fc7dfd-20eb1892b3-

233387065&mc_cid=20eb1892b3&mc_eid=7b61e5049a

• http://www.newyorker.com/culture/culture-desk/the-horror-of-amazons-new-dash-button

• https://www.amazon.com/oc/dash-button

• http://harvardmagazine.com/2014/03/why-big-data-is-a-big-deal http://www.businessinsider.com/big-data-

is-growing-thanks-to-mobile-2013-1http://venturebeat.com/2015/04/03/how-microsofts-using-big-data-to-

predict-traffic-jams-up-to-an-hour-in-advance/

• http://www.engadget.com/2015/04/13/ibm-watson-health-

cloud/?utm_source=Feed_Classic_Full&utm_medium=feed&utm_campaign=Engadget&?ncid=rss_full

?