recent developments in data analytics and big data

22
{ 42 second elevator pitch intro } Mainframes, PDP10’s, Vax VMS, Commondore 64, Apple ][e, IBM PC “servers”, Dialup Modems, MUX’s, Routers, Switches, Firewalls Pre-Internet, The Internet, ISP’s, Napster, MP3, Y2K, DotCom, Web 2.0, Virtualisation, Cloud, Containers, Microservices, WebScale, HPC, MPI / PVM, Hadoop, Spark, Big Data, Openstack, Mesosphere 17k LinkedIn connections, 19k followers on LinkedIn 49k Twitter followers, 66,000 hours of Tech, Telecoms & Business experience, developed & gave away the world’s tiniest Hadoop cluster & Openstack appliance virtual machines, Cloud believer, Hackathons, and I’m a senior editor on Wikipedia

Upload: dez-blanchfield

Post on 14-Apr-2017

3.333 views

Category:

Data & Analytics


0 download

TRANSCRIPT

Page 1: Recent developments in data analytics and big data

{ 42 second elevator pitch intro } Mainframes, PDP10’s, Vax VMS, Commondore 64, Apple ][e, IBM PC “servers”, Dialup Modems, MUX’s, Routers, Switches, Firewalls Pre-Internet, The Internet, ISP’s, Napster, MP3, Y2K, DotCom, Web 2.0, Virtualisation, Cloud, Containers, Microservices, WebScale, HPC, MPI / PVM, Hadoop, Spark, Big Data, Openstack, Mesosphere 17k LinkedIn connections, 19k followers on LinkedIn 49k Twitter followers, 66,000 hours of Tech, Telecoms & Business experience, developed & gave away the world’s tiniest Hadoop cluster & Openstack appliance virtual machines, Cloud believer, Hackathons, and I’m a senior editor on Wikipedia

Page 2: Recent developments in data analytics and big data

What’s new in Big Data & Data Analytics.. the current rate of change is causing us to sprint, in multiple races, at the same time..

Page 3: Recent developments in data analytics and big data

1  Putting a Value on your Data Assets

2  Digital Disruption, Big Data & Analytics

3  Skynet went live somewhere around mid 2012

CONTENTS..

Page 4: Recent developments in data analytics and big data

PUTTING A VALUE ON YOUR DATA ASSETS 1

Page 5: Recent developments in data analytics and big data

LEVERAGING DIGITAL DISRUPTION & PUTTING VALUE ON DATA TO GAIN SIGNIFICANT ADVANTAGE OVER COMPETITORS

Page 6: Recent developments in data analytics and big data

•  CIOs  get  it,  but  other  board  members  may  not  understand  it  yet  

•  Big  Data  is  rarely  viewed  as  being  part  of  a  CEOs  agenda  

•  Value  of  Big  Data  won’t  be  understood  by  CEOs  un?l  it’s  mone?zed  

•  Big  Data  is  oAen  misunderstood  by  CFOs  as  a  risk  or  cost  

•  Mone?ze  Data  by  puFng  it  on  the  balance  sheet  as  an  Asset  

•  Why?  Because  the  value  of  Big  Data  is  rarely  expressed  as  a  Asset  

•  Value  you  Big  Data  as  an  Asset  and  treat  it  as  a  perishable  commodity  

Putting a value on Big Data

Page 7: Recent developments in data analytics and big data

•  We’re all familiar with Physical Assets & Cash

•  We’ve begrudgingly learned to manage Human Capital

•  We’re still catching up with Intellectual Property

•  Most of us of us have a handle on Customer Records

•  Databases, Log-files, and Metadata are now on the radar

•  But too few businesses value their Data as an Asset

•  Yet they know information is valuable, and information is Data !!

Data is often allowed to be a Lazy Asset

Page 8: Recent developments in data analytics and big data

•  ROI estimations on the value in Big Data isn’t trivial

•  Distinguishing cost of gathering & managing Data from cost of doing

business is difficult

•  Data does not have a physical presence & can have an infinite life

•  Value of Data can quickly depreciate if is able to be readily outdated

•  Some Data naturally depreciates in business value over time

•  Other Data gains value by being put to unforeseen commercial use

Data is a difficult Asset to classify & value

Page 9: Recent developments in data analytics and big data

DIGITAL DISRUPTION, BIG DATA & ANALYTICS 2

Page 10: Recent developments in data analytics and big data

ORGANISATIONS WITH MINIMAL PHYSICAL ASSETS HAVE GAINED HIGH MARKET VALUE IN RECORD TIME USING BIG DATA ASSETS

Page 11: Recent developments in data analytics and big data

•  Worlds largest taxi company owns no taxis ( Uber )

•  Largest accommodation provider owns no real estate ( Airbnb )

•  Largest phone companies own no telco infra ( Skype, WeChat )

•  Worlds most valuable retailer has no inventory ( Alibaba )

•  Most popular media owner creates no content ( Facebook )

•  Fastest growing banks have no actual money ( SocietyOne )

•  Worlds largest movie house owns no cinemas ( NetFlix )

•  Largest software vendors don’t write the apps ( Apple & Google)

The Digital Disruption Has Already Happened

Page 12: Recent developments in data analytics and big data

•  Don’t wait for governments or law to catch up

•  A legal precedent wasn’t a precedent until someone created it

•  Data governance & Data policies are a fireable offence

•  Data retention laws VS Right to be forgotten

•  Delete My Account does not actually mean Delete My Data

•  Who owns the data & who does or should have access to it

•  Data Harmonization and what it means to business & consumers

Looking the other way won’t make it go away

Page 13: Recent developments in data analytics and big data

•  Everybody has an A.I. in their hands

•  Facial recognition is now enabled by default

•  Facebook Moments / Apple iPhone / Google Photos

•  Enterprise Search is something old people talk about

•  Internet of Things has happened and 99% of it is not secure

•  Modern aircraft have 6,000 to 10,000 IoT sensors in each wing

•  18,700 daily domestic flights in USA airspace = 43.5 PB per day

Why is Digital Disruption so different

Page 14: Recent developments in data analytics and big data

A recent survey of over 700 information managers found:

•  95% don’t understanding of what big data actually is

•  50% had no  idea  how  to  prepare  for  big  data

•  20% admitted they weren’t even going to try !!

•  < 5% actually had a plan ready to act on

•  < 1% were actually doing something

Doing nothing is not a viable strategy

Page 15: Recent developments in data analytics and big data

SKYNET WENT LIVE AROUND MID 2012 3

Page 16: Recent developments in data analytics and big data

ACCESS TO AND VISIBILITY OF DATA DATA ASSETS IS NOW PAR FOR THE COURSE, BOTH INSIDE AND OUTSIDE THE FIREWALL

Page 17: Recent developments in data analytics and big data

•  StuxNet

•  The Interview “movie”, Sony, X-Box live, Adobe.. OMG !!

•  IRC Bots, Viruses, Trojans and your clients data

•  Botnets can be rented by the hour and have modern API’s

•  Even Siri can in fact find you in all of the following and more:

•  Phone contacts, Email messages, Photos, SMS, iMessage,

Calendars, The Internet, and App data of various forms

If you can dream it up we can code it

Page 18: Recent developments in data analytics and big data

•  If eDiscovery & Data Management of Social Media & Emails are keeping

you awake at night, consider the following landscape challenges:

•  Paper, Photos, Files, Faxes, Emails, Web Pages & PDF’s

•  Windows, Mac OS X, Linux, Unix, Solaris, OS400, Mainframe

•  Fat client apps, Cloud apps, PaaS & SaaS apps, data & logfiles

•  Phone & Tablet platforms:

•  iOS, Android, Firefox OS, Canonical, Blackbery, Sailfish, Open

Alliance, Microsoft 10 Phone OS

The landscape is shifting faster than you are

Page 19: Recent developments in data analytics and big data

•  Hadoop distributions & the “big data in a box” Big iron game

•  Tiny hadoop appliance iTnews “lunch bet”

•  Big Data on your laptop is now the norm

•  Software vendors are building Big Data into their tools & platforms, from Excel

hadoop IAP’s to SAP HANA

•  Bursting into public clouds for instant super computers

•  One size does not fit all, and Failure is the new Black

•  Big Data is what you make it, i.e. Social, Cloud, Email, Fileservers, Intranets,

Websites, The Internet, SMS’s, Bank Records, Phone logs, Human movement

Ecosystems, Clouds & Platform Computing

Page 20: Recent developments in data analytics and big data

•  Occams Razor is not a safe bet •  The simplest answer is not always the correct answer

•  Deep Learning / Machine Learning & Big Data can now give us the tools to dive

so much deeper and look far more broadly

•  600+ public data sources and counting !!

•  Platforms like Anomaly42 have changed the game for court cases •  Spreadsheets found EU$74m in fraud with manual audits

•  A42 tools found EU$2.4b using big data eDiscovery

•  Predicted 5 year value of EU$15b if left to manual discovery with spreadsheets

The answers are often staring you in the face

Page 21: Recent developments in data analytics and big data

•  If you torture data enough, it will talk, when do you stop torturing it

•  You can’t have everything, where would you put it

•  Just because you can’t access it, don’t assume someone else can’t

•  The “dark web” isn’t just an Internet issue, Enterprise networks are a minefield

•  CIA flipped their 80/20 investment rule - Spooks VS data now Data vs Spooks

•  Social media has been used in anger - the USA just killed terrorists based on

data sourced in real time from social media !!

•  Mettadata is a waste of time, we can now auto-classify data if we can reach it

Life, the universe and everything = 42

Page 22: Recent developments in data analytics and big data

THANK YOU. Dez Blanchfield @dez_blanchfield +61 414 464 356 [email protected]