recent developments in data analytics and big data
TRANSCRIPT
{ 42 second elevator pitch intro } Mainframes, PDP10’s, Vax VMS, Commondore 64, Apple ][e, IBM PC “servers”, Dialup Modems, MUX’s, Routers, Switches, Firewalls Pre-Internet, The Internet, ISP’s, Napster, MP3, Y2K, DotCom, Web 2.0, Virtualisation, Cloud, Containers, Microservices, WebScale, HPC, MPI / PVM, Hadoop, Spark, Big Data, Openstack, Mesosphere 17k LinkedIn connections, 19k followers on LinkedIn 49k Twitter followers, 66,000 hours of Tech, Telecoms & Business experience, developed & gave away the world’s tiniest Hadoop cluster & Openstack appliance virtual machines, Cloud believer, Hackathons, and I’m a senior editor on Wikipedia
What’s new in Big Data & Data Analytics.. the current rate of change is causing us to sprint, in multiple races, at the same time..
1 Putting a Value on your Data Assets
2 Digital Disruption, Big Data & Analytics
3 Skynet went live somewhere around mid 2012
CONTENTS..
PUTTING A VALUE ON YOUR DATA ASSETS 1
LEVERAGING DIGITAL DISRUPTION & PUTTING VALUE ON DATA TO GAIN SIGNIFICANT ADVANTAGE OVER COMPETITORS
• CIOs get it, but other board members may not understand it yet
• Big Data is rarely viewed as being part of a CEOs agenda
• Value of Big Data won’t be understood by CEOs un?l it’s mone?zed
• Big Data is oAen misunderstood by CFOs as a risk or cost
• Mone?ze Data by puFng it on the balance sheet as an Asset
• Why? Because the value of Big Data is rarely expressed as a Asset
• Value you Big Data as an Asset and treat it as a perishable commodity
Putting a value on Big Data
• We’re all familiar with Physical Assets & Cash
• We’ve begrudgingly learned to manage Human Capital
• We’re still catching up with Intellectual Property
• Most of us of us have a handle on Customer Records
• Databases, Log-files, and Metadata are now on the radar
• But too few businesses value their Data as an Asset
• Yet they know information is valuable, and information is Data !!
Data is often allowed to be a Lazy Asset
• ROI estimations on the value in Big Data isn’t trivial
• Distinguishing cost of gathering & managing Data from cost of doing
business is difficult
• Data does not have a physical presence & can have an infinite life
• Value of Data can quickly depreciate if is able to be readily outdated
• Some Data naturally depreciates in business value over time
• Other Data gains value by being put to unforeseen commercial use
Data is a difficult Asset to classify & value
DIGITAL DISRUPTION, BIG DATA & ANALYTICS 2
ORGANISATIONS WITH MINIMAL PHYSICAL ASSETS HAVE GAINED HIGH MARKET VALUE IN RECORD TIME USING BIG DATA ASSETS
• Worlds largest taxi company owns no taxis ( Uber )
• Largest accommodation provider owns no real estate ( Airbnb )
• Largest phone companies own no telco infra ( Skype, WeChat )
• Worlds most valuable retailer has no inventory ( Alibaba )
• Most popular media owner creates no content ( Facebook )
• Fastest growing banks have no actual money ( SocietyOne )
• Worlds largest movie house owns no cinemas ( NetFlix )
• Largest software vendors don’t write the apps ( Apple & Google)
The Digital Disruption Has Already Happened
• Don’t wait for governments or law to catch up
• A legal precedent wasn’t a precedent until someone created it
• Data governance & Data policies are a fireable offence
• Data retention laws VS Right to be forgotten
• Delete My Account does not actually mean Delete My Data
• Who owns the data & who does or should have access to it
• Data Harmonization and what it means to business & consumers
Looking the other way won’t make it go away
• Everybody has an A.I. in their hands
• Facial recognition is now enabled by default
• Facebook Moments / Apple iPhone / Google Photos
• Enterprise Search is something old people talk about
• Internet of Things has happened and 99% of it is not secure
• Modern aircraft have 6,000 to 10,000 IoT sensors in each wing
• 18,700 daily domestic flights in USA airspace = 43.5 PB per day
Why is Digital Disruption so different
A recent survey of over 700 information managers found:
• 95% don’t understanding of what big data actually is
• 50% had no idea how to prepare for big data
• 20% admitted they weren’t even going to try !!
• < 5% actually had a plan ready to act on
• < 1% were actually doing something
Doing nothing is not a viable strategy
SKYNET WENT LIVE AROUND MID 2012 3
ACCESS TO AND VISIBILITY OF DATA DATA ASSETS IS NOW PAR FOR THE COURSE, BOTH INSIDE AND OUTSIDE THE FIREWALL
• StuxNet
• The Interview “movie”, Sony, X-Box live, Adobe.. OMG !!
• IRC Bots, Viruses, Trojans and your clients data
• Botnets can be rented by the hour and have modern API’s
• Even Siri can in fact find you in all of the following and more:
• Phone contacts, Email messages, Photos, SMS, iMessage,
Calendars, The Internet, and App data of various forms
If you can dream it up we can code it
• If eDiscovery & Data Management of Social Media & Emails are keeping
you awake at night, consider the following landscape challenges:
• Paper, Photos, Files, Faxes, Emails, Web Pages & PDF’s
• Windows, Mac OS X, Linux, Unix, Solaris, OS400, Mainframe
• Fat client apps, Cloud apps, PaaS & SaaS apps, data & logfiles
• Phone & Tablet platforms:
• iOS, Android, Firefox OS, Canonical, Blackbery, Sailfish, Open
Alliance, Microsoft 10 Phone OS
The landscape is shifting faster than you are
• Hadoop distributions & the “big data in a box” Big iron game
• Tiny hadoop appliance iTnews “lunch bet”
• Big Data on your laptop is now the norm
• Software vendors are building Big Data into their tools & platforms, from Excel
hadoop IAP’s to SAP HANA
• Bursting into public clouds for instant super computers
• One size does not fit all, and Failure is the new Black
• Big Data is what you make it, i.e. Social, Cloud, Email, Fileservers, Intranets,
Websites, The Internet, SMS’s, Bank Records, Phone logs, Human movement
Ecosystems, Clouds & Platform Computing
• Occams Razor is not a safe bet • The simplest answer is not always the correct answer
• Deep Learning / Machine Learning & Big Data can now give us the tools to dive
so much deeper and look far more broadly
• 600+ public data sources and counting !!
• Platforms like Anomaly42 have changed the game for court cases • Spreadsheets found EU$74m in fraud with manual audits
• A42 tools found EU$2.4b using big data eDiscovery
• Predicted 5 year value of EU$15b if left to manual discovery with spreadsheets
The answers are often staring you in the face
• If you torture data enough, it will talk, when do you stop torturing it
• You can’t have everything, where would you put it
• Just because you can’t access it, don’t assume someone else can’t
• The “dark web” isn’t just an Internet issue, Enterprise networks are a minefield
• CIA flipped their 80/20 investment rule - Spooks VS data now Data vs Spooks
• Social media has been used in anger - the USA just killed terrorists based on
data sourced in real time from social media !!
• Mettadata is a waste of time, we can now auto-classify data if we can reach it
Life, the universe and everything = 42
THANK YOU. Dez Blanchfield @dez_blanchfield +61 414 464 356 [email protected]