big data trends
TRANSCRIPT
Collabor8now Ltd
1st December 2016
Steve Dale @stephendaleUnless otherwise noted, this work is licensed under a Creative Commons
Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Trends in Big Data, Data Analytics & AI
Collabor8now Ltd
What is “Big Data”?
Big Data is data whose scale, diversity and complexity require new architecture, techniques, algorithms, and analytics to manage it and extract value and hidden knowledge from it…
Collabor8now Ltd
Big Data – Big Challenges
• Structured e.g. databases• Semi-structured e.g. email, e-forms, HTML, XML• Unstructured e.g. document collections (text),
social interactions (text, images, video, sound)• Machine generated e.g. weblogs, sensor data, etc.
Big Data can be a combination of different data formats:
There is massive growth in unstructured data…but just wait for the IoT!
Collabor8now Ltd
Big Data Challenges
Image Source: IBM
It’s not just how fast data is produced or changed, but the speed at which it must be received,
understood and processed.
Collabor8now Ltd
The credibility gap
Collabor8now Ltd
Time
Dat
a Vol
ume Data available to an organisation
Data an organisation can process
Collabor8now Ltd
Data-driven decisions
Source: PwC Global Data & Analytics Survey 2016
8%
53%
39%
Highly data driven
Rarely data driven
Somewhat data driven
Decision making is best described as
27%
28%
29%
13%
Predictive: What could happen?
Use of analytics is mostly
Prescriptive: What should happen now?
Descriptive: What happened?
Diagnostic: Why did it happen?
Collabor8now Ltd
What do decision-makers need?Strategic decisions are still often based on instinct. But more businesses are beginning to look at sophisticated machine learning algorithms to support decision making.
Our next decision will likely be based on:
Machine Algorithms Human Judgement
59%41%
Source: PwC Global Data & Analytics Survey 2016
A mix of mind and machine
Collabor8now Ltd
AI Lexicon
Collabor8now Ltd
Machine LearningMachine learning techniques are designed to seek out
opportunities to optimise decisions based on the predictive value of large-scale data sets.
Image Source: Tata Consultancy Services
Collabor8now Ltd
Analytical Techniques
Cluster analysis The task of grouping a set of objects in such a way that objects in the same group (cluster) are more similar, in some sense or another, to each other than to those in other groups clusters).
Comparative Analysis. A step-by-step procedure of comparisons and calculations to detect patterns within very large data sets
Descriptive tree analytics A decision support tool that uses a tree-like graph of decisions and their possible consequences including chance event outcomes, resource costs and utility
Factor analysis Used to analyse large numbers of dependent variables to detect certain aspects of the independent variables (factors) affecting those dependent variables.
Machine learning A type of artificial intelligence which provides computers with the ability to learn without being explicitly programmed.
Multivariate analysis The observation and analysis of more than one statistical outcome variable at a time..
Regression analysis A statistical process for estimating relationships between a dependent variable and one or more independent variables.
Segmentation analysis Divides a broad category into subsets that have, or are perceived to have, common features, needs, interests or priorities.
Sentiment analysis The process of identifying and categorising opinions expressed in a piece of text to determine whether the writer’s attitude towards a topic or issue is positive, negative or neutral.
Simulation The imitation of the operation of a real world process or system over time. It requires a model that represents the key characteristics or behaviours of the selected physical or abstract system or process.
Time Series analysis Comprises methods for analysing time series data to extract meaningful statistics and other characteristics of the data.
Collabor8now Ltd
The art and science of decision making
Unlock existing insights. Data do not have to be “big” to be useful. Analysing databases previously mothballed or kept in silos can lead to fresh insights.
1Beware of inherent bias. Important decisions have already taken place before data analysis. Understand the provenance and quality of the data.
2
Invest in talent. Can you give existing employees a foundation in data analysis before recruiting new data scientists?
3Accountability. Be clear about who has decision making rights. Opening up access to data and analysis can allow decisions to be challenged.
4
Collabor8now Ltd
Life in a Big Data World
Collabor8now Ltd
Trends & Predictions
• Power to the business users Information Week 2016
• By 2018, 20% of business content will be authored by machines. Gartner 2016
• Embedding intelligence Gartner 2016
• Shortage of talent A.T. Kearney 2016
• Machine Learning gaining momentum Ovum 2016
• Data-as-a-Service Business Models Forrester 2016
• Real-time insights Forrester 2016
• The start of algorithm markets Forrester 2016
• By 2018, 3 million workers worldwide will be supervised by roboboss. Gartner 2016
Collabor8now Ltd
IBM Watson Editions
Collabor8now Ltd
IBM WatsonTry it for yourself – free!
Go to: http://www.ibm.com/analytics/watson-analytics/ and sign-in with a valid email address. Once your account has been validated, sign-in and you'll see the main Watson interface: https://watson.analytics.ibmcloud.com/
Worth looking at the help videos, and I recommend: - Getting Started- Load your data- Create an Assembled View - Create an Exploration- Create a Prediction
Also – IBM offer regular webinars for new users: http://www.ibm.com/smarterplanet/us/en/ibmwatson/building-with-watson-webinar.html
Collabor8now Ltd
Unless otherwise noted, this work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported License.
Steve Dale @[email protected]
“Errors using inadequate data are much less than those using no data at all” Charles Babbage