The Art and Science of Real Time Analytics Olaf Larson
Proprietary and Confidential - ©2016 Clarity Solution Group, LLC. All Apache and related Apache project trademarks or service marks are the property of the Apache Foundation.
2
Collapsing the speed of decision making?
It is not about real time data, it is about real time action
An opportunity missed is an opportunity lost
3
What we’re going to talk about today
Faster Time to Action
• How can it help
• What hinders success
• What can you do
4
How can faster time to action help you?
Leverage of text mining through unstructured data in real time provided rapid detection of brand damaging communications in social media
• This enabled was quick reaction through PR and social media sites.
• The rapid reaction reduced negative consequences significantly and prevented substantial corporate losses.
5
How can faster time to action help you?
Streaming analytics of online
sales volumes picked up pricing
errors
• System used pattern recognition to quickly identify negative behaviors
• Reduced losses and preventing potential damaged reputation.
6
How can faster time to action help you?
Leverage of near real
time online activity to
shape shopping cart
recommendations for
major retail vendor
7
How can faster time to action help you? (
Leverage of streaming online
activity data merged with
preformed models allowed
for rapid identification of high
risk activity and potential
fraud.
8
Why are some efforts less successful?
Fast data and
processing with
no ability or
need to act
Analytics need to be
aligned with business value
• e.g. rapid reporting of information for actions that are not high value
9
Why are some efforts less successful?
Fast data and processing with
no ability or need to act
Faster data without faster
decisions
Fast analytics that aren’t fast enough
(10 seconds isn’t
actually fast)
Fast data, fast decisions, fast
actions, no impact
Your decision framework
needs to be aligned to
your analytics
• Need ability to shift strategy at speed
Architecture or Technology misaligned • Using standard platforms vs.new architectures
designed to optimize quick pass through and
rapid processing • Trying to push too much computing into
interaction instead of leveraging callable services or restful APIs
11
Collapse the speed of decision making
• Rationalize your decision framework
• Change paradigms
• Improve the decision process
• Get ahead of the curve on detection
• Speed up consumption
• Match platforms and tools to the need
12
Rationalize Your Decision Framework
Key question: what decisions need to be made when given
your current business strategy?
• Pushing real time data through is not without cost and if there is no real need for real time analytics there is a loss of ROI
• Align data availability, tools, and decision engines with framework
•Real Time vs. Near Real Time vs. Daily Batch vs. Monthly Batch vs.
Quarterly Batch
13
Rationalize Your Decision Framework
Create an agile
decision framework
Decentralized decision authority allows better
flexibility to leverage more rapid data and insights
14
Change the paradigm
Recognize
our job is not
just data
exploration
and reporting,
but driving
action
Move from Insight Delivery To Change Agent
15
Really change the paradigm
Deliver decisions
instead of just insights
• Reduce the time to action by capturing the decision algorithm
• Requires:
• Trust in your data and engines
• Trust in your analytics processes and teams
16
Change the paradigm
• Stop measuring your teams on time to insight
• Measure success by actions taken and business outcomes driven
S
u
c
c
e
s
s
17
Apply Process Engineering to Decisions
• Embed analytics into processes
• Make the non-automated decision process more efficient
• Push decisions down stream as far as possible
• Empower decisioning with guard rails and business rules
• Think Golidlocks – not too centralized, but not out of control either
18
Get ahead of the curve on detection
More time is lost waiting to find out you need to take action than actually waiting on analysis to guide actions
oAutomated detection of anomalies and trend changes
oPushed alerts
oForecasting
oLead indicators
19
Speed up consumption
Information Ergonomics
Cut the clutter
Put insights where they will be used not in separate
reports
Provide insight on demand or proactively
20
Align Infrastructure to Framework
Assemble and position your data based on need
Make sure you can move the data and process algorithms in
time
Move ad hoc analytics capabilities near real time
• Spark provides us a common data fabric for computation across batch data integration, Data Science, and realtime streaming data
Tap your “big data” where it sits using rapid analytics tools
Implement agile analytics
Propriet ary and Confidential - ©2016 Clarity Solution Group, LLC
Change Your Culture
• Think processes, not products
• Think enablement and automation
• Align analytics resources with decision processes (not departments or data sources)
• Get and grow the right kind of people
• Think Agile – move quickly and assemble on the fly
22
Collapse the speed of decision making
• Rationalize your decision framework
• Change paradigms
• Improve the decision process
• Get ahead of the curve on detection
• Speed up consumption
• Match platforms and tools to the need
Olaf Larson
Partner
www.clarity-us.com