big data berlin – automating decisions is the next frontier for big data
Post on 12-Jan-2015
858 Views
Preview:
DESCRIPTION
TRANSCRIPT
A U T O M AT I N G D E C I S I O N S – T H E N E X T F R O N T I E R F O R B I G D ATA
L A R S T R I E L O F F – B L U E Y O N D E R – I ’ M @ T R I E L O F F O N T W I T T E R
T H E F R O N T I E RW H A T N O B O D Y T E L L S Y O U A B O U T
W H O I S U S I N G B I G D ATA T O D AY
W H E R E B I G D ATA I S U S E D
Effective Use
Marketing
Finance
Everyone Else
S O U R C E : O W N , P O T E N T I A L LY B I A S E D O B S E R VAT I O N S
W H E R E B I G D ATA I S U S E D
Potential Use
Marketing
Finance
Everyone Else
S O U R C E : O W N , P O T E N T I A L LY B I A S E D O B S E R VAT I O N S
Better Decisions
Faster Data
More Data
T H R E E A P P R O A C H E S
M O R E D ATAD I G I TA L M A R K E T I N G ’ S A P P R O A C H :
FA S T E R D ATAF I N A N C E ’ S A P P R O A C H
B U T W H AT A B O U T B E T T E R D E C I S I O N S ?
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization
Collection
Storage
Analysis
Prediction
Decision
Collection
Storage
Analysis
Prediction
Decision
Physiological
Safety
Love/Belonging
Esteem
Self-Actualization
“… all others bring data.” !
— W. E D W A R D S D E M I N G
H O W D ATA - D R I V E N D E C I S I O N S S H O U L D W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
H U M A N A N A LY Z E S
H U M A N P R E D I C T S
H U M A N D E C I D E S
H O W D ATA - D R I V E N D E C I S I O N S R E A L LY W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
H U M A N A N A LY Z E S
C O M M U N I C AT I O N B R E A K D O W N
H U M A N D E C I D E S
C O M M U N I C AT I O N B R E A K D O W N
Communication Breakdown, It's always the same, I'm having a nervous breakdown, Drive me insane!
— L E D Z E P P E L I N
• Drill-down analysis … misunderstood or distorted• Metrics dashboards … contradictory and confusing• Monthly reports … ignored after two iterations• In-house analyst teams … overworked and powerless
H O W D ATA - D R I V E N D E C I S I O N S R E A L LY W O R K
H T T P : / / D I L B E R T. C O M / S T R I P S / C O M I C / 2 0 0 7 - 0 5 - 1 6 /
H O W D E C I S I O N S R E A L LY S H O U L D W O R K
C O M P U T E R C O L L E C T S
C O M P U T E R S T O R E S
C O M P U T E R A N A LY Z E S
C O M P U T E R P R E D I C T S
C O M P U T E R D E C I D E S
99.9% of all business decisions can be automated
H O W D E C I S I O N S A R E M A D E
Human Decisions
Business Rules
No Decision, Decision-by-default
B U S I N E S S R U L E S = P R O G R A M M I N G
• Business rules are like programs – written by non-programmers
• Business rules can be contradictory, incomplete, and complex beyond comprehension
• Business rules have no built-in feedback mechanism: “It is the rule, because it is the rule”
H U M A N D E C I S I O N M A K I N G
• • System 1: Fast, automatic, frequent, emotional, stereotypic, subconscious
• • System 2: Slow, effortful, infrequent, logical, calculating, conscious
D A N I E L K A H N E M A N N , T H I N K I N G FA S T A N D S L O W
H O W T O D E C I D E FA S TF R E Q U E N T D E C I S I O N M A K I N G M E A N S FA S T D E C I S I O N M A K I N G , M E A N S U S I N G H E U R I S T I C S O R C O G N I T I V E B I A S E S
Anchoring effect
IKEA effect
Confirmation bias
Bandwagon effect
Substitution
Availability heuristic Texas Sharpshooter Fallacy
Rhyme as reason effect
Over-justification effect
Zero-risk bias
Framing effect
Illusory correlationSunk cost fallacy
Overconfidence
Outcome bias
Inattentional Blindness
Benjamin Franklin effect
Hindsight bias
Gambler’s fallacy
Anecdotal evidenceNegativity bias
Loss aversion
Backfire effect
H O W C O M P U T E R S D E C I D E FA S TM A C H I N E L E A R N I N G O F F E R S A N A LT E R N A T I V E T O H U M A N C O G N I T I V E B I A S E S A N D C A N B E M A D E FA S T T H R O U G H B I G D A TA
K-Means Clustering
Naive BayesSupport Vector Machines Affinity Propagation
Least Angle Regression
Nearest Neighbors
Decision Trees
Markov Chain Monte Carlo
Spectral clustering
Restricted Bolzmann Machines
Logistic Regression
Better decisions through predictive applications
H O W P R E D I C T I V E A P P L I C AT I O N S W O R K
C O L L E C T & S T O R E
A N A LY Z E C O R R E L AT I O N S
B U I L D D E C I S I O N M O D E L
D E C I D E & T E S T O P T I M I Z E
W H Y T E S T ?
– R A N D A L L M U N R O E
“Correlation doesn’t imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing ‘look
over there’”
S T O R Y T I M E(Not safe for vegetarians)
T H E G R O U N D B E E F D I L E M M A
T H E G R O U N D B E E F D I L E M M A
Yesterday Today Tomorrow Next Delivery Next Day
In Stock Demand
T H E G R O U N D B E E F D I L E M M A
• Order too much and you will have to throw meat away when it goes bad. You lose money and cows die in vain
• Order too little and you won’t serve all your potential customers. You lose money and customers stay hungry.
A C C U R AT E LY P R E D I C T D E M A N DC H A L L E N G E # 1
A U T O M AT E R E P L E N I S H M E N TC H A L L E N G E # 2
C O L L E C T S T O C K A N D
S A L E S
P R E D I C T D E M A N D
T R A D E O F F C O S T S
C R E AT E O R D E R S I N E R P
S Y S T E M
O P T I M I Z E & R E P E AT
@ B YA N A LY T I C S _ E N @ T R I E L O F F
top related