quantifying and forecasting uncertainty in innovation project management - dr. jose a. briones
DESCRIPTION
Product innovation has been described as the way out of today’s difficult business environment. The rate of success of development projects, in particular disruptive innovation projects remains too low. In this presentation we will discuss the use of @RISK and Probabilistic Decision Analysis in the management of innovation projects with high levels of uncertainty. The launch of the iPad is used as a case studyTRANSCRIPT
QUANTIFYING UNCERTAINTY IN INNOVATION PROJECT
MANAGEMENT
Jose A. Briones, Ph.D.
SpyroTek Performance Solutions
Twitter: @Brioneja
Agenda
Problem Definition New Framework for Innovation
Management Discovery Driven Planning Reverse Income Statement The “Flaw of Averages” Probabilistic Decision Analysis Case Study: Forecast of iPad Sales
www.Brioneja.com Twitter: @Brioneja
Introduction Product innovation has been described as the
way out of today’s difficult business environment.
The rate of success of development projects, in particular disruptive innovation projects remains too low.
We believe that a reason for the low success rate is the erroneous application of analysis methods designed for incremental innovation like NPV and DCF to projects with high levels of uncertainty
www.Brioneja.com Twitter: @Brioneja
Background
In this Chapter of the “Beyond Stage Gate” series we will discuss the use of Discovery Driven Planning and Probabilistic Decision Analysis in the management of innovation projects with high levels of uncertainty.
Probabilistic Decision Analysis, when combined with the right management processes like Discovery Driven Planning is a very effective approach to evaluate and manage the risk and potential of innovation projects
www.Brioneja.com Twitter: @Brioneja
• “We keep rediscovering that the root reason for established companies’ failure to innovate is that managers don’t have good tools to help them understand markets, build brands, find customers, select employees, organize teams, and develop strategy”
• “Some of the tools typically used for financial analysis, and decision making about investments, distort the value, importance, and likelihood of success of investments in innovation”
• “There’s a better way for management teams to grow their companies. But they will need the courage to challenge some of the paradigms of financial analysis and the willingness to develop alternative methodologies”
Clayton M. Christensen
www.Brioneja.com Twitter: @Brioneja
Classical Stage-Gate Process
Project is managed in a linear fashion
“The Stage-Gate system assumes that the proposed strategy is the right strategy, the problem is that except in the case of incremental innovations, the right strategy cannot be completely known in advance” – Clayton M. Christensen
Source: Product Development Institute, Inc.
www.Brioneja.com Twitter: @Brioneja
Challenge
Create a unified framework for innovation projects/product development that 1. Provides the flexibility needed for
innovation to work
2. Still has the metrics needed for proper measurement of progress and resource allocation.
www.Brioneja.com Twitter: @Brioneja
A New Innovation Project Categorization
Tech
nica
l Unc
erta
inty High
Medium
Low
Market UncertaintyLow Medium High
Source: Product Development Management Association
This categorization is based on the most critical variable for new innovation projects: Degree of uncertainty
www.Brioneja.com Twitter: @Brioneja
Resources
Resources
TimeTime
Level
1
2
3
VOC
Idea Generation
Technology Assessment
PrototypeDevelopment
Customer Testing
BusinessCaseValue in Use
Analysis RegulatoryIP Strategy
Supply ChainAnalysis
Roadmap/Timeline
Risk Analysis
The Spiro-Level™ 3-D Approach to InnovationLaunch
Quadrant I
Quadrant IIQuadrant III
Quadrant IV
www.Brioneja.com Twitter: @Brioneja
Overall Project Management
Level 1
• Discovery Driven Planning• Knowledge to Assumption Management
Level 2
• Product/Technology Roadmaps• Multigenerational Product Planning
Level 3
• Linear Stage-Gate• Agile/Lean Development• Rapid Prototyping
www.Brioneja.com Twitter: @Brioneja
Financial Analysis
Level 1
• Reverse Income Statement• Real Options
Level 2
• Probabilistic Decision Analysis
Level 3
• NPV• DCF
www.Brioneja.com Twitter: @Brioneja
Discovery Driven Planning (DDP)
Process for management of innovation and development projects with high levels of uncertainty such as disruptive innovations, new product/new market projects or game changers
1995: First Publication by Rita Gunther McGrath and Ian C. MacMillan (1)
2000: Popularity increases with publication of book “The Entrepreneurial Mindset” (2)
2008: Endorsed by Clayton M. Christensen as an alternative to Stage Gate for high uncertainty projects (3)
2009: Publication of book Discovery Driven Growth (4)
www.Brioneja.com Twitter: @Brioneja
DescriptionDiscovery Driven Planning turns the basic project
map on its head
Discovery Driven Planning is a plan to learn, not to show that you had all the answers when you wrote the plan
“Learn as you go”
DDP uses four basic documents ‘Reverse Income Statement’ models the business
economics. ‘Pro-Forma Operations Specs’ defines operations needed
to run the business. ‘Key Assumptions Checklist’ identifies the business
hurdles and assumptions for the initiative. ‘Milestone Planning Chart’ specifies when the assumption
needs to be tested.www.Brioneja.com Twitter: @Brioneja
Reverse Income Statement
Instead of estimating the market demand and calculating the potential revenue/profit, DDP starts by defining the desired financial target that will meet the company’s targets and strategy
The method then defines the set of assumptions, technical and commercial that MUST be proven true in order for the project to succeed
www.Brioneja.com Twitter: @Brioneja
Generic Reverse Income Statement
www.Brioneja.com Twitter: @Brioneja
Reverse Income StatementRequired Profits (MM$/yr) $20
Return On Sales 20%Target Revenue (MM$/yr) $100Market Share Penetration 30%
Market Size (MM$/yr) $333Unit Price, $ $500
Unit Sales M/yr 200Production Capacity Per Line, Units/Day 100
Production Lines Needed 6Capital Cost of a Production Line, MM$ $1Total Capital Investment Needed, MM$ $6
www.Brioneja.com Twitter: @Brioneja
The Flaw of Averages
The Flaw of Averages states that plans based on the assumption that average conditions will occur are usually wrong
Can you drown in a river that is on average only three
feet deep?
Source: “The Flaw of Averages” Marketingnpv.com
www.Brioneja.com Twitter: @Brioneja
Demand Forecast Problem
Forecast for annual demand: “Between 150,000 and 250,000 units”Average 200,000 units.Plant is sized for average demand estimateProfit is also calculated using average demand
Flaw of averages: Profit WILL be lower than forecastedNo upside to exceeding demand estimate due to plant
capacity limitationsAverage becomes upper limit
www.Brioneja.com Twitter: @Brioneja
Solution: Ranges and Probabilities
Input data and results are expressed as ranges and probabilities80% Chance of number being between 570 and
689K
Source: “The Flaw of Averages” Marketingnpv.com
www.Brioneja.com Twitter: @Brioneja
Probabilistic Decision Analysis Model uses @Risk probabilistic decision analysis
software Monte Carlo simulation Risk and opportunity analysis
Designed for complex projects with high levels of uncertainty Inputs contain high number of variables, either technical or
financial with a high degree of uncertainty, assumptions and dependencies ○ New product development assessment○ Capital spending decisions○ Value chain analysis○ Production and sales forecasting analysis
Eliminates use of “one at a time” cases Analyzes thousands of cases simultaneously Generates a range of outcomes Outcome charts are analyzed to make decisions on direction
www.Brioneja.com Twitter: @Brioneja
Input Ranges Input values are entered in range format –
Width and shape of range are critical inputs Definition of the input ranges is the most critical
step Do not start with the typical value, start with the
range, define the shape of the function (10%, 50%, 90% probability).
There are multiple choices for the shape of the input range:Triangular: Most common for initial
assumptions Normal distribution: Used when more
accurate input data is availablePERT: When data is in form of probabilitiesGamma distribution: Good to model
pricing distributions in B-B cases
5.0% 90.0% 5.0%0.4342 0.6448
0.30
0. 35
0. 40
0. 45
0. 50
0. 55
0. 60
0. 65
0. 70
0. 75
Reverse Income Statement Combined with Probabilistic Decision Analysis
www.Brioneja.com Twitter: @Brioneja
Reverse Income Statement Min Target MaxRequired Profits (MM$/yr) $15 $20 $25
Return On Sales 15% 20% 25%Market Share Penetration 25% 30% 35%
Unit Price, $ $400 $500 $600Production Capacity Per Line, Units/Day 75 100 125Capital Cost of a Production Line, MM$ $0.8 $1.0 $1.4
Define ranges for each input variable
Revenue vs. Market Size
Revenue range is projected w level of probability
www.Brioneja.com Twitter: @Brioneja
8.0% 81.1% 10.8%
$81.00 $120.00
$60
$70
$80
$90
$100
$110
$120
$130
$140
$150
$160
0.000
0.005
0.010
0.015
0.020
0.025
0.030
Target Revenue (MM$/yr)
Target Revenue (MM$/yr)
Minimum $64.17Maximum $151.41Mean $101.06Std Dev $14.69Values 5000
5.0% 90.0% 5.0%
$254.36 $436.48
$150
$200
$250
$300
$350
$400
$450
$500
$550
$600
0.000
0.001
0.002
0.003
0.004
0.005
0.006
0.007
0.008
Market Size (MM$/yr)
Market Size (MM$/yr)
Minimum $191.09Maximum $553.25Mean $338.49Std Dev $54.93Values 5000
Unit Sales vs. Capital
www.Brioneja.com Twitter: @Brioneja
10.5% 79.6% 9.9%
162.5 250.0
100
150
200
250
300
350
0.000
0.002
0.004
0.006
0.008
0.010
0.012
0.014
Unit Sales M/yr
Unit Sales M/yr
Minimum 114.11Maximum 338.25Mean 203.46Std Dev 34.00Values 5000
6.2% 83.8% 9.9%
$4.20 $8.00
$2 $4 $6 $8 $10
$12
$14
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
Total Capital Investment Needed, MM$
Tidal Capital Investment Needed, MM$
Minimum $2.55Maximum $12.64Mean $6.09Std Dev $1.40Values 5000
Capital investment is expressed a range vs. unit sales
Capital Investment Sensitivity Analysis
www.Brioneja.com Twitter: @Brioneja
0.50
-0.45
-0.45
0.44
-0.35-0
.5
-0.4
-0.3
-0.2
-0.1 0.0
0.1
0.2
0.3
0.4
0.5
0.6
Coefficient Value
Unit Price, $
Required Profits (MM$/yr)
Return On Sales
Production Capacity Per Line, Units/Day
Capital Cost of a Production Line, MM$
Total Capital Investment Needed, MM$Regression Coefficients
Sensitivity analysis is a way to focus on the variables that are most important to reduce uncertainty
www.Brioneja.com Twitter: @Brioneja
iPad 2010 Unit Sales (MM) 7.1iPad 2011 Unit Sales (MM) 14.4iPad 2012 Unit Sales (MM) 20.1
Netbook 2010 Unit Sales (MM) 43
Unit Price, $ $650iPad Cost of Materials, $/unit $264.3iPad Cost of Manufacturing, $/unit $10.5iPad Warranty Service, $/unit $20
iPad Indirect Expenses 2010 (MM$/yr) $2,100iPad Indirect Expenses 2011 (MM$/yr) $2,500iPad Indirect Expenses 2012 (MM$/yr) $2,700
Case Study: iPad™ Income Projection A) Static Analysis
Static analysis uses projections prior to launch
iPad Income Projection: Static Analysis
www.Brioneja.com Twitter: @Brioneja
B) Probabilistic Decision Analysis – iPad Income Projection
www.Brioneja.com Twitter: @Brioneja
Min Target MaxiPad 2010 Unit Sales (MM) 1.3 7.1 12.9iPad 2011 Unit Sales (MM) 8.6 14.4 55iPad 2012 Unit Sales (MM) 14.3 20.1 63.3
Netbook 2010 Unit Sales (MM) 36.6 43 58
Unit Price, $ $500 $650 $830iPad Cost of Materials, $/unit $219.0 $264.3 $335iPad Cost of Manufacturing, $/unit $9.0 $10.5 $12iPad Warranty Service, $/unit $15 $20 $25
iPad Indirect Expenses 2010 (MM$/yr) $1,680 $2,100 $2,520iPad Indirect Expenses 2011 (MM$/yr) $2,000 $2,500 $3,000iPad Indirect Expenses 2012 (MM$/yr) $2,160 $2,700 $3,240
iPad Market Share Projections
www.Brioneja.com Twitter: @Brioneja
13.1% 80.8% 6.1%
9.25% 24.25%
0%
5%
10%
15%
20%
25%
30%
35%
0
1
2
3
4
5
6
7
8
iPad Market Share vs. netbooks 2010
iPad Market Share vs. netbooks 2010
Minimum 2.83%Maximum 32.2%Mean 15.6%Std Dev 5.46%Values 5000
13.7% 67.5% 18.8%
32.0% 80.0%
0%
20%
40%
60%
80%
100
%
120
%
140
%
160
%
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
iPad Market Share vs. netbooks 2011
iPad Market Share vs. netbooks 2011
Minimum 17.3%Maximum 143%Mean 57.3%Std Dev 23.7%Values 5000
8.3% 68.1% 23.6%
41.0% 90.0%
20%
40%
60%
80%
100
%
120
%
140
%
160
%
0.0
0.2
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
iPad Market Share vs. netbooks 2012
iPad Market Share vs. netbooks 2012
Minimum 27.5%Maximum 159%Mean 71.7%Std Dev 25.3%Values 5000
Market share analysis projects potential level of penetration and substitution
iPad Revenue Projections
www.Brioneja.com Twitter: @Brioneja
8.6% 75.2% 16.2%
$2,400 $6,350
$0
$1, 0
00
$2, 0
00
$3, 0
00
$4, 0
00
$5, 0
00
$6, 0
00
$7, 0
00
$8, 0
00
$9, 0
00
$10, 0
000.0
0.5
1.0
1.5
2.0
2.5
3.0
Valu
es x
10
-̂4
iPad Revenue 2010 (MM$/yr)
iPad Revenue 2010 (MM$/yr)
Minimum $880.13Maximum $9,811.24Mean $4,685.46Std Dev $1,637.15Values 5000
6.6% 83.9% 9.5%
$8,250 $27,500
$5,0
00
$10, 0
00
$15, 0
00
$20, 0
00
$25, 0
00
$30, 0
00
$35, 0
00
$40, 0
00
$45, 0
00
0
1
2
3
4
5
6
7
Valu
es x
10
-̂5
iPad Revenue 2011 (MM$/yr)
iPad Revenue 2011 (MM$/yr)
Minimum $5,144.81Maximum $41,875.43Mean $17,165.42Std Dev $7,070.77Values 5000
5.0% 90.0% 5.0%
$11,605 $35,590
$5,0
00
$10, 0
00
$15, 0
00
$20, 0
00
$25, 0
00
$30, 0
0 0
$35, 0
0 0
$40, 0
0 0
$45, 0
0 0
$50, 0
0 0
0
1
2
3
4
5
6
Valu
es x
10
-̂5
iPad Revenue 2012 (MM$/yr)
iPad Revenue 2012 (MM$/yr)
Minimum $8,217.53Maximum $46,917.00Mean $21,472.35Std Dev $7,522.30Values 5000
Upper range of revenue shows dramatic upside potential
Max revenue value is from unit sales analysts estimates 7 months after launch
iPad Gross Profit Projections
www.Brioneja.com Twitter: @Brioneja
7.4% 87.1% 5.5%
$252.50 $475.00
$150
$200
$250
$300
$350
$400
$450
$500
$550
$600
0.000
0.001
0.002
0.003
0.004
0.005
0.006
0.007
iPad Gross Profit, $/unit
iPad Gross Profit, $/unit
Minimum $165.47Maximum $562.54Mean $356.74Std Dev $71.92Values 5000
10.0% 85.0% 5.0%
45.25% 62.82%30
%
35%
40%
45%
50%
55%
60%
65%
70%
0
1
2
3
4
5
6
7
iPad Gross Profit Margin
iPad Gross Profit Margin
Minimum 31.6%Maximum 68.8%Mean 53.6%Std Dev 6.12%Values 5000
Gross profit analysis can highlight areas of concern
iPad Income Projections
www.Brioneja.com Twitter: @Brioneja
38.7% 54.1% 7.2%
$50 $2,000
-$2,0
00
-$1, 0
00 $0
$1, 0
00
$2, 0
00
$3, 0
00
$4, 0
00
$5, 0
00
0.0
0.5
1.0
1.5
2.0
2.5
3.0
3.5
4.0
4.5
Valu
es x
10
-̂4
iPad Income 2010 (MM$/yr)
iPad Income 2010 (MM$/yr)
Minimum -$1,869.82Maximum $4,703.02Mean $431.87Std Dev $1,010.77Values 5000
12.0% 77.4% 10.6%
$2,250 $12,500
-$5,0
00 $0
$5, 0
00
$10, 0
00
$15, 0
00
$20, 0
00
$25, 0
00
$30, 0
00
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Valu
es x
10
-̂4
iPad Income 2011 (MM$/yr)
iPad Income 2011 (MM$/yr)
Minimum -$830.65Maximum $26,029.39Mean $6,784.18Std Dev $4,206.06Values 5000
20.5% 64.4% 15.1%
$5,000 $13,750
$0
$5, 0
00
$10, 0
00
$15, 0
0 0
$20, 0
0 0
$25, 0
0 0
$30, 0
0 0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
Valu
es x
10
-̂4
iPad Income 2012 (MM$/yr)
iPad Income 2012 (MM$/yr)
Minimum $467.14Maximum $27,088.90Mean $8,893.70Std Dev $4,560.12Values 5000
Income analysis shows high potential for losses 1st year, but huge profit upside for 3rd year
iPad 2012 Income Sensitivity Analysis
www.Brioneja.com Twitter: @Brioneja
20.5% 68.3% 11.2%
$5,000 $15,000
$0
$5, 0
00
$10, 0
00
$15, 0
00
$20, 0
00
$25, 0
00
$30, 0
00
0.0
0.2
0.4
0.6
0.8
1.0
iPad Income 2012 (MM$/yr)
iPad Income 2012 (MM$/yr)
Minimum $467.14Maximum $27,088.90Mean $8,893.70Std Dev $4,560.12Values 5000
0.86
0.48
-0.17
-0.05
-0.01
-0.2
0. 0 0. 2
0. 4
0. 6
0. 8
1. 0
Coefficient Value
iPad Warranty Service, $/unit
iPad Indirect Expenses 2012 (MM$/yr)
iPad Cost of Materials, $/unit
Unit Price, $
iPad 2012 Unit Sales (MM)
iPad Income 2012 (MM$/yr)Regression Coefficients
Sensitivity analysis shows where more market research is needed to reduce uncertainty
Actual iPad Sales Figures 2010 – 14.8 MM Units
Original forecast 7.1 MM Units
2011 Jan-Mar - 4.7 MM UnitsApr-June - 9.3 MM UnitsJul-Sep – 11.1 MM Units
○ Apple sold more iPads than Dell sold in all its PCs together (10.6 million)
2011 Projection - ~ 40 MM UnitsOriginal forecast 14.1 MM Units
www.Brioneja.com Twitter: @Brioneja
Use of DDP
DDP has been successfully used by oil and pharmaceutical companies to tackle high uncertainty projects
More companies are adapting the method, but there is still strong resistance to leave Stage Gate behind.Stage Gate can be used after the results of
DDP have defined the project and reduced the level of uncertainty
www.Brioneja.com Twitter: @Brioneja
Summary
DDP combined with Probabilistic Decision Analysis provides discipline and framework to the discovery process but at the same time provides the flexibility necessary for the project to change direction or iterate as needed
www.Brioneja.com Twitter: @Brioneja
Contact Information
[email protected] www.Brioneja.com Twitter: @Brioneja
www.Brioneja.com Twitter: @Brioneja
References
1. Discovery Driven Planning, 1995, http://hbr.org/1995/07/discovery-driven-planning/ar/1
2. The Entrepreneurial Mindset, 2000, http://www.amazon.com/Entrepreneurial-Mindset-Continuously-Opportunity-Uncertainty/dp/0875848346/ref=sr_1_1?ie=UTF8&s=books&qid=1274888243&sr=1-1
3. Innovation Killers, 2008, http://hbr.org/product/innovation-killers-how-financial-tools-destroy-you/an/R0801F-PDF-ENG?Ntt=innoation%2520killers
4. Discovery-Driven Growth, 2009, http://www.amazon.com/Discovery-Driven-Growth-Breakthrough-Process-Opportunity/dp/1591396859/ref=sr_1_3?ie=UTF8&s=books&qid=1274887263&sr=8-3
5. Discovery Driven Planning http://faculty.msb.edu/homak/homahelpsite/webhelp/Discovery_Driven_Planning.htm
www.Brioneja.com Twitter: @Brioneja
Backup Data
www.Brioneja.com Twitter: @Brioneja
iPad Market Share vs. netbooks 2010 16.5%iPad Market Share vs. netbooks 2011 33.5%iPad Market Share vs. netbooks 2012 46.7%
iPad Revenue 2010 (MM$/yr) $4,615iPad Revenue 2011 (MM$/yr) $9,360iPad Revenue 2012 (MM$/yr) $13,065
iPad Total Cost, $/unit $294.77iPad Gross Profit, $/unit $355.23iPad Gross Profit Margin 54.7%
iPad Direct Expenses 2010 (MM$/yr) $2,093iPad Direct Expenses 2011 (MM$/yr) $4,245iPad Direct Expenses 2012 (MM$/yr) $5,925
iPad Total Gross Profit 2010 (MM$/yr) $2,522iPad Total Gross Profit 2011 (MM$/yr) $5,115iPad Total Gross Profit 2012 (MM$/yr) $7,140
iPad Income 2010 (MM$/yr) $422iPad Income 2011 (MM$/yr) $2,615iPad Income 2012 (MM$/yr) $4,440