using risk analysis and simulation in project management
TRANSCRIPT
Using Risk Analysis and Simulation in Project
Management Improve Project Plans, Budgets & Schedules
Mike Tulkoff
◦ Overruns the norm: 40 to 200%1
◦ E&Y 2009 Survey 3
96% of managers want to improve risk mgmt. 46% think spending more $ on risk mgmt leads to competitive advantage
Projects are notoriously late & over budget!
Projects in Budget
Projects on Time
Projects met Deliverables
Project Failures
0
10
20
30
40
50
60
70
KPMG 2012 Survey2
1. Morris, P., & Hough, G. (1987). The anatomy of major projects: A study of the reality of project management. Chichester: Wiley.2. https://www.kpmg.com/NZ/en/IssuesAndInsights/ArticlesPublications/Documents/KPMG-Project-Management-Survey-2013.pdf3. https://www.yumpu.com/en/document/view/27686141/the-future-of-risk-protecting-and-enabling-performance-directors-
Project Success Factors Brief history of project management Basic Risk Management Review of Project Management Methods Simulation and Monte Carlo Example Project with Simulation Conclusions Speaker Bio
Agenda
Key Project Success Factors
Adoption & consistent use of
project management methodology
Dedicated project manager
Aligning project goals with business & customer needs
Scope management Effective RISK MANAGEMENT
Effective use of multi-point estimation
4. Meredith, J., & Mantel, S. (1995). Project management: A managerial approach (3rd ed.). New York: Wiley.5. Wilson, J. M. (2003). Gantt charts: A centenary appreciation. European Journal of Operational Research, 149(2), 430-437.6. Moder, J. J., & Phillips, C. R. (1970). Project Management with CPM and PERT (2nd ed.).
A Brief History
Project schedule is most important
tool.4
Gantt invented Gantt chart early 20th century•Earliest network graph•Adapted for project mgmt 1920s.5
1957 DuPont invented Critical Path Management (CPM)•Optimal tradeoff between time and cost
1958 – U.S. Navy and Booz, Allen, and Hamilton invented Program Evaluation Review Technique (PERT) for Polaris Missile Project.•Decreased costs 66% and durations 33%.6
Basic Risk ManagementIdentify Risks, perform risk analysis & plan risk responses (PMI PMBOK 5).
Use Identification Tools
• Documentation, project WBS, SWOT analysis, cross-functional reviews (e.g. legal, financial)
Use Risk Register
• Matrix of identified risks, categories, likelihood, mitigation, owner.
Risk Management is an iterative approach – feedback into the project plan
Simulation & prototype have highest correlations to successful risk mitigation.7
7, Raz, T., & Michael, E. (2001). Use and benefits of tools for project risk management. International Journal of Project Management, 19(1)
Risk Register helps characterize known risks and potential “black swans”.
Probability and project impact Mitigation plans
Identify Risk
Critical Path ◦ Longest chain of dependent steps in a project◦ Determines the time it takes to finish overall
project◦ Any delay along critical path delays whole project
Review of PM Methods - CPM
Single point estimates are error prone & not conducive to risk management
PERT durations/costs use 3-point estimates ◦ a = best case (5% chance or better)◦ m = most likely (90%)◦ b = worst case (5%)
PERT uses a Beta Distribution8
◦ Mean = (a+4m+b) / 6 “Modern” formula is .63 * m + .185*(a+b)corrects for lack of true min and max
◦ Variance = (b-a/6)2
“Modern” formula is (b-a/3.25)2
◦ Standard Deviation = b-a/6 “Modern” is b-a/3.25
8. Source of PERT information: Anderson, M.A and Anderson E.G. (2015) lecture materials from the course Technology Enterprise Design and Implementation at the University of Texas at Austin.
Review of Program Evaluation Review Technique (PERT)
You still need to use a risk register & simulation!
PERTCont’d
Can now calculate overall project probabilities
Expected project cost = 𝞢 activity costsProject cost variance = 𝞢 activity variancesProject duration/cost is normally distributedEstimate the 90% or 95% likely completion time and cost (within a range).
Limitations Garbage in, garbage outDoes not account for true uncertainty
Best tool to analyze uncertainty is simulation!
Why Simulation?Project plans
have variance risk due to
imprecise or overly optimistic
estimates
Risk also comes from predictable & unpredictable events• Have a
disproportionate effect on the project duration & cost
Management of uncertainty is key
Used finance, business, physics, engineering, biology, project management, etc.
Tools used in this presentation include◦ @RISK (Palisades Corp)◦ Project & Excel (Microsoft Corp)
Monte Carlo Process
Monte Carlo Simulation
Model uncertain inputs as
distributions
Generate pseudo random numbers
each iteration
Deterministic
computation
Aggregate output -
probability density
A Trivial Example◦ Roll two 6-sided die ◦ Output is sum of dice◦ After 1000 iterations, results show probabilities
Monte Carlo
Die 1=RiskDiscrete({1,2,3,4,5,6},{0.166666667,0.166666667,0.166666667,0.166666667,0.166666667,0.166666667},RiskStatic(1))
Die 2=RiskDiscrete({1,2,3,4,5,6},{0.166666667,0.166666667,0.166666667,0.166666667,0.166666667,0.166666667},RiskStatic(1))
Sum =RiskOutput("Sum")+SUM(B2:B3)
Sample Project Simulation
Background schematic image courtesy of InvenSense Proj Plan courtesy of Kibbe, Pfau, Reber, Shields, Tulkoff (2015)
Step 1 : Create Project Plan
Create a Work Breakdown Structure (WBS)
Enter tasks into PM tool (e.g. MS Project)
Assign resources & dependencies
Use most likely or optimistic durations for now • Will deal with durations again later
1. Cannot control vendor performance 2. Not accounting for Engineering re-work is a major reason
projects fail.9
3. Task duration variance (use PERT)4. External risks have great effect
◦ In this simulation, they delay the project start date.
Step 2: Identify Risks
Event Probability DelayNo Delay .45 0 daysProblem with funding
.20 30 days
Hiring problems .15 15 daysFreedom to operate issue
.10 90 days
Technology prototype issues
.10 45 days
9. Reichelt, K., & Lyneis, J. (1999). The Dynamics of Project Performance: Benchmarking the Drivers of Cost and Schedule Overrun.European Management Journal, 17(2), 135.
Step 3: Import into Excel @RISKClick project on @RISK ribbon and import MPP file. Note that it draws a Gantt chart. Inputs and Outputs are tied to Excel cells.
Create appropriate input distributions. ◦ There is no “right” answer. ◦ Do what makes sense.
The uncertain inputs that we found can be modeled as a discrete probability distribution with initial duration tied to probability:
Step 4: Model Inputs
Vendor risk (Mold creation task in this example) can be modeled as a Uniform distribution between two bounds.
All values are equal probability. Note this is to illustrate the distribution
◦ would use something more discrete here.
Engineering re-work can be modeled as a normal distribution with some right skew.
Task can be accounted for & simulated as it is uncertain how extensive this will be going into the project.
Task variance can be modeled using PERT (or with Triangle or Trigen distributions)
PERT is a natural fit for project tasks.
Step 5: Add Outputs, Run simulation• Outputs are tied to cells with data that varies
based on varying inputs• Flexibility to also tie values to additional Excel
data, formulas, conditionals• May run simulation using multiple scenarios &
perform sensitivity analysis• Should run at least 1000 iterations
• 10k is better• Directly integrated with Project – uses Project’s
scheduling engine each iteration
Total Task Duration
Critical Path Duration
Step 6: Analyze Output
End Date
Cost
Total duration Tornado
Total Cost Tornado
ConclusionsProjects have inherent task variation risk as well as risk from uncertainty
Projects can be more successful by using a consistent methodology, using multi-point estimates, accounting for re-work, and analyzing/managing risk
Simulation including Monte Carlo is a powerful tool to deal with uncertainty
Risk management is an iterative process
Mike Tulkoff is a Software Engineer with over twenty years of delivering Enterprise Computing solutions. He has spent his career building great products that satisfy market needs and has had technical and managerial roles at both large, global companies and small start-ups. Mike has 12 U.S. patents and holds an MS in Technology Commercialization from the University of Texas at Austin McCombs School of Business and a BS in Computer Science from Georgia Institute of Technology.
Mike is an open networker on LinkedIn. Please feel free to contact him with additional questions, discussion, or consulting inquiries.
About the Presenter