how much risk is too much risk? - boston spinboston-spin.org/slides/039-jan2004-talk.pdf · 2017....
TRANSCRIPT
![Page 1: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/1.jpg)
How Much Risk Is Too Much Risk?
Boston SPINJanuary 20th 2004
The Atlantic Systems Guild, Inc.
![Page 2: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/2.jpg)
“The purpose of risk management is to
make decisions, not to sit around and
admire the risks.” – Rockwell Collins Risk Management Guide
WHAT’S IT ALL ABOUT, ALFIE?WHAT’S IT ALL ABOUT, ALFIE?WHAT’S IT ALL ABOUT, ALFIE?WHAT’S IT ALL ABOUT, ALFIE?WHAT’S IT ALL ABOUT, ALFIE?
![Page 3: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/3.jpg)
“BUT WE CAN’T POSSIBLY MAKE“BUT WE CAN’T POSSIBLY MAKE“BUT WE CAN’T POSSIBLY MAKE“BUT WE CAN’T POSSIBLY MAKE“BUT WE CAN’T POSSIBLY MAKE
THESE DECISIONS . . .”THESE DECISIONS . . .”THESE DECISIONS . . .”THESE DECISIONS . . .”THESE DECISIONS . . .”
“The winning general is the one who can best act on
imperfect information and half formed theories.”
– Napoleon Bonaparte
![Page 4: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/4.jpg)
4. RISK MANAGEMENT:4. RISK MANAGEMENT:4. RISK MANAGEMENT:4. RISK MANAGEMENT:4. RISK MANAGEMENT:
WORLD’S SHORTEST TUTORIAL WORLD’S SHORTEST TUTORIAL WORLD’S SHORTEST TUTORIAL WORLD’S SHORTEST TUTORIAL WORLD’S SHORTEST TUTORIAL
Avoiding the obvious
Explicitly declared uncertainty
Tools of risk management
Risk and value
A test: Did we really do risk management?
![Page 5: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/5.jpg)
You’re blind-sided by a risk that’s happened a
thousand times before.
You have no infrastructure in place to deal
with a risk when it materializes.
You don’t have a useful (early) transition
indicator.
RISK MANAGEMENT ATROCITIESRISK MANAGEMENT ATROCITIESRISK MANAGEMENT ATROCITIESRISK MANAGEMENT ATROCITIESRISK MANAGEMENT ATROCITIES
![Page 6: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/6.jpg)
DENVER INTERNATIONAL AIRPORTDENVER INTERNATIONAL AIRPORTDENVER INTERNATIONAL AIRPORTDENVER INTERNATIONAL AIRPORTDENVER INTERNATIONAL AIRPORT
The automated baggage handling system:
![Page 7: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/7.jpg)
![Page 8: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/8.jpg)
![Page 9: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/9.jpg)
D.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATH
Baggage Handling Software
.
.
Integration testing
.
Acceptance & signoff
Airport opening
1993 1994 1995
![Page 10: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/10.jpg)
D.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATH
Baggage Handling Software
.
.
Integration testing
.
Acceptance & signoff
Airport opening
1993 1994 1995
![Page 11: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/11.jpg)
D.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATH
Baggage Handling Software
.
.
Integration testing
.
Acceptance & signoff
Airport opening
1993 1994 1995
![Page 12: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/12.jpg)
D.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATHD.I.A. PROJECT: CRITICAL PATH
Baggage Handling Software
.
.
Integration testing
.
Acceptance & signoff
Airport opening
1993 1994 1995
![Page 13: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/13.jpg)
![Page 14: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/14.jpg)
You’re blind-sided by a risk that’s happened a
thousand times before.
You have no infrastructure in place to deal
with a risk when it materializes.
You don’t have a useful (early) transition
indicator.
RISK MANAGEMENT ATROCITIESRISK MANAGEMENT ATROCITIESRISK MANAGEMENT ATROCITIESRISK MANAGEMENT ATROCITIESRISK MANAGEMENT ATROCITIES
![Page 15: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/15.jpg)
![Page 16: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/16.jpg)
NOW AVAILABLENOW AVAILABLENOW AVAILABLENOW AVAILABLENOW AVAILABLE
FROM HANSER:FROM HANSER:FROM HANSER:FROM HANSER:FROM HANSER:
www.hanser.de/computer/neu.htm
![Page 17: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/17.jpg)
![Page 18: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/18.jpg)
![Page 19: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/19.jpg)
1. You have zero chance of
delivering before January of
next year.
2. My best guess is you’ll be
done around April 1st . . .
3. but to be at least 50% sure,
you’d better advertise a date
of May 1 or later.
4. To be 100% safe, you’d have to
allow for delivery as late as
end of next year.
![Page 20: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/20.jpg)
![Page 21: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/21.jpg)
![Page 22: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/22.jpg)
![Page 23: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/23.jpg)
RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:
A risk diagram shows explicitly how uncertain we are about
delivery date (or anything else).
![Page 24: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/24.jpg)
RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:
A risk diagram shows explicitly how uncertain we are about
delivery date (or anything else).
RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:
![Page 25: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/25.jpg)
RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:
A risk diagram shows explicitly how uncertain we are about
delivery date (or anything else).
RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:
![Page 26: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/26.jpg)
RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:
A risk diagram shows explicitly how uncertain we are about
delivery date (or anything else).
RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:RISK DIAGRAM:
![Page 27: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/27.jpg)
RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:
Risk
Modeling
ProcessSources of
UncertaintyResultant
Uncertainty
Delivery Date
Size Inflation
Productivity
Variation
etc.
Flaw in size
estimate
![Page 28: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/28.jpg)
RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:
Risk
Modeling
ProcessSources of
UncertaintyResultant
Uncertainty
Delivery Date
Size Inflation
Productivity
Variation
etc.
Flaw in size
estimate
![Page 29: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/29.jpg)
RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:RISK DIAGRAMS IN USE:
Risk
Modeling
ProcessSources of
UncertaintyResultant
Uncertainty
Delivery Date
Size Inflation
Productivity
Variation
etc.
Flaw in size
estimate
![Page 30: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/30.jpg)
SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:
Effort
Size in (for example) Function Points
![Page 31: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/31.jpg)
SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:
Effort
Size in (for example) Function Points
20,000 FP
![Page 32: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/32.jpg)
SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:
Effort
Size in (for example) Function Points
20,000 FP
52 person months
![Page 33: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/33.jpg)
SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:
Effort
Size in (for example) Function Points
20,000 FP
52 person months
“The project will require 52
person-months of effort.”
![Page 34: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/34.jpg)
SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:SENSIBLE PROJECT MANAGEMENT:
Effort
Size in (for example) Function Points
20,000 FP
52 person months
![Page 35: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/35.jpg)
SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:
Effort
Size in (for example) Function Points
20,000 FP
52 person months
![Page 36: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/36.jpg)
SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:
Delivery Date
6/2000 11/2000
9/2000Effort
40 52 70 (Person Months)
“The project will take this much effort.”
![Page 37: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/37.jpg)
40 52 70
Effort
(Person Months)
SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:SENSIBLE RISK MANAGEMENT:
Delivery Date
6/2000 11/2000
9/2000
“The project will take this much time.”
![Page 38: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/38.jpg)
![Page 39: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/39.jpg)
How much risk is too much risk?
![Page 40: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/40.jpg)
INSIGHTS:INSIGHTS:INSIGHTS:INSIGHTS:INSIGHTS:
surprise about where the major risks lie
surprise about the value of low risk projects
surprise about risk quantification
surprise about precision
surprise about aggressively scheduled projects
![Page 41: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/41.jpg)
WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?
![Page 42: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/42.jpg)
WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?
![Page 43: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/43.jpg)
WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?WHERE IS THE BIGGEST RISK?
![Page 44: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/44.jpg)
THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:
The biggest risk an organization faces is lost opportunity, the failure
to choose the right projects.
![Page 45: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/45.jpg)
THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:
The biggest risk an organization faces is lost opportunity, the failure
to choose the right projects.
So . . .
Value is every bit as important as cost (the plusses matter
as much as the minuses).
![Page 46: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/46.jpg)
THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:THE FIRST SURPRISE:
The biggest risk an organization faces is lost opportunity, the failure
to choose the right projects.
So . . .
Value is every bit as important as cost (the plusses matter
as much as the minuses).
Your process for deciding which projects to do is more
important than your process for how to do them.
![Page 47: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/47.jpg)
THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:
![Page 48: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/48.jpg)
. . . so we think we can
have this for you in
seven months . . .
THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:
![Page 49: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/49.jpg)
. . . so we think we can
have this for you in
seven months . . .
THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:THE KEY PROJECT NEGOTIATION:
If we really told him what we think (15-20 months),he’d never do the project.
![Page 50: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/50.jpg)
THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:
The real reason we need to do risk management is not so much to
survive our risks, but to enable risk-taking.
![Page 51: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/51.jpg)
THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:
The real reason we need to do risk management is not so much to
survive our risks, but to enable risk-taking.
Conversely:
A failure to manage risks ensures that no one will take any
but the most minor risks.
![Page 52: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/52.jpg)
THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:THE SECOND SURPRISE:
The real reason we need to do risk management is not so much to
survive our risks, but to enable risk-taking.
Conversely:
A failure to manage risks ensures that no one will take any
but the most minor risks.
Without credible risk management, it is impossible to pursue
meaningful value.
![Page 53: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/53.jpg)
THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:
The only reason to quantify cost (schedule and budget) is to have
something to compare to your quantification of benefit.
![Page 54: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/54.jpg)
THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:
The only reason to quantify cost (schedule and budget) is to have
something to compare to your quantification of benefit.
A failure to quantify benefit assures that:
there is perfect one-way accountability; the project team is
accountable but the client is completely unaccountable.
![Page 55: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/55.jpg)
THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:THE THIRD SURPRISE:
The only reason to quantify cost (schedule and budget) is to have
something to compare to your quantification of benefit.
A failure to quantify benefit assures that:
there is perfect one-way accountability; the project team is
accountable but the client is completely unaccountable.
there is no way to assure that the high value projects get
done (prioritization is a charade).
![Page 56: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/56.jpg)
THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:
There is no sense making cost quantification more precise than value
quantification.
![Page 57: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/57.jpg)
THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:
There is no sense making cost quantification more precise than value
quantification.
So if all the user is willing to say about benefit is:
“We gotta have it!”
![Page 58: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/58.jpg)
THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:THE FOURTH SURPRISE:
There is no sense making cost quantification more precise than value
quantification.
So if all the user is willing to say about benefit is:
“We gotta have it!”
then it’s perfectly reasonable to quantify budget and schedule only
to this extent:
“It’s gonna cost a lot,”
and
“It will be done when it’s done.”
![Page 59: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/59.jpg)
![Page 60: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/60.jpg)
I’m sorry, Tom, but I simply
cannot tolerate delivery any later
than December 31, 2003.
![Page 61: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/61.jpg)
I’m sorry, Tom, but I simply
cannot tolerate delivery any later
than December 31, 2003.
If I limit them to this year they can’t possibly spend more thanI’m willing to pay.
![Page 62: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/62.jpg)
THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:
An aggressive delivery date is often driven by a cost containment
motive, not an urgent date motive.
![Page 63: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/63.jpg)
THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:
An aggressive delivery date is often driven by a cost containment
motive, not an urgent date motive.
The more aggressive the schedule, the more likely it is that
the product is of low value.
![Page 64: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/64.jpg)
THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:THE FIFTH SURPRISE:
An aggressive delivery date is often driven by a cost containment
motive, not an urgent date motive.
The more aggressive the schedule, the more likely it is that
the product is of low value.
An inescapable corollary:
Starting a project late is a
worse sin than finishing it late.
![Page 65: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/65.jpg)
INSIGHTS:INSIGHTS:INSIGHTS:INSIGHTS:INSIGHTS:
surprise about where the major risks lie
surprise about the value of low risk projects
surprise about risk quantification
surprise about precision
surprise about aggressively scheduled projects
![Page 66: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/66.jpg)
I think we’re doing risk management.
Um . . . we are, aren’t we?
![Page 67: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/67.jpg)
THE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISK
MANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TEST
(in six parts):
![Page 68: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/68.jpg)
1. Is there a census of risks with at least 10-20 risks on it?
THE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISK
MANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TEST
(in six parts):
![Page 69: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/69.jpg)
1. Is there a census of risks with at least 10-20 risks on it?
2. Is each risk quantified as to probability and cost and
schedule impact?
THE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISK
MANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TEST
(in six parts):
![Page 70: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/70.jpg)
1. Is there a census of risks with at least 10-20 risks on it?
2. Is each risk quantified as to probability and cost and
schedule impact?
3. Is there at least one early transition indicator associated
with each risk?
THE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISK
MANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TESTMANAGEMENT” TEST
(in six parts):
![Page 71: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/71.jpg)
THE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISK
MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)
4. Does the census include the core risks indicated by
past industry experience?
![Page 72: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/72.jpg)
THE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISK
MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)
4. Does the census include the core risks indicated by
past industry experience?
5. Are risk diagrams used widely to specify both the
causal risks as well as the net result (schedule and
cost) risks?
![Page 73: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/73.jpg)
THE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISKTHE “ARE WE REALLY DOING RISK
MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)MANAGEMENT” TEST (CONTINUED)
4. Does the census include the core risks indicated by
past industry experience?
5. Are risk diagrams used widely to specify both the
causal risks as well as the net result (schedule and
cost) risks?
6. Is the scheduled delivery date significantly different
from the best-case scenario?
![Page 74: How Much Risk Is Too Much Risk? - Boston SPINboston-spin.org/slides/039-Jan2004-talk.pdf · 2017. 9. 12. · How Much Risk Is Too Much Risk? Boston SPIN January 20th 2004 Tim Lister](https://reader036.vdocument.in/reader036/viewer/2022081620/610ed6f391365d71ca0d38ba/html5/thumbnails/74.jpg)