risk biases siefert

29
1 10/16/2010

Upload: naresh-kumar

Post on 10-Apr-2018

232 views

Category:

Documents


0 download

TRANSCRIPT

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 1/29

110/16/2010

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 2/29

CognitiveCognitive

BiasesBiasesin Decisionin DecisionMakingMaking

William Siefert, M.S.

Consequence

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 3/29

310/16/2010

Acknowledgements

Work based on the research done by

Dr Amos Tversky, PhD

Dr Daniel Kahneman, PhD

³Prospect Theory´ Nobel Prize, 2002

Dr Eric Smith, PhD

Dr Paul Slovic, PhD

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 4/29

410/16/2010

³Fear of harm ought to be

proportional not merely to thegravity of the harm, but also to the

 probability of the event.´

Logic, or the Art of Thinking 

Antoine Arnould, 1662

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 5/29

510/16/2010

5 x 5 Risk ³Cube´

Consequence

5

Original

Current

Objectivevs.

Subjective

data

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 6/29

610/16/2010

Present Situation

Risk matrices are recognized by industryas the best way to:

consistently quantify risks, as part of a

repeatable and quantifiable risk management

processRisk matrices involve human:

Numerical judgment

Calibration ± location, gradation

Rounding, Censoring

Data updatingoften approached with under confidence

often distrusted by decision makers

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 7/29

710/16/2010

Goal

M

ore accurate and repeatable SystemsEngineering Decisions

Confidence in correct assessment of 

probability and value

Avoidance of specific mistakes

Recommended actions

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 8/29

810/16/2010

Heuristics and Biases

Daniel Kahneman won the Nobel Prize inEconomics in 2002 "for having integratedinsights from psychological research intoeconomic science, especially concerning

human judgment and decision-makingunder uncertainty.³

Similarities between

cognitive bias experimentsand the risk matrix axes

show that risk matrices are

susceptible to human

biases.

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 9/29

910/16/2010

Anchoring

First impression dominates all further thought

1-100 wheel of fortune spun

Number of African nations in the United Nations?

Small number, like 12, the subjects underestimated

Large number, like 92, the subjects overestimated

Obviating expert opinion

The analyst holds a circular belief that expertopinion or review is not necessary because no

evidence for the need of expert opinion is present.

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 10/29

1010/16/2010

Heuristics and Biases

Presence of cognitive biases

 ± even in extensive and vetted analyses ±can never be ruled out .

Innate human biases, and exterior circumstances, such as the framing or context of a question, can compromiseestimates, judgments and decisions.

It is important to note that subjects oftenmaintain a strong sense that they are

acting rationally while exhibiting biases.

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 11/29

1110/16/2010

Likelihood

1.F

requency of occurrence is objective,discrete

2. Probability is continuous, fiction

"Humans  judge probabilities poorly" 

[Cosmides and Tooby, 1996]

3. Likelihood is a subjective  judgment(unless mathematical)

'Exposure' by pro ject manager  timeless

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 12/29

1210/16/2010

Case Study

Industry risk matrix data1412 original and current risk points

Time of first entry known

Time of last update known

Cost, Schedule and Technical knownSubject matter not known

Biases revealedLikelihood and consequence judgment

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 13/29

1310/16/2010

Magnitude vs. Reliability [Griffin and Tversky, 1992]

M

agnitude perceived more valid Data with outstanding magnitudes but

poor reliability are likely to be chosen

and used

Observation: risk matrices are

magnitude driven, without regard to

reliability

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 14/29

1410/16/2010

1. Estimation in a Pre-Define Scale Bias

Scale magnitude effects judgment [Schwarz, 1990]

Two questions, random 50% of subjects:

Please estimate the average number of hours you

watch television per week:  ____ ____ __X_ ____ ____ ____

1-4 5-8 9-12 13-16 17-20 More

Please estimate the average number of hours youwatch television per week:

  ____ ____ __X_ ____ ____ ____

1-2 3-4 5-6 7-8 9-10 More

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 15/29

1510/16/2010

Severity Amplifiers

Lack of control

Lack of choice

Lack of trust

Lack of warning

Lack of understanding

Manmade

Newness

Dreadfulness Personalization

Recallability

Imminency

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 16/29

1610/16/2010

Situation assessment

5 x 5R

iskM

atrices seek to increaserisk estimation consistency

Hypothesis: Cognitive Bias

information can help improve the

validity and sensitivity of risk matrix

analysis and other Systems

Engineering analysis

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 17/29

1710/16/2010

Prospect Theory

D

ecision-making described withsubjective assessment of:

Probabilities

Values

and combinations in gambles

Prospect Theory breaks subjective

decision making into:

1) preliminary µscreening¶ stage, probabilities and values are subjectively assessed

2) secondary µevaluation¶ stage

combines the subjective probabilities and utilities

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 18/29

1810/16/2010

Humans judge probabilities poorly*

Small probabilities

overestimated

Large probabilities

under estimated

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 19/29

1910/16/2010

Gains and losses are not equal*

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 20/29

2010/16/2010

Subjective Utility

Values considered from

reference pointestablished by the

subject¶s wealth and

perspectiveFraming

Gains and losses are

subjectively valued1-to-2 ratio.

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 21/29

2110/16/2010

 Implication of Pros pect Theory for the Risk Matrix

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 22/29

2210/16/2010

ANALYSES AND OBSERVATIO NS

OF INITIAL DATA

Impediments for the appearance of cognitivebiases in the industry data:

1) Industry data are granular while the predictions

of Prospect Theory are for continuous data2) Qualitative descriptions of 5 ranges of 

likelihood and consequence

non-linear influence in the placement of risk datum

points

Nevertheless, the evidence of cognitive

biases emerges from the data

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 23/29

2310/16/2010

3. Probability Centering Bias

Likelihoodsare pushed

towardL = 3

Symmetric

to a firstorder 

L kelihood Mar i al Distri utio

of Ori i al Poi ts

0

1

2

3

4

5

6

-1 0 1 2 3 4 5 6

o se ue ce

       L       i       k     e       l       i       h     o     o

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 24/29

2410/16/2010

Open Ri nth

0

10

20

30

40

50

60

1 5 9 13 17 21 25 29 33 37 41 45 49 53 57 61 65 69 73 77

Series2

Linear   Series2)

Guess Why the Spike in New Risks

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 25/29

2510/16/2010

Cognitive Biases in Action

Engineers:

1. Schedule consequenceseffect careers

2. Technicalconsequences effect jobperformance reviews

3. Cost consequences areremote and associatedwith management

Higher cognizance of Biases will be valuable atthe engineering level

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 26/29

2610/16/2010

CO NCLUSIO N

First time that the effects of cognitive biases

have been documented within the risk matrix Clear evidence that probability and value

translations, as likelihood and consequence judgments, are present in industry risk matrix

data Steps 1) the translations were predicted by prospect theory,

2) historical data confirmed predictions

Risk matrices are not objective number grids

Subjective, albeit useful, means to verify that riskitems have received risk-mitigating attention.

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 27/29

2710/16/2010

Suggestions for Cognitive Biases improvement

Long-term, institutional rationality

Team approach

Iterations

Public review Expert review

Biases and errors awareness

Requires cultural changes

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 28/29

2810/16/2010

References

L. Cosmides, and J. Tooby, Are humans good intuitive

statisticians after all? Rethinking some conclusions fromthe literature on judgment under uncertainty, Cognition 58(1996), 1-73.

D. Kahneman, and A. Tversky, Prospect theory: Ananalysis of decision under risk, Econometrica 46(2) (1979),171-185.

Nobel, "The Bank of Sweden Prize in Economic Sciencesin memory of Alfred Nobel 2002," 2002. Retrieved March,2006 from Nobel Foundation:http://nobelprize.org/economics/laureates/2002/index.html.

N. Schwarz, Assessing frequency reports of mundane

behaviors: Contributions of cognitive psychology toquestionaire construction, Review of Personality andSocial Psychology 11 (1990), 98-119.

A. Tversky, and D. Kahneman, Advances in prospecttheory: Cumulative representation of uncertainty, Journal

of R

isk and Uncertainty 5 (1992), 297-323

.

8/8/2019 Risk Biases Siefert

http://slidepdf.com/reader/full/risk-biases-siefert 29/29

2910/16/2010

Comments !

Questions ?