cognitive and motivational biases in decision and risk analysis...prof gilberto montibeller bor...

Prof Gilberto Montibeller BOR Summer School 2019 1

Cognitive and Motivational Biases in

Decision and Risk Analysis

Gilberto Montibeller

Professor of Management Science

Head of the Management Science and Operations Group

School of Business and Economics

Loughborough University


Schedule Talk

• Approaches to Decision Making Research

• The practice of Decision and Risk Analysis (DRA)

• Cognitive Biases in DRA Modelling

• Motivational Biases in DRA Modelling

• Debiasing judgments in DRA Modelling

• Group Biases in Modelling


Approaches to Decision Making Research

DecisionMaking

Decision Outcomes

Objectives & Preferences

Uncertainties & Risks

Co

nte

nt

kno

wle

dge

Options

Decision Process

Problem Frame & Structure


Approaches to Decision Making Research

• Normative: how should fully rational decision makers decide? (Decision Theory)

• Descriptive: how do real decision makers decide? (Behavioural Decision Science)

• Prescriptive: how can real decision makers decide better?(Decision Analysis)


The Prescriptive-Descriptive Split

in Decision Analysis

• All research prior to the 1950s (from

Bernoulli to Savage) was prescriptive

• Some researchers criticized the DA

principles of descriptive grounds (Ellsberg,

Allais) already in the 1950s

• Edwards laid the foundation of scientific

descriptive work, but with a prescriptive

agenda


The Allais ParadoxEU(X) = ∑pi ui(X)

Let U($0) = 0; U ($5 million) = 1

• Decision 1:EU(A) = U($1 million)

EU(B) = 0.10 U($5 million) + 0.89 U($1 million) + 0.01 U($0)

EU(B) = 0.10 + 0.89 U($1 million)

As A is preferred to B: EU(A) > EU(B) => U($1 million) > 0.10 + 0.89 U($1 million)

Thus: U($1 million) > 0.91

• Decision 2:EU(C) = 0.11 U($1 million) + 0.89 U($0) = 0.11 U($1 million)

EU(D) = 0.10 U ($5 million) + 0.90 U($0) = 0.10

As D is preferred to C: EU(D) > EU(C) => 0.10 > 0.11 U($1 million)

Thus U($1 million) < 0.91, therefore a paradox.

$1 million

$5 million

$1 million

$0

A

B

0.10

0.89

0.01

Decision 1

$1 million

$5 million

$0

C

D

0.89

0.10

Decision 2

$0

0.11

0.90

Normative models are not descriptively valid!


Imagine that the Netherlands is preparing for the outbreak of an unusual avian flu outbreak, which is expected to kill 600 people. Two alternative programmes have been proposed.

Problem Setting:


Gains-Losses Framing• Framing in terms of gains may elicit risk-averse

behaviour (T1 preferred to T2).

• Framing in terms of losses may elicit risk-seeking behaviour (T4 preferred to T3, a preference reversal from T1 preferred to T2).

Ref: Levin, I. P., Schneider, S. L., & Gaeth, G. J. (1998). All frames are not created equal: a

typology and critical analysis of framing effects. Organizational Behavior and Human Decision

Processes, 76(2), 149-188.

T1

T2

+200 lives

+600 lives

0 lives

1/3

2/3

T3

T4

-400 lives

-600 lives

0 lives

2/3

1/3


Prospect Theory

• People evaluate values as gains or losses relative to some reference level (or status quo)

• People are more risk averse for gains than for losses, and this is captured by the steeper curve in losses than gains.


The Prescriptive-Descriptive Split of the 1970s

• Prescriptive work since 1960:

• 1960’s: Experimental applications of DA

• 1970’s: Multiattribute utility theory and influence

diagrams

• 1980’s: Major applications

• 1990’s: Computerization

• 2000 and beyond: portfolio decision analysis, utility

dependencies (e.g. copulas), etc.


The Prescriptive-Descriptive Split of the 1970s

• Descriptive work

• 1950s and 60s: Early violations of SEU (Allais,

Ellsberg)

• 1970s: Probability Biases and Heuristics (cognitive

illusion paradigm)

• 1980s: Utility biases and Prospect Theory

• 1990s: Generalized expected utility theories and

experiments

• 2000 and beyond: fine tuning Prospect Theory,

heuristics, etc.


Facilitator

Analyst

Group

Process

Space

Modelling

Space

Group

Model

Pro

vid

es in

form

atio

n

Gen

erat

es r

esp

on

ses

Decision Analysis

Informs

model

building

Represents

problem

situation

Facilitation

Methods

Informs

group

facilitation

process

Group Outcomes (e.g. commitment to action, learning)

Model Outcomes (e.g. ranking of alternatives)

Group

discussion

Source:

Franco and Montibeller

(2010). “Facilitated Modelling

in Operational Research.”

European Journal of

Operational Research 205,

no. 3: 489–500.


Value Tree – FAO project


Attributes – FAO projectCriteria Sub-Criteria Attribute

C1: Impact on

International Trade

- Export value in US$ billions/year

C2: Burden of

Disease

- Total Disability-Adjusted Life Years (DALYs) in outbreak cases from

1990 on

C3: Vulnerabilities

due to Food

Consumption

C3.1: Average Serving Average g/day

C3.2: Proportion

Vulnerable Consumers

Proportion (0-100%) consumed by vulnerable groups (toddlers and

elderly)

C3.3: Potential for

Consumer Mishandling

Proportion (0-100%) of LMF products in a given category with an

increased risk as a result of mishandling/poor practices at any time

between final retail and consumption.

C4: Vulnerabilities

due to Food

Production

C4.1: Increased Risk of

Contamination

Proportion (0-100%) of LMF products in a given category with an

increased risk of contamination post kill step.

C4.2: Proportion

without Kill Step

Proportion (0-100%) of LMF products in a given category without a

kill step prior to retail and distribution.

C4.3: Prevalence of

Pathogen

Probability that a LMF is contaminated at a level with any pathogens

with the potential to cause illness in consumers.


Multi-Attribute Value Analysis

𝑽 𝒂 =

𝒊=𝟏

𝑵

𝒘𝒊𝒗𝒊 𝒙𝒊(𝒂

𝒘𝒊 = weight of the i-th attribute (i = 1, 2, …, N)

𝒗𝒊 = partial value of the i-th attribute

The overall value of policy alternative a is given by:

Where:

𝒙𝒊 = performance alternative a on i-th attribute

𝒗𝒊 𝒙∗ = 𝟎 ∀ i ; where 𝒙∗ is the lowest level of the i-th attribute

𝒗𝒊 𝒙∗ = 𝟏𝟎𝟎 ∀ i; where 𝒙∗ is the highest level

of the i-th attribute

𝒊=𝟏

𝑵

𝒘𝒊 = 𝟏

weak-difference independence

condition


Valuation – FAO projectC2: Burden of Disease

Code Category Name Total DALYs in

outbreak cases

from 1990 on

Normalised

Impact (v2)

[Value]

Cat 1 Cereals and Grains 72.53 45.9

Cat 2 Confections and Snacks 60.26 35.4

Cat 3 Dried Fruits and Vegetables 32.78 12.2

Cat 4 Dried Protein Products 136.44 100.0

Cat 5 Nuts and Nut Products 118.51 84.8

Cat 6 Seeds for Consumption 18.42 0.0

Cat 7 Spices, Dried Herb and Tea 80.71 52.8


Overall Value – FAO project


Judgements in Modelling Values

O

ONO2O1

x1

g1

x2

g2 gN

xN

w1 w2 wN

Identifying objectives

Defining

attributes

Eliciting

value

functions

Eliciting weights

...


Supporting a Commercial Law Firm

in deciding the strategy for a commercial dispute

Decision Making Under Uncertainty


Best strategy for the dispute trial


Judgements in Modelling Uncertainty

U1 U2 UM...

Ut

Eliciting

distributions

d1 d2 dM

dTe

Aggregating

distributions

Identifying

Variables


Judgments in ModellingChoices

D

C1

C2

P1,2

P2,1

P2,2

P2, k2

a1

a2

P1,1

P1,k1

CZ

PZ,1

PZ,2

PZ, kZ

aZ

...

...

...

X1,1

Identifying

alternatives

Identifying

uncertainties

X1, k1

XZ, kZ

Eliciting

Probabilities

X1,2

X2, 1

X2, 2

X2, k2...

XZ, 1

XZ, 2

Estimating

Consequences


Two Ways Decision Analysts Deal with Biases

• The easy way

• Biases exist and are harmful

• Decision analysis helps people overcome these

biases

• The hard way

• Some biases can occur in the decision analysis

process whenever a judgment is needed in

the model and may distort the analysis

• Need to understand and correct for these biases

in decision analysis


More vs Less Relevant Biases

More Relevant Biases

• They occur in the tasks of eliciting inputs into a decision and risk analysis (DRA) from experts and decision makers.

• Thus they can significantly distort the results of an analysis.

Less Relevant Biases

• They do not occur or can easily be avoided in the usual tasks of eliciting inputs for DRA


Relevant Cognitive Biases

• Anchoring

• Availability

• Certainty effect

• Equalizing bias

• Gain-loss bias

• Myopic problem

representation

• Omitted variable bias

• Overconfidence

• Scaling biases

• Splitting bias

• Proxy bias

• Range insensitivity

bias

Cognitive biases are distortions of judgments that violate

a normative rules of probability or expected utility


Identifying Objectives

O

ONO2O1

x1

g1

x2

g2 gN

xN

w1 w2wN

Identifying objectives

Defining

attributes

Eliciting

value/utility

functions

Eliciting weights

...

x1(a)

Estimating

impacts


Debiasing

• Older experimental literature shows low efficacy

• Recent literature is more optimistic

• Decision analysts have developed many (mostly

untested) best practices, which we reviewed:

• Prompting

• Challenging

• Counterfactuals

• Hypothetical bets

• Less bias prone techniques

• Involving multiple experts or stakeholders


Identifying Objectives – Biases & Debiasing

Biases:

• Availability bias (C)

• Myopic problem representation (C)

• Omitted variable bias (C)

Debiasing:

• Providing categories

• Prompting for more objectives

• Stimulating creativity

• Employing problem structuring methods


Identifying Objectives – Debiasing Tools Building a Group Causal Map in the

FAO project


• Estimate ranges before central tendency

• Stretch ranges to account for lack of knowledge

• Avoid overconfidence and anchoring biases

• Use multiple individual experts with different perspective

Debiasing: Forecasting Trends


Motivational Biases

• Affect-Influenced Bias

• Confirmation bias

• Undesirability of a negative event or

outcome (precautionary thinking,

pessimism)

• Desirability of a positive event or

outcome (wishful thinking, optimism)

• Desirability of options or choices

Motivational biases are distortions of judgments because

of desires for specific outcomes, events, or actions


Mapping Biases -Modeling Choices

62

D

C1

C2

P1,2

P2,1

P2,2

P2, k2

a1

a2

P1,1

P1,k1

CZ

PZ,1

PZ,2

PZ, kZ

aZ

...

...

...

X1,1

Identifying

alternatives

Identifying

uncertainties

X1, k1

XZ, kZ

Eliciting

Probabilities

X1,2

X2, 1

X2, 2

X2, k2...

XZ, 1

XZ, 2

Estimating

Consequences


Mapping Biases

63

D

C1

C2

P1,2

P2,1

P2,2

P2, k2

a1

a2

P1,1

P1,k1

CZ

PZ,1

PZ,2

PZ, kZ

aZ

...

...

...

X1,1

X1, k1

XZ, kZ

Eliciting

Probabilities

X1,2

X2, 1

X2, 2

X2, k2...

XZ, 1

XZ, 2

• Anchoring bias (C)

• Availability bias (C)

• Equalizing bias (C)

• Gain-loss bias (C)

• Overconfidence bias (C)

• Splitting bias (C)

• Affect-Influenced (M)

• Confirmation bias (M)

• Desirability biases (M)


Current Research Project: Debiasing• Existing literature focused on demonstrate bias

(e.g. overconfidence)

• Few attempts of assessing the effectiveness of debiasing tools in controlled experiments

• No previous attempt of assessing the effectiveness of sophisticated debiasing tools employed by decision analysts in practice

• Ongoing research: testing effectiveness of debiasing tools


Benefits of engaging with groups

• Judgments considering future events:

oContent: Increase of accuracy, pooling of information and perspectives, error checking, motivation gains

oSocial goals: procedural fairness and satisfaction/enjoyment


Benefits of engaging with groups

• Judgments considering group preferences:

oContent: Pooling of information and perspectives, error checking, motivation gains

oSocial goals: procedural fairness, satisfaction/enjoyment, sense of a common purpose, agreement on the way forward


Group biases

• Group biases may affect the quality of judgments (about preferences and future events)

• Groups may increase or attenuate individual biases depending on:

oType of group decision/judgment process

oType and strength of the bias

o Individual preferences among group members


Group biases

• False consensus (but no direct evidence on decision making

or group judgments)

• Groupthink (evidence on incomplete search for alternatives,

consideration of too few objectives, limited search for evidence)

• Group polarisation (evidence on group risk attitude)

• Group escalation of commitment (evidence it

increases sunk-cost bias, impacts on risk-seeking attitude)

• Group overconfidence (evidence it may increase

individual overconfidence, affects all the judgments impacted by individual overconfidence)


Group biases

• False consensus

• Groupthink

• Group polarisation

• Group escalation of commitment

• Group overconfidence

Evidence bias

impact on

modelling task?


Debiasing strategies against group biases

• Engaging with multiple experts with alternative points of view

• Encouraging distinct perspectives

• Using structured elicitation procedures

• Providing effective facilitation

Evidence about effectiveness on

specific group bias/modelling task?

cognitive and motivational biases in decision and risk analysis...prof gilberto montibeller bor...

Documents