spe distinguished lecturer program · expert reliability & uncertainty steve begg, asp,...

Steve Begg, ASP, University AdelaideExpert Reliability & Uncertainty

1SPE DL 2010

SPE Distinguished Lecturer Program

Primary funding is provided by

SThe SPE Foundation through member donations and a contribution from Offshore Europe

The Society is grateful to those companies that allow their professionals to serve as lecturers

2010-2011 SPE Distinguished Lecture Experts and Uncertainty Steve Begg

professionals to serve as lecturers

Additional support provided by AIME

Society of Petroleum Engineers Distinguished Lecturer Programwww.spe.org/dl

Reliability of Expert Judgments and Uncertainty

SPE Distinguished Lecture, 2010-2011

Judgments and Uncertainty Assessments

Steve Begg

Australian School of Petroleum, University of Adelaide

Centre for Improved Business Performance


2SPE DL 2010

Reliability of Expert Judgments and Uncertainty

SPE Distinguished Lecture, 2010-2011

Judgments and Uncertainty Assessments

Steve Begg

Australian School of Petroleum, University of Adelaide“All business proceeds on beliefs, or j d t f b biliti d t Centre for Improved Business Performancejudgments of probabilities, and not on certainties".

Charles W. Eliot

Outline

• The Nature of Uncertainty

P l P b bilit d• People, Probability and Judgment

• Performance of Industry Experts

• Conclusions


Conclusions


3SPE DL 2010

Industry Performance

• Comments & Observations– “Every one of our 10 most important projects failed to

generate the desired return ” (Super Major)generate the desired return. (Super Major)

– “The actual performance of our key assets wasn’t even within the P1 to P99 range.” (Large Independent)

– “I want your guarantee that we will not spend more than the P50 on this project!” (CEO to Manager)

– “a decade of unprofitable growth”; vast majority of


– a decade of unprofitable growth ; vast majority of projects take longer, cost more and produce less than predicted; 1-in-8 of major offshore are “disasters” (IPA)

Taking on a Cult of Mediocrity

• “The last 10 years might be called ‘a decade of unprofitable growth’ for many upstream companies.”p

Ed Merrow, Independent Project Analysis (IPA)

– Based on the analysis of more than 1000 E&P projects:2/3 offshore, average $1Million – $3Billion

– One in eight of all major offshore developments in the last decade falls into the ‘disaster’ category.


failed on two out of three metrics: >40% cost growth, >40% time slippage, produced < 50% than 1st year plan

Average CapEx for these is $670MM

– Record even worse for mega-projectsCapEx of $1 billion or more

Source: UPSTREAM, 23 May 2003


4SPE DL 2010

Taking on a Cult of Mediocrity

-10%

B t ti

Asset costgrowth

BE

TT

ER

-10%

Schedule slip

9 %

First Year Plan

Goal

-5%

0%

5%

10%

15%

Best practice

Industry average

-5%

0%

5%

10%

15%

Best practice

Industry average

20%

95%

75%

Best practice

Industry average

55%The bigger and more important a project

GoalGoal – Why?


30%

25%

20%

Disasters

WO

RS

E

40%

35%

25%

Disasters

30%

35%

Disasters

Ref: Merrow, Ed: Taking on a Cult of Mediocrity, Upstream, 23 May 2003

The bigger and more important a project gets, the more likely it ends up in the

“disaster” category.

The fundamental problem: Industry performance not living up to expectations, or possibilities

Uncertainty



5SPE DL 2010

The fundamental problem: Industry performance not living up to expectations, or possibilities

• People tend to grossly under-estimate uncertainty– number of uncertain factors and the magnitude of

uncertaintyu ce ta ty

– complexity of the relationships between them and therefore un-anticipated non-intuitive outcomes)

• Better decision-making requires accurate (unbiased & appropriate range) uncertainty assessment

– Reduce uncertainty only IF it can change a decision AND expected benefit of reduction is less than its cost


AND expected benefit of reduction is less than its cost

• Naive understanding of economics– NPV “rule” : uncertainty (and/or delay) = Value Loss

– Focus on mitigating downside risk at expense of capturing upside opportunities

Uncertainties and dependencies matter: they are everywhere in the evaluation “system” ……

Prices

Taxes

Drilling CapEx

Export

EconomicsAsset A

OOIP Model

Petro-physics

PredictedProduction

Royalties/PSCSeismic

Geology

OpExProcessing


. . .Production Data

ReservoirSimulation Portfolio

EconomicsAsset 1

EconomicsAsset n

Decline

ProductionAllocation


6SPE DL 2010

…. and we spend a lot of money without really knowing which ones matter

Gross Rock Volume

UncertaintyParameter Impact on NPV

Buy more seismic

Action

Average Porosity

Recovery Factor

Saturation

Oil Price

Facilities

Take more core

Build simulator model

y

Different rock model

Hedge with futures

Cheaper Steel Supply

"There is nothing so inefficient as very efficiently doing the wrong things".

Peter Drucker


Rig Cost

Fiscal Terms

Net:Gross

Continuity

Renegotiate contract

Fire lawyers

More gamma logs

Survey Analogues

Flaw of Using Averages (after Savage)

NCF


NCF


7SPE DL 2010

Averages don’t always work

• For non-linear processes,– reservoir simulation– volumetrics with cut-offs

Mean = 10 Mean = 5

– development alternatives

even if only a single, “best” estimate is required, we still need to use complete range of inputs - cannot use an average input

Model

Y = X2 Z2/

3 21 1 8x z

Si l ti R lt


• Also, P10 (P90) results are NOT given by taking P10 (P90) inputs and running them through the model

ResultY = 4

Y

Simulation Result

0 30

True Mean~ 7.8

The Flaw of Averages:

• Unless model (f) is linear (“model” is a calculation)

average of f( x y z )

average of f( x,y,z,…. )

f( average(x), average(y), average(z),….)


where x,y,z,… are uncertain quantities, and

f is a calculation using x, y, z ....


8SPE DL 2010

Two major fields of study in decision-makingunder uncertainty

• how decision makers

PrescriptiveDescriptive ≠• how decision makers

should choose

• Best decisions given information, alternatives, objectives & preferences

• principle of maximizing expected value (or utility)

• how decision makers actually choose

• psychology of judgment and decision making

• behavioral studies and observations

• heuristics & biases


expected value (or utility)

• traditional decision theory (decision trees, MC Simulation, VoI)

• heuristics & biases

• Sub-optimal, inconsistent illogical decisions

Psychology Mgmt. Science

Technical Work in the context of Decisions and Uncertainty

The role of a Petroleum Geoscientist or Engineer is to support decision-making

• Whether you, your group or company explicitly recognize it or not – technical work is fundamentally about uncertainty assessment for the purpose of making decisions

• If you have a “make the best possible prediction” focus, there is no stopping rule

– you can always reduce uncertainty a bit more (more data, more time, more analysis)


more analysis)

• A decision-driven focus gives a trivially simple stopping rule– Stop when further analysis doesn’t change the decision!!

– From a decision-making perspective we only need to find which option has the greatest value – we don’t (usually) need a precise (= little uncertainty) estimate of that value


9SPE DL 2010

The solution

Betterperformance

BetterDecisions

performance


Better uncertaintyassessment

Results of survey on “reserves” uncertainty-related word meanings


Source: McLane et al, AAPG Bull v92 #10


10SPE DL 2010

Probability: The Language of Uncertainty

• Classical (Theoretical)Number of outcomes representing the occurrence of an event

Total number of possible outcomes

30 d b ll d 70 b ll i b P(R d) 30%– e.g. 30 red balls and 70 green balls in a bag. P(Red) = 30%

• Relative Frequency – Proportion of times an event occurs in the long run– Can be ESTIMATED from sample data ASSUMING identical events

e.g. 15 out of 20 wells drilled were dry holes. P(Dry) = 75%– More accurate with greater sample size. May not apply to future.


• Subjective– Personal degree of belief of the likelihood of a future event occurring

(or of the unknown outcome of a past event)

– May be based on some past similar / analogous occurrences

Uncertainty: not knowing if a statement is true or not

Throw a die and hide top face. What is the probability of a 3? 1/6

Now you getinformation.

Has has the top face changed? No


What is the probability of a 3 now?

Uncertainty is a function of what you know. There is no “right” uncertainty (or PDF)!

p g oHas the probability of a 3 changed? Yes!

1/3


11SPE DL 2010

Uncertainty is in OUR heads – it’s a function of our state of knowledge

Different people can, legitimately, hold

Its not a feature of the “system”. A consequence:

0.333 0.333

p p g ydifferent views about the uncertainty of an

unknown quantity

Person A Person B


1 2 3 5 64Outcome

Prob

.

1 2 3 5 64Outcome

Prob

.

Uncertainty is in YOUR head:What’s it worth to know more?

Another consequence:

Information might have value by virtue of its ability to change probabilitiesits ability to change probabilities

• Betting Game- Win $100 if it is a 3.

- Lose $10 if it is not

• How much is it worth to look at?


How much is it worth to look at (get information about) the centre?

Corollary: you might change your mind on whether to bet or not as a result of getting new information


12SPE DL 2010

Probability is subjective: games of chance – eg coin tossing

Person A Info

(A thinks there is a

Person BInfo(A thinks there is a

small chance the coin is biased

towards heads)

InfoShared

Info


Person A PDF Person B PDF

Heads Tails Heads Tails

Probability is subjective: implications within companies or for joint ventures

Personor

Personor

Company BCompany

AInfo

Company BInfo

SharedInfo


Company A PDFCompany B PDF


13SPE DL 2010

Gambling (probability = repeated outcomes) vs. “Real World” (probability = degree of belief)

Uncertainty Quantification

All Identified Some missed or unknowable

KnownDistributionType

UnknownDistributionType

1. Identify Possible Outcomes 2. Assign Probabilities to Outcomes


KnownParameters

UnknownParametersGames of Chance:

Classical Prob & StatsOil & Gas:Subjective

Uncertainty and Ambiguity

• What is your degree of belief (probability) that Steve is a “wine-drinker”?

• The event “wine-drinker” is ambiguous. It could mean– Prefers wine to beer– Enjoys wine– Drinks a glass of wine once per week– Drinks a glass of wine daily, etc

• The precise definition of the event/statement/outcome is critical to determining our degree of belief in it.


We MUST remove any ambiguity in the definition of uncertain statements (events,

outcomes) to which we wish to assign probabilities


14SPE DL 2010

Uncertainty vs Risk

Uncertainty Risk


• A Risk (noun!) is one possible consequence of uncertainty. It has a negative connotation, which is “personal” to the D-M

- an event that has a negative impact on DM’s objectives

- It is specified by defining the event and assessing its probability,

Don’t take a biassed approach to managing consequences of uncertainty

• Risk is only one outcome of uncertainty - so is Opportunity!– often over-looked – it is a source of value creation

Consequences of Uncertainty

Risk Opportunity


Possibility of loss or injury

A dangerous element or factor

The degree of probability of loss

Possibility of exceeding expectations

Upside potential

A wonderful element or factor


15SPE DL 2010

Uncertainty vs Variability

Variability of all sand-body widths

Width

Uncertainty in individual sand-body width??


y??

Width

Sand 1

?? Sand 2

Uncertainty vs Variability

Variability of all sand-body widths

A di t ib ti th t

Width

Uncertainty in individual sand-body width??

A distribution that describes the variability

of a natural phenomenon is not usually appropriate to

describe the uncertainty in


y??

Width

Sand 1

?? Sand 2

describe the uncertainty in a single occurrence


16SPE DL 2010

Outline

• The Nature of Uncertainty

P l P b bilit d• People, Probability and Judgment

• Performance of Industry Experts

• Conclusions


Conclusions

Adelson’s square: Estimate the gray %

Using a scale of 0% (black) to 100% (white) estimate the % gray of squares A and B



17SPE DL 2010

*Shepard’s Table


Perceptual Limitations as a Metaphor Cognitive Limitations

• Awareness of illusions by itself does Awareness of illusions, by itself, does not produce a more accurate perception.

• Illusions & cognitive errors therefore, can be extremely difficult to


can be extremely difficult to overcome.


18SPE DL 2010

Judging likelihoods of events

• Linda is a 31 years old, single, outspoken and very bright. She majored in philosophy. As a student she was deeply concerned with issues of discrimination and social justice, and also participated in anti-nuclear demonstrations. p p

Which is the more likely alternative?

a) Linda is a bank teller

b) Linda is a bank teller and active in the feminist movement.


Answer:______

*Probability Rules: Addition Rule for Non-Mutually Exclusive Events

Event A and Event B can occur at the same time

Not A or B

AB

Joint probability = P (A and B)


• Multiplication Rule for independent events

P(A and B) = P (A) * P(B)


19SPE DL 2010

*Discussion of Linda Question

• Nearly 90% of respondents choose the second alternative (bank teller and active in the feminist movement), even though this is provably logically incorrect

bank tellers feminists


P(Bank Teller) > P(Bank Teller AND Feminist)

feminist bank tellers

Cognitive Illusions

• The description of Linda is more representative of a feminist bank teller so people, wrongly, conclude it is more likely that she is a feminist and a bank teller

• Kahneman & Tversky (1982)

– “As the amount of detail in a scenario increases, its probability can only decrease steadily, but its representativeness and hence its apparent likelihood may increase.”

“Th li t ti b li i i


– “The reliance on representativeness, we believe, is a primary reason for the unwarranted appeal of detailed scenarios and the illusory sense of insight that such constructions often provide.”

• Implications: consider a “rich” description of a reservoir depositional environment


20SPE DL 2010

Sequences - Representivity Heuristic:The “Law of Small Numbers”

• I have just tossed a fair coin 7 times. You have not seen the result

• You are invited to play a betting game to guess which of the three sequences below is the one I actually observed.

• Which sequence would you bet on

a) HHHHTTT

b) THHTHTT

• Using multiplicative rule for independent eventsP(A&B&C&D.) = P(A)*P(B)*P(C)*P(D) ….

P = (1/2)7 = 1/128

P (1/2)7 1/128


b) THHTHTT

c) TTTTTTTP = (1/2)7 = 1/128

P = (1/2)7 = 1/128

• ALL sequences have the SAME probability and are thus EQUALLY likely (or equally rare!)

*Intuition: Steel Band Problem

• Assume the earth is perfectly flat and a band of steel is places around the equator, which has a circumference of 50,000 km.

• Now we add in an extra 10m of steel, which slightly forces the band off the surface of the earth.

• Estimate by how much?– ………….

– The diameter of a proton, an atom …

Thi k f $ t


– Thickness of a $ note

– ……..

• Answer: 1.6 metres


21SPE DL 2010

Heuristics, Biases & Uncertainty

• Heuristics

– simple rules of thumb and mental shortcutssimple rules of thumb and mental shortcuts

• Biases

– systematic errors that can result from the use of heuristics

• Our “mental wiring” is just not good when it


comes to uncertainty

– Intuition and “gut feel” often significantly wrong

Heuristics, Biases & Uncertainty

• Heuristics

– simple rules of thumb and mental shortcutsHuman beings are not endowed with simple rules of thumb and mental shortcuts

• Biases

– systematic errors that can result from the use of heuristics

• Our “mental wiring” is just not good when it

Human beings are not endowed with rational probabilistic thinking and optimal

behaviour under uncertainty.


comes to uncertainty

– Intuition and “gut feel” often significantly wrongBias & error => poor decisions => undesirable outcomes


22SPE DL 2010

Evidence of Bias: Data from IPA

All 1000+projects

If the Forecasted production is the “Base Case”, we h ld h i t l projects

in the study

No projects

should have approximately as many projects producing more than expected as less than expected !!


0 50 100 150 200 250 300

Basis for development sanction

0% 50% 100% 150% 200% 250% 300%

Outline

• Introduction

Th N t f U t i t• The Nature of Uncertainty

• People, Probability and Judgment

• Performance of Oil & Gas Industry Experts


Industry Experts

• Conclusions


23SPE DL 2010

Refresher: Interpreting PDFs & Percentiles

The area under the PDF between any two points is theprobability of X lying between those two points

0.10

80% ofarea

10% ofarea


P10 P90

10% ofarea

X

Assessing the ability of experts to assign P10-P90 ranges

Lower Limit(P10)

UpperLimit(P90)

80%Chance

10% 10%

Question 1

Question 2

Question 3

etc

E.g. What was daily average oil production in the USA in 2003?


Answer: 7,454,000

g y g p


24SPE DL 2010

Overconfidence Results: Large Industry Sample –using “industry- related” questions

0.30

0.35

pan

ts Expected

Observed

0.10

0.15

0.20

0.25

po

rtio

n o

f P

arti

cip


0.00

0.05

0 1 2 3 4 5 6 7 8 9 10

Pro

p

Questions Correct /10

Exploration Production AbandonDevelopment

Perceived uncertainty

Actual uncertainty

* Experts & Over-confidence

Max

EOR

uncertainty


Time

Min

2D3D

Discovery. drilling

Development. drilling


25SPE DL 2010

US DoE Price Forecasts

Trends Predicted Beginning From the Actual Price of Year ListedTrends Predicted Beginning From the Actual Price of Year Listed

120120

100100

19821982

19811981100100

8080

6060

4040

19841984

19851985

19861986 19871987

Dol

lars

per

Bar

rel

Dol

lars

per

Bar

rel

19831983


after U.S. Department of Energy, 1998after U.S. Department of Energy, 1998

2020

0019751975 19801980 19851985 19901990 19951995 20002000 20052005

YearYear

19911991

19951995Actual

DD

Does it Matter? Overconfidence Model

Biassed –OverconfidentOverconfident

True


12500 15000 17500 20000 22500

Area

OC20 True 10th/20th percentile


26SPE DL 2010

400

on

Economic Impact of Overconfidence

Welsh, Begg & Bratvold (2007) SPE 110765

100

200

300

Rea

l E

(NP

V),

$M

illio

NPV

EV

Assessed (Overconfident) Value


-100

0

0% 5% 10% 15% 20% 25% 30%Overconfidence

Probability Intuition: Assessing co-variation

Present Absent

Present 16 4Seismic

Hydro-carbons

Present 16 4

Absent 4 1

• Which cells of the table are needed to determine whether seismic anomalies are associated with hydrocarbons

Upper Upper Lower Lower

Anomaly


Upper Upper Lower Lower

Left Right Left Right

• According to the data in the table, do seismic anomalies increase the probability of hydrocarbon presence? Yes/No


27SPE DL 2010

Probability Concepts: Conditional Probability in Venn Diagram

Not A or B

A

Joint probability = P (A and B) =

AB

P (A and B) P(A|B) = =

As a proportion of the whole area

The probability of A happening, given that B has occurred1/52 1

P(Q|) = =


P(A|B) P(B)

(Q|)1/4 13

P (A and B) P(B|A) = =

P(A)

The probability of B happening, given that A has occurred1/52 1

P(|Q) = = 1/13 4

Probability Intuition: Assessing co-variation

Present Absent Total

Present 16 4 20

Hydrocarbons

Seismic

• All 4 pieces on information (cells) are required

• Conditional probabilities of HC presence given the presence/absence of seismic anomaly

Absent 4 1 5

Total 20 5 25

Seismic Anomaly


P(HC Present | Anomaly present) = 16/20 = 80%

P(HC Present | Anomaly absent) = 4/5 = 80%

• Probability of HC being present is the same, whether or not a seismic anomaly is present – so no information


28SPE DL 2010

Assessing Co-variation (presumed association): Results

80

90

100

Correct

Incorrect

30

40

50

60

70

80

of

Par

tici

pan

ts


0

10

20

30

First Both

%

Correct Answer

Correct for right reason

Probability Intuition: Assessing Reliability of Predictors

• Historical estimates suggest one in every 1000 blow-out preventers has serious cracks.

• Suppose x-ray analysis is a very good but not• Suppose x-ray analysis is a very good, but not perfect, detector of these cracks.

– If a blow-out preventer has cracks, x-rays will correctlysay it has them 99% of the time

– If a blow-out preventer does not have cracks, x-rays will wrongly say that it has them 2% of the time


• A blow-out preventer has been x-rayed at random and the result was positive!

– What is your intuitive assessment of the chances that is cracked?


29SPE DL 2010

Probability Intuition: Assessing Reliability of Predictors

• Historical estimates suggest one in every 1000 blow-out preventers has serious cracks.

• Suppose x-ray analysis is a very good but not• Suppose x-ray analysis is a very good, but not perfect, detector of these cracks.

– If a blow-out preventer has cracks, x-rays will correctlysay it has them 99% of the time

– If a blow-out preventer does not have cracks, x-rays will wrongly say that it has them 2% of the time

4.7%!P( test positive given crack)=99%

P( crack given test positive)=4.7%


• A blow-out preventer has been x-rayed at random and the result was positive!

– What is your intuitive assessment of the chances that is cracked?

Reliability of predictors: Results

70

80

90

30

40

50

60

0

No

of

Par

tici

pan

ts


0

10

20

0 20 40 60 80 100Estimated Probability


30SPE DL 2010

Intuition is Overrated

• Many decision-makers believe that intuition, repeated experience and their general intelligence will see them through

• Human beings are imperfect information processors

• We can’t always trust our intuition and perception. particularly in an uncertain environment!


• We need to use the appropriatetools and frameworks to address the uncertainties and decisions.

Anchoring - Subtle changes in wording of a question can significantly impact responses

Question Outcome

How long was the movie? 130 mino o g as e o e

How short was the movie?

30

100 min

Question Outcome


How wide are the channels?

How narrow are the channels?


31SPE DL 2010

Anchoring Question: Large Industry Sample

• Alternate versions of question with high and low anchors given to two groups

Hi h A h G “W ld d il i– High Anchor Group: “Were world proved oil reserves in 2003 greater or less than 1721.6 Billion Barrels?”

– Low Anchor Group: “Were world proved oil reserves in 2003 greater or less than 573.9 Billion Barrels?”

B th th k d th ti


• Both groups then asked the same question

– “What is your best estimate of the world proved oil reserves in 2003?”

Anchoring Results: Large Industry Sample

3000

3500

Anchor

1932

1722

1000

1500

2000

2500

mat

ed W

orl

d P

rove

d

ves

2003

+

/-1s

d

Estimate


574

682

0

500

Low HighAnchor Group

Mea

n E

stim

Res

erv


32SPE DL 2010

Anchoring Results: Large Industry Sample

3000

3500

Anchor

1932

1722

1000

1500

2000

2500

mat

ed W

orl

d P

rove

d

ves

2003

+

/-1s

d

EstimateCommon approach in E&P project evaluation:

“Let’s start with a base case and then build some scenarios around it ”


574

682

0

500

Low HighAnchor Group

Mea

n E

stim

Res

erv then build some scenarios around it.

• Unpacking Question (RoW Packed)

– “What % of world oil consumption is accounted for by each of the following regions: North

Unpacking

for by each of the following regions: North America, Europe/Eurasia and the Rest of the World?”

• Unpacking Question (RoW Unpacked)

“What % of world oil consumption is accounted


– What % of world oil consumption is accounted for by each of the following regions: North America, Europe/Eurasia, South and Central America, Middle East, Africa, Asia Pacific and the Rest of the World?”.


33SPE DL 2010

Unpacking Question Results

50

60

Oil

1SD

True:

Packed

Unpacked

20

30

40

mat

ed

% o

f W

orl

d

um

pti

on

(20

03)

+/- Unpacked


0

10

N America Europe/Eur Rest of World

Est

iC

on

s u

Region

Risk Aversion and Incentives

• Your track record hasn’t been too good recently. You can recommend one of two investments. Which one?

P Pf EV $MMPs Pf EV, $MM

“Safe” 80% 20% 10

“Risky” 10% 90% 20

• You should take a corporate (organizational) attitude to risk, not a personal one, and recommend “Risky” based on EV


o a pe so a o e, a d eco e d s y based o

– any other choice (being risk-averse or risk-seeking) is value destroying (money–losing)

• Most incentive policies, focused on reward by outcome, encourage inappropriate risk-aversion, therefore value loss!


34SPE DL 2010

De-biasing

• Create awareness.– Anchoring– Overconfidence– Availability, Vividness & Recency

• Understand the limits of our knowledge.

• Actively challenge ourselves.

– Stop to consider reasons why your judgement might be wrong.


• Abandon false comfort of single-point predictions.– Use ranges instead of single-point estimates.– Use multiple anchors.

• Calibration.– Feedback and accountability.

Reasoning under uncertainty = using the rules of probability

• In terms of practical applicability, probability theory is comparable with geometry;

- both are branches of applied mathematics that are- both are branches of applied mathematics that are directly linked with the problems of daily life.

• While most people have a natural feel for geometry (at least to some extent), many people clearly have trouble developing a good intuition for probability.

• In no other branch of mathematics is it so easy to


o ot e b a c o at e at cs s t so easy tomake mistakes as in probability theory.

- Conditional probabilities, and Bayes theorem in particular, are especially difficult


35SPE DL 2010

Reasoning under uncertainty = using the rules of probability

• In terms of practical applicability, probability theory is comparable with geometry;

- both are branches of applied mathematics that are

“The theory of probabilities is at bottom nothing but common sense reduced to calculus; …- both are branches of applied mathematics that are

directly linked with the problems of daily life.

• While most people have a natural feel for geometry (at least to some extent), many people clearly have trouble developing a good intuition for probability.

• In no other branch of mathematics is it so easy to

;

It teaches us to avoid the illusions which often mislead us;

… there is no science more worthy of our contemplations nor a more useful one for admission to our system of public education ”


o ot e b a c o at e at cs s t so easy tomake mistakes as in probability theory.

- Conditional probabilities, and Bayes theorem in particular, are especially difficult

to our system of public education.Laplace – Theorie Analytique des Probabilites

Conclusions

• In our context, uncertainty is a function of what we know about a situation – its in our heads, not an objective attribute of the “system”- there is no single, “right” probability for an uncertain event

- variability is not the same thing as uncertainty

• Evolution has not “wired” our brains for a good natural ability to assess uncertainty

- training helps, but even industry experts, including those


g y gwhose job it is to deal with uncertainty, are not great

• Do not use intuition to propagate (amalgamate) assessed uncertainties

- use the rules of probability, or Monte Carlo simulation


36SPE DL 2010

Understanding the Limits of Our Knowledge

• To know that we know what we know, and that we do not knowknow, and that we do not know what we do not know, that is true knowledge

Confucius

“It’s not what we don’t know that gets


It s not what we don t know that gets us into trouble, it’s what we know that

ain’t so”Will Rogers

The Pioneers

• Kahneman and Tversky– 2002 Nobel Price in Economics

• Russo and Schoemaker• Russo and Schoemaker– Decision Traps

– Winning Decisions

• Thaler– The Winner’s Curse

• Bazerman


Bazerman– Judgment in Managerial Decision Making

• Plous– The Psychology of Judgment and Decision Making


37SPE DL 2010

Acknowledgements & Further [email protected]

• Prof. Reidar Bratvold– University of Stavanger, Norway

• Dr. Matthew Welsh– University of Adelaide

• Dr. Michael Lee– University of California, Irvine

• SPE 77509– “Would You Know a Good Decision if You Saw One?”

• SPE 84238


– “Shrinks or Quants: Who will improve decision-making?”

• SPE 96423

– “Cognitive Biases in the Petroleum Industry: Impact and Remediation”

Thi k th 50thThis year marks the 50th

anniversary of the SPE Distinguished Lecturer program. Please visit our site to learn more about this amazing program.

www.spe.org/go/DL50

spe distinguished lecturer program · expert reliability & uncertainty steve begg, asp,...

Documents