1 value and utility functions - dartmouth collegecushman/courses/engs41/mark borsuk's...
TRANSCRIPT
5/11/2015
1
Value and Utility Functions
Probability Theory
Value Functions Definition of a value function
Eliciting value functions
Utility Functions Definition of a utility function
Eliciting utility functions
1
Introduction• Probabilities in Decision Analysis▫ Example: Influence diagram for basic risky decision
5/11/2015
2
Introduction• Probabilities in Decision Analysis▫ Example: Decision tree for basic risky decision
• Probability is the most common and theoretically-supported method for describing uncertainty
• Formulated by Kolmogorov’s axioms (1933):
▫ Event space
▫ Probability function P: () [0,1] :
1. P(A) ≥ 0 A
2. P() = 1
3. P(A B) = P(A) + P(B) if A B =
Basics of Probability Theory
5/11/2015
3
• Definitions
▫ Conditional probability:
▫ Independence:)(
)()(
BP
BAPBAP
)()( APBAP
)()()( BPAPBAP
Basics of Probability Theory
• Reversing conditional probabilities:
▫ Conditional probability:
▫ Re-formulation:
▫ Bayes’ Rule:
)(
)()(
BP
BAPBAP
)()()()()( APABPBPBAPBAP
)(
)()()(
BP
APABPBAP
n
iii APABPBP
1
)()()(
Basics of Probability Theory
5/11/2015
4
▫ To apply the mathematical theory of probability to aspects of the real world we need a “physical” or “psychological” interpretation.
▫ At least two different interpretations have been suggested (both of which adhere to Kolmogorov’s axioms):
Frequentist interpretation
Personalist (subjective or Bayesian) interpretation
Interpretations of Probability
• Frequentist Interpretation
▫ The probability of an event represents the long-run frequency of similar events over repeated observations
▫ Useful for statistical inference about underlying regularities in random systems based on repeated measurements
▫ Strictly applies only to learningfrom data
Interpretations of Probability
5/11/2015
5
• But what is the relevant population of similar events for:
▫ Tomorrow’s weather?
▫ Next week’s playoff game?
▫ Next month’s court case?
▫ An earthquake at a nuclear waste storage facility?
Interpretations of Probability
• Subjective Interpretation
▫ The probability of an event represents the “degree of belief” that a person has that the event will occur, given all relevant information currently known to that person.
▫ Explicitly a function of the state of information
▫ Frequencies may be one useful piece of information.
▫ Two or more people can rationally disagree.
▫ “Probability does not exist.” –Bruno de Finetti
Interpretations of Probability
5/11/2015
6
• Subjective Interpretation (cont’d)
▫ Still adheres to Kolmogorov’s axioms (must be coherent)
▫ Bayes’ rule becomes a method for updating beliefs after obtaining new data (information, evidence, etc.)
)(
)()()(
eP
APAePeAP
Constant gNormalizin
PriorLikelihoodPosterior
Interpretations of Probability
• Your aging pet cat seems to not be feeling well. After speaking with some friends you become concerned that she may have chronic kidney disease. If so, then you would need to begin to feed her a special diet. • Based on the comments of your friends, you assess a 10%
probability that she has the disease, but would like to take her to the vet for a test.• The vet performs a blood test and tells you that it came back
positive.• You ask the vet how accurate the test is and he tells you that
when a cat actually has the disease, a positive result will occur 99% of the time, while if the cat does not have the disease a false positive may occur 15% of the time.• What is the probability that your cat has kidney disease?
10%? 99%? 15%? Or something else?
Example of Bayes’ Rule
5/11/2015
7
What is the probability that your cat has kidney disease given the test results?D = has disease + = test positiveD’ = no disease - = test negative
• We want P(D|+), which by Bayes’ rule is:P(+|D) = 0.99P(D) = 0.10
P(+|D’) = 0.15
)'()'|()()|(
)()()(1
DPDPDPDP
DPDPPn
iii
)(
)()()(
P
DPDPDP
Example of Bayes’ Rule
Example of Bayes’ Rule
423.0135.0099.0
099.0
)10.01(15.010.099.0
10.099.0
)(
)()()(
P
DPDPDP
What is the probability that your cat has kidney disease given the test results?
5/11/2015
8
• Subjective Interpretation (cont’d)
▫ The subjective interpretation of probability includes the frequentist interpretation as a limiting case, because for random events it is reasonable to believe that the chance for the next outcome is given by the frequentist probability.
▫ On the other hand, the subjective interpretation has a much wider field of application, as it can be used to describe knowledge or belief about non-repeatable and non-random events.
Interpretations of Probability
Eliciting Probabilities• Elicitation of probabilities
▫ Subjective probabilities can be assessed by elicitation.
▫ They can be made more tractable by comparison with hypothetical bets on lotteries of outcomes.
▫ Formally describes existing knowledge
▫ Can be used to describe knowledge or belief about non-repeatable and non-random events.
▫ Constitutes a “prior”, to be used either on its own or updated using data through a Bayesian process.
5/11/2015
9
Reintroduced tiger will survive in the wildTiger survives $X
Lottery A
Tiger does not survive $0
25% $100
Lottery B
75% $0
Method
1. Set up Lottery B as a reference lottery with a well-specified probability mechanism
2. Adjust the value of the winnings until the decision maker is indifferent
p x X = 0.25 x 100 + .75 x 0
p = 25/X
Fixed Probability Method
Method (cont’d)
3. Use wide brackets and converge slowly
4. Check for consistency
Reintroduced tiger will survive in the wildTiger survives $X
Lottery A
Tiger does not survive $0
25% $100
Lottery B
75% $0
Fixed Probability Method
5/11/2015
10
Reintroduced tiger will survive in the wildTiger survives $100
Lottery A
Tiger does not survive $0
p $100
Lottery B
(1-p) $0
Method
1. Set up Lottery B as a reference lottery with a well-specified probability mechanism
2. Adjust the probability of the reference lottery until the decision maker is indifferent
3. Use wide brackets and converge slowly
4. Check for consistency
Fixed Value Method
Reintroduced tiger will survive in the wildTiger survives $100
Lottery A
Tiger does not survive $0
p $100
Lottery B
(1-p) $0
Problems
1. People may not like the idea of a game
2. People may have a hard time understanding the game
3. Problems such as anchoring may affect results
Fixed Value Method
5/11/2015
11
• Find a partner such that at least one of you is actively looking for a job or summer internship. This person will be the ‘expert’ and the other person will be the ‘analyst’.
• The analyst should elicit from the expert the probability that he/she will have found and committed to a job by June 1, 2014.
• Probability Density Function (PDF)
Probability distribution of X describes the probability of possible results of X
variablerandom continuous a is ifd)(
variablerandom discrete a is if)(
],[Xxxf
XxP
xxP r
l
ril
x
x
xxxi
rlX
Distributions
5/11/2015
12
Distributions• Cumulative Distribution Function (CDF)
Distribution function: Probability for values less than or equal to x:
variablerandom continuous a is if'd)'(
variablerandom discrete a is if)(
Xxxf
XxP
xF xxx
i
X
i
How do we elicit subjective probabilities?
Continuous quantities1. Assess a number of cumulative probabilities to
derive a CDF
2. Assess “ends” and middle of distribution, then a number of midpoints of segments
3. Calculate the derivative of the CDF to get the PDF
Eliciting Probabilities
5/11/2015
13
Climate Sensitivity
• Climate sensitivity, S, is defined as the equilibrium global temperature change in response to a doubling in radiative forcing.• Climate sensitivity is a
variable about which there is much uncertainty.
1. Use reference lotteries to assess a number of cumulative probabilities to derive a CDF
S < 2.0 deg C $100
Lottery A
S > 2.0 deg C $0
p $100
Lottery B
(1-p) $0
Eliciting ProbabilitiesContinuous Quantities
5/11/2015
14
2. Assess “ends” of distribution (0.10 and 0.90 fractiles) and middle (median) by fixing p and assessing corresponding S. Then assess a number of midpoints of segments.
S < x deg C $100
Lottery A
S > x deg C $0
0.90 $100
Lottery B
0.10 $0
Eliciting ProbabilitiesContinuous Quantities
P(S < 1.25) = 0.10P(S < 2.0) = 0.30P(S < 2.25) = 0.5P(S < 2.30) = 0.70P(S < 2.50) = 0.90
P (Delta T < deg)
0.00
0.25
0.50
0.75
1.00
0.50
1.00
1.50
2.00
2.50
3.00
Degrees
Cu
m.
Pro
b.
Eliciting ProbabilitiesContinuous Quantities
Can then take the derivative to get a probability density function.
P(S < x degrees)
5/11/2015
15
Elicitation ExerciseAssess a continuous probability distribution for the amount of snow we will receive in Hanover, NH, next winter (in whatever units you prefer). Do this with a partner using the reference lottery approach. Use either the fixed probability or fixed values approach (you and your partner might each try a different method). Draw your CDF on graph paper. Using slopes, convert this to an approximate PDF. Your challenge is for your distribution to be neither too narrow (i.e., overconfident) nor too wide (underconfident).
Value and Utility Functions
Probability Theory
Value Functions Definition of a value function
Eliciting value functions
Utility Functions Definition of a utility function
Eliciting utility functions
30
5/11/2015
16
• Preferences and Value Functions
21 yy means that the outcome y1 is preferredto the outcome y2.
We want to analyse preferences of possible consequences of management alternatives.
The alternative leading to the most preferred consequences is the one to be chosen.
To find the preferred alternative we need:
an outcome model for decision alternatives, and
a characterization of preferences over consequences.
Preferences are defined for pairs of possible consequences.
Value Functions
31
• Preferences and Value Functions
Definition of a value function:
An value function is a function which assigns each outcome, y1, a real number v(y1), which is larger then v(y2) if y1 is preferred to y2.
Note that in economics, value functions are often called “utility functions.” In the decision sciences, the name utility function is reserved for the description of preferences under uncertainty/risk.
Value Functions
32
5/11/2015
17
313221 thenandif yyyyyy
211221 ororeither yyyyyy
Existence of a value function:
For each preference that is
transitive ( ) and
complete ( )
there exists an ordinal value function representing the preference.
• Preferences and Value Functions
Value Functions
33
• Preferences and Value FunctionsNon-uniqueness of value functions: Value functions are unique up to an
affine transformation (i.e., linear and shift).
Typically scaled to range from 0 to 1.
Interpretation of value functions: Existence as a mathematical
description of preferences, not as a model of thinking of a human decision maker (decision makers do not explicitly maximise values).
Construction of value functions by analysis of preferences.
Value Functions
34
5/11/2015
18
• Preferences and Value Functions
Human preferences are not always transitive and complete; the assumptions can be questioned for a behavioural decision theory. However, the assumptions seem reasonable for a prescriptive theory.
Completeness may limit the field of application of decision theory (no alternatives with “extreme” outcomes).
Value Functions
35
• Elicitation of a Value Function
Single Attribute Case:
Ask for preferences between exemplary outcomes, and construct the value function by interpolation or smoothing.
Do not ask for shape parameters of the value function directly (value functions are mathematical representations of preferences not “stored” in this form by the decision maker).
Value functions are often normalised to the interval [0,1].
After eliciting a value function, consistency checks have to be performed.
Value Functions
36
5/11/2015
19
• Elicitation of a Measurable Value Function
Single Attribute Case:
Direct Rating (not recommended)Ordering of outcomes; specification of values compatible with ordering and with preferences about switching between outcomes.
Difference Standard Sequence TechniqueSet value of worst outcome to zero; set value of outcome with attribute value about 20% of span larger to unity; determine subsequent outcome values so that value increments are unity; normalise value function.
Bisection Method (Midvalue Splitting Technique)Determine worst and best outcome, set values to 0 and 1; identify outcome with value of 0.5; identify outcomes with values of 0.25 and 0.75.
Value Functions
37
Direct Rating – Continuous Case
Ordering of outcomes; specification of values compatible with ordering and with preferences about switching between outcomes
Value Functions
38
5/11/2015
20
Direct Rating – Discrete Case
For a job choice, the location alternatives (from worst to best) are:
Ordering of outcomes; specification of values compatible with ordering and with preferences about switching between outcomes
Value Functions
39
Difference Standard Sequence Technique
1.0
1.0
1.0
1.0
1.0
1
0.8
0.6
0.4
0.2
0
Set value of worst outcome to zero; set value of outcome with attribute value about 20% of span larger to unity; determine subsequent outcome values so that value increments are unity; normalise value function.
Value Functions
40
5/11/2015
21
Difference Standard Sequence Technique to assessValue function for “time spent on a cruise ship”
1
0.857
0.714
0.571
0.429
0.286
0.143
0
Value Functions
41
Bisection Method (Midvalue Splitting Technique)
Determine worst and best outcome, set values to 0 and 1; identify outcome with value of 0.5; identify outcomes with values of 0.25 and 0.75.
Value Functions
42
5/11/2015
22
Typical shape of a single attribute value function:
Decreasing marginal value:
Of special importance for attributes measured in monetary units
Value Functions
43
• Other Parametric Shapes
Value Functions
44
5/11/2015
23
Break and Practice Exercise• Assess a continuous value function for the amount of
sleep you need per day. • Do this with a partner using the Difference Standard
Sequence Technique or the Bisection Method.
45
Summary of Value Functions• Decisions under Certainty with Value Functions
Assumptions:▫ An outcome prediction model gives certain results for all
attributes for all decision alternatives.▫ The value function is elicited from the decision maker as a
function of the attributes (preferences fulfil axioms).
Values can be calculated for all decision alternatives.
Decreasing values lead to a preference ranking of the alternatives.
46
5/11/2015
24
Utility functions should be considered when there is uncertainty and a non-neutral attitude toward risk
Decisions nearly always have uncertainty, however value functions can still be useful:
Value functions are easier to elicit then utility functions;
The application of value functions already leads to a clear separation of predictions from valuations;
Development of the theory for decisions under certainty is a useful step towards the theory under uncertainty/risk;
Uncertainty can still be (partially) taken into account through sensitivity analysis.
Value Functions
47
Value and Utility Functions
Probability Theory
Value Functions Definition of a value function
Eliciting value functions
Utility Functions Definition of a utility function
Eliciting utility functions
48
5/11/2015
25
a probabilistic outcome model (yielding a probability of each possible outcome for each decision alternative)
appropriate characterization of preferences under uncertainty (accounting for attitudes toward risk)
Most decisions are made under uncertainty, therefore we need:
Utility Functions
49
Lotteries
p = 0.5 $1000
p = 0.5 - $1000
?
Would you pay to participate in this lottery?
If not, how much would you need to be paid?
Utility Functions
50
5/11/2015
26
• Preferences and Utility Functions
Definition of a utility function:
A utility function describes preferences for both outcomes and risk such that overall preference between actions with uncertain outcomes is reflected by the expected values of the utilities for each outcome.
This is done by replacing outcomes in the value function setting with lotteries of outcomes, designated Li
p y1
1 – p y2
Utility Functions
51
• Axioms for Expected Utility:
1. Preferences among uncertain events called lotteries Li
can be specified based on consequences and associated probabilities.
2. These preferences fulfill the following axioms:
1. Transitivity and completeness
2. Continuity
3. Independence (substitutability)
313221 thenandIf LLLLLL
211221 ororeither Also, LLLLLL
312321 )1(:thenIf LpLpLpLLL
3323121 )1()1(thenIf LLpLpLpLpLL
Utility Functions
52
5/11/2015
27
• Expected Utility Theory
If these assumptions hold, then…
1. It is possible to construct a utility function to represent your preferences for uncertain outcomes, such that…
2. Your overall preference between lotteries of uncertain outcomes is reflected by the expected utilities of each lottery
p1 U(y1)
1 – p1 U(y2)
p2 U(y3)
1 – p2 U(y4)
Utility Functions
53
• Interpretation of Utility Functions
Existence as a mathematical description of preference under uncertainty, not as a model of thinking of a human decision maker (decision makers do not explicitly maximize utility)
Construction of utility functions by analysis of preferences under uncertainty for an individual
The shape of U(y) is a result of both the shape of the value function and risk attitudes
Utility Functions
54
5/11/2015
28
• Typical risk averse utility function
0
1
Utility
Attributeattribute range
Utility Functions
55
0.96
0.36
0.66
• Risk aversion as a positive risk premium
CertaintyEquivalent
ExpectedOutcome
Risk Premium
Outcome ($000)
CE < E(X) Risk Aversion
ExpectedUtility
Utility Functions
56
5/11/2015
29
• Other attitudes toward risk
Outcome
Utility Functions
57
• Non-uniqueness of utility functions
Utility functions are only ordinal (they do not express strengthof preference)
Utility functions are only unique up to an affine transformation (linear and shift).
Utilities are not absolute, but depend on the attribute ranges
Utility Functions
58
5/11/2015
30
• Elicitation of a Utility Function
Principles:
Ask for preferences between reference lotteries and certain outcomes
Do not ask for shape parameters of the utility function directly
Utility functions are usually normalised to the interval [0,1]
After eliciting a utility function, consistency checks should be performed
It is often useful to compose a utility function as a function of single attribute “sub-utility functions”
Utility Functions
59
• Elicitation of a Utility Function
Single Attribute Case:
Certainty Equivalent Method
1. Assign outcomes of a 50/50 reference lottery to the best and worst possible levels of the attribute; determine the attribute level for which the decision maker would be indifferent between this level with certainty and the reference lottery; this level is called the “certainty equivalent” for the reference lottery and is assigned a utility of 0.5.
2. Repeat by replacing the worst outcome with the outcome determined in step 1 (with utility = 0.5); the resulting certainty equivalent is assigned a utility of 0.75.
3. Repeat step 1 replacing the best outcome with the outcome with utility = 0.5; the resulting certainty equivalent is assigned a utility of 0.25.
Utility Functions
60
5/11/2015
31
• Certainty equivalent method
p = 0.5 $100
Lottery A
p = 0.5 $ 10
Certainty BCE
U(CE) = 0.5 U(100) + 0.5 U(10)
= 0.5 (1) + 0.5 (0)
= 0.5
Utility Functions
61
• Certainty equivalent method
p = 0.5 $100
Lottery A
p = 0.5 CE’
Certainty BCE
U(CE) = 0.5 U(100) + 0.5 U(CE’)
= 0.5 (1) + 0.5 (0.5)
= 0.75
Utility Functions
62
5/11/2015
32
• Certainty equivalent method
0
1
Utility
Attributeattribute range
0.75
0.5
0.25
Utility Functions
63
• Elicitation of a Utility Function
Single Attribute Case:
Probability Method
1. Assign probabilities of a reference lottery values of p and 1-p for the outcomes with the best and worst possible levels of the attribute respectively; set the certainty equivalent to an intermediate value; determine the value of p for which the decision maker would be indifferent between the certainty equivalent and the reference lottery; the value of p is equal to the utility of the certainty equivalent.
2. Repeat with other intermediate values for the certainty equivalent.
Utility Functions
64
5/11/2015
33
p $100
Lottery A
1-p $10
Certainty B
• Probability method
$65
U(65) = p U(100) + (1-p) U(10)
= p (1) + (1-p) (0)
= p
Utility Functions
65
• Probability method
0
1
Utility
Attributeattribute range
0.8
0.6
0.4
0.2
Utility Functions
66
5/11/2015
34
• Functional Forms
U(x) = log(x)
U(x) = 1-exp(-x/R)
U(x) = sqrt(x)
0
1
Utility
Attributeattribute range
Each of which would then need to be normalized to a [0,1] utility range by application of a linear transformation.
Utility Functions
67
Discrete Attributes• Utility Function Assessment▫ Generalized procedure
68
5/11/2015
35
p San Francisco
Lottery A
1-p Ann Arbor
Certainty B Boston
U(BO) = p U(SF) + (1-p) U(AA)
= p (1) + (1-p) (0)
= p
Discrete Attributes• Utility Function Assessment▫ Probability method
69
Practice Exercise
• With a partner, assess each other’s utility function for money over the range $0 to $1000 using the Certainty Equivalent Method or Probability Method.
70