Download - 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour
![Page 1: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/1.jpg)
1
Optimizing Decisions over the Long-term in the Presence of Uncertain
ResponseEdward Kambour
![Page 2: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/2.jpg)
2
Key Question Given a set of choices (product offerings), which
one is optimal? For each choice
Probability of success (sale) Revenue realized given a success
Revenue Management perspective Which of the choices will yield the greatest revenue
• Elicits the best customer response
![Page 3: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/3.jpg)
3
Deterministic world Customer response to each choice is known and
constant Plug in the known success rate and revenue for each
choice and optimize
![Page 4: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/4.jpg)
4
Random World Customer response is random with known parameter
values For example, success rate is 0.5 or realized revenue is
Poisson(7)
How to handle random response Risk neutral – Expected Response Risk averse – Expected Utility Minimax (Minimizes the worst possible outcome) Maximin (Maximizes the best possible outcome)
![Page 5: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/5.jpg)
5
Random World Typically don’t know precisely the demand
parameters Estimate the demand
Parametric estimators of unknown parameters Robust estimators of unknown parameters Non-parametrics
• Sample quantiles, Empirical CDF, …
![Page 6: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/6.jpg)
6
Real World Random response
In either the success rate or the realized revenue
Uncertainty in the demand distribution Based on earlier observations (expert opinion, similar
processes, etc.) The Bayesian approach quantifies this uncertainty in
a directly
![Page 7: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/7.jpg)
7
Effect of Uncertainty What effect does the uncertainty have with
regard to making optimal decisions? Assuming risk neutrality, should decisions be based
only on expected revenue? Is there potential value in exploring choices that do
not yield the best expected revenue as of now? Quantify value in obtaining additional information, i.e.
reducing uncertainty in the demand response
![Page 8: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/8.jpg)
8
Simple Example Choose either to place a $1 bet on a flip of a coin
coming out heads or keep your $1 You estimate that the probability of a head is 0.4 Should you play?
Expected Revenue Don’t play: 0 Do play: 1(p) – 1(1-p) = 2p – 1 = 2(0.4) –1 = -0.2
![Page 9: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/9.jpg)
9
Simple Example Solution Depends
![Page 10: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/10.jpg)
10
Scenario 1 There is no uncertainty
The probability of a head is exactly 0.4 Assuming that you’re risk neutral, you should not play
![Page 11: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/11.jpg)
11
Scenario 2 You know that the coin is either a 2-headed coin
or it is a 2-tailed coin Pr[2-headed] = 0.4 Pr[2-tailed] = 0.6 If you play once
Observe head 2-headed• Continue to play and make $1 every time
Observe tail 2-tailed• Stop after losing $1
![Page 12: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/12.jpg)
12
Scenario 2 (cont.) If you could play up to 10 times
Expected Revenue Don’t play – 0 Do play
• Observe the first flip
» If it’s heads then play the next 9 times
» If it’s tails then quit
• 0.4(10) - 0.6(1) = 3.4
![Page 13: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/13.jpg)
13
Scenario 3 You estimate that the probability of head is 0.4
plus or minus 0.2 Playing once no longer yields perfect information
Playing any finite number of times will not yield perfect information
There is a possibility of long-term positive revenue from playing
Head probabilities greater than 0.5 are plausible Should you play at all?
If so, under what circumstances should you stop playing?
![Page 14: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/14.jpg)
14
Scenario 3 (Bayesian Perspective) You estimate that the probability of head is 0.4
with uncertainty of 0.1 Prior mean = 0.4 Prior standard deviation = 0.1 Beta prior: = 9.2, = 13.8
Beta(9.2,13.8)
0
0.5
1
1.5
2
2.5
3
3.5
4
4.5
0 0.2 0.4 0.6 0.8 1 1.2
p
f(p)
![Page 15: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/15.jpg)
15
Scenario 3 (cont.) Let X be the number of heads in any set of n flips of the
coin and assume X is binomial distributed (iid Bernoulli trials)
Updated uncertainty in the head probability after the n trials Beta * = 9.2 + X, * = 13.8 + n – X Expectation: */(*+*) Variance: **/[(*+*)(*+*)(*+*+1)] Never know for sure
![Page 16: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/16.jpg)
16
Key Points Short term expected revenue is not the only thing
to consider There is also value in obtaining additional
information about the demand distribution because this information can be used to improve long-term revenue performance How do we calculate the value of this information in
terms of future revenue? How much experimentation is optimal?
![Page 17: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/17.jpg)
17
One Approach Evaluate any possible choice at every time
Completely reevaluate after every observation Due to the prior uncertainty, lose independence because
observations share the same unknown parameters Enumerate all possible combinations of observations and
strategies along with their associated probabilities and search for the best
Computationally intensive Complications with continuous variables
![Page 18: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/18.jpg)
18
More Restrictive Approach Define long-term in terms of a number of observations
For example, 10 flips of the coin
Allow for a single experiment with a single conclusion For example, 3 flips of the coin
If 3 heads continue to play over the last 7 Otherwise stop
Calculate the expected net return from an experiment Expected Cost – Expected Revenue from additional observations
![Page 19: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/19.jpg)
19
Expected Cost of Experiment Based on prior distribution on the unknown
parameters and the conditional distribution of the customer response (given the parameters) Calculate the marginal distribution for the data
Calculate the marginal expectation over the trials
Subtract the baseline revenue
( ) ( | ) ( )m x f x
E[ ] ( )X xm x dx
![Page 20: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/20.jpg)
20
Expected Revenue from Trial Expected revenue over the rest of the long-term period
given the decision made after the trial For each choice
Win implies the highest posterior expectation of revenue Pr[win] E[return|win] For the rest of the period (n-m)
• Expected revenue given win is (n-m)E[return|win]
The expected revenue is then the weighted sum Pr[Win](n-m)E[return|win]
Subtract the baseline revenue
![Page 21: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/21.jpg)
21
Scenario 3 (revisited) Set n = 10 Investigate an experiment with m trials Expected cost of the experiment
.2m
Assume we observed m1 heads Updated uncertainty
Beta( = 9.2 + m1, = 13.8 + m – m1)
![Page 22: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/22.jpg)
22
Scenario 3 (cont.) Continue to play if the new expectation on the probability
of a head is greater than 0.5 * = 9.2 + m1 > 13.8 + m – m1 = *
Expected return on the experiment Pr[* > * ](10-m){2E[*/(*+*)| * > * ]-1]} +
(1- Pr[* > * ])(10-m)0 Note
Non-negative If you let n and Pr[* > * ] > 0, then the expected return for any
finite experiment
![Page 23: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/23.jpg)
23
Scenario 3 (Details) * > *
9.2 + m1 > 13.8 + m – m1 2m1 > 4.6 + 0.5m m1 > 2.3 + 0.5m
m1 | p ~ Bin(m,p) m1~BetaBinomial(m,9.2,13.8) */(*+*) = (9.2 + m1) / (23+m)
E[*/(*+*)| * > * ] =
(9.2 +E[m1| m1 > 2.3 + 0.5m])/(23+m)
![Page 24: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/24.jpg)
24
Scenario 3 (Cont.) If m = 5
m1 > 2.3 + 0.5m = 4.8 m1 = 5
* = 9.2 + 5 =14.2
* = 13.8 + 0 = 13.8
Expected Return Pr[* > * ](10-m){2E[*/(*+*)| * > * ]-1]}
Pr[m1 = 5 ](10-5){2(14.2/28)-1} = 0.0175(5)0.0143 = 0.0012
![Page 25: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/25.jpg)
25
Scenario 3 (Cont.) Value of the 5 flip experiment
0.0012 – 0.2(5) = -0.9988
Value of each possible experimentm Exp Ret Exp Cost Net Ret0 0.0000 0.0000 0.00001 0.0000 -0.2000 -0.20002 0.0000 -0.4000 -0.40003 0.0000 -0.6000 -0.60004 0.0000 -0.8000 -0.80005 0.0012 -1.0000 -0.99886 0.0017 -1.2000 -1.19837 0.0023 -1.4000 -1.39778 0.0021 -1.6000 -1.59799 0.0014 -1.8000 -1.798610 0.0000 -2.0000 -2.0000
![Page 26: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/26.jpg)
26
Scenario 4 Equivalent to Scenario 3 except
= 0.092, = 0.138 Uncertainty (prior standard deviation) = 1
Value of Experimentm Exp Ret Exp Cost Net Ret0 0.0000 0.0000 0.00001 0.0000 -0.2000 -0.20002 0.0000 -0.4000 -0.40003 0.0000 -0.6000 -0.60004 0.0000 -0.8000 -0.80005 1.6430 -1.0000 0.64306 1.3148 -1.2000 0.11487 0.9923 -1.4000 -0.40778 0.6617 -1.6000 -0.93839 0.3320 -1.8000 -1.468010 0.0000 -2.0000 -2.0000
![Page 27: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/27.jpg)
27
General Rules of Thumb Larger values of n will lead to longer experiments
The longer the “long-term,” the greater the return on experiments
More uncertainty (larger prior standard deviations) will lead to more experimentation The less you know, the greater the return on
experiments
![Page 28: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/28.jpg)
28
Extensions to the Simple Example Method There are multiple choices
Each with a current prior distribution for the success probability
The revenue achieved for a success may be random for each choice The parameters of the distribution are not known
![Page 29: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/29.jpg)
29
Example Two choices
Choice 1 and Choice 2 The success probabilities are unknown for both choices The realized revenue from a success is random
• Unknown mean and variance
Long term horizon (n) is 50 Experiment
m1 trials for choice 1 m2 trials for choice 2
![Page 30: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/30.jpg)
30
Choice 1: Current Opinion Success Rate (Beta distributed)
Expectation: 0.50 Uncertainty: 0.15
Mean revenue on a success (Normal given the variance) Expectation: 8.00 Uncertainty: 0.41
Variance in revenue on a success (Inverted Gamma) Expectation: 2.07 Uncertainty: 0.05
![Page 31: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/31.jpg)
31
Choice 2: Current Opinion Success Rate (Beta distributed)
Expectation: 0.45 Uncertainty: 0.28
Mean revenue on a success (Normal given the variance) Expectation: 7.00 Uncertainty: 2.11
Variance in revenue on a success (Inverted Gamma) Expectation: 2.11 Uncertainty: 0.08
![Page 32: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/32.jpg)
32
Results Search for the best experiment in terms of net
revenue over the 50 observations
![Page 33: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/33.jpg)
33
Results The best trial consists of a sample of 6
observations 4 with choice 1 2 with choice 2
After the trial the choice with the best posterior mean is used for the remaining 44
![Page 34: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/34.jpg)
34
Future Work Develop efficient optimization routine
Search methods (concave envelope, gradients)
Extend the types of experimentation Multiple experiments
Further investigate the asymptotics
![Page 35: 1 Optimizing Decisions over the Long-term in the Presence of Uncertain Response Edward Kambour](https://reader035.vdocument.in/reader035/viewer/2022062905/5a4d1af57f8b9ab0599810c6/html5/thumbnails/35.jpg)
35