probabilistic bisection search 0.1cm for stochastic root …probabilistic bisection search for...
TRANSCRIPT
Probabilistic Bisection Searchfor Stochastic Root-Finding
Rolf Waeber Peter I. Frazier Shane G. Henderson
Operations Research & Information EngineeringCornell University, Ithaca, NY
Sunday October 14, 2012
INFORMS Annual MeetingPhoenix, AZ
This research is supported by AFOSR YIP FA9550-11-1-0083.
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Problem
0 1
0
g(x)
X*
• Consider a function g : [0, 1]→ R.• Assumption: There exists a unique X∗ ∈ [0, 1] such that
• g(x) > 0 for x < X∗,• g(x) < 0 for x > X∗.
Goal: Find X∗ ∈ [0, 1].
• Can only observe Yn(Xn) = g(Xn) + εn(Xn), where εn(Xn) is anindependent noise with zero mean (median).Decisions:
• Where to place samples Xn for n = 0, 1, 2, . . .
• How to estimate X∗ after n iterations.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 2/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Problem
0 1
0
g(x)
X*
• Consider a function g : [0, 1]→ R.• Assumption: There exists a unique X∗ ∈ [0, 1] such that
• g(x) > 0 for x < X∗,• g(x) < 0 for x > X∗.
Goal: Find X∗ ∈ [0, 1].• Can only observe Yn(Xn) = g(Xn) + εn(Xn), where εn(Xn) is anindependent noise with zero mean (median).
Decisions:• Where to place samples Xn for n = 0, 1, 2, . . .
• How to estimate X∗ after n iterations.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 2/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Problem
0 1
0
Yn(X
n)
X*
• Consider a function g : [0, 1]→ R.• Assumption: There exists a unique X∗ ∈ [0, 1] such that
• g(x) > 0 for x < X∗,• g(x) < 0 for x > X∗.
Goal: Find X∗ ∈ [0, 1].• Can only observe Yn(Xn) = g(Xn) + εn(Xn), where εn(Xn) is anindependent noise with zero mean (median).
Decisions:• Where to place samples Xn for n = 0, 1, 2, . . .
• How to estimate X∗ after n iterations.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 2/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Problem
0 1
0
Yn(X
n)
X*
• Consider a function g : [0, 1]→ R.• Assumption: There exists a unique X∗ ∈ [0, 1] such that
• g(x) > 0 for x < X∗,• g(x) < 0 for x > X∗.
Goal: Find X∗ ∈ [0, 1].• Can only observe Yn(Xn) = g(Xn) + εn(Xn), where εn(Xn) is anindependent noise with zero mean (median).Decisions:
• Where to place samples Xn for n = 0, 1, 2, . . .
• How to estimate X∗ after n iterations.Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 2/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Applications
• Simulation optimization:• g(x) as a gradient
• Sequential statistics:
• Finance:• Pricing American options• Estimating risk measures
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 3/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Approximation (Robbins and Monro, 1951)
0 1
0
g(x)
X*
1. Choose an initial estimate X0 ∈ [0, 1];
2. Determine a tuning sequence (an)n ≥ 0,∑∞
n=0 a 2n <∞, and∑∞
n=0 an =∞.(Example: an = d/n for d > 0.)
3. Xn+1 = Π[0,1](Xn + anYn(Xn)), where Π[0,1] is the projection to [0, 1].
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 4/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
A “Good” Tuning Sequence
0 200 400 600 800 10000
1
n
Xn
X*
an = 1/n, ε
n ~ N(0,0.5)
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 5/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
A “Bad” Tuning Sequence – too Large
0 200 400 600 800 10000
1
n
Xn
X*
an = 10/n, ε
n ~ N(0,0.5)
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 5/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
A “Bad” Tuning Sequence – too Small
0 200 400 600 800 10000
1
n
Xn
X*
an = 0.5/n, ε
n ~ N(0,0.5)
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 5/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Lack of Robustness – εn with Heavy-Tails
0 200 400 600 800 10000
1
n
Xn
X*
an = 1/n, ε
n ~ t
3
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 5/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
A Different Approach
What about a bisection algorithm?
0 1
0
g(x)
X*
• Deterministic bisection algorithm will fail almost surely.• Need to account for the noise.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 6/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
A Different Approach
What about a bisection algorithm?
0 1
0
g(x)
X*
• Deterministic bisection algorithm will fail almost surely.• Need to account for the noise.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 6/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
The Probabilistic Bisection Algorithm
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 7/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
The Probabilistic Bisection Algorithm (Horstein, 1963)
• Input: Zn(Xn) := sign(Yn(Xn)).• Assume a prior density f0 on [0, 1].
0 10
1
2
fn(x)
n = 0, Xn = 0.5, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 8/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
The Probabilistic Bisection Algorithm (Horstein, 1963)
• Input: Zn(Xn) := sign(Yn(Xn)).• Assume a prior density f0 on [0, 1].
0 10
1
2
fn(x)
n = 0, Xn = 0.5, Z
n(X
n) = −1
X*
Xn
0 10
1
2
fn(x)
n = 1, Xn = 0.38462, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 8/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
The Probabilistic Bisection Algorithm (Horstein, 1963)
• Input: Zn(Xn) := sign(Yn(Xn)).• Assume a prior density f0 on [0, 1].
0 10
1
2
fn(x)
n = 0, Xn = 0.5, Z
n(X
n) = −1
X*
Xn
0 10
1
2
fn(x)
n = 1, Xn = 0.38462, Z
n(X
n) = −1
X*
Xn
0 10
1
2
fn(x)
n = 2, Xn = 0.29586, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 8/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
The Probabilistic Bisection Algorithm (Horstein, 1963)
• Input: Zn(Xn) := sign(Yn(Xn)).• Assume a prior density f0 on [0, 1].
0 10
1
2
fn(x)
n = 0, Xn = 0.5, Z
n(X
n) = −1
X*
Xn
0 10
1
2
fn(x)
n = 1, Xn = 0.38462, Z
n(X
n) = −1
X*
Xn
0 10
1
2
fn(x)
n = 2, Xn = 0.29586, Z
n(X
n) = 1
X*
Xn
0 10
1
2
fn(x)
n = 3, Xn = 0.36413, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 8/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Revisited
0 1
0
g(x)
X*
Zn(Xn) =
{sign (g(Xn)) with probability p(Xn),
−sign (g(Xn)) with probability 1− p(Xn).
• The probability of a correct sign p(·) depends on g(·) and the noise(εn)n.
• Stylized Setting:• p(·) is constant.• p(·) is known.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Revisited
0 1
0
g(x)
X*
0 10
0.5
1
p(x)X*
Zn(Xn) =
{sign (g(Xn)) with probability p(Xn),
−sign (g(Xn)) with probability 1− p(Xn).
• The probability of a correct sign p(·) depends on g(·) and the noise(εn)n.
• Stylized Setting:• p(·) is constant.• p(·) is known.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Revisited
0 1
0
g(x)
X*
0 10
0.5
1
p(x)X*
Zn(Xn) =
{sign (g(Xn)) with probability p(Xn),
−sign (g(Xn)) with probability 1− p(Xn).
• The probability of a correct sign p(·) depends on g(·) and the noise(εn)n.
• Stylized Setting:• p(·) is constant.• p(·) is known.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Revisited
0 1
0
g(x)
X*
0 10
0.5
1
p
p(x)X*
Zn(Xn) =
{sign (g(Xn)) with probability p(Xn),
−sign (g(Xn)) with probability 1− p(Xn).
• The probability of a correct sign p(·) depends on g(·) and the noise(εn)n.
• Stylized Setting:• p(·) is constant.
• p(·) is known.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stochastic Root-Finding Revisited
0 1
0
g(x)
X*
0 10
0.5
1
p
p(x)X*
Zn(Xn) =
{sign (g(Xn)) with probability p(Xn),
−sign (g(Xn)) with probability 1− p(Xn).
• The probability of a correct sign p(·) depends on g(·) and the noise(εn)n.
• Stylized Setting:• p(·) is constant.• p(·) is known.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 9/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stylized Setting• g(x) is a step function with a jump at X∗, for example, in edgedetection applications (Castro and Nowak, 2008).
• Sample sequentially at point Xn and use Sm(Xn) =∑m
i=1 Yn,i(Xn) toconstruct an α-level test of power 1 (Siegmund, 1985):
Nn = inf{m : |Sm| ≥ [(m + 1)(log(m + 1) + 2 log(1/α))]1/2
}.
Then PXn=X∗ {Nn <∞} ≤ α, PXn 6=X∗ {Nn <∞} = 1, andPXn<X∗ {SNn(Xn) > 0} ≥ 1− α/2 = pc ,PXn>X∗ {SNn(Xn) < 0} ≥ 1− α/2 = pc .
0 5 10 15 20m
Sm
(Xn)
0 10
0.5
1
p
p(x)X*
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 10/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Stylized Setting• g(x) is a step function with a jump at X∗, for example, in edgedetection applications (Castro and Nowak, 2008).
• Sample sequentially at point Xn and use Sm(Xn) =∑m
i=1 Yn,i(Xn) toconstruct an α-level test of power 1 (Siegmund, 1985):
Nn = inf{m : |Sm| ≥ [(m + 1)(log(m + 1) + 2 log(1/α))]1/2
}.
Then PXn=X∗ {Nn <∞} ≤ α, PXn 6=X∗ {Nn <∞} = 1, andPXn<X∗ {SNn(Xn) > 0} ≥ 1− α/2 = pc ,PXn>X∗ {SNn(Xn) < 0} ≥ 1− α/2 = pc .
0 5 10 15 20m
Sm
(Xn)
0 10
0.5
1
p
p(x)X*
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 10/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
The Probabilistic Bisection Algorithm (Horstein, 1963)
Notation: p(·) = pc ∈ (1/2, 1] and qc = 1− pc .
1. Place a prior density f0 on the root X∗, f0 has domain [0, 1].Example: U(0, 1).
2. For n=0,1,2, . . .(a) Measure at the median Xn := F−1n (1/2).(b) Update the posterior density:
if Zn(Xn) = +1, fn+1(x) ={
2pc · fn(x), if x > Xn,
2qc · fn(x), if x ≤ Xn,
if Zn(Xn) = −1, fn+1(x) ={
2qc · fn(x), if x > Xn,
2pc · fn(x), if x ≤ Xn.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 11/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
The Probabilistic Bisection Algorithm (Horstein, 1963)
Notation: p(·) = pc ∈ (1/2, 1] and qc = 1− pc .
1. Place a prior density f0 on the root X∗, f0 has domain [0, 1].Example: U(0, 1).
2. For n=0,1,2, . . .(a) Measure at the median Xn := F−1n (1/2).(b) Update the posterior density:
if Zn(Xn) = +1, fn+1(x) ={
2pc · fn(x), if x > Xn,
2qc · fn(x), if x ≤ Xn,
if Zn(Xn) = −1, fn+1(x) ={
2qc · fn(x), if x > Xn,
2pc · fn(x), if x ≤ Xn.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 11/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 0, Xn = 0.5, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 1, Xn = 0.61538, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 2, Xn = 0.70414, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 3, Xn = 0.63587, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 4, Xn = 0.55589, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 5, Xn = 0.46446, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 6, Xn = 0.35727, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 7, Xn = 0.43972, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 8, Xn = 0.51955, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 9, Xn = 0.45436, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 10, Xn = 0.39721, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 11, Xn = 0.33187, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 12, Xn = 0.25529, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 13, Xn = 0.3142, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 14, Xn = 0.37124, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 15, Xn = 0.32466, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 16, Xn = 0.3632, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 17, Xn = 0.33085, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 18, Xn = 0.30024, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 19, Xn = 0.25814, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 20, Xn = 0.20046, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 21, Xn = 0.24672, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 22, Xn = 0.27985, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 23, Xn = 0.31321, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 24, Xn = 0.28617, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 25, Xn = 0.2615, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 26, Xn = 0.28243, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 27, Xn = 0.29702, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 28, Xn = 0.32015, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 29, Xn = 0.33658, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 30, Xn = 0.36118, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 31, Xn = 0.38925, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 32, Xn = 0.43944, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 33, Xn = 0.39633, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 34, Xn = 0.42932, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 35, Xn = 0.4563, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 36, Xn = 0.43531, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 37, Xn = 0.41192, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 38, Xn = 0.43177, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 39, Xn = 0.41699, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 40, Xn = 0.39722, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 41, Xn = 0.41399, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 42, Xn = 0.4015, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 43, Xn = 0.41222, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 44, Xn = 0.40403, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 45, Xn = 0.41052, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 46, Xn = 0.40553, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 47, Xn = 0.39812, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 48, Xn = 0.37339, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 49, Xn = 0.35957, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 50, Xn = 0.36904, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 60, Xn = 0.36286, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 70, Xn = 0.37119, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 80, Xn = 0.37225, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 90, Xn = 0.37336, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 100, Xn = 0.3752, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 110, Xn = 0.37371, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 120, Xn = 0.3728, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 130, Xn = 0.37269, Z
n(X
n) = −1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 140, Xn = 0.37108, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Path of Posterior Distributions
0 1
fn(x)
n = 150, Xn = 0.37261, Z
n(X
n) = 1
X*
Xn
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 12/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Comparison to Stochastic Approximation
0 50 100 150 20010
−10
10−8
10−6
10−4
10−2
100
n
|X* − Xn|
Stochastic ApproximationProbabilistic Bisection
0 1
0
g(x)X*
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 13/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Literature Review: Probabilistic Bisection Algorithm
• First introduced in Horstein (1963).
• Discretized version: Burnashev and Zigangirov (1974).
• Feige et al. (1997), Karp and Kleinberg (2007), Ben-Or andHassidim (2008), Nowak (2008), Nowak (2009), ...
• Survey paper: Castro and Nowak (2008)
“The probabilistic bisection algorithm seems to workextremely well in practice, but it is hard to analyze andthere are few theoretical guarantees for it, especiallypertaining error rates of convergence.”
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 14/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Literature Review: Probabilistic Bisection Algorithm
• First introduced in Horstein (1963).
• Discretized version: Burnashev and Zigangirov (1974).
• Feige et al. (1997), Karp and Kleinberg (2007), Ben-Or andHassidim (2008), Nowak (2008), Nowak (2009), ...
• Survey paper: Castro and Nowak (2008)
“The probabilistic bisection algorithm seems to workextremely well in practice, but it is hard to analyze andthere are few theoretical guarantees for it, especiallypertaining error rates of convergence.”
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 14/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Algorithm Analysis
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 15/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Consistency
Setting for probabilistic bisection with power 1 tests:• X∗ ∈ [0, 1] fixed and unknown.• Xn 6= X∗ for any finite n ∈ N.• p(Xn) ≥ pc for all n ∈ N.• pc ∈ (1/2, 1) is an input parameter.
TheoremXn → X∗ almost surely as n→∞.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 16/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Consistency
Setting for probabilistic bisection with power 1 tests:• X∗ ∈ [0, 1] fixed and unknown.• Xn 6= X∗ for any finite n ∈ N.• p(Xn) ≥ pc for all n ∈ N.• pc ∈ (1/2, 1) is an input parameter.
TheoremXn → X∗ almost surely as n→∞.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 16/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Analysis of Posterior Density
0 1
• If Zn = +1 :
fn+1(x) = 2qc · fn(x), x < Xn,
fn+1(x) = 2pc · fn(x), x ≥ Xn,
• If Zn = −1 :
fn+1(x) = 2pc · fn(x), x < Xn,
fn+1(x) = 2qc · fn(x), x ≥ Xn.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 17/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Analysis of Posterior Density
Xn
0 1
• If Zn = +1 :
fn+1(x) = 2qc · fn(x), x < Xn,
fn+1(x) = 2pc · fn(x), x ≥ Xn,
• If Zn = −1 :
fn+1(x) = 2pc · fn(x), x < Xn,
fn+1(x) = 2qc · fn(x), x ≥ Xn.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 17/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Analysis of Posterior Density
Xn
0 1X*
Case I: If X∗ < Xn : P(Zn = +1) = p(Xn) ≥ pc• If Zn = +1 :
fn+1(x) = 2qc · fn(x), x < Xn,
fn+1(x) = 2pc · fn(x), x ≥ Xn,
• If Zn = −1 :
fn+1(x) = 2pc · fn(x), x < Xn,
fn+1(x) = 2qc · fn(x), x ≥ Xn.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 17/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Analysis of Posterior Density
Xn
0 1X*
Case II: If X∗ > Xn : P(Zn = +1) = 1− p(Xn) ≤ pc• If Zn = +1 :
fn+1(x) = 2qc · fn(x), x < Xn,
fn+1(x) = 2pc · fn(x), x ≥ Xn,
• If Zn = −1 :
fn+1(x) = 2pc · fn(x), x < Xn,
fn+1(x) = 2qc · fn(x), x ≥ Xn.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 18/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Analysis of Posterior Density cont.
• The dynamics of fn(x) are very complicated for almost all x ∈ [0, 1].
HOWEVER, the dynamics of fn(X∗) are rather simple:
fn+1(X∗) =
{2pc · fn(X∗), with probability p(Xn) ≥ pc ,2qc · fn(X∗), with probability q(Xn) ≤ qc .
• A sample path of fn(X∗) dominates a sample path of a coupledgeometric random walk (Wn)n with dynamics
Wn+1 =
{2pc ·Wn, with probability pc ,2qc ·Wn, with probability qc .
• The process fn(X∗) behaves almost like a geometric random walkindependently of (Xn)n. The goal is then to locate this geometricrandom walk efficiently!
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 19/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Analysis of Posterior Density cont.
• The dynamics of fn(x) are very complicated for almost all x ∈ [0, 1].HOWEVER, the dynamics of fn(X∗) are rather simple:
fn+1(X∗) =
{2pc · fn(X∗), with probability p(Xn) ≥ pc ,2qc · fn(X∗), with probability q(Xn) ≤ qc .
• A sample path of fn(X∗) dominates a sample path of a coupledgeometric random walk (Wn)n with dynamics
Wn+1 =
{2pc ·Wn, with probability pc ,2qc ·Wn, with probability qc .
• The process fn(X∗) behaves almost like a geometric random walkindependently of (Xn)n. The goal is then to locate this geometricrandom walk efficiently!
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 19/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Analysis of Posterior Density cont.
• The dynamics of fn(x) are very complicated for almost all x ∈ [0, 1].HOWEVER, the dynamics of fn(X∗) are rather simple:
fn+1(X∗) =
{2pc · fn(X∗), with probability p(Xn) ≥ pc ,2qc · fn(X∗), with probability q(Xn) ≤ qc .
• A sample path of fn(X∗) dominates a sample path of a coupledgeometric random walk (Wn)n with dynamics
Wn+1 =
{2pc ·Wn, with probability pc ,2qc ·Wn, with probability qc .
• The process fn(X∗) behaves almost like a geometric random walkindependently of (Xn)n. The goal is then to locate this geometricrandom walk efficiently!
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 19/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Analysis of Posterior Density cont.
• The dynamics of fn(x) are very complicated for almost all x ∈ [0, 1].HOWEVER, the dynamics of fn(X∗) are rather simple:
fn+1(X∗) =
{2pc · fn(X∗), with probability p(Xn) ≥ pc ,2qc · fn(X∗), with probability q(Xn) ≤ qc .
• A sample path of fn(X∗) dominates a sample path of a coupledgeometric random walk (Wn)n with dynamics
Wn+1 =
{2pc ·Wn, with probability pc ,2qc ·Wn, with probability qc .
• The process fn(X∗) behaves almost like a geometric random walkindependently of (Xn)n. The goal is then to locate this geometricrandom walk efficiently!
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 19/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Confidence Intervals for X ∗
• Notation: µ = pc ln 2pc + qc ln 2qc .• For α ∈ (0, 1), define
bn = nµ− n1/2(−0.5 lnα)1/2(ln 2pc − ln 2qc).
• DefineJn = conv(x ∈ [0, 1] : fn(x) ≥ ebn).
TheoremFor α ∈ (0, 1),
P(X∗ ∈ Jn) ≥ 1− α,
for all n ∈ N.
Proof:Application of Hoeffding’s inequality.
�
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 20/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Confidence Intervals for X ∗
• Notation: µ = pc ln 2pc + qc ln 2qc .• For α ∈ (0, 1), define
bn = nµ− n1/2(−0.5 lnα)1/2(ln 2pc − ln 2qc).
• DefineJn = conv(x ∈ [0, 1] : fn(x) ≥ ebn).
TheoremFor α ∈ (0, 1),
P(X∗ ∈ Jn) ≥ 1− α,
for all n ∈ N.
Proof:Application of Hoeffding’s inequality.
�
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 20/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Confidence Intervals for X ∗
• Notation: µ = pc ln 2pc + qc ln 2qc .• For α ∈ (0, 1), define
bn = nµ− n1/2(−0.5 lnα)1/2(ln 2pc − ln 2qc).
• DefineJn = conv(x ∈ [0, 1] : fn(x) ≥ ebn).
TheoremFor α ∈ (0, 1),
P(X∗ ∈ Jn) ≥ 1− α,
for all n ∈ N.
Proof:Application of Hoeffding’s inequality.
�
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 20/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Size of Confidence IntervalTheoremChoose pc ≥ 0.85, α ∈ (0, 1). For 0 < r < µ− qc ln 2pc there exists aN(pc , r , α) ∈ N, such that
P(|Jn| ≤ e−rn,X∗ ∈ Jn) ≥ 1− α,
for all n ≥ N(pc , r , α).
Proof Idea:
0 1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 21/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Size of Confidence IntervalTheoremChoose pc ≥ 0.85, α ∈ (0, 1). For 0 < r < µ− qc ln 2pc there exists aN(pc , r , α) ∈ N, such that
P(|Jn| ≤ e−rn,X∗ ∈ Jn) ≥ 1− α,
for all n ≥ N(pc , r , α).
Proof Idea:
0 1Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 21/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Rate of Convergence
TheoremDefine X̂n to be any point in Jn, then there exists r > 0 such that
E[|X∗ − X̂n|] = O(e−rn).
• This is extremely fast compared to stochastic approximation:
O(e−rn) vs. O(n−1/2).
• And we have true confidence intervals for X∗.• But n is the number of measurement points, what about totalwall-clock time?
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 22/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Rate of Convergence
TheoremDefine X̂n to be any point in Jn, then there exists r > 0 such that
E[|X∗ − X̂n|] = O(e−rn).
• This is extremely fast compared to stochastic approximation:
O(e−rn) vs. O(n−1/2).
• And we have true confidence intervals for X∗.• But n is the number of measurement points, what about totalwall-clock time?
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 22/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Rate of Convergence
TheoremDefine X̂n to be any point in Jn, then there exists r > 0 such that
E[|X∗ − X̂n|] = O(e−rn).
• This is extremely fast compared to stochastic approximation:
O(e−rn) vs. O(n−1/2).
• And we have true confidence intervals for X∗.
• But n is the number of measurement points, what about totalwall-clock time?
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 22/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Rate of Convergence
TheoremDefine X̂n to be any point in Jn, then there exists r > 0 such that
E[|X∗ − X̂n|] = O(e−rn).
• This is extremely fast compared to stochastic approximation:
O(e−rn) vs. O(n−1/2).
• And we have true confidence intervals for X∗.• But n is the number of measurement points, what about totalwall-clock time?
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 22/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Wall-Clock TimeAt each iteration of the Probabilistic Bisection Algorithm:
• Sample sequentially at point Xn and observeSm(Xn) =
∑mi=1 Yn,i(Xn), until
Nn = inf{m : |Sm| ≥ [(m + 1)(log(m + 1) + 2 log(1/α))]1/2
},
then PXn=X∗ {Nn <∞} ≤ α, PXn 6=X∗ {Nn <∞} = 1, and
PXn<X∗ {SNn(Xn) > 0} ≥ 1− α/2 = pc ,PXn>X∗ {SNn(Xn) < 0} ≥ 1− α/2 = pc .
• Wall-clock time: Tn =∑n
i=1 Nn.
0 5 10 15 20m
Sm
(Xn)
0 10
0.5
1
p
p(x)X*
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 23/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Sample Paths
100
101
102
103
104
105
−0.5
0
0.5
Tn
X* − Xn
Robbin−Monro, an = 1/n, ε
n ~ N(0,1)
100
101
102
103
104
105
−0.5
0
0.5
Tn
X* − Xn
Robbin−Monro, an = 1/n, ε
n ~ N(0,1)
100
101
102
103
104
105
−0.5
0
0.5
Tn
X* − Xn
Robbin−Monro, an = 1/n, ε
n ~ N(0,1)
100
101
102
103
104
105
−0.5
0
0.5
Tn
X* − Xn
Bisection, p = 0.75, εn ~ N(0,1)
100
101
102
103
104
105
−0.5
0
0.5
Tn
X* − Xn
Bisection, p = 0.75, εn ~ N(0,1)
100
101
102
103
104
105
−0.5
0
0.5
Tn
X* − Xn
Bisection, p = 0.75, εn ~ N(0,1)
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 24/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Numerical Comparison
100
101
102
103
104
105
10−4
10−3
10−2
10−1
100
Tn
E[|X
* −
Xn|]
Robbins−Monro (an=1/n)
Bisection, Siegmund (pc=0.85)
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 25/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Rate of Convergence in Wall-Clock Time?• Farrell (1964):
Eg(x)[N] ∼ (1/g(x))2 log log(1/|g(x)|) as g(x)→ 0,
and for all tests of power one, if P0(N =∞) > 0, then
limg(x)→0
g(x)2Eg(x)[N] =∞.
Theorem
1. limn→∞ E[|X∗ − Xn|(Tn)1/2] =∞.
2. (|X∗ − Xn|(Tn)1/2)n is not tight.
• Ifg(x)→ 0 as x → X∗,
and if we use Xn as the best estimate of X∗ then the ProbabilisticBisection Algorithm with power one tests is asymptotically slowerthan Stochastic Approximation.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 26/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Rate of Convergence in Wall-Clock Time?• Farrell (1964):
Eg(x)[N] ∼ (1/g(x))2 log log(1/|g(x)|) as g(x)→ 0,
and for all tests of power one, if P0(N =∞) > 0, then
limg(x)→0
g(x)2Eg(x)[N] =∞.
Theorem
1. limn→∞ E[|X∗ − Xn|(Tn)1/2] =∞.
2. (|X∗ − Xn|(Tn)1/2)n is not tight.
• Ifg(x)→ 0 as x → X∗,
and if we use Xn as the best estimate of X∗ then the ProbabilisticBisection Algorithm with power one tests is asymptotically slowerthan Stochastic Approximation.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 26/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Rate of Convergence in Wall-Clock Time?• Farrell (1964):
Eg(x)[N] ∼ (1/g(x))2 log log(1/|g(x)|) as g(x)→ 0,
and for all tests of power one, if P0(N =∞) > 0, then
limg(x)→0
g(x)2Eg(x)[N] =∞.
Theorem
1. limn→∞ E[|X∗ − Xn|(Tn)1/2] =∞.
2. (|X∗ − Xn|(Tn)1/2)n is not tight.
• Ifg(x)→ 0 as x → X∗,
and if we use Xn as the best estimate of X∗ then the ProbabilisticBisection Algorithm with power one tests is asymptotically slowerthan Stochastic Approximation.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 26/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Conjecture
• Xn might not be the best estimate for X∗ when we use power onetests.
• Intuitively, observations where we spend more time should also becloser to X∗, hence an estimator of the form
X̃n =1Tn
n∑i=1
NiXi
should perform better.
• Conjecture: For any β > 0 it holds that
E[|X̃n − X∗|] = O(T− 1
2(1+β)n ),
(if g satisfies some growth conditions).• Sufficient Condition: |Xn − X∗| = O(e−rn) for some r > 0.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 27/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Conjecture
• Xn might not be the best estimate for X∗ when we use power onetests.
• Intuitively, observations where we spend more time should also becloser to X∗, hence an estimator of the form
X̃n =1Tn
n∑i=1
NiXi
should perform better.• Conjecture: For any β > 0 it holds that
E[|X̃n − X∗|] = O(T− 1
2(1+β)n ),
(if g satisfies some growth conditions).
• Sufficient Condition: |Xn − X∗| = O(e−rn) for some r > 0.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 27/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Conjecture
• Xn might not be the best estimate for X∗ when we use power onetests.
• Intuitively, observations where we spend more time should also becloser to X∗, hence an estimator of the form
X̃n =1Tn
n∑i=1
NiXi
should perform better.• Conjecture: For any β > 0 it holds that
E[|X̃n − X∗|] = O(T− 1
2(1+β)n ),
(if g satisfies some growth conditions).• Sufficient Condition: |Xn − X∗| = O(e−rn) for some r > 0.
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 27/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Numerical Comparison Cont.
100
101
102
103
104
105
10−4
10−3
10−2
10−1
100
Tn
E[|X
* −
Xn|]
Robbins−Monro (an=1/n)
Bisection, Siegmund (pc=0.85)
Polyak−RuppertBisection Averaging
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 28/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Conclusions
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 29/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
ConclusionsPositive:
• Provides true confidence interval of the root X∗.
• Works extremely well if there is a jump at g(X∗) (geometric rate ofconvergence).
• Only one tuning parameter.
Drawbacks:• Seems to be asymptotically slower than Stochastic Approximation(but by not much).
• Higher computational cost.
Future Research:• Robustness of algorithm.
• Use parallel computing (very little switching of (Xn)n).
• Extension to higher dimensions.Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 30/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
THANK YOU!
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 31/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
M. Ben-Or and A. Hassidim (2008): The Bayesian learner is optimal for noisy binary search. In49th Annual IEEE Symposium on Foundations of Computer Science. IEEE, pp. 221–230.
M. Burnashev and K. Zigangirov (1974): An interval estimation problem for controlledobservations. Problemy Peredachi Informatsii 10:51–61.
R. Castro and R. Nowak (2008): Active learning and sampling. In Foundations andApplications of Sensor Management, A. O. Hero, D. A. Castañón, D. Cochran and K. Kastella,editors. Springer, pp. 177–200.
R. H. Farrell (1964): Asymptotic behavior of expected sample size in certain one sided tests.The Annals of Mathematical Statistics 35:36–72.
U. Feige, P. Raghavan, D. Peleg and E. Upfal (1997): Computing with noisy information.SIAM Journal of Computing :1001–1018.
M. Horstein (1963): Sequential transmission using noiseless feedback. IEEE Transactions onInformation Theory 9:136–143.
R. Karp and R. Kleinberg (2007): Noisy binary search and its applications. In Proceedings ofthe eighteenth annual ACM-SIAM symposium on Discrete algorithms. Society for Industrial andApplied Mathematics, pp. 881–890.
R. Nowak (2008): Generalized binary search. In 46th Annual Allerton Conference onCommunication, Control, and Computing . pp. 568–574.
R. Nowak (2009): Noisy generalized binary search. In Advances in neural information processingsystems, volume 22. pp. 1366–1374.
H. Robbins and S. Monro (1951): A stochastic approximation method. The Annals ofMathematical Statistics 22:400–407.
D. Siegmund (1985): Sequential Analysis: tests and confidence intervals. Springer.Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 32/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Tuning Parameters
100
101
102
103
104
105
10−4
10−3
10−2
10−1
100
n
Input Parameters d to Robbins−Monro (stepsize d/n), X* ~ U(0,1)
E[|X* − Xn|]
0.10.250.50.7511.5251020
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 32/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Tuning Parameters
100
101
102
103
104
105
10−4
10−3
10−2
10−1
100
n
Input Parameters pc to Bisection Algorithm, X* ~ U(0,1)
E[|X* − Xn|]
0.5250.5750.6250.6750.7250.7750.8250.8750.9250.975
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 32/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 0, Z = −1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 1, Z = −1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 2, Z = 1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 3, Z = 1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 4, Z = −1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 5, Z = 1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 6, Z = 1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 7, Z = −1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 8, Z = 1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 9, Z = 1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32
Rolf Waeber Probabilistic Bisection Search for Stochastic Root-Finding
Higher Dimensions
Boundary detection:
0 0.2 0.4 0.6 0.8 10
0.2
0.4
0.6
0.8
1n = 10, Z = 1
Stochastic Root-Finding Probabilistic Bisection Algorithm Analysis Conclusions References 33/32