lecture 17-cs648
TRANSCRIPT
Randomized AlgorithmsCS648
Lecture 17Miscellaneous applications of Backward analysis
1
MINIMUM SPANNING TREE
2
Minimum spanning tree
3
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6
Minimum spanning tree
Algorithms: โข Primโs algorithmโข Kruskalโs algorithmโข Boruvkaโs algorithm
4
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6
Less known but it is the first algorithm for MST
Minimum spanning tree
: undirected graph with weights on edges, .
Deterministic algorithms:Primโs algorithm 1. O(( + ) log ) using Binary heap2. O( + log ) using Fibonacci heapBest deterministic algorithm: O( + ) bound
โ Too complicated to design and analyzeโ Fails to beat Primโs algorithm using Binary heap
5
Minimum spanning tree
When finding an efficient solution of a problem appears hard, one should strive to design an efficient verification algorithm.
MST verification algorithm: [King, 1990]Given a graph and a spanning tree , it takes O( + ) time todetermine if is MST of .
Interestingly, no deterministic algorithm for MST could use this algorithm to achieve O( + ) time.
6
Minimum spanning tree
: undirected graph with weights on edges, .
Randomized algorithm:Karger-Klein-Tarjanโs algorithm [1995]1. Las Vegas algorithm2. O( + ) expected timeThis algorithm uses โข Random samplingโข MST verification algorithmโข Boruvkaโs algorithmโข Elementary data structure
7
Minimum spanning tree
: undirected graph with weights on edges, .
Randomized algorithm:Karger-Klein-Tarjanโs algorithm [1994]1. Las Vegas algorithm2. O( + ) expected time
โข Random sampling :How close is MST of a random sample of edges to MST of original graph ?
The notion of closeness is formalized in the following slide.
8
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?Answer: ??
9
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
ยฟ๐๐
(๐โ๐)
USING BACKWARD ANALYSIS FORMISCELLANEOUS APPLICATIONS
10
PROBLEM 1SMALLEST ENCLOSING CIRCLE
11
Smallest Enclosing Circle
12
Smallest Enclosing Circle
Question: Suppose we sample points randomly uniformly from a set of points, what is the expected number of points that remain outside the smallest circle enclosing the sample?
13
For = , the answer is
PROBLEM 2SMALLEST LENGTH INTERVAL
14
0 1
Sampling points from a unit interval
Question: Suppose we select points from interval [0,1], what is expected length of the smallest sub-interval ?โข for , it is ?? โข for , it is ??
โข General solution : ??
This bound can be derived using two methods.โข One method is based on establishing a relationship between uniform
distribution and exponential distribution.โข Second method (for nearly same asymptotic bound) using Backward
analysis.
๐
(๐+๐ )๐
PROBLEM 3MINIMUM SPANNING TREE
16
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?
17
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
USING BACKWARD ANALYSIS FORTHE 3 PROBLEMS :
A GENERAL FRAMEWORK
18
A General framework
Let be the desired random variable in any of these problems/random experiment.
โข Step 1: Define an event related to the random variable .
โข Step 2: Calculate probability of event using standard method based on definition. (This establishes a relationship between )
โข Step 3: Express the underlying random experiment as a Randomized incremental construction and calculate the probability of the event using Backward analysis.
โข Step 4: Equate the expressions from Steps 1 and 2 to calculate E[].
19
PROBLEM 3MINIMUM SPANNING TREE
20
A BETTER UNDERSTANDING OF LIGHT EDGES
21
Minimum spanning tree
22
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
13
4 2
19
6
x
b
ac
d
h
y
u
v
181
19
22
10 15
11
5
16
4
31
31
Random sampling
(๐ฝ ,๐ฌ )
(๐ฝ ,๐น)
Minimum spanning tree
23
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
13
4 2
19
6
x
b
ac
d
h
y
u
v
1
19
10 15
11
5
16
4
31
18
2231
MST()
(๐ฝ ,๐ฌ )
(๐ฝ ,๐น)
Minimum spanning tree
24
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
1
19
10 15
11
5
16
4
7
342
3
17
23
2
19
6
31
MST()
(๐ฝ ,๐ฌ )
(๐ฝ ,๐น)๐ฌ ยฟ
Minimum spanning tree
25
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
7
1
19
10
342
3
15
11
5
16
17
23
4 2
19
6
31
MST()
(๐ฝ ,๐ฌ )
(๐ฝ ,๐น)๐ฌ ยฟ
Light
First useful insight
Lemma1: An edge is light with respect to if and only if belongs to MST().
26
Minimum spanning tree
27
b
ac
d
h x
y
u
v
18
7
1
19
22
10
342
3
15
11
5
16
17
23
4 2
19
6
b
ac
d
h x
y
u
v
7
1
19
10
342
3
15
11
5
16
17
23
4 2
19
6
31
MST()
(๐ฝ ,๐ฌ )
(๐ฝ ,๐น)๐ฌ ยฟ
Light heavy
Minimum spanning tree
28
b
ac
d
h x
y
u
v
7
1
3 3
54 2
6
b
ac
d
h x
y
u
v
7
1
19
10
3 3
15
11
5
16
17
4 2
42
23
19
6
MST()
(๐ฝ ,๐ฌ )
(๐ฝ ,๐น)๐ฌ ยฟ
Light heavy
MST()
Is there any relationship among MST(), MST()
and Light edges from ?
Second useful insight
Lemma2: Let โข and โข be the set of all edges from that are light with respect to . Then
MST() = MST()
This lemma is used in the design of randomized algorithm for MST as follows (just a sketch):โข Compute MST of a sample of edges (recursively). Let it be Tโ.โข There will be expected edges light edges among all unsampled edges.โข Recursively compute MST of Tโ edges which are less than on expectation.
29
Light Edge
Definition: Let . An edge is said to be light with respect to if
Question: If and ||= , how many edges from are light with respect to on expectation ?
30
b
ac
d
h x
y
u
v
18
7
1
19
22
10
312
3
15
11
5
16
17
13
4 2
9
6 31
MST() MST()
We shall answer the above question using the Generic framework. But before that, we need to get a better understanding of the
corresponding random variable.
31
๐
๐น
๐ฌ ยฟMST()
32
๐
๐น
๐ฌ ยฟMST()
Light
33
๐
๐น
๐ฌ ยฟLight heavy
MST()
: random variable for the number of light edges in when is a random sample of edges.
: set of all subsets of of size . : number of light edges in when . = ??
34
๐ยฟ๐บโจยฟ โ
๐โ๐บ
๐ (๐ ) ยฟCan you express in terms of
and only ?
Step 1
Question: Let be a uniformly random sample of edges from .What is the prob. that an edge selected randomly from is a light edge ?
35
Two methods to find
Step 2
Calculating using definition
36
Step 2
Calculating using definition
37
๐
๐
๐ฌ ยฟMST()
Light heavy
Light edges=
Step 2
Calculating using definition
: set of all subsets of of size .The probability is equal to
38
๐ (๐)๐โ๐
๐ยฟ๐บโจยฟยฟ
Step 3
Expressing the entire experiment as Randomized Incremental Construction
A slight difficulty in this process is the following:โข The underlying experiment talks about random sample from a set.โข But RIC involves analyzing a random permutation of a set of elements. Question: What is relation between random sample from a set and a random permutation of the set ?
Spend some time on this question before proceeding further.
39
random sample and random permutation
Observation: is indeed a uniformly random sample of
40
Random permutation of
๐ ๐โ๐๐จ ๐ฌ ยฟ
Step 3
The underlying random experiment as Randomized Incremental Construction: โข Permute the edges randomly uniformly.โข Find the probability that th edge is light relative to the first edges.
Question: Can you now calculate probability ?
Spend some time on this question before proceeding further.
41
Step 3
42
Random permutation of
๐๐๐๐๐๐ โฆ
๐ฌ ๐โ๐
Step 3
: a random variable taking value 1 if is a light edge with respect to MST().
43
Random permutation of
๐๐๐๐๐๐ โฆ
๐ฌ ๐โ๐ ๐ฌ {๐ฌ ยฟ๐โ๐
Step 3
: a random variable taking value 1 if is a light edge with respect to MST().
Question: What is relation between and โs?Answer: ??
44
Random permutation of
๐๐๐๐๐๐ โฆ
๐ฌ ๐โ๐ ๐ฌ {๐ฌ ยฟ๐โ๐
๐=๐ยฟยฟ
Calculating ).
: set of all subsets of of size . ) =
depends upon
45
Forward analysis
๐๐๐๐๐๐ โฆ
๐ฌ ๐โ๐
MST()
Random permutation of
Calculating ).
: set of all subsets of of size . )=
46
Backward analysis
๐๐๐๐๐๐ โฆ
๐ฌ ๐
Random permutation of
๐ยฟ ยฟ
= ??
??
47
Backward analysis
๐๐๐๐๐๐ โฆ
๐ฌ ๐
MST()
ยฟMST (๐)โจยฟ๐ยฟ
Random permutation of
Use Lemma 2.
๐ ( ๐ thedge๐๐๐ฅ๐จ๐ง๐ ๐ฌ ยฟMST (๐))
Calculating )
: set of all subsets of of size . )=
48
Backward analysis
๐๐๐๐๐๐ โฆ
๐ฌ ๐
Random permutation of
Combining the two methods for calculating
Using method 1:
Using method 2:
)
Hence:
49
Theorem: If we sample edges uniformly randomly from an undirected graph on vertices and edges, the number of light edges among the unsampled edges will be less than on expectation.
50