algorithms lecture 10 lecturer: moni naor. linear programming in small dimension canonical form of...
TRANSCRIPT
Algorithms
Lecture 10
Lecturer: Moni Naor
Linear Programming in Small Dimension
Canonical form of linear programming
Maximize: c1 ¢x1 + c2 ¢ x2 … cd ¢xd
Subject to: a1,1 ¢ x1 + a1,2 ¢ x2 … a1,d ¢ xd · b1
a2,1 ¢ x1 + a2,2 ¢ x2 … a2,d ¢ xd · b1
...
an,1 ¢ x1 + an,2 ¢ x2 … an,d ¢ xd · bn
n – number of constraints andd – number of variables or dimension
Linear Programming in Two Dimensions
Feasible regionOptimal
vertex
What is special in low dimension
• Only d constraints determine the solution – The optimal value on those d constraints determine the
global one– Problem is reduced to finding those constraints that
matter– We know that equality hold in those constraints
• Generic algorithms:– Fourier-Motzkin: (n/2)2d – Worst case of Simplex: number of bfs/vertices (nd )
Key Observation• If we know that an inequality constraints is
defining: – we can reduce the number variables
• Projection • Substitution
Feasible regionOptimal
vertex
4X1 - 6x 2 =4
Incremental Algorithm BB
Input: A set of n constraints H on d variables Output: the set of defining constraints 0. If |H|=d output BB(H)=H1. Pick a random constraint If h 2 H
recursively find BB(H \ h)
2. If BB(H \ h) does not violate h output BB(H \ h) else project all the constraints onto h and
recursively solve this (n-1,d-1) lp program
Correctness, Termination and Analysis• Correctness: by induction…• Termination: if the non
defining constraints chosen– No need to rerun
• Analysis: probability that h is one the defining constraints is d/n
0. If |H|=d output BB(H)=H1. Pick a random constraint If
h 2 H recursively find BB(H \ h)
2. If BB(H \ h) does not violate h output BB(H \ h)
else project all the constraints onto h and recursively solve this (n-1,d-1) lp program
Analysis
Analysis: probability that h is one the defining constraints is d/n
T(d,n) = d/n T(d-1,n-1) + T(d,n-1)by induction= d/n (d-1)! (n-1) + d! (n-1)= (n-1)d!(1/n +1)· nd!
0. If |H|=d output BB(H)=H1. Pick a random constraint If h 2
H recursively find BB(H \ h)
2. If BB(H \ h) does not violate h output BB(H \ h)
else project all the constraints onto h and recursively solve this (n-1,d-1) lp program
How to improve
The algorithm is wasteful: When the solution does not fit
the new a new is computed from scratch
0. If |H|=d output BB(H)=H1. Pick a random constraint If
h 2 H recursively find BB(H \ h)
2. If BB(H \ h) does not violate h output BB(H \ h)
else project all the constraints onto h and recursively solve this (n-1,d-1) lp program
Random Sampling ideaBuild the basis by adding the constraints in a manner related to history Input: A set of n constraints H on d variables Output: the set of defining constraints 0. If |H|= c d2 return simplex on HS Ã Repeat• Pick random R ½ H of size r• Solve recursively on S [ R solution is u• V = set of constraints in H violated by u• If |V| · t, then S Ã S [ V Until V=
Correctness, Termination and Analysis
Claim: Each time we augment S (S Ã S [ V), we add to S a new constraint from the ``real” basis B of H– If u did not violate any constraint in B it would be optimal– So V must contain an element from B which was not in S before
• Since |B|=d, we can augment S at most d times• Therefore the number of constraints in the recursive call is
|R|+|S| · r +dt• Important factor for analysis: what is the probability of
successful augmentation
Sampling Lemma
For any H and S ½ H The expected (over R) number of constraints V that violate u (optimum on S [ R) is at most nd/r
ProofLet X(R,h) be 1 iff h violated h(S [ R)Need to bound ER [h X(R,h)] = 1 / #R |R|=r h X(R,h) instead consider all subsets Q = R [ h of size r+1= 1 / #R |Q|=r+1 h2 Q X(Q\{h},h)
= (#Q/#R) (r+1)¢ ProbQ,h 2 Q X(Q\{h},h)· n ¢ d / (r+1)
Analysis• Setting t= 2 nd /r implies (from Markov’s inequality):
– number of recursive call until a successful (V · t) augmentation is constantNumber of constraints in recursive call bounded by r+O(d2n/r)
Setting r=d n1/2 means that this is O(r)Total expected running timeT(n) · 2 d T(d n1/2 ) + O(d2 n)
Result O( (log n)log d (Simplex time) ) + O(d2 n)
Can be improved to O(dd +d2n)
Can be improved to O(dd1/2 +d2n) using [Kalai, Matousek-Sharir-Welzl]
References
• Motwani and Raghavan, Randomized Algorithms Chapter 9.10
• Michael Goldwasser, A Survey of Linear Programming in Randomized Subexponential Time http://euler.slu.edu/~goldwasser/publications/SIGACT1995_Abstract.html
• Pioter Indyk’s course at MIT, Geometric Computationhttp://theory.lcs.mit.edu/~indyk/6.838/
• Applet: http://web.mit.edu/ardonite/6.838/linear-programming.htm