probability and randomized algorithms
Post on 07-Apr-2018
225 Views
Preview:
TRANSCRIPT
-
8/6/2019 Probability and Randomized Algorithms
1/14
Probability and
Randomized AlgorithmsProf. Zeph Grunschlag
-
8/6/2019 Probability and Randomized Algorithms
2/14
-
8/6/2019 Probability and Randomized Algorithms
3/14
3
Joint and Conditional
Probability Let X, Y be random variables over the resp.
setsX, Y. (Note,X, Y may/may not be same)
DEF: Joint probabilityPr[x,y] is theprobability that (X,Y) = (x,y). (Probability ofboth occurring simultaneously)
DEF: Conditional probabilityis definedby Pr[x|y] = Pr[x,y] / Pr[y] - assuming thatPr[y] > 0.
-
8/6/2019 Probability and Randomized Algorithms
4/14
4
Independent Variables Random variables are independent if their
probabilities dont depend on each others
values:DEF: X and Y are independent ifPr[x,y] = Pr[x]Pr[y] for allx, y.
LEMMA: Equivalently, X and Y areindependent if (excluding 0-prob.y)x X,y Y,Pr[x|y] = Pr[x]
-
8/6/2019 Probability and Randomized Algorithms
5/14
5
Bayes Theorem
THM: If Pr[y] > 0 then
Pr[x|y] = Pr[y|x] Pr[x] / Pr[y]
-
8/6/2019 Probability and Randomized Algorithms
6/14
6
Binomial Rand. Var.DEF: The product of random variables X, Y isthe random variable XY defined onXYwith distribution Pr[(x,y)] = Pr[x]Pr[y].
Assume X a random variable on {0,1} andlet p = Pr[X=1], q = Pr[X=0]
Repeat experiment n times. I.e., take nindependent copies:
result called Binomial random variableBernoullis Thm:
X1X2 Xn
Pr
n
!i=1
Xi= k
=
n
k
pkqnk
-
8/6/2019 Probability and Randomized Algorithms
7/14
7
Expectation
The average value taken on by a function fon probability distribution X
DEF: The expectation offis defined by:
THM:
COR: For n repetitions of a Binomial randomvariable X consider sum S which counts thenumber outcomes = 1. Then E(S) = np
E(f) = !xX
f(x)px
E(f+g) = E(f) +E(g)
-
8/6/2019 Probability and Randomized Algorithms
8/14
8
Chernoff Bound
Estimates probability that sum of Binomialexperiment deviate from expected sum np
THM:
Note: probability that sum too big falls offexponentially with n
Pr
S (1+!)pn
e
!2
3pn
-
8/6/2019 Probability and Randomized Algorithms
9/14
9
Randomized AlgorithmsEquivalent formulations:
Turing machine with coin flips at everystep of computation
Non-deterministic Turing machine withprobability distribution over computationbranches
Nomenclature (varies from author to author):
Monte-Carlo:
Colloquially any randomized algorithm Complexity theory: NOs always right Las-Vegas: always correct, but may fail BPP: answers correct most of the time
-
8/6/2019 Probability and Randomized Algorithms
10/14
10
Monte Carlo Algorithm
False negative allowed, but no false positivesDEF: A poly-time Monte Carlo algorithmfor the decision problem Pis a poly-time non-
deterministic Turing machine (NDTM) s.t.
Probability measured over coin-flips in TMor equivalently, by taking the ratio ofaccepting branches in NTM to total number
Defines complexity class RP Rand-Poly
Pr[x is accepted] :
1
2x P
= 0 x P
-
8/6/2019 Probability and Randomized Algorithms
11/14
11
Las Vegas Algorithm
Symmetric version of Monte Carlo - no falsenegatives nor false positives but can fail
DEF: A poly-time Las Vegas algorithm is
a poly-time NDTM with a constant >0 forwhich Pr[fail] for all inputs.
Repeat algorithm to make arbitrarily small
Gives class ZPP Zero-Prob-of-error-Poly ZPP= RP co-RP
-
8/6/2019 Probability and Randomized Algorithms
12/14
12
Class BPP
BPP = Bounded-Prob-of-error-Poly Most general class - allow false negatives and
positives. Compensate by insisting answercorrect significantly more than half the time
DEF: A poly-time randomized algorithm forthe decision problem Pis a poly-time NDTMwith a constant >0 for which
Chernoff bound implies may assume =0.25
Pr[x is accepted] : 12 + ! x P 1
2 ! x P
-
8/6/2019 Probability and Randomized Algorithms
13/14
13
Pseudo Random
Sequence
DEF: A pseudo random sequence is adeterministic algorithm from finite bitstringsto infinite bitstrings whose outputs cannot bedistinguished from a random strings by any
BPP algorithm.
-
8/6/2019 Probability and Randomized Algorithms
14/14
14
-bias Detector
Given: A black box fwhich is known a-priori to have some built-in bias in anunknown direction.
Decide: Which direction the bias is in.n =
x = output of length n fromf
c = number of 1s in x
return (c > n/2) // YES if1-bias, NO if 0-bias
Pr[output is correct] > 3/4 thereforethis problem is in BPP so -biassequences are notpseudorandom.
2
(12
!)!2
top related