1 fingerprint 2 verifying set equality verifying set equality v string matching – rabin-karp...
Post on 20-Dec-2015
236 views
TRANSCRIPT
2
Verifying set equalityVerifying set equality
String Matching – Rabin-Karp AlgorithmString Matching – Rabin-Karp Algorithm
16
Density of PrimesDensity of Primes
(x) = número de primos menores ou iguais a x(x) = número de primos menores ou iguais a x (13) = 6(13) = 6– Primos < = do que 13 = 2, 3, 5 , 7, 11 e 13Primos < = do que 13 = 2, 3, 5 , 7, 11 e 13
O valor de O valor de não muda até chegarmos ao não muda até chegarmos ao próximo primo.próximo primo. (13) = (13) = (14) = (14) = (15) = (15) = (16) (16) – Ou seja, Ou seja, aumenta em salto de 1, mas o intervalo aumenta em salto de 1, mas o intervalo
entre esses saltos é irregularentre esses saltos é irregular
17
Density of PrimesDensity of Primes
EEsses intervalos tornam-se cada vez maiores, isto sses intervalos tornam-se cada vez maiores, isto é, a chance de um inteiro escolhido ao acaso ser é, a chance de um inteiro escolhido ao acaso ser
primo diminui quando avançamos para os primo diminui quando avançamos para os números maiores. números maiores.
PERGUNTA:PERGUNTA: O valor de O valor de não poderia ser aproximado por alguma não poderia ser aproximado por alguma
função conhecida?função conhecida?
18
Density of PrimesDensity of Primes
Para um valor elevado de x, Para um valor elevado de x,
(x) ~(x) ~ x/ ln x x/ ln x . .
Ou seja, Ou seja,
lim lim (x)(x) = 1 = 1x x x/ln x x/ln x
23
String MatchingString Matching
Many applications
–While using editor/word processor/browser
–Login name & password checking
–Virus detection
–Header analysis in data communications
–DNA sequence analysis
33
Primality testingPrimality testing A natural number A natural number nn is prime iff the only is prime iff the only
natural numbers dividing natural numbers dividing nn are are 11 and and nn
34
Primality testingPrimality testing A natural number A natural number nn is prime iff the only is prime iff the only
natural numbers dividing natural numbers dividing nn are are 11 and and nn The following are prime: The following are prime: 22,, 3 3,, 5 5,, 7 7,, 11 11,, 13 13,,
35
Primality testingPrimality testing A natural number A natural number nn is prime iff the only is prime iff the only
natural numbers dividing natural numbers dividing nn are are 11 and and nn The following are prime: The following are prime: 22,, 3 3,, 5 5,, 7 7,, 11 11,, 13 13, ,
……aand so are nd so are 12997091299709,,
1548586315485863,,
2280176348922801763489, …, …
36
Primality testingPrimality testing A natural number A natural number nn is prime iff the only is prime iff the only
natural numbers dividing natural numbers dividing nn are are 11 and and nn The following are prime: The following are prime: 22,, 3 3,, 5 5,, 7 7,, 11 11,, 13 13, ,
……aand so are nd so are 12997091299709,,
1548586315485863,,
2280176348922801763489, …, …
There is an infinite number of prime numbersThere is an infinite number of prime numbers
37
Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.
38
Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.
Let p1, p2, … pk be all primes.
Let n = p1 p2 … pk+1,
39
Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.
Let p1, p2, … pk be all primes.
Let n = p1 p2 … pk+1,
n must be composite.
40
Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.
Let p1, p2, … pk be all primes.
Let n = p1 p2 … pk+1,
n must be composite.
there exists a prime p s.t. p | n (fund theo. arithmetic), and p cannot be any of the p1, p2, … pk
41
Primality testingPrimality testingThere is an infinite number of prime numbersThere is an infinite number of prime numbersProof: Let us suppose the number of primes is Finite.
Let p1, p2, … pk be all primes.
Let n = p1 p2 … pk+1,
n must be composite.
there exists a prime p s.t. p | n (fund theo. arithmetic), and p cannot be any of the p1, p2, … pk
Therefore, p1, … pk were not all the prime numbers.
42
Some questions?Some questions? Is Is 22101101-1-1==25353012004564588029934064107512535301200456458802993406410751
prime?prime? How do we check whether a number is How do we check whether a number is
prime?prime? How do we generate huge prime numbers?How do we generate huge prime numbers? Why do we care?Why do we care?
43
Some questions?Some questions? Is Is 22101101-1-1==25353012004564588029934064107512535301200456458802993406410751
prime?prime? How do we check whether a number is How do we check whether a number is
prime?prime? How do we generate huge prime numbers?How do we generate huge prime numbers? Why do we care?Why do we care?
44
Some questions?Some questions? Is Is 22101101-1-1==25353012004564588029934064107512535301200456458802993406410751
prime?prime? How do we check whether a number is How do we check whether a number is
prime?prime? How do we generate huge primeHow do we generate huge prime numbersnumbers?? Why do we care?Why do we care?
45
Some questions?Some questions? Is Is 22101101-1-1==25353012004564588029934064107512535301200456458802993406410751
prime?prime? How do we check whether a number is How do we check whether a number is
prime?prime? How do we generate huge primeHow do we generate huge prime numbersnumbers?? Why do we care?Why do we care?
46
Naïve solution: Finding Naïve solution: Finding the the smallest divisorsmallest divisor
of nof n
– For i=2,..., n do For i=2,..., n do Divide n by i until Divide n by i until n mod i = 0n mod i = 0
Check if i is a divisor of n for some i = 2, ..., n
48
An improvementAn improvement
Check if i is a divisor of n for some i = 2, ..., n
Why can we do that?
49
Theorem:Theorem: Composit numbers have a divisor Composit numbers have a divisor bellow their square root bellow their square root
50
Theorem:Theorem: Composit numbers have a divisor Composit numbers have a divisor bellow their square root bellow their square root
Proof Idea: n composite n = ab, 0 < a b < n
51
Theorem:Theorem: Composit numbers have a divisor Composit numbers have a divisor bellow their square root bellow their square root
Proof Idea: n composite n = ab, 0 < a b < n
a sqrt(n)
52
Theorem:Theorem: Composit numbers have a divisor Composit numbers have a divisor bellow their square root bellow their square root
Proof Idea: n composite n = ab, 0 < a b < n
a sqrt(n)
Otherwise, we obtain ab > n (contradiction!!)
53
Is there a more efficient Is there a more efficient way of checking primality?way of checking primality?
54
Is there a more efficient Is there a more efficient way of checking primality?way of checking primality?
Yes! At least if we are willing to accept
a tiny probability of error.
55
We can prove that a number is not We can prove that a number is not primeprime withoutwithout explicitly finding a explicitly finding a divisordivisor of it of it
Is there a more efficient Is there a more efficient way of checking primality?way of checking primality?
Yes! At least if we are willing to accept
a tiny probability of error.
56
We can prove that a number is not We can prove that a number is not primeprime withoutwithout explicitly finding a explicitly finding a divisordivisor of it of it
Is there a more efficient Is there a more efficient way of checking primality?way of checking primality?
Yes! At least if we are willing to accept
a tiny probability of error.
RANDOMNESS IS USEFUL IN COMPUTATION
57
The The FermatFermat Primality Test Primality Test
FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the
inteintegger er aa, then:, then:
a a p-1p-11(mod p1(mod p))
58
Suppose that ra e sa have are the same modulo p, then we have r = s (mod p) Contradiction!!
Aa, 2a, 3a, ..., (p-1)a quando divididos por p possuem restos diferentes:1, 2, ..., p-1
Proof: • List the first p-1 positive multiple of a:a, 2a, 3a, 4a, ..., (p-1) a
The The FermatFermat Primality Test Primality Test
FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the
inteintegger er aa, then:, then:
a a p-1p-11(mod p1(mod p))
59
Suppose that ra and sa are the same modulo p, then we have r = s (mod p) Contradiction!!
Aa, 2a, 3a, ..., (p-1)a quando divididos por p possuem restos diferentes:1, 2, ..., p-1
Proof: • List the first p-1 positive multiple of a:a, 2a, 3a, 4a, ..., (p-1) a
The The FermatFermat Primality Test Primality Test
FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the
inteintegger er aa, then:, then:
a a p-1p-11(mod p1(mod p))
60
Suppose that ra and sa are the same modulo p, then we have r = s (mod p) Contradiction!!
Aa, 2a, 3a, ..., (p-1)a when divided by p have the different reminders:1, 2, ..., p-1
Proof: • List the first p-1 positive multiple of a:a, 2a, 3a, 4a, ..., (p-1) a
The The FermatFermat Primality Test Primality Test
FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the
inteintegger er aa, then:, then:
a a p-1p-11(mod p1(mod p))
61
a(p-1)(p-1)! = (p-1)! (mod p)
Proof: a. 2a. 3a. ... . (p-1)a 1. 2. 3. ... . (p-1) (mod p)
The The FermatFermat Primality Test Primality Test
FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the
inteintegger er aa, then:, then:
a a p-1p-11(mod p1(mod p))
62
a(p-1)(p-1)! = (p-1)! (mod p)
Dividing by (p-1)! we get the result
Proof: a. 2a. 3a. ... . (p-1)a 1. 2. 3. ... . (p-1) (mod p)
The The FermatFermat Primality Test Primality Test
FermatFermat’s little theorem’s little theorem::If If pp is a prime and p does not divide the is a prime and p does not divide the
inteintegger er aa, then:, then:
a a p-1p-11(mod p1(mod p))
63
A Corollary:A Corollary:
If If pp is a prime then, for any integer is a prime then, for any integer a:a:
aapp a (mod p) a (mod p)
64
A Corollary:A Corollary:
If If pp is a prime then, for any integer is a prime then, for any integer a:a:
aapp a (mod p) a (mod p)
The result is trivial if p divides a: a(ap-1 – 1) 0 (mod p)
If a does not divide a, then we need only multiply the congruence in Fermat´s little theorem by a
to complete the proof
65
A Corollary:A Corollary:
If If pp is a prime then, for any integer is a prime then, for any integer a:a:
aapp a (mod p) a (mod p)
The result is trivial if p divides a: a(ap-1 – 1) 0 (mod p)
If p does not divide a, then we need only multiply the congruence in Fermat´s little theorem by a
to complete the proof
66
Corollary:Corollary:
If If aan n ≠≠a a (mod n)(mod n) , , for some for some aa, , then then nn is not a prime! is not a prime!
Such an a is a Such an a is a witnesswitness to the to the compositeness of compositeness of nn..
The Fermat Test:
Do 100 times:
Pick a random 1<a<n and compute an (mod n).
If an a (mod n), then n is not a prime.
If all 100 tests passed, declare n to be a prime.
69
Is the Fermat test correct?Is the Fermat test correct?
If the Fermat test says that a number If the Fermat test says that a number nn is composite, is composite,then the number then the number nn is indeed a composite number. is indeed a composite number.
If If n n is a prime number, the Fermat test will always say that is a prime number, the Fermat test will always say that nn is is prime. prime.
But,
Can the Fermat test say that a composite Can the Fermat test say that a composite number is prime?number is prime?
What is the What is the probabilityprobability that this will happen? that this will happen?
70
Carmichael numbersCarmichael numbersA composite number n is a Carmichael number
iff an a (mod n) for every integer a.
The first Carmichael numbers are: 561, 1105, 1729, 2465, 2821, 6601, 8911, 10585, 15841, …
On Carmichael numbers, the Fermat test is always wrong!
Carmichael numbers are fairly rare.
71
Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.
72
Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.
Proof: Consider Z*n = {1, 2, ..., n-1}
Let B={x / x Z*n and xn-1 1 (mod n)}
We are going to show that B is subgroup of Z*n
For this:
1. 1 B2. x1, x2 B x1. x2 B3. x B x-1 B
73
Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.
Proof: Consider Z*n = {1, 2, ..., n-1}
Let B={x / x Z*n and xn-1 1 (mod n)}
We are going to show that B is subgroup of Z*n
For this:
1. 1 B : 1n-1 1 (mod n)2. x1, x2 B x1. x2 B3. x B x-1 B
74
Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.
Proof: Consider Z*n = {1, 2, ..., n-1}
Let B={x / x Z*n and xn-1 1 (mod n)}
We are going to show that B is subgroup of Z*n
For this:
1. 1 B : 1n-1 1 (mod n)2. x1, x2 B x1. x2 B3. x B x-1 B
(x1)n-1 1 (mod n)(x2)n-1 1 (mod n)
(x1.x2)n-1 1 (mod n)
75
Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.
Proof: Consider Z*n = {1, 2, ..., n-1}
Let B={x / x Z*n and xn-1 1 (mod n)}
We are going to show that B is subgroup of Z*n
For this:
1. 1 B : 1n-1 1 (mod n)2. x1, x2 B x1. x2 B3. x B x-1 B
(1)n-1 1 (mod n)(x.x-1)n-1 1 (mod n)
(x-1)n-1 1 (mod n)
76
Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.
Proof: Pr(xn-1 1 (mod n)) = Pr(x B)
It can be proved that 1 B and n-1 B and therefore, |B| 2Since the order of a subgroup divides the subgroup we have that
|B| |Z*n | / 2
Pr(x B) 1/2
77
Theorem: (Rabin ’77) If n is a composite number that is not a Carmichael number, then at least half of the numbers between 1 and n are witnesses to the compositeness of n.
Corollary: Let n be a composite number that is not a Carmichael number. If we pick a random number a, 1<a<n, then a is a witness with a probability of at least a 1/2 !
78
““Correctness” of the Correctness” of the Fermat testFermat test
If If nn is prime, the Fermat test is always is prime, the Fermat test is always rightright.. If If nn is a Carmichael number, is a Carmichael number,
the Fermat test is always the Fermat test is always wrong!wrong! If If nn is composite number that is not a Carmichael number, the Fermat test is wrong with a probability of is composite number that is not a Carmichael number, the Fermat test is wrong with a probability of
at most at most 22-100-100
Is an error probability of 2-100 acceptable?Yes!
79
The Rabin-Miller testThe Rabin-Miller test
A fairly simple modification of the Fermat test that A fairly simple modification of the Fermat test that is correct with a probability of at least 1-2is correct with a probability of at least 1-2-100-100 also on also on Carmichael numbers.Carmichael numbers.
Will not be covered in this course.Will not be covered in this course.
80
A probabilistic algorithmA probabilistic algorithmAn algorithm that uses random choices but
outputs the correct result, with high probability,for every input!
Randomness is a very useful algorithmic tool.
Up to 2002, there were no efficient deterministic primality testing algorithms.
In 2002, Agarwal, Kayal and Saxena found a fast deterministic primality testing algorithm.
81
Finding large prime Finding large prime numbersnumbers
The prime number Theorem:
The number of prime numbers smaller than n is asymptotically n / ln n.
Thus, for every number n, there is “likely” to be a prime number between n and n + ln n.
To find a prime number roughly the size of n, simply test n, n+2, n+4, … for primality.
82
Primality testing versus Primality testing versus FactoringFactoring
Fast primality testing algorithms determine that a number n is composite without Fast primality testing algorithms determine that a number n is composite without finding any of its factors.finding any of its factors.
No efficient factoring algorithms are known.No efficient factoring algorithms are known. Factoring a number is believed to be a much harder task.Factoring a number is believed to be a much harder task.
Primality testing - EasyFactoring - Hard
But, factoring is not that hard on a quantum computer!