recurrence relation. outline what is a recurrence relation ? solving linear recurrence relations ...
TRANSCRIPT
Recurrence Relation
Outline
What is a recurrence relation ?
Solving linear recurrence relations
Divide-and-conquer algorithms and recurrence relations
What is a recurrence relation ?
Definition
A recurrence relation for the sequence {an} is an equation that expresses an in terms of one or more of the previous terms of the sequence, namely a0, a1, …, an-1, for all integers n n0, where n0 is a non-negative integer.
A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation.
Example: Given the recurrence relation an=2an-1 -an-2.
3, 5, 7, 9, 11, … satisfies the recurrence relation.
2, 3, 4, 5, 6, … also satisfies the recurrence relation.
Example: Compound interest
Let Pn be the amount of money in the account after n years. If you initially deposit Pn=1000 in your account and the bank gives k % per year with interest compounded annually.
Pn = (1+k/100) Pn-1
P1 = (1+k/100) P0 = (1+k/100) P0
P2 = (1+k/100) P1 = (1+k/100)2 P0
P3 = (1+k/100) P2 = (1+k/100)3 P0
Pn = (1+k/100) Pn-1 = (1+k/100)n P0
Example: Fibonacci numbers
A pair of rabbits is placed on an island. After 2 months old, a pair of rabbits produces another pair each month. How many rabbits are there after n months?
fn =fn-1 + fn-2
month 1-month rabbits
>=2-month rabbits
Totalfn
1 1 0 1
2 0 1 1
3 1 1 2
4 1 2 3
5 2 3 5
6 3 5 8
7 5 8 13
8 8 13 21
Tower of Hanoi
Example: Tower of Hanoi
H(1) = 1
H(n) = 2H(n-1) + 1
H(n) = 2(2H(n-2) +1) +1
= 4H(n-2) + 2 + 1
= 4(2(H(n-3) + 1) + 2 + 1
= 8H(n-3) + 4 + 2 + 1
= 8(2H(n-4) +1) + 4 + 2 + 1
= 16H(n-4) + 8 + 4 + 2 + 1
= 2n-1 + 2n-2 + 2n-3 + … + 1
= 2n - 1
Let x = 2n-1 + 2n-2 + … + 4 + 2 + 1
x +1 = (2n-1 + 2n-2 + … + 4 + 2 + 1) +1
x + 1 = 2n-1 + 2n-2 + … + 4 + 2 + 2
x + 1 = 2n-1 + 2n-2 + … + 4 + 4
x + 1 = 2n-1 + 2n-2 + … + 8
x + 1 = 2n
x = 2n - 1
Linear recurrence relation
Homogeneous and non-homogeneous
What is a linear recurrence relation ?
Linear recurrence relation of degree k
an= c1an-1 + c2an-2 +…+ ckan-k + F(n)
ci is a constant, for i = 1, 2, …, k
ck 0
Linear homogeneous recurrence relation
F(n) = 0
Linear non-homogeneous recurrence relation
F(n) 0
Examples
Linear homogeneous
an = 1.2 an-1 : degree 1
fn = fn-1 + fn-2 : degree 2
an = 3an-3 : degree 3
Non-linear homogeneous
an = a2n-1 + an-2
an = nan-1 - 2an-2
Linear non-homogeneous
an = an-1 + 2n
hn = 2hn-1 + 1
an = 3an-1 + n
Non-linear non-homogeneous
an = a2n-1 + 2n
an = n2 an-1 + n
Solving linear homogeneous recurrence relation
Theorem
The sequence {an} is a solution of the recurrence
relation an = c1 an-1 + c2 an-2 iff an = 1 rn1 + 2
rn2 , where :
• r1 and r2 are two distinct roots of r2 - c1r - c2 = 0,
• a0 = 1 + 2
• a1 = 1 r1 + 2 r2
• [ 1 = (a1 – a0 r2)/(r1 - r2 ) 2 = (a0 r1 –a1)/(r1 - r2 ) ]
Informal proof
Recurrence relation: an = c1 an-1 + c2 an-2
Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0.
Let and be constants.
Show if an = 1 rn1 + 2 rn
2 , the sequence {an} is a
solution of the recurrence relation.
Show if the sequence {an} is a solution of the
recurrence relation, then an = 1 rn1 + 2 rn
2 .
Informal proof
Show if an = 1 rn1 + 2 rn
2 , the sequence {an} is a
solution of the recurrence relation.
Let an = c1 an-1 + c2 an-2 be a recurrence relation.
Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0.
Let and be constants.
Since r1 and r2 are roots of r2 - c1r - c2= 0,
r12 - c1r1 - c2 = 0 (r1
2 = c1r1 + c2) and r22 - c1r2 - c2= 0 (r2
2 = c1r2 + c2).
Informal proof
r12 = c1r1 + c2 and r2
2 = c1r2 + c2.
c1an-1 + c2an-2
= c1(1r1n-1+2r2
n-1) + c2(1r1n-2+2r2
n-2)
= c11r1n-1+ c1 2r2
n-1 + c21r1n-2+ c2 2r2
n-2
= c11r1n-1+ c21r1
n-2 +c1 2r2n-1 + c2 2r2
n-2
= 1r1n-2 (c1r1 + c2) + 2r2
n-2 (c1r2 + c2)
= 1r1n-2 r1
2 + 2r2n-2 r2
2 = 1 r1n + 2 r2
n = an
Informal proof
Show if the sequence {an} is a solution of the recurrence relation, then there are 1 and 2 such that an = 1 rn
1 + 2 rn
2 .
From the initial conditions a0 and a1 ,
a0 = 1 + 2 and a1 = 1 r1 + 2 r2
2 = a0 - 1
1r1 = a1 - 2r2 = a1 - (a0-1) r2 = a1 – a0r2 + 1r2
1 (r1 - r2 ) = a1 – a0r2 >> 1 = (a1 – a0r2 )/(r1 - r2 )
2 =a0-1 =a0-(a1–a0r2 )/(r1-r2 ) = a0r1-a0r2 -a1+a0r2 /(r1-r2 )
2 = (a0r1-a1) / (r1-r2 )
Example
What is the solution of an = an-1 + 2an-2 with a0=2 a1=7 ?
We have c1 = 1 and c2 = 2.
The characteristic equation is r2 - r - 2 = 0, with roots r1= 2 and r2= -1 .
1 = (a1 – a0r2 )/(r1 - r2 ) = (7+2)/(2+1) = 3
2 = (a0r1 – a1)/(r1- r2 ) = (4-7)/(2+1) = -1
an = 1 rn1 + 2 rn
2 = 32n - (-1)n
Example: Fibonacci number
What is the solution of an = an-1 + an-2 with a0=0 a1=1 ?
We have c1 = 1 and c2 = 1.
The characteristic equation is r2 - r - 1 = 0, with roots r1= (1+5)/2 and r2= (1-5)/2 .
1 = (a1 – a0r2 )/(r1 - r2 ) = (1-0)/5 = 1/5
2 = (a0r1 – a1)/(r1- r2 ) = (0-1)/5 = -1/5
an = 1 rn1 + 2 rn
2 = ((1+5)/2)n /5 - ((1-5)/2)n /5
an = ( (1+5)n - (1-5)n )/(52n )
Theorem
The sequence {an} is a solution of the recurrence
relation an = c1 an-1 + c2 an-2 iff an = 1rn0 + 2nrn
0 , where :
• r0 is the only root of r2 - c1r - c2 = 0,
• a0 = 1
• a1 = 1 r0 + 2 r0 = r0(1 + 2)
• [ 1 = a0 2 = a1/r0 – a0 ]
Example
What is the solution of an= 6an-1 -9an-2 with a0=1 a1=6 ?
We have c1 = 6 and c2 = -9.
The characteristic equation is r2 - 6r + 9 = 0, with roots r0= 3.
1 = a0 = 1
2 = a1/r0 – a0 = 6/3-1 = 1
an = 1rn0 + 2nrn
0 = 3n + n 3n = 3n (1+ n).
Theorem
The sequence {an} is a solution of the recurrence
relation an = c1an-1 + c2an-2 +…+ ckan-k iff an = 1rn1
+ 2rn2 +…+ k rn
k , where :
• r1, r2 ,…, rk are k distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0,
• a0 = 1 + 2 +…+ k
• a1 = 1 r1 + 2 r2 +…+ k rk
• a2 = 1 r21 + 2 r2
2 +…+ k r2k
• …
Theorem
The sequence {an} is a solution of the recurrence
relation an = c1an-1 + c2an-2 +…+ ckan-k iff an = 1rn1
+ 2rn2 +…+ k rn
k , where :
• r1, r2 ,…, rk are k distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0,
• a0 = 1 + 2 +…+ k
• a1 = 1 r1 + 2 r2 +…+ k rk
• a2 = 1 r21 + 2 r2
2 +…+ k r2k
• …
Theorem
The sequence {an} is a solution of the recurrence
relation an = c1an-1 + c2an-2 +…+ ckan-k iff
an = (1,0 + 1,1n + …+ 1,m1nm1-1
) rn1 +
(2,0 + 2,1n + …+ 2,m2nm2-1
) rn2 +…+
(t,0 + t,1n + …+ t,mtnmt-1
) rnt , where :
• r1, r2 ,…, rt are t distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0, with multiplicities m1, m2 ,…, mt
Example
What is the solution of an=6an-1-11an-2+6an-3 with a0=2, a1=5, a2=15 ?
We have c1 = 6, c2 = -11 and c3 = 6.
The characteristic equation is r3 - 6r2 + 11r - 6 = 0, with roots r1= 1, r2= 2 and r3= 3 .
a0 = 1 + 2 + 3
a1 = 1 r1 + 2 r2 + 3 r3 = 1 + 22 + 33
a2 = 1 r12 + 2 r2
2 + 3 r32
= 1 + 42 + 93
1 = 1, 2 = -1, 3 = 2
an = 1 – 2n + 23n
Solving linear non-homogeneous recurrence relation
Theorem
If the sequence {an(p)} is a particular solution of the
recurrence relation an = c1 an-1 + c2 an-2 +…+ ck an-k + F(n) then every solution is of the form {an
(p) + an(h)}
, where {an(h)} is a solution of the associated
homogeneous recurrence relation an = c1 an-1 + c2 an-2 +…+ ck an-k.
Example
What are the solutions of an= 3an-1+ 2n with a1=3 ?
We have c1 = 3.
The associated characteristic eqn is r-3=0, with root r = 3.
Then, an(h) = 3n .
Let pn = cn +d.
Then, from an= 3an-1+ 2n, cn +d = 3(c(n-1)+d ) + 2n.
Thus, c = -1 d = -3/2, and an(p) = -n -3/2
an= an(h) + an
(p) = 3n -n -3/2.
From a1= 3 = 3 -1 - 3/2, = 11/6.
The solution is an= (11/6)3n -n -3/2.
Example: summation from 1 to n
What are the solutions of an= an-1+ n with a1=1 ?
We have c1 = 1.
The associated characteristic eqn is r-1=0, with root r = 1.
Then, an(h) = 1n = .
Let pn = n(cn +d) = cn2 +dn.
Then, from an= an-1+ n, cn2 +dn = c(n-1)2+d (n-1) + n.
Thus, cn2 +dn = cn2-2cn+c +d n - d + n , c-d + n(1-2c)=0.
c-d =0 and 1-2c=0. That is, c=d=1/2.
an= an(h) + an
(p) = + n(n+1)/2. From a1= 1 = +1, = 0.
The solution is an= n(n+1)/2.
Divide-and-conquer Algorithms
Divide-and-Conquer Concept
To solve a problem of size n, we solve a sub-problems of size n/b, and combine solutions from sub-problems.
XX1 X2 X3 Xa...n/b n/b n/b n/b
n elements
Binary Search
To search for an element a from a sorted list X of size n elements.
• If a=m, then stop
• If a<m, then search for a from X1.
• If a>m, then search for a from X2.
XX1 X2m
Fast multiplication of large integers
a = 2n A1 + A0. b = 2n B1 + B0.
A1 A0 B1 B0
a b = 22nA1B1 + 2n(A0B1+ A1B0) +A0 B0.
a b = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0 B0.
a b = 22n A1B1 + 2nA1B1 + 2nA1 B0 –2nA0B0 +2nA0B1
– 2nA1B1 + 2n A0 B0 + A0 B0.
10010 00011 10000 10011
Recurrence Relations
In Divide-and-conquer algorithms
Divide-and-Conquer Recurrence Relations
Let f(n) be the number of operations required to solve a problem of size n.
f(n) = a f(n/b) + g(n).
XX1 X2 X3 Xa...n/b n/b n/b n/b
n elements
Binary Search
To search for an element a from a sorted list X of size n elements.
• If a=m, then stop
• If a<m, then search for a from X1.
• If a>m, then search for a from X2.
Let f(n) be the # of comparison in binary search within n elements.
f(n) = f(n/2) + 2.
XX1 X2m
Fast multiplication of large integers
a = 2n A1 + A0. b = 2n B1 + B0.
A1 A0 B1 B0
a b = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0 B0.
Let f(n) be the number of operations needed to multiply two n-bit integers.
f(2n) = 3f(n) + Cn.
10010 00011 10000 10011
Theorem
Let f be an increasing function that satisfies the recurrence relation
f(n) = a f(n/b) + c
whenever n is divisible by b, b is an integer greater than 1, a1, c is a positive real number.
When n=bk, f(n) = t nlogba + s, for some t and s.
When a>1, f(n) is O(nlogba ).
When a=1, f(n) is O(log n).
Examples
Binary Search: f(n) = f(n/2) + 2.
Because a=1, f(n) is O(log n).
Find Max. : f(n) = 2f(n/2) + 2.
Because a>1, f(n) is O(nlog11 ) = O(n) .
Master Theorem
Let f be an increasing function that satisfies the recurrence relation
f(n) = a f(n/b) + cnd
whenever n=bk for some positive integer k, b is an integer greater than 1, a1, c is a positive real number, d is a non-negative real number.
When a<bd, f(n) is O(nd ).
When a=bd, f(n) is O(nd log n).
When a>bd, f(n) is O(nlogba).
Examples
Merge sort: f(n) = 2f(n/2) + n.
a = 2, b = 2, c = 1, d = 1
Because a=bd, f(n) is O(n log n).
Integer multiplication: f(n) = 3f(n/2) + Cn
a = 3, b = 2, c = C, d = 1
Because a>bd, f(n) is O(nlog23).