1.1 data structure and algorithm lecture 7 dynamic programming topics reference: introduction to...
Post on 22-Dec-2015
232 Views
Preview:
TRANSCRIPT
1.1Data Structure and Algorithm
Lecture 7
Dynamic Programming
Topics
Reference: Introduction to Algorithm by Cormen
Chapter 15: Dynamic Programming
1.2Data Structure and Algorithm
Longest Common Subsequence (LCS)
Biologists need to measure similarity between DNA and thus determine how closely related an organism is to another.
They do this by considering DNA as strings of letters A,C,G,T and then comparing similarities in the strings.
Formally, they look at common subsequences in the strings.
Example X = ABCBDAB, Y=BDCABA Subsequences may be: ABA, BCA, BCB, BBA BCBA BDAB etc.
But the Longest Common Subsequences (LCS) are
BCBA and BDAB. How to find LCS efficiently?
1.3Data Structure and Algorithm
Longest Common Subsequence( Brute Force Approach )
if |X| = m, |Y| = n, then there are 2m subsequences of X; we must compare each with Y (n comparisons)
So the running time of the brute-force algorithm is O(n 2m)
Making it impractical for long sequences.
LCS problem has optimal substructure:
solutions of subproblems are parts of the final solution.
Subproblems: “find LCS of pairs of prefixes of X and Y”
1.4Data Structure and Algorithm
LCS: Optimal Substructure
111 and of LCSan is and then , If nmknmknm YXZyxzyx
1 and of LCSan is that implies
then , If
n
nknm
YXZ
yzyx
. and
of LCSany be ,,,let and sequences
be ,,, and ,,,Let
21
2121
YX
zzzZ
yyyYxxxX
k
nm
YXZ
xzyx
m
mknm
and of LCSan is that implies
then , If
1
1.5Data Structure and Algorithm
LCS: Setup for Dynamic Programming
First we’ll find the length of LCS, along the way we will leave “clues” on finding subsequence.
Define Xi, Yj to be the prefixes of X and Y of length i and j respectively.
Define c[i,j] to be the length of LCS of Xi and Yj
Then the length of LCS of X and Y will be c[m,n]
1.6Data Structure and Algorithm
LCS recurrence
Notice the issues…This recurrence is exponential. We don’t know max ahead of time. The subproblems overlap, to find LCS we need to find
LCS of c[i, j-1] and of c[i-1, j]
ji
ji
yxjijicjic
yxjijic
ji
jic
and 0, if ]),1[],1,[max(
, and 0, if 1]1,1[
0or 0 if 0
],[
1.7Data Structure and Algorithm
LCS Algorithm
First we’ll find the length of LCS. Later we’ll modify the algorithm to find LCS itself.
Recall we want to let Xi, Yj to be the prefixes of X and Y of length i and j respectively
And that Define c[i,j] to be the length of LCS of Xi and Yj
Then the length of LCS of X and Y will be c[m,n]
otherwise]),1[],1,[max(
],[][ if1]1,1[],[
jicjic
jyixjicjic
1.8Data Structure and Algorithm
LCS Recursive Solution
We start with i = j = 0 (empty substrings of x
and y)
Since X0 and Y0 are empty strings, their LCS is
always empty (i.e. c[0,0] = 0)
LCS of empty string and any other string is
empty, so for every i and j: c[0, j] = c[i,0] = 0
1.9Data Structure and Algorithm
LCS Recursive Solution
When we calculate c[i,j], we consider two
cases:
First case: x[i]=y[j]: one more symbol in
strings X and Y matches, so the length of LCS
Xi and Yj equals to the length of LCS of
smaller strings Xi-1 and Yi-1 , plus 1
1.10Data Structure and Algorithm
LCS Recursive Solution
Second case: x[i] != y[j]
As symbols don’t match, our solution is not
improved, and the length of LCS(Xi , Yj) is the
same as before (i.e. maximum of LCS(Xi, Yj-1)
and LCS(Xi-1,Yj)
1.11Data Structure and Algorithm
LCS Example
We’ll see how LCS algorithm works on the following
example:
X = ABCB
Y = BDCAB
What is the Longest Common Subsequence
of X and Y?
LCS(X, Y) = BCB
X = A B C B
Y = B D C A B
1.12Data Structure and Algorithm
LCS Example (0)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
X = ABCB; m = |X| = 4Y = BDCAB; n = |Y| = 5Allocate array c[6,5]
1.13Data Structure and Algorithm
LCS Example (1)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
0
0
for i = 1 to m c[i,0] = 0
1.14Data Structure and Algorithm
LCS Example (2)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
for j = 0 to n c[0,j] = 0
1.15Data Structure and Algorithm
LCS Example (3)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0
case i=1 and j=1 A != B but, c[0,1]>=c[1,0] so c[1,1] = c[0,1], and b[1,1] =
1.16Data Structure and Algorithm
LCS Example (4)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0
case i=1 and j=2 A != D but, c[0,2]>=c[1,1] so c[1,2] = c[0,2], and b[1,2] =
0
1.17Data Structure and Algorithm
LCS Example (5)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0
case i=1 and j=3 A != C but, c[0,3]>=c[1,2] so c[1,3] = c[0,3], and b[1,3] =
0 0
1.18Data Structure and Algorithm
LCS Example (6)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 0 1
case i=1 and j=4 A = A so c[1,4] = c[0,3]+1, and b[1,4] =
1.19Data Structure and Algorithm
LCS Example (7)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
000 1 1
case i=1 and j=5 A != B this time c[0,5]<c[1,4] so c[1,5] = c[1, 4], and b[1,5] =
1.20Data Structure and Algorithm
LCS Example (8)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=2 and j=1 B = B so c[2, 1] = c[1, 0]+1, and b[2, 1] =
1.21Data Structure and Algorithm
LCS Example (9)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=2 and j=2 B != D and c[1, 2] < c[2, 1] so c[2, 2] = c[2, 1] and b[2, 2] =
1
1.22Data Structure and Algorithm
LCS Example (10)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=2 and j=3 B != D and c[1, 3] < c[2, 2] so c[2, 3] = c[2, 2] and b[2, 3] =
1 1
1.23Data Structure and Algorithm
LCS Example (11)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=2 and j=4 B != A and c[1, 4] = c[2, 3] so c[2, 4] = c[1, 4] and b[2, 2] =
1 1 1
1.24Data Structure and Algorithm
LCS Example (12)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=2 and j=5 B = B so c[2, 5] = c[1, 4]+1 and b[2, 5] =
1 1 1 2
1.25Data Structure and Algorithm
LCS Example (13)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=3 and j=1 C != B and c[2, 1] > c[3,0] so c[3, 1] = c[2, 1] and b[3, 1] =
1 1 1 2
1
1.26Data Structure and Algorithm
LCS Example (14)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=3 and j= 2 C != D and c[2, 2] = c[3, 1] so c[3, 2] = c[2, 2] and b[3, 2] =
1 1 1 2
1 1
1.27Data Structure and Algorithm
LCS Example (15)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=3 and j= 3 C = C so c[3, 3] = c[2, 2]+1 and b[3, 3] =
1 1 1 2
1 1 2
1.28Data Structure and Algorithm
LCS Example (16)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=3 and j= 4 C != A c[2, 4] < c[3, 3] so c[3, 4] = c[3, 3] and b[3, 3] =
1 1 1 2
1 1 2 2
1.29Data Structure and Algorithm
LCS Example (17)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=3 and j= 5 C != B c[2, 5] = c[3, 4] so c[3, 5] = c[2, 5] and b[3, 5] =
1 1 1 2
1 1 2 22
1.30Data Structure and Algorithm
LCS Example (18)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=4 and j=1 B = B so c[4, 1] = c[3, 0]+1 and b[4, 1] =
1 1 1 2
1 1 2 2
1
2
1.31Data Structure and Algorithm
LCS Example (19)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=4 and j=2 B != D c[3, 2] = c[4, 1] so c[4, 2] = c[3, 2] and b[4, 2] =
1 1 1 2
1 1 2 2
11
2
1.32Data Structure and Algorithm
LCS Example (20)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=4 and j= 3 B != C c[3, 3] > c[4, 2] so c[4, 3] = c[3, 3] and b[4, 3] =
1 1 1 2
1 1 2 2
11 2
2
1.33Data Structure and Algorithm
LCS Example (21)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=4 and j=4 B != A c[3, 4] = c[4, 3] so c[4, 4] = c[3, 4] and b[3, 5] =
1 1 1 2
1 1 2 2
11 22
2
1.34Data Structure and Algorithm
LCS Example (22)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
B
Yj BB ACD
0
0
00000
0
0
0
0 0 10 1
1
case i=4 and j=5 B= B so c[4, 5] = c[3, 4]+1 and b[4, 5] =
1 1 1 2
1 1 2 2
11 22 3
2
1.35Data Structure and Algorithm
LCS Algorithm
LCS-Length(X, Y) m = length(X), n = length(Y)
for i = 1 to m do c[i, 0] = 0 for j = 0 to n do c[0, j] = 0 for i = 1 to m do for j = 1 to n do
if ( xi = = yj ) then c[i, j] = c[i - 1, j - 1] + 1 else if c[i - 1, j]>=c[i, j - 1] then c[i, j] = c[i - 1, j]
else c[i, j] = c[i, j - 1] return c and b
"" , jib
"" , jib
"" , jib
1.36Data Structure and Algorithm
LCS Algorithm Running Time
LCS algorithm calculates the values of each entry of the array c[m,n] So the running time is clearly O(mn) as each entry is done in 3 steps. Now how to get at the solution? We use the arrows we created to guide us. We simply follow arrows back to base case 0
1.37Data Structure and Algorithm
Finding LCSj 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
Yj BB ACD
0
0
00000
0
0
0
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3B
1.38Data Structure and Algorithm
Finding LCS (2)j 0 1 2 3 4 5
0
1
2
3
4
i
Xi
A
B
C
Yj BB ACD
0
0
00000
0
0
0
1000 1
1 21 1
1 1 2
1
22
1 1 2 2 3B
B C BLCS:
1.39Data Structure and Algorithm
Finding LCS (3)
Print_LCS (X, i, j) if i = 0 or j = 0 then return if b[i, j] = “ “ then
Print_LCS (X, i-1, j-1) Print X[i]
elseif b[i, j] = “ “ then Print_LCS (X, i-1, j)
else Print_LCS (X, i, j-1)
Cost: O(m+n)
1.40Data Structure and Algorithm
Element of Dynamic Programming
Optimal Substructure
Overlapping Subproblems
1.41Data Structure and Algorithm
Optimal Substructure
A problem exhibits optimal substructure if an optimal solution contains optimal solutions to its subproblems.
Build an optimal solution from optimal solutions to subproblems
solutions of subproblems are parts of the final solution.
Example :Longest Common Subsequence - An LCS contains within it optimal solutions to the prefixes
of the two input sequences.
Common with Greedy Solution
1.42Data Structure and Algorithm
Overlapping Subproblems
Divide-and-Conquer is suitable when generating brand-new problems at each step of the recursion.
Dynamic-programming algorithms take advantage of overlapping subproblems by solving each subproblem once and then storing the solution in a table where it can be looked up when needed, using constant time per lookup
1.43Data Structure and Algorithm
Dynamic VS Greedy
Dynamic programming uses optimal substructure in a bottom-up fashion First find optimal solutions to subproblems
and, having solved the subproblems, we find an optimal solution to the problem
Greedy algorithms use optimal substructure in a top-down fashion First make a choice – the choice that looks
best at the time – and then solving the resulting subproblem
top related