lecture 6 jianjun hu department of computer science and engineering university of south carolina...
TRANSCRIPT
![Page 1: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/1.jpg)
Lecture 6
Jianjun Hu
Department of Computer Science and Engineering
University of South Carolina
2009.9.
CSCE350 Algorithms and Data Structure
![Page 2: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/2.jpg)
Outline
Review of recursive algorithm running time—cheat sheet
Homogeneous Second-Order Linear Recurrence
Brute Force Strategy for Algorithm Design The art of lazy algorithm is to count/estimate its running time Is it doable within given timeframe?
![Page 3: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/3.jpg)
Time Efficiency of Recursive Algorithms
Steps in mathematical analysis of recursive algorithms:
Decide on parameter n indicating input size
Identify algorithm’s basic operation
Determine worst, average, and best case for input of size n
Set up a recurrence relation and initial condition(s) for C(n)-the number of times the basic operation will be executed for an input of size n (alternatively count recursive calls).
Solve the recurrence to obtain a closed form or estimate the order of magnitude of the solution (see Appendix B)
![Page 4: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/4.jpg)
Three Recurrence Types We know How to Find the Closed-Form Solution
Please related them to the following algorithms we learned in the last class
• Recursive algorithm for n!• Recursive algorithm for Tower of Hanoi• Recursive algorithm for finding the number of digits in the
binary representation of a decimal integer• Recursive algorithm for finding the Fibbonacci numbers
)2()1()(
)1,1()/()(
)1()(
nTbnTanT
banbnTanT
nnTanTk
k
Theorem Master
![Page 5: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/5.jpg)
Important Recurrence Types:
One (constant) operation reduces problem size by one.T(n) = T(n-1) + c T(1) = dSolution: T(n) = (n-1)c + d linear
A pass through input reduces problem size by one.T(n) = T(n-1) + cn T(1) = dSolution: T(n) = [n(n+1)/2 – 1] c + d quadratic
One (constant) operation reduces problem size by half. T(n) = T(n/2) + c T(1) = dSolution: T(n) = c lg n + d logarithmic
A pass through input reduces problem size by half.T(n) = 2T(n/2) + cn T(1) = dSolution: T(n) = cn lg n + d n n log n
![Page 6: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/6.jpg)
A General Divide-and-Conquer Recurrence: Master Theorem
T(n) = aT(n/b) + f (n) where f (n) ∈ Θ(nk)
a < bk T(n) ∈ Θ(nk)
a = bk T(n) ∈ Θ(nk lg n )
a > bk T(n) ∈ Θ(nlog b a)
Note: the same results hold with O instead of Θ.
![Page 7: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/7.jpg)
Solutions to a Homogeneous Second-Order Linear Recurrence with Constant Coefficients
0,0)2()1()( ancTnbTnaT
Characteristic equation
roots
Then
02 cbrar
21, rr
uvvu
nnnTivur
rnrnTrrr
rrnTrr
n
nn
nn
/arctan
)sincos()(
)(
)(
22
2,1
21
2121
and where
complex are if
real is if
real are both and if
![Page 8: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/8.jpg)
Fibonacci numbers
The Fibonacci sequence:0, 1, 1, 2, 3, 5, 8, 13, 21, …
Fibonacci recurrence:F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1
Another example:A(n) = 3A(n-1) – 2A(n-2) A(0) = 1 A(1) = 3
2nd 2nd order linear homogeneous order linear homogeneous recurrence relation recurrence relation
with constant coefficientswith constant coefficients
![Page 9: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/9.jpg)
How to Find the Constants and
Using the initial conditions
For example, for Fibonacci numbers
F(n) = F(n-1) + F(n-2) F(0) = 0 F(1) = 1
Characteristic equation roots
We have
Using F(0)=0 and F(1)=1
012 rr2
512,1
r
nn
nF
2
51
2
51)(
5/15/1 and
![Page 10: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/10.jpg)
Computing Fibonacci numbers
• Definition based recursive algorithm
• Nonrecursive brute-force algorithm
• Explicit formula algorithm
• Logarithmic algorithm based on formula:
FF((nn-1)-1) F F((nn))
FF((nn)) F F((nn+1)+1)
0 10 1
1 11 1=
n
for n≥1, assuming an efficient way of computing matrix powers.
![Page 11: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/11.jpg)
Another Example: calculating a^n
Construct an algorithm for computing an, where a>0 is a constant and the integer n>=0 is the input size. Then analyze its time efficiency
Basic operation a multiplication of two float numbers
Two choices• Nonrecursive algorithm• Recursive algorithm
Any idea to make it faster, e.g., with a better time efficiency?
Can we solve this problem in time?)(logn
![Page 12: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/12.jpg)
Empirical Analysis of Algorithm Time Efficiency
Why? Mathematical analysis is difficult for many algorithms Difficult for average case analysis Difficult for special type of data distribution
How? Purpose: compare efficiency Efficiency measures M: operation counts or time unit Determine characteristics of typical sample input Implement the algorithm and run it Different sample sizes, multiple samples for each size
Profiling tools: measuring time spent on different segments of program can pinpoint bottleneck in a program..
![Page 13: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/13.jpg)
Empirical Analysis of Algorithm Time Efficiency
Data analysis for the empirical data
Scatter Plot
![Page 14: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/14.jpg)
Now We Get to Chapter 3 -- Brute Force
From now on, we are going to learn some basic and general strategies in designing algorithms to solve some typical computing problems.
We will analyze the efficiency of these algorithms using the tools learned in the past several classes
We will learn how to design algorithms with better efficiency
First, let’s talk about the Brute Force strategies– the simplest
![Page 15: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/15.jpg)
Brute Force Strategy for Algorithm Design
• Brute Force is a straightforward approach to solving a problem, usually directly based on the problem’s statement and definitions of the concepts involved
• In many cases, Brute Force does not provide you a very efficient solution
• Brute Force may be enough for moderate size problems with current computers….
![Page 16: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/16.jpg)
Sorting Algorithm
We have discussed two sorting algorithms: Selection Sort and Insertion Sort
What are the basic idea behind the Selection-Sort algorithm?• Scanning the entire given list to find its smallest element and
swap it with the first element• This is a straightforward solution – Brute Force strategy• What is its time efficiency –
An example: sorting the numbers [89 45 68 90 29 34 17]
See pseudocode in Section 3.1
)( 2n
![Page 17: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/17.jpg)
Selection sort
)()()()( 2nnCnCnC averagebestworst
![Page 18: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/18.jpg)
Insertion Sort
?)(),()(),()( 2 nCnnCnnC averagebestworst
![Page 19: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/19.jpg)
Another Brute-Force Application: Bubble Sort
Compare adjacent elements and exchange them if they are out of order
This the result after the first pass, which moves the largest as the rightmost element
90
?
?
? ?
?
?
173429896845
17903429896845
17349029896845
17342990896845
17342990688945
17342990684589
![Page 20: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/20.jpg)
Algorithm in Pseudocode
What is the time efficiency?
]1[][][]1[
20
20
]1..0[
]1..0[
]1..0[
])1..0[(
jAjAjAjA
in-j
n-i
nA
nA
nA
nABubbleSort
and swap if
do to for
do to for
order ascending in sorted Array :Output //
elements orderable of array An :Input //
sort bubble by array sorts algorithm The //
ALGORITHM
)(1 22
0
2
0
nn-
i
-in-
j
![Page 21: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/21.jpg)
Sequential Search – Brute Force
Find whether a search key is present in an array
What is time efficiency of this algorithm?
1
1
][
0
][
)],..0[(
10
-
ini
ii
KiA
i
KnA
KnASearchSequential
..n-AK
return else
return if
do while
ALGORITHM
][ in key Search //
![Page 22: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/22.jpg)
Brute-Force String Matching
Find a pattern in the text: Pattern – ‘NOT’, text – ‘NOBODY_NOTICED_HIM’
Typical Applications – ‘find’ function in the text editor, e.g., MS-Word, Google search
What is the time efficiency of this algorithm?
1
1
][][
0
0
])1..0[],1..0[(
-
imj
jj
jiTjPmj
j
n-m i
mPnTtchBFStringMa
return
return if
and while
do to for
ALGORITHM
![Page 23: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/23.jpg)
Closest-Pair and Convex Hull Problems by Brute Force
Closest-Pair problem• Given n points in a plane, find the closest pair• How to solve this problem and what is the time efficiency of
this algorithm?
Convex-Hull problem• Convex hull is the tightest convex polygon that bounds a set of
n points in a plane• Convex polygon – any two points in this polygon results in the
inclusion of the segment that links these two points also in this polygon
![Page 24: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/24.jpg)
Convex/NonConvex Polygons
![Page 25: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/25.jpg)
Convex Hull
Imagine a rubber band around a set of nails
Nails touched by the band extreme points
P7
P
PP
P
6
35
1
PP
P
82
4
![Page 26: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/26.jpg)
Solve Convex-Hull Problem
Connect any pair of points by a line segment.
Each line segment partitions the plane to the two half planes
If all the n points are on the same side of this line segment
Such a line segment is an edge of the convex-hull polygon
What is the time efficiency of this Brute-Force algorithm?For each possible pair of points, we need to check whether all
the remaining n-2 points are on the same side of the line segment that connects these pair of points.
For Sorting, String Matching, and Convex-Hull problems, we will revisit them by designing more efficient algorithms.
![Page 27: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/27.jpg)
Exhaustive Search
A brute-force approach to combinatorial problem• Generate each and every element of the problem’s domain• Then compare and select the desirable element that satisfies
the set constraints• Involve combinatorial objects such as permutations,
combinations, and subsets of a given set• The time efficiency is usually bad – usually the complexity
grows exponentially with the input size
Three examples• Traveling salesman problem• Knapsack problem• Assignment problem
![Page 28: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/28.jpg)
Traveling Salesman Problem
Find the shortest tour through a given n cities that visits each city exactly once before returning to the starting city
Using graph model: city vertex, road edge, length of the road edge weight.
TSP shortest Hamiltonian Circuit – a cycle that passes through all the vertices of the graph exactly once
Exhaustive search:• List all the possible Hamiltonian circuits (starting from any
vertex)• Ignore the direction • How many candidate circuits do we have? (n-1)!/2• Very high complexity
![Page 29: Lecture 6 Jianjun Hu Department of Computer Science and Engineering University of South Carolina 2009.9. CSCE350 Algorithms and Data Structure](https://reader035.vdocument.in/reader035/viewer/2022062716/56649ddb5503460f94ad1d05/html5/thumbnails/29.jpg)
TSP Example
a
dc
b
5
2
3
1
8 7
a ---> b ---> c --->d ---> a
a ---> b---> d ---> c ---> a
a ---> c ---> b ---> d --->a
a ---> c ---> d ---> b ---> a
a---> d ---> b ---> c ---> a
a ---> d ---> c ---> b ---> a
l = 2 + 8 + 1 + 7 = 18
l = 2 + 3 + 1 + 5 = 11
l = 5 + 8 + 3 + 7 = 23
l = 5 + 1 + 3 + 2 = 11
l = 7 + 3 + 8 + 5 = 23
l = 7 + 1 + 8 + 2 = 18
optimal
optimal
Tour Length