algorithm analysis (algorithm complexity). correctness is not enough it isn’t sufficient that our...
TRANSCRIPT
![Page 1: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/1.jpg)
Algorithm Analysis(Algorithm Complexity)
![Page 2: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/2.jpg)
Correctness is Not Enough
• It isn’t sufficient that our algorithms perform the required tasks.
• We want them to do so efficiently, making the best use of– Space (Storage)– Time (How long will it take, Number of
instructions)
![Page 3: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/3.jpg)
Time and Space
• Time– Instructions take time.– How fast does the algorithm perform?– What affects its runtime?
• Space– Data structures take space.– What kind of data structures can be used?– How does the choice of data structure affect
the runtime?
![Page 4: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/4.jpg)
Time vs. Space
Very often, we can trade space for time:
For example: maintain a collection of students’ with SSN information.– Use an array of a billion elements and have
immediate access (better time)– Use an array of 100 elements and have to
search (better space)
![Page 5: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/5.jpg)
The Right Balance
The best solution uses a reasonable mix of space and time.
– Select effective data structures to represent your data model.
– Utilize efficient methods on these data structures.
![Page 6: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/6.jpg)
Measuring the Growth of Work
While it is possible to measure the work done by an algorithm for a given set of input, we need a way to:
– Measure the rate of growth of an algorithm based upon the size of the input
– Compare algorithms to determine which is better for the situation
![Page 7: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/7.jpg)
7
Worst-Case Analysis
• Worst case running time– Obtain bound on largest possible running time
of algorithm on input of a given size N
– Generally captures efficiency in practice
We will focus on the Worst-Case when analyzing algorithms
![Page 8: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/8.jpg)
Example I: Linear Search Worst Case
Worst Case: match with the last item (or no match)
7 12 5 22 13 32
target = 32
Worst Case: N comparisons
![Page 9: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/9.jpg)
Example II: Binary Search Worst Case
Worst Case: divide until reach one item, or no match,
How many comparisons??
![Page 10: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/10.jpg)
Example II: Binary Search Worst Case
• With each comparison we throw away ½ of the list
N
N/2
N/4
N/8
1
………… 1 comparison
………… 1 comparison
………… 1 comparison
………… 1 comparison
………… 1 comparison
.
.
.
Worst Case: Number of Steps is: Log2N
![Page 11: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/11.jpg)
In General
• Assume the initial problem size is N
• If you reduce the problem size in each step by factor k
– Then, the max steps to reach size 1 LogkN
• If in each step you do amount of work α
– Then, the total amount of work is (α LogkN)
In Binary Search-Factor k = 2, then we have Log2N-In each step, we do one comparison (1)-Total : Log2N
![Page 12: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/12.jpg)
Example III: Insertion Sort Worst Case
Worst Case: Input array is sorted in reverse order
In each iteration i , we do i comparisons.Total : N(N-1) comparisons
![Page 13: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/13.jpg)
Order Of Growth
Log N N 2NN2 N3 N!
Logarithmic
Polynomial Exponential
More efficientLess efficient (infeasible for large N)
![Page 14: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/14.jpg)
14
Why It Matters
• For small input size (N) It does not matter• For large input size (N) it makes all the difference
![Page 15: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/15.jpg)
Order of Growth
![Page 16: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/16.jpg)
Worst-Case Polynomial-Time
• An algorithm is efficient if its running time is polynomial.
• Justification: It really works in practice!– Although 6.02 1023 N20 is technically poly-time, it
would be useless in practice.– In practice, the poly-time algorithms that people
develop almost always have low constants and low exponents.
– Even N2 with very large N is infeasible
![Page 17: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/17.jpg)
Input size N objects
![Page 18: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/18.jpg)
Introducing Big O
• Will allow us to evaluate algorithms.
• Has precise mathematical definition
• Used in a sense to put algorithms into families
LB
![Page 19: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/19.jpg)
Why Use Big-O Notation
• Used when we only know the asymptotic upper bound.
• If you are not guaranteed certain input, then it is a valid upper bound that even the worst-case input will be below.
• May often be determined by inspection of an algorithm.
• Thus we don’t have to do a proof!
![Page 20: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/20.jpg)
Size of Input
• In analyzing rate of growth based upon size of input, we’ll use a variable– For each factor in the size, use a new variable– N is most common…
Examples:– A linked list of N elements– A 2D array of N x M elements– 2 Lists of size N and M elements– A Binary Search Tree of N elements
![Page 21: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/21.jpg)
Formal Definition
For a given function g(n), O(g(n)) is defined to be the set of functions
O(g(n)) = {f(n) : there exist positive constants c and n0 such that 0 f(n) cg(n) for all n n0}
![Page 22: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/22.jpg)
Visual O() Meaning
f(n)
cg(n)
n0
f(n) = O(g(n))
Size of input
Wo
rk d
on
e
Our Algorithm
Upper Bound
![Page 23: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/23.jpg)
Simplifying O() Answers(Throw-Away Math!)
We say 3n2 + 2 = O(n2) drop constants!
because we can show that there is a n0 and a c such that:
0 3n2 + 2 cn2 for n n0
i.e. c = 4 and n0 = 2 yields:
0 3n2 + 2 4n2 for n 2
![Page 24: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/24.jpg)
Correct but Meaningless
You could say
3n2 + 2 = O(n6) or 3n2 + 2 = O(n7)
But this is like answering:• What’s the world record for the mile?
– Less than 3 days.• How long does it take to drive to Chicago?
– Less than 11 years.
O (n2)
![Page 25: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/25.jpg)
Comparing Algorithms
• Now that we know the formal definition of O() notation (and what it means)…
• If we can determine the O() of algorithms…• This establishes the worst they perform.
• Thus now we can compare them and see which has the “better” performance.
![Page 26: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/26.jpg)
Comparing Factors
N
log N
N2
1
Size of input
Wo
rk d
on
e
![Page 27: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/27.jpg)
Do not get confused: O-Notation
O(1) or “Order One”– Does not mean that it takes only one operation
– Does mean that the work doesn’t change as N
changes– Is notation for “constant work”
O(N) or “Order N”– Does not mean that it takes N operations– Does mean that the work changes in a way
that is proportional to N– Is a notation for “work grows at a linear rate”
![Page 28: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/28.jpg)
Complex/Combined Factors
• Algorithms typically consist of a sequence of logical steps/sections
• We need a way to analyze these more complex algorithms…
• It’s easy – analyze the sections and then combine them!
![Page 29: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/29.jpg)
Example: Insert in a Sorted Linked List
• Insert an element into an ordered list…– Find the right location– Do the steps to create the node and add it to
the list
17 38 142head //
Inserting 75
Step 1: find the location = O(N)
![Page 30: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/30.jpg)
Example: Insert in a Sorted Linked List
• Insert an element into an ordered list…– Find the right location– Do the steps to create the node and add it to
the list
17 38 142head //
Step 2: Do the node insertion = O(1)
75
![Page 31: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/31.jpg)
Combine the Analysis
• Find the right location = O(N)• Insert Node = O(1)
• Sequential, so add:– O(N) + O(1) = O(N + 1) =
Only keep dominant factor
O(N)
![Page 32: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/32.jpg)
Example: Search a 2D Array• Search an unsorted 2D array (row, then column)
– Traverse all rows– For each row, examine all the cells (changing columns)
Row
Column
12345
1 2 3 4 5 6 7 8 9 10
O(N)
![Page 33: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/33.jpg)
Example: Search a 2D Array• Search an unsorted 2D array (row, then column)
– Traverse all rows– For each row, examine all the cells (changing columns)
Row
Column
12345
1 2 3 4 5 6 7 8 9 10
O(M)
![Page 34: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/34.jpg)
Combine the Analysis
• Traverse rows = O(N)– Examine all cells in row = O(M)
• Embedded, so multiply:– O(N) x O(M) = O(N*M)
![Page 35: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/35.jpg)
Sequential Steps
• If steps appear sequentially (one after another), then add their respective O().
loop
. . .
endloop
loop
. . .
endloop
N
M
O(N + M)
![Page 36: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/36.jpg)
Embedded Steps
• If steps appear embedded (one inside another), then multiply their respective O().
loop
loop
. . .
endloop
endloop
M N O(N*M)
![Page 37: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/37.jpg)
Correctly Determining O()
• Can have multiple factors:– O(N*M)– O(logP + N2)
• But keep only the dominant factors:
– O(N + NlogN)
– O(N*M + P)
– O(V2 + VlogV)
• Drop constants:
– O(2N + 3N2)
O(NlogN)
remains the same
O(V2)
O(N2)O(N + N2)
![Page 38: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/38.jpg)
Summary
• We use O() notation to discuss the rate at which the work of an algorithm grows with respect to the size of the input.
• O() is an upper bound, so only keep dominant terms and drop constants
![Page 39: Algorithm Analysis (Algorithm Complexity). Correctness is Not Enough It isn’t sufficient that our algorithms perform the required tasks. We want them](https://reader030.vdocument.in/reader030/viewer/2022032806/56649f045503460f94c180f9/html5/thumbnails/39.jpg)