algorithm chapter 2 algorithm analysis

Chapter 2Algorithm Analysis

Introduction

2

algorithmset of simple to solve a problemanalyzed in terms, such as and , required

too long (minutes, hours, years) no goodtoo much memory (terabytes) no good

we can estimate resource usage using a formalmathematical framework to count basic _____________

big-Obig-omegabig-theta

Asymptotic Notation

3

formal definitions

and are for

when we say that is true when is sufficiently large, wemean there exists such that is true for all

we are comparing relative _____________________

Asymptotic Notation: Big-

4

if there exist constants and such that

for all

loosely speaking, is an analog of

example: if andthen since

for all

Asymptotic Notation:

5

if there exist constants and such that

for all


Asymptotic Notation:

6

if there exist constants and suchthat

for all


Relationship of and

7

note that

since

alsoand

since means that there exist andsuch that

for all

Asymptotic Notation: First Observation

8

for sufficiently large

gives an bound on in terms ofgives a bound on in terms of

gives both an and _ boundon in terms of

Asymptotic Notation: Second Observation

9

it is the of constants that matters for thepurposes of asymptotic complexity analysis not the exactvalues of the constants

example: all of the following imply :

for allfor all

for all 00for all 000

Asymptotic Notation: Third Observation

10

we typically constant factors and lower-orderterms, since all we want to capture is growth trends

example:

but the following are discouraged:

true, but bad formtrue, but bad form

Asymptotic Notation: Fourth Observation

11

than , butit can be unduly pessimistic

than , butit can be unduly optimistic

is thethan ,

A

12

an bound can be misleading

example: , since for all

however, for large , the functions andare nothing alike for large

if we know, say,that

upper bounds should be as small as possible!

A

13

an bound can be misleading

example: , since for all

however, for large , the functions andare nothing alike for large

if we know, say,that

lower bounds should be as as possible!

is the Most Informative

14

a relationship is the most :

for all

example:

for all

for large values of , the ratio tends to 2;in this sense,

Typical Growth Rates

15

https://expcode.wordpress.com/2015/07/19/big-o-big-omega-and-big-theta-notation/

Additional Rules

16

Rule 1if and then

(a) (or just)

(b)

Rule 2if is a polynomial of degree , then

Rule 3logk for any constant (logarithms grow very slowly)


17

Function Name____________logarithmiclinear

____________cubic____________


18

http://www.cosc.canterbury.ac.nz/cs4hs/2015/files/complexity-tractability-julie-mcmahon.pdf

Complexity and Tractability

19

assume computer speed of 1 billion ops/sec

T(n)n n n log n n2 n3 n4 n10 2n

10 .01 s .03 s .1 s 1 s 10 s 10s 1 s20 .02 s .09 s .4 s 8 s 160 s 2.84h 1ms30 .03 s .15 s .9 s s 810 s 6.83d 1s40 .04 s .21 s 1.6 s s 2.56ms 121d 18m50 .05 s .28 s s s 6.25ms 3.1y 13d

100 .1 s .66 s 10 s 1ms 100ms 3171y 4 1013y103 1 s 9.96 s 1ms 1s 16.67m 3.17 1013y 32 10283y104 s 130 s 100ms 16.67m 115.7d 3.17 1023y105 s 1.66ms 10s 11.57d 3171y 3.17 1033y106 ms 19.92ms 16.67m 31.71y 3.17 107y 3.17 1043y

Order Comparison

20

the order of an algorithm is useful for purposes

examplean algorithm takes 10ms for input size 60

how long will it take for input size 600 if the order of thealgorithm is linear?

quadratic?

Order Comparison

21

example (cont.)an algorithm takes 10ms for input size 60

how large a problem can be solved in 1s if the order ofthe algorithm is linear?

quadratic?

What to Analyze

22

in order to analyze algorithms, we need a ofcomputation

standard computer with instructionseach operation (+, -, *, /, =, etc.) takes one time unit, evenmemory accessesinfinite memoryobviously unrealistic

Complex Arithmetic Operations

23

examples:count the number of basic operations required to executethe complex arithmetic operationsfor instance, we could compute as

there are , so there arebasic operations to compute plus one more basicoperation to the result

What to Analyze

24

different types of performance are analyzedbest case

may not represent behavioraverage case

often reflects typical behaviordifficult to determine what is averagedifficult to ____________

worst caseon performance for any input

typically considered most importantshould be implementation-independent

Maximum Subsequence Sum Problem

25

example: Maximum Subsequence Sum Problemgiven (possibly negative) integers, A1, A2 N, find themaximum value of

e.g, for input -2, 11, -4, 13, -5, -2, the answer is ____ (A2through A4)

j

ikkA


26

many algorithms to solve this problemwe will focus on four with run time in the table below


27

notescomputer actually run onlooking to compare algorithmsfor inputs, all algorithms perform welltimes do not include _______________

algorithm 4: time to read would surpass time to computereading data is often the bottleneck

for algorithm 4, the run time increases by a factor of 10 asthe problem size increases by a factor of 10 ( )algorithm 2 is ; therefore an increase ofinput by a factor of 10 yields a run time factor of 100

algorithm 1 ( ): run time factor of 1000


28


29

Running Time Calculations

30

at least two ways to estimate the running time of a program________________

as in previous resultsrealistic measurement

_____________helps to eliminate bad algorithms early when severalalgorithms are being consideredanalysis itself provides insight into efficientalgorithmspinpoints , which need special coding

A Simple Example

31

calculate

A Simple Example

32

analysisdeclarations count for ; ignore call costslines 1 and 4 count for 1 time unit eachline 3 counts for 4 time units

2 _________________1 addition1 assignment

line 2 has hidden costsi (1 time unit)

testing i n ( + 1)incrementing i ( )

total: or

A Simple Example

33

analysistoo much work to analyze all programs like this

, we can use to helpline 3 counts for 4 time units, but isline 1 is compared to the for loop

General Rules

34

concentrate on and _____________________Rule 1 FOR loops

running time of the statements inside (including tests)times the number of iterations

Rule 2 Nested loopsanalyze from the _______________running time of the statements times the product of thesizes of all the loops

watch out for loops thatcontribute only a amount of computationcontain statements

Single Loops

35

for (int i = 0; i < n; i++)a[i]+= i;

for (int i = 5; i < n; i++)a[i]+= i;

for (int i = 0; i < n; i++){

if (i >= 7) break;a[i]+= i;

}

Nested Loops

36

for (int i = 0; i < n; ++i)for (int j = 0; j < n; ++j)

++k;amount of computation inside nested loop

must multiply number of times loops execute

for (int i = 0; i < n; ++i)for (int j = i; j < n; ++j)

++k;even though second loop is not executed as often, still

Nested Loops

37

for (int i = 0; i < 7; ++i)for (int j = 0; j < n; ++j)

++k;outer loop is executed a fixed number of times

General Rules (cont.)

38

concentrate on loops and recursive callsRule 3 Consecutive statements

simply the running times

for (i = 0; i < n; ++i)a[i] = 0

for (i = 0; i < n; ++i)for (j = 0; j < n; ++j)

a[i] += a[j] + i + j

work followed by work, so


39

concentrate on loops and recursive callsRule 4 if/else

if (test)S1

elseS2

total running time includes the test plus theof the running times of S1 and S2could be in some cases, but neveran underestimate


40

concentrate on loops and recursive callsfunction calls analyzed firstrecursion

if really just a loop, not difficult (the following is )

long factorial (int n){

if (n <= 1)return 1;

elsereturn n * factorial (n 1);

}


41

previous example not a good use of _____________if recursion used properly, difficult to convert to a simple__________ structure

what about this example?long fib (int n){

if (n <= 1)return 1;

elsereturn fib (n 1) + fib (n 2);

}


42

even worse!extremely inefficient, especially for values _______analysis

for or , is constant: = = 1since the line with recursive calls is not a simpleoperation, it must be handled differently

forwe have seen that this algorithm

similarly, we could show it is

_________________


43

with current recursionlots of redundant work

violating rulethrown away

running time can be reduced substantially with simple for loop

Maximum Subsequence Sum Problem Revisited

44

example: Maximum Subsequence Sum Problemgiven (possibly negative) integers, A1, A2 N, find themaximum value of

e.g, for input -2, 11, -4, 13, -5, -2, the answer is 20 (A2through A4)previously, we reviewed running time results for fouralgorithms

j

ikkA


45

many algorithms to solve this problemwe will focus on four with run time in the table below


46

Algorithm 1

47

Algorithm 1

48

exhaustively tries all ___________________first loop iterates timessecond loop iterates times, which could be small,but we must assume the worstthird loop iterates times, which we must alsoassume to be sizetotal work on line 14 isloopstotal running time is thereforewhat about lines 16-17?

Algorithm 1

49

more precise calculationuse on loops to compute how manytimes line 14 is calculated

compute from the ________________

then

Algorithm 1

50

more precise calculation (cont.)to complete the calculation

Algorithm 2

51

to speed up the algorithm, we can a for loopunnecessary calculation removednote that

new algorithm is

Algorithm 2

52

Algorithm 3

53

algorithm runs even fasterdivide and conquer strategy

divide: problem into two roughly equal subproblemsconquer: solutions to subproblems

maximum subsequence can be in one of three placesentirely in half of inputentirely in half of inputin both halves, the middle

first two cases solved recursivelylast case solved by finding the largest sum in the first halfthat includes the last element of the first half, plus thelargest sum in the second half that includes the firstelement in the second half

Algorithm 3

54

consider the following example

maximum subsequence for the first half: 6maximum subsequence for the second half: 8maximum subsequence that crosses the middle

maximum subsequence in the first half that includes thelast element of the first half: ____maximum subsequence in the second half that includesthe first element in the second half: ____total: ____

Algorithm 3

55

Algorithm 3 (cont.)

56

Algorithm 3

57

notesdriver function needed

gets solution started by calling function with initialparameters that entire array

if left == rightcase

calls to divide the listworking toward base case

lines 18-24 and 26-32 calculate max sums that touch thecentermax3 returns the largest of three values

Algorithm 3

58

analysislet be the time it takes to solve a maximumsubsequence problem of size

if , program takes some constant time to executelines 8-12; thusotherwise, the program must perform two recursive calls

the two for loops access each element in the subarray,with work; thus

all other non-recursive work in the function is constantand can be __________

recursive calls divide the list into two subproblems ifis even (and more strongly, a power of 2)time:

Algorithm 3

59

total time for the algorithm is thereforewhere

note that

is or just

and

isor just

Algorithm 3

60

we can solve the recurrence relation directly (later)

where

for now, note that

thus, if

Algorithm 4

61

algorithm to solve the maximum subsequence sumobservation: we never want a sum

far, but add a running sumif the running sum becomes negative, thestarting point to the first positive element

Algorithm 4

62

Algorithm 4

63

notesif a[i] is negative, it cannot possibly be the element ofa maximum sum, since any such subsequence would beimproved by beginning withsimilarly, any negative subsequence cannot possibly be a_________for an optimal subsequence

this allows us to advance through the original sequencerapidly

correctness may be difficult to determinecan be tested on small input by running against moreinefficient brute-force programs

additional advantage: data need only be read once (_____algorithm) extremely advantageous

Logarithms in the Running Time

64

logarithms will show up regularly in analyzing algorithmswe have already seen them in _____________________strategiesgeneral rule: an algorithm is if it takes constant

time to cut the problem size by a fraction (typically )

if constant time is required to merely reduce the problemby a constant amount (say, smaller by 1), then thealgorithm is

only certain problems fall into the categorysince it would take just to in the inputfor such problems, we assume the input is pre-read

Binary Search

65

binary searchgiven an integer and integers , which are__________ and already in memory, find such that

, or return if is not in the inputobvious solution: scan through the list from left to right tofind

runs in timedoes not take into account that elements are presorted

better solution: check if is in the elementif yes, doneif smaller, apply same strategy to sorted subarrayif greater, use subarray

Binary Search

66

Binary Search

67

analysisall the work done inside the loop takes time periterationfirst iteration of the loop is for

subsequent iterations of the loop this amounttotal running time is therefore

another example using sorted data for fast lookupperiodic table of elements118 elements

at most accesses would be required

68

computes gcd (greatest common divisor) of two integerslargest integer that both

69 70

notesalgorithm computes assuming

if , the values are _____________algorithm works by continually computing until0 is reachedthe last remainder is the answerfor example, if and , the sequence ofremainders is 399, 393, 6, 3, 0

therefore gcd = 3good, fast algorithm

71

analysisneed to determine how many remainders will becomputed

is a good guess, but as can be seen from theexample, the values do not in a uniform way

we can prove that the remainder is at most of itsoriginal value after iterations

this would show that the number of iterations is at most

72

analysis (cont.)Show: if , thenProof: 2 cases

if , then since the remainder is smaller than ,the theorem is trueif , then goes into with aremainder of , and the theorem is true

in our example, is about 20, but we needed only 7operations

the constant can be refined tothe average case ( ) is

Exponentiation

73

exponentiationraising an integer to an power

results are often large, so machine must be able tosupport such valuesnumber of is the measurement of work

obvious algorithm: for uses multiplicationsrecursive algorithm: better

base caseotherwise, if is ,if is ,example:

, ,

Exponentiation

74

Exponentiation

75

analysisnumber of multiplications:

at most, multiplications are required to theproblem (if is odd)

some additional modifications can be made, but care mustbe taken to avoid errors and inefficiencies

Limitations of Worst-Case Analysis

76

analysis can be shown empirically to be an ______________analysis can be tightenedaverage running time may be significantly less than worst-case running time

often, the worst-case is achievable by very ,but still a large overestimate in the caseunfortunately, average-case analysis is extremelycomplex (and in many cases currently unsolved), soworst-

algorithm chapter 2 algorithm analysis

Documents