Algorithm AnalysisAlgorithm AnalysisLectures 3 & 4

ResourcesData Structures & Algorithms Analysis in C++ (MAW): Chap. 2Introduction to Algorithms (Cormen, Leiserson, & Rivest): Chap.1Algorithms Theory & Practice (Brassard & Bratley): Chap. 1

Algorithms

• An algorithm is a well-defined computational procedure that takes some value or a set of values, as input and produces some value, or a set of values as output.

• Or, an algorithm is a well-specified set of instructions to be solve a problem.

Efficiency of Algorithms

• Empirical– Programming competing algorithms and

trying them on different instances

• Theoretical– Determining mathematically the quantity of

resources (execution time, memory space, etc) needed by each algorithm

Analyzing Algorithms• Predicting the resources that the

algorithm requires:• Computational running time• Memory usage• Communication bandwidth

• The running time of an algorithm • Number of primitive operations on a particular

input size• Depends on

– Input size (e.g. 60 elements vs. 70000)– The input itself ( partially sorted input for a sorting

algorithm)

Order of Growth

• The order (rate) of growth of a running time– Ignore machine dependant constants

– Look at growth of T(n) as n notation

• Drop low-order terms• Ignore leading constants• E.g.

– 3n3 + 90n2 – 2n +5 = (n3)

Mathematical Background

Mathematical Background

• Definitions:

– T(N) = O(f(N)) iff c and n0 T(N) c.f(N) when N n0

– T(N) = (g(N)) iff c and n0 T(N) c.g(N) when N n0

– T(N) = (h(N)) iff T(N) = O(h(N)) and

T(N) = (h(N))

Mathematical Background

• Definitions:

– T(N) = o(f(N)) iff c and n0 T(N) c.f(N) when N n0

– T(N) = (g(N)) iff c and n0 T(N) c.g(N) when N n0

Mathematical Background

• Rules:

– If T1(N) = O(f(N)) and T2(N) = O(g(N)) thena) T1(N) + T2(N) = max( O(f(N)),O(g(N))b) T1(N) * T2(N) = O(f(N) * g(N))

– If T(N) is a polynomial of degree k, then T(N) = (Nk)

– Logk N = O(N) for any constant k.

More …1. 3n3 + 90n2 – 2n +5 = O(n3 )2. 2n2 + 3n +1000000 = (n2) 3. 2n = o(n2) ( set membership)

4. 3n2 = O(n2) tighter (n2) 5. n log n = O(n2) 6. True or false:

– n2 = O(n3 )– n3 = O(n2) – 2n+1= O(2n)– (n+1)! = O(n!)

Ranking by Order of Growth

1 n n log n n2 nk

(3/2)n 2n (n)! (n+1)!

Running time calculations• Rule 1 – For Loops

The running time of a for loop is at most the running time of the statement inside the for loop (including tests) times the number of iterations

• Rule 2 – Nested LoopsAnalyze these inside out. The total running time

of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes of all the loops

Running time calculations: Examples

Example 1:sum = 0;for (i=1; i <=n; i++)

sum += n;

Example 2:sum = 0;for (j=1; j<=n; j++)for (i=1; i<=j; i++)sum++;for (k=0; k<n; k++)A[k] = k;

How to rank?