Algorithm Analysis

Math Review – 1.2• Exponents

– XAXB = XA+B

– XA/XB=XA-B

– (XA)B = XAB

– XN+XN = 2XN ≠ X2N

– 2N+2N = 2N+1

• Logarithms– XA=B iff logXB=A– logAB =logCB/logCA; A,B,C > 0, A ≠ 1– logAB = logA + logB; A, B > 0

• Series N

– ∑ 2i = 2N+1-1 i=0

N

– ∑ Ai = AN+1-1/A-1 i=0

N

– ∑ i = N(N+1)/2 i=0

Running Time

• Why do we need to analyze the running time of a program?

• Option 1: Run the program and time it– Why is this option bad?– What can we do about it?

Pseudo-Code

• Used to specify algorithms

• Part English, part code

Algorithm (arrayMax(A, n))

curMax = A[0]

for i=1 i<n i++

if curMax < A[i]

curMax = A[i]

return curMax

Counting Operations

Algorithm (arrayMax(A, n))curMax = A[0]for i=1 i<n i++ if curMax < A[i] curMax = A[i]return curMax

• Best case – 5n • Worst case – 7n-2 • Average case – hard to analyze

Asymptotic Notation

• 7n-2

• n=5 -> 33

• n=100 -> 698

• n=1,000,000 -> 6,999,998

• Running time grows proportionally to n

• What happens as n gets large?

Big-Oh

• T(N) is O(f(N)) if there are positive constants c and n0such that T(N) <= cf(N) when N>=n0

• 7n2-2 is O(n2) n0>=1 c=8

• Upper bound

Omega

• T(N) is Ω(g(N)) if there are positive constants c and n0such that T(N) >= cg(N) when N>=n0

• 7n2-2 is Ω(n2) n0>=1 c=1

• Lower bound

Theta

• T(N) is Θ(h(N)) if and only if T(N) = Oh(N) and T(N) = Ωh(N)

• 7n2-2 is Θ(n2)

• Tightest result possible

Little-Oh

• T(N) is o(p(N)) if T(N) = O(p(N)) and T(N) ≠ Θ(p(N))

• 7n2-2 is o(n3)– O when n0>=8 c=1

• Growth rate of T(N) is smaller than growth rate of p(N)

Rules

• Rule 1– If T1(N) = O(f(N) and T2(N) = O(g(N)), then

• T1(N) + T2(N) = max(O(f(N)), O(g(N)))

• T1(N)* T2(N) = O(f(N)*g(N))

• Rule 2– If T(N) is a polynomial of degree k, then

T(N) = Θ(Nk)

• Rule 3– logkN = O(N) for any constant k. This tells us that

logs grow very slowly.

Examples

• 87n4 + 7n

• 3nlogn + 12logn

• 4n4 + 7n3logn

Terminology

• Logarithmic – O(logn)

• Linear – O(n)

• Quadratic – O(n2)

• Polynomial – O(nk) k>=1

• Exponential – O(an) a>1

Rule 1 – for loops

• The running time of a for loop is at most the running time of the statements inside the for loop (including tests) times the number of iterations.

for(i=0; i<5; i++)

cout << “hello\n”;

Rule 2 – nested loops

• Analyze these inside out. The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the sizes if all of the loops.

for(i=0; i<n; i++) for(j=0; j<n; j++) cout << “hello\n”;

Rule 3 – consecutive statements

• These just add (which means that the max is the one that counts)

for(i=0; i<n; i++)

a[i] = 0;

for(i=0; i<n; i++)

for(j=0; j<n; j++)

a[i]+=a[j]+i+j;

Rule 4 – if/else

• The running time of an if/else statement is never more than the running time of the test plus the larger of the running times of the statements,

if(condition)

S1

else

S2

Example

• Find maximum number in nxn matrix

• Algorithm:

6 4 … 2

12 3 … 9

… … … …

5 8 … 1

0

n-1

n-10

Example

• What is the big-oh running time of this algorithm?

Algorithm:Input: A, ncurMax = A[0][0]for i=0 i<n i++for j=0 j<n j++

if curMax < A[i][j]curMax = A[i][j]

return curMax

Another Example

• Determine how many elements of array 1 match elements of array 2

• Algorithm?

2 4 … 6n-10

6 8 … 3n-10

Another Example

Algorithm:Input: A, B, n

for i=0 i<n i++for j=0 j<n j++

if A[i] == A[j]matches++break

• What is the running time of the algorithm?

2 4 … 6n-10

6 8 … 3n-10

Logs in the Running Time

• An algorithm is O(logN) if it takes constant time to cut the problem size by a fraction.

• Binary search– Given an integer X and integers A0, A1, …,

AN-1, which are presorted and already in memory, find I such that Ai=X, or return i = -1 if X is not in the input.

Binary Search

• Algorithm?

• Running time is at most ceil(log(N-1))+2 which is O(logN)

Example

• 1 – (N-1) - 1

• 2 – (N-1)/2 - 2

• 3 – (N-1)/4 - 4

• 4 – (N-1)/8 - 8

• i – (N-1)/2i-1 - 2i-1 = N-1– difference between high and low is 1

• difference between high and low is 0