Time Complexity

Solving a computational program

• Describing the general steps of the solution– Algorithm’s course

• Use abstract data types and pseudo code – Data structures course

• Implement the solution using a programming language and concrete data structures– Introduction to computer science

Pseudo Code

• Pseudo code is a meta language, used to describe an algorithm for a computer program.

• We use a notation similar to C or Pascal programming language.

• Unlike real code pseudo code uses a free syntax to describe a given problem

Pseudo Code

• Pseudo code does not deal with problems regarding a specific programming language, as data abstraction and error checking

• Use indentation to distinguish between blocks of code (loop and conditional statements)

• Accessing values of an array is expressed with brackets

Pseudo code

• Accessing an attribute of an object is expressed by the attribute name followed by the object in brackets (length[a])

• More on pseudo code style in the text book

What is the running time of these methods ?

Proc2(A[1..n])i 1; j 1; s 0

repeat

if A[i] < A[j]

s s + 1

if j=n

j=i + 1; i i+1

else

j j+1

until i+j > 2n

Proc1(A[1..n])

s 0

for i = 1 .. n do

for j = (i +1) .. n do

if A[i] < A[j]

s s + 1

Analysis of Bubble sort

• Bubble sort (A)– 1. n length[A]– 2. for j n-1 to 1– 3. for i 0 to j – 1– 4. if A[i] > A[i + 1]– 5. Swap (A[i] , A[i +1])

Time Analysis

• Best case – the array is already sorted and therefore no swap operations are required

11 1

1 0 1

( 1)( ) ( (1)) ( ) 1 2 ....( 1)

2

jn n

j i j

n nT n j n

Time Analysis

• Worst case – the array is sorted in descending order and therefore all swap operations will be executed

• For both inputs the solution requires time

1 11 1

1 0 1 0

( 1)( ) ( (1 )) ( 1) 1 ( 1)

2

j jn n

j i j i

n nT n k k k

2( )n

Asymptotic Notation

• Considering two algorithms, A and B, and the running time for each algorithm for a problem of size n is TA(n) and TB(n)

respectively

• It should be a fairly simple matter to compare the two functions TA(n) and TB(n)

and determine which algorithm is the best!

Asymptotic Notation

• Suppose the problem size is n0 and that TA(n0) < TB(n0 ) Then clearly algorithm A is better than algorithm B for problem size n0

• What happens if we do not know the problem size in advance ? If we can show that TA(n) < TB(n) regardless of n then algorithm A is better then algorithm B regardless of the problem size

• Since we don’t know size in advance we tend to compare the asymptotic behavior of the two.

Asymptotic Upper Bound - O

• O(g(n)) is the group of all functions f(n) which are non-negative for all integers, if there exists an integer n0 and a constant

c>0 such that for all integers

0 ,

0 ( ) ( )

n n

f n cg n

Big O notation

f(n)

cg(n)

0n

0 ( ) ( )for all n n f n cg n

Example

2( 1)( ) , ( )

2

n nf n g n n

2( 1): ( )

2

n nprove O n

0 0

2

0

2

, 0 .

( 1)

2

11

2( 1) 1

2 2

find c n s t for all n n

n ncn

for n and c

n nn

Asymptotic lower Bound -

• is the group of all functions f(n) which are non-negative for all integers and

if there exists an integer n0 and a constant

c>0 such that for all integers

( ( ))g n

0 ,

0 ( ) ( )

n n

cg n f n

Asymptotic lower Bound -

cg(n)

f(n)

0n

0 ( ) ( )for all n n f n cg n

Example

2( 1)( ) , ( )

2

n nf n g n n

2( 1): ( )

2

n nprove n

0 0

2

0

2

0

2

, 0 .

( 1)

21

4( 1) 1

2 4

12

2 2

( 1) 1

2 4

find c n s t for all n n

n ncn

we choose c and find n

n nn

nn

n so for n

n nn

Asymptotic tight bound -

• is the group of all functions f(n) which are non-negative for all integers and

if there exists an integer n0 and two

constants such that for all integers

( ( ))g n

1 2, 0c c

0

1 2

,

0 ( ) ( ) ( )

n n

c g n f n c g n

Asymptotic tight Bound -

cg(n)

f(n)

0n

0 ( ) ( ) ( )for all n n dg n f n cg n

dg(n)

Example

• 2

2 2

2

2

( 1)( ) , ( )

2

1 ( 1) 1

2 2 4( 1)

( )2

( ) ( )

n nf n g n n

n nn n

n nn

f n n

Asymptotic Notation

• When we use the term f = O(n) we mean that the function f O(n)

• When we write we mean that the aside from the function

the sum includes an additional function from O(n) which we have no interest of stating explicitly

2 22 3 1 2 ( )n n n O n

22n

Example

• Show that the function f(n)=8n+128 =O(n2) – lets choose c = 1, then

2

2

2

0

( )

8 128

0 8 128

0 ( 8)( 16)

16

f n cn

n n

n n

n n

n

Conventions for using Asymptotic notation

• drop all but the most significant terms– Instead of we write

• drop constant coefficients– Instead of we use

3 2( 3 4)O n n 3( )O n

3(2 )O n 3( )O n

Back to improved bubble sort

• We improve the sorting algorithm so that if in a complete iteration over the array no swap operations were performed, the execution stops

• Best case –

• Worst case –

• Average case –

( )n

2( )n

2( )2

nn n

Comparing functions

( )

lim ( ) ( ( )), ( ) ( ( ))( )

( )lim 0 ( ) ( ( )), ( ) ( ( ))

( )

( )lim ( ) ( ( ))

( )

n

n

n

f ng n O f n g n f n

g n

f nf n O g n f n g n

g n

f nC g n f n

g n

Example

• Compare the functions ! , nn n

1 2 3 .... ....

! ! ( )

....lim ..... 1

! 1 2 3 .... 2 1

! ( )

n n

n

n

n

n n n n n

n n n O n

n n n n n n nn

n n n

n n

Example

• Compare the functions

• The logarithmic base does not change the order of magnitude

log , loga bn n

, 1

log 1log log (log )

logb

a b bb

a b

nn n n

a c

Example

• Compare the functions 22 , 2n

n

2 / 2

/ 2 / 2

2 , 2

2 ( 2 ), 2 ( 2 )

n n

n n n n

we substitute m m

O

Properties of asymptotic notation

• Transitivity: ( ) ( ) ( )g O f and f O h g O h

1 1 1 1

2 2 2 2

1 2 1 1 2

( ) . ( ) ( )

( ) . ( ) ( )

max( , ) ( ) ( ) ( )

( ) ( ( ))

g O f implies c n s t n n g n c f n

f O h implies c n s t n n f n c h n

n n n g n c f n c c h n

g n O h n

Properties of asymptotic notation

• Symmetry ( ) ( ( )) ( ) ( ( ))g n f n f n g n

1 2 0 0 1 2

3 41 2

4 3

( ) ( ( )) , , 0 . ( ) ( ) ( )

1 1,

( ) ( ) ( )

g n f n implies c c n s t n n c f n g n c f n

let c cc c

c g n f n c g n

Properties of asymptotic notation

• Reflexivity: ( ), ( ), ( )f O f f f f f

1

( ) ( ) ( )

( ) ( ( ))

( ) ( ( )) , ( ) ( ( ))

for c and any n

cf n f n cf n

f n f n

f n O f n f n f n

Example

• Give a proof or a counter example to the following statements:

•  12 (2 )n nO

1

2

2 2 2 2 (2 )n n n n

wechoosec since

c O

Example

• 22 (2 )n nO

22 2 2 2 2 (2 )n n n n n n nc O

Example

• 2100 ( )n O n

20

0

2

100 ,

se 1, 11

100 0

n cn n n

choo c n

n n

Example

• 471 2log( 17) (log )n n n O n

471 2 471

471

17 ( )

log 471log

n n n O n

n n

Example

• Show that

– 1.

– 2.

log( !) ( log )n n n

log( !) ( log )n O n n

log( !) ( log )n n n

log( !) ( log )n O n n

!

log( !) log

( 1)

nn n

n n n

c

log( !) ( log )n n n

2

2

0

! ( )2

log( !) log(( ) ) log( ) (log log 2) (log )2 2 2 2 2 2

(log ) log2 2

1 1 1 1(1 )

2 2log 2 log

14,

4

n

n

nn

n n n n n nn n n

n nn cn n

cn n

n c

Example

• Is it true that for any two functions f,g either f=O(g) or g=O(f) ?

n1+(-1)

f(n)=n

g(n)=n

2ng(n)=

1

n even

else