data structure: algorithm and analysis

DATA STRUCTURES ALGORITHM AND ANALYSIS

Rajdeep Chatterjee

Assistant Professor

School of Computer Engineering

KIIT University

OVERVIEW

Algorithm

Analysis of Algorithm

Space Complexity

Time Complexity

Step Counts

Asymptotic Notations

Big-oh Notations

Rate of Growth

Types of Time Complexity

Best Case Complexity

Worst Case Complexity

Average Case Complexity

ALGORITHM

Algorithm is a finite set of well defined

computational instructions written in a proper

sequence in order to solve a problem.

Criteria-

Input

Output

Definiteness

Finiteness

Effectiveness

EXAMPLES

ADD: INTEGER a, b, c

1. Read a & b

2. add a & b

3. store the sum of a & b to c

4. print c

In C, exp(1)

void add(int a, int b){

int c;

c=a+b;

printf(“%d”,c);

}

In C, exp(2)

int add(int a, int b){

int c;

c=a+b;

return c;

}

ANALYZING AN ALGORITHM

Why is it important?

Predict the feasibility requirement of your code

(algorithm).

Usual requirements

Execution time

Memory space

The complexity of the algorithm is determined in

terms of space and time.

Space complexity ← execution time

Time complexity ← memory space

SPACE COMPLEXITY

Amount of computer memory required during

program execution.

Instruction space – space required to store the

code.

Fixed space requirement – input independent

Variable space requirement – input dependent

S = F(I) otherwise S = C

EXAMPLES

Exp(1)

int main(){

printf(“KIIT UNIVERSITY”);

}

Exp(2)

void bubble(int a[], int n){

int i,j;

for(i=0;i<n-1;i++){

for(j=0;j<n-1-i;j++)

if(a[j]>a[j+1])

swap (a[j],a[j+1])

}

}

TIME COMPLEXITY

The amount of computer time that it needs to run

to completion.

It is determined without considering the following

information-

The machine we are executing,

Its machine language instruction set,

The time required by each machine instruction,

The translation, a compiler/interpreter will make from

the source code to machine language.

EXAMPLES

Exp(1)

x=x+1;

Exp(2)

for(i=1 ; i<=n ; i++)

x=x+1;

Exp(3)

for(i=1 ; i<=n ; i++)

for(j=1 ; j<=n ; j++)

x=x+1;

Our concern should be the order of magnitude/ growth of an algorithm

for an input n.

Exp(4)

int main(){


}

STEP COUNT

Instructions / code Step Count

x=x+1 constant, c

int main(){


return 0;

}

?

for(i=1 ; i<=n ; i++)

x=x+1;

n

for(i=1 ; i<=n ; i++)

for(j=1 ; j<=n ; j++)

x=x+1;

n2

EXAMPLE - 1

1. int i, f=1, n=5; -------------- 1 time

2. for(i=1 ; i<=n ; i++) -------------- n+1 times

3. f=f*i; -------------- n times

4. printf(“%d”,f); -------------- 1 time

n i f=f*i , f=1

5 1 1*1=1

2 1*2=2

3 2*3=6

4 6*4=24

5 24*5=120

6

T(n) = 1+(n+1)+n+1 = 2n+3 ≈ O(n)

if n=0 then T(n) = 1+1 ≈ O(1)

EXAMPLE - 2 1. int a=0,b=1,c, i, n=5; ------------- 1

2. printf(“%d %d”,a,b); ------------- 1

3. for(i=1 ; i<=n-2 ; i++ ){ ------------- (n-2)+1 = (n-1)

4. c=a+b; ------------- (n-2)

5. printf(“%d”,c); ------------- (n-2)

6. a=b; ------------- (n-2)

7. b=c; } ------------- (n-2)

i a b c=a+b

1 0 1 1

2 1 1 2

3 1 2 3

4

T(n)= 1+1+(n-1)+2*(n-2) = 2+n-1+2n-4 = 3n-3 ≈ O(n)

EXAMPLE - 3

1. int i, j, n=5, a[5]={12, 2, 51, 35, 7};

2. for(i=0;i<n-1;i++){

3. for(j=0;j<n-1-i;j++)

4. if(a[j]>a[j+1]){

5. a[j] = a[j] + a[j+1];

6. a[j+1] = a[j] - a[j+1];

7. a[j] = a[j] - a[j+1]; }

8. for(i=0 ; i<n ; i++)

9. printf(“%d”, a[i]);

T(n) = ?

ASYMPTOTIC NOTATIONS

O (Big-oh) Notation

Consider a function f(n) which is non-negative for all

integers n=0. we say that f(n)is Big-oh g(n), which we

write f(n)=O(g(n)), if there exits an integer n0 and a

constant c>0 such that for all integers n=n0, f(n)=cg(n).

In other words,

O(g(n))={f(n): ∃ positive constants c and n0 ∋ 0 ≤ f(n) ≤cg(n) ∀ 𝑛 ≥ 𝑛0 }

O-notation to give an upper bound on a function to within a

constant factor.

EXAMPLE

Two students, X & Y

Let, performance of X & Y are TX and TY respectively,

Conclusion is for a large n, TX(n) ≻ TY(n)

So, who is a better professional ? Ans. X

When X outperforms Y and by what factor ? Ans. n0 and c

# Events X Y TX ∼ TY

1 10th 70% 85% TX(1) ≺ TY(1)

2 (10+2)th 65% 78% TX(2) ≺ TY(2)

3 B.Tech 8.8 cgpa 8.7 cgpa TX(3) ≻ TY(3)

4 M.Tech GATE rank 120 GATE rank 1700 TX(4) ≻ TY(4)

5 Ph.D 5 publications 2 publications TX(5) ≻ TY(5)

6 Job 20 lakhs/p.a. 10 lakhs/p.a. TX(6) ≻ TY(6)

EXAMPLE

Suppose C=1 then, f(n)=6n+135

f(n) = cn2

6n +135 = cn2 = n2 (since c=1)

0 = n2 - 6n -135

0 = (n-15)(n+9)

Since (n+9) > 0 for all values n ≥ 0, then (n-15)=0

n0=15

Find n0 when c=2 & c=4.

NOTATIONS

Big-oh : 0≤ f(n) ≤ cg(n)

Theta : 0≤ 𝑐1𝑔 𝑛 ≤ f(n) ≤ c2 𝑔 𝑛 )

Omega : 0 ≤ cg(n) ≤ f(n)

RATE OF GROWTH

𝒍𝒐𝒈𝟐𝒏 n n𝒍𝒐𝒈𝟐𝒏 𝒏𝟐 𝒏𝟑 𝟐𝒏

0 1 0 1 1 2

1 2 2 4 8 4

2 4 8 16 64 16

3 8 24 64 512 256

4 16 64 256 4096 65536

TYPES OF TIME COMPLEXITY

Best Case Time complexity

Exp(1) – Linear search/Binary search

Linear search, key=3

a[0]==key, T(n)=O(1)

Binary search, key=24

m=(0+4)/2 = 2

a[2]==key, T(n)=O(1)

3 5 24 45 78


Worst Case Time complexity


Linear search, key=78 / 80

a[0]==key, T(n)=O(n) / O(n+1) ≈ O(n)

Binary search, key=78 / 80

m=(0+4)/2 = 2 …

a[2]!=key,…

finally T(n)=O(log2 𝑛) ….(*)

3 5 24 45 78


Average Case Time complexity


Linear search, key=24

a[0]==key, T(n)=O((n+1)/2) ≈ O(n)

There are n cases that can occur, i.e. find at the first place, the second place, the third place and so on up to the nth place. If found at the ith place then i comparisons are required. Hence the average number of comparisons over these n cases is:

average = (1+2+3.....+n)/n = (n+1)/2

where the result was used that 1+2+3 ...+n is equal to n(n+1)/2.

Binary search, key=5

m=(0+4)/2 = 2, m=(0+1)/2=0, m=(1+1)/2=1

a[1]==key, finally T(n) ≈ O(log2 𝑛) ….(*)

3 5 24 45 78

SORTING ALGORITHMS

Algorithm Data

Structure

Time

Complexity:

Best

Time

Complexity:

Average

Time

Complexity:

Worst

Space

Complexity:

Worst

Quick Sort Array O(n log(n)) O(n log(n)) O(n2) O(log(n))

Merge sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(n)

Heap sort Array O(n log(n)) O(n log(n)) O(n log(n)) O(1)

Smooth sort Array O(n) O(n log(n)) O(n log(n)) O(1)

Bubble sort Array O(n) O(n2) O(n2) O(1)

Insertion

sort Array O(n) O(n2) O(n2) O(1)

Selection

sort Array O(n2) O(n2) O(n2) O(1)