l-2 performance analysis big-o

8/6/2019 L-2 Performance Analysis Big-O

http://slidepdf.com/reader/full/l-2-performance-analysis-big-o 1/38



2

Algorithms

� Efficiency� Complexity



3

Algorithms

� Algorithms are stepwise solutions to

pr oblems

� There may be more than one algorithm f or a

particular pr oblem



4

Efficiency

� Given several algorithms to solve the same

pr oblem, which algorithm is ³ best´?

� Given an algorithm, is it feasible to use it at all? In

other words, is it efficient enough to be usa ble in practice?

� How much time does the algorithm require?

� How much space (memory) does the algorithm

require?

� In general, both time and space requirements

depend on the algorithm¶s input (typically the

³size´ of the input).



5

Efficiency: measuring time

� Measure time in seconds?

+ is useful in practice

± depends on language, compiler, and pr ocessor.

� Count algorithm steps?

+ does not depend on compiler or pr ocessor

± depends on granularity of steps.

� Count characteristic operations? (e.g.,

arithmetic o ps in math algorithms,

comparisons in searching algorithms)

+ depends only on the algorithm itself



6

Complexity

� For many interesting algorithms, the exact num ber of

o perations is too difficult to analyse mathematically.

� To simplify the analysis:

± identify the fastest-gr owing term

± neglect slower-gr owing terms

± neglect the constant factor in the fastest-gr owing term.

� The resulting f ormula is the algorithm¶s time complexity.

It f ocuses on the growth rate of the algorithm¶s time

requirement.

� Similar ly f or space complexity.



7

Example : analysis of power algorithms (1)

� Analysis of simple power algorithm

(counting multiplications):

No. of multiplications = n

Time taken is appr oximately pr o portional to

n.

Time complexity is of order n. This is

written O(n).



8

O-notation (1)

� We have seen that an O(log n) algorithm is

inherently better than an O(n) algorithm f or

lar ge values of n.

O(log n) signifies a slower gr owth rate than

O(n).

� Complexity O( X ) means ³of order X ́ ,

i.e., gr owing pr o portionally to X .

Here X signifies the gr owth rate,

neglecting slower-gr owing terms and

constant factors.



9

O-notation (2)

� Common time complexities:

O(1) constant time (feasi ble)

O(log n) logarithmic time (feasi ble)

O(n) linear time (feasi ble)

O(n log n) log linear time (feasi ble)

O(n2) quadratic time (sometimes

feasi ble)

O(n3) cubic time (sometimes

feasi ble)

O(2n) exponential time (rarely feasi ble)



10

Gr owth rates (1)

� Comparison of gr owth rates:

F(n)/n Log n n nlogn

5 3 5 1510

4 10 40100

7 100 7001000

10 1000 10,000



11

10 20 30 40

20

40

60

0

80

100

500n

Gr owth rates (2)

� Graphically:

log n

n

n log nn22n



12

Efficiency and Complexity:

Summaryy Efficiency

y How much time or space is required

y Measured in terms of common basic o perations

y Complexity

y How efficiency varies with the size of the task

y Expressed in terms of standard functions of n

y E.g. O(n), O(n2), O(log n), O(n log n)



13

Perf ormance Analysis

� Two criteria are used to judge algorithms:(i) time complexity (ii) space complexity.

� Space Complexity of an algorithm is theamount of memory it needs to run to completion.

� Time Complexity of an algorithm is theamount of CPU time it needs to run to completion.



14

Space Complexity� Memory space S(P) needed by a pr ogram P,

consists of two components:

± A fixed part: needed f or instruction space,

simple varia ble space, constant space etc. c

± A varia ble part: dependent on a particular

instance of input and output data.

S p(instance)� S(P) = c + S p(instance)



15

Space Complexity: Example 11. Algorithm abc (a, b, c)

2. {

3. return a+b+b*c+(a+b-c)/(a+b)+4.0;

4. }

For every instance 3 computer words

required to store varia bles: a, b, and c.Theref ore S p()= 3. S(P) = 3.



16

Space Complexity: Example 21. Algorithm Sum(a[], n)

2. {

3. s:= 0.0;4. for i = 1 to n do

5. s := s + a[i];

6. return s;

7. }



17

Space Complexity: Example 2.� Every instance needs to store array a[] & n.

± Space needed to store n = 1 word.

± Space needed to store a[] = n f loating point

words (or at least n words)

± Space needed to store i and s = 2 words

� S p(n) = (n + 3). Hence S(P) = (n + 3).



18

Time Complexity� Time required T(P) to run a pr ogram P also

consists of two components:

± A fixed part: compile time which is

independent of the pr oblem instance c.

± A varia ble part: run time which depends on the

pr oblem instance

t p(instance)� T(P) = c + t p(instance)



19

Time Complexity� How to measure T(P)?

± Measure experimentally, using a ³sto p watch´

T(P) obtained in secs, msecs.

± Count pr ogram steps T(P) obtained as a step

count.

� Fixed part is usually ignored; only thevaria ble part t p() is measured.



20

Time Complexity� What is a pr ogram step?

± a+ b+ b*c+(a+ b)/(a- b) one step;

± comments zer o steps;

while (<expr>) do step count equal to the

num ber of times <expr> is executed.

for i=<expr> to <expr1> do step countequal to num ber of times <expr1> is checked.



21

Methods to compute the step count

� Intr oduce varia ble count into pr ograms

� Ta bular method

± Determine the total num ber of steps contri buted

by each statement

step per execution v frequency

± add up the contri bution of all statements



22

Time Complexity: Example 1Statements S/E Freq. Total

1 Algorithm Sum(a[],n) 0 - 0

2 { 0 - 0

3 S = 0.0; 1 1 1

4 for i=1 to n do 1 n+1 n+1

5 s = s+a[i]; 1 n n

6 return s; 1 1 1

7 } 0 - 0

2n+3



23

Time Complexity: Example 2Statements S/E Freq. Total

1 Algorithm Sum(a[],n,m) 0 - 0

2 { 0 - 0

3 for i=1 to n do; 1 n+1 n+1

4 for j=1 to m do 1 n(m+1) n(m+1)

5 s = s+a[i][j]; 1 nm nm

6 return s; 1 1 1

7 } 0 - 0

2nm+2n+2



24

Ta bular Method

Step count ta ble f or Pr ogram

steps/executionStatement s/e re uenc Total steps

f loat sum f loat list[ ] int n

{f loat tempsum

int i

f or i i n i

tempsum list[i]

return tempsum}

n n

n n

Total 2n 3



25

p2: Step count ta ble f or matrix addition

Statement s/e requency Total steps

Void add (int a AX_SIZ )

{

int i, ;

or (i 0; i r ow; i )

or ( 0; cols; )

c i a i b[i [j];

0 0 0

0 0 0

0 0 0

1 r ows 1 r ows 1

1 r ows(cols 1) r owscols r ows

1 r owscols r owscols

0 0 0

Total 2r owscols 2r ows 1

MatrixAddition



26

Exercise 1

� Program : Printing out a matrix

void print_matrix(int matrix[ ][MAX _SI E], int

r ows, int cols)

int i, j;

f or (i = 0; i < r ow; i++)

f or (j = 0; j < cols; j++) printf(³%d´, matrix[i][ j]);

printf( ³\n´);



27

Exercise 2� *Program :Matrix multiplication f unction

void mult(int a[ ][MAX _SI E], int b[ ][MAX _SI E], int

c[ ][MAX _SI E])

int i, j, k;

f or (i = 0; i < MAX _SI E; i++)

f or (j = 0; j< MAX _SI E; j++)

c[i][ j] = 0;f or (k = 0; k < MAX _SI E; k++)

c[i][ j] += a[i][k ] * b[k ][ j];



28

Exercise 3

� Program :Matrix product f unction

void pr od(int a[ ][MAX _SI E], int b[ ][MAX _SI E], int

c[ ][MAX _SI E],int r owsa, int cols b, int

colsa)

int i, j, k;

f or (i = 0; i < r owsa; i++)

f or (j = 0; j< cols b; j++)

c[i][ j] = 0;

f or (k = 0; k< colsa; k++)

c[i][ j] += a[i][k ] * b[k ][ j];



29

Exercise4

� Program:Matrix transposition f unction

void transpose(int a[ ][MAX _SI E])

int i, j, temp;

f or (i = 0; i < MAX _SI E-1; i++)

f or (j = i+1; j < MAX _SI E; j++)SWAP (a[i][ j], a[ j][i], temp);



30

Perf ormance Measurement

� Which is better?

± T(P1) = (n+1) or T(P2) = (n2 + 5).

± T(P1) = log (n2 + 1)/n! or T(P2) = nn(nlogn)/n2.

� Complex step count functions are difficultto compare.

� For comparing, µrate of gr owth¶ of time andspace complexity functions is easy andsufficient.



31

log n n n log n n2

n3

2n

0 1 0 1 1 2

1 2 2 4 8 4

2 4 8 16 64 163 8 24 64 512 256

4 16 64 256 4096 65536

5 32 160 1024 32768 4294967296



32

� Some math «

± pr o perties of logarithms:

log b(xy) = log bx + log bylog b (x/y) = log bx - log by

log bxa = alog bx

log ba= logxa/logx b

± pr o perties of exponentials:a( b+c) = a ba c

a bc = (a b)c

a b /ac = a( b-c)

b = a loga b

bc = a c*loga b



33

Important Series



34

Important Series

� Sum of squares:

� Sum of exponents:

� Geometric series:

± Special case when A = 2

� 20

+ 21

+ 22

+ « + 2 N

= 2 N+1

- 1

Nlar gef or 36

)12)(1( 3

1

2 N N N N i

N

i }

!§!

-1k and Nlar gef or |1|

1

1

{

}

!

§k

N i

k N

i

k

1

11

0

!

!

§ A

A A

N N

i

i

§!

!!! N

i

N N i N N S

1

2/)1(21)( -



35

Big O Notation

� Big O of a function gives us µrate of gr owth¶ of

the step count function f (n), in terms of a simple

function g(n), which is easy to compare.� Definition: [Big O] The function f (n) = O(g(n))

( big µoh¶ of g of n) iff there exist positive

constants c and n0 such that f (n) <= c*g(n) f or all

n, n>=n0. See graph on next slide.

� Example: 3n+2 = O(n) because 3n+2 <= 4n f or all

n >= 2. c = 4, n0 = 2.



36

Big O Notation

= n0



37

B

ig O Notation� Example: 10n2+4n+2 = O(n2) because

10n2+4n+2 <= 11n2 f or all n >=5.

� Example: 6*2n+n2 = O(2n) because 6*2n+n2

<=7*2n f or all n>=4.

� Algorithms can be: O(1) constant; O(log

n)logrithmic; O(nlogn); O(n)

linear;O(n2) quadratic; O(n3) cu bic; O(2n)

exponential.



38

B

ig O Notation� Now it is easy to compare time or space

complexities of algorithms. Which

algorithm complexity is better?

± T(P1) = O(n) or T(P2) = O(n2)

± T(P1) = O(1) or T(P2) = O(log n)

± T(P1) = O(2n) or T(P2) = O(n10)

l-2 performance analysis big-o

Documents