algorithms and complexity - polito.it · 2013-10-18 · algorithms and complexity 2 algorithm an...

Complexity Advanced Programming

Algorithms and Complexity

2

Algorithm

An algorithm is a calculation procedure (composed by a finite number of steps) which solves a certain problem, working on a set of input values and producing a set of output values


3

Example

The sorting problems is defined as follows: n  Input set: sequence of numbers <a1, a2,…,an> n  Output set: permutation <a’ 1, a’ 2, …, a’ n> of

input sequence so that a’ 1 ≤ a’ 2 ≤ …≤ a’ n.

4

Insertion Sort

Insert A[ j ] in the sorted sequence A[1.. j --1]


5

Esempio

6

Analysis of algorithms

§  Analysing algorithms means to predict the amount of resources (I/O, memory, time) required by an algorithm during its execution. §  Such analysis should be independent from the kind of hardware platform on which the algorithm is executed. §  We will assume in the rest of the course that the hardware platform is a common machine with a single CPU (Random Access Machine or RAM).


7

Problem Dimension Analysis of algorithms is usually performed with respect to one or more parameters which characterize the dimensions of the problem. Example

n  In the sorting algorithm such parameter is the number of elements of the input sequence.

n  In the multiplication between integer numbers, dimension is given by the number of bits used to represent the operands.

8

Hypothesis

n  Each statement in the pseudo-code requires a fixed time

n  Each statement has a different execution time.


9

Analysis of insertion sort

times

tj is the number of times the statement is repeated, for a

certain value of j

Insert A[ j ] in the ordered sequence A[1 .. j -1 ]

10

Execution time of insertion sort


11

Best Case If the array is already sorted, the tj = 1 for every j. Then: Which can be written as follows

T(n) = an + b

12

Worst Case

If an array is sorted with inverse order at step j, j comparisons and swaps: tj = j. Remember that:


13

Worst Case (2)

Therefore In the worst case:

T(n) = an2+bn+c

14

Average Case

Worst case analysis is important as it defines an upper bound to resources required by an algorithm. In some cases the average case (or mean value) is worthy of being analyzed.


15

Importance of complexity analysis

Analysis of resources of an algorithm (also known as complexity analysis) allows describing performance of the algorithm depending on problem dimension. Choosing an algorithm with lower complexity is the best choice independently from the used technology and it can make a difference for problems with big dimensions.

16

Example Suppose you can get the solution of a problem with 2 algorithms:

n  One with complexity T(n) = 2n2

n  The other with complexity T(n) = 50n log2 n

Suppose you have these two machines: n  Intel Core i7 Extreme 965EE performing 76000 MIPS at 3.2 GHz

for the first algorithm n  AMD Athlon FX-60 performing 18000 MIPS at 2.6 GHz for the

second algorithm. n  MIPS = Millions of Instructions Per Second


17

Example (II) Execution time is proportional to T(n) / MIPS:

n  When n = 1 K ( 103)

Intel - T(n)/76000 = 2*106 / 76000 = 0.026 ms AMD – T(n)/18000 = 50*103*log2 (103) / 18000 =0.027 ms

n When n = 1 M ( 106) Intel – 76000 MIPS : 26310 ms AMD – 18000 MIPS : 55,36 ms

n When n = 1 G ( 109)

Intel – 76000 MIPS : 26,3 Ms = 7305 hours ! = 304 days !!! AMD – 18000 MIPS : 83,04 s

18

Asymptotic Notation In complexity analysis the asymptotic notation is often used to discover the complexity of an algorithm when problem dimension increases. The asymptotic notation is based on 3 notations:

n  Theta Notation Θ n  “Big O” Notation O n  Omega Notation Ω.


19

Θ Notation Given an algorithm of complexity T(n). T(n) = Θ( g(n) ) if and only if there exist three positive constants c1, c2 e n0 so that

0≤ c1g(n) ≤ T(n) ≤ c2g(n) for each n≥n0. In this case g(n) is the asymptotic limit for T(n).

20

“Big O” Notation

T(n) = O( g(n) ) if and only if there exist two positive constants c and n0 so that:

0≤ T(n) ≤ c g(n) for each n≥n0. In this case g(n) is a upper asymptotic limit for T(n).


21

Ω Notation

T(n) = Ω( g(n) ) if and only if there exist two positive constants c and n0 so that

0≤c g(n) ≤T(n) for each n≥n0. In this case g(n) is a lower asymptotic limit for T(n).

22


23

Theorem

Given two functions g(n) and T(n), T(n) = Θ( g(n) ) if and only if

n  T(n) = O( g(n) ) and T(n) = Ω( g(n) )

Problems Complexity

24


25

Empirical complexity measure Using the clock() function: #include <time.h> int main(){ clock_t b= clock(); test_function(); clock_t e = clock(); double elapsed=(double)(e – b)/CLOCKS_PER_SEC; printf("Elapsed: %f seconds\n",elapsed); return 0; }

Example

26

strcpy(destination, source);

mycpy(destination, source);

Vs.


Example

27

strcpy(destination, source);

mycpy(destination, source);

Vs.

void mycpy(char* d,char* s){ char* src; for(src=s;*src!='\0';++src){ char* org=s; char* dst=d; for(; origin<=src; ++org,++dst){ *dst = *origin; } } };

O(n2)

Measurement code

28

int main(){ int i,j; printf("Function,n,Clocks\n"); int n[10]={100,200,300,/*…*/,2000,2500,3000}; for(j=1;j<10;++j){ for(i=0; i<N_MEASURE;++i){ clock_t b= clock(); test_strcpy(n[j]); clock_t e = clock(); printf("strcpy(),%d,%d\n",n[j],(int)(e-b)); }} /*…*/ }


Results

29

y = 0.1498x R² = 0.95331

y = 1.5644x2 - 47.556x R² = 0.99922

0

2000000

4000000

6000000

8000000

10000000

12000000

14000000

16000000

0 500 1000 1500 2000 2500 3000

strcpy()

mycpy()

14 s

463 μs

algorithms and complexity - polito.it · 2013-10-18 · algorithms and complexity 2 algorithm an...

Documents