ds - lecture 2 [time complexity]

8/14/2019 DS - Lecture 2 [Time Complexity]

1/17

Program Efficiency & ComplexityAnalysis of Algorithms


2/17


3/17

Problem Formulation &

SpecificationHalf the battle is knowing what problem

to solveMost problems have no simple precise

specification.Some problems are impossible to

formulate.Identify the problem parameters

Expressing the problem by a formalmodel.Looking for a solution for the modelIn the absence of a solution, discover

about the model


4/17

What is an Algorithm?An algorithm is a definite procedure for solving aproblem in finite number of steps

Algorithm is a well defined computationalprocedure that takes some value(s) as input, andproduces some value(s) as output

Algorithm is finite number of computationalstatements that transform input into the output

Algorithm Definition : A finite set of statementsthat guarantees an optimal solution in finite intervalof time


5/17

What is an Algorithm?Finite sequence of instructions An input should not take the program in an infinite loop

Each instruction having a clear meaning Very subjective. What is clear to me, may not be clear to you.

Each instruction requiring finite amount of effortVery subjective. Finite on a super computer or a P4?

Each instruction requiring finite time to completeVery subjective. 1 min, 1 hr, 1 year or a lifetime?


6/17

Favorite AlgorithmsTakes less memory (Space Efficient)

Smaller execution time

Smaller programming timeTime complexity (most important)


7/17


8/17

Measuring EfficiencyThe efficiency of an algorithm is a

measure of the amount of resourcesconsumed in solving a problem of size

n. The resource we are most interested in is time We can use the same techniques to analyze the

consumption of other resources, such as memory space.

It would seem that the most obvious

way to measure the efficiency of analgorithm is to run it and measure howmuch processor time is neededIs it correct??


9/17

Real Time Execution Vs. Algorithm ComplexitySame algorithms running on different

processors, dont take the same time, why?Processor speedProcessor typeProgramming languageQuality of compilerSize and nature of inputOperating system


10/17

Running Time of an AlgorithmFactors affecting running time:Nature of inputNumber of inputNumber of steps/primitive operations

Running time is measured in terms ofnumber of steps/primitive operations.Generally time grows with size of input, so

running time of an algorithm is usuallymeasured as function of input size.Independent from machine, OSWould not vary from processor to processor

as algorithm steps would remain the same


11/17

Analyzing an AlgorithmFinding running time of an

Algorithm

Running time is measured by number ofsteps/primitive operations performed

Steps means elementary operation like

,+, *,


12/17


13/17

Simple Example (1)// Input: int A[N], array of N integers// Output: Sum of all numbers in array A

int Sum(int A[], int N){

int s=0;

for (int i=0; i< N; i++)

s = s + A[i];

return s;

}

How should we analyse this?


14/17

Simple Example (2)// Input: int A[N], array of N integers

// Output: Sum of all numbers in array A

int Sum(int A[], int N){

int s=0;

for (int i=0; i< N; i++)

s = s + A[i];

return s;

}

1

2 3 4

56 7

8

1,2,8: Once

3,4,5,6,7: Once per each iteration

of for loop, N iterationTotal: 5N + 3

The complexity function of the

algorithm is :f(N) = 5N +3


15/17

Simple Example (3) Growth of5n+3

Estimated running time for different values of N:

N = 10 => 53 steps

N = 100 => 503 stepsN = 1,000=> 5003 stepsN = 1,000,000 => 5,000,003 steps

As N grows, the number of steps grow in linearproportion to N for this function Sum


16/17

What Dominates in Previous

Example?What about the +3 and 5 in 5N+3?As N gets large, the +3 becomes insignificant5 is inaccurate, as different operations require varying

amounts of time and also does not have anysignificant importance

What is fundamental is that the time is linearin N.Asymptotic Complexity: As N gets large,concentrate on the highest order term:

Drop lower order terms such as +3 Drop the constant coefficient of the highest order

term i.e. 5


17/17

Asymptotic ComplexityThe 5N+3 time bound is said to "grow

asymptotically" like N

This gives us an approximation of thecomplexity of the algorithm

Ignores lots of (machine dependent) details,concentrate on the bigger picture

ds - lecture 2 [time complexity]

Documents