yazici kalkan algorithmanalysis 2011 week2a

8/2/2019 Yazici Kalkan AlgorithmAnalysis 2011 Week2a

1/33

Anaysis of Algorihms, A.Yazici & S. Kalkan, Fall 2011 CEng 315 1

Analysis ofAlgorithm

Week 2a

Adnan YAZICISinan KALKAN

Dept. of Computer EngineeringMiddle East Technical Univ.

Ankara - TURKEY


2/33

Measure the height of a tall building

with a barometer

What would be your answer?

One student answered:

I would tie the barometer to the

end of a rope and release the rope

till it touches the ground. The

length of the rope then gives me the

height!

Of course, the instructor rejects theanswer since it doesnt include any

physics


Check the following for two different versions of the `legend:

http://www.snopes.com/college/exam/barometer.asp


3/33

Measure the height of a tall building

with a barometer (contd)

There are other ways:

1. Drop the barometer from the top, and measure the time

it takes to reach the ground.

2. Make a pendulum and time its frequency at the top and

the bottom of the building.

3. Walk down the stairs marking "barometer units" on the

wall.

4. Measure its shadow and the buildings shadow.

Workout the height of the building from barometers

height.



4/33

Analyze/Compare Algorithms

= Pros and Cons of Algorithms =

What are the disadvantages of these?

1. Drop the barometer from the top,

and measure the time it takes to

reach the ground.

2. Make a pendulum and time its

frequency at the top and the

bottom of the building.

3. Walk down the stairs marking

"barometer units" on the wall.

4. Measure its shadow and the

buildings shadow. Workout the

height of the building from

barometers height.


Cons:

1. You lose the barometer; it breaks.

2. You need a loooong rope and how

long that rope is going to be

depends on the answer to the

question.

3. It takes too long. It is too tiring.

4. What if there is no sun?


5/33

Where does the word algorithm

come from?

From a Persian mathematician, astronomer andgeographer: Mohammed ibn-Musa al-Khwarizmi

Algorithmi is the latin form of his name

He contributed to science by

Decimal positional number system(e.g., 32 = 101x3 + 100x2)

Presented the first systematic solutions to

linear and quadratic equations

In fact, the word Algebra comes from one of hisoperators (al-jabr: subtracting a number from both

sides of an equation) for solving equations


Mohammed ibn-Musa

al-Khwarizmi

(780-850)

Source: Wikipedia


6/33

Where does the word algorithm

come from? (contd)

al-Khwarizmi reduced equations to one of thefollowing six forms by using al-jabr (in Arabic:

restoring, completion):

squares equal roots (ax2 = bx)

squares equal number (ax2 = c) roots equal number (bx = c)

squares and roots equal number (ax2 + bx = c)

squares and number equal roots (ax2 + c = bx)

roots and number equal squares (bx + c = ax2)

For example,x2 = 40x 4x2 is reduced to 5x2 =

40x. From this reduced form, it is easily deducable

that the variable is either 0 or 8.


al-Khwarizmis

Compendious Book

on Calculation byCompletion and

Balancing

Source: Wikipedia


7/33

What does algorithm mean?

A procedure or formula for solving a

problem

A set ofinstructions to be followed tosolve a problem

an effective method expressed as a finite

list of well-defined instructions forcalculating a function

step-by-stepprocedure for calculations



8/33

A more formal definitionBetter

suited to us

Starting from an initial state and initial

input (perhaps empty), the instructions

describe a computation that, when executed,will proceed through a finite number of

well-defined successive states, eventually

producing "output" and terminating at afinal ending state.



9/33

A more formal definitionBetter

suited to us

There are three important aspects to thisdefinition:

Computation

Well-defined successive states

Terminate at a final state


Starting from an initial state and initial input (perhaps empty), the instructions describe a

computation that, when executed, will proceed through a finite number of well-defined

successive states, eventually producing "output" and terminating at a final ending state.


10/33

Computation (cont.d)

Problem: Find temperature of the water if

A&B were mixed together.

Any suggestions on how to solve it?


A B


11/33

Computation

Digital vs. analog computation

Sequential vs. parallel computation

Batch vs. interactive computation Evolutionary, molecular, quantum

computation

Physical computation / Digital Physics The whole universe is itself a computation



12/33

Well-defined successive states



13/33

Terminate at a final state



14/33


Algorithms

An

algorithmis a set of instructions to be followed to solve aproblem. In other words, an algorithm is a complete set of rules

that transform the input into the output in a finite number ofsteps.

There can be more than one solution (more than onealgorithm) to solve a given problem.

An algorithm can be implemented using differentprogramming languages on different platforms.

An algorithm should correctly solve the problem.

e.g., for sorting, this means even if (1) the input is already

sorted, or (2) it contains repeated elements. Once we have a correct algorithm for a problem, we have todetermine the efficiency of that algorithm (computationalcomplexity).

i /


15/33

What properties/aspects

algorithms should have other than

performance?

In this course, we

concentrate on

performance.



16/33


Design and Analysis of Algorithms

Aspects of studying algorithms:

1. Designing algorithms: putting the pieces of the puzzles together,

choosing data structures,

selecting the basic approaches to the solution of theproblem,

The most popular design strategies are divide&conquer, greedy,dynamic prog., backtracking, and branch&bound.

2. Expressing and implementing the algorithm

Concerns are:

clearness conciseness

effectiveness

etc.


17/33



3. Analyzing the algorithm

Algorithm analysis is assessing the time and space resourcesrequired by an algorithm as a function of the size of theproblem, without actually implementing the algorithm.

4. Compare UB and LB to see if your solution is good

enoughAnalyzing the algorithm gives us the upper and the lowerbound to solve the problem

5. Validate the algorithm

We show that the algorithm computes the correct answer forall possible legal (or given) inputs


18/33



6. Verifying the algorithm (or program)

An algorithm is said to be correct (verified) if, for every inputinstance, it halts with the correct output.

7. Testing algorithms

There are two phases; Debugging: The process of executing programs on sample data sets to

determine if faulty results occur, and if so, to correct them.

Debugging can only point to the presence of errors, but not to theirabsence

Edsger W. Dijkstra (19302002)

Profiling: the process of executing a correct program on various datasets and measuring the time (and space) it takes to compute the results.
http://en.wikipedia.org/wiki/1930http://en.wikipedia.org/wiki/2002http://en.wikipedia.org/wiki/2002http://en.wikipedia.org/wiki/1930


19/33


Algorithmic Performance

There are two aspects of algorithmic performance:

- Time & space

Time Instructions take time.

How fast does the algorithm perform?

What affects its runtime?

Space Data structures take space

What kind of data structures can be used?

How does choice of data structure affect the runtime?We will focus on time:

How to estimate the time required for an algorithm

How to reduce the time required


20/33

ANALYSIS OF ALGORITHMS



21/33


Analysis of Algorithms

Analysis of Algorithmsanalyze the efficiency of different methods of

solutions.

How do we compare the time efficiency of twoalgorithms that solve the same problem?

Nave Approach: implement these algorithms in aprogramming language (i.e., C++), and run them

to compare their time requirements.


22/33



Comparing the programs (instead of algorithms) has

difficulties.

What data should the program use?

Any analysis must be independent of specific data. Execution time issensitive to the amount of data manipulated, grows as the amount of dataincreases.

What computer should we use?

We should compare the efficiency of the algorithms independently of aparticular computer. Because of the execution speed of the processors, theexecution times for an algorithm on the same data set on two differentcomputers may differ.

How are the algorithms coded?

Comparing running times means comparing the implementations.

We should not compare implementations, because they are sensitive toprogramming style that may cloud the issue of which algorithm is inherentlymore efficient.

absolute measure for an algorithm is not appropriate.


23/33



Analyze algorithms independently of

specific implementations, computers, or data.

To analyze algorithms: First, we start to count the number of significant

operations in a particular solution to assess itsefficiency.

Then, we will express the efficiency of algorithms

using growth functions.


24/33


What is Important?

An array-based list retrieve operation is O(1), a linked-

list-based list retrieve operation is O(n). But insert and delete operations are much easier on a

linked-list-based list implementation.

When selecting the implementation of an Abstract Data Type(ADT), we have to consider how frequently particular ADToperations occur in a given application.

If the problem size is always very small, we can probablyignore the algorithms efficiency.

We have to weigh the trade-offs between an algorithms timerequirement and its memory requirements.


25/33



A running time function, T(n), yieldsthe time required to execute the algorithm of

a problem of size n.

T(n) may contain unspecified constants, which

depend on the characteristics of the ideal

machine.

We cannot determine this function exactly.

T(n) = an2 + bn + c, where a, b and c are unspecified constants.


26/33


The Execution Time of Algorithms (cont.)

Example: Simple If-Statement

Cost Times

if (n < 0) c1 1

absval = -n; c2 1

else

absval = n; c3 1

Total Cost


27/33


General Rules for Estimation

Loops

The running time of a loop is at most the running time of the

statements inside of that loop times the number of iterations.

Nested Loops

Running time of a nested loop containing a statement in theinner most loop is the running time of statement multiplied

by the product of the size of all loops.

Consecutive Statements

Just add the running times of those consecutive statements. If/Else

Never more than the running time of the test plus the larger

of running times of c1 and c2.


28/33


The Execution Time of Algorithms

Example: Simple Loop

Cost Timesi = 1; c1 1

sum = 0; c2 1

while (i


29/33


The Execution Time of Algorithms (cont.)

Example: Nested Loop

Cost Timesi=1; c1 1

sum = 0; c2 1

while (i


30/33



We measure the complexity of an algorithm by identifying a

basic operation and then counting how any times thealgorithm performs that basic operation for an input size n.

problem input of size n

searching a list lists with n elements

sorting a list lists with n elements

multiplying two matrices two n-by-n matrices

traversing a tree tree with n nodes

Towers of Hanoi n disks

Basic operation

comparisoncomparison

multiplication

accessing a node

moving a disk


31/33


Algorithm Growth Rates

An algorithms proportional time requirement isknown as growth rate.The growth rate of T(n) is referred to the

computational complexity of the algorithm.

The computational complexity gives a conciseway of saying how the running time, T(n),varies with n and is independent of any

particular implementation.


32/33


Definition of the Orders of an Algorithm

-Notation: Given two running time functions f(n) and g(n), we say that f(n)is (g(n)) if there exists a real constant c, and a positive integer n0, such

that f(n) c.g(n) for all n n0O-notation gives an upper bound for a function to within a constant factor.

Example: We want to show that 1/2n2+ 3n is O(n2)

f(n) = 1/2n2+ 3n g(n) = n2

to show desired result, need c and n0 such that 0 1/2n2 + 3n c.n2

try c =1

1/2n2 + 3n n2 3n 1/2n2 6 n i.e. n0 = 6.

cg(n)

f(n)

f(n) O(g(n))

Al i h G h R


33/33


Algorithm Growth Rates

We can compare the efficiency of two algorithms bycomparing their growth rates. The lower the growth rate, the faster the algorithm, at least

for large values of n.

For example; T(n) = an2 + bn + c, the growth rate isO(n2)

The goal of the algorithm designer should be an

algorithm with as low a growth rate of the running timefunction, T(n), of that algorithm as possible.

yazici kalkan algorithmanalysis 2011 week2a

Documents