Download - Complexity of an Algorithm lect-2.pps
-
8/12/2019 Complexity of an Algorithm lect-2.pps
1/21
Complexity of an Algorithm
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
2/21
Algorithms
A finite set of steps that specify a sequence of operations to be
carried out in order to solve a specific problem is called an algorithm
Properties of Algorithms:
1. Finiteness- Algorithm must terminate in finite number of steps
2. Absence of Ambiguity-Each step must be clear and unambiguous
3. Feasibility-Each step must be simple enough that it can be easilytranslated into the required language
4. Input-These are zero or more values which are externally supplied to
the algorithm
5. Output-At least one value is produced
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
3/21
Conventions Used for Algorithms
Identifying number-Each algorithm is assigned as identification
number
Comments-Each step may contain a comment in brackets whichindicate the main purpose of the step
Assignment statement-Assignment statement will use colon-equal
notation
Set max:= DATA[1]
Input/Output- Data may be input from user by means of a read
statement
Read: Variable names
Similarly, messages placed in quotation marks, and data in variables
may be output by means of a write statement:
Write: messages or variable names
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
4/21
Selection Logic or conditional flow-
If condition, then:
[end of if structure]
Double alternative
If condition, then
Else:
[End of if structure]
Multiple Alternatives
If condition, then:
Else if condition2, then:
Else if condition3, then
Else:
[End of if structure]C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
5/21
Iteration Logic
Repeat- for loop
Repeat for k=r to s by t:
[End of Loop]
Repeat-While Loop
Repeat while condition:
[End of loop]
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
6/21
Algorithm complexity
An algorithm is a sequence of steps to solve a problem. There can bemore than one algorithm to solve a particular problem and some ofthese solutions may be more efficient than others. The efficiency ofan algorithm is determined in terms of utilization of two resources,
time of execution and memory. This efficiency analysis of analgorithm is called complexity analysis, and it is a very important andwidely-studied subject in computer science. Performancerequirements are usually more critical than memory requirements.Thus in general, the algorithms are analyzed on the basis ofperformance requirements i.e running time efficiency.
Specifically complexity analysis is used in determining how resourcerequirements of an algorithm grow in relation to the size of input.The input can be any type of data. The analyst has to decide whichproperty of the inputshould be measured; the best choice is theproperty that most significantly affects the efficiency-factor we aretrying to analyze. Most commonly, we measure one of the following :
the number of additions, multiplications etc. (for numericalalgorithms).
the number of comparisons (for searching, sorting) the number of data moves (assignment statements)
.
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
7/21
Based on the type of resource variation studied, there are two types ofcomplexities
Time complexity
Space complexity
Space Complexity- The space complexity of an algorithm is amount ofmemory it needs to run to completion. The space needed by aprogram consists of following components:
Instruction space-space needed to store the executable version of
program and is fixed. Data space-space needed to store all constants, variable values and
has further two components:
Space required by constants and simple variables. This space isfixed.
Space needed by fixed sized structured variable such as arrays andstructures.
Dynamically allocated space. This space usually varies.
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
8/21
Environment stack space- Space needed to store information needed toresume the suspended functions. Each time a function is invoked followinginformation is saved on environment stack
Return address i.e from where it has to resume after completion of thecalled function
Values of all local variables and values of formal parameters in functionbeing invoked.
Time complexity-Time complexity of an algorithm is amount of time it needsto run to completion. To measure time complexity, key operations areidentified in a program and are counted till program completes itsexecution. Time taken for various key operations are:
Execution of one of the following operations takes time 1:
1. assignment operation
2. single I/O operations
3. single Boolean operations, numeric comparisons
4. single arithmetic operations5. function return
6. array index operations, pointer dereferences
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
9/21
Running time of a selection statement (if, switch) is the time for thecondition evaluation + the maximum of the running times for theindividual clauses in the selection.
Loop execution time is the number of times the loop body isexecuted + time for the loop check and update operations, + time forthe loop setup.
Always assume that the loop executes the maximum number
of iterations possible
Running time of a function call is 1 for setup + the time for anyparameter calculations + the time required for the execution of thefunction body.
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
10/21
Expressing Space and time complexity: Big O notation
It is very difficult to practically analyze the variation of time requirements
of an algorithm with variation in size of input. A better approach to expressspace/time complexity is in the form of a function f (n) where n is the inputsize for a given instance of problem being solved.
Efficiency (algorithm A) = a function F of some property of A's input.
We have to decide which property of the input we are going to measure;
the best choice is the property that most significantly affects theefficiency-factor we are trying to analyze. For example, the time taken tosort a list is invariably a function of the length of the list. The speed of analgorithm varies with the number of items n.
The most important notation used for expressing this function f(n) is Big O
notation.
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
11/21
Big O notation is a characterization scheme that allows to measure
the properties of algorithm such as performance and/or memory
requirements in general fashion. big O notation
Uses the dominant term of the function
Omits lower order terms and coefficient of dominant
Apart from n (size of input), efficiency measure will depend on
three cases which will decide number of operations to be performed.
Best- Case performance under ideal condition Worst-case performance under most un favorable condition
Average case performance under most probable condition.
Big O notation tries to analyze each algorithm performance in
worst condition. It is the rate of increase of f(n) that is examined as ameasure of efficiency.
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
12/21
Rate of growth: Big O notation
Suppose F is an algorithm and suppose n is the size of input data. Clearly,
complexity f(n) of F increases as n increases. Rate of increase of f(n) is examined by
comparing f(n) with some standard functions such as log2n, n, nlog2n, n2,n3and 2n.
One way to compare f(n) with these standard functions is to use functional Onotation defined as follows:
Definition: If f(n) and g(n) are functions defined on positive integers with the
property that f(n) is bounded by some multiple of g(n) for almost all n. That is,
suppose there exist a positive integer no and a positive number M such that for all
n> no we have|f(n) | M |g(n)|
Then we may write
f(n)=O g(n)
Which is read as f(n)(time taken for number of operations) is of the order of g(n). If
g(n)=n, it means that f(n) is a linear proportional to n. For g(n)=n2, f(n) isproportional to n2. Thus if an array is being sorted using an algorithm with g(n)=n2,
it will take 100 times as long to sort the array that is 10 times the size of another
array.
C#ODE Studio || codstudio.wordpress.com
d l h b d
-
8/12/2019 Complexity of an Algorithm lect-2.pps
13/21
Based on Big O notation, algorithms can be categorized as
Constant time ( O(1)) algorithms
Logarithmic time algorithms (O(logn))
Linear Time algorithm (O(n)
Polynomial or quadratic time algorithm (O(nk))
Exponential time Algorithm (O(kn))
It can be seen that logarithmic function log(n) grows most slowlywhereas kngrows most rapidly and polynomial function nkgrows
in between the two extremities. Big-O notation, concerns onlythe dominant term, low-order terms and constant coefficients areignored in a statement. Thus if g(n) = n2+2n, the variation is takenas O(n2) rather than O(n). Complexities of some well knownsearching and sorting algorithms is:
Linear Search: O(n) Mergesort: O(nlogn)
Binary Search: O(logn)
Bubble sort: O(n2)
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
14/21
RUNNING TIME
let f(n) be the function that defines the time an algorithm takes forproblem size n. The exact formula for f(n) is often difficult to get. We
just need an approximation
What kind of function is f ?
Is it constant? linear? quadratic? ....and, our main concern is large n. Other factors may dominate for
small n
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
15/21
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
16/21
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
17/21
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
18/21
C#ODE Studio || codstudio.wordpress.com
10 100 1000 10,000
102
103
104
10510
6
107
108
n
nlogn
n2
n32n
1
Input size n
Timetaken
Rate of Growth of f(n) with n
logn
-
8/12/2019 Complexity of an Algorithm lect-2.pps
19/21
Other Asymptotic notations for Complexity Analysis
The big O notation defines the upper bound function g(n) for f(n)which represents the time/space complexity of the algorithm on an
input characteristic n . The other such notations are: Omega Notation ()-used when function g(n) defines the lower
bound for function f(n). It is defined as
|f(n)| M|g(n)|
Theta Notation ()-Used when function f(n) is bounded both fromabove and below by the function g(n). It is defined as
c1|g(n)| |f(n)| c2.|g(n)|
Where c1 and c2 are two constants.
Little oh Notation (o)- According to this notation,
f(n)=o g(n) iff f(n) = Og(n) and f(n) g(n)
C#ODE Studio || codstudio.wordpress.com
Ti S T d ff Th b l i h l i bl i
-
8/12/2019 Complexity of an Algorithm lect-2.pps
20/21
Time-Space Tradeoff- The best algorithm to solve a given problem is
one that requires less space in memory and takes less time to
complete its execution. But in practice, it is not always possible to
achieve both of these objectives. Thus, we may have to sacrifice one
at the cost of other. This is known as Time-space tradeoffamongalgorithms. Thus if space is our constraint, we have to choose an
algorithm that requires less space at the cost of more execution time.
On the other hand, if time is our constraint, such as in real time
systems, we have to choose a program that takes less time tocomplete execution at the cost of more space.
C#ODE Studio || codstudio.wordpress.com
-
8/12/2019 Complexity of an Algorithm lect-2.pps
21/21
References
Dinesh Mehta and Sartaj SahniHandbook of Data Structures
and Applications, Chapman and Hall/CRC Press, 2007.
Niklaus Wirth,Algorithms and Data Structures, Prentice Hall,
1985.
Diane Zak, Introduction to programming with c++, copyright
2011 Cengage Learning Asia Pte Ltd
Schaumm Series ,McGraw Hill.
C#ODE Studio || codstudio.wordpress.com
http://en.wikipedia.org/wiki/Sartaj_Sahnihttp://en.wikipedia.org/wiki/Chapman_and_Hallhttp://en.wikipedia.org/wiki/CRC_Presshttp://en.wikipedia.org/wiki/Niklaus_Wirthhttp://en.wikipedia.org/wiki/Prentice_Hallhttp://en.wikipedia.org/wiki/Prentice_Hallhttp://en.wikipedia.org/wiki/Niklaus_Wirthhttp://en.wikipedia.org/wiki/CRC_Presshttp://en.wikipedia.org/wiki/Chapman_and_Hallhttp://en.wikipedia.org/wiki/Sartaj_Sahni