design and analysis of algorithms - 01 - sanlp.org and analysis of algorithms - 01.pdf · course...

Design and Analysis of Algorithms

Dr. M. G. Abbas [email protected]

[email protected] Assistant ProfessorAssistant Professor

COMSATS Institute of Information Technology, Lahore

CoursePurpose: a thorough introduction to the design andPurpose: a thorough introduction to the design and analysis of algorithms

Interesting and important, but theoretical courseg p ,Equations and Formula, but not a Math cou

Text Book:Introduction to Algorithms; Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest & Clifford Stein

Additional Reference Books:Introduction to the design and analysis of algorithms; R C T Lee S S Tseng R C Chang & Y T TsaiR.C. T. Lee, S. S. Tseng, R. C. Chang & Y. T. TsaiThe design and analysis of computer algorithms; Aho, Hopcroft & Ullmanp

2 Dr. M. G. Abbas Malik; COMSATS IIT Lahore

Course1st Mid term Exam 10%1st Mid-term Exam 10%2nd Mid-term Exam 15%Final Exam 50%Assignments and Quizzes 25%gPlus Class participation

80% attendance is compulsory to sit in the final80% attendance is compulsory to sit in the final exam.


AlgorithmThe word “algorithm” derived fromThe word algorithm derived from Mohammed Al-Khowarizmi, 9th century Persian mathematicianPersian mathematician.A procedure to solve a problemWhat is the best algorithm for a given problem?Three things we will learn:

Design a good algorithmg g gAnalyze itKnow when to stopKnow when to stop


Algorithm - DefinitionAn algorithm is any well definedAn algorithm is any well defined computational procedure that takes

fsome value or set of values as input and produces some value or set of values as poutput.A correct algorithm halts with the correctA correct algorithm halts with the correct output for every instance. We can then

th t l ith l th blsay that algorithm solves the problem.


AlgorithmProblem: Add numbersProblem: Add numbersDesign:

an Divide and Conquer: make an algorithm for the addition of two numbers (base case)Generalize this base case to solve the bigger problemgg

Analysis:Is this is a good algorithm?Is this is a good algorithm?What is its complexity?


Algorithm – Addition of NumbersAlgorithm for Base case:Algorithm for Base case:

Add ( X , Y ){

Z ← X + YReturn Z

}}Generalize Algorithm:

Add ( X1 X2 X )Add ( X1 , X2 , … , Xn ){

Z ← X1 + X2 + … + Xn1 2 nReturn Z

}


Famous AlgorithmsC t ti f E lidConstruction of EuclidNewton’s root findingFast Fourier TransformCompression (Huffman, Lempel-Ziv, GIF, MPEG)p ( , p , , )DES, RSA encryptionSimplex algorithm for linear programmingSimplex algorithm for linear programmingShortest Path Algorithm (Dijkstra, Bellman Ford)Dynamic ProgrammingError correcting codes (CDs, DVDs)g ( )


Famous AlgorithmsTCP congestion control IP routingTCP congestion control, IP routingPattern matching (Genomics)Delaunay Triangulation (FEM. Simulation)


Review: InductionSupposeSuppose

S(k) is true for fixed constant k Often k = 0 or 1

S(n) S(n+1) for all n >= kS(n) S(n+1) for all n > kThen S(n) is true for all n >= k


Proof By InductionClaim: S(n) is true for all n >= kClaim: S(n) is true for all n >= kBase case:

Sh f l i t h kShow formula is true when n = kInductive hypothesis:

Assume formula is true for an arbitrary nStep:Step:

Show that formula is then true for n+1


Induction Example: Gaussian Closed Form

Prove 1 + 2 + 3 + + n = n(n+1) / 2Prove 1 + 2 + 3 + … + n = n(n+1) / 2Base case:

If n = 0 then 0 = 0(0+1) / 2If n 0, then 0 0(0+1) / 2If n = 1, then 1 = 1(1+1) / 2

Inductive hypothesis:Inductive hypothesis:Assume 1 + 2 + 3 + … + n = n(n+1) / 2

Step (show true for n+1):Step (show true for n 1):1 + 2 + … + n + n+1 = (1 + 2 + …+ n) + (n+1)= n(n+1)/2 + n+1 = [n(n+1) + 2(n+1)]/2 ( ) [ ( ) ( )]= (n+1)(n+2)/2 = (n+1)(n+1 + 1) / 2


Induction Example:Geometric Closed Form

Prove a0 + a1 + + an = (an+1 - 1)/(a - 1) forProve a + a + … + a = (a - 1)/(a - 1) for all a ≠ 1

Base case: show that a0 = (a0+1 - 1)/(a - 1)Base case: show that a (a 1)/(a 1) a0 = 1 = (a1 - 1)/(a - 1)Inductive hypothesis:Inductive hypothesis:

Assume a0 + a1 + … + an = (an+1 - 1)/(a - 1) Step (show true for n+1):Step (show true for n+1):a0 + a1 + … + an+1 = (a0 + a1 + … + an) +

an+1a= (an+1 - 1)/(a - 1) + an+1 = (an+1+1 - 1)/(a - 1)


Analyzing AlgorithmHow does the algorithm behave as theHow does the algorithm behave as the problem size gets very large?

Running timeMemory/storage requirementsMemory/storage requirementsBandwidth/power requirements/logic gates/etcgates/etc.


Input SizeTime and space complexityTime and space complexity

This is generally a function of the input sizeti lti li tisorting, multiplication

How we characterize input size depends:Sorting: number of input itemsMultiplication: total number of bitspGraph algorithms: number of nodes & edgesg


Running TimeNumber of primitive steps that are executedNumber of primitive steps that are executed

Except for time of executing a function call most statements roughly require the samemost statements roughly require the same amount of time

* by = m * x + bc = 5 / 9 * (t - 32 )z = f(x) + g(y)


AnalysisWorst caseWorst case

Provides an upper bound on running timeA b l t tAn absolute guarantee

Average caseProvides the expected running timeVery useful, but treat with care: what is y ,“average”?

Random (equally likely) inputsRandom (equally likely) inputsReal-life inputs


An Example: Insertion SortInsertionSort(A n) {InsertionSort(A, n) {for i = 2 to n {

key = A[i]j = i - 1;while (j > 0) and (A[j] > key) {

A[j+1] A[j]A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}


An Example: Insertion Sort30 10 40 20 i = ∅ j = ∅ key = ∅

1 2 3 4

j yA[j] = ∅ A[j+1] = ∅

InsertionSort(A, n) {for i = 2 to n {

key = A[i]j i 1j = i - 1;while (j > 0) and (A[j] > key) {

A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}


An Example: Insertion Sort30 10 40 20 i = 2 j = 1 key = 10

1 2 3 4

j yA[j] = 30 A[j+1] = 10



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 30 A[j+1] = 30



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = ∅ A[j+1] = 30



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = ∅ A[j+1] = 10



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 30 A[j+1] = 40



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 40 A[j+1] = 20



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 40 A[j+1] = 40



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 30 A[j+1] = 40



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 30 A[j+1] = 30



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 10 A[j+1] = 30



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 10 A[j+1] = 20



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}



1 2 3 4

j yA[j] = 10 A[j+1] = 20



A[j+1] = A[j]j j 1j = j - 1

}A[j+1] = key

}}}

Done!


Animating Insertion SortCheck out the Animator a java applet at:Check out the Animator, a java applet at:

http://www.cs.hope.edu/~alganim/animator/Animathttp://www.cs.hope.edu/ alganim/animator/Animator.html

Try it out with random, ascending, and descending inputsdesce d g pu s


Insertion SortInsertionSort(A n) {

What is the preconditionfor this loop?



A[j+1] A[j]A[j+1] = A[j]j = j - 1

}}A[j+1] = key

}}


Insertion SortInsertionSort(A n) {InsertionSort(A, n) {for i = 2 to n {


A[j+1] A[j]A[j+1] = A[j]j = j - 1

}}A[j+1] = key

} How many times will}

How many times will this loop execute?


Insertion SortStatement EffortStatement Effort

InsertionSort(A, n) {for i = 2 to n { c1n

key = A[i] c2(n-1)j = i - 1; c3(n-1)

while (j > 0) and (A[j] > key) { c Σtwhile (j > 0) and (A[j] > key) { c4Σti

A[j+1] = A[j] c5 Σ(ti-1)j = j - 1 c6 Σ(ti-1)}

A[j+1] = key c7(n-1)

}}}


Analyzing Insertion SortT(n) = c1n + c2(n-1) + c3(n-1) + c4 Σ2≤ i ≤ nti + c5 Σ2≤ i ≤ n (ti-1) + c6 Σ2≤ i ≤ n (ti-1)+ c7(n-1)

What can T be?Best case -- inner loop body never executedy

ti = 1 T(n) is a linear functionWorst case -- inner loop body executed for all previous lelementsti = i T(n) is a quadratic function


design and analysis of algorithms - 01 - sanlp.org and analysis of algorithms - 01.pdf · course...

Documents