1 decrease-and-conquer approach lecture 06 its033 – programming & algorithms asst. prof. dr....
TRANSCRIPT
1
Decrease-and-Conquer Approach
Lecture 06 ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology (SIIT)
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
2
ITS033Topic 01Topic 01 -- Problems & Algorithmic Problem SolvingProblems & Algorithmic Problem SolvingTopic 02Topic 02 – Algorithm Representation & Efficiency Analysis – Algorithm Representation & Efficiency AnalysisTopic 03Topic 03 - State Space of a problem - State Space of a problemTopic 04Topic 04 - Brute Force Algorithm - Brute Force AlgorithmTopic 05 - Divide and ConquerTopic 05 - Divide and ConquerTopic 06Topic 06 -- Decrease and ConquerDecrease and ConquerTopic 07Topic 07 - Dynamics Programming - Dynamics ProgrammingTopic 08Topic 08 -- Transform and ConquerTransform and ConquerTopic 09Topic 09 - Graph Algorithms - Graph AlgorithmsTopic 10Topic 10 - Minimum Spanning Tree - Minimum Spanning TreeTopic 11Topic 11 - Shortest Path Problem - Shortest Path ProblemTopic 12Topic 12 - Coping with the Limitations of Algorithms Power - Coping with the Limitations of Algorithms Power
http://www.siit.tu.ac.th/bunyarit/its033.phphttp://www.siit.tu.ac.th/bunyarit/its033.phpand and http://www.vcharkarn.com/vlesson/showlesson.php?lessonid=7http://www.vcharkarn.com/vlesson/showlesson.php?lessonid=7
3
This Week Overview Problem size reductionProblem size reduction Insertion SortInsertion Sort Recursive programmingRecursive programming ExamplesExamples
FactorialFactorial Tower of HanoiTower of Hanoi
4
Decrease & Conquer: ConceptLecture 06.1
ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology (SIIT)
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
5
Introduction TThe he decrease-and-conquer decrease-and-conquer technique is based on technique is based on
exploiting the relationship between a solution to a given exploiting the relationship between a solution to a given instance of a problem and a solution to a smaller instance instance of a problem and a solution to a smaller instance of of the same problemthe same problem. .
Once such a relationship is established, it can be exploited Once such a relationship is established, it can be exploited either top down (recursively) or bottom up (without a either top down (recursively) or bottom up (without a recursion). recursion).
6
Introduction There are three major variations of decrease-and-conquer:There are three major variations of decrease-and-conquer:
1. Decrease by a constant1. Decrease by a constant
2. Decrease by a constant factor2. Decrease by a constant factor
3. Variable size decrease3. Variable size decrease
7
Decrease by a constant In the In the decrease-by-a-constant decrease-by-a-constant variation, the size of an variation, the size of an
instance is reduced by the same constant on each iteration instance is reduced by the same constant on each iteration of the algorithm.of the algorithm.
Typically, this constant is equal to Typically, this constant is equal to 11
9
Decrease by a constant
Consider, as an example, the exponentiation problem of Consider, as an example, the exponentiation problem of computing computing an an for positive integer exponents. The for positive integer exponents. The relationship between a solution to an instance of size relationship between a solution to an instance of size n n and and an instance of size an instance of size n n - 1 is obtained by the - 1 is obtained by the
obvious formula: obvious formula: aann = = aann-1-1 x x aa
So the function So the function f (n) f (n) = = an an can be computed either “top down” can be computed either “top down” by using its recursive definitionby using its recursive definition
or “bottom up” by multiplying or “bottom up” by multiplying a a by itself by itself n n - 1 times.- 1 times.
10
Decrease by a Constant Factor The The decrease-by-a-constant-factor decrease-by-a-constant-factor technique suggests technique suggests
reducing a problem’s instance by the same constant factor reducing a problem’s instance by the same constant factor on each iteration of the algorithm.on each iteration of the algorithm.
In most applications, this constant factor is equal to two. In most applications, this constant factor is equal to two.
12
Decrease by a Constant Factor
If the instance of size If the instance of size n n is to compute is to compute aann, the instance of half , the instance of half its size will be to compute its size will be to compute aan/n/22, with the obvious relationship , with the obvious relationship between the two: between the two: aann = = (a(an/n/22))22..
But since we consider instances of the exponentiation But since we consider instances of the exponentiation problem with integer exponents only, the former works only problem with integer exponents only, the former works only for even for even nn. If . If n n is odd, we have to compute is odd, we have to compute aann-1-1 by using the by using the rule for even-valued exponents and then multiply the result rule for even-valued exponents and then multiply the result byby
13
Variable Size Decrease the the variable-size-decrease variable-size-decrease variety of decrease-and-conquer, a variety of decrease-and-conquer, a
size reduction pattern varies from one iteration of an algorithm to size reduction pattern varies from one iteration of an algorithm to another.another.
Euclid’s algorithm for computing the greatest common divisor Euclid’s algorithm for computing the greatest common divisor provides a good example of such a situation.provides a good example of such a situation.
14
Decrease & Conquer: Insertionsort
Lecture 06.2 ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology (SIIT)
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
15
Insertion Sort we consider an application of the decrease-by-one we consider an application of the decrease-by-one
technique to sorting an array technique to sorting an array AA[0..[0..n n - 1]. - 1].
Following the technique’s idea, we assume that the smaller Following the technique’s idea, we assume that the smaller problem of sorting the array problem of sorting the array AA[0..[0..n n - 2] has already been - 2] has already been solved to give us a sorted array of size solved to give us a sorted array of size n n - 1: - 1: AA[0]= [0]= . . . . . . = = AA[[n n - 2]. - 2].
How can we take advantage of this solution to the smaller How can we take advantage of this solution to the smaller problem to get a solution to the original problem by taking problem to get a solution to the original problem by taking into account the element into account the element AA[[n n - 1]?- 1]?
16
Insertion Sort we can scan the sorted subarray from right to we can scan the sorted subarray from right to
left until the first element smaller than or equal left until the first element smaller than or equal to to AA[[n n - 1] is encountered and then insert - 1] is encountered and then insert AA[[n n - 1] - 1] right after that element. =>right after that element. =>straight insertion straight insertion sort sort or simply or simply insertion sortinsertion sort..
Or we can use binary search to find an Or we can use binary search to find an appropriate position for appropriate position for AA[[n n - 1] in the sorted - 1] in the sorted portion of the array. => portion of the array. => binary insertion sortbinary insertion sort..
35
Analysis –Worst Case The basic operation of the algorithm is the key The basic operation of the algorithm is the key
comparison comparison AA[[j j ]]> v> v..
The number of key comparisons in this algorithm The number of key comparisons in this algorithm obviously depends on the nature of the input. obviously depends on the nature of the input.
In the worst case, In the worst case, AA[[j j ]]> v > v is executed the largest is executed the largest numbernumber
of times, i.e., for every of times, i.e., for every j j = = i i - 1, - 1, . . . . . . , 0. Since , 0. Since v v = = AA[[ii], it happens if and only if ], it happens if and only if AA[[j j ]]>A>A[[ii] for ] for j j = = i i - - 1, 1, . . . . . . , 0., 0.
36
Analysis –Worst Case In other words, the worst-case input is an array In other words, the worst-case input is an array
of strictly decreasing values. The number of key of strictly decreasing values. The number of key comparisons for such an input iscomparisons for such an input is
37
Analysis – Best Case In the best case, the comparison In the best case, the comparison AA[[j j ]]> v > v is is
executed only once on every iteration of the executed only once on every iteration of the outer loop. It happens if and only if outer loop. It happens if and only if AA[[i i - 1] = - 1] = AA[[ii] ] for every for every i i =1, =1, . . . . . . , , nn-1, i.e., if the input array is -1, i.e., if the input array is already sorted in ascending order. already sorted in ascending order.
Thus, for sorted arrays, the number of key Thus, for sorted arrays, the number of key comparisons iscomparisons is
38
Decrease & Conquer: Recursive Programming
Lecture 06.3 ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology (SIIT)
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
39
Concept of RecursionConcept of Recursion
A recursive definitionA recursive definition is one which uses the word is one which uses the word or concept being defined in the definition itselfor concept being defined in the definition itself
40
Factorials How is this recursive?How is this recursive?
nn! = ! = nn × ( × (nn-1) × (-1) × (nn-2) × … × 3 × 2 × 1-2) × … × 3 × 2 × 1
= (= (nn-1)!-1)!
So: So: nn! = ! = nn × ( × (nn-1) !-1) ! The factorial function is defined in terms of The factorial function is defined in terms of
itself (i.e. itself (i.e. recursivelyrecursively))
41
Recursive Calculation of Factorials
In order for this to work, we need a In order for this to work, we need a stop casestop case (the simplest (the simplest case)case)
Here: 0! = 1Here: 0! = 1
nn! = ! = nn × ( × (nn-1)!-1)!
44
long Factorial(int n) { long fact = 1;
for (int i=2; i <= n; i++) fact = fact * i; return fact;}
n! = n (n-1) (n-2) … 1, if n > 0
Iterative SolutionIterative Solution
45
Recursive definition - definition in which something isdefined in terms of a smaller version of itself, e.g.
Programming with RecursionProgramming with Recursion
n! =n (n-1)!, if n > 0
1, if n = 0
46
Stopping Condition in a recursive definition is the case for which the solution can be stated nonrecursively
General (recursive) case is the case for which the solution is expressed in terms of a smaller version of itself.
Programming with RecursionProgramming with Recursion
n! =n (n-1)!, if n > 0
1, if n = 0
47
long MyFact (int n) { if (n == 0) return 1;
return (n * MyFact (n – 1));
}
Recursive call - a call made to the function from within the function itself;
Stopping cond.
recursive case
Recursion SolutionRecursion Solution
48
How does this work?
int int MyFactMyFact (int n) (int n) { if (n == 0) // The stop case{ if (n == 0) // The stop case return 1;return 1; elseelse return n * return n * MyFactMyFact (n-1);(n-1); } // factorial} // factorial
int x = MyFact(3);
MyFact(3)
3*MyFact(2)
2*MyFact(1)
1*MyFact(0)1
1
2
6
49
Example #1
#include <vcl.h>#include <vcl.h>#include <stdio.h>#include <stdio.h>#include <conio.h>#include <conio.h>
void iforgot_A(int n)void iforgot_A(int n){{ for (int i=1; i<=n; i++)for (int i=1; i<=n; i++) {{ printf("%d, I will remember to do my homework.\n",i);printf("%d, I will remember to do my homework.\n",i); }}
printf("Maybe NOT!");printf("Maybe NOT!");}}
void main()void main(){{ iforgot_A(5);iforgot_A(5); getch();getch();}}
Iterative programmingIterative programming
>> iforgot_A(5)1, I will remember to do my homework.2, I will remember to do my homework.3, I will remember to do my homework.4, I will remember to do my homework.5, I will remember to do my homework.Maybe NOT!
50
Example #2
#include <vcl.h>#include <vcl.h>#include <stdio.h>#include <stdio.h>#include <conio.h>#include <conio.h>
void iforgot_B(int n)void iforgot_B(int n){{ if (n>0)if (n>0) { { printf("%d, I will remember to do my homework.\n",n);printf("%d, I will remember to do my homework.\n",n); iforgot_B(n-1);iforgot_B(n-1); }} elseelse printf("Maybe NOT!");printf("Maybe NOT!");}}
void main()void main(){{ iforgot_B(5);iforgot_B(5); getch();getch();}}
Recursive programmingRecursive programming
>> iforgot_B(5)5, I will remember to do my homework.4, I will remember to do my homework.3, I will remember to do my homework.2, I will remember to do my homework.1, I will remember to do my homework.Maybe NOT!
51
Writing Recursive Functions
Get an exact definition of the problem to be solved.Get an exact definition of the problem to be solved.
Determine the size of the input of the problem.Determine the size of the input of the problem.
Identify and solve the Identify and solve the stopping condition(s)stopping condition(s) in which in which the problem can be expressed non-recursively.the problem can be expressed non-recursively.
Identify and solve the general case(s) correctly in Identify and solve the general case(s) correctly in terms of a smaller case of the same problem.terms of a smaller case of the same problem.
52
Divide or decrease problemDivide or decrease problem One “step” makes the problem smaller (but of the same type)One “step” makes the problem smaller (but of the same type)
Stopping case (solution is trivial)Stopping case (solution is trivial)
Concept 2 - Concept 2 - Recursive ThinkingRecursive Thinking
53
Recursion as problem solving technique Recursive methods defined in terms of themselvesRecursive methods defined in terms of themselves
In code - will see a call to the method itselfIn code - will see a call to the method itself Can have more than one “activation” of a method going at the Can have more than one “activation” of a method going at the
same timesame time Each activationEach activation
has own values of parametershas own values of parameters Returns to where it was called fromReturns to where it was called from
System keeps track of thisSystem keeps track of this
54
Stopping the recursion
The recursion must always STOP The recursion must always STOP Stopping condition is importantStopping condition is important
Recursive solutions to problemsRecursive solutions to problems
55
Stopping the recursion
General pattern is General pattern is test for stopping conditiontest for stopping condition
if not at stopping condition:if not at stopping condition: eithereither
do one step towards solutiondo one step towards solution call the method again to solve the restcall the method again to solve the rest
or or call the method again to solve most of the problemcall the method again to solve most of the problem do the final stepdo the final step
57
Advantages of Recursion
AdvantagesAdvantages Some problems have complicated iterative solutions, Some problems have complicated iterative solutions,
conceptually simple recursive onesconceptually simple recursive ones Good for dealing with dynamic data structures (size Good for dealing with dynamic data structures (size
determined at run time)determined at run time)
..
58
Disadvantages of Recursion
DisadvantagesDisadvantages Extra method calls use memory space & other Extra method calls use memory space & other
resourcesresources Thinking up recursive solution is hard at firstThinking up recursive solution is hard at first Believing that a recursive solution will workBelieving that a recursive solution will work
59
Why Program Recursively?
Recursive CodeRecursive Code Non-Recursive
Code typically has fewer Code typically has fewer lines.lines.
Code can be conceptually Code can be conceptually simpler, depending on your simpler, depending on your perspectiveperspective
Often easier to maintain!Often easier to maintain!
Code is longer.Code is longer. Code executes faster, Code executes faster,
depending on hardware depending on hardware and programming and programming language.language.
60
This Week’s Practice Write a recursive function to calculate Fibonacci numbersWrite a recursive function to calculate Fibonacci numbers
1 2
1 if 1 or 2
if 2nn n
n nf
f f n
What is the result of f(6) ?What is the result of f(6) ?
62
Fibonacci Recursive Tree
fibo(6)
fibo(5) + fibo(4)
fibo(4) + fibo(3) fibo(3) + fibo(2)
fibo(3) + fibo(2)
fibo(2) + fibo(1)
fibo(2) + fibo(1)fibo(2) + fibo(1)
1 1
1 1 1 1 1
1
63
Decrease & Conquer: Homework
ITS033 – Programming & Algorithms
Asst. Prof. Dr. Bunyarit UyyanonvaraAsst. Prof. Dr. Bunyarit UyyanonvaraIT Program, Image and Vision Computing Lab.
School of Information, Computer and Communication Technology (ICT)
Sirindhorn International Institute of Technology (SIIT)
Thammasat Universityhttp://www.siit.tu.ac.th/bunyarit
[email protected] 5013505 X 2005
64
Homework: Fake-Coin ProblemDesign an algorithm using Decrease and Conquer
approach to solve Fake Coin Problem
Among n identically looking coins, one is fake (lighter than genuine).
Using balance scale to find that fake coin.
How many time do you use the balance to find a fake coin from n coins ?
Is it optimum ?