chapter 10 dynamic programming
DESCRIPTION
Chapter 10 Dynamic Programming. Agenda for This Week. Dynamic Programming Definition Recursive Nature of Computations in DP Forward and Backward Recursion Basic Elements of DP Applications of DP. Dynamic Programming. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/1.jpg)
Chapter 10
Dynamic Programming
![Page 2: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/2.jpg)
2
Agenda for This Week
• Dynamic Programming– Definition– Recursive Nature of Computations in DP– Forward and Backward Recursion– Basic Elements of DP– Applications of DP
![Page 3: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/3.jpg)
3
Dynamic Programming
• An approach to problem solving that decomposes a large problem into a number of smaller problems for finding optimal solution
• Used for problems that are in multistage in nature
• No standard mathematical formulation
![Page 4: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/4.jpg)
4
Stagecoach Problem
• Used to illustrate the features of DP• Concerns a mythical fortune seeker to go to west to
join the gold rush in California during mid-19th century
• A prudent man who was quite concerned about his safety
• Could not go through some territories • Starting and final destination points are fixed• Had considerable choice as to which states to travel
![Page 5: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/5.jpg)
5
Cost Estimates
5 6 7
2 10 6 -
3 9 8 6
4 6 4 7
2 3 4
1 4 3 5
8 9
5 5 9
6 4 8
7 7 5
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
10
8 6
9 4
![Page 6: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/6.jpg)
6
Solving Problem• Select routes that minimizes total cost
• One possible approach is to use trail and error method
• DP starts with a small portion of the original problem and finds the optimal solution
• Optimizes the nest stage using the solution from preceding one
• Continues until all stages are solved
• First decompose it into stages as delineated by the vertical dashed lines
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
![Page 7: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/7.jpg)
7
Recursive Nature of Computations in DP
• Start with the smaller problem where he has completed his journey and has only one to go
• Computations are done recursively
• Solution of one subproblem is used as an input for next subproblem
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
![Page 8: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/8.jpg)
8
DP Concept• Stages are formed by decomposing the original problem
into a number of subproblems
• Shortest path context– Input— any node or state
– Decision Problem— which arc?
– Decision criteria—shortest distance
– Output– any node or state
Decision Problem Decision Criteria
Input Output
![Page 9: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/9.jpg)
9
DP Notation• Xn=input to stage n (output from stage n+1)
• dn=decision variable at stage n (state variable)
• Stage transformation function• Return function—return of a stage
dNXN
Stage NXN-1 Xn
Xn-1 X0X1
d1dn
Stage n Stage 1
rN(xN,dN) rn(xn,dn) r1(x1,d1)
![Page 10: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/10.jpg)
10
Stage one• First stage begins with nodes that are exactly one arc away from
final destination (CA)
• Nodes 8 and 9 are the input nodes or states for stage 1
• Nodes 10 is the output nodes for stage 1
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
![Page 11: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/11.jpg)
11
Decomposing the Problem-Stage two• Second stage begins with all nodes that are exactly two arcs away
from destination and ends with all nodes that are exactly one arc away
• Nodes 5, 6, and 7 are the input nodes for stage 2
• Nodes 8 and 9 are the output nodes for stage 2
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
![Page 12: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/12.jpg)
12
Decomposing the Problem-Stage three• Third stage begins with all nodes that are exactly three arcs away
from destination and ends with all nodes that are exactly two arcs away
• Nodes 2, 3, and 4 are the input nodes for stage 3
• Nodes 5, 6 and 7 are the output nodes for stage 3
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
![Page 13: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/13.jpg)
13
Decomposing the Problem-Stage Four• Forth stage begins with all nodes that are exactly four arcs away
from destination and ends with all nodes that are exactly three arcs away
• Nodes 1 the input node for stage 4
• Nodes 2, 3 and 4 are the output nodes for stage 3
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
Stage 1Stage 4
![Page 14: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/14.jpg)
14
Solving the Problem-Stage one
10
9
86
4
6
4
Input Arc Output Shortest to 10
89
8-109-10
64
1010
![Page 15: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/15.jpg)
15
Solving by DP Notation-Stage 1• X1=input to stage 1 (output from
stage 2)
• d1=decision variable at stage 1 (state variable)
• Stage transformation function
• Return function—return of a stage
f1(x1)= r1(x1,d1)d1
x1
89
10
64
d1*
1010
dnXn
Stage nXn-1
rn(xn,dn)
10
9
86
4
6
4
![Page 16: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/16.jpg)
16
Solving the Problem- Stage two
10
9
8
4
6
4
Input Arc OutputShortest
6
5
6
7
567
5-86-87-9
11109
5
9
4
8
7 5
889
11
10
9Min [(5+6),(9+4)]=11Min [(4+6),(8+4)]=10Min [(7+6),(5+4)]= 9
![Page 17: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/17.jpg)
17
Solving by DP Notation-Stage 2r2(x2,d2)+ f1(x1)d2
x2
567
8 9
111013
13129
d2*
11109
f2(x2)
889
10
9
8
4
6
4
6
5
6
7
5
9
4
8
7 5
11
10
9
![Page 18: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/18.jpg)
18
Solving the Problem- Stage Three
10
9
8
4
6
4
Input Arc OutputShortest
6
5
6
7
234
2-63-74-6
161514
5
9
4
8
7 5
676 11
10
9
2
3
4
10
6
98
6
6 4
7
16
15
14
Min [(10+11),(6+10)]=16Min [(9+11),(8+10), (6+9)]=15Min [(11+6),(10+4),(7+9)]= 14
![Page 19: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/19.jpg)
19
Solving by DP Notation-Stage 3r3(x3,d3)+ f2(x2)d3
x3
234
5 6212017
161814
d3*
161514
f3(x3)
676
7-
1516
10
9
8
4
6
4
6
5
6
7
5
9
4
8
7 5
11
10
9
2
3
4
10
6
98
6
6 4
7
16
15
14
![Page 20: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/20.jpg)
20
Solving the Problem- Stage Four
10
9
8
4
6
4
Input Arc OutputShortest
6
5
6
7
1 18
5
9
4
8
7 5
3
11
10
9
2
3
4
10
6
98
6
6 4
7
16
15
14
1
4
3
5
1-3
18
Min [(4+16),(3+15), (5+14)]=18
![Page 21: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/21.jpg)
21
Solving by DP Notation-Stage 4r4(x4,d4)+ f3(x3)d4
x4
1
2 3
20 18
d3*
18
f4(x4)
3
4
19
10
9
8
4
6
4
6
5
6
7
5
9
4
8
7 5
11
10
9
2
3
4
10
6
98
6
6 4
7
16
15
14
1
4
3
5
18
![Page 22: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/22.jpg)
22
Best Decision at Each stageStage
4321
Arc1-33-77-9
9-10
3
2
7
6
5
9
810
4
1
4
3
5
10
6
8
9
6
7
4
6
5
49
5
87
6
4
![Page 23: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/23.jpg)
23
Dynamic EOQ Models • Dynamic EOQ models are different,
because– Inventory level is reviewed periodically over a
number finite periods– Demand per period, though deterministic, is
dynamic (MRP situation)
• No shortage is allowed and setup cost is incurred each time a production lot is started
![Page 24: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/24.jpg)
24
A Production and Inventory Control Problem
• Making decision on a production quantity for each of periods so that the demand can be met at a minimum cost– N=no. of periods (stages)
– Dn=demand during stage n
– Xn= amount of inventory on hand at beginning of stage n
– dn= production quantity for stage n
– Pn=production capacity in stage n
– Wn=storage capacity at the end of stage
– Cn=production cost per unit in stage
– Hn=holding cost per unit of ending inventory for stage n
![Page 25: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/25.jpg)
25
Example Capacity Cost/unit
Month Demand Production Storage Production Holding
123
233
323
232
175150200
303040
Beginning inventory for month 1is 1 units
![Page 26: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/26.jpg)
26
DP Formulation
Month 1 Month 2 Month 3
D3=
2
P3=
3W
3=2
d3=
?
P2=
2D
2=3
W2=
3d2
=?
D1=
3
P1=
3
W1=
2d1
=?
X3=1 X1X2 X0
r3(x3,d3) r2(x2,d2) r1(x1,d1)
Note: Ending Inventory= Beginning Inventory
+Production-Demand
![Page 27: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/27.jpg)
27
Knapsack Problem
• N items can be put into a knapsack• Each time has a certain weight and a value• Determine how many units of each item to place
in the knapsack to maximize the total value• Constraint is placed on the maximum weight• Similar to a manger who must make a biweekly
selection of jobs from each of four categories to process during a 2-week period
![Page 28: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/28.jpg)
28
Job Data for the Manufacturing Operation • Select certain jobs
during the next 2 weeks such that all the jobs selected can be processed within 10 working days and the total value of the jobs is maximized
Job Category
No. of Jobs to be processed
Estimated Completion time per job
Value rating per job
1 4 1 2
2 3 3 8
3 2 4 11
4 2 7 20
![Page 29: Chapter 10 Dynamic Programming](https://reader036.vdocument.in/reader036/viewer/2022081419/5681482b550346895db54e6d/html5/thumbnails/29.jpg)
29
Formulation• At stage 1, we must decide how many jobs from
category 1 to process• At stage 2, we must decide how many jobs from
category 2 to process, etc.• Value rating is a subjective score (scale 1 to 20)
• xn=No. of days of processing time remaining at the beginning of stage n (state variable)
• dn=No. of jobs processed from category n (decision variable at stage n)