dynamic programming 0-1 knapsack these notes are taken from the notes by dr. steve goddard at...
TRANSCRIPT
![Page 1: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/1.jpg)
Dynamic ProgrammingDynamic Programming0-1 Knapsack0-1 Knapsack
These notes are taken from the notes by Dr. Steve Goddard at
http://www.cse.unl.edu/~goddard/Courses/CSCE310J/Lectures/Lecture8-DynamicProgramming.pdf
![Page 2: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/2.jpg)
Recall: Dynamic Recall: Dynamic ProgrammingProgrammingBasic idea:
◦Optimal substructure: Optimal solution to problem consists of optimal solution to subproblems
◦Overlapping subproblems: Few subproblems in total, many recurring instances of each
◦Memoization: Solve bottom-up, building a table of solved subproblems that are used to solve larger ones
Variations◦“Table” could be 3D, triangular, a tree, etc
![Page 3: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/3.jpg)
0-1 Knapsack Problem0-1 Knapsack ProblemGiven a knapsack with maximum
capacity W, and a set S consisting of n items
Each item i has some weight wi and value vi
Problem: solve a version of 0-1 Knapsack where W, wi, and vi are all integer values◦How to pack the knapsack to achieve
maximum total value of packed items.
![Page 4: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/4.jpg)
Brute Force ApproachBrute Force ApproachSince there are n items, there are 2n
possible combination of itemsGo through all combinations and find
the one with maximum value and total weight less than or equal to W
Running time is (2n)
![Page 5: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/5.jpg)
Can We Do Better?Can We Do Better?Use an algorithm based on dynamic
programmingWe need to carefully identify the
subproblemTry this:
◦If items are labeled 1,…,n, then a subproblem would be to find an optimal solution forSk = {items labeled 1,…,k}
![Page 6: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/6.jpg)
Defining a SubproblemDefining a Subproblem
This is a reasonable subproblem definition
The question:◦Can we define the final solution (Sn) in
terms of subproblems (Sk)?
Unfortunately, we can’t do that
If items are labeled 1,…,n, then a subproblem would be to find an
optimal solution for Sk = {items labeled 1,…,k}
![Page 7: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/7.jpg)
Defining a SubproblemDefining a SubproblemSolution for S4 is not part of the
solution for S5
ItemWeight
wi
Value
viItem
Weight
wi
Value
vi
1 2 3 1 2 3
2 3 4 2 3 4
3 4 5 3 4 5
4 5 8 4 5 8
5 9 10
S4
S5
W = 20
Total weight: 20
Total value: 26
Total weight: 14
Total value: 20
![Page 8: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/8.jpg)
Defining a SubproblemDefining a SubproblemSo our definition of a subproblem is
flawed and we need another one!Let’s add another parameter: w,
which will represent the weight for each subset of items
The subproblem then will be to compute V[k,w]
![Page 9: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/9.jpg)
Recursive Formula for Recursive Formula for SubproblemsSubproblems
It means that the best subset of Sk that has total weight w is:◦The best subset of Sk-1 that has total
weight w OR◦The best subset of Sk-1 that has total
weight w - wk plus the item k
otherwise ,1,,1max
if ,1,
kk
k
vwwkVwkV
wwwkVwkV
![Page 10: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/10.jpg)
Recursive FormulaRecursive Formula
The best subset of Sk that has the total weight w either contains item k or not
First case: wk > w◦ Item k can’t be part of the solution since if it
was, the total weight would be > w, which is unacceptable
Second case: wk ≤ w ◦Then the item k can be in the solution, and we
choose the case with greater value
otherwise ,1,,1max
if ,1,
kk
k
vwwkVwkV
wwwkVwkV
![Page 11: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/11.jpg)
Optimal substructure: Optimal substructure: The 0-1 Knapsack Problem exhibits
optimal substructure◦Consider the most valuable load weighing
at most W pounds If we remove item k from the load, what do we
know about the remaining load? Answer: The remaining load must be the most
valuable load weighing at most W – wk that the thief could take, excluding item k
![Page 12: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/12.jpg)
How many possible subproblems?
![Page 13: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/13.jpg)
How many possible subproblems?nWUse a table of size (n+1) * (W+1)
![Page 14: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/14.jpg)
0-1 Knapsack Algorithm
for w = 0 to WV[0,w] = 0
for k = 1 to nV[k,0] = 0
for k = 1 to nfor w = 0 to W
if w[k] <= w // item k can be part of the solution
if v[k] + V[k-1,w-w[k]] > V[k-1,w]V[k,w] = v[k] + V[k-1,w- w[k]]
elseV[k,w] = V[k-1,w]
else V[k,w] = V[k-1,w] // w[k] > w
![Page 15: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/15.jpg)
ExampleExampleLet’s try this on the following data
◦n = 4 (number of items) ◦W = 5 (maximum weight knapsack can
hold)◦Items (weight, value)
1. (2, 3)2. (3, 4)3. (4, 5)4. (5, 6)
![Page 16: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/16.jpg)
Example (2)Example (2)
Initialization
otherwise ,1,,1max
if ,1,
kk
k
vwwkVwkV
wwwkVwkV
0 1 2 3 4 50 0 0 0 0 0 0
1 0
2 0
3 0
4 0
w
k
![Page 17: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/17.jpg)
Example (3)Example (3)
otherwise ,1,,1max
if ,1,
kk
k
vwwkVwkV
wwwkVwkV
0 1 2 3 4 50 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0
3 0
4 0
w
k
item # weight value
1 2 3
2 3 4
3 4 5
4 5 6
k = 1 w = 1, 2, 3, 4, 5
![Page 18: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/18.jpg)
Example (4)Example (4)
otherwise ,1,,1max
if ,1,
kk
k
vwwkVwkV
wwwkVwkV
0 1 2 3 4 50 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0
4 0
w
k
item # weight value
1 2 3
2 3 4
3 4 5
4 5 6
k = 2 w = 1, 2, 3, 4, 5
![Page 19: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/19.jpg)
Example (5)Example (5)
otherwise ,1,,1max
if ,1,
kk
k
vwwkVwkV
wwwkVwkV
0 1 2 3 4 50 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0
w
k
item # weight value
1 2 3
2 3 4
3 4 5
4 5 6
k = 3 w = 1, 2, 3, 4, 5
![Page 20: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/20.jpg)
Example (6)Example (6)
otherwise ,1,,1max
if ,1,
kk
k
vwwkVwkV
wwwkVwkV
0 1 2 3 4 50 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
w
k
item # weight value
1 2 3
2 3 4
3 4 5
4 5 6
k = 4 w = 1, 2, 3, 4, 5
![Page 21: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/21.jpg)
CommentsCommentsThis algorithm finds only the max
possible value that can be carried in the knapsack◦I.e. The value V[n, W]
To know the items that make this maximum value, we need to back track through the table◦Remember, we did the same thing with
LCS
![Page 22: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/22.jpg)
How to Find Actual ItemsHow to Find Actual ItemsLet k = n and w = WIf V[k, w] ≠ V[k-1, w] then
◦Mark the kth item as in the knapsack◦w = w – wk , k = k – 1
Else◦k = k – 1 // Assume the kth item in not in
the // knapsack
![Page 23: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/23.jpg)
Finding the ItemsFinding the Items
0 1 2 3 4 50 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
wk
0 1 2 3 4 50 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
w
k
0 1 2 3 4 50 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
w
k
0 1 2 3 4 50 0 0 0 0 0 0
1 0 0 3 3 3 3
2 0 0 3 4 4 7
3 0 0 3 4 5 7
4 0 0 3 4 5 7
w
k
item # weight value
1 2 3
2 3 4
3 4 5
4 5 6
![Page 24: Dynamic Programming 0-1 Knapsack These notes are taken from the notes by Dr. Steve Goddard at goddard/Courses/CSCE310J/Lectures/Lecture8-](https://reader035.vdocument.in/reader035/viewer/2022062320/56649d825503460f94a671f8/html5/thumbnails/24.jpg)
Running time?O(nW)
◦pseudo-polynomial◦O(nW) complexity does not contradict the
fact that the 0-1 knapsack problem is NP-complete, since W, unlike n, is not polynomial in the length of the input to the problem.
◦Also remember the DP solution only works if the weights are integers!