knapsack (2)
DESCRIPTION
KnapsackTRANSCRIPT
![Page 1: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/1.jpg)
Brute Force Approaches
• Techniques for finding optimal solutions to hard problems “relatively” quickly
• Backtracking
• Branch and Bound
• Combining efficient solutions with brute force approaches
• Other issues
![Page 2: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/2.jpg)
The Knapsack Problem
• Input– Capacity K
– n items with weights wi and values vi
• Goal– Output a set of items S such that
• the sum of weights of items in S is at most K
• and the sum of values of items in S is maximized
![Page 3: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/3.jpg)
Greedy Does Not Work
• What are possible greedy strategies?– Highest Density First– Highest Value First– Lowest Weight First
• Prove these are not optimal for the 0-1 knapsack problem.
![Page 4: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/4.jpg)
Multi-constraint Knapsack
• Input– Capacity K1 and K2
– n items with integer weights wi, size si, and values vi
• Goal– Output a set of items S such that
• the sum of weights of items in S is at most K1
• the sum of sizes of items in S is at most K2
• and the sum of values of items in S is maximized
• Can we use dynamic programming?
![Page 5: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/5.jpg)
Situation where dynamic programming does not work
What if our problem can’t be described with integers?
wA = 2 vA = $40
wB = vB = $50
wC = 1.98 vC = $100
wD = 5 vD = $95
wE = 3 vE = $30
We have to resort to brute force….
![Page 6: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/6.jpg)
Brute Force
• Generate all possible solutions– With n items, there are 2n solutions to be
generated – Check each to see if they satisfy the constraint– Save maximum solution that satisfies constraint
• Can be represented as a tree
![Page 7: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/7.jpg)
Brute Force: BranchingA
B
C
D
E E
D
E E
C
D
E E
D
E E
B
C
D
E E
D
E E
C
D
E E
D
E E
In Out
Weight = 15.12Value = $315
Weight = 8.98Value = $235
Weight = 9.98Value = $225
![Page 8: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/8.jpg)
Backtracking
• In the tree representation, we can think of the previous algorithm doing a DFS in the tree
• If we reach a point where a solution no longer is feasible, there is no need to continue exploring
• We can “backtrack” from this point and potentially cut off much of the tree and many of the solutions
• In the given example, backtracking would be much more effective if we had even more items or a smaller knapsack capacity
![Page 9: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/9.jpg)
BacktrackingA
B
C
D
E E
D
E E
C
D
E E
D
E E
B
C
D
E E
D
E E
C
D
E E
D
E E
In Out
Weight = 8.98Value = $235
Weight = 9.98Value = $225
2, $40
, $50
1.98, $100
5, $95
3, $30
![Page 10: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/10.jpg)
Branch and Bound
• We can backtrack if we know the best possible solution in current subtree is worse than current best solution obtained so far.
• Estimates for improvement given current ordering– A down can give $315– B down -> $275– C down -> $225– D down -> $125– E down -> $30
![Page 11: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/11.jpg)
Branch and BoundA
B
C
D
E E
D
E E
C
D
E E
D
B
C
D
E E
D
C
In Out
Weight = 7.12Value = $190
Weight = 8.98Value = $235
2, $40
, $50
1.98, $100
5, $95
3, $30
![Page 12: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/12.jpg)
Generating a good bound
• The key to branch and bound is having a good initial bound.
• How might we generate a good initial bound?
![Page 13: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/13.jpg)
Order of items
• Since there is no fixed ordering of items, is there a better way to order the input items?– Highest weight first
• Generate infeasible solutions quickly– Highest density first
• Generate good solution quickly– Which is better depends on input
2, $40
, $50
1.98, $100
5, $95
3, $30
![Page 14: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/14.jpg)
Heaviest on TopD
B
E
A A
C C
E
A
C C
A
C C
B
In Out
Weight = 8.14Value = $145
Weight = 10.0Value = $165
Weight = 9.98Value = $225
Weight = 8.98Value = $235
(Max remaining = $220)
D: 5, $95
B: , $50
E: 3, $30
A: 2, $40
C: 1.98, $100
![Page 15: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/15.jpg)
Best Density on TopC
A
D
B
E E
B
D
B
E E
B
A
In Out
Weight = 8.98Value = $235
C: 1.98, $100 A: 2, $40
D: 5, $95
B: , $50
E: 3, $30
![Page 16: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/16.jpg)
Greedy and Brute Force
• Suppose half the inputs to our knapsack problem all have the same weight.– If all inputs had the same weight, we could
implement a greedy solution– However, since half do not, we cannot use
greedy alone to find optimal solution
• Combine brute-force approach with greedy to find optimal solution more quickly.
![Page 17: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/17.jpg)
The Breakdown...F1
F2
F3
F4
F5
F4
F3
F4 F4
F2
F3
F4 F4
F3
F4 F4
In Out
Greedyon half!
F5 F5 F5 F5 F5 F5 F5 F5 F5
![Page 18: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/18.jpg)
Algorithm
• Backtrack on half the inputs
• At leafs, apply greedy strategy on the other half of the inputs
• Comparison– Pure brute force: O(2n)– Combination: O(n2n/2)
![Page 19: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/19.jpg)
How much better?
• Assumptions– Suppose n=50 – We can test 1,000,000 solutions/second.
• 250 would take over 35 years
• 225 can be generated in half an hour– Plus marginal time to generate greedy solution
for each of the 225 solutions
![Page 20: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/20.jpg)
Another combination
• Suppose half the inputs to our knapsack problem have small integral weights/values while the other half have real weights/values.
• How can we combine approaches to solve this efficiently?
![Page 21: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/21.jpg)
The n-Queens Problem
• Input– Positive integer n
• Task– Place n queens on an n by n chessboard so that
no two queens attack each other (on same row, column, diagonal), or report that this is impossible
• Solve the n-queens problem for n = 1, 2, 3
![Page 22: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/22.jpg)
n=4
• Pure brute force search– 16 squares on a 4 by 4 chessboard– Leads to 16 * 15 * 14 * 13 = 43,680 possible
solutions
• Improvements– At most one queen per row: 44 = 256 possible
solutions– Backtracking: If two queens already attack
before final queen placed, backtrack
![Page 23: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/23.jpg)
Larger values of n
• n=8– At most one queen per row: 88 = 16,777,216– Early backtracking: 2057 nodes total– Time to find first solution: 114 nodes
• n=12– At most one queen per row: 1212 – Early backtracking: 856,189 nodes total– Time to find first solution: 262 nodes
![Page 24: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/24.jpg)
The ProblemIn the U.S. navy, the SEALS are each specially trained in a wide variety of skills so that small teams can handle a multitude of missions.
If there are k different skills needed for a mission, and n SEAL members that can be assigned to the team, find the smallest team that will cover all of the required skills.
Andersen knows hand-to-hand, first aid, and camouflageButler knows hand-to-hand and snaresCunningham knows hand-to-handDouglas knows hand-to-hand, sniping, diplomacy, and snaresEckers knows first-aid, sniping, and diplomacy
![Page 25: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/25.jpg)
Greedy Algorithm
• What is the obvious greedy algorithm for this problem?
• Find a counter-example to the optimality of greedy for this problem.
![Page 26: Knapsack (2)](https://reader033.vdocument.in/reader033/viewer/2022061605/563dba85550346aa9aa65379/html5/thumbnails/26.jpg)
Brute Force Approach
• What is the brute-force approach?
• How can we simplify the problem as far as possible in polynomial time?
• How can we set bounds on the solutions?
• When will we need to do backtracking?
• What order should we test the potential members in when branching?