0/1 Knapsack Problem Explained

100% found this document useful (1 vote)
275 views18 pages
The knapsack problem involves packing items into a knapsack with limited weight capacity to maximize the total value of items. There are two versions: 0-1 knapsack where items are indivisibl…

Uploaded by

pooja0100
  • Introduction to Knapsack Problem
  • Types of Knapsack Problems
  • 0-1 Knapsack Problem Details
  • 0-1 Knapsack Problem: Brute-Force Approach
  • 0-1 Knapsack Problem: Dynamic Programming Approach
  • Defining a Subproblem
  • Recursive Formula for Subproblems
  • 0-1 Knapsack Algorithm
  • Running Time Analysis

Knapsack problem

Given some items, pack the knapsack to get


the maximum total value. Each item has some
weight and some value. Total weight that we can
carry is no more than some fixed number W.
So we must consider weights of items as well as
their values.
Item #
1
2
3

Weight Value
1
8
3
6
5
5
1

Knapsack problem
There are two versions of the problem:
1. 0-1 knapsack problem

Items are indivisible; you either take an item or not. Some


special instances can be solved with dynamic programming

2. Fractional knapsack problem

Items are divisible: you can take any fraction of an item

0-1 Knapsack problem

Given a knapsack with maximum capacity W, and


a set S consisting of n items

Each item i has some weight wi and benefit value


bi (all wi and W are integer values)

Problem: How to pack the knapsack to achieve


maximum total value of packed items?

0-1 Knapsack problem

Problem, in other words, is to find

max bi subject to wi W
iT

iT

The problem is called a 0-1 problem,


because each item must be entirely
accepted or rejected.

0-1 Knapsack problem:


brute-force approach
Lets first solve this problem with a
straightforward algorithm

Since there are n items, there are 2n possible


combinations of items.
We go through all combinations and find the one
with maximum value and with total weight less or
equal to W
Running time will be O(2n)

0-1 Knapsack problem:


dynamic programming approach

We can do better with an algorithm based on


dynamic programming

We need to carefully identify the subproblems

Defining a Subproblem

Given a knapsack with maximum capacity W, and


a set S consisting of n items

Each item i has some weight wi and benefit value


bi (all wi and W are integer values)

Problem: How to pack the knapsack to achieve


maximum total value of packed items?

Defining a Subproblem

We can do better with an algorithm based on


dynamic programming

We need to carefully identify the subproblems


Lets try this:
If items are labeled 1..n, then a subproblem
would be to find an optimal solution for
Sk = {items labeled 1, 2, .. k}
8

Defining a Subproblem
If items are labeled 1..n, then a subproblem would be
to find an optimal solution for Sk = {items labeled
1, 2, .. k}

This is a reasonable subproblem definition.

The question is: can we describe the final solution


(Sn ) in terms of subproblems (Sk)?

Unfortunately, we cant do that.

Defining a Subproblem
w1 =2 w2 =4
b1 =3 b2 =5

w3 =5
b3 =8

Weight Benefit

w4 =3
b4 =4

?
Max weight: W = 20
For S4:
Total weight: 14
Maximum benefit: 20

w1 =2 w2 =4
b1 =3 b2 =5

w3 =5
b3 =8

w5 =9
b5 =10

For S5:
Total weight: 20
Maximum benefit: 26

wi

bi

10

Item

#
S4
S5

Solution for S4 is
not part of the
solution for S !!!

10

Defining a Subproblem

As we have seen, the solution for S4 is not part of the


solution for S5

So our definition of a subproblem is flawed and we


need another one!

11

Defining a Subproblem

Given a knapsack with maximum capacity W, and


a set S consisting of n items

Each item i has some weight wi and benefit value


bi (all wi and W are integer values)

Problem: How to pack the knapsack to achieve


maximum total value of packed items?

12

Defining a Subproblem

Lets add another parameter: w, which will represent


the maximum weight for each subset of items

The subproblem then will be to compute V[k,w], i.e.,


to find an optimal solution for Sk = {items labeled 1,
2, .. k} in a knapsack of size w

13

Recursive Formula for


subproblems

The subproblem will then be to compute V[k,w], i.e.,


to find an optimal solution for Sk = {items labeled 1,
2, .. k} in a knapsack of size w

Assuming knowing V[i, j], where i=0,1, 2, k-1,


j=0,1,2, w, how to derive V[k,w]?

14

Recursive Formula for


subproblems (continued)
Recursive formula for subproblems:
V [k 1, w]
if wk w
V [ k , w]
max{V [k 1, w],V [k 1, w wk ] bk } else

It means, that the best subset of Sk that has total


weight w is:
1) the best subset of Sk-1 that has total weight w,

or

2) the best subset of Sk-1 that has total weight w-wk plus
the item k

15

Recursive Formula
V [k 1, w]
if wk w
V [ k , w]
max{V [k 1, w],V [k 1, w wk ] bk } else

The best subset of Sk that has the total weight w,


either contains item k or not.
First case: wk>w. Item k cant be part of the solution,
since if it was, the total weight would be > w, which
is unacceptable.
Second case: wk w. Then the item k can be in the
solution, and we choose the case with greater value.

16

0-1 Knapsack Algorithm


for w = 0 to W
V[0,w] = 0
for i = 1 to n
V[i,0] = 0
for i = 1 to n
for w = 0 to W
if wi <= w // item i can be part of the solution
if bi + V[i-1,w-wi] > V[i-1,w]
V[i,w] = bi + V[i-1,w- wi]
else
V[i,w] = V[i-1,w]
else V[i,w] = V[i-1,w] // wi > w
17

Running time
for w = 0 to W
O(W)
V[0,w] = 0
for i = 1 to n
V[i,0] = 0
Repeat n
for i = 1 to n
for w = 0 to W
O(W)
< the rest of the code >

times

What is the running time of this


algorithm?
O(n*W)
Remember that the brute-force algorithm
takes O(2n)

18

Common questions

Powered by AI

Introducing a weight parameter allows a structured evaluation of subproblems that aggregate to the solution, reducing complexity from exponential to polynomial time, specifically O(n*W). This makes the problem computationally feasible for larger instances by focusing only on viable solutions within given weight constraints rather than all permutations .

Dynamic programming carefully constructs subproblems by assessing all viable configurations at each step, leading to an optimal solution for the 0-1 knapsack problem. In contrast, a greedy approach exploits fractional selections and makes local optimal decisions based on value-to-weight ratios, which does not guarantee an optimal global solution but is feasible for fractional knapsack problems .

A dynamic programming approach is preferred because it accommodates additional parameters or constraints by breaking down the problem into manageable subproblems, efficiently storing intermediate results to prevent redundant processing. This adaptability is crucial for complex instances where brute-force methods would be computationally prohibitive due to exponential growth in potential solutions .

Defining V[k,w] as the optimal value for the first k items with a maximum weight capacity w allows the solution to be recursively built by considering whether to include each new item based on weight constraints. This recursive relationship enables efficient computation by breaking down the main problem into manageable subproblems and avoiding flawed subproblem definitions as seen in prior attempts .

The brute-force method for the 0-1 knapsack problem assesses all possible combinations of items, resulting in a time complexity of O(2^n). In contrast, the dynamic programming approach reduces this complexity to O(n*W) by solving subproblems and storing their results, which prevents redundant calculations and significantly improves efficiency .

The 0-1 knapsack problem involves items that are indivisible, meaning each item can either be taken or completely left out. In contrast, the fractional knapsack problem allows items to be divisible, meaning any fraction of an item can be chosen .

The case distinction highlights the binary nature of the 0-1 knapsack problem, where each item's inclusion is conditional upon current weight constraints. It reveals that decision-making must consider the trade-off between value gain and weight increase, emphasizing the greedy element inherent in finding an optimum subset of items .

The initialization sets V[0,w] = 0 and V[i,0] = 0 for all i, indicating preparedness by establishing a base case where no items or zero weight capacity yield zero benefit. This ensures that the algorithm can handle all constraints as it recursively builds towards the optimal solution through state transitions .

The initial subproblem definition lacked the maximum weight parameter and attempted to find optimal solutions for subsets only by item count, which did not accurately reflect weight constraints for the overall problem. This led to scenarios where solutions for smaller problems were not applicable to larger ones, showing the importance of including weight as a defining factor .

In the recursive formula, the first case, wk > w, represents the scenario where the item k cannot be included because it would exceed the weight limit. The second case, wk ≤ w, allows item k to be considered, and the solution takes the maximum value between including this item or opting for the previous best solution without it, guiding the construction of the optimal solution .

1
Given some items, pack the knapsack to get 
the maximum total value. Each item has some 
weight and some value. Total weigh
2
Knapsack problem
There are two versions of the problem:
1.
“0-1 knapsack problem”

Items are indivisible; you either take
3
Given a knapsack with maximum capacity W, and 
a set S consisting of n items
Each item i has some weight wi and benefit v
4
Problem, in other words, is to find





T
i
i
T
i
i
W
w
b
 
subject to
 
max
0-1 Knapsack problem
The problem is ca
5
Let’s first solve this problem with a 
straightforward algorithm
Since there are n items, there are 2n possible 
combinati
6
We can do better with an algorithm based on 
dynamic programming
We need to carefully identify the subproblems
0-1 Knapsa
7
Given a knapsack with maximum capacity W, and 
a set S consisting of n items
Each item i has some weight wi and benefit v
8
We can do better with an algorithm based on 
dynamic programming
We need to carefully identify the subproblems
Let’s try
9
If items are labeled 1..n, then a subproblem would be 
to find an optimal solution for Sk = {items labeled 
1, 2, .. k}
Th
10
Max weight: W = 20
For S4:
Total weight: 14
Maximum benefit: 20
w1 =2
b1 =3
w2 =4
b2 =5
w3 =5
b3 =8
w4 =3
b4 =4
wi
bi
10
8

You might also like