BackPack(KnapSack) Dynamic Programming Problems

This blog talks about using dynamic programming to solve the famous 0/1 back pack (knapsack) and its variant problems.

 

BackPack I

Given n items with size Ai, an integer m denotes the size of a backpack. How full you can fill this backpack?

You can not divide any item into small pieces.

Example

If we have 4 items with size [2, 3, 5, 7], the backpack size is 11, we can select [2, 3, 5], so that the max size we can fill this backpack is 10. If the backpack size is 12. we can select [2, 3, 7] so that we can fulfill the backpack.

You function should return the max size we can fill in the given backpack.

Challenge 

O(n x m) time and O(m) memory.

O(n x m) memory is also acceptable if you do not know how to optimize memory.

 

State: T[i][j]: the max size we can get out of the first i items when the backpack size is j. 

Function: T[i][j] = Math.max(T[i - 1][j], T[i - 1][j - A[i - 1]] + A[i - 1]), if j >= A[i - 1];

               T[i][j] = T[i - 1][j], if j < A[i - 1]. 

Initialization: T[i][0] = T[0][j] = 0

Answer: T[A.length][m]

 

O(n * m) time, O(n * m) memory

public class Solution {
    public int backPack(int m, int[] A) {
        if(A == null || A.length == 0){
            return 0;
        }
        int[][] T = new int[A.length + 1][m + 1];
        for(int i = 0; i <= A.length; i++){
            T[i][0] = 0;
        }
        for(int j = 0; j <= m; j++){
            T[0][j] = 0;
        }
        for(int i = 1; i <= A.length; i++){
            for(int j = 1; j <= m; j++){
                if(j >= A[i - 1]){
                    T[i][j] = Math.max(T[i - 1][j], T[i - 1][j - A[i - 1]] + A[i - 1]);                    
                }
                else{
                    T[i][j] = T[i - 1][j];
                }
            }
        }
        return T[A.length][m];
    }
}

 

O(n * m) time, O(m) memory with rolling array

public class Solution {
    public int backPack(int m, int[] A) {
        if(A == null || A.length == 0){
            return 0;
        }
        int[][] T = new int[2][m + 1];
        for(int j = 0; j <= m; j++){
            T[0][j] = 0; 
        }
        for(int i = 1; i <= A.length; i++){
            for(int j = 1; j <= m; j++){
                if(j >= A[i - 1]){
                    T[i % 2][j] = Math.max(T[(i - 1) % 2][j], T[(i - 1) % 2][j - A[i - 1]] + A[i - 1]);                    
                }
                else{
                    T[i % 2][j] = T[(i - 1) % 2][j];
                }
            }
        }
        return T[A.length % 2][m];
    }
}

  

BackPack II

Given n items with size Ai and value Vi, and a backpack with size m. What's the maximum value can you put into the backpack?

You cannot divide item into small pieces and the total size of items you choose should smaller or equal to m.

Example

Given 4 items with size [2, 3, 5, 7] and value [1, 5, 2, 4], and a backpack with size 10. The maximum value is 9.

Challenge 

O(n x m) memory is acceptable, can you do it in O(m) memory?

 

BackPack II is almost identical with BackPack I. The only difference is that in I, we try to maximize the total size, but in II, we try to maximize the total value.

 

State: T[i][j]: the max value we can get out of the first i items when the backpack size is j. 

Function: T[i][j] = Math.max(T[i - 1][j], T[i - 1][j - A[i - 1]] + V[i - 1]), if j >= A[i - 1];

               T[i][j] = T[i - 1][j], if j < A[i - 1]. 

Initialization: T[i][0] = T[0][j] = 0

Answer: T[A.length][m]

 

O(n * m) time, O(n * m) memory

public class Solution {
    public int backPackII(int m, int[] A, int V[]) {
        int[][] T = new int[A.length + 1][m + 1];
        for(int i = 0; i <= A.length; i++){
            T[i][0] = 0;
        }
        for(int j = 0; j <= m; j++){
            T[0][j] = 0;
        }
        for(int i = 1; i <= A.length; i++){
            for(int j = 1; j <= m; j++){
                if(j >= A[i - 1]){
                    T[i][j] = Math.max(T[i - 1][j], T[i - 1][j - A[i - 1]] + V[i - 1]);                    
                }
                else{
                    T[i][j] = T[i - 1][j];
                }
            }
        }
        return T[A.length][m];
    }
}

 

O(n * m) time, O(m) memory with rolling array 

public class Solution {
    public int backPackII(int m, int[] A, int V[]) {
        int[][] T = new int[2][m + 1];
        for(int j = 0; j <= m; j++){
            T[0][j] = 0;
        }
        for(int i = 1; i <= A.length; i++){
            for(int j = 1; j <= m; j++){
                if(j >= A[i - 1]){
                    T[i % 2][j] = Math.max(T[(i - 1) % 2][j], T[(i - 1) % 2][j - A[i - 1]] + V[i - 1]);                    
                }
                else{
                    T[i % 2][j] = T[(i - 1) % 2][j];
                }
            }
        }
        return T[A.length % 2][m];
    }
}

 

Follow up question: If you are allowed to divide any item into small pieces, how does this change affect your algorithm?

A: In this case, we can simply apply a greedy algorithm described in the following.

1. sort all the items in descending order by their value to weight ratio.

2. starting from the item that has the highest ratio, fill the backpack until it is full. If the last item overflows the backpack,

split into a right proportion so that it fits the remaining empty space. 

 

If the toal weights of all items is >=  the size of backpack, it is guaranteed that the backpack is always completely filled.

 

BackPack III

Given n kind of items with size Ai and value Vi( each item has an infinite number available) and a backpack with size m. What's the maximum value can you put into the backpack?

You cannot divide item into small pieces and the total size of items you choose should smaller or equal to m.

Example

Given 4 items with size [2, 3, 5, 7] and value [1, 5, 2, 4], and a backpack with size 10. The maximum value is 15.

 

In II, we only have 1 of each item; In III, we can have infinite number of each item. So after we pick the ith item, 

we should still consider picking the same ith item again as long as the remaining weight is >= A[i - 1]. In II, once

we picked the ith item, we should only consider the max value of the first (i - 1) items with the remaining weight

j - A[i - 1].

 

State: T[i][j]: the max value we can, when we are given the first i items to choose from and the backpack size is j. 

Function: T[i][j] = Math.max(T[i - 1][j], T[i][j - A[i - 1]] + V[i - 1]), if j >= A[i - 1];

               T[i][j] = T[i - 1][j], if j < A[i - 1]. 

Initialization: T[i][0] = T[0][j] = 0

Answer: T[A.length][m]

 

O(n * m) time, O(n * m) memory

public class Solution {
    public int backPackIII(int[] A, int[] V, int m) {
        int[][] T = new int[A.length + 1][m + 1];
        for(int i = 0; i <= A.length; i++){
            T[i][0] = 0;
        }
        for(int j = 0; j <= m; j++){
            T[0][j] = 0;
        }
        for(int i = 1; i <= A.length; i++){
            for(int j = 1; j <= m; j++){
                if(j >= A[i - 1]){
                    T[i][j] = Math.max(T[i - 1][j], T[i][j - A[i - 1]] + V[i - 1]);
                }
                else{
                    T[i][j] = T[i - 1][j];
                }
            }
        }
        return T[A.length][m];
    }
}

 

O(n * m) time, O(m) memory

public class Solution {
    public int backPackIII(int[] A, int[] V, int m) {
        int[][] T = new int[2][m + 1];
        for(int j = 0; j <= m; j++){
            T[0][j] = 0;
        }
        for(int i = 1; i <= A.length; i++){
            for(int j = 1; j <= m; j++){
                if(j >= A[i - 1]){
                    T[i % 2][j] = Math.max(T[(i - 1) % 2][j], T[i % 2][j - A[i - 1]] + V[i - 1]);
                }
                else{
                    T[i % 2][j] = T[(i - 1) % 2][j];
                }
            }
        }
        return T[A.length % 2][m];
    }
}

 

O(n * m) time, O(m) memory

State: T[j]: The max value we can get out of all items when the backpack size is j.

Function: T[j] = Math.max(T[j - A[i]] + V[i], T[j]), for j >= A[i];  

When j >= A[i], it means we can pick 1 more A[i], which may yield a bigger value;

When j < A[i],  we can't pick A[i], no need to do any update check.

Initialization: T[0] = 0

Answer: T[m]

 

This solution is essentially the same with the solution using rolling array. 

public class Solution {
    public int backPackIII(int[] A, int[] V, int m) {
        int[] T = new int[m + 1];
        T[0] = 0;
        for(int i = 0; i < A.length; i++){
            for(int j = A[i]; j <= m; j++){
                T[j] = Math.max(T[j - A[i]] + V[i], T[j]);
            }
        }
        return T[m];
    }
}

 

BackPack IV

Given n items with size nums[i]. nums is an integer array with all positive numbers, no duplicates. An integer target denotes the size of a backpack. Find the number of all possible ways of filling the backpack.

Each item may be chosen unlimited number of times

Example

Given candidate items [2,3,6,7] and target 7,

A solution set is: 
[7]
[2, 2, 3]

return 2

 

State: T[i][j]: the number of all possible ways of filling size j with the first i items.

Function: T[i][j] += T[i - 1][j - k * nums[i - 1]]; for all k that satisfies k * nums[i - 1] <= j

This function basically says for the ith item,  we have have k = 0, 1, or 2...... of it; The stop condition is when k ith items' size is bigger than j.

Initialization: 

T[i][0] = 1; i = 0, 1, ...... nums.length; //This is different with I, II, III; When we need to fill a backpack of size 0, we do have one way of doing it, which is to do nothing.

T[0][j] = 0; j = 1, 2, ....... target;

Answer:

T[nums.length][target]

 

public class Solution {
    public int backPackIV(int[] nums, int target) {
        if(nums == null){
            return -1;
        }
        int[][] T = new int[nums.length + 1][target + 1];
        for(int i = 0; i <= nums.length; i++){
            T[i][0] = 1;
        }
        for(int j = 1; j <= target; j++){
            T[0][j] = 0;
        }
        for(int i = 1; i <= nums.length; i++){
            for(int j = 1; j <= target; j++){
                int k = 0; 
                while(k * nums[i - 1] <= j){
                    T[i][j] += T[i - 1][j - k * nums[i - 1]];
                    k++;
                }
            }    
        }
        return T[nums.length][target];
    }
}

 

Optimization to O(n * m) time,  O(m) memory

State: T[j]: the number of all possible ways of filling size j with all items.

Function: T[j] += T[j - nums[i]], if nums[i] <= j; i = 0, 1, 2, ..... nums.length - 1;

Initialization: T[0] = 1; The only way of filling size 0 with all items is to do nothing.

Answer: T[target]

 

public class Solution {
    public int backPackIV(int[] nums, int target) {
        if(nums == null){
            return -1;
        }
        int[] T = new int[target + 1];
        T[0] = 1;
 8         for(int i = 0; i < nums.length; i++){
 9             for(int j = 1; j <= target; j++){
                if(nums[i] <= j){
                    T[j] += T[j - nums[i]];
                }
            }
        }
        return T[target];
    }
}

 

 

BackPack VI (Same with Combination Sum IV)

Given an integer array nums with all positive numbers and no duplicates, find the number of possible combinations that add up to a positive integer target.

A number in the array can be used multiple times in the combination. 
Different orders are counted as different combinations.

Example

Given nums = [1, 2, 4], target = 4

The possible combination ways are:
[1, 1, 1, 1]
[1, 1, 2]
[1, 2, 1]
[2, 1, 1]
[2, 2]
[4]

return 6

 

The code of BackPack VI is almost identical with BackPack IV. The only difference is that the order of the two level

for-loops is different.

In BackPack I, II, III, IV, the reason that we can use a 1D array is because the 2D array solution only uses the

previous rows' information, so it is a pure space optimization.  BackPack VI is different, it is an 1D dynamic programming problem.  

For a given sum i, we search across the whole array to find all different combinations that sum up to i.

 

BackPack IV: element order does not matter;

BackPack VI: element order matters; [1, 1, 2] and [1, 2, 1] are two different combinations.

 

State: T[j]: the number of all possible ways of summing up to j with all items.

Function: T[j] += T[j - nums[i]], if nums[i] <= j; i = 0, 1, 2, ..... nums.length - 1;

Initialization: T[0] = 1; The only way of summing up to 0 with all items is to not pick any item.

Answer: T[target]

public class Solution {
    public int backPackVI(int[] nums, int target) {
        if(nums == null || target <= 0){
            return -1;
        }
        int[] T = new int[target + 1];
        T[0] = 1;
 8         for(int j = 1; j <= target; j++){
 9             for(int i = 0; i < nums.length; i++){
                if(nums[i] <= j){
                    T[j] += T[j - nums[i]];
                }
            }
        }
        return T[target];
    }
}

 

 

Knapsack With Weight Constraints 

The knapsack problem is a classical problem in combinatorial optimization: Given a set of n items, each with a weight wi and a value vi, determine the items to include in a collection so that the total weight is less than or equal to a given limit w and the total value is as large as possible.

Your goal is to solve a special case of the knapsack problem with additional constraint: the weight of each item is a power of 2.

Find the the set of items with the maximum total value so that their total weight is at most w.

 

Input

The first line contains two integers n and w (1 ≤ n ≤ 100, 0 ≤ w ≤ 10^9), the number of items and the maximal total weight of items in the knapsack.

Each of the next n lines contains two integers wi and vi (1 ≤ wi ≤ 2^30, 0 ≤ vi ≤ 10^6), the weight and the value of i-th item in the set. Each wi is a power of 2.

Output

The maximum total value of items in the selected set.

 

 

BackPack VI Related Problems

Combination Sum IV

Coin Change