0/1 Knapsack Dynamic Programming Optimization, from 2D matrix to 1D matrix

Question

I need some clarification from wikipedia: Knapsack, on the part

This solution will therefore run in O(nW) time and O(nW) space. Additionally, if we use only a 1-dimensional array m[W] to store the current optimal values and pass over this array i+1 times, rewriting from m[W] to m[1] every time, we get the same result for only O(W) space.

I am having trouble understanding how to turn a 2D matrix into a 1D matrix to save space. In addition, to what does rewriting from m[W] to m[1] every time mean (or how does it work).

Please provide some example. Say if I have the set {V,W} --> {(5,4),(6,5),(3,2)} with K = 9.

How would the 1D array look like?

Solution 1

In many dynamic programming problems, you will build up a 2D table row by row where each row only depends on the row that immediately precedes it. In the case of the 0/1 knapsack problem, the recurrence (from Wikipedia) is the following:

m[i, w] = m[i - 1, w] if w_i > w

m[i, w] = max(m[i - 1, w], m[i - 1, w - w_i] + v_i) otherwise

Notice that all reads from the table when filling row i only come from row i - 1; the earlier rows in the table aren't actually used. Consequently, you could save space in the 2D table by only storing two rows - the immediately previous row and the row you're filling in. You can further optimize this down to just one row by being a bit more clever about how you fill in the table entries. This reduces the space usage from O(nW) (O(n) rows and O(W) columns) to O(W) (one or two rows and O(W) columns).

This comes at a cost, though. Many DP algorithms don't explicitly compute solutions as they go, but instead fill in the table, then do a second pass over the table at the end to recover the optimal answer. If you only store one row, then you will get the value of the optimal answer, but you might not know what that optimal answer happens to be. In this case, you could read off the maximum value that you can fit into the knapsack, but you won't necessarily be able to recover what you're supposed to do in order to achieve that value.

Hope this helps!

Solution 2

I know this is an old question. But I had to spend some time searching for this and I'm just documenting the approaches here for anyone's future reference.

Method 1
The straightforward 2D method that uses N rows is:

int dp[MAXN][MAXW];
int solve()
{
    memset(dp[0], 0, sizeof(dp[0]));
    for(int i = 1; i <= N; i++) {
        for(int j = 0; j <= W; j++) {
            dp[i][j] = (w[i] > j) ? dp[i-1][j] : max(dp[i-1][j], dp[i-1][j-w[i]] + v[i]);
        }
    }
    return dp[N][W];
} 

This uses O(NW) space.

Method 2
You may notice that while calculating the entries of the matrix for a particular row, we're only looking at the previous row and not the rows before that. This can be exploited to maintain only 2 rows and keep swapping their roles as current & previous row.

int dp[2][MAXW];
int solve()
{
    memset(dp[0], 0, sizeof(dp[0]));
    for(int i = 1; i <= N; i++) {
        int *cur = dp[i&1], *prev = dp[!(i&1)];
        for(int j = 0; j <= W; j++) {
            cur[j] = (w[i] > j) ? prev[j] : max(prev[j], prev[j-w[i]] + v[i]);
        }
    }
    return dp[N&1][W];
}  

This takes O(2W) = O(W) space. cur is the i-th row and prev is the (i-1)-th row.
Method 3
If you look again, you can see that while we're writing an entry in a row, we're only looking at the items to the left of that in the previous row. We could use this to use a single row and process it right to left so that while we're computing new value for an entry, entries to its left have their old value. This is the 1D table method.

int dp[MAXW];
int solve()
{
    memset(dp, 0, sizeof(dp));
    for(int i =1; i <= N; i++) {
        for(int j = W; j >= 0; j--) {
            dp[j] = (w[i] > j) ? dp[j]: max(dp[j], dp[j-w[i]] + v[i]);
        }
    }
    return dp[W];
}

This also uses O(W) space but just uses a single row. The main reason the inner loop has to be reversed is because when we use dp[j-w[i]], we need the value from the previous iteration of outer loop. For this the j values have to be processed in a large to small fashion.

Test case (from http://www.spoj.com/problems/PARTY/)

N = 10, W = 50
w[] = {0, 12, 15, 16, 16, 10, 21, 18, 12, 17, 18} // 1 based indexing
v[] = {0, 3, 8, 9, 6, 2, 9, 4, 4, 8, 9}

answer = 26