315 Notes, Wednesday, April 30, 2008

Dynamic Programming, ctd.

Last time we touched upon the computation of the n-th Fibonacci number and discovered that the recursive procedure would take time proportional to the number of nodes in the tree of recursive calls, N(n)=1+N(n-1)+N(n-2) for n>1 (that is, counting the root and the nodes in both subtrees of recursive calls that compute the (n-1)-th and (n-2)-th Fibonacci numbers.) You should convince yourself that N(n) grows exponentially, even though you cannot apply the master theorem for recurrence equations.

A quick application of DP to Fib:

Structure: Fib(n)=Fib(n-1)+Fib(n-2) for n>1 (no optimizing here, just computing)
Size: n, the value of the argument.
Optimal solution: smallest sizes (Fib(0)=0, Fib(1)=1, stored in F[0] and F[1]) followed by larger sizes (argument values)
for i from 2 to n do F[i]=F[i-1]+F[i-2]

Trivial, but worth studying. We can now return to our original example of

Matrix Chain Product

The problem is to determine the cheapest (in terms of the number of scalar multiplications) way to multiply a number of matrices in a chain of compatible dimensions.

M_1 x M_2 x ... x M_{n-1} x M_n (*)

The compatibility condition is that the number of rows and columns in consecutive matrices agree: if M_i has d_i rows and q_i columns, then M_{i+1} must have q_i rows and r_i columns, for all i, 1≤i≤n. Since computing a single such product takes d_i.q_i.r_i scalar multiplications, different ways of paranthesising the expression (*) could result in widely different complexity of computing the matrix chain product. The challenge is to find a minimum such complexity, given the dimensions d_0, d_1, ... , d_n (so that M_i has d_{i-1} rows and d_i columns.) Let us call the minimum number of scalar multiplications while computing the matrix chain product M_i x M_{i+1} x ... x M_{j-1} x M_j its cost, m(i,j). The paradigm of Dynamic Programming can guide us to design an algorithm computing it.

Specify the recursive structure of the solution. Whatever the last multiplication is, let's say, (M_1 x ... x M_k) x (M_{k+1} x... x M_n), the component products have to be computed optimally at the cost m(1,k) and m(k+1,n). The final multiplication costs d_0.d_k.d_n. If m(i,j) are available in constant time for all j-i<n,then all we need to do is to find (in O(n) time) the value of k that minimizes m(i,k-1)+m(k,n)+ d_0.d_k.d_n.

Characterize the size of subproblems. The number of matrices in a product, so that the base cases (we can eithr consider 0 or 1 the base case) have the smallest sizes and the overall problem has the largest size (n).

Solve optimally all subproblems in the order of their increasing sizes. Trivially, all M[i,i]=0. Storing the optimal solutions in M[i,j], we can find all M[i,i+1] (1≤1<n) and so on in time O(n) each. This results in an O(n^3) time algorithm.