Left Matrix Division and Numpy Solve

Question

I am trying to convert code that contains the \ operator from Matlab (Octave) to Python. Sample code

B = [2;4]
b = [4;4]
B \ b

This works and produces 1.2 as an answer. Using this web page

http://mathesaurus.sourceforge.net/matlab-numpy.html

I translated that as:

import numpy as np
import numpy.linalg as lin
B = np.array([[2],[4]])
b = np.array([[4],[4]])
print lin.solve(B,b)

This gave me an error:

numpy.linalg.linalg.LinAlgError: Array must be square

How come Matlab \ works with non square matrix for B?

Any solutions for this?

Solution 1

From MathWorks documentation for left matrix division:

If A is an m-by-n matrix with m ~= n and B is a column vector with m components, or a matrix with several such columns, then X = A\B is the solution in the least squares sense to the under- or overdetermined system of equations AX = B. In other words, X minimizes norm(A*X - B), the length of the vector AX - B.

The equivalent in numpy is np.linalg.lstsq:

In [15]: B = np.array([[2],[4]])

In [16]: b = np.array([[4],[4]])

In [18]: x,resid,rank,s = np.linalg.lstsq(B,b)

In [19]: x
Out[19]: array([[ 1.2]])

Solution 2

Matlab will actually do a number of different operations when the \ operator is used, depending on the shape of the matrices involved (see here for more details). In you example, Matlab is returning a least squares solution, rather than solving the linear equation directly, as would happen with a square matrix. To get the same behaviour in numpy, do this:

import numpy as np
import numpy.linalg as lin
B = np.array([[2],[4]])
b = np.array([[4],[4]])
print np.linalg.lstsq(B,b)[0]

which should give you the same solution as Matlab.

Solution 3

You can form the left inverse:

import numpy as np
import numpy.linalg as lin
B = np.array([[2],[4]])
b = np.array([[4],[4]])

B_linv = lin.solve(B.T.dot(B), B.T)
c = B_linv.dot(b)
print('c\n', c)

Result:

c
 [[ 1.2]]

Actually, we can simply run the solver once, without forming an inverse, like this:

c = lin.solve(B.T.dot(B), B.T.dot(b))
print('c\n', c)

Result:

c
 [[ 1.2]]

.... as before

Why? Because:

We have:

Multiply through by B.T, gives us:

Now, B.T.dot(B) is square, full rank, does have an inverse. And therefore we can multiply through by the inverse of B.T.dot(B), or use a solver, as above, to get c.