Least Squares

x

Axbresidual r = 1.36558minimal residual 

r

min

 = 1.00394

When a matrix A is square with full rank, there is a vector

x

that satisfies the equation

Ax=b

for any

b

. However, when A is not square or does not have full rank, such an

x

may not exist, because b does not lie in the range of A. In this case, called the least squares problem, we seek the vector x that minimizes the length (or norm) of the residual vector

r=Ax-b

. The four vectors

Ax

,

b

,

r

, and

r

min

are color coded and the plane is the range of the matrix

A

. The plane shown is the set of all possible vectors

Ax

.