1.1
Multiplication of Using Columns of
Ax
A
[MIT 18.065] Matrix Methods in Data Analysis, Signal Processing, and Machine Learning by Gilbert Strang
Junseo Lee
Ax and the Column Space of A
Ax
A
In this notebook, I will be exploring matrix-vector multiplication, focusing on how it relates to the column space of a matrix and the rank.
A fundamental way of thinking about the matrix-vector multiplicationis to see it as a linear combination of the columns of .
A fundamental way of thinking about the matrix-vector multiplication
Ax
A
Example
Example
Given a matrix , multiply it by a vector . There are two ways to perform this multiplication , each of which yields the same result.
A
2 | 3 |
2 | 4 |
3 | 7 |
x
x 1 |
x 2 |
Ax
In[]:=
A = {{2,3},{2,4},{3,7}}
Out[]=
{{2,3},{2,4},{3,7}}
In[]:=
x={{x1},{x2}}
Inner Products
The matrix and vector can be multiplied as inner products of the rows of with .
A
x
A
x(,)
x
1
x
2
In[]:=
(*Inner Product of A and x *)Ax = A.x
Out[]=
{{2x1+3x2},{2x1+4x2},{3x1+7x2}}
In[]:=
% // MatrixForm
Out[]//MatrixForm=
2x1+3x2 |
2x1+4x2 |
3x1+7x2 |
The product results in a column vector.
31
Outer Product
Alternatively, the matrix-vector multiplication can be performed by forming a linear combination of the columns and with scalars and .
a
1
a
2
x
1
x
2
In[]:=
(* First Column of Matrix A *)a1 = A[[All,{1}]];% // MatrixForm
Out[]//MatrixForm=
2 |
2 |
3 |
In[]:=
(* Second Column of Matrx A *)a2 = A[[All, {2}]];% // MatrixForm
Out[]//MatrixForm=
3 |
4 |
7 |
Forming a linear combination of the columns: +
x
1
a
1
x
2
a
2
In[]:=
Ax = x1 a1 + x2 a2
Out[]=
{{2x1+3x2},{2x1+4x2},{3x1+7x2}}
In[]:=
% // MatrixForm
Out[]//MatrixForm=
2x1+3x2 |
2x1+4x2 |
3x1+7x2 |
We can observe that both approaches results in the same matrix.
All Possible Linear Combinations of the Columns of A and the Column Space
All Possible Linear Combinations of the Columns of and the Column Space
A
When all real numbers and are allowed in the linear combination +, we get infinitely many vectors .
Thus, the combinations of the columns fills out the column space of.
x
1
x
2
a
1
x
1
a
2
x
2
Ax
Thus, the combinations of the columns fills out the column space of
A
In the above example, each is a vector in and all linear combinations produce a plane in . This fills out an infinite plane containing the two lines.
Ax
3
R
Ax+
a
1
x
2
a
2
x
2
3
R
ParametricPlot3D[Flatten[Ax],{x1, -100, 100},{x2,-100,100}, PlotStyle->Opacity[0.5]]
Out[]=
Generally speaking, the vector is in the column space of exactly when has a solution . Then, the solution specifies how we can express as a linear combination +
b(,,)
b
1
b
2
b
3
A
Axb
(,)
x
1
x
2
x
b
a
1
x
2
a
2
x
2
Example
Example
For some , this is impossible because they are not in the column space. For instance, is not in because is not solvable.
b
b
1 |
1 |
1 |
C(A)
Axb
In[]:=
b={1,1,1}
Out[]=
{1,1,1}
In[]:=
(* Check if Ax= b is solvable *)LinearSolve[A, b]
Out[]=
LinearSolve[{{1,3,8},{1,2,6},{0,1,2}},{1,1,1}]
The combinations of three independent columns in give every vector in , thus filling the whole , and the combinations of the columns of an invertible matrix fill its column space: all of .
3
R
b
3
R
3
R
nn
n
R
Independent Columns and the Rank of A
Independent Columns and the Rank of
A
Now that we have seen how the linear combinations of the columns of fills its column space, let’s explore how we can find a basis for the column space of and how we can factor into a product of times .
A
A
A
C
R
Finding the Basis for the Column Space of A
Finding the Basis for the Column Space of
A
To begin with, what is a basis? A basis for a subspace is a full set of independent vectors, and all vectors in the space are linear combinations of the basis vectors.
Then, how do we find a basis? We can create a matrix whose columns come directly from columns of , excluding any column that is a combination of previous columns, or simply linearly dependent columns.
The resulting matrix will have independent columns () and they will be a basis for the column space of .
Then, how do we find a basis? We can create a matrix
C
n
A
The resulting matrix
C
r
r≤n
A
We refer to the number of independent columns in as the rank of , and it is the dimension of its column space.
Although the basis for the column space is not unique, it always contains the same number of vectors.
A
A
Although the basis for the column space is not unique, it always contains the same number of vectors.
Example 1
Example 1
As an example, consider a matrix
A
1 | 3 | 8 |
1 | 2 | 6 |
0 | 1 | 2 |
In[]:=
A = {{1, 3, 8}, {1, 2, 6}, {0, 1, 2}};% // MatrixForm
Out[]//MatrixForm=
1 | 3 | 8 |
1 | 2 | 6 |
0 | 1 | 2 |
Example 2
Example 2
The Rank Theorem
The Rank Theorem
References
Strang, G. (2019). Linear Algebra and Learning from Data. Wellesley-Cambridge Press.