1.2
Matrix-Matrix Multiplication : An Outer Product View
AB
[MIT 18.065] Matrix Methods in Data Analysis, Signal Processing, and Machine Learning by Gilbert Strang
Junseo Lee
Matrix-Matrix Multiplication AB
Matrix-Matrix Multiplication
AB
In this notebook, I will be exploring how we multiply and factor matrices. In particular, I will demonstrate how a matrix can be multiplied by multiplying the columns by rows, and how matrix multiplication can be expressed as a sum of rank one matrices.
Inner Products
Inner Products
Inner products (rows times columns) produce each of the numbers in
ABC
As an example, let us consider the two 3 by 3 matrices and .
A
B
A = Table[Subscript[a,i,j],{i,1,3}, {j,1,3}];%//MatrixForm
Out[]//MatrixForm=
a 1,1 | a 1,2 | a 1,3 |
a 2,1 | a 2,2 | a 2,3 |
a 3,1 | a 3,2 | a 3,3 |
In[]:=
B = Table[Subscript[b,i,j],{i,1,3},{j,1,3}];%//MatrixForm
Out[]//MatrixForm=
b 1,1 | b 1,2 | b 1,3 |
b 2,1 | b 2,2 | b 2,3 |
b 3,1 | b 3,2 | b 3,3 |
If we multiply the two matrices and to produce another matrix , what would be the element in the resultant matrix ?
In the inner product approach, we are taking the dot product of each row and each column to produce each entry in resultant matrix. Mathematically, we can denote this as the following:
A
B
C
(2,3)
C
In the inner product approach, we are taking the dot product of each row and each column to produce each entry in resultant matrix. Mathematically, we can denote this as the following:
Therefore,
(*Take the dot product of the second row of A times the thrid column of B to find the (2,3) element in the resultant matrix C*)c23 = Dot[A[[2, All]], B[[All, 3]]]
Out[]=
a
2,1
b
1,3
a
2,2
b
2,3
a
2,3
b
3,3
Thus, to get the matrix , we simply take the dot product of rows times columns.
(* Produce C = AB by taking the inner prducts (rows times columns) *)CMat = Table[Dot[A[[i, All]], B[[All, j]]], {i, 1, 3}, {j, 1, 3}];%//MatrixForm
Out[]//MatrixForm=
a 1,1 b 1,1 a 1,2 b 2,1 a 1,3 b 3,1 | a 1,1 b 1,2 a 1,2 b 2,2 a 1,3 b 3,2 | a 1,1 b 1,3 a 1,2 b 2,3 a 1,3 b 3,3 |
a 2,1 b 1,1 a 2,2 b 2,1 a 2,3 b 3,1 | a 2,1 b 1,2 a 2,2 b 2,2 a 2,3 b 3,2 | a 2,1 b 1,3 a 2,2 b 2,3 a 2,3 b 3,3 |
a 3,1 b 1,1 a 3,2 b 2,1 a 3,3 b 3,1 | a 3,1 b 1,2 a 3,2 b 2,2 a 3,3 b 3,2 | a 3,1 b 1,3 a 3,2 b 2,3 a 3,3 b 3,3 |
Outer Product
Outer Product
The other way to multiply is columns of times rows of .
One column vector times one row produces a “rank one matrix”.
In[]:=
(* Create a column vector u*)u={{2,2,1}}//Transpose;% // MatrixForm
Out[]//MatrixForm=
2 |
2 |
1 |
In[]:=
(* Create a row vector v *)vT={{3,4,6}};%//MatrixForm
Out[]//MatrixForm=
(
)
3 | 4 | 6 |
In[]:=
(* Compute the outer product of u and v*)uvT=Dot[u,vT];% //MatrixForm
Out[]//MatrixForm=
6 | 8 | 12 |
6 | 8 | 12 |
3 | 4 | 6 |
Now, we can confirm that this matrix is “rank one”.
In[]:=
MatrixRank[uvT]
Out[]=
1
Since the matrix is “rank one”, all columns of are multiple of and all rows are multiples of .
In other words, the dimension of the column space (i.e., the number of independent columns) is the rank of the matrix--a key number. Generally speaking, all nonzero matrices have rank one, and they are the perfect building blocks for every matrix.
= Sum of Rank One Matrices
An important idea is that the product is the sum of columns times rows
As an example, let us consider two 3 by 3 matrices and .
The matrix has 3 columns.
In[]:=
A = Table[{Subscript[a,i]},{i,1,3}]//Transpose;% // MatrixForm
Out[]//MatrixForm=
(
)
a 1 | a 2 | a 3 |
The matrix has 3 rows.
In[]:=
B=Table[{Subscript[b,j]}, {j,1,3}];%//MatrixForm
Out[]//MatrixForm=
b 1 |
b 2 |
b 3 |
Multiply and .
AB = Dot[A,B];%//MatrixForm
Out[]//MatrixForm=
(
)
a 1 b 1 a 2 b 2 a 3 b 3 |
Now manually, sum up columns times rows .
In[]:=
(* Compute the outerproduct of each column of A and each row of B, and sum them up to produce C *)ABSum =Sum[(Dot[{#1[[All,k]]},{#2[[k,All]]}])&[A,B],{k,1,3}];%//MatrixForm
Out[]//MatrixForm=
(
)
a 1 b 1 a 2 b 2 a 3 b 3 |
Confirm whether they are equal
In[]:=
AB == ABSum
Out[]=
True
Example
Example
Let’s now consider a more concrete example of multiplying the matrices and .
We can observe that both methods result in the same matrix!
Why does it matter?
Why does it matter?
In my next post, I will be exploring how to idea can be applied to an orthogonal matrix.