1.2
Matrix-Matrix Multiplication
AB
: An Outer Product View
[MIT 18.065] Matrix Methods in Data Analysis, Signal Processing, and Machine Learning by Gilbert Strang
Junseo Lee

Matrix-Matrix Multiplication
AB

In this notebook, I will be exploring how we multiply and factor matrices. In particular, I will demonstrate how a matrix can be multiplied by multiplying the columns by rows, and how matrix multiplication can be expressed as a sum of rank one matrices.

Inner Products

​
Inner products (rows times columns) produce each of the numbers in
ABC
As an example, let us consider the two 3 by 3 matrices
A
and
B
.
A = Table[Subscript[a,i,j],{i,1,3}, {j,1,3}];​​%//MatrixForm
Out[]//MatrixForm=
a
1,1
a
1,2
a
1,3
a
2,1
a
2,2
a
2,3
a
3,1
a
3,2
a
3,3
In[]:=
B = Table[Subscript[b,i,j],{i,1,3},{j,1,3}];​​%//MatrixForm
Out[]//MatrixForm=
b
1,1
b
1,2
b
1,3
b
2,1
b
2,2
b
2,3
b
3,1
b
3,2
b
3,3
If we multiply the two matrices
A
and
B
to produce another matrix
C
, what would be the
(2,3)
element in the resultant matrix
C
?
In the inner product approach, we are taking the dot product of each row and each column to produce each entry in resultant matrix. Mathematically, we can denote this as the following:
Therefore,
(*Take the dot product of the second row of A times the thrid column of B to find the (2,3) element in the resultant matrix C*)​​c23 = Dot[A[[2, All]], B[[All, 3]]]
Out[]=
a
2,1
b
1,3
+
a
2,2
b
2,3
+
a
2,3
b
3,3
Thus, to get the matrix
, we simply take the dot product of rows times columns.
(* Produce C = AB by taking the inner prducts (rows times columns) *)​​CMat = Table[Dot[A[[i, All]], B[[All, j]]], {i, 1, 3}, {j, 1, 3}];​​%//MatrixForm
Out[]//MatrixForm=
a
1,1
b
1,1
+
a
1,2
b
2,1
+
a
1,3
b
3,1
a
1,1
b
1,2
+
a
1,2
b
2,2
+
a
1,3
b
3,2
a
1,1
b
1,3
+
a
1,2
b
2,3
+
a
1,3
b
3,3
a
2,1
b
1,1
+
a
2,2
b
2,1
+
a
2,3
b
3,1
a
2,1
b
1,2
+
a
2,2
b
2,2
+
a
2,3
b
3,2
a
2,1
b
1,3
+
a
2,2
b
2,3
+
a
2,3
b
3,3
a
3,1
b
1,1
+
a
3,2
b
2,1
+
a
3,3
b
3,1
a
3,1
b
1,2
+
a
3,2
b
2,2
+
a
3,3
b
3,2
a
3,1
b
1,3
+
a
3,2
b
2,3
+
a
3,3
b
3,3
​

Outer Product

The other way to multiply
is columns of
times rows of
.
One column vector
times one row
produces a “rank one matrix”.
In[]:=
(* Create a column vector u*)​​u={{2,2,1}}//Transpose;​​% // MatrixForm
Out[]//MatrixForm=
2
2
1
In[]:=
(* Create a row vector v *)​​vT={{3,4,6}};​​%//MatrixForm
Out[]//MatrixForm=
(
3
4
6
)
In[]:=
(* Compute the outer product of u and v*)​​uvT=Dot[u,vT];​​% //MatrixForm
Out[]//MatrixForm=
6
8
12
6
8
12
3
4
6
Now, we can confirm that this matrix is “rank one”.
In[]:=
MatrixRank[uvT]
Out[]=
1
Since the matrix is “rank one”, all columns of
are multiple of
and all rows are multiples of
.
In other words, the dimension of the column space (i.e., the number of independent columns) is the rank of the matrix--a key number. ​Generally speaking, all nonzero matrices
have rank one, and they are the perfect building blocks for every matrix.
​

= Sum of Rank One Matrices

​
An important idea is that the product
is the sum of columns
times rows
​
As an example, let us consider two 3 by 3 matrices
and
.
The matrix
has 3 columns.
In[]:=
A = Table[{Subscript[a,i]},{i,1,3}]//Transpose;​​% // MatrixForm
Out[]//MatrixForm=
(
a
1
a
2
a
3
)
The matrix
has 3 rows.
In[]:=
B=Table[{Subscript[b,j]}, {j,1,3}];​​%//MatrixForm
Out[]//MatrixForm=
b
1
b
2
b
3
Multiply
and
.
AB = Dot[A,B];​​%//MatrixForm
Out[]//MatrixForm=
(
a
1
b
1
+
a
2
b
2
+
a
3
b
3
)
Now manually, sum up columns
times rows
.
In[]:=
(* Compute the outerproduct of each column of A and each row of B, and sum them up to produce C *)​​ABSum =Sum[(Dot[{#1[[All,k]]},{#2[[k,All]]}])&[A,B],{k,1,3}];​​%//MatrixForm
Out[]//MatrixForm=
(
a
1
b
1
+
a
2
b
2
+
a
3
b
3
)
Confirm whether they are equal
In[]:=
AB == ABSum
Out[]=
True

Example

Let’s now consider a more concrete example of multiplying the matrices
and
.
​
​
We can observe that both methods result in the same matrix!

Why does it matter?

​
​
In my next post, I will be exploring how to idea can be applied to an orthogonal matrix.