Overview

Matrix Multiplication Compatibility

Definition
Examples
Column Perspective
Definition
Examples
Row Perspective
Definition
Examples
Properties
General Properties
The Gramian
Inner Products
Matrix Multiplication Compatibility
Definition

Observation
Suppose we wish to “transform” a vector twice.

B A
Rn Rm Rω
v Bv ABv

We could bypass this intermediary step by “multiplying” A and B together.

m→n ω→m
B A
Rn Rm Rω

AB
ω→n
In short, we have (ω → m) · (m → n) ↭ ω → n.
Matrix Multiplication Compatibility
Examples

Example
Suppse we multiply a 3 → 4 matrix and a 4 → 2 matrix.

A  B 
  ↑ ↑ B A
↑ ↑ ↑ ↑   R2 R4 R3
↑ ↑ ↑ ↑↑ ↑
↑ ↑
↑ ↑ ↑ ↑ AB
↑ ↑
Matrix Multiplication Compatibility
Examples

Example
Suppse we multiply a 3 → 4 matrix and a 4 → 2 matrix.

A  B 
  ↑ ↑  AB  B A
↑ ↑ ↑ ↑   ↑ ↑ R2 R4 R3
↑ ↑ ↑ ↑↑ ↑ = ↑ ↑
↑ ↑
↑ ↑ ↑ ↑ ↑ ↑ AB
↑ ↑

This can also be thought of as (3 → 4) · (4 → 2) ↭ 3 → 2.

Matrix Multiplication Compatibility
Examples

Example
Suppose we multiply a 4 → 2 matrix and a 3 → 4 matrix.

 B  A
↑ ↑  
↑ ↑ ↑ ↑ ↑
 ↑
  ↑ ↑ ↑ ↑
A B
↑ R4 R3→=2 R4
↑
↑ ↑ ↑ ↑
↑ ↑
Matrix Multiplication Compatibility
Examples

Example
Suppose we multiply a 4 → 2 matrix and a 3 → 4 matrix.

 B  A
↑ ↑  
↑ ↑ ↑ ↑ ↑
 ↑
  ↑ ↑ ↑ ↑
A B
↑ R4 R3→=2 R4
↑
↑ ↑ ↑ ↑
↑ ↑

The product BA makes no sense, even though AB does make sense!

Column Perspective
Definition
Definition (Column Perspective)
Suppose A and B fit into a diagram of the form
m→n ω→m
B A
Rn Rm Rω

AB
ω→n
The matrix product AB is given by
 
 
 
AB = 
Ab 1 Ab 2 Ab n 

 

where b 1 , b 2 , . . . , b n are the columns of B.

Column Perspective
Examples

Example
Consider the following matrix product.

 A 
1 2 ' B (
 ↓1 1  1 0 1
 
 ↓ 1 ↓3  1 ↓5 ↓1
0 4

B A
R3 R2 R4

AB
Column Perspective
Examples

Example
Consider the following matrix product.

 A   AB 
1 2 ' B ( 3
 ↓1 1  1 0 1  0 
   
 ↓ 1 ↓3  1 ↓5 ↓1 =  ↓ 4 
0 4 4

Here, A and B fit into the diagram

B A
R3 R2 R4

AB

We multiply A by each column of B to build the columns of AB.

Column Perspective
Examples

Example
Consider the following matrix product.

 A   AB 
1 2 ' B ( 3 ↓ 10
 ↓1 1  1 0 1  0 ↓5 
   
 ↓ 1 ↓3  1 ↓5 ↓1 =  ↓ 4 15 
0 4 4 ↓ 20

Here, A and B fit into the diagram

B A
R3 R2 R4

AB

We multiply A by each column of B to build the columns of AB.

Column Perspective
Examples

Example
Consider the following matrix product.

 A   AB 
1 2 ' B ( 3 ↓ 10 ↓1
 ↓1 1  1 0 1  0 ↓5 ↓2 
   
 ↓ 1 ↓3  1 ↓5 ↓1 =  ↓ 4 15 2 
0 4 4 ↓ 20 ↓4

Here, A and B fit into the diagram

B A
R3 R2 R4

AB

We multiply A by each column of B to build the columns of AB.

Column Perspective
Examples
Example
Consider the following matrix-vector products.

 A     A   
↓ 2 ↓1 1  b1  ↓1 ↓ 2 ↓1 1 b2 ↓1
 1 0
 2 0 0  
 ↓ 1 =  2
 
 2 0 0  
2 =  0

 ↓1 1 ↓2   ↓ 2  ↓1 1 ↓2   0
0 1
1 1 0 0 1 1 0 2

These calculations show that

 A   
B
↓ 2 ↓1 1  
 1 0
 2 0 0  
 ↓ 1 2  = 


 ↓1 1 ↓2   
0 1
1 1 0
Column Perspective
Examples
Example
Consider the following matrix-vector products.

 A     A   
↓ 2 ↓1 1  b1  ↓1 ↓ 2 ↓1 1 b2 ↓1
 1 0
 2 0 0  
 ↓ 1 =  2
 
 2 0 0  
2 =  0

 ↓1 1 ↓2   ↓ 2  ↓1 1 ↓2   0
0 1
1 1 0 0 1 1 0 2

These calculations show that

 A   
B
↓ 2 ↓1 1   ↓1
  1 0  
 2 0 0  ↓ 1 2  =  2 
 ↓1 1 ↓2   ↓2 
0 1
1 1 0 0
Column Perspective
Examples
Example
Consider the following matrix-vector products.

 A     A   
↓ 2 ↓1 1  b1  ↓1 ↓ 2 ↓1 1 b2 ↓1
 1 0
 2 0 0  
 ↓ 1 =  2
 
 2 0 0  
2 =  0

 ↓1 1 ↓2   ↓ 2  ↓1 1 ↓2   0
0 1
1 1 0 0 1 1 0 2

These calculations show that

 A   
B
↓ 2 ↓1 1   ↓1 ↓1
  1 0 
 2 0 0  ↓ 1 2  =  2 0 

 ↓1 1 ↓2   ↓2 0 
0 1
1 1 0 0 2
Row Perspective
Definition

Definition (Row Perspective)

Suppose A and B fit into a diagram of the form
m→n ω→m
B A
Rn Rm Rω

AB
ω→n
The (i, j) entry of the matrix product AB is given by the inner product

↔ith row of A, jth column of B↗

This allows us to fill AB entry-by-entry with inner products.

Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1  
 2 ↓1 ↓2
 3 0 
 0
 
 0  1 =



1 2
2 0
0 0 1
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1  
 2 ↓1 ↓2
 3 0 
 0
 
 0  1 =



1 2
2 0
0 0 1
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1   1
 2 ↓1 ↓2
 3 0 
 0
 
 0  1 =



1 2
2 0
0 0 1
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1   1 ↓2
 2 ↓1 ↓2
 3 0 
 0
 
 0  1 =



1 2
2 0
0 0 1
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1   1 ↓2
 2 ↓1 ↓2
 3 0 
 0
 ↓2 
 0 1 = 
1 2   
2 0
0 0 1
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1   1 ↓2
 2 ↓1 ↓2
 3 0 
 0
 ↓2 ↓1 
 0 1 = 
1 2   
2 0
0 0 1
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1   1 ↓2
 2 ↓1 ↓2
 3 0 
 0
 ↓2 ↓1 
 0 1 = 
1 2   4 
2 0
0 0 1
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1   1 ↓2
 2 ↓1 ↓2
 3 0 
 0
 ↓2 ↓1 
 0 1 = 
1 2   4 1 
2 0
0 0 1
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1   1 ↓2
 2 ↓1 ↓2
 3 0 
 0
 ↓2 ↓1 
 0 1 = 
1 2   4 1 
2 0
0 0 1 2
Row Perspective
Examples

Example
Consider the following matrix product.

 A   
B
1 0 1   1 ↓2
 2 ↓1 ↓2
 3 0 
 0
 ↓2 ↓1 
 0 1 = 
1 2   4 1 
2 0
0 0 1 2 0
Row Perspective
Examples

Observation
The row perspective describes every entry of AB.

 A   AB 
r ↫1 ↔r 1 , b 1 ↗ ↔r 1 , b 2 ↗ ↔r 1 , b n ↗
 r ↫ ) B 
*  ↔r 2 , b 1 ↗ ↔r 2 , b 2 ↗ ↔r 2 , b n ↗ 
 2  
  b1 b2 bn =  
   
r ↫ω ↔r ω , b 1 ↗ ↔r ω , b 2 ↗ ↔r ω , b n ↗

The (i, j) entry is ↔ith row of A, jth column of B↗.

Properties
General Properties

Theorem
Matrix arithmetic obeys the following laws.
additive associativity A + (B + C ) = (A + B) + C
multiplicative associativity (AB)C = A(BC )
additive commutativity A + B = B + A
left distributive A(B + C ) = AB + AC
right distributive (A + B)C = AC + BC
identity rule AIn = Im A = A
trace rule trace(AB) = trace(BA)
transposition rule (AB)↫ = B ↫ A↫
Consequently, we say that transposition is an order-reversing involution.
Properties
General Properties

Example
Consider the products given by

' A (' B ( ' ( ' B (' A ( ' (

1 ↓2 2 ↓2 0 0 2 ↓2 1 ↓2 6 ↓12
= =
↓2 4 1 ↓1 0 0 1 ↓1 ↓2 4 3 ↓6

Here, we find that AB ↘= BA.

Properties
General Properties

Warning
Matrix multiplication is generally noncommutative, meaning AB ↘= BA.

ABABABA ↘= A4 B 3

We must be careful when simplifying expressions of matrix products!

Properties
The Gramian

Definition
The Gramian of a matrix A is G = A↫ A.

 A↭   G 
A
4 ↓6 ' ( 52 14 4
 2 ↓1 4 2 ↓2
=  14 5 ↓2 
↓ 6 ↓1 ↓2
↓ 2 ↓2 4 ↓2 8

The Gramian always satisfies the equation

(A↫ A)↫ = A↫ (A↫ )↫ = A↫ A

The Gramian of an m → n matrix is an n → n symmetric matrix.

Properties
The Gramian

Observation
The row perspective explicitly describes each entry of A↫ A.

 A↭   A↭ A 
a ↫1 ↔a 1 , a 1 ↗ ↔a 1 , a 2 ↗ ↔a 1 , a n ↗

 a ↫2 )

A 
*  ↔a 2 , a 1 ↗ ↔a 2 , a 2 ↗ ↔a 2 , a n ↗ 

  a1 a2 an =  
   
an ↔a n , a 1 ↗ ↔a n , a 2 ↗ ↔a n , a n ↗

The (i, j) entry of A↫ A is ↔ith column, jth column↗.

Properties
Inner Products

Example
Consider two vectors v and w given by
) *↫ ) *↫
v = v1 v 2 vn w = w 1 w2 wn

If we view these vectors as n → 1 matrices, then

 
w1
) 
*  w2 

v ↫ w = v 1 v2 vn   = ↔v , w ↗
 
wn

Inner products can be written as matrix multiplication ↔v , w ↗ = v ↫ w .

Properties
Inner Products

Theorem
↔Av , w ↗ = ↔v , A↫ w ↗
Properties
Inner Products

Theorem
↔Av , w ↗ = ↔v , A↫ w ↗

Proof.
↔Av , w ↗ = (Av )↫ w = (v ↫ A↫ )w = v ↫ (A↫ w ) = ↔v , A↫ w ↗