MA571 Numerical Linear Algebra

Lecture 2: Vectors and Matrices

Rafikul Alam
Department of Mathematics
IIT Guwahati

1 / 21
Outline

Topics:
Vectors in Rn and Cn
Matrix-vector multiplication
Matrix-matrix multiplication
Block matrices
Outer product of vectors

2 / 21
Vectors in Rn
We define Rn to be the set of all ordered n-tuples of real numbers. Thus an n-tuple in
Rn (also called an n-vector) is of the form
 
v1
 ..
row vector: v = [v1 , . . . , vn ] or column vector: v =  .


vn

3 / 21
Vectors in Rn
We define Rn to be the set of all ordered n-tuples of real numbers. Thus an n-tuple in
Rn (also called an n-vector) is of the form
 
v1
 ..
row vector: v = [v1 , . . . , vn ] or column vector: v =  .


vn
We always write a vector in Rn as a column vector. Thus
  

 v1 

n
R := 
 ..  : v , . . . , v ∈ R .
 .  1 n

vn
 

3 / 21
Vectors in Rn
We define Rn to be the set of all ordered n-tuples of real numbers. Thus an n-tuple in
Rn (also called an n-vector) is of the form
 
v1
 ..
row vector: v = [v1 , . . . , vn ] or column vector: v =  .


vn
We always write a vector in Rn as a column vector. Thus
  

 v1 

n
R := 
 ..  : v , . . . , v ∈ R .
 .  1 n

vn
 

   >
v1 v1
Transpose: [v1 , . . . , vn ]> =  ...  and  ...  = [v1 , . . . , vn ].
   

vn vn
3 / 21
Vectors in Cn
We define Cn to be the set of all ordered n-tuples of complex numbers. Thus an
n-tuple in Cn (also called an n-vector) is of the form
 
v1
row vector: v = [v1 , . . . , vn ] or column vector: v =  ... 
 

vn

4 / 21
Vectors in Cn
We define Cn to be the set of all ordered n-tuples of complex numbers. Thus an
n-tuple in Cn (also called an n-vector) is of the form
 
v1
row vector: v = [v1 , . . . , vn ] or column vector: v =  ... 
 

vn
We always write a vector in Cn as column vector. Thus
  
 v1
 

n  .. 
C :=  .  : v1 , . . . , vn ∈ C
 
vn
 

4 / 21
Vectors in Cn
We define Cn to be the set of all ordered n-tuples of complex numbers. Thus an
n-tuple in Cn (also called an n-vector) is of the form
 
v1
row vector: v = [v1 , . . . , vn ] or column vector: v =  ... 
 

vn
We always write a vector in Cn as column vector. Thus
  
 v1
 

n  .. 
C :=  .  : v1 , . . . , vn ∈ C
 
vn
 

Conjugate transpose:   Here z̄ ∗ complex conjugate of z ∈ C.

is the
v̄1 v1
[v1 , . . . , vn ]∗ =  ...  and  ..  = [v̄ , . . . , v̄ ].
  
.  1 n
v̄n vn
4 / 21
Algebraic properties of vectors in Rn and Cn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun

5 / 21
Algebraic properties of vectors in Rn and Cn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun

Then Fn is a vector space over F.

5 / 21
Algebraic properties of vectors in Rn and Cn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun

Then Fn is a vector space over F.

Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .

5 / 21
Algebraic properties of vectors in Rn and Cn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun

Then Fn is a vector space over F.

Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .
Example: A feature vector collects together n different quantities that pertain to a
single thing or object. The entries of a feature vector are called features or attributes.

5 / 21
Algebraic properties of vectors in Rn and Cn
Define addition and scalar multiplication on Fn (F = R or C) as follows:
         
u1 v1 u1 + v1 u1 αu1
 ..   ..   ..  .   . 
 . + . =  and α  ..  =  ..  for α ∈ F.

.
un vn un + vn un αun

Then Fn is a vector space over F.

Standard
 vectors: The vectors
> >  >
e1 := 1 0 · · · 0 , e2 := 0 1 0 · · · 0 , . . . , en := 0 · · · 0 1 are
called standard vectors or canonical vectors in Rn and Cn .
Example: A feature vector collects together n different quantities that pertain to a
single thing or object. The entries of a feature vector are called features or attributes.
For instance, a 5-vector x := [x1 , x2 , x3 , x4 , x5 ]> could give the age, height, weight,
blood pressure, and temperature of a patient admitted to a hospital.
5 / 21
Matrices
Definition: A matrix is an array of numbers. An m × n matrix A has m rows and n
columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn

6 / 21
Matrices
Definition: A matrix is an array of numbers. An m × n matrix A has m rows and n
columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn
 
a1j
 .. 
The j-th column of A : aj :=  .  for j = 1 : n.
amj
 
The i-th row of A: âi := ai1 ai2 · · · ain for i = 1 : m.

6 / 21
Matrices
Definition: A matrix is an array of numbers. An m × n matrix A has m rows and n
columns and is of the form
 
a11 a12 · · · a1n
 a21 a22 · · · a2n 
A= . ..  .
 
. ..
 . . ··· . 
am1 am2 · · · amn
 
a1j
 .. 
The j-th column of A : aj :=  .  for j = 1 : n.
amj
 
The i-th row of A: âi := ai1 ai2 · · · ain for i = 1 : m. Then
   
a11 · · · a1n − â1 −
A =  ... .. ..  =  a a · · · a  =  ..
.
 
. .  1 2 n  .
am1 · · · amn − âm −
6 / 21
Special matrices
An m × n matrix said to be a square matrix if m = n.

7 / 21
Special matrices
An m × n matrix said to be a square matrix if m = n. An m × n matrix D := [dij ] is
said to be a diagonal matrix if dij = 0 for all i 6= j.

7 / 21
Special matrices
An m × n matrix said to be a square matrix if m = n. An m × n matrix D := [dij ] is
said to be a diagonal matrix if dij = 0 for all i 6= j. An n × n diagonal matrix D with
diagonal entries d1 , . . . , dn is given by
 
d1
D = diag(d1 , . . . , dn ) = 
 .. .

.
dn

Identity matrix: An n × n diagonal matrix with all diagonal entries equal to 1 is called
the identity matrix and is denoted by In or I .

7 / 21
Special matrices
An m × n matrix said to be a square matrix if m = n. An m × n matrix D := [dij ] is
said to be a diagonal matrix if dij = 0 for all i 6= j. An n × n diagonal matrix D with
diagonal entries d1 , . . . , dn is given by
 
d1
D = diag(d1 , . . . , dn ) = 
 .. .

.
dn

Identity matrix: An n × n diagonal matrix with all diagonal entries equal to 1 is called
the identity matrix and is denoted by In or I .

Zero matrix: An m × n matrix with all entries 0 is called the zero matrix and is
denoted by Om×n or simply by O.

7 / 21
Special matrices
An m × n matrix said to be a square matrix if m = n. An m × n matrix D := [dij ] is
said to be a diagonal matrix if dij = 0 for all i 6= j. An n × n diagonal matrix D with
diagonal entries d1 , . . . , dn is given by
 
d1
D = diag(d1 , . . . , dn ) = 
 .. .

.
dn

Identity matrix: An n × n diagonal matrix with all diagonal entries equal to 1 is called
the identity matrix and is denoted by In or I .

Zero matrix: An m × n matrix with all entries 0 is called the zero matrix and is
denoted by Om×n or simply by O.
   
1 0 0 0 0 0 0
Example: I := 0 1 0 and O := 0 0 0 0 .
0 0 1 0 0 0 0
7 / 21
Transpose and Conjugate transpose
Transpose: The transpose of an m × n matrix A = [aij ]m×n is the n × m matrix
denoted by A> and is given by A> = [aji ]n×m .

8 / 21
Transpose and Conjugate transpose
Transpose: The transpose of an m × n matrix A = [aij ]m×n is the n × m matrix
denoted by A> and is given by A> = [aji ]n×m .
 >
1 2    >  
1 4 7 1 + i 2 1 + i 3
Example: 4 5 = and =
2 5 8 3 4 + 5i 2 4 + 5i
7 8

8 / 21
Transpose and Conjugate transpose
Transpose: The transpose of an m × n matrix A = [aij ]m×n is the n × m matrix
denoted by A> and is given by A> = [aji ]n×m .
 >
1 2    >  
1 4 7 1 + i 2 1 + i 3
Example: 4 5 = and =
2 5 8 3 4 + 5i 2 4 + 5i
7 8

Conjugate transpose: The conjugate transpose of an m × n complex matrix

A = [aij ]m×n is the n × m matrix denoted by A∗ and is given by

A∗ = [āji ]n×m = ([āij ]m×n )> = (Ā)> ,

where āij is the complex conjugate of aij .

8 / 21
Transpose and Conjugate transpose
Transpose: The transpose of an m × n matrix A = [aij ]m×n is the n × m matrix
denoted by A> and is given by A> = [aji ]n×m .
 >
1 2    >  
1 4 7 1 + i 2 1 + i 3
Example: 4 5 = and =
2 5 8 3 4 + 5i 2 4 + 5i
7 8

Conjugate transpose: The conjugate transpose of an m × n complex matrix

A = [aij ]m×n is the n × m matrix denoted by A∗ and is given by

A∗ = [āji ]n×m = ([āij ]m×n )> = (Ā)> ,

where āij is the complex conjugate of aij .

 
 ∗ −i 3
i 4 1+i
Example: = 4 4 − 5i 
3 4 + 5i 0
1−i 0
8 / 21
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

9 / 21
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

Definition: Let A be an n × n matrix. Then A is said to be

1 symmetric if A> = A
2 skew-symmetric if A> = −A

9 / 21
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

Definition: Let A be an n × n matrix. Then A is said to be

1 symmetric if A> = A
2 skew-symmetric if A> = −A
3 Hermitian if A∗ = A
4 skew-Hermitian if A∗ = −A.

9 / 21
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

Definition: Let A be an n × n matrix. Then A is said to be

1 symmetric if A> = A
2 skew-symmetric if A> = −A
3 Hermitian if A∗ = A
4 skew-Hermitian if A∗ = −A.

Remark: Let A := [aij ]n×n . If A> = −A then ajj = 0 for j = 1 : n.

9 / 21
Transpose and conjugate transpose

Exercise: Let A, B ∈ Fm×n and α ∈ F. Then show that

(a) (A + B)> = A> + B > (b) (αA)> = αA> and (αA)∗ = ᾱA∗ (c) (A> )> = A.

Definition: Let A be an n × n matrix. Then A is said to be

1 symmetric if A> = A
2 skew-symmetric if A> = −A
3 Hermitian if A∗ = A
4 skew-Hermitian if A∗ = −A.

Remark: Let A := [aij ]n×n . If A> = −A then ajj = 0 for j = 1 : n. On the other hand,
if A∗ = −A then Re(ajj ) = 0 for j = 1 : n.

9 / 21
Matrix-vector multiplication
Let A := a1 · · · an ∈ Fm×n and x := [x1 , . . . , xn ]> ∈ Fn . We define the
 

matrix-vector multiplication Ax to be the linear combination of columns of A.

10 / 21
Matrix-vector multiplication
Let A := a1 · · · an ∈ Fm×n and x := [x1 , . . . , xn ]> ∈ Fn . We define the
 

matrix-vector multiplication Ax to be the linear combination of columns of A.

Definition: Matrix-vector multiplication
 
x1
an  ...  = x1 a1 + · · · + xn an .
 
Ax = a1 · · ·


xn

10 / 21
Matrix-vector multiplication
Let A := a1 · · · an ∈ Fm×n and x := [x1 , . . . , xn ]> ∈ Fn . We define the
 

matrix-vector multiplication Ax to be the linear combination of columns of A.

Definition: Matrix-vector multiplication
 
x1
an  ...  = x1 a1 + · · · + xn an .
 
Ax = a1 · · ·


xn

Example:
          
1 0 0 x1 1 0 0 x1
 −1 1 0   x2  = x1  −1  + x2  1  + x3  0  =  −x1 + x2  .
0 −1 1 x3 0 −1 1 −x2 + x3

10 / 21
Matrix-vector multiplication
 
A row vector ai1 · · · ain is a 1 × n matrix. Therefore
 
x1
  . 
ai1 · · · ain  ..  = ai1 x1 + · · · + ain xn .
xn

11 / 21
Matrix-vector multiplication
 
A row vector ai1 · · · ain is a 1 × n matrix. Therefore
 
x1
  . 
ai1 · · · ain  ..  = ai1 x1 + · · · + ain xn .
xn

Example: Matrix-vector multiplication in two ways

        
1 0 0 x1 1 0 0
 1 1 0   x2  = x1  1  + x2  1  + x3  0 
0 1 1 x3 0 1 1
  
1 0 0 x
  
x1
   
=  x1 + x2  =  1 1 0 x 
x2 + x3  
0 1 1 x

11 / 21
Row

and column oriented
 
matrix-vector
 
multiplication
 
a11 ··· a1n x1 a11 a1n
 .. ..   ..  = x  ..  + · · · + x  .. 
 . ··· .  .  1 .  n . 
am1 ··· amn xn am1 amn

12 / 21
Row

and column oriented
 
matrix-vector
 
multiplication
 
a11 ···
a1n x1 a11 a1n
.. ..   .. ..  + · · · + x  .. 
 = x1 
  
 ···
. .  . .  n . 
a · · · amn xn a a
m1    m1   mn
a11 x1 + · · · + a1n xn a11 · · · a1n x
= .. ..
= .
   
.  . 
am1 x1 + · · · + amn xn am1 · · · amn x

12 / 21
Row

and column oriented
 
matrix-vector
 
multiplication
 
a11 ···
a1n x1 a11 a1n
.. ..   .. ..  + · · · + x  .. 
 = x1 
  
 ···
. .  . .  n . 
a · · · amn xn a a
m1    m1   mn
a11 x1 + · · · + a1n xn a11 · · · a1n x
= .. ..
= .
   
.  . 
am1 x1 + · · · + amn xn am1 · · · amn x
 
− â1 −
 
Writing A := a1 · · · an and A =  ..
 , we have
 
.
− âm −
   
a11 x1 + · · · + a1n xn â1 x
Ax = x1 a1 + · · · + xn an =  ..   .. 
 =  . .

.
am1 x1 + · · · + amn xn âm x

12 / 21
Matrix-matrix multiplication
Fact: Let A ∈ Fm×n . Let ei ∈ Fm and ej ∈ Fn be standard unit vectors. Then
 >
Aej is the j-th column a1j · · · amj of A
e>
 
i A is the i-th row ai1 · · · ain of A.

13 / 21
Matrix-matrix multiplication
Fact: Let A ∈ Fm×n . Let ei ∈ Fm and ej ∈ Fn be standard unit vectors. Then
 >
Aej is the j-th column a1j · · · amj of A
e>
 
i A is the i-th row ai1 · · · ain of A.

Let A ∈ Fm×n and B := ∈ Fn×p .

 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

13 / 21
Matrix-matrix multiplication
Fact: Let A ∈ Fm×n . Let ei ∈ Fm and ej ∈ Fn be standard unit vectors. Then
 >
Aej is the j-th column a1j · · · amj of A
e>
 
i A is the i-th row ai1 · · · ain of A.

Let A ∈ Fm×n and B := ∈ Fn×p .

 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

Reason: Define AB to be the m × p matrix such that (AB)x = A(Bx) for all x ∈ Fp .

13 / 21
Matrix-matrix multiplication
Fact: Let A ∈ Fm×n . Let ei ∈ Fm and ej ∈ Fn be standard unit vectors. Then
 >
Aej is the j-th column a1j · · · amj of A
e>
 
i A is the i-th row ai1 · · · ain of A.

Let A ∈ Fm×n and B := ∈ Fn×p .

 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

Reason: Define AB to be the m × p matrix such that (AB)x = A(Bx) for all x ∈ Fp .
Let C := AB be given by C = c1 · · · cp . Let ej ∈ Fp be the standard unit vector.
Then for j = 1 : p, we have Bej = bj and

13 / 21
Matrix-matrix multiplication
Fact: Let A ∈ Fm×n . Let ei ∈ Fm and ej ∈ Fn be standard unit vectors. Then
 >
Aej is the j-th column a1j · · · amj of A
e>
 
i A is the i-th row ai1 · · · ain of A.

Let A ∈ Fm×n and B := ∈ Fn×p .

 
b1 · · · bp
Definition: Define the matrix-matrix multiplication AB by
 
AB := Ab1 · · · Abp .

Reason: Define AB to be the m × p matrix such that (AB)x = A(Bx) for all x ∈ Fp .
Let C := AB be given by C = c1 · · · cp . Let ej ∈ Fp be the standard unit vector.
Then for j = 1 : p, we have Bej = bj and
 
cj = C ej = (AB)ej = A(Bej ) = Abj =⇒ C = Ab1 · · · Abp .

13 / 21
Matrix-matrix

multiplication

−â1 −
..
 ∈ Fm×n , B := b1 · · · ∈ Fn×p . Then
 
Let A =  bp
 
.
−âm −
   
â1 b1 · · · â1 bp â1 B
   . ..  =  ..  .
AB = Ab1 · · · Abp =  .. ··· .   . 
âm b1 · · · âm bp âm B

14 / 21
Matrix-matrix

multiplication

−â1 −
..
 ∈ Fm×n , B := b1 · · · ∈ Fn×p . Then
 
Let A =  bp
 
.
−âm −
   
â1 b1 · · · â1 bp â1 B
   . ..  =  ..  .
AB = Ab1 · · · Abp =  .. ··· .   . 
âm b1 · · · âm bp âm B

Thus if A := [aij ]m×n , B := [bij ]n×p and C := AB = [cij ]m×p then

 
b1j n
  .  X
.
cij = âi bj = ai1 · · · ain  .  = aik bkj .
bnj k=1

14 / 21
Matrix-matrix

multiplication

−â1 −
..
 ∈ Fm×n , B := b1 · · · ∈ Fn×p . Then
 
Let A =  bp
 
.
−âm −
   
â1 b1 · · · â1 bp â1 B
   . ..  =  ..  .
AB = Ab1 · · · Abp =  .. ··· .   . 
âm b1 · · · âm bp âm B

Thus if A := [aij ]m×n , B := [bij ]n×p and C := AB = [cij ]m×p then

 
b1j n
  .  X
.
cij = âi bj = ai1 · · · ain  .  = aik bkj .
bnj k=1

Remark: If A and B are n × n matrices then in general AB 6= BA.

14 / 21
Example
 
  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0

15 / 21
Example
 
  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0
   
  4     −1  
1 3 2   13 1 3 2   5
Ab1 = 1 = and Ab2 = 2 = .
0 −1 1 2 0 −1 1 −2
3 0

15 / 21
Example
 
  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0
   
  4     −1  
1 3 2   13 1 3 2   5
Ab1 = 1 = and Ab2 = 2 = .
0 −1 1 2 0 −1 1 −2
3 0
 
  13 5
Therefore AB = Ab1 Ab2 = .
2 −2

15 / 21
Example


  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0
   
  4     −1  
1 3 2   13 1 3 2   5
Ab1 = 1 = and Ab2 = 2 = .
0 −1 1 2 0 −1 1 −2
3 0
 
  13 5
Therefore AB = Ab1 Ab2 = . On the other hand
2 −2
   
  4 −1     4 −1  
a1 B = 1
b 3 2  1 2  = 13 5 a2 B = 0
and b −1 1  1 2 = 2 −2 .
3 0 3 0

15 / 21
Example


  4 −1
1 3 2
Let A = and B :=  1 2  . Then
0 −1 1
3 0
   
  4     −1  
1 3 2   13 1 3 2   5
Ab1 = 1 = and Ab2 = 2 = .
0 −1 1 2 0 −1 1 −2
3 0
 
  13 5
Therefore AB = Ab1 Ab2 = . On the other hand
2 −2
   
  4 −1     4 −1  
a1 B = 1
b 3 2  1 2  = 13 5 a2 B = 0
and b −1 1  1 2 = 2 −2 .
3 0 3 0
   
a1 B 13 5  
Therefore AB = = = Ab1 Ab2 .
b
a2 B
b 2 −2

15 / 21
Block matrices
Definition: An m × n block matrix (or a partitioned matrix) is a matrix of the form
 
A11 · · · A1n
A :=  ... .. 

··· . 
Am1 · · · Amn

where each Aij is a pi × qj matrix for i = 1 : m and j = 1 : n.

16 / 21
Block matrices
Definition: An m × n block matrix (or a partitioned matrix) is a matrix of the form
 
A11 · · · A1n
A :=  ... .. 

··· . 
Am1 · · · Amn

where each Aij is a pi × qj matrix for i = 1 : m and j = 1 : n.

 
A1j
Then Ai1 · · · Ain is the i-th block row of A and  ...  is the j-th block
   

Amj
column of A.

16 / 21
Block matrices
Definition: An m × n block matrix (or a partitioned matrix) is a matrix of the form
 
A11 · · · A1n
A :=  ... .. 

··· . 
Am1 · · · Amn

where each Aij is a pi × qj matrix for i = 1 : m and j = 1 : n.

 
A1j
   ..
Then Ai1 · · · Ain is the i-th block row of A and  .  is the j-th block


Amj
column of A.
 
1 2 2 0 1 4
 3 4 1 2 3 5 
Example: 
 5 7
 has 2 block rows and 3 block columns.
2 7 8 8 
3 4 1 9 2 2
16 / 21
Block matrix operations
Block matrix addition: Let A := [Aij ]m×n and B := [Bij ]m×n be block matrices such
that size of Aij = size of Bij for i = 1 : m and j = 1 : n. Then A + B := [Aij + Bij ]m×n .

17 / 21
Block matrix operations
Block matrix addition: Let A := [Aij ]m×n and B := [Bij ]m×n be block matrices such
that size of Aij = size of Bij for i = 1 : m and j = 1 : n. Then A + B := [Aij + Bij ]m×n .

Block matrix multiplication: LetP

A := [Aij ]m×n and B := [Bij ]n×p be block matrices. If
the matrix multiplication Cij := nk=1 Aik Bkj is well defined for i = 1 : m and j = 1 : p
then AB is an m × p block matrix given by AB = [Cij ]m×p .

17 / 21
Block matrix operations
Block matrix addition: Let A := [Aij ]m×n and B := [Bij ]m×n be block matrices such
that size of Aij = size of Bij for i = 1 : m and j = 1 : n. Then A + B := [Aij + Bij ]m×n .

Block matrix multiplication: LetP

A := [Aij ]m×n and B := [Bij ]n×p be block matrices. If
the matrix multiplication Cij := nk=1 Aik Bkj is well defined for i = 1 : m and j = 1 : p
then AB is an m × p block matrix given by AB = [Cij ]m×p .
Conformal partition: If an operation on block matrices A and B are well defined then
A and B are said to be partitioned conformably.

17 / 21
Block matrix operations
Block matrix addition: Let A := [Aij ]m×n and B := [Bij ]m×n be block matrices such
that size of Aij = size of Bij for i = 1 : m and j = 1 : n. Then A + B := [Aij + Bij ]m×n .

Block matrix multiplication: LetP

A := [Aij ]m×n and B := [Bij ]n×p be block matrices. If
the matrix multiplication Cij := nk=1 Aik Bkj is well defined for i = 1 : m and j = 1 : p
then AB is an m × p block matrix given by AB = [Cij ]m×p .
Conformal partition: If an operation on block matrices A and B are well defined then
A and B are said to be partitioned conformably.
  
A11 A12 B11 B12 B13
Example: =
A21 A22 B21 B22 B23
 
A11 B11 + A12 B21 A11 B12 + A12 B22 A11 B13 + A12 B23
.
A21 B11 + A22 B21 A21 B12 + A22 B22 A21 B13 + A22 B23

17 / 21
Block matrix multiplication

Example:
 
  1 1 1 1  
1 1 1 1  1 8 6 4 5
 2 2 2 1 1 
 =  10 9
1 1 
 3 6 7 
1 1 1 
3 3 2 2 18 15 10 12
3 2 1 2
 
  1 1 1 1  
1 1 1 1  1 8 6 4 5
 2 2 2 1 1 
1 1  3
 =  10 9 6 7 
1 1 1 
3 3 2 2 18 15 10 12
3 2 1 2

18 / 21
Outer product

Given two vectors x and y in Rn , the standard inner product of x and y is given by

hx, yi = x1 y1 + · · · + xn yn = y> x.

19 / 21
Outer product

Given two vectors x and y in Rn , the standard inner product of x and y is given by

hx, yi = x1 y1 + · · · + xn yn = y> x.

Outer product: The matrix product xy> is an n × n matrix and is given by

   
x1 x1 y1 x1 y2 · · · x1 yn
 x2 y1 x2 y2 · · · x2 yn 
 x2     
>
xy =  .  y1 y2 · · · yn =  . ..  .
 
.. ..
 ..   .. . . . 
xn xn y1 xn y2 · · · xn yn

The product xy> is called the outer product of x ∈ Rn and y ∈ Rn .

19 / 21
Outer product
 >  >
Example: If x := 4 1 3 and y := 3 5 2 then
   
4  12 20 8
xy> =  1  3 5 2 =  3 5 2  .


3 9 15 6

20 / 21
Outer product
 >  >
Example: If x := 4 1 3 and y := 3 5 2 then
   
4  12 20 8
xy> =  1  3 5 2 =  3 5 2  .


3 9 15 6

Outer product of matrices: 

Let X := x1 x2 · · · xn ∈ Rm×n and Y := y1 y2 · · · yn ∈ Rp×n . Then
  

XY > ∈ Rm×p can be written as sum of outer products of vectors

20 / 21
Outer product
 >  >
Example: If x := 4 1 3 and y := 3 5 2 then
   
4  12 20 8
xy> =  1  3 5 2 =  3 5 2  .


3 9 15 6

Outer product of matrices: 

Let X := x1 x2 · · · xn ∈ Rm×n and Y := y1 y2 · · · yn ∈ Rp×n . Then
  

XY > ∈ Rm×p can be written as sum of outer products of vectors

 > 
y1
 y> 
 2 
XY > = x1 x2 · · · xn  .  = x1 y1> + x2 y2> + · · · + xn yn> .
 
 .. 
yn>

20 / 21
Floating-Point Operation (FLOP) count
>
∈ Rn and

Vector-vector operations: Let α ∈ R. Let x := x1 · · · xn
>
y := y1 · · · yn ∈ Rn .


21 / 21
Floating-Point Operation (FLOP) count
>
Vector-vector operations: Let α ∈ R. Let x := x1 · · · xn ∈ Rn and

>
y := y1 · · · yn ∈ Rn . We ignore the lower order terms for flop count.


z ← x + y and d ← α · x require n flops

21 / 21
Floating-Point Operation (FLOP) count
>
Vector-vector operations: Let α ∈ R. Let x := x1 · · · xn ∈ Rn and

>
y := y1 · · · yn ∈ Rn . We ignore the lower order terms for flop count.


z ← x + y and d ← α · x require n flops

z ← α · x + y and s ← hx, yi = nj=1 xj yj require 2n flops
P

21 / 21
Floating-Point Operation (FLOP) count
>
Vector-vector operations: Let α ∈ R. Let x := x1 · · · xn ∈ Rn and

>
y := y1 · · · yn ∈ Rn . We ignore the lower order terms for flop count.


z ← x + y and d ← α · x require n flops

z ← α · x + y and s ← hx, yi = nj=1 xj yj require 2n flops
P

Matrix-vector operations: Let A := a1 · · · an ∈ Rn×n and β ∈ R.

 

z ← Ax = x1 a1 + · · · + xn an and d ← α · Ax + β · y require 2n2 flops

21 / 21
Floating-Point Operation (FLOP) count
>
Vector-vector operations: Let α ∈ R. Let x := x1 · · · xn ∈ Rn and

>
y := y1 · · · yn ∈ Rn . We ignore the lower order terms for flop count.


z ← x + y and d ← α · x require n flops

z ← α · x + y and s ← hx, yi = nj=1 xj yj require 2n flops
P

Matrix-vector operations: Let A := a1 · · · an ∈ Rn×n and β ∈ R.

 

z ← Ax = x1 a1 + · · · + xn an and d ← α · Ax + β · y require 2n2 flops

z ← A> x = a> > > and d ← α · A> x + β · y require 2n2 flops
 
1 x · · · an x

21 / 21
Floating-Point Operation (FLOP) count
>
Vector-vector operations: Let α ∈ R. Let x := x1 · · · xn ∈ Rn and

>
y := y1 · · · yn ∈ Rn . We ignore the lower order terms for flop count.


z ← x + y and d ← α · x require n flops

z ← α · x + y and s ← hx, yi = nj=1 xj yj require 2n flops
P

Matrix-vector operations: Let A := a1 · · · an ∈ Rn×n and β ∈ R.

 

z ← Ax = x1 a1 + · · · + xn an and d ← α · Ax + β · y require 2n2 flops

z ← A> x = a> > > and d ← α · A> x + β · y require 2n2 flops
 
1 x · · · an x

Matrix-matrix operations: Let B := b1 · · · bn ∈ Rn×n and C ∈ Rn×n .

 

D ← AB = Ab1 · · · Abn and D ← α · AB + β · C require 2n3 flops

 

21 / 21
Floating-Point Operation (FLOP) count
>
Vector-vector operations: Let α ∈ R. Let x := x1 · · · xn ∈ Rn and

>
y := y1 · · · yn ∈ Rn . We ignore the lower order terms for flop count.


z ← x + y and d ← α · x require n flops

z ← α · x + y and s ← hx, yi = nj=1 xj yj require 2n flops
P

Matrix-vector operations: Let A := a1 · · · an ∈ Rn×n and β ∈ R.

 

z ← Ax = x1 a1 + · · · + xn an and d ← α · Ax + β · y require 2n2 flops

z ← A> x = a> > > and d ← α · A> x + β · y require 2n2 flops
 
1 x · · · an x

Matrix-matrix operations: Let B := b1 · · · bn ∈ Rn×n and C ∈ Rn×n .

 

D ← AB = Ab1 · · · Abn and D ← α · AB + β · C require 2n3 flops

 

D ← A> B or D ← AB > and D ← α · A> B + β · C require 2n3 flops

21 / 21