WI1403-LR Linear Algebra

Delft University of Technology

Year 2013 – 2014

Michele Facchinelli

Version 1.0 Last modified on February 1, 2017

Preface
This summary was written for the course WI1403-LR Linear Algebra, taught at the Delft University
of Technology. All the material treated is taken from [D. Lay. Linear Algebra and Its Applications.
Pearson, 4th edition, 2014.]
Throughout the summary, references to chapters and sections can be found. These are labelled
with the aid of the symbol § and can be found in the aforementioned book, where exercises and
more explanations are given.

In case of any comments about the content of the summary, please do not hesitate to contact
me at [Link]@[Link].

“Matrices are Roman Catholic —

Rows come before Columns!”

– Linear Algebra professor

i
Changelog
This is version 1.0. Below are listed the changes applied to each version.

Version Date Changes

1.0 February 1, 2017 First version

ii
Contents

1 Linear Equations in Linear Algebra [§1] 1

1.1 Equivalence of Notations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.2 Homogeneous Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.3 Nonhomogeneous Linear Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.4 Linear Independence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1.5 Introduction to Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
1.6 The Matrix of a Linear Transformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2 Matrix Algebra [§2] 4

2.1 Matrix Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.2 The Inverse of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2.3 Characterisation of Invertible Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.4 Subspaces of Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.5 Dimension and Rank . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Determinants [§3] 9
3.1 Introduction to Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Proprieties of Determinants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.3 Cramer’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

4 Eigenvalues and Eigenvectors [§5] 12

4.1 Eigenvectors and Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.2 The Characteristic Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.3 Similarity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
4.4 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.5 Eigenvectors and Linear Transformations . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.6 Complex Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.7 Application to Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.8 Trajectories of Eigenfunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5 Orthogonality and Least Squares [§6] 18

5.1 Inner Product and Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.2 Orthogonal Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
5.3 Orthogonal Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
5.4 The Gram-Schmidt Process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
5.5 Least-square Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

6 Symmetric Matrices [§7] 22

6.1 Diagonalization of Symmetric Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 22
6.2 Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

A Invertible Matrix Theorem I

iii
1 | Linear Equations in Linear Algebra [§1]
1.1 | Equivalence of Notations
Theorem 1.1. Notations.
The matrix equation
Ax = b
the vector equation
x1 a1 + · · · + xn an = b
and the linear system  
a1 ... an |b
all share the same solution set.
1.2 | Homogeneous Linear Systems
A homogeneous linear system is one of the type Ax = 0 , where A is a m × n matrix and 0 is the
zero vector in Rm .
The homogeneous equation
Ax = 0
has a non trivial solution (i.e. not x = 0) if and only if the equation has at least one free variable.

1.3 | Nonhomogeneous Linear Systems

A nonhomogeneous linear system, as seen before, is of the form
Ax = b (1.1)
To describe the solution set of such a system, consider the solution of the same system, but in the
case of b = 0. This specific system will give a solution of the type
x = tv (1.2)
where t is the free variable. To get the solution of the system Ax = b, now one simply has to add
a vector, p for instance, to equation (1.2). Hence, the solution for equation (1.1) is
x = tv + p (1.3)
If equation (1.2) can be seen as a line passing through the origin and the vector v, then equation
(1.3) becomes the eqaution of the line through p parallel to v. Thus the solution set of Ax = b
is a line through p parallel to the solution set of Ax = 0.

1.4 | Linear Independence

A set of vectors is {v 1 , . . . , v p } in Rn is said to be linearly independent if the vector equation
x1 v 1 + · · · + xp v p = 0
has only the trivial solution.
The set {v 1 , . . . , v p } is said to be linearly dependent if there exist weights (or coefficients)
c1 , . . . , cp , not all zero, such that
c1 v 1 + · · · + cp v p = 0
Theorem 1.2. Characterization of Linearly Dependent Sets.
A set S = {v 1 , . . . , v p } of two or more vectors is linearly dependent if and only if at least one
of the vectors in S is a linear combination of the others. In fact, if S is linearly dependent and
v 1 6= 0, then some v j (with j > 1) is a linear combination of the preceding vectors v 1 , . . . , v j−1 .

Theorem 1.3. If a set containes more vectors that there are entries in each vector (or a matrix with
more columns than rows), then the set is linearly dependent. That is, any set S = {v 1 , . . . , v p }
in Rn is linearly dependent if p > n.

1
1.5 | Introduction to Linear Transformations
A matrix equation Ax = b can arise in linear algebra in a way that is not directely connected
with linear combinations of vectors. This happens when we think of the matrix A as an object that
“acts”as a vector x by multiplication to produce a new vector called Ax.
A transformation (or function or mapping) T from Rn to Rm is a rule that assigns to each
vector x in Rn a vector T (x) in Rm . The set Rn is called the domain of T , and Rm is called the
codomain of T . The notation
T : Rn → Rm
indicates that the domain of T is Rn and the codomain is Rm . For x in Rn , the vector T (x) in Rm
is called the image of x (under the action of T ). The set of all images T (x) is called the range of
T.
For each x in Rn , T (x) is computed as Ax, where A is a m × n matrix. A matrix transformation
is usually denoted by x 7→ Ax.

A transformation (or mapping) of T is linear if:

(i) T (u + v) = T (u) + T (v) for all u, v in the domain of T ;
(ii) T (cu) = cT (u) for all scalars c and all u in the domain of T .

If T is a linear transformation, then

T (0) = 0
and
T (cu + dv) = cT (u) + dT (v)
for all vectors u, v in the domain of T and all scalars c, d.

1.6 | The Matrix of a Linear Transformation

Every linear transformation from Rn to Rm is actually a matrix transformation x 7→ Ax. The key
to finding A is to observe that T is completely determined by what it does to the columns of the
n × n identity matrix In .
The columns of  
I2 =
1 0
0 1
   
are e1 = and e2 = . If for a linear transformation T (e1 ) and T (e2 ) are given, x can
1 0
0 1
be rewritten as      
x1
x= = x1 + x2 = x1 e 1 + x2 e 2
1 0
x2
(1.4)
0 1
Since T is a linear transformation

T (x) = x1 T (e1 ) + x2 T (e2 ) (1.5)

The step from equation (1.4) to equation (1.5) explains why the knowledge of T (e1 ) and T (e2 ) is
sufficient to determine T (x) for any x.

Theorem 1.4. Let T : Rn → Rm be a linear transformation. Then there exists a unique matrix A
such that
T (x) = Ax for all x in Rn
In fact, A is the m × n matrix whose jth column is the vector T (ej ), where ej is the jth column of
the of the identity matrix in Rn :
 
A = T (e1 ) · · · T (en ) (1.6)

2
This matrix A in equation (1.6) is called the standard matrix for the linear transformation T .

A mapping T : Rn → Rm is said to be onto Rm if each b in Rm is the image of at least one x in Rn .

A mapping T : Rn → Rm is said to be one-to-one Rm if each b in Rm is the image of at

most one x in Rn .
Theorem 1.5. Let T : Rn → Rm be a linear transformation. Then T in one-to-one if and only if
the equation T (x) = 0 has only the trivial solution.

Theorem 1.6. Let T : Rn → Rm be a linear trasformation and let A be the standard matrix for T .
Then:
(i) T maps Rn onto Rm if and only if the columns of A span Rm ;

(ii) T is one-to-one if and only if the columns of A are linearly independent.

3
2 | Matrix Algebra [§2]
2.1 | Matrix Operations
If A is an m × n matriz, then the scalar entry in the ith row and jth column of A is denoted by aij
and is called the (i, j)-entry of A.
The diagonal entries in an m × n matrix A = [aij ] are a11 , . . . , aii and they form the main
diagonal of A. A diagonal matrix is a square n × n matrix whose nondiagonal entries are zero.
Two examples are the n × n identity matrix, In , and the n × n zero matrix, 0n .
Two matrices are equal if they have the same size and if their corresponding columns are
equal.
The sum A + B is the m × n matrix whose columns are the sums of the corresponding columns
in A and B.

If A is an m × n matrix, and B is an n × p matrix with columns b1 , . . . , bp , then the product

AB is the m × p matrix whose columns are Ab1 , . . . , Abp . That is,
AB = A [ b1 · · · bp ] = [ Ab1 · · · Abp ] (2.1)
From equation (2.1) it is clear that each column of AB is a linear combination of the columns of
A using weights from the corresponding column of B.
If the product AB is defined, then the entry in row i and column j of AB is the sum of the
products of the corresponding entries from row i of A and column j of B. If (A)ij denotes the
(i, j)-entry in AB, and A is an m × n matrix, then
(AB)ij = ai1 b1j + · · · + ain bnj
Warnings:
(i) In general, AB 6= BA;
(ii) The cancellation laws do not hold for matrix multiplication: i.e. if AB = AC , then it is not
true in general that B = C ;
(iii) If a product AB is the zero matrix, you cannot conclude in general that either A = 0 or B = 0.
If A is an n × n matrix and if k is a positive integer, then Ak denotes the power k of the matrix
A. That is Ak denoted the product of k copies of A.
Given an m × n matrix A, the transpose of A is the n × m matrix, denoted  AT , whose
by 
a b
columns are formed from the corresponding rows of A. For instance, let A = , then AT
c d
is  
T a c
A =
b d
Note that the transpose of a product of matrices equals the product of their transposes in the
reverse order. Hence
(AB)T = B T AT

2.2 | The Inverse of a Matrix

An n × n matrix A is said to be invertible if there is an n × n matrix C such that
AC = I and C A = I
where I = In , the n × n identity matrix. In this case, C is the inverse of A. In fact, C is uniquely
determined by A. This unique inverse is denoted by A−1 , so that
AA−1 = I and A−1 A = I
A matrix that is not invertible is sometimes called a singular matrix, and an invertible matrix is
called a nonsingular matrix.

4
  
a b
Theorem 2.1. Let A = . If ad − bc 6= 0, then A is invertible and
c d
 
d −b
A−1 =
1
ad − bc −c a

If ad − bc = 0, then A is not invertible.

The quantity ad − bc is called the determinant of A, and we write
det A = ad − bc
If A is an invertible n × n matrix, then for each b in Rn , the equation Ax = b has the unique
solutions x = A−1 b.

An elementary matrix is one that is obtained by performing a single elementary row opera-
tion on an identity matrix.
If an elementary row operation is performed on an m × n matrix A, the resulting matrix can be
written as EA, where the m × m matrix E ia created by performing the same row operation on Im .
Theorem 2.2. An n × n matrix A is invertible if and only if A is row equivalent to In , and in this
case, any sequence of elementary row operations that reduce A to In also transforms In into A−1 .
Applying theorem 2.2 to an invertible matrix A gives
A ∼ E1 A ∼ · · · ∼ Ep (Ep−1 · · · E1 A) = In
then, the product Ep · · · E1 consists of the inverse of A. Hence

A−1 = Ep · · · E1
If we place A and I side-by-side to form an augmented matrix [ A | I ], then row operations on this
matrix produce identical operations on A and I. Then, if A is row equivalent to I, [ A | I ] is row
equivalent to [ I | A−1 ]. Otherwise A does not have an inverse.

2.3 | Characterisation of Invertible Matrices

Theorem. The Invertible Matrix Theorem.
Let A be a square matrix n × n. Then the following statements are equivalent. That is, for a given
A, the statements are either all true or all false (see appendix A).
(i) A is an invertible matrix;
(ii) A is row equivalent to the n × n identity matrix;

(iii) A has n pivot points;

(iv) The equation Ax = 0 has only the trivial solution;
(v) The columns of A form a linearly independent set;
(vi) The linear transformation x 7→ Ax is one-to-one;

(vii) The equation Ax = b has at least one solution for each b in Rn ;

(viii) The columns of A span Rn ;
(ix) The linear transformation x 7→ Ax maps Rn onto Rn ;
(x) There is an n × n matrix C such that C A = I;

(xi) There is an n × n matrix D such that AD = I;

(xii) AT is an invertible matrix.

5
A linear transformation T : Rn → Rn is said to be invertible if there exists a function S : Rn → Rn
such that
S(T (x)) = x for all x in Rn
T (S(x)) = x for all x in Rn
Hence, T is invertible if and only if its standard matrix A is invertible.

2.4 | Subspaces of Rn
A subspace of Rn is any set H ∈ Rn that has three proprieties:
(i) 0 ∈ H;
(ii) ∀ u, v ∈ H, (u + v) ∈ H;
(iii) ∀ u ∈ H and ∀ c ∈ R, (cu) ∈ H.
In words, a subspace is closed under addition and scalar multiplication. Examples of subspaces
are a plane or a line both through the origin.

The column space of a matrix A is the set ColA of all linear combinations of the columns of
A.
If A = a1 · · · an , with the columns in Rm , then ColA is the same as Span{a1 , . . . , an }.
 

The column space of an m × n matrix is a subspace of Rm . Note that ColA equals Rm only when
the columns of A span Rm .
When a system of linear equations is written in the form Ax = b, the column space of A is the
set of all b for which the system has a solution.

The null space of a matrix A is the set NulA of all solutions of the homogeneous equation
Ax = 0.
Theorem 2.3. The null space of an m × n matrix A is a subspace of Rn . Equivalently, the set of
all solutions of a system Ax = 0 of m homogeneous linear equations in n unknowns is a subspace
of Rn .
Because a subspace typically contains an infinite number of vectors, some problems are handled
best by working with a small finite set of vectors that span the subspace. The smaller the set, the
better. The smallest possible spanning set must be linearly independent.

A basis for a subspace H of Rn is a linearly independent set in H that spans H.

The columns of an invertible n × n matrix form a basis for all of Rn , because they are lin-
early independent and span Rn . One such matrix is the n × n identity matrix. Its columns are
denoted by e1 , . . . , en :    
1 0
e1 =  ...  , . . . , en =  ... 
   

0 1
n
The set {e1 , . . . , en } is called the standard basis for R .

The standard procedure for writing the solution set of Ax = 0 in parametric vector form, ac-
tually identifies a basis for NulA.
Suppose you are given a 3 × 5 matrix A and you are asked to compute the null space of such
matrix:    
a11 · · · a15 x1
A =  ... ..  with solution x =  ..  = x u + x v + x w
 ..
. .   .  2 3 5
a31 ··· a35 x5
The general solution shows that NulA coincides with the set of all linear combinations of u, v and w.
That is, {u, v, w} generates NulA. So {u, v, w} is a basis for NulA.

6
Theorem 2.4. The pivot columns of a matrix A form a basis for the column space of A.

Be careful to use pivot columns of A itself for the basis of ColA. The columns of an echelon form
AR are often not in the column space of A.

2.5 | Dimension and Rank

The main reason for selecting a basis for a subspace H, is that each vector in H can be written
in only one way as a linear combination of the basis vectors of H. To see why, suppose B =
{b1 , . . . , bp } is a basis for H, and suppose a vector x in H can be generated in two ways, say,

x = c1 b1 + · · · + cp bp and x = d1 b1 + · · · + dp bp

Then, subtracting gives

0 = (c1 − d1 )b1 + · · · + (cp − dp )bp
Since B is linearly independent, the weight in this last equation must all be zero. That is, cj = dj
for 1 ≤ j ≤ p, which shows that the two representations are actually the same.
Suppose the set B = {b1 , . . . , bp } is a basis for a subspace H. For each x in H, the
coordinates of x relative to the basis B are the weights c1 , . . . , cp such that x = c1 b1 +· · ·+cp bp ,
and the vector in Rp  
c1
 
[x]B =  ... 
cp
is called the coordinate vector of x (relative to B ) or the B -coordinate vector of x. Hence
x = B [x]B .
The mapping x 7→ [x]B , where B = {b1 , . . . , bp }, is a one-to-one correspondence between H
and Rp that preserves linear combinations. We call such a correspondence as isomorphism, and
we say that H is isomorphic to Rp .

The dimention of a nonzero subspace H, denoted by dimH, is the number of vectors in any
basis for H. The dimention of the zero subspace {0} is defined to be zero.

The rank of a matrix A, denoted by rankA, is the dimention of the column space of A.
Since the pivot columns of A form a basis for ColA, the rank of A is just the number of pivot
columns in A.
Theorem 2.5. The Rank Theorem.
Since the nonpivot columns correspond to the free variables in Ax = 0, if a matrix A has n columns,
then rankA + dimNulA = n.

Theorem 2.6. The Basis Theorem.

Let H be a p-dimentional (i.e. with dimH = p) subspace of Rn . Any linearly independent set of
exactly p elements in H is automatically a basis for H. Also, any set of p elements of H that
spans H is automatically a basis for H.

If A is an m × n matrix, each row of A has n entries and thus can be identified with a vector in Rn .
The set of all linear combinations of the row vector is called the row space of A and is denoted
by RowA, which is also a subspace of Rn . Since the rows of A are identified with the columns of
AT , we could also write ColAT in place of RowA.
If two matrices A and B are row equivalent (A ∼ B), then their row spaces are the same. If B
is in echelon form, the nonzero rows of B form a basis for the row space of A as well as for that
of B.

7
Theorem. The Invertible Matrix Theorem (Continued).
Let A be an n × n matrix. Then the following statements are each equivalent to the statement A
is an invertible matrix (see appendix A).

(xiii) The columns of A form a basis of Rn ;

(xiv) ColA = Rn ;
(xv) dimColA = n;
(xvi) rankA = n;

(xvii) NulA = {0};

(xviii) dimNulA = 0.

8
3 | Determinants [§3]
3.1 | Introduction to Determinants
If A is a 3 × 3 matrix, we can write it as
 
a11 a12 a13
A =  a21 a22 a23 
a31 a32 a33

then its determinant is given by

∆ = det A = |A| = a11 det A11 − a12 det A12 + a13 det A13

In general, for n ≥ 2, the determinant algorithm for an n × n matrix A = [aij ] is

n
X
|A| = det A = (−1)1+j a1j det A1j
j=1

It is useful to define the (i, j)-cofactor of a determinant A = [aij ]. The cofactor is

Cij = (−1)i+j det Aij (3.1)

Then,
det A = a11 C11 + · · · + a1n C1n
This formula is called a cofactor expansion across the first row of A.

Theorem 3.1. In general, the determinant of an n × n matrix A can be computed by cofactor

expansion across any row or down any column. The expansion across the ith row using cofactor
is
Xn
|A| = aij Cij keeping i fixed
j=1

The cofactor expansion down the jth column is

n
X
|A| = aij Cij keeping j fixed
i=1

Theorem 3.2. If A is a triangular matrix, the det A is the product of the entries on the main
diagonal of A.

3.2 | Proprieties of Determinants

Theorem 3.3. Row Operations.
Let A be a square matrix.

(i) If a multiple of one row of A is added to another row to produce a matrix B, then det A = det B;
(ii) If two rows of A are interchanged to produce B, then det B = − det A;
(iii) If one row of A is multiplied by k to produce B, then det B = k · det A.

Suppose a square matrix has been reduced to an echelon form U by row replacements and row
interchanges. If there are r interchanges, then, as seen before,

det A = (−1)r det U

9
Since U is in echelon form (not the reduced echelon form), it is triangular,
 
• ∗ ∗ ∗
 0 • ∗ ∗ 
U=  0 0 • ∗ 


0 0 0 •
and so det U is the product of the diagonal entries u11 , . . . , unn . If A is invertible, the entries uii
are all pivots. Otherwise, at least one unn is zero. Thus
(−1)r · (product of pivots in U) if A invertible

det A =
0 if A not invertible
Theorem 3.4. A square matrix A is invertible if and only if det A 6= 0.

Theorem 3.5. If A is an n × n matrix, then det A = det AT .

Because of theorem 3.5 each statement in theorem 3.3 is also true when the word row is replaced
by the word column.
Theorem 3.6. If A and B are n × n matrices, then det AB = det A · det B.
Suppose that the jth column of A is allowed to vary, and write
 
A = a1 · · · aj−1 x aj+1 · · · an
Define a transformation T from Rn to R by
 
T (x) = det a1 · · · aj−1 x aj+1 ··· an
Then,
T (cx) = cT (x) for all scalars c and all x in Rn
T (u + v) = T (u) + T (v) for all u, v in Rn

3.3 | Cramer’s Rule

For any n × n matrix A and any b in Rn , let Ai (b) be the matrix obtained from A by replacing the
column i by the vector b.  
Ai (b) = a1 · · · b · · · an
Theorem 3.7. Cramer’s Rule.
Let A be an invertible n × n matrix. For any b in Rn , the unique solution x of Ax = b has entries
given by
det Ai (b)
xi = , i = 1, . . . , n
det A
Cramer’s rule leads easily to a general formula for the inverse of an n × n matrix A. The jth
column of A−1 is a vector x that satisfies
Ax = ej
where ej is the jth column of the identity matrix, and the ith entry of x is the (i, j)-entry of A−1 .
By Cramer’s rule,
det Ai (ej )
(i, j)-entry of A−1 = xi =

det A
(3.2)
Then recalling the definition of cofactor of A, equation (3.1), det Ai (ej ) can be written as the cofactor
expansion Cji
det Ai (ej ) = (−1)i+j det Aji = Cji
Thus  
C11 ··· Cn1
A−1
1  . .. .. 
 ..
det A
= . .  (3.3)
C1n ··· Cnn
The matrix of cofactors on the right side of (3.3) is called the adjugate of A, denoted by adjA.

10
Theorem 3.8. An Inverse Formula.
Let A be an invertible n × n matrix. Then

A−1 =
1
det A
adjA

11
4 | Eigenvalues and Eigenvectors [§5]
4.1 | Eigenvectors and Eigenvalues
An eigenvecotr of an n × n matrix A is a nonzero vector x such that Ax = λx for some scalar λ.
A scalar λ is called an eigenvalue of A if there is a nontrivial solution x of Ax = λx; such an x
is called an eigenvector corresponding to λ.
A scalar λ is an eigenvalue of an n × n matrix A if and only if the equation
(A − λI)x = 0 (4.1)
has a non trivial solution. The set of all solutions of (4.1) is just the null space of the matrix
A − λI. So this set is a subspace of Rn and is called the eigenspace of A corresponding to λ. The
eigenspace consists of the zero vector and all the eigenvectors corresponding to λ.
Theorem 4.1. The eigenvalues of a triangular matrix are the entries on its main diagonal.
Zero is an eigenvalue of A if and only if A is not invertible.
Theorem 4.2. If v 1 , . . . , v r are eigenvectors that correspond to distinct eigenvalues λ1 , . . . , λr of
an n × n matrix A, then the set v 1 , . . . , v r is linearly independent.

Theorem. The Invertible Matrix Theorem (Continued).

Let A be an n × n matrix. Then the following statements are each equivalent to the statement A
is an invertible matrix (see appendix A).

(xix) The determinant of A is not zero;

(xx) The number 0 is not an eigenvalue of A.

4.2 | The Characteristic Equation

To find the eigenvalues of an n × n matrix A, one has to find all scalars λ such that the matrix
equation (4.1)
(A − λI)x = 0
has a nontrivial solution. By the Invertible Matrix Theorem (appendix A), this problem is equivalent
to finding all λ such that the matrix A − λI is not invertible. Recalling that the determinant of a
singular (not invertible) matrix is always zero, the following fact will result very useful for solving
exercises.
A scalar λ is an eigenvalue of an n × n matrix A if and only if λ satisfies the characteristic
equation
det(A − λI) = 0 (4.2)
If A is an n × n matrix, then det(A − λI) is a polynomial of degree n called the characteristic
polynomial of A.
The algebraic multiplicity of an eigenvalue λ is its multiplicity as a root of the characteristic
equation.

4.3 | Similarity
If A and B are n × n matrices, then A is similar to B if there is an invertible matrix P such that
P −1 AP = B, or, equivalentely, A = PBP −1 . Writing Q for P −1 , we have Q −1 BQ = A. So B is
also similar to A, and we say simply that A and B are similar. Changing A into P −1 AP is called
a similarity transformation.
Theorem 4.3. If n × n matrices A and B are similar, then they have the same characteristic
polynomial and hence the same eigenvalues (with the same algebraic multiplicities).
Warnings:
(i) Having the same eigenvalues does not mean being similar;
(ii) Similarity is not the same a row equivalence; row operations on a matrix usually changes its
eigenvalues.

12
4.4 | Diagonalization
In many cases, the eigenvalue-eigenvector information contained within a matrix A can be dis-
played in a useful factorizartion of the form A = PDP −1 where D is a diagonal matrix.

A matrix A is said to be diagonalizable if A is similar to a diagonal matrix, that is if A = PDP −1

for some invertible matrix P and some diagonal matrix D.
Theorem 4.4. The Diagonalization Theorem.
An n × n matrix A is diagonalizable if and only if A has n linearly independent eigenvectors.
In fact, A = PDP −1 , with D a diagonal matrix, if and only if the columns of P are n linearly
independent eigenvectors of A. In this case, the diagonal entries of D are eigenvalues of A that
correspond, respectively, to the eigenvectors in P.
In other words, A is diagonalizable if and only if there are enough eigenvectors to form a basis of
Rn . We call such basis an eigenvector basis of Rn .

Diagonalizing Matrices:
To diagonalize A, an n × n matrix:
Step 1 Find the eigenvalues of A using equation (4.2).
Step 2 Find n linearly independent eigenvectors of A.
Step 3 Construct P from the vectors just found.
Step 4 Construct D from the corresponding eigenvalues.

Theorem 4.5. An n × n matrix with n distrinct eigenvalues is diagonalizable.

Notice that the statement in theorem 4.5 is not necessary.

Theorem 4.6. Let A be an n × n matrix whose distinct eigenvalues are λ1 , . . . , λp .

(i) For 1 ≤ k ≤ p, the dimention of the eigenspace for λk is less than or equal to the algebraic
multiplicity of the eigenvalue λk ;
(ii) The matrix A is diagonalizable if and only if the sum of the dimensions of the eigenspaces
equals n, and this happens if and only if (1) the characteristic polynomial factors completely
into linear factors, and (2) the dimension of the eigenspace for each λk , the geometric mul-
tiplicity, equals the algebraic multiplicity of λk ;
(iii) If A is diagonalizable and Bk is a basis for the eigenspace corresponding to λk for each k,
then the total collection of vectors in the set B1 , . . . , Bp forms an eigenvector basis for Rn .

4.5 | Eigenvectors and Linear Transformations

Let V be an n-dimentional vector space, let W be an m-dimensional vector space, and let T be
any linear transformation from V to W . To associate a matrix with T , choose bases B and C for
V and W , respectively.
Given any x in V , the coordinate vector [x]B is in Rn and the coordinate vector of its image,
[T (x)]C , is in Rm .
The connection between [x]B and [T (x)]C can be found in the following way. Let {b1 , . . . , bn }
be the basis B for V . If x = r1 b1 + · · · + rn bn , then
 
r1
 
[x]B =  ... 
rn

and
T (x) = T (r1 b1 + · · · + rn bn ) = r1 T (b1 ) + · · · + rn T (bn ) (4.3)

13
because T is linear. Since the coordinate mapping from W to Rm is linear, equation (4.3) leads
to:
[T (x)]C = r1 [T (b1 )]C + · · · + rn [T (bn )]C (4.4)
Since C-coordinate vectors are in Rm , the vector equation (4.4) can be written as a matrix equation,
namely
[T (x)]C = M[x]B (4.5)
where  
M= [T (b1 )] ··· [T (bn )] (4.6)
The matrix M is a matrix representation of T , called the matrix for T relative to the bases B and
C.
Equation (4.5) says that the action of T on x may be viewed as left-multiplication by M.

In the common case where W is the same as V and the basis C is the same as B , the ma-
trix M in equation (4.6) is called the matrix for T relative to B , or simply the B -matrix for T , and
is denoted by [T ]B .
The B -matrix for T : V → V satisfies

[T (x)]B = [T ]B [x]B for all x is V

Theorem 4.7. Diagonal Matrix Representation.

Suppose A = PDP −1 , where D is a diagonal n × n matrix. If B is the basis for Rn formed from
the columns of P, then D is the B -matrix for the transformation x 7→ Ax.

4.6 | Complex Eigenvalues

A complex scalar λ satisfies det(A − λI) = 0 if and only if there is a nonzero vector x in Cn such
that Ax = λx. We call λ a complex eigenvalue and x a complex eigenvector corresponding to λ.

The complex conjugate of a complex vector x in Cn is the vector x in Cn whose entries are
the complex conjugates of the entries in x. The real and imaginary parts of a complex vector x
are the vectors <x and =x formed from the real and imaginary parts of the entries of x.

Let A be an n × n matrix whose entries are real. Then Ax = Ax = Ax. If λ is an eigenvalue of A

and x is a corresponding eigenvector in Cn , then

Ax = Ax = λx = λx

Hence λ is also an eigenvalue of A, with x a corresponding eigenvector. This shows that, when A
is real, its complex eigenvalues occur in conjugate pairs.

Theorem 4.8. Let A be a real 2 × 2 matrix with a complex eigenvalue λ = a − bi (b 6= 0) and an

associated eigenvector v in C2 . Then
 
−1
  a −b
A = PC P , where P = <v =v and C =
b a

In theorem 4.8, the matrix P provides a change of variable. The action of A amounts to a change
of variables (P), followed by a rotation (C ) and then a return to the original variable (P −1 ).
The matrix C in theorem 4.8 can also be written as
  
r 0 cos φ − sin φ
C=
0 r sin φ cos φ
√
where r is given by a2 + b2 and φ represents the rotation.

14
4.7 | Application to Differential Equations
In many applied problems, several quantities are varying continuously in time, and they are related
by a system of differential equations:

x10 = a11 x1 + · · · + a1n xn

..
.
xn0 = an1 x1 + · · · + ann xn

Here x1 , . . . , xn are differentiable functions of t, with derivatives x10 , . . . , xn0 , and the aij are con-
stants. The crucial feature of this system is that it is linear. To see this, write the system as a
matrix differential equation
x 0 (t) = Ax(t) (4.7)
where
x10 (t)
     
x1 (t) a11 ··· a1n
x(t) =  ...  x 0 (t) =  ...  A= .
     .. .. .. 
. . 
xn (t) xn0 (t) an1 ··· ann
A solution of equation (4.7) is a vector valued function that satisfies (4.7) for all t in some interval
of real numbers.
Equation (4.7) is linear because both differentiation of functions and multiplication of vectors
by a matrix are linear transformations.

For the general solution of equation (4.7), a solution might be a linear combination of functions of
the form
x(t) = veλt (4.8)
for some scalar λ and some fixed nonzero vector v. Observe that
x 0 (t) = λveλt

λveλt = Aveλt
Ax(t) = Aveλt

Since eλt is never zero, x 0 (t) will equal Ax(t) if and only if λv = Av, that is, if and only if λ is
an eigenvalue of A and v is a corresponding eigenvector. Thus each eigenvalue-eigenvector pair
provides a solution (4.8) of x 0 = Ax. Such solutions are sometimes called eigenfunctions of the
differential equation.

For any dynamical system described by x 0 (t) = Ax with A an n × n matrix with n linearly
independent eigenvectors (i.e. with A diagonalizable), a solution can be found in the following
way.
Suppose the eigenfunctions for A are

v 1 eλ1 t , . . . , v n eλn t
 
with v 1 , . . . , v n linearly independent eigenvectors. Let P = v 1 · · · v n , and let D be the
diagonal matrix with entries λ1 , . . . , λn so that A = PDP −1 . Now make a change of variable,
defining a new function y by

y(t) = P −1 x(t) or, equivalentely x(t) = Py(t)

The equation x(t) = Py(t) says that y(t) is the coordinate vector of x(t) relative to the eigenvector
basis. Substitution of Py for x in the equation x 0 = Ax gives

d
(Py) = A(Py) = (PDP −1 )Py = PDy
dt
(4.9)

15
Since P is a constant matrix, the left side of equation (4.9) is Py0 . Left-multiply both sides of
(4.9) by P −1 and obtain y0 = Dy, or
 0    
y1 (t) λ1 · · · 0 y1 (t)
 ..   .. .. ..   .. 
 . = . . .  . 
0
yn (t) 0 · · · λn yn (t)

The change of variables from x to y has decoupled the system of differential equations. Since
y01 = λ1 y1 , we have y1 (t) = c1 eλ1 t , with similar formulas for y2 , . . . , yn . Thus

c1 eλ1 t
 

y(t) = 
 .. 
. 
cn eλn t

the general solution x is

 
x(t) = Py(t) = v1 ··· vn y(t) =

= c1 v 1 eλ1 t + · · · + cn v n eλn t (4.10)

Equation (4.10) is known as the eigenfunction expansion.

In case a real matrix A has a pair of complex eigenvalues λ and λ, with associated eigenvec-
tors v and v, two solutions of x 0 = Ax are

x 1 (t) = veλt and x 2 (t) = veλt

where it can be shown that x 2 (t) = x 1 (t). This two solutions will obvioulsy involve complex
numbers.
For a real matrix A with an eigenvalue λ = a + bi, a corresponding eigenvector v and a
complex solution x 1 (t) = veλt of x 0 = Ax, two real solutions of the same x 0 = Ax are given by

y1 (t) = <x 1 (t) = [ (<v) cos bt − (=v) sin bt ] eat

y2 (t) = =x 1 (t) = [ (<v) sin bt + (=v) cos bt ] eat

Hence, the general solution is
x(t) = c1 y1 (t) + c2 y2 (t)

4.8 | Trajectories of Eigenfunctions

When the matrix A in x 0 (t) = Ax(t) is 2 × 2, algebraic calculations can be supplemented by a
geometric description of a system’s evolution. We can plot the graph of the two eigenfunctions
x 1 (t) and x 2 (t) as a description of what happens to the system as t → ∞. The graph of a single
solution x 1 (t) is called a trajectory of the dynamical system.
The origin of the graph of a dynamical system may be:
(i) an attractor, or sink,
(ii) a repeller, or source, or
(iii) a saddle point.
The origin is called an attractor of the dynamical system when all trajectories tend toward 0. This
occurs whenever both eigenvalues are negative:

λ1 < 0 and λ2 < 0

The direction of greatest attraction is along the line through 0 and the eigenfunction corresponding
to the smaller eigenvalue.

16
 The origin is called a repeller of the dynamical system when trajectories tend away from 0.
This occurs whenever both eigenvalues are positive:

λ1 > 0 and λ2 > 0

The direction of greatest repulsion is the line though 0 and the eigenfunction corresponding to
the eigenvalue of larger magnitude.
The origin is called a saddle point of the dynamical system when some trajectories approach
the origin at first and then change direction and move away from the origin. This occurs whenever
one eigenvalue is positive and the other is negative:

λ1 < 0 and λ2 > 0

The direction of greatest attraction is determined by the eigenfunction for the eigenvalue of smaller
magnitude. The direction of greatest repulsion is determined by the eigenfunction for the eigen-
value of greater magnitude.

In case the matrix A has complex eigenvalues given by λ = a ± bi, the origin may be a spi-
ral point. The rotation is caused by the sine and cosine functions that arise from a complex
eigenvalue. When the real part of the complex eigenvalue is positive (a > 0), the trajectories
spiral outward. When the real part of the complex eigenvalue is negative (a < 0), the trajectories
spiral inward.
If the real part of the eigenvalue is zero (a = 0), the trajectories form ellipses around the
origin.

17
5 | Orthogonality and Least Squares [§6]
5.1 | Inner Product and Orthogonality
If u and v are vectors in Rn
uT v = v T u = u · v
is called the inner product or dot product. If
  
u1 v1
u= .  and v =  .. 
 ..   . 

un vn

the inner product of u and v is

u1 v1 + · · · + un vn
The length or norm of v is the nonnegative scalar ||v|| defined by
√ q
||v|| = v · v = v12 + · · · + vn2 and ||v||2 = v · v

A vector whose length is 1 is called a unit vector. If we divide a nonzero vector v by its length,
we obtain a unit vector u. This process is sometimes called normalising.

For u and v in Rn , the distance between u and v, written as dist(u, v), is the length of the
vector u − v. That is,
dist(u, v) = u − v
Two vectors u and v are orthogonal (to each other) if u · v = 0.

Theorem 5.1. The Pythagorean Theorem.

Two vectors u and v are orthogonal if and only if ||u + v||2 = ||u||2 + ||v||2 .

If a vector z is orthogonal to every vector in a subspace W of Rn , then z is said to be orthogonal

to W . The set of all vectors z that are orthogonal to W is called the orthogonal complement of
W and is denoted by W ⊥ .
A vector x is in W ⊥ if and only if x is orthogonal to every vector is a set that spans W .
Moreover, W ⊥ is a subspace of Rn .

Theorem 5.2. Let A be an m × n matrix. The orthogonal complement of the row space of A is the
null space of A, and the orthogonal complement of the column space of A is the null space of AT :

(RowA)⊥ = NulA and (ColA)⊥ = NulAT

For u and v in either R2 or R3 , the inner product of the two vectors can be written as

u · v = ||u|| ||v|| cos θ

5.2 | Orthogonal Sets

A set of vectors {u1 , . . . , up } in Rn is said to be an orthogonal set if each pair of distinct vector
from the set is orthogonal, that is, if ui · uj = 0 whenever i 6= j.

Theorem 5.3. If S = {u1 , . . . , up } is an orthogonal set of nonzero vectors in Rn , then S is linearly

independent and hence is a basis for the subspace spanned by S.

An orthogonal basis for a subspace W of Rn is a basis for W that is also an orthogonal set.

18
Theorem 5.4. Let {u1 , . . . , up } be an orthogonal basis for a subspace W of Rn . For each y in
W , the weights in the linear combination

y = c1 u1 + · · · + cp up

are given by
y · uj
cj = (j = 1, . . . , p)
uj · uj

Given a nonzero vector u in Rn , consider the problem of decomposing a vector y in Rn into the
sum of two vectors, one a multiple of u and the other orthogonal to u. We wish to write

y = ŷ + z (5.1)

where ŷ = αu for some scalar α and z is some vector orthogonal to u. Let z = y − αu. Then
y − ŷ is orthogonal to u if and only if

0 = (y − αu) · u = y · u − (αu) · u = y · u − α(u · u)

That is, equation (5.1) is satisfied if and only if α = y·u y·u

u·u and ŷ = u·u u. The vector ŷ is called the
orthogonal projection of y onto u, and the vector z is called the component of y orthogonal to
u.
Sometimes ŷ is denoted by projL y and is called the orthogonal projection of y onto L. That
y·u
is
ŷ = projL y = u
u·u
Theorem 5.4 decomposes each y in Span{u1 , . . . , up } into the sum of p projections onto one-
dimensional subspaces that are mutually orthogonal.

A set {u1 , . . . , up } is an orthonormal set if it is an orthogonal set of unit vectors. If W is

the subspace spanned by such a set, then {u1 , . . . , up } is an orthonormal basis for W .

Theorem 5.5. An m × n matrix U has orthonormal columns if and only if U T U = I.

An orthogonal matrix is a square invertible matrix U such that U −1 = U T .

The reflection of a vector y ∈ Rn in L = Span{u}, where u 6= 0 ∈ Rn , is the point reflL y

defined by
reflL y = 2 · projL y − y

5.3 | Orthogonal Projections

Theorem 5.6. The Orthogonal Decomposition Theorem.
Let W be a subspace of Rn . Then each y in Rn can be written uniquely in the form

y = ŷ + z (5.2)

where ŷ is in W and z is in W ⊥ . In fact, if {u1 , . . . , up } is any orthogonal basis of W , then

y · u1 y · up
ŷ = · u1 + · · · + · up
u1 · u1 up · up

and z = y − ŷ.

The vector ŷ in (5.2) is called the orthogonal projection of y onto W and is often written as projW y.

If {u1 , . . . , up } is an orthogonal basis for W and if y happens to be in W , then projW y = y.

19
Theorem 5.7. The Best Approximation Theorem.
Let W be a subspace of Rn , let y be any vector in Rn , and let ŷ be the orthogonal projection of
y onto W . Then ŷ is the closest point in W to y, in the sense that

||y − ŷ|| < ||y − v||

for all v in W distinct from ŷ.

The vector ŷ in theorem 5.7 is called the best approximation to y by elements of W .
Theorem 5.8. If {u1 , . . . , up } is an orthonormal basis for a subspace W of Rn , then

projW y = (y · u1 )u1 + · · · + (y · up )up

 
If U = u1 ··· up , then

projW y = UU T y for all y in Rn

Suppose U is an n × p matrix with orthonormal columns, and let W be the column space of U.
Then
U T Ux = Ip x = x for all x in Rp
UU T y = projW y for all y in Rn
If U is an n × n matrix with orthonormal columns, then U is orthogonal, the columns space W is
all of Rn and UU T y = Iy = y for all y ∈ Rn .

5.4 | The Gram-Schmidt Process

The Gram-Schmidt process is a simple algorithm for producing an orthogonal or orthonormal
basis for any nonzero subspace of Rn . Considering a basis {s1 , . . . , sp }, when the Gram-Schmidt
process is applied, on any vector of the basis sk with 1 < k ≤ p, the components of the vectors
s1 , . . . , sk−1 parallel to sk are removed from sk . In this way only the perpendicular part of sk
stays in the basis, making it orthogonal. The following theorem explains this process.
Theorem 5.9. The Gram-Schmidt Process.
Given a basis {x 1 , . . . , x p } for a nonzero subspace W of Rn , define

v 1 = x1
x2 · v 1
v 2 = x2 − v1
v1 · v1
..
.
xp · v 1 x p · v p−1
v p = xp − v1 − · · · − v p−1
v1 · v1 v p−1 · v p−1
Then {v 1 , . . . , v p } is an orthogonal basis for W . In addition

Span{v 1 , . . . , v k } = Span{x 1 , . . . , x k } for 1 ≤ k ≤ p

An orthonormal basis is constructed easily from an orthogonal basis {v 1 , . . . , v p }: simply normalize
all the v k .
Theorem 5.10. The QR Factorization.
If A is an m × n matrix with linearly independent columns, then A can be factored as A = QR,
where Q is an m × n matrix whose columns form an orthonormal basis for ColA and R is an n × n
upper triagular invertible matrix with positive entries on its diagonal.

To find R , observe that Q T Q = I, because the columns fo Q are orthonormal. Hence

R = QT A

20
5.5 | Least-square Problems
When a solution to a linear system Ax = b is demanded and none exists, the best one can do is
to find an x that makes Ax as close as possible to b.
Think of Ax as an approximation to b. The smaller the distance between b and Ax, given by
||b − Ax||, the better the approximation. The general least-squares problem is to find an x that
makes ||b − Ax|| as small as possible.
If A is m × n and b is in Rm , a least-square solution of Ax = b is an x̂ in Rn such that

||b − Ax̂|| ≤ ||b − Ax||

for all x in Rn .
The most important aspect of the least-square problem is that no matter what x we select, the
vector Ax will necessarely be in the column space, ColA. So we can seek an x that makes Ax the
closest point in ColA to b.

Given A and b, let

b̂ = projColA b
Because b̂ is in the column space of A, the equation Ax = b̂ is consistent, and there is an x̂ in
Rn such that
Ax̂ = b̂ (5.3)
Since b̂ is the closest point in ColA to b, a vector x̂ is a least-squares solution of Ax = b if and
only if x̂ satisfies equation (5.3).

Each least-squares solution of Ax = b satisfies the equation

AT Ax = AT b (5.4)

The matrix equation (5.4) represents a system of equations called the normal equations for Ax = b.
A solution of (5.4) is often denoted by x̂.

Theorem 5.11. The set of least-squares solutions of Ax = b coincides with the nonempty set of
solutions of the normal equations AT Ax = AT b.

Theorem 5.12. Let A be an m × n matrix. The following statements are logically equivalent:
(i) The equation Ax = b has a unique least-squares solution for each b in Rm ;

(ii) The columns of A are linearly independent;

(iii) The matrix AT A is invertible.
When these statements are true, the least-squares solution x̂ is given by

x̂ = (AT A)−1 AT b

When a least-squares solution x̂ is used to produce Ax̂ as an approximation to b, the distance

from b to Ax̂, ||b − b̂|| = ||b − Ax̂||, is called the least-squares error of this approximation.

Theorem 5.13. Given an m × n matrix A with linearly independent columns, let A = QR be a QR

factorization of A, as in theorem 5.10. Then, for each b in Rm , the equation Ax = b has a unique
least-squares solution, given by
x̂ = R −1 Q T b

21
6 | Symmetric Matrices [§7]
6.1 | Diagonalization of Symmetric Matrices
A symmetric matrix is a matrix A such that AT = A. Such a matrix is necessarily square. Its main
diagonal entries may be arbitrary, but its other entries occur in pairs.
Theorem 6.1. If A is symmetric, then any two eigenvectors from different eigenspaces are orthog-
onal.
An n × n matrix A is said to be orthogonally diagonalizable if there are an orthogonal matrix P
(with P −1 = P T ) and a diagonal matrix D such that
A = PDP T = PDP −1 (6.1)
Such a diagonalization requires n linearly independent and orthonormal eigenvectors. If A is
orthogonally diagonalizable as in (6.1), then
AT = (PDP T )T = PDP T = A
Thus A is symmetric. Note that the diagonal entries of D are the eigenvalues λ1 , . . . , λn .
Theorem 6.2. An n × n matrix A is orthogonally diagonalizable if and only if A is a symmetric
matrix.
The set of eigenvalues of a matrix A is sometimes called the spectrum of A, and the following
description of the eigenvalues is called a spectral theorem.
Theorem 6.3. The Spectral Theorem for Symmetric Matrices.
An n × n symmetric matrix A has the following proprieties:

(i) A has n real eigenvalues, counting algebraic multiplicities;

(ii) The dimension of the eigenspace for each eigenvalue λ equals the multiplicity of λ as a root
of the characteristic equation;
(iii) The eigenspaces are mutually orthogonal;

(iv) A is orthogonally diagonalizable.

Using the definition of orthogonal diagonalization given by equation (6.1) we can rewrite a matrix
A as
A = λ1 u1 uT1 + · · · + λn un uTn (6.2)
This representation of A is called spectral decomposition of A because it breaks up A into pieces
determined by the spectrum of A. Each term in equation (6.2) is an n × n matrix of rank 1.
Furthermore, each matrix uj uTj is a projection matrix in the sense that for each x in Rn , the vector
(uj uTj )x is the orthogonal projection of x onto the subspace spanned by uj .

6.2 | Quadratic Forms

A quadratic form on Rn is a function Q defined on Rn whose value at a vector x in Rn can be
computed by an expression of the form Q(x) = x T Ax, where A is an n × n symmetric matrix. The
matrix A is called the matrix of the quadratic form.

If x represents a variable vector in Rn , then a change of variable is an equation of the form

x = Py or equivalentely, y = P −1 x (6.3)
where P is an invertible matrix and y is a new variable vector in Rn .
If the change of variable (6.3) is made in a quadratic form x T Ax, then
x T Ax = yT (P T AP)y (6.4)
and the new quadratic form is P T AP. Since A is symmetric, by theorem 6.2, there is an orthogonal
matrix P such that P T AP is a diagonal matrix D, and the quadratic form in (6.4) becomes yT Dy.

22
Theorem 6.4. The Principal Axes Theorem.
Let A be an n × n symmetric matrix. Then there is an orthogonal change of variable x = Py, that
transforms the quadratic form from x T Ax into a quadratic form yT Dy with no cross-product term.

The columns of P in theorem 6.4 are called the principal axes of the quadratic form x T Ax. The
vector y is the coordinate vector of x relative to the orthonormal basis of Rn given by these
principal axes.

Suppose Q(x) = x T Ax where A is an invertible 2 × 2 symmetric matrix, and let c be a con-

stant. The set of all x in R2 that satisfy

x T Ax = c

either corresponds to an ellipse, a hyperbola, two intersecting lines, a single point or contains no
points at all. If A is a diagonal matrix, the graph is in standard position, otherwise, if A is not
diagonal, the graph is rotated out of strandard position.

When A is an n × n matrix, the quadratic form Q(x) = x T Ax is a real-valued function with

domain Rn .
A quadratic form Q is:
(i) positive definite if Q(x) > 0 for all x 6= 0,
(ii) negative definite if Q(x) < 0 for all x 6= 0,

(iii) indefinite if Q(x) assumes both positive and negative values.

Also Q is said to be positive semidefinite if Q(x) ≥ 0 for all x, and to be negative semidefinite if
Q(x) ≤ 0 for all x.

Theorem 6.5. Quadratic Forms and Eigenvalues.

Let A be an n × n symmetric matrix. Then a quadratic form x T Ax is:
(i) positive definite if and only if the eigenvalues of A are all positive,

(ii) negative definite if and only if the eigenvalues of A are all negative, or
(iii) indefinite if and only if A has both positive and negative eigenvalues.

23
A | Invertible Matrix Theorem
Theorem. The Invertible Matrix Theorem.
Let A be a square matrix n × n. Then the following statements are equivalent. That is, for a given
A, the statements are either all true or all false.
(i) A is an invertible matrix;
(ii) A is row equivalent to the n × n identity matrix;
(iii) A has n pivot points;

(iv) The equation Ax = 0 has only the trivial solution;

(v) The columns of A form a linearly independent set;
(vi) The linear transformation x 7→ Ax is one-to-one;

(vii) The equation Ax = b has at least one solution for each b in Rn ;

(viii) The columns of A span Rn ;
(ix) The linear transformation x 7→ Ax maps Rn onto Rn ;
(x) There is an n × n matrix C such that C A = I;

(xi) There is an n × n matrix D such that AD = I;

(xii) AT is an invertible matrix;
(xiii) The columns of A form a basis of Rn ;

(xiv) ColA = Rn ;
(xv) dimColA = n;
(xvi) rankA = n;
(xvii) NulA = {0};

(xviii) dimNulA = 0;
(xix) The determinant of A is not zero;
(xx) The number 0 is not an eigenvalue of A.

I
This was:

WI1403-LR Linear Algebra

Version 1.0

More summaries are available for the following courses:

• AE1110 Intro to AE I - Module E: Aerodynamics
• AE1110 Intro to AE I - Module F: Flight Mechanics
• AE1108-I Aerospace Materials
• AE1240-I Physics I
• AE1240-II Physics II
• AE3211-II Production of Aerospace Systems
• AE4447 Aircraft Performance Optimization
• WI1402-LR Calculus II
• WI1403-LR Linear Algebra