Mathematical Methods

for Physics & Engineering

Part VI
Matrices: Diagonalization & Applications

Topics Covered:
• Eigenvalues & Eigenvectors: Definitions, the Characteristic Polynomial
• Algebraic & Geometric Multiplicity; When They Differ
• Linear Independence of Eigenvectors from Distinct Eigenvalues (Proof)
• Diagonalization: Necessary & Sufficient Conditions (Full Proof)
• The Change-of-Basis Interpretation: What Diagonalization Means
Geometrically
• Application I: Matrix Powers An via Diagonalization
• Application II: The Matrix Exponential eAt and Systems of ODEs
• Application III: Large Powers, Markov Chains & Steady States
• The Spectral Theorem for Real Symmetric Matrices
• Cayley-Hamilton Theorem and Minimal Polynomial

Based on:
Mathematical Methods for Physicists — Arfken, Weber & Harris
Mathematical Methods in the Physical Sciences — Mary L. Boas
Advanced Engineering Mathematics — Erwin Kreyszig
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Contents

1 Motivation: Why Diagonalize? 2

2 Eigenvalues and Eigenvectors 2

2.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
2.2 The Characteristic Polynomial . . . . . . . . . . . . . . . . . . . . . . . 3
2.3 The Eigenspace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3 Linear Independence of Eigenvectors 6

4 Diagonalization 6
4.1 Definition and the Key Equation . . . . . . . . . . . . . . . . . . . . . 7
4.2 The Procedure for Diagonalization . . . . . . . . . . . . . . . . . . . . 8
4.3 Geometric Interpretation: Change of Basis . . . . . . . . . . . . . . . . 10

5 Application I: Powers of a Matrix 10

6 Application II: The Matrix Exponential 12

6.1 Definition via Power Series . . . . . . . . . . . . . . . . . . . . . . . . . 12
6.2 Computing eAt via Diagonalization . . . . . . . . . . . . . . . . . . . . 13
6.3 Properties of the Matrix Exponential . . . . . . . . . . . . . . . . . . . 13
6.4 Solving Systems of ODEs via the Matrix Exponential . . . . . . . . . . 14

7 Application III: Large Powers and Markov Chains 15

7.1 Asymptotic Behaviour of An . . . . . . . . . . . . . . . . . . . . . . . . 15
7.2 Stochastic Matrices and Markov Chains . . . . . . . . . . . . . . . . . . 15

8 The Spectral Theorem for Real Symmetric Matrices 17

8.1 Statement and Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

9 The Cayley-Hamilton Theorem 19

1
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

10 Non-Diagonalizable Matrices: Jordan Normal Form (Overview) 20

11 Critical Thinking Test Questions 21

2
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

1. Motivation: Why Diagonalize?

A diagonal matrix is the simplest possible matrix: its action on any basis vector is
mere scaling. The central question of diagonalization is:

Given a matrix A, can we find a basis of the vector space

in which A acts by pure scaling — i.e., is diagonal?

Why does this matter?

ˆ Matrix powers: (P DP −1 )n = P Dn P −1 . Since D is diagonal, Dn = diag(λn1 , . . . , λnk )

is trivial.

ˆ Matrix exponential: eAt = P eDt P −1 . Solving coupled linear ODEs reduces to

independent scalar equations.

ˆ Long-time behaviour: The dominant eigenvalue controls An x as n → ∞ — the

basis of Markov chains, PageRank, and quantum mechanics.

ˆ Physics: In quantum mechanics, measuring an observable corresponds to diago-

nalizing a Hermitian operator (Spectral Theorem). Normal modes of oscillation
are eigenvectors of the dynamical matrix.

The plan: build from eigenvalues → eigenvectors → diagonalization theorem → appli-

cations, with full proofs at every step.

2. Eigenvalues and Eigenvectors

2.1. Definitions

Definition: Eigenvalue and Eigenvector

Let A be an n × n matrix over F (= R or C). A scalar λ ∈ F is an eigenvalue
of A if there exists a nonzero vector v ∈ Fn such that

Av = λv.

The vector v ̸= 0 is called an eigenvector of A corresponding to λ.

3
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

How to read Av = λv: The matrix A acts on v by pure scaling — no rotation, no

shearing, only stretching (or shrinking, or reflecting) by the factor λ.
The condition v ̸= 0 is essential: A0 = λ0 trivially for every λ, which would make the
definition meaningless.

2.2. The Characteristic Polynomial

Rearranging Av = λv: (A − λI)v = 0. This is a homogeneous linear system. It has a

nonzero solution if and only if the system matrix is singular, i.e.,:

det(A − λI) = 0.

Definition: Characteristic Polynomial

The characteristic polynomial of an n × n matrix A is

pA (λ) = det(λI − A),

a polynomial of degree n in λ. (We use det(λI − A) rather than det(A − λI) to

make the leading coefficient +1.)
The eigenvalues of A are precisely the roots of pA (λ) = 0.

Remark: Why det(λI − A) is a Polynomial of Degree n

Expanding det(λI − A) along any row, the diagonal entries contribute (λ − aii )
(degree 1 in λ) and off-diagonal products involve fewer than n diagonal terms.
The highest-degree term comes from the main diagonal: ni=1 (λ − aii ), giving
Q

leading term λn . By the Fundamental Theorem of Algebra, pA (λ) has exactly n

roots in C (counted with multiplicity).

General form of the characteristic polynomial:

pA (λ) = λn − (tr A)λn−1 + · · · + (−1)n det(A).

The coefficient of λn−1 is − tr(A) = −(a11 + · · · + ann ) and the constant term is
pA (0) = det(−A) = (−1)n det(A).

Theorem: Trace and Determinant via Eigenvalues

If λ1 , . . . , λn are the eigenvalues of A (counted with multiplicity):
n
X n
Y
tr(A) = λi , det(A) = λi .
i=1 i=1

4
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Proof. Factor the characteristic polynomial (over C): pA (λ) = ni=1 (λ − λi ). At λ = 0:

Q

det(−A) = (−1)n det(A) = ni=1 (−λi ) = (−1)n λi . Hence det(A) = λi .

Q Q Q

The coefficient of λn−1 in (λ − λi ) is − λi . Comparing with − tr(A) from the

Q P
P
expansion above gives tr(A) = λi .

2.3. The Eigenspace

Definition: Eigenspace
For each eigenvalue λ of A, the eigenspace is

Eλ = ker(A − λI) = {v ∈ Fn : Av = λv}.

It is a subspace of Fn (it contains 0 and is closed under addition and scalar

multiplication). The eigenvectors are the nonzero elements of Eλ .

Definition: Algebraic and Geometric Multiplicity

ˆ The algebraic multiplicity ma (λ) of an eigenvalue λ is its multiplicity as a
root of the characteristic polynomial pA (λ).

ˆ The geometric multiplicity mg (λ) is the dimension of the eigenspace Eλ :

mg (λ) = dim Eλ = dim ker(A − λI) = n − rank(A − λI).

Theorem: Inequality Between Multiplicities

For every eigenvalue λ of A:

1 ≤ mg (λ) ≤ ma (λ).

Proof. mg ≥ 1: since λ is an eigenvalue, there exists at least one eigenvector, so Eλ

has dimension ≥ 1.
mg ≤ ma : This is the non-trivial inequality. Let d = mg (λ) and choose an orthonormal
basis e1 , . . . , ed of Eλ . Extend to a basis of Rn : e1 , . . . , ed , f1 , . . . , fn−d . In this basis,
Aek = λek , so !
λId B
A=
0 C
in block form. Therefore
!
(µ − λ)Id −B
pA (µ) = det(µI − A) = det = (µ − λ)d · pC (µ).
0 µIn−d − C

5
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

So (µ − λ)d | pA (µ), meaning ma (λ) ≥ d = mg (λ).

Example: Computing Eigenvalues and Eigenspaces

!
4 1
Let A = .
2 3
Characteristic polynomial:

λ − 4 −1
pA (λ) = = (λ − 4)(λ − 3) − 2 = λ2 − 7λ + 10 = (λ − 5)(λ − 2).
−2 λ − 3

Eigenvalues: λ1 = 5, λ2 = 2. Check: tr = 4 + 3 = 7 = 5 + 2 ✓, det = 12 − 2 =

10 = 5 · 2 ✓.
Eigenspace for λ1 = 5: (A − 5I)v = 0:
! !
−1 1 v1
= 0 ⇒ v1 = v2 .
2 −2 v2
(!)
1
E5 = span . mg (5) = 1 = ma (5).
1
Eigenspace for λ2 = 2: (A − 2I)v = 0:
! !
2 1 v1
= 0 ⇒ 2v1 + v2 = 0.
2 1 v2
( !)
1
E2 = span . mg (2) = 1 = ma (2).
−2

Example: Defective Matrix: mg < ma

!
2 1
Let A = .
0 2
pA (λ) = (λ − 2)2 . !So λ = 2 with ma = 2.
0 1
(A − 2I) = . Null space: v2 = 0, v1 free.
0 0
( !)
1
E2 = span . mg (2) = 1 < ma (2) = 2.
0
This matrix is not diagonalizable (as we shall prove). It is called a defective
matrix.

6
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

3. Linear Independence of Eigenvectors

The following theorem is the analytical engine behind diagonalization.

Theorem: Eigenvectors from Distinct Eigenvalues are Linearly Independent

If λ1 , λ2 , . . . , λk are distinct eigenvalues of A with corresponding eigenvectors
v1 , v2 , . . . , vk , then v1 , . . . , vk are linearly independent.

Proof. By induction on k.
Base case k = 1: A single eigenvector v1 ̸= 0 is trivially linearly independent.
Inductive step: Assume any set of k − 1 eigenvectors from distinct eigenvalues is
linearly independent. Suppose

c1 v1 + c2 v2 + · · · + ck vk = 0. (∗)

Apply A to both sides:

c1 Av1 + c2 Av2 + · · · + ck Avk = 0,

c1 λ1 v1 + c2 λ2 v2 + · · · + ck λk vk = 0. (∗∗)

Subtract λk times (∗) from (∗∗):

c1 (λ1 − λk )v1 + c2 (λ2 − λk )v2 + · · · + ck−1 (λk−1 − λk )vk−1 = 0.

By the induction hypothesis, v1 , . . . , vk−1 are linearly independent. Therefore each

coefficient is zero: cj (λj − λk ) = 0 for j = 1, . . . , k − 1. Since eigenvalues are distinct,
λj ̸= λk , so cj = 0 for all j < k. From (∗): ck vk = 0, and since vk ̸= 0, ck = 0. Hence
all cj = 0, proving linear independence.

4. Diagonalization

7
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

4.1. Definition and the Key Equation

Definition: Diagonalizable Matrix

An n × n matrix A is diagonalizable if there exists an invertible matrix P and
a diagonal matrix D such that

A = P DP −1 ,

equivalently P −1 AP = D.

What are P and D? Write P = [v1 | v2 | · · · | vn ] with columns vi . Then

AP = A[v1 | · · · |vn ] = [Av1 | · · · |Avn ]. Also P D = [v1 | · · · |vn ] diag(λ1 , . . . , λn ) =
[λ1 v1 | · · · |λn vn ].
So AP = P D if and only if Avi = λi vi for each i.

Theorem: The Diagonalization Theorem

An n × n matrix A is diagonalizable if and only if A has n linearly independent
eigenvectors. When this holds:

P = [v1 | · · · |vn ], D = diag(λ1 , . . . , λn ),

where v1 , . . . , vn are linearly independent eigenvectors with corresponding eigen-

values λ1 , . . . , λn .

Proof. (⇒) If A = P DP −1 with P invertible, then AP = P D. The i-th column of this

equation gives Avi = λi vi , so the columns of P are eigenvectors. Since P is invertible,
its columns are linearly independent.
(⇐) If v1 , . . . , vn are linearly independent eigenvectors, form P = [v1 | · · · |vn ]. Then P
is invertible (columns are linearly independent). The equation Avi = λi vi for each i
packages as AP = P D, so A = P DP −1 .

Corollary: Sufficient Condition: Distinct Eigenvalues

If A has n distinct eigenvalues λ1 , . . . , λn , then A is diagonalizable. (The corre-
sponding eigenvectors are automatically linearly independent by the previous
theorem.)

Theorem: Necessary and Sufficient Condition via Multiplicities

A is diagonalizable if and only if for every eigenvalue λ:

mg (λ) = ma (λ).

8
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Proof. Let λ1 , . . . , λs be the distinct eigenvalues with ma (λi ) = ai and mg (λi ) = gi .

A is diagonalizable ⇐⇒ A has n linearly independent eigenvectors ⇐⇒ we can
assemble n linearly independent eigenvectors in total.
One can show (using the fact that eigenvectors from different eigenspaces are linearly
independent of each other) that the maximum number of linearly independent eigen-
vectors is si=1 gi . This equals n = si=1 ai (since the characteristic polynomial factors
P P

as (λ − λi )ai ) if and only if gi = ai for all i.

Q

4.2. The Procedure for Diagonalization

Algorithm:

1. Compute pA (λ) = det(λI − A).

2. Find the roots λ1 , . . . , λs (distinct eigenvalues) and their algebraic multiplicities

ma (λi ).

3. For each λi , solve (A − λi I)v = 0 to find a basis for Eλi ; record mg (λi ) = dim Eλi .

4. A is diagonalizable ⇐⇒ mg (λi ) = ma (λi ) for all i.

5. If yes, form P with eigenvector columns and D = diag(λ1 , . . . , λn ) (repeated

according to multiplicity).

Example: Full Diagonalization of a 3 × 3 Matrix

 
2 0 0
Let A =  1 2 1.
 

−1 0 1
Characteristic polynomial:

λ−2 0 0
pA (λ) = det(λI − A) = −1 λ − 2 −1 .
1 0 λ−1

λ − 2 −1
Expanding along the first row: = (λ − 2) = (λ − 2)[(λ − 2)(λ −
0 λ−1
1) − 0] = (λ − 2)2 (λ − 1).
Eigenvalues: λ1 = 2 with ma = 2; λ2 = 1 with ma = 1.

9
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Eigenspace for λ = 2: (A − 2I)v = 0:

 
0 0 0
A − 2I =  1 0 1  .
 

−1 0 −1

Row reduce: the only non-trivial equation is v1 + v3 = 0, so v1 = −v3 , v2 is free.

Setting (v3 , v2 ) = (1, 0) and (−1, 0) does not give independent solutions; rather
the free parameters are v2 and v3 : v1 = −v3 , v2 , v3 free. So:
   
0 −1
v = v2 1 + v3  0  .
   

0 1
    
 0
 −1  
E2 = span 1 ,  0  . mg (2) = 2 = ma (2). ✓
   
 
0 1
 
Eigenspace for λ = 1: (A − I)v = 0:
 
1 0 0
A − I =  1 1 1 .
 

−1 0 0

Row 1 gives v1 = 0; Row 2 with v1 = 0: v2 + v3 = 0; Row 3: −v1 = 0 (redundant).

So v1 = 0, v2 = −v3 :
  
 0 
 
E1 = span  1  . mg (1) = 1 = ma (1). ✓
 
 
−1
 

Diagonalizing matrices:
   
0 −1 0 2 0 0
P = 1 0 1 , D = 0 2 0 .
   

0 1 −1 0 0 1
     
0 0 0
Verification: AP , column 1: A 1 = 2 = 2 1. ✓
     

0 0 0

10
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

4.3. Geometric Interpretation: Change of Basis

The equation P −1 AP = D is a similarity transformation: it expresses A in the

basis of eigenvectors.
Interpretation: Any vector x can be written in the eigenbasis: x = c1 v1 +· · ·+cn vn =
P c, where c = P −1 x. In the eigenbasis:

λ1 c1
 . 
Ax = A(P c) = P (Dc) = P  ..  .
λn cn

So A acts on coordinates in the eigenbasis by simply multiplying each coordinate ci by

λi . In the eigenbasis, A is diagonal.

Remark: Analogy with Rotating Frames in Physics

In classical mechanics, a rotation matrix R looks complicated in Cartesian
coordinates but becomes diagonal (with entries ±1 or complex phases) when
expressed in its own eigenbasis. Diagonalization is the general algebraic version
of this “choosing the right frame” philosophy. In quantum mechanics, this is
exactly what “choosing a basis of eigenstates of the Hamiltonian” means: in that
basis, time evolution is trivial (each component just picks up a phase e−iEn t/ℏ ).

5. Application I: Powers of a Matrix

Theorem: Matrix Powers via Diagonalization

If A = P DP −1 is diagonalizable, then for any integer n ≥ 0:

An = P Dn P −1 , Dn = diag(λn1 , . . . , λnk ).

Proof. By induction. The base case n = 1 is A = P DP −1 . For n ≥ 2: An = A · An−1 =

(P DP −1 )(P Dn−1 P −1 ) = P D(P −1 P )Dn−1 P −1 = P DDn−1 P −1 = P Dn P −1 .
For Dn : (Dn )ii = λni (since powers of diagonal matrices just raise each diagonal entry to
the given power, as all off-diagonal entries are zero and there is no cross-term).

11
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Remark: Why is Dn = diag(λni )?

By direct computation: D2 = diag(λi ) · diag(λi ) = diag(λ2i ), since (D2 )ij =
2 2
P P
k Dik Dkj = Dii Dij δij ... more precisely, (D )ij = k λi δik · λj δkj = λi δij .

Example: Computing A10 for a 2 × 2 Matrix

!
4 1
Let A = (from Example 2.1, with λ1 = 5, λ2 = 2).
2 3
! !
1 1
Eigenvectors: v1 = , v2 = .
1 −2
! !
1 1 5 0
P = , D= .
1 −2 0 2
! !
−2
1 −1 2/3 1/3
det(P ) = −2 − 1 = −3. P −1 = = .
−3
−1 1 1/3 −1/3
! ! !
1 1 5 0 2/3 1/3
Verify: P DP −1 = =
1 −2 0 2 1/3 −1/3
! ! ! !
5 2 2/3 1/3 10/3 + 2/3 5/3 − 2/3 4 1
= = . ✓
5 −4 1/3 −1/3 10/3 − 4/3 5/3 + 4/3 2 3
Power:
! ! ! !
10
5 0 1 1 9765625 0 2/3 1/3
A10 = P 10
P −1 = .
0 2 1 −2 0 1024 1/3 −1/3
!
9765625 1024
Computing P D10 : = .
9765625 −2048
! ! !
2·9765625+1024 9765625−1024
9765625 1024 2/3 1/3
Then A10 = = 3
2·9765625−2048
3
9765625+2048
9765625 −2048 1/3 −1/3 3 3
!
19532274 9764601
3 3
= 19529202 9767673
.
3 3 ! !
1 1 1 0
Observe: For large n, An ≈ 5n · P −1 = 5n · v1 (P −1 )1· . The
1 −2 0 0
dominant eigenvalue λ1 = 5 controls the asymptotic behaviour.

Example: Fibonacci Sequence via Diagonalization

The Fibonacci recurrence Fn+1 = Fn + Fn−1 can be written as the matrix

12
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

recurrence ! ! !
Fn+1 1 1 Fn
= .
Fn 1 0 Fn−1
| {z }
A
! !
Fn+1 1
So = An (with F1 = 1, F0 = 0).
Fn 0
√
2 1± 5 √
Eigenvalues of A: λ − λ − 1 = 0, so λ1,2 = . Let ϕ = (1 + 5)/2 (the
√ 2
− 5)/2.
golden ratio) and ϕ̂ = (1 ! !
ϕ ϕ̂
Eigenvectors: v1 = (for λ1 = ϕ), v2 = (for λ2 = ϕ̂).
1 1
!
1
Using An = P Dn P −1 and extracting the (1, 1) entry of An :
0
" √ !n √ !n #
ϕn − ϕ̂n 1 1+ 5 1− 5
Fn = √ =√ − .
5 5 2 2
√
This is Binet’s formula. Since |ϕ̂| < 1, ϕ̂n → 0, so Fn ≈ ϕn / 5 for large n,
and Fn+1 /Fn → ϕ.

6. Application II: The Matrix Exponential

6.1. Definition via Power Series

Definition: Matrix Exponential

For an n × n matrix A, the matrix exponential is defined by the absolutely
convergent power series
∞
A
X Ak A2 A3
e = =I +A+ + + ···
k=0
k! 2! 3!

Theorem: Absolute Convergence of eA

The series ∞ k
P
k=0 A /k! converges absolutely for every matrix A.

13
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Proof. Using the sub-multiplicative norm ∥Ak ∥ ≤ ∥A∥k :

∞ ∞
X ∥Ak ∥ X ∥A∥k
≤ = e∥A∥ < ∞.
k=0
k! k=0
k!

Since the scalar series e∥A∥ converges, the matrix series converges absolutely (entry by
entry).

6.2. Computing eAt via Diagonalization

Theorem: Matrix Exponential via Diagonalization

If A = P DP −1 is diagonalizable with D = diag(λ1 , . . . , λn ), then

eAt = P eDt P −1 , eDt = diag(eλ1 t , . . . , eλn t ).

Proof. Substitute the power series definition:

∞ ∞ ∞ ∞
!
X (At)k X Ak tk X (P Dk P −1 )tk X Dk tk
eAt = = = =P P −1 = P eDt P −1 .
k=0
k! k=0
k! k=0
k! k=0
k!
P∞ P∞
For the diagonal matrix: (eDt )ij = k=0 (D
k k
t /k!)ij = k=0 λki tk /k! · δij = eλi t δij .

6.3. Properties of the Matrix Exponential

Proposition: Key Properties of eAt

1. eA·0 = I.
d At
2. e = AeAt = eAt A.
dt
d P Ak tk P k tk−1 P∞ Aj tj
Proof: k = kA = A j=0 = AeAt .
dt k! (k − 1)! j!
3. If AB = BA (matrices commute), then eA+B = eA eB .
Proof: When AB = BA, the binomial theorem P applies: + B)n =
 P (Ak 
Pn n j
n
. Then eA+B = n (A+B) A B

k n−k
= eA eB
P
k=0 k A B n!
= j j! k k!
(by the Cauchy product, valid when series converge absolutely).

4. (eAt )−1 = e−At .

Proof: From property 3 with B = −At: eAt e−At = e0 = I.

14
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

5. If A has eigenvalues λi , then eAt has eigenvalues eλi t with the same eigenvec-
tors.

6.4. Solving Systems of ODEs via the Matrix Exponential

The most powerful application of eAt is solving the coupled system

ẋ(t) = Ax(t), x(0) = x0 .

Theorem: Solution of the Linear ODE System

The unique solution of ẋ = Ax, x(0) = x0 is

x(t) = eAt x0 .

Proof. Existence: Set x(t) = eAt x0 . By property 2: ẋ(t) = AeAt x0 = Ax(t) ✓. At

t = 0: x(0) = e0 x0 = Ix0 = x0 ✓.
Uniqueness: If y(t) is another solution, let z(t) = e−At y(t). Then ż = −Ae−At y +
e−At Ay = 0 (using e−At A = Ae−At ). So z = const = z(0) = y(0) = x0 , giving
y(t) = eAt x0 .

Explicit formula via diagonalization: If A = P DP −1 , write c = P −1 x0 (coordinates

in eigenbasis). Then

eλ1 t c1
 
 . 
x(t) = P eDt c = P  ..  = c1 eλ1 t v1 + · · · + cn eλn t vn .
eλn t cn

Each eigenvector component evolves independently, multiplied by its own exponential

factor.

Example: Solving a Coupled ODE System

! !
3 1 2
Solve ẋ = Ax with A = , x(0) = .
1 3 0
Eigenvalues: pA (λ) = (λ − 3)2 − 1 = 0, so λ = 4!and λ = 2. !
−1 1 1
Eigenvectors: λ = 4: (A − 4I)v = 0: v = 0, so v1 = . λ = 2:
1 −1 1
! !
1 1 1
(A − 2I)v = 0: v = 0, so v2 = .
1 1 −1

15
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

! ! !
1 1 2
Decompose initial condition: x0 = c1 v1 + c2 v2 : c1 + c2 = .
1 −1 0
c1 + c2 = 2, c1 − c2 = 0, so c1 = 1, c2 = 1.
Solution: ! ! !
1 1 e4t + e2t
x(t) = e4t + e2t = .
1 −1 e4t − e2t

Verify: ẋ1 = 4e4t + 2e2t . 3x1 + x2 = 3(e4t + e2t ) + (e4t − e2t ) = 4e4t + 2e2t . ✓

7. Application III: Large Powers and Markov Chains

7.1. Asymptotic Behaviour of An

Theorem: Dominance of the Largest Eigenvalue

If A = P DP −1 and the eigenvalues satisfy |λ1 | > |λ2 | ≥ · · · ≥ |λn | (i.e., λ1 is
strictly dominant), then for any initial vector x0 with c1 = (P −1 x0 )1 ̸= 0:
" n  n #
X λj n→∞
An x0 = λn1 c1 v1 + cj vj −−−→ c1 λn1 v1 (direction converges to v1 ).
j=2
λ1

Proof. An x0 = P Dn P −1 x0 = nj=1 cj λnj vj = λn1 j cj (λj /λ1 )n vj . Since |λj /λ1 | < 1
P P

for j ≥ 2, each term (λj /λ1 )n → 0, leaving c1 λn1 v1 .

7.2. Stochastic Matrices and Markov Chains

Definition: Stochastic (Markov) Matrix

An n × n matrix A is (column) stochastic if all entries are non-negative and
each column sums to 1:
n
X
Aij ≥ 0 for all i, j, Aij = 1 for all j.
i=1

A Markov chain models a random process: Aij is the probability of transitioning

from state j to state i.

16
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Theorem: Steady-State Theorem for Regular Stochastic Matrices

For a regular stochastic matrix A (some power Ak has all positive entries):

1. λ = 1 is an eigenvalue of A.

2. All other eigenvalues satisfy |λj | < 1.

3. limn→∞ An = q1T (each column converges to the unique stationary distribu-

tion q, the normalised eigenvector for λ = 1).

Proof. Part 1: Let 1 = (1, . . . , 1)T . Then 1T A = 1T (column sums are 1), so AT 1 = 1,
meaning 1 is an eigenvector of AT with λ = 1. Hence det(A − I) = det(AT − I) = 0,
so λ = 1 is an eigenvalue of A.
Part 2: Suppose Av = λv with |λ| ≥ 1. Let vk = maxi |vi |. Then |λ||vk | = |(λv)k | =
P P P
|(Av)k | = | j Akj vj | ≤ j Akj |vj | ≤ |vk | j Akj = |vk |. Hence |λ| ≤ 1. The strict
inequality |λ| < 1 for λ ̸= 1 follows from the regularity condition (details require the
Perron-Frobenius theorem).
Part 3: By the dominant eigenvalue theorem, An x0 → c1 q for any x0 with 1T x0 = 1
(probability vector). Normalising, each column of An converges to q.

Example: Markov Chain: Two-State Weather Model

States: Sunny (S) and Rainy (R). Transition probabilities:
!
0.8 0.4
A= .
0.2 0.6

ASS = 0.8: probability sunny → sunny. ARS = 0.2: probability sunny → rainy.
Etc.
Eigenvalues: pA (λ) = (λ−0.8)(λ−0.6)−0.08 = λ2 −1.4λ+0.4 = (λ−1)(λ−0.4).
λ1 = 1, λ2 = 0.4. !
−0.2 0.4
Steady state (λ1 = 1): (A − I)q = 0: q = 0, so qR = qS /2.
0.2 −0.4
Normalising
! (qS + qR = 1): qS = 2/3, qR = 1/3.
2/3
q= .
1/3
!
2/3 2/3
Long-run prediction: limn→∞ An = .
1/3 1/3
Regardless of today’s weather, after many days the probability of sunshine is 2/3.
Rate of convergence: |λ2 | = 0.4, so after n steps the error in any probability
is ∼ 0.4n — geometric decay. After 5 steps: 0.45 ≈ 0.01.

17
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

8. The Spectral Theorem for Real Symmetric Ma-

trices

8.1. Statement and Proof

Real symmetric matrices are the most important class in physics (they arise as Hamil-
tonians, inertia tensors, stress tensors, covariance matrices, and more).

Theorem: Spectral Theorem

Let A be a real symmetric matrix (A = AT , i.e., Aij = Aji ). Then:

1. All eigenvalues of A are real.

2. Eigenvectors corresponding to distinct eigenvalues are orthogonal.

3. A is orthogonally diagonalizable: there exists an orthogonal matrix Q

(QT = Q−1 ) such that A = QDQT .

Proof. Part 1: Eigenvalues are real. Let Av = λv with v ̸= 0 (possibly complex).

Take the conjugate transpose of both sides: v∗ A∗ = λ̄v∗ . Since A is real, A∗ = AT = A.
Multiply Av = λv on the left by v∗ :

v∗ Av = λv∗ v = λ∥v∥2 .

Multiply the conjugate equation v∗ A = λ̄v∗ on the right by v:

v∗ Av = λ̄v∗ v = λ̄∥v∥2 .

Both sides are equal: λ∥v∥2 = λ̄∥v∥2 . Since ∥v∥2 > 0, we get λ = λ̄, i.e., λ ∈ R.
Part 2: Eigenvectors from distinct eigenvalues are orthogonal. Let Au = λu
and Av = µv with λ ̸= µ (all real).

λ u · v = (Au) · v = uT AT v = uT Av = u · (Av) = µ u · v.

(We used A = AT in the middle step.) So (λ − µ)u · v = 0. Since λ =

̸ µ: u · v = 0.
Part 3: Orthogonal diagonalizability. The complete proof uses induction on n: find
one real eigenvector v1 (guaranteed by Part 1), restrict A to the orthogonal complement
v1⊥ , show the restriction is again symmetric (using A = AT ), and apply induction. By
Parts 1 and 2, we obtain an orthonormal basis of eigenvectors, which we assemble as

18
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

columns of Q. Then QT Q = I (columns orthonormal), and A = QDQT .

Example: Orthogonal Diagonalization of a Symmetric Matrix

!
2 1
Let A = .
1 2
Eigenvalues: pA (λ) = (λ − 2)2 − 1 = (λ − 3)(λ − 1). λ1 = 3, λ2 = 1.!
1
Eigenvectors (orthogonal, by Spectral Theorem): λ1 = 3: v1 = . λ2 = 1:
1
!
1
v2 = . Check: v1 · v2 = 1 · 1 + 1 · (−1) = 0. ✓
−1
! !
1 1
Normalise: q1 = √12 , q2 = √12 .
1 −1
! !
1 1 1 3 0
Q= √ , D= , A = QDQT .
2 1 −1 0 1
! ! !
1 1 3 0 1 1
Since QT = Q−1 , verify: QDQT = 12 =
1 −1 0 1 1 −1
! ! ! !
1 3 1 1 1 4 2 2 1
2
= 12 = . ✓
3 −1 1 −1 2 4 1 2

Remark: Physical Application: Principal Axes

The spectral theorem says every real symmetric matrix can be diagonalized by
a pure rotation (orthogonal transformation). In mechanics, this is the theorem
that guarantees principal axes of the inertia tensor exist: there is always a
reference frame in which the inertia tensor is diagonal. The eigenvalues are the
principal moments of inertia.
In elasticity, the stress tensor is symmetric, and its eigenvectors are the principal
stress directions — axes along which there is pure tension or compression with
no shear.

19
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

9. The Cayley-Hamilton Theorem

Theorem: Cayley-Hamilton Theorem

Every matrix satisfies its own characteristic equation: if pA (λ) = λn + cn−1 λn−1 +
· · · + c1 λ + c0 , then

pA (A) = An + cn−1 An−1 + · · · + c1 A + c0 I = 0.

Proof for Diagonalizable A. If A = P DP −1 , then Ak = P Dk P −1 , so

!
X X
pA (A) = ck Ak = P ck D k P −1 = P pA (D) P −1 .
k k

Now pA (D) = diag(pA (λ1 ), . . . , pA (λn )). Since each λi is a root of pA : pA (λi ) = 0 for
all i. Hence pA (D) = 0, and pA (A) = P · 0 · P −1 = 0.
(For non-diagonalizable matrices, the proof uses the Jordan normal form or the adjugate
matrix; the result holds in full generality.)

Corollary: Inverse via Cayley-Hamilton

For an invertible n × n matrix A (i.e., c0 = (−1)n det A ̸= 0):

1 n−1
A−1 = − (A + cn−1 An−2 + · · · + c1 I).
c0

Proof. From pA (A) = 0: An + cn−1 An−1 + · · · + c1 A + c0 I = 0. Multiply on the right

by A−1 : An−1 + cn−1 An−2 + · · · + c1 I + c0 A−1 = 0. Hence A−1 = −c−10 (A
n−1
+ ··· +
c1 I).

Example: Cayley-Hamilton: Expressing A3 in Terms of Lower Powers

!
4 1
For A = with pA (λ) = λ2 − 7λ + 10:
2 3
By Cayley-Hamilton: A2 − 7A + 10I = 0, so A2 = 7A − 10I.
Therefore A3 = A · A2 = A(7A − 10I) = 7A2 − 10A = 7(7A − 10I) − 10A =
39A − 70I. ! ! ! !
4 1 1 0 156 − 70 39 86 39
A3 = 39 − 70 = = .
2 3 0 1 78 117 − 70 78 47

20
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

! ! !
4 4 1
1 18 7
Direct check: A2 = = . A3 = A2 · A =
2 3
2 3 14 11
! ! ! !
18 7 4 1 72 + 14 18 + 21 86 39
= = . ✓
14 11 2 3 56 + 22 14 + 33 78 47

10. Non-Diagonalizable Matrices: Jordan Normal

Form (Overview)

When mg (λ) < ma (λ) for some eigenvalue, A cannot be diagonalized. The closest we
can get is the Jordan normal form.

Definition: Jordan Block and Jordan Normal Form

A Jordan block of size k for eigenvalue λ is the k × k matrix
 
λ 1 0 ··· 0
0 λ 1 ··· 0
 
. .. .. .. 
Jk (λ) =  .
. . . ..
0 ···
 
0 λ 1
0 0 ··· 0 λ

Every matrix A over C is similar to a block-diagonal Jordan normal form J =

diag(Jk1 (λi1 ), . . . , Jkr (λir )): A = P JP −1 , with P (possibly complex) invertible.
Diagonal matrices are the special case where all Jordan blocks are 1 × 1.

Theorem: Powers of a Jordan Block

For a Jordan block Jk (λ):

n n n




λn 1
λn−1 2

λn−2 ··· k−1

λn−k+1
n n
0 λn λn−1 ··· λn−k+2 
 
1 k−2
n
. .. .. 
Jk (λ) =  . . .
. . 
n

n−1
··· λn
 
0 0 1
λ 
0 0 ··· 0 λn

Proof. Write Jk (λ) = λI + N where N is the nilpotent matrix with 1’s on the su-
perdiagonal and 0’s elsewhere. Since λI and N commute: Jk (λ)n = (λI + N )n =
Pk−1 n
n−j j
j=0 j λ N (the sum terminates at j = k − 1 since N k = 0). The (i, j) entry of
N j is δi+j,j (1 on the j-th superdiagonal), giving the stated formula.

21
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Remark: What Jordan Form Reveals

For a Jordan block with |λ| < 1, the entries of J n grow as nk−1 |λ|n . Even though
|λ| < 1 ensures eventual decay to 0, the polynomial factor nk−1 means a defective
system can exhibit transient growth before decaying. This is important in fluid
mechanics and control theory.

11. Critical Thinking Test Questions

10 Critical Thinking Questions

Q1. (Characteristic Polynomial and Invariants) Let A be a 3 × 3 matrix
with characteristic polynomial pA (λ) = λ3 − 6λ2 + 11λ − 6.

(a) Find the eigenvalues.

(b) Compute tr(A), det(A), and the sum of all 2 × 2 principal minors
directly from the polynomial coefficients (without knowing A).
(c) If the eigenvalues are λ1 = 1, λ2 = 2, λ3 = 3, write down the most
general A that has this characteristic polynomial (using the eigenvector
freedom). How many degrees of freedom remain?
(d) Apply Cayley-Hamilton to express A4 in terms of A2 , A, and I.
 
0 1 0
Q2. (Diagonalization: Rigorous Check) Given A = 0 0 1 (com-
 

6 −11 6
panion matrix):

(a) Show that pA (λ) = λ3 − 6λ2 + 11λ − 6.

(b) Find all eigenvalues. Are all algebraic and geometric multiplicities
equal?
(c) Explicitly construct P and D, and compute P −1 directly.
(d) Verify that P −1 AP = D by direct matrix multiplication.
(e) Use this to compute A100 (express in terms of powers of eigenvalues).

Q3. (Matrix
! Exponential: Non-Diagonal Case) For the matrix A =
1 1
(a Jordan block J2 (1)):
0 1

(a) Show that A is not diagonalizable.

22
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

!
0 1
(b) Write A = I + N where N = and show N 2 = 0.
0 0
!
et tet
(c) Use the power series definition to show eAt = .
0 et
d At
(d) Show that dt
e= AeAt , verifying Property 2 of the matrix exponential.
!
1
(e) Solve the system ẋ = Ax, x(0) = .
1
 
3 1 1
Q4. (The Spectral Theorem: Applications) Let A = 1 3 1 (a circu-
 

1 1 3
lant matrix).

(a) Show that v1 = (1, 1, 1)T is an eigenvector. Find its eigenvalue.

(b) Show that v2 = (1, −1, 0)T and v3 = (1, 0, −1)T are eigenvectors. Find
their eigenvalues.
(c) Orthogonalize v2 and v3 using Gram-Schmidt (they share the same
eigenvalue — why are they not automatically orthogonal?).
(d) Write A = QDQT explicitly.
(e) Compute A−1 and eAt using the orthogonal diagonalization.

Q5. (Markov Chain:

 Long-Run
 Behaviour) A Markov chain has transition
0.7 0.2 0.1
matrix A = 0.1 0.6 0.2.
 

0.2 0.2 0.7

(a) Show that λ = 1 is an eigenvalue by showing 1T A = 1T .

(b) Find the stationary distribution q (normalised eigenvector for λ = 1).
(c) Given initial state distribution x0 = (1, 0, 0)T (starting in state 1 with
certainty), use the dominant eigenvalue to predict the long-run state
distribution. After how many steps is the error < 0.01?
(d) Interpret the stationary distribution physically: what fraction of time
does the chain spend in each state in the long run?

Q6. (Cayley-Hamilton: Beyond 2 × 2)

!
1 2
(a) For the matrix A = , use Cayley-Hamilton to compute A−1
3 2
without row reduction.

23
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

(b) For a general 2 × 2 matrix, derive the formula A−1 = 1

det A
((tr A)I − A)
from Cayley-Hamilton.
(c) For a 3 × 3 matrix, Cayley-Hamilton says A3 = c2 A2 + c1 A + c0 I. Use
this to express An for any n ≥ 3 as a linear combination of A2 , A, and
I (find the recurrence for the coefficients).

Q7. (Simultaneous Diagonalization) Two matrices A and B are simultane-

ously diagonalizable if there exists a single invertible P such that both
P −1 AP and P −1 BP are diagonal.

(a) Prove that if A and B are simultaneously diagonalizable, then AB =

BA (they commute).
(b) Prove the converse for diagonalizable matrices: if A and B are both
diagonalizable and AB = BA, then they are simultaneously diago-
nalizable. Hint: Show that each eigenspace of A is invariant under
B.
(c) Give an example of two non-diagonalizable matrices that commute but
are not simultaneously diagonalizable.

Q8. (Quadratic Forms and the Spectral Theorem) A quadratic form is

Q(x) = xT Ax for symmetric A.

(a) Show that Q(x) = ni=1 λi yi2 where y = QT x (change to eigenbasis),

P

giving the principal axis theorem.

(b) The quadratic form is positive definite (Q(x) > 0 for x ̸= 0) if and
only if all eigenvalues are positive. Prove this.
(c) Classify the conic 5x2 + 4xy + 5y 2 = 36 by diagonalizing its associated
matrix. Find the principal axes and the semi-axis lengths.

Q9. (Matrix Functions via Diagonalization) For a diagonalizable matrix

A = P DP −1 and any analytic function f , define f (A) = P f (D)P −1 where
f (D) = diag(f (λi )).
!
0 π
(a) Using this definition, compute sin(A) for A = where sin(D) =
π 0
diag(sin λi ).
(b) Show that (cos A)2 + (sin A)2 = I for any diagonalizable A with real
eigenvalues.
(c) Show that f (A)vi = f (λi )vi for each eigenvector, so f (A) has the same
eigenvectors as A with eigenvalues f (λi ).

24
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

!
√ 5 4
(d) Use this to solve: what is A (a “square root” of A) for A = ?
1 2
(Find a matrix B with B 2 = A.)

Q10. (Defective Matrices

! and Stability) Consider the ODE system ẋ = Ax
−1 1
where A = (a Jordan block J2 (−1)).
0 −1

(a) Show that A is not diagonalizable. Use A = −I + N with N 2 = 0 to

compute eAt directly.
(b) Show that ∥eAt x0 ∥ → 0 as t → ∞ for every x0 , so the origin is
asymptotically stable.
(c) Observe that ∥eAt ∥ ∼ t e−t for large t: the solution initially grows
(transient growth) before decaying. Find the time t∗ at which ∥eAt x0 ∥
is maximised for x0 = (0, 1)T .
(d) Contrast this with the diagonal case A = diag(−1, −1): why is there
no transient growth in that case?

25
Part 6 — Matrix Diagonalization & Applications Mathematical Methods

Summary of Key Results

Eigenvalues & Eigenvectors: Av = λv, v ̸= 0. Eigenvalues = roots of

pA (λ) = det(λI − A).
P Q
tr A = λi , det A = λi . 1 ≤ mg (λ) ≤ ma (λ).

Diagonalization: A = P DP −1 ⇔ n lin. indep. eigenvectors ⇔ mg (λ) = ma (λ)

for all λ.
Distinct eigenvalues ⇒ automatically diagonalizable.

Powers & Exponential: An = P Dn P −1 , Dn = diag(λni ).

eAt = P eDt P −1 , eDt = diag(eλi t ). System ẋ = Ax, x(0) = x0 ⇒ x(t) = eAt x0 .

Spectral Theorem (real symmetric A = AT ): All λi ∈ R; eigenvectors from

√
distinct λ’s orthogonal; A = QDQT , Q orthogonal. Binet: Fn = (ϕn − ϕ̂n )/ 5.

Cayley-Hamilton: pA (A) = 0. Jordan Form: Jk (λ)n : entries nj λn−j on j-th

superdiagonal. Markov: Regular stochastic A has λ = 1 dominant; An → q1T .

End of Part 6 — Matrices: Diagonalization & Applications

26