Eigen Vectors, Eigen Values, Diagonalization

Eigen Vectors
When a matrix A acts on a vector v:

26
• Length may change

20
• Direction may change

al
op
Bh
Question: Are there vectors whose direction does not change?
IT
Yes! These are called Eigenvectors.
II
II-

Eigen Vectors
ics
at

Let A be an n × n matrix. A non-zero vector v is called an eigenvector of

m
he

A if
at
M

Av = λv
g
rin

for some scalar λ.

ee
in
ng

• v ̸= 0
:E

• λ is called the eigenvalue

HI

• A must be square

Understanding the Equation

Av = λv
This means:
• The vector keeps its direction

• Only scaling happens

Geometric meaning:

1
 • λ > 1 : Stretching
• 0 < λ < 1 : Shrinking
• λ < 0 : Direction reversal
• λ = 0 : Collapses to zero
To find λ, we rearrange Av = λv:
(A − λI)v = 0
For a non-trivial solution (v ̸= 0), the matrix (A − λI) must be singular.
Example 1. Characteristic Equation
|A − λI| = 0

26
20
• The expansion of |A − λI| is called the Characteristic Polynomial.

al
• The roots of this polynomial are the Eigenvalues.

op
Bh
Example 2. Find the eigenvalues and vectors of IT
!
II
2 0
A=
II-

0 3
ics

Solution: Characteristic equation

at
m

det(A − λI) = 0
he
at

(2 − λ)(3 − λ) = 0
M

λ = 2, 3.
g
rin

For λ = 2: find non-zero v such that

ee
in

(A − 2I)v = 0
ng

! ! !
:E

0 0 v1 0
=⇒ =
0 1 v2 0
HI

=⇒ v2 = 0, v1 = k.
!
1
So, eigen vectors v = k .
0
Final vector (LI) !
1
v=
0
For λ = 3: find non-zero v such that det(A − λI) = 0.
The final eigen vector (LI) is !
0
v=
1

2
Example 3. Find the eigenvalues and vectors of

!
4 1
A=
2 3
Solution: Char. equation is

det(A − λI) = (4 − λ)(3 − λ) − 2 = 0

=⇒ λ2 − 7λ + 10 = 0
=⇒ (λ − 5)(λ − 2) = 0
=⇒ λ = 5, 2

26
Important Properties

20
al
• Sum of eigenvalues = Trace of matrix

op
Bh
λ1 + · · · + λn = tr(A)
IT
• Product of eigenvalues = Determinant
II
II-
ics

λ1 · · · λn = det(A)
at
m

• Diagonal matrix ⇒ diagonal entries are eigenvalues

he
at

• Triangular matrix ⇒ diagonal entries are eigenvalues

M
g
rin

Eigenspace
ee
in

Let λ be an eigenvalue of A. The eigenspace corresponding to λ is defined

ng

as
:E
HI

Eλ = {v ∈ Rn : Av = λv} ∪ {0}
Equivalently,
Eλ = ker(A − λI).
Important:

• Eigenspace is a subspace of Rn

• It contains the zero vector

• All eigenvectors corresponding to λ lie in Eλ

3
Algebraic Multiplicity (AM)
Let λ be an eigenvalue of A. The algebraic Multiplicity (AM) of λ is:
The number of times λ appears as a root of the characteristic equation.

Example 4. If det(A − λI) = (λ − 2)3 (λ − 5). Then

AM (2) = 3, AM (5) = 1

Geometric Multiplicity (GM)

Let λ be an eigenvalue of A. Geometric Multiplicity (GM) of λ is:

GM (λ) = dim(Eλ )

26
20
That is,

al
GM (λ) = dimension of ker(A − λI)

op
Important relation:

Bh
1 ≤ GM (λ) ≤ AM (λ). IT
II
Example 5. Consider
II-

 
4 1 0
ics

A = 1 4 0
 
at

0 0 2
m
he

• We compute AM, GM and eigenspaces

at
M
g

Solution: Characteristic Polynomial is

rin
ee

4−λ 1 0
in

det(A − λI) = 4−λ

ng

1 0
:E

0 0 2−λ
HI

4−λ 1
= (2 − λ)
1 4−λ

= (2 − λ) (4 − λ)2 − 1

= (2 − λ)(3 − λ)(5 − λ)
Eigenvalues are
λ = 2, 3, 5
Algebraic multiplicities:

AM (2) = 1, AM (3) = 1, AM (5) = 1

4
Each eigenvalue appears once.
Eigenspace for λ = 2  
2 1 0
A − 2I = 1 2 0
 
0 0 0
Solve:
2x + y = 0, x + 2y = 0

x = 0, y = 0, z free
    
0  0 
 

26
E2 = k 0 = span 0
   

20
1  1 
 

al
op
So,

Bh
dim E2 = GM (2) = 1
IT
Eigenspace for λ = 3  
II

1 1 0
II-

A − 3I = 1 1 0 
 
ics

0 0 −1
at
m

Solve:
he

x + y = 0, z=0
at
M

y = −x
g
rin
ee

    
in

1  1 

ng


E3 = x −1 = span −1
   
:E

0  0 
 
HI

So,
dim E3 = GM (3) = 1
Eigenspace for λ = 5
 
−1 1 0
A − 5I =  1 −1 0 
 
0 0 −3

Solve
−x + y = 0, z=0

y=x

5
     
1  1 
 
E5 = k 1 = span 1
   
0  0 
 

So,
dim E5 = GM (5) = 1

Exercise 1. Let !
4 1
A=
0 4

1. Find eigenvalues.

26
20
2. Find Algebraic Multiplicity (AM).

l
pa
3. Find eigenspace of each eigenvalue.

o
Bh
4. Find Geometric Multiplicity (GM).
T
II
Exercise 2. Let
-I
!
2 1
II

A=
1 2
s
ic
at
m

1. Find eigenvalues and eigenvectors.

he
at

2. Find AM and GM.

M

3. Find basis of each eigenspace.

g
in
r

Exercise 3. Let
ee

!
n

3 −1
gi

A=
1 1
En
I:

1. Find the characteristic equation.

H

2. Determine eigenvalues (real/complex).

3. Find eigenspaces.

4. Compute AM and GM.

Exercise 4. Let  
2 1 0
A = 0 2 0
 
0 0 3

1. Find eigenvalues and AM.

6
 2. Find eigenspace for each eigenvalue.
3. Find GM.
Exercise 5. Let  
4 1 −1
A= 1 4 −1
 
−1 −1 4

1. Find eigenvalues.
2. Find AM and GM.
3. Find basis of each eigenspace.

26
Exercise 6. Let  

20
1 1 0

al
A = 0 1 1
 

op
0 0 1

Bh
1. Find eigenvalues.
IT
II

2. Find AM.
II-

3. Find eigenspace and GM.

ics
at

4. Compare AM and GM.

m
he
at

Cayley–Hamilton Theorem
M
g
rin

For a square matrix A, the characteristic polynomial is

ee
in

p(λ) = det(A − λI)

ng
:E

Question:
HI

Can we substitute A into its own characteristic polynomial?

Yes!
This is the Cayley–Hamilton Theorem.

Characteristic Polynomial
Let A be an n × n matrix.
p(λ) = det(A − λI)
It is a polynomial of degree n:
p(λ) = λn + an−1 λn−1 + · · · + a1 λ + a0

7
Cayley–Hamilton Theorem
Every square matrix satisfies its own characteristic equation. If

p(λ) = λn + an−1 λn−1 + · · · + a0

then
p(A) = An + an−1 An−1 + · · · + a0 I = 0
where 0 is the zero matrix.
Let !
2 1
A= .
1 2
The characteristic polynomial is

26
20
p(λ) = det(A − λI) = (2 − λ)2 − 1 = λ2 − 4λ + 3.

al
op
Substitute A:

Bh
p(A) = A2 − 4A + 3I IT
Compute
II
!
II-

5 4
A2 =
ics

4 5
at

Then
m
he

A2 − 4A + 3I = 0
at

Hence verified.
M
g
rin

Important Consequences
ee
in

1. Power Reduction
ng

Any higher power Ak (k > n) can be reduced using CH theorem.

:E

2. Formula for Inverse

HI

If A is invertible:
1
A−1 = − An−1 + an−1 An−2 + · · · + a1 I


a0
Example 6. Let !
2 1
A= .
1 1

The Characteristic polynomial is

λ2 − 3λ + 1

So

8
 A2 − 3A + I = 0
Rearranging gives:

A−1 = 3I − A

Exercise 7. 1. Verify CH theorem for

!
3 1
A=
0 2

2. Use CH theorem to find A−1 for

26
!

20
1 2
A=

al
3 4

op
Bh
Exercise 8. 1. Verify CH theorem IT
 
II
2 0 0
II-

A = 0 3 0
 
ics

0 0 4
at
m

2. Using CH theorem, find A5 for

he
at
M

!
0 1
g

A=
rin

−2 3
ee
in

Diagonalization
ng
:E

Diagonal matrices are easy to compute with.

HI

If  
λ1 0 0
D=0 λ2 0 
 
0 0 λ3
Then:  
λn1 0 0
Dn =  0 λn2 0 
 
0 0 λn3

Goal: Convert a matrix into diagonal form (if possible).

9
Diagonalization
A square matrix A is diagonalizable if there exists an invertible matrix P
such that:

P −1 AP = D
where D is a diagonal matrix.
Equivalently,

A = P DP −1

Theorem

26
An n × n matrix A is diagonalizable

20
al
⇐⇒

op
Bh
A has n linearly independent eigenvectors. IT
II

Important Result:
II-
ics

If A has n distinct eigenvalues, then A is diagonalizable.

at
m

Recall: Algebraic and Geometric Multiplicity

he
at

Let λ be an eigenvalue. Algebraic Multiplicity (AM): Number of times λ

M

appears as a root. Geometric Multiplicity (GM):

g
rin

GM = dim(ker(A − λI))
ee
in
ng

Condition for Diagonalization

:E
HI

Let A be a square matrix of order n. Then A is diagonalizable if and only if

GM = AM for each eigenvalue

Steps to Diagonalize a Matrix

Step 1: Find eigenvalues
det(A − λI) = 0
Step 2: Find eigenvectors Solve (A − λI)x = 0
Step 3: Form matrix P using eigenvectors.
Step 4: Form diagonal matrix D.
Step 5: Verify P −1 AP = D. ]

10
Example 7. Consider #"
4 1
A= .
0 2

Solution: Char. Equation is

det(A − λI) = (4 − λ)(2 − λ) = 0

So, eigenvalues are

λ1 = 4, λ2 = 2
" #
x
For λ = 4: find v1 = such that
y

26
20
" #" # " #
0 1 x 0
(A − 4I)v1 = =

al
0 −2 y 0

op
Bh
" #
1 IT
So, v1 = k , and hence GM (λ = 4) = 1.
0
II
II-

" #
x
For λ = 2: find v2 = such that
ics

y
at
m

" #" # " #

he

2 1 x 0
(A − 2I)V2 = = =⇒ 2x + y = 0.
at

0 0 y 0
M
g

"#
rin

1
So, v2 = k , and hence GM (λ = 2) = 1
ee

−2
in
ng

• Note that AM (λ) = GM (λ) for λ = 4, 2.

:E

• So, A is diagonalizable.
HI

Let " #
h i 1 1
P = v1 v2 =
0 −2
and " # " #
λ1 0 4 0
D= =
0 λ2 0 2
Thus,
A = P DP −1
Matrix is diagonalizable.

11
Example 8. Consider the matrix
" #
2 1
A= .
0 2

The Characteristic equation is

(2 − λ)2 = 0.

So, AM (λ = 2) = 2. Again, for λ = 2, the eigenvector v satisfying

" #" # " #
0 1 v1 0
(A − 2I)v = =
0 0 v2 0

26
20
" #
1

l
is v = . So, GM (λ = 2) = 1. Since GM ̸= AM, Matrix is NOT diagonal-

pa
0

o
Bh
izable.
T
II
Diagonalization and Powers
-I
II

If
s

A = P DP −1
ic
at

Then
m

An = P Dn P −1
he
at

Since Dn is easy to compute, large powers of A become simple.

M
g
in

Summary
r
nee

A matrix is diagonalizable if:

gi
En

• It has n distinct eigenvalues, OR

I:
H

• For each eigenvalue:

GM = AM

Exercise 9. Let !
4 1
A=
0 4

1. Find the eigenvalues of A.

2. Find Algebraic Multiplicity (AM) and Geometric Multiplicity (GM).

3. Is A diagonalizable? Justify.

12
Exercise 10. Let !
2 1
B=
1 2

1. Find eigenvalues and eigenvectors.

2. Find P such that P −1 BP = D.

3. Compute B 5 using diagonalization.

Exercise 11. Let  

3 1 0
C = 0 3 0
 
0 0 2

26
20
1. Find eigenvalues.

al
op
Bh
2. Compute AM and GM for each eigenvalue.
IT
3. Determine whether C is diagonalizable.
II
II-

Exercise 12. Let  

ics

4 1 0
at

D = 1 4 0
 
m

0 0 5
he
at
M

1. Find all eigenvalues.

g
rin

2. Find a basis for each eigenspace.

ee
in

3. Diagonalize the matrix (find P and D).

ng
:E

Exercise 13. Let  

HI

2 0 0
E = 0 2 0
 
0 0 2

1. Find eigenvalues and eigenspaces.

2. Is the matrix diagonalizable?

3. Give an example of a 3 × 3 matrix with only eigenvalue 2 that is not

diagonalizable.

13