EE448/528 Version 1.

0 John Stensby

Chapter 9
Eigenvalues, Eigenvectors and Canonical Forms Under Similarity
Eigenvectors and Eigenvectors play a prominent role in many applications of numerical

linear algebra and matrix theory. In this chapter, we provide basic results on this subject. Then,

we use these results to establish necessary and sufficient conditions for the diagonalization of a

square matrix under a similarity transformation. Finally, we develop the Jordan canonical form of

a matrix, a canonical form the has many applications.

Let T : U → U be a linear operator on a vector space U over the scalar field F. We are
r
interested in non-zero vectors X which map under T into scalar multiples of themselves. That is,
r
we are interested in non-zero vectors X ∈ U that satisfy

r r
T[X] = λX (9-1)

r
for some scalar λ ∈ F. Such a vector X is said to be an eigenvector corresponding to the

eigenvalue λ.

Example
r r r
Let I : U → U be the identity operator. For every X ∈ U, I[X] = X. Here, λ = 1 is an eigenvalue

of I, and every non-zero vector in U is an eigenvector.

Example
r r
Let T : R2 → R2 rotate in a counter clock wise direction T( X ) X
every vector by π/2 radians. The scalar field is R , the set
π/2
of real numbers. Note that no non-zero vector is a scalar

multiple of itself. Hence T : R2 → R2 has no eigenvalues

or eignevectors.

This lack of eigenvalues and eignevectors will not

occur if we use F = C, the field of complex numbers. Hence, in applications where eignevalues

[Link] Page 9-1

EE448/528 Version 1.0 John Stensby

play a role, we use the complex number field.

r r r
Let X and λ be an eigenvector and eigenvalue, respectively, so that T[X] = λX. Let c ∈ F

= C be any non-zero scalar. Then we have

r r
T[cX] = λ(cX) (9-2)

r
so that cX is an eignevector. Hence, eigenvectors are defined up to an arbitrary, non-zero, scalar.

Two or more linearly independent eigenvectors can be associated with a given eigenvalue.

In fact, for a given eigenvalue λ, the set

r r r
Sλ = {X ∈ U : T(X) = λX} (9-3)

r r
is a subspace known as the eigenspace associated with λ (note that 0 is in the eigenspace, but 0 is

not an eigenvector). Finally, the dimension of eigenspace Sλ is known as the geometric

multiplicity of λ. In what follows, we use γ to denote the geometric multiplicity of an eigenvalue.

For a given basis, the transformation T : U → U can be represented by an n×n matrix A.

In terms of this basis, a representation for the eigenvectors can be given. Also, the eigenvalues

and eigenvectors satisfy

r r
(A - λI)X = 0. (9-4)

Hence, the eigenspace associated with eigenvalue λ is just the kernel of (A - λI).

While the matrix representing T is basis dependent, the eigenvalues and eigenvectors are

not. The eigenvalues of T : U → U can be found by computing the eigenvalues of any matrix that

represents T. Let n×n matrix A represent T : U → U with respect to some fixed basis. Then the

eigenvalues are the roots of the nth-order characteristic polynomial

[Link] Page 9-2

EE448/528 Version 1.0 John Stensby

A - λI = det(A - λI) = 0. (9-5)

(note the notation introduced here: A means the determinant of matrix A). The eigenvalues

can be complex or real-valued. They can occur as simple roots or as multiple roots of the

characteristic polynomial. The number of times (i.e., the multiplicity) that λ appears as a root of

det(A - λI) is called the algebraic multiplicity of λ. We use αk to denote the algebraic

multiplicity of eigenvalue λk. A basis for the eigenspace associated with λ can be found by

computing a basis for the kernel of (A - λI).

Example

F1 −3 3 I λ −1 3 −3
A = G3 JJ
−5 3 so that λI - A = −3 λ + 5 −3 = ( λ + 2)2 ( λ − 4) = 0 .
GH 6 −6 4 K −6 6 λ−4

The distinct eigenvalues are λ1 = -2 and λ2 = 4. Eigenvalue λ1 = -2 has algebraic multiplicity α1 =

2, and eigenvalue λ2 = 4 has algebraic multiplicity α2 = 1. Now we find the eigenvectors.

Consider first the eigenvalue λ1 = -2. The matrix

F3 −3 3 I
[A − λI]Y = G3 −3 3 JJ
Yλ1 = −2 GH 6 −6 6 K
r r
has a nullity of two, and X11 = [1 1 0]T and X12 = [-1 0 1]T are two linearly independent

eigenvectors that span the two dimensional eigenspace associated with λ1 = -2 . Hence λ1 = -2

has geometric and algebraic multiplicities of γ1 = α1 = 2. Now, consider λ2 = 4. The matrix

F −3 −3 3 I
[A − λI]Y =G 3 −9 3 JJ
Yλ2 = 4 GH 6 −6 0 K
[Link] Page 9-3
EE448/528 Version 1.0 John Stensby

r
has a nullity of 1, and X2 = [1 1 2]T spans the one-dimensional eigenspace associated with λ2 =
4.

Eigenvector Indexing

From time to time, subscripts and superscripts need to placed on eigenvectors (and the

generalized eigenvectors that are introduced below). In the literature, there is not one indexing

scheme that is predominant (there are faults with all eigenvector indexing schemes). Notice the

indexing scheme that was introduced by the previous example. On some eigenvectors, we placed
r
two subscripts; we wrote Xjk. The first subscript (the "j" subscript) associates the eigenvector
with one of the numerically distinct eigenvalues (each of which can have an algebraic multiplicity

greater than one); we have 1 ≤ j ≤ d, where d is the number of distinct eigenvalues. The second

subscript (the "k" subscript) orders the eigenvector in the set of independent eigenvectors

associated with the "j" eigenvalue; we have 1 ≤ k ≤ γj, where γj is the geometric multiplicity of the

"j" eigenvalue. Sometimes, we place only one subscript on an eigenvector. This one subscript

may associate the eigenvector with a distinct eigenvalue, or it may order the eigenvector in a set

of independent eigenvectors (or it may do both). When one subscript appears, its meaning can be

inferred from context (or its meaning will be stated explicitly). Finally, on an eigenvector,

subscripts are used only when necessary; we will drop all subscripts when they are not needed to

clarify notation.

Eigenvalues of Similar Matrices

Recall that n×n matrices A and B are said to be similar if there exists a nonsingular n×n

matrix P such that A = P-1BP. The matrix representing a linear transformation depends on the

underlying basis; however, all matrices that represent a linear transform are similar to one another.

Furthermore, they have the same eigenvalues and eigenvectors.

Theorem 9-1

Similar matrices have the same eigenvalues and eigenvectors.

Proof: This follows directly from the basic definitions since eigenvalues and eigenvectors are

associated with an underlying linear transformation and not with any particular matrix or vector

[Link] Page 9-4

EE448/528 Version 1.0 John Stensby

r r r r r r
representation. Let β1, β2, ... , βn and β1′, β2′, ... , βn′ denote the “old” and “new” bases,
respectively, for the vector space; in terms of a non-singular transformation matrix P, the “old”
r
and “new” bases are related as shown by (3-27). X and A denote “old” representations for the
r r
eigenvector and matrix, respectively. X′ = P-1X and A′ = P-1AP denotes “new” representations

for the eigenvector and matrix, respectively (see (3-29) and (3-41)). With respect to the “new”
r r
basis, the “old” eigen problem AX = λX becomes

r r
(PA′P-1)PX′ = λ(PX′). (9-6)

After multiplication on the left by P-1, (9-6) becomes the “new” eigen problem

r r
A′X′ = λX′. (9-7)

So, while a similarity transformation changes the matrix and vector representations, it does not

change the underlying linear transformation or its eigenvalues/eigenvectors.♥

Theorem 9-2

For eigenvalue λ, the geometric multiplicity γ does not exceed the algebraic multiplicity α.

Proof: The geometric multiplicity γ of eigenvalue λ is defined independently of any matrix

representing linear transformation T : U → U. The characteristic equation, eigenvalues and

eigenvectors are the same for all matrices that represent T. Hence, to represent transformation T,

we can choose the matrix that makes obvious the proof of this theorem. Let γ be the dimension
r r
of eigenspace Sλ (γ is the geometric multiplicity of λ). Let eigenvectors X1, ... , Xγ be a basis for

eigenspace Sλ (eigenvector subscripts are used here as an index into the set of basis vectors). This

linearly independent set of eigenvectors can be extended to a basis

[Link] Page 9-5

EE448/528 Version 1.0 John Stensby
r r r r r r
X1, X 2 , L , X γ , X γ +1, X γ + 2 , L , X n (9-8)
1442443 144424443
eigenvectors that any other independent
span eigenspace Sλ vectors

r r
of n-dimensional U. The vectors Xγ+1, ... , Xn can be arbitrary as long as they are independent of
r r
each other and independent of the first γ eigenvectors. Now, T(Xi) = λXi for 1 ≤ i ≤ γ. With
respect to (9-8), the matrix A representing T has the form

γ cols n-γ cols

LMλ OP
MM λ
O
A γ ,n−γ PP γ rows

A = MM λ
PP (9-9)
MM PP
MM A n−γ ,n−γ
PP n-γ rows
N Q

Sub-matrix Aγ,n-γ is γ×(n-γ) and A n-γ,n-γ is (n-γ)×(n-γ). These sub-matrices are non-zero, in general;

the values they contain are of no concern to us. From inspection of (9-9), it is evident that the

algebraic multiplicity of λ is at least equal to γ. Hence, for any eigenvalue, the algebraic

multiplicity ≥ geometric multiplicity (α ≥ γ).♥

Theorem 9-3
r r r
Let λ1, λ2, ... , λs be any s distinct eigenvalues, and let X1, X2, ... , Xs (subscripts are used

here to associate an eigenvector with a distinct eigenvalue) be the associated eigenvectors. These

s eigenvectors are linearly independent.

Proof (by contradiction)

Suppose the set of s vectors is dependent. Re-order the eigenvectors so that the first k are

linearly independent and the remaining s-k vectors are dependent on the first k vectors. Then, we

can write the unique representation

[Link] Page 9-6

EE448/528 Version 1.0 John Stensby

r k r
Xs = ∑ ci X i , ci ∈ F , (9-10)
i =1

r r
Since Xs ≠ 0, there are non-zero ci in (9-10). Apply the linear transformation T to (9-10) and
obtain

r k r
λ s Xs = ∑ ciλ i X i . (9-11)
i =1

There are two possibilities. First, if λs = 0, then λi ≠ 0, 1 ≤ i ≤ k, since λ1, ... , λs are distinct. λs =
r r
0 implies that X1, ... , Xk are dependent, a contradiction. The second possibility is that λs ≠ 0, so

that we can write

r k
λ r FG IJ
Xs = ∑ ci i X i .
i =1
λs H K (9-12)

Since there are non-zero ci, and λi/λs ≠ 1 due to distinct eigenvalues, (9-12) is different than

(9-10), a contradiction (since representation (9-10) is unique). Hence, for either possibility, we
r r r
have a contradiction, and the s eigenvectors X1, X2, ... , Xs are independent.♥ Note that the

converse of this theorem is not true (independent eigenvectors are not always associated with

distinct eigenvalues).

Theorem 9-3 tells us a lot about matrices with distinct eigenvalues (distinct eigenvalues

are a common occurrence in practical applications). Matrices with distinct eigenvalues have linear

independent eigenvectors. When this occurs, it is possible to use the n independent eignevectors

to form a basis of n-dimensional U, a useful thing to do when proving theorems.

Let n×n matrix A represent T : U → U with respect to some fixed basis. Suppose T has n
r r r
linearly independent eigenvectors X1, X2, ... , Xn, (subscripts are used to index the eigenvectors in

[Link] Page 9-7

EE448/528 Version 1.0 John Stensby

this set of n independent eigenvectors), and we use them as a basis of n-dimensional space U. We

want to find the matrix D that represents linear transformation T with respect to this basis of

eignevectors. Use these independent eigenvectors to define the n×n transformation matrix

r r r
P ≡ X1 X 2 L X n . (9-13)

Then, with respect to the eigenvector basis, the matrix D that represents T is D = P-1AP. But this

implies that AP = PD, a result that can be written as

r r r r r r
A X1 X2 L Xn = λ1X1 λ2X2 L λnXn

LMλ1 OP
Xn M PP
r r r λ2 (9-14)
= X1 X2 L
MM O
P
N λn Q
.

But, Equation (9-14) leads to the observation that

LMλ1 OP
D=M PP = Xr 1
λ2 r r −1 r r r
MM O
P
X 2 L X n A X1 X 2 L X n (9-15)

N λn Q

Hence, when a basis of eigenvectors is used, the n×n matrix representing T: U → U is diagonal

with the eigenvalues appearing on the diagonal.

If n×n matrix A has distinct eigenvalues, then there is a basis of eigenvectors that can be

used as columns of n×n matrix P. And, with a similarity transformation, matrix P can be used to

diagonalize matrix A. More generally, if each eigenvalue of A has equal geometric and algebraic

[Link] Page 9-8

EE448/528 Version 1.0 John Stensby

multiplicities, then there are n linearly independent eigenvectors, and A can be diagonalized as

described above.

The converse is true as well. That is, if an n×n nonsingular matrix P exists such that P-1AP

is diagonal, then we can conclude

1. The eigenvalues of A appear on the diagonal of P-1AP, and

2. The columns of P are n linearly independent eigenvectors of matrix A.

We have argued the following theorem.

Theorem 9-4

An n×n matrix A is similar to a diagonal matrix D if and only if there are n linearly independent

eigenvectors of A. Furthermore, the eigenvalues of A must appear on the diagonal of D.

Example A =
LM1 −1OP
N2 −1Q
r
det(A - λI) = λ2 + 1 so that λ = ± j are the eigenvalues. λ1 = +j has the eigenvector X1 = [1 1-j]T.
r r r
λ2 = -j has the eigenvector X2 = [1 1+j]T. The eigenvalues are distinct, so X1 and X2 are

independent and

P=
LM 1 1 OP ⇒ P-1AP =
LM j 0 OP
N1 − j 1+ jQ N0 − jQ
LM1 0 −1 OP
Example
MM0
A= 0 1 0
PP has eigenvalues λ1 = 1 (α1 = 2), and λ2 = 2 (α2 = 1). The
N 0 2 Q
eigenvectors are
r r
λ1 = 1 ⇒ X11 = [1 0 0]T and X12 = [0 1 0]T
r
λ2 = 2 ⇒ X2 = [-1 0 1]T
r r r
Note that λ1 has equal algebraic and geometric multiplicities of two. Hence, X11, X12 and X2

comprise a basis of eigenvectors, and we have

[Link] Page 9-9

EE448/528 Version 1.0 John Stensby

LM1 0 −1 OP L 1 0 0 OP
P-1AP = M0
MM0
P= 0 1 0
PP ⇒
MM0 1 0
PP
N 0 1 Q N 0 2 Q
LM1 1 2OP
Example
MM0
A= 0 1 3
PP
N 0 2 Q
has eigenvalues λ1 = 1 (α1 = 2), and λ2 = 2 (α2 = 1). Since nullity(A - λ1I) = 1, we know that λ1

= l has a geometric multiplicity of γ1 = 1 but an algebraic multiplicity of α1 = 2. Hence, there is no

basis of eigenvectors, and matrix A cannot be diagonalized under similarity.

When there is not a basis of eigenvectors, n×n matrix A cannot be diagonalized.

However, we show that a nonsingular n×n matrix P exists such that P-1AP is “almost” diagonal;

our P-1AP has eigenvalues on its diagonal and “1s” immediately above some of the diagonal

eigenvalues. This new “almost diagonal” matrix is called the Jordan Canonical Form for A, and

it has many applications in engineering and the applied sciences. First, we must introduce the

subject of generalized eigenvectors.

Generalized Eigenvectors
r
Let A be an n×n matrix. For an eigenvalue λ, vector X is said to be a generalized

eigenvector of rank k > 0 if

r r
( A − λ I )k X = 0
. (9-16)
k −1 r r
( A − λI ) X≠0

r r r
An “ordinary” eigenvector X is a generalized eigenvector of rank k = 1 since (A - λI)X = 0 and
r r r
(A - λI)0X = X ≠ 0.
r
We develop a chain of generalized eigenvectors. For a given eigenvalue λ, let X be a

generalized eigenvector of rank k. Define the chain of k generalized eigenvectors as

[Link] Page 9-10

EE448/528 Version 1.0 John Stensby
r r
Xk ≡ X
r r r
X k −1 ≡ (A - λI)X = (A - λI)X k
r r r
X k − 2 ≡ (A - λI)2 X = (A - λI)X k −1 . (9-17)

M
r r r
X1 ≡ (A - λI)k-1 X = (A - λI)X 2

A superscript on a vector is not a power; it is used to indicate rank, and it is used as an index! On
r
Xk, the k is used as a rank indicator and index; k is not a power (raising a vector to a power is an

operation that has not been defined!). Now, settle down, get over it! On a vector, the only time

we will use a superscript is when we are working with vectors in a chain of generalized

eigenvectors (we have already described how we want to use the subscript position(s)). On

generalized eigenvectors, superscripts are standard in the literature.

r
For each i, 1 ≤ i ≤ k, Xi is a generalized eigenvector of rank i since

r r r r
(A - λI)i Xi = (A - λI)i (A - λI)k-i X = (A - λI)k X = 0, (9-18)

r r r r
(A - λI)i-1 Xi = (A - λI)i-1 (A - λI)k-i X = (A - λI)k-1 X ≠ 0 . (9-19)

r
Note that X1 is an "ordinary" eigenvector since

r r r r
(A - λI)X1 = (A - λI)(A - λI)k-1X = (A - λI)k X = 0. (9-20)

r r r
As mentioned above, we call X1, X2, ... , Xk a chain of generalized eigenvectors. Now, we

examine some properties that chains have.

[Link] Page 9-11

EE448/528 Version 1.0 John Stensby

Theorem 9-5
r r r
A chain X1, X2, ... , Xk of generalized eigenvectors is linearly independent.
Proof (by contradiction)

For the moment, assume that the vectors in the chain are dependent. Then there exists constants

c1, c2, ... , ck, not all zero, such that

r r r r
c1X1 + c2X2 + ... + ckXk = 0. (9-21)

First, note that for i = 1, 2, ... , k-1 we can write

r r r r
(A - λI)k-1 Xi = (A - λI)k-1(A - λI)k-i X = (A - λI)2k-(i+1) X = 0, (9-22)

a result we will use very soon. Now, apply (A - λI)k-1 to both sides of (9-21) to obtain

r r r r
(A - λI)k-1{ c1X1 + c2X2 + ... + ckXk} = 0. (9-23)

Use (9-22) in (9-23) to obtain

r r
ck(A - λI)k-1 Xk = 0. (9-24)

r r
But, we know that (A - λI)k-1 Xk ≠ 0. Hence, we must have ck = 0 so that (9-21) becomes

r r r r
c1X1 + c2X2 + ... + ck-1Xk-1 = 0. (9-25)

On this equation, repeat the procedure that starts with (9-22). That is, multiply (9-25) by (A -

λI)k-2, and repeat the above argument (that produced ck = 0) to reach the conclusion that ck-1 = 0.

[Link] Page 9-12

EE448/528 Version 1.0 John Stensby

Obviously, this same argument can be repeated a sufficient number of time to conclude that ci = 0,
r r r
1 ≤ i ≤ k. This contradiction leads to the conclusion that the chain X1, X2, ... , Xk is comprised of
linear independent generalized eigenvectors.♥

Theorem 9-6
r
Let λ1 ≠ λ2 be two eigenvalues of n×n matrix A. Suppose X is a generalized eigenvector
r
of rank k associated with λ1 and Y is a generalized eigenvector of rank m associated with λ2.

Define the two chains

r r r r r
Xk = X, and Xi = (A - λ1I)Xi+1 = (A - λ1I)k-i X for i = k-1, k-2, ... , 1 (9-26)

r r r r r
Ym = Y, and Yj = (A - λ2I)Yj+1 = (A - λ2I)m-j Y for j = m-1, m-2, ... , 1 (9-27)

The set of k+m vectors described by (9-26) and (9-27) are linearly independent. Equivalently, any

generalized eigenvector from one chain is independent of the vectors in the other chain.

Proof (by contradiction)

r r r r
Suppose there is an i, 1 ≤ i ≤ k, for which Xi is linearly dependent on the chain Y1, Y2, ... , Ym.

Then, there exists constants c1, ... , cm, not all zero, such that

r m r
Xi = ∑ c j Y j (9-28)
j=1

Multiply (9-28) by (A - λ1I)i, and use the fact that

r r r
(A - λ1I)i Xi = (A - λ1I)i (A - λ1I)k-i X = 0 (9-29)

to obtain

[Link] Page 9-13

EE448/528 Version 1.0 John Stensby

m r r
(A - λ1I)i ∑ cj Y j = 0 (9-30)
j=1

Now, multiply (9-30) by (A - λ2I)m-1, and use the facts

i) (A - λ2I)m-1(A - λ1I)i = (A - λ1I)i (A - λ2I)m-1

r r
ii) (A - λ2I)m-1 Yj = 0 for j = m-1, m-2, ... , 1
to obtain

r r r
(A - λ1I)i (A - λ2I)m-1 cmYm = cm(A - λ1I)i Y1 = 0 (9-31)

r r r
Now, Y1 is an "ordinary" eigenvector: AY1 = λ2Y1, so (9-31) becomes

r r
cm(λ2 - λ1)Y1 = 0. (9-32)

Since λ2 ≠ λ1 we must have cm = 0 so that (9-30) becomes

m −1 r r
(A - λ1I) i
∑ cj Y j = 0 . (9-33)
j=1

Now repeat the argument that started with (9-30) and produced cm = 0. That is, multiply (9-33)

by (A - λ2I)m-2, follow the argument, and conclude that cm-1 = 0. Continue this process to the

conclusion that ci = 0 for i = m, m-1, m-2, ... , 1. This contradiction (the ci's are not all zero)
r r r r r r
leads to the conclusion that Xi is independent of Y1, Y2, ... , Ym. Hence, the two chains Y1, Y2, ...
r r r r
, Ym and X1, X2, ... , Xk contain m+k linearly independent vectors.♥

Theorem 9-7
r r
Let Y and X be generalized eigenvectors of rank m and k, respectively, associated with

the same eigenvalue λ. Define the two chains

[Link] Page 9-14

EE448/528 Version 1.0 John Stensby

r r r r r
Xk = X, and Xi = (A - λI)Xi+1 = (A - λI)k-i X for i = k-1, k-2, ... , 1 (9-34)

r r r r r
Ym = Y, and Yj = (A - λI)Yj+1 = (A - λI)m-j Y for j = m-1, m-2, ... , 1 (9-35)

r r
If the "ordinary" eigenvectors Y1 and X1 are independent, then so are the two chains (i.e., (9-34)
and (9-35) describe m+k independent vectors).

Proof

Similar to the proof of Theorem 9-6.♥

Theorems 9-5, 9-6 and 9-7 provide the basis of our generalized eigenvector theory. Note

that we have shown an important result.

Associated with eigenvalue λ are γ distinct chains of generalized eigenvectors (γ is the geometric multiplicity of λ).

Each chain is "anchored" by an "ordinary" eigenvector (of rank one). In these γ chains, the total number of generalized

eigenvectors is α, the algebraic multiplicity of λ. And, these α vectors are linearly independent.

Note that we have not discussed how many vectors are in each chain. We have argued only that

there are a total of α generalized eigenvectors divided into γ chains associated with λ. While n×n

matrix A may, or may not, have n independent eigenvectors, it always has n independent

generalized eigenvectors.

Eigenvector Indexing - Revisited

It's time once more to consider generalized eigenvector indexing. A generalized

eigenvector can have two subscripts and one superscript. The meaning of the two subscripts are

given above in the section on eigenvector indexing (which is worth reading again). The

superscript is used as both a rank indicator and index into a chain. For example, consider the
r
generalized eigenvector X ljk . The "j" subscript associates the generalized eigenvector with

eigenvalue λj (1 ≤ j ≤ d, where d is the number of numerically distinct eigenvalues). The "k"

[Link] Page 9-15

EE448/528 Version 1.0 John Stensby

subscript associates the generalized eigenvector with a particular chain of independent generalized

eigenvectors for λj (1 ≤ k ≤ γj , where γj is the geometric multiplicity of λj). As described above,

superscript l is a rank indicator, and it is an index into the kth chain of generalized eigenvectors
r
associated with λj. Finally, note that X1jk is the kth "ordinary" eigenvector associated with λj.

Listing of all Generalized Eigenvectors

Let λ1, λ2, ... , λd denote the numerically distinct eigenvalues of an n×n matrix A . For 1 ≤

k ≤ d, eigenvalue λk has an algebraic multiplicity of αk and a geometric multiplicity of γk.

Furthermore, for 1 ≤ k ≤ d, eigenvalue λk is associated with γk separate chains of generalized

eigenvectors containing a total (in all of the γk chains) of αk independent generalized eigenvectors.

Finally, taken all together, for the d numerically distinct eigenvalues, a total of n generalized

eigenvectors exist, considering all of the vectors in all of the chains.

We can list these n generalized eigenvectors. Using the indexing scheme outline above,

we write

R| Xrr 1
11
r2
X11 L
rh
X1111
r2 rh
The α1 generalized eigenvectors for λ1 are | X
1
X12 L X1212
divided into γ 1 chains
S| M 12
M M
||TXr 1
1γ 1
r2
X1γ
1
r h1γ 1
L X1γ
1

R| Xrr 1
21
r
X 221 L
rh
X 2121
r rh
The α 2 generalized eigenvectors for λ 2 are | X
1
X 222 L X 2222
divided into γ 2 chains
S| M 22
M M
(9-36)

||TXr 1
2γ 2
r2
X2 γ
2
r h 2γ
L X2 γ 2
2

M M M M M
r
R|
X1d1
r
X d21
r h d1
L X d1
r1 r rh
|S
The α d generalized eigenvectors for λ d are X d 2 X d2 2 L X d 2d 2
.
divided into γ d chains
r1
M || r2
M M
rh γ
Xd γ |T d
Xd γ
d
L X d γd d
d

[Link] Page 9-16

EE448/528 Version 1.0 John Stensby

Here, hkj, 1 ≤ k ≤ d, 1 ≤ j ≤ γk, denotes the number of generalized eigenvectors in the jth chain

associated with the numerically distinct eigenvalue λk. Integer hkj has to be computed as outlined

in the example given below. As stated in the list given above, we have

γk
α k = ∑ h kj . (9-37)
j=1

Also, we denote the total number of chains as

d
ν= ∑γk . (9-38)
k =1

Finally, for an n×n matrix A, we have

d d γk
n= ∑ α k = ∑ ∑ h kj . (9-39)
k =1 k =1 j=1

An n×n matrix A may, or may not, have n linearly independent eigenvectors. However, it always

has n linearly independent generalized eigenvectors.

Example Reconsider the previous example where

LM1 1 2 OP
MM0
A= 0 1 3
PP
N 0 2 Q
Eigenvalue λ1 = 1 has an algebraic multiplicity of α1 = 2 and a geometric multiplicity of γ1 = 1;
r
X111 = [1 0 0]T is an "ordinary" eigenvector for λ1. Eigenvalue λ2 = 2 has geometric and
r
algebraic multiplicities of 1; X121 = [5 3 1]T is an "ordinary" eigenvector for λ2. We are one

[Link] Page 9-17

EE448/528 Version 1.0 John Stensby

eigenvector short; the matrix A cannot be diagonalized by a similarity transformation. However,

we can find two generalized eigenvectors associated with λ1 = 1. Let's find a chain of length two
r
associated with λ1 = 1. These two generalized eigenvectors, when combined with X121 , will
r
produce a basis of generalized eigenvectors. First, find a non-zero X such that

r
LM
0 1 2
r r
OP
MM
(A − λ1Ι)X = 0 0 3 X ≠ 0
PP
0 0 1N Q
r
LM
0 1 2 0 1 2
r
0 0 5
r r
OP LM OP LM OP
(A − λ1Ι)2 X = 0 0 3 0 0 3 X = 0 0 3 X = 0
MM PP MM PP MM PP
N
0 0 1 0 0 1 0 0 1 QN Q N Q
r
Clearly, X = [0 1 0]T is a generalized eigenvector of rank 2, and we use this vector to write

r2
0LM OP
X11 = 1 MM PP
0 NQ
r1 r
0 1 2 0 LM1 OP LM OP LM OP
MM
X11 = ( A − λ1 )X = 0 0 3 1 = 0 PP MM PP MM PP
0 0 1 0 N
0 QN Q NQ
r r2 r r2
{ X111 , X11 } is a chain of length two associated with λ1 = 1. The vectors X111 = [1 0 0]T, X11 =
r
[0 1 0]T, X121 = [5 3 1]T form a basis of generalized eigenvectors. With respect to this basis,

let's find the matrix A′ that represents the underlying transformation. Define the 3×3 non-singular
r r 2 r1
matrix P ≡ X111 X11 -1
X 21 , and compute A′ = P AP. We compute A′ by considering the
r r 2 r1 r r 2 r1
equivalent equation PA′ = AP ⇔ X111 X11 X 21 A′ = A X111 X11 X 21 so that

[Link] Page 9-18

EE448/528 Version 1.0 John Stensby

r1
1 LM OP
r r2 r
MM PP
AX11 = 0 = 1 ⋅ X111 + 0 ⋅ X11 + 0 ⋅ X121
0 NQ
r2
1 LM OP
r r2 r
MM PP
AX11 = 1 = 1 ⋅ X111 + 1 ⋅ X11 + 0 ⋅ X121
0 NQ
r1
10 LM r
OP r2 r
MM PP
AX 21 = 6 = 0 ⋅ X111 + 0 ⋅ X11 + 2 ⋅ X121
2 N Q

As a result, we see that

−1
LM1 1 0 OP
A′ = P
MM0
AP = 0 1 0 .
PP
N 0 2 Q

Note that A′ has two blocks on its diagonal; we write A′ as

LM J1
OP LM1 1OP, J 2 ≡ 2
A′ =
MM J PQ
P , J1 ≡
N0 1Q
N 2

Matrix A′ is known as the Jordan Canonical Form for matrix A.

Jordan Canonical Form

This procedure can be applied to transform any n×n matrix into its block-diagonal Jordan

canonical form. Let λ1, λ2, ... , λd be the numerically distinct eigenvalues of n×n matrix A. For 1

≤ k ≤ d, let λk have algebraic multiplicity αk and geometric multiplicity of γk. As outlined above,

eigenvalue λk is associated with γk chains containing a total of αk generalized eigenvectors, and

[Link] Page 9-19

EE448/528 Version 1.0 John Stensby

each chain is "anchored" by an "ordinary" eigenvector. As listed by (9-36), there are a total of n

linearly-independent generalized eigenvectors split up into ν chains. We use these n generalized

eigenvectors to define the n×n transformation matrix

r rh r rh r r h1γ
P = [X111 L X1111 X112 L X1212 L X11γ L X1γ1 1
1442443 1442443 144 1
42444 3
chain #1 for λ1 chain #2 for λ1 chain #γ 1 for λ1

r rh r rh r r h2γ 2
X121 L X 2121 X122 L X2222 L X1 γ L X γ L
144 42444 3 144 42444 3 2 2
1444 24442 2
3
chain #1 for λ 2 chain #2 for λ 2 chain #γ 2 for λ 2 (9-40)

M M M

r rh r r hd2 r r h d γd
L X1d1 L X d1d1 X1d 2 L X d 2 L X1dγ L X d γ ] .
144 42444 3 144 42444 3 144 d 42444 3 d
chain #1 for λ d chain #2 for λ d chain #γ d for λ d

By using the similarity transformation A′ = P-1AP, matrix P, given by (9-40), can be used

to transform n×n matrix A into its Jordan Canonical Form. This canonical form is a block

diagonal matrix

LMJ1 OP
A ′ = P −1AP = M PP
J2
MM O
P
(9-41)

N JνQ

made from ν blocks Jk, 1 ≤ k ≤ ν, one block for each chain of generalized eigenvectors. Note that

(9-41) is equivalent to AP = PA′, a matrix equation that can be written as

r rh r r h d γd
AP = A[X111 L X1111 L X1dγ L X d γ ]
1442443 144 d
42444 3d
st
1 chain
νth chain

[Link] Page 9-20

EE448/528 Version 1.0 John Stensby

LMJ1 OP
= [X111 L X11 L X1dγ L X d γ ] M PP = PA′
r rh r rh d γd J2
3 M
11

1442443 O
MN P
d
st 14442444 d
(9-42)
JνQ
1 chain
ν chain th

r r r h jk
Let's examine the structure of a typical block. Consider X1jk , X 2jk , L , X jk , the kth chain

associated with λj, the jth distinct eigenvalue. The Jordan block for this chain is Jp, where

j−1
p = k + ∑γi (9-43)
i =1

From the basic definition of this chain, we have

rh
X jk

r h −1 rh rh rh r h −1
X jk = ( A − λ jI )X jk ⇒ AX jk = λ jX jk + X jk (9-44)

r h −2 r h −1 r h −1 r h −1 r h − 2
X jk = ( A − λ jI )X jk ⇒ AX jk = λ jX jk + X jk
M M M
r r r r r
X1 = ( A − λ jI )X 2 ⇒ AX 2 = λ jX 2 + X1 ,

where we have omitted the common subscripts jk on all generalized eigenvectors. From PA′ =

AP (see (9-42)), we have the requirement

LMXr 1jk r r h jk OP r
X2jk L X jk J p = A X1jk LM r r h jk OP
N Q N X 2jk L X jk ,
Q (9-45)

r r
where p is given by (9-43). However, from (9-44) and the requirement A X1jk = λj X1jk , it is easy

to see that

[Link] Page 9-21

EE448/528 Version 1.0 John Stensby

hjk columns

LMλ j 1 0 OP
MM 00 λj 1
λj O
PP
Jp = M PP
0 (9-46)
hjk rows
MM O
λj 1P
MM λ j PQ
P
N 0

That is, Jp is an hjk×hjk matrix with λj on its diagonal, "1s" on its first "super diagonal", and zeros

everywhere else.

Computational Procedure for Jordan Form

For many low-dimensional problems of practical interest, the Jordan form can be

computed "by hand" without too much effort. A computational procedure for computing the

Jordan form is outlined below.

1. Compute the eigenvalues and "ordinary" eigenvectors of n×n matrix A; determine the algebraic

and geometric multiplicities of the eigenvalues. The distinct eigenvalues are λ1, λ2, ... , λd; for 1 ≤

k ≤ d, eigenvalue λk has algebraic multiplicity αk and geometric multiplicity γk.

2. In γ1 distinct chains, compute a total of α1 independent, generalized eigenvectors for λ1. To

accomplish this, compute (A - λ1I)i for i = 1, 2, ... until the rank of (A - λ1I)k is equal to the rank

of (A - λ1I)k+1. Then, compute a rank k generalized eigenvector and its k-long chain. If k = α1,

go to step #3. Otherwise, look for a second rank-k vector and its chain. If a second rank k

vector does not exist, look for one of rank k-1, and so on, until we have γ1 distinct chains of α1

generalized eigenvectors.

3. Repeat step #2 for the remaining eigenvalues λ2, ... , λd.

4. Write down the Jordan form. For eigenvalue λj, the kth chain is of length hjk (determined in

step #2), and there is an hjk×hjk Jordan block with λj on its diagonal.

[Link] Page 9-22

EE448/528 Version 1.0 John Stensby

In the Jordan form, the ordering of the blocks is not critical. However, it is common to

keep sequential all blocks associated with the same eigenvalue.

LM3 −1 1 1 0 0 OP
MM01 1 −1 −1 0 0
PP
A=M PP
0 2 0 1 1
Example
MM0 0 0 2 −1 −1
PP
MN00 0 0 0
0 0 0
1 1
1 1 Q
Compute the eigenvalues and algebraic multiplicities. Note that det(A - λI) = λ(λ - 2)5, and this

implies that λ1 = 2 with α1 = 5 and λ2 = 0 with α2 = 1. Furthermore, eigenvalue λ1 = 2 has the

two independent eigenvectors

r1 T
X11 = 1 1 0 0 0 0
,
r1 T
X12 = 0 0 1 −1 0 0

so γ1 = 2. Also, λ2 = 0 has the single eigenvector

r T
X 121 = 0 0 0 0 −1 1 ,

so γ2 = 1. Now, compute (A - λ1I)i, for increasing i until the rank no longer changes.

LM 1 −1 1 1 0 0 OP
MM01 −1 −1 −1 0 0 PP
( A − 2I ) = M PP
0 0 0 1 1
has rank equal to 4.
MM0 0 0 0 −1 −1
PP
0 0 0 −1 1
MN00 0 0 0 1 −1 Q
[Link] Page 9-23
EE448/528 Version 1.0 John Stensby

LM0 0 2 2 0 0 OP
MM00 0 2 2 0 0
PP
( A − 2 I )2 = M PP
0 0 0 0 0
has rank equal to 2.
MM0 0 0 0 0 0
PP
0 2 −2
MN00 0
0
0
0 0 −2 2 Q
LM0 0 0 0 0 0 OP
MM00 0 0 0 0 0
PP
( A − 2I)3 = M PP
0 0 0 0 0
has rank equal to 1.
MM0 0 0 0 0 0
PP
0 −4
MN00 0
0
0
0
4
0 4 −4 Q
LM0 0 0 0 0 0 OP
MM00 0 0 0 0 0
PP
( A − 2 I )4 = M PP
0 0 0 0 0
has rank equal to 1.
MM0 0 0 0 0 0
PP
0 8 −8
MN00 0
0
0
0 0 −8 8 Q
The rank of (A - 2I)3 is equal to the rank of (A - 2I)4; hence, there is a rank 3 generalized
r3
eigenvector that is in K((A - 2I)3) but not in K((A - 2I)2). It is easily computed as X11 = [0 0 1
r 3 r r 3 r
0 0 0]T since (A - 2I)3 X11 = 0 but (A - 2I)2 X11 ≠ 0. Now, we compute the first 3-long chain

associated with λ1.

LM2OP LM 1OP LM0OP

r
M
r 3 M0P
2P
r2
M
r 3 M 0P
−1P M
r 3 M 1P
0P

X111 ≡ ( A − 2I)2 X11 =M P, X11 ≡ ( A − 2I)X11 =M P, X11 = M P.

MM0PP MM 0PP MM0PP
MN00PQ MN 00PQ MN00PQ

[Link] Page 9-24

EE448/528 Version 1.0 John Stensby

Since α1 = 5, there are two more generalized eigenvectors associated with λ1; inspection of (A -

2I)2 and (A - 2I) reveals where they are. There is a generalized eigenvector of rank 2 that is in
r2
K((A - 2I)2) but not in K(A - 2I). This rank 2 vector is X12 = [0 0 0 0 1 1]T; note that (A -
r2 r r2 r
2I)2 X12 = 0 but (A - 2I) X12 ≠ 0. Hence, our second chain associated with λ1 = 2 is

LM 0OP LM0OP
r1
M
r 2 M 2P
0
P r 2 M0P
M 0
P
X12 ≡ ( A − 2I )X12 = M P , X12 = M P .
MM−2PP MM0PP
MN 00PQ MN 11PQ

We have 5 generalized eigenvectors associated with λ1 = 2; there are no more. With the
r T
eigenvector X 121 = 0 0 0 0 −1 1 , we have a basis of 6 generalized eigenvectors that we

can use to write the transformation matrix

LM2 1 0 0 0 0 OP
r r 2 r 3 r1 r2 r
M
M0
2 −1 0 0 0 0
PP
X121 = M PP
0 1 2 0 0
P = X111 X11 X11 X12 X12 .
MM0 0 0 −2 0 0
PP
1 −1
MN00 0
0
0 0
0 0 1 1 Q

We write down (no computation is necessary) the Jordan canonical form

LM2 1 0 0 0 0OP
MM00 2 1 0 0 0
PP
A′ = M PP
0 2 0 0 0
.
MM0 0 0 2 1 0
PP
MN00 0
0
0
0
0
0
2
0
0
0 Q
[Link] Page 9-25
EE448/528 Version 1.0 John Stensby

Note that A′ contains the three Jordan blocks

LM2 1 0 OP LM OP
2 1
MM0
J1 = 0
PP
2 1 , J2 =
0 2
, J3 = 0 .
N Q
N 0 2 Q
It is easy to see that A′ satisfies PA′ = AP (so that A′ = P-1AP). As a MatLab exercise, enter P
and A as described above, and type inv(P)*A*P at the command prompt. MatLab will return

the Jordan canonical form A′ given above.

Jordan Form - Sensitivity Issues

Computation of the Jordan form is laborious and time consuming. Also, the Jordan form

in “computationally unstable”; in some cases, a very small perturbation of A can “put back” all of

the missing eigenvectors and remove the superdiagonal of ones. Because of the possible stability

problems, many numerical analysis computer programs do not include the Jordan form (the

Jordan form is not in MatLab proper; it is in MatLab’s symbolic algebra toolbox).

The Jordan form has several applications in state space control theory. Generally

speaking, control engineers will not design a system having a structure that is extremely sensitive

to small perturbations. In Chapter 10, we use the Jordan form to compute functions of matrices.

[Link] Page 9-26