1 Linear Transformations

1.1 Linear Function R → R

A linear function f : R → R is a function which satisfies two conditions

f (x + x0 ) = f (x) + f (x0 ), x, x0 ∈ R;
f (c · x) = c · f (x), c, x ∈ R.

Such a function has the form

f (x) = k · x,

where k ∈ R is some scalar.

1.2 Linear Function Rn → R

A linear function f : Rn → R is a function which satisfies two conditions

f (v + w) = f (v) + f (w), v, w ∈ Rn ;
f (c · v) = c · f (v), v ∈ Rn , c ∈ R.

Such a function has the form

f (v) = k1 · x1 + ... + kn · xn ,

where v = (x1 , ... , xn ), k = (k1 , ... , kn ).

Thus any linear function f : Rn → R has the form

f (v) = k · v

where k ∈ Rn is considered as a vector.

1.3 Linear Function Rn → Rm

A linear function f : Rn → Rm is a function which satisfies two conditions

f (v + w) = f (v) + f (w), v, w ∈ Rn ;
f (c · x) = c · f (x) v ∈ Rn , c ∈ R.

Such a function has the form

f (v) = (a11 · x1 + ... + a1n · xn , ... , am1 · x1 + ... + amn · xn ) ∈ Rm .

Thus any linear function f : Rn → Rm has the form

f (v) = A · v

1
where A is some matrix
 
a11 ... a1n
 
A =  ... .
am1 ... amn

à !
2 2 a11 a1,2
A linear function f : R → R is determined by a matrix A = ,
a21 a22
à ! à ! à !
a11 a1,2 x1 a11 · x1 + a12 · x2
f (x1 , x2 ) = · = .
a21 a22 x2 a21 · x1 + a22 · x2

From this expression easily follows that

à ! à ! à ! à !
1 a11 0 a12
f = , f = ,
0 a21 1 a22
à !
1
so the column vectors of the matrix A are images of basis vectors e1 =
0
à !
0
and e2 = .
1

Theorem 1 Suppose f : Rn → Rm is a linear map. Suppose also that the

images of the basis vectors
     
1 0 0
 0   1   0 
e1 = 


, e2 = 


, ... , en = 



 ...   ...   ... 
0 0 1

are the column vectors

     
a11 a12 an1
 a21   a22   an2 
     
f (e1 ) =  , f (e2 ) =  , ... , f (en ) =  .
 ...   ...   ... 
am1 am2 anm

Then  
a11 ... a1n
 
A =  ... .
am1 ... amn
is the matrix of f .

Example 1. Let f : R2 → R2 be the linear map which is rotation of the

plane by 90◦ clockwise. Find f (2, 3).

2
 The values of basis vectors are

f (1, 0) = (0, −1), f (0, 1) = (1, 0),

à !
0 1
so the matrix of this linear map is . Thus
−1 0
à ! à ! à !
0 1 2 3
f (2, 3) = · = .
−1 0 3 −2

Example 2. Let g : R2 → R2 be the linear map which is the expansion 2

times. Let us find it’s matrix.
The values of basis vectors are

g(1, 0) = (2, 0), g(0, 1) = (0, 2),

à !
2 0
so the matrix of this linear map is .
0 2

Example 3. Let g : R2 → R2 be the linear map which is the unequal

expansion in two perpendicular directions: 2 times in direction x and 3 times
in direction y. Let us find it’s matrix.
The values of basis vectors are

g(1, 0) = (2, 0), g(0, 1) = (0, 3),

à !
2 0
so the matrix of this linear map is .
0 3

Example 4. Let p : R2 → R2 be the projection on x axes: f (x, y) = (x, 0).

Let us find it’s matrix.
The values of basis vectors are

p(1, 0) = (1, 0), p(0, 1) = (0, 0),

à !
1 0
so the matrix of this linear map is .
0 0

Example 5. Let h : R2 → R2 be the linear map which is the reflection with

respect to y axes. Let us find it’s matrix.
The values of basis vectors are

g(1, 0) = (−1, 0), g(0, 1) = (0, 1),

à !
−1 0
so the matrix of this linear map is .
0 1

3
Theorem 2 A linear map F : Rn → Rn given by a matrix A is bijective if
and only if det(A) 6= 0.

Try to prove this!

2 Eigenvalues and Eigenvectors

Let  
a11 ... a1n
 
A =  ... ... ... 
an1 ... ann
be a matrix, which, as we know, defines a linear map F : Rn → Rn defined
by F (x) = A · x.
A scalar λ ∈ R and a nonzero vector x ∈ Rn are called respectively
eigenvalue and eigenvector of A if

A · x = λ · x.

This actually means that the linear map F changes the magnitude of x
but not its direction,
Note that if x is an eigenvector corresponding to an eigenvalue λ then kx
is an eigenvector too: A · (kx) = kA · x = kλx = λ(kx).
The specter of A (denoted by spec(A)) is defined as the set of all eigen-
values λ1 , ..., λk of A.
Eigenspace corresponding to an eigenvalue λ is defined as the subspace
spanned by all eigenvectors corresponding to this eigenvalue.
The geometric degree of an eigenvalue λ is defined as the dimension of its
eigenspace.

Let us observe examples 1-6 from previous section.

à !
0 1
Example 1. Rotation A = . No eigenvalues and eigenvectors.
−1 0
Check!
à !
2 0
Example 2. Expansion 2 times, A = . Eigenvector λ = 2, eigen-
0 2
vector - any nonzero vector, eigenspace - whole R2 . Check!
à !
2 0
Example 3. Unequal expansion A = . Eigenvalues λ1 = 2, λ2 =
0 3
3, corresponding eigenvectors v1 = (1, 0), v2 = (0, 1). Check!

4
 Ã !
1 0
Example 4. Projection on x-axes A = . Eigenvalues λ1 = 1, λ2 =
0 0
0, corresponding eigenvectors v1 = (1, 0), v2 = (0, 1). Check!
à !
−1 0
Example 5. Reflection about the y-axes A = . Eigenvalues
0 1
λ1 = −1, λ2 = 1, corresponding eigenvectors v1 = (1, 0), v2 = (0, 1).
Check!
à !
1 1
Example 6. Horizontal shear A = . Eigenvalue λ = 1, correspond-
0 1
ing eigenvector v1 = (1, 0). Check!

2.0.1 How to Find Eigenvalues and Eigenvectors

These can be found solving the matrix equation A · x = λ · x, equivalently
(A − λI)x = 0, which in its turn is the system


 (a11 − λ)x1 + a12 x2 + ... + a1n xn = 0

 a21 x1 + (a22 − λ)x2 + ... + a2n xn = 0
.


 ... ... ... ...

an1 x1 + an2 x2 + ... + (ann − λ)xn = 0

This is homogenous system so it has a nonzero solution if and only if its

determinant |A − λI| (which is called characteristic polynomial of A) is zero,
so |A − λI| = 0.
So, the eigenvalues can be found from the characteristic equation |A −
λI| = 0 that is
¯ ¯
¯ a11 − λ a12 ... a1n ¯
¯ ¯
¯ a21 a22 − λ ... a2n ¯
¯ ¯
¯ ¯ = 0.
¯
¯ ... ... ... ... ¯
¯
¯ a an2 ... ann − λ ¯
n1

Algebraic degree of an eigenvalue λ∗ ∈ Spec(A) is defined as its multiplic-

ity in characteristic polynomial: AlgDeg(λ) = k if |A − λI| = (λ − λ∗ )k · Q(λ)
where Q(λ) is some polynomial.

The algebraic degree of an eigenvalue λ is more or equal to its geometric

degree.
Example. Find the eigenvalues for the matrix
 
1 1 1
A= 
 1 1 1 .
1 1 1

5
Solution. The characteristic equation looks as
¯ ¯
¯ 1−λ 1 1 ¯
¯ ¯
¯ ¯
¯ 1 1−λ 1 ¯ = 0.
¯ ¯
¯ 1 1 1−λ ¯

Calculating this determinant we obtain

(1 − λ)3 − 3(1 − λ) + 2 = 0, λ3 − 3λ2 = 0, λ2 (λ − 3) = 0,

thus λ1 = 0, λ2 = 3. The algebraic degree of λ1 = 0 is 2, and of λ2 = 3 is 1.

2.0.2 How to Find Eigenvectors

Eigenvectors corresponding to the eigenvalue λ can be found solving the
matrix equation
(A − λI)x = 0
which is equivalent to the system


 (a11 − λ)x1 + a12 x2 + ... + a1n xn = 0

 a21 x1 + (a22 − λ)x2 + ... + a2n xn = 0
.


 ... ... ... ...

an1 x1 + an2 x2 + ... + (ann − λ)xn = 0

Since λ is an eigenvalue the determinant of this system is zero. Thus this

homogenous system has nonzero solutions.

2.1 Examples
Example. Find an eigenvector x corresponding to the eigenvalue λ = 3 of
the matrix  
1 1 1
A=  1 1 1 .


1 1 1
from the previous example.
Solution. We can find x from the matrix equation (A − 3 · I) · x = 0 which
as a system of linear equations looks as
 ¯
¯
 (1 − 3)x1
 + x2 + x3 = 0 ¯
¯
x1 + (1 − 3)x2 + x3 = 0 ¯,

 ¯
x1 + x2 + (1 − 3)x3 = 0 ¯

 ¯
¯
 −2x1
 + x2 + x3 = 0 ¯
¯
x1 − 2x2 + x3 = 0 ¯.

 ¯
x1 + x2 − 2x3 = 0 ¯

6
Rank of the determinant of this system is 2: a nonzero minor is
¯ ¯
¯ −2 1 ¯
¯ ¯
¯ ¯ = −5.
¯ 1 −2 ¯

Thus we can ignore the third equation and the system is equivalent to
( ¯
−2x1 + x2 = −x3 ¯
¯
¯.
x1 + −2x2 = −x3 ¯

Here
¯ ¯ ¯ ¯
¯ −x 1 ¯ ¯ −2 −x3 ¯
¯ 3 ¯ ¯ ¯
∆ = 3, ∆1 = ¯¯ ¯ = 3x3 , ∆2 = ¯ ¯ = 3x3 ,
−x3 −2 ¯ ¯ 1 −x3 ¯

thus
3x3 3x3
x1 = = x3 , x 2 = = x3 .
3 3
So (x3 , x3 , x3 ) is a general solution of our system with exogenous variable x3 .
Taking this variable x3 = 1 we obtain the eigenvector x = (1, 1, 1). As we see
the geometric degree of eigenvalue λ = 3 is 1, as well as its algebraic degree.

Example. Find an eigenvector x corresponding to the eigenvalue λ = 0 of

the same matrix  
1 1 1
A=  1 1 1 .


1 1 1
from the previous example.
Solution. We can find x from the matrix equation (A − 0 · I) · x = 0 which
as a system of linear equations looks as
 ¯
¯
 (1 − 0)x1
 + x2 + x3 = 0 ¯
¯
x1 + (1 − 0)x2 + x3 = 0 ¯,

 ¯
x1 + x2 + (1 − 0)x3 = 0 ¯

 ¯
¯
 x1
 + x2 + x3 = 0 ¯
¯
x1 + x2 + x3 = 0 ¯.

 ¯
x1 + x2 + x3 = 0 ¯

Rank of the determinant of this system is 1, and its general solution is

(x1 = −x2 − x3 , x2 , x3 )

with exogenous variable x2 , x3 . Taking this variables x2 = 1, x3 = 0 we

obtain the eigenvector v = (−1, 1, 0), and taking this variables x2 = 0, x3 = 1
we obtain the eigenvector v = (−1, 0, 1). As we see the geometric degree of
eigenvalue λ = 0 is 2, as well as its algebraic degree.

7
Example. Find the eigenvalues and eigenvectors of the matrix
à !
2 2
A= .
1 3

Solution. The characteristic equation of the matrix A looks as

¯ ¯
¯ 2−λ 2 ¯
A = ¯¯ ¯
¯ = 0 , λ2 − 5λ + 4 = 0.
¯ 1 3−λ ¯

The roots of this equation, that is the eigenvalues are λ1 = 1, λ2 = 4.

The eigenvectors can be found solving the system of equations
(
(2 − λ)x1 + 2x2 = 0
x1 + (3 − λ)x2 = 0

For λ = 1:
( ¯ ¯
(2 − 1)x1 + 2x2 = 0 ¯ x1 + 2x2 = 0 ¯
¯ ¯
¯, ¯
x1 + (3 − 1)x2 = 0 ¯ x1 + 2x2 = 0 ¯

x1 + 2x2 = 0, x1 = 2x2 ,
thus the solution depending on the free parameter x2 is (2x2 , x2 ). Taking,
say, x2 = 1 we obtain the eigenvector v1 = (2, 1).
For λ = 4:
( ¯ ¯
(2 − 4)x1 + 2x2 = 0 ¯ −2x1 + 2x2 = 0 ¯
¯ ¯
¯, ¯
x1 + (3 − 4)x2 = 0 ¯ x1 − x2 = 0 ¯

x1 − x2 = 0, x1 = x2 ,
thus the solution depending on the free parameter x2 is (x2 , x2 ). Taking, say,
x2 = 1 we obtain the eigenvector v1 = (1, 1).

Example. Let à !
1 1
A= .
0 1
(horizontal shear).
Then |A−λI| = (1−λ)2 thus there is one eigenvalue λ = 1 of multiplicity
2. Eigenvectors are solutions of the system
à ! à ! à !
0 1 x 0
· =
0 0 y 0

that is (
0·x+1·y =0
.
0·x+0·y =0

8
The solution of this system is (x, 0), the x-axes, so the geometric multiplicity
of λ = 1 is 1, so it is less then its algebraic multiplicity.

Example. Let
à !
1 0
A= .
0 1
Then |A − λI| = (1 − λ)2 thus there is one eigenvalue λ = 1 of multiplicity
2. Eigenvectors are solutions of the system
à ! à ! à !
0 0 x 0
· =
0 0 y 0

that is (
0·x+0·y =0
.
0·x+0·y =0
The solution of this system is (x, y), the whole R2 so the geometric multi-
plicity of λ = 1 is 2, so it equals to its algebraic multiplicity.

2.1.1 Viett Theorem

Theorem 3 Suppose an n × n matrix A has n eigenvalues λ1 , ..., λn . Then
(i) The determinant of the matrix A equals to the product of eigenvalues

|A| = λ1 · ... · λn ;

(ii) The trace of a matrix A, i.e., the sum of the elements on the main
diagonal, equals to the sum of eigenvalues of A

tr(A) = a11 + ... + ann = λ1 + ... + λn .

à !
2 4
Example. Find the eigenvalues of the matrix A = .
1 2
Solution. The matrix is clearly singular (degenerate, |A| = 0). Therefore
λ1 = 0 is an eigenvalue (why?). By the trace rule λ1 + λ2 = 2 + 2 = 4, thus
λ2 = 4.

2.2 Linearly Independent Eigenvectors

Theorem 4 The eigenvectors of the matrix A corresponding to the different
eigenvalues are linearly independent.

More precisely, suppose λ1 , λ2 , ... λk are eigenvalues of A and λi 6= λj for

all i 6= j, and suppose v1 , ... , vk are corresponding eigenvectors, then they
are linearly independent.

9
 Let us check it for k = 2. We assume λ1 6= λ2 and Av1 = λ1 v1 , Av2 =
λ2 v2 . Suppose v1 , v2 are linearly dependent, say v2 = mv1 , then A · v2 =
A · kv1 = mA · v1 = mλ1 v1 , on the other hand side A · v2 = λ2 v2 = λ2 mv1 ,
thus m(λ1 − λ2 )v1 = 0, this contradicts to λ1 6= λ2 .

Corollary 1 Suppose an n × n matrix A has n different eigenvalues

λ1 , ... , λn . Then the corresponding eigenvectors x(1) , ... , x(n) form a (eigen)basis.

2.3 Representation of a Matrix in Terms of Eigenval-

ues and Eigenvectors
Suppose an n × n matrix A has n eigenvalues λ1 , ... , λn and
 (1)   (n) 
x1 x1
   
x(1) (n)
=  ...  , ... , x =  ... 
x(1)
n x(n)
n

are the corresponding linearly independent eigenvectors. Form two matrixes,

first the diagonal matrix whose diagonal elements are eigenvalues and the
second the matrix whose columns are eigenvectors
  (1)  (n) 
λ1 ... 0 x1 ... x1
Λ =  ... ... ...  , S =  ... ... ... 
  
.
0 ... λn (1)
xn ... xn(n)

Note that since of Theorem 4 the matrix S is invertible.

Theorem 5 A = S · Λ · S −1 .

Example. Find a 3 × 3 matrix A which eigenvalues and eigenvectors are:

λ1 = 3, x(1) = (−3, 2, 1)T ,

λ2 = −2, x(2) = (−2, 1, 0)T
λ3 = 1, x(3) = (−6, 3, 1)T .
   
3 0 0 −3 −2 −6
Solution. Λ =  0 −2 0 



S= 2 1 3  . Then
0 0 1 1 0 1
    −1
−3 −2 −6 3 0 0 −3 −2 −6
   
A = S · Λ · S −1 = 2 1 3   0 −2 0  ·  2 1 3  ,
1 0 1 0 0 1 1 0 1

which can be directly calculated. Ã !

100 41 −30
Example. Find the matrix A , where A = .
56 −41

10
Solution. First find eigenvalues and eigenvectors. The solution of the char-
acteristic equation gives
¯ ¯
¯ 41 − λ −30 ¯
A = ¯¯ ¯
¯ , λ2 − 1 = 0, λ1 = 1, λ2 = −1.
¯ 56 −41 − λ ¯

Furthermore, solving the suitable systems we obtain

à corresponding
! Ãeigenvec-
!
(1) T (2) T 1 0 3 5
tors x = (3, 4) , x = (5, 7) . Thus Λ = , S= .
0 −1 4 7
Then
A100 = (S · Λ · S −1 ) · (S · Λ · S −1 ) · ... · (S · Λ · S −1 ) = S · Λ100 · S −1 =
à ! à !100 à !−1
3 5 1 0 3 5
· · =
Ã
4 !7 Ã 0 −1 ! Ã
4 7 !
3 5 1100 0 7 −5
· 100 · =
Ã
4 7! Ã 0 !(−1) Ã !
−4 Ã 3 !
3 5 1 0 7 −5 1 0
· · = .
4 7 0 1 −4 3 0 1

2.4 Similar Matrices

Two matrices A and B are called similar if there exists an invertible matrix
S such that B = S −1 · A · S.

Theorem 6 Similarity of matrices is an equivalence relation.

Theorem 7 If A and B are similar, then

(i) |A − λI| = |B − λI|;
(ii) spec(A) = spec(B);
(iii) |A| = |B|;
(iv) rank(A) = rank(B);
(iii) tr(A) = tr(B).

2.5 Diagonalization of a Matrix

A square matrix A is called diagonalizable if it is similar to a diagonal matrix,
i.e. if there exists an invertible matrix S such that S −1 · A · S is a diagonal
matrix.

Theorem 8 If an n × n matrix A has n different eigenvalues then it is

diagonalizable.

Indeed, as we already know in this case A = S · Λ · S −1 . Then, multiplying

this equality by S −1 and S respectively from right and left we obtain

S −1 · A · S = S −1 · (S · Λ · S −1 ) · S = Λ,

11
which is diagonal matrix.
Thus the existence of n distinct eigenvalues is a sufficient condition for
diagonalizability, but not necessary:
à !
1 0
Example. The identity matrix is already diagonal, nevertheless it
0 1
has two equal eigenvalues λ1 = λ2 = 1. By the way, any vector v ∈ R2 is an
eigenvector.
Furthermore, there areà nondiagonalizable
! matrixes:
1 1
Example. The matrix has two equal eigenvalues λ1 = λ2 = 1
0 1
and the corresponding eigenvector is v = (1, 0), so in this case the algebraic
degree is 2 and the geometric degree is 1 (see above). This matrix is not
diagonalizable. Ã !
0 −1
Example. The matrix has no real eigenvalues, consequently no
1 0
eigenveqtors. This matrix is not diagonalizable.

Which n × n matrices are diagonlizable?

1. Matrices with n distinct eigenvalues.
2. Matrices with n linearly independent eigenvectors.
3. Symmetric matrices (A = At ).
Let us prove the last proposition for a 2 × 2 symmetric matrix
à !
a b
A= .
b d

First let us prove that A has only real eigenvalues:

¯ ¯
¯ a−λ b ¯
¯ ¯
|A−λI| = ¯¯ ¯ = (a−λ)·(d−λ)−b2 = λ2 −(a+d)·λ+ad−b2 = 0,
b d−λ ¯

the discriminant of this quadratic equation D = (a − d)2 + 4b2 ≥ 0, thus the

characteristic quadratic equation has only real roots.
Consider two cases.
1. Suppose we have a multiple rootà λ1 =!λ2 , it happens when D = 0, that is
a 0
if a = d, b = 0, in this case A = is already a diagonal matrix.
0 a
2. Now assume that λ1 6= λ2 . By Theorem above two distinct real eigenvalues
guarantee the diagonalizability.

12
 Exercises
à !
−1 3
1. Let .
2 0
(a) Check that λ Ã = 2!is an eigenvalue of A.
1
(b) Check that is a corresponding eigenvector of A.
1
(c) Find all eigenvalues and corresponding eigenvectors of 
A. 
1 0 2
 
2. Find the eigenvalues and eigenvectors for the matrix  0 5 0  .
à !
3 0 2
a b
3. Suppose is a Markov matrix, that is a + c = 1, b + d = 1.
c d
Show that λ = 1 is it’s eigenvector.  
a b c
4. Find eigenvalues of an upper-triangular matrix   0 d e .


0 0 f
5. For each of the following matrix A find diagonal matrix Λ and invertible
matrix S so that A = S · Λ · S −1
à ! à !
3 0 1 −1
(a) . (b) .
1 2 2 4
   
3 −1 0 4 −2 2
  
(c)  −1 2 −1 . (d)  0 1 0 
.
0 −1 3 1 0 1

Exercises 23.1-23.7, 23.15.

Homework
1. Exercise 23.2 Ã !
a b
2. Show that a 2 × 2 symmetric matrix has real eigenvalues. In
b d
which case it has just one eigenvalue?
à !
a b
3. Show that a 2 × 2 symmetric matrix has two orthogonal
b d
eigenvectors (hint: in the case of two eigenvalues λ1 6= λ2 consider the inner
product Av1 · v2 and use Av1 · v2 = v1 · AT v2 , in the case λ1 = λ2 characterize
A).
à !
a b
4. Show that each symmetric 2 × 2 matrix can be diagonalized
b d
by an orthogonal matrix P . √ √
5. Find a and b for which two vectors v1 = ( 22 , a) and v2 = (b, 22 ) form
an orthnormal basis of R2 .

13
 Summary

Linear map f : Rn → Rm : f (v + w) = f (v) + f (w), f (c · x) = c · f (x).

f (v) = A·v where A is a matrix whose columns are f (e1 ), ... , f (en ) ∈ Rm .
f : Rn → Rn is bijective iff det(A) 6= 0
λ ∈ R and a nonzero vector x ∈ Rn are called respectively eigenvalue
and eigenvector of A if A · x = λ · x.
spec(A) is the set of all eigenvalues of A.
Eigenspace of λ: the subspace spanned by all its eigenvectors.
The geometric degree of λ is dim of its eigenspace.
Eigenvalues of A are solutions of characteristic equation det(A−λI) =
0.
Eigenvectors of eigenvalue λ are solutions of (A − λI)v = 0.
Algebraic degree of λ∗ ∈ spec(A) is its multiplicity in det(A − λI) = 0.
Algebraic degree ≥ geometric degree.
Viett Theorem: If A has n eigenvalues λ1 , ... , λn then |A| = λ1 · ... · λn
and tr(A) = a11 + ... + ann = λ1 + ... + λn .
If {λ1 , λ2 , ... λk } = spec(A) and i 6= j ⇒ λi 6= λj then corresponding
eigenvectors v1 , ... , vk are lin. indep.
If A has n different eigenvalues, then corresponding eigenvectors form
eigenbasis.
If A has n eigenvalues λ1 , ... , λn and eigenbasis (x(1) , ... , x(n) ) then A =
   (1) (n) 
λ1 ... 0 x1 ... x1
SΛS −1 or Λ = S −1 AS where Λ =   
 ... ... ...  S =  ... ... ...  .


0 ... λn x(1)
n ... x(n)
n
A and B are similar if B = S −1 · A · S. In this case |A − λI| = |B −
λI|, spec(A) = spec(B), |A| = |B|, rank(A) = rank(B), tr(A) = tr(B).

14