APSC 174

Lecture Notes

c 2009-2017
Abdol-Reza Mansouri

Department of Mathematics and Statistics

Queen’s University
 About These Notes

Dear APSC-174 student,

These notes are meant to be detailed and expanded versions of the class-
room lectures you will be attending. Each section in these lecture notes will be
covered in one or (often) more classroom lectures.
These lecture notes will be slightly and gradually updated and completed
as the classroom lectures progress. Some parts of the lecture notes may even be
updated and slightly modified after the corresponding classroom lectures have
been delivered. It is therefore recommended that you always refer to the latest
version of these lecture notes, which will be posted on the course website.
It is strongly recommended that you read these lecture notes carefully
as a complement to the classroom lectures. Almost every section in these notes
contains examples which have been worked out in detail. It is strongly recom-
mended that you examine those in detail, and it is equally strongly recommended
that you work out in detail the examples that have been left to the reader. Last
but not least, it is strongly recommended that you attempt to solve the problems
at the end of each section.
I hope you will find these notes useful, and I would appreciate your
feedback. So please feel free to e-mail me at mansouri@[Link] . I look
forward to hearing from you.

Abdol-Reza Mansouri
January 2017
Contents

Section 0 1
Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
Quantifiers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
Mappings and Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Section 1 19
Systems of Linear Equations . . . . . . . . . . . . . . . . . . . . . 20
Number of Solutions of Systems of Linear Equations . . . . . . . 21
Applications of Systems of Linear Equations . . . . . . . . . . . . 24

Section 2 33

Section 3 45

Section 4 51

Section 5 59

Section 6 69

Section 7 79

Section 8 95

Section 9 107

Section 10 117

Section 11 131

Section 12 151

Section 13 157
Applications of Matrix Multiplication . . . . . . . . . . . . . . . 165

3
4 CONTENTS

Section 14 175

Section 15 193
Applications of eigenvalues and eigenvectors . . . . . . . . . . . . 203
Section 0

Study Topics
• A Review of Basic Notions

1
2 SECTION 0

In this section, we shall review some of the basic mathematical notions that we
will be using throughout the course.

Sets
One of the basic notions that we shall deal with throughout the course
is that of set. By a set we mean nothing other than a collection of objects;
these objects are then called the elements or points of the set. Some simple
examples of sets are:

• the set of all integers greater than or equal to 0 and less than or equal to
5,

• the set of all letters in the Greek alphabet,

• the set of all words in a given book,

• the set of all common words in the English language,

• the set of all common words in the French language,

• the set of all integers greater than or equal to 0 (denoted by N),

• the set of all integers (denoted by Z),

• the set of all real numbers greater than or equal to 0 and less than or equal
to 1,

• the set of all real numbers (denoted by R),

• the set of all complex numbers (denoted by C),

• ...

Note that in the list of examples given above, the first 5 sets in the list have a
finite number of elements; the last 5, on the other hand, have infinitely many
elements.

Let us now denote the first set in the list above by the capital letter A. By
definition of the set A, the elements (or points) of A are all the integers greater
than or equal to 0, and less than or equal to 5. We can therefore write the set
A explicitely as:

A = {0, 1, 2, 3, 4, 5}.

The curly brace “{” to the left indicates the beginning of the set, and the curly
brace “}” to the right indicates its end. The symbols enclosed by the curly
braces and separated by commas “,” are the elements of the set (namely the
integers 0, 1, 2, 3, 4 and 5).
SETS 3

As far as a set goes, the order in which its elements are written is of no conse-
quence; hence, we can equivalently write the set A as:

A = {2, 5, 0, 4, 3, 1}

or as

A = {0, 5, 2, 1, 3, 4}

or any other permutation of the elements. On the other hand, when writing
down the elements of the set, we shall not allow any repetition of any element;
for example, we shall never write A as {0, 0, 1, 2, 3, 4, 5} or as {0, 1, 2, 3, 4, 5, 4},
or in any other such manner: Each element of the set must appear only once.

Let now S be any set; if some entity, say x, is an element of the set S, then we
write

x∈S

to denote this membership property. If x is not an element of S, then we write

x∈
/S

to denote this. For example, for our set A above, we can write 0 ∈ A, since the
integer 0 is an element of A. On the other hand, the integer 8 is not an element
of A, and therefore we write 8 ∈
/ A. Similarly, the integer 5 is an element of the
set N, and we can therefore write 5 ∈ N.√On the other hand, the real number
√
2 is not an element of N, and we write 2 ∈ / N.

There is a set containing no elements; it is called the empty set, and it is

denoted by the symbol ∅ (it is also often denoted by {}).

Now that we have defined what we mean by a set and what we mean by an
element of a set, we can define what we mean by two sets being equal:
Definition 1. Let A and B be two sets. A is said to be equal to B if A and B
have exactly the same elements. We write this as A = B.
In other words, we have A = B if and only if the following two conditions are
met:
(i) Every element of A is also an element of B, and
(ii) every element of B is also an element of A.
For example, defining the set E as E = {0, 1, 2, 3} and the set F as F =
{2, 1, 3, 0}, we can write

E=F
4 SECTION 0

since E and F have exactly the same elements, and hence, by our definition, are
equal (recall again that, for a set, the order in which its elements are listed is
of no importance). On the other hand, defining the set G as G = {3, 0, 2, 7, 1},
we can write

E 6= G,

since E and G do not have exactly the same elements; in particular, we can see
that 7 ∈ G whereas 7 ∈/ E, and this shows that the elements of E and G are
not exactly the same.

Writing down the elements of a set explicitly is sometimes plainly impossible.

Let B for example be the set of all integers greater than or equal to 5. How
shall we describe B ? Certainly not by listing all its elements! Recalling that N
denotes the set of all integers greater than or equal to 0, we can write:

B = {x ∈ N : x ≥ 5}.

The expression to the right of the equality sign should be read as “the set of
all elements x in N such that x ≥ 5”; in that expression, the colon “:” is to be
read as “such that”. It is worth noting that in some texts the vertical bar “|”
is used in place of the colon “:”, and hence the set B could also be written as

B = {x ∈ N | x ≥ 5},

with the vertical bar “|” to be read as “such that” just like we did with the
semicolon. These are some of the standard ways of defining sets. Note that
this way of defining sets is not restricted only to sets having infinitely many
elements; for example, defining the set A as:

A = {5, 2, 3, 4, 6},

we could also write A as

A = {x ∈ N : x ≥ 2 and x ≤ 6};

indeed, it is immediate to verify that we have the equality of sets

{5, 2, 3, 4, 6} = {x ∈ N : x ≥ 2 and x ≤ 6},

since every element of the set to the left is also an element of the set to the
right, and vice-versa. Similarly, we also have the equalities

A = {x ∈ N : x > 1 and x < 7},

and

A = {x ∈ N : x > 1 and x ≤ 6},

SETS 5

and so on. These are all equivalent ways of defining the set A.

As another example, consider the set [0, 1] consisting of all real numbers greater
than or equal to 0 and less than or equal to 1. We can write:

[0, 1] = {x ∈ R : x ≥ 0 and x ≤ 1}.

We now define relations and operations between sets.

Definition 2. Let A and B be two sets. A is said to be a subset of B if every
element of A is also an element of B; we write this as A ⊂ B.
For example, every element of N is also and element of R, i.e. N is a subset of
R, and we can therefore write

N ⊂ R,

On the other hand, it is not true that every element of R is also an element of
N, and hence R is not a subset of N.
Using the “subset” relation, we can therefore state that two sets A and B are
equal if and only if we have both A ⊂ B and B ⊂ A; we can write this more
formally as:

A = B ⇔ A ⊂ B and B ⊂ A.

In the expression above, the symbol ⇔ should be read as “if and only if” or “is
equivalent to”.

Given two sets S and T , we can construct a new set from them as follows:
Definition 3. Let S and T be sets. We denote by S ∩ T the set of all elements
which are both in S and in T . We call S ∩ T the intersection of the sets S and
T.
For example, if A = {0, 1, 2, 3}, B = {2, 3, 4, 5}, and C = {5, 6, 7}, then we can
write:

A∩B = {2, 3},

B∩C = {5},
A∩C = ∅.

The following properties of set intersection are easy to verify (and strongly
recommended to the reader):
Lemma 1. Let S, T, U be sets. We have:

S ∩T = T ∩ S,
S ∩ (T ∩ U ) = (S ∩ T ) ∩ U.
6 SECTION 0

Given two sets S and T , we can construct a new set from them in yet another
way:
Definition 4. Let S and T be sets. We denote by S ∪ T the set of all elements
which are in either S or T or both. We call S ∪ T the union of the sets S and T .
For example, for the sets A, B, C defined just above, we have:

A∪B = {0, 1, 2, 3, 4, 5},

B∪C = {2, 3, 4, 5, 6, 7},
A∪C = {0, 1, 2, 3, 5, 6, 7}.

The following properties of set union are easy to verify (and their verification is
again strongly recommended to the reader):
Lemma 2. Let S, T, U be sets. We have:

S∪T = T ∪ S,
S ∪ (T ∪ U ) = (S ∪ T ) ∪ U.

We shall construct new sets in yet another way:

Definition 5. Let S and T be sets. We denote by S \ T the set of all elements of
S which are not in T . We call S \ T the set difference of the sets S and T (in
that order).
For example, for the same sets A, B, C defined just above by A = {0, 1, 2, 3},
B = {2, 3, 4, 5}, and C = {5, 6, 7}, we have:

A\B = {0, 1},

B\A = {4, 5},
A\C = A,
C \A = C,
B\C = {2, 3, 4},
C \B = {6, 7}.

It is easy to verify that for any set S, we have S \ S = ∅.

The following properties of set difference are again easy to verify (and their
verification is again strongly recommended to the reader):
Lemma 3. Let S, T, U be sets. We have:

S \ (T ∩ U ) = (S \ T ) ∪ (S \ U ),
S \ (T ∪ U ) = (S \ T ) ∩ (S \ U ).

We shall construct new sets in yet another way:

SETS 7

Definition 6. Let S and T be sets. We denote by S × T the set of all pairs of the
form (s, t) where s ∈ S and t ∈ T . We call S × T the Cartesian product of the
sets S and T .

NOTE: It is important to recall that a pair (s, t) is an ordered list consisting

(in that order) of s and t; in a pair of elements, order does matter. As a result,
two pairs (a, b) and (c, d) are considered equal if and only if we have both a = c
and b = d. Hence, the pair (a, b) is not equal to the pair (b, a), unless we have
a = b; if a 6= b, then the pairs (a, b) and (b, a) denote distinct pairs and are
not equal. Hence, a pair of elements is very different from a set of elements,
since, as we saw, in a set, order does not matter. So if a 6= b, the sets {a, b} and
{b, a} will denote identical sets, whereas the pairs (a, b) and (b, a) will denote
distinct pairs. One other key difference is that in a pair of elements, the two
elements are allowed to be identical (i.e. repetitions are allowed), whereas this
is not the case for a set. So, the pair of integers (1, 1) is a valid pair, whereas the
set {1, 1} is not correctly written (since the element 1 is repeated) and should
be instead written as {1}.

Let us consider some examples. If E and F are sets defined by E = {0, 1, 2}

and F = {4, 5}, then we have the following Cartesian products:

E×E = {(0, 0), (0, 1), (0, 2), (1, 0), (1, 1), (1, 2), (2, 0), (2, 1), (2, 2)},
E×F = {(0, 4), (0, 5), (1, 4), (1, 5), (2, 4), (2, 5)},
F ×E = {(4, 0), (5, 0), (4, 1), (5, 1), (4, 2), (5, 2)},
F ×F = {(4, 4), (4, 5), (5, 4), (5, 5)}.

Note that if S and T are two sets, the set S × T is in general distinct from the
set T × S (as you can see on the previous example). Note also that if either S
or T is the empty set then S × T will also be the empty set.

We can use the Cartesian product operation to construct yet more sets from
a given set S as follows: We can construct not just S × S (as we did above),
but also (S × S) × S, and ((S × S) × S) × S, and ... We shall write these last
ones simply as S × S × S, and S × S × S × S, and ... (i.e. by removing the
parentheses). So, just as we had S × S be the set of all pairs (s1 , s2 ) with s1
and s2 elements of S, we will have:

• S × S × S = set of all triples (s1 , s2 , s3 ) with s1 , s2 , s3 elements of S,

• S × S × S × S = set of all quadruples (s1 , s2 , s3 , s4 ) with s1 , s2 , s3 , s4

elements of S,

• ···

• S × S × · · · × S (n times) is the set of all n-tuples (s1 , s2 , · · · , sn ) with

s1 , s2 , · · · , sn elements of S.
8 SECTION 0

We shall denote the n−fold Cartesian product S × S × · · · × S of S with itself by

S n , with the convention that S 1 = S. Generalizing what we saw for pairs, two
n−tuples (a1 , · · · , an ) and (b1 , · · · , bn ) in S n are said to be equal if and only if
we have a1 = b1 , a2 = b2 , · · · , an = bn .

We shall very soon encounter the following sets:

• R2 , the set of all pairs of real numbers, i.e. the set of all pairs of the form
(a, b) where a, b ∈ R (shorthand for “a and b are both elements of R”),

• R3 , the set of all triples of real numbers, i.e. the set of all triples of the
form (a, b, c) where a, b, c ∈ R,

• and more generally, Rn , the set of all n-tuples of real numbers, i.e. the
set of all n−tuples of the form (a1 , a2 , · · · , an ) with a1 , a2 , · · · , an ∈ R.

Quantifiers
Let S be a set, and suppose all elements of S satisfy some property. We want
to express this fact formally. To make this more concrete, consider the set N of
all integers greater than or equal to 0; it is clear that every element x of N is
such that x + 1 is also an element of N. We can write this statement formally
(and succintly) as:

∀x ∈ N : x + 1 ∈ N.

The symbol ∀ in the expression above is to be read as “for any”, or, equivalently,
as “for all”. The symbol ∀ is called the universal quantifier. The colon symbol
“:” in the above expression should be read as “we have” or “the following holds”.

Using the universal quantifier, many otherwise length statements can be written
formally in a very succint way. For example, if S, T are two sets, we know that
S ⊂ T if and only if every element of S is also an element of T . We can write
this formally as:

S ⊂ T ⇔ ∀x ∈ S : x ∈ T

where again the symbol of equivalence ⇔ is meant to be read as “if and only
if” or “is equivalent to”.

Let now S be a set, and suppose there is at least one element in S which
satisfies some property. We want to express this fact formally. Again, to make
this more concrete, consider the set N of all integers greater than or equal to 0;
it is clear that there are elements in N (namely all multiples of 2) which, after
division by 2, yield again an integer in N. Clearly, this is not true of all elements
in N (consider any odd integer). We can write the statement “there exists at
MAPPINGS AND FUNCTIONS 9

least one element in N such that dividing it by 2 yields again an element of N”

as:
x
∃x ∈ N : ∈ N.
2
In the above expression, the symbol ∃ should be read as “there exists”, and is
called the existential quantifier. The colon symbol “:” in the above expres-
sion should be read as “such that”.

Using the existential quantifier also allows writing expressions formally and
succintly. For example, if S, T are two sets, then we know their intersection
S ∩ T is non-empty if and only if (by definition of intersection), there is an
element in S which is also an element in T . We can write this formally as:

S ∩ T 6= ∅ ⇔ ∃x ∈ S : x ∈ T

Mappings and Functions

The following definition is fundamental:

Definition 7. Let S and T be two sets. A mapping (also called function) from
S to T is a rule which assigns to each element of S one and only one
element of T .

To indicate that f is a function from S to T , we shall write:

f :S → T
x 7→ f (x)

The above expression should be read as: “f is a mapping from the set S to the
set T which assigns to every element x of S the element f (x) of T ”.
In the expression above, S is called the domain set of the function f , T is
called the target set of f . Each function is uniquely defined by specifying its
domain set, its target set, and the rule by which it associates to each element
of the domain set a unique element of the target set.

Let us immediately consider some examples:

1. The mapping

f :Z → N
x 7 → x2

is the mapping with domain set Z and target set N, defined by assigning
to each integer x its squared value x2 .
10 SECTION 0

2. The mapping
f :R → R
x 7 → sin(x)
is the mapping with domain set and target set both equal to R, defined
by assigning to each real number x its sine sin(x).
3. The mapping
f :N×N → R
x
(x, y) 7→
y2 +1
is the mapping with domain set the Cartesian product N × N (i.e. N2 ) and
target set R which assigns to every pair (x, y) of integers the real number
x
y 2 +1 .

4. The mapping
f : R3 → R2
(x, y, z) 7→ (x + y, x − z)
is the mapping with domain set R3 and target set R2 which assigns to every
triple (x, y, z) of real numbers the pair (x + y, x − z) of real numbers.

It is important to note that given two sets S and T , not every rule of assignement
from S to T defines a function; the rule of assignment will define a function from
S to T if to every element of S it associates a unique element of T . To make
this clear, consider the set N for both the domain set and the target set. Suppose
to each element x of N we assign an element y of N if x is a multiple of y (i.e.
if y divides x). For example, we would assign to the element 2 of N both the
element 1 of N (since 2 is a multiple of 1) and the element 2 of N (since 2 is a
multiple of 2), and so on. Does this rule of assignment define a function from
N to N ? NO! For the simple reason that some elements of the domain set (for
example 2) are assigned to more than one element of the target set.
Consider now the rule of assignment defined by assigning to every element x of
N the element y of N if the relation y = x − 5 is satisfied. Does this rule of
assignment define a function from N to N ? NO! For the simple reason that some
elements of the domain set (namely the integers 0, 1, 2, 3, 4 are not assigned to
any element at all of the target set.

It is important to point out that a mapping is defined by the data consisting of

its domain set, its target set, and the law by which each element of the domain
set is assigned a unique element of the target set. As a result, the two mappings
f and g defined by
f :N → N
x 7 → x3
MAPPINGS AND FUNCTIONS 11

and
g:N → R
x 7→ x3
are considered to be distinct mappings (even though they both map each
integer to its cube) since they do not have the same target set in their definition.
In what follows, we shall assume that all the domain sets and target sets involved
in the discussion are non-empty (since there would be not much to say if they
were!).

It is important to a given mapping f : S → T from some set S to some set T is

an entity in its own right; we can therefore talk about the set of all mappings
from S to T . A given mapping from S to T will therefore be some element of
that set.

Let now S, T be sets, and let f : S → T be a mapping from S to T . It may

happen that distinct elements get mapped under f to distinct elements of T .
This case is important enough to warrant a definition.
Definition 8. Let f : S → T be a mapping from S to T . f is said to be injective
(or one-to-one) if ∀x, y ∈ S: x 6= y ⇒ f (x) 6= f (y).
Equivalently, f : S → T is injective if ∀x, y ∈ S : f (x) = f (y) ⇒ x = y.

For example, the mapping

f :R → R
x 7→ x3
is injective, whereas the mapping
g:R → R
x 7→ x2
is not.

If f : S → T is some mapping, it may also happen that each element of T is

mapped to under f by some element of S (i.e. no element of T is left out). This
case is also important enough to warrant a definition.
Definition 9. Let f : S → T be a mapping from S to T . f is said to be surjective
(or onto) if ∀x ∈ T , ∃y ∈ S : x = f (y), i.e. every element of T is the image under
f of some element of S.

For example, the mapping

f :Z → Z
x 7→ x + 1
12 SECTION 0

is surjective, whereas the mapping

g:N → N
x 7→ x + 1

is not.

Let now S, T, U be three sets, and let f : S → T and g : T → U be two

mappings. We can construct a new mapping from f and g as follows:

Definition 10. Let f : S → T and g : T → U be mappings. We define the

composition g ◦ f of g and f to be the mapping with domain set S and target
set U , and which assigns to every element x of S the element g(f (x)) of U .

In other words, the mapping g ◦ f is defined as:

g◦f :S → U
x 7→ g(f (x))

It is clear why g ◦ f is called the composition of g and f : an element x of S is

first mapped to the element f (x) of T , and g then maps the element f (x) of T
to the element g(f (x)) of U .
It is important to note from this definition that one cannot always compose any
two arbitrary functions f and g; for the composition g ◦ f to make sense, the
target set of f has to be equal to the domain set of g.

Let now S be any set; we denote by idS the mapping

idS : S → S
x 7→ x

i.e., the mapping which assigns to each element of S itself. idS is called the
identity mapping of S for the obvious reason that it maps each element of S
to itself. It is easy to verify that for any set S, the identity mapping idS of S is
both injective and surjective.

Let now S, T be two sets, and let f : S → T be a mapping. The composition

f ◦ idS is defined, since the target set of idS (namely S) is equal to the domain
set of f , and the domain and target set of f ◦ idS are exactly those of f .
Furthermore, ∀x ∈ S:

f ◦ idS (x) = f (idS (x)) = f (x),

and hence f ◦idS = f , i.e. the two mappings f ◦idS and f are one and the same,
since they have identical domain and target sets, and since they have the same
COMPLEX NUMBERS 13

exact rule of assignement. It is possible to verify in the same exact manner that
the composition idT ◦ f is defined, and that idT ◦ f = f .

Let now f : S → T be a mapping from S to T ; an important special case

happens when each element of the target set T is mapped to under f by exactly
one element of the domain set S; clearly this implies that f is both injective
and surjective. This case is also important enough to warrant a definition.
Definition 11. Let f : S → T be a mapping from S to T . f is said to be bijective
if ∀x ∈ T , there exists one and only one element y ∈ S such that x = f (y).
Let now f : S → T be a mapping from S to T and assume f is bijective. The
fact that f is bijective now allows us to define a mapping g : T → S (note that
g has domain set T and target set S, i.e. the exact opposite of f ) as follows:
For each x ∈ T , we know there is exactly one element y ∈ S which satisfies
f (y) = x; we define g(x) to be precisely that element y of S. In other words
(and don’t forget again that f is assumed bijective), for each x ∈ T we define
by g(x) that unique element of S which satisfies f (g(x)) = x. It is easy to verify
that, with this definition, g is a mapping from T to S, and that, furthermore, g
itself is bijective. Note that the compositions f ◦ g : T → T and g ◦ f : S → S
are both defined, and that ∀x ∈ S: g ◦ f (x) = x, and ∀x ∈ T : f ◦ g(x) = x. This
shows that f ◦ g = idT and g ◦ f = idS . For this reason, g is called the inverse
mapping of f and is denoted by f −1 (not to be confused with “one over ...”!).
Once again, the inverse mapping f −1 of f exists only and only when f is
bijective.

Complex Numbers
We will be dealing with complex numbers only at the very end of this course
(when eigenvalues and eigenvectors begin to appear), and since it is assumed
here that you have learned or soon will learn complex numbers in different
settings, we keep the treatment of complex numbers to the bare minimum.
Formally, a complex number can be considered as a pair of real numbers, i.e.
an element of R2 , with addition of pairs and multiplication of pairs defined as
follows:

∀(a, b), (c, d) ∈ R2 : (a, b) + (c, d) = (a + c, b + d),

2
∀(a, b), (c, d) ∈ R : (a, b) × (c, d) = (ac − bd, ad + bc).

It can be verified that two operations satisfy the usual properties of addition
and multiplication that are satisfied for real numbers. The set of all complex
numbers is denoted by C.
Informally, you can think of a complex number as a number of the form a + ib,
where a and b are real numbers, and where the symbol i satisfies i2 = −1 (i.e.
i is the square root of −1). Adding and multiplying complex numbers then
proceeds just as with real numbers, with the proviso that i2 = −1.
14 SECTION 0

For example, adding the complex numbers a+ib and c+id (where again a, b, c, d
are real numbers) yields (using the same grouping rules as for real numbers)

(a + ib) + (c + id) = (a + c) + i(b + d),

and multiplying them yields (using again the same grouping rules as for real
numbers)

(a + ib)(c + id) = a(c + id) + ib(c + id) = ac + a(id) + ibc + ib(id)

= ac + iad + ibc + i2 bd = ac + iad + ibc − bd
= (ac − bd) + i(ad + bc),

where, in the second to last step, we have replaced i2 by −1, as stipulated. As

an exercise, try to relate the expressions obtained here to the ones above with
the addition and multiplication of pairs!

For the complex number z = a + ib, where a, b ∈ R, the real number a is called
the real part of z, whereas the real number b is called the imaginary part of
z. Note that the complex number i can be written as

i = 0 + i1,

which shows that i has real part 0 and imaginary part 1. On the other hand, if
a ∈ R is any real number, we can write a as

a = a + i0,

which shows that the real number a has real part a (i.e. is equal to its own real
part) and has imaginary part 0.

As a very simple example of manipulation of real numbers, we all know that for
a, b ∈ R, we have the factorization:

a2 − b2 = (a − b)(a + b).

Using complex numbers, we can now factorize a2 + b2 as follows:

a2 + b2 = (a − ib)(a + ib).

Assume now that we wish to find a complex number z which satisfies z 2 + 1 = 0.

Since

z 2 + 1 = (z + i1)(z − i1) = (z + i)(z − i),

there are only two complex numbers which satisfy the equation z 2 + 1 = 0,
namely the complex numbers i and −i. Similarly, if we wish to find a complex
number z which satisfies z 2 + 2 = 0, we can write
√ √
z 2 + 2 = (z + i 2)(z − i 2),
COMPLEX NUMBERS 15

√ √
and hence the roots of the equation z 2 + 2 = 0 are given by i 2 and −i 2.

Let now a, b ∈ R, and consider the complex number z = a + ib. The complex
conjugate of z is the complex number denoted by z̄ and defined by z̄ = a − ib.
Note that we have:

z z̄ = (a + ib)(a − ib) = a2 + b2 .

Using the complex conjugate z̄ of z, we can easily compute 1z for z 6= 0. Note

first that with z = a + ib (with a, b ∈ R), having z 6= 0 is equivalent to having
a = b = 0, and hence z 6= 0 is equivalent to having a, b not both equal to 0. Let
then z = a + ib (again, with a, b ∈ R) with z 6= 0. We can write:
1 1 z̄ a − ib a b
= = = 2 2
= 2 2
−i 2 ,
a + ib z z z̄ a +b a +b a + b2
1 a b
which shows that a+ib has real part a2 +b2 and imaginary part − a2 +b2 (don’t

forget the minus sign!).

As an example, we can easily compute 1i , and we obtain:

1 1
= = −i,
i 0 + i1
1
whereas 1+i yields

1 1 1 1
= = −i .
1+i 1 + i1 2 2
PROBLEMS:

1. (a) Let A = {0, 1, 2}, B = {2, 3, 4}. Compute A ∩ B, A ∪ B, A × B,

B × A.
(b) Let A = ∅, B = {1, 2}. Compute A ∩ B, A ∪ B, A × B, B × A.
(c) Let A = {0, 1, 2}, B = ∅. Compute A ∩ B, A ∪ B, A × B, B × A.
(d) Let A = {0, 1, 2}, B = N. Compute A ∩ B, A ∪ B, A × B, B × A.
(e) Let A = {0, 1, 2}, B = R. Compute A ∩ B, A ∪ B, A × B, B × A.
(f) Let A = N, B = R. Compute A ∩ B, A ∪ B, A × B, B × A.
(g) Let A = N, B = Z. Compute A ∩ B, A ∪ B, A × B, B × A.
(h) Let A = Z, B = R. Compute A ∩ B, A ∪ B, A × B, B × A.
2. (a) Let A, B be sets; show that A ∪ B = B ∪ A.
(b) Let A, B be sets; show that A ∩ B = B ∩ A.
16 SECTION 0

(c) Let A, B be sets; show that A ⊂ A ∪ B.

(d) Let A, B be sets; show that A ∩ B ⊂ A.
(e) Let A, B, C be sets; show that (A ∩ B) ∩ C = A ∩ (B ∩ C).
(f) Let A, B, C be sets; show that (A ∪ B) ∪ C = A ∪ (B ∪ C).
(g) Let A, B, C be sets; show that (A ∪ B) ∩ C = (A ∩ C) ∪ (B ∩ C)
(h) Let A, B, C be sets; show that (A ∩ B) ∪ C = (A ∪ C) ∩ (B ∪ C)
(i) Let A, B be sets; show that A ⊂ B ⇒ A ∩ B = A.
(j) Let A, B be sets; show that A ⊂ B ⇒ A ∪ B = B.
(k) Let A, B be sets; show that A ∩ B = A ∪ B ⇒ A = B.
3. For each of the cases below, determine whether the given rule of assign-
ment defines a valid function with domain set and target set both equal to
N; in case it does, determine whether the function is injective, surjective,
bijective.
(a) To each x ∈ N, we assign x.
(b) To each x ∈ N, we assign x2 .
(c) To each x ∈ N, we assign x − 1.
(d) To each x ∈ N, we assign x3 .
x
(e) To each x ∈ N, we assign3.
√
(f) To each x ∈ N, we assign x.
(g) To each x ∈ N, we assign sin(x).
4. For each of the cases below, determine whether the given rule of assign-
ment defines a valid function with domain set and target set both equal to
Z; in case it does, determine whether the function is injective, surjective,
bijective.
(a) To each x ∈ Z, we assign x.
(b) To each x ∈ Z, we assign x4 .
(c) To each x ∈ Z, we assign x − 1.
(d) To each x ∈ Z, we assign x5 .
x
(e) To each x ∈ Z, we assign5.
√
(f) To each x ∈ Z, we assign x + 1.
(g) To each x ∈ Z, we assign cos(x).
5. For each of the cases below, determine whether the given rule of assign-
ment defines a valid function with domain set equal to N2 and target set
N; in case it does, determine whether the function is injective, surjective,
bijective.
(a) To each pair (x, y) ∈ N2 , we assign x + y.
COMPLEX NUMBERS 17

(b) To each pair (x, y) ∈ N2 , we assign x − y.

(c) To each pair (x, y) ∈ N2 , we assign xy.
x
(d) To each pair (x, y) ∈ N2 , we assign y.
x
(e) To each pair (x, y) ∈ N2 , we assign y+1 .
x
(f) To each pair (x, y) ∈ N2 , we assign y 2 +1 .

6. For each of the cases below, determine whether the given rule of assign-
ment defines a valid function with domain set equal to N2 and target set
R; in case it does, determine whether the function is injective, surjective,
bijective.

(a) To each pair (x, y) ∈ N2 , we assign x + y.

(b) To each pair (x, y) ∈ N2 , we assign x − y.
(c) To each pair (x, y) ∈ N2 , we assign −xy.
x
(d) To each pair (x, y) ∈ N2 , we assign y.
x
(e) To each pair (x, y) ∈ N2 , we assign y+1 .
x
(f) To each pair (x, y) ∈ N2 , we assign y 2 +1 .

7. For each of the following cases, determine whether the compositions f ◦ f ,

f ◦ g, g ◦ f , and g ◦ g are defined; if so, explicit them.

(a)

f :Z → Z
x 7→ x + 1

and

g:Z → R
x2 − 1
x 7→
x2 + 1

(b)

f :Z → Z
x 7→ x2

and

g:N → R
1
x 7→
x+1
18 SECTION 0

(c)

f :R → R
x 7 → x2

and

g:R → R
1
x 7→
x2 +1

(d)

f :N → Z
x 7 → x2

and

g:Z → N
x 7→ x3

(e)

f :R → R
1
x 7→
x2 +4
and

g:Z → R
1
x 7→
x2 + 2
Section 1

Study Topics
• Systems of Linear Equations
• Number of Solutions to Systems of Linear Equations

19
20 SECTION 1

Systems of Linear Equations

Linear algebra is ubiquitous in engineering, mathematics, physics, and
economics. Indeed, many problems in those fields can be expressed as problems
in linear algebra and solved accordingly. An important example of application
of linear algebra which we will use as an illustration throughout these lectures,
and as a motivation for the theory, is the problem of solving a system of linear
equations. Systems of linear equations appear in all problems of engineering; for
example, the problem of computing loop currents or node voltages in a passive
electric circuit, as well as computing forces and torques in a mechanical system
lead to systems of linear equations.
By a system of linear equations (with real coefficients), we mean a set
of equations of the form:


 a11 x1 + a12 x2 + a13 x3 + · · · a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + · · · a2n xn = b2
(E)

 a31 x1 + a32 x2 + a33 x3 + · · · a3n xn = b3

am1 x1 + am2 x2 + am3 x3 + · · · amn xn = bm
where the real numbers a11 , a12 , · · · , a1n , a21 , a22 , · · · , a2n , · · · , am1 , am2 , · · · , amn
and b1 , b2 , · · · , bm are given real numbers, and we wish to solve for the real
numbers x1 , x2 , · · · , xn ; equivalently, we wish to solve for the n−tuple of real
numbers (x1 , x2 , · · · , xn ). Each n−tuple (x1 , x2 , · · · , xn ) of real numbers satis-
fying the system (E) is called a solution of the system of linear equations (E).
In this system, the integer m is called the number of equations, whereas the
integer n is called the number of unknowns. Here are a few examples of
systems of linear equations:
(a) Consider the system

2x = 3,
where we wish to solve for the real number x; this is a system of linear
equations with one equation (i.e. m = 1) and one unknown (i.e. n = 1).
(b) Consider now the system

2x = 3,
3x = 4,
where again we wish to solve for the real number x; this is a system of
linear equations with two equations (i.e. m = 2) and one unknown (i.e.
n = 1).
(c) Consider now the system

2x + y = 2,
x − y = 3,
where we now wish to solve for the real numbers x and y, or, equivalently,
for the pair of real numbers (x, y); this is a system of linear equations
with two equations (i.e. m = 2) and two unknowns (i.e. n = 2).
 21

(d) Consider now the system


x + y + z = 0,
y − z = 3,

where we now wish to solve for the real numbers x, y and z, or, equivalently,
for the triple of real numbers (x, y, z); this is a system of linear equations
with two equations (i.e. m = 2) and three unknowns (i.e. n = 3).

Number of Solutions of Systems of Linear Equations

Now that it is clear what we mean by a system of m linear equations
in n unknowns, let us examine some examples. Let us begin with the simplest
possible example, that is, a system of linear equations with one equation (m = 1)
and one unknown (n = 1).
To start things off, consider the following system:

(A) 2x = 3,

where we wish to solve for the unknown real number x. Multiplying both sides
of the equation 2x = 3 by 21 , we obtain the equation:
1 1
2x = 3,
2 2
which, upon simplification, yields:
3
x= .
2
We conclude that the system of linear equations (A) has a unique solution, and
it is given by the real number x = 32 .
Let us still deal with the simplest case of systems of linear equations with
one equation and one unknown (i.e. m = n = 1), and consider now the system:

(B) 0x = 0,

where we again wish to solve for the real number x. It is clear that any real
number x satisfies the equation 0x = 0; we conclude therefore that the system
(B) has infinitely many solutions.
Still remaining with the simplest case of systems of linear equations with one
equation and one unknown (i.e. m = n = 1), consider now instead the system:

(C) 0x = 2,

where we again wish to solve for the real number x. It is clear that there is no
real number x that satisfies the equation 0x = 2; we conclude therefore that
the system (C) has no solution.
We can recapitulate our findings for the three examples that we considered as
follows:
22 SECTION 1

• System (A) has exactly one solution.

• System (B) has infinitely many solutions.

• System (C) has no solution.

Let us now increase the complexity by one notch and examine systems of
linear equations with 2 equations and 2 unknowns (i.e. m = n = 2). To begin
with, let us consider the following system:

x + y = 2,
(A′ )
x − y = 1,

where we wish to solve for the pair (x, y) of real numbers. If the pair (x, y)
is a solution of the system (A′ ), then we obtain from the first equation that
x = 2 − y and from the second equation that x = 1 + y. Hence, if the pair (x, y)
is a solution of the system (A′ ), then we must have x = 2 − y and x = 1 + y;
this in turn implies that we must have 2 − y = 1 + y, which then implies that
2y = 1, from which we obtain that y = 12 . From the relation x = 2 − y, we
then obtain that x = 2 − 12 = 23 . Note that we could have also used the relation
x = 1 + y instead, and we would still have obtained x = 32 . What we have
therefore established is that if the pair (x, y) is a solution of the system (A′ ),
then we must have x = 32 and y = 12 , i.e., the pair (x, y) must be equal to the
pair ( 23 , 21 ); conversely, it is easy to verify, by simply substituting in values, that
the pair ( 32 , 21 ) is actually a solution to the system (A′ ); indeed, we have:

3 1 4
+ = = 2,
2 2 2
3 1 2
− = = 1,
2 2 2

as expected. We conclude therefore that the system (A′ ) of linear equations has
exactly one solution, and that this solution is given by the pair ( 32 , 12 ).
Consider now the following system of linear equations, again having 2 equa-
tions (i.e. m = 2) and 2 unknowns (i.e. n = 2):

x + y = 1,
(B ′ )
2x + 2y = 2,

where again we wish to solve for the pair (x, y) of real numbers. Applying the
same “method” as previously, we obtain the relation x = 1 − y from the first
equation and the relation 2x = 2 − 2y from the second equation; multiplying
both sides of this last relation by 12 yields, upon simplification x = 1 − y.
Proceeding as before, we use the two relations obtained, namely x = 1 − y and
x = 1 − y, obtaining as a result the equation 1 − y = 1 − y, which then yields
 23

0y = 0. Clearly, any real number y satisfies the relation 0y = 0, and for each
such y, the corresponding x is given by x = 1 − y. In other words, if the pair
(x, y) is a solution of the system (B ′ ), then we must have x = 1−y, i.e., we must
have (x, y) = (1 − y, y). Conversely, for any real number y, the pair (1 − y, y)
is a solution of the system (B ′ ), as can be directly verified. Indeed,

(1 − y) + y = 1,
2(1 − y) + 2y = 2 − 2y + 2y = 2,

as expected. We conclude from these manipulations that the system (B ′ ) of

linear equations has infinitely many solutions, and they are all of the form
(1 − y, y) with y any real number.
Consider now the following system of linear equations, again having 2 equa-
tions (i.e. m = 2) and 2 unknowns (i.e. n = 2):

′ x + y = 1,
(C )
2x + 2y = 0,

where again we wish to solve for the pair (x, y) of real numbers. Applying the
same “method” as previously, we obtain the relation x = 1 − y from the first
equation, and the relation 2x = −2y from the second equation; multiplying both
sides of this last relation by 12 yields, upon simplification, the relation x = −y.
We have therefore that if the pair (x, y) is a solution of system (C ′ ), then we
must have x = 1 − y and x = −y, and hence, we must also have as a result
that 1 − y = −y, which yields 0y = 1. Since there exists no real number y such
that 0y = 1, we conclude that there exists no pair (x, y) of real numbers which
would be a solution to system (C ′ ). We conclude therefore that system (C ′ ) has
no solution.
We recapitulate our findings concerning systems (A′ ), (B ′ ) and (C ′ ) as follows:

• System (A′ ) has exactly one solution.

• System (B ′ ) has infinitely many solutions.

• System (C ′ ) has no solution.

Note that these are exactly the cases that we encountered for the systems
(A), (B) and (C), which consisted of only one equation in one unknown.
Some very natural questions that come to mind at this point are:

• Can a system of linear equations, with whatever number of equations and

whatever number of unknowns, have a number of solutions different from
these, i.e. a number of solutions different from 0, 1 and ∞ ?

• Can we come up with a system of linear equations that instead has exactly
2 solutions ? or exactly 17 solutions ? or 154 solutions ? ...
24 SECTION 1

• Is there anything special with 0, 1, ∞, or did we stumble on these by

accident ?

• ... and many such questions!

We shall see in the next few lectures that these are the only cases that
we can encounter with systems of linear equations; that is, a system of linear
equations, with whatever number of equations and whatever number of un-
knowns, can have either exactly one solution, infinitely many solutions, or
no solution at all; it is instructive to contrast this with the case of polyno-
mial equations of degree 2 in one real variable, i.e. equations of the form
ax2 + bx + c = 0, with a 6= 0, which we know can never have infinitely many
solutions (recall that they can have exactly 0, 1, or 2 solutions).
In order to get to a point where we can prove this non-trivial result, we
shall first develop the necessary tools. This will be done in the next few
lectures.
Later on, we will also see how to quickly and systematically solve systems
of linear equations (as opposed to the ad hoc scheme we have been using) ...

Applications of Systems of Linear Equations

Systems of linear equations have numerous applications in engineering and ap-
plied sciences. We provide here a number of application examples of systems of
linear equations. These by no means exhaust the rich diversity of contexts in
which systems of linear equations naturally arise.

• Chemical Balance Equations: The fundamental law of conservation

of mass implies that, in a chemical reaction, the number of atoms of each
element involved in the reaction must be the same before and after the
reaction. The mathematical expression of this principle is called a chemical
balance equation, and arises as a system of linear equations. To give
a simple example, hydrochloric acid (HCl) combines with sodium (N a)
to yield salt (N aCl) and hydrogen gas (H2 ); we can write the chemical
reaction as follows:

x1 HCl + x2 N a → x3 N aCl + x4 H2 ,

where x1 and x2 denote the number of molecules of hydrochloric acid and

sodium, respectively, involved in the reaction, and x3 and x4 the number of
molecules of salt and hydrogen, respectively, obtained from the reaction.
The balance equations are obtained by stipulating that the number of
atoms of each element must be conserved in the reaction, i.e., must be the
same before and after the reaction. In this particular reaction, we have
the elements H, Cl, N a. There are x1 atoms of hydrogen (H) before the
reaction, and 2x4 after the reaction; we must therefore have x1 = 2x4 , i.e.,
x1 − 2x4 = 0. There are x1 atoms of chlorine (Cl) before the reaction,
and x3 atoms after the reaction; we must therefore have x1 = x3 , i.e.
 25

x1 − x3 = 0. Finally, there are x2 atoms of sodium (N a) before the

reaction, and x3 after; we must therefore have x2 = x3 , i.e. x2 − x3 = 0.
Hence, the balance equations for the given chemical reaction are given by
the following system of linear equations:

x1 − 2x4 = 0,
x1 − x3 = 0,
x2 − x3 = 0.

• Network Flow: Many problems in engineering, economics, etc., can be

abstracted as network flow problems: The basic data is a graph, consisting
of vertices (also called nodes) and directed edges between pairs of vertices,
as well as a numeric weight attached to each edge; in the context of electric
circuits, this weight would denote the intensity of the electric current
flowing from one node to another, while in the context of economics, and
more specifically, transportation problems, it could denote the amount
of goods that are conveyed from one city to another (both represented
as vertices of the graph). Just as with chemical balance equations, we
assume here a conservation law of the form: “What goes into a vertex
must equal what comes out of a vertex” (in the context of electric circuits
this conservation law is known as Kirchoff’s current law). We illustrate
this with an example: Consider then the graph:

A
x1 x5
1 5

3
B D

x3 x4
x2 2

consisting of four vertices (or nodes), labelled A,B,C,D, and edges connect-
ing them. The numeric weight attached to each edge denotes the number
of units flowing through that edge, and we write the so-called “node bal-
ance” equations by equating at each node the total weights of incoming
and outgoing edges. If at a given node, there are no incoming (resp. out-
going) edges, then the total weight of incoming (resp. outgoing) edges
for that node is 0. For the graph depicted above, x1 , x2 , x3 , x4 , x5 are the
unknown weights that have to be determined from the node balance equa-
tions. Here, we have a total of 1+x5 units flowing into node A, and a total
of 5 + x1 units flowing out of node A; equating these yields the equation
1+x5 = 5+x1 , or, equivalently, the equation x1 −x5 = −4. At node B, we
26 SECTION 1

have a total of x1 + x3 flowing in, and a total of 1 + 3 + x2 = 4 + x2 flowing

out; equating these yields the equation x1 − x2 + x3 = 4. At node C, we
have a total of x2 + x4 units flowing in, and a total of 2 + x3 units flowing
out; equating these yields the equation x2 + x3 = 2 + x4 , or, equivalently,
x2 + x3 − x4 = 2. At node D, we have a total of 5 + 3 + 2 = 10 units
flowing in, and x4 + x5 units flowing out; equating these yields the equa-
tion x4 + x5 = 10. Hence, we have obtained the following node balance
equations:

x1 − x5 = −4,
x1 − x2 + x3 = 4,
x2 + x3 − x4 = 2,
x4 + x5 = 10,

which form a system of linear equations.

• Polynomial Interpolation: Suppose the position x(t) of a particle at

time t is given by the relation x(t) = at2 + bt + c, where the real coeffi-
cients a, b, c are unknown and have to be determined from experimental
data. Suppose we observe the particle at times t = 0, 1, 2, and suppose
we determine experimentally that x(0) = 1, x(1) = 2, x(2) = 4. Hence,
equating x(0) = a02 + b0 + c = c with x(0) = 1 yields the equation c = 1,
equating x(1) = a12 + b1 + c = a + b + c with x(1) = 2 yields the equa-
tion a + b + c = 2, and equating x(2) = a22 + b2 + c = 4a + 2b + c with
x(2) = 4 yields the equation 4a + 2b + c = 4. We have therefore obtained
the following system of linear equations

c = 1,
a+b+c = 2,
4a + 2b + c = 4,

from which we can determine the unknowns a, b, c.

• Steady-State Temperature Distribution:

Consider a rectangular thin metal plate in which the temperature is at

steady-state (i.e does not change as a function of time), and assume the
temperature on the boundary is known; we wish to determine the temper-
ature at points interior to the plate. We can consider the simpler problem
of determining the temperature at regularly placed points on the plate.
Consider then the following mesh, which represents points on the plate
and on its boundary:
 27

10 20 20 10

x1 x2
15 18

20 30 30 20
The temperature at each mesh point is indicated next to that mesh point.
There are two points on this mesh which are in the interior of the plate
(i.e. not on its boundary), with respective temperatures x1 and x2 ; we
wish to determine these two temperatures. The fundamental laws of heat
propagation imply that at steady-state, the temperature at each interior
mesh point is the average of the temperatures of its neighboring mesh
points. Applying this law to the two interior mesh points (with respective
temperatures x1 and x2 ) yields:
1
x1 = (15 + 20 + x2 + 30),
4
1
x2 = (x1 + 20 + 18 + 30),
4
which is equivalent to the system of linear equations:
4x1 − x2 = 65,
−x1 + 4x2 = 68,
from which x1 , x2 can be obtained. Note that the temperatures obtained
at the two interior points are merely approximations to the exact temper-
atures; however, the finer the mesh (i.e. the more grid points), the better
the approximation.
PROBLEMS:

For each of the following systems of linear equations, identify the number of
solutions using the same “procedure” as in the examples treated in this lecture
(this should convince you that there can be only the three cases mentioned in
this Lecture):
1.

2x = 3,
4x = 5,
where we wish to solve for the real number x.
2.

2x = 3,
4x = 6,
where we wish to solve for the real number x.
28 SECTION 1

3.

2x + y = 1,
4x + 2y = 3,

where we wish to solve for the pair (x, y) of real numbers.

4.


 2x + y = 1,

10x + 5y = 5,

 6x + 3y = 3,

20x + 10y = 10,

where we wish to solve for the pair (x, y) of real numbers.

5.

 2x + y = 1,
10x + 5y = 5,

x − y = 1,

where we wish to solve for the pair (x, y) of real numbers.

6.


 2x − y + z = 0,

x + y = 2,

 x − y = 1,

y − z = 0,

where we wish to solve for the triple (x, y, z) of real numbers.

7.

x + y − z = 0,

where we wish to solve for the triple (x, y, z) of real numbers.

8.

 x + y − z = 0,
x + y = 1,

y + z = 2,

where we wish to solve for the triple (x, y, z) of real numbers.

9.

x + y − z − w = 0,
x + y = 2,

where we wish to solve for the quadruple (x, y, z, w) of real numbers.

 29

10.

 x + y − z − w = 0,
−x − y = 2,

z + w = 3,

where we wish to solve for the quadruple (x, y, z, w) of real numbers.

11. Methane (CH4 ) combines with oxygen gas (O2 ) to yield carbon dioxide
(CO2 ) and water (H2 O); write the chemical balance equations for this
reaction as a system of linear equations.

12. Tin oxide (SnO2 ) combines with hydrogen gas (H2 ) to yield tin (Sn) and
water (H2 O); write the chemical balance equations for this reaction as a
system of linear equations.

13. Iron (F e) combines with sulfuric acid (H2 SO4 ) to yield ferric sulfate
(F e2 (SO4 )3 ) and hydrogen gas (H2 ); write the chemical balance equa-
tions for this reaction as a system of linear equations.

14. Propane (C3 H8 ) combines with oxygen gas (O2 ) to produce water (H2 O)
and carbon dioxide (CO2 ); write the chemical balance equations for this
reaction as a system of linear equations.

15. Write the node balance equations (as a system of linear equations in the
unknowns x1 , x2 , x3 ) for the following graph:

A x1

4
x2

B x3 D E

x1
x2
x3
C

16. Write the node balance equations (as a system of linear equations in the
unknowns x1 , x2 , x3 , x4 , x5 ) for the following graph:
30 SECTION 1

A x1

2 3

3 x5
B D E

x4
x2 7
x3
C

17. Write the node balance equations (as a system of linear equations in the
unknowns x1 , x2 , x3 , x4 , x5 , x6 ) for the following graph:
x6
A F

x1 x5
x4 5x4
3x2
x1 2x3
B D E
7
2x6
x5 2x1
x3
C

18. The voltage V at the output of an electric device is related to the input
current I to the device by the polynomial function V (I) = aI 3 +bI 2 +cI+d,
where the real coefficients a, b, c, d are unknown; we wish to determine
a, b, c, d from experimental data. We experimentally measure the output
voltage at the input current values I = 0, 1, 2, 3, and we determine from
measurement that V (0) = 0.2, V (1) = 2.1, V (2) = 5.7, and V (3) =
10.4. Write the system of linear equations (in the unknowns a, b, c, d)
corresponding to these measurements.
19. The elongation l of a metal bar is modelled as a function of temperature
T by the polynomial function l(T ) = aT 4 + bT 3 + cT 2 + dT + e, where the
real coefficients a, b, c, d, e are unknown; we wish to determine a, b, c, d, e
from experimental data. We experimentally measure the elongation of the
bar at temperatures T = 0, 1, 2, 3, 4, and we determine from measurement
that l(0) = 0.11, l(1) = 2.21, l(2) = 4.35, l(3) = 7.26, l(4) = 9.77. Write
the system of linear equations (in the unknowns a, b, c, d, e) corresponding
to these measurements.
20. Consider a square thin metal plate with temperature at steady state and
 31

with known boundary temperature; we represent this thin metal plate by

the following square mesh:
10 20 30 40

x1 x2
15 45

20 x3 x4 50

20 30 40 50
The steady-state temperature at each mesh point is indicated next to
that mesh point. The four mesh points interior to the plate have respec-
tive steady-state temperatures x1 , x2 , x3 , x4 ; write the system of linear
equations that governs the relation between these temperatures.
21. Consider a square thin metal plate with temperature at steady state and
with known boundary temperature; we represent this thin metal plate by
the following square mesh:
10 20 30 40 50

x1 x2 x3
15 50

x4 x5 x6
20 55

x7 x8 x9
25 60

30 35 40 50 60
The steady-state temperature at each mesh point is indicated next to that
mesh point. The nine mesh points interior to the plate have respective
steady-state temperatures x1 , x2 , x3 , x4 , x5 , x6 , x7 , x8 , x9 ; write the system
of linear equations that governs the relation between these temperatures.
32 SECTION 1
Section 2

Study Topics
• Real vector spaces
• Examples of real vector spaces

33
34 SECTION 2

The basic notion in linear algebra is that of vector space. In this lecture, we
give the basic definition and review a number of examples.
Definition 12. Let V be a set, with two operations defined on it:
(i) An operation denoted by “+” and called addition, defined formally as a
mapping + : V × V → V which maps a pair (v, w) in V × V to the element
v + w of V;
(ii) An operation denoted by “·” and called multiplication by a scalar, defined
formally as a mapping · : R × V → V which maps a pair (α, v) in R × V
(i.e. α ∈ R and v ∈ V) to the element α · v of V.
With these two operations in place, V is said to be a real vector space if the
following properties are verified:
1. The operation + is associative, i.e. for any x, y, z in V, we have:

x + (y + z) = (x + y) + z

2. There exists an element in V, called the zero vector and denoted by 0,

such that for any x in V, we have:

x+0=0+x=x

3. For any x in V, there exists an element in V denoted by −x (and usually

called the inverse or opposite of x) such that:

x + (−x) = (−x) + x = 0

4. The operation + is commutative, i.e. for any x, y in V, we have:

x+y=y+x

5. For any α, β in R and any x in V, we have:

α · (β · x) = (αβ) · x

6. For any α in R and any x, y in V, we have:

α · (x + y) = α · x + α · y

7. For any α, β in R and any x in V, we have:

(α + β) · x = α · x + β · x

8. For any x in V, we have:

1·x=x
 35

We shall usually denote the vector space by (V, +, ·) and sometimes only by V,
when there is no risk of confusion about what the addition and multiplication
by scalar operation are. Each element of V is called a vector.

IMPORTANT NOTE: If given some set V and operations “+” and “·” de-
fined in some way, one wants to verify whether or not (V, +, ·) is a real vector
space, before going through the axioms one by one it is important to first
verify that the operations “+” and “·” are both well-defined. To illus-
trate this point with an example, if we let V = N2 and we define as the addition
operation on V the operation “+̃” on V by stipulating that ∀(x1 , y1 ) ∈ N2 and
√
∀(x2 , y2 ) ∈ N2 we define (x1 , y1 )+̃(x2 , y2 ) to be the pair (x1 x2 , y1 y2 ), then
this operation will not be well-defined since for some pairs (x1 , y1 ), (x2 , y2 ) in
N2 the pair (x1 , y1 )+̃(x2 , y2 ) will not be in N2 ; for
√ example, if we take the pairs
(1, 1) and (2, 2), we will have (1, 1)+̃(2, 2) = (2, 2) which is not an element of
N2 . Hence, we can stop right here and declare that since the operation +̃ is not
well-defined, (V, +̃, ·) will not be a real vector space (no matter how we define
the operation “·”).

We now examine a number of examples in order to develop some familiarity

with this concept.
(a) Consider the usual set R of real numbers, with the addition operation “+”
being the usual addition of real numbers, and the multiplication by scalar
operation “·” the usual multiplication operation of real numbers. It is
easy to verify that, with these two operations, R satisfies all the axioms of
a real vector space. We can write therefore that (R, +, ·) is a real vector
space.
(b) Let R2 denote the set of all pairs (x, y) with x and y real numbers. We
define an addition operation “+” on R2 as follows: If (x1 , y1 ) and (x2 , y2 )
are two elements of R2 , we define:
(x1 , y1 ) + (x2 , y2 ) = (x1 + x2 , y1 + y2 ).
We also define a multiplication by scalar operation “·” on R2 as follows:
For any α in R and any (x, y) in R2 , we define:
α · (x, y) = (αx, αy).
It is easy to verify that endowed with these two operations, R2 satisfies all
the axioms of a real vector space. We can write therefore that (R2 , +, ·)
is a real vector space.
(c) Let F (R; R) denote the set of all functions f : R → R, i.e. the set of all
real-valued functions of a real variable. We define the addition operation
“+” on F (R; R) as follows: If f, g are two elements of F (R; R), we define
(f + g) : R → R by:
(f + g)(t) = f (t) + g(t), for all t in R.
36 SECTION 2

We also define a multiplication by scalar operation “·” on F (R; R) as

follows: For any α in R and any f in F (R; R), we define the function
(α · f ) : R → R by:

(α · f )(t) = αf (t), for all t in R.

It is easy to verify that endowed with these two operations, F (R; R) sat-
isfies all the axioms of a real vector space. We can write therefore that
(F (R; R), +, ·) is a real vector space.

(d) Let now F0 (R; R) denote the set of all functions f : R → R which satisfy
f (0) = 0, i.e. the set of all real valued functions of a real variable which
vanish at 0. Formally, we write:

F0 (R; R) = {f ∈ F (R; R)|f (0) = 0}.

Note that F0 (R; R) is a subset of F (R; R). Note also that we can define
the addition operation “+” on F0 (R; R) the same way we defined it on
F (R; R); indeed, if f, g are two elements of F0 (R; R), then the function
f + g satisfies (f + g)(0) = f (0) + g(0) = 0 and hence f + g is an element
of F0 (R; R) as well. Similarly, if we define the multiplication by scalar
operation “·” on F0 (R; R) the same way we defined it on F (R; R), we
have that if α is in R and f is in F0 (R; R), then the function α · f satisfies
(α · f )(0) = αf (0) = 0, and as a result, the function α · f is an element of
F0 (R; R) as well. Just as easily as with F (R; R), it is immediate to verify
that endowed with these two operations, F0 (R; R) satisfies all the axioms
of a real vector space. We can write therefore that (F0 (R; R), +, ·) is a
real vector space.

(e) Let Rn denote the set of all n−tuples (x1 , x2 , · · · , xn ) of real numbers
(where n is any integer ≥ 1); Rn is defined as the nth Cartesian product
of R with itself, i.e. Rn = R × R × · · · × R (n times). Note that we saw
the special case of this construction for n = 1 in Example (a) above, and
for n = 2 in Example (b) above. We define on Rn the addition operation
“+” as follows: For any n−tuples (x1 , x2 , · · · , xn ) and (y1 , y2 , · · · , yn ) of
real numbers, we define:

(x1 , x2 , · · · , xn ) + (y1 , y2 , · · · , yn ) = (x1 + y1 , x2 + y2 , · · · , xn + yn ).

We also define a multiplication by scalar operation “·” on Rn as follows:

For any α in R and any element (x1 , x2 , · · · , xn ) of Rn , we define:

α · (x1 , x2 , · · · , xn ) = (αx1 , αx2 , · · · , αxn ).

It is easy to verify that endowed with these two operations, Rn satisfies all
the axioms of a real vector space. We can write therefore that (Rn , +, ·)
is a real vector space.
 37

(f) Let (Rn )0 denote the set of all n−tuples (x1 , x2 , · · · , xn ) of real numbers
which satisfy x1 + x2 + · · · + xn = 0. Note that (Rn )0 is a subset of Rn .
Furthermore, we can define on (Rn )0 the addition operation “+” and the
multiplication by scalar operation “·” in exactly the same way that we
defined them on Rn . Indeed, if (x1 , x2 , · · · , xn ) and (y1 , y2 , · · · , yn ) are
two elements of (Rn )0 , then the n−tuple (x1 + y1 , x2 + y2 , · · · , xn + yn ) is
also in (Rn )0 since

(x1 + y1 ) + · · · + (xn + yn ) = (x1 + · · · + xn ) + (y1 + · · · + yn )

= 0+0
= 0.

Similarly, if α is in R and (x1 , x2 , · · · , xn ) is an element of (Rn )0 , then the

n−tuple α · (x1 , x2, · · · , xn ) is also in (Rn )0 , since

αx1 + αx2 + · · · αxn = α(x1 + x2 + · · · + xn )

= 0.

It is easy to verify that endowed with these two operations, (Rn )0 sat-
isfies all the axioms of a real vector space. We can write therefore that
((Rn )0 , +, ·) is a real vector space.

Let us now consider examples of sets with operations on them which do not
make them real vector spaces:
(g) On the set Rn , which we have defined in Example (e) above, define an
addition operation, which we denote by +̃ to distinguish it from the one
defined in Example (e), as follows: For any n−tuples (x1 , x2 , · · · , xn ) and
(y1 , y2 , · · · , yn ) of real numbers, define:

(x1 , x2 , · · · , xn )+̃(y1 , y2 , · · · , yn ) = (x1 + 2y1 , x2 + 2y2 , · · · , xn + 2yn ).

Define the multiplication by scalar operation “·” as in Example (e). It

is easy to verify that endowed with these two operations, Rn does not
satisfy all the axioms of a real vector space. We can therefore write that
(Rn , +̃, ·) is not a real vector space.
(h) On the set F (R; R), defined in Example (c) above, define the addition
operation + exactly as in Example (c), but define the multiplication by
scalar operation by ˜· (to distinguish it from the one defined in Example
(c)) as follows: For any α in R and f in F (R; R), define the element α˜·f
of F (R; R) as follows:

(α˜·f )(t) = α(f (t))2 , for all t in R.

It is easy to verify that endowed with these two operations F (R; R) does
not satisfy all the axioms of a real vector space. We can write therefore
that (F (R; R), +,˜·) is not a real vector space.
38 SECTION 2

Now that the concept of real vector space is hopefully getting more concrete,
let us prove the following simple (and very intuitive) results for general vector
spaces:

Theorem 1. Let (V, +, ·) be a real vector space. Then:

• For any v in V, we have 0 · v = 0;

• For any α in R, we have α · 0 = 0.

Proof. Let us first prove the first statement. Since 0 = 0 + 0, we have, of course:

0 · v = (0 + 0) · v = 0 · v + 0 · v,

where we have used Property (7) of a vector space to get this last equality; now
by property (3) of a vector space, there exists an element in V, which we denote
by −0 · v, such that −0 · v + 0 · v = 0; adding −0 · v to both sides of the above
equality yields:

−0 · v + 0 · v = −0 · v + (0 · v + 0 · v),

which, by Property (1) of a vector space is equivalent to the equality:

−0 · v + 0 · v = (−0 · v + 0 · v) + 0 · v,

which, by the fact that −0 · v + 0 · v = 0 (Property (3)) is equivalent to the

equality

0 = 0 + 0 · v,

which, by Property (2) of a vector space, is equivalent to

0 = 0 · v,

i.e. 0 · v = 0.
Let us now prove the second statement. Since 0 = 0 + 0 (Property (2)), we
have:

α · 0 = α · (0 + 0) = α · 0 + α · 0,

where we have used Property (6) to get this last equality; By property (3), there
exists an element in V, which we denote by −α · 0, such that −α · 0 + α · 0 = 0;
adding −α · 0 to both sides of the above equality yields:

−α · 0 + α · 0 = −α · 0 + (α · 0 + α · 0),

which, by Property (1) is equivalent to

−α · 0 + α · 0 = (−α · 0 + α · 0) + α · 0,
 39

which by the fact that −α · 0 + α · 0 = 0 (Property (3)) is equivalent to the

equality

0 = 0 + α · 0,

which, by Property (2), is equivalent to

0 = α · 0,

i.e., α · 0 = 0.

Recall that, by definition of a real vector space, if (V, +, ·) happens to be a real

vector space, and if v is any element of V, then there is an element in V which
we denote by −v (and which we call the “inverse” or “opposite” of v) and which
satisfies:

v + (−v) = −v + v = 0

(where 0 denotes the zero vector of V). All we know is that such an element
does exist in V; we don’t know yet whether for each choice of v in V such an
element −v is unique or not, nor do we know how to compute it. The following
theorem answers these two questions:

Theorem 2. Let (V, +, ·) be a real vector space. Then:

(a) For any v in V, the element −v is uniquely defined.

(b) For any v in V, we have: −v = (−1) · v.

Proof. Let us prove (a) first. Let then v ∈ V be any element of V, and assume
the element w ∈ V satisfies the defining conditions for −v, i.e. it satisfies:

w + v = v + w = 0.

We shall show that we necessarily have w = −v; this will prove that for any
v ∈ V, the element −v is uniquely defined. Since

w + v = 0,

we obtain (by adding −v to the right on both sides of the equality):

(w + v) + (−v) = 0 + (−v),

and using axioms (1) and (2) of a real vector space, we obtain:

w + (v + (−v)) = −v,

and using axiom (3), we obtain:

w + 0 = −v,
40 SECTION 2

and, finally, using axiom (2), we obtain:

w = −v,
which is exactly what we wanted to prove. Hence this proves (a). We now prove
(b). Let then v ∈ V be any element of V. Note that by axiom (8) of a real
vector space we have v = 1 · v, hence (using axiom (7) of a real vector space as
well as the previous theorem):
(−1) · v + v = (−1) · v + 1 · v = (−1 + 1) · v = 0 · v = 0,
and, similarly,
v + (−1) · v = 1 · v + (−1) · v = (1 + (−1)) · v = 0 · v = 0,
Hence, we have shown that the element (−1) · v of V satisfies:
v + (−1) · v = (−1) · v + v = 0,
and by the result proved in part (a), this shows that
(−1) · v = −v,
which is exactly what we wanted to prove.

To recap the previous theorem: If you are given an element v in a real vector
space (V, +, ·), you can compute its inverse −v simply by computing (−1) · v
(i.e. by “multiplying the vector v by the real number −1”); as simple as that!

Before closing this section, we consider an important example of a real vector

cn (to distinguish
space which we will deal with very frequently. We denote it by R
n
it from R ) and call it the vector space of all column n−vectors with real
entries. It is the set of all elements of the form
 
x1
 x2 
 
 .. 
 . 
xn
where x1 , x2 , · · · , xn are real numbers. The addition operation “+” and multi-
plication 
by scalar cn as follows: For any α in R
 operation
 “·” are defined on R

x1 y1
 x2   y2 
    cn , we define:
and any  .  and  .  in R
 ..   .. 
xn yn
     
x1 y1 x1 + y1
 x2   y2   x2 + y2 
     
 ..  +  ..  =  .. 
 .   .   . 
xn yn xn + yn
 41

and
   
x1 αx1
 x2   αx2 
   
α· .. = .. 
 .   . 
xn αxn

It is easily verified (just as for Rn in Example (e) above) that endowed with
these two operations, R cn is a real vector space.

A WORD ON NOTATION:

In the remainder of these notes, we will omit the multiplication by a scalar

symbol “·” when multiplying a real number and a vector together, as this should
cause no confusion. Hence, if α is a real number and v is an element of some
vector space (i.e. a vector), we will write simply αv instead of α·v. For example,
if (a, b) is an element of the vector space R2 (the set ofall pairs
 of real numbers),
a
c3
we will write α(a, b) instead of α · (a, b). Similarly, if  b  is an element of R
c 
a
(the set of all real column vectors with 3 entries), we will write simply α  b 
  c
a
instead of α ·  b .
c
We will however continue to denote a vector space by a triple of the form (V, +, ·)
(even though we will drop the “·” when actually writing the multiplication by a
real number); occasionally, we will also denote a vector space only by V instead
of (V, +, ·).

PROBLEMS:

1. Show that the set of all nonnegative integers N = {0, 1, 2, 3, · · · }, with

addition and multiplication defined as usual is not a real vector space,
and explain precisely why.

2. Show that the set of all integers Z = {· · · , −3, −2, −1, 0, 1, 2, 3, · · ·}, with
addition and multiplication defined as usual is not a real vector space,
and explain precisely why.

3. Show that the set of all non-negative real numbers R+ = {x ∈ R|x ≥ 0}

(with addition and multiplication defined as for R) is not a real vector
space, and explain precisely why.
42 SECTION 2

4. Show that the set of all non-positive real numbers R− = {x ∈ R|x ≤ 0}

(with addition and multiplication defined as for R) is not a real vector
space, and explain precisely why.
5. Show that the subset of R defined by S = {x ∈ R| − 1 ≤ x ≤ 1} (with
addition and multiplication defined as for R) is not a real vector space,
and explain precisely why.
6. Let F1 (R; R) be the set of all functions f : R → R which satisfy f (0) = 1.
Show that with addition and multiplication defined as for F (R; R) (see
Example (c) of this Lecture), F1 (R; R) is not a real vector space, and
explain precisely why.
7. Let F[−1,1] (R; R) be the set of all functions f : R → R which satisfy
−1 ≤ f (x) ≤ 1, for all x ∈ R. Show that with addition and multiplication
defined as for F (R; R) (see Example (c) of this Lecture), F[−1,1] (R; R) is
not a real vector space, and explain precisely why.
8. Consider the set R2 of all pairs (x, y) of real numbers. Define an “addition”
operation, which we denote by “+̃”, as follows: For any (x, y) and (u, v)
in R2 , we define (x, y)+̃(u, v) to be the pair (y + v, x + u). Define the
multiplication operation · as in Example (b) (i.e. α · (x, y) = (αx, αy)). Is
(R2 , +̃, ·) a real vector space ? Explain precisely why or why not.
9. Consider the set R2 of all pairs (x, y) of real numbers. Define an “addition”
operation, which we again denote by “+̃”, as follows: For any (x, y) and
(u, v) in R2 , we define (x, y)+̃(u, v) to be the pair (x + v, y + u). Define the
multiplication operation · as in Example (b) (i.e. α · (x, y) = (αx, αy)). Is
(R2 , +̃, ·) a real vector space ? Explain precisely why or why not.
10. Consider the subset V1 of R2 consisting of all pairs (x, y) of real numbers
such that x+y = 1. If we define the addition and multiplication operations
on V1 as we did for R2 in Example (b) of this Lecture, do we obtain a real
vector space ? Explain precisely why or why not.
11. Consider the subset V2 of R2 consisting of all pairs (x, y) of real numbers
such that x+y = 0. If we define the addition and multiplication operations
on V2 as we did for R2 in Example (b) of this Lecture, do we obtain a real
vector space ? Explain precisely why or why not.
12. Consider now the subset V3 of R2 consisting of all pairs (x, y) of real
numbers such that 2x−3y = 1. If we define the addition and multiplication
operations on V3 as we did for R2 in Example (b) of this Lecture, do we
obtain a real vector space ? Explain precisely why or why not.
13. Consider now the subset V3 of R2 consisting of all pairs (x, y) of real
numbers such that 2x−3y = 0. If we define the addition and multiplication
operations on V4 as we did for R2 in Example (b) of this Lecture, do we
obtain a real vector space ? Explain precisely why or why not.
 43

14. Let a1 , a2 , · · · , an be given real numbers, and consider the subset V of

Rn (defined in Example (e) of this Lecture) consisting of all n−tuples
(x1 , x2 , · · · , xn ) of real numbers which satisfy

a1 x1 + a2 x2 + · · · + an xn = 0.

With addition and multiplication operations defined as in Example (e) of

this Lecture, is V a real vector space ? Explain precisely why or why not.

15. Let a1 , a2 , · · · , an be given real numbers, and consider the subset V of

Rn (defined in Example (e) of this Lecture) consisting of all n−tuples
(x1 , x2 , · · · , xn ) of real numbers which satisfy

a1 x1 + a2 x2 + · · · + an xn = 1.

With addition and multiplication operations defined as in Example (e) of

this Lecture, is V a real vector space ? Explain precisely why or why not.
16. Let a1 , a2 , · · · , an be given real numbers, and consider the subset W of
Rn (defined in Example (e) of this Lecture) consisting of all n−tuples
(x1 , x2 , · · · , xn ) of real numbers which satisfy

a1 x1 + a2 x2 + · · · + an xn ≥ 0.

With addition and multiplication operations defined as in Example (e) of

this Lecture, is W a real vector space ? Explain precisely why or why not.
17. Let S = {ξ} be a set consisting of a single element, denoted ξ. Define
the addition operation “+” on S as follows: ξ + ξ = ξ, and define the
multiplication (by a scalar) operation “·” on S as follows: For any α ∈ R,
α · ξ = ξ. Show that (S, +, ·) is a real vector space.
18. Let (V, +, ·) be a real vector space. By the definition of a real vector
space (see Property (2)), we know that there exists a “special” element
in V, which we denote by 0 and call the zero vector of V, which has the
property that for any v in V, v + 0 = 0 + v = v. We wish to show that
this “special” element is actually unique. To do so, prove that if some
b satisfies Property (2) of a real
(possibly other) element of V, call it 0,
b
vector space, then we must have 0 = 0, i.e. there is no element in V
other than the zero vector 0 itself, which satisfies Property (2).
19. Consider the set R2 of all pairs (x, y) of real numbers. Define an “addition”
operation, which we denote by “+̃”, as follows: For any (x, y) and (u, v)
in R2 , we define (x, y)+̃(u, v) to be the pair (x + u − 1, y + v − 2). Define
the multiplication operation “ ˜· ” by α˜·(x, y) = (αx − α + 1, αy − 2α + 2),
for any α ∈ R and (x, y) ∈ R2 . Show that (R2 , +̃,˜·) is a real vector space.
What is the zero vector of (R2 , +̃,˜·) ?
44 SECTION 2
Section 3

Study Topics
• Vector subspaces
• Examples and properties of vector subspaces

45
46 SECTION 3

We now define another important notion, that of a subspace of a vector space.

Definition 13. Let (V, +, ·) be a real vector space, and let W be a subset of V
(i.e. W ⊂ V). W is said to be a vector subspace (or, for short, subspace) of
(V, +, ·) (or, for short, of V) if the following properties hold:
(i) The zero element 0 of V is in W, that is, 0 ∈ W.
(ii) For any x and y in W, we have x + y ∈ W.
(iii) For any α in R and any x in W, we have αx ∈ W.
Before going further, we examine a number of examples:
(a) Recall the real vector space (Rn , +, ·) of all real n−tuples (x1 , x2 , · · · , xn )
which was defined in Lecture 2; recall also the subset (Rn )0 of Rn (defined
in the same lecture), consisting of all n−tuples (x1 , x2 , · · · , xn ) in Rn for
which x1 + x2 + · · ·+ xn = 0 (i.e. all entries add up to 0). Let us show that
(Rn )0 is a vector subspace of the real vector space (Rn , +, ·): To do this,
we have to show that properties (i),(ii), and (iii) of a vector subspace are
verified. Let us begin with property (i); the zero element 0 of Rn is the
n−tuple (0, 0, · · · , 0) (all zero entries); since 0 + 0 + · · · + 0 = 0, it is clear
that the n-tuple (0, 0, · · · , 0) is in (Rn )0 , i.e. 0 ∈ (Rn )0 ; hence, property
(i) of a vector subspace is verified. Let us now check Property (ii): Let
(x1 , x2 , · · · , xn ) and (y1 , y2 , · · · , yn ) be two elements of (Rn )0 ; we have to
show that the n−tuple (x1 , x2 , · · · , xn ) + (y1 , y2 , · · · , yn ), which is defined
as the n−tuple (x1 + y1 , x2 + y2 , · · · , xn + yn ), is in (Rn )0 . Now, since

(x1 + y1 ) + · · · + (xn + yn ) = (x1 + · · · + xn ) + (y1 + · · · + yn )

= 0+0
= 0,

it follows that the n−tuple (x1 + y1 , x2 + y2 , · · · , xn + yn ) is in (Rn )0 , i.e.

(x1 , x2 , · · · , xn ) + (y1 , y2 , · · · , yn ) is in (Rn )0 . Hence Property (ii) is veri-
fied. Finally, let us check Property (iii); let then α ∈ R be any real number
and let (x1 , x2 , · · · , xn ) be any element of (Rn )0 ; we have to show that
the n−tuple α(x1 , x2 , · · · , xn ), which is defined as (αx1 , αx2 , · · · , αxn ), is
in (Rn )0 . Now, since

αx1 + αx2 + · · · αxn = α(x1 + x2 + · · · + xn )

= α(0)
= 0,

it follows that the n−tuple (αx1 , αx2 , · · · , αxn ) is in (Rn )0 , i.e. α(x1 , x2 , · · · , xn )
is in (Rn )0 . Hence, Property (iii) is verified as well. We conclude that
(Rn )0 is a vector subspace of the real vector space (Rn , +, ·).
(b) Let (V, +, ·) be a real vector space. Then V is itself a vector subspace of
V.
 47

(c) Let (V, +, ·) be a real vector space. Then the set {0} consisting of the
zero element 0 of V alone is a vector subspace of V.
(d) Recall the real vector space (F (R; R), +, ·) of all real-valued functions of
a real variable defined in Lecture 2, as well as the subset F0 (R; R) of
F (R; R) defined in the same lecture (recall F0 (R; R) consists of all real-
valued functions f of a real variable for which f (0) = 0). It is easy to
verify that F0 (R; R) is a vector subspace of F (R; R).

For completeness, let us also consider examples of subsets which are not vector
subspaces:
(e) Recall again the real vector space (Rn , +, ·) of all real n−tuples (x1 , x2 , · · · , xn )
which was defined in Lecture 2; let (Rn )1 denote the subset of Rn consist-
ing of all n−tuples (x1 , x2 , · · · , xn ) of Rn for which x1 + x2 + · · · + xn = 1
(i.e. the entries add up to 1). It is easy to verify that (Rn )1 is not a
vector subspace of (Rn , +, ·).
(f) Recall now again the real vector space (F (R; R), +, ·) defined in Lecture
2, and let F1 (R; R) denote the subset of (F (R; R), +, ·) consisting of all
functions f ∈ (F (R; R), +, ·) for which f (0) = 1. It is easy to verify that
F1 (R; R) is not a vector subspace of (F (R; R), +, ·).
Vector subspaces of a given vector space have the following important property:
Theorem 3. Let (V, +, ·) be a real vector space, and let W1 ⊂ V and W2 ⊂ V
be two vector subspaces of V; then their intersection W1 ∩ W2 is also a vector
subspace of V.
Proof: To prove that W1 ∩ W2 is a vector subspace of V , we have to verify
that W1 ∩ W2 satisfies the three properties of a vector subspace.
(i) We begin by showing the first property, namely that the zero element 0
of V is in W1 ∩ W2 . Since W1 is by assumption a vector subspace of V,
the zero element 0 is in W1 ; similarly, since W2 is by assumption a vector
subspace of V, the zero element 0 is in W2 . Hence, the zero element 0
is in W1 and in W2 , that is, 0 is in the intersection W1 ∩ W2 of W1
and W2 .
(ii) Let us now prove the second property of a vector subspace, namely that
for any x and y in W1 ∩ W2 , x + y is also in W1 ∩ W2 . For this, let us
take x and y in W1 ∩ W2 ; we have to show that x + y is also in W1 ∩ W2 .
Since x is in W1 ∩ W2 , it is in W1 ; similarly, since y is in W1 ∩ W2 , it is
also in W1 . Since by assumption W1 is a vector subspace of V, x + y is
also in W1 . We now repeat this procedure for W2 instead of W1 ; since
x is in W1 ∩ W2 , it is in W2 ; similarly, since y is in W1 ∩ W2 , it is also
in W2 . Since by assumption W2 is a vector subspace of V, x + y is also
in W2 . Hence, we have obtained that x + y is in W1 and in W2 ; hence,
x + y is in W1 ∩ W2 .
48 SECTION 3

(ii) Let us now prove the third and last property of a vector subspace, namely
that for any α in R and any x in W1 ∩ W2 , then αx is in W1 ∩ W2 .
Since x is in W1 ∩ W2 , it is in W1 ; since by assumption W1 is a vector
subspace of V, αx is also in W1 . Similarly, since x is in W1 ∩ W2 , it is
also in W2 ; since by assumption W2 is a vector subspace of V, αx is also
in W2 . We have therefore obtained that αx is in W1 and in W2 ; hence,
αx is in W1 ∩ W2 .

We conclude that W1 ∩ W2 satisfies the three properties of a vector subspace.

This proves the theorem. 

Remark 1. It is important to note that if (V, +, ·) is a real vector space and

W1 , W2 two vector subspaces of V, then their union W1 ∪ W2 is in general not
a vector subspace of V.

PROBLEMS:

1. Consider the real vector space (R2 , +, ·) defined in Example (b) of Lecture
2. For each of the following subsets of R2 , determine whether or not they
are a vector subspace of R2 :

(a) S = set of all (x, y) in R2 such that 6x + 8y = 0.

(b) S = set of all (x, y) in R2 such that 6x + 8y = 1.
(c) S = set of all (x, y) in R2 such that x = 0.
(d) S = set of all (x, y) in R2 such that y = 0.
(e) S = set of all (x, y) in R2 such that x = 3.
(f) S = set of all (x, y) in R2 such that y = 5.
(g) S = set of all (x, y) in R2 such that x2 + y 2 = 1.
(h) S = set of all (x, y) in R2 such that xy = 0.
(i) S = set of all (x, y) in R2 such that xy = 1.

2. Consider the real vector space (F (R; R), +, ·) defined in Example (c) of
Lecture 2. For each of the following subsets of F (R; R), determine whether
or not they are a vector subspace of F (R; R):

(a) S = set of all f in F (R; R) such that f (1) = 0.

(b) S = set of all f in F (R; R) such that f (1) = 0 and f (2) = 0.
(c) S = set of all f in F (R; R) such that f (1) = f (2).
(d) S = set of all f in F (R; R) such that f (1) = 1 + f (2).
(e) S = set of all f in F (R; R) such that f (1) = f (2) and f (2) = f (3).
(f) S = set of all f in F (R; R) such that f (1) = f (2) and f (4) = 0.
 49

(g) S = set of all f in F (R; R) such that f (x) = 0 for all −1 ≤ x ≤ 1.

(h) S = set of all f in F (R; R) such that f (x) = 2 for all −1 ≤ x ≤ 1.
(i) S = set of all f in F (R; R) such that f (x) ≥ 0 for all x ∈ R.
(j) S = set of all f in F (R; R) such that f (x) > 0 for all x ∈ R.
(k) S = set of all f in F (R; R) such that f (x) = f (−x) for all x ∈ R.
(l) S = set of all f in F (R; R) such that f (x) = −f (−x) for all x ∈ R.
(m) S = set of all f in F (R; R) such that f (x) + f (2x) = 0 for all x ∈ R.
(n) S = set of all f in F (R; R) such that f (x) + (f (2x))2 = 0 for all
x ∈ R.
(o) S = set of all f in F (R; R) such that f (x) + f (x2 + 1) = 0 for all
x ∈ R.
(p) S = set of all f in F (R; R) such that f (x) + f (x2 + 1) = 1 for all
x ∈ R.
(q) S = set of all f in F (R; R) such that f (x) − f (x + 1) = 0 for all
x ∈ R.
(r) S = set of all f in F (R; R) such that f (x) + f (x + 1) + f (x + 2) = 0
for all x ∈ R.
(s) S = set of all f in F (R; R) such that f (x) + f (x + 1) + f (x + 2) = 1
for all x ∈ R.
3. For the real vector space (R2 , +, ·), defined in Example (b) of Lecture 2,
give an example of two vector subspaces V1 and V2 such that their
union V1 ∪ V2 is not a vector subspace of R2 (See Remark 1 in this
Lecture).
4. Let (V, +, ·) be a real vector space, and let the subset W of V be a
vector subspace of V. Show that with the same addition operation “+”
and multiplication (by a scalar) operation “·” as in V, W satisfies all
the properties of a real vector space, and hence, is itself (with those two
operations) a real vector space.
50 SECTION 3
Section 4

Study Topics
• Linear combinations of vectors
• Linear span of a finite set of vectors

51
52 SECTION 4

We know that in a real vector space, we can add two vectors and multiply
a vector by a real number (aka scalar). A natural question that comes to
mind is: if we have a real vector space V and we take, say, two vectors v1 and
v2 of V, what can we get by doing all these possible operations on these
two vectors, i.e. by taking the set of all vectors of the form αv1 + βv2 (with
α and β being real numbers) ?
We shall examine this question in this lecture, and we shall see at the end of this
lecture how it relates to our original problem, that of understanding systems
of linear equations.
Definition 14. Let (V, +, ·) be a real vector space, and let v1 , · · · , vp be a finite
number of elements of V (with p ≥ 1). The expression

α1 v1 + α2 v2 + · · · + αp vp ,

with α1 , α2 , · · · , αp real numbers, is called a linear combination of the vectors

v1 , v2 , · · · , vp ; we also sometimes call it “the linear combination of the vectors
v1 , v2 , · · · , vp with respective coefficients α1 , α2 , · · · , αp ”.
If an element v of V can be written as

v = α1 v1 + α2 v2 + · · · + αp vp ,

with α1 , α2 , · · · , αp real numbers, then we say that v is a linear combination of

the vectors v1 , v2 , · · · , vp (with respective coefficients α1 , · · · , αp ). We denote by
S(v1 ,v2 ,··· ,vp ) the set of all linear combinations of the vectors v1 , v2 , · · · , vp ,
that is:

S(v1 ,v2 ,··· ,vp ) = {α1 v1 + α2 v2 + · · · + αp vp |α1 , α2 , · · · , αp ∈ R}.

It is clear that S(v1 ,v2 ,··· ,vp ) is a subset of V; indeed, since v1 , v2 , · · · , vp are in
V , multiplying them by real numbers and adding them up gives us something
still in V, since V is a vector space. So we can write:

S(v1 ,v2 ,··· ,vp ) ⊂ V.

But there is more! Indeed, the following result shows that S(v1 ,v2 ,··· ,vp ) is not
just any old subset of V; rather, it is a vector subspace of V:
Proposition 1. Let (V, +, ·) be a real vector space, and let v1 , · · · , vp be a finite
number of elements of V (where p ≥ 1). The subset S(v1 ,v2 ,··· ,vp ) of V consisting
of all linear combinations of the vectors v1 , · · · , vp is a vector subspace of
V.
Proof. To prove this result, we have to show that S(v1 ,v2 ,··· ,vp ) satisfies the
three properties that a vector subspace of V should satisfy. We verify these
properties one by one:
(i) We have to show that the zero vector 0 of V is also an element of
S(v1 ,v2 ,··· ,vp ) ; to do this, we have to show that the zero vector 0 can be
 53

written as a linear combination α1 v1 +α2 v2 +· · ·+αp vp of v1 , v2 , · · · , vp

for some choice of the scalars α1 , α2 , · · · , αp . It is clear that choosing
α1 , α2 , · · · , αp to be all zero yields the desired linear combination; indeed,
we can write:

0 = 0v1 + 0v2 + · · · + 0vp ,

and this shows that the zero vector 0 can indeed be expressed as a linear
combination of the vectors v1 , v2 , · · · , vp ; hence the zero vector 0 is an
element of S(v1 ,v2 ,··· ,vp ) , i.e. 0 ∈ S(v1 ,v2 ,··· ,vp ) .
(ii) Let x and y be two elements of S(v1 ,v2 ,··· ,vp ) ; we have to show that the
vector x + y is also in S(v1 ,v2 ,··· ,vp ) . But since by assumption x is in
S(v1 ,v2 ,··· ,vp ) , it must be that

x = α1 v1 + α2 v2 + · · · αp vp

for some real numbers α1 , α2 , · · · , αp , by definition of the set S(v1 ,v2 ,··· ,vp )
itself. Similarly, since by assumption y is in S(v1 ,v2 ,··· ,vp ) , it must be that

y = β1 v1 + β2 v2 + · · · βp vp

for some real numbers β1 , β2 , · · · , βp , by definition of S(v1 ,v2 ,··· ,vp ) . Hence,
the sum x + y of x and y can be written as:

x+y = (α1 v1 + · · · αp vp ) + (β1 v1 + · · · βp vp )

= (α1 + β1 )v1 + · · · (αp + βp )vp ,

which shows that x + y itself is a linear combination of the vectors

v1 , v2 , · · · , vp ; hence x + y is an element of S(v1 ,v2 ,··· ,vp ) , i.e. x + y ∈
S(v1 ,v2 ,··· ,vp ) .
(iii) Let now x be an element of S(v1 ,v2 ,··· ,vp ) and γ be a real number; we have
to show that the vector γx is also in S(v1 ,v2 ,··· ,vp ) . Since, by assumption,
x is in S(v1 ,v2 ,··· ,vp ) , it must be that

x = α1 v1 + α2 v2 + · · · αp vp

for some real numbers α1 , α2 , · · · , αp , by definition of the set S(v1 ,v2 ,··· ,vp )
itself. Hence, we can write:

γx = γ(α1 v1 + α2 v2 + · · · αp vp )
= (γα1 )v1 + · · · (γαp )vp ,

which shows that γx itself is a linear combination of the vectors

v1 , v2 , · · · , vp ; hence γx is in S(v1 ,v2 ,··· ,vp ) , i.e. γx ∈ S(v1 ,v2 ,··· ,vp ) .
We conclude that S(v1 ,v2 ,··· ,vp ) satisfies all three properties of a vector subspace
of V; hence S(v1 ,v2 ,··· ,vp ) is a vector subspace of V.
54 SECTION 4

Before going any further, we give a name to S(v1 ,v2 ,··· ,vp ) :

Definition 15. The vector subspace S(v1 ,v2 ,··· ,vp ) of V is called the subspace of
V generated by the vectors v1 , v2 , · · · , vp ; it is also called the linear span
of the vectors v1 , v2 , · · · , vp .

Now that S(v1 ,v2 ,··· ,vp ) has an honest name (actually two!), we examine a num-
ber of examples:

(a) Let (V, +, ·) be a real vector space, and let v1 = 0 be the zero vector
of V. The vector subspace of V generated by v1 is clearly seen to be the
zero subspace {0} of V.

(b) Consider the real vector space (R2 , +, ·) defined previously, and let v1 =
(1, 0) and v2 = (0, 1); it is easy to verify that the subspace of (R2 , +, ·)
generated by v1 , v2 is V itself.

(c) Consider the real vector space (R3 , +, ·) defined previously, and consider
the two elements v1 = (1, −1, 0) and v2 = (0, 1, −1) of R3 . Let us show
that the vector subspace S(v1 ,v2 ) of R3 spanned by v1 and v2 is the set of
all (a, b, c) in R3 such that a + b + c = 0, that is, we wish to show that

S(v1 ,v2 ) = {(a, b, c) ∈ R3 |a + b + c = 0}.

To show this equality, we will show first that we have the inclusion

S(v1 ,v2 ) ⊂ {(a, b, c) ∈ R3 |a + b + c = 0},

following which we will show that we have the inclusion

{(a, b, c) ∈ R3 |a + b + c = 0} ⊂ S(v1 ,v2 ) ;

these two inclusions will, together, show the desired equality. Let us then
begin by showing the first inclusion. For this, let us take an arbitrary
element x in S(v1 ,v2 ) , and show that it is also in the set {(a, b, c) ∈ R3 |a +
b + c = 0}. By definition of S(v1 ,v2 ) , x can be written as:

x = αv1 + βv2 ,

for some real numbers α and β. Hence, we have:

x = αv1 + βv2
= α(1, −1, 0) + β(0, 1, −1)
= (α, −α, 0) + (0, β, −β)
= (α, −α + β, −β);

clearly, since α + (−α + β) + (−β) = 0, we have that x is in the set

{(a, b, c) ∈ R3 |a + b + c = 0}. This proves the first inclusion.
 55

To prove the second inclusion, let us take an arbitrary element y in the

set {(a, b, c) ∈ R3 |a + b + c = 0}, and show that it is also in the linear span
S(v1 ,v2 ) of v1 and v2 . Let then y = (α, β, γ) ∈ {(a, b, c) ∈ R3 |a+b+c = 0};
since α + β + γ = 0, we deduce that β = −(α + γ), and hence, we can
write y as:

y = (α, −(α + γ), γ)

= (α, −α − γ, γ)
= (α, −α, 0) + (0, −γ, γ)
= α(1, −1, 0) + (−γ)(0, 1, −1)
= αv1 + (−γ)v2 ,

that is, y is a linear combination of v1 and v2 (with coefficients α and −γ,

respectively) and hence is in S(v1 ,v2 ) . This proves the second inclusion,
and hence, together with the first inclusion, we obtain the desired result.
We now have enough tools at our disposal to revisit our original “experiments”
with systems of linear equations and to have some basic understanding of what
it is that happens when a system of linear equations has a solution or has no
solution. In order to make things concrete, let us consider two examples:
1. Let us begin with the following system of 3 linear equation in 2 unknowns:

 x1 − x2 = 1,
(A) 2x1 + x2 = 0,

x1 − 2x2 = 2,

where we wish to solve for the pair (x1 , x2 ) of real numbers. Let us now
recall  c3 , +, ·) of all column vectors of the form
the real vector space (R

a
 b  where a, b, c are real numbers. Consider the elements v1 , v2 , w of
c
c 3
R defined as follows:
     
1 −1 1
v1 =  2  , v2 =  1  , w =  0  ,
1 −2 2

Let now x1 , x2 be real numbers. We have, by definition of the operations

“+” and “·” in R c3 :
   
1 −1
x1 · v1 + x2 · v2 = x1 ·  2  + x2 ·  1 
1 −2
     
x1 −x2 x1 − x2
=  2x1  +  x2  =  2x1 + x2  .
x1 −2x2 x1 − 2x2
56 SECTION 4

Hence, the pair (x1 , x2 ) of real numbers satisfies the system (A) of linear
equations above if and only if we have:

x1 · v1 + x2 · v2 = w;

It follows immediately from this that:

• If w is not in the linear span of v1 and v2 , then system (A) has no

solution,
• if w is in the linear span of v1 and v2 , then system (A) has at least
one solution (but we can’t yet tell how many!).

2. Consider now the following system of 2 linear equations in 2 unknowns:


x1 − x2 = 1,
(B)
−x1 + x2 = 1,

where we wish to solve for the pair (x1 , x2 ) of real numbers. In order
to understand whether this system has a solution or not, consider the
c2 :
following vectors in R
     
1 −1 1
v1 = , v2 = ,w = .
−1 1 1

Let now (x1 , x2 ) be a pair of real numbers; we clearly have:

     
1 −1 x1 − x2
x1 · v1 + x2 · v2 = x1 · + x2 · = .
−1 1 −x1 + x2

It follows that the pair (x1 , x2 ) is a solution of system (B) if and only if
we have:

x1 · v1 + x2 · v2 = w;

Hence, we again have:

• If w is not in the linear span of v1 and v2 , then system (B) has no

solution,
• if w is in the linear span of v1 and v2 , then system (B) has at least
one solution (again we can’t yet tell how many!).

So we now have some basic “picture” of what it means for a system of linear
equations to have, or not to have, a solution: If some given vector happens to lie
in some given subspace, then the system has a solution (at least one); otherwise,
it has no solution.
In order to get to a point where we can explain why that number of solutions
is always 0,1, or ∞, we have to further sharpen our tools. This is what we will
do in the following sections.
 57

PROBLEMS:

1. Consider the (by now familiar) real vector space R2 , consisting of all pairs
(x, y) of real numbers, with the usual addition and scalar multiplication
operations. Consider the vectors v1 , v2 , v3 , v4 , v5 in R2 defined as follows:

v1 = (0, 0), v2 = (1, 0), v3 = (−1, 0), v4 = (0, 3), v5 = (2, 1).

(a) Is v1 in the linear span of {v2 } ?

(b) Is v1 in the linear span of {v3 , v4 } ?
(c) Is v2 in the linear span of {v1 , v4 } ?
(d) Is v2 in the linear span of {v1 } ?
(e) Is v2 in the linear span of {v2 , v4 } ?
(f) Is v3 in the linear span of {v2 } ?
(g) Is v3 in the linear span of {v4 } ?
(h) Is v4 in the linear span of {v1 , v2 , v3 } ?
(i) Is v5 in the linear span of {v2 } ?
(j) Is v5 in the linear span of {v1 , v2 , v3 } ?
(k) Is v5 in the linear span of {v2 , v4 } ?
(l) Is v5 in the linear span of {v3 , v4 } ?

2. Consider the real vector c4

 space
 R (see Lecture 2) consisting of all column
x
 y 
vectors of the form  
 z  with x, y, z, w real, with the usual addition
w
and scalar multiplication operations. Define the vectors v1 , v2 , v3 , v4 in
c4 as follows:
R
       
1 2 1 0
 0   1   1   0 
    
v1 =   , v2 =   , v3 =   , v4 =    ,
0 0 1 1 
0 0 0 1

For each of these four vectors, determine whether they are in the linear
span of the other three.
3. Consider the (familiar) real vector space R3 consisting of all triples of the
form (x, y, z) with x, y, z real numbers, with the usual addition and scalar
multiplication operations. For each of the following list of vectors in R3 ,
determine whether the first vector is in the linear span of the last two:
(a) (1, 1, 1), (1, 2, 1), (1, 3, 1)
(b) (0, 0, 0), (1, 2, 1), (1, 3, 1)
58 SECTION 4

(c) (1, 2, 1), (1, 1, 1), (1, 3, 1)

(d) (0, 1, 0), (1, 2, 1), (1, 3, 1)
(e) (1, 1, 1), (2, 5, 0), (3, 2, 0)
(f) (1, 0, 1), (0, 2, 0), (0, 3, 0)
(g) (1, 1, 1), (0, 0, 0), (2, 2, 2)
(h) (1, 1, 0), (1, 0, 1), (0, 1, 1)
(i) (3, 2, 1), (3, 2, 2), (1, 1, 1)
(j) (0, 0, 0), (3, 2, 1), (4, 3, 1)
(k) (1, 2, 1), (2, 2, 1), (1, 3, 1)
4. Consider the real vector space F (R; R) of all real-valued functions of a
real variable (defined in Lecture 2), with the usual addition and scalar
multiplication operations. Consider the elements f1 , f2 , f3 , f4 in F (R; R),
defined as follows:

0, t < 0
f1 (t) =
t, t ≥ 0

0, t < 0
f2 (t) =
3t + 1, t ≥ 0

0, t < 0
f3 (t) =
t2 , t ≥ 0

0, t < 0
f4 (t) =
3t2 + 2t + 1, t ≥ 0

For each of these four elements (i.e. vectors) in F (R; R), verify whether
or not they are in the linear span of the other three.
Section 5

Study Topics
• Linear dependence of a set of vectors
• Linear independence of a set of vectors

59
60 SECTION 5

Consider the vector space R2 of all pairs (x, y) of real numbers, which we have
defined previously and used numerous times. Consider the following subsets of
R2 :
S1 = {(3, 2), (12, 8)}, S2 = {(3, 2), (0, 1)}.
What are the similarities and differences between S1 and S2 ? Well, they both
contain two vectors each; in that, they are similar; another similarity is that
they both contain the vector (3, 2). What about differences ? Notice that the
second vector in S1 , namely (12, 8), is a scalar multiple of (3, 2), i.e. (12, 8)
can be obtained by multiplying (3, 2) by some real number (in this case, the real
number 4); indeed, we have:
4(3, 2) = (4 × 3, 4 × 2) = (12, 8).
We can also obviously claim that (3, 2) is a scalar multiple of (12, 8) (this time
with a factor of 41 ), since we can write:
1 1 1
(12, 8) = ( × 12, × 8) = (3, 2).
4 4 4
On the other hand, no such thing is happening with S2 ; indeed, (0, 1) is not
a scalar multiple of (3, 2), since mutiplying (3, 2) by a real number can never
yield (0, 1) (prove it!). Similary, multiplying (0, 1) by a real number can never
yield (3, 2) (prove it also!).
So we have identified a fundamental difference between S1 and S2 : Whereas the
two elements of S1 are related, in the sense that they are scalar multiples of
one another, those of S2 are completely unrelated (in the sense that they are
not scalar multiples of each other).
Before going further, we generalize this observation into a definition:
Definition 16. Let (V, +, ·) be a real vector space, and let S = {v1 , v2 , · · · , vp }
be a finite subset of V.
(i) The subset S is said to be linearly independent if for any α1 , · · · , αp ∈ R,
the relation
α1 v1 + α2 v2 + · · · + αp vp = 0
implies that α1 = α2 = · · · = αp = 0.
(ii) The subset S is said to be linearly dependent if it is not linearly in-
dependent.
Equivalent ways to re-state these definitions are as follows:
(i) The subset S is linearly independent if the only real numbers α1 , α2 , · · · , αp
which yield
α1 v1 + α2 v2 + · · · + αp vp = 0
are given by α1 = α2 = · · · = αp = 0.
 61

(ii) The subset S is linearly independent if whenever the linear combination

α1 v1 + · · · + αp vp is equal to 0, it must be that α1 , · · · , αp are all 0.

(iii) The subset S is linearly dependent if there exist real numbers α1 , · · · , αp

not all 0, but for which the linear combination α1 v1 + · · · + αp vp is never-
theless equal to 0.

Before going any further, let us show that the set S1 above is a linearly de-
pendent subset of R2 , and that the set S2 above is a linearly independent
subset of R2 . Let us begin with S1 : Let α1 = −4 and α2 = 1; we have:

α1 (3, 2) + α2 (12, 8) = −4(3, 2) + 1(12, 8)

= (−12, −8) + (12, 8)
= (0, 0)
= 0

(recall that the zero element 0 of R2 is the pair (0, 0)). Hence we have found real
numbers α1 , α2 which are not both zero but such that the linear combination
α1 (3, 2) + α2 (12, 8) is the zero vector 0 = (0, 0) of R2 . This proves that S1 is a
linearly dependent subset of R2 .
Let us now show that the set S2 above is a linearly independent subset of
R2 . To do this, we have to show that if for some real numbers α1 , α2 we have

α1 (3, 2) + α2 (0, 1) = (0, 0),

then, necessarily, we must have α1 = 0 and α2 = 0. Note that:

α1 (3, 2) + α2 (0, 1) = (3α1 , 2α1 ) + (0, α2 )

= (3α1 , 2α1 + α2 ).

Hence, if the linear combination α1 (3, 2) + α2 (0, 1) is to be equal to the zero

vector (0, 0) of R2 , we must have (3α1 , 2α1 + α2 ) = (0, 0), or, equivalently, we
must have 3α1 = 0 and 2α1 + α2 = 0; the relation 3α1 = 0 implies α1 = 0, and
substituting this value of α1 in the second relation yields α2 = 0. Hence, we
have shown that if the linear combination α1 (3, 2)+α2 (0, 1) is to be equal to the
zero vector (0, 0) for some real numbers α1 and α2 , then we must have α1 = 0
and α2 = 0. This proves that the subset S2 of R2 is linearly independent.
At this point, we may wonder how these notions of linear dependence and inde-
pendence tie in with our initial observations about the subsets S1 and S2 of R2 ,
namely, that in S1 one of the elements can be written as a scalar multiple of the
other element, whereas we cannot do the same thing with the two elements of
S2 ; how does this relate to linear dependence and independence of the subsets
? The answer is given in the theorem below:

Theorem 4. Let (V, +, ·) be a real vector space, and let S = {v1 , v2 , · · · , vp } be

a finite subset of V . Then:
62 SECTION 5

(i) If S is a linearly dependent subset of V, then at least one of the elements

of S can be written as a linear combination of the other elements of
S;
(ii) conversely, if at least one of the elements of S can be written as a linear
combination of the other elements of S, then S is a linearly depen-
dent subset of V.
Proof. Let us first prove (i): Since S is assumed linearly dependent, there do
exist real numbers α1 , α2 , · · · , αp not all 0, such that:
α1 v1 + α2 v2 + · · · + αp vp = 0.
Since α1 , α2 , · · · , αp are not all equal to 0, at least one of them must be non-
zero; Assume first that α1 is non-zero, i.e. α1 = 6 0. From the above equality, we
deduce:
α1 v1 = −α2 v2 − α3 v3 − · · · − αp vp ,
1
and multiplying both sides of this last equality by α1 (which we can do since
α1 6= 0), we obtain:
α2 α3 αp
v1 = − v2 − v3 − · · · − vp ,
α1 α1 α1
which shows that v1 is a linear combination of v2 , v3 , · · · , vp . Now if it is
α2 which happens to be non-zero, and not α1 , we repeat the same procedure,
but with α2 and v2 instead, and we obtain that v2 is a linear combination of
v1 , v3 , v4 , · · · , vp . If instead, it is α3 which is non-zero, then we repeat the same
procedure with α3 and v3 ; and so on. Since one of the α1 , α2 , · · · , αp is non-
zero, we will be able to write one of the v1 , v2 , · · · , vp as a linear combination
of the others.
Let us now prove (ii): Assume then, that one of the v1 , v2 , · · · , vp can be
written as a linear combination of the others. Again, for simplicity, assume it
is v1 which can be written as a linear combination of the other vectors, namely
v2 , v3 , · · · , vp (if it is another vector instead of v1 , we do the same thing for
that vector); hence, there exist real numbers α2 , α3 , · · · , αp such that:
v1 = α2 v2 + α3 v3 + · · · + αp vp .
We can write the above equality equivalently as:
−v1 + α2 v2 + α3 v3 + · · · + αp vp = 0,
and, equivalently, as:
α1 v1 + α2 v2 + α3 v3 + · · · + αp vp = 0,
where α1 = −1; but this shows that the above linear combination is the zero
vector 0, and yet not all of α1 , α2 , · · · , αp are 0 (for the good reason that α1 =
−1). This shows that the subset S = {v1 , v2 , · · · , vp } of V is linearly dependent.
 63

We now examine a number of examples.

(a) Recall the real vector space c3

 R consisting of all column vectors (with 3
a
real entries) of the form  b , where a, b, c are real numbers. Consider
c
the subset S of R c3 given by:
     
1 0 0
S = { 0  ,  1  ,  0 }.
0 0 1

c3 : Let then
Let us show that S is a linearly independent subset of R
α1 , α2 , α3 be three real numbers such that
       
1 0 0 0
α1  0  + α2  1  + α3  0  =  0  ;
0 0 1 0

we have to show that this implies that α1 = α2 = α3 = 0. We have:

       
1 0 0 α1
α1  0  + α2  1  + α3  0  =  α2  ,
0 0 1 α3

and the equality

   
α1 0
 α2  =  0 
α3 0

does imply that α1 = α2 = α3 = 0. Hence, S is a linearly independent

c3 .
subset of R
(b) Recall the real vector space F (R; R) of all real-valued functions on R;
consider the two elements f1 , f2 in F (R; R), defined as follows:
 
1, t < 0, 0, t < 0,
f1 (t) = f2 (t) =
0, t ≥ 0, 1, t ≥ 0.

We wish to show that {f1 , f2 } is a linearly independent subset of

F (R; R). Let then α1 , α2 be two real numbers such that

α1 f1 + α2 f2 = 0;

(recall that the zero element 0 of F (R; R) is defined to be the constant 0

function). Hence, we must have

α1 f1 (t) + α2 f2 (t) = 0,
64 SECTION 5

for all t in R; in particular, this relation should hold for t = 1 and t = −1.
But for t = 1, we have:

α1 f1 (1) + α2 f2 (1) = α2 ,

and for t = −1, we get:

α1 f1 (−1) + α2 f2 (−1) = α1 .

Hence we must have α2 = 0 and α1 = 0, which shows that {f1 , f2 } is a

linearly independent subset of F (R; R).
(c) Let (V, +, ·) be any real vector space, and let S = {0} be the subset of
V consisting of the zero element alone. Let us show that S is a linearly
dependent subset of V: For this, consider the linear combination 1 · 0;
we evidently have 1 · 0 = 0; on the other hand, the (unique) coefficient
of this linear combination (namely the real number 1) is non-zero. This
shows that {0} is a linearly dependent subset of V.

Consider now a real vector space (V, +, ·), and let S and T be two finite subsets
of V such that S ⊂ T (i.e. S is itself a subset of T ); suppose we know that S
is a linearly dependent subset of V; what can we then say about T ? The
following lemma does answer this question.
Lemma 4. Let (V, +, ·) be a real vector space, and S, T two finite subsets of V
such that S ⊂ T . If S is linearly dependent, then T is also linearly dependent.
Proof. Assume S has p elements and T has q elements (necessarily, we have
q ≥ p since S is a subset of T ). Let us then write:

S = {v1 , v2 , · · · , vp },
T = {v1 , v2 , · · · , vp , vp+1 , vp+2 , · · · , vq }.

Since S is by assumption a linearly dependent subset of V, there do exist

real numbers α1 , α2 , · · · , αp not all zero, such that the linear combination

α1 v1 + α2 v2 + · · · + αp vp

is equal to the zero vector 0 of V, i.e.,

α1 v1 + α2 v2 + · · · + αp vp = 0.

But then, the linear combination

α1 v1 + α2 v2 + · · · + αp vp + 0vp+1 + · · · + 0vq

is also equal to 0, and yet, not all of the coefficients in the above linear com-
bination are zero (since not all of the α1 , α2 , · · · , αp are zero); this proves that
T is a linearly dependent subset of V.
 65

Remark 2. Let (V, +, ·) be a real vector space. We have seen in Example (c) above
that the subset {0} of V consisting of the zero vector alone is linearly dependent;
it follows from the previous lemma that if S is any finite subset of V such that
0 ∈ S, then S is linearly dependent.
Remark 3. The previous lemma also implies the following: If a finite subset S of a
vector space V is linearly independent, then any subset of S is linearly independent
as well.
Remark 4. Let (V, +, ·) be a real vector space, and let {v1 , · · · , vp } be a finite
subset of V. We will sometimes write “v1 , · · · , vp are linearly independent (resp.
dependent) elements of V” instead of “{v1 , · · · , vp } is a linearly independent (resp.
dependent) subset of V”; the two statements in quotes are meant to say exactly the
same thing. One reason we will occasionally prefer to express linear dependence or
independence of v1 , · · · , vp by writing a statement such as “v1 , · · · , vp are linearly
independent (resp. dependent)” is because this does not require v1 , · · · , vp to
be all distinct; if, however, we were to write “the subset {v1 , · · · , vp } is linearly
independent (resp. dependent)”, this would assume that v1 , · · · , vp are all distinct.
To give a concrete example, let v1 ∈ V be any element of V. Then we can say that
the vectors v1 and v1 are linearly dependent (since, for example, (1)·v1 +(−1)·v1 =
0V , i.e. there is a linear combination of the vectors v1 and v1 that gives the
zero vector of V but in which at least one coefficient is non-zero); on the other
hand, the subset of V consisting of the vectors v1 and v1 is the subset {v1 } (not
{v1 , v1 } Remember that repetitions are not allowed when writing sets by listing
their elements), so we would have no way of discussing the linear dependence or
independence of the vectors v1 and v1 by constructing a subset of V from them.
Remark 5. Continuing the previous remark, it should be pointed out that when
we write a statement such as “v1 , · · · , vp are linearly independent (resp. depen-
dent) elements of V” we mean that the “collection” or “family” consisting of the
vectors v1 , · · · , vp is linearly independent (resp. dependent), and not that each of
v1 , · · · , vp is separately linearly independent (resp. dependent). In other words, the
usage of the plural here is not meant in the usual sense it is commonly employed.
This is because linear dependence/independence is a collective property of all the
vectors involved. To give an example, a statement such as “the numbers 2, 4, 6 are
even” means that 2 is even, 4 is even, and 6 is even, whereas a statement such as
“v1 , v2 , v3 are linearly independent” means that the collection or family consisting
of the vectors v1 , v2 , v3 is linearly independent, i.e. for any α1 , α2 , α3 ∈ R such
that α1 · v1 + α2 · v2 + α3 · v3 = 0V it necessarily follows that α1 = α2 = α3 = 0.

PROBLEMS:

1. Consider the following vectors in the real vector space R2 :

v1 = (1, 0), v2 = (3, 0), v3 = (0, 0), v4 = (1, 1), v5 = (2, 1).
66 SECTION 5

For each of the following finite subsets of R2 , specify whether they are
linearly dependent or linearly independent:
(a) {v1 }
(b) {v1 , v2 }
(c) {v2 }
(d) {v3 }
(e) {v1 , v4 }
(f) {v2 , v4 }
(g) {v3 , v4 }
(h) {v4 , v5 }
(j) {v1 , v4 , v5 }
(k) {v2 , v4 , v5 }
(l) {v1 , v2 , v4 , v5 }
(m) {v1 , v5 }

2. Consider the following vectors in the real vector space Rc4 :

         
1 2 1 0 0
 0   1   1   0   1 
v1 =        
 0  , v2 =  0  , v3 =  1  , v4 =  1  , v5 =  0
 .

0 0 0 1 1

For each of the following finite subsets of R2 , specify whether they are
linearly dependent or linearly independent:
(a) {v1 }
(b) {v1 , v2 }
(c) {v2 }
(d) {v3 }
(e) {v1 , v4 }
(f) {v2 , v4 }
(g) {v3 , v4 }
(h) {v4 , v5 }
(j) {v1 , v4 , v5 }
(k) {v2 , v4 , v5 }
(l) {v1 , v2 , v4 , v5 }
(m) {v1 , v5 }
(n) {v1 , v2 , v3 , v4 , v5 }
(o) {v3 , v4 , v5 }
 67

(p) {v2 , v3 , v4 , v5 }
3. Consider the real vector space (F (R; R), +, ·) defined in Lecture 2, and
consider the elements f1 , f2 , f3 , f4 in F (R; R), defined as follows:

f1 (t) = 1, ∀t ∈ R,
f2 (t) = t, ∀t ∈ R,
f3 (t) = (t + 1)2 , ∀t ∈ R,
f4 (t) = t2 + 5, ∀t ∈ R,
f5 (t) = t3 , ∀t ∈ R,
f6 (t) = t3 + 2t2 , ∀t ∈ R,

For each of the following finite subsets of F (R; R), specify whether they
are linearly dependent or linearly independent:
(a) {f1 }
(b) {f1 , f2 }
(c) {f1 , f2 , f3 }
(d) {f1 , f2 , f3 , f4 }
(e) {f2 , f3 , f4 }
(f) {f4 , f5 , f6 }
(g) {f3 f4 , f5 , f6 }
(h) {f2 , f3 , f4 , f5 , f6 }
(j) {f1 , f2 , f3 , f4 , f5 , f6 }
4. Let (V, +, ·) be a real vector space, and let S be a finite subset of V. Show
directly (without using Lemma 1 or Remark 2 of these lectures) that if 0
is in S then S is a linearly dependent subset of V .
68 SECTION 5
Section 6

Study Topics
• Linear dependence/independence and linear combinations
• Application to systems of linear equations (or, we now have
the complete answer to our initial question!)

69
70 SECTION 6

Let (V, +, ·) be a real vector space, and let S = {v1 , v2 , · · · , vp } be a finite

subset of V. We know that if the element v of V is in the linear span of
the vectors v1 , v2 , · · · , vp , then v can be written as some linear combination
of v1 , v2 , · · · , vp , that is, we can find real numbers α1 , α2 , · · · , αp such that

α1 v1 + α2 v2 + · · · αp vp = v;

A very natural question at this point is: In how many different ways can
this same v be expressed as a linear combination of v1 , v2 , · · · , vp ? Does there
exist another p−tuple of real numbers, say (γ1 , γ2 , · · · , γp ), distinct from the
p−tuple (α1 , α2 , · · · , αp ) such that we also have

γ1 v1 + γ2 v2 + · · · γp vp = v ?

The complete answer to this question is given in the following theorems:

Theorem 5. Let (V, +, ·) be a real vector space, and let S = {v1 , v2 , · · · , vp }
be a finite subset of V. Let v ∈ V be in the linear span of v1 , v2 , · · · , vp .
If S is a linearly independent subset of V, then v can be expressed only in
a unique way as a linear combination of v1 , v2 , · · · , vp ; i.e., there is a unique
p−tuple (α1 , α2 , · · · , αp ) of real numbers such that:

α1 v1 + α2 v2 + · · · αp vp = v;

Proof. To prove this result, we have to show that if v (which is assumed to be

in the linear span of v1 , v2 , · · · , vp ) can be expressed as two linear combinations
of v1 , v2 , · · · , vp , then those two linear combinations must be one and the same.
Assume then that we have two p−tuples of real numbers (α1 , α2 , · · · , αp ) and
(γ1 , γ2 , · · · , γp ) such that

α1 v1 + α2 v2 + · · · αp vp = v

and

γ1 v1 + γ2 v2 + · · · γp vp = v.

We have to show that the p−tuples (α1 , α2 , · · · , αp ) and (γ1 , γ2 , · · · , γp ) are

equal, i.e. that α1 = γ1 , α2 = γ2 , · · · , αp = γp . This will show (obviously!)
that those two linear combinations are exactly the same. Now these last two
equalities imply:

α1 v1 + α2 v2 + · · · αp vp = γ1 v1 + γ2 v2 + · · · γp vp ,

which then implies (by putting everything on the left-hand side of the “=” sign):

(α1 − γ1 )v1 + (α2 − γ2 )v2 + · · · (αp − γp )vp = 0;

but by linear independence of {v1 , v2 , · · · , vp }, we must have:

(α1 − γ1 ) = (α2 − γ2 ) = · · · = (αp − γp ) = 0,

 71

which, equivalently, means:

α1 = γ1 ,
α2 = γ2 ,
···
αp = γp ,

i.e. the two linear combinations

α1 v1 + α2 v2 + · · · αp vp = v

and

γ1 v1 + γ2 v2 + · · · γp vp = v.

are one and the same.

Conversely, we have the following:

Theorem 6. Let (V, +, ·) be a real vector space, and let S = {v1 , v2 , · · · , vp }

be a finite subset of V. Assume that any v ∈ V that is in the linear span
of v1 , v2 , · · · , vp can be expressed only in a unique way as a linear combi-
nation of v1 , v2 , · · · , vp ; i.e., for any v ∈ S(v1 ,··· ,vp ) there is a unique p−tuple
(α1 , α2 , · · · , αp ) of real numbers such that:

α1 v1 + α2 v2 + · · · αp vp = v;

Then, S is a linearly independent subset of V.

Proof. Consider the zero vector 0 of V, and Let α1 , α2 , · · · , αn ∈ R be such

that

α1 v1 + α2 v2 + · · · + αp vp = 0;

to show that S is linearly independent, we have to show that this last equality
implies that α1 , α2 , · · · , αp are all zero. Note first that 0 is in the linear span of
v1 , v2 , · · · , vp ; indeed, we can write:

0 = 0v1 + 0v2 + · · · + 0vp .

Since we have assumed that any element in the linear span of v1 , v2 , · · · , vp can
be written only in a unique way as a linear combination of v1 , v2 , · · · , vp , and
since we have written 0 as the following linear combinations of v1 , v2 , · · · , vp :

0 = α1 v1 + α2 v2 + · · · + αp vp ,
0 = 0v1 + 0v2 + · · · + 0vp ,
72 SECTION 6

it follows that these two linear combinations must be one and the same, i.e.
their respective coefficients must be equal, i.e., we must have:

α1 = 0,
α2 = 0,
···
αp = 0.

Hence, to summarize the previous steps, we have shown that

α1 v1 + α2 v2 + · · · + αp vp = 0

implies α1 = α2 = · · · = αp = 0. This proves that S is a linearly independent

subset of V.
Now that we know the full story for the case when S is a linearly independent
subset of V, let us examine the case when S is a linearly dependent subset
of V:
Theorem 7. Let (V, +, ·) be a real vector space, and let S = {v1 , v2 , · · · , vp } be
a finite subset of V. Let v ∈ V be in the linear span of v1 , v2 , · · · , vp . If S is a
linearly dependent subset of V, then v can be expressed in infinitely many
distinct ways as a linear combination of v1 , v2 , · · · , vp ; i.e., there exist infinitely
many distinct p−tuples (α1 , α2 , · · · , αp ) of real numbers such that:

α1 v1 + α2 v2 + · · · αp vp = v;

Proof. Let us assume then that the finite subset S = {v1 , v2 , · · · , vp } of V is

linearly dependent, and let v be any element in the linear span of S. We
have to “produce” or “manufacture” infinitely many distinct p−tuples of real
numbers (α1 , α2 , · · · , αp ) for which we have:

α1 v1 + α2 v2 + · · · αp vp = v.

Since v is assumed to be in the linear span of S, we know that there is at least

one such p−tuple. Let that p−tuple be (λ1 , · · · , λp ) (to change symbols a bit!),
i.e. assume we have

λ1 v1 + λ2 v2 + · · · λp vp = v.

Since S = {v1 , v2 , · · · , vp } is assumed to be a linearly dependent subset of

V, we know (by definition of linear dependence) that there exist real numbers
β1 , β2 , · · · , βp not all 0, such that:

β1 v1 + β2 v2 + · · · + βp vp = 0.

Consider now for each real number µ the p−tuple of real numbers given by:

(λ1 + µβ1 , λ2 + µβ2 , · · · , λp + µβp );

 73

since at least one of β1 , β2 , · · · , βp is non-zero, we obtain infinitely many

distinct p−tuples of real numbers in this way, one for each µ ∈ R. To see this,
note that the entry in the p−tuple

(λ1 + µβ1 , λ2 + µβ2 , · · · , λp + µβp )

corresponding to that particular βi which is non-zero will vary as µ varies, and

will never have the same value for two different values of µ.
Finally, note that for each µ in R, the p−tuple of real numbers

(λ1 + µβ1 , λ2 + µβ2 , · · · , λp + µβp )

yields:

(λ1 + µβ1 )v1 + · · · + (λp + µβp )vp = (λ1 v1 + · · · + λp vp )

+ µ(β1 v1 + · · · + βp vp )
= v + µ0
= v.

This last calculation shows that the linear combination of v1 , v2 , · · · , vp with

coefficients given by the entries of the p−tuple (λ1 + µβ1 , · · · , λp + µβp ) yields
the vector v; since we have shown that there are infinitely many distinct
p−tuples (λ1 + µβ1 , · · · , λp + µβp ) (one for each µ in R), this shows that v can
be written as a linear combination of v1 , v2 , · · · , vp in infinitely many distinct
ways.
Conversely, we have the following:
Theorem 8. Let (V, +, ·) be a real vector space, and let S = {v1 , v2 , · · · , vp }
be a finite subset of V. Assume that there exists a vector v ∈ V which is in the
linear span of v1 , v2 , · · · , vp and such that it can be expressed as two distinct
linear combinations of v1 , v2 , · · · , vp , i.e. there exist two distinct p−tuples
(α1 , α2 , · · · , αp ) and (β1 , β2 , · · · , βp ) of real numbers such that:

v = α1 v1 + α2 v2 + · · · αp vp ,
v = β1 v1 + β2 v2 + · · · βp vp ;

Then, S is a linearly dependent subset of V.

Proof. Subtracting the second linear combination from the first yields:

0 = v − v = (α1 − β1 )v1 + (α2 − β2 )v2 + · · · (αp − βp )vp ,

and since the two n−tuples (α1 , α2 , · · · , αp ) and (β1 , β2 , · · · , βp ) are assumed
distinct, there should be some integer i in {1, 2, · · · , p} such that αi 6= βi , i.e.
such that (αi − βi ) 6= 0; but this implies that the above linear combination of
v1 , v2 , · · · , vp is equal to the zero vector 0 but does not have all its coefficients
zero; as a result, S is linearly dependent.
74 SECTION 6

APPLICATIONS TO SYSTEMS OF LINEAR EQUATIONS

We can now give a complete answer to our initial question:

• why is it that a system of linear equations can have only 0,1, or infinitely
many solutions ?

Consider then the system of linear equations



 a11 x1 + a12 x2 + a13 x3 + · · · a1n xn = b1

a21 x1 + a22 x2 + a23 x3 + · · · a2n xn = b2
(E)

 a31 x1 + a32 x2 + a33 x3 + · · · a3n xn = b3

am1 x1 + am2 x2 + am3 x3 + · · · amn xn = bm

where the aij and the bk are given real numbers. We would like to know how
many distinct n−tuples (x1 , x2 , · · · , xn ) of real numbers are a solution to that
system of equations (i.e. satisfy all the equalities above).
In order to answer this question using the tools we have developed, consider the
real vector space (R d m , +, ·) which we are by now familiar with, and consider the

vectors v1 , v2 , · · · , vn in Rd m defined as follows:

     
a11 a12 a1n
 a21   a22   a2n 
     
v1 =   31  a  , v 2 =  a
 32 
 , · · · , vn =  a3n  ,
 
 ···   ···   ··· 
am1 am2 amn

d
as well as the vector w in R m defined as:

 
b1
 b2 
 
w=  b3  .

 ··· 
bm

It is clear that the n−tuple of real numbers (x1 , x2 , · · · , xn ) is a solution to

the system (E) of linear equations above if and only if it satisfies:

x1 v1 + x2 v2 + · · · + xn vn = w.

We can therefore state:

• If w is not in the linear span of v1 , v2 , · · · , vp , then there is no n−tuple

of real numbers (x1 , x2 , · · · , xn ) satisfying the equality x1 v1 + x2 v2 + · · · +
xn vn = w; in this case therefore, the system of linear equations (E) has
NO SOLUTION;
 75

• If now w is in the linear span of v1 , v2 , · · · , vp , then there are 2 subcases:

– If the vectors v1 , v2 , · · · , vp are linearly independent, then there
is a unique n−tuple of real numbers (x1 , x2 , · · · , xn ) satisfying the
equality x1 v1 + x2 v2 + · · · + xn vn = w; in this case therefore, the
system of linear equations (E) has a UNIQUE SOLUTION;
– If on the other hand the vectors v1 , v2 , · · · , vp are linearly depen-
dent, then there are infinitely many n−tuples of real numbers
(x1 , x2 , · · · , xn ) satisfying the equality x1 v1 + x2 v2 + · · ·+ xn vn = w;
in this case therefore, the system of linear equations (E) has IN-
FINITELY MANY SOLUTIONS;

PROBLEMS:

1. For each of the following systems of linear equations, study the number of
solutions by formulating and analyzing the corresponding linear algebra
problem, as done in the last section of this Lecture under the heading
“Application to Systems of Linear Equations”.
(a)

2x = 4,
4x = 5,

(b)

2x = 4,
4x = 8,

(c)

0x = 1,

(d)

0x = 0,

(e)

 2x + y = 1,
10x + 5y = 5,

x − y = 1,

(f)

 x + y − z = 0,
x + y = 1,

y + z = 2,
76 SECTION 6

(g)

x + y − z − w = 0,
x + y = 2,

(h)

 x + y − z − w = 0,
−x − y = 2,

z + w = 3,

(i)

2x + y = 1,
4x + 2y = 2,

(j)

2x + y = 1,
4x + 2y = 3,

(k)

2x + y = 1,
x = 1,

(l)

2x + y = 1,
y = 0,

(m)

x + y + z = 1,

(n)

x + y + z = 1,
x=0

(o)


 x + y + z = 1,

x=0

 y=0

z=0
 77

(p)

 x + y + z = 1,
x=0

y=0

(q)

 x + y + z = 1,
2x + 2y + 2z = 2

y=0

(r)


 x + y + z = 1,

2x + 2y + 2z = 2

 x=0

y=0

(s)


 x + y + z = 1,


 2x + 2y + 2z = 2
x=0



 y=0

z=0
78 SECTION 6
Section 7

Study Topics
• Systems of linear equations in upper triangular form
• Solution by back-substitution
• Gaussian Elimination to put systems of linear equations in up-
per triangular form
• Augmented matrix of a system of linear equations
• Elementary Row operations

79
80 SECTION 7

We now take a little pause from theory and examine a quick and efficient
approach to solving systems of linear equations. Recall that in Section 1,
we solved systems of linear equations using the ad hoc (and quite primitive!)
procedure of isolating one variable from one of the equations, substituting its
expression in terms of the other variables in all other equations, and repeating
the procedure. Such a scheme could be suitable for small systems of linear
equations (i.e. having few equations and few unknowns), but quickly becomes
unwieldy for larger systems of linear equations. In this section, we will learn a
systematic approach to solving such systems. The approach we will learn is often
called “Gaussian Elimination and Substitution”. To motivate the approach, we
begin with a few examples.
Consider first the system of linear equations given by

x1 + 3x2 − 4x3 + 2x4 = 5,

x2 + 3x3 − 2x4 = 1,
x3 + 4x4 = 2,
2x4 = 6.

Such a system is said to be in upper-triangular form or row-echelon form

since (beyond the obvious fact of its appearance) it has the property that the
coefficient of the first variable (namely x1 ) is zero on row 2 and beyond, the
coefficient of the second variable (namely x2 ) is zero on row 3 and beyond, the
coefficient of the third variable (namely x3 ) is zero on row 4 and beyond, ...
(and this is precisely what gives it its “triangular” appearance!).
Solving a system of linear equations which is in upper-triangular (aka row-
echelon) form is extremely easy, and is accomplished through a process known
as back-substitution. To illustrate this process, we return to our example
above. We begin with the last row and gradually make our way up (this
is precisely where the “back” in “back-substitution” comes from). The equation
on the last row is given by

2x4 = 6,

from which we immediately obtain that x4 = 3. We now substitute this value

of x4 that we just obtained in all other equations, and we obtain:

x1 + 3x2 − 4x3 + 6 = 5,
x2 + 3x3 − 6 = 1,
x3 + 12 = 2,

which, after rearranging, yields the system of linear equations:

x1 + 3x2 − 4x3 = −1,

x2 + 3x3 = 7,
x3 = −10.
 81

Note that this system of linear equations now involves only the variables x1 , x2 , x3
(since we already solved for x4 ), and is also in upper-triangular (aka row-echelon)
form. We begin again with the last row. The equation on the last row is given
by

x3 = −10,

which yields directly that x3 = −10. We now substitute this value of x3 that
we just obtained in all other equations, and we obtain:

x1 + 3x2 + 40 = −1,
x2 − 30 = 7,

which, after rearranging, yields the system of linear equations:

x1 + 3x2 = −41,
x2 = 37.

Note that this system of linear equations now involves only the variables x1 , x2
(since we already solved for x3 and x4 ), and is also in upper-triangular form.
We begin again with the last row. The equation on the last row is given by

x2 = 37,

which yields directly that x2 = 37. We now substitute this value of x2 that
we just obtained into the first equation (since it is the only equation left!), and
we obtain:

x1 + 111 = −41,

which, after rearranging, yields the system of linear equations

x1 = −152.

We have therefore solved our original system, and we have found that the
unique solution to our system is given by (x1 , x2 , x3 , x4 ) = (−152, 37, −10, 3).
The important thing to note is how systematic and painless the whole procedure
was.
We now examine yet another example. Consider the system of linear equations
given by:

2x1 + 4x2 − x3 + 3x4 = 8,

x3 − x4 = 6.

This is again a system of linear equations in upper-triangular (aka row-echelon)

form; indeed, the coefficient of the first variable (namely x1 ) is zero in the second
equation. Solving for x3 in the second equation yields

x3 = x4 + 6,
82 SECTION 7

and substituting this value of x3 into the first equation yields

2x1 + 4x2 − (x4 + 6) + 3x4 = 8,
i.e., after rearranging,
2x1 + 4x2 + 2x4 − 6 = 8,
and solving for x1 in this equation yields
x1 = −2x2 − x4 + 7.
We have thus obtained that for any real numbers x2 and x4 , the 4−tuple of
real numbers given by
(−2x2 − x4 + 7, x2 , x4 + 6, x4 )
is a solution to our system of linear equations. Note that this system has
infinitely many solutions (we sometimes say that we have parametrized our
solutions by x2 and x4 ).
Let us examine one last example; consider the system of linear equations given
by
2x1 + x2 − x3 = 5,
0x3 = 2.
This is again a system of linear equations in upper-triangular form; indeed,
the coefficient of the first variable (namely x1 ) is zero in the second equation.
Solving for x3 in the second equation yields no solution; indeed there is no
real number x3 that satisfies 0x3 = 2. We conclude therefore that this system
of linear equations has no solution.
It is now quite clear from the few examples we have seen that systems of linear
equations in upper-triangular form are very easy to solve thanks to the back-
substitution scheme, and that the fact that the system happens to have a unique
solution, infinitely many solutions, or no solution at all, appears clearly during
the back-substitution process.
Now that we have seen the benefit of systems of linear equations in upper-
triangular form, a natural question we can ask ourselves is whether given a
system of linear equations, we can somehow “transform it into upper-triangular
form” in such a way that the solutions remain exactly the same as for the original
system; can such a transformation always be done ? The answer is a deafen-
ing “YES”, and before introducing the systematic scheme for transforming any
system of linear equations in upper triangular form, we motivate it on a simple
example.
Consider therefore the system of linear equations given by
2x1 + x2 + x3 = 5,
4x1 + x2 + 3x3 = 2,
6x1 + x2 + 4x3 = 0.
 83

Let us multiply the first row by −2 and add the result to the second row;
denoting row 1 by R1, row 2 by R2, and row 3 by R3, the operation we are
describing consists of replacing R2 by −2R1+R2. We shall denote this operation
as follows:

−2R1 + R2 → R2.

After this operation, our system becomes:

2x1 + x2 + x3 = 5,
0x1 − x2 + x3 = −8,
6x1 + x2 + 4x3 = 0.

To see that this system has exactly the same solution (or solutions) as our
original system, note that we can go back to our original system from this
system by replacing the second row with 2 times row 1 + row 2. Whenever two
systems of linear equations have exactly the same solutions, we say that they
are equivalent. By the row operation described by

−2R1 + R2 → R2,

we have therefore gone from our original system of linear equations to an equiv-
alent system. Consider now the (equivalent) system we have obtained:

2x1 + x2 + x3 = 5,
0x1 − x2 + x3 = −8,
6x1 + x2 + 4x3 = 0.

Let us multiply the first row by −3 and add the result to the third row; the
operation we are describing consists of replacing R3 by −3R1 + R3, and can be
denoted as follows:

−3R1 + R3 → R3.

After this operation, our system becomes:

2x1 + x2 + x3 = 5,
0x1 − x2 + x3 = −8,
0x1 − 2x2 + x3 = −15,

which we can also rewrite (by eliminating the terms having coefficient 0) as:

2x1 + x2 + x3 = 5,
−x2 + x3 = −8,
−2x2 + x3 = −15.

Note again that this system is equivalent to the previous one, and therefore,
to our original system. The solutions to this last system are exactly those of
84 SECTION 7

our original system, and vice-versa. Note also that this system is not upper-
triangular, but we’re almost there! To get there, let us now multiply the
second row by −2 and add the result to the third row; the operation we are
describing consists of replacing R3 by −2R2 + R3, and we shall denote it as
follows:
−2R2 + R3 → R3.
After this operation, our system becomes:
2x1 + x2 + x3 = 5,
−x2 + x3 = −8,
−x3 = 1.
Note again that this system is equivalent to the previous one, and therefore to
our original system, and that this last system is now indeed in upper-triangular
form; we can now easily solve this system by back-substitution, and we obtain
that the unique solution of this system, and hence of our original system, is
given by the triple (x1 , x2 , x3 ) = (−1/2, 7, −1).
Let us now describe a systematic approach that captures what we have done;
this process is known as Gaussian Elimination. Before we get there however,
we need some more terminology.
Definition 17. Consider the system of linear equations in m equations and n un-
knowns given by
a1,1 x1 + a1,2 x2 + · · · a1,n xn = b1 ,
a2,1 x1 + a2,2 x2 + · · · a2,n xn = b2 ,
···
am,1 x1 + am,2 x2 + · · · am,n xn = bm .

The augmented matrix of this system is the table of real numbers with m rows
and n + 1 columns given by
 
a1.1 a1,2 ··· a1,n | b1
 a2.1 a2,2 ··· a2,n | b2 
 .
 ··· 
am.1 am,2 ··· am,n | bm
(Note: The vertical bars “|” in this table are meant to visually separate the coef-
ficients that multiply the unknowns from those that appear on the right-hand side
of the equation).
For example, the augmented matrix corresponding to the system of linear equa-
tions
2x1 + x2 + x3 = 5,
4x1 + x2 + 3x3 = 2,
6x1 + x2 + 4x3 = 0.
 85

is given by the table (with 3 rows and 4 columns):

 
2 1 1 | 5
 4 1 3 | 2 .
6 1 4 | 0

It is important to note that, conversely, an m × (n + 1) table of real numbers of

the form
 
a1.1 a1,2 · · · a1,n | b1
 a2.1 a2,2 · · · a2,n | b2 
 .
 ··· 
am.1 am,2 · · · am,n | bm

uniquely defines a system of m linear equations in n unknowns, namely the

system:

a1,1 x1 + a1,2 x2 + · · · a1,n xn = b1 ,

a2,1 x1 + a2,2 x2 + · · · a2,n xn = b2 ,
···
am,1 x1 + am,2 x2 + · · · am,n xn = bm .

Hence, we can represent any system of linear equations by its augmented matrix,
and vice-versa.
We now define what it means for a system of linear equations – or, equivalently,
its augmented matrix, to be in upper-triangular, aka row-echelon form:

Definition 18. The augmented matrix

 
a1.1 a1,2 · · · a1,n | b1
 a2.1 a2,2 · · · a2,n | b2 
 .
 ··· 
am.1 am,2 · · · am,n | bm

is said to be in row-echelon form if the following two conditions are met:

1. Each row with all entries equal to 0 is below every row having at least one
nonzero entry,

2. the leftmost non-zero entry on each row is to the right of the leftmost non-zero
entry of the preceding row.

For example, the augmented matrices

     
2 1 1 | 5 0 2 1 | 5 2 1 1 3 | 5
 0 1 3 | 2 ,  0 0 0 | 2 ,  0 0 3 2 | 2 
0 0 3 | 0 0 0 0 | 0 0 0 0 3 | 0
 86 SECTION 7

are in row-echelon form, whereas the augmented matrices

     
2 1 1 | 5 0 2 1 | 5 2 1 1 3 | 5
 0 1 3 | 2 ,  0 0 0 | 0 ,  0 0 3 2 | 2 
1 0 3 | 0 0 0 0 | 2 0 0 3 3 | 0
are not.
We now formalize the operations we performed on the system of linear equations
that we began with in order to put it in upper-triangular form. We define some
more terminology.
Definition 19. Let
 
a1.1 a1,2 ··· a1,n | b1
 a2.1 a2,2 ··· a2,n | b2 
 .
 ··· 
am.1 am,2 ··· am,n | bm

be the augmented matrix of a system of linear equations. An elementary row

operation on this table consists of one of the following operations:
1. Multiplying a given row by a non-zero scalar;
2. Exchanging any two rows;
3. Adding to a given row a scalar multiple of any other row.
The key results (which we shall prove later, when dealing with matrices) are the
following:
Theorem 9. let T be the augmented matrix corresponding to a system of linear
equations, and let T ′ be the augmented matrix obtained from T by a sequence of
elementary row operations. The system of linear equations corresponding to
T ′ is equivalent to the system of linear equations corresponding to T (i.e. they
have exactly the same solutions).
Theorem 10. let T be the augmented matrix corresponding to a system of linear
equations; then there does exist a sequence of elementary row operations
(by no means unique) such that the augmented matrix T ′ obtained from T by that
sequence of elementary row operations is in row-echelon form.
In light of these two theorems, the Gaussian elimination/substitution pro-
cedure for systematically solving systems of linear equations can be described
as follows:
Step 1: Write down the augmented matrix of the system of linear equations;
Step 2: Transform the augmented matrix in row-echelon form through a
sequence of elementary row operations;
Step 3: Solve the system corresponding to the row-echelon augmented
matrix obtained in Step 2 by back-substitution.
 87

We now illustrate this approach on a number of examples:

1. Consider the system of linear equations given by:

x2 + x3 = 1,
2x1 + x2 − x3 = 0,
4x1 + x2 + 2x3 = 5.

Step 1: The augmented matrix of this system is the 3 × 4 table given by:
 
0 1 1 | 1
 2 1 −1 | 0  .
4 1 2 | 5

Step 2: We put this augmented matrix in row-echelon form through a

sequence of elementary row operations. Exchanging rows 1 and 2 (we
denote this by R1 ↔ R2) yields the augmented matrix
 
2 1 −1 | 0
 0 1 1 | 1 .
4 1 2 | 5

Let us now multiply row 1 by −2 and add the result to row 3 (we denote
this by −2R1 + R3 → R3); we obtain the augmented matrix
 
2 1 −1 | 0
 0 1 1 | 1 .
0 −1 4 | 5

Let us now add row 2 to row 3 (we denote this by R2 + R3 → R3); we

obtain the augmented matrix
 
2 1 −1 | 0
 0 1 1 | 1 .
0 0 5 | 6

This augmented matrix is now in row-echelon form, and it corresponds to

the following system of linear equations:

2x1 + x2 − x3 = 0,
x2 + x3 = 1,
5x3 = 6.

Step 3: Solving this last system by back-substitution yields x3 = 65 ,

7
x2 = − 15 , x1 = 10 . Hence, we can state that the unique solution of our
7
original system of linear equations is given by (x1 , x2 , x3 ) = ( 10 , − 51 , 65 ).
88 SECTION 7

2. Consider now the system of linear equations given by:

x2 + x3 = 2,
2x2 + 2x3 = 2,
x1 + x3 = 1.
Step 1: The augmented matrix of this system is the 3 × 4 table given by:
 
0 1 1 | 2
 0 2 2 | 2 .
1 0 1 | 1

Step 2: We put this augmented matrix in row-echelon form through a

sequence of elementary row operations. Exchanging rows 1 and 3 (we
denote this by R1 ↔ R3) yields the augmented matrix
 
1 0 1 | 1
 0 2 2 | 2 .
0 1 1 | 2
Let us now multiply row 2 by −1/2 and add the result to row 3 (we denote
this by − 21 R2 + R3 → R3); we obtain the augmented matrix
 
1 0 1 | 1
 0 2 2 | 2 .
0 0 0 | 1
This augmented matrix is now in row-echelon form, and it corresponds to
the following system of linear equations:
x1 + x3 = 1,
2x2 + 2x3 = 2,
0x3 = 1.
Step 3: Solving this last system by back-substitution involves first solving
for x3 from the equation 0x3 = 1. Clearly there is no real number that
satisfies this equation. We conclude that our original system of linear
equations has no solution.
3. Consider now the system of linear equations given by:
x1 + x3 − x4 = 2,
2x1 + 2x4 = 0,
x1 + x2 − x4 = 1.
Step 1: The augmented matrix of this system is the 3 × 5 table given by:
 
1 0 1 −1 | 2
 2 0 0 2 | 0 .
1 1 0 −1 | 1
 89

Step 2: We put this augmented matrix in row-echelon form through a

sequence of elementary row operations. Let us multiply row 1 by −2 and
add the result to row 2 (we denote this by −2R1 + R2 → R2); we obtain
the augmented matrix
 
1 0 1 −1 | 2
 0 0 −2 4 | −4  .
1 0 0 −1 | 1

This augmented matrix is not in row-echelon form yet; let us now multiply
row 1 by −1 and add the result to row 3 (we denote this by −R1 + R3 →
R3); we obtain the augmented matrix
 
1 0 1 −1 | 2
 0 0 −2 4 | −4  .
0 0 −1 0 | −1

This augmented matrix is still not in row-echelon form yet; let us now
multiply row 2 by −1/2 and add the result to row 3 (we denote this by
− 12 R2 + R3 → R3); we obtain the augmented matrix
 
1 0 1 −1 | 2
 0 0 −2 4 | −4  .
0 0 0 −2 | 1

and this augmented matrix is now indeed in row-echelon form, and it

corresponds to the linear system of equations

x1 + x3 − x4 = 2,
−2x3 + 4x4 = −4,
−2x4 = 1.

Step 3: Solving this last system by back-substitution involves first solving

for x4 from the last equation, namely −2x4 = 1. This yields x4 = − 12 .
Substituting this value of x4 in the remaining equations yields the system
1
x1 + x3 + = 2,
2
−2x3 − 2 = −4,

which, after rearranging, yields:

3
x1 + x3 = ,
2
−2x3 = −2,
90 SECTION 7

Solving for x3 from the last equation yields x3 = 1, and substituting this
value of x3 in the first equation yields
3
x1 + 1 = ,
2

i.e.
1
x1 = .
2

We have completed the back-substitution process, and we have found that

x1 = 12 , x2 is any real number, x3 = 1, and x4 = − 21 . We conclude that
our original system of equations has infinitely many solutions, and each
4−tuple of the form ( 21 , x2 .1, − 21 ), with x2 any real number, is a solution
to our original system.
Remark 6. An augmented matrix is said to be in reduced row-echelon form if:
• It is in row echelon form, and
• the first non-zero entry in each row is 1, and
• the first non-zero entry in each row is the only non-zero entry in its column.
Any augmented matrix can be put in reduced row-echelon form using a sequence
of elementary row operations; such a sequence of operations is then called Gauss-
Jordan elimination.

PROBLEMS:

1. Solve each of the following systems of linear equations using the Gaussian
Elimination and Back-Substitution method described in this Section, i.e.
for each system:

(i) Write the augmented matrix,

(ii) convert the augmented matrix in row-echelon form using elementary
row operations,
(iii) finally solve the resulting system by back-substitution.

(a)

x1 + x2 + x3 = 0,
2x1 + x3 = 1,
x1 − 3x3 = 2.
 91

(b)

x1 + x2 + x3 = 0,
x1 + x3 = 1,
x1 − 3x3 = 2,
x1 + 2x3 = 1.

(c)

x1 + x2 + x3 = 0,
x1 + 2x3 = 1,

(d)

x1 + x2 + x3 = 0,
x1 + x2 = −1,

(e)

x1 + x2 + x4 = 0,
x1 + x3 = 1,
x1 − x4 = 2,
x1 + x2 = 1.

(f)

2x3 + x4 = 2,
x1 + x2 = 0,
x1 − x3 = 1,
x3 + x4 = 0.

(g)

2x3 + x4 = 2,
x1 + x2 = 0,
x1 − x3 = 1,
x3 + x4 = 0,
x1 + x2 + x3 + x4 = 1.

(h)

x2 + x3 = 1,
x1 − x4 = 1,
92 SECTION 7

2. Consider the system of equations given by

x1 + x2 + x3 = 1 + a,
2x1 + 2x2 + 4x3 = 2,
3x1 + 3x2 + 2x3 = a,

where a is a real number. By putting this system in row-echelon form,

determine for which values of a it has no solution.

3. Consider the system of equations given by

x1 + x2 + x3 = a,
x1 + x3 = 2a,
4x1 + 4x3 = a,

where a is a real number. By putting this system in row-echelon form,

determine for which values of a it has no solution.

4. Consider the system of equations given by

x1 + x2 + 2x3 + 2x4 = a,
x1 + x2 + 2x4 = a,
2x1 + 4x4 = 1,
−x1 + x2 − 2x4 = 2.

where a is a real number. By putting this system in row-echelon form,

determine for which values of a it has no solution.

5. For each of the following chemical reactions, solve the corresponding chem-
ical balance equations using Gaussian elimination and back-substitution:

(a) CH4 + O2 CO2 + H2 O

(b) SnO2 + H2 Sn + H2 O

(c) F e + H2 SO4 F e2 (SO4 )3 + H2

(d) C3 H8 + O2 H2 O + CO2

6. Solve the node balance equations (as a system of linear equations in the
unknowns x1 , x2 , x3 ) for the following graph:
 93

A x1
−2
x2

B x3 D E

7
2x2
x3
C

7. Solve the node balance equations (as a system of linear equations in the
unknowns x1 , x2 , x3 , x4 ) for the following graph:

A
3x1

2 3

3x4 2x3
B D E

7
2x2 7x1
x3
C

8. Solve the node balance equations (as a system of linear equations in the
unknowns x1 , x2 , x3 , x4 , x5 , x6 ) for the following graph:
x6
A F

x1 x5
x4 5x4
3x2
x1 2x3
B D E
7
2x6
x5 2x1
x3
C

9. The voltage V at the output of an electric device is related to the input

current I to the device by the polynomial function V (I) = aI 2 + bI + c,
where the real coefficients a, b, c are unknown; we wish to determine a, b, c
94 SECTION 7

from experimental data. We experimentally measure the output voltage at

the input current values I = 0, 1, 2, and we determine from measurement
that V (0) = −5, V (1) = 5, V (2) = 21. Solve the system of linear equations
(in the unknowns a, b, c) corresponding to these measurements.
10. The velocity v of a particle is modelled as a function of time t by the
polynomial function v(t) = at3 + bt2 + ct + d, where the real coefficients
a, b, c, d are unknown; we wish to determine a, b, c, d from experimental
data. We experimentally measure the velocity of the particle at times
t = 0, 1, 3, 5, and we determine from measurement that v(0) = 5, v(1) = 2,
v(3) = 2, v(5) = 50. Solve the system of linear equations (in the unknowns
a, b, c, d) corresponding to these measurements.
11. Consider a square thin metal plate with temperature at steady state and
with known boundary temperature; we represent this thin metal plate by
the following square mesh:
10 20 20 10

x1 x2
15 15

15 x3 x4 15

20 30 30 20
The steady-state temperature at each mesh point is indicated next to that
mesh point. The four mesh points interior to the plate have respective
steady-state temperatures x1 , x2 , x3 , x4 ; solve the system of linear equa-
tions (in the unknowns x1 , x2 , x3 , x4 ) that governs the relations between
these temperatures.
Section 8

Study Topics
• Generating set for a vector space
• Basis of a vector space
• Coordinates of a vector with respect to a basis

95
96 SECTION 8

Let (V, +, ·) be a real vector space, and let v1 , v2 , · · · , vp be a finite number

of vectors in V. We are by now (hopefully) familiar with the notion of linear
combination of the vectors v1 , v2 , · · · , vp : A vector v in V is said to be a
linear combination of v1 , v2 , · · · , vp if it is equal to the vector

α1 v1 + α2 v2 + · · · + αp vp

for some choice of real numbers α1 , α2 , · · · , αp . We have seen in previous lec-

tures that the set of all linear combinations of v1 , v2 , · · · , vp , which we have
denoted by S(v1 ,v2 ,··· ,vp ) , is a vector subspace of V.
The case where S(v1 ,v2 ,··· ,vp ) is V itself is of particular interest, since it means
that any vector v in V is some linear combination of just v1 , v2 , · · · , vp . Due
to the importance of this special case, we make it into a definition:

Definition 20. Let (V, +, ·) be a real vector space, and let S = {v1 , v2 , · · · , vp }
be a finite subset of V. S is said to be a generating set for the vector space V
if any vector v in V can be written as a linear combination of v1 , v2 , · · · , vp ; that
is, for any v in V, we can find real numbers α1 , α2 , · · · , αp such that:

v = α1 v1 + α2 v2 + · · · + αp vp .

Let us examine a few examples:

c3 , consider the three vectors

(a) in the (by now familiar) real vector space R
v1 , v2 , v3 defined as follows:
     
1 0 0
v1 =  0  , v2 =  1  , v3 =  0  ;
0 0 1

It is easy to verify that none of the sets {v1 , v2 }, {v2 , v3 }, and {v1 , v3 }
is a generating set for R c3 , but that the set {v1 , v2 , v3 } is a generating
c 3
set for R . To be sure, let us do some of these verifications. Let us first
prove that {v1 , v2 , v3 } is a generating set for Rc3 : To do this we have to
c3
show that any element of R can be written as a linear combination of
v1 , v2 , v3 . Let then v be any element in R c3 ; by definition of Rc3 , v is a
real column vector, with three entries, i.e. is of the form
 
a
v =  b ,
c

for some real numbers a, b and c; but then, it is clear that we have the
equality

v = av1 + bv2 + cv3 ,

 97

since, writing it out in detail, we have:

     
1 0 0
av1 + bv2 + cv3 = a  0  + b  1  + c  0 
0 0 1
     
a 0 0
=  0 + b + 0 
0 0 c
 
a
=  b ,
c
which is none other than v. This proves our assertion that any element in
c3 can be written as a linear combination of v1 , v2 , v3 , and hence shows
R
c3 .
that {v1 , v2 , v3 } is indeed a generating set for R
Let us now show that {v1 , v2 } is not a generating set for R c3 ; for this it
c 3
is enough to find one vector in R which cannot be written as a linear
combination of v1 and v2 ; consider then the vector v in R c3 defined by:
 
0
v= 0 
1
Let us show there is no linear combination of v1 and v2 that equals v.
How can we show this ? Well, we can assume that there does exist such a
linear combination, and show that we are then led to a contradiction. Let
us then assume that there do exist real numbers α1 , α2 such that
v = α1 v1 + α2 v2 ,
or, equivalently,
     
0 1 0
 0  = α1  0  + α2  1  ,
1 0 0
that is,
       
0 α1 0 α1
 0  =  0  +  α2  =  α2  ;
1 0 0 0
 
0
But we have a contradiction since the two column vectors  0  and
  1
α1
 α2  can never be equal, no matter what α1 and α2 are (since their
0
98 SECTION 8

equality would imply that 1 = 0). This shows that there is no linear
combination of v1 and v2 that equals v; hence, {v1 , v2 } is not a generating
c3 .
set for R
(b) Consider now the (equally familiar) real vector space Rc2 of all real column
vectors with two entries. Consider the subset S = {v1 , v2 , v3 } of R c2 ,
c2 are defined as:
where the vectors v1 , v2 , v3 in R
     
1 0 −1
v1 = , v2 = , v3 = .
0 1 0

c2 . To do this, we
Let us show that S = {v1 , v2 , v3 } is a generating set for R
have to show that any element c2
  in R can be written as a linear combination
a
of v1 , v2 , v3 . Let then (with a, b real numbers) be any element of
b
c2 ; it is clear that we can write:
R
 
a
= av1 + bv2 + 0v3 ,
b
 
a c2 can be written as a linear combination
that is, any element of R
b
c2 .
of v1 , v2 , v3 . This shows that S is indeed a generating set for R
 
a
Let us note now that can also be expressed as the following linear
b
combination of v1 , v2 , v3 :
 
a
= 0v1 + bv2 − av3 ,
b

and even as the following linear combination:

 
a
= 2av1 + bv2 + av3 ,
b

and even as:

 
a
= 3av1 + bv2 + 2av3 ,
b

and so on ... (you get the idea). This shows that any element in Rc2 can
be written as many distinct linear combinations of v1 , v2 , v3 .
The case where a given generating set of a vector space is such that any vector
in that vector space can be expressed as a linear combination of vectors in the
generating set in a unique way is of particular interest, since then there is no
ambiguity as to how the vector should be written as a linear combination of the
 99

elements of the generating set. Note that this was the case with Example (a)
above (prove it!), but not with Example (b).
A very natural question at this point is therefore the following: Suppose we
have a vector space V, and we are given a finite subset S of V which is also
a generating set for V. We then know of course that any element of V can
be written as some linear combination of the elements of S; how then, can we
determine whether that is the only possible linear combination that yields the
desired element ? We already have the answer to that question from Theorem
4 of Lecture 6; we can therefore state:

Theorem 11. Let (V, +, ·) be a real vector space, and let S = {v1 , · · · , vp } be a
finite subset of V such that:

(i) S is a generating set for V, and

(ii) S is a linearly independent of V;

Then, any element of V can be expressed in a unique way as a linear combination

of elements of S.

Finite subsets of V which happen to be both generating and linearly inde-

pendent are of particular importance, as we will see later; we shall therefore
give them a special name. However, for a reason that will become clear soon,
we will care about the order, i.e. we will consider not just subsets of V,
but rather ordered subsets, or equivalently p−tuples of vectors. We therefore
define:

Definition 21. Let (V, +, ·) be a real vector space, and let v1 , v2 , · · · , vp ∈ V.

The p−tuple (v1 , v2 , · · · , vp ) is said to be a basis of V if

(i) {v1 , v2 , · · · , vp } is a generating set for V, and

(ii) the vectors v1 , v2 , · · · , vp are linearly independent.

It is important to note again that in a p−tuple order is important, and that

is what distinguishes it from a set; for example, the 2−tuples of vectors (v1 , v2 )
and (v2 , v1 ) are distinct unless v1 = v2 . Similarly, the 3−tuples (v1 , v2 , v3 ) and
(v3 , v2 , v1 ) are distinct unless v1 = v3 , and so on.

Before going further, let us consider two familiar examples:

1. Consider the real vector space (R2 , +, ·), and consider the elements (0, 1),(1, 1),(2, 2),(1, 0)
of R2 . Consider the following few tuples of vectors made from these 4 vec-
100 SECTION 8

tors:
B1 = ((0, 1)), B2 = ((0, 1), (1, 0)),
B3 = ((1, 0), (0, 1)), B4 = ((0, 1), (1, 1)),
B5 = ((0, 1), (2, 2)), B6 = ((1, 1), (2, 2)),
B7 = ((1, 1), (1, 0), (2, 2)), B8 = ((1, 1), (1, 0)),
B9 = ((1, 1), (0, 1)), B10 = ((1, 1), (1, 0), (2, 2)),
B11 = ((2, 2), (1, 0)), B12 = ((1, 0), (2, 2)),
B13 = ((0, 1), (2, 2)), B14 = ((0, 1), (2, 2), (1, 1)),
B15 = ((2, 2), (0, 1)), B16 = ((2, 2)),
B17 = ((2, 2), (1, 1)), B18 = ((2, 2), (0, 1), (1, 1), (0, 1)),
B19 = ((2, 2), (2, 2), (2, 2), (2, 2)), B20 = ((0, 1), (2, 2), (2, 2), (2, 2));
It is easy to verify that among these 20 tuples, only B2 , B3 , B4 , B5 , B8 ,
B9 , B11 , B12 , B13 , B15 , are a basis of the real vector space R2 . Again it
is important to remember that the two tuples B2 and B3 are not the
same since, although their constituent elements are the same (namely the
two vectors (1, 0) and (0, 1)), their order in B2 and B3 is not the same;
similarly, the two tuples B4 and B9 are not the same, the two tuples B5
and B15 are not the same, ...
c3 , +, ·), and consider the vectors
2. Consider the real vector space (R
     
1 0 0
v1 =  0  , v2 =  1  , v3 =  0  ,
0 0 1
     
1 1 0
v4 =  1  , v5 =  1  , v6 =  1  ,
0 1 1
c3 . Here again, it is easy to verify that each of the 3−tuples (v1 , v2 , v3 ),(v2 , v1 , v3 ),(v3 , v1 , v2 ),(v
of R
(and still many more!) are distinct bases of the real vector space R c3 ; on
the other hand, the tuples (v1 ), (v1 , v2 ), (v3 ), (v1 , v1 , v1 ), (v5 , v6 ) (to
give only a few examples) do not form a basis of R c3 .

The attentive reader may have noticed that in the case of the real vector space
R2 given above, all the tuples which did form a basis of R2 happened to have
exactly the same number of vectors (namely 2); similarly, in the case of the real
vector space Rc3 , all the tuples which did form a basis of R
c3 happened to have
exactly the same number of vectors (namely 3). We will see in the next lecture
that this is not accidental, and that in a vector space, all bases have the same
number of elements.
Now that we have defined the notion of basis for a real vector space, we define
another important notion, that of components of a vector with respect to
a basis:
 101

Definition 22. Let (V, +, ·) be a real vector space, let B = (v1 , v2 , · · · , vp ) be a

basis of V, and let v ∈ V. The p−tuple (α1 , α2 , · · · , αp ) of real numbers is called
the component vector or coordinate vector of v with respect to the basis B
if we have:

α1 v1 + α2 v2 + · · · + αp vp = v.

The real number α1 is called the first component (or first coordinate) of v with
respect to the basis B, the real number α2 is called the second component (or
second coordinate) of v with respect to the basis B, ... and so on.
It is absolutely essential to note the following point:
• If B = (v1 , v2 , · · · , vp ) is a basis of V and v is an element of V, then v has
at least one component vector with respect to B (since {v1 , v2 , · · · , vp }
is a generating set for V); furthermore, since the vectors v1 , v2 , · · · , vp
are linearly independent, there is only a unique p−tuple (α1 , · · · , αp )
of real numbers for which α1 v1 + α2 v2 + · · · + αp vp is equal to v; i.e. to
each v in V there corresponds only a unique component vector. This is
why in the previous definition, we wrote “the component vector of v with
respect to B” and not “a component vector of v with respect to B”.
It is also important to make note of the following point:
• If B = (v1 , v2 , · · · , vp ) is a basis of V and v is an element of V, we have
defined the component vector of v with respect to B to be some p−tuple
of real numbers, i.e. some element of Rp ; we will find it occasionally
convenient to denote the component vector of v by an element of R cp
instead, i.e. by a column vector with p real entries.
We close this section with three examples:

(a) Consider the real vector space (R2 +, ·), and let W denote the subset of
R2 consisting of all pairs (x, y) of real numbers such that x = y. It is easy
to verify that W is a vector subspace of R2 , and hence, by Problem 4 of
Lecture 3, W (with the operations of addition and scalar multiplication
it receives from R2 ) is itself a real vector space. Consider the vector
v1 = (1, 1) of R2 ; clearly, v1 is an element of W, since W is the subset
of R2 consisting of all pairs (x, y) with x = y, and the pair (1, 1) satisfies
this condition. Consider the “1−tuple” B = (v1 ); let us show that B is a
basis of W. To do this, we have to show two things:
(i) That {v1 } is a generating set for W, and
(ii) that {v1 } is a linearly independent subset of W.
To show (i), namely that {v1 } is a generating set for W, we have to show
that any v ∈ W can be written as some linear combination of v1 ; let
then v = (x, y) be an element of W; since (x, y) is in W, we have (by
definition of W) x = y. Hence, v = (x, x) = x(1, 1) = xv1 , which shows
102 SECTION 8

that v is a linear combination of v1 . To recapitulate, we have shown that

any element of W is a linear combination of v1 ; this proves that {v1 } is
a generating set for W. Let us now show (ii), i.e. that {v1 } is a linearly
independent set. Let then α be any real number such that αv1 = 0; we
have to show that we must then have α = 0. Now, αv1 = 0 is equivalent
to α(1, 1) = (0, 0), which is equivalent to (α, α) = (0, 0), which implies
α = 0. Hence, we have shown that αv1 = 0 implies α = 0, and this proves
that {v1 } is a linearly independent subset of W. We have thus shown (i)
and (ii), and hence, we have shown that B is a basis of W.
Any element of W is a pair of the form (a, a) for some a ∈ R; Let us
compute the component vector of (a, a) with respect to the basis B.
We have:

(a, a) = a(1, 1) = av1 ,

which shows that the component vector of the vector (a, a) of W with
respect to the basis B is the 1−tuple (a). (Note: It may sound a bit
akward to talk about 1−tuples, but think of them as a list with only one
entry!).
(b) Consider the real vector space (R3 , +, ·), and let v1 = (1, 0, 0), v2 =
(0, 1, 0), v3 = (0, 0, 1), v4 = (1, 1, 0), v5 = (1, 1, 1). It is easy to verify
that each of the 3−tuples of vectors B1 = (v1 , v2 , v3 ), B2 = (v2 , v3 , v1 ),
and B3 = (v1 , v4 , v5 ) is a basis of R3 . Let now v = (a, b, c) be an element
of R3 . It is easy to verify that the component vector of v with respect to
B1 is (a, b, c) (i.e. v itself!), whereas its component vector with respect to
the basis B2 is (b, c, a), and its component vector with respect to the basis
B3 is (a − b, b − c, c).
(c) Consider again the real vector space (R3 , +, ·), and let W be the subset
of R3 consisting of all triples (x, y, z) with x + y + z = 0. It is easy to
verify that W is a vector subspace of R3 . By Problem 4 of Lecture 3, we
know that W (with the operations of addition and scalar multiplication
it receives from R3 ) is itself a real vector space. Consider the elements
v1 , v2 of W defined by:

v1 = (1, 0, −1),
v2 = (0, 1, −1).

Define B to be the 2−tuple B = (v1 , v2 ). Let us show that B is indeed

a basis for the real vector space W. Once again, to show this, we must
show two things:
(i) That {v1 , v2 } is a generating set for W, and
(ii) that {v1 , v2 } is a linearly independent subset of W.
Let us first show (i): We have to show that any element of W is some
linear combination of v1 and v2 . Let then v = (x, y, z) be any element in
 103

W. Since v is in W, we must have (by definition of W) x + y + z = 0,

i.e. z = −x − y. Hence, v = (x, y, −x − y). Now note that we can write:
v = (x, y, −x − y)
= (x, 0, −x) + (0, y, −y)
= x(1, 0, −1) + y(0, 1, −1)
= xv1 + yv2 ,
which shows that v is a linear combination of v1 and v2 . We have thus
shown that any element of W is a linear combination of v1 and v2 , and
this shows that {v1 , v2 } is a generating set for W.
Let us now show (ii). Let then α, β be real numbers such that αv1 +βv2 =
0; we must show that this necessarily implies that α = β = 0. Now,
αv1 + βv2 = 0 is equivalent to
α(1, 0, −1) + β(0, 1, −1) = (0, 0, 0),
which is equivalent to
(α, 0, −α) + (0, β, −β) = (0, 0, 0),
which is equivalent to
(α, β, −α − β) = (0, 0, 0),
which implies α = β = 0. Hence, we have shown that if α, β are real
numbers such that αv1 + βv2 = 0 then it must follow that α = β = 0.
This proves (ii), i.e. that {v1 , v2 } is a linearly independent subset of
W. We have therefore shown (i) and (ii), i.e., we have shown that B is a
basis of W.
Let now (a, b, c) be any element of W; let us compute the component
vector of (a, b, c) with respect to the basis B. Since (a, b, c) is in W, we
must have (by definition of W) a + b + c = 0; it follows that c = −a − b,
i.e.
(a, b, c) = (a, b, −a − b) = a(1, 0, −1) + b(0, 1, −1) = av1 + bv2 ,
which shows that the component vector of the element (a, b, c) of W with
respect to the basis B is the pair (a, b).

PROBLEMS:

1. Consider the real vector space (R3 , +, ·), and consider the following vectors
in R3 :
v1 = (1, 0, 0), v2 = (1, 1, 0), v3 = (1, 2, 1), v4 = (0, 0, 3),
104 SECTION 8

(a) Show that {v1 } is not a generating set for R3 .

(b) Show that {v2 } is not a generating set for R3 .
(c) Show that {v3 } is not a generating set for R3 .
(d) Show that {v4 } is not a generating set for R3 .
(e) Show that {v1 , v2 } is not a generating set for R3 .
(f) Show that {v1 , v3 } is not a generating set for R3 .
(g) Show that {v2 , v3 } is not a generating set for R3 .
(h) Show that {v1 , v4 } is not a generating set for R3 .
(i) Show that {v1 , v2 , v3 } is a generating set for R3 .
(j) Show that {v1 , v2 , v4 } is a generating set for R3 .
(k) Show that {v1 , v3 , v4 } is a generating set for R3 .
(l) Show that {v2 , v3 , v4 } is a generating set for R3 .
(m) Show that {v1 , v2 , v3 , v4 } is a generating set for R3 .

2. Consider the real vector space c3 , +, ·), and let W be the subset of R
(R c3
 
x
consisting of all elements  y  of Rc3 for which x + y − z = 0. It is easy
z
to verify that W is a vector subspace of R c3 , and hence, is itself a real
vector space.
Consider now the following vectors in W:
       
1 0 1 0
v1 =  0  , v2 =  2  , v3 =  −1  , v4 =  0  .
1 2 0 0

(a) Show that {v1 } is not a generating set for W.

(b) Show that {v2 } is not a generating set for W.
(c) Show that {v3 } is not a generating set for W.
(d) Show that {v4 } is not a generating set for W.
(e) Show that {v1 , v4 } is not a generating set for W.
(f) Show that {v2 , v4 } is not a generating set for W.
(g) Show that {v3 , v4 } is not a generating set for W.
(h) Show that {v1 , v2 } is a generating set for W.
(i) Show that {v2 , v3 } is a generating set for W.
(j) Show that {v1 , v3 } is a generating set for W.
(k) Show that {v1 , v2 , v3 } is a generating set for W.
(l) Show that {v1 , v2 , v3 , v4 } is a generating set for W.
 105

3. Consider the real vector space (R4 , +, ·), and let W be the subset of R4
consisting of all 4−tuples (x, y, z, w) of real numbers for which 3x+y−w =
0 and z − 2w = 0. It is easy to verify that W is a vector subspace of W,
and hence, is itself a real vector space. Consider the following vectors in
W:

v1 = (1, 0, 6, 3), v2 = (0, 1, 2, 1), v3 = (1, 1, 8, 4)

(a) Show that {v1 } is not a generating set for W.

(b) Show that {v2 } is not a generating set for W.
(c) Show that {v3 } is not a generating set for W.
(d) Show that {v1 , v2 } is a generating set for W.
(e) Show that {v1 , v3 } is a generating set for W.
(f) Show that {v2 , v3 } is a generating set for W.
(g) Show that {v1 , v2 , v3 } is a generating set for W.
4. Consider the real vector space (R2 , +, ·), and consider the following vectors
in R2 :

v1 = (1, 1), v2 = (−1, 1), v3 = (0, 1), v4 = (1, 2)

(a) Show that (v1 ) is not a basis of R2 .

(b) Show that (v2 ) is not a basis of R2 .
(c) Show that (v3 ) is not a basis of R2 .
(d) Show that (v4 ) is not a basis of R2 .
(e) Show that B1 = (v1 , v2 ) is a basis of R2 .
(f) Show that B2 = (v1 , v3 ) is a basis of R2 .
(g) Show that B3 = (v1 , v4 ) is a basis of R2 .
(h) Show that B4 = (v2 , v3 ) is a basis of R2 .
(i) Show that B5 = (v3 , v2 ) is a basis of R2 .
(j) Show that B6 = (v3 , v4 ) is a basis of R2 .
(k) Show that (v1 , v2 , v3 ) is not a basis of R2 .
(l) Show that (v1 , v2 , v4 ) is not a basis of R2 .
(m) Show that (v1 , v2 , v3 , v4 ) is not a basis of R2 .
(n) Compute the component vectors of v1 with respect to each of the
bases B1 , · · · , B6 .
(o) Compute the component vectors of v2 with respect to each of the
bases B1 , · · · , B6 .
(p) Compute the component vectors of v3 with respect to each of the
bases B1 , · · · , B6 .
106 SECTION 8

(q) Compute the component vectors of v4 with respect to each of the

bases B1 , · · · , B6 .
5. Consider the real vector space (R4 , +, ·), and let W be the subset of R4
consisting of all 4−tuples (x, y, z, w) of real numbers for which x + y + z −
w = 0 and y − z = 0. It is easy to verify that W is a vector subspace of
R4 , and hence, is itself a real vector space. Consider the following vectors
in W:

v1 = (1, 0, 0, 1), v2 = (0, 1, 1, 2), v3 = (1, 1, 1, 3), v4 = (1, −1, −1, −1).

(a) Show that (v1 ) is not a basis of W.

(b) Show that (v2 ) is not a basis of W.
(c) Show that (v3 ) is not a basis of W.
(d) Show that (v4 ) is not a basis of W.
(e) Show that B1 = (v1 , v2 ) is a basis of W.
(f) Show that B2 = (v1 , v3 ) is a basis of W.
(g) Show that B3 = (v3 , v1 ) is a basis of W.
(h) Show that B4 = (v1 , v4 ) is a basis of W.
(k) Show that (v1 , v2 , v3 ) is not a basis of W.
(l) Show that (v1 , v2 , v4 ) is not a basis of W.
(m) Show that (v1 , v3 , v4 ) is not a basis of W.
(n) Compute the component vectors of v1 with respect to each of the
bases B1 , · · · , B4 .
(o) Compute the component vectors of v2 with respect to each of the
bases B1 , · · · , B4 .
(p) Compute the component vectors of v3 with respect to each of the
bases B1 , · · · , B4 .
(q) Compute the component vectors of v4 with respect to each of the
bases B1 , · · · , B4 .
Section 9

Study Topics
• Finite-dimensional vector spaces
• Dimension of a real vector space

107
108 SECTION 9

We know from elementary geometry that we can represent real numbers (i.e.
elements of the real vector space R) by points on a line, pairs of real numbers
(i.e. elements of the real vector space R2 ) by points in a plane, triples of real
numbers (i.e. elements of the real vector space R3 ) by points in space, ... The
question we may ask at this point is how the difference between, say, a line and
a plane, manifests itself in a difference between the vector spaces R and R2 .
Or how the difference between a plane and space manifests itself in a difference
between the vector spaces R2 and R3 . In other words, how is the “line-like”
feature of the real vector space R captured in the vector space structure of R,
how is the “plane-like” feature of the real vector space R2 captured in the vector
space structure of R2 , how is the “space-like” feature of the real vector space
R3 captured in the vector space structure of R3 , ... In one word: How do these
“geometric” concepts get expressed in terms of linear algebra ? We shall soon
see that one point of contact between linear algebra and geometry is given by
the notion of dimension.
Definition 23. Let (V, +, ·) be a real vector space.
• V is said to be finite-dimensional if there exists an integer N ≥ 0 such
that any subset of V containing N + 1 elements is linearly dependent.
The smallest integer N for which this holds is then called the dimension
of V (equivalently, V is said to have dimension N ).
• V is said to be infinite-dimensional if it is not finite-dimensional.
It is important to note the following points:
• By the above definition, if V has dimension N , then any subset of V
containing N + 1 elements is linearly dependent, and as a result, any
subset of V containing N + 2 or more elements is also linearly dependent.
• By the above definition, if V has dimension N , then any subset of V
containing N + 1 elements is linearly dependent, but there exists at least
one linearly independent subset of V containing exactly N elements,
since otherwise the dimension of V would be strictly less than N .
• V is infinite-dimensional if for any positive integer N , no matter how
large, there exists a linearly independent subset of V containing ex-
actly N elements.
Let’s recapitulate:
• V is said to have dimension N if and only if the following two conditions
are met:
(i) There exists a linearly independent subset of V containing ex-
actly N elements,
(ii) Any subset of V containing N +1 elements is linearly dependent.
Let us now examine some simple examples:
 109

1. Recall the real vector space (F (R; R), +, ·) consisting of all functions f :
R → R (i.e. all real-valued functions of a real variable). Let us show that
F (R; R) is an infinite-dimensional vector space. To do this, we have
to show that for any integer N ≥ 0 (no matter how large), there exists a
linearly independent subset of F (R; R) containing exactly N elements.
Let then N be any integer ≥ 0. Consider the following elements f1 , f2 , · · · , fN
in F (R; R) defined as follows:

f1 (t) = 0 for t 6= 1 and f1 (1) = 1,

f2 (t) = 0 for t 6= 2 and f2 (2) = 1,
f3 (t) = 0 for t 6= 3 and f3 (3) = 1,
···
fN (t) = 0 for t 6= N and fN (N ) = 1.

(You may want to draw the graphs of these functions just to see what
they look like). In other words, if k is an integer in the range 1 ≤ k ≤ N ,
fk is the function from R to R defined to be 0 everywhere except at
k (where it takes the value 1). Let us show that {f1 , f2 , · · · , fN } is a
linearly independent subset of F (R; R). Let then α1 , α2 , · · · , αN be
real numbers such that

α1 f1 + α2 f2 + · · · + αN fn = 0.

(Recall that the zero vector 0 of the vector space F (R; R) is the function
from R to R which maps every real number to zero). We have to show
that α1 ,α2 ,· · · ,αN must all be zero. Now, the equality

α1 f1 + α2 f2 + · · · + αN fn = 0.

is equivalent to

α1 f1 (t) + α2 f2 (t) + · · · + αN fn (t) = 0, ∀t ∈ R,

and in particular, this last equality must hold for particular choices of t,
such as t = 1, t = 2, · · · , t = N . But since f1 (1) = 1 and f2 (1) = f3 (1) =
· · · = fN (1) = 0, the above equality, for the particular choice of t = 1
becomes:

α1 = 0.

Similarly, since f2 (2) = 1 and f1 (2) = f3 (2) = · · · = fN (2) = 0, that same

equality, for the particular choice of t = 2 becomes:

α2 = 0.
110 SECTION 9

Continuing in this way with, successively, t = 3, t = 4, · · · , t = N , we

obtain successively that α3 = 0, α4 = 0, · · · , αN = 0. Hence, we have
shown that if α1 , α2 , · · · , αN are real numbers such that

α1 f1 + α2 f2 + · · · + αN fn = 0,

then it follows that α1 = α2 = · · · = αN = 0. This proves that {f1 , f2 , · · · , fN }

is a linearly independent subset of F (R; R).
Let us recapitulate: For any integer N ≥ 0, we have constructed a
linearly independent subset of F (R; R) containing exactly N ele-
ments; this proves that F (R; R) is an infinite-dimensional real vector
space.
2. Let V be a set containing only one element, which we denote by ξ (the
zero vector), that is, V = {ξ}; define addition and scalar multiplication
on V as follows:

ξ+ξ = ξ
α·ξ = ξ, ∀α ∈ R.

It is easy to verify that (V, +, ·) is a real vector space and that ξ itself is
the zero vector of this vector space. Let us show that V has dimension
0. Let N = 0; there is only one possible subset of V containing N +
1 = 1 elements, namely {ξ}, which is V itself, and since that subset
contains the zero vector (since ξ itself is the zero vector), that subset is
linearly dependent. It follows from the definition of dimension that V is
0−dimensional.
3. Let now V = R, with the usual addition and multiplication operations (we
have already seen that with these operations, R is a real vector space). Let
us show that V has dimension 1. For this, we have to show that we can
find a linearly independent subset of R containing exactly one element,
but that any subset of R containing two elements is linearly dependent.
Consider then the subset of R given by {1} (i.e. the subset containing
only the vector 1 of R). Let α ∈ R, and assume α · 1 = 0; since α · 1 = α,
this implies α = 0, which shows that {1} is a linearly independent subset
of R. Let us now show that any subset of R containing two elements is
linearly dependent. Let then {v1 , v2 } be any subset of R; we have to show
that there exists a linear combination αv1 + βv2 of v1 , v2 that is equal to
0 but such that α, β are not both zero. Consider the following three cases:
(i) If v1 = 0, then with α = 1, β = 0, we obtain αv1 +βv2 = 0, and since
α, β are not both zero (since α = 1), this proves linear dependence
of {v1 , v2 } in this case.
(ii) If v2 = 0, then with α = 0, β = 1, we obtain αv1 +βv2 = 0, and since
α, β are not both zero (since β = 1), this proves linear dependence
of {v1 , v2 }.
 111

(iii) If v1 6= 0 and v2 6= 0, then, with α = 1, β = − vv12 (remember that

since V = R in this example, our vectors are real numbers so we
can divide by them if they are non-zero), we have αv1 + βv2 = 0,
and since α, β are not both zero, this proves linear dependence of
{v1 , v2 }.

Hence, we have linear dependence of {v1 , v2 } in all cases. To recapitulate,

we have shown that:

• There exists a linearly independent subset of R containing exactly

one element,
• Any subset of R containing two elements is linearly dependent.

This proves that R has dimension 1.

The reader may feel at this point that the dimension of a real vector space may
not be such an easy thing to compute: After all, it took us quite a few lines
above to compute the dimension of a simple vector space such as R. Recall
again that to prove that a real vector space has dimension N , we have to do
two things:

• Find N linearly independent vectors in that vector space,

• Show that any subset containing N + 1 vectors is linearly dependent.

The first item is usually easy to accomplish; it is enough to find one linearly
independent subset containing exactly N elements. The difficulty comes from
the second item: How does one prove that any subset containing N +1 elements
is necessarily linearly dependent ?
The following theorems will yield an extremely simple way to compute the
dimension of a finite-dimensional real vector space.

Theorem 12. Let (V, +, ·) be a finite-dimensional real vector space of di-

mension N . Let {v1 , · · · , vp } be a finite subset of V containing p vectors. If
{v1 , · · · , vp } is a linearly independent subset of V then p ≤ N .

Proof. The proof follows directly from the definition of dimension. If we had p >
N , then that would mean that p ≥ N + 1; since V is assumed to have dimension
N , we know (by definition of dimension) that any subset of V containing N + 1
or more elements has to be linearly dependent; hence, if p > N , i.e. equivalently
p ≥ N + 1, the subset {v1 , · · · , vp } has N + 1 or more elements, and hence
is linearly dependent. Hence, if we assume that {v1 , · · · , vp } is linearly
independent, then we must have p ≤ N .

Theorem 13. Let (V, +, ·) be a finite-dimensional real vector space of di-

mension N . Let {v1 , · · · , vp } be a finite subset of V containing p vectors. If
{v1 , · · · , vp } is a generating set for V of V then p ≥ N .
112 SECTION 9

Proof. We will show that, under the assumption that {v1 , · · · , vp } is a gen-
erating set for V, we cannot have p < N . So let us begin by assuming that
{v1 , · · · , vp } is a generating set for V, and that p < N ; we will show that this
leads to a contradiction.
Since V is assumed to have dimension N , there does exist (by definition of
dimension) a linearly independent subset of V containing exactly N ele-
ments; let {e1 , · · · , eN } denote that subset. Since {v1 , · · · , vp } is a generating
set for V, each of e1 , e2 , · · · , eN can be expressed as a linear combination of
v1 , · · · , vp . Let us start with e1 ; we can write:

e1 = α1 v1 + α2 v2 + · · · + αp vp ,

for some real numbers α1 , α2 , · · · , αp . Since the subset {e1 , · · · , eN } is linearly

independent, none of e1 , · · · , eN is the zero vector; in particular, e1 6= 0,
and hence, at least one of α1 , α2 , · · · , αp must be non-zero (since if all of
α1 , α2 , · · · , αp were zero, we would obtain e1 = 0 from the linear combination
above). Assume with no loss of generality (relabel the vectors if necessary) that
it is α1 which is non-zero, i.e. α1 6= 0. We can then write:
1 α2 αp
v1 = e1 − v2 − · · · − vp ,
α1 α1 α1
and this shows that {e1 , v2 , v3 , · · · , vp } is a generating set for V (since
{v1 , v2 , · · · , vp } itself is assumed to be a generating set for V). Hence e2 can
be written as some linear combination of e1 , v2 , v3 , · · · , vp , i.e.

e2 = λ1 e1 + λ2 v2 + λ3 v3 + · · · + λp vp ,

for some real numbers λ1 , · · · , λp . Note that at least one of λ2 , λ3 , · · · , λp is

non-zero, since if all of λ2 , λ3 , · · · , λp were zero, we would obtain e2 = λ1 e1 ,
contradicting the linear independence of {e1 , · · · , eN }. Assume with no loss of
generality (relabel the vectors if necessary) that it is λ2 which is non-zero, i.e.
λ2 6= 0. This then yields:
λ1 1 λ3 λp
v2 = − e1 + e2 − v3 − · · · − vp ,
λ2 λ2 λ2 λ2
This last equality, together with the equality
1 α2 αp
v1 = e1 − v2 − · · · − vp ,
α1 α1 α1
and the assumption that {v1 , v2 , · · · , vp } is a generating set for V, implies
that {e1 , e2 , v3 , v4 · · · , vp } is a generating set for V. Continuining in this
way, we obtain successively that {e1 , e2 , e3 , v4 · · · , vp } is a generating set for
V, that {e1 , e2 , e3 , e4 , v5 · · · , vp } is a generating set for V, ..., and finally, that
{e1 , e2 , e3 , · · · , ep } is a generating set for V, in other words, that the linear
span of {e1 , e2 , e3 , · · · , ep } is the whole vector space V. Note that we have
assumed p < N ; hence, the number of elements of {e1 , e2 , e3 , · · · , ep } is strictly
 113

smaller than the number of elements of {e1 , e2 , e3 , · · · , ep , ep+1 , · · · , eN }. In

particular, eN is in the linear span of {e1 , e2 , e3 , · · · , ep }, and this contradicts
the assumed linear independence of {e1 , e2 , e3 , · · · , ep , ep+1 , · · · , eN }. Hence,
we cannot have p < N ; in other words, we must have p ≥ N .

We can now combine the previous two theorems in the following theorem:

Theorem 14. Let (V, +, ·) be a finite-dimensional real vector space of dimension

N . Let B = (v1 , v2 , · · · , vp ) be a basis for V. Then, we must have p = N .

Proof. Since (v1 , v2 , · · · , vp ) is assumed to be a basis for V, {v1 , v2 , · · · , vp }

is both a generating subset for V and a linearly independent subset of
V; the first property implies p ≥ N , and the second property implies p ≤ N .
Together, they imply p = N .

We can establish the following two very important corollaries of the previous
theorem:

(i) In a finite-dimensional real vector space, all bases have the same num-
ber of elements (and that number is the dimension of the vector space);

(ii) To compute the dimension of a finite-dimensional real vector space,

it is enough to find a basis for it; the dimension of the vector space is
then equal to the number of elements of the basis.

Let us illustrate the application of these results on some examples:

1. Consider the (by now extremely familiar) real vector space (R2 , +, ·). Let
us compute the dimension of R2 . As we have seen above, all we need to
do is find a basis for R2 ; the dimension of R2 will then be equal to the
number of elements of that basis. Consider then the following vectors in
R2 :

v1 = (1, 0), v2 = (0, 1).

We have already seen that (v1 , v2 ) is a basis of R2 ; since it has 2 elements,

we conclude that R2 is a real vector space of dimension 2.

2. Consider the (equally familiar) real vector space (R3 , +, ·). Let us compute
the dimension of R3 . As we have seen above, all we need to do is find a
basis for R3 ; the dimension of R3 will then be equal to the number of
elements of that basis. Consider the following vectors in R3 :

v1 = (1, 0, 0), v2 = (0, 1, 0), v3 = (0, 0, 1).

We have already seen that (v1 , v2 , v3 ) is a basis of R3 ; since it has 3

elements, we conclude that R3 is a real vector space of dimension 3.
114 SECTION 9

3. More generally, let n be an integer ≥ 1 and consider the real vector space
(Rn , +, ·). Let us compute the dimension of Rn . Proceeding as before, let
us construct a basis for Rn ; the dimension of Rn will then be given by the
number of elements in that basis. For this, consider the following elements
of Rn :

v1 = (1, 0, 0, 0, · · · , 0),
v2 = (0, 1, 0, 0, · · · , 0),
v3 = (0, 0, 1, 0, · · · , 0),
···
vn = (0, 0, 0, 0, · · · , 1).

(i.e. vk is the n−tuple with the k th entry equal to 1 and all other entries
equal to 0). It is easy to verify that (v1 , v2 , v3 , · · · , vn ) is a basis of Rn .
Since it has exactly n elements, this shows that Rn has dimension n.

4. Consider the vector subspace W of R2 defined as the set of all (x, y) in

R2 with x = y. Consider the vector v1 = (1, 1) in W. We have already
seen (in the previous lecture) that (v1 ) is a basis of W. Since it has one
element, we deduce that W is a real vector space of dimension 1.

5. Consider now the vector subspace W of R3 defined as the set of all (x, y, z)
in R3 with x + y + z = 0. Consider the vectors v1 = (1, 0, −1) and
v2 = (0, 1, −1) of W. We have already seen (in the previous lecture) that
(v1 , v2 ) is a basis of W. Since it has 2 elements, we conclude that W is
a real vector space of dimension 2.

Let us restate once more the following important point:

• To compute the dimension of a real vector space, it is enough to find

a basis for that vector space. The dimension of the vector space is
then equal to the number of elements of that basis.

PROBLEMS:

1. Let C(R; R) be the real vector space of all continuous functions from R to
R (with addition and scalar multiplication defined as in F (R; R)). Show
that C(R; R) is an infinite-dimensional vector space.

2. Consider the vector subspace W of R2 defined as the set of all (x, y) ∈ R2

such that x + y = 0. Show that W has dimension 1.

3. Consider the vector subspace W of R2 defined as the set of all (x, y) ∈ R2

such that 2x + 3y = 0. Show that W has dimension 1.
 115

4. Consider the vector subspace W of R2 defined as the set of all (x, y) ∈ R2

such that 5x − 7y = 0. Show that W has dimension 1.

5. Consider the vector subspace W of R2 defined as the set of all (x, y) ∈ R2

such that x = 0. Show that W has dimension 1.

6. Consider the vector subspace W of R2 defined as the set of all (x, y) ∈ R2

such that y = 0. Show that W has dimension 1.

7. Consider the vector subspace W of R3 defined as the set of all (x, y, z) ∈ R3

such that x − y = 0. Show that W has dimension 2.

8. Consider the vector subspace W of R3 defined as the set of all (x, y, z) ∈ R3

such that 2x + y + z = 0. Show that W has dimension 2.

9. Consider the vector subspace W of R3 defined as the set of all (x, y, z) ∈ R3

such that x = 0. Show that W has dimension 2.

10. Consider the vector subspace W of R3 defined as the set of all (x, y, z) ∈ R3
such that y = 0. Show that W has dimension 2.

11. Consider the vector subspace W of R3 defined as the set of all (x, y, z) ∈ R3
such that z = 0. Show that W has dimension 2.

12. Consider the vector subspace W of R3 defined as the set of all (x, y, z) ∈ R3
such that x + y = 0 and z = 0. Show that W has dimension 1.

13. Consider the vector subspace W of R3 defined as the set of all (x, y, z) ∈ R3
such that x − z = 0 and x + y + z = 0. Show that W has dimension 1.

14. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x + y + z + w = 0. Show that W has dimension 3.

15. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x + w = 0. Show that W has dimension 3.

16. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x − y = 0. Show that W has dimension 3.

17. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x = 0. Show that W has dimension 3.

18. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that w = 0. Show that W has dimension 3.

19. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x + y = 0 and z + w = 0. Show that W has dimension 2.

20. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x+y +z +w = 0 and z −w = 0. Show that W has dimension
2.
116 SECTION 9

21. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x + 2w = 0 and 2z + w = 0. Show that W has dimension 2.
22. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x − y = 0 and z + w = 0 and y + w = 0. Show that W has
dimension 1.
23. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x + 2y + z = 0 and z = 0 and 2y + w = 0. Show that W
has dimension 1.
24. Consider the vector subspace W of R4 defined as the set of all (x, y, z, w) ∈
R4 such that x + 2y + z + w = 0 and x + y = 0 and y − z = 0. Show that
W has dimension 1.
Section 10

Study Topics
• Linear Transformations
• Range and Kernel of a Linear Transformation

117
118 SECTION 10

In the previous lectures, we have studied vector spaces by themselves; for ex-
ample, we have started from a real vector space, and we have found subspaces
of that vector space, generating sets for that vector space, linearly dependent or
independent subsets for that vector space, bases for that vector space, ... and so
on. We now consider functions between vector spaces, i.e. functions from
one vector space to another. Among these functions, there are some which have
a desirable property (called linearity) which we will precisely define shortly;
these functions are called linear, and are the main object of study of this and
the next few lectures.
A WORD ON TERMINOLOGY: We will often use the words mapping
and transformation instead of function – They will all mean the same thing.
Definition 24. Let V and W be two real vector spaces, and let L : V → W
be a mapping from V to W. L is said to be a linear mapping (also called a
linear transformation or a linear function) if the following two properties are
verified:
1. For any v1 , v2 ∈ V, L(v1 + v2 ) = L(v1 ) + L(v2 );
2. For any α ∈ R and any v ∈ V, L(αv) = αL(v).
Before going further, let us give some examples of linear mappings:
1. Let V be a real vector space, and let L : V → V be the identity mapping
of V, defined by L(v) = v for all v ∈ V; it is easy to verify that L is a
linear mapping.
2. Let V be a real vector space, let α ∈ R be any real number, and let
L : V → V be the mapping defined by L(v) = αv for all v ∈ V; it is easy
to verify that L is a linear mapping.
3. Let V and W be real vector spaces, and let 0W denote the zero vector of
W. Let L : V → W denote the zero mapping from V to W, defined
by L(v) = 0W for all v ∈ V (i.e. everything in V is mapped to the zero
vector of W); here again, it is easy to verify that L is a linear mapping.
4. Consider the familiar vector spaces R2 and R, and let L : R2 → R be the
mapping defined by L((x, y)) = 2x + 3y for all (x, y) ∈ R2 . Let us show
that L is a linear mapping:
(i) Let v1 = (x1 , y1 ) ∈ R2 and v2 = (x2 , y2 ) ∈ R2 . We have to show
that L(v1 + v2 ) = L(v1 ) + L(v2 ). We have:
L(v1 + v2 ) = L((x1 , y1 ) + (x2 , y2 ))
= L((x1 + x2 , y1 + y2 ))
= 2(x1 + x2 ) + 3(y1 + y2 )
= (2x1 + 3y1 ) + (2x2 + 3y2 )
= L((x1 , y1 )) + L((x2 , y2 ))
= L(v1 ) + L(v2 ),
 119

which shows that the first property of a linear mapping is verified.

(ii) Let v = (x, y) ∈ R2 and α ∈ R. We have to show that L(αv) =
αL(v). We have:

L(αv) = L(α(x, y))

= L((αx, αy))
= 2(αx) + 3(αy)
= α(2x + 3y)
= αL((x, y))
= αL(v),

which shows that the second property of a linear mapping is verified.

Both defining properties of a linear mapping are verified; we have therefore

shown that the mapping L of this example is a linear mapping.

5. Consider the familiar real vector spaces R2 and R3 , and let L : R2 → R3 be

the mapping defined by L((x, y)) = (x−y, x+y, 2x+5y) for all (x, y) ∈ R2 .
It is easy to verify that L is a linear mapping.

6. Consider again the familiar real vector spaces R2 and R3 , and let L : R2 →
R3 be the mapping defined by L((x, y)) = (x, y, 0) for all (x, y) ∈ R2 . It
is easy to verify that L is a linear mapping.

7. Consider once more the familiar real vector spaces R2 and R3 , and let
L : R3 → R2 be the mapping defined by L((x, y, z)) = (x, y) for all
(x, y, z) ∈ R3 . It is easy to verify that L is a linear mapping.

8. Consider now the familiar real vector spaces R3 and R, and let L : R3 → R
be the mapping defined by L((x, y, z)) = x + y + z for all (x, y, z) ∈ R3 .
It is easy to verify that L is a linear mapping.

9. Consider now the real vector spaces F (R; R) and R (recall F (R; R) is the
real vector space of all functions from R to R), and let L : F (R; R) → R
be the mapping defined by L(f ) = f (0) for all f ∈ F (R; R). Let us prove
that L is a linear mapping:

(i) We have to show that ∀f1 , f2 ∈ F (R; R), we have L(f1 + f2 ) =

L(f1 ) + L(f2 ). We have:

L(f1 + f2 ) = (f1 + f2 )(0)

= f1 (0) + f2 (0)
= L(f1 ) + L(f2 ),

which shows that the first property of a linear mapping is verified.

120 SECTION 10

(ii) Let now f ∈ F (R; R) and α ∈ R; we have to show that L(αf ) =

αL(f ). We have:
L(αf ) = (αf )(0)
= αf (0)
= αL(f ),
which shows that the second property of a linear mapping is verified.
Both defining properties of a linear mapping are verified; we have therefore
shown that the mapping L of this example is a linear mapping.
10. Consider again the real vector spaces F (R; R) and R, and let L : F (R; R) →
R be the mapping defined by L(f ) = 5f (1) + 7f (2) for all f ∈ F (R; R).
It is easy to verify that L is indeed a linear mapping.
11. Consider now the real vector spaces C([0, 1]; R) and R, where C([0, 1]; R)
denotes the set of all continuous functions from [0, 1] to R (with addition
and scalar multiplication defined as for F (R; R); C([0, 1]; R) is actually a
vector subspace of F (R; R) (prove it as an exercise!)). Define the mapping
R1
L : C([0, 1]; R) → R by L(f ) = 0 f (t)dt, for all f ∈ C([0, 1]; R). Let us
prove that L is a linear mapping:
(i) We have to show that ∀f1 , f2 ∈ C([0, 1]; R), we have L(f1 + f2 ) =
L(f1 ) + L(f2 ). We have:
Z 1
L(f1 + f2 ) = (f1 + f2 )(t)dt
0
Z 1
= (f1 (t) + f2 (t))dt
0
Z 1 Z 1
= f1 (t)dt + f2 (t)dt
0 0
= L(f1 ) + L(f2 ),
which shows that the first property of a linear mapping is verified.
(ii) Let now f ∈ C([0, 1]; R) and α ∈ R; we have to show that L(αf ) =
αL(f ). We have:
Z 1
L(αf ) = (αf )(t)dt
0
Z 1
= (αf (t))dt
0
Z 1
= α f (t)dt
0
= αL(f ),
which shows that the second property of a linear mapping is verified.
 121

Both defining properties of a linear mapping are verified; we have therefore

shown that the mapping L of this example is a linear mapping.

Before going further and giving examples of mappings between vector spaces
which are not linear, we prove the following useful and simple theorem:

Theorem 15. Let V and W be real vector spaces; let 0V denote the zero vector
of V, and let 0W denote the zero vector of W. let L : V → W be a linear
mapping. Then, we have:

L(0V ) = 0W .

What the above theorem says, in other words, is the following: if L : V → W is

linear, then it must map the zero vector of V to the zero vector of W. Where
can we use this theorem ? Well, if we ever stumble upon a mapping L : V → W
which maps the zero vector of V to something other than the zero vector of
W, then we can be sure that L is not a linear mapping. Before going further,
we prove the theorem:

Proof. Let α = 0 (i.e. α is the real number zero); we have α0V = 0V (we have
already shown that any real number times the zero vector is equal to the zero
vector, and in particular if that real number happens to be zero), and therefore,
we can write:

L(0V ) = L(α0V )
= αL(0V )
= 0L(0V )
= 0W

since, as we have already shown, the real number 0 times any vector of a vector
space is equal to the zero vector of that vector space.

Let us now examine mappings between vector spaces, which are not linear.

1. Let V and W be real vector spaces, and let w ∈ W be a given vector in

W such that w 6= 0W (i.e. w is not the zero vector of W). Define the
mapping L : V → W by L(v) = w, for all v ∈ V. We have:

L(0V ) = w 6= 0W ,

i.e. L maps the zero vector of V to something other than the zero vector
of W; it follows therefore from the previous theorem that the mapping L
is not linear.

2. Let V be a real vector space, let λ ∈ R be a real number, and let v1 be a

vector in V such that v1 6= 0V (i.e. v1 is distinct from the zero vector of
122 SECTION 10

V). Let L : V → V be the mapping defined by L(v) = λv + v1 , for all

v ∈ V. We have:

L(0V ) = λ0V + v1
= 0V + v1
= v1 6= 0V ,

and since L maps the zero vector of V to something other than the zero
vector of V, it follows from the previous theorem that the mapping L is
not linear.
3. Consider the real vector spaces R2 and R, and let L : R2 → R be the
mapping defined by L((x, y)) = x2 + y 2 for all (x, y) ∈ R2 . Let us show
that the mapping L is not linear. The previous theorem is not of much
help here, since L does indeed map the zero vector of R2 (namely the pair
(0, 0)) to the zero vector of R (namely the real number 0); indeed, we
have:

L((0, 0)) = 02 + 02 = 0 + 0 = 0;

so we cannot use the previous theorem to show that L is not linear, unlike
what we did in the previous examples.
So how do we proceed ? Well, we have to go back to the definition. From
there, it is clear that it is enough to find two vectors v1 , v2 in R2 for which
L(v1 + v2 ) is not equal to L(v1 ) + L(v2 ), or a real number α and a vector
v in R2 such that L(αv) is not equal to αL(v), in order to conclude that
L is not a linear mapping.
Let us choose for example α = 2 and v = (1, 1). For these choices of α
and v, we have:

L(αv) = L(2(1, 1)) = L(2, 2) = 22 + 22 = 4 + 4 = 8,

αL(v) = 2L((1, 1)) = 2(12 + 12 ) = 2(1 + 1) = 4.

Hence, we have found some real number α (namely α = 2) and some vector
v in R2 (namely v = (1, 1)) for which L(αv) is not equal to αL(v). This
shows that the second property in the definition of a linear mapping does
not hold for our mapping L, and we conclude from this that the mapping
L is not linear.
4. Consider the real vector space R2 , and let L : R2 → R2 be the mapping
defined by L((x, y)) = (x3 + y, x − y) for all (x, y) ∈ R2 . It is easy to verify
here as well that the mapping L defined here is not linear.
5. Consider the real vector space C([0, 1]; R) of all continuous functions from
[0, 1] to R, and define the mapping L : C([0, 1]; R) → R from the real
vector space C([0, 1]; R) to the real vector space R as follows: L(f ) =
R1
0
(f (t))2 dt for all f ∈ C([0, 1]; R). Let us show that L is not a linear
 123

mapping. For this, let us show that the second property in the definition
of a linear mapping fails in this case. Let then α = 2, and define the vector
f ∈ C([0, 1]; R) as follows (recall that in the real vector space C([0, 1]; R),
a vector is nothing other than a continuous function from [0, 1] to R):
f (t) = 1, for all t ∈ R (in other words, f is defined to be the constant
function on [0, 1] which maps every t ∈ [0, 1] to the real number 1). Clearly,
f is continuous on [0, 1] with values in R, and hence is an honest element
of C([0, 1]; R). With these choices of α ∈ R and f ∈ C([0, 1]; R), let us
compute L(αf ) and αL(f ). We have:
Z 1
L(αf ) = ((αf )(t))2 dt
0
Z 1
= (αf (t))2 dt
0
Z 1
= α2 (f (t))2 dt
0
Z 1
2
= α (f (t))2 dt
0
Z 1
= 22 (1)2 dt
0
Z 1
= 4 dt
0
= 4,

whereas
Z 1
αL(f ) = α (f (t))2 dt
0
Z 1
= 2 (1)2 dt
0
Z 1
= 2 dt
0
= 2,

which shows that for these choices of α ∈ R and f ∈ C([0, 1]; R), L(αf )
is not equal to αL(f ), and this shows that the second property in the
definition of a linear mapping does not hold for our mapping L, and we
conclude from this that the mapping L is not linear.

Now that we have seen examples of linear (and non-linear) mappings between
vector spaces, let us examine an interesting property of linear mappings: Con-
sider three real vector spaces V, W, and Z, let L1 : V → W be a mapping
from the real vector space V to the real vector space W, and L2 : W → Z a
a mapping from the real vector space W to the real vector space Z. We know
124 SECTION 10

we can then define a mapping from V to Z, which we denote by L2 ◦ L1 (pay

attention to the order!) as follows:

• L2 ◦ L1 : V → Z is defined to be the mapping from V to Z which maps

every element v ∈ V to the element L2 (L1 (v)) of Z.

L2 ◦ L1 is called the composition of the two mappings L1 and L2 ; its definition

is very intuitive: If we are given v in V and we want to compute L2 ◦ L1 (v),
we first “apply” L1 to v, obtaining the element L1 (v) of W. Since L1 (v) is in
W and since the mapping L2 maps from W (to wherever), it makes sense to
“apply” L2 to L1 (v). We then “apply” L2 to L1 (v), obtaining the element of
L2 (L1 (v)) of Z, and it is this last element that we denote by L2 ◦ L1 (v).
Let us now again consider three real vector spaces V, W, and Z, let L1 : V → W
be a mapping from the real vector space V to the real vector space W, and
L2 : W → Z a a mapping from the real vector space W to the real vector space
Z; assume now that L1 and L2 are both linear mappings. What can we say
then about the mapping L2 ◦L1 ? The answer is given by the following theorem:

Theorem 16. Let V, W, and Z be real vector spaces, and let L1 : V → W and
L2 : W → Z be linear mappings. Then, the mapping L2 ◦ L1 : V → Z is also
linear.

Proof. We have to show that for any v1 , v2 ∈ V, L2 ◦ L1 (v1 + v2 ) is equal to

L2 ◦ L1 (v1 ) + L2 ◦ L1 (v2 ), and that for any α ∈ R and any v ∈ V, L2 ◦ L1 (αv)
is equal to α(L2 ◦ L1 (v)). Let us then prove these properties in turn:

(i) Let v1 , v2 ∈ V; we have, using the linearity of L1 and L2 :

L2 ◦ L1 (v1 + v2 ) = L2 (L1 (v1 + v2 ))

= L2 (L1 (v1 ) + L1 (v2 ))
= L2 (L1 (v1 )) + L2 (L1 (v2 ))
= L2 ◦ L1 (v1 ) + L2 ◦ L1 (v2 ),

which proves the first property in the definition of a linear mapping.

(ii) Let now α ∈ R and v ∈ V; we have, using the linearity of L1 and L2 :

L2 ◦ L1 (αv) = L2 (L1 (αv))

= L2 (αL1 (v))
= αL2 (L1 (v))
= α(L2 ◦ L1 (v)),

which proves the second property in the definition of a linear mapping.

We have shown that L2 ◦ L1 satisfies both properties of the definition of a linear

mapping; it follows that L2 ◦ L1 is a linear mapping.
 125

Let us now consider two real vector spaces V and W and a linear mapping
L : V → W. Consider the set of all v in V which are mapped to the zero vector
0W of W by L, i.e. the set of all v ∈ V for which L(v) = 0W . Let us give a
name to the set of all such elements of V; we denote by ker(L) (and we call this
“Kernel of L or Null Space of L”) the set of all v ∈ V for which L(v) = 0W ;
we can write this statement more formally as:
ker(L) = {v ∈ V|L(v) = 0W }.
What do we know about ker(L) ? Well, the first thing we know about ker(L),
which follows straight from its definition, is that it is a subset of V. But as
the following theorem will show, ker(L) is not just any old subset of V; rather,
it is a vector subspace of V.
Theorem 17. Let V and W be real vector spaces, and let L : V → W be a linear
mapping. Then, the kernel ker(L) of L is a vector subspace of V.
Proof. To show that ker(L) is a vector subspace of the real vector space V, we
have to show that:
(i) The zero vector 0V of V is in ker(L),
(ii) for any v1 and v2 in ker(L), v1 + v2 is also in ker(L),
(iii) for any v in ker(L) and any α in R, αv is also in ker(L).
• We know from the first theorem we proved in this lecture that (by linearity
of L) L(0V ) = 0W , from which we obtain that 0V is indeed in ker(L)
(by definition of ker(L)); i.e. we have shown property (i), namely that
0V ∈ ker(L).
• Let us now prove property (ii). Let then then v1 , v2 ∈ ker(L); we have
to show that v1 + v2 is also in ker(L), i.e. that L(v1 + v2 ) = 0W . But,
by linearity of L and the fact that v1 and v2 are assumed to be in ker(L)
(and therefore L(v1 ) = 0W and L(v2 ) = 0W ):
L(v1 + v2 ) = L(v1 ) + L(v2 )
= 0W + 0W
= 0W ,
which proves that v1 + v2 is also in ker(L). This proves property (ii).
• Let us now prove property (iii). Let then then v ∈ ker(L) and α ∈ R; we
have to show that αv is also in ker(L), i.e. that L(αv) = 0W . But, by
linearity of L and the fact that v is assumed to be in ker(L) (and therefore
L(v) = 0W ):
L(αv) = αL(v)
= α0W
= 0W ,
which proves that αv is also in ker(L). This proves property (iii).
126 SECTION 10

We have shown that ker(L) satisfies all three properties of a vector subspace;
this shows that ker(L) is a vector subspace of V.
Recall that a mapping L : V → W is said to be injective or one-to-one if for
any two elements v1 , v2 ∈ V with v1 6= v2 it must follow that L(v1 ) 6= L(v2 );
equivalently, L is injective if for any v1 , v2 ∈ V, the equality L(v1 ) = L(v2 )
implies v1 = v2 (i.e two distinct elements in V cannot map to the same value
under L). The following theorem relates the property of L being injective to
the kernel ker(L) of L.
Theorem 18. Let L : V → W be a linear mapping from the real vector space V
to the real vector space W. We have:
• L is injective if and only if the kernel ker(L) of L is equal to {0V }, i.e.
ker(L) = {0V }.
Proof. (i) Assume first that L is injective; we have to show then that if v ∈
ker(L), then v = 0V (i.e. the only element of ker(L) is the zero vector of
V). Let then v ∈ ker(L). Then, L(v) = 0W (by definition of v being an
element of the kernel ker(L) of L). On the other hand, we also know that
L(0V ) = 0W . We therefore have:
L(v) = L(0V ).
Since L is assumed injective, it must follow that v = 0V . Hence, we have
shown that if v is any element in ker(L), then it must follow that v = 0V ;
this shows that ker(L) = {0V }.
(ii) Let us now assume that ker(L) = {0V }; we have to show then that for
any v1 , v2 ∈ V, the equality L(v1 ) = L(v2 ) implies that v1 = v2 . Let
then v1 , v2 ∈ V and assume that L(v1 ) = L(v2 ). Hence, by linearity of
L:
L(v1 − v2 ) = L(v1 + (−1)v2 )
= L(v1 ) + L((−1)v2 )
= L(v1 ) + (−1)L(v2 )
= L(v1 ) − L(v2 )
= 0W ,
which shows that v1 − v2 ∈ ker(L) (since L(v1 − v2 ) = 0W ). Since we
have assumed that ker(L) = {0V }, it must follow that v1 − v2 = 0V ,
i.e. that v1 = v2 . This is what we wanted to show, and this establishes
injectivity of L.

Let us now consider again a linear mapping L : V → W from a real vector

space V to a real vector space W. Consider now the subset Im(L) of W (Note:
 127

subset of W, not of V like last time!) defined formally by:

Im(L) = {L(v) ∈ W|v ∈ V};

Here are, in plain English, two equivalent definitions for Im(L):

• Im(L) is the set of all values in W that L can take.

• w is in Im(L) if and only if there exists a v in V such that w = L(v).

Im(L) is called the Range of L or Image of L.

What do we know about Im(L) ? Well, by its very definition, it is a subset of
W; but as the following result will show, Im(L) is not just any old subset of
W, rather, it is a vector subspace of W.

Theorem 19. Let V and W be real vector spaces, and let L : V → W be a linear
mapping. Then, the range Im(L) of L is a vector subspace of W.

Proof. We have to prove that Im(L) satisfies the three properties that a vector
subspace of W should satisfy, namely, we have to show that:

(i) The zero vector 0W of W is in Im(L),

(ii) for any v1 and v2 in Im(L), v1 + v2 is also in Im(L),

(iii) for any v in Im(L) and any α in R, αv is also in Im(L).

• We know from the first theorem we proved in this lecture that (by linearity
of L) L(0V ) = 0W , from which we obtain that there exists an element in
V (namely the zero vector of V) which is mapped by L to the zero vector
0W of W; this shows that 0W is in the range of L, i.e. 0W ∈ Im(L).
Hence, we have shown property (i).

• Let us now prove property (ii). Let then then w1 , w2 ∈ Im(L); we have
to show that w1 + w2 is also in Im(L), i.e. we have to show that w1 + w2
is L of something in V. Since we have w1 ∈ Im(L), by definition of
Im(L), there must exist a vector in V, call it v1 , such that L(v1 ) = w1 .
Similarly, since we have w2 ∈ Im(L), by definition of Im(L), there must
exist a vector in V, call it v2 , such that L(v2 ) = w2 . Consider now the
vector v in V defined by v = v1 + v2 (i.e. v is defined to be the sum of
the two vectors v1 and v2 ). We have:

L(v) = L(v1 + v2 )
= L(v1 ) + L(v2 )
= w1 + w2 ,

which proves that w1 + w2 is in the range of L, i.e. w1 + w2 ∈ Im(L)

(since we have been able to write w1 + w2 as L of something in V). This
proves property (ii).
128 SECTION 10

• Let us now prove property (iii). Let then then w ∈ Im(L) and α ∈ R; we
have to show that αw is also in Im(L), i.e. we have to show that αw is L
of something in V. We have:

L(αv) = αL(v)
= αw,

which proves that αw is in Im(L) (since we have written αw as L of

something in V). This proves property (iii).

We have shown that Im(L) satisfies all three properties of a vector subspace;
this shows that Im(L) is a vector subspace of W.

We close this section with some important terminology and an important the-
orem which we shall merely state and not prove:

Definition 25. Let V, W be real vector spaces, assume V is finite-dimensional,

and let L : V → W be a linear mapping.

(i) The rank of L is defined to be the dimension of Im(L), and is denoted

by rank(L).

(ii) The nullity of L is defined to be the dimension of ker(L), and is denoted

by nullity(L).

The following important theorem is known as the Rank-Nullity theorem:

Theorem 20. Let V, W be real vector spaces, and let L : V → W be a linear

mapping. Assume V is finite-dimensional, and let N denote the dimension of
V. Then:

rank(L) + nullity(L) = N.

PROBLEMS:

Show which of the following mappings between real vector spaces are linear and
which are not linear:

1. L : R → R, defined by L(x) = 2x + 1.

2. L : R → R, defined by L(x) = x2 + x.

3. L : R → R2 , defined by L(x) = (x, 3x).

4. L : R → R2 , defined by L(x) = (x + 5, 3x).

5. L : R → R2 , defined by L(x) = (x + x2 , 2x).

 129

6. L : R → R2 , defined by L(x) = (3x, 2x − 1).

7. L : R2 → R2 , defined by L((x, y)) = (x + y, x − y).
8. L : R2 → R2 , defined by L((x, y)) = (x2 + y, x − y 2 ).
9. L : R2 → R2 , defined by L((x, y)) = (x2 + y 2 , x2 − y 2 ).
10. L : R2 → R3 , defined by L((x, y)) = (x + y, x − y, xy).

11. L : R2 → R3 , defined by L((x, y)) = (x + y, x − y, x).

12. L : R2 → R3 , defined by L((x, y)) = (x + y, x − y, 3y).
13. L : R2 → R3 , defined by L((x, y)) = (x + y + 1, x − y, 3y).
14. L : R3 → R3 , defined by L((x, y, z)) = (x + z, x − y, 3z).
15. L : R3 → R3 , defined by L((x, y, z)) = (z, x, y).
16. L : R3 → R3 , defined by L((x, y, z)) = (z, y, x).
17. L : R3 → R3 , defined by L((x, y, z)) = (z, x, z).
18. L : R3 → R3 , defined by L((x, y, z)) = (x, x, x).
19. L : R3 → R3 , defined by L((x, y, z)) = (x, x, x + 1).
20. L : R3 → R3 , defined by L((x, y, z)) = (x, x2 , x3 ).
21. L : R3 → R3 , defined by L((x, y, z)) = (zx, xy, yz).
22. L : R3 → R3 , defined by L((x, y, z)) = (2x + y, y − 3z, x + y + z).
23. L : R3 → R3 , defined by L((x, y, z)) = (x2 , y 2 , z 2 ).
24. L : R3 → R2 , defined by L((x, y, z)) = (x, y).
25. L : R3 → R2 , defined by L((x, y, z)) = (x, z).
26. L : R3 → R2 , defined by L((x, y, z)) = (y, z).
27. L : R3 → R2 , defined by L((x, y, z)) = (y, x).
28. L : R3 → R, defined by L((x, y, z)) = x + y + z.
29. L : R3 → R, defined by L((x, y, z)) = x + y + z + 2.
30. L : R3 → R, defined by L((x, y, z)) = x2 + y 2 + z 2 .
31. L : R3 → R, defined by L((x, y, z)) = x3 + y 3 + z 3 .
32. L : R3 → R, defined by L((x, y, z)) = xyz.
33. L : F (R; R) → R defined by L(f ) = 2f (−1) + 5f (1).
130 SECTION 10

34. L : F (R; R) → R defined by L(f ) = f (0)f (1).

35. L : F (R; R) → R defined by L(f ) = f (−2) + (f (1))2 .
R1
36. L : C([0, 1]; R) → R defined by L(f ) = 1/2 f (t)dt.
R 1/2 R1
37. L : C([0, 1]; R) → R defined by L(f ) = 0
f (t)dt − 1/2
f (t)dt.
R 1/2 R1
38. L : C([0, 1]; R) → R defined by L(f ) = 0
f (t)dt − 1/2
(f (t))2 dt.
R 1/2 R1
39. L : C([0, 1]; R) → R defined by L(f ) = 1 + 0 f (t)dt − 1/2 f (t)dt.
R 1/2 R1
40. L : C([0, 1]; R) → R defined by L(f ) = f (3/4) − 0
f (t)dt − 1/2
f (t)dt.
R 1/2 R1
41. L : C([0, 1]; R) → R defined by L(f ) = (f (3/4))2 − 0
f (t)dt− 1/2
f (t)dt.
Section 11

Study Topics
• Real Matrices
• Linear Transformations Defined by Matrices
• Range and Kernel of a Matrix

131
132 SECTION 11

In the previous lecture, we have studied a special class of functions from one
real vector space to another, namely those that were linear, and we called them
linear functions, or, equivalently, linear mappings, or equivalently, linear
transformations. In this lecture, we introduce the notion of a real “matrix”.
What is a real matrix ? Think of it as a table, with, say m rows and n columns,
of real numbers. That’s it! So what’s the relation between a matrix and a linear
transformation ? As we will see in this lecture, a real matrix with m rows and
n columns will allow us to define a linear transformation from R cn to Rdm.

Definition 26. Let m and n be integers ≥ 1. A real matrix with m rows and
n columns (also called a real m × n matrix) is a table (or array) of the form:
 
a1,1 a1,2 · · · a1,n
 a2,1 a2,2 · · · a2,n 
 
 .. .. ..  ,
 . . ··· . 
am,1 am,2 ··· am,n
where all the entries (i.e. a1,1 , a1,2 , a1,3 , · · · ) are real numbers.
ai,j is called the entry of the matrix on row i and column j.
Let us look immediately at some examples:
 √ 
2
1. The matrix is a real 2 × 1 real matrix (2 rows, 1 column).
−π
 
2 0√
2. The matrix is a 2 × 2 real matrix (2 rows, 2 columns).
−1 1/ 3
 
1 0
3. The matrix  −1 1  is a 3 × 2 real matrix (3 rows, 2 columns).
2 3
 
1 0 −2 1
4. The matrix is a 2×4 real matrix (2 rows, 4 columns).
−1 1 2 3


5. The matrix 1 3 2 1 5 is a 1 × 5 real matrix (1 row, 5 columns).
6. The matrix (−7) is a 1 × 1 real matrix (1 row, 1 column).
It is important to point out that the place of the elements in a matrix matters;
exchanging the place of two elements in the matrix will change the matrix
 if
1 0
those two elements are not equal. For example, the matrices and
  0 0
0 0
are not equal. More generally, if A is defined to be the m × n real
0 1
matrix
 
a1,1 a1,2 · · · a1,n
 a2,1 a2,2 · · · a2,n 
 
A= . .. ..  ,
 .. . ··· . 
am,1 am,2 · · · am,n
 133

and B is defined to be the m × n real matrix

 
b1,1 b1,2 · · · b1,n
 b2,1 b2,2 · · · b2,n 
 
B= . .. .. ,
 .. . ··· . 
bm,1 bm,2 · · · bm,n

then we have A = B if and only if ai,j = bi,j for all 1 ≤ i ≤ m and all 1 ≤ j ≤ n,
i.e. the corresponding entries of A and B must be equal.
Definition 27. Let
 
a1,1 a1,2 ··· a1,n
 a2,1 a2,2 ··· a2,n 
 
A= .. .. .. ,
 . . ··· . 
am,1 am,2 ··· am,n

be a real m × n matrix. Let j ∈ {1, 2, · · · , n}. The j th column vector of A,

denoted by A;j is defined to be the element of R dm given by:

 
a1,j
 a2,j 
 
 
A;j =  a3,j  .
 .. 
 . 
am,j

Similarly, letting i ∈ {1, 2, · · · , m}, the ith row vector of A, denoted by Ai; is
defined to be the element of Rn given by:

Ai; = (ai,1 , ai,2 , ai,3 , · · · , ai,n ).

We shall very often write the row vector Ai; without the separating commas,
i.e., as:


Ai; = ai,1 ai,2 ai,3 · · · ai,n ;

despite the lack of the separating commas, Ai; will still be understood as an n−tuple
of real numbers, i.e. as an element of Rn .
 
3 2
Let us look at an example: Consider the 3 × 2 real matrix  −1 0 ; its first
  0 1
3
column vector is the element of R c3 given by  −1 , and its second column
  0
2
vector is the element of R c3 given by  0 . Its first row vector is the element
1
134 SECTION 11

of R2 given by (3, 2), its second row vector is the element of R2 given by (−1, 0),
and finally, its third row vector is the element of R2 given by (0, 1). As was
mentionned above, we shall often omit the separating commas when writing
row vectors of matrices;

for example, we shall write the first row vector of this

matrix as 3
2 instead, and its second and third row vectors as −1 0
and 0 1 , respectively, and we shall still consider them as elements of R2
(i.e. pairs of real numbers), despite the absence of the separating commas.
Definition 28. Let m, n be integers ≥ 1. We denote the set of all m × n real
matrices by Mm,n (R).
In other words, an element of Mm,n (R) is nothing other than a real m × n
matrix, and any real m × n matrix is an element of Mm,n (R).
We now define two operations on Mm,n (R), namely addition of real matri-
ces and multiplication of a real matrix by a real number. Let us define
the addition operation first. Let then
 
a1,1 a1,2 · · · a1,n
 a2,1 a2,2 · · · a2,n 
 
A= . .. ..  ,
 .. . ··· . 
am,1 am,2 ··· am,n
and
 
b1,1 b1,2 ··· b1,n
 b2,1 b2,2 ··· b2,n 
 
B= .. .. .. ,
 . . ··· . 
bm,1 bm,2 ··· bm,n
be two real m × n matrices. We define the sum of A and B, and we denote by
A + B, the real m × n matrix given by:
 
a1,1 + b1,1 a1,2 + b1,2 · · · a1,n + b1,n
 a2,1 + b2,1 a2,2 + b2,2 · · · a2,n + b2,n 
 
A+B = .. .. .. ;
 . . ··· . 
am,1 + bm,1 am,2 + bm,2 ··· am,n + bm,n
in other words, A + B is the real m × n matrix obtained by adding together the
corresponding entries of the matrices A and B.
Let now α be a real number; we define the product of the matrix A and the real
number α, and we denote by α · A (the “·”, denoting the scalar multiplication
operation – as with vectors, we shall drop the “·” in the notation very soon ...),
the real m × n matrix given by:
 
αa1,1 αa1,2 · · · αa1,n
 αa2,1 αa2,2 · · · αa2,n 
 
α·A=  .. .. .. ;
 . . ··· . 
αam,1 αam,2 · · · αam,n
 135

in other words, α · A is the real m × n matrix obtained by scaling by α each

entry of A.
We have now defined an addition operation (denoted by the usual “+” sym-
bol), as well as a scalar multiplication operation (denoted by the usual “·”
symbol), on the set Mm,n (R) of all real m × n matrices. What do we gain from
this ? The answer is given by the next theorem:

Theorem 21. (Mm,n (R), +, ·) is a real vector space. The zero vector 0 of
this vector space is the real m × n matrix with all entries equal to 0, i.e.,
 
0 0 ··· 0
 0 0 ··· 0 
 
0= . . . ;
 .. .. · · · .. 
0 0 ··· 0

The proof of this theorem is immediate and is left to the reader. The bottom
line is that we can think of real m × n matrices themselves as vectors (in the
real vector space Mm,n (R)) ...
Let now again m, n be integers ≥ 1 and let A ∈ Mm,n (R) be the matrix defined
by:
 
a1,1 a1,2 · · · a1,n
 a2,1 a2,2 · · · a2,n 
 
A= . .. ..  ,
 .. . ··· . 
am,1 am,2 · · · am,n

cn
where the ai,j are some given real numbers. Consider the function LA from R
d
to R cn → R
m , i.e. the function L : R
A
d m defined as follows: Let

 
x1
 x2 
 
 
v =  x3 
 .. 
 . 
xn

cn ; LA (v) is defined to be the element of R

be any element of R d m given by:

 
a1,1 x1 + a1,2 x2 + · · · + a1,n xn
 a2,1 x1 + a2,2 x2 + · · · + a2,n xn 
 
LA (v) =  .. .
 . 
am,1 x1 + am,2 x2 + · · · + am,n xn

NOTE: We shall very often denote LA (v) by Av.

Let us examine some examples before going further:
136 SECTION 11

1. Consider the real 1 × 4 matrix A given by A = −1 1 0 3 ; The
mapping LA defined by A is the mapping LA : R c4 → R (we consider R
c1
to be just R itself!) given by the following rule: Let
 
x1
 x2 
v=  x3 


x4

c4 ; LA (v) is defined to be the element of R given by:

be any element of R

LA (v) = (−1)x1 + (1)x2 + (0)x3 + (3)x4

= −x1 + x2 + 3x4 .

Again, as indicated above, we shall often write simply Av instead of LA (v),

and in this case, we will write therefore

Av = −x1 + x2 + 3x4 .


−1 2
2. Consider now the real 3 × 2 matrix A given by A =  3 0 ; The
4 5
c2 c3
mapping LA defined by A is the mapping LA : R → R given by the
following rule: Let
 
x1
v=
x2

c2 ; LA (v) is defined to be the element of R

be any element of R c3 given by:
   
−x1 + 2x2 −x1 + 2x2
LA (v) =  3x1 + 0x2  =  3x1 
4x1 + 5x2 4x1 + 5x2

and again, we shall often simply write Av instead of LA (v), i.e., we shall
write
 
−x1 + 2x2
Av =  3x1 .
4x1 + 5x2

3. Let n be an integer ≥ 1 and let A be the real n × n matrix given by:

 
1 0 0 ··· 0
 0 1 0 ··· 0 
 
 1 ··· 0 
A= 0 0 ,
 .. .. .. .. 
 . . . ··· . 
0 0 0 ··· 1
 137

i.e. the matrix A has 1’s on the diagonal, and 0’s everywhere else. As we
cn → R
have seen, A defines therefore a function LA : R cn as follows: Let
 
x1
 x2 
 
 x3 
v= 
 .. 
 . 
xn

cn ; LA (v) is defined to be the element of R

be any element of R cn given by:
   
1x1 + 0x2 + · · · + 0xn x1
 0x1 + 1x2 + · · · + 0xn   x2 
   
LA (v) =  .. = ..  = v,
 .   . 
0x1 + 0x2 + · · · + 1xn xn

i.e., for any v ∈ Rcn , we have: LA (v) = v. This shows that LA is nothing
other than the identity mapping of R cn (i.e. the function which maps
every vector in Rcn to itself); for this reason, the matrix A in this example
is usually called the n×n identity matrix. Hence, with A being the n×n
identity matrix, we can write (in keeping with our simplified notation):
Av = v for every v ∈ R cn .

Let now A be a real m × n matrix (i.e. an element of Mm,n (R); what can we
say about the function LA : Rcn → R dm ? Does it have any special property ?

The answer is given by the following theorem:

cn → R
Theorem 22. Let A ∈ Mm,n (R); the mapping LA : R d m defined by A is

linear.

Proof. Let
 
a1,1 a1,2 ··· a1,n
 a2,1 a2,2 ··· a2,n 
 
A= .. .. .. 
 . . ··· . 
am,1 am,2 ··· am,n

be any element in Mm,n (R), and consider the mapping LA : R cn → R d

m . To

show that LA is linear, we have to show the following two properties:

cn , we have LA (v1 + v2 ) = LA (v1 ) + LA (v2 ), and

(i) ∀v1 , v2 ∈ R

cn , ∀α ∈ R, we have LA (αv) = αLA (v).

(ii) ∀v ∈ R
138 SECTION 11

Let us begin by showing the first property. For this, let

   
x1 y1
 x2   y2 
   
 x3   
v1 =   , v2 =  y3  ,
 ..   .. 
 .   . 
xn yn

be any two elements in Rcn ; we have to show that LA (v1 +v2 ) = LA (v1 )+LA (v2 ).
Let us first calculate LA (v1 + v2 ), and before we do this, let us first calculate
v1 + v2 . We have:
 
x1 + y1
 x2 + y2 
 
 
v1 + v2 =  x3 + y3  ,
 .. 
 . 
xn + yn
and therefore, by definition of LA ,
 
a1,1 (x1 + y1 ) + a1,2 (x2 + y2 ) + · · · + a1,n (xn + yn )
 a2,1 (x1 + y1 ) + a2,2 (x2 + y2 ) + · · · + a2,n (xn + yn ) 
 
LA (v1 + v2 ) =  .. .
 . 
am,1 (x1 + y1 ) + am,2 (x2 + y2 ) + · · · + am,n (xn + yn )
Separating the xi ’s from the yj ’s in the expression obtained, we can write:
 
(a1,1 x1 + · · · + a1,n xn ) + (a1,1 y1 + · · · + a1,n yn )
 (a2,1 x1 + · · · + a2,n xn ) + (a2,1 y1 + · · · + a2,n yn ) 
 
LA (v1 + v2 ) =  .. 
 . 
(am,1 x1 + · · · + am,n xn ) + (am,1 y1 + · · · + am,n yn )
   
a1,1 x1 + · · · + a1,n xn a1,1 y1 + · · · + a1,n yn
 a2,1 x1 + · · · + a2,n xn   a2,1 y1 + · · · + a2,n yn 
   
=  .. + .. 
 .   . 
am,1 x1 + · · · + am,n xn am,1 y1 + · · · + am,n yn
= LA (v1 ) + LA (v2 ).
cn , we have LA (v1 +
This shows that property (i) holds, namely, that ∀v1 , v2 ∈ R
v2 ) = LA (v1 ) + LA (v2 ).
Let us now prove property (ii); for this, let α ∈ R be any real number, and let
 
x1
 x2 
 
 
v =  x3 
 .. 
 . 
xn
 139

cn ; we have to show that LA (αv) = αLA (v). Let us first

be any element in R
compute αv; we have:
   
x1 αx1
 x2   αx2 
   
   
αv = α  x3  =  αx3  ,
 ..   .. 
 .   . 
xn αxn
and therefore,
 
a1,1 (αx1 ) + a1,2 (αx2 ) + · · · + a1,n (αxn )
 a2,1 (αx1 ) + a2,2 (αx2 ) + · · · + a2,n (αxn ) 
 
LA (αv) =  .. 
 . 
am,1 (αx1 ) + am,2 (αx2 ) + · · · + am,n (αxn )
 
α(a1,1 x1 + a1,2 x2 + · · · + a1,n xn )
 α(a2,1 x1 + a2,2 x2 + · · · + a2,n xn ) 
 
=  .. 
 . 
α(am,1 x1 + am,2 x2 + · · · + am,n xn )
 
a1,1 x1 + a1,2 x2 + · · · + a1,n xn
 a2,1 x1 + a2,2 x2 + · · · + a2,n xn 
 
= α .. 
 . 
am,1 x1 + am,2 x2 + · · · + am,n xn
= αLA (v).

This proves property (ii), namely that ∀v ∈ R cn and ∀α ∈ R, we have LA (αv) =

αLA (v).
Hence, LA satisfies properties (i) and (ii) and is therefore a linear mapping.
We now show the converse to the previous result, i.e. we show that if L :
cn → R
R dm is a linear mapping, then there exists a real m × n matrix C such

that L = LC ; in other words, any linear mapping from R cn to R

d m is actually

defined by a real m × n matrix.

Theorem 23. Let L : Rcn → R d
m be a linear mapping. Then, there exists C ∈

Mm,n (R) such that L = LC .

cn → R
Proof. Let L : R d m be a linear mapping.
cn as follows:
Define the vectors e1 , e2 , · · · , en ∈ R
     
1 0 0
 0   1   0 
     
     
e1 =  0  , e2 =  0  , · · · en =  0 ,
 ..   ..   .. 
 .   .   . 
0 0 1
140 SECTION 11

i.e. for i ∈ {1, · · · , n}, ei has entry 1 on row i and entry 0 everywhere else.
cn , and it is called the canonical basis
Note that (e1 , e2 , · · · , en ) is a basisof R 
x1
 x2 
 
  cn
of R . Note also that any element  x3  ∈ R
cn can be written as
 .. 
 . 
xn
 
x1
 x2 
 
 x3 
  = x1 · e1 + x2 · e2 + · · · + xn · en .
 .. 
 . 
xn

Hence, we have, by linearity of L:

 
x1
 x2 
 
 x3 
L( ) = L(x1 · e1 + x2 · e2 + · · · + xn · en )
 .. 
 . 
xn
= x1 · L(e1 ) + x2 · L(e2 ) + · · · + xn · L(en ).

d
L(e1 ), L(e2 ), · · · , L(en ) are all elements of R m , i.e. are column vectors with m

real entries; denote their entries as follows:

     
c1,1 c1,2 c1,n
 c2,1   c2,2   c2,n 
     
 c3,1   c3,2   c3,n 
L(e1 ) =   , L(e2 ) =   , · · · L(en ) =  ;
 ..   ..   .. 
 .   .   . 
cm,1 cm,2 cm,n

Let now C ∈ Mm,n (R) be the real m × n matrix constructed from the entries
of L(e1 ), L(e2 ), · · · , L(en ) as follows:
 
c1,1 c1,2 ··· c1,n
 c2,1 c2,2 ··· c2,n 
 
C= .. .. .. .
 . . ··· . 
cm,1 cm,2 ··· cm,n

Note that the first column vector of C is nothing other than L(e1 ), the second
column vector of C is nothing other than L(e2 ), and so on.
 141

 
x1
 x2 
 
 x3  cn
We have, for any element  ∈R :
 .. 
 . 
xn
 
x1  
  c1,1 x1 + c1,2 x2 + · · · + c1,n xn
 x2   
   c2,1 x1 + c2,2 x2 + · · · + c2,n xn 
LC ( x3 ) =  .. 
 ..   . 
 . 
cm,1 x1 + cm,2 x2 + · · · + cm,n xn
xn
     
c1,1 c1,2 c1,n
 c2,1   c2,2   c2,n 
     
 c3,1   c3,2   c3,n 
= x1 ·   + x2 ·   + · · · + xn ·  
 ..   ..   .. 
 .   .   . 
cm,1 cm,2 cm,n
= x1 · L(e1 ) + x2 · L(e2 ) + · · · + xn · L(en )
 
x1
 x2 
 
 
= L( x3 ).
 .. 
 . 
xn

Hence, LC = L.

Now that we have shown that for any real m × n matrix A the mapping LA :
cn → R
R d m is linear (and the converse result), we can ask some further questions;

in particular, we can try (in the spirit of the previous lecture) to identify the
kernel and range of LA ...
Before we go any further, a few words on notation and terminology:
• As has been stated a number of times already, we shall often write Av
instead of LA (v); in that notation, the linearity of LA that we have just
established means that A(v1 + v2 ) = Av1 + Av2 and A(αv) = αAv.
• We shall denote the kernel of LA by ker(A) instead of the more cumber-
some (but correct!) notation ker(LA ), and we shall often just say “kernel
of the matrix A” instead of the more cumbersome “kernel of the linear
mapping LA defined by the matrix A”.
• Similarly, we shall denote the range of LA by Im(A) instead of the more
cumbersome (but correct!) notation Im(LA ), and we shall often just say
“range (or image) of the matrix A” instead of the more cumbersome
“range (or image) of the linear mapping LA defined by the matrix A”.
142 SECTION 11

Let then
 
a1,1 a1,2 ··· a1,n
 a2,1 a2,2 ··· a2,n 
 
A= .. .. .. ,
 . . ··· . 
am,1 am,2 ··· am,n

be a real m × n matrix (i.e. an element of Mm,n (R)), and let us first try to
identify the range of A, i.e. the range of the linear mapping LA : R cn →
d m
R . Recall that the range of LA , which we shall simply write as Im(A), is
d
the set of all LA (v) in R cn ; i.e., writing this formally, we have:
m with v in R

cn }.
Im(A) = {LA (v)|v ∈ R

Let now
 
x1
 x2 
 
 x3 
v= 
 .. 
 . 
xn

cn ; we can write:
be any element in R
 
a1,1 x1 + a1,2 x2 + · · · + a1,n xn
 a2,1 x1 + a2,2 x2 + · · · + a2,n xn 
 
LA (v) =  .. 
 . 
am,1 x1 + am,2 x2 + · · · + am,n xn
     
a1,1 a1,2 a1,n
 a2,1   a2,2   a2,n 
     
= x1  .  + x2  .  + · · · + xn  . 
 ..   ..   .. 
am,1 am,2 am,n
= x1 A;1 + x2 A;2 + · · · xn A;n ,

where, as we have already seen, A;1 denotes the first column vector of A, A;2
denotes the first column vector of A, ..., and so on. Recall that the column
vectors of A, namely A;1 , A;2 , · · · , A;n are all elements of the real vector space
d
R m (since A is m × n, i.e. has m rows!).

So what have we shown ? Well, we have shown that if we take a vector

 
x1
 x2 
 
 x3 
 
 .. 
 . 
xn
 143

 
x1
 x2 
 
cn , then LA (
in R  x3 
) is nothing other than the linear combination
 .. 
 . 
xn

x1 A;1 + x2 A;2 + · · · xn A;n

of the column vectors of A, with the coefficients of this linear combination

(namely
  the real numbers x1 , x2 , · · · , xn ) given by the entries of the vector
x1
 x2 
 
 x3 
 . This shows that any element in Im(A) is a linear combination
 .. 
 . 
xn
of the column vectors of A, i.e. a linear combination of A;1 , A;2 , · · · , A;n .
Conversely, we shall show that any linear combination of the column
vectors of A is in Im(A); indeed, let the vector w ∈ R dm be a linear combination

of A;1 , A;2 , · · · , A;n , i.e. assume there exist real numbers α1 , α2 , · · · , αn such
that:

w = α1 A;1 + α2 A;2 + · · · αn A;n .

We therefore have:
     
a1,1 a1,2 a1,n
 a2,1   a2,2   a2,n 
     
w = α1  ..  + α2  ..  + · · · + αn  .. 
 .   .   . 
am,1 am,2 am,n
 
a1,1 α1 + a1,2 α2 + · · · + a1,n αn
 a2,1 α1 + a2,2 α2 + · · · + a2,n αn 
 
=  .. 
 . 
am,1 α1 + am,2 α2 + · · · + am,n αn
 
α1
 α2 
 
 
= LA ( α3 ),
 .. 
 . 
αn

which shows that w is in Im(A) (since it is LA of something in Rcn !).

Let us recapitulate what we have shown: We have shown that if A is a real
m × n matrix, then Im(A) (i.e. the range of the linear mapping LA : Rcn → Rdm)

is the set of all linear combinations of the column vectors of A; but we

already have a name for this! The set of all linear combinations of the column
144 SECTION 11

d
vectors of A is nothing other than the vector subspace of R m generated by the

column vectors of A, i.e. the linear span of the column vectors of A. We have
therefore proved the following theorem:

Theorem 24. Let A be a real m × n matrix; the range Im(A) of the matrix A
(i.e. the range of the linear mapping LA : R cn → R
d m ) is the linear span of the

column vectors A;1 , A;2 , · · · , A;n of A.

Let us consider some examples:

 
1 0 1
1. Let A be the real 2 × 3 matrix given by A = ; then, Im(A) is
2 3 5
     
the vector subspace of R c2 generated by the subset { 1 , 0 , 1 }
 2  3   5
c2 ; in other words, Im(A) is the linear span of { 1 0 1
of R , , }.
2 3 5
It is easy to show that the linear span of this set is R c2 itself (for exam-
ple, one can easily show that any vector in c2 can be written as a linear
R   
1 0
combination of the first two vectors, i.e. and ), and hence
2 3
Im(A) = R c2 .
 
−1 0
 0 1 
2. Let now A be the real 4 × 2 matrix given by A =   −2
; then,
3 
1 −3
   
−1 0
 0   1 
Im(A) is the vector subspace of R c4 generated by the subset {   }
 −2  ,  3 
1 −3
   
−1 0
   
c4 ; in other words, Im(A) is the linear span of { 0  ,  1 }.
of R  −2   3 
1 −3
c 4
Could it be the case here that Im(A) is R itself ? Well, we know from
Lecture 8 that R c4 has dimension 4; we also know from Theorem 10 of
Lecture 8 that if a subset S of an N −dimensional real vector space V is
a generating set for
 V, then
 the number
 of elements of S must be ≥ N .
−1 0
 0   1 
In our case, S = {   
 −2  ,  3 }, and therefore has only 2 elements
1 −3
(and 2 is not ≥ 4); we conclude that Im(A) 6= Rc4 .

The observations made in these two examples can be packaged into a theorem:
 145

Theorem 25. Let A be a real m × n matrix (i.e. with m rows and n columns). We
have:
d
• If Im(A) = R m , then we must have n ≥ m. Hence, if n < m, then Im(A)
d
cannot be equal to R m.

Proof. We know that Im(A) is equal to the linear span of the n column vectors
d
A;1 , A;2 , · · · , A;n of A. Recall that each of A;1 , A;2 , · · · , A;n is a vector in R m . If

Im(A) = R d, then the subset {A;1 , A;2 , · · · , A;n } is a generating set for R
m d m;

that generating subset has n elements and R d m has dimension m (we have

seen this in Lecture 8), and it follows therefore from Theorem 10 of Lecture 8
that if Im(A) = R d m then we must have n ≥ m.

Now that we have investigated the range Im(A) of a real matrix A, let us examine
its kernel ker(A) (again, by ker(A) we mean the kernel of the linear map LA
defined by A, i.e. ker(LA )). Let then
 
a1,1 a1,2 ··· a1,n
 a2,1 a2,2 ··· a2,n 
 
A= .. .. .. ,
 . . ··· . 
am,1 am,2 ··· am,n

be a real m × n matrix, and let us try to get some information about the kernel
ker(A) of A. By definition of ker(A), cn
  a vector v of R is in ker(A) if and only if
0
 ..  dm ); let us be
LA (v) = 0Rdm (where 0 d
R m =  .  denotes the zero vector of R
0
more precise. Let then
 
x1
 x2 
 
 x3 
v= 
 .. 
 . 
xn

cn ; We have seen already that

be any element in R

LA (v) = x1 A;1 + x2 A;2 + · · · xn A;n ,

where A;1 , A;2 , · · · , A;n are the column vectors of A.

Assume now that A;1 , A;2 , · · · , A;n are linearly independent. Then, the
equality

LA (v) = 0Rd
m
146 SECTION 11

 
0
 
(where 0Rd
m =  ...  denotes the zero vector of R
d m ) is equivalent to the

0
equality

x1 A;1 + x2 A;2 + · · · xn A;n = 0Rd

m,

which, by linear independence of A;1 , A;2 , · · · , A;n , implies x1 = x2 = · · · =

xn = 0, i.e. v = 0Rcn , i.e. v is the zero vector of R cn . In other words, we have
shown that if the column vectors of A are linearly independent, then the
kernel of LA (which we denote by ker(A)) is just {0Rcn }, i.e. ker(A) = {0Rcn }.
Let us now prove the converse. Assume therefore that ker(A) = {0Rcn }; we
would like to show that the column vectors of A are then linearly independent.
Let then α1 , α2 , · · · , αn be real numbers such that

α1 A;1 + α2 A;2 + · · · + αn A;n = 0Rd

m;

cn be
we want to show that α1 , α2 , · · · , αn must then all be zero. Now let v ∈ R
given by:
 
α1
 α2 
 
 
v =  α3  .
 .. 
 . 
αn

The equality

α1 A;1 + α2 A;2 + · · · + αn A;n = 0Rd

m;

can be written

LA (v) = 0Rd
m

which implies v ∈ ker(A) (by definition of ker(A)), and since we have assumed
that ker(A) = {0Rcn }, it follows that v = 0Rcn , i.e.
   
α1 0
 α2   0 
   
 α3   0 
 = ,
 ..   .. 
 .   . 
αn 0

which implies α1 = α2 = · · · = αn = 0, as desired. This proves that the column

vectors A;1 , A;2 , · · · , A;n of A are linearly independent.
In summary, we have proved the following theorem:
 147

Theorem 26. Let A ∈ Mm,n (R) (i.e. A is a real m×n matrix). We have ker(A) =
{0Rcn } if and only if the column vectors of A are linearly independent.

We close this section with some important terminology and an important the-
orem which is a direct adaptation of the Rank-Nullity Theorem of Lecture
9:

Definition 29. Let A ∈ Mm,n (R), i.e. A is a real m × n matrix.

(i) The rank of A is defined to be the dimension of Im(A), and is denoted

by rank(A).

(ii) The nullity of A is defined to be the dimension of ker(A), and is denoted

by nullity(A).

Theorem 27. Let A ∈ Mm,n (R); then:

rank(A) + nullity(A) = n,

i.e. rank(A) + nullity(A) is equal to the number of columns of A.

Proof. The result follows directly from the Rank-Nullity Theorem of Lecture
9, taking into account the fact that:

• rank(A) is by definition the dimension of Im(LA ) (which we have denoted

Im(A)),

• nullity(A) is by definition the dimension of ker(LA ) (which we have

denoted ker(A)),
cn to the real vector
• LA is a linear mapping from the real vector space R
d m
space R ,
cn is a real vector space of dimension n.
• R

PROBLEMS:

1. For each of the following choices for the matrix A, do the following:

• Specify
 the linear
 transformation LA that it defines (for example, if
1 0 c2 → R
c2 defined by
A= , then LA is the mapping LA : R
 2 1    
x1 x1 x1 c2 ,
LA ( )= , for every ∈R
x2 2x1 + x2 x2
• specify its range Im(A) as well as the dimension of Im(A),
148 SECTION 11

• specify its kernel ker(A) as well as the dimension of ker(A);

• verify the Rank-Nullity theorem for A (i.e. verify that dim(Im(A)) +
dim(ker(A)) = number of columns of A).


(a) A = −1 2 .
 
−1
(b) A = .
2


(c) A = 1 0 3 .
 
3 2 1
(d) A = .
0 0 0
 
0 0
(e) A = .
0 0
 
1 0
(f) A = .
0 1
 
1 0 0
(g) A = .
0 1 0
 
1 0 0
(h) A =  0 1 0 .
0 0 1
 
1 2
(i) A =  2 1 .
−1 3
 
1 0
 1 −1 
(j) A = 
 0
.
0 
1 0
 
1 0 0
 1 −1 1 
(k) A =  0
.
0 2 
1 0 −1
 
2 0 0
(l) A =  0 2 0 .
0 0 2
 
5 0 0 0
(m) A = .
0 5 0 0
 
0 5 0 0
(n) A = .
0 0 5 0
 
0 0 5 0
(o) A = .
0 0 0 5
 149

 
5 0 0 0
 0 5 0 0 

(p) A =  .
0 0 5 0 
0 0 0 5
 
3 0 0 0
(q) A = .
0 −2 0 0
 
0 3 0 0
(r) A = .
0 0 −1 0
 
1 0 2
 2 −1 4 
(s) A = 
 −2
.
0 −4 
−1 1 1
 
1 0 2
 2 −1 4 
(t) A = 
 −2
.
0 −4 
−1 1 −2
 
1 2 2
 2 4 4 

(u) A =  .
−2 −4 −4 
−1 −2 −2
 
1 2 2
 2 4 4 
(v) A = 
 −2
.
−4 −4 
−1 2 −2
 
1 0 2 −1
(x) A =  3 0 6 −3 .
−2 0 −4 2
 
1 1 2 −1
(y) A =  3 0 6 −3 .
−2 0 −4 2
 
1 1 2 −1
(z) A =  3 0 6 3 .
−2 0 −4 2
150 SECTION 11
Section 12

Study Topics
• Systems of Linear Transformations Revisited
• Solving Systems of Linear Equations

151
152 SECTION 12

Consider the system of linear equations in m equations and n unknowns given

by:


 a1,1 x1 + a1,2 x2 + a1,3 x3 + · · · a1,n xn = b1

a2,1 x1 + a2,2 x2 + a2,3 x3 + · · · a2,n xn = b2
(E)

 a3,1 x1 + a3,2 x2 + a3,3 x3 + · · · a3,n xn = b3

am,1 x1 + am,2 x2 + am,3 x3 + · · · am,n xn = bm

where the real numbers a1,1 , a1,2 , · · · , am,n and b1 , b2 , · · · , bm are given real
numbers, and we wish to solve for the real numbers x1 , x2 , · · · , xn .
Can we write this system in terms of matrices and vectors ? Before answering
this question, you may ask what the point of writing such a system in terms of
matrices and vectors would be. The answer to this last question is quite simple:
If we are able to write that system of equations in terms of matrices and vectors,
then with the linear algebra knowledge we have accumulated so far, we may be
able to say something interesting about that system of equations: For example,
whether or not it has a solution, how many solutions it does have, and so on,
and even beyond this, in case it does have a solution, how to go about finding
that solution. So let us not waste any time and let us try to express that system
of linear equations using matrices and vectors.
Let then A ∈ Mm,n (R) be the real m × n matrix defined by
 
a1,1 a1,2 · · · a1,n
 a2,1 a2,2 · · · a2,n 
 
A= . .. ..  ,
 .. . ··· . 
am,1 am,2 ··· am,n

d
and let b ∈ R m be the real vector defined by

 
b1
 b2 
 
b =  . .
 . .
bm

It is easy to verify that the real numbers x1 , x2 , · · · , xn satisfy the above system
of linear equations if and only if the vector v ∈ R cn defined by
 
x1
 x2 
 
v =  . .
 .. 
xn

satisfies the vector equation (so called because the unknown is now a vector
cn ):
in R

LA (v) = b.
 153

Note that in keeping with our simplified notation (i.e. writing Av instead of
LA (v)), we can write this last equation simply as

Av = b.

Before going further, let us now see how easily we can prove that our original
system of linear equations (E) can have only 0, 1 or infinitely many solutions.
We already proved this in Lecture 6, but let us prove it again here.
From the discussion above, we need only show therefore that the equation
LA (v) = b, where A ∈ Mm,n (R) and b ∈ R d m are given, and where we wish

to solve for the unknown vector v ∈ R cn , can have only 0, 1 or infinitely many
solutions. We have the following cases:

(Case 1) b ∈/ Im(A). In this case, there is no v ∈ Rcn which satisfies LA (v) = b

(since otherwise b would be in Im(A)), i.e. there is no solution.
cn which satisfies
(Case 2) b ∈ Im(A). In this case, there is at least one vector in R
the equation LA (v) = b; let v1 be one such vector (i.e. LA (v1 ) = b). To
find out how many other vectors satisfy that equation, we consider the
following two subcases:
(Case 2a) ker(A) = {0Rcn }. In this subcase LA is one-to-one and hence there
is a unique v ∈ R cn which satisfies LA (v) = b, namely v1 .
(Case 2b) ker(A) 6= {0Rcn }. In this subcase, there is a non-zero vector w ∈
cn , we
ker(A), i.e w 6= 0Rcn . Since ker(A) is a vector subspace of R
have αw ∈ ker(A), ∀α ∈ R. Then, ∀α ∈ R:

LA (v1 + αw) = LA (v1 ) + LA (αw) = b + αLA (w) = b + 0Rd

m = b,

which shows that v1 + αw is also a solution to the vector equation

LA (v) = b. There are infinitely many distinct vectors of the form
v1 + αw (one for each choice of the real number α), and hence in this
subcase, there are infinitely many solutions to the vector equation
LA (v) = b.
And this completes the proof!
There are two important points to take away from this:
1. The ease with which that non-trivial result is proved,
2. The key role played by the range Im(A) and the kernel ker(A) in de-
termining the number of solutions to the equation LA (v) = b.
Now that we have seen that a system of linear equations can be written very
compactly using matrices and vectors in the form of a vector equation LA (v) = b
(with A, b given, and v the unknown), let us try to use this connection to further
our understanding of systems of linear equations. In particular, let us examine
the following question:
154 SECTION 12

• Let A ∈ Mm,n (R); under what conditions on A does there exist, for
d
each choice of b ∈ R m , a unique solution to the equation L (v) = b ?
A

The answer to this important question is given in the next theorem:

Theorem 28. Let A ∈ Mm,n (R). We have the following:

d
(a) If, for each b ∈ R cn which satisfies LA (v) =
m , there exists a unique v ∈ R

b, then m = n and the column vectors of A are linearly independent.

(b) Conversely, if m = n and the column vectors of A are linearly indepen-

d
dent, then for each b ∈ R cn which satisfies
m , there exists a unique v ∈ R

LA (v) = b.

Proof. (a) Assume that for each b ∈ R d m there exists a unique v ∈ R cn which
satisfies LA (v) = b; since for each b ∈ R d m the equation L (v) = b
A
d m
has a solution, it follows that Im(A) = R , which implies that m ≤ n;
furthermore, since that solution is unique, it follows that ker(A) = {0Rcn },
which implies that m ≥ n. Since we have both m ≤ n and m ≥ n, it
follows that m = n.

(b) Assume now that m = n and that the column vectors of A are linearly
independent. Since the n column vectors A;1 , A;2 , · · · , A;n of A are ele-
ments of R cn (since m = n), since they are assumed to be linearly inde-
pendent, and since R cn has dimension n, it follows (from Lecture 8) that
{A;1 , A;2 , · · · , A;n } is also a generating set for R cn , i.e. that Im(A) = R cn .
Hence, for each b ∈ R d m , there is at least one vector v ∈ R cn which satisfies
LA (v) = b. Furthermore, since the column vectors of A are assumed to
be linearly independent, it follows (from Lecture 10) that ker(A) = {0Rcn },
and therefore (again from lecture 10), for each b ∈ R d m , there is a unique

v∈R cn which satisfies LA (v) = b.

Now that, thanks to this last theorem, we have the answer to the question we
posed earlier, let us examine how we could go about solving an equation of the
form LA (v) = b, with A a square n × n real matrix.

NOTE: For simplicity, we will denote the set of all square n × n real matrices
simply by Mn (R).

cn . We wish to find v ∈ R
Let then A ∈ Mn (R), and let b ∈ R cn which satisfies

LA (v) = b.

Assume that there exist some matrix B ∈ Mn (R) such that LB (LA (v)) = v
cn , i.e., such that LB ◦ LA (v) = v for all v ∈ R
for all v ∈ R cn ; how can the
 155

knowledge of this matrix B help us solve our equation ? Well, if some vector of
cn , say w, satisfies the equation
R
LA (w) = b,
then, applying LB to both sides, it must also satisfy the equation
LB (LA (w)) = LB (b),
cn , it follows
and since we assumed B was such that LB (LA (v)) = v for all v ∈ R
that w must also satisfy
w = LB (b),
i.e. w must be equal to LB (b), i.e. the solution to the equation LA (v) = b is
given by v = LB (b); in other words, we have solved the equation!
Let us examine this idea on an example before going further:
 
4 1
• Let A ∈ M2 (R) be the square 2 × 2 matrix defined by A = , and
3 1
 
c2 be the vector defined by b = b1 c2
let b ∈ R . We wish to find v ∈ R
b2
such that LA (v) = b. Note that since the column vectors of A are linearly
independent (as can be easily verified), it follows that for any b ∈ Rc2 , the
equation LA (v) = b has a unique solution.
 Letnow B ∈ M2 (R)be the 
1 −1 x1
square 2 × 2 matrix defined by B = . Let now v =
−3 4 x2
c2 . We have:
be any vector in R
LB ◦ LA (v) = LB (LA (v))
 
4x1 + x2
= LB ( )
3x1 + x2
 
(4x1 + x2 ) − (3x1 + x2 )
=
−3(4x1 + x2 ) + 4(3x1 + x2 )
 
x1
=
x2
= v,

We have shown therefore that ∀v ∈ R c2 , LB ◦ LA (v) = v. Thanks to this,

and in line with our previous discussion, we can solve the vector equation
LA (v) = b; indeed, let v ∈ Rc2 satisfy LA (v) = b. Then, applying LB to
both sides yields the equality:
LB ◦ LA (v) = LB (b),
i.e., equivalently,
   
b1 b1 − b2
v = LB (b) = LB ( )= .
b2 −3b1 + 4b2
156 SECTION 12

Hence, thanks to the fact that we got our hands on that special matrix
B, we were able to solve our vector equation.
The natural questions one may ask at this point are:
• How did this matrix B show up ?
• How does one find such a matrix B in a systematic way ?
We will have the complete answer to these questions in the next few lectures.
Section 13

Study Topics
• Matrix Multiplication

157
158 SECTION 13

Let m, n, p be given integers ≥ 1, let A ∈ Mm,n (R) be the real m × n matrix

defined by
 
a1,1 · · · a1,n
 ..  ,
A =  ... ··· . 
am,1 · · · am,n

and let B ∈ Mn,p (R) be the real n × p matrix defined by

 
b1,1 · · · b1,p
 ..  .
B =  ... ··· . 
bn,1 ··· bn,p

Before going further, let us pay attention to the fact that A has m rows and n
columns whereas B has n rows and p columns; in particular the number
of columns of A is equal to the number of rows of B.
We know that A defines the linear mapping

cn → R
LA : R d m,

and similarly, B defines the linear mapping

cp → R
LB : R cn .

cn and since LB maps to R

Note that since LA maps from R cn , we can compose
these two mappings and obtain the mapping LA ◦ LB : Rcp → R d m , which is

defined by:

cp .
LA ◦ LB (v) = LA (LB (v)), ∀v ∈ R

What do we know about LA ◦ LB ? Well, it is the composition of two linear

mappings (namely LA and LB ) hence it is also linear. But there is more!
We will soon see that LA ◦ LB is actually equal to LC where C is a real m × p
matrix obtained from A and B ...
Let us begin by computing LA ◦ LB (v). Let then v ∈ R cp be given by
 
x1
 x2 
 
v =  . ;
 .. 
xp

then, we obtain:
 
b1,1 x1 + · · · + b1,p xp
 .. 
LB (v) =  . ,
bn,1 x1 + · · · + bn,p xp
 159

and hence
 
a1,1 (b1,1 x1 + · · · + b1,p xp ) + · · · + a1,n (bn,1 x1 + · · · + bn,p xp )
 .. 
LA ◦ LB (v) =  . 
am,1 (b1,1 x1 + · · · + b1,p xp ) + · · · + am,n (bn,1 x1 + · · · + bn,p xp )
 
(a1,1 b1,1 + · · · + a1,n bn,1 )x1 + · · · + (a1,1 b1,p + · · · a1,n bn,p )xp
 .. 
=  . 
(am,1 b1,1 + · · · + am,n bn,1 )x1 + · · · + (am,1 b1,p + · · · am,n bn,p )xp
= LC (v),

where C ∈ Mm,p (R) is the real m × p matrix given by

 
c1,1 c1,2 ··· c1,p
 c2,1 c2,2 ··· c2,p 
 
C = .. .. .. 
 . . ··· . 
cm,1 cm,2 ··· cm,p

with ci,j given by

ci,j = ai,1 b1,j + ai,2 b2,j + ai,3 b3,j + · · · + ai,n bn,j

Xn
= ai,k bk,j
k=1

for every 1 ≤ i ≤ m and 1 ≤ j ≤ p. We call C the product of A and B and

we write C = AB. NOTE: C has the same number of rows as A, and the same
number of columns as B.
Let us consider a few examples:
 
1 2 0
1. Let A be the 2 × 3 matrix given by A = , and let B be the
  3 5 1
1 0
2 × 2 matrix given by B = ; we wish to compute the product
3 5
AB of A and B. But this is not possible, since the number of columns
of A is not equal to the number of rows of B. Hence, the product AB
does not make any sense! On the other hand, the number of columns of
B is indeed equal to the number of rows of A, so it does make sense to
compute the product BA, and we obtain the 2 × 3 real matrix (with the
“·” denoting ordinary multiplication):
 
1·1+0·3 1·2+0·5 1·0+0·1
BA =
3·1+5·3 3·2+5·5 3·0+5·1
 
1 2 0
=
18 31 5
160 SECTION 13

2. Let A be the real m × n matrix given by

 
a1,1 · · · a1,n
 ..  ,
A =  ... ··· . 
am,1 · · · am,n

and let v be the real n × 1 matrix (i.e. the column vector) given by:
 
x1
 x2 
 
v =  . ;
 .. 
xn

Since the number of columns of A (namely n) is equal to the number of

rows of v, it makes sense to consider the product Av of these two matrices,
and we obtain the m × 1 real matrix:
 
a1,1 x1 + · · · + a1,n xn
 a2,1 x1 + · · · + a2,n xn 
 
Av =  .. 
 . 
am,1 x1 + · · · + am,n xn

which is nothing other than LA (v) !!!! This is why we shall often simply
write Av instead of LA (v) ...

3. Let now A ∈ Mn (R) be the real n × n (square) matrix given by

 
1 0 0 ··· 0
 0 1 0 ··· 0 
 
 1 ··· 0 
A= 0 0 ,
 .. .. .. .. 
 . . . ··· . 
0 0 0 ··· 1

(i.e. 1’s on the diagonal, and 0’s everywhere else), and let B ∈ Mn,p (R)
be a real n × p matrix. Since the number of columns of A (namely n) is
equal to the number of rows of B, it makes sense to consider the matrix
product AB. AB has n rows and p columns, just like B. Furthermore, a
simple computation shows that AB = B for any B ∈ Mn,p (R). For this
reason, (as we have seen before), this particular matrix A is called the
n × n identity matrix.

4. Let now A ∈ M3,2 (R) be the real 3 × 2 matrix given by

 
1 1
A =  0 2 ,
−1 1
 161

and B ∈ M2,3 (R) be the real 2 × 3 matrix given by

 
1 3 0
B= ;
2 1 1

since the number of columns of A (namely 2) is equal to the number of

rows of B, it makes sense to consider the matrix product AB, and we
obtain after a simple computation that AB is the 3 × 3 real matrix given
by:
 
3 4 1
AB =  4 2 2 .
1 −2 1

Note that since in this particular case the number of columns of B (namely
3) is equal to the number of rows of A, it also makes sense to compute
the matrix product BA, and we obtain after a simple computation that
BA is the 2 × 2 real matrix given by:
 
1 7
BA = .
1 5

Note that in this particular example where both AB and BA made sense,
AB and BA ended up being of different types: AB is 3 × 3 whereas BA is
2 × 2. The next example shows that even when AB and BA are of the
same type, they are not necessarily equal ...
5. Let now A and B both be real 2 × 2 matrices (so both matrix products
AB and BA make sense in this case) given by:
   
0 1 0 0
A= , B= .
0 0 1 0

We obtain for AB:

 
1 0
AB = ,
0 0

whereas
 
0 0
BA = ,
0 1

which shows that AB is not equal to BA. So keep in mind that ma-
trix multiplication is NOT like ordinary multiplication of real
numbers! (Another way to put this is that matrix multiplication is not
commutative).
6. Let A ∈ Mm,n (R) be a real m × n matrix, and let B ∈ Mn,p (R) be
a real n × p matrix. Denote the p column vectors of B by (in order)
162 SECTION 13

B;1 , B;2 , · · · , B;p , i.e. we can write B = (B;1 B;2 · · · B;p ). It is easy to
verify that the matrix product AB has columns vectors given (in order)
by AB;1 , AB;2 , · · · , AB;p , i.e., we can write AB = (AB;1 AB;2 · · · AB;p ).
This is often useful in proofs ...

Now that we have seen the “mechanics” of matrix multiplication, let us examine
some of its properties:

Theorem 29. Matrix multiplication satisfies the following properties:

(i) Let m, n, p, q be integers ≥ 1. ∀A ∈ Mm,n (R), ∀B ∈ Mn,p (R), and ∀C ∈

Mp,q (R), we have:

(AB)C = A(BC),

i.e. matrix multiplication is associative.

(ii) Let m, n, p be integers ≥ 1. ∀A ∈ Mm,n (R), ∀B, C ∈ Mn,p (R), we have:

A(B + C) = AB + AC

(iii) Let m, n, p be integers ≥ 1. ∀A, B ∈ Mm,n (R), ∀C ∈ Mn,p (R), we have:

(A + B)C = AC + BC

(iv) Let m, n, p be integers ≥ 1. ∀A ∈ Mm,n (R), ∀B ∈ Mn,p (R), ∀λ ∈ R, we

have:

A(λB) = (λA)B = λ(AB).

NOTE: What statement (i) of the theorem says is that we can first multiply A
and B to obtain AB, and then multiply this with C to obtain (AB)C; we can
also first multiply B and C to obtain BC and then multiply A with BC to obtain
A(BC). Both give the same result. Hence, we can remove the parentheses and
simply write ABC instead of either (AB)C or A(BC).

We shall only prove (i); Properties (ii),(iii),(iv) can be directly verified by com-
putation and are left as an exercise to the reader.

Proof. Recall that the matrix product AB is defined by LAB = LA ◦ LB ; sim-

ilarly, the matrix product (AB)C is defined by L(AB)C = LAB ◦ LC . Hence,
L(AB)C = (LA ◦LB )◦LC , and since composition of functions is associative,
we have

L(AB)C = (LA ◦ LB ) ◦ LC = LA ◦ (LB ◦ LC ) = LA ◦ LBC = LA(BC) ,

which implies (AB)C = A(BC), and this proves (i).

 163

Let us now see on some simple examples how we can apply matrix opera-
tions to the solution of systems
of linear equations:
1. Consider the system of linear equations given by

2x + 3y = 5
3x + y = 7,

where we wish to solve for the pair (x, y) of real numbers which satisfies
that system, if such a pair does exist. We can write this equation in matrix
form as:
    
2 3 x 5
= ,
3 1 y 7
 
2 3
and since the column vectors of the 2 × 2 matrix are linearly
3 1
independent we know that this  system hasa unique solution. Consider
− 17 3
7
now the 2 × 2 matrix given by 3 . It is easy to verify that
7 − 72
 1 3
   
−7 7 2 3 1 0
3 = ,
7 − 72 3 1 0 1
which is nothing other than the 2 × 2 identity matrix. Hence, multiplying
both sides of the equation
    
2 3 x 5
= ,
3 1 y 7
 1 3

−7 7
by the matrix 3 , we obtain
7 − 72
 1 3
    1 3
 
−7 7 2 3 x −7 7 5
3 = ,
7 − 72 3 1 y 3
7 − 27 7
i.e., equivalently,
    16

1 0 x 7
= 1 ,
0 1 y 7

and since
    
1 0 x x
= ,
0 1 y y
we finally obtain the solution to our system of equations as
   16 
x 7
= 1 .
y 7
164 SECTION 13

2. More generally, consider the system of linear equations (in n equations

and n unknowns) given by

a1,1 x1 + · · · a1,n xn = b1
a2,1 x1 + · · · a2,n xn = b2
..
.
an,1 x1 + · · · an,n xn = bn

where we wish to solve for the n−tuples (x1 , x2 , · · · , xn ) of real numbers.

We can write this system in matrix form as:

Av = b,

where A ∈ Mn (R) is the square n × n matrix given by

 
a1,1 a1,2 · · · a1,n
 a2,1 a2,2 · · · a2,n 
 
A= . .. ..  ,
 .. . ··· . 
an,1 an,2 ··· an,n

b ∈ Mn,1 (R) is the n × 1 matrix (i.e. column vector) given by

 
b1
 b2 
 
b= . ,
 .. 
bn

and, finally,
 
x1
 x2 
 
v= .. ,
 . 
xn

is the n × 1 matrix (i.e. column vector) we wish to solve for. Assume

there exists a matrix B ∈ Mn (R) such that the matrix product of B and
A, i.e. BA, is equal to the n × n identity matrix. Then, multiplying (to
the left) both sides of the equality

Av = b,

by B yields

BAv = Bb,
 165

and since BA is the n × n identity matrix, we obtain

v = Bb,
i.e. we have the solution to our original system of equations. In the next
lectures, we will see when such a matrix B does exist and how to compute
it.

Applications of Matrix Multiplication

The notion of matrix multiplication (together with other linear algebraic notions
we have studied so far) can lead to interesting engineering applications. We
present two of these in what follows.
• Message Scrambling
Suppose we wish to communicate a message to another party, but in such
a way that it could not be easily decoded by a third party. Assume also
that the message is to be sent as a string of integers greater than or equal
to 0 (in some large enough range, say 0 to 1000), arranged in matrix form.
The simplest way to proceed would be, say, to assign the integer 1 to the
letter “a”, the integer 2 to the letter “b”, and so on, until the integer 26,
assigned to the letter “z” (assume for simplicity that we care only about
lower case letters, and no other symbols other than the space symbol and
the period – it is of course easy to see how these ideas can be generalized).
Assume also that the integer 0 would be assigned to the space symbol,
and the integer 27 to the period “.”. We could then transmit the sentence
“hello world.” as the 1 × 12 matrix

8 5 12 12 15 0 23 15 18 12 4 27 ,

and we could similarly transmit longer messages (the corresponding matrix

will of course have the same number of columns as there are characters in
the message to be sent). What is the problem with this approach ? The
main problem is that the code can be easily broken. Indeed, the statistics
regarding the frequency of occurence of individual letters in the English
alphabet are well known (e.g. the letter “e” occurs the most often), and
from these statistics it would be possible (assuming the message is long
enough) to find out which integer was assigned to which letter. Note
that even a slightly less straightforward scheme, such as asssigning 100
to “a”, 22 to “b”, 33 to “c”, and so on, would still suffer from the same
problem: The correspondence between the integers and the letters (i.e.
the “code”) could be recovered by examining the frequency of occurence
of the individual integers.
 
4 3
Is there any way around this ? Well, consider the matrices A = ,
  1 1
1 −3
and B = . Note that all the entries of A are integers greater
−1 4
166 SECTION 13

than or equal to 0. Note also that the product AB is the 2 × 2 identity

matrix. These properties will be essential in what follows. Consider now
the following more involved way of transmitting the above message, i.e.
the matrix

8 5 12 12 15 0 23 15 18 12 4 27 ;

We first chop it up in blocks of length 2, and then multiply by matrix A on

the right; this is of course possible since A has

two rows. The first block
of length 2 is the 1 × 2 submatrix


8 5 ; multiplying it by A yields
the 1 × 2 matrix 37 29 . The
next block of length 2 in our message
above is the submatrix

12 12 ; multiplying it by A yields the 1 ×
2
matrix 60 48 . The next block of length 2 is the submatrix

15 0 ;
multiplying it by A yields the 1 × 2 matrix
60 45 . The next block
of length 2 is the submatrix 23 15 ; multiplying it by A yields the


1 × 2 matrix


107 84 . The next block of length 2 is the submatrix


18 12 ; multiplying it by A yields the 1 × 2 matrix 84 66 . The


final block of length 2 is the submatrix 4 27 ; multiplying it by A


yields the 1 × 2 matrix 43 39 . After all this, we would put together
the blocks of length 2 we have obtained as a result of multiplication by A,
and we would send our message as the following 1 × 12 matrix:

37 29 60 48 60 45 107 84 84 66 43 39 .

What we have done is “scramble” the original message; the code cannot
be easily recovered from the occurence statistics of the integers. But how
do we recover our original message ? Well, the party we are sending our
message to is aware that they need matrix B to recover the message. All
they need to do is break down the message they have received (i.e. the
1 × 12 matrix above) into blocks of length 2, and multiply each block on
the right by B. Since AB is the 2 × 2 identity matrix, the original message
can be recovered. For example, multiplying

the first block of length 2 of
the received message,
i.e. 37 29 by matrix B on the right yields the
1 × 2 matrix 8 5 . Continuing this procedure for all subsequent 1 × 2
blocks yields the original message.
There are more sophisticated variations of this approach; we could for ex-
ample scramble the message even more by arranging the original message
as the following 2 × 6 matrix:

 
8 5 12 12 15 0
;
23 15 18 12 4 27
 167

 
8 5
We would then take the first 2 × 2 block, i.e. the submatrix ,
  23 15
37 29
multiply it on the right by A, obtaining . We would then do
107 84
the same with the next 2 × 2 block, and so on. We would then arrange
all the 2 × 2 blocks thus obtained as a 2 × 6 matrix which we would send
to the other party; the receiving party would then take the consecutive
2 × 2 blocks of this 2 × 6 matrix, multiply them on the right by B, thereby
recovering the original 2 × 6 matrix containing the message. Compared
to our original approach, this would scramble the original message even
more, making it even harder to recover the correspondence between the
letters and the integers.
Note that we could also scramble the original message more by taking A
and B to be 3 × 3 (or even larger) square matrices; the key properties of
A and B are that A should have integer entries (greater than or equal to
0) and that the product AB should be equal to the identity matrix. The
requirement that A have integer entries greater than or equal to 0 will
ensure that if the original unscrambled message is encoded using integers
greater than or equal to 0, so will the scrambled message. The requirement
that the product AB be equal to the identity matrix ensures that the
original message can be recovered from the scrambled message.

• Process Control
Consider two reservoirs containing a certain liquid, and assume the level
of the liquid in the
 reservoirs
 at time n (n = 0, 1, 2, . . .) is represented by
yn 2
the vector xn = ∈ R̂ , with yn representing the level in the first
zn
reservoir at time n, and zn representing the level in the second reservoir
at time n. Assume also that from time n to time n + 1, a certain fraction
of the liquid in both reservoirs is lost due to evaporation. We also assume
that a certain fraction of the liquid in the first reservoir is transferred to
the second reservoir through some channel in going from time n to time
n + 1. Finally, we assume that a certain amount of liquid is injected in
the first reservoir at times n = 0, 1, 2, . . .. This process can be modelled
mathematically as follows:

xn+1 = Axn + Bun ,

where A is a real 2 × 2 matrix representing the evaporation ofthe  liquids

1
and the transfer from the first to the second reservoir, B = , and
0
un represents the amount of liquid injected in the first reservoir at time
n (a negative value would represent an amount of liquid taken from the
first reservoir). For example, if in going from time n to time n + 1, 25
percent of the liquid in the first reservoir and 10 percent of the liquid in
the second reservoir are lost due to evaporation, and 20 percent of the
168 SECTION 13

liquid in the first reservoir is transferred

 to
 the second reservoir, matrix A
.75 0
would be equal to the matrix , and the level of the liquid in
.2 .9
the reservoirs at times n = 0, 1, 2, . . . would be governed by the equation:
      
yn+1 .75 0 yn 1
= + un , n = 0, 1, 2, . . .
zn+1 .2 .9 zn 0
 
y0
The vector represents the level of the liquid in the reservoirs at
z0
the initial time n = 0; assuming at time n = 0 the amount u0 of liquid
is
 injected
 into the first reservoir, the above equation gives us the vector
y1
, which represents the level of the liquid in the reservoirs at time
z1
n = 1. With u1 denoting the amount of liquid injected in the first reservoir
 
y2
at time n = 1, we then obtain from the above equation the vector ,
z2
which represents the level of the liquid in the  reservoirs
 at time n = 2.
yn
Continuing in this way we obtain the vector for all n = 0, 1, 2, . . ..
zn
A natural question at this point is whether we can make the reservoirs have
a desired level of liquid at a given time by suitably choosing u0 , 
u1 , u2 ,. . ..
y0
For example, assume that the reservoirs are initially empty, i.e. =
  z0
0
; by suitably choosing u0 , u1 , u2 , can we make the level of liquid in
0  
10
the reservoirs at time n = 3 be equal to ? In other words, by
20
 
y3
suitably choosing u0 , u1 , u2 , can we make the vector x3 = be
  z3
10
equal to the vector ? To answer this question, we return to the
20
governing equation for xn , i.e.

xn+1 = Axn + Bun , n = 0, 1, 2, . . .

We have:

x1 = Ax0 + Bu0 ,

and hence

x2 = Ax1 + Bu1 = A(Ax0 + Bu0 ) + Bu1 = A2 x0 + ABu0 + Bu1 ,

and finally

x3 = Ax2 + Bu2 = A(A2 x0 + ABu0 + Bu1 ) + Bu2

= A3 x0 + A2 Bu0 + ABu1 + Bu2 .
 169

Since we have assumed that the reservoirs are initially empty, i.e. x0 is
the zero vector, we actually have:

x3 = A2 Bu0 + ABu1 + Bu2 .

The question
 is now: Can we choose u0 , u1 , u2 such that x3 be equal to the
10
vector ? A simple calculation shows that the vectors B, AB, A2 B
20
span R̂2 ; hence not  only can
 we choose u0 , u1 , u2 to ensure that x3 is
10
equal to the vector – for example choosing u0 = 0, u1 = 100
20
and u2 = −65 yields the desired outcome – we can actually, by a proper
choice of u0 , u1 , u2 , make sure that x3 is any desired vector of R̂2 . In the
language of process control, the vector xn is called the state of the system
at time n, whereas un is called the control at time n.

PROBLEMS:

1. For each of the following choices for the matrices A and B, do the following:

• Specify whether it makes sense to consider the matrix product AB

(and indicate why or why not), and if
the product AB makes sense, do compute it.
• Specify whether it makes sense to consider the matrix product BA
(and indicate why or why not), and if
the product BA makes sense, do compute it.
 

1 2
(a) A = −1 2 , B = .
0 −1
 

1 0
(b) A = −1 2 , B = .
0 1
 

0 1
(c) A = −1 2 , B = .
1 0



(d) A = −1 2 , B = 0 1 .
 

0 1
(e) A = −1 2 , B =  1 0 .
2 2
   
−1 2 1 −3
(g) A = , B= .
3 5 5 1
   
0 1 2 −3
(h) A = , B= .
0 0 0 0
170 SECTION 13

   
−1 2 1 0
(i) A = , B= .
3 5 0 1
   
−1 2 0 1
(j) A = , B= .
3 5 1 0
 
−1 2  
1 1
(k) A =  3 5 , B= .
0 1
7 4
 
−1 2  
1 0
(l) A =  3 5 , B= .
0 1
7 4
 
−1 2  
0 1
(m) A =  3 5 , B= .
1 0
7 4
   
−1 2 1 2 2
(n) A =  3 5 , B =  0 1 −3 .
7 4 −2 0 1
   
−1 2 1 0 0
(o) A =  3 5 , B= 0 2 0 .
7 4 0 0 −1
   
−1 2 0 1 0
(p) A =  3 5 , B =  1 5 7 .
7 4 0 0 1
   
−1 2 5 1 0 0
(q) A =  3 −2 5  , B =  0 1 0 .
7 0 4 0 0 1
   
−1 2 5 1 0 0
(r) A =  3 −2 5  , B =  0 0 1 .
7 0 4 0 1 0
   
−1 2 5 0 0 1
(s) A =  3 −2 5  , B =  0 1 0 .
7 0 4 1 0 0
   
−1 2 5 0 1 0
(t) A =  3 −2 5  , B =  1 0 0 .
7 0 4 0 0 1
 
−2 1  
1 2 1
(u) A =  1 2 , B= .
0 3 5
3 0
   
−2 1 1 0 0
(v) A =  1 0 , B= 0 1 0 .
3 2 0 0 1
 171

   
−1 3 0 1 0
(w) A =  0 0  , B =  1 0 0 .
1 5 0 0 1
 
−3 3  
  1
(x) A= 2 7 , B= .
1
−2 −5
   
−1 2 1
(y) A =  3 5  , B =  2 .
7 4 1
   
−5 −2 1 0 1 0
(z) A =  −3 1  , B =  1 2 0 0 .
−2 0 0 3 0 1

2. We wish to transmit a message, encoded as the following 2 × 6 matrix in a

secure way (in that matrix, as in the example in the text, the integer 1 to
the letter “a”, the integer 2 to the letter “b”, and so on, and the integer
0 is assigned to the
space symbol, and the integer 27 to the period “.” (the first half of the
message is on the first row, and the second half on the second row):

 
16 12 5 1 19 5
M= .
0 3 1 12 12 27

We wish to do so by scrambling the message before sending it, and have

the receiving party unscramble the received
 message.
 We will do the
5 4
scrambling using the 2 × 2 matrix A = ; the receiving party will
1 1
unscramble the received message
 
1 −4
and recover the original message M using matrix B = . The
−1 5
scrambling will be done by taking consecutive 2 × 2 blocks from M (note
that there are 3 such blocks in M ) and multiplying them on the right by
matrix A. The result of these
multiplications will form the scrambled message. At the receiving it, the
receiving party will take the received (scrambled) 2 × 6 matrix, break it
into consecutive 2 × 2 blocks and multiply each on the right by B, thereby
recovering the original message M .

(a) Verify directly that AB is the 2 × 2 identity matrix.

172 SECTION 13

(b) Compute the scrambled message using A (according to the scram-

bling procedure above).
(c) Recover the original message by directly unscrambling the scrambled
message using matrix B (using the unscrambling procedure detailed
above).

3. We wish to transmit the same message as above message, encoded now as

the following 1 × 12 matrix, in a secure way:

M= 16 12 5 1 19 5 0 3 1 12 12 27 .

We wish to do so by scrambling the message before sending it, and have

the receiving party unscramble the received
 message.
 We will do the
1 0 0
scrambling using the 3 × 3 matrix A =  0 3 2 ; the
0 1 1
receiving party will unscramble the received message
 
1 0 0
and recover the original message M using matrix B =  0 1 −2 .
0 −1 3
The scrambling will be done by taking consecutive 1 × 3 blocks from M
(note that there are 4 such blocks in M ) and multiplying
them on the right by matrix A. The result of these
multiplications will form the scrambled message. At the receiving it, the
receiving party will take the received (scrambled) 1 × 12 matrix, break it
into consecutive 1 × 3 blocks and multiply each on the right by B, thereby
recovering the original message M .

(a) Verify directly that AB is the 3 × 3 identity matrix.

(b) Compute the scrambled message using A (according to the scram-
bling procedure above).
(c) Recover the original message by directly unscrambling the scrambled
message using matrix B (using the unscrambling procedure detailed
above).

4. Two reservoirs containing a certain

 liquid are involved in a chemical pro-
yn
cess. With the vector xn = representing the level of the liquids
zn
 173

in the reservoirs at time n = 0, 1, 2, . . . (with yn being the level in the first

reservoir, and
zn in the second), we assume that due to various physical processes (evap-
oration, leaks from one reservoir to the other, external injection of liquid
in the reservoirs) the equation governing the level of fluids in the reservoirs
is given by

   
.5 .1 0
xn+1 = xn + un , n = 0, 1, 2, · · ·
.2 .7 1

where un denotes the amount of liquid injected in the second reservoir at

time n.

(a) Assuming
 that the reservoirs are empty at time n = 0, i.e. assuming
0
x0 = , can we suitably choose u0 to ensure that x1 will be
0
equal to
 
1
?
1
(b) Assuming
 that the reservoirs are empty at time n = 0, i.e. assuming
0
x0 = , can we suitably choose u0 , u1 to ensure that x2 will be
0
equal to
 
1
?
1
 
1
(c) Assuming that the initial fluid levels are given by x0 = , can
0
 
2
we suitably choose u0 to ensure that x1 will be equal to ?
2
 
1
(d) Assuming that the initial fluid levels are given by x0 = , can
0
 
2
we suitably choose u0 , u1 to ensure that x2 will be equal to ?
2

5. Three reservoirs containing a certain

 liquid are involved in a chemical
yn
process. With the vector xn =  zn  representing the level of the
wn
liquids in the reservoirs at time n = 0, 1, 2, . . . (with yn being the level in
the first reservoir,
174 SECTION 13

zn in the second, and wn in the third), we assume that due to various

physical processes (evaporation, leaks from one reservoir to the other,
external injection of liquid in the reservoirs) the equation governing the
level of fluids in the reservoirs is given by

   
1 1 0 1
xn+1 = 0 1 0  xn +  0  un , n = 0, 1, 2, · · ·
0 0 1 0

where un denotes the amount of liquid injected in the first reservoir at

time n.

(a) Assuming
 that
 the reservoirs are empty at time n = 0, i.e. assuming
0
x0 = , can we suitably choose u0 to ensure that x1 will be
0
equal to
 
1
 1 ?
1
(b) Assuming
 that the reservoirs are empty at time n = 0, i.e. assuming
0
x0 = , can we suitably choose u0 , u1 to ensure that x2 will be
0
equal to
 
1
 1 ?
1
(c) Assuming
 that
 the reservoirs are empty at time n = 0, i.e. assuming
0
x0 = , can we suitably choose u0 , u1 , u2 to ensure that x3 will
0
be equal to
 
1
 1 ?
1
Section 14

Study Topics
• Invertible Square Matrices
• Determinant of a Square Matrix

175
176 SECTION 14

Let A ∈ Mn (R) be the square n × n real matrix given by

 
a1,1 a1,2 · · · a1,n
 a2,1 a2,2 · · · a2,n 
 
A= . .. .
 .. . ··· ··· 
an,1 an,2 ··· an,n
We have already seen in the previous lectures what use we could make of the
existence of a real n × n matrix B such that the matrix product BA be equal to
the n × n identity matrix. Indeed (let us review this one more time!), assume
we wish to solve the system of linear equations given by
a1,1 x1 + a1,2 x2 + · · · a1,n xn = b1
a2,1 x1 + a2,2 x2 + · · · a2,n xn = b2
···
an,1 x1 + an,2 x2 + · · · an,n xn = bn

where the ai,j are the same real numbers as in the matrix A above and are
assumed given, the real numbers b1 , · · · , bn are assumed given, and we wish
to solve for the real numbers x1 , · · · , xn . We can write this system of linear
equations in matrix form as:

Av = b,

where A is the same matrix as defined above, b is the n × 1 matrix (i.e. column
vector) defined by
 
b1
 b2 
 
b =  . ,
 .. 
bn

and v is the n × 1 matrix (i.e. column vector) defined by

 
x1
 x2 
 
v= . 
 .. 
xn
and which we wish to solve for. Again, Av denotes the matrix product of
matrix A and matrix v; we have already seen that the n × 1 matrix (i.e. column
vector) Av is nothing other than LA (v). Let us see again what we can gain by
writing the original system of linear equations in matrix form:
Assuming there exists a real matrix B ∈ Mn (R) such that
BA = I,
 177

 
1 0 ··· 0
 0 1 ··· 0 
 
where I =  .. .. ..  is the n × n identity matrix, we obtain, after
 . . . 
0 0 ··· 1
multiplying on the left both sides of the equation Av = b by the matrix B:

B(Av) = Bb,

which is equivalent to (by associativity of matrix multiplication):

(BA)v = Bb,

which is equivalent to:

Iv = Bb,

which is equivalent to:

v = Bb,

since Iv = v. In other words, knowledge of the matrix B has led us to

the solution of the equation Av = b: Namely, the solution to this equation
is given by the n × 1 matrix (i.e. column vector) v = Bb.
We are naturally led to the following definition:

Definition 30. Let A ∈ Mn (R) be a real n × n matrix. A is said to be invertible

if there exists a real n × n matrix B such that AB = BA = I, where I is the n × n
identity matrix.

Remark 7. The reader may rightly wonder why we require in the above definition
that both AB and BA be equal to the n × n identity matrix I (since all we cared
about in solving our matrix equation was that BA be equal to I; we didn’t care
about AB). It is easy to show (but we won’t do it here) that AB = I implies
BA = I, and vice-versa. This is due to the fact that A and B are both square
matrices (and as an application of the rank-nullity theorem).
To recapitulate, we could have defined “A invertible” just by requiring the existence
of B ∈ Mn (R) such that BA = I; this would have been equivalent to the above
definition.

Let now A ∈ Mn (R) and assume A invertible; by definition of invertibility,

this means that there exists B ∈ Mn (R) such that AB = BA = I (I denoting
again the n × n identity matrix). A natural question at this point is: Is there
only one matrix B which satisfies this, or are there many ? The answer is
given in the following theorem.

Theorem 30. Let A ∈ Mn (R). If A is invertible, then there exists a unique

B ∈ Mn (R) such that AB = BA = I (I denoting again the n× n identity matrix).
178 SECTION 14

Proof. Assume there exist n×n matrices B, C ∈ Mn (R) such that AB = BA =

I and AC = CA = I; we have to show that B must then be equal to C. We
have:

B = BI = B(AC) = (BA)C = IC = C,

which is exactly what we wished to prove.

This leads naturally to the following definition:

Definition 31. Let A ∈ Mn (R) and assume A is invertible. The unique matrix
B ∈ Mn (R) which satisfies AB = BA = I is called the inverse of A and is
denoted A−1 .

Before going further, let us highlight a few key points:

(i) The definition of invertible matrices applies only to square matrices.

(ii) it does not make any sense to talk about invertibility of an m × n matrix
with m 6= n.

Let us now consider a few simple examples:

1. Consider the n × n identity matrix I; it is easy to see that the inverse

of I is I itself since we have II = I. In particular, I is invertible.

2. Let A be the n × n zero matrix (i.e. the n × n matrix with all entries
equal to zero); since for any B ∈ Mn (R) the matrix products BA and
AB are equal to the zero matrix, it follows that there is no B ∈ Mn (R)
for which the matrix products BA and AB be equal to the n × n identity
matrix; hence the n × n zero matrix is not invertible.
 
5 1
3. Let A be the 2 × 2 matrix given by A = ; it is easy to verify that
4 1
 
1 −1
the 2 × 2 matrix given by B = satisfies AB = BA = I (I
−4 5
being the 2 × 2 identity matrix); hence A is invertible and B is its inverse.

The following questions arise naturally at this point: Consider an n × n real

matrix A;

• How do we find out whether or not A is invertible ?

• In case A is invertible, how do we compute the inverse of A ?

The following important theorem addresses the first question:

Theorem 31. Let A ∈ Mn (R) be a real n × n matrix. We have:

• A is invertible if and only if the column vectors of A are linearly in-

dependent.
 179

Equivalently, A is invertible if and only if A has rank n.

Proof. (i) Assume first that A is invertible, and let B ∈ Mn (R) be its inverse
(i.e. AB = BA = I, where I is the n×n identity matrix). We wish to show
that this implies that the column vectors of A are linearly independent.
From what we have seen in Lecture 10, showing that the column vectors of
A are linearly independent is equivalent to showing that ker(A) = {0Rcn }
cn ); equivalently, we
(i.e. the kernel of A contains only the zero vector of R
need to show that if v ∈ ker(A), then we must have v = 0Rcn . Let then
v ∈ ker(A); then, by definition of ker(A), we have:

Av = 0Rcn ,

and multiplying both sides of the above equation by the matrix B (the
inverse of A), we obtain:

B(Av) = B0Rcn = 0Rcn ,

i.e., equivalently (by associativity of matrix multiplication),

(BA)v = 0Rcn ,

i.e.

v = 0Rcn ,

since BA = I and Iv = v. Hence, we have shown that v ∈ ker(A) implies

v = 0Rcn . This proves that ker(A) = {0Rcn }, which (by Lecture 10) implies
that the column vectors of A are linearly independent, as desired.
(ii) Assume now that the column vectors of A are linearly independent; we
need to show that A is then invertible. Since the column vectors of A are
cn → R
assumed linearly independent, it follows that the linear map LA : R cn
defined by matrix A is one-to-one (i.e. injective); furthermore, it follows
from the rank-nullity theorem that the range (i.e. image) Im(A) of matrix
cn , i.e. that the linear map LA is onto (i.e. surjective). It
A is all of R
follows that there exists a real n × n matrix B such that:

LB ◦ LA (v) = LA ◦ LB (v) = v, cn ,
∀v ∈ R

i.e.

BA = AB = I,

where I is the n × n identity matrix. Hence, A is invertible.

Before going further, let us summarize how we can apply this last ex-
tremely important theorem: Suppose you are given a real n × n matrix A;
then,
180 SECTION 14

• If the column vectors of A form a linearly independent family of

cn , then A is invertible;
R
cn ,
• If the column vectors of A form a linearly dependent family of R
then A is not invertible.
Let us now look at a few examples before going further:
 
1 0 ··· 0
 0 1 ··· 0 
 
1. Let I =  . .. ..  be the n × n identity matrix; we have already
 .. . . 
0 0 ··· 1
seen that I is invertible (and it’s inverse is I itself). Now we know why I
is invertible: It’s n column vectors are linearly independent, as can be
easily verified (and as we have seen many times earlier!).
 
1 1 2
2. Consider the 3 × 3 real matrix A =  −1 1 0 ; Is A invertible ?
2 3 5
Well, by the previous theorem, we have to see whether or not the column
vectors of A are linearly independent. Let A;1 , A;2 , A;3 denote the three
column vectors of A; it is easy to see that
A;1 + A;2 − A;3 = 0Rc3 ,

which shows that the column vectors of A are not linearly independent;
hence, by the previous theorem, A is not invertible.
 
1 2 2
3. Consider the 3 × 3 real matrix A =  0 1 0 ; Is A invertible ?
2 3 5
Again, by the previous theorem, we have to see whether or not the column
vectors of A are linearly independent. Let A;1 , A;2 , A;3 denote the three
column vectors of A; it is easy to verify that {A;1 , A;2 , A;3 } is a linearly
independent subset of R c3 ; hence, it follows from the previous theorem
that A is invertible.
We now introduce an extremely important function of matrices, the deter-
minant. Before, however, we need to introduce some notation:
Definition 32. Let A ∈ Mn (R) be a square n × n matrix. Let i, j be integers ≥ 1
and ≤ n. We denote by [A]i,j the (n − 1) × (n − 1) matrix obtained from A by
deleting row i and column j.
Let us see consider a few examples:
 
2 1
1. Consider the real 2 × 2 matrix A = ; we have:
3 4

[A]1,1 = (4), [A]1,2 = (3), [A]2,1 = (1), [A]2,2 = (2).

 181

 
5 1 0
2. Consider the real 3 × 3 matrix A =  −1 2 3 ; we have:
7 4 1
     
2 3 −1 3 −1 2
[A]1,1 = , [A]1,2 = , [A]1,3 = ,
4 1 7 1 7 4
     
1 0 5 0 5 1
[A]2,1 = , [A]2,2 = , [A]2,3 = ,
4 1 7 1 7 4
     
1 0 5 0 5 1
[A]3,1 = , [A]3,2 = , [A]3,3 = .
2 3 −1 3 −1 2

We are now ready to define the determinant of a square matrix:

Definition 33. Let A ∈ Mn (R) be a square n×n real matrix. The determinant
det(A) of A is the real number defined as follows:
(i) If n = 1, i.e. A = (a) for some real number a, then det(A) = a;
(ii) If n > 1, then det(A) is recursively defined as follows:
n
X
det(A) = (−1)1+j a1,j det([A]1,j ).
j=1

Let us again consider some examples:

 
a b
1. Let A = be a real 2 × 2 matrix. We have:
c d
det(A) = (−1)1+1 a det([A]1,1 ) + (−1)1+2 b det([A]1,2 )
= (−1)2 a det([A]1,1 ) + (−1)3 b det([A]1,2 )
= a det((d)) − b det((c))
= ad − bc.
You should learn this identity by heart:
 
a b
det( ) = ad − bc.
c d
 
a b c
2. Let now A =  d e f  be a real 3 × 3 matrix. We have:
g h i
det(A) = (−1)1+1 a det([A]1,1 ) + (−1)1+2 b det([A]1,2 ) + (−1)1+3 c det([A]1,3 )
= a det([A]1,1 ) − b det([A]1,2 ) + c det([A]1,3 )
     
e f d f d e
= a det( ) − b det( ) + c det( )
h i g i g h
= a(ei − f h) − b(di − f g) + c(dh − eg).
(You should definitely not try to memorize this last identity!).
182 SECTION 14

We state without proof the following extremely important result:

Theorem 32. Let A ∈ Mn (R) be a real n × n matrix. We have:
• A is invertible if and only if det(A) 6= 0.
Let us recapitulate: We now have two distinct ways to establish whether or
not a given n × n matrix A is invertible:
1. Consider the column vectors A;1 , A;2 , · · · , A;n of A: If they are linearly
independent then A is invertible; if they are linearly dependent, then
A is not invertible.
2. Compute det(A): If det(A) 6= 0, then A is invertible; if det(A) = 0, then
A is not invertible.
The reader may wonder: Why not always use the first scheme to verify invert-
ibility of a given n × n matrix ? Or why not always use the second scheme for
that purpose ? What is the use of having these two schemes (which at the end
give the same result) ? Well, in some cases, the first scheme may be easier to
apply, and in other cases, the second. Let us illustrate this on two examples:
 
−1 2 1 −4
 0 3 3 7 
1. Let A =  
 2 3 5 −3 ; The question is: is A invertible ? To see
1 1 2 3
whether or not A is invertible, we could either:
• Check whether or not the column vectors of A are linearly inde-
pendent, or
• compute the determinant det(A) of A and check whether or not it
is distinct from zero.
Well, we can immediately see that the third column vector of A is
the sum of the first two; this shows that the column vectors of A are
linearly dependent, and hence A is not invertible. We could have also
computed det(A) and found it to be zero, but this would have been mildly
tedious.
 
1 0 0 0
 2 0 0 1 
2. Let now A =  
 0 1 2 0 ; The question is: is A invertible ?
0 1 0 1
Here again, to answer this question, we could either check linear depen-
dence/independence of the column vectors of A or we could compute the
determinant det(A) of A. It happens that in this example computing
det(A) is extremely easy, due to so many entries of A being zero. Apply-
ing the definition of the determinant, we obtain:
 
0 0 1  
  1 2
det(A) = det( 1 2 0 ) = det( ) = −2,
1 0
1 0 1
 183

and hence det(A) 6= 0, which shows that A is invertible.

Now that we have seen how to use the determinant function to verify whether
or not a given real n × n matrix A is invertible, let us see how we can compute
the inverse A−1 of A. Before going further, we need an important definition:

Definition 34. Let M ∈ Mm,n (R) be a real m × n matrix. The transpose of

M , denoted M T , is the n × m real matrix defined by: (M T )i,j = (M )j,i , for all
1 ≤ i ≤ n and 1 ≤ j ≤ m, where by the notation (C)a,b we mean the entry of
matrix C on row a and column b.

In other words, the entry of M T on row i and column j is exactly the entry of
M on row j and column i. A very simple way to remember how to construct
the transpose AT of A is as follows: The first row of A becomes the first column
of AT , the second row of A becomes the second column of AT , ..., the last row
of A becomes the last column of AT .
Let us illustrate this on a few examples:
 
1 2 −3 7
1. Let A = be a real 2 × 4 matrix; its transpose AT is
−1 4 5 0
the real 4 × 2 matrix given by
 
1 −1
 2 4 
AT =  −3
.
5 
7 0

 
7 3 −3
2. Let A =  −1 0 5  be a real 3 × 3 matrix; its transpose AT is the
2 2 1
real 3 × 3 matrix given by
 
7 −1 2
AT =  3 0 2 
−3 5 1

 
1
3. Let A = be a real 2 × 1 matrix; its transpose AT is the real 1 × 2
2
matrix given by


AT = 1 2 .

Remark 8. Clearly, if A is a square matrix, then its transpose AT is again a square

matrix of the same size; i.e. if A ∈ Mn (R) then AT ∈ Mn (R).

We are now ready to compute the inverse A−1 of an invertible matrix A ∈

Mn (R):
184 SECTION 14

Theorem 33. Let A ∈ Mn (R) be a real n × n matrix. Assume A is invertible.

Then, the inverse A−1 of A is given by:
1
A−1 = CT ,
det(A)
where the matrix C ∈ Mn (R) is defined as follows: The entry (C)i,j of C on row
i and column j is given by:

(C)i,j = (−1)i+j det([A]i,j ),

and where (recall!) [A]i,j is the (n − 1) × (n − 1) matrix obtained from A by

removing row i and column j.
Let us apply this result to compute the inverse of an invertible 2 × 2 matrix:
Let then
 
a b
A=
c d

be an invertible 2 × 2 matrix, i.e. such that det(A) = 6 0, i.e. such that

ad − bc 6= 0. Applying the previous result to compute the inverse A−1 of A, we
obtain:
 
1 d −b
A−1 = .
ad − bc −c a

Let us verify that we indeed have A−1 A = AA−1 = I (I being the 2 × 2 identity
matrix); we have:
    
1 d −b a b 1 ad − bc 0
A−1 A = =
ad − bc −c a c d ad − bc 0 ad − bc
 
1 0
= ,
0 1
      
a b 1 d −b 1 a b d −b
AA−1 = =
c d ad − bc −c a ad − bc c d −c a
 
1 ad − bc 0
=
ad − bc 0 ad − bc
 
1 0
= .
0 1

We end this lecture by stating (without proof) a number of extremely impor-

tant properties of the determinant function:
Theorem 34. Let A ∈ Mn (R) be a real n × n matrix. We have:

det(A) = det(AT ).

In other words, the determinant of a matrix is equal to the determinant of its

transpose.
 185

Theorem 35. Let A, B ∈ Mn (R) be real n × n matrices. We have:

det(AB) = det(A) det(B).

In other words, the determinant of a product of matrices is the product of the

invidual determinants.

It follows from this last theorem that if A ∈ Mn (R) is invertible, then det(A−1 ) =
1
det(A) , i.e. the determinant of the inverse is the inverse of the determinant. In-
deed, since we have AA−1 = I (where I is the n × n identity matrix), we obtain
det(AA−1 ) = det(A) det(A−1 ) = det(I), and since we have already computed
that det(I) = 1, we obtain det(A) det(A−1 ) = 1, from which it follows that
1
det(A−1 ) = det(A) .

Theorem 36. Let A ∈ Mn (R). We have, ∀i ∈ {1, 2, · · · , n}:

n
X
det(A) = (−1)i+j ai,j det([A]i,j ),
j=1

and ∀j ∈ {1, 2, · · · , n}:

n
X
det(A) = (−1)i+j ai,j det([A]i,j ),
i=1

where ai,j denotes the entry of A on row i and column j, and (as before) [A]i,j
denotes the (n− 1)× (n− 1) matrix obtained from A by removing row i and column
j.

Remark 9. Note that the definition we gave for det(A) corresponds to the first
formula above for i = 1, i.e. “expansion along the first row of A”. What the first
formula in the above theorem says is that we can compute det(A) by “expanding
along any row of A” (not just the first one), and the second formula says that we
can compute det(A) by “expanding along any column of A”.

The previous result together with the previous remark lead us to the following
observation: Since we can compute det(A) by expanding along any row or col-
umn of A that we please, we may as well choose the row or column of A having
the most number of zero entries. We illustrate this idea on a few examples:

1. Let A ∈ Mn (R) be given by

 
a1,1 0 ··· 0
 a2,1 a2,2 ··· 0 
 
A= . .. .. .. ;
 .. . . . 
an,1 an,2 ··· an,n
186 SECTION 14

such a matrix is called lower triangular (since all the entries above the
diagonal are zero). Let us compute det(A). In line with the preceding
observation, it is to our advantage to expand along the row or column of
A having the most number of zero entries. Let us then compute det(A)
by expanding along the first row of A. Since the only possibly non-zero
entry on the first row of A is a1,1 , we obtain, upon applying the formula
for the determinant:
 
a2,2 0 ··· 0
 a3,2 a3,3 · · · 0 
 
det(A) = a1,1 det  . . .. ..  ,
 .. .. . . 
an,2 an,3 ··· an,n
and again computing this new determinant by row expansion along the
first row yields:
 
a3,3 0 ··· 0
 a4,3 a4,4 ··· 0 
 
det(A) = a1,1 a2,2 det  . .. .. .. ,
 .. . . . 
an,3 an,4 ··· an,n
and iterating this procedure yields:
det(A) = a1,1 a2,2 a3,3 · · · an,n ,
i.e., in other words, the determinant of a lower triangular matrix A is
the product of the diagonal entries of A.
2. Let now A ∈ Mn (R) be given by
 
a1,1 a1,2 ··· a1,n
 0 a2,2 ··· a2,n 
 
A= . .. .. .. ;
 . . . . . 
0 0 ··· an,n
such a matrix is called upper triangular (since all the entries below the
diagonal are zero). Note that for this particular matrix, it would be most
advantageous to compute the determinant by expanding along the first
column. Note also that we can use our previous computation for lower
triangular matrices to compute det(A). Indeed, note that the transpose
AT of matrix A is lower-triangular, and hence det(AT ) is the product of
the diagonal entries of AT . We have also seen that transposing a matrix
does not change its determinant, i.e. det(AT ) = det(A). Putting all this
together, we obtain:
 
a1,1 0 ··· 0
 a1,2 a2,2 · · · 0 
 
det(A) = det(AT ) = det  . . . ..  = a1,1 a2,2 a3,3 · · · an,n ,
 .. .. .. . 
a1,n a2,n · · · an,n
 187

i.e., in other words, the determinant of an upper triangular matrix A is

also the product of the diagonal entries of A.

3. Let now A ∈ Mn (R) be given by

 
a1,1 0 ··· 0
 0 a 2,2 ··· 0 
 
A= .. .. ;
 0 0 . . 
0 0 ··· an,n

such a matrix is called diagonal (since all the entries outside the diag-
onal are zero). Note that a diagonal matrix is both upper- and lower-
triangular. Hence, applying our previous results, we obtain:

det(A) = a1,1 a2,2 a3,3 · · · an,n ,

i.e., in other words, the determinant of an diagonal matrix A is also the

product of the diagonal entries of A.

Let now A ∈ Mn (R) be a real n × n matrix, and let us write A as

A = A;1 · · · A;j · · · A;n ,

where A;j denotes the j th column vector of A. If we now multiply the j th column
vector of A by some real number α (but only the j th column vector of A), we
obtain the matrix


A;1 · · · αA;j · · · A;n ,

which has exactly the same column vectors as A except for the j th one (which
has been multiplied by α. The following theorem indicates how the determinants
of those matrices are related:

Theorem 37. For any α ∈ R, we have:

det( A;1 · · · αA;j · · · A;n ) = α det( A;1 ··· A;j ··· A;n ).

Let now A ∈ Mn (R) be a real n × n matrix, and let α ∈ R; recall that αA is the
matrix obtained by multiplying each entry of A by α. How is det(αA) related
to det(A) ? The following theorem answers this question:

Theorem 38. ∀A ∈ Mn (R), ∀α ∈ R, we have:

det(αA) = αn det(A).

Proof. Writing A in terms of its column vectors as

A= A;1 ··· A;j ··· A;n ,
188 SECTION 14

we obtain:


αA = αA;1 ··· αA;j ··· αA;n ,

and it follows from the previous theorem that

det(αA) = det( αA;1 ··· αA;j ··· αA;n )


= αn det( A;1 ··· A;j ··· A;n )
= αn det(A).

Remark 10. Note that it follows from the previous theorem that for any A ∈
Mn (R), we have:

det(−A) = (−1)n det(A).

Let now A ∈ Mn (R) be a real n × n matrix, which we write again in terms of

its column vectors as


A = A;1 · · · A;i · · · A;j · · · A;n ,

where again A;i denotes the ith column vector of A and A;j its j th column
vector; what happens to the determinant if we interchange these two column
vectors, i.e. how is the determinant of the matrix


A;1 · · · A;j · · · A;i · · · A;n ,

(which is obtained from A by interchanging column vectors i and j) related to

the determinant of the matrix


A = A;1 · · · A;i · · · A;j · · · A;n ?

The answer is given in the following theorem:

Theorem 39. Let A ∈ Mn (R) be a real n × n matrix given (in terms of its column
vectors) by


A = A;1 · · · A;i · · · A;j · · · A;n ,

and let à be the real n× n matrix obtained from A by interchanging column vectors
i and j (where i 6= j), i.e.,


à = A;1 · · · A;j · · · A;i · · · A;n .

Then:

det(Ã) = − det(A);

in other words, interchanging two distinct column vectors changes the sign of the
determinant.
 189

Let now A ∈ Mn (R) be a real n × n matrix given (again in terms of its column
vectors) by


A = A;1 · · · A;j · · · A;n ,

cn be a column vector; let us add v to the j th column vector of A

and let v ∈ R
(and only to that column vector); we obtain the n × n matrix


A;1 · · · A;j + v · · · A;n

which differs from A only in the j th column vector. How is the determinant of
this last matrix related to the determinant of A ?

Theorem 40. Let A ∈ Mn (R) be a real n × n matrix given in terms of its column
vectors by


A = A;1 · · · A;j · · · A;n ,

and let v ∈ Rcn be a column vector; consider the matrix obtained from A by adding
th
v to the j column vector of A, i.e. the matrix


A;1 · · · A;j + v · · · A;n ,

and let


A;1 ··· v ··· A;n ,

be the matrix obtained by replacing the j th column vector of A by v (and keeping

all other column vectors the same as those of A). We have:



det( A;1 · · · A;j + v · · · A;n ) = det( A;1 · · · A;j · · · A;n )


+ det( A;1 · · · v · · · A;n ).

PROBLEMS:

1. For each of the following choices for the matrix A, establish whether or
not A is invertible in the following two distinct ways:

(i) Check whether or not the column vectors of A are linearly indepen-
dent.
(ii) Compute the determinant det(A) of A to see whether or not it is
non-zero.
 
−1 2
(a) A = .
2 1
190 SECTION 14

 
1 2
(b) A = .
2 0
 
1 2
(c) A = .
0 2
 
1 0
(d) A = .
0 1
 
0 1
(e) A = .
1 0
 
0 1
(f) A = .
0 1
 
1 1
(g) A = .
0 0
 
1 1
(h) A = .
0 1
 
1 1
(i) A = .
1 1
 
3 2
(j) A = .
6 4
 
−2 0
(k) A = .
0 −2
 
−3 0
(l) A = .
7 0
 
−1 2
(m) A = .
2 −4
 
1 0 3
(n) A =  2 1 2 .
3 3 3
 
3 2 1
(o) A =  2 0 1 .
0 0 1
 
1 2 1
(p) A =  2 4 2 .
3 5 2
 
1 1 0
(q) A =  2 7 0 .
3 −3 0
 
1 0 0
(r) A =  1 1 0 .
1 1 1
 191

 
1 0 0
(s) A =  0 1 0 .
0 0 1
 
1 1 1
(t) A =  0 1 1 .
0 0 1
 
1 1 1
(u) A =  0 1 1 .
1 0 1
 
2 1 3
(v) A =  7 −1 −1 .
5 −2 2
 
0 1 0
(w) A =  −1 2 3 .
−1 5 4
 
1 0 0 0
 1 −1 0 0 
(x) A=  1
.
2 −3 0 
1 5 −7 −2
 
1 0 0 −1
 1 −1 1 2 
(y) A=  0 −2 2
.
3 
1 0 0 −1
 
2 0 1 0
 0 −2 5 0 
(z) A=  0
.
1 0 2 
1 1 −1 −1

2. The following problems involve the design of scrambling and unscrambling

matrices for secure message communication (see Section 13 for more details
on this topic; the example matrices given there should not be used). In the
problems that follow, the matrices of similar dimension should be distinct.

(a) Design a 2 × 2 scrambling matrix A and a corresponding 2 × 2 un-

scrambling matrix B, and use A to scramble the following message

(by considering consecutive 1×2 blocks): 0 1 15 2 4 4 7 20 .
(b) Design a 2 × 2 scrambling matrix A and a corresponding 2 × 2 un-
scrambling matrix B, and use A to scramble  the following message

0 1 15 2
(by considering consecutive 2 × 2 blocks): .
4 4 7 20
(c) Design a 3 × 3 scrambling matrix A and a corresponding 3 × 3 un-
scrambling matrix B, and use A to scramble the following message


(by considering consecutive 1 × 3 blocks): 7 2 9 18 13 23 .
192 SECTION 14

(d) Design a 3 × 3 scrambling matrix A and a corresponding 3 × 3 un-

scrambling matrix B, and use A  to scramble the
 following message
7 2 9
(by considering a 2 × 3 block): .
18 13 23
(e) Design a 4 × 4 scrambling matrix A and a corresponding 4 × 4 un-
scrambling matrix B, and use A to scramble the following message

(by considering consecutive 1×4 blocks): 0 1 15 2 4 4 7 20 .
(f) Design a 4 × 4 scrambling matrix A and a corresponding 4 × 4 un-
scrambling matrix B, and use A  to scramble thefollowing message
0 1 15 2
(by considering a 2 × 4 block): .
4 4 7 20
Section 15

Study Topics
• Eigenvalues and eigenvectors of an endomorphism
• Eigenvalues and eigenvectors of a square matrix

193
194 SECTION 15

Let V be a real vector space, and let L : V → V be a linear transformation

(note here that L maps from V back to V and not to some other vector space;
a linear mapping from a vector space back to itself is also called an endomor-
phism). We have the following important definition:
Definition 35. Let v ∈ V with v 6= 0 (i.e. v is not the zero vector of V); v is said
to be an eigenvector of the linear transformation L if there exists a real number
λ such that:
L(v) = λv.
The real number λ in the above relation is called an eigenvalue of L.
We then say that v is an eigenvector of L associated to the eigenvalue λ.
Let us note the following key points before going further:
1. L is a mapping from V to V, and not from V to some other vector space
W.
2. An eigenvector a of linear transformation L : V → V is by definition a
vector other than the zero vector 0 of V; hence, even though L(0) = 0 =
λ0, 0 is not considered to be an eigenvector of L.
3. For a real number λ ∈ R to be an eigenvalue of L, there must exist a
non-zero vector v ∈ V such that L(v) = λv; if no such vector exists, λ
cannot be an eigenvalue of L.
Let us now consider a number of examples:
1. Let V be a real vector space, and let α ∈ R be a real number. Let
L : V → V be the mapping defined by L(v) = αv, ∀v ∈ V. It is easy to
verify that L is a linear mapping. Furthermore (by definition of L), for
any v ∈ V with v 6= 0, we have:
L(v) = αv,
which shows that v is an eigenvector of L associated to the eigenvalue
α. Hence, for the linear mapping L of this example, any non-zero vector
in V is an eigenvector of L (associated to the eigenvalue α).
2. Let C ∞ (R; R) denote the real vector space of all functions f : R → R
which are differentiable of any order (i.e. f can be differentiated as many
times as we want). Let L : C ∞ (R; R) → C ∞ (R; R) be the mapping defined
by:
L(f ) = f ′ ,
where f ′ denotes the first derivative of f . It is easy to verify that L is
a linear mapping. Let us try to find an eigenvector of L. Let α ∈ R be
a real number, and let f ∈ C ∞ (R; R) be defined by:
f (t) = eαt , ∀t ∈ R.
 195

Let us compute L(f ). We have, ∀t ∈ R:

d αt
(L(f ))(t) = (e ) = αeαt = αf (t) = (αf )(t),
dt
which shows that

L(f ) = αf.

Since f is not the zero vector of C ∞ (R; R), and since L(f ) = αf , we
conclude that f is an eigenvector of L associated to the eigenvalue α.

3. Let L : R2 → R2 be defined by L((x, y)) = (2x, 3y) for all (x, y) ∈ R2 . It is

easy to verify that L is linear. Let now v1 ∈ R2 be defined by v1 = (1, 0).
Clearly v1 is not equal to the zero vector (0, 0) of R2 . Furthermore,

L(v1 ) = L((1, 0)) = (2, 0) = 2(1, 0) = 2v1 ,

which shows that v1 is an eigenvector of L associated to the eigenvalue

2.
Let now v2 ∈ R2 be defined by v2 = (0, 1). Clearly v2 is not equal to the
zero vector (0, 0) of R2 . Furthermore,

L(v2 ) = L((0, 1)) = (0, 3) = 3(0, 1) = 3v2 ,

which shows that v2 is an eigenvector of L associated to the eigenvalue

3.

Let now A ∈ Mn (R) be a real n × n matrix (note that A is a square matrix),

and consider the linear mapping LA : R cn → R cn defined by A; Note that LA
cn
maps from R back to itself (this is why we chose A to be a square matrix!),
and as a result, we can try to find eigenvectors and eigenvalues of LA .

• NOTE: We will just say “eigenvalue (resp. eigenvector) of A” instead of

“eigenvalue (resp. eigenvector) of LA ”.

Let then λ ∈ R; Assume λ is an eigenvalue of A. This means that there must

cn such that
exist a non-zero vector v ∈ R

LA (v) = λv,

i.e. (using matrix notation) such that

Av = λv.

This in turn implies that

λv − Av = 0Rcn ,
196 SECTION 15

i.e., that
(λI − A)v = 0Rcn ,
where I denotes the n × n identity matrix. But then, this last equality means
nothing other than that v is in the kernel ker(λI − A) of the matrix λI − A.
Since v is assumed to be distinct from the zero vector of R cn and since it lies
in the kernel ker(λI − A) of λI − A, it follows that ker(λI − A) must contain
more than just the zero vector of R cn ! But then this means that λI − A is not
invertible (otherwise ker(λI − A) would have contained only the zero vector of
cn ), i.e. that det(λI − A) = 0 !
R
Conversely, assume the real number λ ∈ R is such that det(λI −A) = 0; this then
implies that the kernel ker(λI − A) of the matrix λI − A is not equal to {0Rcn },
i.e. there does exist a vector v 6= 0Rcn in ker(λI − A); in other words, we have
both v 6= 0Rcn and (λI − A)v = 0Rcn , and this last equality implies Av = λv.
To recapitulate: We have shown that if λ ∈ R is such that det(λI − A) = 0,
then there exists a v ∈ Rcn such that v 6= 0 cn and Av = λv; this shows that λ
R
is an eigenvalue of A.
We have therefore proved the following important theorem:
Theorem 41. Let A ∈ Mn (R) be a real n × n matrix. Let λ ∈ R. We have:
• λ is an eigenvalue of A if and only if det(λI − A) = 0 (where again I
denotes the n × n identity matrix).
The importance of this theorem lies in that it will provide us a systematic way
to compute the eigenvalues of any n × n matrix. This theorem has the following
useful corollary:
Theorem 42. Let A ∈ Mn (R) be a real n × n matrix. Then 0 is an eigenvalue of
A if and only if A is not invertible.
Proof. 0 being an eigenvalue of A is equivalent (by the last theorem) to det(0I −
A) = 0, i.e. to det(−A) = 0, i.e. to (−1)n det(A) = 0, i.e to det(A) = 0,
which itself is equivalent to A being not invertible (by what we saw in the last
lecture).
Let now A ∈ Mn (R) be given by
 
a1,1 a1,2 ··· a1,n
 a2,1 a2,2 ··· a2,n 
 
A= . .. .. .
 .. . ··· . 
an,1 an,2 ··· an,n
We can then write, ∀λ ∈ R:
 
λ − a1,1 −a1,2 ··· −a1,n
 −a2,1 λ − a2,2 ··· −a2,n 
 
det(λI − A) = det  .. .. .. .
 . . ··· . 
−an,1 −an,2 ··· λ − an,n
 197

An explicit computation of this last determinant yields that det(λI − A) has

the form:

det(λI − A) = λn + cn−1 λn−1 + cn−2 λn−2 + · · · + c2 λ2 + c1 λ + c0 ,

where the real numbers c0 , c1 , · · · , cn−2 , cn−1 are determined by the entries of
matrix A (i.e. by the ai,j ).
It is important to note the following key points:

1. For any real n × n matrix A, det(λI − A) is a polynomial of degree n

in λ,

2. the coefficients of the polynomial det(λI −A) are determined by the entries
of matrix A,

3. the coefficient of the highest order term (i.e. λn ) in the polynomial

det(λI − A) is always equal to 1 (we say that the polynomial det(λI − A)
is monic),

4. the roots of the polynomial det(λI − A) are exactly the eigenvalues of

A.

Due to the importance of the polynomial (in λ) defined by det(λI − A), we give
it a special name:

Definition 36. Let A ∈ Mn (R) be a real n × n matrix. The polynomial of degree

n in λ given by det(λI − A) is called the characteristic polynomial of A.

Now we know how to compute the eigenvalues of a given matrix A ∈

Mn (R); all we have to do is proceed as follows:

(i) Compute the characteristic polynomial of A,

(ii) Find the roots of the characteristic polynomial of A, i.e. find all the
scalars λ which satisfy det(λI − A) = 0 ; these are the eigenvalues of A.

Let us consider a few examples to illustrate the procedure highlighted above:

 
2 1
1. Let A ∈ M2 (R) be given by A = ; suppose we wish to find all
1 2
the eigenvalues of A. How would we proceed ? Well, as detailed above,
all we have to do is:

(i) Compute the characteristic polynomial det(λI − A) of A (where I is

the 2 × 2 identity matrix),
(ii) find the roots of that characteristic polynomial, i.e. all λ which satisfy
det(λI − A) = 0.
198 SECTION 15

The characteristic polynomial det(λI − A) is easily computed to be:

 
λ−2 −1
det(λI − A) = det( )
−1 λ−2
= (λ − 2)2 − 1
= λ2 − 4λ + 3.

Note that this characteristic polynomial is, as expected, a monic polyno-

mial of degree 2 (since A is 2 × 2).
The eigenvalues of A are given by solving the equation

det(λI − A) = 0,

i.e. by solving

λ2 − 4λ + 3 = 0;

we immediately find that the only solutions are given by λ = 1 and λ = 3.

Hence, we conclude that the set of eigenvalues of A is given by the set
{1, 3}.
2. Let now I ∈ Mn (R) denote (as usual) the n × n identity matrix; let us
try to find the eigenvalues of I. Again, as detailed above, all we have to
do is:
(i) Compute the characteristic polynomial det(λI − I) of I,
(ii) find the roots of that characteristic polynomial, i.e. all λ which satisfy
det(λI − I) = 0.
The characteristic polynomial det(λI − I) of I is easily computed to be:

det(λI − I) = det((λ − 1)I)

= (λ − 1)n det(I)
= (λ − 1)n ,

since, as we have already seen, det(I) = 1.

The eigenvalues of I are given by solving the equation

det(λI − I) = 0,

i.e. by solving

(λ − 1)n = 0

we immediately find that the only solution is given by λ = 1. Hence, we

conclude that the real number 1 is the only eigenvalue of the n× n identity
matrix I, i.e. the set of eigenvalues of I is given by the set {1}.
 199

3. Let now A ∈ Mn (R) denote the n × n zero matrix (i.e. all entries of A
are zero); let us try to find the eigenvalues of A. Again, as detailed above,
all we have to do is:
(i) Compute the characteristic polynomial det(λI − A) of A,
(ii) find the roots of that characteristic polynomial, i.e. all λ which satisfy
det(λI − A) = 0.
The characteristic polynomial det(λI − A) of A is easily computed to be:

det(λI − A) = det(λI)
= λn det(I)
= λn .

The eigenvalues of A are given by solving the equation

det(λI − A) = 0,

i.e. by solving

λn = 0.

We immediately find that the only solution is given by λ = 0. Hence, we

conclude that the real number 0 is the only eigenvalue of the n × n zero
matrix A, i.e. the set of eigenvalues of A is given by the set {0}.
 
a b c
4. Let now A ∈ M3 (R) be given by A =  0 d e  for some real num-
0 0 f
bers a, b, c, d, e, f ; again, suppose we wish to find all the eigenvalues of A.
Again, as detailed above, all we have to do is:
(i) Compute the characteristic polynomial det(λI − A) of A (where I is
the 3 × 3 identity matrix),
(ii) find the roots of that characteristic polynomial, i.e. all λ which satisfy
det(λI − A) = 0.

The characteristic polynomial det(λI − A) is easily computed to be:

 
λ−a −b −c
det(λI − A) = det( 0 λ−d −e )
0 0 λ−f
= (λ − a)(λ − d)(λ − f )
= λ3 − (a + d + f )λ2 + (ad + af + df )λ − adf.

Note that this characteristic polynomial is, as expected, a monic polyno-

mial of degree 3 (since A is 3 × 3).
200 SECTION 15

The eigenvalues of A are given by solving the equation

det(λI − A) = 0,

i.e. by solving

(λ − a)(λ − d)(λ − f ) = 0

we immediately find that the only solutions are given by λ = a and λ = d,

and λ = f . Hence, we conclude that the set of eigenvalues of A is given
by the set {a, d, f }.
Remark 11.
  Before going further, let us consider the 2 × 2 real matrix A =
0 −1
and let us try to compute its eigenvalues. According to the procedure
1 0
outlined above, we begin by computing the characteristic polynomial det(λI − A)
of A (where I denotes the 2 × 2 identity matrix). We obtain:
 
λ 1
det(λI − A) = det
−1 λ
= λ2 + 1.

The eigenvalues of A are then given by all the scalars λ which satisfy

det(λI − A) = 0,

i.e. which satisfy

λ2 + 1 = 0.

But there is no real number λ which satisfies λ2 + 1 = 0! Hence, we can say

that A has no real eigenvalue. On the other hand, the complex numbers i
and −i are the two solutions to the above equation (recall that i2 = −1); hence
we can say that this particular matrix A has only complex eigenvalues and no
real eigenvalue.
In this lecture, we will restrict ourselves to n × n matrices which have only real
eigenvalues.
Now that we have a “recipe” for computing all the eigenvalues of a given real
n × n matrix A, how do we compute the corresponding eigenvectors ? We
begin with a definition:
Definition 37. Let A ∈ Mn (R) be a real n × n matrix, and let λ ∈ R be an
cn given by ker(λI − A) is called the
eigenvalue of A. The vector subspace of R
eigenspace of A associated to the eigenvalue λ of A.
It is important to note the following points:
cn (see the lecture on the Kernel
(i) ker(λI −A) is indeed a vector subspace of R
of a linear mapping).
 201

(ii) Any non-zero vector v in ker(λI − A) is an eigenvector of A associated

with eigenvalue λ; indeed, let v ∈ ker(λI − A) with v 6= 0. We then have

(λI − A)v = 0,

which implies

Av = λIv,

which implies

Av = λv,

and since v 6= 0, this shows that v is an eigenvector of A associated with

eigenvalue λ.

These observations naturally lead us to a “recipe” for computing eigenvectors of

a given real n×n matrix A. How do we do it ? Well, as suggested above, we first
compute all the eigenvalues of A, and then compute the eigenspace associated
to each eigenvalue. Each non-zero vector in the eigenspace associated to a given
eigenvalue will be an eigenvector associated to that eigenvalue.
Let us illustrate this “recipe” on an example:
 
2 1
1. Let A ∈ M2 (R) be given by A = (as before); we have already
1 2
computed all the eigenvalues of A, and we know that the set of eigenvalues
of A is the set {1, 3}. Let us now compute the eigenspaces associated to
each of these eigenvalues:
The eigenspace associated to the eigenvalue 1 of A is the vector subspace
c2 given by:
of R
   
1 − 2 −1 −1 −1
ker(1I − A) = ker( ) = ker( ),
−1 1 − 2 −1 −1

in other words, the eigenspace of A associated

 toeigenvalue 1 is nothing
−1 −1
other than the kernel of the matrix , and we already know
−1 −1
very well how to compute kernels of matrices! So let’s do it! We have:
   
−1 −1 x c2 | − x − y = 0},
ker( ) = { ∈R
−1 −1 y
   
x −1 −1
and therefore, ∈ ker( ) implies
y −1 −1
     
x x 1
= =x .
y −x −1
202 SECTION 15

 
c2 be defined by v1 = 1
Let v1 ∈ R . The above calculation shows
−1
that {v1 } is a generating set for the eigenspace of A associated with
eigenvalue 1; furthermore, it is easy to verify that {v1 } is a linearly inde-
pendent set. Hence, (v1 ) is a basis for the eigenspace of A associated
to the eigenvalue 1, and we conclude therefore that that eigenspace is one-
dimensional. In particular, v1 itself is an eigenvector of A associated
with eigenvalue 1. Let us verify this last claim: Clearly v1 6= 0, and we
have
      
2 1 1 2−1 1
Av1 = = = = v1 = 1v1 ,
1 2 −1 1−2 −1

as expected.
Let us now compute the eigenspace of A associated to the eigenvalue 3
of A. The eigenspace associated to the eigenvalue 3 of A is the vector
c2 given by:
subspace of R
   
3 − 2 −1 1 −1
ker(3I − A) = ker( ) = ker( ),
−1 3 − 2 −1 1

in other words, the eigenspace of A associated  to eigenvalue

 3 is noth-
1 −1
ing other than the kernel of the matrix ; again, this is
−1 1
something we know very well how to compute! We have:
   
1 −1 x c2 |x − y = 0},
ker( ) = { ∈R
−1 1 y
   
x 1 −1
and therefore, ∈ ker( ) implies
y −1 1
     
x x 1
= =x .
y x 1
 
c2 1
Let v2 ∈ R be defined by v2 = . The above calculation shows
1
that {v2 } is a generating set for the eigenspace of A associated with
eigenvalue 3; furthermore, it is easy to verify that {v2 } is a linearly inde-
pendent set. Hence, (v2 ) is a basis for the eigenspace of A associated
to the eigenvalue 3, and we conclude therefore that that eigenspace is one-
dimensional. In particular, v2 itself is an eigenvector of A associated
with eigenvalue 3. Let us verify this last claim: Clearly v2 6= 0, and we
have
      
2 1 1 2+1 3
Av2 = = = = 3v2 ,
1 2 1 1+2 3

as expected.
 203

Let now A ∈ Mn (R), let k ≥ 1 be an integer, and let us consider the k th

power Ak of A (i.e. AA · · · A (k times)); is there a simple relation between the
eigenvalues A and those of Ak ? The answer is given by the following theorem:

Theorem 43. Let A ∈ Mn (R) and let λ be an eigenvalue of A. Then for any
integer k ≥ 1, λk is an eigenvalue of Ak .
cn , v 6= 0,
Proof. Since by assumption λ is an eigenvalue of A, there exists v ∈ R
such that Av = λv; hence:

Ak v = (Ak−1 A)v = Ak−1 (Av) = Ak−1 (λv) = λ(Ak−1 v),

(where we have used both the associativity and linearity of matrix multiplica-
tion) and upon iterating this procedure, we obtain

Ak v = λ(Ak−1 v) = λ2 (Ak−2 v) = λ3 (Ak−3 v) = · · · = λk v,

and since v 6= 0, this shows that λk is an eigenvalue of Ak . Note in passing

that we have also shown that if v is an eigenvector of A then v remains an
eigenvector of Ak for any integer k ≥ 1.

Applications of eigenvalues and eigenvectors

The most common way to locate information on the web is to do a keyword
search using a search engine such as GoogleT M . But the web contains billions
of pages, and, typically, tens of thousands of pages match the keywords searched.
Yet, GoogleT M manages to order the results of the search in a way which agrees
very closely with our own notion of relative importance or relevance of the pages.
How does GoogleT M do it ? Well, for a start, search engines such as GoogleT M
constantly crawl the web, going from one link to the next, in order to inventory
every single page. This is then used to set up a huge matrix (4 billion by 4 billion)
that captures the links between the pages (rows and columns are indexed by
the 4 billion pages, and a link from a page to another would be represented by
a non-zero positive real number in the matrix entry corresponding to that row
and column). The whole problem of ranking pages becomes that of finding an
eigenvector of that huge matrix associated with eigenvalue 1. For a fascinating
account of the mathematics behind GoogleT M ’s PageRank algorithm, the reader
is strongly encouraged to read the article “The $ 25,000,000,000 Eigenvector:
The Linear Algebra behind Google” by Kurt Bryan and Tanya Leise, published
in SIAM Review, Volume 48, No. 6, 2006.

PROBLEMS:

1. For each of the following choices for the matrix A:

(i) Compute the set of all eigenvalues of A.

204 SECTION 15

(ii) For each eigenvalue of A, determine a basis for the corresponding

eigenspace.
 
0 0
(a) A = .
0 0
 
0 1
(b) A = .
0 0
 
1 0
(c) A = .
0 1
 
1 1
(d) A = .
0 1
 
1 0
(e) A = .
1 1
 
1 1
(f) A = .
1 1
 
2 0
(g) A = .
0 2
 
2 1
(h) A = .
0 2
 
5 0
(i) A = .
0 3
 
5 1
(j) A = .
0 3
 
7 0
(k) A = .
0 −2
 
7 1
(l) A = .
0 −2
 
7 0
(m) A = .
1 −2
 
3 1
(n) A = .
1 3
 
3 2
(o) A = .
2 3
 
3 1
(p) A = .
4 3
 
3 1
(q) A = .
9 3
 
3 1
(r) A = .
0 3
 205

 
3 2
(s) A = .
0 3
 
1 0 0
(t) A= 0 1 0 .
0 0 1
 
1 1 0
(u) A= 0 1 1 .
0 0 1
 
1 0 1
(v) A= 0 1 0 .
0 0 1
 
1 1 1
(w) A= 0 1 1 .
0 0 1
 
2 0 0
(x) A= 0 5 0 .
0 0 7
 
2 1 0
(y) A= 0 5 1 .
0 0 7
 
2 0 1
(z) A= 0 5 0 .
0 0 7
 
2 0 0 0
 0 2 0 0 
(z1) A=
 0 0
.
7 1 
0 0 0 7
 
2 1 0 0
 0 2 0 0 
(z2) 
A= .
0 0 7 1 
0 0 0 7
 
2 1 0 0
 0 2 0 0 
(z3) A=
 0 0
.
7 0 
0 0 0 7
 
2 0 0 0
 0 2 0 0 
(z4) A=
 0 0
.
7 0 
0 0 0 7
 
2 1 0 0
 0 3 0 0 
(z5) A=
 0 0
.
4 1 
0 0 0 6
206 SECTION 15

 
2 0 0 0
 0 3 0 0 

(z6) A =  .
0 0 4 1 
0 0 0 6
 
2 0 0 0
 0 3 0 0 
(z7) A = 
 0
.
0 4 0 
0 0 0 6