Linear Algebra

CSC 114
LINEAR ALGEBRA

By Prof. Kingsley Muka

Page 1 of 83
 Linear Algebra

LINEAR ALGEBRA FOR COMPUTER SCIENCE CSC114 (2 units)

Linear algebra is a fundamental branch of Mathematics which handles the study of vector spaces
and linear transformations.

It plays an important role in different fields such as Physics, Computer Science, Robotics,
Engineering etc.

Learning Objectives:

The objectives of the course are to:

(i) Explain vectors and the various forms in which they appear
(ii) Illustrate the importance of vectors in Computer Science
(iii) Explain matrices and their algebraic structures
(iv) Illustrate Eigenvalues and Eigenvectors
(v) Explain the idea behind vector spaces and other mathematical concept tied to it
(vi) Illustrate matrix, Eigenvalues and Eigenvectors with Computer Science related
problems

Learning Outcomes:

At the end of the course, students are expected to be able to:

(1) Explain what vectors are

(2) Describe the various mathematical operations performed on vectors as well as solve related
problems
(3) Explain the nature of matrices and various algebraic operations performed on matrices as
well as solve related problems
(4) Solve for the Eigenvalues and Eigenvectors of square matrices
(5) Explain the concept of vector space and its components
(6) Describe the rationale behind linear independence, basis, dimension and linear
transformation on vector spaces.
(7) Apply these knowledge area in Computer problem domains e.g. Graphics

Page 2 of 83
 Linear Algebra

Course Content

Week 1 General Introduction/Vectors

Week 2 Vectors and its operations

Week 3 Linear Systems and Solutions

Week 4 Methods for solving Linear Systems (Matrix Based)

Week 5 Revision/Continuous Assessment

Week 6 Vector Space over a Real Field

Week 7 Vector Space Cont.

Week 8 Subspaces

Week 9 Linear dependence & Independence

Week 10 Revision/Continuous Assessment

Week 11 Basis and Dimension

Week 12 Linear Transformation

Week 13 Null Space

Week 14 Revision/Continuous Assessment

Week 15 Application of Linear Algebra to Computer Science

Page 3 of 83
 Linear Algebra

VECTORS IN 2- AND 3-DIMENSIONAL SPACE(S)

1.1. INTRODUCTION

In Science and Engineering, two kinds of quantities are usually encountered. These
are scalars and vectors quantities.

A scalar is a quantity that is determined by its magnitude. Example include

distance, speed, temperature, age, current, mass, pressure, density, work, energy
and power. Vector quantity characterized by both magnitude and direction (except
for zero vector). Examples are displacement which is distance in a specified
direction, velocity, force, acceleration, momentum, electromotive force, and
weight.

The difference between speed and velocity can be shown by considering a car
moving round a circle as shown in fig1.1

\\

Fig 1.1 car movingC in a circular motion

At each points A, B and C, the speed of the car can be 20km/h as could be seen on
the car speedometer. However, the velocities are different due to directional
changes of the car. Therefore, the magnitude of velocity is the speed.

To describe a vector geometrically, the concept of directed line segment drawn to

scale with an arrow will be introduced. This is, so that its length gives the
magnitude and its orientation gives the direction of the vector.

A vector (arrow) has a tail called its initial point and a tip called its terminal point.
Therefore the position of a point B relative to another point A can be completely
described by vector from point A, the initial point to the point B, the terminal
point such that the vector specifies both the distance and direction from A to B.
Page 4 of 83
 Linear Algebra

Vectors are denoted by uppercase letter of initial and terminal points with an arrow
on top. For example . When the initial and terminal points are
known or assumed, upper or lower case letter can be used with an arrow on top, for
example . In most text, vectors are denoted by lowercase bold face
letters a,b,v etc.

1.2. EQUALITY OF VECTORS

Two vectors a and b are equal, written , if they have the same length and the
same direction. Hence, a vector can be arbitrarily translated: that is, it initial point
can be chosen arbitrarily.

𝑎
𝑏⃑
• Fig 1.2(a) Equal vectors
𝑎
𝑏⃑
• Fig 1.2(b) Equal length but different direction

• Fig 1.2(c) same direction but different length (Not Equal)

1.3. VECTOR COMPONENT AND CARTESIAN SYSTEM

A single vector is made up of two or more vectors called vector components.

Let be a given vector with initial point and terminal point Q

then, the three coordinate differences

(1.1)

are called components of vector with respect to that coordinate system.

Page 5 of 83
 Linear Algebra

(1.2)

A vector can be expressed as sum of its vector components which are in different
directions. These particular directions are called reference directions for the vector
and each component vector can be expressed in terms of a unit vector in that
direction.

Note: Unit Vector is a vector whose magnitude is one.

When the reference directions are mutually perpendicular (orthogonal) such a

reference system is called the rectangular Cartesian coordinate system. In the three
dimensional system, O is generally taken as the centre or reference origin and axes
are OX, OY and OZ such that rotation of x-axis into y-axis about z-axis is
accomplished by the right-hand rule.

For a Cartesian coordinate system: given the position vector of a point

. The vector with origin as the initial point and A as the terminal
point
z
A

r
o
y

Fig 1.3: Position vector of a point

In components, . Thus the length of r can readily be expressed in

terms of components because

Using Pythagoras Theorem,

(1.3)
The magnitude of vector , is given as

(1.4)

Page 6 of 83
 Linear Algebra

The vector r in fig1.3, is called position vector because it has the origin as
the initial point of r and as the corresponding terminal point, the
coordinates of r equals the components of A.

This suggests that each point in space can be determined by a vector called the
position vector of the point. That is, a point in space is an ordered triple .
of real numbers.

The xy- plane is the plane that contains the x- and y- axes; the yz- plane contains
the y- and z- axes; the xz- plane contains the x- and z- axes. These three coordinate
planes divide space into eight parts called octants.
z

yz plane
xz plane
o
y
xy plane

x
Fig 1.4. Octant

1.4. VECTOR ALGEBRA

1.4.1. ADDITION OF VECTORS

If a particle moves from X to Y the vector XY represents the displacement of Y

with respect to X. if the particle further moves from Y to a new point Z, the
displacement of Z with respect to Y is the vector YZ. The displacement of Z with
respect to X is represented by XZ. Thus the vector XZ is the resultant of the
vectors XY and YZ. So, we write

XY+YZ=XZ (1.5)

Page 7 of 83
 Linear Algebra

This is the fundamental law of addition of displacements. Geometrically, place the

vectors as shown in figure 1.5, whereby the two sides of the triangle XY and YZ
drawn to scale to represent the two given vectors and the third side XZ represents
the resultant. Y

x
Figure 1.5: vector
addition

This is the triangle method for the addition of vectors.

An alternative way of adding vectors is called parallelogram method. This is done

by drawing the vectors to scale so that they form adjacent sides of a parallelogram.

Note that for a particular parallelogram, opposite sides are equal, the resultant of
the two vectors is the diagonal drawn from the corner with two tails to the corner
with two heads as shown in figure 1.6

b a

a b

Figure 1.6: Parallelogram Law

Two vectors a, b can be added as arrows, placed such that the terminal point of a
coincides with the initial point of b. the resultant then will be c which is a vector
from the initial point of a to the terminal point of b. This shows that the resultant
vector from the initial point of b to the final point of a in fig 1.6 is also c. That is,
(1.6)

hence, vector addition is commutative.

Any number of vectors can be compounded (added) into a single vector by

represented application of the addition laws of two vectors. Thus,

AB+BC+CD=AD, (1.7)
Page 8 of 83
 Linear Algebra

as shown in figure 1.7 B

C

A D
Figure 1.7 compounded vectors.
b
c
b+c
a
a+b

a+b+c
Figure 1.8 Associative law.

Given vectors a, b, c as shown in fig1.8, observe that

(1.8)

The addition satisfies the associative law.

1.5. MULTIPLICATION OF A VECTOR BY A SCALAR

If m is a scalar and A is a vector, then the multiplication of the vector A by the

scalar m is with magnitude m times the magnitude of A and the direction as
that of A, if m is positive. If m is negative, the direction will be opposite to the
direction of A.
1.6. ZERO VECTORS

The difference of two equal vectors will give us a zero vector, which is a vector of
zero length and with no particular (arbitrary) direction.
(1.8)
The zero/null vector is an identity element for vector addition, that is

(1.9)
1.7. UNIT VECTOR

A vector whose magnitude is one is called a unit vector. If given direction a unit
vector can be written as to mean a unit vector in the direction of AB. A vector
AB can be written as the product of its magnitude and unit vector in its direction.
Page 9 of 83
 Linear Algebra

That is

(1.10)
Where AB is the scalar component specifying the magnitude while is the unit
vector specifying the direction. Therefore,

(1.11)
1.6. LAWS OF VECTOR ALGEBRA

If a, b and c are vectors and m and n are scalars then,

• Commutative law-

• Associative law-

• Distributive law for multiplication

• Distributive law for addition

Example 1.1

Find the sum of

(a) PQ, -TQ, PS, ST

(b) VX, XY, -ZY, ZX Solution

(a) PQ+(-TQ)+PS+ST=PQ+QT+PS+ST
=PT+PT
=2PT
(b) VX+XY+(-ZY)+ZX=VX+XY+YZ+ZX
=VY+YZ+ZX
=VZ+ZX=VX
1.8. UNIT VECTOR i, j, k

Another way of writing the vector is

(1.12)
Page 10 of 83
 Linear Algebra

In this representation i, j, k are the unit vectors in the positive directions of the axes
of a Cartesian coordinate systems, see figure 1.9
z
k

i j
x y
Fig 1.9: the unit vectors i, j, k

Hence, in components
The right hand side of (1.12) is a sum of three vectors parallel to the three axes
z

x
y
Figure 1:10 the representation of (1.12)

The triple of vectors i, j, k is called a standard basis of R3.

Page 11 of 83
 Linear Algebra

All vectors with real numbers as components

3
for the real vector space R with the two algebraic operations of vector addition
and scalar multiplication R3 has dimension 3.

A vector expressed as , implies that vector a has 2 units in the x

direction, 3 units in the y direction and 5 units in the z direction.

With reference to the origin, the end points of vector a is determined by a point in
space given by (2, 3, 5). 2i, 3j, and 5k are the vectors components of a, while 2, 3,
5 are the components in the x, y and z direction.

Generally, a vector v is written as

(1.13)
This can also be expresses using the orthogonal Cartesian format

(1.14)
Consider a vector OP in three dimensions whose initial point is the origin (i.e.
position vector) as shown in figure 1.11
j
P

𝑣𝑧 o 𝑣𝑦�

k 𝑣𝑥� i

Figure 1.11: position vector OP

The magnitude of OP is

(1.15)
The magnitude is also referred to as length or norm (or Euclidean norm) of the
vector. The magnitude of any vector represented as sum of vector components in
orthogonal coordinate system is the positive root of the squares of its components.

Page 12 of 83
 Linear Algebra

Example 1.2 (Components and Length or Magnitude of a Vector)

Find the components of the vector v with initial point A and terminal point B. Find

(i)

(ii)
Solution

1. The vector v with initial point and terminal point has

the components

Given as

2. The vector v with initial point and

terminal point has the components

Given as

Find the terminal point B of the Vector a whose components are as given and
initial point A.

Page 13 of 83
 Linear Algebra

(ii)

Solution

(i) has terminal point given as

(ii)

Example 1.4 (Addition and Scalar Multiplication)

If

Find

(i) 3a; -a
(ii) (a+b)+c; a+(b+c)
(iii) 6(c-b); 6c-6b
(iv) 3a+4b; -3a-4b Solution

(i)

(ii)

Page 14 of 83
 Linear Algebra

(iii)

(iv)

Example 1.5

Find the unit vector u in the direction of v if the vector v has initial point
and terminal point .

Solution

Recall the component of

Hence

The unit vector

Example 1.6

If , find the length of a, and the direction cosines of a.

Solution

Length or Magnitude of a

Recall a , therefore

Let be the angle vector a makes with the coordinates axes. Then the direction
cosines are:

Page 15 of 83
 Linear Algebra

Exercise

1. Given vector

(a) Find the resultant in terms of components and its magnitude.

(b) Find vector such that the resultant of vectors has no

component in x- and y- directions.

2. For what value of u, is the resultant of vectors

and parallel to the xy plane?
1.8. PRODUCT OF VECTORS

As shown in the previous discussion that the product of a vector and a scalar gives
a vector, which is parallel to the original vector.

In this section, product of vectors will be discussed. There are two types of product
of two vectors, these are: the dot (or scalar or inner) product of two vectors and the
cross (or vector or outer) product of two vectors.

1.8.1 DOT PRODUCT (SCALAR OR INNER PRODUCT)

The scalar product of two vectors is also known as dot or inner product. The dot
product of two vectors A and B written as is the product of their length times
the cosine of their angle.

(1.16)
(1.17)

Where , the angle between A and B when the initial points of the vector
coincide.

Page 16 of 83
 Linear Algebra

A A A
𝜃 𝜃
𝜃
B B B

Figure 1.2: Angle between vectors A and B.

where where ; and where .

Theorem 1
The dot product (inner product) of two nonzero vectors A and B is zero, if and only
if the vectors are perpendicular.
The angle can be determined as follows

(1.18)
In components,
if

and
then,

Recall
From the definition of dot product

Similarly,

Also

It follows that the dot product of vectors A and B is:

(1.19)

Page 17 of 83
 Linear Algebra

Thus, the dot product of two vectors A and B is the sum of the products of their
corresponding components.
Substituting (1.19) into (1.18) yield

(1.20)
If A=B, then (1.19) becomes

(1.21)
Similarly,
Substituting (1.19) into (1.21), to obtain

(1.22)
1.8.2. PROPERTIES OF SCALAR PRODUCT

Given three vectors A, B, C and scalars m and n then i.

Commutative law of vectors-
ii. Distributive law-
iii. Homogenity- iv.
Positivity-
v.
If , that is A and B are parallel and in the same direction then

If , that is A and B are parallel but in opposite direction then

Example 1.7

Find the scalar product and the lengths of vectors and

as well as the angle between them.
Solution

Page 18 of 83
 Linear Algebra

Class Exercise

Given and . Find the angle between a and b.

Exercise: Inner Product (Kreyszig, (2011))

Let
Find

1.
2.
3.
4.
5.

6.

Page 19 of 83
 Linear Algebra

7.
8.
9.
10.

1.9. VECTOR PRODUCT

The vector product of two vectors A and B written as (pronounced A cross

B) is defined as

The length of vector V is given as

(1.23)

Where is the angle between A and B measured in the positive sense and is the
unit vector perpendicular to the plane of A and B, see figure 2.3
𝑒𝑛�
B

𝜃
A
Figure 1.3: unit vector perpendicular to plane From
(1.23), see that

(1) The magnitude of is the product of the length of A and B and the
numerical value of the sine of the angle between the vectors.
(2) The vector has direction perpendicular to the plane of A and B. That
is, it is perpendicular to A and Band it is also oriented that A rotated into B
about by the right hand rule, though not more than
(i) If , then we define
(ii) If A and B lie in the same straight line, that is A and B have the same or
opposite directions, then is so that . In that case
, so that

Page 20 of 83
 Linear Algebra

(iii) If both vectors A and B are non-zero vectors then, vector V has length as
given in equation (1.23).

1.10. RIGHT HANDED CARTESIAN SYSTEM

The system is called right handed if the corresponding unit vectors i, j, k are in the
positive directions of the axes as shown in figure 2.4a. These form a right – handed
triple. The system is called left – handed if the sense of k is reversed as shown in
figure 1.4b z
k i j

j k
i x
y
x y z
(a) Right – handed (b) left – handed
Figure 1.4: the two types of Cartesian coordinate system

In application, the right – handed system is preferred

If , and
Then

Page 21 of 83
Linear Algebra

Page 22 of 83
 Linear Algebra

LINEAR SYSTEMS AND SOLUTIONS 2.1

LINEAR EQUATION

An equation of this form:

(2.1)

Is called a linear equation in variables and

Equation (2.1) is an algebraic representation of a line in xy-plane. In general, a linear equation in

m variables is given in the form

(2.2)

Where are real constants

A linear equation does not involve any products, or roots of variables. Also, variables do not
appear as arguments for trigonometric, logarithmic or exponential functions. The exponent of the
variable is first power.

Example 2.1

The following equations are linear:

1.
2.
3.

Example 2.2

Some examples of nonlinear equations are:

1.
2.
3.

Page 23 of 83
 Linear Algebra

2.2 SOLUTION OF LINEAR EQUATION

Sequence of m real numbers such that that satisfy the

equation (2.2) is called solution to the linear equations (2.2). The set of all solutions of the
equation (2.2) is called the solution set to (2.2).

Example 2.3

Find the solution set of the following:

(i)
(ii)
(iii)
(iv)

Solution

The solution set to (i) and (iii) are explained while (ii) and (iv) are left for students to attempt.

(i) Assign arbitrary value to x and solve for y or choose arbitrary value for y and solve for x.

Let x=s, then

The formula describe the solution set in terms of arbitrary parameter s. Particular solution can be
obtained by substituting specific values for s.

Example, for s=5, yields x=5, and y=3. For s=3/2, x=3/2 and y=2.

If we set y=v, then the solution set is and y=v

This formula is different from previous, however they both yield the same solution set as s and v
varies over all possible real numbers. For example if v=2,
Therefore, x=3/2 and y=2.

(ii) To determine the solution to the third linear equation, assign two arbitrary values to any
two variables and solve for the third variable as follows:

Let and

then, and

Page 24 of 83
 Linear Algebra

2.3 SYSTEM OF LINEAR EQUATION

A finite set of linear equations in the variables in the form:

(2.3)

is called a system of linear equation or linear system. Where are

real constants called coefficients.

A sequence of real numbers is called a solution of the system (2.3), if

satisfies every equation in the system. The system of linear equation (2.3) can be
written in compact form as

(2.4)

Where

A is an matrix. A matrix is a rectangular array of numbers.

X is an m-dimensional column vectors and B is an n-dimensional column vectors. The number of

equations in (2.4) is n while the numbers of unknowns is m.

The system of linear equation (2.4) can also be written in the form

called the Augmented Matrix.

Example 2.4

Write the augmented matrix for the system:

Page 25 of 83
 Linear Algebra

Solution

2.4. CONSISTENT LINEAR SYSTEM

Consider a general system of two linear equations in x and y unknowns. Each equation is a line,
which as shown in figure 2.1

(2.5)

these two lines and have solution at the points of intersection.

y y y 𝑙
𝑙 𝑙 𝑙
𝑙
𝑙 𝑙

x x
(a) Parallel x (b) Intercept at all points (c) Intercept at a point

Fig. 2.1: Graphs of equations and

In figure 2.1(a), both lines are parallel, hence no point of interception. This implies no solution.
In figure 2.1(b), both lines intercept at all points, hence the system has infinitely many solutions.
In figure 2.1(c), both lines intercept at one and only one point and therefore (2.5) has a unique
solution. System of linear equation (2.5) that has solution is said to be consistent and system of
linear equations that do not have solution are said to be inconsistent.

In general, the system (2.3) is said to be consistent; if it has a solution and inconsistent otherwise.

2.5. ELEMENTARY ROW OPERATIONS

Two or more systems of Linear equations can have same solution sets. Set of linear systems that
have same solution set are said to be (row) equivalent.

Page 26 of 83
 Linear Algebra

Methods for solving system of linear equations, usually involve replacing given system by a
simpler equivalent system. The process through which this is done is via the application of
sequence of the three elementary row operations:

(i) Interchanging two rows

(ii) Multiplying any row by a nonzero constant
(iii) Adding a multiple of one row to another row

Example 2.5

Use the row operations to obtain row equivalent to the system

which has solution x=1, y=2, and z=-1.

Solution

, Using
row operations as follows:

Equivalent system is

Students can verify with the solution: x=1, y=2, and z=-1. The new system can further be reduced
by obtaining new third row as follows:

an equivalent system to the original equation is

Page 27 of 83
 Linear Algebra

This is new system is in form called triangular system. The solution to new system can easily be
obtained by method of Back substitution as

x=1, y=2, and z=-1.

Class Exercise

(1) Given an augmented matrix

obtain equivalent systems.

(2)

Obtain equivalent systems.

Note that, two augmented matrices corresponding to linear systems that have same solution are
said to be row equivalent and are denoted using ~ (tilde), for example.

2.6. DEGENERATE AND NON-DEGENERATE EQUATIONS

A linear equation is said to be degenerate, if it has the form

, (2.6)

that is, all the coefficient It is non-degenerate linear equations otherwise.

Page 28 of 83
 Linear Algebra

If in (2.6), b=0, then any constant is a solution to (2.6). If ,

then (2.5) has no solution.

2.7. ECHELON FORM

A system of linear equation is in echelon form, if no equation is degenerate and if the leading
unknown in each equations to the right of the leading unknown of the preceding equation. As
given in the augmented matrix below.

where and and

All zero rows, if any in the echelon matrix are on the bottom of the matrix.

Example

The following are Augmented matrix in echelon

(1)

(2)

Page 29 of 83
 Linear Algebra

(3)

(4)

The augmented matrix in echelon form can also be expressed as follows:

(2.7)

where and

In system (2.7) the number of equations r is less than or equal to the number of unknown m. That
is If r<m the system is called under determined, r=m, it is called a determined system.
While if r>m, the system is called over determined.

The variables in the echelon system (2.7) are called free variables if is not the
lead variable in any equation (i.e. lead variable in an echelon system is the variable
corresponding to the pivot element).

In the case of under determined system where r<m, m-r variables become free and values can be
assigned to the m-r variables to obtain solution to the system.

Example 2.6 (Schaum’s series 1991)

Consider the system

The system is in Echelon form. The leading unknowns are x and z. hence, free variables are y
and t. To solve the echelon system, assign parameters to y and t as and obtain

Example 2.7

(1) The system

Page 30 of 83
 Linear Algebra

Reduce to Echelon form using the row operations

Or

The third equation is a multiple of the second. Hence, the system has two equations and three
unknowns. The free variable is z.

Hence, we assign a parameter and solve for others. That is is

the general solution.

(3) The system

Reduced to Echelon form

Perform the operation yields the degenerate equation

The system has no solution.

2.8. REDUCED ROW ECHELON FORM

Equivalent augmented matrices can be put in a canonical form called the reduced Row Echelon
Form. Canonical form is unique augmented matrix in reduced row echelon form. The reduced
row echelon form is the augmented matrix of the form;

Page 31 of 83
 Linear Algebra

The pivot element is 1 and other entries in its column are zero. The * element are non-zero values
in each row and column. The lead variable is to the right of preceding equation.

Augmented matrix in echelon form can help in determining, if the system has no solution,
infinite solutions or unique solution. If at any point, a degenerate row of the form

(2.8)

is obtained, then the system has no solution if and infinitely many solutions if

It has a unique solution, if there is no degenerate row.

In reduced row echelon form, the solution when it exist can easily be obtained by method of
back substitution.

Example 2.8

Determine the reduced row echelon form for the following

augmented matrices:

(1)

(2)

Page 32 of 83
 Linear Algebra

(3)

Solution

(1)

In reduced row echelon form

Page 33 of 83
 Linear Algebra

From this reduced row echelon form, observe that the solution is

The new system is the reduced row echelon of the augmented matrix. The system will the
augmented matrix

has no solution since in the reduced roe echelon form.

Page 34 of 83
 Linear Algebra

Using the row operation , to obtain row equivalent

another sequence operations: :

’ Canonical

form is obtained using the operation

Assign the free variable say and obtain a solution set

. That is

Class Exercise

(a) Give the canonical form of the following augmented matrices and state the nature of their
solution where it exists:

Page 35 of 83
 Linear Algebra

Home Work
(iv) Determine the values of that will make the solution to the system with the
augmented matrix

(i) unique solution (ii) infinitely many solution (iii) no solution.

Page 36 of 83
 Linear Algebra

Matrices and Matrix Operations

3.0 Matrix

Matrix is a rectangular array of numbers. The vertical arrangements are called the
columns while the horizontal arrangements are called the rows.

The size of a matrix is determined by the total numbers of rows and columns.

The matrix in (2.4) has size 𝑛 × 𝑚. The column vector X in (2.4), is a matrix with
one column and n rows (it is also known as a column matrix).

A row matrix is a matrix with one row and m columns also known as row vector.

Example 3.1

An example of a row vector with four columns is (1 −1 5 0)

𝑎11 𝑎12 ⋯ 𝑎1𝑚
𝑎21 𝑎22 ⋯ 𝑎2𝑚
The matrix 𝐴 = ( ⋮ ⋮ ⋮ , in (2.4) can also be written as
𝑎𝑛1 𝑎𝑛2 ⋯ 𝑎𝑛𝑚

𝐴 = [𝑎𝑖𝑗 ]𝑛×𝑚
(3.1)
3.1 Square Matrix

A matrix with equal rows and columns is called a square matrix.

Suppose in (2.4) 𝑛 = 𝑚, so that

𝐴 = [𝑎𝑖𝑗 ]𝑚×𝑚 ,
(3.2)

Then we say matrix A is a square matrix of order m. The entries 𝑎𝑖𝑗 , such that 𝑖 = 𝑗
are called the diagonal element.

A square matrix with zero entries in off diagonal and 1’s on the main diagonal is
called unit matrix or identity matrix I. An n-dimensional unit matrix is usually
written as 𝐼𝑛 .

That is,
Page 37 of 83
 Linear Algebra

1 0 0 ⋯ 0
0 1 0 ⋯ 0
𝐼𝑛 = 0 0 1 ⋯ 0
⋮ ⋮ ⋮ ⋱ ⋮
(0 0 0 ⋯ 1)
A square matrix with zero entries except along the main diagonal is called diagonal
matrix. That is,
∗ 0 ⋯ 0
0 ∗ ⋯ 0
𝐴=( ,
⋮ ⋮ ⋱ ⋮
0 0 ⋯ ∗
Where *are real constants.

Diagonal matrices are usually written as 𝑑𝑖𝑎𝑔 ,𝐴-.

3.2 Trace

Trace of a square matrix A, is the sum of the diagonal entries of matrix 𝐴 =

[𝑎𝑖𝑗 ]𝑛×𝑛

𝑡𝑟(𝐴) = ∑𝑚
𝑖=1 𝑎𝑖𝑖 (3.3)
Example 3.2

Compute the trace of the matrix

−5 1 3
(7 0 4+
8 −1 6
Solution

−5 1 3
𝑡𝑟 ( 7 0 4+ = −5 + 0 + 6 = 1
8 −1 6
3.4 Transpose of Matrix

Interchanging rows and columns of a matrix, in such a way that the first row
become the first column of the new matrix, second row become second column and

Page 38 of 83
 Linear Algebra

so on. The new matrix is said to be the transpose of the former. That is, suppose
𝐴 = (𝑎𝑖𝑗 )𝑛×𝑚 then

𝐴𝑇 = (𝑎𝑗𝑖 )𝑛×𝑚
(3.6)

If
𝑎11 𝑎12 𝑎13 ⋯ 𝑎1𝑛
𝑎21 𝑎22 𝑎23 ⋯ 𝑎2𝑛
𝐴=( ⋮ ⋮ ⋮ ⋮ ,,
𝑎𝑚1 𝑎𝑚2 𝑎𝑚3 ⋯ 𝑎𝑚𝑛

then
𝑎11 𝑎12 ⋯ 𝑎𝑚1
𝑎12 𝑎22 ⋯ 𝑎𝑚2
𝐴 = 𝑎13
𝑇 𝑎23 ⋯ 𝑎𝑚3
⋮ ⋮ ⋮
(𝑎1𝑛 𝑎2𝑛 ⋯ 𝑎𝑚𝑛 )

Example 3.3

Give the transpose of the matrix

−5 1 3
𝐵=( 7 0 4+
8 −1 6
Solution
−5 7 8
𝑇
𝐵 =( 1 0 −1+
3 4 6
Properties of Matrix Transpose
(a) (𝐴𝑇 )𝑇 = 𝐴
(b) (𝐴 𝐵)𝑇 = 𝐴𝑇 𝐵𝑇
(c) (𝐾𝐴)𝑇 = 𝐾𝐴𝑇
(d) (𝐴𝐵)𝑇 = 𝐵𝑇 𝐴𝑇
(e) (𝐴𝑇 ) 1 = (𝐴 1 )𝑇

3.4.1 Symmetric and Asymmetric Matrices

Page 39 of 83
 Linear Algebra

Matrix A is said to be symmetric if 𝐴 = 𝐴𝑇 and asymmetric if 𝐴 = −𝐴𝑇 .

Example 3.4

Observe that the given matrices satisfy the above definitions

1 7 2
(i) 𝐴 = (7 0 3+ is symmetric
2 3 5
0 2 3
(ii) 𝐴 = (−2 0 6+ is Asymmetric
−3 −6 0

Properties of symmetric matrix

If A and B are symmetric and 𝛼 a scalar then

(a) 𝐴𝑇 is symmetric
(b) 𝐴 𝐵 is symmetric
(c) 𝛼𝐴 is symmetric
(d) 𝐴 1 is symmetric

3.5 Matrix Addition

Algebraic Operations such as addition and subtraction can be carried out between
two or more matrices of equal sizes.

That is, given matrices

𝐴 = [𝑎𝑖𝑗 ]𝑚×𝑛 , 𝐵 = [𝑏𝑖𝑗 ]𝑚×𝑛 and 𝐶 = [𝑐𝑖𝑗 ]𝑚×𝑛 then the following operations holds

(i) 𝐴 + 𝐵 = [𝑎𝑖𝑗 ]𝑚×𝑛 [𝑏𝑖𝑗 ]𝑚×𝑛 = [𝑎𝑖𝑗 𝑏𝑖𝑗 ]𝑚×𝑛

(ii) (𝐴 + 𝐵) 𝐶 = [𝑎𝑖𝑗 + 𝑏𝑖𝑗 ]𝑚×𝑛 [𝑐𝑖𝑗 ]𝑚×𝑛 = [(𝑎𝑖𝑗 𝑏𝑖𝑗 ) − 𝑐𝑖𝑗 ]𝑚×𝑛

Example 3.5

Perform the operations on the following matrices if possible and state if not.

Page 40 of 83
 Linear Algebra

−5 1 3 −1 0 3
−1 0 3 7
𝐴=( 7 0 4+ , 𝐵 = ( 6 −1 2+ , 𝐶 = . / , 𝐷=
6 −1 2 −9
8 −1 6 7 −9 1
1 3 3 8
. /
7 4 −1 6
(i) 𝐴−𝐶
(ii) 𝐴+𝐵
(iii) 𝐵+𝐷
(iv) 𝐷−𝐶
(v) 𝐶+𝐷

Solution
(i) 𝐴 − 𝐶 = [𝑎𝑖𝑗 ]3×3 − [𝑐𝑖𝑗 ]2×4
The operations can not hold because they are of different sizes

−6 1 6
(ii) 𝐴 + 𝐵 = [𝑎𝑖𝑗 ] + [𝑏𝑖𝑗 ] = ( 13 −1 6+
3×3 3×3
15 −10 7

(iii) 𝐵 + 𝐷 = [𝑏𝑖𝑗 ]3×3 + [𝑑𝑖𝑗 ]2×4 Not Possible

2 3 0 1
(iv) 𝐷 − 𝐶 = [𝑑𝑖𝑗 ]2×4 − [𝑐𝑖𝑗 ]2×4 = . /
1 5 −3 15

0 3 6 15
(v) 𝐶 + 𝑐 = [𝑐𝑖𝑗 ]2×4 − [𝑑𝑖𝑗 ]2×4 = . /
13 3 1 −3

Algebraic operation of multiplying a matrix by real or complex number known as

scalar is done by multiplying each element of the matrix by the constant.

Let 𝛼 be the scalar (say),

𝛼𝐵 = 𝛼[𝑏𝑖𝑗 ]𝑚×𝑛 = [𝛼𝑏𝑖𝑗 ]𝑚×𝑛

The matrix [𝛼𝑏𝑖𝑗 ]𝑚×𝑛 is a scalar multiple of matrix [𝑏𝑖𝑗 ]𝑚×𝑛 .

Example 3.6

Perform the following operations, given

Page 41 of 83
 Linear Algebra

−1 0 3
−1 0 3 7
𝐵 = ( 6 −1 2+ and 𝐶 = . /
6 −1 2 −9
7 −9 1
(i) 3B
(ii) -7C

Solution
−3 0 9
(i) 3𝐵 = [3𝑏𝑖𝑗 ]3×3 = ( 18 −3 6+
21 −27 3
7 0 −21 −49
(ii) −7𝐶 = [−7𝑐𝑖𝑗 ] =. /
2×4 −42 7 −14 63

3.7 Product of two matrices

The product AB of matrices A and B is possible if their sizes are such that, matrix
𝐴 = [𝑎𝑖𝑗 ]𝑚×𝑝 and 𝐵 = [𝑏𝑖𝑗 ]𝑝×𝑛 . That is the column of matrix A must be equal to
the rows of matrix B for the product AB to be possible and the size of matrix AB is
𝑚 × 𝑛.

The entries of matrix ,𝐴𝐵-𝑚×𝑛 is obtained as follows

For matrix
𝑎11 𝑎12 𝑎13 ⋯ 𝑎1𝑝
𝑎21 𝑎22 𝑎23 ⋯ 𝑎2𝑝
𝐴=( ⋮ ⋮ ⋮ ⋮ ,
𝑎𝑚1 𝑎𝑚2 𝑎𝑚3 ⋯ 𝑎𝑚𝑝

and
𝑏11 𝑏12 𝑏13 ⋯ 𝑎1𝑛
𝑏 𝑏22 𝑏23 ⋯ 𝑎2𝑛
𝐵 = ( 21
⋮ ⋮ ⋮ ⋮ ,,
𝑏𝑝1 𝑏𝑝2 𝑏𝑝3 ⋯ 𝑎𝑝𝑛

then,

𝐴𝐵 = [𝑎𝑖𝑝 𝑏𝑝𝑗 ]𝑚×𝑛 = (𝑎11 𝑏11 + 𝑎12 𝑏21 + ⋯ + 𝑎1𝑝 𝑏𝑝1 )𝑖1 .

The first element on first row, first column is given as

Page 42 of 83
 Linear Algebra

,𝐴𝐵-11 = 𝑎11 𝑏11 + 𝑎12 𝑏21 + ⋯ + 𝑎1𝑝 𝑏𝑝1 = ∑ 𝑎1𝑘 𝑏𝑝1

𝑘=1

Second element on row one is

𝑝

,𝐴𝐵-12 = 𝑎11 𝑏12 + 𝑎12 𝑏22 + ⋯ + 𝑎1𝑝 𝑏𝑝2 = ∑ 𝑎1𝑘 𝑏𝑝2 ;

𝑘=1

Similarly, the pth element on first row is

𝑝

,𝐴𝐵-1𝑝 = 𝑎11 𝑏1𝑛 + 𝑎12 𝑏2𝑛 + ⋯ + 𝑎1𝑝 𝑏𝑝𝑛 = ∑ 𝑎1𝑘 𝑏𝑝𝑛 .

𝑘=1

The product matrix AB, has elements as shown in

∑𝑝𝑘=1 𝑎1𝑘 𝑏𝑝1 ∑𝑝𝑘=1 𝑎1𝑘 𝑏𝑝2 ⋯ ∑𝑝𝑘=1 𝑎1𝑘 𝑏𝑝𝑛

∑𝑝𝑘=1 𝑎2𝑘 𝑏𝑝1 ∑𝑝𝑘=1 𝑎2𝑘 𝑏𝑝2 ⋯ ∑𝑝𝑘=1 𝑎2𝑘 𝑏𝑝𝑛
𝐴𝐵 = (3.4)
⋮ ⋮ ⋮
𝑝 𝑝 𝑝
∑ 𝑎 𝑏 ∑𝑘=1 𝑎𝑚𝑘 𝑏𝑝2 ⋯ ∑𝑘=1 𝑎𝑚𝑘 𝑏𝑝𝑛
( 𝑘=1 𝑚𝑘 𝑝1 )
From (3.4), observe that the ij element of AB is given as

(𝐴𝐵)𝑖𝑗 = ∑𝑝𝑘=1 𝑎𝑖𝑘 𝑏𝑝𝑗 , 𝑖 = 1,2, … , 𝑚; 𝑗 = 1,2, … , 𝑛

(3.5)

(3.5) is called the row-column rule for matrix multiplication.

Example 3.8

−3 0 1
−1 2
−1 0 3 7 6 −1 0
If 𝐴 = ( 1 −3+, 𝐵 = . / and 𝐶 = ( ,
6 −1 2 −9 7 0 0
0 1
1 1 1
Compute (i) 𝐴 ∙ 𝐵 (ii) CA (iii) BC

Solution

Page 43 of 83
 Linear Algebra

−1 2
−1 0 3 7
(i) 𝐴𝐵 = ( 1 −3+ . /
6 −1 2 −9
0 1
1 + 12 0 − 2 −3 + 4 −7 − 18
= (−1 − 18 0 + 3 3−6 7 + 27 +
0+6 0−1 0+2 0−9
13 −2 1 −25
= (−19 3 −3 34 +
6 −1 2 −9

𝐴 = [𝑎𝑖𝑗 ]3×2 , 𝐵 = [𝑏𝑖𝑗 ]2×4 , AB is of size 3 × 4

−3 0 1
−1 2
6 −1 0
(i) 𝐶𝐴 = ( , ( 1 −3+
7 0 0
0 1
1 1 1
3 + 0 + 0 −6 + 0 + 1
−6 − 1 + 0 12 + 3 + 0
=( ,
−7 + 0 + 0 14 + 0 + 0
−1 + 1 + 0 2 − 3 + 1
3 −5
−7 15
=( ,
−7 14
0 0
−3 0 1
−1 0 3 7 6 −1 0 31 7 6
(ii) 𝐵𝐶 = . /( ,=. /
6 −1 2 −9 7 0 0 −19 −8 −3
1 1 1

Class Exercise
1 7 2 1 0 2
1. Given 𝐴 = (3 1 0+ , 𝐵 = (3 2 1+
4 3 5 2 1 2

Compute (i) AB (ii) BA

3 2
1 7 2
2. Given 𝐴 = (1 4 + , 𝐵=. /
3 1 0
2 10

Compute (i) AB (ii) BA. In each case give the size of the new matrix.

Properties of Matrix Arithmetic

Given that the sizes of matrices are valid for the following operations, then the
following holds. Where a, b, and c are scalars.

Page 44 of 83
 Linear Algebra

(1) 𝐴 + 𝐵 = 𝐵 + 𝐴 (Commutative law for matrix addition)

(2) 𝐴 + (𝐵 + 𝐶) = (𝐴 + 𝐵) + 𝐶 (Associative law for matrix addition)
(3) 𝐴(𝐵𝐶) = (𝐴𝐵)𝐶 (Associative law for matrix multiplication)
(4) 𝐴(𝐵 𝐶) = 𝐴𝐵 𝐴𝐶 (left distributive law)
(5) (𝐴 𝐵)𝐶 = 𝐴𝐶 𝐵𝐶 (right distributive law)
(6) 𝑎(𝐵 𝐶) = 𝑎𝐵 𝑎𝐶
(7) (𝑎 𝑏)𝐶 = 𝑎𝐶 𝑏𝐶
(8) 𝑎(𝑏𝐶) = (𝑎𝑏)𝐶
(9) 𝑎(𝐵𝐶) = 𝐵(𝑎𝐶)

3.8 Orthogonal Matrix

A matrix B is said to be orthogonal if

𝐵𝑇 = 𝐵 1
(3.7)

Example 3.9
1 2 2
Show that the matrix 𝐵 = 1/9 ( 2 1 −2+
−2 2 −1
3.10 Inverse Matrix

A matrix A is said to be the inverse of matrix B if the following holds

𝐴𝐵 = 𝐵𝐴 = 𝐼
(3.8)

where I is the identity matrix.

1
Usually, the inverse of matrix B is written as 𝐵

That is,
1
𝐵𝐵 = 𝐵 1𝐵 = 𝐼 (3.9)

1 𝑎𝑑𝑗(𝐵)
𝐵 = |𝐵|
,
(3.10)

Where 𝑎𝑑𝑗(𝑏) is called the adjoint of matrix B while |𝐵| is called the determinant
of matrix B.
Page 45 of 83
 Linear Algebra

1
If |𝐵| = 0, then the matrix B is said to be singular and the inverse 𝐵 does not
exist.

If |𝐵| ≠ 0, then the matrix B is called non singular matrix and it is said to be
invertible.
𝑏11 𝑏12
If in (3.9), 𝐵 = ( *,
𝑏21 𝑏22

Then, |𝐵| = 𝑏11 𝑏22 − 𝑏12 𝑏21

𝑏22 −𝑏12
𝑎𝑑𝑗(𝐵) = ( *,
−𝑏21 𝑏11

Hence, it is invertible if 𝑏11 𝑏22 − 𝑏12 𝑏21 ≠ 0

3.11 Computing Inverse of Matrix Using Elementary Row Operations

The inverse of matrix, say 𝐴 = [𝑎𝑖𝑗 ]𝑚×𝑛 can be derived using the elementary Row
Operations without necessary computing its adjunct and determinant of matrix A.
The process followed in reducing an argument matrix say [A,b], where A is a
square matrix to its canonical form can be used in obtaining A 1 . Matrix A is
invertible if its reduced row echelon does not contain a zero row. If it contains a
zero row then it is singular and hence not invertible. If an n×n matrix is invertible,
then the process of obtaining canonical form of augmented matrix [A,b] can be
used by adjoining unit matrix I n as A, I n  . As elementary row operations is
applied, then [ I n , A1 ] is obtained in finite sequence of steps.

Example 3.10

In example 2.8 question 1, the square matrix in the augment matrix ,𝐴, 𝐵- is

  2 1 1
 
A    2 1 5  , Compute A 1
 1 4 3 


Solution

Page 46 of 83
 Linear Algebra

The canonical form of the augmented matrix is given in example 2.8. The row
operations applied shall be employed to obtain its inverse.

The row operations used in example 2.8 are indicated in the Table 3.1. The inverse
of matrix

 - 2 -1 1 
 
A   - 2 1 5  is found to be
 1 4 3
 

  17 1 3
 
 14 2 7 
1 4
A 1  
11
 14 2 7 
 9 1 2
 
 14 2 7 

TABLE 3.1 Computation of Inverse of Matrix A

  2 1 1 1 0 0
   
  2 1 5  0 1 0
 1 4 3  0 0 1
  

R2  R1  R2 new  1 0 0
 
 1 1 0
2 R3  R1  R3new  1 0 2
 
  2 1 1 
 
 0 2 4
 0 7 7 


7 R2  2 R3  R3new  1 0 0 
 
 1 1 0 
  2 1 1    9 7  4
   
 0 2 4
 0 0 14 


Page 47 of 83
 Linear Algebra

1  1 
R1  R1new  0 0 
2  2 
1  1 1
0 
R2  R2 new  2 
2 2
 9 1 2
1
R2  R3new  
14  14 2 7 

 1 1 
1 
 2 2 
0 1 2 
0 0 1 

 

1
R1  R2  R1  1 1 
 0 
2  4 4 
 1 1
0 
 3  2 2 
1 0  9 1  2
 2   
0 1 2   14 2 7 
0 0 1 

   1 1 
 0 
 4 4 
 3  1 1
0 
1 0   2 2 
 2 
9 1  2
0 1 2 
 
0 0 1   14 2 7 

 

3
R3  R1  R1new   17 1 3
 
2  14 2 7 
 11 1  4
 2 R3  R2  R2 new  14 2 7 
 9 1  2
 
1 0 0  14 2 7 
 
 0 1 0
0 0 1
 

Page 48 of 83
 Linear Algebra

Class Exercise:

Compute the inverse of the following matrices using the elementary row operations

2 2 4    2 2  8
1 7    
i)   ii)   1 3 4  iii)  1 1 0 
 4 1  1  2  3  3 3 2 
   

3.12 Matrix Based Method for Solving Linear Systems

In previous sections, the elementary row operations were used to determine the
solution to the system of linear equation (2.5), that is,

𝐴𝑋 = 𝐵

The two methods are known as Gaussian Elimination and Gauss Jordan methods
respectively.

Other matrix based methods include matrix inversion method, crammers rule, LU
decomposition.
3.12.1 Matrix Inversion Method

Matrix inversion method involves the use of inverse of a matrix. That is,

𝑋 = 𝐴 1𝐵 (3.11)

Example 3.11

Find the solution to the system of linear equations:

−𝑥 + 2𝑦 + 5𝑧 = 1
(i) −3𝑥 + 5𝑦 + 𝑧 = −1
−1 + 2𝑧

Solution
(i) The system 𝐴𝑋 = 𝐵 has matrix
−1 2 5 1 𝑥
𝐴 = (−3 5 1+ , 𝐵 = (−1+ , 𝑋 = (𝑦)
−1 0 2 2 𝑧

Page 49 of 83
 Linear Algebra

The inverse of A is
1
1 10 −4 23
𝐴 = (5 3 −14+,
25
5 −2 1

Therefore,

1 10 −4 23
1
1
𝑋=𝐴 𝐵= (5 3 −14+ (−1+
25
5 −2 1 2
32
−
25
26
= −
25
9
( 25 )
32 26 9
𝑥=− ,𝑦 = − ,𝑧 =
25 25 25
Exercise

Use the Matrix Inversion method to find the solution to the following systems:
𝑥 + 𝑦 − 5𝑧 = 2
(i) 3𝑥 − 𝑦 − 𝑧 = 5
𝑥 + 2𝑦 + 2𝑧 = 0
x yz 6
(ii) 3 x  3 y  4 z  20
2 x  y  3 z  13
10x  2 y  3 z  23
(iii) 2 x  10 y  5 z  33
3 x  4 y  10 z  41
3x  4 y  5 z  18
(iv) 2 x  y  8 z  13
5 x  2 y  7 z  20

Page 50 of 83
 Linear Algebra

4.0 VECTOR SPACES

Vectors have being thought of as a quantity with two components; magnitude and
direction. Mathematically, we have seen vectors as lists of number in ℝ𝑛 . That is
vector as having components.

Concept of a vector is extended herein by the use of basic properties of vectors in

ℝ𝑛 as axioms. These axioms, when satisfied by a set of object prove that those
objects are like known vectors.

Definition 4.1

A vector space (over ℝ or C) is a set V with two operations + (addition) and ∙

(multiplication by scalar) satisfying the following axioms for all 𝑢, 𝑣 ∈ 𝑉 and
𝑐, 𝑑 ∈ ℝ.
(1) If 𝑢, 𝑣 ∈ 𝑉 then 𝑢 + 𝑣 ∈ 𝑉 (additive closure)
(2) If 𝑢, 𝑣 ∈ 𝑉 then 𝑢 + 𝑣 = 𝑣 + 𝑢 ∈ 𝑉 (additively commutative)
(3) (𝑢 + 𝑣) + 𝑤 = 𝑢 + (𝑣 + 𝑤) (additive associativity)
(4) There is a special vector 0 ∈ 𝑉, such that 𝑢 + 0 = 𝑢 ∈ 𝑉 for all u in V
(5) For every 𝑢 ∈ 𝑉 there exist 𝑤 ∈ 𝑉 such that 𝑢 + 𝑤 = 0 (additive inverse)
that is, 𝑤 = −𝑢 for this to hold
(6) For 𝑐 ∈ ℝ and 𝑣 ∈ 𝑉, 𝑐 ∙ 𝑣 ∈ 𝑉 (scalar multiplication)
(7) 𝑐, 𝑑 ∈ ℝ and 𝑣 ∈ 𝑉 then (𝑐 + 𝑑) ∙ 𝑣 = 𝑐 ∙ 𝑣 + 𝑑 ∙ 𝑣 (distributive)
(8) 𝑐 ∈ ℝ, 𝑢, 𝑣 ∈ 𝑉 then 𝑐 ∙ (𝑢 + 𝑣) = 𝑐 ∙ 𝑢 + 𝑐 ∙ 𝑣
(9) 𝑐, 𝑑 ∈ ℝ and 𝑣 ∈ 𝑉 then (𝑐𝑑) ∙ 𝑣 = 𝑐 ∙ (𝑑 ∙ 𝑣) (distributive)
(10) 1 ∙ 𝑣 = 𝑣 for all 𝑣 ∈ 𝑉

Example (Schaum’s Series (1991))

4.1 Space 𝑲𝒏

Let K be an arbitrary field. The notation 𝐾 𝑛 is used to denote the set of n-tuples of
elements in K. Here, 𝐾 𝑛 is viewed as a vector space over K, where vector addition
and scalar multiplication are defined by

(𝑎1 , 𝑎2 , … , 𝑎𝑛 ) + (𝑏1 , 𝑏2 , … , 𝑏𝑛 ) = (𝑎1 + 𝑏1 , 𝑎2 + 𝑏2 , … , 𝑎𝑛 + 𝑏𝑛 )

and

𝑘(𝑎1 , 𝑎2 , … , 𝑎𝑛 ) = (𝑘𝑎1 , 𝑘𝑎2 , … , 𝑘𝑎𝑛 )

The zero vector in 𝐾 𝑛 is the n-tuple of zeros

Page 51 of 83
 Linear Algebra

0 = (0, 0, … , 0𝑛 )

and the negative of a vector is defined as

−(𝑎1 , 𝑎2 , … , 𝑎𝑛 ) = (−𝑎1 , −𝑎2 , … , −𝑎𝑛 )

The axioms can be shown to be satisfied

4.2 Matrix Space 𝑴𝒎,𝒏

𝑀𝑚,𝑛 is the set of all 𝑚 × 𝑛 matrices over an arbitrary field K.

Then 𝑀𝑚×𝑛 is a vector space over K with respect to the usual operations of matrix
addition and multiplication.

Proof

Let 𝐴, 𝐵, 𝐶 ∈ 𝑀𝑚×𝑛 and any scalar 𝑐, 𝑑 ∈ 𝐾, then

(i) 𝐴 + 𝐵 ∈ 𝑀𝑚×𝑛
(ii) (𝐴 + 𝐵) + 𝐶 = 𝐴 + (𝐵 + 𝐶)
(iii) 𝐴+0=𝐴
(iv) 𝐴 + (−𝐴) = 0
(v) 𝐴+𝐵 =𝐵+𝐴
(vi) 𝑐(𝐴 + 𝐵) = 𝑐𝐴 + 𝑐𝐵
(vii) (𝑐 + 𝑑)𝐴 = 𝑐𝐴 + 𝑑𝐴
(viii) (𝑐𝑑)𝐴 = 𝑐(𝑑𝐴)
(ix) 1∙𝐴 =𝐴

Example 4.1

The vector space of 2x2 matrix

Let V be a set of 2x2 matrices with real entries and take the vector space operations
on V to be the usual matrix addition and scalar multiplication. That is,
𝑢11 𝑢12 𝑣11 𝑣12 𝑢11 + 𝑣11 𝑢12 + 𝑣12
𝑢 + 𝑣 = .𝑢 𝑢22 / + .𝑣21 𝑣22 / = .𝑢21 + 𝑣21 𝑢22 + 𝑣22 /
21

𝑢11 𝑢12 𝑘𝑢11 𝑘𝑢12

𝑘𝑢 = 𝑘 .𝑢 𝑢22 / = ( *
21 𝑘𝑢21 𝑘𝑢22
Page 52 of 83
 Linear Algebra

The end results of both operations resulted into a 2x2 matrix. Both operations show
that the set V is closed under addition and scalar multiplication.

Class Exercise

Verify that the axioms of vector space are satisfied. Hint take
0 0 −𝑣11 −𝑣12
0=0 1 𝑎𝑛𝑑 − 𝑣 = .−𝑣 −𝑣22 /
0 0 21

Note: Denote the space by the symbol 𝑀𝑚𝑛 , of which the 2x2 matrix space is
denoted by 𝑀22 .

Home work
(i) Show that the set of all polynomials p(t) form a vector space over field R with respect to the
usual operations of addition of polynomials and multiplication of polynomials by a constant
(𝑡) = 𝑎 + 𝑎1 𝑡 + 𝑎2 𝑡 2 + ⋯ + 𝑎𝑛 𝑡 𝑛 , 𝑛 = 0,1,2, … ; 𝑎𝑖 ∈ , 1 = 1,2, …

Theorem 4.1

Suppose W is a subset of a vector space V, then W is a subspace of V if and only if

the following holds:
(i) 0∈ .
(ii) W is closed under vector addition, that is for every 𝑢, 𝑣 ∈ , then 𝑢 + 𝑣 ∈ .
(iii) W is closed under scalar multiplication that is, for every 𝑢 ∈ , 𝑐 ∈ 𝐾, the multiple 𝑐𝑢 ∈ .

See other examples of vector spaces in relevant texts.

It is important to state thus, that imposing any two operations on any set V would
lead the set to not fulfilling axioms of vector space.

Example 4.2

If V is the set of n-tuples with positive components, and if the standard operations
from ℝ𝑛 are used, then V is not closed under scalar multiplication, due to the fact
that for a nonzero u in V, a scalar multiplied ku will contain negative component if
k is a negative scalar and hence the end result will not be in V.

Page 53 of 83
 Linear Algebra

4.3 Subspaces

A subset W of a vector space V is called a subspace of V if W is itself a vector

space under the addition and scalar multiplication defined on V.

Theorem 4.3

If W is a set of one or more vectors in a vector space V, then W is a subspace of V

if and only if the following conditions are satisfied
(a) If u and v are vectors in W, then 𝑢 + 𝑣 is in W
(b) If k is a scalar and u is a vector in W, the 𝑘𝑢 is in W

Proof (see recommended text)

Theorem 4.4

If 1 , 2 , … , 𝑟 are subspaces of a vector space V, then the intersection of the

subspaces (that is 1 ∩ 2 ∩ … ∩ 𝑟 ) is also a subspace of V.

Proof (see recommended text)

4.3.1 Example of Subspaces

(1) Let 𝑉 = 𝑀𝑛×𝑛 the space of 𝑛 × 𝑛 matrices. Then the subset W of (upper) triangular matrices and
the subset of symmetric matrices are subspaces of V, since they are non empty and closed under
matrix addition and scalar multiplication.
(2) (𝑡) is the vector space of polynomials. If (𝑡) denote polynomials of degree 𝑠 for any fixed
s, then (𝑡) is a subspace of (𝑡)

4.4 Linear combination

Definition 4.2

If w is vector in a vector space V, then w is said to be a linear combination of the

vectors 𝑣1 , 𝑣2 , … , 𝑣𝑟 in V, if w can be expressed in the form

𝑤 = 𝜆1 𝑣1 + 𝜆2 𝑣2 + ⋯ + 𝜆𝑟 𝑣𝑟

Where 𝜆1 , 𝜆2 , … , 𝜆𝑟 are scalars.

Page 54 of 83
 Linear Algebra

These scalars are called the coefficients of the linear combinations.

Example 4.3 (Linear Combinations)

Consider the vectors 𝑢 = (1, 2, −1) and 𝑣 = (6, 4, 2) in ℝ3

(i) Show that 𝑤 = (9, 2, 7) is a linear combination of u and v
(ii) Show that 𝑣 = (4, −1, 8) is not a linear combination of u and v

Solution
(i) In order for W to be a linear combination of u and v, there must exist 𝜆1 and 𝜆2 such that

(9, 2, 7) = 𝜆1 (1, 2, −1) + 𝜆2 (6, 4, 2)

(9, 2, 7) = (𝜆1 , 2𝜆1 , −𝜆1 ) + (6𝜆2 , 4𝜆2 , 2𝜆2 )

(9, 2, 7) = (𝜆1 + 6𝜆2 , 2𝜆1 + 4𝜆2 , −𝜆1 + 2𝜆2 )

That is,

𝜆1 + 6𝜆2 = 9 (i)

2𝜆1 + 4𝜆2 = 2 (ii)

−𝜆1 + 2𝜆2 = 7 (iii)

From (i), 𝜆1 = 9 − 6𝜆2

Substituting 𝜆1 into (iii) yields

−9 + 6𝜆2 + 2𝜆2 = 7

8𝜆2 = 16

𝜆2 = 2

Hence, 𝜆1 = 9 − 6(2) = −3

Clearly if 𝜆1 = −3 and 𝜆2 = 2 satisfy (ii)

2(−3) + 4(2) = −6 + 8 = 2

Therefore, 𝜆1 = −3 and 𝜆2 = 2 is a solution

Page 55 of 83
 Linear Algebra

Hence, W is a linear combination of u and v

(ii) 𝑞 = 𝜆1 𝑢 + 𝜆2 𝑣
(4, −1, 8) = 𝜆1 (1, 2, −1) + 𝜆2 (6, 4, 2)

𝜆1 + 6𝜆2 = 4 (i)

2𝜆1 + 4𝜆2 = −1 (ii)

−𝜆1 + 2𝜆2 = 8 (iii)

From (i), 𝜆1 = 4 − 6𝜆2

Substituting 𝜆1 into (iii) yields

−4 + 6𝜆2 + 2𝜆2 = 8

8𝜆2 = 12
3
𝜆2 =
2
3
𝜆1 = 4 − 6 ( * = 4 − 9 = −5
2
Check if (ii) is satisfied by
3
𝜆1 = −5 and 𝜆2 =
2

3
2(−5) + 4 ( * = −10 + 6 = −4
2
Since this is not true it means that there is no 𝜆1 and 𝜆2 that satisfy the equation
𝑞 = 𝜆1 𝑢 + 𝜆2 𝑣

Hence, q is not a linear combination of u and v.

Theorem 4.5

If 𝑆 = *𝑤1 , 𝑤2 , … , 𝑤𝑟 + is a non-empty set of vectors in a vector space V, then:

Page 56 of 83
 Linear Algebra

The set W of all possible linear combinations of the vectors in S is a subspace of V

Proof

Le W be the set of all possible linear combinations of the vectors in S.

The requirement here is to show that W is closed under addition and scalar
multiplication

To prove closure under addition, let

𝑢 = 𝛼1 𝑤1 + 𝛼2 𝑤2 + ⋯ + 𝛼𝑟 𝑤𝑟 and 𝑣 = 𝜆1 𝑤1 + 𝜆2 𝑤2 + ⋯ + 𝜆𝑟 𝑤𝑟 be two vectors

in W.

It follows that

𝑢 + 𝑣 = (𝛼1 + 𝜆1 )𝑤1 + (𝛼2 + 𝜆2 )𝑤2 + ⋯ + (𝛼𝑟 + 𝜆𝑟 )𝑤𝑟

Which is a linear combination of vectors, in S. thus, W is closed under addition

Next, let k be a scalar, then

𝑘𝑢 = 𝑘𝛼1 𝑤1 + 𝑘𝛼2 𝑤2 + ⋯ + 𝑘𝛼𝑟 𝑤𝑟

Hence, W is closed under scalar multiplication.

Therefore, W is a subspace of V.

Exercise
1. Let 2 be the set of all polynomials of degree less than or equal to 2.
Let 𝑣1 = 𝑡 2 − 1, 𝑣2 = 𝑡 2 + 3𝑡 − 5 And 𝑣3 = 𝑡 be vectors in 2 . Show that the quadratic
polynomial 𝑥 = 7𝑡 2 − 15 is a linear combination of *𝑣1 , 𝑣2 , 𝑣3 +.
2. Let 2 be the set of all polynomials of degree less than or equal to 2.
Let 𝑝1 = 𝑡 2 + 2𝑡 − 1, 𝑝2 = 2𝑡 + 1 and 𝑝3 = 5𝑡 2 + 2𝑡 − 3 be vectors in 2 . Show that the
following are linear combinations of these vectors 𝑝1 , 𝑝2 and 𝑝3
a. 𝑥 = 4𝑡 2 − 2𝑡 − 3
b. 𝑥 = − 2𝑡 2 − 2
c. 𝑥 = 6

4.5 Linear Span

Definition 4.3

Page 57 of 83
 Linear Algebra

If 𝑆 = *𝑤1 , 𝑤2 , … , 𝑤𝑟 + is a non-empty set of vectors in vector space V, then the

subspace W of V that consists all possible linear combinations of the vectors in S is
called the subspace of V generated by S, and we say that the vectors 𝑤1 , 𝑤2 , … , 𝑤𝑟
spans W.

We denote this subspace as

= 𝑆𝑝𝑎𝑛*𝑤1 , 𝑤2 , … , 𝑤𝑟 +

or = 𝑆𝑝𝑎𝑛(𝑆)

In other words, for a non-empty set 𝑆 = *𝑤1 , 𝑤2 , … , 𝑤𝑟 +, of a vector space all

possible linear combinations of vectors in S yields a subspace of the vector space V
generated by S.

Example 4.4
1 0
The standard unit vector 𝑒1 = . / and 𝑒2 = . / span ℝ2 because a linear
0 1
combination of these vectors 𝑒1 and 𝑒2 can produce any vector in ℝ2

Example 4.5

Determine whether the vectors 𝑣1 = (1, 1, 2), 𝑣2 = (1, 0, 1) and 𝑣3 = (2, 1, 3)

span the vector space ℝ3 .

Solution

To test if *𝑣1 , 𝑣2 , 𝑣3 + spans the vector space ℝ3

We check if an arbitrary vector 𝑝 = (𝑝1 , 𝑝2 , 𝑝3 ) in ℝ3 can be expresses as a linear

combination of 𝑣1 , 𝑣2 𝑎𝑛𝑑 𝑣3 .

𝑝 = 𝜆1 𝑣1 + 𝜆2 𝑣2 + 𝜆3 𝑣3

That is,

𝑝 = 𝜆1 (1, 1, 2) + 𝜆2 (1, 0, 1) + 𝜆3 (2, 1, 3)

𝑝1 = 𝜆1 + 𝜆2 + 2𝜆3

𝑝2 = 𝜆1 + 𝜆3

Page 58 of 83
 Linear Algebra

𝑝3 = 2𝜆1 + 𝜆2 + 3𝜆3

This can be written as

1 1 2 𝜆1 𝑝1
(1 0 1+ (𝜆2 + = (𝑝2 +
2 1 3 𝜆3 𝑝3

a system of linear equation. The task will be to find out if the system is consistent.

Verify that it is not consistent, hence, the vectors 𝑣1 , 𝑣2 𝑎𝑛𝑑 𝑣3 do not span ℝ3

Example 4.6

The standard unit vectors in ℝ𝑛 are of the form

𝑒1 = (1, 0, 0, … , 0), 𝑒2 = (0, 1, 0, … , 0), … , 𝑒𝑛 = (0, 0, 0, … , 1)

Every vector 𝑢 = (𝑢1 , 𝑢2 , … , 𝑢𝑛 ) in ℝ𝑛 can be expressed as

𝑢 = 𝑢1 𝑒1 + 𝑢2 𝑒2 + ⋯ + 𝑢𝑛 𝑒𝑛

That is, u is a linear combination of vectors 𝑒1 , 𝑒2 , … , 𝑒𝑛

Hence, the unit vectors 𝑒1 , 𝑒2 , … , 𝑒𝑛 spans ℝ𝑛

Example 4.7

The vectors 𝑖 = (1, 0, 0), 𝑗 = (0, 1, 0) and 𝑘 = (0, 0, 1) spans ℝ3 , since every
vector 𝑤 = (𝑤1 , 𝑤2 , 𝑤3 ) in the ℝ3 space can be expressed as

𝑤 = (𝑤1 , 𝑤2 , 𝑤3 ) = 𝑤1 (1, 0, 0) + 𝑤2 (0, 1, 0) + 𝑤3 (0, 0, 1)

= 𝑤1 𝑖 + 𝑤2 𝑗 + 𝑤3 𝑘

Example 4.8

A spanning set for 𝑛

The polynomials 1, 𝑥, 𝑥 2 , … , 𝑥 𝑛 span the vector space 𝑛 , set of polynomials with

degree less than or equal n. each polynomial q in 𝑛 can be expressed as the linear
combination of 1, 𝑥, 𝑥 2 , … , 𝑥 𝑛 as follows

𝑞 = 𝑎 + 𝑎1 𝑥 + ⋯ + 𝑎𝑛 𝑥 𝑛
Page 59 of 83
 Linear Algebra

Hence, 𝑛 = 𝑆𝑝𝑎𝑛*1, 𝑥, 𝑥 2 , … , 𝑥 𝑛 +.

4.6 Linear Independence

Definition 4.4

If 𝑆 = *𝑣1 , 𝑣2 , … , 𝑣𝑟 + is a set of two or more vectors in a vector space V, then S is

said to be linearly independent set if no vector in S can be expressed as a linear
combination of the others.

If a set is not linearly independent, it is said to be linearly dependent.

Theorem 4.6

A non-empty set 𝑆 = *𝑣1 , 𝑣2 , … , 𝑣𝑟 + in a vector space V is linearly independent, if

and only if the coefficients satisfying the vector equation

𝜆1 𝑣1 + 𝜆2 𝑣2 + ⋯ + 𝜆𝑟 𝑣𝑟 = 0

are such that 𝜆1 = 𝜆2 = ⋯ = 𝜆𝑟 = 0

If there is any scalar 𝜆𝑖 ≠ 0, 𝑖 = 1,2, … , 𝑟. Then the vectors are said to be linearly
dependent.

Example 4.8

Linear independence of the standard unit vectors in ℝ𝑛 .

The most basic linearly independent set in ℝ𝑛 is the set of standard unit vectors

𝑒1 = (1, 0, 0, … , 0), 𝑒2 = (0, 1, 0, … , 0), … , 𝑒𝑛 = (0, 0, 0, … , 1)

Example 4.9

If in example above, we are in ℝ3 , then 𝑒1 = (1, 0, 0), 𝑒2 = (0, 1, 0) and 𝑒3 =

(0, 0, 1)

𝑒1 = 𝑖, 𝑒2 = 𝑗 𝑎𝑛𝑑 𝑒3 = 𝑘

To prove linear independence,

Page 60 of 83
 Linear Algebra

𝜆1 𝑒1 + 𝜆2 𝑒2 + 𝜆3 𝑒3 = 0

(𝜆1 , 0, 0) + (0, 𝜆2 , 0) + (0, 0, 𝜆3 )

(𝜆1 , 𝜆2 , 𝜆3 ) = (0, 0, 0)

∴ 𝜆1 = 0, 𝜆2 = 0 𝑎𝑛𝑑 𝜆3 = 0

Hence, 𝑒1 , 𝑒2 and 𝑒3 are linearly independent.

Example 4.10

Determine if the vectors 𝑣1 = (1, −2, 3), 𝑣2 = (5, 6, −1) and 𝑣3 = (3, 2, 1) are
linearly independent

Solution

𝜆1 𝑣1 + 𝜆2 𝑣2 + 𝜆3 𝑣3 = 0

⟹ (𝜆1 + 5𝜆2 + 3𝜆3 , −2𝜆1 + 6𝜆2 + 2𝜆3 , 3𝜆1 − 𝜆2 + 𝜆3 ) = (0, 0, 0)

𝜆1 + 5𝜆2 + 3𝜆3 = 0

−2𝜆1 + 6𝜆2 + 2𝜆3 = 0

3𝜆1 − 𝜆2 + 𝜆3 = 0

Rewrite the system using matrix

1 5 3 𝜆1 0
(−2 6 2|𝜆2 + = (0+
3 −1 1 𝜆3 0
The augmented matrix is
1 5 30
(−2 6 2|0+
3 −1 1 0
Using the appropriate row operations

1 ⟹ 1 1 5 3 0
2 1 + 2 ⟹ 2 (0 16 8 |0+
−3 1 + 3 ⟹ 2 0 −16 −8 0
Page 61 of 83
 Linear Algebra

1 1 5 30
2 (0 16 8|0+
2 + 3 0 0 00
The last row of the augmented matrix is zero, hence the vectors are linearly
dependent. Also, the system has infinitely many solutions as can be seen

Let 𝜆3 = 𝑠

16 𝜆2 + 8𝜆3 = 0
1 1
⟹ 𝜆2 = − 𝜆3 = − 𝑠
2 2
𝜆1 + 5𝜆2 + 3𝜆3 = 0 ⟹ 𝜆1 = 3𝜆3 − 5𝜆2
1 5
⟹ 𝜆1 = −38 − 5 (− 𝑠* = −3𝑠 + 𝑠
2 2
1
= 𝑠
2
𝜆1 −𝑠/2 −1/2
(𝜆2 + = (−𝑠/2+ = 𝑠 (−1/2+ , 𝑓𝑜𝑟 𝑠 ∈ ℝ
𝜆3 1 1
Hence, the vectors 𝑣1 , 𝑣2 , 𝑎𝑛𝑑 𝑣3 𝑎𝑟𝑒 𝑙𝑖𝑛𝑒𝑎𝑟𝑙𝑦 𝑑𝑒𝑝𝑒𝑛𝑑𝑒𝑛𝑡.

Example 4.11

Let V be the vector space of continuous function defined on the real line.
Determine if the vectors 𝐴 = cos 2 𝑥 , 𝐵 = sin2 𝑥 and 𝐶 = 5 are linearly
independent.

Solution

The linear combination

𝜆1 𝐴 + 𝜆2 𝐵 + 𝜆3 𝐶 = 0

Page 62 of 83
 Linear Algebra

For 𝑥 = 0

𝜆1 cos 2 0 + 𝜆2 sin2 0 + 𝜆3 5 = 0

𝜆1 = 0, 𝜆2 ≠ 0 𝑎𝑛𝑑 𝜆3 = 0

Hence, they are linearly dependent.

Alternatively, recall that

cos 2 𝑥 + sin2 𝑥 = 1,

If the above equation is multiplied by 5, obtain

5 cos 2 𝑥 + 5 sin2 𝑥 = 5

It implies that vector C forms a linear combination of A and B. hence, they are
linearly dependent.

Exercise
1. Let V be the vector space of continuous functions defined on the real line.
Let 𝑢 = cos 2 𝑥 , 𝑣 = sin2 𝑥 and 𝑤 = 2 be vectors in V. Show that the vectors u, v and w are
linearly dependent.
2. Show that the following vectors are linearly dependent
(a) 𝑢1 = (−1, 2, 4) and 𝑢2 = (5, −10, −20) in ℝ3
(b) 𝑢1 = (3, −1), 𝑢2 = (4, 5) and 𝑢3 = (−4, 7) in ℝ2
3. Determine if the vectors (−3, 0, 4), (5, −1, 2) and (1, 1, 3) in ℝ3 are linearly dependent or
linearly independent.
4. Determine if the vectors (1, 1, 1), (1, 2, 3) and (2, −1, 1) in ℝ3 are linearly dependent or linearly
independent.

4.7 Basis vectors

Definition 4.7

Consider the n vectors 𝑣1 , 𝑣2 , … , 𝑣𝑛 in the vector space V, these vectors form a

basis of V, if
(i) 𝑣1 , 𝑣2 , … , 𝑣𝑛 spans V,
(ii) 𝑣1 , 𝑣2 , … , 𝑣𝑛 are linearly independent.

Page 63 of 83
 Linear Algebra

We can write the n vectors 𝑣1 , 𝑣2 , … , 𝑣𝑛 𝑎𝑠 𝑎 𝑠𝑒𝑡

𝐵 = *𝑣1 , 𝑣2 , … , 𝑣𝑛 +

These are called basis vectors.

Definition 4.8

A set. 𝐵 = *𝑣1 , 𝑣2 , … , 𝑣𝑛 + of vectors is a basis of V, if every vector v in V can be

expressed uniquely as a linear combination of the vectors in S.

Example 4.12

Show that the following vectors form a basis for 𝑀22 space

The standard basis for 𝑀22 are

1 0 0 1 0 0 0 0
𝐴=. /,𝐵 = . /,𝐶 = . / , 𝑎𝑛𝑑 𝐷 = . /
0 0 0 0 1 0 0 1
The task will be to show that these vectors spans 𝑀22 and that they are linearly
independent.

(𝑖)𝜆1 𝐴 + 𝜆2 𝐵 + 𝜆3 𝐶 + 𝜆4 𝐷 = 0, 𝑖𝑚𝑝𝑙𝑖𝑒𝑠
1 0 0 1 0 0 0 0 0 0
𝜆1 . / + 𝜆2 . / + 𝜆3 . / + 𝜆4 . /=. /
0 0 0 0 1 0 0 1 0 0

𝜆1 𝜆2 0 0
( *=. /
𝜆3 𝜆4 0 0
That is 𝜆1 = 𝜆2 = 𝜆3 = 𝜆4 = 0
Hence, the vectors A, B, C and D are linearly independent.

𝑝1 𝑝2
(ii) Let = .𝑝 𝑝4 / be a vector,
3

1 0 0 1 0 0 0 0
Then = 𝑝1 . / + 𝑝2 . / + 𝑝3 . / + 𝑝4 . /
0 0 0 0 1 0 0 1

Hence, vectors A, B, C, and D spans 𝑀22 , and conclude that

A, B, C, and D form a basis for vector space 𝑀22

Page 64 of 83
 Linear Algebra

Example 4.13

Show that the following vectors do not form a basis for 𝑀22 space
1 0 0 1
𝐴=. /,𝐵 = . /
0 1 1 0
Solution
(i.) 𝜆1 𝐴 + 𝜆2 𝐵 = 0

𝜆 0 0 𝜆2 0 0
( 1 *+( *=. /
0 𝜆1 𝜆2 0 0 0
𝜆1 𝜆2 0 0
( *=. /
𝜆2 𝜆1 0 0
Implies 𝜆1 = 𝜆2 = 0. Therefore, vectors A and B are linearly independent.
𝑐1 𝑐2
(ii.) Let 𝐶 = .𝑐 𝑐4 / be a vector in 𝑀22 ,
3

Then
𝑐1 𝑐2
.𝑐
3 𝑐4 / = 𝜆1 𝐴 + 𝜆2 𝐵

implies that
𝜆 𝜆2 𝑐1 𝑐2
( 1 * = .𝑐 𝑐4 /
𝜆2 𝜆1 3

𝜆1 = 𝑐1 , 𝜆2 = 𝑐2 , 𝜆1 = 𝑐4 , 𝜆2 = 𝑐3

Vector A and B span 𝑀22 , if and only if 𝑐1 = 𝑐4 and 𝑐2 = 𝑐3 . Since that is not the
case, the vector A and B do not span 𝑀22 . Therefore, it is not a basis for 𝑀22 .

Theorem 4.7

Let 𝐵 = *𝑣1 , 𝑣2 , … , 𝑣𝑛 + the set be a basis for a vector space V. Every vector in V
can be expressed uniquely as a linear combination of the basis vectors.

Proof

Page 65 of 83
 Linear Algebra

Consider an arbitrary vector q in V. Since 𝐵 = *𝑣1 , 𝑣2 , … , 𝑣𝑛 + is a basis for the

vector space V, there exist scalars 𝜆1 , 𝜆2 , … , 𝜆𝑛 such that

𝜆1 𝑣1 + 𝜆2 𝑣2 + ⋯ + 𝜆𝑛 𝑣𝑛 = 𝑞 (𝑖)

Suppose vector q can be expressed as a linear combination of the basis vector as

𝛼1 𝑣1 + 𝛼2 𝑣2 + ⋯ + 𝛼𝑛 𝑣𝑛 = 𝑞 (𝑖𝑖)

Where 𝛼1 , 𝛼2 , … , 𝛼𝑛 are scalars

Subtracting (ii) from (i), gives

(𝜆1 − 𝛼1 )𝑣1 + (𝜆2 − 𝛼2 )𝑣2 + ⋯ + (𝜆𝑛 − 𝛼𝑛 )𝑣𝑛 = 0

∴ 𝜆1 − 𝛼1 = 0, 𝜆2 − 𝛼2 = 0, … , 𝜆𝑛 − 𝛼𝑛 = 0

𝜆1 = 𝛼1 , 𝜆2 = 𝛼2 , … , 𝜆𝑛 = 𝛼𝑛

Therefore the basis representation of any vector is unique.

Exercise
(1) Show that the set *1, 𝑡 − 1, (𝑡 − 1)2 + forms a basis for the vector space 2 (where 2 is vector
space of polynomials of degree 2)
(2) Write the vector 𝑝 = 𝑡 2 + 1 in terms of the basis vectors *1, 𝑡 − 1, (𝑡 − 1)2 +

4.8 Dimension
Definition 4.8

The number of vectors in a basis of a non-zero vector space V is the dimension of

the vector space

The number of vectors in a basis gives the dimension of the vector space V. it is
usually denoted by dim(𝑉).

Definition 4.9

If n vectors *𝑣1 , 𝑣2 , … , 𝑣𝑛 + form a basis for a vector space V, then dim(𝑉) = 𝑛

4.9 Finite Dimensional Vector Space

Page 66 of 83
 Linear Algebra

Finite dimensional vector space V, is a vector space with a finite number of vectors
in its basis.

Definition 4.10

If a finite number of vectors form a basis for a vector space V, then it is said to be
finite dimensional otherwise infinite dimensional.

Example 4.14
(1) The standard basis for vector space 𝑀22 is
1 0 0 1 0 0 0 0
𝑢=. /,𝑣 = . /,𝑤 = . /,𝑥 = . /
0 0 0 0 1 0 0 1
Hence, dimension dim(𝑀22 ) = 4.
Note, it can be shown that for 𝑀𝑚𝑛 space, the dim(𝑀𝑚𝑛 ) = 𝑚𝑛
(2) The standard basis for the vector space ℝ𝑛 is
1 0 0 0
0 1 0 0
𝑒1 = 0 , 𝑒2 = 0 , 𝑒3 = 1 , … , 𝑒𝑛 = ⋮
⋮ ⋮ ⋮ 0
0
( ) 0
( ) 0
( ) (1)
𝑛
There are n basis in ℝ
Hence, dim(ℝ𝑛 ) = 𝑛

Exercise
1. Find the dimension of the following vector spaces and exhibit a basis for each space
a. ℝ6
b. 𝑀33
c. 𝑀32

4.10 Linear Transformation

A linear transformation (or linear mapping) is a special transformation which

preserves scalar multiplication and vector addition.

Definition 4.11

A function 𝑇: 𝑉 → is a linear transformation, if for all 𝑢, 𝑣 ∈ 𝑉 and 𝜆 ∈

(i) 𝑇(𝑢 + 𝑣) = 𝑇(𝑢) + 𝑇(𝑣) (T preserves vector addition)
(ii) 𝑇(𝜆𝑢) = 𝜆𝑇(𝑢) (T preserves vector scalar multiplication)

Page 67 of 83
 Linear Algebra

Example 4.15
3 3
Let 𝑇: → defined by
𝑥 𝑥+𝑦
𝑇 (𝑦) = ( 𝑦 + 𝑧 +
𝑧 0
3 3
Show that 𝑇: → is a linear transformation.

Solution
𝑢1 𝑣1
Let 𝑢, 𝑣 ∈ 3
that is 𝑢 = (𝑢2 + and 𝑣 = (𝑣2 +
𝑢3 𝑣3
Then
𝑢1 + 𝑣1 (𝑢1 + 𝑣1 ) + (𝑢2 + 𝑣2 )
𝑇(𝑢 + 𝑣) = 𝑇 (𝑢2 + 𝑣2 + = ((𝑢2 + 𝑣2 ) + (𝑢3 + 𝑣3 )+
𝑢3 + 𝑣3 0
(𝑢1 + 𝑣1 ) + (𝑢2 + 𝑣2 )
= ((𝑢2 + 𝑣2 ) + (𝑢3 + 𝑣3 )+ … … … (𝑖)
0
Also,
𝑢1 + 𝑢2 𝑣1 + 𝑣2
𝑇(𝑢) = (𝑢2 + 𝑢3 + , 𝑇(𝑣) = (𝑣2 + 𝑣3 +
0 0
𝑢1 + 𝑢2 𝑣1 + 𝑣2
𝑇(𝑢) + 𝑇(𝑣) = (𝑢2 + 𝑢3 + + (𝑣2 + 𝑣3 +
0 0
𝑢1 + 𝑢2 + 𝑣1 + 𝑣2
= (𝑢2 + 𝑢3 + 𝑣2 + 𝑣3 + … … … (𝑖𝑖)
0

It follows that 𝑇(𝑢 + 𝑣) = 𝑇(𝑢) + 𝑇(𝑣)

Also,
Page 68 of 83
 Linear Algebra

𝜆𝑢1 𝜆𝑢1 + 𝜆𝑢2 𝜆(𝑢1 + 𝑣1 ) 𝑢1 + 𝑢2

𝑇(𝜆𝑢) = 𝑇 (𝜆𝑢2 + = (𝜆𝑢2 + 𝜆𝑢3 + = (𝜆(𝑢2 + 𝑣2 )+ = 𝜆 (𝑢2 + 𝑢3 + … … . (𝑖𝑖𝑖)
𝜆𝑢3 0 0 0
𝑢1 𝑢1 + 𝑢2
𝜆𝑇(𝑢) = 𝜆𝑇 (𝑢2 + = 𝜆 (𝑢2 + 𝑢3 + … … . (𝑖𝑣)
𝑢3 0

Since (iii) = (iv), it follows that 𝑇(𝜆𝑢) = 𝜆𝑇(𝑢)

Observe that
𝑥 𝑥+𝑦 1 1 0 𝑥
𝑇 (𝑦) = ( 𝑦 + 𝑧 + = (0 1 1+ (𝑦)
𝑧 0 0 0 0 𝑧
Exercise

𝑥 𝑥 1 1
3 3
(1) Show that the transformation 𝑇: → defined by 𝑇 0𝑦1 = 𝐴 .𝑦/ where 𝐴 = (1 −1+ is
0 1
linear
3 3
(2) Show that the transformation 𝑇: → defined by 𝑇(𝑢) = 𝑥 2 is not linear
1 0
(3) Consider the transformation 𝑇: 3 → 3 given by 𝑇(𝑢) = 𝐴𝑢 where 𝐴 = . /
1 1
2 −1 −1
Determine the vector 𝑇(𝑢), where u is given by (i) 𝑢 = . /, (ii) 𝑢 = . /, (iii) 𝑢 = . /,
3 2 5

NULL SPACE

Recall, the system of linear equation (2.5) if the right hand vector B = 0, that is

𝐴𝑋 = 0 (4.1)

The system (4.1) is said to be a homogenous

The system 92.5) is inhomogenous.

Let N(A) be a solution space to the homogenous system (4.1). That is the set of all
solution to the system (4.1)

Definition

The set of vectors X in N(A) which satisfies the homogenous system (4.1) is called
the null space of matrix A.

The dimension of the null space is called the nullity of matrix A.

Page 69 of 83
 Linear Algebra

Example

Determine the null space and the nullity of the 𝐴𝑋 = 0, where matrix
1 5 3 𝑥1
𝐴 = (−2 6 2+ and 𝑋 = (𝑥2 +
3 −1 1 𝑥3

Solution

The augmented matrix of the system is

1 1 5 30
2 (−2 6 2|0+
3 3 −1 10
with appropriate Row operations to obtain equivalent systems

1 1 5 30
2 1 + 2 → 2 (0 16 8|0+
3 1 − 3 = 3 0 16 8 0

1
1 5 30
2 (0 16 8|0+
2− 3 = 3 0 0 00
Hence, the lead variables are 𝑥1 and 𝑥2 and free variable is 𝑥3 . Therefore set
𝑥3 = 𝑠, where 𝑠 ∈ .

16𝑥2 + 8𝑥3 = 0
1
𝑥2 = − 𝑠
2
𝑥1 + 5𝑥2 + 3𝑥3 = 0
5 𝑠
𝑥1 = −5𝑥2 − 3𝑥3 = 𝑠 − 3𝑠 = −
2 3

Page 70 of 83
 Linear Algebra

1
𝑥1 −
2
(𝑥2 + = 𝑠 1
𝑥3 −
2
( 1 )
1
−
2
The solution is a scalar multiple of the vector 𝑋 = (− 1,. X is a basis vector.
2
1
The null space is the set of vectors

𝑁(𝐴) = *𝑠𝑋|𝑠 ∈ ℝ+

The nullity is 1.

Example 2

Determine the null space and the nullity of the system 𝐴𝑋 = 0, where
1 1 2 −1 𝑥1
4 −5 −6 2 𝑥2
𝐴=( , and 𝑋 = (𝑥 ,
1 −8 −12 5 3
2 −7 −10 4 𝑥4

Solution

Put the system in augmented matrix and carryout appropriate row operations to
obtain equivalent row.

1 1 1 2 −1 0
2 4 −5 −6 2 0
( | ,
3 1 −8 −12 5 0
4 2 −7 −10 4 0

11 1 2 −1 0
4 1− 2 0 9 14 −6 0
( | ,
1− 3 0 9 14 −6 0
2 1− 4 0 9 14 −6 0

Page 71 of 83
 Linear Algebra

1 1 1 2 −1 0
2 0 9 14 −6 0
( | ,
2− 3 = 3 0 0 0 0 0
2− 4 = 4 0 0 0 0 0
The leading variables are 𝑥1 and 𝑥2 while 𝑣3 and 𝑣4 are free variables

Therefore, set 𝑥3 = 𝑠 and 𝑥4 = 𝑡

9𝑥2 + 14𝑠 − 6𝑡 = 0

9𝑥2 = 6𝑡 − 14𝑠
6 14
𝑥2 = 𝑡 − 𝑠
9 9
From equation 1

𝑥1 = 𝑣4 − 2𝑣3 − 4𝑣4
6 14
= 𝑡 − 2𝑠 − ( 𝑡 − 𝑠*
9 9
6 14
= 𝑡 − 𝑡 − 2𝑠 + 𝑠
9 9
3 4
= 𝑡− 𝑠
9 9
Hence,

1 4 1 4
𝑥1 𝑡 − 𝑠 −
3 9 3 9
𝑥2 2 14
(𝑥 , = 2 𝑡 − 14 𝑠 = 𝑡 +𝑠 −
3 3 9 3 9
𝑥4 𝑠 0 1
( 𝑡 ) (1) ( 0 )
The solution vector is a scalar multiples of vectors

Page 72 of 83
 Linear Algebra

1 4
−
3 9
2 14
𝑢= and 𝑣 = −
3 9
0 1
(1) ( 0 )
The null space is the set

𝑁(𝐴) = *𝑡𝑢 + 𝑠𝑣|𝑡, 𝑠 ∈ ℝ+

Nullity is 2.

Exercise

Determine the null space and nullity of the following systems

1 −2 −3 𝑥1 0
(i) (4 −5 −6+ (𝑥2 + = (0+
7 −8 −9 𝑥3 0
−3 1 −1 𝑥 0
(ii) (2 5 −7+ (𝑦) = (0+
4 8 −4 𝑧 0

Page 73 of 83
 Linear Algebra

5.0 Eigenvalues and Eigenvectors

Consider the linear transformation

𝑇(𝑢) = 𝐴𝑢 (5.1)

Were A is a square matrix. It is such that

𝑇(𝑢): 𝑢 → 𝜆𝑢
That is,

𝐴𝑢 = 𝜆𝑢 (5.2)

Matrix A transforms vector u by scalar multiplying vector u

The transformation only changes the length of the vector u, unless 𝜆 = 1. for a
negative scalar 𝜆, the direction of the vector is changed.

The task here is to find a non-zero vector u called the Eigenvector corresponding to
the scalar 𝜆 called the eigenvalue.

To do this, we rewrite (5.2) as

𝐴𝑢 = 𝜆𝐼𝑢 (5.3)

Where I is the identity matrix of same size as matrix A. (students are to verify that
𝜆𝐼𝑢 = 𝜆𝑢)

From (5.3),

𝐴𝑢 − 𝜆𝐼𝑢 = 0

(𝐴 − 𝜆𝐼)𝑢 = 0
(5.4)

Equation (5.4) is an homogeneous equation.

The equation (5.4) will have infinitely many solutions, if the last row equivalent
matrix (𝐴 − 𝜆𝐼) is degenerate. That will occur only if the determinant of matrix
(𝐴 − 𝜆𝐼) is zero.

That is,

Page 74 of 83
 Linear Algebra

𝑑𝑒𝑡(𝐴 − 𝜆𝐼) = 0 (5.5)

Note- the need for infinitely many solution is based on the fact that the vector
𝑢 = 0 is a trivial solution to (5.4). Of which our interest is on non-zero vector
solution u.

(5.5) is known as the characteristics equation. The roots of the characteristics

equations are the eigenvalues of square matrix A, while the null space of (5.4)
corresponding to each eigenvalue is the eigenvector u.

Example 5.1

Compute the eigenvectors and eigenvalues of the matrix

1 3
𝐴=. /
5 −1
Solution

Let 𝑢, 𝜆 be the eigenvector and eigenvalues respectively, so that

𝐴𝑢 − 𝜆𝐼𝑢 = 0

(𝐴 − 𝜆𝐼)𝑢 = 0 (i)

substituting A and I into (i) to obtain

1 3 1 0 𝑢1
∴ (. /−𝜆. /) .𝑢 / = 0
5 −1 0 1 2

1−𝜆 3 𝑢1
. / .𝑢 / = 0
5 −1 − 𝜆 2
(ii)

The corresponding characteristics equation is

1−𝜆 3 1−𝜆 3
det . /=| | = (1 − 𝜆)(−1 − 𝜆) − 12 = 0
5 −1 − 𝜆 5 −1 − 𝜆
(−1 − 𝜆 + 𝜆 + 𝜆2 ) − 15 = 0

𝜆2 − 1 − 15 = 𝜆2 − 16 = 0

⟹ 𝜆 = 4 𝑜𝑟 𝜆 = −4
Page 75 of 83
 Linear Algebra

To compute eigenvector u corresponding to 𝜆 = 4,

substitute 𝜆 = 4 into (ii) to obtain

1−4 3 𝑢1
. / .𝑢 / = 0
5 −1 − 4 2

−3 3 𝑢1
⟹. / .𝑢 / = 0
5 −5 2

using appropriate row operation

1 −3 3 𝑢1
. /. / = 0
5 1 +3 2 → 2 0 0 𝑢2

free variable is 𝑢2 . So set 𝑢2 = 𝑡,

obtain

−3𝑢1 + 3𝑢2 = −3𝑢1 + 3𝑡 = 0

𝑢1 = 𝑡
𝑢1 𝑡 1
.𝑢 / = . / = 𝑡 . /
2 𝑡 1
where 𝑡 ≠ 0
1
The eigenvector is . /
1
Eigenvector corresponding to 𝜆 = −4 is obtained as
5 3 𝑢1 0
. / .𝑢 / = . /
5 3 2 0
using appropriate row operation, we have

1 5 3 𝑢1 0
. / .𝑢 / = . /.
1 − 2 → 2 0 0 2 0

Set 𝑢2 = 𝑠,

5𝑢1 + 3𝑢2 = 5𝑢1 + 3𝑠 = 0

Page 76 of 83
 Linear Algebra

3
𝑢1 = − 𝑠
5

𝑢1 3 3
− 𝑠 −
.𝑢 / = ( 5 ) = 𝑠 ( 5+ , 𝑠 ≠ 0
2
𝑠 1

3
−
The eigenvector 𝑢 = ( 5)
1
Exercises

Find the eigenvalues and eigenvectors of the following matrices

7 0
(i) 𝐴=. /
3 −4
−2 5
(ii) 𝐴=. /
−1 4
−1 4
(iii) 𝐴=. /
2 1

5.1 Application of Linear Algebra

Matrices are very useful and efficient way to store information

Example

In computer graphics, you will encounter image files with a gif extension. These
files are actually just matrices: at the start of the file the size of the matrix is given,
and then each entity of the matrix is a number indicating the color of a particular
pixel in the image.

The resulting matrix then has it rows shuffled a bit: by listing say every eighth row,
then a web browser downloading the file can start displaying an incomplete version
of the picture before the download is complete.

Finally, a compression algorithm is applied to the matrix to reduce the size of the
file.

Example

Page 77 of 83
 Linear Algebra

Graphs occur in many applications, ranging from telephone networks to airline

routes. In the subject of graph theory, a graph is just a collection of vertices and
some edges connecting vertices. A matrix can be used to indicate how many edges
attach one vertex to another.

Consider the graph

1 2

3 4
The adjacency matrix for this graph is:

0 1 1 1
 
1 0 0 1
M 
1 0 0 1
 
1 0 
 1 1

Compute 𝑀2

3 1 1 2
 
1 2 2 1
M 
2

1 2 2 1
 
2 3 
 1 1

The space of 𝑟 × 𝑘 matrices 𝑀𝑟𝑘 is a vector space with addition and scalar
multiplication defined as follows

𝑀 + 𝑁 = (𝑚𝑖𝑗 ) + (𝑟𝑖𝑗 ) = (𝑚𝑖𝑗 + 𝑟𝑖𝑗 )

𝑟𝑀 = 𝑟(𝑚𝑖𝑗 ) = (𝑟𝑚𝑖𝑗 ).

Example

We have seen how black-and-white images can be represented by a single matrix

with entries given by 8-bit unsigned integers. (In other words, each entry is a value
between 0 and 255, inclusive. The value 0 represents black and the value 255
represents white. The values in between represent various shades of grey.)

Let A be an m×n matrix representing a black-and-white image. We will now look at

how we can perform some basic image manipulations using matrix algebra on A
Page 78 of 83
 Linear Algebra

Flipping an image upside down

To flip an image upside down, all we need to do is to reorder the rows of A

so that the first row becomes the last, the second row becomes the second last, and
so on. We can do this using a sequence of row swapping. Consider the m×m matrix
0 0 ⋯ 0 1
0 0 ⋯ 1 0
𝑀= ⋮ ⋰ ⋮
0 1 ⋯ 0 0
[1 0 ⋯ 0 0]
Note that M is a 0-1 matrix with ones on the diagonal from the top right to the
bottom left and it can obtained from the m×m identity matrix by reordering the
rows so that the first row becomes the last, the second row becomes the second
last, and so on. Then the image flipped upside down is represented by the matrix
MA

Rotating to the left by 90 degrees

Let us look at a small example to see what happens to the matrix when the image is
rotated to the left by 90 degrees.
1 2 3 4
Suppose that 0 1
5 6 7 8
4 8
3 7
. What we want to end up with is the matrix 𝐵 = [ ]
2 6
1 5
1 5
2 6
Note that 𝐴𝑇 = [ ]
3 7
4 8
To obtain B, we can simply flip the matrix AT upside down.

Colour inversion

Page 79 of 83
 Linear Algebra

Colour inversion involves flipping the colour of each pixel. In terms of the entries
of the matrix, we want each value v to become 255−v. In particular, a 0 will
become a 255 and a 255 will become a 0. This can be accomplished by

255𝐽𝑚×𝑛 − 𝐴

where J is the m×n matrix of all 1's. (Using the letter J to denote a matrix of all
ones is a common convention.)

Vertical blurring

If we take a copy of an image and slide it up one pixel, and another copy and slide
it one pixel down, the average of the superimposition of the slided copies and the
original image will create a vertically blurred version of the image.

At the pixel level, we can replace a pixel by the average of the pixel itself and its
immediate neigbours above and below. For example, if the matrix entry for a pixel,
call it p, has value 100 and the pixel immediately above has value 80 and the one
immediately below right has value 201, then the new value of p is
80 + 100 + 201
= 127.
3
We want to do this for all pixels, noting that if a pixel is on the top or bottom edge,
then it has only one neighbour. Can the operation described be accomplished using
matrix operations?

We look at a small example to illustrate the ideas. Suppose that

1 3
3 5
𝐵=[ ]
5 7
7 9
Suppose that we want to replace the first row with the average of the first two
rows. What we can do is to add the second row to the first row and then multiply
the first row by 12. These are both elementary row operations. We can therefore
form the product MA where
Page 80 of 83
 Linear Algebra

1 1
0 0
2 2
𝑀= 0 0 0 0
0 0 0 0
[0 0 0 0]
The result is
2 4
3 5
[ ]
5 7
7 9
To replace the third row with the average of rows 2, 3, and 4, we can form the
product NA with
1 0 0 0
0 1 0 0
𝑁= 1 1 1
0
3 3 3
[0 0 0 1]

We can accomplish the blurring in one stroke by forming the product BA where
1 1
0 0
2 2
1 1 1
0
𝐵= 3 3 3
1 1 1
0
3 3 3
1 1
[0 0
2 2]

In general, to blur an image vertically represented by an m×n matrix A, we form

the product BA where B is the following m×m matrix:

Page 81 of 83
 Linear Algebra

1 1
0 0 ⋯ 0 0 0 0
2 2
1 1 1
0 ⋱ 0 0 0 0
3 3 3
1 1 1
0 0 ⋱ 0 0 0
3 3 3
⋮ ⋱ ⋱ ⋱ ⋱ ⋱ ⋮ ⋮ ⋮
⋮ ⋮ ⋱ ⋱ ⋱ ⋱ ⋱ ⋮ ⋮
⋮ ⋮ ⋮ ⋱ ⋱ ⋱ ⋱ ⋱ ⋮
1 1 1
0 0 0 ⋱ ⋱ 0
3 3 3
1 1 1
0 0 0 0 ⋱ ⋱
3 3 3
1 1
[ 0 0 0 0 ⋯ 0 0
2 2]
Note that the blurring that results is hardly noticeable. To increase the amount of
blurring, simply form the product 𝐵𝑘 𝐴 for values of k larger than 1. With k=20, the
blurring becomes quite noticeable. The following table shows the results for
various values of k

Interestingly, B is invertible. In other words, vertical blurring as described above

can be undone! However, we must be careful not to jump to such a conclusion too
quickly. The reason is that when we do the averaging, we might end up with
numbers that are not integers. The numbers will have to be rounded before they can
represent pixels as 8-bit integers. Once rounding is done, deblurring will not
necessarily give us back the original image.

To reduce the impact of rounding, we can first convert the 8-bit integers to 24-bit
or 32-bit floating point values before we perform any processing. Once all the
desired processing has been done, the floating point value are converted back to 8-
bit integers. Such a conversion process is adopted by many image editing software
packages.

Exercises

Page 82 of 83
 Linear Algebra

How would you flip an image left to right using only the operations discussed
above?

How would you blur an image horizontally?

Page 83 of 83