Linear Algebra

Applied Sciences and Humanities

Unit 1 Matrices

Study Guide

Dr. Monika Naik || Ms. Monali Darji

ASH, PIET
Parul University

1
 Linear Algebra

INDEX

Sr.
Topics Subtopics Page No.
No.
1.0 Matrices Definition 4
1.1 Basis of matrices and its Trace, Symmetric, Skew-symmetric, 4-6
types singular and non-singular matrix,
Orthogonal matrix
1.2 System of Linear Definition 7
Equations
1.2.1 Linear Equations Definition 7

1.2.2 System of Linear Types of system of Linear equations, 7-9

Equations Geometrical Interpretation of system of
Linear Equations
1.2.3 Augmented Matrix Definition 7

1.2.4 Condition of Consistency Types of Solutions 7

for non-homogeneous
system
1.3 Row Echelon Form and Definition 7
Reduced Row Echelon
Form of a matrix
1.4 Methods of Solving - 7
system of Linear
Equations
1.4.1 Gauss Elimination Method Method Explanation and Examples 10

1.4.2 LU Decomposition Method Explanation and Examples 22

Method
1.5 Homogeneous Equations Method Explanation and Examples 17

1.6 Rank of a Matrix Definition 22

1.6.1 Methods for finding Rank By Determinant Method 22-29

of a matrix By Row echelon Form
1.7 Eigenvalues and Eigen Definition and Results 29-32
vectors

2
 Linear Algebra

1.8 Algebraic Multiplicity Definition and Examples 33-36

and Geometric
Multiplicity
1.9 Cayley Hamiltonian Statement and Examples 37-39
Theorem
1.10 Diagonalization Definition and Examples 39-41
1.11 Application of Matrices - 41
1.11.1 Coding and decoding Examples 41-44

1.11.2 Principal Component Definition and Examples 44-47

Analysis
— References Standard textbooks and datasheets cited 48

3
 Linear Algebra

Road map of matrices

1.0 Matrix:
A Matrix is a rectangular array of numbers (or functions) enclosed in brackets. These number or functions
are called entries or elements of the matrix.
e.g.
4 −2 1 sin 𝑥 cos 𝑥
[ ],[ ]
0 3 5 − cos 𝑥 sin 𝑥

Operations on Matrices:

Addition of two matrices:

One of the basic operations that can be performed on matrices is the addition operation. Just as we add two
or more integers, two or more matrices can also be added in a similar way.

Multiplication by a scalar

Multiplication of two matrices:

4
 Linear Algebra

1.1 Basic of Matrix and its types

Row Matrix:
A matrix having only one row and any number of columns, is called a row matrix or row vector.
e.g.: A =  2 5 − 3 4

Column Matrix:
A matrix, having only one column and any number of rows, is called a column matrix or column vector.

1
e.g.: A =  −3
 
 4 

Zero or Null Matrix:

A matrix, whose all the elements are zero, is called a zero or null matrix and is denoted by 0.

0 0 0 
0 0 
e.g. [ ], 0 0 0 
0 0  
0 0 0 

Square Matrix:
A matrix, in which the number of rows is equal to the number of columns, is called a square matrix,

1 3 2
e.g.  −1 4 −5
 
 2 6 8 

Diagonal Matix:
A square matrix, all of whose non-diagonal elements are zero and at least one diagonal element is non-
zero, is called a diagonal matrix.

1 0 0 
e.g. 0 4 0 
 
0 0 8 

Unit or Identity Matix:

5
 Linear Algebra

A diagonal matrix, all whose diagonal elements are unity, is called a unit or identity matrix and is denoted
by I,
1 0 0 
1 0  
e.g.   , 0 1 0
0 1  0 0 1 
 
Scalar Matrix:
A diagonal matrix, all of whose elements are equal, is called a scalar matrix,

2 0 0
4 0 
e.g.   , 0 2 0 
0 4 0 0 2
 
Upper Triangular Matrix:
A square matrix, in which all the elements below the diagonal are zero, is called an upper triangular matrix.

1 3 2 
e.g. 0 4 −5
 
0 0 8 

Lower Triangular Matrix:

A square matrix, in which all the elements above the
diagonal are zero, is called a lower triangular matrix.

 1 0 0
e.g.  −1 4 0 
 
 4 6 8 

Trace of a matrix:
If 𝐴 is a square matrix, the trace of 𝐴, denoted by
𝑡𝑟(𝐴)and is defined to be the sum of entries on the main
diagonal of 𝐴. The trace of 𝐴 is undefined if𝐴 is not a
square matrix.
4 5
e.g. If 𝐴 = [ ], then 𝑡𝑟(𝐴) = 4 + 6 = 10.
10 6
Transpose of a Matrix:
A matrix obtained by interchanging rows and columns of a matrix is called transpose of a matrix and is
denoted by AT or A' .
 1 −1
1 0 −4
If A = aij  , then A = a ji 
T
e.g. A =  0 2  then, AT =  
 
mXn nX m
 −4 1   −1 2 1

6
 Linear Algebra

Determinant of a matrix
If A is a square matrix, then determinant of A is
represented as A or det(A).
1 3 2
A = −1 4 −5
2 6 8
Symmetric matrix: - For any square matrix A,
if 𝐴 = 𝐴𝑇 then, it is known as symmetric matrix.
Skew-symmetric matrix: - For any square
matrix A, if 𝐴 = −𝐴𝑇 then it is known as Skew
symmetric matrix.
Singular and non-singular matrix: -
For any square matrix A, if|𝐴| ≠ 0, then it is
known as non-singular matrix and if |𝐴| =
0 then it is known as singular matrix.
e.g.
2 1
𝐴=[ ] ⇒ |𝐴| = 8 − 8 = 0 ⇒
8 4
Singular Matrix
1 2
𝐴=[ ] ⇒ |𝐴| = 4 − 6 = −2 ≠ 0
3 4
⇒ Non − Singular Matrix
Orthogonal Matrix:- The matrix is said to be an
orthogonal matrix if the product of a matrix and
its transpose give an identity value. i.e. 𝐴𝐴𝑇 = 𝐼
e.g.
Given A is an orthogonal matrix because
−1 0
𝐴=[ ]
0 1
−1 0
⸫ 𝐴𝑇 = [ ]
0 1
−1 0 −1 0 1 0
𝐴𝐴𝑇 = [ ][ ]=[ ]=𝐼
0 1 0 1 0 1
Exercise:

1. Identify which of the following matrices are square, diagonal, upper or lower triangular,
symmetric, or skew-symmetric. Find the trace of the matrices, if possible:
 2 −3 4  1 3 6  1 0 0 
(i) A =  −4 3 −2 (ii) B =  −1 2 2  (iii) C = 0 1 0 
   
 3 2 4   0 1 1  0 0 1 

7
 Linear Algebra

2. Are the given matrices orthogonal or not?

 2 2 1  Cos  0 Sin   1 2 2 1 3 1 
    1 −2 1
A =  −2 1 2 , B =  0 1 0 , C =   , D =  2 1 −2 , E = 3 1 2

 1 −2 2  − Sin  0 Cos   2 1  3
 −2 2 1  1 2 2
1 2 
 3 3 a
 
 2 1
3. Find a, b, c if A = b  is orthogonal.
3 3 
 
2 − 2 c
 3 3 

Elementary Transformations:
Elementary transformation is any one of the following operations on a matrix.
(i) The interchange of any two rows (or columns)
Ri j : Interchange of ith and jth row.
(ii) The multiplication of the elements of any row (or column) by any non-zero number.
kRi : Multiplication of ith row by a non-zero number k.
(iii) Addition of k times the jth row to the ith row
Ri + kR j : Addition of k times the jth row to the ith row.
1.2 System of Linear Equations:

1.2.1 Linear Equations: Any straight line in the 𝑥𝑦-plane can be represented algebraically by equation of
the form 𝑎𝑥 + 𝑏𝑦 = 𝑐, where 𝑎&𝑏 are real numbers.

1.2.2 System of Linear Equations:

A system of linear equation is a collection of one or more linear equations involving the same variables.
A linear system of m linear equations in n variables: An arbitrary system of m linear equations in n variables
𝑥1 , 𝑥2 , 𝑥3 , … . . 𝑥𝑛 is a set of equations of the form
𝑛

∑ 𝑎𝑖𝑗 𝑥𝑗 = 𝑏𝑖 (𝑖 = 1,2,3, … , 𝑚, 𝑗 = 1,2,3, … , 𝑛)

𝑗=1
A system of linear equations has either
1. No solutions, or
2. Exactly one solution, or
3. Infinitely many solutions

7
 Linear Algebra

Geometrical interpretation of system of Linear equations

Remarks:
(i) The system is said to be consistent if we get infinitely many solutions or unique solution.
(ii) The system is said to be inconsistent if we get No solution.

Schematic diagram for system of Linear equations

1.2.3 Augmented matrix:

A system of 𝑚 equations in 𝑛 unknowns can be abbreviated by writing only the rectangular array of
numbers.

8
 Linear Algebra

𝑎11 𝑎12 …𝑎1𝑛 |𝑏1

𝑎21 𝑎22…𝑎2𝑛 |𝑏2
[ ]
⋮ ⋮ ⋮ ⋮ ⋮
𝑎𝑚1 𝑎𝑚2 …𝑎𝑚𝑛 |𝑏𝑚
This is known as augmented matrix.
e.g.
Find the augmented matrix for each of the following system of linear equations:
2𝑥1 + +2𝑥3 = 1
3𝑥1 − 𝑥2 + 4𝑥3 = 7
6𝑥1 + 𝑥2 − 𝑥3 = 0
2 0 2 |1
Then, augmented matrix is given by[3 −1 4 |7].
6 1 −1|0
1.2.4 Condition of Consistency for non-homogeneous system:

(1) If there is a zero row to left of the augmentation bar but the last entry of this row is non-zero then the
system has no solution.
1 0 2 |1
e.g.: [0 1 4|7]
𝟎 𝟎 𝟎|𝟒
(2) If at least one of the columns on the left of the augmentation bar has zero element pivot entry, then the
system has infinitely many solutions.
1 0 2 |1
e.g.: [0 1 4|7]
𝟎 𝟎 𝟎|𝟎
(3) If all the rows having the leading entry 1 then the system has unique solution.
1 0 2 |1
e.g.: [0 1 4|7]
𝟎 𝟎 𝟏|𝟒

1.3 Row-Echelon (RE)form and Row-Reduced Echelon (RRE) form of a matrix

Definition: A rectangular matrix is in row-echelon form (or echelon form) if it has the following three
properties:

[Link] first element in each row must be non-zero and equals to 1, that is called leading entry 1.
[Link] the leading 1’s must be on the right-hand side of the matrix.
[Link] any zero row is available, then it must be below to the all-leading 1.

If the matrix satisfies the 4th property (i.e., In each column except leading 1 if all entries are zero) then
row-echelon form (RE form) becomes reduced row echelon form (RRE form).

9
 Linear Algebra

Illustrations of REF and RREF

Exercise
1. Which of the following matrices are in row-echelon or echelon form?
1 1 0 1 1  1 0 0 0
1 0 0 1 2 0 1 0 0 0 2 0 2 2  0 1 1 0 
   
(a) 0 1 0 (b) 0 1 0 (c) 0 1 0 (d)     (e)  
      0 0 0 3 3  0 0 0 0 
0 0 1 0 0 0 0 2 0    
0 0 0 0 4  0 0 0 1 
2. Which of the following matrices are in reduced row-echelon or reduced echelon form?
1 0 0 1 0 0 0 1 0  1 0 0 1 0 0 0 1 0 
0 0 
(a) 0 1 0 (b) 0 1 0 (c) 0 0 1 (d) 0 0 1 (e) 0 0 0 (f) 1 0 0 (g) 
         

           
0 0 1 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 
1.4 Methods of solving system of linear equations:
1.4.1 Gauss – Elimination Method: (Using Row Echelon Form)
1.4.2 LU Decomposition Method

10
 Linear Algebra

Schematic diagram for Methods of Solution of system of Linear Equations

Case-1: Unique solution
Example: Solve the following system by gauss- Elimination method
𝑥+𝑦+𝑧 =6
𝑥 + 2𝑦 + 3𝑧 = 14
2𝑥 + 4𝑦 + 7𝑧 = 30
Solution:
The matrix form of the given system is
1 1 1 𝑥 6
𝑦
[1 2 3] [ ] = [14]
2 4 7 𝑧 30
The augmented matrix is
1 1 1| 6
[𝐴|𝐵] = [1 2 3|14]
2 4 7|30
Now, to convert the given augmented matrix in row-echelon form we apply elementary operations as
following.
𝑅2 → 𝑅2 − 𝑅1 , 𝑅3 → 𝑅3 − 2𝑅1
1 1 1| 6
[0 1 2 | 8 ]
0 2 5|18
𝑅3 → 𝑅3 − 2𝑅2
1 1 1|6
[0 1 2|8]
0 0 1|2
The corresponding system of equation is
𝑥+𝑦+𝑧 =6
𝑦 + 2𝑧 = 8
𝑧=2
By using back substitution of 𝑧 = 2 in 𝑦 + 2𝑧 = 8, we get 𝑦 = 4 and 𝑧 = 2 & 𝑦 = 4 in 𝑥 + 𝑦 + 𝑧 = 6 we
get 𝑥 = 0.
𝑥 = 0, 𝑦 = 4, 𝑧 = 2 is unique solution of given system.

11
 Linear Algebra

Case-2: No solution
Example: Solve the following system of equation by Gauss elimination.
−2𝑏 + 3𝑐 = 1
3𝑎 + 6𝑏 − 3𝑐 = −2

6𝑎 + 6𝑏 + 3𝑐 = 5
Solution:
The matrix form of the given system is
0 −2 3 𝑥 1
[3 6 −3] [𝑦] = [−2]
6 6 3 𝑧 5
The augmented matrix is
0 −2 3 | 1
[3 6 −3|−2]
6 6 3|5
𝑅1 ↔ 𝑅2
3 6 −3|−2
[0 −2 3 | 1 ]
6 6 3|5

𝑅1 → (1⁄3)𝑅1
1 2 −1|−2/3
[0 −2 3 | 1 ]
6 6 3| 5
𝑅3 → 𝑅3 − 6𝑅1
1 2 −1|−2/3
[0 −2 3 | 1 ]
0 −6 9 | 9

𝑅3 → 𝑅3 − 3𝑅2

1 2 −1|−2/3
[0 −2 3| 1 ]
0 0 0| 6

The system of linear equation is

𝑎 + 2𝑏 − 𝑐 = −2/3
−2𝑏 + 3𝑐 = 1
0𝑎 + 0𝑏 + 0𝑐 = 6 is not possible.
This shows that the system has no solution

Case-3: Infinitely many solutions

Example: Solve the following system by Gauss elimination method.
4𝑥 − 2𝑦 + 6𝑧 = 8
𝑥 + 𝑦 − 3𝑧 = −1
15𝑥 − 3𝑦 + 9𝑧 = 21
Solution:

12
 Linear Algebra

The matrix form of the given system is

4 −2 6 𝑥 8
[1 1 −3] [ 𝑦 ] = [ −1]
15 −3 9 𝑧 21
The augmented matrix is
4 −2 6 | 8
[𝐴|𝐵] = [ 1 1 −3|−1]
15 −3 9 | 21

𝑅1 ↔ 𝑅2
1 1 −3|−1
[ 4 −2 6 | 8 ]
15 −3 9 | 21

𝑅2 → 𝑅2 − 4𝑅1 , 𝑅3 → 𝑅3 − 15𝑅1
1 1 −3|−1
[0 −6 18 | 12 ]
0 −18 54 | 36

𝑅2 → (−1/6)𝑅1 , 𝑅3 → (−1/6)𝑅3
1 1 −3|−1
[0 1 −3|−2]
0 1 −3|−2
𝑅3 → 𝑅3 − 𝑅2
1 1 −3|−1
[0 1 −3|−2]
0 0 0|0

The corresponding system of equations is

𝑥 + 𝑦 − 3𝑧 = −1
𝑦 − 3𝑧 = −2
Assigning the free variable 𝑧 an arbitrary value 𝑡,
𝑦 = 3𝑡 − 2,
𝑥 = −1 − 3𝑡 + 2 + 3𝑡 = 1

Hence, 𝑥 = 1, 𝑦 = 3𝑡 − 2, 𝑧 = 𝑡 is solution of the given system of equations.

Since 𝑡 is arbitrary real number, The system has infinitely many solutions.
Example: Consider the following system
𝑥+𝑦+𝑧 =6
𝑥 + 2𝑦 + 3𝑧 = 10
𝑥 + 2𝑦 + 𝜆𝑧 = 𝜇
For what values of 𝜆 and𝜇 the system has (i) infinitely many solutions (ii) unique solution and (iii) no
solution.
Solution: The Augmented matrix is
1 1 1| 6
[1 2 3|10]
1 2 𝜆| 𝜇
𝑅2 → 𝑅2 − 𝑅1 , 𝑅3 → 𝑅3 − 𝑅1

13
 Linear Algebra

1 1 1 | 6
[0 1 2 | 4 ]
0 1 𝜆 − 1|𝜇 − 6

𝑅3 → 𝑅3 − 𝑅2

1 1 1 | 6
[0 1 2 | 4 ]
0 0 𝜆 − 3|𝜇 − 10
(i) If 𝜆 − 3 = 0 and 𝜇 − 10 = 0, that is if 𝜆 = 3 and 𝜇 = 10 then the system has infinitely many solutions.
(ii)If𝜆 − 3 = 0 then the system has a unique solution. That is 𝜆 ≠ 3 and 𝜇 can possess any real value.
(iii)If 𝜆 − 3 = 0 and𝜇 − 10 ≠ 0, that is if 𝜆 = 3 and 𝜇 ≠ 10 then the system does not have any solution.

Example:
Solve the following system by gauss elimination method.
−1 3 4
+ + = 30
𝑥 𝑦 𝑧
3 2 1
+ − =9
𝑥 𝑦 𝑧
2 1 2
− + = 10
𝑥 𝑦 𝑧
Solution:
1 1 1
Let 𝑢 = 𝑥 , 𝑣 = 𝑦 , 𝑤 = 𝑧
Then the system of equations
−𝑢 + 3𝑣 + 4𝑤 = 30
3𝑢 + 2𝑣 − 𝑤 = 9
2𝑢 − 𝑣 + 2𝑤 = 10
The matrix form of the system is
−1 3 4 𝑢 30
[3 𝑣
2 −1] [ ] = [ 9 ]
2 −1 2 𝑤 10
The augmented matrix is
−1 3 4 |30
[3 2 −1| 9 ]
2 −1 2 |10

𝑅1 → (−1)𝑅1
1 −3 −4|−30
[3 2 −1| 9 ]
2 −1 2 | 10

𝑅2 → 𝑅2 − 3𝑅1 , 𝑅3 → 𝑅3 − 2𝑅1
1 −3 −4|−30
[0 11 11 | 99 ]
0 5 10 | 70

1 1
𝑅2 → ( ) 𝑅2 , 𝑅3 → ( ) 𝑅3
11 5

14
 Linear Algebra

1 −3 −4|−30
[0 1 1| 9 ]
0 1 2 | 14

𝑅3 → 𝑅3 − 2𝑅2
1 −3 −4|−30
[0 1 1| 9 ]
0 0 1| 5
The corresponding system of equations is
𝑢 − 3𝑣 − 4𝑤 = −30
𝑣+𝑤 =9
𝑤=5
By doing back substitution we get
1
𝑣+5=9⇒𝑣 =4⇒𝑦 =
4
1
𝑢 − 12 − 20 = −30 ⇒ 𝑢 = 2 ⇒ 𝑥 =
2
1 1 1
Hence, 𝑥 = 2 , 𝑦 = 4 , 𝑧 = 5 is required unique solution of the system.
Example:
Solve the system of linear equations 𝑥 − 2𝑦 + 𝑧 = 1; −𝑥 + 𝑦 − 𝑧 = 0; 2𝑥 − 𝑦 + 𝑧 = −1 using Gauss
Elimination method.
Solution: The Matrix form of the system of equation
1 −2 1 𝑥 1
𝑦
[−1 1 −1] [ ] = [ 0 ]
2 −1 1 𝑧 −1
1 −2 1 𝑥 1
Let A= [−1 1 −1] , 𝑋 = [𝑦] 𝐵 = [ 0 ]
2 −1 1 𝑧 −1
1 −2 1 1
[𝐴|𝐵] = [−1 1 −1 0 ]
2 −1 1 −1
𝑅2 → 𝑅1 + 𝑅2 𝑅3 → 𝑅3 − 2𝑅1
1 −2 1 1
~ [0 −1 0 1 ]
0 3 −1−3
𝑅2 → −𝑅2
1 −2 1 1
~ [0 1 0 −1]
0 3 −1−3
𝑅3 → 𝑅3 − 3𝑅2
1 −2 1 1
~ [0 1 0 −1]
0 0 −1 0
−𝑧 = 0 => 𝑧 = 0
𝑦 = −1
𝑥 − 2𝑦 + 𝑧 = 1
𝑥 = −1
Hence, the required solution is 𝑥 = −1, 𝑦 = −1 𝑎𝑛𝑑 𝑧 = 0
Example: Solve the system of linear equation using Gauss Elimination method

15
 Linear Algebra

𝑥 + 𝑦 + 𝑧 = 6; 𝑥 + 2𝑦 + 3𝑧 = 14; 2𝑥 + 4𝑦 + 7𝑧 = 30
1 1 1| 6
Solution: [𝐴|𝐵] = [1 2 3 | 14]
2 4 7 | 30
𝑅2 → 𝑅2 − 𝑅1 𝑅3 → 𝑅3 − 2𝑅1
1 1 1| 6
~ [0 1 2 | 8 ]
0 2 51| 8
𝑅3 → 𝑅3 − 2𝑅2
1 1 1 | 6
~ [0 1 2 | 8 ]
0 0 1 | 2
𝑧=2
𝑦 + 2𝑧 = 8
∴𝑦=4
𝑥=0
Therefore, required solution is 𝑥 = 0; 𝑦 = 4; 𝑧 = 8
Example: Solve the following system by Gauss elimination Method
2x + y + z = 0
3 x + 2 y + 3z = 18
x + 4 y + 9 z = 16
Solution: The corresponding augmented matrix is
2 1 1 | 0 
 A | B  =  3 2 3 | 18 R1 →   R1
1
2
1 4 9 | 16 
 1 1 
1 2 2 | 0 
 
3 2 3 | 18  R2 → R2 − 3R1 , R3 → R3 − 3R1
1 4 9 | 16 
 
 

16
 Linear Algebra

 1 1 
1 2 2 | 0
 
0 1 3
| 18  R2 → 2 R2
 2 2 
 7 17
| 16 
0 
 2 2 
 1 1 
1 2 2 | 0
 
0 1 3 7
| 18  R3 → R3 − R2
 2 2  2
 7 17
| 16 
0 
 2 2 
 1 1 
1 2 2 | 0 
  1
0 1 3 | 36  R3 → − R3
0 2
0 −2 | −110 
 
 
 1 1 
1 2 2 | 0
 
0 1 3 | 36 
0 0 1 | 55 
 
 

The resulting system is an upper triangular, which yields,

1 1
x+ y+ z =0
2 2
y + 3z = 36
z = 55
Back substitution gives the solution:
z = 55, y = −129, x = 37
Example: Solve the system of Linear equations by Gauss elimination method:
x − 2y + w = 3
1
− x + 2 y + z − w = −7
2
4 x − 8 y + 6 z + 7 w = −3
Solution: The corresponding augmented matrix is

17
 Linear Algebra

 1 
1 −2 0 | 3
−1
 A | B  =  −1 2 1 | −7  R2 → R2 + R1 , R3 → R3 − 4 R1
 2 
4 −8 6 | −3 
 7 
 1 
1 −2 0 | 3 
0 1
0 1 | −4  R3 → R3 − 6 R2
 2 
0 0 6 | −15
 3 
 1 
1 −2 0 | 3
0 1
0 1 | −4 
 2 
0 0 0
0
| 9
 
The resultant system is,
x − 2y + w = 3
1
z + w = −4
2
0=9
The last equation, 0 = 9 is never satisfied. Thus, the system has no solution. Hence, the system is
inconsistent.
Example: Find the value of  so that the equations:
2x + y + 2z = 0
x + y + 3z = 0 have a nontrivial solution.
4x + 3y +  z = 0
Solution: The corresponding
2 1 2 | 0
1
 A | B  = 1 1 3 | 0  R1 → R1
2
 4 3  | 0 
 1 
1 2
1
| 0
 
1 1 3 | 0 R2 → R2 − R1 , R3 → R3 − 4 R1
4 3  | 0
 
 
 1 
1 2
1 
 | 0
0 1
2 | 0 R2 → 2 R2
 2 
0 1  −4
| 0
 
 

18
 Linear Algebra

 1 
1 2
1
| 0 
  1
0 1 4 | 0 R1 → R1 − R2 , R3 → R3 − R2
0 2
1  − 4 | 0
 
 
1 0 −1 | 0 
0 1 4 | 0 

0 0  − 8 | 0 

Thus, the system to have the nontrivial solution,  − 8 = 0,  = 8

Example: Find the equation of the circle in the form x 2 + y 2 + 2 gx + 2hy + c = 0 which passes through the
points ( 6,8) , (8, 4 ) and ( 3,9 ) .
Solution: Substituting the values of x and y, we obtain the following in terms of g, h and c:
12 g + 16h + c = −100
16 g + 8h + c = −80
6 g + 18h + c = −90
On solving this system by using Gauss elimination method. The corresponding augmented matrix is
12 16 1 | −100 
 A | B  = 16 8 1 | −80  R1 →   R1
1
 12 
 6 18 1 | −90 
 4 1 −25 
1 3 12 | 3 
 
16 8 1 | −80  R2 → R2 − 16 R1 , R3 → R3 − 6 R1
6 18 1 | −90 
 
 
 4 1 −25 
1 3 12 | 3 
 
0 −40 −1 160   3 
| R2 →  −  R2
 3 3 3   40 
 | 
0 1 −40 
10
 2 

19
 Linear Algebra

 4 1 
1 3 12 −25 
 | 3 
0 1 1 | −4  R3 → R3 − 10 R2
 40 
 | −40 
0 10 1 
 2 
 4 1 
1 3 −25 
12 |
 3 
0 1 1
| −4  R3 → 4 R3
 40 
 | 0 
0 0 1 
 4 
 4 1 
1 3 −25 
12 |
 3 
0 1 1
| −4 
 40 
 | 0 
0 0 1 
 

This gives the following system:

4 1 −25
g + h+ c =
3 12 3
1
h + c = −4
40
c=0
Solving, we get c = 0, h = −4, g = −3
The equation of a circle is x 2 + y 2 − 6 x − 8 y = 0

Example: Suppose an electrical manufacturing company produces three types of articles line conditioner,
servo stabilizer and UPS. The total number of productions is 50 units. The company produce UPS twice
the number of line conditioners. Again the total of servo stabilizers and line conditioners is two more than
the number of UPS. Find the actual number of each items being produced by the company.
Solution: Let x, y, z be the numbers of line conditioners, servo stabilizers, UPS respectively. From the
given data, we can construct the following system of equations:
x + y + z = 50
2x − y = 0
x+ y−z =2
The corresponding augmented matrix is

20
 Linear Algebra

1 1 1 | 50 
 A | B  = 2 0 −1 | 0  R2 → R2 − 2 R1 , R3 → R3 − R1
1 1 −1 | 2 
1 1 1 | 50 
0 1
 −2 −3 | −100  R2 → − R2
2
0 0 −2 | −48 
1 1 1 
 | 50 
0 1 3 | 50   1
R3 →  −  R3
 2   2
0 0 −2 | −48
 
1 1 1 
 | 50 
0 1 3 | 50 
 2 
0 0 1 | 24 
 

The system is
x + y + z = 50
3
y + z = 50
2
z = 24
Solution of this system is z = 24, y = 14, x = 12
Therefore, the number of line conditioners = 12, servo stabilizer = 14, UPS=24

Exercise:
Solve the following system of equations by using Gauss elimination method.
(1) 𝑥 + 𝑦 + 2𝑧 = 9 Ans:
2𝑥 + 4𝑦 − 3𝑧 = 1 𝑥 = 1, 𝑦 = 2, 𝑧 = 3
3𝑥 + 6𝑦 − 5𝑧 = 0

(2)3𝑥 + 𝑦 − 3𝑧 = 13 Ans: No solution as the augmented matrix in row-

2𝑥 − 3𝑦 + 7𝑧 = 5 echelon form is
2𝑥 + 19𝑦 − 47𝑧 = 32 1 1/3 −1 |13/3
[0 1 −27/11| 1 ]
0 0 0 | 5

(3) 2𝑥 + 2𝑦 + 2𝑧 = 0 Ans: Infinitely many solutions. The solution set is

−2𝑥 + 5𝑦 + 2𝑧 = 1 −3𝑘−1 1−4𝑘
{( 7 , 7 , 𝑘)/ k R}.
8𝑥 + 𝑦 + 4𝑧 = −1
(4) 𝑥 + 𝑦 + 2𝑧 = 9; Answer: 𝑥 = 1; 𝑦 = 2; 𝑧 = 3
2𝑥 + 4𝑦 − 3𝑧 = 1; 3𝑥 + 6𝑦 − 5𝑧 = 0
(5) −𝑥 + 3𝑦 + 4𝑧 = 30; 3𝑥 + 2𝑦 − 𝑧 Answer: 𝑥 = 2; 𝑦 = 4; 𝑧 = 5
= 9; 2𝑥 − 𝑦 + 2𝑧 = 10
(6) 𝑥 + 2𝑦 + 𝑧 = 5 ; 𝑥 = 2, 𝑦 = 1, 𝑧 = 1

21
 Linear Algebra

3𝑥 − 𝑦 + 𝑧 = 6 ; 𝑥 + 𝑦 + 4𝑧 = 7
(7) 𝑥 + 𝑦 + 𝑧 = 6; 𝑥 = 0, 𝑦 = 4, 𝑧 = 2
𝑥 + 2𝑦 + 3𝑧 = 14;
2𝑥 + 4𝑦 + 7𝑧 = 30.
(8) Find the values of  and  so that the equations:
2 x + 3 y + 5z = 9
7x + 3y − 2z = 8
2x + 3y +  z = 
has (i) unique solution (ii) no solution (iii) infinitely many solutions
(9) Find the quadratic equation y = ax 2 + bx + c that passes through the point (3,18), (2,9) and (-2,13)

LU Decomposition in Linear Algebra:

LU decomposition is a method of decomposing a square matrix A into the product of two matrices: A
lower triangular matrix L and an upper triangular matrix U.
This is particularly useful for solving linear systems of equations, computing determinants, and
performing matrix inversion efficiently.
If A is a matrix, the LU decomposition expresses it as:
Where:
(i)L is a lower triangular matrix (all elements above the diagonal are zero).
(ii) U is an upper triangular matrix (all elements below the diagonal are zero).

In some cases, a permutation matrix P may also be needed to reorder the rows of A to ensure stability,
giving:

Procedure for LU Decomposition:

22
 Linear Algebra

Consider the system of linear equations in three variables:

𝑎11𝑥 + 𝑎12 𝑦 + 𝑎13 𝑧 = 𝑏1 ; 𝑎21𝑥 + 𝑎22 𝑦 + 𝑎23 𝑧 = 𝑏2 ; 𝑎31𝑥 + 𝑎32 𝑦 + 𝑎33 𝑧 = 𝑏3

These can be written in the form of AX =B as:

𝑎11 𝑎12 𝑎13 𝑥 𝑏1

𝑎
[ 21 𝑎22 𝑎23 ] [𝑦]=[𝑏2 ]
𝑎31 𝑎32 𝑎33 𝑧 𝑏3

𝑎11 𝑎12 𝑎13 𝑥 𝑏1

Here A= [𝑎21 𝑎22 𝑎23 ] , X= [𝑦] , B = [𝑏2 ]
𝑎31 𝑎32 𝑎33 𝑧 𝑏3

Now follow the steps given below to solve the above system of linear equations by LU Decomposition
method.

Step-1: Generate a matrix A = LU such that L is the lower triangular matrix with principal diagonal
elements being equal to 1 and U is the upper triangular matrix.

1 0 0 𝑢11 𝑢12 𝑢13

That means, L = [𝑙21 1 0] and U = [ 0 𝑢22 𝑢23 ]
𝑙31 𝑙32 1 0 0 𝑢33

Step 2: Now, we can write AX = B as: LUX = B….(1)

Step 3: Let us assume UX = Y….(2)

𝑦1
Where Y = [𝑦2 ]
𝑦3

Step 4: From equations (1) and (2), we have; LY = B

On solving this equation, we get y1, y2, y3.

Step 5: Substituting Y in equation (2), we get UX = Y

By solving equation, we get X= (x , y, z)

The above process is also called the Method of Triangularisation.

Conditions for LU Decomposition

- Let A must be a square matrix.

23
 Linear Algebra

- LU decomposition may fail if A is singular or if pivot elements are zero.

Example: Solve the following system using LU Decomposition Method:

𝑥 + 5𝑦 + 𝑧 = 14,2𝑥 + 𝑦 + 3𝑧 = 13,3𝑥 + 𝑦 + 4𝑧 = 17
Solution: We can solve the system using LU Decomposition
Let A = LU and Substitute into AX=B
So LUX =B for X to solve the system…..(1)
Let UX=Y ∴ LY=B and UX=Y
First we solve LY = B and then solve UX=Y for X
We need to find L and U such that A =LU.

1 5 1 1 0 0 𝑢11 𝑢12 𝑢13

[2 1 3] =[ 𝑙 21 1 0] [ 0 𝑢22 𝑢23 ]
3 1 4 𝑙31 𝑙32 1 0 0 𝑢33

𝑢11 𝑢12 𝑢13

= [𝑙21 𝑢11 𝑙21 𝑢12 + 𝑢22 𝑙21 𝑢13 + 𝑢23 ]
𝑙31 𝑢11 𝑙31 𝑢12 + 𝑙32 𝑢22 𝑙31 𝑢13 + 𝑙32 𝑢23 + 𝑢33

𝑢11 =1 , 𝑢12 =5, 𝑢13 = 1

𝑙21 𝑢11 =2 ⇒ 𝑙21 = 2

And 𝑙31 𝑢11 = 3 ⇒ 𝑙31 = 3

𝑙21 𝑢12 + 𝑢22 =1 ⇒ 𝑢22 = −9 and

𝑙21 𝑢13 + 𝑢23 = 3 ⇒ 𝑢23 = 1

𝑙31 𝑢12 + 𝑙32 𝑢22 =1 ⇒ 𝑙32 = 14/9

𝑙31 𝑢13 + 𝑙32 𝑢23 + 𝑢33 =4 ⇒ 𝑢33 = −5/9

1 0 0 1 5 1
𝐴 = 𝐿𝑈 = [2 1 0] [0 −9 1 ]
3 14/9 1 0 0 −5/9

From (i) LUX=B ⇒ LY=B

1 0 0 𝑦1 14
[2 1 0] [𝑦2 ] =[13]
3 14/9 1 𝑦3 17
14
𝑦1 = 14 , 2𝑦2 + 𝑦3 =13 , 3𝑦1 + 𝑦2 + 𝑦3 = 17
9

⇒ 𝑦1 = 14 , 𝑦2 = −15 , 𝑦3 = −5/3

24
 Linear Algebra

Now UX=Y

1 5 1 𝑥 14
[0 −9 1 ] [𝑦 ] [ −15 ]
0 0 −5/9 𝑧 −5/3

∴ 𝑥 + 5𝑦 + 𝑧 = 14 ,−9𝑦 + 𝑧 = −15, z = 3

∴ 𝑥 = 1 ,𝑦 = 2, z = 3

Example-17: Solve the following system using LU Decomposition Method:

𝑥 + 𝑦 − 𝑧 = 4, 𝑥 − 2𝑦 + 3𝑧 = −6,2𝑥 + 3𝑦 + 𝑧 = 7
1 1 −1 𝑥 4
Solution: [1 −2 3 ] [𝑦] = [−6]
2 3 1 𝑧 7
AX=B
We can solve the system using LU Decomposition
Let A = LU and Substitute into AX=B
So LUX =B for X to solve the system…..(1)
Let UX=Y ∴ LY=B and UX=Y
First we solve LY = B and then solve UX=Y for X
We need to find L and U such that A =LU.
1 1 −1 1 0 0 𝑢11 𝑢12 𝑢13
[1 −2 3 ] [𝑙21 1 0] [ 0 𝑢22 𝑢23 ]
2 3 1 𝑙31 𝑙32 1 0 0 𝑢33
𝑢11 𝑢12 𝑢13
𝑙 𝑢
[ 21 11 𝑙 𝑢
21 12 + 𝑢22 𝑙 21 13 + 𝑢23
𝑢 ]
𝑙31 𝑢11 𝑙31 𝑢12 + 𝑙32 𝑢22 𝑙31 𝑢13 + 𝑙32 𝑢23 + 𝑢33

𝑢11 =1 , 𝑢12 =1, 𝑢13 = -1

𝑙21 𝑢11 =1 ⇒ 𝑙21 = 1 and 𝑙31 𝑢11 = 2 ⇒ 𝑙31 = 2

𝑙21 𝑢12 + 𝑢22 =-2 ⇒ 𝑢22 = −1 and 𝑙21 𝑢13 + 𝑢23 = 3 ⇒ 𝑢23 = 2

𝑙31 𝑢12 + 𝑙32 𝑢22 = 3 ⇒ 𝑙32 = −1/3

𝑙31 𝑢13 + 𝑙32 𝑢23 + 𝑢33 =1 ⇒ 𝑢33 = 11/3

1 0 0 1 1 −1
A =LU = [1 1 0] [0 −1 2 ]
2 −1/3 1 0 0 11/3

Now we know that AX=B

For finding solution of equation, we are using

25
 Linear Algebra

LUX=B ⇒ LY=B

1 0 0 𝑦1 14
[1 1 0] [𝑦2 ] [13]
2 −1/3 1 𝑦3 17

⇒ 𝑦1 = 4 , 𝑦2 = −10 , 𝑦3 = −13/3

1 5 1 𝑥 4
Now UX=Y implies [0 −9 1 ] [𝑦 ] [ −10 ]
0 0 −5/9 𝑧 −13/3

∴ 𝑥 = 1 ,𝑦 = 2, z = -1

Exercise:

1. Solve the system of equations x + y + z = 1, 3x + y – 3z = 5 and x – 2y – 5z = 10 by LU

decomposition method. (Ans: (6, -7,2))
2. Solve the below given system of equations by LU decomposition.
x + y + z = 1, 4x + 3y – z = 6, 3x + 5y + 3z = 4
3. Find the solution of the system of equations by LU decomposition.
x + 2y + 3z = 9, 4x + 5y + 6z = 24, 3x + y – 2z = 4

Classification of types of Solution of Systems of Linear equations

1.5 HOMOGENEOUS EQUATIONS

A system of linear equations in terms of 𝑥1, 𝑥2, 𝑥3, … 𝑥𝑛 having the matrix form AX= O, where A is 𝑚 × 𝑛
coefficient matrix, X is 𝑛 × 1 column matrix, O is a 𝑚 × 1 zero column matrix is called a system of
homogeneous equations
e.g.: 𝑥 + 𝑦 + 𝑧 = 0 ; 𝑥 + 2𝑦 − 𝑧 = 0; 𝑥 + 3𝑦 + 2𝑧 = 0

26
 Linear Algebra

e.g.: 𝑥 + 𝑦 = 0 ; 𝑥 + 2𝑦 = 0

Homogeneous equations are never inconsistent. They always have the solution “all variables = 0”. The
solution (0, 0, …, 0) is often called the trivial solution. Any other solution is called nontrivial solution.

Example: Solve the following system: Example: Solve the following system
4𝑥 + 3𝑦 − 𝑧 = 0
3𝑥 + 4𝑦 + 𝑧 = 0
5𝑥 + 𝑦 − 4𝑧 = 0
Solution: The matrix form of the system is
4 3 −1 𝑥 0
[3 4 1 ] [𝑦] = [0] Solution:
5 1 −4 𝑧 0
The augmented matrix is
[𝐴|𝐵]
4 3 −1|0
[3 4 1 |0]
5 1 −4|0 =

=
=

=
The last equation does not give any information
about the equations.
Let.

___________________________
=
Exercise: Solve the following system of
equations.
(1) Ans: Infinitely many
𝑥+𝑦−𝑧+𝑤 =0 solutions.
𝑥 − 𝑦 + 2𝑧 − 𝑤 = 0 The solution set is

27
 Linear Algebra

3𝑥 + 𝑦 +𝑤 =0 {(t/4 , −7𝑡/4, 𝑡 )/ t =
R}.
(2) 2𝑥 + 𝑦 + 3𝑧 = 0 Ans: Trivial solution
𝑥 + 2𝑦 =0 𝑥 = 0, 𝑦 = 0, 𝑧 = 0
𝑦+𝑧 =0

The required solution is𝑥 = 0, 𝑦 = 0, 𝑧 = 0 which is

trivial solution.
1.6 Rank of a Matrix
The positive integer 𝑟 is said to be a rank of a matrix 𝐴 if it possesses the following properties:
(i)There is at least one minor of order 𝑟 which is non-zero.
(ii) Every minor of order greater than 𝑟 is zero.
Remarks:
1. Rank of matrix 𝐴 is denoted by 𝜌(𝐴)
2. The rank of matrix remains unchanged by elementary transformation
3. 𝜌(𝐴𝑇 ) = 𝜌(𝐴)
4. The rank of the product of two matrices always less than or equal to the rank of either matrix (i.e.,
𝜌(𝐴𝐵) ≤ 𝜌(𝐴) or 𝜌(𝐴𝐵) ≤ 𝜌(𝐵)).

28
 Linear Algebra

Flowcharts to find rank for 2 X2 and 3 X3 matrices

1.6.1 Methods for finding Rank of a Matrix:
❖ Method-1: Rank of a Matrix by Determinant Matrix
Consider a square matrix 𝐴 of order 𝑟.
• Step-1: Find the determinant of 𝐴. If 𝑑𝑒𝑡(𝐴) ≠ 0 then 𝜌(𝐴) = 𝑟. Otherwise 𝜌(𝐴) < 𝑟.
• Step-2: Find the all-possible minors of order 𝑟 − 1. If any one of them is non-zero then order is
𝑟 − 1, otherwise 𝜌(𝐴) < 𝑟 − 1.
• Step-3: By continuing this process upto the non-zero determinant.
Example: Find the rank the following matrices by determinant method:
𝟐 𝟑 𝟒
a. 𝑨 = [𝟒 𝟑 𝟏]
𝟏 𝟐 𝟒

29
 Linear Algebra

2 3 4
Solution: Given, 𝐴 = [4 3 1] then 𝑑𝑒𝑡(𝐴) ≠ 0. Hence, the 𝜌(𝐴) = 3
1 2 4
𝟏 𝟐 𝟑
b. 𝑨 = [𝟐 𝟑 𝟒]
𝟑 𝟓 𝟕
1 2 3
Solution: Given, 𝐴 = [2 3 4] then 𝑑𝑒𝑡(𝐴) = 0. Hence, the rank of 𝐴 is less than 3.
3 5 7
3 4
Now, minor of 1 = | | = 21 − 20 = 1 ≠ 0. Hence, 𝜌(𝐴) = 2.
5 7
𝟒 𝟐 𝟑
c. 𝑨 = [ 𝟖 𝟒 𝟔]
𝟑
−𝟐 −𝟏 − 𝟐
4 2 3
Solution: Given, 𝐴 = [ 8 4 6 ] then 𝑑𝑒𝑡(𝐴) = 0. Hence 𝜌(𝐴) < 3.
3
−2 −1 − 2

Consider all the minors of order 2, i.e.,

2 3 4 3
4 2 2 3 4 3 4 2 3 3| = 0
| | = 0, | | = 0, | | = 0, | | = 0, | | = 0, |
8 4 4 6 8 6 −2 −1 −1 − −2 −
2 2
Here, all the minors of order 2 are zero. There rank is less than 2. Hence, 𝜌(𝐴) = 1.
 1 2 −1 −4 
d. A =  2 4 3 5 
 −1 −2 6 −7 

Solution: Here, the order of matrix 𝐴 is 3 × 4. Hence the rank of 𝐴 is maximum 3 as we can find the
square matrix of order 3. Therefore, consider all the minors of order 3, i.e.,
1 2 −1 2 −1 −4 1 2 −4 1 −1 −4
|2 4 3 | = 0, | 4 3 5 | = 0, | 2 4 5 | = 0, | 2 3 5 | = −120
−1 −2 6 −2 6 −7 −1 −2 −7 −1 6 −7
Here, one minor of rank 3 is not equal to zero. Hence, 𝜌(𝐴) = 3.
Example: Find the value of p for which the following matrix A will be of
(i) rank one (ii) rank two (iii) rank three
3 p p
A =  p 3 p 
 p p 3 
Solution:

30
 Linear Algebra

3 p p
det ( A ) = p 3 p
p p 3
= 3(9 − p 2 ) − p (3 p − p 2 ) + p ( p 2 − 3 p )
= (3 − p )(9 + 3 p − p 2 − p 2 )
= ( p − 3)(2 p 2 − 3 p − 9)
= ( p − 3)( p − 3)(2 p + 3)
= ( p − 3) 2 (2 p + 3)
(i ) If p = 3
 3 3 3
A = 3 3 3
3 3 3
det( A) = 0
And all the minors of order 2 are zero.
Hence,  ( A) = 1
Rank of A will be 1 if p = 3
(ii) Rank of A will be 2 if det(A)=0, but p  3
( p − 3) 2 (2 p + 3) = 0 but p  3
2p +3 = 0
−3
p=
2
(iii) Rank of A will be 3 if det( A)  0
( p − 3) 2 (2 p + 3)  0
−3
p  3, p 
2
3 − x 2 2 

Example: For what value of x, will the matrix A = 1 4− x 0  be of rank

 −2 −4 1 − x 
(i) Equal to 3 (ii) less than 3

31
 Linear Algebra

3− x 2 2
det( A) = 1 4− x 0
−2 −4 1 − x
= (3 − x)[(4 − x)(1 − x) − 0] − 2(1 − x) + 2(−4 + 8 − 2 x)
= (3 − x)(4 − x)(1 − x) + 2(3 − x)
= (3 − x)( x 2 − 5 x + 6)
= (3 − x)( x − 3)( x − 2)
= − ( x − 3) 2 ( x − 2)
Solution:
(i )  ( A) = 3 if det( A)  0
( x − 3) 2 ( x − 2)  0
x  2,3
(ii )  ( A)  3 if det( A) = 0
( x − 3) 2 ( x − 2) = 0
x = 2,3
Example: Find the possible values of p, for which the following matrix A will have
(i) Rank 1 (ii) rank 2 (iii) rank 3
p p 2
A =  2 p p 
 p 2 p 
Ans: (i) p=2; (ii) p=-2; (iii) p  −1, p  2
 3 1 2

Example: Find the value of  for which rank of the matrix A = −1 4 5

 
 7 2  
(i) Is less than 3 (ii) equal to 3

❖ Method-2: Rank of a Matrix by Row Echelon Form

The Rank of a Matrix in Row Echelon Form is equal to the number of non-zero rows of the matrix.
1 3 −1
e.g. A=|0 1 4 | ; the matrix 𝐴 is in Row Echelon form with two non-zero rows. Hence, rank of matrix
0 0 0
𝐴 is 2.
Example: Find the rank the following matrices by reducing to Row Echelon Form:

32
 Linear Algebra

5 3 14 4  1 2 3 −1
 −2 −1 −3 −1
(a) A = 0 1 2 1 
(b) A =  
1 −1 2 0 1 0 1 1
 
Solution: Given  0 1 1 −1
Solution: Given
5 3 14 4 
A = 0 1 2 1   1 2 3 −1
 −2 −1 −3 −1
1 −1 2 0  A= 
1 0 1 1
By applying row-operations  
 0 1 1 −1
1 −1 2 0
By applying row-operations
R13 0 1 2 1 
5 3 14 4  1 2 3 −1
0 3 3 −3
1 −1 2 0 R2 + 2 R1 , R3 − R1  
0 −2 −2 2 
R3 − 5 R1 0 1 2 1   
0 8 4 4  0 1 1 −1
1 2 3 −1
1 −1 2 0  0 1 1 −1
R3 − 8 R2 0 1 2 1  R24  
0 −2 −2 2 
0 0 −12 −4   
0 3 3 −3
 
1 −1 2 0  1 2 3 −1
 1   0 1 1 −1
 −  R3 0 1 2 1  R3 + 2 R2 , R4 − 3R2  
 12  0 0 1 1  0 0 0 0 
   
 3 0 0 0 0 
The equivalent matrix is in Row-Echelon The equivalent matrix is in Row-Echelon
Form. Form.
Number of non-zero rows = 3 .Hence, Number of non-zero rows = [Link], 𝜌(𝐴) =
𝜌(𝐴) = 3 2

Exercise:
2 4 1
(1) Find the ranks of 𝐴, 𝐵, 𝐴𝐵 and verify 𝜌(𝐴𝐵) ≤ 𝜌(𝐴) or 𝜌(𝐴𝐵) ≤ 𝜌(𝐵)) where 𝐴 = [3 6 2],
4 8 3
1 2 3
𝐵 = [3 1 2]
4 3 5

33
 Linear Algebra

(2) Find the rank the following matrices by reducing to Row Echelon Form:
 3 −2 0 −1 −7 
1 1 −1 1  0 2 2 1 −5  1 2 3 1 1 −1 1
(i) A = 1 −1 2 −1 , (ii) A =  (iii)[4 5 6] , (iv)[1 −1 2 −1]
1 −2 −3 −2 1 3 1
3 1 0 1    7 8 9 0 1
0 1 2 1 6 ,

1 2 3
(v) [2 3 4]
3 4 5
Remarks:
(1) If 𝜌(𝐴) ≠ 𝜌(𝐴|𝐵) then the system is inconsistent.
(2) If 𝜌(𝐴) = 𝜌(𝐴|𝐵) then the system is consistent.
(3) If 𝜌(𝐴) < 𝑛 then there are infinitely many solutions (𝑛 is the number of unknowns)
(4) If 𝜌(𝐴) = 𝑛 then there is a unique solution.

Schematic diagram for types of solutions

Example: What conditions must b1 , b2 , b3 satisfy for the system to be

x1 + 2 x2 + 3x3 = b1 , 2 x1 + 5 x2 + 3x3 = b2 , x1 + 8 x3 = b3 be consistent?
Solution:
1 2 3 | b1 
 A | B  =  2 5 3 | b2  R2 → R2 − 2 R1 , R3 → R3 − R1
1 0 8 | b3 
1 2 3 | b1 
0 1 −3 | b2 − 2b1  R3 → R3 + 2 R2

0 2 5 | b3 − b1 
1 2 3 | b1 
0 1 −3 | b2 − 2b1  R3 → (−1) R3

0 0 −1 | b3 + 2b2 − 5b1 
1 2 3 | b1 
0 1 −3 | b2 − 2b1 

0 0 1 | −b3 − 2b2 + 5b1 
Since, rank (A) = rank (A | B) = 3, the solution is unique for every choice of the values of b1 , b2 , b3 .

34
 Linear Algebra

Schematic diagram for Eigen values and Eigen vectors concepts

1.7 Eigen values and Eigen vectors:

Let 𝐴 be 𝑛 × 𝑛 matrix, then there exists a real number 𝜆 and a nonzero vector 𝑋 such that
𝐴𝑋 = 𝜆𝑋
then, 𝜆 is called as the eigen value or characteristic value or latent root or proper roots of the matrix𝐴, and
𝑋 is called as eigen vector or characteristic vector or real vector corresponding to eigen value 𝜆 of the
matrix 𝐴.
As, 𝐴𝑋 = 𝜆𝑋 ⇒ 𝐴𝑋 − 𝜆 𝐼𝑛 𝑋 = 0 ⇒ (𝐴 − 𝜆 𝐼𝑛 )𝑋 = 0
Which is a homogeneous system, and has a nontrivial solution if |𝐴 − 𝜆 𝐼𝑛 | = 0
Remarks:
1. An Eigen vector is never the zero vector.
2. The matrix [A –𝜆In] is known as the characteristic matrix of A.
3. The determinant of (A –𝜆In) after expansion gives the polynomial in 𝜆, it is known as the
characteristic polynomial of the matrix A of order 𝑛 × 𝑛 and is of degree n.
4. |𝐴 − 𝜆𝐼𝑛 |=0 is called the characteristic equation of matrix A.
5. The root of the characteristic equation is known as characteristic value or eigenvalues of the
matrix.
6. The set of all characteristic roots (Eigen values) of the matrix A is called the spectrum of A.
7. Let A be 𝑛 × 𝑛 matrix and 𝜆 be an eigen value for A. Then the set𝐸𝜆 = {𝑋/𝐴𝑋 = 𝜆𝑋} is called
the Eigen space of 𝜆.
Results:

35
 Linear Algebra

1. The Eigen values of a diagonal matrix are its diagonal elements.

2. The sum of Eigen values of an 𝑛 × 𝑛 matrix is its trace and their product is |𝐴|.
3. For the upper triangular (lower triangular) 𝑛 × 𝑛 matrix A, the Eigen values are its
diagonal elements.
Example:
1 4
If 𝐴 = | |, find the Eigen values for the given matrices:
3 2
(i) 𝐴, (ii) 𝐴𝑇 , (iii) 𝐴−1 , (iv) 4𝐴−1, (v) 𝐴2 , (vi) 𝐴2 − 2𝐴 + 𝐼,
(vii) 𝐴3 + 2𝐼

1 4
Solution: Given, 𝐴 = | |
3 2
The characteristic equation of matrix 𝐴 is
A −  I2 = 0
1−  4
 = 0  (1 −  )( 2 −  ) − 12 = 0   2 − 3 − 10 = 0  (  − 5 )(  + 2 ) = 0
3 2−
  = 5 or  = −2

Eigenvalues of 𝐴 = 𝜆 5, −2
Eigenvalues of 𝐴𝑇 = 𝜆𝑇 5, −2
Eigenvalues of 𝐴−1 = 𝜆−1 1 1
,−
5 2
Eigenvalues of 4𝐴−1 = 4𝜆−1 4
, −2
5
Eigenvalues of 𝐴2 = 𝜆2 25, 4
Eigenvalues of 𝐴2 − 2𝐴 + 𝐼 = 𝜆2 − 2𝜆 + 1 16, 9
Eigenvalues of 𝐴3 + 2𝐼 = 𝜆3 + 2 127, −6

3 2
Example: Find the Eigen values of 𝐴 = | |
3 8
3 2
Solution: Given 𝐴 = | |, then the characteristic equation of matrix 𝐴 is
3 8
A −  I2 = 0
3− 2
 = 0  ( 3 −  )(8 −  ) − 6 = 0   2 − 11 + 18 = 0  (  − 9 )(  − 2 ) = 0
3 8−
 1 = 9 or 2 = 2
1 0 
Example: If A =   then find the eigen values of A2 and A−1 .
0 2 
Solution: The eigen values are diagonal elements are 1,2 as A is a diagonal matrix.
Therefore, eigen values of A2 are (1) 2 , (2) 2 implies 1, 4
11
and eigen values of A−1 are , .
12

36
 Linear Algebra

1 2 −3
 
Example: If A = 0 2 3 then find the eigen values of AT and 5A .
 
0 0 2 
Solution: Since A is an upper triangular matrix, the eigen values of A are 1,2,2.
Therefore, the eigen values of AT are 1,2,2
The eigen values of 5A are 5,10,10

Types of Eigen Values

Eigenvalues are non- Eigenvalues are Eigenvalues are

repeated, whether repeated and the repeated and the
matrix is symmetric matrix is non- matrix is symmetric
or non-symmetric symmetric

−2 −8 −12
Example: Find the eigen values and eigen vector of the matrix A= [ 1 4 4 ]
0 0 1

Solution:
The characteristic equation is |𝐴 − 𝜆𝐼𝑛 |=0
−2 −  −8 −13
1 4− 4 =0
0 0 1− 
  3 − S1 2 + S2 − A = 0
S1 = tr ( A) = −2 + 4 + 1 = 3
S2 = Sum of minors of diagonal entries
4 4 −2 −12 1 4
= + + = 4−2+0 = 2
0 1 0 1 0 0
A = −2 ( 4 ) + 8 (1) − 12 ( 0 ) = −8 + 8 = 0
∴ characteristic equation is
 3 − 3 2 + 2 = 0
  (  2 − 3 + 2 ) = 0 We suppose z = k , y = 0, x + 4 z = 0
 z = k , y = 0, x = −4 z = −4k
  = 0 or  2 − 3 + 2 = 0
  = 0 or (  − 2 )(  − 1) = 0
  = 0 or  = 1or  = 2
Here, one can observe that all eigenvalues are non-
repeated and matrix is non-symmetric. Therefore, eigen vector space for 1 = 0
When 1 = 0

37
 Linear Algebra

When 3 = 2

Therefore, we suppose
x + 4 y + 6 z = 0, − 2 z = 0, y = k
 z = 0, y = k , x = −4k
Therefore, eigen vector space is

Therefore, eigen vector space for 1 = 0 is

When 2 = 1

We suppose z = 0, y = k , x + 2 y = 0
 z = 0, y = k , x = −2 z = −2k

( Continue on previous page …)

Therefore, Eigen vector space for 3 = 2

1.8 Algebraic multiplicity and Geometric multiplicity

38
 Linear Algebra

Let 𝐴 be 𝑛 × 𝑛 matrix and 𝜆 be an eigen value for 𝐴 . If 𝜆 occurs ( k  1) times then 𝑘 is called the
Algebraic multiplicity of 𝜆, and the number of basis vectors is called Geometric multiplicity.
−2 2 −3
Example: Find Eigen values and Eigen vectors of the matrix. 𝐴 = [ 2 1 −6]. Also determine
−1 −2 0
algebraic and geometric multiplicity.
Solution: The characteristic equation is |𝐴 − 𝜆𝐼𝑛 | = 0.

Algebraic Multiplicity of 𝜆 = −3 is 2 and of 𝜆 = 5 is 1.

We solve the following homogeneous system:

Case I: When 𝜆1 = 5 Case II: When 𝜆2 = −3, 𝜆3 = −3

= =

=
=

=
which is in Row-Echelon from.
We suppose

39
 Linear Algebra

x2 = k1 , x3 = k2 , x1 + 2 x2 − 3x3 = 0  x1 = −2k1 + 3k2

Therefore, Eigen space is for𝜆2 = −3, 𝜆3 = −3 is
k1 ( −2,1,0) + k2 (3,0,1) / k1, k2  R
= Hence, Geometric multiplicity of 𝜆2 = −3 is 2 and of𝜆 =
5 is 1.

which is in Row-Echelon from.

x = k , x2 + 2 x3 = 0  x2 = −2k ,
We suppose 3
x1 + x3 = 0  x1 = −k
Therefore, Eigen space is for 𝜆1 = 5 is
k ( −1, −2,1) / k  R
0 1 1
Example: Find Eigen values and Eigen vectors of the matrix. 𝐴 = [1 0 1]. Also determine algebraic
1 1 0
and geometric multiplicity.
Solution: The characteristic equation is|𝐴 − 𝜆𝐼𝑛 | = 0.

Algebraic Multiplicity of 𝜆 = −1 is 2 and of 𝜆 = 2 is 1.

Case-1:𝜆1 = 2 Case-2: 𝜆2 = −1, 𝜆3 = −1

40
 Linear Algebra

Let𝑥3 = 𝑘1, 𝑥2 = 𝑘2,

⇒ 𝑥1 + 𝑥2 + 𝑥3 = 0 ⇒ 𝑥1 = −𝑘1 −𝑘2, .
Therefore, eigen space is for𝜆2 = −1, 𝜆3 = −1
is k1 ( −1, 0,1) + k2 ( −1,1, 0 ) / k1 , k2  R
Hence, Geometric Multiplicity of𝜆2 = −1 is 2 and 𝜆1 =
2 of is 1.

Let

.Therefore, eigen space is for 𝜆1 = 2 is

k (1,1,1) / k  R
1 2 2
Example: Determine algebraic and geometric multiplicity of matrix 𝐴 = [ 0 2 1].
−1 2 2
Answer: 𝜆 = 1,2,2 therefore algebraic multiplicity of 𝜆 = 2 is 2 and geometric multiplicity is 1. For 𝜆 =
1, A.M. is 1 and G.M. is 1.
−2 −8 −12
Example: Find Eigen value and Eigen vector for following matrix[ 1 4 4 ]
0 0 1
Solution: |A-λI|=0. Also, find A.M.

−2 − 𝜆 −8 −12
| 1 4−𝜆 4 |=0
0 0 1−𝜆
∴ (−2 − 𝜆)((4 − 𝜆) × (1 − 𝜆) − 4 × 0) − (−8)(1 × (1 − 𝜆) − 4 × 0) + (−12)(1 × 0 − (4 − 𝜆) × 0)
=0

∴ (−2 − 𝜆)((4 − 5𝜆 + 𝜆2 ) − 0) + 8((1 − 𝜆) − 0) − 12(0 − 0) = 0

∴ (−2 − 𝜆)(4 − 5𝜆 + 𝜆2 ) + 8(1 − 𝜆) − 12(0) = 0

41
 Linear Algebra

∴ (−8 + 6𝜆 + 3𝜆2 − 𝜆3 ) + (8 − 8𝜆) − 0 = 0

∴ (−𝜆3 + 3𝜆2 − 2𝜆) = 0

∴ −𝜆(𝜆 − 1)(𝜆 − 2) = 0

∴ 𝜆 = 0𝑜𝑟(𝜆 − 1) = 0𝑜𝑟(𝜆 − 2) = 0

∴ 𝑇ℎ𝑒 𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝑡ℎ𝑒 𝑚𝑎𝑡𝑟𝑖𝑥 𝐴 𝑎𝑟𝑒 𝑔𝑖𝑣𝑒𝑛 𝑏𝑦 𝜆 = 0,1,2

1. Eigenvectors for λ=0 2. Eigenvectors for λ=1 3. Eigenvector for λ=2
−2
−4 −4 𝑣3 = [ 1 ]
𝑣1 = [ 1 ] 𝑣2 = [ 0 ] 0
0 1
A.M. for all the three eigen values are 1.
Exercise-7
Find the Eigen value and Eigenvector of the matrices
2 1 0 1 2 2
a. [−1 0 1] , b. [ 0 2 1]
0 0 1 −1 2 2

Schematic diagram of matrix related results

Theorem: Every square matrix can be decomposed as a sum of symmetric and skew-symmetric matrices.
Proof: Let 𝐴 be 𝑚 × 𝑛 matrix.
1 1
Let 𝐵 = 2 (𝐴 + 𝐴𝑇 ) 𝑎𝑛𝑑 𝐶 = = 2 (𝐴 − 𝐴𝑇 ) be two matrices.

Obviously, 𝐴 = 𝐵 + 𝐶
1 𝑇 1 1 1
Now,𝐵 𝑇 = [2 (𝐴 + 𝐴𝑇 )] = 2 [(𝐴 + 𝐴𝑇 )]𝑇 = 2 [𝐴𝑇 + (𝐴𝑇 )𝑇 ] = 2 (𝐴𝑇 + 𝐴) = 𝐵

As 𝐵 𝑇 = 𝐵 , 𝐵 is symmetric.

42
 Linear Algebra

𝑇
𝑇
1 𝑇 )]
1 1 1
𝐶 =[ (𝐴 −𝐴 = [(𝐴 − 𝐴𝑇 )]𝑇 = [𝐴𝑇 − (𝐴𝑇 )𝑇 ] = (𝐴𝑇 − 𝐴) = −𝐶
2 2 2 2
Therefore,𝐶 𝑇 = −𝐶 , 𝐶 is skew- symmetric.
Therefore, 𝐴 is a sum of symmetric and skew-symmetric matrices.
1.9 Cayley –Hamilton Theorem:
Every square matrix satisfies its own characteristic equation i.e. The theorem states that, for a square matrix
𝐴 of order 𝑛, if |𝐴 − 𝜆𝐼𝑛 | = 0.
1 4
Example: Verify Cayley-Hamilton theorem and hence find the inverse of 𝐴 = [ ] and 𝐴4 .
2 3
Solution: The characteristic equation for given matrix is
|𝐴 − 𝜆𝐼2 | = 0.

Now, by putting 𝜆 = 𝐴, we have

1 4 1 4 1 4 1 0 9 16  4 16 5 0 0 0
A2 − 4 A − 5I =     −4  −5 = − − =  =O
 2 3  2 3 2 3 0 1 8 17  8 12 0 5 0 0
Hence, Cayley-Hamilton theorem verified.
Now, by using Cayley-Hamilton theorem, we have
A2 − 4 A − 5 I = 0 , by applying 𝐴−1 on both the sides

A−1 ( A2 − 4 A − 5I ) = A−1 ( 0 )
 A − 4 I − 5 A−1 = 0
 5 A−1 = A − 4 I
1 1  1 4  1 0   1  −3 4 
 A−1 = ( A − 4I ) =    −4  =  
5 5  2 3 0 1   5  2 −1
And for 𝐴4 , applying 𝐴2 both the sides

A2 ( A2 − 4 A − 5 I ) = A 2 ( 0 )
 A4 − 4 A3 − 5 A2 = 0
 A4 = 4 A3 + 5 A2
 41 84  9 16   209 416 
 A4 = 4   +5   A4 =  
 42 83 8 17   208 417 
2 1 1
Example: Find the characteristics equation of the matrix 𝐴 = [0 1 0]and hence prove that
1 1 2

43
 Linear Algebra

Solution: The characteristics equation is

By Cayley-Hamilton Theorem
..............(1)
Now,
𝐴8 − 5𝐴7 + 7𝐴6 − 3𝐴4 − 5𝐴3 + 8𝐴2 − 2𝐴 + 𝐼

Exercise:
1 2 −2
(1)Verify Cayley-Hamilton theorem and hence find the inverse of 𝐴 = [−1 3 0 ] and 𝐴4 .
0 −2 1
1 2 3
(2) Compute 𝐴9 − 6𝐴8 + 10𝐴7 − 3𝐴6 + 𝐴 + 𝐼, where 𝐴 = [−1 3 1]
1 0 2
2 −1 1
−1
(3) Using Cayley-Hamilton theorem find 𝐴 for the following matrix ⌊−1 2 −1⌋ .
1 −1 2
−1 1 −1
(4) Using Cayley-Hamilton theorem find 𝐴 for A=[ ]
2 3

2.0 Diagonalization of a matrix:

An 𝑛 × 𝑛 matrix 𝐴 is diagonalizable if and only if 𝐴 has 𝑛 linearly independent eigenvectors.

44
 Linear Algebra

OR
If 𝑛 × 𝑛 matrix 𝐴 has a basis of eigenvectors, then D = P −1 AP is diagonal, with the eigenvalues of 𝐴 as
the entries on the main diagonal. Here,𝑃 is the matrix with these eigenvectors as column vectors.
Also, Dn = P −1 An P and An = PDn P −1 3

Example-34: Find a matrix 𝑃 that diagonalizes matrix 𝐴 and determine 𝑃 −1 𝐴𝑃

Solution : =
The characteristic equation is |𝐴 − 𝜆𝐼𝑛 | = 0

For  = 1

For  = 3

=
=

=
=

=
=

45
 Linear Algebra

=
=

For  = 2

Example: Find a matrix P that diagonalizes A and determine P-1APwhere

Also find A10 and find eigenvalues of A2.

Solution :
The characteristic equation is =
|𝐴 − 𝜆𝐼𝑛 | = 0

46
 Linear Algebra

𝑦 = 𝑘, 6𝑥 = 0 => 𝑥 = 0
∴ (𝑥, 𝑦) = {𝑘(0,1)/𝑘 ∈ 𝑅}

Now,
For  = 1

Suppose 𝑥 = 𝑘, 6𝑥 − 2𝑦 = 0
𝑥 = 𝑘, 𝑦 = 3𝑘
∴ (𝑥, 𝑦) = {𝑘(1,3)/𝑘 ∈ 𝑅}
For  = −1

( Continue on previous page …)

Eigenvalues of A2 are: 12 = 1 and(−1)2 = 1.

1.11 Application of Matrices:

1.11.1 Coding and Encoding
Example
Consider the message to be sent: BEST WISHES
We take the standard codes as follows:
A = 1 ; B =2 ; ........; Z = 26 and Space = 0
1. We convert the above message in to a stream of numerical values as follows:
BEST WISHES 2 5 19 20 0 23 9 19 8 5 19
2. We construct the message matrix M with this stream of numerals as

M = which is of order 4 X 3

3. Then we perform the product MA, where A is an arbitrary nonsingular matrix given by

47
 Linear Algebra

 1
1 0
3
1 0 −1  
  1 −1 
A = 0 2 2  whose inverse is given by A = 0
−1
 2 3
0 0 3   
0 1
0
 3 
4. Then we get matrix
 2 5 19   2 10 65 
 20 0 23 1 0 −1  20 0 49 
X = MA =   0 2 2  = 
 9 19 8     9 38 53
  0 0 3   
 5 19 0   5 38 33

 1 
1 0
 2 10 65   3   2 5 19 
 20 0 49     
 0 1 1   20 0 23
5. M = XA = 
−1
− =
 9 38 53   2 3   9 19 8 
  
1   5 19 0 
 5 38 33  0 0
 3 
This matrix M is converted into numerical message as
2 5 19 20 0 23 9 19 8 5 19
This stream of numerical is converted into the text message as
2 5 19 20 0 23 9 19 8 5 19: BEST WISHES
Example
The Encoding Process
1. Convert the Message to Numbers
The word "MATH" corresponds to the numbers: 13, 1, 20, 8.
2. Create the Message Matrix (M)
Group the numbers into a matrix. Since there are 4 letters, we can form a 2x2 matrix.
13 20
M = 
1 8 
3. Choose an Invertible Encoding Matrix (A)
Select a square encoding matrix that has an inverse (its determinant must be non-zero).
We will use a simple 2x2 matrix as an example.
2 1
A= 
 3 2

48
 Linear Algebra

The determinant of A is det(A) = (2 * 2) - (1 * 3) = 4 - 3 = 1.

Since the determinant is 1 (not zero), the matrix is invertible.
4. Encode the Message
Multiply the message matrix (M) by the encoding matrix (A) to get the coded matrix (C).
 2 1  13 20
C = AXM =   
 3 2  1 8 
 ( 2 13) + (11) ( 3 13) + ( 2 1)   26 + 1 39 + 2   27 41
C= = = ]
( 2  20 ) + (1 8 ) ( 3  20 ) + ( 2  8) 40 + 8 60 + 16   48 76

The encoded message is the sequence of numbers from matrix C: 27, 41, 48, This is what is transmitted.
The Decoding Process
The receiver gets the sequence 27, 41, 48, 76 and knows the encoding matrix A.
1. Form the Coded Matrix (C)
 27 48
The receiver arranges the numbers back into the 2x2 matrix form: C =  
 41 76
2. Find the Inverse of the Encoding Matrix
To decode, the receiver must find the inverse of the encoding matrix A.
For a 2 x 2 matrix
a b  1
A=  , the inverse is A−1 = [a * d − b * c]
 c d  det( A)

1  2 −1  2 −1
det( A) = 1 . The inverse is: A−1 =  =
1 −3 2   −3 2 
3. Decode the Message
Multiply the coded matrix (C) by the inverse matrix ( A−1 ) to recover the original message matrix
(M).
 2 −1  27 48
M = A−1 X C =  X  
 −3 2   41 76 
 (2 X 27) + ( −1X 41) (2 X 48) + ( −1X 76) 
M = 
(−3 X 27) + (2 X 41) ( −3 X 48) + (2 X 76) 
 54 − 41 96 − 76  13 20 
M =  = 
 −81 + 82 −144 + 152   1 8 
4. Convert Numbers back to the Message
The resulting numbers are 13, 1, 20, 8, which, using the correspondence table, translate back to M,

49
 Linear Algebra

A, T, H. The original message is recovered.

1.11.2 Principal Component Analysis (PCA)
- PCA (Principal Component Analysis) is a dimensionality reduction technique and helps us to reduce the
number of features in a dataset while keeping the most important information.
- It changes complex datasets by transforming correlated features into a smaller set of uncorrelated
components.
- It helps us to remove redundancy, improve computational efficiency and make data easier to visualize and
analyze

Principal Component Analysis (PCA)

- Principal Component Analysis (PCA) is a linear algebra–based technique used to reduce the
number of variables (dimensions) in a dataset while keeping maximum information (variance).
- In simple words, PCA converts correlated variables into a new set of uncorrelated variables called
principal components using eigenvalues and eigenvectors.
- Linear Algebra behind PCA is it is completely based on matrices, eigenvalues, and eigenvectors.
- Main concepts used in PCA are Mean, Covariance matrix, eigenvalues, eigen-vectors and Matrix
Transformation.
- Principal components are orthogonal
- Eigenvectors = directions
- Eigenvalues = importance
- PCA is a dimension reduction technique

Need for PCA

- To reduce dimensionality of large datasets
- To remove redundancy in data
- To visualize high-dimensional data in 2D or 3D
- To speed up calculations in machine learning
- To remove noise from data

Example: Find PCA for the matrix

50
 Linear Algebra

Step 1: Arrange Data in Matrix Form

Suppose we have a dataset with two variables x and y:

Each row = observation, each column = variable.

Step 2: Mean Centering the Data

Subtract mean of each column from the data:

Mean of column 1 = 4
Mean of column 2 = 6

 −2 −2
X c e n t e r e d =  0 0 
 2 2 
Step 3: Find Covariance Matrix
1
Formula: 𝑐𝑜𝑣 = 𝑛−1 𝑋𝑐𝑇 𝑋𝑐

 4 4
For above data: cov =  
 4 4

Step 4: Find Eigenvalues and Eigenvectors

Solve:

Eigenvalues:

Corresponding eigenvectors:

1 1
v1 =   , v2 =  
1  −1

Interpretation of Eigenvalues and Eigenvectors

Eigenvector → direction of new axis (principal component)

51
 Linear Algebra

Eigenvalue → amount of variance in that direction

Largest eigenvalue = most important principal component

1
Here, eigenvalue 8 is largest → keep eigenvector  
1

Step 5: Form Principal Components

1
Principal Component 1 ( PC1 ) is the eigenvector with highest eigenvalue: PC1 =  
1

This becomes the new axis.

Step 6: Project Data onto Principal Component

Z = X centered PC1

This reduces 2D data → 1D data.

3 1
Example: Given covariance matrix:  
1 3
Characteristic equation:
Eigenvalues: So, first principal component corresponds to eigenvalue 4.
1 2
Example: Find the First Principal Component of Data Matrix: [2 3]
3 4
𝐒𝐭𝐞𝐩 𝟏: Mean of column 1 = 2
Mean of column 2 = 3
Mean of columns = (2, 3)

Step 2: Mean Centered Matrix:

1 − 2 2 − 3 −1 −1
Subtract column mean from each element:[2 − 2 3 − 3]=[ 0 0]
3−2 4−3 1 1
1
Step 3: Covariance Matrix: 𝑐𝑜𝑣 = 𝑛−1 𝑋𝑐𝑇 𝑋𝑐
−1 0 1
𝑋𝑐𝑇 = [ ]
−1 0 1
2 2
𝑋𝑐𝑇 𝑋𝑐 = [ ]
2 2
1 2 2
Since n = 3, 𝑐𝑜𝑣 = 3−1 [ ]
2 2

1 1
𝑐𝑜𝑣 = [ ]
1 1

52
 Linear Algebra

Step 4: Find Eigenvalues

Solve: 1−λ 1
Cov −  I = 0  | |=0
1 1−λ
(1 − λ)2 − 1 = 0 ⇒ λ2 − 2λ = 0
Λ1 = 2, λ2 = 0
Eigenvalues: 𝜆1 = 2, 𝜆2 = 0
First Principal Component corresponds to 𝜆1 .:

4 2
Example: Find PCA Using Given Covariance Matrix [ ]
2 3
𝟒 𝟐
Given Covariance Matrix 𝑨 = [ ]
𝟐 𝟑
Solution:
|𝐴 − 𝜆𝐼| = 0
4−λ 2
[ ]=0
2 3−λ
(4 − 𝜆)(3 − 𝜆) − 4 = 0
𝜆2 − 7λ + 8 = 0
(λ − 4)(λ − 3) = 0
𝜆1 = 4, 𝜆2 = 3
First principal component corresponds to λ = 4
It explains maximum variance in the data

53
 Linear Algebra

References:
1. Elementary Linear Algebra (Textbook) By Howard Anton, Chris Rorres| Willy India Edition|
9thEdition
2. Advanced Engineering Mathematics, Erwin Kreyszig, John Wiley & Sons.
3. Linear Algebra and Its Applications By G. Strang |Brooks/Cole India
4. Linear Algebra, By K. Hoffman and R. Kunze |Prentice Hall

54
 Linear Algebra

55