LINEAR ALGEBRA 1 & 2

Irena Penev
Computer Science Institute
of Charles University (IÚUK)
 To my students

Irena Penev i
Linear Algebra 1 & 2
Preface

These lecture notes are intended to accompany the two-semester linear algebra
sequence (Linear Algebra 1 & 2) for computer science students at the Faculty of
Mathematics and Physics, Charles University. While the Czech version of the course
has a set textbook (namely, Milan Hladı́k’s excellent Lineárnı́ algebra (nejen) pro
informatiky), no such textbook exists for the English version of the course. To be
sure, a great many linear algebra textbooks have been published in English, and
some of them have quite a lot to recommend them. Unfortunately, none of the
textbooks (in English) that I am familiar with are quite suitable for this course, and
this is essentially because they are either too advanced or not advanced enough. The
former generally assume a level of mathematical maturity that cannot reasonably
be expected of a first-year university student (even one with a solid high school
mathematics background), and they also tend to have few numerical examples. The
latter do not cover the material in sufficient generality for the purposes of this course
(for example, they do not introduce groups and fields, and in particular, they only
cover real or perhaps complex vector spaces, rather than vector spaces over arbitrary
fields). These lecture notes are intended to bridge the gap by being just “advanced
enough,” with a relatively large number of numerical examples throughout. It is
my hope that these notes will be of use to the students taking this linear algebra
sequence in the future, as well as to the instructors teaching it.
Finally, a small note on technology: some of the routine computations in these
lecture notes were done entirely by hand, but others (especially those involving row
reduction of large matrices) were done with the help of online calculators. I have
primarily used [Link] (which, among other
things, enables row reduction of matrices with entries in Zp , for a prime number p),
and occasionally also [Link]

May 14, 2024 Irena Penev

Irena Penev ii
Linear Algebra 1 & 2
A note on revisions

These lecture notes may occasionally be modified and updated. I fix typos and other
errors as I discover them. More substantial modifications may be made between two
academic years when I am teaching this course, or during academic years when I
am not teaching this course. The last substantial revision (not counting minor error
correction) was made on September 29, 2025.

Irena Penev

Irena Penev iii

Linear Algebra 1 & 2
Contents

0 Preliminaries 1
0.1 Mathematical induction . . . . . . . . . . . . . . . . . . . . . . . . . 1
0.1.1 Induction with more than one base case . . . . . . . . . . . . 4
0.1.2 Strong induction . . . . . . . . . . . . . . . . . . . . . . . . . 7
0.2 Modular arithmetic. Arithmetic in Zn . . . . . . . . . . . . . . . . . 9
0.2.1 Modular arithmetic . . . . . . . . . . . . . . . . . . . . . . . 9
0.2.2 Arithmetic in Zn . Fermat’s Little Theorem . . . . . . . . . . 15
0.3 Complex numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
0.3.1 Complex numbers: definition, basic properties, and examples 22
0.3.2 The Fundamental Theorem of Algebra . . . . . . . . . . . . . 27

1 Systems of linear equations. Vectors and matrices 30

1.1 An informal introduction to fields . . . . . . . . . . . . . . . . . . . . 30
1.2 Vectors and matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.2.1 Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
1.2.2 Column vectors (or simply vectors) . . . . . . . . . . . . . . . 33
1.2.3 Row vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34
1.2.4 Specifying matrices in terms of their rows or columns . . . . 34
1.3 Systems of linear equations . . . . . . . . . . . . . . . . . . . . . . . 35
1.3.1 The augmented matrix and the coefficient matrix of a linear
system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
1.3.2 Elementary row operations . . . . . . . . . . . . . . . . . . . 39
1.3.3 Row reduction . . . . . . . . . . . . . . . . . . . . . . . . . . 43
1.3.4 Solving linear systems . . . . . . . . . . . . . . . . . . . . . . 55
1.3.5 Solving systems of linear equations via back substitution . . . 69
1.3.6 A few more remarks about the (reduced) row echelon form . 72
1.3.7 Proof of Theorem 1.3.6 and Corollaries 1.3.7 and 1.3.8 . . . . 76
1.4 Algebraic operations on vectors and matrices . . . . . . . . . . . . . 80
1.4.1 Vector addition, vector subtraction, and scalar-vector multipli-
cation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
1.4.2 Vector addition and scalar multiplication in R2 . . . . . . . . 82
1.4.3 Linear combinations of vectors . . . . . . . . . . . . . . . . . 83

Irena Penev iv
Linear Algebra 1 & 2
 1.4.4 Matrix-vector multiplication . . . . . . . . . . . . . . . . . . . 86
1.5 Matrix-vector equations . . . . . . . . . . . . . . . . . . . . . . . . . 90
1.5.1 Matrix-vector equations and linear span . . . . . . . . . . . . 99
1.6 The rank of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 105
1.6.1 Rank and the number of solutions of a matrix-vector equation 107
1.6.2 Matrices of full rank . . . . . . . . . . . . . . . . . . . . . . . 109
1.7 Matrix operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
1.7.1 Matrix addition, matrix subtraction, and scalar-matrix multi-
plication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112
1.7.2 Matrix multiplication . . . . . . . . . . . . . . . . . . . . . . 113
1.7.3 Properties of matrix addition and multiplication . . . . . . . 118
1.7.4 Matrix powers . . . . . . . . . . . . . . . . . . . . . . . . . . 122
1.8 The transpose of a matrix . . . . . . . . . . . . . . . . . . . . . . . . 122
1.9 Solving matrix equations of the form AX = B and XA = B . . . . . 124
1.9.1 Solving matrix equations of the form AX = B . . . . . . . . 124
1.9.2 Solving matrix equations of the form XA = B . . . . . . . . 130
1.10 A first look at linear functions and their matrices . . . . . . . . . . . 133
1.10.1 Linear functions: definition and examples . . . . . . . . . . . 133
1.10.2 The images of lines under linear functions f : Rm → Rn . . . 137
1.10.3 Matrix transformations. The standard matrix of a linear function138
1.10.4 Checking the existence and uniqueness of linear functions with
certain specifications . . . . . . . . . . . . . . . . . . . . . . . 142
1.10.5 Some geometric examples . . . . . . . . . . . . . . . . . . . . 149
1.10.6 Making new linear functions out of old ones . . . . . . . . . . 153
1.10.7 One-to-one and onto functions. Bijections and their inverses . 155
1.10.8 A first look at isomorphisms . . . . . . . . . . . . . . . . . . 158
1.11 Invertible matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 161
1.11.1 Invertible matrices: definition and uniqueness of inverses . . . 161
1.11.2 Computing the inverse of an invertible matrix . . . . . . . . . 162
1.11.3 Basic properties of invertible matrices . . . . . . . . . . . . . 164
1.11.4 Invertible matrices, isomorphisms, and rank . . . . . . . . . . 167
1.11.5 Elementary matrices and row reduction . . . . . . . . . . . . 169
1.11.6 Proof of Theorem 1.11.4 . . . . . . . . . . . . . . . . . . . . . 171
1.11.7 The Invertible matrix theorem (version 1) . . . . . . . . . . . 172

2 Groups and permutations. Fields 174

2.1 Monoids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
2.2 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
2.2.1 Groups: definition and basic properties . . . . . . . . . . . . 175
2.2.2 Groups of matrices and vectors. The general linear group . . 180
2.2.3 Subgroups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
2.3 Permutations and the symmetric group . . . . . . . . . . . . . . . . 183

Irena Penev v
Linear Algebra 1 & 2
 2.3.1 Cycle notation . . . . . . . . . . . . . . . . . . . . . . . . . . 184
2.3.2 The sign of a permutation. Even and odd permutations . . . 188
2.3.3 Transpositions . . . . . . . . . . . . . . . . . . . . . . . . . . 189
2.3.4 The alternating group An . . . . . . . . . . . . . . . . . . . . 194
2.3.5 Symmetries of polygons . . . . . . . . . . . . . . . . . . . . . 194
2.3.6 Inversions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
2.3.7 Permutation matrices . . . . . . . . . . . . . . . . . . . . . . 199
2.4 Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
2.4.1 Fields: definition, examples, and basic properties . . . . . . . 205
2.4.2 Finite fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . 208
2.4.3 The fraction notation in fields . . . . . . . . . . . . . . . . . . 209
2.4.4 The characteristic of a field . . . . . . . . . . . . . . . . . . . 209
2.4.5 Algebraically closed fields . . . . . . . . . . . . . . . . . . . . 210

3 Vector spaces 213

3.1 Vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 213
3.1.1 Vector (linear) subspaces . . . . . . . . . . . . . . . . . . . . 215
3.1.2 Linear combinations and linear span . . . . . . . . . . . . . . 217
3.1.3 Making new vector (sub)spaces out of old ones . . . . . . . . 222
3.2 Bases of vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 223
3.2.1 Linear independence . . . . . . . . . . . . . . . . . . . . . . . 223
3.2.2 Bases of vector spaces: definition and basic properties . . . . 224
3.2.3 “Shrinking” a spanning set to a basis . . . . . . . . . . . . . . 230
3.2.4 The Steinitz exchange lemma . . . . . . . . . . . . . . . . . . 232
3.2.5 The dimension of a finite-dimensional vector space . . . . . . 236
3.2.6 On the dimension of some vector spaces obtained from old ones240
3.2.7 A very brief introduction to infinite bases . . . . . . . . . . . 242
3.3 The column space, row space, and null space of a matrix. The rank
of a matrix revisited . . . . . . . . . . . . . . . . . . . . . . . . . . . 243
3.3.1 The column space and row space of a matrix (and their rela-
tionship with rank) . . . . . . . . . . . . . . . . . . . . . . . . 243
3.3.2 The rank of a matrix and elementary row and column operations254
3.3.3 Matrices of full rank . . . . . . . . . . . . . . . . . . . . . . . 256
3.3.4 The rank of matrix products. Left and right inverses of a matrix258
3.3.5 Extending a linearly independent set of vectors in Fn to a basis
of Fn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
3.3.6 The null space of a matrix . . . . . . . . . . . . . . . . . . . . 264
3.3.7 The Invertible Matrix Theorem (version 2) . . . . . . . . . . 268

4 Linear functions 271

4.1 Linear functions: definition, examples, and basic properties . . . . . 271
4.1.1 Basic properties of linear functions . . . . . . . . . . . . . . . 273
4.1.2 Making new linear functions out of old ones . . . . . . . . . . 275

Irena Penev vi
Linear Algebra 1 & 2
 4.1.3 A remark on infinity . . . . . . . . . . . . . . . . . . . . . . . 276
4.2 The image and kernel of a linear function. The rank-nullity theorem 277
4.2.1 One-to-one linear functions and kernel . . . . . . . . . . . . . 280
4.2.2 The rank of a linear function. The rank-nullity theorem . . . 280
4.2.3 The effect of a linear function on linearly independent and
spanning sets . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
4.2.4 Computing bases of the images and preimages of subspaces
under linear functions . . . . . . . . . . . . . . . . . . . . . . 291
4.3 Linear functions and bases . . . . . . . . . . . . . . . . . . . . . . . . 302
4.4 Isomorphisms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
4.4.1 Basic properties of isomorphisms . . . . . . . . . . . . . . . . 306
4.4.2 Isomorphism and dimension . . . . . . . . . . . . . . . . . . . 309
4.4.3 An application of isomorphisms: transforming polynomials
and matrices into vectors . . . . . . . . . . . . . . . . . . . . 310
4.5 Matrices of linear functions between non-trivial, finite-dimensional
vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326
4.5.1 Change of basis (transition) matrices . . . . . . . . . . . . . . 339
4.5.2 Similar matrices . . . . . . . . . . . . . . . . . . . . . . . . . 343
4.5.3 Checking the existence and uniqueness of linear functions with
certain specifications: examples with polynomials and matrices 347

5 Affine subspaces and affine functions 368

5.1 Affine subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 368
5.2 Affine functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 371
5.2.1 Making new affine functions out of old ones . . . . . . . . . . 372
5.2.2 The image of an affine subspace under an affine function . . . 372
5.2.3 The preimage of an affine subspace under an affine function . 375
5.3 Affine combinations and affine hulls . . . . . . . . . . . . . . . . . . 379
5.4 Affine frames and affine bases . . . . . . . . . . . . . . . . . . . . . . 381
5.4.1 Affine frames . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
5.4.2 Affine independence . . . . . . . . . . . . . . . . . . . . . . . 383
5.4.3 Affine bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . 384
5.4.4 A relationship between affine bases and affine frames . . . . . 385

6 Scalar (inner) products, norms, and orthogonality 389

6.1 The scalar product . . . . . . . . . . . . . . . . . . . . . . . . . . . . 389
6.1.1 The scalar product in real vector spaces . . . . . . . . . . . . 389
6.1.2 The scalar product in complex vector spaces . . . . . . . . . 394
6.1.3 Orthogonality . . . . . . . . . . . . . . . . . . . . . . . . . . . 396
6.2 The norm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 397
6.2.1 The norm induced by a scalar product . . . . . . . . . . . . . 398
6.2.2 The Pythagorean theorem, the Cauchy–Schwarz inequality,
and the triangle inequality . . . . . . . . . . . . . . . . . . . . 400

Irena Penev vii

Linear Algebra 1 & 2
 6.2.3 The norm in general . . . . . . . . . . . . . . . . . . . . . . . 402
6.2.4 Other examples of norms . . . . . . . . . . . . . . . . . . . . 404
6.3 Orthogonal and orthonormal bases. Gram-Schmidt orthogonalization 405
6.3.1 Vector projection . . . . . . . . . . . . . . . . . . . . . . . . . 405
6.3.2 Orthogonal and orthonormal sets. Orthogonal and orthonor-
mal bases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 407
6.3.3 Coordinate vectors with respect to orthogonal and orthonormal
bases. Fourier coefficients . . . . . . . . . . . . . . . . . . . . 409
6.3.4 Gram-Schmidt orthogonalization . . . . . . . . . . . . . . . . 410
6.4 The orthogonal complement of a subspace . . . . . . . . . . . . . . . 421
6.5 Orthogonal projection onto a subspace . . . . . . . . . . . . . . . . . 427
6.6 Orthogonal projection onto subspaces of Rn . . . . . . . . . . . . . . 431
6.7 The method of least squares . . . . . . . . . . . . . . . . . . . . . . . 436
6.7.1 Data fitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . 438
6.8 Orthogonal matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . 441
6.8.1 Orthogonal matrices: definition and characterization . . . . . 441
6.8.2 Making new orthogonal matrices out of old ones . . . . . . . 442
6.8.3 The Householder matrix . . . . . . . . . . . . . . . . . . . . . 444
6.8.4 The Givens matrix . . . . . . . . . . . . . . . . . . . . . . . . 446
6.8.5 Orthogonal matrices, length, and angles . . . . . . . . . . . . 448
6.9 Scalar product, coordinate vectors, and matrices of linear functions . 448

7 Determinants 452
7.1 Determinants: definition, examples, and basic properties . . . . . . . 452
7.1.1 Some matrices whose determinants are zero . . . . . . . . . . 457
7.2 The linearity of determinants in one row or one column . . . . . . . 459
7.3 Computing determinants via elementary row and column operations 462
7.3.1 Computing the determinant of a triangular matrix . . . . . . 462
7.3.2 Determinants and elementary row and column operations . . 464
7.4 Determinants and matrix invertibility . . . . . . . . . . . . . . . . . 468
7.4.1 The Invertible Matrix Theorem (version 3) . . . . . . . . . . 469
7.5 The multiplicative property of determinants . . . . . . . . . . . . . . 470
7.6 Laplace expansion . . . . . . . . . . . . . . . . . . . . . . . . . . . . 473
7.7 Cramer’s rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 480
7.8 The adjugate matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 482
7.9 The Vandermonde matrix . . . . . . . . . . . . . . . . . . . . . . . . 486
7.10 Determinants and volume . . . . . . . . . . . . . . . . . . . . . . . . 488
7.11 Common roots of polynomials via determinants . . . . . . . . . . . . 498

8 Eigenvalues and eigenvectors 503

8.1 Eigenvectors, eigenvalues, and eigenspaces . . . . . . . . . . . . . . . 503
8.1.1 Eigenvectors and eigenvalues of linear functions . . . . . . . . 503
8.1.2 Eigenvectors and eigenvalues of square matrices . . . . . . . . 507

Irena Penev viii

Linear Algebra 1 & 2
 8.2 The characteristic polynomial . . . . . . . . . . . . . . . . . . . . . . 510
8.2.1 The characteristic polynomial of a square matrix . . . . . . . 510
8.2.2 Some numerical examples . . . . . . . . . . . . . . . . . . . . 513
8.2.3 Eigenvectors and eigenvalues of similar matrices . . . . . . . 522
8.2.4 A relationship between the spectrum, trace, and determinant
of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 523
8.2.5 Eigenvalues and invertibility . . . . . . . . . . . . . . . . . . . 524
8.2.6 The Invertible Matrix Theorem (version 4) . . . . . . . . . . 525
8.2.7 The characteristic polynomial and spectrum of a linear function526
8.2.8 Proof of Theorem 8.2.3 . . . . . . . . . . . . . . . . . . . . . 533
8.3 The Cayley-Hamilton theorem . . . . . . . . . . . . . . . . . . . . . 535
8.4 Eigenvectors and linear independence. Eigenbases . . . . . . . . . . . 538
8.5 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 543
8.5.1 Diagonal matrices and their powers . . . . . . . . . . . . . . . 543
8.5.2 Eigenbases and diagonal matrices of linear functions . . . . . 545
8.5.3 Matrix diagonalization . . . . . . . . . . . . . . . . . . . . . . 548
8.6 The Jordan normal form . . . . . . . . . . . . . . . . . . . . . . . . . 558
8.6.1 The Jordan normal form: definitions and statements of main
theorems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 559
8.6.2 Computing the Jordan normal form of a square matrix . . . . 570
8.6.3 A brief outline of the proof of Theorems 8.6.2 and 8.6.4 . . . 577
8.6.4 Invariant subspaces and the uniqueness of the Jordan normal
form: the proof of Theorem 8.6.1 . . . . . . . . . . . . . . . . 584
8.6.5 Generalized eigenspaces, nilpotent linear functions, and the
existence of the Jordan normal form of a square matrix . . . 593
8.6.6 The proof of Theorems 8.6.2, 8.6.4, and 8.6.6 . . . . . . . . . 613
8.6.7 Computing the Jordan normal form of a square matrix (once
more): computing both J and P . . . . . . . . . . . . . . . . 614
8.7 Symmetric matrices and orthogonal diagonalization . . . . . . . . . . 650
8.7.1 Symmetric and Hermitian matrices . . . . . . . . . . . . . . . 650
8.7.2 Orthogonal diagonalizability . . . . . . . . . . . . . . . . . . 653
8.7.3 The spectral theorem for symmetric matrices . . . . . . . . . 656
8.7.4 Diagonally orthogonalizing a symmetric matrix . . . . . . . . 658

9 Bilinear and quadratic forms 660

9.1 A formula for products of the form xT Ay . . . . . . . . . . . . . . . 660
9.2 Bilinear forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 661
9.2.1 Matrix congruence . . . . . . . . . . . . . . . . . . . . . . . . 667
9.2.2 Symmetric bilinear forms on vector spaces over fields of char-
acteristic other than 2 . . . . . . . . . . . . . . . . . . . . . . 669
9.3 Quadratic forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
9.3.1 Quadratic forms and their matrices . . . . . . . . . . . . . . . 669

Irena Penev ix
Linear Algebra 1 & 2
 9.3.2 Matrix congruence and quadratic forms . . . . . . . . . . . . 672
9.4 Quadratic forms on Rn . . . . . . . . . . . . . . . . . . . . . . . . . . 673
9.4.1 Matrix congruence and the signature of a symmetric matrix
with real entries . . . . . . . . . . . . . . . . . . . . . . . . . 673
9.4.2 Sylvester’s law of inertia . . . . . . . . . . . . . . . . . . . . . 679
9.4.3 Quadratic forms on R2 . . . . . . . . . . . . . . . . . . . . . . 681

10 Matrix definiteness 683

10.1 Matrix definiteness: definition and basic properties . . . . . . . . . . 683
10.2 Methods of testing for positive definiteness . . . . . . . . . . . . . . 688
10.2.1 The recursive test of positive definiteness . . . . . . . . . . . 688
10.2.2 The Gaussian elimination test of positive definiteness . . . . 693
10.2.3 Sylvester’s criterion of positive definiteness . . . . . . . . . . 697
10.3 The Cholesky decomposition of positive definite matrices . . . . . . 699
10.4 Positive definite matrices and the scalar product . . . . . . . . . . . 703

Bibliography 708

Irena Penev x
Linear Algebra 1 & 2
This page is intentionally left blank.
Chapter 0. Preliminaries 1

Chapter 0

Preliminaries

Notation: In what follows, we will use the following notation:

ˆ N is the set of all natural numbers (positive integers);

ˆ N0 is the set of all non-negative integers;

ˆ Z is the set of all integers;

ˆ Q is the set of all rational numbers;

ˆ R is the set of all real numbers;

ˆ C is the set of all complex numbers.

0.1 Mathematical induction

Mathematical induction is a proof technique that can be used to prove that a certain
statement holds for all positive integers n.
Let P (n) be a statement about the number n. In order to prove that P (n) holds
for every positive integer n, it suffices to prove the following two statements:

ˆ Base case: P (1) is true;

ˆ Induction step: for every positive integer n,

if P (n) is true , then P (n + 1) is true.
| {z }
“induction hypothesis”

Why does this work? Here is the intuition: We are trying to prove an infinite
sequence of statements, namely,

P (1), P (2), P (3), P (4), . . .

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 2

By the base case, P (1) is true. By the induction step, since P (1) is true, P (2) is
also true. Again by the induction step, since P (2) is true, so is P (3). Once again by
the induction step, since P (3) is true, so is P (4). And so on! Thus, P (n) is true for
all positive integers n. Schematically, this is shown in the diagram below.

ind. step ind. step ind. step ind. step

P (1) =⇒ P (2) =⇒ P (3) =⇒ P (4) =⇒ ...

base
case
n(n+1)
Example 0.1.1. Prove that 1 + 2 + · · · + n = 2 for all positive integers n.
n(n+1)
Solution. Let P (n) be the statement that 1 + 2 + · · · + n = 2 . Thus:
1·(1+1)
ˆ P (1) is the statement that 1 = 2 ;
2·(2+1)
ˆ P (2) is the statement that 1 + 2 = 2 ;
3·(3+1)
ˆ P (3) is the statement that 1 + 2 + 3 = 2 ;

ˆ etc.

We need to prove that the statement P (n) is true for all positive integers n.
1·(1+1)
Base case: n = 1. Obviously, 1 = 2 . Thus, P (1) is true.
Induction step: Fix a positive integer n, and assume inductively that P (n) is true.
We must show that P (n + 1) is true.
The induction hypothesis states that 1 + 2 + · · · + n
= n(n+1)
2 . Using this, we
(n+1) (n+1)+1
must prove that 1 + 2 + · · · + n + (n + 1) = 2 . We compute:

1 + 2 + · · · + n + (n + 1) = (1 + 2 + · · · + n) + (n + 1)

by the
n(n+1)
= 2 + (n + 1) induction
hypothesis

= (n + 1)( n2 + 1)


(n+1) (n+1)+1
= 2 .

Thus, P (n + 1) is true. This completes the induction.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 3

Sometimes, the base case may be different from n = 1. This may happen if we
need to prove that a statement P (n) is true for all n ≥ k, where k is an integer other
than 1. (Typically, we will have k = 0 or k > 1. However, in principle, k may even
be a negative integer.) In this case, the base case will be n = k, i.e. we will need to
prove the following two statements:
ˆ Base case: P (k) is true;
ˆ Induction step: for every integer n ≥ k,
if P (n) is true , then P (n + 1) is true.
| {z }
“induction hypothesis”

Here, the intuition is similar to what we saw above. We are trying to prove an
infinite sequence of statements, namely
P (k), P (k + 1), P (k + 2), P (k + 3), . . .
By the base case, P (k) is true. By the induction step, since P (k) is true, P (k + 1) is
also true. Again by the induction step, since P (k + 1) is true, so is P (k + 2). Once
again by the induction step, since P (k + 2) is true, so is P (k + 3). And so on! Thus,
P (n) is true for all integers n ≥ k. Schematically, this is shown in the diagram below.

ind. step ind. step ind. step ind. step

P (k) =⇒ P (k + 1) =⇒ P (k + 2) =⇒ P (k + 3) =⇒ ...

base
case
Example 0.1.2. Prove that 3n < 2n for all integers n ≥ 4.
Proof. Since we are proving the statement for integers n ≥ 4, our base case is n = 4.
Base case: n = 4. Clearly, 3 · 4 = 12 < 16 = 24 .
Induction step: Fix an integer n ≥ 4, and assume inductively that 3n < 2n . We
must show that 3(n + 1) < 2n+1 . We observe the following:
3(n + 1) = 3n + 3

< 2n + 3 by the induction hypothesis

< 2n + 22

< 2n + 2n because n > 2

= 2n+1
Thus, the statement is true for n + 1. This completes the induction.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 4

0.1.1 Induction with more than one base case

Suppose that k is an integer, and that we wish to prove inductively that P (n) holds for
all integers n ≥ k. Ordinarily, we would expect n = k to be the base case. However,
suppose that we do not know how to prove the implication “P (n) =⇒ P (n + 1),”
but we do know how to prove that “P (n) =⇒ P (n + ℓ),” where ℓ is some positive
integer (other than 1). In this case, we will have a slightly modified induction step
(“P (n) =⇒ P (n + ℓ)” instead of “P (n) =⇒ P (n + 1)”), and we will have ℓ base cases,
namely, P (k), P (k + 1), . . . , P (k + ℓ − 1). More precisely, we will need to prove the
following:
ˆ Base case: P (k), P (k + 1), . . . , P (k + ℓ − 1) are true;
ˆ Induction step: for every integer n ≥ k,
if P (n) is true , then P (n + ℓ) is true.
| {z }
“induction hypothesis”

Indeed, this is enough to show that P (n) holds for all integers n ≥ k. The intuition
behind this is given in the diagram below.

ind. step ind. step ind. step

P (k) =⇒ P (k + ℓ) =⇒ P (k + 2ℓ) =⇒ ...

ind. step ind. step ind. step

P (k + 1) =⇒ P (k + ℓ + 1) =⇒ P (k + 2ℓ + 1) =⇒ ...

.. .. ..
. . .

ind. step ind. step ind. step

P (k + ℓ − 1) =⇒ P (k + 2ℓ − 1) =⇒ P (k + 3ℓ − 1) =⇒ ...

base
case

Example 0.1.3. Suppose you have an unlimited number of 3 Kč stamps and 5 Kč
stamps (and no other stamps). Show that you can pay any amount of postage greater
or equal to 8 Kč (as long as it is in whole Kč).
Solution. We need to show that any integer n ≥ 8 (our postage in Kč) can be
expressed in the form
n = 3a + 5b,
where a and b are non-negative integers (the number of 3 Kč and 5 Kč stamps,
respectively, that we can use to pay our n Kč postage). We will prove this by

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 5

induction on n. Obviously, if we can pay n Kč using our stamps, then we can also
pay (n + 3) Kč: we simply use one 3 Kč stamp more. In other words, if the statement
is true for n, then it is also true for n + 3. This means that we will need three base
cases: n = 8, n = 9, and n = 10. Let us give the details.
Base case: We must show that for each n ∈ {8, 9, 10}, there exist non-negative
integers a and b such that n = 3a + 5b. But this is clearly true:

ˆ 8 = 3 · 1 + 5 · 1;

ˆ 9 = 3 · 3 + 5 · 0;

ˆ 10 = 3 · 0 + 5 · 2.

Induction step: Fix an integer n ≥ 8, and assume inductively that the statement is
true for n. We must show that it is true for n + 3. By the induction hypothesis, there
exist non-negative integers a and b such that n = 3a+5b. But then n+3 = 3(a+1)+5b,
and so the statement holds for n + 3. This completes the induction.

Suppose, again, that k is an integer, and that we wish to prove inductively

that P (n) holds for all integers n ≥ k. However, suppose that we are not able
to prove the implication “P (n) =⇒ P (n + 1),” but that we are able to prove
that P (n), P (n + 1), . . . , P (n + ℓ − 1) together imply P (n + ℓ), where ℓ is some
positive integer (other than 1). In this case, we will again have ℓ base cases, namely,
P (k), P (k + 1), . . . , P (k + ℓ − 1). More precisely, we will need to prove the following:

ˆ Base case: P (k), P (k + 1), . . . , P (k + ℓ − 1) are true;

ˆ Induction step: for every integer n ≥ k,

if P (n), P (n + 1), . . . , P (n + ℓ − 1) are all true, then P (n + ℓ) is true.
| {z }
“induction hypothesis”

Once again, this is enough to show that P (n) holds for all integers n ≥ k. The
intuition behind this is as follows:

ˆ P (k), P (k + 1), . . . , P (k + ℓ − 1) hold by the base case;

ˆ since P (k), P (k + 1), . . . , P (k + ℓ − 1) hold, the induction hypothesis guarantees

that P (k + ℓ) holds;

ˆ now P (k + 1), P (k + 2), . . . , P (k + ℓ) hold, and so by the induction hypothesis,

P (k + ℓ + 1) holds;

ˆ now P (k+2), P (k+3), . . . , P (k+ℓ+1) hold, and so by the induction hypothesis,

P (k + ℓ + 2) holds;

ˆ and so on!

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 6

For the case when ℓ = 2, the idea behind this is illustrated in the diagram below.

ind. step ind. step ind. step ind. step

P (k + 1) =⇒ P (k + 2) =⇒ P (k + 3) =⇒ P (k + 4) =⇒ ...

P (k)

base
case

Example 0.1.4. The Fibonacci numbers are defined as follows:

ˆ F (1) = F (2) = 1;

ˆ F (n + 2) = F (n) + F (n + 1) for all positive intgers n.

√ √
(1+ 5)n −(1− 5)n
Prove that F (n) = √
2n 5
for all positive integers n.

Solution. The general term is defined in terms of the previous two terms. Thus,
instead of one base case, we have two: n = 1 and n = 2.
Remark: If the general term were defined in terms of, say, the previous fifteen
terms, then we would have fifteen base cases!
Base case: For n = 1, we have:
√ √ √
(1+ 5)1 −(1− 5)1 2√5
1
2 5
√ = 2 5
= 1 = F (1).

For n = 2, we have:
√ √ √ √ √
(1+ 5)2 −(1− 5)2 (1+2 5+5)−(1−2 5+5) 4√5
√
22 5
= √
4 5
= 4 5
= 1 = F (2).

Thus, the statement is true for n = 1 and n = 2.

Induction step: Fix a positive integer n, and assume inductively that the statement
is true for n and n + 1. We must show that it is true for n + 2.
By the induction hypothesis, we have that
√ √
(1+ 5)n −(1− 5)n
ˆ F (n) = √
2n 5
;
√ n+1 √
−(1− 5)n+1
ˆ F (n + 1) = (1+ 5) 2n+1 √
5
.
√ n+2 √
−(1− 5)n+2
We must show that F (n + 2) = (1+ 5) 2n+2 √
5
. We compute:

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 7

(∗)
F (n + 2) = F (n) + F (n + 1)
√ √ √ √
(∗∗) (1+ 5)n −(1− 5)n (1+ 5)n+1 −(1− 5)n+1
= 2 n
√
5
+ 2 n+1
√
5
√ √ √ √ √ √
4(1+ 5)n −4(1− 5)n 2(1+ 5)(1+ 5)n −2(1− 5)(1− 5)n
= 2n+2
√
5
+ 2 n+2
√
5
√ √ √ √
(6+2 5)(1+ 5)n −(6−2 5)(1− 5)n
= √
2n+2 5
√ √ √ √
(1+ 5)2 (1+ 5)n −(1− 5)2 (1− 5)n
= √
2n+2 5
√ √
(1+ 5)n+2 −(1− 5)n+2
= √
2n+2 5
,

where (*) follows from the definition of Fibonacci numbers, and (**) follows from
the induction hypothesis. This completes the induction.

0.1.2 Strong induction

We now discuss a type of induction (sometimes called “strong induction”) that lacks
a base case. Again, let P (n) be a statement about the number n. In order to prove
that P (n) holds for every positive integer n, it suffices to prove the following:

ˆ Induction step: for every positive integer n,

if P (1), . . . , P (n − 1) are all true, then P (n) is true.
| {z }
“induction hypothesis”

Here is a slightly different way of writing the same thing:

ˆ Induction step: for every positive integer n,

if P (i) is true for all positive integers i < n, then P (n) is true.
| {z }
“induction hypothesis”

Why does this make sense? Here is the intuition. Suppose that we have proven
the induction step above. For n = 1, the induction hypothesis is vacuously true,1
and so it follows that P (1) is true. Put in another way, for n = 1, the induction
step essentially says “if ‘nothing,’ then P (1) is true,” which is the same as “P (1) is
true.” What about P (2), P (3), P (4), etc.? Since P (1) is true, the induction step
guarantees that P (2) is true. Now P (1), P (2) are true; so, by the induction step,
P (3) is true. Now P (1), P (2), P (3) are true; so, by the induction step, P (4) is true.
And so on! The intuition behind this is summarized in the diagram below.
1
This is because there are no positive integers i < 1.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 8

ind. step ind. step ind. step ind. step

P (1) =⇒ P (2) =⇒ P (3) =⇒ P (4) =⇒ ...

follows from “nothing”

via the induction step

As before, slight variations on the theme are possible. In particular, for a fixed
integer k, we may wish to prove by strong induction that P (n) holds for all integers
n ≥ k. In this case, it is enough to prove the following:
ˆ Induction step: for every integer n ≥ k,
if P (k), . . . , P (n − 1) are all true, then P (n) is true.
| {z }
“induction hypothesis”

Another way of writing the same thing is as follows:

ˆ Induction step: for every integer n ≥ k,
if P (i) is true for all integers i such that k ≤ i < n, then P (n) is true.
| {z }
“induction hypothesis”

Example 0.1.5. Prove that every integer n ≥ 2 can be written as a product of one
or more prime numbers.
Proof. Fix an integer n ≥ 2, and assume inductively that each of 2, . . . , n − 1 can be
written as a product of primes.2 We must show that n can be written as a product
of primes.
Clearly, n is either prime or composite.
Suppose first that n is prime. Then, obviously, n can be written as a product of
primes, namely
n = |{z} n .
prime

Suppose now that n is composite. Then there exist integers n1 , n2 such that
2 ≤ n1 , n2 < n and n = n1 n2 . By the induction hypothesis, n1 and n2 can be
written as products of primes. Set n1 = p1 · · · · · pk and n2 = q1 · · · · · qℓ , where
p1 , . . . , pk , q1 , . . . , qℓ are prime numbers. Then n = n1 n2 = p1 · · · · · pk · q1 · · · · · qℓ .
Thus, n is a product of primes. This completes the induction.
2
In other words, we are assuming that for all integers m such that 2 ≤ m < n, m can be written
as a product of primes. Note that if n = 2, then we are in fact not assuming anything because there
are no integers m satisfying 2 ≤ m < 2.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 9

0.2 Modular arithmetic. Arithmetic in Zn

0.2.1 Modular arithmetic
Given n ∈ N and m ∈ Z, we write n | m if m is divisible by n, that is, if there exists
some k ∈ Z such that m = kn.
Given n ∈ N and a, b ∈ Z, we say that a and b are congruent modulo n, and we
write a ≡ b (mod n) or a ≡n b, provided that n|(a − b), i.e. a − b = kn for some
k ∈ Z; equivalently, we have that a ≡ b (mod n) provided that a and b leave the
same remainder when divided by n (where the remainder is required to be one of
the integers 0, 1, . . . , n − 1). Note that for a positive integer n and an integer a, we
have that a is divisible by n (equivalently: a is a multiple of n) if and only if a ≡ 0
(mod n).

Example 0.2.1. All the following hold:

ˆ 2 ≡ 17 (mod 3); ˆ 2 ̸≡ 17 (mod 2);

ˆ −13 ≡ 8 (mod 7); ˆ −13 ̸≡ 8 (mod 5);

ˆ −1 ≡ 7 (mod 4); ˆ −1 ̸≡ 7 (mod 6).

Remarks:
ˆ For fixed n ∈ N, every integer is congruent modulo n to exactly one of
the following n integers: 0, . . . , n − 1. As we shall see, doing arithmetic
modulo n essentially boils down to doing arithmetic with only n values (namely
0, . . . , n − 1), as opposed to infinitely many. This is quite useful for certain
applications.
ˆ Congruence modulo n can be visualized in terms of an “n-hour clock” (see
the picture below for the case n = 5). Suppose we are given an integer a,
and we wish to determine which of 0, 1, . . . , n − 1 it is congruent to modulo n.
Obviously, if a = 0, then a ≡ 0 (mod n). If a is positive, then we start at 0 and
make a clockwise steps; the number we finish at is the number we need. For
example, we have that 14 ≡ 4 (mod 5) because if we start at 0 and then make
14 steps clockwise on the 5-hour clock, we finish at 4. On the other hand, if a
is negative, then we make |a| = −a many counterclockwise steps. For example,
we have that −7 ≡ 3 (mod 5) because if we start at 0 and then make 7 steps
counterclockwise on the 5-hour clock, then we finish at 3.
0

4
1

3 2

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 10

Proposition 0.2.2. Let n ∈ N and a, b, c ∈ Z. Then the following hold:

(a) a ≡ a (mod n);

(b) if a ≡ b (mod n), then b ≡ a (mod n);

(c) if a ≡ b (mod n) and b ≡ c (mod n), then a ≡ c (mod n).

Proof. (a) and (b) are obvious. For (c), assume that a ≡ b (mod n) and b ≡ c
(mod n). Then n | (a − b) and n | (b − c), i.e. there exist k, ℓ ∈ Z such that a − b = kn
and b − c = ℓn. But then

a − c = (a − b) + (b − c) = kn + ℓn = (k + ℓ)n,

i.e. n | (a − c). Thus, a ≡ c (mod n).

Remark: Proposition 0.2.2 states that congruence modulo n is an “equivalence

relation” on Z. (If you are not yet familiar with equivalence relations, you will soon
learn about them in Discrete Math.)

Proposition 0.2.3. Let n ∈ N and a, b, c, d ∈ Z, and assume that a ≡ b (mod n)

and c ≡ d (mod n). Then:

(a) a + c ≡ b + d (mod n);

(b) a − c ≡ b − d (mod n);

(c) ac ≡ bd (mod n).

Proof. Since a ≡ b (mod n), we have that n|(a − b), and so there exists some k ∈ Z
such that a − b = kn. Similarly, since c ≡ d (mod n), there exists some ℓ ∈ Z such
that c − d = ℓn.
To prove (a), we observe that

(a + c) − (b + d) = (a − b) + (c − d) = kn + ℓn = (k + ℓ)n,


and so n | (a + c) − (b + d) . Thus, a + c ≡ b + d (mod n). This proves (a).
For (b), we observe that

(a − c) − (b − d) = (a − b) − (c − d) = kn − ℓn = (k − ℓ)n,


and so n | (a − c) − (b − d) . Thus, a − c ≡ b − d (mod n). This proves (b).
Finally, for (c), we have that

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 11

ac − bd = ac − ad + ad − bd

= a(c − d) + (a − b)d

= aℓn + knd

= (aℓ + dk)n,

and so n | (ac − bd). Thus, ac ≡ bd (mod n). This proves (c).

Warning: Do not divide!!! For example, we have that 4 ≡ 8 (mod 4), but if we
divide both sides by 2, we get 2 ̸≡ 4 (mod 4).

Proposition 0.2.4. Let n ∈ N and a, b ∈ Z. Assume that a ≡ b (mod n). Then

at ≡ bt (mod n) for all integers t ≥ 0.

Proof. We proceed by induction on t.

Base case: t = 0. By definition, r0 = 1 for all integers r.3 So, a0 = 1 = b0 , and so
by Proposition 0.2.2(a), we have that a0 ≡ b0 (mod n).
Induction case: Fix a non-negative integer t, and assume inductively that at ≡ bt
(mod n). Since we also have that a ≡ b (mod n), Proposition 0.2.3(c) implies that
at a ≡ bt b (mod n), i.e. that at+1 ≡ bt+1 (mod n). This completes the induction.

Remark: In what follows, we will repeatedly use Propositions 0.2.2, 0.2.3, and 0.2.4
without explicitly stating this.

Example 0.2.5. Compute the last digit of 20182019 .

Solution. In principle, we could compute the value of 20182019 , and then simply
check what its last digit is. However, 20182019 is an enormous number, and so this is
impractical (even with the help of a computer). However, note that the last digit of
a non-negative integer is simply its remainder when divided by 10. So, we need only
figure out which of 0, 1, . . . , 9 the number 20182019 is congruent to modulo 10.4
Clearly, 2018 ≡ 8 (mod 10), and so 20182019 ≡ 82019 (mod 10). Now, note the
following:

ˆ 81 ≡ 8 (mod 10);

ˆ 82 ≡ 4 (mod 10);
3
In fact, r0 = 1 for all real numbers r.
4
If we were looking for the last two digits, then we would be considering congruence modulo 100;
for the last three digits, we would need congruence modulo 1000, etc.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 12

ˆ 83 ≡ 2 (mod 10);

ˆ 84 ≡ 6 (mod 10);

ˆ 85 ≡ 8 (mod 10).
This looks like a periodic pattern! The general formula will be as in the Claim below
(and we prove the Claim by mathematical induction).

Claim. For all integers k ≥ 0, we have the following:

ˆ 84k+1 ≡ 8 (mod 10);

ˆ 84k+2 ≡ 4 (mod 10);
ˆ 84k+3 ≡ 2 (mod 10);
ˆ 84k+4 ≡ 6 (mod 10).

Proof of the Claim. We proceed by induction on k.

Base case: For k = 0, we have:
ˆ 84·0+1 = 8 ≡10 8;

ˆ 84·0+2 = 82 = 64 ≡10 4;
(∗)
ˆ 84·0+3 = 83 = 8 · 82 ≡ 10 8 · 4 = 32 ≡10 2, where for (*), we used the fact that
82 ≡10 4 (proven above);
(∗)
ˆ 84·0+4 = 84 = 8 · 83 ≡ 10 8 · 2 = 16 ≡10 6, where for (*) we used the fact that
83 ≡10 2 (proven above).
Thus, the claim is true for k = 0.
Induction step: Fix a non-negative integer k, and assume inductively that the
statement is true for k.5 We must show that it is true for k + 1.6 We saw in the
5
So, we are assuming that all the following hold:
ˆ 84k+1 ≡ 8 (mod 10);
ˆ 84k+2 ≡ 4 (mod 10);
ˆ 84k+3 ≡ 2 (mod 10);
ˆ 84k+4 ≡ 6 (mod 10).
6
So, we must prove all the following:
ˆ 84(k+1)+1 ≡ 8 (mod 10);
ˆ 84(k+1)+2 ≡ 4 (mod 10);
ˆ 84(k+1)+3 ≡ 2 (mod 10);
ˆ 84(k+1)+4 ≡ 6 (mod 10).

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 13

base case that 84 ≡ 6 (mod 10), and consequently, for all non-negative integers ℓ, we
have that
84(k+1)+ℓ = 84 · 84k+ℓ ≡10 6 · 84k+ℓ .
In the following calculations, (*) follows from what we just showed,7 and (**) follows
from the induction hypothesis. We compute:
(∗) (∗∗)
ˆ 84(k+1)+1 ≡ 10 6 · 84k+1 ≡ 10 6 · 8 = 48 ≡10 8;
(∗) (∗∗)
ˆ 84(k+1)+2 ≡ 10 6 · 84k+2 ≡ 10 6 · 4 = 24 ≡10 4;
(∗) (∗∗)
ˆ 84(k+1)+3 ≡ 10 6 · 84k+3 ≡ 10 6 · 2 = 12 ≡10 2;
(∗) (∗∗)
ˆ 84(k+1)+4 ≡ 10 6 · 84k+4 ≡ 10 6 · 6 = 36 ≡10 6.
This completes the induction. ♦
Since 2019 = 4 · 504 + 3, the Claim guarantees that 82019 ≡ 2 (mod 10). Conse-
quently,
20182019 ≡ 82019 ≡ 2 (mod 10),
and it follows that the last digit of 20182019 is 2.

Notation: For an , an−1 , . . . , a0 ∈ {0, 1, . . . , 9}, we define:

n
ak 10k .
P
an an−1 . . . a0 :=
k=0

Thus, an an−1 . . . a0 is the number whose first digit is an ,8 whose second digit is an−1 ,
and so on.

Proposition 0.2.6. Let a = an an−1 . . . a0 . Then a ≡ an + an−1 + · · · + a0 (mod 9).

Therefore, a positive integer is divisible by 9 if and only if the sum of its digits is
divisible by 9.
Proof. By definition, an integer is divisible by 9 if and only if it is congruent to 0
modulo 9. So, the second statement of the proposition follows immediately from
the first. It remains to prove the first statement. Note that 10 ≡ 1 (mod 9). So, by
Proposition 0.2.4, we have that 10k ≡ 1 (mod 9) for all non-negative integers k. It
follows that for all k ∈ {0, . . . , n}, we have that ak · 10k ≡ ak (mod 9). Consequently,
n n
ak 10k ≡9
P P
a = an an−1 . . . a0 = ak = an + an−1 + · · · + a0 ,
k=0 k=0

which is what we needed to show.

7
That is, from the fact that 84(k+1)+ℓ ≡10 6 · 84k+ℓ for all non-negative integers ℓ.
8
It is possible that this first digit is zero. We could eliminate this possibility, but that would
result in a messier definition.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 14

Proposition 0.2.7. Let a = an an−1 . . . a0 . Then a ≡ an + an−1 + · · · + a0 (mod 3).

Therefore, a positive integer is divisible by 3 if and only if the sum of its digits is
divisible by 3.

Proof. The proof is completely analogous to that of Proposition 0.2.6: just replace 9
with 3 throughout.

Example 0.2.8. Show that the equation

x2 + y 2 = 10z+2 − 1

has no non-negative integer solutions.

Solution. We will show that for all non-negative integers x, y, z, we have that

x2 + y 2 ̸≡ 10z+2 − 1 (mod 4).

This will immediately imply that the equation x2 +y 2 = 10z+2 −1 has no non-negative
integer solutions.
First, note that 100 ≡ 0 (mod 4). So, for a non-negative integer z, we have that

10z+2 − 1 = 100 · 10z − 1 ≡ 0 · 10z − 1 ≡ −1 (mod 4).

On the other hand:

ˆ 02 ≡ 0 (mod 4);

ˆ 14 ≡ 1 (mod 4);

ˆ 22 ≡ 0 (mod 4);

ˆ 32 ≡ 1 (mod 4).

Since every integer is congruent to one of 0, 1, 2, 3 modulo 4, it follows that the square
of any integer is congruent to either 0 or 1 modulo 4. It follows that the sum of two
squares is congruent to 0, 1, or 2 modulo 4, and none of these three numbers (0 , 1,
or 2) is congruent to −1 modulo 4. Consequently, for integers x and y, we have that

x2 + y 2 ̸≡ −1 (mod 4).

Thus, for non-negative integers x, y, z, we have that

x2 + y 2 ̸≡ 10z+2 − 1 (mod 4),

and we are done.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 15

0.2.2 Arithmetic in Zn . Fermat’s Little Theorem

Given n ∈ N and a ∈ Z, we set

[a]n := x ∈ Z | x ≡ a (mod n) ;

note that [a]n = a + kn | k ∈ Z .9 Note also that a ∈ [a]n , since a ≡ a (mod n).


We define 
Zn := [a]n | a ∈ Z .

Proposition 0.2.9. Let n ∈ N and a, b ∈ Z. Then:

(a) if a ≡ b (mod n), then [a]n = [b]n ;

(b) if a ̸≡ b (mod n), then [a]n ∩ [b]n = ∅.

Proof. This follows from the fact that, by Proposition 0.2.2, congruence modulo n is
an equivalence relation on Z. If you are not familiar with the theory of equivalence
relations, here is a detailed proof.
We first prove (a). Suppose that a ≡ b (mod n). It suffices to show that [a]n ⊆ [b]n
(the proof of the reverse inclusion is analogous). Fix x ∈ [a]n . Then x ≡ a (mod n).
Since a ≡ b (mod n), Proposition 0.2.2 guarantees that x ≡ b (mod n). Consequently,
x ∈ [b]n , and we deduce that [a]n ⊆ [b]n . This proves (a).
It remains to prove (b). We prove the contrapositive: if [a]n ∩ [b]n ̸= ∅, then a ≡ b
(mod n). So, assume that [a]n ∩ [b]n ̸= ∅, and fix some x ∈ [a]n ∩ [b]n . Since x ∈ [a]n ,
we have that x ≡ a (mod n), and since x ∈ [b]n , we have that x ≡ b (mod n). But
now by Proposition 0.2.2, we have that a ≡ b (mod n). This proves (b).

Note that for n ∈ N, every integer is congruent to exactly one of 0, . . . , n − 1

modulo n; by Proposition 0.2.9, it follows that for all x ∈ Z, the set [x]n is equal to
exactly one of the following: [0]n , . . . , [n − 1]n . This implies that, in fact:

Zn = [0]n , . . . , [n − 1]n .

Moreover, by Proposition 0.2.9, no two of 0, . . . , n − 1 are congruent to each other

modulo n, and consequently, [0]n , . . . , [n − 1]n are pairwise disjoint. We now deduce
that the sets [0]n , . . . , [n − 1]n form a “partition” of Z, that is:
9
For example:
ˆ [0]2 = {. . . , −4, −2, 0, 2, 4, . . . };
ˆ [1]2 = {. . . , −3, −1, 1, 3, 5, . . . };
ˆ [0]3 = {. . . , −6, −3, 0, 3, 6, . . . };
ˆ [1]3 = {. . . , −5, −2, 1, 4, 7, . . . };
ˆ [2]3 = {. . . , −4, −1, 2, 5, 8, . . . }.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 16

ˆ Z = [0]n ∪ · · · ∪ [n − 1]n , and

ˆ the sets [0]n , . . . , [n − 1]n are pairwise disjoint.10

If you are familiar with “equivalence relations,” then note that congruence modulo n
is an equivalence relation on Z (by Proposition 0.2.2), and the sets [0]n , . . . , [n − 1]n
are the associated equivalence classes.

Notation: When working in Zn , we often write simply 0, . . . , n − 1 instead of

[0]n , . . . , [n − 1]n , respectively. We may do this only if we have previously made it
clear that our numbers (which are technically sets of integers) are in Zn .

Example 0.2.10. For n = 2, [0]2 = {2t | t ∈ Z} and [1]2 = {1 + 2t | t ∈ Z},11 and

we have that Z2 = {[0]2 , [1]2 }. Typically, we write simply Z2 = {0, 1}, but technically,
0 stands for the set [0]2 , and 1 stands for [1]2 .

Recall that, by Proposition 0.2.3, for all n ∈ N and a, a′ , b, b′ ∈ Z, if a ≡ a′ (mod n)

and b ≡ b′ (mod n), then a + b ≡ a′ + b′ (mod n), a − b ≡ a′ − b′ (mod n), and
ab ≡ a′ b′ (mod n); equivalently, if [a]n = [a′ ]n and [b]n = [b′ ]n , then [a+b]n = [a′ +b′ ]n ,
[a − b]n = [a′ − b′ ]n , and [ab]n = [a′ b′ ]n . Thus, we may define addition, subtraction,
and multiplication in Zn as follows. For n ∈ N and a, b ∈ Z, we define

ˆ [a]n + [b]n = [a + b]n ;

ˆ [a]n − [b]n = [a − b]n ;

ˆ [a]n [b]n = [ab]n .

As we would expect (and as our next proposition summarizes), addition and mul-
tiplication in Zn are commutative and associative, and multiplication is distributive
over addition in Zn .

Proposition 0.2.11. Let n ∈ N. Then all the following hold:

(a) addition and multiplication are commutative in Zn , that is, for all a, b ∈ Zn , we
have that a + b = b + a and ab = ba;

(b) addition and multiplication are associative in Zn , that is, for all a, b, c ∈ Zn , we
have that (a + b) + c = a + (b + c) and (ab)c = a(bc);

(c) multiplication is distributive over addition in Zn , that is, for all a, b, c ∈ Zn , we

have that a(b + c) = ab + ac.
10
This means that no two of [0]n , . . . , [n − 1]n have an element in common. In other words, for all
distinct i, j ∈ {0, . . . , n − 1}, we have [i]n ∩ [j]n = ∅.
11
In other words, [0]2 is the set of all even numbers, and [1]2 is the set of all odd numbers.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 17

Proof. This essentially follows from the definition of Zn , from the fact that addi-
tion and multiplication are commutative and associative in Z, and from the fact
that multiplication is distributive over addition in Z. We give the details for the
commutativity of addition in Zn ; the rest is left as an easy exercise.
Fix a, b ∈ Zn . Then there exist a′ , b′ ∈ Z such that a = [a′ ]n and b = [b′ ]n . We
now have that
a + b = [a′ ]n + [b′ ]n

= [a′ + b′ ]n by the definition of addition in Zn

= [b′ + a′ ]n by the commutativity of addition in Z

= [b′ ]n + [a′ ]n by the definition of addition in Zn

= b + a.

This proves that addition is commutative in Zn .

Let us now take a look at the addition and multiplication tables for Zn , for a few
small values of n.

Example 0.2.12. Below are the addition and multiplication tables for Z2 .

+ [0]2 [1]2 · [0]2 [1]2

[0]2 [0]2 [1]2 [0]2 [0]2 [0]2
[1]2 [1]2 [0]2 [1]2 [0]2 [1]2

If we omit square brackets and subscripts (as we usually do), we obtain the addition
and multiplication tables for Z2 shown below.

+ 0 1 · 0 1
0 0 1 0 0 0
1 1 0 1 0 1

Example 0.2.13. Below are the addition and multiplication tables for Z3 .12

+ 0 1 2 · 0 1 2
0 0 1 2 0 0 0 0
1 1 2 0 1 0 1 2
2 2 0 1 2 0 2 1
12
Remember, in this context, 0 stands for [0]3 , 1 stands for [1]3 , and 2 stands for [2]3 .

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 18

Example 0.2.14. Below are the addition and multiplication tables for Z4 .13

+ 0 1 2 3 · 0 1 2 3
0 0 1 2 3 0 0 0 0 0
1 1 2 3 0 1 0 1 2 3
2 2 3 0 1 2 0 2 0 2
3 3 0 1 2 3 0 3 2 1

Example 0.2.15. Below are the addition and multiplication tables for Z5 .14

+ 0 1 2 3 4 · 0 1 2 3 4
0 0 1 2 3 4 0 0 0 0 0 0
1 1 2 3 4 0 1 0 1 2 3 4
2 2 3 4 0 1 2 0 2 4 1 3
3 3 4 0 1 2 3 0 3 1 4 2
4 4 0 1 2 3 4 0 4 3 2 1

Remark/Notation: Note that for all positive integers n, each number a in Zn has
a unique “additive inverse,” denoted by −a, i.e. the number (element of Zn ) that we
need to add to a in order to obtain 0 (here, 0 = [0]n ). When using square brackets
and subscripts, we do, of course, get −[a]n = [−a]n = [n − a]n for all positive integers
n and all integers a. However, we will usually work in Zn without such brackets.
For small values of n, we get the following:

ˆ in Z2 : −0 = 0, −1 = 1;

ˆ in Z3 : −0 = 0, −1 = 2, −2 = 1;

ˆ in Z4 : −0 = 0, −1 = 3, −2 = 2, −3 = 1;

ˆ in Z5 : −0 = 0, −1 = 4, −2 = 3, −3 = 2, −4 = 1.

Remark: Note that for n = 2, 3, 5, every non-zero member of Zn has a “multiplicative

inverse,” i.e. a number that we can multiply it by to get 1. However, for n = 4, this
is not the case. As Theorem 0.2.16 and Corollary 0.2.17 (see below) show, this is
not an accident.

Theorem 0.2.16. Let n ∈ N and a ∈ Z be relatively prime.15 Then there exists

some b ∈ Z such that ab ≡ 1 (mod n), and therefore, [a]n [b]n = [1]n .
13
Remember, in this context, 0 stands for [0]4 , 1 stands for [1]4 , 2 stands for [2]4 , and 3 stands for
[3]4 .
14
Remember, in this context, 0 stands for [0]5 , 1 stands for [1]5 , 2 stands for [2]5 , 3 stands for [3]5 ,
and 4 stands for [4]5 .
15
This means that the greatest common divisor of n and a, denoted by gcd(n, a), is 1. In other
words, the only positive integer that divides both n and a is 1.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 19

Proof. Let us show that no two of 0, a, 2a, . . . , (n − 1)a are congruent modulo n.16
Suppose otherwise, and fix distinct i, j ∈ {0, . . . , n − 1} such that ia ≡ ja (mod n).
Then (i − j)a ≡ 0 (mod n), that is, n|(i − j)a. Since n and a are relatively prime, it
follows that n|(i − j). But this is impossible because i, j ∈ {0, . . . , n − 1} and i ̸= j,
and so 0 < |i − j| < n. Thus, no two of 0, a, 2a, . . . , (n − 1)a are congruent modulo n.
We know that every integer is congruent modulo n to one of the following n
integers: 0, 1, 2, . . . , n − 1. We showed above that no two of the following n integers
are congruent to each other modulo n: 0, a, 2a, . . . , (n − 1)a. It follows that (exactly)
one of 0, a, 2a, . . . , (n − 1)a is congruent to 1 modulo n. In other words, for exactly
one value of b ∈ {0, 1, 2, . . . , n − 1}, we have that ba ≡ 1 (mod n). For this b,
we have that ab ≡ 1 (mod n), and therefore, [a]n [b]n = [1]n . This completes the
argument.

Corollary 0.2.17. Let p ∈ N be a prime number. Then:

(a) for all a ∈ Z such that a is not a multiple of p, there exists some b ∈ Z such that
ab ≡ 1 (mod p), and consequently, [a]p [b]p = [1]p ;

(b) for all a ∈ Zp \ {0}, there exists some b ∈ Zp \ {0} such that ab = 1.17

Proof. We first prove (a). Since p is a prime number, every integer that is not a
multiple of p is relatively prime to p; (a) now follows from Theorem 0.2.16.
Statement (b) immediately follows from (a). Indeed, fix a ∈ Zp \ {0}. Then
there exists an integer a′ ∈ {1, . . . , p − 1} such that a = [a′ ]p . By (a), there exists
an integer b′ such that a′ b′ ≡ 1 (mod p). We now set b := [b′ ]p , and we see
that ab = [a′ ]p [b′ ]p = [a′ b′ ]p = [1]p . Moreover, b ̸= 0, since (in Zp ) we have that
a · 0 = 0 ̸= 1 = ab. This proves (b).

Corollary 0.2.17(b) states that, for a prime number p, every number in Zp \ {0}
has a multiplicative inverse. Fermat’s Little Theorem (below) is a strengthening
of Corollary 0.2.17 in that it gives an actual formula for this multiplicative inverse.
However, before stating and proving Fermat’s Little Theorem, we need some notation
(which will be used in the proof). For non-negative integers n, we define n! (read “n
factorial”) recursively, as follows:

ˆ 0! := 1;

ˆ (n + 1)! := n! · (n + 1) for all non-negative integers n.

So, for a positive integer n, we have n! = 1 · 2 · · · · · n.

Fermat’s Little Theorem. If p ∈ N is a prime number, and a ∈ Z is not a multiple

of p, then ap−1 ≡ 1 (mod p).
16
Note that this implies that [a]n , [2a]n , . . . , [(n − 1)a]n are pairwise distinct.
17
Here, 0 = [0]p and 1 = [1]p .

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 20

Proof. Fix a prime number p ∈ N. Let a ∈ Z, and assume that a is not a multiple of
p. As in the proof of Theorem 0.2.16, no two of 0, a, 2a, . . . , (p − 1)a are congruent
modulo p.18 Since every integer is congruent to exactly one of 0, 1, . . . , p−1 modulo p,
it follows that there exists some rearrangement (i.e. permutation) r1 , . . . , rp−1 of the
sequence 1, . . . , p − 1 such that
ˆ a ≡ r1 (mod p);

ˆ 2a ≡ r2 (mod p);
..
.

ˆ (p − 1)a ≡ rp−1 (mod p).

It now follows that

a · 2a · · · · · (p − 1)a ≡ r1 r2 . . . rp−1 (mod p),

| {z } | {z }
=(p−1)!ap−1 =(p−1)!

and so (p − 1)!ap−1 ≡ (p − 1)! (mod p). But now

(ap−1 − 1)(p − 1)! ≡ 0 (mod p),

 
that is, p | (ap−1 − 1)(p − 1)! . Since p is prime, we see that p and (p − 1)! are
relatively prime. It follows that p | (ap−1 − 1), and consequently, ap−1 ≡ 1 (mod p),
which is what we needed to show.

For a positive integer n and for a ∈ Zn , we define powers of a recursively, as

follows:
ˆ a0 = 1 (where 1 := [1]n );

ˆ am+1 = am a for all non-negative integers m.

So, for a positive integer m, we have the familiar formula

am = a
| · ·{z
· · · a},
m

where it is understood that the multiplication on the right-hand-side is in Zn . With

this set-up, we can restate Fermat’s Little Theorem in two ways, as follows.
18
This is exactly the same as in the proof of Theorem 0.2.16, but for the sake of completeness,
here is the full proof. Suppose that some two of 0, a, . . . , (p − 1)a are congruent modulo p. Fix
distinct i, j ∈ {0, 1, . . . , p − 1} such that ia ≡ ja (mod p). Then (i − j)a ≡ 0 (mod p), that is,
p|(i − j)a. Since p is prime and does not divide a, we see that p|(i − j). But this is impossible
because i, j ∈ {0, . . . , p − 1} and i =
̸ j, and so 0 < |i − j| < p. Thus, no two of 0, a, 2a, . . . , (p − 1)a
are congruent modulo p.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 21

Fermat’s Little Theorem. If p ∈ N is a prime number, and a ∈ Z is not a multiple

p−1
of p, then [a]p = [1]p .

Fermat’s Little Theorem. If p ∈ N is a prime number and a ∈ Zp \ {0}, then

ap−1 = 1.

Multiplicative inverses. Suppose that p is a prime number and that a ∈ Zp \ {0}.

By Fermat’s Little Theorem, ap−2 is a “multiplicative inverse” of a, i.e. if we multiply
a by ap−2 (on either side), we obtain 1.19 Moreover, it is easy to see that ap−2 is the
only multiplicative inverse of a in Zp . Indeed, if b ∈ Zp satisfies ab = 1 (which is
equivalent to ba = 1, by Proposition 0.2.11), then by multiplying both sides by ap−2 ,
we obtain
p−2 p−2 · 1,
| {z · a} b = a
a
=ap−1 =1

and consequently, b = ap−2 . So, we can say that ap−2 is the multiplicative inverse of
a (denoted by a−1 ), and we write

a−1
|{z} = ap−2
multiplicative
inverse of a

Note, however, that for small values of the prime number p, it is easier to read off
the multiplicative inverses of non-zero numbers in Zp from the multiplication table
for Zp than it is to compute the (p − 2)-th powers of those numbers. Thus, by taking
a quick look at the multiplication tables for Z2 , Z3 , and Z5 , we get the following:

ˆ in Z2 : 1−1 = 1;

ˆ in Z3 : 1−1 = 1, 2−1 = 2;

ˆ in Z5 : 1−1 = 1, 2−1 = 3, 3−1 = 2, 4−1 = 4.

0.3 Complex numbers

Remark: This section is written for students who have already studied complex
numbers, but need a bit of a refresher. For this reason, most of the proofs are omitted,
and more importantly, there are relatively few examples. If you have never seen
complex numbers before, you might want to learn about them from a high school
algebra textbook. After that, you can read this section to check your understanding.
19
That is: a · ap−2 = ap−2 · a = 1.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 22

0.3.1 Complex numbers: definition, basic properties, and examples

To define complex numbers, we first introduce the imaginary unit number, denoted
by i, which satisfies
i2 = −1.
A complex number is any number of the form z = a + bi, where a and b are real
numbers; the real part of the complex number z is the real number a, and the
imaginary part of z is the real number b. The real and imaginary part of a complex
number z are denoted by Re(z) and Im(z), respectively. For example, we have the
following:
ˆ Re(2 + i) = 2 and Im(2 + i) = 1;
ˆ Re(−3i) = 0 and Im(−3i) = −3;
ˆ Re(7) = 7 and Im(7) = 0.
Note that real numbers are precisely those complex numbers whose imaginary part
is zero.
The set of all complex numbers is denoted by C. Complex numbers can be
visualized in the “complex plane.” This plane has two axes: the real axis (denoted
by Re) and the imaginary axis (denoted by Im). A complex number z = a + bi
(where a, b ∈ R) can be visualized in the complex plane as in the picture below.
Im

z = a + ib
b

a Re

Note that real numbers are precisely those complex numbers that lie on the real axis.
We define addition and multiplication of complex numbers as follows. Given
complex numbers z1 = a1 + b1 i and z2 = a2 + b2 i (where a1 , b1 , a2 , b2 ∈ R), we define
ˆ z1 + z2 = (a1 + b1 i) + (a2 + b2 i) := (a1 + a2 ) + (b1 + b2 )i;
ˆ z1 z2 = (a1 + b1 i)(a2 + b2 i) = (a1 a2 − b1 b2 ) + (a1 b2 + b1 a2 )i.
The definition of addition is natural. The idea behind the definition of multiplication
is that we are supposed to get something like this:
(∗)
(a1 + b1 i)(a2 + b2 i) = i2
a1 a2 + a1 b2 i + b1 a2 i + b1 b2 |{z}
=−1

= (a1 a2 − b1 b2 ) + (a1 b2 + b1 a2 )i,

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 23

where in (*), we assumed that the addition and multiplication of complex numbers
are commutative and associative, and that multiplication is distributive over addition
for complex numbers. However, this is not something that we can assume; it is
something we would need to prove, using the appropriate definitions.

Proposition 0.3.1. All the following hold:

(a) addition and multiplication in C are commutative, that is, for all z1 , z2 ∈ C, we
have that z1 + z2 = z2 + z1 and z1 z2 = z2 z1 ;

(b) addition and multiplication in C are associative, that is, for all z1 , z2 , z3 ∈ C, we
have that (z1 + z2 ) + z3 = z1 + (z2 + z3 ) and (z1 z2 )z3 = z1 (z2 z3 );

(c) multiplication is distributive over addition in C, that is, for all z1 , z2 , z3 ∈ C, we

have that z1 (z2 + z3 ) = z1 z2 + z1 z3 .

Proof. This essentially follows from the definition of addition and multiplication in C,
and from the fact that addition and multiplication are commutative and associative
in R, and multiplication is distributive over addition in R. Let us prove in detail
that addition in C is associative; the rest is left as a straightforward exercise.
Fix z1 , z2 , z3 ∈ C; we must show that (z1 + z2 ) + z3 = z1 + (z2 + z3 ). Set
z1 = a1 + b1 i, z2 = a2 + b2 i, and z3 = a3 + b3 i, where a1 , b1 , a2 , b2 , a3 , b3 ∈ R. We
now compute:
 
(z1 + z2 ) + z3 = (a1 + b1 i) + (a2 + b2 i) + (a3 + b3 i)

(∗)
 
= (a1 + a2 ) + (b1 + b2 )i + (a3 + b3 i)

(∗)
   
= (a1 + a2 ) + a3 + (b1 + b2 ) + b3 i

(∗∗)
   
= a1 + (a2 + a3 ) + b1 + (b2 + b3 ) i

(∗)
 
= (a1 + b1 i) + (a2 + a3 ) + (b2 + b3 )i

(∗)
 
= (a1 + b1 i) + (a2 + b2 i) + (a3 + b3 i)

= z1 + (z2 + z3 ),

where each instance of (*) follows from the definition of addition in C, and (**)
follows from the fact that addition in R is associative. This proves that addition in
C is associative.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 24

Powers of complex numbers are defined in the usual way. For a complex number
z, we define
ˆ z 0 := 1;
ˆ z m+1 := z m z for all non-negative integers m.
So, for all positive integers m, we have the familiar expression
z m = z| .{z
. . z} .
m

Next, for a complex number z = a + bi (where a, b ∈ R), we define the following:

ˆ the complex conjugate of z is z := a − bi;
√
ˆ the modulus (or absolute value) of z is |z| := a2 + b2 .
Geometrically, the complex conjugate of a complex number z is obtained by reflecting
z about the Re axis, as shown in the picture below. Obviously, z = z. Further,
note that z = z if and only if the complex number z is in fact a real number,
i.e. Im(z) = 0. We also note that the modulus of a complex number is the usual
Pythagorean distance between that complex number and the origin in the complex
plane. Note that the modulus of a complex number z is a non-negative real number,
and moreover, we have that |z| = 0 if and only if z = 0.
Im

z = a + ib
b

a Re
−b
z = a − ib

Proposition 0.3.2. For all complex numbers z = a + bi (with a, b ∈ R), we have

that
zz = a2 + b2 = |z|2 .
Proof. Fix a complex number z = a + bi, where a, b ∈ R. We then have that
zz = (a + bi)(a − bi)

= (a2 − b(−b)) + (a(−b) + ba)i

= a2 + b2

= |z|2 ,

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 25

which is what we needed to show.

Note that Proposition 0.3.2, in particular, establishes that multiplying a complex

number z by its conjugate produces a real number; that real number is zero if and
only if z = 0.
Let us now explain how division works in C. First of all, given a complex number
z = a + bi (with a, b ∈ R) and a real number r ̸= 0, we have
z a
r = r + rb i.

Now suppose that z1 and z2 = ̸ 0 are complex numbers. To compute zz12 , we need to
transform the denominator into a non-zero real number. We do this by multiplying
both the numerator and the denominator by z2 , at which point (by Proposition 0.3.2)
the denominator becomes |z2 |2 , which is a non-zero real number, and we can divide
as above. Let us take a look at an example.

Example 0.3.3. Compute the following quotients:

7−6i 1 2−3i 4−2i
(a) 3+2i ; (b) 2−i ; (c) 5 ; (d) 2−i .

Solution. (a) We multiply both the numerator and the denominator by 3 + 2i = 3−2i,
and we obtain
7−6i (7−6i)(3−2i) 9−32i 9 32
3+2i = (3+2i)(3−2i) = 9+4 = 13 − 13 i.

(b) We multiply both the numerator and the denominator by 2 − i = 2 + i, and

we obtain
1 2+i 2+i 2 1
2−i = (2−i)(2+i) = 4+1 = 5 + 5 i.

(c) The denominator is a real number, and so we have

2−3i 2
5 = 5 − 35 i.

(d) We could multiply both the numerator and the denominator by 2 − i = 2 + i.

However, in this particular case, it is easier to compute as follows:

4−2i 2(2−i) (∗)

2−i = 2−i = 2,

where (*) was obtained by canceling out the common factor 2 − i in the numerator
and the denominator.

Finally, we give some properties of the complex conjugate and the modulus of a
complex number (see Propositions 0.3.4 and 0.3.5 below).

Proposition 0.3.4. For all z1 , z2 ∈ C, the following hold:

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 26

(a) z1 + z2 = z1 + z2 ;

(b) z1 − z2 = z1 − z2 ;

(c) z1 z2 = z1 z2 ;

(d) if z2 ̸= 0, then z1 /z2 = z1 /z2 .

Moreover, for all z ∈ C and non-negative integers m, we have that

(e) z m = (z)m .

Proof. We prove (c). The rest is left as an exercise. Fix a complex numbers
z1 = a1 + b1 i and z2 = a2 + b2 i (with a1 , b1 , a2 , b2 ∈ R). We then have that

z1 z2 = (a1 + b1 i)(a2 + b2 i)

= (a1 a2 − b1 b2 ) + (a1 b2 + b1 a2 )i

= (a1 a2 − b1 b2 ) − (a1 b2 + b1 a2 )i.

On the other hand,

  
z1 z2 = a1 + b1 i a2 + b2 i

= (a1 − b1 i)(a2 − b2 i)

= (a1 a2 − b1 b2 ) + (−a1 b2 − b1 a2 )i

= (a1 a2 − b1 b2 ) − (a1 b2 + b1 a2 )i.

So, z1 z2 = z1 z2 . This proves (c).

Proposition 0.3.5. For all z1 , z2 ∈ C, the following hold:

(a) |z1 z2 | = |z1 ||z2 |;

(b) if z2 ̸= 0, then |z1 /z2 | = |z1 |/|z2 |.

Moreover, for all z ∈ C, the following hold:

(c) | − z| = |z|;

(d) for all non-negative integers m, we have |z m | = |z|m .

Proof. We prove (a); the rest is left as an exercise. Fix complex numbers z1 = a1 +b1 i
and z2 = a2 + b2 i (where a1 , b1 , a2 , b2 ∈ R). We then have that

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 27

|z1 z2 | = |(a1 + b1 i)(a2 + b2 i)|

= |(a1 a2 − b1 b2 ) + (a1 b2 + b1 a2 )i|

p
= (a1 a2 − b1 b2 )2 + (a1 b2 + b1 a2 )2
p
= a21 a22 + 2a1 a2 b1 b2 + b21 b22 + a21 b22 − 2a1 b2 b1 a2 + b21 a22
p
= a21 a22 + b21 b22 + a21 b22 + b21 a22 .

On the other hand, we have that

|z1 ||z2 | = |a1 + b1 i||a2 + b2 i|

p p
= a21 + b21 a22 + b22
p
= a21 a22 + a21 b22 + b21 a22 + b21 b22
p
= a21 a22 + b21 b22 + a21 b22 + b21 a22 .

It follows that |z1 z2 | = |z1 ||z2 |. This proves (a).

0.3.2 The Fundamental Theorem of Algebra

A root of a polynomial p(x) with complex coefficients is a complex number c such
that p(c) = 0. For example, 1 + i is a root of the polynomial p(x) = x2 − 2x + 2
because
p(1 + i) = (1 + i)2 − 2(1 + i) + 2 = 0.
In the particular case of p(x) = x2 − 2x + 2, the roots could have been found via the
familiar quadratic equation. There exist formulas for finding the complex roots of all
third and fourth degree polynomials with complex coefficients, but no such formula
exists for polynomials of degree five or more (although in some special cases, we may
be able to use various tricks to find the roots of these higher-degree polynomials).
Nevertheless, we do have the following existence result. (A constant polynomial is
a polynomial of the form p(x) = c, where c is a fixed constant/number.)

The Fundamental Theorem of Algebra. Any non-constant polynomial with

complex coefficients has a complex root.

Remark: The Fundamental Theorem of Algebra is an existence result in the sense

that it guarantees the existence of a complex root for any non-constant polynomial
with complex coefficients, even though we might not be able to actually compute
this root. Of course, every real number is complex. So, the Fundamental Theorem

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 28

of Algebra, in particular, implies that every non-constant polynomial with real

coefficients has a complex root (which may or may not be a real number). For
instance, the polynomial p(x) = x2 + 1 is a non-constant polynomial with real (in
fact, rational) coefficients, but it has no real roots. It does, of course, have two
complex roots, namely i and −i.
We will not prove the Fundamental Theorem of Algebra here. There are no
known elementary proofs of this theorem: all the known proofs of the Fundamental
Theorem of Algebra rely on advanced mathematics, such as complex analysis or
topology.
The Fundamental Theorem of Algebra implies that any polynomial p(x) with
complex coefficients and of degree n ≥ 1 can be factored into n linear factors. More
precisely, for such a polynomial p(x), there exist complex numbers a, α1 , . . . , αℓ such
that a = ̸ 0 and such that α1 , . . . , αℓ are pairwise distinct, and positive integers
n1 , . . . , nℓ satisfying n1 + · · · + nℓ = n, such that

p(x) = a(x − α1 )n1 . . . (x − αℓ )nℓ ,

and moreover, this factorization into linear factors is unique up a permutation of

the αi ’s and the corresponding ni ’s. (We omit the proof.) Here, a is the leading
coefficient of p(x), i.e. the coefficient in front of xn . Complex numbers α1 , . . . , αℓ are
the roots of p(x) with multiplicities n1 , . . . , nℓ , respectively. If we think of each αi as
being a root “ni times” (due to its multiplicity), then we see that the n-th degree
polynomial p(x) has exactly n complex roots. This is often summarized as follows:
“every n-th degree polynomial (with n ≥ 1) with complex coefficients has exactly n
complex roots, when multiplicities are taken into account.”
As we already mentioned, there are formulas that allow us to compute the roots
of polynomials with complex coefficients of degree at most four. However, no such
formulas exist for polynomials (with complex coefficients) of degree n ≥ 5: we know
that all such polynomials have n complex roots (when multiplicities are taken into
account), but in general, there is no formula for computing these roots. In fact, not
only is no such formula known, but using Galois theory, one can show that no such
formula can exist for polynomials of degree at least five. (Once again, we may be able
to use various tricks to compute the roots of some special high-degree polynomials.
However, none of these tricks will work in the general case.)

Complex roots of polynomials with real coefficients. Every real number

can be seen as a complex number. So, it makes sense to speak of complex roots of
polynomials with real coefficients, as in the case of the theorem below. Recall that,
geometrically, the complex conjugate of a complex number z is obtained by reflecting
z about the Re axis in the complex plane. Moreover, for a complex number z, we
have that z = z if and only if z is a real number.

Irena Penev
Linear Algebra 1 & 2
Chapter 0. Preliminaries 29

Theorem 0.3.6. Let p(x) be any polynomial with real coefficients, and let z ∈ C.
Then z is a root of p(x) if and only if its complex conjugate z is a root of p(x).

Proof. Set p(x) = an xn + · · · + a1 x + a0 , where a0 , a1 , . . . , an ∈ R. Then we have the

following sequence of equivalences:

p(z) = 0 ⇐⇒ p(z) = 0

⇐⇒ an z n + · · · + a1 z + a0 = 0

(∗)
⇐⇒ an (z)n + · · · + a1 (z) + a0 = 0

(∗∗)
⇐⇒ an (z)n + · · · + a1 z + a0 = 0

⇐⇒ p(z) = 0,

where (*) follows from Proposition 0.3.4, and (**) follows from the fact that
a0 , a1 , . . . , an and 0 are real numbers.

Remark: Note that Theorem 0.3.6 implies that the complex roots of a non-constant
polynomial are symmetric about the Re axis in the complex plane. Some (or perhaps
all) of those roots may lie on the Re axis, i.e. they may be real numbers. So, if we
plot the roots of a non-constant polynomial with real coefficients as points in the
complex plane, we may obtain a picture that looks something like this (see below;
complex conjugate pairs are colored with the same color).
Im

Re

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 30

Chapter 1

Systems of linear equations.

Vectors and matrices

1.1 An informal introduction to fields

In this chapter, we will assume that F is a fixed “field.” A formal definition of a field
will be given in chapter 2 (see section 2.4). For now, we note that all the following
are fields:

ˆ the field Q of rational numbers;

ˆ the field R of real numbers;

ˆ the field C of complex numbers;

ˆ the field Zp , where p is a prime number.1

Each field is equipped with two operations: addition and multiplication. These two
operations are commutative and associative, and multiplication is distributive over
addition.2 It is also relevant that every field has an “additive identity” 0 and a
“multiplicative identity” 1, which satisfy a + 0 = 0 + a = a and a · 1 = 1 · a = a for
all elements a of the field. Every element a of a field has a corresponding “additive
inverse,” denoted by −a, which is a number that we can add to a in order to obtain
0. For example:
√ √ √ √
ˆ the additive inverse of 17 in R is − 17, since 17 + (− 17) = 0 in R.
1
If n ∈ N is not prime, then Zn is not a field.
2
This means that for all elements a, b, c of the field, the following are satisfied:
ˆ a + b = b + a and ab = ba (commutativity of addition and multiplication);
ˆ (a + b) + c = a + (b + c) and (ab)c = a(bc) (associativity of addition and multiplication);
ˆ a(b + c) = ab + ac (distributive property of multiplicaiton over addition).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 31

ˆ the additive inverse of 2 − i in C is −2 + i, since (2 − i) + (−2 + i) = 0 in C;

ˆ the additive inverse of 3 in Z5 is 2 (and we write −3 = 2), since 3 + 2 = 0 in Z5 ;
ˆ the additive inverse of 4 in Z5 is 1 (and we write −4 = 1), since 4 + 1 = 0 in Z5 ;
ˆ the additive inverse of 2 in Z3 is 1 (and we write −2 = 1), since 2 + 1 = 0 in Z3 .
Equally importantly, every non-zero element a of a field has a “multiplicative
inverse,” denoted by a−1 , which is a number we can multiply a by in order to obtain
1. For example:
√ √
ˆ the multiplicative inverse of 17 in R is √117 , because 17 · √117 = 1 in R;

ˆ the multiplicative inverse of 2 − i is 2

5 + 51 i, because (2 − i)( 25 + 15 i) = 1 in C;
– this is obtained by computing:
1 2+i 2+i 2
2−i = (2−i)(2+i) = 5 = 5 + 15 i;

ˆ the multiplicative inverse of 3 in Z5 is 2 (and we write 3−1 = 2), since 3 · 2 = 1

in Z5 ;
ˆ the multiplicative inverse of 4 in Z5 is 4 (and we write 4−1 = 4), since 4 · 4 = 1
in Z5 ;
ˆ the multiplicative inverse of 2 in Z3 is 2 (and we write 2−1 = 2), since 2 · 2 = 1
in Z3 .

Remark: When working over Zp (for a prime number p), it is a good idea to first
write out the addition and multiplication tables for Zp , because this allows us to
easily identify additive and multiplicative inverses: for a given a ∈ Zp , we simply
read off from the tables what number we need to add to a to get zero, and (assuming
a ̸= 0) what number we need to multiply it by to get 1.

Warning: The following are not fields: N, Z, Zn (where n is a positive integer that
is not prime).

For the remainder of this chapter, you may assume that the field F in
question is one of the following: Q, R, C, or Zp (where p is a prime
number). However, everything that we prove in this chapter does in
fact hold for general fields F, not just the ones listed above.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 32

1.2 Vectors and matrices

1.2.1 Matrices
A matrix is a rectangular array of numbers (typically, elements of some field). An
n × m matrix (read “n by m matrix”) is a matrix with n rows and m columns.
Consider, for example, the following matrices:
   
  3 1 3 3 2
1 0 2
A = , B =  2 5 , C =  1 −1 −5  .
1 3 4
1 0 −2 2 3

A is a 2 × 3 matrix, B is a 3 × 2 matrix, and C is a 3 × 3 matrix. A square matrix

is one that has the same number of rows and columns. So, C is a square matrix,
but A and B are not square matrices. The main diagonal of a square matrix is the
diagonal between the upper left corner and the bottom right corner. For example,
the main diagonal of the square matrix C is colored red (below).
 
3 3 2
C =  1 −1 −5 
−2 2 3

The rows of a matrix are enumerated from top to bottom, whereas the columns
are enumerated from left to right. The i, j-th entry of a matrix is the entry that
appears in the i-th row (from the top) and j-th column (from the left) in the matrix.
A matrix A can be specified as follows:
 
A = ai,j n×m .

This notation indicates that the matrix A is of size n × m (i.e. has n rows and m
columns), and the i, j-th
 entry (i.e. the entry in the i-th row and j-th column) is
ai,j . So, if A = ai,j n×m , then we have that
 
a1,1 a1,2 . . . a1,m
 a2,1 a2,2 . . . a2,m 
A =  .
 
.. .. .. ..
 . . . . 
an,1 an,2 . . . an,m

A zero matrix is a matrix all of whose entries are 0 (where the 0 comes from the field
that we are working with). The zero matrix of size n × m is denoted by On×m . For
example,  
0 0 0 0
O2×4 = .
0 0 0 0
A non-zero matrix is a matrix that has at least one non-zero entry.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 33

Notation: If F is a field, then the set of all n × m matrices with entries in F is

denoted by Fn×m .
Terminology: A real matrix is a matrix whose entries are real numbers, whereas a
complex matrix is a matrix whose entries are complex numbers.

1.2.2 Column vectors (or simply vectors)

A column vector, or simply vector, is a matrix with just one column. Here are some
examples of vectors (in this case, vector entries are real numbers):
 
  1
  −13  2 
1  0   
a= , b= , c= 0 
.
−3  0  
 −1 
π
1
Vectors are typically denoted by bold letters (e.g. a,  u, x) or by letters with an arrow
0
 .. 
on top (e.g. ⃗a, ⃗u, ⃗x). The zero vector (i.e. vector  . ) is denoted by 0 or ⃗0. (The
0
number of entries in a zero vector should either be made explicit or be clear from
context.) A non-zero vector is a vector that has at least one non-zero entry.

Notation: If F is a field, then the set of all (column) vectors with n entries, all of
them in F, is denoted by Fn . (Thus, Fn = Fn×1 .)
 
2 3 a1
Geometric interpretation of vectors in R and R . A vector a = in R2
a2
can be represented in the two-dimensional Euclidean space either as a point (see the
picture below, on the left) or as a line segment with an arrow starting at the origin
(see the picture below, on the right).
x2 x2

a a
a2 a2

a1 x1 a1 x1
 
0
The zero vector 0 = is simply the origin.
0
 
a1
A vector a =  a2  in R3 has a similar geometric interpretation in the three-
a3
dimensional Euclidean space (see the picture below).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 34

x3 x3
a3 a3
a
a

a2 x2 a2 x2
a1 a1
x1 x1

0
Once again, the zero vector 0 =  0  is simply the origin.
0
Vectors in Rn for n ≥ 4 are higher-dimensional analogs of vectors in R2 and R3 .

1.2.3 Row vectors

A row vector is a matrix with only one row. For example, the following are row
vectors (in this case, vector entries are real numbers):

ˆ a = 1 −3 ;
 

ˆ b = −13 0 0 π ;
 

ˆ c = 1 2 0 −1 1 .
 

In these lecture notes, row vectors will appear less commonly than column vectors.
The set of all row vectors with n entries, all of them in some field F, is denoted by
F1×n (i.e. exactly the same way as the set of all 1 × n matrices with entries in F).

1.2.4 Specifying matrices in terms of their rows or columns

The columns of a matrix can be seen as (column) vectors, and matrices can be
specified in terms of their columns. When we specify a matrix A ∈ Fn×m (where F
is some field) in the form
 
A = a1 . . . am ,

we mean that a1 , . . . , am are the columns of A (appearing in that order from left to
right in the matrix A), and moreover, a n
1 , . . ., am arevectors
 in F . For example,

  1 1 3
if A = a1 a2 a3 , where a1 = , a2 = , and a3 = , then
2 0 4
 
1 1 3
A= .
2 0 4
Similarly, the rows of a matrix can be seen as row vectors, and matrices can be
specified in terms of their rows. When we specify a matrix A ∈ Fn×m (where F is

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 35

some field) in the form 

r1
A =  ...  ,
 

rn
we mean that r1 , . . . , rn are the rows of A (appearing in that order from top to
bottom in the matrix A), 1×m .
 and moreover, r1 , . . . , rn are row vectors in F
r1    
For example, if A = , where r1 = 1 2 1 3 and r2 = 3 4 4 3 ,
r
  2
1 2 1 3
then A = .
3 4 4 3

1.3 Systems of linear equations

A linear equation in the variables x1 , . . . , xm is an equation that can be written in
the form
a1 x1 + · · · + am xm = b,
where b and the coefficients a1 , . . . , an are elements of some field F. For example, the
equation
x1 − 3(x2 − x1 ) = 7x3 − 4,
with coefficients understood to be in R, is a linear equation because it can be
algebraically rearranged to have the following form:
4x1 − 3x2 − 7x3 = −4,
which is obviously a linear equation. On the other hand, equations
√
x31 + x2 = 17 and x1 − x2 = 5
√
are not linear because of x31 and x2 .
A system of linear equations, or a linear system, is a collection of one or more
linear equations involving the same variables, say x1 , . . . , xm (and with coefficients
from the same field). For example, the following is a linear system (here, the
coefficients are assumed to be in R):
√
2x1 + 7x2 − πx4 = − 3
−3x2 + 17x3 − 3x4 = 2
11
x1 + x2 − 2x3 + 7x4 = 2

Remark: Typographically, we normally arrange equations in our system so that

the terms involving the same variable are below each other (i.e. visually in the same
column).
A solution of a linear system in variables x1 , . . . , xm is a list s1 , . . . , sm of numbers
(from the same field as the coefficients of the system) such that each equation becomes
a true statement when s1 , . . . , sm are substituted for x1 , . . . , xm , respectively.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 36

Example 1.3.1. Consider the linear system

x1 + 2x2 − x3 = 9
2x2 + 3x3 = 16
x1 + x2 − x3 = 4

with coefficients in R. Then

x1 = 1
x2 = 5
x3 = 2
is a solution of the system above.

Example 1.3.2. Consider the linear system

x1 + x2 = 0
2x1 + x2 = 1

with coefficients in Z3 . Then

x1 = 1
x2 = 2
is a solution of the system above.

The set of solutions or solution set of a linear system is the set of all solutions of
that system. Our main goal in this section is to describe a procedure for finding the
solution set of any linear system.
A linear system may have no solutions, may have a unique solution (i.e. exactly
one solution), or may have more than one solution. A system that has at least one
solution is called consistent; a system that has no solutions is said to be inconsistent.

Linear systems with real coefficients. Consider the following system of two
linear equations in two variables, with coefficients in R.

a1,1 x1 + a1,2 x2 = b1
a2,1 x1 + a2,2 x2 = b2

Let us assume that at least one of the coefficients a1,1 , a1,2 is non-zero, and similarly,
that at least one of the coefficients a2,1 , a2,2 is non-zero. Then each of the two
equations above defines a line in the plane. There are three possibilities for these
two lines:

1. the two lines may intersect in one point (in this case, the system has a unique
solution, and in particular, it is consistent);

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 37

x2 a1,1x1 + a1,2x2 = b1

x1
a2,1x1 + a2,2x2 = b2

2. the two lines may be distinct, parallel lines (in this case, the system has no
solutions, i.e. it is inconsistent);
x2 a1,1x1 + a1,2x2 = b1

a2,1x1 + a2,2x2 = b2

x1

3. the two lines may be identical (in this case, the system has infinitely many
solutions, and in particular, the system is consistent).3
x2
a1,1x1 + a1,2x2 = b1
a2,1x1 + a2,2x2 = b2

x1

On the other hand, suppose that we have a system of two linear equations in
three variables (with coefficients in R).
a1,1 x1 + a1,2 x2 + a1,3 x3 = b1
a2,1 x1 + a2,2 x2 + a2,3 x3 = b2
3
Note that the two lines may be identical even if the two equations are different. For instance,
x1 + x2 = 1 and 2x1 + 2x2 = 2 define the same line.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 38

Similarly to the above, let us assume that at least one of the coefficients a1,1 , a1,2 , a1,3
is non-zero, and that at least one of the coefficients a2,1 , a2,2 , a2,3 is non-zero. Then
each of the two equations above defines a plane in the three-dimensional Euclidean
space. Those two planes may intersect in a line (in which case the system has
infinitely many solutions, and in particular, the system is consistent); or the two
planes may be distinct and parallel (in which case, the system has no solutions, i.e.
it is inconsistent); or the two planes may be identical (in which case the system has
infinitely many solutions, and in particular, the system is consistent).

1.3.1 The augmented matrix and the coefficient matrix of a linear

system
Suppose we are given a system of n linear equations in m variables, as follows.
a1,1 x1 + a1,2 x2 + . . . + a1,m xm = b1
a2,1 x1 + a2,2 x2 + . . . + a2,m xm = b2
..
.
an,1 x1 + an,2 x2 + . . . + an,m xm = bn
The coefficient matrix of this system is the n × m matrix
 
a1,1 a1,2 . . . a1,m
 a2,1 a2,2 . . . a2,m 
A =  . ..  .
 
.. ..
 .. . . . 
an,1 an,2 . . . an,m
To fully capture our linear system, we need a bigger matrix, called the “augmented
matrix” of the linear system. We start with the coefficient matrix A, and then we
form the vector whose entries are the numbers to the right of the equality sign:
 
b1
 b2 
b =  . .
 
.
 . 
bn
The augmented matrix of our linear system is the n × (m + 1) matrix
 
a1,1 a1,2 . . . a1,m b1
   a2,1 a2,2 . . . a2,m b2 
A b =  . ..  .
 
.. .. ..
 .. . . . . 
an,1 an,2 . . . an,m bn
Obviously, a linear system is fully “encoded” by its augmented matrix. The vertical
dotted line is optional, but visually, it helps separate the coefficients to the left of
the equality sign (those that form the coefficient matrix) from the numbers to the
right of the equality sign.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 39

Example 1.3.3. Find the coefficient matrix and the augmented matrix of the linear
system below (with coefficients understood to be in R).

3x1 + 2x2 + 5x3 = 7

3x2 − x3 = 0
 
3 2 5
Solution. The coefficient matrix of the linear system is , whereas the
0 3 −1
 
3 2 5 7
augmented matrix is .
0 3 −1 0
Example 1.3.4. Find the coefficient matrix and the augmented matrix of the linear
system below (with coefficients understood to be in Z3 ).

2x1 + x3 + 2 = x2
x2 + x3 = 2x1
Solution. We first algebraically rearrange the system above to get it into standard
form (below).4
2x1 + 2x2 + x3 = 1
x1 + x2 + x3 = 0
We can now easily read off the
 two matrices that we need. The coefficient
 matrix of the
2 2 1 2 2 1 1
linear system is , whereas the augmented matrix is .
1 1 1 1 1 1 0

1.3.2 Elementary row operations

Two linear systems (with the same variables) are equivalent if they have exactly the
same solution set. Now, suppose we are given a system of linear equations such as
the one below (with coefficients understood to be in some field F).

a1,1 x1 + a1,2 x2 + . . . + a1,m xm = b1

a2,1 x1 + a2,2 x2 + . . . + a2,m xm = b2
..
.
an,1 x1 + an,2 x2 + . . . + an,m xm = bn
We would like to manipulate this system in a way that allows us to “read off” the
solution set of the system. There are three basic ways that we can manipulate the
4
Remember: We are working in Z3 ! We manipulated the first equation as follows.
1. First, we added 2x2 to both sides of the equation to obtain 2x1 + 2x2 + x3 + 2 = 0. (We used
the fact that x2 + 2x2 = (1 + 2)x2 = 0x2 = 0.)
2. Then, we added 1 to both sides of the equation to obtain 2x1 + 2x2 + x3 = 1. (We used the
fact that 2 + 1 = 0.)
We manipulated the second equation by adding x1 to both sides to obtain x1 + x2 + x3 = 0. (We
used the fact that 2x1 + x2 = (2 + 1)x2 = 0x2 = 0.)

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 40

system in a way that does not change the solution set (i.e. in a way that produces
an equivalent linear system). We list these three operations/manipulations below
(the scalars that we mention always belong to the same field as the coefficients of
the linear system in question), and we illustrate each operation with an example (in
each example, the coefficients are understood to be in R.)

1. Swap (interchange) two equations.

ˆ For example, by swapping the first and third equation of the linear system
on the left, we obtain the linear system on the right.
x1 + 3x2 − 2x3 = −1 x1 + x2 + 2x3 = 2
1 1
2 x1 + 2x3 = 0 −→ x
2 1 + 2x3 = 0
x1 + x2 + 2x3 = 2 x1 + 3x2 − 2x3 = −1

It is obvious that this operation does not alter the solution set.

2. Multiply one equation by a non-zero scalar.

ˆ For example, by multiplying the second equation of the linear system on
the left by 2, we obtain the linear system on the right.
x1 + x2 + 2x3 = 2 x1 + x2 + 2x3 = 2
1
2 x1 + 2x3 = 0 −→ x1 + 4x3 = 0
x1 + 3x2 − 2x3 = −1 x1 + 3x2 − 2x3 = −1

Let us explain why this does not alter the solution set. Suppose we have multi-
plied the i-th equation of our linear system by some scalar α ̸= 0. Obviously,
all solutions of the old system are still solutions of the new system. On the
other hand, by multiplying the i-th equation of the new system by α−1 (the
multiplicative inverse of α), we get the old system back.5 So, any solution of
the new system is a solution of the old system as well.
Warning: Do not multiply an equation by 0, since that “kills” the equation!

3. Add a scalar multiple of one equation to another equation.

ˆ For example, by adding (−1) times the second equation to the third
equation of the linear system on the left, we obtain the linear system on
the right.
x1 + x2 + 2x3 = 2 x1 + x2 + 2x3 = 2
x1 + 4x3 = 0 −→ x1 + 4x3 = 0
x1 + 3x2 − 2x3 = −1 3x2 − 6x3 = −1
5
In the example above, we would multiply the second equation of the linear system on the right
1
by 2
in order to obtain the linear system on the left.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 41

Let us explain why this does not alter the solution set. Suppose we have added
α times the i-th equation to the j-th equation (where i ̸= j). Obviously, any
solution of the old system is also a solution of the new system. On the other
hand, if we start with the new system, then add −α times the i-th equation to
the j-th equation, we get the old system back.6 So, any solution of the new
system is a solution of the old system as well.
Instead of manipulating systems linear systems in this way, we can manipulate
their augmented matrices. There are three types of “elementary row operations” on
matrices (with entries understood to be in some field F), which we list below. (The
scalars that we mention always belong to the same field as the entries of the matrix
in question.) We illustrate each type of elementary row operation with an example
(in our examples, the matrix entries are assumed to be in R).
1. Swap (interchange) two rows.
ˆ We denote the operation of swapping rows i and j (i ̸= j) by “Ri ↔ Rj .”
ˆ For example, we can swap the first and third row of the matrix on the
left to obtain the matrix on the right.
   
1 3 −2 −1 1 1 2 2
R ↔R
 1 0 2 0  1
∼ 3  1 0 2 0 
2 2
1 1 2 2 1 3 −2 −1
2. Multiply one row by a non-zero scalar.
ˆ We denote the operation of multiplying row i by a scalar α = ̸ 0 by
“Ri → αRi .”
ˆ For instance, we can multiply the second row of the matrix on the left by
2 to obtain the matrix on the right.
   
1 1 2 2 1 1 2 2
R2 →2R2
 1 0 2 0  ∼  1 0 4 0 
2
1 3 −2 −1 1 3 −2 −1
3. Add a scalar multiple of one row to another row.
ˆ We denote the operation of adding scalar α times row i to row j (i ̸= j)
by “Rj → Rj + αRi .”
ˆ For example, we can add (−1) times the second row to the third row of
the matrix on the left to obtain the matrix on the right.
   
1 1 2 2 1 1 2 2
R3 →R3 +(−1)R2
 1 0 4 0  ∼  1 0 4 0 
1 3 −2 −1 0 3 −6 −1
6
In the example above, we would add 1 times the second equation to the third equation of the
system on the right to obtain the system on the left.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 42

Note: Instead of “R3 → R3 + (−1)R2 ,” we could also have written (and

we typically do write) just “R3 → R3 − R2 .”

Importantly, all elementary row operations are reversible:

1. we can undo (reverse) the operation of swapping two rows (“Ri ↔ Rj ”) by

applying the same operation again;

2. we can undo (reverse) the operation of multiplying row i by a scalar α ̸= 0

(“Ri → αRi ”) by multiplying row i by α−1 (“Ri → α−1 Ri ”);

3. we can undo (reverse) the operation of adding scalar α times row i to another
row j (“Rj → Rj +αRi ”) by adding −α times row i to row j (“Rj → Rj −αRi ”).

Remark: Solving systems of linear equations is our primary motivation for introduc-
ing elementary row operations. However, we can, in principle, perform elementary
row operations on any matrix (with entries in some field), even one that was not
obtained as an augmented matrix of a linear system. We will, indeed, do this at
various points in these lecture notes. However, for now, it is useful to think of
elementary row operations on matrices as a more compact way of performing the
corresponding operations on linear systems.

Terminology/Notation: If one matrix can be obtained from another via some

sequence of elementary row operations, then the two matrices are said to be row
equivalent. If matrices A and B are row equivalent, then we write A ∼ B. Note that
any two row equivalent matrices are of the same size (i.e. have the same number of
rows and the same number of columns), and their entries belong to the same field.

Remark: Clearly, if two matrices with at least two columns (and with entries in
some field F) are row equivalent, then they encode equivalent linear systems (as
augmented matrices).7

The following proposition states that, for a field F, row equivalence is an equiva-
lence relation on the set Fn×m .

Proposition 1.3.5. Let F be a field. Then all the following hold:

(a) for all A ∈ Fn×m , A ∼ A;

(b) for all A, B ∈ Fn×m , if A ∼ B, then B ∼ A;

(c) for all A, B, C ∈ Fn×m , if A ∼ B and B ∼ C, then A ∼ C.

7
A matrix that only has one column is not the augmented matrix of any linear system. That
said, according to our definition, two one-column matrices (i.e. two column vectors) can be row
equivalent.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 43

Proof. (a) Fix A ∈ Fn×m . By, for example, multiplying the first row of A by 1 (i.e.
by applying the elementary row operation “R1 → 1R1 ”), we obtain the original
matrix A; so, A ∼ A.
(b) Fix A, B ∈ Fn×m , and assume that A ∼ B. Then by applying some sequence
R1 , . . . , Rk of elementary row operations to A, we obtain the matrix B. But we know
that elementary row operations are reversible! For each i ∈ {1, . . . , k}, let Ri′ be the
elementary row operation that reverses (undoes) the elementary row operation Ri .
If we apply the sequence Rk′ , . . . , R1′ of elementary row operations to B, we obtain
the matrix A. So, B ∼ A.
(c) Fix A, B, C ∈ Fn×m , and assume that A ∼ B and B ∼ C. Since A ∼ B, we
know that B can be obtained by applying some sequence R1 , . . . , Rk of elementary
row operations to A. Similarly, since B ∼ C, we know that C can be obtained by
applying some sequence Rk+1 , . . . , Rk+ℓ of elementary row operations to B. But now
if we apply the sequence R1 , . . . , Rk , Rk+1 , . . . , Rk+ℓ to A, we get C.

1.3.3 Row reduction

A zero row of a matrix is a row in which all entries are zero, and a non-zero row is a
row that has at least one non-zero entry. (Zero and non-zero columns are defined
analogously.) The leading entry of a non-zero row is the leftmost non-zero entry of
that row.
A matrix is in row echelon form (or simply echelon form), abbreviated REF, if it
satisfies the following two conditions:

1. all non-zero rows are above any zero rows;

2. each leading entry of a non-zero row (other than the top row) is in a column
strictly to the right of the column containing the leading entry of the row right
above.8

If, in addition, the matrix satisfies the following two conditions, then it is in reduced
row echelon form (or simply reduced echelon form), abbreviated RREF:

3. the leading entry in each non-zero row is 1;

4. each leading 1 is the only non-zero entry in its column.

If a matrix is in row echelon form (resp. reduced row echelon form), then we also say
that the matrix is a row echelon matrix (resp. reduced row echelon matrix).
8
So, all entries in a column below a leading entry of a row are zeros.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 44

Schematically, a matrix in row echelon form looks like this (here, ■’s represent
non-zero numbers, and ∗’s represent arbitrary numbers):

0 ■ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗
 
 0 0 0 ■ ∗ ∗ ∗ ∗ ∗ ∗ 
 
 0 0 0 0 ■ ∗ ∗ ∗ ∗ ∗ 
 0 0 0 0 0 0 0 ■ ∗ ∗ .
 
 
 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0

On the other hand, a matrix in reduced row echelon form schematically looks like
this (∗’s represent arbitrary numbers):

0 1 ∗ 0 0 ∗ ∗ ∗ ∗
 
0
 0 0 0 1 0
 ∗ ∗ 0 ∗ ∗ 

 0 0 0 0 1
 ∗ ∗ 0 ∗ ∗ 
.
 0 0 0 0 0
 0 0 1 ∗ ∗ 

 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0

A pivot position of a matrix in row echelon form is the position of a leading

entry of a non-zero row, and a pivot column of a matrix in row echelon form is any
column that contains a pivot position. In our diagram representing a matrix in row
echelon form, the pivot positions are the positions of the black squares, and the
pivot columns are the columns containing those black squares. In the special case of
matrices in reduced row echelon form, the pivot positions are the positions of the
leading 1’s of the non-zero rows, and the pivot columns are the columns containing
those leading 1’s. For example, the matrix below is in reduced row echelon form (the
*’s are arbitrary numbers from the field in question); its pivot positions are boxed,
and the pivot columns (four of them) are the ones with the boxed entries.
 
0 1 ∗ 0 0 ∗ ∗ 0 ∗ ∗
 0 0 0 1
 0 ∗ ∗ 0 ∗ ∗  
 0 0 0 0 1 ∗ ∗ 0 ∗ ∗ 
 
 
 0 0 0 0
 0 0 0 1 ∗ ∗  
 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0

Theorem 1.3.6. Every matrix (with entries in some field) is row equivalent to a
unique matrix in reduced row echelon form.
Corollary 1.3.7. If two row equivalent matrices (with entries in some field) are
both in row echelon form, then they have exactly the same pivot positions and the
indices of their pivot columns are exactly the same.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 45

Remark: Suppose that A, B ∈ R4×5 , and that the pivot positions of A are positions
(1, 1) and (2, 3), so that the pivot columns of A are the first and third column of A.
Then by Corollary 1.3.7, the pivot positions of B are also (1, 1) and (2, 3), and the
pivot columns of B are the first and third column of B. The “shape” of both A and
B will be as in the diagram below.
 
■ ∗ ∗ ∗ ∗
 0 0 ■ ∗ ∗ 
 
 0 0 0 0 0 
0 0 0 0 0

We postpone the proof of Theorem 1.3.6 and Corollary 1.3.7 to subsection 1.3.7.
For now, we introduce some terminology and notation. By Theorem 1.3.6, every
matrix A (with entries in some field) is row equivalent to a unique matrix in reduced
row echelon form, which we call the reduced row echelon form of A, denoted by
RREF(A). A row echelon form of a matrix A is any matrix that is in row echelon
form and is row equivalent to A. A matrix may have more than one row echelon
form (i.e. it may be row equivalent to more than one matrix in row echelon form),
but by Corollary 1.3.7, all row echelon matrices of a given matrix have the same
“shape,” i.e. their “black squares” are in the same place. The pivot positions of an
arbitrary matrix A are the pivot positions of any matrix in row echelon form that
is row equivalent to A; by Corollary 1.3.7, this is well defined. A pivot column of
an arbitrary matrix A is any column of A that contains a pivot position of A. Note
that if we have computed the reduced row echelon form of a matrix A, then we can
immediately identify the pivot positions and the pivot columns of A.
Remark/Terminology: The number of pivot columns of a matrix A (equivalently:
the number of pivot positions of A, or the number of non-zero rows in a row echelon
form of A) has a special name: it is called the rank of A and is denoted by rank(A).
We will study rank in more detail in section 1.6, and it will subsequently reappear in
various contexts throughout these lecture notes.
The following corollary of Theorem 1.3.6 is also postponed to subsection 1.3.7.

Corollary 1.3.8. Two matrices (with entries in some field) are row equivalent if
and only if they have the same reduced row echelon form.

The row reduction algorithm. We now describe an algorithm, called the row
reduction algorithm, that transforms any matrix (with entries in some field) into
a row equivalent matrix that is in reduced row echelon form.9 The algorithm has
two parts: the “forward phase” and the “backward phase.” The forward phase
transforms the matrix into one in row echelon form. The backward phase transforms
9
We note this algorithm proves the existence part of Theorem 1.3.6, but not the uniqueness part.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 46

a matrix in row echelon form into one in reduced row echelon form. The forward
phase of the row reduction algorithm is also called “Gaussian elimination.” The
entire row reduction algorithm (with both the forward and the backward phase) is
also called the “Gauss-Jordan elimination.” In the description of the algorithm, we
will use the word “pivot” to mean the actual number that is in the pivot position in
question (or that we intend to move into the pivot position).10 We now describe the
row reduction algorithm.

Forward phase:
1. Begin with the leftmost non-zero column. This is a pivot column. The pivot
position is at the top of the column.11
2. Select a non-zero entry in the pivot column as a pivot. If necessary, interchange
rows to move this entry into the pivot position.
3. Use elementary row operations of the form “Rj → Rj + αRi ” (where row i
contains the pivot position in question, row j is below row i,12 and α is a
suitable scalar) to create zeros in all positions below the pivot position.
4. Cover (or ignore) the row containing the pivot position, as well as all the rows
(if any) above it. Apply steps 1-4 to the submatrix that remains. Repeat the
process until there are no more non-zero rows to modify.
Backward phase:
5. Beginning with the rightmost pivot column and working upward and to the
left, create zeros above each pivot position. If a pivot is not 1, make it 1 by a
scaling operation (“Ri → αRi ,” for a suitable scalar α ̸= 0).

A couple of implementations of the row reduction algorithm are given below (see
Examples 1.3.9 and 1.3.10). In each case, we first implement the forward phase, and
then we implement the backward phase. In the forward phase, we use a horizontal
dotted line as a visual aid: it separates the rows that have already been processed
(those are the ones above the horizontal dotted line) from the ones that have not yet
been processed (those are the ones below the horizontal dotted line). Moreover, the
pivot column that we have identified and are currently processing (as per step 1 or
step 5) is colored red.
10
Most texts never actually define the word (noun) “pivot.” Instead, they only define “pivot
positions” and “pivot columns,” as we did above. If they use the word “pivot” (as a noun), they
do so informally. We also use it slightly informally, but the advantage is that we get a simpler
description of the row reduction algorithm.
11
In the initial iteration, this means that the pivot position is in the top row. However, in
subsequent iterations, it will mean that the pivot position is “in the top row if we ignore the rows
that we have already processed and are done with.”
12
So, j > i.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 47

Example 1.3.9. Apply the row reduction algorithm to the matrix A below (with
entries understood to be in R) in order to compute its reduced row echelon form.
 
0 −3 −6 3 4 −1
A :=  2 1 −4 13 −4 3 
2 3 0 11 −6 5

Solution. We first implement the forward phase of the algorithm in order to transform
the matrix into one in row echelon form, as follows.
 
0 −3 −6 3 4 −1
A =  2 1 −4 13 −4 3 
2 3 0 11 −6 5
 
2 3 0 11 −6 5
R1 ↔R3
∼  2 1 −4 13 −4 3 
0 −3 −6 3 4 −1
 
2 3 0 11 −6 5
R2 →R2 −R1
∼  0 −2 −4 2 2 −2 
0 −3 −6 3 4 −1
 
R3 →R3 − 32 R2
2 3 0 11 −6 5
∼  0 −2 −4 2 2 −2  .
0 0 0 0 1 2

The forward phase of the row reduction algorithm is now complete: our matrix
is in row echelon form. It remains to implement the backward phase in order to
transform the matrix into one in reduced row echelon form. We compute:
 
2 3 0 11 −6 5 by the
A ∼  0 −2 −4 2 2 −2  forward
0 0 0 0 1 2 phase
 
R1 →R1 +6R3 2 3 0 11 0 17
R2 →R2 −2R3
∼  0 −2 −4 2 0 −6 
0 0 0 0 1 2
 
R2 →− 12 R2
2 3 0 11 0 17
∼  0 1 2 −1 0 3 
0 0 0 0 1 2

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 48

 
2 0 −6 14 0 8
R1 →R1 −3R2
∼  0 1 2 −1 0 3 
0 0 0 0 1 2
 
R1 → 21 R1
1 0 −3 7 0 4
∼  0 1 2 −1 0 3  .
0 0 0 0 1 2

The backward phase of row reduction is now complete: our matrix is in reduced row
echelon form. Thus,
 
1 0 −3 7 0 4
RREF(A) =  0 1 2 −1 0 3  .
0 0 0 0 1 2

Example 1.3.10. Apply the row reduction algorithm to the matrix B below (with
entries understood to be in Z3 ) in order to compute its reduced row echelon form.
 
0 1 1 0 2
 2 1 0 1 1 
B :=   2 1 1 1 1 


1 0 2 2 1

Solution. We will compute keeping the addition and multiplication tables for Z3
(below) in mind.

+ 0 1 2 · 0 1 2
0 0 1 2 0 0 0 0
1 1 2 0 1 0 1 2
2 2 0 1 2 0 2 1

We first implement the forward phase of the algorithm in order to transform the
matrix into one in row echelon form, as follows.
 
0 1 1 0 2
 2 1 0 1 1 
B = 
 2 1 1 1 1


1 0 2 2 1
 
1 0 2 2 1
R1 ↔R4  2 1 0 1 1 
∼ 
 2

1 1 1 1 
0 1 1 0 2

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 49

 
1 0 2 2 1
R2 →R2 +R1
R3 →R3 +R1  0 1 2 0 2 
∼ 
 0

1 0 0 2 
0 1 1 0 2
 
1 0 2 2 1
R3 →R3 +2R2
R4 →R4 +2R2  0 1 2 0 2 
∼ 
 0

0 1 0 0 
0 0 2 0 0
 
1 0 2 2 1
R4 →R4 +R3  0 1 2 0 2 
∼ 
 0

0 1 0 0 
0 0 0 0 0

The forward phase of our row reduction algorithm is now complete: our matrix
is in row echelon form. It remains to implement the backward phase in order to
transform the matrix into one in reduced row echelon form.
   
1 0 2 2 1 1 0 0 2 1
R1 →R1 +R3
(∗)  0 1 2 0 2  R2 →R2 +R3  0 1 0 0 2 
B ∼   0 0 1 0 0 
 ∼  0 0 1 0 0 ,
 

0 0 0 0 0 0 0 0 0 0

where (*) follows from the forward phase (above). The backward phase of row
reduction is now complete: our matrix is in reduced row echelon form. Thus,
 
1 0 0 2 1
 0 1 0 0 2 
RREF(B) =   0 0 1 0 0 .


0 0 0 0 0

When row reducing, we do not normally draw the horizontal dotted line and color
the pivot column that we are processing (as we did in Examples 1.3.9 and 1.3.10). It
is, however, good practice to indicate which elementary row operations are being
performed at each stage. Let us take a look at a couple of additional examples
(Examples 1.3.11 and 1.3.12 below) in which we omit the horizontal dotted line and
pivot column coloring, but carefully indicate which elementary row operation(s) we
are performing. For extra clarity, we also indicate the beginning and end of the
forward and backward phase of the row reduction algorithm (though this is not
strictly necessary).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 50

Example 1.3.11. Apply the row reduction algorithm to the matrices C1 and C2
below (with entries understood to be in Z2 ) in order to compute their reduced row
echelon form.
 
  1 1 1 1
1 1 1 1 1 1  1 0 1 0 
C1 :=  1 1 1 0 0 1  C2 :=   0 1 0 1 

1 1 0 0 1 1
1 0 1 1

Solution. We will compute keeping the addition and multiplication tables for Z2
(below) in mind.
+ 0 1 · 0 1
0 0 1 0 0 0
1 1 0 1 0 1

We first row reduce the matrix C1 , as follows:

 
1 1 1 1 1 1
start of the
C1 =  1 1 1 0 0 1 
forward phase
1 1 0 0 1 1
 
R2 →R2 +R1 1 1 1 1 1 1
R3 →R3 +R1
∼  0 0 0 1 1 0 
0 0 1 1 0 0

  end of the
1 1 1 1 1 1
R2 ↔R3 forward phase
∼  0 0 1 1 0 0 
(the matrix is in
0 0 0 1 1 0
row echelon form)
 
R1 →R1 +R3 1 1 1 0 0 1
R2 →R2 +R3 start of the
∼  0 0 1 0 1 0 
backward phase
0 0 0 1 1 0

  end of the
1 1 0 0 1 1 backward phase
R1 →R1 +R2
∼  0 0 1 0 1 0  (the matrix is in
0 0 0 1 1 0 reduced row
echelon form).

The last matrix from the calculation above is in reduced row echelon form, and we
conclude that

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 51

 
1 1 0 0 1 1
RREF(C1 ) =  0 0 1 0 1 0 .
0 0 0 1 1 0

We now row reduce C2 , as follows:

 
1 1 1 1
 1 0 1 0  start of the
C2 = 
 0 1 0

1  forward phase
1 0 1 1
 
1 1 1 1
R2 →R2 +R1
R4 →R4 +R1  0 1 0 1 
∼ 
 0

1 0 1 
0 1 0 0
 
1 1 1 1
R3 →R3 +R2
R4 →R4 +R2  0 1 0 1 
∼ 
 0

0 0 0 
0 0 0 1
 
1 1 1 1 end of the
R3 ↔R4  0 1 0 1  forward phase
∼ 
 0

0 0 1  (the matrix is in
0 0 0 0 row echelon form)
 
1 1 1 0
R1 →R1 +R3
R2 →R2 +R3  0 1 0 0  start of the
∼ 
 0

0 0 1  backward phase
0 0 0 0

  end of the
1 0 1 0
 0 backward phase
R1 →R1 +R2 1 0 0 
∼ 
 0
 (the matrix is in
0 0 1 
reduced row
0 0 0 0
echelon form).

The last matrix from the calculation above is in reduced row echelon form, and
we conclude that  
1 0 1 0
 0 1 0 0 
RREF(C2 ) =   0 0
.
0 1 
0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 52

Remark: Note that in our solution of Example 1.3.11, we applied elementary row
operations of only two types: “Ri ↔ Rj ” and “Ri → Ri + Rj ” (where i ̸= j in
both cases). This is not an accident! It is because we were working over Z2 , and Z2
contains only one non-zero element (number), namely 1. Of course, elementary row
operations of the type “Ri → 1Ri ” and “Ri → Ri + 0Rj ” (for i = ̸ j) are legal, but
they leave the matrix unchanged.

Example 1.3.12. Apply the row reduction algorithm to the matrices D1 and D2
below (with entries understood to be in Z5 ) in order to compute their reduced row
echelon form.
 
  4 3 2 1
2 1 0 2 3  0 1 2 3 
D1 :=  4 2 2 1 2  D2 :=   1 2 1 3 

3 4 1 2 2
2 1 3 3

Solution. We will compute keeping the addition and multiplication tables for Z5
(below) in mind.

+ 0 1 2 3 4 · 0 1 2 3 4
0 0 1 2 3 4 0 0 0 0 0 0
1 1 2 3 4 0 1 0 1 2 3 4
2 2 3 4 0 1 2 0 2 4 1 3
3 3 4 0 1 2 3 0 3 1 4 2
4 4 0 1 2 3 4 0 4 3 2 1

We first row reduce the matrix D1 , as follows:

 
2 1 0 2 3
 4 start of the
D1 = 2 2 1 2 
forward phase
3 4 1 2 2
 
R2 →R2 +3R1 2 1 0 2 3
R3 →R3 +R1
∼  0 0 2 2 1 
(∗)
0 0 1 4 0
 
2 1 0 2 3
R3 →R3 +2R2 end of the
∼  0 0 2 2 1 
forward phase
0 0 0 3 2

R1 →3R1  
R2 →3R2 1 3 0 1 4
R3 →2R3 start of the
∼  0 0 1 1 3 
(∗∗) backward phase
0 0 0 1 4
Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 53

 
R1 →R1 +4R3 1 3 0 0 0
R2 →R2 +4R3 end of the
∼  0 0 1 0 4 
(∗∗∗) backward phase.
0 0 0 1 4

The last matrix from the calculation above is in reduced row echelon form, and we
conclude that  
1 3 0 0 0
RREF(D1 ) =  0 0 1 0 4 .
0 0 0 1 4
Before moving on to the matrix D2 , let us make some comments on the calculation
above. In (*), we observe that 3 = −2 in Z5 , and so the elementary row operation
“R2 → R2 +3R1 ” is exactly the same as the elementary row operation “R2 → R2 −2R1 ”
(and indeed, we could have written “R2 → R2 − 2R1 ” instead of “R2 → R2 + 3R1 ”).
In any case, the goal was to turn the first entry of the second row into 0. We can
accomplish this using the fact that 4 + 3 · 2 = 0 (in Z5 ), or using the fact that
4−2·2 = 0 (again, in Z5 ). Similar remarks apply to (***): instead of “R1 → R1 +4R3 ”
and “R2 → R2 + 4R3 ,” we could have written “R1 → R1 − R3 ” and “R2 → R2 − R3 ,”
respectively, and we would have obtained the same result. Finally, in (**), we were
turning all pivots into 1’s, using the fact that, in Z5 , we have that 2−1 = 3 and
3−1 = 2.
We now row reduce the matrix D2 , as follows:
 
4 3 2 1
 0 1 2 3  start of the
D2 = 
 1

2 1 3  forward phase
2 1 3 3
 
4 3 2 1
R3 →R3 +R1
R4 →R4 +2R1  0 1 2 3 
∼ 
 0

0 3 4 
0 2 2 0
 
4 3 2 1
R4 →R4 −2R2  0 1 2 3  this is the same as
∼  
 0 0 3 4  “R4 → R4 + 3R2 ”
0 0 3 4
 
4 3 2 1 this is the same as
R4 →R4 −R3  0 1 2 3  “R4 → R4 + 4R3 ”
∼ 
 0

0 3 4  end of the
0 0 0 0 forward phase

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 54

  turning the leading

1 2 3 4
R1 →4R1
 0 entry of each
R3 →2R3 1 2 3 
∼ 
 0
 non-zero row into 1
0 1 3 
start of the
0 0 0 0
backward phase
 
1 2 0 0
R1 →R1 +2R3
R2 →R2 +3R3  0 1 0 2 
∼ 
 0

0 1 3 
0 0 0 0
 
1 0 0 1
R1 →R1 +3R2  0 1 0 2  end of the
∼ 
 0

0 1 3  backward phase.
0 0 0 0

The last matrix from the calculation above is in reduced row echelon form, and we
conclude that  
1 0 0 1
 0 1 0 2 
RREF(D2 ) =   0
.
0 1 3 
0 0 0 0

Warning: When working over Zp (where p is a prime number), all the entries
inside of your matrices should only be the numbers 0, 1, . . . , p − 1. As you process
your matrix, you may possibly get negative numbers −1, . . . , −(p − 1) inside of your
matrix, but your final answer should never contain negative numbers. For example,
if you are working over Z3 , you should turn any −1’s into 2’s, and any −2’s into 1’s.
In any case, when row reducing a matrix with entries in Zp (for a prime number p),
you should never (i.e. at no stage of the algorithm) have any numbers greater than
p − 1, and you should never have any fractions inside of your matrix!

Reasonable deviations from the row reduction algorithm. When computing

the reduced row echelon form of a matrix, it is in principle legal to apply any
elementary row operation at any stage (since this always produces a row equivalent
matrix). However, to efficiently turn a matrix into one in reduced row echelon form,
we should more or less follow the row reduction algorithm as described, because
otherwise, our calculation may become very long and very messy. That said, slight
deviations from the algorithm are sometimes a good idea. In particular, it is often
a good idea to rescale one or more rows at the beginning or in the middle of the

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 55

algorithm in order to eliminate fractions (when working over R), or perhaps to turn
the leading entry of one or more rows into 1’s. For instance, in Example 1.3.12, we
could have started our row reduction algorithm for D1 by rescaling the first row so
that the leading entry would become 1 (and then proceeding from there):
   
2 1 0 2 3 1 3 0 1 4
R →3R
D1 =  4 2 2 1 2  1 ∼ 1  4 2 2 1 2  .
3 4 1 2 2 3 4 1 2 2

We could also have chosen to rescale all three rows so that the leading entries in all
of them become 1:
  R1 →3R1  
2 1 0 2 3 R2 →4R2 1 3 0 1 4
R3 →2R3
D1 =  4 2 2 1 2  ∼  1 3 3 4 3 .
3 4 1 2 2 1 3 2 4 4
In any case, the basic idea of the row reduction algorithm is that, in the forward
phase, we identify pivot columns from left to right and we systematically “clean them
up” downward (i.e. all the entries below the pivot position that we are processing get
turned into 0), whereas in the backward phase, we identify pivot columns from right
to left and we “clean them up” upward (i.e. all the entries above the pivot position
that we are processing get turned into 0). This basic procedure should be respected,
since significant deviations from it may lengthen the procedure very considerably.

1.3.4 Solving linear systems

To find the solution set of a linear system, we proceed as follows. First, we form the
augmented matrix of our linear system, and using row reduction, we find the reduced
row echelon form of that matrix. Then, we “translate” this matrix (in reduced row
echelon form) into the linear system that it encodes. The linear system that we
obtain is equivalent to the one that we started with, that is, the two systems have
exactly the same solution set. We now read off the solution set as follows.

1. If the rightmost column of the augmented matrix (the one to the right of the
vertical dotted line) is a pivot column, then the system is inconsistent, i.e. it
has no solutions.

ˆ For example, suppose that by row reduction, we obtained the following

matrix (say, with coefficients in R).
 
1 0 −1 0
 0 1 5 0 
 
 0 0 0 1 
0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 56

This matrix encodes the following linear system:

x1 − x3 = 0
x2 + 5x3 = 0
0 = 1
0 = 0

Because of the equation “0 = 1,” the system is inconsistent (i.e. it has no

solutions).

2. If the rightmost column of the augmented matrix (the one to the right of the
vertical dotted line) is not a pivot column, but all the other columns are pivot
columns, then the system has a unique solution.

ˆ For example, suppose that by row reduction, we obtained the following

matrix (say, with coefficients in R).
 
1 0 0 −5
 0 1 0 0 
 
 0 0 1 3 
0 0 0 0

This matrix encodes the following linear system:

x1 = −5
x2 = 0
x3 = 3
0 = 0

This system is consistent and has a unique solution, which we can imme-
diately read off, as follows.

x1 = −5
x2 = 0
x3 = 3

3. If the rightmost column of the augmented matrix (the one to the right of the
vertical dotted line) is not a pivot column, and at least one of the other columns
is also not a pivot column, then the system has more than one solution, which
we read off as follows. The variables that correspond to the non-pivot columns
(we call these variables free variables) may take any value; these values (called
parameters) are denoted by letters such as r, s, t. The variables that correspond
to the pivot columns are called basic, and we solve for them in terms of our
parameters. This form of solution is called the parametric form of the solution;
we will also refer to it as the general solution.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 57

ˆ For example, suppose that by row reduction, we obtained the following

matrix (say, with coefficients in R).
 
1 2 0 5 6 0
 0 0 1 −1 7 −3 
0 0 0 0 0 0
This matrix encodes the linear system below.
x1 + 2x2 + 5x4 + 6x5 = 0
x3 − x4 + 7x5 = −3
0 = 0
The system is consistent and has more than one solution. The variables
variables x2 , x4 , x5 are free (because the non-pivot columns of the aug-
mented matrix to the left of the vertical dotted line are columns 2, 4, 5).
The remaining variables are basic. We now read off the solutions as
follows:
x1 = −2r − 5s − 6t
x2 = r
x3 = s − 7t − 3
x4 = s
x5 = t where r, s, t ∈ R.
Remark: Do not forget to specify which field your parameters come
from! Here, we have “r, s, t ∈ R” because the coefficients of our system
are in R.

Specifying the number of solutions of a linear system. An inconsistent linear

system has zero solutions. A consistent system may have a unique solution (i.e.
exactly one solution), or it may have more than one solution. A consistent system
with no free variables has a unique solution. A consistent system that has at least
one free variable has more than one solution, since each free variable can take an
arbitrary value from the field F in question. If our field is infinite (for example, if it
is Q, R, or C), then a consistent system with at least one free variable has infinitely
many solutions. On the other hand, if our field F is finite, and our linear system is
consistent with exactly k free variables, then the number of solutions of our system
is precisely |F|k (where |F| is the cardinality of F, i.e. the number of elements in F).
In particular, if F = Zp for some prime number p, then a consistent system with
exactly k free variables has exactly pk solutions.
Example 1.3.13. Solve the linear system below (with coefficients understood to be
in R), and specify how many solutions it has.
−3x2 − 6x3 + 3x4 + 4x5 = −1
2x1 + x2 − 4x3 + 13x4 − 4x5 = 3
2x1 + 3x2 + 11x4 − 6x5 = 5

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 58

Solution. The augmented matrix of this linear system is the matrix A below.
 
0 −3 −6 3 4 −1
A =  2 1 −4 13 −4 3 
2 3 0 11 −6 5

This is precisely the matrix from Example 1.3.9. The reduced row echelon form of
this matrix (computed in Example 1.3.9) is
 
1 0 −3 7 0 4
RREF(A) =  0 1 2 −1 0 3  .
0 0 0 0 1 2

We see from RREF(A) that the rightmost column (the one to the right of the vertical
dotted line) of the augmented matrix A of our linear system is not a pivot column;
therefore, our linear system is consistent. We further see from RREF(A) that the
pivot columns of A are its first, second and fifth column; so, the basic variables of
our linear system are x1 , x2 , x5 , whereas the remaining variables (namely, x3 , x4 ) are
free. Now, RREF(A) is the augmented matrix of the linear system below, which is
equivalent to our original linear system.

x1 − 3x3 + 7x4 = 4
x2 + 2x3 − x4 = 3
x5 = 2

We read off the solutions as follows:

x1 = 3s − 7t + 4
x2 = −2s + t + 3
x3 = s
x4 = t
x5 = 2 where s, t ∈ R.

Our linear system is consistent and has two free variables. Since the field R is infinite,
it follows that the number of solutions is infinite.
Optional: It is is easy to make mistakes when row reducing, and so it is generally a
good idea to check our solutions. We do this by plugging in our general solution into
the original system and checking that we get true statements. In this example, we
get the following.

−3(−2s + t + 3) − 6s + 3t + 4 · 2 = −1
2(3s − 7t + 4) + (−2s + t + 3) − 4s + 13t − 4 · 2 = 3
2(3s − 7t + 4) + 3(−2s + t + 3) + 11t − 6 · 2 = 5

By simplifying the left-hand-side, we see that all the equalities above are correct.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 59

Remark: When checking solutions, all the parameters should cancel out! If, after
simplifying, one of our equations became something like 2s + 7 = −2 or −t = 1, this
would tell us that we miscomputed somewhere. It does not matter that equalities
such as 2s + 7 = −2 or −t = 1 work for some values of s and t. They are supposed
to work for all possible values of the parameters. If they fail to work for some values,
then we know that we made a mistake somewhere and need to compute again.
Example 1.3.14. Solve the linear system below (with coefficients understood to be
in Z3 ), and specify how many solutions it has.
x2 + x3 = 2
2x1 + x2 + x4 = 1
2x1 + x2 + x3 + x4 = 1
x1 + 2x3 + 2x4 = 1
Solution. The augmented matrix of this linear system is the matrix B below.
 
0 1 1 0 2
 2 1 0 1 1 
B =   2

1 1 1 1 
1 0 2 2 1
This is precisely the matrix from Example 1.3.10. The reduced row echelon form of
this matrix (computed in Example 1.3.10) is
 
1 0 0 2 1
 0 1 0 0 2 
RREF(B) =   0 0 1
.
0 0 
0 0 0 0 0
We see from RREF(B) that the rightmost column (the one to the right of the vertical
dotted line) of the augmented matrix B of our linear system is not a pivot column;
therefore, our linear system is consistent. We further see from RREF(B) that the
pivot columns of B are its first, second, and third column; so, the basic variables
of our linear system are x1 , x2 , x3 , whereas the remaining variable (namely, x4 ) is
free. Now, RREF(B) is the augmented matrix of the linear system below, which is
equivalent to our original linear system.
x1 + 2x4 = 1
x2 = 2
x3 = 0
0 = 0
We read off the solutions as follows:
x1 = t + 1
x2 = 2
x3 = 0
x4 = t where t ∈ Z3 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 60

Our linear system is consistent and has one free variable. Since the field Z3 has three
elements, the number of solutions is 31 = 3.
Remark: To get x1 , we computed x1 = −2x4 + 1 = x4 + 1 = t + 1, where we used
the fact that in Z3 , we have that −2 = 1.
Optional: We check our solutions by plugging them into our original system.

2 + 0 = 2
2(t + 1) + 2 + t = 1
2(t + 1) + 2 + 0 + t = 1
(t + 1) + 2 · 0 + 2t = 1

By simplifying the left-hand-side, we see that all the equalities above are correct.
Here, it is important to remember that we are working in Z3 . For example, the
left-hand-side of the second equality simplifies as follows:

2(t + 1) + 2 + t = 2t + 2 + 2 + t = (2 + 1)t + 2 + 2 = 1,
| {z } | {z }
=0 =1

which is what we were supposed to get.

Example 1.3.15. Solve the linear systems (⋆) and (⋆⋆) below (with coefficients
understood to be in Z2 ), and specify how many solutions they have.

x1 + x2 + x3 + x4 + x5 = 1 
x1 + x2 + x3 = 1 (⋆)
x1 + x2 + x5 = 1



x1 + x2 + x3 = 1 

x1 + x3 = 0

(⋆⋆)
x2 = 1 

x1 + x3 = 1


Solution. We begin by solving the linear system (⋆). Its augmented matrix is the
matrix C1 below.  
1 1 1 1 1 1
C1 =  1 1 1 0 0 1 
1 1 0 0 1 1
This is precisely the matrix C1 from Example 1.3.11. The reduced row echelon form
of this matrix (computed in Example 1.3.11) is
 
1 1 0 0 1 1
RREF(C1 ) =  0 0 1 0 1 0 .
0 0 0 1 1 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 61

We see from RREF(C1 ) that the rightmost column (the one to the right of the
vertical dotted line) of the augmented matrix C1 of the linear system (⋆) is not a
pivot column; therefore, the linear system (⋆) is consistent. We further see from
RREF(C1 ) that the pivot columns of C1 are its first, third, and fourth column; so,
the basic variables of the linear system (⋆) are x1 , x3 , x4 , whereas the remaining
variables (namely, x2 , x5 ) are free. Now, RREF(C1 ) is the augmented matrix of the
linear system below, which is equivalent to our original linear system (⋆).

x1 + x2 + x5 = 1
x3 + x5 = 0
x4 + x5 = 0

We read off the solutions as follows:

x1 = s+t+1
x2 = s
x3 = t
x4 = t
x5 = t where s, t ∈ Z2 .

The system (⋆) is consistent and has two free variables. Since the field Z2 has two
elements, the number of solutions is 22 = 4.
Remark: Remember, in Z2 , we have that −1 = 1, and consequently, −s = s and
−t = t. We used this to solve for our basic variables (x1 , x3 , x4 ).
Optional: We check our solutions by plugging them into our original system (⋆).

(s + t + 1) + s + t + t + t = 1
(s + t + 1) + s + t = 1
(s + t + 1) + s + t = 1

By simplifying the left-hand-side, we see that all the equalities above are correct.
Here, it is important to keep in mind that we are working in Z2 . For example, the
left-hand-side of the first equality simplifies to

(s + t + 1) + s + t + t + t = (1 + 1)s + (1 + 1 + 1 + 1)t + 1 = 1,
| {z } | {z }
=0 =0

which is what we were supposed to get.

It remains to solve the linear system (⋆⋆). Its augmented matrix is the matrix
C2 below.  
1 1 1 1
 1 0 1 0 
C2 =   0 1 0 1 


1 0 1 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 62

This is precisely the matrix C2 from Example 1.3.11. The reduced row echelon form
of this matrix (computed in Example 1.3.11) is
 
1 0 1 0
 0 1 0 0 
RREF(C2 ) =   0 0 0
.
1 
0 0 0 0

We see from RREF(C2 ) that the rightmost column (the one to the right of the
vertical dotted line) of the augmented matrix C2 of the linear system (⋆⋆) is a pivot
column; therefore, the linear system (⋆⋆) is inconsistent. (The number of solutions is
zero.)
Remark: We cannot check this answer (since there are no solutions to plug into the
system). We can only hope that we did not make any mistakes in our calculation!
When in doubt, redo the whole calculation from scratch.

Example 1.3.16. Solve the linear systems (⋆) and (⋆⋆) below (with coefficients
understood to be in Z5 ), and specify how many solutions they have.

2x1 + x2 + 2x4 = 3 
4x1 + 2x2 + 2x3 + x4 = 2 (⋆)
3x1 + 4x2 + x3 + 2x4 = 2



4x1 + 3x2 + 2x3 = 1 

x2 + 2x3 = 3

(⋆⋆)
x1 + 2x2 + x3 = 3 

2x1 + x2 + 3x3 = 3


Solution. We begin by solving the linear system (⋆). Its augmented matrix is the
matrix D1 below.  
2 1 0 2 3
D1 :=  4 2 2 1 2 
3 4 1 2 2
This is precisely the matrix D1 from Example 1.3.12. The reduced row echelon form
of this matrix (computed in Example 1.3.12) is
 
1 3 0 0 0
RREF(D1 ) =  0 0 1 0 4  .
0 0 0 1 4

We see from RREF(D1 ) that the rightmost column (the one to the right of the
vertical dotted line) of the augmented matrix D1 of the linear system (⋆) is not a

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 63

pivot column; therefore, the linear system (⋆) is consistent. We further see from
RREF(D1 ) that the pivot columns of D1 are its first, third, and fourth column; so,
the basic variables of the linear system (⋆) are x1 , x3 , x4 , whereas the remaining
variable (namely, x2 ) is free. Now, RREF(D1 ) is the augmented matrix of the linear
system below, which is equivalent to our original linear system (⋆).
x1 + 3x2 = 0
x3 = 4
x4 = 4
We read off the solutions as follows:
x1 = 2t
x2 = t
x3 = 4
x4 = 4 where t ∈ Z5 .
The system (⋆) is consistent and has one free variable. Since the field Z5 has five
elements, the number of solutions is 51 = 5.
Optional: We check our solutions by plugging them into our original system (⋆).
2(2t) + t + 2·4 = 3
4(2t) + 2t + 2 · 4 + 4 = 2
3(2t) + 4t + 4 + 2·4 = 2
By simplifying the left-hand-side, we see that all the equalities above are correct. It is
important to keep in mind that we are working in Z5 . For example, the left-hand-side
of the first equality simplifies to
2(2t) + t + 2 · 4 = (2 · 2 + 1)t + (2 · 4 ) = 3,
|{z}
| {z }
=0 =3

which is what we were supposed to get.

It remains to solve the linear system (⋆⋆). Its augmented matrix is the matrix
D2 below.  
4 3 2 1
 0 1 2 3 
D2 :=   1 2

1 3 
2 1 3 3
This is precisely the matrix D2 from Example 1.3.12. The reduced row echelon form
of this matrix (computed in Example 1.3.12) is
 
1 0 0 1
 0 1 0 2 
RREF(D2 ) =   0 0 1
.
3 
0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 64

We see from RREF(D2 ) that the rightmost column (the one to the right of the
vertical dotted line) of the augmented matrix D2 of the linear system (⋆⋆) is not a
pivot column; therefore, the linear system (⋆⋆) is consistent. We further see from
RREF(D2 ) that all the columns of D2 other than the rightmost one (i.e. all the
columns to the left of the vertical dotted line) are pivot columns. So, all three
variables of the linear system (⋆⋆) are basic, and consequently, this linear system has
a unique solution. RREF(D2 ) is the augmented matrix of the linear system below.

x1 = 1
x2 = 2
x3 = 3
0 = 0

We now see that

x1 = 1
x2 = 2
x3 = 3
is the unique solution of the linear system (⋆⋆). In particular, the linear system (⋆⋆)
has exactly one solution.
Optional: We check our solution by plugging it into our original system (⋆⋆).

4·1 + 3·2 + 2·3 = 1

2 + 2·3 = 3
1 + 2·2 + 3 = 3
2·1 + 2 + 3·3 = 3

By simplifying the left-hand-side, we see that all the equalities above are correct.
(Again, we must keep in mind that we are computing in Z5 ).

Homogeneous linear systems. A homogeneous linear system is a linear system of

the form
a1,1 x1 + a1,2 x2 + . . . + a1,m xm = 0
a2,1 x1 + a2,2 x2 + . . . + a2,m xm = 0
..
.
an,1 x1 + an,2 x2 + . . . + an,m xm = 0
where the coefficients ai,j are all from some field F (and 0 is also understood to be
from that same field F). Such a system is always consistent: x1 = x2 = · · · = xm = 0
is a solution, called the trivial solution. A non-trivial solution of a homogeneous
linear system is a solution that is not trivial. Some homogeneous linear systems have
only the trivial solution, whereas others also have non-trivial solutions. (The former
happens when there are no free variables, and the latter happens when there is at
least one free variable.)

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 65

We note that when working with homogeneous linear systems, we typically row
reduce only the coefficient matrix, and not the augmented matrix (see Example 1.3.17
below).
Example 1.3.17. Solve the homogeneous linear system below, with coefficients
understood to be in R.
2x1 − 4x2 + 6x4 = 0
2x1 − 4x2 + 2x3 − 2x4 = 0
How many solutions does this homogeneous linear system have? Does it have any
non-trivial solutions?
Proof. The coefficient matrix of our homogeneous linear system is
 
2 −4 0 6
A :=
2 −4 2 −2
We row reduce this matrix as follows:
 
2 −4 0 6
A =
2 −4 2 −2
 
R2 →R2 −R1 2 −4 0 6
∼
0 0 2 −8

R1 → 12 R1
 
R2 → 12 R2 1 −2 0 3
∼ .
0 0 1 −4
The last matrix from the calculation above is in reduced row echelon form, and so
 
1 −2 0 3
RREF(A) = .
0 0 1 −4
Remark: We must keep in mind that A is the coefficient matrix of  our linear
system. The augmented matrix of our linear system would be A 0 . Since zero
columnsremain unchanged

when we perform elementary row operations, the matrix
RREF A 0 is obtained by adding a zero column to the right of RREF(A).
However, we do not normally write all this. We simply keep track of it mentally.

We now continue our computation. We see from the matrix RREF(A) that the
pivot columns of the coefficient matrix A are its first and third column. So, x1 , x3 are
the basic variables, and x2 , x4 are the free variables. Further, we see from RREF(A)
that our original linear system is equivalent to the linear system below.
x1 − 2x2 + 3x4 = 0
x3 − 4x4 = 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 66

We now read off the solutions as follows:

x1 = 2s − 3t
x2 = s
x3 = 4t
x4 = t where s, t ∈ R.

Since our system has free variables (in fact, two of them), and since we are working
over the infinite field R, we see that our system has infinitely many solutions. In
particular, our system has a non-trivial solution (in fact, it has infinitely many of
them).
Optional: We can check our solutions by plugging them into the original linear
system, as follows.
2(2s − 3t) − 4s + 6t = 0
2(2s − 3t) − 4s + 2(4t) − 2t = 0
By simplifying the left-hand-side, we see that both equalities above are correct.

An example with parameters. We complete this subsection with an example of a

linear system in which some of the coefficients depend on a “parameter,” which can
take many possible values (in Example 1.3.18 below, the parameter k may be any
real number). As we solve the system, we treat this parameter as a fixed constant.
If some (or all) of the coefficients of a linear system involve one or more parameters,
then the solutions of the system may depend on these parameters, and moreover,
the number of solutions may depend on these parameters.
Example 1.3.18. Solve the linear system below (with coefficients understood to be
in R), and specify how many solutions it has.

x1 + 2x2 + 3x3 = −1
2x1 + 3x2 + 4x3 = 2
2
4x1 + 7x2 + (k + 9)x3 = k − 1

Here, k is a parameter (which should be treated as a fixed real constant), and x1 , x2 , x3

are the variables.
Remark: Because of k 2 , it might seem that this system is not linear. However, k is
a constant, and not a variable. The variables that we need to solve for are x1 , x2 , x3 ,
and so the system is indeed linear.

Solution. The augmented matrix of the given linear is system is

 
1 2 3 −1
A :=  2 3 4 2 .
2
4 7 k +9 k−1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 67

First, we perform a sequence of elementary row operations on the matrix A, loosely

following the row reduction algorithm, in order to simplify the matrix as much as
possible. Importantly, we make sure we never divide by zero.
 
1 2 3 −1
A =  2 3 4 2 
2
4 7 k +9 k−1
 
R2 →R2 −2R1 1 2 3 −1
R3 →R3 −4R1
∼  0 −1 −2 4 
2
0 −1 k − 3 k + 3
 
1 2 3 −1
R3 →R3 −R2
∼  0 −1 −2 4 
0 2
0 k −1 k−1
 
1 2 3 −1
R2 →−R2
∼  0 1 2 −4 
2
0 0 k −1 k−1
 
1 0 −1 7
R1 →R1 −2R2
∼  0 1 2 −4 
2
0 0 k −1 k−1
| {z }
=:B

At this point, we would like to divide the bottom row by k 2 − 1, but there is a
potential problem: it is possible that k 2 − 1 = 0. Clearly, k 2 − 1 = 0 if and only if
k = ±1. Thus, we will consider three cases: when k = 1, when k = −1, and when
k ̸= ±1.
Case 1: k = 1.
By plugging in k = 1 into the matrix B, we obtain the following:
 
1 0 −1 7
A ∼ B =  0 1 2 −4  .
0 0 0 0

Our matrix is in reduced row echelon form, and it is the augmented matrix of the
linear system below.
x1 − x3 = 7
x2 + 2x3 = −4
0 = 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 68

The system is consistent, and its solutions are

x1 = t + 7
x2 = −2t − 4
x3 = t where t ∈ R.

Since the system has a free variable, and since the field R is infinite, we see that our
system has infinitely many solutions.
Case 2: k = −1.
By plugging in k = −1 into the matrix B, we obtain
 
1 0 −1 7
A ∼ B =  0 1 2 −4  .
0 0 0 −2

The bottom row corresponds to the equation 0 = −2, and we deduce that our system
is inconsistent. (The number of solutions is zero.)
Case 3: k ̸= ±1.
In this case, we have that k 2 − 1 ̸= 0, which means that we may divide by k 2 − 1.
We now continue row reducing as follows:
 
1 0 −1 7
A ∼ B =  0 1 2 −4 
2
0 0 k −1 k−1
 
R3 → 1
R3
1 0 −1 7
k2 −1
∼  0 1 2 −4 
k−1
0 0 1 k2 −1
 
1 0 −1 7
=  0 1 2 −4 
1
0 0 1 k+1

1
 
1 0 0 7+ k+1
R1 →R1 +R3
R2 →R2 −2R3 
 0 1 0 −4 − 2

∼  k+1


1
0 0 1 k+1

7k+8
 
1 0 0 k+1
 0 1 0 − 4k+6  .
 
=  k+1 
1
0 0 1 k+1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 69

Our last matrix is in reduced row echelon form. We now see that
7k+8
x1 = k+1

x2 = − 4k+6
k+1
1
x3 = k+1

is the unique solution of our linear system. In particular, our linear system has
exactly one solution.

Optional: In Cases 1 and 3, in which we concluded that our system is consistent,

we can check our solutions by plugging them into the original system.
Case 1: k = 1.
We plug in k = 1, plus the solutions that we obtained for our system in Case 1,
and we obtain the following.

(t + 7) + 2(−2t − 4) + 3t = −1
2(t + 7) + 3(−2t − 4) + 4t = 2
2
4(t + 7) + 7(−2t − 4) + (1 + 9)t = 1 − 1

After simplifying, we see that all three equalities above are correct.
Case 3: k ̸= ±1.
We plug in the solutions that we obtained in Case 3 into our original system,
and we obtain the following.
7k+8 4k+6 1




k+1 + 2 − k+1 + 3 k+1 = −1
2 7k+8 4k+6 1




k+1 + 3 − k+1 + 4 k+1 = 2
4 7k+8 4k+6
+ (k 2 + 9) 1




k+1 + 7 − k+1 k+1 = k−1

After simplifying, we see that all three equalities above are correct.

1.3.5 Solving systems of linear equations via back substitution

In this subsection, we present a way of solving linear systems by performing only
the forward phase of the row reduction algorithm on the augmented matrix (and
thus transforming it into a matrix in row echelon form, but not necessarily reduced
row echelon form), and then solving the equivalent system via “back substitution.”
Rather than explaining the general principle, we give a couple of examples. We note
that the systems in Examples 1.3.19 and 1.3.20 (below) are precisely those from
Examples 1.3.13 and 1.3.14, respectively, and so, unsurprisingly, we get the same
answer.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 70

Example 1.3.19. Find the solution set of the following system of linear equations
(with coefficients in R).

−3x2 − 6x3 + 3x4 + 4x5 = −1

2x1 + x2 − 4x3 + 13x4 − 4x5 = 3
2x1 + 3x2 + 11x4 − 6x5 = 5

Solution. The augmented matrix of this linear system is the matrix A below.
 
0 −3 −6 3 4 −1
A =  2 1 −4 13 −4 3 
2 3 0 11 −6 5

This is precisely the matrix from Example 1.3.9. By performing only the forward phase
of the row reduction algorithm on the matrix A (see the solution of Example 1.3.9
for the details), we see that
 
2 3 0 11 −6 5
A ∼  0 −2 −4 2 2 −2  ,
0 0 0 0 1 2

where the pivot columns are in red for emphasis. Since the rightmost column (the
one to the right of the vertical dotted line) is not a pivot column, our system is
consistent. The pivot columns are the first, second, and fifth column. Therefore,
x1 , x2 , x5 are the basic variables, while the remaining variables (namely, x3 , x4 ) are
free. Now, the matrix above is the augmented matrix of the linear system below
(which is equivalent to our original system).

2x1 + 3x2 + 11x4 − 6x5 = 5

− 2x2 − 4x3 + 2x4 + 2x5 = −2
x5 = 2

The free variables x3 , x4 become arbitrary parameters, say x3 = s and x4 = t (where

s, t ∈ R). We plug this into the linear system above, and we obtain the following.

2x1 + 3x2 + 11t − 6x5 = 5

− 2x2 − 4s + 2t + 2x5 = −2
x5 = 2

We now solve for the basic variables, working our way from the bottom up. From
the bottom equation, we get x5 = 2. If we plug x5 = 2 into the equation above (the
second-from-bottom one), we get −2x2 − 4s + 2t + 2 · 2 = −2. By solving for x2 , we
get x2 = −2s + t + 3. We now plug in both x2 = −2s + t + 3 and x5 = 2 into the top
equation, which yields 2x1 + 3(−2s + t + 3) + 11t − 6 · 2 = 5. By solving for x1 , we

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 71

obtain x1 = 3s − 7t + 4. By putting all this together, we obtain the general solution

of our linear system:

x1 = 3s − 7t + 4
x2 = −2s + t + 3
x3 = s
x4 = t
x5 = 2 where s, t ∈ R.

We note that this is exactly the same as the solution that we obtained in Exam-
ple 1.3.13.

Example 1.3.20. Find the solution set of the following system of linear equations
(with coefficients in Z3 ).

x2 + x3 = 2
2x1 + x2 + x4 = 1
2x1 + x2 + x3 + x4 = 1
x1 + 2x3 + 2x4 = 1

Solution. The augmented matrix of this linear system is the matrix B below.
 
0 1 1 0 2
 2 1 0 1 1 
B =   2

1 1 1 1 
1 0 2 2 1

This is precisely the matrix from Example 1.3.10. By only performing the for-
ward phase of the row reduction algorithm on the matrix B (see the solution of
Example 1.3.10 for the details), we obtain
 
1 0 2 2 1
 0 1 2 0 2 
B ∼   0 0 1 0 0 ,


0 0 0 0 0

where the pivot columns are in red for emphasis. Since the rightmost column (the
one to the right of the vertical dotted line) is not a pivot column, our system is
consistent. The pivot columns are the first, second, and third column. Therefore,
x1 , x2 , x3 are the basic variables, whereas the variable x4 is free. Now, the matrix
above is the augmented matrix of the system below (which is equivalent to our
original system).
x1 + 2x3 + 2x4 = 1
x2 + 2x3 = 2
x3 = 0
0 = 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 72

The free variable x4 becomes an arbitrary parameter, say x4 = t (where t ∈ Z3 ). We

now plug this into the linear system above to obtain the following.
x1 + 2x3 + 2t = 1
x2 + 2x3 = 2
x3 = 0
0 = 0
Finally, we solve for the basic variables, working our way from the bottom up. The
bottom equation (“0 = 0”) gives us no information, so we ignore it. The equation
above it yields x3 = 0. We then plug that into the equation right above to obtain
x2 + 2 · 0 = 2; solving for x2 , we get x2 = 2. We now plug in both x2 = 2 and x3 = 0
into our top equation, and we obtain x1 + 2 · 0 + 2t = 1. Solving for x1 (and keeping
in mind that in Z3 , we have −2t = t), we obtain x1 = t + 1. By putting all this
together, we obtain the general solution of our linear system:
x1 = t+1
x2 = 2
x3 = 0
x4 = t where t ∈ Z3 .
We note that this is exactly the same as the solution that we obtained in Exam-
ple 1.3.14.

Remark: Computers typically use back substitution to solve linear systems. However,
when computing by hand (especially when the numbers are reasonably nice), it is
more convenient to find the reduced row echelon form of the augmented matrix
and then read off the solutions, as described in subsection 1.3.4.

1.3.6 A few more remarks about the (reduced) row echelon form
Submatrices. A submatrix of a matrix A is any matrix obtained from A by possibly
deleting some rows and some columns. For example, the matrix
 
1 1 2 2
B =  0 2 3 5 
3 5 5 3
is a submatrix of the matrix
 
1 2 1 2 1 2
 0 1 2 3 4 5 
A = 
 8

4 2 1 2 4 
3 4 5 5 4 3
because B can be obtained from A by deleting the third row and the second and
fifth column, as shown below.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 73

 
1 2 1 2 1 2
 0 1 2 3 4 5 
 
 8 4 2 1 2 4 
3 4 5 5 4 3

Submatrices and row equivalence. Suppose that A and B are row equivalent
matrices. If A′ is a submatrix of A obtained by possibly deleting some columns of A
(and no rows), and B ′ is the submatrix of B obtained by deleting the corresponding
columns of B, then A′ and B ′ are also row equivalent. Indeed, any sequence of
elementary row operations that transforms A into B will transform A′ into B ′ . For
example, we have the following (matrix entries are assumed to be in R):
   
0 1 2 0 1 2 1 2 1 2 1 1
R1 ↔R2
 1 2 1 2 1 1  ∼  0 1 2 0 1 2 
3 8 3 8 5 5 3 8 3 8 5 5
 
1 2 1 2 1 1
R3 →R3 −3R1
∼  0 1 2 0 1 2 
0 2 0 2 2 2
 
R3 → 12 R3
1 2 1 2 1 1
∼  0 1 2 0 1 2 .
0 1 0 1 1 1

If we delete, say, the first, fourth, and fifth column throughout, we get the following:
   
0 1 2 0 1 2 1 2 1 2 1 1
R1 ↔R2
 1 2 1 2 1 1  ∼  0 1 2 0 1 2 
3 8 3 8 5 5 3 8 3 8 5 5
 
1 2 1 2 1 1
R3 →R3 −3R1
∼  0 1 2 0 1 2 
0 2 0 2 2 2
 
R3 → 21 R3
1 2 1 2 1 1
∼  0 1 2 0 1 2 .
0 1 0 1 1 1

Warning: Do not delete rows! Deleting rows may destroy row equivalence.

Submatrices and the row echelon  form. If A and B are matrices with the same
number of rows, then we denote by A B the matrix obtained by placing A and

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 74

B next to each other (A is to the left and B is to the right). For example, if
   
1 2 3 4 0 3
A =  2 3 4 5  and B =  2 0 ,
3 4 5 6 0 1

then  
  1 2 3 4 0 3
A B =  2 3 4 5 2 0 .
3 4 5 6 0 1
     

Further,
 when
 we write A1 B1 ∼ A2 B2 or RREF A1 B1 =
A2 B2 , we implicitly assume that A1 is of the same size as A2 (i.e. A1 and A2
have the same number of rows and the same number of columns), and that B1 and
B2 are of the same size.
It follows immediately from the appropriate definitions that if a matrix A is in
row echelon form, then any submatrix of A that lies in the upper-left corner of A
is also in row echelon form (see the diagram below for illustration; the submatrix
in question is in red). However, other submatrices of a matrix in row echelon form
need not be in row echelon form.

0 ■ ∗ ∗ ∗ ∗ ∗ ∗ ∗ ∗
 
 0 0 0 ■ ∗ ∗ ∗ ∗ ∗ ∗ 
 
 0 0 0 0 ■ ∗ ∗ ∗ ∗ ∗ 
 
 0 0 0 0 0 0 0 ■ ∗ ∗ 
 
 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0
Similarly, if a matrix A is in reduced row echelon form, then any submatrix of A
that lies in the upper-left corner of A is also in reduced row echelon form (see the
diagram below for illustration; the submatrix in question is in red). However, other
submatrices of a matrix in reduced row echelon form need not be in reduced row
echelon form.
0 1 ∗ 0 0 ∗ ∗ 0 ∗ ∗
 
 0 0 0 1 0 ∗ ∗ 0 ∗ ∗ 
 
 0 0 0 0 1 ∗ ∗ 0 ∗ ∗ 
 
 0 0 0 0 0 0 0 1 ∗ ∗ 
 
 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0
 
In particular, this means that if a matrix A B is in row echelon form (resp.
reduced row echelon form), then A is also in row echelon form (resp. reduced row
echelon form).
Further, note that if A is a matrix in row echelon form (resp. reduced row echelon
form), then adding an arbitrary number of zero columns to the right of the matrix

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 75

A, or an arbitrary number of zero rows on the bottom of the matrix A, produces

another matrix in row echelon form (resp. reduced row echelon form).13

Proposition 1.3.21. Let F be a field, let A1 , A2 ∈ Fn×m , and let B1 , B2 ∈ Fn×p .

Then both the following hold:
   
(a) if A1 B1 ∼ A2 B2 , then A1 ∼ A2 and B1 ∼ B2 ;
 
 
(b) if RREF A1 B1 = A2 B2 , then RREF(A1 ) = A2 .

Warning: The converse

 of (a)
 does  that is, it is possible that A1 ∼ A2
 not hold,
and B1 ∼ B2 , but A1 B1 ̸∼ A1 B1 . Furthermore, in (b), it is possible
that RREF(B1 ) ̸= B2 (because B2 need not be in reduced row echelon form).
   
Proof. (a) Assume that A1 B1 ∼ A2 B2 . Then  some sequence  of elemen-

tary row operations transforms the matrix A1 B1 into the matrix A2 B2 .
If we apply that same sequence of elementary row operations to the matrix A1 , we
obtain the matrix A2 ; similarly, if we apply that same sequence of elementary row
operations to the matrix B1 , we obtain the matrix B2 . This proves that A1 ∼ A2
and B1 ∼ B2 .
 
   
 (b) Assume
 that RREF A1 B1 = A2 B2 . Then A1 B1 ∼
 A2 B2 , and so by (a), we have that A1 ∼ A2 . Moreover, since the matrix
A2 B2 is in reduced row echelon form, so is the matrix A2 .14 So, RREF(A1 ) =
A2 .

Recall that the n × m zero matrix in Fn×m (where F is some field) is the n × m
matrix, all of whose entries are 0;15 this matrix is denoted by On×m . As our next
proposition shows, the converse of Proposition 1.3.21 holds in the special case when
B1 = B2 = On×p .

Proposition 1.3.22. Let F be a field, let A1 , A2 ∈ Fn×m , and let On×p be the zero
matrix in Fn×p . Then both the following hold:
   
(a) A1 ∼ A2 if and only if A1 On×p ∼ A2 On×p ;
 
 
(b) RREF(A1 ) = A2 if and only if RREF A1 On×p = A2 On×p .
13
As a matter of fact, we can insert zero columns into a matrix in row echelon form anywhere
(left, right, middle), and we will obtain another matrix in row echelon form; similar remarks apply
to matrices in reduced row echelon form. However, zero rows can only be added to the bottom, or
at least below any non-zero rows (if we insert zero rows elsewhere, a matrix in row echelon form will
no longer be in row echelon form).
14
However, B2 need not be in reduced row echelon form!
15
The 0 is from our field F.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 76

Proof. (a) Note that elementary row operations leave any zero columns unaffected.
Therefore, any sequence
 of elementary
  row operations
 that transforms A1 into A2
also transforms
 A 1 O n×p  into A 2  n×p , and vice versa. So, A1 ∼ A2 if
O
and only if A1 On×p ∼ A2 On×p .
   
(b) By (a), we have that A1 ∼ A2 if and only if A1 On×p ∼ A2 On×p .
Moreover, it follows immediately from the definition that A2 is in reduced row
echelon form if and only if the matrix A2 On×p is in reduced  row echelon


form.
 This proves
 that RREF(A1 ) = A2 if and only if RREF A1 On×p =
A2 On×p .

1.3.7 Proof of Theorem 1.3.6 and Corollaries 1.3.7 and 1.3.8

The existence part of Theorem 1.3.6 essentially follows from the row reduction
algorithm. To prove uniqueness, we need the following lemma.
Lemma 1.3.23. Let F be a field. If two reduced row echelon matrices in Fn×m are
row equivalent, then they are in fact equal.
Proof. We keep the number of rows (n) fixed, and we proceed by induction on the
number of columms (m). More precisely, we fix a positive integer n, and we prove
(by induction on m) that for all positive integers m, if two reduced row echelon
matrices in Fn×m are row equivalent, then they are in fact equal.
Base case: m = 1. There are exactly two reduced row echelon matrices in Fn×1 ,
namely,    
0 1
 0   0 
   
 0   0 
  and  .
 ..   .. 
 .   . 
0 0
(Here, the matrix/vector on the left has n many 0’s, and the matrix/vector on the
right has n − 1 many 0’s.) Since no elementary row operation will transform a
zero column into a non-zero column, we see that these two matrices are not row
equivalent.
Induction step: Fix a positive integer m, and assume inductively that the claim is
true for m. We must prove that it is true for m + 1. Fix two row equivalent reduced
row echelon matrices A and B in Fn×(m+1) . We must show that A = B.
Let A′ and B ′ be the n × m matrices obtained by deleting the rightmost column
of A and B, respectively. Since A and B are in reduced row echelon form, so are A′
and B ′ .16 Next, since A and B are row equivalent, so are A′ and B ′ (indeed, any
sequence of elementary row operations that transforms A into B also transforms A′
into B ′ ). So, by the induction hypothesis, A′ = B ′ . Thus, A and B are of the form
16
This follows straight from the definition of a matrix in reduced row echelon form.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 77

   
a1,1 . . . a1,m c1 a1,1 . . . a1,m d1
ˆ A =  ... .. .. .. , ˆ B =  ... .. .. .. ,
 
. . .  . . . 
an,1 . . . an,m cn an,1 . . . an,m dn

where the ai,j ’s, the ci ’s, and the di ’s are some elements of the field F. (Here, the
submatrix
  of both A and B to the left of the vertical dotted line is A′ = B ′ =
ai,j n×m .) We must show that c1 = d1 , . . . , cn = dn .
Assume that A′ = B ′ has exactly k pivot columns, and assume that those pivot
columns are columns number j1 , . . . , jk (appearing from left to right in A′ = B ′ ,
so that j1 < . . . jk ≤ m). Thus, the matrix A′ = B ′ has precisely k non-zero rows,
and for each i ∈ {1, . . . , m}, the leading 1 of the i-th row of A′ = B ′ is in the ji -th
column. So, schematically, the matrix A′ = B ′ looks as shown in the diagram below
(the first k rows are non-zero, the pivot columns are in red, indices of the pivot
columns are shown on top, row indices are shown on the left, and the horizontal
dotted line separates the non-zero rows from the zero rows).

j1 j2 jk
∗ ∗ ∗ ∗ ∗
 
1 0 1 0 0 0
2 
 0 0 0 1 0 ∗ ∗ 0 ∗ ∗ 


 0 0 0 0 1 ∗ ∗ 0 ∗ ∗ 

k 
 0 0 0 0 0 0 0 1 ∗ ∗ 

 0 0 0 0 0 0 0 0 0 0 
n 0 0 0 0 0 0 0 0 0 0

Now, consider the linear systems (⋆) and (⋆⋆) below, whose augmented matrices
are A and B, respectively. (Note that this means that they have the same coefficient
matrix, namely A′ = B ′ .)

a1,1 x1 + . . . + a1,m xm = c1  
.. (⋆)
. 
an,1 x1 + . . . + an,m xm = cn



a1,1 x1 + . . . + a1,m xm = d1 

.. (⋆⋆)
. 
an,1 x1 + . . . + an,m xm = dn


Since A and B are row equivalent, the linear systems (⋆) and (⋆⋆) are equivalent,
i.e. they have exactly the same solutions. In particular, (⋆) and (⋆⋆) are both either
consistent or both inconsistent.
Suppose first that the linear systems (⋆) and (⋆⋆) are both inconsistent. Since (⋆)
is inconsistent, the rightmost column of A is a pivot column. Since A is in reduced

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 78

row echelon form, and since it has exactly k pivot columns to the left of its rightmost
column (i.e. to the left of the vertical dotted line), we see that the rightmost column
of A has 1 in the (k + 1)-th row and 0’s everywhere else. Schematically, the matrix
A is of the following form (where to the left of the vertical dotted line, we have the
matrix A′ = B ′ , and the horizontal dotted line is inherited from A′ = B ′ ):

∗ ∗ ∗ ∗ ∗
 
0 1 0 0 0 0

 0 0 0 1 0 ∗ ∗ 0 ∗ ∗ 0 


 0 0 0 0 1 ∗ ∗ 0 ∗ ∗ 0 
.

 0 0 0 0 0 0 0 1 ∗ ∗ 0 

 0 0 0 0 0 0 0 0 0 0 1 
0 0 0 0 0 0 0 0 0 0 0
In other words, we have that ck+1 = 1 and ci = 0 for all i ∈ {1, . . . , n} \ {k}. Since
(⋆⋆) is also inconsistent, a completely analogous argument shows that dk+1 = 1 and
di = 0 for all i ∈ {1, . . . , n} \ {k}. It follows that c1 = d1 , . . . , cn = dn , and we are
done.
Suppose now that the linear systems (⋆) and (⋆⋆) are both consistent. Since
(⋆) is consistent, the rightmost column of its augmented matrix A is not a pivot
column. Since A is in reduced row echelon form, and since it has exactly k pivot
columns to the left of its rightmost column (i.e. to the left of the vertical dotted
line), we see that the bottom n − k many rows of A are all zero, and in particular,
ck+1 = · · · = cn = 0. Schematically, the matrix A is of the following form (where to
the left of the vertical dotted line, we have the matrix A′ = B ′ , and the horizontal
dotted line is inherited from A′ = B ′ ):
 
0 1 ∗ 0 0 ∗ ∗ 0 ∗ ∗ c1
 0 0 0 1 0 ∗ ∗ 0 ∗ ∗ c2 
 .. 

 0 0 0 0 1 ∗ ∗ 0 ∗ ∗ . 
.

 0 0 0 0 0 0 0 1 ∗ ∗ ck 

 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0

Since (⋆⋆) is also consistent, a completely analogous argument shows establishes that
dk+1 = · · · = dn = 0. Schematically, the matrix B is of the following form (where to
the left of the vertical dotted line, we have the matrix A′ = B ′ , and the horizontal
dotted line is inherited from A′ = B ′ ):
 
0 1 ∗ 0 0 ∗ ∗ 0 ∗ ∗ d1
 0 0 0 1 0 ∗ ∗ 0 ∗ ∗ d2 
 .. 

 0 0 0 0 1 ∗ ∗ 0 ∗ ∗ . 
.

 0 0 0 0 0 0 0 1 ∗ ∗ dk 

 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 79

We have now shown that ck+1 , . . . , cn , dk+1 , . . . , dn are all zero. It remains to
show that c1 = d1 , . . . , ck = dk .
We consider the systems (⋆) and (⋆⋆). In both of those systems, the basic
variables are the variables xj1 , . . . , xjk , and the remaining variables are free. We first
consider the linear system (⋆). If we set all the free variables to 0 and solve for the
basic variables, we get the following solution of the system (⋆):

ˆ xj = 0 for all j ∈ {1, . . . , m} \ {j1 , . . . , jk };

ˆ and xji = ci for all i ∈ {1, . . . , k}.

Now, since the linear systems (⋆) and (⋆⋆) are equivalent, this solution of (⋆) is also
a solution of (⋆⋆). But if we plug it into (⋆⋆), we obtain the following.

c1 = d1
c2 = d2
..
.
ck = dk 
0 = 0 
0 = 0 

.. n−k
. 


0 = 0


In particular, c1 = d1 , . . . , ck = dk . This completes the argument.

We are now ready to prove Theorem 1.3.6, restated below.

Theorem 1.3.6. Every matrix (with entries in some field) is row equivalent to a
unique matrix in reduced row echelon form.

Proof. The row reduction algorithm transforms any matrix into one in reduced
row echelon form; these two matrices are row equivalent because the row reduction
algorithm is simply a particular sequence of elementary row operations. This proves
the existence part of the theorem: every matrix is row equivalent to at least one
matrix in reduced row echelon form.
It remains to prove uniqueness. Fix any matrix A (with entries in some field),
and suppose that it is row equivalent to matrices A1 and A2 , both in reduced row
echelon form. But then A1 ∼ A ∼ A2 ; consequently (by the transitivity of row
equivalence, see Proposition 1.3.5(c)), we have that A1 ∼ A2 . But now Lemma 1.3.23
guarantees that A1 = A2 . This proves uniqueness, and we are done.

We complete this subsection by proving Corollaries 1.3.7 and 1.3.8.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 80

Corollary 1.3.7. If two row equivalent matrices (with entries in some field) are
both in row echelon form, then they have exactly the same pivot positions and the
indices of their pivot columns are exactly the same.

Proof. Let A and B be row equivalent matrices in row echelon form, both with
entries in some field F. By performing the backward phase of the row reduction
algorithm on the matrix A, we obtain the matrix RREF(A), which is in reduced
row echelon form, is row equivalent to A, and (by the description of the backward
phase of the row reduction algorithm) has exactly the pivot positions as A. Similarly,
RREF(B) is in reduced row echelon form, is row equivalent to B, and has exactly
the same pivot positions as B. Now, we have that
(∗)
RREF(A) ∼ A ∼ B ∼ RREF(B),

where (*) is true by hypothesis. So, by the transitivity of row equivalence, it follows
that RREF(A) ∼ RREF(B). Since both RREF(A) and RREF(B) are in row echelon
form, Theorem 1.3.6 implies that they are in fact equal. But now both A and B
have exactly the same pivot positions as the matrix RREF(A) = RREF(B), and
consequently, the indices of the pivot columns of A and B are also the same.

Corollary 1.3.8. Two matrices (with entries in some field) are row equivalent if
and only if they have the same reduced row echelon form.

Proof. Fix two matrices A and B (with entries in some field). By Theorem 1.3.6,
the matrix A is row equivalent to a unique matrix in reduced row echelon form,
denoted by RREF(A). Similarly, the matrix B is row equivalent to a unique matrix
in reduced row echelon form, denoted by RREF(B).
Suppose first that A ∼ B. Then RREF(A) ∼ A ∼ B ∼ RREF(B), and so by
the transitivity of row equivalence, we have that RREF(A) ∼ RREF(B). But now
Lemma 1.3.23 guarantees that RREF(A) = RREF(B).
Conversely, suppose that RREF(A) = RREF(B). Then A ∼ RREF(A) =
RREF(B) ∼ B. By the transitivity of row equivalence, it follows that A ∼ B, and
we are done.

1.4 Algebraic operations on vectors and matrices

1.4.1 Vector addition, vector subtraction, and scalar-vector multi-
plication
We can add and subtract vectors, and we can also multiply them by scalars, as
follows. Let F be some field.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 81

ˆ Given two vectors in Fn , say

  

x1 y1
x =  ...  and y =  ...  ,
   

xn yn
we define the sum of x and y by
 
x 1 + y1
x + y :=  ..
,
 
.
x n + yn
and the difference of x and y by
 
x 1 − y1
x − y :=  ..
,
 
.
x n − yn

where the sums xi + yi and differences xi − yi (for i = 1, . . . , n) are computed

in the field F.
ˆ Given a vector  
x1
x =  ... 
 

xn
in Fn and a scalar α ∈ F, we define the scalar-vector product of α and x by
 
αx1
αx :=  ...  ,
 

αxn
where the products αx1 , . . . , αxn are computed in the field F.

Terminology: A scalar multiple of a vector x ∈ Fn (where F is some field) is any

vector of the form αx, where α ∈ F.
Notation: By convention, for a vector x and a scalar α, we write αx, but we do
not write xα. In other words, by convention, we have “scalar times vector,” but not
“vector times scalar.”
Example 1.4.1. Consider the vectors
  

0 1
 1   0 
x =  
 2  and y =  
 2 
2 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 82

in Z43 . Then
     
 
0+1 1 2·0 0
 1+0   1   2·1   2 
 2+2  =  1
x+y =    
 and 2x = 
 2·2  =  1
  .

2+1 0 2·2 1

(Reminder: In Z3 , we have that 2 + 2 = 1, 2 + 1 = 0, and 2 · 2 = 1.)

1.4.2 Vector addition and scalar multiplication in R2

   
a1 b1
To add two vectors in R2 , say a = and b = , we apply the “parallelo-
a2 b2
gram rule,” as shown below.
x2

a+b
a2 + b2 a
a2
b2 b
a1 b1 a2 + b2 x1

Scalar
 multiplication can be interpreted as follows. Suppose we are given a vector

a1
a= and a scalar c ∈ R. If c > 0, then ca is the vector that points in the
a2
same direction as a, but whose length is scaled by c.
x2 x2
ca2
c>0
a2 ca
a

a1 x1 ca1 x1

On the other hand, if c < 0, then ca is the vector that points in the opposite direction
to a, but whose length is scaled by |c| = −c.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 83

x2
(c < 0)
a2
a
ca1
a1 x1
ca

ca2

If c = 0, then ca = 0,
 which
 is simply
 the origin.
a1 b1
For vectors a = and b = in R2 , we note that a − b = a + (−1)b,
a2 b2
which yields the geometric interpretation below.
x2

a
a−b
b
x1
−b

For vectors in R3 , we have a similar geometric interpretation, only in the three-

dimensional Euclidean space.

1.4.3 Linear combinations of vectors

Suppose F is some field. A linear combination of vectors v1 , . . . , vk in Fn is any sum
of the form
k
P
α i vi = α 1 v1 + · · · + α k vk ,
i=1

where α1 , . . . , αk are scalars from

 the field
 F.  
5 0 −3
For example, in R3 , vectors  6 ,  3 , and  −9  are linear combinations
5 0 −3
   
1 1
of the vectors  3  and  0  because
1 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 84

     
5 1 1
ˆ  6 = 2 3 + 3 0 ;
   
5 1 1
         
0 1 1 1 1
ˆ  3  =  3  −  0  = 1  3  + (−1)  0 ;
0 1 1 1 1
       
−3 1 1 1
ˆ  −9  = −3  3  = (−3)  3  + 0  0 .
−3 1 1 1
   
2 1
Similarly, is a linear combination of the vector in Z23 because
1 2
   
2 1
= 2 .
1 2

We note that in Fn (where F is a field), the zero vector 0 is a linear combination of

any vectors v1 , . . . , vk because

0 = 0v1 + · · · + 0vk .

Moreover, we define the “empty sum” of vectors in Fn (or the sum of an “empty list”
of vectors in Fn ) to be 0, where 0 is the zero vector in Fn .

Linear span. The linear span (or simply span) of vectors v1 , . . . , vk in Fn (where F
is a field), denoted by Span({v1 , . . . , vk }) or simply Span(v1 , . . . , vk ), is the set of
all linear combinations of vectors v1 , . . . , vk . In other words,
k
nP o
Span(v1 , . . . , vk ) = αi vi | α1 , . . . , αk ∈ F .
i=1

So, by definition, a vector v belongs to Span(v1 , . . . , vk ) if and only if it can be written

as a linear combination the vectors v1 , . . . , vk . As a special case, the empty sum of
vectors is equal to the zero vector, and so Span(∅) = {0}. Obviously, Span(0) = {0}.
We will study the linear span in more generality in chapter 3 (see subsection 3.1.2).
Here, let us try to give a geometric intuition for the special case of Rn . As we discussed
above, Span(∅) = {0} and Span(0) = {0}. If v ̸= 0, then Span(v) = {αv | α ∈ R}
is the line through the origin containing v: indeed, Span(v) is the set of all scalar
multiples of v, which is precisely the line through 0 and v (this is illustrated below
for the special case of R2 ).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 85

x2

Span(v)

x1

What if we have two vectors v1 and v2 ? If neither of those vectors is a scalar multiple
of the other (and in particular, neither of the two vectors is 0), then Span(v1 , v2 ) is
the plane through0, v1, v2 . The case
 that
 is particularly easy to visualize is that of
1 0
the vectors e1 :=  0  and e2 :=  1  in R3 :
0 0
 
n o n a1 o
Span(e1 , e2 ) = a1 e1 + a2 e2 | a1 , a2 ∈ R =  a2  | a1 , a2 ∈ R ,
0

which is simply the x1 x2 -plane in R3 , shown below.

x3

 
0 


e2 = 1 



0
 
 
1 


e1 = 0 



x2
0
 

Span(e1, e2)

x1

But what if we have two vectors, one of which is a scalar multiple of the other? If
v1 , v2 ∈ Rn , with v2 = αv1 for some scalar α ∈ R and v1 ̸= 0, then Span(v1 , v2 ) is
the line through the origin, v1 , and v2 . In the case of R2 , this is illustrated below.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 86

x2

Span(v1, v2)

v2 = αv1

v1

x1

In general, for vectors v1 , . . . , vk in Rn , the set Span(v1 , . . . , vk ) is the smallest “flat”

(point, line, plane, or higher dimensional generalization) containing the origin and
all the vectors v1 , . . . , vk .

1.4.4 Matrix-vector multiplication

Suppose that F is some field. Given a matrix A ∈ Fn×m and a vector x ∈ Fm , say
 
x1
x =  ...  ,
 
A = a1 . . . am and
 

xm
we define the matrix-vector product Ax as follows:
m
P
Ax := xi ai = x1 a1 + · · · + xm am .
i=1

Thus, Ax is a linear combination of the columns of A, and the weights/scalars in

front of the columns are determined by the entries of the vector x.
Note that, for the matrix-vector product Ax to be defined, two conditions must
be satisfied:
ˆ entries of the matrix A and entries of the vector x must belong to the same
field;
ˆ the number of columns of A must be the same as the number of entries of x.
Schematically, we have the following:
A
|{z} x
|{z} = |{z}
Ax .
∈Fn×m ∈Fm ∈Fn

Example 1.4.2. Consider the matrix A ∈ R3×2 and vector x ∈ R2 , given below:
 
−1 2  
2
A =  2 0  and x = .
3
3 −2

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 87

Then
       
−1 2   −1 2 4
2
Ax =  2 0  = 2 2  + 3 0  =  4 .
3
3 −2 3 −2 0

Example 1.4.3. Consider the matrix A ∈ Z2×3

2 and vector x ∈ Z32 , given below:
 
  1
1 1 0
A = and x =  1 .
1 0 1
0

Then
 
  1        
1 1 0  1  = 1 1 +1 1 +0 0 0
Ax = = .
1 0 1 1 0 1 1
0

Proposition 1.4.4. Let F be a field, let a1 , . . . , am ∈ Fn , and set A :=

 
a1 . . . am .
Then
Span(a1 , . . . , am ) = Ax | x ∈ Fm .


Proof. We compute:

Span(a1 , . . . , am ) = x1 a1 + · · · + xm am | x1 , . . . , xm ∈ F
 
x1
n  .  o
= a1 . . . am  ..  | x1 , . . . , xm ∈ F
xm

Ax | x ∈ Fm .

=

This completes the argument.

Remark:
 Suppose
 that a1 , . . . , am ∈ Fn , where F is some field, and set A :=
a1 . . . am . Proposition 1.4.4 states that Span(a1 , . . . , am ), which we defined
as the set of all linear combinations of the vectors a1 , . . . , am , is in fact the set of
all possible matrix-vector products Ax (where our matrix A = a1 . . . am is
fixed, and the vector x ∈ Fm is allowed to vary). We note that Span(a1 , . . . , am ),
the span of the columns of A, has a special name: it is called the “column space”
of the matrix A, and it is denoted by Col(A). We will study the column space of a
matrix in more detail in chapter 3 (see section 3.3).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 88

The standard basis vectors. Let F be a field. For each positive integer n and
index i ∈ {1, . . . , n}, the vector eni is the vector in Fn whose i-th entry is 1, and all
of whose other entries are 0’s (here, both 0 and 1 are understood to belong to the
field F). Schematically, for each index i ∈ {1, . . . , n}, the vector eni is given by
 
0
 .. 
 . 
 
 0 
n
 
ei =   1  ←− i-th entry

 0 
 
 .. 
 . 
0

(here, we have exactly one 1, and we have n − 1 many 0’s). When n is clear from
context, we drop the superscript n, and we write e1 , . . . , en instead of en1 , . . . , enn ,
respectively. Vectors e1 , . . . , en are called the standard basis vectors of Fn , and the
set En := {e1 , . . . , en } is called the standard basis of Fn . We note that any vector
 
v1
v =  ... 
 

vn

in Fn can be expressed as a linear combination of the standard basis vectors e1 , . . . , en

in a unique way, namely

v = v1 e1 + · · · + vn en .

As our next proposition shows, multiplying a matrix by the i-th standard basis
vector yields the i-th column of the matrix that we started with.
 
Proposition 1.4.5. Let F be a field, and let A = a1 . . . am be a matrix in
Fn×m . Then for all indices i ∈ {1, . . . , m}, we have that Aem
i = ai .

Proof. Fix i ∈ {1, . . . , m}. Then

 
0
 .. 
 . 
 
 0 
 
Aem

= a1 . . . ai−1 ai ai+1 . . . am  1  ←− i-th entry
i  
 0 
 
 .. 
 . 
0

= 0a1 + · · · + 0ai−1 + 1ai + 0ai+1 + · · · + 0am = ai ,

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 89

which is what we needed to show.

The identity matrix. For a field F, the identity matrix in Fn×n is the n × n matrix
 n
e1 . . . enn .

In :=

In other words, the identity matrix In is the n×n matrix with 1’s on the main diagonal
and 0’s elsewhere (where the 1’s and the 0’s are from the field F). Schematically, we
have that  
1 0 0 ... 0 0
 0 1 0 ... 0 0 
 
 0 0 1 ... 0 0 
In =  . . . .
 
. . 
 .. .. .. . . .. .. 
 
 0 0 0 ... 1 0 
0 0 0 . . . 0 1 n×n
for all positive integers n. For small values of n, we have:
 
  1 0 0
  1 0
I1 = 1 , I2 = , I3 =  0 1 0  .
0 1
0 0 1

As our next proposition shows, if we multiply the identity matrix by a vector, we

obtain that same vector.

Proposition 1.4.6. Let F be a field. Then for all vectors v ∈ Fn , we have that
In v = v.
 
v1
Proof. For any vector v =  ...  in Fn , we have that
 

vn
 
v1
 v2 
en1 en2 enn

In v = ...
 
 .. 
 . 
vn

= v1 en1 + v2 en2 + · · · + vn enn

     
1 0 0
 0   1   0 
= v1   + v2   + · · · + vn 
     
.. .. .. 
 .   .   . 
0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 90

 
v1
 v2 
= = v,
 
 .. 
 . 
vn

which is what we needed.

Multiplying by zero. Recall that, for a field F, the zero matrix in Fn×m , denoted
by On×m , is the n × m matrix, all of whose entries are 0 (where the 0 is from the
field F.

Proposition 1.4.7. Let F be a field. Then both the following hold:

(a) for all v ∈ Fm , we have that On×m v = 0;17

(b) for all matrices A ∈ Fn×m , we have that A0 = 0.18

Proof. This readily follows from the definition of matrix-vector multiplication.

1.5 Matrix-vector equations

A matrix-vector equation is an equation of the form Ax = b, where the matrix A
and vector b are known, and the vector x is unknown. Once again, the entries of A
and b must come from the same field F. Moreover, the number of rows of A must
be the same as the number of entries of b. Any solution x will then be a vector in
Fm , where m is the number of columns of A.
As we shall see, a matrix-vector equation is equivalent to a system of linear
equations. Before considering the general case, let us first take a look at an example.
Consider the following matrix A and vector b (with entries understood to be in R):
   
1 2 3 7
A = , b= .
4 5 6 8

We now transform the matrix-vector equation Ax = b into a system of linear

equations, as follows.

17
Here, the zero vector 0 belongs to Fn .
18
Here, the first 0 belongs to Fm , whereas the second 0 belongs to Fn . Or, if we color code for
convenience, in the expression A0 = 0, we have that 0 ∈ Fm and 0 ∈ Fn .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 91

 
  x1  
1 2 3  x2  = 7
Ax = b ⇐⇒ 4 5 6 8
x3
| {z }
=x
       
1 2 3 7
⇐⇒ x1 + x2 + x3 =
4 5 6 8
   
x1 + 2x2 + 3x3 7
⇐⇒ =
4x1 + 5x2 + 6x3 8

x1 + 2x2 + 3x3 = 7
⇐⇒
4x1 + 5x2 + 6x3 = 8

Note that the augmented matrix of the linear system that we obtained is
 
  1 2 3 7
A b = .
4 5 6 8

Let us now consider the general case. Suppose that F is afield,A ∈ Fn×m is a
b1
n
   .. 
matrix, and b ∈ F is a vector. Set A = ai,j n×m and b =  . . We transform
bn
the matrix-vector equation Ax = b into a system of linear equations, as follows.
    
a1,1 a1,2 . . . a1,m x1 b1
 a2,1 a2,2 . . . a2,m   x2   b2 
=  . 
    
 .. .. . . .
. .
. .
Ax = b ⇐⇒
  
 . . . .  .   . 
an,1 an,2 . . . an,m xm bn
| {z }
=x
       
a1,1 a1,2 a1,m b1
 a2,1   a2,2   a2,m   b2 
⇐⇒ x1   + x2   + · · · + xm   = 
       
.. .. .. .. 
 .   .   .   . 
an,1 an,2 an,m bn
   
a1,1 x1 + a1,2 x2 + · · · + a1,m xm b1
 a2,1 x1 + a2,2 x2 + · · · + a2,m xm   b2 
⇐⇒  = 
   
 .. .. 
 .   . 
an,1 x1 + an,2 x2 + · · · + an,m xm bn

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 92



 a1,1 x1 + a1,2 x2 + · · · + a1,m xm = b1
 a2,1 x1 + a2,2 x2 + · · · + a2,m xm = b2

⇐⇒ ..


 .
an,1 x1 + an,2 x2 + · · · + an,m xm = bn


So, solving the matrix-vector equation Ax = b boils down to solving a system of

linear equations. The augmented matrix of this linear system is the matrix
 
a1,1 a1,2 . . . a1,m b1
   a2,1 a2,2 . . . a2,m b2 
A b =  . ..  .
 
. .. . . ..
 . . . . . 
an,1 an,2 . . . an,m bn
 
The matrix A b will also be referred to as the augmented matrix of the matrix-
vector equation Ax = b. As in the case of linear systems, a matrix-vector equation
Ax = b may have no solutions, may have exactly one solution, or may have more
than one solution. A matrix-vector equation that has at least one solution is called
consistent; a matrix-vector equation that has no solutions is said to be inconsistent.

Example 1.5.1. Solve the matrix-vector equation Ax = b, where

   
1 2 2
A = and b = ,
3 6 6

with entries understood to be in R. How many solutions does the matrix-vector

equation Ax = b have?

Solution. The augmented matrix of the matrix-vector equation Ax = b is

 
  1 2 2
A b = .
3 6 6
 

We now row reduce in order to find RREF A b , as follows:
   
  1 2 2 R2 →R2 −3R1 1 2 2
A b = ∼ .
3 6 6 0 0 0

The last matrix from the computation above is in reduced row echelon form, and we
deduce that
 
  1 2 2
RREF A b = .
0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 93

 

The matrix RREF A b is the augmented matrix of the linear system below.

x1 + 2x2 = 2
0 = 0

The system is consistent, with one free variable (namely, x2 ). We read off the
solutions as follows.
x1 = −2s + 2
x2 = s, where s ∈ R.

So, the general solution of the matrix-vector equation Ax = b is

 
−2s + 2
x = , where s ∈ R.
s

Here is another way to write the general solution of the matrix-vector equation
Ax = b:    
2 −2
x = +s , where s ∈ R.
0 1

Remark: We obtained this second form of the solution by separating the constant
part of x form the part with the parameter, and then factoring out the parameter,
as follows:  
−2s + 2
x =
s
   
2 −2s
= +
0 s
   
2 −2
= +s , where s ∈ R.
0 1

The set of solutions of the matrix-vector equation Ax = b is

n  −2s + 2  o n 2  
−2
 o
|s∈R = +s |s∈R .
s 0 1

Since the parameter s can take infinitely many values (because R is infinite), the
matrix-vector equation Ax = b has infinitely many solutions.

Remark: The solution

 set from Example
 1.5.1 has a geometric interpretation.
n −2 o −2
Indeed, s | s ∈ R = Span( ) is the line that passes through the
1 1
 
−2
origin and the point (this is the blue line in the picture below). The solution
1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 94

n 2  
−2
 o  
2
set +s | s ∈ R is obtained by shifting this line by the vector ,
0 1 0
 
2
i.e. by adding the vector to each point on the line (this vector is shown in
0
n 2  
−2
 o
purple in the picture below). The solution set +s | s ∈ R is the
0 1
red line in the picture below.
x2
 
−2
1  
2
0

n 2    x1
−2 o
+s |s∈R
0 1
n  −2  o
s |s∈R
1

Example 1.5.2. Solve the matrix-vector equation Ax = b, where

   
1 2 0 1 2
A =  1 0 1 0  and b =  2  ,
2 2 1 1 0

with entries understood to be in Z3 . How many solutions does the matrix-vector

equation Ax = b have?

Solution. The augmented matrix of the matrix-vector equation Ax = b is

 
  1 2 0 1 2
A b =  1 0 1 0 2 .
2 2 1 1 0
 

We now row reduce in order to find RREF A b , as follows:
 
  1 2 0 1 2
A b =  1 0 1 0 2 
2 2 1 1 0
 
R2 →R2 +2R1 1 2 0 1 2
R3 →R3 +R1
∼  0 1 1 2 0 
0 1 1 2 2

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 95

 
1 2 0 1 2
R3 →R3 +2R2
∼  0 1 1 2 0 
0 0 0 0 2
 
1 2 0 1 2
R3 →2R3
∼  0 1 1 2 0 
0 0 0 0 1
 
1 2 0 1 0
R1 →R1 +R3
∼  0 1 1 2 0 
0 0 0 0 1
 
1 0 1 0 0
R1 →R1 +R2
∼  0 1 1 2 0 .
0 0 0 0 1

The last matrix from the computation above is in reduced row echelon form, and we
deduce that
 
  1 0 1 0 0
RREF A b =  0 1 1 2 0 .
0 0 0 0 1
 
 
We see from RREF A b that the rightmost column of A b is a pivot
column; consequently, the matrix-vector equation Ax = b is inconsistent, i.e. the
solution set of the equation Ax = b is ∅. (The number of solutions of the matrix-
vector equation Ax = b is zero.)

Remark: In the solution of Example 1.5.2, we could in fact have stopped as soon as
we got the red matrix (despite the fact that this matrix is not in reduced row echelon
form). This is because the bottom row of the red matrix encodes the equation
0
 = 2, which is has no solutions. Indeed, as soon as we obtain a row of the form
0 . . . 0 ■ , where ■ is a non-zero number, we can stop row reducing, and we
can deduce that the system has no solutions (because this row encodes the equation
0 = ■, and ■ is non-zero).

Example 1.5.3. Solve the matrix-vector equation Ax = b, where

   
1 1 1
A = and b = ,
1 0 1

with entries understood to be in Z2 . How many solutions does the matrix-vector

equation Ax = b have?

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 96

Solution. The augmented matrix of the matrix-vector equation Ax = b is

 
  1 1 1
A b = .
1 0 1
 

We now row reduce in order to find RREF A b , as follows:
 
  1 1 1
A b =
1 0 1
 
R2 →R2 +R1 1 1 1
∼
0 1 0
 
R1 →R1 +R2 1 0 1
∼ .
0 1 0

The last matrix from the computation above is in reduced row echelon form, and we
deduce that
 
  1 0 1
RREF A b = .
0 1 0

We now see that the matrix-vector equation Ax = b has a unique solution, namely
 
1
x = .
0
n 1 o
The solution set of the matrix-vector equation Ax = b is . (The number of
0
solutions of the matrix-vector equation Ax = b is one.)
Example 1.5.4. Solve the matrix-vector equation Ax = b, where
   
1 1 0 1 1
A =  1 0 1 0  and b =  0  ,
0 1 1 1 1
with entries understood to be in Z2 . How many solutions does the matrix-vector
equation Ax = b have?
Solution. The augmented matrix of the the
matrix-vector equation Ax = b is
 
  11 0 1 1
A b =  1 0 1 0 0 .
01 1 1 1
 

We now row reduce in order to find RREF A b , as follows:

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 97

 
  1 1 0 1 1
A b =  1 0 1 0 0 
0 1 1 1 1
 
1 1 0 1 1
R2 →R2 +R1
∼  0 1 1 1 1 
0 1 1 1 1
 
1 1 0 1 1
R3 →R3 +R2
∼  0 1 1 1 1 
0 0 0 0 0
 
1 0 1 0 0
R1 →R1 +R2
∼  0 1 1 1 1 .
0 0 0 0 0

The last matrix from the computation above is in reduced row echelon form, and we
deduce that
 
  1 0 1 0 0
RREF A b =  0 1 1 1 1 .
0 0 0 0 0
 
 
We see from RREF A b that the rightmost column of A b is not a
pivot
 column,
 and so the matrix-vector equation Ax = b is consistent. The matrix
A b has two non-pivot columns to the left of the vertical dotted line, namely,
the third and fourth column. So, the third and fourth entry of the solution x of
Ax = b become arbitrary parameters. The general solution of the matrix-vector
equation Ax = b is the following:19
19
Normally, we do indeed
 read off

the solutions of the matrix-vector equation Ax = b straight
from the matrix RREF A b . However, let us give a slightly

more detailed explanation of
how exactly we got our answer. The matrix RREF A b is the augmented matrix of the
linear system below.
x1 + x3 = 0
x2 + x3 + x4 = 1
0 = 0
The system is consistent, with two free variables (namely, x3 and x4 ). We read off its solution as
follows.
x1 = s
x2 = s + t + 1
x3 = s
x4 = t where s, t ∈ Z2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 98

 
s
 s+t+1 
x =  , where s, t ∈ Z2 .
 s 
t
We can also write the general solution of the matrix-vector equation Ax = b as
follows:      
0 1 0
 1   1   1 
x =   0  + s 1  + t 0 ,
     where s, t ∈ Z2 .
0 0 1

Remark: We obtained this second form of the general solution by separating the
constant part from the parts associated with each parameter:

      
s 0 s 0
 s+t+1   1   s   t 
x =   =  + + 
 s   0   s   0 
t 0 0 t
    
0 1 0
 1   1
 + t 1 ,
  
= 
 0  + s 1
 
  0  where s, t ∈ Z2 .
0 0 1

The solution set of the matrix-vector equation Ax = b is

       
s 0 1 0
n s + t + 1  n 1 
  + s 1  + t 1 
o    o
  | s, t ∈ Z2 =  | s, t ∈ Z2 .
 s   0   1   0 
t 0 0 1

There are two parameters (namely, s and t), and each of them can take two values
(because |Z2 |). So, the matrix-vector equation Ax = b has 22 = 4 solutions.20
So, the general solution of the matrix-vector equation Ax = b is
   
x1 s
 x2   s+t+1 
x =   x3  = 
  , where s, t ∈ Z2 .
s 
x4 t

20
Since there are only four solutions, we could easily list all of them:

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 99

1.5.1 Matrix-vector equations and linear span

Let F be a field, and consider vectors a1 , . . . , am (m ≥ 1) and b in Fn . How
do we determine if b is a linear combination of the vectors a1 , . . . , am , that is, if
b ∈ Span(a1 , . . . , am )? Set A = a1 . . . am , and recall that by Proposition 1.4.4,
we have that
Span(a1 , . . . , am ) = {Ax | x ∈ Fm }.
So, b ∈ Span(a1 , . . . , am ) if and only if there exists some x ∈ Fm such that Ax = b.
In other words, b ∈ Span(a1 , . . . , am ) if and only if the matrix-vector equation
Ax = b is consistent. Thus, determining whether b is a linear combination of the
vectors a1 , . . . , am boils down to solving the matrix-vector equation Ax = b, which
we know how to do.
We give two examples. The solution to the first one (Example 1.5.5) aims to
carefully justify all the steps. The solution to the second one (Example 1.5.6) is
more concise, but follows the same process. When solving examples by yourself, you
should aim to give the amount of detail given in the solution to Example 1.5.6.

 
0
 1 
ˆ for s = 0 and t = 0, we have the solution  0 ;


0
 
0
 0 
ˆ for s = 0 and t = 1, we have the solution  0 ;


1
 
1
 0 
ˆ for s = 1 and t = 0, we have the solution  1 ;


0
 
1
 1 
ˆ for s = 1 and t = 1, we have the solution  1 .


1
       
0 0 1 1
n 1   0   0   1 o
So, the solution set is 
 0  ,  0  ,  1  ,  1  . However, we usually leave the final answer
      

0 1 0 1
in parametric form, rather than listing all possible solutions one by one.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 100

Example 1.5.5. Consider the vectors

      

2 1 1 1
 0   0   1   2 
 1 ,
a1 =   a2 =  2 ,

 1 ,
a3 =  
 0 ,
a4 =  

0 0 1 2
    
  
2 0 2 0
 1   0   0   1 
 0 ,
b= 
 1 ,
c= 
 2 ,
d= 
 2 ,
e= 

1 1 0 1

with entries understood to be in Z3 . For each of the vectors b, c, d, e, determine if

it can be expressed as a linear combination of the vectors a1 , a2 , a3 , a4 (that is, if it
belongs to Span(a1 , a2 , a3 , a4 )), and if so, express it as such a linear combination,
and explain whether your answer is unique.

Solution. Set
 
2 1 1 1
   0 0 1 2 
A := a1 a2 a3 a4 = 
 1
.
2 1 0 
0 0 1 2

We have that b ∈ Span(a1 , a2 , a3 , a4 ) if and only if the matrix-vector equation

Ax = b is consistent (and similar for the vectors c, d, and e). So, the obvious way
to proceed would be to solve four matrix-vector equations, namely, Ax = b, Ax = c,
Ax = d, and Ax = e. However, this would require row reducing four times! We can
solve the problem more efficiently by forming the matrix (color coded for emphasis)
 
2 1 1 1 2 0 2 0
   0 0 1 2 1 0 0 1 
A b c d e =   1 2 1 0 0 1 2 2 ,


0 0 1 2 1 1 0 1

and then row reducing to obtain

 
1 2 0 1 2 0 0 1
 
 0 0 1 2 1 0 0 1 
RREF A b c d e = 
 0

0 0 0 0 1 0 0 
0 0 0 0 0 0 1 0

A′ b′ c′ d′ e′
 
=: .

A′ b′ c′ d′ e′ , and consequently,
   
We now have that A b c d e ∼

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 101

ˆ A′ b′ ;
  
A b ∼

ˆ A c ∼ A′ c ′ ;
   

ˆ A d ∼ A′ d′ ;
   

ˆ A e ∼ A′ e′ .
   

(Indeed, the same sequence of′ elementary row operations that transforms the
 matrix
′ c′ d′ e′
   
A b c d e into A b will transform matrices A b ,
A  c , A d , and A e into matrices A b′ , A′ c′ , A′ d′ ,
       ′    

and A′ e′ , respectively.)
From this point on, we deal with the vectors b, c, d, e separately.

Vector b. We need to check whether there exist scalars x1 , x2 , x3 , x4 ∈ Z3 such that

b = x1 a1 + x2 a2 + x3 a3 + x4 a4 ,

and if so, to find such xi ’s. Theabove

 is equivalent to solving the matrix-vector
′ ′

equation Ax = b. Since A b ∼ A b , we can “read off” the solutions of
the matrix-vector equation Ax = b (if they exist) from the matrix
 
1 2 0 1 2
 ′  0 0 1 2 1 
A b′

=   0 0 0 0 0 .


0 0 0 0 0

We see that our equation Ax = b is consistent, and that the general solution is
   
x1 s + 2t + 2
 x2   s 
x =   x3  = 
  , where s, t ∈ Z3 .
t+1 
x4 t

However, we were asked to find just one particular solution, and not the general
solution. In principle, we could choose any values from Z3 for the parameters s and
t, but it is easiest to choose s = t = 0, which yields
   
x1 2
 x2   0 
x =   x3  =  1  .
  

x4 0

So, b is indeed a linear combination of the vectors a1 , a2 , a3 , a4 , that is, b ∈

Span(a1 , a2 , a3 , a4 ), and we have that

b = 2a1 + 0a2 + 1a3 + 0a4 = 2a1 + a3 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 102

(Both b = 2a1 + 0a2 + 1a3 + 0a4 and b = 2a1 + a3 are acceptable as a final answer,
though the second form is more common.) We note that our solution is not unique,
because we had parameters for which we chose particular values.
Remark:  Here is a slightly different way to proceed. By looking at the matrix
′

A b , we see that the matrix-vector equation Ax = b is consistent. We do not
need the general solution, but only one possible solution. So, we cross out (or simply
′ ′
ignore) the non-pivot columns of A b to the left of the vertical dotted line,21


and we read off the (unique) solution that remains after we eliminated those columns.
Here, it is important to remember which xi corresponds to which column.
 
1 2 0 1 2
0 0 1 2 1 
A′ b′
  
= 
 0

0 0 0 0 
0 0 0 0 0
↑ ↑
x1 x3

We now read off x1 = 2 and x3 = 1, and we get b = 2a1 + a3 . Because we crossed out
some non-pivot columns (which correspond to arbitrary parameters), our solution is
not unique.

Vector c. We need to check whether there exist scalars x1 , x2 , x3 , x4 ∈ Z3 such that

c = x1 a1 + x2 a2 + x3 a3 + x4 a4 ,

and if so, to find such xi ’s. The above

 is equivalent to solving the matrix-vector
′ ′

equation Ax = c. Since A c ∼ A c , we can “read off” the solutions of
the matrix-vector equation Ax = c (if they exist) from the matrix
 
1 2 0 1 0
 ′  0 0 1 2 0 
A c′

=   0 0 0 0 1 .


0 0 0 0 0

The rightmost column (the one to the right of the vertical dotted line) is a pivot
column, and it follows that the matrix-vector equation Ax = c is inconsistent.
Consequently, the vector c is not a linear combination of the vectors a1 , a2 , a3 , a4 ,
i.e. c ∈
/ Span(a1 , a2 , a3 , a4 ).

Vector d. We need to check whether there exist scalars x1 , x2 , x3 , x4 ∈ Z3 such that

d = x1 a1 + x2 a2 + x3 a3 + x4 a4 ,
21
This has the effect of assigning the value 0 to free variables that correspond to the crossed out
non-pivot columns.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 103

and if so, to find such xi ’s. Theabove

 is equivalent to solving the matrix-vector
equation Ax = d. Since A d ∼ A′ d′ , we can “read off” the solutions of


the matrix-vector equation Ax = d (if they exist) from the matrix

 
1 2 0 1 0
 ′  0 0 1 2 0 
A d′

=   0 0 0 0 0 .


0 0 0 0 1
The rightmost column (the one to the right of the vertical dotted line) is a pivot
column, and it follows that the matrix-vector equation Ax = d is inconsistent.
Consequently, the vector d is not a linear combination of the vectors a1 , a2 , a3 , a4 ,
i.e. d ∈
/ Span(a1 , a2 , a3 , a4 ).
Remark: The matrix A′ d′ is not in reduced row echelon form, but this does
 

not matter. The important point is that we have a row of the form 0 0 0 0 ■ ,
where ■ is non-zero. Since the equation 0 = ■ is inconsistent (whenever ■ is
non-zero), we see that our matrix-vector equation Ax = d is inconsistent, and
consequently, d ∈/ Span(a1 , a2 , a3 , a4 ).

Vector e. We need to check whether there exist scalars x1 , x2 , x3 , x4 ∈ Z3 such that

e = x1 a1 + x2 a2 + x3 a3 + x4 a4 ,
and if so, to find such xi ’s. The above
 is equivalent to solving the matrix-vector
equation Ax = e. Since A e ∼ A′ e′ , we can “read off” the solutions of


the matrix-vector equation Ax = e (if they exist) from the matrix

 
1 2 0 1 1
 ′  0 0 1 2 1 
A e′

=   0 0 0 0 0 


0 0 0 0 0
We see that our equation Ax = e is consistent, and that the general solution is
   
x1 s + 2t + 1
 x2   s 
x =   x3  =
  , where s, t ∈ Z3 .
 t+1 
x4 t
We only need one solution, and so we set s = t = 0,22 which yields
   
x1 1
 x2   0 
x =   x3  =  1  .
  

x4 0
22
We could choose other values for s and t (for example, s = 2 and t = 1), but it is simplest to
choose s = t = 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 104

So, e is indeed a linear combination of the vectors a1 , a2 , a3 , a4 , that is, e ∈

Span(a1 , a2 , a3 , a4 ), and we have that

e = 1a1 + 0a2 + 1a3 + 0a4 = a1 + a3 .

Remark: As  in the case

 of the vector b, we could also simply cross out the non-pivot
columns of A′ e′ to the left of the vertical dotted line, and read off the (unique)
solution that remains after we eliminated those columns. Again, we must keep track
of which xi corresponds to which column.
 
1 2 0 1 1
 0 0 1 2 1 
A′ ′
 
e = 
 0

0 0 0 0 
0 0 0 0 0
↑ ↑
x1 x3

We now read off x1 = 1 and x3 = 1, and we get e = a1 + a3 . Because we crossed out

some non-pivot columns (which correspond to arbitrary parameters), our solution is
not unique.

Example 1.5.6. Consider the vectors

       
1 0 −2 0
 2   1   −1   0 
 2 ,
a1 =   a2 =  2 ,

 2 ,
a3 =  
 1 ,
a4 =  

1 4 10 2
   
  
2 7 3 1
 2   12   8   2 
 3 ,
b= 
 10  ,
c= 
 11  ,
d= 
 4 ,
e= 

1 −1 13 6

with entries understood to be in R. For each of the vectors b, c, d, e, determine if

it can be expressed as a linear combination of the vectors a1 , a2 , a3 , a4 (that is, if it
belongs to Span(a1 , a2 , a3 , a4 )), and if so, express it as such a linear combination,
and explain whether your answer is unique.

Solution. Set
 
1 0 −2 0
   2 1 −1 0 
A = a1 a2 a3 a4 = 
 2
.
2 2 1 
1 4 10 2

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 105

We now form the following matrix (color coded for emphasis):

 
1 0 −2 0 2 7 3 1
   2 1 −1 0 2 12 8 2 
A b c d e =   2
.
2 2 1 3 10 11 4 
1 4 10 2 1 −1 13 6
 
We find the reduced row echelon form of the matrix A b c d e , and we
cross out any non-pivot columns to the left of the vertical dotted line.
 
1 0 −2 0 0 7 3 −1
 0 1 3 0 0 −2 2 2 
 
RREF A b c d e =  
 0 0 0 1 0 0 1 −1 
0 0 0 0 1 0 0 1

We see from the matrix above that b, e ∈ / Span(a1 , a2 , a3 , a4 ), whereas c, d ∈

Span(a1 , a2 , a3 , a4 ). Moreover, vectors c and d can be expressed as linear combina-
tions of the vectors a1 , a2 , a3 , a4 , as follows:
ˆ c = 7a1 − 2a2 ;
ˆ d = 3a1 + 2a2 + a4 .
Since we crossed out a non-pivot column (which corresponds to an arbitrary parame-
ter), the two expressions above are not unique.

1.6 The rank of a matrix

The rank of a matrix A (with entries in some field F), denoted by rank(A), is the
number of pivot columns of A. Equivalently, rank(A) is the number of pivot positions
of A, or the number of non-zero rows of any row echelon form of A. To find the rank
of a matrix, we first find some row echelon form of that matrix (e.g. by performing
the forward phase of row reduction; the backward phase is optional), and we count
the number of pivot columns (or alternatively, the number of pivot positions, or the
number of non-zero rows) of that row echelon matrix.
Example 1.6.1. Find the rank of each of the following matrices.
 
0 −3 −6 3 4 −1
(a) A =  2 1 −4 13 −4 3 , with entries understood to be in R;
2 3 0 11 −6 5
 
0 1 1 0 2
 2 1 0 1 1 
(b) B = 
 2 1 1 1
, with entries understood to be in Z3 .
1 
1 0 2 2 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 106

Solution#1. (a) In Example 1.3.9, we computed

 
1 0 −3 7 0 4
RREF(A) =  0 1 2 −1 0 3  .
0 0 0 0 1 2

The matrix RREF(A) has three pivot columns (equivalently: three pivot positions
or three non-zero rows), and so rank(A) = 3.
(b) In Example 1.3.10, we computed
 
1 0 0 2 1
 0 1 0 0 2 
RREF(B) = 
 0
.
0 1 0 0 
0 0 0 0 0

The matrix RREF(B) has three pivot columns (equivalently: three pivot positions
or three non-zero rows), and so rank(B) = 3.

Solution#2. (a) In Example 1.3.9, we saw that the matrix A is row equivalent to
the following matrix in row echelon form:
 
2 3 0 11 −6 5
 0 −2 −4 2 2 −2  .
0 0 0 0 1 2

This row echelon matrix has three pivot columns (equivalently: three pivot positions
or three non-zero rows), and so rank(A) = 3.
(b) In Example 1.3.10, we saw that the matrix B is row equivalent to the following
matrix in row echelon form:
 
1 0 2 2 1
 0 1 2 0 2 
 0 0 1 0 0 .
 

0 0 0 0 0

This row echelon matrix has three pivot columns (equivalently: three pivot positions
or three non-zero rows), and so rank(B) = 3.

Proposition 1.6.2. Row equivalent matrices (with entries in some field) have the
same rank.
Proof. Fix row equivalent matrices A and B (with entries in some field). By the
definition of rank, rank(A) is equal to the number of pivot columns of A, which is
precisely the number of pivot columns of RREF(A). Similarly, rank(B) is equal to
the number of pivot columns of RREF(B). But since A and B are row equivalent,
Corollary 1.3.8 guarantees that RREF(A) = RREF(B). So, rank(A) = rank(B).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 107

Proposition 1.6.3. Let A be an n × m matrix (with entries in some field F). Then
rank(A) ≤ min{n, m}.23

Proof. By definition, rank(A) is equal to the number of pivot columns of A, and

consequently, rank(A) is at most the number of columns of A, which is m. So,
rank(A) ≤ m.
On the other hand, rank(A) is equal to the number of non-zero rows of RREF(A),
and consequently, rank(A) is at most the number of rows of RREF(A); since A and
RREF(A) have the same number of rows, we deduce that rank(A) is at most the
number of rows of A, which is n. So, rank(A) ≤ n.

Terminology: For a field F and a matrix A ∈ Fn×m (so, A has n rows and m
columns):

ˆ if rank(A) = n, then A is said to have full row rank;24

ˆ if rank(A) = m, then A is said to have full column rank;25

ˆ if rank(A) = min{n, m}, then A is said to have full rank;26

ˆ if rank(A) < min{n, m}, then A is said to be rank-deficient.

1.6.1 Rank and the number of solutions of a matrix-vector equation

As our next theorem shows, the number of solutions of a matrix-vector equation
Ax = b can easily be determined if we know the size of the matrix A (i.e. the

number
of rows and columns of A) and we also know rank(A) and rank A b .

Theorem 1.6.4. Let F be a field, and let A ∈ Fn×m and b ∈ Fn . Then

 
rank(A) ≤ rank( A b ) ≤ rank(A) + 1.

Moreover, all the following hold:

   
(a) if rank( A b ) ̸= rank(A) (and consequently, rank( A b ) = rank(A) + 1),
then the matrix-vector equation Ax = b is inconsistent.
 
(b) if rank( A b ) = rank(A) = m, then the matrix-vector equation Ax = b has
a unique solution.
23
This means that rank(A) ≤ n (i.e. rank(A) is at most the number of rows of A) and rank(A) ≤ m
(i.e. rank(A) is at most the number of columns of A).
24
In this case, Proposition 1.6.3 guarantees that n ≤ m, i.e. the number of rows of A is no greater
than the number of columns.
25
In this case, Proposition 1.6.3 guarantees that m ≤ n, i.e. the number of columns of A is no
greater than the number of rows.
26
So, A has full rank if and only if it has full row rank or full column rank.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 108

 
(c) if rank( A b ) = rank(A) < m, then the matrix-vector equation Ax = b has
more than one solution, and more precisely,

(c.1) if the field F is finite, then the matrix-vector equation Ax = b has exactly
|F|m−rank(A) many solutions,27
(c.2) if the field F is infinite, then the matrix-vector equation Ax = b has infinitely
many solutions.
28
   

Proof. First, set U c = RREF A b , so that RREF(A)  = U . Next,
let (⋆) be the linear system whose augmented matrix is A b . Obviously, the
linear system (⋆) and the matrix-vector equation Ax = b have the same number of
solutions, and we can move back and forth between the linear system (⋆) and the
matrix-vector equations Ax = b in a naturalway. 
Let us first prove that rank(A) ≤ rank( A b ) ≤ rank(A) + 1. The pivot
columns of U c are precisely the pivot columns of U , plus possibly the rightmost
column (namely,
 the column c to the right of the vertical
 dotted line). If the rightmost
column of U c is a pivot column, then rank( A b ) = rank(A) + 1, and
otherwise, rank( A b ) = rank(A). This proves that
 
rank(A) ≤ rank( A b ) ≤ rank(A) + 1.
 
We now prove (a). Suppose that rank(A)  ̸
= rank( A b ). By what we just
showed, this  implies  that rank( A b  )
= rank(A) + 1 and that the rightmost
column of U c = RREF A b is a pivot column. It follows that the
linear system (⋆) is inconsistent, and consequently, that the matrix-vector equation
Ax = b is inconsistent. This proves (a).
It remains
 to prove
 (b) and (c), which we prove simultaneously.  Assume

that rank( A b
) = rank(A). Then the rightmost column of U c =
is not a pivot column,29 and it follows that the system (⋆) is

RREF A b
consistent. Further, exactly rank(A) many columns of U are pivot columns, and
consequently, the system (⋆) has precisely rank(A) many basic variables. The re-
maining m − rank(A) many variables of (⋆) are free variables. If rank(A) = m, it
follows that (⋆) has no free variables, and we deduce that the linear system (⋆) has a
unique solution, and consequently, that the matrix-vector equation Ax = b has a
unique solution. This proves (b). We may now assume that rank(A) < m. So, the
linear system (⋆) has at least one free variable, and therefore, it has more than one
solution. Each free variable can take any value from the field F, and the values of
the basic variables are fully determined by the values of the free variables. So, if F
is infinite, then the number of solutions of (⋆) is infinite,30 and if F is finite, then
27
As usual, |F| is the cardinality of F, i.e. the number of elements of F.
28
This is “obvious,” but it also follows from Proposition 1.3.21(b).
29
This follows from the discussion above (the second paragraph of the proof).
30
This is because each free variable can take infinitely many values, and there is at least one free
variable.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 109

(⋆) has precisely |F|m−rank(A) many solutions.31 Since the number of solutions of
the linear system (⋆) is the same as the number of solutions of the matrix-vector
equation Ax = b, this proves (c).

1.6.2 Matrices of full rank

In this subsection, we prove a couple of corollaries of Theorem 1.6.4 for matrices
of full rank (see Corollaries 1.6.5 and 1.6.6 below). By definition, a matrix of full
rank has full column rank or full row rank (possibly both). We deal with these
two cases separately. Finally, at the end of the subsection, we prove Theorem 1.6.8,
which deals with square matrices of full rank (note that such matrices have both
full column rank and full row rank).

Matrices of full column rank. In a matrix of full column rank, all columns are
pivot columns. So, the reduced row echelon form of such a matrix is of the form
 
1 0 0 ... 0 0
 0 1 0 ... 0 0 
 
 0 0 1 ... 0 0 
 .. .. .. . . .. .. 
 
 . . . . . . 
 
 0 0 0 ... 1 0 
 0 0 0 ... 0 1 ,
 
 
 0 0 0 ... 0 0 
 
 0 0 0 ... 0 0 
 
 . . . .
 .. .. .. . . ... ... 


0 0 0 ... 0 0
where the zero rows on the bottom are optional. More precisely, if we have an n × m
matrix of full column rank,32 then the reduced row echelon form of that matrix
is obtained from the identity matrix Im by adding n − m many zero rows to the
bottom.
A homogeneous matrix-vector equation is a matrix-vector equation of the form
Ax = 0. Note that such an equation is always consistent: indeed, x = 0 is a solution,
called the trivial solution.
Corollary 1.6.5. Let F be a field, and let A ∈ Fn×m . Then the following are
equivalent:
(a) rank(A) = m (i.e. A has full column rank);
31
This is because each free variable can take |F| many different values, and there are precisely
m − rank(A) many free variables.
32
Note that this means that rank(A) = m ≤ n. Indeed, since the n × m matrix A has full
column rank, we have that rank(A) = m. On the other hand, by Proposition 1.6.3, we have that
rank(A) ≤ n. So, rank(A) = m ≤ n.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 110

(b) the homogeneous matrix-vector equation Ax = 0 has only the trivial solution (i.e.
the solution x = 0);

(c) there exists some vector b ∈ Fn such that the matrix-vector equation Ax = b
has a unique solution;

(d) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has at most one
solution.

Proof. It is enough to prove the implications shown in the diagram below.

(a)
~ =⇒ (d)
w
w 
(c) ⇐= (b)

In fact, the implications “(d) =⇒ (b)” and “(b) =⇒ (c)” are obvious. It remains to
prove the implications “(c) =⇒ (a)” and “(a) =⇒ (d).”
We first prove the implication “(c) =⇒ (a).” Assume that (c) is true, and fix a
vector b ∈ Fn such that the matrix-vector equation Ax = b has a unique solution. In
particular, the matrix-vector
 equation
 Ax = b is consistent, and so Theorem 1.6.4(a)
guarantees that rank( A b ) = rank(A). Moreover, by Proposition 1.6.3 and
Theorem 1.6.4(c), we have that rank(A) = m.33 Thus, (a) holds.
It remains to prove the implication “(a) =⇒ (d).” Assume that (a) is true,
i.e. that rank(A) = m, and fix a vector b ∈ Fn . We must show  that
 the matrix-
vector equation Ax = b has at most one solution. If rank( A b ) ̸= rank(A),
then Theorem 1.6.4(a) guarantees that  the matrix-vector
 equation Ax = b has no
solutions. On the other hand, if rank( A b ) = rank(A), then since rank(A) = m,
Theorem 1.6.4(b) guarantees that the matrix-vector equation Ax = b has a unique
solution. In either case, the matrix-vector equation Ax = b has at most one solution,
i.e. (d) holds.

Matrices of full row rank. Note that matrices of full row rank are precisely those
matrices whose reduced row echelon form has no zero rows.

Corollary 1.6.6. Let F be a field, and let A ∈ Fn×m . Then the following are
equivalent:

(a) rank(A) = n (i.e. A has full row rank);

(b) for all vectors b ∈ Fn , the matrix-vector equation Ax = b is consistent.

33
Indeed, by Proposition 1.6.3, we have that rank(A) ≤ m. If rank(A) < m, then Theorem 1.6.4(c)
would imply that Ax = b has more than one solution, a contradiction. So, rank(A) = m.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 111

Proof. Suppose first that (a) holds. We must prove (b). Fix any b ∈ Fn . Then

n = rank(A) by (a)
 
≤ rank( A b ) by Theorem 1.6.4

by Proposition
  1.6.3,
≤ n since A b is an
n × (m + 1) matrix,
 
and it follows that rank( A b ) = rank(A) = n. But now Theorem 1.6.4 guaran-
tees that the matrix-vector equation Ax = b is consistent. Thus, (b) holds.
Suppose now that (a) is false; we must show that (b) is false, i.e. that there
exists some b ∈ Fn such that the matrix-vector equation Ax = b is inconsistent.
Since A is an n × m matrix and rank(A) ̸= n, Proposition 1.6.3 guarantees that
rank(A) ≤ n − 1. Now, set U := RREF(A), and let R1 , . . . , Rk be some sequence of
elementary row operations that transforms A into U , and for each i ∈ {1, . . . , k},
let Ri′ be the elementary row operation that reverses (undoes) the elementary row
operation Ri .34 Since U has n rows and r := rank(A) ≤ n − 1, we see that  the (r + 1)-

th row of U is a zero row. Then the rightmost column of the matrix U er+1
is a pivot column,35 and consequently, the matrix-vector equation U x = er+1 is
inconsistent. Now, we perform the elementary row operations Rk′ , . . . , R1′ on the
A b for some vector b ∈ Fn .
 
matrix U er+1  , and we  obtain  the matrix

Since matrices U er+1 and A b are row equivalent, the matrix-vector
equations U x = er+1 and Ax = b are equivalent. Since the matrix-vector equation
U x = er+1 is inconsistent, it follows that the matrix-vector equation Ax = b is also
inconsistent. Thus, (b) is false.

Square matrices of full rank. We now consider the special case of square matrices
of full rank. We begin with a simple proposition.

Proposition 1.6.7. Let F be a field. Then for all square matrices A ∈ Fn×n , we
have that rank(A) = n if and only if RREF(A) = In . In particular, rank(In ) = n.

Proof. In is a matrix in reduced row echelon form, and it has n pivot columns; so,
rank(In ) = n. Moreover, it is clear that In is the only reduced row echelon form
matrix in Fn×n of rank n.
Now, fix any
matrix A ∈ Fn×n . By Proposition 1.6.2, we have that rank(A) =
rank RREF(A) . Since In is the only reduced row echelon form matrix in Fn×n of
rank n, it follows that rank(A) = n if and only if RREF(A) = In .
34
See subsection 1.3.2.
35
Here, er+1 is the (r + 1)-th standard basis vector of Fn , i.e. the vector whose (r + 1)-th entry is
1, and all of whose other entries are 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 112

Proposition 1.6.7 and Corollaries 1.6.5 and 1.6.6 readily yield the following
theorem.

Theorem 1.6.8. Let F be a field, and let A ∈ Fn×n be a square matrix. Then the
following are equivalent:

(a) rank(A) = n (i.e. the square matrix A has full rank);

(b) RREF(A) = In ;

(c) the homogeneous matrix-vector equation Ax = 0 has only the trivial solution (i.e.
the solution x = 0);

(d) there exists some vector b ∈ Fn such that the matrix-vector equation Ax = b
has a unique solution;

(e) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has a unique solution;

(f ) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has at most one

solution;

(g) for all vectors b ∈ Fn , the matrix-vector equation Ax = b is consistent.

Proof. By Proposition 1.6.7, (a) and (b) are equivalent, and by Corollary 1.6.6, (a)
and (g) are equivalent. Further, Corollary 1.6.5 guarantees that (a), (c), (d), and (f)
are equivalent. Obviously, (e) implies (f). We complete the proof by showing that (a)
implies (e). Assume that (a) holds, and fix a vector b ∈ Fn . Since A is a square
matrix, (a) guarantees that A has both full column rank and full row rank. Since
A has full column rank, Corollary 1.6.5 guarantees that Ax = b has at most one
solution. On the other hand, since A has full row rank, Corollary 1.6.6 guarantees
that Ax = b is consistent, i.e. has at least one solution. It now follows that the
matrix-vector equation Ax = b has exactly one solution, i.e. (e) holds.

1.7 Matrix operations

1.7.1 Matrix addition, matrix subtraction, and scalar-matrix multi-
plication
   
Suppose that F is a field. Given matrices A = ai,j n×m
and B = bi,j n×m
in
Fn×m , and given a scalar c, we define

ˆ A + B := ai,j + bi,j n×m ;

 

ˆ A − B := ai,j − bi,j n×m ;

 

ˆ cA := cai,j .
 

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 113

Thus, we add (resp. subtract) matrices by adding (resp. subtracting) corresponding

entries, i.e.
ˆ ai,j n×m + bi,j n×m = ai,j + bi,j n×m ;
     

ˆ ai,j n×m − bi,j n×m = ai,j − bi,j n×m .

     

Similarly, we multiply a matrix by a scalar (on the left) by multiplying each entry of
the matrix by that scalar, i.e.
ˆ c ai,j n×m = cai,j n×m .
   

Notation: By convention, for a matrix A and scalar c, we write cA, but we do not
write Ac. In other words, by convention, we have “scalar times matrix,” but not
“matrix times scalar.”

1.7.2 Matrix multiplication

Let F be a field, and suppose that we are given two matrices, A ∈ Fn×m and
B ∈ Fm×p , where B = b1 . . . bp . We define
 
AB := Ab1 . . . Abp

Note that AB ∈ Fn×p .

Note that, for the product AB to be defined, the number of columns of A must
be the same as the number of rows of B. The matrix AB has the same number of
rows as A, and the same number of columns as B. Schematically, we get:

(n × m) · (m × p) = (n × p).

Example 1.7.1. Let

 
  1 1
1 2 −1
A = and B =  −2 1 ,
0 −3 1
0 −1

with entries understood to be in R. Compute AB.

Solution. We set
   
1 1
b1 =  −2  and b2 =  1  ,
0 −1
   
so that B = b1 b2 . Then AB = Ab1 Ab2 .
We compute

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 114

 
  1      
1 2 −1  −2  1 2 −1
Ab1 = = 1 + (−2) +0
0 −3 1 0 −3 1
0
       
1 −4 0 −3
= + + =
0 6 0 6

and
 
  1      
1 2 −1  1  1 2 −1
Ab2 = = 1 +1 + (−1)
0 −3 1 0 −3 1
−1
       
1 2 1 4
= + + = ,
0 −3 −1 −4

which yields  
  −3 4
AB = Ab1 Ab2 = .
6 −4

Multiplication by the identity or zero matrix. As our next proposition shows,

by multiplying a matrix by an appropriately sized identity matrix (on the left or
on the right), we obtain the original matrix. On the other hand, multiplying a
matrix by an appropriately sized zero matrix (on the left or on the right) yields an
appropriately sized zero matrix.

Proposition 1.7.2. Let F be a field, let m, n, p be positive integers, and let A ∈ Fn×m
be a matrix. Then all the following hold:

(a) In A = AIm = A;

(b) AOm×p = On×p ;

(c) Op×n A = Op×m .

Proof. Parts (b) and (c) readily follow from the appropriate
 definitions (the
 details
are left as an easy exercise). Let us prove (a). Set A = a1 . . . am . To show
that In A = A, we compute:
 
In A = In a1 . . . am

  by the definition of
= In a1 . . . In am
matrix multiplication

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 115

 
= a1 . . . am by Proposition 1.4.6

= A.

On the other hand, to show that AIm = A, we compute:

AIm = A em . . . em
 
1 m

by the definition of
Aem . . . Aem
 
= 1 m
matrix multiplication
 
= a1 . . . am by Proposition 1.4.5

= A.

This proves (a).

Another way to compute the product of two matrices. Suppose we are

 given

v1
a matrix A ∈ Fn×m and a vector v ∈ Fm . Set A = ai,j and v =  ... .
   
n×m
vm
Then by the definition of a matrix-vector product, we have that Av ∈ Fn , and
moreover, we have the following:
  
a1,1 . . . a1,k . . . a1,m v1
 .. .. .. .. ..   .. 
 .
 . . . . 

 . 
Av =  ai,1 . . . ai,k . . .
 ai,m  

 vk  
 .. .. .. .. ..   .. 
 . . . . .   . 
an,1 . . . an,k . . . an,m vm
     
a1,1 a1,k a1,m
 ..   ..   .. 
 .   .   . 
     
= v1  ai,1  + · · · + vk 
 
 ai,k
 + · · · + vm  ai,m 
  
 ..   ..   .. 
 .   .   . 
an,1 an,k an,m

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 116

 
a1,1 v1 + · · · + a1,k vk + · · · + a1,m vm
 .. 

 . 

=  ai,1 v1 + · · · + ai,k vk + · · · + ai,m vm 
 
 .. 
 . 
an,1 v1 + · · · + an,k vk + · · · + an,m vm

m
 
P
 a1,k vk 
 k=1 
 .. 
 m .
 

 P 
=  k=1 ai,k vk
 .

..
 
 
 m .
 

 P 
an,k vk
k=1

m
P
So, the i-th entry of the vector Av is ai,k vk .
k=1
Let us now consider the product of two matrices. Suppose we are
 given matrices
n×m m×p

A∈F and B ∈ F , and set A = ai,j n×m and B = bi,j m×p . The matrix
AB belongs to F n×p . We would like to compute the i, j-th entry of the matrix AB
in terms of the entries of A and B. The i, j-th entry of AB is precisely the i-th entry
of the j-th column of AB, and by the  definition
 of matrix product, the j-th column
b1,j
of AB is the vector Abj , where bj =  ...  is the j-th column of B. Using the
 

bm,j
formula for the matrix-vector product that we obtained above, we see that the i-th
m
P
entry of the vector Abj is ai,k bk,j . So, the i, j-th entry of the n × p matrix AB is
k=1
m
P
ai,k bk,j .
k=1
Here is a way to visualize the product of two matrices. To obtain the i, j-th entry
of the matrix AB, we focus on the i-th row of A and j-th column of B. We then
take the sum of the products of the corresponding entries of this row and column,
and we obtain the i, j-th entry of AB. Schematically, this is represented below. The
matrix A is on the bottom-left, the matrix B is on the top-right, and the matrix AB
is on the bottom-right (squeezed between A and B). The i-th row of A is in red, the
m
P
j-th column of B is in blue, and the i, j-th entry of AB is ai,k bk,j .
k=1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 117

 
b1,1 . . . b1,j ... b1,p
 .. .. .. .. .. 
 .
 . . . . 
 bk,1 . . . bk,j ... bk,p 
 
 .. .. .. .. .. 
 . . . . . 
 b
  m,1 . . . bm,j . . . bm,p 
a1,1 . . . a1,k . . . a1,m
 .. .. .. .. ..   
 .
 . . . . 


 m


P
 ai,1 . . . ai,k . . . ai,m




 ai,k bk,j 

 .. .. .. .. ..   k=1 
 . . . . .   
an,1 . . . an,k . . . an,m

Another way to write this is as follows:

m
 
    P
ai,j n×m
bi,j m×p
= ai,k bk,j ,
k=1 n×p

where in each of the three matrices, the expression between the square brackets is
the general form of the i, j-th entry (i.e. the entry in the i-th row and j-th column)
of the matrix in question.

Example 1.7.3. Let

   
1 0 1 0 1
A = and B = ,
1 1 1 1 0

with entries understood to be in Z2 . Compute the matrix AB.

Solution. We compute as shown below (the rows of A are color coded, as are the
columns of B).
 
1 0 1
1 1 0
   
1 0 1·1+0·1 1·0+0·1 1·1+0·0
1 1 1·1+1·1 1·0+1·1 1·1+1·0
By performing arithmetic (in Z2 ) on the entries of the matrix that we obtained, we
get:  
1 0 1
AB = .
0 1 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 118

Example 1.7.4. Let

 
1 2  
1 0 −1
A =  −1 0  and B = ,
2 3 0
2 4

with entries understood to be in R. Compute the matrix AB.

Solution. We compute as shown below.

 
1 0 −1
2 3 0
   
1 2 (1 · 1 + 2 · 2) (1 · 0 + 2 · 3) (1 · (−1) + 2 · 0)
 −1 0   ((−1) · 1 + 0 · 2) ((−1) · 0 + 0 · 3) ((−1) · (−1) + 0 · 0) 
2 4 (2 · 1 + 4 · 2) (2 · 0 + 4 · 3) (2 · (−1) + 4 · 0)

By performing arithmetic (in R) on the entries of the matrix that we obtained, we

get  
5 6 −1
AB =  −1 0 1 .
10 12 −2

1.7.3 Properties of matrix addition and multiplication

Theorem 1.7.5. For any matrices A, B, and C, and any scalars α and β, the
following hold (provided the matrices are of compatible size for the operation in
question, and the entries of our matrices and our scalars all belong to the same
field F):

(a) (α + β)A = αA + βA; (f ) A(B + C) = AB + AC;

(b) (αβ)A = α(βA) (g) (AB)C = A(BC);

(c) A + B = B + A; (h) (αA)B = α(AB);

(d) (A + B) + C = A + (B + C); (i) A(αB) = α(AB).

(e) (A + B)C = AC + BC;

Proof. Parts (a)-(d) readily follow from the definition of the relevant operations and
from the properties of addition and multiplication in the field F (listed in section 1.1).
We prove (a) to illustrate the principle; the proof of (b)-(d) is left as an exercise. Fix

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 119

in Fn×m , and fix scalars α, β ∈ F. Then

 
a matrix A = ai,j n×m
 
(α + β)A = (α + β) ai,j n×m

  by the definition of the

= (α + β)ai,j n×m scalar-matrix product

by the distributive
  property of
= αai,j + βai,j n×m multiplication over
addition in F

    by the definition of
= αai,j n×m
+ βai,j n×m matrix addition

    by the definition of the

= α ai,j n×m
+β ai,j n×m scalar-matrix product

= αA + βA.

Thus, (a) holds.

Next, we prove (e). Fix matrices A = ai,j n×m and B = bi,j n×m in Fn×m ,
   

and fix a matrix C = ci,j m×p in Fm×p . We compute:

 

     
(A + B)C = ai,j n×m
+ bi,j n×m
ci,j m×p

   
= ai,j + bi,j n×m
ci,j m×p

m
 
P
= (ai,k + bi,k )ck,j
k=1 n×p
  m  m
P  
(∗) P
= ai,k ck,j + bi,k ck,j
k=1 k=1 n×p

m m
   
P P
= ai,k ck,j + bi,k ck,j
k=1 n×p k=1 n×p

(∗∗)
= AC + BC,

where (*) follows from the fact that addition distributes over multiplication in the
field F, (**) follows from the formula for matrix multiplication that we obtained in

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 120

subsection 1.7.2, and the rest follows from the appropriate definitions. This proves
(e). The proof of (f) is similar.
We now prove (g). Fix matrices A = ai,j n ×n in Fn1 ×n2 , B = bi,j n ×n
   
1 2 2 3
in Fn2 ×n3 , and C = ci,j n ×n in Fn3 ×n4 . Clearly, both (AB)C and A(BC)
 
3 4
are matrices in Fn1 ×n4 . To prove that these two matrices are equal, it suffices to
prove that their corresponding entries are equal. So, fix indices i ∈ {1, . . . , n1 } and
j ∈ {1, . . . , n4 }. We must show that the i, j-th entry of (AB)C is equal to the i, j-th
entry of A(BC).
We first compute n2 the i, j-th n2
entry of (AB)C. The i-th
n2
 row of the n1 × n3
P P P
matrix AB is ai,k bk,1 ai,k bk,2 . . . ai,k bk,n3 . The j-th column of
k=1  k=1 k=1
c1,j
 c2,j 
the n3 × n4 matrix C is  . . So, the i, j-th entry of the n1 × n4 matrix (AB)C
 
 .  .
cn3 ,j
n3  P
P n2 
is ( ai,k bk,ℓ )cℓ,j .
ℓ=1 k=1
We now compute  of A(BC). The i-th row of the n1 × n2
the i, j-th entry
matrix A is ai,1 ai,2 . . . ai,n2 . The j-th column of the n2 × n4 matrix
 n3 
P
 b1,k ck,j 
 k=1n3

 P 
 b2,k ck,j 
BC is  k=1 . So, the i, j-th entry of the n1 × n4 matrix (AB)C is
 
 .. 
 n3 .
 

 P 
bn2 ,k ck,j
k=1
Pn2  n3
P 
ai,ℓ ( bℓ,k ck,j ) .
ℓ=1 k=1
n3  P
P n2  Pn2  n3
P 
It now remains to show that ( ai,k bk,ℓ )cℓ,j = ai,ℓ ( bℓ,k ck,j ) . For
ℓ=1 k=1 ℓ=1 k=1
this, we compute:

by the distributive
n3  P
P n2  n3  P
P n2  property of
( ai,k bk,ℓ )cℓ,j = ai,k bk,ℓ cℓ,j
ℓ=1 k=1 ℓ=1 k=1 multiplication over
addition in F

n2  P
P n3  by swapping the
= ai,k bk,ℓ cℓ,j P
k=1 ℓ=1 two ’s

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 121

by the distributive
n2 
P n3
P  property of
= ai,k ( bk,ℓ cℓ,j )
k=1 ℓ=1 multiplication over
addition in F

by swapping the
n2 
P n3
P  names of the
= ai,ℓ ( bℓ,k ck,j )
ℓ=1 k=1 dummy variables
k and ℓ,

and we obtain the equality that we needed. Thus, (AB)C = A(BC). This proves (g).
The proof of (h) and (i) is left as an exercise.

Warning: Matrix multiplication is not commutative, that is, for matrices A and B,

AB Z
= BA.


In fact, it is possible that one of AB and BA is defined, while the other one is not.
(For instance, if A ∈ F2×3 and B ∈ F3×4 , where F is some field, then AB is defined,
but BA is not.) Moreover, it is possible that both AB and BA are defined, but are
not of the same size. (For instance, if A ∈ F2×3 and B ∈ F3×2 , where F is some field,
then AB ∈ F2×2 and BA ∈ F3×3 .) Finally, it is possible that AB and BA are both
defined,
 and are of the same
 size,
 but AB = ̸ BA. Consider, for example, matrices
1 1 1 0
A= and B = , with the 1’s and 0’s understood to be in some
1 1 0 0
   
1 0 1 1
field F. Then AB = , but BA = , and we see that AB ̸= BA.
1 0 0 0

Recall that a vector is simply a matrix with one column. So, the following
corollary is a special case of Theorem 1.7.5.

Corollary 1.7.6. For any matrices A, B, vectors u, v, and w, and scalars α and
β, the following hold (provided the matrices and vectors are of compatible size for
the operation in question, and the entries of our matrices, the entries of our vectors,
and our scalars all belong to the same field F):
(a) (α + β)u = αu + βu; (f ) A(u + v) = Au + Av;
(b) (αβ)u = α(βu);
(g) (AB)u = A(Bu);
(c) u + v = v + u;
(h) (αA)u = α(Au);
(d) (u + v) + w = u + (v + w);
(i) A(αu) = α(Au).
(e) (A + B)u = Au + Bu;

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 122

1.7.4 Matrix powers

We can define powers of square matrices in a natural way, as follows. For a field F
and a square matrix A ∈ Fn×n , we define
ˆ A0 := In ;

ˆ Am+1 := Am A for all non-negative integers m.

So, by convention, we set A0 := In , and for any positive integer m, we have that

Am = A
| .{z
. . A},
m

where we did not have to indicate parentheses since, by Theorem 1.7.5(g), matrix
multiplication is associative.

1.8 The transpose of a matrix

Given a matrix A ∈ Fn×m (where F is a field), the transpose of A, denoted by AT , is
the matrix in Fm×n such that the i, j-th entry of AT is the j, i-th entry of A, for all
indices i ∈ {1, . . . , m} and j ∈ {1, . . . , n}. In other words, to form AT from A, the
columns of A (from left to right) become the rows of AT (from top to bottom), and
likewise, the rows of A (from top to bottom) become the columns of AT (from left
to right). Schematically, we have the picture below.
 
♦ ♦ ♦ ♦
   ∗ ∗ ∗ ∗ 
♦ ∗ ∗ ∗ ∗ ∗ ♢ 
 ∗ ∗ ∗ ∗ 

 ♦ ∗ ∗ ∗ ∗ ∗ ♢   
A =   ♦ ∗ ∗ ∗ ∗ ∗ ♢ 
 −→ AT =   ∗ ∗ ∗ ∗ 

 ∗ ∗ ∗ ∗ 
♦ ∗ ∗ ∗ ∗ ∗ ♢ 
 ∗ ∗ ∗ ∗ 


♢ ♢ ♢ ♢
 
  1 4
1 2 3
For example, if A = , then AT =  2 5 .
4 5 6
3 6

Column vectors as transposes of row vectors. In order to save space, we often

specify column vectors in terms of transposes of row vectors. For
 instance,
 we often
u1
 T  u2 
write something like u = u1 u2 . . . un instead of u =  . .
 
 .. 
un

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 123

Proposition 1.8.1. For any matrices A and B, and any scalar α, the following
hold (provided the matrices are of compatible size for the operation in question, and
the entries of our matrices and our scalar belong to the same field F):

(a) (AT )T = A; (c) (αA)T = αAT

(b) (A + B)T = AT + B T ; (d) (AB)T = B T AT .

n×m
Proof. Parts (a), (b), and (c)are obvious.
m×p
 Let us prove
  Fix matrices A ∈ F n×p
(d).
and B ∈ F , and set A = ai,j n×m and B = bi,j m×p . Clearly, AB ∈ F ,
and so (AB)T ∈ Fp×n . On the other hand, we have that B T ∈ Fp×m and AT ∈ Fm×n ,
and so B T AT ∈ Fp×n . So, both (AB)T and B T AT are p × n matrices with entries
in F. It remains to show that the corresponding entries of (AB)T and B T AT are
the same. Fix indices i ∈ {1, . . . , p} and j ∈ {1, . . . , n}; we will show that the i, j-th
entry of (AB)T is equal to the i, j-th entry of B T AT .
By the definition of matrix transpose, the i, j-th entry of (AB)T is equal to the
m
P
j, i-th entry of AB, which is equal to aj,k bk,i .
k=1
We now compute the i, j-th entry of B T AT . We observe that i-th row of the
matrix B T is b1,i b2,i . . . bm,i ,36 whereas the j-th column of the matrix
 
T 37
AT is aj,1 aj,2 . . . aj,m So, the i, j-th entry of the matrix B T AT is

.
m
P m
P
b1,i aj,1 + b2,i aj,2 + · · · + bm,i aj,m = bk,i aj,k = aj,k bk,i .
k=1 k=1
We have now shown that the corresponding entries of the p × n matrices (AB)T
and B T AT are the same, and we deduce that (AB)T = B T AT . This proves (d).

Remark: Proposition 1.8.1(d) and an easy induction on k readily imply that if

A1 , . . . , Ak are matrices with entries in some field F, and of sizes that are compatible
for the product A1 . . . Ak to be defined, then (A1 . . . Ak )T = ATk . . . AT1 . The details
are left as an exercise.

Recall that, by Proposition 1.4.5, if we multiply a matrix A by the i-th standard

basis vector (on the right), we obtain the i-th column of the matrix. To obtain the
i-th row of a matrix, we should multiply it on the left by the transpose of the i-th
standard basis vector (see Proposition 1.8.2 below). For example,
 
1 2
  3 4   
0 0 1 0   = 5 6 ,
| {z } 5 6
 
=eT3 7 8
36
Indeed, this is the transpose of the i-th column of B.
37
Indeed, this is the transpose of the j-th row of A.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 124

as we can easily verify by routine matrix multiplication.

Proposition 1.8.2. Let F be a field, and let

 
r1
A =  ... 
 

rn

be a matrix in Fn×m .38 Then for all i ∈ {1, . . . , n}, we have that

eTi A = ri ,

where ei is the i-th standard basis vector of Fn .

Proof. First of all, we note that AT = rT1 . . . rTn , i.e. vectors rT1 , . . . , rTn are the
 

columns of AT , appearing from left to right in AT . We will apply Proposition 1.4.5

to AT , as follows. For any index i ∈ {1, . . . , n}, we have that

eTi A = (AT ei )T by Proposition 1.8.1

by Proposition 1.4.5, since

= (rTi )T
rTi is the i-th column of AT

= ri by Proposition 1.8.1,

which is what we needed to show.

1.9 Solving matrix equations of the form AX = B and

XA = B
1.9.1 Solving matrix equations of the form AX = B
Example 1.9.1. Consider the matrices
  
1 2 3 4 5 3
A =  −1 3 1 −2  and B =  −3 1  ,
0 1 0 3 3 0

with entries understood to be in R. Solve the matrix equation AX = B.39 How many
solutions does the equation AX = B have?
38
So, r1 , . . . , rn are the rows of A, appearing in that order from top to bottom in A.
39
Note that solutions of the matrix equation AX = B are 4 × 2 real matrices.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 125

     
Solution #1. Set X = x1 x2 and B = b1 b2 . Then AX = Ax1 Ax2 ,
and so the equation AX = B is equivalent to
   
Ax1 Ax2 = b1 b2 .

So, we need to solve two matrix-vector equations, namely Ax1 = b1 and Ax2 = b2 .
We solve these two equations one by one.
First,
 we solve
 the matrix-vector equation Ax1 = b1 . We form the augmented
matrix A b1 and we row reduce to obtain its reduced row echelon form:
31 35
   
  1 2 3 4 5 1 0 0 4 4
A b1 =  −1 3 1 −2 −3  ∼  0 1 0 3 3 .
13 13
0 1 0 3 3 0 0 1 −4 −4

We now read off the solutions for x1 :

 31
− 4 s + 35

4
 
 
 −3s + 3 
 
x1 =   , where s ∈ R.
 13 13

 4 s− 4
 

 
s

We now solve the matrix-vector equation Ax2 = b2 . We form the augmented

matrix A b2 and we row reduce to obtain its reduced row echelon form:
31
   
  1 2 3 4 3 1 0 0 4 0
A b2 =  −1 3 1 −2 1  ∼  0 1 0 3 0 .
13
0 1 0 3 0 0 0 1 −4 1

We now read off the solutions for x2 :

− 31
 
4 t
 
 
 −3t 
 
x2 =   , where t ∈ R.
 13 
 t+1 
 4 
 
t

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 126

 
We now read off the general solution for X = x1 x2 :

− 31 35
− 31
 
4 s+ 4 4 t
 
 
 −3s + 3 −3t 
 
X =   , where s, t ∈ R.
 13 13 13

 4 s− 4 t + 1
 
4 
 
s t

There are two parameters (namely, s and t), and they can each take infinitely many
values (because R is infinite). So, the equation AX = B has infinitely many solutions.
Remark: Note that the parameters (namely, s and t) from the solution above are
different for different columns! This is because the equations Ax1 = b1 and Ax2 = b2
are solved independently, and so the parameter that appears in x1 is independent of
the one that appears in x2 .

Remark: Solution #1 is correct, but rather inefficient. We had to solve a separate

matrix-vector equation for each column of B,40 and each of these matrix-vector
equations involved forming an augmented matrix and finding its reduced row echelon
form. Luckily, we can do better by essentially solving these two matrix-vector
equations simultaneously.
 
Solution #2. We first form the matrix A B and row reduce to find its reduced
row echelon form.
 
  1 2 3 4 5 3
A B =  −1 3 1 −2 −3 1 
0 1 0 3 3 0

After row reducing, we obtain the following matrix (the columns to the right of the
vertical dotted line are color coded for easier reference):
31 35
 
  1 0 0 4 4 0
RREF A B =  0 1 0 3 3 0 .
13 13
0 0 1 −4 −4 1

We now read off the columns of X one by one. We read off the first column of X by
reading off the solutions of the matrix-vector equation encoded by the matrix obtained
by taking the submatrix to the left of the vertical dotted line, plus the first column
40
Since B has two columns, this translated into two matrix-vector equations. In general, if B has
m columns, we get m matrix-vector equations.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 127

 
41
to the right of the vertical dotted line (i.e. the red column) of RREF A B .
We read off the second column of X by reading off the solutions of the matrix-vector
equation encoded by the matrix obtained by taking the submatrix to the left of the
vertical dotted line, plus the second
 column to the right of the vertical dotted line

42
(i.e. the blue column) of RREF A B . The solutions are as follows:43

− 31 35
− 31
 
4 s+ 4 4 t
 
 
 −3s + 3 −3t 
 
X =   , where s, t ∈ R.
 13 13 13

 4 s− 4 4 t+1 
 
 
s t

There are two parameters (namely, s and t), and they can each take infinitely
many values (because R is infinite). So, the equation AX = B has infinitely many
solutions.

Recipe for solving matrix equations of the form AX = B. Suppose that A is

an n × m matrix and B is an n × p matrix (both with entries in some field F), and
we wish to solve the matrix equation AX = B.44 We proceed as follows:
 
1. We form the n × (m + p) matrix A B and find its reduced row echelon
form.
 

2. We check if RREF A B has a row of the form
 
0 ... 0 ∗ ... ∗ ,

where at least one of the ∗’s (to the right of the vertical dotted line) is non-zero.
41
This is the matrix in question:
31 35
 
1 0 0 4 4
 0 1 0 3 3 .
0 0 1 − 13
4
− 13
4

42
This is the matrix in question:
31
 
1 0 0 4
0
 0 1 0 3 0 .
0 0 1 − 13
4
1

43
Remember to use different parameters for different columns!
44
Note that solutions of the matrix equation AX = B are m × p matrices.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 128

(a) If such a row exists, then the matrix equation AX = B is inconsistent

(i.e. has no solutions).45
(b) If no such row exits, then the matrix equation AX = B is consistent (i.e.
has at least one solution). For each k ∈ {1, . . . , p},46 we read off the k-th
 

column of X by focusing on the part of RREF A B  to the left 
of
the vertical dotted line, plus the k-th column of RREF A B to
the right of the vertical dotted line.
ˆ If there are any free variables, remember to use different letters for the
parameters in different columns, as in the solution of Example 1.9.1.
Example 1.9.2. Consider the matrices
   
1 1 1 4 3 1 3
 1 1 1   4 3 1 3 
A =   1 2 −1 
 and B = 
 2
,
1 1 3 
1 2 −1 2 1 2 3

with entries understood to be in R. Solve the matrix equation AX = B.47 How many
solutions does the equation AX = B have?
Solution. We first form the matrix
 
1 1 1 4 3 1 3
   1 1 1 4 3 1 3 
A B =  .
 1 2 −1 2 1 1 3 
1 2 −1 2 1 2 3
After row reducing, we obtain
 
1 0 3 6 5 0 3
   0 1 −2 −2 −2 0 0 
RREF A B =   0 0
.
0 0 0 1 0 
0 0 0 0 0 0 0
 

By considering the third row of RREF A B , we see that the matrix equation
AX = B is inconsistent, i.e. it has no solutions.

Example 1.9.3. Consider the matrices

  

1 1 1 1 1 1 1
A =  1 1 0 0  and B =  0 1 1 ,
1 0 1 0 0 1 0
45
 
Indeed, suppose we got a row of the form 0 . . . 0 ∗ . . . ∗ , where the k-th ∗ to the
right of the vertical dotted line is non-zero. Then there are no solutions for the k-th column of the
matrix X, and therefore, there are no solutions for the matrix X, either.
46
Remember: p is the number of columns of B, and therefore, the number of columns of X.
47
Note that solutions of the matrix equation AX = B are 3 × 4 real matrices.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 129

with entries understood to be in Z2 . Solve the matrix equation AX = B. How many

solutions does the equation AX = B have?
Solution. We first form the matrix
 
  1 1 1 1 1 1 1
A B =  1 1 0 0 0 1 1 .
1 0 1 0 0 1 0
After row reducing, we obtain
 
   1 0 0 1 1 1 0
RREF A B =  0 1 0 1 1 0 1 .
0 0 1 1 1 0 0
We now read off the solutions:
 
t1 + 1 t2 + 1 t3
 t1 + 1 t2 t3 + 1 
X =   , where t1 , t2 , t3 ∈ Z2 .
 t1 + 1 t2 t3 
t1 t2 t3
There are three parameters (namely, t1 , t2 , t3 ), and each of them can take two values
(because |Z2 | = 2). So, the total number of solutions of the equation AX = B is
23 = 8.
Example 1.9.4. Consider the matrices
   
1 1 1 0 1 1 2
A =  0 1 2  and B =  2 1 1 0 ,
1 2 2 1 2 2 2
with entries understood to be in Z3 . Solve the matrix equation AX = B. How many
solutions does the equation AX = B have?
Solution. We first form the matrix
 
  1 1 1 0 1 1 2
A B =  0 1 2 2 1 1 0 .
1 2 2 1 2 2 2
After row reducing, we obtain
 
  1 0 0 2 0 0 2
RREF A B =  0 1 0 0 1 1 0 .
0 0 1 1 0 0 0
We now see that the equation AX = B has a unique solution, namely,
 
2 0 0 2
X =  0 1 1 0 .
1 0 0 0
(The number of solutions of the matrix equation AX = B is one.)

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 130

1.9.2 Solving matrix equations of the form XA = B

Suppose we are asked to solve a matrix equation of the form XA = B. This
equation is equivalent to the equation (XA)T = B T , which is, in turn, equivalent to
AT X T = B T .48 Using the methods from the previous section, we solve the equation
AT X T = B T for X T , and then we take the transpose of the solution(s) to obtain X.

Example 1.9.5. Consider the matrices

 
  5 5 1 −2
1 2 0 −1
A = and B =  4 3 1 −1  ,
3 1 1 0
2 4 0 −2

with entries understood to be in R. Solve the matrix equation XA = B.49 How many
solutions does the equation XA = B have?

Solution. Note that XA = B if and only if AT X T = B T . We first find all the

matrices X T that satisfy AT X T = B T , and then we take the transpose to obtain all
the matrices X that satisfy XA = B. First, we have
   
1 3 5 4 2
 2 1   5 3 4 
AT =   0 1 
 and BT =   1
.
1 0 
−1 0 −2 −1 −2

We now form the matrix

 
1 3 5 4 2
 2 1 5 3 4 
AT BT
 
= 
 0
,
1 1 1 0 
−1 0 −2 −1 −2

and by row reducing, we obtain

 
1 0 2 1 2
   0 1 1 1 0 
RREF AT BT = 
 0
.
0 0 0 0 
0 0 0 0 0

Using the matrix above, we can solve for X T . There is only one solution, namely:
 
T 2 1 2
X = .
1 1 0
48
We are using Proposition 1.8.1(d).
49
Note that solutions of the matrix equation XA = B are 3 × 2 real matrices.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 131

Thus, the equation XA = B has a unique solution, namely:

 
2 1
X =  1 1 .
2 0

(The number of solutions of the matrix equation XA = B is one.)

Example 1.9.6. Consider the matrices

 
1 1 0 1  
0 1 1 0
A =  1 0 1 1  and B = ,
1 1 1 1
0 1 1 0

with entries understood to be in Z2 . Solve the matrix equation XA = B.50 How

many solutions does the equation XA = B have?

Solution. Note that XA = B if and only if AT X T = B T . We first find all the

matrices X T that satisfy AT X T = B T , and then we take the transpose to obtain all
the matrices X that satisfy XA = B. First, we have
   
1 1 0 0 1
 1 0 1  T =  1 1 .
 
AT =   0 1 1 
 and B  1 1 
1 1 0 0 1

We now form the matrix

 
1 1 0 0 1
 1 0 1 1 1 
AT BT
 
= 
 0
,
1 1 1 1 
1 1 0 0 1

and by row reducing, we obtain:

 
1 0 1 1 0
   0 1 1 1 0 
RREF AT BT = 
 0
.
0 0 0 1 
0 0 0 0 0

By considering the third row of RREF AT B T

 

, we see that the equation
AT X T = B T has no solutions. Consequently, the original matrix equation XA = B
has no solutions either (i.e. the number of solutions of XA = B is zero).
50
Note that solutions of the matrix equation XA = B are 2 × 3 matrices with entries in Z2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 132

Example 1.9.7. Consider the matrices

 
1 1 1 1 1  
1 0 1 0 1
A =  1 2 1 2 1  and B = ,
2 3 2 3 2
3 0 3 0 3
with entries understood to be in Z5 . Solve the matrix equation XA = B.51 How
many solutions does the equation XA = B have?
Proof. Note that XA = B if and only if AT X T = B T . We first find all the matrices
X T that satisfy AT X T = B T , and then we take the transpose to obtain all the
matrices X that satisfy XA = B. First, we have
   
1 1 3 1 2
 1 2 0   0 3 
T
  T
 
A =  1 1 3 
  and B =  1 2 .

 1 2 0   0 3 
1 1 3 1 2
We now form the matrix
 
1 1 3 1 2

 1 2 0 0 3 

AT BT
 
= 
 1 1 3 1 2 ,

 1 2 0 0 3 
1 1 3 1 2
and by row reducing, we obtain:
 
1 0 1 2 1
 

 0 1 2 4 1 

RREF AT BT = 
 0 0 0 0 0 .

 0 0 0 0 0 
0 0 0 0 0
Using the matrix above, we can solve for X T :
 
4s + 2 4t + 1
X T =  3s + 4 3t + 1  , where s, t ∈ Z5 .
s t
Thus, the general solution of the matrix equation XA = B is
 
4s + 2 3s + 4 s
X = , where s, t ∈ Z5 .
4t + 1 3t + 1 t
Since we have two parameters (namely, s and t), each of which can take five
values (because |Z5 | = 5), we see that the matrix equation XA = B has 52 = 25
solutions.
51
Note that solutions of the matrix equation XA = B are 2 × 3 matrices with entries in Z5 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 133

1.10 A first look at linear functions and their matrices

In this section, we introduce “linear functions” from Fm to Fn (where F is some
field). In chapter 4, we will study linear functions in a more general setting.

1.10.1 Linear functions: definition and examples

For a field F, a function f : Fm → Fn is said to be a linear function (or a linear
transformation) if it satisfies the following two conditions (axioms):

1. for all vectors u, v ∈ Fm , we have that f (u + v) = f (u) + f (v);

2. for all vectors u ∈ Fm and scalars α ∈ F, we have that f (αu) = αf (u).

Proposition 1.10.1. Let F be a field, and let f : Fm → Fn be a linear function.

Then for all vectors v1 , . . . , vk ∈ Fm and all scalars α1 , . . . , αk ∈ F, we have that
k
P  k
P
f α i vi = αi f (vi ),
i=1 i=1

or, written in another way, that

 
f α1 v1 + · · · + αk vk = α1 f (v1 ) + · · · + αk f (vk ).

Proof. This follows from the definition of a linear function via an easy induction
on k. The details are left as an exercise.

Example 1.10.2. Determine whether the following functions are linear (and prove
your answer):

(a) the function f : R3 → R2 given by

 
 x1  
x1 − x2 + x3

f  x2  =
x1 + x2
x3

for all x1 , x2 , x3 ∈ R.

(b) the function g : Z22 → Z42 given by

 
x 1
 x   x1 + x2 
1
g =  
x2  x2 
1

for all x1 , x2 ∈ Z2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 134

(c) The function h : Z33 → Z23 given by

 
 x1  
x1 + x2

h  x2  =
x1 x2
x3

for all x1 , x2 , x3 ∈ Z3 .
Remark: To show that a function is linear, we must show that it satisfies both
axioms from the definition of a linear function; in particular, axiom 1 must hold for
all vectors u and v, and axiom 2 must hold for all vectors u and scalars α. On
the other hand, to show that a function is not linear, it is enough to show that
it fails to satisfy at least one of the axioms 1 and 2 from the definition of a linear
function. To show that a function does not satisfy axiom 1, it is enough to exhibit
one particular pair of vectors u and v for which that axiom does not hold.
Similarly, to show that a function does not satisfy axiom 2, it is enough to exhibit
one particular vector u and one particular scalar α for which axiom 2 fails.

Solution of Example 1.10.2. (a) The function f is linear. We prove this by verifying
the axioms of a linearfunction
 for the function
 f , as follows.
u1 v1
1. Fix vectors u =  u2  and v =  v2  in R3 . We must show that f (u + v) =
u3 v3
f (u) + f (v). For this, we compute:
     
 u1 v1   u1 + v1 
f (u + v) = f  u2  +  v2  = f  u2 + v2 
u3 v3 u3 + v3
 
(∗) (u1 + v1 ) − (u2 + v2 ) + (u3 + v3 )
=
(u1 + v1 ) + (u2 + v2 )
 
(u1 − u2 + u3 ) + (v1 − v2 + v3 )
=
(u1 + u2 ) + (v1 + v2 )
   
u1 − u2 + u3 v1 − v2 + v3
= +
u1 + u2 v1 + v2
  
u1   v1 
(∗∗)

= f  u2  + f  v2 
u3 v3

= f (u) + f (v),
where both (*) and (**) follow from the definition of f .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 135

 
u1
2. Fix a vector u =  u2  in R3 and a scalar α ∈ R. We must show that
u3
f (αu) = αf (u). For this, we compute
   
 u1   αu1 
f (αu) = f α  u2  = f  αu2 
u3 αu3
 
(∗) αu1 − αu2 + αu3
=
αu1 + αu2
 
α(u1 − u2 + u3 )
=
α(u1 + u2 )
 
u1 − u2 + u3
= α
u1 + u2


u1 
(∗∗)

= αf  u2 
u3

= αf (u),
where both (*) and (**) follow from the definition of f .
We have now shown that f satisfies both axioms from the definition of a linear
function. So, f is linear, as we had claimed.

(b) The function g is not linear because it does not satisfy axiom 1 of the
definition 52 To see this, we consider, for example, the vectors
  of a linear function.

1 1
u= and v = in Z22 , and we observe that
1 1
 
0
 1
 
1
  0

 0 
g(u + v) = g + = g = 
 0 ,

1 1 0
1

whereas

52
In fact, g also fails to satisfy axiom 2 (details?). However, to show that g is not linear, it is
enough to show that it fails to satisfy at least one of the two axioms.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 136

     
1 1 0
 1   1   0   0   0 
g(u) + g(v) = g +g =  +  =  .
1 1  1   1   0 
1 1 0

As we can see, g(u + v) ̸= g(u) + g(v), and we deduce that g is not linear.

(c) The function h is not linear because it does not satisfy axiom 2 of the
definition
 of a linear function. To see this, we consider, for example, the vector
1
u =  2  in Z33 and the scalar α = 2 in Z3 , and we observe that
0
   
 1   2   2+1   0 
ˆ h(αu) = h 2  2  = h  1  = = ;
2·1 2
0 0
 
 1  
1+2
    
0 0
ˆ αh(u) = 2h  2  =2 =2 = .
1·2 2 1
0

As we can see, h(αu) ̸= αh(u), and we deduce that h is not linear.

Proposition 1.10.3. Let F be a field, and let f : Fm → Fn be a linear function.

Then f (0) = 0.53

Proof. We observe that

(∗)
f (0) = f (0 · 0) = 0f (0) = 0,

where (*) follows from the fact that f is linear.54

Remark: Proposition 1.10.3 can sometimes be used to show that a function is

not linear. For example, for the function g from Example 1.10.2(b), we have that
g(0) ̸= 0, and so g is not linear. However, note that the converse of Proposition 1.10.3
fails: it is possible that a function f : Fm → Fn (where F is some field) satisfies
f (0) = 0, but that the function f is still not linear. For instance, the function h
from Example 1.10.2(c) satisfies h(0) = 0, but h is nevertheless not linear.
53
Note that in f (0) = 0, we have that 0 ∈ Fm , whereas 0 ∈ Fn . So, the two zero vectors aren’t
actually the same (unless m = n). Furthermore, 0 (from the proof of Proposition 1.10.3) is the zero
element of the field F.
54
In particular, we are using axiom 2 of the definition of a linear function.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 137

1.10.2 The images of lines under linear functions f : Rm → Rn

Suppose that f : Rm → Rn is a linear function. It turns out that the image of
any line in Rm under f is either a line in Rn or a point in Rn (technically, a set
that contains only one point/vector of Rn ; we can think of such one-point sets as
“degenerate lines”). This is one of the reasons why linear functions are called linear.
Let us give a formal proof. Lines through the origin in Rm are simply sets of the
form
Span(a) = {αa | α ∈ R},
where a is a non-zero vector in Rm . Any line in Rm is obtained by shifting a line
through the origin by some vector b ∈ Rm (if b = 0, then our line still passes though
the origin). So, consider some line

L := b + Span(a) = {b + αa | α ∈ R},

where a ̸= 0 and b are fixed vectors in Rm (this is illustrated below for the special
case of R2 ).
x2
L = b+Span(a)

Span(a)

b
a
x1

For any point b + αa (α ∈ R) on the line L, we have that

(∗) (∗∗)
f (b + αa) = f (b) + f (αa) = f (b) + αf (a)

where both (*) and (**) follow from the linearity of f , but in (*) we used axiom 1
from the definition of a linear function, and in (**) we used axiom 2. So, the image
of our line L under f , denoted by f [L], is
 
f [L] = {f (b) + αf (a) | α ∈ R} = f (b) + Span f (a) .

If f (a) ̸= 0, then f [L] is a line in Rn . On the other hand, if f (a) = 0, then

f [L] = {f (b)}, which is a one-point subset (“degenerate line”) of Rn .
We also remark that linear functions f : Rm → Rn map line segments onto line
segments (possibly degenerate ones, i.e. those that contain only one point). The
proof is similar to the above and is left as an exercise.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 138

We note, however, that not all functions f : Rm → Rn that map lines to lines (or
points) are linear. An obvious example might be a function f : Rn → Rn given by
f (x) = x + b for all x ∈ Rn , where b is a fixed non-zero vector in Rn . This function
is not linear because f (0) ̸= 0, and we know (by Proposition 1.10.3) that all linear
functions map 0 to 0. However, even if a function f : Rm → Rn maps lines to lines
(or points) and maps 0 to 0, it might still fail to be linear. For example, consider
the function g : R2 → R2 given by
 x   3 
1 x1
g = for all x1 , x2 ∈ R.
x2 0

This function is not linear,55 although it does map all lines onto either lines or points,
and it does map 0 to 0. In particular, g maps any non-vertical line in R2 onto
the x1 -axis, and it maps any vertical line onto a one-point set, as illustrated in the
picture below.
x2     x2
x1 x31
  7→  
x2 0
 

a3 
0
a x1 x1

1.10.3 Matrix transformations. The standard matrix of a linear

function
Proposition 1.10.4. Let F be a field, let A ∈ Fn×m be a matrix, and define
f : Fm → Fn by setting f (x) = Ax for all x ∈ Fm . Then f is a linear function.
Proof. By Corollary 1.7.6, the following hold:
(i) for all vectors u, v ∈ Fm , we have A(u + v) = Au + Av;
(ii) for all vectors u ∈ Fm and scalars α ∈ F, we have that A(αu) = α(Au).
But now we have the following:
1. for all vectors u, v ∈ Fm , we have that
(i)
f (u + v) = A(u + v) = Au + Av = f (u) + f (v);
55
This is “obvious,” but here is a formal proof:
  1   2  
8
 
2
 
1
  1 
g 2 = g = ̸= = 2 = 2g ,
1 2 0 0 0 1

and so g does not satisfy axiom 2 from the definition of a linear function, and it follows that g is
not linear.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 139

2. for all vectors u ∈ Fm and scalars α ∈ F, we have that

(ii)
f (αu) = A(αu) = α(Au) = αf (u).

So, f is linear.

Mappings of the form x 7→ Ax, where A is some matrix, are sometimes called
matrix transformations. By Proposition 1.10.4, all matrix transformations are linear.
Let us try to describe matrix
 transformations in a bit more detail. Suppose we are
given a matrix A = ai,j n×m in F n×m (where F is some field), and define the
function f : Fm → Fn by setting f (x) = Ax for all x ∈ Fm . But now for all vectors
T
in Fm , we have the following:

x = x1 . . . xm
  
a1,1 a1,2 . . . a1,m x1
 a2,1 a2,2 . . . a2,m  x2 
f (x) = Ax =
  
 .. .. .. ..  .. 
 . . . .  . 
an,1 an,2 . . . an,m xm
 
a1,1 x1 + a1,2 x2 + · · · + a1,m xm
 a2,1 x1 + a2,2 x2 + · · · + a2,m xm 
= .
 
 ..
 . 
an,1 x1 + an,2 x2 + · · · + an,m xm

So, our matrix transformation maps each vector x ∈ Fm to a vector in Fn , each of

whose entries is a linear combination of the entries of x, and the scalars/weights
are determined by the corresponding row of the matrix A. Note that the function f
from Example 1.10.2 has this form, whereas the functions g and h from the same
example do not.
By Proposition 1.10.4, every matrix transformation is a linear function. Inter-
estingly, a converse of sorts also holds (see Theorem 1.10.6 below). We begin with
another important theorem, which readily implies Theorem 1.10.6.

Theorem 1.10.5. Let F be a field, and let a1 , . . . , am be any vectors in Fn . Then there
exists a unique linear function f : Fm → Fn that satisfies f (e1 ) = a1 , . . . , f (em ) =
am , where e1 , . . . , em are the standard basis vectors of Fm . Moreover, this linear
function f is given by f (x) = Ax for all x ∈ Fm , where A = a1 . . . am .

Proof. Existence. Define f : Fm → Fn by setting f (x) = Ax for all x ∈ Fm . Then

f is a matrix transformation, and so by Proposition 1.10.4, it is linear. Moreover,
for all indices i ∈ {1, . . . , m}, we have that
  (∗)
f (ei ) = Aei = a1 . . . am ei = ai ,

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 140

where (*) follows from Proposition 1.4.5.

Uniqueness. Suppose that f : Fm → Fn is any linear function that satisfies
f (e1 ) = a1 , . . . , f (em ) = am . We must show that f (x) = Ax for all x ∈ Fm . Fix any
T
in Fm . Then we have that x = x1 e1 + · · · + xm em , and

vector x = x1 . . . xm
we compute:  
f (x) = f x1 e1 + · · · + xm em

(∗)
= x1 f (e1 ) + · · · + xm f (em )

(∗∗)
= x1 a1 + · · · + xm am
 
x1
(∗∗∗)   . 
= a1 . . . am  .. 
xm

= Ax,
where (*) follows from the linearity of f (and more precisely, from Proposition 1.10.1),
(**) follows from the fact that f (e1 ) = a1 , . . . , f (em ) = am , and (***) follows from
the definition of matrix-vector multiplication.

Remark: Theorem 1.10.5 essentially states that we can fully determine a linear
function f : Fm → Fn (where F is a field) by simply specifying what the standard
basis vectors of Fm get mapped to. Moreover, we can choose what the standard
basis vectors get mapped to arbitrarily (i.e. we can map them to any vectors of Fn
that we like).

As a corollary of Theorem 1.10.5, we obtain the following theorem, which essen-

tially states that all linear functions f : Fm → Fn (where F is a field) are in fact
matrix transformations.

Theorem 1.10.6. Let F be a field, and let f : Fm → Fn be a linear function. Then

there exists a unique matrix A (called the standard matrix of f ) such that for all
x ∈ Fm , we have that f (x) = Ax. Moreover, the standard matrix A of f is given by
 
A = f (e1 ) . . . f (em ) ,

where e1 , . . . , em are the standard basis vectors of Fm .

Proof. Existence. Set a1 := f (e1 ), . . . , am := f (em ) and

   
A := a1 . . . am = f (e1 ) . . . f (em ) .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 141

Then by Theorem 1.10.5, we have that f (x) = Ax for all x ∈ Fm .56 This proves
existence.
Uniqueness. Let B = b1 . . . bm be any matrix in Fn×m such that f (x) = Bx
 

for all x ∈ Fm . Then for all i ∈ {1, . . . , m}, we have that

(∗)
f (ei ) = Bei = bi

where (*) follows from Proposition 1.4.5. Consequently,

   
B = b1 . . . bm = f (e1 ) . . . f (em ) .

This proves uniqueness.

Example 1.10.7. Find the standard matrix of the linear function f : R3 → R2 given
by  
 x1  
x1 − x2 + x3

f  x2  =
x1 + x2
x3
for all x1 , x2 , x3 ∈ R. (The fact that f is linear was proven in the solution of
Example 1.10.2(a).)

Solution. The standard matrix of f is

 
  1 −1 1
A := f (e1 ) f (e2 ) f (e3 ) = .
1 1 0

 
x1
Remark: Note that for all vectors x =  x2  in R3 , we really do have
x3
 
  x1  
1 −1 1  x2  (∗) x1 − x2 + x3
Ax = = = f (x),
1 1 0 x1 + x2
x3

where (*) was obtained by matrix-vector multiplication.

For any set X, the identity function on X is the function IdX : X → X given by
IdX (x) = x for all x ∈ X. The following proposition is obvious, but useful to keep in
mind.

Proposition 1.10.8. Let F be a field. Then the identity function IdFn : Fn → Fn is

linear, and its standard matrix is the identity matrix In .
56
Indeed, f : Fm → Fn is a linear function that satisfies f (e1 ) = a1 , . . . , f (em ) = am . So, by
Theorem 1.10.5, we have that f (x) = Ax for all x ∈ Fm .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 142

Proof. Obviously, the identity function IdFn satisfies the two axioms from the defini-
tion of a linear function, and by Theorem 1.10.6, its standard matrix is
   
IdFn (e1 ) . . . IdFn (en ) = e1 . . . en = In .

Alternatively, we observe that for any vector x ∈ Fn , we have that

(∗) (∗∗)
IdFn (x) = x = In x,

where (*) follows from the definition of the identity function, and (**) follows from
Proposition 1.4.6. So, IdFn is a matrix transformation and is therefore linear (by
Proposition 1.10.4), and its standard matrix is In .

1.10.4 Checking the existence and uniqueness of linear functions

with certain specifications
Suppose that F is a field, and b1 , . . . , bk ∈ Fm and c1 , . . . , ck ∈ Fn are vectors. How
would we determine if there exists a linear function f : Fm → Fn that satisfies
f (bi ) = f (ci ) for all i ∈ {1, . . . , k}? If it is exists, how do we tell if it is unique?
Since linear functions f : Fm → Fn are precisely the matrix transformations, this
boils down to solving matrix equations of the form XA = B, which we studied in
section 1.9. We will see this in the examples below, but first, let us make some
general remarks.
A linear function with specifications of the sort described above does not always
 1   1 
exist. For example, no linear function f : R2 → R2 satisfies f = and
0 1

 2    
3
f = , since any such function would violate axiom 2 of the definition
0 2
of a linear function.57 Moreover, if a function of this type does exist, it need not
be unique. For example, there is more than one linear function g : R2 → R2 that
satisfies g(e1 ) = e2 . In fact, there are infinitely many such functions, since we can
arbitrarily choose the value of g(e2 ), as per Theorem 1.10.5.

Example 1.10.9. Prove that there exists a unique linear function f : Z32 → Z52 that
satisfies
 T   T
ˆ f 1 0 1 = 1 1 0 0 1 ,
 1   1 
57
Indeed, if f : R2 → R2 is a linear function that satisfies f = , then it also satisfies
0 1
 2    1  (∗)
 1  
1
 
2

f = f 2 = 2f = 2 = ,
0 0 0 1 2

where (*) follows from the linearity of f .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 143

 T  T
ˆ f

1 1 1 = 1 1 1 1 1 ,
 T  T
ˆ f

0 1 1 = 1 1 1 1 1 ,

and find its standard matrix.

Solution. To simplify notation, we set

     
1 1 0
b1 := 0 ,
  b2 := 1  ,
 b3 :=  1  ,
1 1 1
     
1 1 1

 1 


 1 


 1 

c1 := 
 0 ,
 c2 := 
 1 ,
 c3 := 
 1 .

 0   1   1 
1 1 1

We need to prove that there exists a unique linear function f : Z32 → Z52 that satisfies
f (bi ) = ci for all i ∈ {1, 2, 3}. This is equivalent to proving that there exists a unique
matrix A ∈ Z5×3 2 (the standard matrix of f ) such that Abi = ci for all i ∈ {1, 2, 3}.
So, we are looking for the matrix A ∈ Z5×3 2 that satisfies
Ab1 = c1 , Ab2 = c2 , Ab3 = c3 .
This is equivalent to
   
A b1 b2 b3 = c1 c2 c3 ,
| {z } | {z }
=:B =:C

in which the matrix A is the unknown (and matrices B and C are known). So, we
proceed as in subsection 1.9.2. We take the transpose of both sides of the equation
above to obtain B T AT = C T , we form the matrix
 T
b1 cT1
  
 T 1 0 1 1 1 0 0 1
CT =  bT2 cT2  =  1 1 1 1 1 1 1 1  ,

B
bT3 cT3 0 1 1 1 1 1 1 1
and we row reduce to obtain
 
  1 0 0 0 0 0 0 0
RREF BT CT =  0 1 0 0 0 1 1 0 .
0 0 1 1 1 0 0 1
We now see that the equation B T AT = C T has a unique solution for AT , namely,
 
0 0 0 0 0
AT =  0 0 1 1 0 .
1 1 0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 144

So, the equation AB = C has a unique solution for A, namely,

 
0 0 1
 0 0 1 
 
A =   0 1 0 .

 0 1 0 
0 0 1

We have now shown that there exists a unique linear function f : Z32 → Z53 that
satisfies f (bi ) = ci for all i ∈ {1, 2, 3}, and that its standard matrix is the matrix A
above. (The existence and uniqueness of f follow from the existence and uniqueness
of A.)

Remark: Now that we have computed the standard matrix A of f , we can easily
T
in Z32 , we

compute a formula for f , as follows. For all vectors x = x1 x2 x3
have:    
0 0 1   x3
 0 0 1  x1  x3 
  x2  (∗)
   
f (x) = Ax =   0 1 0  =  x2  ,
 
 0 1 0  x3  x2 
0 0 1 x3
where (*) was obtained via standard matrix-vector multiplication.

Example 1.10.10. Determine if there exists a linear function f : Z33 → Z23 that
satisfies all the following:
 
 1   1 
ˆ f  2  = ;
1
0
 
 2   0 
ˆ f  1  = ;
1
1
 
 1   2 
ˆ f  2  = ;
0
1
 
 0   1 
ˆ f  0  = .
2
1
If such a linear function f exists, determine if it is unique.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 145

Solution. To simplify notation, we set

       
1 2 1 0
b1 :=  2  , b2 :=  1  , b3 :=  2  , b4 :=  0  ,
0 1 1 1
       
1 0 2 1
c1 := , c2 := , c3 := , c4 := .
1 1 0 2

We need to check whether there exists a linear function f : Z33 → Z23 that satisfies
f (bi ) = ci for all i ∈ {1, 2, 3, 4}, and if so, whether it is unique. This is equivalent
to determining whether there exists a matrix A ∈ Z2×3 3 (the standard matrix of f )
such that Abi = ci for all i ∈ {1, 2, 3, 4}. So, we have a system of four equations (in
which the unknown is the matrix A):

Ab1 = c1 , Ab2 = c2 , Ab3 = c3 , Ab4 = c4 .

This is equivalent to the equation

   
A b1 b2 b3 b4 = c1 c2 c3 c4 ,
| {z } | {z }
=:B =:C

in which the matrix A is the unknown (and matrices B and C are known). We
proceed as in subsection 1.9.2. We take the transpose of both sides of the equation
above to obtain B T AT = C T , we form the matrix
 T
b1 cT1
  
1 2 0 1 1
 T  bT2 cT2   2 1 1 0 1 
CT

B =   bT3 cT3  =  1 2 1 2 0  ,
  

bT4 cT4 0 0 1 1 2

and we row reduce to obtain

 
1 2 0 1 1
   0 0 1 1 2 
RREF BT CT = 
 0
.
0 0 0 0 
0 0 0 0 0

We now see that the equation B T AT = C T is consistent, and that the following is
the general solution for AT :
 
t1 + 1 t2 + 1
AT =  t1 t2  , where t1 , t2 ∈ Z3 .
1 2

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 146

By taking the transpose of the matrix above, we obtain the general solution of the
equation AB = C:
 
t1 + 1 t1 1
A = , where t1 , t2 ∈ Z3 .
t2 + 1 t2 2
So, the equation AB = C (with the unknown A) has a solution, but because of the
two parameters (namely, t1 and t2 ), it is not unique. It follows that there exists a
linear function f : Z33 → Z23 that satisfies f (bi ) = ci for all i ∈ {1, 2, 3, 4}, but such a
linear function f is not unique.

Example 1.10.11. Determine if there exists a linear function f : R3 → R2 that

satisfies all the following:
 
1  
2
ˆ f ( −2 ) = ;
−3
2
 
2  
4
ˆ f ( −4 ) =
  ;
−6
4
 
0  
−2
ˆ f ( −1 ) =
  ;
−1
0
 
−2  
−2
ˆ f ( 7 ) = .
7
−4
If such a linear function f exists, determine if it is unique.
Solution. To simplify notation, we set
       
1 2 0 −2
b1 :=  −2  , b2 :=  −4  , b3 :=  −1  , b4 :=  7 
2 4 0 −4
       
2 4 −2 −2
c1 := , c2 := , c3 := , c4 := .
−3 −6 −1 7

We need to check whether there exists a linear function f : R3 → R2 that satisfies

f (bi ) = ci for all i ∈ {1, 2, 3, 4}, and if so, whether it is unique. This is equivalent
to determining whether there exists a matrix A ∈ R2×3 (the standard matrix of f )
such that Abi = ci for all i ∈ {1, 2, 3, 4}. So, we have a system of four equations (in
which the unknown is the matrix A):
Ab1 = c1 , Ab2 = c2 , Ab3 = c3 , Ab4 = c4 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 147

This is equivalent to the equation

   
A b1 b2 b3 b4 = c1 c2 c3 c4 ,
| {z } | {z }
=:B =:C

in which the matrix A is the unknown (and matrices B and C are known). We
proceed as in subsection 1.9.2. We take the transpose of both sides of the equation
above to obtain B T AT = C T , we form the matrix
 T
b1 cT1
  
1 −2 2 2 −3
 T  bT2 cT2   2 −4 4 4 −6 
CT

B =   bT3 cT3  =  0 −1
  ,
0 −2 −1 
bT4 cT4 −2 7 −4 −2 7

and we row reduce to obtain

 
1 0 2 0 −4
   0 1 0 0 0 
RREF BT CT = 
 0
.
0 0 1 1/2 
0 0 0 0 0

We see from the third row of the matrix above that the equation AB = C (where
A is the unknown) is inconsistent. Therefore, there does not exist a linear function
f : R3 → R2 that satisfies the property that f (bi ) = ci for all i ∈ {1, 2, 3, 4}.

Example 1.10.12. Determine if there exists a linear function f : Z43 → Z33 that
satisfies all the following:
 T   T
ˆ f 1 2 1 2 = 1 1 0 ;
 T  T
ˆ f

2 2 2 2 = 2 0 1 ;
 T  T
ˆ f

1 0 1 0 = 1 2 1 ;
 T  T
ˆ f

0 1 0 1 = 0 1 1 ;
 T  T
ˆ f

1 1 0 1 = 0 0 0 ;
 T  T
ˆ f

0 0 1 1 = 0 1 0 .

If such a linear function f exists, determine if it is unique.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 148

Solution. To simply notation, we set

     
1 2 1
 2   2   0 
b1 := 
 1 ,
 b2 := 
 2
,
 b3 := 
 1 ,


2 2 0
     
0 1 0
 1   1   0 
b4 := 
 0 ,
 b5 := 
 0 ,
 b6 := 
 1 ,


1 1 1

and we further set

     
1 2 1
c1 :=  1  , c2 :=  0  , c3 :=  2  ,
0 1 1
     
0 0 0
c4 :=  1  , c5 :=  0  , c6 :=  1  .
1 0 0

We need to check whether there exists a linear function f : Z43 → Z33 that satisfies
f (bi ) = ci for all i ∈ {1, . . . , 6}, and if so, whether it is unique. This is equivalent
to determining whether there exists a matrix A ∈ Z3×4 3 (the standard matrix of f )
such that Abi = ci for all i ∈ {1, . . . , 6}. So, we have a system of six equations (in
which the unknown is the matrix A):

Ab1 = c1 , Ab2 = c2 , Ab3 = c3 , Ab4 = c4 , Ab5 = c5 , Ab6 = c6 .

This is equivalent to the equation

   
A b1 b2 b3 b4 b5 b6 = c1 c2 c3 c4 c5 c6 ,
| {z } | {z }
=:B =:C

in which the matrix A is the unknown (and matrices B and C are known). We
proceed as in subsection 1.9.2. We take the transpose of both sides of the equation
above to obtain B T AT = C T , we form the matrix
 T
b1 cT1
  
1 2 1 2 1 1 0
 bT2 cT2   2 2 2 2 2 0 1 
 T T
  
 T T
  b3 c3   1 0 1 0 1 2 1 
B C =  T

T  =  0 1 0 1 0 1 1 ,
  
 b4 c4   
 bT cT   1 1 0 1 0 0 0 
5 5
bT6 cT6 0 0 1 1 0 1 0

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 149

and by row reducing, we obtain

 
1 0 0 0 0 2 2

 0 1 0 0 1 0 0 

 0 0 1 0 1 0 2 
BT CT
 

RREF =  .

 0 0 0 1 2 1 1 

 0 0 0 0 0 0 0 
0 0 0 0 0 0 0

So, the equation B T AT = C T has a unique solution for AT , namely,

 
0 2 2
 1 0 0 
AT =   1
,
0 2 
2 1 1

and we deduce that the equation AB = C has a unique solution for A, namely,
 
0 1 1 2
A =  2 0 0 1 .
2 0 2 1

It now follows that there exists a unique linear function f : Z43 → Z33 such that
f (bi ) = ci for all i ∈ {1, . . . , 6}, and moreover, the standard matrix of f is the
matrix A above.
Remark: Now that we have computed the standard matrix A of f , we can easy
compute a formula for f :
 
  x1  
0 1 1 2  x2 + x3 + 2x4
x2 

f (x) = Ax =  2 0 0 1    x3  =
 2x1 + x4 .
2 0 2 1 2x1 + 2x3 + x4
x4
T
in Z43 .

for all vectors x = x1 x2 x3 x4

1.10.5 Some geometric examples

In this subsection, we consider some linear functions f : R2 → R2 that have a
nice geometric interpretation, and we find their standard matrices. We will not
formally prove that these functions are all linear. To convince yourself that they
are linear, think about what happens geometrically to sums and scalar multiples of
vectors under these functions. We also note that the functions that we consider in
this subsection have higher-dimensional analogs, which you can try to think about.
However, we will not discuss higher dimensions in this subsection. In what follows,
e1 and e2 are the standard basis vectors of R2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 150

Rotation. The function rθ : R2 → R2 that rotates each vector about the origin
counterclockwise by the angle θ (see the picture below) is linear, and its standard
matrix is  
  cos θ − sin θ
rθ (e1 ) rθ (e2 ) = .
sin θ cos θ
x2

rθ (u)

u
θ
x1

Note that rotating by the angle θ clockwise is the same as rotating by the angle
−θ counterclockwise (which is why it is enough to consider only counterclockwise
rotation, as long as we allow negative angles as well).

Orthogonal projection. Given a line L in R2 that passes through the origin, the
orthogonal projection projL : R2 → R2 onto L (see the picture below) is linear.
x2
L
projL(u)

x1

We cannot yet compute the standard matrix of orthogonal projection onto an arbitrary
line through the origin; we will be able to do so only after we have developed a lot
more theory (see Corollary 6.6.4). However, we can already compute this matrix in
some special cases. Consider the projection projx1 : R2 → R2 onto the x1 -axis and
the projection projx2 : R2 → R2 onto the x2 -axis (illustrated below). Note that for a
     
u1 2 u1 0
vector u = in R , we have projx1 (u) = and projx2 (u) = .
u2 0 u2
x2

projx2 (u)
u

projx1 (u) x1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 151

The standard matrix of projx1 is

 
    1 0
projx1 (e1 ) projx1 (e2 ) = e1 0 = ,
0 0

and the standard matrix of projx2 is

 
    0 0
projx2 (e1 ) projx2 (e2 ) = 0 e2 = .
0 1

Reflection. Given a line L in R2 that passes through the origin, the reflection
refL : R2 → R2 about the line L (see the picture below) is linear.
x2
refL(u)
L

x1

As in the case of orthogonal projections, we cannot yet compute the standard matrix
of the reflection about an arbitrary line through the origin; we will only be able to
do so once we have developed a lot more theory (see subsection 6.8.3). However,
we can already compute this matrix in some special cases. Consider the reflection
refx1 : R2 → R2 about the x1 -axis and the reflection refx2 : R2→ R2 about the
u1
x2 -axis (illustrated below). Note that for a vector u = in R2 , we have
u2
   
u1 −u1
refx1 (u) = and refx2 (u) = .
−u2 u2
x2

u
refx2 (u)
x1

refx1 (u)

The standard matrix of refx1 is

 
    1 0
refx1 (e1 ) refx1 (e2 ) = e1 −e2 = ,
0 −1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 152

and the standard matrix of refx2 is

 
    −1 0
refx2 (e1 ) refx2 (e2 ) = −e1 e2 = .
0 1

Scaling. Given a scalar α ∈ R, the function that scales each vector in R2 by α (see
the picture below) is linear. The standard matrix of this linear function is
 
  α 0
αe1 αe2 = .
0 α
x2 x2
scaling by α
αu2
u2 αu
u
u1 x1 αu1 x1

(If α = 0, then scaling by α is the same as mapping each vector to the origin.)
Horizontal Shear.  A horizontal
 shear in R2 is a mapping
 from
 R2 to R2 given by
1 k u1 u1 + ku2
the formula u 7→ u, i.e. by the formula 7→ , where k
0 1 u2 u2
is a fixed real constant. This mapping has the effect of horizontally tilting objects
in the coordinate plane (while keeping the vertical component unchanged). This is
illustrated below for the cases when k = 1 and k = −1.
 
x2 1 1 x2
u 7→  u
0 1
 
 
−1 
 
1  0 
2

 
1
  
     
1 1 1

   
x1 x1
−1  1
 
  −2   

−1
 
−1
 
0
−1
 

−1
 

 
x2 1 −1  x2
u 7→  u
0 1
 
0
   
 −1   1 
 
1
 
−2 
   
1 1 
 
1

   
x1 x1
 −1   1
−1 −1
     
0
 
  2
−1
 
−1
 

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 153

Vertical Shear. A vertical

 shear in R2 is a mapping
  from
2 2
 R to R given by the
1 0 u1 u1
formula u 7→ u, i.e. by the formula 7→ , where k is
k 1 u2 ku1 + u2
a fixed real constant. This mapping has the effect of vertically tilting objects in
the coordinate plane (while keeping the horizontal component unchanged). This is
illustrated below for the cases when k = 1 and k = −1.
 
x2 1 0 x2
u 7→  u
 
1 1  1
 
2
   
 −1   1  

1
 
1

 −1 
 
0

x1   x1
1
   
 −1   1 
 

−1
 
−1

0
 
 −1 
−2
 

 
x2 1 0 x2
u 7→  u
−1 1
 
    −1 
−1  1

 

 

  2 
1

1 1 
 
0

x1   x1
    −1 
−1  1 
 
 
0
−1 −1
   

 
 1
−2
 

1.10.6 Making new linear functions out of old ones

Given functions f : A → B and g : B → C (where A, B, and C are sets), we define
the composition of
functions g and f to be the function g ◦ f : A → C given by
(g ◦ f )(a) = g f (a) for all a ∈ A (see the diagram below).
g◦f

f g
A B C
Proposition 1.10.13. Let F be a field. Then all the following hold:
(a) for all linear functions f, g : Fm → Fn , the function f + g is linear,58 and
moreover, if A and B (both in Fn×m ) are the standard matrices of f and g,
respectively, then A + B is the standard matrix of f + g;
58
As usual, the function f + g : Fm → Fn is defined by (f + g)(u) = f (u) + g(u) for all u ∈ Fm .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 154

(b) for all linear functions f : Fm → Fn and scalars α ∈ F, the function αf is

linear,59 and moreover, if A ∈ Fn×m is the standard matrix of f , then αA is the
standard matrix of αf ;

(c) for all linear functions f : Fp → Fm and g : Fm → Fn , the function g ◦ f is

linear,60 and moreover, if A ∈ Fm×p and B ∈ Fn×m are the standard matrices
of f and g, respectively, then BA is the standard matrix of g ◦ f .61
g ◦ f , BA

f, A g, B
Fp Fm Fn

Proof. We prove (c). Parts (a) and (b) are left as an exercise. Fix linear functions
f : Fp → Fm and g : Fm → Fn . Let A ∈ Fm×p be the standard matrix of f , and let
B ∈ Fn×m be the standard matrix of g. Then for any u ∈ Fp , we have that
(∗) (∗∗) (∗∗∗)
(g ◦ f )(u) = g(f (u)) = g(Au) = B(Au) = (BA)u,

where (*) follows from the fact that A is the standard matrix of f , (**) follows from the
fact that B is the standard matrix of g, and (***) follows from Corollary 1.7.6(g). We
have now shown that g◦f is a matrix transformation, and so (by Proposition 1.10.4) it
is linear. Moreover, since (by the calculation above) we have that (g ◦ f )(u) = (BA)u
for all vectors u ∈ Fp , we see that BA is the standard matrix of g ◦ f .

Example 1.10.14.
(a) Find the standard matrix of the linear function f1 : R2 → R2 that first reflects
about the x1 -axis and then rotates about the origin counterclockwise by 90◦ .

(b) Find the standard matrix of the linear function f2 : R2 → R2 that first rotates
about the origin counterclockwise by 90◦ and then reflects about the x1 -axis.
You may assume that f1 and f2 are indeed linear.
Solution. We solve the problem in two ways: first, by checking what the linear
functions f1 and f2 map the standard basis vectors to, and second, by multiplying
matrices as in Proposition 1.10.13(c).
Computing directly. (a) We observe that f1 (e1 ) = e2 and f1 (e2 ) = e1 . Conse-
quently, the standard matrix of f1 is
 
    0 1
f1 (e1 ) f1 (e2 ) = e2 e1 = .
1 0
59
As usual, the function αf : Fm → Fn is defined by (αf )(u) = α(f (u)) for all u ∈ Fm .
60
As usual, the function g ◦ f : Fp → Fn is defined by (g ◦ f )(u) = g(f (u)) for all u ∈ Fp .
61
Note that BA ∈ Fn×p .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 155

(b) We observe that f2 (e1 ) = −e2 and f2 (e2 ) = −e1 . Consequently, the standard
matrix of f1 is
 
    0 −1
f2 (e1 ) f2 (e2 ) = −e2 −e1 = .
−1 0

Computing via matrix multiplication. We use Proposition 1.10.13(c). First, we

note that the standard matrix of refx1 : R2 → R2 , the reflection about the x1 -axis, is
 
    1 0
A = refx1 (e1 ) refx1 (e2 ) = e1 −e2 = ,
0 −1

whereas the standard matrix of r90◦ : R2 → R2 , the counterclockwise rotation by 90◦

about the origin, is
 
    0 −1
B = r90 (e1 ) r90 (e2 )
◦ ◦ = e2 −e1 = .
1 0

Note that
f1 = r90◦ ◦ refx1 and f2 = refx1 ◦ r90◦ .
So, by Proposition 1.10.13(c), the standard matrix of f1 is
    
0 −1 1 0 0 1
BA = = ,
1 0 0 −1 1 0

whereas by the standard matrix of f2 is

    
1 0 0 −1 0 −1
AB = = .
0 −1 1 0 −1 0

Remark: Our two methods produced the same final answer, as expected.

1.10.7 One-to-one and onto functions. Bijections and their inverses

For a set X, we define the function IdX : X → X by setting IdX (x) = x for all
x ∈ X. IdX is called the identity function on X.
A function f : A → B is said to be

ˆ one-to-one (or injective, or an injection) if for all a1 , a2 ∈ A such that a1 ̸= a2 ,

we have f (a1 ) ̸= f (a2 );62
62
Equivalently, f : A → B is one-to-one if for all a1 , a2 ∈ A such that f (a1 ) = f (a2 ), we have that
a1 = a2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 156

ˆ onto (or surjective, or a surjection) if for all b ∈ B, there exists some a ∈ A

such that f (a) = b;

ˆ bijective or a bijection if it is both one-to-one and onto.

Proposition 1.10.15. Let f : A → B be a function. Then the following are
equivalent:
(a) f is a bijection;

(b) there exists some function g : B → A such that g ◦ f = IdA and f ◦ g = IdB .
Proof. Suppose first that (a) holds. Then for all b ∈ B, there exists a unique a ∈ A
such that f (a) = b.63 We now define g : B → A by, for each b ∈ B, letting g(b) be
the unique a ∈ A such that f (a) = b. Then clearly, g ◦ f = IdA and f ◦ g = IdB .64
Suppose now that (b) holds, and fix a function g : B → A such that g ◦ f = IdA
and f ◦ g = IdB . We first show that f is one-to-one. Fix a1 , a2 ∈ A such that
f (a1 ) = f (a2 ). Then

a1 = IdA (a1 )

= (g ◦ f )(a1 ) because g ◦ f = IdA

= g f (a1 )


= g f (a2 ) because f (a1 ) = f (a2 )

= (g ◦ f )(a2 )

= IdA (a2 ) because g ◦ f = IdA

= a2 .

So, f is one-to-one. We now show that f is onto. Fix b ∈ B, and set a := g(b). Then

f (a) = f (g(b)) = (f ◦ g)(b) = IdB (b) = b.

So, f is onto. We have now shown that f is both one-to-one and onto, and so f is a
bijection, i.e. (a) holds.

Proposition 1.10.16. Let f : A → B be a bijection. Then there exists a unique

function g : B → A such that g ◦ f = IdA and f ◦ g = IdB .
63
The existence of such an a follows from the fact that f is onto, and the uniqueness of a follows
from the fact that f is one-to-one.
64
Indeed, for all a ∈ A, we have that (g ◦ f )(a) = g(f (a)) = a. On the other hand, fix b ∈ B, and
let a be the unique element of A such that f (a) = b; then (f ◦ g)(b) = f (g(b)) = f (a) = b.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 157

Proof. The existence of g follows immediately from Proposition 1.10.15. It remains

to prove uniqueness. So, suppose that functions g1 , g2 : B → A satisfy
ˆ g1 ◦ f = IdA and f ◦ g1 = IdB ;

ˆ g2 ◦ f = IdA and f ◦ g2 = IdB .

We must show that g1 = g2 . Fix b ∈ B. Since f is onto, there exists some a ∈ A
such that f (a) = b. We now have that

g1 (b) = g1 (f (a)) because f (a) = b

= (g1 ◦ f )(a)

= IdA (a) because g1 ◦ f = IdA

= (g2 ◦ f )(a) because g2 ◦ f = IdA

= g2 (f (a))

= g2 (b) because f (a) = b.

So, g1 = g2 .

Terminology/Notation: If f : A → B is a bijection, then the unique function

g : B → A that satisfies g ◦ f = IdA and f ◦ g = IdB (i.e. the function g from
Proposition 1.10.16) is called the inverse of f and is denoted by f −1 . Note that this
means that:
ˆ f −1 ◦ f = IdA ;

ˆ f ◦ f −1 = IdB ;

ˆ for all a ∈ A and b ∈ B, we have that b = f (a) if and only if a = f −1 (b).

Note that the inverse of a bijection is also a bijection (by Proposition 1.10.15), and
moreover, (f −1 )−1 = f .

Proposition 1.10.17. Let f : A → B and g : B → C be functions. Then all the

following hold:
(a) if f and g are one-to-one, then g ◦ f is also one-to-one;

(b) if f and g are onto, then g ◦ f is also onto;

(c) if f and g are bijections, then g ◦ f is also a bijection, and moreover, (g ◦ f )−1 =
f −1 ◦ g −1 (see the diagram below).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 158

g◦f

f g
A B C
−1 −1
f g

(g ◦ f )−1 = f −1 ◦ g −1

Proof. (a) Assume that f and g are one-to-one; we must show that
g ◦ f is one-to-one.


Fix a1 , a2 ∈ A such that (g ◦ f )(a1 ) = (g ◦ f )(a2 ), i.e. g f (a1 ) = g f (a2 ) . Since
g is one-to-one, we have that f (a1 ) = f (a2 ). Since f is one-to-one, we have that
a1 = a2 . This proves that g ◦ f is one-to-one.
(b) Assume that f and g are onto; we must show that g ◦ f is onto. Fix c ∈ C.
Since g is onto, there exists some b ∈ B such that g(b) = c. Since f is onto, there
exists some a ∈ A such that f (a) = b. But now


(g ◦ f )(a) = g f (a) = g(b) = c.

This proves that g ◦ f is onto.

(c) Assume that f and g are bijections. By definition, this means that they are
one-to-one and onto, and so by (a) and (b), g ◦ f is one-to-one and onto, i.e. g ◦ f
is a bijection. It remains to show that (g ◦ f )−1 = f −1 ◦ g −1 . Fix any c ∈ C. Set
b := g −1 (c) and a := f −1 (b), so that g(b) = c and f (a) = b. Then


(g ◦ f )(a) = g f (a) = g(b) = c,

and consequently, (g ◦ f )−1 (c) = a. On the other hand,

(f −1 ◦ g −1 )(c) = f −1 g −1 (c) = f −1 (b) = a.

Thus, (g ◦ f )−1 (c) = (f −1 ◦ g −1 )(c), and we deduce that (g ◦ f )−1 = f −1 ◦ g −1 .

1.10.8 A first look at isomorphisms

As the following theorem shows, we can easily check whether a linear function is
one-to-one or onto by computing the rank of its standard matrix.

Theorem 1.10.18. Let F be a field, let f : Fm → Fn be a linear function, and let

A ∈ Fn×m be the standard matrix of f . Then both the following hold:

(a) f is one-to-one if and only if rank(A) = m (i.e. A has full column rank);

(b) f is onto if and only if rank(A) = n (i.e. A has full row rank).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 159

Proof. (a) We have the following sequence of equivalent statements:

(∗) for all b ∈ Fn , f (x) = b

f is one-to-one ⇐⇒
has at most one solution

(∗∗) for all b ∈ Fn , Ax = b

⇐⇒
has at most one solution,

(∗∗∗)
⇐⇒ rank(A) = m,

where (*) follows from the definition of a one-to-one function, (**) follows from the
fact that A is the standard matrix of f , and (***) follows from Corollary 1.6.5.
(b) We have the following sequence of equivalent statements:

(∗) for all b ∈ Fn , f (x) = b

f is onto ⇐⇒
has at least one solution

for all b ∈ Fn , Ax = b
(∗∗)
⇐⇒ has at least one solution
(i.e. Ax = b is consistent)

(∗∗∗)
⇐⇒ rank(A) = n,

where (*) follows from the definition of an onto function, (**) follows from the fact
that A is the standard matrix of f , and (***) follows from Corollary 1.6.6.

Let F be a field. A function f : Fm → Fn is an isomorphism if it is both linear

and a bijection. As Theorem 1.10.19 (below) shows, if we know the standard matrix
of a linear function, then we can easily determine whether that linear function is an
isomorphism. Moreover, Theorem 1.10.19 implies, in particular, that for a field F,
there can be no isomorphism from Fm to Fn for m ̸= n.

Theorem 1.10.19. Let F be a field, let f : Fm → Fn be a linear function, and let

A ∈ Fn×m be the standard matrix of f . Then the following are equivalent:

(a) f is an isomorphism;

(b) rank(A) = m = n (i.e. A is a square matrix of full rank).

Proof. Suppose first that (a) holds. Since f is a one-to-one linear function, The-
orem 1.10.18(a) guarantees that rank(A) = m. On the other hand, since f is an
onto linear function, Theorem 1.10.18(b) guarantees that rank(A) = n. But now
m = rank(A) = n, and (b) follows.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 160

Suppose now that (b) holds. Then by Theorem 1.10.18(a), f is one-to-one, and
by Theorem 1.10.18(b), f is onto. So, f is a bijection. Since f is also linear (by
hypothesis), we deduce that f is an isomorphism, i.e. (a) holds.

Proposition 1.10.20. Let F be a field, and let f : Fn → Fn be an isomorphism.

Then f −1 : Fn → Fn is also an isomorphism.

Proof. Since f : Fn → Fn is an isomorphism, it is, in particular, a bijection; conse-

quently, f has an inverse f −1 : Fn → Fn , which is also a bijection. So, to show that
f −1 is an isomorphism, it suffices to show that f −1 is linear.
First, fix v1 , v2 ∈ Fn . We must show that f −1 (v1 + v2 ) = f −1 (v1 ) + f −1 (v2 ).
Set u1 := f −1 (v1 ) and u2 := f −1 (v2 ), so that f (u1 ) = v1 and f (u2 ) = v2 . Then

f −1 (v1 + v2 ) = f −1 f (u1 ) + f (u2 )

f −1 f (u1 + u2 )


= because f is linear

= (f −1 ◦ f )(u1 + u2 )

= IdFn (u1 + u2 )

= u1 + u2

= f −1 (v1 ) + f −1 (v2 ).

Next, fix v ∈ Fn and α ∈ F. We must show that f −1 (αv) = αf −1 (v). Set

u := f −1 (v), so that f (u) = v. Then

f −1 (αv) = f −1 αf (u)

f −1 f (αu)


= because f is linear

= (f −1 ◦ f )(αu)

= IdFn (αu)

= αu

= αf −1 (v).

We have now proven that f −1 linear. This completes the argument.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 161

1.11 Invertible matrices

1.11.1 Invertible matrices: definition and uniqueness of inverses
A square matrix A ∈ Fn×n (where F is a field) is invertible if there exists a matrix
B ∈ Fn×n , called an inverse of A, such that AB = BA = In . A square matrix that
is not invertible is called non-invertible. As we shall see, the inverse of an invertible
matrix is in fact unique (see Proposition 1.11.1 below). In subsection 1.11.2, we will
describe a simple procedure for determining whether a square matrix is invertible,
and if so, for finding its inverse.
Terminology: Inverible matrices are also called non-singular or non-degenerate,
whereas non-invertible matrices are also called singular or degenerate. The Czech
term for an invertible matrix is “regulárnı́ matice,” and for this reason, Czech
mathematicians sometimes use the term “regular matrix” instead of “invertible
matrix”; however, this usage (“regular matrix”) is quite rare in the English speaking
world. In these notes, we will consistently use the term “invertible matrix.”

Proposition 1.11.1. Let F be a field, and let A ∈ Fn×n be an invertible matrix.

Then A has a unique inverse.

Notation: The unique inverse of A is denoted by A−1 .

Proof. Since A is invertible, it has an inverse, and we just need to show that it is
unique. So, suppose that B, C ∈ Fn×n are both inverses of A, so that AB = BA = In
and AC = CA = In . Then

B = BIn by Proposition 1.7.2

= B(AC) because AC = In

by the associativity of
= (BA)C matrix multiplication
(see Theorem 1.7.5(g))

= In C because BA = In

= C by Proposition 1.7.2.

This completes the argument.

 
1 1
Example 1.11.2. The matrix A := in R2×2 is invertible, and its inverse
0 1
 
−1 1 −1
is A := , which we can easily verify by checking that
0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 162

     
1 1 1 −1 1 −1 1 1
= I2 and = I2 .
0 1 0 1 0 1 0 1

We complete this subsection with a technical proposition whose proof is very

similar to that of Proposition 1.11.1 (above).

Proposition 1.11.3. Let F be a field, and let A, B ∈ Fn×n . Assume that A is

invertible and that AB = In or BA = In . Then A−1 = B.

Proof. We prove the proposition for the case when BA = In . The proof of the other
case (i.e. the case when AB = In ) is similar and is left as an easy exercise. Since A
is invertible, it has an inverse A−1 , and we have that AA−1 = In . We now compute:

B = BIn by Proposition 1.7.2

= B(AA−1 ) because AA−1 = In

by the associativity of
= (BA)A−1 matrix multiplication
(see Theorem 1.7.5(g))

= In A−1 because BA = In

= A−1 by Proposition 1.7.2.

This completes the argument.

Remark: Note that Proposition 1.11.3 can only be applied if we already know
that A is invertible. Once we have developed a lot more theory, we will be able to
eliminate this hypothesis and show that if A, B ∈ Fn×n are square matrices that
satisfy AB = In , then both A and B are invertible and are each other’s inverses
(see Corollary 3.3.20). However, we cannot prove this stronger statement yet, and
therefore, we cannot use it yet.

1.11.2 Computing the inverse of an invertible matrix

The following theorem (whose proof we postpone to subsection 1.11.6) gives us a
recipe for determining whether a square matrix is invertible, and if so, for finding
the inverse of that matrix. We state the theorem and give a few examples. However,
as we develop our theory in the remainder of this section, we will not rely on
Theorem 1.11.4 (in fact, we will need to develop sufficient theory in order to actually
prove this theorem).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 163

n×n be a square matrix, and set

Theorem
  F be a
field, let A ∈ F
 1.11.4. Let
U B = RREF A In , where each of U and B has n columns. Then

(a) if U = In , then A is invertible and B = A−1 ;

(b) if U ̸= In , then A is not invertible.

Example 1.11.5. Consider the following matrices.

 
1 2
(a) A = , with entries understood to be in R;
3 4
 
1 1 0
(b) B =  0 1 1 , with entries understood to be in Z2 ;
1 1 1
 
1 2 0
(c) C =  1 1 1 , with entries understood to be in Z3 .
2 0 1
For each of these three matrices, determine if the matrix is invertible, and if so, find
its inverse.

Solution. (a) We form the matrix

 
  1 2 1 0
A I2 = ,
3 4 0 1

and by row reducing, we obtain

 
  1 0 −2 1
RREF A I2 = 3 1 .
0 1 2 −2
 

The submatrix of RREF A I2 to the left of the vertical dotted line is I2 . So,
A is invertible, and its inverse is
 
−1 −2 1
A = 3 1 .
2 −2

(b) We form the matrix

 
  1 1 0 1 0 0
B I3 =  0 1 1 0 1 0 ,
1 1 1 0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 164

and by row reducing, we obtain

 
  1 0 0 0 1 1
RREF B I3 =  0 1 0 1 1 1 .
0 0 1 1 0 1
 

The submatrix of RREF B I3 to the left of the vertical dotted line is I3 . So,
B is invertible, and its inverse is
 
0 1 1
B −1 =  1 1 1 .
1 0 1

(c) We form the matrix

 
  1 2 0 1 0 0
C I3 =  1 1 1 0 1 0 ,
2 0 1 0 0 1

and by row reducing, we obtain

 
 
1 0 2 0 0 2
RREF C I3 =  0 1 2 0 1 1 .
0 0 0 1 1 2
 

The submatrix of RREF C I3 to the left of the vertical dotted line is not I3 .
So, C is not invertible.

Remark: Because it is easy to miscompute when row reducing, it is not a bad

idea to check our answers. Suppose that we have computed the inverse A−1 of an
invertible n × n matrix A (with entries in some field F). We can check if our answer
is correct by computing the matrix products AA−1 and A−1 A, and verifying that
we get In in both cases. Actually, in practice, it is more or less enough to check
that one of AA−1 = In and A−1 A = In holds. The theoretical justification for this
is given by Corollary 3.3.20 (which we cannot prove yet), but for now, the point is
that this answer checking is not a formal part of our solution/calculation: we only
do it in order to increase our own confidence that we have not miscomputed.

1.11.3 Basic properties of invertible matrices

Matrix invertibility and matrix-vector equations. Theorem 1.11.6 (below) is
one of the main reasons we care about invertible matrices. Note that it implies that
if the coefficient matrix of a linear system is invertible, then that linear system has
a unique solution.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 165

Theorem 1.11.6. Let F be a field, and let A ∈ Fn×n be an invertible matrix. Then
for all vectors b ∈ Fn , the matrix-vector equation Ax = b has a unique solution, and
that solution is A−1 b.

Proof. Fix any vector b ∈ Fn . To show that A−1 b is indeed a solution of the
matrix-vector equation Ax = b, we compute
(∗) −1 (∗∗)
A(A−1 b) = | {z })b = In b
(AA = b,
=In

where (*) follows from Corollary 1.7.6(g), and (**) follows from Proposition 1.4.6.
So far, we have proven that A−1 b is a solution of the matrix-vector equation
Ax = b. It remains to prove uniqueness. Fix any solution x0 ∈ Fn of the matrix-
vector equation Ax = b. Then Ax0 = b, and consequently, A−1 (Ax0 ) = A−1 b. We
now compute:
(∗) −1 (∗∗)
A−1 b = A−1 (Ax0 ) = | {z A})x0 = In x0
(A = x0 .
=In

where once again, (*) follows from Corollary 1.7.6(g), and (**) follows from Proposi-
tion 1.4.6. This proves that A−1 b is in fact the unique solution of the matrix-vector
equation Ax = b.

Example 1.11.7. Set

   
1 1 2
A := and b := ,
0 1 −3

with entries understood to be in R. Solve the matrix-vector equation Ax = b.

Solution. As we saw in Example 1.11.2, the matrix A is invertible, and its inverse is
 
1 −1
A−1 = .
0 1

So, by Theorem 1.11.6, the matrix-vector equation Ax = b has a unique solution,

namely     
−1 1 −1 2 5
x = A b = = .
0 1 −3 −3

Remark: We saw in subsection 1.11.2 how one can check if a square matrix (with
entries in some field) is invertible, and if so, how one can compute its inverse.
However, if we do not already know whether A is invertible (or we know that A is
invertible, but have not yet computed its inverse), then the most efficient way to

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 166

solve our matrix-vector equation Ax = b is by row reducing the augmented matrix

A b . Using the formula x = A−1 b is only efficient if we already happen to


know that A is invertible and have already computed its inverse A−1 for some reason
other than solving the equation Ax = b.

Making new invertible matrices out of old ones. We complete this subsection
by proving an easy, but important, proposition about invertible matrices and their
inverses.

Proposition 1.11.8. Let F be a field. Then all the following hold:

(a) the identity matrix In is invertible and is its own inverse (i.e. In−1 = In );

(b) if a matrix A ∈ Fn×n is invertible, then its inverse A−1 is also invertible, and
moreover, (A−1 )−1 = A;

(c) if a matrix A ∈ Fn×n is invertible, then its transpose AT is also invertible, and
moreover, (AT )−1 = (A−1 )T ;

(d) if matrices A, B ∈ Fn×n are invertible matrices, then AB is also invertible, and
moreover, (AB)−1 = B −1 A−1 ;

(e) if matrices A1 , . . . , Ak ∈ Fn×n are invertible, then the matrix A1 . . . Ak is also

invertible, and moreover, (A1 . . . Ak )−1 = A−1 −1
k . . . A1 ;

(f ) if a matrix A ∈ Fn×n is invertible, then for all non-negative integers m, the

matrix Am is also invertible, and moreover, (Am )−1 = (A−1 )m .

Proof. Part (a) follows immediately from the fact that In In = In .

Let us prove (b). Fix an invertible matrix A ∈ Fn×n . Since AA−1 = A−1 A = In ,
we see that A−1 is invertible that that its inverse is A. This proves (b).
Next, we prove (c). Fix an invertible matrix A ∈ Fn×n . Then
(∗)
AT (A−1 )T = (A−1 A)T = InT = In ,

where (*) follows from Proposition 1.8.1(d). An analogous argument shows that
(A−1 )T AT = In . So, AT is invertible and its inverse is (A−1 )T . This proves (c).
We now prove (d). Fix invertible matrices A, B ∈ Fn×n . It suffices to show
that (AB)(B −1 A−1 ) = (B −1 A−1 )(AB) = In . For this, we compute (using the
associativity of matrix multiplication):

ˆ (AB)(B −1 A−1 ) = A(BB −1 )A−1 = AIn A−1 = AA−1 = In ;

ˆ (B −1 A−1 )(AB) = B −1 (A−1 A)B = B −1 In B = B −1 B = In .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 167

This proves (d).

Part (e) follows from (d) via an easy induction on k (the details are left as an
exercise). Part (f) follows from (a) when m = 0 (this is because A0 = In for all
matrices A ∈ Fn×n ), and is a special case of (e) when m ≥ 1.

Notation: For a field F, an invertible matrix A ∈ Fn×n , and a positive integer m, we

define A−m := (A−1 )m . By Proposition 1.11.8(f), we also have that A−m = (Am )−1 ,
as we would expect. Note that this is only defined if A is invertible (and is undefined
otherwise).

1.11.4 Invertible matrices, isomorphisms, and rank

As our next theorem shows, invertible matrices are precisely the standard matrices
of isomorphisms, or equivalently, the square matrices of full rank. Recall from
Theorem 1.10.19 that if f : Fm → Fn (where F is some field) is an isomorphism, then
m = n.

Theorem 1.11.9. Let F be a field, let A ∈ Fn×n be a square matrix, and let
f : Fn → Fn be given by f (x) = Ax for all x ∈ Fn . Then f is linear and its standard
matrix is A. Furthermore, the following are equivalent:

(a) f is an isomorphism;

(b) A is invertible;

(c) RREF(A) = In ;

(d) rank(A) = n.

Moreover, in this case,65 f −1 is an isomorphism and its standard matrix is A−1 .

Proof. The function f is a matrix transformation, and so by Proposition 1.10.4, it

is linear. The fact that A is its standard matrix follows from the definition of a
standard matrix.
By Theorem 1.10.19, (a) and (d) are equivalent, and by Proposition 1.6.7, (c)
and (d) are equivalent. So, (a), (c), and (d) are equivalent. Moreover, Proposi-
tion 1.10.20 guarantees that if f is an isomorphism, then so is f −1 . It now suffices
to prove the following:

(1) if f is an isomorphism, then A is invertible, and moreover, the standard matrix

of f −1 is A−1 ;66
65
“In this case” means “if (a), (b), (c), and (d) hold,” or equivalently (since (a), (b), (c), and (d)
are equivalent): “if one of (a), (b), (c), and (d) holds.”
66
Note that (1) states that (a) implies (b), and moreover, that if (a) holds, then the standard
matrix of f −1 is A−1 .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 168

(2) if A is invertible, then f is an isomorphism.67

We first prove (1). Assume that f is an isomorphism. Then by Proposition 1.10.20,
f −1 : Fn → Fn is an isomorphism; let B ∈ Fn×n be the standard matrix of the
isomorphism f −1 . We must show that A is invertible and that B = A−1 . Since f
and f −1 are linear, Proposition 1.10.13(c) guarantees that f ◦ f −1 and f −1 ◦ f are
also linear, and moreover, that their standard matrices are AB and BA, respectively.
On the other hand, we have that f −1 ◦ f = f ◦ f −1 = IdFn , and clearly, the standard
matrix of IdFn is In .68 So, AB = BA = In . But now A is invertible and B is its
inverse, i.e. B = A−1 .
It remains to prove (2). Assume that A is invertible. We must show that f is
an isomorphism. By hypothesis, f is linear; it remains to show that f is a bijection.
Define g : Fn → Fn by setting g(u) = A−1 u for all u ∈ Fn . (So, g : Fn → Fn
is the linear function whose standard matrix is A−1 .) Our goal is to show that
f ◦ g = g ◦ f = IdFn . In view of Proposition 1.10.15, this will imply that f is a
bijection, which is what we need. But indeed, for any u ∈ Fn , we have that
ˆ (f ◦ g)(u) = f g(u) = A(A−1 u) = (AA−1 )u = In u = u;

ˆ (g ◦ f )(u) = g f (u) = A−1 (Au) = (A−1 A)u = In u = u.

This proves that f ◦ g = g ◦ f = IdFn , and it follows that f is indeed a bijection.

As an easy corollary of Proposition 1.11.8 and Theorem 1.11.9, we obtain the
following.
Corollary 1.11.10. Let F be a field, and let A ∈ Fn×n be a square matrix. Then
the following are equivalent:
(a) A is invertible;
(b) AT is invertible;
(c) rank(A) = n;
(d) rank(AT ) = n.
Proof. By Theorem 1.11.9 applied to the matrix A, we have that (a) and (c) are
equivalent. Similarly, by Theorem 1.11.9 applied to the matrix AT , we have that (b)
and (d) are equivalent. By Proposition 1.11.8(c) applied to the matrix A, we
have that (a) implies (b). On the other hand, Proposition 1.11.8(c) applied to AT
guarantees that if AT is invertible, then so is (AT )T = A, and so (b) implies (a).
This completes the argument.
Remark: By Corollary 1.11.10, a square matrix (with entries in some field) has full
rank if and only if its transpose has full rank. In fact, the rank of any matrix is equal
to the rank of its transpose (see Corollary 3.3.11), but we cannot prove this yet.
67
Note that (2) states that (b) implies (a).
68
This is obvious, but it also follows from Proposition 1.10.8.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 169

1.11.5 Elementary matrices and row reduction

An elementary matrix is any matrix obtained by performing one elementary row
operation on an identity matrix In . For an elementary row operation performed on a
matrix with n rows, the elementary matrix that corresponds to this elementary row
operation is the matrix obtained by performing that same elementary row operation
on the identity matrix In . Let us consider some examples.

1. The elementary matrix that corresponds to swapping rows 2 and 4 (“R2 ↔ R4 ”)

of a matrix with 5 rows is
 
1 0 0 0 0
 0 0 0 1 0 
 
 0 0 1 0 0 .
 
 0 1 0 0 0 
0 0 0 0 1

2. The elementary matrix that corresponds to multiplying the second row of a

matrix with three rows by a scalar α ̸= 0 (“R2 → αR2 ”) is
 
1 0 0
 0 α 0 .
0 0 1

3. The elementary matrix that corresponds to adding α times the third row to
the second row (“R2 → R2 + αR3 ”) of a matrix with three rows is
 
1 0 0
 0 1 α .
0 0 1

Proposition 1.11.11. Let F be a field, and let A ∈ Fn×m be a matrix. Then both
the following hold:

(a) if R is any elementary row operation (performed on a matrix with n rows and
with entries in F) and E is the corresponding elementary matrix, then the matrix
obtained from A by performing R on it is precisely the matrix EA;

(b) if R1 , . . . , Rk are elementary row operations (performed on a matrix with n rows

and with entries in F) and E1 , . . . , Ek ∈ Fn×n are, respectively, the corresponding
elementary matrices, then the matrix obtained from A by performing R1 , . . . , Rk
(in that order) on it is precisely the matrix Ek . . . E1 A.

Remark: In part (b), note the swapping of order between R1 , . . . , Rk on the one
hand and Ek . . . E1 on the other.

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 170

Proof. We first prove (a). Consider any elementary row operation R performed on a
matrix with n rows (and with entries in the field F). Define fR : Fn → Fn by, for
each u ∈ Fn , letting f (u) be the vector obtained by performing the elementary row
operation R on u. It is easy to see that fR is linear.69 So, fR has a standard matrix.
But clearly, the standard matrix of fR is precisely the matrix E.70
Now, fix any matrix A ∈ Fn×m , and set A = a1 . . . am . Then


  (∗)  
EA = Ea1 . . . Eam = fR (a1 ) . . . fR (am ) =: M,

where (*) follows from the fact that E is the standard matrix of fR . But obviously,
the matrix M is precisely the matrix obtained by performing the elementary row
operation R on A. This proves (a).
Part (b) follows from part (a) via an easy induction on k (the details are left as
an exercise).

Remark: Schematically (but somewhat informally), Proposition 1.11.11(b) yields

the following:
R1 R2 R3 R4 Rk
A ∼ E1 A ∼ E2 E1 A ∼ E3 E2 E1 A ∼ ... ∼ Ek . . . E3 E2 E1 A.
Proposition 1.11.12. Let F be a field. Then all the following hold:
(a) elementary matrices in Fn×n are invertible;

(b) the inverse of an elementary matrix in Fn×n is an elementary matrix in Fn×n ;

(c) a matrix A ∈ Fn×n is invertible if and only if there exist elementary matrices
E1 , . . . , Ek such that A = E1 . . . Ek (that is, a matrix is invertible if and only if
it can be written as a product of elementary matrices).
Proof. We prove (a) and (b) simultaneously. Let R be an elementary row operation
performed on a matrix with n rows (and with entries in the field F), and let E be the
elementary matrix that corresponds to R. Let R′ be the elementary row operation
that “undoes” R,71 and let E ′ be the elementary matrix that corresponds to R′ . But
now Proposition 1.11.11 guarantees that EE ′ = E ′ E = In .72 This proves that E is
invertible, and that its inverse is the elementary matrix E ′ . This proves (a) and (b).
69
Check this!
70
 
Indeed, the standard matrix of R is fR (e1 ) . . . fR (en ) , which is precisely the matrix
obtained from In by applying the elementary row operation R to it, and this matrix is precisely the
elementary matrix E.
71
See subsection 1.3.2.
72
Let us explain this in detail. By Proposition 1.11.11, applying the elementary row operation R
(resp. R′ ) to a matrix in Fn×n is the same as multiplying that matrix on the left by the elementary
matrix E (resp. E ′ ). If we apply R to the matrix In , and then apply R′ to the resulting matrix, we
obtain In back. So, if we multiply In by E on the left, and then multiply the resulting matrix by E ′
on the left, we obtain In ; so, E ′ EIn = In , and consequently, E ′ E = In . Analogously, EE ′ = In .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 171

Let us now prove (c). The fact that products of elementary matrices are invertible
follows immediately from part (a) and from the fact that (by Proposition 1.11.8(e))
products of invertible matrices are invertible. For the reverse direction, we fix an
arbitrary invertible matrix A ∈ Fn×n , and we show that A can be written as a product
of elementary matrices. Since A is invertible, Proposition 1.11.9 guarantees that
RREF(A) = In . In particular, A and In are row equivalent, and it follows that we can
transform In into A via some sequence R1 , . . . , Rk of elementary row operations. For
each index i ∈ {1, . . . , k}, let Ei ∈ Fn×n be the elementary matrix that corresponds
to the elementary row operation Ri . But then by Proposition 1.11.11(b), we have
that A = Ek . . . E1 In = Ek . . . E1 . This proves (c).

As a corollary of Propositions 1.11.11(b) and 1.11.12(c), we obtain the following

theorem.

Theorem 1.11.13. Let F be a field, and let A, B ∈ Fn×m . Then the following are
equivalent:

(a) A ∼ B;

(b) there exist elementary matrices E1 , . . . , Ek ∈ Fn×n such that B = E1 . . . Ek A;

(c) there exists an invertible matrix C ∈ Fn×n such that B = CA.

Proof. By definition, (a) is equivalent to:

(a’) B can be obtained from A via some sequence of elementary row operations.

But Proposition 1.11.11(b) guarantees that (a’) and (b) are equivalent, and Propo-
sition 1.11.12(c) guarantees that (b) and (c) are equivalent. This completes the
argument.

1.11.6 Proof of Theorem 1.11.4

We are now ready to prove Theorem 1.11.4, restated below for the reader’s conve-
nience.
n×n be a square matrix, and set
Theorem
  F be a
field, let A ∈ F
 1.11.4. Let
U B = RREF A In , where each of U and B has n columns. Then

(a) if U = In , then A is invertible and B = A−1 ;

(b) if U ̸= In , then A is not invertible.

Proof. By Theorem1.11.9, we have

 that A 
is invertible if and only if RREF(A) = In ,
and since U B = RREF A In 73
, we have that RREF(A) = U . So, if
U ̸= In , then A is not invertible; this proves (b) holds. Assume now that U = In ,
73
This is “obvious,” but it also follows from Proposition 1.3.21(b).

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 172

so
 that A is invertible. To prove (a), it now remains to show that B = A−1 . Since
74

A In and In B are row equivalent, Theorem 1.11.13 guarantees that
n×n
  
there exists an invertible
 matrix
  C ∈ F  75such that
 C A  In =
 In B
 .
But
 note that C A In = CA C . So, CA C = C A In =
In B , which in turn implies that CA = In and C = B, and consequently,
BA = In . But we already saw that A is invertible, and so Proposition 1.11.3
guarantees that A−1 = B.

1.11.7 The Invertible matrix theorem (version 1)

The following theorem, which essentially summarizes the results of this section (plus
Theorems 1.6.8 and 1.10.18), gives several equivalent characterizations of invertible
matrices. Later in these lecture notes (see subsections 3.3.7, 7.4.1, and 8.2.6), we
will add several more equivalent characterizations of invertible matrices.

Warning: The Invertible Matrix Theorem only works for square matrices. Do not
attempt to apply it to matrices that are not square (or to linear functions whose
standard matrices are not square)!

The Invertible Matrix Theorem (version 1). Let F be a field, and let A ∈ Fn×n
be a square matrix. Further, let f : Fn → Fn be given by f (x) = Ax for all x ∈ Fn .76
Then the following are equivalent:

(a) A is invertible (i.e. A has an inverse);

(b) AT is invertible;

(c) RREF(A) = In ;

= In B for some matrix B ∈ Fn×n ;

 
 
(d) RREF A In

(e) rank(A) = n;
74
     

This is because In B = U B = RREF A In  .
75

 This is “obvious,”
 but here are the details. Set A = a1 . . . an and set C =
c1 . . . cn . Then
   
C A In = C a1 . . . an e1 . . . en
 
= Ca1 ... Can Ce1 ... Cen
 
= Ca1 ... Can c1 ... cn
 
= CA C .

76
Since f is a matrix transformation, Proposition 1.10.4 guarantees that f is linear. Moreover, A
is the standard matrix of f .

Irena Penev
Linear Algebra 1 & 2
Chapter 1. Systems of linear equations. Vectors and matrices 173

(f ) rank(AT ) = n;

(g) A is a product of elementary matrices;

(h) the homogeneous matrix-vector equation Ax = 0 has only the trivial solution (i.e.
the solution x = 0);

(i) there exists some vector b ∈ Fn such that the matrix-vector equation Ax = b
has a unique solution;

(j) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has a unique solution;

(k) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has at most one
solution;

(l) for all vectors b ∈ Fn , the matrix-vector equation Ax = b is consistent;

(m) f is one-to-one;

(n) f is onto;

(o) f is an isomorphism.

Proof. By Theorem 1.11.9, (a), (c), (e), and (o) are equivalent, and by Corol-
lary 1.11.10, (a), (b), and (f) are equivalent. By Theorem 1.11.4, (a) and (d) are
equivalent. By Proposition 1.11.12(c), we have that (a) and (g) are equivalent. So
far, we have shown that (a), (b), (c), (d), (e), (f), (g), and (o) are equivalent.
Next, by Theorem 1.6.8, (e), (h), (i), (j), (k), and (l) are equivalent.
Finally, by Theorem 1.10.18(a), we have that (m) and (e) are equivalent, and by
Theorem 1.10.18(b), we have that (n) and (e) are equivalent. This completes the
argument.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 174

Chapter 2

Groups and permutations.

Fields

2.1 Monoids
A monoid is an ordered pair (S, ◦), where S is a set and ◦ is a binary operation on S
(i.e. ◦ : S × S → S), satisfying the following two axioms:
1. the operation ◦ is associative, i.e. for all a, b, c ∈ S, we have that
a ◦ (b ◦ c) = (a ◦ b) ◦ c;

2. there exists some e ∈ S, called the identity element of (S, ◦), such that for all
a ∈ S, we have that
e◦a = a and a ◦ e = a.

Proposition 2.1.1. Every monoid has a unique identity element.

Proof. Let (S, ◦) be a monoid. By definition (in particular, by axiom 2), the monoid
(S, ◦) has an identity element; we must show that this identity element is unique.
Suppose that e1 , e2 are identity elements of (S, ◦).1 Then
(∗) (∗∗)
e1 = e1 ◦ e2 = e2
where (*) follows from the fact that e2 is the identity element of the monoid (S, ◦),
and (**) follows from the fact that e1 is the identity element of the monoid (S, ◦).
So, the identity element of the monoid (S, ◦) is unique.
1
This means that the following hold:
ˆ for all a ∈ S, we have that e1 ◦ a = a and a ◦ e1 = a;
ˆ for all a ∈ S, we have that e2 ◦ a = a and a ◦ e2 = a.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 175

Example 2.1.2. All the following are monoids:

1. (N0 , +); 2. (Z, +); 3. (Q, +); 4. (R, +); 5. (C, +).

In each of the above, 0 is the identity element.

Remark: (N, +) is not a monoid, since it does not have an identity element.

Example 2.1.3. All the following are monoids (“·” denotes multiplication):

1. (N0 , ·); 3. (Z, ·); 5. (R, ·);

2. (N, ·); 4. (Q, ·); 6. (C, ·).

In each of the above, 1 is the identity element.

Example 2.1.4. All the following are monoids (“·” denotes multiplication):

1. (N, ·); 3. (Q \ {0}, ·); 5. (C \ {0}, ·).

2. (Z \ {0}, ·); 4. (R \ {0}, ·);

In each of the above, 1 is the identity element.

2.2 Groups
2.2.1 Groups: definition and basic properties
A group is an ordered pair (G, ◦), where G is a set and ◦ is a binary operation on G
(i.e. ◦ : G × G → G) that satisfy the following three axioms:
1. the operation ◦ is associative, i.e. for all a, b, c ∈ G, we have that

a ◦ (b ◦ c) = (a ◦ b) ◦ c;

2. there exists some e ∈ G, called the identity element of (G, ◦), such that for all
a ∈ G, we have that

e◦a = a and a ◦ e = a;

3. for all a ∈ G, there exists some a′ ∈ G, called the inverse of a, such that

a ◦ a′ = e and a′ ◦ a = e.

An abelian group is a group (G, ◦) that satisfies the following additional axiom:

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 176

4. the operation ◦ is commutative, i.e. for all a, b ∈ G, we have that

a ◦ b = b ◦ a.

A non-abelian group is a group that is not abelian.

Remark: Note that the first two axioms (axioms 1 and 2) from the definition
of a group are precisely the monoid axioms. So, every group is a monoid. By
Proposition 2.1.1, it follows that the identity element e of a group is unique. In
particular, the third axiom (axiom 3) makes sense.

Terminology/Notation: If the operation ◦ of the group (G, ◦) is clear from context,

then we may say that G is a group, rather than that (G, ◦) is a group. However, this
is only done if there is no chance of confusion, and so when in doubt, you should
specify the operation. Sometimes, we say “G is a group under the operation ◦,”
which means exactly the same thing as “(G, ◦) is a group.”

Proposition 2.2.1. Each element of a group has a unique inverse.

Proof. Let (G, ◦) be a group, and let e be its identity element. Fix some g ∈ G. By
the definition of a group (and in particular, by axiom 3), g has an inverse in the
group (G, ◦); we must show that this inverse is unique. Let g1 and g2 be inverses of
g in the group (G, ◦).2 Then
g1 = g1 ◦ e because e is the identity element of (G, ◦)

= g1 ◦ (g ◦ g2 ) because g2 is an inverse of g

= (g1 ◦ g) ◦ g2 because ◦ is associative

= e ◦ g2 because g1 is an inverse of g

= g2 because e is the identity element of (G, ◦).

We have now shown that g1 = g2 . So, the inverse of g is unique.

Notation: Typically, the (unique) inverse of an element g of a group (G, ◦) is

denoted by g −1 . However, when the group operation is denoted by + (note: this
is typically done only if the group is abelian), then the inverse of an element g is
denoted by −g.

2
This means that both the following hold:
ˆ g ◦ g1 = e and g1 ◦ g = e;
ˆ g ◦ g2 = e and g2 ◦ g = e.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 177

Example 2.2.2. All the following are abelian groups:

1. (Z, +); 2. (Q, +); 3. (R, +); 4. (C, +).

In each of the above cases, the identity element is 0, and the inverse of a group
element g is −g.3

Note that the monoid (N0 , +) is not a group because elements other than 0 do
not have inverses, and so axiom 3 from the definition of a group is not satisfied.

Example 2.2.3. All the following are abelian groups:

1. (Q \ {0}, ·); 2. (R \ {0}, ·); 3. (C \ {0}, ·).

In each of the above cases, the identity element is 1, and the inverse of a group
element g is g −1 = g1 .4

Remark: Monoids (Q, ·), (R, ·), and (C, ·) are not groups because, in each of those
cases, 0 does not have an inverse element. Note also that (Z \ {0}, ·) is not a group
because elements other than 1 and −1 do not have inverses.
Remark: It might now seem that all groups are abelian. However, this is not
the case: we will see examples of non-abelian groups in subsection 2.2.2 and in
section 2.3.

Proposition 2.2.4. Let (G, ◦) be a group with identity element e. Then all the
following hold (here, the inverse of a group element g is denoted by g −1 ):

(a) for all a, b, c ∈ G, if a ◦ b = a ◦ c, then b = c;

(b) for all a, b, c ∈ G, if b ◦ a = c ◦ a, then b = c;

(c) for all a, b ∈ G, there exists a unique x ∈ G such that a ◦ x = b;

(d) for all a, b ∈ G, there exists a unique x ∈ G such that x ◦ a = b;

(e) for all a ∈ G, (a−1 )−1 = a;5

(f ) for all a, b ∈ G, (a ◦ b)−1 = b−1 ◦ a−1 .

Proof. We first prove (a). Fix a, b, c ∈ G, and assume that a ◦ b = a ◦ c. Then

3
√ √
For example, in the group (R, +), the inverse of 13 √ is − 13.
4
For example, in the group (R \ {0}, ·), the inverse of 13 is √113 .
5
So, the inverse of the inverse of a is equal to a.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 178

because e is the identity

b = e◦b
element of (G, ◦)

= (a−1 ◦ a) ◦ b because a−1 ◦ a = e

= a−1 ◦ (a ◦ b) because ◦ is associative

= a−1 ◦ (a ◦ c) because a ◦ b = a ◦ c

= (a−1 ◦ a) ◦ c because ◦ is associative

= e◦c because a−1 ◦ a = e

because e is the identity

= c
element of (G, ◦).

This proves (a). The proof of (b) is similar.

Next, we prove (c). Fix a, b ∈ G. We must show that there exists a unique x ∈ G
such that a ◦ x = b. For existence, we set x := a−1 ◦ b, and we observe that

a◦x = a ◦ (a−1 ◦ b) because x = a−1 ◦ b

= (a ◦ a−1 ) ◦ b because ◦ is associative

= e◦b because a ◦ a−1 = e

becuase e is the identity

= b
element of (G, ◦).

Uniqueness follows from (a). This proves (c). The proof of (d) is similar.
We now prove (e). Fix a ∈ G. It suffices to show that a−1 ◦ (a−1 )−1 = a−1 ◦ a,
for then (a) will guarantee that (a−1 )−1 = a, which is what we need. Since (a−1 )−1
is the inverse of a−1 , we know that a−1 ◦ (a−1 )−1 = e. On the other hand, since a−1
is the inverse of a, we have that a−1 ◦ a = e. Thus, a−1 ◦ (a−1 )−1 = a−1 ◦ a. As
explained above, this implies that (a−1 )−1 = a. This proves (e).
It remains to prove (f). Fix a, b ∈ G. We observe that

(a ◦ b) ◦ (b−1 ◦ a−1 ) = a ◦ (b ◦ b−1 ) ◦ a because ◦ is associative

= a ◦ e ◦ a−1 because b ◦ b−1 = e

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 179

because e is the identity

= a ◦ a−1
element of (G, ◦)

= e because a ◦ a−1 = e,

and similarly,

(b−1 ◦ a−1 ) ◦ (a ◦ b) = b−1 ◦ (a−1 ◦ a) ◦ b because ◦ is associative

= b−1 ◦ e ◦ b because a−1 ◦ a = e

because e is the identity

= b−1 ◦ b
element of (G, ◦)

= e because b−1 ◦ b = e.

We have now shown that (a ◦ b) ◦ (b−1 ◦ a−1 ) = e and (b−1 ◦ a−1 ) ◦ (a ◦ b) = e. It

follows that (a ◦ b)−1 = b−1 ◦ a−1 . This proves (f).

The case of Zn and Zp . For Zn (where n is a positive integer) and Zp (where

p is a prime number), we have Proposition 2.2.5 (below). We note that part (b)
crucially relies on Fermat’s Little Theorem, stated and proven in subsection 0.2.2,
and restated below for the reader’s convenience.

Fermat’s Little Theorem. If p ∈ N is a prime number and a ∈ Zp \ {0}, then

ap−1 = 1.

Proposition 2.2.5.

(a) For all positive integers n, (Zn , +) is an abelian group whose identity element is
0 := [0]n .

(b) For all prime numbers p, (Zp \ {0}, ·) is an abelian group whose identity element
is 1 := [1]p .

Proof. (a) Fix a positive integer n. The fact that + (“addition”) is an associative and
commutative binary operation on Zn follows from Proposition 0.2.11. The identity
element of Zn is 0 := [0]n . For each element [a]n in Zn (where a ∈ Z), the additive
inverse of [a]n is [−a]n = [n − a]n . So, (Zn , +) is an abelian group with identity
element [0]n .
(b) Fix a prime number p. By Proposition 0.2.11, we know that · (“multiplication”)
is an associative and commutative binary operation on Zp . However, the question

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 180

is whether multiplication remains a binary operation on Zp \ {0}, that is, whether

Zp \ {0} is “closed under multiplication,” that is, whether the product of two numbers
in Zp \ {0} is always another number in Zp \ {0}.6 So, fix a, b ∈ Z such that [a]p and
[b]p are both non-zero (in Zp ), i.e. p divides neither a nor b. Since p is prime, p does
not divide the product ab,7 and consequently, [a]p [b]p = [ab]p = ̸ 0. So, multiplication
is indeed a binary operation on Zp \ {0}. The identity element of (Zp \ {0}, ·) is
1 := [1]p . Moreover, by Fermat’s Little Theorem, each number a ∈ Zp \ {0} has
a multiplicative inverse, namely, ap−2 . This proves that (Zp \ {0}, ·) is indeed an
abelian group.

Remark: If n is a positive integer that is not prime, then (Zn \ {0}, ·) is not a group.
Indeed, if n = 1, then Zn \{0} is empty and therefore not a group under any operation
(no group is empty, since it must, at a minimum, contain an identity element). On
the other hand, if n ≥ 2 is a composite number, say n = pq for some integers p, q ≥ 2,
then we have that [p]n , [q]n ∈ Zn \ {0}, but [p]n [q]n = [pq]n = [n]n = 0, and it follows
that Zn \ {0} is not closed under multiplication, i.e. multiplication is not a binary
operation on Zn \ {0}.

2.2.2 Groups of matrices and vectors. The general linear group

Let F is a field. Since we have not formally studied fields yet, you may assume for now
that F is one of the following: Q, R, C, or Zp (where p is a prime number). However,
the examples given in this subsection work for all fields, not just the four listed above.
First of all, it is obvious that (Fn×m , +) is an abelian group whose  identity element
is the zero matrix On×m ; the (additive) inverse of a matrix ai,j n×m in the group
(Fn×m , +) is the matrix −ai,j n×m (i.e. the n × m matrix whose i, j-th entry is
 

−ai,j for all indices i ∈ {1, . . . , n} and j ∈ {1, . . . , m}). In particular, (Fn , +) is an
abelian group (with identity element 0).8 More interestingly, consider the set GLn (F)
of all invertible matrices in Fn×n . GLn (F) is a group under matrix multiplication,
called the general linear group of degree n over the field F. The identity element of
GLn (F) is the identity matrix In , and the inverse of a matrix A in GLn (F) is the
matrix A−1 (the usual matrix inverse that we studied in section 1.11). The group
GL1 (F) is abelian (because multiplication is commutative in the field F). However,
for n ≥ 2, the group GLn (F) is not abelian. Let us first check this for n = 2, and
then we will generalize. Consider the following two matrices in F2×2 :
   
1 1 1 0
A2 := , B2 := .
0 1 1 1
6
In other words, the question is whether the product of two non-zero numbers in Zp is always a
non-zero number in Zp .
7
Remark: This is why we care about p being prime! If p were not prime, then this implication
would be invalid.
8
We are using the fact that, by definition, Fn = Fn×1 .

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 181

Both of these matrices have rank 2, and so by the Invertible Matrix Theorem (see
subsection 1.11.7), they are both invertible and therefore belong to GL2 (F). However,
we have that
    
1 1 1 0 1+1 1
ˆ A2 B 2 = = ,
0 1 1 1 1 1
    
1 0 1 1 1 1
ˆ B 2 A2 = = .
1 1 0 1 1 1+1
Since 1 + 1 ̸= 1,9 we see that A2 B2 ̸= B2 A2 , and so GL2 (F) is not abelian. Let us
now generalize this. Fix an integer n ≥ 2, and consider the following two matrices
in Fn×n :
   
1 0 0 ... 0 0 1 1 0 0 ... 0 0 0
 0 1 0 ... 0 0 0   0 1 0 ... 0 0 0 
   
 0 0 1 ... 0 0 0   0 0 1 ... 0 0 0 
   
An :=  ... ... ... . . . ... ... ...  , Bn :=  ... ... ... . . . ... ... .
   
   
 0 0 0 ... 1 0 0   0 0 0 ... 1 0 0 
   
 0 0 0 ... 0 1 0   0 0 0 ... 0 1 0 
0 0 0 ... 0 0 1 1 0 0 ... 0 0 1
It is then easy to see that rank(An ) = rank(Bn ) = n, and so by the Invertible Matrix
Theorem (see subsection 1.11.7), An and Bn are both invertible and consequently
belong to GLn (F). Moreover, the 1,1-th entry (the one in the upper left corner) of
An Bn is 1 + 1, whereas the 1,1-th entry of Bn An is 1. Since 1 + 1 ̸= 1, we see that
An Bn ̸= Bn An , and it follows that the group GLn (F) is not abelian.
Remark: The fact that 1 + 1 ̸= 1 is obviously true for the fields that we are familiar
with. But in fact, it is true in any field F, not just those that we have seen so far,
and it essentially follows from the fact that 1 ̸= 0 (which is true for any field; see
axiom 3 from the definition of a field in section 2.4). On the other hand, 1 + 1 + 1 = 1
is true in some fields (for example, it is true for the field Z2 ).

2.2.3 Subgroups
A subgroup of a group (G, ◦) is a group (H, ⋄) such that H ⊆ G and for all a, b ∈ H, we
have that a ⋄ b = a ◦ b. If (H, ⋄) is a subgroup of (G, ◦), then we write (H, ⋄) ≤ (G, ◦).
Here, ⋄ is the restriction of ◦ to H, and it is important that a ⋄ b = a ◦ b ∈ H for
all a, b ∈ H (otherwise, H is not “closed under” ⋄, which means that ⋄ is not a
binary operation on H, and in particular, (H, ⋄) is not a group). Normally, we do
not notationally distinguish between ⋄ and ◦, and we speak about (H, ◦) being a
subgroup of (G, ◦), where it is understood from context that the operation ◦ from
(H, ◦) is the restriction of the the binary operation ◦ on G to H.
9
See the Remark at the end of this subsection.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 182

Example 2.2.6. Every group (G, ◦) has the following two (possibly identical) sub-
groups: (G, ◦) and ({e}, ◦), where e is the identity element of G.10

Example 2.2.7. (Z, +) ≤ (Q, +) ≤ (R, +) ≤ (C, +).

Example 2.2.8. (Q \ {0}, ·) ≤ (R \ {0}, ·) ≤ (C \ {0}, ·).

Theorem 2.2.9. Let (G, ◦) be a group with identity element e, and with the inverse
of an element a ∈ G denoted by a−1 . Then for all H ⊆ G, we have that (H, ◦) is a
subgroup of (G, ◦) if and only if all the following hold:

(i) e ∈ H;

(ii) H is closed under ◦, that is, for all a, b ∈ H, we have that a ◦ b ∈ H;

(iii) H is closed under inverses, that is, for all a ∈ H, we have that a−1 ∈ H.

Proof. Fix H ⊆ G. Suppose first that (i), (ii), and (iii) hold. By (ii), the binary
operation ◦ on G can be restricted to H (so that it becomes a binary operation
on H). The fact that ◦ is associative in (H, ◦) follows simply from the fact that ◦
is inherited from the group (G, ◦), where it is associative. By (i), H contains an
identity element, and by (iii), every element of H has an inverse in (H, ◦).
Suppose, conversely, that (H, ◦) is a subgroup of (G, ◦). Then (ii) holds, because ◦
(properly restricted) is a binary operation on H. It remains to prove that (i) and (iii)
hold. Since H is a group, it must have an identity element eH , and each element
of H must have inverse in (H, ◦). The question is whether the identity element of
(H, ◦) is the same as in (G, ◦), and similar for inverses.11 We first deal with the
identity element. If we compute in (H, ◦), we have that eH ◦ eH = eH (because eH
is the identity element of (H, ◦)), and if we compute in (G, ◦), then we have that
eH ◦ e = eH (because e is the identity element of (G, ◦)). But now eH ◦ eH = eH ◦ e,
and so by Proposition 2.2.4(a) applied to (G, ◦), we have that eH = e. So, e ∈ H,
and it follows that (i) holds. Finally, fix a ∈ H. Since (H, ◦) is a group, a has an
inverse a′ in (H, ◦), so that a ◦ a′ = eH = e. On the other hand, if we compute
in (G, ◦), we get that a ◦ a−1 = e. It follows that a ◦ a′ = a ◦ a−1 , and so by
Proposition 2.2.4(a) applied to (G, ◦), we have that a′ = a−1 , and consequently,
a−1 ∈ H. This proves (ii).
10
If G = {e}, then the subgroups (G, ◦) and ({e}, ◦) are obviously identical. On the other hand,
if G has at least two elements, then the subgroups (G, ◦) and ({e}, ◦) are distinct, and it is possible
that (G, ◦) has other subgroups as well.
11
Could it be that eH ̸= e, i.e. that (H, ◦) has an identity element, but one that is different from
the identity element of (G, ◦)? Could something similar happen with inverses? Actually, this cannot
happen, but we need to prove that!

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 183

2.3 Permutations and the symmetric group

A permutation of a set X is any bijection from X to itself. The set of all permutations
of X is denoted by Sym(X). As usual, IdX is the identity function on X, i.e.
IdX : X → X is given by IdX (x) = x for all x ∈ X.
Note that for any set X, (Sym(X), ◦) is a group, called the symmetric group
on X (here, ◦ is the composition of functions). Let us justify this. First of all, by
Proposition 1.10.17(c),12 the composition of two permutations of X is a permutation
of X, and so ◦ is indeed a binary operation on Sym(X). Moreover, it is clear that ◦
is associative; indeed, for any π, σ, τ ∈ Sym(X), we have that π ◦ (σ ◦ τ ) = (π ◦ σ) ◦ τ ,
because for all x ∈ X, we have the following:



π ◦ (σ ◦ τ ) (x) = π (σ ◦ τ )(x)


= π σ(τ (x))


= (π ◦ σ) τ (x)


= (π ◦ σ) ◦ τ (x).

The identity element of this group is the identity function IdX on X. The inverse
element of any permutation π ∈ Sym(X) is the inverse permutation π −1 . (Since
permutations are bijections, they have inverse functions, and moreover, those inverses
are also bijections; see the comment following the proof of Proposition 1.10.15. We
deduce that the inverse of a permutation of X is another permutation of X.)
If a set X has at most two elements, then it is easy to see that the group Sym(X)
is abelian. However, if X has at least three elements, then Sym(X) is not abelian,
as we now show. Suppose that |X| ≥ 3, and let a, b, c be pairwise distinct elements
of X. Let σ, τ : X → X be defined as follows:13
ˆ σ(a) = b, σ(b) = a, and σ(x) = x for all x ∈ X \ {a, b};

ˆ τ (a) = c, τ (c) = a, and τ (x) = x for all x ∈ X \ {a, c}.

Clearly, σ, τ ∈ Sym(X). But now
ˆ (σ ◦ τ )(a) = σ τ (a) = σ(c) = c;

ˆ (τ ◦ σ)(a) = τ σ(a) = τ (b) = b.

Since b ̸= c, we have that (σ ◦ τ )(a) ̸= (τ ◦ σ)(a). So, σ ◦ τ ̸= τ ◦ σ, and it follows

that Sym(X) is not abelian.
12
We apply Proposition 1.10.17(c) for A = B = C = X.
13
The permutation σ swaps (“transposes”) a and b, while leaving all other elements of X fixed.
Similarly, the permutation τ swaps (“transposes”) a and c, while leaving all other elements of X
fixed. For more on transpositions, see subsection 2.3.3.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 184

We particularly often consider Sym(X) for the case when X = {1, . . . , n} for
some positive integer n. The set Sym({1, . . . , n}) is also denoted by Sym(n), Symn ,
or Sn . In these lecture notes, we will consistently use the notation Sn . The group
(Sn , ◦) is called the symmetric group of degree n. Note that |Sn | = n!.
A permutation π ∈ Sn can be represented in the following way:
 
1 2 ... n
π = .
π(1) π(2) . . . π(n)
So, in the top row, we have numbers 1, 2, . . . , n, and in the bottom row, we have
those same numbers in some order (determined by the permutation π). For example,
the permutation π ∈ S4 given by π(1) = 3, π(2) = 2, π(3) = 4, and π(4) = 1 can be
represented as follows:  
1 2 3 4
π = .
3 2 4 1

2.3.1 Cycle notation

Suppose we are given the following permutation in S9 :
 
1 2 3 4 5 6 7 8 9
π = .
3 6 2 4 9 7 1 8 5

We can represent this permutation geometrically, as shown below (the cycles are
color coded for easier reference).
7 9

1 6
4 8
3 2 5

We can “encode” the picture that we obtained as a “product of disjoint cycles”:

π = (13267)(4)(59)(8).

The above is also referred to as a “disjoint cycle decomposition” of the permutation π.

The disjoint cycle decomposition of a permutation is unique up to cyclic permutation
of the elements within each cycle, and up to a reordering of the cycles. For example,
the permutation π above can also be expressed as follows:

π = (95)(26713)(8)(4).

However, the first disjoint cycle decomposition14 is canonical/standard because it

satisfies the following two properties:
14
That is, the disjoint cycle decomposition π = (13267)(4)(59)(8).

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 185

ˆ within each cycle, the smallest number appears first;

ˆ the first elements of the cycles from the disjoint cycle decomposition form an
increasing sequence.15

Usually, the canonical representation is preferred, but occasionally, it may be more

practical to use a non-canonical one. When the n from Sn is clear from context,
one-element cycles may be omitted. So, if we know that we are working in S9 , then
we may omit the one-element cycles (4) and (8) from the representation above, and
write simply
π = (13267)(59).
In this case, the cycles (4) and (8) are understood from context. However, we can
only do this when n has been specified beforehand! Otherwise, cycles of length one
must be included.
Notation: When there is danger of confusion, we put commas between elements
within cycles. For instance, if we are working in S12 , then (123) is ambiguous. To
avoid ambiguity, we write (1, 2, 3) or (12, 3), as appropriate. However, if we are
working in Sn , where n is a single-digit number, then there is no danger of confusion,
and so we normally omit commas.
Let us consider some more examples.

Example 2.3.1. Find the disjoint cycle decompositions of the following permutations.
 
1 2 3 4 5
(a) π1 =
2 5 4 3 1
 
1 2 3 4 5 6
(b) π2 =
3 2 4 1 6 5
 
1 2 3 4 5
(c) π3 =
2 5 1 3 4
Solution. We have:

(a) π1 = (125)(34);

(b) π2 = (134)(2)(56);

(c) π3 = (12543).

Note that in (b), we could also have written π ∈ S6 , π = (134)(56).

15
Indeed, 1 < 4 < 5 < 8.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 186

It is also easy to go the other way around: from the disjoint cycle decomposition
to the table representation, i.e. representation of the form
 
1 2 ... n
π = .
π(1) π(2) . . . π(n)

For instance, we see that

 
1 2 3 4 5 6
(143)(26)(5) =
4 6 1 3 5 2

and  
1 2 3 4 5 6
(154362) = .
5 1 6 3 4 2

Compositions of permutations. By Proposition 1.10.17(c), the composition of

two permutations in Sn is another permutation in Sn . For instance, in S5 , we have
the following (with permutations color coded for easier reference):
     
1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
◦ = .
1 3 2 5 4 2 4 5 1 3 3 5 4 1 2

As usual with function composition, we apply permutations from right to left

with
 respect to ◦. So, in the case above, we first apply the blue permutation 
1 2 3 4 5 1 2 3 4 5
, and then we apply the red permutation .
2 4 5 1 3 1 3 2 5 4
For instance, 1 first gets mapped to 2 via the blue permutation, and then 2 gets
mapped to 3 via the red permutation. So, the composition above maps 1 to 3.
We can similarly compose permutations specified in terms of their disjoint cycle
decompositions. For instance,

(1)(23)(45) ◦ (124)(35) = (134)(25).

Again we apply permutations from right to left with respect to ◦. So, in the case
above, we first apply the blue permutation (124)(35), and then we apply the red
permutation (1)(23)(45). However, within each permutation (separated by ◦’s from
the other permutations), we read from left to right. For instance, in the blue
permutation (124)(35), 1 gets mapped to 2, 2 gets mapped to 4, and 4 gets mapped
to 1.
Again, when the n from Sn is clear from context, we may omit one-element cycles.
For instance, in S5 , we have

(154) ◦ (245)(13) ◦ (25) = (135).

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 187

Here, certain one-element cycles are understood from context. In particular, (154) =
(154)(2)(3), (25) = (1)(25)(3)(4), and (135) = (135)(2)(4). So, the above expression
can be rewritten as

(154)(2)(3) ◦ (245)(13) ◦ (1)(25)(3)(4) = (135)(2)(4).

Inverses of permutations. The inverse of a permutation π in Sn can be obtained

by starting with a disjoint cycle decomposition of π, and then reversing the order of
elements in all cycles, i.e. turning each cycle of the form (a1 a2 . . . ak ) into (ak . . . a2 a1 ).
Pictorially, we get the same cycles, only with arrows reversed (see the picture below).

ak ak

a1 a2 a1 a2
(a1a2 . . . ak ) (ak . . . a2a1)

For example, in S7 :
ˆ if π1 = (143)(2576), then π1−1 = (341)(6752) = (134)(2675);

7 7
3 4 3 4
5 5
6 6
1 2 1 2
π1 π1−1

ˆ if π2 = (15)(2)(3476), then π2−1 = (51)(2)(6743) = (15)(2)(3674).

5 7 5 7
4 4
6 6
1 2 3 1 2 3
π2 π2−1

Notation: The identity permutation in Sn is often denoted simply by 1. So, in this

context, we have that
1 = (1)(2) . . . (n).
If we wish to emphasize n (or if we need to avoid confusion with other kinds of 1 that
may appear in our proof/computation), then we can denote the identity permutation
in Sn by 1n .

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 188

2.3.2 The sign of a permutation. Even and odd permutations

Given a positive integer n and a permutation π ∈ Sn , the sign of π, denoted by
sgn(π), is given by
sgn(π) = (−1)n−k ,
where k is the number of cycles in the disjoint cycle decomposition of π including
the one-element cycles. For instance, for π1 = (1367)(2)(45) in S7 , we have

sgn(π1 ) = (−1)7−3 = 1,

whereas for π2 = (12)(345)(6)(7) in S7 , we have

sgn(π2 ) = (−1)7−4 = −1.

Equivalently, for π ∈ Sn , we have that

′ ′
sgn(π) = (−1)n −k ,
where k ′ is the number of cycles in some disjoint cycles in some disjoint cycle
decomposition of π (possibly with some one-element cycles omitted), and n′ is the
number of elements in those k ′ cycles. The two definitions are equivalent because if
d is the number of omitted one-element cycles in some disjoint cycle decomposition
of π , then n = n′ + d, and if we write the complete disjoint cycle decomposition of π
including all one-element cycles, then we get k = k ′ +d many cycles. So, n−k = n′ −k ′ ,
′ ′
and consequently, (−1)n−k = (−1)n −k . For instance, for π3 = (123)(45) in S7 , we
have
sgn(π3 ) = (−1)5−2 = −1.
Note that the one-element cycles (6) and (7) are implicitly understood for π3 , that
is, π3 = (123)(45)(6)(7). And indeed, we have

sgn(π3 ) = (−1)7−4 = −1,

as before.
Remark: Note that for all positive integers n, the identity permutation in Sn has sign
1. This is because the identity permutation in Sn has disjoint cycle decomposition
(1)(2) . . . (n), and so its sign is (−1)n−n = (−1)0 = 1.

Terminology: Permutations whose sign is +1 are called even, and permutations

whose sign is −1 are called odd. Since the sign of the identity permutation is +1,
the identity permutation is even.

Proposition 2.3.2. Let n ≥ 2 be an integer, and let π be a permutation in Sn .

Then sgn(π −1 ) = sgn(π).
Proof. This follows from the fact that π and π −1 have the same number of cycles in
their disjoint cycle decompositions (when the one-element cycles are included).

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 189

2.3.3 Transpositions
Slightly informally, a transposition is a permutation that swaps two elements and
fixes all the remaining ones. More formally, given an integer n ≥ 2, a transposition in
Sn is a permutation π ∈ Sn for which there exist distinct i, j ∈ {1, . . . , n} such that
π(i) = j, π(j) = i, and π(ℓ) = ℓ for all ℓ ∈ {1, . . . , n} \ {i, j}. Such a transposition
is typically denoted by (ij), and the n − 2 many one-element cycles are implicitly
understood. For instance, the following permutation in S5 is a transposition:
 
1 2 3 4 5
= (25).
1 5 3 4 2

Note that this transposition could also have been written in the form (1)(25)(3)(4).
More commonly, one-element cycles are omitted.
Remark: Every transposition is its own inverse, that is, for any transposition
τ = (ij) in Sn (n ≥ 2), we have that τ −1 = τ .
Remark: The sign of any transposition is −1,16 and so transpositions are odd.
As we shall see, for n ≥ 2, any permutation can be written as a composition of
transpositions. For instance, in S7 , we have

(134)(2657) = (13) ◦ (34) ◦ (26) ◦ (65) ◦ (57).

The correctness of the above can easily be verified by checking that the image of
each element of {1, . . . , 7} under the permutations (134)(2657) and (13) ◦ (34) ◦ (26) ◦
(65) ◦ (57) is the same. Moreover, this works in general, as the following proposition
shows.

Proposition 2.3.3. Let n ≥ 2 be an integer. Then any permutation in Sn can be

written as a composition of transpositions.

Proof. The identity permutation in Sn can be written in the form (12) ◦ (12).17 Let
us now suppose that π is some permutation in Sn other than the identity. Then π
can be written as the product of one or more disjoint cycles of length at least two
(one-element cycles are omitted in our expression, but are understood from context).
Let us say we have k cycles of length at least two, as follows (to help the reader, the
cycles are color coded):

π = (a11 a12 . . . a1ℓ1 ) . . . (ak1 ak2 . . . akℓk ),

16
This follows straight from the definition of the sign of a permutation. Indeed, if τ is a
transposition in Sn (n ≥ 2), then the disjoint cycle decomposition of τ consists of one cycle of length
two and n − 2 many cycles of length one, and consequently, it consists of n − 1 cycles total (when
cycles of length one are included). So, sgn(τ ) = (−1)n−(n−1) = −1.
17
Actually, the identity permutation can also be written as an “empty” composition of transposi-
tions.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 190

where the aji ’s are pairwise distinct, and ℓ1 , . . . , ℓk ≥ 2. But then we have

π = (a11 a12 ) ◦ (a12 a13 ) ◦ · · · ◦ (a1ℓ1 −1 a1ℓ1 ) ◦ · · · ◦ (ak1 ak2 ) ◦ (ak2 ak3 ) ◦ · · · ◦ (akℓk −1 akℓk ),

and so π is the composition of transpositions.

Example 2.3.4. Express each of the following permutations in S6 as the composition

of transpositions.
 
1 2 3 4 5 6
(a) π1 = ;
1 5 2 3 4 6
 
1 2 3 4 5 6
(b) π2 = ;
2 1 3 5 6 4
 
1 2 3 4 5 6
(c) π3 = .
3 6 5 2 1 4
Solution. To help the reader, we color code the cycles that we obtain, as well as the
transpositions that correspond to them.
 
1 2 3 4 5 6
(a) π1 = = (2543) = (25) ◦ (54) ◦ (43);
1 5 2 3 4 6
 
1 2 3 4 5 6
(b) π2 = = (12)(456) = (12) ◦ (45) ◦ (56);
2 1 3 5 6 4
 
1 2 3 4 5 6
(c) π3 = = (135)(264) = (13) ◦ (35) ◦ (26) ◦ (64).
3 6 5 2 1 4

We note that the same permutation can be expressed as the composition of

transpositions in more than one way. For instance, in S5 , we have:
ˆ (12345) = (12) ◦ (23) ◦ (34) ◦ (45);

ˆ (12345) = (12) ◦ (23) ◦ (34) ◦ (45) ◦ (35) ◦ (35);

ˆ (12345) = (15) ◦ (14) ◦ (13) ◦ (12);

ˆ (12345) = (35) ◦ (35) ◦ (23) ◦ (23) ◦ (15) ◦ (14) ◦ (13) ◦ (12) ◦ (35) ◦ (35).
However, as we shall see, for any given permutation π in Sn , where n ≥ 2, in all
representations of π as a composition of transpositions, the number of transposi-
tions is of the same parity (i.e. it is either always even or always odd). We prove
this in Theorem 2.3.6. However, to prove Theorem 2.3.6, we need the following
technical proposition, which essentially states that composing a permutation with a
transposition results in a sign change.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 191

Proposition 2.3.5. Let n ≥ 2 be an integer. Then for all π, τ ∈ Sn such that τ is a

transposition, we have that sgn(τ ◦ π) = sgn(π ◦ τ ) = −sgn(π).

Warning: In general, τ ◦ π Z
=π ◦ τ.


Proof. The Claim below proves one part of the proposition (“sgn(τ ◦ π) = −sgn(π)”).
The other part (“sgn(π ◦ τ ) = −sgn(π)”) can be proven using the Claim and certain
basic properties of permutations (as we shall see below).

Claim. For all π, τ ∈ Sn such that τ is a transposition, we have that

sgn(τ ◦ π) = −sgn(π).

Proof of the Claim. Fix π, τ ∈ Sn , and assume that τ = (ij) is a transposition

(here, i and j are some two distinct elements of {1, . . . , n}). There are two cases to
consider: when i and j are in the same cycle of the disjoint cycle decomposition of
π, and when they are in different cycles.
Case 1: i and j are in the same cycle of the disjoint cycle decomposition of π.
After possibly swapping the order of our disjoint cycles, and cyclically permuting the
elements of the cycle that contains i and j, we may assume that our disjoint cycle
decomposition of π is given by

π = (i a1 . . . ap j b1 . . . bq )(c11 . . . c1ℓ1 ) . . . (cr1 . . . crℓr ).

Note: Here, i and j are both in the red cycle. The remaining cycles (the ones that
do not contain i and j) are colored blue.18
In the permutation τ ◦ π, the red cycle essentially gets “split up” into two, while
the blue cycles remain unaffected, as follows:

τ ◦ π = (ij) ◦ (i a1 . . . ap j b1 . . . bq )(c11 . . . c1ℓ1 ) . . . (cr1 . . . crℓr )

= (i a1 . . . ap )(j b1 . . . bq )(c11 . . . c1ℓ1 ) . . . (cr1 . . . crℓr ) .

| {z }
=:π ′

The correctness of the above can easily be verified by checking that the permutations
τ ◦ π and π ′ do indeed map each element of

{1, . . . , n} = {i, a1 , . . . , ap , j, b1 , . . . , bq , c11 , . . . , c1ℓ1 , . . . , cr1 , . . . , crℓr }

to the same element.19

18
It is possible that r = 0, so that π consists only of the red cycle. It is also possible that p = 0 (in
this case, the red cycle is (i j b1 . . . bq )), or that q = 0 (in this case, the red cycle is (i a1 . . . ap j)).
If p = q = 0, then the red cycle is simply (ij).
19
Here, a picture may help. The diagram below represents the permutation π ′ (obviously). But by
considering what each element of {1, . . . , n} = {i, a1 , . . . , ap , j, b1 , . . . , bq , c11 , . . . , c1ℓ1 , . . . , cr1 , . . . , crℓr }

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 192

We now see that the disjoint cycle decomposition of τ ◦ π has one cycle more than
the disjoint cycle decomposition of π, and it follows that sgn τ ◦ π = −sgn(π),20

which is what we needed to show.

Case 2: i and j are in different cycles of the disjoint cycle decomposition of π.
After possibly swapping the order of our disjoint cycles, and cyclically permuting the
elements of the cycles that contain i and j, we may assume that our disjoint cycle
decomposition of π is given by
π = (i a1 . . . ap )(j b1 . . . bq )(c11 . . . c1ℓ1 ) . . . (cr1 . . . crℓr )
Note: Here, i and j are in the two red cycles. The remaining cycles (the ones that
do not contain i and j are colored blue.21
We then have that
π = (i a1 . . . ap )(j b1 . . . bq )(c11 . . . c1ℓ1 ) . . . (cr1 . . . crℓr )

(∗)
= (ij) ◦ (i a1 . . . ap j b1 . . . bq )(c11 . . . c1ℓ1 ) . . . (cr1 . . . crℓr ),
where (*) follows from the argument given in Case 1. We now compose both sides
with τ = (ij) on the left, and we obtain
(ij) ◦ π = (ij) ◦ (ij) ◦ (i a1 . . . ap j b1 . . . bq )(c11 . . . c1ℓ1 ) . . . (cr1 . . . crℓr ).
Since (ij) = τ and (ij) ◦ (ij) = 1n ,22 we deduce that
τ ◦ π = (i a1 . . . ap j b1 . . . bq )(c11 . . . c1ℓ1 ) . . . (cr1 . . . crℓr ).
As we can see, in the permutation τ ◦ π, the two red cycles of π essentially get
“merged” into one, while the blue cycles remain unaffected. But now the disjoint
cycle decomposition of τ ◦ π has one

cycle less than the disjoint cycle decomposition
23
of π, and it follows that sgn τ ◦ π = −sgn(π), which is what we needed to show.
This completes the proof of the Claim. ♦
gets mapped to by the permutation τ ◦ π, we see that the diagram below also represents the
permutation τ ◦ π.
c1ℓ1 crℓr
ap a1 bq b1 ...
i j c11 cr1

20
Indeed, the disjoint cycle decomposition of π has r + 1
cycles, whereas the disjoint cycle
decomposition of τ ◦ π has r + 2 cycles. Therefore, sgn τ ◦ π = (−1)n−(r+2) = (−1)n−(r+1)−1 =
−(−1)n−(r+1) = −sgn(π).
21
It is possible that p = 0, q = 0, or r = 0. Similar remarks apply as in Case 1.
22
Here, 1n is the identity permutation in Sn .
23
Indeed, the disjoint cycle decomposition of π has r + 2
cycles, whereas the disjoint cycle
decomposition of τ ◦ π has r + 1 cycles. Therefore, sgn τ ◦ π = (−1)n−(r+1) = (−1)n−(r+2)+1 =
−(−1)n−(r+2) = −sgn(π).

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 193

Now, fix π, τ ∈ Sn such that τ is a transposition. By the Claim, we have that

sgn(τ ◦ π) = −sgn(π). On the other hand,
 
sgn(π ◦ τ ) = sgn (π ◦ τ )−1 by Proposition 2.3.2

by Proposition 1.10.17(c)
sgn τ −1 ◦ π −1


=
(or by Proposition 2.2.4(f))

because τ is a transposition,
sgn τ ◦ π −1


=
and so τ −1 = τ

by the Claim applied to

= −sgn(π −1 )
π −1 and τ

= −sgn(π) by Proposition 2.3.2.

This completes the argument.

Theorem 2.3.6. Let n ≥ 2. Then for any permutation π ∈ Sn , if π can be expressed

as a composition of r transpositions, then

(a) sgn(π) = (−1)r ;

(b) π is an even permutation if and only if r is even;

(c) π is an odd permutation if and only if r is odd.

Proof. Clearly, (b) and (c) follow from (a). Part (a) follows from Proposition 2.3.5
by an easy induction on r. Let us give the details. We prove the following statement:
“for every positive integer r and permutation π ∈ Sn , if π is the composition of r
transpositions, then sgn(π) = (−1)r .”
Base case: r = 1. Note that if π is the composition of one transposition, i.e. π is
itself a transposition, then π is odd, and we have that sgn(π) = −1 = (−1)r .
Induction step: Fix a positive integer r, and assume that for any permutation
π ∈ Sn , if π is the composition of r transpositions, then sgn(π) = (−1)r . Now, fix a
permutation π ∈ Sn in Sn such that π can be expressed as the composition of r + 1
transpositions, say π = (a0 a′0 )◦(a1 a′1 )◦· · ·◦(ar a′r ). Then by the induction hypothesis,
π ′ := (a1 a′1 ) ◦ · · · ◦ (ar a′r ) satisfies sgn(π ′ ) = (−1)r . But since π = (a0 a′0 ) ◦ π ′ ,
Proposition 2.3.5 guarantees that sgn(π) = −sgn(π ′ ). So, sgn(π) = −sgn(π ′ ) =
−(−1)r = (−1)r+1 . This completes the induction.

Theorem 2.3.7. Let n ≥ 2 be an integer, and let σ, π ∈ Sn . Then sgn(σ ◦ π) =

sgn(σ)sgn(π).

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 194

Proof. This easily follows from Proposition 2.3.3 and Theorem 2.3.6. Let us give
the details. By Proposition 2.3.3, we can express σ and π as compositions of
transpositions, say

ˆ σ = (s1 s′1 ) ◦ (s2 s′2 ) ◦ · · · ◦ (sk s′k );

ˆ π = (t1 t′1 ) ◦ (t2 t′2 ) ◦ · · · ◦ (tℓ t′ℓ ).

By Theorem 2.3.6(a), we have that sgn(σ) = (−1)k and sgn(π) = (−1)ℓ . On the
other hand, σ ◦ π = (s1 s′1 ) ◦ (s2 s′2 ) ◦ · · · ◦ (sk s′k ) ◦ (t1 t′1 ) ◦ (t2 t′2 ) ◦ · · · ◦ (tℓ t′ℓ ), and so
again by Theorem 2.3.6(a), we have that sgn(σ ◦ π) = (−1)k+ℓ . So, sgn(σ ◦ π) =
(−1)k+ℓ = (−1)k (−1)ℓ = sgn(σ)sgn(π).

2.3.4 The alternating group An

For an integer n ≥ 2, let An be the set of all even permutations in Sn . Let us show
that (An , ◦) is a subgroup of (Sn , ◦), where ◦ is the composition of functions. We
apply Theorem 2.2.9. The identity element of Sn is the identity permutation 1n ,
which is obviously even, and therefore belongs to An . Next, by Theorem 2.3.7, a
composition of two even permutations is even, and consequently, An is closed under ◦.
Finally, by Proposition 2.3.2, the sign of a permutation in Sn is equal to the sign of
its inverse, and in particular, the inverse of an even permutation is even; so, An is
closed under inverses. Theorem 2.2.9 now guarantees that An is indeed a subgroup
of Sn .
Terminology: For an integer n ≥ 2, the group (An , ◦) is called the alternating
group of degree n. Typically, we just say that An is the alternating group of degree
n, and the operation ◦ (composition of functions) is understood from context.
We remark that the set of odd permutations in Sn (n ≥ 2), call it On ,24 does not
form a subgroup of Sn . Indeed, the identity permutation 1n is even and therefore
does not belong to On ; so, by Theorem 2.2.9, On is not a subgroup of Sn .

2.3.5 Symmetries of polygons

If we start with a (not necessarily symmetric, and not necessarily convex) polygon
on n vertices, and we label its vertices 1, . . . , n, then the collection of symmetries
of the polygon can be interpreted as a subgroup of the group Sn . For example, the
group of symmetries of an equilateral triangle with vertices labeled 1, 2, 3 is the entire
group S3 . This corresponds to the identity function (which we can also think about
as rotation by 0◦ ), two rotations, and three reflections, as indicated in the picture
below.
24
On is not standard notation for the set of odd permutations in Sn ; in fact, no standard notation
exists for this set. However, An is indeed the standard notation for the set of even permutations in
Sn .

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 195

3 3 3

120◦ 240◦
1 2 1 2 1 2
(1)(2)(3) (123) (132)

3 3 3

1 2 1 2 1 2
(12)(3) (1)(23) (13)(2)

However, for an isosceles (but non-equilateral) triangle with vertices labeled 1, 2, 3,

where 12 is the base, the group of symmetries is {(1)(2)(3), (12)(3)}, where (1)(2)(3)
is simply the identity, and (12)(3) is the reflection about the axis passing through
the vertex 3 and the midpoint of the base 12.

3 3

1 2 1 2
(1)(2)(3) (12)(3)

Finally, if we have a triangle with three sides of different length, and with vertices
labeled 1, 2, 3, then its group of symmetries is just {(1)(2)(3)}, i.e. its only element
is the identity permutation.
3

1 2
(1)(2)(3)

In the case of a square, we have eight symmetries: the identity (which we can also
think of as rotation by 0◦ ), three rotations, and four reflections, shown below. The
resulting subgroup of S4 has 8 elements, whereas the group S4 itself has 4! = 24
elements. Of course, if we relabeled the vertices, we would get a different group.
However, it would be the same as the group that we obtained with this labeling, up
to a relabeling of the elements (vertices). The technical term is “isomorphic”: the
new group would be isomorphic to the old one.25
25
We will study vector space isomorphism in chapter 4, but we will not go into group isomorphism
in any detail.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 196

4 3 4 3 4 3 4 3

90◦ 180◦ 270◦

1 2 1 2 1 2 1 2

(1)(2)(3)(4) (1234) (13)(24) (1432)

4 3 4 3 4 3 4 3

1 2 1 2 1 2 1 2

(1)(24)(3) (13)(2)(4) (12)(34) (14)(23)

We can, of course, also consider symmetries of polygons with more than three
vertices. The regular n-gon has 2n symmetries: it has n rotations (by nk 360◦
counterclockwise for k = 0, 1, . . . , n − 1, where for k = 0, we simply get the identity)
and n reflections. If n is odd, then the reflections are about axes passing through one
vertex and the midpoint of opposite side, and if n is even, then n2 many reflections
are through two opposite vertices, and the remaining n2 reflections are through the
midpoints of the opposite sides. (This is illustrated below for the cases when n = 5
and n = 6). Note that the symmetric group Sn has n! many elements, whereas the
group of symmetries of a regular n-gon has only 2n symmetries.

Terminology and notation: The group of symmetries of a regular polygon is

called the dihedral group of that polygon. Unfortunately, notation is not entirely
consistent in the literature. Some texts denote the dihedral group of the regular
n-gon (n ≥ 3) by Dn , whereas others denote it by D2n .

It is also possible to consider non-regular, and even non-convex polygons. For

example, for the polygon shown in the picture below, the group of symmetries has
four elements, as in the picture.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 197

5 4 5 4 5 4 5 4

180◦

6 3 6 3 6 3 6 3

1 2 1 2 1 2 1 2
(1)(2)(3)(4)(5)(6) (14)(25)(36) (12)(36)(45) (15)(24)(3)(6)

2.3.6 Inversions
In this subsection, we give another way of computing the sign of a permutation. Let
n be a positive integer. An inversion of a permutation π ∈ Sn is an ordered pair
(i, j) of numbers in {1, . . . , n} such that i < j and π(i) > π(j).

Example 2.3.8. The permutation

 
1 2 3 4 5 6 7
π =
2 1 3 6 5 4 7

in S7 has the following four inversions: (1, 2), (4, 5), (4, 6), (5, 6).

By Theorem 2.3.9 (below), the parity of a permutation is equal to the parity of

the number of its inversions. Since the permutation π from Example 2.3.8 has an
even number of inversions, Theorem 2.3.9 implies that π is even, i.e. sgn(π) = 1.
Note that the permutation π from Example 2.3.8 can be also written in the form
π = (12)(3)(46)(5)(7), and so by definition, its sign is sgn(π) = (−1)7−5 = (−1)2 = 1,
which is the same as what we got using inversions. Let us now formally state and
prove the theorem.

Theorem 2.3.9. Let n be a positive integer. Then all permutations π ∈ Sn satisfy

sgn(π) = (−1)r , where r is the number of inversions of π.

Proof. We proceed by induction on the number r of inversions.

Base case: r = 0. The only permutation with no inversions is the identity
permutation,26 and its sign is 1. Since (−1)0 = 1, this is what we needed.
26
This is “obvious,” but let us give the details. Fix a permutation π ∈ Sn that has no inversions.
Then, in particular, we have that π(1) < π(2) < · · · < π(n). Since π(1), . . . , π(n) all belong to the
n-element set {1, . . . , n}, it follows that π(1) = 1, . . . , π(n) = n, i.e. π is the identity permutation.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 198

Induction step: Fix a non-negative integer r, and assume inductively that any
permutation in Sn that has exactly r inversions has sign (−1)r . We must show that
any permutation in Sn that has exactly r + 1 inversions has sign (−1)r+1 .
Fix a permutation π ∈ Sn , and assume that it has exactly r + 1 inversions. (Note
that this implies that n ≥ 2.) In particular, π has at least one inversion, and it
follows that there exists some p ∈ {1, . . . , n − 1} such that (p, p + 1) is an inversion
of π (otherwise, we would have that π(1) < π(2) < · · · < π(n), and then π would
be the identity permutation, contrary  to the fact that  it has at least one inversion).
Now, consider the transposition τ := π(p)π(p + 1) in Sn , and set π ′ := τ ◦ π, so
that
 
1 ... p−1 p p+1 p+2 ... n
π′ =
π(1) . . . π(p − 1) π(p + 1) π(p) π(p + 2) . . . π(n)

Then π ′ has exactly r inversions, i.e. exactly one inversion less than π has. To see
this, we note the following:

ˆ inversions (i, j) of π such that i, j ∈

/ {p, p + 1} are still inversions of π ′ ;

ˆ inversions of the form (i, p) of π correspond to inversions (i, p + 1) of π ′ ;

ˆ inversions of the form (i, p + 1) of π, where i < p, correspond to inversions

(i, p) of π ′ ;

ˆ inversions of the form (p, j) of π, where p + 1 < j, correspond to inversions

(p + 1, j) of π ′ ;

ˆ inversions of the form (p + 1, j) of π correspond to inversions (p, j) of π ′ ;

ˆ π ′ has no other inversions, and in particular (p, p + 1) is not an inversion of π ′ .

But now
by the induction hypothesis,
(−1)r = sgn(π ′ )
since π ′ has exactly r inversions

= sgn(τ ◦ π) because π ′ = τ ◦ π

by Proposition 2.3.5,
= −sgn(π)
since τ is a transposition,

and it follows that sgn(π) = (−1)r+1 . This completes the induction.

Remark: In the induction step of the proof of Theorem 2.3.9, it was important
that we chose an inversion of the form (p, p + 1), and not just any inversion of our

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 199

permutation π. To explain why, let us take a look at an example. Consider the

permutation  
1 2 3 4 5 6 7
π =
2 1 3 6 5 4 7
from Example
 2.3.8. We
 could choose the inversion (4, 5), and consider the transpo-
sition τ := π(4)π(5) = (65) = (56) and the permutation
 
1 2 3 4 5 6 7
π ′ := τ ◦ π = (56) ◦
2 1 3 6 5 4 7
 
1 2 3 4 5 6 7
=
2 1 3 5 6 4 7

Note that π ′ has three inversions,27 whereas π has four.28 If we had, instead, chosen
an arbitrary inversion of π, then the number of inversions would not necessarily
decrease by one, and we could not apply the induction hypothesis. Indeed, suppose
we chose the inversion (4, 6) of our  permutation
 π from Example 2.3.8 and then
′
considered the transposition τ := π(4)π(6) = (64) = (46) and the permutation
 
1 2 3 4 5 6 7
π ′′ := τ′ ◦ π = (46) ◦
2 1 3 6 5 4 7
 
1 2 3 4 5 6 7
= .
2 1 3 4 5 6 7

Note that π ′′ has only one inversion (namely, (1, 2)), whereas π has four.

2.3.7 Permutation matrices

A permutation matrix is a square matrix that has exactly one 1 in each row and each
column, and has 0’s everywhere else. Below are all the possible 3 × 3 permutation
matrices.      
1 0 0 1 0 0 0 0 1
 0 1 0   0 0 1   0 1 0 
0 0 1 0 1 0 1 0 0
     
0 1 0 0 1 0 0 0 1
 1 0 0   0 0 1   1 0 0 
0 0 1 1 0 0 0 1 0
27
Those inversions are (1, 2), (4, 6), (5, 6).
28
Those inversions are (1, 2), (4, 5), (4, 6), (5, 6), as we saw in Example 2.3.8.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 200

Remark: The 0’s and 1’s in permutation matrices may belong to any field F of our
choice. Of course, we have not formally studied fields yet, and so for now, you may
assume that the entries of our permutation matrices belong to one of the fields F
that we already know, namely, Q, R, C, or Zp for some prime number p. As usual,
in what follows, we will denote by e1 , . . . , en the standard basis vectors of Fn , where
F is the field to which the 0’s and 1’s of our permutation matrices belong. The
important point, though, is that in the remainder of this section, we will never need
to add two non-zero numbers, and whenever we multiply two numbers, at least one
of the two numbers will be 0 or 1. So, it does not matter which particular field we
are working in, and therefore, for the remainder of the section, we will not emphasize
this.
Obviously, identity matrices are permutation matrices. Moreover, n × n permuta-
tion matrices are precisely the matrices that can be obtained from the identity matrix
In by reordering (i.e. permuting) rows, or alternatively, by reordering (i.e. permuting)
columns. So, the columns of an n × n permutation matrix are the standard basis
vectors e1 , . . . , en (appearing in some order in that matrix), whereas the rows are
eT1 , . . . , eTn (again, appearing in some order in that matrix).
For a positive integer n and a permutation π ∈ Sn , we define the matrix ofthe
permutation π, denoted by Pπ , to be the n × n matrix that has 1 in the i, π(i) -th
entry for each each index i ∈ {1, . . . , n}, and has 0 in all other entries. In other
words, for each index i ∈ {1, . . . , n}, the i-th row of the matrix Pπ is eTπ(i) . For
example, for the permutation
 
1 2 3 4 5 6
π = ,
2 4 1 6 5 3

in S6 , we obtain the 6 × 6 permutation matrix

 
0 1 0 0 0 0
 0 0 0 1 0 0 
 
 1 0 0 0 0 0 
Pπ =   0
.
 0 0 0 0 1 

 0 0 0 0 1 0 
0 0 1 0 0 0

Obviously, for a positive integer n, the matrix of the identity permutation 1n in Sn

is precisely the identity matrix In , i.e. P1n = In .
Our next proposition states that, as we would expect, matrices of permutations
are indeed permutation matrices (i.e. they have exactly one 1 in each row and each
column, and they have 0’s elsewhere).
Proposition 2.3.10. Let n be a positive integer, and let π ∈ Sn . Then Pπ is a
permutation matrix.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 201

Proof. Obviously, Pπ is an n×n matrix, all of whose entries are 0’s and 1’s. Moreover,
by the definition of Pπ , we have that for each index i ∈ {1, . . . , n}, the i-th row of Pπ
is the row vector eTπ(i) . So, Pπ has exactly one 1 in each row. Note that this means
that the matrix Pπ has exactly n entries that are 1, whereas all the other entries are
0’s.
It remains to show that the matrix Pπ has exactly one 1 in each column. Since
Pπ has exactly n many 1’s, it is enough to show that no column has more than one
1. Since the rows of Pπ (from top to bottom) are eTπ(1) , . . . , eTπ(n) , and since all those
row vectors are pairwise distinct (because π is a permutation), we see that no two
rows of Pπ have a 1 in the same position. So, no column of Pπ has more than one 1,
and we are done.

Remark: By Proposition 2.3.10, the matrix of a permutation is a permutation

matrix. What about the converse: is every permutation matrix the matrix of some
permutation? The answer to this question is “yes,” and it follows from a simple
counting argument. Let n be a positive integer. The n × n permutation matrices
are precisely those n × n matrices whose columns are the standard basis vectors
e1 , . . . , en , appearing in some order. There are n! many ways to order the vectors
e1 , . . . , en , and consequently, there are n! many n × n permutation matrices. On the
other hand, |Sn | = n!, and consequently, there are n! many matrices of permutations
in Sn (we are using the fact that different permutations have different matrices). So,
the number of n × n permutation matrices is the same as the number of matrices of
permutations in Sn . It now follows from Proposition 2.3.10 that n × n permutation
matrices are precisely the matrices of permutations in Sn .

Proposition 2.3.11. Let n be a positive integer, and let π ∈ Sn be a permutation.

Then both the following hold:

(a) for all indices i ∈ {1, . . . , n}, we have that eTi Pπ = eπ(i) , i.e. the i-th row of Pπ
is eTπ(i) ;

(b) for all indices j ∈ {1, . . . , n}, we have that Pπ ej = eπ−1 (j) , i.e. the j-th column
of Pπ is eπ−1 (j) .

Consequently, in terms of its rows and columns, Pπ can be written as follows:

eTπ(1)
 

=  ...  =
 
Pπ eπ−1 (1) . . . eπ−1 (n) .
 

eTπ(n)

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 202

Proof. The last statement of the proposition follows immediately from (a) and (b).29
So, it is enough to prove (a) and (b).
(a) Fix an index i ∈ {1, . . . , n}. By Proposition 1.8.2, eTi Pπ is precisely the i-th
row of the matrix Pπ , and by the definition of the matrix Pπ , its i-th row is precisely
eπ(i) .
(b) Fix an index j ∈ {1, . . . , n}. By Proposition 1.4.5, Pπ ej is precisely the j-th
column of the matrix Pπ . Set i := π −1 (j), so that j = π(i). By (a), the i-th row
of Pπ is the row vector eTπ(i) = eTj . So, Pπ has 1 in its (i, j)-th entry. Since Pπ is a
permutation matrix (by Proposition 2.3.10), and therefore has exactly one 1 in each
column, it follows that the j-th column of Pπ is ei = eπ−1 (j) .

Propositions 2.3.12 and 2.3.13 (below) readily follow from Proposition 2.3.11.
Proposition 2.3.12. Let n be a positive integer, and let π ∈ Sn . Then
Pπ−1 = PπT .
Proof. We have that
eTπ−1 (1)
 
(∗)
  T .. (∗)
PπT = eπ−1 (1) . . . eπ−1 (n) =  = Pπ−1 ,
 
. 
T
eπ−1 (n)

where both instances of (*) follow from Proposition 2.3.11.

Proposition 2.3.13. Let n be a positive integer, and let σ and π be permutations

in Sn . Then
Pσ◦π = Pπ Pσ .
Remark: Note that swapping of order of σ and π: Pσ◦π = Pπ Pσ .

Proof. It suffices to show that matrices Pσ◦π and Pπ Pσ have the same corresponding
rows. Fix an index i ∈ {1, . . . , n}. By Proposition 1.8.2, the i-th row of the matrix
Pσ◦π is eTi Pσ◦π , and the i-th row of the matrix Pπ Pσ is eTi (Pπ Pσ ). So, we just need
to show that eTi Pσ◦π = eTi (Pπ Pσ ). But follows easily via repeated application of
Proposition 2.3.11(a). Indeed, we have that
(∗) (∗) (∗)
eTi (Pπ Pσ ) = (eTi Pπ )Pσ = eTπ(i) Pσ = e
= eT
(σ◦π)(i) = eTi Pσ◦π ,
σ π(i)

where all three instances of (*) follow from Proposition 2.3.11(a).

29
The “last statement of the proposition” is the statement that
 T
eπ(1)


Pπ =  ..  
eπ−1 (1) . . . eπ−1 (n) .
 =
 
.
eTπ(n)

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 203

Theorem 2.3.14 (below) easily follows from Propositions 2.3.12 and 2.3.13, and it
states that permutation matrices are invertible, and moreover, that the inverse of a
permutation matrix is equal to the transpose of that permutation matrix.

Theorem 2.3.14. Let n be a positive integer, and let π ∈ Sn . Then Pπ is invertible,

and moreover,
Pπ−1 = Pπ−1 = PπT .

Proof. The fact that Pπ−1 = PπT follows immediately from Proposition 2.3.12. It
remains to show that Pπ is invertible, and that its inverse is Pπ−1 . We will denote
the identity permutation in Sn by 1n , so that π ◦ π −1 = 1n and π −1 ◦ π = 1n . We
now compute:
(∗)
Pπ Pπ−1 = Pπ−1 ◦π = P1n = In ,
where (*) follows immediately from Proposition 2.3.13. Analogously, Pπ−1 Pπ = In .
So, Pπ and Pπ−1 are invertible and are each other’s inverses. This completes the
argument.

Remark: A matrix Q ∈ Rn×n is orthogonal if it satisfies QT Q = In . We will

study orthogonal matrices in chapter 6 (see section 6.8). For now, we note that
Theorem 2.3.14 guarantees that permutation matrices are orthogonal (as long as we
consider the 0’s and 1’s in those matrices as belonging to R, rather than to some
other field).

Permuting the rows and columns of a matrix. As our next theorem shows,
multiplying a matrix by a permutation matrix on the left permutes the rows of the
original matrix. On the other hand, multiplying a matrix by a permutation matrix
on the right permutes the columns of the original matrix. More precisely, we have
the following.
 
r1
Theorem 2.3.15. Let A =  ...  = a1 . . . am be an n × m matrix with
   

rn
entries in some field F.30 Then all the following hold:

(a) for all π ∈ Sn , we have that

 
rπ(1)
Pπ A =  ...  ;
 

rπ(n)
30
Since we have not formally studied fields yet, you may assume for now that F is one of the fields
that you are already familiar with, namely, Q, R, C, or Zp for some prime number p. However,
Theorem 2.3.15 is true for all fields F, not just those particular ones.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 204

(b) for all π ∈ Sm , we have that

 
APπ = aπ−1 (1) . . . aπ−1 (m) ;

(c) for all π ∈ Sm , we have that

APπT
 
= aπ(1) . . . aπ(m) .

Proof. We first prove (b). Fix any permutation π ∈ Sm . In what follows, e1 , . . . , em

are the standard basis vectors of Fm . We compute:
 
APπ = A eπ−1 (1) . . . eπ−1 (m) by Proposition 2.3.11

  by the definition of
= Aeπ−1 (1) . . . Aeπ−1 (m)
matrix multiplication
 
= aπ−1 (1) . . . aπ−1 (m) by Proposition 1.4.5.

This proves (b).

For (c), we note that for any permutation π ∈ Sm , we have that
(∗)
APπT = APπ−1

(∗∗)  
= a(π−1 )−1 (1) . . . a(π−1 )−1 (m)

(∗∗∗)  
= aπ(1) . . . aπ(m) ,

where (*) follows from Proposition 2.3.12, (**) follows from (b) applied to the matrix
A and the permutation π −1 , and (***) follows from the fact that (π −1 )−1 = π.
It remains to prove (a). Fix any permutation π ∈ Sn . We first consider the
matrix (Pπ A)T , and we compute:

(Pπ A)T = AT PπT by Proposition 1.8.1(d)

rT1 . . . rTn PπT

 
=

h i by (c), applied to the matrix

= rTπ(1) . . . rTπ(n) AT = rT1 . . . rTn and
the permutation π
 
rπ(1) 
 ..  T

=  .  .
rπ(n)

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 205

By taking the transpose of both sides, we obtain

 
rπ(1)
Pπ A =  ...  ,
 

rπ(n)

which is what we needed. This proves (a).

2.4 Fields
2.4.1 Fields: definition, examples, and basic properties
A field is an ordered triple (F, +, ·), where F is a set, and + and · are binary operations
on F (i.e. functions from F × F to F), called addition and multiplication, respectively,
satisfying the following axioms:

1. addition and multiplication are associative, that is, for all a, b, c ∈ F, we have
that a + (b + c) = (a + b) + c and a · (b · c) = (a · b) · c;

2. addition and multiplication are commutative, that is, for all a, b ∈ F, we have
that a + b = b + a and a · b = b · a;

3. there exist distinct elements 0F , 1F ∈ F such that for all a ∈ F, a + 0F = a

and a · 1F = a; 0F is called the additive identity of F, and 1F is called the
multiplicative identity of F;

4. for every a ∈ F, there exists an element in F, denoted by −a and called the

additive inverse of a, such that a + (−a) = 0F ;

5. for all a ∈ F \ {0F }, there exists an element in F, denoted by a−1 and called
the multiplicative inverse of a, such that a · a−1 = 1F ;

6. multiplication is distributive over addition, that is, for all a, b, c ∈ F, we have

that a · (b + c) = (a · b) + (a · c).

Example 2.4.1. All the following are fields:

1. (Q, +, ·); 2. (R, +, ·); 3. (C, +, ·).

Remark: Note that (Z, +, ·) is not a field. This is because elements of Z \ {−1, 0, 1}
do not have multiplicative inverses. As we shall see, (Zp , +, ·) is a field for every
prime number p (see Theorem 2.4.3).

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 206

Notation:

ˆ If operations + and · are understood from context, then we typically just say
“field F” instead of “field (F, +, ·).”

ˆ For a, b ∈ F, we typically write ab instead of a · b, and we typically write a − b

instead of a + (−b).

ˆ As usual, unless parentheses indicate otherwise, we perform multiplication

before performing addition. So, for a, b, c ∈ F, we write ab + c instead of
(a · b) + c, and similarly, we write a + bc instead of a + (b · c).

Remarks:

ˆ Axioms 1, 2, and 3 above imply that (F, +) and (F, ·) are monoids with identity
elements 0F and 1F , respectively. Proposition 2.1.1 guarantees that 0F and 1F
are unique.

– When there is no danger of confusion, we write 0 and 1 instead of 0F and

1F , respectively.

ˆ Axioms 1, 2, 3, and 4 imply that (F, +) is an abelian group with identity

element 0F . By Proposition 2.2.1, this implies that each element a ∈ F has a
unique additive inverse −a.

ˆ By Proposition 2.4.2 (below), for any a, b ∈ F \ {0F }, we have ab ̸= 0F , i.e.

ab ∈ F \ {0F }. This, together with axioms 1 and 3, implies that (F \ {0F }, ·)
is a monoid with identity element 1F . Next, by Proposition 2.4.2, and by
axioms 2 (commutativity of addition) and 3 (0F = ̸ 1F ), we have that we have
that a0F = 0F a = 0F ̸= 1F . This, together with axiom 5 implies that the
multiplicative inverse of any element a ∈ F \ {0F } also belongs to F \ {0F }. So,
(F \ {0F }, ·) is an abelian group with identity element 1F . By Proposition 2.2.1,
it follows that every element a ∈ F \ {0F } has a unique multiplicative inverse
a−1 .

ˆ By axioms 2 and 6, for all a, b, c ∈ F, we have that (b + c) · a = (b · a) + (c · a),

or written in a simplified manner, (b + c)a = ba + ca.31

Proposition 2.4.2. Let (F, +, ·) be a field. Then all the following hold:

(a) for all a ∈ F, 0a = a0 = 0;

(b) for all a, b ∈ F, if ab = 0, then a = 0 or b = 0;

31 ax. 2. ax. 6. ax. 2.
Indeed, for a, b, c ∈ F, we have that (b + c)a = a(b + c) = ab + ac = ba + ca.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 207

(c) for all a ∈ F, (−1)a = −a.32

Proof. We first prove (a). Fix a ∈ F. Since multiplication in the field F is com-
mutative, we know that 0a = a0. So, it suffices to show that a0 = 0. First, note
that
(∗) (∗∗)
a0 = a(0 + 0) = a0 + a0,
where (*) follows from the fact that 0 + 0 = 0 (because 0 is the additive identity of
the field), and (**) follows from axiom 6 of the definition of a field. We have now
established that a0 = a0 + a0, and it follows that

because −(a0) is the

0 = −(a0) + a0
additive inverse of a0

because a0 = a0 + a0
= −(a0) + (a0 + a0)
(proven above)

= (−(a0) + a0) + a0 because + is associative

because −(a0) is the

= 0 + a0
additive inverse of a0

because 0 is the additive

= a0
identity of the field F.

Thus, a0 = 0. This proves (a).

Next, we prove (b). Fix a, b ∈ F such that ab = 0. We may assume that b ̸= 0, for
otherwise we are done. But now b has a multiplicative inverse b−1 , and we compute:
(∗) (∗∗) (∗∗∗)
a = a · 1 = a(bb−1 ) = (ab)b−1 = 0b−1 = 0,

where (*) follows from the associativity of multiplication, (**) follows from the fact
that ab = 0, and (***) follows from (a).
It remains to prove (c). Fix a ∈ F. First, we have that
(∗)
0 = 0a = (1 − 1)a = 1a + (−1)a = a + (−1)a,

where (*) follows from (a). Consequently,

32
This statement may require some clarification. Here, −a is the additive inverse of a. On the
other hand, (−1)a is the product of −1 (the additive inverse of the multiplicative identity) and a.
So, −a is not simply a shorthand for (−1)a. The two quantities are indeed equal, but this requires
proof!

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 208

because 0 is the additive

−a = −a + 0
identity of the field F

because 0 = a + (−1)a
= −a + (a + (−1)a)
(proven above)

= (−a + a) + (−1)a because + is associative

because −a is the additive

= 0 + (−1)a
inverse of a

because 0 is the additive

= (−1)a
identity of the field F.

This proves (c).

2.4.2 Finite fields

In this subsection, we show that (Zp , +, ·) is a field for all prime numbers p (see The-
orem 2.4.3 below). We will use Fermat’s Little Theorem, proven in subsection 0.2.2,
and restated below for the reader’s convenience.

Fermat’s Little Theorem. If p ∈ N is a prime number and a ∈ Zp \ {0}, then

ap−1 = 1.

Theorem 2.4.3. For every prime number p, (Zp , +, ·) is a field.

Proof. By Proposition 0.2.11, addition and multiplication are associative and com-
mutative in Zp , and multiplication is distributive over addition in Zp . So, (Zp , +, ·)
satisfies axioms 1, 2, and 6 from the definition of a field. Further, 0 := [0]p is the
additive identity and 1 := [1]p is the multiplicative identity of (Zp , +, ·). Moreover,
̸ [1]p , since 0 ̸≡ 1 (mod p).33 Thus, (Zp , +, ·) satisfies axiom 3 from the
[0]p =
definition of a field. Further, for all a ∈ Z, the additive inverse of [a]p in (Zp , +, ·) is
[−a]p , and so axiom 4 is satisfied. Finally, by Fermat’s Little Theorem, every number
a ∈ Zp \ {0} has a multiplicative inverse, namely, ap−2 , and it follows that axiom 5
is satisfied. This proves that (Zp , +, ·) is indeed a field, which is what we needed to
show.

Remark: For a positive integer n that is not prime, (Zn , +, ·) is not a field. If n = 1,
then this follows from the fact that Zn = Z1 has only one element, whereas every field
has at least two elements (namely, the additive and multiplicative identities, which
cannot be equal by axiom 3 of the definition of a field). Now, let us suppose that n ≥ 2
33
This follows from the fact that p ≥ 2.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 209

is composite, say n = pq where p, q ≥ 2 are integers. Then [p]n [q]n = [pq]n = [n]n = 0.
So, if (Zn , +, ·) were a field, Proposition 2.4.2(b) would imply that at least one of
[p]n and [q]n is 0, a contradiction.

Finally, we state the following theorem without proof.

Theorem 2.4.4. Let n ≥ 2 be an integer. Then there exists a field of size n if and
only if n is a power of a prime.34 Moreover, if n is a power of a prime, then up to
“isomorphism” (i.e. up to renaming the operations and elements of the field), there is
exactly one field of size n, and it is denoted by Fn .35

Proof. Omitted.

Remark: For a prime number p, we have that Fp = Zp . However, if n = pm , where

p is a prime number and m ≥ 2 is an integer, then Fn ̸= Zn (this is because Fn is a
field, but by the Remark following the proof of Theorem 2.4.3, Zn is not a field).

2.4.3 The fraction notation in fields

Let F be a field. For a ∈ F \ {0}, we sometimes use the notation a1 instead of a−1 (the
multiplicative inverse of a in the field F). For instance, in Z3 , we have 11 = 1−1 = 1
and 12 = 2−1 = 2 (because in Z3 , we have that 2 · 2 = 1). In a similar vein, for scalars
a, b ∈ F such that b ≠ 0, we sometimes write ab instead of b−1 a. For example, in
Z5 , we have that 3−1 = 2 (because 3 · 2 = 1), and so 43 = 3−1 · 4 = 2 · 4 = 3. It is
sometimes more convenient to use the notation a1 instead of a−1 , and ab instead of
b−1 a. However, when working over a finite field such as Zp (for a prime number p),
we never leave a fraction as a final answer, and instead, we always simplify.

2.4.4 The characteristic of a field

The characteristic of a field F is the smallest positive integer n (if it exists) such
that, in the field F, we have that

1 + · · · + 1 = 0,
| {z }
n

where the 1’s and the 0 are understood to be in the field F. If no such n exists, then
char(F) := 0. Note that fields Q, R, and C all have characteristic 0. On the other
hand, for all prime numbers p, we have that char(Zp ) = p.

Theorem 2.4.5. The characteristic of any field is either 0 or a prime number.

34
“n is a power of a prime” means that there exists some prime number p and a positive integer
m such that n = pm .
35
Technically, the field is (Fn , +, ·), but we typically write just Fn .

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 210

Proof. Let F be a field. We may assume that char(F) ̸= 0, for otherwise we are done.
So, char(F) is a positive integer. By the definition of a field, we have that 1 ̸= 0,36
and so char(F) ≥ 2. Now, suppose that char(F) is not prime, and fix integers p, q ≥ 2
such that char(F) = pq. Then
  
1 + ··· + 1 1 + ··· + 1 = 1 + · · · + 1 = 0.
| {z } | {z } | {z }
p q pq

Since F is a field, Proposition 2.4.2(b) guarantees that at least one of the numbers
1 + · · · + 1 and 1 + · · · + 1 is zero. But this is impossible since 0 < p, q < char(F).
| {z } | {z }
p q

2.4.5 Algebraically closed fields

Polynomials. Given a field F, a polynomial with coefficients in F is an expression
of the form
p(x) = an xn + an−1 xn−1 + · · · + a1 + a0 ,
where n is a non-negative integer, and a0 , a1 , . . . , an−1 , an , called the coefficients of
p(x), are some elements of F. A constant polynomial is a polynomial of the form
p(x) = a0 for some constant (i.e. fixed element of the field) a0 . The zero polynomial
is the constant polynomial

p(x) = 0. The degree of a non-zero polynomial p(x),
denoted by deg p(x) , is the largest non-negative integer n for which the coefficient
in front of xn is non-zero. As a convention, the degree of the zero polynomial is
defined to be −∞. Two polynomials are equal if their corresponding coefficients are
equal.
Essentially, the polynomials that we study are the same as the ones that you
saw in high school, except that the coefficients may belong to an arbitrary field. In
particular, we may add, subtract, and multiply polynomials. This is done in the
usual way, except that we have to keep in mind which particular field we are working
over. For example:

ˆ x2 + x + 2x + 1 = x2 + 3x + 1 when the coefficients are considered to be

in R;

ˆ x2 + x + 2x + 1 = x2 + 1 when the coefficients are considered to be in Z3 .

Roots of polynomials. For a polynomial p(x) with coefficients in a field F, a

root of p(x) is a number a ∈ F such that p(a) = 0. For example, the polynomial
p(x) = x2 − 3x + 2, seen as a polynomial with coefficients in R, has two real roots,
namely, 1 and 2. We could have computed these roots using the familiar quadratic
equation, and we can check that they really are roots of our polynomial by computing
36
Here, 1 and 0 are understood to be in the field F.

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 211

p(1) = 12 − 3 · 1 + 2 = 0 and p(2) = 22 − 3 · 2 + 2 = 0, where the computation was

in R.
Now, consider the polynomial

q(x) = x2 + 1.

Does q(x) have a root? If we consider q(x) to be a polynomial with real coefficients
and we ask if it has real roots, then the answer is that it has none. If we consider q(x)
to be a polynomial with complex coefficients, then we see that it has two complex
roots, namely i and −i. On the other hand, if we consider q(x) to be a polynomial
with coefficients in Z2 , then we see that 1 is a root, since q(1) = 12 + 1 = 0 (in Z2 ).
If we consider q(x) to be a polynomial with coefficients in Z3 , and we ask if it has
any roots in Z3 , then we see that the answer is “no”: none of 0, 1, 2 is a root, as we
can see by direct computation.

Algebraically closed fields. An algebraically closed field is a field F that has the
property that every non-constant polynomial with coefficients in F has a root in F.
By the Fundamental Theorem of Algebra (see subsection 0.3.2), C is an algebraically
closed field. On the other hand, R is not algebraically closed (as our example above
demonstrates), and similarly, neither is Q. Moreover, no finite field is algebraically
closed. To see this, consider any finite field F = {f1 , . . . , ft } (t ≥ 2), and consider
the polynomial
p(x) = (x − f1 ) . . . (x − ft ) + 1,
which is a polynomial of degree t with coefficients in F. Then for each i ∈ {1, . . . , t}, we
have that p(fi ) = 1, and consequently, fi is not a root of p(x). Since F = {f1 , . . . , ft },
we see that p(x) has no roots in F.
Thus, of the fields that we have seen so far, namely, Q, R, C, and Zp (where p is
a prime number), only the field C is algebraically closed. Other algebraically closed
fields do exist, but they will not be discussed in these lecture notes.

Factoring polynomials into linear terms. It can be shown (though we will

not give a formal proof) that any non-constant polynomial with coefficients in
an algebraically closed field F can be factored into linear terms in a unique way.
More precisely, if p(x) is a polynomial of degree n ≥ 1, and with coefficients in an
algebraically closed field F, then there exist numbers a, α1 , . . . , αℓ in F such that
a ̸= 0 and such that α1 , . . . , αℓ are pairwise distinct, and positive integers n1 , . . . , nℓ
satisfying n1 + · · · + nℓ = n, such that

p(x) = a(x − α1 )n1 . . . (x − αℓ )nℓ .

Moreover, a, α1 , . . . , αℓ , n1 , . . . , nℓ are uniquely determined by the polynomial p(x),

up to a permutation of the αi ’s and the corresponding ni ’s. Here, a is the leading
coefficient of p(x), i.e. the coefficient in front of xn . Numbers α1 , . . . , αℓ are the roots

Irena Penev
Linear Algebra 1 & 2
Chapter 2. Groups and permutations. Fields 212

of p(x) with multiplicities n1 , . . . , nℓ , respectively. If we think of each αi as being a

root “ni times” (due to its multiplicity), then we see that the n-th degree polynomial
p(x) has exactly n roots in F. This is often summarized as follows: “every n-th
degree polynomial (with n ≥ 1) with coefficients in an algebraically closed field has
exactly n roots in that field, when multiplicities are taken into account.”
When a field F is not algebraically closed, then some of its non-constant polyno-
mials can be factored into linear terms, while others cannot.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 213

Chapter 3

Vector spaces

3.1 Vector spaces

Let F be a field with additive identity 0 and multiplicative identity 1. In what follows,
we shall refer to elements of F as scalars. A vector space (or linear space) over the
field F is a set V , together with a binary operation + on V (called vector addition)
and an operation · : F × V → V (called scalar multiplication), satisfying the following
axioms:

1. (V, +) is an abelian group; the identity element of (V, +) is denoted by 0 (“zero

vector”), and for any vector v ∈ V , the inverse of v in (V, +) is dented by −v;

2. for all vectors v ∈ V , we have 1v = v;

3. for all vectors v ∈ V and scalars α, β ∈ F, we have (α + β)v = αv + βv;

4. for all vectors v ∈ V and scalars α, β ∈ F, we have (αβ)v = α(βv);

5. for all vectors u, v ∈ V and scalars α ∈ F, we have α(u + v) = αu + αv.

Terminology: A real vector space is a vector space over the field R, and a complex
vector space is a vector space over the field C.

Example 3.1.1. Let F be a field. Then all the following are vector spaces over F (in
each case, vector addition and scalar multiplication are defined in the natural way):

1. Fn ;

2. Fn×m ;

3. the set of all functions from F to F;

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 214

4. the set PF of all polynomials (in one varialble, typically x) with coefficients in
the field F;1

ˆ Notation: Some texts use the notation F[x] instead of PF (if x is the
variable used in the polynomials in question).

5. for a non-negative integer n, the set PnF of all polynomials of degree at most n
and with coefficients in F.2

Note that each of the cases above, elements of our vector space are considered vectors
(even if they do not “look like” vectors, i.e. even if they are matrices, functions, or
polynomials).

Remark: In these lecture notes, we will see quite a few examples with polynomials,
especially in chapter 4. So, a remark on when two polynomials are equal is in order.
For a field F, two polynomials in PF are equal precisely when their corresponding
coefficients are the same.3 For instance, polynomials p1 (x) = x2 + 2x + 3 and
p1 (x) = 2x2 − x + 3 in PR are not equal, and we write p1 (x) ̸= p2 (x). This is because
p1 (x) and p2 (x) do not have the same corresponding coefficients. One might object
that p1 (0) = p2 (0) and p1 (3) = p2 (3), and so p1 (x) and p2 (x) are “sometimes equal.”
This does not matter. The point is that polynomials p1 (x) and p2 (x) are the different
as polynomials (because they do not have the same corresponding coefficients).
For an even more interesting example, consider polynomials q1 (x) = x4 + x3 and
q2 (x) = x5 + x in PZ2 . Then q1 (x) ̸= q2 (x) (because q1 (x) and q2 (x) do not have
the same corresponding coefficients), even though q1 (0) = q2 (0) and q1 (1) = q2 (1),
and 0 and 1 are the only elements of Z2 ! On the other hand, note that polynomials
r1 (x) = (x + 3)2 and r2 (x) = x(x + 6) + 9 in PR are equal because if we write them
in the standard form,4 we see that they have the same corresponding coefficients.
Indeed, r1 (x) = x2 + 6x + 9 and r2 (x) = x2 + 6x + 9, and so r1 (x) = r2 (x).

If you have studied calculus, here is another example of a vector space.

Example 3.1.2. The following are real vector spaces (with vector addition and
scalar multiplication defined in the usual way):

1. the set of continuous functions from R to R;

2. the set of differentiable functions from R to R.

1
One could also consider polynomials in more than one variable (say, x1 , . . . , xk ) and with
coefficients in F. This, too, is a vector space over F.
2
The notation Pn F is not fully standard (there is no fully standard notation for this), but it is the
notation that we will use in these lecture notes.
3
“Their corresponding coefficients are the same” means that for each non-negative integer k,
the two polynomials have the same coefficient in front of xk . This includes the free coefficient: the
coefficient in front of x0 = 1.
4
That is, in the form an xn + · · · + a1 x + a0 for some coefficients a0 , a1 , . . . , an ∈ F.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 215

We note that for any field F, we have the trivial vector space {0} over the field
F. In this vector space, vector addition and scalar multiplication are defined in
the obvious way: 0 + 0 = 0 and α0 = 0 for all scalars α ∈ F. A vector space is
non-trivial if it contains at least one non-zero vector.

Proposition 3.1.3. Let V be a vector space over a field F. Then all the following
hold:

(a) for all v ∈ V , 0v = 0;5

(b) for all α ∈ F, α0 = 0;

(c) for all v ∈ V and α ∈ F, if αv = 0, then α = 0 or v = 0;

(d) for all v ∈ V , (−1)v = −v.6

Proof. The proof is similar to that of Proposition 2.4.2. We prove (a), and we leave
the rest as an exercise. Fix v ∈ V . Then 0v = (0 + 0)v = 0v + 0v, and consequently,

0 = −(0v) + 0v

= −(0v) + 0v + 0v because 0v = 0v + 0v

= 0 + 0v because −(0v) + 0v = 0

= 0v.

This proves (a).

3.1.1 Vector (linear) subspaces

Let V be a vector space over a field F. A vector subspace (or linear subspace or
simply subspace) of V is a set U ⊆ V such that U is itself a vector space over
F, when equipped with the vector addition and scalar multiplication operations
“inherited” from V .7 This means that we add two vectors of U using the vector
addition operation from V , and similar for scalar multiplication. Moreover, U must
be “closed under” vector addition and scalar multiplication from V , that is, that
for all u1 , u2 ∈ U , we have that u1 + u2 ∈ U and that for all u ∈ U and α ∈ F, we
have that αu ∈ U (where vector addition and scalar multiplication are those from
the vector space V ).
5
Here, 0 is the zero of the field F, and 0 is the zero vector in V .
6
Here, −1 is the additive inverse of the multiplicative identity of the field F, and in particular,
−1 is a scalar. So, (−1)v is the product of the scalar −1 and the vector v. On the other hand, −v
is the additive inverse of the vector v.
7
Note that the field F must be the same for U and V !

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 216

Remark: It is obvious that the subspace relation is transitive. More precisely, for
any vector space V over a field F, if U is a subspace of V , and W is a subspace of U ,
then W is a subspace of V . Informally, we would say that “a subspace of a subspace
is a subspace” (see the picture below).

U
W

Example 3.1.4. Let V be a vector space over a field F. Then V is a subspace of

itself, and {0} is a subspace of V .

Terminology: For a vector space V over a field F, the trivial subspace of V is the
subspace {0}. A non-trivial subspace of V is one that contains at least one non-zero
vector. A subspace U of V is proper if U ⫋ V .

Example 3.1.5. Let n be a positive integer, and let F be a field. Then PnF is a
subspace of PF .

If you have studied calculus, here is another example.

Example 3.1.6. The real vector space of differentiable functions from R to R is a

subspace of the real vector space of continuous functions from R to R, which is in
turn a subspace of the real vector space of all functions from R to R.

Theorem 3.1.7 (below) is a vector space analog of Theorem 2.2.9 (which dealt
with groups). Moreover, our proof of Theorem 3.1.7 relies on Theorem 2.2.9.

Theorem 3.1.7. Let V be a vector space over a field F, and let U ⊆ V . Then U is
a subspace of V if and only if the following three conditions are satisfied:

(i) 0 ∈ U ;8

(ii) U is closed under vector addition, that is, for all u, v ∈ U , we have that
u + v ∈ U;

(iii) U is closed under scalar multiplication, that is, for all u ∈ U and α ∈ F, we
have that αu ∈ U .
8
Here, 0 is the zero vector in the vector space V .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 217

Proof. Suppose first that (i), (ii), and (iii) are satisfied; we must show that U is
a subspace of V . By (ii), the restriction of + to U × U (denoted + ↾ (U × U ), or
just + for simplicity) is a binary operation on U ,9 and by (iii), the restriction of ·
to F × U (denoted by · ↾ (F × U ), or just · for simplicity) is indeed a function from
F × U to U . So, U is equipped with both the vector addition operation and the
scalar multiplication operation. Next, U satisfies axioms 2-5 from the definition of
a vector space because the vector space V satisfies those axioms and because the
vector addition and scalar multiplication operations in U are inherited from V . It
remains to show that U satisfies axiom 1 from the definition of a vector space, that
is, that U is an abelian group under vector addition. Since (V, +) is an abelian
group (because V is a vector space), it suffices to show that (U, +) is a subgroup of
(V, +).10 By (i), we have that 0 ∈ U , and by (ii), we have that U is closed under
vector addition. Moreover, by (iii) and by Proposition 3.1.3(d), for all u ∈ U , we
have that −u = (−1)u ∈ U ,11 and so U is closed under additive (vector) inverses.
Theorem 2.2.9 now guarantees that (U, +) is a subgroup of (V, +). This proves that
U is indeed a subspace of V .
Suppose now that U is a subspace of V ; we must show that (i), (ii), and (iii)
hold. Since the vector addition and scalar multiplication operations of the vector
space U are inherited from the ones for V , we see that (ii) and (iii) hold. Moreover,
since U is a vector space, we know that it contains the zero vector, call it 0U .12
We must show that 0U = 0.13 Since 0U is the identity element of (U, +), we see
that 0U + 0U = 0U . Since 0U ∈ V and 0 is the identity element of V , we see that
0U + 0 = 0U . So, 0U + 0U = 0U + 0. By now adding −0U to both sides of the
equation,14 and we obtain 0U = 0. So, (i) holds.

3.1.2 Linear combinations and linear span

In section 1.4, we introduced the linear span of vectors in Fn (where F is a field).
Here, we generalize the concept to arbitrary vector spaces, as follows.
Suppose that V is a vector space over a field F. Given vectors u1 , . . . , uk ∈ V ,
we say that a vector v ∈ V is a linear combination of u1 , . . . , uk if there exist scalars
α1 , . . . , αk ∈ F such that
v = α1 u1 + · · · + αk uk .
9
In other words, we have that + ↾ (U × U ) : U × U → U .
10
If (U, +) is a subgroup of (V, +), then in particular, (U, +) is a group, and it must be abelian
because (V, +) is abelian.
11
Indeed, by Proposition 3.1.3(d), we have that −u = (−1)u, and by (iii), we have that (−1)u ∈ U .
12
Since U is a subspace of V , we know, in particular, that (U, +) is an abelian group, and
consequently, it contains a (unique) identity element. We call this identity element 0U .
13
Here, 0 is the zero vector in V , and i.e. the identity element of the abelian group (V, +). Since
(U, +) is an abelian group, it must have an identity element, and we call this identity element 0U .
However, could it be that 0U ̸= 0, so that (i) potentially fails? We show that this cannot happen.
We argue similarly as in the proof of Theorem 2.2.9, only with different notation.
14
Here, −0U is the additive inverse of the vector 0U in V .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 218

The linear span (or simply span) of the set of vectors {u1 , . . . , uk }, denoted by
Span({u1 , . . . , uk }) or Span(u1 , . . . , uk ), is the set of all linear combinations of
u1 , . . . , uk , i.e.

Span(u1 , . . . , uk ) = α1 u1 + · · · + αk uk | α1 , . . . , αk ∈ F .

As a convention, we define the “empty sum” of vectors in V to be 0 (the zero vector

in V ),15 and consequently, Span(∅) = {0}.
Given a vector space V over a field F, and given vectors u1 , . . . , uk ∈ V , we say
that {u1 , . . . , uk } is a spanning set of V , or that that the set {u1 , . . . , uk } spans V ,
or that vectors u1 , . . . , uk span V , provided that V = Span(u1 , . . . , uk ). (Note that
∅ is a spanning set of the trivial vector space {0} over a field F.)
   
1 0
Example 3.1.8. Consider vectors e1 =  0  and e2 =  1  in R3 . Then
0 0
 
x1
Span(e1 , e2 ) = { x2  | x1 , x2 ∈ R}. So, Span(e1 , e2 ) is the x1 x2 -plane in the
0
Euclidean space R . 3

x3

e2
0
x2
e1

U = Span(e1, e2)

x1

Example 3.1.9. Consider the polynomials 1, x, x2 in PR . Then Span(1, x, x2 ) =

{a2 x2 + a1 x + a0 | a0 , a1 , a2 ∈ R} = P2R .

 
Remark: By Proposition 1.4.4, for a field F and a matrix A = a1 . . . am in
Fn×m , we have that

Span(a1 , . . . , am ) = Ax | x ∈ Fm .


Consequently, for all vectors b ∈ Fn , we have that b ∈ Span(a1 , . . . , am ) if and

only if the matrix-vector equation Ax = b is consistent. (See also Examples 1.5.5
15
An “empty sum” might be the sum α1 u1 + · · · + αk uk , where k = 0 (and so we do not actually
have any ui ’s or αi ’s).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 219

and 1.5.6 for a couple of numerical examples.) Moreover, as Proposition 3.1.10

(below) shows, checking whether a finite set of vectors in Fn is a spanning set of Fn
is quite straightforward. (However, Proposition 3.1.10 only works for Fn , and not for
arbitrary vector spaces.)

Proposition 3.1.10. Let F be a field, and let a1 , . . . , am (m ≥ 1) be some vectors

in Fn . Set A := a1 . . . am . Then the following are equivalent:


(a) vectors a1 , . . . , am span Fn ;

(b) for all b ∈ Fn , the matrix-vector equation Ax = b is consistent;

(c) rank(A) = n (i.e. A has full row rank).

Proof. By Corollary 1.6.6, (b) and (c) are equivalent. On the other hand, the fact
that (a) and (b) are equivalent essentially follows from the fact that

Span(a1 , . . . , am ) = Ax | x ∈ Fm .


Indeed, we have the following sequence of equivalent statements:

vectors a1 , . . . , am span Fn ⇐⇒ Span(a1 , . . . , am ) = Fn

Ax | x ∈ Fm = Fn

⇐⇒

⇐⇒ ∀b ∈ Fn ∃x ∈ Fm s.t. Ax = b

⇐⇒ ∀b ∈ Fn : Ax = b is consistent.

Thus, (a) and (b) are indeed equivalent. This completes the argument.

Theorem 3.1.11. Let V be a vector space over a field F, and let u1 , . . . , uk ∈ V

(k ≥ 0).16 Then all the following hold:

(a) u1 , . . . , uk ∈ Span(u1 , . . . , uk );

(b) Span(u1 , . . . , uk ) is a subspace of V ;

(c) for all subspaces U of V such that u1 , . . . , uk ∈ U , Span(u1 , . . . , uk ) is a subspace

of U ;

(d) Span(u1 , . . . , uk ) is precisely the intersection of all subspaces of V that contain

the vectors u1 , . . . , uk .
16
If k = 0, then u1 , . . . , uk is an empty list of vectors, the set {u1 , . . . , uk } is empty, and
Span(u1 , . . . , uk ) = {0}.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 220

Proof. To prove (a), we simply observe that for all i ∈ {1, . . . , k}, we have that

ui = 0u1 + · · · + 0ui−1 + 1ui + 0ui+1 + · · · + 0uk ,

and so ui ∈ Span(u1 , . . . , uk ).
Next, we prove (b). It suffices to show that Span(u1 , . . . , uk ) satisfies (i), (ii)
and (iii) from Theorem 3.1.7, that is, that all the following hold:

(i) 0 ∈ Span(u1 , . . . , uk );

(ii) Span(u1 , . . . , uk ) is closed under vector addition, that is, for all vectors v1 , v2 ∈
Span(u1 , . . . , uk ), we have that v1 + v2 ∈ Span(u1 , . . . , uk );

(iii) Span(u1 , . . . , uk ) is closed under scalar multiplication, that is, for all vectors
v ∈ Span(u1 , . . . , uk ) and scalars α ∈ F, we have that αv ∈ Span(u1 , . . . , uk ).

For (i), we simply note that 0 = 0u1 + · · · + 0uk ∈ Span(u1 , . . . , uk ).

Next, we prove (ii). Fix v1 , v2 ∈ Span(u1 , . . . , uk ). Then there exist scalars
α1 , . . . , αk , β1 , . . . , βk ∈ F such that v1 = α1 u1 +· · ·+αk uk and v2 = β1 u1 +· · ·+βk uk .
But now
v1 + v2 = (α1 u1 + · · · + αk uk ) + (β1 u1 + · · · + βk uk )

= (α1 + β1 )u1 + · · · + (αk + βk )uk ,

and we deduce that v1 + v2 ∈ Span(u1 , . . . , uk ). This proves (ii).

It remains to prove (iii). Fix v ∈ Span(u1 , . . . , uk ) and α ∈ F. Since v ∈
Span(u1 , . . . , uk ), we see that there exist scalars α1 , . . . , αk ∈ F such that v =
α1 u1 + · · · + αk uk . But now


αv = α α1 u1 + · · · + αk uk = (αα1 )u1 + · · · + (ααk )uk ,

and so αv ∈ Span(u1 , . . . , uk ). This proves (iii). We have now proven (b).

Claim. For all subspaces U of V such that u1 , . . . , uk ∈ U , we have that

Span(u1 , . . . , uk ) ⊆ U .

Proof of the Claim. Fix any subspace U of V that contains u1 , . . . , uk ; we must

show that Span(u1 , . . . , uk ) ⊆ U . Fix any v ∈ Span(u1 , . . . , uk ). Then there exist
scalars α1 , . . . , αk ∈ F such that v = α1 u1 + · · · + αk uk . Since U is a subspace of
V , it satisfies (ii) and (iii) from Theorem 3.1.7. Since u1 , . . . , uk ∈ U , (iii) from
Theorem 3.1.7 guarantees that α1 u1 , . . . , αk uk ∈ U ; but then (ii) from Theorem 3.1.7
guarantees that α1 u1 + · · · + αk uk ∈ U , i.e. v ∈ U . So, Span(u1 , . . . , uk ) ⊆ U . ♦
We now prove (c). Fix any subspace U of V such that u1 , . . . , uk ∈ U . By
the Claim, we have that Span(u1 , . . . , uk ) ⊆ U , and by (a), Span(u1 , . . . , uk ) is a

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 221

subspace of V . So, U is a vector space, and Span(u1 , . . . , uk ) is subset of U that is a

vector space in its own right (when equipped with the vector addition and scalar
multiplication operations inherited from U ).17 So, by definition, Span(u1 , . . . , uk ) is
a subspace of U . This proves (c).
It remains to prove (d). By (a) and (b), Span(u1 , . . . , uk ) is itself a subspace of
V that contains u1 , . . . , uk . So, the intersection of all subspaces of V that contain
u1 , . . . , uk is a subset of Span(u1 , . . . , uk ).18 On the other hand, by the Claim,
Span(u1 , . . . , uk ) is a subset of each subspace of V that contains the vectors u1 , . . . , uk ,
and consequently, of the intersection of all such subspaces. This proves (d).

Remark: In some texts, for a vector space V over a field F, and for vectors
u1 , . . . , uk ∈ V , the linear span (or simply span) of {u1 , . . . , uk } is defined to be the
intersection of all subspaces of V that contain u1 , . . . , uk . By Theorem 3.1.11, this
definition is equivalent to the one that we gave at the beginning of this subsection.

Rescaling vectors in spanning sets. The following proposition readily follows

from the relevant definitions, but is may be useful to state explicitly.

Proposition 3.1.12. Let V be a vector space over a field F, let v1 , . . . , vk ∈ V , and

let α1 , . . . , αk ∈ F \ {0}. Then

Span(v1 , . . . , vk ) = Span(α1 v1 , . . . , αk vk ).

Proof. We need to prove two inclusions:

(i) Span(v1 , . . . , vk ) ⊆ Span(α1 v1 , . . . , αk vk );

(ii) Span(α1 v1 , . . . , αk vk ) ⊆ Span(v1 , . . . , vk ).

We prove (i); the proof of (ii) is similar and is left as an exercise. Fix any vector
v ∈ Span(v1 , . . . , vk ). Then, by definition, there exist scalars β1 , . . . , βk ∈ F such
that
v = β 1 v1 + · · · + β k vk .
Since scalars α1 , . . . , αk are all non-zero, they have multiplicative inverses α1−1 , . . . , αk−1 ,
respectively. We now have that

v = β1 v1 + · · · + βk vk = (β1 α1−1 )(α1 v1 ) + · · · + (βk αk−1 )(αk vk ),

and so v ∈ Span(α1 v1 , . . . , αk vk ). This proves (i).

17
Technically, both Span(u1 , . . . , uk ) and U are equipped with the vector addition and scalar
multiplication operations inherited from V . However, since Span(u1 , . . . , uk ) ⊆ U ⊆ V , we in fact
have that Span(u1 , . . . , uk ) is equipped with the vector addition and scalar multiplication operations
inherited from U (where U inherited those two operations from V ).
18
T using the fact that for all non-empty collections I of sets, and for every set A ∈ I,
Here, we are
we have that X∈I X ⊆ A.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 222

Remark: In Proposition 3.1.12, it is important that the scalars α1 , . . . , αk are all

non-zero, and indeed, the proposition becomes false without this hypothesis. For
example, for the standard basis vectors e1 , e2 in R2 , we have that Span(e1 , e2 ) = R2 ,
n x  o
1
but Span(1e1 , 0e2 ) = | x1 ∈ R , which is a proper subspace of R2 .
0

3.1.3 Making new vector (sub)spaces out of old ones

The Cartesian product of two vector spaces. Suppose we are given two vector
spaces, say U and W , over a field F.19 Then the Cartesian product

U × W := (u, w) | u ∈ U, w ∈ W

can be turned into a vector space over F in a natural way. We define vector addition
in U × W by setting

(u1 , w1 ) + (u2 , w2 ) := (u1 + u2 , w1 + w2 ),

for all u1 , u2 ∈ U and w1 , w2 ∈ W , where in the first coordinate (“u1 + u2 ”) we

applied addition from the vector space U , and in the second coordinate (“w1 + w2 ”)
we applied vector addition from the vector space W . Scalar multiplication in U × W
(with scalars from the field F) is defined in an equally natural way, i.e. by setting

α(u, w) := (αu, αw)

for all α ∈ F, u ∈ U , and w ∈ W . The zero vector of U × W is the vector

0U ×W := (0U , 0W ), where 0U is the zero vector of the vector space U , and 0W is
the zero of the vector space W . The additive inverse of a vector (u, w) in U × W is
the vector (−u, −w), where −u (resp. −w) is the additive inverse of u (resp. w) in
the vector space U (resp. W ). It is straightforward to verify that all the axioms of a
vector space hold for U × W (with vector addition and scalar multiplication defined
as above). Indeed, this simply follows from the fact that those axioms hold for U
and W , and the details are left as an exercise.

The intersection and sum of linear subspaces. Suppose that V is a vector

space over a field F, and that U and W are subspaces of V . Using Theorem 3.1.7, it
can easily be verified that U ∩ W is a subspace of V , as is

U + W := u + w | u ∈ U, w ∈ W .

The details are left as an exercise.

19
The field F must be the same for both U and W ! Otherwise, the construction does not work.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 223

3.2 Bases of vector spaces

3.2.1 Linear independence
Given a vector space V over a field F, and given vectors v1 , . . . , vk ∈ V , we say
that {v1 , . . . , vk } is a linearly independent set, or that vectors v1 , . . . , vk are linearly
independent, if for all scalars α1 , . . . , αk ∈ F such that
α1 v1 + · · · + αk vk = 0,
we have that α1 = · · · = αk = 0. In other words, vectors v1 , . . . , vk are linearly
independent if and only if the equation α1 v1 + · · · + αk vk = 0 has only the “trivial
solution,” i.e. the solution α1 = · · · = αk = 0. On the other hand, if vectors
v1 , . . . , vk are not linearly independent, then we say that they are linearly dependent,
or that the {v1 , . . . , vk } is a linearly dependent set. So, vectors v1 , . . . , vk are linearly
dependent if and only if there exist scalars α1 , . . . , αk ∈ F, not all zero, such that
α1 v1 + · · · + αk vk = 0. We note that ∅ is a linearly independent set in any vector
space.
As Proposition 3.2.1 (below) shows, for a field F, it is easy to check if a finite set
of vectors in Fn is linearly independent. Note, however, that linear independence is
defined for general vector spaces, and not just for Fn .
Proposition
 3.2.1. Let F be a field, and let a1 , . . . , am (m ≥ 1) be vectors in Fn .
Set A := a1 . . . am . Then the following are equivalent:
(a) vectors a1 , . . . , am are linearly independent;
(b) the homogeneous matrix-vector equation Ax = 0 has only the trivial solution (i.e.
the solution x = 0);
(c) rank(A) = m (i.e. A has full column rank).
Proof. By Corollary 1.6.5, (b) and (c) are equivalent. Let us show that (a) and (b)
are equivalent. We have the following sequence of equivalent statements:
vectors a1 , . . . , am are linearly independent

the equation x1 a1 + · · · + xm am = 0 has only the

⇐⇒
trivial solutuon (i.e. the solution x1 = · · · = xm = 0)
 
x1
the equation a1 . . . am  ...  = 0 has only the
  
⇐⇒ | {z }
=A xm
trivial solutuon (i.e. the solution x1 = · · · = xm = 0)

the homogeneous matrix-vector equation Ax = 0 has

⇐⇒
only the trivial solution (i.e. the solution x = 0).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 224

Thus, (a) and (b) are equivalent. This completes the argument.

For linear independence, we have the following analog of Proposition 3.1.12.

Proposition 3.2.2. Let V be a vector space over a field F, let v1 , . . . , vk ∈ V , and

let α1 , . . . , αk ∈ F \ {0}. Then the set {v1 , . . . , vk } is linearly independent if and
only if the set {α1 v1 , . . . , αk vk } is linearly independent.

Proof. This readily follows from the definition of linear independence and is left as
an exercise.

3.2.2 Bases of vector spaces: definition and basic properties

A finite basis (or simply basis) of a vector space V over a field F is a set {v1 , . . . , vk }
of vectors in V that satisfies the following two conditions:

1. {v1 , . . . , vk } is linearly independent in V ;

2. {v1 , . . . , vk } is a spanning set of V , i.e. Span(v1 , . . . , vk ) = V .

A vector space is finite-dimensional if it has a finite basis. A vector space that

does not have a finite basis is infinite-dimensional.
Not all vector spaces have a finite basis. For example, for any field F, the vector
space PF (over F) of all polynomials with coefficients in F is infinite-dimensional (see
Proposition 3.2.5). It is, indeed, possible to define a basis more generally, so that it
may possibly be an infinite set. This is briefly discussed in subsection 3.2.7. However,
with the exception of subsection 3.2.7, these lecture notes deal only with finite bases.
Notation: Suppose that V is a vector-space over a field F. If V is finite-dimensional
(i.e. has a finite basis), then we write dim(V ) < ∞. On the other hand, if V is
infinite-dimensional (i.e. does not have a finite basis), then we write dim(V ) = ∞.
Remarks: Suppose that V is a vector space over a field F.

ˆ Obviously, any subset of a linearly independent set of vectors in V is linearly

independent. Similarly, any superset of a spanning set of V is a spanning set
of V .20

ˆ {0} is not a linearly independent set in V (because 1 · 0 = 0 and 1 ̸= 0); so,

by the previous bullet point, no linearly independent set of vectors in V , and
in particular, no basis of V , contains the zero vector.

ˆ ∅ is a basis of the trivial vector space {0} (over any field F), and in particular,
{0} is finite dimensional. In fact, ∅ is the unique basis of {0} (because, by the
previous bullet point, no linearly independent set contains 0).
20
A set A is a superset of a set B provided that B ⊆ A.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 225

ˆ Suppose we are given vectors v1 , . . . , vk ∈ V , and we are trying to check

if {v1 , . . . , vk } is a spanning set of V , i.e. whether V = Span(v1 , . . . , vk )
(this is one of the two conditions from the definition of a basis). Obvi-
ously, Span(v1 , . . . , vk ) ⊆ V , and so the only question is whether V ⊆
Span(v1 , . . . , vk ). But “V ⊆ Span(v1 , . . . , vk )” simply means “every vector
in V is a linear combination of vectors v1 , . . . , vk .” So, the second condition
from the definition of a basis holds if and only if every vector in V is a linear
combination of vectors v1 , . . . , vk .

ˆ In the literature, there is a bit of ambiguity about whether (finite) bases are
sets or ordered sets. An “ordered set” is a set in which order and repetitions
matter. For instance, {1, 2, 3}, {1, 2, 2, 3}, and {3, 1, 2} are the same as sets,
but they are pairwise distinct as ordered sets. In what follows, we will implicitly
treat finite sets (when discussed in the context of linearly independent sets,
spanning sets, and bases) as ordered, and in particular, we will care about
repetitions. It is important to note that no linearly independent set (and in
particular, no basis), may contain more than one copy of the same vector.
Indeed, if v1 , . . . , vk is a list of vectors that contains more than one copy of
some vector (say, vi = vj for some i ̸= j), then we can set αi = 1, αj = −1,
and αk = 0 for all k ∈ {1, . . . , n} \ {i, j}, and we get α1 v1 + · · · + αn vn = 0;
so v1 , . . . , vn are not linearly independent.

– In what follows, if A and B are ordered sets (possibly with repeating

elements), then A ⊆ B means that every element of A appears at least as
many times in B as in A. Moreover, for x ∈ A, A \ {x} is the set obtained
from A by deleting one copy of x.

Example 3.2.3. Let F be a field. Then the standard basis En = {en1 , . . . , enn } of Fn
(defined in subsection 1.4.4) is indeed a basis of Fn .21

Example 3.2.4. Let F be a field. Then

           
n 1 0 0 1 0 0 0 0 0 0 0 0 o
 0 0 , 0 0 , 1 0 , 0 1 , 0 0 , 0 0 
0 0 0 0 0 0 0 0 1 0 0 1

is a basis of F3×2 .22

21
Let us check this! We first show that En = {en n
1 , . . . , en } is linearly independent. Fix scalars
T
α1 , . . . , αn ∈ F such that α1 e1 + · · · + αn en = 0. Clearly, α1 en
n n n

1 + · · · + αn en = α1 . . . αn .
 T n n
So, α1 . . . αn = 0, and it follows that α1 = · · · = αn = 0. So, En = {e1 , . . . , en } is
linearly independent. Let us now show that Span(en n n
1 , . . . , en ) = F , i.e. that every vector in V
T
is a linear combination of en n

1 , . . . , en . Fix any v ∈ V , and set v = v1 . . . vn . But now
v = v1 e1 + · · · + vn en , i.e. v is a linear combination of e1 , . . . , en . So, En = {e1 , . . . , en
n n n n n
n } is indeed
a basis of Fn .
22
Proof?

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 226

Proposition 3.2.5. Let F be a field. Then PF is inifnite-dimensional. On the other

hand, for all non-negative integers n, {1, x, . . . , xn } is a basis of PnF , and in particular,
PnF is finite-dimensional.

Proof. It is clear that for any non-negative integer n, {1, x, . . . , xn } is a basis of PnF .23
Let us show that PF is infinite-dimensional. We must show that PF does not have
a (finite) basis. Fix any finite set {p1 (x), . . . , pk (x)} (k ≥ 0) of polynomials in PF ;
we must show that this
set is not a basis of PF .24Let d be any non-negative integer
such that deg pi (x) ≤ d for all i ∈ {1, . . . , k}. Then any linear combination of
the polynomials p1 (x), . . . , pk (x) is
a polynomial of degree at most d, and it follows
that xd+1 ∈ / Span p1 (x), . . . , pk (x) . So, {p1 (x), . . . , pk (x)} is not a spanning set of
PF , and consequently, it is not a basis of PF . This proves that PF does not have a
finite basis, and consequently, PF is infinite-dimensional.

As Proposition 3.2.6 (below) shows, for a field F, we can easily check if a finite
set of vectors in Fn is a basis of Fn . Note, however, that Proposition 3.2.6 only works
for Fn , and not for general vector spaces V .

Proposition 3.2.6. Let F be  a field, and let a1 , . . . , am (m ≥ 1) be vectors in

Fn . Set A := a1 . . . am . Then {a1 , . . . , am } is a basis of Fn if and only if
rank(A) = n = m (i.e. A is a square matrix of full rank). In particular, every basis
of Fn contains exactly n vectors.

Proof. By Proposition 3.2.1, vectors a1 , . . . , am are linearly independent if and only

if rank(A) = m, and by Proposition 3.1.10, vectors a1 , . . . , am span Fn if and only if
rank(A) = n. So, {a1 , . . . , am } is a basis of Fn if and only if rank(A) = m = n.

Remark: By the Invertible Matrix Theorem (version 1; see subsection 1.11.7), square
matrices of full rank are precisely the invertible matrices. So, Proposition 3.2.6 yields
another characterizations of invertible matrices: a matrix in Fn×n (where F is a field)
is invertible if and only if its columns form a basis of Fn .
Remark: By Proposition 3.2.6, every basis of Fn (where F is a field) contains exactly
n vectors. In fact (see Theorem 3.2.16), if V is any finite-dimensional vector space,
23
Here are the details. Fix a non-negative integer n. To show that {1, x, . . . , xn } is linearly
independent, we fix α0 , α1 , . . . , αn ∈ F such that α0 · 1 + α1 x + · · · + αn xn = 0. The 0 on
the right-hand-side denotes the zero polynomial, i.e. the polynomial whose coefficients are all
zero. So, α0 = α1 = · · · = αn = 0, and we deduce that {1, x, . . . , xn } is linearly independent.
On the other hand, by definition, for every p(x) ∈ Pn F , there exist a0 , a1 , . . . , an ∈ F such that
p(x) = a0 + a1 x + · · · + an xn . But then p(x) = a0 · 1 + a1 x + · · · + an xn , and so p(x) is a
linear combination of 1, x, . . . , xn . Thus, {1, x, . . . , xn } is a spanning set of Pn F . This proves that
{1, x, . . . , xn } is indeed a basis of Pn
F .
24



For example, we can take d := max{0, deg p1 (x) , . . . , deg pk (x) }. (Here, we needed to put 0
into the set so that d would be defined even in the case when k = 0: the empty set does not have a
minimum!)

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 227

then all bases of V are of the same size (i.e. contain exactly the same number of
vectors). However, to prove this, we first need to develop some more theory.

Theorem 3.2.7. Let V be a vector space over a field F, and let v1 , . . . , vn ∈ V .

Then the following are equivalent:
(i) {v1 , . . . , vn } is a basis of V ;
(ii) for all vectors v ∈ V , there exist unique scalars α1 , . . . , αn ∈ F such that
v = α 1 v1 + · · · + α n v n .
Proof. Suppose first that (i) holds; we must show that (ii) holds. Fix v ∈ V . We must
show that there exist unique scalars α1 , . . . , αn ∈ F such that v = α1 v1 + · · · + αn vn .
Since {v1 , . . . , vn } is a basis of V , and consequently a spanning set of V , we know
that every vector in V is a linear combination of the vectors v1 , . . . , vn . This proves
existence. It remains to prove uniqueness. Fix scalars α1 , . . . , αn , β1 , . . . , βn ∈ F
such that v = α1 v1 + · · · + αn vn and v = β1 v1 + · · · + βn vn . Then
α1 v1 + · · · + αn vn = β1 v1 + · · · + βn vn ,
and consequently,
(α1 − β1 )v1 + · · · + (αn − βn )vn = 0.
Since {v1 , . . . , vn } is linearly independent (because it is a basis of V ), we deduce
that α1 − β1 = · · · = αn − βn = 0. So, α1 = β1 , . . . , αn = βn . This proves uniqueness,
and (ii) follows.
Suppose now that (ii) holds; we must show that (i) holds. By (ii), every vector
in V is a linear combination of the vectors v1 , . . . , vn , and so V = Span(v1 , . . . , vn ).
It remains to show that {v1 , . . . , vn } is linearly independent. Clearly, the equation
α1 v1 + · · · + αn vn = 0 has a solution, namely α1 = · · · = αn = 0; by (ii), this solution
is unique, and we deduce that the set {v1 , . . . , vn } is linearly independent. Thus, (i)
holds.

Remark/Notation: Theorem 3.2.7 is one of the main reasons we care about bases.
Suppose B = {b1 , . . . , bn } (n ≥ 1) is a basis of a vector space V over a field F. Then
by Theorem 3.2.7, to every vector v ∈ V , we can associate a unique vector
 
α1
:=  ... 
 
v
 
B
αn
in Fn such that v = α1 b1 + · · · + αn bn ; the vector v B is called the coordinate
 

vector of v associated with the basis B. So, V is in a sense “equivalent” to Fn . The

technical term here is “isomorphic”: V is “isomorphic” to Fn . We will discuss this
more formally in chapter 4 (see Proposition 4.3.1).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 228

Example 3.2.8. Let F be a field.

(a) Consider the basis En = {e1 , . . . , en } of Fn . Then for all x ∈ Fn , we have that
x E = x.25
 
n

(b) Consider the basis B := {1, x, . . . , xn } of PnF . Then for all polynomials p(x)
 =
n n

an x + · · · + a1 x + a0 in PF (where an , . . . , a1 , a0 ∈ F), we have that p(x) B =
 T
a0 a1 . . . an .

(c) Consider the basis

           
n 1 0 0 1 0 0 0 0 0 0 0 0 o
C :=  0 0 , 0 0 , 1 0 , 0 1 , 0 0 , 0 0 
0 0 0 0 0 0 0 0 1 0 0 1

of F3×2 . Then for all matrices

 
a1,1 a1,2
A =  a2,1 a2,2  ,
a3,1 a3,1
T
in F3×2 , we have that .26
  
A C
= a1,1 a1,2 a2,1 a2,2 a3,1 a3,1

Remark: When working with coordinate vectors, we must always specify which
basis we are working with. This is because the same vector of a given finite-
dimensional vector space may have different coordinate vectors with respect to
different bases.
Remark: Note that if we change the order of basis elements, then coordinate vectors
change. This is, in fact, the main reason for treating bases as ordered sets, rather
than simply sets. For instance, consider the following two bases of R2×2 :
n 1 0   0 1   0 0   0 0 o
ˆ B1 = , , , ;
0 0 0 0 1 0 0 1
n 1 0   0 0   0 1   0 0 o
ˆ B2 = , , , .
0 0 1 0 0 0 0 1
25
 T  
Indeed, for any x = x1 ... xn , we have that x = x1 e1 + · · · + xn en , and so x En
=
 T
x1 . . . xn = x.      
1 0 0 1 0 0
26
Indeed, we have that A = a1,1  0 0  + a1,2  0 0  + a2,1  1 0  +
0 0 0 0 0 0
     
0 0 0 0 0 0  
a2,2  0 1  + a3,1  0 0  + a3,2  0 0 , and it follows that A C
=
0 0 1 0 0 1
 T
a1,1 a1,2 a2,1 a2,2 a3,1 a3,1 .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 229

These two bases are the same except for the order in which matrices appear in them
(the second and third matrix get swapped). But note that for any matrix
 
a b
A =
c d
T T
in R2×2 , we have that A B = a b c d
     
and A B = a c b d .
    1 2
So, A B ̸= A B (unless b = c).
1 2

The following proposition follows immediately from the definition of a coordinate

vector, but it is useful to keep in mind.

Proposition 3.2.9. Let B = {b1 , . . . , bn } (n ≥ 1) be a basis of a vector space V

over a field F. Then for all i ∈ {1, . . . , n}, we have that bi B = eni .

Proof. Fix i ∈ {1, . . . , n}. Then

bi = 0b1 + · · · + 0bi−1 + 1bi + 0bi+1 + · · · + 0bn

and consequently,  
0
 .. 
 . 
 
 0 
   
bi  1  ← i-th entry
=  
B
 0 
 
 .. 
 . 
0
= eni .
 
i.e. bi B

Rescaling basis vectors. The following proposition states that if we rescale the
vectors of a basis using non-zero scalars, then we obtain another basis of the same
vector space.

Proposition 3.2.10. Let V be a vector space over a field F, let v1 , . . . , vn ∈ V ,

and let α1 , . . . , αn ∈ F \ {0}. Then {v1 , . . . , vn } is a basis of V if and only if
{α1 v1 , . . . , αk vn } is a basis of V .

Proof. This follows immediately from the definition of a basis and from Proposi-
tions 3.1.12 and 3.2.2.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 230

3.2.3 “Shrinking” a spanning set to a basis

The main goal of this subsection is to show that every spanning set of a vector space
has a subset that is a basis of that vector space (see Theorem 3.2.14). We begin
with a couple of technical propositions.
Proposition 3.2.11. Let V be a vector space over a field F, and let a1 , . . . , ak ∈ V .
Set A := {a1 , . . . , ak }. Then the following hold:
(a) A is linearly independent if and only if no vector in A is a linear combination
of the other vectors in A;27

(b) if A is a spanning set of V , and some vector ai ∈ A is a linear combination of

the other vectors in A, then A \ {ai } is a spanning set of V .28
Proof. We first prove (a). We prove the following equivalent statement: “A is linearly
dependent if and only if some vector of A is a linear combination of the other vectors
in A.”
Suppose first that A is linearly dependent. Then there exist scalars α1 , . . . , αk ∈ F,
not all zero, such that α1 a1 + · · · + αk ak = 0. Fix an index i ∈ {1, . . . , k} such that
αi ̸= 0. Then αi has a multiplicative inverse αi−1 , and we see that

ai = −αi−1 α1 a1 − · · · − αi−1 αi−1 ai−1 − αi−1 αi+1 ai+1 − · · · − αi−1 α1 ak .

So, ai is a linear combination of the other vectors in A.

Suppose now that some vector in A is a linear combination of the other vectors
in A. Say, ai is a linear combination of the vectors a1 , . . . , ai−1 , ai+1 , . . . , ak . Then
there exist scalars α1 , . . . , αi−1 , αi+1 , . . . , αk ∈ F such that

ai = α1 a1 + · · · + αi−1 ai−1 + αi+1 ai+1 + · · · + αk ak .

We now set αi = −1, and we observe that

α1 a1 + · · · + αi−1 ai−1 + αi ai + αi+1 ai+1 + · · · + αk ak = 0.

Since not all of α1 , . . . , αk are zero (indeed, αi ̸= 0), we see that A = {a1 , . . . , ak } is
linearly dependent. This proves (a).
We now prove (b). Assume that A is a spanning set of V , and that some
ai ∈ A is a linear combination of the other vectors in A. Then there exist scalars
α1 , . . . , αi−1 , αi+1 , . . . , αk ∈ F such that

ai = α1 a1 + · · · + αi−1 ai−1 + αi+1 ai+1 + · · · + αk ak .

27
If A contains more than one copy of the same vector, then we treat each copy as distinct. So,
when expressing a vector v in A as a linear combination of the “other” vectors in A, we are allowed
to use any additional copies of v (if there are any) in that linear combination.
28
If ai appears more than once in A, then A \ {ai } is understood to be the set obtained from A
by removing only one copy of ai .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 231

Now, fix any vector v ∈ V . We must show that v is a linear combination of vectors
in A \ {ai } = {a1 , . . . , ai−1 , ai+1 , . . . , ak }. Since A = {a1 , . . . , ak } is a spanning set
of V , we know that there exist scalars β1 , . . . , βk ∈ F such that v = β1 a1 + · · · + βk ak .
We now compute:

v = β1 a1 + · · · + βi−1 ai−1 + βi ai + βi+1 ai+1 + · · · + βk ak

= β1 a1 + · · · + βi−1 ai−1 +
+βi (α1 a1 + · · · + αi−1 ai−1 + αi+1 ai+1 + · · · + αk ak )
+βi+1 ai+1 + · · · + βk ak

= (β1 + βi α1 )a1 + · · · + (βi−1 + βi αi−1 )ai−1 +

+(βi+1 + βi αi+1 )ai+1 + · · · + (βk + βi αk )ak .

So, v is a linear combination of vectors a1 , . . . , ai−1 , ai+1 , . . . , ak , and (b) follows.

We note that Proposition 3.2.11(a) can be slightly strengthened as follows.

Proposition 3.2.12. Let V be a vector space over a field F, and let a1 , . . . , ak ∈ V .

Then the set A := {a1 , . . . , ak } is linearly dependent if and only if there exists some
index i ∈ {1, . . . , k} such that ai is a linear combination of a1 , . . . , ai−1 .29

Proof. Exercise.

Proposition 3.2.13. Let V be a vector space over a field F, and let B = {b1 , . . . , bk }
be a spanning set of V . Let B ′ ⊆ B be such that every vector in B \ B ′ is a linear
combination of vectors in B ′ . Then B ′ is a spanning set of V .

Proof. Choose a set B

e such that

ˆ B′ ⊆ B
e ⊆ B,

ˆ B
e is a spanning set of V ;

ˆ subject to the above, B

e is as small as possible.

(The fact that Be exists follows from the fact that B ′ ⊆ B ⊆ B, and B is a spanning
e = B ′ , then we are done. So, assume that B ′ ⫋ B,
set of V .) If B e and fix some
v∈B ′
e \ B . Then v is a linear combination of the other vectors in B
e (because v is a
′ e \ {v} is
linear combination of the vectors in B ), and so by Proposition 3.2.11(b), B
a spanning set of V . But now B \ v contradicts the minimality of B.
e e
29
By definition, 0 is a linear combination of the empty set/list of vectors, and so if a1 = 0, then
a1 is, in fact, a linear combination of the empty list a1 , . . . , ai−1 .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 232

Our next theorem (Theorem 3.2.14) states that, given any spanning set B of a
vector space V over a field F, we can obtain a basis of V by possibly removing some
vectors from B. As we shall see later (see Theorem 3.2.19), any linearly independent
set in a finite-dimensional vector space can be extended to a basis; however, we
cannot prove this yet.

Theorem 3.2.14. Let V be a vector space over a field F, and let B = {b1 , . . . , bk }
be a spanning set of V . Then some subset of B is a basis of V .

Proof. Let B ′ ⊆ B be a spanning set of V that has as few elements as possible.30

We claim that B ′ is a basis of V . It suffices to show that B ′ is linearly independent.
Suppose otherwise. Then Proposition 3.2.11(a) guarantees that some b ∈ B ′ is a
linear combination of the other vectors in B ′ ; but then by Proposition 3.2.11(b),
B ′ \ {b} is a spanning set of V , contrary to the minimality of B ′ .

3.2.4 The Steinitz exchange lemma

In this subsection, we prove a technical result called the Steinitz exchange lemma. It
essentially states that if we are given a linearly independent set A and a spanning set
B of a vector space V , then we can extend A to a spanning set A ∪ B ′ of V of the
same size as B, and moreover, we can choose B ′ so that it is a subset of B. (So, our
spanning set A ∪ B ′ is obtained by adding to A some vectors from B. It is possible
that B ′ = ∅, but the important point is that we do not add to A any vectors that
are not in B.)
We note that the Steinitz exchange lemma has a number of important corollaries,
which we discuss in our next subsection (see subsection 3.2.5). Perhaps the most
important of these corollaries is the fact that any two bases of a finite-dimensional
vector space are of the same cardinality, i.e. contain the same number of vectors (see
Theorem 3.2.16).31 This will allow us to define the “dimension” of a finite-dimensional
vector space as the number of vectors in any basis of that vector space.
The proof of the Steinitz exchange lemma essentially proceeds by induction
on |A \ B| (i.e. the numbers of vectors that belong to A, but not to B), using
Lemma 3.2.15 (a technical lemma stated and proven below).

Lemma 3.2.15. Let V be a vector space over a field F. Let a1 , . . . , ak , b1 , . . . , bℓ ∈ V ,

and assume that a1 , . . . , ak are pairwise distinct and that b1 , . . . , bℓ are pairwise
distinct. Assume furthermore that A := {a1 , . . . , ak } is a linearly independent set in
V , and that B := {b1 , . . . , bℓ } is a spanning set of V . Then for all a ∈ A \ B, there
exists some b ∈ B \ A such that (B \ {b}) ∪ {a} is a spanning set of V .
30
Let us explain why B ′ exists. Clearly, B has a subset (namely itself) that is a spanning set of
V . Of all subsets of B that span V , we choose B ′ to be one of minimum size.
31
We have already proven this for Fn , where F is a field (see Proposition 3.2.6). However,
Theorem 3.2.16 deals with general finite-dimensional vector spaces, and not just Fn .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 233

A
a

b a

B (B \ {b}) ∪ {a}

Proof. We may assume that A ̸⊆ B, for otherwise, the lemma is vacuously true.32
Fix any a ∈ A \ B. Then there exists an index i ∈ {1, . . . , k} such that a = ai . Since
ai ∈ V = Span(B), we know that there exist scalars α1 , . . . , αℓ ∈ F such that

ai = α1 b1 + · · · + αℓ bℓ .

Claim. There exists an index j ∈ {1, . . . , ℓ} such that αj ̸= 0 and

bj ∈ B \ A.

Proof of the Claim. Suppose otherwise. Then for all j ∈ {1, . . . , ℓ} such that αj =
̸ 0,
33
we have that bj ∈ B ∩ A ⊆ A \ {ai }. But now ai is a linear combination of the
other vectors in the linearly independent set A,34 contrary to Proposition 3.2.11(a). ♦
Using the Claim, we fix an index j ∈ {1, . . . , ℓ} such that αj =
̸ 0 and bj ∈ B \ A.
We will show that (B \{bj })∪{ai } is a spanning set of V (this will complete the proof
of the lemma). Since bj ̸= ai ,35 we see that (B \ {bj }) ∪ {ai } = (B ∪ {ai }) \ {bj },
and we need to show that (B ∪ {ai }) \ {bj } is a spanning set of V . Since B is a
spanning set of V , so is B ∪ {ai }. In view of Proposition 3.2.11(b), it now suffices
to show that bj is a linear combination of the other vectors in B ∪ {ai }. Since
ai = α1 b1 + · · · + αℓ bℓ , we see that

αj bj = ai − α1 b1 − · · · − αj−1 bj−1 − αj+1 bj+1 − · · · − αℓ bℓ .

Since αj ̸= 0, we know that αj has a multiplicative inverse αj−1 , and we deduce that

bj = αj−1 ai − αj−1 α1 b1 − · · · − αj−1 αj−1 bj−1 − αj−1 αj+1 bj+1 − · · · − αj−1 αℓ bℓ .

So, bj is indeed a linear combination of the other vectors in B ∪ {ai }, and we are
done.

32
Indeed, if A ⊆ B, then there are no vectors a ∈ A \ B, and so there is nothing to prove.
33
We are using the fact that ai ∈ A \ B, and so B ∩ A ⊆ A \ {ai }.
34
Indeed, since ai = α1 b1 + · · · + αℓ bℓ , we see that ai is a linear combination of those bj ’s for
which αj ̸= 0. But all such bj ’s belong to A \ {ai }.
35
This is because ai ∈ A, whereas bj ∈ B \ A.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 234

The Steinitz exchange lemma. Let V be a vector space over a field F, let
a1 , . . . , ak , b1 , . . . , bℓ ∈ V , and assume that a1 , . . . , ak are pairwise distinct and that
b1 , . . . , bℓ are pairwise distinct. Assume furthermore that A := {a1 , . . . , ak } is a
linearly independent set in V , and assume that B := {b1 , . . . , bℓ } is a spanning set
of V . Then k ≤ ℓ (i.e. |A| ≤ |B|). Moreover, there exists a set B ′ ⊆ B \ A such that
|B ′ | = |B| − |A| = ℓ − k and A ∪ B ′ is a spanning set of V .

B′ A B′

Remark: Since
A∩B ′ = ∅ (because B ′ ⊆ B \A), we have that |A∪B ′ | = |A|+|B ′ | =

|A| + |B| − |A| = |B|. So, the “new” spanning set A ∪ B ′ is of the same size as the
“old” spanning set B.
Proof. We may assume that A ̸⊆ B, for otherwise, the result is immediate.36 Set
p := |A ∩ B|.37 After possibly permuting the elements of A and B, we may assume
that the following hold:
ˆ a1 = a1 , . . . , ap = bp ;
ˆ {ap+1 , . . . , ak } ∩ {bp+1 , . . . , bℓ } =
̸ ∅.
We now prove a technical claim.
Claim. For all t ∈ {0, . . . , k − p}, there exist pairwise distinct indices
i1 , . . . , it ∈ {p + 1, . . . , ℓ} such that
 
{a1 , . . . , ap } ∪ {ap+1 , . . . , ap+t } ∪ {bp+1 , . . . , bℓ } \ {bi1 , . . . , bit }

is a spanning set of V .

A ak
..
ap+t+1
ap+t
..
ap+2
ap+1

ap = bp
.. bi 2 bi 1 bi t
a1 = b1 B

36
Indeed, if A ⊆ B, then |A| ≤ |B|, and we may set B ′ := B \ A.
37
Since A ̸⊆ B, we see that p < k.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 235

Proof of the Claim. We proceed by induction on t, using Lemma 3.2.15.

For t = 0, we need only show that {a1 , . . . , ap } ∪ {bp+1 , . . . , bℓ } is a spanning set
of V . But note that {a1 , . . . , ap } ∪ {bp+1 , . . . , bℓ } = B, and by hypothesis, B is a
spanning set of V .
Now, fix some t ∈ {0, . . . , k − p − 1}, and assume inductively that the statement
is true for t, i.e. that there exist pairwise distinct indices i1 , . . . , it ∈ {p + 1, . . . , ℓ}
such that
 
Bt := {a1 , . . . , ap } ∪ {ap+1 , . . . , ap+t } ∪ {bp+1 , . . . , bℓ } \ {bi1 , . . . , bit }

is a spanning set of V . Now, ap+t+1 ∈ A \ Bt , and so by Lemma 3.2.15, there exists

some b ∈ Bt \A such that (Bt \{b})∪{ap+t+1 } is a spanning set of V . Since Bt \A =
{bp+1 , . . . , bℓ } \ {bi1 , . . . , bit }, there exists some it+1 ∈ {p + 1, . . . , ℓ} \ {i1 , . . . , it }
such that b = bit+1 . Now i1 , . . . , it , it+1 are pairwise distinct indices in {p + 1, . . . , ℓ},
and
(Bt \ {b}) ∪ {ap+t+1 } = {a1 , . . . , ap } ∪ {ap+1 , . . . , ap+t , ap+t+1 }∪
∪ {bp+1 , . . . , bℓ } \ {bi1 , . . . , bit , bit+1 }

is a spanning set of V . This completes the induction. ♦

We now apply the Claim for t = k − p, and we get that there exist pairwise
distinct indices i1 , . . . , ik−p ∈ {p + 1, . . . , ℓ} such that
 
C := {a1 , . . . , ap } ∪ {ap+1 , . . . , ak } ∪ {bp+1 , . . . , bℓ } \ {bi1 , . . . , bik−p }

is a spanning set of V . But note that |C| = ℓ = |B| and A ⊆ C. Thus, |A| ≤ |C| = |B|,
and so k ≤ ℓ. Next, set B ′ := {bp+1 , . . . , bℓ } \ {bi1 , . . . , bik−p }. Then B ′ ⊆ B \ A,
|B ′ | = (ℓ − p) − (k − p) = ℓ − k = |B| − |A|, and C = A ∪ B ′ is a spanning set of V .
This completes the argument.

Remark: For technical reasons (in order to get the set B ′ ), the Steinitz exchange
lemma assumes that the sets A and B contain no repetitions.38 However, if we only
care about the “|A| ≤ |B|” part of the Steinitz exchange lemma (which is what we
usually care about), then this assumption is not necessary. Indeed, suppose that V
is a vector space over a field F, and suppose that A is a linearly independent set of
vectors in V and that B is a spanning set of V (with repetitions allowed). Since A
is linearly independent, it contains no repetitions; however, B may possibly contain
repetitions. But then we let Be be the set obtained from B by eliminating repetitions.
Then B is still a spanning set of V , and by the Steinitz exchange lemma, we get that
e
|A| ≤ |B|
e ≤ |B|.
38
Actually, it would be possible to state and prove a version of the Steinitz exchange lemma that
allows repetitions. However, this would be notationally messy.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 236

3.2.5 The dimension of a finite-dimensional vector space

The following theorem is perhaps the most important corollary of the Steinitz
exchange lemma.
Theorem 3.2.16. Let V be a finite-dimensional vector space over a field F. Then
all bases of V are of the same size.
Proof. We apply the Steinitz exchange lemma twice. Fix bases {u1 , . . . , um } and
{v1 , . . . , vn } of V . Since {u1 , . . . , um } is linearly independent and {v1 , . . . , vn } is a
spanning set of V , the Steinitz exchange lemma guarantees that m ≤ n. On the other
hand, since {v1 , . . . , vn } is a linearly independent set and {u1 , . . . , um } is a spanning
set of V , the Steinitz exchange lemma guarantees that n ≤ m. So, m = n.

The dimension of a finite-dimensional vector space V over a field F, denoted by

dim(V ), is the number of elements in any basis of V (by Theorem 3.2.16, this is well
defined).

Remarks:
ˆ Note that dim({0}) = 0 (where {0} is understood to be a vector space over an
arbitrary field F), because ∅ is a basis of {0}.
ˆ For any field F, we have that dim(Fn ) = n, because the standard basis of Fn
has n elements. We note, however, that the standard basis is not the only basis
of Fn (except in some very special cases; see Proposition 3.2.6).

Theorem 3.2.17 (below) is another important corollary of the Steinitz exchange

lemma. It essentially states that if V is a finite-dimensional vector space, then any
linearly independent set in V is of size at most dim(V ), and any spanning set of V is
of size at most V . Schematically (and informally), we can summarize this is follows:

|linearly independent set of V | ≤ dim(V ) ≤ |spanning set of V |.

Theorem 3.2.17. Let V be a finite-dimensional vector space over a field F, and set
n := dim(V ). Then both the following hold:
(a) every linearly independent set of vectors in V has at most n vectors;
(b) every spanning set of V has at least n vectors.
Proof. Fix a basis B = {b1 , . . . , bn } of V . Then B is both a linearly independent
set and a spanning set of V . Now, by the Steinitz exchange lemma, the number of
vectors in any linearly independent set of V is at most the number of vectors in the
spanning set B of V , which is n; so, (a) holds. On the other hand, by the Steinitz
exchange lemma, any spanning set of V has at least as many vectors as the linearly
independent set B; so, (b) holds.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 237

By Theorem 3.2.17(a), linearly independent sets in any finite-dimensional vector

space have bounded size (bounded above by the dimension of the vector space in
question). On the other hand, by Proposition 3.2.18 (below), infinite-dimensional
vector spaces have linearly independent sets of arbitrarily large (finite) size. For
instance, if F is a field, then for any positive integer n, {1, x, x2 , . . . , xn } is a linearly
independent set in PF (the vector space of all polynomials with coefficients in F).

Proposition 3.2.18. Let V be an infinite-dimensional vector space over a field F.

Then for every non-negative integer n, V has a linearly independent set of size n.

Proof. We proceed by induction on n. For n = 0, we observe that ∅ is a linearly

independent set of size 0 in V . Next, fix a non-negative integer n, and assume
that V has a linearly independent set of size n, say {a1 , . . . , an }. Then {a1 , . . . , an }
is not a spanning set of V , for otherwise, it would be a basis of V , contrary to
the fact that V is infinite-dimensional. Thus, Span(a1 , . . . , an ) ⫋ V ; fix some
an+1 ∈ V \ Span(a1 , . . . , an ). We now claim that {a1 , . . . , an , an+1 } is a linearly
independent set in V . Fix α1 , . . . , αn , αn+1 ∈ F such that

α1 a1 + · · · + αn an + αn+1 an+1 = 0.
−1 −1
If αn+1 ̸= 0, then an+1 = −αn+1 α1 a1 − · · · − αn+1 αn an , contrary to the fact that
an+1 ∈/ Span(a1 , . . . , an ). So, αn+1 = 0, and it follows that

α1 a1 + · · · + αn an = 0.

But since {a1 , . . . , an } is a linearly independent set, it follows that α1 = · · · = αn = 0.

We have now shown that α1 = · · · = αn = αn+1 = 0, and we deduce that the set
{a1 , . . . , an , an+1 } is linearly independent. This completes the induction.

Our next theorem states that any linearly independent set of vectors in a finite-
dimensional vector space can be extended to a basis of that vector space.

Theorem 3.2.19. Let V be a finite-dimensional vector space over a field F, and

let {a1 , . . . , ak } be a linearly independent set of vectors in V . Then there exists some
basis of V that contains all of a1 , . . . , ak .

Proof. Set n := dim(V ). By Theorem 3.2.17, any linearly independent set of

vectors in V has at most n vectors; in particular, k ≤ n (because {a1 , . . . , ak }
is linearly independent). Now, let A be a linearly independent set that contains
vectors a1 , . . . , ak , and subject to that, is of maximum possible size.39 Set A =
39
Let us explain why A exists. There exists at least one linearly independent set that contains
vectors a1 , . . . , ak , namely, the set {a1 , . . . , ak }. On the other hand, all linearly independent sets
are of size at most n, and in particular, there is an upper bound on the size of linearly independent
sets containing a1 , . . . , ak . So, A exists.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 238

{a1 , . . . , ak , ak+1 , . . . , ak+ℓ }. We claim that A is a basis of V . Since A is linearly

independent, it suffices to show that A is a spanning set of V . Fix v ∈ V ; we
must show that v is a linear combination of vectors in A. If v ∈ A, then this is
immediate.40 So, assume that v ∈ / A. Then by the maximality of A, the set {v} ∪ A
is not linearly independent. So, there exist scalars α0 , α1 , . . . , αk+ℓ ∈ F, not all zero,
such that
α0 v + α1 a1 + · · · + αk+ℓ ak+ℓ = 0.
If α0 = 0, then at least one of α1 , . . . , αk+ℓ is non-zero and α1 a1 + · · · + αk+ℓ ak+ℓ = 0,
contrary to the fact that A is linearly independent. So, α0 ̸= 0, it follows that
v = (−α0−1 α1 )a1 + · · · + (−α0−1 αk+ℓ )ak+ℓ ,
and we see that v is a linear combination of vectors in A. This proves that A is a
basis of V , and we are done.

Remark: Suppose that V is a vector space over a field F. By Theorem 3.2.14, any
(finite) spanning set of V contains a subset that is a basis of V ; in particular, if a
vector space has a (finite) spanning set, then it is finite-dimensional.41 On the other
hand, by Theorem 3.2.19, if V is finite-dimensional, then any linearly independent
set in V can be extended to a basis of V .42

Theorems 3.2.14 and 3.2.19 together yield the following corollary.

Corollary 3.2.20. Let V be a finite-dimensional vector space over a field F, and
set n := dim(V ). Then both the following hold:
(a) any linearly independent set of n vectors of V is a basis of V ;
(b) any set of n vectors of V that spans V is a basis of V .
Proof. We first prove (a). Let A be any linearly independent set of vectors in V
such that |A| = n. By Theorem 3.2.19, V has a basis A′ such that A ⊆ A′ . Since
dim(V ) = n, we see that |A′ | = n. Since |A| = n and A ⊆ A′ , it follows that A = A′ .
So, A is a basis of V (because A′ is). This proves (a).
It remains to prove (b). Let B be any set of n vectors of V such that V = Span(B).
Then by Theorem 3.2.14, V has a basis B ′ such that B ′ ⊆ B. Since dim(V ) = n,
we see that |B ′ | = n. Since |B| = n and B ′ ⊆ B, it follows that B ′ = B. So, B is a
basis of V (because B ′ is). This proves (b).
40
Indeed, suppose v ∈ A. Then there exists an index i ∈ {1, . . . , k + ℓ} such that v = ai . Now
set αi = 1, and for all j ∈ {1, . . . , k + ℓ}, set αj = 0. Then v = α1 a1 + · · · + αk+ℓ ak+ℓ , and so v is
linear combination of vectors in A.
41
It is possible to define infinite spanning sets, and we will very briefly discuss this in subsec-
tion 3.2.7. However, except in subsection 3.2.7, we only consider finite spanning sets in these lecture
notes.
42
Actually, Theorem 3.2.19 can be generalized to all vector spaces (including infinite-dimensional
ones), as long as we allow infinite bases. However, the proof would be significantly more complicated,
and it would involve some relatively advanced set theory (in particular, “Zorn’s lemma”).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 239

Theorem 3.2.21. Let V be a finite-dimensional vector space over a field F, and let
U be a subspace of V . Then all the following hold:

(a) U is finite-dimensional;

(b) dim(U ) ≤ dim(V );

(c) if dim(U ) = dim(V ), then U = V .

Proof. Set n := dim(V ). Since U is a subspace in V , any linearly independent set

of vectors in U is also linearly independent in V , and by Theorem 3.2.17(a), any
such set contains at most n vectors. Now, let {u1 , . . . , uk } be a linearly independent
set of vectors in U of maximum possible size.43 (Then k ≤ n.) Let us show that
{u1 , . . . , uk } spans U . Fix u ∈ U ; we must show that u is a linear combination of
the vectors u1 , . . . , uk . If u ∈ {u1 , . . . , uk }, then this is immediate. So, assume that
u∈ / {u1 , . . . , uk }. By the maximality of {u1 , . . . , uk }, we see that {u, u1 , . . . , uk } is
linearly dependent. So, there exist scalars α0 , α1 , . . . , αk ∈ F, not all zero, such that
α0 u + α1 u1 + · · · + αk uk = 0. If α0 = 0, then α1 u1 + · · · + αk uk = 0 and at least one
of the scalars α1 , . . . , αk is non-zero, contrary to the fact that {u1 , . . . , uk } is linearly
independent. So, α0 ̸= 0, and we deduce that u = (−α0−1 α1 )u1 + · · · + (−α0−1 αk )uk .
So, u ∈ Span(u1 , . . . , uk ), and we deduce that {u1 , . . . , uk } is a spanning set of U .
So, {u1 , . . . , uk } is a basis of U , and it follows that U is finite-dimensional, with
dim(U ) = k. So, dim(U ) = k ≤ n = dim(V ). This proves (a) and (b). It remains to
prove (c). Suppose that dim(U ) = dim(V ), i.e. k = n. But now {u1 , . . . , uk } is a
linearly independent set of n vectors in V , and so Corollary 3.2.20 guarantees that
{u1 , . . . , uk } is a basis of V . So, U = Span(u1 , . . . , uk ) = V , and we are done.

Warning: Theorem 3.2.21(c) fails if V is infinite-dimensional! Infinite-dimensional

vector spaces can have proper subspaces that are infinite-dimensional. For example,
p(x) ∈ PR | p(0) = 0 is an infinite-dimensional proper subspace of PR .44


Subspaces of Rn . Let us consider a geometric interpretation of subspaces in Rn .

ˆ The only 0-dimensional subspace of Rn is {0}.45

ˆ 1-dimensional subspaces of Rn are lines though the origin. Indeed, suppose

that {a} is a basis of a subspace U of Rn . Then a ̸= 0 (by linear independence),
and we see that U = Span(a) is the line through the origin and a. (This
is illustrated below for the case of R2 .) So, 1-dimensional subspaces of Rn
essentially look like copies of R1 inside of Rn .
43
It is possible that k = 0. In this case, our linearly independent set is empty.
44
Details?
45
This holds for any vector space V (not just Rn ), as long as the zero vector is from the vector
space V in question. Recall that we defined Span(∅) = {0}, and obviously, ∅ is linearly independent.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 240

x2

U = Span(a)

0
x1

ˆ 2-dimensional subspaces of Rn are planes through the origin. Indeed, suppose

that {a1 , a2 } is a basis of a subspace U of Rn . By linear independence, a1 , a2 are
both non-zero and are not scalar multiples of each other. So, U = Span(a1 , a2 ) is
the plane through the origin and through a1 and a2 . For example, the subspace
of R3 whose basis is {e1 , e2 } is simply the x1 x2 -plane in R3 (illustrated below).
In general, 2-dimensional subspaces of Rn look like copies of R2 inside of Rn
(of course, those copies of R2 , i.e. planes, may possibly be “tilted,” i.e. not
formed by any two of the coordinate axes of Rn ); however, they must all pass
through the origin.
x3

e2
0
x2
e1

U = Span(e1, e2)

x1

ˆ In general, for a positive integer m ≤ n, an m-dimensional subspace of Rn

looks like a copy of Rm inside of Rn . Again, our copy of Rm may possibly be
“tilted,” i.e. not be formed by any m of the n axes of Rn . However, it must pass
through the origin.

3.2.6 On the dimension of some vector spaces obtained from old

ones
Recall from subsection 3.1.3 that if U and W are vector spaces over a field F, then
U × W is also a vector space over F (with vector addition and scalar multiplication
defined in a natural way, as explained in subsection 3.1.3).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 241

Proposition 3.2.22. Let U and W be finite-dimensional vector spaces over a field F.

Then the vector space U × W is finite-dimensional, and moreover,

dim(U × W ) = dim(U ) + dim(W ).

Proof (outline). Let 0U be the zero vector of the vector space U , and let 0W be the
zero of the vector space W . Set m := dim(U ) and n := dim(W ), and fix a basis
{u1 , . . . , um } of U and a basis {w1 , . . . , wn } of W . It is then straightforward to
check that n o
(u1 , 0W ), . . . , (um , 0W ), (0U , w1 ), . . . , (0U , wn )

is a basis of U ×W (the details are left as an exercise), and consequently, dim(U ×W ) =

m + n = dim(U ) + dim(W ).

Recall from subsection 3.1.3 that if V is a vector space over a field F, and
U and W are subspaces of V , then U ∩ W and U + W are also subspaces of V .
Theorem 3.2.23 (below) specifies the relationship between the dimensions of these
four subspaces of V .

Theorem 3.2.23. Let V be a finite-dimensional vector space over a field F, and let
U and W be subspaces of V . Then U ∩ W and U + W are subspaces of V . Moreover,
U , W , U ∩ W , and U + W are all finite-dimensional and satisfy

dim(U + W ) + dim(U ∩ W ) = dim(U ) + dim(W ).

Proof (outline). The fact that U ∩ W and U + W are subspaces of V follows from
the discussion in subsection 3.1.3. Since V is finite-dimensional, Theorem 3.2.21
guarantees that all its subspaces are finite dimensional; in particular, U , W , U ∩ W ,
and U + W are all finite-dimensional. Set m := dim(U ), n := dim(W ), and
p := dim(U ∩ W ). Fix a basis {v1 , . . . , vp } of U ∩ W . Then {v1 , . . . , vp } is a
linearly independent set in the finite-dimensional vector space U , and so by Theo-
rem 3.2.19, it can be extended to a basis {v1 , . . . , vp , u1 , . . . , um−p } of U .46 Similarly,
{v1 , . . . , vp } can be extended to a basis {v1 , . . . , vp , w1 , . . . , wn−p } of W .47 It is now
straightforward to check that
n o
v1 , . . . , vp , u1 , . . . , um−p , w1 , . . . , wn−p

is a basis of U + W (the details are left as an exercise). So,

dim(U + W ) = p + (m − p) + (n − p) = m + n − p.
46
We are using the fact that dim(U ) = m, and so to extend the linearly independent set
{v1 , . . . , vp } to a basis of U , we must add precisely m − p suitably chosen vectors to this set.
47
We are using the fact that dim(W ) = n, and so to extend the linearly independent set
{v1 , . . . , vp } to a basis of W , we must add precisely n − p suitably chosen vectors to this set.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 242

It now follows that

dim(U + W ) + dim(U ∩ W ) = (m + n − p) + p

= m+n

= dim(U ) + dim(W ),

which is what we needed to show.

If V is a vector space over a field F and U and W are its subspaces such that
U ∩ W = {0} and V = U + W , then we say that V is the direct sum of U and W , and
we write V = U ⊕ W . If V = U ⊕ W is also finite-dimensional, then Theorem 3.2.23
immediately implies that dim(V ) = dim(U ) + dim(W ).48 Moreover, we have the
following theorem.

Theorem 3.2.24. Let V be a vector space over a field F, and let U and W be
subspaces of V such that V = U ⊕ W . Then for all v ∈ V , there exist unique u ∈ U
and w ∈ W such that v = u + w.

Proof. Exercise.

3.2.7 A very brief introduction to infinite bases

We can define a basis of a vector space in more generality, as follows. Let V be a
vector space over a field F, and let B ⊆ V . (B may possibly be infinite. However,
we do not allow repetitions in B.)

ˆ B is said to be linearly independent provided that for all pairwise distinct

vectors v1 , . . . , vk ∈ V , and all scalars α1 , . . . , αk ∈ F, if α1 v1 + · · · + αk vk = 0,
then α1 = · · · = αk = 0.49

ˆ Span(B) = α1 v1 + · · · + αk vk | v1 , . . . , vk ∈ B, α1 , . . . , αk ∈ F .50


ˆ B is a basis of V if it satisfies the following two conditions:

1. B is linearly independent;
2. V = Span(B).

With a basis defined in this way, it is possible to show that every vector space has
a (possibly infinite) basis. However, the proof uses “Zorn’s lemma” (an equivalent of
the “Axiom of Choice,” which is studied in set theory) and is non-constructive. So,
48
This is because dim(U ∩ W ) = 0.
49
So, B is linearly independent if and only if all finite subsets of B are linearly independent.
50
So, Span(B) is the set of vectors that can be expressed as a linear combination of finitely many
vectors in B.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 243

it is possible to show that every vector space has a basis, but for some vector spaces,
we have no idea what a basis might look like. For instance, consider the set of all
functions from R to R; this set is a vector space (over R) and therefore has a basis,
but it is not known what a basis of this vector space might look like.
In some cases, though, we can get a “nice” infinite basis. For instance, PR has a
basis {1, x, x2 , x3 , x4 , . . . }.

3.3 The column space, row space, and null space of a

matrix. The rank of a matrix revisited
3.3.1 The column space and row space of a matrix (and their
relationship with rank)
For a field F and a matrix A ∈ Fn×m , we define the following:
ˆ the column space of A, denoted by Col(A), is the subspace of Fn spanned by
the columns of A;51
ˆ the row space of A, denoted by Row(A), is the subspace of F1×m spanned by
the rows of A.52
We can easily relate the column space to the row space using transposes, as the
following proposition shows.
Proposition 3.3.1. Let F be a field, and let A ∈ Fn×m be a matrix. Then both the
following hold:
(a) Col(A) = uT | u ∈ Row(AT ) ;


(b) Row(A) = uT | u ∈ Col(AT ) .



Proof. This is essentially “obvious,” since transposes turn rows into columns and
vice versa. However, let us give a formal proof of (a). The proof of (b) is similar and
is left as an exercise.  T 
a1
  T  .. 
Set A = a1 . . . am , so that A =  . . By definition, we have that
aTm
Col(A) = Span(a1 , . . . , am ) and Row(AT ) = Span(aT1 , . . . , aTm ). Now, to prove (a),
we must prove the following two inclusions:
51
 
More precisely, if A = a1 . . . am , then Col(A) := Span(a1 , . . . , am ). The fact that
Col(A) really is a subspace of Fn follows from Theorem 3.1.11.
 
r1
52
More precisely, if A =  ...  (i.e. r1 , . . . , rn are the rows of A, appearing in A in that order,
 

rn
from top to bottom), then Row(A) := Span(r1 , . . . , rn ). The fact that Row(A) really is a subspace
of F1×m follows from Theorem 3.1.11.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 244

(1) Col(A) ⊆ uT | u ∈ Row(AT ) ;



(2) uT | u ∈ Row(AT ) ⊆ Col(A).



We first prove (1). Fix any u ∈ Col(A). Then u ∈ Span(a1 , . . . , am ), and so

there exist scalars α1 , . . . , αm ∈ F such that u = α1 a1 + · · · + αm am . By taking the
transpose of both sides, we get

uT = (α1 a1 + · · · + αm am )T

= α1 aT1 + · · · + αm aTm ,

and it follows that uT ∈ Span(aT1 , . . . , aTm ) = Row(AT ). Since u = (uT )T , this

proves (1).53
Let us now prove (2). Fix u ∈ Row(AT ). Then u ∈ Span(aT1 , . . . , aTm ), and so
there exist scalars α1 , . . . , αm such that u = α1 aT1 + · · · + αm aTm . By taking the
transpose of both sides, we get that

uT = (α1 aT1 + · · · + αm aTm )T

= α1 (aT1 )T + · · · + αm (aTm )T

= α1 a1 + · · · + αm am ,

and it follows that uT ∈ Span(a1 , . . . , am ) = Col(A). This proves (2).

Proposition 3.3.2. Let F be a field, and let A ∈ Fn×m be a matrix. Then both the
following hold:

(a) Col(A) = {Ax | x ∈ Fm };

(b) Row(A) = {xA | x ∈ F1×n }.54

 
Proof. For (a), we set A = a1 . . . am , and we observe that

Col(A) = Span(a1 , . . . , am ) by the definition of Col(A)

Ax | x ∈ Fm

= by Proposition 1.4.4.

For (b), we use part (a) and Proposition 3.3.1(b), as follows:

 T
Row(A) = u | u ∈ Col(AT ) by Proposition 3.3.1(b)

53
Indeed, we have shown that any element u of Col(A) is the transpose of some element (namely, uT )
of Row(AT ). So, we have proven (1).
54
Note that in the expression xA, we have that x is a row vector with n entries.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 245

 T T
= (A x) | x ∈ Fn by (a)
 T
= x A | x ∈ Fn

xA | x ∈ F1×n .

=

This completes the argument.

Our main goal in this section is to give a recipe for finding a basis of the column
space and the row space of a matrix (see Theorems 3.3.4 and 3.3.9). As we shall see,
both of those spaces have dimension precisely rank(A). We begin with a technical
proposition.
Proposition 3.3.3. Let F be a field, let a1 , . . . , ak ∈ Fn , and let B ∈ Fn×n be an
invertible matrix. Then both the following hold:
(a) {a1 , . . . , ak } is linearly independent if and only if {Ba1 , . . . , Bak } is linearly
independent;

(b) for all v ∈ Fn , v ∈ Span(a1 , . . . , ak ) if and only if Bv ∈ Span(Ba1 , . . . , Bak );

Proof. We first prove (a). Suppose first that {a1 , . . . , ak } is linearly independent. We
must show that {Ba1 , . . . , Bak } is linearly independent. Fix scalars α1 , . . . , αk ∈ F
such that
α1 Ba1 + · · · + αk Bak = 0.
Since B is invertible, it has an inverse B −1 . By multiplying both sides of the equation
above by B −1 (on the left), we obtain

α1 a1 + · · · + αk ak = 0.

Since {a1 , . . . , ak } is linearly independent, we have that α1 = · · · = αk = 0. So,

{Ba1 , . . . , Bak } is linearly independent.
Suppose, conversely, that {Ba1 , . . . , Bak } is linearly independent. We must show
that {a1 , . . . , ak } is linearly independent. Fix scalars α1 , . . . , αk ∈ F such that

α1 a1 + · · · + αk ak = 0.

We now multiply both sides by B (on the left), and we obtain

α1 (Ba1 ) + · · · + αk (Bak ) = 0.

Since {Ba1 , . . . , Bak } is linearly independent, it follows that α1 = · · · = αk = 0. So,

{a1 , . . . , ak } is linearly independent. This completes the proof of (a).
We now prove (b). Fix v ∈ Fn . Suppose first that v ∈ Span(a1 , . . . , ak ). Then
there exist scalars α1 , . . . , αk such that v = α1 a1 + · · · + αk ak . By multiplying

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 246

both sides by B (on the left), we get Bv = α1 (Ba1 ) + · · · + αk (Bak ), and so

Bv ∈ Span(Ba1 , . . . , Bak ).
Suppose, conversely, that Bv ∈ Span(Ba1 , . . . , Bak ). Then there exist scalars
α1 , . . . , αk ∈ F such that

Bv = α1 (Ba1 ) + · · · + αk (Bak ).

Since B is invertible, it has an inverse B −1 . We now multiply both sides of the

equation by B −1 (on the left), and we obtain v = α1 a1 + · · · + αk ak . So, v ∈
Span(a1 , . . . , ak ). This proves (b).

Theorem 3.3.4. Let F be a field, and let A ∈
Fn×m . Then the pivot columns of A
form a basis of Col(A). Moreover, dim Col(A) = rank(A).

Remark/Warning: To get a basis of Col(A), we need to take the pivot columns of

the original matrix A, not of RREF(A).
Remark: If A has no pivot columns (which can only happen if A is the zero matrix,
in which case Col(A) = {0}), then ∅ is the (unique) basis of Col(A).

Proof. Since r := rank(A) is equal to the number of pivot columns of A the first
statement implies the second.  
It remains to prove the first statement. Set A = a1 . . . am . Let ai1 , . . . , air
(with 1 ≤ i1 < · · · < ir ≤ m) be the pivot columns of A. We must show that
{ai1 , . . . , air } is a basis of Col(A).
Set U := RREF(A). Then A ∼ U , and so by Theorem 1.11.13, there
 exists an
invertible matrix B ∈ Fn×n such that U = BA = Ba1 . . . Bam . But now


since U = RREF(A), we see that all the following hold:

(i) Bai1 , . . . , Bair are the pivot columns of U ;

(ii) for all j ∈ {1, . . . , r}, we have that Baij = enj ;55

(iii) in any column of U , only the top r entries may possibly be non-zero (the other
entries are all zero).

Clearly, {en1 , . . . , enr } is a linearly independent set; so, by (ii), {Bai1 , . . . , Bair } is a
linearly independent set. Consequently, by Proposition 3.3.3(a), {ai1 , . . . , air } is a
linearly independent set. Moreover, it is clear that any vector in Fn in which only the
top r entries may possibly be non-zero (and the other entries are all zero), is a linear
combination ofvectors en1 , . . . , enr . So, (i), (ii), and (iii) together imply that every
column of U = Ba1 . . . Bam is a linear combination of vectors Bai1 , . . . , Bair.
But now by Proposition 3.3.3(b), we see that every column of A = a1 . . . am
is a linear combination of vectors ai1 , . . . , air . So, by Propositon 3.2.13, {ai1 , . . . , air }
55
As usual, en n n
1 , . . . , en are the standard basis vectors of F .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 247

is a spanning set of Col(A).56 It now follows that {ai1 , . . . , air } is a basis of Col(A),
and we are done.

Example 3.3.5. Consider the matrix

 
1 2 −1 2 1
A :=  2 4 1 3 1 ,
4 8 −1 7 3
with entries understood to be in R. Find a basis A of Col(A).
Solution. By performing the “forward” phase of the row reduction algorithm, we get
that  
1 2 −1 2 1
A ∼  0 0 3 −1 −1  ,
0 0 0 0 0
and we see that the pivot columns of A are its first and third column, and by
Theorem 3.3.4, those two columns form a basis of Col(A). So,
   
n 1 −1 o
A =  2 , 1 
4 −1
is a basis of Col(A).

Warning: Make sure you use the pivot columns of A itself, and not of one of its
row echelon forms!

Remark: We could also have computed

 
1 2 0 5/3 2/3
RREF(A) =  0 0 1 −1/3 −1/3 
0 0 0 0 0
and identified the pivot columns of A that way. However, if all we need to do is
identify the pivot columns, then this is not necessary: we can identify the pivot
columns from any row echelon form of A, not just from its reduced row echelon
form.

Example 3.3.6. Consider the matrix

 
1 1 2 2 0
 1 2 1 2 1 
A = 
 2

0 0 1 1 
1 0 0 2 1
56
By definition, {a1 , . . . , am } is a spanning set of Col(A). By what we just showed, every vector
in {a1 , . . . , am } is a linear combination of vectors in {ai1 , . . . , air }. So, by Propositon 3.2.13
{ai1 , . . . , air } is a spanning set of Col(A).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 248

and vectors    
1 2
 1   0 
b =  
 1  and  2 ,
c =  

1 1
all with entries in Z3 .
(a) Find a basis A for Col(A).

(b) Find the coordinate vectors of all the columns of A with respect to the basis A.
 
(c) Determine whether b ∈ Col(A), and if so, compute the coordinate vector b A ;
 
(d) Determine whether c ∈ Col(A), and if so, compute the coordinate vector c A .

Solution. Set A = a1 a2 a3 a4 a5 .57 We form the matrix

 

   
A b c = a1 a2 a3 a4 b c
 
1 1 2 2 0 1 2
 1 2 1 2 1 1 0 
= 
 2
.
0 0 1 1 1 2 
1 0 0 2 1 1 1

Because we need to compute some coordinate vectors (and not just

 a basis of Col(A)),
we need to find the reduced row echelon form of A b c , not just any row
echelon form. By row reducing, we get
 
1 0 0 2 0 0 1
   0 1 2 0 0 0 1 
RREF A b c =  0 0 0 0 1 0 0 ,


0 0 0 0 0 1 0

where for emphasis, we colored the pivot columns to the left of the vertical dotted line
red and to the right of the vertical dotted line blue. By focusing on the submatrix
to the left of the vertical dotted line, we see that the pivot columns of A are its first,
second, and fifth column. So, by Theorem 3.3.4,
     
1 1 0
n 1   2   1 o
A := {a1 , a2 , a5 } =  , , 
 2   0   1 
1 0 1
        

1 1 2 2 0
 1   2  1   2
, and a5 =  1 .
57
   
So, a1 = 
 2 , a2 =  0
  , a3 = 
  0 , a4 =  1
 
  1 
1 0 0 2 1

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 249

is a basis of Col(A). We now need to express all the columns of A as a linear

combination of the pivot columns, which allows us to form the coordinate vectors of
the columns of A in terms
 of thebasis

A of Col(A). We simply read this off from
the submatix RREF A b c to the left of the vertical dotted line, as follows:
 
1
ˆ a1 = 1a1 + 0a2 + 0a5 , and so a1 A =  0 ;
 

0
 
0
ˆ a2 = 0a1 + 1a2 + 0a5 , and so a2 A = 1 ;
 

0
 
0
ˆ a3 = 0a1 + 2a2 + 0a5 , and so a3 A =  2 ;
 

0
 
2
ˆ a4 = 2a1 + 0a2 + 0a5 , and so a4 A = 0 ;
 

0
 
0
ˆ a5 = 0a1 + 0a2 + 1a5 , and so a5 A =  0 .
 

Remark: Note that our solution roughly follows the recipe from subsection 1.5.1,
albeit with some modifications. For each pivot column ai , we simply get that
that it is equal to “1 times itself, plus 0 times all the other pivot columns.”
 For


each non-pivot column ai , we focus on the submatrix of RREF A b c
consisting of its leftmost i columns, we ignore any non-pivot columns other than ai
itself, and we express ai as a linear combination of the pivot columns to the left of
it (for the pivot columns to the right of it, we simply get the weight/scalar 0). For
example,  for the non-pivot

column a4 , we first focus on the first four columns of
RREF A b c :
 
1 0 0 2
 0 1 2 0 
 0 0 0 0 ,
 

0 0 0 0
we ignore any non-pivot columns other than the fourth column itself,58 and we
express the fourth column as a linear combination of the pivot columns. This yields
a4 = 2a1 + 0a2 . To express a4 as a linear combination of all the pivot columns
58
In this case, we ignore the third column.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 250

of A, we simply add the remaining pivot columns with weight 0 at the front, i.e.
a4 = 2a1 + 0a2 + 0a5 . Now we can read off the coordinate vector of a4 with respect
   T
to A: a4 A = 2 0 0 .
It remains to deal with vectors b and c. Here again, we essentially follow the
recipe from subsection 1.5.1. Since b is a pivot column of A b c , we see that
b cannot be written as a linear combination of the columns of A, and  it follows

that b ∈ / Col(A). On the other hand, c is not a pivot column of A b c ;
consequently, c can indeed be expressed as a linear combination
 of the
columns of A,
and in particular, c ∈ Col(A). From the matrix RREF A b c , we read off
   T
c = 1a1 + 1a2 + 0a5 , which yields c A = 1 1 0 .

Using Proposition 3.3.1 and Theorem 3.3.4, we can also compute a basis of the
row space of a matrix (see Example 3.3.7 below). However, we will later give another
way of computing a basis of the row space (see Theorem 3.3.9), one that implies that
the dimension of the row space of a matrix is equal to the rank of that matrix.

Example 3.3.7. Consider the matrix

 
1 0 0 2 1
 2 0 0 1 2 
A := 
 1
,
2 1 1 2 
1 1 2 2 1

with entries understood to be in Z3 . Compute a basis of Row(A).

Solution. We first take the transpose of A:

 
1 2 1 1
 0 0 2 1 
 
AT =   0 0 1 2 .

 2 1 1 2 
1 2 2 1

By row reducing, we get

 
1 2 0 0

 0 0 1 0 

RREF(AT ) = 
 0 0 0 1 .

 0 0 0 0 
0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 251

Thus, the pivot columns of AT are its first, third, and fourth column. By Theo-
rem 3.3.4, it follows that
     
1 1 1
n 0   2   1 
    
o
 0 , 1 , 2 
     
 2   1   2 
1 2 1

is a basis of Col(AT ). But by Proposition 3.3.1, we have that Row(A) = {uT | u ∈

Col(AT )}. So, by simply turning columns into rows, we get that
n     o
1 0 0 2 1 , 1 2 1 1 2 , 1 1 2 2 1

is a basis of Row(A).

Proposition 3.3.8. Let F be a field. Then any two row equivalent matrices in Fn×m
have the same row space.
Proof. We begin by showing that applying one elementary row operation to a matrix
does not alter the row space (see the Claim below).

Claim. Let A, B ∈ Fn×m be matrices such that B is obtained from A

by performing one elementary row operation. Then Row(A) = Row(B).
   
a1 b1
Proof of the Claim. Set A =  ...  and B =  ...  (so, a1 , . . . , an are the rows
   

an bn
of A appearing in that order in A, from top to bottom, and similar for B). By
definition, Row(A) = Span(a1 , . . . , an ) and Row(B) = Span(b1 , . . . , bn ).
Since B is obtained from A by performing one elementary row operation R, we
know that A can be obtained from B by performing one elementary row operation
(the one that “undoes” R). So, it is enough to show that Row(A) ⊆ Row(B), for
then an analogous argument will establish that Row(B) ⊆ Row(A), and then the
result will follow.
If B is obtained by swapping two rows of A, then obviously, Row(A) = Row(B).
Next, suppose that B is obtained by multiplying one row of A (say, the i-th row)
by a non-zero scalar α ∈ F. Then bi = αai and bj = aj for all j ∈ {1, . . . , n} \ {i}.
Now, fix v ∈ Row(A); we must show that v ∈ Row(B). Since v ∈ Row(A), there
exist scalars α1 , . . . , αn ∈ F such that v = α1 a1 + · · · + αn an . But now
v = α1 a1 + · · · + αi−1 ai−1 + αi ai + αi+1 ai+1 + · · · + αn an

= α1 a1 + · · · + αi−1 ai−1 + (αi α−1 )(αai ) + αi+1 ai+1 + · · · + αn an

= α1 b1 + · · · + αi−1 bi−1 + (αi α−1 )bi + αi+1 bi+1 + · · · + αn bn ,

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 252

and so v ∈ Row(B). Thus, Row(A) ⊆ Row(B).

Finally, suppose that B is obtained from A by adding a scalar multiple of one
row to another row. Then there exist distinct indices i, j ∈ {1, . . . , n} and a scalar
α ∈ F such that bj = aj + αai , and bk = ak for all k ∈ {1, . . . , n} \ {j}.59 Now, fix
v ∈ Row(A). Then there exist scalars α1 , . . . , αn ∈ F such that v = α1 a1 +· · ·+αn an .
We now set βi := αi − αj α, and we set βk := αk for all k ∈ {1, . . . , n} \ {i}. Then

βi bi + βj bj = (αi − αj α)ai + αj (aj + αai ) = αi ai + αj aj ,

whereas βk bk = αk ak for all k ∈ {1, . . . , n} \ {i, j}. Thus,

β1 b1 + · · · + βn bn = α1 a1 + · · · + αn an = v,

and it follows that v ∈ Row(B). Thus, Row(A) ⊆ Row(B). ♦

Now, fix row equivalent matrices A, B ∈ Fn×m . Then there exists a sequence
R1 , . . . , Rk of elementary row operations such that, by starting with A and then
successively applying R1 , . . . , Rk to it, we obtain B. By the Claim, each time
we apply an elementary row operation, the row space remains unchanged. So,
Row(A) = Row(B).60

Theorem 3.3.9. Let F be a field, let A ∈ Fn×m , and let U be any matrix in row
echelon form that is row equivalent to A. 61 Then the non-zero rows of U form a


basis of Row(A). Moreover, dim Row(A) = rank(A).

Remark: If U has no non-zero rows (which can only happen if A is the zero matrix,
in which case Row(A) = {0}, where 0 is the zero vector in F1×m ), then ∅ is a basis
of Row(A).

Proof. Since r := rank(A) is equal to the number of non-zero rows of U , the first
statement implies the second. Moreover, by Proposition 3.3.8, Row(A) = Row(U ).
So, it suffices to show that the non-zero rows of U form a basis of Row(U ). Let
u1 , . . . , ur be the non-zero rows of U , appearing in that order (from top to bottom)
in U .62 We must show that {u1 , . . . , ur } is a basis of Row(U ). Clearly, Row(U ) =
Span(u1 , . . . , ur ). It remains to show that {u1 , . . . , ur } is a linearly independent set.
Fix scalars α1 , . . . , αr ∈ F such that α1 u1 + · · · + αr ur = 0. We must show that
α1 = · · · = αr = 0. Suppose otherwise, and let i ∈ {1, . . . , r} be the smallest index
such that αi ̸= 0. We may assume that the leading entry (i.e. the leftmost non-zero
59
So, we applied the elementary row operation “Rj → Rj + αRi .”
60
Technically, we are doing an induction on the number of elementary row operations. (Details?)
61
It may be that U = RREF(A), but this assumption is not necessary. U may be any matrix in
row echelon form obtained from A via a sequence of elementary row operations. For instance, U
may be the matrix obtained from A by performing only the “forward” part of the row reduction
algorithm in order to transform A into a matrix in row echelon form.
62
It is possible that r = 0. In that case, we have that U = On×m (and consequently, A = On×m ),
and u1 , . . . , ur is an empty list of vectors.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 253

entry) of the row ui is in position j. But since U is in row echelon form, the leading
entries of ui+1 , . . . , ur are all strictly to the right of the leading entry of ui , and so
their j-th entry is 0. Since α1 = · · · = αi−1 = 0 (by the minimality of i), it follows
that the j-th entry of α1 u1 + · · · + αr ur is non-zero,63 contrary to the fact that
α1 u1 + · · · + αr ur = 0.

Example 3.3.10. Consider the matrix

 
0 3 −6 6 4 −5
 3 −7 8 −5 8 9 
A =   3 −9 12 −9 6 15 


0 1 −2 2 2 1

with entries understood to be in R.

(a) Compute rank(A).

(b) Find a basis of Col(A).

(c) Find a basis of Row(A).

Solution. By performing the “forward” part of the row reduction algorithm, we see
that the following matrix is a row echelon form of A:
 
3 −9 12 −9 6 15
 0 2 −4 4 2 −6 
U =   0
.
0 0 0 1 4 
0 0 0 0 0 0

(a) The matrix U has three pivot columns, and so rank(A) = 3.

(b) The pivot columns of U are its first, second, and fifth column. So, the pivot
columns of A are its first, second, and fifth column, and so those columns of A form
a basis of Col(A). More precisely, the following is a basis of Col(A):
     
0 3 4
n  3   −7   8  o
 ,
 3   −9  ,  6  .
  

0 1 2

(c) The non-zero rows of U form a basis of Row(A). So, the following is a basis
of Row(A):
n     o
3 −9 12 −9 6 15 , 0 2 −4 4 2 −6 , 0 0 0 0 1 4 .
63
Indeed, it is equal to αi ui,j , where ui,j is the i, j-th entry of the matrix U . Since αi and ui,j
are both non-zero, we see that αi ui,j ̸= 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 254

Remark: We could also have computed a basis of Row(A) using the method
described in Example 3.3.7 (i.e. by finding a basis of Col(AT ) and then taking the
transpose of the basis vectors in order to obtain a basis of Row(A)). However, this
would have meant having to row reduce twice. Indeed, we would row reduce A in
order to find a basis of Col(A), and then we would row reduce AT in order to find
a basis of Row(A). On the other hand, the method from Example 3.3.7 has the
advantage that it produces a basis of Row(A) all of whose elements are rows of the
original matrix A itself (which was not the case for the basis of Row(A) that we
obtained above).

Corollary 3.3.11. Let F be a field, and let A ∈ Fn×m . Then both the following hold:



(a) dim Col(A) = dim Row(A) = rank(A);

(b) rank(A) = rank(AT ).

Proof. Part (a) follows immediately from Theorems 3.3.4 and 3.3.9. For (b), we
observe that


rank(A) = dim Col(A) by (a)

= dim Row(AT )


by Proposition 3.3.1

= rank(AT ) by (a),

and we are done.

3.3.2 The rank of a matrix and elementary row and column opera-
tions
We introduced elementary row operations in subsection 1.3.2, and we have used them
in many contexts throughout these lecture notes. Elementary column operations are
defined completely analogously, only for columns instead of rows. Let us be fully
precise. There are three types of elementary column operations on a matrix (with
entries in some field F), listed below.

1. Swap (interchange) two columns.

ˆ We denote the operation of swapping columns i and j (i ̸= j) by “Ci ↔

Cj .”

2. Multiply one column by a non-zero scalar.

ˆ We denote the operation of multiplying column i by a scalar α =

̸ 0 by
“Ci → αCi .”

3. Add a scalar multiple of one column to another column.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 255

ˆ We denote the operation of adding scalar α times column i to column j

(i ̸= j) by “Cj → Cj + αCi .”

Elementary column operations should not be used for solving linear systems. How-
ever, Proposition 1.6.2 and Corollary 3.3.11(b) readily imply that both elementary
row operations and elementary column operations preserve rank. This is stated
formally by the proposition below.

Proposition 3.3.12. Elementary row operations and elementary column operations

both preserve the rank of a matrix. More precisely, if a matrix A′ can be obtained
from a matrix A (with entries in some field F) by performing an elementary row or
column operation on A, then rank(A) = rank(A′ ).

Proof. By Proposition 1.6.2, elementary row operations preserve rank.64 Suppose

now that a matrix A′ can be obtained from a matrix A ∈ Fn×m (where F is some
field) via an elementary column operation. Then (A′ )T can be obtained from AT via
an elementary row operation, and therefore, AT and (A′ )T are row equivalent. We
now deduce that
(∗) (∗∗) (∗)
rank(AT ) rank (A′ )T rank(A′ ),


rank(A) = = =

where both instances of (*) follow from Corollary 3.3.11(b), and (**) follows from
Proposition 1.6.2.

Example 3.3.13. Compute the rank of the matrix

 
−1 2 −3 5
A =  2 −4 6 7 
1 −1 3 7

with entries understood to be in R.

Remark: We could, of course, simply row reduce the matrix A, and count the number
of pivot columns (or the number of non-zero rows) of the matrix RREF(A). However,
to illustrate Proposition 3.3.12, we will instead apply a sequence of elementary row
and column operations to the matrix A.

Solution. We compute:
 
−1
 2 −3 5 
rank(A) = rank  2 −4 6 7 
1 −1 3 7

64
Indeed, if one matrix can be obtained from another via an elementary row operation, then the
two matrices are row equivalent (by definition). So, by Proposition 1.6.2, such two matrices have
the same rank.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 256

C2 →C2 +2C1  
C3 →C3 −3C1 −1 0 0 0 

C4 →C4 +5C1
= rank  2 0 0 17 
1 0 0 12
 
R2 →R2 +2R1 −1 0 0 0 

R3 →R3 +R1
= rank  0 0 0 17 
0 0 0 12
 
12
R3 →R3 − 17 R2
−1 0 0 0 

= rank  0 0 0 17 
0 0 0 0

= 2.

3.3.3 Matrices of full rank

Theorem 3.3.14. Let F be a field, and let A ∈ Fn×m . Then all the following hold:

(a) the columns of A are linearly independent if and only if rank(A) = m (i.e. A has
full column rank);

(b) the columns of A span Fn (i.e. Col(A) = Fn ) if and only if rank(A) = n (i.e. A
has full row rank);

(c) the rows of A are linearly independent if and only if rank(A) = n (i.e. A has full
row rank);

(d) the rows of A span F1×m (i.e. Row(A) = F1×m ) if and only if rank(A) = m (i.e.
A has full column rank).

Proof. Part (a) follows from Proposition 3.2.1, and part (b) follows from Proposi-
tion 3.1.10.
Further, by Corollary 3.3.11(b), we have that rank(AT ) = rank(A). In particular,
A has full column rank if and only if A has full row rank,65 and AT has full row
T

65
Indeed AT is an m × n matrix, and so

AT has full column rank ⇐⇒ rank(AT ) = n

(∗)
⇐⇒ rank(A) = n

⇐⇒ A has full row rank,

where (*) follows from Corollary 3.3.11(b).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 257

rank if and only if A has full column rank.66 So, by applying (a) to AT , we obtain (c),
and by applying (b) to AT , we obtain (d).

Recall that a square matrix has full rank if and only if it is invertible (see
Corollary 1.11.10). For square matrices of full rank, Theorem 3.3.14 yields the
following corollary.

Corollary 3.3.15. Let F be a field, and let A ∈ Fn×n be a square matrix. Then
the following are equivalent:

(a) rank(A) = n;

(b) rank(AT ) = n;

(c) the columns of A are linearly independent;

(d) the columns of A span Fn (i.e. Col(A) = Fn );

(e) the columns of A form a basis of Fn ;

(f ) the rows of A are linearly independent;

(g) the rows of A span F1×n (i.e. Row(A) = F1×n );

(h) the rows of A form a basis of F1×n .

Proof. By Corollary 3.3.11(b), we have that rank(A) = rank(AT ), and so (a) and (b)
are equivalent.67
The fact that (a) and (c) are equivalent follows from Theorem 3.3.14(a),68 the
fact that (a) and (d) are equivalent follows from Theorem 3.3.14(b),69 , and the fact
that that (a) and (e) are equivalent follows from Proposition 3.2.6.
So far, we have shown that (a), (b), (c), (d), and (e) are equivalent.
The equivalence of (a), (c), (d), and (e) applied to AT yields the equivalence
of (b), (f), (g), and (h). This completes the argument.
66
Indeed, AT is an m × n matrix, and so

AT has full row rank ⇐⇒ rank(AT ) = m

(∗)
⇐⇒ rank(A) = m

⇐⇒ A has full column rank,

where (*) follows from Corollary 3.3.11(b).

67
Alternatively, the equivalence of (a) and (b) follows from Corollary 1.11.10.
68
Alternatively, this follows from Proposition 3.2.1.
69
Alternatively, this follows from Proposition 3.1.10

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 258

3.3.4 The rank of matrix products. Left and right inverses of a

matrix
Using the results of subsection 3.3.1 (and also of section 1.11), we can show that
multiplying a matrix by an invertible matrix (on the left or the right) leaves the
rank unchanged. More precisely, we have the following proposition.

Proposition 3.3.16. Let F be a field, and let A ∈ Fn×m . Then all the following
hold:

(a) for all invertible matrices S ∈ Fn×n , we have that rank(SA) = rank(A);

(b) for all invertible matrices S ∈ Fm×m , we have that rank(AS) = rank(A);

(c) for all invertible matrices S1 ∈ Fn×n and S2 ∈ Fm×m , we have that rank(S1 AS2 ) =
rank(A).

Proof. We first prove (a). Fix an invertible matrix S ∈ Fn×n . By Theorem 1.11.13,
A and SA are row equivalent, and so by Proposition 1.6.2, they have the same rank.
This proves (a).
We now prove (b). Fix an invertible matrix S ∈ Fm×m . Then by the Invertible
Matrix Theorem (version 1; see subsection 1.11.7), S T is also invertible. We now
compute:

rank(AS) = rank (AS)T

by Corollary 3.3.11(b)

= rank S T AT


by Proposition 1.8.1(d)

= rank AT by (a), since S T is invertible

= rank(A) by Corollary 3.3.11(b).

This proves (b).

Finally, for (c), we fix invertible matrices S1 ∈ Fn×n and S2 ∈ Fm×m , and we
observe that
(a) (b)
rank(S1 AS2 ) = rank(AS2 ) = rank(A),
and we are done.

Our next theorem states that the rank of a product of two matrices is no greater
than the minimum of the ranks of the two matrices.

Theorem 3.3.17. Let F be a field, and let A ∈ Fn×m and B ∈ Fm×p . Then
n o
rank(AB) ≤ min rank(A), rank(B) .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 259

   
Proof. Set A = a1 . . . am and B = b1 . . . bp . We must show that
rank(AB) ≤ rank(A) and rank(AB) ≤ rank(B).
 We first prove that rank(AB) ≤ rank(A). By definition, we have that AB =
Ab1 . . . Abp , and in particular, every column of AB is a linear combination of
the columns of A,70 i.e. every column of AB belongs to Span(a1 , . . . , am ) = Col(A).
Since Col(A) is a subspace of Fn (and is therefore a vector space in its own right),
Theorem 3.1.11(b) now guarantees that Col(AB) = Span(Ab1 , . . . , Abp ) is a subspace
of Col(A).71 Since Col(A) is finite-dimensional, Theorem 3.2.21 now implies that



dim Col(AB) ≤ dim Col(A) ,

and we deduce that

(∗)

(∗)
rank(AB) = dim Col(AB) ≤ dim Col(A) = rank(A),

where both instances of (*) follow from Theorem 3.3.4 (or alternatively, from Corol-
lary 3.3.11(a)).
We have now shown that rank(AB) ≤ rank(A). A completely analogous argument
shows that rank(B T AT ) ≤ rank(B T ), and we deduce that
(∗) (∗)
rank (AB)T = rank(B T AT ) ≤ rank(B T )


rank(AB) = = rank(B)

where both instances of (*) follow from Corollary 3.3.11(b).

Corollary 3.3.18. Let F be a field, and let A, B ∈ Fn×n . Then AB is invertible if

and only if A and B are both invertible.

Proof. If A and B are invertible, then Proposition 1.11.8(d) guarantees that AB

is invertible. For the other direction, assume that AB is invertible. Then by
the Invertible Matrix Theorem (version 1; see subsection 1.11.7), we have that
rank(AB) = n, and it suffices to show that rank(A) =  n and rank(B) = n. By
Theorem 3.3.17, we have that n = rank(AB) ≤ min rank(A), rank(B) , and it
follows that rank(A) ≥ n and rank(B) ≥ n. But since A and B are n × n matrices,
Proposition 1.6.3 now implies that rank(A) = n and rank(B) = n, and we are
done.

Left and right inverses. Suppose that A ∈ Fn×m , where F is some field. A left
inverse of A is a matrix B ∈ Fm×n such that BA = Im , and a right inverse of A is a
70
T
is any vector in Fm , then Ax = x1 a1 + · · · + xm am , which is

Indeed, if x = x1 . . . xm
a linear combination of the columns of A.
71
Indeed, we have shown that every column of AB belongs to Col(A), that is, that Ab1 , . . . , Abp ∈
Col(A). Since Col(A) is a vector space (because it is a subspace of Fn ), Theorem 3.1.11(b) guarantees
that Span(Ab1 , . . . , Abp ) is a subspace of Col(A). But Span(Ab1 , . . . , Abp ) = Col(AB). So,
Col(AB) is a subspace of Col(A).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 260

matrix C ∈ Fm×n such that AC = In . Thus, a left inverse (resp. right inverse) of a
matrix A is a matrix that we can multiply A by on the left (resp. on the right) in
order to obtain the identity matrix of the appropriate size. Consider, for example,
matrices
 
  1/2 0
2 0 0
A1 = and A2 =  0 0  ,
0 0 1/3
0 3
with entries understood to be in R. Then A1 A2 = I2 , and consequently, A1 is a left
inverse of A2 , and A2 is a right inverse of A1 . Obviously, a matrix need not have a
left or a right inverse. For example, zero matrices have no left inverses and no right
inverses. On the other hand, a matrix may possibly have more than one left inverse
or more than one right inverse.72 However, as Corollary 3.3.19 (below) shows, any
matrix A that has both a left inverse and a right inverse is in fact invertible (and
in particular, square), and moreover, both its left inverse and its right inverse are
unique are equal to A−1 .
Corollary 3.3.19. Let F be a field, let A ∈ Fn×m be a matrix, and assume that
B ∈ Fm×n is a left inverse of A (i.e. BA = Im ) and that C ∈ Fm×n is a right inverse
of A (i.e. AC = In ). Then A is invertible (and in particular square, i.e. m = n),
and B = C = A−1 .
Proof. First, we have that

(∗) (∗∗)  (∗∗∗)

m = rank(Im ) = rank(BA) ≤ min rank(B), rank(A) ≤ n,

where (*) follows from the fact that BA = Im , (**) follows from Theorem 3.3.17,
and (***) follows from Proposition 1.6.3 (because A is an n × m matrix and B is an
m × n matrix). Since AC = In , an analogous argument establishes that n ≤ m (we
simply use the fact that AC = In instead of BA = Im ). So, m = n. In particular,
we have that A, B, C ∈ Fn×n , and that BA = Im = In and AC = In . But now
B = BIn = B(|{z}
AC ) = (|{z}
BA )C = In C = C.
=In =In

So, BA = In and AB = AC = In . Thus, A is invertible, and its inverse is B = C.

This completes the argument.

Remark: Corollary 3.3.19 is the reason that we defined invertibility only for square
matrices. Any reasonable definition of an invertible matrix would entail the existence
of both a left and a right inverse for that matrix, and by Corollary 3.3.19, only
square matrices can have both a left and a right inverse.

As a corollary of Theorem 3.3.17 for square matrices, we get the following.

72
Examples?

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 261

Corollary 3.3.20. Let F be field, and let A, B ∈ Fn×n be such that AB = In or

BA = In . Then AB = BA = In , i.e. A and B are both invertible and are each
other’s inverses.

Remark: Note that Corollary 3.3.20 implies that if a square matrix A has a left or
a right inverse B, then B is in fact a “two-sided inverse” of A, i.e. the (ordinary)
inverse of A, and in particular, A is invertible.

Proof. By symmetry, we may assume that AB = In . The argument is analogous in

the case when BA = In (in that case, we simply swap the roles of A and B). We
then have that
n = rank(In )

= rank(AB) because AB = In
n o
≤ min rank(A), rank(B) by Theorem 3.3.17
n o
≤ max rank(A), rank(B)

by Proposition 1.6.3, since

≤ n
A and B are n × n matrices,

and it follows that

n o n o
n = min rank(A), rank(B) = max rank(A), rank(B) = n,

which in turn implies that rank(A) = rank(B) = n. But now the Invertible Matrix
Theorem (version 1; see subsection 1.11.7) guarantees that A and B are invertible.
Since AB = In , Proposition 1.11.3 now implies that A−1 = B and B −1 = A.

3.3.5 Extending a linearly independent set of vectors in Fn to a

basis of Fn
Proposition 3.3.21. Let F be a field, let {a1 , . . . , ak } be a linearly independent set
of vectors in Fn , and
 let {b1 , . . . , bn } be any basis of Fn . Then the pivot columns of
the matrix C := a1 . . . ak b1 . . . bn form a basis of Fn that extends the


linearly independent set {a1 , . . . , ak }.73

     
Proof. Set A := a1 . . . ak and B := b1 . . . bn , so that C = A B .
73
“Extends” simply means “contains as a subset.”

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 262

Since the columns of B form a basis of Fn , it is clear that Col(C) = Fn .74

Since the columns of the n × k matrix A are linearly independent, Theo-
rem 3.3.14(a) guarantees that rank(A) = k, and it follows that all columns of
A are pivot columns.75 Next, the pivot columns of C are the pivot columns of A (i.e.
all the columns of A, by what we just showed), plus possibly some columns of B;
moreover, by Theorem 3.3.4, the pivot columns of C form a basis of Span(C) = Fn .
This proves that the pivot columns of C indeed form a basis of Fn that extends
{a1 , . . . , ak }.

Remark: Proposition 3.3.21 will work for any basis {b1 , . . . , bn } of Fn . However,
in practice, it is usually easiest to use the standard basis En = {e1 , . . . , en } of Fn .

Example 3.3.22. Consider the vectors

  

1 0
 1   1 
a1 =  0 
 and a2 = 
 0 ,


0 1
with entries understood to be in Z2 . Extend the linearly independent set {a1 , a2 } to
a basis of Z42 . (Assume that the set {a1 , a2 } is indeed linearly independent.)
Solutution. We apply Proposition 3.3.21 to the linearly independent set {a1 , a2 } and
the standard basis E4 = {e1 , e2 , e3 , e4 } of Z42 . We form the matrix
 
1 0 1 0 0 0
   1 1 0 1 0 0 
C := a1 a2 e1 e2 e3 e4 =   0 0 0 0 1 0 .


0 1 0 0 0 1
By row reducing, we obtain
 
1 0 0 1 0 1
 0 1 0 0 0 1 
RREF(C) = 
 0
,
0 1 1 0 1 
0 0 0 0 1 0
74
Indeed, we have that

Fn ⊇ Col(C) because C ∈ Fn×(k+n)

= Span(a1 , . . . , ak , b1 , . . . , bn ) by the definition of Col(C)

⊇ Span(b1 , . . . , bn )

= Fn because {b1 , . . . , bn } is a basis of Fn ,

and it follows that Col(C) = Fn .

75
Indeed, since rank(A) = k, we know that A has k pivot columns. Since A has precisely k
columns, it follows that all columns of A are pivot columns.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 263

and we see that the pivot columns of C are the first, second, third, and fifth column.
It follows that
       
1 0 1 0
n 1   1   0
    0 o
{a1 , a2 , e1 , e3 } =  ,
 0   , , 
0   0   1 
0 1 0 0

is a basis of Z42 that extends {a1 , a2 }.

Proposition 3.3.23. Let F be a field, let a1 , . . . , ak ∈ Fn be arbitrary vectors, and

n

let {b1 , . . . , bn } be any basis of F . Set C := a1 . . . ak b1 . . . bn . Then
the pivot columns of C to the left of the vertical dotted line form a basis A of
Span(a1 , . . . , ak ), and all the pivot columns of C form a basis C of Fn that extends
A (i.e. satisfies A ⊆ C).
     
Proof. Set A := a1 . . . ak and B := b1 . . . bn , so that C = A B .
By definition, we have that Col(A) = Span(a1 , . . . , ak ).
First, by Theorem  the pivot columns of C form some basis C of Col(C).
3.3.4,
Next, since C = A B , and since the columns of B form a basis of Fn, we have


that Col(C) = Fn .76 So, C is a basis of Fn . Now, the pivot columns of C = A B

to the left of the vertical dotted line are precisely the pivot columns of A, and by
Theorem 3.3.4, the pivot columns of A form a basis A of Col(A) = Span(a1 , . . . , ak ).
Obviously, A ⊆ C. This completes the argument.

Example 3.3.24. Consider the vectors

      
1 2 1 2
 0   0   1   1 
 2 ,
a1 =  
 1 ,
a2 =  
 2 ,
a3 =  
 1 ,
a4 =  

0 0 2 2

with entries understood to be in Z3 . Find a basis A of Span(a1 , a2 , a3 , a4 ), and find

a basis C of Z43 that extends A (i.e. that satisfies A ⊆ C).

Solution. We apply Proposition 3.3.23 to the vectors a1 , a2 , a3 , a4 and the standard

basis E4 = {e1 , e2 , e3 , e4 } of Z43 . We form the matrix
 
C := a1 a2 a3 a4 e1 e2 e3 e4
 
1 2 1 2 1 0 0 0
 0 0 1 1 0 1 0 0 
= 
 2
.
1 2 1 0 0 1 0 
0 0 2 2 0 0 0 1
76
Here, the argument is the same as in the proof of Proposition 3.3.21.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 264

By row reducing, we obtain

 
1 2 0 1 0 0 2 1
 0 0 1 1 0 0 0 2 
RREF(C) = 
 0
,
0 0 0 1 0 1 0 
0 0 0 0 0 1 0 1
and we see that the pivot columns of C are the first, third, fifth and sixth column.
Since the first and third column are to the left of the vertical dotted line, while the
fifth and sixth are to the right, we see that
   
1 1
n 0   1 o
A := {a1 , a3 } =  , 
 2   2 
0 2
is a basis of Span(a1 , a2 , a3 , a4 ), and that
       
1 1 1 0
n 0   1   0   1 o
C := {a1 , a3 , e1 , e2 } =  ,
 2   2
, ,
  0   0


0 2 0 0

is a basis of Z43 that extends A.

3.3.6 The null space of a matrix

For field F and a matrix A ∈ Fn×m , we define the null space of A, denoted by Nul(A),
to be the set of all solutions of the homogeneous matrix-vector equation Ax = 0, i.e.
Nul(A) := x ∈ Fm | Ax = 0 .


Notation: In some texts, notation Ker(A) is used instead of Nul(A). “Ker” stands
for “kernel.”
Proposition 3.3.25. Let F be a field, and let A ∈ Fn×m . Then Nul(A) is a subspace
of Fm .
Terminology: The dimension of Nul(A) is called the nullity of the matrix A.

Proof. We apply Theorem 3.1.7. First, A0 = 0, and so 0 ∈ Nul(A). Next, if

u, v ∈ Nul(A), then
A(u + v) = Au + Av

= 0+0 because u, v ∈ Nul(A)

= 0,

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 265

and so u + v ∈ A. Finally, if u ∈ Nul(A) and α ∈ F, then

A(αu) = α(Au)

= α0 because u ∈ Nul(A)

= 0,
and so αu ∈ Nul(A). Theorem 3.1.7 now guarantees that Nul(A) is a subspace
of Fm .
Proposition 3.3.26. Let F be a field, and let A ∈ Fm×n . Then the columns of A
are linearly independent if and only if Nul(A) = {0}.
Proof. This essentially follows from the definition of Nul(A) and from Proposi-
tion 3.2.1. Indeed, by definition, Nul(A) is the set of all solutions of the homogeneous
matrix-vector equation Ax = 0; consequently,
the homogeneous matrix-vector equation Ax = 0
Nul(A) = {0} ⇐⇒
has only the trivial solution (i.e. the solution x = 0)

(∗)
⇐⇒ the columns of A are linearly independent,
where (*) follows from Proposition 3.2.1.
Example 3.3.27. Let
 
1 0 1 0 1
A =  0 1 0 1 0 ,
1 1 1 1 1


with entries understood to be in Z2 . Find a basis of Nul(A). What is dim Nul(A) ?
Proof. We begin by finding the general solution of the homogeneous matrix-vector
equation Ax = 0.77 By row reducing, we get
 
1 0 1 0 1
RREF(A) =  0 1 0 1 0 .
0 0 0 0 0
The general solution of Ax = 0 is
 
r+t
 s 
 
x =   r
,
 where r, s, t ∈ Z2 ,
 s 
t
77
As discussed in subsection 1.3.4 (see Example 1.3.17), when solving a homogeneous linear
system, we need only row reduce the coefficient matrix, and not the whole augmented matrix. The
same obviously applies to homogeneous matrix-vector equations.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 266

that is,
     
1 0 1

 0 


 1 


 0 

x = r
 1  + s
  0  + t
  0 ,
 where r, s, t ∈ Z2 .
 0   1   0 
0 0 1

So,      
1 0 1
n
 0  
  1  0 o
  
B := 
 1 ,
  0, 0 
  
 0   1  0 
0 0 1


is a basis of Nul(A), and it follows that dim Nul(A) = 3.

Remark: It is obvious that B spans the solution set of Ax = 0, that is, that B is
a spanning set of Nul(A). However, it might not be immediately obvious that B
is linearly independent. In view of Proposition 3.2.11(a), it suffices to show that
no vector in B is a linear combination of the other vectors in B. But this follows
immediately from the fact that each vector in B has a non-zero entry in a position
in which all the other vectors of B have a zero. The non-zero entries in question are
boxed below:      
1 0 1
n 0   1   0 o
    
B :=  1 , 0 , 0  .
     
 0   1   0 
0 0 1
Note that the boxed entries correspond to the entries in which our parameters (in
this case, r, s, t) appear for the first time, counting from bottom up, in our general
solution of the homogeneous matrix-vector equation Ax = 0, as indicated below:
 
r+t
 s 
 
x =   r ,
 where r, s, t ∈ Z2 .
 s 
t

Remark: Let A ∈ Fn×m (where F is some field).

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 267

1. We note that dim Nul(A) (i.e. the number of vectors in any basis of Nul(A))
will always be equal to the number of free variables in the general solution
of the homogeneous matrix-vector equation Ax = 0. We omit a fully formal
proof of this, but the basic argument is as in our solution of Example 3.3.27.

2. If the homogeneous matrix-vector equation Ax = 0 has only the trivial solution,

i.e. the solution x = 0, then Nul(A)

= {0}, and ∅ is the (unique) basis of
Nul(A). In this case, dim Nul(A) = 0, which is consistent with the point made
above (since in this case, our general solution of the homogeneous matrix-vector
equation Ax = 0 has no free variables).

The rank–nullity theorem (matrix version). Let F be a field, and let A ∈ Fn×m .
Then

rank(A) + dim Nul(A) = m
|{z} .
= number of
columns of A

Remark: We give a slightly informal proof of the theorem. We will give a fully formal
proof of the (more general) rank-nullity theorem for linear functions in chapter 4
(see subsection 4.2.2). As we shall see in subsection 4.2.2, the rank-nullity theorem
for linear functions immediately implies the rank-nullity theorem for matrices.

Proof (outline/informal). By definition, rank(A) is equal to the number of pivot

columns of A. On the other hand, when we compute the general solution of Ax = 0,
the number of free variables is equal to the number of non-pivot columns of A, and
the number of vectors

in a basis of Nul(A) is equal to the number of free variables.78
So, dim Nul(A) is equal to
the number of non-pivot columns of A. It now follows
that rank(A) + dim Nul(A) is equal to the number of columns of A, and we are
done.

Remark: The diagram below (informally) summarizes the idea behind the rank-
nullity theorem for matrices A ∈ Fn×m (where F is a field).


rank(A) + dim Nul(A) = m
|{z}
| {z } | {z }
= number of = number of = number of
pivot non-pivot columns of A
columns of A columns of A

= number of = number of
basic variables free variables
78
This last part (“the number of vectors in a basis of Nul(A) is equal to the number of free
variables”) is not fully justified, but you should be able to convince yourself that this is true by
reading the solution of Example 3.3.27 above.

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 268

Proposition 3.3.28 (below) states that performing elementary row operations on

a matrix, or deleting its zero rows, does not alter the null space of the matrix. This
follows immediately from the definition of the null space, but is useful to state as
a separate proposition for easy future reference (in particular, we will need this in
section 6.6).

Proposition 3.3.28. Let F be a field.

(a) If A, B ∈ Fn×m are row equivalent matrices, then Nul(A) = Nul(B).

(b) If a matrix C is obtained from a matrix A ∈ Fn×m by possibly deleting some

zero rows of A, then Nul(A) = Nul(C).
n×m are row equivalent.
Proof. We firstprove (a).
 Assume
  matrices A, B ∈ F79
that
Then matrices A 0 and B 0 are row equivalent, and consequently, the
matrix-vector equations Ax = 0 and Bx = 0 are equivalent (i.e. have exactly the
same solutions). It follows that

Nul(A) = {x ∈ Fm | Ax = 0} by definition

because the matrix-vector equations

= {x ∈ Fm | Bx = 0}
Ax = 0 and Bx = 0 are equivalent

= Nul(B) by definition.

This proves (a).

For (b), we note that if a matrix C is obtained from a matrix A by possibly
deleting some zero rows of A, then the matrix-vector equations Ax = 0 and Cx = 0
are equivalent,80 and similarly to the above, it follows that
Nul(A) = {x ∈ Fm | Ax = 0} by definition

because the matrix-vector equations

= {x ∈ Fm | Cx = 0}
Ax = 0 and Cx = 0 are equivalent

= Nul(C) by definition.
This proves (b).

3.3.7 The Invertible Matrix Theorem (version 2)

In subsection 1.11.7, we stated and proved the Invertible Matrix Theorem (version 1).
Using the results of the present section, we can now extend that theorem, as follows.
79
This is “obvious,” but it also follows from Proposition 1.3.22(a).
80
Details?

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 269

The Invertible Matrix Theorem (version 2). Let F be a field, and let A ∈ Fn×n
be a square matrix. Further, let f : Fn → Fn be given by f (x) = Ax for all x ∈ Fn .81
Then the following are equivalent:
(a) A is invertible (i.e. A has an inverse);

(b) AT is invertible;

(c) RREF(A) = In ;

= In B for some matrix B ∈ Fn×n ;

 
 
(d) RREF A In

(e) rank(A) = n;

(f ) rank(AT ) = n;

(g) A is a product of elementary matrices;

(h) the homogeneous matrix-vector equation Ax = 0 has only the trivial solution (i.e.
the solution x = 0);

(i) there exists some vector b ∈ Fn such that the matrix-vector equation Ax = b
has a unique solution;

(j) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has a unique solution;

(k) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has at most one
solution;

(l) for all vectors b ∈ Fn , the matrix-vector equation Ax = b is consistent;

(m) f is one-to-one;

(n) f is onto;

(o) f is an isomorphism;

(p) there exists a matrix B ∈ Fn×n such that BA = In (i.e. A has a left inverse);

(q) there exists a matrix C ∈ Fn×n such that AC = In (i.e. A has a right inverse);

(r) the columns of A are linearly independent;

(s) the columns of A span Fn (i.e. Col(A) = Fn );

(t) the columns of A form a basis of Fn ;

(u) the rows of A are linearly independent;

81
Since f is a matrix transformation, Proposition 1.10.4 guarantees that f is linear. Moreover, A
is the standard matrix of f .

Irena Penev
Linear Algebra 1 & 2
Chapter 3. Vector spaces 270

(v) the rows of A span F1×n (i.e. Row(A) = F1×n );

(w) the rows of A form a basis of F1×n ;

(x) Nul(A) = {0} (i.e. dim Nul(A) = 0).

Proof. The fact that (a)-(o) are equivalent follows from the Invertible Matrix Theorem
(version 1; see subsection 1.11.7). By definition, (a) implies (p) and (q). On the
other hand, Corollary 3.3.20 guarantees that any one of (p) and (q) implies (a).
By Corollary 3.3.15, (r)-(w) are equivalent to each other, as well as to (e). Finally,
Proposition 3.3.26 guarantees that (r) and (x) are equivalent. This completes the
argument.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 271

Chapter 4

Linear functions

4.1 Linear functions: definition, examples, and basic

properties
We have already studied linear functions from Fm to Fn , for a field F (see section 1.10).
The concept of a linear function can easily be extended to a more general setting,
that of arbitrary vector spaces, as follows.
Given vector spaces U and V over a field F, we say that a function f : U → V is
linear provided it satisfies the following two conditions (axioms):
1. for all vectors u1 , u2 ∈ U , we have that f (u1 + u2 ) = f (u1 ) + f (u2 );
2. for all vectors u ∈ U and scalars α ∈ F, we have that f (αu) = αf (u).
If the linear function f is also a bijection, then we say that it is an isomorphism,
and that the vector spaces U and V are isomorphic. Linear functions are also called
linear transformations.
Remark: We note that in the definition of a linear function, the two vector spaces
(the domain and the codomain of the function) must be over the same field F.

Let us now take a look at a couple of examples (Examples 4.1.1 and 4.1.2). Recall
that a real vector space is a vector space over the field R.

Example 4.1.1. Let PR be the real vector space of all polynomials with coefficients
in R. Show that the function D : PR → PR given by
n n
ak xk kak xk−1
P
P
D =
k=0 k=1

for all integers n ≥ 0 and real numbers a0 , . . . , ak , is linear.

Remark: If you have studied calculus, you will recognize this as the formula for the
derivative of polynomial functions.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 272

Solution. We need to check that D satisfies the two axioms from the definition of a
linear function.
1. Fix p(x), q(x) ∈ PR . Then there exists an integer n ≥ 0 and real numbers
a0 , . . . , an , b0 , . . . , bn such that
n n
ak xk bk xk .
P P
p(x) = and q(x) =
k=0 k=0



Remark: Here, n is some non-negative integer such that deg p(x) , deg q(x) ≤ n.
The inequality may possibly be strict, i.e. it is possible that an = 0 or bn = 0.
We now compute:
n
 P n

ak xk + bk xk


P
D p(x) + q(x) = D
k=0 k=0

n
P 
= D (ak + bk )xk
k=0

n
k(ak + bk )xk−1
P
=
k=1

n
P n
 P 
= kak xk−1 + kbk xk−1
k=1 k=1

n
P  n
P 
= D ak xk + D ak xk
k=0 k=0



= D p(x) + D q(x) .

2. Fix p(x) ∈ PF and α ∈ F. Then there exists an integer n ≥ 0 and real numbers
a0 , . . . , an such that
n
ak xk .
P
p(x) =
k=0

We now compute:
 Pn

ak xk


D αp(x) = D α
k=0

n
P 
= D (αak )xk
k=0

n
k(αak )xk−1
P
=
k=0

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 273

n
kak xk−1
P
= α
k=0

n
P 
= αD ak xk
k=0


= αD p(x) .

From 1. and 2., we conclude that D is linear.

Here is another example, for those who have studied calculus.

Example 4.1.2. Let Diff(R) be the real vector space of all differentiable functions
from R to R, and let Func(R) be the real vector space of all functions from R to R.
Show that the function D : Diff(R) → Func(R) given by D(f ) = f ′ for all f ∈ Diff(R)
is linear. (As usual, f ′ denotes the derivative of f .)

Proof. 1. Fix f, g ∈ Diff(R). Then by the properties of the derivative, we have that
D(f + g) = (f + g)′ = f ′ + g ′ = D(f ) + D(g).
2. Fix f ∈ Diff(R) and α ∈ R. Then by the properties of the derivative, we have
that D(αf ) = (αf )′ = αf ′ = αD(f ).
From 1. and 2., we conclude that D is linear.

In the remainder of this section, we prove some basic properties of linear functions.
For the most part (though not exclusively), these are generalizations of the results
that we proved in section 1.10 for linear functions f : Fm → Fn (where F is a field).
As we shall see, most of the results readily generalize to linear functions between
arbitrary vectors spaces (over the same field), with one exception: linear functions
between general vector spaces do not have standard matrices. It is in fact possible to
define the matrix of a linear function between non-trivial,1 finite-dimensional vectors
spaces, but such matrices depend on the particular choice of basis of the domain and
codomain (see section 4.5).

4.1.1 Basic properties of linear functions

Theorem 4.1.3. Let U and V be vector spaces over a field F, and let f : U → V be
a function. Then the following are equivalent:

(i) f is linear;
1
Recall that a vector space is trivial if it only contains the zero vector (i.e. if its dimension is 0),
and it is non-trivial if it contains at least one non-zero vector (i.e. if its dimension is greater than 0).

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 274

(ii) for all vectors u1 , u2 ∈ U and scalars α1 , α2 ∈ F, we have that

f (α1 u1 + α2 u2 ) = α1 f (u1 ) + α2 f (u2 ).

Proof. This easily follows from the definition of a linear function. The details are
left as an exercise.

Proposition 4.1.4. Let U and V be vector spaces over a field F, and let f : U → V
be a linear function. Then for all u1 , u2 ∈ U , we have that

f (u1 − u2 ) = f (u1 ) − f (u2 ).

Proof. Fix u1 , u2 ∈ U . Then

(∗)
f (u1 − u2 ) + f (u2 ) = f (u1 − u2 + u2 ) = f (u1 ),

where (*) follows from the linearity of f . By subtracting f (u2 ) from both sides, we
get that f (u1 − u2 ) = f (u1 ) − f (u2 ), which is what we needed to show.
Alternatively, we could observe that for all u1 , u2 ∈ U , we have the following:

f (u1 − u2 ) = f (u1 + (−1)u2 ) by Proposition 3.1.3(d)

= f (u1 ) + f ((−1)u2 ) by the linearity of f

= f (u1 ) + (−1)f (u2 ) by the linearity of f

= f (u1 ) − f (u2 ) by Proposition 3.1.3(d).

The following proposition is an analogue of Proposition 1.10.1 for general linear

functions.

Proposition 4.1.5. Let U and V be vector spaces over a field F, and let f : U → V be
a linear function. Then for all vectors u1 , . . . , uk ∈ U and all scalars α1 , . . . , αk ∈ F,
we have that
P k  Pk
f αi ui = αi f (ui ),
i=1 i=1

of, written in another way, that

 
f α1 u1 + · · · + αk uk = α1 f (u1 ) + · · · + αk f (uk ).

Proof. This follows from the definition of a linear function via an easy induction on
k. The details are left as an exercise.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 275

We remark that our next proposition (Proposition 4.1.6) was already proven in the
context of linear functions from Fm to Fn , where F is a field (see Proposition 1.10.3).
Here, we prove the proposition more generally, with an essentially identical proof.

Proposition 4.1.6. Let U and V be vector spaces over a field F, and let f : U → V
be a linear function. Then f (0) = 0.2

Proof. We observe that

(∗) (∗∗) (∗)
f (0) = f (0 · 0) = 0f (0) = 0,

where both instances of (*) follows from Proposition 3.1.3(a), and (**) follows from
the fact that f is linear.

4.1.2 Making new linear functions out of old ones

Our first proposition (Proposition 4.1.7 below) is a generalization of Proposi-
tion 1.10.13 for linear functions between arbitrary vector spaces. The proof of
Proposition 1.10.13 relied on standard matrices. However, linear functions in general
need not have an associated standard matrix (for example, the derivative function
D from Example 4.1.1, and also from Example 4.1.2, does not have an associated
standard matrix). So, our proof of Proposition 4.1.7 does not rely on matrices, and
instead, it uses the definition of a linear function.

Proposition 4.1.7. Let U , V , and W be vector spaces over a field F. Then all the
following hold:

(a) for all linear functions f, g : U → V , the function f + g is linear;3

(b) for all linear functions f : U → V and scalars α ∈ F, the function αf : U → V

is linear;4

(c) for all linear functions f : U → V and g : V → W , the function g ◦ f is linear.5

g◦f

f g
U V W
2
Technically, this means f (0U ) = 0V , where 0U is the zero vector in U , and 0V is the zero vector
in V .
3
As usual, the function f + g : U → V is defined by (f + g)(u) = f (u)
+ g(u) for all u ∈ U .
4
As usual, the function αf : U → V is defined by (αf )(u) = α f (u) for
all u ∈ U .
5
As usual, the function g ◦ f : U → W is defined by (g ◦ f )(u) = g f (u) for all u ∈ U .

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 276

Proof. We prove (c). The proofs of (a) and (b) are left as an exercise. Fix linear
functions f : U → V and g : V → W . We must show that g ◦ f is linear.
1. Fix u1 , u2 ∈ U . Then


(g ◦ f )(u1 + u2 ) = g f (u1 + u2 )


= g f (u1 ) + f (u2 ) because f is linear



= g f (u1 ) + g f (u2 ) because g is linear

= (g ◦ f )(u1 ) + (g ◦ f )(u2 ).

2. Fix u ∈ U and α ∈ F. Then

(∗)
(∗∗)

(g ◦ f )(αu) = g f (αu) = g αf (u) = αg f (u) = α(g ◦ f )(u),

where (*) follows from the fact that f is linear, and (**) follows from the fact that g
is linear.
From 1. and 2., we conclude that g ◦ f is linear.

Given vector spaces U and V over a field F, the set of all linear functions from U to
V is denoted by Hom(U, V ).6 We note that Hom(U, V ) is a vector space over F. The
vector addition and scalar multiplication operations in Hom(U, V ) are the addition
and scalar multiplication of functions; by parts (a) and (b) of Proposition 4.1.7,
Hom(U, V ) is indeed closed under the addition and scalar multiplication of functions.
The zero vector in Hom(U, V ) is the zero function, i.e. the function f0 : U → V given
by f0 (u) = 0V for all u ∈ U , where 0V is the zero of the vector space V .

4.1.3 A remark on infinity

In the context of vector space dimension, we adopt the convention that n ≤ ∞ for all
non-negative integers n, and that ∞ = ∞. Readers who have some familiarity with
infinite cardinals might object to “∞ = ∞” on the grounds that “there is more than
one kind of infinity, and some infinities are bigger than others.” We will not worry
about this in these lecture notes. In particular, when we write dim(U ) = dim(V )
for some vector spaces U and V , we mean that either U and V have the same
finite dimension, or U and V are both infinite-dimensional. On the other hand,
“dim(U ) < dim(V ),” in particular, means that U if finite-dimensional (and unless
specified otherwise, V may possibly be infinite-dimensional).
6
Linear functions are sometimes called “homomorphisms,” which is where the notation Hom(U, V )
comes from.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 277

4.2 The image and kernel of a linear function. The

rank-nullity theorem
We begin with a definition. Suppose we are given a function f : A → B (not
necessarily linear). Then we define the following:

ˆ for all subsets A′ ⊆ A, the set f [A′ ] := {f (a) | a ∈ A′ } is called the image of
A′ under the function f ;

ˆ the set Im(f ) := f [A] is called the image of f ;

ˆ for all subsets B ′ ⊆ B, the set f −1 [B ′ ] := {a ∈ A | f (a) ∈ B ′ } is called the

preimage of B ′ under f .

We note that in some texts, the image of f is called the “range of f .”

Remark: If f : A → B is a bijection, then it has an inverse function f −1 : B → A.
In this case, for B ′ ⊆ B, the notation f −1 [B ′ ] can be interpreted in two ways: as
the preimage of B ′ under f , and as the image of B ′ under the inverse function f −1 .
However, in both cases, f −1 [B ′ ] is the same subset of A, which is why we usually do
not need to specify which interpretation we have in mind.

Given a linear function f : U → V , where U and V are vector spaces over a

field F, the kernel of f is defined to be the set

Ker(f ) := {u ∈ U | f (u) = 0}.

Note that this means that Ker(f ) = f −1 [{0}], i.e. Ker(f ) is the preimage of the set
{0} under f . We further note the kernel is only defined for linear functions, and
not for general functions.
In the case of linear functions from Fm to Fn (where F is a field), Proposition 4.2.1
(below) gives the correspondence between the image and kernel of the linear function
on the one hand, and the column and null space of the standard matrix on the other
hand. Note, however, that the image and kernel are defined for all linear functions,
not just those from Fm to Fn (see Example 4.2.2 below).

Proposition 4.2.1. Let F be a field, let f : Fm → Fn be a linear function, and let

A ∈ Fn×m be the standard matrix of f . Then both the following hold:

(a) Im(f ) = Col(A);

(b) Ker(f ) = Nul(A).

Proof. For (a), we observe that

(∗) (∗∗)
Col(A) = {Ax | x ∈ Fm } = {f (x) | x ∈ Fm } = Im(f ),

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 278

where (*) follows from Proposition 3.3.2(a), and (**) follows from the fact that A is
the standard matrix of f .
For (b), we observe that
(∗)
Nul(A) = {x ∈ Fm | Ax = 0} = {x ∈ Fm | f (x) = 0} = Ker(f ),

where (*) follows from the fact that A is the standard matrix of f .

Example 4.2.2. Let PR be the real vector space of all polynomials with coefficients
in R. Consider the function D : PR → PR given by
n n
ak xk ) = kak xk−1
P P
D(
k=0 k=1

for all integers n ≥ 0 and real numbers a0 , . . . , ak . By Example 4.1.1, D is linear.

Clearly, Ker(D) is the set of all constant polynomials, and Im(D) is the set of all
polynomials (i.e. Im(D) = PR ).

As an easy corollary of Theorem 3.1.7, we get the following theorem.

Theorem 4.2.3. Let U and V be vector spaces over a field F, and let f : U → V be
a linear function. Then all the following hold:

(a) for all subspaces U ′ of U , we have that f [U ′ ] is a subspace of V ;

(b) Im(f ) is a subspace of V ;

(c) for all subspaces V ′ of V , we have that f −1 [V ′ ] is a subspace of U ;

(d) Ker(f ) is a subspace of U .

Proof. Since U is a subspace of itself, (a) implies (b). Similarly, since Ker(f ) =
f −1 [{0}] and {0} is a subspace of V , we have that (c) implies (d). So, it suffices
to prove (a) and (c). We prove those two parts using Theorem 3.1.7. To avoid any
possible confusion, we will denote the zero vectors of the vector spaces U and V by
0U and 0V , respectively.
We first prove (a). Fix a subspace U ′ of U . We must show that f [U ′ ] is a
subspace of V . Since f : U → V and U ′ ⊆ U , we have that f [U ′ ] ⊆ V . In view of
Theorem 3.1.7, it now suffices to prove the following:

(i) 0V ∈ f [U ′ ];

(ii) for all v1 , v2 ∈ f [U ′ ], we have that v1 + v2 ∈ f [U ′ ];

(iii) for all v ∈ f [U ′ ] and α ∈ F, we have that αv ∈ f [U ′ ].

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 279

We first prove (i). Since U ′ is a subspace of U , Theorem 3.1.7 guarantees that

0U ∈ U ′ . On the other hand, by Proposition 4.1.6, we have that f (0U ) = 0V , and it
follows that 0V ∈ f [U ′ ]. This proves (i).
Next, we prove (ii). Fix v1 , v2 ∈ f [U ′ ]; we must show that v1 + v2 ∈ f [U ′ ].
Since v1 , v2 ∈ f [U ′ ], we know that there exist u1 , u2 ∈ U ′ such that v1 = f (u1 ) and
v2 = f (u2 ). Since U ′ is a subspace of U , we have that u1 + u2 ∈ U ′ .7 But now we
have that
(∗) (∗∗)
v1 + v2 = f (u1 ) + f (u2 ) = f (u1 + u2 ) ∈ f [U ′ ],

where (*) follows from the fact that f is linear, and (**) follows from the fact that
u1 + u2 ∈ U ′ . This proves (ii).
It remains to prove (iii). Fix v ∈ f [U ′ ] and α ∈ F; we must show that αv ∈ f [U ′ ].
Since v ∈ f [U ′ ], we know that there exists some u ∈ U ′ such that v = f (u). Since
U ′ is a subspace of U , we have that αu ∈ U ′ .8 But now we have that

(∗) (∗∗)
αv = αf (u) = f (αu) ∈ f [U ′ ],

where (*) follows from the fact that f is linear, and (**) follows from the fact that
αu ∈ U ′ .
We have now proven (i), (ii), and (iii). So, by Theorem 3.1.7, we have that f [U ′ ]
is a subspace of V . This proves (a).
It remains to prove (c). Fix a subspace V ′ of V . We must show that f −1 [V ′ ] is a
subspace of U . Since f : U → V and V ′ ⊆ V , we have that f −1 [V ′ ] ⊆ U . In view of
Theorem 3.1.7, it now suffices to prove the following:

(i) 0U ∈ f −1 [V ′ ];

(ii) for all u1 , u2 ∈ f −1 [V ′ ], we have that u1 + u2 ∈ f −1 [V ′ ];

(iii) for all u ∈ f −1 [V ′ ] and α ∈ F, we have that αu ∈ f −1 [V ′ ].

We first prove (i). Since V ′ is a subspace of V , Theorem 3.1.7 guarantees that

0V ∈ V ′ . On the other hand, by Proposition 4.1.6, we have that f (0U ) = 0V . So,
f (0U ) = 0V ∈ V ′ , and it follows that 0U ∈ f −1 [V ′ ]. This proves (i).
Next, we prove (ii). Fix u1 , u2 ∈ f −1 [V ′ ]; we must show that u1 + u2 ∈ f −1 [V ′ ],
i.e. that f (u1 + u2 ) ∈ V ′ . Since u1 , u2 ∈ f −1 [V ′ ], we have that v1 := f (u1 ) and
v2 := f (u2 ) belong to V ′ . Since V ′ is a subspace of V , we have that v1 + v2 ∈ V ′ .
We now have that
(∗)
f (u1 + u2 ) = f (u1 ) + f (u2 ) = v1 + v2 ∈ V ′ ,
7
This actually follows from the definition of a subspace. However, it is also possible to use
Theorem 3.1.7.
8
Again, this follows from the definition of a subspace. Alternatively, we can use Theorem 3.1.7.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 280

where (*) follows from the fact that f is linear. This proves (ii).
It remain to prove (iii). Fix u ∈ f −1 [V ′ ] and α ∈ F; we must show that
αu ∈ f −1 [V ′ ], i.e. that f (αu) ∈ V ′ . Since u ∈ f −1 [V ′ ], we know that v := f (u)
belongs to V ′ . Since V ′ is a subspace of V , we have that αv ∈ V ′ . We now have that
(∗)
f (αu) = αf (u) = αv ∈ V ′ ,

where (*) follows from the fact that f is linear. This proves (iii).
We have now proven (i), (ii), and (iii). So, by Theorem 3.1.7, we have that
f −1 [V ′ ] is a subspace of U . This proves (c).

4.2.1 One-to-one linear functions and kernel

Theorem 4.2.4. Let U and V be vector spaces over a field F, and let f : U → V be
a linear function. Then f is one-to-one if and only if Ker(f ) = {0}.

Proof. To avoid any possible confusion, we denote by 0U the zero vector of the vector
space U , and we denote by 0V the zero vector of the vector space V . We need to
show that f is one-to-one if and only if Ker(f ) = {0U }.
Suppose first that f is one-to-one. By Proposition 4.1.6, we have that f (0U ) = 0V ,
and it follows that 0U ∈ Ker(f ). It remains to show that 0U is the only element
of Ker(f ). So, fix any u ∈ Ker(f ). Then f (u) = 0V = f (0U ), and so since f is
one-to-one, we have that u = 0U . This proves that Ker(f ) = {0U }.
Suppose now that Ker(f ) = {0U }. Fix u1 , u2 ∈ U , and assume that f (u1 ) =
f (u2 ); we must show that u1 = u2 . We note that
(∗) (∗∗)
f (u1 − u2 ) = f (u1 ) − f (u2 ) = 0V ,

where (*) follows from Proposition 4.1.4, and (**) follows from the fact that f (u1 ) =
f (u2 ). So, u1 − u2 ∈ Ker(f ). Since Ker(f ) = {0U }, it follows that u1 − u2 = 0U ,
and consequently, u1 = u2 . This proves that f is one-to-one.

4.2.2 The rank of a linear function. The rank-nullity theorem

Suppose that U and V are vector spaces over a field F, and that f : U → V is a linear
function. By Theorem 4.2.3, Im(f ) is a subspace of V , and Ker(f ) is a subspace of
U . The rank of f is defined to be


rank(f ) := dim Im(f ) ,

and the nullity of f is dim(Ker(f )). We note that both the rank and the nullity of f
may possibly be infinite.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 281

f
Ker(f ) Im(f )

U V
Proposition 4.2.5. Let U and V be vector spaces over a field F, and let f : U → V
be a linear function. Then rank(f ) ≤ dim(V ).

Proof. We may assume that n := dim(V ) is finite, for otherwise, this is immediate.
By Theorem 4.2.3,

Im(f ) is a subspace of V , and so by Theorem 3.2.21, we have
that dim Im(f ) ≤ dim(V ), i.e. rank(f ) ≤ dim(V ).

By Proposition 4.2.4, a linear function if one-to-one if and only if its nullity is 0.

As our next proposition shows, a linear function with a finite-dimensional codomain
is onto if and only if the rank of the linear function is equal to the dimension of its
codomain.

Proposition 4.2.6. Let U and V be vector spaces over a field F, and let f : U → V
be a linear function. Assume that V is finite-dimensional. Then f is onto if and
only if rank(f ) = dim(V ).

Proof. We have the following sequence of equivalent statements:

(∗)
f is onto ⇐⇒ Im(f ) = V

(∗∗)

⇐⇒ dim Im(f ) = dim(V )

(∗∗∗)
⇐⇒ rank(f ) = dim(V ),

where (*) follows from the definition of an onto function, (**) follows from Theo-
rem 3.2.21 (since Im(f ) is a subspace of V , and V is finite-dimensional), and (***)
follows from the definition of rank.

Warning: Proposition 4.2.6 only applies to linear functions that have a finite-
dimensional codomain. Do not apply Proposition 4.2.6 to linear functions with
an infinite-dimensional codomain!

In the case of a linear function f : Fm → Fn (where F is a field), there is a

natural relationship between the rank and nullity of f and the rank and nullity of
the standard matrix of f , as our next proposition shows.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 282

Proposition 4.2.7. Let F be a field, let f : Fm → Fn be a linear function, and let

A ∈ Fn×m be the standard matrix of f . Then both the following hold:

(a) rank(f ) = rank(A);

(b) dim Ker(f ) = dim Nul(A) .

Proof. By Proposition 4.2.1, we have that Im(f ) = Col(A) and Ker(f ) = Nul(A).
The latter immediately implies (b). For (a), we observe that


(∗)
rank(f ) = dim Im(f ) = dim Col(A) = rank(A),

where (*) follows from Theorem 3.3.4.

As we pointed out above, both the rank and the nullity of a linear function
f : U → V (where U and V are vector spaces over a field F) may possibly be infinite.
However, as the rank-nullity theorem for linear functions (below) states, if the domain
U is finite-dimensional, then both Im(f ) and Ker(f ) are finite-dimensional, and
moreover, the sum of their dimensions (i.e. the sum of rank and nullity of f ) is
precisely dim(U ). We also note that, together with Proposition 4.2.7, the rank-
nullity theorem for linear functions immediately implies the rank-nullity theorem for
matrices (the details are below).

The rank–nullity theorem (linear function version). Let U and V be vector

spaces over a field F, and assume that U is finite-dimensional. Then every linear
function f : U → V satisfies


rank(f ) + dim Ker(f ) = dim(U ),

and in particular, both Ker(f ) and Im(f ) are finite-dimensional.

Proof. By Theorem 4.2.3, Ker(f ) is a subspace of U , and Im(f ) is a subspace

of V . Next, since U is finite-dimensional, Theorem 3.2.21 guarantees
that its
subspace Ker(f )
is finite-dimensional and satisfies dim Ker(f ) ≤ dim(U ). Set
k := dim Ker(f ) and
m := dim(U ) (so, k ≤ m). By definition, we have that
rank(f ) = dim Im(f ) . Thus, to complete the proof, we need only exhibit
a basis of
Im(f ) of size m − k. Indeed, this will imply rank(f ) = dim Im(f ) = m − k, and
the result will follow immediately.9
We proceed as follows. Fix a basis {u1 , . . . , uk } of Ker(f ).10 Then {u1 , . . . , uk }
is a linearly independent set in a finite-dimensional vector space U ; so, by The-
orem 3.2.19, {u1 , . . . , uk } can be extended to a basis {u1 , . . . , uk , uk+1 , . . . , um }
9


Indeed,
if rank(f ) = dim Im(f ) = m − k, then Im(f ) is finite-dimensional and rank(f ) +
dim Ker(f ) = (m − k) + k = m = dim(U ).

10
We are using the fact that dim Ker(f ) = k, and so every basis of Ker(f ) has k elements.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 283

of 11 We will complete the proof by showing that the (m − k)-element set

 U.
f (uk+1 ), . . . , f (um ) is a basis of Im(f ).
Before proceeding with the technical details, we give a picture of our set-up (see
below). Since u1 , . . . , uk all belong to Ker(f ), the function f maps them all to 0.
On the other hand, as we shall prove (see Claim 1), vectors f (uk+1 ), . . . , f (um ) are
linearly independent, and therefore, they are pairwise distinct and non-zero. (Thus,
the picture below is indeed correct, but have not proven this yet!)

f Im(f )

um f (um)
... ...
uk+2 f (uk+2)
Ker(f ) uk+1 f (uk+1)
uk
...
0
u2
u1

U V
Claim 1. Vectors f (uk+1 ), . . . , f (um ) are linearly independent.

Proof of Claim 1. Fix scalars αk+1 , . . . , αm ∈ F such that

αk+1 f (uk+1 ) + · · · + αm f (um ) = 0.

We must show that αk+1 = · · · = αm = 0. Note that

(∗)
f (αk+1 uk+1 + · · · + αm um ) = αk+1 f (uk+1 ) + · · · + αm f (um ) = 0,

where (*) follows from the fact that f is linear (and more precisely, from Propo-
sition 4.1.5). But now we have that αk+1 uk+1 + · · · + αm um ∈ Ker(f ). Since
{u1 , . . . , uk } is a basis of Ker(f ), we have that αk+1 uk+1 + · · · + αm um is a linear
combination of the vectors u1 , . . . , uk , i.e. there exist scalars α1 , . . . , αk ∈ F such
that
αk+1 uk+1 + · · · + αm um = α1 u1 + · · · + αk uk .
But this implies that

−α1 u1 − · · · − αk uk + αk+1 uk+1 + · · · + αm um = 0.

Since vectors u1 , . . . , uk , uk+1 , . . . , um are linearly independent (because they form a

basis of U ), we deduce that −α1 = · · · = −αk = αk+1 = · · · = αm = 0. In particular,
αk+1 = · · · = αm = 0, and it follows that vectors f (uk+1 ), . . . , f (um ) are indeed
linearly independent, which is what we needed to show. ♦
11
We are using the fact that dim(U ) = m, and so every basis of U has m elements.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 284

Claim 2. Im(f ) = Span f (uk+1 ), . . . , f (um ) .

Proof of Claim 2. By definition, f (uk+1 ), . . . , f (um ) ∈ Im(f ). Since Im(f ) is a

subspace of V (and therefore a vector
space in its own right), Theorem 3.1.11
guarantees that Span f (uk+1 ), . . . , f (um ) is a subspace (and in particular,

a subset)
of Im(f ). It remains to show that Im(f ) ⊆ Span f (uk+1 ), . . . , f (um ) , i.e. that every
vector in Im(f ) is a linear combination of the vectors f (uk+1 ), . . . , f (um ).
Fix v ∈ Im(f ). Then there exists some u ∈ U such that v = f (u). Since
{u1 , . . . , um } is a basis of U , we know that there exist scalars α1 , . . . , αm ∈ F such
that u = α1 u1 + · · · + αm um . We now have the following:

v = f (u)

= f (α1 u1 + · · · + αm um )

(∗)
= α1 f (u1 ) + · · · + αm f (um )

(∗∗)
= αk+1 f (uk+1 ) + · · · + αm f (um ),

where (*) follows from the fact that f is linear (and more precisely, from Proposi-
tion 4.1.5), and (**) follows from the fact that f (u1 ) = · · · = f (uk ) = 0 (because
u1 , . . . , uk ∈ Ker(f )). This proves that v is indeed a linear combination of the
vectors f (uk+1 ), . . . , f (um ), and we are done. ♦

Clearly, Claims 1 and 2 together imply that f (uk+1 ), . . . , f (um ) is a basis of
Im(f ). This completes the argument.

We are now ready to give a fully formal proof of the rank-nullity theorem for
matrices, one that relies on the rank-nullity theorem for linear functions, which we
just proved.
The rank–nullity theorem (matrix version). Let F be a field, and let A ∈ Fn×m .
Then

rank(A) + dim Nul(A) = m
|{z} .
= number of
columns of A

Proof. Let f : Fm → Fn be given by f (u) = Au for all u ∈ Fm . By Proposition 1.10.4,

f is linear, and obviously, A is the standard matrix of f . We now have that
(∗) (∗∗)
dim(Fm ) = m,



rank(A) + dim Nul(A) = rank(f ) + dim Ker(f ) =

where (*) follows from Proposition 4.2.7, and (**) follows from the rank-nullity
theorem for linear functions.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 285

Dimension considerations. The following is an easy corollary of the rank-nullity

theorem for linear functions.

Corollary 4.2.8. Let U and V be vector spaces over a field F, and let f : U → V
be a linear function. Then

rank(f ) ≤ min dim(U ), dim(V ) .


Remark: By definition, rank(f ) = dim Im(f ) . So, Corollary 4.2.8 states that the
dimension of the image of a linear function is at most the dimension of the domain
and also at most the dimension of the codomain. We note that in Corollary 4.2.8,
vector spaces U and V may possibly be infinite-dimensional.

Proof of Corollary 4.2.8. The fact that rank(f ) ≤ dim(V ) follows from Proposi-
tion 4.2.5. It remains to show that rank(f ) ≤ dim(U ). If dim(U ) = ∞, then this is
immediate. So, let us assume that U is finite-dimensional. Then

(∗)
rank(f ) ≤ rank(f ) + dim Ker(f ) = dim(U ),

where (*) follows from the rank-nullity theorem for linear functions.

Corollary 4.2.9. Let U and V be vector spaces over a field F, and let f : U → V
be a linear function. Then for any subspace U ′ of U , we have that

dim f [U ′ ] ≤ min dim(U ′ ), dim(V ) .



U′ f [U ′]

U V
Proof. Consider the function f ′ := f ↾ U ′ (the restriction of f to U ′ ).12 Since U ′ is a
subspace of U and f : U → V is linear, we have that f ′ : U ′ → V is also linear. So,
(∗)
dim f [U ′ ] = dim f ′ [U ′ ] = dim Im(f ′ ) min dim(U ′ ), dim(V ) ,




≤

where (*) follows from Corollary 4.2.8.

Geometric considerations. First of all, recall that subspaces of a Euclidean

space Rk are {0}, lines through the origin, planes through the origin, and higher
dimensional generalizations. Now, suppose that f : Rm → Rn is a linear function.
By Theorem 4.2.3(c), for any subspace U of the domain Rm , we have

that f [U ] is a
n
subspace of the codomain R , and by Corollary 4.2.9, dim f [U ] ≤ dim(U ). This
12
So, f ′ : U ′ → V is given by f ′ (u) = f (u) for all u ∈ U .

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 286

implies that f maps {0} onto {0}, maps any line through the origin onto either a
line through the origin or {0}, maps planes through the origin onto either planes
through the origin or lines through the origin or {0}. Similar remarks apply to
higher-dimensional generalizations of subspaces of Rm and Rn . (Compare these
remarks to the discussion in subsection 1.10.2.)

Linear functions between vector spaces of the same finite dimension. By the
Invertible Matrix Theorem (see subsection 3.3.7), for a linear function f : Fn → Fn
(where F is a field), the following are equivalent:

ˆ f is one-to-one;

ˆ f is onto;

ˆ f is an isomorphism.

Here, we assumed that the domain and the codomain of f are the same (namely, Fn ).
Using Theorem 4.2.4 (which states that a linear function is one-to-one if and only if
its kernel is {0}) and the rank-nullity theorem for linear functions, we can generalize
this to linear functions between two vector spaces of the same finite dimension.

Corollary 4.2.10. Let U and V be finite-dimensional vector spaces over a field

F, and assume that dim(U ) = dim(V ). Let f : U → V be a linear function. Then
the following are equivalent:

(i) f is one-to-one;

(ii) f is onto;

(iii) f is a bijection (and therefore an isomorphism).

Warning: Corollary 4.2.10 only works if U and V (the domain and codomain of our
linear function f ) are of the same finite dimension. Do not attempt to apply the
corollary to linear functions between infinite-dimensional vector spaces, or between
vector spaces of different dimension.

Proof of Corollary 4.2.10. By definition, (i) and (ii) together are equivalent to (iii).
So, it suffices to prove that (i) and (ii) are equivalent. By Theorem 4.2.4, we have
that f is one-to-one if and only if Ker(f ) = {0}, and by the rank-nullity theorem for
linear functions, we have that


rank(f ) + dim Ker(f ) = dim(U ).

We now have the following sequence of equivalent statements:

f is one-to-one ⇐⇒ Ker(f ) = {0} by Theorem 4.2.4

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 287

⇐⇒ dim Ker(f ) = 0

by the rank-nullity
⇐⇒ rank(f ) = dim(U )
theorem

by the definition
⇐⇒ dim Im(f ) = dim(U )
of rank(f )

because
⇐⇒ dim Im(f ) = dim(V )
dim(U ) = dim(V )

by Theorem 3.2.21, since

⇐⇒ Im(f ) = V
V is finite-dimensional

⇐⇒ f is onto V .

So, (i) and (ii) are equivalent. This completes the argument.

4.2.3 The effect of a linear function on linearly independent and

spanning sets
Theorem 4.2.11. Let U and V be vector spaces over a field F, and let f : U → V
be a linear function. Let u1 , . . . , uk ∈ U , and set U ′ := Span(u1 , . . . , uk ). Then all
the following hold:

(a) U ′ is a subspace of U , and f [U ′ ] is a subspace of V ;

(b) f [U ′ ] = f Span(u
 

1 , . . . , uk ) = Span f (u1 ), . . . , f (uk ) , i.e. vectors f (u1 ), . . . , f (uk )
span f [U ′ ] = f Span(u1 , . . . , uk ) ;
 

(c) dim f [U ′ ] ≤ dim(U ′ ) ≤ k.

uk
U′ f [U ′]
f (uk )

u2 f (u1)
u1

U V

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 288

Proof. The fact that U ′ is a subspace of U follows immediately from Theorem 3.1.11,
and the fact that f [U ′ ] is a subspace of V follows from Theorem 4.2.3(a). This
proves (a).
For (b), we have the following:


Span f (u1 ), . . . , f (uk ) = α1 f (u1 ) + · · · + αk f (uk ) | α1 , . . . , αk ∈ F

(∗)
n
o
= f α1 u1 + · · · + αk uk | α1 , . . . , αk ∈ F

(∗∗) 
= f (u) | u ∈ Span(u1 , . . . , uk )
 
= f Span(u1 , . . . , uk )

= f [U ′ ],

where (*) follows from the linearity of the f (and more precisely, from Proposi-
tion
 4.1.5), and (**) follows from the fact that, by definition, Span(u1 , . . . , uk ) =
α1 u1 + · · · + αk uk | α1 , . . . , αk ∈ F .
It remains to prove (c). By hypothesis, {u1 , . . . , uk } is a spanning set of U ′ of
size k. So, by Theorem 3.2.14, some subset of that spanning set, say {ui1 , . . . , uim }
(with 1 ≤ i1 < · · · < im ≤ k) is a basis of U ′ . So, dim(U ′ ) = m ≤ k. But now
{ui1 , . . . , uim } is a spanning set of U ′ . So, by part (b) applied to the set {ui1 , . . . , uim }
(rather than to {u1 , . . . , uk }), we get that {f (ui1 ), . . . , f (uim )} is a spanning set of
f [U ′ ]. We now apply Theorem 3.2.14 again, and we deduce that some
subset of
′ ′
{f (ui1 ), . . . , f (uim )} is a basis of f [U ], and consequently, dim f [U ] ≤ m. This
proves (c).

As an easy corollary of Theorem 4.2.11 for the case when the vectors u1 , . . . , uk
span the domain U , we obtain the following.

Corollary 4.2.12. Let U and V be vector spaces over a field F, let f : U → V

be a linear function, and let {u1 , . . . , uk } 
be a spanning set of U . Then Im(f ) =



Span f (u1 ), . . . , f (uk ) and rank(f ) = dim Span f (u1 ), . . . , f (uk ) ≤ k.

Proof. By hypothesis, U = Span(u1 , . . . , uk ).
So, by Theorem 4.2.11(b), we have

that Im(f ) = f [U ] = Span
f (u1 ), . . . , f
(uk ) , and by Theorem 4.2.11(c), we have
that rank(f ) = dim Im(f ) = dim f [U ] ≤ k.13

13



Here, the fact that rank(f

) = dim Im(f ) = dim f [U ] follows from the appropriate definitions.
The fact that dim f [U ] ≤ k follows from Theorem 4.2.11(c).

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 289

Theorem 4.2.13. Let U and V be vector spaces over a field F, let f : U → V be a

linear function, and let u1 , . . . , uk ∈ U . Then all the following hold:
(a) if f is one-to-one and vectors u1 , . . . , uk are linearly independent in U , then
vectors f (u1 ), . . . , f (uk ) are linearly independent in V ;

(b) if vectors f (u1 ), . . . , f (uk ) are linearly independent in V , then vectors u1 , . . . , uk

are linearly independent in U ;

(c) if f is onto and vectors u1 , . . . , uk span U , then vectors f (u1 ), . . . , f (uk ) span V ;

(d) if f is one-to-one and vectors f (u1 ), . . . , f (uk ) span V , then vectors u1 , . . . , uk

span U .
Remark: Schematically (and informally), Theorem 4.2.13 can be summarized as
shown in the diagram below.
linear
f : U −→ V

if f is 1-1
u1 , . . . , uk are =⇒ f (u1 ), . . . , f (uk ) are
(a)-(b)
linearly independent ⇐= linearly independent
always

if f is onto
=⇒
(c)-(d) u1 , . . . , uk span U f (u1 ), . . . , f (uk ) span V
⇐=
if f is 1-1

Proof of Theorem 4.2.13. Part (c) essentially follows from Corollary 4.2.12, so let us
prove it first. Assume that f is onto and that vectors u1 , . . . , uk span U . But then
(∗) (∗∗)

V = Im(f ) = Span f (u1 ), . . . , f (uk ) ,

where (*) follows from the fact that f is onto, and (**) follows from Corollary 4.2.12.
So, vectors f (u1 ), . . . , f (uk ) span V . This proves (c).
Next, we prove (d). Assume that f is one-to-one and that vectors f (u1 ), . . . , f (uk )
span V ; we must show that vectors u1 , . . . , uk span U , i.e. that any vector in U
can be written as a linear combination of the vectors u1 , . . . , uk . Fix u ∈ U . Since
f (u) ∈ V and since vectors f (u1 ), . . . , f (uk ) span V , we know that there exist scalars
α1 , . . . , αk ∈ F such that f (u) = α1 f (u1 ) + · · · + αk f (uk ). But now
(∗)
f (u) = α1 f (u1 ) + · · · + αk f (uk ) = f (α1 u1 + · · · + αk uk ),

where (*) follows from the fact that f is linear (and more precisely, from Proposi-
tion 4.1.5). Since f is one-to-one, we deduce that u = α1 u1 + · · · + αk uk , and so u
is indeed a linear combination of the vectors u1 , . . . , uk . This proves (d).

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 290

We now prove (a). Assume that f is one-to-one and that vectors u1 , . . . , uk are
linearly independent in U . We must show that vectors f (u1 ), . . . , f (uk ) are linearly
independent in V . Fix scalars α1 , . . . , αk ∈ F such that

α1 f (u1 ) + · · · + αk f (uk ) = 0.

We must show that α1 = · · · = αk = 0. First, since f is linear (and more precisely,

by Proposition 4.1.5), we have that α1 f (u1 ) + · · · + αk f (uk ) = f (α1 u1 + · · · + αk uk ).
So, f (α1 u1 + · · · + αk uk ) = 0, and it follows that α1 u1 + · · · + αk uk ∈ Ker(f ). But
since f is one-to-one, Theorem 4.2.4 guarantees that Ker(f ) = {0}, and we deduce
that α1 u1 + · · · + αk uk = 0. Since vectors u1 , . . . , uk are linearly independent, we
deduce that α1 = · · · = αk = 0. This proves (a).
It remains to prove (b). Assume that vectors f (u1 ), . . . , f (uk ) are linearly
independent in V . We must show that vectors u1 , . . . , uk are linearly independent in
U . Fix scalars α1 , . . . , αk ∈ F such that

α1 u1 + · · · + αk uk = 0.

We must show that α1 = · · · = αk = 0. We have the following:

(∗) (∗∗) (∗∗∗)
α1 f (u1 ) + · · · + αk f (uk ) = α1 f (u1 + · · · + αk uk ) = f (0) = 0,

where (*) follows from the fact that f is linear (and more precisely, from Propo-
sition 4.1.5), (**) follows from the fact that α1 u1 + · · · + αk uk = 0, and (***)
follows from Proposition 4.1.6. But now since vectors f (u1 ), . . . , f (uk ) are linearly
independent, we now have that α1 = · · · = αk = 0. This proves (b).

Dimension considerations. As we know, for any function f : A → B, where A

and B are finite sets, the following hold:

ˆ if f is one-to-one, then |A| ≤ |B|;

ˆ if f is onto, then |A| ≥ |B|;

ˆ if f is a bijection, then |A| = |B|.

(Actually, the above is true even if we allow A and B to be infinite, but to make
sense of the statement, we would need infinite cardinals. We omit the details.) In the
case of linear functions, Theorem 4.2.14 (below) gives us a very similar statement,
only involving dimension (rather than cardinality) of the domain and codomain. We
note that Theorem 4.2.14 is an easy corollary of Theorem 4.2.13. We also note that
the vector spaces U and V from the statement of Theorem 4.2.14 may possibly be
infinite-dimensional.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 291

Theorem 4.2.14. Let U and V be vector spaces over a field F, and let f : U → V
be a linear function. Then all the following hold:

(a) if f is one-to-one, then dim(U ) ≤ dim(V );

(b) if f is onto, then dim(U ) ≥ dim(V );

(c) if f is an isomorphism, then dim(U ) = dim(V ).

Proof. Obviously, (a) and (b) together imply (c). So, it is enough to prove (a)
and (b).
(a) We prove the contrapositive: we assume that dim(U ) > dim(V ) (and in
particular, dim(V ) is finite), and we prove that f is not one-to-one. Set n := dim(V ).
Since dim(U ) > dim(V ), we know that U has a linearly independent set of size
greater than n. (Indeed, if U is finite-dimensional, then any one of its bases is a
linearly independent set of size dim(U ) > n, and if U is infinite-dimensional, then
Proposition 3.2.18 guarantees that U has linearly independent sets of any finite size.)
So, fix a linearly independent set {u1 , . . . , uk } of U , with k > n. Since dim(V ) = n,
Theorem 3.2.17(a) guarantees that the set f (u1 ), . . . , f (uk ) is linearly dependent.
But now Theorem 4.2.13(a) guarantees that f is not one-to-one.
(b) Assume that f is onto; we must show that dim(U ) ≥ dim(V ). We may
assume that n := dim(U ) is finite, for otherwise, we are done. We must show that
dim(V ) ≤ n. Fix any basis {u1 , . . . , un } of U . In particular, vectors u1 , . . . , un span
U , and so since f is onto, Theorem 4.2.13(c) guaranteesthat vectors f (u1 ), . . . , f (un )
span V . But then by Theorem 3.2.14, some subset of f (u1 ), . . . , f (un ) is a basis
of V , and it follows that dim(V ) ≤ n.

4.2.4 Computing bases of the images and preimages of subspaces

under linear functions
In this subsection, we consider linear functions f : Fm → Fn (where F is a field). By
Theorem 4.2.3, for every subspace U of the domain Fm , f [U ] is a subspace of the
codomain Fn , and for every subspace V of the codomain Fn , f −1 [V ] is a subspace of
the domain Fm . We would like to compute bases of f [U ] and f −1 [V ].

Computing a basis of the image of a subspace of the domain of a linear

function. Proposition 4.2.15 (below) is an easy (and computationally useful)
corollary of Theorem 4.2.11.

Proposition 4.2.15. Let F be a field, let f : Fm → Fn be a linear function, let

A ∈ Fn×m be the standard matrix of f , let u1 , . . . , uk ∈ Fm (k ≥ 1), and set
U := Span(u1 , . . . , uk ). Then
  
f [U ] = Col A u1 . . . uk ,

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 292

 
and moreover, the pivot columns of the matrix A u1 . . . uk form a basis of
f [U ].

Proof. First, we compute:

 
f [U ] = f Span(u1 , . . . , uk )

(∗)

= Span f (u1 ), . . . , f (uk )

(∗∗)
 
= Col f (u1 ) . . . f (uk )

(∗∗∗)
 
= Col Au1 . . . Auk

(∗∗∗∗)
  
= Col A u1 . . . uk ,

where (*) follows from Theorem 4.2.11(b), (**) follows from the definition of the
column space, and (***) follows from the fact that A is the standard matrix of f ,
and (****) follows from the definition of matrix multiplication. By Theorem 3.3.4,
the pivot columns of a matrix form a basis of the column space of that matrix, and
the result follows.

Example 4.2.16. Let f : Z52 → Z42 be the linear function whose standard matrix is
 
1 1 0 0 1
 0 1 1 0 1 
A =   1 0 1 0 0 ,


1 1 0 1 1

and consider the vectors

       
1 1 0 1
 1   0   1   0 
       
 1
u1 =  1 0 0
, u2 =  , u3 =  , u4 =  
      
 1   0   1   0 
1 1 0 1

in Z52 . Set U := Span(u1 , u2 , u3 , u4 ). Find a basis of f [U ].

 
Solution. Our goal is to find the pivot columns of the matrix A u1 u2 u3 u4 ,
since by Proposition 4.2.15, those columns form a basis of f [U ]. First, by multiplying
matrices, we obtain

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 293

 
  1 1 0 1
1 1 0 0 1 
 0 1 1 0 1 0 
  1 0 1   
A u1 u2 u3 u4 = 
 1 0
 1 1 0 0 
1 0 0 
 
 1 0 1 0 
1 1 0 1 1
1 1 0 1
 
1 0 1 0
 1 0 1 1 
= 
 0
.
0 0 1 
0 0 0 0

By row reducing, we obtain

 
1 0 1 0
    0 0 0 1 
RREF A u1 u2 u3 u4 = 
 0 0
.
0 0 
0 0 0 0
 
As we can see, the pivot columns of A u1 u2 u3 u4 are its first and fourth
column. Therefore,    
1 0
n 1   1 o
 , 
 0   1 
0 0
is a basis of f [U ].

Example 4.2.17. Let f : R4 → R3 be the linear function whose standard matrix is

 
1 2 0 3
A =  2 4 1 4 ,
1 2 1 1

and consider the vectors

     
−5 −3 −1
 1   0   −1 
u1 = 
 2 ,
 u2 = 
 2 ,
 u3 = 
 2 


1 1 1

in R4 . Set U := Span(u1 , u2 , u3 ). Find a basis of f [U ].

 
Solution. Our goal is to find the pivot columns of the matrix A u1 u2 u3 ,
since by Proposition 4.2.15, those columns form a basis of f [U ]. We compute:

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 294



 −5 −3 −1
  
1 2 0 3  0 0 0
 
 2 4 1 4  1 0 −1 
A u1 u2 u3 =  2
 =  0 0 0 .
2 2 
1 2 1 1 0 0 0
1 1 1
 
So, A u1 u2 u3 is a zero matrix, and consequently, it has no pivot columns. It
follows that ∅ is a basis of f [U ].

Remark: So, we effectively got that f [U ] = {0}. The only basis of the trivial vector
space {0} is the empty basis, i.e. ∅.

Computing a basis of the preimage of a subspace of the codomain of a

linear function.

Proposition 4.2.18. Let F be a field, let f : Fm → Fn be a linear function, let

A ∈ Fn×m be the standard matrix of f , let v1 , . . . , vk ∈ Fn (k ≥ 1), and set
V := Span(v1 , . . . , vk ) Then
 
−1
n
m k
  x o
f [V ] = x ∈ F | ∃y ∈ F s.t. A v1 . . . vk =0
y
 
n
m k x   o
= x ∈ F | ∃y ∈ F s.t. ∈ Nul A v1 . . . vk .
y
T
in Fm ,
  
Proof. Set A = a1 . . . am . Then for all vectors x = x1 . . . xm
we have the following sequence of equivalent statements:

x ∈ f −1 [V ]

⇐⇒ f (x) ∈ Span(v1 , . . . , vk )
| {z }
=V

(∗)
⇐⇒ Ax ∈ Span(v1 , . . . , vk )

⇐⇒ x a + · · · + xm am ∈ Span(v1 , . . . , vk )
|1 1 {z }
=Ax

(∗∗)
⇐⇒ ∃α1 , . . . , αk ∈ F s.t. x1 a1 + · · · + xm am = α1 v1 + · · · + αk vk

⇐⇒ ∃α1 , . . . , αk ∈ F s.t. x1 a1 + · · · + xm am − α1 v1 − · · · − αk vk = 0

(∗∗∗)
⇐⇒ ∃y1 , . . . , yk ∈ F s.t. x1 a1 + · · · + xm am + y1 v1 + · · · + yk vk = 0

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 295

 
x1
 .. 
 . 
 
   xm 
⇐⇒ ∃y1 , . . . , yk ∈ F s.t. a1 . . . am v1 . . . vk  =0
 y 1


 .. 
 . 
yk
 
x
∃y ∈ Fk s.t.
 
⇐⇒ A v1 . . . vk =0
y
 
(∗∗∗∗) x  
⇐⇒ ∃y ∈ Fk s.t. ∈ Nul A v1 . . . vk ,
y

where (*) follows from the fact that A is the standard matrix of f , (**) follows from
the definition of span, (***) follows by performing the substitution yi := −αi for all
i ∈ {1, . . . , k}, and (****) follows from the definition of the null space. The result is
now immediate.

Example 4.2.19. Consider the linear function f : R4 → R5 whose standard matrix

is  
1 0 0 0
 0 −2 −4 0 
 
A =   −2 −3 −6 1 ,

 4 0 0 0 
2 −1 −2 0
and consider the following vectors in R5 :
T
ˆ v1 = −1

6 9 −4 1 ;
T
ˆ v2 =

2 2 −2 8 5 ;
T
ˆ v3 =

0 0 0 −1 0 ;
T
ˆ v4 =

0 −2 −3 −1 −1 ;
T
ˆ v5 =

0 −1 −2 1 0 ;
T
ˆ v6 =

−3 −1 2 −11 −6 .

Set V := Span(v1 , . . . , v6 ). Find a basis of f −1 [V ].

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 296

Solution. We apply Proposition 4.2.18. We first form the matrix

 
C := A v1 v2 v 3 v 4 v5 v6
 
1 0 0 0 −1 2 0 0 0 −3

 0 −2 −4 0 6 2 0 −2 −1 −1 

= 
 −2 −3 −6 1 9 −2 0 −3 −2 2 ,

 4 0 0 0 −4 8 −1 −1 1 −11 
2 −1 −2 0 1 5 0 −1 0 −6

and we find the general solution of the matrix-vector equation

 
  x
A v1 v2 v 3 v 4 v5 v6 = 0,
| {z } y
=C

where the vector x has four entries (because A has four columns) and the vector y
has six entries (because we have six vectors v1 , . . . , v6 ). By row reducing, we obtain
 
1 0 0 0 −1 2 0 0 0 −3
 0 1 2 0 −3 −1 0 1 0 0 
 
RREF(C) =  0 0 0 1 −2 −1 0 0 0 −2 

.
 0 0 0 0 0 0 1 1 0 0 
0 0 0 0 0 0 0 0 1 1

So, the general solution of our matrix-vector equation is

q − 2r + 3t
 
 −2p + 3q + r − s 
 

 p 

 

 2q + r + 2t 

x  q 
=  , where p, q, r, s, t ∈ R.
y 
 r 


 −s 


 s 

 −t 
t

But as per Proposition 4.2.18, we only need x! So, we simply ignore the part below
the horizontal dotted line, and we obtain
 
q − 2r + 3t
 −2p + 3q + r − s 
x =  , where p, q, r, s, t ∈ R.
 p 
2q + r + 2t

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 297

By separating parameters, we obtain

      
  
0 1 −2 0 3
 −2   3   1   −1   0 
x = p  1 +q 0 +r 0
   
  0  + t 0
 + s   ,
 where p, q, r, s, t ∈ R.
0 2 1 0 2

In view of Proposition 4.2.18, we now have that

         
0 1 −2 0 3
n  −2  3 1 −1  0
o
f −1 [V ] =
       
p  | p, q, r, s, t ∈ R
 1  + q 0  + r 0  + s 0  + t 0
      

0 2 1 0 2
         
0 1 −2 0 3
  −2   3   1   −1   0 
= Span  1 , 0
 
  0 , 0 , 0
,     

0 2 1 0 2
 
0 1 −2 0 3
  −2 3 1 −1 0 
= Col  1
 .
0 0 0 0 
0 2 1 0 2
| {z }
=:B

We note that the five vectors that we obtained in the second-to-last line above are
not necessarily linearly independent,14 and so to find an actual basis of f −1 [V ], we
row reduce the matrix B and use Theorem 3.3.4. Indeed, Theorem 3.3.4 guarantees
that the pivot columns of B form a basis of Col(B) = f −1 [V ]. By row reducing, we
obtain  
1 0 0 0 0
 0 1 0 0 7/5 
RREF(B) =   0 0 1 0 −4/5  .


0 0 0 1 17/5
Thus, the pivot columns of B are its leftmost four columns, and those four columns
14
In fact, we can immediately see that they are not linearly independent: no five vectors in R4 are
linearly independent (by Theorem 3.2.17(a)). More generally, though, the reason our computation
does not necessarily yield linearly independent vectors is because we “cut off” the entries below the
vertical dotted line.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 298

form a basis of f −1 [V ]. So, our final answer is that

       
0 1 −2 0
n  −2   3   1   −1  o
 1 , 0 , 0 , 0 
       

0 2 1 0

is a basis of f −1 [V ].

Example 4.2.20. Consider the linear function f : Z53 → Z53 whose standard matrix
is  
1 1 0 1 1
 2 1 2 0 2 
 
A =   1 1 0 0 2 ,
 2 1 2 1 1 
1 1 0 0 2
and consider the following vectors in Z53 :
         
1 0 0 2 1

 1 


 1 


 1 


 1 


 0 

v1 = 
 1 ,
 v2 = 
 0 ,
 v3 = 
 1 ,
 v4 = 
 1 ,
 v5 = 
 2 .

 0   2   0   0   1 
0 1 0 0 0

Set V := Span(v1 , v2 , v3 , v4 , v5 ). Find a basis of f −1 [V ].

Solution. We apply Proposition 4.2.18. We first form the matrix

 
C := A v1 v2 v3 v4 v5
 
1 1 0 1 1 1 0 0 2 1

 2 1 2 0 2 1 1 1 1 0 

= 
 1 1 0 0 2 1 0 1 1 2 ,

 2 1 2 1 1 0 2 0 0 1 
1 1 0 0 2 0 1 0 0 0

and we find the general solution of the matrix-vector equation

 
  x
A v1 v 2 v 3 v4 v5 = 0,
| {z } y
=C

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 299

where x has five entries (because A has five columns) and y also has five entries
(because we have five vectors v1 , . . . , v5 ). By row reducing, we obtain
 
1 0 2 0 0 0 1 0 0 1
 0 1 1 0 2 0 0 0 0 2 
 
RREF(C) =   0 0 0 1 2 0 0 0 0 0 .

 0 0 0 0 0 1 2 0 2 1 
0 0 0 0 0 0 0 1 2 1

So, the general solution of our matrix-vector equation is

p + 2r + 2t
 
 2p + q + t 
 

 p 

 

 q 

x  q 
=  , where p, q, r, s, t ∈ Z3 .
y 
 r + s + 2t 


 r 


 s + 2t 

 s 
t

For x, we get
 
p + 2r + 2t

 2p + q + t 

x = 
 p ,
 where p, q, r, t ∈ Z3 .
 q 
q

Remark: The parameter s does not appear in the vector x, and so from this point
on, that parameter plays no role in our solution.
By separating parameters, we get
       
1 0 2 2
 2   1   0   1 
       
x = p  1 +q 0 +r
    0  + t
  0 ,
 where p, q, r, t ∈ Z3 .
 0   1   0   0 
0 1 0 0

In view of Proposition 4.2.18, we now have that

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 300

       
1 0 2 2
n  2   1   0   1  o
f −1 [V
       
] = p 1 +q 0 +r 0  + t 0  | p, q, r, t ∈ Z3
       
 0   1   0   0 
0 1 0 0
       
1 0 2 2

 2  
  1  
  0  
  1 

= Span 
 1 ,
  0 ,
  0 ,
  0 

 0   1   0   0 
0 1 0 0
 
1 0 2 2

 2 1 0 1 

= Col 
 1 0 0 0  .

 0 1 0 0 
0 1 0 0
| {z }
=:B

By row reducing, we get

 
1 0 0 0

 0 1 0 0 

RREF(B) = 
 0 0 1 0 .

 0 0 0 1 
0 0 0 0
So, all four columns of B are pivot columns, and by Theorem 3.3.4, the pivot columns
of B form a basis of Col(B). So,
       
1 0 2 2
n       1 
 2   1   0  
o
 1 , 0 , 0 , 0 
       
 0   1   0   0 
0 1 0 0
is a basis of f −1 [V ].

Example 4.2.21. Consider the linear function f : Z32 → Z52 whose standard matrix
is  
1 1 0
 1 1 0 
 
A =   1 1 1 ,

 0 1 1 
0 1 1

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 301

and consider the following vectors in Z52 :

     
1 0 1

 0 


 1 


 1 

v1 = 
 0 ,
 v2 = 
 0 ,
 v3 = 
 0 .

 0   1   1 
1 1 0

Set V := Span(v1 , v2 , v3 ). Find a basis of f −1 [V ].

Solution. We apply Proposition 4.2.18. We first form the matrix

 
1 1 0 1 0 1
 1 1 0 0 1 1 
   
C := A v1 v2 v3 =   1 1 1 0 0 0 ,

 0 1 1 0 1 1 
0 1 1 1 1 0

and we find the general solution of the matrix-vector equation

 
  x
A v1 v 2 v3 v4 = 0,
| {z } y
=C

where x has three entries (because A has three columns) and y also has three entries
(because we have three vectors v1 , v2 , v3 ). By row reducing, we obtain
 
1 0 0 0 0 0
 0 1 0 0 0 0 
 
RREF(C) =   0 0 1 0 0 0 .

 0 0 0 1 0 1 
0 0 0 0 1 1

So, the general solution of our matrix-vector equation is

 
0
 0 
   
x  0 
=  t ,
 where t ∈ Z2 ,
y  
 t 
t

and in particular, x = 0. In view of Proposition 4.2.18, we now have that f −1 [V ] =

{0}, and consequently, ∅ is the (unique) basis of f −1 [V ].

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 302

4.3 Linear functions and bases

We now consider a particularly important type of linear function, which is in fact
an isomorphism. Recall that a non-trivial vector space is one that contains at least
one non-zero vector, or equivalently, one that has strictly positive (possibly infinite)
dimension. Now, suppose that V is a non-trivial, finite-dimensional vector space over
a field F, and that B = {v1 , . . . , vn } is a basis of V . By Theorem 3.2.7, every vector
of V can be written as linear combination of the vectors v1 , . . . , vn in a unique way,
that is, for all vectors v ∈ V , there exist unique scalars α1 , . . . , αn ∈ F such that

v := α1 v1 + · · · + αn vn ,

and the coordinate vector of v with respect to the basis B is defined to be

 
α1
v B :=  ...  .
   

αn

As our next proposition shows, · B : V → Fn is an isomorphism. In fact, it is the

 

single most important isomorphism that we will encounter in these lecture notes. It
̸ 0)
essentially allows us to “translate” vectors of an n-dimensional vector space (n =
into vectors in Fn . We will see some numerical examples that rely on coordinate
vectors in subsection 4.4.3, after we have developed some more theory.
Proposition 4.3.1. Let V be a non-trivial, finite-dimensional   vector space over
a field F, and let B = {v1 , . . . , vn } is a basis of V . Then · B : V → Fn is an
isomorphism.
 
Proof. We start by proving that · B is linear.
       
1. Fix x, y ∈ V . We must show that x + y B = x B + y B . Set x B =
 T    T
α1 . . . αn and y B = β1 . . . βn . Then x = α1 v1 + · · · + αn vn and
y = β1 v1 + · · · + βn vn ; consequently,

x + y = (α1 + β1 )v1 + · · · + (αn + βn )vn ,

   T
and so x+y B
= α1 + β 1 . . . αn + βn . We now have that
     
α1 + β 1 α1 β1

x+y

=  ..  ..   ..     
x B+ y B.
 =  . + .  =
 
B .
αn + βn αn βn

   T
2. Fix x ∈ V and α ∈ F. Set x B = α1 . . . α n . Then x =
 
α1 v1 + · · · + αn vn ; consequently, αx = αα1 v1 + · · · + ααn vn , and so αx B =

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 303

 T
αα1 . . . ααn . We now have that
   
αα1 α1
αx B =  ...  = α  ..  = α  x  .
    
.  B
ααn αn

By 1. and 2., f is linear.

 
It remains to show that · B is a bijection, i.e. that it is one-to-one and
onto Fn . Since V and Fn are both n dimensional, Corollary 4.2.10 guarantees
that f is one-to-one if and only if f is onto Fn . So, it is enough to show that
T
f is onto Fn . Fix α1 . . . αn ∈ Fn . Set v := α1 v1 + · · · + αn vn . Then

T
. So, · B is onto Fn . This completes the argument.
    
v B = α1 . . . α n

Theorem 4.3.2 (below) is one of the main reasons we care about bases. It
essentially states that, given vector spaces U and V over a field F, where U is finite-
dimensional with a basis B, we can uniquely determine a linear function f : U → V
by specifying what the basis vectors from B get mapped to (and we get to determine
arbitrarily what vectors of V those basis vectors get mapped to). We note that
Theorem 4.3.2 can, in fact, be generalized to infinite-dimensional domains U , but
this would require working with infinite bases, and we omit the details.
Theorem 4.3.2. Let U and V be vector spaces over a field F, and assume that U
is finite-dimensional. Let B = {u1 , . . . , un } be a basis of U , and let v1 , . . . , vn ∈
V .15 Then there exists a unique linear function f : U → V such that f (u1 ) =
v1 , . . . , f (un ) = vn . Moreover, if the vector space U is non-trivial (i.e. n ̸= 0), then
this unique linear function f : U → V satisfies the following: for all u ∈ U , we have
that
f (u) = α1 v1 + · · · + αn vn ,
T
. On the other hand, if U is trivial (i.e. U = {0}),16
  
where u B = α1 . . . αn
then f : U → V is given by f (0) = 0.
Proof. Suppose first that the vector space U is trivial, i.e. n = 0 and U = {0}.
Then the function f : U → V given by f (0) = 0 is obviously linear, and moreover,
it vacuously satisfies f (u1 ) = v1 , . . . , f (un ) = vn (because n = 0, and so both
u1 , . . . , un and v1 , . . . , vn are empty lists of vectors). The uniqueness of f follows
from Proposition 4.1.6.
̸ 0.
From now on, we assume that the vector space U is non-trivial, i.e. that n =
We must prove the existence and the uniqueness of the linear function f satisfying
the required properties.
15
Here, v1 , . . . , vn are arbitrary vectors in V . They are not necessarily pairwise distinct.
16
Note that in this case, we have that n = 0 and B = ∅.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 304

Existence. Let f : U → V be defined as in the statement of the theorem, i.e. for all
u ∈ U , we set
f (u) = α1 v1 + · · · + αn vn ,
   T
where u B = α1 . . . αn . Note that this means that for all α1 , . . . , αn ∈ F,
we have that

f (α1 u1 + · · · + αn un ) = α1 v1 + · · · + αn vn .

Let us show that f is linear and satisfies f (u1 ) = v1 , . . . , f (un ) = vn . For the
latter, we note that for all i ∈ {1, . . . , n}, we have that

f (ui ) = f (0u1 + · · · + 0ui−1 + 1ui + 0ui+1 + · · · + 0un )

= 0v1 + · · · + 0vi−1 + 1vi + 0vi+1 + · · · + 0vn

= vi .

This proves that f (u1 ) = v1 , . . . , f (un ) = vn .

Let us now show that f is linear. We verify that f satisfies the two axioms from
the definition of a linear function.  
1. Fix x, y ∈ U . We must show that f (x + y) = f (x) + f (y). Set x B =
 T    T  
α1 . . . αn and y B = β1 . . . βn . We then have that x + y B =
 T 17
α1 + β1 . . . αn + βn , and we see that

(∗)
f (x + y) = (α1 + β1 )v1 + · · · + (αn + βn )vn

= (α1 v1 + · · · + αn vn ) + (β1 v1 + · · · + βn vn )

(∗∗)
= f (x) + f (y),

where both (*) and (**) follow from the construction of f .  

2. Fix u ∈ U and α ∈ F. We must show that f (αu) = αf (u). Set u B =
17
   T  T
Indeed, since x B = α1 . . . αn and y = β1 . . . βn , we have that x =
α1 u1 + · · · + αn un and y = β1 u1 + · · · + βn un , and consequently,

x+y = (α1 u1 + · · · + αn un ) + (β1 u1 + · · · + βn un )

= (α1 + β1 )u1 + · · · + (αn + βn )un ,

   T
and so x+y B
= α1 + β1 ... αn + βn .

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 305

T T
,18 and we see that
   
α1 . . . αn . Then αu B
= αα1 . . . ααn

(∗) (∗∗)
f (αu) = (αα1 )v1 + · · · + (ααn )vn = α(α1 v1 + · · · + αn vn ) = αf (u),

where both (*) and (**) follow from the construction of f .

By 1. and 2., we see that f is linear. This completes the proof of existence.
Uniqueness. Let f1 , f2 : U → V be linear functions that satisfy f1 (u1 ) =
v1 , . . . , f1 (un ) = vn and f2 (u1 ) = v1 , . . . , f2 (un ) = vn . We must show that f1 = f2 .
   T
Fix u ∈ U . We must show that f1 (u) = f2 (u). Set u B = α1 . . . αn .
Then
f1 (u) = f1 (α1 u1 + · · · + αn un )

by the linearity of f1
= α1 f1 (u1 ) + · · · + αn f1 (un ) (and more precisely,
by Proposition 4.1.5)

because
= α 1 v1 + · · · + α n vn
f1 (u1 ) = v1 , . . . , f1 (un ) = vn

because
= α1 f2 (u1 ) + · · · + αn f2 (un )
f2 (u1 ) = v1 , . . . , f2 (un ) = vn

by the linearity of f2
= f2 (α1 u1 + · · · + αn un ) (and more precisely,
by Proposition 4.1.5)

= f2 (u).

Thus, f1 = f2 . This proves uniqueness.

Corollary 4.3.3. Let U and V be vector spaces over a field F, and assume that U
is finite-dimensional. Let {u1 , . . . , uk } be a linearly independent set of vectors in U ,
and let v1 , . . . , vk ∈ V .19 Then there exists a linear function f : U → V such that
f (u1 ) = v1 , . . . , f (uk ) = vk . Moreover, if V is non-trivial, then this linear function
f is unique if and only if {u1 , . . . , uk } is a basis of U .
Remark: If V is trivial (i.e. V = {0}, and consequently v1 = · · · = vk = 0), then
there exists exactly one function from U to V , this function maps all elements of U
to 0, and obviously, it is linear.
18
   T
Indeed, since u B
= α1
αn ..., we have that u = α1 u1 + · · · + αn un . Consequently,
   T
αu = (αα1 )u1 + · · · + (ααn )un , and so αu B = αα1 . . . ααn .
19
Here, v1 , . . . , vk are arbitrary vectors in V . They are not necessarily pairwise distinct.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 306

Proof of Corollary 4.3.3 (outline). Using Theorem 3.2.19, we extend {u1 , . . . , uk } to

a basis of U , and then we apply Theorem 4.3.2. The details are left as an exercise.

4.4 Isomorphisms
Recall that, for vector spaces U and V over a field F, a function f : U → V is an
isomorphism if it is linear and a bijection.
Vector spaces U and V (over the same field F) are isomorphic, and we write
U∼ = V , if there exits an isomorphism f : U → V .

4.4.1 Basic properties of isomorphisms

Proposition 4.4.1 (below) generalizes Proposition 1.10.20 to isomorphisms between ar-
bitrary vector spaces. The proof is essentially identical to that of Proposition 1.10.20.

Proposition 4.4.1. Let U and V be vector spaces over a field F, and let f : U → V
be an isomorphism. Then f −1 : V → U is also an isomorphism.
f
U V
f −1

Proof. Since f : U → V is an isomorphism, it is, in particular, a bijection; conse-

quently, f has an inverse f −1 : V → U , which is also a bijection. So, to show that
f −1 is an isomorphism, it suffices to show that f −1 is linear.
First, fix v1 , v2 ∈ V . We must show that f −1 (v1 + v2 ) = f −1 (v1 ) + f −1 (v2 ). Set
u1 := f −1 (v1 ) and u2 := f −1 (v2 ), so that f (u1 ) = v1 and f (u2 ) = v2 . Then

f −1 (v1 + v2 ) = f −1 f (u1 ) + f (u2 )

f −1 f (u1 + u2 )


= because f is linear

= (f −1 ◦ f )(u1 + u2 )

= IdU (u1 + u2 )

= u1 + u2

= f −1 (v1 ) + f −1 (v2 ).

Next, fix v ∈ V and α ∈ F. We must show that f −1 (αv) = αf −1 (v). Set

u := f −1 (v), so that f (u) = v. Then

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 307

f −1 (αv) f −1 αf (u)


=

f −1 f (αu)


= because f is linear

= (f −1 ◦ f )(αu)

= IdU (αu)

= αu

= αf −1 (v).

We have now proven that f −1 linear. This completes the argument.

Proposition 4.4.2. Let U , V , and W be vector spaces over a field F, and let
f : U → V and g : V → W be isomorphisms. Then g ◦ f : U → W is an
isomorphism.
g◦f

f g
U V W
Proof. Since f : U → V and g : V → W are linear functions (because they are
isomorphisms), Proposition 4.1.7 guarantees that their composition g ◦ f : U → W
is also linear. Since f : U → V and g : V → W are bijections, Proposition 1.10.17
guarantees that g ◦ f : U → W is also a bijection. So, g ◦ f : U → W is linear and a
bijection, i.e. it is an isomorphism.

Theorem 4.4.3. Let U , V , and W be vector spaces over a field F. Then all the
following hold:
(a) U ∼
= U;
(b) if U ∼
= V , then V ∼
= U;
(c) if U ∼
= V and V ∼
= W , then U ∼
= W.
Proof. (a) Clearly, IdU : U → U (the identity function on U ) is an isomorphism. So,
U=∼ U.

(b) Suppose that U ∼ = V . Then there exists an isomorphism f : U → V . But

then by Proposition 4.4.1, f −1 : V → U is also an isomorphism. So, V ∼ = U.
(c) Suppose that U ∼ = V and V ∼ = W . Then there exist isomorphisms f : U → V
and g : V → W . But then by Proposition 4.4.2, g ◦ f : U → W is an isomorphism.
So, U ∼
= W.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 308

The final result of this subsection (Theorem 4.4.4 below) essentially states that
isomorphisms map linearly independent sets to linearly independent set, spanning
sets to spanning sets, and bases to bases. As we shall see, it is an easy corollary of
Theorem 4.2.13.

Theorem 4.4.4. Let U and V be vector spaces over a field F, let f : U → V be an

isomorphism, and let u1 , . . . , uk ∈ U . Then all the following hold:

(a) vectors u1 , . . . , uk are linearly independent in U if and only if vectors f (u1 ), . . . , f (uk )
are linearly independent in V ;

(b) vectors u1 , . . . , uk span U if and only if vectors f (u1 ), . . . , f (uk ) span V ;


(c) {u1 , . . . , uk } is a basis of U if and only if f (u1 ), . . . , f (uk ) is a basis of V .

Proof. Since f is an isomorphism, it is, by definition, a linear function that is both

one-to-one and onto. Thus, (a) follows from Theorem 4.2.13(a-b), and (b) follows
from Theorem 4.2.13(c-d). Finally, since a basis of a vector space is simply a linearly
independent set of vectors that spans that vector space, parts (a) and (b) together
imply (c).

Proposition 4.4.5 (below) is a converse of sorts of Theorem 4.4.4(c). It essentially

states that any linear function that (injectively) maps a basis onto a basis is an
isomorphism.

Proposition 4.4.5. Let U and V be finite-dimensional vector spaces over a field

F. Assume that dim(U ) = dim(V ) =: n. Let {u1 , . . . , un } be a basis of U , and let
{v1 , . . . , vn } be a basis of V . Then there exists a unique linear function f : U → V
such that f (u1 ) = v1 , . . . , f (un ) = vn . Moreover, this linear function f is an
isomorphism.

f
un vn
... ...

u3 v3
u2 v2
u1 v1

U V
Proof. The existence and uniqueness of the linear function f follows from The-
orem 4.3.2. We need to show that the linear function f is in fact an isomor-
phism. But by hypothesis, U and V are finite-dimensional vector spaces satisfying

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 309

dim(U ) = dim(V ), and so by Corollary 4.2.10, it is enough to show that f is onto.

Fix v ∈ V . Since {v1 , . . . , vn } is a basis of V , we know that there exist scalars
α1 , . . . , αn ∈ F such that v = α1 v1 + · · · + αn vn . But now
(∗)
f (α1 u1 + · · · + αn un ) = α1 f (u1 ) + · · · + αn f (un )

= α 1 v1 + · · · + α n vn

= v,
where (*) follows from the linearity of f (and more precisely, from Proposition 4.1.5).
So, f is onto, and we are done.

4.4.2 Isomorphism and dimension

By Theorem 4.2.14(c), any two isomorphic vector spaces have the same dimension.
Theorem 4.4.6 (below) guarantees that, in the case of finite-dimensional vector
spaces, the converse is also true: any two vector spaces (over the same field) that have
the same finite dimension are isomorphic. We give two proofs (both of them quite
short) of this result. One of the proofs relies on coordinate vectors and Theorem 4.4.3,
whereas the other one relies on Proposition 4.4.5.
Theorem 4.4.6. Let U and V be finite-dimensional vector spaces over a field F.
Then U and V are isomorphic if and only if dim(U ) = dim(V ).
Warning: This theorem is only true for finite-dimensional vector spaces, and it
becomes false for infinite-dimensional ones.

Proof#1. If U and V are isomorphic, then Theorem 4.2.14(c) guarantees that

dim(U ) = dim(V ).20 Suppose, conversely, that dim(U ) = dim(V ) =: n. Fix any basis
 = {b1 , . . . , bnn} of U and any basis nC = {c1 , . . . , cn } of V . By Proposition 4.3.1,
B
· B : U → F and · C : V → F are both isomorphisms, and consequently,
U∼ = Fn and V ∼ = Fn . But now Theorem 4.4.3 guarantees that U ∼ =V.
Proof#2. If U and V are isomorphic, then Theorem 4.2.14(c) guarantees that
dim(U ) = dim(V ). Suppose, conversely, that dim(U ) = dim(V ) =: n. Fix a basis
B = {b1 , . . . , bn } of U and a basis C = {c1 , . . . , cn } of V . Then by Proposition 4.4.5,
there exists a unique linear function f : U → V such that f (b1 ) = c1 , . . . , f (bn ) =
cn , and moreover, this linear function f is an isomorphism. So, U and V are
isomorphic.

We complete this subsection with a technical proposition that slightly generalizes

Theorem 4.2.14(c).
20 ∼ V , then by definition, there exists an isomorphism f : U → V , and so by
Indeed, if U =
Theorem 4.2.14(c), we have that dim(U ) = dim(V ).

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 310

Proposition 4.4.7. Let U and V be a vector spaces over a field F, and let f : U → V
be an isomorphism, and let U ′ ⊆ U . Then U ′ is a subspace of U if and only if
V ′ := f [U ′ ] is a subspace of V . Moreover, in this case,21 all the following hold:

(a) the function f ′ : U ′ → V ′ given by f ′ (u) = f (u) for all u ∈ U ′ is an isor-

mophism;22

(b) U ′ ∼
= V ′;

(c) dim(U ′ ) = dim(V ′ ).23

U′ f V ′ = f [U ′]

U V
Proof. First of all, since f is an isomorphism (and in particular, a bijection), we
see that f −1 [V ′ ] = U ′ . Further, since f : U → V , Proposition 4.4.1 guarantees that
f −1 : V → U is an isomorphism. Now, if U ′ is a subspace of U , then Theorem 4.2.3
guarantees that f [U ′ ] = V ′ is a subsapce of V ′ . On the other hand, if V ′ is a subspace
of V , then Theorem 4.2.3 implies that f −1 [V ′ ] = U ′ is a subspace of U .
From now on, we assume that U ′ is a subspace of U and V ′ is a subspace of V .
We construct the function f ′ : U ′ → V ′ by setting f ′ (u) = f (u) for all u ∈ V ′ , and
in part (a). Since f : U → V is an isomorphism, and since U ′ and V ′ are subspaces
of U and V , respectively, we see that f ′ : U ′ → V ′ is also an isomorphism,24 and
consequently, U ′ ∼ = V ′ . So, (a) and (b) hold. Part (c) follows from part (a) and from
Theorem 4.2.14(c) applied to U ′ , V ′ , and f ′ .

4.4.3 An application of isomorphisms: transforming polynomials

and matrices into vectors
By Theorem 4.4.6, for all positive integers n and fields F, every n-dimensional vector
space V over F is isomorphic to Fn . Moreover, by Proposition 4.3.1, given any basis B
of such a vector space V , the coordinate function · B : V → Fn is an isomorphism.
 

This is useful because we have developed powerful computational tools for vectors in
Fn . By using isomorphisms, we can reduce problems of computing in an arbitrary
21
That is: if U ′ is a subspace of U and V ′ is a subspace of V .
22
So, we constructed f ′ by restricting both the domain and the codomain of f . This is well defined
because for all u ∈ U ′ , we have that f (u) ∈ f [U ′ ] = V ′ .
23
So, either U ′ and V ′ have the same finite dimension, or they are both infinite-dimensional.
24
This follows from the definition of an isomorphism. Details?

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 311

n-dimensional vector space to problems of computing in Fn , which we know how to

do in many cases.

Remark: When working with coordinate vectors, we must always specify the
basis that we are working with (i.e. with respect to which the coordinate vectors are
computed). Choosing a different basis will, in general, produce different coordinate
vectors. For instance, consider the real vector space P2R of all polynomials of degree
at most 2 and with coefficients in R. There are two “obvious” bases to chose for P2R ,
namely A1 = {1, x, x2 } and A2 = {x2 , x, 1}. For a polynomial p(x) = a2 x2 + a1 x + a0
(with a0 , a1 , a2 ∈ R), we have
   
  a0   a2
p(x) A =  a1  and p(x) A =  a1  .
1 2
a2 a0

As we can see, the coordinate vectors are different (whenever a0 ̸= a2 ), which is why
we have to be careful to specify what basis we are working with.
First of all, using Proposition 4.3.1 and Theorem 4.4.4, we can “translate”
Propositions 3.1.10, 3.2.1, and 3.2.6 into statements for arbitrary non-trivial, finite-
dimensional vector spaces, as follows.

Proposition 4.4.8. Let V be a non-trivial, finite-dimensional vector space over a

field F, and let B = {b1 , . . . , bn } be a basis of V. Let v1 , . . . , vm (m
 ≥ 1) be some

vectors in V , and for all i ∈ {1, . . . , n}, set ai := vi B . Set A := a1 . . . am .
Then all the following hold:

(a) {v1 , . . . , vm } is a linearly independent set in V if and only if rank(A) = m (i.e.

A has full column rank);

(b) {v1 , . . . , vm } is a spanning set of V if and only if rank(A) = n (i.e. A has full
row rank);

(c) {v1 , . . . , vm } is a basis of V if and only if rank(A) = n = m (i.e. A is a square

matrix of full rank).

Proof. By Proposition 4.3.1, · B : V → Fn is an isomorphism. So, Theorem 4.4.4

 

guarantees that the following hold:

(a’) {v1 , . . . , vm } is a linearly independent set in V if and only if {a1 , . . . , am } is a

linearly independent set in Fn ;

(b’) {v1 , . . . , vm } is a spanning set of V if and only if {a1 , . . . , am } is a spanning

set of Fn ;

(c’) {v1 , . . . , vm } is a basis of V if and only if {a1 , . . . , am } is a basis of Fn .

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 312

Now (a) follows from (a’) and Proposition 3.2.1; (b) follows from (b’) and Proposi-
tion 3.1.10; and (c) follows from (c’) and Proposition 3.2.6.

Example 4.4.9. Consider the following sets of polynomials (with coefficients under-
stood to be in R):

(a) A = x2 + x, x3 + 1, x, x2 + 1 ;


(b) B = 3x3 + 2x2 + x + 1, 6x3 + 4x2 + 5x + 6, 5x + 6, 2x + 2 ;



(c) C = x3 + 1, x3 + x2 , x2 + x, x + 1, 1, x ;


(d) D = x3 , 2x2 + 3x, 4x3 + 5x + 6 .



For each of the four sets above, determine whether

ˆ it is linearly independent in P3R ;

ˆ it spans P3R ;

ˆ it is a basis of P3R .

Solution. In what follows, we will use the basis P = 1, x, x2 , x3 of P3R .



(a) We set
T
ˆ a1 := x2 + x P = 0 1 1 0 ;
  

T
ˆ a2 := x3 + 1
  
P
= 1 0 0 1 ;
T
ˆ a3 :=
  
x P
= 0 1 0 0 ;
T
ˆ a4 := x2 + 1
  
P
= 1 0 1 0 ;

Further, we set
 
0 1 0 1
   1 0 1 0 
A := a1 a2 a3 a4 = 
 1
.
0 0 1 
0 1 0 0

By row reducing, we get that RREF(A) = I4 , and consequently, rank(A) = 4. So, by

Proposition 4.4.8, A is a basis of P3R , and in particular, it is a linearly independent
set in P3R , as well as a spanning set of P3R .
(b) We set
T
ˆ b1 := 3x3 + 2x2 + x + 1 P = 1 1 2 3 ;
  

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 313

T
ˆ b2 := 6x3 + 4x2 + 5x + 6
  
P
= 6 5 4 6 ;
T
ˆ b3 :=
  
5x + 6 P
= 6 5 0 0 ;
T
ˆ b4 :=
  
2x + 2 P
= 2 2 0 0 .

Further, we set
 
1 6 6 2
   1 5 5 2 
B := b1 b2 b3 b4 = 
 2
.
4 0 0 
3 6 0 0

By row reducing, we get that

 
1 0 0 2
 0 1 0 −1 
RREF(B) = 
 0
,
0 1 1 
0 0 0 0

and consequently, rank(B) = 3. So, by Proposition 4.4.8, B is not a linearly

independent set in P3R , is not a spanning set of P3R , and is not a basis of P3R .
(c) We set
T
ˆ c1 := x3 + 1 P = 1 0 0 1 ;
  

T
ˆ c2 := x3 + x2
  
P
= 0 0 1 1 ;
T
ˆ c3 := x2 + x
  
P
= 0 1 1 0 ;
T
ˆ c4 :=
  
x+1 P
= 1 1 0 0 ;
T
ˆ c5 :=
  
1 P
= 1 0 0 0 ;
T
ˆ c6 :=
  
x P
= 0 1 0 0 .

Further, we set
 
1 0 0 1 1 0
   0 0 1 1 0 1 
C := c1 c2 c3 c4 c5 c6 = 
 0
.
1 1 0 0 0 
1 1 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 314

By row reducing, we get that

 
1 0 0 1 0 1
 0 1 0 −1 0 −1 
RREF(C) = 
 0
,
0 1 1 0 1 
0 0 0 0 1 −1

and consequently, rank(C) = 4. So, by Proposition 4.4.8, C is not linearly indepen-

dent, it is spanning set of P3R , and it is not a basis of P3R .
Remark: Since dim(P3R ) = 4 (because P = {1, x, x2 , x3 } is a basis of P3R ), and since
C contains six vectors (polynomials), Theorem 3.2.17 guarantees that C is not a
linearly independent set in P3R , and consequently, it is not a basis of P3R . However, to
determine whether C spans P3R , we did in fact have to row reduce.
(d) We set
T
ˆ d1 := x3 P = 0 0 0 1 ;
  

T
ˆ d2 := 2x2 + 3x
  
P
= 0 3 2 0 ;
T
ˆ d3 := 4x3 + 5x + 6
  
P
= 6 5 0 4 .
Further, we set
 
0 0 6
   0 3 5 
D := d1 d2 d3 = 
 0
.
2 0 
1 0 4

By row reducing, we get that

 
1 0 0
 0 1 0 
RREF(D) = 
 0
,
0 1 
0 0 0

and consequently, rank(D) = 3. So, by Proposition 4.4.8, D is linearly independent,

but it is not a spanning set of P3R , and it is not a basis of P3R .
Remark: Since dim(P3R ) = 4 (because P = {1, x, x2 , x3 } is a basis of P3R ), but D
contains only three vectors (polynomials), Theorem 3.2.17 guarantees that D is not a
spanning set of P3R , and consequently, it is not a basis of P3R . However, to determine
whether D is linearly independent, we had to row reduce.

Example 4.4.10. Consider the following sets of matrices (with coefficients under-
stood to be in Z3 ):

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 315

n 1 2
 
0 1
 
0 0
 
2 0
o
(a) A = , , , ;
0 0 0 2 2 1 1 0

n 1       o
1 1 1 2 1 2 0
(b) B = , , , .
0 1 0 2 0 2 1 1
For each of the two sets above, determine whether
ˆ it is linearly independent in Z32×2 ;
ˆ it spans Z2×2
3 ;

ˆ it is a basis of Z2×2
3 .
Solution. In what follows, we will use the basis
n 1 0   0 1   0 0   0 0 o
M = , , ,
0 0 0 0 1 0 0 1
of Z2×2
3 .
(a) We set
   
1 2 T
ˆ a1 :=

= 1 2 0 0 ;
0 0 M
   
0 1 T
ˆ a2 :=

= 0 1 0 2 ;
0 2 M
   
0 0 T
ˆ a3 :=

= 0 0 2 1 ;
2 1 M
   
2 0 T
ˆ a4 :=

= 2 0 1 0 .
1 0 M
Further, we set
 
1 0 0 2
   2 1 0 0 
A := a1 a2 a3 a4 = 
 0
.
0 2 1 
0 2 1 0
By row reducing, we get that
 
1 0 0 2
 0 1 0 2 
RREF(A) = 
 0
,
0 1 2 
0 0 0 0
and consequently, rank(A) = 3. So, by Proposition 4.4.8, A is not a linearly
independent set in Z2×2 2×2 2×2
2 , is not a spanning set of Z2 , and is not a basis of Z2 .
(b) We set

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 316

   
1 1 T
ˆ b1 :=

= 1 1 0 1 ;
0 1 M
   
1 1 T
ˆ b2 :=

= 1 1 0 2 ;
0 2 M
   
2 1 T
ˆ b3 :=

= 2 1 0 2 ;
0 2 M
   
2 0 T
ˆ b4 :=

= 2 0 1 1 .
1 1 M

Further, we set
 
1 1 2 2
   1 1 1 0 
B := b1 b2 b3 b4 = 
 0
.
0 0 1 
1 2 2 1

By row reducing, we get that RREF(B) = I4 , and consequently, rank(B) = 4. So, by

Proposition 4.4.8, B is a basis of Z2×2
3 , and in particular, it is a linearly independent
set in Z2×2
3 , as well as a spanning set of Z2×2
3 .

Finding a basis of the span of polynomials or matrices. We now present

three very similar examples, each dealing with finding a basis of the span of a set
of polynomials or matrices. In Example 4.4.11, we give full theoretical justification
of each step (citing the appropriate theorems and propositions). Examples 4.4.12
and 4.4.13 are very similar, but we do not justify the steps in as much detail.
When solving problems by yourself, you should aim for the level of detail given in
Examples 4.4.12 and 4.4.13.

Example 4.4.11. Consider the following polynomials in PZ2 :

ˆ p1 (x) = x3 + x + 1; ˆ p4 (x) = x + 1;

ˆ p2 (x) = x3 + x2 + 1; ˆ p5 (x) = x2 ;

ˆ p3 (x) = x2 + 1; ˆ p6 (x) = x3 + 1.


Set U := Span p1 (x), . . . , p6 (x) . Find a basis B of U . What is dim(U )? For
each i ∈ {1, . . . , 6} such that pi (x) is not in the basis B, express pi (x) as a linear
combination of the basis vectors in B.

Solution. Note that polynomials p1 (x), . . . , p6 (x) are all
of degree at most 3, and
they all belong to P3Z2 . Thus, U = Span p1 (x), . . . , p6 (x) is a subspace of P3Z2 . We

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 317

know that
A = {1, x, x2 , x3 }
is a basis of P3Z2 and (by Proposition 4.3.1) that · A : P3Z2 → Z42 is an iso-
 
 
morphism. Next, by Theorem 4.2.11, the image of U under · A is precisely
    
V := Span p1 (x) A , . . . , p6 (x) A , and moreover, Proposition 4.4.7 guaran-
tees that when we restrict the domain of · A to U and the codomain to V ,25 we
 

obtain an isomorphism. So, we first solve the problem  for the coordinate vectors 
     
p1 (x) A , . . . , p6 (x) A and the subspace V = Span p1 (x) A , . . . , p6 (x) A
 
of V , and then using the fact that · A is an isomorphism, we “translate” the


solution back to p1 (x), . . . , p6 (x) and U = Span p1 (x), . . . , p6 (x) .
We first read off the coordinate vectors of our six polynomials with respect to
the basis A:
T T
ˆ ˆ
     
p1 (x) A
= 1 1 0 1 ; p4 (x) A
= 1 1 0 0 ;
T T
ˆ ˆ
     
p2 (x) A
= 1 0 1 1 ; p5 (x) A
= 0 0 1 0 ;
T T
ˆ ˆ
     
p3 (x) A
= 1 0 1 0 ; p6 (x) A
= 1 0 0 1 .

We now form the matrix

 
1 1 1 1 0 1
       1 0 0 1 0 0 
A = p1 (x) A
... p6 (x) A
= 
 0
,
1 1 0 1 0 
1 1 0 0 0 1
and by row reducing, we obtain the following (pivot columns are in red, and non-pivot
columns are in blue):
 
1 0 0 1 0 0
 0 1 0 1 0 1 
RREF(A) =   0 0 1 1 0 0 .


0 0 0 0 1 1
Since the pivot columns of A are its first, second, third, and fifth column, Theo-
rem 3.3.4 guarantees that
n        o
C := p1 (x) A , p2 (x) A , p3 (x) A , p5 (x) A
      
is a basis of Col(A) = Span p1 (x) A , p2 (x) A , . . . , p6 (x) A = V . As
we pointed out above, the function obtained from · A : PZ2 → Z42 by re-
 


stricting the domain to U = Span p1 (x), . . . , p6 (x) and the codomain to V =
25
 
This is well defined because the image of U under · A
is precisely V .

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 318

    
Span p1 (x) A , . . . , p6 (x) A is an isomorphism. So, since C is a basis of V ,
Theorem 4.4.4(c) guarantees that
n o
B := p1 (x), p2 (x), p3 (x), p5 (x)

is a basis of U . Since U has a four-vector basis, we see that dim(U ) = 4.

It remains to express p4 (x) and p6 (x) as a linear combination of the polynomials
in the basis B. First, we see from the matrix RREF(A) that the following hold:

ˆ p4 (x) A = p1 (x) A + p2 (x) A + p3 (x) A ;

       

ˆ p6 (x) A = p2 (x) A + p5 (x) A .

     

But now
       
p4 (x) A
= p1 (x) A
+ p2 (x) A
+ p3 (x) A

(∗)  
= p1 (x) + p2 (x) + p3 (x) A

and
      (∗)  
p6 (x) A
= p2 (x) A
+ p5 (x) A
= p2 (x) + p5 (x) A
,
 
where both instances of (*) follow from the fact that · A is linear (because it is an
 
isomorphism). But · A is also one-to-one (again, because it is an isomorphism);
it follows that

ˆ p4 (x) = p1 (x) + p2 (x) + p3 (x),

ˆ p6 (x) = p2 (x) + p5 (x),

and we are done.

Optional: It is not a bad idea to check whether our expressions for p4 (x) and p6 (x)
are correct (to make sure we did not miscompute). So, we compute:

p1 (x) + p2 (x) + p3 (x) = (x3 + x + 1) + (x3 + x2 + 1) + (x2 + 1)

= (x3 + x3 ) + (x2 + x2 ) + x + (1 + 1 + 1)

= x+1

= p4 (x)

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 319

and
p2 (x) + p5 (x) = (x3 + x2 + 1) + x2

= x3 + (x2 + x2 ) + 1

= x3 + 1

= p6 (x).
As we can see, our expressions for p4 (x) and p6 (x) are correct.

Example 4.4.12. Consider the following polynomials in PZ3 :

ˆ p1 (x) = x4 + 2; ˆ p4 (x) = 2x4 + x3 + x2 + 1;

ˆ p2 (x) = x3 + x2 ; ˆ p5 (x) = 2x + 1.

ˆ p3 (x) = x4 + x3 + x2 + 2;


Set U := Span p1 (x), . . . , p5 (x) . Find a basis B of U . What is dim(U )? For
each i ∈ {1, . . . , 5} such that pi (x) is not in the basis B, express pi (x) as a linear
combination of the basis vectors in B.
Solution. Note that polynomials p1 (x), . . . , p5 (x) are all
of degree at most 4, and
they all belong to P4Z3 . Thus, U = Span p1 (x), . . . , p5 (x) is a subspace of P4Z3 . We
know that
A = {1, x, x2 , x3 , x4 }
is a basis of P4Z3 . The coordinate vectors of p1 (x), . . . , p5 (x) with respect to the basis
A are as follows:
T
ˆ p1 (x) A = 2 0 0 0 1 ;
  

T
ˆ p2 (x) A = 0 0 1 1 0 ;
  

T
ˆ p3 (x) A = 2 0 1 1 1 ;
  

T
ˆ p4 (x) A = 1 0 1 1 2 ;
  

T
ˆ p5 (x) A = 1 2 0 0 0 .
  

We form the matrix

 
2 0 2 1 1
     

 0 0 0 0 2 

A = p1 (x) A
... p5 (x) A
= 
 0 1 1 1 0 ,

 0 1 1 1 0 
1 0 1 2 0

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 320

and by row reducing, we obtain the following (pivot columns are in red, and non-pivot
columns are in blue):
 
1 0 1 2 0
 0 1 1 1 0 
 
RREF(A) =   0 0 0 0 1 .

 0 0 0 0 0 
0 0 0 0 0
We see that the pivot columns of A are its first, second, and fifth column. Therefore,
n      o
C := p1 (x) A , p2 (x) A , p5 (x) A
    
is a basis of Col(A) = Span p1 (x) A , . . . , p5 (x) A . Consequently,
n o
B := p1 (x), p2 (x), p5 (x)


is a basis of U = Span p1 (x), . . . , p5 (x) , and it follows that dim(U ) = 3.
It remains to express p3 (x) and p4 (x) as a linear combination of the vectors
(polynomials) in B. First, we have that
 (∗)   (∗∗) 
ˆ p3 (x) A = p1 (x) A + p2 (x) A = p1 (x) + p2 (x) A ,
   

(∗) (∗∗)
ˆ
       
p4 (x) A
=2 p1 (x) A
+ p2 (x) A
= 2p1 (x) + p2 (x) A
,
where both instances of (*) were obtained  from the matrix RREF(A), and both
instances of (**) follow from the fact that · A : P4Z3 → Z53 is linear (because it is an

 
isomorphism). Since · A is also one-to-one (again, because it is an isomorphism),
we get that
ˆ p3 (x) = p1 (x) + p2 (x),
ˆ p4 (x) = 2p1 (x) + p2 (x),
and we are done.

Optional: Let us check that our expressions for p3 (x) and p4 (x) are correct. We
compute:
p1 (x) + p2 (x) = (x4 + 2) + (x3 + x2 ) = x4 + x3 + x2 + 2 = p3 (x)
and
2p1 (x) + p2 (x) = 2(x4 + 2) + (x3 + x2 ) = 2x4 + x3 + x2 + 1 = p4 (x).
As we can see, our expressions for p3 (x) and p4 (x) are correct.

Example 4.4.13. Consider the following matrices in R2×2 :

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 321

     
−2 1 5 −2 0 0
ˆ M1 = ; ˆ M3 = ; ˆ M5 = ;
3 −2 −6 6 3 0
     
1 0 −3 0 7 −2
ˆ M2 = ; ˆ M4 = ; ˆ M6 = .
0 2 0 −6 −9 10
 
Set U := Span M1 (x), . . . , M6 (x) . Find a basis B for U . What is dim(U )? For
each i ∈ {1, . . . , 6} such that Mi (x) is not in the basis B, express Mi (x) as a linear
combination of the basis vectors in B.

Solution. Consider the basis

n 1 0   0 1   0 0   0 0 o
A := , , ,
0 0 0 0 1 0 0 1

of R2×2 . We form the coordinate vectors

     
−2 5 0
 1   −2   0 
ˆ M1 A =  ˆ M3 A =  ˆ M5 A = 
     
 3 ;  −6 ;  3 ;
  

−2 6 0
     
1 −3 7
 0   0   −2 
ˆ M2 A =  ˆ M4 A =  ˆ M6 A = 
     
 0 ;  0 ;  −9 .
  

2 −6 10

We now form the matrix

 
−2 1 5 −3 0 7
       1 0 −2 0 0 −2 
A := M1 A
... M6 A
=  ,
 3 0 −6 0 3 −9 
−2 2 6 −6 0 10

and by row reducing, we obtain the following (pivot columns are in red, and non-pivot
columns are in blue):
 
1 0 −2 0 0 −2
 0 1 1 −3 0 3 
RREF(A) =  .
 0 0 0 0 1 −1 
0 0 0 0 0 0

We see that the pivot columns of A are its first, second, and fifth column. Therefore,
n      o
C := M 1 A , M 2 A , M5 A

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 322

   

is a basis of Col(A) = Span A
, . . M1
. , M 6 A . Consequently,
n o
B := M1 , M2 , M5
 
is a basis of U = Span M1 , . . . , M6 , and it follows that dim(U ) = 3.
It remains to express M3 , M4 , M6 as a linear combination of the vectors (matrices)
in C. First, we have that
 (∗)  (∗∗) 
ˆ M3 A = −2 M1 A + M2 A = −2M1 + M2 A ,
    

(∗) (∗∗)
ˆ
     
M4 A
= −3 M2 A
= −3M2 A
,
(∗)  (∗∗) 
ˆ
       
M6 A
= −2 M1 A
+3 M2 A
− M5 A = −2M1 + 3M2 − M5 A ,
where all three instances of (*) can be read  off from RREF(A), and all three
instances of (**) follow from the fact that · A : R2×2 → R4 is linear (because it is
an isomorphism). But since · A : R2×2 → R4 is also one-to-one (again, because it
 

is an isomorphism), we now get that

ˆ M3 = −2M1 + M2 ,
ˆ M4 = −3M2 ,
ˆ M6 = −2M1 + 3M2 − M5 ,
and we are done.

Optional: Let us check that our expressions for M3 , M4 , M6 are correct.

First, for M3 , we compute:
     
−2 1 1 0 5 −2
−2M1 + M2 = −2 + = = M3 ,
3 −2 0 2 −6 6
and we see that our expression for M3 is correct.
Next, for M4 , we compute:
   
1 0 −3 0
−3M2 = −3 = = M4 ,
0 2 0 −6
and we see that our expression for M4 is correct.
Finally, for M6 , we compute:
     
−2 1 1 0 0 0
−2M1 + 3M2 − M5 = −2 +3 −
3 −2 0 2 3 0
 
7 −2
=
−9 10

= M6 ,

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 323

and we see that our expression for M6 is correct.

Extending a basis of a subspace to a basis of the whole vector space.

Our next example is similar to Example 3.3.24 from subsection 3.3.5, only we have
polynomials instead of column vectors. We will use coordinate vectors to transform
polynomials into column vectors, and we will rely on Proposition 3.3.23.

Example 4.4.14. Consider the following polynomials in P3Z3 :

ˆ p1 (x) = x3 + 1; ˆ p3 (x) = x2 + 2x + 1

ˆ p2 (x) = 2x3 + 2; ˆ p4 (x) = 2x3 + x2 + 2x.

Find a basis BU of U := Span p1 (x), p2 (x), p3 (x), p4 (x) , extend it to a basis B of
P3Z3 , and for each i ∈ {1, 2, 3, 4} such that pi (x) ∈/ B, express pi (x) as a linear
combination of the basis vectors
 in B. Finally, for each i ∈ {1, 2, 3, 4}, compute the
coordinate vector pi (x) B .
Solution. We know that
A := {1, x, x2 , x3 }
is a basis of P3Z3 , and we let V be the image of U under the isomorphism
 
· A
.
Further, we consider the standard basis
n      o
1 A , x A , x2 A , x3 A
 
E4 = {e1 , e2 , e3 , e4 } =

of Z43 . We now form the 4 × 8 matrix C whose columns are the coordinate vectors of
the polynomials
p1 (x), p2 (x), p3 (x), p4 (x), 1, x, x2 , x3
with respect to the basis A. Here is the matrix C explicitly (with tiny font so that it
would fit on the page):
h  i
x2 x3
              
C := p1 (x) A
p2 (x) A
p3 (x) A
p4 (x) A
1 A
x A .
A A

We then have that

 
1 2 1 0 1 0 0 0
 0 0 2 2 0 1 0 0 
C = 
 0
.
0 1 1 0 0 1 0 
1 2 0 2 0 0 0 1
By row reducing, we obtain
 
1 2 0 2 0 0 0 1
 0 0 1 1 0 0 1 0 
RREF(C) = 
 0
,
0 0 0 1 0 2 2 
0 0 0 0 0 1 1 0

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 324

and we see that the pivot columns of C are its first, third, fifth, and sixth column.
By Proposition 3.3.23, the pivot columns of C to the left of the vertical dotted line
form a basis of V , and all the pivot columns of C together form a basis of Z43 . So,
n    o
p1 (x) A , p3 (x) A

is a basis of V , whereas
n        o
p1 (x) A
, p3 (x) A
, 1 A
, x A

is a basis of Z43 that extends our basis of V . Since ·

 
A
is an isomorphism, we see
that n o
BU := p1 (x), p3 (x)
is a basis of U , and that
n o
B := p1 (x), p3 (x), 1, x

is a basis of P3Z3 that extends our basis BU of U . We can read off from RREF(C)
that
ˆ p2 (x) = 2p1 (x),

ˆ p4 (x) = 2p1 (x) + p3 (x).

Finally, we read off the coordinate vectors:
T
ˆ p1 (x) B = 1 0 0 0 ,
  

T
ˆ
  
p2 (x) B
= 2 0 0 0 ,
T
ˆ
  
p3 (x) B
= 0 1 0 0 ,
T
ˆ
  
p4 (x) B
= 2 1 0 0 ,
and we are done.

Example 4.4.15. Consider the following matrices in Z32×2 :

       
2 0 1 0 1 0 2 0
M1 = , M2 = , M3 = , M4 = .
0 0 0 0 2 2 2 2

Find a basis BU of U := Span(M1 , M2 , M3 , M4 ), extend it to a basis B of Z2×2

3 , and
for each i ∈ {1, 2, 3, 4} such that Mi ∈
/ B, express Mi as a linear combination of the
basis
 vectors
 in B. Finally, for each i ∈ {1, 2, 3, 4}, compute the coordinate vector
Mi B .

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 325

Solution. We proceed similarly as in Example 4.4.14. Set

       
1 0 0 1 0 0 0 0
A1 := , A2 := , A3 := , A4 := .
0 0 0 0 1 0 0 1

We know that
A := {A1 , A2 , A3 , A4 }
is a basis of Z2×2
 
3 , and we let V be the image of U under the isomorphism · A
.
Further, we consider the standard basis
n        o
E4 = {e1 , e2 , e3 , e4 } = A1 A , A2 A , A3 A , A4 A

of Z43 . We now form the 4 × 8 matrix C whose columns are the coordinate vectors of
the matrices
M1 , M 2 , M 3 , M 4 , A 1 , A 2 , A 3 , A 4
with respect to the basis A. Here is the matrix C explicitly (as in Example 4.4.14,
the font is tiny so that the matrix would fit on the page):
                 
C := M1 A
M2 A
M3 A
M4 A
A1 A
A2 A
A3 A
A4 A .

We then have that

 
2 1 1 2 1 0 0 0
 0 0 0 0 0 1 0 0 
C = 
 0

0 2 2 0 0 1 0 
0 0 2 2 0 0 0 1

By row reducing, we obtain

 
1 2 0 2 2 0 0 2
 0 0 1 1 0 0 0 2 
RREF(C) = 
 0
,
0 0 0 0 1 0 0 
0 0 0 0 0 0 1 2

and we see that the pivot columns of C are its first, third, sixth, and seventh column.
By Proposition 3.3.23, the pivot columns of C to the left of the vertical dotted line
form a basis of V , and all the pivot columns of C together form a basis of Z4 . So,
n    o
M1 A , M3 A

is a basis of V , whereas
n        o
M1 A
, M3 A
, A2 A
, A3 A

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 326

is a basis of Z43 that extends our basis of V . Since · A is an isomorphism, we see

 

that
BU := {M1 , M3 }
is a basis of U , and that
B := {M1 , M3 , A2 , A3 }

is a basis of Z2×2
3 that extends our basis BU of U . We can read off from RREF(C)
that
ˆ M2 = 2M1 ,
ˆ M4 = 2M1 + M3 .
Finally, we can read off the coordinate vectors:
T
ˆ M1 B = 1 0 0 0 ,
  

T
ˆ
  
M2 B
= 2 0 0 0 ,
T
ˆ
  
M3 B
= 0 1 0 0 ,
T
ˆ
  
M4 B
= 2 1 0 0 ,
and we are done.

4.5 Matrices of linear functions between non-trivial,

finite-dimensional vector spaces
As we have already mentioned, linear functions between general vector spaces do not
have standard matrices. However, as our next theorem shows, we can associate a
matrix to a linear function between non-trivial,26 finite-dimensional vector spaces,
provided we have first specified a basis of the domain and a basis of the codomain.
Theorem 4.5.1. Let U and V be non-trivial, finite-dimensional vector spaces over
a field F. Let B = {b1 , . . . , bm } be a basis of U , let C = {c1 , . . . , cn } be a basis of
V , and let f : U → V be a linear function. Then exists a unique matrix in Fn×m ,
denoted by C f B and called the matrix of f with respect to B and C, such that for
all u ∈ U , we have that
     
C
f B
u B
= f (u) C
.
 
Moreover, the matrix C f B is given by
       
C
f B = f (b1 ) C . . . f (bm ) C .
26
Recall that a vector space V over a field F is non-trivial if it contains at least one non-zero
vector, or equivalently, if dim(V ) > 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 327

Proof. Existence. Fix u ∈ U . We must show that

        
f (b1 ) C . . . f (bm ) C u B = f (u) C .
   T
Set u B
= β1 . . . βm , so that u = β1 b1 + · · · + βm bm . We then compute:
      
f (b1 ) C
... f (bm ) C
u B
 
β1
f (bm ) C  ... 
     
= f (b1 ) ...

C
βm
   
= β1 f (b1 ) C
+ · · · + βm f (bm ) C

(∗)  
= β1 f (b1 ) + · · · + βm f (bm ) C

(∗∗) 

= f β1 b1 + · · · + βm bm C
 
= f (u) C
,

where (*) follows from the fact that · C : V → Fn is an isomorphism (and in

 

particular, a linear function), and (**) follows from the fact that f is linear.27
Uniqueness. Fix any matrix A = a1 . .. am  in Fn×m
 
 that has the property
that for all u ∈ U , we have that A u B = f (u) C . We must show that
     
A = f (b1 ) C . . . f (bm ) C . We prove this by showing that the two
 
matrices have the same corresponding columns, that is, that ai = f (bi ) C for all
indices i ∈ {1, . . . , m}. Indeed, for all i ∈ {1, . . . , m}, we have the following:

ai = Aem
i by Proposition 1.4.5

because bi B = em
 
i
 
= A bi B (by Proposition 3.2.9)
 
= f (bi ) C
by the choice of A.
     
This proves that A = f (b1 ) C
... f (bm ) C
, and we are done.

Remark: Suppose that U and V are non-trivial, finite-dimensional vector spaces

over a field F, that B = {b1 , . . . , bm } and C = {c1 , . . . , cn } are bases of U and V ,
respectively, and that f : U → V is a linear function, as in Theorem 4.5.1. Then the
27
Technically, both (*) and (**) also rely on Proposition 4.1.5.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 328

uniqueness part of Theorem 4.5.1 guarantees that if A ∈ Fn×m is any matrix that
satisfies the property that for all u ∈ U , we have that
   
A u B = f (u) C ,
 
then we in fact have that A = C f B . We will use this observation repeatedly (see
the proofs of Theorem 4.5.3, Theorem 4.5.4, and Lemma 4.5.11).
 
Remark: Note that matrices of the form C f B are generalizations of standard
m n
 Indeed, if F is a field and f : F → F is a linear function, then
matrices. the matrix
f E is precisely the standard matrix of f . (As usual, Em = {e1 , . . . , em
m

En m m } is
m n n n
the standard basis of F , and En = {e1 , . . . , en } is the standard basis of F .)28

n 1   1 o
Example 4.5.2. Consider the basis B = , of R2 , and consider the
0 1
unique linear function f : R2 → R2 that satisfies the following:
 1   1 
ˆ f = ,
0 0
 1   2 
ˆ f = .
1 2
 
Compute the matrix B f B .
 1   2 
x2 x2 f =
1 2 f (u)

 
1
1 f
u


1
 x1  1   1  x1
f =
0 0 0
n 1   1 o
Remark: The fact that B = , is a basis of R2 follows from the fact that
0 1
 1 1 
rank = 2 and from the Invertible Matrix Theorem (see subsection 3.3.7).
0 1
The existence and uniqueness of the linear function f follows from Theorem 4.3.2.
28
This is “obvious,” but here are the details. Let F be a field, let f : Fm → Fn be a linear function,
and let A be the standard matrix of f . Then for all u ∈ Fm , we have the following:
  (∗) (∗∗) (∗)  
A u Em
= Au = f (u) = f (u) En
,

where both instances of (*) follow from Example 3.2.8(a) (or alternatively, from Proposition 3.2.9),
and (**) follows from the fact that A is thestandard matrix
  of f . But now the uniqueness part of
Theorem 4.5.1 guarantees that A = E f E , i.e. E f E is the standard matrix of f .
n m n m

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 329

Solution. Using the formula from Theorem 4.5.1, we compute:

         
  1 1
B
f B = f f
0 B
1 B
         
1 2
=
0 B
2 B
 
1 0
= .
0 2

By Proposition 4.1.7, sums, scalar multiples, and compositions of linear functions

are linear (as long as the domains and codomains are compatible). Theorem 4.5.3
(below) describes the matrices of sums, scalar multiples, and compositions or linear
functions, and it is a generalization of Proposition 1.10.13.

Theorem 4.5.3. Let U , V , and W be non-trivial, finite-dimensional vector spaces

over a field F. Let B = {b1 , . . . , bm } be a basis of U , let C = {c1 , . . . , cn } be a basis
of V , and let D = {d1 , . . . , dp } be a basis of W . Then all the following hold:

(a) for all linear functions f, g : U → V , the function f + g is linear,29 and moreover,
     
C
f +g B = C f B+C g B;

(b) for all linear functions f : U → V and scalars α ∈ F, the function αf is linear,30
and moreover,    
C
αf B
= α C
f B
;

(c) for all linear functions f : U → V and g : V → W , the function g ◦ f is linear,31

and moreover,      
D
g ◦ f B
= D
g C C
f B
.
   
g ◦ f, D
g C C
f B

   
f, C
f B
g, D
g C

U V W

B C D
29
As usual, the function f + g : U → V is defined by (f + g)(u) = f (u) + g(u) for all u ∈ U .
30
As usual, the function αf : U → V is defined by (αf )(u) = α(f (u)) for all u ∈ U .
31
As usual, the function g ◦ f : U → W is defined by (g ◦ f )(u) = g(f (u)) for all u ∈ U .

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 330

Proof. We prove (c). The proofs of (a) and (b) are left as an exercise. The fact
 g ◦ f is linear
that  follows
 from
 Proposition 4.1.7(c). It remains to show that
D
g ◦ f B
=D g C C
f B
.

Claim. For all u ∈ U , we have that

         
D
g C C
f B
u B = (g ◦ f )(u) D .

Proof of the Claim. For all u ∈ U , we have the following:

               
D
g C C
f B
u B
= D
g C C
f B
u B

   
= D
g C
f (u) C


= g f (u) D
 
= (g ◦ f )(u) D
.

This proves the Claim. ♦

The Claim and the uniqueness part of Theorem 4.5.1 now imply that
     
D
g◦f B = D g C C f B,

which is what we needed to show.

We can use matrices of linear functions to determine various properties of those

linear functions. Notably, we have Theorem 4.5.4 below. We remark that parts (a)
and (b) of Theorem 4.5.4 generalize Proposition 4.2.7, part (c) essentially follows
from Theorem 4.2.4,32 parts (d) and (e) generalize Theorem 1.10.18, and parts (f)
and (g) generalize (one part of) Theorem 1.11.9.

Theorem 4.5.4. Let U and V be non-trivial, finite-dimensional vector spaces over

a field F. Let B = {b1 , . . . , bm } be a basis of U , let C = {c1 , . . . , cn } be a basis of V ,
and let f : U → V be a linear function.33 Then all the following hold:
   
(a) rank(f ) = rank C f B ;

    
(b) dim Ker(f ) = dim Nul C f B ;
   
(c) f is one-to-one if and only if Nul C
f B
= {0};
32
Actually, it follows from Theorem 4.2.4 and Theorem 4.5.4(b).
33
Note that this means that dim(U ) = m, dim(V ) = n, and C f B ∈ Fn×m .
 

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 331

     
(d) f is one-to-one if and only if rank C f B = m (i.e. the matrix C f B has
full column rank);
     
(e) f is onto if and only if rank C f B = n (i.e. the matrix C f B has full
row rank);
 
(f ) f is an isomorphism if and only if the matrix C f B is invertible (and in
particular, square);
   −1
(g) if f is an isomorphism, then B f −1 C = C f B
 
.

Proof. We first prove (a). By Theorem 4.5.1, we have that

       
C
f B = f (b1 ) C . . . f (bm ) C .

We now compute:
(∗)


rank(f ) = dim Span f (b1 ), . . . , f (bm )
!
(∗∗)
    
= dim Span f (b1 ) C , . . . , f (bm ) C

!
     
= dim Col f (b1 ) C
... f (bm ) C

!
   
= dim Col C
f B

(∗∗∗)
   
= rank C f B ,

where (*) follows from the fact that B = {b1 , . . . , bm } is a spanning set of U
(because it is a basis of U ) and from Corollary 4.2.12, (**) follows from the fact that
· C : V → Fn is an isomorphism and from Propositions 4.2.11(b) and 4.4.7(c),34
 

and (***) follows from Theorem 3.3.4. This proves (a).

34
Let us explain this in more detail. Since · C : V → Fn is an isomorphism, Proposition 4.4.7(c)
 
 
guarantees that V ′ := Span f (b1 ), . . . , f (bm ) and the image of V ′ under · C have the same
 

dimension. On the other hand, by Theorem 4.2.11(b), the image of V ′ under · C is precisely
 
    
equal to Span f (b1 ) C , . . . , f (bm ) C . This justifies (**).

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 332

For (b), we first observe that

(∗)
rank(f ) + dim Ker(f ) = dim(U )

= m

(∗∗)
        
= rank C
f B
+ dim Nul C f B

where (*) follows from the rank-nullity theorem for linear

 functions,
 and (**) follows
from the rank-nullity theorem for matrices (since C f B is an n × m matrix).
   

But by (a), we have that rank(f ) = rank C f B . Therefore, dim Ker(f ) =
    
dim Nul C f B . This proves (b).

For (c), we have the following sequence of equivalent statements:

(∗)
f is one-to-one ⇐⇒ Ker(f ) = {0}


⇐⇒ dim Ker(f ) = 0

(∗∗)
    
⇐⇒ dim Nul C f B =0
   
⇐⇒ Nul C f B = {0},

where (*) follows from Theorem 4.2.4, and (**) follows from part (b).
For (d), we have the following sequence of equivalent statements:
(∗)
   
f is one-to-one ⇐⇒ Nul C f B = {0}
!
   
⇐⇒ dim Nul C f B =0

(∗∗)
   
⇐⇒ rank C f B = m,

where (*) follows from part (c), and (**) follows from the rank-nullity theorem for

Irena Penev
Linear Algebra 1 & 2
 Chapter 4. Linear functions 333

matrices.35
For (e), we have the following sequence of equivalent statements:
(∗)
f is onto ⇐⇒ rank(f ) = dim(V )

(∗∗)
⇐⇒ rank(f ) = n

(∗∗∗)
   
⇐⇒ rank C f B = n,

where (*) follows from Proposition 4.2.6, (**) follows from the fact that dim(V ) = n,
and (***) follows from part (a).
Next, we prove (f). Suppose first that f is an isomorphism. Then by Theo-
rem 4.2.14(c) (or alternatively, by Theorem 4.4.6), we have that dim(U ) = dim(V ),
i.e. m = n. In particular, C f B is an n × n matrix. Next, since f is an iso-
morphism
  (and in particular, an onto linear function), part (d) guarantees that
 
rank C f B = n. But now the Invertible Matrix Theorem (see subsection 1.11.7
 
or 3.3.7) guarantees that C f B is invertible.
   
Suppose, conversely, that C f B is invertible. In particular, C f B is a square
 
matrix, and it follows that m = n (because C f B is an n × m matrix). Now,
 
since C f B is an invertible n × n matrix, the Invertible Matrix Theorem (see
   
subsection 1.11.7 or 3.3.7) implies that rank C f B = n. But now parts (d)
and (e) guarantee that f is one-to-one and onto,36 and consequently, a bijection.
Since f is also linear (by hypothesis), it follows that f is an isomorphism. This
proves (f).
It remains to prove (g). Suppose that f is an isomorphism. Then by Theo-
rem 4.2.14(c) (or alternatively,
 by  Theorem 4.4.6), we have that m = dim(U ) =
dim(V ) = n. Consequently, C f B is an n × n matrix. Moreover, by (f), the matrix
 
C
f B
is invertible. But now for all v ∈ V , we have the following:
   −1     −1 h   i
f f −1 (v)

C
f B
v C
= C
f B C

35
 
Indeed, since C
f is an n × m matrix, the rank-nullity theorem for matrices guarantees that
B
!
       
rank C f B + dim Nul C f B = m,

and (**) immediately follows.

36
We are also using the fact that m = n, and so part (d) applies.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 334

   −1    
f −1 (v)
 
= C
f B C
f B B

!
   −1
f −1 (v)
   
= C
f B C
f B B
| {z }
=In

f −1 (v)
 
= B
.

The uniqueness part of Theorem 4.5.1 now implies that

 −1     −1
B
f C
= C
f B
.

This proves (f).

Example 4.5.5. Let U and V be real vector spaces. Let B = {b1 , b2 , b3 } be a basis
of U , and let C = {c1 , c2 , c3 } be a basis of V . Let f : U → V be the unique linear
function that satisfies the following:
ˆ f (b1 ) = −2c1 + c3 ;

ˆ f (b2 ) = c1 + 3c2 + c3 ;

ˆ f (b3 ) = c2 − 2c3 .
(The existence and uniqueness of f follow from Theorem 4.3.2.)
ˆ Compute the matrix C f B .
 

ˆ Compute rank(f ) and dim Ker(f )).

ˆ If f one-to-one? Is f onto? If f an isomorphism?

Solution. Using Theorem 4.5.1, we compute:
         
C
f B
= f (b 1 ) C
f (b 2 ) C
f (b3 ) C
       
= −2c1 + c3 C
c1 + 3c2 + c3 C
c2 − 2c3 C
 
−2 1 0
0 3
=  1 .
1 1 −2
 

By row reducing, we obtain RREF C f B = I3 . So, by Theorem 4.5.4, we see
 

that rank(f ) = rank C f B = 3, and consequently, by the rank-nullity theorem,


we have that dim Ker(f ) = dim(U ) − rank(f ) = 3 − 3 = 0. By Theorem 4.5.4, f is
one-to-one, onto, and an isomorphism.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 335

Example 4.5.6. Let U and V be vector spaces over Z3 . Let B = {b1 , b2 , b3 } be a

basis of U , and let C = {c1 , c2 } be a basis of V . Let f : U → V be the unique linear
function that satisfies the following:

ˆ f (b1 ) = c1 + c2 ;

ˆ f (b2 ) = c1 + 2c2 ;

ˆ f (b3 ) = 2c1 + c2 .

(The existence and uniqueness of f follow from Theorem 4.3.2.)

ˆ Compute the matrix C f B .

 

ˆ Compute rank(f ) and dim Ker(f )).

ˆ If f one-to-one? Is f onto? If f an isomorphism?

Proof. First of all, we observe that dim(U ) = 3 and dim(V ) = 2, and in particular,
dim(U ) > dim(V ). So, by Theorem 4.2.14, f is not one-to-one, and f is not an
isomorphism. Meanwhile, Theorem 4.2.14 gives us no information about whether f
is onto; we will determine this by computing.
Using Theorem 4.5.1, we compute:
         
C
f B = f (b1 ) C f (b2 ) C f (b3 ) C
       
= c1 + c2 C
c1 + 2c2 C
2c1 + c2 C
 
1 1 2
= .
1 2 1

By row reducing, we obtain

 
 
1 0 0
RREF C f B = .
0 1 2
 

So, by Theorem 4.5.4, we see that rank(f ) = rank C f B = 2, and consequently,


by the rank-nullity
  theorem, dim Ker(f ) = dim(U ) − rank(f ) = 3 − 2 = 1. Since
the matrix C f B has full row rank, Theorem 4.5.4 guarantees that f is onto.
(Alternatively, since rank(f ) = dim(V ), Proposition 4.2.6 guarantees that f is
onto.)

Example 4.5.7. Let U and V be vector spaces over Z2 . Let B = {b1 , b2 , b3 } be a

basis of U , and let C = {c1 , c2 , c3 , c4 } be a basis of V . Let f : U → V be the unique
linear function that satisfies the following:
ˆ f (b1 ) = c1 + c2 + c4 ;

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 336

ˆ f (b2 ) = c1 + c2 + c3 ;

ˆ f (b3 ) = c3 + c4 .

(The existence and uniqueness of f follow from Theorem 4.3.2.)

ˆ Compute the matrix C f B .

 

ˆ Compute rank(f ) and dim Ker(f )).

ˆ If f one-to-one? Is f onto? If f an isomorphism?

Proof. First of all, we observe that dim(U ) = 3 and dim(V ) = 4, and in particular,
dim(U ) < dim(V ). So, by Theorem 4.2.14, f is not onto, and f is not an isomorphism.
Meanwhile, Theorem 4.2.14 gives us no information about whether f is one-to-one;
we will determine this by computing.
Using Theorem 4.5.1, we compute:
         
C
f B = f (b1 ) C f (b2 ) C f (b3 ) C
       
= c1 + c2 + c4 C
c1 + c2 + c3 C
c3 + c4 C
 
1 1 0
 1 1 0 
= 
 0
.
1 1 
1 0 1

By row reducing, we obtain

 
1 0 1
 
 0 1 1 
RREF C f B = 
 0
.
0 0 
0 0 0
 

So, by Theorem 4.5.4, we see that rank(f ) = rank C f B = 2, and consequently,


by the rank-nullity
  theorem, dim Ker(f ) = dim(U )−rank(f ) = 3−2 = 1. Since the
matrix C f B does not have full column rank, Theorem 4.5.4 guarantees that f is
not one-to-one. (Alternatively, since Ker(f ) is not trivial, Theorem 4.2.4 guarantees
that f is not one-to-one.)

Suppose that U and V are non-trivial, finite-dimensional vector spaces over a

field F, that B = {b1 , . . . , bm } is a basis of U , and that C = {c1 , . . . , cn } is a basis
of V . By Theorem 4.5.1, to every linear function  f : U → V , we can associate a
unique matrix A ∈ Fn×m (which we denoted by C f B ) such that for all u ∈ U , we

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 337

   
have that A u B = f (u) C . How about the converse? Is it true that for every
matrix A ∈ Fn×m , there exists a linear function f : U → V such that A = C f B ?
 

As our next proposition shows, this is indeed true, but the proof is not completely
obvious: it relies on Theorems 4.3.2, 4.5.1, and 4.5.3(c).

Proposition 4.5.8. Let U and V be non-trivial, finite-dimensional vector spaces

over a field F, let B = {b1 , . . . , bm } be a basis of U , and let C = {c1 , . . . , cn } be a
n×m , there exists a unique linear function
basis of V . Then for every matrix  A∈F
f : U → V such that A = C f B .

Proof. Existence. As usual, Em = {em m

1 , . . . , em } is the standard basis of F , and
m
n n n
En = {e1 , . . . , en } is the standard basis of F . Using Theorem 4.3.2, we let f1 : U →
Fm be the unique linear function such that f1 (b1 ) = em m
1 , . . . , f1 (bm ) = em , and we
let f2 : Fn → V be the unique linear function such that f2 (en1 ) = c1 , . . . , f2 (enn ) = cn .
(Note that Proposition 4.4.5 guarantees that f1 and f2 are actually isomorphisms, but
we will not actually use this fact.) Next, let fA : Fm → Fn be given by fA (u) = Au
for all u ∈ U ; then fA is linear (by  Proposition
 1.10.4), and moreover, A is the
standard matrix of fA , i.e. A = E fA E . Finally, set f := f2 ◦ fA ◦ f1 (see the
n m  
diagram below). Our goal is to show that A = C f B .
f := f2 ◦ fA ◦ f1

f1 fA f2

bm em
m enn cn
... ... ... ...
b2 em
2 en2 c2
b1 em
1 en1 c1

U Fm Fn V

B Em En C
First, we have that
  (∗)  
C
f B
= C
f2 ◦ fA ◦ f1 B

(∗∗)      
= C
f2 En En
fA Em Em
f1 B

(∗∗∗)    
= C
f2 En
A Em
f1 B
,

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 338

where (*) follows from the fact that f = f2 ◦fA ◦f1 ,(**) follows from Theorem 4.5.3(c),
and (***) follows from the fact that A = E fA E . It is now enough to show that
    n m

E
f1 B = Im and C
f2 E = In , for it will then follow that
m n

     
C
f B
= C
f2 En
A Em
f1 B
= In AIm = A,
| {z } | {z }
=In =Im
 
which is what we need. For Em
f1 B
, we compute:

(∗)
  h     i
Em
f1 B
= f1 (b1 ) Em
... f1 (bm ) Em

(∗∗)
h  i
em em
  
= 1 Em
... m Em

(∗∗∗)
em . . . em
 
= 1 m = Im ,

where (*) follows from Theorem 4.5.1, (**) follows from the construction of f1 , and
(***) follows fromExample  3.2.8(a) (or alternatively, from Proposition 3.2.9). The
computation for C f2 E is similar and is left as an exercise. This proves existence.
n
 
Uniqueness. Suppose that f, g : U → V are linear functions such that C f B = A
 
and C g B = A. We must show that f = g. First of all, note that for all indices
i ∈ {1, . . . , m}, we have that
           
f (bi ) C = C f B bi B = C g B bi B = g(bi ) C ,
| {z } | {z }
=A =A

and consequently, f (bi ) = g(bi ) (because · C : V → Fn is an isomorphism

 

and therefore one-to-one). But now since B = {b1 , . . . , bm } is a basis of U and

f, g : U → V are linear, the uniqueness part of Theorem 4.3.2 guarantees that
f = g.

Remark: Suppose that U and V are non-trivial, finite-dimensional vector spaces

over a field F, and recall from subsection 4.1.2 that Hom(U, V ), the set of all linear
functions from U to V , is a vector space over the field F (vector addition and scalar
multiplication in this vector space are the usual addition and scalar multiplication
of functions). Set m := dim(U ) and n := dim(V ), and let B = {b1 , . . . , bm } and
C = {c1 , . . . ,cn } be bases of U and V , respectively. By Theorem 4.5.1 and Proposi-
tion 4.5.8, C · B : Hom(U, V ) → Fn×m is a bijection,37 and by Theorem 4.5.3(a-b),
37
The fact that C · B : Hom(U, V ) → Fn×m is a well-defined function follows from Theorem 4.5.1.
 

The fact that this function is one-to-one and onto (i.e. a bijection) follows from Proposition 4.5.8.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 339

: Hom(U, V ) → Fn×m is in fact an isomorphism.

 
it is also a linear function. So, C
· B
 
By Theorem 4.2.14(c), it follows that dim Hom(U, V ) = dim(Fn×m ) = nm.

4.5.1 Change of basis (transition) matrices

Recall that for any set X, the function IdX : X → X is defined by IdX (x) = x for
all x ∈ X; the function IdX is called the identity function on X.

 vector space V over a field F, and bases B

Given a non-trivial, finite-dimensional

and C of V , we call the matrix C IdV B the change of basis matrix from B to C or
the transition matrix from B to C.
Proposition 4.5.9. Let V be a non-trivial, finite-dimensional vector space over
 1 , . . . ,bn } and C = {c1 , . . . , cn } be bases of V . Then the
a field F, and let B = {b
change of basis matrix C IdV B satisfies the property that
     
C
IdV B v B = v C.

for all vectors v ∈ V . Moreover, this matrix is given by the formula

       
C
IdV B = b1 C . . . bn C .

Proof. The first statement follows straight from the definition of a change of basis
matrix; indeed, for all vectors v ∈ V , we have that
       
C
IdV B v B = IdV (v) C = v C.

For the second statement, we observe that

  (∗)      
C
IdV B
= IdV (b1 ) C
... IdV (bn ) C
     
= b1 C
... bn C

where (*) follows from Theorem 4.5.1.

Proposition 4.5.10. Let V be a non-trivial, finite-dimensional vector space over

a field F, and let B = {b1 , . . . , b n } and C = {c1 , . . . , cn } be bases of V . Then the
change of basis matrices C IdV B and B IdV C are invertible, and moreover, they
are each other’s inverses.
Proof.
 Clearly,
 IdV : V →
 V is an isomorphism, and so by Theorem 4.5.4(f), matrices
C
IdV B
and B
IdV C
are both invertible. Moreover,
(∗) (∗∗)
   −1
Id−1
   
C
IdV B
= C V B
= B
IdV C
,

where (*) follows from the fact that Id−1

V = IdV , and (**) follows from Theo-
rem 4.5.4(g). This completes the argument.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 340

For the special case of Fn (where F is a field), we get a nice formula for change
of basis matrices (see Theorem 4.5.12 below). First, we need a lemma.
Lemma 4.5.11. Let F be a field, let En = {e1 , . . . , en }be the standard basis of Fn ,
and let B = {b1 , . . . , bn } be any basis of Fn . Set B := b1 . . . bn . Then B is
invertible, and moreover,
IdFn E = B −1 .
   
En
IdFn B = B and B n
 
Proof. Let us first prove that E IdFn B = B. In view of the uniqueness part of
n
Theorem 4.5.1, it suffices to show that for all v ∈ Fn , we have that B v B =
 
T
v E . So, fix a vector v ∈ Fn , and set v B = β1 . . . βn
    
, so that
n
v = β1 b1 + · · · + βn bn . Then
 
β1
n
b1 . . . bn  ...  =
    P (∗)  
B v B = βi bi = v = v E ,

n
i=1
βn

where (*) follows from Proposition [Link] This proves that E IdFn B = B. The
 
n
fact that B is invertible and that B IdFn E = B −1 now follows from Proposi-
 
n
tion 4.5.10.

Theorem 4.5.12. Let F be a field, and let B = {b1 , . . . ,bn } and C = {c  1 , . . . , cn }

be two bases of n . Set B := b . . . b and C := c . . . c
 F
 1 n 1 n . Then the
matrix C IdFn B is invertible, and it is given by the formula

IdFn B = C −1 B.
 
C
 
Proof. The fact that C IdFn B is invertible follows from Proposition 4.5.10. To
 
prove that the formula for C IdFn B is correct, we observe that
(∗) (∗∗)
C −1 B,
       
C
IdFn B
= C IdFn ◦ IdFn B
= C
IdFn En En
IdFn B
=

where (*) follows from Theorem 4.5.3, and (**) follows from Lemma 4.5.11.

Proposition 4.5.13 (below) is an immediate corollary of Theorem 4.5.3(c). We

state it as a separate proposition because it is used particularly often for computation.
Proposition 4.5.13. Let U and V be non-trivial, finite-dimensional vector spaces
over a field F, let B1 and B2 be bases of U , let C1 and C2 be bases of V , and let
f : U → V be a linear function. Then
   
C
f B
= C
IdV ◦ f ◦ IdU B
2 2 2 2

     
= C2
IdV C1 C1
f B1 B1
IdU B2
.
38
Alternatively, it follows from Example 3.2.8(a).

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 341

Proof. This follows immediately from Theorem 4.5.3(c).

Let us now return to the linear function f from Example 4.5.2: we would like to
compute the standard matrix of this linear function.
n 1   1 o
Example 4.5.14. Consider the basis B = , of R2 , and consider the
0 1
unique linear function f : R2 → R2 that satisfies the following:
 1   1 
ˆ f = ;
0 0
 1   2 
ˆ f = .
1 2
Compute the standard matrix of the linear function f .
 
  1 0
Solution. In Example 4.5.2, we saw that B f B = . Now, we set B :=
0 2
   
1 1 39 1 −1
, and we compute B −1 = . Then the standard matrix of f is
0 1 0 1
       
E2
f E2
= E2
IdR2 B B
f B B
IdR2 E2
by Proposition 4.5.13

B −1
 
= B B
f B
by Lemma 4.5.11
   
1 1 1 0 1 −1
=
0 1 0 2 0 1
 
1 1
= .
0 2

Optional: Let us check that our answer is correct. Indeed, we have that
      1 
1 1 1 1
ˆ = =f ;
0 2 0 0 0
      1 
1 1 1 2
ˆ = =f .
0 2 1 2 1
So, our answer is correct.
39
So, the columns of B are the vectors of the basis B, arranged from left to right in the order in
which they appear in B.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 342

A characterization of change of basis matrices. We complete this subsection

with Proposition 4.5.15 (below), which essentially states that change of basis matrices
are precisely the invertible matrices.
Proposition 4.5.15. Let F be a field, let A ∈ Fn×n be a matrix, and let V be any
n-dimensional vector space over the field F. Then the following are equivalent:
(a) A is invertible;
 
(b) for all bases B of V , there exists a basis C of V such that A = C
IdV B
;
 
(c) for all bases C of V , there exists a basis B of V such that A = C
IdV B
;
 
(d) there exist bases B and C of V such that A = C IdV B .
Proof. Clearly, it is enough to prove the implications shown in the diagram below.
(a)

(b) (d) (c)

Since V has at least one n-element basis (because dim(V ) = n), we see that (b)
implies (d), and that (c) implies (d).40 Further, by Proposition 4.5.10, (d) implies (a).
It remains to show that (a) implies (b) and (c). We prove the former; the proof of
the latter is similar and is left as an exercise.
So, assume that (a) is true; we must prove (b). Fix any basis B = {b1, . . . , bn }
of V ;41 we must construct a basis C = {c1 , . . . , cn } of V such that A = C IdV B .
Using Proposition
  4.5.8, we let f : V → V be the (unique) linear function such
that A = B f B . Since A is invertible, Theorem 4.5.4(f) guarantees that f is
an isomorphism. Then by Proposition 4.4.1, f −1 : V → V is also an isomorphism.
For each index i ∈ {1, . . . , n}, we set ci := f −1 (bi ). Since f −1 : V → V is an
isomorphism
 −1 and B = {b1 , . . . , bn } is a basis of V , Theorem 4.4.4(c) implies that
f (b1 ), . . . , f −1 (bn ) = {c1, . . . , cn } =: C is also a basis of V .
Now, we claim that A = C IdV B . First, we note that
(∗)
f −1 ◦ f f −1 f −1
      
  
C
IdV B
= C B
= C B B
f B = C B
A,
| {z }
=A

where (*) follows from Theorem 4.5.3(c). It now suffices to show that C f −1 B = In ,
 
 
for it will then immediately follow that A = C IdV B , which is what we need. We
compute:

40
We need the fact that V has at least one n-element basis, since that means that (b) and (c) are
not just “vacuously true” (due to there not being any bases of V ).
41
Since dim(V ) = n, all bases of V have n elements.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 343

(∗)
f −1 f −1 (b1 ) f −1 (bn )
       
C B
= C
... C
     
= c1 C
... cn C

(∗∗)
en1 . . . enn
 
= = In ,

where (*) follows from Theorem 4.5.1, and (**) follows from Proposition 3.2.9. This
proves (b), and we are done.

4.5.2 Similar matrices

Let F be a field. Given matrices A, B ∈ Fn×n , we say that A is similar to B if there
exists an invertible matrix P ∈ Fn×n such that B = P −1 AP . By Proposition 4.5.16
(below), matrix similarity is an equivalence relation on Fn×n .

Proposition 4.5.16. Let F be a field. Then all the following hold:

(a) for all matrices A ∈ Fn×n , A is similar to A;

(b) for all matrices A, B ∈ Fn×n , if A is similar to B, then B is similar to A;

(c) for all matrices A, B, C ∈ Fn×n , if A is similar to B and B is similar to C, then

A is similar to C.

Proof. (a) Fix a matrix A ∈ Fn×n . Then A = In−1 AIn , and it follows that A is
similar to itself.
(b) Fix a matrices A, B ∈ Fn×n , and assume that A is similar to B. Then
there exists an invertible matrix P ∈ Fn×n such that B = P −1 AP . But then
A = P BP −1 = (P −1 )−1 BP −1 , and it follows that B is similar to A.
(c) Fix matrices A, B, C ∈ Fn×n , and assume that A is similar to B and that
B is similar to C. Then there exist invertible matrices P, Q ∈ Fn×n such that
B = P −1 AP and C = Q−1 BQ. But now

C = Q−1 BQ

= Q−1 (P −1 AP )Q

= (Q−1 P −1 )A(P Q)

= (P Q)−1 A(P Q),

and it follows that A is similar to C.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 344

Remark: By Proposition 4.5.16(b), the similarity relation on Fn×n (where F is a

field) is symmetric. Consequently, we may speak of matrices A, B ∈ Fn×n as being
similar or not being similar to each other. In particular, in what follows, we will
often write something like “let A, B ∈ Fn×n be similar matrices.” This means that
A is similar to B and vice versa.

Proposition 4.5.17. Let F be a field, and let A, B ∈ Fn×n be similar matrices, say
B = P −1 AP for some invertible matrix P ∈ Fn×n . Then A is invertible if and only
if B is invertible, and in this case, B −1 = P −1 A−1 P and A−1 = P B −1 P −1 .

Proof. Since B = P −1 AP , we have that A = P BP −1 . Since P and P −1 are

invertible,42 Proposition 1.11.8(e) guarantees that A is invertible if and only if B is
invertible.43 Suppose now that A and B are invertible. Then
(∗) (∗∗)
B −1 = (P −1 AP )−1 = P −1 A−1 (P −1 )−1 = P −1 A−1 P,

where (*) follows from Proposition 1.11.8(e), and (**) follows from Proposition 1.11.8(b).
But now since B −1 = P −1 A−1 P , we immediately get that A−1 = P B −1 P −1 . This
completes the argument.

Proposition 4.5.18. Let F be a field, and let A, B ∈ Fn×n be similar matrices,

say B = P −1 AP for some invertible matrix P ∈ Fn×n . Then for all non-negative
integers m, we have that B m = P −1 Am P , and in particular, Am and B m are similar.
Moreover, if A and B are invertible,44 then we in fact have that B m = P −1 Am P for
all integers m.

Proof. We first prove that B m = P −1 Am P for all non-negative integers m. We

proceed by induction on m. For m = 0, we note that B 0 = In and P −1 A0 P =
P −1 In P = P −1 P = In , and so B 0 = P −1 A0 P . Now, fix a non-negative integer m,
and assume inductively that B m = P −1 Am P . We then have that

B m+1 = BmB

(∗) −1 m −1
= (P
| {zA P})(P
| {zAP})
=B m =B

−1
= P −1 Am (P
|P{z })AP
=In

42
Since P is invertible, Proposition 1.11.8(b) guarantees that P −1 is also invertible, and moreover,
that (P −1 )−1 = P .
43
Indeed, if A is invertible, then we have that P −1 , A, P are all invertible, and so by Proposi-
tion 1.11.8(e), B = P −1 AP is invertible. Similarly, if B is invertible, then we have that P, B, P −1
are all invertible, and so by Proposition 1.11.8(e), A = P BP −1 is invertible.
44
By Proposition 4.5.17, A is invertible if and only if B is invertible.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 345

= P −1 Am AP

= P −1 Am+1 P,

where in (*) we used the induction hypothesis (for the fact that B m = P −1 Am P ),
plus the fact that B = P −1 AP (by hypothesis). This completes the induction.
Assume now that A and B are invertible. By Proposition 4.5.17, we have that
B −1 = P −1 A−1 P . But now by an argument completely analogous to the above,45 we
get that for all non-negative integers m, we have that (B −1 )m = P −1 (A−1 )m P , that
is, B −m = P −1 A−m P . Combined with the above, this implies that B m = P −1 Am P
for all integers m.

Our next theorem essentially states that two n × n matrices are similar if and
only if they represent the same linear function from an n-dimensional vector space
to itself, but possibly with respect to different bases.

Theorem 4.5.19. Let F be a field, let B, C ∈ Fn×n be matrices, and let V be an

n-dimensional vector space over the field F. Then the following are equivalent:

(a) B and C are similar;

 
(b) for all bases B of V and linear functions f : V → V such that B = B
f B
,
 
there exists a basis C of V such that C = C f C ;
 
(c) for all bases C of V and linear functions f : V → V such that C = C
f C
,
 
there exists a basis B of V such that B = B f B ;

(d) there exist

 bases B and C of V and a linear function f : V → V such that
B = B f B and C = C f C .

Proof. Clearly, it is enough to prove the implications shown in the diagram below.
(a)

(b) (d) (c)

But since matrix similarity in Fn×n is symmetric (by Proposition 4.5.16(b)), the
proofs of the implications “(a) =⇒ (b)” and “(a) =⇒ (c)” are completely analogous,
as are the proofs of the implications “(b) =⇒ (d)” and “(c) =⇒ (d).” So, it is enough
to prove the implications shown in the diagram below.
45
We simply replace A and B with A−1 and B −1 , respectively, in the argument above.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 346

(a)

(d) (c)

First, we assume (a) and prove (c). Assume that  C is a basis of V and that
f : V → V is a linear function such that C = C f C . We must show that there
 
exists a basis B of V such that B = B f B . By (a), matrices B and C are similar,
which by definition means that there exists an invertible matrix P ∈ Fn×n such that
B = P −1 CP . Since P is invertible, Proposition 4.5.15 guarantees that there exists a
basis B of V such that P = C IdV B .46 But now we have that
 

B = P −1 CP
   −1    
= C
IdV B C
f C C
IdV B

     
= B
IdV C C
f C C
IdV B
by Proposition 4.5.10
 
= B
IdV ◦ f ◦ IdV B
by Theorem 4.5.3(c)
 
= B
f B
.

This proves (c).

Next, we assume (c) and prove (d). Since V is an n-dimensional vector space,
it has a basis C of size n. Next, by Proposition
  4.5.8, there exists a (unique) linear
function f : V → V such that C = C f C . But then by (c), there exists a basis B
of V such that B = B f B . This proves (d).47
 

Finally, we assume (d) and prove (a). Using (d),  we fix bases B and C of V
and a linear function f : V → V such that B = B f B and C = C f C . Set
P := B IdV C . By Proposition 4.5.10, P is invertible and satisfies P −1 = C IdV B .
   

We now compute:

P −1 BP = C IdV B B f B B IdV C
     

(∗)  
= C
IdV ◦ f ◦ IdV C
46
We are relying on the “(a) =⇒ (c)” implication of Proposition 4.5.15.
47
Remark: The implication “(c) =⇒ (d)” may seem trivial, but in fact it is not! To get this
implication, we need to make
 sure
 that (c) is not just “vacuously true” due to there not existing any
C and f such that C = C f C . The existence of the basis C follows immediately from dimension
 
considerations, but the existence of a linear function f : V → V such that C = C f C only follows
from the not entirely trivial Proposition 4.5.8.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 347

 
= C
f C

= C,

where (*) follows from Theorem 4.5.3(c). So, B and C are similar. This proves (a),
and we are done.

Corollary 4.5.20. Let F be a field, and let B, C ∈ Fn×n be similar matrices. Then
rank(B) = rank(C).

Proof. This follows immediately from the definition of matrix similarity and from
Proposition 3.3.16(c).48 However, let us give a different proof, one relying on
Theorem 4.5.19 (in order to illustrate how Theorem 4.5.19 can be used).
Since B and C are similar, Theorem 4.5.19 guarantees that there exist bases
B and C of Fn and a linear function f : Fn → Fn such that B = B f B and
C = C f C .49 But then
 

     
rank(B) = rank B f B because B = B
f B

= rank(f ) by Theorem 4.5.4(a)

   
= rank C f C by Theorem 4.5.4(a)
 
= rank(C) because C = C
f C
,

and we are done.

4.5.3 Checking the existence and uniqueness of linear functions

with certain specifications: examples with polynomials and
matrices
In this subsection, we give a few examples similar to those from subsection 1.10.4,
except that we do not work with linear functions of the form f : Fm → Fn (where F is
a field), but with linear functions between various other vector spaces (in particular,
our examples involve matrices and polynomials). All examples in this subsection are
of the following general form: we are given vector spaces U and V over some field F
(where at least U is non-trivial and finite-dimensional), and we are asked whether
there exists a linear function f : U → V that maps certain specified vectors from
U to certain specified vectors from V , and if so, whether this linear function f is
48
Details?
49
Here, we are using the “(a) =⇒ (d)” part of Theorem 4.5.19, with V := Fn .

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 348

unique. In some examples, we are further asked to determine various properties of

such a linear function f (for example, we may need to determine whether f is an
isomorphism). In our solutions, we will rely on matrices of linear functions with
respect to the most natural bases of the domain and codomain (natural for the vector
spaces in question, with no regard to the particular linear function f ).

Remark: When working with matrices of linear functions with respect to particular
bases, or when working with coordinate vectors, we must always explicitly specify
the bases that we are working with. If bases are not explicitly specified, then our
proof/solution is at best incomplete, and at worst incorrect.

Example 4.5.21. Consider the following matrices with entries in Z2 :

   
1 0 1 1 0 0
ˆ M1 = ; ˆ M5 = ;
1 0 1 1 0 0
   
0 1 0 1 0 0
ˆ M2 = ; ˆ M6 = ;
1 0 1 0 1 0
   
0 1 0 0 0 0
ˆ M3 = ; ˆ M7 = ;
1 0 0 0 1 0
   
0 0 1 0 0 0
ˆ M4 = ; ˆ M8 = .
0 0 0 0 0 1

Further, consider the following polynomials with coefficients in Z2 :

ˆ p1 (x) = x3 + x2 + x + 1; ˆ p5 (x) = x5 + x2 + 1;

ˆ p2 (x) = x4 + x2 + x + 1; ˆ p6 (x) = x5 + x4 + x2 + x;

ˆ p3 (x) = x5 + x4 + x2 + 1; ˆ p7 (x) = x2 + x;

ˆ p4 (x) = x3 ; ˆ p8 (x) = x5 + x.

(a) Prove that there exists a unique linear function f : Z2×3 2 → P5Z2 that satisfies the
property that f (Mi ) = pi (x) for all indices i ∈ {1, . . . , 8}.


(b) Find rank(f ) and dim Ker(f ) .

(c) Is f one-to-one? Is it onto? Is it an isomorphism?

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 349

(d) Find a formula for the linear function f , that is, fill in the blank in the following:
 a 
1,1 a1,2 a1,3
f =
a2,1 a2,2 a2,3
∀a1,1 , a1,2 , a1,3 , a2,1 , a2,2 , a2,3 ∈ Z2 .

(e) If f is an isomorphism, then find a formula for f −1 , that is, fill in the blank in
the following:
 
f −1 a5 x5 + a4 x4 + a3 x3 + a2 x2 + a1 x + a0 =

∀a0 , a1 , a2 , a3 , a4 , a5 ∈ Z2 .

Solution. In our solution, we will use the basis

n 1 0 0   0 1 0   0 0 1 
M := , , ,
0 0 0 0 0 0 0 0 0
     
0 0 0 0 0 0 0 0 0 o
, ,
1 0 0 0 1 0 0 0 1

of Z2×3 2 3 4 5 of P5 .

2 , and the basis P := 1, x, x , x , x , x Z2

(a) We proceed as in subsection 1.10.4, except that instead of solving for the
standard matrix of our linear function (which does not exist, since we are working
 
with vector spaces of matrices and polynomials), we will solve for the matrix P f M .
We need our linear function f to satisfy f (M  pi (x) for all indices i ∈ {1, . . . , 8},
 i) =
and consequently, our (unknown) matrix P f M should satisfy
     
P
f M
Mi M
= pi (x) P

for all indices i ∈ {1, . . . , 8}. This is equivalent to

             
P
f M
M 1 M
. . . M 8 M
= p 1 (x) P
... p8 (x) P
.
| {z } | {z }
=:M =:P

Here,
 matrices
 M and P can easily be computed (see below), whereas the matrix
P
f M is the unknown that we need to solve for. We proceed as in subsection 1.9.2.
We first take the transpose of both sides of the equation above, and we obtain
   T
MT P f M = PT,
   T
which we solve for P
f M
. We form the matrix

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 350

  T  T 
M1 M
p1 (x) P

MT PT

=
 .. .. 
. .
 
 
 T  T
M8 M
p8 (x) P

 
1 0 1 1 0 1 1 1 1 1 0 0

 0 1 0 1 0 1 1 1 1 0 1 0 


 0 1 0 1 0 0 1 0 1 0 1 1 

 0 0 1 0 0 0 0 0 0 1 0 0 
=  ,

 1 0 0 1 0 0 1 0 1 0 0 1 


 1 0 0 0 1 0 0 1 1 0 1 1 

 0 0 0 0 1 0 0 1 1 0 0 0 
0 0 0 0 0 1 0 1 0 0 0 1

and we row reduce to obtain

 
1 0 0 0 0 0 0 0 0 0 1 1

 0 1 0 0 0 0 0 0 0 0 0 1 


 0 0 1 0 0 0 0 0 0 1 0 0 

   0 0 0 1 0 0 1 0 1 0 1 0 
MT PT

RREF =  .

 0 0 0 0 1 0 0 1 1 0 0 0 


 0 0 0 0 0 1 0 1 0 0 0 1 

 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0
   T
We now read off the (unique) solution for P
f M
:
 
0 0 0 0 1 1

 0 0 0 0 0 1 

   T  0 0 0 1 0 0 
P
f M
=  .

 1 0 1 0 1 0 

 0 1 1 0 0 0 
0 1 0 0 0 1
 
By taking the transpose, we obtain the (unique) solution for the matrix P
f M
:
 
0 0 0 1 0 0

 0 0 0 0 1 1 

   0 0 0 1 1 0 
P
f M
=  

 0 0 1 0 0 0 

 1 0 0 1 0 0 
1 1 0 0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 351

 
The existence and uniqueness of the matrix P f M guarantees the existence and
uniqueness of the linear function f : Z2×3 2 → P5Z2 that satisfies the property that
f (Mi ) = pi (x) for all indices i ∈ {1, . . . , 8}.
 
Remark: In the above, the existence and uniqueness of the matrix P f M implied
the existence and uniqueness of the linear function f with the specifications from
the statement
  of the example. If we had obtained more than one solution for the
matrix P f M , this would have implied that a linear function f with the given
specifications exists,
  but is not unique. On the other hand, if there had been no
solutions for P f M , this would have meant that no linear function f with the
given specifications exists.
 

(b) By row reducing, we see that RREF P f M = I6 . Consequently,

(∗)
   
rank(f ) = rank P
f M
= 6,

where (*) follows from Theorem 4.5.4(a). On
the other hand, by the rank-nullity
theorem, we have that rank(f ) + dim Ker(f ) = dim(Z2×3
2 ), and it follows that

dim Ker(f ) = dim(Z2×3

2 ) − rank(f ) = 6 − 6 = 0.

(c) Since dim Ker(f ) = 0, Theorem 4.2.4 guarantees that f is one-to-one. Since
rank(f ) = 6 = dim(P5Z2 ), Proposition 4.2.6 guarantees that f is onto.50 Since the
linear function f is one-to-one and onto, it is an isomorphism.
 
(d) Using the matrix P f M , we can easily read off the formula for f , as follows.
For a1,1 , a1,2 , a1,3 , a2,1 , a2,2 , a2,3 ∈ Z2 , we compute:
       
a1,1 a1,2 a1,3    a1,1 a1,2 a1,3
f = P f M
a2,1 a2,2 a2,3 P
a2,1 a2,2 a2,3 M
  
0 0 0 1 0 0 a1,1

 0 0 0 0 1 1 
 a1,2 

 0 0 0 1 1 0  a1,3 
=   

 0 0 1 0 0 0 
 a2,1 

 1 0 0 1 0 0  a2,2 
1 1 0 0 0 1 a2,3

50
Alternatively, since the domain and the codomain of the linear function f have the same
finite dimension, and since f is one-to-one, Corollary 4.2.10 guarantees that f is also onto and an
isomorphism.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 352

 
a2,1

 a2,2 + a2,3 

 a2,1 + a2,2 
=  

 a1,3 

 a1,1 + a2,1 
a1,1 + a1,2 + a2,3

(a1,1 + a1,2 + a2,3 )x5 +

   
  +(a1,1 + a2,1 )x4 + a1,3 x3 +  
=  
  +(a2,1 + a2,2 )x2 +
  .
 
+(a2,2 + a2,3 )x + a2,1 P
 
Since · P
is an isomorphism (and in particular, one-to-one), we deduce that

(a1,1 + a1,2 + a2,3 )x5 +



+(a1,1 + a2,1 )x4 + a1,3 x3 +
 a  
1,1 a1,2 a1,3

f =
a2,1 a2,2 a2,3 
 +(a2,1 + a2,2 )x2 +
+(a2,2 + a2,3 )x + a2,1


for all a1,1 , a1,2 , a1,3 , a2,1 , a2,2 , a2,3 ∈ Z2 . This is the formula that we needed.
(e) As we saw in part (c), f is an isomorphism. Let us find a formula for f −1 .
First, we have that
 
1 0 0 0 1 0
 0 1 1 0 1 1 
 
 −1  (∗)
   −1  0 0 0 1 0 0 
M
f P
= P
f M
= 
 1 0
,
 0 0 0 0 

 1 0 1 0 0 0 
1 1 1 0 0 0

where (*) follows from Theorem 4.5.4(g). We now proceed similarly as in part (d).
For all a0 , a1 , a2 , a3 , a4 , a5 ∈ Z2 , we have the following:
h   i
f −1 a5 x5 + a4 x4 + a3 x3 + a2 x2 + a1 x + a0
M

f −1 a5 x5 + a4 x4 + a3 x3 + a2 x2 + a1 x + a0
   
= M P P
  
1 0 0 0 1 0 a0

 0 1 1 0 1 1 
 a1 

 0 0 0 1 0 0  a2 
=   

 1 0 0 0 0 0 
 a3 

 1 0 1 0 0 0  a4 
1 1 1 0 0 0 a5

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 353

 
a0 + a4

 a1 + a2 + a4 + a5 

 a3 
=  

 a0 

 a0 + a2 
a0 + a1 + a2
   
a0 + a4 a1 + a2 + a4 + a5 a3
= .
a0 a0 + a2 a0 + a1 + a2 M
 
Since · M
is an isomorphism (and in particular, one-to-one), it follows that
 
f −1 a5 x5 + a4 x4 + a3 x3 + a2 x2 + a1 x + a0
 
a0 + a4 a1 + a2 + a4 + a5 a3
=
a0 a0 + a2 a0 + a1 + a2

for all a0 , a1 , a2 , a3 , a4 , a5 ∈ Z2 . This is the formula for f −1 that we needed.

Optional: Because it is easy to miscompute, it is a good idea to check our formulas
for f and f −1 . Let us first check our formula for f . For each index i ∈ {1, . . . , 8}, we
compute f (Mi ) using the formula that we obtained in part (d), and we check that we
do indeed get f (Mi ) = pi (x). (If for some i ∈ {1, . . . , 8}, we get that f (Mi ) ̸= pi (x),
it means that we made a mistake somewhere.) Here, we only do the computation
for i = 1 in order to demonstrate the general principle. The rest is similar routine
computation. So, for i = 1, we compute:
 1 0 1 
f (M1 ) = f
1 0 1

= (1 + 0 + 1)x5 + (1 + 1)x4 + 1x3 + (1 + 0)x2 + (0 + 1)x + 1

= x3 + x2 + x + 1 = p1 (x),

which is what we were supposed to get.

We check our formula for f −1 in a similar way. For each index i ∈ {1, . . . , 8}, we
compute f −1 pi (x) using the formula


that we obtained in part (d), and we check
−1 p (x) = M . (If for some i ∈ {1, . . . , 8}, we get that


that we do indeed get f i i
f −1 pi (x) ̸= Mi , it means that we made a mistake somewhere.) Once again, we

only do the computation for i = 1 in order to demonstrate the general principle. The
rest is similar routine computation. So, for i = 1, we compute:

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 354

f −1 p1 (x) f −1 x3 + x2 + x + 1



=
 
1+0 1+1+0+0 1
=
1 1+1 1+1+1
 
1 0 1
= = M1 ,
1 0 1

which is what we were supposed to get. Alternatively, having verified the formula
for f , we can verify the formula for f −1 by checking that f −1 ◦ f = IdZ2×3 and that
2
f ◦ f −1 = IdP5 (using our formulas for f and f −1 ).
Z2

Example 4.5.22. Consider the following polynomials with coefficients in Z2 :

ˆ p1 (x) = 1; ˆ q1 (x) = x4 ;

ˆ p2 (x) = x + 1; ˆ q2 (x) = x3 + x2 ;

ˆ p3 (x) = x2 + x + 1; ˆ q3 (x) = x2 + 1;

ˆ p4 (x) = x3 + x2 + x + 1; ˆ q4 (x) = x.

(a) Prove that there exists a unique linear function f : P3Z2 → PZ2 that satisfies the


property that f pi (x) = qi (x) for all indices i ∈ {1, 2, 3, 4}.


(b) Compute rank(f ) and dim Ker(f ) .

(c) If f one-to-one?

Remark: f is a linear function from a finite-dimensional vector space

to an infinite-dimensional vector space, and so (by Theorem 4.2.14(b))
it is not onto, and therefore, it is not an isomorphism.

(d) Find a formula for the linear function f , that is, fill in the blank in the following:

f a3 x3 + a2 x2 + a1 x + a0 =


∀a0 , a1 , a2 , a3 ∈ Z2 .

Solution. We proceed similarly as in Example 4.5.21. In what follows, we consider

the basis P3 = {1, x, x2 , x3 } for P3Z2 , and the basis P4 = {1, x, x2 , x3 , x4 } for P4Z2 .
(a) Since polynomials q1 (x), q2 (x), q3 (x), q4 (x) are all of degree at most four,
we see that, if the desired linear function f exists, then Im(f ) is a subspace of

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 355

P4Z2 .51 Therefore, we may temporarily consider f as having codomain P4Z2 , which
is a non-trivial,
 finite-dimensional vector space.52 We will now
solve for the matrix
P4
f P3
. We need our linear function f to satisfy f pi (x) = qi (x) for all indices
 
i ∈ {1, 2, 3, 4}, and consequently, our (unknown) matrix P f B should satisfy
4
     
P
f P pi (x) P = qi (x) P
4 3 3 4

for all indices i ∈ {1, 2, 3, 4}. This is equivalent to

  h     i h     i
P4
f P3
p 1 (x) P3
. . . p4 (x) P3 = q 1 (x) P4
... q4 (x) P4 .
| {z } | {z }
=:P =:Q

Here,
 matrices
 P and Q can easily be computed (see below), whereas the matrix
P4
f P is the unknown that we need to solve for. We proceed as in subsection 1.9.2.
3
We take the transpose of both sides, and we obtain
   T
PT P f P = QT ,
4 3
   T
which we solve for P4
f . We form the matrix
P3
  T  T 
p1 (x) P q1 (x) P
3 4

PT QT

= 
 .. .. 
. .

 
 T  T
p4 (x) P q4 (x) P
3 4

 
1 0 0 0 0 0 0 0 1
 1 1 0 0 0 0 1 1 0 
= 
 1
,
1 1 0 1 0 1 0 0 
1 1 1 1 0 1 0 0 0
and we row reduce to obtain
 
1 0 0 0 0 0 0 0 1
   0 1 0 0 0 0 1 1 1 
RREF PT QT = 
 0
.
0 1 0 1 0 0 1 0 
0 0 0 1 1 1 1 0 0
51
Here is a fully rigorous justification. Since polynomials p1 (x), p2 (x), p3 (x), p4 (x) span P3Z2
(because they form a basis of P3Z2 ), we have that
(∗)


 (∗∗)

Im(f ) = Span f p1 (x) , . . . , f p4 (x) = Span q1 (x), . . . , q4 (x) ,


where (*) follows from Corollary 4.2.12, and (**) follows from the fact that f pi (x) = qi (x) for
all i ∈ {1, 2, 3, 4}. Since q1 (x), . . . , q4 (x) ∈ P4Z2 , Theorem 3.1.11 now guarantees that Im(f ) is a
subspace of P4Z2 .
52
This is important, because in order to form a matrix of a linear function (with respect to suitably
chosen bases), we need both the domain and the codomain to be non-trivial, finite-dimensional
vector spaces. As we know, the vector space PZ2 is not finite-dimensional.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 356

   T
We now read off the (unique) solution for P4
f P3
:
 
0 0 0 0 1
   T  0 0 1 1 1 
P4
f P3
= 
 1
.
0 0 1 0 
1 1 1 0 0
 
By taking the transpose, we obtain the (unique) solution for the matrix P4
f P3
:
 
0 0 1 1
 

 0 0 0 1 

P4
f P3
= 
 0 1 0 1 .

 0 1 1 0 
1 1 0 0
 
The existence and uniqueness of the matrix P f P guarantees the existence and
4 3
uniqueness of the linear function f : P3Z2 → PZ2 that satisfies f pi (x) = qi (x) for all

i ∈ {1, 2, 3, 4}.53
(b) By row reducing, we obtain
 
1 0 0 0
   

 0 1 0 0 

RREF P f P =  0 0 1 0 ,
4 3  
 0 0 0 1 
0 0 0 0
   
and so rank P f P = 4. Therefore, by Theorem 4.5.4(a), we have that
4 3
rank(f ) = 4. For the kernel, we compute:
(∗)
dim P3Z2 − rank(f )



dim Ker(f ) = = 4−4 = 0,

where (*) follows from the rank-nullity theorem.

(c) Since dim Ker(f ) = 0, Theorem 4.2.4 guarantees that f is one-to-one.
 
(d) We use our formula for the matrix P f P . For all a0 , a1 , a2 , a3 ∈ Z2 , we
4 3
have the following:

f a3 x3 + a2 x2 + a1 x + a0 P


4

53
Technically,
we proved the existence and uniqueness of the linear function f : P3Z2 → P4Z2 that
satisfies f pi (x) = qi (x) for all i ∈ {1, 2, 3, 4}. But obviously, we can simply expand the codomain
(from P4Z2 to PZ2 ).

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 357

a3 x3 + a2 x2 + a1 x + a0
   
= P4
f P3 P3

   
0 0 1 1   a2 + a3
 0 0 0 1  a0  a3 
   a1   
= 
 0 1 0 1  
  a2  = 
 a1 + a3 

 0 1 1 0   a1 + a2 
a3
1 1 0 0 a0 + a1

(a0 + a1 )x4 + (a1 + a2 )x3 + (a1 + a3 )x2 + a3 x + (a2 + a3 )

 
= P4
.
 
Since · P4
is an isomorphism (and in particular, one-to-one), it follows that

f (a3 x3 + a2 x2 + a1 x + a0 ) = (a0 + a1 )x4 + (a1 + a2 )x3 + (a1 + a3 )x2 +

+a3 x + (a2 + a3 )

for all a0 , a1 , a2 , a3 , a4 ∈ Z2 .
Optional: We can check that our formula

for f is correct by verifying that it indeed
satisfies the property that f pi (x) = qi (x) for all indices i ∈ {1, 2, 3, 4}. Here, we
only compute this for i = 4 in order to demonstrate the general principle. The rest
is similar routine computation. So, for i = 4, we compute:

f p4 (x) = f (x3 + x2 + x + 1)

= (1 + 1)x4 + (1 + 1)x3 + (1 + 1)x2 + 1x + (1 + 1)

= x = q4 (x),

which is what we were supposed to get.

Example 4.5.23. Consider the following matrices with entries in Z2 :

   
1 1 1 1 1
ˆ M1 = ; ˆ N1 = ;
0 1 1 1 1
   
1 0 1 0 1
ˆ M2 = ; ˆ N2 = ;
1 1 1 0 1
   
0 1 1 0 1
ˆ M3 = ; ˆ N3 = ;
1 0 0 1 0
   
1 0 1 1 0
ˆ M4 = ; ˆ N4 = .
0 1 0 1 1

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 358

Determine if there exists a linear function f : Z2×22 → Z2×3

2 such that f (Mi ) = Ni
for all i ∈ {1, 2, 3, 4}. If such a linear function f exists, determine if it is unique,
and if it is not, determine the number of such linear functions f .

Remark: In this particular case, it is not very hard to see that f does not exist.
Indeed, we can see that M3 = M1 + M2 , and so any linear function f : Z2×2
2 → Z2×3
2
satisfying f (M1 ) = N1 and f (M2 ) = N2 must also satisfy
(∗)
f (M3 ) = f (M1 + M2 ) = f (M1 ) + f (M2 ) = N1 + N2 ̸= N3 ,

where (*) follows from the linearity of f . However, we give a solution that illustrates
the general principle, which we can also use in those situations when the non-existence
of the function in question is not quite so obvious (and also when the function with
the given specifications does in fact exist).

Solution. We proceed as in our solution to Example 4.5.21(a). We set

       
1 0 0 1 0 0 0 0
A1 := , A2 := , A3 := , A4 := ,
0 0 0 0 1 0 0 1

and we further set

     
1 0 0 0 1 0 0 0 1
B1 := , B2 := , B3 := ,
0 0 0 0 0 0 0 0 0
     
0 0 0 0 0 0 0 0 0
B4 := , B5 := , B6 := .
1 0 0 0 1 0 0 0 1

In our solution, we will use the basis A := {A1 , A2 , A3 , A4 } of Z2×2 2 and the basis
B := {B1 , B2 , B3 , B4 , B5 , B6 } of Z2×3
2 . Instead of directly solving for the linear
function f : Z2×2
2  → Z
2
2×3
satisfying f (M i ) = Ni for all i ∈ {1, 2, 3, 4}, we will solve
6×4
for the matrix B f A in Z2 satisfying
     
B
f A
Mi A
= Ni B

for all i ∈ {1, 2, 3, 4}. This is equivalent to

             
B
f A M1 A . . . M4 A = N1 A . . . N4 A .
| {z } | {z }
=:M =:N
 
Matrices M and N can easily be computed, whereas the matrix B f A is the
unknown that we need to solve for. We proceed as in subsection 1.9.2. We first take
the transpose of both sides of the equation above, and we obtain
   T
MT B f A = NT ,

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 359

   T
which we solve for B
f A
. We form the matrix
  T  T 
M1 A
N1 B

MT NT

=
 .. .. 
. .
 
 
 T  T
M4 A
N4 B
 
1 1 0 1 1 1 1 1 1 1
 1 0 1 1 1 0 1 1 0 1 
= 
 0
,
1 1 0 1 0 1 0 1 0 
1 0 0 1 1 1 0 0 1 1

and we row reduce to obtain

 
1 0 0 1 0 0 1 0 1 1
   0 1 0 0 0 0 1 1 0 0 
RREF MT NT = 
 0
.
0 1 0 0 1 1 1 1 0 
0 0 0 0 1 1 1 0 0 0
   54
Because of the fourth row of RREF MT NT , we see that the equation
  T   T
MT B f A = N T has no solutions for B f A . Consequently, the equa-
 
   
tion B f A M = N has no solutions for B f A . This implies that there is no
linear function f : Z2×2
2 → Z2×3
2 satisfying f (Mi ) = Ni for all i ∈ {1, 2, 3, 4}.

Example 4.5.24. Consider the following polynomials and vectors, the former with
coefficients in Z3 , and the latter with entries in Z3 :
T
ˆ p1 (x) = x2 + x + 1; ˆ v1 =

0 1 ;
T
ˆ p2 (x) = 2x + 1; ˆ v2 =

2 1 ;
T
ˆ p3 (x) = 2x2 + 1; ˆ v3 =

1 1 ;
T
ˆ p4 (x) = x + 2; ˆ v4 =

1 2 ;
T
ˆ p5 (x) = x2 + 2x; ˆ v5 =

1 0 .

Determine if there exists a linear function f : P2Z3 → Z23 such that f pi (x) = vi for

all i ∈ {1, . . . , 5}. If such a linear function f exists, determine if it is unique, and if
54
 
The point is that the fourth row is of the form 0 ... 0 ∗ ... ∗ , where at least one
of the ∗’s is non-zero.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 360

it is not, determine the number of such linear functions f .

Solution. In our solution, we will use the basis P = {1, x, x2 } of P2Z3 , and the
standard basis E2 = {e1 , e2 } of Z
23 . Instead of directly solving for the linear function
f : P2Z3 → Z23 satisfying f pi (x) = vi for all i ∈ {1, . . . , 5}, we will solve for the
matrix E f P in Z2×3
 
3 satisfying
2

     
E2
f P
pi (x) P
= vi E
| {z }2
=vi

for all i ∈ {1, . . . , 5}. This is equivalent to

         
E2
f P
p 1 (x) P
. . . p 5 (x) P
= v 1 . . . v 5 .
| {z } | {z }
=:P =:M
 
Matrices P and M can easily be computed, whereas the matrix E f P is the
2
unknown that we need to solve for. We take the transpose of both sides of the
equation above, and we obtain
  T
PT = MT ,

E2
f P

   T
which we solve for E2
f P
. We form the matrix
 
  T  1 1 1 0 1
p1 (x) P
v1T  1 2 0 2 1 

PT MT

=
 .. ..  =

1 0 2 1 1

,
. . 
  
  

p5 (x)
T
v5T
 2 1 0 1 2 
P 0 2 1 1 0

and by row reducing, we obtain

 
1 0 2 1 1
 

 0 1 2 2 0 

RREF PT MT = 
 0 0 0 0 0 .

 0 0 0 0 0 
0 0 0 0 0
   T
We can now read off the general solution for E2
f P
:
 
   T t1 + 1 t2 + 1
E2
f P
=  t1 + 2 t2 , where t1 , t2 ∈ Z3 .
t1 t2

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 361

 
By taking the transpose, we obtain the general solution for E2
f P
, as follows:
 
  t1 + 1 t1 + 2 t1
E2
f P
= , where t1 , t2 ∈ Z3 .
t2 + 1 t2 t2
 
Since we got more than one solution for the matrix E f P , we deduce that there
2

exists a linear function f : P2Z3 → Z23 satisfying f pi (x) = vi for all i ∈ {1, . . . , 5},
but that such a linear function f is not unique.
It remains to determine the exact number of linear functions
 f with the given
specifications. Our general solution for the matrix E f P has two parameters,
2  
each of which can take any value from Z3 . So, the number of solutions for E f P
2
is 32 = 9, and consequently, there are exactly nine linear functions f : P2Z3 → Z23


satisfying f pi (x) = vi for all i ∈ {1, . . . , 5}.

Our final example of this subsection (see Example 4.5.25 below) is slightly more
complicated. As we shall see, the linear function satisfying the specifications from
that example is not unique. Moreover, we will need to construct examples of linear
functions satisfying those specifications, and also having all possible ranks (that
is, possible subject to the constraints of the example). Our solution will rely on
Proposition 3.3.12, which states that both elementary row operations and elementary
column operations preserve the rank of a matrix.

Example 4.5.25. Consider the following polynomials with coefficients in Z3 :

ˆ p1 (x) = x3 + 1; ˆ p3 (x) = x2 + 2x + 1

ˆ p2 (x) = 2x3 + 2; ˆ p4 (x) = 2x3 + x2 + 2x.

Further, consider the following matrices with entries understood to be in Z3 :

   
2 0 1 0
ˆ M1 = ; ˆ M3 = ;
0 0 2 2
   
1 0 2 0
ˆ M2 = ; ˆ M4 = .
0 0 2 2

Prove that there exists a linear function f : P3Z3 → Z2×2

3 such that f pi (x) = Mi for
all indices i ∈ {1, 2, 3, 4}. What are all the possible ranks that such a linear function
f can have? For each possible rank, find a formula for one linear function f that
has that rank (and satisfies the specifications above).55 Can f be an isomorphism?
55
There may be more than one correct answer. However, we are not asked to find all possible
correct answers: any one will do.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 362

If so, find a formula for one such isomorphism,56 and also for its inverse.

Solution. In our solution, we will use the basis P = {1, x, x2 , x3 } of P3Z3 , as well as
the basis M = {A1 , A2 , A3 , A4 } of Z2×2
3 , where
       
1 0 0 1 0 0 0 0
A1 = , A2 = , A3 = , A4 = .
0 0 0 0 1 0 0 1

We need to solve for the linear function f : P3Z3 → Z2×2

3 that satisfies f pi (x) =
Mi for all indices i ∈ {1, 2, 3, 4}. Equivalently, we need to solve for the matrix
M
f P in Z4×43 that satisfies
     
M
f P
pi (x) P
= M i M

for all i ∈ {1, 2, 3, 4}. This is equivalent to

             
M
f P
p1 (x) P
. . . p 4 (x) P
= M 1 M
. . . M 4 M
.
| {z } | {z }
=:P =:M
 
Matrices P and M can easily be computed, whereas the matrix M f P is the
unknown that we need to solve for. We take the transpose of both sides of the
equation above, and we obtain
   T
PT M f P = MT ,
   T
which we solve for M
f P
. We form the matrix
  T  T 
p1 (x) P
M1 M

PT MT
 
=  .. .. 
. .

 
 T  T
p4 (x) P
M4 M
 
1 0 0 1 2 0 0 0
 2 0 0 2 1 0 0 0 
= 
 1
.
2 1 0 1 0 2 2 
0 2 1 2 2 0 2 2
By row reducing, we obtain
 
1 0 0 1 2 0 0 0
   0 1 2 1 1 0 1 1 
RREF PT MT = 
 0
.
0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0
56
Once again, there may be more than one correct answer. We are not asked to find all possible
answers: any one will do (provided it exists to begin with).

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 363

   T
We now read off the general solution for M
f P
:
 
2t1 + 2 2t2 2t3 2t4
   T  s1 + 2t1 + 1 s2 + 2t2 s3 + 2t3 + 1 s4 + 2t4 + 1 
M
f P
= 
 s1
,
s2 s3 s4 
t1 t2 t3 t4

where s1 , s2 , s3 , s4 , t1 , t2 , t3, t4 ∈ Z3 . By taking the transpose, we obtain the general

solution for the matrix M f P :
 
2t1 + 2 s1 + 2t1 + 1 s1 t1
   2t2 s2 + 2t2 s2 t2 
M
f P
= 
 2t3
,
s3 + 2t3 + 1 s3 t3 
2t4 s4 + 2t4 + 1 s4 t4

where s1 , s2 , s3 , s4 , t1 , t2 , t3 , t4 ∈ Z3 . In particular, our matrix equation

 
M
f P P = M
 
is consistent, i.e. has a solution for M f P , and we deduce that there exists a linear
function f : P3Z3 → Z2×2


3 that satisfies f pi (x) = Mi for all indices i ∈ {1, 2, 3, 4}.
We
 now examine the rank of f . By Theorem 4.5.4(a), we have that rank(f ) =
  
rank M f P , and by Proposition 3.3.12, elementary row operations and elemen-
tary column operations both preserve the rank of a matrix.  We will now perform
elementary row and column operations on our matrix M f P until we obtain a
matrix in which the leftmost columns are definitely pivot columns (these columns
will contain only numbers and not parameters), the columns in the middle can be
arranged to be pivot columns or not (depending on our choice of the values for the
parameters s1 , s2 , s3 , s4 , t1 , t2 , t3 , t4 ), and the rightmost columns are zero columns
(which are definitely not pivot columns).57 We compute:
   
rank(f ) = rank M f P

 
2t1 + 2 s1 + 2t1 + 1 s1 t1
  2t s2 + 2t2 s2 t2  
2
= rank  2t3

s3 + 2t3 + 1 s3 t3 
2t4 s4 + 2t4 + 1 s4 t4

57
Actually, in this particular example, we will end up with a matrix that has two pivot columns,
and two columns that can be arranged to be pivot columns or not (depending on the values we
assign to our parameters); there will be no zero columns in the end.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 364

 
2t1 + 2 2t1 + 1 s1 t1
  2t 2t2 s2 t2  
C2 →C2 +2C3 2
= rank  2t3

2t3 + 1 s3 t3 
2t4 2t4 + 1 s4 t4
 
2 1 s1 t1
C1 →C1 +C4  0
C2 →C2 +C4 0 s2 t2  
= rank 
 0

1 s3 t3 
0 1 s4 t4
 
1 1 s1 t1
 0 0 s2 t2  
C1 →2C1
= rank 
 0

1 s3 t3 
0 1 s4 t4
 
C2 →C2 +2C1 1 0 0 0
C3 →C3 +2s1 C1  0
C4 →C4 +2t1 C1 0 s2 t2  
= rank 
 0

1 s3 t3 
0 1 s4 t4
 
1 0 0 0
 0 1 s3 t3  
R2 ↔R3
= rank 
 0

0 s2 t2 
0 1 s4 t4
 
1 0 0 0
 0 1 s3 t3 
R4 →R4 +2R2
= rank 
 0

0 s2 t2 
0 0 2s3 + s4 2t3 + t4
 
1 0 0 0
C3 →C3 +2s3 C2  0
C4 →C4 +2t3 C2 1 0 0 
= rank 
 0
 ,
0 s2 t2 
0 0 2s3 + s4 2t3 + t4

where s1 , s2 , s3 , s4 , t1 , t2 , t3 , t4 ∈ Z3 . The last matrix that we obtained is a 4 × 4

matrix, and its first two columns are clearly pivot columns. So, 2 ≤ rank(f ) ≤ 4. It is
now easy to see that f can have three possible ranks: 2, 3, 4. Each of these three values
can easily be obtained by varying the values of the parameters s1 , s2 , s3 , s4 , t1 , t
2 , t3 , t4 .
We now find a formula for one possible linear function f satisfying f pi (x) = Mi
for all i ∈ {1, 2, 3, 4}, and having the specified rank.
Rank 2: By setting, for example, s1 = s2 = s3 = s4 = t1 = t2 = t3 = t4 = 0, we

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 365

obtain rank(f ) = 2.58 In this case, we have

 
2 1 0 0
   0 0 0 0 
M
f P =   0
.
1 0 0 
0 1 0 0

Then for all a0 , a1 , a2 , a3 ∈ Z3 , we get

f a3 x3 + a2 x2 + a1 x + a0 M = a3 x3 + a2 x2 + a1 x + a0

    
M
f P P
  
2 1 0 0 a0
 0 0 0 0   a1 
= 
 0
 
1 0 0   a2 
0 1 0 0 a3
 
2a0 + a1
 0 
=  
 a1 
a1
   
2a0 + a1 0
= .
a1 a1 M
 
Since · M
is an isomorphism (and in particular, one-to-one), we get the formula
 
2a0 + a1 0
x3 x2


f a3 + a2 + a1 x + a0 =
a1 a1

for all a0 , a1 , a2 , a3 ∈ Z3 .
Rank 3: By setting, for example, s2 = 1 and s1 = s3 = s4 = t1 = t2 = t3 = t4 =
0, we obtain rank(f ) = 3. In this case, we have
 
2 1 0 0
   0 1 1 0 
M
f P =   0 1
.
0 0 
0 1 0 0

Then for all a0 , a1 , a2 , a3 ∈ Z3 , we get

58
There are many other possible choices for the parameters s1 , s2 , s3 , s4 , t1 , t2 , t3 , t4 that will also
yield rank(f ) = 2. However, we only need to exhibit one single example, and this is arguably the
simplest one.

Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 366

f a3 x3 + a2 x2 + a1 x + a0 a3 x3 + a2 x2 + a1 x + a0

    
M
= M
f P P
  
2 1 0 0 a0
 0 1 1 0   a1 
= 
 0
 
1 0 0   a2 
0 1 0 0 a3
 
2a0 + a1
 a1 + a2 
=  
 a1 
a1
   
2a0 + a1 a1 + a2
= .
a1 a1 M
 
Since · M
is an isomorphism (and in particular, one-to-one), we get the formula
 
2a0 + a1 a1 + a2
x3 x2


f a3 + a2 + a1 x + a0 =
a1 a1

for all a0 , a1 , a2 , a3 ∈ Z3 .
Rank 4: By setting, for example, s2 = t4 = 1 and s1 = s3 = s4 = t1 = t2 = t3 =
0, we obtain rank(f ) = 4. In this case, we have
 
2 1 0 0
   0 1 1 0 
M
f P =   0 1
.
0 0 
2 0 0 1

Then for all a0 , a1 , a2 , a3 ∈ Z3 , we get

f a3 x3 + a2 x2 + a1 x + a0 M = a3 x3 + a2 x2 + a1 x + a0

    
M
f P P
  
2 1 0 0 a0
 0 1 1 0   a1 
= 
 0
 
1 0 0   a2 
2 0 0 1 a3
 
2a0 + a1
 a1 + a2 
=  
 a1 
2a0 + a3
   
2a0 + a1 a1 + a2
= .
a1 2a0 + a3 M
Irena Penev
Linear Algebra 1 & 2
Chapter 4. Linear functions 367

 
Since · M
is an isomorphism (and in particular, one-to-one), we get the formula
 
2a0 + a1 a1 + a2
x3 x2


f a3 + a2 + a1 x + a0 =
a1 2a0 + a3

for all a0 , a1 , a2 , a3 ∈ Z3 . Finally, we note that our linear function f : P3Z3 → Z2×2
3
(given by the formula that we just obtained) is an isomorphism. Indeed, since
rank(f ) = 4 = dim(Z2×2 3 ), Proposition 4.2.6 guarantees that f is onto. Since
the domain and codomain of f have the same finite dimension, Corollary 4.2.10
guarantees that f is in fact an isomorphism. It remains to compute a formula for
f −1 . First, we note that
 
2 0 1 0
 −1  (∗)
   −1 (∗∗)  0 0 1 0 
P
f M
= M
f P
=   0 1 2 0 ,


2 0 1 1

where (*) follows from Theorem 4.5.4(g), and (**) is obtained via routine computation.
Then for all a, b, c, d ∈ Z3 , we have that
  a b      
−1
 −1  a b
f = P f M
c d P
c d M
  
2 0 1 0 a
 0 0 1 0   b 
= 
 0
 
1 2 0  c 
2 0 1 1 d
 
2a + c
 c 
= 
 b + 2c 


2a + c + d

(2a + c + d)x3 + (b + 2c)x2 + cx + (2a + c)

 
= P
.
 
Since · P
is an isomorphism (and in particular, one-to-one), it follows that
 a b 
f −1 = (2a + c + d)x3 + (b + 2c)x2 + cx + (2a + c)
c d

for all a, b, c, d ∈ Z3 .

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 368

Chapter 5

Affine subspaces and affine

functions

Terminology: So far, we have typically referred to vector/linear subspaces of a

vector space simply as “subspaces.” In this chapter, we will study a generalization of
linear subspaces, called “affine subspaces.” To avoid any confusion, in this chapter,
we will not use the term “subspace” and will instead always write either “linear
subspace” or “affine subspace.” However, in subsequent chapters, we will again use
the term “subspace” to mean “linear subspace.”

5.1 Affine subspaces

An affine subspace of a vector space V over a field F is any set of the form

a+U := {a + u | u ∈ U },

where a is a vector in V and U is a linear subspace of V .

a+U

a U

Thus, an affine subspace of V is obtained by shifting a linear subspace U of V by

some vector a.

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 369

Remark: For a vector space V over a field F:

ˆ every linear subspace U of V is also an affine subspace of V , since U = 0 + U ;1

ˆ V is an affine subspace of itself (because V is a linear subspace of itself);

ˆ for every vector a ∈ V , {a} is an affine subspace of V , since {a} = a + {0} and
{0} is a linear subspace of V .

Geometric considerations. As we know, linear subspaces of Rn are {0}, lines

through the origin, planes through the origin, and higher dimensional generalizations.
So, affine subspaces of Rn are {a} (for any vector a ∈ Rn ), lines, planes, and
higher dimensional generalizations (these lines, planes, and higher dimensional
generalizations may, but need not, pass through the origin).

As Theorem 5.1.1 (below) states, for an affine subspace M = a + U of a vector

space V over a field F (where a is a vector and U a linear subspace of V ), the vector
a need not be unique (indeed, it can be any vector in M ),2 but the linear subspace
U is unique (it depends only on M , and not on the vector a).3

a + U = a′ + U

a a′ U

Theorem 5.1.1. Let V be a vector space over a field F, and let M = a + U be an

affine subspace of V , where a is a vector and U a linear subspace of V . Then all the
following hold:

(a) a ∈ M (and in particular, M ̸= ∅);

(b) for all a′ ∈ M , we have that M = a′ + U ;

(c) for all vectors a′ and linear subspaces U ′ of V such that M = a′ + U ′ , we have
that U ′ = U ;
1
Moreover, as we shall see, linear subspaces of V are precisely those affine subspaces of V that
contain 0 (see Corollary 5.1.2).
2
This follows from Theorem 5.1.1(b)
3
This follows from Theorem 5.1.1(c).

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 370

(d) for all b ∈ V \ M , we have that M ∩ (b + U ) = ∅.

Proof. (a) Since U is a linear subspace of V , Theorem 3.1.7 guarantees that 0 ∈ U ,

and consequently, a = a + 0 ∈ a + U = M .
(b) Fix a′ ∈ M . Since a′ ∈ M = a + U , there exists some u′ ∈ U such that
a′ = a + u′ . Now, we must show that M = a′ + U .
Let us first show that M ⊆ a′ + U . Fix x ∈ M . Since M = a + U , there exists
some u ∈ U such that x = a + u. Then x = a + u = (a′ − u′ ) + u = a′ + (u − u′ ).
Since u, u′ ∈ U , and U is a linear subspace of V , we have that u − u′ ∈ U ; so,
x = a′ + (u − u′ ) ∈ a′ + U . This proves that M ⊆ a′ + U .
Let us now show that a′ + U ⊆ M . Fix u ∈ U ; we must show that a′ + u ∈ M .
But note that a′ + u = a + u′ + u. Since u′ , u ∈ U , and U is a linear subspace of
V , we have that u′ + u ∈ U ; consequently, a′ + u = a + u′ + u ∈ a + U = M . This
proves that a′ + U ⊆ M .
We have now shown that M = a + U , which is what we needed.
(c) Fix a vector a′ and a linear subspace U ′ of V such that M = a′ + U ′ . By (a),
we have that a′ ∈ M , and so by (b), we have that M = a′ + U . So, a′ + U ′ = a′ + U ,
and we deduce that U ′ = U .4
(d) Fix b ∈ V \ M . We must show that M ∩ (b + U ) = ∅. Suppose otherwise,
and fix x ∈ M ∩ (b + U ). Since x ∈ M = a + U , there exists some u1 ∈ U such
that x = a + u1 ; on the other hand, since x ∈ b + U , there exists some u2 ∈ U
such that x = b + u2 . So, a + u1 = b + u2 , and it follows that b = a + (u1 − u2 ).
Since u1 , u2 ∈ U , and since U is a linear subspace of V , we have that u1 − u2 ∈ U ;
consequently, b = a+(u1 −u2 ) ∈ a+U = M , contrary to the fact that b ∈ V \M .

Given a vector space V over a field F, we define the dimension of an affine

subspace M = a + U of V (where a is a vector and U a linear subspace of V ) to be

dim(M ) := dim(U ).

By Theorem 5.1.1(c), this is well defined.

Corollary 5.1.2. Let V be a vector space over a field F. Then linear subspaces of V
are precisely those affine spaces of V that contain 0. In other words, for all U ⊆ V ,
the following are equivalent:

(i) U is a linear subspace of V ;

(ii) U is an affine subspace of V and 0 ∈ U .

This is “obvious,” but here is a formal proof. By symmetry, it suffices to show that U ′ ⊆ U .
4

Fix u′ ∈ U ′ . Then a′ + u′ ∈ a′ + U ′ = a′ + U , and it follows that there exists some u ∈ U such that
a′ + u′ = a′ + u. By subtracting a′ from both sides, we get u′ = u; since u ∈ U , we deduce that
u′ ∈ U . So, U ′ ⊆ U .

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 371

Proof. Fix U ⊆ V . Suppose first that (i) holds. Then 0 ∈ U (by Theorem 3.1.7),
and moreover, U = 0 + U . So, (ii) holds.
Suppose now that (ii) holds. Since U is an affine subspace of V , we know that
there exists a vector a ∈ V and a linear subspace U ′ of V such that U = a + U ′ .
Moreover, by (ii), we have that 0 ∈ U , and so by Theorem 5.1.1(b), we have that
U = 0 + U ′ . So, U = U ′ . Since U ′ is a linear subspace of V , we see that (i) holds.

Recall that the intersection of two linear subspaces is a linear subspace (see
subsection 3.1.3). In the case of affine subspaces, we have the following corollary.
Corollary 5.1.3. Let V be a vector space over a field F, and let M1 and M2 be
affine subspaces of V . Then either M1 ∩ M2 = ∅, or M1 ∩ M2 is an affine subspace
of V .
Proof. We may assume that M1 ∩ M2 ̸= ∅, for otherwise we are done. Fix any
a ∈ M1 ∩ M2 . By Theorem 5.1.1, M1 and M2 can be written as M1 = a + U1 and
M2 = a+U2 , for some linear subspaces U1 and U2 of V . Then U := U1 ∩U2 is a linear
subspace of V (see subsection 3.1.3). Moreover, it is clear that M1 ∩ M2 = a + U ,5
and so M1 ∩ M2 is an affine subspace.

5.2 Affine functions

Suppose that V1 and V2 are vector spaces over a field F. A function f : V1 → V2 is
called an affine function if there exists a linear function g : V1 → V2 and a vector
b ∈ V2 such that for all x ∈ V1 , we have that f (x) = g(x) + b.
Obviously, every linear function f is affine (we simply take g := f and b := 0).
Moreover, we have the following proposition.
Proposition 5.2.1. Let V1 and V2 be vector spaces over a field F, and let f : V1 → V2
be an affine function. Then f is linear if and only if f (0) = 0.
Proof. If f is linear, then Proposition 4.1.6 guarantees that f (0) = 0. For the reverse
implication, we assume that f (0) = 0, and we show that f is linear. Since f is an
affine function, we know that there exists a linear function g : V1 → V2 and a vector
b ∈ V2 such that for all x ∈ V1 , we have that f (x) = g(x) + b. But now
(∗)
0 = f (0) = g(0) + b = 0+b = b
where (*) follows from the fact that g is linear, and so g(0) = 0 (by Proposition 4.1.6).
So, f (x) = g(x) for all x ∈ V1 , that is, f = g. Since g is linear, so is f .
5
This is “obvious,” but here is a full proof. It is clear that a + U ⊆ M1 ∩ M2 . For the reverse
inclusion, we fix some x ∈ M1 ∩ M2 , and we show that x ∈ a + U . Since x ∈ M1 = a + U1 , we know
that there exists some u1 ∈ U1 such that x = a + u1 . Similarly, since x ∈ M2 = a + U2 , there exists
some u2 ∈ U2 such that x = a + u2 . So, a + u1 = a + u2 , and consequently, u1 = u2 . Since u1 ∈ U1
and u2 ∈ U2 , we deduce that u1 = u2 belongs to U1 ∩ U2 = U . But now x = a + u1 ∈ a + U , and
we are done.

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 372

5.2.1 Making new affine functions out of old ones

Theorem 5.2.2 (below) is an analog of Theorem 4.1.7 for affine functions.
Theorem 5.2.2. Let V1 , V2 , V3 be vector spaces over a field F. Then all the following
hold:
(a) for all affine functions f1 , f2 : V1 → V2 , we have that f1 + f2 is an affine function;
(b) for all affine functions f : V1 → V2 and scalars α, we have that αf is an affine
function;
(c) for all affine functions f1 : V1 → V2 and f2 : V2 → V3 , we have that f2 ◦ f1 is an
affine function.
f2 ◦ f1

f1 f2
V1 V2 V3

Proof. We prove (c). The proofs of (a) and (b) are left as an exercise. Fix affine
functions f1 : V1 → V2 and f2 : V2 → V3 . Since f1 : V1 → V2 is an affine function,
there exists a linear function g1 : V1 → V2 and a vector b2 ∈ V2 such that for all
x ∈ V1 , we have that f1 (x) = g1 (x) + b2 . Similarly, since f2 : V2 → V3 is an affine
function, there exists a linear function g2 : V2 → V3 and a vector b3 ∈ V3 such that
for all x ∈ V2 , we have that f2 (x) = g2 (x) + b3 . But now for all x ∈ V1 , we have that


(f2 ◦ f1 )(x) = f2 f1 (x)


= f2 g1 (x) + b2


= g2 g1 (x) + b2 + b3


= g2 g1 (x) + g2 (b2 ) + b3 because g2 is linear


= (g2 ◦ g1 )(x) + g2 (b2 ) + b3 .
Since g1 and g2 are linear, Proposition 4.1.7(c) guarantees that g2 ◦ g1 is linear. On
the other hand, g2 (b2 ) + b3 is a vector in V3 . So, f2 ◦ f1 is an affine function.

5.2.2 The image of an affine subspace under an affine function

By Theorem 4.2.3(a), for a linear function f : U → V (where U and V are vector
spaces over a field F), and for a linear subspace U ′ of U , we have that f [U
′ ] is
a subspace of V , and moreover, by Corollary 4.2.9, we have that dim f [U ′ ] ≤
min dim(U ′ ), dim(V ) . In the case of affine functions and affine subspaces, we have


the following theorem.

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 373

Theorem 5.2.3. Let V1 and V2 be vector spaces over a field F, let f : V1 → V2 be

an affine function given by

f (x) = g(x) + b for all x ∈ V1 ,

where g : V1 → V2 is a linear function and b is a fixed vector in V2 , and let let

M1 = a1 + U1 be an affine subspace of V1 (where a1 is a vector and U1 a linear
subspace of V1 ). Then


f [M1 ] = g(a1 ) + b + g[U1 ],

and consequently, f [M1 ] is an affine subspace of V2 . Moreover,


dim f [M1 ] ≤ min dim(M1 ), dim(V ) .

Proof. First, we have the following:


f [M1 ] = f (x) | x ∈ M1

= f (a1 + u) | u ∈ U1 becuase M1 = a1 + U1

 because f (x) = g(x) + b

= g(a1 + u) + b | u ∈ U1
for all x ∈ V1

= g(a1 ) + g(u) + b | u ∈ U1 because g is linear


= g(a1 ) + b + g(u) | u ∈ U1


= g(a1 ) + b + g[U1 ].

Since g : V1 → V2 is a linear function and U1 is a linear subspace of V1 , The-

orem 4.2.3(a)

guarantees that g[U1 ] is a linear subspace of V2 . So, f [M1 ] =
g(a1 ) + b + g[U1 ] is an affine subspace
of V2 .
It remains to show that dim f [M1 ] ≤ dim(M1 ). For this, we observe the
following:



dim f [M1 ] = dim g(a1 ) + b + g[U1 ]

by definition, since
= dim g[U1 ] g[U1 ] is a linear
subspace of V2

≤ min dim(U1 ), dim(V ) by Corollary 4.2.9

 by definition, since
= min dim(M1 ), dim(V )
Irena Penev
M1 = a1 + U1 .
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 374

This completes the argument.

Corollary 5.2.4. Let V1 and V2 be vector spaces over a field F, and let f : V1 → V2
be an affine function given by

f (x) = g(x) + b for all x ∈ V1 ,

where g : V1 → V2 is a linear function and b is a fixed vector in V2 . Then

Im(f ) = g(a1 ) + b + Im(g),

and consequently, Im(f ) is an affine subspace of V2 . Moreover,


dim Im(f ) = rank(g) ≤ min dim(V1 ), dim(V2 ) .

Proof. Since V1 is a linear subspace of itself, we have the following:

Im(f ) = f [V1 ]

= f [0 + V1 ]


= g(0) + b + g[V1 ] by Theorem 5.2.3

because g is linear
and therefore (by
= b + g[V1 ]
Proposition 4.1.6)
satisfies g(0) = 0

= b + Im(g).

Since g is linear, Theorem 4.2.3(b) guarantees that Im(g) is a linear subspace of V2 ,

and it follows that Im(f ) is an affine subspace of V2 . Finally, we have the following:



dim Im(f ) = dim b + Im(g)

by definition, since
= dim Im(g) Im(g) is a linear
subspace of V2

by definition, since
= rank(g)
g is linear

≤ min dim(V1 ), dim(V2 ) by Corollary 4.2.8.

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 375

This completes the argument.

Geometric considerations. Suppose that f : Rm → Rn is an affine (possibly

linear) function. As we know, affine subspaces of Rm are points (technically, sets
that contain exactly one point), lines, planes, and higher dimensional generalizations.
Theorem 5.2.3 guarantees that f maps every m
n

affine subspace M of R onto an affine
subspace of R , and moreover, dim f [M ] ≤ dim(M ). So, f maps lines onto lines or
points, and it maps planes onto planes, lines, or points. Obvious higher-dimensional
generalizations apply. (Compare these remarks to the discussion in subsection 1.10.2.)

5.2.3 The preimage of an affine subspace under an affine function

By Theorem 4.2.3(c), for any linear function f : U → V (where U and V are vector
spaces over a field F), and any linear subspace V ′ of V , we have that f −1 [V ′ ] is a linear
subspace of the domain U . Theorem 5.2.5 (below) is an analog of Theorem 4.2.3(c)
for affine functions and affine subspaces. Note that it is not quite true that the
preimage of an affine subspace under an affine function must be an affine subspace.
This is because such a preimage may possibly be empty.6 (Note that this never
happens with preimages of linear subspaces under linear functions: such preimages
always contain at least the zero vector.) However, as Theorem 5.2.5 states, if the
preimage of an affine subspace of the codomain under an affine function is not empty,
then that preimage is indeed an affine subspace of the domain, as we would expect.

Theorem 5.2.5. Let V1 and V2 be vector spaces over a field F, and let f : V1 → V2
be an affine function given by

f (x) = g(x) + b for all x ∈ V1 ,

where g : V1 → V2 is a linear function and b is a fixed vector in V2 . Further, let

M2 = a2 + U2 be an affine subspace of V2 (where a2 is a vector and U2 a linear
subspace of V2 ). Then both the following hold:

(a) for all a1 ∈ f −1 [M2 ], we have that f −1 [M2 ] = a1 + g −1 [U2 ];

(b) f −1 [M2 ] is either empty or an affine subspace of V1 .

Remark: If f −1 [M2 ] = ∅ (which is possible), then (a) is vacuously true (since in

this case, there are no vectors a1 in f −1 [M2 ]).
6
For a simple example, consider any two vector spaces U and V over the same field F, and assume
that V is non-trivial. Consider the function f : U → V that maps all elements of U to the zero
vector (i.e. f (u) = 0 for all u ∈ U ). Then f is a linear (and therefore affine) function. Now choose
any vector b ∈ V \ {0}. Then {b} = b + {0} is an affine subspace of the codomain V , and we have
that f −1 [{b}] = ∅.

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 376

Proof. We first prove (a). Fix a1 ∈ f −1 [M2 ]. Then f (a1 ) ∈ M2 , and so by Theo-
rem 5.1.1, we have that M2 = f (a1 ) + U2 for some linear subspace U2 of V2 . We now
have the following:

f −1 [M2 ] = {x ∈ V1 | f (x) ∈ M2 }

= {x ∈ V1 | f (x) ∈ f (a1 ) + U2 }

= {x ∈ V1 | g(x) + b ∈ (g(a1 ) + b) + U2 }

= {x ∈ V1 | g(x) ∈ g(a1 ) + U2 }

= {x ∈ V1 | g(x) − g(a1 ) ∈ U2 }

(∗)
= {x ∈ V1 | g(x − a1 ) ∈ U2 }

(∗∗)
= a1 + {y ∈ V1 | g(y) ∈ U2 }

= a1 + g −1 [U2 ],

where (*) follows from the fact that g is linear, and in (**) we set y = x − a1 .7 This
proves (a).
It remains to prove (b). We may assume that f −1 [M2 ] ̸= ∅, for otherwise
we are done. Fix any a1 ∈ f −1 [M2 ]. Then by (a), we have that f −1 [M2 ] =
a1 + g −1 [U2 ]. Since g : V1 → V2 is a linear function and U2 is a linear subspace
of V2 , Theorem 4.2.3(c) guarantees that g −1 [U2 ] is a linear subspace of V1 . So,
f −1 [M2 ] = a1 + g −1 [U2 ] is an affine subspace of V1 . This proves (b).

Corollary 5.2.6. Let V1 and V2 be vector spaces over a field F, and let f : V1 → V2
be an affine function given by

f (x) = g(x) + b for all x ∈ V1 ,

where g : V1 → V2 is a linear function and b is a fixed vector in V2 . Further, let c be

any vector in V2 . Then both the following hold:

(a) if a ∈ V1 is any solution of the equation f (x) = c,8 then the solution set of the
equation f (x) = c is a + Ker(g);

(b) the solution set of the equation f (x) = c is either empty or an affine subspace of
V1 .
7
We are also using the fact that V1 = {x − a1 | x ∈ V1 }. (Proof?)
8
This simply means that f (a) = c.

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 377

Remark: If the equation f (x) = c has no solutions (which is possible), then (a) is
vacuously true.

Proof. We first prove (a). Suppose that a ∈ V1 is any solution of the equation
f (x) = c; we must show that the solution set of the equation f (x) = c is precisely
the set a + Ker(g). Now, note that the solution set of the equation f (x) = c is
precisely the set f −1 [{c}], and in particular, we have that a ∈ f −1 [{c}]. Moreover,
since {c} = c + {0}, and {0} is a linear subspace of V2 , we have that {c} is an affine
subspace of V2 . But now
We can now apply Theorem 5.2.5, as follows:
(∗) (∗∗)
f −1 [{c}] = a + g −1 [{0}] = a + Ker(g)

where (*) follows from Theorem 5.2.5, and (**) follows from the definition of Ker(g).
This proves (a).
It remains to prove (b). We may assume that the equation f (x) = c is consistent,
for otherwise, its solution set is empty, and we are done. Fix any solution a of
the equation f (x) = c. Then by (a), the solution set of the equation f (x) = c
is a + Ker(g). But by Theorem 4.2.3(d), Ker(g) is a linear subspace of V1 . So,
a + Ker(g) is an affine subspace of V1 . This proves (b).

Corollary 5.2.7. Let F be a field, and let A ∈ Fn×m and b ∈ Fn . Then both the
following hold:
(a) if a is any solution of the matrix-vector equation Ax = b, then the solution set
of Ax = b is a + Nul(A);

(b) if the matrix-vector equation Ax = b is consistent, then its solution set is an

affine subspace of Fm .
Proof. Let fA : Fm → Fn be given by fA (x) = Ax for all x ∈ Fm . By Proposi-
tion 1.10.4, fA is a linear (and therefore affine) function. If a is any solution of
Ax = b, then it is also a solution of fA (x) = b, and by Corollary 5.2.6, the latter is
precisely equal to
(∗)
a + Ker(fA ) = a + Nul(A),
where (*) follows from Proposition 4.2.1(b). This proves (a). Part (b) follows from (a)
and from the fact that Nul(A) is a subspace of Fm (by Proposition 3.3.25).

Let us take a look at a simple example illustrating Corollary 5.2.7. Consider the
the following matrix and vector, with entries understood to be in Z3 :
   
1 2 2 2 1 2
 2 2 0 0 1   2 
A :=   0 2 0 2 0 ,
 b :=   1 .


1 1 2 1 1 0

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 378

Let us solve the matrix-vector equation Ax = b. We form the augmented matrix

 
1 2 2 2 1 2
   2 2 0 0 1 2 
A b =   0 2 0
,
2 0 1 
1 1 2 1 1 0
and by row reducing, we obtain
 
1 0 0 2 2 2
   0 1 0 1 0 2 
RREF A b = 
 0
.
0 1 2 1 1 
0 0 0 0 0 0
We see that the matrix-vector equation Ax = b is consistent, and that the general
solution of this equation is
 
s+t+2
 2s + 2 
 
x =   s + 2t + 1  ,
 where s, t ∈ Z3 .
 s 
t
By separating parameters, we obtain
     
2 1 1
 2   2   0 
     
x =   1  + s 1  + t
   2 ,
 where s, t ∈ Z3 .
 0   1   0 
0 0 1
So, the solution set of the equation Ax = b is
           
2 1 1 2 1 1

n  2  
 2 
 0 
  o 
 2 
 
 2  
  0 

 1  + s  1  + t  2  | s, t ∈ Z3 =  1  + Span  1 , 2  .
           
 0   1   0   0   1   0 
0 0 1 0 0 1
 T
But note that a := 2 2 1 0 0 is one solution of the matrix-vector equation
Ax = b, and that the null space of A is precisely
   
1 1
   0 
 2  

Nul(A) = Span  1 , 2  .
   
 1   0 
0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 379

So, the solution set of Ax = b is precisely a + Nul(A), which is consistent with

Corollary 5.2.7.

Geometric considerations. Suppose that we are given a matrix A ∈ Rn×m and a

vector b ∈ Rn . By Corollary 5.2.7(b), the solution set of the matrix-vector equation
Ax = b is either empty or an affine subspace of Rm , i.e. a point (technically, a set that
contains exactly one point), a line, a plane, or a higher-dimensional generalization in
Rm .

5.3 Affine combinations and affine hulls

Recall from analytic geometry that if x and y are distinct points (vectors) in R2 ,
then the line in R2 that passes through x and y is {tx + (1 − t)y | t ∈ R}. This in
fact holds for all distinct points x and y in Rn (not just R2 ). Affine combinations
are a generalization of this concept.

{tx + (1 − t)y | t ∈ R}

Suppose that x1 , . . . , xn (n ≥ 1) are vectors in a vector space V over a field

F. An affine combination of x1 , . . . , xn is any sum of the form α1 x1 + · · · + αn xn ,
where α1 , . . . , αn ∈ F satisfy α1 + · · · + αn = 1. The set of all affine combinations
of x1 , . . . , xn , denoted Aff(x1 , . . . , xn ), is called the affine hull (or affine span) of
x1 , . . . , xn . So, we have that
n
nP n
P o
Aff(x1 , . . . , xn ) := αi xi | α1 , . . . , αn ∈ F, αi = 1 .
i=1 i=1

Since xi = 0x1 + · · · + 0xi−1 + 1xi + 0xi+1 + · · · + 0xn for all i ∈ {1, . . . , n}, we see that
x1 , . . . , xn ∈ Aff(x1 , . . . , xn ). As Theorem 5.3.1 (below) shows, affine subspaces of V
are precisely those non-empty subsets of V that are closed under affine combinations.
As a corollary (see Corollary 5.3.2), we deduce that all affine hulls are affine subspaces
of V . We note that Theorem 5.3.1 are the affine subspace analogs of Theorems 3.1.7
and 3.1.11(b), respectively.

Theorem 5.3.1. Let V be a vector space over a field F, and let M ⊆ V . Then the
following are equivalent:

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 380

(i) M is an affine subspace of V ;

(ii) M is non-empty and closed under affine combinations, that is, for all vectors
x1 , . . . , xn ∈ M and α1 , . . . , αn ∈ F such that α1 + · · · + αn = 1, we have that
α1 x1 + · · · + αn xn ∈ M .
Proof. Assume first that (i) holds. Let us prove (ii). Set M = a + U , where a is
a vector and U a linear subspace of V , as in the definition of an affine subspace.
By Theorem 5.1.1(a), we have that a ∈ M , and in particular, M ̸= ∅. It remains
to show that M is closed under affine combinations. Fix x1 , . . . , xn ∈ M , and fix
α1 , . . . , αn ∈ F such that α1 + · · · + αn = 1; we must show that α1 x1 + · · · + αn xn
belongs to M . Since x1 , . . . , xn ∈ M = a + U , there exist vectors u1 , . . . , un ∈ U
such that x1 = a + u1 , . . . , xn = a + un . We now have that

α1 x1 + · · · + αn xn = α1 (a + u1 ) + · · · + αn (a + u1 )

= (α1 + · · · + αn )a + (α1 u1 + · · · + αn un )
| {z }
=1

= a + (α1 u1 + · · · + αn un ).
| {z }
:=u

Since u1 , . . . , un ∈ U , and U is a linear subspace of V , we have that u ∈ U . So,

α1 x1 + · · · + αn xn = a + u ∈ a + U = M . This proves (ii).
Conversely, suppose that (ii) holds. We must prove (i). Using the fact that M ̸= ∅,
we fix some a ∈ M . Set U := {x − a | x ∈ M }. Clearly, M = a + U . It remains to
show that U is a linear subspace of V . By Theorem 3.1.7, it suffices to show that
0 ∈ U , and that U is closed under vector addition and scalar multiplication.
First, since a ∈ M , we have that 0 = a − a ∈ U .
Next, fix u1 , u2 ∈ U . We must show that u1 + u2 ∈ U . Since u1 , u2 ∈ U , there
exist x1 , x2 ∈ M such that u1 = x1 − a and u2 = x2 − a. Then
 
u1 + u2 = (x1 − a) + (x2 − a) = 1x1 + 1x2 + (−1)a − a.
| {z }
:=y

Since x1 , x2 , a ∈ M and 1 + 1 + (−1) = 1, and since (ii) holds, we see that y ∈ M .

But now u1 + u2 = y − a ∈ U .
Finally, fix u ∈ U and α ∈ F; we must show that αu ∈ U . Since u ∈ U , we know
that there exists some x ∈ M such that u = x − a. But now
 
αu = α(x − a) = αx + (1 − α)a − a.
| {z }
:=y

Since x, a ∈ M , and since (ii) holds, we have that y = αx + (1 − α)a ∈ M . But now
αu = y − a ∈ U .

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 381

We have now shown that U is a linear subspace of V , and it follows that (i)
holds.

Corollary 5.3.2. Let x1 , . . . , xn (n ≥ 1) be vectors in a vector space V over a field

F. Then M := Aff(x1 , . . . , xn ) is an affine subspace of V .

Proof. Since x1 , . . . , xn ∈ M , we see that M ̸= ∅. In view of Theorem 5.3.1, it now

suffices to show that M is closed under affine combinations. Fix y1 , . . . , ym ∈ M
and α1 , . . . , αm ∈ F such that α1 + · · · + αm = 1. We must show that y :=
α1 y1 + · · · + αm ym belongs to M . Since y1 , . . . , ym ∈ M , we see that for all
i ∈ {1, . . . , m}, yi is an affine combination
Pnof vectors x1 , P
. . . , xn , that is, there exist
scalars βi,1 , . . . , βi,n ∈ F such that yi = j=1 βi,j xj and nj=1 βi,j = 1. But now
m
P m
P n
P  n P
P m
y = α i yi = αi βi,j xj = αi βi,j xj .
i=1 i=1 j=1 j=1 i=1

n}, we set γj := m
P Pn
For each j ∈ {1, . . . , P i=1 αi βi,j . Then y = j=1 γn xj . It now
remains to show that nj=1 γj = 1, for this will imply that y is an affine combination
of x1 , . . . , xn , that is, that y ∈ M , which is what we need to show. We compute:
n
P n P
P m m
P n
P  (∗) m
P
γj = αi βi,j = αi βi,j = αi = 1,
j=1 j=1 i=1 i=1 j=1 i=1
Pn
where (*) follows from the fact that j=1 βi,j = 1. This completes the argument.

Corollary 5.3.3. Let V be a vector space over a field F, let M be an affine subspace
of V , and let x1 , . . . , xn (n ≥ 1) be vectors in V . Then the following are equivalent:

(i) M = Aff(x1 , . . . , xn );

(ii) x1 , . . . , xn ∈ M , and every vector in M is an affine combination of x1 , . . . , xn .

Proof. Obviously, (i) implies (ii). For the reverse implication, we assume that (ii)
holds, and we prove (i). Since every vector in M is an affine combination of
x1 , . . . , xn , we have that M ⊆ Aff(x1 , . . . , xn ). Let us prove the reverse inclusion.
Fix x ∈ Aff(x1 , . . . , xn ). By (ii), we have that x1 , . . . , xn ∈ M , and by Theorem 5.3.1,
we know that M is closed under affine combinations. Since x is an affine combination
of x1 , . . . , xn , we deduce that x ∈ M . This proves that Aff(x1 , . . . , xn ) ⊆ M . Thus, (i)
holds.

5.4 Affine frames and affine bases

We have extensively studied bases of (finite-dimensional) vector spaces. For affine
subspaces, we have two analogues of bases: “affine frames” and “affine bases.”

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 382

5.4.1 Affine frames

Let n be a non-negative integer, and let M be an n-dimensional affine subspace of
a vector space V over a field F. An affine frame of M is an ordered (n + 1)-tuple
(a, u1 , . . . , un ) of vectors of V such that M can be written in the form M = a + U ,
where U is a linear subspace of V ,9 and {u1 , . . . , un } is a basis of U .

Remark: Suppose that M is an affine subspace of a vector space V over a field F,

and that a, u1 , . . . , un ∈ V . It then follows from the definition that (a, u1 , . . . , un ) is
an affine frame of M if and only if vectors u1 , . . . , un are linearly independent and
M = a + Span(u1 , . . . , un ).10

Remark: Infinite-dimensional affine subspaces do not have affine frames.

By Theorem 3.2.7, if {v1 , . . . , vn } is a basis of a vector space V over a field

F, then every vector in V can be written as a linear combination of the vectors
v1 , . . . , vn in a unique way. Our next theorem is an analogue of this result for affine
subspaces and affine frames.

Theorem 5.4.1. Let M be an affine subspace of a vector space V over a field F, and
let (a, u1 , . . . , un ) be an affine frame of M . Then for all x ∈ M , there exist unique
scalars α1 , . . . , αn ∈ F such that x = a + α1 u1 + · · · + αn un .

Proof. Set U := Span(u1 , . . . , un ), so that M = a + U . Fix x ∈ M . We must show

that there exist unique scalars α1 , . . . , αn ∈ F such that x = a + α1 u1 + · · · + αn un .
We first prove existence. Since x ∈ M = a + U , there exists some u ∈ U such
that x = a + u. Since u ∈ U = Span(u1 , . . . , un ), we know that there exist scalars
α1 , . . . , αn such that u = α1 u1 + · · · + αn un . So, x = a + α1 u1 + · · · + αn un . This
proves existence.
Let us prove uniqueness. Fix scalars α1 , . . . , αn , β1 , . . . , βn ∈ F such that x =
a + α1 u1 + · · · + αn un and x = a + β1 u1 + · · · + βn un . Then a + α1 u1 + · · · + αn un =
a+β1 u1 +· · ·+βn un , and consequently, (α1 −β1 )u1 +· · ·+(αn −βn )un = 0. Since the
set {u1 , . . . , un } is linearly independent,11 we have that α1 − β1 = · · · = αn − βn = 0.
It follows that αi = βi for all i ∈ {1, . . . , n}. This proves uniqueness.
9
By Theorem 5.1.1(c), the linear subspace U is unique, i.e. it depends only on M , and not on
the choice of a.
10
This is “obvious,” but here is a formal proof. Suppose first that vectors u1 , . . . , un are linearly
independent and M = a + Span(u1 , . . . , un ). Then {u1 , . . . , un } is a basis of U := Span(u1 , . . . , un ),
and we deduce that (a, u1 , . . . , un ) is an affine frame of M = a + U . Suppose conversely that
(a, u1 , . . . , un ) is an affine frame of M . Then there exists some linear subspace U of V such that
M = a + U and such that {u1 , . . . , un } is a basis of U . But then vectors u1 , . . . , un are linearly
independent and U = Span(u1 , . . . , un ), and consequently, M = a + U = a + Span(u1 , . . . , un ).
11
This follows from the fact that (a, u1 , . . . , un ) is an affine frame of M = a + U , and consequently,
{u1 , . . . , un } is a basis of U .

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 383

5.4.2 Affine independence

Given vectors x1 , . . . , xn in a vector space V over a field F, we say that vectors
x1 , . . . , xn ∈ V are affinely independent, or that the set {x1 , . . . , xn } is affinely
independent, if for all α1 , . . . , αn ∈ F such that
α 1 x1 + · · · + α n xn = 0 and α1 + · · · + αn = 0,
we have that α1 = · · · = αn = 0.
Proposition 5.4.2. Let V be a vector space over a field F, and let x0 , x1 , . . . , xn
(n ≥ 0) be vectors in V . Then the following are equivalent:
(i) x0 , x1 , . . . , xn are affinely independent;
(ii) there exists some i ∈ {0, 1, . . . , n} such that vectors
x0 − xi , . . . , xi−1 − xi , xi+1 − xi , . . . , xn − xi
are linearly independent;
(iii) for all i ∈ {0, 1, . . . , n}, vectors
x0 − xi , . . . , xi−1 − xi , xi+1 − xi , . . . , xn − xi
are linearly independent.
Proof. Obviously, (iii) implies (ii). We will show that (ii) implies (i), and that (i)
implies (iii).
Suppose that (ii) holds. Let us prove (i). By (ii) and by symmetry, we may assume
that x1 −x0 , . . . , xn −x0 are linearly independent. Now, fix scalars α0 , α1 , . . . , αn ∈ F
such that α0 x0 + α1 x1 + · · · + αn xn = 0 and α0 + α1 + · · · + αn = 0. We must
show that α0 = α1 = · · · = αn = 0. Since α0 + α1 + · · · + αn = 0, we have that
α0 = −α1 − · · · − αn , and so
0 = α0 x0 + α1 x1 + · · · + αn xn

= (−α1 − · · · − αn )x0 + α1 x1 + · · · + αn xn

= α1 (x1 − x0 ) + · · · + αn (xn − x0 )
Since vectors x1 − x0 , . . . , xn − x0 are linearly independent, we see that α1 = · · · =
αn = 0. Since α0 = −α1 − · · · − αn , it follows that α0 = 0. This proves (i).
Suppose now that (i) holds. Let us prove (iii). By symmetry, it suffices to show
that x1 − x0 , . . . , xn − x0 are linearly independent. Fix scalars α1 , . . . , αn ∈ F such
that α1 (x1 − x0 ) + · · · + αn (xn − x0 ) = 0. Then
(−α1 − · · · − αn )x0 + α1 x1 + · · · + αn xn = 0.
| {z }
:=α0

Since x0 , x1 , . . . , xn are affinely independent, we now get that α0 = α1 = · · · = αn = 0,

and we deduce that (iii) holds.

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 384

5.4.3 Affine bases

Let M be an affine subspace of a vector space V over a field F. An affine basis
(also called a barycentric frame) of M is a non-empty ordered set {x0 , x1 , . . . , xn } of
vectors in M such that
ˆ vectors x0 , x1 , . . . , xn are affinely independent;
ˆ M = Aff(x0 , x1 , . . . , xn ).
As we shall see (see Theorem 5.4.4 and the remark following it), for any non-negative
integer n, every affine basis of an n-dimensional affine subspace contains exactly
n + 1 vectors.
Remark: Suppose that M is an affine subspace of a vector space V over a field
F, and let x0 , x1 , . . . , xn be vectors in V . In view of Corollary 5.3.3, we have that
{x0 , x1 , . . . , xn } is an affine basis of M if and only if all the following hold:
1. x0 , x1 , . . . , xn ∈ M ;
2. vectors x0 , x1 , . . . , xn are affinely independent;
3. every vector in M can be expressed as an affine combination of x0 , x1 , . . . , xn .
Recall that by Theorem 3.2.7, if {v1 , . . . , vn } is a basis of a vector space V over a
field F, then every vector in V can be written as a linear combination of the vectors
v1 , . . . , vn in a unique way. Theorem 5.4.1 was an analogue of this result for affine
frames. Theorem 5.4.3 (below) is an analogue of that same result for affine bases.
Theorem 5.4.3. Let M be an affine subspace of a vector space V over a field F,
and let {x0 , x1 , . . . , xn } be an affine basis of M . Then for all x ∈ M , there exist
unique scalars α0 , α1 , . . . , αn ∈ F, called the barycentric
P coordinates Pof x with respect
to the affine basis {x0 , x1 , . . . , xn }, such that x = ni=0 αi xi and ni=0 αi = 1.
Proof.
Pn Fix x ∈PM . The existence of scalars α0 , α1 , . . . , αn ∈ F such that x =
n
i=0 i xi and
α i=0 αi = 1 follows from the fact that M = Aff(x0 , x1 , . . . , xn ).
It remains to prove uniqueness. So, fix α0 , α1 , . . . , αn , β0 , β1 , . . . , βn ∈ F such
that
ˆ x = ni=0 αi xi and ni=0 αi = 1;
P P

ˆ x = ni=0 βi xi and ni=0 βi = 1.

P P

Then ni=0 αi xi = ni=0 βi xi , and we deduce that ni=0 (αi − βi )xi = 0. On the
P P P
other hand,
Pn Pn  P n 
(αi − βi ) = αi − βi = 1 − 1 = 0.
i=0 i=0 i=0
Since vectors x0 , x1 , . . . , xn are affinely independent, we now deduce that α0 − β0 =
α1 − β1 = · · · = αn − βn = 0. Therefore, αi = βi for all i ∈ {0, 1, . . . , n}. This proves
uniqueness.

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 385

5.4.4 A relationship between affine bases and affine frames

Theorem 5.4.4. Let V be a vector space over a field F, let M be an affine subspace
of V , and let x0 , x1 , . . . , xn ∈ V . Then the following are equivalent:
(i) {x0 , x1 , . . . , xn } is an affine basis of M ;
(ii) (x0 , x1 − x0 , . . . , xn − x0 ) is an affine frame of M .
Remark: Since every affine frame of an n-dimensional affine subspace contains n + 1
vectors, Theorem 5.4.4 implies that every affine basis of an n-dimensional affine
subspace contains exactly n + 1 vectors.

Proof. First, we know that (i) and (ii) are, respectively, equivalent to (1) and (2)
below:
(1) vectors x0 , x1 , . . . , xn are affinely independent and

M = Aff(x0 , x1 , . . . , xn );

(2) vectors x1 − x0 , . . . , xn − x0 are linearly independent and

M = x0 + Span(x1 − x0 , . . . , xn − x0 ).

So, it suffices to show that (1) and (2) are equivalent. By Proposition 5.4.2, vectors
x0 , x1 , . . . , xn are affinely independent if and only if vectors x1 − x0 , . . . , xn − x0
are linearly independent. It now remains to show that Aff(x0 , x1 , . . . , xn ) = x0 +
Span(x1 − x0 , . . . , xn − x0 ). For this, we compute:
Aff(x0 , x1 , . . . , xn )

= α0 x0 + α1 x1 + · · · + αn xn | α0 , α1 , . . . , αn ∈ F, α0 + α1 + · · · + αn = 1

= (1 − α1 − · · · − αn )x0 + α1 x1 + · · · + αn xn | α1 , . . . , αn ∈ F

= x0 + α1 (x1 − x0 ) + · · · + αn (xn − x0 ) | α1 , . . . , αn ∈ F

= x0 + Span(x1 − x0 , . . . , xn − x0 ).
This completes the argument.

Remark: If M = {a} is a one-element affine subspace of a vector space V over a

field F, then (a) is the (unique) affine frame and {a} the (unique) affine basis of M .

Remark: Suppose we are given a matrix A ∈ Fn×m and a vector b ∈ Fn (where F

is a field). By Corollary 5.2.7, the solution set of the matrix-vector equation Ax = b
is either empty or an affine subspace of Fm . Moreover, we have the following:

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 386

ˆ if the matrix-vector equation Ax = b is inconsistent, then its solution set is

empty, and consequently, it is not and affine subspace of Fm and therefore
does not have an affine frame or an affine basis;

ˆ if the matrix-vector equation Ax = b has a unique solution, say x0 , then {x0 }

is the solution set of Ax = b, and we see that (x0 ) is the (unique) affine frame
and {x0 } the (unique) affine basis of the solution set of Ax = b;

ˆ if the matrix-vector equation Ax = b has more than one solution, then an

affine frame and an affine basis of the solution set of Ax = b can be computed
by following the procedure from the solution of Example 5.4.5 (below).

Example 5.4.5. Consider the following matrix and vector, both with entries in Z2 :
   
1 0 0 1 1 1 1
 1 1 1 0 0 1   1 
A=  0 1
, b =  .
1 1 1 0   0 
1 0 1 0 1 0 0

Show that the matrix-vector equation Ax = b is consistent, and consequently (by

Corollary 5.2.7(b)), an affine subspace of Z62 . Find an affine frame and an affine
basis of the solution set of Ax = b.

Solution. We form the augmented matrix

 
1 0 0 1 1 1 1
   1 1 1 0 0 1 1 
A b =  0 1
,
1 1 1 0 0 
1 0 1 0 1 0 0

and by row reducing, we obtain

 
1 0 0 1 1 1 1
   0 1 0 0 1 1 1 
RREF A b = 
 0
.
0 1 1 0 1 1 
0 0 0 0 0 0 0

So, the general solution of the matrix-vector equation Ax = b is

 
r+s+t+1
 s+t+1 
 
 r+t+1 
x =    where r, s, t ∈ Z2 .
 r 

 s 
t

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 387

In particular, the matrix-vector equation Ax = b is consistent, and so by Corol-

lary 5.2.7(b), the solution set of this equation is an affine subspace of Z62 .
Now, the solution set of the matrix-vector equation Ax = b is the following (color
coding is for emphasis):
 
r+s+t+1
 s+t+1 
n
 r+t+1

 o
S :=   | r, s, t ∈ Z2

 r 

 s 
t
       
1 1 1 1
 1   0   1   1 
n
 1



 1



 0



 1

 o
=   + r  + s  + t  | r, s, t ∈ Z2

 0 


 1 


 0 


 0 

 0   0   1   0 
0 0 0 1
       
1 1 1 1

 1 


 0  
  1  
  1 

 1    1   0   1 
=   + Span  ,  ,   .

 0 


 1  
  0  
  0 

 0   0   1   0 
0 0 0 1

We now see that        

1 1 1 1
 1   0   1   1 

 1
 
  1
 
  0
 
  1


 ,  ,  ,  

 0  
  1  
  0  
  0 

 0   0   1   0 
0 0 0 1
is an affine frame of the solution set S of Ax = b, whereas (by Theorem 5.4.4)
             
1 1 1 1 1 1 1
 1   0   1   1   1   1   1 
n
 1
 
  1
 
  1
 
  0
 
  1
 
  1
 
  1
o

 ,  + ,  + ,  + 

 0  
  1  
  0  
  0  
  0  
  0  
  0 

 0   0   0   1   0   0   0 
0 0 0 0 0 1 0

Irena Penev
Linear Algebra 1 & 2
Chapter 5. Affine subspaces and affine functions 388

       
1 0 0 0
 1   1   0   0 
n
 1
 
  0
 
  1
 
  0
o

=  , , , 

 0  
  1  
  0  
  0 

 0   0   1   0 
0 0 0 1

is an affine basis of S.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 389

Chapter 6

Scalar (inner) products, norms,

and orthogonality

6.1 The scalar product

So far, we have worked with vector spaces over arbitrary fields F. In this chapter, we
impose some additional structure on vector spaces, namely the “scalar product” (also
called “inner product”) and the “norm.” A scalar product is a way of multiplying
two vectors and obtaining a scalar. A norm is a way of measuring the distance of
a vector from the origin, or alternatively, measuring the length of a vector. As a
trade-off for imposing this additional structure, we restrict ourselves to vector spaces
over only two fields: R and C. The theory that we develop in this chapter would not
work for vector spaces over general fields F.
Terminology: Vector spaces over R are called real vector spaces, and vector spaces
over C are called complex vector spaces.

6.1.1 The scalar product in real vector spaces

A scalar product (also called inner product) in a real vector space V is a function
⟨·, ·⟩ : V × V → R that satisfies the following four axioms:

r.1. for all x ∈ V , ⟨x, x⟩ ≥ 0, and equality holds if and only if x = 0;

r.2. for all x, y, z ∈ V , ⟨x + y, z⟩ = ⟨x, z⟩ + ⟨y, z⟩;

r.3. for all x, y ∈ V and α ∈ R, ⟨αx, y⟩ = α⟨x, y⟩;

r.4. for all x, y ∈ V , ⟨x, y⟩ = ⟨y, x⟩.

The name “scalar product” comes from the fact that we multiply two vectors
and obtain a scalar as a result.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 390

Note that axioms r.2 and r.3 from the definition above guarantee that the scalar
product in a real vector space V is linear in the first variable (when we keep the
second variable fixed). But in fact, axioms r.2, r.3, and r.4 guarantee that it is linear
in the second variable as well (when we keep the first variable fixed). More precisely,
we have the following:
r.2’. for all x, y, z ∈ V , ⟨x, y + z⟩ = ⟨x, y⟩ + ⟨x, z⟩;
r.3’. for all x, y ∈ V and α ∈ R, ⟨x, αy⟩ = α⟨x, y⟩.
To see that r.2’ holds, note that for all x, y, z ∈ V , we have the following:
r.4 r.2 r.4
⟨x, y + z⟩ = ⟨y + z, x⟩ = ⟨y, x⟩ + ⟨z, x⟩ = ⟨x, y⟩ + ⟨x, z⟩.
On the other hand, to see that r.3’ holds, note that for all x, y ∈ V and α ∈ R, we
have the following:
r.4 r.3 r.4
⟨x, αy⟩ = ⟨αy, x⟩ = α⟨y, x⟩ = α⟨x, y⟩.

The standard scalar product in Rn . Perhaps the best known example of a

scalar product is the “standard scalar product” (sometimes also called the “dot
T
product”) in Rn . The standard scalar product of vectors x = x1 . . . xn

and
 T n
y = y1 . . . y n in R is given by
n
P
x · y := x i yi .
i=1

(By Proposition 6.1.1, this really is a scalar product in Rn .) For example, for vectors
T T
in R3 , we compute:
 
1 −2 5 and −3 2 1
   
1 −3
 −2  ·  2  = 1 · (−3) + (−2) · 2 + 5 · 1 = −2.
5 1
T T
in Rn , we
 
Note that for vectors x = x1 . . . xn and y = y1 . . . yn
have that

y1  n 
 . 
xT y =
 P  
x1 .
. . . xn  .  = x y
i i = x·y .
i=1
yn
So, if we identify 1 × 1 matrices with scalars, then we simply get that

x · y = xT y

for all x, y ∈ Rn .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 391

Proposition 6.1.1. The standard scalar product in Rn is a scalar product.

Proof. We need to check that the standard scalar product · in Rn satisfies the four
axioms from the definition of a scalar product in a real vector space.
T
in Rn , we have that

r.1. For a vector x = x1 . . . xn
n (∗)
x2i
P
x·x = ≥ 0,
i=1

and (*) is an equality if and only if x1 = · · · = xn = 0, i.e. if and only if x = 0.

 T  T  T
r.2. For vectors x = x1 . . . xn , y = y1 . . . yn , and z = z1 . . . zn
in Rn , we have that
n
P
(x + y) · z = (xi + yi )zi
i=1

n
P n
 P 
= xi zi + yi zi
i=1 i=1

= x · z + y · z.
T T
in Rn and a
 
r.3. For vectors x = x1 . . . xn and y = y1 . . . yn
scalar α ∈ R, we have that
n
P n
P
(αx) · y = (αxi )yi = α xi yi = α(x · y).
i=1 i=1

T T
in Rn , we have
 
r.4. For vectors x = x1 . . . xn and y = y1 . . . yn
that
Pn Pn
x·y = x i yi = yi xi = y · x.
i=1 i=1

This proves that the standard scalar product in Rn really is a scalar product.

We note that a similar  type  of scalar product can be defined for matrices.
n×m

Indeed, for matrices A = ai,j n×m and B = bi,j n×m in R , we can define
n Pm
aij bij . It is easy to verify that this really is a scalar product in Rn×m
P
⟨A, B⟩ =
i=1 j=1
(the proof is similar to that of Proposition 6.1.1).

Remark: The standard scalar product is only one of many possible scalar products in
Rn . A full characterization of all possible scalar products in Rn (and more generally,
in all non-trivial, finite-dimensional real vector spaces) is given by Theorem 10.4.1.

An example with integrals. For readers who have studied calculus, Proposi-
tion 6.1.2 (below) gives an example of a scalar product involving integrals.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 392

Proposition 6.1.2. Let a, b ∈ R be such that a < b, and let C[a,b] be the (real)
vector space of all continuous functions from the closed interval [a, b] to R.1 Then
the function ⟨·, ·⟩ : C[a,b] × C[a,b] → R defined by

Rb
⟨f, g⟩ := f (x)g(x)dx
a

for all f, g ∈ C[a,b] is a scalar product.

Proof. We must verify that the four axioms from the definition of a scalar product
are satisfied. We first prove that axioms r.2, r.3, and r.4 are satisfied, and then we
prove that axiom r.1 is satisfied (our proof of r.1 relies on r.3 and r.4, which is why
we prove r.1 last).
r.2. For f1 , f2 , f3 ∈ C[a,b] , we have that

Rb  
⟨f1 + f2 , f3 ⟩ = f1 (x) + f2 (x) f3 (x)dx
a

Rb  
= f1 (x)f3 (x) + f2 (x)f3 (x) dx
a

Rb Rb
= f1 (x)f3 (x)dx + f2 (x)f3 (x)dx
a a

= ⟨f1 , f3 ⟩ + ⟨f2 , f3 ⟩.

r.3. For f1 , f2 ∈ C[a,b] and α ∈ R, we have that

Rb  
⟨αf1 , f2 ⟩ = αf1 (x) f2 (x)dx
a

Rb
= α f1 (x)f2 (x)dx
a

= α⟨f1 , f2 ⟩.

r.4. For f1 , f2 ∈ C[a,b] , we have that

Rb Rb
⟨f1 , f2 ⟩ = f1 (x)f2 (x)dx = f2 (x)f1 (x)dx = ⟨f2 , f1 ⟩.
a a
1
Recall from calculus that all such functions are integrable.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 393

r.1. Let f ∈ C[a,b] . Then

Rb (∗)
⟨f, f ⟩ = f (x)2 dx ≥ 0,
a

where (*) follows from the fact that f (x)2 ≥ 0 for all x ∈ [a, b]. If f (x) = 0 for all
x ∈ [a, b], then obviously, ⟨f, f ⟩ = 0. Suppose now that there exists some x0 ∈ [a, b]
such that f (x0 ) ̸= 0. We must show that ⟨f, f ⟩ > 0.
Suppose first that f (x0 ) > 0. Set m = f (x2 0 ) . (Clearly, m > 0.) Then since f
is continuous on [a, b], there exist a0 , b0 ∈ R such that a ≤ a0 ≤ x0 ≤ b0 ≤ b and
a0 < b0 , and such that for all x ∈ [a0 , b0 ], we have that f (x) ≥ m.2 We now compute:

Rb
⟨f, f ⟩ = f (x)2 dx
a

Ra0 Rb0 Rb
= f (x)2 dx + f (x)2 dx + f (x)2 dx
a a0 b0

(∗) Rb0
≥ f (x)2 dx
a0

(∗∗)
≥ m2 (b0 − a0 )

> 0,

where (*) follows from the fact that f (x)2 ≥ 0 for all x ∈ [a, a0 ] ∪ [b0 , b], and (**)
follows from the fact that f (x)2 ≥ m2 for all x ∈ [a0 , b0 ].
Suppose now that f (x0 ) < 0. Then −f (x0 ) > 0. So, by an argument completely
analogous to the above (applied to −f instead of f ), we obtain ⟨−f, −f ⟩ > 0. We
now use axioms r.3 and r.4 (which we have already verified) to obtain the following:
r.3 r.4 r.3
⟨−f, −f ⟩ = −⟨f, −f ⟩ = −⟨−f, f ⟩ = ⟨f, f ⟩.

Since ⟨−f, −f ⟩ > 0, it follows that ⟨f, f ⟩ > 0.

2
This is essentially because, by the continuity of f , we have that “x ≈ x0 =⇒ f (x) ≈ f (x0 ).”
So, since f (x0 ) > m, there exists some (sufficiently small) subinterval [a0 , b0 ] of [a, b] such that
x0 ∈ [a0 , b0 ] and such that for all x ∈ [a0 , b0 ], we have that f (x) ≥ m. Here is a formal proof, for
those readers who would like one. Set ε := m = f (x2 0 ) . By the continuity of f , there existssome δ > 0
such that for all x ∈ [a, b], if |x − x0 | < δ, then |f (x) − f (x0 )| < ε. Now, set a0 := max a, x0 − 2δ
and b0 := min b, x0 + 2δ . Then a ≤ a0 ≤ x0 ≤ b0 ≤ b and a0 < b0 . Moreover, by construction,


for all x ∈ [a0 , b0 ], we have that |x − x0 | ≤ 2δ < δ, and consequently, |f (x) − f (x0 )| < ε, i.e.
f (x0 ) − ε < f (x) < f (x0 ) + ε. Since f (x0 ) = 2m and ε = m, we deduce that for all x ∈ [a0 , b0 ], we
have that m < f (x) < 3m, and in particular, f (x) ≥ m.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 394

6.1.2 The scalar product in complex vector spaces

Recall from section 0.3 that the complex conjugate of a complex number z = a + bi
(where a, b ∈ R) is defined to be the number z := a − bi.

A scalar product (also called inner product) in a complex vector space V is a

function ⟨·, ·⟩ : V × V → C that satisfies the following four axioms:

c.1. for all x ∈ V , ⟨x, x⟩ is a real number, ⟨x, x⟩ ≥ 0 , and equality holds if and
only if x = 0;

c.2. for all x, y, z ∈ V , ⟨x + y, z⟩ = ⟨x, z⟩ + ⟨y, z⟩;

c.3. for all x, y ∈ V and α ∈ C, ⟨αx, y⟩ = α⟨x, y⟩;

c.4. for all x, y ∈ V , ⟨x, y⟩ = ⟨y, x⟩.

Note that axioms c.2 and c.3 from the definition above guarantee that the scalar
product in a complex vector space V is linear in the first variable (when we keep the
second variable fixed). Unlike in the real case, it is not linear in the second variable
(when we keep the first variable fixed). We do, however, have the following:

c.2’. for all x, y, z ∈ V , ⟨x, y + z⟩ = ⟨x, y⟩ + ⟨x, z⟩;

c.3’. for all x, y ∈ V and α ∈ C, ⟨x, αy⟩ = α⟨x, y⟩.

To see that c.2’ holds, note that for all x, y, z ∈ V , we have the following:
c.4
⟨x, y + z⟩ = ⟨y + z, x⟩

c.2
= ⟨y, x⟩ + ⟨z, x⟩

= ⟨y, x⟩ + ⟨z, x⟩

c.4
= ⟨x, y⟩ + ⟨x, z⟩.

On the other hand, to see that c.3’ holds, note that for all x, y ∈ V and α ∈ C, we
have the following:
c.4 c.3 c.4
⟨x, αy⟩ = ⟨αy, x⟩ = α⟨y, x⟩ = α ⟨y, x⟩ = α⟨x, y⟩.

The standard scalar product in Cn . The standard scalar product of vectors

T T
in Cn is given by
 
x = x1 . . . xn and y = y1 . . . yn
n
P
x · y := x i yi .
i=1

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 395

(By Proposition 6.1.3, this really is a scalar product in Cn .) For example, for vectors
T T
in C2 , we compute:
 
1 − 2i −2 + i and 2 + i 1 + 3i
   
1 − 2i 2+i
· = (1 − 2i)(2 + i) + (−2 + i)(1 + 3i)
−2 + i 1 + 3i

= (1 − 2i)(2 − i) + (−2 + i)(1 − 3i)

= 1 + 2i.
Proposition 6.1.3. The standard scalar product in Cn is a scalar product.
Proof. We need to check that the standard scalar product · in C satisfies the four
axioms from the definition of a scalar product in a complex vector space.
c.1. For a vector x = x1 . . . xn in Cn , we have that
 

n (∗) n (∗∗)
|xi |2
P P
x·x = xi xi = ≥ 0,
i=1 i=1

where (*) follows from Proposition 0.3.2. Moreover, note that the inequality (**) is
an equality if and only if x1 = · · · = xn = 0, i.e. if and only if x = 0.
 T  T  T
c.2. For vectors x = x1 . . . xn , y = y1 . . . yn , and z = z1 . . . zn
in Cn , we have that
n
P
(x + y) · z = (xi + yi )zi
i=1

n
P  n
P 
= x i zi + yi zi
i=1 i=1

= x · z + y · z.
T T
in Cn and a
 
c.3. For vectors x = x1 . . . xn and y = y1 . . . yn
scalar α ∈ C, we have that
n
P n
P
(αx) · y = (αxi )yi = α xi yi = α(x · y).
i=1 i=1

T T
in Cn , we have
 
c.4. For vectors x = x1 . . . x n and y = y1 . . . y n
that
n
P n
P n
P n
P
x·y = xi yi = x i yi = x i yi = yi xi = y · x.
i=1 i=1 i=1 i=1

This proves that the standard scalar product in Cn really is a scalar product.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 396

6.1.3 Orthogonality
Given a real or complex vector space V , equipped with a scalar product ⟨·, ·⟩, we say
that vectors x and y in V are orthogonal, and we write x ⊥ y, if ⟨x, y⟩ = 0. When
our scalar product is the standard scalar product in Rn , this corresponds to the
usual geometric interpretation (a detailed explanation is given in subsection 6.2.1).
However, for general scalar products, this is how we define orthogonality.3

Proposition 6.1.4. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩. Then all the following hold:

(a) for all vectors x, y ∈ V , we have that x ⊥ y if and only if y ⊥ x;

(b) for all vectors x, y ∈ V and scalars α, β,4 if x ⊥ y then (αx) ⊥ (βy);

(c) for all vectors x ∈ V , we have that x ⊥ 0 and 0 ⊥ x.

Proof. We prove the proposition for the case when V is a complex vector space. The
real case is similar but slightly easier (because we do not have to deal with complex
conjugates).
(a) For vectors x, y ∈ V , we have the following sequence of equivalences:

x⊥y ⇐⇒ ⟨x, y⟩ = 0 by definition

⇐⇒ ⟨y, x⟩ = 0 by c.4

⇐⇒ ⟨y, x⟩ = 0

⇐⇒ y⊥x by definition.

(b) Fix vectors x, y ∈ V and scalars α, β ∈ C, and assume that x ⊥ y. Then we

compute:
⟨αx, βy⟩ = α⟨x, βy⟩ by c.3

= αβ⟨x, y⟩ by c.3’

= αβ0 beause x ⊥ y

= 0.
3
For example, for the scalar product defined on C[−π,π] in Proposition 6.1.2, we have that
Rπ
sin x ⊥ cos x, since sin x, cos x = sin x cos xdx = 0.
−π
4
Here, α and β are real or complex numbers, depending on whether V is a real or complex vector
space.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 397

So, (αx) ⊥ (βy).

(c) Fix any vector x ∈ V . We then have that
c.3
⟨0, x⟩ = ⟨00, x⟩ = 0⟨0, x⟩ = 0,

and so 0 ⊥ x. The fact that x ⊥ 0 now follows from (a).

Suppose that V is a real or complex vector space, equipped with a scalar product
⟨·, ·⟩. For a vector v ∈ V and a set of vectors A ⊆ V ,5 we say that v is orthogonal
to A, and we write v ⊥ A, provided that v is orthogonal to all vectors in A.6 For
sets of vectors A, B ⊆ V ,7 we say that A is orthogonal to B, and we write A ⊥ B, if
every vector in A is orthogonal to every vector in B.

Proposition 6.1.5. Let V be a real or complex vector space, equipped with a

scalar product ⟨·, ·⟩. Let a1 , . . . , ap , b1 , . . . , bq ∈ V , and assume that {a1 , . . . , ap } ⊥
{b1 , . . . , bq }. Then Span(a1 , . . . , ap ) ⊥ Span(b1 , . . . , bq ).

Proof. Fix a ∈ Span(a1 , . . . , ap ) and b ∈ Span(b1 , . . . , bq ). Then there exist scalars

α1 , . . . , αp , β1 , . . . , βq such that a = α1 a1 + · · · + αp ap and b = β1 b1 + · · · + βq bq .
We now compute:
p
DP q
P E
⟨a, b⟩ = αi ai , βj bj
i=1 j=1

p D
P q
P E
= αi ai , βj bj by r.2 or c.2
i=1 j=1

p P
P q
= ⟨αi ai , βj bj ⟩ by r.2’ or c.2’
i=1 j=1 | {z }
(∗)
=0

= 0,

where (*) follows from Proposition 6.1.4(b) and from the fact that {a1 , . . . , ap } ⊥
{b1 , . . . , bq }. This proves that a ⊥ b, and the result follows.

6.2 The norm

In this section, we introduce the notion of a “norm” || · || in a real or complex
vector space V . The idea is that for a vector x ∈ V , ||x|| is the distance from x to
5
A may, but need not be, a subspace of V .
6
By definition, this means that for all a ∈ A, we have that ⟨v, a⟩ = 0.
7
Again, A and B may or may not be subspaces of V .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 398

the origin, or alternatively, the length of the vector x; ||x|| is always supposed to
be a non-negative real number (even if V is a complex vector space). For vectors
x, y ∈ V , ||x − y|| is supposed to be the distance between x and y. Distance can
be defined in a variety of ways. We first study norms induced by a scalar product
(see subsections 6.2.1 and 6.2.2). The definition of a norm in general is given in
subsection 6.2.3, and some additional examples of norms are given in subsection 6.2.4.

6.2.1 The norm induced by a scalar product

Given a scalar product ⟨·, ·⟩ in a real or complex vector space V , we define the norm
in V induced by ⟨·, ·⟩ to be the function || · || : V → R given by
p
||x|| := ⟨x, x⟩

for all x ∈ V . In view of r.1 and c.1, for all x ∈ V , we have that ||x|| is a non-negative
real number,8 and moreover, ||x|| = 0 if and only if x = 0.

Proposition 6.2.1. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩ and the induced norm || · ||. Then for all vectors x ∈ V and scalars α,9
we have that
||αx|| = |α| ||x||.

Proof. If the vector space V is real, then for all vectors x ∈ V and scalars α ∈ R, we
have that
p (∗) p p
||αx|| = ⟨αx, αx⟩ = α2 ⟨x, x⟩ = |α| ⟨x, x⟩ = |α| ||x||,

where (*) follows from r.3 and r.3’.

On the other hand, if the vector space V is complex, then for all vectors x ∈ V
and scalars α ∈ C, we have that
p
||αx|| = ⟨αx, αx⟩
p
= αα⟨x, x⟩ by c.3 and c.3’
p
= |α|2 ⟨x, x⟩ by Proposition 0.3.2
p
= |α| ⟨x, x⟩

= |α| ||x||.

This completes the argument.

8
This happens even if V is a complex vector space.
9
So, α is a real or complex number, depending on whether the vector space V is real or complex.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 399

Note that if || · || is the norm induced by the standard scalar product in Rn ,

T
in Rn , we have that

then for all vectors x = x1 . . . xn
s
√ n
x2i .
P
||x|| = x·x =
i=1

So, we simply get the standard Euclidean length in Rn . We note that if x =

T T
are non-zero vectors in Rn , then we have
 
x1 . . . x n and y = y1 . . . yn
that
x · y = ||x|| ||y|| cos θ,
where θ is the angle between x and y. To see this, consider the triangle formed by x,
y, and z := x − y,10 and let θ be the angle between x and y in this triangle.

y z=x−y

θ
x

We then compute
||z||2 = z · z

= (x − y) · (x − y)

= x · x −x · y − y · x + y · y
|{z} | {z }
=||x||2 =||y||2

= ||x||2 + ||y||2 − 2x · y
On the other hand, the Law of Cosines (for triangles) tells us that
||z||2 = ||x||2 + ||y||2 − 2||x|| ||y|| cos θ.
So, ||x||2 + ||y||2 − 2x · y = ||x||2 + ||y||2 − 2||x|| ||y|| cos θ, and consequently,
x · y = ||x|| ||y|| cos θ,
as we had claimed. Note that this means that non-zero vectors x, y ∈ Rn are
orthogonal in the usual geometric sense (i.e. the angle between them is 90◦ ) if and
only if x · y = 0.11
10
This triangle may possibly be “degenerate” (i.e. one-dimensional). This happens if x and y are
scalar multiples of each other.
11
This is because for an angle θ, with 0◦ ≤ θ ≤ 180◦ , we have that cos θ = 0 if and only if θ = 90◦ .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 400

Warning: The formula x · y = ||x|| ||y|| cos θ that we obtained above only works
for the standard scalar product in Rn and the norm induced by it. Do not attempt
to use it for general scalar products!

6.2.2 The Pythagorean theorem, the Cauchy–Schwarz inequality,

and the triangle inequality
The Pythagorean theorem. Let V be a real or complex vector space, equipped
with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Then for all x, y ∈ V such
that x ⊥ y, we have that

||x + y||2 = ||x||2 + ||y||2 .

Proof. Fix x, y ∈ V such that x ⊥ y. Then ⟨x, y⟩ = 0 and ⟨y, x⟩ = 0. So,

||x + y||2 = ⟨x + y, x + y⟩

= ⟨x, x⟩ + ⟨x, y⟩ + ⟨y, x⟩ + ⟨y, y⟩

| {z } | {z } | {z } | {z }
=||x||2 =0 =0 =||y||2

= ||x||2 + ||y||2 ,

which is what we needed to show.

The Cauchy–Schwarz inequality. Let V be a real or complex vector space, equipped

with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Then

|⟨x, y⟩| ≤ ||x|| ||y||

for all x, y ∈ V .

Proof. Fix x, y ∈ V . We may assume that ⟨x, y⟩ = ̸ 0, for otherwise, the result is
immediate. Note that this implies that x, y ̸= 0, and consequently, ||x||, ||y|| =
̸ 0.
We set
z := ⟨y,y⟩
⟨x,y⟩ x − y,

and we compute
(∗)
⟨z, y⟩ = ⟨ ⟨y,y⟩
⟨x,y⟩ x − y, y⟩ = ⟨y,y⟩
⟨x,y⟩ ⟨x, y⟩ − ⟨y, y⟩ = 0,

where (*) follows from r.2 and r.3 if V is a real vector space, or from c.2 and c.3
if V is a complex vector space. We have now shown that z ⊥ y, and so by the
Pythagorean theorem, we have that

||z + y||2 = ||z||2 + ||y||2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 401

⟨y,y⟩
But by construction, z + y = ⟨x,y⟩ x, and consequently:

(∗) ||y||2
||z + y|| = || ⟨y,y⟩
⟨x,y⟩ x|| = | ⟨y,y⟩
⟨x,y⟩ | ||x|| =
|⟨y,y⟩|
|⟨x,y⟩| ||x|| = |⟨x,y⟩| ||x||,

where (*) follows from Proposition 6.2.1. So,

||y||4
|⟨x,y⟩|2
||x||2 = ||z + y||2 = ||z||2 + ||y||2 ≥ ||y||2 ,

which yields
||y||4
|⟨x,y⟩|2
||x||2 ≥ ||y||2 .
|⟨x,y⟩|2
Since ⟨x, y⟩ and ||y|| are both non-zero, we have that ||y||2
is defined and positive.
|⟨x,y⟩|2
So, we may multiply both sides of the inequality above by ||y||2 to obtain

||x||2 ||y||2 ≥ |⟨x, y⟩|2 .

By taking the square root of both sides, we get

||x|| ||y|| ≥ |⟨x, y⟩|,

which is what we needed to show.

Corollary 6.2.2. For all x1 , . . . , xn , y1 , . . . , yn ∈ R, we have that

n
P 2 n
P n
 P 
xi yi ≤ x2i yi2 .
i=1 i=1 i=1

Proof. If we consider the standard scalar product in Rn , the Cauchy-Schwarz in-

equality yields s s
n n n
2 yi2 .
P P P
x i yi ≤ xi
i=1 i=1 i=1

for all x1 , . . . , xn , y1 , . . . , yn ∈ R. By squaring both sides, we obtain the desired

inequality.

As a further corollary of the Cauchy-Schwarz inequality, we obtain the following.

The triangle inequality. Let V be a real or complex vector space, equipped with a
scalar product ⟨·, ·⟩ and the induced norm || · ||. Then

||x + y|| ≤ ||x|| + ||y||

for all x, y ∈ V .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 402

y
x

x+y

Proof. We prove the result for the case when V is a complex vector space. The
real case is similar but slightly easier (because we do not have to deal with complex
conjugates). We first remark that for all complex numbers z = a+ib (where a, b ∈ R),
we have that
ˆ z + z = 2a = 2Re(z);
√
ˆ Re(z) = a ≤ |a| ≤ a2 + b2 = |z|.

Now, fix x, y ∈ V . Then we have the following:

||x + y||2 = ⟨x + y, x + y⟩

(∗)
= ⟨x, x⟩ +⟨x, y⟩ + ⟨y, x⟩ + ⟨y, y⟩
| {z } | {z }
=||x||2 =||y||2

= ||x||2 + ||y||2 + ⟨x, y⟩ + ⟨y, x⟩

(∗∗)
= ||x||2 + ||y||2 + ⟨x, y⟩ + ⟨x, y⟩

||x||2 + ||y||2 + 2Re ⟨x, y⟩

=

≤ ||x||2 + ||y||2 + 2|⟨x, y⟩|

(∗∗∗)
≤ ||x||2 + ||y||2 + 2||x|| ||y||

= (||x|| + ||y||)2 ,

where (*) follows from c.2 and c.2’, (**) follows from c.4, and (***) follows from the
Cauchy-Schwarz inequality. By taking the square root of both sides, we obtain

||x + y|| ≤ ||x|| + ||y||,

which is what we needed to show.

6.2.3 The norm in general

A norm in a real or complex vector space V is a function || · || : V → R that satisfies
the following three axioms:

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 403

n.1. for all vectors x ∈ V , we have that ||x|| ≥ 0, and equality holds if and only if
x = 0;

n.2. for all vectors x ∈ V and scalars α,12 we have that ||αx|| = |α| ||x||;

n.3. for all vectors x, y ∈ V , we have that ||x + y|| ≤ ||x|| + ||y||.

As stated at the beginning of section 6.2, a norm in a real or complex vector space
V gives a way of measuring the distance of a vector from the origin, or equivalently,
measuring the length of a vector. The norm of a vector is always a non-negative
real number (regardless of whether the vector space is real or complex). We note
that n.3 is referred to as the “triangle inequality.” The idea is that vectors x, y, and
x + y form a triangle (see the picture below), and the length of the third side cannot
be greater than the sum of lengths of the other two sides.

y
x

x+y

It follows from the results of subsection 6.2.1 that any norm induced by a scalar
product in a real or complex vector space V really is a norm, i.e. it is a function
from V to R that satisfies axioms n.1, n.2, and n.3 above. The fact that axiom n.1 is
satisfied is immediate from the construction of a norm induced by a scalar product,
the fact that n.2 is satisfied follows from Proposition 6.2.1, and the fact that n.3 is
satisfied follows from the triangle inequality proven in subsection 6.2.2.

Unit vectors. Suppose that V is a real or complex vector space, equipped with a
norm || · ||. A vector v ∈ V is called a unit vector if ||v|| = 1. (In view of n.1, any
unit vector is, in particular, a non-zero vector.) For notational convenience, given a
vector v and a scalar α ̸= 0, we often write αv instead of α−1 v or α1 v. In particular,
v
for a non-zero vector v ∈ V , we may write ||v|| (as in Proposition 6.2.3 below).

Proposition 6.2.3. Let V be a real or complex vector space, equipped with a norm
v
|| · ||. Then for all non-zero vectors v ∈ V , we have that ||v|| > 0 and that || ||v|| || = 1,
v
and in particular, ||v|| is a unit vector.

Proof. Fix a non-zero vector v ∈ V . By n.1, we have that ||v|| > 0. We further have
that
v n.2 1 (∗) 1
|| ||v|| || = | ||v|| | ||v|| = ||v|| ||v|| = 1,
where (*) follows from the fact that ||v|| > 0. This completes the argument.
12
So, α is a real or complex number, depending on whether the vector space V is real or complex.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 404

Terminology/Remark: Suppose that V is a real or complex vector space, equipped

with a norm || · ||. To normalize a non-zero vector v in V means to multiply that
1
vector by ||v|| (“divide by its length”). By Proposition 6.2.3, when we normalize a
non-zero vector, we produce a unit vector.

6.2.4 Other examples of norms

For a positive integer p, we define the p-norm in Rn , denoted by || · ||p , by setting
n
P 1
p
||x||p := |xi |p
i=1
T
in Rn . We omit the proof of the fact that this really is

for all x = x1 . . . xn
n
a norm in R . We do note, however, that for p = 2, we get
s
n
x2i ,
P
||x||2 =
i=1

which is precisely the norm induced by the standard scalar product in Rn , i.e. the
standard Euclidean norm in Rn . For p = 1, we get
n
P
||x||1 = |xi |.
i=1

We note that the || · ||1 norm is sometimes called the “Manhattan norm.” This is
because streets and avenues in Manhattan form a perfect grid (more or less), and so
|| · ||1 gives the actual walking distance between two places in Manhattan (see the
picture below).

Another norm of interest is the so called “Chebyshev distance” in Rn , denoted by

|| · ||∞ . It is defined by

||x||∞ := max |x1 |, . . . , |xn |

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 405

T
in Rn .13

for all vectors x = x1 . . . xn
The unit disk in a real or complex vector space V , equipped with a norm || · ||, is
the set
{x ∈ V | ||x|| ≤ 1}.
The unit disks in R2 with respect to the norms || · ||2 , || · ||1 , and || · ||∞ are represented
in the picture below.
x2 x2 x2
1 1 1

−1 1 x1 −1 1 x1 −1 1 x1

−1 −1 −1

unit disk in R2 unit disk in R2 unit disk in R2

w.r.t. || · ||2 w.r.t. || · ||1 w.r.t. || · ||∞

Finally, if you have studied calculus, recall that for a, b ∈ R such that a < b,
C[a,b] is the (real) vector space of all continuous functions from [a, b] to R. For a real
number p ≥ 1, we have the norm || · ||p on C[a,b] given by
 Rb 1
p
||f ||p = |f (x)|p
a

for all f ∈ C[a,b] , and we also have the norm || · ||∞ on C[a,b] given by
||f ||∞ = max |f (x)|
x∈[a,b]

for all f ∈ C[a,b] . Once again, we omit the proof of the fact that || · ||p (for a real
number p ≥ 1) and || · ||∞ really are norms in C[a,b] .

6.3 Orthogonal and orthonormal bases. Gram-Schmidt

orthogonalization
6.3.1 Vector projection
Suppose we are given a real or complex vector space V , equipped with a scalar
product ⟨·, ·⟩. For a non-zero vector u ∈ V and any vector v ∈ V , the orthogonal
projection of v onto u is the vector
⟨v,u⟩
proju (v) := ⟨u,u⟩ u.
13
It is not hard to check that || · ||∞ really is a norm in Rn , i.e. that it satisfies axioms n.1, n.2,
and n.3. The details are left as an exercise.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 406

Since u ̸= 0, r.1 or c.1 guarantees that ⟨u, u⟩ > 0, and so the expression above is
well defined (that is, we are not dividing by zero). As we can see, proju (v) is a scalar
multiple of u.

proju(v) u

As the picture above suggests, for any scalar α ̸= 0, the projection of v onto αu is
the same as the projection of v onto u. Indeed, if V is a complex vector space, then
we have that
⟨v,αu⟩
projαu (v) = ⟨αu,αu⟩ (αu) by definition

⟨v,αu⟩
= α⟨u,αu⟩ (αu) by c.3

α⟨v,u⟩
= αα⟨u,u⟩ (αu) by c.3’

⟨v,u⟩
= ⟨u,u⟩ u

= proju (v) by definition.

If V is a real vector space, then the proof is similar to the above, except that we use
r.3 and r.3’ instead of c.3 and c.3’, respectively, and instead of α, we simply have α.
Moreover, we have the following proposition.

Proposition 6.3.1. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩. Let u be a non-zero vector in V , let v be any vector in V , and set
z := v − proju (v). Then z ⊥ u.

v
z = v−proju(v)

proju(v) u

Proof. We compute

⟨z, u⟩ = v − proju (v), u

⟨v,u⟩
= v− ⟨u,u⟩ u, u

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 407

(∗) ⟨v,u⟩
= ⟨v, u⟩ − ⟨u,u⟩ ⟨u, u⟩

= ⟨v, u⟩ − ⟨v, u⟩

= 0,

where (*) follows from r.2 and r.3 (in the real case) or from c.2 and c.3 (in the
complex case). This proves that z ⊥ u, which is what we needed to show.

6.3.2 Orthogonal and orthonormal sets. Orthogonal and orthonor-

mal bases
Suppose we are given a real or complex vector space V , equipped with a scalar
product ⟨·, ·⟩ and the induced norm || · ||. An orthogonal set of vectors in V is a set
of pairwise orthogonal vectors in V . An orthonormal set of vectors is an orthogonal
set of unit vectors (i.e. vectors of length 1). An orthogonal basis (resp. orthonormal
basis) of V is an orthogonal (resp. orthonormal) set in V that is also a basis of V .

Proposition 6.3.2. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩ and the induced norm || · ||. Then both the following hold:

(a) any orthogonal set of non-zero vectors in V is linearly independent;

(b) any orthonormal set of vectors in V is linearly independent.

Proof. Any orthonormal set of vectors is an orthogonal set of non-zero vectors

(because 0 is not a unit vector). So, (a) immediately implies (b).
It remains to prove (a). Fix an orthogonal set {u1 , . . . , uk } of non-zero vectors
in V . We must show that this set is linearly independent. Fix scalars α1 , . . . , αk
such that
α1 u1 + · · · + αk uk = 0.
We must show that α1 = · · · = αk = 0. Fix any i ∈ {1, . . . , k}. Then
(∗)
⟨α1 u1 + · · · + αk uk , ui ⟩ = ⟨0, ui ⟩ = 0,
| {z }
=0

where (*) follows from Proposition 6.1.4(c). On the other hand, note that
(∗) (∗∗)
⟨α1 u1 + · · · + αk uk , ui ⟩ = α1 ⟨u1 , ui ⟩ + · · · + αk ⟨uk , ui ⟩ = αi ⟨ui , ui ⟩,

where (*) follows from r.2 and r.3 (in the real case) or from c.2 and c.3 (in the
complex case), and (**) follows from the fact that {u1 , . . . , uk } is an orthogonal

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 408

set.14 So,
αi ⟨ui , ui ⟩ = 0.
Since ui ̸= 0, r.1 or c.1 guarantees that ⟨ui , ui ⟩ = ̸ 0; consequently, αi = 0. Since
i ∈ {1, . . . , k} was chosen arbitrarily, it follows that α1 = · · · = αk = 0, and we are
done.

Proposition 6.3.3. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩ and the induced norm || · ||. Let {u1 , . . . , uk } be an orthogonal set of
vectors in V . Then all the following hold:
(a) for all scalars α1 , . . . , αk , we have that {α1 u1 , . . . , αk uk } is an orthogonal set
of vectors;
n o
(b) if vectors u1 , . . . , uk are all non-zero, then ||uu11 || , . . . , ||uukk || is an orthonormal
set of vectors, and consequently, an orthonormal basis of Span(u1 , . . . , uk );
n o
(c) if {u1 , . . . , uk } is an orthogonal basis of V , then ||uu11 || , . . . , ||uukk || is an or-
thonormal basis of V .
Proof. Part (a) follows immediately from Proposition 6.1.4(b), and part (c) is a
special case of part (b).15n It remains to prove o (b). Assume that vectors u1 , . . . , uk
u1 uk
are all non-zero. By (a), ||u1 || , . . . , ||uk || is an orthogonal set. On the other hand,
by Proposition 6.2.3, vectors ||uu11 || , . . . , ||uukk || are all unit vectors. So, by definition,
n o
u1 uk
||u1 || , . . . , ||uk || is an orthonormal set. In particular, by Proposition 6.3.2(b), the
n o
set ||uu11 || , . . . , ||uukk || is linearly independent. Moreover, by Proposition 3.1.12, we
  n o
have that Span(u1 , . . . , uk ) = Span ||uu11 || , . . . , ||uukk || . So, ||uu11 || , . . . , ||uukk || is an
orthonormal basis of Span(u1 , . . . , uk ). This proves (b).

Proposition 6.3.4. Let V be a finite-dimensional real or complex vector space,

equipped with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Set n := dim(V ).
Then both the following hold:
(a) any orthogonal set of n non-zero vectors in V is an orthogonal basis of V ;
(b) any orthonormal set of n vectors in V is an orthonormal basis of V .
Proof. By Proposition 6.3.2(a), any orthogonal set of non-zero vectors is linearly
independent, and by Corollary 3.2.20(a), any linearly independent set of size n in an
n-dimensional vector space is a basis of that vector space. This proves (a). Part (b)
follows from (a), since any orthonormal set of vectors is, in particular, an orthogonal
set of non-zero vectors (because 0 is not a unit vector).
14
In particular, we have that ⟨uj , ui ⟩ = 0 for all j ∈ {1, . . . , k} \ {i}.
15
Indeed, if {u1 , . . . , uk } is an orthogonal
n basis of V , othen vectors u1 , . . . , uk are all non-zero and
u1 uk
V = Span(u1 , . . . , uk ), and so by (b), ||u1 ||
, . . . , ||u k ||
is an orthonormal basis of V .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 409

6.3.3 Coordinate vectors with respect to orthogonal and orthonor-

mal bases. Fourier coefficients
If we have an orthogonal basis of a real or complex vector space (equipped with a
scalar product and the norm induced by it), then every vector in that vector space
can be expressed as a linear combination of those basis vectors in a particularly nice
way, that is, we have a convenient formula for the coefficients in front of the basis
vectors (see Theorem 6.3.5). If our basis is orthonormal, then we get an even nicer
formula for the coefficients (see Corollary 6.3.6).
Theorem 6.3.5. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩. Let B = {u1 , . . . , un } be an orthogonal basis of V . Then for all
v ∈ V , we have that
n n
P P ⟨v,ui ⟩
v = projui (v) = ⟨ui ,ui ⟩ ui ,
i=1 i=1

and consequently,
h iT
  ⟨v,u1 ⟩ ⟨v,un ⟩
v B
= ⟨u1 ,u1 ⟩ ... ⟨un ,un ⟩ .

Proof. The second statement follows from the first and from the definition of a
coordinate vector. It remains to prove the first statement. Fix a vector v ∈ V . By
⟨v,ui ⟩
definition, for all i ∈ {1, . . . , n}, we have that projui (v) = ⟨u i ,ui ⟩
ui . So, it suffices to
show that
n
P ⟨v,ui ⟩
v = ⟨ui ,ui ⟩ ui .
i=1
Since v ∈ V and {u1 , . . . , un } is a basis of V , there exist scalars α1 , . . . , αn such that
n
P
v = α i ui .
i=1

Now, fix any index j ∈ {1, . . . , n}. We then have that

DPn E (∗) Pn (∗∗)
⟨v, uj ⟩ = αi ui , uj = αi ⟨ui , uj ⟩ = αj ⟨uj , uj ⟩,
i=1 i=1

where (*) follows from r.2 and r.3 (in the real case) or from c.2 and c.3 (in the complex
case), and (**) follows from the fact that u1 , . . . , un are pairwise orthogonal. Since
uj ̸= 0 (because {u1 , . . . , un } is a basis of V ), r.1 or c.1 guarantees that ⟨uj , uj ⟩ =
̸ 0,
and we deduce that
⟨v,u ⟩
αj = ⟨uj ,ujj ⟩ .
Since j ∈ {1, . . . , n} was chosen arbitrarily, we now deduce that
n n
P P ⟨v,ui ⟩
v = αi ui = ⟨ui ,ui ⟩ ui ,
i=1 i=1

and we are done.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 410

Corollary 6.3.6. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩ and the induced norm || · ||. Let B = {u1 , . . . , un } be an orthonormal
basis of V . Then for all v ∈ V , we have that
n
P
v = ⟨v, ui ⟩ ui ,
i=1

and consequently,
   T
v B
= ⟨v, u1 ⟩ . . . ⟨v, un ⟩ .

Terminology: Coefficients ⟨v, ui ⟩ from Corollary 6.3.6 are called Fourier coefficients.

Proof. Since {u1 , . . . , un } is an orthonormal basis of V , we know that ||u1 || = · · · =

||un || = 1, and consequently (by the construction of || · ||), we have that ⟨u1 , u1 ⟩ =
· · · = ⟨un , un ⟩ = 1. The result now follows immediately from Theorem 6.3.6.

6.3.4 Gram-Schmidt orthogonalization

In this subsection, we describe the “Gram-Schmidt orthogonalization process,” which
gives a recipe for transforming an arbitrary basis of a real or complex vector space
(equipped with a scalar product and the norm induced by it) into an orthogonal
(and even orthonormal) basis. But first, we need a technical proposition.

Proposition 6.3.7. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩. Let {u1 , . . . , uk } be an orthogonal set of non-zero vectors in V . Let
k k
P P ⟨v,ui ⟩
v ∈ V , and set y := projui (v) = ⟨ui ,ui ⟩ ui and z := v − y. Then all the
i=1 i=1
following hold:

(a) {u1 , . . . , uk , z} is an orthogonal set of vectors;

(b) z = 0 if and only if v ∈ Span(u1 , . . . , uk );

(c) Span(u1 , . . . , uk , v) = Span(u1 , . . . , uk , z).

Proof. First of all, Proposition 6.3.2 guarantees that {u1 , . . . , uk } is a linearly indepen-
dent set, and we deduce that {u1 , . . . , uk } is an orthogonal basis of Span(u1 , . . . , uk ).
Let us first prove (a). By hypothesis, vectors u1 , . . . , uk are pairwise orthogonal.
On the other hand, for each j ∈ {1, . . . , k}, we have the following:
k
P ⟨v,ui ⟩
⟨z, uj ⟩ = ⟨v − ⟨ui ,ui ⟩ ui , uj ⟩
i=1

(∗) k
P ⟨v,ui ⟩
= ⟨v, uj ⟩ − ⟨ui ,ui ⟩ ⟨ui , uj ⟩
i=1

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 411

(∗∗) ⟨v,uj ⟩
= ⟨v, uj ⟩ − ⟨uj ,uj ⟩ ⟨uj , uj ⟩

= ⟨v, uj ⟩ − ⟨v, uj ⟩

= 0,

where (*) follows from r.2 and r.3 (in the real case) or from c.2 and c.3 (in the
complex case), and (**) follows from the fact that {u1 , . . . , uk } is an orthogonal set.
Thus, {u1 , . . . , uk , z} is an orthogonal set of vectors. This proves (a).
k
P ⟨v,ui ⟩
Next, we prove (b). Clearly, z = 0 if and only if v = ⟨ui ,ui ⟩ ui . If v =
i=1
k
P ⟨v,ui ⟩
⟨ui ,ui ⟩ ui , then v is a linear combination of the vectors u1 , . . . , uk , and conse-
i=1
quently, v ∈ Span(u1 , . . . , uk ). On the other hand, if v ∈ Span(u1 , . . . , uk ), then
k
P ⟨v,ui ⟩ 16 This proves (b).
Theorem 6.3.5 guarantees v = ⟨ui ,ui ⟩ ui .
i=1

Finally, we prove (c). Fix any vector x ∈ V . We must show that x ∈

Span(u1 , . . . , uk , v) if and only if x ∈ Span(u1 , . . . , uk , z). We prove both direc-
tions (as we shall see, they are very similar).
Suppose first that x ∈ Span(u1 , . . . , uk , v). Then there exist scalars α1 , . . . , αk , β
such that x = α1 u1 + · · · + αk uk + βv. But now

x = α1 u1 + · · · + αk uk + βv

k
P 
= αi ui + β(y + z)
i=1

k
P  k
 P 
⟨v,ui ⟩

= α i ui + β ⟨ui ,ui ⟩ ui +z
i=1 i=1

k
P 
⟨v,ui ⟩

= αi + β ⟨u i ,ui ⟩
u i + βz,
i=1

and we deduce that x ∈ Span(u1 , . . . , uk , z).

Suppose, conversely, that x ∈ Span(u1 , . . . , uk , z). Then there exist scalars
α1 , . . . , αk , β such that x = α1 u1 + · · · + αk uk + βz. But now

x = α1 u1 + · · · + αk uk + βz

16
This is because {u1 , . . . , uk } is an orthogonal basis of Span(u1 , . . . , uk ).

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 412

k
P 
= αi ui + β(v − y)
i=1

k
P   k 
P ⟨v,ui ⟩

= α i ui + β v − ⟨ui ,ui ⟩ u i
i=1 i=1

k
P 
⟨v,ui ⟩

= αi − β ⟨u i ,ui ⟩
u i + βv,
i=1

and we deduce that x ∈ Span(u1 , . . . , uk , v). This proves (c).

The Gram-Schmidt orthogonalization process (version 1). Let V be a real or

complex vector space, equipped with a scalar product ⟨·, ·⟩ and the induced norm || · ||,
and let v1 , . . . , vk be linearly independent vectors in V . For all ℓ ∈ {1, . . . , k}, set
ℓ−1 ℓ−1
P P ⟨vℓ ,ui ⟩
uℓ := vℓ − projui (vℓ ) = vℓ − ⟨ui ,ui ⟩ ui .
i=1 i=1
n o
u1 uk
Then {u1 , . . . , uk } is an orthogonal basis of Span(v1 , . . . , vk ), and ||u1 || , . . . , ||uk ||
is an orthonormal basis of Span(v1 , . . . , vk ).
Remark: Before proceeding with the proof, it may be helpful to note that the
sequence u1 , . . . , uk is obtained (recursively) as follows:
ˆ u1 := v1 ;

ˆ u2 := v2 − proju1 (v2 );
 
ˆ u3 := v3 − proju1 (v3 ) + proju2 (v3 ) ;

..
.
 
ˆ uk := vk − proju1 (vk ) + proju2 (vk ) + · · · + projuk−1 (vk ) .

This describes precisely the sequence u1 , . . . , uk from the statement of the Gram-
Schmidt orthogonalization process (version 1), only in a less compact (but perhaps
more readable) form. We also note that it may be helpful to read Example 6.3.8
before reading the proof below, since Example 6.3.8 illustrates the Gram-Schmidt
orthogonalization process on a concrete numerical example.

Proof. We first prove that {u1 , . . . , uk } is an orthogonal basis of Span(v1 , . . . , vk ).

For each ℓ ∈ {1, . . . , k}, we set Uℓ := Span(v1 , . . . , vℓ ), and we prove (inductively)
that {u1 , . . . , uℓ } is an orthogonal basis of Uℓ . Obviously, this is enough, because for
k = ℓ, we get that {u1 , . . . , uk } is an orthogonal basis of Uk = Span(v1 , . . . , vk ).

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 413

Since {v1 , . . . , vk } is linearly independent, we see that v1 , . . . , vk are all non-zero,

and in particular, {v1 } is linearly independent. Since U1 = Span(v1 ) and u1 = v1 ,
we deduce that {u1 } is a basis of U1 , and this basis is obviously orthogonal (since it
contains only one vector).
Now, fix ℓ ∈ {1, . . . , k − 1}, and assume inductively that {u1 , . . . , uℓ } is an
orthogonal basis of Uℓ . We must show that {u1 , . . . , uℓ , uℓ+1 } is an orthogo-
nal basis of Uℓ+1 . Since {u1 , . . . , uℓ } and {v1 , . . . , vℓ } are two bases of Uℓ ,17 it
is clear that Span(u1 , . . . , uℓ , vℓ+1 ) = Span(v1 , . . . , vℓ , vℓ+1 ) = Uℓ+1 .18 On the
other hand, by the construction of uℓ+1 and by Proposition 6.3.7(c), we have that
Span(u1 , . . . , uℓ , vℓ+1 ) = Span(u1 , . . . , uℓ , uℓ+1 ). So, Span(u1 , . . . , uℓ , uℓ+1 ) = Uℓ+1 .
Since dim(Uℓ+1 ) = ℓ + 1,19 the fact that {u1 , . . . , uℓ , uℓ+1 } spans Uℓ+1 implies that
{u1 , . . . , uℓ , uℓ+1 } is in fact a basis of Uℓ+1 (this follows from Corollary 3.2.20).
By the induction hypothesis, vectors u1 , . . . , uℓ are pairwise orthogonal non-zero
vectors,20 and so by the construction of uℓ+1 and by Proposition 6.3.7(a), we have
that u1 , . . . , uℓ , uℓ+1 are pairwise orthogonal. So, {u1 , . . . , uℓ , uℓ+1 } is an orthogonal
basis of Uℓ+1 . This completes the induction.
We have now shown that {u1 , . . . , uk } is
n an orthogonalobasis of Span(v1 , . . . , vk ).
By Proposition 6.3.3(b), this implies that ||uu11 || , . . . , ||uukk || is an orthonormal basis
of Span(v1 , . . . , vk ). This completes the argument.

Example 6.3.8. Consider the following linearly independent vectors in R4 :

     
3 −5 8
 4   10   19 
v1 =   −4  ,
 v2 =   2 ,

 11  .
v3 =  

3 11 −2

Set U := Span(v1 , v2 , v3 ). Using the Gram-Schmidt orthogonalization process (ver-

sion 1):

(a) compute an orthogonal basis of U (with respect to the standard scalar product ·
in R4 ).

(b) compute an orthonormal basis of U (with respect to the standard scalar product ·
in R4 and the norm || · || induced by it).
17
The fact that {u1 , . . . , uℓ } is a basis of Uℓ follows from the induction hypothesis. The fact that
{v1 , . . . , vℓ } is a basis of Uℓ follows from the fact that {v1 , . . . , vℓ } is linearly independent (because
{v1 , . . . , vk } is linearly independent) and Uℓ = Span(v1 , . . . , vℓ ) (by construction).
18
Details?
19
This is because {v1 , . . . , vℓ+1 } is a basis of Uℓ+1 .
20
The fact that u1 , . . . , uℓ are all non-zero follows from the fact that {u1 , . . . , uℓ } is a basis of Uℓ
(by the induction hypothesis).

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 414

Remark: To see that v1 , v2 , v3 really are linearly independent, we compute

 
1 0 0
   0 1 0 
RREF v 1 v 2 v3 =   0 0 1 ,


0 0 0
   
and we deduce that rank v1 v 2 v3 = 3, i.e. v1 v2 v3 has full column
rank. So, by Theorem 3.3.14(a), vectors v1 , v2 , v3 are linearly independent.

Solution. (a) First, we set

 
3
 4 
u1 := v1 = 
 −4  .


Next, we compute:
  
−5 3
  
 10  4 
·
  
−4
   
−5 2
 





 3
v2 ·u1
 10  11 3  4 
u2 := v2 − u1 ·u1 u1  2 −
=      
 −4 
3 3
  
11  4  4  3
·
  
−4 −4
 
  
  
3 3
  

−5 3
 10 
 4 
(−5)·3+10·4+2·(−4)+11·3  
 2 −
=  
3·3+4·4+(−4)·(−4)+3·3  −4 

11 3
 
−8
 6 
= 
 6 .


Finally, we compute:
 
u3 := v3 − uv31 ·u·u1 u1 +
1 v3 ·u2
u2 ·u2 u2

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 415

     
8 3 8 −8
     
 19  4   19  6 
· ·
     
−4
     
11 11 6 −8 !
   
8 




 3 





 19  −2 3  4  −2 8  6 
=  11  −  −4  +
         
3 3 −8 −8  6 
−2
     
 4  4  3  6  6  8
· ·
     
−4 −4 6 6
   
     
     
3 3 8 8
  
  
8 3 −8 !
 19 
 4 
8·3+19·4+11·(−4)+(−2)·3   6 
8·(−8)+19·6+11·6+(−2)·8 
 
=  11  − 3·3+4·4+(−4)·(−4)+3·3  −4  +
 
(−8)·(−8)+6·6+6·6+8·8  6 
−2 3 8
 
9
 12 
=  12  .
 

−9

So,      
3 −8 9
n  4   6   12 o
B := {u1 , u2 , u3 } =  −4  ,  6  ,  12
     

3 8 −9
is an orthogonal basis of U .
(b) To obtain an orthonormal basis of U , we normalize the vectors of the
orthogonal basis B of U that we obtained in part (a). First, we compute:
v   
3 3
u
u
√ 4 
· 4 
u  
||u1 || = u1 · u1 = u
u
t −4   −4 
3 3
p
= 3 · 3 + 4 · 4 + (−4) · (−4) + 3 · 3
√
= 5 2

v   
u −8 −8
u
√ u 6 
· 6 
u  
||u2 || = u2 · u2 =
6 6 
u  
t
8 8

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 416

p
= (−8) · (−8) + 6 · 6 + 6 · 6 + 8 · 8
√
= 10 2

v   
9 9
u
u
√ u 12   12
u   
||u3 || = u3 · u3 = · 
t 12   12
u 
−9 −9
p
= 9 · 9 + 12 · 12 + 12 · 12 + (−9) · (−9)
√
= 15 2.

We now see that

    
3 −4 3
n
u1 u2 u3
o n
1 
 4  1 
 3  1 
 4 o
C := ||u1 || , ||u2 || , ||u3 || = √ , √
5 2  −4  5 2 
, √ 
3  5 2 4 
3 4 −3
is an orthonormal basis of U .

Remark: Suppose that V is a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩ and the induced norm || · ||. Suppose, furthermore, that we are given
a list v1 , . . . , vk of vectors in V , which may possibly be linearly dependent. How
would we find an orthogonal (or orthonormal) basis of Span(v1 , . . . , vk )? In this
case, we would first need to find a basis of Span(v1 , . . . , vk ), and then apply the
Gram-Schmidt process to that basis. If V = Rn , then Theorem  3.3.4 guarantees

that such a basis is formed by the pivot columns of the matrix v1 . . . vk . A
numerical example is given below.
Example 6.3.9. Consider the following vectors in R3 :
       
1 −2 3 5
v1 =  2  , v2 =  −4  , v3 =  5  , v4 =  9  .
1 −2 −1 1
Set U := Span(v1 , v2 , v3 , v4 ). Compute an orthogonal basis of U (with respect to the
standard scalar product · in R3 ).
Solution. First, we form the matrix
 
  1 −2 3 5
A := v1 v2 v3 v4 =  2 −4 5 9 ,
1 −2 −1 1

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 417

and by row reducing, we obtain

 
1 −2 0 2
RREF(A) =  0 0 1 1 .
0 0 0 0
So, the pivot columns of A are its first and third column, and therefore, {v1 , v3 }
is a basis of Col(A) = Span(v1 , v2 , v3 , v4 ). We now apply the Gram-Schmidt
orthogonalization process to vectors v1 , v3 . First, we set:
 
1
u1 := v1 =  2  .
1
Further, we compute:
  
3 1
  
   5 · 2   
3   
1
v3 ·u1 −1 1
u2 := v2 − u1 ·u1 u1 =  5 −     2 
1 1
−1  
 2 · 2 
 1
  
1 1
   
3 1
3·1+5·2+(−1)·1 
=  5 − 1·1+2·2+1·1 2 
−1 1
 
1
=  1 .
−3
We now see that
   
n 1 1 o
B := {u1 , u2 } =  2 , 1 
1 −3
is an orthogonal basis of U .

Remark: We could obtain an orthonormal basis of U (with respect to the standard

scalar product · and the induced norm || · ||) by normalizing
√ the vectors
√ in the
orthogonal basis B of U . We simply compute ||u1 || = 6 and ||u2 || = 11, and we
deduce that
 √   √ 
n o n 1/√6 1/√11 o
u1 u2
C := ||u1 || , ||u2 || =  2/ 6  ,  1/ 11 
√ √
1/ 6 −3/ 11
is an orthonormal basis of U .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 418

The Gram-Schmidt orthogonalization process (version 2). Let V be a real or

complex vector space, equipped with a scalar product ⟨·, ·⟩ and the induced norm || · ||,
and let v1 , . . . , vk be linearly independent vectors in V . For all ℓ ∈ {1, . . . , k}, set
ℓ−1
P ℓ−1
P
uℓ = v ℓ − projzi (vℓ ) = vℓ − ⟨vℓ , zi ⟩ zi ;
i=1 i=1

uℓ
zℓ = ||uℓ || .

Then {z1 , . . . , zk } is an orthonormal basis of Span(v1 , . . . , vk ).

The proof of correctness of the Gram-Schmidt orthogonalization process (ver-

sion 2) is similar to that of version 1, and we omit it. Let us, however, explain the
main difference. The Gram-Schmidt orthogonalization process (version 2) recursively
constructs two sequences of vectors, namely, u1 , . . . , uk and z1 , . . . , zk , as follows:

ˆ u1 = v1 ;

ˆ z1 = u1
||u1 || ;

ˆ u2 = v2 − projz1 (v2 );

ˆ z2 = u2
||u2 || ;
 
ˆ u3 = v3 − projz1 (v3 ) + projz2 (v3 ) ;

ˆ z3 = u3
||u3 || ;

..
.
 
ˆ uk = vk − projz1 (vk ) + projz2 (vk ) + · · · + projzk−1 (vk ) ;
uk
ˆ zk = ||uk || .

So, at each step, we obtain a vector uℓ that is orthogonal to the previously constructed
vectors z1 , . . . , zℓ−1 , and then we normalize uℓ to obtain the unit vector zℓ that
points in the same direction as uℓ . (In version 1, we skip this normalization process
during our recursive construction. At the very end, we may optionally normalize all
the vectors in the orthonormal basis that we obtain and thus create an orthonormal
basis.) Note that, for all ℓ ∈ {1, . . . , k} and i ∈ {1, . . . , ℓ − 1}, we have that
projzi (vℓ ) = ⟨vℓ , zi ⟩ zi . This is because ||zi || = 1, and so ⟨zi , zi ⟩ = 1, and therefore,
projzi (vℓ ) = ⟨v ℓ ,zi ⟩
⟨zi ,zi ⟩ zi = ⟨vℓ , zi ⟩ zi .
We now return to Example 6.3.8, and we compute an orthonormal basis using
the Gram-Schmidt process (version 2).

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 419

Example 6.3.10. Consider the following linearly independent vectors in R4 :

     
3 −5 8
 4   10   19 
v1 =   −4 
, v 2 = 
 2 
, v 3 =  11  .
 

3 11 −2

Set U := Span(v1 , v2 , v3 ). Using the Gram-Schmidt orthogonalization process (ver-

sion 2), compute an orthonormal basis of U (with respect to the standard scalar
product · in R4 and the norm || · || induced by it).

Proof. We set  
3
 4 
u1 := v1 = 
 −4  ,


3
√ √
we compute ||u1 || = u1 · u1 = 5 2, and we set
 
3
u1 1 
 4 
z1 := ||u1 || = √ .
5 2  −4 
3

Next, we set

u2 := v2 − (v2 · z1 ) z1
      
−5 −5 3 3
 10    10  1 
 4 
 1 
 4 

 2 −  2
=    · √  √ 
 5 2  −4  5 2  −4 
11 11 3 3
 
−8
 6 
=  6 .
 

8
√ √
We compute ||u2 || = u2 · u2 = 10 2, and we set
 
−4
u2
 3 
1 
z2 := ||u2 || = √ .
5 2 3 
4

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 420

Next, we set
 
u3 := v3 − (v3 · z1 ) z1 + (v3 · z2 ) z2


     
8 8 3 3
 19   19  1 
 4 
 1 
 4 

 11  −
=   11  ·
   √  √  +
5 2  −4  5 2  −4 
−2 −2   3  3 
8 −4 −4 !
  19  3 
 1  3
1 
 

+  11  · 5 2  3
 √  √  
 5 2 3 
−2 4 4

9
 12 
=   12  .


−9
√ √
We compute ||u3 || = u3 · u3 = 15 2, and we set
 
3
u3
 4 
1 
z3 := ||u3 || = √ .
5 2 4 
−3
We now have that
     
3 −4 3
n
1 
 4  1 
 3  1 
 4 o
C := {z1 , z2 , z3 } = √ , √
5 2  −4  5 2 
, √ 
3  5 2 4 
3 4 −3
is an orthonormal basis of U .

We complete this subsection with the following important corollary of the Gram-
Schmidt orthogonalization process.
Corollary 6.3.11. Let V be a finite-dimensional real or complex vector space,
equipped with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Let U be a subspace
of V . Then all the following hold:
(a) U has an orthogonal basis;
(b) any orthogonal basis of U can be extended to an orthogonal basis of V ;21
21
This means that for any orthogonal basis B of U , there exists an orthogonal basis C of V such
that B ⊆ C.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 421

(c) U has an orthonormal basis;

(d) any orthonormal basis of U can be extended to an orthonormal basis of V .22

Proof. Since V is finite-dimensional, Theorem 3.2.21 guarantees that the subspace

U of V is also finite-dimensional. Consider any basis {v1 , . . . , vk } of U . Then the
Gram-Schmidt orthogonalization process (version 1) applied to the vectors v1 , . . . , vk
yields
n a sequence o of vectors u1 , . . . , uk such that {u1 , . . . , uk } is an orthogonal and
u1 uk
||u1 || , . . . , ||uk || an orthonormal basis of U = Span(v1 , . . . , vk ). This proves (a)
and (c).
For (b), consider any orthogonal basis {v1 , . . . , vk } of U , and using Theo-
rem 3.2.19, extend it to a basis {v1 , . . . , vk , vk+1 , . . . , vn } of V . We apply the Gram-
Schmidt orthogonalization process (version 1) to the sequence v1 , . . . , vk , vk+1 , . . . , vn ,
and we obtain a sequence u1 , . . . , uk , uk+1 , . . . , un such that {u1 , . . . , uk , uk+1 , . . . , un }
is an orthogonal basis of V . However, since v1 , . . . , vk were pairwise orthogonal to
begin with, we see from the description of the Gram-Schmidt orthogonalization pro-
cess that u1 = v1 , . . . , uk = vk . So, the orthogonal basis {u1 , . . . , uk , uk+1 , . . . , un }
of V extends the orthogonal basis {v1 , . . . , vk } of U . This proves (b).
For (d), consider any orthonormal basis {u1 , . . . , uk } of U . In particular, the
basis {u1 , . . . , uk } of U is orthogonal, and so by (b), it can be extended to an
orthogonal
n basiso {u1 , . . . , uk , uk+1 , . . . , un } of V . Then by Proposition 6.3.3(c),
u1 un
||u1 || , . . . , ||un || is an orthonormal basis of V . But since the basis {u1 , . . . , uk }
of U is orthonormal, we know that ||u1 || = · · · = ||u n k || = 1, and ito follows that
u1 uk u1 un
||u1 || = u1 , . . . , ||uk || = uk . So, our orthonormal basis ||u1 || , . . . , ||un || of V in fact
extends the orthonormal basis {u1 , . . . , uk } of U . This proves (d).

6.4 The orthogonal complement of a subspace

Suppose we are given a real or complex vector space V , equipped with a scalar
product ⟨·, ·⟩. For a set A ⊆ V ,23 the orthogonal complement of A, denoted by A⊥ ,
is the set of all vectors in V that are orthogonal to A, that is,

A⊥ = {v ∈ V | v ⊥ A}

= {v ∈ V | v ⊥ a ∀a ∈ A}

= {v ∈ V | ⟨v, a⟩ = 0 ∀a ∈ A}.

22
This means that for any orthonormal basis B of U , there exists an orthonormal basis C of V
such that B ⊆ C.
23
Here, A may or may not be a subspace of V .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 422

Proposition 6.4.1. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩. Let A, B ⊆ V . Then

(a) A⊥ is a subspace of V ;24

(b) if A ⊆ B, then A⊥ ⊇ B ⊥ .

Proof. (a) We use Theorem 3.1.7.

First, by Proposition 6.1.4(c), we have that 0 ⊥ v for all v ∈ V . In particular,
0 ⊥ a for all a ∈ A. So, 0 ∈ A⊥ .
Next, fix vectors x1 , x2 ∈ A⊥ . Then for all a ∈ A, we have that ⟨x1 , a⟩ =
⟨x2 , a⟩ = 0, and consequently,

⟨x1 + x2 , a⟩ = ⟨x1 , a⟩ + ⟨x2 , a⟩ = 0 + 0 = 0,

i.e. (x1 + x2 ) ⊥ a. So, x1 + x2 ∈ A⊥ .

Finally, fix a vector x ∈ A⊥ and a scalar α. Then for all a ∈ A, we have that
⟨x, a⟩ = 0, and consequently,

⟨αx, a⟩ = α⟨x, a⟩ = α0 = 0,

i.e. αx ⊥ a. So, αx ∈ A⊥ .
By Theorem 3.1.7, it now follows that A⊥ is a subspace of V .
(b) Suppose that A ⊆ B. Then any vector that is orthogonal to all vectors in B
is, in particular, orthogonal to all vectors in A. So, A⊥ ⊇ B ⊥ .

Proposition 6.4.2. Let V be a real or complex vector space, equipped with a scalar
product ⟨·, ·⟩. Let u1 , . . . , uk ∈ V . Then {u1 , . . . , uk }⊥ = Span(u1 , . . . , uk )⊥ .

Proof. Since {u1 , . . . , uk } ⊆ Span(u1 , . . . , uk ), Proposition 6.4.1(b) guarantees that

{u1 , . . . , uk }⊥ ⊇ Span(u1 , . . . , uk )⊥ . Let us prove the reverse inclusion. Fix x ∈
{u1 , . . . , uk }⊥ . We must show that x ∈ Span(u1 , . . . , uk )⊥ . Fix u ∈ Span(u1 , . . . , uk ).
Then there exist scalars α1 , . . . , αk such that u = α1 u1 + · · · + αk uk . But now

⟨u, x⟩ = ⟨α1 u1 + · · · + αk uk , x⟩

= α1 ⟨u1 , x⟩ + · · · + αk ⟨uk , x⟩

(∗)
= α1 0 + · · · + αk 0

= 0,

24
Note that it is possible that A = ∅. In this case, we simply get that A⊥ = V . This is because
every vector in V is (vacuously) orthogonal to every vector in the empty set.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 423

where (*) follows from the fact that x ∈ {u1 , . . . , uk }⊥ . This proves that x ⊥ u, and
consequently, x ∈ Span(u1 , . . . , uk )⊥ .

Recall from subsection 3.1.3 that if V is a vector space over a field F, and U and
W are subspaces of V , then

U +W := {u + w | u ∈ U, w ∈ W }

is a subspace of V . Moreover, recall from subsection 3.2.6, that if V = U + W and

U ∩ W = {0}, then we say that V is the direct sum of U and W , and we write
V = U ⊕ W.
Theorem 6.4.3. Let V be a finite-dimensional real or complex vector space, equipped
with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Let U be a subspace of V .25
Then U ⊥ is a subspace of V , and all the following hold:
(a) if {u1 , . . . , uk } is an orthogonal basis of U , and {u1 , . . . , uk , uk+1 , . . . , un } is an
extension of that basis to an orthogonal basis of V ,26 then {uk+1 , . . . , un } is an
orthogonal basis of U ⊥ ;

(b) if {u1 , . . . , uk } is an orthonormal basis of U , and {u1 , . . . , uk , uk+1 , . . . , un } is

an extension of that basis to an orthonormal basis of V ,27 then {uk+1 , . . . , un }
is an orthonormal basis of U ⊥ ;

(c) (U ⊥ )⊥ = U ;

(d) V = U ⊕ U ⊥ , that is, V = U + U ⊥ and U ∩ U ⊥ = {0};

(e) dim(V ) = dim(U ) + dim(U ⊥ ).

Proof. By Proposition 6.4.1(a), U ⊥ is a subspace of V . It remains to prove (a)-(e).
We first prove (a). Assume that {u1 , . . . , uk } is an orthogonal basis of U , and
that {u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an orthogonal basis
of V . We must show that {uk+1 , . . . , un } is an orthogonal basis of U ⊥ . Clearly,
{uk+1 , . . . , un } is an orthogonal set of vectors, and so it suffices to show that
{uk+1 , . . . , un } is in fact a basis of U ⊥ . We already know that {uk+1 , . . . , un } is
linearly independent (because it is a subset of the basis {u1 , . . . , uk , uk+1 , . . . , un }
of V ), and so we need only show that Span(uk+1 , . . . , un ) = U ⊥ .
Let us first prove that Span(uk+1 , . . . , un ) ⊇ U ⊥ . Fix x ∈ U ⊥ . Then x ∈ V , and
so since {u1 , . . . , un } is an orthogonal basis of V , Theorem 6.3.5 guarantees that
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=1
25
By Theorem 3.2.21, the fact that V is finite-dimensional implies that U is also finite-dimensional.
26
The existence of {u1 , . . . , uk } and {u1 , . . . , uk , uk+1 , . . . , un } follows from Corollary 6.3.11(a,b).
27
The existence of {u1 , . . . , uk } and {u1 , . . . , uk , uk+1 , . . . , un } follows from Corollary 6.3.11(c,d).

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 424

Since x ∈ U ⊥ , and since u1 , . . . , uk ∈ U , we know that ⟨x, ui ⟩ = 0 for all i ∈

{1, . . . , k}. Consequently,
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=k+1

Thus, x is a linear combination of the vectors uk+1 , . . . , un , and we deduce that

x ∈ Span(uk+1 , . . . , un ). This proves that Span(uk+1 , . . . , un ) ⊇ U ⊥ .
For the reverse inclusion, we fix an arbitrary x ∈ Span(uk+1 , . . . , un ), and we
show that x ∈ U ⊥ . Fix scalars αk+1 , . . . , αn such that

x = αk+1 uk+1 + · · · + αn un .

Fix any u ∈ U ; we must show that x ⊥ u. Since {u1 , . . . , uk } is a basis of U , we

know that there exist scalars α1 , . . . , αk such that

u = α1 u1 + · · · + αk uk .

Now, if V is a real vector space, then we have that

⟨x, u⟩ = ⟨αk+1 uk+1 + · · · + αn un , α1 u1 + · · · + αk uk ⟩

n
P k
P (∗)
= αi αj ⟨ui , uj ⟩ = 0,
i=k+1 j=1

where (*) follows from the fact that {u1 , . . . , un } is an orthogonal set. Similarly, if
V is a complex vector space, then we have that

⟨x, u⟩ = ⟨αk+1 uk+1 + · · · + αn un , α1 u1 + · · · + αk uk ⟩

n
P k
P (∗)
= αi αj ⟨ui , uj ⟩ = 0,
i=k+1 j=1

where (*) follows from the fact that {u1 , . . . , un } is an orthogonal set. In either case,
we get that x ⊥ u, and consequently, x ∈ U ⊥ . It follows that Span(uk+1 , . . . , un ) ⊆
U ⊥ . This proves (a). Part (b) follows immediately from part (a).28
It remains to prove (c), (d), and (e). First, since V is finite-dimensional, so
is U . So, by Corollary 6.3.11(a), U has an orthogonal basis {u1 , . . . , uk }. By
Corollary 6.3.11(b), the orthogonal basis {u1 , . . . , uk } of U can be extended to
28
This is “obvious,” but here are the details. Assume that {u1 , . . . , uk } is an orthonormal basis
of U , and that {u1 , . . . , uk , uk+1 , . . . , un } is an extension of that basis to an orthonormal basis of
V . Then {u1 , . . . , uk } is, in particular, an orthogonal basis of U , and {u1 , . . . , uk , uk+1 , . . . , un } is
an extension of that basis to an orthogonal basis of V . So, by (a), {uk+1 , . . . , un } is an orthogonal
basis of U ⊥ . But all vectors in {uk+1 , . . . , un } are unit vectors (because {u1 , . . . , uk , uk+1 , . . . , un }
is an orthonormal basis of V ). So, {uk+1 , . . . , un } is an orthonormal basis of U ⊥ . This proves (b).

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 425

an orthogonal basis {u1 , . . . , uk , uk+1 , . . . , un } of V . By (a), {uk+1 , . . . , un } is an

orthogonal basis of U ⊥ . But then {uk+1 , . . . , un , u1 , . . . , uk } is an orthogonal basis
of V that extends {uk+1 , . . . , un },29 and so by (a) applied to the vector space U ⊥ ,
we have that {u1 , . . . , uk } is an orthogonal basis of (U ⊥ )⊥ . But now {u1 , . . . , uk } is
a basis of both U and (U ⊥ )⊥ , and it follows that U = (U ⊥ )⊥ , i.e. (c) holds. Further,
we have the following:

ˆ dim(U ) = k, since {u1 , . . . , uk } is a basis of U ;

ˆ dim(U ⊥ ) = n − k, since {uk+1 , . . . , un } is a basis of U ⊥ ;

ˆ dim(V ) = n, since {u1 , . . . , uk , uk+1 , . . . , un } is a basis of V .

It now immediately follows that dim(V ) = dim(U ) + dim(U ⊥ ), i.e. (e) holds.
Finally, we prove (d). Let us first show that U ∩ U ⊥ = {0}. Since U and U ⊥ are
both subspaces of V , they both contain 0, and consequently, 0 ∈ U ∩ U ⊥ . Now, fix
any u ∈ U ∩ U ⊥ ; we must show that u = 0. Since u ∈ U and u ∈ U ⊥ , we have that
u ⊥ u, i.e. ⟨u, u⟩ = 0. But then by the definition of a scalar product, we have that
u = 0. This proves that U ∩ U ⊥ = {0}. It remains to show that V = U + U ⊥ . It
is clear that U + U ⊥ ⊆ V , and so we need only show that V ⊆ U + U ⊥ . Fix any
v ∈ V . Since {u1 , . . . , uk , uk+1 , . . . , un } is a basis of V , we know that there exist
scalars α1 , . . . , αn such that v = α1 u1 + · · · + αn un . Set v1 := α1 u1 + · · · + αk uk
and v2 := αk+1 uk+1 + · · · + αn un . Then v = v1 + v2 . Since {u1 , . . . , uk } is a basis
of U , we see that v1 ∈ U , and since {uk+1 , . . . , un } is a basis of U ⊥ , we see that
v2 ∈ U ⊥ . So, v = v1 + v2 belongs to U + U ⊥ , and it follows that V ⊆ U + U ⊥ . This
proves (d), and we are done.

As a corollary of Theorem 6.4.3(a-b), we obtain the following computationally

useful proposition.

Proposition 6.4.4. Let V be a finite-dimensional real or complex vector space,

equipped with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Let {v1 , . . . , vk } be
any linearly independent set of vectors V , and let {v1 , . . . , vk , vk+1 , . . . , vn } be an
extension of that linearly independent set to a basis of V .30 Set U := Span(v1 , . . . , vk ).

(a) If the Gram-Schmidt orthogonalization process (version 1) is applied to input vec-

tors v1 , . . . , vk , vk+1 , . . . , vn to produce output vectors u1 , . . . , uk , uk+1 , . . . , un ,
then both the following hold:

ˆ {u1 , . . . , uk } is an orthogonal basis of U , and {uk+1 , . . . , un } is an orthog-

onal basis of U ⊥ ;
29
Indeed, since {u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V , so is
{uk+1 , . . . , un , u1 , . . . , uk } (we simply reordered vectors).
30
By Theorem 3.2.19, any linearly independent set of vectors in a finite-dimensional vector space
can be extended to a basis of that vector space.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 426

n o n o
uk uk+1
ˆ u1
||u1 || , . . . , ||uk || is an orthonormal basis of U , and un
||uk+1 || , . . . , ||un || is
an orthonormal basis of U ⊥ .

(b) If the Gram-Schmidt orthogonalization process (version 2) is applied to input vec-

tors v1 , . . . , vk , vk+1 , . . . , vn to produce output vectors z1 , . . . , zk , zk+1 , . . . , zn ,
then {z1 , . . . , zk } is an orthonormal basis of U , and {zk+1 , . . . , zn } is an or-
thonormal basis of U ⊥ .

Proof. Let us prove (a). The Gram-Schmidt orthogonalization process (version 1)

applied to v1 , . . . , vk produces vectors u1 , . . . , uk , which form an orthogonal ba-
sis of U . If we now continue the Gram-Schmidt orthogonalization process (ver-
sion 1) with vectors vk+1 , . . . , vn , then we obtain vectors uk+1 , . . . , un such that
{u1 , . . . , uk , uk+1 , . . . , un } is an orthogonal basis of V . But now by Theorem 6.4.3(a),
we know that {uk+1 , . . . , un } is an orthogonal basis of U ⊥ . If we then normal-
ize our vecotrs u1 , . . . , uk , uk+1 , . n . . , un , then Proposition
o 6.3.3(b) guarantees that
u1 uk
we obtain an orthonormal basis ||u1 || , . . . , ||uk || of U and an orthonormal basis
n o
uk+1
||uk+1 || , . . . , un
||un || of U ⊥ . This proves (a).

The proof of (b) is similar to that of (a), except that we apply Theorem 6.4.3(b)
instead of Theorem 6.4.3(a).

Example 6.4.5. Consider the following vectors in R4 :

      
1 2 0 2
 1   2   3   4 
a1 = 
 1 ,
 a2 =  2 ,
 a3 =  3 ,
 a4 = 
 4 .


0 0 3 2

Compute an orthonormal basis of U := Span(a1 , a2 , a3 , a4 ) and an orthonormal basis

of U ⊥ .

Solution. First, we need to find a basis of U and extend it to a basis of R4 . For this,
we use Proposition 3.3.23. We consider the standard basis E4 = {e1 , e2 , e3 , e4 } of
R4 , and we form the matrix
 
C := a1 a2 a3 a4 e1 e2 e3 e4
 
1 2 0 2 1 0 0 0
 1 2 3 4 0 1 0 0 
= 
 1
.
2 3 4 0 0 1 0 
0 0 3 2 0 0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 427

By row reducing, we obtain

 
1 2 0 2 0 0 1 −1
 0 0 1 2/3 0 0 0 1/3 
RREF(C) =  .
 0 0 0 0 1 0 −1 1 
0 0 0 0 0 1 −1 0

As we can see, the pivot columns of C are its first, third, fifth, and sixth column.
So, by Proposition 3.3.23, {a1 , a3 } is a basis of U , and {a1 , a3 , e1 , e2 } is a basis of
R4 that extends {a1 , a3 }. By applying the Gram-Schmidt orthogonalization process
(version 2) to the vectors a1 , a3 , e1 , e2 , we obtain the following vectors:
 √  √ 
−2/√15

1/√3
 1/ 3   1/ 15 
z1 =  √  √ 
 1/ 3  ,  1/ 15  ,
z2 = 
√
0 3/ 15
 √   
2/√10 0√
 −1/ 10  1/ 2 
√ √ 

z3 = 
 −1/ 10
,
  −1/ 2  .
z4 = 
√
2/ 10 0

By Proposition 6.4.4(b), {z1 , z2 } is an orthonormal basis of U , whereas {z3 , z4 } is

an orthonormal basis of U ⊥ .

Remark: We could also have applied the Gram-Schmidt orthogonalization process

(version 1) to the vectors a1 , a3 , e1 , e2 , and then normalized the output vectors.
We would have gotten the same vectors z1 , z2 , z3 , z4 as above. Proposition 6.4.4(a)
would then imply that {z1 , z2 } is an orthonormal basis of U , whereas {z3 , z4 } is an
orthonormal basis of U ⊥ .

6.5 Orthogonal projection onto a subspace

Theorem 6.5.1. Let V be a finite-dimensional real or complex vector space, equipped
with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Let U be a subspace of V , and
let x ∈ V . Then there exists a unique vector xU ∈ U that has the property that

||x − xU || = min ||x − u||.

u∈U

Moreover, if {u1 , . . . , uk } is an orthogonal basis of U , then this vector xU is given

by the formula
k k
P P ⟨x,ui ⟩
xU = projui (x) = ⟨ui ,ui ⟩ ui .
i=1 i=1

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 428

Remark: Note that if {u1 , . . . , uk } happens to be an orthonormal basis of U ,

then we get that ⟨u1 , u1 ⟩ = · · · = ⟨uk , uk ⟩ = 1, and so the formula for xU from
Theorem 6.5.1 turns into
k
P k
P
xU = projui (x) = ⟨x, ui ⟩ ui .
i=1 i=1

Moreover, we note that if x ∈ U , then xU = x, since in this case, the expression

||x − u|| (for u ∈ U ) is minimized for u = x.

Proof of Theorem 6.5.1. Using Corollary 6.3.11, we fix an orthogonal basis {u1 , . . . , uk }
of U , and we extend it to an orthogonal basis {u1 , . . . , uk , uk+1 , . . . , un } of V . By
Theorem 6.4.3(a), {uk+1 , . . . , un } is an orthogonal basis of U ⊥ . Set
k
⟨x,ui ⟩
u∗ :=
P
⟨ui ,ui ⟩ ui .
i=1

(So, u∗ is defined via the formula from the statement of the theorem. The reason we
call it u∗ rather than xU is because we have not proven the existence and uniqueness
of xU yet. However, this is just a minor stylistic matter!) Since u∗ is a linear
combination of the vectors u1 , . . . , uk , which form a basis of U , we see that u∗ ∈ U .
Now, fix any u ∈ U . We must show that ||x − u∗ || ≤ ||x − u||, and that equality
holds if and only if u∗ = u. Clearly, this is sufficient to prove the theorem.
Let us first prove that (u∗ − u) ⊥ (x − u∗ ). Since u∗ , u ∈ U , and since U is a
subspace of V , it is clear that u∗ − u ∈ U . So, it suffices to show that x − u∗ ∈ U ⊥ .
By Theorem 6.3.5, we have that
n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui ,
i=1

and it follows that

n
⟨x,ui ⟩
x − u∗ =
P
⟨ui ,ui ⟩ ui .
i=k+1
So, x−u∗ is a linear combination of the vectors uk+1 , . . . , un ; since those n−k vectors
form a basis of U ⊥ , it follows that x − u∗ ∈ U ⊥ . This proves that (u∗ − u) ⊥ (x − u∗ ).
Now that we have shown that vectors u∗ − u and x − u∗ are orthogonal to each
other, we can apply the Pythagorean theorem (see subsection 6.2.2) to them, as
follows:
||x − u||2 = ||(x − u∗ ) + (u∗ − u)||2

(∗)
= ||x − u∗ ||2 + ||u∗ − u||2

≥ ||x − u∗ ||2 ,
where (*) follows from the Pythagorean theorem. Consequently, we have that
||x − u∗ || ≤ ||x − u||. Moreover, the inequality above is an equality if and only if
||u∗ − u|| = 0, i.e. if and only if u∗ = u. This completes the argument.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 429

Terminology/Notation: The vector xU from Theorem 6.5.1 is called the orthogonal

projection of x onto U .

Corollary 6.5.2. Let V be a finite-dimensional real or complex vector space, equipped

with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Let u be any non-zero vector
in V , and set U := Span(u).31 Then for every x ∈ V , we have that
⟨x,u⟩
xU = proju (x) = ⟨u,u⟩ u.

Proof. Clearly, {u} is an orthogonal basis of U . So, the result follows immediately
from Theorem 6.5.1.

Corollary 6.5.3. Let V be a finite-dimensional real or complex vector space, equipped

with a scalar product ⟨·, ·⟩ and the induced norm || · ||. Let U be a subspace of V , and
let x ∈ V . Then
x = xU + xU ⊥ .
Moreover, this is the unique way of expressing x as a sum of a vector in U and a
vector in U ⊥ .32

Proof. By Corollary 6.3.11, U has an orthogonal basis {u1 , . . . , uk }, and moreover,

this basis can be extended to an orthogonal basis {u1 , . . . , uk , uk+1 , . . . , un } of V .
By Theorem 6.4.3(a), we have that {uk+1 , . . . , un } is an orthogonal basis of U ⊥ .
Now, by Theorem 6.5.1, we have that
k n
P ⟨x,ui ⟩ P ⟨x,ui ⟩
xU = ⟨ui ,ui ⟩ ui and xU ⊥ = ⟨ui ,ui ⟩ ui .
i=1 i=k+1

On the other hand, by Theorem 6.3.5, we have that

n
P ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui .
i=1

Consequently,
n k
P  n
 P 
P ⟨x,ui ⟩ ⟨x,ui ⟩ ⟨x,ui ⟩
x = ⟨ui ,ui ⟩ ui = ⟨ui ,ui ⟩ ui + ⟨ui ,ui ⟩ ui = xU + xU ⊥ .
i=1 i=1 i=k+1

It remains to prove the uniqueness part of the corollary. So, suppose that y ∈ U
and z ∈ U ⊥ are such that x = y + z. We must prove that y = xU and z = xU ⊥ . We
have that
xU + xU ⊥ = x = y + z,
31
So, U is a one-dimensional subspace of V .
32
This means that for all y ∈ U and z ∈ U ⊥ , if x = y + z, then y = xU and z = xU ⊥ .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 430

and consequently,
xU − y = z − xU ⊥ .
But xU − y ∈ U and z − xU ⊥ ∈ U ⊥ . Since U ∩ U ⊥ = {0} (by Theorem 6.4.3(d)), it
follows that xU − y = z − xU ⊥ = 0, and consequently, y = xU and z = xU ⊥ . This
completes the argument.

Remark: We note that the uniqueness part of Corollary 6.5.3 could also have been
obtained as an immediate consequence of Theorems 6.4.3(d) and 3.2.24. However,
note that the proof of Theorem 3.2.24 is actually quite similar to the proof of the
uniqueness part of Corollary 6.5.3 that we gave above.

The linearity of orthogonal projection onto a subspace. Suppose that V is

a finite-dimensional real or complex vector space, equipped with a scalar product
⟨·, ·⟩ and the induced norm || · ||, and suppose that U is a subspace of V . We
can then define the function projU : V → V by setting projU (x) = xU for all
x ∈ V (where xU is the orthogonal projection of x onto U , as in Theorem
6.5.1).
Clearly, projU (u) = u for all u ∈ U . Moreover, we have that Im projU = U and
projU [U ] = U . Using the formula from Theorem 6.5.1, we can easily see that the
function projU is linear. Indeed, if {u1 , . . . , uk } is any orthogonal basis of U (such a
basis exists by Corollary 6.3.11), then the following hold:

ˆ for all x, y ∈ V , we have that

(∗) k
P ⟨x+y,ui ⟩
projU (x + y) = ⟨ui ,ui ⟩ ui
i=1

(∗∗) k
P ⟨x,ui ⟩+⟨y,ui ⟩
= ⟨ui ,ui ⟩ ui
i=1

k
P  k
P 
⟨x,ui ⟩ ⟨y,ui ⟩
= ⟨ui ,ui ⟩ ui + ⟨ui ,ui ⟩ ui
i=1 i=1

(∗)
= projU (x) + projU (y),

where both instances of (*) follow from Theorem 6.5.1, and (**) follows from
r.2 or c.2;

ˆ for all x ∈ V and scalars α, we have that

(∗) k
P ⟨αx,ui ⟩
projU (αx) = ⟨ui ,ui ⟩ ui
i=1

(∗∗) k
P α⟨x,ui ⟩
= ⟨ui ,ui ⟩ ui
i=1
Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 431

k
P ⟨x,ui ⟩
= α ⟨ui ,ui ⟩ ui
i=1

(∗)
= αprojU (x),

where both instances of (*) follow from Theorem 6.5.1, and (**) follows from
r.3 or c.3.

6.6 Orthogonal projection onto subspaces of Rn

In this section, we assume that Rn is equipped with the standard scalar product ·
and the induced norm || · ||. Recall that if we identify 1 × 1 matrices with scalars,
then we have that x · y = xT y for all x, y ∈ Rn .
Now, suppose that U is a subspace of Rn . As we saw above (see the comment
following the proof of Corollary 6.5.3), projU : Rn → Rn is linear (and its image is
U ). Since projU is linear, it has a standard matrix (note that this matrix belongs
to Rn×n ). In this section, we give formulas for the standard matrices of orthogonal
projections onto various subspaces of Rn .
In section 3.3, we defined the row space of a matrix A to be the span of the rows
of A, and Proposition 3.3.1(b) states that Row(A) = uT | u ∈ Col(AT ) . In this
section, it will be convenient to slightly modify the definition of the row space, as
follows:
Row(A) := Col(AT ).
So, we (re)defined the row space of a matrix to be the span of the transposes of its
rows. For example, for the matrix
 
1 2 1 2
A =  2 3 2 3 ,
3 4 3 4

we have that  
1 2 3
 2 3 4 
AT = 
 1
,
2 3 
2 3 4
and consequently,

    
1 2 3
 2   3   4 
Row(A) = Span 
 1 , 2
  ,  .
  3 
2 3 4

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 432

(If this change of definition bothers you, then every time you see Row(□), mentally
replace it with Col(□T ).)

Theorem 6.6.1. Let A ∈ Rn×m . Then Row(A)⊥ = Nul(A) and Row(A) = Nul(A)⊥ .

Proof. In view of Theorem 6.4.3(c), it suffices to show that Row(A)⊥ = Nul(A).33

Set  T 
a1
 .. 
A =  . ,
aTn
so that Row(A) = Span(a1 , . . . , an ).34 Now, for all vectors x ∈ Rm , we have the
following sequence of equivalences:

x ∈ Nul(A) ⇐⇒ Ax = 0

aT1
 
 .. 
⇐⇒  . x = 0
aTn
 
a1 · x
⇐⇒ ..
=0
 
 .
an · x

⇐⇒ ai · x = 0 ∀i ∈ {1, . . . , n}

⇐⇒ ai ⊥ x ∀i ∈ {1, . . . , n}

⇐⇒ x ∈ {a1 , . . . , an }⊥

(∗)
⇐⇒ x ∈ Span(a1 , . . . , an )⊥

⇐⇒ x ∈ Row(A)⊥ ,

where (*) follows from the fact that {a1 , . . . , am }⊥ = Span(a1 , . . . , am )⊥ (by Propo-
sition 6.4.2). This proves that Nul(A) = Row(A)⊥ , and we are done.
33
Indeed, by Theorem 6.4.3(c), we have that (Row(A)⊥ )⊥ = Row(A). So, if Row(A)⊥ = Nul(A),
then Nul(A)⊥ = (Row(A)⊥ )⊥ = Row(A).
34
Indeed, we have the following:
 
Row(A) = Col(AT ) = Col a1 . . . an = Span(a1 , . . . , an ).

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 433

Corollary 6.6.2. Let A ∈ Rn×m . Then all the following hold:

(a) Nul(AT A) = Nul(A);

(b) Row(AT A) = Row(A);

(c) rank(AT A) = rank(A).

Proof. We first prove (a). Note that AT A ∈ Rm×m , and that both Nul(A) and
Nul(AT A) are subspaces of Rm . Now, fix any x ∈ Rm . We must show that
x ∈ Nul(AT A) if and only if x ∈ Nul(A).
Suppose first that x ∈ Nul(A). Then Ax = 0, and consequently, AT Ax = 0. So,
x ∈ Nul(AT A).
Suppose, conversely, that x ∈ Nul(AT A). Then AT Ax = 0, and it follows that
xT AT Ax = 0. But note that xT AT Ax = (Ax)T (Ax) = (Ax) · (Ax) = ||Ax||2 ;
consequently, ||Ax||2 = 0. It follows that ||Ax|| = 0, and therefore, Ax = 0, i.e.
x ∈ Nul(A). This proves (a).
For (b), we observe that

Row(AT A) = Nul(AT A)⊥ by Theorem 6.6.1

= Nul(A)⊥ by (a)

= Row(A) by Theorem 6.6.1.

Finally, for (c), we have the following:

rank(AT A) = dim Row(AT A)

by Theorem 3.3.9


= dim Row(A) by (b)

= rank(A) by Theorem 3.3.9.

This completes the argument.

Theorem 6.6.3. Let A ∈ Rn×m be a matrix of rank m (i.e. A is a matrix of full

column rank). Then the matrix A(AT A)−1 AT is the standard matrix of orthogonal
projection onto Col(A), that is, for all x ∈ Rn , the orthogonal projection of x onto
C := Col(A) is given by

xC = A(AT A)−1 AT x.

Proof. Fix x ∈ Rn . We must first check that the expression A(AT A)−1 AT x is
defined and belongs to C = Col(A). First, note that AT A ∈ Rm×m , and that by
Corollary 6.6.2(a), we have that rank(AT A) = rank(A) = m. So, by the Invertible

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 434

Matrix Theorem,35 AT A is invertible, and we see that (AT A)−1 is defined and
belongs to Rm×m . Since A ∈ Rn×m , (AT A)−1 ∈ Rm×m , and AT ∈ Rm×n , we see
that A(AT A)−1 AT ∈ Rn×n ; since x ∈ Rn , we see that A(AT A)−1 AT x is defined and
belongs to Rn . Meanwhile, (AT A)−1 AT x is a vector in Rm , and so
 
A(AT A)−1 AT x = |{z} A (AT A)−1 AT x
n×m
| {z }
∈R ∈Rm

is a linear combination of the columns of A. By definition, this means that

A(AT A)−1 AT x ∈ Col(A) = C.
In view of Corollary 6.5.3, it is now enough to prove that

x − A(AT A)−1 AT x ∈ C ⊥ ,

for it will then follow that xC = A(AT A)−1 AT x,36 which is what we need to show.
But note that
(∗)
C ⊥ = Col(A)⊥ = Row(AT )⊥ = Nul(AT ),

where (*) follows from Theorem 6.6.1. So, it in fact suffices to show that the vector
x − A(AT A)−1 AT x belongs to Nul(AT ). For this, we compute:
 
AT x − A(AT A)−1 AT x = AT x − AT A(AT A)−1 AT x = 0.
| {z }
=Im

This proves that x − A(AT A)−1 AT x ∈ Nul(AT ), and we are done.

 
Remark: Suppose that we are given a non-zero matrix A = a1 . . . am in
Rn×m , and that we need to compute the standard matrix of orthogonal projection
onto Col(A). If rank(A) = m (i.e. A has full column rank), then the matrix that
we need is A(AT A)−1 AT , as per Theorem 6.6.3. But what if rank(A) < m? In that
case, we let B be the matrix obtained from A by deleting all the non-pivot columns
of A. By Theorem 3.3.4, the columns of B form a basis of Col(A), and we see that
Col(A) = Col(B). Moreover, all the columns of B are pivot columns, and so B has
full column rank. But now the matrix B satisfies the hypotheses of Theorem 6.6.3. So,
the standard matrix of orthogonal projection onto Col(A) = Col(B) is B(B T B)−1 B T .

As a special case of Theorem 6.6.3, we get the following.

35
See subsection 1.11.7 or 3.3.7.
36
Indeed, if we can show that x − A(AT A)−1 AT x ∈ C ⊥ , then we get that

x = A(AT A)−1 AT x + x − A(AT A)−1 AT x ,

| {z } | {z }
∈C ∈C ⊥

which (by Corollary 6.5.3) implies that xC = A(AT A)−1 AT x and xC ⊥ = x − A(AT A)−1 AT x.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 435

Corollary 6.6.4. Let a be a non-zero vector in Rn . Then the standard matrix of

orthogonal projection onto the line L := Span(a) is the matrix

a(aT a)−1 aT = a(a · a)−1 aT = 1 T

a·a aa .

Consequently, for every vector x ∈ Rn , we have that

1 T
xL = projL (x) = a·a aa x.

Proof. We have that L = Span(a) = Col(a), where we think of the vector a simply
as a one-column matrix. Moreover, since a ̸= 0, we know that rank(a) = 1, i.e. the
one-column matrix a has full column rank. The result now follows immediately from
Theorem 6.6.3.

Theorem 6.6.5. Let U be a subspace of Rn , and let P ∈ Rn×n be the standard

matrix of projU . Then In −P is the standard matrix of projU ⊥ , that is, for all x ∈ Rn ,
the orthogonal projection of x onto U ⊥ is given by xU ⊥ = (In − P )x.

Proof. We observe that for all x ∈ Rn , we have that

(∗) (∗∗)
(In − P )x = In x − P x = x − xU = xU ⊥ ,

where (*) follows from the fact that P is the standard matrix of projU ,37 and (**)
follows from Corollary 6.5.3. So, In − P is indeed the standard matrix of projU ⊥ .

Corollary 6.6.6. Let A ∈ Rn×m be a matrix of rank n (i.e. A is a matrix of full

row rank). Then the matrix Im − AT (AAT )−1 A is the standard matrix of orthogonal
projection onto N := Nul(A), that is, for all x
∈ Rm , the orthogonal projection of x
onto N is given by xN = Im − AT (AAT )−1 A x.

Proof. First, note that

(∗)
Nul(A) = Row(A)⊥ = Col(AT )⊥ .

where (*) follows from Theorem 6.6.1. Note further that AT ∈ Rm×n and that
(by Corollary 3.3.11) rank(AT ) = rank(A) = n, i.e. AT has full column rank. So,
by Theorem 6.6.3, the standard matrix of orthogonal projection onto Col(AT )
is AT (AAT )−1 A. Finally, by Theorem 6.6.5, the standard matrix of orthogonal
projection onto Col(AT )⊥ = Nul(A) is Im − AT (AAT )−1 A. This completes the
argument.
37
Technically, the fact that P is the standard matrix of projU guarantees that P x = xU . The fact
that In x = x follows from Proposition 1.4.6.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 436

Remark: Suppose that we are given a matrix A ∈ Rn×m , and that we need to
compute the standard matrix of orthogonal projection onto Nul(A). If rank(A) = n
(i.e. A has full row rank), then the matrix that we need is Im − AT (AAT )−1 A, as per
Corollary 6.6.6. But what if rank(A) < n? If A = On×m (i.e. A is a zero matrix),
then Nul(A) = Rm , and so the standard matrix of orthogonal projection onto Nul(A)
is the identity matrix Im . Assume now that A ̸= On×m . In this case, we let B be
the matrix obtained from RREF(A) by deleting any zero rows that RREF(A) may
have. By Proposition 3.3.28, Nul(A) = Nul(B). But the matrix B has full row rank,
which means that we can apply Corollary 6.6.6 to it. So, the standard matrix of
orthogonal projection onto Nul(A) = Nul(B) is Im − B T (BB T )−1 B.

6.7 The method of least squares

In some real-world applications, we may be interested in finding the best approxi-
mate solution to a (possibly inconsistent) matrix-vector equation Ax = b. More
formally, suppose we are given a norm || · || on Rn , a matrix A ∈ Rn×m , and a vector
b ∈ Rn . We would then like to find a vector x for which

||Ax − b||

is as small as possible. If Ax = b is consistent, then any solution of that equation will

minimize ||Ax − b||. However, what if the equation Ax = b is inconsistent? Then
the answer will obviously depend on which norm that we are using. In the remainder
of this section, we will work only with the norm induced by the standard scalar
product in Rn , i.e. the standard Euclidean norm. Recall that this is the norm || · ||
given by
√ p
||x|| = x·x = x21 + · · · + x2n
T
in Rn .

for all vectors x = x1 . . . xn

Theorem 6.7.1. Let A ∈ Rn×m and b ∈ Rn . Then the matrix-vector equation

AT Ax = AT b

is consistent, and moreover, its solution set is precisely the set of vectors x in Rm
that minimize the expression
||Ax − b||.

Proof. We are looking for vectors x ∈ Rm that minimize the expression ||Ax − b||.
Our goal is to show is that the vectors we are looking for are precisely those that
satisfy AT Ax = AT b.
By Proposition 3.3.2(a), we have that C := Col(A) = {Ax | x ∈ Rm }. So, we
are in fact looking for the solutions x of the equation Ax = bC , because by the

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 437

definition of bC , such x’s are precisely the ones for which ||Ax − b|| is minimized.
Moreover, by Corollary 6.5.3, b = bC + bC ⊥ is the only way to decompose b as a
sum of a vector in C and a vector in C ⊥ . So, we are looking for those x’s for which
b − Ax ∈ C ⊥ . But note that
(∗)
C ⊥ = Col(A)⊥ = Row(AT )⊥ = Nul(AT ),
where (*) follows from Theorem 6.6.1. So, we in fact looking for vectors x for which
b − Ax ∈ Nul(AT ), i.e. those that satisfy AT (b − Ax) = 0, which is obviously
equivalent to AT Ax = AT b.
It remains to show that the equation AT Ax = AT b is consistent. By our argument
above, a vector x ∈ Rm satisfies AT Ax = AT b if and only if it satisfies the equation
Ax = bC . Since the latter equation is consistent (this follows from the definition of
C and the existence of bC ), so is the former.

Terminology: Suppose we are given a matrix A ∈ Rn×m and a vector b ∈ Rn .

Vectors x ∈ Rm that minimize the expression ||Ax − b|| are called the least-squares
solutions of the equation Ax = b (such solutions are often denoted by x̂), whereas
the number
minm ||Ax − b||
x∈R
is called the least-squares error for the equation Ax = b. By Theorem 6.7.1, the
equation Ax = b has at least one least-squares solution x̂, and consequently, the
least-squares error is defined and is equal to ||Ax̂ − b||.
Remark: Obviously, if Ax = b is consistent, then the least-squares solutions of
Ax = b are precisely the solutions of the equation Ax = b itself. This is because if
Ax = b is consistent, then the solutions of that equation minimize the expression
||Ax − b|| (indeed, ||Ax − b|| = 0 if and only if Ax = b). Moreover, the matrix-vector
equation Ax = b is consistent if and only if the least-squares error of this equation
is zero.
Example 6.7.2. Let
  

1 −2 3
 −1 2   1 
A = 
 0
 and b = 
 −4  ,

3 
2 5 2
with entries understood to be in R. Find all least-squares solutions x̂ of Ax = b, as
well as the least-squares error. Is the equation Ax = b consistent?
Solution. We apply Theorem 6.7.1. So, we need to find the solutions of the equation
AT Ax̂ = AT b. We first compute
   
T 6 6 T 6
A A = and A b = ,
6 42 −6

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 438

and then we compute

 

T T
 1 0 4/3
RREF A A A b = .
0 1 −1/3

It follows that  
4/3
x̂ =
−1/3
is the unique solution of the matrix-vector equation AT Ax̂ = AT b, and consequently,
the unique least-squares solution of the matrix-vector equation Ax = b.
The least-squares error of Ax = b is
   
1 −2   3
 −1 2  4/3  1 
||Ax̂ − b|| = ||   − −4  ||

 0 3  −1/3
2 5 2
 
−1
 −3  √
= || 
 3  || = 2 5.


−1

Since the least-squares error of the equation Ax = b is strictly positive, we see that
the equation is inconsistent.

Remark: In the example above, the equation Ax = b had a unique least-squares

solution x̂, and we obtained the least-squares error of Ax = b by computing ||Ax̂−b||.
But what if Ax = b had more than one least-squares solution? In that case, we
would choose one least-squares solution x̂ (any one will do), and we would compute
||Ax̂ − b||. By the definition of a least-squares solution, the value of ||Ax̂ − b|| is
the same regardless of which least-squares solution x̂ we choose.

6.7.1 Data fitting

Suppose we are given a collection of two or more data points, and we wish to find
a line that best fits them. How would we do this? First, let us be a bit more
precise. We will be plotting our data points, say (x0 , y0 ), (x1 , y1 ), . . . , (xn , yn ), in R2 .
Most commonly, the x-axis is time (measured in whatever time units happen to be
convenient for the problem that we are studying), whereas the y-axis is the quantity
that we are measuring, such as population size, the average global temperature, the
number of products of a certain type produced or consumed in a given region, etc.
We are looking for a line f (x) = ax + b that best fits our data points (see the graph
below).

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 439

y
f (x) = ax + b

(xn, yn)
(x0, y0) (x1, y1)

(x2, y2)
x

So, we set up a system of linear equations shown below.

ax0 + b = y0
ax1 + b = y1
..
.
axn + b = yn

This linear
 system can be rewritten as the matrix-vector equation below, where the
a
vector is the unknown.
b
   
x0 1 y0
 x1 1  
 a
 y1 
= 
  
.. .. ..
 b
  
 . .  . 
xn 1 yn

Except in rare cases, the system above will  be inconsistent. For this reason, we
â
will look for the least-squares solution(s) of the system, which yields the line
b̂
fˆ(x) = âx + b̂. This (approximate) solution minimizes the following quantity:
     
x0 1 y0 ax0 + b − y0
 x1 1     y1   ax1 + b − y1 
 a
||  . .  −  .  || = ||   ||
    
..
 .. ..  b  ..   . 
xn 1 yn axn + b − yn

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 440

 
f (x0 ) − y0
 f (x1 ) − y1 
= ||   ||
 
..
 . 
f (xn ) − yn
s
n
P
2
= f (xi ) − yi .
i=0

So, we are effectively minimizing the sum of squares of the vertical distances between
our data points and the line, i.e. the sum of squares of the lengths of the purple
dotted line segments shown in the graph above.
Example 6.7.3. Using the method of least squares, find the line that best fits the
data points (1, 2), (2, 3), (3, 3), (5, 6).
Solution. We are looking for the function f (x) = ax + b that best fits these four
data points. We get the linear system below.
1a + b = 2
2a + b = 3
3a + b = 3
5a + b = 6
At a glance, we can see that this system is inconsistent; so, we will not be able to find
an exact solution and will instead have to settle for an approximate
  one. This system
a
can be rewritten as a matrix-vector equation below, where is the unknown.
b
   
1 1   2
 2 1  a  3 

 3 1  b
 =   3 


5 1 6
We multiply both sides by the transpose of the matrix on the left, and we get
the following (where a and b became â and b̂, respectively, because we are now
approximating):
   
  1 1     2
1 2 3 5   2 1  â 1 2 3 5   3 .
 
=
1 1 1 1  3 1  b̂ 1 1 1 1  3 
5 1 6
After performing matrix multiplication, we obtain
    
39 11 â 47
= .
11 5 b̂ 14

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 441

We now form the augmented matrix of the matrix-vector equation above, and we
row reduce to obtain:
  39 11 47   
1 0 81/74

RREF = .
11 5 14 0 1 29/74

This yields the least-squares solution

   
â 81/74
= .
b̂ 29/74

So, the line that best fits our data points is

fˆ(x) = 81
74 x + 29
74 .

6.8 Orthogonal matrices

Throughout this section, we assume that Rn is equipped with the standard scalar
product · and the induced norm || · ||.

6.8.1 Orthogonal matrices: definition and characterization

A matrix Q ∈ Rn×n is orthogonal if it satisfies QT Q = In . Obviously, matrices In
and −In are orthogonal. Moreover, by Theorem 2.3.14, permutation matrices are
orthogonal (as long as we consider the 0’s and 1’s in those matrices as being real
numbers). The matrices mentioned so far all have entries only −1, 0, 1. However,
there are many other orthogonal matrices: a couple of important examples will be
given in subsection 6.8.3. In this subsection, we prove a theorem (Theorem 6.8.1
below) that gives several equivalent characterizations of orthogonal matrices.

Theorem 6.8.1. Let Q ∈ Rn×n . Then the following are equivalent:

(a) Q is orthogonal (i.e. satisfies QT Q = In );

(b) Q is invertible and satisfies Q−1 = QT ;

(c) QQT = In ;

(d) QT is orthogonal;

(e) Q is invertible and Q−1 is orthogonal;

(f ) the columns of Q form an orthonormal basis of Rn ;

(g) the columns of QT form an orthonormal basis of Rn .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 442

Proof. By Corollary 3.3.20, we have that (a), (b), and (c) are equivalent. More-
over, since (QT )T = Q, we have that (c) and (d) are equivalent. This proves
that (a), (b), (c), and (d) are equivalent.
Next, (b) and (d) together imply (e). Suppose now that (e) holds. Then by
applying “(a) =⇒ (b)” to the matrix Q−1 , we see that Q−1 is invertible and satisfies
(Q−1 )−1 = (Q−1 )T . Consequently, Q−1 = QT , and it follows that (b) holds.
So far, we have established that (a), (b), (c), (d), and (e) are equivalent.
Let us now show that (a) and (f) are equivalent. Set Q = q1 . . . qn . Then
 T 
q1
 qT  
 2 
QT Q =  .  q1 q2 . . . qn

.
 . 
qTn
 
q 1 · q1 q1 · q2 . . . q1 · qn
 q 2 · q1 q2 · q2 . . . q2 · qn 
=  .
 
.. .. .. ..
 . . . . 
qn · q1 qn · q2 . . . qn · qn

So, QT Q = In if and only if {q1 , . . . , qn } is an orthonormal set. But by Proposi-

tion 6.3.4(b), any orthonormal set of n vectors in Rn is in fact an orthonormal basis
of Rn . It now follows that (a) and (f) are equivalent. Analogously, (d) and (g) are
equivalent. This completes the argument.

6.8.2 Making new orthogonal matrices out of old ones

Proposition 6.8.2. Let
rT1
 

. . . qn =  ... 
  
Q= q1


rTn
be an orthogonal matrix in Rn . Then all the following hold:
 
(a) for all α1 , . . . , αn ∈ {−1, 1}, the matrix α1 q1 . . . αn qn is orthogonal;

α1 rT1
 

(b) for all α1 , . . . , αn ∈ {−1, 1}, the matrix  ...  is orthogonal;

 

αn rTn
(c) the matrix −Q is orthogonal.
Remark: Proposition 6.8.2 guarantees that if we multiply one row or one column of
an orthogonal matrix by −1, then the resulting matrix is again orthogonal. Obviously,
we can iterate the process and obtain a sequence of orthogonal matrices.

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 443

 
Proof. We first prove (a). Fix α1 , . . . , αn ∈ {−1, 1}. Since Q = q1 . . . qn
is orthogonal, Theorem 6.8.1 guarantees that {q1 , . . . , qn } is an orthonormal basis
of Rn . But then {α1 q1 , . . . , αn qn } is also an orthonormal basis
 of Rn ,38 and so
once again by Theorem 6.8.1, the matrix α1 q1 . . . αn qn is orthogonal. This
proves (a).
Next, we prove (b). Fix α1 , .. . , αn ∈ {−1,
 1}. Since Q is orthogonal, Theo-
T
rem 6.8.1 guarantees that Q = r1 , . . . , rn is also orthogonal. By (a) applied
to the orthogonal matrix QT , we get that

 α 1 r 1 , . . . , α n
 r n is orthogonal. We
now apply Theorem 6.8.1 to the matrix α1 r1 , . . . , αn rn , and we deduce that its
transpose is orthogonal. This proves (b).
Finally, part (c) is simply a special case of (a) for α1 = · · · = αn = −1.

Proposition 6.8.3. If Q1 , Q2 ∈ Rn×n are orthogonal, then so is their product Q1 Q2 .

Proof. Assume Q1 , Q2 ∈ Rn×n are orthogonal. Then QT1 Q1 = In and QT2 Q2 = In ,

and consequently,

(Q1 Q2 )T (Q1 Q2 ) = QT2 QT1 Q1 Q2 = QT2 Q2 = In .

| {z }
=In

So, Q1 Q2 is indeed orthogonal.

Proposition 6.8.4. Let Q1 ∈ Rm×m and Q2 ∈ Rn×n be orthogonal matrices. Then

the (m + n) × (m + n) matrix
 
Q1 Om×n
Q =
On×m Q2

is an orthogonal matrix in R(m+n)×(m+n) .

Proof. By hypothesis, we have that QT1 Q1 = Im and QT2 Q2 = In . We now compute:

38
This is “obvious,” but here are the details. First of all, since {q1 , . . . , qn } is an orthonormal
basis of Rn , it is, in particular, an orthogonal set of unit vectors in Rn . By Proposition 6.3.3(a), the
fact that {q1 , . . . , qn } is an orthogonal set implies that {α1 q1 , . . . , αn qn } is also an orthogonal set.
On the other hand, for all i ∈ {1, . . . , n}, we have that
(∗) (∗∗) (∗∗∗)
||αi qi || = |αi | ||qi || = ||qi || = 1,

where (*) follows from Proposition 6.2.1, (**) follows from the fact that αi ∈ {−1, 1}, and (***)
follows from the fact that qi is a unit vector. But now {α1 q1 , . . . , αn qn } is an orthonormal set
of n vectors in Rn , and so Proposition 6.3.4(b) guarantees that {α1 q1 , . . . , αn qn } is in fact an
orthonormal basis of Rn .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 444

QT1
  
Om×n Q1 Om×n
QT Q =
On×m QT2 On×m Q2

QT1 Q1 + Om×n On×m QT1 Om×n + Om×n Q2

 
=
On×m Q1 + QT2 On×m On×m Om×n + QT2 Q2

QT1 Q1 Om×n
 
=
On×m QT2 Q2
 
Im Om×n
=
On×m In

= Im+n .

So, Q is indeed an orthogonal matrix.

6.8.3 The Householder matrix

For a non-zero vector a in Rn , the Householder matrix is the n × n matrix
2 2
H(a) := In − aT a
aaT = In − T
a·a aa .

To see that H(a) really is an orthogonal matrix, we perform the following simple
calculation:
2 2
H(a)T H(a) = (In − T T
a·a aa ) (In − T
a·a aa )

2 2
= (InT − T T
a·a (aa ) ) (In − T
a·a aa )

2 T 2 T
= (In − a·a aa ) (In − a·a aa )

= In − 4
a·a aa
T + 4
a aT a aT
(a·a)2 |{z}
=a·a

4 T 4 T
= In − a·a aa + a·a aa

= In .

Let us now discuss the geometric meaning of this matrix. First of all, we observe
that if U is a subspace of Rn , then for any vector x ∈ Rn , the reflection of x about
U is given by
refU (x) := x + 2(xU − x) = 2xU − x,

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 445

and the reflection of x about U ⊥ is given by

(∗)
refU ⊥ (x) := x + 2(xU ⊥ − x) = x − 2xU = −refU (x),
where (*) follows from the fact that x = xU + xU ⊥ (by Corollary 6.5.3); so, either
one of xU and xU ⊥ can be obtained from the other by reflecting about the origin.
U⊥

xU ⊥ − x
x
xU ⊥ xU − x
refU ⊥ (x)

xU U

refU (x)

Now, by Corollary 6.6.4, the standard matrix of orthogonal projection onto the line
L := Span(a) is

a(aT a)−1 aT = a(a · a)−1 aT = 1 T

a·a aa .

So, if x is any vector in Rn , then the projection of x onto L is given by

1 T
xL = a·a aa x,

the reflection of x about L is given by

refL (x) = 2xL − x

2 T
= a·a aa x − In x

2 T


= a·a aa − In x

= −H(a)x,

and the reflection of x about L⊥ is given by

refL⊥ (x) = −refL (x) = H(a)x.

We now see that −H(a) is the standard matrix of reflection about the line L =
Span(a), whereas the Householder matrix H(a) itself is the standard matrix of
reflection about L⊥ .39 In the case of R2 , this is illustrated in the picture below.
39
Alternatively, the Householder matrix H(a) is the standard matrix of the linear transformation
that first reflects about the line L = Span(a) and then reflects about the origin. As we saw above,
this linear transformation is precisely the reflection about L⊥ .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 446

x2

L
a
−H(a)x
xL xL − x

x
x1

H(a)x L⊥

Remark: Suppose that a is a non-zero vector in Rn . Then the standard matrix of

reflection about the line L := Span(a) in Rn is an orthogonal matrix. Indeed, as
we saw above, the Householder matrix H(a) is an orthogonal matrix. By Propo-
sition 6.8.2(c), it follows that −H(a) is also an orthogonal matrix, and as we saw
above, −H(a) is the standard matrix of reflection about the line L = Span(a) in Rn .

6.8.4 The Givens matrix

Given an integer n ≥ 2, indices i, j ∈ {1, . . . , n} such that i < j, and real
 numbers c
2 2

and s such that c + s = 1, we define the Givens matrix Gi,j (c, s) = gi,j n×n as
follows:

ˆ gi,i = gj,j = c;

ˆ gk,k = 1 for all k ∈ {1, . . . , n} \ {i, j};

ˆ gi,j = −s;

ˆ gj,i = s;

ˆ all other entries of the matrix are zero.

So, the matrix Gi,j (c, s) can be represented as follows:

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 447

i j

1 0 ... 0 0 0 ... 0 0 0 ... 0

0 1 ... 0 0 0 ... 0 0 0 ... 0
.. .. .. .. .. .. .. .. .. .. ..
. . . . . . . . . . .
0 0 ... 1 0 0 ... 0 0 0 ... 0
i 0 0 ... 0 c 0 . . . 0 −s 0 ... 0
0 0 ... 0 0 1 ... 0 0 0 ... 0
.. .. . .. .. .. .. .. ..
Gi,j (c, s) = . . . . . .. . . . . . . 0
0 0 ... 0 0 0 ... 1 0 0 ... 0
j 0 0 ... 0 s 0 ... 0 c 0 ... 0
0 0 ... 0 0 0 0 0 1 0
.. .. .. .. .. .. .. .. . . ..
. . . . . . . . . .
0 0 ... 0 0 0 0 0 0 ... 1

It is not hard to check that the columns of Gi,j (c, s) form an orthonormal set of
vectors in Rn , and therefore (by Proposition 6.3.4) an orthonormal basis of Rn . So,
by Theorem 6.8.1, the Givens matrix Gi,j (c, s) really is orthogonal. Let us now give
a geometric interpretation of this matrix. Since c2 + s2 = 1, we see that there exists
a real number (angle in radians) θ such that c = cos θ and s = sin θ. With this
set-up, we see that Gi,j (c, s) represents rotation about the origin by angle θ in the
xi xj -plane. This is particularly easy to see in the case when n = 2. In that case, we
have that    
c −s cos θ − sin θ
G1,2 (c, s) = = ,
s c sin θ cos θ
which is precisely the standard matrix of counterclockwise rotation about the origin
by angle θ.
x2

c = cos θ
G1,2(c, s)u s = sin θ

u
θ
x1

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 448

6.8.5 Orthogonal matrices, length, and angles

be an orthogonal matrix in Rn×n . Then all
 
Theorem 6.8.5. Let Q = qi,j n×n
the following hold:

(a) for all x, y ∈ Rn , (Qx) · (Qy) = x · y;

(b) for all x ∈ Rn , ||Qx|| = ||x||;

(c) for all i, j ∈ {1, . . . , n}, |qi,j | ≤ 1.

Proof. (a) For x, y ∈ Rn , we have the following:

(Qx) · (Qy) = (Qx)T (Qx) = xT QT Q y = xT y = x · y.

| {z }
=In

(b) For x ∈ Rn , we have the following:

p (a) √
||Qx|| = (Qx) · (Qx) = x · x = ||x||.

(c) By Theorem 6.8.1, the columns of Q form an orthonormal basis. In particular,

all columns of Q are unit vectors, and it follows that all entries of Q have absolute
value at most 1.

Remark: By Theorem 6.8.5(b), multiplication by an orthogonal matrix (on the

left) preserves vector length. On the other hand, recall that for non-zero vectors
x, y ∈ Rn , we have that x · y = ||x|| ||y|| cos θ, where θ is the angle between x and y.
So, Theorem 6.8.5(a-b) implies that multiplication (on the left) by an orthogonal
matrix preserves angles between non-zero vectors.

6.9 Scalar product, coordinate vectors, and matrices of

linear functions
Proposition 6.9.1. Let V be a real or complex vector space, equipped with the scalar
product ⟨·, ·⟩ and the induced norm ||·||, and let B = {u1 , . . . , un } be an orthonormal
basis of V . Let · be the standard scalar product in Rn or Cn (depending on whether
the vector space V is real or complex). Then for all x, y ∈ V , we have that
   
⟨x, y⟩ = x B· y B.

Proof. We prove the result for the case when V is a complex vector space.40 The
proof for the real case is similar but slightly easier (because we do not have to deal
40
In this case, · is the standard scalar product in Cn .

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 449

with complex conjugates). Fix x, y ∈ V . Since B is an orthonormal basis of V ,

Corollary 6.3.6 guarantees that
n
P n
P
x = ⟨x, ui ⟩ ui and y = ⟨y, ui ⟩ ui ,
i=1 i=1

and consequently,
   
⟨x, u1 ⟩ ⟨y, u1 ⟩

x

=  .. and

y

=  ..
.
   
B .  B .
⟨x, un ⟩ ⟨y, un ⟩
We now compute:
n
DP n
P E
⟨x, y⟩ = ⟨x, ui ⟩ ui , ⟨y, ui ⟩ ui
i=1 i=1

n
DP n
P E
= ⟨x, ui ⟩ ui , ⟨y, uj ⟩ uj
i=1 j=1

n P
P n D E
= ⟨x, ui ⟩ ui , ⟨y, uj ⟩ uj
i=1 j=1

n P
P n
= ⟨x, ui ⟩⟨y, uj ⟩ ⟨ui , uj ⟩
i=1 j=1

(∗) n
P
= ⟨x, ui ⟩⟨y, ui ⟩
i=1
   
⟨x, u1 ⟩ ⟨y, u1 ⟩
.. ..
= ·
   
 . . 
⟨x, un ⟩ ⟨y, un ⟩
   
= x B
· y B
,

where (*) follows from the fact that B = {u1 , . . . , un } is an orthonormal set.

Theorem 6.9.2. Let U and V be non-trivial, finite-dimensional real vector spaces.

Assume that U is equipped with a scalar product ⟨·, ·⟩U and the induced norm || · ||U ,
and that V is equipped with a scalar product ⟨·, ·⟩V and the induced norm || · ||V .
Let BU = {u1 , . . . , um } and BV = {v1 , . . . , vn } be orthonormal bases of U and
V , respectively, and let f : U → V be a linear function. Then the following two
statements are equivalent:

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 450

 
(i) the columns of the n × m matrix B f B form an orthonormal set of vectors
V U
in Rn (with respect to the standard scalar product · and the induced norm
|| · ||);41

(ii) f preserves the scalar product, that is, for all vectors x, y ∈ U , we have that
f (x), f (y) V = ⟨x, y⟩U .
   
Proof. Set B f B = c1 . . . cm . We observe that
V U

cT1
 
 cT2 
)T
    
(B f f =   c1 c2 . . . cm
 
V BU BV BU ..
 . 
cTm
 
c1 · c1 c1 · c2 ... c1 · cm
 c2 · c1 c2 · c2 ... c2 · cm 
=  .
 
.. .. .. ..
 . . . . 
cm · c1 cm · c2 . . . cm · cm

)T
   
So, we see that (i) holds if and only if ( B f BU BV
f BU
= Im .
V

Next, by Proposition 6.9.1, the following hold for all x, y ∈ U :

   
(1) ⟨x, y⟩U = x B · y B ;
U U
   
(2) f (x), f (y) V
= f (x) BV
· f (y) BV
.

Now, for all x, y ∈ U , we have that

(2)    
f (x), f (y) V
= f (x) BV
· f (y) BV

 
T  
= f (x) BV
f (y) BV

     T      
= BV
f BU
x BU BV
f BU
y BU

 
T  
T    
= x BU BV
f BU BV
f BU
y BU
.

)T
   
Suppose first that (i) holds. Then ( B f BU BV
f BU
= Im , and conse-
V

41
 
However, despite Theorem 6.8.1, this does not necessarily mean that the matrix B f B is
  V U
orthogonal. This is because B f B is an n × m matrix, and it is possible that m ̸= n, in which
  V U
case B f B is not a square matrix. Only square matrices can be orthogonal!
V U

Irena Penev
Linear Algebra 1 & 2
Chapter 6. Scalar (inner) products, norms, and orthogonality 451

quently, for all x, y ∈ U , we have that

T
)T
       
f (x), f (y) V
= x BU
(B f BU BV
f BU
y BU
| V {z }
=Im

 
T  
= x BU
y BU
   
= x BU
· y BU

(1)
= ⟨x, y⟩U .

Thus, (ii) holds.

Suppose now that (ii) holds. Then for all i, j ∈ {1, . . . , m}, we have that

em m
   
i · ej = ui B · uj B
U U

(1)
= ⟨ui , uj ⟩U

(ii)
= f (ui ), f (uj ) V

(2)    
= f (ui ) BV
· f (uj ) BV
   
   

= BV
f BU
ui BU
· BV
f BU
uj BU

em em
 
 

= BV
f BU i · BV
f BU j

= ci · cj .

So, for all distinct i, j ∈ {1, . . . , n}, we have that ci · cj = em m

i · ej = 0, and it
follows that {c1 , . . . , cn } is an orthogonal set of vectors
p in Rn . On the other hand,
√ m
for all i ∈ {1, . . . , m}, we have that ||ci || = ci · ci = ei · em
m
i = ||ei || = 1. Thus,
n
{c1 , . . . , cn } is an orthonormal set of vectors in R , that is, (i) holds.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 452

Chapter 7

Determinants

7.1 Determinants: definition, examples, and basic prop-

erties
 
The determinant of a matrix A = ai,j n×n
with entries in some field F, denoted
by det(A) or |A|, is defined by
P n
Q
det(A) := sgn(σ) ai,σ(i)
σ∈Sn i=1

P
= sgn(σ)a1,σ(1) a2,σ(2) . . . an,σ(n) .
σ∈Sn

Let us try to explain this definition. Each permutation σ ∈ Sn gives us one way
of selecting one entry of A out of each row and each column: we select entries
a1,σ(1) , . . . , an,σ(n) , multiply them together, and then multiply that product by sgn(σ),
which yields the product sgn(σ)a1,σ(1) . . . an,σ(n) .1 We then sum up all products of
this type (there are |Sn | = n! many of them), and we obtain the determinant of our
matrix.
Note that if the entries of our square matrix belong to a field of characteristic 2
(i.e. a field in which 1 + 1 = 0, such as the field Z2 ), then 1 = −1, and so sgn(σ) can
be ignored (because it is always equal to 1). However, if our field is of characteristic
 
1 1 2 3 4
For example, for n = 4 and σ = = (134)(2), we select the boxed entries
3 2 4 1
below,
a1,1 a1,2 a1,3 a1,4
 
 a2,1 a2,2 a2,3 a2,4 
A = ,
 
a3,1 a3,2 a3,3 a3,4

 
a4,1 a4,2 a4,3 a4,4
and we obtain the product sgn(σ)a1,3 a2,2 a3,4 a4,1 = a1,3 a2,2 a3,4 a4,1 , since sgn(σ) = 1.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 453

other than 2 (i.e. if 1 + 1 ̸= 0 in our field, and consequently, 1 ̸= −1), then we must
keep track of sgn(σ) in each summand from the definition of a determinant. (For a
more detailed discussion of the characteristic of a field, see subsection 2.4.4.)

Notation: We typically write

a1,1 a1,2 . . . a1,n

a2,1 a2,2 . . . a2,n
.. .. .. ..
. . . .
an,1 an,2 . . . an,n

instead of  
a1,1 a1,2 . . . a1,n
 a2,1 a2,2 . . . a2,n 
det   .
 
.. .. .. ..
 . . . . 
an,1 an,2 . . . an,n

Remark: Only square matrices have determinants. Moreover, the determinant of

a matrix in Fn×n is always a scalar in F.

The following proposition easily follows from the definition of a determinant. We

note that permutation matrices were discussed in subsection 2.3.7.

Proposition 7.1.1. Let n be a positive integer, and let π ∈ Sn , and consider the
matrix Pπ of the permutation π (where the 0’s and 1’s in Pπ can be considered as
belonging to an arbitrary field F). Then

det(Pπ ) = sgn(π).
 
Proof. Set Pπ = pi,j n×n
, so that

1 if j = π(i)
pi,j =
0 if j ̸= π(i)

for all i, j ∈ {1, . . . , n}. By definition,

P
det(Pπ ) = sgn(σ)p1,σ(1) p2,σ(2) . . . pn,σ(n) .
σ∈Sn

The only permutation σ ∈ Sn for which none of p1,σ(1) , p2,σ(2) , . . . , pn,σ(n) is 0 is the
permutation σ = π. So,
(∗)
det(Pπ ) = sgn(π)p1,π(1) p2,π(2) . . . pn,π(n) = sgn(π),

where (*) follows from the fact that pi,π(i) = 1 for all i ∈ {1, . . . , n}.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 454

Remark: Note that the identity matrix In is the matrix of the identity permutation
1 in Sn . Since sgn(1) = 1, Proposition 7.1.1 guarantees that det(In ) = 1.

Proposition 7.1.2. We have the following formulas for the determinants of 1 × 1,

2 × 2, and 3 × 3 matrices (with entries in some field F):

(a) a1,1 = a1,1 ;2

a1,1 a1,2
(b) = a1,1 a2,2 − a1,2 a2,1 ;
a2,1 a2,2

a1,1 a1,2 a1,3 

a1,1 a2,2 a3,3 + a1,2 a2,3 a3,1 + a1,3 a2,1 a3,2
(c) a2,1 a2,2 a2,3 =
−a1,3 a2,2 a3,1 − a1,1 a2,3 a3,2 − a1,2 a2,1 a3,3 .
a3,1 a3,2 a3,3

Proof. (a) S1 has just one element, namely σ1 = (1), with sgn(σ1 ) = 1. So, we have
that
a1,1 = sgn(σ1 )a1,σ1 (1) = a1,1 .

(b) S2 has two elements, listed below, along with their signs.
 
1 2
ˆ σ1 = = (1)(2), with sgn(σ1 ) = 1;
1 2
 
1 2
ˆ σ2 = = (12), with sgn(σ2 ) = −1.
2 1
So, we have that

a1,1 a1,2
= sgn(σ1 )a1,σ1 (1) a2,σ1 (2) + sgn(σ2 )a1,σ2 (1) a2,σ2 (2)
a2,1 a2,2
= a1,1 a2,2 − a1,2 a2,1 .

(c) S3 has six elements, listed below, along with their signs.
 
1 2 3
ˆ σ1 = = (1)(2)(3), with sgn(σ1 ) = 1;
1 2 3
 
1 2 3
ˆ σ2 = = (123), with sgn(σ2 ) = 1;
2 3 1
 
1 2 3
ˆ σ3 = = (132), with sgn(σ3 ) = 1;
3 1 2
2
Be careful not to confuse this with the absolute value! (The notation is admittedly some-
what unfortunate/ambiguous.)

If there is any danger of confusion, it is always possible to write
det a1,1 instead of a1,1 .

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 455

 
1 2 3
ˆ σ4 = = (13)(2), with sgn(σ4 ) = −1;
3 2 1
 
1 2 3
ˆ σ5 = = (1)(23), with sgn(σ5 ) = −1;
1 3 2
 
1 2 3
ˆ σ6 = = (12)(3), with sgn(σ6 ) = −1.
2 1 3
So, we have that


 sgn(σ1 )a1,σ1 (1) a2,σ1 (2) a3,σ1 (3)

 +sgn(σ2 )a1,σ2 (1) a2,σ2 (2) a3,σ2 (3)
a1,1 a1,2 a1,3


+sgn(σ3 )a1,σ3 (1) a2,σ3 (2) a3,σ3 (3)

a2,1 a2,2 a2,3 =
+sgn(σ4 )a1,σ4 (1) a2,σ4 (2) a3,σ4 (3)
a3,1 a3,2 a3,3


+sgn(σ5 )a1,σ5 (1) a2,σ5 (2) a3,σ5 (3)





+sgn(σ6 )a1,σ6 (1) a2,σ6 (2) a3,σ6 (3)


 a1,1 a2,2 a3,3



 +a1,2 a2,3 a3,1
+a1,3 a2,1 a3,2

=

 −a1,3 a2,2 a3,1
−a1,1 a2,3 a3,2




−a1,2 a2,1 a3,3 .


Determinants of 2 × 2 and 3 × 3 matrices can be represented schematically, as

shown below.
+ + + − − −
+ − a1,1 a1,2 a1,3 a1,1 a1,2
a1,1 a1,2
a2,1 a2,2 a2,3 a2,1 a2,2
a2,1 a2,2
a3,1 a3,2 a3,3 a3,1 a3,2

We multiply the entries along each of the red lines and add them up, and then we
multiply the entries along each of the blue lines and subtract them. In each case,
the result we get is precisely the formula from Proposition 7.1.2. For example, we
can compute the determinant of the matrix
 
1 2
A =
3 4

in R2×2 by forming the diagram

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 456

+ −
1 2
3 4

and the computing

1 2
det(A) = = 1 · 4 − 2 · 3 = −2.
3 4

Similarly, we can compute the determinant of the matrix

 
1 2 3
B =  4 5 6 
7 8 9

in R3×3 by forming the diagram

+ + + − − −
1 2 3 1 2
4 5 6 4 5
7 8 9 7 8

and then computing

1 2 3
det(B) = 4 5 6
7 8 9

= 1·5·9+2·6·7+3·4·8−3·5·7−1·6·8−2·4·9

= 0.

Warning: Do not try this with matrices of larger size!

Theorem 7.1.3. Let F be a field. For all A ∈ Fn×n , we have that

det(AT ) = det(A).

Proof. We set A = ai,j n×n and AT = aTi,j n×n . So, for all i, j ∈ {1, . . . , n},
   

we have aTi,j = aj,i . Now, we compute:

n
det(AT ) aTi,σ(i)
P Q
= sgn(σ)
σ∈Sn i=1

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 457

P n
Q
= sgn(σ) aσ(i),i
σ∈Sn i=1

P n
Q
= sgn(σ) aj,σ−1 (j)
σ∈Sn j=1

(∗) n
sgn(σ −1 )
P Q
= aj,σ−1 (j)
σ∈Sn j=1

P n
Q
= sgn(π) aj,π(j)
π∈Sn j=1

= det(A),

where (*) follows from Proposition 2.3.2.

7.1.1 Some matrices whose determinants are zero

Proposition 7.1.4. Let F be a field, and let A = ai,j n×n be a matrix in Fn×n .
 

If A has a zero row or a zero column,3 then det(A) = 0.

Proof. In view of Theorem 7.1.3, it suffices to consider the case when A has a zero
row.4 Suppose that that the p-th row of A is a zero row. Then for all σ ∈ Sn , we
have that ap,σ(p) = 0. Consequently,
P
det(A) = sgn(σ)a1,σ(1) . . . ap,σ(p) . . . an,σ(n) = 0,
σ∈Sn | {z }
=0

which is what we needed to show.

Proposition 7.1.5. Let F be a field, and let A = ai,j n×n be a matrix in Fn×n .
 

If A has two identical rows or two identical columns, then det(A) = 0.

Proof. In view of Theorem 7.1.3, it suffices to consider the case when A has two
identical rows.5 So, suppose that for some distinct p, q ∈ {1, . . . , n}, the p-th and
3
A zero row is a row in which all entries are zero. Similarly, a zero column is a column in which
all entries are zero.
4
Let us explain this in more detail. Suppose we have shown that if a matrix in Fn×n has a
zero row, then its determinant is zero. Suppose now that B is a matrix in Fn×n that has a zero
(∗) (∗∗)
column. Then B T has a zero row, and we see that det(B) = det(B T ) = 0, where (*) follows from
Theorem 7.1.3, and (**) follows from the fact that B T has a zero row.
5
Let us explain this in more detail. Suppose we have shown that if a matrix in Fn×n has two
identical rows, then its determinant is zero. Let us prove this for matrices with two identical columns.
Suppose B is a matrix in Fn×n with two identical columns. Then B T has two identical rows, and
(∗) (∗∗)
we see that det(B) = det(B T ) = 0, where (*) follows from Theorem 7.1.3, and (**) follows from
the fact that B T has two identical rows.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 458

q-th row of A are the same. (In particular, n ≥ 2.) Now, let An be the alternating
group of degree n, i.e. the group of all even permutations in Sn , and let On be the
set of all odd permutations in Sn .6 Obviously,

Sn = An ∪ On and An ∩ On = ∅.

Next, consider the transposition τ = (pq). By Proposition 2.3.2, for all σ ∈ Sn , we

have that sgn(σ ◦ τ ) = −sgn(σ); it then readily follows that On = {σ ◦ τ | σ ∈ An },7
and obviously, for all distinct σ1 , σ2 ∈ An , we have that σ1 ◦ τ ̸= σ2 ◦ τ .8
Qn Qn
Claim. For all σ ∈ Sn , we have that i=1 ai,σ(i) = i=1 ai,σ◦τ (i) .

Proof of the Claim. Fix σ ∈ Sn . First, note that

(∗)
ˆ ap,σ(p) = ap,σ◦τ (q) = aq,σ◦τ (q) ,
(∗)
ˆ aq,σ(q) = aq,σ◦τ (p) = ap,σ◦τ (p) ,

where both instances of (*) follow from the fact that the p-th and q-th row of A
are the same. So, ap,σ(p) aq,σ(q) = ap,σ◦τ (p) aq,σ◦τ (q) . On the other hand, it is clear
Qn for all i ∈
that Qn{1, . . . , n} \ {p, q}, we have that ai,σ(i) = ai,σ◦τ (i) . It follows that
i=1 ai,σ(i) = i=1 ai,σ◦τ (i) , which is what we needed to show. ♦

We now compute:
P
det(A) = sgn(σ)a1,σ(1) . . . an,σ(n)
σ∈Sn

P P
= sgn(σ) a1,σ(1) . . . an,σ(n) + sgn(π) a1,π(1) . . . an,π(n)
σ∈An | {z } π∈On | {z }
=1 =−1

6
Unlike An , On is not a group.
7
Let us check this. First, suppose that σ ∈ An . Then
(∗) (∗∗)
sgn(σ ◦ τ ) = −sgn(σ) = −1,

where (*) follows from Proposition 2.3.2, and (**) follows from the fact that σ is even. So, σ ◦ τ ∈ On .
Conversely, suppose that π ∈ On . Set σ := π ◦ τ . Then
(∗) (∗∗)
sgn(σ) = sgn(π ◦ τ ) = −sgn(π) = 1,

where (*) follows from Proposition 2.3.2, and (**) follows from the fact that π is odd. So, σ ∈ An .
But τ is a transposition, and consequently, τ −1 = τ . So, π = π ◦ τ ◦ τ = σ ◦ τ .
8
This follows from the fact that τ is a bijection. So, if we had that σ1 ◦ τ = σ2 ◦ τ , then we would
have that
σ1 = σ1 ◦ τ ◦ τ −1 = σ2 ◦ τ ◦ τ −1 = σ2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 459

P P
= a1,σ(1) . . . an,σ(n) − a1,π(1) . . . an,π(n)
σ∈An π∈On

P P
= a1,σ(1) . . . an,σ(n) − a1,σ◦τ (1) . . . an,σ◦τ (n)
σ∈An σ∈An

(∗)
= 0,

where (*) follows from the Claim.

7.2 The linearity of determinants in one row or one

column
In general, for matrices A, B ∈ Fn×n (where F is some field) and a scalar α ∈ F, we
have that

det(A + B) Z
= det(A) + det(B)
 and det(αA) Z
= αdet(A).


We do, however, have the following proposition.

Proposition 7.2.1. Let F be a field, and let a1 , . . . , ap−1 , ap+1 , . . . , an ∈ Fn . Then

both the following hold:

(a) the function fCp : Fn → F given by

 
fCp (x) = det a1 . . . ap−1 x ap+1 . . . an

for all x ∈ Fn is linear;

(b) the function fRp : Fn → F given by

aT1
 
 .. 
 . 
 T 
 ap−1
 
T

 x
fRp (x) = det  

 aT 
 p+1 
 .
 ..


aTn

for all x ∈ Fn is linear.

Remark: Before reading the proof, the reader might want to take a look at
Example 7.2.2 (below), since it illustrates how Proposition 7.2.1 can be used in
practice.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 460

Proof. Clearly, (b) and Theorem 7.1.3 imply (a).9 So, it suffices to prove (b).
We first set up some notation. For each index i ∈ {1, . . . , n} \ {p}, we set
T
, so that aTi = ai,1 . . . ai,n . Now, let us prove that
  
ai = ai,1 . . . ai,n
fRp is linear.
T T
1. Fix x, y ∈ Fn , and set x = x1 . . . xn
 
and y = y1 . . . yn . We
compute:

aT1
 
a1,1 ... a1,n
 ..  .. . .. ..
 .  . .
T
 
 ap−1

 ap−1,1 . . . ap−1,n
(x + y) T  x
fRp (x + y) = det    = 1 + y1 . . . xn + yn
 aT  a p+1,1 . . . ap+1,n
 p+1 
 .
..
 .
.. .. ..
  . .
aTn an,1 ... an,n

P  
= sgn(σ)a1,σ(1) . . . ap−1,σ(p−1) xσ(p) + yσ(p) ap+1,σ(p+1) . . . an,σ(n)
σ∈Sn

P
= sgn(σ)a1,σ(1) . . . ap−1,σ(p−1) xσ(p) ap+1,σ(p+1) . . . an,σ(n)
σ∈SP
n
+ sgn(σ)a1,σ(1) . . . ap−1,σ(p−1) yσ(p) ap+1,σ(p+1) . . . an,σ(n)
σ∈Sn

a1,1 . . . a1,n a1,1 . . . a1,n

.. .. .. .. .. ..
. . . . . .
ap−1,1 . . . ap−1,n ap−1,1 . . . ap−1,n
= x1 ... xn + y1 ... yn
ap+1,1 . . . ap+1,n ap+1,1 . . . ap+1,n
.. .. .. .. .. ..
. . . . . .
an,1 . . . an,n an,1 . . . an,n

aT1 aT1
   
 ..   .. 
 .   . 
 T   T 
 ap−1
  ap−1
  
T
 
 + det  yT
=  x
det  = fRp (x) + fRp (y).

  
 aT   aT 
 p+1   p+1 
 .  .
 ..  ..
 
 
aTn aTn

9
Details?

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 461

T
2. Fix x ∈ Fn and α ∈ F, and set x =

x1 . . . xn . We compute:

aT1
 
a1,1 . . . a1,n
 ..  .. .. ..
 .  . . .
 T 
 ap−1
 ap−1,1 . . . ap−1,n

T

fRp (αx) =  αx
det  = αx1 . . . αxn


 aT  ap+1,1 . . . ap+1,n
 p+1 
 . .. .. ..
 ..

 . . .
aTn an,1 . . . an,n

P  
= sgn(σ)a1,σ(1) . . . ap−1,σ(p−1) αxσ(p) ap+1,σ(p+1) . . . an,σ(n)
σ∈Sn

P
= α sgn(σ)a1,σ(1) . . . ap−1,σ(p−1) xσ(p) ap+1,σ(p+1) . . . an,σ(n)
σ∈Sn

aT1
 
a1,1 . . . a1,n
.. .. ..  .. 
. . .  . 
 T 
ap−1,1 . . . ap−1,n  ap−1
 
T

= α x1 ... xn  x
= α det  
 = αfRp (x).
ap+1,1 . . . ap+1,n  aT 
 p+1 
.. .. ..  .
 ..

. . . 
an,1 . . . an,n aTn

By 1. and 2., fRp is linear, i.e. (b) holds.

Example 7.2.2. By Proposition 7.2.1, we have the following (entries are understood
to be in R, and the row/column being manipulated is in red to facilitate reading):

1 2 1 1 1 1 1 1 1
ˆ 2 3 4 = 2 2 4 + 2 1 4 ;
0 1 5 0 −2 5 0 3 5

3 2 4 1 2 4
ˆ 6 −1 0 =3 2 −1 0 ;
−3 0 5 −1 0 5

1 2 3 1 2 3 1 2 3
ˆ 2 2 3 = 2 2 3 + 2 2 3 ;
7 3 −2 4 4 −2 3 −1 0

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 462

2 −2 4 1 −1 2
ˆ 1 0 −2 =2 1 0 −2 .
2 1 4 2 1 4

As an easy corollary of Proposition 7.2.1, we obtain the following.

Proposition 7.2.3. Let F be a field, let A ∈ Fn×n , and let α ∈ F. Then

det αA = αn det(A).

Proof. We apply Proposition 7.2.1 n times, once to each row (or alternatively, once
to each column) of αA, and the result follows.10

7.3 Computing determinants via elementary row and

column operations
In this section, we examine how performing elementary row and column operations
(introduced in subsections 1.3.2 and 3.3.2, respectively) affects the value of the
determinant, and how we can use these operations to compute the determinant of a
square matrix. As we pointed out in subsection 3.3.2, elementary column operations
should not be used for solving linear systems. However, it turns out that both
elementary row operations and elementary column operations behave well with
respect to determinants, i.e. they change the value of the determinant in a way that
we can describe precisely, as we shall see.

7.3.1 Computing the determinant of a triangular matrix

in Fn×n (where F is some field), we say that
 
Given a square matrix A = ai,j n×n

ˆ A is upper triangular if all entries of A below the main diagonal are zero, i.e. if
for all i, j ∈ {1, . . . , n} such that i > j, we have that ai,j = 0;

ˆ A is lower triangular if all entries of A above the main diagonal are zero, i.e. if
for all i, j ∈ {1, . . . , n} such that i < j, we have that ai,j = 0;

ˆ A is triangular if it is upper triangular or lower triangular.

A schematic representation of an upper triangular and a lower triangular matrix is

given below (*’s represent arbitrary elements of the field F, and the main diagonal is
in red in both cases).
10
Details?

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 463

   
∗ ∗ ∗ ... ∗ ∗ ∗ 0 0 ... 0 0

 0 ∗ ∗ ... ∗ ∗  

 ∗ ∗ 0 ... 0 0  

 0 0 ∗ ... ∗ ∗  

 ∗ ∗ ∗ ... 0 0  
 .. .. .. . . .. ..   .. .. .. . . .. .. 

 . . . . . . 

 . . . . . . 
 0 0 0 ... ∗ ∗   ∗ ∗ ∗ ... ∗ 0 
0 0 0 ... 0 ∗ ∗ ∗ ∗ ... ∗ ∗

upper triangular matrix lower triangular matrix

Note that any square matrix in row echelon form is in fact an upper triangular
matrix. (However, not all upper triangular matrices are in row echelon form.) So,
the row reduction algorithm performed on a square matrix will, in particular, yield
an upper triangular matrix.
It turns out that the determinant of any triangular matrix is particularly easy to
compute, as we now show.
 
Proposition 7.3.1. Let F be a field, and let A = ai,j n×n be a triangular matrix
in Fn×n . Then
n
Q
det(A) = ai,i = a1,1 a2,2 . . . an,n ,
i=1
that is, det(A) is equal to the product of entries on the main diagonal of A.
Remark: For example, we can compute the determinants of the following matrices
in R3×3 as follows:

1 2 3 1 0 0
ˆ 0 4 5 = 1 · 4 · 6 = 24; ˆ 2 3 0 = 1 · 3 · 6 = 18.
0 0 6 4 5 6

Proof. Note that the transpose of an upper triangular matrix is a lower triangular
matrix, and moreover, the main diagonal remains unchanged when we take the
transpose of a square matrix. So, in view of Theorem 7.1.3, it suffices to prove the
result for the case when A is lower triangular. Now, note that for all σ ∈ Sn \ {1},11
there exists some index i ∈ {1, . . . , n} such that i < σ(i),12 and consequently,
ai,σ(i) = 0 (since A is lower triangular). It follows that for all σ ∈ Sn \ {1}, we have
that a1,σ(1) a2,σ(2) . . . an,σ(n) = 0, and consequently,

11
Recall that 1 is the identity permutation in Sn .
12
This is “obvious,” but here is a formal proof. Fix a permutation σ ∈ Sn \ {1}, and let
i ∈ {1, . . . , n} be minimal with the property that σ(i) ̸= i. Set j := σ(i). Then
ˆ σ(1) = 1, . . . , σ(i − 1) = i − 1,
ˆ σ(i) = j ̸= i.
If j < i, then σ(j) = j = σ(i), contrary to the fact that σ is a permutation (and in particular,
one-to-one). So, j > i, i.e. σ(i) > i.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 464

P
det(A) = sgn(σ)a1,σ(1) a2,σ(2) . . . an,σ(n)
σ∈Sn

= sgn(1)a1,1 a2,2 . . . an,n

= a1,1 a2,2 . . . an,n .

This completes the argument.

7.3.2 Determinants and elementary row and column operations

be a matrix in Fn×n .
 
Theorem 7.3.2. Let F be a field, and let A = ai,j n×n
Then all the following hold:

(a) if a matrix B is obtained by swapping two rows or swapping two columns of A,

then
det(B) = −det(A);

(b) if a matrix B is obtained by multiplying some row or some column of A by a

scalar α ∈ F \ {0}, then

det(B) = αdet(A) and det(A) = α−1 det(B);

(c) if a matrix B is obtained from A by adding a scalar multiple of one row (resp.
column) of A to another row (resp. column) of A, then

det(B) = det(A).

Proof. In view of Theorem 7.1.3, it suffices to prove the result for row operations
only.
(a) Fix distinct indices p, q ∈ {1, . . . , n}, and suppose
  that B is obtained by
swapping rows p and q of A (“Rp ↔ Rq ”). Set B = bi,j n×n , so that

ˆ for all j ∈ {1, . . . , n}, we have that bp,j = aq,j and bq,j = ap,j ;

ˆ for all i ∈ {1, . . . , n} \ {p, q} and j ∈ {1, . . . , n}, we have that bi,j = ai,j .

Next, consider the transposition τ = (pq) in Sn .

n
Q n
Q
Claim. For all σ ∈ Sn , we have that bi,σ(i) = ai,σ◦τ (i) .
i=1 i=1

Proof of the Claim. First, we note that

ˆ bp,σ(p) = aq,σ(p) = aq,σ◦τ (q) ;

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 465

ˆ bq,σ(q) = ap,σ(q) = ap,σ◦τ (p) .

So, bp,σ(p) bq,σ(q) = ap,σ◦τ (p) aq,σ◦τ (q) . On the other hand, for all i ∈ {1, . . . , n} \ {p, q},
n
Q n
Q
we have that bi,σ(i) = ai,σ◦τ (i) . It follows that bi,σ(i) = ai,σ◦τ (i) , which is what
i=1 i=1
we needed to show. ♦
We now compute:
P Qn
det(B) = sgn(σ) i=1 bi,σ(i)
σ∈Sn

(∗) P Qn
= sgn(σ) i=1 ai,σ◦τ (i)
σ∈Sn

(∗∗) P  Q
n
= − sgn(σ ◦ τ ) i=1 ai,σ◦τ (i)
σ∈Sn

P Qn
= − sgn(σ ◦ τ ) i=1 ai,σ◦τ (i)
σ∈Sn

P Qn
= − sgn(π) i=1 ai,π(i)
π∈Sn

= −det(A),

where (*) follows from the Claim, and (**) follows from Proposition 2.3.2. This
proves (a).
(b) Fix an index p ∈ {1, . . . , n} and a scalar α ∈ F \ {0}, and suppose that B is ob-
tained by multiplying the p-th row of A by α (“Rp → αRp ”). By Proposition 7.2.1(b),
the determinant is linear in the p-th row, and we deduce that det(B) = αdet(A).
Since α ̸= 0, we deduce that det(A) = α−1 det(B). This proves (b).
(c) Fix distinct indices p, q ∈ {1, . . . , n} and a scalar α ∈ F, and suppose thatB is
obtained by adding α times row p to row q (“Rq → Rq + αRp ”). Set B = bi,j n×n ,
so that

ˆ for all j ∈ {1, . . . , n}, we have that bq,j = aq,j + αap,j ;

ˆ for all i ∈ {1, . . . , n} \ {q} and j ∈ {1, . . . , n}, we have that bi,j = ai,j .

We now compute (the q-th row is in red for emphasis):

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 466

a1,1 ... a1,n

.. .. ..
. . .
aq−1,1 ... aq−1,n
det(B) = aq,1 + αap,1 . . . aq,n + αap,n
aq+1,1 ... aq+1,n
.. .. ..
. . .
an,1 ... an,n

a1,1 . . . a1,n a1,1 . . . a1,n

.. .. .. .. .. ..
. . . . . .
aq−1,1 . . . aq−1,n aq−1,1 . . . aq−1,n
(∗)
= aq,1 . . . aq,n +α ap,1 . . . ap,n
aq+1,1 . . . aq+1,n aq+1,1 . . . aq+1,n
.. .. .. .. .. ..
. . . . . .
an,1 . . . an,n an,1 . . . an,n
| {z } | {z }
=det(A) (∗∗)
= 0

= det(A),

where (*) follows from the fact that the determinant is linear in the q-th row (by
Proposition 7.2.1(b)), and (**) follows from the fact that any square matrix with
two identical rows (in this case, the p-th and q-th row) has determinant zero (by
Proposition 7.1.5).

Example 7.3.3. Compute the determinant of the matrix below (with entries under-
stood to be in R).  
2 4 6
A =  2 4 4 
3 3 7

Solution. We perform elementary row operations on A (keeping track of the way

that this changes the value of the determinant, as per Theorem 7.3.2) until we
transform A into a matrix in row echelon form. Square matrices in row echelon form
are upper triangular, and so by Proposition 7.3.1, we can obtain their determinant
by multiplying the entries on the main diagonal. We now compute:

2 4 6
det(A) = 2 4 4
3 3 7

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 467

2 4 6
R2 →R2 −R1
= 0 0 −2
3 3 7

2 4 6
R2 ↔R3
= − 3 3 7
0 0 −2

R1 → 21 R1
1 2 3
= −2 3 3 7
0 0 −2

1 2 3
R2 →R1 −3R1
= −2 0 −3 −2
0 0 −2

(∗)
= (−2)1(−3)(−2)

= −12,

where (*) follows by taking the determinant of an upper triangular matrix.

Example 7.3.4. Compute the determinant of the matrix below (with entries under-
stood to be in Z3 ).  
1 2 1 1 2
 1 1 0 2 1 
 
A =   2 0 1 1 2 

 2 2 0 0 1 
1 0 2 1 2
Solution. Here, we just notice that the second column is the sum of the first and third.
This allows us to turn the second column into a zero column via two elementary
column operations, which implies that det(A) = 0. The detailed computation is as
follows:
1 2 1 1 2
1 1 0 2 1
det(A) = 2 0 1 1 2
2 2 0 0 1
1 0 2 1 2

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 468

1 1 1 1 2
1 0 0 2 1
C2 →C2 −C1
= 2 1 1 1 2
2 0 0 0 1
1 2 2 1 2

1 0 1 1 2
1 0 0 2 1
C2 →C2 −C3
= 2 0 1 1 2
2 0 0 0 1
1 0 2 1 2

(∗)
= 0,

where (*) follows from the fact that a matrix with a zero column has determinant
zero (by Proposition 7.1.4). We could also have noticed that the matrix in the second
line of the computation above has two identical columns, and so by Proposition 7.1.5,
its determinant is zero. Finally, we note that our two elementary column operations
could also have been written as “C2 → C2 + 2C1 ” and “C2 → C2 + 2C3 ,” since in
Z3 , we have that −1 = 2.

7.4 Determinants and matrix invertibility

Theorem 7.4.1. Let F be a field, and let A ∈ Fn×n . Then A is invertible if and
only if det(A) ̸= 0.

Proof. We can transform A into a matrix in reduced row echelon form via a sequence
of elementary row operations. By Theorem 7.3.2, each elementary row operation has
the effect of multiplying the value of the determinant by some non-zero
scalar. So,
there exists some scalar α ∈ F \ {0} such that

det(A) = αdet RREF(A) . Therefore,
det(A) = 0 if and only if det RREF(A) = 0. Moreover, RREF(A) is an upper
triangular matrix, and so (by Proposition 7.3.1) its determinant is zero if and only if
at least one entry on its main diagonal is zero. We now have the following sequence
of equivalent statements:


det(A) = 0 ⇐⇒ det RREF(A) = 0

at least one entry on the main

⇐⇒
diagonal of RREF(A) is 0

(∗)
⇐⇒ RREF(A) ̸= In

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 469

(∗∗)
⇐⇒ A is not invertible,

where (*) follows from the fact that RREF(A) is a square matrix in reduced row ech-
elon form, and (**) follows from the Invertible Matrix Theorem (see subsection 1.11.7
or 3.3.7). It now obviously follows that A is invertible if and only if det(A) ̸= 0,
which is what we needed to show.

7.4.1 The Invertible Matrix Theorem (version 3)

In this subsection, we expand the previous version of the Invertible Matrix Theorem
(see subsection 3.3.7) to include Theorem 7.4.1.
The Invertible Matrix Theorem (version 3). Let F be a field, and let A ∈ Fn×n
be a square matrix. Further, let f : Fn → Fn be given by f (x) = Ax for all x ∈ Fn .13
Then the following are equivalent:
(a) A is invertible (i.e. A has an inverse);

(b) AT is invertible;

(c) RREF(A) = In ;

= In B for some matrix B ∈ Fn×n ;

 
 
(d) RREF A In

(e) rank(A) = n;

(f ) rank(AT ) = n;

(g) A is a product of elementary matrices;

(h) the homogeneous matrix-vector equation Ax = 0 has only the trivial solution (i.e.
the solution x = 0);

(i) there exists some vector b ∈ Fn such that the matrix-vector equation Ax = b
has a unique solution;

(j) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has a unique solution;

(k) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has at most one
solution;

(l) for all vectors b ∈ Fn , the matrix-vector equation Ax = b is consistent;

(m) f is one-to-one;

(n) f is onto;
13
Since f is a matrix transformation, Proposition 1.10.4 guarantees that f is linear. Moreover, A
is the standard matrix of f .

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 470

(o) f is an isomorphism;

(p) there exists a matrix B ∈ Fn×n such that BA = In (i.e. A has a left inverse);

(q) there exists a matrix C ∈ Fn×n such that AC = In (i.e. A has a right inverse);

(r) the columns of A are linearly independent;

(s) the columns of A span Fn (i.e. Col(A) = Fn );

(t) the columns of A form a basis of Fn ;

(u) the rows of A are linearly independent;

(v) the rows of A span F1×n (i.e. Row(A) = F1×n );

(w) the rows of A form a basis of F1×n ;

(x) Nul(A) = {0} (i.e. dim Nul(A) = 0);

(y) det(A) ̸= 0.

Proof. Items (a)-(x) are the same as those from the Invertible Matrix Theorem
(version 2) from subsection 3.3.7. The equivalence of (a) and (y) follows from
Theorem 7.4.1.

7.5 The multiplicative property of determinants

Suppose that F is some field. In general, for matrices A, B ∈ Fn×n and a scalar
α ∈ F, we have that

det(A + B) Z
= det(A) + det(B)
 and det(αA) Z
= αdet(A).


However, as we shall see (see Theorem 7.5.2 below), we do have that

det(AB) = det(A)det(B).

We begin with a technical proposition. (Recall from subsection 1.11.5 that an

elementary matrix is any matrix obtained by performing one elementary row operation
on an identity matrix In .14 )

Proposition 7.5.1. Let F be a field, let A, E ∈ Fn×n , and assume that E is an

elementary matrix. Then det(EA) = det(E)det(A).
14
Here, it is possible that E = In . In this case, we can take R to be the multiplication of the first
row by the scalar 1.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 471

Proof. Let R be an elementary row operation that corresponds to the elementary ma-
trix E, i.e. E is the matrix obtained by performing R on In . By Proposition 1.11.11(a),
EA is the matrix obtained by performing R on A. Now, by Theorem 7.3.2, there
exists some scalar α ∈ F \ {0} such that for any matrix M ∈ Fn×n , the determinant
of the matrix obtained by performing the elementary row operation R on M is
αdet(M ). So,

ˆ det(E) = αdet(In ) = α;

ˆ det(EA) = αdet(A).

It follows that
det(EA) = αdet(A) = det(E)det(A),
which is what we needed to show.

We are now ready to prove the multiplicative property of determinants.

Theorem 7.5.2. Let F be a field, and let A, B ∈ Fn×n . Then

det(AB) = det(A)det(B).

Proof. Suppose first that at least one of A, B is non-invertible. Then by Corol-

lary 3.3.18, AB is also non-invertible. But by Theorem 7.4.1, non-invertible matrices
have determinant zero, and so det(AB) = 0 = det(A)det(B).15
From now on, we assume that A and B are both invertible. Therefore, by
the Invertible Matrix Theorem (see subsection 1.11.7, 3.3.7, or 7.4.1), each of
them can be written as a product of elementary matrices, say A = E1A . . . EpA
and B = E1B . . . EqB , where E1A , . . . , EpA , E1B , . . . , EqB are elementary matrices. So,
AB = E1A . . . EpA E1B . . . EqB . By repeatedly applying Proposition 7.5.1, we get that

ˆ det(A) = det(E1A ) . . . det(EpA );

ˆ det(B) = det(E1B ) . . . det(EqB );

ˆ det(AB) = det(E1A ) . . . det(EpA )det(E1B ) . . . det(EqB ).

But now

det(AB) = det(E1A ) . . . det(EpA ) det(E1B ) . . . det(EqB ) = det(A)det(B),

| {z }| {z }
=det(A) =det(B)

which is what we needed to show.

15
If A is non-invertible, then det(A) = 0, and if B is non-invertible, then det(B) = 0. In either
case, det(A)det(B) = 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 472

Corollary 7.5.3. Let F be a field, and let A ∈ Fn×n be an invertible matrix. Then

det(A−1 ) = 1
det(A) .

Proof. Since AA−1 = In , we see that

(∗)
det(A)det(A−1 ) = det(AA−1 ) = det(In ) = 1,

where (*) follows from Theorem 7.5.2. We now see that det(A−1 ) = 1
det(A) , which is
what we needed to show.

Similar matrices and determinants. Recall from subsection 4.5.2 that matrices
A, B ∈ Fn×n (where F is a field) are said to be similar if there exists an invertible
matrix P ∈ Fn×n such that B = P −1 AP . Using Theorem 7.5.2 and Corollary 7.5.3,
we can easily show that similar matrices have the same determinant.

Corollary 7.5.4. Let F be a field, and let A and B be similar matrices in Fn×n .
Then det(A) = det(B).

Proof. Since A and B are similar, there exists an invertible matrix P ∈ Fn×n such
that B = P −1 AP . We then have that
det(B) = det(P −1 AP )

= det(P −1 )det(A)det(P ) by Theorem 7.5.2

1
= det(P ) det(A)det(P ) by Corollary 7.5.3

= det(A),

which is what we needed to show.

Determinants of (some) linear functions. Suppose that V is a non-trivial,

finite-dimensional vector space over a field F, and that f : V → V is a linear function.
Then we define the determinant of f to be
   
det(f ) := det B f B ,

where B is any basis of V . Let us explain why this is well defined, that is, why the
value of det(f ) that we get depends only on f , and not on the particular choice of
the  B. Suppose
 basis   that C is any basis of V . Then by Theorem 4.5.19, matrices
B
f B
and C
f C
are similar, and consequently (by Corollary 7.5.4), they have
the same determinant. So, det(f ) is well defined.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 473

Remark: Note that we defined determinants only for linear functions whose domain
and codomain are one and the same, and moreover, are finite-dimensional and
non-null.

Determinants of orthogonal matrices. As another corollary of Theorem 7.5.2,

we obtain the following.

Corollary 7.5.5. Let A be an orthogonal matrix in Rn×n . Then det(A) = ±1 (i.e.

det(A) is either +1 or −1).

Proof. Since A is orthogonal, it satisfies AT A = In (by definition). Therefore,

(∗) (∗∗)
1 = det(In ) = det(AT A) = det(AT )det(A) = det(A)2 ,

where (*) follows from Theorem 7.5.2, and (**) follows from Theorem 7.1.3. But
now we see that det(A) = ±1, which is what we needed to show.

Warning: The converse of Corollary 7.5.5 is false, i.e. matrices whose

 determinant

1 1
is ±1 need not be orthogonal. For example, the matrix A = satisfies
2 3
det(A) = 1, but A is not orthogonal. More generally, suppose that A is any invertible
matrix in Rn×n . Then by Theorem 7.4.1, we have that det(A) ̸= 0. We now form
1
the matrix B by multiplying one row or one column of A by det(A) , and we see that
det(B) = 1. However, B need not be orthogonal.

7.6 Laplace expansion

Our goal in this section is to prove a formula (“Laplace explansion”) for the determi-
nant of a square matrix in terms of determinants of smaller matrices. We begin with
a technical proposition.

Proposition 7.6.1. Let F be a field, and let A ∈ F(n−1)×(n−1) (where n ≥ 2) and

a ∈ Fn−1 . Then
 A 0  
det = det(A).
aT 1 n×n
 
A 0  
Proof. First, set = ai,j , so that all the following hold:
aT 1 n×n
n×n

ˆ A=
 
ai,j (n−1)×(n−1)
;

ˆ an,n = 1;

ˆ for all i ∈ {1, . . . , n − 1}, ai,n = 0 ;

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 474

ˆ for all j ∈ {1, . . . , n − 1}, an,j is the j-th entry of the vector a.

Next, for all σ ∈ Sn−1 , let σ ∗ ∈ Sn be given by σ ∗ (i) = σ(i) for all i ∈ {1, . . . , n − 1}
and σ ∗ (n) = n. So, for any σ ∈ Sn−1 , the disjoint cycle decomposition of σ ∗ is
obtained by adding the one-element cycle (n) to the disjoint cycle decomposition
of σ, and consequently, sgn(σ) = sgn(σ ∗ ).16 Set Sn∗ := {σ ∗ | σ ∈ Sn−1 } = {π ∈ Sn |
π(n) = n}. We then have the following:
P
det(A) = sgn(σ)a1,σ(1) . . . an−1,σ(n−1)
σ∈Sn−1

sgn(σ ∗ )a1,σ∗ (1) . . . an−1,σ∗ (n−1) an,σ∗ (n)

P
=
σ∈Sn−1 | {z }
=1

P
= sgn(π)a1,π(1) . . . an−1,π(n−1) an,π(n)
∗
π∈Sn

(∗) P
= sgn(π)a1,π(1) . . . an−1,π(n−1) an,π(n)
π∈Sn

 A 0
 
= det ,
aT 1 n×n

where (*) follows from the fact that for all π ∈ Sn \ Sn∗ , we have that i := π −1 (n) ̸= n
(because π(n) ̸= n), and so ai,π(i) = ai,n = 0.
 
We now introduce some terminology and notation. For a matrix A = ai,j n×n
(where n ≥ 2) with entries in some field F, and for indices p, q ∈ {1, . . . , n}, Ap,q
is the (n − 1) × (n − 1) matrix obtained from A by deleting the p-th row and q-th
column (see Example 7.6.2 below). The determinants

det(Ai,j ), with i, j ∈ {1, . . . , n}

are referred to as the first minors of A, whereas numbers

Ci,j := (−1)i+j det(Ai,j ) with i, j ∈ {1, . . . , n}

are referred to as the cofactors of A.

16
Indeed, fix any σ ∈ Sn−1 , and suppose that the disjoint cycle decomposition of σ contains
exactly k cycles (when cycles of length one are included). Then the disjoint cycle decomposition
of σ ∗ contains exactly k + 1 cycles (when cycles of length one are included). But now sgn(σ) =
(−1)(n−1)−k = (−1)n−k−1 = (−1)n−(k+1) = sgn(σ ∗ ).

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 475

Example 7.6.2. For the matrix



1 2 3
A =  4 5 6 ,
7 8 9

we have the following:

     
5 6 4 6 4 5
ˆ A1,1 = ; ˆ A1,2 = ; ˆ A1,3 = ;
8 9 7 9 7 8
     
2 3 1 3 1 2
ˆ A2,1 = ; ˆ A2,2 = ; ˆ A2,3 = ;
8 9 7 9 7 8
     
2 3 1 3 1 2
ˆ A3,1 = ; ˆ A3,2 = ; ˆ A3,3 = .
5 6 4 6 4 5

We now prove a recursive formula for computing determinants in terms of minors or

cofactors. It allows us to compute the determinant of a square matrix in terms of
determinants of smaller square matrices. This formula is called “Laplace expansion”
or “cofactor expansion.”
 
Laplace expansion. Let F be a field, and let A = ai,j n×n (where n ≥ 2) be a
matrix in Fn×n . Then both the following hold:
(a) [expansion along the i-th row] for all i ∈ {1, . . . , n}, we have that
n
(−1)i+j ai,j det(Ai,j );
P
det(A) =
j=1

(b) [expansion along the j-th column] for all j ∈ {1, . . . , n}, we have that
n
(−1)i+j ai,j det(Ai,j ).
P
det(A) =
i=1

Remark: If we write Ci,j := (−1)i+j det(Ai,j ) for all i, j ∈ {1, . . . , n} (so, the Ci,j ’s
Pn
are the cofactors of A), then the formula from (a) becomes det(A) = ai,j Ci,j , and
j=1
n
P
the formula from (b) becomes det(A) = ai,j Ci,j . This is why Laplace expansion
i=1
is also referred to as “cofactor expansion.”

Proof. In view of Theorem 7.1.3, it is enough to prove (b). Fix j ∈ {1, . . . , n}. We
must show that
n
(−1)i+j ai,j det(Ai,j ).
P
det(A) =
i=1

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 476

  n
P
First, set A = a1 . . . an . Then aj = ai,j ei , and so
i=1
 
det(A) = det a1 . . . aj−1 aj aj+1 . . . an

 n

P
= det a1 . . . aj−1 ai,j ei aj+1 . . . an
i=1

(∗) n
P  
= ai,j det a1 . . . aj−1 ei aj+1 . . . an ,
i=1

where (*) follows from Proposition 7.2.1(a). Fix an arbitrary index i ∈ {1, . . . , n}.
To complete the proof, it now suffices to show that
 
det a1 . . . aj−1 ei aj+1 . . . an = (−1)i+j det(Ai,j ).

By iteratively performing n − j column swaps on the matrix

 
Bi := a1 . . . aj−1 ei aj+1 . . . an ,

we can obtain the matrix

 
Ci := a1 . . . aj−1 aj+1 . . . an ei .

By iteratively performing n − i row swaps on the matrix Ci , we can obtain the matrix
 
Ai,j 0
,
aT 1

where aT is the row vector of length n − 1 obtained from the i-th row of A by deleting
its j-th entry.17 Since swapping two rows or two columns has the effect of changing
the sign of the determinant, we see that

det(Bi ) = (−1)n−j det(Ci )

 A 0

i,j
= (−1)n−j (−1)n−i det
aT 1

(∗)
= (−1)2n−i−j det(Ai,j )

= (−1)i+j det(Ai,j ),

17
T
So, aT = , and in particular aT ∈ Fn−1 .

ai,1 ... ai,j−1 ai,j+1 ... ai,n

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 477

where (*) follows from Proposition 7.6.1. This completes the argument.

Example 7.6.3. Consider the matrix

 
2 0 1
A =  3 4 5 ,
7 0 8

with entries understood to be in R. Compute det(A) in two ways:

(a) via Laplace expansion along the third row;

(b) via Laplace expansion along the second column.

Solution. (a) We compute:

2 0 1
det(A) = 3 4 5
7 0 8

0 1 2 1 2 0
= (−1)3+1 7 + (−1)3+2 0 + (−1)3+3 8
4 5 3 5 3 4

0 1 2 0
= 7 +8 = 36.
4 5 3 4
| {z } | {z }
=−4 =8

(b) We compute:

2 0 1
det(A) = 3 4 5
7 0 8

3 5 2 1 2 1
= (−1)1+2 0 + (−1)2+2 4 + (−1)3+2 0
7 8 7 8 3 5

2 1
= 4 = 36.
7 8
| {z }
=9

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 478

Example 7.6.4. Compute the determinant of the matrix

 
1 2 0 −1 −2
 3 −4 0 −2 −1 
 
A =   1 2 2 0 1 
,
 1 0 0 0 2 
2 −1 0 1 3

with entries understood to be in R.

Solution. As a general rule, it is best to expand along a row or column that has a lot
of zeros (if such a row or column exists), since that reduces the amount of calculation
that we need to perform. In the calculation below, the row or column along which
we are about to expand is in red (to facilitate reading).

1 2 0 −1 −2
1 2 −1 −2
3 −4 0 −2 −1
3 −4 −2 −1
det(A) = 1 2 2 0 1 = (−1)3+3 2
1 0 0 2
1 0 0 0 2
2 −1 1 3
2 −1 0 1 3

1 2 −1 −2
3 −4 −2 −1
= 2
1 0 0 2
2 −1 1 3

2 −1 −2 1 2 −1 !
= 2 (−1)3+1 1 −4 −2 −1 + (−1)3+4 2 3 −4 −2
−1 1 3 2 −1 1

 2 −1 −2 1 2 −1 
= 2 −4 −2 −1 −2 3 −4 −2 = 78,
−1 1 3 2 −1 1
| {z } | {z }
=−11 −25

where the determinants of the two 3 × 3 matrices from the last line can be obtained in
various ways: Laplace expansion, elementary row/column operations, or our diagram
for computing determinants of 3 × 3 matrices (described section 7.1).

Example 7.6.5. Compute the determinant of the matrix

 
1 −1 2
A =  −2 4 1 ,
3 −3 5

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 479

with entries understood to be in R.

Solution. We combine various methods for computing determinants, as follows:

1 −1 2
det(A) = −2 4 1
3 −3 5

1 0 2
C2 →C2 +C1
= −2 2 1
3 0 5

1 2 Laplace expansion
= (−1)2+2 2
3 5 along 2nd column
| {z }
=−1

= −2.

Using Laplace expansion, one can easily show (see Theorem 7.6.6 and Corol-
lary 7.6.7 below) that the determinant of a matrix obtained by arranging several
square matrices along the main diagonal and placing zeros everywhere else is equal
to the product of the determinants of those square matrices along the main diagonal.

Theorem 7.6.6. Let F be a field, and let A ∈ Fn×n and B ∈ Fm×m be square
matrices. Then
 A On×m 

det = det(A) det(B).
Om×n B

Proof (outilne). This can be proven (for example) by induction on n, via Laplace
expansion along the leftmost column. The details are left as an exercise.

Corollary 7.6.7. Let F be a field, and let A1 ∈ Fn1 ×n1 , A2 ∈ Fn2 ×n2 , . . . , Ak ∈
Fnk ×nk be square matrices. Then
 
A1 On1 ×n2 . . . On1 ×nk
  On2 ×n1 A2 . . . On2 ×nk  k
Q
det  = det(Ai ).
 
.. .. .. .. 
 . . . .  i=1
Onk ×n1 Onk ×n2 ... Ak

Proof. This follows from Theorem 7.6.6 via an easy induction on k.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 480

7.7 Cramer’s rule

Before stating Cramer’s rule, we set up some notation. For a matrix A ∈ Fn×n , a
vector b ∈ Fn , and an index j ∈ {1, . . . , n}, we denote by Aj (b) the matrix obtained
from A by replacing the j-th column of A with b. For example, for
   
1 1 1 4
A =  0 2 2  and b =  5 ,
0 0 3 6

we have that
     
4 1 1 1 4 1 1 1 4
A1 (b) =  5 2 2  , A2 (b) =  0 5 2  , A3 (b) =  0 2 5  .
6 0 3 0 6 3 0 0 6

Further, in the remainder of this section, it will be convenient to use the fraction
notation discussed in subsection 2.4.3.

Cramer’s rule. Let F be a field, and let A be an invertible matrix in Fn×n , and let
b ∈ Fn . Then the matrix-vector equation Ax = b has a unique solution, namely



T
det A1 (b) det A2 (b) det An (b)
x = ... .
det(A) det(A) det(A)

Proof. Since A is invertible, we know that the matrix-vector equation Ax = b

has a unique solution, namely, x = A−1 b. Now, for this solution x, we set x =
 T
x1 . . . x n . Our goal is to show that



T
det A1 (b) det A2 (b) det An (b)
x = ... .
det(A) det(A) det(A)

Fix an index j ∈ {1, . . . , n}. We must show that

det Aj (b)
xj = det(A) .
 
Pn A = a1 . . . an . Using the fact that Ax = b and the fact that Ax =
Set
i=1 xi ai (by the definition of matrix-vector multiplication), we compute:
 
det(Aj (b)) = det a1 . . . aj−1 b aj+1 . . . an
 
= det a1 . . . aj−1 Ax aj+1 . . . an

 n

P
= det a1 . . . aj−1 xi ai aj+1 . . . an
i=1
Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 481

(∗) n
P  
= xi det a1 . . . aj−1 ai aj+1 . . . an
i=1

(∗∗)
 
= xj det a1 . . . aj−1 aj aj+1 . . . an

= xj det(A),

where (*) follows from Proposition 7.2.1(a), and (**) follows from the fact that for
all i ∈ {1, . . . , n} \ {j}, the matrix
 
a1 . . . aj−1 ai aj+1 . . . an

has two identical columns and therefore (by Proposition 7.1.5) has determinant zero.
We have now shown that


det Aj (b) = xj det(A).

Since A is invertible, Theorem 7.4.1 guarantees that det(A) ̸= 0. So, we can divide
both sides of the equality above by det(A) to obtain


det Aj (b)
xj = det(A) .

This completes the argument.

Example 7.7.1. Let

  
2 1 0 1
A =  0 2 2  and b =  1 ,
1 1 1 0
with entries understood to be in Z3 . Solve the matrix-vector equation Ax = b.
Solution. Note that det(A) = 2, and in particular, A is invertible (by Theorem 7.4.1).
So, Cramer’s rule applies. We compute:
1 1 0
ˆ det A1 (b) =


1 2 2 = 2;
0 1 1

2 1 0
ˆ det A2 (b) =


0 1 2 = 1;
1 0 1

2 1 1
ˆ det A3 (b) =


0 2 1 = 0.
1 1 0

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 482

By Cramer’s rule, Ax = b has a unique solution, namely




T
det A1 (b) det A2 (b) det A3 (b)
x =
det(A) det(A) det(A)

 2 1 0
T
= 2 2 2

 T
= 1 2 0 .

7.8 The adjugate matrix

Given a field F and a matrix A ∈ Fn×n (n ≥ 2), with cofactors i+j
 Ci,j = (−1) det(Ai,j )
(for i, j ∈ {1, . . . , n}), the cofactor matrix of A is the matrix Ci,j n×n . The adjugate
matrix (also called the classical adjoint) of A, denoted by adj(A), is the transponse
of the cofactor matrix of A, i.e.
  T
adj(A) := Ci,j n×n
.

So, the i, j-th entry of adj(A) is the cofactor Cj,i (note the swapping of the indices).

Example 7.8.1. Consider the matrix

 
1 1 1
A =  0 2 2 ,
0 0 3

with entries understood to be in R. Compute the cofactor and adjugate matrices of

the matrix A.

Solution. For all i, j ∈ {1, 2, 3}, we let Ci,j = (−1)i+j det(Ai,j ). We compute:

2 2
ˆ C1,1 = (−1)1+1 = 6;
0 3

0 2
ˆ C1,2 = (−1)1+2 = 0;
0 3

0 2
ˆ C1,3 = (−1)1+3 = 0;
0 0

1 1
ˆ C2,1 = (−1)2+1 = −3;
0 3

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 483

1 1
ˆ C2,2 = (−1)2+2 = 3;
0 3

1 1
ˆ C2,3 = (−1)2+3 = 0;
0 0

1 1
ˆ C3,1 = (−1)3+1 = 0;
2 2

1 1
ˆ C3,2 = (−1)3+2 = −2;
0 2

1 1
ˆ C3,3 = (−1)3+3 = 2.
0 2
So, the cofactor matrix of A is
   
C1,1 C1,2 C1,3 6 0 0
 C2,1 C2,2 C2,3  =  −3 3 0 .
C3,1 C3,2 C3,3 0 −2 2

The adjugate matrix of A is the transpose of the cofactor matrix, i.e.

 
6 −3 0
adj(A) =  0 3 −2  .
0 0 2

Theorem 7.8.2. Let F be a field, and let A ∈ Fn×n (n ≥ 2). Then

adj(A) A = A adj(A) = det(A)In .

Consequently, if A is invertible, then A−1 = 1

det(A) adj(A).

Proof. Let us first show that the first statement implies the second. Indeed, if A is
invertible, then det(A) ̸= 0, and so if the first statement holds, then we get that
   
1 1
det(A) adj(A) A = A det(A) adj(A) = In ,

and consequently, A−1 = 1

det(A) adj(A).

It remains to prove the first statement, i.e. that adj(A) A = A adj(A) = det(A)In .
We will do this by proving that the matrices adj(A) A, A adj(A), and det(A)In have
the same corresponding entries. Fix indices i, j ∈ {1, . . . , n}. The i, j-th entry of the
matrix det(A)In is det(A) if i = j, and is zero if i ̸= j. We must show this holds for
the i, j-th entry of the matrices adj(A) A and A adj(A) as well.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 484

 
We first consider the matrix adj(A) A. The i-th row of adj(A) is C1,i . . . Cn,i ,
 T
Pn the j-th column of A is a1,j . . . an,j
and . So, the i, j-th entry of adj(A) A is
a C
k=1 k,j k,i . Now, let B 1 be the matrix obtained
Pn by replacing the i-th column of
A by the j-th column of A. Then det(B1 ) = k=1 ak,j Ck,i (via Laplace expansion
along the i-th column of B1 ). But if i = j, then det(B1 ) = det(A) (because B1 = A),
and if i ≠ j, then det(B1 ) = 0 (because B1 has two identical columns, namely, the
i-th and j-th column).18
 
We now consider the matrix A adj(A). The i-th row of A is ai,1 . . . ai,n ,
 T
and the j-th Pncolumn of adj(A) is Cj,1 . . . Cj,n . So, the i, j-th entry of
A adj(A) is k=1 ai,k Cj,k . Now, let B2 be the matrix
Pn obtained by replacing the j-th
row of A by the i-th row of A. Then det(B2 ) = k=1 ai,k Cj,k (via Laplace expansion
along the j-th row of B2 ). But if i = j, then det(B2 ) = det(A) (because B2 = A),
and if i ̸= j, then det(B2 ) = 0 (because B2 has two identical rows, namely, the i-th
and j-th row).19

Example 7.8.3. Show that the matrix

 
1 1 1
A =  0 2 2 ,
0 0 3

(with entries understood to be in R) is invertible, and using Theorem 7.8.5, find its
inverse A−1 .

Solution. The matrix A is upper triangular, and so its determinant can be computed
by multiplying the entries along the main diagonal. So, det(A) = 1 · 2 · 3 = 6. Since
det(A) ̸= 0, Theorem 7.4.1 guarantees that A is invertible. In Example 7.8.1, we
compute the adjugate matrix of A:
 
6 −3 0
adj(A) =  0 3 −2  .
0 0 2

So, by Theorem 7.8.5, we have that

   
6 −3 0 1 −1/2 0
A−1 = 1
det(A) adj(A) = 1
6 0 3 −2  =  0 1/2 −1/3  .
0 0 2 0 0 1/3

18
By Proposition 7.1.5, the determinant of a square matrix with two identical columns is zero.
19
By Proposition 7.1.5, the determinant of a square matrix with two identical rows is zero.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 485

Corollary 7.8.4. Let F be a field, and let a, b, c, d ∈ F. Then the matrix

 
a b
A =
c d

is invertible if and only if ad ̸= bc, and in this case, the inverse of A is given by the
formula  
−1 1 d −b
A = ad−bc .
−c a

Proof. By Theorem 7.4.1, we know that A is invertible if and only if det(A) ̸= 0.

Clearly, det(A) = ad − bc, and it follows that A is invertible if and only if ad − bc ̸= 0,
i.e. if and only if ad ̸= bc.
Now, assume that A is invertible, so that ad ̸= bc. We first compute the cofactors
Ci,j of A:

ˆ C1,1 = (−1)1+1 det(A1,1 ) = d; ˆ C1,2 = (−1)1+2 det(A1,2 ) = −c;

ˆ C2,1 = (−1)2+1 det(A2,1 ) = −b; ˆ C2,2 = (−1)2+2 det(A2,2 ) = a.

The cofactor matrix of A is

   
C1,1 C1,2 d −c
= .
C2,1 C2,2 −b a

The adjugate matrix of A is the transpose of the cofactor matrix, i.e.

 
d −b
adj(A) = .
−c a

By Theorem 7.8.5, we now have that

 
d −b
A−1 = 1
det(A) adj(A) = 1
ad−bc ,
−c a

which is what we needed to show.

We complete this section with another proof of a slightly weaker version of Theo-
rem 7.8.2, one that only applies to invertible matrices A. We give this proof in order
to illustrate a nice application of Cramer’s rule.

Theorem 7.8.5. Let F be a field, and let A be an invertible matrix in Fn×n . Then

A−1 = 1
det(A) adj(A).

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 486

Proof. Since A is invertible, Theorem 7.4.1 guarantees that det(A) ̸= 0, and in

1
particular, the expression det(A) adj(A) is defined. We will prove the theorem by
−1
showing that matrices A and det(A) 1
adj(A) have the same corresponding entries.
Fix indices i, j ∈ {1, . . . , n}. By the definition of adj(A), we see that the i, j-th entry
1
of the matrix det(A) adj(A) is
(−1)j+i det(Aj,i )
det(A) .
We will use Cramer’s rule to show that this is also the i, j-th entry of the matrix
A−1 .
Set A−1 = a∗1 . . . a∗n . Since AA−1 = In , we have that
 

A a∗1 . . . a∗n
   
= e1 . . . en ,
and consequently (by the definition of matrix-vector multiplication), that
Aa∗1 . . . Aa∗n
   
= e1 . . . en .
In particular, the two matrices above have the same j-th column, and so Aa∗j = ej ,
i.e. a∗j is the solution of the equation Ax = ej (this solution is unique because A is
invertible). So, by Cramer’s rule, we have that


T
∗ det A1 (ej ) det An (ej )
aj = ... .
det(A) det(A)

The i-th entry of a∗j is

det Ai (ej )
det(A) .
By Laplace explansion along the i-th column, we get that
det Ai (ej ) = (−1)j+i det(Aj,i ).

So, the i-th entry of a∗j (which is precisely the i, j-th entry of A−1 ) is
(−1)j+i det(Aj,i )
det(A) ,
which is what we needed to show.

7.9 The Vandermonde matrix

For a positive integer n and real numbers a0 , a1 , . . . , an , we define the matrix
 
1 1 ... 1
 a0 a1 . . . an 
 
 2 2 2 
V (a0 , a1 , . . . , an ) :=  a0 a1 . . . an  ,
 .. .. . . .. 
 . . . . 
a0 a1 . . . ann (n+1)×(n+1)
n n

called the Vandermonde matrix.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 487

Proposition 7.9.1. For all positive integers n and real numbers a0 , a1 , . . . , an , we

have that
  Q n i−1
Q Q
det V (a0 , a1 , . . . , an ) = (ai − aj ) = (ai − aj ).
i>j i=1 j=0

and consequently, the matrix V (a0 , a1 , . . . , an ) is invertible if and only if a0 , a1 , . . . , an

are pairwise distinct.

Proof. By Theorem 7.4.1, we know that a square matrix is invertible if and only if its
determinant is non-zero. So, the second statement of the proposition follows immedi-
ately from the first, i.e. from the formula for the determinant of the Vandermonde
matrix.
We prove the formula for the determinant of the Vandermonde by induction on
n. For n = 1, we note that for any a0 , a1 ∈ R, we have that
  1 1
det V (a0 , a1 ) = = (a1 − a0 ).
a0 a1

Now, fix a positive integer n, and assume inductively that our formula is correctfor
n, i.e. that for all real numbers a0 , a1 , . . . , an , we have that det V (a0 , a1 , . . . , an ) =
Qn Qi−1
i=1 j=0 (ai − aj ). We must show that the formula is correct for n + 1. Fix
a0 , a1 , . . . , an , an+1 ∈ R; we will show that
  n+1
Q i−1
Q
det V (a0 , a1 , . . . , an , an+1 ) = (ai − aj ).
i=1 j=0

If some two of the numbers a0 , a1 , . . . , an , an+1 are the same, then this is obvious.
Indeed, in this case, the matrix V (a0 , a1 , . . . , an , an+1 ) has two identical columns
and
Qn+1therefore
Qi (by Proposition 7.1.5) has determinant zero, and on the other hand,
i=1 j=0 (a i − aj ) = 0 (because one of the factors is zero). So, from now on, we
may assume that a0 , a1 , . . . , an , an+1 are pairwise distinct.
Set
1 1 ... 1 1
a0 a1 . . . an t
a20 a21 . . . a2n t2
f (t) := .. .. .. .. .. ,
. . . . .
an0 an1 . . . ann tn
an+1
0 an+1
1 . . . an+1
n tn+1
so that f (an+1 ) = V (a0 , a1 , . . . , an , an+1 ). By performing Laplace expansion along
the rightmost column, we see that f (t) is a polynomial of degree n + 1, and that its

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 488

leading coefficient (i.e. coefficient in front of tn+1 ) is

1 1 ... 1
a0 a1 . . . an
(∗) n i−1
k := a20 a21 . . . a2n =
Q Q
(ai − aj ),
.. .. .. . i=1 j=0
. . . ..
an0 an1 . . . ann

where (*) follows from the induction hypothesis. Moreover, for each i ∈ {0, 1, . . . , n},
f (ai ) is the determinant of a square matrix that has two identical columns and is
therefore (by Proposition 7.1.5) equal to zero. Thus, a0 , a1 , . . . , an are all roots of
the (n + 1)-th degree polynomial f (t). So, f (t) can be factored as
n
Q n i−1
Q Q n
 Q 
f (t) = k (t − aj ) = (ai − aj ) (t − aj ) .
j=0 i=1 j=0 j=0

Consequently,
  n+1
Q i−1
Q
det V (a0 , a1 , . . . , an , an+1 ) = f (an+1 ) = (ai − aj ).
i=1 j=0

This completes the induction.

7.10 Determinants and volume

Throughout this section, we assume that Rn is equipped with the standard scalar
product · and the induced norm || · ||.

For a parallelogram, we have the familiar formula

 
area of
= (length of base) × (height).
parallelogram

height

base

We have a similar formula for the volume of a parallelepiped:

 
volume of
= (area of base) × (height).
parallelepiped

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 489

height

base

We would now like to generalize this to arbitrary dimensions. Given vectors

v1 , . . . , vm ∈ Rn , the m-parallelepiped determined by vectors v1 , . . . , vm is the set
n o
c1 v1 + · · · + cm vm | c1 , . . . , cm ∈ R, 0 ≤ c1 , . . . , cm ≤ 1 .

For instance, given two vectors v1 , v2 ∈ R2 , neither of which is a scalar multiple of

the other, the 2-parallelepiped determined by v1 , v2 is just the usual parallelogram
determined by these two vectors (see the picture below).

v2

v1

For vectors v1 , v2 ∈ Rn , neither of which is a scalar multiple of each other, the

2-parallelepiped determined by v1 , v2 is still a parallelogram, but this parallelogram
lies in the plane (2-dimensional subspace) Span(v1 , v2 ) of Rn . What happens if one
of v1 , v2 ∈ Rn is a scalar multiple of the other, say v2 = αv1 for some scalar α ∈ R?
Then the 2-parallelepiped determined by v1 and v2 is just set
n o
c1 v1 + c2 v2 | c1 , c2 ∈ R, 0 ≤ c1 , c2 ≤ 1
n o
= c1 v1 + c2 αv1 | c1 , c2 ∈ R, 0 ≤ c1 , c2 ≤ 1
n o
= (c1 + c2 α)v1 | c1 , c2 ∈ R, 0 ≤ c1 , c2 ≤ 1
n o
= c(1 + α)v1 | c ∈ R, 0 ≤ c ≤ 1 ,

which is 1-dimensional (a line segment) if v1 ̸= 0, and is 0-dimensional (containing

only the zero vector) if v1 = 0. We can think of these as “degenerate parallelograms.”

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 490

Similarly, for three linearly independent vectors v1 , v2 , v3 ∈ Rn , the 3-parallelepiped

defined by v1 , v2 , v3 is just the usual parallelepiped whose edges are determined by
these three vectors (see the picture below).

v3

v2
v1
0

If {v1 , v2 , v3 } is not linearly independent, then the 3-parallelepiped determined by

v1 , v2 , v3 is either a parallelogram, or a line segment, or {0}, depending on the
dimension of Span(v1 , v2 , v3 ). Once again, we can think of these as “degenerate
parallelepipeds.”
For more than three vectors, we get higher-dimensional generalizations.
We would now like to define the “volume” (more precisely, the “m-volume”) of
an m-parallelepiped in Rn . We do this recursively, as follows.

ˆ The 1-volume of the 1-parallelepiped determined by the vector v1 ∈ Rn is

defined to be
V1 (v1 ) := ||v1 ||.

ˆ For a positive integer m, the (m + 1)-volume of the (m + 1)-parallelepiped

determined by the vectors v1 , . . . , vm , vm+1 ∈ Rn is defined to be
⊥
Vm+1 (v1 , . . . , vm , vm+1 ) := Vm (v1 , . . . , vm ) ||vm+1 ||,
⊥
where vm+1 = projSpan(v1 ,...,vm )⊥ (vm+1 ).20

In this recursive formula, the m-parallelepiped determined by the vectors v1 , . . . , vm

⊥
is our “base” and ||vm+1 || is our “height.” So, we get the formula
 
(m + 1)-volume of
= (m-volume of base) × (height).
(m + 1)-parallelepiped

Note that 1-volume represents (1-dimensional) length, 2-volume represents (2-

dimensional) area, and 3-volume represents (3-dimensional) volume. For m ≥ 4,
m-volume is an m-dimensional generalization of these concepts.
Obviously, we would like volume to be non-negative and invariant under vector
reordering (i.e. the m-volume of an m-parallelepiped should not change if we merely
reorder the vectors determining this m-parallelepiped). The former readily follows
20 ⊥
Equivalently (by Corollary 6.5.3): vm+1 = vm+1 − projSpan(v1 ,...,vm ) (vm+1 ).

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 491

from the definition of volume (see Proposition 7.10.1 below). However, the latter
is not entirely obvious. We prove this in Corollary 7.10.4, which in fact follows
from the formula for volume as the determinant of a certain square matrix (see
Theorem 7.10.2).

Proposition 7.10.1. Let v1 , . . . , vm ∈ Rn . Then Vm (v1 , . . . , vm ) ≥ 0, and equality

holds if and only if {v1 , . . . , vm } is a linearly dependent set.
⊥
Proof. For each i ∈ {1, . . . , m − 1}, set vi+1 := projSpan(v1 ,...,vi )⊥ (vi+1 ). It then
follows from our recursive definition of V (v1 , . . . , vm ) that

Vm (v1 , . . . , vm ) = ||v1 || ||v2⊥ || . . . ||vm

⊥ ||.

Since the length of any vector is non-negative, we see that Vm (v1 , . . . , vm ) ≥ 0.

Moreover, equality holds if and only if at least one of the vectors v1 , v2⊥ , . . . , vm⊥ is 0.

We will show that the latter happens if and only if the set {v1 , . . . , vm } is linearly
dependent.
Suppose first that at least one of v1 , v2⊥ , . . . , vm⊥ is 0. If v = 0, then obvi-
1
ously, {v1 , . . . , vm } is linearly dependent. Suppose now that vi⊥ = 0 for some
index i ∈ {2, . . . , m}. Then projSpan(v1 ,...,vi−1 ) (vi ) = vi − vi⊥ = vi , and so vi ∈
Span(v1 , . . . , vi−1 ), i.e. vi is a linear combination of the vectors v1 , . . . , vi−1 . So,
{v1 , . . . , vm } is linearly dependent.
Suppose now that {v1 , . . . , vm } is linearly dependent. Then by Proposition 3.2.12,
there exists some i ∈ {1, . . . , m} such that vi is a linear combination of the vectors
v1 , . . . , vi−1 . If i = 1, then v1 = 0. On the other hand, if i ≥ 2, then vi ∈
Span(v1 , . . . , vi−1 ), and so vi⊥ = 0.21

Theorem 7.10.2. Let a1 , . . . , am ∈ Rn , and set A := a1 . . . am . Then

 

p
Vm (a1 , . . . , am ) = det(AT A).
 
Proof. For each i ∈ {1, . . . , m}, set Ai := a1 . . . ai . We will prove inductively
q
that for all i ∈ {1, . . . , m}, we have that Vi (a1 , . . . , ai ) = det(ATi Ai ). Obviously,
this is enough, since Am = A.
For i = 1, we observe that
T 
AT1 A1 =
   
a1 a1 = a1 · a1 ,

and consequently,
√
q
det(AT1 A1 ) = a1 · a1 = ||a1 || = V1 (a1 ).
21
Indeed, if vi ∈ Span(v1 , . . . , vi−1 ), then projSpan(v1 ,...,vi−1 ) (vi ) = vi , and consequently, vi⊥ =
vi − projSpan(v1 ,...,vi−1 ) (vi ) = 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 492

We may now assume that m ≥ 2, for otherwise we are done by what weqjust showed.
Fix i ∈ {1, . . . , m − 1}, and assume inductively that Vi (a1 , . . . , ai ) = det(ATi Ai ).
q
We must show that Vi+1 (a1 , . . . , ai , ai+1 ) = det(ATi+1 Ai+1 ). Set

||
ˆ ai+1 := projSpan(a1 ,...,ai ) (ai+1 );

ˆ a⊥
i+1 := projSpan(a1 ,...,ai )⊥ (ai+1 ).

|| ||
By Corollary 6.5.3, we have that ai+1 = ai+1 + a⊥
i+1 . Since ai+1 ∈ Span(a1 , . . . , ai ),
||
there exist scalars c1 , . . . , ci ∈ R such that ai+1 = c1 a1 + · · · + ci ai , and consequently,

||
a⊥
i+1 = ai+1 − ai = ai+1 − c1 a1 − · · · − ci ai .

Now, let Bi+1 be the matrix obtained from Ai+1 by replacing the rightmost column
of Ai+1 by a⊥
i+1 , i.e.

a1 . . . ai a⊥
 
Bi+1 := i+1 .

Then
aT1 aT1
   
 ..   .. 
T
Bi+1 = 
 .  = 
  . .

 aTi   aTi 
(a⊥
i+1 )
T aTi+1 − c1 aT1 − · · · − ci aTi
T
So, Bi+1 can be obtained from ATi+1 via the following sequence of i elementary row
operations:

ˆ Ri+1 → Ri+1 − c1 R1 ;
..
.

ˆ Ri+1 → Ri+1 − ci Ri .

Let E1 , . . . , Ei be the elementary matrices corresponding to these i elementary row

operations, so that Bi+1 T = E . . . E AT , and consequently, B T T
i 1 i+1 i+1 = Ai+1 E1 . . . Ei .
By Theorem 7.3.2(c), we see that det(E1 ) = · · · = det(Ei ) = 1.22 We now compute:
 
det(Bi+1 T B = det (Ei . . . E1 ATi+1 )(Ai+1 E1T . . . EiT )
i+1 )

(∗)
= det(Ei ) . . . det(E1 )det(ATi+1 Ai+1 )det(E1T ) . . . det(EiT )
22
Indeed, for each j ∈ {1, . . . , i}, the matrix Ej is obtained by performing the row operation
Ri+1 → Ri+1 − cj Rj on the identity matrix Ii+1 , and so by Theorem 7.3.2(c), we have that
det(Ej ) = det(Ii+1 ) = 1.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 493

(∗∗)
= det(Ei ) . . . det(E1 ) det(ATi+1 Ai+1 ) det(E1 ) . . . det(Ei )
| {z } | {z } | {z } | {z }
=1 =1 =1 =1

= det(ATi+1 Ai+1 ),

where (*) follows from Theorem

 7.5.2, and (**) follows from Theorem 7.1.3. But
⊥
note that Bi+1 = Ai ai+1 , and so

ATi
 
T B Ai a⊥
 
Bi+1 i+1 =
(a⊥
i+1 )
T i+1

ATi Ai ATi a⊥
 
= i+1
(a⊥ T ⊥ T ⊥
i+1 ) Ai (ai+1 ) ai+1

ATi Ai
 
(∗) 0
= ,
0T ||a⊥
i+1 ||
2

where in (*), we used the fact that a⊥

i+1 is orthogonal to the columns of A, and so
A ai+1 = 0, and we also used the fact that (a⊥
T ⊥ 23 T ⊥ ⊥ ⊥ ⊥ 2
i+1 ) ai+1 = ai+1 · ai+1 = ||ai+1 || .
We now compute:

det(ATi+1 Ai+1 ) = T B
det(Bi+1 i+1 )

ATi Ai 0
=
0T ||a⊥
i+1 ||
2

(∗)
= (−1)(i+1)+(i+1) ||a⊥ 2 T
i+1 || det(Ai Ai )

= det(ATi Ai ) ||a⊥
i+1 ||
2

(∗∗)
= Vi (a1 , . . . , ai )2 ||a⊥
i+1 ||
2

(∗∗∗)
= Vi+1 (a1 , . . . , ai , ai+1 )2 ,

where (*) follows by Laplace expansion along the rightmost column, (**) follows from
23
Indeed,
aT1 a1 · a⊥
   
i+1
 .  ⊥ ..
AT a⊥ =  ..  ai+1 = .
 
i+1  .
⊥
aTi a1 · ai+1
Since a⊥ ⊥ ⊥ ⊥ T ⊥
i+1 ∈ Span(a1 , . . . , ai ) , we see that a1 · ai+1 = · · · = ai · ai+1 = 0, and so A ai+1 = 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 494

the induction hypothesis, and (***) follows from the definition of Vi+1 (a1 , . . . , ai , ai+1 ).
Since Vi+1 (a1 , . . . , ai , ai+1 ) ≥ 0 (by Proposition 7.10.1), we may now take the square
root of both sides to obtain
q
Vi+1 (a1 , . . . , ai , ai+1 ) = det(ATi+1 Ai+1 ).

This completes the induction.

The following corollary of Theorem 7.10.2 gives a geometric interpretation of the

determinant.

Corollary 7.10.3. Let a1 , . . . , an ∈ Rn . Then

 
Vn (a1 , . . . , an ) = |det( a1 . . . an )|.
 
Proof. First of all, we note that A := a1 . . . an is an n×n matrix (with entries
in R), and so it has a determinant. We now compute:
p
Vn (a1 , . . . , an ) = det(AT A) by Theorem 7.10.2
p
= det(AT )det(A) by Theorem 7.5.2
p
= det(A)2 by Theorem 7.1.3

= |det(A)|.

This completes the argument.

Our next corollary states that the m-volume of an m-parallelepiped remains

unchanged if we merely change the order of the vectors that determine our m-
parallelepiped.

Corollary 7.10.4. Let a1 , . . . , am ∈ Rn and σ ∈ Sm . Then Vm (a1 , . . . , am ) =

Vm (aσ(1) , . . . , aσ(m) ).
   
Proof. Set A := a1 . . . am and Aσ := aσ(1) . . . aσ(m) , and consider Pσ ,
the matrix of the permutation σ.24 By Theorem 2.3.15(c), we have that Aσ = APσT ,
and by Proposition 7.1.1, we have that det(Pσ ) = sgn(σ). But now
(∗) p
Vm (aσ(1) , . . . , aσ(m) ) = det(ATσ Aσ )
q

= det (APσT )T (APσT )

24
We discussed permutation matrices in subsection 2.3.7.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 495

p
= det(Pσ AT APσT )

(∗∗) p
= det(Pσ )det(AT A)det(PσT )

(∗∗∗) p
= det(Pσ )det(AT A) det(Pσ )
p
= sgn(σ)2 det(AT A)
p
= det(AT A)

(∗)
= Vm (a1 , . . . , am ),

where both instances of (*) follow from Theorem 7.10.2, (**) follows from Theo-
rem 7.5.2, and (***) follows from Theorem 7.1.3.

Corollary 7.10.5. Let v1 , . . . , vn ∈ Rn , and let A ∈ Rn×n . Then

Vn (Av1 , . . . , Avn ) = |det(A)| Vn (v1 , . . . , vn ).

   
Proof. Set B := v1 . . . vn and C := Av1 . . . Avn = AB. Note that A,
B, and C = AB all belong to Rn×n , and so all three matrices have determinants.
We now compute:
(∗) p
Vn (Av1 , . . . , Avn ) = det(C T C)
r  
= det (AB)T (AB)

p
= det(B T AT AB)

(∗∗) p
= det(B T )det(AT )det(A)det(B)

(∗∗∗) p
= det(A)2 det(B T )det(B)

(∗∗) p
= det(A)2 det(B T B)
p
= |det(A)| det(B T B)

(∗)
= |det(A)| Vn (v1 , . . . , vn ),

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 496

where both instances of (*) follow from Theorem 7.10.2, both instances of (**) follow
from Theorem 7.5.2, and (***) follows from Theorem 7.1.3.
Remark: For a1 , . . . , am ∈ Rn (m ̸= n) and A ∈ Rn×n , the formula from Corol-
lary 7.10.5 fails, i.e.
= |det(A)| Vm (v1 , . . . , vm ).
Vm (Av1 , . . . , Avm ) Z

   
1 1 0
For instance, for m = 1 and n = 2, we can take v1 = and A = ,
0 0 0
so that Av1 = v1 . Then V1 (Av1 ) = V1 (v1 ) = ||v1 || = 1, but det(A) = 0, and so
V1 (Av1 ) ̸= |det(A)| V1 (v1 ).

Suppose that Ω is any object in Rn for which n-volume Vn (Ω) can be defined.
We will not go into the technical details of how this can be done, but the idea is
that we approximate Ω with ever smaller n-dimensional hypercubes; the sum of
n-volumes of those n-hypercubes (which are simply n-parallelepipeds, and so we
know how to compute their n-volume) will give us an ever better approximation of
the n-volume of Ω that we wish to define. To obtain the actual n-volume of Ω, we
take the limit of these ever-finer approximations. If the limit exists, then Ω will have
an n-volume (defined to be this limit). If the limit does not exist, then n-volume
is undefined for Ω. (It is actually pretty difficult to construct Ω for which volume
is undefined! Any reasonably pretty object Ω will have a volume, although that
volume may possibly be zero.) Now, suppose we are given a matrix A ∈ Rn×n . We
consider the linear function fA : Rn → Rn whose standard matrix is A (i.e. for all
x ∈ Rn , we have fA (x) = Ax). Then each of the small n-hypercubes gets mapped
onto a small n-parallelepiped; if the small n-hypercubes each had volume V , then by
Corollary 7.10.5, the small n-parallelepipeds that these n-hypercubes get mapped
onto via fA will have volume |det(A)| V . So, we get the following formula for the
n-volume of the image of Ω under fA :
Vn (fA [Ω]) = |det(A)| Vn (Ω).
For the case n = 2, see the picture below.

Ω fA[Ω]
fA(x) = Ax

Example 7.10.6. Let a and b be positive real numbers. Compute the area (i.e.
2-volume) of the region bounded by the ellipse whose equation is
x21 x22
a2
+ b2
= 1.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 497

Solution. We need compute the area of the region

n x 
x2 x22
o
1
E := | x1 , x2 ∈ R, a21 + b2
≤1 .
x2

Consider the unit disk

n x  o
1
D := | x1 , x2 ∈ R, x21 + x22 ≤ 1
x2

and the matrix  

a 0
A = .
0 b
Let fA : R2 → R2 be the linear function whose standard matrix is A, so that for all
T
∈ R2 , we have

x1 x2
 x  
a 0

x1
 
ax1

1
fA = = .
x2 0 b x2 bx2
x2 x2

1 D fA(x) = Ax E = fA[D]
b
−1 1 −a a
x1  
x1
a 0
A= 
  −b
−1 0 b

We now see that

n  x  o
1
fA [D] = fA | x1 , x2 ∈ R, x21 + x22 ≤ 1
x2
n  ax  o
1
= | x1 , x2 ∈ R, x21 + x22 ≤ 1
bx2
n y  o
1
= | y1 , y2 ∈ R, ( ya1 )2 + ( yb2 )2 ≤ 1
y2
n y 
y12 y22
o
1
= | y1 , y2 ∈ R, a2
+ b2
≤1
y2

= E.

Therefore, the area of E is

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 498

area(E) = |det(A)| area(D) = abπ.

| {z } | {z }
=ab =12 π

7.11 Common roots of polynomials via determinants

In this section, we show how determinants can help us determine whether two
polynomials have a common root, as long as the field that we are working over is
algebraically closed. Algebraically closed fields were discussed in subsection 2.4.5.
As we discussed in that subsection, of all the fields that we have seen in these lecture
notes, only the field C is algebraically closed (although other algebraically closed
fields do exist). The fact that C is algebraically closed follows immediately from the
Fundamental Theorem of Algebra (see subsection 0.3.2).
As we saw in subsection 2.4.5, any non-constant polynomial with coefficients in
an algebraically closed field has a root in that field.25 However, there is no general
formula for computing roots of such polynomials, even when the field that we are
working over is the familiar field C of complex number. So, it may be surprising
that, given arbitrary polynomials p(x) and q(x) with coefficients in an algebraically
closed field F, we can use determinants to determine whether p(x) and q(x) have a
common root, i.e. whether there exists a number x0 ∈ F for which we have p(x0 ) = 0
and q(x0 ) = 0 (see Theorem 7.11.1 below). However, the determinant in question
will only tell us whether such a common root exists; it provides no information on
how one might actually compute such a root. The theorem below is proven for
arbitrary algebraically closed fields (and it becomes false if the field in question is
not algebraically closed). However, if this level of abstraction bothers you, feel free
to assume that the field in question is C (which is the only algebraically closed field
that we will ever see in these lecture notes).

Theorem 7.11.1. Let F be Pan algebraically closed field. Let

Pnm andi n be positive
m i (a
integers, and let p(x) = i=0 ai x m ̸
= 0) and q(x) = i=0 bi x (bn ̸= 0) be
polynomials with coefficients in F. Let P be the n × (n + m) matrix whose j-th row
(for j ∈ {1, . . . , n}) is
h i
0 . . . 0 am am−1 . . . a0 0 . . . 0 ,
| {z } | {z }
j−1 n−j

and let Q be the m × (n + m) matrix whose j-th row (for j ∈ {1, . . . , m}) is
h i
0 . . . 0 bn bn−1 . . . b0 0 . . . 0 .
| {z } | {z }
j−1 m−j

25
This does not hold in general if the field in question is not algebraically closed!

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 499

Then p(x) and q(x) have a common root in F if and only if

 P 
det = 0.
Q

Remark: For example, if m = 3 and n = 5, so that

ˆ p(x) = a3 x3 + a2 x2 + a1 x + a0 ,

ˆ q(x) = b5 x5 + b4 x4 + b3 x3 + b2 x2 + b1 x + b0 ,

then we have
a3 a2 a1 a0 0 0 0 0
 

 0 a3 a2 a1 a0 0 0 0  
 

 0 0 a3 a2 a1 a0 0 0  
P  0 0 0 a3 a2 a1 a0 0 
=   .
Q 
 0 0 0 0 a3 a2 a1 a0 


 b5 b4 b3 b2 b1 b0 0 0  
 0 b5 b4 b3 b2 b1 b0 0 
0 0 b5 b4 b3 b2 b1 b0 8×8

Proof of Theorem 7.11.1. We begin by proving a technical claim.

Claim. Polynomials p(x) and q(x) have a common root in F if and only
if there exist non-zero polynomials r(x) and s(x) with coefficients in F
that satisfy the following:

ˆ deg r(x) ≤ n − 1;

ˆ deg s(x) ≤ m − 1;

ˆ r(x)p(x) + s(x)q(x) = 0.

Proof of the Claim. Suppose first that p(x) and q(x) have a common root in
q(x) p(x)
F, say α. Then we set r(x) := x−α and s(x) := − x−α , and we observe that





deg r(x) = deg q(x) − 1 = n − 1, deg s(x) = deg p(x) − 1 = m − 1, and
q(x)p(x) p(x)q(x)
r(x)p(x) + s(x)q(x) = x−α − x−α = 0.

Suppose conversely there
exist non-zero polynomials

r(x) and s(x) with coeffi-
cients in F such that deg r(x) ≤ n−1, deg s(x) ≤ m−1, and r(x)p(x)+s(x)q(x) =
0. Then r(x)p(x) and s(x)q(x) are non-constant polynomials with coefficients in F,
and they have
exactly the same roots with the same corresponding multiplicities.
Since deg p(x) = m, we know that p(x)
has exactly m roots in F (when multiplicities
are taken into account).26 But deg s(x) ≤ m − 1, and so at least one of the roots of
26
Here, we are using the fact that F is algebraically closed.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 500

p(x) either fails to be a root of s(x), or is a root of s(x) but has smaller multiplicity
in s(x) than in p(x). This root of p(x) must therefore be a root of q(x).27 ♦
In view of the P Claim,i it now suffices to determine if there exist non-zero poly-
nomials r(x) = n−1
Pm−1 i
i=0 ic x and s(x) = i=0 di x such that r(x)p(x) + s(x)q(x) = 0.
So, we need to determine if there exist c0 , . . . , cn−1 , d0 , . . . , dm−1 ∈ F such that at
least one of c0 , . . . , cn−1 is non-zero and at least one of d0 , . . . , dm−1 is non-zero, and
such that
 n−1
X m
 X   m−1
X n
 X 
ci xi ai xi + di xi bi xi = 0.
i=0 i=0 i=0 i=0
| {z } | {z } | {z } | {z }
=r(x) =p(x) =s(x) =q(x)

But obviously, if c0 , . . . , cn−1 are all zero, then d0 , . . . , dm−1 are all zero, and vice
versa. So, we in fact need to determine if the above equality holds for some numbers
c0 , . . . , cn−1 , d0 , . . . , dm−1 ∈ F, at least one of which is non-zero. We now write
the polynomial on the left-hand-side in the standard form, and we set all the
coefficients that we obtain equal to zero.28 This yields a system of n + m linear
equations in the variables cn−1 , . . . , c0 , dm−1 , . . . , d0 (we treat am , . . . , a0 , bn , . . . , b0
as constants). In each equation, we arrange the variables cn−1 , . . . , c0 , dm−1 , . . . , d0
in this order from left to right. We arrange the equations for the coefficients in front
of xn+m−1 , . . . , x1 , x0 from top to bottom. We then rewrite this linear system as a
matrix-vector equation
 T
A cn−1 . . . c0 dm−1 . . . d0 = 0,
 
P 29
and we observe that the coefficient matrix A satisfies AT = .
Q
27
We are using the fact that r(x)p(x) and s(x)q(x) have the same roots with the same corresponding
multiplicities.
28
We can do this since our polynomial is identically zero, i.e. it is zero as a polynomial. This
means precisely that all its coefficients are zero.
29
For example, if m = 3 and n = 5, so that
ˆ p(x) = a3 x3 + a2 x2 + a1 x + a0 ,
ˆ q(x) = b5 x5 + b4 x4 + b3 x3 + b2 x2 + b1 x + b0 ,
ˆ r(x) = c4 x4 + c3 x3 + c2 x2 + c1 x + c0 ,
ˆ s(t) = d2 x2 + d1 x + d0 ,
then our equation becomes
4
X 3
 X  2
X 5
 X 
ci xi ai xi + d i xi bi xi = 0,
i=0 i=0 i=0 i=0
| {z } | {z } | {z } | {z }
=r(x) =p(x) =s(x) =q(x)

which yields the system of linear equations below (we consider the coefficients in front of

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 501

We now have the following sequence of equivalent statements:

 T
p(x) and q(x) have A cn−1 . . . c0 dm−1 . . . d0 =0
⇐⇒
a common root in F has a non-zero solution

(∗)
⇐⇒ A is non-invertible
 
(∗) P
⇐⇒ AT = is non-invertible
Q

(∗)
 P 
⇐⇒ det = 0,
Q
where all three instances of (*) follow from the Invertible Matrix Theorem (version 3;
x7 , x6 , x5 , x4 , x3 , x2 , x1 , x0 from top to bottom, and we arrange the variables c4 , c3 , c2 , c1 , c0 , d2 , d1 , d0
from left to right).

c4 c3 c2 c1 c0 d2 d1 d0

x7 a3 c4 + b5 d 2 = 0
x6 a2 c4 + a3 c3 + b4 d 2 + b5 d 1 = 0
x5 a1 c4 + a2 c3 + a3 c2 + b3 d 2 + b4 d 1 + b5 d0 = 0
x4 a0 c4 + a1 c3 + a2 c2 + a3 c1 + b2 d 2 + b3 d 1 + b4 d0 = 0
x3 a0 c3 + a1 c2 + a2 c1 + a3 c0 + b1 d 2 + b2 d 1 + b3 d0 = 0
x2 a0 c2 + a1 c1 + a2 c0 + b0 d 2 + b1 d 1 + b2 d0 = 0
x1 a0 c1 + a1 c0 + b0 d 1 + b1 d0 = 0
x0 a0 c0 + b0 d0 = 0

This linear system, in turn, translates into the following matrix-vector equation:
  
a3 0 0 0 0 b5 0 0 c4
  c3 
 a2 a3 0 0 0 b4 b5 0   

 a1 a2 a3 0 0 b3 b4 b5   c2 
  
  c1  = 0.
 a0 a1 a2 a3 0 b2 b3 b4   

 0 a0 a1 a2 a3 b1 b2 b3   c0 
  
 0 0 a0 a1 a2 b0 b1 b2    d2 
 

 0 0 0 a0 a1 0 b0 b1   d1 
0 0 0 0 a0 0 0 b0 d0

Note that the transpose of the coefficient matrix that we obtained is precisely the matrix
 
a3 a2 a1 a0 0 0 0 0
 0 a3 a2 a1 a0 0 0 0 
 
 0 0 a3 a2 a1 a0 0 0 
   
P  0 0 0 a3 a2 a1 a0 0 
=   0

Q  0 0 0 a3 a2 a1 a0  
 b5 b4 b3 b2 b1 b0 0 0 
 
 0 b5 b4 b3 b2 b1 b0 0 
0 0 b5 b4 b3 b2 b1 b0 8×8

from the Remark following the statement of the theorem.

Irena Penev
Linear Algebra 1 & 2
Chapter 7. Determinants 502

see subsection 7.4.1). This completes the argument.

Example 7.11.2. Determine whether the polynomials p(x) = 5x3 − 2x2 + x − 4 and
q(x) = 7x2 − 6x − 1 have a common complex root.

Solution. In this case, it is easy to see that p(1) = 0 and q(1) = 0, and so 1 is a
common root of p(x) and q(x). However, let us use Theorem 7.11.1, in order to
illustrate how this theorem can be applied.
Using the notation of Theorem 7.11.1, we have that m = 3, n = 2, and the
matrices P and Q are given by
 
5 −2 1 −4 0
ˆ P = ;
0 5 −2 1 −4
 
7 −6 −1 0 0
ˆ Q= 0 7 −6 −1 0 .
0 0 7 −6 −1
We now have that
5 −2 1 −4 0
  0 5 −2 1 −4
P
det = 7 −6 −1 0 0 = 0.
Q
0 7 −6 −1 0
0 0 7 −6 −1

Theorem 7.11.1 now guarantees that p(x) and q(x) have a common complex root.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 503

Chapter 8

Eigenvalues and eigenvectors

Remark: In our study of eigenvalues and eigenvectors, we will often make reference to
“algebraically closed fields.” Algebraically closed fields were covered in subsection 2.4.5,
which the reader may wish to review before reading the present chapter. In a nutshell,
a field F is algebraically closed if every non-constant polynomial with coefficients in
F has a root in F. It can be shown that if F is an algebraically closed field, then any
non-constant polynomial with coefficients in F can be factored into linear terms. Of
all the fields that we have seen in these lecture notes, only C is algebraically closed.
(Other algebraically closed fields exist, but they are not discussed in these lecture
notes.) Fields Q, R, and Zp (where p is a prime number) are not algebraically closed.

8.1 Eigenvectors, eigenvalues, and eigenspaces

8.1.1 Eigenvectors and eigenvalues of linear functions
Suppose that V is a vector spaces over a field F, and that f : V → V is a linear
function. An eigenvector of f is a vector v ∈ V \ {0} for which there exists a scalar
λ ∈ F, called the eigenvalue of f associated with the eigenvector v, such that

f (v) = λv.

Under these circumstances, we also say that v is an eigenvector of f associated with

the eigenvalue λ. So, the eigenvectors of f are those non-zero vectors in V that
simply get scaled by f , and the eigenvalues are the scalars that the eigenvectors get
scaled by. By definition, an eigenvector cannot be 0, but an eigenvalue may possibly
be 0.
Remark: Note that eigenvectors and eigenvalues are only defined for those linear
functions whose domain is the same as the codomain.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 504

Example 8.1.1. Consider the linear function f : R2 → R2 given by

 x  
−1 0

x1
 
−x1

1
f = =
x2 0 1 x2 x2

for all x1 , x2 ∈ R. So, f isthe reflection

 about the x2 -axis (see the picture below),
−1 0
and its standard matrix is .
0 1
x2

f (v) v

x1

As usual, e1 and e2 are the standard basis vectors of R2 . Then

ˆ e1 is an eigenvector of f associated with the eigenvalue λ1 := −1, since

f (e1 ) = −e1 = λ1 e1 ;

ˆ e2 is an eigenvector of f associated with the eigenvalue λ2 := 1, since f (e2 ) =

e2 = λ2 e2 .
x2

f (e2) = e2

f (e1) = −e1 e1 x1

Example 8.1.2. Consider the linear function f : R2 → R2 given by

 x  
0 −1

x1
 
−x2

1
f = =
x2 1 0 x2 x1

for all x1 , x2 ∈ R. So, f is the counterclockwise rotation by ◦

 90 about the origin
0 −1
(see the picture below), and its standard matrix is . This function has no
1 0
eigenvectors (and consequently, it has no eigenvalues), since it does not simply scale
any non-zero vector in R2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 505

x2

f (v)
90◦ v

x1

Example 8.1.3. Consider the linear function f : C2 → C2 given by

 x  
0 −1

x1
 
−x2

1
f = =
x2 1 0 x2 x1

for all x1 , x2 ∈ C. (This is the same formula as the one from Example 8.1.2, except
that we are now working over C, rather than over R.) Then
 
i
ˆ v1 := is an eigenvector of f associated with the eigenvalue λ1 := i, since
1
   
−1 i
f (v1 ) = = i = λ1 v1 ;
i 1
 
−i
ˆ v2 := is an eigenvector of f associated with the eigenvalue λ2 := −i,
1
since    
−1 −i
f (v2 ) = = (−i) = λ2 v2 .
−i 1

Remark: It may be somewhat surprising that the linear function f from Exam-
ple 8.1.2 has no eigenvectors and no eigenvalues, whereas the one from Example 8.1.3
has them. As we shall see once we learn how to actually compute eigenvalues and
eigenvectors (this will involve finding roots of polynomials), the essential difference
is that C is an algebraically closed field, whereas R is not.

Eigenspaces. For a linear function f : V → V , where V is a vector space over a

field F, and for a scalar λ ∈ F, we define

Eλ (f ) := {v ∈ V | f (v) = λv}.

(∗)
Note that 0 ∈ Eλ (f ), since f (0) = 0 = λ0, where (*) follows from Proposition 4.1.6
(since f is linear). The set Eλ (f ) can be defined for any scalar λ, but it is only
interesting in the case when λ is an eigenvalue of V , in which case Eλ (f ) is called
the eigenspace of f associated with the eigenvalue λ. Note that, for an eigenvalue
λ of f , the elements of the eigenspace Eλ (f ) are precisely the zero vector and the

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 506

eigenvectors of f associated with λ.1 On the other hand, if λ is not an eigenvalue

of f , then we simply have that Eλ (f ) = {0}, and we do not refer to Eλ (f ) as an
eigenspace.

Proposition 8.1.4. Let V be a vector space over a field F, and let f : V → V be a

linear function. Then both the following hold:

(a) for all scalars λ ∈ F, Eλ (f ) is a subspace of V , and this subspace is non-trivial

(i.e. contains at least one non-zero vector) if and only if λ is an eigenvalue of f ;

(b) for all distinct scalars λ1 , λ2 ∈ F, we have that Eλ1 (f ) ∩ Eλ2 (f ) = {0}.

Proof. (a) Fix a scalar λ ∈ F. We first show that Eλ (f ) is a subsapce of V . In view

of Theorem 3.1.7, it suffices to show that Eλ (f ) contains 0 and is closed under vector
addition and scalar multiplication.
We already saw in the discussion above that 0 ∈ Eλ (f ).2 Next, for vectors
v1 , v2 ∈ Eλ (f ), we have that

f (v1 + v2 ) = f (v1 ) + f (v2 ) because f is linear

= λv1 + λv2 because v1 , v2 ∈ Eλ (f )

= λ(v1 + v2 ),

and consequently, v1 + v2 ∈ Eλ (f ). Finally, for a vector v ∈ Eλ (f ) and a scalar

α ∈ F, we have that

f (αv) = αf (v) because f is linear

= α(λv) because v ∈ Eλ (f )

= λ(αv),

and it follows that αv ∈ Eλ (f ). This proves that Eλ (f ) is indeed a subspace of V .

The fact that the subspace Eλ (f ) is non-trivial if and only if λ is an eigenvalue
of f follows immediately from the appropriate definitions.3
1
By definition, 0 cannot be an eigenvector.
2 (∗)
Here is the proof once again. We have that f (0) = 0 = λ0, where (*) follows from Proposi-
tion 4.1.6 (since f is linear). So, by the definition of Eλ (f ), we have that 0 ∈ Eλ (f ).
3
Here is a detailed proof. Suppose first that the subspace Eλ (f ) is non-trivial, and fix some
v ∈ Eλ (f ) \ {0}. But then v is a non-zero vector in V that satisfies f (v) = λv, which by definition
means that v is an eigenvector of V , and that λ is the associated eigenvalue. On the other hand, if λ
is an eigenvalue of V , then by definition, there exists a non-zero vector v ∈ V such that f (v) = λv,
which means that v ∈ Eλ (f ), and so the subspace Eλ (f ) contains a non-zero vector and is therefore
non-trivial.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 507

(b) Fix distinct scalars λ1 , λ2 ∈ F. By (a), Eλ1 (f ) and Eλ2 (f ) are both subspaces
of V , and consequently, 0 ∈ Eλ1 (f ) ∩ Eλ2 (f ). Now, fix any v ∈ Eλ1 (f ) ∩ Eλ2 (f ).
Since v ∈ Eλ1 (f ), we have that f (v) = λ1 v, and since v ∈ Eλ2 (f ), we have that
f (v) = λ2 v. So, λ1 v = λ2 v, and consequently, (λ1 − λ2 )v = 0. Since λ1 − λ2 ̸= 0
(because λ1 ̸= λ2 ), Proposition 3.1.3(c) guarantees that v = 0. This proves that
Eλ1 (f ) ∩ Eλ2 (f ) = {0}.

Terminology: Suppose that V is a vector space over a field F, and that λ is

an eigenvalue of a linear function f : V →
V . The geometric multiplicity of the
eigenvalue λ is defined to be dim Eλ (f ) . So, the geometric multiplicity of an
eigenvalue is the dimension of the associated eigenspace.
Remark: Suppose that V is a vector space over a field F, and that λ is an eigenvalue
of a linear function f : V → V . Then the elements of the eigenspace Eλ (f ) are
precisely the vector 0 and the eigenvectors of f associated with the eigenvalue λ.
Since no linearly independent set of vectors contains 0, it follows that all vectors in
any linearly independent set of vectors in Eλ (f ) are eigenvectors of f associated with
λ. Consequently, all vectors in any basis of Eλ (f ) are eigenvectors of f associated
with the eigenvalue λ.

8.1.2 Eigenvectors and eigenvalues of square matrices

Let F be a field, and let A ∈ Fn×n be a square matrix. An eigenvector of A is a
vector v ∈ Fn \ {0} for which there exists a scalar λ ∈ F, called the eigenvalue of A
associated with the eigenvector v, such that

Av = λv.

Under these circumstances, we also say that v is an eigenvector of A associated with

the eigenvalue λ.
Remark: Note that eigenvectors and eigenvalues are only defined for square
matrices. Eigenvectors are, by definition, non-zero, whereas eigenvalues may possibly
be zero.

Eigenspaces. For a square matrix A ∈ Fn×n (where F is some field), and for a
scalar λ ∈ F, we define

Eλ (A) := {v ∈ Fn | Av = λv}.

If λ is an eigenvalue of A, then Eλ (A) is called the eigenspace of A associated with

the eigenvalue λ. Note that, for an eigenvalue λ of A, the elements of the eigenspace
Eλ (A) are precisely the zero vector and the eigenvectors of A associated with λ. On
the other hand, if λ is not an eigenvalue of A, then we simply have that Eλ (A) = {0},
and we do not refer to Eλ (A) as an eigenspace.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 508

Proposition 8.1.5. Let F be a field, let f : Fn → Fn be a linear function, and let A

be the standard matrix of f . Then f and A have exactly the same eigenvalues and
the associated eigenectors. Moreover, for all eigenvalues λ of f and A, we have that
Eλ (f ) = Eλ (A).

Proof. This follows immediately from the appropriate definitions.

Propositions 8.1.4 and 8.1.5 immediately imply the following proposition.

Proposition 8.1.6. Let F be a field, and let A ∈ Fn×n be a square matrix. Then all
the following hold:

(a) for all scalars λ ∈ F, Eλ (A) is a subspace of Fn , and this subspace is non-trivial
(i.e. contains at least one non-zero vector) if and only if λ is an eigenvalue of A;

(b) for all distinct scalars λ1 , λ2 ∈ F, we have that Eλ1 (A) ∩ Eλ2 (A) = {0}.

Proof. Consider the function fA : Fn → Fn , given by fA (v) = Av for all vectors

v ∈ Fn . Then fA is linear (by Proposition 1.10.4), and moreover, A is the standard
matrix of fA . So, by Proposition 8.1.5, we have that for all λ ∈ F, Eλ (A) = Eλ (fA ).
The result now follows immediately from Proposition 8.1.4.

Terminology: Suppose that F is a field, and that λ is an eigenvalue of a square

matrix A ∈
Fn×n . The geometric multiplicity of the eigenvalue λ is defined to be
dim Eλ (A) . So, the geometric multiplicity of an eigenvalue is the dimension of the
associated eigenspace.

Proposition 8.1.7. Let V be a non-trivial, finite-dimensional vector space over a

field F, let B = {b1 , . . . , bn } be a basis of V , and let f : V → V be a linear function.
Then for all λ ∈ F, we have that
    n  o
Eλ B f B = v B | v ∈ Eλ (f ) .
 
Consequently, the linear function f and the matrix B f B have exactly the same
eigenvalues, with exactly the same corresponding geometric multiplicities.

Proof. Fix a scalar λ ∈ F. We first note that,

  by Proposition
 8.1.4, Eλ (f ) is a
subspace of V , and by Proposition 8.1.6, Eλ B f B is a subspace of Fn . Now,
we claim that     n  o
Eλ B f B = v B | v ∈ Eλ (f ) .

For this, we must prove the following two inclusions:

    n  o
(1) Eλ B f B ⊆ v B | v ∈ Eλ (f ) ;

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 509

n  o    
(2) v B
| v ∈ E λ (f ) ⊆ E λ B f B
.
 T    
We first prove (1). Fix any vecor x = x1 . . . xn in Eλ B f B . Set
 
v := x1 b1 + · · · + xn bn , so that v B = x. We must show that v ∈ Eλ (f ). We
compute:
      (∗)   (∗∗)  
f (v) B = B f B v B = λ v B = λv B ,
| {z } | {z }
=x =x
  
where (*) follows from the fact that x ∈ Eλ B f B , and (**) follows from the
   
linearity of · B . Since · B is an isomorphism (and in particular, one-to-one),
we see that f (v) = λv. By definition, this means that v ∈ Eλ (f ). This proves (1).
Let us now prove (2). Fix any v ∈ Eλ (f ). Then
     
B
f B v B = f (v) B
 
= λv B
because v ∈ Eλ (f )
   
= λ v B because · B is linear,
     
and it follows that v B ∈ Eλ B f B . This proves (2).
We have now proven both (1) and (2), and it follows that
    n  o
Eλ B f B = v B | v ∈ Eλ (f ) .
   
Thus, Eλ B f B is the image of Eλ (f ) under the isomorphism · B : V → Fn .
 

So, by Proposition 4.4.7, we have that

      
dim Eλ (f ) = dim Eλ B f B .
In particular, the subspace Eλ (f ) of V is non-trivial if and only if the subspace
Eλ B f B of Fn is non-trivial, and so by Propositions 8.1.4 and 8.1.6, λ is an
 
 
eigenvalue of f if and only if it is an eigenvalue of B f B . Moreover, by what we just
 
proved, if λ is an eigenvalue of f and B f B , then the corresponding eigenspaces
have the same dimension, that is, the eigenvalue λ has the same geometric multiplicity

with respect to the linear function f and with respect to the matrix B f B .

Remark: In view of Propositions 8.1.5 and 8.1.7, we see that the study of eigenvalues
and eigenvectors of linear functions from a non-trivial, finite-dimensional vector space
to itself is essentially equivalent to the study of eigenvalues and eigenvectors of square
matrices. The computational tools that we develop for finding eigenvectors and
eigenvalues will primarily be for square matrices. On the other hand, some of the
theoretical results that we prove will be for linear functions instead, and we will
obtain corresponding results for matrices as more or less immediate corollaries.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 510

8.2 The characteristic polynomial

In this section, we introduce the “characteristic polynomial” of a square matrix
(see subsection 8.2.1), and we show that its roots are precisely the eigenvalues of
the matrix in question. We define the “algebraic multiplicity” of an eigenvalue to
be its multiplicity as a root of the characteristic polynomial. As we shall see, the
geometric multiplicity of an eigenvalue is no greater than its algebraic multiplicity
(see Theorem 8.2.3). We also show that a square matrix is invertible if and only
if 0 is not its eigenvalue (see Proposition 8.2.11), and we add this to our previous
version of the Invertible Matrix Theorem (see subsection 7.4.1) to produce the fourth
and final version of the Invertible Matrix Theorem (see subsection 8.2.6). Finally,
in subsection 8.2.7, we introduce characteristic polynomial of a linear function
(having the same non-trivial, finite-dimensional vector space both for its domain and
codomain).

8.2.1 The characteristic polynomial of a square matrix

Given a field F and a matrix A ∈ Fn×n , the characteristic polynomial of A is defined
to be

pA (λ) := det λIn − A .
The characteristic equation of A is the equation


det λIn − A = 0.

So, the roots of the characteristic polynomial of A are precisely the solutions of the
characteristic equation of A.

Example 8.2.1. Compute the characteristic polynomial of the following matrix in

C3×3 :  
1 −2 3
A =  −1 0 2 .
2 −1 −3
Solution. The characteristic polynomial of A is:

λ − 1 2 −3
pA (λ) = det(λI3 − A) = 1 λ −2 = λ3 + 2λ2 − 9λ − 3.
−2 1 λ + 3

Remark: For a field F and a matrix A ∈ Fn×n , the characteristic polynomial

pA (λ) = det(λIn − A) is a polynomial of degree n, with leading coefficient 1, i.e. the
coefficient in front of λn in pA (λ) is 1. In some texts, the characteristic polynomial

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 511

is defined to be det(A − λIn ). By Proposition 7.2.3, we have that det(A − λIn ) =

(−1)n det(λIn − A), and so the polynomials det(λIn − A) and det(A − λIn ) have
exactly the same roots, with the same corresponding multiplicities, which is what
we will actually care about when it comes to the characteristic polynomial. The
main advantage of using det(λIn − A) rather than det(A − λIn ) is that the former
polynomial has leading coefficient 1, whereas the latter has leading coefficient (−1)n ,
which is −1 if n is odd.

Theorem 8.2.2. Let F be a field, let A ∈ Fn×n , and let λ0 ∈ F. Then

Eλ0 (A) = Nul λ0 In − A = Nul A − λ0 In .

Moreover, the following are equivalent:

(1) λ0 is an eigenvalue of A;

(2) λ0 is a root of the characteristic polynomial of A, i.e. pA (λ0 ) = 0;

(3) λ0 is a solution of the characteristic equation of A, i.e. det λ0 In − A = 0.
n , we have that λ I − A v = 0 if and only if


Proof. Obviously,

for all v ∈ F
0
n
A − λ0 In v = 0. So, Nul λ0 In − A = Nul A − λ0 In . Further, we compute:

v ∈ Fn | Av = λ0 v

Eλ0 (A) =

v ∈ Fn | Av = λ0 In v

=

v ∈ Fn | (λ0 In − A)v = 0

=


= Nul λ0 In − A .

It remains to show that (1), (2), and (3) are equivalent. The fact that (2) and (3)
are equivalent follows immediately from the appropriate definitions. It remains to
prove that (1) and (3) are equivalent. For this, we have the following sequence of
equivalent statements:

λ0 is an eigenvalue of A (∗)
| {z } ⇐⇒ Eλ0 (A) ̸= {0}
(1)

(∗∗)

⇐⇒ Nul λ0 In − A ̸= {0}

(∗∗∗) the matrix λ0 In − A

⇐⇒
is not invertible


(∗∗∗) det λ0 In − A = 0
⇐⇒ | {z }
Irena Penev
(3)
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 512

where (*) follows

from Proposition 8.1.6, (**) follows from the fact that Eλ0 (A) =
Nul λ0 In − A (which we proved above), and both instances of (***) follow from
the Invertible Matrix Theorem (version 3; see subsection 7.4.1). This completes the
argument.

Algebraic multiplicities of eigenvalues. By Theorem 8.2.2, the eigenvalues of a

square matrix are precisely the roots of its characteristic polynomial. With this in
mind, we define the “algebraic multiplicity” of an eigenvalue as follows. For a field
F, a matrix A ∈ Fn×n , and an eigenvalue λ0 of A, the algebraic multiplicity of the
eigenvalue λ0 is its multiplicity as a root of the characteristic polynomial of A, or
in other words, it is the largest integer k such that (λ − λ k
k 4

0 ) | pA (λ), i.e. such that
(λ − λ0 ) divides the polynomial pA (λ). Since deg pA (λ) = n, the sum of algebraic
multiplicities of the eigenvalues of the matrix A ∈ Fn×n is at most n; if the field F is
algebraically closed, then the sum of algebraic multiplicities of the eigenvalues of A is
exactly n.5 In view of Theorem 8.2.2, this implies that if the field F is algebraically
closed, then A has exactly n eigenvalues when algebraic multiplicities are taken into
account. However, this is not necessarily true if F is not algebraically closed (in that
case, it is even possible that A has no eigenvalues at all).

Theorem 8.2.3. Let F be a field, and let A ∈ Fn×n . Then the geometric multiplicity
of any eigenvalue of A is no greater than the algebraic multiplicity of that eigenvalue.

Remark: Schematically, Theorem 8.2.3 states that for an eigenvalue λ of a matrix

A ∈ Fn×n (where F is an arbitrary field), we have that:

geometric multiplicity of λ ≤ algebraic multiplicity of λ.

We postpone the proof of Theorem 8.2.3 to the very end of this section (see
subsection 8.2.8); no result of the present section relies on this theorem.

The spectrum of a square matrix. The spectrum of a square matrix A ∈ Fn×n is

the multiset of all eigenvalues of A, with algebraic multiplicities taken into account.6
For example, if a matrix A ∈ C5×5 has eigenvalues 1 (with algebraic multiplicity 1),
1 + i (with algebraic multiplicity 2), and 1 − i (with algebraic multiplicity 2), then
the spectrum of A is {1, 1 + i, 1 + i, 1 − i, 1 − i}. In general, the spectrum of a matrix
4
In other words, k is the largest integer such that there exists some polynomial q(λ) with
coefficients in F such that pA (λ) = (λ − λ0 )k q(λ).
5
Indeed, if F is algebraically closed, then the characteristic polynomial pA (λ) can be written as a
product of linear factors, and there are n of those factors. If F is not algebraically closed, we might
or might not be able to factor pA (λ) in this way, which is why the sum of algebraic multiplicities of
the eigenvalues of A is at most n (possibly strictly smaller than n).
6
This means that the number of times that an eigenvalue appears in the spectrum is equal to the
algebraic multiplicity of that eigenvalue. The order in which we list the eigenvalues in the spectrum
does not matter, but repetitions do matter.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 513

A ∈ Fn×n (where F is a field) has at most n elements; if the field F is algebraically

closed, then the spectrum of A has exactly n elements.

The spectral radius. For a matrix A ∈ Cn×n , the spectral radius of A, denoted
by ρ(A), is the maximum absolute value of any eigenvalue of A. For example, if
the spectrum of a matrix A ∈ C5×5 is {1, 1 + i, 1 + i, 1 − i, 1 −√ i}, then the spectral
radius of A is ρ(A) = max{|1|, |1 + i|, |1 + i|, |1 − i|, |1 − i|} = 2.7
In view of Theorems 0.3.6 and 8.2.2, we can visualize the complex eigenvalues
of an n × n matrix A with real entries (however, we consider A to be a matrix in
the vector space Cn×n , so that it can have complex eigenvalues). Its characteristic
polynomial pA (λ) is of degree n and has real coefficients. By Theorem 0.3.6, the roots
of this polynomial come in conjugate pairs,8 and moreover, by Theorem 8.2.2, those
roots are precisely the eigenvalues of A. The eigenvalues all lie in the complex plane,
in the disk centered at the origin and with radius ρ(A), and they are symmetric
about the real axis. Visually, the eigenvalues λ1 , λ2 , λ3 , λ4 , λ5 of a matrix A ∈ C5×5
with real entries might appear as in the picture below (the conjugate pairs are color
coded for emphasis).
Im

λ1
ρ
λ3

λ5 Re
λ4
λ2

8.2.2 Some numerical examples

Example 8.2.4. Consider the following matrix in C3×3 :
 
4 0 −2
A =  2 5 4 .
0 0 5

(a) Compute the characteristic polynomial pA (λ) of the matrix A.

(b) Compute all the eigenvalues of A and their algebraic multiplicities, and compute
the spectrum of A.
7
√ √ √
Indeed, |1| = 1, |1 + i| = 2, and |1 − i| = 2. So, ρ(A) = 2.
8
Each real root is its own conjugate pair.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 514

(c) For each eigenvalue λ of A, compute a basis of the eigenspace Eλ (A) and specify
the geometric multiplicity of the eigenvalue λ.

Solution. (a) The characteristic polynomial of A is:

pA (λ) = det(λI3 − A)

λ−4 0 2
= −2 λ − 5 −4
0 0 λ−5

(∗)
= (λ − 4)(λ − 5)2

= λ3 − 14λ2 + 65λ − 100,

where the easiest way to obtain (*) is via Laplace expansion along the second column.
Remark: We did not really need to expand in the last line. We only really care
about the roots of the characteristic polynomial, and it is more convenient to have a
form that is already factored. So, pA (λ) = (λ − 4)(λ − 5)2 is a “better” answer than
pA (λ) = λ3 − 14λ2 + 65λ − 100, although they are both correct.
(b) From part (a), we see that A has two eigenvalues, namely, the eigenvalue
λ1 = 4 (with algebraic multiplicity 1), and the eigenvalue λ2 = 5 (with algebraic
multiplicity 2). So, the spectrum of A is {4, 5, 5}.
(c) For each i ∈ {1, 2}, we have that


Eλi (A) = Nul λi I3 − A ,

which is precisely the set of all solutions of the characteristic equation

(λi I3 − A)x = 0.

Let us now compute a basis of each of the two eigenspaces.

For λ1 = 4, we have that
   
λ1 − 4 0 2 0 0 2
λ1 I3 − A =  −2 λ1 − 5 −4  =  −2 −1 −4  ,
0 0 λ1 − 5 0 0 −1

and that
1
 
1 2 0
RREF(λ1 I3 − A) =  0 0 1 .
0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 515

Consequently, the general solution of the equation (λ1 I3 − A)x = 0 is

     
−t/2 −1/2 −1
x =  t  = t  1  = 2t  2  , with t ∈ C.
0 0 0
 
n−1 o
So,  2  is a basis of the eigespace Eλ1 (A) = Nul A − λ1 In ,9 and we see

0
that the eigenvalue λ1 = 4 has geometric multiplicity 1.
For λ2 = 5, we have that
   
λ2 − 4 0 2 1 0 2
λ2 I3 − A =  −2 λ2 − 5 −4  =  −2 0 −4  ,
0 0 λ2 − 5 0 0 0

and that  
1 0 2
RREF(λ2 I3 − A) =  0 0 0  .
0 0 0
Consequently, the general solution of the equation (λ2 I3 − A)x = 0 is
     
−2t 0 −2
x =  s  = s 1  + t 0 , with s, t ∈ C.
t 0 1

  
n0 −2 o

So,  1  ,  0  is a basis of the eigenspace Eλ2 (A) = Nul A − λ2 In , and
0 1
we see that the eigenvalue λ2 = 5 has geometric multiplicity 2.

Example 8.2.5. Consider the following matrix in R2×2 :

" 1 #
√ − √1
B = 2 2 .
√1 √1
2 2

(Note that B is the standard matrix of counterclockwise rotation by 45◦ about the
origin in R2 .)

(a) Compute the characteristic polynomial pB (λ) of the matrix B.

 
n −1/2 o
9  1  is a basis of Eλ1 . However, it is nicer to get integers (when
It is also true that
0
possible).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 516

(b) Compute all the (real) eigenvalues of B and their algebraic multiplicities.

Remark: Since we consider B to be a matrix in R2×2 , we need to

look for real eigenvalues only.

(c) For each eigenvalue λ of B, compute a basis of the eigenspace Eλ (B) and specify
the geometric multiplicity of the eigenvalue λ.

Solution. (a) The characteristic polynomial of B is:

pB (λ) = det(λI2 − B)

λ − √12 √1
2
=
− √12 λ − √12

= (λ − √1 )(λ − √1 ) − √1 (− √1 )
2 2 2 2
√
= λ2 − 2λ + 1.

(b,c) We need to find any real roots that the polynomial pB (λ) may have, i.e.
any real solutions that the quadratic equation
√
λ2 − 2λ + 1 = 0
√
may have. The discriminant of this quadratic equation is (− 2)2 − 4 · 1 · 1 = −2 < 0,
and it follows that the equation has no real solutions. Therefore, B has no real
eigenvalues, and it follows that the spectrum of B is empty.

Remark: The fact that B has no eigenvectors (and consequently, no eigenvalues)

also follows from geometric considerations. Indeed, B is the standard matrix of
counterclockwise rotation by 45◦ about the origin in R2 . So, no non-zero vector in
R2 simply gets scaled when we multiply it on the left by B, which by definition
means that B has no eigenvectors.

Example 8.2.6. Consider the following matrix in C2×2 :

" 1 #
√ − √1
C = 2 2 .
√1 √1
2 2

Remark: This is the same as the matrix B from Example 8.2.5, but this
time, we consider the matrix to be in C2×2 .

(a) Compute the characteristic polynomial pC (λ) of the matrix C.

(b) Compute all the eigenvalues of C and their algebraic multiplicities.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 517

Remark: Since we consider C to be a matrix in C2×2 , we need to look

for complex eigenvalues. (Note that all real numbers are complex!
So, if our eigenvalues ended up being real, they would still count as
complex eigenvalues.)
(c) For each eigenvalue λ of C, compute a basis of the eigenspace Eλ (C) and specify
the geometric multiplicity of the eigenvalue λ.
Solution. (a) The characteristic polynomial of C is the same as the characteris-
tic polynomial of the matrix B from Example 8.2.5, since the two characteristic
polynomials are computed in exactly the same way. Indeed,
pC (λ) = det(λI2 − C)

λ − √12 √1
2
=
− √12 λ − √12

= (λ − √1 )(λ − √1 ) − √1 (− √1 )
2 2 2 2
√
= λ2 − 2λ + 1.
(b) We need to find the (complex) roots of the characteristic polynomial pC (λ),
together with their algebraic multiplicities. The quadratic equation
√
λ2 − 2λ + 1 = 0
| {z }
=pC (λ)

has solutions
√ √ √ √ √
−(− 2)± (− 2)2 −4·1·1 2± −2 1±i
λ1,2 = 2·1 = 2 = √ ,
2

that is,
1+i 1−i
λ1 = √
2
and λ2 = √ .
2
Complex numbers λ1 and λ2 are the eigenvalues of the matrix C, and they each
have algebraic
n multiplicity
o 1, since pC (λ) = (λ − λ1 )(λ − λ2 ). The spectrum of C is
1+i 1−i
{λ1 , λ2 } = √2 , √2 .
(c) For each i ∈ {1, 2}, the eigenspace Eλi (C) is precisely the set of solutions of
the characteristic equation
(λi I2 − C)x = 0.

1+i
For λ1 = √ ,
2
we have that
" # " #
λ1 − √12 √1
2
√i
2
√1
2
λ1 I2 − C = = ,
− √12 λ1 − √1
2
− √12 √i
2

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 518

and that  
1 −i
RREF(λ1 I2 − C) = .
0 0
Consequently, the general solution of the equation (λ1 I2 − C)x = 0 is
   
it i
x = = t , with t ∈ C.
t 1
n i o

So, is a basis of the eigenspace Eλ1 (C) = Nul λ1 I2 − C , and we see that
1
the eigenvalue λ1 = 1+i
√ has geometric multiplicity 1.
2
1−i
For λ2 = √ ,
2
we have that
" # " #
λ2 − √12 √1
2
− √i2 √1
2
λ 2 I2 − C = = ,
− √12 λ2 − √1
2
− √12 − √i2

and that  
1 i
RREF(λ2 I2 − C) = .
0 0
Consequently, the general solution of the equation (λ2 I2 − C)x = 0 is
   
−it −i
x = = t , with t ∈ C.
t 1
n  −i  o

So, is a basis of the eigenspace Eλ2 (C) = Nul λ2 I2 − C , and we see that
1
the eigenvalue λ2 = 1−i
√ has geometric multiplicity 1.
2

Eigenvectors and eigenvalues of triangular matrices. By Proposition 7.3.1,

the determinant of a triangular matrix is equal to the product of its entries on the
main diagonal. This immediately yields the following proposition.
 
Proposition 8.2.7. Let F be a field, and let A = ai,j n×n be a triangular matrix
in Fn×n . Then the characteristic polynomial of A is
n
Q
pA (λ) = (λ − ai,i ) = (λ − a1,1 )(λ − a2,2 ) . . . (λ − an,n ),
i=1

the eigenvalues of A are precisely the entries of A on its main diagonal, and more-
over, the algebraic multiplicity of each eigenvalue is precisely the number of times
that it appears on the main diagonal of A.10 Consequently, the spectrum of A is
10
However, the geometric multiplicity may possibly be smaller, as Example 8.2.8 shows. We note,
however, that the geometric multiplicity will never be larger than this, as per Theorem 8.2.3 (which
we have not proven yet).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 519

{a1,1 , a2,2 , . . . , an,n }, i.e. the multiset formed precisely by the main diagonal entries
of A, with each number appearing in the spectrum of A the same number of times as
on the main diagonal of A.

Proof. Since A is triangular, so is the matrix λIn − A; so, the determinant of λIn − A
can be computed simply by multiplying its entries on the main diagonal. It follows
that the characteristic polynomial of A is

pA (λ) = det(λIn − A) = (λ − a1,1 )(λ − a2,2 ) . . . (λ − an,n ),

and the result follows.

Example 8.2.8. Consider the following matrix in C5×5 :

 
1 2 0 0 0
 0 2 0 0 0 
 
A =   0 0 1 1 3 .

 0 0 0 3 3 
0 0 0 0 3

(a) Compute the characteristic polynomial pA (λ) of the matrix A.

(b) Compute all the eigenvalues of A and their algebraic multiplicities, and compute
the spectrum of A.

(c) For each eigenvalue λ of A, compute a basis of the eigenspace Eλ (A) and specify
the geometric multiplicity of the eigenvalue λ.

Solution. (a) The matrix A is upper triangular, and so its characteristic polynomial
is
λ − 1 −2 0 0 0

0 λ−2 0 0 0
pA (λ) = det λI5 − A = 0 0 λ − 1 −1 −3
0 0 0 λ − 3 −3
0 0 0 0 λ−3

= (λ − 1)2 (λ − 2)(λ − 3)2 .

(b) We see from part (a) that A has three eigenvalues, namely, λ1 = 1 (with
algebraic multiplicity 2), λ2 = 2 (with algebraic multiplicity 1), and λ = 3 (with
algebraic multiplicity 2).11 So, the spectrum of A is {1, 1, 2, 3, 3}.
11
We could also have obtained the same answer by noticing that A is triangular, and that 1
appears twice on the main diagonal of A, 2 appears once, and 3 appears twice.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 520

(c) For each i ∈ {1, 2, 3}, the eigenspace Eλi (A) is precisely the set of solutions
of the characteristic equation

(λi I5 − A)x = 0.

For λ1 = 1, we have that

 
0 −2 0 0 0

 0 −1 0 0 0 

λ 1 I5 − A = 
 0 0 0 −1 −3 ,

 0 0 0 −2 −3 
0 0 0 0 −2

and that  
0 1 0 0 0

 0 0 0 1 0 

RREF(λ1 I5 − A) = 
 0 0 0 0 1 .

 0 0 0 0 0 
0 0 0 0 0
Consequently, the general solution of the equation (λ1 I5 − A)x = 0 is
     
s 1 0
 0   0   0 
     
x =   t  = s 0  + t 1 ,
     with s, t ∈ C.
 0   0   0 
0 0 0

So,    
1 0
n
 0  
  0 o


 0 ,
  1 

 0   0 
0 0


is a basis of the eigenspace Eλ1 (A) = Nul λ1 I5 − A , and we see that the eigenvalue
λ1 = 1 has geometric multiplicity 2.
For λ2 = 2, we have that
 
1 −2 0 0 0

 0 0 0 0 0 

λ 2 I5 − A = 
 0 0 1 −1 −3 ,

 0 0 0 −1 −3 
0 0 0 0 −1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 521

and that  
1 −2 0 0 0

 0 0 1 0 0 

RREF(λ2 I5 − A) = 
 0 0 0 1 0 .

 0 0 0 0 1 
0 0 0 0 0
Consequently, the general solution of the equation (λ2 I5 − A)x = 0 is
   
2t 2
 t   1 
   
x =   0  = t 0 ,
   with t ∈ C.
 0   0 
0 0

So,  
2
n
 1 o


 0 

 0 
0


is a basis of the eigenspace Eλ2 (A) = Nul λ2 I5 − A , and we see that the eigenvalue
λ2 = 2 has geometric multiplicity 1.
For λ3 = 3, we have that
 
2 −2 0 0 0

 0 1 0 0 0 

λ 3 I5 − A = 
 0 0 2 −1 −3 ,

 0 0 0 0 −3 
0 0 0 0 0

and that  
1 0 0 0 0
 0 1 0 0 0 
1 − 12 0
 
RREF(λ3 I5 − A) = 
 0 0 

 0 0 0 0 1 
0 0 0 0 0
Consequently, the general solution of the equation (λ3 I5 − A)x = 0 is
     
0 0 0
 0   0   0 
 t 
 = t 1  = t  1 ,
   
x =   2   2  2  with t ∈ C.
 t   1   2 
0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 522

So,  
0
n
 0 o


 1 

 2 
0
is a basis of the eigenspace Eλ3 (A), and we see that the eigenvalue λ3 = 3 has
geometric multiplicity 1.

8.2.3 Eigenvectors and eigenvalues of similar matrices

Recall from subsection 4.5.2 that matrices A, B ∈ Fn×n (where F is a field) are said
to be similar if there exists an invertible matrix P ∈ Fn×n such that B = P −1 AP . By
Proposition 4.5.16, matrix similarity is an equivalence relation on Fn×n . Moreover,
by Theorem 4.5.19, two matrices in Fn×n are similar if and only if they are matrices
of the same linear function from an n-dimensional vector space (over F) to itself,
but possibly with respect to different bases. When it comes to eigenvalues and
eigenvectors, we have the following result for similar matrices.

Theorem 8.2.9. Let F be a field, and let A, B ∈ Fn×n be similar matrices. Then
A and B have the same characteristic polynomial, as well as the same eigenvalues,
with the same corresponding algebraic multiplicities, and the same corresponding
geometric multiplicities. Moreover, A and B have the same spectrum.

Warning: Similar matrices A and B need not have the same eigenspaces, that is,
for an eigenvalue λ of A and B:

Eλ (A) Z
= Eλ (B)


Proof. Let us first show that A and B have the same eigenvalues with the same
corresponding geometric multiplicities. Since A and B are similar, Theorem 4.5.19
guarantees that there
  exists a linear
 function
 f : Fn → Fn and bases A and B of Fn
such that A = A f A and B = B f B . But then by Proposition 8.1.7, the linear
 
function f and the matrix A = A f A have exactly the same eigenvalues, with
exactly the same corresponding
  geometric multiplicities, and the same holds for f
and the matrix B = B f B . So, A and B have exactly the same eigenvalues with
exactly the same corresponding geometric multiplicities.
It now remains to show that A and B have the same characteristic polynomial,
since this will (by definition) imply that A and B have the same spectrum, and in
particular, that the eigenvalues of A and B have the same corresponding algebraic
multiplicities. Since A and B are similar, we know that there exists an invertible
matrix P ∈ Fn×n such that B = P −1 AP . We now compute:

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 523

pB (λ) = det λIn − B

det λIn − P −1 AP


=

det P −1 (λIn − A)P

=

det(P −1 ) det λIn − A det(P )

= by Theorem 7.5.2

1


= det(P ) det λIn − A det(P ) by Corollary 7.5.3


= det λIn − A

= pA (λ).

This completes the argument.

Remark: The converse of Theorem 8.2.9 is false: two matrices in Fn×n (where F is
a field) that have the same characteristic polynomial, as well as the same eigenvalues,
with the same corresponding algebraic multiplicities, and the same corresponding
geometric multiplicities, need not be similar. We will see examples of this when we
study the “Jordan normal form” (see section 8.6).

8.2.4 A relationship between the spectrum, trace, and determinant

of a matrix
 
The trace of a square matrix A = ai,j n×n with entries in some field F is defined
to be trace(A) := ni=1 ai,i , i.e. the trace of A is the sum of entries on the main
P
diagonal of A. For example, for the matrix
 
1 2 3
A =  4 5 6 
7 8 9
in C3×3 , we have that trace(A) = 1 + 5 + 9 = 15.
Theorem 8.2.10. Let F be a field, let A = ai,j n×n be a matrix in Fn×n , and
 

assume that {λ1 , . . . , λn } is the spectrum of A. Then

(a) det(A) = λ1 . . . λn ;
(b) trace(A) = λ1 + · · · + λn .
Warning: Theorem 8.2.10 only applies if the spectrum of the matrix A ∈ Fn×n
contains n eigenvalues (counting algebraic multiplicities)! This will always be the
case if the field F is algebraically closed (for example, if F = C), but need not be
the case otherwise.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 524

Proof. By definition, we have that pA (λ) = det(λIn − A). On the other hand,
since {λ1 , . . . , λn } is the spectrum of A (and A is an n × n matrix), we see that
pA (λ) = (λ − λ1 ) . . . (λ − λn ).
(a) By setting λ = 0, we obtain

pA (0) = (0 − λ1 ) . . . (0 − λn ) = (−1)n λ1 . . . λn .

On the other hand, we have that

(∗)
pA (0) = det(0In − A) = det(−A) = (−1)n det(A),

where (*) follows from Proposition 7.2.3. It now follows that (−1)n λ1 . . . λn =
(−1)n det(A), and consequently, det(A) = λ1 . . . λn .
(b) We will compute the coefficient in front of λn−1 in the characteristic polynomial
pA (λ) in two ways.
First, since pA (λ) = (λ − λ1 ) . . . (λ − λn ), it is clear that the coefficient in front
of λn−1 is −λ1 − · · · − λn .
On the other hand, we have that
λ − a1,1 −a1,2 . . . −a1,n
−a2,1 λ − a2,2 . . . −a2,n
pA (λ) = det(λIn − A) = .. .. .. .. .
. . . .
−an,1 −an,2 . . . λ − an,n
We now use the definition of the determinant: the only permutation σ ∈ Sn that pro-
duces a polynomial with λn−1 appearing
 with it (with
 a possibly non-zero coefficient)
1 2 . . . n 12
is the identity permutation σ = , and clearly, this permutation
1 2 ... n
is even, i.e. has sign 1. So, the coefficient in front of λn−1 in pA (λ) is equal to the
coefficient of λn−1 in the product (λ − a1,1 )(λ − a2,2 ) . . . (λ − an,n ), and this coefficient
is precisely −a1,1 − a2,2 − · · · − an,n = −trace(A).
We have now computed the coefficient in front of λn−1 in the polynomial pA (λ) in
two ways: we got −λ1 − · · · − λn the first time, and we got −trace(A) the second time.
So, −λ1 − · · · − λn = −trace(A), and it follows that trace(A) = λ1 + · · · + λn .

8.2.5 Eigenvalues and invertibility

Proposition 8.2.11. Let F be a field, and let A ∈ Fn×n . Then A is invertible if
and only if 0 is not an eigenvalue of A.
Proof. It suffices to show that 0 is an eigenvalue of A if and only if A is not invertible.
We have the following sequence of equivalent statements:

12
Note that the identity permutation encodes the selection of the entire main diagonal.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 525

0 is eigenvalue of A ⇐⇒ det(0In − A) = 0 by Theorem 8.2.2

⇐⇒ det(−A) = 0

⇐⇒ (−1)n det(A) = 0 by Proposition 7.2.3

⇐⇒ det(A) = 0

⇐⇒ A is not invertible by Theorem 7.4.1.

This completes the argument.

8.2.6 The Invertible Matrix Theorem (version 4)

In this subsection, we state and prove the fourth and final version of the Invertible
Matrix Theorem. This version of the theorem is obtained from the previous one (see
subsection 7.4.1) by adding the eigenvalue condition from Proposition 8.2.11 as the
last item. Note that our final version of the Invertible Matrix Theorem uses up all
26 letters of the English alphabet!
The Invertible Matrix Theorem (version 4). Let F be a field, and let A ∈ Fn×n
be a square matrix. Further, let f : Fn → Fn be given by f (x) = Ax for all x ∈ Fn .13
Then the following are equivalent:
(a) A is invertible (i.e. A has an inverse);

(b) AT is invertible;

(c) RREF(A) = In ;

= In B for some matrix B ∈ Fn×n ;

 
 
(d) RREF A In

(e) rank(A) = n;

(f ) rank(AT ) = n;

(g) A is a product of elementary matrices;

(h) the homogeneous matrix-vector equation Ax = 0 has only the trivial solution (i.e.
the solution x = 0);

(i) there exists some vector b ∈ Fn such that the matrix-vector equation Ax = b
has a unique solution;

(j) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has a unique solution;
13
Since f is a matrix transformation, Proposition 1.10.4 guarantees that f is linear. Moreover, A
is the standard matrix of f .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 526

(k) for all vectors b ∈ Fn , the matrix-vector equation Ax = b has at most one
solution;

(l) for all vectors b ∈ Fn , the matrix-vector equation Ax = b is consistent;

(m) f is one-to-one;

(n) f is onto;

(o) f is an isomorphism;

(p) there exists a matrix B ∈ Fn×n such that BA = In (i.e. A has a left inverse);

(q) there exists a matrix C ∈ Fn×n such that AC = In (i.e. A has a right inverse);

(r) the columns of A are linearly independent;

(s) the columns of A span Fn (i.e. Col(A) = Fn );

(t) the columns of A form a basis of Fn ;

(u) the rows of A are linearly independent;

(v) the rows of A span F1×n (i.e. Row(A) = F1×n );

(w) the rows of A form a basis of F1×n ;

(x) Nul(A) = {0} (i.e. dim Nul(A) = 0);

(y) det(A) ̸= 0;

(z) 0 is not an eigenvalue of A.

Proof. Items (a)-(y) are the same as those from the Invertible Matrix Theorem
(version 3) from subsection 7.4.1. The equivalence of (a) and (z) follows from
Proposition 8.2.11.

8.2.7 The characteristic polynomial and spectrum of a linear func-

tion
In section 7.5, we defined the determinant of a linear function whose domain and
codomain and one and the same non-trivial, finite-dimensional vector space. Before
proceeding, let us recall this definition. Suppose that V is a non-trivial, finite-
dimensional vector space over a field F, and that f : V → V is a linear function.
Then we define the determinant of f to be
   
det(f ) := det B f B ,

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 527

where B is any basis of V . As we explained in section 7.5, the reason that  det(f )
is well defined is because, by Theorem 4.5.19, all matrices of the form B f B are
similar, and therefore (by Corollary 7.5.4) have the same determinant.
We can similarly define the characteristic polynomial of linear functions, as long
as their domain and codomain are one and the same non-trivial, finite-dimensional
vector space. So, let us once again suppose that V is a non-trivial, finite-dimensional
vector space over a field F. As usual, IdV is the identity function on V , i.e. it is
the function IdV : V → V given by IdV (v) = v for all v ∈ V . The characteristic
polynomial of a linear function f : V → V is defined to be the polynomial

   
pf (λ) := det λIdV − f = det B λIdV − f B ,

where B is any basis of V . As per our discussion above, the polynomial pf (λ) depends
only on f , and not on the particular choice of the basis B. The characteristic equation
of f is the equation

det λIdV − f = 0.
So, the roots of the characteristic polynomial of f are precisely the solutions of the
characteristic equation of f .
By Theorem 8.2.9, similar matrices have the same characteristic polynomial. In
view of the characterization of similar matrices given by Theorem 4.5.19, it should
not be surprising that the characteristic polynomial of a linear function is exactly
the same as the characteristic polynomial of its matrices (as long as we use the
same basis for the domain and codomain). More precisely, we have the following
proposition.

Proposition 8.2.12. Let V be a non-trivial, finite-dimensional vector space over

 B be any basis of V , let f : V → V be a linear function, and set
a field F, let
B := B f B . Then pf (λ) = pB (λ).

Proof. Set B = {b1 , . . . , bn }, and as usual, let En = {e1 , . . . , en } be the standard

basis of Fn . By Theorem 4.5.1, we have that
       
B
IdV B
= IdV (b 1 ) B
. . . IdV (b n ) B
     
= b1 B
... bn B
 
= e1 . . . en = In .

We now compute:


pf (λ) = det λIdV − f by definition
   
= det B λIdV − f B by definition

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 528

     
= det λ B IdV B − B f B by Theorem 4.5.3


= det λIn − B

= pB (λ) by definition.

This completes the argument.

We also have the following analog of Theorem 8.2.2.

Theorem 8.2.13. Let V be a non-trivial, finite-dimensional vector space over a

field F, let f : V → V be a linear function, and let λ0 ∈ F. Then



Eλ0 (f ) = Ker λ0 IdV − f = Ker f − λ0 IdV .

Moreover, the following are equivalent:

(1) λ0 is an eigenvalue of f ;

(2) λ0 is a root of the characteristic polynomial of f , i.e. pf (λ0 ) = 0;

(3) λ0 is a solution of the characteristic equation of f , i.e. det λ0 IdV − f = 0.

Proof. We proceed similarly as in
the proof of Theorem 8.2.2. Obviously,

for all
v ∈ V , we have

that λ0 IdV − f
(v) = 0 if and only if f − λ0 IdV (v) = 0. So,
Ker λ0 IdV − f = Ker f − λ0 IdV . Further, we compute:

Eλ0 (A) = v ∈ V | f (v) = λ0 v

= v ∈ V | f (v) = (λ0 IdV )(v)

v ∈ Fn | (λ0 IdV − f )(v) = 0


=


= Ker λ0 IdV − f .

It remains to show that (1), (2), and (3) are equivalent. The fact that (2)
and (3) are equivalent follows immediately from the definition of the characteristic
polynomial and the characteristic equation of f . It remains to show that
 (1) and (2)
are equivalent. For this, we fix any basis B of V , and set B := B f B . We then
have the following sequence of equivalent statements:
(∗)
λ0 is an eigenvalue of f ⇐⇒ λ0 is an eigenvalue of B
| {z }
(1)

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 529

(∗∗)
⇐⇒ λ0 is a root of pB (λ)

(∗∗∗)
⇐⇒ λ0 is a root of pf (λ),
| {z }
(2)

where (*) follows from Proposition 8.1.7, (**) follows from Theorem 8.2.2, and (***)
follows from the fact that pf (λ) = pB (λ) (by Proposition 8.2.12).

Suppose that f : V → V is a linear function, where V is a non-trivial, finite-

dimensional vector space over a field F. In view of Theorem 8.2.13, we may define
the algebraic multiplicity of an eigenvalue λ0 of f to be the largest positive integer k
such that (λ − λ0 )k divides the polynomial pf (λ). The spectrum of f is the multiset
of all the eigenvalues of f , with algebraic multiplicities taken into account.
Proposition 8.2.14. Let V be a non-trivial, finite-dimensional vector space over
a field F, let f : V → V be a linear function, and let B be any basis of V . Then
f and B f B have the same characteristic polynomial, and the same spectrum.
 
Moreover, f and B f B have exactly the same eigenvalues, with exactly the same
corresponding geometric multiplicities, and exactly the same corresponding algebraic
multiplicities.
 
Proof. The fact that f and B f B have the same eigenvalues, with the same
geometric multiplicities, follows immediately from Proposition 8.1.7. The fact that
they have the same characteristic polynomial (and consequently the
 same spectrum)
follows immediately from Proposition 8.2.12. Since f and B f B have the same
spectrum, their eigenvalues have the same algebraic multiplicities.

As a special case for linear functions of the form f : Fn → Fn (where F is a field)

and their standard matrices, we have the following proposition.
Proposition 8.2.15. Let F be a field, let f : Fn → Fn be a linear function, and let A
be the standard matrix of f . Then f and A have the same characteristic polynomial
and the same spectrum. Moreover, for each eigenvalue λ of f and A, all the following
hold:
ˆ the algebraic multiplicity of λ as an eigenvalue of f is the same as the algebraic
multiplicity of λ as an eigenvalue of A;
ˆ the geometric multiplicity of λ as an eigenvalue of f is the same as the geometric
multiplicity of λ as an eigenvalue of A;
ˆ Eλ (f ) = Eλ (A).
 
Proof. Since A is the standard matrix of f , we have that A = E f E , where En is
n n
the standard basis of Fn . The result now follows immediately from Propositions 8.1.5
and 8.2.14.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 530

Example 8.2.16. Consider the function f : P2R → P2R given by

f a2 x2 + a1 x + a0 = (a2 + a0 )x2 − a1 x + (a2 + a0 )

for all a0 , a1 , a2 ∈ R. Prove that f is linear, and compute its characteristic polynomial
and spectrum. Identify the eigenvalues of f , and for each eigenvalue λ of f , determine
its geometric and algebraic multiplicity, and compute a basis of the eigenspace Eλ (f ).

Solution. We first show that f is linear.

1. Fix polynomials p(x), q(x) ∈ P2R . Then there exist some a0 , a1 , a2 , b0 , b1 , b2 ∈ R
such that p(x) = a2 x2 + a1 x + a0 and q(x) = b2 x2 + b1 x + b0 . We now compute:
 
f p(x) + q(x) = f (a2 x2 + a1 x + a0 ) + (b2 x2 + b1 x + b0 )

 
= f (a2 + b2 )x2 + (a1 + b1 )x + (a0 + b0 )

(a2 + b2 ) + (a0 + b0 ) x2 − (a1 + b1 )x + (a2 + b2 ) + (a0 + b0 )

=

(a2 + a0 )x2 − a1 x + (a2 + a0 ) + (b2 + b0 )x2 − b1 x + (b2 + b0 )

=



= f p(x) + f q(x) .

2. Fix a polynomial p(x) ∈ P2R and a scalar α ∈ R. Since p(x) ∈ P2R , there exist
some a0 , a1 , a2 ∈ R such that p(x) = a2 x2 + a1 x + a0 . We now compute:
 
f αp(x) = f α(a2 x2 + a1 x + a0 )

 
= f (αa2 )x2 + (αa1 )x + (αa0 )

αa2 + αa0 x2 − (αa1 )x + αa2 + αa0

=

= α (a2 + a0 )x2 − a1 x + (a2 + a0 )

= αf p(x) .

From 1 and 2, we see that f is linear.

Now, consider the basis A = {1, x, x2 } of f . We compute:
 
A := A f A

(∗)
f (x2 )
       
= f (1) A
f (x) A A

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 531

x2 + 1 x2 + 1
       
= A
−x A A
 
1 0 1
=  0 −1 0  ,
1 0 1

where (*) follows from Theorem 4.5.1. We can now compute the characteristic
polynomial of f , as follows:
(∗)
pf (λ) = fA (λ) = det λI3 − A)

λ−1 0 −1
= 0 λ+1 0
−1 0 λ−1

(∗∗) λ − 1 −1
= (λ + 1)
−1 λ − 1

(λ + 1) (λ − 1)2 − 1


=

= λ(λ − 2)(λ + 1),

where (*) follows from Proposition 8.2.12, and (**) is obtained via Laplace expansion
along the second column. Optionally, we can compute the product above to obtain
pf (λ) = λ3 − λ2 − 2λ, although the characteristic polynomial is more useful in
factored form. The roots of the characteristic polynomial pf (λ) are λ1 = 0, λ2 = 2,
and λ3 = −1, each with multiplicity 1. So, the eigenvalues of f are λ1 = 0, λ2 = 2,
and λ3 = −1, each with algebraic multiplicity 1. The spectrum of f is {0, 2, −1}.
It remains to compute a basis of each of the three eigenspaces, and to determine
the geometric multiplicity of each eigenvalue. For each index i ∈ {1, 2, 3}, we will
first compute a basis of the eigenspace Eλi (A), and then we will use Proposition 8.1.7,
plus Theorem 4.4.4(c), to “translate” this bases into a basis of Eλi (f ).
We first deal with the eigenvalue λ1 = 0. We compute:
   

 −1 0 −1  1 0 1
RREF λ1 I3 − A =  0 1 0  =  0 1 0 .
−1 0 −1 0 0 0
 
n −1 o

Therefore,  0  is a basis of Eλ1 (A) = Nul λ1 I3 − A . But note that
1
 
−1  2 
 0  = x −1 A.
1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 532

By
 Proposition 8.1.7, Eλ1 (A) is simply the image of Eλ1 (f ) under the isomorphism
· A . So, by Theorem 4.4.4(c), {x2 − 1} is a basis of Eλ1 (f ). In particular, the


geometric multiplicity of λ1 = 0 as an eigenvalue of f is 1.

We next deal with the eigenvalue λ2 = 2. We compute:
   

 1 0 −1  1 0 −1
RREF λ2 I3 − A = RREF  0 3 0  =  0 1 0 .
−1 0 1 0 0 0
 
n 1 o

Therefore,  0  is a basis of Eλ2 (A) = Nul λ2 I3 − A . Note that
1
 
1  2 
 0  = x +1 A.
1

So, {x2 + 1} is basis of Eλ2 (f ). (The argument is analogous to the one we gave for
λ1 .) In particular, the geometric multiplicity of λ2 = 2 as an eigenvalue of f is 1.
Finally, we deal with the eigenvalue λ3 = −1. We compute:
   

 −2 0 −1  1 0 0
RREF λ3 I3 − A = RREF  0 0 0  =  0 0 1 .
−1 0 −2 0 0 0
 
n 0 o

Therefore,  1  is a basis of Eλ3 (A) = Nul λ3 I3 − A . Note that
0
 
0  
 1  = x A.
0

So, {x} is basis of Eλ3 (f ). (The argument is analogous to the one we gave for λ1 .)
In particular, the geometric multiplicity of λ3 = −1 as an eigenvalue of f is 1.
We now summarize our results as follows.

ˆ The characteristic polynomial of f is pf (λ) = λ(λ − 2)(λ + 1).

ˆ The spectrum of f is {0, 2, −1}.

ˆ The linear function f has three eigenvalues, namely λ1 = 0, λ2 = 1, and

λ3 = −1, and each of these three eigenvalues has algebraic multiplicity 1 and
geometric multiplicity 1.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 533

ˆ We have the following bases of the three eigenspaces of f :

– {x2 − 1} is a basis of Eλ1 (f ) = E0 (f );

– {x2 + 1} is a basis of Eλ2 (f ) = E2 (f );
– {x} is a basis of Eλ3 (f ) = E−1 (f ).

Optional: Since it is easy to miscompute, it is not a bad idea to check that each
vector in a basis of an eigenspace of f associated with λi (for i ∈ {1, 2, 3}) really is
an eigenvector of f associated with λi . For this, we compute:

ˆ f (x2 − 1) = (1 − 1)x2 − 0x + (1 − 1) = 0 = 0(x2 − 1) = λ1 (x2 − 1);

ˆ f (x2 + 1) = (1 + 1)x2 + 0x + (1 + 1) = 2x2 + 2 = 2(x2 + 1) = λ2 (x2 + 1);

ˆ f (x) = (0 + 0)x2 − 1x + (0 + 0) = (−1)x = λ3 x.

8.2.8 Proof of Theorem 8.2.3

In this subsection, we prove Theorem 8.2.3, which states that the geometric multiplic-
ity of an eigenvalue of a square matrix is no greater than the algebraic multiplicity
of that eigenvalue. In fact, it will be a bit more convenient to first prove an analog
of this theorem for linear functions (see Theorem 8.2.17 below), and to then derive
Theorem 8.2.3 as in immediate corollary.

Theorem 8.2.17. Let V be a non-trivial, finite-dimensional vector space over a

field F, and let f : V → V be a linear function. Then the geometric multiplicity of
any eigenvalue of f is no greater than the algebraic multiplicity of that eigenvalue.

Proof. Suppose that λ0 is an eigenvalue of f of geometric multiplicity k. We must

show that the eigenvalue λ0 has algebraic multiplicity at least k, that is, that
(λ − λ0 )k | pf (λ). The goal is to find a basis B of V for which it can  easily be shown
that (λ − λ0 )k divides the polynomial pB (λ), where B = B f B ; this is enough
because, by Proposition 8.2.12, pf (λ) = pB (λ).
Since the geometric multiplicity of the eigenvalue λ0 of f is k, we see that
the eigenspace Eλ0 (f ) has a k-element basis, say {b1 , . . . , bk }.14 In particular,
{b1 , . . . , bk } is a linearly independent set of vectors in V , and so by Theorem 3.2.19,
it can be extended to a basis B = {b1 , . . . , bk , bk+1 , . . . , bn } of V . As usual En =
{en1 , . . . , enn } is the standard basis of Fn . We now compute:

14
Obviously, b1 , . . . , bk are all eigenvectors of f associated with the eigenvalue λ0 , and they form
a linearly independent set of vectors in Fn .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 534

 
B := B
f B

(∗)          
= f (b1 ) B
... f (bk ) B
f (bk+1 ) B
... f (bn ) B

(∗∗)          
= λ 0 b1 B
... λ 0 bk B
f (bk+1 ) B
... f (bn ) B

λ0 en1 . . . λ0 enk
     
= f (bk+1 ) B
... f (bn ) B
 
λ 0 Ik    
= f (bk+1 ) B
... f (bn ) B
,
O(n−k)×k

where (*) follows from Theorem 4.5.1, and (**) follows from the fact that b1 , . . . , bk ∈
Eλ0 (f ).
If k = n, then we have that B = λ0 In , and so by Proposition 8.2.7, pB (λ) =
(λ − λ0 )n , and in particular, (λ − λ0 )k | pB (λ).
From now on, we may assume that k < n. We then have that

(λ − λ0 )Ik
pB (λ) = det λIn − B = C ,
O(n−k)×k

where
     
C = λek+1 − f (bk+1 ) B
. . . λen − f (bn ) B (n−k)×n
.

Thus, pB (λ) is of the form

λ − λ0 0 ... 0 ∗ ∗ ... ∗
0 λ − λ0 ... 0 ∗ ∗ ... ∗
.. .. .. .. .. .. . . ..
. . . . . . . .
0 0 . . . λ − λ0 ∗ ∗ ... ∗
pB (λ) = ,
0 0 ... 0 ∗ ∗ ... ∗
0 0 ... 0 ∗ ∗ ... ∗
.. .. .. .. .. .. . . ..
. . . . . . . .
0 0 ... 0 ∗ ∗ ... ∗

where the red submatrix in the upper-left corner (to the left of the vertical dotted
line, and above the horizontal dotted line) is of size k × k. By iteratively performing
Laplace expansion along the first column, we see that pB (λ) has a factor (λ − λ0 )k .
This completes the argument.

We are now ready to prove Theorem 8.2.3, restated below for the reader’s
convenience.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 535

Theorem 8.2.3. Let F be a field, and let A ∈ Fn×n . Then the geometric multiplicity
of any eigenvalue of A is no greater than the algebraic multiplicity of that eigenvalue.

Proof. Let fA : Fn → Fn be given by fA (x) = Ax for all x ∈ Fn . Then fA is linear

(by Proposition 1.10.4), and its standard matrix is A. By Proposition 8.2.15, A
and fA have exactly the same eigenvalues, with the same corresponding geometric
multiplicities, and the same corresponding algebraic multiplicities. The result now
follows from Theorem 8.2.17 applied to the linear function fA .

8.3 The Cayley-Hamilton theorem

The famous Cayley-Hamilton theorem essentially states that every square matrix is a
root of its own characteristic polynomial. (Here, we need to treat the free coefficient
of the characteristic polynomial as that coefficient times the identity matrix of the
appropriate size.) For example, for the matrix
 
1 2
A = ,
3 4

with entries understood to be in R or C, we have that

λ − 1 −2
pA (λ) = det(λI2 − A) = = λ2 − 5λ − 2,
−3 λ − 4

and we see that

 2    
1 2 1 2 1 0
A2 − 5A − 2I2 = −5 −2
3 4 3 4 0 1
     
7 10 5 10 2 0
= − −
15 22 15 20 0 2
 
0 0
= .
0 0

We note that the proof of the Cayley-Hamilton theorem relies on the adjugate
matrix (see section 7.8) and on Theorem 7.8.2. Let us now state and prove the
Cayley-Hamilton theorem.

The Cayley-Hamilton theorem. Let F be a field, let A ∈ Fn×n be a square matrix,

and let pA (λ) = λn + an−1 λn−1 + · · · + a1 λ + a0 be the characteristic polynomial of A.
Then
An + an−1 An−1 + · · · + a1 A + a0 In = On×n .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 536

Proof. If n = 1, then the result is immediate.15 So, assume that n ≥ 2. By

Theorem 7.8.2 applied to the matrix λIn − A (where λ is a variable), we get that

(λIn − A) adj(λIn − A) = det(λIn − A)In .

Now, note that each cofactor of the matrix λIn − A is a polynomial (in variable λ) of
degree at most λn−1 . Since the entries of adj(λIn − A) are precisely the cofactors of
λIn − A, it follows that each entry of adj(λIn − A) is a polynomial (in the variable λ)
of degree at most n − 1. So, the matrix adj(λIn − A) can be expressed in the form

adj(λIn − A) = λn−1 Bn−1 + λn−2 Bn−2 + · · · + λB1 + B0 ,

for some matrices B0 , B1 , . . . , Bn−1 ∈ Fn×n . Consequently,

(λIn − A)(λn−1 Bn−1 + λn−2 Bn−2 + · · · + λB1 + B0 ) = det(λIn − A)In .

| {z } | {z }
=adj(λIn −A) :=RHS
| {z }
:=LHS

For the left-hand-side, we have

LHS = (λIn − A)(λn−1 Bn−1 + · · · + λB1 + B0 )

= λn Bn−1 + λn−1 (Bn−2 − ABn−1 ) + λn−2 (Bn−3 − ABn−2 )+

+ · · · + λ(B0 − AB1 ) − AB0 .

For the right-hand-side, we have

RHS = det(λIn − A)In = pA (λ)In

= (λn + an−1 λn−1 + an−2 λn−1 + · · · + a1 λ + a0 )In

= λn In + λn−1 an−1 In + λn−2 an−2 In + · · · + λa1 In + a0 In .

The corresponding coefficients in front of λi (for i ∈ {0, 1, . . . , n}) must be equal on

the left-hand-side (LHS) and the right-hand-side (RHS). This yields the following
n + 1 equations.
Bn−1 = In
Bn−2 − ABn−1 = an−1 In
Bn−3 − ABn−2 = an−2 In
..
.
B0 − AB1 = a1 In
−AB0 = a0 In
15
Indeed, suppose that n = 1, and consider any matrix A = a1,1 in F1×1 . Then pA (λ) =
 

det(λI1 − A) = det( λ − a1,1 ) = λ − a1,1 , and we see that A − a1,1 I1 = O1×1 .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 537

We now multiply the first (top) equation by An on the left, the second equation by
An−1 on the left, the third equation by An−2 on the left, and so on. (The (n + 1)-th
equation, i.e. the bottom one, gets multiplied by A0 = In on the left, i.e. it remains
unchanged). This yields the following.

An Bn−1 = An
An−1 B n
n−2 − A Bn−1 = an−1 A
n−1

An−2 Bn−3 − An−1 Bn−2 = an−2 An−2

..
.
AB0 − A2 B1 = a1 A
−AB0 = a0 In

We now add up the equations that we obtained. On the left-hand-side, the sum is
obviously On×n . So, the right-hand-side must also sum up to On×n , i.e.

An + an−1 An−1 + an−2 An−2 + · · · + a1 A + a0 In = On×n .

But this is precisely what we needed to show.

Corollary 8.3.1. Let F be a field. For all matrices A ∈ Fn×n , both the following
hold:

(a) An ∈ Span(In , A, A2 , . . . , An−1 ), i.e. An is a linear combination of the matrices

In , A, A2 , . . . , An−1 ;

(b) if A is invertible, then A−1 ∈ Span(In , A, A2 , . . . , An−1 ), i.e. A−1 is a linear

combination of the matrices In , A, A2 , . . . , An−1 .

Proof. Fix a matrix A ∈ Fn×n , and consider its characteristic polynomial pA (λ) =
λn + an−1 λn−1 + an−2 λn−2 + · · · + a1 λ + a0 .
(a) By the Cayley-Hamilton theorem, we have that

An + an−1 An−1 + · · · + aa A2 + a1 A + a0 In = On×n .

Consequently,

An = −a0 In − a1 A − a2 A2 − · · · − an−1 An−1 .

Thus, An is a linear combination of the matrices In , A, A2 , . . . , An−1 .

(b) Assume that A is invertible. Proposition 8.2.11 then guarantees that 0 is
not an eigenvalue of A. Since the eigenvalues of A are precisely the roots of the
characteristic polynomial of A, we have that pA (0) ̸= 0; since pA (0) = a0 , it follows
that a0 ̸= 0.16
16
Indeed, 0 is not a root of pA (λ), and so pA (0) ̸= 0. But pA (0) = a0 , and it follows that a0 ̸= 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 538

Now, by the Cayley-Hamilton theorem, we have that

An + an−1 An−1 + · · · + a2 A2 + a1 A + a0 In = On×n .

We multiply both sides of the equation by A−1 on the right, and we obtain

An−1 + an−1 An−2 + · · · + a2 A + a1 In + a0 A−1 = On×n ,

and consequently,

a0 A−1 = −a1 In − a2 A − · · · − an−1 An−2 − An−1 .

Since a0 ̸= 0, this implies that

an−1 n−2
A−1 = − aa01 In − a2
a0 A − ··· − a0 A − 1 n−1
a0 A .

So, A−1 is a linear combination of the matrices In , A, A2 , . . . , An−1 .

8.4 Eigenvectors and linear independence. Eigenbases

For a finite-dimensional vector space V over a field F and a linear function f : V → V ,
an eigenbasis of V associated with f is a basis B of V such that all vectors in B are
eigenvectors of f . Similarly, for an field F and a matrix A ∈ Fn×n , an eigenbasis of
Fn associated with A is a basis B of Fn such that all vectors in B are eigenvectors
of A. Eigenbases do not always exist, and one of our goals in this section is to
determine when they do and do not exist. As we shall see in section 8.5, eigenbases
play a crucial role in matrix “diagonalization.”
Our first proposition of the section (Proposition 8.4.1 below) states that, for a
linear function f : V → V , where V is a vector space over a field F, any (finite)
set of eigenvectors of V associated with pairwise distinct eigenvalues is linearly
independent.

Proposition 8.4.1. Let V be a vector space over a field F, let f : V → V be a linear

function, let λ1 , . . . , λk ∈ F be pairwise distinct eigenvalues of f , associated with
eigenvectors v1 , . . . , vk , respectively. Then {v1 , . . . , vk } is a linearly independent set.

Proof. We will prove inductively that for all i ∈ {0, . . . , k}, the set {v1 , . . . , vi } is
linearly independent. For i = 0, we have that {v1 , . . . , vi } = ∅, which is obviously
a linearly independent set. Now, fix an index i ∈ {0, . . . , k − 1}, and assume
inductively that the set {v1 , . . . , vi } is linearly independent. We must show that
{v1 , . . . , vi , vi+1 } is linearly independent. Fix scalars α1 , . . . , αi , αi+1 ∈ F such that

α1 v1 + · · · + αi vi + αi+1 vi+1 = 0.

If we multiply both sides of the equation above by λi+1 , we obtain

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 539

(1) λi+1 α1 v1 + · · · + λi+1 αi vi + λi+1 αi+1 vi+1 = 0.

If, on the other hand, we apply the function f to both sides and also use the fact
that f (0) = 0 (by Proposition 4.1.6), then we obtain


(2) f α1 v1 + · · · + αi vi + αi+1 vi+1 = 0.
We now compute:
(2)

0 = f α1 v1 + · · · + αi vi + αi+1 vi+1

(∗)
= α1 f (v1 ) + · · · + αi f (vi ) + αi+1 f (vi+1 )

(∗∗)
= α1 λ1 v1 + · · · + αi λi vi + αi+1 λi+1 vi+1 ,

where (*) follows from the linearity of f (and more precisely, from Proposition 4.1.5),
and (**) follows from the fact that v1 , . . . , vi , vi+1 are eigenvectors of f associated
with eigenvalues λ1 , . . . , λi , λi+1 , respectively. Combining this with (1), we obtain

α1 λ1 v1 + · · · + αi λi vi + αi+1 λi+1 vi+1

= λi+1 α1 v1 + · · · + λi+1 αi vi + λi+1 αi+1 vi+1 .

By subtracting one side from the other and factoring, we get

α1 (λ1 − λi+1 )v1 + · · · + αi (λi − λi+1 )vi = 0

By the induction hypothesis, vectors v1 , . . . , vi are linearly independent, and it

follows that α1 (λ1 − λi+1 ) = · · · = α1 (λi − λi+1 ) = 0. Since λ1 − λi+1 , . . . , λi − λi+1
are all non-zero (because λ1 , . . . , λi , λi+1 are pairwise distinct), we deduce that
α1 = · · · = αi = 0. Plugging this into our equation α1 v1 + · · · + αi vi + αi+1 vi+1 = 0,
we get
αi+1 vi+1 = 0.
But vi+1 is an eigenvector of f , and so by definition, vi+1 ̸= 0. So, αi+1 = 0.
We have now shown that α1 = · · · = αi = αi+1 = 0, and we deduce that the set
{v1 , . . . , vi , vi+1 } is linearly independent. This completes the induction.

Proposition 8.4.2. Let V be a vector space over a field F, let f : V → V be a linear

function, and let λ1 , λ2 , . . . , λk ∈ F be pairwise distinct eigenvalues of f . For each
i ∈ {1, . . . , k}, let vi,1 , . . . , vi,ti be linearly independent eigenvectors of f associated
with the eigenvalue λi . Then the eigenvectors

v1,1 , . . . , v1,t1 , v2,1 , . . . , v2,t2 , . . . , vk,1 , . . . , vk,tk

are linearly independent.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 540

Proof. Fix scalars α1,1 , . . . , α1,t1 , α2,1 , . . . , α2,t2 , . . . , αk,1 , . . . , αk,tk ∈ F such that

k 
P 
αi,1 vi,1 + · · · + αi,ti vi,ti = 0.
i=1

Now, for each i ∈ {1, . . . , k}, set vi := αi,1 vi,1 + · · · + αi,ti vi,ti , that is

ˆ v1 := α1,1 v1,1 + · · · + α1,t1 v1,t1 ;

ˆ v2 := α2,1 v2,1 + · · · + α2,t2 v2,t2 ;

..
.

ˆ vk := αk,1 vk,1 + · · · + αk,tk vk,tk .

So,
v1 + v2 + · · · + vk = 0.
Now, note that for each i ∈ {1, . . . , k}, the vector vi is a linear combination of
vectors in Eλi (f );17 since (by Proposition 8.1.4) Eλi (f ) is a subsapce of V and is
therefore closed under linear combinations, it follows that vi ∈ Eλi (f ). Consequently,
for each i ∈ {1, . . . , k}, vi is either 0 or an eigenvector of f associated with the
eigenvalue λi . We claim that v1 = v2 = · · · = vk = 0. Suppose otherwise. After
possibly permuting the order of the λi ’s and the corresponding vi ’s, we may assume
that there exists some ℓ ∈ {1, . . . , k} such that v1 , . . . , vℓ are all non-zero (and are
consequently eigenvectors of f associated with λ1 , . . . , λℓ ), while vℓ+1 , . . . , vk are all
zero. So,
v1 + · · · + vℓ = 0,
and it follows that {v1 , . . . , vℓ } is a linearly dependent set. But this contradicts
Proposition 8.4.1. We have now shown that v1 = · · · = vk = 0. So, for all indices
i ∈ {1, . . . , k}, we have that αi,1 vi,1 + · · · + αi,ti vi,ti = 0; since vectors vi,1 , . . . , vi,ti
are linearly independent, it follows that αi,1 = · · · = αi,ti = 0. Since this holds for
all indices i ∈ {1, . . . , k}, we deduce that the eigenvectors

v1,1 , . . . , v1,t1 , v2,1 , . . . , v2,t2 , . . . , vk,1 , . . . , vk,tk

are linearly independent, which is what we needed to show.

Theorem 8.4.3. Let V be a non-trivial, finite-dimensional vector space over a field

F, and set n := dim(V ). Let f : V → V be a linear function, let λ1 , . . . , λk be all the
(distinct) eigenvalues of f , and let B1 , . . . , Bk be bases of the associated eigenspaces
Eλ1 (f ), . . . , Eλk (f ), respectively. Set B := B1 ∪ · · · ∪ Bk . Then all the following hold:
17
Indeed, since vi,1 , . . . , vi,ti are eigenvectors of f associated with the eigenvalue λi , they all
belong to Eλi (f ). By construction, vi is a linear combination of these eigenvectors.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 541

(a) B is a linearly independent set of eigenvectors of f ;

(b) dim Eλ1 (f ) + · · · + dim Eλk (f ) ≤ n, i.e. the sum of geometric multiplicities
of the eigenvalues of f is at most n;

(c) V has an eigenbasis associated with f if and only if the sum of geometric
multiplicities of the eigenvalues of f is n, and in this case, B is such an eigenbasis;

(d) V has an eigenbasis associated with f if and only if the sum of algebraic mul-
tiplicities of the eigenvalues of f is n, and the geometric multiplicity of each
eigenvalue is equal to its algebraic multiplicity; in this case, B is an eigenbasis
of V associated with the linear function f .

Proof. Part (a) follows immediately from Proposition 8.4.2. Part (b) follows from (a)
and from the fact that, by Theorem 3.2.17(a), any linearly independent set of vectors
in an n-dimensional vector space contains at most n vectors.
Let us prove (c). Suppose first that the sum of geometric multiplicities of the
eigenvalues of f is equal to n. Then B is a linearly independent set of size n in the
n-dimensional vector space V . So, by Corollary 3.2.20(a), B is a basis of V . Since all
vectors in B are eigenvectors of f , it follows that B is an eigenbasis of V associated
with f .
Suppose, conversely, that V has an eigenbasis C associated with f ; since dim(V ) =
n, we see that |C| = n. Since all vectors in C are eigenvecors of f , we see that they all
belong to Eλ1 (f ) ∪ · · · ∪ Eλk (f ). But since the basis C of V is, in particular, linearly
independent, we see that it cannot contain more than dim Eλi (f ) many vectors from
Eλi (f ) for any index i ∈ {1, . . . , k}.18 So, |C| ≤ dim Eλ1 (f ) + · · · + dim Eλk (f ) .

But now we have that

(b)
n = |C| ≤ dim Eλ1 (f ) + · · · + dim Eλk (f ) ≤ n,



and it follows that dim Eλ1 (f ) + · · · + dim Eλk (f ) = n, i.e. the sum of geometric
multiplicities of the eigenvalues of f is n. This proves (c).
It remains to prove (d). If the sum of algebraic multiplicities of the eigenvalues
of f is equal to n, and the geometric multiplicity of each eigenvalue is equal to its
algebraic multiplicity, then obviously, the sum of geometric multiplicities of f is
equal to n, and so by (c), V has an eigenbasis associated with f , and B is one such
eigenbasis. For the converse, assume that V has an eigenbasis C associated with
f . Let λ1 , . . . , λk be the eigenvalues of f , with geometric multiplicities g1 , . . . , gk ,
respectively, and algebraic multiplicities a1 , . . . , ak , respectively. By (c), we have
that g1 + · · · + gk = n. On the other hand, the characteristic polynomial of f is
of degree n, we see that the sum of algebraic multiplicitis of f is at most n, i.e.
18
Once again, we are using Theorem 3.2.17(a).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 542

a1 + · · · + ak ≤ n. But by Theorem 8.2.17, the geometric multiplicity of an eigenvalue

of f is no greater than the algebraic multiplicity of that eigenvalue, that is, gi ≤ ai
for all indices i ∈ {1, . . . , n}. We now have that
n = g1 + · · · + gk ≤ a1 + · · · + ak ≤ n,
and we deduce that a1 + · · · + ak = n and that gi = ai for all i ∈ {1, . . . , k}. This
proves (d).

Corollary 8.4.4. Let V be a non-trivial, finite-dimensional vector space over a

field F, and set n := dim(V ). If a linear function f : V → V has n distinct
eigenvalues, then V has an eigenbasis associated with f .
Proof. Let f : V → V be a linear function that has n distinct eigenvalues, say
λ1 , . . . , λn . By the definition of an eigenvalue,
19 we have that dim E



λi (f ) ≥ 1 for
all i ∈ {1, . . . , n}. Consequently, dim Eλ1 (f ) + · · · + dim
Eλn (f ) ≥ n. On
the
other hand, Theorem
8.4.3(b) guarantees
that dim Eλ1 (f ) + · · · + dim E λn (f ) ≤ n.
Thus, dim Eλ1 (f ) + · · · + dim Eλn (f ) = n, and so by Theorem 8.4.3(c), V has an
eigenbasis associated with f .

Propositions 8.4.1 and 8.4.2 can easily be “translated” into the language of
matrices, as can Theorem 8.4.3. The case of Propositions 8.4.1 and 8.4.2 is left
as an easy exercise for the reader. In the case of Theorem 8.4.3, we obtain the
Theorem 8.4.5 (below).
Theorem 8.4.5. Let F be a field, and let A ∈ Fn×n . Let λ1 , . . . , λk be all the
(distinct) eigenvalues of A, and let B1 , . . . , Bk be bases of the associated eigenspaces
Eλ1 (A), . . . , Eλk (A), respectively. Set B := B1 ∪ · · · ∪ Bk . Then all the following hold:
(a) B is a linearly independent set of eigenvectors of A;



(b) dim Eλ1 (A) + · · · + dim Eλk (A) ≤ n, i.e. the sum of geometric multiplicities
of the eigenvalues of A is at most n;
(c) Fn has an eigenbasis associated with A if and only if the sum of geometric
multiplicities of the eigenvalues of A is n, and in this case, B is such an eigenbasis;
(d) Fn has an eigenbasis associated with A if and only if the sum of algebraic
multiplicities of the eigenvalues of A is n, and the geometric multiplicity of each
eigenvalue is equal to its algebraic multiplicity; in this case, B is an eigenbasis
of Fn associated with the matrix A.
Proof. Define fA : Fn → Fn by setting fA (v) = Av for all v ∈ Fn . Then fA is
linear (by Proposition 1.10.4), and moreover, A is the standard matrix of fA . The
result now follows immediately from Proposition 8.2.15 (applied to fA and A) and
Theorem 8.4.3 (applied to fA ).
19
Or alternatively: by Proposition 8.1.4.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 543

Corollary 8.4.6. Let F be a field, and let A ∈ Fn×n . If A has n distinct eigenvalues,
then Fn has an eigenbasis associated with A.
Proof. Define fA : Fn → Fn by setting fA (x) = Ax for all x ∈ Fn . Then fA is linear
(by Proposition 1.10.4), and A is the standard matrix of A. The result now follows
immediately from Proposition 8.1.5 and Corollary 8.4.4.

8.5 Diagonalization
8.5.1 Diagonal matrices and their powers
For a field F, a square matrix D ∈ Fn×n is diagonal if all its entries off the main
diagonal are zero (the entries on the main diagonal may or may not be zero). For
scalars λ1 , λ2 , . . . , λn ∈ F, D(λ1 , λ2 , . . . , λn ) is the n × n matrix with λ1 , λ2 , . . . , λn
on the main diagonal (appearing in that order) and 0’s everywhere else, i.e.
 
λ1 0 . . . 0
 0 λ2 . . . 0 
D(λ1 , λ2 , . . . , λn ) :=  .
 
. .. . . .. 
 . . . . 
0 0 . . . λn
 
= λ1 e1 . . . λn en ,
where as usual, e1 , . . . , en are the standard basis vectors of Fn .
Note that diagonal matrices are, in particular, triangular. So, Propositions 7.3.1
and 8.2.7 apply. More precisely, for scalars λ1 , . . . , λn ∈ F (where F is a field), and
for the diagonal matrix D := D(λ1 , . . . , λn ), we have the following:
ˆ det(D) = λ1 . . . λn ;
ˆ pD (λ) = (λ − λ1 ) . . . (λ − λn ).

Proposition 8.5.1. Let F be a field, let λ1 , . . . , λn ∈ F (n ≥ 1) be arbitrary scalars,

and set D := D(λ1 , . . . , λn ). Then both the following hold:
T
in Fn , we have that

(a) for all vectors x = x1 . . . xn
 
λ 1 x1
Dx =  ...  ;
 

λ n xn

a1 . . . an in Fm×n , we have that

 
(b) for all matrices A =
 
AD = λ1 a1 . . . λ n an .
T
in Fn . We then compute:

Proof. (a) Fix a vector x = x1 . . . x n

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 544

 
x1
. . . λn en  ... 
 
Dx = λ1 e1


xn

(∗)
= x1 (λ1 e1 ) + · · · + xn (λn en )

= (λ1 x1 )e1 + · · · + (λn xn )en

 
λ 1 x1
 .. 
=  . ,
λ n xn

where (*) follows from the definition of matrix-vector multiplication.

(b) Fix a matrix A = a1 . . . an in Fm×n . We then compute:
 

 
AD = A λ1 e1 . . . λn en

(∗)  
= A(λ1 e1 ) . . . A(λn en )
 
= λ1 (Ae1 ) . . . λn (Aen )

(∗∗)  
= λ1 a1 . . . λn an ,
where (*) follows from the definition of matrix multiplication, and (**) follows from
Proposition 1.4.5.

Proposition 8.5.2 (below) states that if the product of two diagonal matrices is
another diagonal matrix.
Proposition 8.5.2. Let F be a field, and let λ1 , . . . , λn , µ1 , . . . , µn ∈ F (n ≥ 1) be
arbitrary scalars. Then

D(λ1 , . . . , λn ) D(µ1 , . . . , µn ) = D(λ1 µ1 , . . . , λn µn ).

Proof. We compute:
 
D(λ1 , . . . , λn ) D(µ1 , . . . , µn ) = λ1 e1 . . . λn en D(µ1 , . . . , µn )

(∗)  
= µ1 (λ1 e1 ) . . . µn (λn en )
 
= (λ1 µ1 )e1 . . . (λn µn )en

= D(λ1 µ1 , . . . , λn µn ),

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 545

where (*) follows from Proposition 8.5.1(b).

Proposition 8.5.3. Let F be a field, let λ1 , . . . , λn ∈ F (n ≥ 1), and set D :=

D(λ1 , . . . , λn ). Then both the following hold:
(a) for all non-negative integers m, we have that Dm = D(λm m
1 , . . . , λn );

(b) D is invertible if and only if λ1 , . . . , λn are all non-zero, and in this case, we
have that Dm = D(λm m
1 , . . . , λn ) for all integers m.

Proof. Part (a) follows from Proposition 8.5.2 via an easy induction on m (the details
are left as an exercise).
We now prove (b). By Theorem 7.4.1, we know that D is invertible if and only if
det(D) ̸= 0. Since det(D) = λ1 . . . λn (because D is diagonal), we deduce that D is
invertible if and only if λ1 , . . . , λn are all non-zero.
Now assume that D is invertible, so that λ1 , . . . , λn are all non-zero and therefore
have multiplicative inverses. Then
(∗)
D(λ1 , . . . , λn ) D(λ−1 −1
1 , . . . , λn ) = D(λ1 λ−1 −1
1 , . . . , λ n λ n ) = In ,
| {z }
=D

where (*) follows from Proposition 8.5.2. Corollary 3.3.20 now guarantees that
D−1 = D(λ−1 −1
1 , . . . , λn ). To obtain (b), we simply apply (a) twice: first to the
diagonal matrix D = D(λ1 , . . . , λn ), and then to the diagonal matrix D−1 =
D(λ−1 −1 20
1 , . . . , λn ).

8.5.2 Eigenbases and diagonal matrices of linear functions

Suppose that V is a non-trivial, finite-dimensional vector space over a field F, and
suppose that f : V → V is a linear function. Theorem 8.4.3 gave us a criterion
for determining whether V has an eigenbasis associated with the linear function f .
Theorem 8.5.4 (below) states that if such an eigenbasis exists, then the matrix of
f with respect to that basis is diagonal. As we saw in subsection 8.5.1, diagonal
matrices have particularly nice computational properties.
Theorem 8.5.4. Let V be a non-trivial, finite-dimensional vector space, let B =
{v1 , . . . , vn } be a basis of V , and let f : V → V be a linear function.
  Then B is
an eigenbasis of V associated with f if and only if the matrix B f B is diagonal.
Moreover, in this case, we have that
 
B
f B
= D(λ1 , . . . , λn ),
20
Indeed, by applying (a) to the diagonal matrix D = D(λ1 , . . . , λn ), we get that Dm =
D(λm m
1 , . . . , λn ) for all non-negative integers m. On the other hand, by applying (a)
to the di-
agonal matrix D−1 = D(λ−1 −1
1 , . . . , λn ), we get that (D ) = D (λ−1
−1 m m −1 m
1 ) , . . . , (λn ) , that is,
D−m = D(λ−m 1 , . . . , λ−m
n ) for all non-negative integers m. Combined, these two facts tell us that
Dm = D(λm m
1 , . . . , λn ) for all integers m.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 546

where λ1 , . . . , λn are the eigenvalues of f associated with the eigenvectors v1 , . . . , vn ,

respectively.

Proof. Suppose first that B is an eigenbasis of V associated with f . Then, by

definition, vectors v1 , . . . , vn are eigenvectors of f , and we let λ1 , . . . , λn , respectively,
be the associated eigenvalues. Then f (vi ) = λi vi for all indices i ∈ {1, . . . , n}, and
we have the following:
       
B
f B = f (v1 ) B . . . f (v1 ) B by Theorem 4.5.1
     
= λ 1 v1 B
... λ n vn B
 
= λ1 e1 . . . λn en

= D(λ1 , . . . , λn ).
 
Conversely, suppose that the matrix B f B is diagonal, and let λ1 , . . . , λn be
the entries of this matrix on the main diagonal, so that
   
B
f B = D(λ1 , . . . , λn ) = λ1 e1 . . . λn en .

We will show that the basis vectors v1 , . . . , vn are eigenvectors of f with associated
eigenvalues λ1 , . . . , λn , respectively. Fix any index i ∈ {1, . . . , n}; we
 must
 show that
f (vi ) = λi vi . Since vi is the i-th basis vector of B, we have that vi B = ei . We
now compute:
     
f (vi ) B = B f B vi B
 
= λ1 e1 . . . λn en ei

(∗)
= λi ei
 
= λi vi B

(∗∗)  
= λ i vi B
,

where
  (*) follows   from Proposition 1.4.5, and (**) follows from the linearity of
· B . Since · B is an isomorphism (and in particular, one-to-one), it follows that
f (vi ) = λi vi , which is what we needed to show.

Remark: Suppose that V is a non-trivial, finite-dimensional vector space over a

field F. By Theorems 4.3.2 and 8.5.4, linear functions from V to V that have a

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 547

diagonal matrix are precisely those that can be defined starting from some basis, and
then scaling each of the basis elements. Indeed, suppose that B = {v1 , . . . , vn } is
some basis of V , and that λ1 , . . . , λn ∈ F are some scalars. By Theorem 4.3.2, there
exists a unique linear function f : V → V such that f (vi ) = λi vi (for the special
case of R2 , see the picture below). But then by Theorem 8.5.4, we have that
 
B
f B = D(λ1 , . . . , λn ).

By Theorem 8.5.4, the converse also holds.

λ1v1
f λ2v2
v1

v2

Example 8.5.5. Consider the function f : P2R → P2R given by

f a2 x2 + a1 x + a0 = (a2 + a0 )x2 − a1 x + (a2 + a0 )

for all a0 , a1 , a2 ∈ R. We showed in Example 8.2.16 that f is linear. Moreover, in

that example, we obtained the following:

ˆ the characteristic polynomial of f is pf (λ) = λ(λ − 2)(λ + 1).

ˆ the spectrum of f is {0, 2, −1}.

ˆ the linear function f has three eigenvalues, namely λ1 = 0, λ2 = 1, and

λ3 = −1, and each of these three eigenvalues has algebraic multiplicity 1 and
geometric multiplicity 1.

ˆ we have the following bases of the three eigenspaces of f :

– {x2 − 1} is a basis of Eλ1 (f ) = E0 (f );

– {x2 + 1} is a basis of Eλ2 (f ) = E2 (f );
– {x} is a basis of Eλ3 (f ) = E−1 (f ).

Since dim(P2R ) = 3, Theorem 8.4.3(c) guarantees that B = {x2 − 1, x2 + 1, x} is an

eigenbasis of P2R associated
  with the linear function f . Theorem 8.5.4 now guarantees
that the matrix B f B is diagonal, and moreover, that
 
  0 0 0
B
f B
= D(λ1 , λ2 , λ3 ) = D(0, 2, −1) =  0 2 0 .
0 0 −1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 548

8.5.3 Matrix diagonalization

A matrix A ∈ Fn×n (where F is a field) is diagonalizable if it is similar to a diagonal
matrix. To diagonalize a diagonalizable matrix A means to compute a diagonal matrix
D and an invertible matrix P such that D = P −1 AP (equivalently: A = P DP −1 ).
Theorem 8.5.6 (below) gives a necessary and sufficient condition for a matrix to be
diagonalizable, and it essentially gives us a recipe for actually diagonalizing such a
matrix. We note that one reason why we care about diagonalizable matrices is that,
using Propositions 4.5.18 and 8.5.3, we can easily compute a formula for an arbitrary
power of a diagonalizable matrix A, at least provided we have actually diagonalized
it first (see Examples 8.5.10 and 8.5.11).

Theorem 8.5.6. Let F be a field, and let A ∈ Fn×n be a matrix. Then A is

diagonalizable if and only if Fn has an eigenbasis associated with A. Moreover, if
P = {p1 , . . . , pn } is any eigenbasis of Fn associated with A, and λ1 , . . . , λn are the
eigenvalues of A associated with the eigenvectors p1 , . . . , pn , respectively, then

D = P −1 AP and A = P DP −1 ,
 
where D = D(λ1 , . . . , λn ) and P = p1 . . . pn .

Remark: We could, in principle, obtain Theorem 8.5.6 as a corollary of Theo-

rem [Link] However, in this particular case, it is actually not much more difficult
to prove Theorem 8.5.4 “from scratch,” i.e. using matrices only.

Proof. Suppose first that Fn has an eigenbasis associated with A, and let P =
{p1 , . . . , pn } be such an eigenbasis. Let λ1 , . . . , λn be the eigenvalues of A associated
with the eigenvectors p1 , . . . , pn , so that Api = λi pi for all indices i ∈ {1, . . . , n}.
Further, set D := D(λ1 , . . . , λn ) and P := p1 . . . pn , as in the statement
of the theorem. Since the columns of P form a basis of Fn , the Invertible Matrix
Theorem (see subsection 8.2.6) guarantees that P is invertible. Now, it suffices to
show that P D = AP , since this will imply that D = P −1 AP and A = P DP −1
(because P is invertible). We compute:
 
PD = λ1 p1 . . . λn pn by Proposition 8.5.1(b)

  because Api = λi pi
= Ap1 . . . Apn
for all i ∈ {1, . . . , n}

  by the definition of
= A p1 . . . pn
| {z } matrix multiplication
=P

= AP.
21
Try it!

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 549

Suppose, conversely, that A is diagonalizable, and fix matrices D, P ∈ Fn×n such

that D −1
 is diagonal, P  is invertible, and D = P AP . Set D = D(λ1 , . . . , λn ) and
P = p1 . . . pn . Since P is invertible, the Invertible Matrix Theorem (see
subsection 8.2.6) guarantees that its columns form a basis of Fn (and in particular,
p1 , . . . , pn are all non-zero). Now, let us show that the columns of P are eigenvectors
of A. Since D = P −1 AP , we have that AP = P D. But note that by the definition
of matrix multiplication, we have that
   
AP = A p1 . . . pn = Ap1 . . . Apn ,

whereas by Proposition 8.5.1(b), we have that

 
PD = λ1 p1 . . . λn pn .

Since AP = P D, we deduce that

   
Ap1 . . . Apn = λ1 p1 . . . λn pn .

But this implies that for all i ∈ {1, . . . , n}, we have that Api = λi pi , and so pi
is an eigenvector of A associated with the eigenvalue λi .22 It now follows that
{p1 , . . . , pn } is an eigenbasis of Fn associated with the matrix A. This completes
the argument.

Corollary 8.5.7. Let F be a field, and let A ∈ Fn×n . If A has n distinct eigenvalues,
then A is diagonalizable.

Proof. Assume that A has n distinct eigenvalues. By Corollary 8.4.6, Fn has an

eigenbasis associated with A. So, by Theorem 8.5.6, A is diagonalizable.

Diagonalizing a matrix. Note that Theorems 8.4.5 and 8.5.6 together give us a
recipe for determining whether a matrix A ∈ Fn×n is diagonalizable, and if so, for
diagonalizing it (i.e. for finding a diagonal matrix D and an invertible matrix P ,
both in Fn×n , such that D = P −1 AP ). We proceed as follows.

1. We compute the characteristic polynomial pA (λ) and its roots. By Theo-

rem 8.2.2, the roots of pA (λ) are the eigenvalues of A, and we can read off the
algebraic multiplicities of those eigenvalues from the polynomial pA (λ).

ˆ Computing the roots of pA (λ) is the computationally tricky part, since

there is no formula for computing the roots of a high-degree polynomial. If
we cannot figure out how to compute the roots of pA (λ), then we are stuck:
the matrix A may or may not be diagonalizable, but computationally, we
cannot diagonalize it.
22
Note that we are also using the fact that pi ̸= 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 550

2. If the sum of algebraic multiplicities of the eigenvalues of A is less than n, then

by Theorem 8.4.5, Fn does not have an eigenbasis associated with A, and so
by Theorem 8.5.6, A is not diagonalizable.

3. From now on, we assume that the sum of algebraic multiplicities of the eigen-
values of A, call them λ1 , . . . , λk , is n. We then compute a basis Bi for each
eigenspace Eλi (A), which allows us to compute the geometric multiplicities of
all the eigenvalues of A.

4. If the geometric multiplicity of some eigenvalue of A is smaller than its algebraic

multiplicity,23 then by Theorem 8.4.5, Fn does not have an eigenbasis associated
with A, and so by Theorem 8.5.6, A is not diagonalizable.

5. From now on, we assume that the geometric multiplicity of each eigenvalue of
A is equal to its algebraic multiplicity. Theorem 8.4.5 then guarantees that Fn
has an eigenbasis associated with A, and moreover, that B = B1 ∪ · · · ∪ Bk is
one such eigenbasis.

6. By Theorem 8.5.6, A is diagonalizable. We now follow the recipe from Theo-

rem 8.5.6 to actually diagonalize A.

7. We form the matrix P whose columns are precisely the vectors in the eigenbasis
B. We form the diagonal matrix D, where on the main diagonal we place the
eigenvalues of A, taking care that, for each i ∈ {1, . . . , n}, the i-th entry on
the main diagonal of D is the eigenvalue associated with the i-th column of P
(which is, by construction, an eigenvector of A). Now D = P −1 AP .
Example 8.5.8. Consider the following matrix in C3×3 :
 
4 0 −2
A =  2 5 4 .
0 0 5
Determine whether A is diagonalizable, and if so, diagonalize it.
Solution. The matrix A is precisely the matrix from Example 8.2.4. In that example,
we determined that A has two eigenvalues, namely, λ1 = 4 (with algebraic multiplicity
1 and geometric multiplicity 1) and λ2 = 5 (with algebraic multiplicity 2 and geometric
multiplicity 2). Since the sum of algebraic multiplicities of the eigenvalues of A
is 3, and since the geometric multiplicity of each eigenvalue of A is equal to its
algebraic multiplicity, we see   3 × 3 matrix A is indeed diagonalizable. In
that the
n −1 o
Example 8.2.4, we saw that  2  is a basis of the eigespace Eλ1 (A), and that
0
23
By Theorem 8.2.3, the geometric multiplicity of an eigenvalue is either smaller than or equal to
the algebraic multiplicity of that eigenvalue.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 551

   
n 0 −2 o
 1  ,  0  is a basis of the eigenspace Eλ2 (A). So, we set
0 1
   
4 0 0 −1 0 −2
D :=  0 5 0  and P :=  2 1 0 ,
0 0 5 0 0 1
and we see that D = P −1 AP .
Example 8.5.9. Consider the following matrix in C5×5 :
 
1 2 0 0 0
 0 2 0 0 0 
 
A =   0 0 1 1 3 .

 0 0 0 3 3 
0 0 0 0 3
Determine whether A is diagonalizable, and if so, diagonalize it.
Solution. The matrix A is precisely the matrix from Example 8.2.8. In that example,
we determined that A has three eigenvalues, namely λ1 = 1 (with algebraic multi-
plicity 2 and geometric multiplicity 2), λ2 = 2 (with algebraic multiplicity 1 and
geometric multiplicity 1), and λ3 = 3 (with algebraic multiplicity 2 and geometric
multiplicity 1). Since the geometric multiplicity of the eigenvalue λ3 = 3 is strictly
smaller than the algebraic multiplicity, we see that A is not diagonalizable.

Matrix powers of diagonalizable matrices. If A is an arbitrary square matrix

with entries in some field F, then it is not easy to find a nice formula for arbitrary
powers of A. On the other hand, if A happens to be diagonalizable, then this can
easily be done using Propositions 4.5.18 and 8.5.3, at least provided we have actually
diagonalized the matrix first.
Example 8.5.10. Consider the following matrix in C3×3 :
 
4 0 −2
A =  2 5 4 .
0 0 5
Find a formula for Am , where m is an arbitrary non-negative integer. Does the
formula also work for negative integers m?
Solution. This is the matrix from Example 8.5.8. In that example, we computed
matrices D, P ∈ C3×3 such that D is diagonal, P is invertible, and D = P −1 AP .
The matrices in question were
   
4 0 0 −1 0 −2
D :=  0 5 0  and P :=  2 1 0 .
0 0 5 0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 552

We then compute  
−1 0 −2
P −1 =  2 1 4 .
0 0 1
Then for all non-negative integers m, we have the following:
(∗)
Am = P Dm P −1
  m  
−1 0 −2 4 0 0 −1 0 −2
(∗∗)
=  2 1 0   0 5m 0   2 1 4 
0 0 1 0 0 5m 0 0 1
| {z }| {z }| {z }
=P =Dm =P −1

4m 2 · 4m − 2 · 5m
 
0
(∗∗∗)
=  −2 · 4m + 2 · 5m 5m −4m+1 + 4 · 5m 
0 0 5m

4m 0 2(4m − 5m )
 

=  2(5m − 4m ) 5m 4(5m − 4m )  ,
0 0 5m

where (*) follows from Proposition 4.5.18, (**) follows from Proposition 8.5.3, and
(***) follows by simple matrix multiplication.
It remains to check whether our formula for Am also works for negative integers
m. If A is not invertible, then Am is not defined for negative integers m (and in
particular, the formula does not work for negative m). On the other hand, if A is
invertible, then Proposition 4.5.18 guarantees that our formula for Am does in fact
work for negative integers m. To see if A is invertible, we compute
(∗)
det(A) = det(D) = 4 · 5 · 5 = 100 ̸= 0

where (*) follows from Corollary 7.5.4, since matrices A and D are similar. Since
det(A) ̸= 0, Theorem 7.4.1 guarantees that A is invertible. So, our formula for Am
does in fact work for negative numbers m as well.
To summarize, we have shown that

4m 0 2(4m − 5m )
 

Am =  2(5m − 4m ) 5m 4(5m − 4m ) 
0 0 5m

for all integers m (positive, negative, and zero).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 553

Optional: Since it is easy to miscompute, it is not a bad idea to check that the
formula that we obtained is correct. We can do this by induction, as follows. For
m = 0, we have

40 0 2(40 − 50 )
 
 2(50 − 40 ) 50 4(50 − 40 )  = I3 = A0 .
0 0 50

Now, fix a non-negative integer m, and assume inductively that

4m 0 2(4m − 5m )
 

Am =  2(5m − 4m ) 5m 4(5m − 4m )  .
0 0 5m

We now compute:

4m 0 2(4m − 5m )
  
4 0 −2
Am+1 =  2(5m − 4m ) 5m 4(5m − 4m )   2 5 4 
0 0 5 m 0 0 5
| {z }| {z }
(∗) =A
= Am

4m+1 8 · 4m − 10 · 5m
 
0
(∗∗)
=  10 · 5m − 8 · 4m 5m+1 20 · 5m − 16 · 4m 
0 0 5m+1

4m+1 2(4m+1 − 5m+1 )

 
0
=  2(5m+1 − 4m+1 ) 5m+1 4(5m+1 − 4m+1 )  ,
0 0 5m+1

where (*) follows from the induction hypothesis, and (**) follows via simple matrix
multiplication. This completes the induction and proves that our formula is correct.
(Technically, we have only shown that our formula is correct for non-negative
integers m. We could also prove inductively that the formula is true for negative
integers m, but in practice, we need not bother. This is because this part is optional
anyway, and it simply serves to increase our confidence that we didn’t make any
mistakes in our computation.)

Example 8.5.11. Consider the following matrix in C2×2 :

 
6 −2
A = .
6 −1

Find a formula for Am , where m is an arbitrary non-negative integer. Does the

formula also work for negative integers m?

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 554

Solution. We will first try to diagonalize A. We start by computing and factoring

the characteristic polynomial of A:

pA (λ) = det(λI2 − A)

λ−6 2
=
−6 λ + 1

= λ2 − 5λ + 6

= (λ − 2)(λ − 3).

So, A has two eigenvalues: λ1 = 2 and λ2 = 3, each with algebraic multiplicity 1.

By Corollary 8.5.7, A is diagonalizable. So, let us diagonalize it.
First, we compute a basis of Eλ1 (A). We have the following:
  −4 2   
1 −1/2

RREF(λ1 I2 − A) = RREF = .
−6 3 0 0
n 1 o
So, is a basis of Eλ1 (A) = Nul(λ2 I2 − A).
2
We now compute a basis of Eλ2 (A). We have the following:
  −3 2   
1 −2/3

RREF(λ2 I2 − A) = RREF = .
−6 4 0 0
n 2 o
So, is a basis of Eλ2 (A) = Nul(λ2 I2 − A).
3
Now, for    
2 0 1 2
D := and P := ,
0 3 2 3
we have that D = P −1 AP . Moreover, we compute
 
  1 0 −3 2
RREF P I2 = ,
0 1 2 −1

and we deduce that  

−3 2
P −1 = .
2 −1
We note that
(∗) (∗∗)
det(A) = det(D) = 2 · 3 = 6 ̸= 0,

where (*) follows from Corollary 7.5.4 (because A and D are similar), and (**) follows
from Proposition 7.3.1 (because D is diagonal, and in particular, triangular). By the

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 555

Invertible Matrix Theorem (see subsection 8.2.6), it follows that A is invertible. So,
by Proposition 4.5.18, the following holds for all integers m:

Am = P Dm P −1

2m 0
   
1 2 −3 2 by Proposition 8.5.3,
=
2 3 0 3m 2 −1 since D is diagonal

−3 · 2m + 4 · 3m 2m+1 − 2 · 3m
 
= .
−3 · 2m+1 + 2 · 3m+1 2m+2 − 3m+1

This is the formula that we need, and as we saw, it works for all integers m (positive,
negative, and zero).

Optional: Let us check that our answer is correct, at least for non-negative integers
m. For m = 0, we have the following:

−3 · 20 + 4 · 30 20+1 − 2 · 30
 
= I2 = A0 .
−3 · 20+1 + 2 · 30+1 20+2 − 30+1

Now, fix a non-negative integer m, and assume inductively that

−3 · 2m + 4 · 3m 2m+1 − 2 · 3m
 
A m = .
−3 · 2m+1 + 2 · 3m+1 2m+2 − 3m+1

We now compute:

−3 · 2m + 4 · 3m 2m+1 − 2 · 3m
  
6 −2
Am+1 =
−3 · 2m+1 + 2 · 3m+1 2m+2 − 3m+1 6 −1
| {z } | {z }
(∗) =A
= Am

−6 · 2m + 12 · 3m 4 · 2m − 6 · 3m
 
(∗∗)
=
−6 · 2m+1 + 6 · 3m+1 4 · 2m+1 − 3 · 3m+1

−3 · 2m+1 + 4 · 3m+1 2m+2 − 2 · 3m+1

 
= ,
−3 · 2m+2 + 2 · 3m+2 2m+3 − 3m+2

where (*) follows from the induction hypothesis, and (**) follows via simple matrix
multiplication. So, our formula is correct.

Reading off the spectrum and bases of the eigenspaces of a square matrix
from its diagonalization. Suppose that we have successfully diagonalized a square
matrix A ∈ Fn×n (where F is a field), that is, that we have computed a diagonal matrix

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 556

D and an invertible matrix P , both in Fn×n , such that D = P −1 AP . Then we can

easily read off the spectrum and a basis of each eigenspace of A, as Proposition 8.5.12
(below) shows. We note that this proposition essentially summarizes various facts
about diagonalizable matrices that we have proven already, but it is convenient to
have them stated in one proposition.

Proposition 8.5.12. Let F be a field, and let A ∈ Fn×n . Assume −1

 that D = P AP ,
where D = D(λ1 , . . . , λn ) is a diagonal and P = p1 . . . pn an invertible matrix,
both in Fn×n . Then the characteristic polynomial of A is
n
Q
pA (λ) = (λ − λi ) = (λ − λ1 ) . . . (λ − λn ),
i=1

and the spectrum of A is {λ1 , . . . , λn }. Moreover, for each eigenvalue λ0 of A,24

the algebraic and geometric multiplicity of λ0 are both equal to the number of times
that λ0 appears on the main diagonal of D, and moreover, if λ0 appears precisely in
positions i1 , . . . , ik of the main diagonal of D, then the corresponding columns of P
(i.e. vectors pi1 , . . . , pik ) form a basis of the eigenspace Eλ0 (A). Finally, {p1 , . . . , pn }
is an eigenbasis of Fn associated with the matrix A.

Proof. First, we we note that

(∗) (∗∗)
pA (λ) = pD (λ) = (λ − λ1 ) . . . (λ − λn )

where (*) follows from Theorem 8.2.9 (because A and D are similar), and (**) follows
from Proposition 8.2.7 (because D is diagonal, and in particular, triangular). It now
immediately follows that the spectrum of A is {λ1 , . . . , λn }.

Claim. For all indices i ∈ {1, . . . , n}, pi is an eigenvector of A associated

with the eigenvalue λi .

Proof of the Claim. See the proof of Theorem [Link] ♦

24
So, λ0 ∈ {λ1 , . . . , λn }, since {λ1 , . . . , λn } is the spectrum of A.
25
For the sake of completeness, here is the proof again, copy-pasted from the proof of Theorem 8.5.6.
Since D = P −1 AP , we have that AP = P D. But note that by the definition of matrix multiplication,
we have that    
AP = A p1 . . . pn = Ap1 . . . Apn ,
whereas by Proposition 8.5.1(b), we have that
 
PD = λ1 p1 ... λn pn .

Since AP = P D, we deduce that

   
Ap1 . . . Apn = λ1 p1 ... λn pn .

But this implies that for all i ∈ {1, . . . , n}, we have that Api = λi pi , and so pi is an eigenvector of
A associated with the eigenvalue λi .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 557

Using the Claim, we can easily show that {p1 , . . . , pn } is an eigenbasis of Fn

associated with the matrix A. Indeed, since P is invertible, the Invertible Matrix
Theorem (see subsection 8.2.6) implies that the columns of P form a basis of Fn .
On the other hand, by the Claim, each column of P is an eigenvector of A. So,
{p1 , . . . , pn } is indeed an eigenbasis of Fn associated with the matrix A.
Now, suppose that λ0 is any eigenvalue of A. Since Fn has an eigenbasis (namely,
{p1 , . . . , pn }) associated with A, Theorem 8.4.5 guarantees that the geometric
multiplicity of each eigenvalue of A is equal to its algebraic multiplicity. In particular,
the geometric multiplicity of λ0 is equal to its algebraic multiplicity, which is precisely
the number of times that it appears on the main diagonal of D.
Let us suppose that the eigenvalue λ0 appears precisely k times on the main
diagonal of D (so that both the algebraic and geometric multiplicity of the eigenvalue
λ0 is k), and that it appears precisely in positions i1 , . . . , ik , so that λ0 = λi1 = · · · =
λik . We must show that B0 := {pi1 , . . . , pik } is a basis of the eigenspace Eλ0 (A).
First of all, by the Claim, all vectors in B0 are eigenvectors of A associated with the
eigenvalue λ0 , and so they belong to Eλ0 (A). Moreover, since {p1 , . . . , pn } is a basis
of Fn (proven above), the set B0 is linearly independent. So, B0 is
a set of k linearly
independent vectors in the eigenspace Eλ0 (A). Since dim Eλ0 (A) = k (because the
geometric multiplicity of the eigenvalue λ0 is k), Corollary 3.2.20 guarantees that B0
is in fact a basis of Eλ0 (A).

Example 8.5.13. Consider the following matrices in C6×6 (color coded for empha-
sis):
   
5 0 0 0 0 0 1 3 8 8 3 4

 0 4 0 0 0 0 


 2 8 0 0 0 2 

 0 0 5 0 0 0   5 4 6 4 5 0 
D =  , P =  .

 0 0 0 3 0 0 


 0 5 8 5 4 3 

 0 0 0 0 4 0   1 0 8 0 3 0 
0 0 0 0 0 4 0 2 0 3 0 2

It can be checked that P is invertible (for example, we can compute that det(P ) =
−1020 ̸= 0, and so by Theorem 7.4.1, P is invertible). We now set A = P DP −1 , so
that D = P −1 AP . Then by Proposition 8.5.12, all the following hold:
ˆ the characteristic polynomial of A is
pA (λ) = (λ − 3)(λ − 4)3 (λ − 5)2 ;

ˆ the spectrum of A is {5, 4, 5, 3, 4, 4}, which we can optionally reorder as

{3, 4, 4, 4, 5, 5};
ˆ the eigenvalues of A are 3 (with algebraic and geometric multiplicity 1), 4 (with
algebraic and geometric multiplicity 3), and 5 (with algebraic and geometric
multiplicity 2);

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 558

ˆ we can read off bases of the eigenspaces E3 (A), E4 (A), and E5 (A), as follows:

– a basis of E3 (A) is
 
8
 0 
n
 4
o

  ,

 5 

 0 
3
– a basis of E4 (A) is
     
3 3 4
 8   0   2 
n
 4
 
  5
 
  0
o

 , ,  ,

 5  
  4  
  3 

 0   3   0 
2 0 2

– a basis of E5 (A) is
   
1 8
 2   0 
n
 5
 
  6
o

 ,  ;

 0  
  8 

 1   8 
0 0
ˆ the columns of P form an eigenbasis of Cn associated with the matrix A.

8.6 The Jordan normal form

This section is by far the most technical one of these lecture notes. So, some reading
instructions are in order.
In subsection 8.6.1, we introduce matrices in “Jordan normal form” (also known
as “Jordan matrices”), which are upper triangular matrices of a particular kind.
In subsection 8.6.1, we also state our main results concerning such matrices. In
particular, we will see that if F is an algebraically closed field,26 then every
matrix A ∈ Fn×n is similar to some Jordan matrix J ∈ Fn×n (see Theorem 8.6.2);
the matrix J is in some sense unique, and it is referred to it as the “Jordan normal
form of A.” Moreover, Theorem 8.6.6 gives a recipe for computing the Jordan normal
form of a given square matrix (with entries in an algebraically closed field F), and
subsection 8.6.2 contains a couple of worked out examples applying this theorem.
26
Algebraically closed fields were discussed in subsection 2.4.5.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 559

In subsection 8.6.3, we give an outline of the proof of our main theorems. For most
readers, it is enough to read subsections 8.6.1, 8.6.2, and 8.6.3.
The remaining subsections (namely, subsections 8.6.4, 8.6.5, 8.6.6, and 8.6.7) are
highly technical and can be considered “optional reading” for particularly ambitious
students. Subsections 8.6.4, 8.6.5, 8.6.6 together give a formal proof of our main
theorems (the theorems stated in subsection 8.6.1). Subsection 8.6.7 is computational,
and it is essentially a more ambitious version of subsection 8.6.2. Suppose we are given
a square matrix A ∈ Fn×n , where F is an algebraically closed field. By Theorem 8.6.2,
the matrix A is similar to some matrix in Jordan normal form. In other words,
there exists a Jordan matrix J and an invertible matrix P , both in Fn×n , such that
J = P −1 AP . In subsection 8.6.2, we give a recipe for computing the Jordan matrix
J. In subsection 8.6.7, we give a recipe for computing both J and P ; the correctness
of this recipe essentially follows from the proofs given in subsections 8.6.4, 8.6.5,
and 8.6.6. We note, however, that readers who just wish to learn how to compute J
and P mechanically, without necessarily understanding why the procedure works,
can simply follow the steps described in subsection 8.6.7 (without having read the
proofs from subsections 8.6.4, 8.6.5, and 8.6.6 first).

8.6.1 The Jordan normal form: definitions and statements of main

theorems
In what follows, it will be notationally useful to define the “direct sum” of matrices.
So, suppose that F is a field and A ∈ Fn1 ×n1 and B ∈ Fn2 ×n2 are square matrices.
Then the direct sum of A and B is the (n1 + n2 ) × (n1 + n2 ) matrix
 
A On1 ×n2
A ⊕ B := .
On2 ×n1 B

More generally, for square matrices A1 ∈ Fn1 ×n1 , A2 ∈ Fn2 ×n2 , . . . , Ak ∈ Fnk ×nk , we
define the direct sum of A1 , A2 . . . , Ak to be the (n1 +n2 +· · ·+nk )×(n1 +n2 +· · ·+nk )
matrix
 
A1 On1 ×n2 . . . On1 ×nk
 On ×n A2 . . . On2 ×nk 
2 1
A1 ⊕ A2 ⊕ · · · ⊕ Ak :=  .
 
.. .. .. ..
 . . . . 
Onk ×n1 Onk ×n2 ... Ak
For example:
 
1 2 0 0 0 0
   3 4 0 0 0 0 
  1 2 3  
1 2    0 0 1 2 3 0 
⊕ 4 5 6 ⊕ 1
  =  .
3 4  0 0 4 5 6 0 
7 8 9  
 0 0 7 8 9 0 
0 0 0 0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 560

For a field F, a scalar λ0 ∈ F, and a positive integer t, the Jordan block Jt (λ0 ) is
defined to be following t × t matrix (with entries understood to be in F):
 
λ0 1 0 . . . 0 0
 0 λ0 1 . . . 0 0 
 
Jt (λ0 ) =  ... .. .. . . .. ..  .

 . . . . . 

 0 0 0 . . . λ0 1 
0 0 0 . . . 0 λ0 t×t

Thus, Jt (λ0 ) is a matrix in Ft×t , it has all λ0 ’s on the main diagonal, all 1’s on the
diagonal right above the main diagonal, and 0’s everywhere else. For example:

ˆ J1 (λ0 ) = λ0 ;
 

 
λ0 1
ˆ J2 (λ0 ) = ;
0 λ0
 
λ0 1 0
ˆ J3 (λ0 ) =  0 λ0 1 ;
0 0 λ0
 
λ0 1 0 0
 0 λ0 1 0 
ˆ J4 (λ0 ) =  0 0 λ0 1 ;


0 0 0 λ0
 
λ0 1 0 0 0
 0 λ0 1 0 0 
ˆ J5 (λ0 ) = 
 
 0 0 λ0 1 0 .

 0 0 0 λ0 1 
0 0 0 0 λ0

A Jordan matrix (also called a matrix in Jordan normal form) is any matrix that is
a direct sum of one or more Jordan blocks. Thus, a Jordan matrix is a matrix of the
form
 
Jt1 (λ1 ) O ... O
 O Jt2 (λ2 ) . . . O 
Jt1 (λ1 ) ⊕ Jt2 (λ2 ) ⊕ · · · ⊕ Jtℓ (λℓ ) =  ,
 
.. .. . . .
.
 . . . . 
O O . . . Jtℓ (λℓ )

where λ1 , . . . , λℓ are scalars in F, t1 , . . . , tℓ are positive integers, and the O’s are zero
matrices of appropriate sizes. For instance, the following is a Jordan matrix with
four Jordan blocks, namely J3 (5), J2 (2), J1 (2), and J3 (5):

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 561

5 1 0 0 0 0 0 0 0
 

 0 5 1 0 0 0 0 0 0 


 0 0 5 0 0 0 0 0 0 


 0 0 0 2 1 0 0 0 0 

J3 (5) ⊕ J2 (2) ⊕ J1 (2) ⊕ J3 (5) = 
 0 0 0 0 2 0 0 0 0 .


 0 0 0 0 0 2 0 0 0 


 0 0 0 0 0 0 5 1 0 

 0 0 0 0 0 0 0 5 1 
0 0 0 0 0 0 0 0 5

Remark: Every diagonal matrix is a Jordan matrix. Moreover, note that a Jordan
matrix is diagonal if and only if all its Jordan blocks are of size 1 × 1. On the other
hand, if some Jordan block of a Jordan matrix J is of larger size (i.e. is of size t × t
for some t ≥ 2), then J will have at least one 1 on the diagonal right above the main
diagonal.
Remark: Not all matrices that have an arbitrary main diagonal, all 0’s and 1’s on
the diagonal right above the main one, and 0’s everywhere else, are Jordan matrices.
For example, the matrix  
2 1
0 3
is not a Jordan matrix (because it is not a direct sum of Jordan blocks).

Theorem 8.6.1. Let F be a field, and let J1 , J2 ∈ Fn×n be Jordan matrices. Then J1
and J2 are similar if and only if they have exactly the same Jordan blocks (counting
repetitions, but not counting the order in which the blocks appear in the two matrices).
The proof of Theorem 8.6.1 is postponed to subsection 8.6.4. For now, let us
point out that the “if”(“⇐=”) part is fairly easy, whereas the “only if” (“=⇒”) part
requires some work. The “if” part, i.e. the fact that two Jordan matrices that have the
same Jordan blocks, counting repetitions, are indeed similar essentially follows from
the fact that similar matrices represent the same linear function, only with respect to
(possibly) different bases (see Theorem 4.5.19). A change in the order of Jordan blocks
corresponds to a change in the order of basis vectors. For a formal proof, see Proposi-
tion 8.6.12. For now, let us take a look at a special case in order to gain some intuition.
Suppose that V is a finite-dimensional vector space over a field F, that f : V → V
is a linear function, and that B = {a1 , . . . , at1 , b1 , . . . , bt2 , c1 , . . . , ct3 , d1 , . . . , dt4 }
(with t1 , t2 , t3 , t4 ≥ 1) is a basis of V such that
 
Jt1 (λ1 ) O O O
   O Jt2 (λ2 ) O O 
B
f B =  .
 O O Jt3 (λ3 ) O 
O O O Jt4 (λ4 )

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 562

Then for the basis C = {b1 , . . . , bt2 , d1 , . . . , dt4 , a1 , . . . , at1 , c1 , . . . , ct3 } of V , we have
the following:
 
Jt2 (λ2 ) O O O
   O Jt4 (λ4 ) O O 
C
f C =  .
 O O Jt1 (λ1 ) O 
O O O Jt3 (λ3 )
   
By Theorem 4.5.19, matrices B
f B
and C
f C
are similar, and so the two Jordan
matrices above are similar.
We now state two theorems involving Jordan matrices, namely, Theorems 8.6.2
and 8.6.4 below. As we shall see, the two theorems are equivalent (in the sense that
either one easily implies the other). The proofs of the two theorems are long and
technical; a proof outline is given in subsection 8.6.3, and a full proof is given in
subsections 8.6.4, 8.6.5, 8.6.6.

Theorem 8.6.2. Assume that F is an algebraically closed field, and let A ∈ Fn×n
be a square matrix. Then A is similar to a matrix J in Jordan normal form. Moreover,
this matrix J is unique up to a reordering of the Jordan blocks.

Terminology/Remark: Suppose that A ∈ Fn×n is a matrix, where F is some

algebraically closed field. Then any Jordan matrix that is similar to A is called
a Jordan normal form of A. As we saw above, reordering the Jordan blocks of a
Jordan matrix produces a Jordan matrix that is similar to the original one. So, if J
is a Jordan normal form of A, then any Jordan matrix obtained from J by merely
rearranging the order in which the Jordan blocks appear along the main diagonal is
also a Jordan normal form of A. However, by the uniqueness part of Theorem 8.6.2,
this exhausts the possibilities for different Jordan normal forms of A: any two Jordan
normal forms of A have exactly the same Jordan blocks (with repetitions taken into
account).
The following is an immediate corollary of Theorems 8.6.1 and 8.6.2 (plus
Proposition 4.5.16).

Corollary 8.6.3. Let F be an algebraically closed field, and let A, B ∈ Fn×n .

Then A and B are similar if and only if they have the same Jordan normal form.
More precisely, the following are equivalent:

(a) A and B are similar;

(b) there exists a Jordan matrix J ∈ Fn×n such that both A and B are similar to J;

(c) there exist Jordan matrices JA , JB ∈ Fn×n such that A is similar to JA , B is

similar to JB , and the Jordan matrices JA and JB can be obtained from each
other by possibly rearranging the order of the Jordan blocks.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 563

Proof (assuming Theorems 8.6.1 and 8.6.2). In what follows, we will use the fact
that, by Proposition 4.5.16, matrix similarity is an equivalence relation on Fn×n . We
will prove “(a) =⇒ (b) =⇒ (c) =⇒ (a).”
We first assume (a) and prove (b). By Theorem 8.6.2, A is similar to a Jordan
matrix J ∈ Fn×n . Since matrix similarity is an equivalence relation on Fn×n , it
follows that B is similar to J.27 This proves (b).
Next, we assume (b) and prove (c). By (b), A and B are both similar to the
same Jordan matrix J ∈ Fn×n . But then we simply set JA := J and JB := J, and (c)
follows.
Finally, we assume (c) and prove (a). Let JA and JB be as in part (c). Then
by Theorem 8.6.1, JA and JB are similar. Since matrix similarity is an equivalence
relation on Fn×n , we deduce that A is similar to B,28 i.e. (a) holds.

Remark: As we know, the field C is algebraically closed, and so Corollary 8.6.3

applies to matrices in Cn×n . On the other hand, R is not algebraically closed, and
so we cannot apply Corollary 8.6.3 to matrices in Rn×n , or at least not directly. For
a way around this, see Theorem 8.6.7.

Theorem 8.6.4. Let V be a non-trivial, finite-dimensional vector space over an

algebraically closed field F, and let f : V → V be a linear function. Then there
 
exists a basis B such that the matrix B f B is in Jordan normal form. Moreover,
this matrix
  the following sense: if B1 and B2 are bases of V such that
is unique in
both B f B and B f B are in Jordan normal form, then these two matrices
1 1 2 2
are the same up to a reordering of the Jordan blocks.

Remarks:
1. Theorems 8.6.2 and 8.6.4 only hold for algebraically closed fields. The only
algebraically closed field that we have seen is C, but others do exist.

2. Theorem 4.5.19 essentially states that two n × n matrices are similar if and
only if they represent the same linear function from an n-dimensional vector
space to itself, only possibly with respect to different bases. It is then easy to
show that Theorems 8.6.2 and Theorems 8.6.4 are equivalent in the sense that
either one of them (combined with Theorem 4.5.19) readily implies the other.
The details are left as an exercise.

3. As we saw in section 8.5, not all square matrices are diagonalizable, i.e. there
are square matrices that are not similar to any diagonal matrix. However, as
long as we are working over an algebraically closed field, Theorem 8.6.2
guarantees that any square matrix is similar to a matrix that is “almost
27
Indeed, we have that A is similar to both B and and J. So, B is similar to J.
28
Indeed, we have that A is similar to JA , that JA is similar to JB , and that JB is similar to B.
So, A is similar to B.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 564

diagonal,” namely to its Jordan normal form. However, in the special case
when a square matrix A is diagonalizable, the Jordan normal form of A is any
diagonal matrix D that is similar to A.29

4. Since every Jordan matrix is upper triangular, its eigenvalues, together with
their algebraic multiplicities, can easily be read off from the Jordan matrix
itself (see Proposition 8.2.7): the eigenvalues are precisely the entries along
the main diagonal of the Jordan matrix, and the algebraic multiplicity of each
eigenvalue is the number of times that it appears on the main diagonal. For
instance, the eigenvalues of the Jordan matrix

5 1 0 0 0 0 0 0 0
 

 0 5 1 0 0 0 0 0 0 


 0 0 5 0 0 0 0 0 0 


 0 0 0 2 1 0 0 0 0 

J3 (5) ⊕ J2 (2) ⊕ J1 (2) ⊕ J3 (5) = 
 0 0 0 0 2 0 0 0 0 


 0 0 0 0 0 2 0 0 0 


 0 0 0 0 0 0 5 1 0 

 0 0 0 0 0 0 0 5 1 
0 0 0 0 0 0 0 0 5

are 5 (with algebraic multiplicity 6) and 2 (with algebraic multiplicity 3).

5. Perhaps more interestingly, the geometric multiplicity of each eigenvalue of a

Jordan matrix J can also be read off quite easily: the geometric multiplicity of
each eigenvalue λ is precisely the number of Jordan blocks of the form Jt (λ)
that appear along the main diagonal of J.30 For instance, for the Jordan matrix
above, the geometric multiplicity of the eigenvalue 5 is 2, and the geometric
multiplicity of the eigenvalue 2 is also 2.

6. By Theorem 8.2.9, similar matrices have the same eigenvalues, with the same
corresponding algebraic multiplicities, and the same corresponding geometric
multiplicities. So, if we know the Jordan normal form of a matrix A, then
we can easily read off the eigenvalues of A, together with their algebraic and
geometric multiplicities. We note, however, that two square matrices of the
same size, and with exactly the same eigenvalues, with the same corresponding
algebraic and geometric multiplicities, need not be similar. Indeed, it is easy
to construct two Jordan matrices that have different Jordan blocks, but have
the same eigenvalues with the same corresponding algebraic and geometric
multiplicities. By Theorem 8.6.1, such matrices are not similar. For a concrete
example, consider the Jordan matrices J2 (λ) ⊕ J2 (λ) and J3 (λ) ⊕ J1 (λ), where
29
All such diagonal matrices D have the spectrum of A on the main diagonal (in some order),
and they are all similar to each other.
30
Check this!

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 565

λ is an arbitrary scalar from the field in question; these two matrices have
only one eigenvalue, namely λ, with algebraic multiplicity 4 and geometric
multiplicity 2, but they have different Jordan blocks and are therefore not
similar.

Example 8.6.5. Let A1 , A2 , A3 ∈ C7×7 be matrices whose Jordan normal forms are
J1 , J2 , J3 , respectively, as follows:
 
0 0 0 0 0 0 0
 0 1 1 0 0 0 0 
 
 0 0 1 1 0 0 0 
ˆ J1 = 
 
 0 0 0 1 0 0 0 ;

 0 0 0 0 1 0 0 
 
 0 0 0 0 0 0 1 
0 0 0 0 0 0 0
 
1 0 0 0 0 0 0
 0 0 1 0 0 0 0 
 
 0 0 0 0 0 0 0 
ˆ J2 = 
 
 0 0 0 0 0 0 0 ;

 0 0 0 0 1 1 0 
 
 0 0 0 0 0 1 1 
0 0 0 0 0 0 1
 
0 0 0 0 0 0 0

 0 1 1 0 0 0 0 

 0 0 1 0 0 0 0 
ˆ J3 = 
 
 0 0 0 1 1 0 0 .


 0 0 0 0 1 0 0 

 0 0 0 0 0 0 1 
0 0 0 0 0 0 0
Determine which (if any) of A1 , A2 , A3 are similar. Then, for each i ∈ {1, 2, 3},
compute its characteristic polynomial and spectrum, and find all the eigenvalues of
Ai , along with their algebraic and geometric multiplicities.
Solutution. We first identify the Jordan blocks of the the three Jordan matrices. In
each matrix, we use colors to indicate the Jordan blocks.
 
0 0 0 0 0 0 0
 0 1 1 0 0 0 0 
 
 0 0 1 1 0 0 0 
ˆ J1 = 
 
 0 0 0 1 0 0 0  = J1 (0) ⊕ J3 (1) ⊕ J1 (1) ⊕ J2 (0);

 0 0 0 0 1 0 0 
 
 0 0 0 0 0 0 1 
0 0 0 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 566

 
1 0 0 0 0 0 0

 0 0 1 0 0 0 0 

 0 0 0 0 0 0 0 
ˆ J2 = 
 
0 0 0 0 0 0 0  = J1 (1) ⊕ J2 (0) ⊕ J1 (0) ⊕ J3 (1);
 

 0 0 0 0 1 1 0 

 0 0 0 0 0 1 1 
0 0 0 0 0 0 1
 
0 0 0 0 0 0 0

 0 1 1 0 0 0 0 

 0 0 1 0 0 0 0 
ˆ J3 = 
 
0 0 0 1 1 0 0  = J1 (0) ⊕ J2 (1) ⊕ J2 (1) ⊕ J2 (0).
 

 0 0 0 0 1 0 0 

 0 0 0 0 0 0 1 
0 0 0 0 0 0 0

We see that J1 and J2 have the same Jordan blocks (counting repetitions), and so
A1 and A2 are similar. On the other hand, the Jordan blocks of the matrix J3 are
different from those of J1 and J2 , and so A3 is not similar to A1 and A2 .
For each i ∈ {1, 2, 3}, we see that the characteristic polynomial of Ai is
(∗) (∗∗)
pAi (λ) = pJi (λ) = λ3 (λ − 1)4 ,

where (*) follows from the fact that Ai and Ji are similar (we are using Proposi-
tion 8.2.9), and (**) from the fact that the Jordan matrix Ji is upper triangular
(we are using Proposition 8.2.7). Finally, we see from the matrices J1 , J2 , J3 , that
A1 , A2 , A3 all have spectrum {0, 0, 0, 1, 1, 1, 1}, and that they all have exactly two
eigenvalues: the eigenvalue 0 with algebraic multiplicity 3 and geometric multiplicity
2, and the eigenvalue 1 with algebraic multiplicity 4 and geometric multiplicity 2.

Computing the Jordan normal form. The following theorem allows us to

actually compute the Jordan normal form of a square matrix (with entries in an
algebraically closed field F).

Theorem 8.6.6. Let F be an algebraically closed field, let A ∈ Fn×n , and let

λ1 , . . . , λ 1 , . . . , λ k , . . . , λ k
| {z } | {z }
m1 mk

be the spectrum of A, where λ1 , . . . , λk are pairwise distinct eigenvalues of A and

m1 , . . . , mk are positive integers.31 Then A is similar to a matrix J ∈ Fn×n in
Jordan normal form that has the following properties:
31
Since F is algebraically closed, we know that m1 + · · · + mk = n.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 567

(i) each Jordan block of the Jordan matrix J is of the form Jt (λi ) for some
i ∈ {1, . . . , k} and t ∈ {1, . . . , mi };

(ii) for each i ∈ {1, . . . , k} and each positive integer r, the Jordan matrix J has
exactly
rank (A − λi In )r−1 − rank (A − λi In )r

many Jordan blocks Jt (λi ) satisfying t ≥ r.

Moreover, A is similar to any Jordan matrix in Fn×n that satisfies conditions (i)
and (ii) above.

The proof of Theorem 8.6.6 is given in subsection 8.6.6 (the proof relies on the
rather technical results of subsections 8.6.4 and 8.6.5). For now, let us just note
that Theorem 8.6.6 does indeed allow us to compute the Jordan normal form of a
square matrix A with entries in an algebraically closed field F, as long as we are able
to factor its characteristic polynomial into linear terms.32 Indeed, condition (i) of
Theorem 8.6.6 tells us what sorts of Jordan blocks the Jordan normal form of A may
possibly have. Condition (ii) gives us an easy way to compute the number of Jordan
blocks of each type. Indeed, using the set-up and notation from Theorem 8.6.6, we
consider an eigenvalue λi of A, and we fix a positive integer r. Then the number of
Jordan blocks Jr (λi ) in the Jordan normal form of A is exactly
! !
r−1
r
r
r+1

rank (A − λi In ) − rank (A − λi In ) − rank (A − λi In ) − rank (A − λi In ) .
| {z } | {z }
= number of Jordan blocks = number of Jordan blocks
Jt (λi ) satisfying t ≥ r Jt (λi ) satisfying t ≥ r + 1

So, we can compute both the possible types of Jordan blocks that the Jordan normal
form of A may have, and the exact number of blocks of each possible type. The
reader may have noticed that we in fact get an exact formula

rank (A − λi In )r−1 + rank (A − λi In )r+1 − 2 rank (A − λi In )r

for the number of Jordan blocks Jr (λi ) in the Jordan normal form of A. However, it
is arguably easier to memorize the formula for the number of Jordan blocks of the
form Jt (λi ) satisfying t ≥ r. For a couple of numerical examples, see subsection 8.6.2.

Matrix similarity over a field. A field F1 is a subfield of a field F2 if the following

three conditions are satisfied:
32
Any non-constant polynomial with coefficients in an algebraically closed field F can be factored
into linear terms (with coefficients in F). However, this is merely an existence result: actually
computing the linear factors may be extremely difficult or even impossible. If we get stuck factoring
the characteristic polynomial into linear terms, then Theorem 8.6.6 is of no use to us (computationally
speaking).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 568

ˆ F1 ⊆ F2 ;

ˆ for all a, b ∈ F1 , the sum a + b is the same in F1 and in F2 ;

ˆ for all a, b ∈ F1 , the product ab is the same in F1 and in F2 .

For example, Q is a subfield of both R and C, and R is a subfield of C. On the other

hand, for distinct prime numbers p and q, Zp is not a subfield of Zq (even if p < q).
Moreover, for a prime number p, Zp is not a subfield of any one of Q, R, or C. It
can be shown that any field is a subfield of some algebraically closed field, but the
proof of this fact is beyond the scope of these lecture notes, and we omit it. However,
let us just point out that the field R is a subfield of the algebraically closed field C.33
For a field F, we say that n × n matrices A and B with entries in F are similar
over F if there exists an invertible matrix P ∈ Fn×n such that B = P −1 AP . The
reader will have noticed that this is simply our usual definition of matrix similarity
in Fn×n . However, if F is a subfield of some larger field F, e then it makes sense to
speak of A and B being (or not being) similar over F, or of them being (or not being)
similar over F.e In fact, it can be shown that the two notions are equivalent. More
precisely, it can be shown that if F is a subfield of F,e then n × n matrices A and B,
with entries in F, are similar over F if and only if they are similar over F, e that is,
the following are equivalent:

ˆ there exists an invertible matrix P ∈ Fn×n such that B = P −1 AP ;

ˆ there exists an invertible matrix P ∈ F

en×n such that B = P −1 AP .

We will not prove this in full generality, since it would involve theory that is beyond
the scope of these lecture notes. However, for the special case of R and C, we can
give a proof that that is both fully formal and fairly simple (see Theorem 8.6.7
below). Before turning to the special case of R and C, let us explain what similarity
over different fields had to do with the Jordan normal form. Suppose that we need
to check if two n × n matrices, call them A and B, with entries in some field F,
are similar (over F). We first extend F to an algebraically closed field F.e Then the
following are equivalent:

ˆ A and B are similar over F;

ˆ A and B are similar over F;

e

ˆ A and B have the same Jordan normal form in F

en×n (up to a reordering of
the Jordan blocks).

(The equivalence of the second and third item above follows from Corollary 8.6.3.)
So, if we can compute the Jordan normal forms of A and B in F en×n , then we can
immediately determine if A and B are similar over F. Of course, actually computing
33
Of course, Q is also a subfield of the algebraically closed field C.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 569

the Jordan normal forms of A and B (in F en×n ) may be very difficult or even
impossible, essentially because we might not succeed in factoring the characteristic
polynomials pA (λ) and pB (λ).34
We now turn to the special case of R and C. As we pointed out above, we have
all the tools that we need to prove that two n × n matrices with real entries are
similar over R if and only if they are similar over C.
Theorem 8.6.7. Two n × n matrices with real entries are similar over R if and
only if they are similar over C.
Proof. Fix n×n matrices A and B with real entries. We must show that the following
are equivalent:
ˆ A and B are similar over R, that is, there exists an invertible matrix P ∈ Rn×n
such that B = P −1 AP ;
ˆ A and B are similar over C, that is, there exists an invertible matrix P ∈ Cn×n
such that B = P −1 AP .
If A and B are similar over R, then they are obviously similar over C. For the
converse, we assume that A and B are similar over C, and we prove that they
are similar over R. Fix an invertible matrix P ∈ Cn×n such that B = P −1 AP ,
so that P B = AP . Obviously, there exist n × n matrices R and Q with real
entries such that P = R + iQ.35 So, (R + iQ)B = A(R + iQ), and consequently,
RB + i(QB) = AR + i(AQ). By separating the real and imaginary parts (and
relying on the fact A, B, R, Q all have only real entries), we get that RB = AR and
QB = AQ.36 Therefore,
(R + cQ)B = A(R + cQ)
for all c ∈ C. It now suffices to show that there exists a real number c such that the
matrix R + cQ is invertible; we will then have that B = (R + cQ)−1 A(R + cQ), so
that A and B are similar over R (because all entries of R + cQ are real), which is
what we need.
Consider the polynomial
q(x) := det(R + xQ).
34
Since Fe is algebraically closed, the characteristic polynomials pA (λ) and pB (λ) can be factored
into linear terms with coefficients in F.e However, as we have pointed out a number of times already,
this is only an existence statement: we have no general recipe for factoring.  
35 2 + i −3i 2 0
For example, if P = , then we have that P = R + iQ for R := and
1 − 2i −7 1 −7
 
1 −3
Q := .
−2 0
36
If R is invertible, then we have that B = R−1 AR, and so A and B are similar over R, and we
are done. We are similarly done if Q is invertible. Unfortunately, it is possible that neither R nor Q
is invertible, which is why we are not done yet.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 570

Since R and Q are n × n matrices with real entries, we see that q(x) is a polynomial
with real coefficients and of degree at most n. Now, since P is invertible, the
Invertible Matrix Theorem (see subsection 8.2.6) guarantees that det(P ) ̸= 0. So,
q(i) = det(R + iQ) = det(P ) ̸= 0. We have now shown that q(x) is a non-zero
polynomial with real coefficients and of degree at most n. Therefore, q(x) has at
most n complex roots, and in particular, it has at most n real roots. Thus, there
exists a real number c such that q(c) ̸= 0.37 But then det(R + cQ) = q(c) ̸= 0, and so
the Invertible Matrix Theorem (see subsection 8.2.6) guarantees R + cQ is invertible.
This completes the argument.

Remark: In view of Theorem 8.6.7, we can rely on the Jordan normal form to check
whether two n × n matrices with real entries are similar over R, even though the
field R is not algebraically closed. Indeed, for n × n matrices A and B with real
entries, Corollary 8.6.3 and Theorem 8.6.7 together guarantee that the following are
equivalent:

ˆ A and B are similar over R;

ˆ A and B are similar over C;

ˆ A and B have the same Jordan normal form in Cn×n (up to a reordering of
the Jordan blocks).

If we manage to factor the characteristic polynomials of A and B, then we can

compute the Jordan normal forms of A and B (seen as matrices in Cn×n ) using
Theorem 8.6.6. Importantly, the Jordan normal form of a square matrix with real
entries may have entries that are not real.

8.6.2 Computing the Jordan normal form of a square matrix

In this subsection, we show how Theorem 8.6.6 can be used to compute the Jordan
normal form of a matrix A in Cn×n . We give two fully worked out examples
(Examples 8.6.8 and 8.6.9 below). In order to get a full picture of how Theorem 8.6.6
is used in the general case, we need our matrix A to be large enough. So, in our first
example, we find the Jordan normal form of a 10 × 10 matrix, and in our second
example, we do this for a 13 × 13 matrix. We note that the matrix powers and
ranks in these examples are all computed with the help of a calculator (this type of
computation would take a very long time if we were to do everything by hand).

37
We can even choose c to be one of 0, 1, . . . , n. Indeed, since q(x) has at most n complex roots,
we know that at least one of 0, 1, . . . , n is not a root of q(x). We can choose c to be this non-root.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 571

Example 8.6.8. Consider the following matrix in C10×10 :

3 1 0 0 0 0 0 −1 0 1
 
 −3 1 5 2 −2 −4 −7 4 −1 3 
 
 0
 1 3 0 0 0 0 −1 0 1 

 −2 −1 3 4 −1 −2 −3 2 −1 2 
 
 −1 0 2 1 2 −2 −1 1 0 1 
A :=   .
 −1 0 1 0 0 2 −1 0 0 1 

 1
 1 −2 −1 1 2 7 −2 1 −2 

 −1 0 1 0 0 −1 0 3 1 0 
 
 0 0 0 0 0 0 1 0 3 −1 
1 1 −2 −1 1 2 5 −2 1 0

Using Theorem 8.6.6, compute the Jordan normal form of A.

Solution. First of all, we compute the characteristic polynomial of A, and we factor

it into linear terms:

pA (λ) = det λI10 − A = (λ − 3)8 (λ − 2)2 .

We see that the eigenvalues of A are λ1 = 3 (with algebraic multiplicity 8) and

λ2 = 2 (with algebraic multiplicity 2). So, all of our Jordan blocks will be of the form
Jt (3) and Jt (2) for various positive integers t. We now deal with the two eigenvalues
separately, as follows.
We first deal with the eigenvalue λ1 = 3. We compute the matrices (A − λ1 I10 )r
for r = 0, 1, 2, 3, . . . along with their ranks. We keep computing until we get the
same rank twice in a row. We obtain:
 
ˆ rank (A − λ1 I10 )0 = 10;38
 
ˆ rank (A − λ1 I10 )1 = 7;
 
ˆ rank (A − λ1 I10 )2 = 4;
 
ˆ rank (A − λ1 I10 )3 = 2;
 
ˆ rank (A − λ1 I10 )4 = 2.

We have now obtained the same rank twice in a row, and so we can stop. We
compute:
   
ˆ rank (A − λ1 I10 )0 − rank (A − λ1 I10 )1 = 3;
38
By definition, we have that (A − λ1 I10 )0 = I10 , and obviously, rank(I10 ) = 10.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 572

   
ˆ rank (A − λ1 I10 )1 − rank (A − λ1 I10 )2 = 3;
   
ˆ rank (A − λ1 I10 ) − rank (A − λ1 I10 ) = 2;
2 3

   
ˆ rank (A − λ1 I10 )3 − rank (A − λ1 I10 )4 = 0.

By Theorem 8.6.6, the Jordan normal form of A will contain:

ˆ three Jordan blocks Jt (λ1 ) = Jt (3) with t ≥ 1;

ˆ three Jordan blocks Jt (λ1 ) = Jt (3) with t ≥ 2;

ˆ two Jordan blocks Jt (λ1 ) = Jt (3) with t ≥ 3;

ˆ zero Jordan blocks Jt (λ1 ) = Jt (3) with t ≥ 4.

Keeping in mind that for any positive integer r, the number of Jordan blocks
Jr (λ1 ) = Jr (3) in the Jordan normal form of A is equal to
   
number of Jordan blocks number of Jordan blocks
− ,
Jt (λ1 ) satisfying t ≥ r Jt (λ1 ) satisfying t ≥ r + 1

we conclude that the Jordan normal form of A will contain exactly two Jordan blocks
J3 (λ1 ) = J3 (3),39 and exactly one Jordan block J2 (λ1 ) = J2 (3). The Jordan normal
form of A contains no other Jordan blocks of the form Jt (λ1 ) = Jt (3).
It remains to deal with the eigenvalue λ2 = 2. We compute the matrices
(A − λ2 I10 )r for r = 0, 1, 2, 3, . . . along with their ranks. We keep computing until
we get the same rank twice in a row. We obtain:
 
ˆ rank (A − λ2 I10 )0 = 10;
 
ˆ rank (A − λ2 I10 )1 = 9;
 
ˆ rank (A − λ2 I10 )2 = 8;
 
ˆ rank (A − λ2 I10 )3 = 8.

We have now obtained the same rank twice in a row, and so we can stop. We
compute:
   
ˆ rank (A − λ2 I10 )0 − rank (A − λ2 I10 )1 = 1;
39
Indeed, it contains two Jordan blocks Jt (3) with t ≥ 3, but zero Jordan blocks Jt (3) with t ≥ 4.
So, the number of Jordan blocks Jt (3) with t = 3 is 2 − 0 = 2.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 573

   
ˆ rank (A − λ2 I10 )1 − rank (A − λ2 I10 )2 = 1;
   
ˆ rank (A − λ2 I10 ) − rank (A − λ2 I10 ) = 0.
2 3

By Theorem 8.6.6, the Jordan normal form of A will contain:

ˆ one Jordan block Jt (λ2 ) = Jt (2) with t ≥ 1;

ˆ one Jordan block Jt (λ2 ) = Jt (2) with t ≥ 2;

ˆ zero Jordan blocks Jt (λ2 ) = Jt (2) with t ≥ 3.

Consequently, the Jordan normal form of A will contain exactly one Jordan block
J2 (λ2 ) = J2 (2), and it will contain no other Jordan blocks of the form Jt (λ2 ) = Jt (2).
Putting everything together, we get that the Jordan normal form of A is the
following (color coded for ease of reading):

J := J3 (λ1 ) ⊕ J3 (λ1 ) ⊕ J2 (λ1 ) ⊕ J2 (λ2 )

= J3 (3) ⊕ J3 (3) ⊕ J2 (3) ⊕ J2 (2)

3 1 0 0 0 0 0 0 0 0
 

 0 3 1 0 0 0 0 0 0 0 


 0 0 3 0 0 0 0 0 0 0 


 0 0 0 3 1 0 0 0 0 0 

 0 0 0 0 3 1 0 0 0 0 
=  .

 0 0 0 0 0 3 0 0 0 0 


 0 0 0 0 0 0 3 1 0 0 


 0 0 0 0 0 0 0 3 0 0 

 0 0 0 0 0 0 0 0 2 1 
0 0 0 0 0 0 0 0 0 2

We remark that we could have written our Jordan blocks in a different order, but in
any case, the Jordan blocks would have to be the same as above (counting repetitions).
For instance, J2 (3) ⊕ J3 (3) ⊕ J2 (2) ⊕ J3 (3) is also a Jordan normal form of A.

Remark: It is acceptable to leave J3 (3) ⊕ J3 (3) ⊕ J2 (3) ⊕ J2 (2) (color coded or not)
as a final answer, without exhibiting the actual 10 × 10 matrix with its 100 entries.
It is not acceptable to leave J3 (λ1 ) ⊕ J3 (λ1 ) ⊕ J2 (λ1 ) ⊕ J2 (λ2 ) as a final answer.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 574

Example 8.6.9. Consider the following matrix in C13×13 :

 
4 0 0 3 0 −1 2 0 0 0 0 2 0
 1
 5 1 −3 0 1 −2 1 1 0 1 −2 0 

 −1 −2 3 4 0 −6 3 1 −2 2 −3 4 −2 
 
 0
 −3 0 10 −1 −2 4 0 −2 −1 0 4 0 

 0
 0 0 −1 4 3 −1 0 0 0 1 −1 2 

 0 0 0 0 0 4 0 0 0 0 0 0 0 
 
A :=  0 3 0 −12 1 2 −4 0 2 1 −2 −6 0 .
 
 1 1 1 −1 0 6 −1 3 1 −2 3 −2 2 
 
 0 −1 0 3 0 −1 2 0 3 0 0 2 0 
 
 −1 −1 −1 0 −4 3 −1 −1 4 −2 3 −2 
 
4
 
 0
 1 0 −3 0 1 −2 0 1 0 4 −2 0 

 0 2 0 3 1 1 2 0 1 1 2 4 0 
0 0 0 0 0 −2 0 0 0 0 0 0 2

Using Theorem 8.6.6, compute the Jordan normal form of A.

Solution. We proceed similarly as in Example 8.6.8. First, we compute the charac-

teristic polynomial of A, and we factor it into linear terms:

pA (λ) = det λI13 − A = (λ − 4)10 (λ − 2)3 .

We see that the eigenvalues of A are λ1 = 4 (with algebraic multiplicity 10) and
λ2 = 2 (with algebraic multiplicity 3). So, all of our Jordan blocks will be of the form
Jt (4) and Jt (2) for various positive integers t. We now deal with the two eigenvalues
separately, as follows.
We first deal with the eigenvalue λ1 = 4. We compute the matrices (A − λ1 I13 )r
for r = 0, 1, 2, 3, . . . along with their ranks. We keep computing until we get the
same rank twice in a row. We obtain:
 
ˆ rank (A − λ1 I13 )0 = 13;
 
ˆ rank (A − λ1 I13 )1 = 9;
 
ˆ rank (A − λ1 I13 )2 = 6;
 
ˆ rank (A − λ1 I13 )3 = 5;
 
ˆ rank (A − λ1 I13 )4 = 4;
 
ˆ rank (A − λ1 I13 )5 = 3;

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 575

 
ˆ rank (A − λ1 I13 )6 = 3.

We have now obtained the same rank twice in a row, and so we can stop. We
compute:
   
ˆ rank (A − λ1 I13 )0 − rank (A − λ1 I13 )1 = 4;
   
ˆ rank (A − λ1 I13 )1 − rank (A − λ1 I13 )2 = 3;
   
ˆ rank (A − λ1 I13 )2 − rank (A − λ1 I13 )3 = 1;
   
ˆ rank (A − λ1 I13 )3 − rank (A − λ1 I13 )4 = 1;
   
ˆ rank (A − λ1 I13 )4 − rank (A − λ1 I13 )5 = 1;
   
ˆ rank (A − λ1 I13 )5 − rank (A − λ1 I13 )6 = 0.

By Theorem 8.6.6, the Jordan normal form of A will contain:

ˆ four Jordan blocks Jt (λ1 ) = Jt (4) with t ≥ 1;

ˆ three Jordan blocks Jt (λ1 ) = Jt (4) with t ≥ 2;

ˆ one Jordan block Jt (λ1 ) = Jt (4) with t ≥ 3;

ˆ one Jordan block Jt (λ1 ) = Jt (4) with t ≥ 4;

ˆ one Jordan block Jt (λ1 ) = Jt (4) with t ≥ 5;

ˆ zero Jordan blocks Jt (λ1 ) = Jt (4) with t ≥ 6.

Keeping in mind that for any positive integer r, the number of Jordan blocks
Jr (λ1 ) = Jr (4) in the Jordan normal form of A is equal to
   
number of Jordan blocks number of Jordan blocks
− ,
Jt (λ1 ) satisfying t ≥ r Jt (λ1 ) satisfying t ≥ r + 1

we conclude that the Jordan normal form of A will contain exactly one Jordan
block J5 (λ1 ) = J5 (4), two Jordan blocks J2 (λ1 ) = J2 (4), and one Jordan block
J1 (λ1 ) = J1 (4). The Jordan normal form of A contains no other Jordan blocks of
the form Jt (λ1 ) = Jt (4).
It remains to deal with the eigenvalue λ2 = 2. We compute the matrices
(A − λ2 I13 )r for r = 0, 1, 2, 3, . . . along with their ranks. We keep computing until
we get the same rank twice in a row. We obtain:

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 576

 
ˆ rank (A − λ2 I13 )0 = 13;
 
ˆ rank (A − λ2 I13 ) = 11;
1

 
ˆ rank (A − λ2 I13 )2 = 10;
 
ˆ rank (A − λ2 I13 )3 = 10.

We have now obtained the same rank twice in a row, and so we can stop. We
compute:
   
ˆ rank (A − λ2 I13 )0 − rank (A − λ2 I13 )1 = 2;
   
ˆ rank (A − λ2 I13 )1 − rank (A − λ2 I13 )2 = 1;
   
ˆ rank (A − λ2 I13 )2 − rank (A − λ2 I13 )3 = 0.

By Theorem 8.6.6, the Jordan normal form of A will contain:

ˆ two Jordan blocks Jt (λ2 ) = Jt (2) with t ≥ 1;

ˆ one Jordan block Jt (λ2 ) = Jt (2) with t ≥ 2;

ˆ zero Jordan block Jt (λ2 ) = Jt (2) with t ≥ 3.

Consequently, the Jordan normal form of A will contain exactly one Jordan block
J2 (λ2 ) = J2 (2), one Jordan block J1 (λ2 ) = J1 (2), and no other Jordan blocks of the
form Jt (λ2 ) = Jt (2).
Putting everything together, we get that the Jordan normal form of A is the
following (color coded for ease of reading):

J := J5 (λ1 ) ⊕ J2 (λ1 ) ⊕ J2 (λ1 ) ⊕ J1 (λ1 ) ⊕ J2 (λ2 ) ⊕ J1 (λ2 )

= J5 (4) ⊕ J2 (4) ⊕ J2 (4) ⊕ J1 (4) ⊕ J2 (2) ⊕ J1 (2)

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 577

 
4 1 0 0 0 0 0 0 0 0 0 0 0

 0 4 1 0 0 0 0 0 0 0 0 0 0 


 0 0 4 1 0 0 0 0 0 0 0 0 0 


 0 0 0 4 1 0 0 0 0 0 0 0 0 


 0 0 0 0 4 0 0 0 0 0 0 0 0 

0 0 0 0 0 4 1 0 0 0 0 0 0
 
 
= 0 0 0 0 0 0 4 0 0 0 0 0 0 .
 

0 0 0 0 0 0 0 4 1 0 0 0 0
 
 
0 0 0 0 0 0 0 0 4 0 0 0 0
 
 
 
 0 0 0 0 0 0 0 0 0 4 0 0 0 
 

 0 0 0 0 0 0 0 0 0 0 2 1 0 

 0 0 0 0 0 0 0 0 0 0 0 2 0 
0 0 0 0 0 0 0 0 0 0 0 0 2

As in Example 8.6.8, we could have written our Jordan blocks in a different order, as
long as we preserved any repetitions. For example, J1 (2) ⊕ J2 (4) ⊕ J1 (4) ⊕ J2 (2) ⊕
J5 (4) ⊕ J2 (4) is also a Jordan normal form of A.

Remark: Suppose we are given a matrix A ∈ Fn×n , where F is an algebraically closed

field. We saw above how we can compute the Jordan normal form of A, that is, how
we can find a Jordan matrix J ∈ Fn×n that is similar to A. Could we also compute
an invertible matrix P ∈ Fn×n for which J = P −1 AP ? This is indeed possible, but
it is significantly more complicated than just computing a suitable Jordan matrix J.
Unfortunately, any example that illustrates the procedure in full generality (more
or less) requires a great deal of long and laborious computation.40 For sufficiently
brave readers, a recipe and a couple of examples are given in subsection 8.6.7.

8.6.3 A brief outline of the proof of Theorems 8.6.2 and 8.6.4

As we pointed out in subsection 8.6.1, Theorems 8.6.2 and 8.6.4 are equivalent, i.e.
in view of Theorem 4.5.19, either one of these two theorems easily implies the other.
More precisely, the existence parts of Theorems 8.6.2 and 8.6.4 are equivalent, as
are the uniqueness parts of the two theorems. In what follows, we will outline the
proof of the existence part of Theorem 8.6.4, and the proof of the uniqueness part
of Theorem 8.6.2. (In passing, we will also say a few words about the proof of
Theorem 8.6.6.) The uniqueness part is easier, and so we outline that first.

Uniqueness. Let us outline the proof of the uniqueness part of Theorem 8.6.2,
restated below for the reader’s convenience.
40
It really is quite long and quite laborious, even if we use a calculator throughout.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 578

Theorem 8.6.2. Assume that F is an algebraically closed field, and let A ∈ Fn×n
be a square matrix. Then A is similar to a matrix J in Jordan normal form. Moreover,
this matrix J is unique up to a reordering of the Jordan blocks.

So, let us suppose that F is a field,41 and assume that a matrix A ∈ Fn×n is
similar to a Jordan matrix J ∈ Fn×n . Clearly, it suffices to show that the types
of Jordan blocks that appear in J are fully determined by A, as is the number of
Jordan blocks of each type. (For a formal statement of what we are trying to prove,
see the somewhat lengthy statement of Proposition 8.6.17.)
First of all, since A and J are similar, Theorem 8.2.9 guarantees that these two
matrices have the same characteristic polynomial and the same spectrum. Since the
Jordan matrix J is upper triangular, it has precisely its spectrum (equivalently: the
spectrum of A) on the main diagonal. In particular, all Jordan blocks of J are of
the form Jt (λ), where λ is an eigenvalue of A, and t is a positive integer no greater
than the algebraic multiplicity of λ as an eigenvalue of A. For each eigenvalue λ and
positive integer r, we would like to compute the number of Jordan blocks Jt (λ) that
appear in J and satisfy t ≥ r. If we can show that this number depends only on A
(and not on the particular choice of J), then we are done, since for fixed λ and r,
the number of Jordan blocks Jr (λ) in J is equal to
   
number of Jordan blocks number of Jordan blocks
− .
Jt (λ) satisfying t ≥ r Jt (λ) satisfying t ≥ r + 1

We now fix an eigenvalue λ (say, of algebraic multiplicity m) of A, and we consider

the matrix J − λIn ; this matrix is a Jordan matrix (because J is), and moreover,
each Jordan block Jt (λ′ ) of J corresponds to a Jordan block Jt (λ′ − λ) of J − λIn in
the obvious way. In particular, blocks Jt (λ) of J correspond to the blocks Jt (0) of
J − λIn in the natural way. So, we just need to count the number of blocks Jt (0) in
J − λIn satisfying t ≥ r, for all possible values of r. Here, it will be important to
note that the sizes of the Jordan blocks Jt (0) in J − λIn sum up to m, whereas the
sizes of the remaining Jordan
blocks of J − λIn sum up to n − m. Now, the idea is
to compute rank (J − λIn )r for r = 0, 1, 2, 3, . . . .
We first make a couple of key observations. The first observation is that if
A1 , . . . , Ak are square matrices, then

ˆ rank(A1 ⊕ · · · ⊕ Ak ) = rank(A1 ) + · · · + rank(Ak ), and

ˆ (A1 ⊕ · · · ⊕ Ak )r = Ar1 ⊕ · · · ⊕ Ark for all non-negative integers r.

So, the r-th power of a Jordan matrix is equal to the direct sum of the r-th powers
of its Jordan blocks, and the rank of the r-th power of a Jordan matrix is equal to
the sum of ranks of the r-th powers of its Jordan blocks.
41
For the uniqueness part, we do not need F to be algebraically closed. Algebraic closure matters
only for the existence part.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 579

The second observation is that for any matrix A with t columns and entries in F,
the matrix AJt (0) is obtained from A by first adding a zero column to the left, and
then deleting the rightmost column of the resulting matrix; it then follows by an
easy induction on r that


 It if r = 0





r   O It−r

(t−r)×r
Jt (0) = if 1 ≤ r ≤ t − 1

 Or×r Or×(t−r)




if r ≥ t

Ot×t
for all positive
 integers t and r (see Proposition 8.6.13). From here, we can easily

r  42
read off rank Jt (0) . Meanwhile, the remaining Jordan blocks of J − λIn (i.e.
those that have a number other than 0 on the main diagonal) are invertible matrices
(because their determinant is non-zero);43 therefore, all powers of these matrices
are invertible and have full rank.44 So, for any positive integer r, the r-th powers
of the Jordan blocks of J − λIn that have a non-zero on the main diagonal end up
contributing n − m to the rank of (J − λIn )r . Meanwhile, the contribution that the
r-th power of a Jordan block Jt (0) of J − λIn
makes to the rank of (J − λIn )r can
r
easily be read off from our formula for Jt (0) above.
Thus, we can in fact obtain a formula for rank (J − λIn )r that depends on the

number of Jordan blocks Jt (0) for various values of t. With a little bit of computation,
it can be shown that, for each positive integer r, the number of Jordan blocks Jt (0)
satisfying t ≥ r in the Jordan matrix J − λIn is precisely

rank (J − λIn )r−1 − rank (J − λIn )r .

As discussed above, this is precisely the number of Jordan blocks Jt (λ) satisfying
t ≥ r in J. But since A and J are similar, so are A − λIn and A − λIn , and by
Proposition 4.5.18, so are all of their corresponding powers. By Corollary 4.5.20,
similar matrices have the same rank. So, the number of Jordan blocks Jt (λ) satisfying
t ≥ r in J is in fact

rank (A − λIn )r−1 − rank (A − λIn )r .

This essentially completes the proof of the uniqueness part of Theorems 8.6.2. For
the full details, see subsection 8.6.4.

r  
r 
42
Indeed, we get that rank Jt (0) = t − r if r ≤ t − 1, and that rank Jt (0) = 0 if r ≥ t.
43
Any Jordan block is an upper triangular matrix, and so (by Proposition 7.3.1) its determinant
can be computed by multiplying the entries on the main diagonal. If these entries are non-zero,
then the determinant is non-zero. By the Invertible Matrix Theorem (see subsection 8.2.6), any
square matrix whose determinant is non-zero is invertible.
44
By Proposition 1.11.8(f), all powers of an invertible matrix are invertible, and by the Invertible
Matrix Theorem (see subsection 8.2.6), invertible matrices have full rank.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 580

Remark: The formula that we obtained above is precisely the same as the one from
Theorem 8.6.6. However, the argument above (fortified with all the technical details)
does not actually prove Theorem 8.6.6. It only proves that if a square matrix A
is similar to a Jordan matrix J, then the Jordan blocks of J are as specified in
Theorem 8.6.6. To prove that a matrix A ∈ Fn×n (where F is an algebraically closed
field) is indeed similar to at least one Jordan matrix,45 we need the existence part of
Theorem 8.6.2, to which we now turn.

Existence. We now give a brief outline of the proof of the existence part of
Theorem 8.6.4, restated below for the reader’s convenience.

Theorem 8.6.4. Let V be a non-trivial, finite-dimensional vector space over an

algebraically closed field F, and let f : V → V be a linear function. Then there
 
exists a basis B such that the matrix B f B is in Jordan normal form. Moreover,
this matrix
  the following sense: if B1 and B2 are bases of V such that
is unique in
both B f B and B f B are in Jordan normal form, then these two matrices
1 1 2 2
are the same up to a reordering of the Jordan blocks.

First, we need a couple of definitions. For a function f : A → A (where A is

some set), we set f 0 := IdA (the identity function on A), and for all non-negative
integers k, we set f k+1 := f k ◦ f . So, for all positive integers k, we have that

f k := f ◦ · · · ◦ f .
| {z }
k

The function f k is called the k-th iterate of f .

Next, for a linear function f : V → V (where V is a vector space over a field F),
a subspace U of V is said to be f -invariant if f [U ] ⊆ U , i.e. if for all u ∈ U , we have
that f (u) ∈ U . Under these circumstances, it is possible to restrict both the domain
and the codomain of f to U . More precisely, we can define f |U : U → U by setting
f |U (u) = f (u) for all u ∈ U ; obviously, f |U is linear (since f is).
Here is an outline of the proof of the existence part of Theorem 8.6.4. Let
f : V → V be a linear function, where V is a non-trivial, finite-dimensional vector
space over an algebraically closed field F, as in the statement of Theorem 8.6.4. Let

(λ1 , . . . , λ1 , . . . , λk , . . . , λk )
| {z } | {z }
m1 mk

be the spectrum of f , where λ1 , . . . , λk ∈ F are pairwise distinct, and where

m1 , . . . , mk are positive integers. Since F is algebraically closed, we know that
m1 + · · · + mk = dim(V ) =: n.
45
Here, it is important that F is algebraically closed: the statement becomes false otherwise.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 581

By Theorem 8.2.13, for any

eigenvalue λ of f , the eigenspace of f associated with
λ is Eλ (f ) = Ker f − λIdV . We can generalize this as follows. For each eigenvalue
λ of f , we define the generalized eigenspace of f associated with λ to be the set

Gλ (f ) := v ∈ V | ∃r ∈ N0 s.t. (f − λIdV )r (v) = 0



∞
S 
r 
= Ker f − λIdV .
r=0

It is easy to show that Gλ (f ) is a subspace of V (we simply check that it satisfies the
three conditions from Theorem 3.1.7), and obviously, eigenspace Eλ (f ) = Ker(f −λIn )
is a subspace of the generalized eigenspace of Gλ (f ). Now, in the first part of our
proof, we show that all generalized eigenspaces of f are f -invariant subspaces of V ,
and moreover, that V is the direct sum of the generalized eigenspaces of f , that is,

V = Gλ1 (f ) ⊕ · · · ⊕ Gλk (f ).

We now consider the restrictions f1 , . . . , fk of our linear function f to the generalized

eigenspaces Gλ1 (f ), . . . , Gλk (f ), respectively.46 It can be shown that for each index
i ∈ {1, . . . , k}, fi has exactly one eigenvalue, namely λi , and the algebraic multiplicity
of this eigenvalue is mi . Further, it is easy to show that for any bases B1 , . . . , Bk
of Gλ1 (f ), . . . , Gλk (f ), respectively, we have that B := B1 ∪ · · · ∪ Bk is a basis of V ,
and moreover, that
     
B
f B = B f1 B ⊕ · · · ⊕ B fk B .
1 1 k k
   
If we can choose our bases B1 , . . . , Bk so that the matrices B f1 B , . . . , B fk B
  1 1 k k
are Jordan matrices, then the matrix B f B will also be a Jordan matrix. So, we
may handle the restrictions f1 , . . . , fk of f separately. Since each of these restrictions
has only one eigenvalue, this effectively reduces the problem to the case when the
linear function f : V → V has only one eigenvalue.
Now, note that if λ is the only eigenvalue of f : V → V , then 0 is the only
eigenvalue of f − λIdV , and moreover, for any basis B of V , we have that
   
B
f B
= B
f − λIdV B
+ λIn .
   
Thus, B f B is a Jordan matrix if and only if B f − λIdV B is a Jordan matrix.
So, we may consider f − λIdV instead of f . In other words, we have reduced our
problem to the case when the only eigenvalue of f is 0.
From now on, we deal only with the case when f has exactly one eigenvalue,
namely 0. Because F is algebraically closed, this eigenvalue has algebraic multiplicity
n = dim(V ), i.e. pf (λ) = λn . It can then be shown that f is “nilpotent,” that is,
46
In other words, for each i ∈ {1, . . . , k}, we set fi := f |Gλi (f ) , which is well defined because
Gλi (f ) is f -invariant.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 582

that some iterate f p of f is the zero function,47 or equivalently, that Ker(f p ) = V .

Thus, we have reduced the problem of proving the existence part of Theorem 8.6.4
to the case when f is nilpotent.  
The goal is now to construct a basis B of V such that the matrix B f B is in
Jordan normal form. Our basis will be the union of suitably chosen “Jordan chains”
of the form {f r−1 (u), f r−2 (u), . . . , f (u), u}, where u ∈ Ker(f r ) \ Ker(f r−1 ), so that
f r (u) = 0, but f r−1 (u) ̸= 0. We will say that such a Jordan chain is started by u.48
So, our basis B will be of the form
n o
B = f a1 −1 (u1 ), . . . , f (u1 ), u1 , . . . , f aℓ −1 (uℓ ), . . . , f (uℓ ), uℓ

for some vectors u1 , . . . , uℓ ∈ V and positive integers a1 , . . . , aℓ such that for all
indices i ∈ {1, . . . , ℓ}, we have that f ai (ui ) = 0 and f ai −1 (ui ) ̸= 0. These Jordan
chains need to be chosen with care, so that they together produce a basis. We will say
a few words about this below, but let us first explain why a basis of this type is actually
useful. The point is that each Jordan chain {f ai −1 (ui ), f ai −2 (u), . . . , f(ui ),ui } in
our basis B will correspond to a Jordan block Jai (0) in the matrix B f B . To
gain some intuition for how the Jordan chains behave, let us suppose that B =
{f r−1 (u), f r−2 (u), . . . , f (u), u} is our whole basis, i.e. our basis B consists of only one
Jordan chain; in this particular case, we would in fact have that r = dim(V ) = n, and
so B = {f n−1 (u), f n−2 (u), . . . , f (u), u}. Then, using the formula from Theorem 4.5.1,
we would obtain
f f n−1 (u) B f f n−2 (u) B . . .
   
 
 
   
B
f B = f f (u) B f (u) B

f n (u) f n−1 (u) f 2 (u)

         
= B B
... B
f (u) B

f n−1 (u) f 2 (u)

         
= 0 B B
... B
f (u) B

0 en1 . . . enn−2 enn−1

 
=

= Jn (0).

The computation above works when B consists of only one Jordan chain. If B consists
of multiple Jordan chains, then we simply get multiple Jordan blocks.
We now briefly outline the construction of the basis B discussed above. The
argument proceeds by (strong) induction on dim(V ). Let p be an integer such that
f p is a zero function, i.e. Ker(f p ) = V . The goal is to show that for all vectors
v1 , . . . , vt ∈ V that are “linearly independent over Ker(f p−1 )” (which means that they
47
This essentially follows from the Cayley-Hamilton theorem (stated and proven in section 8.3).
For the details, see Proposition 8.6.26.
48
Granted, this terminology is slightly unfortunate, since the Jordan chain rather “ends” with u.
The point, however, is that once we have chosen u, the Jordan chain is uniquely determined.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 583

are linearly independent in the ordinary sense, and in addition, Span(v1 , . . . , vt ) ∩

Ker(f p−1 ) = {0}), there exists a basis B of V that is the union of some number of
pairwise disjoint Jordan chains, and t of those chains are started by our preselected
vectors v1 , . . . , vt . With a little bit of preprocessing, we may assume that f p−1 is
not a zero function, i.e. Ker(f p−1 ) ⫋ Ker(f p ) = V . We may further assume that
v1 , . . . , vt is a maximal list of linearly independent vectors over Ker(f p−1 ), since
otherwise, we simply extend this list to a maximal one (this is possible because V is
finite-dimensional).
Now, if p = 1, then any basis of V that extends {v1 , . . . , vt } will do.49 Otherwise,
it can be shown that f (v1 ), . . . , f (vt ) are linearly independent over Ker(f p−2 ). Then,
we apply the induction hypothesis to the vector space Ker(f p−1 ), the linear function
f (more precisely: its restriction to Ker(f p−1 )), and the vectors f (v1 ), . . . , f (vt ).
We obtain a basis of Ker(f p−1 ) that is the union of pairwise disjoint Jordan chains,
t of which are started by f (v1 ), . . . , f (vt ). This basis is of the form

{f p−1 (v1 ), f p−2 (v1 ), . . . , f 2 (v1 ), f (v1 )}

..
.
p−1 p−2 (v ), . . . , f 2 (v ), f (v )}
S
S {f at+1(v
−1
t ), f t t t
{f (vt+1 ), f at+1 −2 (vt+1 ), . . . , f 2 (vt+1 ), f (vt+1 ), vt+1 }
..
S . at+s −1
{f (vt+s ), f at+s −2 (vt+s ), . . . , f 2 (vt+s ), f (vt+s ), vt+s },

for some vectors vt+1 , . . . , vt+s ∈ Ker(f p−1 ) and positive integers at+1 , . . . , at+s such
that f at+i (vi ) = 0 and f at+i −1 (vi ) ̸= 0 for all i ∈ {1, . . . , s}. We extend the chains
started by f (v1 ), . . . , f (vt ) by adding to them the vectors v1 , . . . , vt , respectively.
These “extended” chains, plus the remaining (“non-extended”) chains form the basis
B of V that we need:
{f p−1 (v1 ), f p−2 (v1 ), . . . , f 2 (v1 ), f (v1 ), v1 }
 
 .. 
 . 
p−1 p−2 2
 S 
 {f (vt ), f (vt ), . . . , f (vt ), f (vt ), vt } 
B :=   S a −1 a −2 2
.
 {f t+1 (vt+1 ), f t+1 (vt+1 ), . . . , f (vt+1 ), f (vt+1 ), vt+1 }  
 .. 
S . at+s −1
 
{f (vt+s ), f at+s −2 (vt+s ), . . . , f 2 (vt+s ), f (vt+s ), vt+s }

Remark: This completes our outline of the proofs of Theorems 8.6.2 and 8.6.4. The
full technical details are given in subsections 8.6.4, 8.6.5, and 8.6.6.
49
This is because, if p = 1, then every non-zero vector u starts the one-element Jordan chain {u}.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 584

8.6.4 Invariant subspaces and the uniqueness of the Jordan normal

form: the proof of Theorem 8.6.1
In this subsection, we prove Theorem 8.6.1. We note that this in fact proves the
uniqueness (but not the existence) part of Theorems 8.6.2 and 8.6.4. We further
note that no result of this subsection requires that the field that we are working over
be algebraically closed. Algebraic closure will become relevant in subsection 8.6.5,
which deals with the existence part of Theorems 8.6.2 and 8.6.4.
The proof of Theorem 8.6.1 has two parts. First, we prove that any if two matrices
are the direct sums of the same matrices, but possibly arranged in a different order,
then those two matrices are in fact similar (see Proposition 8.6.12); this immediately
implies that two Jordan matrices that have exactly the same Jordan blocks (counting
repetitions) are similar. Then, we show that if a square matrix A is similar to a
Jordan matrix J, then the types of Jordan blocks appearing in J depend only on A,
as does the number of Jordan blocks of each possible type (see Proposition 8.6.17).50
Thus, any two Jordan matrices similar to a given square matrix A have exactly the
same number of Jordan blocks (counting repetitions, but not counting the order
in which they appear). Since matrix similarity is an equivalence relation, this, in
particular, implies that any two similar Jordan matrices have exactly the same
Jordan blocks (counting repetitions, but not counting the order in which the Jordan
blocks appear in the two matrices).
Before turning to the proofs, we note that the formula for the number of Jordan
blocks of each type from Proposition 8.6.17 is precisely the one given in Theorem 8.6.6
(stated in subsection 8.6.1). Thus, it may seem that Proposition 8.6.17 immediately
implies Theorem 8.6.6. This, however, is not the case. Indeed, Proposition 8.6.17 does
not state that any matrix A ∈ Fn×n (where F is a field) is similar to a Jordan matrix
J ∈ Fn×n . For general fields F, the statement is not even true. For algebraically
closed fields F, we do indeed get the existence of such a Jordan matrix J, but this
requires a lot more work (see subsections 8.6.5 and 8.6.6).

Invariant subspaces. Suppose that V is a vector space over a field F and that
f : V → V is a linear function. A subspace U of V is said to be f -invariant (or
invariant for f ) if f [U ] ⊆ U , that is, if for all u ∈ U , we have that f (u) ∈ U . Under
these circumstances (i.e. if U is an f -invariant subspace of V ), we may define the
function f |U : U → U given by f |U (u) = f (u) for all u ∈ U ; thus, f |U is obtained by
restricting both the domain and the codomain of f to U (which we can do because
U is f -invariant), and obviously, f |U is linear (because f is linear).
Recall from subsection 3.2.6 that if V is a vector space over a field F and
U1 and U2 are its subspaces such that U1 ∩ U2 = {0} and V = U1 + U2 , then
we say that V is the direct sum of U1 and U2 , and we write V = U1 ⊕ U2 . If
50
We note that Proposition 8.6.17 immediately implies the uniqueness part of Theorem 8.6.2, as
outlined in subsection 8.6.3.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 585

V = U1 ⊕ U2 is also finite-dimensional, then Theorem 3.2.23 immediately implies

that dim(V ) = dim(U1 ) + dim(U2 ).51
Obviously, the direct sum of subspaces can be generalized to more than two
subspaces. Suppose that V is a vector space over a field F, and suppose that
U1 , . . . , Uk (k ≥ 1) are its subspaces such that Ui ∩ Uj = {0} for all distinct
i, j ∈ {1, . . . , k} and such that V = U1 + · · · + Uk .52 Under these circumstances, we
say that V is the direct sum of U1 , . . . , Uk , and we write V = U1 ⊕ · · · ⊕ Uk .

Proposition 8.6.10. Let V be a finite-dimensional vector space over a field F,

let B be a basis of V , and let (B1 , . . . , Bk ) be a partition of B, i.e. assume that
B1 , . . . , Bk are pairwise disjoint and that B = B1 ∪ · · · ∪ Bk .53 For all i ∈ {1, . . . , k},
set Ui := Span(Bi ), so that Bi is a basis of Ui .54 Then V = U1 ⊕ · · · ⊕ Uk .

Proof. This essentially follows from the appropriate definitions. The details are left
as a straightforward exercise.

Proposition 8.6.11. Let V be a non-trivial, finite-dimensional vector space over

a field F, and let f : V → V be a linear function. Assume that V = U1 ⊕ · · · ⊕ Uk
(k ≥ 1), where U1 , . . . , Uk are non-trivial, f -invariant subspaces of V . To simplify
notation, we set fi := f |Ui for all i ∈ {1, . . . , k}. Then for any bases B1 , . . . , Bk
of U1 , . . . , Uk , respectively, we have that B := B1 ∪ · · · ∪ Bk is a basis of V , and
moreover,      
B
f B
= B
f1 B
⊕ · · · ⊕ B
fk B
.
1 1 k k

Consequently,
pf (λ) = pf1 (λ) . . . pfk (λ),
i.e. the characteristic polynomial of f is equal to the product of the characteristic
polynomials of f1 , . . . , fk .

Proof. For each index i ∈ {1, . . . , k}, fix a basis Bi = {bi,1 , . . . , bi,mi } of Ui . Set
B := B1 ∪ · · · ∪ Bk = {b1,1 , . . . , b1,m1 , . . . , bk,1 , . . . , bk,mk }.

Claim 1. B is a basis of V .

Proof of Claim 1. Since V is finite-dimensional and satisfies V = U1 ⊕ · · · ⊕ Uk , we

know that dim(V ) = dim(U1 ) + · · · + dim(Uk ) = m1 + · · · + mk .55 Since B contains
51
This is because dim(U1 ∩ U2 ) = 0.
52
Here, U1 + · · · + Uk is defined in the obvious way:

U1 + · · · + Uk := {u1 + · · · + uk | u1 ∈ U1 , . . . , uk ∈ Uk }.
53
Here, we allow B1 , . . . , Bk to possibly be empty, although in practice, we will apply the proposition
only to the case when they are all non-empty.
54
The fact that Bi is linearly independent follows from the fact that B is linearly independent
(because it is a basis of V ), and by construction, Bi is a spanning set of Ui .
55
This readily follows from Theorem 3.2.23 via an easy induction on k.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 586

precisely m1 + · · · + mk many vectors, it now suffices to show that B is a spanning

set of V , for Theorem 3.2.20(b) will then immediately imply that B is a basis of V .
Fix any v ∈ V . Since V = U1 ⊕ · · · ⊕ U2 , there exist vectors u1 ∈ U1 , . . . , uk ∈ Uk
such that v = u1 + · · · + uk . For each i ∈ {1, . . . , k}, we fix scalars αi,1 , . . . , αi,mi ∈ F
such that
ui = αi,1 bi,1 + · · · + αi,mi bi,mi ;
the existence of such scalars αi,1 , . . . , αi,mi follows from the fact that ui ∈ Ui and
the fact that Bi = {bi,1 , . . . , bi,mi } is a basis of Ui . But now
k
P k
P
v = ui = (αi,1 bi,1 + · · · + αi,mi bi,mi ),
i=1 i=1

and we see that v is a linear combination of the vectors in B. We now conclude that
B is indeed a spanning set of V , and is therefore (as we discussed above) a basis of
V. ♦

Claim 2. For all i ∈ {1, . . . , k} and j ∈ {1, . . . , mi }, we have that

0
 
..
.
 
 
 
 0  
    
f (bi,j )  fi (bi,j ) Bi
=  ,
B 

 0 

 .. 
 . 
0

with exactly m1 + · · · + mi−1 many 0’s in the top block,56 and exactly
mi+1 + · · · + mk many 0’s in the bottom block.57

Proof of Claim 2. Fix indices i ∈ {1, . . . , k} and j ∈ {1, . . . , mi }. Since Ui is f -

invariant, we have that f (bi,j ) ∈ Ui . Therefore, there exist scalars αi,1 , . . . , αi,mi
such that
Pmi
fi (bi,j ) = f (bi,j ) = αi,ℓ bi,ℓ .
ℓ=1
and consequently,
   T
fi (bi,j ) Bi
= αi,1 . . . αi,mi

and
   T
f (bi,j ) B
= 0 . . . 0 αi,1 . . . αi,mi 0 ... 0 ,
56
 
Meaning: above the block f (bi,j ) B .
i
57
 
Meaning: below the block f (bi,j ) B .
i

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 587

with m1 + · · · + mi−1 many 0’s on the left (these 0’s correspond to the coefficients
0 in front of the basis vectors b1,1 , . . . , b1,m1 , . . . , bi−1,1 , . . . , bi−1,mi−1 ), and with
mi+1 + · · · + mk many 0’s on the right (these 0’s correspond to the coefficients 0
in front of the basis vectors bi+1,1 , . . . , bi+1,mi+1 , . . . , bk,1 , . . . , bk,mk ). The result is
now immediate. ♦
     
Claim 3. B
f B
= B1
f1 B1
⊕ ··· ⊕ B fk Bk
.
k

Proof of Claim 3. This follows immediately from Theorem 4.5.1 and from Claim 2. ♦
Now, set n := dim(V ). By Claim 1, we have that  n = m1 + · · · + mk , and
consequently, In = Im1 ⊕ · · · ⊕ Imk . Further, set B = B f B . Then
 
λIn − B = λIn − B f B

(∗)
       
= (λIm1 ) ⊕ · · · ⊕ (λImk ) − B f1 B ⊕ · · · ⊕ B fk B
1 1 k k

       
= λIm1 − B f1 B1
⊕ · · · ⊕ λImk − B fk B
1 k k

where (*) follows from Claim 3. Consequently,

(∗)

pf (λ) = pB (λ) = det λIn − B
!
       
= det λIm1 − B f1 B1
⊕ · · · ⊕ λImk − B fk Bk
1 k

(∗∗) k
Q     (∗) k
Q
= det λImi − B fi B = pfi (λ),
i i
i=1 i=1

where both instances of (*) follow from Proposition 8.2.12, and (**) follows from
Corollary 7.6.7. This completes the argument.

Proposition 8.6.12. Let A1 , . . . , Ak be square matrices with entries in some field

F, and let σ ∈ Sk . Then matrices A := A1 ⊕ · · · ⊕ Ak and Aσ := Aσ(1) ⊕ · · · ⊕ Aσ(k)
are similar.

Remark: Proposition 8.6.12 immediately implies that any two Jordan matrices that
have exactly the same Jordan blocks (counting repetitions) are similar.

Proof. For each index i ∈ {1, . . . , k}, assume that the square matrix Ai is of size
mi ×mi . Set m := m1 +· · ·+mk , so that A is of size m×m, and let Em = {e1 , . . . , em }
be the standard basis of Fm . Let f : Fm → Fm be given
 by f (x) = Ax, so that f linear
and A is the standard matrix of f , i.e. A = E f E . For each index i ∈ {1, . . . , k},
m m

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 588

we set Bi := {em1 +···+mi−1 +1 , . . . , em1 +···+mi−1 +mi },58 and we set Ui := Span(Bi ), so
that Bi is a basis of Ui . It is then easy to see that for each i ∈ {1, . . . , k}, the subspace
Ui of Fm is f -invariant,59 and moreover, that Ai = B fi B , where fi := f |Ui .60
i i
Now, set Em σ := B σ m
σ(1) ∪ · · · ∪ Bσ(k) ; obviously, Em is a basis of F , since it was
obtained by simply permuting the vectors of the standard basis Em of Fm . It then
follows from Proposition 8.6.11 that
     
σ
Em
f σ
Em
= Bσ(1)
fσ(1) Bσ(1)
⊕ · · · ⊕ Bσ(k)
fσ(k) Bσ(k)
= Aσ .
| {z } | {z }
=Aσ(1) =Aσ(k)

   
By Theorem 4.5.19, matrices A = Em
f Em
and Aσ = σ
Em
f σ
Em
are similar. This
completes the argument.

Counting the number of Jordan blocks. Suppose we are are given a matrix
A ∈ Fn×n (where F is a field), and that we know that this matrix is similar to a
Jordan matrix J ∈ Fn×n . In what follows, we would like to determine the types
of Jordan blocks that the Jordan matrix J can have, and also to compute the
number of blocks of each possible type (see Proposition 8.6.17 below); combined
with Proposition 8.6.12, this will allow us to prove Theorem 8.6.1, which states that
two Jordan matrices are similar if and only if they have exactly the same Jordan
blocks (counting repetitions). This, in turn, will allow us to prove uniqueness part
of Theorems 8.6.2 and 8.6.4. Proposition 8.6.17 will also be one of the ingredients
of the proof of Theorem 8.6.6. We begin with four simple technical propositions
(Propositions 8.6.13, 8.6.14, 8.6.15, and 8.6.16).

Proposition 8.6.13. Let F be a field, and let t be a positive integer. Then both the
following hold:

(a) for all matrices A = a1 . . . at in Fs×t , we have that

 

 
AJt (0) = 0 a1 . . . at−1 ,

i.e. AJt (0) is the s × t matrix obtained from A by first adding a zero column to
the left, and then deleting the rightmost column of the resulting matrix;
58
So, the vectors of B1 are the first m1 vectors of Em , the vectors of B2 are the subsequent m2
vectors of Em , and so on.
59
Here are the details. Fix an index i ∈ {1, . . . , k}. Then for all ej ∈ Bi , Aej is the j-th column of
A, and we see from the construction of A that this column is a linear combination of the vectors in
Bi , i.e. f (ej ) = Aej ∈ Span(Bi ) = Ui . Since Ui = Span(Bi ) and since f is linear, it readily follows
that f [Ui ] ⊆ Ui .
60
Details?

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 589

(b) we have that



 It if r = 0





r   O It−r

(t−r)×r
Jt (0) = if 1 ≤ r ≤ t − 1

 O r×r Or×(t−r)




if r ≥ t

Ot×t

for all non-negative integers r.

Proof. Clearly, (b) follows from (a) via aneasy induction
 on r. So, let us prove (a).
Fix a positive integer t and a matrix A = a1 . . . at in Fs×t . By definition, we
have that  
Jt (0) = 0 e1 . . . et−1 ,
where e1 , . . . , et are the standard basis vectors of Ft . We now compute:
 
AJt (0) = A 0 e1 . . . et−1

  by the definition of
= A0 Ae1 . . . Aet−1
matrix multiplication
 
= 0 a1 . . . at−1 by Proposition 1.4.5.

This proves (a), and we are done.

Proposition 8.6.14. Let F be a field. Then



r   t if λ ̸= 0
rank Jt (λ) = t − r if λ = 0 and r ≤ t − 1
0 if λ = 0 and r ≥ t


for all λ ∈ F and all positive integers t and non-negative integers r.

Proof. Fix λ ∈ F. If λ = 0, then the result follows immediately from Proposi-
̸ 0, and fix a positive integer t and a non-negative
tion 8.6.13. So, assume that λ =
integer r. The Jordan block Jt (λ) is an upper triangular t × t
matrix with all λ’s
on the main diagonal, and so by Proposition 7.3.1, det Jt (λ) = λt ̸= 0. So, by
the Invertible Matrix Theorem (see subsection

r 8.2.6), the matrix Jt (λ) is invertible.
Therefore, by Proposition 1.11.8(f), Jt (λ) is also invertible, and consequently (by
the Invertible Matrix Theorem) has rank t.

Proposition 8.6.15. Let A1 , . . . , Ak be square matrices with entries in some field

F. Then for all non-negative integers r, we have that

(A1 ⊕ · · · ⊕ Ak )r = Ar1 ⊕ · · · ⊕ Ark .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 590

Proof. This readily follows from the definition of matrix multiplication.

Proposition 8.6.16. Let A1 , . . . , Ak be square matrices with entries in some field

F. Then
rank(A1 ⊕ · · · ⊕ Ak ) = rank(A1 ) + · · · + rank(Ak ).
Proof. For each index i ∈ {1, . . . , k}, assume that the square matrix Ai is of size
mi × mi . Set A := A1 ⊕ · · · ⊕ Ak and m := m1 + · · · + mk , so that A is of size m × m.
We first perform row reduction on the top m1 many rows of A (while ignoring the
remaining rows) in order to transform the matrix formed by the top m1 many rows
of A into one in row echelon form. Then, we row reduce the matrix formed by the
subsequent m2 many rows of A in order to turn that submatrix of A into one in row
echelon form. We continue like this until we reach the bottom of our matrix A. This
produces a matrix that contains exactly rank(A1 ) + · · · + rank(Ak ) many non-zero
rows. Moreover, after “moving” any zero rows to the bottom of this matrix, we obtain
a matrix in row echelon form that still has precisely rank(A1 ) + · · · + rank(Ak ) many
non-zero rows. This matrix is row equivalent to A, and so (by Proposition 1.6.2), we
have that rank(A) = rank(A1 ) + · · · + rank(Ak ).

Proposition 8.6.17. Let F be a field, let A ∈ Fn×n , and let


λ1 , . . . , λ 1 , . . . , λ k , . . . , λ k
| {z } | {z }
m1 mk

be the spectrum of A, where λ1 , . . . , λk are pairwise distinct eigenvalues of A and

m1 , . . . , mk are positive integers. Assume that A is similar to a Jordan matrix
J ∈ Fn×n . Then
ˆ each Jordan block of the Jordan matrix J is of the form Jt (λi ) for some
i ∈ {1, . . . , k} and t ∈ {1, . . . , mi };

ˆ for each i ∈ {1, . . . , k} and each positive integer r, the Jordan matrix J has
exactly
rank (A − λi In )r−1 − rank (A − λi In )r

many Jordan blocks Jt (λi ) satisfying t ≥ r.

Proof. Since A and J are similar, Theorem 8.2.9 guarantees that they have the same
spectrum. Since the matrix J is upper triangular (because it is a Jordan matrix),
Proposition 8.2.7 guarantees that J has the following entries on the main diagonal
(counting repetitions, but possibly in a different order):

λ1 , . . . , λ 1 , . . . , λ k , . . . , λ k .
| {z } | {z }
m1 mk

Note that this implies that m1 + · · · + mk = n. It also proves that each Jordan block
of J is of the form Jt (λi ) for some i ∈ {1, . . . , k} and t ∈ {1, . . . , mi }.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 591

It remains to compute the number of Jordan blocks of each type in the matrix J.
We begin with a simple technical claim.

Claim 1. For all scalars λ ∈ F and positive integers r, matrices (A−λIn )r

and (J − λIn )r are similar and therefore have the same rank.

Proof of Claim 1. Fix λ ∈ F. First of all, since A and J are similar, we know that
there exists an invertible matrix P ∈ Fn×n such that J = P −1 AP . Then

P −1 (A − λIn )P = P −1 AP − λP −1 In P = J − λi In ,

and so A − λIn and J − λIn are similar. Now, fix a positive integer r. By Proposi-
tion 4.5.18, matrices (A−λIn )r and (J −λIn )r are similar, and so by Corollary 4.5.20,
these two matrices have the same rank. ♦
Now, fix an index i ∈ {1, . . . , k}. We must show that
for each positive
integer
r, the Jordan matrix J has exactly rank (A − λi In )r−1 − rank (A − λi In )r many
Jordan blocks Jt (λi ) satisfying t ≥ r. Let a1 , . . . , aℓ be a non-decreasing sequence of
positive integers such that the Jordan blocks of J of the form Jt (λi ) are precisely the
blocks Ja1 (λi ), . . . , Jaℓ (λi ), counting repetitions and appearing in some order along
the main diagonal of J.61 Clearly, a1 + · · · + aℓ = mi .

Claim 2. For all non-negative integers r and indices q ∈ {0, 1, . . . , k}, if

a1 ≥ · · · ≥ aq ≥ r ≥ aq+1 ≥ · · · ≥ ak ,62 then
q
rank (A − λi In )r

P
= (n − mi ) + (aj − r).
j=1

Proof of Claim 2. First of all, since J is a Jordan matrix, so is the matrix J − λi In ,

and moreover, any Jordan block Jt (λj ) of J (with j ∈ {1, . . . , k}) corresponds to a
Jordan block Jt (λj − λi ) of J − λi In in the obvious way. In particular, the Jordan
blocks Jt (λi ) of J correspond to the Jordan blocks Jt (0) of J − λi In in the obvious
way, and we see that the Jordan blocks of J − λi In of the form Jt (0) are precisely
the blocks Ja1 (0), . . . , Jaℓ (0) (counting repetitions, and appearing in some order in
J − λi In ). Let J1 , . . . , Js be the Jordan blocks of the Jordan matrix J − λi In other
than Ja1 (0), . . . , Jak (0) (counting repetitions). So, J − λi In is the direct sum of the
Jordan blocks J1 , . . . , Js , Ja1 (0), . . . , Jak (0), arranged in some order.
61
Of course, the Jordan matrix J may contain other Jordan blocks as well, but those other Jordan
blocks do not have the eigenvalue λi on their main diagonal.
62
If q = 0, then this means that r ≥ a1 ≥ · · · ≥ ak . On the other hand, if q = k, then we have
that a1 ≥ · · · ≥ ak ≥ r.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 592

Now, fix a positive integer r and an index q ∈ {0, 1, . . . , k} such that a1 ≥ · · · ≥

aq ≥ r ≥ aq+1 ≥ · · · ≥ ak . Then

(∗)
rank (A − λi In )r rank (J − λi In )r



=

(∗∗) s
P k
 P 
r 
= rank(Jjr ) + rank Jaj (0) ,
j=1 j=1

where (*) follows from Claim 1, and (**) follows from Propositions 8.6.15 and 8.6.16.
By Proposition 8.6.14 we know that for all ℓ ∈ {1, . . . , s}, Jsr is a matrix of full rank,
and in particular, we have that

(1) rank(J1r ) + · · · + rank(Jsr ) = n − mi .63

Proposition 8.6.14 further implies the following:


r 
(2) for all j ∈ {1, . . . , q}, we have that rank Jaj (0) = aj − r (because r ≤ aj );

r 
(3) for all j ∈ {q + 1, . . . , k}, we have that rank Jaj (0) = 0 (because r ≥ aj ).

It now follows that

s
P k
 P 
r 
rank (A − λi In )r rank(Jjr ) +


= rank Jaj (0)
j=1 j=1

(∗) q
P
= (n − mi ) + (aj − r),
j=1

where (*) follows from (1), (2), and (3). ♦

Now, fix a positive integer r, and let q be the number of Jordan blocks Jt (λi ) of
J satisfying t ≥ r. Then a1 ≥ · · · ≥ aq ≥ r > aq+1 ≥ · · · ≥ ak ,64 and consequently,
a1 ≥ · · · ≥ aq ≥ r − 1 ≥ aq+1 ≥ · · · ≥ ak . By applying Claim 2 first to r and then to
r − 1, we obtain the following:
q
ˆ rank (A − λi In )r = (n − mi ) +

P
(aj − r);
j=1

q
ˆ rank (A − λi In )r−1 = (n − mi ) +

P
(aj − (r − 1)).
j=1

r−1 − rank (A − λi In )r = q, and we are done.

Consequently, rank (A − λi In )
63
We are using the fact that a1 + · · · + aℓ = mi .
64
If q = 0, then we simply have that r > a1 ≥ · · · ≥ ak . On the other hand, if q = k, then
a1 ≥ · · · ≥ ak ≥ r.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 593

We are now ready to prove Theorem 8.6.1, restated below for the reader’s
convenience.
Theorem 8.6.1. Let F be a field, and let J1 , J2 ∈ Fn×n be Jordan matrices. Then J1
and J2 are similar if and only if they have exactly the same Jordan blocks (counting
repetitions, but not counting the order in which the blocks appear in the two matrices).
Proof. If J1 and J2 have the same Jordan blocks (counting repetitions), then Propo-
sition 8.6.12 guarantees that they are similar. On the other hand, if J1 and J2 are
similar, then Proposition 8.6.17 guarantees that they have exactly the same Jordan
blocks (counting repetitions).65

8.6.5 Generalized eigenspaces, nilpotent linear functions, and the

existence of the Jordan normal form of a square matrix
The main goal of this subsection is to prove Theorem 8.6.30, which is the existence
part of Theorem 8.6.4. As an immediate consequence of Theorem 8.6.30, we obtain
Corollary 8.6.31, which is the existence part of 8.6.2.

Iterated linear functions. Suppose that A is a set and f : A → A is a function.66

We define f 0 := IdA ,67 and for all non-negative integers k, f k+1 := f k ◦ f . So, for
all positive integers k, we have that

fk = f ◦ · · · ◦ f .
| {z }
k

The function f k is called the k-th iterate of f .

Proposition 8.6.18. Let V be a vector space over a field F, and let f : V → V be a
linear function. Then all the following hold:
(a) functions f 0 , f 1 , f 2 , f 3 , . . . are all linear;68

(b) {0} = Ker(f 0 ) ⊆ Ker(f 1 ) ⊆ Ker(f 2 ) ⊆ Ker(f 3 ) ⊆ . . . ;

(c) for all non-negative integers r, if Ker(f r ) = Ker(f r+1 ), then Ker(f r ) = Ker(f r+1 ) =
Ker(f r+2 ) = Ker(f r+3 ) = . . . ;

(d) for all non-negative integers r and all scalars λ ∈ F, both Ker(f r ) and Im(f r )
are (f + λIdV )-invariant subspaces of V ;69
65
Indeed, we set A := J1 . Then A is similar both to J1 and to J2 , and we simply apply
Proposition 8.6.17 twice: first to A and J1 , and then to A and J2 .
66
Here, it is important that the domain and the codomain of f are the same.
67
As usual, IdA is the identity function on A, that is, IdA : A → A is given by IdA (a) = a for all
a ∈ A.
68
Note that this implies that Ker(f 0 ), Ker(f 1 ), Ker(f 2 ), Ker(f 3 ), . . . are all defined.
69
Since f and IdV are linear, Proposition 4.1.7 guarantees that f + λIdV is linear for all λ ∈ F.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 594

(e) for all positive integers r, and all v1 , . . . , vr ∈ V , if vi ∈ Ker(f i ) \ Ker(f i−1 ) for
all i ∈ {1, . . . , r}, then the set {v1 , . . . , vr } is linearly independent;

(f ) for all non-negative integers r, all i ∈ {0, . . . , r}, and all v ∈ V , we have that
v ∈ Ker(f r ) if and only if f r−i (v) ∈ Ker(f i );

(g) for all positive integers r and vectors v ∈ Ker(f r ) \ Ker(f r−1 ), both the following
hold:

ˆ for all i ∈ {1, . . . , r}, f r−i (v) ∈ Ker(f i ) \ Ker(f i−1 );

ˆ the set f r−1 (v), f r−2 (v), . . . , f 2 (v), f (v), v is linearly independent.


Proof. We prove (a)-(g) in order, with one exception: we prove (c) last, because our
proof of (c) relies on (f).
(a) The function f 0 = IdV is obviously linear, and the function f 1 = f is linear
by assumption. The result now follows from Proposition 4.1.7(c) via an obvious
induction.
(b) First of all, by definition, f 0 = IdV , and so Ker(f 0 ) = {0}. Now, fix a
non-negative integer r. We must show that Ker(f r ) ⊆ Ker(f r+1 ). Fix v ∈ Ker(f r ).
Then

(∗) (∗∗)
f r+1 (v) = f f r (v) = f (0) = 0,
where (*) follows from the fact that v ∈ Ker(f r ), and (**) follows from the fact that
f is linear and from Proposition 4.1.6. This proves that Ker(f r ) ⊆ Ker(f r+1 ).
(d) Fix a non-negative integer r and a scalar λ ∈ F. We first show that Ker(f r )
is (f + λIdV )-invariant. Fix any v ∈ Ker(f r ), so that f r (v) = 0; we must show that
(f + λIdV )(v) ∈ Ker(f r ). We compute:
(∗)
 
f r (f + λIdV )(v) = f r+1 (v) + λf r (v)

f f r (v) + λf r (v)


=

(∗∗)
= f (0) +λ0 = 0,
|{z}
(∗∗∗)
= 0

where (*) follows from the linearity of f r , (**) follows from the fact that v ∈
Ker(f r ) (and so f r (v) = 0), and (***) follows from the linearity of f and from
Proposition 4.1.6. Thus, (f + λIdV )(v) ∈ Ker(f r ), and it follows that Ker(f r ) is
(f + λIdV )-invariant.
It remains to show that Im(f r ) is (f + λIdV )-invariant. Fix any v ∈ Im(f r );
we must show that (f + λIdV )(v) ∈ Im(f r ). Since v ∈ Im(f r ), we know that there

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 595

exists some u ∈ V such that v = f r (u). We now compute:

(f + λIdV )(v) = f (v) + λv

f f r (u) + λf r (u)


=

f r f (u) + λf r (u)


=

(∗)
f r f (u) + λu ,


=
| {z }
=:w

where (*) follows from the linearity of f r . We have now obtained that (f +λIdV )(v) =
f r (w), and we deduce that (f + λIdV )(v) ∈ Im(f r ). This proves that Im(f r ) is
(f + λIdV )-invariant.
(e) Fix a positive integer r and vectors v1 , . . . , vr ∈ V , and assume that vi ∈
Ker(f i ) \ Ker(f i−1 ) for all i ∈ {1, . . . , r}. We must show that the set {v1 , . . . , vr } is
linearly independent. Fix scalars α1 , . . . , αr ∈ F such that

α1 v1 + · · · + αr vr = 0.

We must show that α1 = · · · = αr = 0. Suppose otherwise, and let k ∈ {1, . . . , r} be

maximal with the property that αk ̸= 0. Then

α1 v1 + · · · + αk vk = 0.

We apply f k−1 to both sides of the equation above, and using the linearity of f k−1
and Proposition 4.1.6, we obtain

α1 f k−1 (v1 ) + · · · + αk f k−1 (vk ) = 0.

In view of (b), we have that v1 , . . . , vk−1 ∈ Ker(f k−1 ), and consequently, f k−1 (v1 ) =
· · · = f k−1 (vk−1 ) = 0. Thus, αk f k−1 (vk ) = 0. Since αk = ̸ 0, it follows that
f k−1 (vk ) = 0, and consequently, vk ∈ Ker(f k−1 ), a contradiction.
(f) Fix a non-negative integer
integer r, an index i ∈ {0, . . . , r}, and a vector
v ∈ V . Then f r (v) = f i f r−i (v) , and consequently, we have the following sequence
of equivalences:

v ∈ Ker(f r ) ⇐⇒ f r (v) = 0

f i f r−i (v) = 0


⇐⇒

⇐⇒ f r−i (v) ∈ Ker(f i ).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 596

(g) Fix a positive integer r and a vector v ∈ Ker(f r ) \ Ker(f r−1 ). By (f), we
know that f r−i (v)∈ Ker(f i ) \ Ker(f i−1 ) for all i ∈ {1, . . . , r}.70 But now by (e), we
have that the set f r−1 (v), f r−2 (v), . . . , f 2 (v), f (v), v is linearly independent.
(c) Fix a non-negative integer r, and assume that Ker(f r ) = Ker(f r+1 ). We must
show that Ker(f r ) = Ker(f r+1 ) = Ker(f r+2 ) = Ker(f r+3 ) = . . . . Clearly, it suffices
to show that for all non-negative integers j, we have that Ker(f r+j ) = Ker(f r+j+1 ).
Suppose otherwise. In view of (b), this means that there exists some non-negative
integer j such that Ker(f r+j ) ⫋ Ker(f r+j+1 ). Fix some v ∈ Ker(f r+j+1 ) \ Ker(f r+j ).
Then (f) guarantees that f j (v) ∈ Ker(f r+1 ) \ Ker(f r ), contrary to the fact that
Ker(f r ) = Ker(f r+1 ).

Proposition 8.6.19. Let V be a non-trivial, finite-dimensional vector space, and let

f : V → V be a linear function. Then there exists a (unique) non-negative integer p
such that
Ker(f 0 ) ⫋ Ker(f 1 ) ⫋ Ker(f 2 ) ⫋ . . . ⫋ Ker(f p ) = Ker(f p+1 ) = . . .
| {z }
={0}

Moreover, for this integer p, we have that p ≤ dim Ker(f p ) ≤ dim(V ) and V =

Ker(f p ) ⊕ Im(f p ).

Remark: It is possible that p = 0. In this case, we simply have that {0} = Ker(f 0 ) =
Ker(f 1 ) = Ker(f 2 ) = . . . . In view of Theorem 4.2.4, this happens precisely when
our linear function f is one-to-one.

Proof. Set n := dim(V ).71

Claim 1. Ker(f n ) = Ker(f n+1 ).

Proof of Claim 1. Suppose otherwise. Then by Proposition 8.6.18(b-c), we have that

Ker(f 0 ) ⫋ Ker(f 1 ) ⫋ · · · ⫋ Ker(f n ) ⫋ Ker(f n+1 ). For each i ∈ {1, . . . , n + 1}, we
fix some vi ∈ Ker(f i ) \ Ker(f i−1 ). By Proposition 8.6.18(e), {v1 , . . . , vn , vn+1 } is
linearly independent. But this is impossible since by Theorem 3.2.17(a), all linearly
independent sets in the n-dimensional vector space V are of size at most n. ♦
Now, using Claim 1, we fix the smallest non-negative integer p such that Ker(f p ) =
Ker(f p+1 ). By Proposition 8.6.18(b-c), we have that

Ker(f 0 ) ⫋ Ker(f 1 ) ⫋ Ker(f 2 ) ⫋ . . . ⫋ Ker(f p ) = Ker(f p+1 ) = . . . .

| {z }
={0}
70
Let us check this in full detail. Fix i ∈ {1, . . . , r}. Since v ∈ Ker(f r ), part (f) applied to r, i,
and v implies that f r−i (v) ∈ Ker(f i ). On the other hand, since v ∈ / Ker(f r−1 ), part (f) applied to
r−i (r−1)−(i−1)
r − 1, i − 1, and v, implies that f (v) = f / Ker(f i−1 ).
(v) ∈
71
Since V is non-trivial and finite-dimensional, we have that n is a positive integer.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 597

Claim 2. p ≤ dim Ker(f p ) ≤ n.

Proof of Claim 2. Since Ker(f p ) is a subspace of
the n-dimensional subspace V ,

p

guarantees that dim Ker(f ) ≤ n. It remains to show that
Theorem 3.2.21(b)
p
p ≤ dim Ker(f ) . If p = 0, then this is immediate. So, we may assume that p ≥ 1.
We now proceed similarly as in Claim 1. By the choice of p, we have that

Ker(f 0 ) ⫋ Ker(f 1 ) ⫋ Ker(f 2 ) ⫋ . . . ⫋ Ker(f p ).

| {z }
={0}

For all i ∈ {1, . . . , p}, we fix vi ∈ Ker(f i ) \ Ker(f i−1 ). By Proposition 8.6.18(e),
vectors v1 , . . . , vp are linearly independent, and obviously, they all belong to Ker(f p ).
By Theorem
3.2.17(a), all linearly independent sets
in Ker(f p ) are of size at most
dim Ker(f p ) , and we deduce that p ≤ dim Ker(f p ) . ♦
It remains to show that V = Ker(f p ) ⊕ Im(f p ), that is, that Ker(f p ) ∩ Im(f p ) =
{0} and V = Ker(f p ) + Im(f p ).
We first show that Ker(f p ) ∩ Im(f p ) = {0}. Since both Ker(f p ) and Im(f p ) are
subspaces of V , they both contain 0, and so 0 ∈ Ker(f p ) ∩ Im(f p ). Now, fix any
v ∈ Ker(f p ) ∩ Im(f p ); we must show that v = 0. Since v ∈ Im(f p ), we know that
there exists some u ∈ V such that f p (u) = v. We now apply f p to both sides of the
equation, and we obtain f 2p (u) = f p (v) = 0, where the last equality follows from
the fact that v ∈ Ker(f p ). But now u ∈ Ker(f 2p ) = Ker(f p ),72 and we deduce that
f p (u) = 0, i.e. v = 0. This proves that Ker(f p ) ∩ Im(f p ) = {0}.
It remains to show that V = Ker(f p ) + Im(f p ). We compute:
(∗)
     
dim Ker(f p ) + Im(f p ) = dim Ker(f p ) + Im(f p ) + dim Ker(f p ) ∩ Im(f p )

(∗∗) (∗∗∗∗)
dim Ker(f p ) + dim Im(f p )



= = dim(V ),
| {z }
(∗∗∗)
= rank(f p )

where (*) follows from the fact that Ker(f p ) ∩ Im(f p ) = {0} (proven above), (**)
follows from Theorem 3.2.23, (***) follows from the definition of rank(f p ), and
(****) follows from the rank-nullity theorem. Since V is finite-dimensional, Theo-
rem 3.2.21(c) now guarantees that Ker(f p ) + Im(f p ) = V , and we are done.

Generalized eigenvectors and generalized eigenspaces. Suppose that V is a

vector space over a field F, and that f : V → V is a linear function, and that λ ∈ F is
a scalar. Then Propositions 4.1.7 and 8.6.18(a) together guarantee that the function
72
We saw above that Ker(f p ) = Ker(f p+1 ) = Ker(f p+2 ) = . . . . In particular, Ker(f p ) = Ker(f 2p ).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 598

(f − λIdV )r : V → V is linear for all linear non-negative integers r, and consequently,

we can define the set
Gλ (f ) := v ∈ V | ∃r ∈ N0 s.t. (f − λIdV )r (v) = 0


∞
S 
r 
= Ker f − λIdV .
r=0

If λ is an eigenvalue of f , then the set Gλ (f ) is called the generalized eigenspace of

f associated with the eigenvalue λ.73

Proposition 8.6.20. Let V be a vector space over a field F, and let f : V → V be a

linear function. Then all the following hold:

(a) for all scalars λ ∈ F, Gλ (f ) is an f -invariant subspace of V , and moreover,

Eλ (f ) is a subspace of Gλ (f );74

(b) for all scalars λ ∈ F, the subspace Gλ (f ) is non-trivial if and only if λ is an

eigenvalue of f ;

(c) for all distinct λ1 , λ2 ∈ F, we have that Gλ1 (f ) ∩ Gλ2 (f ) = {0}.

Proof. To simplify notation, for each scalar λ ∈ F, we set Gλ := Gλ (f ) and Eλ :=

Eλ (f ). We begin with a sequence of technical claims.

Claim 1. For all scalars λ ∈ F, Gλ is a subspace of V .

S∞ 1. Fix ar scalar λ ∈ F, and to simplify notation, set g := f − λIdV , so

Proof of Claim
that Gλ = r=0 Ker(g ). We must show that Gλ is a subspace of V . It suffices to
verify that Gλ satisfies the three conditions from Theorem 3.1.7.
First, we note that g 0 (0) = IdV (0) = 0, and we deduce that 0 ∈ Ker(g 0 ) ⊆ Gλ .
Next, fix u, v ∈ Gλ . Then there exist non-negative integers p and q such that
u ∈ Ker(g p ) and v ∈ Ker(g q ). By symmetry, we may assume that p ≤ q. Then by
Proposition 8.6.18(b), we have that Ker(g p ) ⊆ Ker(g q ), and we deduce that u, v ∈
Ker(g q ). Since Ker(g q ) is a subspace of V ,75 we deduce that u + v ∈ Ker(g q ) ⊆ Gλ .
Finally, fix u ∈ Gλ and α ∈ F. Then there exists a non-negative integer p such
that u ∈ Ker(g p ). We once again use the fact that Ker(g p ) is a subspace of V , and
we deduce that αu ∈ Ker(g p ) ⊆ Gλ .
We have now verified that Gλ satisfies the three conditions from Theorem 3.1.7,
and we deduce that Gλ is indeed a subspace of V . ♦
73
If λ is not an eigenvalue of f , the set Gλ (f ) is still defined, but we do not refer to it as an
eigenspace.
74
Recall that Eλ (f ) = {v ∈ V | f (v) = λv}. If λ is an eigenvalue of f , then we referred to Eλ (f )
as the eigenspace of f associated with λ.
75
This follows from Theorem 4.2.3(d) applied to the linear function g p .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 599

Claim 2. For all scalars λ ∈ F, Eλ is a subspace of Gλ .

Proof of Claim 2. Fix a scalar λ ∈ F. By Proposition 8.1.4, Eλ is a subspace of V ,

and by Claim 1, Gλ is a subspace of V . So, it suffices to show that Eλ ⊆ Gλ . So, fix
any v ∈ Eλ , so that f (v) = λv. But then (f − λIdV )(v) = 0, and it follows that
v ∈ Ker(f − λIdV ) ⊆ Gλ . ♦

Claim 3. For all scalars
λ ∈ F, positive integers

p, and vectors v ∈ V ,
if v ∈ Ker (f − λIdV )p \ Ker (f − λIdV )p−1 , then the scalar λ is an
eigenvalue of f , and (f − λIdV )p−1 (v) is an eigenvector of f associated
with the eigenvalue λ.

Proof of Claim 3. Fix a scalar λ ∈
F, a positive integer
p, and a vector v ∈ V .

Assume that v ∈ Ker (f − λIdV )p \ Ker (f − λIdV )p−1 . We must show that
w := (f − λIdV )p−1 (v) is a non-zero vector in V that satisfies f (w) = λw.
Set g := f − λIdV , so that v ∈ Ker(g p ) \ Ker(g p−1 ) and w = g p−1 (v). By
Proposition 8.6.18(f), we have that w ∈ Ker(g 1 ) \ Ker(g 0 ) = Ker(g) \ {0}. In
particular, we have that g(w) = 0, which means that (f − λIdV )(w) = 0, i.e.
f (w) = λw. Since w ̸= 0, we deduce that λ is indeed an eigenvalue of f , and that
w is an eigenvector of f associated with the eigenvalue λ. ♦

Claim 4. For all scalars λ, λ′ ∈ F, Gλ is an (f −λ′ IdV )-invariant subspace

of V .

Proof of Claim 4. Fix scalars λ, λ′ ∈ F. By Claim 1, Gλ is a subspace of V , and we just

need to show that Gλ is an (f − λ′ IdV )-invariant. Fix a vector v ∈ Gλ ; we must show
that (f − λ′ IdV )(v) ∈ Gλ . Since v ∈ G
λ , we know that there exists a non-negative

integer r such that v ∈ Ker (f −λIdV )r . By Proposition 8.6.18(d), Ker (f −λIdV )r
is (f − λ′ IdV )-invariant,76 and so (f − λ′ IdV )(v) ∈ Ker (f − λIdV )r ⊆ Gλ . ♦

We are now ready to prove (a) and (b). Fix a scalar λ ∈ F. By Claims 1 and 4,
Gλ is an f -invariant subspace of V .77 Moreover, by Claim 2, Eλ is a subspace of
Gλ . This proves (a). For (b), suppose first that Gλ is non-trivial, and fix some
p .


v ∈ Gλ \ {0}. Let p be the
smallest positive integer such that v ∈ Ker (f − λIdV )
Then v ∈ Ker (f −λIdV )p \Ker (f −λIdV )p−1 , and so by Claim 3, λ is an eigenvalue

of f . Suppose, conversely, that λ is an eigenvalue of f . Then Proposition 8.1.4

guarantees that Eλ is a non-trivial subspace of V , and so by Claim 2, Gλ is also
non-trivial. This proves (b).
It remains to prove (c). Fix distinct scalars λ1 , λ2 ∈ F. We must show that
Gλ1 ∩ Gλ2 = {0}. By Claim 1, Gλ1 and Gλ2 are both subspaces of V , and so they
76
Indeed, we set fe := f − λIdV and λe := λ − λ′ . By Proposition 8.6.18(d), we have that Ker(fer )
is (f + λIdV )-invariant, i.e. Ker (f − λIdV )r is (f − λ′ IdV )-invariant.


e e
77
We apply Claim 4 to λ′ := 0.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 600

both contain 0, i.e. 0 ∈ Gλ1 ∩ Gλ2 . Now, fix any u ∈ Gλ1 ∩ Gλ2 ; we must show that
u ̸= 0. Fix the smallest non-negative integer p such that u ∈ Ker (f − λ1 IdV )p .

If p = 0, then u = 0,78 and we
are done. We may
therefore assume that p ≥ 1,

so that u ∈ Ker (f − λ1 IdV )p \ Ker (f − λ1 IdV )p−1 . Then by Claim 3, λ1 is an
eigenvalue of f , and w := (f − λ1 IdV )p−1 (u) is an eigenvector of f associated with
the eigenvalue λ1 . On the other hand, by Claim 4, Gλ2 is (f − λ1 IdV )-invariant. So,
since u ∈ Gλ2 , we have that w ∈ Gλ2 . We will derive a contradiction by showing
that w = 0, contrary to the fact that w is an eigenvector of f .

Claim 5. For all non-negative integers k, we have that (f −λ2 IdV )k (w) =
(λ1 − λ2 )k w.

Proof of Claim 5. We proceed by induction on k. For the base case, we compute:

(f − λ2 IdV )0 (w) = IdV (w) = w = (λ1 − λ2 )0 (w).

Now, fix a non-negative integer k, and assume inductively that (f − λ2 IdV )k (w) =
(λ1 − λ2 )k w. We then compute:

(f − λ2 IdV )k+1 (w) (f − λ2 IdV )k f (w) − λ2 IdV (w)

=

(∗)
= (f − λ2 IdV )k (λ1 w − λ2 w)

(f − λ2 IdV )k (λ1 − λ2 )w


=

(∗∗)
= (λ1 − λ2 )(f − λ2 IdV )k (w)

(∗∗∗)
= (λ1 − λ2 )(λ1 − λ2 )k w

= (λ1 − λ2 )k+1 w,

where (*) follows from the fact that w is an eigenvector of f associated with the
eigenvalue λ1 (and so f (w) = λ1 w), (**) follows from the linearity of (f − λ2 IdV )k ,
and (***) follows from the induction hypothesis. This completes the induction. ♦
Now, suppose that w ∈
Gλ2 . Then there exists a non-negative integer k such
that w ∈ Ker (f − λ2 IdV )k . But now

(∗) (∗∗)
0 = (f − λ2 IdV )k (w) = (λ1 − λ2 )k (w),

where (*) follows from the fact that w ∈ Ker (f − λ2 IdV )k , and (**) follows from

78
This is because Ker (f − λ1 IdV )0 = Ker(IdV ) = {0}.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 601

Claim 5. Since λ1 = ̸ λ2 , it follows that w = 0, contrary to the fact that w is an

eigenvector of f . This completes the proof of (c).

We now introduce some terminology. Suppose that V is a vector space over a

field F, that f : V → V is a linear function, and that λ ∈ F is an eigenvalue of f .
We then define the following:

ˆ a generalized eigenvector of f associated with λ is any non-zero vector v ∈ V

such that for some positive integer r, we have that (f − λIdV )r (v) = 0;79

ˆ the rank of a generalized eigenvector v of f associated with λ is the smallest

positive integer r such that (f − λIdV )r (v) = 0;80

ˆ for a generalized eigenvector v of f associated with λ and of rank r, the Jordan

chain started by v is the set

(f − λIdV )r−1 (v), . . . , (f − λIdV )2 (v), (f − λIdV )(v), v .



Note that the elements of the generalized eigenspace Gλ (f ) are precisely the general-
ized eigenvectors of f associated with λ. Moreover, by Proposition 8.1.4, we have
that Eλ (f ) = Ker(f − λIdV ). So, eigenvectors of f associated with λ are precisely
the the generalized eigenvectors of rank 1 of f associated with λ.
Remark: In view of Proposition 8.6.20(c), any generalized eigenvector of a linear
function f : V → V (where V is a vector space over a field F) is associated with
exactly one eigenvalue of f .
Remark: By Proposition 8.6.18(g), every Jordan chain is a linearly independent
set. We note that the basis B from the proof of Theorem 8.6.4 will turn out to be
the union of pairwise disjoint Jordan chains (associated with various eigenvalues).

Theorem 8.6.21. Let V be a non-trivial, finite-dimensional vector space over an

algebraically closed field F, let f : V → V be a linear function, and let

λ1 , . . . , λ 1 , . . . , λ k , . . . , λ k
| {z } | {z }
m1 mk

be the spectrum of f , where λ1 , . . . , λk are pairwise distinct eigenvalues of f and

m1 , . . . , mk are positive integers. Then V = Gλ1 (f ) ⊕ · · · ⊕ Gλk (f ). Moreover, for
each index i ∈ {1, . . . , k}, all the following hold:

ˆ Gλi (f ) is an f -invariant subspace of V of dimension mi ,

79
The reason we specify that r is positive (rather than merely non-negative) is simply that
(f − λIdV )0 = IdV , and so there are no non-zero vectors v ∈ V such that (f − λIdV )0 (v) = 0.
80
Note that this means that v ∈ Ker(f r ) \ Ker(f r−1 ).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 602

ˆ there exists a positive integer pi ≤ mi such that

     
Ker (f − λi IdV )0 ⫋ Ker (f − λi IdV )1 ⫋ Ker (f − λi IdV )2 ⫋ ...
| {z }
={0}

   
... ⫋ Ker (f − λi IdV )pi = Ker (f − λi IdV )pi +1 = ...,
 
and in particular, Gλi (f ) = Ker (f − λi IdV )pi .

ˆ λi is the only eigenvalue of f |Gλ (f ) , and the algebraic multiplicity of the

i
eigenvalue λi of f |Gλ (f ) is mi .
i

Remark: Since the field F is algebraically closed, we know that pf (λ) can be factored
into linear terms, and so pf (λ) = (λ−λ1 )m1 . . . (λ−λk )mk and m1 +· · ·+mk = dim(V ).

Proof. Set n := dim(V ). To simplify notation, for all indices i ∈ {1, . . . , k}, we
∞
set Gi := Gλi (f ) and gi := f − λi IdV ,81 so that Gi = Ker gir . Now, by
S

i=0
Proposition 8.6.19 applied to gi (for i ∈ {1, . . . , k}), we know that there exists a
(unique) non-negative integer pi such that

Ker(gi0 ) ⫋ Ker(gi1 ) ⫋ Ker(gi2 ) ⫋ . . . ⫋ Ker(gipi ) = Ker(gipi +1 ) = . . . ,

and consequently, Gi = Ker(gipi ); but in fact, since Gi ̸= {0} (because λi is an

eigenvalue of f , and Gi contains all eigenvectors of f associated with λi ), we see
that pi ≥ 1.

Claim 1. For all i ∈ {1, . . . , k}, both Gi and Im(gipi ) are f -invariant
subspaces of V , and moreover, V = Gi ⊕ Im(gipi ).

Proof of Claim 1. Fix an index i ∈ {1, . . . , k}. By Proposition 8.6.19 applied gi , we

know that V = Ker(gipi ) ⊕ Im(gipi ) = Gi ⊕ Im(gipi ). The fact that Gi = Ker(gipi ) and
Im(gipi ) are f -invariant now follows from Proposition 8.6.18(d) applied to the linear
function gi and the scalar λi .82 ♦
To simplify notation, for each index i ∈ {1, . . . , k}, we set fi := f |Gi . The
fact that the functions f1 , . . . , fk are well defined follows from the fact that, by
Claim 1, G1 , . . . , Gk are f -invariant subspaces of V . Moreover, since f is linear, so
are f1 , . . . , fk .

Claim 2. For all indices i ∈ {1, . . . , k}, we have that pfi (λ) = (λ − λi )mi ,
dim(Gi ) = mi , and pi ≤ mi .
81
Since f and IdV are linear, Proposition 4.1.7 guarantees that gi is linear as well.
82
Indeed, by Proposition 8.6.18(d), both Ker(gipi ) and Im(gipi ) are (gi + λi IdV )-invariant, and by
construction, gi + λi IdV = f .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 603

Proof of Claim 2. Fix an index i ∈ {1, . . . , k}. By the definition of the characteristic
polynomial, we know that the degree of pfi (λ) is equal to dim(Gi ), and by Proposi-
tion 8.6.19 applied to gi , we know that pi ≤ dim(Gi ). So, it suffices to show that
pfi (λ) = (λ − λi )mi .
By Claim 1, Im(gipi ) is an f -invariant subspace of V , and so hi := f |Im(gpi ) is
i
well defined. Further, by Claim 1 and by Proposition 8.6.11, we know that

pf (λ) = pfi (λ) phi (λ).

Since pf (λ) = (λ − λ1 )m1 . . . (λ − λk )mk , it is now enough to show that fi has no

eigenvalues other than λi , and that λi is not an eigenvalue of hi . This essentially
follows from Proposition 8.6.20, but we give the full details below, as follows.
We first show that fi has no eigenvalues other than λi . Suppose that u ∈ Gi \ {0}
is an eigenvector of fi associated with an eigenvalue λ0 of fi . Then u is an eigenvector
of f associated with λ0 ,83 and so by Proposition 8.6.20(a), u ∈ Gλ0 , and consequently,
u ∈ Gλ0 (f ) ∩ Gλi (f ). Since u ̸= 0, Proposition 8.6.20(c) guarantees that λ0 = λi .
This proves that fi indeed has no eigenvalues other than λi .
It remains to show that λi is not an eigenvalue of hi . Fix any u ∈ Im(gipi ) such
that hi (u) = λi u; we must show that u = 0. Now, we have that f (u) = hi (u) = λi u,
and so u ∈ Ker(f − λi IdV ) ⊆ Gi . Thus, u ∈ Gi ∩ Im(gipi ). But by Claim 1, we have
that V = Gi ⊕ Im(gipi ), which in particular means that Gi ∩ Im(gipi ) = {0}. Thus,
u = 0. This proves that λi is not an eigenvalue of hi . ♦
In view of Claims 1 and 2, it now just remains to show that V = G1 ⊕· · ·⊕Gk . Set
U := G1 +· · ·+Gk . In view of Proposition 8.6.20(c), it follows that U = G1 ⊕· · ·⊕Gk .
But now
(∗) (∗∗)
dim(U ) = dim(G1 ) + · · · + dim(Gk ) = m1 + · · · + mk = n,

where (*) follows from Theorem 3.2.23, and (**) follows from Claim 2. Thus,
U is an n-dimensional subspace of the n-dimensional vector space V , and so by
Theorem 3.2.21(c), we have that U = V , that is, V = G1 ⊕ · · · ⊕ Gk . This completes
the argument.

Linear independence over a subspace. Given a vector space V over a field F, a

subspace U of V , and vectors v1 , . . . , vt ∈ V (t ≥ 0), we say that vectors v1 , . . . , vt
are linearly independent over U , or that the set {v1 , . . . , vt } is linearly independent
over U , provided that the following two conditions are satisfied:
ˆ vectors v1 , . . . , vt are linearly independent;

ˆ Span(v1 , . . . , vt ) ∩ U = {0}.
83 (∗)
Indeed, u is a non-zero vector of V that satisfies f (u) = fi (u) = λ0 u, where (*) follows from
the fact that u is an eigenvector of fi associated with the eigenvalue λ0 .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 604

Proposition 8.6.22. Let V be a vector space over a field F, let U be a subspace of

V , and let v1 , . . . , vt ∈ V . Assume that vectors v1 , . . . , vt are linearly independent
over U . Then v1 , . . . , vt ∈/ U.

Proof. Suppose otherwise, and fix an index i ∈ {1, . . . , t} such that vi ∈ U . Then
vi ∈ Span(v1 , . . . , vt ) ∩ U = {0}, and consequently, vi = 0. But this is impossible
since vectors v1 , . . . , vt are linearly independent.

Remark: The converse of Proposition 8.6.22 is false, even if we additionally assume

that v1 , . . . , vt are linearly independent. For example, consider the subspace
n x  o
U := |x∈R
x

of R2 , and consider the standard basis vectors e1 and e2 of R2 . Then vectors e1 , e2

are linearly independent and do not belong to U , but Span(e1 , e2 ) ∩ U = R2 ∩ U =
U ̸= {0}, and so e1 , e2 are not linearly independent over U .

Proposition 8.6.23. Let V be a vector space over a field F, let U be a finite-

dimensional subspace of V , and let v1 , . . . , vt ∈ V . Set m := dim(U ). Then the
following are equivalent:

(a) vectors v1 , . . . , vt are linearly independent over U ;

(b) for all scalars α1 , . . . , αt ∈ F, if α1 v1 + · · · + αt vt ∈ U , then α1 = · · · = αt = 0;

(c) for all bases B = {b1 , . . . , bm } of U , vectors b1 , . . . , bm , v1 , . . . , vt are linearly

independent;

(d) there exists a basis B = {b1 , . . . , bm } of U such that vectors b1 , . . . , bm , v1 , . . . , vt

are linearly independent.

Remark: Some texts take (b) as the definition of linear independence over U .
By Proposition 8.6.23, and in particular, by the equivalence of (a) and (b), this
alternative definition is equivalent to our definition.

Proof. We will prove the implications “(a) =⇒ (b) =⇒ (c) =⇒ (d) =⇒ (a).” Since
U is m-dimensional and therefore has at least one basis of size m, the implication
“(c) =⇒ (d)” is immediate. We must prove the remaining three implications.
First, we assume (a), and we prove (b). Fix scalars α1 , . . . , αt ∈ F, and assume
that α1 v1 +· · ·+αt vt ∈ U . Then α1 v1 +· · ·+αt vt ∈ Span(v1 , . . . , vt )∩U . But by (a),
we have that Span(v1 , . . . , vt ) ∩ U = {0}, and we deduce that α1 v1 + · · · + αt vt = 0.
But once again by (a), vectors v1 , . . . , vt are linearly independent. So, α1 = · · · =
αt = 0. This proves (b).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 605

Next, we assume (b), and we prove (c). Fix any basis B = {b1 , . . . , bm } of U . We
must show that vectors b1 , . . . , bm , v1 , . . . , vt are linearly independent. Fix scalars
α1 , . . . , αm , β1 , . . . , βt ∈ F such that α1 b1 + · · · + αm bm + β1 v1 + · · · + βt vt = 0. Then
β1 v1 + · · · + βt vt = −α1 b1 − · · · − αm bm ∈ Span(b1 , . . . , bm ) = U , and so by (b),
we have that β1 = · · · = βt = 0. Consequently, α1 b1 + · · · + αm bm = 0, and so since
B = {b1 , . . . , bm } is linearly independent (because it is a basis of U ), we have that
α1 = · · · = αm = 0. We have now shown that α1 = · · · = αt = β1 = · · · = βm = 0,
and (c) follows.
Finally, we assume (d), and we prove (a). Using (d), we fix a basis B =
{b1 , . . . , bm } of U such that vectors b1 , . . . , bm , v1 , . . . , vt are linearly independent.
It is clear that {v1 , . . . , vt } is linearly independent, and we just need to show that
Span(v1 , . . . , vt ) ∩ U = {0}. Since Span(v1 , . . . , vt ) and U are both subspaces
of V , they both contain 0, and so 0 ∈ Span(v1 , . . . , vt ) ∩ U . Now, fix any u ∈
Span(v1 , . . . , vt ) ∩ U ; we must show that u = 0. Since u ∈ Span(v1 , . . . , vt ), we
know that there exist scalars α1 , . . . , αt ∈ F such that u = α1 v1 + · · · + αt vt . On
the other hand, since u ∈ U , and since B = {b1 , . . . , bm } is a basis of U , we know
that there exist scalars β1 , . . . , βm ∈ F such that u = β1 b1 + · · · + βm bm . But now
α1 v1 + · · · + αt vt = β1 b1 + · · · + βm bm , and consequently, β1 b1 + · · · + βm bm −
α1 v1 − · · · − αt vt = 0. Since {b1 , . . . , bm , v1 , . . . , vt } is linearly independent, we
deduce that β1 = · · · = βm = −α1 = · · · = −αt = 0. So, u = 0. This proves that
Span(v1 , . . . , vt ) ∩ U = {0}, and (a) follows.

Proposition 8.6.24. Let V be a vector space over a field F, let r be a positive

integer, and let v1 , . . . , vt ∈ V (t ≥ 0). If vectors v1 , . . . , vt are linearly independent
over Ker(f r ), then vectors f (v1 ), . . . , f (vt ) are linearly independent over Ker(f r−1 ).
Proof. Assume that vectors v1 , . . . , vt are linearly independent over Ker(f r ). We
must show that vectors v1 , . . . , vt are linearly independent over Ker(f r−1 ). We use
the equivalence of (a) and (b) from Proposition 8.6.23. Fix scalars α1 , . . . , αt ∈ F
such that v := α1 f (v1 ) + · · · + αt f (vt ) ∈ Ker(f r−1 ); we must show that α1 = · · · =
αt = 0.84 By the linearity of f , we have that v = f (α1 v1 + · · · + αt vt ). Since
v ∈ Ker(f r−1 ), Proposition 8.6.18(f) guarantees that α1 v1 + · · · + αt vt ∈ Ker(f r ).
But since vectors v1 , . . . , vt are linearly independent over Ker(f r ), Proposition 8.6.23
guarantees that α1 = · · · = αt = 0.

Nilpotent linear functions. A linear function f : V → V (where V is a vector

space over a field F) is said to be nilpotent if there exists a positive integer p such
that f p is the zero function,85 i.e. Ker(f p ) = V . Nilpotent matrices can be defined
in an analogous way: a square matrix A ∈ Fn×n (where F is a field) is nilpotent if
there exists a positive integer p such that Ap = On×n .
84
By the equivalence of (a) and (b) from Proposition 8.6.23, this will immediately imply that
vectors f (v1 ), . . . , f (vt ) are linearly independent over Ker(f r−1 ), which is what we need to show.
85
This simply means that f p (v) = 0 for all v ∈ V .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 606

Proposition 8.6.25. Let F be a field, and let A ∈ Fn×n be a square matrix. Then
the following are equivalent:
(a) A is nilpotent;
(b) pA (λ) = λn ;
(c) A has only one eigenvalue, namely 0, and the algebraic multiplicity of this
eigenvalue is n.
Remark: Note that Proposition 8.6.25 immediately implies that the only eigenvalue
of a nilpotent n × n matrix is 0, and moreover, the algebraic multiplicity of this
eigenvalue is n.

Proof. Obviously, (b) and (c) are equivalent. We will complete the proof by showing
that (a) and (b) are equivalent, that is, that A is nilpotent if and only if pA (λ) = λn .
If pA (λ) = λn , then the Cayley-Hamilton theorem (see section 8.3) guarantees
that An = On×n , and consequently, A is nilpotent.
For the reverse implication, we assume that A is nilpotent, and we prove that
pA (λ) = λn . Using the fact that A is nilpotent, we fix a positive integer p such that
Ap = On×n . Now, recall the following factoring formula:
 p−1 
xp − y p = (x − y) xp−i−1 y i .
P
i=0

If we plug in x := λIn and y := A into the formula above, we get

 p−1 
λp In − Ap = (λIn − A) λp−i−1 Ai .
P
i=0

Taking the determinant of both sides, and keeping the multiplicative property of
determinants in mind (see Theorem 7.5.2), we get that
 p−1
P p−i−1 i 
det(λp In − Ap ) = det(λIn − A) det λ A .
i=0

Since Ap = On×n , we have that det(λp In − Ap ) = det(λp In ) = λpn . On the other

hand, by definition, we have that pA (λ) = det(λIn − A). So,
 p−1
P p−i−1 i 
λpn = pA (λ) det λ A ,
i=0

and it follows that pA (λ) | λpn .

Since pA (λ) is a polynomial of degree n with leading
coefficient 1, we deduce that pA (λ) = λn .

Proposition 8.6.26. Let V be a non-trivial, finite-dimensional vector space over a

field F, and let f : V → V be a linear function. Set n := dim(V ). Then the following
are equivalent:

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 607

(a) f is nilpotent;

(b) pf (λ) = λn ;

(c) f has only one eigenvalue, namely 0, and the algebraic multiplicity of this
eigenvalue is n.

Proof. Obviously, (b) and (c) are equivalent, and we just need to prove that (a)
and (b) are equivalent. For this, we simply “translate” Proposition 8.6.25 into
the language
  of linear functions, as follows. Let B be any basis of V , and set
B := B f B ; by Proposition 8.2.12, we have that pf (λ) = pB (λ). We note that
Theorem 4.5.3(c) and an easyinduction on p imply that for all non-negative integers
p, we have that B p = B f p B . We now have the following sequence of equivalent
statements:
f is nilpotent ⇐⇒ ∃p ∈ N s.t. f p is the zero function

fp
 
⇐⇒ ∃p ∈ N s.t. B B
= On×n

⇐⇒ ∃p ∈ N s.t. B p = On×n .

⇐⇒ B is nilpotent

(∗)
⇐⇒ pB (λ) = λn

⇐⇒ pf (λ) = λn ,

where (*) follows from Proposition 8.6.25.

We now introduce some terminology and notation. Suppose that f : V → V is a

nilpotent linear function, where V is a non-trivial, finite-dimensional vector space
over some field F. For a vector v ∈ V \ {0}, we define

Jf (v) := f r−1 (v), f r−2 (v), . . . , f 2 (v), f (v), v ,



where r is the smallest positive integer such that f r (v) = 0.86 Here, we have that
v ∈ Ker(f r ) \ Ker(f r−1 ), i.e. v is a generalized eignvector of rank r of the nilpotent
function f and associated with the eigenvalue 0, and Jf (v) is the Jordan chain
started by v. By Proposition 8.6.18(g), Jf (v) is a linearly independent set. Now, a
basis B of V is canonical with respect to the nilpotent linear function f if it is of the
form
B = Jf (u1 ) ∪ · · · ∪ Jf (uk ),
86
Such an r exists because f is nilpotent.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 608

where u1 , . . . , uk are some non-zero vectors in V , and Jf (u1 ), . . . , Jf (uk ) are pairwise
disjoint. So, a canonical basis of V associated with the nilpotent linear function
f is a basis that is the union of pairwise disjoint Jordan chains. Our goal is to
prove Theorem 8.6.28, which states that such a basis always exists. We begin with a
technical proposition, which readily implies Theorem 8.6.28.

Proposition 8.6.27. Let V be a non-trivial, finite-dimensional vector space over

a field F, let f : V → V be a nilpotent linear function, and let p be a positive
integer such that f p is the zero function, i.e. Ker(f p ) = V . Then for all vectors
v1 , . . . , vt ∈ V (t ≥ 0) that are linearly independent over Ker(f p−1 ), there exist non-
zero vectors vt+1 , . . . , vt+s ∈ V (s ≥ 0) such that Jf (v1 ), . . . , Jf (vt+s ) are pairwise
disjoint and B = Jf (v1 ), . . . , Jf (vt+s ) is a basis of V .

Remark: Note that the basis B is canonical with respect to f .

Proof. We may assume inductively that the proposition is true for non-trivial, finite-
dimensional vector spaces of dimension strictly smaller than dim(V ).
To simplify notation, for each non-negative integer r, we set Kr := Ker(f r ). So,
Kp = V . Let us first explain why we may assume that Kp−1 ⫋ V . Let q be the
smallest non-negative integer such that Kq = V (so, q ≤ p). Since K0 = {0} and V
is non-trivial, we know that K0 ⫋ V , and in particular, q ≥ 1. Now, suppose that
q < p. By Proposition 8.6.19, we then have that

K0 ⫋ K1 ⫋ K2 ⫋ . . . ⫋ Kq = Kq+1 = . . . ,
|{z} |{z}
={0} =V

and in particular, Kp−1 = Kp = V . So, by Proposition 8.6.22, if vectors v1 , . . . , vt ∈

V are linearly independent over Kp−1 , then t = 0, that is, v1 , . . . , vt is in fact an
empty list of vectors. Therefore, we just need to show that there exists a basis of V
that is canonical with respect to f . Thus, we may simply prove the proposition with
q instead of p, since any canonical basis of V that works for q also works for p.
In view of the discussion above, we assume from now on that Kp−1 ⫋ V . By
Proposition 8.6.19, we have that

K0 ⫋ K1 ⫋ K2 ⫋ . . . ⫋ Kp = Kp+1 = . . . ,
|{z} |{z}
={0} =V

Now, fix vectors v1 , . . . , vt ∈ V (t ≥ 0) that are linearly independent over Kp−1 .

In view of Proposition 8.6.22, we have that v1 , . . . , vt ∈ Kp \ Kp−1 .87 Fix any basis
Np−1 of Kp−1 . By Proposition 8.6.23, Np−1 ∪ {v1 , . . . , vt } is linearly independent,
and so by Theorem 3.2.19, it can be extended to a basis Np−1 ∪{v1 , . . . , vt , z1 , . . . , zℓ }
of V . Once again by Proposition 8.6.23, vectors v1 , . . . , vt , z1 , . . . , zℓ are linearly
87
Indeed, v1 , . . . , vt ∈ Kp because V = Kp . On the other hand, since v1 , . . . , vt are linearly
independent over Kp−1 , Proposition 8.6.22 guarantees that v1 , . . . , vt ∈
/ Kp−1 .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 609

independent over Kp−1 . So, we may now assume that ℓ = 0, for otherwise, we simply
consider vectors v1 , . . . , vt , z1 , . . . , zℓ instead of v1 , . . . , vt . With this set-up, Np−1 ∪
{v1 , . . . , vt } is a basis of V . Note that this implies that t = dim(V ) − dim(Kp−1 ).
Suppose first that p = 1, so that {0} = K0 ⫋ Ker(f ) = V .88 Then Jf (vi ) = {vi }
for all i ∈ {1, . . . , t}. Moreover, we have that Np−1 = ∅,89 and therefore, B =
{v1 , . . . , vt } is a basis of V .90 But note that B = Jf (v1 ) ∪ · · · ∪ Jf (vt ), and obviously,
the Jordan chains Jf (v1 ), . . . , Jf (vt ) are pairwise disjoint.91 So, B satisfies the
requirements from the statement of the proposition, and we are done.
From now on, we may assume that p ≥ 2. Note that this implies that {0} = K0 ⫋
Kp−1 ⫋ Kp = V , and in particular, by Theorem 3.2.21, 0 < dim(Kp−1 ) < dim(V ).
So, we will be able to apply the induction hypothesis to the vector space Kp−1 .
By Proposition 8.6.18(d), Kp−1 is f -invariant, and so g := f |Kp−1 is well defined
and obviously linear (because f is linear). Clearly, g p−1 is the zero function,92 and
in particular, g is nilpotent. Moreover, it is clear that for all i ∈ {0, . . . , p − 1}, we
have that Ker(g i ) = Ki .93
By Proposition 8.6.18(f), vectors f (v1 ), . . . , f (vt ) belong to Kp−1 , and by Propo-
sition 8.6.24, they are linearly independent over Kp−2 . We now apply the induc-
tion hypothesis applied to Kp−1 , the nilpotent linear function g, and the vectors
f (v1 ), . . . , f (vt ), and we deduce that there exist non-zero vectors vt+1 , . . . , vt+s ∈
Kp−1 (s ≥ 0) such that the Jordan chains



Jg f (v1 ) , . . . , Jg f (vt ) , Jg (vt+1 ), . . . , Jg (vt+s )

are pairwise disjoint and such that

C := Jg f (v1 ) ∪ · · · ∪ Jg f (vt ) ∪ Jg (vt+1 ) ∪ · · · ∪ Jg (vt+s )

is a basis of Kp−1 . We will show that

B := Jf (v1 ) ∪ · · · ∪ Jf (vt ) ∪ Jf (vt+1 ) ∪ · · · ∪ Jf (vt+s )

is the basis of V that we need.

88
Note that this means that f itself is a zero function.
89
This is because Np−1 is now a basis of Kp−1 = K0 = Ker(f 0 ) = Ker(IdV ) = {0}.
90
This is because Np−1 ∪ {v1 , . . . , vt } is a basis of V , and Np−1 = ∅.
91
Indeed, {v1 , . . . , vt } is a basis of V , and in particular, v1 , . . . , vt are pairwise distinct. So, the
Jordan chains Jf (vi ) = {vi } (i ∈ {1, . . . , t}) are pairwise disjoint.
92
Indeed, for any u ∈ Kp−1 , we have that
(∗) (∗∗)
g p−1 (u) = f p−1 (u) = 0,

where (*) follows from the fact that g = f |Kp−1 , and (**) follows from the fact that u ∈ Kp−1 .
93
This readily follows from the fact that g = f |Kp−1 , but here is a formal proof. Fix an index
i ∈ {0, . . . , p − 1}. Clearly, for all u ∈ Ker(g i ), we have that f i (u) = g i (u) = 0, and so u ∈ Ki . This
proves that Ker(g i ) ⊆ Ki . For the reverse inclusion, fix any u ∈ Ki . Since Ki ⊆ Kp−1 , we know
that u ∈ Kp−1 , and in particular, g i (u) is defined. But now g i (u) = f i (u) = 0, and we deduce that
u ∈ Ker(g i ). This proves that Ki ⊆ Ker(g i ).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 610

First of all, since g = f |Kp−1 , it is clear that for any u ∈ Kp−1 , we have that
Jg (u) = Jf (u). Therefore,



C = Jf f (v1 ) ∪ · · · ∪ Jf f (vt ) ∪ Jf (vt+1 ) ∪ · · · ∪ Jf (vt+s ),



and the Jordan chains Jf f (v1 ) , . . . , Jf f (vt ) , Jf (vt+1 ), . . . , Jf (vt+s ) are
pairwise
disjoint. Moreover, for all i ∈ {1, . . . , t}, we have that Jf (vi ) = Jf f (vi ) ∪ {vi }.
Since vectors v1 , . . . , vt are pairwise distinct (because they are linearly independent)
and do not belong to Kp−1 , whereas all vectors of C do belong to Kp−1 , we see that
the Jordan chains Jf (v1 ), . . . , Jf (vt ), Jf (vt+1 ), . . . , Jf (vt+s ) are pairwise disjoint.
Now, C is a basis of Kp−1 , and vectors v1 , . . . , vt are linearly independent over
Kp−1 . So, by Proposition 8.6.23, C ∪ {v1 , . . . , vt } is linearly independent. But B was
obtained from C ∪ {v1 , . . . , vt } by simply rearranging the elements of the ordered
set C ∪ {v1 , . . . , vt } (and placing v1 , . . . , vt in the appropriate places). So, B is
linearly independent. Moreover, |B| = |C| + t. Since t = dim(V ) − dim(Kp−1 ), and
since |C| = dim(Kp−1 ) (because C is a basis of Kp−1 ), we have that |B| = dim(V ).
Corollary 3.2.20(a) now guarantees that B is a basis of V . This completes the
argument.

Theorem 8.6.28. Let V be a non-trivial, finite-dimensional vector space over a

field F, and let f : V → V be a nilpotent linear function. Then V has a basis that is
canonical with respect to f .

Proof. This follows immediately from Proposition 8.6.27, as follows. Since f is

nilpotent, there exists a positive integer p such that f p is the zero function. We
now apply Proposition 8.6.27 to f , p, and the empty list of vectors (i.e. t = 0),
and we deduce that there exist vectors v1 , . . . , vs ∈ V such that the Jordan chains
Jf (v1 ), . . . , Jf (vs ) are pairwise disjoint, and such that B = Jf (v1 ) ∪ · · · ∪ Jf (vs ) is
a basis of V . By definition, this basis B is canonical with respect to f .

Theorem 8.6.29. Let V be a non-trivial, finite-dimensional vector space over a

 →V be a nilpotent linear function. Then
field F, and set n := dim(V ). Let f : V
there exists a basis B of V such that B f B is a Jordan matrix.

Proof. Using Theorem 8.6.28, we fix a canonical basis B of V with respect to f . By

the definition of a canonical basis, there exist non-zero vectors u1 , . . . , uk ∈ V such
that the Jordan chains Jf (u1 ), . . . , Jf (uk ) are pairwise disjoint and such that

B = Jf (u1 ) ∪ · · · ∪ Jf (uk ).
 
Our goal is to show that B f B is a Jordan matrix.
For each i ∈ {1, . . . , k}, let ai be the positive integer for which ui ∈ Ker(f ai ) \
Ker(f ai −1 ), so that
Jf (ui ) = f ai −1 (ui ), f ai −2 (ui ), . . . , f 2 (ui ), f (ui ), ui , and set
Ui := Span Jf (ui ) , so that Ji (ui ) is a basis of Ui . We will show that B f B =
Ja1 (0) ⊕ · · · ⊕ Jak (0).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 611

Claim. V = U1 ⊕ · · · ⊕ Uk . Moreover, subspaces U1 , . . . , Uk of V are

f -invariant.

Proof of the Claim. The fact that V = U1 ⊕ · · · ⊕ Uk follows immediately from

Proposition 8.6.10. It remains to show that U1 , . . . , Uk are f -invariant. Fix an index
i ∈ {1, . . . , k} and a vector u ∈ Ui . Then there exist scalars α0 , . . . , αai −1 ∈ F such
that
i −1
aP
u = αj f j (ui ).
j=0

By applying f to both sides of the equation, and by using the linearity of f , we

obtain
i −1
aP
f (u) = αj f j+1 (ui ).
j=0

But note that f (ai −1)+1 (ui ) = f ai (ui ) = 0, and so we in fact have that

i −2
aP i −1
aP
f (u) = αj f j+1 (ui ) = αj−1 f j (ui ).
j=0 j=1

So, f (u) ∈ Span f a1 −1 (u1 ), . . . , f 2 (u1 ), f (u1 ) ⊆ Span Jf (ai ) = Ui . ♦

In view of the Claim, for each i ∈ {1, . . . , k}, we may define fi := f |Ui . By the
Claim and Proposition 8.6.11, we have that
     
B
f B
= J (u )
f1 J (u )
⊕ · · · ⊕ J (u )
fk J (u )
.
f 1 f 1 f k f k

On the other hand, for all indices i ∈ {1, . . . , k}, we have the following:
 
J (u )
fi J (u )
f i f i

(∗)
h  i
fi f ai −1 (ui ) J fi f ai −2 (ui ) J

 
 

= ... fi ui Jf (ui )
f (ui ) f (ui )

h  i
f ai (ui ) f ai −1 (ui )
    
= Jf (ui ) Jf (ui )
... f (ui ) Jf (ui )

(∗∗)  
= 0 e1 . . . eai −1 = Jai (0),

where (*) follows from Theorem 4.5.1, and where in (**), e1 , . . . , eai are the standard
basis vectors of Fai . It now follows that
     
B
f B = J (u ) f1 J (u ) ⊕ · · · ⊕ J (u ) fk J (u )
f 1 f 1 f k f k

= Ja1 (0) ⊕ · · · ⊕ Jak (0),

which completes the argument.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 612

The existence part of Theorems 8.6.2 and 8.6.4. We are now ready to prove
the main result of this section. We first prove the existence part of Theorem 8.6.4
(see Theorem 8.6.30 below); as we shall see, it readily follows from Theorems 8.6.21
and 8.6.29. Corollary 8.6.31 readily follows from Theorem 8.6.30, and it constitutes
the existence part of Theorem 8.6.2.
Theorem 8.6.30. Let V be a non-trivial, finite-dimensional vector space over an
algebraically closed field F, and let f : V → V be a linear function. Then there
 
exists a basis B of V such that B f B is a Jordan matrix.
Proof. Set n := dim(V ), and let

λ1 , . . . , λ 1 , . . . , λ k , . . . , λ k
| {z } | {z }
m1 mk

be the spectrum of f , where λ1 , . . . , λk are pairwise distinct eigenvalues of f and

m1 , . . . , mk are positive integers. Since the field F is algebraically closed, we know
that m1 + · · · + mk = n.
By Theorem 8.6.21, we know that V = Gλ1 (f ) ⊕ · · · ⊕ Gλk (f ), and that for all
indices i ∈ {1,
. . . , k}, the generalized eigenspace Gλi (f ) is f -invariant and satisfies
dim Gλi (f ) = mi . To simplify notation, for each index i ∈ {1, . . . , k}, we set
Gi := Gλi (f ) and fi := f |Gi . Moreover, for each i ∈ {1, . . . , k}, we let pi be the
smallest positive integer such that

Gi = Ker (f − λi IdV )pi = Ker (fi − λi IdGi )pi .

Now, for each index i ∈ {1, . . . , k}, we proceed as follows. Obviously, the linear
function fi − λi IdGi is nilpotent.94 Using Theorem  8.6.29, for each i ∈ {1, . . . , k},
we fix a basis Bi such that Ji := B fi − λi IdGi B is a matrix in Jordan normal
i i
form; by Theorem 8.6.29, we know that all the Jordan blocks of the Jordan matrix
Ji are of the form Jt (0) for some positive integer t. But now
  (∗)     (∗∗)
B
f B
= B
f − λi IdGi B
+ λi B IdGi B = Ji + λi Imi ,

where (*) follows from Theorem 4.5.3, and (**) follows from the definition of Ji and
from formula from Theorem
 4.5.1. Since the matrix Ji is in Jordan normal form,
so is the matrix B f B , and moreover, we see that each Jordan block Jt (0) of the
former corresponds to a Jordan block Jt (λi ) of the latter.
Now, set B := B1 ∪ · · · ∪ Bk . Then by Proposition 8.6.11, we have that
     
B
f B = B f1 B ⊕ · · · ⊕ B fk B .
1 1 k k
     
Since B1
f1 B1
,...,B fk Bk
are Jordan matrices, so is B
f B
.
k
94 pi pi
Indeed, for all v ∈ Gi , we have that (fi − λi IdGi ) (v) = (f − λi IdV ) (v) = 0, and it follows
that (fi − λi IdGi )pi is a zero function. So, fi − λi IdGi is nilpotent.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 613

Corollary 8.6.31. Let F be an algebraically closed field, and let A ∈ Fn×n be a

square matrix. Then A is similar to a matrix in Jordan normal form.
Proof. Define fA : Fn → Fn by setting fA (u) = Au for all u ∈ Fn ; then fA is linear

(by Proposition 1.10.4), and clearly, its standard matrix is A, i.e. A = E fA E ,
n n
where En is the standard basis of Fn . By Theorem 8.6.30, there exists a basis
B of Fn such that the matrix J := B fA B is a Jordan matrix. Finally, by

   
Theorem 4.5.19, matrices A = E fA E and J = B fA B are similar. This
n n
completes the argument.

8.6.6 The proof of Theorems 8.6.2, 8.6.4, and 8.6.6

We are finally ready to prove Theorems 8.6.2, 8.6.4, and 8.6.6, restated below for the
reader’s convenience.
Theorem 8.6.2. Assume that F is an algebraically closed field, and let A ∈ Fn×n
be a square matrix. Then A is similar to a matrix J in Jordan normal form. Moreover,
this matrix J is unique up to a reordering of the Jordan blocks.
Proof. The existence part follows from Corollary 8.6.31. Uniqueness follows from
Theorem 8.6.1, since any two Jordan matrices that are similar to A are (by Proposi-
tion 4.5.16) similar to each other.

Theorem 8.6.4. Let V be a non-trivial, finite-dimensional vector space over an

algebraically closed field F, and let f : V → V be a linear function. Then there
 
exists a basis B such that the matrix B f B is in Jordan normal form. Moreover,
this matrix
  the following sense: if B1 and B2 are bases of V such that
is unique in
both B f B and B f B are in Jordan normal form, then these two matrices
1 1 2 2
are the same up to a reordering of the Jordan blocks.
Proof. The existence part follows immediately from Theorem 8.6.30. The uniqueness
part is a consequence of Theorem 8.6.1,
 as
 we now explain.
  Suppose that B1 and
B2 are bases of V such that both B f B and B f B are in Jordan normal
1 1 2 2
form. By Theorem 4.5.19, these two matrices are similar. But now Theorem 8.6.1
guarantees that these two matrices have exactly the same Jordan blocks (counting
repetitions).

Theorem 8.6.6. Let F be an algebraically closed field, let A ∈ Fn×n , and let

λ1 , . . . , λ 1 , . . . , λ k , . . . , λ k
| {z } | {z }
m1 mk

be the spectrum of A, where λ1 , . . . , λk are pairwise distinct eigenvalues of A and

m1 , . . . , mk are positive integers.95 Then A is similar to a matrix J ∈ Fn×n in
Jordan normal form that has the following properties:
95
Since F is algebraically closed, we know that m1 + · · · + mk = n.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 614

(i) each Jordan block of the Jordan matrix J is of the form Jt (λi ) for some
i ∈ {1, . . . , k} and t ∈ {1, . . . , mi };
(ii) for each i ∈ {1, . . . , k} and each positive integer r, the Jordan matrix J has
exactly
rank (A − λi In )r−1 − rank (A − λi In )r

many Jordan blocks Jt (λi ) satisfying t ≥ r.

Moreover, A is similar to any Jordan matrix in Fn×n that satisfies conditions (i)
and (ii) above.
Proof. By Theorem 8.6.2, A is similar to a matrix J in Jordan normal form. The fact
that the matrix J satisfies (i) and (ii) follows immediately from Proposition 8.6.17.
On the other hand, it is clear that any Jordan matrix in Fn×n that satisfies (i)
and (ii) has exactly the same Jordan blocks as our Jordan matrix J, and is therefore
(by Theorem 8.6.1) similar to J. Since matrix similarity in Fn×n is an equivalence
relation (by Proposition 4.5.16), it follows that any Jordan matrix in Fn×n that
satisfies (i) and (ii) is indeed similar to A.

8.6.7 Computing the Jordan normal form of a square matrix (once

more): computing both J and P
Generalied eigenvectors and generalized eigenspaces of square matrices.
In what follows, we will need to adapt to matrices some of the terminology and
notation that we originally introduced for linear functions. For a field F, a square
matrix A ∈ Fn×n , and an eigenvalue λ of A, we define the following:
ˆ a generalized eigenvector of A associated with λ is any non-zero vector v ∈ Fn
such that for some positive integer r, we have that (A − λIn )r v = 0;96
ˆ the rank of a generalized eigenvector v of A associated with λ is the smallest
positive integer r such that (A − λIn )r v = 0;97
ˆ for a generalized eigenvector v of A associated with λ and of rank r, the Jordan
chain started by v is the set
(A − λIn )r−1 v, . . . , (A − λIn )2 v, (A − λIn )v, v ;


ˆ the generalized eigenspace of A associated with λ is the set

Gλ (A) := v ∈ V | ∃r ∈ N0 s.t. (A − λIn )r v = 0


∞
S 
r 
= Nul A − λIn ,
r=0
96
The reason we specify that r is positive (rather than merely non-negative) is simply that
(A − λIn )0 = In , and so there are no non-zero vectors v ∈ Fn such that (A − λIn )0 v = 0.
97
Note that this means that v ∈ Nul(Ar ) \ Nul(Ar−1 ).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 615

i.e. the set whose elements are precisely the generalized eigenvectors of A
associated with λ, plus the vector 0.

Let fA : Fn → Fn given by fA (v) = Av, so that fA is the linear function whose

standard matrix is A. So, all the results from subsection 8.6.5 that apply to the linear
function fA can be translated into results for the matrix A (we note, however, that
some of those results only hold when F is an algebraically closed field). Importantly,
if F is an algebraically closed field, then Theorem 8.6.21 guarantees that Fn is the
direct sum of the generalized eigenspaces of A. Moreover, by Proposition 8.6.18(g),
all Jordan chains are linearly independent.

Computing J and P . Suppose that F is an algebraically closed field, and that

A ∈ Fn×n is a square matrix. We saw in subsection 8.6.2 how Theorem 8.6.6 can
be used to compute the Jordan normal form of A. Suppose we would like to do
more: we would like to compute both a Jordan matrix J ∈ Fn×n and an invertible
matrix P ∈ Fn×n such that J = P −1 AP . Here, we give a recipe for doing precisely
that. We will not give a fully formal proof of the correctness of our recipe, except
to note that it essentially follows from our proof of Theorems 8.6.2 and 8.6.4 (see
subsections 8.6.4, 8.6.5, and 8.6.6). The steps are as follows.

1. Compute the characteristic polynomial pA (λ) of A, and factor this polynomial

into linear terms.98

2. Compute the spectrum


λ1 , . . . , λ1 , . . . , λk , . . . , λk
| {z } | {z }
m1 mk

of A, where λ1 , . . . , λk ∈ F are pairwise distinct and m1 , . . . , mk are positive

integers.

3. For each index i ∈ {1, . . . , k}, we process the eigenvalue λi as follows.

(a) We form the matrix Ai := A − λi In , and we compute the matrices Ari

and their ranks for non-negative integers r. We keep
computing until the
rank becomes n − mi , that is, until we get rank Api i = n − mi for some
positive integer pi . For this pi , we will have

Nul(A0i ) ⫋ Nul(A1i ) ⫋ . . . ⫋ Nul(Api i ) = Nul(Api i +1 ) = . . . ,

| {z } | {z }
={0} =Gλi (A)

98
The fact that pA (λ) can be factored into linear terms follows from the fact that the field F is
algebraically closed. Of course, we only know that such a factorization exists and do not have an
actual formula/algorithm for finding it. If we get stuck factoring pA (λ), then we are well and truly
stuck: we will not be able to compute the matrices J and P that we need.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 616

and we will further have rank A0i > rank A1i > · · · > rank Api i −1 >

rank Api i = rank Api i +1 = rank Api i +2 = . . . ,99 and in particular,

rank Api i = rank Api i +1 will be the first instance of “repeating rank.”100

ˆ The largest Jordan block of J associated with the eigenvalue λi will

be Jpi (λi ), and there may possibly be more than one copy of this
block in J.
(b) For each r ∈ {0, 1, . . . , pi }, we compute a basis Ni,r of Nul(Ari ).101
(c) We compute the Jordan blocks Jt (λi ) of the Jordan matrix J using the
formula from Theorem 8.6.6, as in subsection [Link]
(d) First, we deal with the Jordan blocks of the form Jpi (λi ) of J. Each
of these blocks will correspond to a Jordan chain Api i −1 u, . . . , Ai u, u


associated with A and λi ; the goal is to find the vectors u that start
these Jordan chains (one vector u per Jordan block Jpi (λi ) in J). We
proceed as follows. We extend the basis Ni,pi −1 of Nul(Api i −1 ) to a basis
of Nul(Api i ). For this, we can use Proposition 3.3.21 (the needed matrix
is formed by the vectors of Ni,pi −1 and of Ni,pi , with a vertical dotted
line placed between them). Each vector u that we added to Ni,pi −1 to
form our basis of Nul(Api i ) starts a Jordan chain Api i −1 u, . . . , Ai u, u


corresponding to one of the Jordan blocks Jpi (λi ).

(e) If J contains no Jordan blocks associated with λi other than blocks Jpi (λi ),
then we are done processing the eigenvalue λi . Otherwise, we proceed as
follows.
(f) Suppose that t < pi , that J has at least one Jordan block Jt (λi ), and that
we have already dealt with Jordan blocks associated with λi and of size
99
Let us justify this
(and in particular, explain why such a pi exists). By Proposition 8.6.21, we
have that dim Gλi (A) = mi . Moreover, Proposition 8.6.21 guarantees that there exists a positive
integer pi such that

Nul(A0i ) ⫋ Nul(A1i ) ⫋ ... ⫋ Nul(Api i ) = Nul(Api i +1 ) = ...

| {z } | {z }
={0} =Gλ (A)
i


  
For this pi , we have that dim Nul Api i = dim Gλi (A) = mi , and so by the rank-nullity theorem,


rank Api i = n − mi . On the other hand, Theorem 3.2.21 guarantees that 0 = dim Nul A0i


<
 
pi

, and consequently (by the rank-nullity theorem), n = rank A0i > · · · >


· · · < dim Nul Ai
pi

that Nul Ari = Nul Api i , and consequently (by



rank Ai = pi − mi . For integers r
> pi , we have
the rank-nullity theorem), rank Ari = rank Api i = n − mi .


100
In subsection 8.6.2, our recipe said that we should keep computing until we get “repeating rank”
for the first time. Here, we give a bit of a shortcut: we keep computing until we get rank n − mi for
the first time, and we are guaranteed that the rank will start repeating itself after that.
101
Note that A0i = In , and consequently, Nul(A0i ) = Nul(In ) = {0} and Ni,0 = ∅.
102
Since we are currently processing the eigenvalue λi , we are only interested in the Jordan blocks
associated with this particular eigenvalue. The Jordan blocks associated with the other eigenvalues
are computed when we process those other eigenvalues.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 617

greater than t × t. We now take the basis Ni,t−1 of Nul At−1

i , and we
add to it all the generalized eigenvectors of rank t of A associated with λi
that come from the Jordan chains that we have already generated.103 Let
−
us call the resulting set Ni,t . This is a linearly independent set of vectors
t −
to a basis of Nul(Ati )


in Nul Ai . Using Proposition 8.6.19, we extend Ni,t
−
(the columns of the needed matrix are the vectors of Ni,t and the vectors
−
of Ni,t , with a vertical dotted line placed between the vectors of Ni,t and
−
the vectors of Ni,t ). Each vector u that we added to Ni,t to create a basis
of Nul(Ati ) starts a Jordan chain At−1

i u, . . . , Ai u, u that corresponds to
one of our Jordan blocks Jt (λi ). We continue this process until we have
created a Jordan chain for each Jordan block of J associated with λi .104

4. We form J as the direct sum of the Jordan blocks that we have computed
103
Let us be more precise. Suppose we have already generated a Jordan chain As−1 i u, . . . , Ai u, u
for some s > t. Then As−ti u is the unique generalized eigenvector of rank t of A associated with the
eigenvalue λi that comes from this Jordan chain. We select such a generalized eigenvector of rank t
out of each Jordan chain associated with A and λi that we have already generated.
104
Let us consider an example. Suppose that we have determined (using Theorem 8.6.6) that the
Jordan blocks of J associated with the eigenvalue λi are precisely

J18 (λi ), J9 (λi ), J9 (λi ), J9 (λi ), J5 (λi ), J5 (λi ), J3 (λi ), J1 (λi ), J1 (λi ),

counting repetitions. Suppose that we have already generated the needed Jordan chains that
correspond to the Jordan blocks J18 (λi ), J9 (λi ), J9 (λi ), J9 (λi ), and that we are currently trying to
generate the Jordan chains that correspond to the Jordan blocks J5 (λi ), J5 (λi ). Suppose that the
Jordan chains that we have already generated are the following:
ˆ the Jordan chain {A17 16
i u1 , Ai u1 , . . . , Ai u1 , u1 } corresponding to the Jordan block J18 (λi ),
ˆ the Jordan chain {Ai w1 , Ai w1 , . . . , Ai w1 , w1 } corresponding to the first Jordan block J9 (λi ),
8 7

ˆ the Jordan chain {A8i w2 , A7i w2 , . . . , Ai w2 , w2 } corresponding to the second Jordan block
J9 (λi ),
ˆ the Jordan chain {A8i w3 , A7i w3 , . . . , Ai w3 , w3 } corresponding to the third Jordan block J9 (λi ).
Since we are now processing the Jordan blocks J5 (λi ), J5 (λi ), we first to need to identify the
generalized eigenvectors of rank 5 of the matrix A associated with the eigenvalue λi from the four
Jordan chains above. These are the vectors A13 4 4 4
i u1 , Ai w1 , Ai w2 , Ai w3 . Now we form the set

−
:= Ni,4 ∪ A13 4 4 4

Ni,5 i u1 , Ai w1 , Ai w2 , Ai w3 .

−
The set Ni,5 is linearly independent in Nul(A5i ), and so using Proposition 3.3.21, we can extend it
−
to a basis of Nul(A5i ) (the columns of the needed matrix are the vectors of Ni,5 , followed by the
−
vectors of Ni,5 , with a vertical dotted line between the vectors of Ni,5 and the vectors of Ni,5 ).
Because we have two Jordan blocks J5 (λi ), J5 (λi ), we will need to add exactly two vectors, say
−
z1 , z2 , to Ni,5 in order to obtain a basis of Nul(A5i ). (Vectors z1 , z2 will be the vectors of Ni,5 that
Proposition 3.3.21 gives us. More precisely, they will be the the pivot columns to the right of the
vertical dotted line of the matrix that we formed when we used Proposition 3.3.21.) Now z1 , z2 are
generalized eigenvectors of rank 5 of A associated with the eigenvalue λi , and they will start the
Jordan chains of length five corresponding to our two Jordan blocks J5 (λi ), J5 (λi ). So, the Jordan
chains that correspond to our two Jordan blocks J5 (λi ), J5 (λi ) are {A4i z1 , A3i z1 , A2i z1 , Ai z1 , z1 } and
{A4i z2 , A3i z2 , A2i z2 , Ai z2 , z2 }.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 618

(associated with all the eigenvalues of A), and we form P using the corresponding
Jordan chains. We make sure that the Jordan blocks of J and the Jordan
chains in P are placed in a corresponding order.

Some numerical examples. We now revisit Examples 8.6.8 and 8.6.9 from
subsection 8.6.2. In each case, we are given a square matrix A with entries in C. In
Examples 8.6.8 and 8.6.9, we computed the Jordan normal form of the matrix in
question. We will now compute both a Jordan matrix J and an invertible matrix P
such that J = P −1 AP .

Example 8.6.32. Consider the following matrix in C10×10 :

3 1 0 0 0 0 0 −1 0 1
 

 −3 1 5 2 −2 −4 −7 4 −1 3 


 0 1 3 0 0 0 0 −1 0 1 


 −2 −1 3 4 −1 −2 −3 2 −1 2 

 −1 0 2 1 2 −2 −1 1 0 1 
A :=  .

 −1 0 1 0 0 2 −1 0 0 1 


 1 1 −2 −1 1 2 7 −2 1 −2 


 −1 0 1 0 0 −1 0 3 1 0 

 0 0 0 0 0 0 1 0 3 −1 
1 1 −2 −1 1 2 5 −2 1 0

Compute a Jordan matrix J and an invertible matrix P , both in C10×10 , such that
J = P −1 AP .

Remark: This is the matrix from Example 8.6.8. In that example, we computed
the matrix J. Here, we will see how to compute the matrix P .

Solution. We first compute the characteristic polynomial of A, and we factor it into

linear terms:

pA (λ) = det λI10 − A = (λ − 3)8 (λ − 2)2

The eigenvalues of A are λ1 = 3 (with algebraic multiplicity 8) and λ2 = 2 (with

algebraic multiplicity 2). We handle the two eigenvalues separately.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 619

The eigenvalue λ1 = 3. To simplify notation, we write

0 1 0 0 0 0 0 −1 0 1
 

 −3 −2 5 2 −2 −4 −7 4 −1 3 


 0 1 0 0 0 0 0 −1 0 1 


 −2 −1 3 1 −1 −2 −3 2 −1 2 

 −1 0 2 1 −1 −2 −1 1 0 1 
A1 := A − λ1 I10 =  .

 −1 0 1 0 0 −1 −1 0 0 1 


 1 1 −2 −1 1 2 4 −2 1 −2 


 −1 0 1 0 0 −1 0 0 1 0 

 0 0 0 0 0 0 1 0 0 −1 
1 1 −2 −1 1 2 5 −2 1 −3

We proceed as follows. We compute the matrices Ar1 for r = 0, 1, 2, 3, . . . , and

we simultaneously compute their ranks. Our matrix A is of size 10 × 10, and the
algebraic multiplicity of the eigenvalue λ1 is 8. So, we
will keep computing until
we reach the first positive integer p for which rank Ap1 = 10 − 8 = 2. For this
p, we will have that Nul Ap1 = Nul Ap+1 = Nul Ap+2




1 1 = . . . , and consequently,
p
p+1
p+2

rank A1 = rank A1
= rank A1 = . . . . For r = 0, 1, . . . , p, we also compute
a basis N1,r of Nul Ar1 . Here is our computation.
r = 0: A01 = I10 , rank A01 = 10. A basis of Nul A01 is N1,0 := ∅.

r = 1:
0 1 0 0 0 0 0 −1 0 1
 

 −3 −2 5 2 −2 −4 −7 4 −1 3 


 0 1 0 0 0 0 0 −1 0 1 


 −2 −1 3 1 −1 −2 −3 2 −1 2 

 −1 0 2 1 −1 −2 −1 1 0 1 
A11 =  .

 −1 0 1 0 0 −1 −1 0 0 1 


 1 1 −2 −1 1 2 4 −2 1 −2 


 −1 0 1 0 0 −1 0 0 1 0 

 0 0 0 0 0 0 1 0 0 −1 
1 1 −2 −1 1 2 5 −2 1 −3

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 620

By row reducing, we obtain

1 0 0 1 −1 0 0 1 0 0
 

 0 1 0 0 0 0 0 −1 0 0 


 0 0 1 1 −1 0 0 1 0 0 


 0 0 0 0 0 1 0 0 0 0 

1
 0 0 0 0 0 0 1 0 0 0 
RREF(A1 ) =  ,

 0 0 0 0 0 0 0 0 1 0 


 0 0 0 0 0 0 0 0 0 1 


 0 0 0 0 0 0 0 0 0 0 

 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0

and we see that rank A11 = 7. The following is a basis of Nul A11 :

−1 1 −1
     

 0  
  0  
  1 


 −1  
  1  
  −1 

 1   0   0 
n
 0
 
  1
 
  0
o

N1,1 :=  , ,  .

 0  
  0  
  0 


 0  
  0  
  0 


 0  
  0  
  1 

 0   0   0 
0 0 0

r = 2:
−1 −1 2 1 −1 −1 −2 2 −1 0
 

 0 0 0 0 0 0 0 0 0 0 


 −1 −1 2 1 −1 −1 −2 2 −1 0 


 1 1 −2 −1 1 2 4 −2 1 −2 

 0 0 0 0 0 1 1 0 0 −1 
A21 =  

 1 0 −1 0 0 1 2 0 0 −2 


 0 0 0 0 0 0 0 0 0 0 


 1 0 −1 0 0 1 2 0 0 −2 

 0 0 0 0 0 0 −1 0 0 1 
0 0 0 0 0 0 −1 0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 621

By row reducing, we obtain

1 0 −1 0 0 0 0 0 0 0
 

 0 1 −1 −1 1 0 0 −2 1 2 


 0 0 0 0 0 1 0 0 0 0 


 0 0 0 0 0 0 1 0 0 −1 

2
 0 0 0 0 0 0 0 0 0 0 
RREF(A1 ) =  ,

 0 0 0 0 0 0 0 0 0 0 


 0 0 0 0 0 0 0 0 0 0 


 0 0 0 0 0 0 0 0 0 0 

 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0

and we see that rank A21 = 4. The following is a basis of Nul A11 :

1 0 0 0 0 0
           

 1  
  1  
  −1  
  2  
  −1  
  −2 


 1  
  0  
  0  
  0  
  0  
  0 

 0   1   0   0   0   0 
n
 0
 
  0
 
  1
 
  0
 
  0
 
  0
o

N1,2 :=  , , , , ,  .

 0  
  0  
  0  
  0  
  0  
  0 


 0  
  0  
  0  
  0  
  0  
  1 


 0  
  0  
  0  
  1  
  0  
  0 

 0   0   0   0   1   0 
0 0 0 0 0 1

r = 3:
−1 0 1 0 0 −1 −3 0 0 3
 

 0 0 0 0 0 0 0 0 0 0 


 −1 0 1 0 0 −1 −3 0 0 3 


 0 0 0 0 0 0 0 0 0 0 

 −1 0 1 0 0 −1 −2 0 0 2 
A31 =  

 −1 0 1 0 0 −1 −3 0 0 3 


 0 0 0 0 0 0 0 0 0 0 


 −1 0 1 0 0 −1 −3 0 0 3 

 0 0 0 0 0 0 1 0 0 −1 
0 0 0 0 0 0 1 0 0 −1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 622

By row reducing, we obtain:

1 0 −1 0 0 1 0 0 0 0
 

 0 0 0 0 0 0 1 0 0 −1 


 0 0 0 0 0 0 0 0 0 0 


 0 0 0 0 0 0 0 0 0 0 

3
 0 0 0 0 0 0 0 0 0 0 
RREF(A1 ) =  ,

 0 0 0 0 0 0 0 0 0 0 


 0 0 0 0 0 0 0 0 0 0 


 0 0 0 0 0 0 0 0 0 0 

 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0

and we see that rank(A31 ) = 2. The following is basis of Nul A31 :

0 1 0 0 −1 0 0 0
               

 1  
  0  
  0  
  0  
  0  
  0  
  0  
  0 


 0  
  1  
  0  
  0  
  0  
  0  
  0  
  0 

 0   0   1   0   0   0   0   0 
n
 0
 
  0
 
  0
 
  1
 
  0
 
  0
 
  0
 
  0
o

N1,3 :=  , , , , , , ,  .

 0  
  0  
  0  
  0  
  1  
  0  
  0  
  0 


 0  
  0  
  0  
  0  
  0  
  0  
  0  
  1 


 0  
  0  
  0  
  0  
  0  
  1  
  0  
  0 

 0   0   0   0   0   0   1   0 
0 0 0 0 0 0 0 1

Since rank A31 = 2,105 we know that rank A31 = rank A41 = rank A51 = . . . , and

in particular, we can now stop computing powers of A1 .

Next, we compute the Jordan blocks of our Jordan matrix J that correspond to
the eigenvalue λ1 . This is done precisely as in Example 8.6.8 (using Theorem 8.6.6).
In Example 8.6.8, we saw that J contains two Jordan blocks J3 (λ1 ) = J3 (3) and one
Jordan block J2 (λ1 ) = J2 (3), and that it contains no other Jordan blocks associated
with the eigenvalue λ1 = 3. We now need to generate the Jordan chains corresponding
to these Jordan blocks.
We first generate the Jordan chains corresponding to the two Jordan blocks
J3 (λ1 ). (Note that each of these Jordan chains will contain three vectors and will
2 u, A u, u}.) We extend our basis N 2 to a basis of


be of the form {A 1 1 1,2 of Nul A 1
Nul A31 . We use Proposition 3.3.21. We form the matrix whose columns are the

vectors of N1,2 , followed by the vectors of N1,3 , with a vertical dotted line between
105
Recall that A is of size 10 × 10, whereas the eigenvalue λ1 is of algebraic multiplicity 8. So, we
needed to compute powers of A1 until we got rank 10 − 8 = 2.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 623

the vectors of N1,2 and the vectors of N1,3 . The matrix we obtain is the following:

1 0 0 0 0 0 0 1 0 0 −1 0 0 0
 

 1 1 −1 2 −1 −2 1 0 0 0 0 0 0 0 


 1 0 0 0 0 0 0 1 0 0 0 0 0 0 


 0 1 0 0 0 0 0 0 1 0 0 0 0 0 


 0 0 1 0 0 0 0 0 0 1 0 0 0 0 
.

 0 0 0 0 0 0 0 0 0 0 1 0 0 0 


 0 0 0 0 0 1 0 0 0 0 0 0 0 1 


 0 0 0 1 0 0 0 0 0 0 0 1 0 0 

 0 0 0 0 1 0 0 0 0 0 0 0 1 0 
0 0 0 0 0 1 0 0 0 0 0 0 0 1

By row reducing this matrix, we see that its pivot columns are all its columns to
the left of the vertical dotted line, plus the first and fifth column to
the right of the
vertical dotted line. So, N1,2 can be extended to a basis of Nul A31 by adding the
first and fifth vector of the basis N1,3 to it. The first and fifth vector of N1,3 are the
following vectors:
T
ˆ u1 := 0 1 0 0 0 0 0 0 0 0 ;


T
ˆ u2 :=

−1 0 0 0 0 1 0 0 0 0 .

These two vectors will start our first two Jordan chains. More precisely, one of our
Jordan blocks J3 (λ1 ) corresponds to the Jordan chain {A21 u1 , A1 u1 , u1 }, whereas
the other Jordan block J3 (λ1 ) corresponds to the Jordan chain {A21 u2 , A1 u2 , u2 }.
If we compute the values of all these vectors and arrange them into a matrix, we
obtain the following (the vertical dotted line separates the two Jordan chains):

−1 1 0 0 0 −1
 

 0 −2 1 0 −1 0 


 −1 1 0 0 0 0 


 1 −1 0 1 0 0 

 0 0 0 1 −1 0 
A21 u1 A1 u1 u1 A21 u2 A1 u2
 
u2 =  .

 0 0 0 0 0 1 


 0 1 0 0 1 0 


 0 0 0 0 0 0 

 0 0 0 0 0 0 
0 1 0 0 1 0

It remains to compute a Jordan chain that corresponds to the Jordan block J2 (λ1 ).
(This Jordan chain will contain two vectors, and it will be of the form {A1 u, u}.)
We first identify the generalized eigenvectors of rank 2 of A associated with the
eigenvalue λ1 inside the Jordan chains (associated with λ1 ) that we have already

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 624

−
created. These are the vectors A1 u1 and A1 u2 . Now, N1,2 := N1,1 ∪ {A1 u1 , A1 u2 } is
2


a linearly independent set of vectors in Nul A 1 , and we would like to extend it to a
2


basis of Nul A1 . We once again use Proposition 3.3.21. We form the matrix whose
−
columns are the vectors of the linearly independent set N1,2 = N1,1 ∪ {A1 u1 , A1 u2 },
followed by the vectors of N1,2 , with a vertical dotted line between the vectors of
the two sets. The matrix that we obtain is the following:

−1 1 −1 1 0 1 0 0 0 0 0
 

 0 0 1 −2 −1 1 1 −1 2 −1 −2 


 −1 1 −1 1 0 1 0 0 0 0 0 


 1 0 0 −1 0 0 1 0 0 0 0 


 0 1 0 0 −1 0 0 1 0 0 0 
.

 0 0 0 0 0 0 0 0 0 0 0 


 0 0 0 1 1 0 0 0 0 0 1 


 0 0 1 0 0 0 0 0 1 0 0 

 0 0 0 0 0 0 0 0 0 1 0 
0 0 0 1 1 0 0 0 0 0 1

By row reducing, we see that the pivot columns of this matrix are all the columns to
the left of the vertical dotted line, plus the fifth column to the right of the vertical

−
dotted line. So, N1,2 = N1,1 ∪ {A1 u1 , A1 u2 } can be extended to a basis of Nul A21
by adding the fifth vector of N1,2 to it. The fifth vector of N1,2 is the following:
T
ˆ u3 = 0 −1 0 0 0 0 0 0 1 0 .


This vector starts our Jordan chain. More precisely, our Jordan block J2 (λ1 ) corre-
sponds to the Jordan chain {A1 u3 , u3 }. By computing and placing this chain into a
matrix, we obtain:
−1 0
 
 1 −1 
 
 −1 0 
 
 0 0 
 
   0 0 
A1 u3 u3 =   0
.
 0 

 0 0 
 
 1 0 
 
 0 1 
0 0

We are now done processing the eigenvalue λ1 = 3.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 625

The eigenvalue λ2 = 2. To simplify notation, we write

1 1 0 0 0 0 0 −1 0 1
 

 −3 −1 5 2 −2 −4 −7 4 −1 3 


 0 1 1 0 0 0 0 −1 0 1 


 −2 −1 3 2 −1 −2 −3 2 −1 2 

 −1 0 2 1 0 −2 −1 1 0 1 
A2 := A − λ2 I10 =  .

 −1 0 1 0 0 0 −1 0 0 1 


 1 1 −2 −1 1 2 5 −2 1 −2 


 −1 0 1 0 0 −1 0 1 1 0 

 0 0 0 0 0 0 1 0 1 −1 
1 1 −2 −1 1 2 5 −2 1 −2

We now proceed as we did in the case of the eigenvalue λ1 = 3. We compute the

matrices Ar2 for r = 0, 1, 2, 3, . . . , and we simultaneously compute their ranks. Our
matrix A is of size 10 × 10, and the algebraic multiplicity of the eigenvalue λ2 is
2. So, we
will keep computing until we reach the first positive integer p for which
rank A2 = 10 − 2 = 8. For this p, we will have that Nul A2 = Nul Ap+1
p p


2 =
Nul Ap+2 p
p+1
p+2



2 = . . . , and consequently, rank A 2 = rank A
2 = rank A 2 = . . .
r
For r = 0, 1, . . . , p, we also compute a basis N2,r of Nul A2 . Here is our computation.
r = 0: A02 = I10 , rank A02 = 10. A basis of Nul A02 is N2,0 := ∅.

r = 1:
1 1 0 0 0 0 0 −1 0 1
 

 −3 −1 5 2 −2 −4 −7 4 −1 3 


 0 1 1 0 0 0 0 −1 0 1 


 −2 −1 3 2 −1 −2 −3 2 −1 2 

 −1 0 2 1 0 −2 −1 1 0 1 
A12 =  .

 −1 0 1 0 0 0 −1 0 0 1 


 1 1 −2 −1 1 2 5 −2 1 −2 


 −1 0 1 0 0 −1 0 1 1 0 

 0 0 0 0 0 0 1 0 1 −1 
1 1 −2 −1 1 2 5 −2 1 −2

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 626

By row reducing, we obtain

1 0 0 0 0 0 0 −1 0 0
 

 0 1 0 0 0 0 0 0 0 0 


 0 0 1 0 0 0 0 −1 0 0 


 0 0 0 1 0 0 0 0 0 0 

1
 0 0 0 0 1 0 0 −1 0 0 
RREF(A2 ) =  ,

 0 0 0 0 0 1 0 −1 0 0 


 0 0 0 0 0 0 1 0 0 0 


 0 0 0 0 0 0 0 0 1 0 

 0 0 0 0 0 0 0 0 0 1 
0 0 0 0 0 0 0 0 0 0

and we see that rank A12 = 9. The following is a basis of Nul A12 :

1
 

 0 


 1 

 0 
n
 1
o

N2,1 :=   .

 1 


 0 


 1 

 0 
0

r = 2:
0 1 2 1 −1 −1 −2 0 −1 2
 

 −6 −3 10 4 −4 −8 −14 8 −2 6 


 −1 1 3 1 −1 −1 −2 0 −1 2 


 −3 −1 4 2 −1 −2 −2 2 −1 2 

 −2 0 4 2 −1 −3 −1 2 0 1 
A22 =  .

 −1 0 1 0 0 0 0 0 0 0 


 2 2 −4 −2 2 4 9 −4 2 −4 


 −1 0 1 0 0 −1 2 1 2 −2 

 0 0 0 0 0 0 1 0 1 −1 
2 2 −4 −2 2 4 9 −4 2 −4

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 627

By row reducing, we obtain

1 0 0 0 0 0 0 −1 0 0
 

 0 1 0 0 0 0 0 0 0 0 


 0 0 1 0 0 0 0 −1 0 0 


 0 0 0 1 0 0 0 0 0 0 

2
 0 0 0 0 1 0 0 −1 0 −1 
RREF(A2 ) =  ,

 0 0 0 0 0 1 0 −1 0 0 


 0 0 0 0 0 0 1 0 0 0 


 0 0 0 0 0 0 0 0 1 −1 

 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0

and we see that rank A22 = 8. The following is a basis of Nul A22 :

1 0
   

 0  
  0 


 1  
  0 

 0   0 
n
 1
 
  1
o

N2,2 :=  ,  .

 1  
  0 


 0  
  0 


 1  
  0 

 0   1 
0 1

Since rank A22 = 8,106 we know that rank A22 = rank A32 = rank A42 = . . . , and

in particular, we can now stop computing powers of A2 .

Next, we compute the Jordan blocks of our Jordan matrix J that correspond
to the eigenvalue λ2 . We already did this in Example 8.6.8, and we saw that J
contains one Jordan block J2 (λ2 ) = J2 (2). We must compute a Jordan chain that
corresponds to this Jordan block. (This Jordan chain will contain two vectors, and
it will be of the
form {A2 u, u}.) We proceed as follows. We first extend our basis
1 2


N2,1 of Nul A2 to a basis of Nul A2 . We use Proposition 3.3.21. We form the
matrix whose columns are the vectors of N2,1 , followed by the vectors of N2,2 , with a
vertical dotted line between the vectors of N2,1 and the vectors of N2,2 . The matrix
106
Recall that A is of size 10 × 10, whereas the eigenvalue λ2 is of algebraic multiplicity 2. So, we
needed to compute powers of A2 until we got rank 10 − 2 = 8.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 628

that we obtain is the following:

1 1 0
 

 0 0 0 


 1 1 0 


 0 0 0 


 1 1 1 
.

 1 1 0 


 0 0 0 


 1 1 0 

 0 0 1 
0 0 1

By row reducing, we see that the pivot columns of this matrix are its first column
(the only column to the left of the vertical dotted line), plus the second column after
2


the vertical dotted line. So, N2,1 can be extended to a basis of Nul A2 by adding
the second vector of the basis N2,2 to it. The second vector of N2,2 is the following
vector:
T
ˆ w1 := 0 0 0 0 1 0 0 0 1 1 .


The Jordan chain corresponding to the Jordan block J2 (λ2 ) will therefore be the
chain {A2 w1 , w1 }. In a matrix form, we get

1 0
 

 0 0 


 1 0 


 0 0 

   1 1 
A2 w1 w1 =  .

 1 0 


 0 0 


 1 0 

 0 1 
0 1

We are now done processing the eigenvalue λ2 = 2.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 629

The matrices J and P . We now put everything together. Our Jordan matrix is

J := J3 (λ1 ) ⊕ J3 (λ1 ) ⊕ J2 (λ1 ) ⊕ J2 (λ2 )

= J3 (3) ⊕ J3 (3) ⊕ J2 (3) ⊕ J2 (2)

3 1 0 0 0 0 0 0 0 0
 

 0 3 1 0 0 0 0 0 0 0 


 0 0 3 0 0 0 0 0 0 0 


 0 0 0 3 1 0 0 0 0 0 

 0 0 0 0 3 1 0 0 0 0 
=  .

 0 0 0 0 0 3 0 0 0 0 


 0 0 0 0 0 0 3 1 0 0 


 0 0 0 0 0 0 0 3 0 0 

 0 0 0 0 0 0 0 0 2 1 
0 0 0 0 0 0 0 0 0 2

Our matrix P consists of the Jordan chains that we have created, where we make
sure that we place the Jordan chains in the order that corresponds to the order in
which we placed our Jordan blocks in J. Our matrix P is the following (with vertical
dotted lines places between different Jordan chains to facilitate reading):
 2
A1 u1 A1 u1 u1 A21 u2 A1 u2 u2 A1 u3 u3 A2 w1 w1

P :=

−1 1 0 0 0 −1 −1 0 1 0
 

 0 −2 1 0 −1 0 1 −1 0 0 


 −1 1 0 0 0 0 −1 0 1 0 


 1 −1 0 1 0 0 0 0 0 0 

 0 0 0 1 −1 0 0 0 1 1 
=  .

 0 0 0 0 0 1 0 0 1 0 


 0 1 0 0 1 0 0 0 0 0 


 0 0 0 0 0 0 1 0 1 0 

 0 0 0 0 0 0 0 1 0 1 
0 1 0 0 1 0 0 0 0 1

For this choice of J and P , we have that J = P −1 AP .

Optional: Because our computation is so long and complicated, it is very easy to

miscompute (even if we use a calculator throughout). Therefore, it is a good idea to
check our answer. We ask a calculator to check whether our matrix P is invertible,

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 630

and if so, to compute its inverse. Indeed, we obtain

3 0 −5 −1 1 3 2 −2 0 −1
 
 1 0 −2 −1 1 1 2 −1 0 −1 
 
 2 1 −3 −1 1 2 4 −2 1 −2 
 
 −2 0 3 1 0 −2 0 1 0 0 
 
−1
 −1 0 2 1 −1 −1 −1 1 0 1 
P =   −1 0
.
 1 0 0 0 0 0 0 0 

 −1 0 1 0 0 −1 0 1 0 0 
 
 0 0
 0 0 0 0 1 0 1 −1 

 1 0 −1 0 0 1 0 0 0 0 
0 0 0 0 0 0 −1 0 0 1

Next, we ask the calculator to compute the product P −1 AP . We are in luck: we

do indeed get the Jordan matrix J that we computed above. So, our answer is
correct.

Example 8.6.33. Consider the following matrix in C13×13 :

 
4 0 0 3 0 −1 2 0 0 0 0 2 0
 1
 5 1 −3 0 1 −2 1 1 0 1 −2 0 

 −1 −2 3 4 0 −6 3 1 −2 2 −3 4 −2 
 
 0 −3 0 10 −1 −2 4 0 −2 −1 0 4 0 
 
 0
 0 0 −1 4 3 −1 0 0 0 1 −1 2 

 0 0 0 0 0 4 0 0 0 0 0 0 0 
 
A :=  0 3 0 −12 1 2 −4 0 2 1 −2 −6 0 .
 
 1 1 1 −1 0 6 −1 3 1 −2 3 −2 2 
 
 0 −1 0 3 0 −1 2 0 3 0 0 2 0 
 
 −1 −1 −1 0 −4 3 −1 −1 4 −2 3 −2 
 
4
 
 0
 1 0 −3 0 1 −2 0 1 0 4 −2 0 

 0 2 0 3 1 1 2 0 1 1 2 4 0 
0 0 0 0 0 −2 0 0 0 0 0 0 2

Compute a Jordan matrix J and an invertible matrix P , both in C5×5 , such that
J = P −1 AP .
Remark: This is the matrix from Example 8.6.9. In that example, we computed
the matrix J. Here, we will see how to compute the matrix P .

Solution. We first compute the characteristic polynomial of A, and we factor it into

linear terms:

pA (λ) = det λI13 − A) = (λ − 4)10 (λ − 2)3 .

The eigenvalues of A are λ1 = 4 (with algebraic multiplicity 10) and λ2 = 2 (with

algebraic multiplicity 3). We handle the two eigenvalues separately.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 631

The eigenvalue λ1 = 4. To simplify notation, we write

A1 := A − λ1 I13
 
0 0 0 3 0 −1 2 0 0 0 0 2 0

 1 1 1 −3 0 1 −2 1 1 0 1 −2 0 


 −1 −2 −1 4 0 −6 3 1 −2 2 −3 4 −2 


 0 −3 0 6 −1 −2 4 0 −2 −1 0 4 0 


 0 0 0 −1 0 3 −1 0 0 0 1 −1 2 

0 0 0 0 0 0 0 0 0 0 0 0 0 
 

=  0 3 0 −12 1 2 −8 0 2 1 −2 −6 0 
 
1 1 1 −1 0 6 −1 −1 1 −2 3 −2 2 
 

0 −1 0 3 0 −1 2 0 −1 0 0 2 0 
 

−1 −1 −1 0 −4 3 −1 −1 0 −2 3 −2 
 
 4
 

 0 1 0 −3 0 1 −2 0 1 0 0 −2 0 

 0 2 0 3 1 1 2 0 1 1 2 0 0 
0 0 0 0 0 −2 0 0 0 0 0 0 −2

We proceed as follows. We compute the matrices Ar1 for r = 0, 1, 2, 3, . . . , and

we simultaneously compute their ranks. Our matrix A is of size 13 × 13, and
the algebraic multiplicity of the eigenvalue λ1 is 10. So, we
will keep computing
until we reach the first positive integer p for which rank Ap1 = 13 − 10 = 3. For
this p, we will have Nul Ap1 = Nul Ap+1 = Nul Ap+2




1 1 = . . . , and consequently,
p
p+1
p+1

rank A1 = rank A1
= rank A1 = . . . . For r = 0, 1, . . . , p, we also compute
a basis N1,r of Nul Ar1 . Here is our computation.
r = 0: A01 = I13 , rank A01 = 13. A basis of Nul A01 is N1,0 := ∅.

r = 1:
 
0 0 0 3 0 −1 2 0 0 0 0 2 0

 1 1 1 −3 0 1 −2 1 1 0 1 −2 0 


 −1 −2 −1 4 0 −6 3 1 −2 2 −3 4 −2 


 0 −3 0 6 −1 −2 4 0 −2 −1 0 4 0 


 0 0 0 −1 0 3 −1 0 0 0 1 −1 2 

0 0 0 0 0 0 0 0 0 0 0 0 0 
 

A11 =  0 3 0 −12 1 2 −8 0 2 1 −2 −6 0 .
 
1 1 1 −1 0 6 −1 −1 1 −2 3 −2 2 
 

0 −1 0 3 0 −1 2 0 −1 0 0 2 0 
 

−1 −1 −1 0 −4 3 −1 −1 0 −2 3 −2 
 
 4
 

 0 1 0 −3 0 1 −2 0 1 0 0 −2 0 

 0 2 0 3 1 1 2 0 1 1 2 0 0 
0 0 0 0 0 −2 0 0 0 0 0 0 −2

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 632

By row reducing, we obtain

 
1 0 1 0 0 0 0 0 0 −1 0 1 0
 0 1 0 0 0 0 0 0 1 0 0 0 0 
 
 0 0 0 1 0 0 0 0 0 0 0 2 0 
 
 0
 0 0 0 1 0 0 0 −1 1 0 0 0 

 0 0 0 0 0 1 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 1 0 0 0 0 −2 0 
 
RREF(A11 ) =  0 0 0 0 0 0 0 1 0 1 0 0 0 ,
 
 0 0 0 0 0 0 0 0 0 0 1 −1 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 1 
 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0

and we see that rank A11 = 9. The following is a basis of Nul A11 :

       
−1 0 1 −1
 0   −1   0   0 
       
 1   0   0   0 
       
 0   0   0   −2 
       
 0   1   −1   0 
       
n 0   0   0   0
       
o
N1,1 :=  0 , 0 , 0 , 2  .
       
 0   0   −1   0
       

 0   1   0   0
       

       
 0   0   1   0 
       
 0   0   0   1 
       
 0   0   0   1 
0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 633

r = 2:
 
0 1 0 0 1 0 0 0 0 1 0 0 0

 1 −1 1 3 −1 0 2 1 0 −1 1 2 0 


 −2 3 −2 9 3 8 6 −6 0 −1 2 2 4 


 −2 2 −2 0 2 0 0 −2 0 2 −2 0 0 


 0 −1 0 0 −1 −4 0 0 0 −1 0 0 −4 

0 0 0 0 0 0 0 0 0 0 0 0 0 
 

A21 =  2 −2 2 12 −2 0 8 2 0 −2 6 4 0 .
 
1 −2 1 −9 −2 −8 −6 5 0 2 −3 −2 −4 
 

−1 1 −1 0 1 0 0 −1 0 1 −1 0 0 
 

−1 −1 −3 4 −2 −1 0 2 −1 −2
 
 2 2 4 
 

 1 −1 1 0 −1 0 0 1 0 −1 1 0 0 

 1 −1 1 −12 −1 0 −8 1 0 −1 −3 −4 0 
0 0 0 0 0 4 0 0 0 0 0 0 4

By row reducing, we obtain

 
1 0 1 0 0 0 0 0 0 −1 0 1 −1
 0 1 0 0 1 0 0 0 0 1 0 0 0 
 
 0 0 0 1 0 0 2/3 0 0 0 0 2/3 0 
 
 0 0 0 0 0 1 0 0 0 0 0 0 1 
 
 0 0 0 0 0 0 0 1 0 1 0 0 1 
 
 0 0 0 0 0 0 0 0 0 0 1 −1 0 
 
RREF(A21 ) =  0 0 0 0 0 0 0 0 0 0 0 0 0 ,
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 634

and we see that rank A21 = 6. The following is a basis of Nul A21 :

             
−1 0 0 0 1 −3 1
 0   −1   0   0   −1   0   0 
             
 1   0   0   0   0   0   0 
             
 0   0   −2   0   0   −2   0 
             
 0   1   0   0   0   0   0 
             
n  0   0   0   0   0   0   −1  o
             
N1,2 :=  0 , 0 , 3 , 0 , 0 , 0 , 0  .
             
 0   0   0   0   −1   0   −1 
             
 0   0   0   1   0   0   0 
             
             
 0   0   0   0   1   0   0 
             
 0   0   0   0   0   3   0 
             
 0   0   0   0   0   3   0 
0 0 0 0 0 0 1

r = 3:
 
0 0 0 0 0 0 0 0 0 0 0 0 0

 0 1 0 3 1 0 2 0 0 1 0 2 0 


 0 −1 0 −42 −1 −16 −28 8 0 7 −12 −16 −8 


 0 0 0 −6 0 0 −4 0 0 0 0 −4 0 


 0 0 0 0 0 8 0 0 0 0 0 0 8 

0 0 0 0 0 0 0 0 0 0 0 0 0 
 

A31 =  0 0 0 −18 0 0 −12 0 0 0 −8 −4 0 .
 
0 1 0 39 1 16 26 −8 0 −7 12 14 8 
 

0 0 0 −3 0 0 −2 0 0 0 0 −2 0 
 

0 −1 0 −3 −1 −8 −2 0 0 −1 0 −2 −8 
 

 

 0 0 0 3 0 0 2 0 0 0 0 2 0 

 0 0 0 27 0 0 18 0 0 0 8 10 0 
0 0 0 0 0 −8 0 0 0 0 0 0 −8

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 635

By row reducing, we obtain

 
0 1 0 0 1 0 0 0 0 1 0 0 0
 0 0 0 1 0 0 2/3 0 0 0 0 2/3 0 
 
 0 0 0 0 0 1 0 0 0 0 0 0 1 
 
 0 0 0 0 0 0 0 1 0 1 0 0 1 
 
 0
 0 0 0 0 0 0 0 0 0 1 −1 0 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
RREF(A31 ) =  0 0 0 0 0 0 0 0 0 0 0 0 0 ,
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0

and we see that rank A31 = 5. The following is a basis of Nul A31 :

               
1 0 0 0 0 0 0 0
 0   0   −1   0   0   −1   0   0 
               
 0   1   0   0   0   0   0   0 
               
 0   0   0   −2   0   0   −2   0 
               
 0   0   1   0   0   0   0   0 
               
n 0   0   0   0   0   0   0 −1
               
  o
N1,3 :=  0 , 0 , 0 , 3 , 0 , 0 , 0 , 0  .
               
 0   0   0   0   0   −1   0 −1
               
  
 0   0   0   0   1   0   0 0
               
  
               
 0   0   0   0   0   1   0   0 
               
 0   0   0   0   0   0   3   0 
               
 0   0   0   0   0   0   3   0 
0 0 0 0 0 0 0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 636

r = 4:
 
0 0 0 0 0 0 0 0 0 0 0 0 0

 0 1 0 0 1 0 0 0 0 1 0 0 0 


 0 −2 0 96 −2 32 64 −16 0 −18 32 32 16 


 0 −2 0 0 −2 0 0 0 0 −2 0 0 0 


 0 0 0 0 0 −16 0 0 0 0 0 0 −16 

0 0 0 0 0 0 0 0 0 0 0 0 0 
 

A41 =  0 2 0 48 2 0 32 0 0 2 16 16 0 
 
0 1 0 −96 1 −32 −64 16 0 17 −32 −32 −16 
 

0 −1 0 0 −1 0 0 0 0 −1 0 0 0 
 

−1 0 −1 0 0 −1
 
 0 0 16 0 0 0 16 
 

 0 1 0 0 1 0 0 0 0 1 0 0 0 

 0 1 0 −48 1 0 −32 0 0 1 −16 −16 0 
0 0 0 0 0 16 0 0 0 0 0 0 16

By row reducing, we obtain:

 
0 1 0 0 1 0 0 0 0 1 0 0 0
 0 0 0 1 0 0 2/3 0 0 0 1/3 1/3 0 
 
 0 0 0 0 0 1 0 0 0 0 0 0 1 
 
 0 0 0 0 0 0 0 1 0 1 0 0 1 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
RREF(A41 ) =  0 0 0 0 0 0 0 0 0 0 0 0 0 ,
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 637

and we see that rank A41 = 4. The following is a basis of Nul A41 :

             
1 0 0 0 0 0 0
 0   0   −1   0   0   −1   0 
             
 0   1   0   0   0   0   0 
             
 0   0   0   −2   0   0   −1 
             
 0   0   1   0   0   0   0 
             
n 0   0   0   0   0   0   0 
             
N1,4 :=  0 , 0 , 0 , 3 , 0 , 0 , 0 ,
             
 0   0   0   0   0   −1   0 
             
 0   0   0   0   1   0   0 
             
             
 0   0   0   0   0   1   0 
             
 0   0   0   0   0   0   3 
             
 0   0   0   0   0   0   0 
0 0 0 0 0 0 0
   
0 0

 0  
  0 


 0  
  0 


 −1  
  0 


 0  
  0 

0 −1
   
   o
0 , 0  .
   

0 −1
   
   
0 0
   
   
   
 0   0 
   

 0  
  0 

 3   0 
0 1

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 638

r = 5:
 
0 0 0 0 0 0 0 0 0 0 0 0 0

 0 0 0 0 0 0 0 0 0 0 0 0 0 


 0 0 0 −240 0 −64 −160 32 0 32 −80 −80 −32 


 0 0 0 0 0 0 0 0 0 0 0 0 0 


 0 0 0 0 0 32 0 0 0 0 0 0 32 

0 0 0 0 0 0 0 0 0 0 0 0 0 
 

A51 =  0 0 0 −96 0 0 −64 0 0 0 −32 −32 0 .
 
0 0 0 240 0 64 160 −32 0 −32 80 80 32 
 

0 0 0 0 0 0 0 0 0 0 0 0 0 
 

0 0 −32 0 −32 
 
 0 0 0 0 0 0 0 0
 

 0 0 0 0 0 0 0 0 0 0 0 0 0 

 0 0 0 96 0 0 64 0 0 0 32 32 0 
0 0 0 0 0 −32 0 0 0 0 0 0 −32

By row reducing, we obtain:

 
0 0 0 1 0 0 2/3 0 0 0 1/3 1/3 0
 0 0 0 0 0 1 0 0 0 0 0 0 1 
 
 0 0 0 0 0 0 0 1 0 1 0 0 1 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
RREF(A51 ) =  0 0 0 0 0 0 0 0 0 0 0 0 0 ,
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 639

and we see that rank A51 = 3. The following is a basis of Nul A51 :

             
1 0 0 0 0 0 0
 0   1   0   0   0   0   0 
             
 0   0   1   0   0   0   0 
             
 0   0   0   0   −2   0   0 
             
 0   0   0   1   0   0   0 
             
n 0   0   0   0   0   0   0 
             
N1,5 =  0 , 0 , 0 , 0 , 3 , 0 , 0 ,
             
 0   0   0   0   0   0   −1 
             
 0   0   0   0   0   1   0 
             
             
 0   0   0   0   0   0   1 
             
 0   0   0   0   0   0   0 
             
 0   0   0   0   0   0   0 
0 0 0 0 0 0 0
     
0 0 0

 0  
  0  
  0 


 0  
  0  
  0 


 −1  
  −1  
  0 


 0  
  0  
  0 

0 0 −1
     
     o
0 , 0 , 0
     
 
0 0 −1
     
     
0 0 0
     
     
     
 0   0   0 
     

 3  
  0  
  0 

 0   3   0 
0 0 1

Since rank A51 = 3,107 we know that rank A51 = rank A61 = rank A71 = . . . , and

in particular, we can now stop computing powers of A1 .

Now, we compute the Jordan blocks of our Jordan matrix J associated with the
eigenvalue λ1 = 4. This is done precisely as in Example 8.6.9 (using Theorem 8.6.6).
In Example 8.6.9, we saw that J contains one Jordan block J5 (λ1 ) = J5 (4), two
Jordan blocks J2 (λ1 ) = J2 (4), and one Jordan block J1 (λ1 ) = J1 (4). The Jordan
normal form of A contains no other Jordan blocks of the form Jt (λ1 ) = Jt (4). We
now need to generate the Jordan chains corresponding to these Jordan blocks.
We first generate the Jordan chain corresponding to the Jordan block J5 (λ1 ).
(Note that this Jordan chain will contain five vectors and will be of the form
107
Recall that A is of size 13 × 13, whereas the algebraic multiplicity of λ1 is 10. So, we needed to
compute powers of A1 until we got rank 13 − 10 = 3.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 640

{A41 u, A31 u, A21 u, A1 u, u}.) We extend our basis N1,4 to a basis of Nul A51 . We

use Proposition 3.3.21. We form the matrix whose columns are the vectors of N1,4 ,
followed by the vectors of N1,5 , with a vertical dotted line between the vectors of
N1,4 and the vectors of N1,5 . The matrix that we obtain is the following:
 
1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0
 0 0 −1 0 0 −1 0 0 0 0 1 0 0 0 0 0 0 0 0 
 
 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 
 
 0 0
 0 −2 0 0 −1 −1 0 0 0 0 0 −2 0 0 −1 −1 0 

 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 
 
 0 0 0 0 0 0 0 0 −1 0 0 0 0 0 0 0 0 0 −1 
 
 0 0 0 3 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 .
 
 0 0 0 0 0 −1 0 0 −1 0 0 0 0 0 0 −1 0 0 −1 
 
 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 
 
 
 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 
 
 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 3 0 0 
 
 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 3 0 
0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1

By row reducing this matrix, we see that its pivot columns are all its columns to the
left of the vertical dotted line, plus the second column to the
right of the vertical
5
dotted line. So, N1,4 can be extended to a basis of Nul A1 by adding the second
vector of the basis N1,5 to it. The second vector of N1,5 is the following vector:
T
ˆ u1 := 0 1 0 0 0 0 0 0 0 0 0 0 0 .


The vector u1 will start the Jordan chain that corresponds to the Jordan block
J5 (λ1 ). This Jordan chain is {A41 u1 , A31 u1 , A21 u1 , A1 u1 , u1 }. If we compute the values
of all these vectors and arrange them into a matrix, we obtain the following:
 
0 0 1 0 0
 1
 1 −1 1 1 
 −2 −1 3 −2 0 
 
 −2 0 2 −3 0 
 
 0
 0 −1 0 0 
0 0 0 0 0
 
 4  
A1 u1 A31 u1 A21 u1 A1 u1 u1

=  2 0 −2 3 0 .
 
 1 1 −2 1 0 
 
 −1 0 1 −1 0 
 
 −1 −1 2 −1 0 
 
 
 1
 0 −1 1 0 

 1 0 −1 2 0 
0 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 641

Next, we generate the Jordan chains corresponding to the two Jordan chains
corresponding to the two Jordan blocks J2 (λ1 ). Each of these Jordan chains will
contain two vectors and will be of the form {A1 u, u}. We first identify the generalized
eigenvectors of rank 2 of A associated with the eigenvalue λ1 inside the Jordan chains
(associated with λ1 ) that we have already created. There is exactly on such vector,
−
namely, A31 u1 . Now, N1,2 := N1,1 ∪ {A31 u1 } is a linearly independent set of vectors
in Nul A1 , and we would like to extend it to a basis of Nul A21 . We once again
2

use Proposition 3.3.21. We form the matrix whose columns are the vectors of the
−
linearly independent set N1,2 = N1,1 ∪ {A31 u1 }, followed by the vectors of N1,2 , with
a vertical dotted line between the vectors of the two sets. The matrix that we obtain
is the following:
 
−1 0 1 −1 0 −1 0 0 0 1 −3 1
 0 −1 0 0 1 0 −1 0 0 −1 0 0 
 
 1
 0 0 0 −1 1 0 0 0 0 0 0 

 0
 0 0 −2 0 0 0 −2 0 0 −2 0 

 0
 1 −1 0 0 0 1 0 0 0 0 0 

 0 0 0 0 0 0 0 0 0 0 0 −1 
 
 0 0 0 2 0 0 0 3 0 0 0 0 
 
 0 0 −1 0 1 0 0 0 0 −1 0 −1 
 
 0 1 0 0 0 0 0 0 1 0 0 0 
 
0 −1
 
 0 0 1 0 0 0 0 1 0 0 
 
 0 0 0 1 0 0 0 0 0 0 3 0 
 
 0 0 0 1 0 0 0 0 0 0 3 0 
0 0 0 0 0 0 0 0 0 0 0 1

By row reducing this matrix, we see that its pivot columns are all its columns to the
left of the vertical dotted line, plus the third and seventh column to
the right of the
−
vertical dotted line. So, N1,2 can be extended to a basis of Nul A21 by adding the
third and the seventh vector of N1,2 to it. The third and the seventh vector of N1,2
are the following:
T
ˆ u2 := 0 0 0 −2 0 0 3 0 0 0 0 0 0 ;


T
ˆ u3 :=

1 0 0 0 0 −1 0 −1 0 0 0 0 1 .

These two vectors will start our two Jordan chains that correspond to the two
Jordan blocks J2 (λ1 ). These two Jordan chains are {A1 u2 , u2 } and {A1 u3 , u3 }. If
we compute the values of these vectors and arrange them into a matrix, we obtain

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 642

the following (the vertical dotted line separates the two Jordan chains):
 
0 0 1 1
 0
 0 −1 0 

 1 0 2 0 
 
 0 −2 2 0 
 
 −1 0 −1 0 
 
0 0 0 −1
 
   
A1 u2 u2 A1 u3 u3 =  0 3 −2 0 .
 
 −1 0 −2 −1 
 
 0 0 1 0 
 
 
 1 0 2 0 
 
 0
 0 −1 0 

 0 0 −1 0 
0 0 0 1

Finally, we generate the Jordan chain that corresponds to the Jordan block
J1 (λ1 ). This Jordan chain will contain only one vector, which we find as follows.
We first identify the generalized eigenvectors of rank 1 of A associated with the
eigenvalue λ1 inside the Jordan chains (associated with λ1 ) that we have already
generated. There are exactly three such vectors, namely, A41 u1 , A1 u2 , A1 u3 . Now,
−
N1,1 := N1,0 ∪ {A41 u1 , A1 u2 , A1 u3 } = {A41 u1 , A1 u2 , A1 u3 } is a linearly independent
set in Nul A11 , and we would like to extend it to a basis of Nul A11 . We once again

use Proposition 3.3.21. We form the matrix whose columns are the vectors of the
−
linearly independent set N1,1 = {A41 u1 , A1 u2 , A1 u3 }, followed by the vectors of N1,1 ,
−
with a vertical dotted line between the vectors of N1,1 and the vectors of N1,1 . The
matrix that we obtain is the following:
 
0 0 1 −1 0 1 −1
 1
 0 −1 0 −1 0 0 

 −2 1 2 1 0 0 0 
 
 −2 0 2 0 0 0 −2 
 
 0 −1 −1 0 1 −1 0 
 
 0 0 0 0 0 0 0 
 
 2 0 −2 0 0 0 2 .
 
 1 −1 −2 0 0 −1 0 
 
 −1 0 1 0 1 0 0 
 
 −1
 
1 2 0 0 1 0 
 
 1
 0 −1 0 0 0 1 

 1 0 −1 0 0 0 1 
0 0 0 0 0 0 0
By row reducing this matrix, we see that its pivot columns are all its columns to the
left of the vertical dotted line, plus the second column to the right of the vertical

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 643

−
can be extended to a basis of Nul A11 by adding the second


dotted line. So, N1,1
vector of N1,1 to it. The second vector of N1,1 is the following:
T
ˆ u4 := 0 −1 0 0 1 0 0 0 1 0 0 0 0 .


This vector forms the Jordan chain {u4 } that corresponds to the Jordan block J1 (λ1 ).
We are now done processing the eigenvalue λ1 = 4.

The eigenvalue λ2 = 2. To simplify notation, we write

A2 := A − λ2 I13
 
2 0 0 3 0 −1 2 0 0 0 0 2 0

 1 3 1 −3 0 1 −2 1 1 0 1 −2 0 


 −1 −2 1 4 0 −6 3 1 −2 2 −3 4 −2 


 0 −3 0 8 −1 −2 4 0 −2 −1 0 4 0 


 0 0 0 −1 2 3 −1 0 0 0 1 −1 2 

0 0 0 0 0 2 0 0 0 0 0 0 0 
 

=  0 3 0 −12 1 2 −6 0 2 1 −2 −6 0 .
 
1 1 1 −1 0 6 −1 1 1 −2 3 −2 2 
 

0 −1 0 3 0 −1 2 0 1 0 0 2 0 
 

−1 −1 −1 0 −4 3 −1 −1 2 −2 3 −2 
 
 4
 

 0 1 0 −3 0 1 −2 0 1 0 2 −2 0 

 0 2 0 3 1 1 2 0 1 1 2 2 0 
0 0 0 0 0 −2 0 0 0 0 0 0 0

We proceed similarly as in the case of the eigenvalue λ1 . We compute matrices Ar2

for r = 0, 1, 2, 3, . . . , and we simultaneously compute their ranks. Our matrix A is of
size 13 × 13, and the algebraic multiplicity of the eigenvalue λ2 is 3. So, we will keep
p

computing until we reach the first positive integer p for which rank A2 = 13−3 = 10.
For this p, we will have that Nul Ap2 = Nul Ap+1 = Nul Ap+2




2 2 = . . . , and
consequently, rank Ap2 = rank Ap+1 p+2




2
= rank A 2 = . . . . For r = 0, 1, . . . , p, we
r
also compute a basis N2,r of Nul A2 . Here is our computation.
r = 0: A02 = I13 , rank A02 = 13. A basis of Nul A02 is N2,0 := ∅.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 644

r = 1:
 
2 0 0 3 0 −1 2 0 0 0 0 2 0

 1 3 1 −3 0 1 −2 1 1 0 1 −2 0 


 −1 −2 1 4 0 −6 3 1 −2 2 −3 4 −2 


 0 −3 0 8 −1 −2 4 0 −2 −1 0 4 0 


 0 0 0 −1 2 3 −1 0 0 0 1 −1 2 

0 0 0 0 0 2 0 0 0 0 0 0 0 
 

A12 =  0 3 0 −12 1 2 −6 0 2 1 −2 −6 0 .
 
1 1 1 −1 0 6 −1 1 1 −2 3 −2 2 
 

0 −1 0 3 0 −1 2 0 1 0 0 2 0 
 

−1 −1 −1 0 −4 3 −1 −1 2 −2 3 −2 
 
 4
 

 0 1 0 −3 0 1 −2 0 1 0 2 −2 0 

 0 2 0 3 1 1 2 0 1 1 2 2 0 
0 0 0 0 0 −2 0 0 0 0 0 0 0

By row reducing, we obtain

 
1 0 0 0 0 0 0 0 0 0 0 0 0

 0 1 0 0 0 0 0 0 0 0 0 0 0 


 0 0 1 0 0 0 0 1 0 0 0 0 0 


 0 0 0 1 0 0 0 0 0 0 0 0 0 


 0 0 0 0 1 0 0 0 0 0 0 0 1 

0 0 0 0 0 1 0 0 0 0 0 0 0 
 

RREF(A12 ) =  0 0 0 0 0 0 1 0 0 0 0 0 0 ,
 
0 0 0 0 0 0 0 0 1 0 0 0 0 
 

0 0 0 0 0 0 0 0 0 1 0 0 −1 
 

 
 0 0 0 0 0 0 0 0 0 0 1 0 0 
 

 0 0 0 0 0 0 0 0 0 0 0 1 0 

 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 645

and we see that rank A12 = 11. A basis of Nul A12 is

   
0 0
 0   0 
   
 −1   0 
   
 0   0 
   
 0   −1 
   
n 0   0 o
   
N2,1 :=  0 , 0  .
   
 1   0 
   
 0   0 
   
   
 0   1 
   
 0   0 
   
 0   0 
0 1

r = 2:
 
4 1 0 12 1 −4 8 0 0 1 0 8 0

 5 7 5 −9 −1 4 −6 5 4 −1 5 −6 0 


 −6 −5 −2 25 3 −16 18 −2 −8 7 −10 18 −4 


 −2 −10 −2 28 −2 −8 16 −2 −8 −2 −2 16 0 


 0 −1 0 −4 3 8 −4 0 0 −1 4 −4 4 

0 0 0 0 0 4 0 0 0 0 0 0 0 
 

A22 =  2 10 2 −36 2 8 −20 2 8 2 −2 −20 0 .
 
5 2 5 −13 −2 16 −10 5 4 −6 9 −10 4 
 

−1 −3 −1 12 1 −4 8 −1 0 1 −1 8 0 
 

−5 −2 −5 2 −12 10 −5 −4 6 −9 10 −4 
 
 13
 

 1 3 1 −12 −1 4 −8 1 4 −1 5 −8 0 

 1 7 1 0 3 4 0 1 4 3 5 0 0 
0 0 0 0 0 −4 0 0 0 0 0 0 0

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 646

By row reducing, we obtain:

 
1 0 0 0 0 0 0 0 0 0 0 0 0

 0 1 0 0 0 0 0 0 0 0 0 0 0 


 0 0 1 0 0 0 0 1 0 0 0 0 0 


 0 0 0 1 0 0 0 0 0 0 0 0 0 


 0 0 0 0 1 0 0 0 0 0 0 0 1 

0 0 0 0 0 1 0 0 0 0 0 0 0 
 

RREF(A22 ) =  0 0 0 0 0 0 1 0 0 0 0 1 0 ,
 
0 0 0 0 0 0 0 0 1 0 0 0 0 
 

0 0 0 0 0 0 0 0 0 1 0 0 −1 
 

 
 0 0 0 0 0 0 0 0 0 0 1 0 0 
 

 0 0 0 0 0 0 0 0 0 0 0 0 0 

 0 0 0 0 0 0 0 0 0 0 0 0 0 
0 0 0 0 0 0 0 0 0 0 0 0 0

and we see that rank A22 = 10. A basis of Nul A22 is

     
0 0 0
 0   0   0 
     
 −1   0   0 
     
 0   0   0 
     
 0   0   −1 
     
n 0   0   0
     
o
N2,2 :=  0  ,  −1  ,  0  .
     
 1   0   0
     

 0   0   0
     

     
 0   0   1 
     
 0   0   0 
     
 0   1   0 
0 0 1

Since rank A22 = 10,108 we know that rank A22 = rank A32 = rank(A42 ) = . . . , and

in particular, we can now stop computing powers of A2 .

We now compute the Jordan blocks of the Jordan matrix J associated with the
eigenvalue λ2 = 2. This is done precisely as in Example 8.6.9 (using Theorem 8.6.6).
In Example 8.6.9, we saw that J contains one Jordan block J2 (λ2 ) = J2 (2), one
Jordan block J1 (λ2 ) = J1 (2), and no other Jordan blocks of the form Jt (λ2 ) = Jt (2).
We first generate the Jordan chain corresponding to the Jordan block J2 (λ2 ).
(Note that this Jordan chain will contain two vectors and will be of the form {A2 u, u}.)
108
Recall that A is of size 13 × 13, whereas the eigenvalue λ2 is of algebraic multiplicity 3. So, we
need to compute powers of A2 until we got rank 13 − 3 = 10.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 647

We extend our basis N2,1 to a basis of Nul A22 . We use Proposition 3.3.21. We form

the matrix whose columns are the vectors of N2,1 , followed by the vectors of N2,2 ,
with a vertical dotted line between the vectors of N2,1 and the vectors of N2,2 . The
matrix that we obtain is the following:
 
0 0 0 0 0
 0 0 0 0 0 
 
 −1 0 −1 0 0 
 
 0 0 0 0 0 
 
 0 −1 0 0 −1 
 
 0 0 0 0 0 
 
 0 0 0 −1 0 .
 
 1 0 1 0 0 
 
 0 0 0 0 0 
 
 
 0 1 0 0 1 
 
 0 0 0 0 0 
 
 0 0 0 1 0 
0 1 0 0 1
By row reducing, we see that the pivot columns of the matrix above are both columns
to the left of the vertical dotted line, plus the second column to the
right of the
2
vertical dotted line. So, N2,1 can be extended to a basis of Nul A2 by adding the
second vector of N2,2 to it. The second vector of N2,1 is the following vector:
T
ˆ w1 := 0 0 0 0 0 0 −1 0 0 0 0 1 0 .


The vector w1 will starts the Jordan chain {A2 w1 , w1 } that corresponds to the
Jordan block J2 (λ2 ). If we place this Jordan chain into a matri, we obtain the
following:  
0 0
 0 0 
 
 1 0 
 
 0 0 
 
 0 0 
 
0 0
 
   
A2 w1 w1 =  0 −1  .
 
 −1 0 
 
 0 0 
 
 
 0 0 
 
 0 0 
 
 0 1 
0 0

It remains to generate a Jordan chain that corresponds to the Jordan block

J1 (λ2 ) of the Jordan matrix J. This Jordan chain will contain only one vector,

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 648

which we find as follows. We first identify the generalized eigenvectors of rank 1 of A

associated with the eigenvalue λ2 inside the Jordan chains (associated with λ2 ) that
we have already generated. There is exactly one such vector, namely, A2
w1 . Now,
−
N2,1 := N2,0 ∪ {A2 w1 } = {A2 w1 } is a linearly independent set in Nul A12 , and we
would like to extend it to a basis of Nul A12 .

We once again use Proposition 3.3.21. We form the matrix whose columns are
−
the unique vector of the linearly independent set N2,1 = {A2 w1 }, followed by the
−
vectors of N2,1 , with a vertical dotted line between the vector of N2,1 and the vectors
of N2,1 . The matrix that we obtain is the following:
 
0 0 0
 0 0 0 
 
 1 −1 0 
 
 0 0 0 
 
 0
 0 −1 

 0 0 0 
 
 0 0 0 .
 
 −1 1 0 
 
 0 0 0 
 
 
 0 0 1 
 
 0 0 0 
 
 0 0 0 
0 0 1

By row reducing this matrix, we see that its pivot columns are its one column to the
left of the vertical dotted line, plus the second column to the
right of the vertical
−
dotted line. So, N2,1 can be extended to a basis of Nul A12 by adding the second
vector of N2,1 to it. The second vector of N2,1 is the following:
T
ˆ w2 := 0 0 0 0 −1 0 0 0 0 1 0 0 1 .


This vector forms the Jordan chain {w2 } that corresponds to the Jordan block
J1 (λ2 ).
We are now done processing the eigenvalue λ2 = 2.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 649

The matrices J and P . We now put everything together. Our Jordan matrix is

J := J5 (λ1 ) ⊕ J2 (λ1 ) ⊕ J2 (λ1 ) ⊕ J1 (λ1 ) ⊕ J2 (λ2 ) ⊕ J1 (λ2 )

= J5 (4) ⊕ J2 (4) ⊕ J2 (4) ⊕ J1 (4) ⊕ J2 (2) ⊕ J1 (2)

 
4 1 0 0 0 0 0 0 0 0 0 0 0

 0 4 1 0 0 0 0 0 0 0 0 0 0 


 0 0 4 1 0 0 0 0 0 0 0 0 0 


 0 0 0 4 1 0 0 0 0 0 0 0 0 


 0 0 0 0 4 0 0 0 0 0 0 0 0 

0 0 0 0 0 4 1 0 0 0 0 0 0
 
 
=  0 0 0 0 0 0 4 0 0 0 0 0 0 .
 
0 0 0 0 0 0 0 4 1 0 0 0 0
 
 
0 0 0 0 0 0 0 0 4 0 0 0 0
 
 
 
 0 0 0 0 0 0 0 0 0 4 0 0 0 
 

 0 0 0 0 0 0 0 0 0 0 2 1 0 

 0 0 0 0 0 0 0 0 0 0 0 2 0 
0 0 0 0 0 0 0 0 0 0 0 0 2

Our matrix P consists of the Jordan chains that we have generated, where we make
sure that we place the Jordan chain in the order that correspoinds to the order in
which we places our Jordan blocks in J. Our matrix P is the following (with vertical
dotted lines between different Jordan chains to facilitate reading):
 4
A1 u1 A31 u1 A21 u1 A1 u1 u1 A1 u2 u2 A1 u3 u3 u4 A2 w1 w1 w2

P :=
 
0 0 1 0 0 0 0 1 1 0 0 0 0

 1 1 −1 1 1 0 0 −1 0 −1 0 0 0 


 −2 −1 3 −2 0 1 0 2 0 0 1 0 0 


 −2 0 2 −3 0 0 −2 2 0 0 0 0 0 


 0 0 −1 0 0 −1 0 −1 0 1 0 0 −1 

0 0 0 0 0 0 0 0 −1 0 0 0 0 
 

=  2 0 −2 3 0 0 3 −2 0 0 0 −1 0 
 
1 1 −2 1 0 −1 0 −2 −1 0 −1 0 0 
 

−1 0 1 −1 0 0 0 1 0 1 0 0 0 
 

−1 −1 2 −1 0
 
 1 0 2 0 0 0 0 1 
 

 1 0 −1 1 0 0 0 −1 0 0 0 0 0 

 1 0 −1 2 0 0 0 −1 0 0 0 1 0 
0 0 0 0 0 0 0 0 1 0 0 0 1

Optional: Let us check our anger. We first check that our matrix P is invertible,

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 650

and we compute its inverse. Indeed, the calculator tells us that

 
1 0 0 −3 0 1 −2 0 0 0 1 −2 0
 0
 0 0 0 −1 0 0 0 1 −1 0 0 0 

 1 0 1 0 0 0 0 1 0 0 1 0 0 
 
 0 0 0 3 0 0 2 0 0 0 0 2 0 
 
 0 1 0 0 1 0 0 0 0 1 0 0 0 
 
 −1 0 0 0 −1 −2 0 0 1 0 1 0 −1 
 
P −1 =  0

0 0 −2 0 0 −1 0 0 0 −1 −1 0 .

 0 0 −1 0 0 1 0 −1 0 0 −1 0 0 
 
 0 0 0 0 0 −1 0 0 0 0 0 0 0 
 
 
 0 0 0 0 0 0 0 0 1 0 1 0 0 
 
 0
 0 0 0 0 2 0 −1 0 −1 0 0 1 

 0 0 0 −3 0 0 −2 0 0 0 −1 −1 0 
0 0 0 0 0 1 0 0 0 0 0 0 1

Next, we ask the calculator to compute the product P −1 AP , and indeed, we obtain
our matrix J. So, our answer is correct.

8.7 Symmetric matrices and orthogonal diagonalization

As usual, the complex conjugate of a complex number z is denoted by z. Recall from
section 6.1 that the standard scalar product in Rn or Cn , denoted by ·, was defined
as follows:
T T
ˆ for all vectors x = x1 . . . xn in Rn :
 
and y = y1 . . . yn
n
P
x·y = xk yk ;
k=1

T T
ˆ for all vectors x = in Cn :
 
x1 . . . xn and y = y1 . . . yn
n
P
x·y = xk yk .
k=1

Throughout this section, we shall denote by || · || the norm induced by the standard
scalar product · in Rn or Cn (as appropriate). In particular, orthogonality and
orthonormality will always be assumed to be with respect to the standard scalar
product and the induced norm.

8.7.1 Symmetric and Hermitian matrices

For any field F, a matrix A ∈ Fn×n is symmetric if AT = A. If F = C, then it
turns out that symmetric matrices are less interesting than the so-called “Hermitian

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 651

matrices.” For a matrix A = ai,j n×m in Cn×m , we set A = ai,j n×m , i.e. A
   

is an n × m matrix such that for all indices i ∈ {1, . . . , n} and j ∈ {1, . . . , m}, the
i, j-th entry of A is ai,j (the complex conjugate of ai,j ). The Hermitian transpose of
a A is the matrix A∗ = (A)T . For example, for
 
−1 + i 3 2i
A := ,
1 + 2i 4 − 2i 3

we have the following:

 
  −1 − i 1 − 2i
−1 − i 3 −2i ∗
A= , A =  3 4 + 2i  .
1 − 2i 4 + 2i 3
−2i 3

A square matrix A ∈ Cn×n is Hermitian if A∗ = A. For example, the matrix

 
−1 1+i 2−i
 1−i 2 −3 + i 
2+i −3 − i 0

is Hermitian. Note that all entries on the main diagonal of a Hermitian matrix are
real. Note also that if all entries of a matrix in Cn×n happen to be real, then that
matrix is Hermitian if and only if it is symmetric.

Proposition 8.7.1. For all x ∈ Cn , we have that x∗ x = ||x||2 .

T
in Cn , we have that

Proof. For any vector x = x1 . . . xn
 
x1
n
x∗ x = . . . xn  ...  = xk xk = x · x = ||x||2 ,
  P
x1

k=1
xn

which is what we needed.

The basic algebraic properties of the Hermitian transpose are essentially the same
as those of the ordinary transpose, as we can see by comparing Proposition 1.8.1
(which deals with the ordinary transpose) and Proposition 8.7.2 (below).

Proposition 8.7.2. For all matrices A, B ∈ Cn×m and scalars α ∈ C, the following
hold:

(a) (A∗ )∗ = A; (c) (αA)∗ = αA∗ ;

(b) (A + B)∗ = A∗ + B ∗ ; (d) (AB)∗ = B ∗ A∗ .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 652

Proof. This follows from the appropriate definitions. The details are left as an easy
exercise.

Theorem 8.7.3. All eigenvalues of a Hermetrian matrix are real.

Remark: Recall that the field C is algebraically closed, and consequently, every
matrix in Cn×n has n complex eigenvalues (with algebraic multiplicities taken into
account). So, Theorem 8.7.3 states that if A is a Hermitian matrix in Cn×n , then all
n eigenvalues of A (with algebraic multiplicities taken into account) are real.

Proof. Let A ∈ Cn×n be a Hermitian matrix, let λ be any eigenvalue of A, and let
x be an associated eigenvector of A. After possibly normalizing the eigenvector x
x
(i.e. replacing x by ||x|| ), we may assume that x is a unit vector, i.e. that it satisfies
||x|| = 1. Then Ax = λx, and we compute:

x∗ Ax = x∗ (λx) because Ax = λx

= λ(x∗ x)

= λ||x||2 by Proposition 8.7.1

= λ because ||x|| = 1.

But now we have the following:

λ = x∗ Ax

= x ∗ A∗ x because A is Hermitian

= x∗ A∗ (x∗ )∗ by Proposition 8.7.2(a)

= (x∗ Ax)∗ by Proposition 8.7.2(d)

where we consider λ as
= λ∗
a 1 × 1 complex matrix

where we consider λ as
= λ
a complex number.

We have now shown that λ = λ, and it follows that λ is a real number.

Corollary 8.7.4. Every symmetric matrix in Rn×n has n real eigenvalues (with
algebraic multiplicities taken into account). In other words, for every symmetric
matrix A ∈ Rn×n , the sum of algebraic multiplicities of its distinct (real) eigenvalues
is n.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 653

Proof. Consider any symmetric matrix A ∈ Rn×n . If we consider A as a matrix

in Cn×n , then A is Hermitian, and so Theorem 8.7.3 guarantees that all complex
eigenvalues of A are in fact real. Finally, the fact that A has n complex eigenvalues
follows from the fact that C is algebraically closed.

8.7.2 Orthogonal diagonalizability

Recall from section 6.8 that a matrix Q ∈ Rn×n is orthogonal if QT Q = In . By
Theorem 6.8.1, the following are equivalent for any matrix Q in Rn×n :

ˆ Q is orthogonal;

ˆ Q is invertible and satisfies Q−1 = QT ;

ˆ the columns of Q form an an orthonormal basis of Rn .

In what follows, we will repeatedly use the fact that the three statements above are
equivalent, without explicitly mentioning Theorem 6.8.1.
Let us say that a matrix A ∈ Rn×n is orthogonally diagonalizable if there exists a
diagonal matrix D and an orthogonal matrix Q, both in Rn×n , such that D = QT AQ.
Since orthogonal matrices Q are invertible and satisfy QT = Q−1 , we see that
orthogonally diagonalizable matrices are, in particular, diagonalizable in the usual
sense. The main result of this subsection is Theorem 8.7.6, which states that a
matrix in Rn×n is orthogonally diagonalizable if and only if it is symmetric. The
proof proceeds by induction on n, and in the induction step, it will be convenient to
reduce the problem to the case when the matrix has an eigenvalue 0. To this end,
we will use the following technical proposition.

Proposition 8.7.5. Let A ∈ Rn×n and λ0 ∈ R. Then all the following hold:

(a) λ0 is an eigenvalue of A if and only if 0 is an eigenvalue of A − λ0 In , and

moreover, Eλ0 (A) = E0 (A − λ0 In );109

(b) A is symmetric if and only if A − λ0 In is symmetric;

(c) A is diagonalizable if and only if A − λ0 In is digonalizable;

(d) A is orthogonally disagonalizable if and only if A − λ0 In is orthogonally diago-

nalizable.

Proof. (a) For all v ∈ Rn , we have that Av = λ0 v if and only if (A−λ0 In )v = 0 = 0v,
and we deduce that v ∈ Eλ0 (A) if and only if v ∈ E0 (A − λ0 In ). Thus, Eλ0 (A) =
E0 (A − λ0 In ). In particular, Eλ0 (A) is non-trivial if and only if E0 (A − λ0 In ) is
109
Here, Eλ0 (A) = E0 (A − λ0 In ) holds even if λ0 is not an eigenvalue of A. In that case, we simply
have that Eλ0 (A) = E0 (A − λ0 In ) = {0}.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 654

non-trivial, and consequently (by definition, or alternatively, by Proposition 8.1.6(a)),

λ0 is an eigenvalue of A if and only if 0 is an eigenvalue of A − λ0 In .
(b) First, we note that (In )T = AT − λ0 InT = AT − λ0 In ; consequently, (A −
λ0 In )T = A − λ0 In if and only if AT = A, i.e. A − λ0 In is symmetric if and only if
A is symmetric.
(c) Suppose first that A is diagonalizable. Then there exist a diagonal matrix D
and an invertible matrix P , both in Rn×n , such that D = P −1 AP . But then

P −1 (A − λ0 In )P = P −1 AP − P −1 (λ0 In )P

−1 −1
| {zAP} −λ0 P
= P | {z P}
=D =In

= D − λ0 In ,

and obviously, D − λ0 In is diagonal. So, A − λ0 In is diagonalizable. The proof of

the converse is analogous.110
(d) This is completely analogous to the proof of (c), except that instead of P and
P −1 (where P ∈ Rn×n is an invertible matrix), we have Q and QT (where Q ∈ Rn×n
is an orthogonal matrix).

Theorem 8.7.6. A matrix in Rn×n is orthogonally diagonalizable if and only if it is

symmetric.
Proof. Let us first show that orthogonally diagonalizable matrices are symmeteric.
Fix any orthogonally diagonalizable matrix A ∈ Rn×n . Let D be a diagonal and Q
an orthogonal matrix, both in Rn×n , such that D = QT AQ. Then A = QDQT , and
we see that
(∗)
AT = (QDQT )T = (QT )T DT QT = QDQT = A,

where in (*), we used the fact that DT = D, since D is diagonal. Thus, A is

symmetric.
It remains to prove the reverse implication: symmetric matrices in Rn×n are
orthogonally diagonalizable. We proceed by induction on n.
For n = 1, the result is immediate: indeed, if A ∈ R1×1 , then A is diagonal, and
we can take D := A and Q := I1 to obtain D = QT AQ.111
Now, fix a positive integer n, and assume inductively that every symmetric matrix
in Rn×n is orthogonally diagonalizable. Fix any symmetric matrix A ∈ R(n+1)×(n+1) ;
110
Indeed, set A′ := A − λ0 In and λ′0 := −λ0 . Then an argument completely analogous to the
above shows that if A′ = A − λ0 In is diagonalizable, then so is A′ − λ′0 In = A.
111
Obviously, I1 is orthogonal.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 655

we must show that A is orthogonally diagonalizable. By Corollary 8.7.4, A has

n + 1 real eigenvalues (with algebraic multiplicities taken into account). Let λ0 ∈ R
be an eigenvalue of A. In view of Proposition 8.7.5, we may assume that λ0 = 0,
for otherwise, we simply consider A − λ0 In instead of A.112 Let x0 ∈ Rn be an
eigenvector of A associated with the eigenvalue 0, so that Ax0 = 0. After possibly
normalizing the eigenvector x0 (i.e. replacing x0 by ||xx00 || ), we may assume that
||x0 || = 1. Now, using Corollary
 6.3.11(d), we let{x0 , x1 , . . . , xn } be an orthonormal
basis of R n+1 .113 Set S := x0 x1 . . . xn ; then S is an orthogonal matrix.
Now, since A is symmetric, so is S T AS; indeed,

T (∗)
S T AS = S T AT S = S T AS,

where in (*), we used the fact that AT = A (since A is symmetric). Moreover, it is

easy to see that the first (i.e. leftmost) column of S T AS is 0; indeed:

S T AS = S T A x0 x1 . . . xn
 

by the definition of
S T Ax0 S T Ax1 . . . S T Axn
 
=
matrix multiplication

S T 0 S T Ax1 . . . S T Axn
 
= because Ax0 = 0

0 S T Ax1 . . . S T Axn
 
= .

We now know that S T AS ∈ R(n+1)×(n+1) is a symmetric matrix, and that its leftmost
column is 0. So, there exists a symmetric matrix A0 ∈ Rn×n such that

0 0T
 
T
S AS = .
0 A0

By the induction hypothesis, A0 is orthogonally diagonalizable, i.e. there exist

a diagonal matrix D0 and an orthogonal matrix Q0 , both in Rn×n , such that
D0 = QT0 A0 Q0 . Now, set

0 0T 1 0T
   
ˆ D := ; ˆ R := .
0 D0 (n+1)×(n+1) 0 Q0 (n+1)×(n+1)
112
Let us explain this in more detail. By Proposition 8.7.5(a-b), A − λ0 In is symmetric and has
eigenvalue 0. On the other hand, if we can show that A − λ0 In is orthogonally diagonalizable, then
Proposition 8.7.5(d) will guarantee that A is also orthogonally diagonalizable. So, we may consider
A − λ0 In and 0 instead of A and λ0 , respectively.
113
Indeed, {x0 } is an orthonormal basis of the subspace U := Span(x0 ) of Rn+1 , and so by
Corollary 6.3.11(d), {x0 } can be extended to an orthonormal basis of Rn+1 .

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 656

Clearly, D is diagonal (because D0 is diagonal), and R is orthogonal (because

Q0 is orthogonal).114 Since R and S are orthogonal, Proposition 6.8.3 guarantees
that Q := SR is also orthogonal. Finally, we compute:

QT AQ = (SR)T A(SR)

= RT (S T AS)R

1 0T 0 0T 1 0T
   
=
0 QT0 0 A0 0 Q0

0T
 
0
= T
0 Q0 A0 Q0

0 0T
 
=
0 D0

= D,

and we are done.

8.7.3 The spectral theorem for symmetric matrices

For a matrix A ∈ Rn×n , an orthogonal eigenbasis (resp. orthonormal eigenbasis) of
Rn associated with A is an orthogonal (resp. orthonormal) basis of Rn , all of whose
vectors are eigenvectors of A. Theorem 8.7.6, combined with what we know about
diagonalization and orthogonal matrices, readily yields the following theorem.
The spectral theorem for symmetric matrices. For every matrix A ∈ Rn×n ,
the following are equivalent:
(a) A is symmetric;

(b) A is orthogonally diagonalizable;

(c) Rn has an orthonormal eigenbasis associated with A;

(d) Rn has an orthogonal eigenbasis associated with A;

(e) Rn has an eigenbasis associated with A, and the eigenspaces of A are pairwise
orthogonal;
114
Indeed,

0T 0T 0T 0T
      
1 1 1 (∗) 1
RT R = = = = In+1 ,
0 QT0 0 Q0 0 QT Q0 0 In

where (*) follows from the fact that Q is orthogonal.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 657

(f ) A has n pairwise orthogonal eigenvectors.115

Proof. Fix a matrix A. To simplify notation, for each eigenvalue λ of A, we set

Eλ := Eλ (A). To prove the theorem, it is enough to prove the implications shown in
the diagram below.
(a) (b) (c)

(e) (d) (f )

The fact that (a) and (b) are equivalent follows from Theorem 8.7.6. Let us prove
that (d) and (f) are equivalent. Since any basis of Rn contains exactly n vectors, it
is clear that (d) implies (f). Let us now assume (f) and prove (d). Using (f), we fix
pairwise orthogonal eigenvectors v1 , . . . , vn of A. By the definition of an eigenvector,
v1 , . . . , vn are all non-zero. So, by Proposition 6.3.4(a), {v1 , . . . , vn } is an orthogonal
basis of Rn . But now {v1 , . . . , vn } is an orthogonal eigenbasis of Rn associated with
A, and we see that (d) holds.
Next, we assume (b) and prove (e). Using (b), we fix a diagonal matrix D
and an orthogonal matrix Q, both in Rn×n , such that D = QT AQ = Q−1 AQ. By
Proposition 8.5.12, the columns of Q form an eigenbasis of Rn associated with
A, and since Q is orthogonal, this basis is orthonormal. It remains to show that
the eigenspaces of A are pairwise orthogonal. So, suppose that λ1 and λ2 are
distinct eigenvalues of A; we must show that Eλ1 ⊥ Eλ2 . By Proposition 8.5.12,
the eigenvalues of A are precisely the entries on the main diagonal of D, and in
particular, both λ1 and λ2 appear on the main diagonal of D. Now, suppose the
eigenvalue λ1 appears (precisely) in entries i1 , . . . , ik1 of the main diagonal of D;
then by Proposition 8.5.12, columns number i1 , . . . , ik1 of Q form a basis B1 of Eλ1 .
Similarly, suppose that the eigenvalue λ2 appears (precisely) in entries j1 , . . . , jk2 of
the main diagonal of D; then by Proposition 8.5.12, columns number j1 , . . . , jk2 of
Q form a basis B2 of Eλ2 . But since Q is orthogonal, we know that its columns form
an orthonormal basis of Rn . In particular, B1 ⊥ B2 . Proposition 6.1.5 then implies
that Span(B1 ) ⊥ Span(B2 ), that is, Eλ1 ⊥ Eλ2 . This proves (e).
Next, we assume (e) and prove (d). Let λ1 , . . . , λk be the distinct eigenvalues of A,
and for all i ∈ {1, . . . , k}, let Bi be a basis of Eλi ; after possibly applying the Gram-
Schmidt orthogonalization procedure to Bi , we may assume that Bi is orthogonal.
By (e), Rn has an eigenbasis associated with A, and so by Proposition 8.4.5(c),
the sum of geometric multiplicities of A is n, and moreover, B := B1 ∪ · · · ∪ Bk
is an eigenbasis of Rn associated with A. Since the eigenspaces of A are pairwise
orthogonal, we see that B1 , . . . , Bk are orthogonal to each other. Since B1 , . . . , Bk
115
This means that some n eigenvectors of A are pairwise orthogonal. It does not mean that A
has exactly n eigenvectors (which happen to be orthogonal).

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 658

are orthogonal sets of vectors, we deduce that the eigenbasis B = B1 ∪ · · · ∪ Bk is

orthogonal, i.e. (d) holds.
Next, we assume (d) and prove (c). Using (d), we fix an orthogonal eigenbasis
{v1 , . . . , vn } of Rn associated with A. For all i ∈ {1, . . . , n}, set ui := ||vvii || ; since
vi is an eigenvector of A associated with the eigenvalue λi , so is ui . In view of
Proposition 6.3.3(c), we now deduce that {u1 , . . . , un } is an orthogonal eigenbasis of
Rn associated with A. This proves (c).
Finally, we assume (c) and prove (b). Using (c), we fix an orthonormal eigenbasis
{v1 , . . . , vn } of Rn associated with A. For each i ∈ {1, . . . , n}, we let λi be the
eigenvalue
 of A associated
 with the eigenvector vi . Set D := D(λ1 , . . . , λn ) and
Q := v1 . . . vn . By Theorem 8.5.6, A is diagonalizable and satisfies D =
Q−1 AQ. But since the columns of Q form an orthonormal basis of Rn , we see that
the matrix Q is orthogonal and therefore satisfies Q−1 = QT . Thus, D = QT AQ,
and (b) follows.

8.7.4 Diagonally orthogonalizing a symmetric matrix

By Theorem 8.7.6, every symmetric matrix in Rn×n can be orthogonally diagonalized,
and in fact, the proof of Theorem 8.7.6 gives us a recipe of sorts for orthogonally
diagonalizing such a matrix. However, that recipe is not particularly practical, and we
are better off using the spectral theorem from subsection 8.7.3 instead. Suppose we
are given a symmetric matrix A ∈ Rn×n , which we wish to orthogonally diagonalize.
So, our goal is to construct a diagonal matrix D and an orthogonal matrix Q, both
in Rn×n , such that D = QT AQ. We proceed as follows. First, we compute the
characteristic polynomial of A, we factor it, and we find all the (real) eigenvalues of
A along with their algebraic multiplicities. Since A is orthogonally diagonalizable
(and in particular, diagonalizable), Theorems 8.4.5(d) and 8.5.6 together guarantee
that Rn has an eigenbasis associated with A, and moreover, that the sum of algebraic
multiplicities of the eigenvalues of A is n, and that the geometric multiplicity of
each eigenvalue is equal to its algebraic multiplicity. Next, for each eigenvalue
λ of A, we compute a basis Bλ of the eigenspace Eλ (A), and then we apply the
Gram-Schmidt orthogonalization process to Bλ in order to obtain an orthonormal
basis Cλ of Eλ (A). In view of the spectral theorem, we see that the union C of the
Cλ ’s is an orthonormal eigenbasis of Rn associated with A.116 We now form the
diagonal matrix D by placing the eigenvalues of A on the main diagonal of D (while
116
Let us justify this in more detail. Since A is diagonalizable, Theorems 8.4.5(d) and 8.5.6 together
guarantee that C is an eigenbasis of Rn associated with A. Since C is the union of the Cλ ’s, and since
each Cλ is an orthonormal set of vectors, it only remains to explain why the Cλ ’s are orthogonal to
each other. But this follows immediately from the spectral theorem (see subsection 8.7.3): since A
is symmetric, its eigenspaces are orthogonal to each other, and in particular, the bases Cλ of those
eigenspaces are orthogonal to each other.

Irena Penev
Linear Algebra 1 & 2
Chapter 8. Eigenvalues and eigenvectors 659

respecting the algebraic/geometric multiplicity of each eigenvalue),117 and we form

Q by arranging the vectors of our orthonormal eigenbasis C into a matrix (while
respecting the order from D).118 Since the columns of Q form an orthonormal basis
of Rn , we see that Q is orthogonal, and so Q−1 = QT . But now Theorem 8.5.6
guarantees that D = Q−1 AQ = QT AQ.

Example 8.7.7. Orthogonally diagonalize the following symmetric matrix in R3×3 :

 
3 −2 4
A =  −2 6 2 .
4 2 3

Solution. First, we compute the characteristic polynomial of A:

pA (λ) = det(λI3 − A)

λ−3 2 −4
= 2 λ − 6 −2
−4 −2 λ − 3

= λ3 − 12λ2 + 21λ + 98

= (λ + 2)(λ − 7)2 .

Thus, A has two eigenvalues: λ1 = −2 (with algebraic multiplicity 1) and λ2 = 7

 T
(with algebraic multiplicity 2). We now compute a basis B1 = { −2 −1 2 }
 T  T
of Eλ1 (A) and a basis B2 = { −1 2 0 , 1 0 1 } of Eλ2 (A). Next, we
apply the Gram-Schmidt orthogonalization process to B1 and B2 . This yields
T
an orthonormal basis C1 = { − 23 − 31 23

} of Eλ1 , and an orthonormal basis
h iT h iT
1 2 4 2 5
C 2 = { − √5 √5 0 , 3 √5 3 √5 3 √5 } of Eλ2 . We now set
  √ √ 
−2 0 0 −2/3 −1/√5 4/(3√5)
D :=  0 7 0  and Q :=  −1/3 2/ 5 2/(3√5)  .
0 0 7 2/3 0 5/(3 5)

Now D is diagonal, Q is orthogonal, and D = QT AQ.

117
In other words, if {λ1 , . . . , λn } is the spectrum of A, then we set D := D(λ1 , . . . , λn ). Recall
that the geometric multiplicity of any eigenvalue of A is equal to the algebraic multiplicity of that
eigenvalue.
118
So, if an eigenvalue λ of A has algebraic/geometric multiplicity k and appears (precisely) in
entries i1 , . . . , ik of the main diagonal of D, then columns number i1 , . . . , ik of Q should be the
vectors of the orthonormal basis Cλ of the eigenspace Eλ (A).

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 660

Chapter 9

Bilinear and quadratic forms

9.1 A formula for products of the form xT Ay

For a matrix A ∈ Fn×n and vectors x, y ∈ Fn (where F is some field), the product
xT Ay is a 1 × 1 matrix. Whenever convenient, we will treat 1 × 1 matrices simply
as scalars. Moreover, the following proposition gives a formula for products of the
form xT Ay, which we will use throughout the chapter.

Proposition 9.1.1.  F be a field, let En = {e1 , . . . , en } be the standard basis of

Let
Fn , and let A = ai,j n×n be a matrix in Fn×n . Then both the following hold:


T T
in Fn , we have
 
(a) for all vectors x = x1 . . . x n and y = y1 . . . y n
that
n P
n
xT Ay =
P
ai,j xi yj ;
i=1 j=1

(b) for all indices i, j ∈ {1, . . . , n}, we have that eTi Aej = ai,j .

Proof. Obviously, (a) implies (b). So, let us prove (a). For any vectors x =
T T
in Fn , we have the following:
 
x1 . . . x n and y = y1 . . . yn
  
a1,1 a1,2 . . . a1,n y1
 a2,1 a2,2 . . . a2,n  y2 
xT Ay

= x1 x2 . . . xn
  
 .. .. .. ..  .. 
 . . . .  . 
an,1 an,2 . . . an,n yn
 Pn 
a1,j yj
Pj=1
n
j=1 a2,j yj
 
 
= x1 x2 . . . xn 

.. 
Pn .
 
j=1 an,j yj

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 661

n
P n
P  n P
P n
= xi ai,j yj = ai,j xi yj .
i=1 j=1 i=1 j=1

This proves (a).

9.2 Bilinear forms

A bilinear form on a vector space V over a field F is a function f : V × V → F that
satisfies the following four axioms:

b.1. for all x1 , x2 , y ∈ V , we have that f (x1 + x2 , y) = f (x1 , y) + f (x2 , y);

b.2. for all x, y ∈ V and α ∈ F, we have that f (αx, y) = αf (x, y);

b.3. for all x, y1 , y2 ∈ V , we have that f (x, y1 + y2 ) = f (x, y1 ) + f (x, y2 );

b.4. for all x, y ∈ V and α ∈ F, we have that f (x, αy) = αf (x, y).

The bilinear form f is said to be symmetric if it further satisfies the property that
f (x, y) = f (y, x) for all x, y ∈ V .

Remark: In section 6.1, we defined scalar products in real and complex vector spaces.
We note that every scalar product ⟨·, ·⟩ in a real vector space V is a symmetric
bilinear form. Indeed, r.2, r.3, r.2’, and r.3’ (see subsection 6.1.1) are precisely the
axioms b.1, b.2, b.3, and b.4, respectively. Moreover, by r.4, scalar products in
real vector spaces are symmetric. However, scalar products in non-trivial complex
vector spaces are not bilinear forms, since c.1 and c.3’ (see subsection 6.1.2) together
contradict axiom b.4.1 Indeed, if ⟨·, ·⟩ is a scalar product in a non-trivial complex
vector space V , then for any x ∈ V \ {0}, c.1 guarantees that ⟨x, x⟩ = ̸ 0, and so
c.3’
⟨x, ix⟩ = i⟨x, x⟩ = −i⟨x, x⟩ ̸= i⟨x, x⟩,

and we see that b.4 does not hold.

Proposition 9.2.1. Let V be a vector space over a field F, and let f be a bilinear
form on V . Then all the following hold:

(a) for all x ∈ V , we have that f (x, 0) = 0;

(b) for all y ∈ V , we have that f (0, y) = 0;

(c) f (0, 0) = 0.
1
If our complex vector space V is trivial, then there is no contradiction. Indeed, in this case,
there is exactly one scalar product in V = {0}, namely, the function ⟨·, ·⟩ : V × V → C given by
⟨0, 0⟩ = 0. This is obviously a symmetric bilinear form, but it is not very interesting!

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 662

Proof. For (a), we fix a vector x ∈ V , and we compute:

b.3
f (x, 0) = f (x, 0 + 0) = f (x, 0) + f (x, 0).

By subtracting f (x, 0) from both sides, we obtain 0 = f (x, 0). This proves (a). The
proof of (b) is similar. Finally, (c) is a special case of (a) for x = 0.

Theorem 9.2.2. Let V be a non-trivial, finite-dimensional vector space over a field

F, and let B = {b1 , . . . , bn } be a basis of V .

(a) For every matrix A = ai,j n×n in Fn×n , the function f : V × V → F given by
 

 T  
f (x, y) = x B
A y B
for all x, y ∈ V

is a bilinear form on V , and moreover, all the following hold:

(a.1) f (bi , bj ) = ai,j for all i, j ∈ {1, . . . , n},

Pn n
P  P n Pn
(a.2) f ci bi , dj bj = ai,j ci dj for all c1 , . . . , cn , d1 , . . . , dn ∈ F,
i=1 j=1 i=1 j=1
(a.3) f is symmetric if and only if A is symmetric.
 
(b) For every bilinear form f on V , there exists a unique matrix A = ai,j n×n in
Fn×n , called the matrix of the bilinear form f with respect to the basis B, that
satisfies the property that
 T  
f (x, y) = x B
A y B
for all x, y ∈ V .

Moreover, the entries of the matrix A are given by ai,j = f (bi , bj ) for all indices
i, j ∈ {1, . . . , n}.

Proof. (a) Fix a matrix A = ai,j n×n in Fn×n , and define f : V × V → F by

 

setting
 T  
f (x, y) = x B A y B for all x, y ∈ V .
Let us first check that f is bilinear. We must check that f satisfies axioms b.1-b.4.
For b.1, we observe that for all vectors x1 , x2 , y ∈ V , we have the following:
 T  
f (x1 + x2 , y) = x1 + x2 B
A y B

(∗)
    T  
= x1 B
+ x2 B
A y B

 T     T   
= x1 B
A y B
+ x2 B
A y B

= f (x1 , y) + f (x2 , y),

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 663

 
where (*) follows from the linearity of · B . Thus, f satisfies b.1, and similarly, it
satisfies b.3. For b.2, we observe that for all vectors x, y ∈ V and scalars α ∈ F, we
have the following:
 T  
f (αx, y) = αx B
A y B

(∗)
   T  
= α x B A y B
 T   
= α x B
A y B

= αf (x, y),
 
where (*) follows from the linearity of · B . Thus, f satisfies b.2, and similarly, it
satisfies b.4. This proves that f is indeed bilinear.
Next, to prove (a.1), we fix indices i, j ∈ {1, . . . , n}, and we compute:
T (∗)
= eTi Aej
  
f (bi , bj ) = bi B
A bj B
= ai,j ,

where (*) follows from Proposition 9.1.1(b).

For (a.2), we fix scalars c1 , . . . , cn , d1 , . . . , dn ∈ F, and we compute:
n
P n
P  (∗) n P
P n (a.1) n P
P n
f ci bi , dj bj = ci dj f (bi , bj ) = ai,j ci dj ,
i=1 j=1 i=1 j=1 i=1 j=1

where (*) follows from the fact that f is bilinear.

It remains to prove (a.3). Suppose first that A is symmetric. Then for all
x, y ∈ V , we have that
 T  
f (x, y) = x B
A y B

(∗)
 T   T
= x B
A y B

T
AT
  
= y B
x B

(∗∗)  T  
= y B
A x B

= f (y, x),
 T  
where (*) follows from the fact that x B A y B is a 1 × 1 matrix (and is
therefore symmetric), and (**) follows from the fact that A is symmetric. So, f is
symmetric.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 664

Suppose, conversely, that f is symmetric. Then for all indices i, j ∈ {1, . . . , n},
we have the following:
(a.1) (∗) (a.1)
ai,j = f (bi , bj ) = f (bj , bi ) = aj,i ,

where (*) follows from the fact that f is symmetric. So, A is symmetric.
 
(b) Fix a bilinear form f on V . First of all, if A = ai,j n×n is any matrix in
T
Fn×n that satisfies the property that f (x, y) = x B A y B for all x, y ∈ V ,
  

then (a) guarantees that ai,j = f (bi , bj ) for all indices i, j ∈ {1, . . . , n}. This, in
particular, proves the uniqueness part of (b).  
For existence, we must show that the matrix A = ai,j n×n given by the formula
ai,j = f (bi , bj ) for all indices i, j ∈ {1, . . . , n}, does indeed satisfy the property that
 T  
f (x, y) = x B A y B for all x, y ∈ V . So, fix vectors x, y ∈ V . Since B =
{b1 , . . . , bn } is a basis of V , we know that there exist scalars c1 , . . . , cn , d1 , . . . , dn ∈ F
T
such that x = ni=1 ci bi and y = nj=1 dj bj , so that x B = c1 . . . cn
P P   
and
   T
y B = d1 . . . dn . We then compute:
n
P n
P 
f (x, y) = f ci bi , dj bj
i=1 j=1

(∗) n P
P n
= ci dj f (bi , bj )
i=1 j=1 | {z }
=ai,j

n P
P n
= ai,j ci dj
i=1 j=1

(∗∗)  T  
= x B
A y B
,

where (*) follows from the fact that f is bilinear, and (**) follows from Proposi-
tion 9.1.1(a).

Corollary 9.2.3. Let F be a field, and let En = {e1 , . . . , en } be the standard basis
of Fn .

(a) For every matrix A = ai,j n×n in Fn×n , the function f : Fn × Fn → F given
 

by
f (x, y) = xT Ay for all x, y ∈ Fn
is a bilinear form on Fn , and moreover, all the following hold:

(a.1) f (ei , ej ) = ai,j for all i, j ∈ {1, . . . , n},

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 665

n P
P n  T
(a.2) f (x, y) = ai,j xi yj for all vectors x = x1 . . . xn and y =
i=1 j=1
T
Fn ,

y1 . . . yn in
(a.3) f is symmetric if and only if A is symmetric.

(b) For every bilinear form f on Fn , there exists a unique matrix A =

 
ai,j n×n
in Fn×n that satisfies the property that

f (x, y) = xT Ay for all x, y ∈ Fn .

Moreover, the entries of the matrix A are given by ai,j = f (ei , ej ) for all indices
i, j ∈ {1, . . . , n}.

Proof. This is simply a special case of Theorem 9.2.2 for V := Fn and B := En .

Remark: Corollary 9.2.3 implies that, for a field F, the bilinear forms on Fn are
precisely the functions f : Fn × Fn → F given by
   
x1 y1
n Pn
for all x =  ...  and y =  ..  in Fn ,
P
f (x, y) = ai,j xi yj
  
. 
i=1 j=1
xn yn

where the ai,j ’s are some scalars in F. Moreover, such a bilinear form is symmetric
if and only if ai,j = aj,i for all indices i, j ∈ {1, . . . , n}. The
 matrix of this bilinear
form with respect to the standard basis En of Fn is ai,j n×n (so, the i, j-th entry
of the matrix is the coefficient in front of xi yj from the formula for f above). For
example, functions f1 , f2 : R2 × R2 → R given by the formulas

ˆ f1 (x, y) = x1 y1 − 3x1 y2 − 3x2 y1 + 7x2 y2 ,

ˆ f2 (x, y) = x1 y1 − 2x1 y2 + 3x2 y1 − 3x2 y2 ,

T T
in R2 , are bilinear forms on R2 . The
 
for all x = x1 x2 and y = y1 y2
bilinear form f1 is symmetric, whereas the bilinear form f2 is not. The matrices
of thebilinear forms
 f1 and f2 with respect
 to the standard basis E2 of R2 are
1 −3 1 −2
A1 = and A2 = , respectively. Note that A1 is symmetric,
−3 7 3 −3
whereas A2 is not; this is consistent with the fact that f1 is symmetric, whereas f2
is not.

Theorem 4.3.2 essentially states that a linear function can be fully determined by
specifying what the vectors of some basis of the domain get mapped to. For bilinear
forms, Theorem 9.2.2 yields the following analogous result.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 666

Corollary 9.2.4. Let V be a non-trivial, finite-dimensional  vector

 space over a
field F, let B = {b1 , . . . , bn } be a basis of V , and let A = ai,j n×n be a matrix in
Fn×n . Then there exists a unique bilinear form f on V that satisfies the property
that f (bi , bj ) = ai,j for all indices i, j ∈ {1, . . . , n}. Moreover, the matrix of this
bilinear form with respect to the basis B is precisely the matrix A.

Proof. Existence. By Theorem 9.2.2(a), the function f : V × V → F given by

 T  
f (x, y) = x B
A y B
for all x, y ∈ V

is bilinear, and moreover, part (a.1) of Theorem 9.2.2(a) guarantees that f (bi , bj ) =
ai,j for all indices i, j ∈ {1, . . . , n}. Clearly, A is the matrix of the bilinear form f
with respect to the basis B.
Uniqueness. Suppose that f ′ is any bilinear form on V that satisfies f ′ (bi , bj ) = ai,j
for all i, j ∈ {1, . . . , n}. Then Theorem 9.2.2(b) guarantees that the matrix of the
′

bilinear form f with respect to the basis B is precisely the matrix A = ai,j n×n ,
T
i.e. f ′ (x, y) = x B A y B for all x, y ∈ V .
  

Theorem 9.2.5 (Change of basis for bilinear forms). Let V be a non-trivial, finite-
dimensional vector space over a field F, let f be a bilinear form on V , and let B and
C be bases of V . Further, let B be the matrix of f with respect to B, and let C be
the matrix of f with respect to C. Then
 T  
C = B
IdV C
B B
IdV C
.

Proof. For all x, y ∈ V , we have that

(∗)  T  
f (x, y) = x B
B y B

     T      
= B
IdV C
x C
B B
IdV C
y C

 T   T    
= x C B
IdV C
B B
IdV C
y C
,

where (*) follows from the fact that B is the matrix of the bilinear form f with
respect to the basis B. But now we have that
 T  
B
IdV C
B B
IdV C

is the matrix of the bilinear form f with respect to the basis C, that is, C =
T
IdV C B B IdV C .2
  
B
2
Note that we are using the uniqueness part of Theorem 9.2.2(b).

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 667

9.2.1 Matrix congruence

Let F be a field. A matrix A ∈ Fn×n is said to be congruent to a matrix B ∈ Fn×n
if there exists an invertible matrix P ∈ Fn×n such that B = P T AP . Like matrix
similarity (see Proposition 4.5.16), matrix congruence is an equivalence relation on
Fn×n , as we now show.

Proposition 9.2.6. Let F be a field. Then all the following hold:

(a) for all matrices A ∈ Fn×n , A is congruent to A;

(b) for all matrices A, B ∈ Fn×n , if A is congruent to B, then B is congruent to A;

(c) for all matrices A, B, C ∈ Fn×n , if A is congruent to B and B is congruent to

C, then A is congruent to C.

Proof. (a) Fix a matrix A ∈ Fn×n . Then A = InT AIn , and it follows that A is
congruent to itself.
(b) Fix a matrices A, B ∈ Fn×n , and assume that A is congruent to B. Then there
exists an invertible matrix P ∈ Fn×n such that B = P T AP . By Proposition 1.11.8(c),
P T is invertible and satisfies (P T )−1 = (P −1 )T . So,

A = (P T )−1 BP −1 = (P −1 )T BP −1 ,

and it follows that B is congruent to A.

(c) Fix matrices A, B, C ∈ Fn×n , and assume that A is congruent to B and that
B is congruent to C. Then there exist invertible matrices P, Q ∈ Fn×n such that
B = P T AP and C = QT BQ. We now compute:

C = QT BQ

= QT (P T AP )Q

= (QT P T )A(P Q)

= (P Q)T A(P Q).

By Proposition 1.11.8(d), the matrix P Q is invertible, and we deduce that A is

congruent to C.

Remark: By Proposition 9.2.6(b), the congruence relation on Fn×n (where F is a

field) is symmetric. Consequently, we may speak of matrices A, B ∈ Fn×n as being
congruent or not being congruent to each other. In particular, we may write
something like “let A, B ∈ Fn×n be congruent matrices.” This means that A is
congruent to B and vice versa.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 668

Theorem 4.5.19 essentially states that two square matrices are similar if and only
if they represent the same linear function, but possibly with respect to different
bases. Theorem 9.2.7 (below) is an analog of Theorem 4.5.19 for congruent matrices:
it states that two square matrices are congruent if and only if they represent the
same bilinear form, but possibly with respect to different bases.

Theorem 9.2.7. Let F be a field, let B, C ∈ Fn×n be matrices, and let V be an

n-dimensional vector space over the field F. Then the following are equivalent:

(a) B and C are congruent;

(b) for all bases B of V and bilinear forms f on V such that B is the matrix of f
with respect to B, there exists a basis C of V such that C is the matrix of f with
respect to C;

(c) there exist bases B and C of V and a bilinear form f on V such that B is the
matrix of f with respect to B, and C is the matrix of f with respect to C.

Proof. We will prove the implications “(a) =⇒ (b) =⇒ (c) =⇒ (a).”

We first assume (a) and prove (b). By (a), there exists an invertible matrix
P ∈ Fn×n such that C = P T BP . Now, to prove (b), we fix a basis B of V and a
bilinear form f on V such that B is the matrix of f with respect to B. Since P is
invertible,
 Proposition
 4.5.15 guarantees that there exists a basis C of V such that
P = B IdV C . But then Theorem 9.2.5 guarantees that the matrix of the bilinear
form f with respect to the basis C is precisely the matrix
T
= P T BP
  
B
IdV C
B B
IdV C
= C.

This proves (b).

Next, we assume (b) and prove (c). Fix any basis B of V , and define f : V ×V → F
 T  
by setting f (x, y) = x B B y B for all x, y ∈ V . By Theorem 9.2.2, f is a
bilinear form on V , and obviously, B is the matrix of f with respect to the basis B.
Using (b), we now fix a basis C of V such that C is the matrix of the bilinear form
f with respect to C. We have now constructed bases B and C of V , and a bilinear
form f on V , such that B is the matrix of f with respect to B, and C is the matrix
of f with respect to C. This proves (c).
Finally, we assume (c) and prove (a). Using (c), we fix bases B and C and a
 of f with respect to B, and C is
bilinear form f on V such that B is the matrix
the matrix of f with respect to C. Set P := B IdV C . By Proposition 4.5.15, P is
invertible, and by Theorem 9.2.5, we have that C = P T BP . This proves (a).

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 669

9.2.2 Symmetric bilinear forms on vector spaces over fields of char-

acteristic other than 2
Recall from subsection 2.4.4 that the characteristic of a field F, denoted by char(F),
is the smallest positive integer n (if it exists) such that
1 + · · · + 1 = 0,
| {z }
n

where the 1’s and 0’s are understood to be in the field F. If no such n exists, then
char(F) := 0. Note that fields Q, R, and C all have characteristic 0. On the other
hand, for all prime numbers p, we have that char(Zp ) = p. By Theorem 2.4.5, the
characteristic of any field is either 0 or a prime number.
Proposition 9.2.8 (below) applies to bilinear forms over vector spaces of character-
istic other than 2. In such fields, we can divide by 2 := 1 + 1, since 2 = 1 + 1 ̸= 0. The
only field of characteristic 2 that we have seen is Z2 , but other fields of characteristic
2 do exist.

Proposition 9.2.8. Let f and g be symmetric bilinear forms on a vector space V

over a field F of characteristic other than 2, and assume that for all x ∈ V , we have
that f (x, x) = g(x, x). Then f = g.
Proof. Fix x, y ∈ V . We must show that f (x, y) = g(x, y). By hypothesis, all the
following hold:
(1) f (x, x) = g(x, x);
(2) f (y, y) = g(y, y);
(3) f (x + y, x + y) = g(x + y, x + y).
On the other hand, since f and g are bilinear, we have that
(4) f (x + y, x + y) = f (x, x) + f (x, y) + f (y, x) + f (y, y);
(5) g(x + y, x + y) = g(x, x) + g(x, y) + g(y, x) + g(y, y).
By (1)-(5), it follows that f (x, y) + f (y, x) = g(x, y) + g(y, x). But since f and g
are symmetric, we further have that f (x, y) = f (y, x) and g(x, y) = g(y, x), and it
follows that 2f (x, y) = 2g(x, y). Since char(F) ̸= 2 (and consequently, 2 = 1 + 1 ̸= 0
in our field F), we deduce that f (x, y) = g(x, y).

9.3 Quadratic forms

9.3.1 Quadratic forms and their matrices
A quadratic form on a vector space V over a field F is any function q : V → F
for which there exists a bilinear form f : V × V → F such that q(x) = f (x, x)

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 670

for all x ∈ V . Quadratic forms are defined for vector spaces over fields of any
characteristic. However, in all our results below, we assume that the field in question
is of characteristic other than 2, so that we can divide by 2.

Theorem 9.3.1. Let q be a quadratic form on a vector space V over a field F of

characteristic other than 2. Then there exists a unique symmetric bilinear form
f on V such that for all x ∈ V , we have that q(x) = f (x, x). Furthermore, if the
vector space V is non-trivial and finite-dimensional, then for any basis B of V , there
exists a unique symmetric matrix A ∈ Fn×n such that
 T  
q(x) = x B A x B for all x ∈ V ,

and moreover, this unique symmetric matrix A is precisely the matrix of the symmetric
bilinear form f with respect to the basis B.
Terminology: The symmetric matrix A from the statement of Theorem 9.3.1 is
called the matrix of the quadratic form q with respect to the basis B. For emphasis,
we may optionally refer to A as the symmetric matrix of the quadratic form q with
respect to the basis B.
Warning: There may possibly exist more than one matrix A ∈ Fn×n that satisfies
 T  
the property that q(x) = x B A x B for all x ∈ V . However, only one such
matrix is symmetric. This (unique) symmetric matrix is the one that we refer to as
the matrix of q with respect to B.

Proof. We first prove the existence and uniqueness of the symmetric bilinear form f .
By the definition of a quadratic form, there exists some bilinear form h on V such
that for all x ∈ V , we have that q(x) = h(x, x). Now, using the fact that char(F) ̸= 2,
we define f : V × V → F by setting
 
f (x, y) = 21 h(x, y) + h(y, x) for all x ∈ V .

It is then straightforward to check that f is a symmetric bilinear form on V .3

Moreover, for all x ∈ V , we have that
 
q(x) = h(x, x) = 12 h(x, x) + h(x, x) = f (x, x),

which is what we needed. This completes the proof of existence. Uniqueness follows
immediately from Proposition [Link]
Let us now assume that the vector space V is non-trivial and finite-dimensional,
and let B be a basis of V . Let A ∈ Fn×n be the matrix of the bilinear form f with
3
Details?
4
This is “obvious,” but here are the details. Suppose that f1 and f2 are symmetric bilinear forms
on V such that q(x) = f1 (x, x) and q(x) = f1 (x, x) for all x ∈ V . Then f1 (x, x) = f2 (x, x) for all
x ∈ V . But then by Proposition 9.2.8, we have that f1 = f2 .

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 671

respect to the basis B; by Theorem 9.2.2, the matrix A is symmetric. Obviously,

 T  
q(x) = f (x, x) = x B A x B for all x ∈ V .
For uniqueness, suppose that A′ ∈ Fn×n is a symmetric matrix such that
T
A′
  
q(x) = x B
x B
for all x ∈ V .

We must show that A′ = A. Define f ′ : V × V → F by setting

T
f ′ (x, y) = A′
  
x B
y B
for all x, y ∈ V .

By Theorem 9.2.2(a), f ′ is a symmetric bilinear form. But then for all x ∈ V , we have
that f ′ (x, x) = q(x) = f (x, x), and so by Proposition 9.2.8, f ′ = f . The uniqueness
part of Theorem 9.2.2(b) now guarantees that A′ = A, and we are done.

Remark: Let F be a field. Then quadratic forms q on Fn are all of the form
n P
n T
in Fn ,
P 
q(x) = bi,j xi xj for all x = x1 . . . xn
i=1 j=1

where the bi,j ’s are some elements of F.5 If char(F) ̸= 2, then the matrix of
such a quadratic form q with respect to the standard basis En of Fn is the matrix
A = ai,j n×n whose entries are given by ai,j = 12 (bi,j +bj,i ) for all i, j ∈ {1, . . . , n}.6


Indeed, by construction, A is symmetric, and we see that for all vectors x =

T
in Fn , we have the following:

x1 . . . x n

(∗) n P
n
xT Ax
P
= ai,j xi xj
i=1 j=1

n P
n
P 1
= 2 (bi,j + bj,i )xi xj
i=1 j=1

n P
 P n n P
n

1

P
= 2 bi,j xi xj + bj,i xi xj
i=1 j=1 i=1 j=1

n P
P n
= bi,j xi xj = q(x),
i=1 j=1

where (*) follows from Proposition 9.1.1(a).

5
This is because for every quadratic form q on Fn , there exists a bilinear form f on Fn×n such
that q(x) = f (x, x) for all x ∈ Fn . By Corollary 9.2.3(b), the bilinear form f is of the form
T T
f (x, y) = n
P Pn  
i=1 j=1 bi,j xi yj for all vectors x = x1 . . . xn and y = y1 . . . yn in
n
F , where the bi,j ’s are some elements of F.
6
In particular, we have that ai,i = 12 (bi,i + bi,i ) = bi,i for all i ∈ {1, . . . , n}.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 672

Example 9.3.2. Consider the quadratic form q on R3 given by

q(x) = 3x21 + 2x1 x2 − 4x1 x3 + 5x22 − 6x2 x3 + 2x33
T
in R3 . Then the matrix of q with respect to the

for all vectors x = x1 x2 x3
3
standard basis E3 of R is the matrix
 
3 1 −2
A :=  1 5 −3  .
−2 −3 2
Corollary 9.3.3 (Change of basis for quadratic forms). Let V be a non-trivial,
finite-dimensional vector space over a field F of characteristic other than 2, let q
be a quadratic form on V , and let B and C be bases of V . Further, let B be the
(symmetric) matrix of q with respect to B, and let C be the (symmetric) matrix of q
with respect to C. Then
 T  
C = B IdV C B B IdV C .
Proof. By Theorem 9.3.1, there exists a unique symmetric bilinear form f on V such
that for all x ∈ V , we have that q(x) = f (x, x). Theorem 9.3.1 further guarantees
that B (resp. C) is the matrix of the bilinear form f with respect to the basis B
(resp. C) of Fn . The result now follows immediately from Theorem 9.2.5.

9.3.2 Matrix congruence and quadratic forms

The following theorem is an analogue of Theorem 9.2.7 for symmetric matrices and
quadratic forms.
Theorem 9.3.4. Let F be a field of characteristic other than 2, let B, C ∈ Fn×n be
symmetric matrices, and let V be an n-dimensional vector space over the field F.
Then the following are equivalent:
(a) B and C are congruent;
(b) for all bases B of V and quadratic forms q on V such that B is the matrix of q
with respect to B, there exists a basis C of V such that C is the matrix of q with
respect to C;
(c) there exist bases B and C of V and a quadratic form q on V such that B is the
matrix of q with respect to B, and C is the matrix of q with respect to C.
Proof. We will prove the implications “(a) =⇒ (b) =⇒ (c) =⇒ (a).”
We first assume (a) and prove (b). Suppose that B is a basis of V and q a quadratic
form on V such that B is the matrix of q with respect to B. Using Theorem 9.3.1,
we fix the (unique) symmetric bilinear form f on V such that q(x) = f (x, x) for all
x ∈ V ; Theorem 9.3.1 guarantees that B is the matrix of the bilinear form f with
respect to B. Then by Theorem 9.2.7,7 we know that there exists a basis C such that
7
We are using the “(a) =⇒ (b)” implication from Theorem 9.2.7.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 673

C is the matrix of the bilinear form f with respect to C. But then once again by
Theorem 9.3.1, we have that C is the matrix of q with respect to C. This proves (b).
Next, we assume (b) and prove (c). Fix any basis B of V , and define f : V ×V → F
 T  
by setting f (x, y) = x B B y B for all x, y ∈ V . By Theorem 9.2.2, f is a
symmetric bilinear form on V , and obviously, B is the matrix of f with respect to B.
But then by Theorem 9.3.1, B is the matrix of the quadratic form q with respect to
B. So, by (b), there exists a basis C of V such that C is the matrix of the quadratic
form q with respect to C. We have now constructed bases B and C of V , and a
quadratic form q on V , such that B is the matrix of q with respect to B, and C is
the matrix of q with respect to C. This proves (c).
Finally, we assume (c) and prove (a). Using (c), we fix bases B and C of V , as
well as a quadratic form q on V , such that B is the matrix of q with respect to B,
and C is the matrix of q with respect to C. Using Theorem 9.3.1, we fix the (unique)
symmetric bilinear form f on V such that q(x) = f (x, x) for all x ∈ V . But then
Theorem 9.3.1 guarantees that B is the matrix of the bilinear form f with respect
to B, and that C is the matrix of the bilinear form f with respect to C. So, by
Theorem 9.2.7,8 B and C are congruent, i.e. (a) holds.

9.4 Quadratic forms on Rn

Throughout this section, orthogonality and orthonormality in Rn are assumed to be
with respect to the standard scalar product · and the induced norm || · ||.

9.4.1 Matrix congruence and the signature of a symmetric matrix

with real entries
By Corollary 8.7.4, any symmetric matrix in Rn×n has n real eigenvalues (when
algebraic multiplicities are taken into account). With this in mind, we define the
signature of a symmetric matrix A ∈ Rn×n to be the ordered triple (n+ , n− , n0 ),
where

ˆ n+ is the number of positive eigenvalues of A (counting algebraic multiplicities),

ˆ n− is the number of negative eigenvalues of A (counting algebraic multiplicities),

ˆ n0 := n − n+ − n− .

Note that 0 is an eigenvalue of A if and only if n0 > 0, and in this case, the algebraic
multiplicity of the eigenvalue 0 is precisely n0 . For example, if the spectrum of a
symmetric matrix in R9×9 is {0, 0, 1, 1, −2, −2, 5, 6, −7}, then the signature of that
matrix is (4, 3, 2).
8
We are using the “(c) =⇒ (a)” implication from Theorem 9.2.7.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 674

The main goal of this subsection is to prove Theorem 9.4.3, which states that
two symmetric matrices in Rn×n are congruent if and only if they have the same
signature. We begin by proving Proposition 9.4.1, which essentially states that any
symmetric matrix with real entries is congruent to a diagonal matrix with only
1’s, −1’s, and 0’s on the main diagonal, where the number of 1’s, −1’s, and 0’s is
determined by the signature of the symmetric matrix that we started with.
Proposition 9.4.1. Let A be a symmetric matrix in Rn×n with signature (n+ , n− , n0 ).
Then there exists an invertible matrix R ∈ Rn×n with pairwise orthogonal columns
such that
RT AR = D 1, . . . , 1, −1, . . . , −1, 0, . . . , 0 .


| {z } | {z } | {z }
n+ n− n0

Proof. By the spectral theorem for symmetric matrices (see subsection 8.7.3), we know
that A is orthogonally diagonalizable. So, let D = D(λ1 , . . . , λn ) be a diagonal and
Q an orthogonal matrix, both in Rn×n , such that D = QT AQ. By Proposition 8.5.12,
{λ1 , . . . , λn } is the spectrum of A. After possibly permuting the λi ’s and the
corresponding columns of the orthogonal matrix Q, we may assume that the first n+
many λi ’s are positive, the subsequent n− many λi ’s are negative, and the final n0
many λi ’s are 0.9 Now, set
( 1
√ if λi ̸= 0
|λi |
ℓi :=
1 if λi = 0
9
More precisely, we are assuming the following:
ˆ for all i ∈ {1, . . . , n+ }, we have that λi > 0;
ˆ for all i ∈ {n+ + 1, . . . , n+ + n− }, we have that λi < 0;
ˆ for all i ∈ {n+ + n− + 1, . . . , n}, we have that λi = 0.
The fact that we may assume this essentially follows from our recipe for orthogonally diagonalizing
a symmetric matrix, which allows us to place the eigenvalues of A in any order we like on the
diagonal matrix D (as long as we respect algebraic multiplicities, and as long as we take care to
place the suitable eigenvectors of A in Q in the corresponding order). However, let us give a fully
formal justification for why we may permute the λi ’s in this way. We use permutation matrices (see
subsection 2.3.7). Since {λ1 , . . . , λn } is the spectrum of A, we know that the number of positive,
negative, and zero λi ’ (counting repetitions) is exactly n+ , n− , and n0 , respectively. So, there exists
a permutation σ ∈ Sn such that all the following hold:
ˆ for all i ∈ {1, . . . , n+ }, we have that λσ(i) > 0;
ˆ for all i ∈ {n+ + 1, . . . , n+ + n− }, we have that λσ(i) < 0;
ˆ for all i ∈ {n+ + n− + 1, . . . , n}, we have that λσ(i) = 0.
As usual, Pσ is the matrix of the permutation σ. But then by Theorem 2.3.15, we have that
Pσ DPσT = D(λσ(1) , . . . , λσ(n) ). Moreover, by Proposition 2.3.14, Pσ and PσT are orthogonal, and so
since Q is also orthogonal, Proposition 6.8.3 guarantees that QPσT is orthogonal. But note that

(QPσT )T A(QPσT ) = Pσ (QT AQ) PσT = Pσ DPσT = D(λσ(1) , . . . , λσ(n) ),

| {z }
=D

which is what we needed.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 675

for all indices i ∈ {1, . . . , n}, and set L := D(ℓ1 , . . . , ℓn ) and R := QL. Since both Q
and L are invertible,10 so is R. Moreover, since L is diagonal, Proposition 8.5.1(b)
guarantees that the columns of R = QL are scalar multiples of the columns of Q; since
the columns of Q are pairwise orthogonal (by Theorem 6.8.1),11 Proposition 6.1.4(b)
guarantees that the columns of R are pairwise orthogonal. Finally, we compute:

RT AR = (QL)T A(QL)

= LT QT AQ L
| {z }
=D

= LT DL

(∗)
= LDL

= D(ℓ1 , . . . , ℓn ) D(λ1 , . . . , λn ) D(ℓ1 , . . . , ℓn )

(∗∗)
= D(λ1 ℓ21 , . . . , λn ℓ2n ),

(∗∗∗)

= D 1, . . . , 1, −1, . . . , −1, 0, . . . , 0 ,
| {z } | {z } | {z }
n+ n− n0

where (*) follows from the fact that L is diagonal and therefore symmetric, (**)
follows from Proposition 8.5.2, and (***) follows from the fact that, by construction,

 1 if λi > 0
λi ℓ2i = −1 if λi < 0
0 if λi = 0


for all indices i ∈ {1, . . . , n}, plus the fact that the first n+ many λi ’s are positive,
the subsequent n− many λi ’s are negative, and the final n0 many λi ’s are zero.

Example 9.4.2. Consider the following symmetric matrix in R4×4 :

 
0 2 1 1
 2 0 −1 −1 
A =   1 −1
.
0 −2 
1 −1 −2 0
10
Since Q is orthogonal, Theorem 6.8.1 guarantees that it is invertible. On the other hand, L is a
diagonal matrix, and all its entries on the main diagonal are non-zero; so, by Proposition 8.5.3(b),
L is invertible.
11
By Theorem 6.8.1, the columns of the orthogonal matrix Q form an orthonormal basis of Rn .
In particular, this means that the columns of Q are pairwise orthogonal.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 676

Compute the signature (n+ , n− , n0 ) of A, and compute an invertible matrix R with

pairwise orthogonal columns such that

RT AR = D 1, . . . , 1, −1, . . . , −1, 0, . . . , 0 .


| {z } | {z } | {z }
n+ n− n0

Solution. We proceed as in the proof of Proposition 9.4.1. First, we orthogonally

diagonalize the matrix A, as follows. We compute and factor the characteristic
polynomial of A:

pA (λ) = det(λI4 − A) = λ(λ − 2)2 (λ + 4).

So, the spectrum of A is {2, 2, −4, 0}. In other words, the eigenvalues of A are λ1 = 2
(with algebraic multiplicity 2), λ2 = −4 (with algebraic multiplicity 1), and λ3 = 0
(with algebraic multiplicity 1). Clearly, the signature of A is (2, 1, 1). It remains to
compute the matrix R.
Remark: It is important to list any positive eignvalues first, any negative eigenvalues
second, and any zero eigenvalues last. This is to ensure that on the main diagonal of
the diagonal matrix RT AR, any 1’s appear first, followed by any −1’s, followed by
any 0’s, as specified by the requirements of the example.
We now compute an orthonormal basis of each of the three eigenspaces. We
process our eigenvalues separately (first positive, then negative, then zero).
For the eigenvalue λ1 = 2, we compute:
 
1 −1 0 0
 0 0 1 1 
RREF(λ1 I4 − A) = RREF(2I4 − A) = 
 0
.
0 0 0 
0 0 0 0
 T  T
So, B1 := 1 1 0 0 , 0 0 −1 1 is a basis of Eλ1 (A). After perform-
ing the Gram-Schmidt orthogonalization process on B1 ,12 we obtain an orthonormal
 √ √ T  √ √ T
basis C1 := 1/ 2 1/ 2 0 0 , 0 0 −1/ 2 1/ 2 of Eλ1 (A).
For the eigenvalue λ2 = −4, we compute:
 
1 0 0 1
 0 1 0 −1 
RREF(λ2 I4 − A) = RREF(−4I4 − A) =  .
 0 0 1 −1 
0 0 0 0
12
In this particular case, the basis B1 is already orthogonal, and so we just need to normalize the
two vectors in it. However, this is an accident: in the general case, we may need to perform the full
Gram-Schmidt orthogonalization process.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 677

 T
So, B2 := −1 1 1 1 is a basis of Eλ2 (A). After performing the Gram-
Schmidt orthogonalization process on B2 ,13 we obtain an orthonormal basis C2 :=
 T
−1/2 1/2 1/2 1/2 of Eλ2 (A).
For the eigenvalue λ3 = 0, we compute:
 
1 0 0 −1
 0 1 0 1 
RREF(λ3 I4 − A) = RREF(−A) =  .
 0 0 1 −1 
0 0 0 0
 T
So, B3 := 1 −1 1 1 is a basis of Eλ3 (A). After performing the Gram-
Schmidt orthogonalization process on B3 ,14 we obtain an orthonormal basis C3 :=
 T
1/2 −1/2 1/2 1/2 of Eλ3 (A).
We now set D := D(λ1 , λ2 , λ2 , λ3 ) = D(2, 2, −4, 0) and
 √
−1/2

1/√2 0 1/2
 1/ 2
√0 1/2 −1/2 
Q :=  .
 0 −1/√2 1/2 1/2 
0 1/ 2 1/2 1/2

Next, we set L := D( √12 , √12 , 12 , 1),15 and we set

 
1/2 0 −1/4 1/2
(∗)  1/2 0 1/4 −1/2 
R := QL =  ,
 0 −1/2 1/4 1/2 
0 1/2 1/4 1/2

where (*) follows from Proposition 8.5.1(b) (or simply via routine matrix multiplica-
tion). Then R is an invertible matrix that has pairwise orthogonal columns, and we
have that
RT AR = D(1, 1, −1, 0),
as in the proof of Proposition 9.4.1.

13
Since B2 contains only one vector, all we actually need to do is to normalize the one vector
in B2 .
14
Since B3 contains only one vector, we just need to normalize the one vector in B3 .
15
Here, we started with D = D(2, 2, −4, 0), and we formed the matrix L by setting

L := D( √1 , √1 , √ 1 , 1) = D( √12 , √1 , 1 , 1),
2 2
|2| |2| |−4|

as in the proof of Proposition 9.4.1.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 678

Theorem 9.4.3. Two symmetric matrices in Rn×n are congruent if and only if they
have the same signature.
Proof. Fix symmetric matrices B, C ∈ Rn×n , and suppose first that B and C both
have the same signature, say (n+ , n− , n0 ). Proposition 9.4.1 then guarantees B and
C are both congruent to the diagonal matrix


D := D 1, . . . , 1, −1, . . . , −1, 0, . . . , 0 .
| {z } | {z } | {z }
n+ n− n0

By Proposition 9.2.6, matrix congruence is an equivalence relation on Rn×n ; so, since

B and C are congruent to the same matrix D, they are also congruent to each other.
Suppose, conversely, that B and C are congruent. Let (p, q, n − p − q) be the
signature of B, and let (s, t, n − s − t) be the signature of C; we must show that
(p, q, n−p−q) = (s, t, n−s−t). Clearly, it suffices to show that p = s and p+q = s+t.
First, by Proposition 9.4.1, B is congruent to the matrix


DB := D 1, . . . , 1, −1, . . . , −1, 0, . . . , 0 ,
| {z } | {z } | {z }
p q n−p−q

and C is congruent to the matrix

DC := D 1, . . . , 1, −1, . . . , −1, 0, . . . , 0 .
| {z } | {z } | {z }
s t n−s−t

Proposition 9.2.6 then guarantees that DB and DC are congruent to each other. By
definition, this means that there exists an invertible matrix P ∈ Rn×n such that
DC = P T DB P ; we will use this to prove that p + q = r + s. On the other hand, by
Theorem 9.4.1,16 there exist bases B = {b1 , . . . , bn } and C = {c1 , . . . , cn } of Rn , as
well as a quadratic form qe on Rn , such that DB is the matrix of qe with respect to B,
and DC is the matrix of qe with respect to C; we will use this to prove that p = s.
We first show that p + q = s + t. Clearly, rank(DB ) = p + q and rank(DC ) = s + t,
and so it is enough to show that rank(DB ) = rank(DC ). Since the matrix P is
invertible, the Invertible Matrix Theorem (see subsection 8.2.6) guarantees that P T
is also inverible. But then
(∗)
rank(DC ) = rank(P T DB P ) = rank(DB ),

where (*) follows from Proposition 3.3.16 (since P T and P are both invertible).
It remains to show that p = s. Suppose otherwise. By symmetry, we may
assume that p > s. Now consider the subspaces UB := Span(b1 , . . . , bp ) and
UC := Span(cs+1 , . . . , cn ) of Rn . Then by Theorem 3.2.23, we have that
dim(UB ) + dim(UC ) = dim(UB + UC ) + dim(UB ∩ UC ).
But note that
16
We are using the “(a) =⇒ (c)” implication from Theorem 9.4.1

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 679

ˆ dim(UB ) + dim(UC ) = p + (n − s) = n + (p − s) > n,

ˆ dim(UB + UC ) ≤ dim(Rn ) = n.
So, dim(UB ∩ UC ) > 0, and it follows that UB ∩ UC contains some non-zero vector u.
   T    T
Set u B = x1 . . . xn and u C = y1 . . . yn . Then at least one
of x1 , . . . , xp is non-zero, xp+1 = · · · = xn = 0, and y1 = · · · = ys = 0. We now have
that
T  (∗)
ˆ qe(u) = u B DB u B = x21 + · · · + x2p > 0,
 

T (∗)
ˆ qe(u) = 2 2
  
u C
DC u C
= −ys+1 − · · · − ys+t ≤ 0,
where for both instances of (*), we used the formula from Proposition 9.1.1(a). We
have now derived a contradiction, and it follows that p = s. This completes the
argument.

9.4.2 Sylvester’s law of inertia

Suppose that F is a field and that D = D(a1 , . . . , an ) is a diagonal matrix in Fn×n .
T
in Fn , we have that

Then for all vectors x = x1 . . . xn
xT Dx = a1 x21 + · · · + an x2n ,
as can be seen via routine computation, or by applying Proposition 9.1.1(a). This is
a particularly nice formula, and for this reason, if q is a quadratic form over a field
F, it is helpful to have a basis B with respect to which the matrix of q is diagonal.
Sylvester’s law of inertia (below) states that when V = Rn , such a basis B always
exists.
As we shall see, Sylvester’s law of inertia is essentially a “translation” of Proposi-
tion 9.4.1 and Theorem 9.4.3 into the language of quadratic forms. Before formally
stating and proving the law, we need a couple of definitions. The signature of a
quadratic form q on Rn is defined to be the signature of the matrix of q with respect
to any basis B of Rn . By Theorems 9.3.4 and 9.4.3, this is well defined. Indeed, by
Theorem 9.3.4,17 matrices of q with respect to all possible bases of Rn are congruent
to each other, and so by Theorem 9.4.3, they all have the same signature. A polar
basis of Rn associated with the quadratic form q is any orthogonal basis B of Rn
such that the matrix of q with respect to B is a diagonal matrix with only 1’s, −1’s,
and 0’s on the main diagonal.
Sylvester’s law of inertia. Let q be a quadratic form on Rn , and let (n+ , n− , n0 )
be the signature of q. Then Rn has a polar basis B associated with q. Moreover, for
any basis C of Rn such that the matrix C of q with respect to C is diagonal, with only
1’s, −1’s, and 0’s on the main diagonal, the following holds: the number of 1’s, −1’s,
and 0’s on the main diagonal of C is n+ , n− , and n0 , respectively.
17
Here, we are using the “(c) =⇒ (a)” implication from Theorem 9.3.4.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 680

Remark: The basis C from the second sentence of Sylvester’s law of inertia is not
assumed to be polar, i.e. it is possible that it is not orthogonal.

Proof. Let A be the matrix of the quadratic form q with respect to the standard
basis En of Rn ; then the signature of A is (n+ , n− , n0 ).
We first prove the existence of the polar basis B. Set


D := D 1, . . . , 1, −1, . . . , −1, 0, . . . , 0 .
| {z } | {z } | {z }
n+ n− n0

By Proposition 9.4.1, there exists an invertible matrix R ∈ Rn×n with pairwise

orthogonal columns such that D = RT AR. Since R is invertible, the Invertible
Matrix Theorem (see subsection 8.2.6) guarantees that its columns form a basis B
of Rn ; since the columns of R are pairwise orthogonal, the basis B is orthogonal.
Moreover,
 byTheorem 4.5.1 (or alternatively, by Lemma 4.5.11), we have that
R = E IdV B , so that
n

 T  
D = En
IdV B
A En
IdV B
.

But now Theorem 9.3.3 guarantees that D is the matrix of q with respect to B. We
have already seen that the basis B is orthogonal, and we deduce that B is a polar
basis of Rn associated with q.
Now, fix any basis C of Rn such that the matrix of q with respect to C is a diagonal
matrix C with only 1’s, −1’s, and 0’s on the main diagonal. By Theorem 9.3.4,
matrices A and C are congruent, and so by Theorem 9.4.3, they have the same
signature, which is (n+ , n− , n0 ). Since the matrix C is diagonal, we know its entries
on the main diagonal form its spectrum;18 so, the number of 1’s, −1’s, and 0’s on
the main diagonal of C is n+ , n− , and n0 , respectively.

Example 9.4.4. Consider the quadratic form q on R4 given by the formula

q(x) = 4x1 x2 + 2x1 x3 + 2x1 x4 − 2x2 x3 − 2x2 x4 − 4x3 x4

T
in R4 . Compute the signature (n+ , n− , n0 )

for all vectors x = x1 x2 x3 x4
of q, a polar basis B of R4 associated with q, and the matrix D of q with respect to B.

Solution. First of all, the (symmetric) matrix of q with respect to the standard basis
E4 of R4 is the matrix
 
0 2 1 1
 2 0 −1 −1 
A :=   1 −1
.
0 −2 
1 −1 −2 0
18
This follows from Proposition 8.2.7.

Irena Penev
Linear Algebra 1 & 2
Chapter 9. Bilinear and quadratic forms 681

This is precisely the matrix from Example 9.4.2. In the solution to that example, we
saw that the signature of A is (2, 1, 1), and so by definition, the signature of q is also
(2, 1, 1). Moreover, in the solution to Example 9.4.2, we saw that matrix
 
1/2 0 −1/4 1/2
 1/2 0 1/4 −1/2 
R :=  
 0 −1/2 1/4 1/2 
0 1/2 1/4 1/2

is invertible with pairwise orthogonal columns, and that RT AR = D(1, 1, −1, 0).
The columns of the matrix R form a polar basis
       
1/2 0 −1/4 1/2
n  1/2   0   1/4
    −1/2  o
B :=  0  ,  −1/2  ,  1/4
   ,
  1/2 


0 1/2 1/4 1/2

of R4 associated with the quadratic form q. The matrix of q with respect to the
polar basis B is the matrix D := RT AR = D(1, 1, −1, 0).

9.4.3 Quadratic forms on R2

For quadratic forms on R2 , there exist only six possible signatures (n+ , n− , n0 ),
namely, the following:

ˆ (2, 0, 0); ˆ (0, 2, 0);

ˆ (1, 0, 1); ˆ (0, 1, 1);

ˆ (1, 1, 0); ˆ (0, 0, 2).

Thus, the graph of any quadratic form q on R2 has the same general shape as one
of the six graphs shown below (the one that has the same signature as q).19 The
actual graph of the quadratic form q would be obtained by starting with one of
the six graphs below (the one that has the same signature as q), and then possibly
stretching or shrinking the graph along the x1 - and x2 -axes (the coordinate axes of
the domain), and then possibly rotating it about the vertical axis x3 ; this to account
for the fact that a polar basis B of R2 associated with q is not necessarily equal to
the standard basis E2 = {e1 , e2 }, but the vectors of B are indeed orthogonal to each
other (by the definition of a polar basis).
19
My thanks to Milan Hladı́k, who generated these six graphs and shared them with me.

Irena Penev
Linear Algebra 1 & 2
 Poznamenejme, že báze, vůči níž matice kvadratické formy je diagonální, se nazývá polární báze. Tedy
báze z věty 12.13 je příkladem polární báze, ale typicky existují i další. Dá se také ukázat, že polární báze
existuje nejen pro reálné prostory, ale i pro prostory nad libovolným tělesem charakteristiky různé od 2.
Geometrický význam Sylvestrova zákona setrvačnosti spočívá v tom najít vhodný souřadný systém (tj.
bázi), ve kterém má kvadratická forma jednoduchý diagonální tvar. Algebraický pohled na věc je ten, že
danou symetrickou matici A transformujeme na diagonální tvar pomocí úprav S TAS, kde S je regulární.
Chapter 9. Bilinear and quadratic forms 682
Příklad 12.14 (Kvadratické formy v R2 ). Podle Sylvestrova zákona mají kvadratické formy v R2 v pod-
statě jeden z následujících tvarů v souřadném systému vhodné báze.

50 0

40 −10

30 −20

20 −30

10 −40

0 −50
5 5
5 5
0 0
0 0

−5 −5 −5 −5

x21 + x22 −x21 − x22

25 0

20 −5

15 −10

10 −15

5 −20

0 −25
5 5
5 5
0 0
0 0

−5 −5 −5 −5

x21 −x21

30 1

20
0.5
10

0 0

−10
−0.5
−20

−30 −1
5 5
5 5
0 0
0 0

−5 −5 −5 −5

x21 − x22 0

Remark: Suppose that q is a quadratic form on R2 , and that B = {b1 , b2 } is a

polar basis of R2 associated with q. The vectors in the polar basis B = {b1 , b2 } may
possibly have a different orientation from those of the standard basis E2 = {e1 , e2 }
(see the picture below, on the left). However, this change of orientation does not
affect the graph of q, essentially due to the squaring. Note also that if {b1 , b2 } is
a polar basis, then so is {−b1 , b2 }, and one of those two polar bases will have the
same orientation as the standard basis E2 = {e1 , e2 }.
x2 x2

e2 b2 e2 b2

b1
e1 x1 e1 x1
−b1

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 683

Chapter 10

Matrix definiteness

Notation: Throughout this chapter, we will be working over the field R of real
numbers. So, in our numerical examples, we will not emphasize the fact that
the matrix in question has real entries (this will simply be assumed throughout).
Moreover, throughout this chapter, · denotes the standard scalar product in Rn , and
|| · || denotes the norm in Rn induced by the standard scalar product (for the relevant
definitions, see subsections 6.1.1 and 6.2.1).

10.1 Matrix definiteness: definition and basic properties

A symmetric matrix A ∈ Rn×n is said to be

ˆ positive definite if xT Ax > 0 for all x ∈ Rn \ {0};

ˆ positive semi-definite if xT Ax ≥ 0 for all x ∈ Rn ;

ˆ negative definite if xT Ax < 0 for all x ∈ Rn \ {0};

ˆ negative semi-definite if xT Ax ≤ 0 for all x ∈ Rn ;

ˆ indefinite if it is neither positive semi-definite nor negative semi-definite.

Remark: Obviously, any positive definite matrix is positive semi-definite, and any
negative definite matrix if negative semi-definite.
Remark: We note that the definitions above would also make sense without the
requirement that A be symmetric. However, for any matrix A ∈ Rn×n , the matrix
1 T 1 n
2 (A + A ) is symmetric, and for all vectors x ∈ R , we have that

1 1
xT + AT ) x T + 12 (xT AT x)


2 (A = 2 (x Ax)

1

T
Indeed, 1
2
(A + AT ) = 12 (AT + A) = 21 (A + AT ), and so 1
2
(A + AT ) is symmetric.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 684

(∗) 1
= T
2 (x Ax) + 12 (xT AT x)T

1
= T
2 (x Ax) + 12 (xT Ax)

= xT Ax,

where (*) follows from the fact that xT Ax is a 1 × 1 matrix, and is consequently
symmetric. So, instead of considering an arbitrary square matrix A, we can consider
the symmetric matrix 12 (A + AT ) instead. This is important because some tests of
definiteness only work if we assume that the matrix in question is symmetric.
Remark: Matrix definiteness plays an important role in optimization, though these
lecture notes do not cover this. Another reason for caring about positive definite
matrices in particular is that it turns out that a bilinear form on a non-trivial,
finite-dimensional real vector space is a scalar product if and only if the matrix of
this bilinear form with respect to some basis of the vector space in question is a
positive definite matrix (see section 10.4). In fact, it can be shown that one such
matrix is positive definite if and only if all such matrices are positive definite (for a
precise statement, see Theorem 10.4.1).

Proposition 10.1.1. For every symmetric matrix A ∈ Rn×n , both the following
hold:

(a) A is positive definite if and only if −A is negative definite;

(b) A is positive semi-definite if and only if −A is negative semi-definite.

Proof. Fix a symmetric matrix A ∈ Rn×n . For (a), we have the following sequence
of equivalent statements:

A is positive definite ⇐⇒ xT Ax > 0 ∀x ∈ Rn \ {0}

⇐⇒ −xT Ax < 0 ∀x ∈ Rn \ {0}

⇐⇒ xT (−A)x < 0 ∀x ∈ Rn \ {0}

⇐⇒ −A is negative definite.

This proves (a). The proof of (b) is very similar.

Remark: In view of Proposition 10.1.1, results for positive (semi-)definite matrices

can easily be translated into corresponding results for negative (semi-)definite matri-
ces. So, it makes sense to focus on positive (semi-)definite matrices. In these lecture
notes (and in particular, in sections 10.2, 10.3, and 10.4), we will mostly focus on

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 685

positive definite matrices, which are somewhat easier to deal with than the more
general positive semi-definite ones.
By Corollary 8.7.4, any symmetric matrix in Rn×n has n real eigenvalues (when
algebraic multiplicities are taken into account). The following theorem specifies the
relationship between the signature (defined in subsection 9.4.1) and the definiteness
of a symmetric matrix with real entries.

Theorem 10.1.2. Let A ∈ Rn×n be a symmetric matrix, and let (n+ , n− , n0 ) be the
signature of A. Then all the following hold:

(a) A is positive definite if and only if n+ = n (i.e. all eigenvalues of A are positive);

(b) A is positive semi-definite if and only if n+ + n0 = n (i.e. all eigenvalues of A

are non-negative);

(c) A is negative definite if and only if n− = n (i.e. all eigenvalues of A are negative);

(d) A is negative semi-definite if and only if n− + n0 = n (i.e. all eigenvalues of A

are non-positive);

(e) A is indefinite if and only if n+ and n− are both non-zero (i.e. A has at least
one positive and at least one negative eigenvalue).

Proof. Obviously, (b) and (d) together imply (e).2 So, we just need to prove (a)-(d).
Here, we prove (a). The proofs of (b)-(d) are similar.
Suppose first that A is positive definite. Fix an eigenvalue λ of A, and let x
be an associated eigenvector of A; after possibly normalizing the eigenvector x (i.e.
x
replacing x by ||x|| ), we may assume that ||x|| = 1. Then

(∗) (∗∗) (∗∗∗)

0 < xT Ax = xT (λx) = λ(xT x) = λ(x · x) = λ||x||2 = λ,

where (*) follows from the fact that A is positive definite and x ̸= 0, (**) follows
from the fact that x is an eigenvector of A associated with the eigenvalue λ, and
(***) follows from the fact that ||x|| = 1. We have now shown that λ > 0; since λ
was an arbitrarily chosen eigenvalue of A, we deduce that all eigenvalues of A are
positive, i.e. n+ = n.
2
Indeed, by definition, n+ , n− , n0 ≥ 0 and n+ + n− + n0 = n. Moreover, by definition, A is
indefinite if and only if it is neither positive semi-definite nor negative semi-definite. So, assuming (b)
and (d), we have the following sequence of equivalent statements:
(∗)
A is indefinite ⇐⇒ n+ + n0 < n and n− + n0 < n

⇐⇒ n+ , n− > 0,

where (*) follows from (b) and (d).

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 686

Suppose conversely that n+ = n, i.e. that all eigenvalues of A are positive. Since
A is symmetric, the spectral theorem for symmetric matrices (see subsection 8.7.3)
guarantees that Rn has an orthonormal eigenbasis B = {x1 , . . . , xn } associated
with A. For each i ∈ {1, . . . , n}, let λi be the eigenvalue of A associated with the
eigenvector xi . Set λ0 := min{λ1 , . . . , λn }; since all eigenvalues of A are positive, we
see that λ0 > 0. Now, fix any x ∈ Rn \ {0}. Since B is a basis of Rn , we know that
there exist scalars α1 , . . . , αn ∈ R such that x = α1 x1 + · · · + αn xn . Since x ̸= 0, at
least one of α1 , . . . , αn is non-zero. We now compute:
n
T n
xT Ax
P P

= αi xi A α j xj
i=1 j=1

n P
n
αi αj xTi Axj
P
=
i=1 j=1

n P
n because each xj is an
αi αj xTi (λj xj )
P
= eigenvector of A associated
i=1 j=1 with the eigenvalue λj

n P
n
λj αi αj (xTi xj )
P
=
i=1 j=1

n P
P n
= λj αi αj (xi · xj )
i=1 j=1

n because x1 , . . . , xn are
λi αi2 (xi · xi )
P
= pairwise orthogonal (by
i=1 the orthonormality of B)

n
λi αi2 ||xi ||2
P
=
i=1

n because x1 , . . . , xn are
λi αi2
P
= unit vectors (by the
i=1 orthonormality of B)

n because λ0 = min{λ1 , . . . , λn }
λ0 αi2
P
≥
i=1 and α12 , . . . , αn2 ≥ 0

because λ0 > 0 and at least

> 0
one of α1 , . . . , αn is non-zero.

Thus, A is positive definite. This proves (a).

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 687

Corollary 10.1.3. Let A ∈ Rn×n be a symmetric matrix.

(a) If A is positive definite, then det(A) and trace(A) are both positive.

(b) If A is positive semi-definite, then det(A) and trace(A) are both non-negative.

Proof. Since A is symmetric, Corollary 8.7.4 guarantees that it has n real eigenvalues
(with algebraic multiplicities taken into account). So, let {λ1 , . . . , λn } be the spectrum
of A. By Theorem 8.2.10, we have that det(A) = λ1 . . . λn and trace(A) = λ1 +· · ·+λn .
By Theorem 10.1.2(a), all eigenvalues of a positive definite matrix are positive, and it
follows that (a) holds. Similarly, by Theorem 10.1.2(b), all eigenvalues of a positive
semi-definite matrix are non-negative, and it follows that (b) holds.

The main diagonal of a square matrix A ∈ Rn×n is positive (resp. non-negative,

negative, non-positive) if all entries on the main diagonal of A are positive (resp.
non-negative, negative, non-positive).

Proposition 10.1.4. The main diagonal of any positive definite (resp. positive
semi-definite, negative definite, negative semi-definite) matrix is positive (resp. non-
negative, negative, non-positive).

Proof. Fix a matrix A = ai,j n×n in Rn×n . Then for all indices i ∈ {1, . . . , n}, we
 

have that eTi Aei = ai,i .3 The result now follows from the appropriate definitions.4

Proposition 10.1.5. Let A, B ∈ Rn×n and α ∈ R. Then all the following hold:

(a) if A and B are both positive definite (resp. positive semi-definite, negative definite,
negative semi-definite), then A+B is positive definite (resp. positive semi-definite,
negative definite, negative semi-definite);

(b) if A is positive definite (resp. positive semi-definite, negative definite, negative

semi-definite) and α > 0, then αA is positive definite (resp. positive semi-definite,
negative definite, negative semi-definite);

(c) if A is positive definite (resp. positive semi-definite, negative definite, negative

semi-definite) and α < 0, then αA is negative definite (resp. negative semi-
definite, positive definite, positive semi-definite);

(d) if A ∈ Rn×n is positive definite (respectively: negative definite), then A is

invertible and its inverse A−1 is positive definite (respectively: negative definite).
3
This can be seen directly, but it also follows from Proposition 9.1.1(b).
4
Let us explain this in a bit more detail. Suppose that A is positive definite. Then for each
i ∈ {1, . . . , n}, we have that ai,i = eTi Aei > 0, i.e. the main diagonal of A is positive. Similar remarks
apply for the cases of positive semi-definiteness, negative definiteness, and negative semi-definiteness.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 688

Proof. The proof of (a), (b), and (c) is left as a straightforward exercise. Let us
prove (d). We prove the result for positive definite matrices; the proof for negative
definite matrices is similar, or alternatively, it can be reduced to the case of positive
definiteness via Proposition 10.1.1(a).
Assume that A is positive definite. Then by Theorem 10.1.2(a), all eigenvalues of
A are positive, and in particular, 0 is not an eigenvalue of A. So, by the Invertible
Matrix Theorem (see subsection 8.2.6), A is invertible. We must show that A−1 is
positive definite.
First, we show that A−1 is symmetric. By Proposition 1.11.8(c), AT is invertible
and satisfies (AT )−1 = (A−1 )T . But AT = A (because A is positive definite and
therefore symmetric), and it follows that A−1 = (A−1 )T . So, A−1 is symmetric.
In view of Theorem 10.1.2(a), it now suffices to show that all eigenvalues of A−1
are positive. So, let λ ∈ R be an eigenvalue of A−1 , and let v ∈ Rn \ {0} be an
associated eigenvector. Since A−1 is invertible, the Invertible Matrix Theorem once
again guarantees that λ ̸= 0, and in particular, λ1 is defined. We now compute:

1 (∗) 1 −1 1 −1 1
Av = λ A(λv) = λ A(A v) = λ (AA
| {z })v = λ v,
=In

where (*) follows from the fact that v is an eigenvector of A−1 associated with the
eigenvalue λ. But now v is an eigenvector of A associated with the eigenvalue λ1 ,
and it follows that λ1 > 0 (because all eigenvalues of A are positive). So, λ > 0.

10.2 Methods of testing for positive definiteness

In this section, we present three tests of positive definiteness:
ˆ the recursive test of positive definiteness (see Theorem 10.2.3 in subsec-
tion 10.2.1);
ˆ the Gaussian elimination test of positive definiteness (see Theorem 10.2.6 in
subsection 10.2.2);
ˆ Sylvester’s criterion of positive definiteness (see Theorem 10.2.9 in subsec-
tion 10.2.3).
Of these three tests, the first is arguably the least convenient for computing (at least
if we are computing by hand), but it is important because we will rely on it to prove
the correctness of the other two tests.

10.2.1 The recursive test of positive definiteness

In what follows, for a matrix A ∈ Rn×n (n ≥ 2) and indices i, j ∈ {1, . . . , n}, we will
denote by Ai,j the submatrix of A obtained by deleting the i-th row and j-th column
of A (as in section 7.6).

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 689

ai,j n×n (n ≥ 2) be a symmetric matrix in Rn×n ,


Proposition 10.2.1. Let A =
 T
assume that a1,1 ̸ 0, and set a := a2,1 . . . an,1
= , so that

a1,1 aT
 
A = .
a A1,1

Let Ae be the matrix obtained from A by (sequentially or simultaneously) performing

the following elementary row operations on A:
a2,1
ˆ R2 → R2 − a1,1 R1 ;
a3,1
ˆ R3 → R3 − a1,1 R1 ;

..
.
an,1
ˆ Rn → Rn − a1,1 R1 .

Then " #
a1,1 aT
A
e = 1 .
0 A1,1 − a1,1 aaT

Proof. Set
rT2
 

A1,1 =  ...  ,
 

rTn
so that the row vectors rT2 , . . . , rTn (in R1×(n−1) ) are the rows of A1,1 (from top to
bottom).5
Since none of the elementary row
 operations modified the first row of A, we see
e is a1,1 aT . On the other hand, for each i ∈ {2, . . . , n}, the

that the first row of A
h i
i-th row of Ae is 0 rTi − ai,1 aT , whereas the (i − 1)-th row of the (n − 1) × (n − 1)
a1,1
1 ai,1 T
matrix A1,1 − a1,1 aaT is T
ri − a1,1 a . The result is now immediate.

Proposition 10.2.2. Let α ∈ R, a ∈ Rn , and A ∈ Rn×n . If A is symmetric, then

A − αaaT is also symmetric.

Proof. Assume that A is symmetric. Then

T (∗)
A − αaaT = AT − α(aT )T aT = A − αaaT ,

where in (*), we used the fact that A is symmetric and so AT = A. This proves that
A − αaaT is indeed symmetric.
5
T
So, r2 , . . . , rn ∈ Rn−1 , and A1,1 =

r2 ... rn .

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 690

Theorem 10.2.3 (The recursive test of positive definiteness). Let n be a positive

α aT
 
integer, and let A = (with α ∈ R, a ∈ Rn , and A′ ∈ Rn×n ) be a symmetric
a A′
matrix in R(n+1)×(n+1) . Then A is positive-definite if and only if α > 0 and A′ − α1 aaT
is positive definite.

Remark: If α ̸= 0, then Proposition 10.2.2 guarantees that the matrix A′ − α1 aaT

is symmetric, and Proposition 10.2.1 guarantees that

aT
 
α
0 A′ − α1 aaT

is the matrix obtained from A by (sequentially or simultaneously) performing the

elementary row operations of the form “Ri → Ri + βi R1 ” (for i ∈ {2, . . . , n}), with
the βi ’s chosen so that, with the exception of the 1, 1-th entry, the leftmost column
becomes zero.

Proof. Suppose first that A is positive definite. By Proposition 10.1.4, we have

that α > 0, and in particular, the matrix A′ − α1 aaT is defined (i.e. we are not
dividing by zero). We must show that this matrix is positive definite. First of all, by
Proposition 10.2.2, the matrix A′ − α1 aaT is symmetric. Now, fix any x ∈ Rn \ {0}.
Then
xT (A′ − α1 aaT )x = xT A′ x − α1 (xT aaT x)

α aT − α1 aT x
  
(∗)
− α1 aT x xT
 
=
a A′ x
| {z }
:=y
(∗∗)
= yT Ay > 0

where (**) follows from the fact that A is positive definite and y ̸= 0 (because
x ̸= 0), and (*) follows from the following computation:
 1 T 
 α aT −aT x + aT x
  
 1 T −αa x  1 T
− α a x xT T

= − a x x
a A′ x α − α1 aaT x + A′ x
 
0
− α1 aT x xT
 
=
A′ x − α1 aaT x

= xT A′ x − α1 xT aaT x.

This proves that A′ − α1 aaT is indeed positive definite.

Suppose conversely that α > 0 and A′ − α1 aaT is positive definite. We must show

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 691

that A is positive definite. By hypothesis, A is symmetric. Now, fix any x ∈ Rn+1 ;

we must
 show
 that xT Ax ≥ 0, and that equality holds if and only if x = 0. Set
x0
x= , where x0 ∈ R and z ∈ Rn . We now compute:
z

α aT
  
x0
xT Ax zT
 
= x0
a A′ z

= αx20 + x0 aT z + x0 zT a + zT A′ z

(∗)
= αx20 + 2x0 aT z + zT A′ z

= zT (A′ − α1 aaT )z + α1 zT aaT z + 2x0 aT z + αx20

√
= zT (A′ − α1 aaT )z + ( √1α aT z)2 + 2x0 aT z + ( αx0 )2

 √ 2 (∗∗)
= zT (A′ − α1 aaT )z + √1 aT z
α
+ αx0 ≥ 0,

where in (*), we used the fact that x0 zT a is a 1 × 1 (and consequently symmetric)

matrix, and so x0 zT a = (x0 zT a)T = x0 aT z; and where for the inequality (**), we
used the fact that zT (A′ − α1 aaT )z ≥ 0, since A′ − α1 aaT is positive definite. It
remains to show that the inequality (**) is an equality if and only if x = 0. If x = 0,
then x0 = 0 and z = 0, and it is obvious that the inequality (**) is an equality.
Suppose now that the inequality (**) is an equality. Then zT (A′ − α1 aaT )z = 0
√
and √1α aT z + αx0 = 0. The former implies that z = 0 (since A′ − α1 aaT is
√
positive definite). But now since √1α aT z + αx0 = 0, we deduce that x0 = 0. So,
 
x0
x= = 0. This proves that A is positive definite.
z

Remark: Theorem 10.2.3 allows us to reduce the problem of determining whether a

given symmetric matrix is positive definite to the problem of determining whether a
smaller symmetric matrix is positive definite (this is why the test is called “recursive”).
We keep computing suitable smaller matrices until we either obtain a non-positive
number in the upper left corner of our matrix (in this case, the input matrix is not
positive definite), or obtain a 1 × 1 matrix whose sole entry is positive (in this case,
the input matrix is positive definite). Obviously, a matrix in R1×1 is positive definite
if and only if its unique entry is positive.6

6
Indeed, suppose we are given a matrix A = a in R1×1 . Then for all x = x in R1 \ {0},
   

we have that xT Ax = ax2 , which has the same sign as a.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 692

Example 10.2.4. Using Theorem 10.2.3, determine whether the matrix

 
4 −2 4
A :=  −2 10 1 
4 1 6
is positive definite.
Solution. The matrix A is symmetric, and so Theorem 10.2.3 applies. As we shall
see, we will need to apply Theorem  10.2.3 twice.
α1 aT1
    
−2 ′ 10 1
First, set α1 := 4, a1 := , and A1 := , so that A = .
4 1 6 a1 A′1
We have that α1 > 0, and so by Theorem 10.2.3, A is positive definite if and only if
A1 := A′1 − α11 a1 aT1 is positive definite. We compute:
     
′ 1 T 10 1 1 −2   9 3
A1 = A1 − α1 a1 a1 = −4 −2 4 = .
1 6 4 3 2

α2 aT2
 
Next, set α2 := 9, a2 := 3 , and A′2 := 2 , so that A1 =
   
. We
a2 A′2
have that α2 > 0, and so by Theorem 10.2.3, A1 is positive definite if and only if
A2 := A′2 − α12 a2 aT2 is positive definite. We compute

A2 = A′2 − α12 a2 aT2 =

  1     
2 −9 3 3 = 1 .

Since the only entry of A2 is positive, we see that A2 is positive definite. So, A is
positive definite.

Example 10.2.5. Using Theorem 10.2.3, determine whether the matrix

 
2 −2 2 0
 −2 3 0 1 
A :=   2

0 6 0 
0 1 0 2
is positive definite.
Solution. The matrix A is symmetric, and so Theorem 10.2.3 applies. First, set
   
−2 3 0 1
α1 = 2, a1 =  2  , A′1 =  0 6 0  ,
0 1 0 2

α1 aT1
 
so that A = . We have that α1 > 0, and so by Theorem 10.2.3, A is
a1 A′1
positive definite if and only if A1 := A′1 − α11 a1 aT1 is positive definite. We compute:

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 693

   
3 0 1 −2 
A′1 − 1 T  0 6 0  − 1  2  −2 2 0 T

A1 = α1 a1 a1 = 2
1 0 2 0
 
1 2 1
=  2 4 0 .
1 0 2

Next, set
   
2 4 0
α2 := 1, a2 := , A′2 = ,
1 0 2

α2 aT2
 
so that A1 = . We have that α2 > 0, and so by Theorem 10.2.3, A1 is
a2 A′2
positive definite if and only if A2 := A′2 − α12 a2 aT2 is positive definite. We compute:
     
′ 1 T 4 0 1 2   0 −2
A2 = A2 − α2 a 2 a 2 = −1 2 1 = .
0 2 1 −2 1
The matrix A2 has a zero on the main diagonal, and so by Proposition 10.1.4, A2 is
not positive definite. So, A is not positive definite.

10.2.2 The Gaussian elimination test of positive definiteness

Theorem 10.2.6 (The Gaussian elimination test of positive definiteness). Let
A ∈ Rn×n be a symmetric matrix. Then the following algorithm correctly determines
whether A is positive definite.
ˆ Step 0: Set A1 := A, and go to Step 1.

ˆ For j ∈ {1, . . . , n}, and assuming the matrix Aj has already been generated, we
proceed as follows.
Step j:
– If the main diagonal of Aj is not positive,7 then the algorithm returns
the answer that A is not positive definite and terminates.
– If the main diagonal of Aj is positive and j = n, then the algorithm
returns the answer that A is positive definite and terminates.
– If the main diagonal of Aj is positive and j ≤ n − 1, then for each index
i ∈ {j + 1, . . . , n}, we add a suitable scalar multiple of the j-th row of Aj
to the i-th row of Aj so that the i, j-th entry of the matrix becomes zero;8
we call the resulting matrix Aj+1 , and we go to Step j + 1.
7
This means that the main diagonal of Aj has at least one non-positive entry.
8
The point is to turn all the entries of the j-th column below the main diagonal into zeros.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 694

Remark: The algorithm performs a modified version of the “forward” part of the
row reduction algorithm. It only performs elementary row operations of the form
“Ri → Ri + αRj ,” where i > j (i.e. row i is below row j), and where α is chosen so
that the i, j-th entry of the matrix becomes zero; moreover, these operations (which
add scalar multiples of row j to the rows below it) are performed only in Step j.9
We keep modifying our matrix until we either obtain a zero or a negative number on
the main diagonal (in this case, our input matrix is not positive definite), or until we
transform our matrix into an upper triangular matrix with a positive main diagonal
(in this case, our input matrix is positive definite). To fully understand how this
algorithm works, the reader may wish to take a look at Examples 10.2.7 and 10.2.8
before reading the proof of the theorem.

Proof. We proceed by induction on n. 

For n = 1, we fix a matrix A = a1,1 in R1×1 (obviously, A is symmetric), and
we observe that our algorithm sets A1 := A, and then if a1,1 > 0, returns the answer
that A is positive definite, and if a1,1 ≤ 0, returns the answer that A is not positive
definite. Obviously, this is correct.
Now, fix an integer n ≥ 2, and assume inductively that the algorithm is correct
when applied
 to symmetric matrices in R(n−1)×(n−1) . Fix a symmetric matrix
A = ai,j n×n in R n×n . Set A1 := A, as per Step 0 of the algorithm. If the
main diagonal of A1 contains an entry that is zero or negative, then the algorithm
terminates after Step 1, having determined that A is not positive definite (this is
correct by Proposition 10.1.4).
From now on, we may assume that the main diagonal of A1 is positive, and in
particular, a1,1 > 0. In this case, Step 1 performs the following elementary row
operations on the matrix A1 = A:
a2,1
ˆ R2 → R2 − a1,1 R1 ;
a3,1
ˆ R3 → R3 − a1,1 R1 ;

..
.
an,1
ˆ Rn → Rn − a1,1 R1 .

This transforms entries 2, . . . , n − 1 of the first column into 0. The resulting matrix
is A2 . But note that our matrix A1 = A is of the form

a1,1 aT
 
A1 = A = ,
a A1,1
9
Essentially, we use the j, j-th entry of the matrix Aj to “clean up” the j-th column below the
main diagonal, i.e. to turn all entries of the j-th column below the main diagonal into zeros. Note
that at the start of Step j, the leftmost j − 1 many columns have already been processed, so that
they have all zeros below the main diagonal.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 695

 T
where a = a2,1 . . . an,1 , and A1,1 is the matrix obtained from A by deleting
the first row and first column. So, by Proposition 10.2.2, the matrix A2 that we
obtain after the Step 1 is precisely the matrix
" #
a1,1 aT
A2 = 1 .
0 A1,1 − a1,1 aaT

1
By Proposition 10.2.2, A1,1 − a1,1 aaT is symmetric, and the remainder of our algorithm
1
only manipulates the (n − 1) × (n − 1) submatrix A1,1 − a1,1 aaT of A2 (while leaving
the top row and leftmost column of A2 unchanged). Moreover, by Theorem 10.2.3,
1
A is positive definite if and only if A1,1 − a1,1 aaT is positive definite. The result now
readily follows from the induction hypothesis.

Example 10.2.7. Using Theorem 10.2.6, determine whether the matrix

 
4 −2 4
A :=  −2 10 1 
4 1 6

is positive definite.

Solution. The matrix A is symmetric, and so Theorem 10.2.6 applies. We perform

the modified version of the “forward” part of the row reduction algorithm described
in Theorem 10.2.6, as follows (the dotted lines isolate the submatrix in the lower
right corner that is still being processed):
 
4 −2 4
A =  −2 10 1  Step 0
4 1 6
| {z }
=:A1

R2 →R2 + 12 R1
 
4 −2 4
R3 →R3 −R1
∼  0 9 3  Step 1
0 3 2
| {z }
=:A2

 
R3 →R3 − 13 R2
4 −2 4
∼  0 9 3  Step 2.
0 0 1
| {z }
=:A3

We have now obtained an upper triangular matrix with a positive main diagonal.
So, by Theorem 10.2.6, A is positive definite. (This answer is returned by Step 3 of

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 696

the algorithm from Theorem 10.2.6, at which point the algorithm terminates.)

Remark: Normally, we do not actually number our steps, and we do not name the
matrices Ai ; here, we did it for the sake of extra clarity. The horizontal and vertical
dotted lines are also optional, but they are useful for visually keeping track of the
submatrix being processed, and so it is not a bad idea to include them.

Example 10.2.8. Using Theorem 10.2.6, determine whether the matrix

 
2 −2 2 0
 −2 3 0 1 
A :=   2

0 6 0 
0 1 0 2

is positive definite.

Solution. The matrix A is symmetric, and so Theorem 10.2.6 applies. We perform

the modified version of the “forward” part of the row reduction algorithm described
in Theorem 10.2.6, as follows:
 
2 −2 2 0
 −2 3 0 1 
A = 
 2

0 6 0 
0 1 0 2
 
2 −2 2 0
R2 →R2 +R1
R3 →R3 −R1  0 1 2 1 
∼ 
 0

2 4 0 
0 1 0 2
 
2 −2 2 0
R3 →R3 −2R2
R4 →R4 −R2  0 1 2 1 
∼  .
 0 0 0 −2 
0 0 −2 1

We have now obtained a zero on the main diagonal of our matrix, and so by
Theorem 10.2.6, the matrix A is not positive definite.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 697

10.2.3 Sylvester’s criterion of positive definiteness

Given any n × n matrix A, and any index k ∈ {1, . . . , n}, we let A(k) be the k × k
matrix in the upper left corner of A. For example, if
 
1 2 3
A =  4 5 6 ,
7 8 9

then we have that

 
  1 2 3
1 2
A(1) = A(2) = A(3)
 
1 , , =  4 5 6 .
4 5
7 8 9

Clearly, for any n × n matrix A, we have that A(n) = A.

Theorem 10.2.9 (Sylvester’s criterion of positive definiteness). For all symmetric

matrices A ∈ Rn×n , the following are equivalent:

(i) A is positive definite;

(ii) det A(1) , . . . , det A(n) > 0.

Proof. We first assume (i) and prove (ii). In view of Corollary 10.1.3(a), it suffices
to show that the matrices A(1) , . . . , A(n) are all positive definite. Obviously, these n
matrices are all symmetric (because A is symmetric). Now, fix an index k ∈ {1, . . . , n}
T
in Rk \ {0}; we must show that xTk A(k) xk > 0.

and a vector xk = x1 . . . xk
 T
Set x := x1 . . . xk 0 . . . 0 (with n − k zeros to the right of the vertical
dotted line, so that x ∈ Rn \ {0}). Then
(∗)
xTk A(k) xk = xT Ax > 0,

where (*) follows from the fact that A is positive definite and x ̸= 0 (because xk =
̸ 0).
So, A(k) is positive definite, and we deduce that (ii) holds.
We now assume (ii) and prove (i). We may assume inductively that the statement
is true for smaller matrices. More precisely, we assume that the following holds.

Induction hypothesis: For all
n′ ∈ {1, . . . , n
− 1}, and all symmetric

′ ′ ′
matrices B ∈ Rn ×n , if det B (1) , . . . , det B (n ) > 0, then B is positive
definite.

Set A = ai,j n×n . Then a1,1 = det A(1) > 0. If n = 1, so that A = a1,1 ,
 
 

then it is clear that A is positive definite, and we are done. So, from now, we assume
that n ≥ 2. Let A e be the matrix obtained from A by (sequentially or simultaneously)
performing the following elementary row operations on A:

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 698

a2,1
ˆ R2 → R2 − a1,1 R1 ;
a3,1
ˆ R3 → R3 − a1,1 R1 ;

..
.
an,1
ˆ Rn → Rn − a1,1 R1 .

1
Set B := A1,1 − a1,1 aaT ; obviously, B ∈ R(n−1)×(n−1) , and by Proposition 10.2.2, B
is symmetric. By Proposition 10.2.1, we have that
" #
T
a1,1 aT
 
a 1,1 a
A = 1 = ,
aaT
e
0 A1,1 − a1,1 0 B

and since we have already checked that a1,1 > 0, Theorem 10.2.3 guarantees that A
is positive definite if and only if B is positive definite. Thus, it is enough to show
that the symmetric matrix B ∈ R(n−1)×(n−1)
is positive definite. By the induction
hypothesis, it suffices to show that det B (1) , . . . , det B (n−1) > 0.

Fix an index k ∈ {1, . . . , n − 1}; we must show that det B (k) > 0. First of

all, by performing Laplace
expansion along the first column of A e(k+1) , we obtain

(k+1) (k)


det A e = a1,1 det B ; since a1,1 > 0 (and in particular, a1,1 ̸= 0), it follows
(k) 1 (k+1) e(k+1) is obtained



that det B = a1,1 det Ae . On the other hand, note that A
from A(k+1) by repeatedly adding a scalar multiple of one row to another; by
Theorem 7.3.2(c), this type of elementary row operation does not

alter the value
e(k+1) = det A(k+1) . We now deduce


of the determinant, and it follows that det A
that

(∗)
det B (k) = 1 det A e(k+1) = 1 det A(k+1)



a1,1 a1,1 > 0,

where (*) follows from the fact that a1,1 > 0 and det A(k+1) > 0. We now conclude

that (ii) holds, and we are done.

Example 10.2.10. Using Sylvester’s criterion of positive definiteness, determine

whether the matrix  
4 −2 4
A :=  −2 10 1 
4 1 6
is positive definite.

Solution. The matrix A is symmetric, and so we can use Sylvester’s criterion of

positive definiteness. We compute:

ˆ det A(1) = 4 = 4 > 0;

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 699

4 −2
ˆ det A(2) =


= 36 > 0;
−2 10

4 −2 4
ˆ det A(3) =


−2 10 1 = 36 > 0.
4 1 6
All three determinants are positive, and so by Sylvester’s criterion of positive
definiteness, the matrix A is positive definite.

Example 10.2.11. Using Sylvester’s criterion of positive definiteness, determine

whether the matrix  
2 −2 2 0
 −2 3 0 1 
A :=   2

0 6 0 
0 1 0 2
is positive definite.

Solution. The matrix A is symmetric, and so we can use Sylvester’s criterion of

positive definiteness. We compute:

ˆ det A(1) = 2 = 2 > 0;

2 −2
ˆ det A(2) =


= 2 > 0;
−2 3

2 −2 2
ˆ det A(3) =


−2 3 0 = 0.
2 0 6

Since det A(3) is not positive, Sylvester’s criterion of positive definiteness guarantees

that A is not positive definite.

Remark: Note that we did not need to compute det A(4) . In general, if we obtain

det A(k) ≤ 0 for some k, then we do not need to compute further, and we simply

deduce that A is not positive definite.

10.3 The Cholesky decomposition of positive definite

matrices
Proposition 10.3.1. Let L ∈ Rn×n be a lower triangular matrix with a positive
main diagonal. Then the matrix A := LLT is positive definite.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 700

Proof. First of all, we have that AT = (LLT )T = (LT )T LT = LLT = A, and so A is

symmetric. Now, fix a vector x ∈ Rn \ {0}. We must show that xT Ax > 0. We have
the following:
xT Ax = xT LLT x = (LT x)T (LT x) = (LT x) · (LT x) = ||LT x||2 .
Since LT is upper triangular (because L is lower triangular), Proposition 7.3.1
guarantees that det(L) is equal to the product of the main diagonal entries of LT ,
and by hypothesis, all those entries are positive. Thus, det(LT ) ̸= 0, and so by the
Invertible Matrix Theorem (see subsection 8.2.6), LT is invertible. Since x ̸= 0, it
follows that LT x ̸= 0. Consequently, ||LT x|| > 0, and it follows that xT Ax > 0.
This proves that A is positive definite.

Theorem 10.3.2 (Cholesky decomposition). For every positive definite matrix

A ∈ Rn×n , there exists a unique lower triangular matrix L ∈ Rn×n with a positive
main diagonal and satisfying A = LLT .
Proof. We proceed by induction on n.
For n = 1, we fix a positive definite matrix A = a in R1×1 , and we note that
 
√
a > 0 (because A is positive definite). We set L := a , and we observe that
T
A = LL . The uniqueness of L is obvious.
Now, fix a positive integer n, and assume the theorem is true for positive definite
matrices in Rn×n . Fix a positive definite matrix A ∈ R(n+1)×(n+1) , and set
α aT
 
A = ,
a A′
where α ∈ R, a ∈ Rn , and A′ ∈ Rn×n . By Theorem 10.2.3, we have that α > 0 and
that the matrix A′ − α1 aaT is positive definite. By the induction hypothesis, there
exists a unique lower triangular matrix L′ ∈ Rn×n with a positive main diagonal and
such that A′ − α1 aaT = L′ L′T . We now set
" √ #
α 0
L := √1 a L′
.
α n×n

Clearly, L is lower triangular with a positive main diagonal. Moreover, we have that
" √ #" √ #
T
α 0T α √1α aT
LL = √1 a L′
α 0 L′T

aT
 
α
= 1 T ′ ′T
a α aa + L L

α aT
 
= = A.
a A′

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 701

It remains to show that L is unique. For this, suppose that L1 ∈ R(n+1)×(n+1) is

a lower triangular matrix with a positive main diagonal and satisfying A = L1 LT1 ;
we must show that L1 = L. Set

β 0T
 
L1 = ,
b L′1

where β is some positive real number, b is some vector in Rn , and L′1 is some lower
triangular matrix in Rn×n with a positive main diagonal. Then

β 0T β bT
 2
βbT
   
T β
A = L1 L1 = = .
b L′1 0 L′T 1 βb bbT + L′1 L′T 1

We now have that

 2
βbT aT
  
β α
= A = .
βb bbT + L′1 L′T
1 a 1 T ′ ′T
α aa + L L

But then β 2 = α, βb = a, and bbT + L′1 L′T

1 =√
1 T ′ ′T
α aa + L L . This, together with
the fact that β > 0, yields the fact that β = α, b = √1α a, and L′1 L′T ′ ′T
1 = LL .
Moreover, by the uniqueness of L′ , we have that L′1 = L′ .10 Thus,
" √ #
β 0T
 
α 0
L1 = = √1 a L′
= L.
b L′1 α

This proves the uniqueness of L.

 
There is also an algorithm that, for a positive definite matrix A = ai,j n×n
in Rn×n , computes the Cholesky decomposition of A, i.e. computes the (unique)
lower triangular matrix L = ℓi,j n×n in Rn×n with a positive main diagonal and
 

satisfying A = LLT . We construct the matrix L column by column, from left to

right. Each column is constructed from top to bottom. Here is the algorithm.

1. We construct the first (i.e. leftmost) column of L as follows:

√
ˆ ℓ1,1 := a1,1 ,
a
ˆ ℓi,1 := √ i,1
a1,1 for all i ∈ {2, . . . , n}.

2. For all j ∈ {2, . . . , n}, assuming we have constructed the first (i.e. leftmost)
j − 1 columns of L, we construct the j-th column of L as follows (from top to
bottom):
10
Indeed, L′ is the unique lower triangular matrix in Rn×n with a positive main diagonal such that
A − α1 aaT = L′ L′T . Since L′1 is a lower triangular matrix in Rn×n with a positive main diagonal
′

such that L′1 L′T

1 = L L
′ ′T
= A′ − α1 aaT , the uniqueness of L′ guarantees that L′1 = L′ .

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 702

ˆ ℓi,j := 0 for all i ∈ {1, . . . , j − 1},

s
j−1
ˆ ℓj,j := aj,j −
P 2
ℓj,k ,
k=1
 j−1 
ˆ ℓi,j := 1 P
ℓj,j ai,j − ℓi,k ℓj,k for all i ∈ {j + 1, . . . , n}.
k=1

We omit the proof of correctness of the construction above, but it essentially follows
from Theorem 10.2.3 and from the proof of Theorem 10.3.2.
Example 10.3.3. Compute the Cholesky decomposition of the positive definite matrix
 
4 −2 4
A :=  −2 10 1  .
4 1 6
(The fact that A is positive definite follows from the solution of Example 10.2.4, 10.2.7,
or 10.2.10.)
 
Proof. We construct the matrix L = ℓi,j 3×3 as follows.
1. The first column of L is:
√ √
ˆ ℓ1,1 := a1,1 = 4 = 2,
a
ˆ ℓ2,1 := √ 2,1
a1,1 = −2
√
4
= −1,
a3,1
ˆ ℓ3,1 := √
a1,1 = 4
√
4
= 2.

2. The second column of L is:

ˆ ℓ1,2 := 0,
q
ˆ ℓ2,2 := a2,2 − ℓ22,1 = 10 − (−1)2 = 3,
p
   
ˆ ℓ3,2 := ℓ2,2
1
a3,2 − ℓ3,1 ℓ2,1 = 13 1 − 2 · (−1) = 1.

3. The third column of L is:

ˆ ℓ1,3 := 0,
ˆ ℓ2,3 := 0,
q
ˆ ℓ3,3 := a3,3 − (ℓ23,1 + ℓ23,2 ) = 6 − (22 + 12 ) = 1.
p

So, we have that  

2 0 0
L :=  −1 3 0  .
2 1 1
We see that the matrix L that we obtained is lower triangular with a positive main
diagonal, and it is straightforward to check that A = LLT . So, our computation is
correct.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 703

Remark: The main reason for interest in the Cholesky decomposition for positive
definite matrices is that it allows us to solve equations of the form Ax = b (where A
is positive definite) faster, as well as to compute the inverse of A faster. We omit
the details.

10.4 Positive definite matrices and the scalar product

Recall from subsection 6.1.1 that a scalar product (also called inner product) in a
real vector space V is a function ⟨·, ·⟩ : V × V → R that satisfies the following four
axioms:

r.1. for all x ∈ V , ⟨x, x⟩ ≥ 0, and equality holds if and only if x = 0;

r.2. for all x, y, z ∈ V , ⟨x + y, z⟩ = ⟨x, z⟩ + ⟨y, z⟩;

r.3. for all x, y ∈ V and α ∈ R, ⟨αx, y⟩ = α⟨x, y⟩;

r.4. for all x, y ∈ V , ⟨x, y⟩ = ⟨y, x⟩.

Moreover, as we saw in subsection 6.1.1, these four axioms imply the following:

r.2’. for all x, y, z ∈ V , ⟨x, y + z⟩ = ⟨x, y⟩ + ⟨x, z⟩;

r.3’. for all x, y ∈ V and α ∈ R, ⟨x, αy⟩ = α⟨x, y⟩.

As we saw in section 9.2, every scalar product in a real vector space is, in
particular, a symmetric bilinear form on that vector space. Furthermore, we have
the following theorem.

Theorem 10.4.1. Let V be a non-trivial, finite-dimensional real vector space, and

let ⟨·, ·⟩ be a bilinear form on V . Then the following are equivalent:

(i) ⟨·, ·⟩ is a scalar product in V ;

(ii) for all bases B of V , the matrix B of the bilinear form ⟨·, ·⟩ with respect to the
basis B is positive definite;

(iii) there exists a basis B of V such that the matrix B of the bilinear form ⟨·, ·⟩
with respect to the basis B is positive definite.

Proof. It is enough to prove the following sequence of implications: “(i) =⇒ (ii) =⇒

(iii) =⇒ (i).” The implication “(ii) =⇒ (iii)” is obvious, and so in fact, we just need
to prove the implications “(i) =⇒ (ii)” and “(iii) =⇒ (i).”
We first assume (i) and prove (ii). Fix any basis B = {b1 , . . . , bn } of V , and let
B be the matrix of the bilinear form ⟨·, ·⟩ with respect to the basis B. Since (i) holds,
the bilinear form ⟨·, ·⟩ is symmetric, and so by Theorem 9.2.2(a), the matrix B is

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 704

T
in Rn ; we must

also symmetric. Now, fix any non-zero vector x = x1 . . . xn
T
show that x Bx > 0. Set v := x1 b1 + · · · + xn bn , so that v B = x. Since x ̸= 0,
 
and since · B is an isomorphism, we have that v ̸= 0. Then

T (∗) (∗∗)
xT Bx =
  
v B
B v B
= ⟨v, v⟩ > 0,

where (*) follows from the fact that B is the matrix of the bilinear form ⟨·, ·⟩, and
(**) follows from (i), and more precisely, from the axiom r.1. This proves that B is
positive definite, and (ii) follows.
We now assume (iii) and prove (i). First of all, since ⟨·, ·⟩ is a bilinear form, it
satisfies axioms r.2 and r.3 from the definition of a scalar product; it remains to
show that it satisfies axioms r.1 and r.4. Using (iii), we fix a basis B of V such that
the matrix B of the bilinear form ⟨·, ·⟩ with respect to the basis B is positive definite.
Since B is positive definite, it is in particular symmetric, and so by Theorem 9.2.2(a),
the bilinear form ⟨·, ·⟩ is also symmetric, i.e. r.4 holds. It remains to show that r.1
holds. First, we have that
(∗) T
= 0T B0 = 0,
  
⟨0, 0⟩ = 0 B
B 0 B

 ⟨·, ·⟩
where (*) follows from the fact that B is the matrix of the bilinear form
with respect to the basis B. Now, fix any vector x ∈ V \ {0}. Since · B is an
 
isomorphism, we see that x B ̸= 0. We then have that

(∗)     (∗∗)
⟨x, x⟩ = x B
B x B
> 0,

where (*) follows from the fact that B is the matrix of the bilinear form ⟨·, ·⟩, and
(**) follows from the fact that B is positive definite and x B = ̸ 0. Thus, r.1 holds.
This proves (i).

Corollary 10.4.2. For any function ⟨·, ·⟩ : Rn ×Rn → R, the following are equivalent:

(i) ⟨·, ·⟩ is a scalar product on Rn ;

(ii) there exists a positive definite matrix A ∈ Rn×n such that for all x, y ∈ Rn , we
have ⟨x, y⟩ = xT Ay.

Proof. First of all, we may assume that the function ⟨·, ·⟩ : Rn × Rn → R is a

symmetric bilinear form on Rn , since each of (i) and (ii) individually implies this.
Indeed, if (i) holds, then ⟨·, ·⟩ is a symmetric bilinear form on Rn ,11 and if (ii) holds,
then Corollary 9.2.3(a) guarantees that ⟨·, ·⟩ is a symmetric bilinear form on Rn .12
11
This was explained at the beginning of section 9.2.
12
We are also using the fact that any positive definite matrix is, in particular, symmetric.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 705

Now, let A be the matrix of the bilinear form ⟨·, ·⟩ with respect to the standard
basis En of Rn . Then for all x, y ∈ Rn , we have that
T
x E A y E = xT Ay.
  
⟨x, y⟩ =
n n

Moreover, by Theorem 10.4.1, we have that ⟨·, ·⟩ is a scalar product in Rn if and

only if A is positive definite. This completes the argument.

Example 10.4.3. Consider the bilinear form ⟨·, ·⟩ on R3 given by

⟨x, y⟩ = 5x1 y1 + 2x1 y2 − 2x1 y3 + 2x2 y1 + x2 y2 − 2x3 y1 + 6x3 y3

T T
in R3 .
 
for all x = x1 x2 x3 and y = y1 y2 y3

(a) Compute the matrix of the bilinear form ⟨·, ·⟩ with respect to the standard basis
E3 of R3 .

(b) Is the bilinear form ⟨·, ·⟩ symmetric?

(c) Is the bilinear form ⟨·, ·⟩ a scalar product in R3 ?

Solution. (a) The matrix of the bilinear form ⟨·, ·⟩ is

 
5 2 −2
A =  2 1 0 .
−2 0 6

Remark: Note that the i, j-th entry of this matrix is simply the coefficient in front
of xi yj , for all indices i, j ∈ {1, 2, 3}.
(b) The matrix A from part (a) is symmetric, and so Theorem 9.2.2(a) guarantees
that ⟨·, ·⟩ is also symmetric.
(c) By Corollary 10.4.2, we know that ⟨·, ·⟩ is a scalar product in R3 if and only if
the matrix A is positive definite. We already saw that A is symmetric. We can now
perform any of our tests of positive definiteness on the matrix A. When computing
by hand with small-ish matrices, it is usually easiest to use Sylvester’s criterion of
positive definiteness (see Theorem 10.2.9). We compute:

ˆ det A(1) = det

 

5 = 5 > 0;

5 2
ˆ det A(2) =


= 1 > 0;
2 1

5 2 −2
ˆ det A(3) =


2 1 0 = 2 > 0.
−2 0 6

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 706

So, by Sylvester’s criterion of positive definiteness, the matrix A is positive definite,

and it follows that ⟨·, ·⟩ is a scalar product in A.

Example 10.4.4. Consider the bilinear form ⟨·, ·⟩ on R4 given by

⟨x, y⟩ = 2x1 y1 + x1 y3 + x1 y4 + x2 y2 + x3 y1 − x3 y3 + 5x3 y4 + x4 y1 + 5x4 y3 + 2x4 y4

T T
in R4 .
 
for all x = x1 x2 x3 x4 and y = y1 y2 y3 y4

(a) Compute the matrix of the bilinear form ⟨·, ·⟩ with respect to the standard basis
E4 of R4 .

(b) Is the bilinear form ⟨·, ·⟩ symmetric?

(c) Is the bilinear form ⟨·, ·⟩ a scalar product in R4 ?

Solution. (a) The matrix of the bilinear form ⟨·, ·⟩ is

 
2 0 1 1
 0 1 0 0 
A =  .
 1 0 −1 5 
1 0 5 2

(b) Since the matrix A from part (a) is symmetric, Theorem 9.2.2(a) guarantees
that the bilinear form ⟨·, ·⟩ on R4 is also symmetric.
(c) By Corollary 10.4.2, ⟨·, ·⟩ is a scalar product in R4 if and only if A is positive
definite. But since the main diagonal of A is not positive,13 Proposition 10.1.4
guarantees that A is not positive definite. We now conclude that ⟨·, ·⟩ is not a scalar
product in R4 .

Example 10.4.5. Consider the bilinear form ⟨·, ·⟩ on R3 given by

⟨x, y⟩ = 7x1 y1 + 3x1 y2 + 4x2 y1 + 3x2 y2 + x3 y3

T T
in R3 .
 
for all x = x1 x2 x3 and y = y1 y2 y3

(a) Compute the matrix of the bilinear form ⟨·, ·⟩ with respect to the standard basis
E3 of R3 .

(b) Is the bilinear form ⟨·, ·⟩ symmetric?

(c) Is the bilinear form ⟨·, ·⟩ a scalar product in R3 ?

13
Indeed, the third entry of the main diagonal is −1.

Irena Penev
Linear Algebra 1 & 2
Chapter 10. Matrix definiteness 707

Solution. (a) The matrix of the bilinear form ⟨·, ·⟩ is

 
7 3 0
A =  4 3 0 .
0 0 1

(b) The matrix A from part (a) is not symmetric, and so by Theorem 9.2.2(a),
the bilinear form ⟨·, ·⟩ on R3 is not symmetric, either.
(c) Since the bilinear form ⟨·, ·⟩ is not symmetric, it is not a scalar product.

Irena Penev
Linear Algebra 1 & 2
Bibliography 708

Bibliography

[1] Otto Bretscher, Linear algebra with applications (4th ed.), Pearson, Upper
Saddle River, 2009.

[2] Evan Dummit, Linear Algebra (part 4): Eigenvalues, Diagonalization, and the
Jordan Form (v. 3.00, 2022), [Link]
/linalgthy_4_eigenvalues_diagonalization_jordan_form.pdf (accessed
April 18, 2024).

[3] Alexey Frolov, Cholesky decomposition (In AlgoWiki), [Link]

[Link]/en/Cholesky_decomposition (accessed September 26, 2023).

[4] Milan Hladı́k, Lineárnı́ algebra (nejen) pro informatiky, MatfyzPress, Praha,
2019.

[5] Zoran Kadelburg, Vladimir Mićić, Srdjan Ognjanović, Analiza sa algebrom 2,

Krug, Beograd, 1997.

[6] David C. Lay, Linear algebra and its applications (3rd ed.), Pearson, Boston,
2006.

[7] marco2013, Similarity of real matrices over C (In Stack Exchange), https:
//[Link]/q/1129770 (accessed April 18, 2024).

[8] Iver Mølgaard Ottosen, The row reduction Algorithm, [Link]

[Link]/~ottosen/MMA2011/[Link] (accessed September 26, 2023).

[9] Permutation matrix (In Wikipedia), [Link]

utation_matrix (accessed September 26, 2022).

[10] Alexander Shen, Matematicheskaya induktsiya (5th ed.), MCNMO, Moscow,

2016 (in Russian). [Link] (accessed Septem-
ber 26, 2023).

[11] Georgi E. Shilov, Linear Algebra, Dover, New York, 1977.

[12] Gilbert Strang, Introduction to Linear algebra (3rd ed.), Wellesley-Cambridge,

Wellesley, 2003.
Bibliography 709

[13] Mark Wildon, A short proof of the existence of Jordan normal form, https:
//[Link]/~uvah099/Maths/[Link] (accessed April 18,
2024).

[14] Alex Zorn, Prove that the characteristic polynomial of a nilpotent matrix is xn
(In Stack Exchange), [Link] (accessed
April 18, 2024).

Irena Penev
Linear Algebra 1 & 2