Math 412: Advanced Linear Algebra

Lecture Notes

Lior Silberman
 These are rough notes for the Fall 2019 course, compiled January 10, 2023.
Copyright Lior Silberman. These notes are available for traditional academic reuse (with attri-
bution), and are specifically excluded from the terms of UBC Policy 81.
 Contents

Introduction 4
0.1. Goals and course plan (Lecture 1) 4
0.2. Review 5
0.3. Euler’s Theorem 6
Chapter 1. Constructions 7
1.1. Overall plan 7
1.2. Direct sum, direct product (Lectures 2-4) 7
1.3. Quotients (Lecture 5) 9
1.4. Hom spaces and duality (Lectures 6-8) 10
1.5. Multilinear algebra and tensor products (Lectures 9-14) 14
Chapter 2. Structure Theory: The Jordan Canonical Form 19
2.1. Introduction (Lecture 15) 19
2.2. The minimal polynomial (Lecture 16) 20
2.3. Generalized eigenspaces and Cayley–Hamilton (Lectures 17-18) 22
2.4. Nilpotent maps and Jordan blocks (Lectures 19-20) 25
2.5. The Jordan canonical form (Lecture 21) 27
Chapter 3. Vector and matrix norms 31
3.1. Norms on vector spaces (Lecture 22) 31
3.2. Norms on matrices (Lectures 23-24) 33
3.3. Example: eigenvalues and the power method (Lecture 25) 34
3.4. Sequences and series of vectors and matrices (Lectures 26-27) 35
3.5. The exponential series (Lecture 28) 37
3.6. Invertibility and the resolvent (Lecture 29) 40
3.7. Holomorphic calculus 41
Chapter 4. Vignettes 42
4.1. The exponential map and structure theory for GLn (R) 42
4.2. Representation Theory of Groups 42

3
 Introduction

Lior Silberman, lior@[Link], [Link]

Office: Math Annex 1112
Phone: 604-827-3031
For administrative details see the syllabus.
0.1. Goals and course plan (Lecture 1)
Math Metamath Skills
Reinforce basics Abstraction Hard problems
• Vector space, subspace Multiple ideas
• Linear independence, basis
• Linear map
• Eigenvalue, eigenvector
New ideas Constructions Abstract examples
• Direct sum, product Universal properties
• Hom(U,V ) and duality
• Quotients
• Tensor products
Structure theory for linear maps Canonical forms
• Matrix decompositions Algorithms
• LU, LL† and Computation
• Minimal poly, Cayley–Hamilton
• Jordan canonical form
Analysis Combining fields
• Norms
• Holomorphic calculus (etX )

4
 0.2. Review
0.2.1. Basic definitions. We want to give ourselves the freedom to have scalars other than real
or complex.
D EFINITION 1 (Fields). A field is a quintuple (F, 0, 1, +, ·) such that (F, 0, +) and (F \ {0} , 1, ·)
are abelian groups, and the distributive law ∀x, y, z ∈ F : x(y + z) = xy + xz holds.
L EMMA 2. In a field x · 0 = 0.
P ROOF. x ·0 = x ·(0 +0) = x ·0 +x ·0. Adding −(x ·0) and using the axioms gives 0 = x ·0. 
C OROLLARY 3. The associative and commutative laws for multiplication also holds for prod-
ucts involving 0.
E XERCISE 4. Generalized associative and commutative laws holds for both addition and mul-
tiplication.
E XAMPLE 5. R, C, Q. F2 (via addition and multiplication tables; ex: show this is a field), F p .
E XERCISE 6. Every finite field has pr elements for some prime p and some integer r ≥ 1. Fact:
there is one such field for every prime power.
D EFINITION 7. A vector space over a field F is a quadruple (V, 0, +, ·) where (V, 0, +) is an
abelian group, and · : F ×V → V is a map such that:
(1) 1F v = v.
(2) α (β v) = (αβ ) v.
(3) (α + β ) (v + w) = αv + β v + αw + β w.
L EMMA 8. 0F · v = 0, a · 0 = 0.
P ROOF. See Lemma 2. 
E XAMPLE 9. {0}, Rn , F X .
E XERCISE 10 (PS1). Check vector space axioms for F X
0.2.2. Linear maps and subspaces.
0.2.3. Bases and dimension. Fix a vector space V .
D EFINITION 11. Let S ⊂ V .
• v ∈ V depends on S if there are {vi }ri=1 ⊂ S and {ai }ri=1 ⊂ F such that v = ∑ri=1 ai vi [empty
sum is 0]
• Write SpanF (S) ⊂ V for the set of vectors that depend on S.
• Call S linearly dependent if some v ∈ S depends on S \ {v}, equivalently if there are
distinct {vi }ri=1 ⊂ S and {ai }ri=1 ⊂ F not all zero such that ∑ri=1 ai vi = 0.
• Call S linearly independent if it is not linearly dependent.
A XIOM 12 (Axiom of choice). Every vector space has a basis.

5
 0.3. Euler’s Theorem
Let G = (V, E) be a connected planar graph. A face of G is a finite connected component of
R2 \ G.
T HEOREM 13 (Euler). v − e + f = 1.
P ROOF. Arbitrarily orient the edges. Let ∂E : RE → RV be defined by f ((u, v)) = 1v − 1u ,
∂F : RF → RE be given by the sum of edges around the face.
L EMMA 14. ∂F is injective.
P ROOF. Faces containing boundary edges are independent. Remove them and repeat. 
L EMMA 15. Ker ∂E = Im ∂F .
P ROOF. Suppose a combo of edges is in the kernel. Following a sequence with non-zero
coefficients gives a closed loop, which can be expressed as a sum of faces. Now subtract a multiple
to reduce the number of edges with non-zero coefficients. 
L EMMA 16. Im(∂E ) is the the set of functions with total weight zero.
P ROOF. Clearly the image is contained there. Conversely, given f of total weight zero move
the weight to a single vertex using elements of the image. [remark: quotient vector spaces] 
Now dim RE = dim Ker ∂E + dim Im ∂E = dim Im ∂F + dim Im ∂E so
e = f + (v − 1) .

R EMARK 17. Using F2 coefficients is even simpler.

6
 CHAPTER 1

Constructions

1.1. Overall plan

(1) Direct sums
(a) External direct sum of two spaces
(b) Internal direct sum of two spaces
(c) Abstract direct sum of two or more spaces (through internal direct sum, not universal
property)
(2) Quotients
(a) Definition
(b) Examples
(c) Universal property in PS
(3) Duality
(a) Hom space
(b) Dual space
(c) Dual basis
(d) Dual of infinite direct sum leads to notion

0 of Ndirect product
(e) Pairings and bilinear forms (but F ⊕N ' F in PS3)
(f) dual map defined (properties in PS3)
(4) Tensor products
(a) Bilinear forms and maps
(b) The tensor product
(c) Universal properties (for direct sum, product, quotient, tensor product)
(d) Symmetric and antisymmetric tensors

1.2. Direct sum, direct product (Lectures 2-4)

Fix a field F.
1.2.1. Simplest case (Lecture 2).
C ONSTRUCTION 18 (External direct sum). Let U,V be vector spaces. Their direct sum, de-
noted U ⊕V , is the vector space whose underlying set is U ×V , with coordinate-wise addition and
scalar multiplication.
L EMMA 19. This really is a vector space.
R EMARK 20. The Lemma serves to review the definition of vector space.
P ROOF. Every property follows from the respective properties of U,V . 
R EMARK 21. Direct products of groups are discussed in 322.
7
 L EMMA 22. dimF (U ⊕V ) = dimF U + dimF V .
R EMARK 23. This Lemma serves to review the notion of basis.
P ROOF. Let BU , BV be bases of U,V respectively. Then {(u, 0V )}u∈BU t {(0U , v)}v∈BV is a
basis of U ⊕V . 
E XAMPLE 24. Rn ⊕ Rm ' Rn+m .
1.2.2. Internal sum and direct sum (Lecture 3). A key situation is when U,V are subspaces
of an “ambient” vector space W .
L EMMA 25. Let W be a vector space, U,V ⊂ W . Then SpanF (U ∪V ) = {u + v | u ∈ U, v ∈ V }.
P ROOF. RHS contained in the span by definition. It is a subspace (non-empty, closed under
addition and scalar multiplication) which contains U,V hence contains the span. 
D EFINITION 26. The space in the previous lemma is called the sum of U,V and denoted U +V .
L EMMA 27. Let U,V ⊂ W . There is a unique homomorphism U ⊕ V → U + V which is the
identity on U,V .
P ROOF. Define f ((u, v)) = u + v. Check that this is a linear map. 
P ROPOSITION 28 (Dimension of sums). dimF (U +V ) = dimF U + dimF V − dimF (U ∩V ).
P ROOF. Consider the map f of Lemma 27. It is surjective by Lemma 25. Moreover Ker f =
{(u, v) ∈ U ⊕V | u + v = 0W }, that is
Ker f = {(w, −w) | w ∈ U ∩V } ' U ∩V .
Since dimF Ker f + dimF Im f = dim (U ⊕V ) the claim now follows from Lemma 22. 
R EMARK 29. This was a review of that formula. Alternative proof by starting from a basis of
U ∩V and extending to bases of U,V , which is basically revisiting the proof of the formula.
D EFINITION 30 (Internal direct sum). We say the sum is direct if f is an isomorphism.
T HEOREM 31. For subspaces U,V ⊂ W TFAE
(1) The sum U +V is direct and equals W ;
(2) U +V = W and U ∩V = {0}
(3) Every vector w ∈ W can be uniquely written in the form w = u + vv.
P ROOF. (1) ⇒ (2): U +V = W by assumption, U ∩V = Ker f .
(2) ⇒ (3): the first assumption gives existence, the second uniqueness.
(3) ⇒ (1): existence says f is surjective, uniqueness says f is injective. 
1.2.3. Finite direct sums (Lecture 4). Three possible notions: (U ⊕V ) ⊕ W , U ⊕ (V ⊕W ),
vector space structure on U × V × W . These are all the same. Not just isomorphic (that is, not
just same dimension), but also isomorphic when considering the extra structure of the copies of
U,V,W . How do we express this?
D EFINITION 32. W is the internal direct sum of its subspaces {Vi }i∈I if it spanned by them
and each vector has a unique representation as a sum of elements of Vi (either as a finite sum of
non-zero vectors or as a zero-extended sum).
8
 R EMARK 33. This generalizes the notion of “linear independence” from vectors to subspaces.
L EMMA 34. Each of the three candidates contains an embedded copy of U,V,W and is the
internal direct sum of the three images.
P ROOF. Easy. 
P ROPOSITION 35. Let A, B each be the internal direct sum of embedded copies of U,V,W .
Then there is a unique isomorphism A → B respecting this structure.
P ROOF. Build up the isomorphism from the pieces. 
R EMARK 36. (1) Proof only used result of Lemma, not specific structure; but (2) proof im-
plicitly relies on isomorphism to U ×V ×W ; (3) We used the fact that a map can be defined using
values on copies of U,V,W (4) Exactly same proof as the facts that a function on 3d space can be
defined on bases, and that that all 3d spaces are isomorphic.
• Dimension of ⊕iVi by induction.
D EFINITION 37. Abstract arbitrary direct sum.
• Block diagonality (but it’s also a practice problem in PS2)
R EMARK 38. Infinite direct sums are a supplement to PS2.

1.3. Quotients (Lecture 5)

Recall that for a group G and a normal subgroup N, we can endow the quotient G/N with group
structure (gN) (hN) = (gh) N.
• This is well-defined, gives group.
• Have quotient map q : G → G/N given by g 7→ gN.
• Homomorphism theorem: any f : G → H factors as G → G/ Ker( f ) follows by isomor-
phism.
• If N < M < G with both N, M normal then q(M) ' M/N is normal in G/N and (G/N) / (M/N) '
(G/M).
Now do the same for vector spaces.
L EMMA 39. Let V be a vector space, W a subspace. Let π : V → V /W be the quotient as
abelian groups. Then there is a unique vector space structure on V /W making π a surjective
linear map.
P ROOF. We must set α (v +W ) = αv + W . This is well-defined and gives the isomorphism.
Use quotient map to verify vector space axioms. 
d
E XAMPLE 40. V = C1 (0, 1), space of continuously differentiable functions. Then dx :V→
d 1


C(0, 1) vanishes on R1 and hence induces a map dx : C (0, 1)/R1 → C(0, 1). Note that the
inverse of this map is what we call “indefinite integral” – whose images is exactly an equivalence
class “function+c”.
E XAMPLE 41. The definite integral is a linear function which vanishes on functions which
are non-zero at countably many points. More generally, integration works on functions modulu
functions which are zero a.e.
9
 FACT 42. The properties above persist for vector spaces.
• How to use quotients: “kill off” part of the vector spaces that is irrelevant (linear maps
vanish there) or already understood.
R EMARK 43. Bijection of subspaces of quotietns and universal property are a problem in PS2
1.4. Hom spaces and duality (Lectures 6-8)
1.4.1. Hom spaces (Lecture 5 continued or start of lecture 6).
D EFINITION 44. HomF (U,V ) will denote the space of F-linear maps U → V .
L EMMA 45. HomF (U,V ) ⊂ V U is a subspace, hence a vector space.
D EFINITION 46. V 0 = HomF (V, F) is called the dual space.
Motivation 1: in PDE. Want solutions in some function space V . Use that V 0 is much bigger to
find solutions in V 0 , then show they are represented by functions.
1.4.2. The dual space, finite dimensions.
N OTE 47. In lecture ignore infinite dimensions (but make statements which are correct in
general).
C ONSTRUCTION 48 (Dual basis). Let B = {bi }i∈I ⊂ V be a basis. Write v ∈ V uniquely as
v = ∑i∈I ai bi (almost all ai = 0) and set ϕi (v) = ai .
L EMMA 49. These are linear functionals.
P ROOF. Represent αv + v0 in the basis. 
E XAMPLE 50. V = F n with standard basis, get ϕi (x) = xi . Note every functional has the form
ϕ(x) = ∑ni=1 ϕ(ei )ϕi (x).
R EMARK 51. Alternative construction: ϕi is the unique linear map to F satisfying ϕi (b j ) = δi, j .
L EMMA 52. The dual basis is linearly independent. It is spanning iff dimF V < ∞.
P ROOF. Evaluate a linear combination at b j .
If V is finite-dimensional, enumerate the basis as {bi }ni=1 . Then for any ϕ ∈ V 0 and any v ∈ V
write v = ∑i ai bi and then
ϕ(v) = ∑ ai ϕ(bi ) = ∑ (ϕ(bi )) ϕi (v)
i i
so
ϕ = ∑ (ϕ(bi )) ϕi ∈ SpanF {ϕi } .
i
In the infinite-dimensional case let φ = ∑i∈I ϕi . Then φ is a well-defined linear functional
which depends on every coordinate hence not in the span of the {ϕi }. 
R EMARK 53. This isomorphism V → V 0 is not canonical: the functional ϕi depends on the
whole basis B and not only on bi , and the dual basis transforms differently from the original basis
under change-of-basis.
The argument above used evaluation – let’s investigate that more.
10
 P ROPOSITION 54 (Double dual). Given v ∈ V consider the evaluation map ev : V 0 → F given by
ev (ϕ) = ϕ(v). Then v 7→ ev is a linear injection V ,→ V 00 , an isomorphism iff V is finite-dimensional.
P ROOF. The vector space structure on V 0 (and on F V in general) is such that ev is linear. That
the map v 7→ ev is linear follows from the linearity of the elements of V 0 . For injectivity let v ∈ V
be non-zero. Extending v to a basis, let ϕv be the element of the dual such that ϕv (v) = 1. Then
ev (ϕv ) 6= 0 so ev 6= 0. If dimF V = n then dimF V 0 = n and thus dimF V 00 = n and we have an
isomorphism. 
The map V ,→ V 00 is natural: the image ev of v is intrinsic and does not depend on a choice of
basis.
1.4.3. The dual space, infinite dimensions (Lecture 7).
L EMMA 55 (Interaction with past constructions). We have
(1) (V /U)0 ,→ V 0 as {ϕ ∈ V | ϕ(U) = {0}}.
(2) (U ⊕V )0 ' U 0 ⊕V 0 .
P ROOF. Universal property. 
C OROLLARY 56. Since (F)0 ' F, it follows by induction that (F n )0 ' F n .
What about infinite sums?
• The universal property gives a bijection ( i∈I Vi )0 ←→ i∈I Vi0 , more generally
L Ś
!
1:1
M ą
HomF Vi , Z ←→ Homi (Vi , Z) .
i∈I i∈I
– LHS has a vector space structure – should get one on the right.
– Which leads us to observe
Ś that:
• Any Cartesian product i∈I Wi has a natural vector space structure, coming from point-
wise addition and scalar multiplication.
– Note that the underlying set is
ą
Wi = { f | f is a function with domain I and ∀i ∈ I : f (i) ∈ Wi }
i∈I
( )
[
= f:I→ Wi | f (i) ∈ Wi .
i∈I
∗ RMK: AC means Cartesian products nonempty, but our sets have a distin-
guished element so this is not an issue.
def
– Define α (wi )i∈I + (w0i )i∈I = (αwi + w0i )i∈I . This gives a vector space structure.
– Denote theLresulting vector space ∏i∈I Wi and called it the direct product of the Wi .
• The bijection ( i∈I Vi )0 ←→ ∏i∈I Vi0 is now a linear isomorphism [in fact, the vector space
structure on the right is the one transported by the isomorphism].
We now investigate ∏i Wi in general.
• Note that it contains a copy of each Wi (map w ∈ Wi to the sequence which has w in the
ith position, and 0 at every other position).
11
 • And these copies are linearly independent: if a sum of such vectors from distinct Wi is
zero, then every coordinate was zero.
• Thus ∏i Wi contains i∈I Wi as an internal direct sum.
L

– This subspace is exactly the subset {wL

∈ ∏i Wi | supp(w) is finite}.
– And in fact, that subspace proves that i∈I Wi exists.
– But ∏i Wi contains many other vectors – it is much bigger.
E XAMPLE 57. R⊕N ⊂ RN – and the latter is the dual!
C OROLLARY 58. The dual of an infinite-dimensional space is much bigger than the sum of the
duals, and the double dual is bigger yet.
1.4.4. Question: we only have finite sums in linear algebra. What about infinite sums?
Answer: no infinite sums in algebra. Definition of ∑∞ n=1 an = A from real analysis relies on analytic
properties of A (it’s a number close to the partial sums), not algebraic properties.
But, calculating sums can be understood in terms of linear functionals.
L EMMA 59 (Results from Calc II, reinterpreted). Let S ⊂ RN denote the set of sequences a
such that ∑∞n=1 an converges.
⊕N
(1) R ⊂ S ⊂ RN is a linear subspace.
(2) Σ : S → R given by Σ(a) = ∑∞
n=1 an is a linear functional.

Philosophy: Calc I,II made element-by-element statements, but using linear algebra we can ex-
press them as statements on the whole space.
Now questions about summing are questions about intelligently extending the linear functional
Σ to a bigger subspace. BUT: if an extension is to satisfy every property of summing series, it is
actually the trivial (no) extension.
For more information let’s talk about limits of sequences instead (once we have notions of
generalized limits of sequences we can apply them to the sequence of partial sums of a series).
D EFINITION 60. Let c ⊂ `∞ ⊂ RN be the sets of convergent, respectively bounded sequences.
L EMMA 61. c ⊂ `∞ are subspaces, and limn→∞ : c → R is a linear functional.
E XAMPLE 62. Let C : RN → RN be the Cesàro map (Ca)N = N1 ∑N n=1 an . This is clearly linear.
Let CS = C (c) be the set of sequences which are Cesàro-convergent, and set L ∈ CS0 by L(a) =
−1

limn→∞ (Ca). This is clearly linear (composition of linear maps). For example, the sequence
(0, 1, 0, 1, · · · ) now has the limit 21 .
L EMMA 63. If a ∈ c then Ca ∈ c and they have the same limit. Thus L above is an extension of
limn→∞ .
T HEOREM 64. There are two functionals LIM, limω ∈ (`∞ )0 (“Banach limit”, “limit along
ultrafilter”, respectively) such that:
(1) They are positive (map non-negative sequences to non-negative sequences);
(2) Agree with limn→∞ on c;
(3) And, in addition
(a) LIM ◦ S = LIM where S : `∞ → `∞ is the shift.
(b) limω (an bn ) = (limω an ) (limω bn ).
12
 1.4.5. Pairings and bilinear forms (Lecture 8). Goal: identify the dual of a vector space in
concerete terms. For this we need an abstract notion of dual not tied to the particular realization
V 0 (similar to how we have an abstract notion of direct sum as “space generated by independent
copies of Vi ” which is not tied to the concrete realization as the subspace of tuples of finite support
in ∏i Vi ).
O BSERVATION 65. The evaluation map V ×V 0 → F given by
hv, ϕi = ϕ(v)
is bilinear (=linear in each variable).
Note that linearity in the first variable is equivalent to the linearity of ϕ, while linearity in the
second variable is equivalent to the definition of the vector space structure on V 0 .
D EFINITION 66 (Pairings / bilinear maps). For any two vector spaces U,V a (bilinear) pairing
between U,V is a map
h·, ·i : U ×V → F
which is linear in each variable separately. Similarly we define a bilinear map U ×V → Z.
E XAMPLE 67. The standard inner product on F n : hu, vi = ∑i ui vi . More generally, given B ∈
Mm×n (F) have a pairing on F m × F n given by
hu, vi = ∑ ui Bi j v j .
i

More generally, given any bilinear pairing of U,V choose bases {ui }i∈I ⊂ U, v j j∈J ⊂ V and
define the Gram matrix by
Bi j = ui , v j .
We can then compute the pairing of any two vectors: by the distributive law (“FOIL”)
* +
∑ aiui, ∑ b j v j = ∑ a j Bi j b j .
i i i, j
Conversely, any matrix B defines a bilinear pairing (aside: this is a linear bijection if you give
pairings the obvious vector space structure).
1.4.6. Pairings: duality and degeneracy. Fix a bilinear formh·, ·i : U ×V → F. Then for any
u ∈ U we get a map ϕu : V → F by ϕu (v) = hu, vi.
(1) ϕu is linear (∈ V 0 ) iff the pairing is linear in the second variable.
(2) The map U → V 0 given by u → ϕu is linear iff the pairing is linear in the first variable.
We conclude that every pairing gives a map U → V 0 , and equivalently also a map V → U 0 .
L EMMA 68. We have a linear bijection {pairings on U ×V } ←→ HomF (U,V 0 )
P ROOF. The inverse map associates to each f ∈ HomF (U,V 0 ) the bilinear form
hu, vi f = ( f (u)) (v) .

D EFINITION 69. Call the bilinear map non-degenerate if both maps U → V 0 , V → U 0 are
embeddings.
13
 L EMMA 70. A pairing is non-degenerate iff for every non-zero u ∈ U there is v ∈ V such that
hu, vi 6= 0 and conversely.
Key idea: if the map V → U 0 associated to a pairing is bijective, then we can use V as a model
for U 0 via the pairing.
E XAMPLE 71. The dot product is a non-degenerate pairing F n × F n hence identifies (F n )0 with
F n.
Two further examples from functional analysis:
First we fix a compact topological space X. Then for any finite Borel measure µ on X and any
continuous f ∈ C(X) we have the integral
Z
f dµ .

This is a bilinear pairing C(X) × {finite signed measures on X} which is non-degenerate.

T HEOREM 72 (Riesz representation theorem). Let X be compact. Then every continuous lin-
ear functional on C(X) is given by a finite measure (that is, the continuous dual C(X)0 can be
represented by the space of measures).
Second, the inner product on Hilbert space is a non-degenerate pairing!
T HEOREM 73 (Riesz representation theorem). Let H be a Hilbert space. Then every continu-
ous linear functional on H is of the form hu, ·i.
1.4.7. The dual of a linear map (Lecture 8, continued).
C ONSTRUCTION 74. Let T ∈ Hom(U,V ). Set T 0 ∈ Hom(V 0 ,U 0 ) by (T 0 ϕ) (v) = ϕ (T v).
L EMMA 75. This is a linear map Hom(U,V ) → Hom(V 0 ,U 0 ). An isomorphism if U,V finite-
dimensional.
L EMMA 76. (T S)0 = S0 T 0
P ROOF. PS3 

1.5. Multilinear algebra and tensor products (Lectures 9-14)

1.5.1. Bilinear forms (Lecture 9).
D EFINITION 77. Let {Vi }i∈I be vector spaces, W another vector space. A function f : ×i∈I
Vi → W is said to be multilinear if it is linear in each variable.
E XAMPLE 78 (Bilinear maps). (1) f (x, y) = xy is bilinear F 2 → F.
(2) The map (T, v) 7→ T v is a multilinear
map Hom(V,W ) ×V → W .
(3) For a matrix A ∈ Mn,m (F) have x, y 7→ t xAy on F n × F m .
(4) For ϕ ∈ U 0 , ψ ∈ V 0 have (u, v) 7→ ϕ(u)ψ(v), and finite combinations of those.
R EMARK 79. A bilinear function on U ×V is not the same as a linear function on U ⊕V . For
example: is f (au, av) equal to a f (u, v) or to a2 f (u, v)? That said, Vi was universal for maps
L

from Vi . It would be nice to have a space which is universal for multilinear maps. We only discuss
the finite case.
14
 E XAMPLE 80. A multilinear function B : U × {0} → F has B(u, 0) = B(u, 0 · 0) = 0 · B(u, 0) =
0. A multilinear function B : U × F → F has B(u, x) = B(u, x · 1) = xB(u, 1) = xϕ(u) where ϕ(u) =
B(u, 1) ∈ U 0 .

We can reduce everything to Example 78(3): Fix bases {ui } , v j . Then
!
B ∑ xiui, ∑ y j v j = ∑ xi B(ui , v j )y j = t xBy
i i i, j
where Bi j = B(ui , v j ). Note: xi = ϕi (u) where {ϕi } is the dual basis. Conclude that


(1.5.1) B = ∑ B ui , v j ϕi ψ j .
i, j


Easy to check that this is an expansion in a basis (check against ui , v j ). We have shown:

P ROPOSITION 81. The set ϕi ψ j i, j is a basis of the space of bilinear forms U ×V → F.
C OROLLARY 82. The space of bilinear forms on U ×V has dimension dimF U · dimF V .
R EMARK 83. Also works in infinite dimensions, since can have the sum (1.5.1) be infinite –
every pair of vectors only has finite support in the respective bases.
1.5.2. The tensor product (Lecture 10-11). Now let’s fix U,V and try to construct a space
that will classify bilinear maps on U ×V .
• Our space will be generated by terms u ⊗ v on which we can evaluate f to get f (u, v).
• Since f is multilinear, f (au, bv) = ab f (u, v) so need (au) ⊗ (bv) = ab (u ⊗ v).
• Similarly, since f (u1 + u2 , v) = f (u1 , v) + f (u2 , v) want (u1 + u2 ) ⊗ (v1 + v2 ) = u1 ⊗ v1 +
u2 ⊗ v1 + u1 ⊗ v2 + u2 ⊗ v2 .
C ONSTRUCTION 84 (Tensor product). Let U,V be spaces. Let X = F ⊕(U×V ) be the formal
span of all expressions of the form {u ⊗ v}(u,v)∈U×V . Let Y ⊂ X be the subspace spanned by
{(au) ⊗ (bv) − ab (u ⊗ v) | a, b ∈ F, (u, v) ∈ U ×V }
and
{(u1 + u2 ) ⊗ (v1 + v2 ) − (u1 ⊗ v1 + u2 ⊗ v1 + u1 ⊗ v2 + u2 ⊗ v2 ) | ∗∗} .
Then set U ⊗V = X/Y and let ι : U ×V → U ⊗V be the map ι(u, v) = (u ⊗ v) +Y .
T HEOREM 85. ι is a bilinear map. For any space W any any bilinear map f : U × V → W ,
there is a unique linear map f˜ : U ⊗V → W such that f = f˜ ◦ ι.
P ROOF. Uniqueness is clear, since f˜(u ⊗ v) = f (u, v) fixes f˜ on a generating set. For existence
we need to show that if f˜ : X → W is defined by f˜(u ⊗ v) = f (u, v) then f˜ vanishes on Y and hence
descends to U ⊗V . 
P ROPOSITION 86. Let BU , BV be bases for U,V respectively. Then {u ⊗ v | u ∈ BU , v ∈ BV } is
a basis for U ⊗V .
P ROOF. Spanning: use bilinearity of ι. Independence: let ϕu u∈B ⊂ U 0 , ψv v∈B ⊂ V 0
 
U V
be the dual bases. Then ϕu ψv is a bilinear map U × V → F, and the sets {u ⊗ v}(u,v)∈BU ×BV and

ϕ]u ψv (u,v)∈B ×B are dual bases. 
U V

15
 C OROLLARY 87. dimF (U ⊗V ) = dimF U · dimF V .
E XAMPLE 88. Examples of tensor products
(1) Rn ⊗ Rm encoded as matrices.
(2) Polynomial algebra: F[x] ⊗ F[y] ' F[x, y]
(3) Functions on product spaces: let X,Y be compact then C(X) ⊗C(Y ) dense in C(X ×Y ).
• Note Fubini’s Theorem can be obtained from this.
(4) Quantum mechanics: state space for two particles is the (completion of the) tensor product
of the state spaces for the individual particles.
• Note that many states are not “pure tensors”.
1.5.3. The Universal Property (Lecture 12). Abstract view of tensor products (note the in-
definite article!)
D EFINITION 89 (Abstract tensor product). A tensor product of the spaces U,V is a pair (W, ι)
where W is a vector space, ι : U × V → W is bilinear, and for every bilinear map f : U × V → Z
there is a unique f¯ ∈ HomF (W, Z) such that f = f¯ ◦ ι.
R EMARK 90. Informally, ι is the “most general bilinear map on U ×V .
E XAMPLE 91. We show that the image of ι spans W . Indeed if not there would be a non-zero
functional f¯ ∈ W 0 vanishing on the span of that image, and then f¯ ◦ ι = 0 ◦ ι would both be the
zero bilinear form, violating uniqeness.
Before we explain how to use this property, we return to the example of direct sum.
D EFINITION 92 (Abstract direct sum). A direct sum of {Vi }i∈I is a space W and maps ei ∈
HomF (Vi ,W ) such that for every system of maps fi ∈ HomF (Vi , Z) there is a unique f¯ : W → Z
such that fi = f¯ ◦ ei for each i.
P ROPOSITION 93. Direct sums are unique up to a unique isomorphism.
P ROOF. Suppose W 0 is another direct sum with system of inclusions {e0i }i∈I . Then W 0 is a
space with system of maps, so by hypothesis there is a unique f¯0 : W → W 0 such that e0i = f¯0 ◦ ei
(note that this is the key requirement from an isomorphism preserving the structure, so we see both
that such a map exists and that it is unique, but we don’t know it is an isomorphism yet).
By symmetry there is also f¯ : W 0 → W such that ei = f¯ ◦ e0i .
Next, note that f¯ ◦ f¯0 and idW are both maps W → W and they satisfy
f¯ ◦ f¯0 ◦ ei = f¯ ◦ f¯0 ◦ ei

= f¯ ◦ e0i choice of f¯0

= ei choice of f¯
= idW ◦ei
so by the universal property (applied to the system of maps ei with target W ), f¯ ◦ f¯0 = idW . By
symmetry we also have f¯0 ◦ f¯ = idW 0 and we are done. 
Consider now direct products. Reversing all the arrows (on the board, modify the definition in
red ink), we get:
16
 D EFINITION 94 (Abstract direct product). A direct product of {Vi }i∈I is a space W and maps
πi ∈ HomF (W,Vi ) such that for every system of maps fi ∈ HomF (Z,Vi ) there is a unique f¯ : Z → W
such that fi = πi ◦ f¯ for each i.
P ROPOSITION 95. Direct products are unique up to a unique isomorphism.
P ROOF. Reverse all the arrows in the previous proof. 
P ROPOSITION 96. Tensor products are unique up to a unique isomorphism.
P ROOF. Let (W, ι), (W 0 , ι 0 ) be two tensor products of U,V . Then since ι 0 : U × V → W 0 is
bilinear there is a unique f¯0 : W → W 0 such that ι 0 = f¯0 ◦ ι.
R EMARK . Note that this is a basic requirement for an “isomorphism of tensor products”: it
must identify the vector representing u ⊗ v on both sides. Using the universal property we saw
both that we can actually identify these vectors and that this identification extends to a linear map
of the tensor product spaces.
Continuing with the proof, for the same reason there is f¯ : W 0 → W such that ι = f¯ ◦ ι 0 .
R EMARK . One can now finish the proof by noting, for example, that f¯ ◦ f¯0 fixes elements of
the form u ⊗ v and that these span the tensor product. But we prefer a proof which doesn’t “look
under the hood” and use the vectors in the vector space.
Finally, we have
f¯ ◦ f¯0 ◦ ι = f¯ ◦ f¯0 ◦ ι

= f¯ ◦ ι 0 choice of f¯0
=ι choice of f¯
= idW ◦ι .
We have shown that the bilinear map ι : U ×V → W is represented by both homomorphisms f¯ ◦ f¯0
and idW so they must be equal, and by symmetry we conclude that f¯0 ◦ f¯ = idW 0 as well so that
f¯, f¯0 are the desired isomorphisms. 
R EMARK 97. This point of view leads to “category theory” where one forgets about the specific
algebraic structure under consideration (here vector spaces) and considers only the statements
about objects and homomorphisms. This way theorems about “direct sums”, say, apply for any
construction of direct sum regardless of the underlying algebraic structures.
For example, we get direct sums (and direct products) of groups, rings, modules, vector spaces.
But we also get direct sums of topological spaces (this turns out to be the disjoint union) and direct
products of topological spaces (this is the Tychonoff product).
1.5.4. Extension of scalars. Let K/F be a field extension.
P ROPOSITION 98. The field operations of K endow it with the structure of an F-vectorspace.
def
L EMMA -D EFINITION 99. Let V be an F-vectorspace. Then VK = K ⊗F V has the natural
structure of a K-vectorspace.
T HEOREM 100. Let B ⊂ V be an F-basis. Then {1K ⊗ v}v∈B ⊂ VK is a K-basis
17
 E XERCISE 101. The map V 7→ VK is functorial: every f ∈ Hom(U,V ) extends naturally to a
map fK : UK → VK (take fK = 1K ⊗ f ), with the same matrix as f (with respect to bases at in the
Theorem). If L/K if a further extension then there is a natural transformation VL ' (VK )L .
E XERCISE 102. Extension of scalars respects all the constructions: direct sum, direct product,
quotient, kernel, image, tensor product: (U ⊗F V )K ' UK ⊗K VK .
1.5.5. Symmetric and antisymmetric tensor products (Lecture 13). For motivation, think
of U ⊗U as the state space of a pair of identical quantum particles. What happens when we swap
them? Represent swapping them by the obvious map T ∈ EndF (U ⊗U).
FACT 103. Some fundamental particles (“Bosons”) always have states in the +1-eigenspace.
Other particles (“Fermions”) always have states in the −1-eigenspace. Note that a state like
u ⊗ u is permitted to Bosons, but prohibited (!) to Fermions. This is called the “Fermi exclusion
principle”.
A SSUMPTION 104. For this section, char(F) = 0.
Let (12) ∈ S2 act on V ⊗V by exchanging the factors (why is this well-defined?).
L EMMA 105. Let T ∈ EndF (U) satisfy T 2 = Id. Then U is the direct sum of the two eigenspaces.
V2
D EFINITION 106. Sym2 V and V are the eigenspaces.
P ROPOSITION 107. Generating sets and bases.
In general, let Sk act on V ⊗k .
• What do we mean by that? Well, this classifies n-linear maps V × · · · ×V → Z. Universal
property gives isom of (U ⊗V ) ⊗W , U ⊗ (V ⊗W ).
• Why action well-defined? After all, the set of pure tensors is nonlinear. So see first as
multilinear map V n → V ⊗n .
• Single out Symk V , k V . Note that there are other representations.
V

• Claim: bases
Vk
1.5.6. Bases of Symk , , determinants (Lecture 14).
E XAMPLE 108. 2 R3 is three-dimensional, which explains the cross-product as giving anti-
V

symmetric 2-tensors rather than vectors.

P ROPOSITION 109. Symmetric/antisymmetric tensors are generating sets; bases coming from
subsets of basis.
 
Tool: the maps Pk± : V ⊗k → V ⊗k given by Pk± (v1 ⊗ · · · ⊗ vk ) = k!1 ∑σ ∈Sk (±)σ vσ (1) ⊗ · · · ⊗ vσ (k) .

L EMMA 110. These are well defined (extensions of linear maps). Fix elements of Symk V ,
V respectively, images are in those subspaces (check τ ◦ Pk± = (±)τ Pk± ). Conclude that image
Vk

is spanned by image of basis.

E XAMPLE 111. Exterior forms of top degree and determinants.

18
 CHAPTER 2

Structure Theory: The Jordan Canonical Form

2.1. Introduction (Lecture 15)

2.1.1. The two paradigmatic problems. Fix a vector space V of dimension n < ∞ (in this
chapter, all spaces are finite-dimensional unless stated otherwise), and a map T ∈ End(V ). We will
try for two kinds of structural results:
(1) [“decomposition”] T = RS where R, S ∈ End(V ) are “simple”
(2) [“canonical form”] There is a basis {vi }ni=1 ⊂ V in which the matrix of T is “simple”.
E XAMPLE 112. (From 1st course)
(1) (Gaussian elimination) Every matrix A ∈ Mn (F) can be written in the form A = E1 · · · Ek ·
Arr where Ei are “elementary” (row operations or rescaling) and Arr is row-reduced.
(2) (Spectral theory) Suppose T is diagonable. Then there is a basis in which T is diagonal.
As an example of how to use (1), suppose det(A) is defined for matrices by column expan-
sion. Then can show (Lemma 1) that det(EX) = det(E) det(X) whenever E is elementary and that
(Lemma 2) det(AX) = det(A) det(X) whenever A is row-reduced. One can then prove
T HEOREM 113. For all A, B, det(AB) = det(A) det(B).
P ROOF. Let D = {A | ∀X : det(AX) = det(A) det(X)}. Then we know that all Arr ∈ D and that
if A ∈ D then for any elementary E, det((EA)X) = det(E(AX)) = det(E) det(AX) = det(E) det(A) det(X) =
det(EA) det(X) so EA ∈ D as well. It now follows from Gauss’s Theorem that D is the set of all
matrices. 

2.1.2. Triangular matrices.

D EFINITION 114. A ∈ Mn (F) is upper (lower) triangular if ...
Significance: these are very good for computation. For example:
L EMMA 115. The lower-triangular matrix L is invertible iff its diagonal entries are non-zero.
The proof is:
A LGORITHM 116 (Forward-substitution). Let L be lower-triangular with non-zero diagonal
bi −∑i−1
j=1 li j x j
entries. Then the solution to Lx = b is given by xi = lii for i = 1, 2, . . . , n.
R EMARK 117. Note that the algorithm does exactly as many multiplications as non-zero entries
in U. Hence better than Gaussian elimination for general matrix (O(n3 )), really good for sparse
matrix, and doesn’t require storing the matrix entries only the way to calculate ui j (in particular no
need to find inverse).
19
 E XERCISE 118. (1) Express this as a formula for the inverse of a lower-triangular matrix (2)
Develop the backward-substitution algorithm for upper-triangular matrices and find a formula for
their inverses.
C OROLLARY 119. If A = LU we can efficiently solve Ax = b.
Note that we don’t like to store inverses. For example, because they are generally dense matri-
ces even if L,U are sparse.
We now try to look for a vector-space interpretation of being triangular. For this note that if
U ∈ Mn (F) is upper-triangular then
Ue1 = u11 e1 ∈ Span {e1 }
Ue2 = u12 e1 + u22 e2 ∈ Span {e1 , e2 }
.. .
. = ..
Uek ∈ Span {e1 , . . . , ek }
.. .
. = .. .
In particular, we found a family of subspaces Vi = Span {e1 , . . . , ei } such that U(Vi ) ⊂ Vi , such
that{0} = V0 ⊂ V1 ⊂ · · · ⊂ Vn = F n and such that dimVi = i.
E XERCISE 120 (Cholesky decomposition). If A is positive-definite, then A = LLt for a lower-
triangular matrix L.
T HEOREM 121. T ∈ End(V ) has an upper-triangular matrix wrt some basis iff there are T -
invariant subspaces {0} = V0 ⊂ V1 ⊂ · · · ⊂ Vn = F n with dimVi = i.
P ROOF. We just saw necessity. For sufficiency, given Vi choose for 1 ≤ i ≤ n, vi ∈ Vi \ Vi−1 .
These exist (the dimension increases by 1), are a linearly independent set (each vector is inde-
pendent of its predecessors) and the first i span Vi (by dimension count). Finally for each i,
T vi ∈ T (Vi ) ⊂ Vi = Span {v1 , . . . , vi } so the matrix of T in this basis is upper triangular. 
2.2. The minimal polynomial (Lecture 16)
Recall we have an n-dimensional F-vector space V .
• A key tool for studying linear maps is studying polynomials in the maps (we saw how to
analyze maps satisfying T 2 = Id, for example).
• We will construct a gadget (the “minimal polynomial”) attached to every linear map on
V . It is a polynomial, and will tell us a lot about the map.
• Computationally speaking, this polynomial cannot be found efficiently. It is a tool of
theorem-proving in abstract algebra.
D EFINITION 122. Given a polynomial f ∈ F[x], say f = ∑di=0 ai xi and a map T ∈ End(V ) set
(with T 0 = Id)
d
f (T ) = ∑ ai T i .
i=0
L EMMA 123. Let f , g ∈ F[x]. Then ( f + g) (T ) = f (T ) + g(T ) and ( f g) (T ) = f (T )g(T ). In
other words, the map f 7→ f (T ) is a linear map F[x] → End(V ), also respecting multiplication (“a
map of F-algebras”, but this is beyond our scope).
20
 P ROOF. Do it yourself. 
• Given a linear map our first instinct is to study the kernel and the image. [Aside: the
kernel is an ideal in the algebra].
• We’ll examine the kernel and leave the image for later.
L EMMA 124. There is a non-zero polynomial f ∈ F[x] such that f (T ) = 0. In fact, there is
such f with deg f ≤ n2 .
2
P ROOF. F[x] is infinite-dimensional while EndF (V ) is finite-dimensional. Specifically, dimF F[x]≤n =
n + 1 while dimF EndF (V ) = n2 .
2 
R EMARK 125. We will later show (Theorem of Cayley–Hamilton) that the characteristic poly-
nomial PT (x) = det (x Id −T ) from basic linear algebra has this property.
• Warning: we are about to divide polynomials with remainder.
P ROPOSITION 126. Let I = { f ∈ F(x) | f (T ) = 0}. Then I contains a unique non-zero monic
polynomial of least degree, say m(x), and I = {g(x)m(x) | g ∈ F[x]} is the set of multiples of m.
P ROOF. Let m ∈ I be a non-zero member of least degree. Dividing by the leading coefficient
we may assume m monic. Now suppose m0 is another such. Then m − m0 ∈ I (this is a subspace)
is of strictly smaller degree. It must therefore be the zero polynomial, and m is unique. Clearly if
g ∈ F[x] then (gm) (T ) = g(T )m(T ) = 0. Conversely, given any f ∈ I we can divide with remainder
and write f = qm + r for some q, r ∈ F[x] with deg r < deg m. Evaluating at T we find r(T ) = 0,
so r = 0 and f = qm. 
D EFINITION 127. Call m(x) = mT (x) the minimal polynomial of T .
R EMARK 128. We will later prove directly that deg mT (x) ≤ n.
E XAMPLE 129. (Minimal polynomials)
(1) T = Id,
 m(x) = x − 1.
1
(2) T = , T 2 = 0 but T 6= 0 so mT (x) = x2 .
   
1 1 1 2
, T2 = so T 2 − Id = 2 (T − Id) so T 2 − 2T + Id = 0 so mT (x)|(x −


(3) T =
1 1
2 2
 T−
1) . But  Id 6= 0 so mT (x) = (x − 1) .
1
(4) T = , T 2 = Id so mT (x) = x2 − 1 = (x − 1) (x + 1).
1
   
1 1
• In the eigenbasis the matrix is – we saw this in a previous class.
±1 −1
 
−1
(5) T = , T 2 = − Id so mT (x) = x2 + 1.
1
(a) If F = Q or F = R this is irreducible. No better basis.  
1
(b) If F = C (or Q(i)) then factor mT (x) = (x − i) (x + i) and in the eigenbasis
±i
 
−i
the matrix has the form .
i
21
 d
(6) V = F[x]<n (polynomials of degree less than n), T = dx . Then T n = 0 but T n−1 6= 0
(why?) so mT (x) = xn .
(7) [To be proved in problem set] Let D = diag (a1 , . . . , an ) be diagonal, its entries being the
distinct numbers {b1 , · · · , br } (perhaps with repetition). Then its minimal polynomial is
∏ri=1 (x − br ) [cf (1),(4),(5)]
We now connect the minimal polynomial with the spectrum.
L EMMA 130 (Spectral calculus). Suppose that T v = λ v. Then f (T )v = f (λ )v.
P ROOF. Work it out at home. 
R EMARK 131. The same proof shows that if the subspace W is T -invariant (T (W ) ⊂ W ) then
W is f (T )-invariant for all polynomials f .
C OROLLARY 132. If λ is an eigenvalue of T then mT (λ ) = 0. In particular, if mT (0) 6= 0 then
T is invertible (0 is cannot be eigenvalue)
We now use the minimality of the minimal polynomial.
T HEOREM 133. T is invertible iff mT (0) 6= 0.
P ROOF. Suppose that T is invertible and that ∑di=1 ai T i = 0 [note a0 = 0 here]. Then this is not
the minimal polynomial since multiplying by T −1 also gives
d−1
∑ ai+1T i = 0 .
i=0

C OROLLARY 134. λ ∈ F is an eigenvalue of T iff λ is a root of mT (x).
P ROOF. Let S = T −λ Id. Then mS (x) = mT (x+λ ). Then λ ∈ SpecF (T ) ⇐⇒ S not invertible ⇐⇒
mS (0) = 0 ⇐⇒ mT (λ ) = 0. 
R EMARK 135. The characteristic polynomial PT (x) also has this property – this is how eigen-
values are found in basic linear algebra.

2.3. Generalized eigenspaces and Cayley–Hamilton (Lectures 17-18)

R EMARK 136. A slower schedule covers 2.3.1 in one lecture, 2.3.2 and half of 2.3.3 in another,
and finishes 2.3.3 in a third lecture.
2.3.1. Generalized Eigenspaces (Lecture 17). Continue with T ∈ EndF (V ), dimF (V ) = n.
Recall that T is diagonable iff V is the direct sum of the eigenspace. For non-diagonable maps we
need something more sophisticated.
P ROBLEM 137. Find a matrix A ∈ M2 (F) which only has a 1-d eigenspace.
D EFINITION 138. Call v ∈ V a generalized eigenvector of T if for some λ ∈ F and k ≥ 1,
(T − λ )k v = 0. Let Vλ ⊂ V denote the set of generalized λ -eigenvectors and 0. Call λ a generalized
eigenvalue of T if Vλ 6= {0}.
In particular, if T v = λ v then v ∈ Vλ .
22
 P ROPOSITION 139 (Generalized eigenspaces). (1) Each Vλ is a T -invariant subspace.
(2) Let λ 6= µ. Then (T − µ) is invertible on Vλ .
(3) Vλ 6= {0} iff λ ∈ SpecF (T ).
0
P ROOF. Let v, v0 ∈ Vλ be killed by (T − λ )k , (T − λ )k respectively. Then αv + β v0 is killed by
0
(T − λ )max{k,k } . Also, (T − λ )k T v = T (T − λ )k v = 0 so T v ∈ Vλ as well.
Let v ∈ Ker(T − µ) be non-zero. By Lemma 130, for any k we have (T − λ )k v = (µ − λ )k v 6= 0
so v ∈
/ Vλ .
Finally, given λ and non-zero v ∈ Vλ let k be minimal such that (T − λ )k v = 0. Then (T − λ )k−1 v
is non-zero and is an eigenvector of eigenvalue λ . 
⊂ V is direct.
L
T HEOREM 140. The sum λ ∈SpecF (T ) Vλ

P ROOF. Let ∑ri=1 vi = 0 be a minimal dependence with vi ∈ Vλi for distinct λi . Applying
(T − λr )k for k large enough to kill vr we get the dependence.
r−1
∑ (T − λr )k vi = 0 .
i=1
k
Now (T − λr ) vi ∈ Vλi since these are T -invariant subspaces, and for 1 ≤ i ≤ r − 1 is non-zero
since T − λr is invertible there. This shorter dependence contradicts the minimality. 
R EMARK 141. The sum 
may very well be empty – there are non-trivial maps without eigen-
−1
values (for example ∈ M2 (R)).
1
2.3.2. Algebraically closed fields. We all know that sometimes linear maps fail to have eigen-
values, even though they “should”. In this course we’ll blame the field, not the map, for this
deficiency.
D EFINITION 142. Call the field F algebraically closed if every non-constant polynomial f ∈
F[x] has a root in F. Equivalently, if every non-constant polynomial can be written as a product of
linear factors.
FACT 143 (Fundamental theorem of algebra). C is algebraically closed.
R EMARK 144. Despite the title, this is a theorem of analysis.
Discussion. The goal is to create enough eigenvalues so that the generalized eigenspaces ex-
plain all of V . The first point of view is that we can simple “define the problem away” by restrict-
ing to the case of algebraically closed fields. But this isn’t enough, since sometimes we are given
maps
 over
 other fields. This already
 appears in the diagonable case, dealt with in 223: we can view
−1 −1
∈ M2 (R) instead as ∈ M2 (C), at which point it becomes diagonable. In other
1 1
words, we can take a constructive point of view:
• Starting with any field F we can “close it” by repeatedly adding roots to polynomial
equations until we can’t, obtaining an “algebraic closure” F̄ [the difficulty is in showing
the process eventually stops].
– This explains the “closed” part of the name – it’s closure under an operation.
23
 – [Q: do you need the full thing? A: In fact, it’s enough to pass to the splitting field of
the minimal polynomial]
• We now make this work for linear maps, with three points of view:
(1) (matrices) Given A ∈ Mn (F) view it as A ∈ Mn (F̄), and apply the theory there.
(2) (linear
Ln
maps) Given T ∈ EndF (V ), fix a basis {vi }ni=1 ⊂ V , make the formal span
V̄ = i=1 F̄vi and extends T to V̄ by the property of having the same matrix.
(3) (coordinate free) Given V over F set V̄ = F̄ ⊗F V (considering F̄ as an F-vectorspace),
and extend T (by T̄ = IdF̄ ⊗F T ).
2.3.3. The direct sum decomposition and Cayley–Hamilton (Lecture 18).
L EMMA 145. Suppose F is algebraically closed and that 1 ≤ dimF V < ∞. Then every T ∈
EndF (V ) has an eigenvector.
P ROOF. mT (x) has roots. 
We suppose now that F is algebraically closed, in other words that every linear map has an
eigenvalue. The following is the key structure theorem for linear maps:
L
T HEOREM 146. (with F algebraically closed) We have V = λ ∈SpecF (T ) Vλ .

P ROOF. Let mT (x) = ∏ri=1 (x − λi )ki and let W = ri=1 Vλi . Supposing that W 6= V , let V̄ =
L

V /W and consider the quotient map T̄ ∈ EndF (V̄ ) defined by T̄ (v +W ) = T v+W . Since dimF V̄ ≥
1, T̄ has an eigenvalue there. We first check that this eigenvalue is one of the λi . Indeed, for any
polynomial f ∈ F[x], f (T̄ ) (v +W ) = ( f (T )v)+W , and in particular mT (T̄ ) = 0 and hence mT̄ |mT .
Renumbering the eigenvalues, we may assume V̄λr 6= {0}, and let v ∈ V be such that v+W ∈ V̄λr
ki ki
/ W . Since ∏r−1
is non-zero, that is v ∈ r−1
i=1 (T̄ − λi ) is invertible on V̄λr , u = ∏i=1 (T − λi ) v ∈
/ W.
kr
But (T − λr ) u = mT (T )v = 0 means that u ∈ VλR ⊂ W , a contradiction. 
P ROPOSITION 147. In mT (x) = ∏ri=1 (x − λi )ki , the number ki is the minimal k such that
(T − λi )k = 0 on Vλi .

P ROOF. Let Ti be the restriction of T to Vλi . Then (Ti − λi )k is the minimal polynomial by
assumption. But mT (Ti ) = 0. It follows that (x − λi )k |mT and hence that k ≤ ki . Conversely, since

k j k
∏ j6=i T − λ j is invertible on Vλi , we see that (T − λi ) i = 0 there, so ki ≥ k. 
Summary of the construction so far:
• F algebraically closed field, dimF V = n, T ∈ EndF (V ).
• mT (x) = ∏ri=1 (x − λi )ki the minimal polynomial.
• Then V = ri=1 Vλi where on Vλi we have(T − λi )ki = 0 but (T − λi )ki −1 6= 0.
L

We now study the restriction of T to each Vλi , via the map N = T − λi , which is nilpotent of degree
ki .
D EFINITION 148. A map N ∈ EndF (V ) such that N k = 0 for some k is called nilpotent. The
smallest such k is called its degree of nilpotence.
L EMMA 149. Let N ∈ EndF (V ) be nilpotent. Then its degree of nilpotence is at most dimF V .
P ROOF. Exercise. 
24
 P ROOF. Define subspaces Vk by V0 = V and Vi+1 = N(Vi ). Then V = V0 ⊃ V1 · · · ⊃ Vi ⊃ · · · .
If at any stage Vi = Vi+1 then Vi+ j = Vi for all j ≥ 1, and in particular Vi = {0} (since Vk = 0). It
follows that for i < k, dimVi+1 < dimVi and the claim follows. 
C OROLLARY 150 (Cayley–Hamilton Theorem). Suppose F is algebraically closed. Then
mT (x)|pT (x) and, equivalently, pT (T ) = 0. In particular, deg mT ≤ dimF V .
Recall the that the characteristic polynomial of T is the polynomial pT (x) = det (x Id −T ) of
degree dimF V , and that is also has the property that λ ∈ SpecF (T ) iff pT (λ ) = 0.
P ROOF. The linear map x Id −T respects the decomposition V = ri=1 Vλi . We thus have
L

pT (x) = ∏ri=1 pT Vλ (x). Since pT Vλ (x) has the unique root λ , it is the polynomial (x − λ )dimF Vλ ,
i
so
r
pT (x) = ∏ (x − λi )dimVλi .
i=1
Finally, ki is the degree of nilpotence of (T − λi ) on Vλi . Thus ki ≤ dimF̄ Vλi 
We now resolve a lingering issue:
L EMMA 151. The minimal polynomial is independent of the choice of the field. In particular,
the Cayley–Hamilton Theorem holds over any field.
P ROOF. Whether 1, T, . . . , T d−1 ⊂ EndF (V ) are linearly dependent or not does not depend

on the field. 
T HEOREM 152 (Cayley–Hamilton). Over any field we have mT (x)|pT (x) or, equivalently,
pT (T ) = 0.
P ROOF. Extend scalars to an algebraic closure. This does not change either of the polynomials
mT , pT . 

2.4. Nilpotent maps and Jordan blocks (Lectures 19-20)

2.4.1. Jordan blocks (Lecture 19). We finally turn to the problem of finding good bases for
linear maps, starting with the nilpotent case. Here F can be an arbitrary field.
To start with, consider the filtration {0} ( Ker(N) ( Ker(N 2 ) ( · · · ( V = Ker(N k ) where k is
the degree of nilpotence. Choose a basis as follows:
(1) Choose a basis of size m1 in Ker(N).
(2) Add m2 vectors to get a basis of Ker(N 2 ) ⊃ Ker(N)
..
.
(3) Add mk vectors to get a basis of V = Ker(N k ).
What is the matrix of N in this basis? It is block-upper triangular. On the diagonal the blocks are
mi × mi , all zero. Above each block is a block of size mi−1 × mi where every column is non-zero
(every if v ∈ Ker(N r ) \ Ker(N r−1) then Nv ∈ Ker(N r−1 ) \ Ker(N r−2 )). Above that no information.
We now try to improve this by carefully choosing the basis to encode more of the action of N.
L EMMA 153. Let N ∈ End(V ) be nilpotent. Let B ⊂ V be a set of vectors such that N(B) ⊂
B ∪ {0}. Then B is linearly independent iff B ∩ Ker(N) is.
25
 P ROOF. One direction is clear. For the converse, let ∑ri=1 ai vi = 0 be a minimal dependence in
B. Apply N s where s is maximal such that for some i, N s vi 6= 0 (perhaps s = 0). Then
r
∑ aiN svi = 0
i=1

where each N s vi ∈ B ∪ {}. If some N s vi = 0 the new dependence is shorter. Thus they are all
non-zero then by the maximality of s they are all in Ker(N), so we obtain a dependence among
B ∩ Ker(N), a contradiction. 
C OROLLARY 154. Let N ∈ End(V ) and let v ∈ V be non-zero such that N k v = 0 for some k
k−1
(wlog minimal). Then N i v i=0 is linearly independent.


k−1
P ROOF. N is nilpotent on Span N i v i=0 , this set is invariant, and its intersection with Ker N


is exactly N k−1 v 6= {0}.




2.4.2. Jordan canonical form for nilpotent maps (Lecture 20). Our goal is now to decom-
pose V as a direct sum of N subspaces (“Jordan blocks”) each of which has a basis as in the
Corollary.
T HEOREM 155 (Jordan form for nilpotent maps). Let N ∈ EndF (V ) be nilpotent. We then have
a decomposition V = rj=1 V j where each V j is an N-invariant Jordan block.
L

   
1 2 3 1

E XAMPLE 156. A = 1  2 3 = 1 1 2 3 .
 
−1 −2 −3 −1
   
1
1
2


• A = 1   1 2 3  1  1 2 3 = 0, so A is nilpotent. The characterstic poly-
−1 −1
3
nomial must be x .          
 1  3 1  3 1 
• The image of A is Span  1  . Since A −1 = 1 , Span
     −1 , 1 
 
−1 0 −1 0 −1 
  
is a block.        
2  3 1 2 
• Taking any other vector in the kernel (say, −1) we get the basis −1 ,  1  , −1
0  0 −1 0 
in which A has the matrix
  
0 1
 0 .
(0)

P ROOF. Let N have degree of nilpotence d and kernel W . For 1 ≤ k ≤ d define Wk = Im(N k ) ∩
W , so that W0 = W ⊃ W1 ⊃ Wd = {0}. Now choose a basis C of W compatible with this de-
composition – in other words choose subsets Ck ⊂ Wk such that ∪k≥k0 Ck is a basis for Wk0 . Let
C = ∪d−1 ki
k=0 Ck = {vi }i∈I and for each i define ki by vi ∈ Cki . Choose ui such that N ui = vi , and for
26
 (
v j≥1
1 ≤ j ≤ ki set vi, j = N ki − j ui so that vi,1 = ui and in general Nvi, j = i, j−1 . It is clear that
0 j=1
 ki 
SpanF vi, j j=1 is a Jordan block, and that B = ui, j i, j is a union of Jordan blocks. 

• The set B is linearly independent: by construction, N(B) ⊂ B ∪ {0} and B ∩ W = C is

independent.
• The set B is spanning: We prove by induction on k ≤ d that SpanF (B) ⊃ Ker(N k ). This
is clear for k = 0; suppose the result for 0 ≤ k < d, and let v ∈ Ker(N k+1 ). Then N k v ∈ Wk ,
so we can write
Nkv = ∑ ai vi
i:ki ≥k

ai N k vi,k+1 .


= ∑
i:ki ≥k

It follows that !
Nk v − ∑ ai vi,k+1 = 0.
i:ki ≥k

By induction, v − ∑i:ki ≥k ai vi,k ∈ SpanF (B), and it follows that v ∈ SpanF (B).
D EFINITION 157. A Jordan basis is a basis as in the Theorem.
L EMMA 158. Any Jordan basis for N has exactly dimF Wk−1 − dimF Wk blocks of length k.
Equivalently, up to permuting the blocks, N has a unique matrix in Jordan form.
  
P ROOF. Let vi, j be a Jordan basis. Then Ker N = Span vi,1 , while vi, j | ki ≥ k, j ≤ ki − k
is a basis for Im(N k ). Clearly vi,1 | ki ≥ k then spans Wk and the claim follows.



2.5. The Jordan canonical form (Lecture 21)

T HEOREM 159 (Jordan canonical form). Let T ∈ EndF (V ) and suppose that mT splits into lin-
ear factors in F (for example, that F is algebraically closed). Then there is a basis vλ ,i, j λ ,i, j of V
(
v j≥1
such that vλ ,i, j i, j ⊂ Vλ is a basis, and such that (T − λ ) vλ ,i, j = λ ,i, j−1

. Furthermore,
0 j=1
writing Wλ = Ker(T −λ ) for the eigenspace, we have for each λ , that
1 ≤ i ≤ dimF Wλ and that the
number of i such that 1 ≤ j ≤ k is exactly dimF (T − λ )k−1Vλ ∩Wλ − dimF (T − λ )kVλ ∩Wλ .


Equivalently, T has a unique matrix in Jordan canonical form up to permuting the blocks.
C OROLLARY 160. The algebraic multipicity of λ is dimF Vλ . The geometric multiplicity is the
number of blocks.
 
2 2 3
E XAMPLE 161 (Jordan forms). (1) A1 =  1 3 3  = I + A. This has character-
−1 −2 −2
3
istic polynomial (x − 1) , A1 − I = A and we are back in example 156.
27
 (2) (taken from
 Wikibooks:Linear  Algebra) pB (x) = (x − 6)4 . 
7 1 2 2 1 1 2 2
 1 4 −1 −1 0  1 −2 −1 −1
Let B = −2 1 5 −1, B = B − 6I = −2 1 −1 −1. Gaussian elimina-
  
1 1 2 8 1 1 2 2
   
3 −3 0 0 0 3 3 3
 1 −2 −1 −1 02 0 3 3 3
tion shows B0 = E −3 3
, B = 
0 −6 −6 −6 and B = 0. Thus
 03
0 0
0 0 0 0 0 3 3 3
0 t
Ker B = {(x, y, z, w) | x = y = −(z + w)} is two-dimensional. We see that the image of
B02 is spanned by (3, 3, −6, 3)t , which is (say) B0 (2, −1, −1, 2)t which (being the last
column) was  B0
(0,0, 0,1)t . 
Another
  vector
 in the t
 kernel is (−1, −1, 1, 0) , and we get the
 0
 2 3 −1  
 0 −1  3  −1
Jordan basis   ,   ,   ,   .
       

 0 −1 −6 1  
 1 2 3 0 
 
4 0 1 0
 2 2 3 0 4 2 2
(3) C = −1 0 2 0 acting on V = R with pC (x) = (x − 2) (x − 3) . Then C − 2I =

4 0 1 2
     
2 0 1 0 1 0 1 0 0 0 0 0
, C −3I =  2 −1 3 , (C − 3I)2 = −3 1 −4 0. Thus
 2 0 3 0  0  

−1 0 0 0 −1 0 −1 0   0 0 0 0
4 0 1 0 4 0 1 −1 −1 0 2 1
Ker (C − 2I) = Span {e2 , e4 }, which must be the 2d generalized eigenspace V2 giving two
1×1 blocks. For λ = 3, Ker (C − 3I) = {(x, y, z, w)t | z = y = −x, w = 3x} = Span {(1, −1, −1, 3)t }.
This isn’t the whole generalized eigenspace, and

Ker (C − 3I)2 = (x, y, z, w)t | y = 3x + 4z, w = x − 2z = Span (1, −1, −1, 3)t , (1, 3, 0, 1)t .
 

This must be the generalized eigenspace V3 , since it’s 2d. We need to find the im-
age of (C − 3I) [V3 ]. One vector is in the kernel, so we try the other one, and indeed
 − 3I)
(C  (1,
 3, 1)t =
0, −1,−1,
(1,  3). This gives us a 2x2 block,
 so in the basis 

 0 0 1 1  (2)
       
1 , 0 , −1 , 3 the matrix has the form  (2) 
  
. Note
 0 0 −1 0  3 1 

 0 
1 3 1  3
how the image of (C − 3I)2 is exactly V2 (why?)
28
(4) V = R6 . pD (x) = t 6 + 3t 5 −10t 3 −15t 2 −9t−2 = (t + 1)5 (t−2):
0 0 0 0 −1 −1 1 0 0 0 −1 −1
   
 0 −8 4 −3 1 −3  0 −7 4 −3 1 −3
−3 13 −8 6 2 9 −3 13 −7 6 2 9
   
D= , D+I =  ,
−2 14 −7 4 2 10  −2 14 −7 5 2 10 
 1 −18 11 −11 2 −6  1 −18 11 −11 3 −6
−1 19 −11 10 −2 7 −1 19 −11 10 −2 8
1 −1 0 1 −2 −3 0 0 0 0 0 0
   
−2 −16 9 −11 4 −3  0 −54 27 −27 0 −27
−1 37 −18 17 2 21  0 108 −54 54 0 54 
   
(D + I)2 =  , (D + I)3
= .
1 35 −18 19 −2 15  0 108 −54 54 0 54 
 
−1 −53 27 −28 2 −24 0 −162 81 −81 0 −81
2 52 −27 29 −4 21 0 162 −81 81 0 81
(5) First, V2 must be a 1-dimensional eigenspace. Gaussian elimination finds the eigenvector
(01, −2, −2, 3, −3)t . Next, V−1 must be 5-dimensional. Row-reduction gives: D + I →
1 0 0 0 −1 −1 2 0 −1 3 −4 −5
   
0 0 0 1 0 −1/2 0 2 −1 1 0 1
0 1 0 0 1 3/2 0 0 0 0 0 0
   
2
, (D + I) →  . So the Ker(D + I)
   
0 0 1 0 2 3/2  0 0 0 0 0 0

0 0 0 0 0 0  0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0
2
is two-dimensional (since (D + I) 6= 0 there will be a block of size at least 3; since
(D + I)3 has rank one, it has the 5d kernel V−1 = {x | x3 = 2x2 + x4 + x6 } so the largest
block is 3, and so the other block must have size 2. We need a vector from the generalized
eigenspace in the image of (D + I)2 . Since (D + I)3 e1 = 0 but the first column of (D + I)2
is non-zero, we see that (D + I)2 e1 = (1, −2, −1, 1, −1, 2)t has preimage (D + I) e1 =
(1, 0, −3, −2, 1, −1)t , and we obtain our first block. Next, we need an eigenvector in the
kernel and image of D + I, but any vector in the kernel is also in the image (no blocks
of size 1), so we cam take any vector in Ker(D + I) independent of the one we already
have. Using the row-reduced form we see that (1, −1, −2, 0, 1, 0)t is such a vector. Then
we solve
1 0 0 0 −1 −1 1
   
 0 −7 4 −3 1 −3 −1
−3 13 −7 6 2 9 −2
   
x =  ,
−2 14 −7 5 2 10  0
 
 1 −18 11 −11 3 −6 1
−1 19 −11 10 −2 8 0

finding for example the vector (1, 0, −1, −1, 0, 0)t and our second block. We conclude
0 1 1 1 1 1 
           

1 −2 0 0 −1 0 

 

           
           
−2 −1 −3 0 −2 −1

that in the basis   ,   ,   ,   ,   ,   the matrix has the

−2  1  −2 0  0  −1

 3  −1  1  0  1   0  


 

−3 2 −1 0 0 0
29
form 

2  

 −1 1 



 −1 1  


 −1 


 −1 1 
−1

30
 CHAPTER 3

Vector and matrix norms

For the rest of the course our field of scalars is either R or C.

3.1. Norms on vector spaces (Lecture 22)

3.1.1. Review of metric spaces.
D EFINITION 162. A metric space is a pair (X, dX ) where X is a set, and dX : X × X → R≥0 is
a function such that for all x, y, z ∈ X, dX (x, y) = 0 iff x = y, dX (x, y) = dX (y, z) and (the triangle
inequality) dX (x, z) ≤ dX (x, y) + dX (y, z).
N OTATION 163. For x ∈ X and r ≥ 0 we write BX (x, r) = {y ∈ X | dX (x, y) ≤ r} for the closed
ball of radius r around x, B◦X (x, r) = {y ∈ X | dX (x, y) < r} for the open ball.
D EFINITION 164. Let (X, dX ) , (Y, dY ) be metric spaces and let f : X → Y be a function.
(1) We say f is continuous if ∀x ∈ X : ∀ε > 0 : ∃δ > 0 : f (BX (x, δ )) ⊂ BY ( f (x), ε).
(2) We say f is uniformly continuous ∀ε > 0 : ∃δ > 0 : ∀x ∈ X : f (BX (x, δ )) ⊂ BY ( f (x), ε).
(3) We say f is Lipschitz continuous if in (2) we can take δ = ε/L, in other words if for all
x 6= x0 ∈ X,
dY f (x), f (x0 ) ≤ LdX x, x0 .

In that case we let k f kLip denote the smallest L for which this holds.

Clearly (3) ⇒ (2) ⇒ (1).

L EMMA 165. The composition of two functions of type (1),(2),(3) is again a function of that
type. In particular, k f ◦ gkLip ≤ k f kLip kgkLip .

D EFINITION 166. We call the metric space (X, dX ) complete if every Cauchy sequence con-
verges.
3.1.2. Norms. Fix a vector space V .
D EFINITION 167. A norm on V is a function k·k : V → R≥0 such that kvk = 0 iff v = 0,
kαvk = |α| kvk and ku + vk ≤ kuk + kvk. A normed space is a pair (V, k·k).
L EMMA 168. Let k·k be a norm on V . Then the function d(u, v) = ku − vk is a metric.
E XERCISE 169. The map k7→k d is a bijection between norms on V and metrics on V which are
(1) translation-invariant d(u, v) = d (u + w, v + w) and (2) 1-homogenous: d (αu, αv) = |α| d (u, v).
The restriction of a norm to a subspace is a norm.
31
 3.1.3. Finite-dimensional examples.
E XAMPLE 170. Standard norms on Rn and Cn :
(1) The supremum norm kvk∞ = max {|vi |}ni=1 , parametrizing uniform convergence.
(2) kvk1 = ∑ni=1 |vi |.
 1/2
n 2
(3) The Euclidean norm kvk2 = ∑i=1 |vi | , connected to the inner product hu, vi =
n
∑i=1 ui vi (prove 4 inequality from this by squaring norm of sum).
(4) For 1 < p < ∞, kvk p = (∑ni=1 |vi | p )1/p .
P ROOF. These functions are clearly homogeneous, and clearly are non-zero if v 6= 0; the only
non-trivial part is the triangle inequality (“Minkowsky’s inequality”). This is easy for p = 1, ∞,
well-known for p = 2. Other cases resolved in supplement to PS8. 
E XERCISE 171. Show that lim p→∞ kvk p = kvk∞ .
We have a geometric interpretation. The unit ball of a norm is the set B = B(0, 1) = {v ∈ V | kvk ≤ 1}.
1
This determines the norm ( kvk is the largest α such that αv ∈ B). Now applying a linear map to B
gives a the ball of a new norm.
E XERCISE 172. Draw the unit balls for k·k p on R2 , R3 for p = 1, 2, ∞.
P ROPOSITION 173 (Pullback). Let T : U ,→ V be an injectivel linear map. Let k·kV be a norm
def
on V . Then kuk = kT ukV defines a norm on U.
P ROOF. Easy check. 
3.1.4. Infinite-dimensional examples. Now the norm comes first, the space second.
E XAMPLE 174. For a set X let `∞ (X) = f ∈ F X | sup {| f (x)| : x ∈ X} < ∞ , k f k∞ = supx∈X | f (x)|.


P ROOF. The map k·k∞ : F X → [0, ∞] satisfies the axioms of a norm, suitably extended to in-
clude the value ∞. That the set of vectors of finite norm is a subspace follows from the scaling and
triangle inequalities. 
R EMARK 175. A vector space with basis B can be embedded into `∞ (B) (we’ve basically seen
this).
p
E XAMPLE 176. ` p (N) = a ∈ F N : ∑∞

i=1 |ai | < ∞ with the obvious norm.
In the continuous case we a construction from earlier in the course:
.
p
D EFINITION 177. L p (R) = { f : R → F [measurable] |
R
R | f (x)| dx < ∞} { f | f = 0 a.e.} with
the natural norm.
R EMARK 178. The quotient is essential: for actual functions, can have | f (x)| p dx = 0 without
R

f = 0 exactly. In particular, elements of L p (R) don’t have specific values.

FACT 179. In each equivalence class in L p (R) there is at most one continuous representative.
So part of PDE is about whether an L p solution can be promoted to a continuous functiuon. We
give an example theorem:
2 2
R Let Ω ⊂ RR be a domain, and let f ∈ L (Ω) satisfy ∆ f =
T HEOREM 180 (Elliptic regularity).
λ f distributionally:: for g ∈ Cc (Ω), Ω f ∆g = λ f g. Then there is a smooth function f¯ such that
∞

∆ f = λ f pointwise and such that f = f¯ almost everywhere.

32
 3.1.5. Converges in the norm. While there are many norms on Rn , it turns out that there is
only one notion of convergence.
L EMMA 181. Every norm on Rn is a continuous function.
P ROOF. Let M = maxi kei k. Then
n n
kxk = ∑ xiei ≤ ∑ |xi | kei k ≤ M kxk1 .
i=1 i=1
In particular,
kxk − y ≤ x−y ≤ M x−y 1
.

D EFINITION 182. Call two norms equivalent if there are 0 < m ≤ M such that m kxk ≤ kxk0 ≤
M kxk holds for all x ∈ V .
E XERCISE 183. This is an equivalence relation. The norms are equivalent iff the same se-
quences of vectors satisfy limn→∞ xn = 0.
T HEOREM 184. All norms on Rn (and Cn ) are equivalent.
P ROOF. It is enough to show that they are all equivalent to k·k1 . Accordingly let k·k be any
other norm. Then the Lemma shows that there is M such that
kxk ≤ M kxk1 .
Next, the “sphere”{x | kxk1 = 1} is closed and bounded, hence compact. Accordingly let m =
min {kxk | kxk1 = 1}. Then m > 0 since k0k1 = 0 6= 1. Finally, for any x 6= 0 we have
kxk x
= ≥m
kxk1 kxk1
x
since kxk1 1 = 1. It follows that
m kxk1 ≤ q kxk ≤ M kxk1


3.2. Norms on matrices (Lectures 23-24)

D EFINITION 185. Let U,V be normed spaces. A map T : U → V is called bounded if there is
M ≥ 0 such that kT ukV ≤ M kukU for all u ∈ U. The smallest such M is called the (operator) norm
of T .
R EMARK 186. Motivation: Let U be the space of initial data for an evolution equation (say
wave, or heat). Let V be the space of possible states at time t. Let T be “time evolution”. Then a
key part of PDE is finding norms in which T is bounded as a map from U to V . This shows that
solution exist, and that they are unique.
 
1 1
E XAMPLE 187. The identity map has norm 1. Now consider the matrix A = acting
0 1
on R2 .
33
 (1) As a map from `1 → `1 we have
   
x x
A = |x + y| + |y| ≤ 2 ,
y 1 y 1
with equality if x = 0. Thus kAk1 = 2.
(2) Next,
  2 √
x 2 2 3+ 5 2
A = |x + y| + |y| ≤ x + y2 .
y 2 2
(3) Finally,  
x
A = max {|x + y|, |y|} ≤ 2 max {|x|, |y|}
y ∞
with equality if x = y, Thus kAk∞ = 2.
E XAMPLE 188. Consider Dx : Cc∞ (R) → Cc∞ (R). This is not bounded in any norm (consider
f (x) = e2πikx ).
L EMMA 189. Every map of finite-dimensional spaces is bounded.
P ROOF. Identify U with Rn . Then the k·kU is equivalent with k·k1 , so there is A such that
kuk1 ≤ A kukU . Now the map u 7→ kT ukV is 1-homogenous and satisfies the triangle inequality, so
by the proof of Lemma 181 there is B so that kT ukV ≤ B kuk1 ≤ (AB) kukU . 
L EMMA 190. Let T, S be bounded and composable. Then ST is bounded and kST k ≤ kSk kT k.
P ROOF. For any u ∈ U, kST ukW ≤ kSk kT ukV ≤ kSk kT k kukU . 
P ROPOSITION 191. The operator norm is a norm on Homb (U,V ), the space of bounded maps
U → V.
P ROOF. For any S, T ∈ Homb (U,V ), |α| kT k + kSk is a bound for αT + S. Since the zero map
is bounded it follows that Homb (U,V ) ⊂ Hom(U,V ) is a subspace, and setting α = 1 gives the
triangle inequality. If T 6= 0 then there is u such that T u 6= 0 at which point
kT uk
kT k ≥ > 0.
kuk
Finally, k(αT ) uk = |α| kT uk ≤ |α| kT k kuk so kαT k ≤ |α| kT k. But then
1 1
kT k = αT ≤ kαT k
α |α|
gives the reverse inequality. 

3.3. Example: eigenvalues and the power method (Lecture 25)

Let A be diagonable. Want eigenvalues of A. Raising A to large powers selects the eigenvalue
with largest component.
• Algorithm: multiply by A and renormalize.
• Advantage: if A sparse only need to multiply by A.
• Rate of convergence related to spectral gap.
34
 3.4. Sequences and series of vectors and matrices (Lectures 26-27)
3.4.1. Completeness (Lecture 26).
D EFINITION 192. A metric space is complete if every Cauchy sequence in it converges.
E XAMPLE 193. R. Rn in any norm. Hom(Rn , Rm ) (because isom to Rmn ).
FACT 194. Any metric space has a completion. [note associated universal property and hence
uniqueness]
T HEOREM 195. Let (U, k·kU ) , (V, k·kV ) be normed spaces with V complete. Then Homb (U,V )
is complete with respect to the operator norm.
P ROOF. Let {Tn }∞ n=1 be a Cauchy sequences of linear maps. For fixed u ∈ U, the sequence
{Tn u} is Cauchy: k(Tn u − Tm u)kV ≤ kTn − Tm k kuk. It is therefore convergent – call the limit
T u. This is linear since αTn u + Tn u0 converges to αT u + T u0 while Tn (αu + u0 ) converges to
T (αu + u0 ).
Since |kTn k − kTm k| ≤ kTn − Tm k, the norms themselves are a Cauchy sequences of real num-
bers, in particular a convergent sequence. Now for fixed u, we have kT ukV = limn→∞ kTn ukV . We
have the pointwise bound kTn uk ≤ kTn k kukU . Passing to the limit we find
 
kT ukV ≤ lim kTn k kukU
n→∞
so T is bounded. Finally, given ε let N be such that if m, n ≥ N then kTn − Tm k ≤ ε. Then for any
u ∈ U,
kTn u − Tm uk ≤ kTn − Tm k kukU ≤ ε kukU .
Letting m → ∞ and using the continuity of the norm, we get that if n ≥ N then
kTn u − T uk ≤ ε kukU .
Since u was arbitrary this shows that kTn − T k ≤ ε for n ≥ N and we are done. 
E XAMPLE 196. Let K be a compact space. Then C(K), the space of continuous functions on
K, is complete wrt k·k∞ .
P ROOF. Continuous functions on a compact space are bounded. Let { fn }∞ n=1 ⊂ C(K) be a
∞
Cauchy sequence. Then for fixed x ∈ X, { fn (x)}n=1 ⊂ C is a Cauchy sequence, hence convergent to
some f (x) ∈ C. To see the convergence is in the norm, give ε > 0 let N be such that k fn − fm k∞ ≤ ε
for n, m ≥ N. Then for any x,
| fn (x) − fm (x)| ≤ ε .
Letting m → ∞ we find for all n ≤ N that | fn (x) − f (x)| ≤ ε, that is
k fn − f k∞ ≤ ε.
Finally, we need to show that f is continuous. Given x ∈ X and ε > 0 let N be as above and let
n ≥ N. For any x, the continuity of fn gives a neighbourhood of x where | fn (x) − fn (y)| ≤ ε. Then
| f (x) − f (y)| ≤ | f (x) − fn (x)| + | fn (x) − fn (y)| + | fn (y) − f (y)| ≤ 3ε
in that neighbourhood, so f is continuous at x. 
E XERCISE 197. Generalize this example:
(1) Show that `∞ (X) is complete for any set X.
35
 (2) For a general (topological) space X, show that Cb (X) = C(X) ∩ `∞ (X) is complete with
respect to the supremum norm.
Now fix a complete normed space V .
(3) For a set X write `∞ (X;V ) for the space of bounded functions X → V . Then `∞ (X;V ) is
complete.
(4) Cb (X;V ) = C(X;V ) ∩ `∞ (X;V ) is complete.
3.4.2. Series of vectors and matrices (Lecture 27). Fix a complete normed space V .
D EFINITION 198. Say the series ∑∞ ∞
n=1 vn converges absolutely if ∑n=1 kvn kV < ∞.
P ROPOSITION 199. If ∑∞ ∞ ∞
n=1 vn converges absolutely it converges, and k∑n=1 vn kV ≤ ∑n=1 kvn kV .
P ROOF. Standard. 
E XAMPLE 200 (Exponential series). Let V be a complete normed space and let T ∈ Endb (V ).
def 1 n
Then exp(T ) = ∑∞
n=0 n! T converges absolutely.
Another instance of this phenomenon:
T HEOREM 201 (Weierstrass’s M-test). Let X be a (topological) space, fn : X → V contin-
uous. Suppose that we have Mn such that k fn (x)kV ≤ Mn holds for all x ∈ X. Suppose that
M = ∑∞n=1 Mn < ∞. Then ∑n f n converges uniformly to a continuous function F : X → V .
P ROOF. By the Proposition this amounts to showing that the space Cb (X,V ) (continuous func-
tions X → V with k f (x)kV bounded) is complete with respect to the supremum norm k f k∞ =
sup {k f (x)kV : x ∈ X}. 
We will apply this to power series of matrices.
1
E XAMPLE 202. Let k·k be some operator norm on Mn (R), and let A ∈ Mn (R). For 0 < R < kAk
(any R > 0 if A = 0) and z ∈ C with |z| ≤ R consider the series
∞
∑ znAn .
n=0
We have kAn k≤ kAk (operator norm!) so that kzn An k ≤ (R kAk)n . Since ∑∞
n n
n=0 (R kAk) con-
verges, we see that our series converges and the sum is continuous in z (and in A). Taking the
1
union we get convergence in |z| < kAk . The limit is (Id −zA)−1 (incidentally showing this is in-
vertible).
1
R EMARK 203. In fact, the radius of convergence is ρ(A) .

3.4.3. Vector-valued limits and derivatives. We recall facts about vector-valued limits.
L EMMA 204 (Limit arithmetic). Let U,V,W be normed spaces. Let ui (x) : X → U, αi (x) : X →
F, T (x) : X → Homb (U,V ), S(x) : X → Homb (V,W ). Then, in each case supposing the limits on
the right exist, the limits on the left exist and equality holds:
(1) limx→x0 (α1 (x)u1 (x) + α2 (x)u2 (x)) = (limx→x0 α1 (x)) (limx→x0 u1 (x))+(limx→x0 α2 (x)) (limx→x0 u2 (x)).
(2) limx→x0 T (x)u(x) = (limx→x0 T (x)) (limx→x0 u(x)).
(3) limx→x0 S(x)T (x) = (limx→x0 S(x)) (limx→x0 T (x)).
P ROOF. Same as in R, replacing |·| with k·kV . 
36
 We can also differentiate vector-valued functions (see Math 320 for details)

D EFINITION 205. Let X ⊂ Rn be open. Say that f : X → V is strongly differentiable at x0 if

there is a bounded linear map L : Rn → V such that
k f (x0 + h) − f (x0 ) − LhkV
lim = 0.
h→0 khkRn
In that case we write D f (x0 ) for L.

It is clear that differentiability at x0 implies continuity at x0 .

L EMMA 206 (Derivatives). Let U,V,W be normed spaces. Let ui (x) : X → U, T (x) : X →
Homb (U,V ), S(x) : X → Homb (V,W ) be differentiable at x0 . Then the derivatives on the left exist
and take the following values:
(1) D (u1 + u2 ) (x0 ) = Du1 (x0 ) + Du2 (x0 ).
(2) D (T u) (x0 )(h) = (DT (x0 )(h) · u(x0 )) + T (x0 ) · Du(x0 )(h).
(3) D (ST ) (x0 )(h) = (DS(x0 )(h) · T (x0 )) + (S(x0 ) · DT (x0 )(h)).

P ROOF. Same as in R, replacing |·| with k·kV . 

3.5. The exponential series (Lecture 28)

We apply Weierestrass’s M-test to power series.

T HEOREM 207. Let X be a (topological) space, fn : X → V continuous. Suppose that we have

Mn such that k fn (x)kV ≤ Mn holds for all x ∈ X with M = ∑∞n=1 Mn < ∞. Then ∑n f n converges
uniformly to a continuous function F : X → V and kF(x)kV ≤ M for all x ∈ X.

C OROLLARY 208. Let V be a complete normed space, and let ∑n an zn be a power series with
radius of convergence R. Then for any A ∈ Endb (V ), ∑n an An converges absolutely if kAk < R,
uniformly in {kAk ≤ R − ε}

P ROOF. Let X = V = Endb (V ), fn (A) = an An , so that k fn (A)k ≤ |an | kAkn . For T < R we have
∑n |an |T n < ∞ and hence uniform convergence in {kAk ≤ T }. 

We therefore fix a normed space V , and and plug matrices A ∈ Endb (V ) into power series.
k
E XAMPLE 209. exp(A) = ∑k Ak! converges everywhere.

R EMARK 210. We’ll look at two kinds of matrix-valued series:

(1) Power series with matrix coefficients: f (t) = ∑∞ n
n=0 Ant . Here, t is a scalar and An ∈
Endb (V ).
(2) Plugging in matrices into power series: given f (z) = ∑∞ n ∞ n
n=0 an z set f (A) = ∑n=0 an A .

3.5.1. Basic properties.

L EMMA 211. exp(tA) exp(sA) = exp((t + s)A).

37
 P ROOF. The series converge absolutely, so the product converges in any order. We thus have
! !
∞
(tA)k ∞
(sA)l t k s` Ak+`
exp(tA) exp(sA) = ∑ ∑ = ∑ k!`!
k=0 k! l=0 l! k,l
∞
t k s` Ak+` ∞
Am m!
= ∑ ∑ k!`! = ∑ m! ∑ k!`! t k s`
m=0 k+l=m m=0 k+l=m
∞
Am
= ∑ m! (t + s)m = exp ((t + s)A) .
m=0

Recap: multiplication of absolutely convergent series.

L EMMA 212. Let ∑n un converge absolutely, Then it converges in any reordering and its sum
is unchanged.
P ROOF. Let σ ∈ SN . Given N > 0 let K > max σ −1 ({0, 1, . . . , N}). Then

K N
∑ uσ (k) − ∑ un = ∑ uσ (k)
k=0 n=0 0≤k≤K
σ (k)>N

≤ ∑ uσ (k)
0≤k≤K
σ (k)>N

≤ ∑ kunk .
n>N

Now given ε > 0 let N be large enough such that ∑n>N kun k < ε (exists by absolute convergence).
Then for K large enough as above,
K ∞
∑ uσ (k) − ∑ un ≤ ∑ un + ∑ kunk
k=0 n=0 n>N n>N
≤ 2 ∑ kunk ≤ 2ε .
n>N


P ROPOSITION 213. Let A = ∑n an , B = ∑m bm be convergent series of positive real numbers.
Then ∑n,m an bm converges to AB.

P ROOF. Let S ⊂ N2 be finite so that ∑(n,m)∈S an bm is a partial sum. Then for N large enough
we have S ⊂ {0, 1, . . . , N}2 so that
! !
∑ an bm ≤ ∑ an bm = ∑ an ∑ bm ≤ AB
(m,n)∈S 0≤n,m≤N n≤N m≤N

38
and the series converges. To evaluate the limit it’s enough to note that the subsequence of partial
sums ! !
∑ an bm = ∑ an ∑ bm
0≤n,m≤N n≤N m≤N
evidently converges to AB. 
T HEOREM 214. Let A = ∑∞ ∞
n=0 An , B = ∑m=0 Bm be absolutely convergent (An ∈ Homb (V,W ),
Bm ∈ Homb (U,V )). Then ∑m,n≥0 An Bm converges absolutely to AB.
P ROOF. Since kAn Bm k ≤ kAn k kBm k absolute convergence follows from the Proposition and
the convergence of ∑n kAn k and ∑m kBm k. To evaluate the sum we again take the “square” partial
sums: ! !
∑ An Bm = ∑ An ∑ Bm −−−→ AB .
n≤N m≤N N→∞
0≤n,m≤N


3.5.2. Differentiation and application to constant-coefficient differential equations.

d
C OROLLARY 215. dt exp(tA) = A exp(tA) = exp(tA)A.
exp(hA)−Id h k
P ROOF. At t = 0 we have = A + ∑∞ k+1 and
h k=1 (k+1)! A

∞
hk ∞
|h|k exp(|h| kAk − 1 − kAk |h|
∑ (k + 1)! A k+1
≤ ∑ (k + 1)! kAkk+1 ≤ |h|
−−→ 0 .
h→0
k=1 k=1

In general we have
exp ((t + h)A) − exp(tA) exp(hA) − Id
= exp(tA) −−→ exp(tA)A .
h h h→0

That A exp(tA) = exp(tA)A follows from considering partial sums. 

Consider the system of differential equations
(
d
dt v(t) = Av(t)
v(0) = v0
where A is a bounded map.
P ROPOSITION 216. The system has the unique solution v(t) = exp(At)v0 .
d
P ROOF. We saw dt exp(At)v0 = A (exp(At)v0 ). Conversely, suppose v(t) is any solution. Then
d  −At     
e v(t) = e−At (−A) (v(t)) + e−At (Av(t))
dt
= e−At (−A + A) v(t) = 0 .
It remains to prove: 
L EMMA 217. Let f : [0, 1] → V be differentiable. If f 0 (t) = 0 for all t then f is constant.
39
 P ROOF. Suppose f (t0 ) 6= f (0). Let ϕ ∈ V 0 be a bounded linear functional such that ϕ ( f (t0 ) − f (0)) 6=
0. Then ϕ ◦ f : [0, 1] → Ris differentiable and its derivative is 0:
   
ϕ ( f (t + h)) − ϕ ( f (t)) f (t + h) − f (t) f (t + h) − f (t)
lim = lim ϕ = ϕ lim = ϕ( f 0 (t)) .
h→0 h h→0 h h→0 h
But (ϕ ◦ f ) (t0 ) − (ϕ ◦ f ) (0) = ϕ ( f (t0 ) − f (0)) 6= 0, a contradiction. 
R EMARK 218. If V is finite-dimensional, every linear functional is bounded. If V is infinite-
dimensional the existence of ϕ is a serious fact (“Hahn–Banach Theorem”)
Now consider a linear ODE with constant coefficients:
( n
d n−1 (k)
dt n u(t) = ∑k=0 ak u (t)
u(k) (0) = wk 0 ≤ k ≤ n−1.
We solve this system via the auxilliary vector
 
v(t) = u(t), u0 (t), · · · , u(n−1) (t) .
We then have
dv(t)
= Av
dt
where A is the companion matrix
0 1 0 0 0
 
0 0 1 0 0 

A= ... 
0 0 0 0  .
0 0 0 0 1 
a0 a1 · · · an−2 an−1
(companion to the polynomial xn − ∑n−1 k
k=0 ak x ). It follows that

v(t) = eAt w .
Idea: bring A to Jordan form so easier to take exponential.

3.6. Invertibility and the resolvent (Lecture 29)

Say we have a matrix A we’d like to invert. Idea: write A = D + E where we know to invert D.
Then A = D(I + D−1 E), so if D−1 E < 1 we have

−1 ∞
n
−1
I +D E = ∑ −D−1 E
n=0
and
∞
n
A−1 = ∑ −D−1 E D−1
n=0
(in particular, A is invertible).

3.6.1. Application: Gauss-Seidel and Jacobi iteration.

40
 3.6.2. Application: the resolvent. Let V be a complete normed space. Let T be an operator
on V . Define the resolvent set of T to be the set of z ∈ Cfor which T − z Id has a bounded inverse.
Define the spectrum σ (T ) to be the complement of the resolvent set. This contains the actual
eigenvalues (λ such that Ker (T − λ ) is non-trivial) but also λ where T − λ is not surjective, and
λ where an inverse to T − λ exists but is unbounded).
T HEOREM 219. The resolvent set is open, and the function (“resolvent function”) ρ(T ) →
Endb (V ) given by z 7→ R(z) = (z Id −T )−1 is holomorphic.
P ROOF. Suppose z0 − T has a bounded inverse. We need to invert z − T for z close to z0 .
1
Indeed, if |z − z0 | < (z −T then
k 0 )−1 k
∞
− ∑ (T − z0 )n+1 (z − z0 )n
n=0
converges and furnishes the requisite inverse. It is evidently holomorphic in z in the indicated
ball. 
d2 d2
E XAMPLE 220. Let Ω ⊂ R2 be a bounded domain with nice boundary, ∆ = dx 2 + dy2 the

Laplace operator (say defined on f ∈ C∞ (Ω) vanishing on the boundary). Then ∆ is unbounded,
but its resolvent is nice. For example, R(iε) only has eigenvalues. It follows that the spectrum of
∆ consists of eigenvalues, that is for λ ∈ σ (∆) there is f ∈ L2 (Ω) with ∆ f = λ f (and f ∈ C∞ by
elliptic regularity).
3.7. Holomorphic calculus
D EFINITION 221. Let f (z) = ∑n an zn . Define f (A) = ∑∞ n
n=0 an A .

L EMMA 222. S f (A)S−1 = f (SAS−1 ).

P ROPOSITION 223. ( f ◦ g) (A) = f (g(A)) if it all works.
T HEOREM 224. det(exp(A)) = exp(Tr(A)).

41
 CHAPTER 4

Vignettes

Sketches of applications of linear algebra to group theory.

Key Idea: linearization – use linear tools to study non-linear objects.
4.1. The exponential map and structure theory for GLn (R)
Our goal is to understand the (topologically) closed subgroups of G = GLn (R).
Idea: to a subgroup H assign the logarithms of the elements of H. If H was commutative this
would be a subspace.
D EFINITION 225. Lie(H) = {X ∈ Mn (R) | ∀t : exp(tX) ∈ H}.
R EMARK 226. Clearly this is invariant under scaling. In fact, enough to take small t, and even
just a sequence of t tending to zero (since {t | exp(tX) ∈ H} is a closed subgroup of R).
T HEOREM 227. Lie(H) is a subspace of Mn (R), closed under [X,Y ].
 m
tY m 1
P ROOF. For t ∈ Rand m ∈ Z≥1 , exp tX tX+tY




m exp m = Id + m + O( m2
) −−−→ exp (tX + tY ).
m→∞
Thus If X,Y ∈ Lie(H) then also X +Y ∈ Lie(H). 
• Classify subgroups of G containing A by action on Lie algebra and finding eigenspaces.
4.2. Representation Theory of Groups
E XAMPLE 228 (Representations). (1) Structure of GLn (R): let A act on Mn (R).
(2) M manifold, G acting on M, thus acting on Hk (M) and
H k (M).
(3) Angular momentum: O(3) acting by rotation on L2 R3 .

42