Linear Algebra and Calculus Lecture 14

Eric Sandin Vidal

Spring 2026

Dot Product
In this lecture we will define and work with the notion of length of a vector.

Definition. The inner product (or in this case, dot product) between two vectors u and v
in Rn , written as ⟨u, v⟩ (respectively, u · v), is a real value given by , that is
 
v
   .1 
u1 · · · un  ..  = .
vn

Theorem. Let u, v and w be vectors in Rn , and let c be a scalar. Then

1. u · v =

2. (u + v) · w =

3. (cu) · v =

4. u · u ≥ 0, and u · u = 0 if and only if

Definition. The length (or norm) of v is the nonnegative scalar ∥v∥ defined by
√
∥v∥ = v·v = and ∥v∥2 = v · v.

Remark. This norm is what is called the Euclidean norm, but there are many other norms
and ways to define length. In this course, we use Euclidean norms.

Remark. Notice that ∥cv∥ = |c|∥v∥.

A vector whose length is 1 is called a unit vector. If we divide a nonzero vector v by its
length we obtain a unit vector u because the length of u is (1/∥v∥)∥v∥. The process of creating
u from v is sometimes called normalizing v, and we say that u is in the same direction as v.

Definition. For u and v in Rn , the distance between u and v, written as dist(u, v), is the
length of the vector u − v. That is,

dist(u, v) = .

1
 Consider R2 or R3 and two lines through the origin determined by vectors u and v. The
two lines are geometrically perpendicular if and only if the distance from u to v is the same as
the distance from u to −v. This is equivalent to requiring the squares of the distances to be
the same. Now,

dist(u, −v)2 = ∥u − (−v)∥2

= ∥u + v∥2
= (u + v) · (u + v)
= u · (u + v) + v · (u + v)
=u·u+u·v+v·u+v·v
= ∥u∥2 + ∥v∥2 + 2u · v.

Similarly, interchanging v and −v shows that

dist(u, v)2 = ∥u∥2 + ∥ − v∥2 + 2u · (−v)

= ∥u∥2 + ∥v∥2 − 2u · v.

The two squared distances are equal if and only if 2u · v = −2u · v, which holds if and only if
u · v = 0.

Definition. Two vectors u and v in Rn are orthogonal (to each other) if and only if .

Orthogonal Complements
Definition. If a vector z is orthogonal to every vector in a subspace W of Rn , then z is said to
be orthogonal to W . The set of all vectors z that are orthogonal to W is called the orthogonal
complement of W and is denoted by .

Proposition. Let W be a vector subspace.

1. A vector x is in W ⊥ if and only if x is orthogonal to every vector in a set that spans W .

2. W ⊥ is a subspace of Rn .

Theorem. Let A be an m × n matrix. Then (Col A)⊥ = .

Orthogonal Sets
Definition. A set of vectors {u1 , . . . , up } in Rn is said to be an orthogonal set if each pair of
distinct vectors from the set is orthogonal, that is, if ui · uj = 0 whenever i ̸= j. An orthogonal
set is also called orthonormal set if every vector in the set is a .

Theorem. If S = {u1 , . . . , up } is an orthogonal set of nonzero vectors in Rn , then S is

and hence is for the subspace spanned by S.

Definition. An orthogonal basis for a subspace W of Rn is a basis for W that is also an

orthogonal set.

2
Theorem. Let {u1 , . . . , up } be an orthogonal basis for a subspace W of Rn . For each y ∈ W ,
the weights in the linear combination

y = c1 u1 + · · · + cp up

are given by
cj = , j = 1, . . . , p.

Orthogonal Projections
Let {u1 , . . . , u5 } be an orthogonal basis for R5 and let

y = c1 u1 + · · · + c5 u5 .

Consider the subspace W = Span{u1 , u2 }, and say we want to write y as the sum of a vector
z1 in W and a vector z2 in W ⊥ , that is,

y = c1 u1 + c2 u2 +c3 u3 + c4 u4 + c5 u5
| {z }| {z }
z1 z2

where
z1 = c1 u1 + c2 u2 is in Span{u1 , u2 }
and
z2 = c3 u3 + c4 u4 + c5 u5 is in .
To show that z2 is in W ⊥ , it suffices to show that z2 is orthogonal to the vectors in the basis
{u1 , u2 } for W . Using properties of the inner product, compute

z2 · u1 = (c3 u3 + c4 u4 + c5 u5 ) · u1 = c3 (u3 · u1 ) + c4 (u4 · u1 ) + c5 (u5 · u1 ) =

because u1 is orthogonal to u3 , u4 , and u5 . A similar calculation shows that z2 · u2 = . Thus

z2 is in W ⊥ .

Theorem. (The Orthogonal Decomposition Theorem) Let W be a subspace of Rn . Then each

y in Rn can be written uniquely in the form

y=y
b+z

b is in W and z is in W ⊥ . In fact, if {u1 , . . . , up } is any orthogonal basis of W , then

where y

y
b=

and z = y − y
b.

The vector yb in the previous equation is called the orthogonal projection of y onto W
and is often written as .

Proposition. If y ∈ W = Span{u1 , . . . , up }, then projW y = y.

3
Theorem. (The Best Approximation Theorem) Let W be a subspace of Rn , let y be any vector
in Rn , and let y
b be the orthogonal projection of y onto W . Then y
b is the closest point in W to
y, in the sense that
∥y − yb∥ <
for all v in W distinct from y
b.

Example. Let      
1 0 0
1 1 0
x1 = 
1 ,
 x2 = 
1 ,
 x3 = 
1 .


1 1 1
Then {x1 , x2 , x3 } is clearly linearly independent and thus is a basis for a subspace W of R4 .
To construct an orthogonal basis for W , we use the following process:
Step 1. Let v1 = x1 and W1 = Span{x1 } = Span{v1 }.
Step 2. Let v2 be the vector produced by subtracting from x2 its projection onto the
subspace W1 . That is, let
x2 · v1
v2 = x2 − projW1 x2 = x2 − v1 .
v1 · v1
Since v1 = x1 ,      
0 1 −3/4
1 3 1  1/4
1 − 4 1 =  1/4 .
v2 =      

1 1 1/4
Now v2 is the component of x2 orthogonal to x1 , and {v1 , v2 } is an orthogonal basis for the
subspace W2 spanned by x1 and x2 .
Step 2’ (Optional). If appropriate, scale v2 to simplify later computations. Since v2
has fractional entries, it is convenient to scale it by a factor of 4 and replace {v1 , v2 } by the
orthogonal basis    
1 −3
1
 , v2′ =  1 .
 
v1 = 1  1
1 1
Step 3. Let v3 be the vector produced by subtracting from x3 its projection onto the
subspace W2 . Use the orthogonal basis {v1 , v2′ } to compute this projection onto W2 :

x3 · v1 x3 · v2′ ′
projW2 x3 = v1 + ′ v.
v1 · v1 v2 · v2′ 2

Calculating each projection:

   
1 0
x3 · v1 2 1 2/3
  
v1 =  = ,
v1 · v1 4 1 2/3
1 2/3

4
and  
−3
x3 · v2′ ′ 2  1
v =  .
v2′ · v2′ 2 12  1
1
So,      
0 0 0
0 2/3 −2/3
v3 = x3 − projW2 x3 = 
1 − 2/3 =  1/3 .
    

1 2/3 1/3
Thus, {v1 , v2′ , v3 } is an orthogonal set of nonzero vectors, and since W is three-dimensional,
by the Basis Theorem, {v1 , v2′ , v3 } is an orthogonal basis for W .

Theorem. (The Gram-Schmidt Process) Given a basis {x1 , . . . , xp } for a nonzero subspace W
of Rn , define:
v1 = x1 ,
x2 · v1
v2 = x2 − v1 ,
v1 · v 1
x3 · v 1 x3 · v 2
v3 = x3 − v1 − v2 ,
v1 · v 1 v2 · v 2
..
.
xp · v1 xp · v 2 xp · vp−1
vp = xp − v1 − v2 − · · · − vp−1 .
v1 · v1 v2 · v2 vp−1 · vp−1
Then {v1 , . . . , vp } is an orthogonal basis for W . In addition,

Span{v1 , . . . , vk } = for 1 ≤ k ≤ p.