MA110:

Lecture 13

Saurav Bhaumik
Department of Mathematics
IIT Bombay
Spring 2025

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

Inner Product and Norm
Let K := R, the set of real numbers, or K := C, the set of
complex numbers. For a scalar α ∈ K, we denote its conjugate
by α. If α ∈ R, then of course, α = α.
   
x1 y1
 ..   .. 
Consider column vectors x :=  .  and y :=  .  in Kn×1 .
xn yn
∗ 
The conjugate transpose (or the adjoint) x := x 1 · · · x n


of x is a row vector in K1×n . The inner product of x with y

is defined by

⟨x, y⟩ := x∗ y = x 1 y1 + · · · + x n yn .

Note: If K = R, then ⟨x, y⟩ is just the scalar product of

x := (x1 , . . . , xn ) and y := (y1 , . . . , yn ) in Rn .

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

The inner product function ⟨· , ·⟩ : Kn×1 × Kn×1 → K has the
following crucial properties. For x, y, z ∈ Kn×1 and α, β ∈ K,
1. ⟨x, x⟩ ≥ 0 and ⟨x, x⟩ = 0 ⇐⇒ x = 0 (positive definite),
2. ⟨x, αy + βz⟩ = α⟨x, y⟩ + β⟨x, z⟩ (linear in 2nd variable),
3. ⟨y, x⟩ = ⟨x, y⟩ (conjugate symmetric).
From the above three crucial properties, conjugate linearity in
the 1st variable follows: ⟨αx + βy, z⟩ = α⟨x, z⟩ + β⟨y, z⟩.
 T
Let x := x1 · · · xn ∈ Kn×1 . We define the norm of x by

∥x∥ := ⟨x, x⟩1/2 = (|x1 |2 + · · · + |xn |2 )1/2 .

For n = 1, the norm of x ∈ K is the absolute value |x| of x.

Clearly, max{|x1 |, . . . , |xm |} ≤ ∥x∥ ≤ |x1 | + · · · + |xm |.
If x ∈ Kn×1 and ∥x∥ = 1, then we say that x is a unit vector.

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

Theorem
Let x, y ∈ Kn×1 . Then
(i) (Schwarz Inequality) |⟨x, y⟩| ≤ ∥x∥∥y∥.
(ii) (Triangle Inequality) ∥x + y∥ ≤ ∥x∥ + ∥y∥.
 T  T
Proof. Suppose x := x1 · · · xn and y := y1 · · · yn .
(i) If ∥x∥ = 0 or ∥y∥ = 0, then x = 0 or y = 0. Hence we are
done. Now let ∥x∥ = ̸ 0 and ∥y∥ ≠ 0. Then
|x j | |yj | 1  |xj |2 |yj |2 
≤ + for j = 1, . . . , n,
∥x∥ ∥y∥ 2 ∥x∥2 ∥y∥2
since |αβ| = |α| |β| ≤ (|α|2 + |β|2 )/2 for all α, β ∈ K. Hence
n
X ∥x∥∥y∥
|⟨x, y⟩| ≤ |x j | |yj | ≤ (1 + 1) = ∥x∥∥y∥.
j=1
2

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

(ii) Since ⟨x, y⟩ + ⟨y, x⟩ = 2 R ⟨x, y⟩, we see that

∥x + y∥2 = ⟨x + y, x + y⟩ = ∥x∥2 + ∥y∥2 + 2 R ⟨x, y⟩

≤ ∥x∥2 + ∥y∥2 + 2 |⟨x, y⟩|
≤ ∥x∥2 + ∥y∥2 + 2 ∥x∥∥y∥ (by the Schwarz inequality)
= (∥x∥ + ∥y∥)2 .

Thus ∥xx + y ∥ ≤ ∥xx ∥ + ∥yy ∥.

We observe that the norm function ∥ · ∥ : Kn×1 → K satisfies
the following three crucial properties:
(i) ∥x∥ ≥ 0 for all x ∈ Kn×1 and ∥x∥ = 0 ⇐⇒ x = 0,
(ii) ∥αx∥ = |α|∥x∥ for all α ∈ K and x ∈ Kn×1 ,
(iii) ∥x + y∥ ≤ ∥x∥ + ∥y∥ for all x, y ∈ Kn×1 .

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

The properties of the norm function allow us to define the
distance between two vectors in Kn×1 . Let x, y ∈ Kn×1 . Then
the distance between x and y is defined by

d(x, y) := ∥x − y∥.

The distance function d : Kn×1 × Kn×1 → K has the following

analogous properties.
(i) d(x, y) ≥ 0 for all x, y ∈ Kn×1 , d(x, y) = 0 ⇐⇒ x = y,
(ii) d(x, y) = d(y, x) for all x, y ∈ Kn×1 ,
(iii) d(x, y) ≤ d(x, z) + d(z, y) for all x, y, z ∈ Kn×1 .
The inner product defined earlier allows us to say when two
column vectors are perpendicular to each other.

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

Orthogonality
Let x, y ∈ Kn×1 . We say that x and y are orthogonal (to
each other) if ⟨x, y⟩ = 0, and then we write x ⊥ y.
Clearly, x ⊥ x ⇐⇒ ∥x∥ = 0 ⇐⇒ x = 0.
Let E be a subset of Kn×1 , and define
E ⊥ := {y ∈ Kn×1 : y ⊥ x for all x ∈ E }.
It is easy to see that E ⊥ is a subspace of Kn×1 .
Proposition (Pythagoras Theorem)
Let x, y ∈ Kn×1 . If x ⊥ y, then ∥x + y∥2 = ∥x∥2 + ∥y∥2 .

Proof.
∥x + y∥2 = ⟨x + y, x + y⟩ = ⟨x + y, x⟩ + ⟨x + y, y⟩
= ⟨x, x⟩ + ⟨y, x⟩ + ⟨x, y⟩ + ⟨y, y⟩
= ∥x∥2 + 0 + 0 + ∥y∥2 = ∥x∥2 + ∥y∥2 .
Saurav Bhaumik, IIT Bombay MA110: Lecture 13
We now introduce an important concept.
Let y be a nonzero vector in Kn×1 . For x ∈ Kn×1 , define

⟨y, x⟩
Py (x) := y.
⟨y, y⟩

It is called the (perpendicular) projection of the vector x in

the direction of the vector y. Note that Py : Kn×1 → Kn×1 is
a linear map and its image space is one dimensional. Also,
Py (y) = y, so that (Py )2 := Py ◦ Py = Py .

b
x
x − Py (x)
b

b
Py (x)
y

Note that Py (x) is a scalar multiple of y for every x ∈ Kn×1 .

Saurav Bhaumik, IIT Bombay MA110: Lecture 13
An important property of the projection of a vector in the
direction of another (nonzero) vector is the following:
Proposition
Let y ∈ Kn×1 be nonzero. Then for every x ∈ Kn×1 ,


x − Py (x) ⊥ y.

Proof. Let x ∈ Kn×1 . The result follows from

⟨y, x⟩
⟨y, x − Py (x)⟩ = ⟨y, x⟩−⟨y, Py (x)⟩ = ⟨y, x⟩− ⟨y, y⟩ = 0.
⟨y, y⟩

Let E be a subset of Kn×1 . Then E is said to be orthogonal

if any two (distinct) element of E are orthogonal (to each
other), that is, x ⊥ y for all x, y in E with x ̸= y.
For example, E := {0, e1 + e2 , e1 − e2 } is an orthogonal
subset of Kn×1 .
Saurav Bhaumik, IIT Bombay MA110: Lecture 13
Proposition
Let E be a subset of Kn×1 . If E is orthogonal and if 0 ̸∈ E ,
then E is linearly independent.

Proof. Let x1 , . . . , xk be distinct vectors in E , and let

α1 , . . . , αk be scalars such that α1 x1 + · · · + αk xk = 0. Fix
j ∈ {1, . . . , k}. Since ⟨xj , xi ⟩ = 0 for all i ̸= j, we obtain
k
X
0 = ⟨xj , α1 x1 + · · · + αk xk ⟩ = αℓ ⟨xj , xℓ ⟩ = αj ⟨xj , xj ⟩.
ℓ=1

But ⟨xj , xj ⟩ =
̸ 0 since xj ̸= 0. Hence αj = 0.
The converse of the above proposition is not true, that is, a
linear linearly independent subset of Kn×1 need not be
orthogonal. For example, the subset {e1 , e1 + e2 } of Kn×1 is
linearly independent, but not orthogonal.
Saurav Bhaumik, IIT Bombay MA110: Lecture 13
We now address the question: Can we modify a linearly
independent set E to construct an orthogonal set, retaining
the span of the elements in E at each step of the procedure?
Let E be an ordered linearly independent set of column
vectors. Suppose x1 is the first vector in E . Then x1 ̸= 0. Let
y1 := x1 . Let x2 be the second vector in E . If x2 is not
orthogonal to y1 , then it makes sense to subtract from x2 , the
projection of x2 in the direction of y1 , so that
y2 := x2 −Py1 (x2 ) is orthogonal to y1 . Also, in replacing x2 by
y2 , we do not alter the span of {x1 , x2 } since y2 is a linear
combination of x2 and y1 , and x2 is a linear combination of y2
and y1 , where y1 = x1 .
Let x3 be the third vector in E . If x3 is not orthogonal to y1
and y2 , then we may subtract from x3 , the projections of x3 in
the directions of y1 and y2 . Then y3 := x3 − Py1 (x3 ) − Py2 (x3 )
is orthogonal to y1 as well as to y2 . We can see this as follows.
Saurav Bhaumik, IIT Bombay MA110: Lecture 13
 ⟨y1 , y3 ⟩ = ⟨y1 , x3 − Py1 (x3 )⟩ − ⟨y1 , Py2 (x3 )⟩ = 0 − 0 = 0,

and similarly ⟨y3 , y2 ⟩ = 0. We illustrate these vectors in the

following figure.

b
y3 = x3 − Py1 (x3 ) − Py2 (x3 )

x3
b
Py1 (x3 ) b

b Py2 (x3 )
y1 b
b

y2
b

Py1 (x3 ) + Py2 (x3 )

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

This procedure can be continued to yield the famous
Gram-Schmidt Orthogonalization Process (G-S OP)
Let (x1 , . . . , xk ) be an ordered linearly independent set in
Kn×1 . Define y1 := x1 .
Let 1 ≤ j < k. Suppose we have found y1 , . . . , yj in Kn×1
such that the set {y1 , . . . , yj } is orthogonal, and also
span{y1 , . . . , yj } = span{x1 , . . . , xj }. Define

yj+1 := xj+1 − Py1 (xj+1 ) − · · · − Pyj (xj+1 ).

Then span{y1 , . . . , yj+1 } = span{x1 , . . . , xj+1 } since

yj+1 ∈ span{y1 , . . . , yj , xj+1 } = span{x1 , . . . , xj , xj+1 } and
xj+1 ∈ span{y1 , . . . , yj , yj+1 }.
To show that the set {y1 , . . . , yj+1 } is orthogonal, it is enough
to show that yj+1 ∈ {y1 , . . . , yj }⊥ .

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

Let i ∈ {1, . . . , j}. Then

⟨yi , yj+1 ⟩ = ⟨yi , xj+1 − Py1 (xj+1 ) − · · · − Pyj (xj+1 )⟩

= ⟨yi , xj+1 ⟩ − ⟨yi , Py1 (xj+1 )⟩ − · · · − ⟨yi , Pyj (xj+1 )⟩
= ⟨yi , xj+1 ⟩ − ⟨yi , Pyi (xj+1 )⟩ (since yi ⊥ yj , i ̸= j)
= ⟨yi , xj+1 − Pyi (xj+1 )⟩
= 0 (by the important property of the projection).

We remark that since the set {x1 , . . . , xk } is linearly

independent, all vectors y1 , y2 , . . . , yk constructed in the G-S
OP are nonzero: Clearly, y1 = x1 ̸= 0. Also, if yj+1 = 0 for
some j ∈ {1, . . . , k − 1}, then xj+1 would belong to
span{y1 , . . . , yj } = span{x1 , . . . , xj }.
This completes the construction of the G-S OP.

Saurav Bhaumik, IIT Bombay MA110: Lecture 13

An orthogonal set whose elements are unit vectors is called an
orthonormal set.
Any orthogonal set whose elements are nonzero vectors can
always be turned into an orthonormal set by dividing each
element by its own norm.
Thus given an ordered linearly independent set (x1 , . . . , xk ),
we can construct an ordered orthogonal set (y1 , . . . , yk ) by the
G-S OP, and if we let uj := yj /∥yj ∥ for j = 1, . . . , k, then
(u1 , . . . , uk ) is an ordered orthonormal set such that
span{x1 , . . . , xk } = span{y1 , . . . , yk } = span{u1 , . . . , uk }.
Example
For j = 1, . . . , n, let xj := j(e1 + · · · + ej ). Then
E := (x1 , . . . , xn ) is an ordered linearly independent subset of
Kn×1 . We claim that the G-S OP gives yj := jej for
j = 1, . . . , n. Indeed, y1 := x1 = e1 . Also, assuming that
yj = jej , we see that
Saurav Bhaumik, IIT Bombay MA110: Lecture 13
yj+1 = xj+1 − Py1 (xj+1 ) − · · · − Pyj (xj+1 )
= (j + 1)(e1 + · · · + ej+1 ) − (j + 1)e1 − · · · − (j + 1)ej
= (j + 1)ej+1 .

Hence our claim is justified. Since ∥yj ∥ = j for each j, we let

uj := yj /j, so that uj = ej for each j = 1, . . . , n. Clearly,
(u1 , . . . , uk ) is an ordered orthonormal set in Kn×1 .
Let V be a subspace of Kn×1 . An orthonormal basis for V is
a basis for V which is an orthonormal subset of V .
The G-S OP enables us to modify a given basis for a subspace
of Kn×1 to an orthonormal basis for that subspace.
Also, we can expand an orthonormal set in V to a possibly
larger orthonormal set in V as follows.
Saurav Bhaumik, IIT Bombay MA110: Lecture 13
Proposition
Let V be a subspace of Kn×1 , and let u1 , . . . , uk be an
orthonormal set in V . Then there is an orthonormal basis for
V which contains u1 , . . . , uk .

Proof. If span{u1 , . . . , uk } = V , then there is nothing to prove.

Now suppose span{u1 , . . . , uk } = ̸ V . Let dim V = r . Since
the set {u1 , . . . , uk } =
̸ V is linearly independent, there are
yk+1 , . . . , yr in V such that {u1 , . . . , uk , yk+1 , . . . , yr } is a
basis for V . By the G-S OP, we can find uk+1 , . . . , ur in V
such that the set {u1 , . . . , uk , uk+1 , . . . , ur } is orthonormal and
its span is equal to span{u1 , . . . , uk , yk+1 , . . . , yr } = V .

Corollary
Every nonzero vector subspace V has an orthonormal basis.

Proof. If 0 ̸= x1 ∈ V , then extend {x1 /∥x1 ∥} to an o. n. basis.

Saurav Bhaumik, IIT Bombay MA110: Lecture 13