Chapter 4

Subspaces and Their Properties

除了標註※之簡報外，其餘採用李宏毅教授之投影片教材
Subspaces
(Chapter 4.1)
Subspace
• A vector set V is called a subspace of a vector
space W if it has the following three properties:
• 1. The zero vector 0 belongs to V
• 2. If u and w belong to V, then u+w belongs to V
Closed under (vector) addition
• 3. If u belongs to V, and c is a scalar, then cu
belongs to V
Closed under scalar multiplication

2+3 is linear combination

Examples

Subspace of
R3?
Property 1. 0  W 6(0)  5(0) + 4(0) = 0
Property 2. u, v  W  u+v  W
u = [ u1 u2 u3 ]T, v = [ v1 v2 v3 ]T u+v=[ u1+v1 u2+v1 u3+v1 ]T
6(u1+v1)  5(u2+v2) + 4(u3+v3)
= (6u1  5u2 + 4u3 ) + (6v1  5v2 + 4v3 ) = 0 + 0 = 0
Property 3. u  W  cu  W
6(cu1)  5(cu2) + 4(cu3) = c(6u1  5u2 + 4u3) = c0 = 0
 Examples
V = {cw  c  R} Subspace?

Subspace? u  S1, u  0  u  S1

1  1 1  1
Subspace? 1,  1   S2 but 1   1   S2
     

Rn Subspace? {0} Subspace? zero subspace

Subspace vs. Span
• The span of a vector set is a subspace
Let = , ,⋯, =

Property 1. ∈

Property 2. , ∈ , + ∈V

Property 3. ∈ ,c ∈

Span Subspace
Next lecture
Column Space and Row Space

• Column space of an matrix A is the span of

its columns. It is denoted as Col A.

If matrix A represents a function A: Rn  Rm

Col A is the range of the function
(a subspace in Rm)
• Row space of an matrix A is the span of its
rows. It is denoted as Row A.

(Row A is a subspace in Rn)

Column Space = Range
• The range of a linear transformation is the same as
the column space of its matrix.
Linear Transformation

Standard matrix

 Range of T =
RREF
• Original Matrix A vs. its RREF R
• Columns:
• The relations between the columns are the
same.
• The span of the columns are different.

• Rows:
• The relations between the rows are changed.
• The span of the rows are the same.
 Consistent
Ax = b have a solution (consistent)
b is the linear combination of columns of A
b is in the span of the columns of A
b is in Col A
2 2
u  1  Col A? v  1  Col A?
   
1  3

Solving Ax = u Solving Ax = v

RREF([A u]) = RREF([A v]) =

Null Space
• The null space of an matrix A is the solution
set of Ax=0. It is denoted as Null A.
Null A = { v  Rn : Av = 0 }
The solution set of the homogeneous linear
equations Av = 0.

• Null A is a subspace in Rn

A linear function is Null space only

one-to-one contain 0
Null Space - Example

Find a generating set for the null space of T.

The null space of T is the set of solutions to Ax = 0

1 −1 2 1 −1 0
= =
−1 1 −3 0 0 1

= 1
= 1
0
a generating set for the null space
 Basis
(Chapter 4.2)
 Basis Why nonzero?

• Let V be a nonzero subspace of Rn. A basis B for V is

a linearly independent generating set of V.
{e1, e2, , en} is a basis for Rn.
1. {e1, e2, , en} is independent
2. {e1, e2, , en} generates Rn.

{ } is a basis for R2

…… any two independent

{ }{ }{ }
vectors form a basis for R2
Basis
• The pivot columns of a matrix form a basis for its
column space.

RREF

pivot columns

Col A = Span
Property
• (a) S is contained in Span S Basis is always in
its subspace

• (b) If a finite set S’ is contained in Span S, then Span S’ is

also contained in Span S
Span S
• Because Span S is a subspace
Span S’
• (c) For any vector z, Span S = Span S∪{z} if and only if z
belongs to the Span S
z
z
Theorem
• 1. A basis is the smallest generating set.
• 2. A basis is the largest independent vector set in
the subspace.
• 3. Any two bases for a subspace contain the same
number of vectors.
• The number of vectors in a basis for a nonzero
subspace V is called dimension of V (dim V).
 Every basis of Rn
Theorem 3 has n vectors.

• The number of vectors in a basis for a subspace V is

called the dimension of V, and is denoted dim V
• The dimension of zero subspace is 0

dim R2 =2 dim R3=3

 ●

Example

Find dim V
=3 −5 +6
dim V = 3

Basis? Independent vector set that

generates V
 More from Theorems
Any two bases for a subspace contain the same number
of vectors.
Rm have a basis {e1, e2, , em} All bases have m vectors
dim Rm = m
A basis is the smallest generating set.
A vector set generates Rm must contain at least m vectors.
Because a basis is the smallest generating set
Any other generating set has at least m vectors.

A basis is the largest independent set in the subspace.

Any independent vector set in Rm contain at most m vectors.
Theorem 1
A basis is the smallest generating set.

If there is a generating set S for subspace V,

The size of basis for V is smaller than or equal to S.

Reduction Theorem
There is a basis containing in any generating
set S.
S can be reduced to a basis for V by removing
some vectors.
Theorem 1 – Reduction Theorem
所有的 generating set 心中都有一個 basis

S can be reduced to a basis for V by removing some

vectors.
Suppose S = {u1, u2, , uk} is a generating set of
subspace V
Subspace Let A = [ u1 u2  uk ].

The basis of Col A is

the pivot columns of A Subset of S
 Theorem 1 – Reduction Theorem
所有的 generating set 心中都有一個 basis

Subspace = = = Span
1 2 −1 2 1 2 Smallest generating set
= −1 , −2 , 1 , 2 , 3 , 6
2 4 −3 2 0 3
−3 6 2 0 3 9 Generation set

RREF
=
Theorem 2
A basis is the largest independent set in the subspace.

If the size of basis is k, then you cannot find more than k

independent vectors in the subspace.

Extension Theorem
Given an independent vector set S in the space
S can be extended to a basis by adding more vectors
Theorem 2 – Extension Theorem
Independent set:
我不是一個 basis 就是正在成為一個 basis
There is a subspace V
Given a independent vector set S (elements of S are in V)
If Span S = V, then S is a basis
If Span S ≠ V, find v1 in V, but not in Span S
S = S ⋃ {v1} is still an independent set
If Span S = V, then S is a basis
If Span S ≠ V, find v2 in V, but not in Span S
S = S ⋃ {v2} is still an independent set V
…… You will find the basis in the end.
 Textbook P245
Null B
Theorem 3 Null C

• Any two bases of a subspace V contain the same

number of vectors
Suppose {u1, u2, …, uk} and {w1, w2, …, wp} are two bases of V.
Let A = [u1 u2  uk] and B = [w1 w2  wp].
Since {u1, u2, …, uk} spans V,  ci  Rk s.t. Aci = wi for all i
 A[c1 c2  cp] = [w1 w2  wp]  AC = B
Suppose Cx = 0 for some x  Rp  ACx = Bx = 0
B is independent vector set  x= 0  c1 c2  cp are independent
ci  Rk  p  k
Reversing the roles of the two bases one has k  p  p = k.
 Theorem 4.9
• If V and W are subspaces of Rn with V contained in
W, then dim V dim W
• If dim V = dim W, V=W
• Proof:
BV is a basis of V, V in W, BV in W
BV is an independent set in W
By extension theorem, BV is in the basis of W dim V ≤ dim W
If dim V = dim W =k
BV is a linear independent set in W, with k elements
It is also the span of W
 R3 is the only 3-dim subspace of itself
L
W
The 2-dim subspace
with basis {u,v}
u
The 0-dim
subspace v

The 1-dim subspace

with basis {u}
Concluding Remarks
• 1. A basis is the smallest generation set.
• 2. A basis is the largest independent vector set in
the subspace.
• 3. Any two bases for a subspace contain the same
number of vectors.
• The number of vectors in a basis for a nonzero
subspace V is called dimension of V (dim V).
Confirming that
a set is a Basis
(Chapter 4.2)
Intuitive Way
• Definition: A basis B for V is an independent
generating set of V.

Is C a basis of V ?
Independent? yes
Generating set? difficult

generates V
 Another way
Find a basis for V
• Given a subspace V, assume that we already know that dim
V = k. Suppose S is a subset of V with k vectors
If S is independent S is basis
If S is a generating set S is basis

Dim V = 2 (parametric representation) Is C a basis of V ?

C is a subset of V with 2 vectors

C is a basis of V
Independent? yes
 Assume that dim V = k. Suppose
Another way S is a subset of V with k vectors

If S is independent S is basis

By the extension theorem, we can add more

vector into S to form a basis.
However, S already have k vectors, so it is already
a basis.

If S is a generating set S is basis

By the reduction theorem, we can remove some
vector from S to form a basis.
However, S already have k vectors, so it is already
a basis.
 ●

Example
• Is B a basis of V ?

Independent set in V? yes

Dim V = ? 3 B is a basis of V.
 ●

Example
• Is B a basis of V = Span S ? B is a subset of V with 3
vectors

1 0 0 −2/3
= 0 1 0 1/3
0 0 1 2/3
0 0 0 0
1 0 0
= 0 1 0
0 0 1 B is a basis of V.
0 0 0
Dimension of Basis
(Chapter 4.3)
Col A = Range
• Basis: The pivot columns of A form a basis for Col A.

 Col A = Span

pivot columns pivot columns

• Dimension:
Dim (Col A) = number of pivot columns
= rank A
Rank A (revisit)
Maximum number of Independent Columns

Number of Pivot Columns

Number of Non-zero rows

Number of Basic Variables

Dim (Col A): dimension of column space

Dimension of the range of A

Row A
• Basis: Nonzero rows of RREF(A)

RREF
R=

Row A = Row R a basis of Row R

(The elementary row operations = a basis of Row A
do not change the row space.)
• Dimension: Dim (Row A) = Number of Nonzero rows
= Rank A
Rank A (revisit)
Maximum number of Independent Columns

Number of Pivot Column

Number of Non-zero rows

Number of Basic Variables

Dim (Col A): dimension of column space = Dim (Row A)

Dimension of the range of A = Dim (Col AT)

Rank A = Rank AT

• Proof

Rank A = Dim (Col A)

Rank A = Dim (Row A)

= Dim (Col AT) = Rank AT
 Example 2, P256

3 1 −2 1 5 10 1 0 1
Null A =
1 0 1 0 1
=
01−50 4
−5−2 5 −5 −3 00 0 1−2
−2−1 3 2 −10 00 0 0 0
• Basis:
• Solving Ax = 0
• Each free variable in the parametric representation of
the general solution is multiplied by a vector.
• The vectors form the basis.

=− − −1 −1
+ + =0 =5 −4 5 −4
−5 +4 =0 = 1 + 0
= (free) 0 2
−2 =0 =2 0 1
= (free)
Basis
Null A
• Basis:
• Solving Ax = 0
• Each free variable in the parametric representation of
the general solution is multiplied by a vector.
• The vectors form the basis.
• Dimension:
Dim (Null A) = number of free variables
= Nullity A
= n - Rank A
 Dimension Theorem

Dim (Col A) Dim (Null A) If A is mxn

=Rank A =n - Rank A Dim (Rn) =n
Dim of Range + Dim of Null = Dim of Domain

A A

range
 Four fundamental
subspaces of
Rn Rm
: Rn → Rm

(A): ATy Rn ← Rm ∶ Col(A): Ax

dim r dim r

one-to-one

Rn Rm

N(A): Ax=0 N(AT): ATy=0

dim n-r dim m-r

※
 No Solution

One Solution

Infinite Solutions
+
z

※
 The Meaning of Matrix Transpose

A=

: Rn → Rm : Rm → Rn
x Ax
ATy y

AT
Preservation of dot product in Rn and Rm
※
 Finite vs. Infinite-dimension
Vector Space
• Care has to be taken when dealing with infinite-
dimension vector spaces.
• E.g. Consider the “vector space” containing all
polynomial functions with basis P={1, , , …}
Is it really a vector space?
No!

which does not converge to a polynomial function.

※
Coordinate System
(Chapter 4.4)
Coordinate System
• Each coordinate system is a “viewpoint” for vector
representation.
• The same vector is represented differently in
different coordinate systems.
• Different vectors can have the same
representation in different coordinate systems.
• A vector set B can be considered as a coordinate
system for Rn if:
• 1. The vector set B spans the Rn
• 2. The vector set B is independent
B is a basis of Rn
Coordinate System
• Let vector set B= , ,⋯, be a basis for a subspace
Rn
B is a coordinate system

• For any v in Rn, there are unique scalars , ,⋯, such

that = + + ⋯+

B -coordinate vector of v:

B
(用 B 的觀點來看 )
 New Coordinate System B
E =
1
=
−1
{e1, e2} is a coordinate system 1 1
6 8
8
4 E =
4 4B =
6
−2

8
−2
8 8
=8 +4 =6 + (−2)
4 4
Vector
1 0 2 0
= = = =
0 1 0.5 0.5

2 2
3 3
4
for left
2.5

4
2 +3 =
2.5
Vector
1 0 2 0
= = = =
0 1 0.5 0.5

2 home 2
home
3 3
4
for left
2.5

4
2 +3 =
2.5
 New Coordinate System B
E =
1
=
−1
{e1, e2} is a coordinate system 1 1
6
8
4 E =
4 B=
6
−2

8
−2
8 8
=8 +4 =6 + (−2)
4 4

E= (standard vectors) E
E is Cartesian coordinate system (直角坐標系)
Coordinate System
• A vector set B can be considered as a coordinate
system for Rn if:
• 1. The vector set B spans the Rn
Every vector should have a representation

• 2. The vector set B is independent

Unique representation

B is a basis of Rn
Why Basis?
• Let vector set B= , , ⋯ , be independent.
• Any vector v in Span B can be uniquely represented as a
linear combination of the vectors in B.
• That is, there are unique scalars , , ⋯ , such that =
+ + ⋯+
• Proof:
Unique? = + + ⋯+
= + + ⋯+
− + − + ⋯+ − =0
B is independent a1  b1 = a2  b2 =  = ak  bk = 0
Change Coordinate
(Chapter 4.4)
Coordinate System
• Let vector set B= , ,⋯, be a basis for a subspace
Rn
B is a coordinate system

• For any v in Rn, there are unique scalars , ,⋯, such

that = + + ⋯+

B -coordinate vector of v:

B
(用 B 的觀點來看 )
Other System → Cartesian
1 1 1 3
B= 1 , −1 , 2 B= 6
1 1 2 −2
1 1 1 7
= 3 1 + 6 −1 − 2 2 = −7
1 1 2 5

1 4 7 3
C= 2 , 5 , 8 C= 6
3 6 9 −2
1 4 7 13
=3 2 +6 5 −2 8 = 20
3 6 9 27
Other System → Cartesian
• Let vector set B= be a basis for a
subspace Rn
• Matrix

Given B , how to find v? B

B (matrix-vector product)
Cartesian → Other System
1 1 1 1
= −4 B= 1 , −1 , 2 find [v]B
4 1 1 2

[v]B

1 1 1
= 1 −1 2 B is invertible (?) independent
1 1 2

B B
Cartesian Other System
• Let B = {b1 , b2 , , bn}

B
B

= c1b1 + c2b2 +  + cnbn

Let B= be a basis of Rn. B

(Standard vector)
 ●

Equation of ellipse
Rotate 45◦

2 2 3
3

x2 y2
2
 2 1 ?
3 2
 ●

Equation of ellipse
Use another coordinate system

B ={ }
2 3

What is the equation of the ellipse

in the new coordinate system?

( x) 2 ( y) 2
2
 2 1
3 2
 ●

Equation of ellipse

B ={ }

( x) 2 ( y) 2
2
 2 1
3 2
Linear Function in
Coordinate System
(Chapter 4.5)
Basic Idea
Simple Function
Another output’
Input’
coordinate
system

Cartesian
coordinate Input Output
system Complex Function
Basic Idea
Simple Function
Another output’
Input’
coordinate
system

Cartesian
coordinate Input Output
system Complex Function
Sometimes a function can be
complex ……
• T: reflection about a line L through the origin in R2
Sometimes a function can be
complex ……
• T: reflection about a line L through the origin in R2

special case: L is the horizontal axis

′ ′ ′
= =−
Describing the function in another
coordinate system
• T: reflection about a line L through the origin in R2

In another coordinate system B ….

B=
=−

=
Describing the function in another
coordinate system
• T: reflection about a line L through the origin in R2

In another coordinate system B …. B

Input and output
B matrix of T:
are both in B
=
B=
B B = B
B =
=−
B=
B B = − B
B =−
Flowchart
B

B B
reflection about the
horizontal line

reflection about a line L

 Linear Operator vs. Matrix

B
B B

C
C C
Corresponding matrix of operator T depends on
the coordinate system
※
 B

B B
reflection about the
horizontal line
同一件事情
不同的詮釋

reflection about a line L

Flowchart
B
B B
B coordinate
system

Cartesian
coordinate
system

B B
similar similar
• Example: reflection operator T about the line y = (1/2)x

0.4 0.2 2 −1
= =
−1 −0.2 0.4 1 2
=
2 y = (1/2)x
B
B B
2
=
1

0 y = (1/2)x
= =?
1

1
=
0
B
• Example: reflection operator T about the line y = (1/2)x

0.4 0.2 2 −1
= =
−0.2 0.4 1 2

B
B B
−0.5 y = (1/2)x
=
1

4
=
2

=?

B
Example (P279)

B =?
3 0 1 B B
= 1 1 0
−1 −1 3 B=

1 1 2
= 1 2 1
1 3 1
3 −9 8 is known
−1 3 −3
B=
1 6 1 = B
Example (P279) Determine T

b1 c1 b2 c2 b3 c3

e1 e2 e3 B B B-1c B-1c B-1c

1 2 3
B
b1, b2, b3 as a
coordinate system
{b1, b2, b3} is a
basis of R3

b1 b2 b3 c1 c2 c3
Example (P279) Determine T

B= =
[ ]= B = =

e1 e2 e3 B B B-1c B-1c B-1c

1 2 3
B
b1, b2, b3 as a
coordinate system
{b1, b2, b3} is a
basis of R3

b1 b2 b3 c1 c2 c3
Conclusion

B
B coordinate
B B
system
B=

Cartesian
coordinate
system
= B