Engineering Math

May 30, 2023

2
CONTENTS

1 Introduction 13

2 Algebra and Notation 15

2.1 Sets And Set Notation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
2.2 Well Ordering And Induction . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.3 The Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.4 Polar Form Of Complex Numbers . . . . . . . . . . . . . . . . . . . . . . 21
2.5 Roots Of Complex Numbers . . . . . . . . . . . . . . . . . . . . . . . . . 21
2.6 The Quadratic Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.7 The Complex Exponential . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.8 Dividing Polynomials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.9 The Fundamental Theorem Of Algebra . . . . . . . . . . . . . . . . . . . 26
2.10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3 Integrals, Functions of One Variable 31

3.1 Properties of the Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
3.2 Uniform Convergence of Continuous Functions . . . . . . . . . . . . . . . 42
3.3 Uniform Convergence And The Integral . . . . . . . . . . . . . . . . . . . 44

4 Some Important Improper Integrals 45

4.1 Gamma Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
4.2 Laplace Transforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

I Linear Algebra And Multivariable Calculus 51

5 Fundamentals 53
5.1 Fn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53
5.2 Algebra in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
5.3 Geometric Meaning Of Vector Addition In R3 . . . . . . . . . . . . . . . . 56
5.4 Lines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
5.5 Distance in Rn . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
5.6 Geometric Meaning Of Scalar Multiplication In R3 . . . . . . . . . . . . . 63
5.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64
5.8 Physical Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
5.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3
4 CONTENTS
6 Vector Products 75
6.1 The Dot Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
6.2 The Geometric Significance Of The Dot Product . . . . . . . . . . . . . . . 78
6.2.1 The Angle Between Two Vectors . . . . . . . . . . . . . . . . . . . 78
6.2.2 Work And Projections . . . . . . . . . . . . . . . . . . . . . . . . 79
6.2.3 The Dot Product And Distance In Cn . . . . . . . . . . . . . . . . 82
6.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
6.4 The Cross Product . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85
6.4.1 The Box Product . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
6.5 Proof of the distributive law . . . . . . . . . . . . . . . . . . . . . . . . . 90
6.5.1 Torque . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91
6.5.2 Center Of Mass . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92
6.5.3 Angular Velocity . . . . . . . . . . . . . . . . . . . . . . . . . . . 93
6.6 Vector Identities And Notation . . . . . . . . . . . . . . . . . . . . . . . . 94
6.7 Planes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
6.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

7 Systems Of Equations 105

7.1 Systems Of Equations, Algebraic Procedures . . . . . . . . . . . . . . . . 105
7.1.1 Elementary Operations . . . . . . . . . . . . . . . . . . . . . . . . 105
7.1.2 Gauss Elimination . . . . . . . . . . . . . . . . . . . . . . . . . . 107
7.1.3 Balancing Chemical Reactions . . . . . . . . . . . . . . . . . . . . 117
7.1.4 Dimensionless Variables∗ . . . . . . . . . . . . . . . . . . . . . . 119
7.2 MATLAB And Row Reduced Echelon Form . . . . . . . . . . . . . . . . . 122
7.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

8 Matrices 129
8.1 Addition And Scalar Multiplication Of Matrices . . . . . . . . . . . . . . . 129
8.2 Multiplication of Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 132
8.3 Linear Transformations and Matrices . . . . . . . . . . . . . . . . . . . . . 135
8.4 Multiplication of Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 136
8.4.1 The Transpose . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
8.5 Some Examples of Linear Functions on Rn . . . . . . . . . . . . . . . . . 139
8.5.1 Rotations in R2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139
8.5.2 Projections . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
8.5.3 Rotations About A Particular Vector . . . . . . . . . . . . . . . . . 142
8.6 The Inverse of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
8.6.1 The Identity And Inverses . . . . . . . . . . . . . . . . . . . . . . 144
8.6.2 Finding The Inverse Of A Matrix . . . . . . . . . . . . . . . . . . 145
8.7 MATLAB And Matrix Arithmetic . . . . . . . . . . . . . . . . . . . . . . 151
8.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152

9 Subspaces Spans and Bases 159

9.1 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
9.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 165
CONTENTS 5

10 Eigenvalues and Eigenvectors 173

10.1 Definition of Eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
10.2 An Introduction to Determinants . . . . . . . . . . . . . . . . . . . . . . . 174
10.2.1 Cofactors and 2 × 2 Determinants . . . . . . . . . . . . . . . . . . 174
10.2.2 The Determinant of a Triangular Matrix . . . . . . . . . . . . . . . 177
10.2.3 Properties of Determinants . . . . . . . . . . . . . . . . . . . . . . 178
10.2.4 Finding Determinants Using Row Operations . . . . . . . . . . . . 179
10.3 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181
10.3.1 A Formula For The Inverse . . . . . . . . . . . . . . . . . . . . . . 181
10.3.2 Finding Eigenvalues Using Determinants . . . . . . . . . . . . . . 183

11 Matrices and The Inner Product 185

11.1 Eigenvalues and Eigenvectors . . . . . . . . . . . . . . . . . . . . . . . . . 186
11.2 Using Matlab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
11.3 Distance and Unitary Matrices . . . . . . . . . . . . . . . . . . . . . . . . 190
11.4 Schur’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191
11.5 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196
11.6 Approximations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 198
11.6.1 Fredholm Alternative . . . . . . . . . . . . . . . . . . . . . . . . . 198
11.6.2 Least Squares . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 200
11.6.3 Regression lines . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
11.6.4 Identifying the Closest Point . . . . . . . . . . . . . . . . . . . . . 204
11.6.5 Using MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
11.7 The Singular Value Decomposition∗ . . . . . . . . . . . . . . . . . . . . . 207
11.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

12 Vector Valued Functions 217

12.1 Vector Valued Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . 217
12.2 Vector Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 218
12.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 220
12.4 Continuous Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
12.4.1 Sufficient Conditions For Continuity . . . . . . . . . . . . . . . . . 222
12.5 Limits Of A Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223
12.6 Properties Of Continuous Functions . . . . . . . . . . . . . . . . . . . . . 226
12.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
12.8 Open And Closed Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . 229
12.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

13 Some Fundamentals∗ 235

13.1 Combinations Of Continuous Functions . . . . . . . . . . . . . . . . . . . 235
13.2 The Nested Interval Lemma . . . . . . . . . . . . . . . . . . . . . . . . . 238
13.3 Convergent Sequences, Sequential Compactness . . . . . . . . . . . . . . . 239
13.4 Continuity And The Limit Of A Sequence . . . . . . . . . . . . . . . . . . 241
13.5 The Extreme Value Theorem And Uniform Continuity . . . . . . . . . . . 242
13.6 Convergence of Functions . . . . . . . . . . . . . . . . . . . . . . . . . . 243
13.7 Root Test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
13.8 Convergence of Sums . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
13.9 Connected Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 247
6 CONTENTS
13.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251

14 Vector Valued Functions Of One Variable 253

14.1 Limits Of A Vector Valued Function Of One Real Variable . . . . . . . . . 253
14.2 The Derivative And Integral . . . . . . . . . . . . . . . . . . . . . . . . . 254
14.2.1 Geometric And Physical Significance Of The Derivative . . . . . . 256
14.2.2 Differentiation Rules . . . . . . . . . . . . . . . . . . . . . . . . . 257
14.2.3 Leibniz’s Notation . . . . . . . . . . . . . . . . . . . . . . . . . . 260
14.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
14.4 Line Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
14.4.1 Arc Length And Orientations . . . . . . . . . . . . . . . . . . . . . 262
14.4.2 Line Integrals And Work . . . . . . . . . . . . . . . . . . . . . . . 265
14.4.3 Another Notation For Line Integrals . . . . . . . . . . . . . . . . . 268
14.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
14.6 Independence Of Parametrization∗ . . . . . . . . . . . . . . . . . . . . . . 270
14.6.1 Hard Calculus . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 270
14.6.2 Independence Of Parametrization . . . . . . . . . . . . . . . . . . 272

15 Motion On A Space Curve 275

15.1 Space Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
15.1.1 Some Simple Techniques . . . . . . . . . . . . . . . . . . . . . . . 278
15.2 Geometry Of Space Curves∗ . . . . . . . . . . . . . . . . . . . . . . . . . 279
15.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282

16 Functions Of Many Variables 285

16.1 Review Of Limits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
16.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
16.3 The Directional Derivative And Partial Derivatives . . . . . . . . . . . . . 288
16.3.1 The Directional Derivative . . . . . . . . . . . . . . . . . . . . . . 288
16.3.2 Partial Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . 289
16.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
16.5 Mixed Partial Derivatives . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
16.6 Partial Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . 294
16.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295

17 The Derivative Of A Function Of Many Variables 297

17.1 The Derivative Of Functions Of One Variable . . . . . . . . . . . . . . . . 297
17.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
17.3 The Derivative Of Functions Of Many Variables . . . . . . . . . . . . . . . 300
17.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
17.5 C1 Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 306
17.6 The Chain Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 310
17.6.1 The Chain Rule For Functions Of One Variable . . . . . . . . . . . 310
17.6.2 The Chain Rule For Functions Of Many Variables . . . . . . . . . . 310
17.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
17.7.1 Related Rates Problems . . . . . . . . . . . . . . . . . . . . . . . 316
17.7.2 The Derivative Of The Inverse Function . . . . . . . . . . . . . . . 318
17.7.3 Proof Of The Chain Rule . . . . . . . . . . . . . . . . . . . . . . . 319
17.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 320
CONTENTS 7

17.9 The Gradient . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322

17.10The Gradient And Tangent Planes . . . . . . . . . . . . . . . . . . . . . . 324
17.11Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 326

18 Optimization 329
18.1 Local Extrema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 329
18.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 331
18.3 The Second Derivative Test . . . . . . . . . . . . . . . . . . . . . . . . . . 332
18.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335
18.5 Lagrange Multipliers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 337
18.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 342
18.7 Proof Of The Second Derivative Test∗ . . . . . . . . . . . . . . . . . . . . 345

19 The Riemannn Integral On Rn 349

19.1 Methods For Double Integrals . . . . . . . . . . . . . . . . . . . . . . . . 349
19.1.1 Density And Mass . . . . . . . . . . . . . . . . . . . . . . . . . . 353
19.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 354
19.3 Methods For Triple Integrals . . . . . . . . . . . . . . . . . . . . . . . . . 355
19.3.1 Definition Of The Integral . . . . . . . . . . . . . . . . . . . . . . 355
19.3.2 Iterated Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . 356
19.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 359
19.4.1 Mass And Density . . . . . . . . . . . . . . . . . . . . . . . . . . 360
19.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 362

20 The Integral In Other Coordinates 365

20.1 Polar Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 365
20.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 367
20.3 Cylindrical And Spherical Coordinates . . . . . . . . . . . . . . . . . . . . 368
20.3.1 Volume and Integrals in Cylindrical Coordinates . . . . . . . . . . 369
20.3.2 Volume And Integrals in Spherical Coordinates . . . . . . . . . . . 371
20.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 377
20.5 The General Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . 379
20.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382
20.7 The Moment Of Inertia And Center Of Mass . . . . . . . . . . . . . . . . . 384
20.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 385

21 The Integral on Two Dimensional Surfaces In R3 389

21.1 The Two Dimensional Area In R3 . . . . . . . . . . . . . . . . . . . . . . 389
21.2 Surfaces Of The Form z = f (x, y) . . . . . . . . . . . . . . . . . . . . . . 393
21.3 MATLAB and Graphing Surfaces . . . . . . . . . . . . . . . . . . . . . . 394
21.4 Piecewise Defined Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . 395
21.5 Flux Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395
21.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 396

22 Calculus Of Vector Fields 399

22.1 Divergence And Curl Of A Vector Field . . . . . . . . . . . . . . . . . . . 399
22.1.1 Vector Identities . . . . . . . . . . . . . . . . . . . . . . . . . . . 400
22.1.2 Vector Potentials . . . . . . . . . . . . . . . . . . . . . . . . . . . 401
22.1.3 The Weak Maximum Principle . . . . . . . . . . . . . . . . . . . . 402
8 CONTENTS
22.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 403
22.3 The Divergence Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 404
22.3.1 Coordinate Free Concept Of Divergence . . . . . . . . . . . . . . . 408
22.4 Some Applications Of The Divergence Theorem . . . . . . . . . . . . . . . 409
22.4.1 Hydrostatic Pressure . . . . . . . . . . . . . . . . . . . . . . . . . 409
22.4.2 Archimedes Law Of Buoyancy . . . . . . . . . . . . . . . . . . . . 410
22.4.3 Equations Of Heat And Diffusion . . . . . . . . . . . . . . . . . . 410
22.4.4 Balance Of Mass . . . . . . . . . . . . . . . . . . . . . . . . . . . 412
22.4.5 Balance Of Momentum . . . . . . . . . . . . . . . . . . . . . . . . 412
22.4.6 The Reynolds Transport Formula . . . . . . . . . . . . . . . . . . 418
22.4.7 Frame Indifference . . . . . . . . . . . . . . . . . . . . . . . . . . 420
22.4.8 Bernoulli’s Principle . . . . . . . . . . . . . . . . . . . . . . . . . 422
22.4.9 The Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . . 423
22.4.10 A Negative Observation . . . . . . . . . . . . . . . . . . . . . . . 423
22.4.11 Volumes Of Balls In Rn . . . . . . . . . . . . . . . . . . . . . . . 423
22.4.12 Electrostatics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425
22.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 426

23 Stokes And Green’s Theorems 429

23.1 Green’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 429
23.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 434
23.3 Stoke’s Theorem From Green’s Theorem . . . . . . . . . . . . . . . . . . . 435
23.3.1 The Normal and the Orientation . . . . . . . . . . . . . . . . . . . 438
23.3.2 The Mobeus Band . . . . . . . . . . . . . . . . . . . . . . . . . . 440
23.4 A General Green’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . 441
23.4.1 Conservative Vector Fields . . . . . . . . . . . . . . . . . . . . . . 442
23.4.2 Some Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . 446

24 Moving Coordinate Systems 447

24.1 The Acceleration In Polar Coordinates . . . . . . . . . . . . . . . . . . . . 447
24.2 Planetary Motion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 449
24.2.1 The Equal Area Rule, Kepler’s Second Law . . . . . . . . . . . . . 450
24.2.2 Inverse Square Law, Kepler’s First Law . . . . . . . . . . . . . . . 450
24.2.3 Kepler’s Third Law . . . . . . . . . . . . . . . . . . . . . . . . . . 453
24.3 The Angular Velocity Vector . . . . . . . . . . . . . . . . . . . . . . . . . 454
24.4 Angular Velocity Vector on Earth . . . . . . . . . . . . . . . . . . . . . . . 455
24.5 Coriolis Force and Centripetal Force . . . . . . . . . . . . . . . . . . . . . 457
24.6 Coriolis Force on the Rotating Earth . . . . . . . . . . . . . . . . . . . . . 458
24.7 The Foucault Pendulum∗ . . . . . . . . . . . . . . . . . . . . . . . . . . . 460
24.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 462

25 Curvilinear Coordinates 465

25.1 Basis Vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 465
25.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 468
25.3 Curvilinear Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469
25.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472
25.5 Transformation of Coordinates. . . . . . . . . . . . . . . . . . . . . . . . . 474
25.6 Differentiation and Christoffel Symbols . . . . . . . . . . . . . . . . . . . 475
CONTENTS 9

25.7 Gradients and Divergence . . . . . . . . . . . . . . . . . . . . . . . . . . . 478

25.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 481
25.9 Curl and Cross Products . . . . . . . . . . . . . . . . . . . . . . . . . . . 482

26 Implicit Function Theorem* 485

26.1 More Continuous Partial Derivatives . . . . . . . . . . . . . . . . . . . . . 489
26.2 The Method Of Lagrange Multipliers . . . . . . . . . . . . . . . . . . . . . 490
26.3 The Local Structure Of C1 Mappings∗ . . . . . . . . . . . . . . . . . . . . 492

II Differential Equations 495

27 Determinants 497
27.1 Basic Techniques And Properties . . . . . . . . . . . . . . . . . . . . . . . 497
27.1.1 Cofactors And 2 × 2 Determinants . . . . . . . . . . . . . . . . . . 497
27.1.2 The Determinant Of A Triangular Matrix . . . . . . . . . . . . . . 501
27.1.3 Properties Of Determinants . . . . . . . . . . . . . . . . . . . . . . 502
27.1.4 Finding Determinants Using Row Operations . . . . . . . . . . . . 503
27.2 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 505
27.2.1 A Formula For The Inverse . . . . . . . . . . . . . . . . . . . . . . 505
27.2.2 Finding Eigenvalues Using Determinants . . . . . . . . . . . . . . 509
27.2.3 Cramer’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . 511
27.3 MATLAB And Determinants . . . . . . . . . . . . . . . . . . . . . . . . . 513
27.4 The Cayley Hamilton Theorem∗ . . . . . . . . . . . . . . . . . . . . . . . 513
27.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 516

28 The Mathematical Theory Of Determinants∗ 525

28.0.1 The Function sgn . . . . . . . . . . . . . . . . . . . . . . . . . . . 525
28.1 The Determinant . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
28.1.1 The Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527
28.1.2 Permuting Rows Or Columns . . . . . . . . . . . . . . . . . . . . 528
28.1.3 A Symmetric Definition . . . . . . . . . . . . . . . . . . . . . . . 529
28.1.4 The Alternating Property Of The Determinant . . . . . . . . . . . . 529
28.1.5 Linear Combinations And Determinants . . . . . . . . . . . . . . . 530
28.1.6 The Determinant Of A Product . . . . . . . . . . . . . . . . . . . . 531
28.1.7 Cofactor Expansions . . . . . . . . . . . . . . . . . . . . . . . . . 531
28.1.8 Formula For The Inverse . . . . . . . . . . . . . . . . . . . . . . . 533
28.1.9 Cramer’s Rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535

29 First Order Scalar ODE 537

29.1 First Order Linear Equations . . . . . . . . . . . . . . . . . . . . . . . . . 537
29.2 Bernouli Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 545
29.3 Separable Differential Equations, Stability . . . . . . . . . . . . . . . . . . 546
29.4 Homogeneous Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 554
29.5 Exact Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 555
29.6 The Integrating Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . 557
29.7 The Case Where M, N Are Affine Linear . . . . . . . . . . . . . . . . . . . 562
29.8 Linear and Nonlinear Differential Equations . . . . . . . . . . . . . . . . . 564
29.9 Computer Algebra Methods . . . . . . . . . . . . . . . . . . . . . . . . . 567
10 CONTENTS
29.9.1 MATLAB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 567
29.10Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 569

30 Laplace Transform Methods 581

30.1 Linear O.D.E. With Constant Coefficients . . . . . . . . . . . . . . . . . . 581
30.2 First Order Systems, Constant Coefficients . . . . . . . . . . . . . . . . . . 586
30.2.1 Some Technical Considerations∗ . . . . . . . . . . . . . . . . . . 587
30.2.2 Solving a First Order System . . . . . . . . . . . . . . . . . . . . . 589
30.2.3 Using a Computer Algebra System . . . . . . . . . . . . . . . . . . 591
30.3 Homogeneous Particular and General Solutions . . . . . . . . . . . . . . . 593
30.4 Higher Order Scalar Linear Equations . . . . . . . . . . . . . . . . . . . . 597

31 Numerical Solutions For Systems 601

31.1 A Few Numerical Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 601
31.2 Using MATLAB to Find Solutions . . . . . . . . . . . . . . . . . . . . . . 603
31.3 Stability of Equilibrium Points . . . . . . . . . . . . . . . . . . . . . . . . 604
31.4 Periodic Orbits, Poincare Bendixon Theorem . . . . . . . . . . . . . . . . 608
31.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 609

32 Solutions Near a Regular Singular Point 611

32.1 The Euler Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 611
32.2 Some Simple Observations on Power Series . . . . . . . . . . . . . . . . . 615
32.3 Regular Singular Points . . . . . . . . . . . . . . . . . . . . . . . . . . . . 615
32.4 Abel’s Formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 618
32.5 Finding the Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 619
32.6 The Bessel Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 626
32.6.1 The Case where ν = 0 . . . . . . . . . . . . . . . . . . . . . . . . 626
32.6.2 The Case of ν Not an Integer . . . . . . . . . . . . . . . . . . . . . 627
32.6.3 Case Where ν is an Integer . . . . . . . . . . . . . . . . . . . . . . 628
32.7 Other Properties of Bessel Functions . . . . . . . . . . . . . . . . . . . . . 630
32.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 633

33 Boundary Value Problems, Fourier Series 637

33.1 Boundary Value Problems . . . . . . . . . . . . . . . . . . . . . . . . . . 637
33.2 Eigenvalue Problems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 638
33.3 Fourier Series . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 640
33.4 Mean Square Approximation . . . . . . . . . . . . . . . . . . . . . . . . . 643
33.5 Pointwise Convergence of Fourier Series . . . . . . . . . . . . . . . . . . . 647
33.5.1 Explanation of Pointwise Convergence Theorem . . . . . . . . . . 648
33.5.2 Mean Square Convergence . . . . . . . . . . . . . . . . . . . . . . 652
33.6 Integrating and Differentiating Fourier Series . . . . . . . . . . . . . . . . 654
33.7 Odd and Even Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . 658
33.8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659

34 Some Partial Differential Equations 669

34.1 Laplacian in Orthogonal Curvilinear
Coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 669
34.2 Heat and Wave Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 670
34.2.1 Heat Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 670
CONTENTS 11

34.2.2 The Wave Equation . . . . . . . . . . . . . . . . . . . . . . . . . . 675

34.3 Nonhomogeneous Problems . . . . . . . . . . . . . . . . . . . . . . . . . 678
34.4 Laplace Equation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683
34.4.1 Rectangles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 683
34.4.2 Circular Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 686
34.5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 689

III Fundamentals of Complex Analysis 695

35 Analytic Functions 697
35.1 Cauchy Riemann Equations . . . . . . . . . . . . . . . . . . . . . . . . . . 697
35.2 The Cauchy Riemann Equations . . . . . . . . . . . . . . . . . . . . . . . 698
35.3 Contour Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 700
35.4 Cauchy Integral Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . 705
35.5 Primitives and Cauchy Goursat Theorem . . . . . . . . . . . . . . . . . . . 706
35.6 Functions Differentiable on a Disk, Zeros . . . . . . . . . . . . . . . . . . 709
35.7 Liouville’s Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 717
35.8 Riemann Sphere . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 718
35.9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 719

36 Isolated Singularities and Analytic Functions 727

36.1 Open Mapping Theorem for Complex Valued Functions . . . . . . . . . . . 727
36.2 Functions Analytic on an Annulus . . . . . . . . . . . . . . . . . . . . . . 730
36.3 Isolated Singularities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 734
36.4 Meromorphic Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 736
36.5 The Residue Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 737
36.6 Evaluation of Improper Integrals . . . . . . . . . . . . . . . . . . . . . . . 738
36.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 747

37 Some Fundamental Functions and Transforms 751

37.1 Gamma Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 751
37.2 Laplace Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 752
37.3 Fourier Transform . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 754
37.4 The Inversion of Laplace Transforms . . . . . . . . . . . . . . . . . . . . . 758
37.5 The Bromwich Integral . . . . . . . . . . . . . . . . . . . . . . . . . . . . 759
37.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 764

IV Probability and Statistics 767

38 Probability 769
38.1 Improper Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 769
38.2 Combinations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 771
38.3 The Binomial Theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . 772
38.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 774
38.5 Counting and Basic Probability . . . . . . . . . . . . . . . . . . . . . . . . 774
38.6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 777
38.7 General Considerations Probability . . . . . . . . . . . . . . . . . . . . . . 779
12 CONTENTS
38.8 Moment Generating Functions . . . . . . . . . . . . . . . . . . . . . . . . 787
38.9 Independence and Conditional Probability . . . . . . . . . . . . . . . . . . 791

39 Statistical Tests 797

39.1 The Distribution of nS2 /σ 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 798
39.1.1 Confidence Intervals for Variance . . . . . . . . . . . . . . . . . . 802
39.2 The T and F Distributions . . . . . . . . . . . . . . . . . . . . . . . . . . . 805
39.2.1 The T Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 805
39.2.2 Confidence Intervals for the Mean . . . . . . . . . . . . . . . . . . 807
39.2.3 Testing For Two Different Means . . . . . . . . . . . . . . . . . . 810
39.2.4 The F Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . 812
39.2.5 Confidence Intervals for the Ratio of Two Variances . . . . . . . . 814
39.3 Maximum Likelihood Estimates . . . . . . . . . . . . . . . . . . . . . . . 816
39.4 Quadratic Forms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 818
39.5 Linear Regression . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 825
39.6 Goodness of Fit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 832
39.7 Contingency Tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 839

A The Theory Of The Riemannn Integral∗ 843

A.1 An Important Warning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843
A.2 Basic Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 843
A.3 Basic Properties . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 846
A.4 Which Functions Are Integrable? . . . . . . . . . . . . . . . . . . . . . . . 849
A.5 Iterated Integrals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 857
A.6 The Change Of Variables Formula . . . . . . . . . . . . . . . . . . . . . . 861
A.7 Some Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 869

B A Rigid Body Rotating About a Point 871

C Lagrangian Mechanics 877

C.1 The Spinning Top and the Euler Angles . . . . . . . . . . . . . . . . . . . 880
Copyright © 2018, You are welcome to use this, including copying it for use in classes
or referring to it on line but not to publish it for money.
Chapter 1

Introduction

This book is on multivariable calculus. It is to follow a calculus course which is devoted

primarily to calculus of functions of one variable. It is not an advanced calculus course
but is intended to serve as an elementary presentation of calculus of many variables. As
part of this presentation, there is a short review of topics which are often omitted from
single variable calculus courses. There is also a short treatment of linear algebra because
multivariable calculus is dependent on linear algebra. For example, the derivative is a linear
transformation. The determinant is used in change of variables formulas. You really
don’t understand Lagrange multipliers without some linear algebra concepts, the second
derivative test is most easily remembered in terms of eigenvalues and one could go on
like this. Indeed, multivariable calculus is really all about using linear algebra concepts to
approximate nonlinear analysis ideas so if you don’t have any concepts from linear algebra
understood, then multivariable calculus can seem a little mysterious. The first part of the
book, consisting of multivariable calculus and linear algebra will fit in one semester. The
second part will fit in a second semester. I have left out the fluff which usually clogs our
differential equations classes and replaced it with MATLAB. I taught differential equations
way too often to pretend that it has anything sufficiently significant to justify the time spent
on it. I think that one of the major difficulties people have with this subject is more about
factoring polynomials than anything of mathematical significance.
While the book does contain a fairly complete presentation of linear algebra, it is not
necessary to read all of this in order to do the multivariable calculus portion of the book.
It suffices to consider that which includes matrices and linear transformations and eigen-
values. The first part of my book on calculus of one and many variables has the relevant
material on R so if one has read it, there will be no need to re read what is repeated in this
book. Those interested in linear algebra would do better to read a linear algebra book like
those on my web page.
I have called the book Engineering math because it is not limited to multivariable cal-
culus and linear algebra. It also has the necessary material on differential equations and a
part devoted to basic complex analysis. It concludes with a short introduction to probability
and statistics. I have tried to emphasize those aspects of complex analysis which are in my
opinion of most use. It is an elementary treatment of this subject, not the type of thing in a
graduate text.
The material on probability and statistics is intended to explain some of the difficult
topics in statistics. This is a subject which has been automated to a remarkable extent

13
14 CHAPTER 1. INTRODUCTION

and doing the applications amounts to using the right software at this point. I think that
understanding why certain things are true needs to be presented and this is the emphasis in
this book. However, really difficult mathematical issues are not included, especially those
things which really need Lebesgue integration and measure theory to understand. Also,
I am emphasizing moment generating functions rather than characteristic functions. I am
emphasizing confidence intervals more than hypothesis testing. It seems to me that this
is easier to understand with less jargon and is sufficient to draw conclusions. It is not a
complete book on mathematical statistics, just an introduction to some of the important
ideas. Not everything is proved in this section because some of the proofs are too long.
However, I am making every effort to at least make it plausible.
Chapter 2

Algebra and Notation

The reader should be familiar with most of the topics in this chapter. However, it is often
the case that set notation is not familiar and so a short discussion of this is included first.
Complex numbers are then considered in somewhat more detail. Many of the applications
of linear algebra and differential equations require the use of complex numbers, so this is
the reason for this introduction.

2.1 Sets And Set Notation

A set is just a collection of things called elements. Often these are also referred to as points
in calculus. For example {1, 2, 3, 8} would be a set consisting of the elements 1,2,3, and
8. To indicate that 3 is an element of {1, 2, 3, 8} , it is customary to write 3 ∈ {1, 2, 3, 8} .
9∈/ {1, 2, 3, 8} means 9 is not an element of {1, 2, 3, 8} . Sometimes a rule specifies a set.
For example you could specify a set as all integers larger than 2. This would be written as
S = {x ∈ Z : x > 2} . This notation says: the set of all integers, x, such that x > 2.
If A and B are sets with the property that every element of A is an element of B, then
A is a subset of B. For example, {1, 2, 3, 8} is a subset of {1, 2, 3, 4, 5, 8} , in symbols,
{1, 2, 3, 8} ⊆ {1, 2, 3, 4, 5, 8} . It is sometimes said that “A is contained in B” or even “B
contains A”. The same statement about the two sets may also be written as {1, 2, 3, 4, 5, 8} ⊇
{1, 2, 3, 8}.
The union of two sets is the set consisting of everything which is an element of at least
one of the sets, A or B. As an example of the union of two sets {1, 2, 3, 8} ∪ {3, 4, 7, 8} =
{1, 2, 3, 4, 7, 8} because these numbers are those which are in at least one of the two sets.
In general
A ∪ B ≡ {x : x ∈ A or x ∈ B} .
Be sure you understand that something which is in both A and B is in the union. It is not an
exclusive or.
The intersection of two sets, A and B consists of everything which is in both of the sets.
Thus {1, 2, 3, 8} ∩ {3, 4, 7, 8} = {3, 8} because 3 and 8 are those elements the two sets have
in common. In general,
A ∩ B ≡ {x : x ∈ A and x ∈ B} .
The symbol [a, b] where a and b are real numbers, denotes the set of real numbers x,
such that a ≤ x ≤ b and [a, b) denotes the set of real numbers such that a ≤ x < b. (a, b)

15
16 CHAPTER 2. ALGEBRA AND NOTATION

consists of the set of real numbers x such that a < x < b and (a, b] indicates the set of
numbers x such that a < x ≤ b. [a, ∞) means the set of all numbers x such that x ≥ a and
(−∞, a] means the set of all real numbers which are less than or equal to a. These sorts
of sets of real numbers are called intervals. The two points a and b are called endpoints
of the interval. Other intervals such as (−∞, b) are defined by analogy to what was just
explained. In general, the curved parenthesis indicates the end point it sits next to is not
included while the square parenthesis indicates this end point is included. The reason that
there will always be a curved parenthesis next to ∞ or −∞ is that these are not real numbers.
Therefore, they cannot be included in any set of real numbers.
A special set which needs to be given a name is the empty set also called the null set,
denoted by 0. / Thus 0/ is defined as the set which has no elements in it. Mathematicians
like to say the empty set is a subset of every set. The reason they say this is that if it were
not so, there would have to exist a set A, such that 0/ has something in it which is not in A.
However, 0/ has nothing in it and so the least intellectual discomfort is achieved by saying
0/ ⊆ A.
If A and B are two sets, A \ B denotes the set of things which are in A but not in B. Thus

A \ B ≡ {x ∈ A : x ∈
/ B} .

Set notation is used whenever convenient.

To illustrate the use of this notation relative to intervals consider three examples of
inequalities. Their solutions will be written in the notation just described.

Example 2.1.1 Solve the inequality 2x + 4 ≤ x − 8

x ≤ −12 is the answer. This is written in terms of an interval as (−∞, −12].

Example 2.1.2 Solve the inequality (x + 1) (2x − 3) ≥ 0.

3
The solution is x ≤ −1 or x ≥ . In terms of set notation this is denoted by (−∞, −1] ∪
2
3
[ , ∞).
2
Example 2.1.3 Solve the inequality x (x + 2) ≥ −4.

This is true for any value of x. It is written as R or (−∞, ∞) .

2.2 Well Ordering And Induction

Mathematical induction and well ordering are two extremely important principles in math.
They are often used to prove significant things which would be hard to prove otherwise.

Definition 2.2.1 A set is well ordered if every nonempty subset S, contains a smallest ele-
ment z having the property that z ≤ x for all x ∈ S.

Axiom 2.2.2 Any set of integers larger than a given number is well ordered.
2.2. WELL ORDERING AND INDUCTION 17

In particular, the natural numbers defined as

N ≡ {1, 2, · · · }

is well ordered.
The above axiom implies the principle of mathematical induction. The symbol Z de-
notes the set of all integers. Note that if a is an integer, then there are no integers between
a and a + 1.

Theorem 2.2.3 (Mathematical induction) A set S ⊆ Z, having the property that a ∈ S and
n + 1 ∈ S whenever n ∈ S contains all integers x ∈ Z such that x ≥ a.

Proof: Let T consist of all integers larger than or equal to a which are not in S. The
theorem will be proved if T = 0. / If T ̸= 0/ then by the well ordering principle, there would
have to exist a smallest element of T, denoted as b. It must be the case that b > a since by
definition, a ∈/ T. Thus b ≥ a + 1, and so b − 1 ≥ a and b − 1 ∈ / S because if b − 1 ∈ S, then
b − 1 + 1 = b ∈ S by the assumed property of S. Therefore, b − 1 ∈ T which contradicts
the choice of b as the smallest element of T. (b − 1 is smaller.) Since a contradiction is
obtained by assuming T ̸= 0, / it must be the case that T = 0/ and this says that every integer
at least as large as a is also in S. ■
Mathematical induction is a very useful device for proving theorems about the integers.

n (n + 1) (2n + 1)
Example 2.2.4 Prove by induction that ∑nk=1 k2 = .
6
▶
By inspection, if n = 1 then the formula is true. The sum yields 1 and so does the
formula on the right. Suppose this formula is valid for some n ≥ 1 where n is an integer.
Then
n+1 n
n (n + 1) (2n + 1)
∑ k2 = ∑ k2 + (n + 1)2 = 6
+ (n + 1)2 .
k=1 k=1
The step going from the first to the second line is based on the assumption that the formula
is true for n. This is called the induction hypothesis. Now simplify the expression in the
second line,
n (n + 1) (2n + 1)
+ (n + 1)2 .
6
This equals  
n (2n + 1)
(n + 1) + (n + 1)
6
and
n (2n + 1) 6 (n + 1) + 2n2 + n (n + 2) (2n + 3)
+ (n + 1) = =
6 6 6
Therefore,
n+1
(n + 1) (n + 2) (2n + 3) (n + 1) ((n + 1) + 1) (2 (n + 1) + 1)
∑ k2 = 6
=
6
,
k=1

showing the formula holds for n + 1 whenever it holds for n. This proves the formula by
mathematical induction.
18 CHAPTER 2. ALGEBRA AND NOTATION

1 3 2n − 1 1
Example 2.2.5 Show that for all n ∈ N, · ··· <√ .
2 4 2n 2n + 1
1 1
If n = 1 this reduces to the statement that < √ which is obviously true. Suppose
2 3
then that the inequality holds for n. Then
√
1 3 2n − 1 2n + 1 1 2n + 1 2n + 1
· ··· · <√ = .
2 4 2n 2n + 2 2n + 1 2n + 2 2n + 2
1
The theorem will be proved if this last expression is less than √ . This happens if
2n + 3
and only if
 2
1 1 2n + 1
√ = >
2n + 3 2n + 3 (2n + 2)2
which occurs if and only if (2n + 2)2 > (2n + 3) (2n + 1) and this is clearly true which may
be seen from expanding both sides. This proves the inequality.
Lets review the process just used. If S is the set of integers at least as large as 1 for which
the formula holds, the first step was to show 1 ∈ S and then that whenever n ∈ S, it follows
n + 1 ∈ S. Therefore, by the principle of mathematical induction, S contains [1, ∞) ∩ Z,
all positive integers. In doing an inductive proof of this sort, the set S is normally not
mentioned. One just verifies the steps above. First show the thing is true for some a ∈ Z
and then verify that whenever it is true for m it follows it is also true for m + 1. When this
has been done, the theorem has been proved for all m ≥ a.

2.3 The Complex Numbers

Recall that a real number is a point on the real number line. Just as a real number should be
considered as a point on the line, a complex number is considered a point in the plane which
can be identified in the usual way using the Cartesian coordinates of the point. Thus (a, b)
identifies a point whose x coordinate is a and whose y coordinate is b. In dealing with com-
plex numbers, such a point is written as a + ib. For example, in the following picture, I have
graphed the point 3 + 2i. You see it corresponds to the point in the plane whose coordinates
are (3, 2) .
Multiplication and addition are defined in the most obvious way sub-
2
•3 + 2i ject to the convention that i = −1. Thus,
(a + ib) + (c + id) = (a + c) + i (b + d)

and

(a + ib) (c + id) = ac + iad + ibc + i2 bd

= (ac − bd) + i (bc + ad) .

Every non zero complex number a + ib, with a2 + b2 ̸= 0, has a unique multiplicative
inverse.
1 a − ib a b
= 2 2
= 2 2
−i 2 .
a + ib a + b a +b a + b2
You should prove the following theorem.
2.3. THE COMPLEX NUMBERS 19

Theorem 2.3.1 The complex numbers with multiplication and addition defined as above
form a field satisfying all the field axioms. These are the following list of properties.

1. x + y = y + x, (commutative law for addition)

2. x + 0 = x, (additive identity).
3. For each x ∈ R, there exists −x ∈ R such that x + (−x) = 0, (existence of additive
inverse).
4. (x + y) + z = x + (y + z) , (associative law for addition).
5. xy = yx, (commutative law for multiplication). You could write this as x × y = y × x.
6. (xy) z = x (yz) , (associative law for multiplication).
7. 1x = x, (multiplicative identity).
8. For each x ̸= 0, there exists x−1 such that xx−1 = 1.(existence of multiplicative in-
verse).
9. x (y + z) = xy + xz.(distributive law).

Something which satisfies these axioms is called a field. In this book, the field of most
interest will be the field of complex numbers or the field of real numbers. You have seen
in earlier courses that the set of real numbers with the usual operations also satisfies the
above axioms. The field of complex numbers is denoted as C and the field of real numbers
is denoted as R. An important construction regarding complex numbers is the complex
conjugate denoted by a horizontal line above the number. It is defined as follows.

a + ib ≡ a − ib.

What it does is reflect a given complex number across the x axis. Algebraically, the follow-
ing formula is easy to obtain.


a + ib (a + ib) = (a − ib) (a + ib)
= a2 + b2 − i (ab − ab) = a2 + b2 .

Observation 2.3.2 The conjugate of a sum of complex numbers equals the sum of the com-
plex conjugates and the conjugate of a product of complex numbers equals the product of
the conjugates. To illustrate, consider the claim about the product.

(a + ib) (c + id) = (ac − bd) + i (bc + ad) = (ac − bd) − i (bc + ad)



a + ib c + id = (a − ib) (c − id) = (ac − bd) − i (bc + ad)

Showing the claim works for a sum is left for you. Of course this means the conclusion
holds for any finite product or finite sum. Indeed, for zk a complex number, the associative
law of multiplication above gives

z1 · · · zn = (z1 · · · zn−1 ) (zn ) = (z1 · · · zn−1 ) (zn )

Now by induction, the first product in the above can be split up into the product of the
conjugates. Similar observations hold for sums.
20 CHAPTER 2. ALGEBRA AND NOTATION

Definition 2.3.3 Define the absolute value of a complex number as follows.

p
|a + ib| ≡ a2 + b2 .

Thus, denoting by z the complex number z = a + ib,

|z| = (zz)1/2 .

Also from the definition, if z = x + iy and w = u + iv are two complex numbers, then
|zw| = |z| |w| . You should verify this. ▶

Notation 2.3.4 Recall the following notation.

n
∑ a j ≡ a1 + · · · + an
j=1

There is also a notation which is used to denote a product.

n
∏ a j ≡ a1 a2 · · · an
j=1

The triangle inequality holds for the absolute value for complex numbers just as it does
for the ordinary absolute value.

Proposition 2.3.5 Let z, w be complex numbers. Then the triangle inequality holds.

|z + w| ≤ |z| + |w| , ||z| − |w|| ≤ |z − w| .

Proof: Let z = x + iy and w = u + iv. First note that

zw = (x + iy) (u − iv) = xu + yv + i (yu − xv)

and so |xu + yv| ≤ |zw| = |z| |w| .

|z + w|2 = (x + u + i (y + v)) (x + u − i (y + v))

= (x + u)2 + (y + v)2 = x2 + u2 + 2xu + 2yv + y2 + v2

≤ |z|2 + |w|2 + 2 |z| |w| = (|z| + |w|)2 ,
so this shows the first version of the triangle inequality. To get the second,

z = z − w + w, w = w − z + z

and so by the first form of the inequality

|z| ≤ |z − w| + |w| , |w| ≤ |z − w| + |z|

and so both |z| − |w| and |w| − |z| are no larger than |z − w| and this proves the second
version because ||z| − |w|| is one of |z| − |w| or |w| − |z|. ■
With this definition, it is important to note the following. Be sure to verify this. It is not
too hard but you need to do it.
2.4. POLAR FORM OF COMPLEX NUMBERS 21
q
Remark 2.3.6 : Let z = a + ib and w = c + id. Then |z − w| = (a − c)2 + (b − d)2 . Thus
the distance between the point in the plane determined by the ordered pair (a, b) and the
ordered pair (c, d) equals |z − w| where z and w are as just described.

For example, consider the distance between (2, 5) and (1, 8) . From the distance formula
q √
this distance equals (2 − 1)2 + (5 − 8)2 = 10. On the other hand, letting z = 2 + i5 and
√
w = 1 + i8, z − w = 1 − i3 and so (z − w) (z − w) = (1 − i3) (1 + i3) = 10 so |z − w| = 10,
the same thing obtained with the distance formula.

2.4 Polar Form Of Complex Numbers

Complex numbers, are often written in the so called polar form which is described next.
Suppose z = x + iy is a complex number. Then
!
p x y
x + iy = x2 + y2 p +ip .
x2 + y2 x 2 + y2

Now note that !2 !2

x y
p + p =1
x2 + y2 x 2 + y2
and so !
x y
p ,p
x2 + y2 x2 + y2
is a point on the unit circle. Therefore, there exists a unique angle θ ∈ [0, 2π) such that
x y
cos θ = p , sin θ = p .
x 2 + y2 x 2 + y2

the complex number is then r (cos θ + i sin θ ) where θ is this angle just
The polar form of p
described and r = x2 + y2 ≡ |z|.

p
r= x2 + y2 r x + iy = r(cos(θ ) + i sin(θ ))
θ

2.5 Roots Of Complex Numbers

A fundamental identity is the formula of De Moivre which follows.

Theorem 2.5.1 Let r > 0 be given. Then if n is a positive integer,

[r (cost + i sint)]n = rn (cos nt + i sin nt) .

22 CHAPTER 2. ALGEBRA AND NOTATION

Proof: It is clear the formula holds if n = 1. Suppose it is true for n.

[r (cost + i sint)]n+1 = [r (cost + i sint)]n [r (cost + i sint)]
which by induction equals
= rn+1 (cos nt + i sin nt) (cost + i sint)
= rn+1 ((cos nt cost − sin nt sint) + i (sin nt cost + cos nt sint))
= rn+1 (cos (n + 1)t + i sin (n + 1)t)
by the formulas for the cosine and sine of the sum of two angles. ■
Corollary 2.5.2 Let z be a non zero complex number. Then there are always exactly k kth
roots of z in C.
Proof: Let z = x + iy and let z = |z| (cost + i sint) be the polar form of the complex
number. By De Moivre’s theorem, a complex number
r (cos α + i sin α) ,
is a kth root of z if and only if
rk (cos kα + i sin kα) = |z| (cost + i sint) .

This requires rk = |z| and so r = |z|1/k and also both cos (kα) = cost and sin (kα) = sint.
This can only happen if
kα = t + 2lπ
for l an integer. Thus
t + 2lπ
α= ,l ∈ Z
k
and so the kth roots of z are of the form
    
1/k t + 2lπ t + 2lπ
|z| cos + i sin , l ∈ Z.
k k
Since the cosine and sine are periodic of period 2π, there are exactly k distinct numbers
which result from this formula. ■
Example 2.5.3 Find the three cube roots of i.




First note that i = 1 cos π2 + i sin π2 . Using the formula in the proof of the above
corollary, the cube roots of i are
    
(π/2) + 2lπ (π/2) + 2lπ
1 cos + i sin
3 3
where l = 0, 1, 2. Therefore, the roots are
       
π  π  5 5 3 3
cos + i sin , cos π + i sin π , cos π + i sin π .
6 6 6 6 2 2
√   √  
3 1 − 3 1
Thus the cube roots of i are +i , +i , and −i.
2 2 2 2
The ability to find kth roots can also be used to factor some polynomials.
2.6. THE QUADRATIC FORMULA 23

Example 2.5.4 Factor the polynomial x3 − 27.

First find the cube roots of 27. By the above procedure using De Moivre’s theorem,
√ ! √ !
−1 3 −1 3
these cube roots are 3, 3 +i , and 3 −i . Therefore, x3 − 27 =
2 2 2 2
√ !! √ !!
−1 3 −1 3
(x − 3) x − 3 +i x−3 −i .
2 2 2 2
  √    √ 
Note also x − 3 −1
2 + i 2
3
x − 3 −1
2 − i 2
3
= x2 + 3x + 9 and so

x3 − 27 = (x − 3) x2 + 3x + 9

where the quadratic polynomial x2 +3x +9 cannot be factored without using complex num-
bers.
3
Note that even though√ the polynomial√ x − 27 has all real coefficients, it has some
−1 3 −1 3
complex zeros, +i and −i . These zeros are complex conjugates of each
2 2 2 2
other. It is always this way. You should show this is the case. To see how to do this, see
Problems 17 and 18 below.
Another fact for your information is the fundamental theorem of algebra. This theorem
says that any polynomial of degree at least 1 having any complex coefficients always has
a root in C. This is sometimes referred to by saying C is algebraically complete. Gauss is
usually credited with giving a proof of this theorem in 1797 but many others worked on it
and the first completely correct proof was due to Argand in 1806. For more on this theo-
rem, you can google fundamental theorem of algebra and look at the interesting Wikipedia
article on it. Proofs of this theorem usually involve the use of techniques from calculus
even though it is really a result in algebra. A proof and plausibility explanation is given
later.

2.6 The Quadratic Formula

The quadratic formula √
−b ± b2 − 4ac
x=
2a
gives the solutions x to
ax2 + bx + c = 0
where a, b, c are real numbers. It holds even if b2 − 4ac < 0. This is easy to show from the
above. There are exactly two square roots to this number b2 − 4ac from the above methods
using De Moivre’s theorem. These roots are of the form
p  π   π  p
4ac − b2 cos + i sin = i 4ac − b2
2 2
and     
p 3π 3π p
4ac − b2 cos + i sin = −i 4ac − b2
2 2
24 CHAPTER 2. ALGEBRA AND NOTATION

Thus the solutions, according to the quadratic formula are still given correctly by the above
formula.
Do these solutions predicted by the quadratic formula continue to solve the quadratic
equation? Yes, they do. You only need to observe that when you square a square root of a
complex number z, you recover z. Thus
√ !2 √ !
−b + b2 − 4ac −b + b2 − 4ac
a +b +c
2a 2a
√ !
−b + b2 − 4ac
 
1 2 1 1 p 2
=a b − c − 2 b b − 4ac + b +c
2a2 a 2a 2a
  1  p
1  p 2 
= − b b − 4ac + 2ac − b2 + b b2 − 4ac − b2 + c = 0
2a 2a
√
2
Similar reasoning shows directly that −b− 2ab −4ac also solves the quadratic equation.
What if the coefficients of the quadratic equation are actually complex numbers? Does
the formula hold even in this case? The answer is yes. This is a hint on how to do Problem
27 below, a special case of the fundamental theorem of algebra, and an ingredient in the
proof of some versions of this theorem.

Example 2.6.1 Find the solutions to x2 − 2ix − 5 = 0.

Formally, from the quadratic formula, these solutions are

√
2i ± −4 + 20 2i ± 4
x= = = i ± 2.
2 2
Now you can check that these really do solve the equation. In general, this will be the case.
See Problem 27 below.

2.7 The Complex Exponential

It was shown above that every complex number can be written in the form

r (cos θ + i sin θ )

where r ≥ 0. Laying aside the zero complex number, this shows that every non zero com-
plex number is of the form eα (cos β + i sin β ) . We write this in the form eα+iβ . Having
done so, does it follow that the expression preserves the most important property of the
function t → e(α+iβ )t for t real, that
 ′
e(α+iβ )t = (α + iβ ) e(α+iβ )t ?

By the definition just given which does not contradict the usual definition in case β = 0 and
the usual rules of differentiation in calculus,
 ′
′
e(α+iβ )t = eαt (cos (βt) + i sin (βt))
= eαt [α (cos (βt) + i sin (βt)) + (−β sin (βt) + iβ cos (βt))]
2.8. DIVIDING POLYNOMIALS 25

Now consider the other side. From the definition it equals

(α + iβ ) eαt (cos (βt) + i sin (βt)) = eαt [(α + iβ ) (cos (βt) + i sin (βt))]

= eαt [α (cos (βt) + i sin (βt)) + (−β sin (βt) + iβ cos (βt))]
which is the same thing. This is of fundamental importance in differential equations. It
shows that there is no change in going from real to complex numbers for ω in the consid-
eration of the problem y′ = ωy, y (0) = 1. The solution is always eωt . The formula just
discussed, that
eα (cos β + i sin β ) = eα+iβ
is Euler’s formula.

2.8 Dividing Polynomials

It will be very important to be able to work with polynomials in certain parts of linear
algebra to be presented later. It is surprising how useful this junior high material will be.

Definition 2.8.1 A polynomial is an expression of the form an λ n + an−1 λ n−1 + · · · + a1 λ +

a0 , an ̸= 0 where the ai are numbers. Two polynomials are equal means that the coefficients
match for each power of λ . The degree of a polynomial is the largest power of λ . Thus the
degree of the above polynomial is n. Addition of polynomials is defined in the usual way
as is multiplication of two polynomials. The leading term in the above polynomial is an λ n .
The coefficient of the leading term is called the leading coefficient. It is called a monic
polynomial if an = 1.

Note that the degree of the zero polynomial is not defined in the above. The following
is called the division algorithm.

Lemma 2.8.2 Let f (λ ) and g (λ ) ̸= 0 be polynomials. Then there exist polynomials, q (λ )

and r (λ ) such that
f (λ ) = q (λ ) g (λ ) + r (λ )
where the degree of r (λ ) is less than the degree of g (λ ) or r (λ ) = 0. These polynomials
q (λ ) and r (λ ) are unique.

Proof: Suppose that f (λ ) − q (λ ) g (λ ) is never equal to 0 for any q (λ ). If it is, then

the conclusion follows. Now suppose

r (λ ) = f (λ ) − q (λ ) g (λ )

and the degree of r (λ ) is m ≥ n where n is the degree of g (λ ). Say the leading term of
r (λ ) is bλ m while the leading term of g (λ ) is b̂λ n . Then letting a = b/b̂ , aλ m−n g (λ ) has
the same leading term as r (λ ). Thus the degree of r1 (λ ) ≡ r (λ ) − aλ m−n g (λ ) is no more
than m − 1. Then
q1 (λ )
 
z }| {
r1 (λ ) = f (λ ) − q (λ ) g (λ ) + aλ m−n g (λ ) = f (λ ) − q (λ ) + aλ m−n  g (λ )

 
26 CHAPTER 2. ALGEBRA AND NOTATION

Denote by S the set of polynomials f (λ ) − g (λ ) l (λ ) . Out of all these polynomials,

there exists one which has smallest degree r (λ ). Let this take place when l (λ ) = q (λ ).
Then by the above argument, the degree of r (λ ) is less than the degree of g (λ ). Otherwise,
there is one which has smaller degree. Thus f (λ ) = g (λ ) q (λ ) + r (λ ).
As to uniqueness, if you have r (λ ) , r̂ (λ ) , q (λ ) , q̂ (λ ) which work, then you would
have
(q̂ (λ ) − q (λ )) g (λ ) = r (λ ) − r̂ (λ )
Now if the polynomial on the right is not zero, then neither is the one on the left. Hence this
would involve two polynomials which are equal although their degrees are different. This
is impossible. Hence r (λ ) = r̂ (λ ) and so, matching coefficients implies that q̂ (λ ) = q (λ ).
■

2.9 The Fundamental Theorem Of Algebra

The fundamental theorem of algebra states that every non constant polynomial having co-
efficients in C has a zero in C. If C is replaced by R, this is not true because of the
example, x2 + 1 = 0. This theorem is a very remarkable result and notwithstanding its ti-
tle, all the most straightforward proofs depend on either analysis or topology. It was first
mostly proved by Gauss in 1797. The first complete proof was given by Argand in 1806.
The proof given here follows Rudin [31]. See also Hardy [20] for a similar proof, more
discussion and references. The shortest proof is found in the theory of complex analysis.
First I will give an informal explanation of this theorem which shows why it is is reasonable
to believe in the fundamental theorem of algebra.

Theorem 2.9.1 Let p (z) = an zn + an−1 zn−1 + · · · + a1 z + a0 where each ak is a complex

number and an ̸= 0, n ≥ 1. Then there exists w ∈ C such that p (w) = 0.

To begin with, here is the informal explanation. Dividing by the leading coefficient an ,
there is no loss of generality in assuming that the polynomial is of the form

p (z) = zn + an−1 zn−1 + · · · + a1 z + a0

If a0 = 0, there is nothing to prove because p (0) = 0. Therefore, assume a0 ̸= 0. From

the polar form of a complex number z, it can be written as |z| (cos θ + i sin θ ). Thus, by
DeMoivre’s theorem,
zn = |z|n (cos (nθ ) + i sin (nθ ))
It follows that zn is some point on the circle of radius |z|n
Denote by Cr the circle of radius r in the complex plane which is centered at 0. Then
if r is sufficiently large and |z| = r, the term zn is far larger than the rest of the poly-
nomial. It is on the circle of radius |z|n while the other terms are on circles of fixed
multiples of |z|k for k ≤ n − 1. Thus, for r large enough, Ar = {p (z) : z ∈ Cr } describes
a closed curve which misses the inside of some circle having 0 as its center. It won’t
be as simple as suggested in the following picture, but it will be a closed curve thanks
to De Moivre’s theorem and the observation that the cosine and sine are periodic. Now
shrink r. Eventually, for r small enough, the non constant terms are negligible and so Ar
is a curve which is contained in some circle centered at a0 which has 0 on the outside.
2.9. THE FUNDAMENTAL THEOREM OF ALGEBRA 27
Ar
• a0 Ar r large Thus it is reasonable to believe that for some r dur-
ing this shrinking process, the set Ar must hit 0. It
•0
follows that p (z) = 0 for some z.
r small
For example, consider the polynomial x3 + x +
1 + i. It has no real zeros. However, you could let z = r (cost + i sint) and insert this into
the polynomial. Thus you would want to find a point where
(r (cost + i sint))3 + r (cost + i sint) + 1 + i = 0 + 0i
Expanding this expression on the left to write it in terms of real and imaginary parts, you
get on the left
r3 cos3 t − 3r3 cost sin2 t + r cost + 1 + i 3r3 cos2 t sint − r3 sin3 t + r sint + 1

Thus you need to have both the real and imaginary parts equal to 0. In other words, you
need to have
r3 cos3 t − 3r3 cost sin2 t + r cost + 1, 3r3 cos2 t sint − r3 sin3 t + r sint + 1 = (0, 0)

for some value of r and t. First here is a graph of this parametric function of t for t ∈ [0, 2π]
on the left, when r = 4. Note how the graph misses the origin 0 + i0. In fact, the closed
curve surrounds a small circle which has the point 0 + i0 on its inside.
r too big r too small r just right
50 2 4
2
y 0 y 0 y
0
-50 -2 -2
-50 0 50 -2 0 2 -4 -2 0 2 4 6
x x x
Next is the graph when r = .5. Note how the closed curve is included in a circle which
has 0 + i0 on its outside. As you shrink r you get closed curves. At first, these closed
curves enclose 0 + i0 and later, they exclude 0 + i0. Thus one of them should pass through
this point. In fact, consider the curve which results when r = 1. 386 which is the graph on
the right. Note how for this value of r the curve passes through the point 0 + i0. Thus for
some t, 1.3862 (cost + i sint) is a solution of the equation p (z) = 0.
Now here is a rigorous proof for those who have studied analysis.
Proof. Suppose the nonconstant polynomial p (z) = a0 + a1 z + · · · + an zn , an ̸= 0, has
no zero in C. Since lim|z|→∞ |p (z)| = ∞, there is a z0 with
|p (z0 )| = min |p (z)| > 0
z∈C
p(z+z0 )
Then let q (z) = p(z0 ) . This is also a polynomial which has no zeros and the minimum of
|q (z)| is 1 and occurs at z = 0. Since q (0) = 1, it follows q (z) = 1 + ak zk + r (z) where r (z)
consists of higher order terms. Here ak is the first coefficient which is nonzero. Choose a
sequence, zn → 0, such that ak zkn < 0. For example, let −ak zkn = (1/n). Then

|q (zn )| = 1 + ak zk + r (z) ≤ 1 − 1/n + |r (zn )| = 1 + ak zkn + |r (zn )| < 1

for all n large enough because |r (zn )| is small compared with ak zkn since |r (zn )| involves
only higher order terms and ak zkn < 0. This is a contradiction. ■
28 CHAPTER 2. ALGEBRA AND NOTATION

2.10 Exercises
1 1 1
1. Prove by induction that ∑nk=1 k3 = n4 + n3 + n2 .
4 2 4
1 √
2. Prove by induction that whenever n ≥ 2, ∑nk=1 √ > n.
k
3. Prove by induction that 1 + ∑ni=1 i (i!) = (n + 1)!.
4. The binomial theorem states (x + y)n = ∑nk=0 nk xn−k yk where

         
n+1 n n n n
= + if k ∈ [1, n] , ≡1≡
k k k−1 0 n
Prove the binomial theorem by induction. Next show that
 
n n!
= , 0! ≡ 1
k (n − k)!k!
▶
5. Let z = 5 + i9. Find z−1 .
6. Let z = 2 + i7 and let w = 3 − i8. Find zw, z + w, z2 , and w/z.
7. Give the complete solution to x4 + 16 = 0.
8. Graph the complex cube roots of 8 in the complex plane. Do the same for the four
fourth roots of 16. ▶
9. If z is a complex number, show there exists ω a complex number with |ω| = 1 and
ωz = |z| .
10. De Moivre’s theorem says [r (cost + i sint)]n = rn (cos nt + i sin nt) for n a positive
integer. Does this formula continue to hold for all integers n, even negative integers?
Explain. ▶
11. You already know formulas for cos (x + y) and sin (x + y) and these were used to
prove De Moivre’s theorem. Now using De Moivre’s theorem, derive a formula for
sin (5x) and one for cos (5x). ▶
12. If z and w are two complex numbers and the polar form of z involves the angle θ
while the polar form of w involves the angle φ , show that in the polar form for zw
the angle involved is θ + φ . Also, show that in the polar form of a complex number
z, r = |z| .
13. Factor x3 + 8 as a product of linear factors.
14. Write x3 + 27 in the form (x + 3) x2 + ax + b where x2 + ax + b cannot be factored

any more using only real numbers.

15. Completely factor x4 + 16 as a product of linear factors.
16. Factor x4 + 16 as the product of two quadratic polynomials each of which cannot be
factored further without using complex numbers.
2.10. EXERCISES 29

17. If z, w are complex numbers prove zw = zw and then show by induction that ∏nj=1 z j =
∏nj=1 z j . Also verify that ∑m m
k=1 zk = ∑k=1 zk . In words this says the conjugate of a
product equals the product of the conjugates and the conjugate of a sum equals the
sum of the conjugates.
18. Suppose p (x) = an xn + an−1 xn−1 + · · · + a1 x + a0 where all the ak are real numbers.
Suppose also that p (z) = 0 for some z ∈ C. Show it follows that p (z) = 0 also.
19. Show that 1 + i, 2 + i are the only two zeros to

p (x) = x2 − (3 + 2i) x + (1 + 3i)

so the zeros do not necessarily come in conjugate pairs if the coefficients are not real.
20. I claim that 1 = −1. Here is why.
√ √ q √
−1 = i2 = −1 −1 = (−1)2 = 1 = 1.

This is clearly a remarkable result but is there something wrong with it? If so, what
is wrong?
21. De Moivre’s theorem is really a grand thing. I plan to use it now for rational expo-
nents, not just integers.

1 = 1(1/4) = (cos 2π + i sin 2π)1/4 = cos (π/2) + i sin (π/2) = i.

Therefore, squaring both sides it follows 1 = −1 as in the previous problem. What

does this tell you about De Moivre’s theorem? Is there a profound difference between
raising numbers to integer powers and raising numbers to non integer powers?
22. Review Problem 10 at this point. Now here is another question: If n is an integer, is
it always true that (cos θ − i sin θ )n = cos (nθ ) − i sin (nθ )? Explain.
23. Suppose you have any polynomial in cos θ and sin θ . By this I mean an expression
of the form ∑m n β
α=0 ∑β =0 aαβ cos θ sin θ where aαβ ∈ C. Can this always be written
α

in the form ∑m+n b cos γθ + ∑n+m

γ=−(n+m) γ
c sin τθ ? Explain.
τ=−(n+m) τ

24. Suppose p (x) = an xn + an−1 xn−1 + · · · + a1 x + a0 is a polynomial and it has n zeros,

z1 , z2 , · · · , zn

listed according to multiplicity. (z is a root of multiplicity m if the polynomial f (x) =

(x − z)m divides p (x) but (x − z) f (x) does not.) Show that

p (x) = an (x − z1 ) (x − z2 ) · · · (x − zn ) .

25. Give the solutions to the following quadratic equations having real coefficients.

(a) x2 − 2x + 2 = 0
(b) 3x2 + x + 3 = 0
(c) x2 − 6x + 13 = 0
30 CHAPTER 2. ALGEBRA AND NOTATION

(d) x2 + 4x + 9 = 0
(e) 4x2 + 4x + 5 = 0

26. Give the solutions to the following quadratic equations having complex coefficients.
Note how the solutions do not come in conjugate pairs as they do when the equation
has real coefficients.

(a) x2 + 2x + 1 + i = 0
(b) 4x2 + 4ix − 5 = 0
(c) 4x2 + (4 + 4i) x + 1 + 2i = 0
(d) x2 − 4ix − 5 = 0
(e) 3x2 + (1 − i) x + 3i = 0

27. Prove the fundamental theorem of algebra for quadratic polynomials having coef-
ficients in C. That is, show that an equation of the form ax2 + bx + c = 0 where
a, b, c are complex numbers, a ̸= 0 has a complex solution. Hint: Consider the fact,
noted earlier that the expressions given from the quadratic formula do in fact serve
as solutions.
Chapter 3

Integrals, Functions of One

Variable

One cannot very well study integrals of functions of many variables without some knowl-
edge of integrals of one variable.
I assume the reader is familiar with the usual techniques for finding antiderivatives
and integrals such as partial fractions, integration by parts and integration by substitution.
These topics are usually done very well in beginning calculus courses so I am not giving
lots of exercises and examples related to formal symbol pushing techniques. However, the
typical calculus book does not even give a complete explanation of why the integral of a
continuous function exists.
I do not wish this book to be based on the kind of thing encountered in religion where
we are asked to choose to believe without any good reason for doing so or even a very
coherent description of what we are to believe. I do not wish to disparage religion since I
am a religious man myself who chooses to believe many things with no solid evidence, even
in the presence of obvious contradictions and patent absurdities, but math should not be this
way. Nor should it in any way resemble magic. This is why I am attempting to give rational
explanations. Sometimes these may fall flat, but at least I am giving it a try which is more
than can be said of the typical undergraduate presentation of courses related to calculus. If
you don’t even understand why the integral exists, then what is the meaning of everything
dependent on the integral? These things become nothing more than meaningless ritual and
speculation (religion). This short chapter is on the fundamental questions related to the
integral which are usually not discussed in undergraduate calculus. For a more complete
treatment of Riemann integration which includes what is here, see my book on calculus of
functions of one and many variable or the single variable advanced calculus book for a lot
more.
The fundamental issues depend not on techniques of integration or some stupid geo-
metric reasoning but on completeness of R.

Definition 3.0.1 One of the equivalent definitions of completeness of R is that if S is any

nonempty subset of R which is bounded above, then there exists a least upper bound for S
and if S is bounded below, then there exists a greatest lower bound for S. The least upper
bound of S is denoted as sup (S) or sometimes as l.u.b. (S) while the greatest lower bound
of S is denoted as inf (S) sometimes as g.l.b. (S). If there is no upper bound for S we say

31
32 CHAPTER 3. INTEGRALS, FUNCTIONS OF ONE VARIABLE

sup (S) = ∞. If there is no lower bound, we say inf (S) = −∞.

The words mean exactly what they say. sup (S) is a number with the property that
s ≤ sup (S) for all s ∈ S and out of all such “upper bounds” it is the smallest. inf (S) has the
property that inf (S) ≤ s for all s ∈ S and if l ≤ s for all s ∈ S, then l ≤ inf (S) . In words,
it is the largest lower bound and sup (S) is the smallest upper bound. Here the meaning of
small and large are as follows. To say that x is smaller than y means that x ≤ y which also
says that y is larger than x.
A consequence of this axiom is the nested interval lemma, Lemma 3.0.2.

Lemma 3.0.2 Let Ik = ak , bk and suppose that for all k = 1, 2, · · · ,

 

Ik ⊇ Ik+1 .

Then there exists a point, c ∈ R which is an element of every Ik . If

lim bk − ak = 0
k→∞

then there is exactly one point in all of these intervals.

Proof: Since Ik ⊇ Ik+1 , this implies

ak ≤ ak+1 , bk ≥ bk+1 . (3.1)

Consequently, letting k ≤ l,
al ≤ al ≤ bl ≤ bk . (3.2)
Thus n o n o
c ≡ sup al : l = 1, 2, · · · = sup al : l = k, k + 1, · · · ≤ bk

because bk is an upper bound for all the al. Thenc ≥ al for all l. In other words x ≥ ak for
all k. Also c ≤ bk for all k. Therefore, c ∈ ak , bk for all k.
If the length of these intervals converges to 0, then there can be at most one point in
their intersection since otherwise, you would have two different points c, d and the length
of the kth interval would then be at least as large as |d − c| but both of these points would
need to be in intervals having smaller length than this which can’t happen. ■

Corollary 3.0.3 Suppose

 {xn } is a sequence contained in [a, b]. Then there exists x ∈ [a, b]
and a subsequence xnk such that limk→∞ xnk = x.

Proof: Consider a sequence of closed intervals contained in [a, b] , I1 , I2 , · · · where Ik+1

is one half
 of Ik and  each Ik contains xn for infinitely many values of n. Thus I1 = [a, b] , I2 is
either a, a+b
2 or a+b2 , b , depending on which one contains xn for infinitely many values
of n. If both intervals have this property, just pick one. Let xnk ∈ Ik and let xnk+1 ∈ Ik+1 with
nk+1 > nk . This is possible to do because each Ik contains xn for infinitely many values of
n. Then by the nested interval lemma, there exists a unique point x contained in all of these
intervals and x − xnk < (b − a) /2k . ■
The next corollary is the extreme value theorem from calculus.
 33

Corollary 3.0.4 If f : [a, b] → R is continuous, then there exists xM ∈ [a, b] such that

f (xM ) = sup { f (x) : x ∈ [a, b]}

and there exists xm ∈ [a, b] such that

f (xm ) = inf { f (x) : x ∈ [a, b]}

Proof: From the definition of inf { f (x) : x ∈ [a, b]} , there exists xn ∈ [a, b] such that

f (xn ) ≤ inf { f (x) : x ∈ [a, b]} + 1/n

That is, limn→∞ f (xn ) = inf { f (x) : x ∈ [a, b]} . This is called a minimizing sequence. There-
fore, there is a subsequence xnk which converges to x ∈ [a, b] . By continuity of f it
follows that

inf { f (x) : x ∈ [a, b]} = lim f xnk = f (x)
k→∞

The case where f achieves its maximum is similar. You just use a maximizing sequence.
■

Corollary 3.0.5 If {xn } is a Cauchy sequence, then it converges.

Proof: The Cauchy sequence is contained in some closed interval [a, b]. This is be-
cause, letting ε = 1, it follows that there exists N such that if m, n ≥ N, then |xn − xm | < 1. In
particular, for all n ≥ N, |xn − xN | < 1. Therefore, |xn | ≤ max {|xN | + 1, |x1 | , |x2 | , · · · , |xN |}
for
 all n. By Corollary 3.0.3, there is a subsequence of the Cauchy sequence, denoted as
xnk which converges to some x ∈ [a, b]. Since the original sequence is a Cauchy se-
quence, letting ε > 0 be given, there is N such that if k, l ≥ N, then |xk − xl | < ε/2 and
xnk − x < ε/2. Thus if m ≥ N, then
ε ε
|x − xm | ≤ |x − xnm | + |xnm − xm | < + =ε
2 2
Indeed, if m ≥ N, then nm ≥ N because {xnm }∞ m=1 is a subsequence. Thus the original
Cauchy sequence converges to x. ■
Actually, the convergence of every Cauchy sequence is equivalent to completeness and
so it gives another way of defining completeness in contexts where no order is available.
Recall completeness means that every nonempty set bounded above (below) has a least
upper bound (greatest lower bound). More consideration of this issue is a good topic for
advanced calculus courses. This standard definition involving least upper bounds depends
on an order. One can prove that if you have the least upper bound property described in the
above definition, then you also have the greatest lower bound property also described there
and the other way around.
The Riemann integral pertains to bounded functions which are defined on a bounded
interval. Let [a, b] be a closed interval. A set of points in [a, b], {x0 , · · · , xn } is a partition if

a = x0 < x1 < · · · < xn = b.

Such partitions are denoted by P or Q.

34 CHAPTER 3. INTEGRALS, FUNCTIONS OF ONE VARIABLE

Definition 3.0.6 A function f : [a, b] → R is bounded if the set of values of f is contained

in some interval. Thus

sup { f (x) : x ∈ [a, b]} < ∞, inf { f (x) : x ∈ [a, b]} > −∞

Letting P denote a partition,

∥P∥ ≡ max {|xi+1 − xi | : i = 0, · · · , n − 1} .

A Riemann sum for a bounded f corresponding to a partition P = {x0 , · · · , xn } is a sum of

the form
n
∑ f ≡ ∑ f (yi ) (xi − xi−1 )
P i=1

where yi ∈ [xi−1 , xi ]. Then there are really many different Riemann sums corresponding to
a given partition, depending on which yi is chosen.

For example, suppose f is a function with positive values. The above Riemann sum
involves adding areas of rectangles. Here is a picture:

The area under the curve is close to the sum of the areas of these rectangles and one
would imagine that this would become an increasingly good approximation if you included
more and narrower rectangles.

Definition 3.0.7 A bounded function defined on an interval [a, b] is Riemann integrable

means that there exists a number I such that for every ε > 0, there exists a δ > 0 such
that whenever ∥P∥ < δ , and ∑P f is some Riemann sum corresponding to this partition, it
follows that

∑ f −I <ε
P

This is written as
lim
∥P∥→0 P
∑f =I
and when this number exists, it is denoted by
Z b
I= f (x) dx
a

One of the big theorems is on the existence of the integral whenever f is a continuous
function. This requires a technical lemma which follows.

Lemma 3.0.8 Let f : [a, b] → R be continuous. Then for every ε > 0 there exists a δ > 0
such that if |x − y| < δ , x, y ∈ [a, b] , it follows that | f (x) − f (y)| < ε.
 35

Proof: If not, then there exists ε > 0 and xn , yn , |xn − yn | < 1/n but

| f (xn ) − f (yn )| ≥ ε.

By Corollary 3.0.3, there is a subsequence xnk and point x ∈ [a, b] such that limk→∞ xnk =
x. Then it follows that also limk→∞ ynk = x also because

ynk − x ≤ ynk − xnk + xnk − x

and both of the terms on the right converge to 0. But then by continuity of f ,




0 = f (x) − f (x) = lim f xnk − f ynk
k→∞



which is impossible because f xnk − f ynk ≥ ε for all k. ■
With this preparation, here is the major result on the existence of the integral of a
continuous function.

Theorem 3.0.9 Let f : [a, b] → R be continuous. Then ab f (x) dx exists. In fact, there
R

exists a sequence δ m converging to 0 such that if ∥P∥ < δ m , and if ∑P f is a Riemann sum
for P, then
Z b
2
∑ f − f dx < (b − a)
P a m
1
δ m is defined to be such that if |x − y| < δ m , then | f (x) − f (y)| < m and the sequence is
decreasing.

Proof: Consider a partition P given by a = x0 < x1 < · · · < xn = b. Then you could add
in another point as follows:

a = x0 < x1 < · · · < xi−1 < x∗ < xi < · · · < xn = b

Denote this one by Q. Then if you have a Riemann sum,

n

∑ f = ∑ f (y j ) x j − x j−1
P j=1

You could write this sum in the following form.

i−1 n
x j − x j−1 + f (yi ) (x∗ − xi−1 ) + f (yi ) (xi − x∗ ) +



∑ f (y j ) ∑ f (y j ) x j − x j−1
j=1 j=i+1

In fact, you could continue adding in points and doing the same trick and thereby write the
original sum in terms of any partition containing P. If R is a partition containing P and if
δ m corresponds to ε = 1/m in the above Lemma with · · · > δ m > δ m+1 · · · 3.0.8, then one
can conclude that if ∥P∥ < δ m , then

1
∑ f −∑ f ≤
m
(b − a)
P R
36 CHAPTER 3. INTEGRALS, FUNCTIONS OF ONE VARIABLE

Now if ∥P∥ , ∥Q∥ < δ m ,let R = P ∪ Q. Then

∑ f −∑ f ≤ ∑ f −∑ f + ∑ f −∑ f
P Q P R R Q
1 1 2
≤ (b − a) + (b − a) = (b − a)
m m m
Let M ≥ max {| f (x)| : x ∈ [a, b]} . Then all Riemann sums are in the interval

[−M (b − a) , M (b − a)]

Now let ( )
Sn ≡ ∑ f : ∥P∥ < δ n
P

Then Sn ⊇ Sn+1 for all n thanks to the fact that the δ n are decreasing. Let

In = [inf (Sn ) , sup (Sn )]

These are nested intervals contained in [−M (b − a) , M (b − a)] and so there exists I con-
tained in them all. However, from the above computation,
2
sup (Sn ) − inf (Sn ) ≤ (b − a)
n
and so there is only one such I. Hence for any ε > 0 given, there exists δ > 0 such that if
∥P∥ < δ , then

∑ f −I <ε ■
S

We say that a bounded function f defined on an interval [a, b] is Riemann integrable if

the above integral exists. This is written as f ∈ R ([a, b]). The above just showed that every
continuous function is Riemann integrable.
Not all bounded functions are Riemann integrable. For example, let x ∈ [0, 1] and
(
1 if x ∈ Q
f (x) ≡ (3.3)
0 if x ∈ R \ Q

This has no Riemann integral because you can pick a sequence of partitions Pn , such that
∥Pn ∥ < 1/n and each partition point is rational. Then for your Riemann sums, take the
value of the function at the left end point. The resulting Riemann sum will always equal 1.
But you could just as easily pick your point yi in the Riemann sum to equal an irrational
number and these Riemann sums will all equal 0. Therefore, the condition for integrability
is violated for ε = 1/4.
If you can partition the interval [a, b] into finitely many intervals [zi−1 , zi ], such that a
function f is continuous on each [zi−1 , zi ] , then the function will be integrable on [a, b].
This is roughly the claim of the next theorem.

Definition 3.0.10 A bounded function f : [a, b] → R is called piecewise continuous if there

are points zi such that a = z0 < z1 < · · · < zn = b and continuous functions gi : [zi−1 , zi ] → R
such that for t ∈ (zi−1 , zi ) , gi (t) = f (t).
 37

Corollary 3.0.11 Let f : [a, b] → R be piecewise continuous. Then f is Riemann inte-

grable. Also
Z b n Z zi
f dt = ∑ gi dt (3.4)
a i=1 zi−1

Proof: Let Pi be a partition for [zi−1 , zi ] . Since there are only finitely many of these
intervals, there exists δ > 0 such that if ∥Pi ∥ < δ , then for each i,
Z zi
∑ gi − zi−1
gi dt < ε
Pi

Let M f be an upper bound for | f | on [a, b], Mg an upper bound for all |gi |. Now let ∥P∥ <
δ < ε where P is a partition of [a, b] , these points denoted as x j . Let P̂i be those points of
P which are in (zi−1 , zi ] and let Pi consist of P̂i along with zi−1 and zi . Thus ∥Pi ∥ < δ . Then
for yi ∈ [xi−1 , xi ] ,

n Z zi n Z zi

∑ gi dt − ∑ f ≤ ∑ gi dt − ∑ f (y j ) x j − x j−1
i=1 zi−1 P i=1 zi−1
x j ∈P̂i

Now for x j ∈ P̂i ,
f (y j ) = gi (y j ) except maybe at end points where these differ by no more
than 2 M f + Mg ≡ 2M. Thus the above is no more than
n Z zi
n

≤ ∑ gi dt − ∑ gi (y j ) x j − x j−1 + ∑ 4 M f + Mg δ
i=1 zi−1 x j ∈Pi i=1

< nε + 4Mnδ < ε (n + 4Mn)

Since ε is arbitrary, this shows that f is indeed Riemann integrable and equals 3.4. ■
Note that what has actually been shown is that if a bounded function f satisfies f = gi
on [zi−1 , zi ] except for possibly the end points, and gi is Riemann integrable on [zi−1 , zi ] ,
then f is Riemann integrable on [a, b]. The above proof applies with no change.
It is important to notice that the integral is linear. That is, for α, β numbers and f , g
piecewise continuous functions,
Z b Z b Z b
(α f + β g) dx = α f dx + β gdx
a a a

This is easy to see because such linearity holds for sums. Thus
Z b
(α f + β g) dx ≡ lim ∑α f +βg
a ∥P∥→0 P
Z b Z b
= lim α ∑ f + β ∑ g = α f dx + β gdx
∥P∥→0 P P a a

I leave the details to you. Actually, this works under the assumption that f , g are Riemann
integrable but in this case, you have to show that a linear combination of Riemann inte-
grable functions is Riemann integrable. This is not hard but I don’t want to waste time on
it.
The above is the Riemann integral. There is another integral which can be proved to be
equivalent to the above. It is called the Darboux integral.
38 CHAPTER 3. INTEGRALS, FUNCTIONS OF ONE VARIABLE

Definition 3.0.12 For P a partition a = x0 < · · · < xn = b and

Mi ≡ {sup f (x) : x ∈ [xi−1 , xi ]} ,

mi ≡ inf {inf f (x) : x ∈ [xi−1 , xi ]}

for f a bounded function. Then the upper sum and lower sum are respectively
n
U ( f , P) ≡ ∑ Mi (xi − xi−1 ) ,
i=1
n
L ( f , P) ≡ ∑ mi (xi − xi−1 )
i=1

I ≡ inf{U ( f , P) where P is a partition}

I ≡ sup{L ( f , P) where P is a partition}.
We say that f is Darboux integrable if I¯ = I and the Darboux integral is the common value
of these.

Note that I and I are well defined real numbers and this definition of an integral really
says that there is a unique number between all the upper sums and lower sums. If f is
Riemann integrable, then it is not hard to see it is Darboux integrable. Indeed, from the
definition, there exists P such that whenever you have a Riemann sum for P,
Z b
f dx − ∑ f < ε
a P

In particular, this shows after a little consideration that

|U ( f , P) − L ( f , P)| < 2ε,

Thus, since ε is arbitrary, there can’t be more than one number between all the upper
and lower sums and this number must be the Riemann integral. One can also show that
every lower sum is no larger than every upper sum, even if taken with respect to different
partitions. In this book, we are mainly interested in piecewise continuous functions and
so once you know these are Riemann integrable, it follows automatically that they are
Darboux integrable. It can be shown that the two definitions are equivalent, but this is not
needed in this book. I think it is a little more convenient to use the Darboux approach when
dealing with the theory of the integral of a function of many variables which is done in the
appendix. Either integral handles the functions of most interest and gives the same answer
for these. However, both of these integrals have been obsolete for over a hundred years.

3.1 Properties of the Integral

To find the integral of a continuous function, one can often use another method which
is much easier than taking the limit of Riemann sums. This other method is called the
fundamental theorem of calculus. There are two forms to this theorem, one enabling the
computation of the integral and another which gives the existence of a function whose
derivative is a given function.
3.1. PROPERTIES OF THE INTEGRAL 39

Theorem 3.1.1 Suppose F ′ (x) = f (x) where f is a continuous function on [a, b] . Then
Z b
f (x) dx = F (b) − F (a) (3.5)
a

Proof: Let ε > 0 be given and let P be a partition a = x0 < x1 < · · · < xn = b such that
whenever ŷi ∈ [xi−1 , xi ] ,
Z b n
f (x) dx − ∑ f (ŷi ) (xi − xi−1 ) < ε (3.6)
a i=1

Then from the mean value theorem, there exists yi ∈ (xi−1 , xi ) such that
n
F (b) − F (a) = ∑ (F (xi ) − F (xi−1 ))
i=1
n n
= ∑ F ′ (yi ) (xi − xi−1 ) = ∑ f (yi ) (xi − xi−1 )
i=1 i=1

Let ŷi in 3.6 be equal to yi just described. Then with the above, this shows that
Z b
f (x) dx − (F (b) − F (a)) < ε
a

Since ε is arbitrary, this verifies 3.5. ■

R2
Example 3.1.2 Find 0 cos (t) dt.

Note that cos (t) = sin′ (t) and so the above integral is sin (2) − sin (0) = sin (2).
Rb
Example 3.1.3 Find a αdx.

A function whose derivative is α is x → αx. Therefore, this integral is αb − αa =

α (b − a).
The integral ab f (t) dt has been defined when f is continuous and a < b. What if a > b?
R

The following definition tells what this equals.

Definition 3.1.4 Let [a, b] be an interval and let f be piecewise continuous on [a, b] or
more generally Riemann integrable on this interval. Then
Z a Z b
f (t) dt ≡ − f (t) dt
b a
Rb
Observation 3.1.5 With the above definition, a dx is linear satisfying
Z b Z b Z b
(α f + β g) dx = α f dx + β gdx
a a a
Rb
if a < b or b < a. Also a αdx = αb − αa if a < b or b < a.
40 CHAPTER 3. INTEGRALS, FUNCTIONS OF ONE VARIABLE

Note that this definition must hold if we want to continue to use Theorem 3.1.1. With
this definition, one can give a convenient theorem. It holds for general Riemann integrable
functions. However, I am stating it only for the case of most interest, piecewise continuous
ones because I am basing the argument on Corollary 3.0.11. As noted, this corollary will
end up holding in greater generality with very little change in the proof.

Theorem 3.1.6 Suppose a, b, c are all points in some interval on which f is piecewise
continuous. Then Z b Z c Z c
f (t) dt + f (t) dt = f (t) dt (3.7)
a b a

Proof: case 1: a < b < c In this case, 3.7 follows from Corollary 3.0.11.
case 2: a < c < b In this case, Corollary 3.0.11 implies
Z c Z b Z b
f (x) dx + f (x) dx = f (x) dx
a c a

and so
Z c Z b Z b
f (x) dx = f (x) dx − f (x) dx
a a c
Z b Z c
= f (x) dx + f (x) dx
a b

case 3: c < a < b In this case, Corollary 3.0.11 implies

Z a Z b Z b
f (x) dx + f (x) dx = f (x) dx
c a c

so
Z a Z b Z c
f (x) dx + f (x) dx + f (x) dx = 0
c a b
Z b Z c Z c
f (x) dx + f (x) dx = f (x) dx
a b a

This includes all cases and proves the theorem. ■

Next is the triangle inequality.

Proposition 3.1.7 Let a, b be in an interval on which f is piecewise continuous (or Rie-

mann integrable). Then
Z b Z b
f (x) dx ≤ | f (x)| dx
a a

Proof: I will give the proof for almost the only case of interest in this book, piecewise
continuous. If f is piecewise continuous, then so is | f |. Hence there is no problem with ex-
istence of the integral. Suppose first that a > b. Then, since | f | − f , | f | + f are nonnegative
functions, all Riemann sums are nonnegative and so their limit is also nonnegative. Hence
Z a Z a Z a
0 ≤ (| f (x)| − f (x)) dx = | f (x)| dx − f (x) dx
Zb a Zb a Zb a
0 ≤ (| f (x)| + f (x)) dx = | f (x)| dx + f (x) dx
b b b
3.1. PROPERTIES OF THE INTEGRAL 41

and so
Z a Z a Z b
f (x) dx ≤ | f (x)| dx = | f (x)| dx
b b a
Z a Z a Z b
− f (x) dx ≤ | f (x)| dx = | f (x)| dx
b b a

It follows that in this case where a > b,

Z a Z b Z b
f (x) dx = f (x) dx ≤ | f (x)| dx
b a a

The argument is the same in case a < b except you work with ab rather than ba . ■
R R

With these basic properties of the integral, here is the other form of the fundamental
theorem of calculus. This major theorem, due to Newton and Leibniz shows the existence
of an “anti-derivative” for any continuous function.
Theorem 3.1.8 Let f be continuous on [a, b]. Also let
Z t
F (t) ≡ f (x) dx
a

Then for every t ∈ (a, b) ,

F ′ (t) = f (t) .
Proof: For t ∈ (a, b) and |h| sufficiently small, t + h ∈ (a, b). Always let h be this small.
Then, from the above properties of integrals in Proposition 3.1.7, and Theorem 3.1.6,
Z t+h Z t
1 t+h

F (t + h) − F (t) 1
Z
= f (x) dx − f (x) dx = f (x) dx
h h a a h t
Now from Observation 3.1.5,
Z t+h
1
f (t) dt = f (t)
h t

Therefore, by the properties of the integral given above,

F (t + h) − F (t) 1 t+h 1 t+h
Z Z
− f (t) = f (x) dx − f (t) dx
h h t h t
1 t+h
Z
= ( f (x) − f (t)) dx
h t
Z t+h
1
≤ | f (x) − f (t)| dx
|h| t
Now if |h| is small enough, | f (x) − f (t)| < ε by continuity of f at x. Therefore, for |h| this
small, Z t+h
F (t + h) − F (t) 1
− f (t) ≤ εdx = ε
h |h| t
Since ε is arbitrary, it follows from the definition of the limit that
F (t + h) − F (t)
lim = f (t) ■
h→0 h
42 CHAPTER 3. INTEGRALS, FUNCTIONS OF ONE VARIABLE

Corollary 3.1.9 For F (t) defined as above, it is also true that F ′ (a) = f (a) and F ′ (b) =
f (b) provided the derivatives are taken respectively from the right and from the left.

Proof: You repeat the above argument paying attention to the sign of h. Otherwise
there is no change. ■

Definition 3.1.10 When F ′ (t) = f (t) for t on some interval, the function
R
t → F (t) is
R
called
an anti-derivative for f . The set of all anti-derivatives is denoted as f dx. Thus f dx is
a collection of functions, not a number, while ab f (x) dx is a number.
R

R
Proposition 3.1.11 Suppose F, G ∈ f dx for x in some interval. Then there exists a con-
stant C such that F (x) +C = G (x).

Proof: It comes from the mean value theorem. By assumption (G − F)′ = 0 and so if
x0 is a fixed point in the interval, then if x is another point, (G − F) (x) − (G − F) (x0 ) =
(G − F)′ (z) (x − x0 ) for some z between x and x0 . But by assumption, (G − F)′ (z) = 0 and
so (G − F) (x) must equal (G − F) (x0 ) for all x in the interval. Let C = (G − F) (x0 ) . ■

3.2 Uniform Convergence of Continuous Functions

Suppose for each n ∈ N, fn is a continuous function defined on some interval [a, b] . Also
suppose that for each fixed x ∈ [a, b] , limn→∞ fn (x) = f (x). This is called pointwise con-
vergence. Does it follow that f is continuous on [a, b]? The answer is NO. Consider the
following
fn (x) ≡ xn for x ∈ [0, 1]
Then limn→∞ fn (x) exists for each x ∈ [0, 1] and equals
(
1 if x = 1
f (x) ≡
0 if x ̸= 1

You should verify this is the case. This limit function is not continuous. Indeed, it has a
jump at x = 1. Here are graphs of the first few of these functions.
y

x
If you want the convergence to carry continuity with it you need something more than
pointwise convergence.

Definition 3.2.1 Let { fn } be a sequence of functions defined on D. Then fn is said to

converge uniformly to f on D if
 
lim ∥ fn − f ∥∞ ≡ lim sup | fn (x) − f (x)| = 0
n→∞ n→∞ x∈D

∥·∥∞ is called a norm.

3.2. UNIFORM CONVERGENCE OF CONTINUOUS FUNCTIONS 43

The following picture illustrates the above definition.

f

The dotted lines define sort of a tube centered about the graph of f and the graph of the
function fn fits in this tube for all n sufficiently large. The tube can be made as narrow as
desired.
It is convenient to observe the following properties of ∥·∥∞ , written ∥·∥ for short.

Lemma 3.2.2 The norm ∥·∥∞ satisfies the following properties.

∥ f ∥ ≥ 0 and equals 0 if and only if f = 0 (3.8)

For α a number,
∥α f ∥ = |α| ∥ f ∥ (3.9)
∥ f + g∥ ≤ ∥ f ∥ + ∥g∥ (3.10)

Proof: The first claim 3.8 is obvious. As to 3.9, it follows fairly easily.

∥α f ∥ ≡ sup |α f (x)| = sup |α| | f (x)| = |α| sup | f (x)| = |α| ∥ f ∥

x∈D x∈D x∈D

The last follows from

| f (x) + g (x)| ≤ | f (x)| + |g (x)| ≤ ∥ f ∥ + ∥g∥

Therefore,
sup | f (x) + g (x)| ≡ ∥ f + g∥ ≤ ∥ f ∥ + ∥g∥ ■
x∈D

Now with this preparation, here is the main result.

Theorem 3.2.3 Let fn be continuous on D and suppose limn→∞ ∥ fn − f ∥ = 0. Then f is

also continuous. If each fn is uniformly continuous, then f is uniformly continuous.

Proof: Let ε > 0 be given and let x ∈ D. Let n be such that ∥ fn − f ∥ < ε3 . By continuity
of fn there exists δ > 0 such that if |y − x| < δ , then | fn (y) − fn (x)| < ε3 . Then for such y,

| f (y) − f (x)| ≤ | f (y) − fn (y)| + | fn (y) − fn (x)| + | fn (x) − f (x)|

ε ε ε ε
< ∥ f − fn ∥ + + ∥ fn − f ∥ < + + = ε
3 3 3 3
and so this shows that f is continuous. To show the claim about uniform continuity, use the
same string of inequalities above where δ is chosen so that for any pair x, y with |x − y| <
δ , | fn (y) − fn (x)| < ε3 . Then the above shows that if |x − y| < δ , then | f (x) − f (y)| < ε
which satisfies the definition of uniformly continuous. ■
This implies the following interesting corollary about a uniformly Cauchy sequence of
continuous functions.
44 CHAPTER 3. INTEGRALS, FUNCTIONS OF ONE VARIABLE

Definition 3.2.4 Let { fn } be a sequence of continuous functions defined on [a, b]. It is said
to be uniformly Cauchy if for every ε > 0 there exists nε such that if m, k > nε

∥ fm − fk ∥ < ε

Corollary 3.2.5 Suppose { fn } is a uniformly Cauchy sequence of functions defined on D.

Then there exists a unique continuous function f such that limn→∞ ∥ fn − f ∥ = 0. If each fn
is uniformly continuous, then so is f .

Proof: The hypothesis implies that { fn (x)} is a Cauchy sequence in R for each x. By
completeness of R, this sequence converges for each x. Let f (x) ≡ limn→∞ fn (x). Then by
continuity of y → |y − fn (x)| , for each x,

| f (x) − fn (x)| = lim | fm (x) − fn (x)| ≤ lim inf ∥ fm − fn ∥ < ε

m→∞ m→∞

provided n is sufficiently large. Since x is arbitrary, this shows that

∥ f − fn ∥ ≡ sup | f (x) − fn (x)| ≤ ε

x∈[a,b]

if n is large enough. this says limn→∞ ∥ fn − f ∥ = 0. Now the continuity of f follows from
Theorem 3.2.3. How many such functions f are there? There can be only one because f (x)
must equal the limit of fn (x). ■

3.3 Uniform Convergence And The Integral

It turns out that uniform convergence is very agreeable in terms of the integral. The follow-
ing is the main result.

Theorem 3.3.1 Let fn be continuous and converging uniformly to f on [a, b]. Then it
follows f is also continuous and
Z b Z b
f dx = lim fn dx
a n→∞ a

Proof: RThe uniform convergence implies f is also continuous. See Theorem 3.2.3.
Therefore, ab f dx exists. Using the triangle inequality and definition of ∥·∥ described ear-
lier in conjunction with this theorem,
Z b Z b Z b
f (x) dx − fn (x) dx = ( f (x) − fn (x)) dx
a a a
Z b Z b
≤ | f (x) − fn (x)| dx ≤ ∥ f − fn ∥ dx
a a
≤ ∥ f − fn ∥ (b − a)

which is given to converge to 0 as n → ∞. ■

Chapter 4

Some Important Improper

Integrals

4.1 Gamma Function

This belongs to a larger set of ideas concerning improper integrals. I will just give enough
of an introduction to this to present the very important gamma function. The Riemann
integral only is defined for bounded functions which are defined on a bounded interval.
If this is not the case, then the integral has not been defined. Of course, just because the
function is bounded does not mean the integral exists as mentioned above, but if it is not
bounded, then there is no hope for it at all. However, one can consider limits of Riemann
integrals. The following definition is sufficient to deal with the gamma function in the
generality needed in this book.
Definition 4.1.1 We say that f defined on [0, ∞) is improper Riemann integrable if it is
Riemann integrable on [δ , R] for each R > 1 > δ > 0 and the following limits exist.
Z ∞ Z 1 Z R
f (t) dt ≡ lim f (t) dt + lim f (t) dt
0 δ →0+ δ R→∞ 1

The gamma function is defined by

Z ∞
Γ (α) ≡ e−t t α−1 dt
0
whenever α > 0.
Lemma 4.1.2 The limits in the above definition exist for each α > 0.
Proof: Note first that as δ → 0+, the Riemann integrals
Z 1
e−t t α−1 dt
δ

increase. Thus limδ →0+ δ1 e−t t α−1 dt either is +∞ or it will converge to the least upper
R

bound thanks to completeness of R. However,

Z 1
1
t α−1 dt ≤
δ α

45
46 CHAPTER 4. SOME IMPORTANT IMPROPER INTEGRALS

so the limit of these integrals exists. Also e−t t α−1 ≤ Ce−(t/2) for suitable C if t > 1. This
is obvious if α − 1 < 0 and in the other case it is also clear because exponential growth
exceeds polynomial growth. Thus
Z R Z R
e−t t α−1 dt ≤ Ce−(t/2) dt ≤ 2Ce(−1/2) − 2Ce(−R/2) ≤ 2Ce(−1/2)
1 1

Thus these integrals also converge as R → ∞. It follows that Γ (α) makes sense. ■
This gamma function has some fundamental properties described in the following propo-
sition. In case the improper integral exists, we can obviously compute it in the form
Z 1/δ
lim f (t) dt
δ →0+ δ

which is used in what follows. Thus also the usual algebraic properties of the Riemann
integral are inherited by the improper integral.

Proposition 4.1.3 For n a positive integer, n! = Γ (n + 1). In general, Γ (1) = 1, Γ (α + 1) =

αΓ (α)
R −1  −1

Proof: First of all, Γ (1) = limδ →0 δδ e−t dt = limδ →0 e−δ − e−(δ ) = 1. Next,
for α > 0,
Z −1
" Z −1
#
δ −1 δ
Γ (α + 1) = lim e−t t α dt = lim −e−t t α |δδ +α e−t t α−1 dt
δ →0 δ δ →0 δ

Z δ −1
!
−δ α −(δ −1 ) −α −t α−1
= lim e δ −e δ +α e t dt = αΓ (α)
δ →0 δ

Now it is defined that 0! = 1 and so Γ (1) = 0!. Suppose that Γ (n + 1) = n!, what of
Γ (n + 2)? Is it (n + 1)!? if so, then by induction, the proposition is established. From
what was just shown,

Γ (n + 2) = Γ (n + 1) (n + 1) = n! (n + 1) = (n + 1)!

and so this proves the proposition. ■

The properties of the gamma function also allow for a fairly easy proof about differen-
tiating under the integral in a Laplace transform. First is a definition.

Definition 4.1.4 A function φ has exponential growth on [0, ∞) if there are positive con-
stants λ ,C such that |φ (t)| ≤ Ceλt for all t.

Theorem 4.1.5 Let f (s) = 0∞ e−st φ (t) dt where t → φ (t) e−st is improper Riemann in-
R

tegrable for all s large enough and φ has exponential growth. Then for s large enough,
f (k) (s) exists and equals 0∞ (−t)k e−st φ (t) dt.
R

Proof: Suppose true for some k ≥ 0. By definition it is so for k = 0. Then always

assuming s > λ , |h| < s − λ , where |φ (t)| ≤ Ceλt , λ ≥ 0,

f (k) (s + h) − f (k) (s) e−(s+h)t − e−st

Z ∞
= (−t)k φ (t) dt
h 0 h
4.2. LAPLACE TRANSFORMS 47

e−ht − 1
Z ∞   Z ∞  
k −st
= (−t) e φ (t) dt = (−t)k e−st (−t) eθ (h,t) φ (t) dt
0 h 0

where θ (h,t) is between −ht and 0, this by the mean value theorem. Thus by mean value
theorem again,
f (k) (s + h) − f (k) (s)
Z ∞
− (−t)k+1 e−st φ (t) dt
h 0
Z ∞ Z ∞
≤ |t|k+1 Ceλt e−st eθ (h,t) − 1 dt ≤ t k+1Ceλt e−st eα(h,t) |ht| dt
0 0
Z ∞ Z ∞
≤ t k+2Ceλt e−st |h| et|h| dt = C |h| t k+2 e−(s−(λ +|h|))t dt
0 0
Let u = (s − (λ + |h|))t, du = (s − (λ + |h|)) dt. Then the above equals
Z ∞ k+2
u 1
C |h| e−u du
0 s − (λ + |h|) (s − (λ + |h|))

C |h| C |h|
Z ∞
= k+3
e−u uk+2 du = Γ (k + 3)
(s − (λ + |h|)) 0 (s − (λ + |h|))k+3
Thus, as h → 0, this converges to 0 and so this proves the theorem. ■
The function s → f (s) in the above theorem is called the Laplace transform of φ .

4.2 Laplace Transforms

Suppose f is piecewise continuous on each interval [0, R], meaning that it is bounded on
that interval and equals a continuous function on each of finitely many closed subintervals
except for the end points as described in Definition 3.0.10. Then from Corollary 3.0.11,
t → f (t) is integrable. So is t → e−st f (t) . It is tacitly assumed that f is as just described in
all that follows. It is much nicer to formulate this in terms of the Lebesgue integral however
and use a condition of measurability instead of all this piecewise continuous nonsense.

Definition 4.2.1 We say that a function defined on [0, ∞) has exponential growth if for some
λ ≥ 0, and C > 0,
| f (t)| ≤ Ceλt

Note that this condition is satisfied if | f (t)| ≤ a + beλt . You simply pick C > max (a, b)
and observe that a + beλt ≤ 2Ceλt .

Proposition 4.2.2 Let f have exponential growth and be continuous except for finitely
many points in [0, R] for each R. Then
Z R
lim f (t) e−st dt ≡ L f (s)
R→∞ 0

exists for every s > λ where | f (t)| ≤ eλt . That limit is denoted as
Z ∞
f (t) e−st dt.
0
48 CHAPTER 4. SOME IMPORTANT IMPROPER INTEGRALS

Proof: Let Rn → ∞. Then for Rm < Rn ,

Z Rm Z Rn Z Rn
f (t) e−st dt − f (t) e−st ≤ | f (t)| e−st dt
0 0 Rm
Z Rn
≤ e−(s−λ )t dt ≤ e−(s−λ )Rm
Rm

The elementary
nR computationsoare left to the reader. Then this converges to 0 as Rm → ∞. It
∞
Rn −st
follows that 0 f (t) e dt is a Cauchy sequence and so it converges to I ∈ R. The
n=1
above computation shows that if R̂n also converges to ∞ as n → ∞, then
Z Rn Z R̂n
lim f (t) e−st = lim f (t) e−st
n→∞ 0 n→∞ 0

and so the limit does indeed exist and this is the definition of the following improper integral
(t) e−ts dt. ■
R∞
0 f
Certain properties are obvious. For example,
1. If a, b scalars and if g, f have exponential growth, then for all s large enough,
L (a f + bg) (s) = aL ( f ) (s) + bL (g) (s)

2. If f ′ (t) exists and has exponential growth, and so does f (t) then for s large enough,
L f ′ (s) = − f (0) + sL ( f ) (s)

One can also compute Laplace transforms of many standard functions without much
difficulty. That which is most certainly not obvious is the following major theorem. This
is the thing which is omitted from virtually all ordinary differential equations books, and
it is this very thing which justifies the use of Laplace transforms. Without it or something
like it, the whole method is nonsense. I am following [37]. This theorem says that if you
know the Laplace transform, this will determine the function it came from at every point of
continuity of this function. The proof is fairly technical but only involves the theory of the
integral which was presented in this chapter.
Theorem 4.2.3 Let φ have exponential growth and have finitely many discontinuities on
every interval [0, R] and let f (s) ≡ L (φ ) (s). Then if t is a point of continuity of φ , it
follows that
   k+1
(−1)k (k) k

k
φ (t) = lim f .
k→∞ k! t t
Thus φ (t) is determined by its Laplace transform at every point of continuity.
Proof: First note that for k a positive integer, you can change the variable letting ku = t
and obtain
kk+1 ∞ −u
k kk+1 ∞ −t  t k 1
Z Z
e u du = e dt
k! 0 k! 0 k k
The details involve doing this on finite intervals using the theory of the Riemann integral
developed earlier and then passing to a limit. Thus the above equals
1 1 1
Z ∞
e−t t k dt = Γ (k + 1) = k! = 1
k! 0 k! k!
4.2. LAPLACE TRANSFORMS 49

To see this, use integration by parts.

Now assuming that |φ (u)| ≤ Ceλ u , then from what was just shown,

kk+1 kk+1 −u
k
Z ∞
k Z ∞
e−u u φ (u) du − φ (1) = e u (φ (u) − φ (1)) du
k! 0 0 k!
Assuming φ is continuous at 1, the improper integral is of the form
Z 1−δ k+1
k 1+δ kk+1 −u
k

k Z
e−u u (φ (u) − φ (1)) du + e u (φ (u) − φ (1)) du
0 k! 1−δ k!
Z ∞ k+1
k
k
+ e−u u (φ (u) − φ (1)) du
1+δ k!
Consider the first integral in the above. Letting K be an upper bound for

|φ (u) − φ (1)|

on [0, 1] ,
Z 1−δ k+1
k 1−δ kk+1 −u
k

k Z
e−u u (φ (u) − φ (1)) du ≤ K e u du
0 k! 0 k!

kk+1  −(1−δ ) k
≤K e (1 − δ ) (1 − δ )
k!
k+1 k
Now this converges to 0 as k → ∞. In fact, for a < 1, limk→∞ k k! (e−a a) = 0 because of
kk+1 k
the ratio test which shows that for a < 1, ∑k k! (e−a a) < ∞ which implies the kth term
converges to 0. Here a = 1 − δ . Next consider the last integral. This obviously converges
to 0 because of the exponential growth of φ . In fact,
Z ∞ k+1
k ∞ kk+1 −u
k 

k Z 
e−u u (φ (u) − φ (1)) du ≤ e u a + beλ u du
1+δ k! 1+δ k!

Now changing the variable letting uk = t, and doing everything on finite intervals followed
by passing to a limit, the absolute value of the above is dominated by

kk+1 −t  t k 1 
Z ∞ 
e a + beλ (t/k) dt
k(1+δ ) k! k k
1 −t k 
Z ∞ 
= e t a + beλ (t/k) dt for some a, b ≥ 0
k(1+δ ) k!

Z k(1+δ )
1 −t k  1 −t k 
Z ∞  
= e t a + beλ (t/k) dt − e t a + beλ (t/k) dt
0 k! 0 k!
However, the limit as k → ∞ of the integral on the right equals the improper integral on
the left. Thus this converges to 0 as k → ∞. Thus all that is left to consider is the middle
integral in which δ was chosen such that |φ (u) − φ (1)| < ε. Thus
Z 1+δ k+1
k ∞ kk+1 −u
k

k Z
e−u u (φ (u) − φ (1)) du ≤ ε e u du = ε
1−δ k! 0 k!
50 CHAPTER 4. SOME IMPORTANT IMPROPER INTEGRALS

It follows that if φ is continuous at 1,

Z ∞ k+1
k
k
lim e−u u (φ (u) − φ (1)) du = 0
k→∞ 0 k!
k+1 k
and so 0∞ k k! (e−u u) φ (u) du = φ (1). Now you simply replace φ (u) with φ (tu) where
R

φ is continuous at t. This function of u still has exponential growth and is continuous at

u = 1. Thus we obtain
Z ∞ k+1
k
k
lim e−u u φ (tu) du = φ (t)
k→∞ 0 k!
Now use Theorem 4.1.5 on Z ∞
f (s) ≡ e−st φ (t) dt
0

This theorem says that for large s, f (k) (s) exists and equals (−u)k e−su φ (u) du. Then
R∞
0
   k+1   k+1
(−1)k (k) k (−1)k
 Z ∞
k k
f = (−u)k e−(k/t)u φ (u) du
k! t t k! 0 t

Now letting v = ut , this reduces to

  k+1
(−1)k kk+1 ∞ −kv k
Z ∞ k
Z
k −kv
(− (tv)) e φ (tv)tdv = e v φ (tv) dv
k! 0 t k! 0

which was shown above to converge to φ (t). ■

 Part I

Linear Algebra And

Multivariable Calculus

51
Chapter 5

Fundamentals

5.1 Fn
The notation, Fn refers to the collection of ordered lists of n numbers. These numbers can
be either real or complex numbers. More precisely, consider the following [Link]
we mean real numbers, the symbol Rn is used. If nothing is specified, assume the symbol
means Cn .

Definition 5.1.1 Define

Fn ≡ (x1 , · · · , xn ) : x j ∈ F for j = 1, · · · , n .


(x1 , · · · , xn ) = (y1 , · · · , yn ) if and only if for all j = 1, · · · , n, x j = y j . When

(x1 , · · · , xn ) ∈ Rn ,

it is conventional to denote (x1 , · · · , xn ) by the single bold face letter x. The numbers x j are
called the coordinates. The set

{(0, · · · , 0,t, 0, · · · , 0) : t ∈ R }

for t in the ith slot is called the ith coordinate axis coordinate axis, the xi axis for short.
The point 0 ≡ (0, · · · , 0) is called the origin. Points in Rn are also called vectors.

Thus (1, 2, 4) ∈ R3 and (2, 1, 4) ∈ R3 but (1, 2, 4) ̸= (2, 1, 4) because, even though the
same numbers are involved, they don’t match up. In particular, the first entries are not
equal.
Why would anyone be interested in such a thing? First consider the case when n = 1.
Then from the definition, R1 = R. Recall that R is identified with the points of a line. Look
at the number line again. Observe that this amounts to identifying a point on this line with
a real number. In other words a real number determines where you are on this line. Now
suppose n = 2 and consider two lines which intersect each other at right angles as shown
in the following picture.

53
54 CHAPTER 5. FUNDAMENTALS

6 • (2, 6)
(−8, 3)
• 3
2
−8

Notice how you can identify a point shown in the plane with the ordered pair (2, 6) .
You go to the right a distance of 2 and then up a distance of 6. Similarly, you can identify
another point in the plane with the ordered pair (−8, 3) . Go to the left a distance of 8 and
then up a distance of 3. The reason you go to the left is that there is a − sign on the eight.
From this reasoning, every ordered pair determines a unique point in the plane. Conversely,
taking a point in the plane, you could draw two lines through the point, one vertical and the
other horizontal and determine unique points x1 on the horizontal line in the above picture
and x2 on the vertical line in the above picture, such that the point of interest is identified
with the ordered pair (x1 , x2 ) . In short, points in the plane can be identified with ordered
pairs similar to the way that points on the real line are identified with real numbers. Now
suppose n = 3. As just explained, the first two coordinates determine a point in a plane.
Letting the third component determine how far up or down you go, depending on whether
this number is positive or negative, this determines a point in space. Thus, (1, 4, −5) would
mean to determine the point in the plane that goes with (1, 4) and then to go below this
plane a distance of 5 to obtain a unique point in space. You see that the ordered triples
correspond to points in space just as the ordered pairs correspond to points in a plane and
single real numbers correspond to points on a line.
You can’t stop here and say that you are only interested in n ≤ 3. What if you were
interested in the motion of two objects? You would need three coordinates to describe
where the first object is and you would need another three coordinates to describe where
the other object is located. Therefore, you would need to be considering R6 . If the two
objects moved around, you would need a time coordinate as well. As another example,
consider a hot object which is cooling and suppose you want the temperature of this object.
How many coordinates would be needed? You would need one for the temperature, three
for the position of the point in the object and one more for the time. Thus you would need
to be considering R5 . Many other examples can be given. Sometimes n is very large. This
is often the case in applications to business when they are trying to maximize profit subject
to constraints. It also occurs in numerical analysis when people try to solve hard problems
on a computer.
There are other ways to identify points in space with three numbers but the one pre-
sented is the most basic. In this case, the coordinates are known as Cartesian coordinates
after Descartes1 who invented this idea in the first half of the seventeenth century. I will
often not bother to draw a distinction between the point in n dimensional space and its
Cartesian coordinates.
1 René Descartes 1596-1650 is often credited with inventing analytic geometry although it seems the ideas were

actually known much earlier. He was interested in many different subjects, physiology, chemistry, and physics
being some of them. He also wrote a large book in which he tried to explain the book of Genesis scientifically.
Descartes ended up dying in Sweden.
5.2. ALGEBRA IN Rn 55

5.2 Algebra in Rn
There are two algebraic operations done with points of Rn . One is addition and the other is
multiplication by numbers, called scalars. Yes, numbers =scalars.

Definition 5.2.1 If x ∈ Rn and a is a number, also called a scalar, then ax ∈ Rn is defined

by
ax = a (x1 , · · · , xn ) ≡ (ax1 , · · · , axn ) . (5.1)
This is known as scalar multiplication. If x, y ∈ Rn then x + y ∈ Rn and is defined by

x + y = (x1 , · · · , xn ) + (y1 , · · · , yn )
≡ (x1 + y1 , · · · , xn + yn ) (5.2)

An element of Rn x ≡ (x1 , · · · , xn ) is often called a vector. The above definition is known

as vector addition.

With this definition, the algebraic properties satisfy the conclusions of the following
theorem. The conclusions of this theorem are called the vector space axioms. There are
many other examples.

Theorem 5.2.2 For v, w vectors in Rn and α, β scalars, (real numbers), the following
hold.
v + w = w + v, (5.3)
the commutative law of addition,

(v + w) + z = v+ (w + z) , (5.4)

the associative law for addition,

v + 0 = v, (5.5)
the existence of an additive identity

v+ (−v) = 0, (5.6)

the existence of an additive inverse, Also

α (v + w) = α v+ αw, (5.7)

(α + β ) v = α v+ β v, (5.8)
α (β v) = αβ (v) , (5.9)
1v = v. (5.10)
In the above 0 = (0, · · · , 0).

You should verify these properties all hold. For example, consider 5.7.

α (v + w) = α (v1 + w1 , · · · , vn + wn ) = (α (v1 + w1 ) , · · · , α (vn + wn ))

= (αv1 + αw1 , · · · , αvn + αwn ) = (αv1 , · · · , αvn ) + (αw1 , · · · , αwn ) = αv + αw.

As usual, subtraction is defined as x − y ≡ x+ (−y) .

56 CHAPTER 5. FUNDAMENTALS

5.3 Geometric Meaning Of Vector Addition In R3

It was explained earlier that an element of Rn is an n tuple of numbers and it was also shown
that this can be used to determine a point in three dimensional space in the case where n = 3
and in two dimensional space, in the case where n = 2. This point was specified relative to
some coordinate axes.
Consider the case where n = 3 for now. If you draw an arrow from the point in three
dimensional space determined by (0, 0, 0) to the point (a, b, c) with its tail sitting at the
point (0, 0, 0) and its point at the point (a, b, c) , this arrow is called the position vector of
the point determined by u ≡ (a, b, c) . One way to get to this point is to start at (0, 0, 0)
and move in the direction of the x1 axis to (a, 0, 0) and then in the direction of the x2 axis
to (a, b, 0) and finally in the direction of the x3 axis to (a, b, c) . It is evident that the same
arrow (vector) would result if you began at the point v ≡ (d, e, f ) , moved in the direction
of the x1 axis to (d + a, e, f ) , then in the direction of the x2 axis to (d + a, e + b, f ) , and
finally in the x3 direction to (d + a, e + b, f + c) only this time, the arrow would have its
tail sitting at the point determined by v ≡ (d, e, f ) and its point at (d + a, e + b, f + c) . It
is said to be the same arrow (vector) because it will point in the same direction and have
the same length. It is like you took an actual arrow, the sort of thing you shoot with a bow,
and moved it from one location to another keeping it pointing the same direction. This is
illustrated in the following picture in which v + u is illustrated. Note the parallelogram
determined in the picture by the vectors u and v.

x3 v u+v

u
x2
x1

Thus the geometric significance of (d, e, f ) + (a, b, c) = (d + a, e + b, f + c) is this. You

start with the position vector of the point (d, e, f ) and at its point, you place the vector
determined by (a, b, c) with its tail at (d, e, f ) . Then the point of this last vector will be
(d + a, e + b, f + c) . This is the geometric significance of vector addition. Also, as shown
in the picture, u + v is the directed diagonal of the parallelogram determined by the two
vectors u and v.
The following example is art.

Example 5.3.1 Here is a picture of two vectors u and v.

v
Sketch a picture of u + v, u − v, and u+2v.
5.4. LINES 57

First here is a picture of u + v. You first draw u and then at the point of u you place the
tail of v as shown. Then u + v is the vector which results which is drawn in the following
pretty picture.

v
u
u+v
Next consider u − v. This means u+ (−v) . From the above geometric description of
vector addition, −v is the vector which has the same length but which points in the opposite
direction to v. Here is a picture.
−v

u + (−v)
u

Finally consider the vector u+2v. Here is a picture of this one also.

2v
u
u + 2v

5.4 Lines
To begin with consider the case n = 1, 2. In the case where n = 1, the only line is just
R1 = R. Therefore, if x1 and x2 are two different points in R, consider

x = x1 + t (x2 − x1 )

where t ∈ R and the totality of all such points will give R. You see that you can always
solve the above equation for t, showing that every point on R is of this form. Now consider
the plane. Does a similar formula hold? Let (x1 , y1 ) and (x2 , y2 ) be two different points in
R2 which are contained in a line l. Suppose that x1 ̸= x2 . Then if (x, y) is an arbitrary point
on l,

(x, y)
(x2 , y2 )

(x1 , y1 )
Now by similar triangles,
y2 − y1 y − y1
m≡ =
x2 − x1 x − x1
58 CHAPTER 5. FUNDAMENTALS

and so the point slope form of the line, l, is given as

y − y1 = m (x − x1 ) .

If t is defined by
x = x1 + t (x2 − x1 ) ,
you obtain this equation along with

y = y1 + mt (x2 − x1 )
= y1 + t (y2 − y1 ) .

Therefore,
(x, y) = (x1 , y1 ) + t (x2 − x1 , y2 − y1 ) .
If x1 = x2 , then in place of the point slope form above, x = x1 . Since the two given points
are different, y1 ̸= y2 and so you still obtain the above formula for the line. Because of this,
the following is the definition of a line in Rn .

Definition 5.4.1 A line in Rn containing the two different points x1 and x2 is the collection
of points of the form
x = x1 + t x2 − x1

where t ∈ R. This is known as a parametric equation and the variable t is called the
parameter.

Often t denotes time in applications to Physics. Note this definition agrees with the
usual notion of a line in two dimensions and so this is consistent with earlier concepts.

Lemma 5.4.2 Let a, b ∈ Rn with a ̸= 0. Then x = ta + b ∈ R, is a line.

Proof: Let x1 = b and let x2 −x1 = a so that x2 ̸= x1 . Then ta + b = x1 +t x2 − x1

and so x = ta + b is a line containing the two different points x1 and x2 . ■

Definition 5.4.3 The vector a in the above lemma is called a direction vector for the line.

Definition 5.4.4 Let p and q be two points in Rn , p ̸= q. The directed line segment from
p to q, denoted by −
→ is defined to be the collection of points
pq,

x = p + t (q − p) , t ∈ [0, 1]

with the direction corresponding to increasing t. In the definition, when t = 0, the point p is
obtained and as t increases other points on this line segment are obtained until when t = 1,
you get the point q. This is what is meant by saying the direction corresponds to increasing
t.

Think of −→ as an arrow whose point is on q and whose base is at p as shown in the

pq
following picture.
q

p
This line segment is a part of a line from the above Definition.
5.4. LINES 59

Example 5.4.5 Find a parametric equation for the line through the points (1, 2, 0) and
(2, −4, 6) .

Use the definition of a line given above to write

(x, y, z) = (1, 2, 0) + t (1, −6, 6) , t ∈ R.

The vector (1, −6, 6) is obtained by (2, −4, 6) − (1, 2, 0) as indicated above.
The reason for the word, “a”, rather than the word, “the” is there are infinitely many
different parametric equations for the same line. To see this replace t with 3s. Then you
obtain a parametric equation for the same line because the same set of points is obtained.
The difference is they are obtained from different values of the parameter. What happens is
this: The line is a set of points but the parametric description gives more information than
that. It tells how the points are obtained. Obviously, there are many ways to trace out a
given set of points and each of these ways corresponds to a different parametric equation
for the line.

Example 5.4.6 Find a parametric equation for the line which contains the point (1, 2, 0)
and has direction vector (1, 2, 1) .

From the above this is just

(x, y, z) = (1, 2, 0) + t (1, 2, 1) , t ∈ R. (5.11)

Sometimes people elect to write a line like the above in the form

x = 1 + t, y = 2 + 2t, z = t, t ∈ R. (5.12)

This is a set of scalar parametric equations which amounts to the same thing as 5.11.
There is one other form for a line which is sometimes considered useful. It is the so
called symmetric form. Consider the line of 5.12. You can solve for the parameter t to
write
y−2
t = x − 1,t = , t = z.
2
Therefore,
y−2
x−1 = = z.
2
This is the symmetric form of the line.

Example 5.4.7 Suppose the symmetric form of a line is

x−2 y−1
= = z + 3.
3 2
Find the line in parametric form.
x−2 y−1
Let t = 3 ,t = 2 and t = z + 3. Then solving for x, y, z, you get

x = 3t + 2, y = 2t + 1, z = t − 3, t ∈ R.

Written in terms of vectors this is

(2, 1, −3) + t (3, 2, 1) = (x, y, z) , t ∈ R.

60 CHAPTER 5. FUNDAMENTALS

5.5 Distance in Rn
How is distance between two points in Rn defined?

Definition 5.5.1 Let x = (x1 , · · · , xn ) and y = (y1 , · · · , yn ) be two points in Rn . Then

|x − y| to indicates the distance between these points and is defined as
!1/2
n
2
distance between x and y ≡ |x − y| ≡ ∑ |xk − yk | .
k=1

This is called the distance formula. Thus |x| ≡ |x − 0| . The symbol B (a, r) is defined by

B (a, r) ≡ {x ∈ Rn : |x − a| < r} .

This is called an open ball of radius r centered at a. It gives all the points in Rn which are
closer to a than r.

First of all note this is a generalization of the notion of distance in R. There the distance
between two points x and y was given by the absolute value of their difference. Thus |x − y|
 1/2
is equal to the distance between these two points on R. Now |x − y| = (x − y)2 where
the square root is always the positive square root. Thus it is the same formula as the above
definition except there is only one term in the sum. Geometrically, this is the right way
to define distance which is seen from the Pythagorean theorem. Consider the following
picture in the case that n = 2.

(y1 , y2 )

(x1 , x2 ) (y1 , x2 )
There are two points in the plane whose Cartesian coordinates are (x1 , x2 ) and (y1 , y2 )
respectively. Then the solid line joining these two points is the hypotenuse of a right triangle
which is half of the rectangle shown in dotted lines. What is its length? Note the lengths
of the sides of this triangle are |y1 − x1 | and |y2 − x2 | . Therefore, the Pythagorean theorem
implies the length of the hypotenuse equals
 1/2  1/2
|y1 − x1 |2 + |y2 − x2 |2 = (y1 − x1 )2 + (y2 − x2 )2

which is just the formula for the distance given above.

Now suppose n = 3 and let (x1 , x2 , x3 ) and (y1 , y2 , y3 ) be two points in R3 . Consider the
following picture in which one of the solid lines joins the two points and a dotted line joins
the points (x1 , x2 , x3 ) and (y1 , y2 , x3 ) .
5.5. DISTANCE IN Rn 61

(y1 , y2 , y3 )

(y1 , y2 , x3 )

(x1 , x2 , x3 ) (y1 , x2 , x3 )
By the Pythagorean theorem, the length of the dotted line joining the following two
points (x1 , x2 , x3 ) and (y1 , y2 , x3 ) equals
 1/2
(y1 − x1 )2 + (y2 − x2 )2

while the length of the line joining (y1 , y2 , x3 ) to (y1 , y2 , y3 ) is just |y3 − x3 | . Therefore,
by the Pythagorean theorem again, the length of the line joining the points (x1 , x2 , x3 ) and
(y1 , y2 , y3 ) equals
( )1/2
 1/2 2
2 2 2
(y1 − x1 ) + (y2 − x2 ) + (y3 − x3 )
 1/2
= (y1 − x1 )2 + (y2 − x2 )2 + (y3 − x3 )2 ,

which is again just the distance formula above.

This completes the argument that the above definition is reasonable. Of course you
cannot continue drawing pictures in ever higher dimensions but there is no problem with
the formula for distance in any number of dimensions. Here is an example.

Example 5.5.2 Find the distance between the points in R4 ,

a = (1, 2, −4, 6) , b = (2, 3, −1, 0)

Use the distance formula and write

|a − b|2 = (1 − 2)2 + (2 − 3)2 + (−4 − (−1))2 + (6 − 0)2 = 47

√
Therefore, |a − b| = 47.
All this amounts to defining the distance between two points as the length of a straight
line joining these two points. However, there is nothing sacred about using straight lines.
One could define the distance to be the length of some other sort of line joining these points.
It won’t be done in this book but sometimes this sort of thing is done.
Another convention which is usually followed, especially in R2 and R3 is to denote the
first component of a point in R2 by x and the second component by y. In R3 it is customary
to denote the first and second components as just described while the third component is
called z.

Example 5.5.3 Describe the points which are at the same distance between (1, 2, 3) and
(0, 1, 2) .
62 CHAPTER 5. FUNDAMENTALS

Let (x, y, z) be such a point. Then

q q
(x − 1)2 + (y − 2)2 + (z − 3)2 = x2 + (y − 1)2 + (z − 2)2 .

Squaring both sides

(x − 1)2 + (y − 2)2 + (z − 3)2 = x2 + (y − 1)2 + (z − 2)2

and so
x2 − 2x + 14 + y2 − 4y + z2 − 6z = x2 + y2 − 2y + 5 + z2 − 4z
which implies
−2x + 14 − 4y − 6z = −2y + 5 − 4z
hence
2x + 2y + 2z = −9. (5.13)
Since these steps are reversible, the set of points which is at the same distance from the two
given points consists of the points (x, y, z) such that 5.13 holds.
The following lemma is fundamental. It is a form of the Cauchy Schwarz inequality.

Lemma 5.5.4 Let x = (x1 , · · · , xn ) and y = (y1 , · · · , yn ) be two points in Rn . Then

n
∑ xi yi ≤ |x| |y| . (5.14)
i=1

Proof: Let θ be either 1 or −1 such that

n n n
θ ∑ xi yi = ∑ xi (θ yi ) = ∑ xi yi
i=1 i=1 i=1

and consider p (t) ≡ ∑ni=1 (xi + tθ yi )2 . Then for all t ∈ R,

n n n
0 ≤ p (t) = ∑ xi2 + 2t ∑ xi θ yi + t 2 ∑ y2i
i=1 i=1 i=1
n
= |x|2 + 2t ∑ xi θ yi + t 2 |y|2
i=1

If |y| = 0 then 5.14 is obviously true because both sides equal zero. Therefore, assume
|y| ̸= 0 and then p (t) is a polynomial of degree two whose graph opens up. Therefore, it
either has no zeroes, two zeros or one repeated zero. If it has two zeros, the above inequality
must be violated because in this case the graph must dip below the x axis. Therefore, it
either has no zeros or exactly one. From the quadratic formula this happens exactly when
!2
n
4 ∑ xi θ yi − 4 |x|2 |y|2 ≤ 0
i=1

and so
n n
∑ xi θ yi = ∑ xi yi ≤ |x| |y|
i=1 i=1
5.6. GEOMETRIC MEANING OF SCALAR MULTIPLICATION IN R3 63

as claimed. This proves the inequality. ■

There are certain properties of the distance which are obvious. Two of them which
follow directly from the definition are
|x − y| = |y − x| ,
|x − y| ≥ 0 and equals 0 only if y = x.
The third fundamental property of distance is known as the triangle inequality. Recall that
in any triangle the sum of the lengths of two sides is always at least as large as the third
side. The following corollary is equivalent to this simple statement.
Corollary 5.5.5 Let x, y be points of Rn . Then
|x + y| ≤ |x| + |y| .
Proof: Using the Cauchy Schwarz inequality, Lemma 5.5.4,
n
|x + y|2 ≡ ∑ (xi + yi )2
i=1
n n n
= ∑ xi2 + 2 ∑ xi yi + ∑ y2i
i=1 i=1 i=1
2 2
≤ |x| + 2 |x| |y| + |y|
= (|x| + |y|)2
and so upon taking square roots of both sides,
|x + y| ≤ |x| + |y|
■

5.6 Geometric Meaning Of Scalar Multiplication In R3

As discussed earlier, x = (x1 , x2 , x3 ) determines a vector. You draw the line from 0 to
x placing the point of the vector on x. What is the length of this vector? The length
of this vector is defined to equal |x| as in Definition 5.5.1. Thus the length of x equals
q
x12 + x22 + x32 . When you multiply x by a scalar α, you get (αx1 , αx2 , αx3 ) and the length
of this vector is defined as
r  q
(αx1 )2 + (αx2 )2 + (αx3 )2 = |α| x12 + x22 + x32 .

Thus the following holds.

|αx| = |α| |x| .
In other words, multiplication by a scalar magnifies the length of the vector. What about
the direction? You should convince yourself by drawing a picture that if α is negative, it
causes the resulting vector to point in the opposite direction while if α > 0 it preserves the
direction the vector points. One way to see this is to first observe that if α ̸= 1, then x and
αx are both points on the same line. Note that there is no change in this when you replace
R3 with Rn .
64 CHAPTER 5. FUNDAMENTALS

5.7 Exercises
1. Verify all the properties 5.3-5.10.
2. Compute the following

(a) 5 (1, 2, 3, −2) + 6 (2, 1, −2, 7)

(b) 5 (1, 2, −2) − 6 (2, 1, −2)
(c) −3 (1, 0, 3, −2) + (2, 0, −2, 1)
(d) −3 (1, −2, −3, −2) − 2 (2, −1, −2, 7)
(e) − (2, 2, −3, −2) + 2 (2, 4, −2, 7)

3. Find symmetric equations for the line through the points (2, 2, 4) and (−2, 3, 1) .
4. Find symmetric equations for the line through the points (1, 2, 4) and (−2, 1, 1) .
5. Symmetric equations for a line are given. Find parametric equations of the line.
x+1 2y+3
(a) 3 = 2 = z+7
2x−1 2y+3
(b) 3 = 6 = z−7
x+1
(c) 3 = 2y + 3 = 2z − 1
1−2x 3−2y
(d) 3 = 2 = z+1
x−1 2y−3
(e) 3 = 5 = z+2
x+1 3−y
(f) 3 = 5 = z+1

6. Parametric equations for a line are given. Find symmetric equations for the line if
possible. If it is not possible to do it explain why.

(a) x = 1 + 2t, y = 3 − t, z = 5 + 3t
(b) x = 1 + t, y = 3 − t, z = 5 − 3t
(c) x = 1 + 2t, y = 3 + t, z = 5 + 3t
(d) x = 1 − 2t, y = 1, z = 1 + t
(e) x = 1 − t, y = 3 + 2t, z = 5 − 3t
(f) x = t, y = 3 − t, z = 1 + t

7. The first point given is a point contained in the line. The second point given is a
direction vector for the line. Find parametric equations for the line, determined by
this information.

(a) (1, 2, 1) , (2, 0, 3)

(b) (1, 0, 1) , (1, 1, 3)
(c) (1, 2, 0) , (1, 1, 0)
(d) (1, 0, −6) , (−2, −1, 3)
(e) (−1, −2, −1) , (2, 1, −1)
5.7. EXERCISES 65

(f) (0, 0, 0) , (2, −3, 1)

8. Parametric equations for a line are given. Determine a direction vector for this line.

(a) x = 1 + 2t, y = 3 − t, z = 5 + 3t
(b) x = 1 + t, y = 3 + 3t, z = 5 − t
(c) x = 7 + t, y = 3 + 4t, z = 5 − 3t
(d) x = 2t, y = −3t, z = 3t
(e) x = 2t, y = 3 + 2t, z = 5 + t
(f) x = t, y = 3 + 3t, z = 5 + t

9. A line contains the given two points. Find parametric equations for this line. Identify
the direction vector.

(a) (0, 1, 0) , (2, 1, 2)

(b) (0, 1, 1) , (2, 5, 0)
(c) (1, 1, 0) , (0, 1, 2)
(d) (0, 1, 3) , (0, 3, 0)
(e) (0, 1, 0) , (0, 6, 2)
(f) (0, 1, 2) , (2, 0, 2)

10. Draw a picture of the points in R2 which are determined by the following ordered
pairs.

(a) (1, 2)
(b) (−2, −2)
(c) (−2, 3)
(d) (2, −5)

11. Does it make sense to write (1, 2) + (2, 3, 1)? Explain.

12. Draw a picture of the points in R3 which are determined by the following ordered
triples.

(a) (1, 2, 0)
(b) (−2, −2, 1)
(c) (−2, 3, −2)

13. You are given two points in R3 , (4, 5, −4) and (2, 3, 0) . Show the distance from the
point (3, 4, −2) to the first of these points is the same as the distance from this point
to the second of the original pair of points. Note that 3 = 4+2 5+3
2 , 4 = 2 . Obtain a
theorem which will be valid for general pairs of points (x, y, z) and (x1 , y1 , z1 ) and
prove your theorem using the distance formula.
14. A sphere is the set of all points which are at a given distance from a single given
point. Find an equation for the sphere which is the set of all points that are at a
distance of 4 from the point (1, 2, 3) in R3 .
66 CHAPTER 5. FUNDAMENTALS

15. A parabola is the set of all points (x, y) in the plane such that the distance from the
point (x, y) to a given point (x0 , y0 ) equals the distance from (x, y) to a given line.
The point (x0 , y0 ) is called the focus and the line is called the directrix. Find the
equation of the parabola which results from the line y = l and (x0 , y0 ) a given focus
with y0 < l. Repeat for y0 > l.

16. A sphere centered at the point (x0 , y0 , z0 ) ∈ R3 having radius r consists of all points
(x, y, z) whose distance to (x0 , y0 , z0 ) equals r. Write an equation for this sphere in
R3 .
17. Suppose the distance between (x, y) and (x′ , y′ ) were defined to equal the larger of
the two numbers |x − x′ | and |y − y′ | . Draw a picture of the sphere centered at the
point (0, 0) if this notion of distance is used.
18. Repeat the same problem except this time let the distance between the two points be
|x − x′ | + |y − y′ | .

19. If (x1 , y1 , z1 ) and (x2 , y2 , z2 ) are two points such that |(x
i , yi , zi )| = 1 for i = 1, 2, show
2 y1 +y2 z1 +z2
that in terms of the usual distance, x1 +x 2 , 2 , 2 < 1. What would happen if
you used the way of measuring distance given in Problem 17 (|(x, y, z)| = maximum
of |z| , |x| , |y| .)?
20. Give a simple description using the distance formula of the set of points which are at
an equal distance between the two points (x1 , y1 , z1 ) and (x2 , y2 , z2 ) .

21. Suppose you are given two points (−a, 0) and (a, 0) in R2 and a number r > 2a. The
set of points described by

(x, y) ∈ R2 : |(x, y) − (−a, 0)|



+ |(x, y) − (a, 0)| = r}

is known as an ellipse. The two given points are known as the focus points of the

2  2
ellipse. Find α and β such that this is in the form αx + βy = 1. This is a nice
exercise in messy algebra.
22. Suppose you are given two points (−a, 0) and (a, 0) in R2 and a number r < 2a. The
set of points described by

(x, y) ∈ R2 : |(x, y) − (−a, 0)|



− |(x, y) − (a, 0)| = r}

is known as hyperbola. The two given points are known as the focus points of the

2  2
hyperbola. Simplify this to the form αx − βy = 1. This is a nice exercise in
messy algebra.
23. Let (x1 , y1 ) and (x2 , y2 ) be two points in R2 . Give a simple description using the
distance formula of the perpendicular bisector of the line segment joining these two
points. Thus you want all points (x, y) such that |(x, y) − (x1 , y1 )| = |(x, y) − (x2 , y2 )| .
24. Show that |αx| =|α||x| whenever x ∈ Rn for any positive integer n.
5.8. PHYSICAL VECTORS 67

5.8 Physical Vectors

Suppose you push on something. What is important? There are really two things which are
important, how hard you push and the direction you push.

Definition 5.8.1 Force is a vector. The magnitude of this vector is a measure of how hard
it is pushing. It is measured in units such as Newtons or pounds or tons. Its direction is the
direction in which the push is taking place.

Of course this is a little vague and will be left a little vague until the presentation of
Newton’s second law later. See the appendix on this or any physics book.
Vectors are used to model force and other physical vectors like velocity. What was just
described would be called a force vector. It has two essential ingredients, its magnitude
and its direction. Think of vectors as directed line segments or arrows as shown in the
following picture in which all the directed line segments are considered to be the same
vector because they have the same direction, the direction in which the arrows point, and
the same magnitude (length).

Because of this fact that only direction and magnitude are important, it is always possi-
ble to put a vector in a certain particularly simple form. Let −
→ be a directed line segment
pq
−→
or vector. Then from Definition 5.4.4 it follows that pq consists of the points of the form

p + t (q − p)

where t ∈ [0, 1] . Subtract p from all these points to obtain the directed line segment con-
sisting of the points
0 + t (q − p) , t ∈ [0, 1] .
The point in Rn , q − p, will represent the vector.
Geometrically, the arrow − → was slid so it points in the same direction and the base is
pq,
at the origin 0. For example, see the following picture.

In this way vectors can be identified with points of Rn .

Definition 5.8.2 Let x = (x1 , · · · , xn ) ∈ Rn . The position vector of this point is the vector
whose point is at x and whose tail is at the origin (0, · · · , 0). If x = (x1 , · · · , xn ) is called
68 CHAPTER 5. FUNDAMENTALS

a vector, the vector which is meant, is this position vector just described. Another term
associated with this is standard position. A vector is in standard position if the tail is
placed at the origin.

It is customary to identify the point in Rn with its position vector.

The magnitude of a vector determined by a directed line segment − → is just the distance
pq
between the point p and the point q. By the distance formula this equals
!1/2
n
2
∑ (qk − pk ) = |p − q|
k=1

1/2
and for v any vector in Rn the magnitude of v equals ∑nk=1 v2k = |v|.

Example 5.8.3 Consider the vector v ≡ (1, 2, 3) in Rn . Find |v| .

First, the vector is the directed line segment (arrow) which has its base at 0 ≡ (0, 0, 0)
and its point at (1, 2, 3) . Therefore,
p √
|v| = 12 + 22 + 32 = 14.

What is the geometric significance of scalar multiplication? If a represents the vector

v in the sense that when it is slid to place its tail at the origin, the element of Rn at its point
is a, what is rv?
!1/2 !1/2
n n
2 2
|rv| = ∑ (rai ) = ∑ r2 (ai )
k=1 k=1
!1/2
n
2 1/2
∑ a2i


= r = |r| |v| .
k=1

Thus the magnitude of rv equals |r| times the magnitude of v. If r is positive, then the
vector represented by rv has the same direction as the vector v because multiplying by the
scalar r, only has the effect of scaling all the distances. Thus the unit distance along any
coordinate axis now has length r and in this re-scaled system the vector is represented by
a. If r < 0 similar considerations apply except in this case all the ai also change sign. From
now on, a will be referred to as a vector instead of an element of Rn representing a vector
as just described. The following picture illustrates the effect of scalar multiplication.

v 2v −2v

Note there are n special vectors which point along the coordinate axes. These are

ei ≡ (0, · · · , 0, 1, 0, · · · , 0)
5.8. PHYSICAL VECTORS 69

where the 1 is in the ith slot and there are zeros in all the other spaces. See the picture in
the case of R3 .
z

e3

e1 e2 y

The direction of ei is referred to as the ith direction. Given a vector v = (a1 , · · · , an ) ,

ai ei is the ith component of the vector. Thus ai ei = (0, · · · , 0, ai , 0, · · · , 0) and so this vector
gives something possibly nonzero only in the ith direction. Also, knowledge of the ith
component of the vector is equivalent to knowledge of the vector because it gives the entry
in the ith slot and for v = (a1 , · · · , an ) , v = ∑nk=1 ai ei .
What does addition of vectors mean physically? Suppose two forces are applied to
some object. Each of these would be represented by a force vector and the two forces
acting together would yield an overall force acting on the object which would also be
a force vector known as the resultant. Suppose the two vectors are a = ∑nk=1 ai ei and
b = ∑nk=1 bi ei . Then the vector a involves a component in the ith direction, ai ei while the
component in the ith direction of b is bi ei . Then it seems physically reasonable that the
resultant vector should have a component in the ith direction equal to (ai + bi ) ei . This is
exactly what is obtained when the vectors a and b are added.
n
a + b = (a1 + b1 , · · · , an + bn ) = ∑ (ai + bi ) ei
i=1

Thus the addition of vectors according to the rules of addition in Rn which were pre-
sented earlier, yields the appropriate vector which duplicates the cumulative effect of all
the vectors in the sum.
What is the geometric significance of vector addition? Suppose u, v are vectors

u = (u1 , · · · , un ) , v = (v1 , · · · , vn )

Then u + v = (u1 + v1 , · · · , un + vn ) . How can one obtain this geometrically? Consider the
−→
directed line segment, 0u and then, starting at the end of this directed line segment, follow
−−−−−−→
the directed line segment u (u + v) to its end u + v. In other words, place the vector u in
standard position with its base at the origin and then slide the vector v till its base coincides
with the point of u. The point of this slid vector, determines u + v. To illustrate, see the
following picture
70 CHAPTER 5. FUNDAMENTALS

u u+v

Note the vector u + v is the diagonal of a parallelogram determined from the two vec-
tors u and v and that identifying u + v with the directed diagonal of the parallelogram
determined by the vectors u and v amounts to the same thing as the above procedure.
An item of notation should be mentioned here. In the case of Rn where n ≤ 3, it is
standard notation to use i for e1 , j for e2 , and k for e3 . Now here are some applications of
vector addition to some problems.

Example 5.8.4 There are three ropes attached to a car and three people pull on these
ropes. The first exerts a force of 2 i+ 3 j− 2k Newtons, the second exerts a force of 3 i+
5j + k Newtons and the third exerts a force of 5 i − j+ 2k. Newtons. Find the total force
in the direction of i.

To find the total force add the vectors as described above. This gives 10 i+ 7j + k
Newtons. Therefore, the force in the i direction is 10 Newtons.
As mentioned earlier, the Newton is a unit of force like pounds.

Example 5.8.5 An airplane flies North East at 100 miles per hour. Write this as a vector.

A picture of this situation follows.

The vector has length 100. Now using that vector as the√ hypotenuse of a right triangle
having equal
√ sides, the
√ sides should be each of length 100/ 2. Therefore, the vector would
be 100/ 2i + 100/ 2j.
This example also motivates the concept of velocity.

Definition 5.8.6 The speed of an object is a measure of how fast it is going. It is measured
in units of length per unit time. For example, miles per hour, kilometers per minute, feet
per second. The velocity is a vector having the speed as the magnitude but also specifying
the direction.
√ √
Thus the velocity vector in the above example is 100/ 2i + 100/ 2j.

Example 5.8.7 The velocity of an airplane is 100i+j +k measured in kilometers per hour
and at a certain instant of time its position is (1, 2, 1) . Here imagine a Cartesian coordinate
5.9. EXERCISES 71

system in which the third component is altitude and the first and second components are
measured on a line from West to East and a line from South to North. Find the position of
this airplane one minute later.
Consider the vector (1, 2, 1) , is the initial position vector of the airplane. As it moves,
the position vector changes. After one minute the airplane has moved in the i direction a
1
distance of 100 × 60 = 53 kilometer. In the j direction it has moved 60 1
kilometer during this
1
same time, while it moves 60 kilometer in the k direction. Therefore, the new displacement
vector for the airplane is
   
5 1 1 8 121 121
(1, 2, 1) + , , = , ,
3 60 60 3 60 60
Example 5.8.8 A certain river is one half mile wide with a current flowing at 4 miles per
hour from East to West. A man swims directly toward the opposite shore from the South
bank of the river at a speed of 3 miles per hour. How far down the river does he find himself
when he has swam across? How far does he end up swimming?
Consider the following picture.

3
4

You should write these vectors in terms of components. The velocity of the swimmer in
still water would be 3j while the velocity of the river would be −4i. Therefore, the velocity
of the swimmer is −4i + 3j. Since the component of velocity in the direction across the
√ is 3, it follows the trip takes 1/6 hour or 10 minutes.
river The speed at which he travels is
42 + 32 = 5 miles per hour and so he travels 5 × 16 = 56 miles. Now to find the distance
downstream he finds himself, note that if x is this distance, x and 1/2 are two legs of a right
triangle whose hypotenuse equals 5/6 miles. Therefore, by the Pythagorean theorem the
distance downstream is
2
q
(5/6)2 − (1/2)2 = miles.
3

5.9 Exercises
1. The wind blows from the South at 40 kilometers per hour and an airplane which
travels at 400 kilometers per hour in still air is heading East. Find the actual velocity
of the airplane.
2. ↑In the above problem, find the position of the airplane after two hours.
3. ↑In the above problem, if the airplane is to travel due east, in what direction should
it head in order to achieve this?
4. The wind blows from West to East at a speed of 50 miles per hour and an airplane
which travels at 300 miles per hour in still air is heading North West. What is the
velocity of the airplane relative to the ground? What is the component of this velocity
in the direction North?
72 CHAPTER 5. FUNDAMENTALS

5. In the situation of Problem 4 how many degrees to the West of North should the
airplane head in order to fly exactly North. What will be the speed of the airplane
relative to the ground?
6. In the situation of 5 suppose the airplane uses 34 gallons of fuel every hour at that air
speed and that it needs to fly North a distance of 600 miles. Will the airplane have
enough fuel to arrive at its destination given that it has 63 gallons of fuel?
7. An airplane is flying due north at 150 miles per hour. A wind is pushing the airplane
due east at 40 miles per hour. After 1 hour, the plane starts flying 30◦ East of North.
Assuming the plane starts at (0, 0) , where is it after 2 hours? Let North be the
direction of the positive y axis and let East be the direction of the positive x axis.
8. City A is located at the origin while city B is located at (300, 500) where distances
are in miles. An airplane flies at 250 miles per hour in still air. This airplane wants
to fly from city A to city B but the wind is blowing in the direction of the positive y
axis at a speed of 50 miles per hour. Find a unit vector such that if the plane heads
in this direction, it will end up at city B having flown the shortest possible distance.
How long will it take to get there?
9. A certain river is one half mile wide with a current flowing at 2 miles per hour from
East to West. A man swims directly toward the opposite shore from the South bank
of the river at a speed of 3 miles per hour. How far down the river does he find
himself when he has swam across? How far does he end up swimming?
10. A certain river is one half mile wide with a current flowing at 2 miles per hour from
East to West. A man can swim at 3 miles per hour in still water. In what direction
should he swim in order to travel directly across the river? What would the answer to
this problem be if the river flowed at 3 miles per hour and the man could swim only
at the rate of 2 miles per hour?

11. Three forces are applied to a point which does not move. Two of the forces are
2i + j + 3k Newtons and i − 3j + 2k Newtons. Find the third force.
12. Three forces are applied to a point which does not move. Two of the forces are
i + j + 3k Newtons and i − 3j − 2k Newtons. Find the third force.

13. The total force acting on an object is to be 2i + j + k Newtons. A force of −i + j + k

Newtons is being applied. What other force should be applied to achieve the desired
total force?
14. The total force acting on an object is to be i + j + 3k Newtons. A force of −i − j + k
Newtons is being applied. What other force should be applied to achieve the desired
total force?
15. A bird flies from its nest 5 km. in the direction 60◦ north of east where it stops to
rest on a tree. It then flies 10 km. in the direction due southeast and lands atop a
telephone pole. Place an xy coordinate system so that the origin is the bird’s nest,
and the positive x axis points east and the positive y axis points north. Find the
displacement vector from the nest to the telephone pole.
5.9. EXERCISES 73

16. A car is stuck in the mud. There is a cable stretched tightly from this car to a tree
which is 20 feet long. A person grasps the cable in the middle and pulls with a force
of 100 pounds perpendicular to the stretched cable. The center of the cable moves
two feet and remains still. What is the tension in the cable? The tension in the cable
is the force exerted on this point by the part of the cable nearer the car as well as the
force exerted on this point by the part of the cable nearer the tree.
74 CHAPTER 5. FUNDAMENTALS
Chapter 6

Vector Products

6.1 The Dot Product

There are two ways of multiplying vectors which are of great importance in applications.
The first of these is called the dot product, also called the scalar product and sometimes
the inner product.

Definition 6.1.1 Let a, b be two vectors in Rn define a · b as

n
a·b ≡ ∑ ak bk .
k=1

With this definition, there are several important properties satisfied by the dot product.
In the statement of these properties, α and β will denote scalars and a, b, c will denote
vectors.

Proposition 6.1.2 The dot product satisfies the following properties.

a·b = b·a (6.1)

a · a ≥ 0 and equals zero if and only if a = 0 (6.2)

(αa + β b) · c = α (a · c) + β (b · c) (6.3)
c · (αa + β b) = α (c · a) + β (c · b) (6.4)
2
|a| = a · a (6.5)

You should verify these properties. Also be sure you understand that 6.4 follows from
the first three and is therefore redundant. It is listed here for the sake of convenience.

Example 6.1.3 Find (1, 2, 0, −1) · (0, 1, 2, 3) .

This equals 0 + 2 + 0 + −3 = −1.

Example 6.1.4 Find the magnitude of a = (2, 1, 4, 2) . That is, find |a| .

75
76 CHAPTER 6. VECTOR PRODUCTS
p
This is (2, 1, 4, 2) · (2, 1, 4, 2) = 5.
The dot product satisfies a fundamental inequality known as the Cauchy Schwarz in-
equality. It has already been proved but here is another proof. This proof will be based
only on the above axioms for the dot product.

Theorem 6.1.5 The dot product satisfies the inequality

|a · b| ≤ |a| |b| . (6.6)

Furthermore equality is obtained if and only if one of a or b is a scalar multiple of the

other.

Proof: First note that if b = 0, both sides of 6.6 equal zero and so the inequality holds
in this case. Indeed,
a · 0 = a· (0 + 0) = a · 0 + a · 0
so a · 0 = 0. Therefore, it will be assumed in what follows that b ̸= 0.
Define a function of t ∈ R

f (t) = (a + tb) · (a + tb) .

Then by 6.2, f (t) ≥ 0 for all t ∈ R. Also from 6.3,6.4,6.1, and 6.5

f (t) = a · (a + tb) + tb · (a + tb)

= a · a + t (a · b) + tb · a + t 2 b · b
= |a|2 + 2t (a · b) + |b|2 t 2 .

Now
!
2 2 a·b |a|2
f (t) = |b| t + 2t +
|b|2 |b|2
 !2 !2 
2
a·ba·b a·b |a|
= |b|2 t 2 + 2t+ − + 2
|b|2 |b|2 |b|2 |b|
 !2  !2 
2
2 a·b |a| a · b
= |b| t+ 2 + 2 −  ≥ 0
|b| |b| |b|2
 
for all t ∈ R. In particular f (t) ≥ 0 when t = − a · b/ |b|2 which implies
!2
|a|2 a·b
− ≥ 0. (6.7)
|b|2 |b|2

Multiplying both sides by |b|4 ,

|a|2 |b|2 ≥ (a · b)2
which yields 6.6.
From Theorem 6.1.5, equality holds in 6.6 whenever one of the vectors is a scalar
multiple of the other. It only remains to verify this is the only way equality can occur.
6.1. THE DOT PRODUCT 77

If either vector equals zero, then equality is obtained in 6.6 so it can be assumed both
 and that equality is obtained in 6.7. This implies that f (t) = 0 when
vectors are non zero
2
t = − a · b/ |b| and so from 6.2, it follows that for this value of t, a+tb = 0 showing
a = −tb. ■
You should note that the entire argument was based only on the properties of the dot
product listed in 6.1 - 6.5. This means that whenever something satisfies these properties,
the Cauchy Schwartz inequality holds. There are many other instances of these properties
besides vectors in Rn .
The Cauchy Schwartz inequality allows a proof of the triangle inequality for distances
in Rn in much the same way as the triangle inequality for the absolute value.

Theorem 6.1.6 (Triangle inequality) For a, b ∈ Rn

|a + b| ≤ |a| + |b| (6.8)

and equality holds if and only if one of the vectors is a nonnegative scalar multiple of the
other. Also
||a| − |b|| ≤ |a − b| (6.9)

Proof: By properties of the dot product and the Cauchy Schwarz inequality,

|a + b|2 = (a + b) · (a + b) = (a · a) + (a · b) + (b · a) + (b · b)
= |a|2 + 2 (a · b) + |b|2 ≤ |a|2 + 2 |a · b| + |b|2
≤ |a|2 + 2 |a| |b| + |b|2 = (|a| + |b|)2 .

Taking square roots of both sides you obtain 6.8.

It remains to consider when equality occurs. If either vector equals zero, then that vec-
tor equals zero times the other vector and the claim about when equality occurs is verified.
Therefore, it can be assumed both vectors are nonzero. To get equality in the second in-
equality above, Theorem 6.1.5 implies one of the vectors must be a multiple of the other.
Say b = αa. If α < 0 then equality cannot occur in the first inequality because in this case

(a · b) = α |a|2 < 0 < |α| |a|2 = |a · b|

Therefore, α ≥ 0.
To get the other form of the triangle inequality, a = a − b + b so

|a| = |a − b + b| ≤ |a − b| + |b| .

Therefore,
|a| − |b| ≤ |a − b| (6.10)
Similarly,
|b| − |a| ≤ |b − a| = |a − b| . (6.11)
It follows from 6.10 and 6.11 that 6.9 holds. This is because ||a| − |b|| equals the left side
of either 6.10 or 6.11 and either way, ||a| − |b|| ≤ |a − b| . ■
78 CHAPTER 6. VECTOR PRODUCTS

6.2 The Geometric Significance Of The Dot Product

6.2.1 The Angle Between Two Vectors
Given two vectors a and b, the included angle is the angle between these two vectors which
is less than or equal to 180 degrees. The dot product can be used to determine the included
angle between two vectors. To see how to do this, consider the following picture.
b
θ a
a−b

By the law of cosines,

|a − b|2 = |a|2 + |b|2 − 2 |a| |b| cos θ .

Also from the properties of the dot product,

|a − b|2 = (a − b) · (a − b) = |a|2 + |b|2 − 2a · b

and so comparing the above two formulas,

a · b = |a| |b| cos θ . (6.12)

In words, the dot product of two vectors equals the product of the magnitude of the two vec-
tors multiplied by the cosine of the included angle. Note this gives a geometric description
of the dot product which does not depend explicitly on the coordinates of the vectors.

Example 6.2.1 Find the angle between the vectors 2i + j − k and 3i + 4j + k.

The dot product of these two vectors equals 6 + 4 − 1 = 9 and the norms are
√ √
4+1+1 = 6
√ √
and 9 + 16 + 1 = 26. Therefore, from 6.12 the cosine of the included angle equals
9
cos θ = √ √ = . 720 58
26 6
Now the cosine is known, the angle can be determines by solving the equation cos θ = .
720 58. This will involve using a calculator or a table of trigonometric functions. The an-
swer is θ = . 766 16 radians or in terms of degrees, θ = . 766 16 × 360 ◦
2π = 43. 898 . Recall
how this last computation is done. Set up a proportion .76616 = 2π because 360◦ corre-
x 360

sponds to 2π radians. However, in calculus, you should get used to thinking in terms of
radians and not degrees. This is because all the important calculus formulas are defined in
terms of radians.

Example 6.2.2 Let u, v be two vectors whose magnitudes are equal to 3 and 4 respectively
and such that if they are placed in standard position with their tails at the origin, the angle
between u and the positive x axis equals 30◦ and the angle between v and the positive x
axis is -30◦ . Find u · v.
6.2. THE GEOMETRIC SIGNIFICANCE OF THE DOT PRODUCT 79

From the geometric description of the dot product in 6.12

u · v = 3 × 4 × cos (60◦ ) = 3 × 4 × 1/2 = 6.

Observation 6.2.3 Two vectors are said to be perpendicular if the included angle is π/2
radians (90◦ ). You can tell if two nonzero vectors are perpendicular by simply taking their
dot product. If the answer is zero, this means they are perpendicular because cos θ = 0.

Example 6.2.4 Determine whether the two vectors 2i + j − k and 1i + 3j + 5k are per-
pendicular.

When you take this dot product you get 2 + 3 − 5 = 0 and so these two are indeed
perpendicular.

Definition 6.2.5 When two lines intersect, the angle between the two lines is the smaller
of the two angles determined.

Example 6.2.6 Find the angle between the two lines, (1, 2, 0) + t (1, 2, 3) and (0, 4, −3) +
t (−1, 2, −3) .

These two lines intersect, when t = 0 in the first and t = −1 in the second. It is only
a matter of finding the angle between the direction vectors. One angle determined is given
by
−6 −3
cos θ = = . (6.13)
14 7
We don’t want this angle because it is obtuse. The angle desired is the acute angle given by
3
cos θ = .
7
It is obtained by using replacing one of the direction vectors with −1 times it.

6.2.2 Work And Projections

Our first application will be to the concept of work. The physical concept of work does
not in any way correspond to the notion of work employed in ordinary conversation. For
example, if you were to slide a 150 pound weight off a table which is three feet high and
shuffle along the floor for 50 yards, sweating profusely and exerting all your strength to
keep the weight from falling on your feet, keeping the height always three feet and then
deposit this weight on another three foot high table, the physical concept of work would
indicate that the force exerted by your arms did no work during this project even though the
muscles in your hands and arms would likely be very tired. The reason for such an unusual
definition is that even though your arms exerted considerable force on the weight, enough
to keep it from falling, the direction of motion was at right angles to the force they exerted.
The only part of a force which does work in the sense of physics is the component of the
force in the direction of motion (This is made more precise below.). The work is defined
to be the magnitude of the component of this force times the distance over which it acts in
the case where this component of force points in the direction of motion and (−1) times
the magnitude of this component times the distance in case the force tends to impede the
motion. Thus the work done by a force on an object as the object moves from one point
80 CHAPTER 6. VECTOR PRODUCTS

to another is a measure of the extent to which the force contributes to the motion. This
is illustrated in the following picture in the case where the given force contributes to the
motion.

F p2
F⊥
θ
F ||
p1

In this picture the force, F is applied to an object which moves on the straight line from
p1 to p2 . There are two vectors shown, F || and F ⊥ and the picture is intended to indicate
that when you add these two vectors you get F while F || acts in the direction of motion
and F ⊥ acts perpendicular to the direction of motion. Only F || contributes to the work
done by F on the object as it moves from p1 to p2 . F || is called the component of the
force in the direction of motion. From trigonometry, you see the magnitude of F || should
equal |F | |cos θ | . Thus, since F || points in the direction of the vector from p1 to p2 , the
total work done should equal

|F | −
p− →
1 p2 cos θ = |F | |p2 − p1 | cos θ

If the included angle had been obtuse, then the work done by the force, F on the object
would have been negative because in this case, the force tends to impede the motion from
p1 to p2 but in this case, cos θ would also be negative and so it is still the case that the work
done would be given by the above formula. Thus from the geometric description of the dot
product given above, the work equals

|F | |p2 − p1 | cos θ = F · (p2 −p1 ) .

This explains the following definition.

Definition 6.2.7 Let F be a force acting on an object which moves from the point p1 to the
point p2 . Then the work done on the object by the given force equals F · (p2 − p1 ) .

The concept of writing a given vector F in terms of two vectors, one which is parallel
to a given vector D and the other which is perpendicular can also be explained with no
reliance on trigonometry, completely in terms of the algebraic properties of the dot product.
As before, this is mathematically more significant than any approach involving geometry
or trigonometry because it extends to more interesting situations. This is done next.

Theorem 6.2.8 Let F and D be nonzero vectors. Then there exist unique vectors F || and
F ⊥ such that
F = F || + F ⊥ (6.14)
where F || is a scalar multiple of D, also referred to as

projD (F ) ,

and F ⊥ · D = 0. The vector projD (F ) is called the projection of F onto D.

6.2. THE GEOMETRIC SIGNIFICANCE OF THE DOT PRODUCT 81

Proof: Suppose 6.14 and F || = αD. Taking the dot product of both sides with D and
using F ⊥ · D = 0, this yields
F · D = α |D|2
which requires α = F · D/ |D|2 . Thus there can be no more than one vector F || . It follows
F ⊥ must equal F − F || . This verifies there can be no more than one choice for both F ||
and F ⊥ .
Now let
F ·D
F || ≡ D
|D|2
and let
F ·D
F ⊥ = F − F || = F − D
|D|2
F ·D
Then F || = α D where α = . It only remains to verify F ⊥ · D = 0. But
|D |2

F ·D
F ⊥ · D = F · D− D · D = F · D − F · D = 0.
|D|2

Example 6.2.9 Let F = 2 i+ 7j − 3k Newtons. Find the work done by this force in moving
from the point (1, 2, 3) to the point (−9, −3, 4) along the straight line segment joining these
points where distances are measured in meters.

According to the definition, this work is

(2 i+ 7j − 3k) · (−10i − 5j + k) = −20 + (−35) + (−3) = −58 Newton meters.

Note that if the force had been given in pounds and the distance had been given in feet,
the units on the work would have been foot pounds. In general, work has units equal to
units of a force times units of a length. Instead of writing Newton meter, people write joule
because a joule is by definition a Newton meter. That word is pronounced “jewel” and it is
the unit of work in the metric system of units. Also be sure you observe that the work done
by the force can be negative as in the above example. In fact, work can be either positive,
negative, or zero. You just have to do the computations to find out.

Example 6.2.10 Find proju (v) if u = 2i + 3j − 4k and v = i − 2j + k.

From the above discussion in Theorem 6.2.8, this is just

1
(i − 2j + k) · (2i + 3j − 4k) (2i + 3j − 4k)
4 + 9 + 16
−8 16 24 32
= (2i + 3j − 4k) = − i − j + k.
29 29 29 29
Example 6.2.11 Suppose a, and b are vectors and b⊥ = b − proja (b) . What is the mag-
nitude of b⊥ in terms of the included angle?
82 CHAPTER 6. VECTOR PRODUCTS

! !
2 b·a b·a
|b⊥ | = (b − proja (b)) · (b − proja (b)) = b− a · b− a
|a|2 |a|2
!2 !
2 (b · a)2 b·a 2 2 (b · a)2
= |b| − 2 + |a| = |b| 1−
|a|2 |a|2 |a|2 |b|2
= |b|2 1 − cos2 θ = |b|2 sin2 (θ )

where θ is the included angle between a and b which is less than π radians. Therefore,
taking square roots, |b⊥ | = |b| sin θ .

6.2.3 The Dot Product And Distance In Cn

It is necessary to give a generalization of the dot product for vectors in Cn . This definition
reduces to the usual one in the case the components of the vector are real.
Definition 6.2.12 Let x, y ∈ Cn . Thus x = (x1 , · · · , xn ) where each xk ∈ C and a similar
formula holding for y. Then the dot product of these two vectors is defined to be
x · y ≡ ∑ x j y j ≡ x1 y1 + · · · + xn yn .
j

Notice how you put the conjugate on the entries of the vector y. It makes no difference
if the vectors happen to be real vectors but with complex vectors you must do it this way.
The reason for this is that when you take the dot product of a vector with itself, you want
to get the square of the length of the vector, a positive number. Placing the conjugate on
the components of y in the above definition assures this will take place. Thus
2
x · x = ∑ x jx j = ∑ x j ≥ 0.
j j

If you didn’t place a conjugate as in the above definition, things wouldn’t work out cor-
rectly. For example,
(1 + i)2 + 22 = 4 + 2i
and this is not a positive number.
The following properties of the dot product follow immediately from the definition and
you should verify each of them.
Properties of the dot product:
1. u · v = v · u.
2. If a, b are numbers and u, v, z are vectors then (au + bv) · z = a (u · z) + b (v · z) .
3. u · u ≥ 0 and it equals 0 if and only if u = 0.
The norm is defined in the usual way.
Definition 6.2.13 For x ∈ Cn ,
!1/2
n
|x| ≡ ∑ |xk | 2
= (x · x)1/2
k=1
6.2. THE GEOMETRIC SIGNIFICANCE OF THE DOT PRODUCT 83

As in the case of Rn , the Cauchy Schwarz inequality is of fundamental importance.

First here is a simple lemma.

Lemma 6.2.14 If z ∈ C there exists θ ∈ C such that θ z = |z| and |θ | = 1.

z
Proof: Let θ = 1 if z = 0 and otherwise, let θ = . Recall that for z = x + iy, z = x − iy
|z|
and zz = |z|2 . ■

Theorem 6.2.15 (Cauchy Schwarz)The following inequality holds for xi and yi ∈ C.

!1/2 !1/2
n n n
2 2
|(x · y)| = ∑ xi yi ≤ ∑ |xi | ∑ |yi | = |x| |y| (6.15)
i=1 i=1 i=1

Proof: Let θ ∈ C such that |θ | = 1 and θ (x · y) = |x · y| . Then from the properties of

the dot product,

x + t θ̄ y · x + t θ̄ y = (x · x) + t 2 θ̄ θ (y · y) + t x · θ̄ y + t θ̄ y · x

= (x · x) + t 2 (y · y) + tθ (x · y) + t x·θ̄ y

= (x · x) + t 2 (y · y) + tθ (x · y) + tθ (x · y)
= |x|2 + 2t |x · y| + t 2 |y|2 ≥ 0

If |y| = 0, this can only happen if x · y = 0 and so the inequality holds. If |y| ̸= 0, then you
have a parabola which opens up and has at most one real zero. Therefore, by the quadratic
formula,
4 |x · y|2 − 4 |x|2 |y|2 ≤ 0
which yields the Cauchy Schwarz inequality. ■
By analogy to the case of Rn , length or magnitude of vectors in Cn can be defined.

Definition 6.2.16 Let z ∈ Cn . Then |z| ≡ (z · z)1/2 .

Theorem 6.2.17 For length defined in Definition 6.2.16, the following hold.

|z| ≥ 0 and |z| = 0 if and only if z = 0 (6.16)

If α is a scalar, |αz| = |α| |z| (6.17)

|z + w| ≤ |z| + |w| . (6.18)

Proof: The first two claims are left as exercises. To establish the third, you use the
same argument which was used in Rn .

|z + w|2 = (z + w, z + w) = z · z + w · w + w · z + z · w
= |z|2 + |w|2 + 2 Re w · z ≤ |z|2 + |w|2 + 2 |w · z|
≤ |z|2 + |w|2 + 2 |w| |z| = (|z| + |w|)2 .

All other considerations such as open and closed sets and the like are identical in this more
general context with the corresponding definition in Rn . The main difference is that here
the scalars are complex numbers. ■
84 CHAPTER 6. VECTOR PRODUCTS

Definition 6.2.18 Suppose you have a vector space, V and for z, w ∈ V and α a scalar
a norm is a way of measuring distance or magnitude which satisfies the properties 6.16 -
6.18. Thus a norm is something which does the following.

||z|| ≥ 0 and ||z|| = 0 if and only if z = 0 (6.19)

If α is a scalar, ||αz|| = |α| ||z|| (6.20)

||z + w|| ≤ ||z|| + ||w|| . (6.21)
Here is understood that for all z ∈ V, ||z|| ∈ [0, ∞).

6.3 Exercises
1. Find (1, 2, 3, 4) · (2, 0, 1, 3) .
2. Use formula 6.12 to verify the Cauchy Schwartz inequality and to show that equality
occurs if and only if one of the vectors is a scalar multiple of the other.
3. For u, v vectors in R3 , define the product u ∗ v ≡ u1 v1 + 2u2 v2 + 3u3 v3 . Show the
axioms for a dot product all hold for this funny product. Prove the Cauchy Schwarz
inequality |u ∗ v| ≤ (u ∗ u)1/2 (v ∗ v)1/2 . Hint: Do not try to do this with methods
from trigonometry.
4. Find the angle between the vectors 3i − j − k and i + 4j + 2k.
5. Find the angle between the vectors i − 2j + k and i + 2j − 7k.

6. Find proju (v) where v = (1, 0, −2) and u = (1, 2, 3) .

7. Find proju (v) where v = (1, 2, −2) and u = (1, 0, 3) .
8. Find proju (v) where v = (1, 2, −2, 1) and u = (1, 2, 3, 0) .
9. Does it make sense to speak of proj0 (v)?

10. If F is a force and D is a vector, show projD (F ) = (|F | cos θ ) u where u is the unit
vector in the direction of D, u = D/ |D| and θ is the included angle between the
two vectors F and D. |F | cos θ is sometimes called the component of the force, F
in the direction, D.

11. A boy drags a sled for 100 feet along the ground by pulling on a rope which is 20
degrees from the horizontal with a force of 40 pounds. How much work does this
force do?
12. A girl drags a sled for 200 feet along the ground by pulling on a rope which is 30
degrees from the horizontal with a force of 20 pounds. How much work does this
force do?
13. A large dog drags a sled for 300 feet along the ground by pulling on a rope which is
45 degrees from the horizontal with a force of 20 pounds. How much work does this
force do?
6.4. THE CROSS PRODUCT 85

14. How much work in Newton meters does it take to slide a crate 20 meters along a
loading dock by pulling on it with a 200 Newton force at an angle of 30◦ from the
horizontal?
15. An object moves 10 meters in the direction of j. There are two forces acting on this
object F 1 = i + j + 2k, and F 2 = −5i + 2 j− 6k. Find the total work done on the
object by the two forces. Hint: You can take the work done by the resultant of the
two forces or you can add the work done by each force. Why?
16. An object moves 10 meters in the direction of j + i. There are two forces acting on
this object F 1 = i + 2j + 2k, and F 2 = 5i + 2 j− 6k. Find the total work done on
the object by the two forces. Hint: You can take the work done by the resultant of
the two forces or you can add the work done by each force. Why?
17. An object moves 20 meters in the direction of k + j. There are two forces acting on
this object F 1 = i + j + 2k, and F 2 = i + 2 j− 6k. Find the total work done on the
object by the two forces. Hint: You can take the work done by the resultant of the
two forces or you can add the work done by each force.
18. If a, b, and c are vectors. Show that (b + c)⊥ = b⊥ + c⊥ where b⊥ = b− proja (b) .
19. In the discussion of the reflecting mirror which directs all rays to a particular point
(0, p) . Show that for any choice of positive C this point is the focus of the parabola
and the directrix is y = p − C1 .
20. Suppose you wanted to make a solar powered oven to cook food. Are there reasons
for using a mirror which is not parabolic? Also describe how you would design a
good flash light with a beam which does not spread out too quickly.
h i
21. Show that (a · b) = 14 |a + b|2 − |a − b|2 .

22. Prove from the axioms of the dot product the parallelogram identity which is the
following: |a + b|2 + |a − b|2 = 2 |a|2 + 2 |b|2 .
23. Suppose f , g are two continuous functions defined on [0, 1] . Define the inner product
( f · g) = 01 f (x) g (x) dx. Show this dot product satisfies conditions 6.1 - 6.5. Explain
R

why the Cauchy Schwarz inequality continues to hold in this context and state the
Cauchy Schwarz inequality in terms of integrals.

6.4 The Cross Product

The cross product is the other way of multiplying two vectors in R3 . It is very different
from the dot product in many ways. First the geometric meaning is discussed and then
a description in terms of coordinates is given. Both descriptions of the cross product are
important. The geometric description is essential in order to understand the applications
to physics and geometry while the coordinate description is the only way to practically
compute the cross product.

Definition 6.4.1 Three vectors a, b, c form a right handed system if when you extend the
fingers of your right hand along the vector a and close them in the direction of b, the thumb
points roughly in the direction of c.
86 CHAPTER 6. VECTOR PRODUCTS

For an example of a right handed system of vectors, see the following picture.

In this picture the vector c points upwards from the plane determined by the other
two vectors. You should consider how a right hand system would differ from a left hand
system. Try using your left hand and you will see that the vector c would need to point in
the opposite direction as it would for a right hand system.
From now on, the vectors i, j, k will always form a right handed system. To repeat,
if you extend the fingers of our right hand along i and close them in the direction j, the
thumb points in the direction of k.

j
i
The following is the geometric description of the cross product. It gives both the direc-
tion and the magnitude and therefore specifies the vector.

Definition 6.4.2 Let a and b be two vectors in R3 . Then a × b is defined by the following
two rules.

1. |a × b| = |a| |b| sin θ where θ is the included angle.

2. a × b · a = 0, a × b · b = 0, and a, b, a × b forms a right hand system.

Note that |a × b| is the area of the parallelogram spanned by a and b.

|b|sin(θ )
b

θ a

The cross product satisfies the following properties.

a × b = − (b × a) , a × a = 0, (6.22)
6.4. THE CROSS PRODUCT 87

For α a scalar,
(αa) ×b = α (a × b) = a× (αb) , (6.23)
For a, b, and c vectors, one obtains the distributive laws,

a× (b + c) = a × b + a × c, (6.24)

(b + c) × a = b × a + c × a. (6.25)
Formula 6.22 follows immediately from the definition. The vectors a × b and b × a
have the same magnitude, |a| |b| sin θ , and an application of the right hand rule shows they
have opposite direction. Formula 6.23 is also fairly clear. If α is a nonnegative scalar, the
direction of (αa) ×b is the same as the direction of a × b,α (a × b) and a× (αb) while
the magnitude is just α times the magnitude of a × b which is the same as the magnitude
of α (a × b) and a× (αb) . Using this yields equality in 6.23. In the case where α < 0,
everything works the same way except the vectors are all pointing in the opposite direction
and you must multiply by |α| when comparing their magnitudes. The distributive laws are
much harder to establish but the second follows from the first quite easily. Thus, assuming
the first, and using 6.22,

(b + c) × a = −a× (b + c) = − (a × b + a × c) = b × a + c × a.

A proof of the distributive law is given in a later section for those who are interested.
Now from the definition of the cross product,

i × j = k, j × i = −k
k × i = j, i × k = −j
j × k = i, k × j = −i

With this information, the following gives the coordinate description of the cross product.

Proposition 6.4.3 Let a = a1 i + a2 j + a3 k and b = b1 i + b2 j + b3 k be two vectors. Then

a × b = (a2 b3 − a3 b2 ) i+ (a3 b1 − a1 b3 ) j+ (a1 b2 − a2 b1 ) k. (6.26)

Proof: From the above table and the properties of the cross product listed,

(a1 i + a2 j + a3 k) × (b1 i + b2 j + b3 k) =

a1 b2 i × j + a1 b3 i × k + a2 b1 j × i + a2 b3 j × k+
+a3 b1 k × i + a3 b2 k × j
= a1 b2 k − a1 b3 j − a2 b1 k + a2 b3 i + a3 b1 j − a3 b2 i
= (a2 b3 − a3 b2 ) i+ (a3 b1 − a1 b3 ) j+ (a1 b2 − a2 b1 ) k (6.27)
■
It is probably impossible for most people to remember 6.26. Fortunately, there is a
somewhat easier way to remember it.

i j k
a×b = a1 a2 a3 (6.28)
b1 b2 b3
88 CHAPTER 6. VECTOR PRODUCTS

where you expand the determinant along the top row. This yields

(a2 b3 − a3 b2 ) i− (a1 b3 − a3 b1 ) j+ (a1 b2 − a2 b1 ) k (6.29)

which is the same as 6.27. If you have not seen determinants, it doesn’t matter all you need
here is how to evaluate 2 × 2 and 3 × 3 determinants. First consider 2 × 2 determinants.

x y
= xw − yz
z w

and
a b c
y z x z x y
x y z =a −b +c .
v w u w u v
u v w
Here is the rule: You look at an entry in the top row and cross out the row and column
which contain that entry. If the entry is in the ith column, you multiply (−1)1+i times the
determinant of the 2 × 2 which remains. This is the cofactor. You take the element in the
top row times this cofactor and add all such terms. The rectangular array enclosed by the
vertical lines is called a matrix and will be discussed more later.

Example 6.4.4 Find (i − j + 2k) × (3i − 2j + k) .

Use 6.28 to compute this.

i j k
−1 2 1 2 1 −1
1 −1 2 = i− j+ k
−2 1 3 1 3 −2
3 −2 1
= 3i + 5j + k.

Example 6.4.5 Find the area of the parallelogram determined by the vectors

(i − j + 2k) , (3i − 2j + k) .

These are the same two vectors in Example 6.4.4.

From Example 6.4.4 and the geometric description of the cross √

product, the area
√is just
the norm of the vector obtained in Example 6.4.4. Thus the area is 9 + 25 + 1 = 35.

Example 6.4.6 Find the area of the triangle determined by (1, 2, 3) , (0, 2, 5) , and (5, 1, 2) .

This triangle is obtained by connecting the three points with lines. Picking (1, 2, 3) as
a starting point, there are two displacement vectors (−1, 0, 2) and (4, −1, −1) such that the
given vector added to these displacement vectors gives the other two vectors. The area of
the triangle is half the area of the parallelogram determined by (−1, 0, 2) and√ (4, −1, −1) .
Thus (−1, 0, 2) × (4, −1, −1) = (2, 7, 1) and so the area of the triangle is 21 4 + 49 + 1 =
3
√
2 6.

Observation 6.4.7 In general, if you have three points (vectors) in R3 , P, Q, R the area
of the triangle is given by
1
|(Q − P ) × (R − P )| .
2
6.4. THE CROSS PRODUCT 89

P R

6.4.1 The Box Product

Definition 6.4.8 A parallelepiped determined by the three vectors a, b, and c consists of
{r a+ sb + tc : r, s,t ∈ [0, 1]} .
That is, if you pick three numbers, r, s, and t each in [0, 1] and form r a+ sb + tc, then the
collection of all such points is what is meant by the parallelepiped determined by these
three vectors.
The following is a picture of such a thing.

a×b

c
θ b
a

You notice the area of the base of the parallelepiped, the parallelogram determined by
the vectors a and b has area equal to |a × b| while the altitude of the parallelepiped is
|c| cos θ where θ is the angle shown in the picture between c and a × b. Therefore, the
volume of this parallelepiped is the area of the base times the altitude which is just
|a × b| |c| cos θ = a × b · c.
This expression is known as the box product and is sometimes written as [a, b, c] . You
should consider what happens if you interchange the b with the c or the a with the c. You
can see geometrically from drawing pictures that this merely introduces a minus sign. In
any case the box product of three vectors always equals either the volume of the paral-
lelepiped determined by the three vectors or else minus this volume.
Example 6.4.9 Find the volume of the parallelepiped determined by the vectors i + 2j −
5k, i + 3j − 6k,3i + 2j + 3k.
According to the above discussion, pick any two of these, take the cross product and
then take the dot product of this with the third of these vectors. The result will be either the
desired volume or minus the desired volume.
i j k
(i + 2j − 5k) × (i + 3j − 6k) = 1 2 −5 = 3i + j + k
1 3 −6
90 CHAPTER 6. VECTOR PRODUCTS

Now take the dot product of this vector with the third which yields

(3i + j + k) · (3i + 2j + 3k) = 9 + 2 + 3 = 14.

This shows the volume of this parallelepiped is 14 cubic units.

There is a fundamental observation which comes directly from the geometric definitions
of the cross product and the dot product.

Lemma 6.4.10 Let a, b, and c be vectors. Then (a × b) ·c = a· (b × c) .

Proof: This follows from observing that either (a × b) ·c and a· (b × c) both give the
volume of the parallelepiped or they both give −1 times the volume. ■

6.5 Proof of the distributive law

Here is another proof of the distributive law for the cross product. Let x be a vector. From
the above observation,

x · a× (b + c) = (x × a) · (b + c) = (x × a) · b+ (x × a) · c
= x · a × b + x · a × c = x· (a × b + a × c) .

Therefore,
x· [a× (b + c) − (a × b + a × c)] = 0
for all x. In particular, this holds for x = a× (b + c) − (a × b + a × c) and this shows that
the following holds: a× (b + c) = a × b + a × c and this proves the distributive law for
the cross product another way.

Observation 6.5.1 Suppose you have three vectors, u = (a, b, c) , v = (d, e, f ) , and w =
(g, h, i) . Then u · v × w is given by the following.

i j k
u·v×w = (a, b, c) · d e f
g h i
e f d f d e
= a −b +c
h i g i g h
 
a b c
= det  d e f .
 

g h i

The message is that to take the box product, you can simply take the determinant of the
matrix which results by letting the rows be the rectangular components of the given vectors
in the order in which they occur in the box product.
6.5. PROOF OF THE DISTRIBUTIVE LAW 91

6.5.1 Torque
Imagine you are using a wrench to loosen a nut. The idea is to turn the nut by applying
a force to the end of the wrench. If you push or pull the wrench directly toward or away
from the nut, it should be obvious from experience that no progress will be made in turning
the nut. The important thing is the component of force perpendicular to the wrench. It is
this component of force which will cause the nut to turn. For example see the following
picture.

F
F⊥ θ

R
F
θ

In the picture a force, F is applied at the end of a wrench represented by the posi-
tion vector R and the angle between these two is θ . Then the tendency to turn will be
|R| |F ⊥ | = |R| |F | sin θ , which you recognize as the magnitude of the cross product of R
and F . If there were just one force acting at one point whose position vector is R, perhaps
this would be sufficient, but what if there are numerous forces acting at many different
points with neither the position vectors nor the force vectors in the same plane; what then?
To keep track of this sort of thing, define for each R and F, the torque vector
τ ≡ R×F.
This is also called the moment of the force, F . That way, if there are several forces acting
at several points the total torque can be obtained by simply adding up the torques associated
with the different forces and positions.
Example 6.5.2 Suppose R1 = 2i − j+3k, R2 = i+2 j− 6k meters and at the points de-
termined by these vectors there are forces, F 1 = i − j+2k and F 2 = i − 5j + k Newtons
respectively. Find the total torque about the origin produced by these forces acting at the
given points.
It is necessary to take R1 × F 1 + R2 × F 2 . Thus the total torque equals
i j k i j k
2 −1 3 + 1 2 −6 = −27i − 8j − 8k Newton meters
1 −1 2 1 −5 1
Example 6.5.3 Find if possible a single force vector F which if applied at the point
i + j + k will produce the same torque as the above two forces acting at the given points.
This is fairly routine. The problem is to find F = F1 i + F2 j + F3 k which produces the
above torque vector. Therefore,
i j k
1 1 1 = −27i − 8j − 8k
F1 F2 F3
92 CHAPTER 6. VECTOR PRODUCTS

which reduces to (F3 − F2 ) i+ (F1 − F3 ) j+ (F2 − F1 ) k = −27i − 8j − 8k. This amounts

to solving the system of three equations in three unknowns, F1 , F2 , and F3 ,

F3 − F2 = −27, F1 − F3 = −8, F2 − F1 = −8

However, there is no solution to these three equations. (Why?) Therefore no single force
acting at the point i + j + k will produce the given torque.

6.5.2 Center Of Mass

The mass of an object is a measure of how much stuff there is in the object. An object has
mass equal to one kilogram, a unit of mass in the metric system, if it would exactly balance
a known one kilogram object when placed on a balance. The known object is one kilogram
by definition. The mass of an object does not depend on where the balance is used. It
would be one kilogram on the moon as well as on the earth. The weight of an object is
something else. It is the force exerted on the object by gravity and has magnitude gm
where g is a constant called the acceleration of gravity. Thus the weight of a one kilogram
object would be different on the moon which has much less gravity, smaller g, than on the
earth. An important idea is that of the center of mass. This is the point at which an object
will balance no matter how it is turned.

Definition 6.5.4 Let an object consist of p point masses m1 , · · · , m p with the position of the
kth of these at Rk . The center of mass of this object R0 is the point satisfying
p
∑ (Rk − R0 ) × gmk u = 0
k=1

for all unit vectors u.

The above definition indicates that no matter how the object is suspended, the total
torque on it due to gravity is such that no rotation occurs. Using the properties of the cross
product !
p p
∑ Rk gmk − R0 ∑ gmk ×u = 0 (6.30)
k=1 k=1

for any choice of unit vector u. You should verify that if a × u = 0 for all u, then it must
be the case that a = 0. Then the above formula requires that
p p
∑ Rk gmk − R0 ∑ gmk = 0.
k=1 k=1

p
dividing by g, and then by ∑k=1 mk ,
p
∑k=1 Rk mk
R0 = p . (6.31)
∑k=1 mk

This is the formula for the center of mass of a collection of point masses. To consider
the center of mass of a solid consisting of continuously distributed masses, you need the
methods of calculus.
6.5. PROOF OF THE DISTRIBUTIVE LAW 93

Example 6.5.5 Let m1 = 5, m2 = 6, and m3 = 3 where the masses are in kilograms. Sup-
pose m1 is located at 2i + 3j + k, m2 is located at i − 3j + 2k and m3 is located at
2i − j + 3k. Find the center of mass of these three masses.

Using 6.31
5 (2i + 3j + k) + 6 (i − 3j + 2k) + 3 (2i − j + 3k) 11 3 13
R0 = = i− j + k
5+6+3 7 7 7

6.5.3 Angular Velocity

Definition 6.5.6 In a rotating body, a vector Ω is called an angular velocity vector if the
velocity of a point having position vector u relative to the body is given by Ω × u.

The existence of an angular velocity vector is the key to understanding motion in a

moving system of coordinates. It is used to explain the motion on the surface of the ro-
tating earth. For example, have you ever wondered why low pressure areas rotate counter
clockwise in the upper hemisphere but clockwise in the lower hemisphere? To quantify
these things, you will need the concept of an angular velocity vector. Details are presented
later for interesting examples. Here is a simple example. In the above example, think of a
coordinate system fixed in the rotating body. Thus if you were riding on the rotating body,
you would observe this coordinate system as fixed but it is not fixed.

Example 6.5.7 A wheel rotates counter clockwise about the vector i + j + k at 60 revo-
lutions per minute. This means that if the thumb of your right hand were to point in the
direction of i + j + k your fingers of this hand would wrap in the direction of rotation.
Find the angular velocity vector for this wheel. Assume the unit of distance is meters and
the unit of time is minutes.

Let ω = 60 × 2π = 120π. This is the number of radians per minute corresponding to

60 revolutions per minute. Then the angular velocity vector is 120π √ (i + j + k) . Note this
3
gives what you would expect in the case the position vector to the point is perpendicular to
i + j + k and at a distance of r. This is because of the geometric description of the cross
product. The magnitude of the vector is r120π meters per minute and corresponds to the
speed and an exercise with the right hand shows the direction is correct also. However, if
this body is rigid, this will work for every other point in it, even those for which the position
vector is not perpendicular to the given vector. A complete analysis of this is given later.

Example 6.5.8 A wheel rotates counter clockwise about the vector i + j + k at 60 rev-
olutions per minute exactly as in Example 6.5.7. Let {u1 , u2 , u3 } denote an orthogonal
right handed system attached to the rotating wheel in which u3 = √13 (i + j + k) . Thus
u1 and u2 depend on time. Find the velocity of the point of the wheel located at the point
2u1 + 3u2 − u3 . Note this point is not fixed in space. It is moving.

Since {u1 , u2 , u3 } is a right handed system like i, j, k, everything applies to this sys-
tem in the same way as with i, j, k. Thus the cross product is given by

u1 u2 u3
(au1 + bu2 + cu3 ) × (du1 + eu2 + f u3 ) = a b c
d e f
94 CHAPTER 6. VECTOR PRODUCTS

Therefore, in terms of the given vectors ui , the angular velocity vector is 120πu3 . The
velocity of the given point is

u1 u2 u3
0 0 120π = −360πu1 + 240πu2
2 3 −1

in meters per minute. Note how this gives the answer in terms of these vectors which are
fixed in the body, not in space. Since ui depends on t, this shows the answer in this case
does also. Of course this is right. Just think of what is going on with the wheel rotating.
Those vectors which are fixed in the wheel are moving in space. The velocity of a point in
the wheel should be constantly changing. However, its speed will not change. The speed
will be the magnitude of the velocity and this is
p
(−360πu1 + 240πu2 ) · (−360πu1 + 240πu2 )

which from the properties of the dot product equals

q √
(−360π)2 + (240π)2 = 120 13π

because the ui are given to be orthogonal.

6.6 Vector Identities And Notation

To begin with consider u × (v × w) and it is desired to simplify this quantity. It turns
out this is an important quantity which comes up in many different contexts. Let u =
(u1 , u2 , u3 ) and let v and w be defined similarly.

i j k
v×w = v1 v2 v3 = (v2 w3 − v3 w2 ) i+ (w1 v3 − v1 w3 ) j+ (v1 w2 − v2 w1 ) k
w1 w2 w3

Next consider u× (v × w) which is given by

i j k
u× (v × w) = u1 u2 u3 .
(v2 w3 − v3 w2 ) (w1 v3 − v1 w3 ) (v1 w2 − v2 w1 )

When you multiply this out, you get

i (v1 u2 w2 + u3 v1 w3 − w1 u2 v2 − u3 w1 v3 ) + j (v2 u1 w1 + v2 w3 u3 − w2 u1 v1 − u3 w2 v3 )

+k (u1 w1 v3 + v3 w2 u2 − u1 v1 w3 − v2 w3 u2 )
and if you are clever, you see right away that

(iv1 + jv2 + kv3 ) (u1 w1 + u2 w2 + u3 w3 ) − (iw1 + jw2 + kw3 ) (u1 v1 + u2 v2 + u3 v3 ) .

Thus
u× (v × w) = v (u · w) − w (u · v) . (6.32)
6.6. VECTOR IDENTITIES AND NOTATION 95

A related formula is

(u × v) × w = − [w× (u × v)]
= − [u (w · v) − v (w · u)]
= v (w · u) − u (w · v) . (6.33)

This derivation is simply wretched and it does nothing for other identities which may arise
in applications. Actually, the above two formulas, 6.32 and 6.33 are sufficient for most
applications if you are creative in using them, but there is another way. This other way
allows you to discover such vector identities as the above without any creativity or any
cleverness. Therefore, it is far superior to the above nasty computation. It is a vector
identity discovering machine and it is this which is the main topic in what follows.
There are two special symbols, δ i j and ε i jk which are very useful in dealing with vector
identities. To begin with, here is the definition of these symbols.

Definition 6.6.1 The symbol δ i j , called the Kroneker delta symbol is defined as follows.
(
1 if i = j
δij ≡ .
0 if i ̸= j

With the Kroneker symbol i and j can equal any integer in {1, 2, · · · , n} for any n ∈ N.

Definition 6.6.2 For i, j, and k integers in the set, {1, 2, 3} , ε i jk is defined as follows.

 1 if (i, j, k) = (1, 2, 3) , (2, 3, 1) , or (3, 1, 2)

ε i jk ≡ −1 if (i, j, k) = (2, 1, 3) , (1, 3, 2) , or (3, 2, 1) .

0 if there are any repeated integers


The subscripts i jk and i j in the above are called indices. A single one is called an index.
This symbol ε i jk is also called the permutation symbol.

The way to think of ε i jk is that ε 123 = 1 and if you switch any two of the numbers in the
list i, j, k, it changes the sign. Thus ε i jk = −ε jik and ε i jk = −ε k ji etc. You should check that
this rule reduces to the above definition. For example, it immediately implies that if there
is a repeated index, the answer is zero. This follows because ε ii j = −ε ii j and so ε ii j = 0.
It is useful to use the Einstein summation convention when dealing with these symbols.
Simply stated, the convention is that you sum over the repeated index. Thus ai bi means
∑i ai bi . Also, δ i j x j means ∑ j δ i j x j = xi . When you use this convention, there is one very
important thing to never forget. It is this: Never have an index be repeated more than once.
Thus ai bi is all right but aii bi is not. The reason for this is that you end up getting confused
about what is meant. If you want to write ∑i ai bi ci it is best to simply use the summation
notation. There is a very important reduction identity connecting these two symbols.

Lemma 6.6.3 The following holds.

ε i jk ε irs = (δ jr δ ks − δ kr δ js ) .

Proof: If { j, k} ̸= {r, s} then every term in the sum on the left must have either ε i jk
or ε irs contains a repeated index. Therefore, the left side equals zero. The right side also
96 CHAPTER 6. VECTOR PRODUCTS

equals zero in this case. To see this, note that if the two sets are not equal, then there is one
of the indices in one of the sets which is not in the other set. For example, it could be that
j is not equal to either r or s. Then the right side equals zero.
Therefore, it can be assumed { j, k} = {r, s} . If i = r and j = s for s ̸= r, then there is
exactly one term in the sum on the left and it equals 1. The right also reduces to 1 in this
case. If i = s and j = r, there is exactly one term in the sum on the left which is nonzero and
it must equal -1. The right side also reduces to -1 in this case. If there is a repeated index in
{ j, k} , then every term in the sum on the left equals zero. The right also reduces to zero in
this case because then j = k = r = s and so the right side becomes (1) (1) − (−1) (−1) = 0.
■

Proposition 6.6.4 Let u, v be vectors in Rn where the Cartesian coordinates of u are

(u1 , · · · , un ) and the Cartesian coordinates of v are (v1 , · · · , vn ). Then u · v = ui vi . If u, v
are vectors in R3 , then
(u × v)i = ε i jk u j vk .
Also, δ ik ak = ai .

Proof: The first claim is obvious from the definition of the dot product. The second is
verified by simply checking that it works. For example,

i j k
u×v ≡ u1 u2 u3
v1 v2 v3

and so
(u × v)1 = (u2 v3 − u3 v2 ) .
From the above formula in the proposition,

ε 1 jk u j vk ≡ u2 v3 − u3 v2 ,

the same thing. The cases for (u × v)2 and (u × v)3 are verified similarly. The last claim
follows directly from the definition. ■
With this notation, you can easily discover vector identities and simplify expressions
which involve the cross product.

Example 6.6.5 Discover a formula which simplifies (u × v) ×w.

From the above reduction formula,

((u × v) ×w)i = ε i jk (u × v) j wk = ε i jk ε jrs ur vs wk

= −ε jik ε jrs ur vs wk = − (δ ir δ ks − δ is δ kr ) ur vs wk
= − (ui vk wk − uk vi wk ) = u · wvi − v · wui
= ((u · w) v − (v · w) u)i .

Since this holds for all i, it follows that

(u × v) ×w = (u · w) v − (v · w) u.
6.7. PLANES 97

6.7 Planes
You have an idea of what a plane is already. It is the span of some vectors. However, it can
also be considered geometrically in terms of a dot product. To find the equation of a plane,
you need two things, a point contained in the plane and a vector normal to the plane. Let
p0 = (x0 , y0 , z0 ) denote the position vector of a point in the plane, let p = (x, y, z) be the
position vector of an arbitrary point in the plane, and let n denote a vector normal to the
plane. This means that
n· (p − p0 ) = 0
whenever p is the position vector of a point in the plane. The following picture illustrates
the geometry of this idea.

n
p
p0

Expressed equivalently, the plane is just the set of all points p such that the vector
p − p0 is perpendicular to the given normal vector n.

Example 6.7.1 Find the equation of the plane with normal vector n = (1, 2, 3) containing
the point (2, −1, 5) .

From the above, the equation of this plane is just

(1, 2, 3) · (x − 2, y + 1, z − 3) = x − 9 + 2y + 3z = 0

Example 6.7.2 2x + 4y − 5z = 11 is the equation of a plane. Find the normal vector and a
point on this plane.

You can write this in the form 2 x − 11

2 + 4 (y − 0) + (−5) (z − 0) = 0.
Therefore, a
11
normal vector to the plane is 2i + 4j − 5k and a point in this plane is 2 , 0, 0 . Of course
there are many other points in the plane. The thing which makes perfect sense is the angle
between two vectors. The angle between two planes requires some definition. If you think
about it geometrically, you could imagine infinitely many angles between two lines both of
which lie in one of the planes and which intersect at a point on a line of intersection of two
planes.

Definition 6.7.3 Suppose two planes intersect in a line. The angle between the planes is
defined to be the angle which is no more than π/2 between normal vectors to the respective
planes.

Example 6.7.4 Find the angle between the two planes x + 2y − z = 6 and 3x + 2y − z = 7.
98 CHAPTER 6. VECTOR PRODUCTS

The two normal vectors are (1, 2, −1) and (3, 2, −1) . Therefore, the cosine of the angle
desired is
(1, 2, −1) · (3, 2, −1)
cos θ = q q = . 872 87
12 + 22 + (−1)2 32 + 22 + (−1)2
Now use a calculator or table to find what the angle is. cos θ = . 872 87, Solution is :
{θ = . 509 74} . This value is in radians.
Sometimes you need to find the equation of a plane which contains three points. Con-
sider the following picture.

(a2 , b2 , c2 )

(a0 , b0 , c0 ) a
(a1 , b1 , c1 )
You have plenty of points but you need a normal. This can be obtained by taking a × b
where a = (a1 − a0 , b1 − b0 , c1 − c0 ) and b = (a2 − a0 , b2 − b0 , c2 − c0 ) .
Example 6.7.5 Find the equation of the plane which contains the three points
(1, 2, 1) , (3, −1, 2) , and (4, 2, 1) .
You just need to get a normal vector to this plane. This can be done by taking the cross
products of the two vectors
(3, −1, 2) − (1, 2, 1) and (4, 2, 1) − (1, 2, 1)
Thus a normal vector is (2, −3, 1)×(3, 0, 0) = (0, 3, 9) . Therefore, the equation of the plane
is
0 (x − 1) + 3 (y − 2) + 9 (z − 1) = 0
or 3y + 9z = 15 which is the same as y + 3z = 5. When you have what you think is the
plane containing the three points, you ought to check it by seeing if it really does contain
the three points.
Example 6.7.6 Find the equation of the plane which contains the three points
(1, 2, 1) , (3, −1, 2) , and (4, 2, 1) .
You just need to get a normal vector to this plane. This can be done by taking the cross
products of the two vectors
(3, −1, 2) − (1, 2, 1) and (4, 2, 1) − (1, 2, 1)
Thus a normal vector is (2, −3, 1)×(3, 0, 0) = (0, 3, 9) . Therefore, the equation of the plane
is
0 (x − 1) + 3 (y − 2) + 9 (z − 1) = 0
or 3y + 9z = 15 which is the same as y + 3z = 5.
6.7. PLANES 99

Proposition 6.7.7 If (a, b, c) ̸= (0, 0, 0) , then ax + by + cz = d is the equation of a plane

with normal vector ai + bj + ck. Conversely, any plane can be written in this form.

Proof: One of a, b, c is nonzero. Suppose for example that c ̸= 0. Then the equation
can be written as  
d
a (x − 0) + b (y − 0) + c z − =0
c
Therefore, 0, 0, dc is a point on the plane and a normal vector is ai+bj +ck. The converse

follows from the above discussion involving the point and a normal vector. ■

Example 6.7.8 Find the equation of the plane containing the points (1, 2, 3) and the line
(0, 1, 1) + t (2, 1, 2) = (x, y, z).

There are several ways to do this. One is to find three points and use the above pro-
cedures. Let t = 0 and then let t = 1 to get two points on the line. This yields the three
points (1, 2, 3) , (0, 1, 1) , and (2, 2, 3) . Then a normal vector is obtained by fixing a point
and taking the cross product of the differences of the other two points with that one. Thus
in this case, fixing (0, 1, 1) , a normal vector is

(1, 1, 2) × (2, 1, 2) = (0, 2, −1)

Therefore, an equation for the plane is

0 (x − 0) + 2 (y − 1) + (−1) (x − 3) = 0

Simplifying this yields

2y + 1 − x = 0

Example 6.7.9 Find the equation of the plane which contains the two lines, given by the
following parametric expressions in which t ∈ R.

(2t, 1 + t, 1 + 2t) = (x, y, z) , (2t + 2, 1, 3 + 2t) = (x, y, z)

Note first that you don’t know there even is such a plane. However, if there is, you could
find it by obtaining three points, two on one line and one on another and then using any of
the above procedures for finding the plane. From the first line, two points are (0, 1, 1) and
(2, 2, 3) while a third point can be obtained from second line, (2, 1, 3) . You need a normal
vector and then use any of these points. To get a normal vector, form (2, 0, 2) × (2, 1, 2) =
(−2, 0, 2) . Therefore, the plane is −2x + 0 (y − 1) + 2 (z − 1) = 0. This reduces to z − x = 1.
If there is a plane, this is it. Now you can simply verify that both of the lines are really in
this plane. From the first, (1 + 2t) − 2t = 1 and the second, (3 + 2t) − (2t + 2) = 1 so both
lines lie in the plane.
One way to understand how a plane looks is to connect the points where it intercepts
the x, y, and z axes. This allows you to visualize the plane somewhat and is a good way to
sketch the plane. Not surprisingly these points are called intercepts.

Example 6.7.10 Sketch the plane which has intercepts (2, 0, 0) , (0, 3, 0) , and (0, 0, 4) .
100 CHAPTER 6. VECTOR PRODUCTS
z

x
You see how connecting the intercepts gives a fairly good geometric description of the
plane. These lines which connect the intercepts are also called the traces of the plane. Thus
the line which joins (0, 3, 0) to (0, 0, 4) is the intersection of the plane with the yz plane. It
is the trace on the yz plane.

Example 6.7.11 Identify the intercepts of the plane 3x − 4y + 5z = 11.

x y z
The easy way to do this is to divide both sides by 11. Thus (11/3) + (−11/4) + (11/5) = 1.
The intercepts are (11/3, 0, 0) , (0, −11/4, 0) and (0, 0, 11/5) . You can see this by letting
both y and z equal to zero to find the point on the x axis which is intersected by the plane.
The other axes are handled similarly.

6.8 Exercises
1. Show that if a × u = 0 for all unit vectors u, then a = 0.
2. If you only assume 6.30 holds for u = i, j, k, show that this implies 6.30 holds for
all unit vectors u.
3. Let m1 = 5, m2 = 1, and m3 = 4 where the masses are in kilograms and the distance
is in meters. Suppose m1 is located at 2i − 3j + k, m2 is located at i − 3j + 6k and
m3 is located at 2i + j + 3k. Find the center of mass of these three masses.
4. Let m1 = 2, m2 = 3, and m3 = 1 where the masses are in kilograms and the distance
is in meters. Suppose m1 is located at 2i − j + k, m2 is located at i − 2j + k and m3
is located at 4i + j + 3k. Find the center of mass of these three masses.
5. Find the angular velocity vector of a rigid body which rotates counter clockwise
about the vector i−2j + k at 40 revolutions per minute. Assume distance is mea-
sured in meters.
6. Let {u1 , u2 , u3 } be a right handed system with u3 pointing in the direction of
i−2j + k and u1 and u2 being fixed with the body which is rotating at 40 revo-
lutions per minute. Assuming all distances are in meters, find the constant speed of
the point of the body located at 3u1 + u2 − u3 in meters per minute.
7. Find the area of the triangle determined by the three points (1, 2, 3) , (4, 2, 0) and
(−3, 2, 1) .
8. Find the area of the triangle determined by the three points (1, 0, 3) , (4, 1, 0) and
(−3, 1, 1) .
6.8. EXERCISES 101

9. Find the area of the triangle determined by the three points (1, 2, 3) , (2, 3, 4) and
(0, 1, 2) . Did something interesting happen here? What does it mean geometrically?
10. Find the area of the parallelogram determined by the vectors (1, 2, 3) and (3, −2, 1) .

11. Find the area of the parallelogram determined by the vectors (1, 0, 3) and (4, −2, 1) .
12. Find the area of the parallelogram determined by the vectors (1, −2, 2) and (3, 1, 1) .
13. Find the volume of the parallelepiped determined by the vectors i−7j −5k, i−2j −
6k,3i + 2j + 3k.

14. Find the volume of the parallelepiped determined by the vectors i + j − 5k, i + 5j −
6k,3i + j + 3k.
15. Find the volume of the parallelepiped determined by the vectors i+6j +5k, i+5j −
6k,3i + j + k.
16. Suppose a, b, and c are three vectors whose components are all integers. Can you
conclude the volume of the parallelepiped determined from these three vectors will
always be an integer?
17. What does it mean geometrically if the box product of three vectors gives zero?
18. Find the equation of the plane through the three points (1, 2, 3) , (2, −3, 1) , (1, 1, 7) .

19. It is desired to find an equation of a plane containing the two vectors a and b and the
point 0. Using Problem 17, show an equation for this plane is

x y z
a1 a2 a3 =0
b1 b2 b3

That is, the set of all (x, y, z) such that

a2 a3 a1 a3 a1 a2
x −y +z =0
b2 b3 b1 b3 b1 b2

20. Using the notion of the box product yielding either plus or minus the volume of the
parallelepiped determined by the given three vectors, show that

(a × b) ·c = a· (b × c)

In other words, the dot and the cross can be switched as long as the order of the
vectors remains the same. Hint: There are two ways to do this, by the coordinate
description of the dot and cross product and by geometric reasoning.
21. Is a× (b × c) = (a × b) × c? What is the meaning of a × b × c? Explain. Hint: Try
(i × j) ×j.
102 CHAPTER 6. VECTOR PRODUCTS

22. Verify directly that the coordinate description of the cross product a × b has the
property that it is perpendicular to both a and b. Then show by direct computation
that this coordinate description satisfies

|a × b|2 = |a|2 |b|2 − (a · b)2 = |a|2 |b|2 1 − cos2 (θ )

where θ is the angle included between the two vectors. Explain why |a × b| has
the correct magnitude. All that is missing is the material about the right hand rule.
Verify directly from the coordinate description of the cross product that the right
thing happens with regards to the vectors i, j, k. Next verify that the distributive law
holds for the coordinate description of the cross product. This gives another way to
approach the cross product. First define it in terms of coordinates and then get the
geometric properties from this.
23. Discover a vector identity for u× (v × w) .
24. Discover a vector identity for (u × v) · (z × w) .

25. Discover a vector identity for (u × v) × (z × w) in terms of box products.

26. Simplify (u × v) · (v × w) × (w × z) .

27. Simplify |u × v|2 + (u · v)2 − |u|2 |v|2 .

28. Prove that ε i jk ε i jr = 2δ kr .

 
29. If A is a 3 × 3 matrix such that A = u v w where these are the columns of
the matrix A. Show that det (A) = ε i jk ui v j wk .

30. If A is a 3 × 3 matrix, show ε rps det (A) = ε i jk Ari A p j Ask .

31. Suppose A is a 3 × 3 matrix and det (A) ̸= 0. Show using 30 and 28 that

1
A−1


ks
= ε rps ε i jk A p j Ari .
2 det (A)

32. When you have a rotating rigid body with angular velocity vector Ω then the velocity,
u′ is given by u′ = Ω × u. It turns out that all the usual calculus rules such as the
product rule hold. Also, u′′ is the acceleration. Show using the product rule that for
Ω a constant vector
u′′ = Ω× (Ω × u) .
It turns out this is the centripetal acceleration. Note how it involves cross products.
33. Find the planes which go through the following collections of three points. In case
the plane is not well defined, explain why.

(a) (1, 2, 0) , (2, −1, 1) , (3, 1, 1)

(b) (3, 1, 0) , (2, 1, 1) , (−3, 1, −1)
(c) (2, 1, 1) , (−2, 3, 1) , (0, 4, 2)
(d) (1, 0, 1) , (2, 0, 1) , (0, 1, 1)
6.8. EXERCISES 103

34. A point is given along with a line. Find the equation for the plane which contains the
line as well as the point.

(a) (1, 2, 1) , (1, −1, 1) + t (1, 0, 1)

(b) (2, 1, −1) , (1, 1, 1) + t (2, −1, 1)
(c) (−1, 2, 3) , (−1, 1, 1) + t (2, 1, 1)
(d) (2, 0, 1) , (2, 1, 1) + t (−1, 1, 1)
104 CHAPTER 6. VECTOR PRODUCTS
Chapter 7

Systems Of Equations

7.1 Systems Of Equations, Algebraic Procedures

7.1.1 Elementary Operations
Consider the following example.

Example 7.1.1 Find x and y such that

x + y = 7 and 2x − y = 8. (7.1)

The set of ordered pairs, (x, y) which solve both equations is called the solution set.

You can verify that (x, y) = (5, 2) is a solution to the above system. The interesting
question is this: If you were not given this information to verify, how could you determine
the solution? You can do this by using the following basic operations on the equations,
none of which change the set of solutions of the system of equations.

Definition 7.1.2 Elementary operations are those operations consisting of the following.

1. Interchange the order in which the equations are listed.

2. Multiply any equation by a nonzero number.

3. Replace any equation with itself added to a multiple of another equation.

Example 7.1.3 To illustrate the third of these operations on this particular system, con-
sider the following.
x+y = 7
2x − y = 8

The system has the same solution set as the system

x+y = 7
.
−3y = −6

105
106 CHAPTER 7. SYSTEMS OF EQUATIONS

To obtain the second system, take the second equation of the first system and add −2 times
the first equation to obtain
−3y = −6.
Now, this clearly shows that y = 2 and so it follows from the other equation that x + 2 = 7
and so x = 5.
Of course a linear system may involve many equations and many variables. The so-
lution set is still the collection of solutions to the equations. In every case, the above
operations of Definition 7.1.2 do not change the set of solutions to the system of linear
equations.
Theorem 7.1.4 Suppose you have two equations, involving the variables,
(x1 , · · · , xn )
E1 = f1 , E2 = f2 (7.2)
where E1 and E2 are expressions involving the variables and f1 and f2 are constants. (In
the above example there are only two variables, x and y and E1 = x + y while E2 = 2x − y.)
Then the system E1 = f1 , E2 = f2 has the same solution set as
E1 = f1 , E2 + aE1 = f2 + a f1 . (7.3)
Also the system E1 = f1 , E2 = f2 has the same solutions as the system, E2 = f2 , E1 = f1 . The
system E1 = f1 , E2 = f2 has the same solution as the system E1 = f1 , aE2 = a f2 provided
a ̸= 0.
Proof: If (x1 , · · · , xn ) solves E1 = f1 , E2 = f2 then it solves the first equation in E1 =
f1 , E2 + aE1 = f2 + a f1 . Also, it satisfies aE1 = a f1 and so, since it also solves E2 = f2 it
must solve E2 + aE1 = f2 + a f1 . Therefore, if (x1 , · · · , xn ) solves E1 = f1 , E2 = f2 it must
also solve E2 + aE1 = f2 + a f1 . On the other hand, if it solves the system E1 = f1 and
E2 + aE1 = f2 + a f1 , then aE1 = a f1 and so you can subtract these equal quantities from
both sides of E2 +aE1 = f2 +a f1 to obtain E2 = f2 showing that it satisfies E1 = f1 , E2 = f2 .
The second assertion of the theorem which says that the system E1 = f1 , E2 = f2 has the
same solution as the system, E2 = f2 , E1 = f1 is seen to be true because it involves nothing
more than listing the two equations in a different order. They are the same equations.
The third assertion of the theorem which says E1 = f1 , E2 = f2 has the same solution
as the system E1 = f1 , aE2 = a f2 provided a ̸= 0 is verified as follows: If (x1 , · · · , xn ) is a
solution of E1 = f1 , E2 = f2 , then it is a solution to E1 = f1 , aE2 = a f2 because the second
system only involves multiplying the equation, E2 = f2 by a. If (x1 , · · · , xn ) is a solution
of E1 = f1 , aE2 = a f2 , then upon multiplying aE2 = a f2 by the number 1/a, you find that
E2 = f2 . ■
Stated simply, the above theorem shows that the elementary operations do not change
the solution set of a system of equations.
Here is an example in which there are three equations and three variables. You want to
find values for x, y, z such that each of the given equations are satisfied when these values
are plugged in to the equations.
Example 7.1.5 Find the solutions to the system,

x + 3y + 6z = 25
2x + 7y + 14z = 58 (7.4)
2y + 5z = 19
7.1. SYSTEMS OF EQUATIONS, ALGEBRAIC PROCEDURES 107

To solve this system replace the second equation by (−2) times the first equation added
to the second. This yields the system

x + 3y + 6z = 25
y + 2z = 8 (7.5)
2y + 5z = 19

Now take (−2) times the second and add to the third. More precisely, replace the third
equation with (−2) times the second added to the third. This yields the system

x + 3y + 6z = 25
y + 2z = 8 (7.6)
z=3

At this point, you can tell what the solution is. This system has the same solution as the
original system and in the above, z = 3. Then using this in the second equation, it follows
y + 6 = 8 and so y = 2. Now using this in the top equation yields x + 6 + 18 = 25 and so
x = 1. This process is called back substitution.
Alternatively, in 7.6 you could have continued as follows. Add (−2) times the bottom
equation to the middle and then add (−6) times the bottom to the top. This yields

x + 3y = 7, y = 2, z = 3

Now add (−3) times the second to the top. This yields

x = 1, y = 2, z = 3,

a system which has the same solution set as the original system. This avoided back substi-
tution and led to the same solution set.

7.1.2 Gauss Elimination

A less cumbersome way to represent a linear system is to write it as an augmented matrix.
For example the linear system, 7.4 can be written as
 
1 3 6 | 25
 2 7 14 | 58  .
 

0 2 5 | 19

It has exactly the

 same
 information asthe original
 system but here it 
is understood there
1 3 6
is an x column,  2  , a y column,  7  and a z column,  14  . The rows corre-
     

0 2 5
spond to the equations in the system. Thus the top row in the augmented matrix corresponds
to the equation,
x + 3y + 6z = 25.
Now when you replace an equation with a multiple of another equation added to itself, you
are just taking a row of this augmented matrix and replacing it with a multiple of another
108 CHAPTER 7. SYSTEMS OF EQUATIONS

row added to it. Thus the first step in solving 7.4 would be to take (−2) times the first row
of the augmented matrix above and add it to the second row,
 
1 3 6 | 25
 0 1 2 | 8 .
 

0 2 5 | 19

Note how this corresponds to 7.5. Next take (−2) times the second row and add to the
third,  
1 3 6 | 25
 0 1 2 |8 
 

0 0 1 |3
This augmented matrix corresponds to the system

x + 3y + 6z = 25
y + 2z = 8
z=3

which is the same as 7.6. By back substitution you obtain the solution x = 1, y = 6, and
z = 3.
In general a linear system is of the form

a11 x1 + · · · + a1n xn = b1
..
. , (7.7)
am1 x1 + · · · + amn xn = bm

where the xi are variables and the ai j and bi are constants. This system can be represented
by the augmented matrix
 
a11 · · · a1n | b1
 . .. . 
 .
 . . | ..  . (7.8)

am1 · · · amn | bm

Changes to the system of equations in 7.7 as a result of an elementary operations translate

into changes of the augmented matrix resulting from a row operation. Note that Theorem
7.1.4 implies that the row operations deliver an augmented matrix for a system of equations
which has the same solution set as the original system.

Definition 7.1.6 The row operations consist of the following

1. Switch two rows.

2. Multiply a row by a nonzero number.
3. Replace a row by a multiple of another row added to it.

Gauss elimination is a systematic procedure to simplify an augmented matrix to a re-

duced form. In the following definition, the term “leading entry” refers to the first nonzero
entry of a row when scanning the row from left to right.
7.1. SYSTEMS OF EQUATIONS, ALGEBRAIC PROCEDURES 109

Definition 7.1.7 An augmented matrix is in echelon form if

1. All nonzero rows are above any rows of zeros.

2. Each leading entry of a row is in a column to the right of the leading entries of any
rows above it.

How do you know when to stop doing row operations? You might stop when you have
obtained an echelon form as described above, but you certainly should stop doing row
operations if you have gotten a matrix in row reduced echelon form described next.

Definition 7.1.8 An augmented matrix is in row reduced echelon form if

1. All nonzero rows are above any rows of zeros.

2. Each leading entry of a row is in a column to the right of the leading entries of any
rows above it.
3. All entries in a column above and below a leading entry are zero.
4. Each leading entry is a 1, the only nonzero entry in its column.

Example 7.1.9 Here are some matrices which are in row reduced echelon form.

 
  1 0 0 0
1 0 0 5 8 0  
 0   0 1 0 0 
0 1 2 7 0   
, 0 0 1 0 .


 0 0 0 0 0 1   
0 0 0 1
 
0 0 0 0 0 0
 
0 0 0 0

Example 7.1.10 Here are matrices in echelon form which are not in row reduced echelon
form but which are in echelon form.
 
  1 3 5 4
1 0 6 5 8 2  
 0 0 2 2 7 3   0 2 0 7 
 
, 0 0 3 0 
  

 0 0 0 0 0 1   
 0 0 0 1 

0 0 0 0 0 0
0 0 0 0

Example 7.1.11 Here are some matrices which are not in echelon form.
 
0 0 0 0  
    0 2 3 3
 1 2 3 3  1 2 3
  1 5 0 2 
   
 0 1 0 2 ,  2 4 −6  ,  .
   7 5 0 1 
0 0 0 1 4 0 7
 
0 0 1 0
 
0 0 0 0

Definition 7.1.12 A pivot position in a matrix is the location of a leading entry in an ech-
elon form resulting from the application of row operations to the matrix. A pivot column is
a column that contains a pivot position.
110 CHAPTER 7. SYSTEMS OF EQUATIONS

For example consider the following.

Example 7.1.13 Suppose  

1 2 3 4
A= 3 2 1 6 
 

4 4 4 10
Where are the pivot positions and pivot columns?

Replace the second row by −3 times the first added to the second. This yields
 
1 2 3 4
 0 −4 −8 −6  .
 

4 4 4 10

This is not in reduced echelon form so replace the bottom row by −4 times the top row
added to the bottom. This yields
 
1 2 3 4
 0 −4 −8 −6  .
 

0 −4 −8 −6

This is still not in reduced echelon form. Replace the bottom row by −1 times the middle
row added to the bottom. This yields
 
1 2 3 4
 0 −4 −8 −6 
 

0 0 0 0

which is in echelon form, although not in reduced echelon form. Therefore, the pivot
positions in the original matrix are the locations corresponding to the first row and first
column and the second row and second columns as shown in the following:
 
1 2 3 4
 3 2 1 6 
 

4 4 4 10

Thus the pivot columns in the matrix are the first two columns.
The following is the algorithm for obtaining a matrix which is in row reduced echelon
form.

Algorithm 7.1.14

This algorithm tells how to start with a matrix and do row operations on it in such a
way as to end up with a matrix in row reduced echelon form.

1. Find the first nonzero column from the left. This is the first pivot column. The
position at the top of the first pivot column is the first pivot position. Switch rows if
necessary to place a nonzero number in the first pivot position.
7.1. SYSTEMS OF EQUATIONS, ALGEBRAIC PROCEDURES 111

2. Use row operations to zero out the entries below the first pivot position.
3. Ignore the row containing the most recent pivot position identified and the rows above
it. Repeat steps 1 and 2 to the remaining sub-matrix, the rectangular array of numbers
obtained from the original matrix by deleting the rows you just ignored. Repeat the
process until there are no more rows to modify. The matrix will then be in echelon
form.
4. Moving from right to left, use the nonzero elements in the pivot positions to zero out
the elements in the pivot columns which are above the pivots.
5. Divide each nonzero row by the value of the leading entry. The result will be a matrix
in row reduced echelon form.

This row reduction procedure applies to both augmented matrices and non augmented
matrices. There is nothing special about the augmented column with respect to the row
reduction procedure.

Example 7.1.15 Here is a matrix.

 
0 0 2 3 2
 
 0 1 1 4 3 
 

 0 0 1 2 2 

0 0 0 0 0
 
 
0 0 0 2 1

Do row reductions till you obtain a matrix in echelon form. Then complete the process by
producing one in row reduced echelon form.

The pivot column is the second. Hence the pivot position is the one in the first row and
second column. Switch the first two rows to obtain a nonzero entry in this pivot position.
 
0 1 1 4 3
 
 0 0 2 3 2 
 
 0 0 1 2 2 
 
 0 0 0 0 0 
 
0 0 0 2 1

Step two is not necessary because all the entries below the first pivot position in the resulting
matrix are zero. Now ignore the top row and the columns to the left of this first pivot
position. Thus you apply the same operations to the smaller matrix
 
2 3 2
 1 2 2 
.
 

 0 0 0 
0 2 1

The next pivot column is the third corresponding to the first in this smaller matrix and the
second pivot position is therefore, the one which is in the second row and third column.
112 CHAPTER 7. SYSTEMS OF EQUATIONS

In this case it is not necessary to switch any rows to place a nonzero entry in this position
because there is already a nonzero entry there. Multiply the third row of the original matrix
by −2 and then add the second row to it. This yields
 
0 1 1 4 3
 
 0 0 2 3 2 
 
 0 0 0 −1 −2  .
 
 0 0 0 0 0 
 
0 0 0 2 1

The next matrix the steps in the algorithm are applied to is

 
−1 −2
 0 0 .
 

2 1

The first pivot column is the first column in this case and no switching of rows is necessary
because there is a nonzero entry in the first pivot position. Therefore, the algorithm yields
for the next step  
0 1 1 4 3
 
 0 0 2 3 2 
 
 0 0 0 −1 −2  .
 
 0 0 0 0 0 
 
0 0 0 0 −3
Now the algorithm will be applied to the matrix
!
0
−3

There is only one column and it is nonzero so this single column is the pivot column.
Therefore, the algorithm yields the following matrix for the echelon form.
 
0 1 1 4 3
 
 0 0 2 3 2 
 
 0 0 0 −1 −2  .
 
 0 0 0 0 −3 
 
0 0 0 0 0

To complete placing the matrix in reduced echelon form, multiply the third row by 3 and
add −2 times the fourth row to it. This yields
 
0 1 1 4 3
 
 0 0 2 3 2 
 
 0 0 0 −3 0 
 
 0 0 0 0 −3 
 
0 0 0 0 0
7.1. SYSTEMS OF EQUATIONS, ALGEBRAIC PROCEDURES 113

Next multiply the second row by 3 and take 2 times the fourth row and add to it. Then add
the fourth row to the first.  
0 1 1 4 0
 
 0 0 6 9 0 
 
 0 0 0 −3 0  .
 
 0 0 0 0 −3 
 
0 0 0 0 0
Next work on the fourth column in the same way.
 
0 3 3 0 0
 
 0 0 6 0 0 
 
 0 0 0 −3 0 
 
 0 0 0 0 −3
 

0 0 0 0 0
Take −1/2 times the second row and add to the first.
 
0 3 0 0 0
 
 0 0 6 0 0 
 
 0 0 0 −3 0  .
 
 0 0 0 0 −3 
 
0 0 0 0 0
Finally, divide by the value of the leading entries in the nonzero rows.
 
0 1 0 0 0
 
 0 0 1 0 0 
 
 0 0 0 1 0 .
 
 0 0 0 0 1 
 
0 0 0 0 0
The above algorithm is the way a computer would obtain a reduced echelon form for
a given matrix. It is not necessary for you to pretend you are a computer but if you like
to do so, the algorithm described above will work. The main idea is to do row operations
in such a way as to end up with a matrix in echelon form or row reduced echelon form
because when this has been done, the resulting augmented matrix will allow you to describe
the solutions to the linear system of equations in a meaningful way. When you do row
operations until you obtain row reduced echelon form, the process is called the Gauss
Jordan method. Otherwise, it is called Gauss elimination.
Example 7.1.16 Give the complete solution to the system of equations, 5x+10y−7z = −2,
2x + 4y − 3z = −1, and 3x + 6y + 5z = 9.
The augmented matrix for this system is
 
2 4 −3 −1
 5 10 −7 −2 
 

3 6 5 9
114 CHAPTER 7. SYSTEMS OF EQUATIONS

Multiply the second row by 2, the first row by 5, and then take (−1) times the first row and
add to the second. Then multiply the first row by 1/5. This yields
 
2 4 −3 −1
 0 0 1 1 
 

3 6 5 9
Now, combining some row operations, take (−3) times the first row and add this to 2 times
the last row and replace the last row with this. This yields.
 
2 4 −3 −1
 0 0 1 1 .
 

0 0 1 21
One more row operation, taking (−1) times the second row and adding to the bottom yields.
 
2 4 −3 −1
 0 0 1 1 .
 

0 0 0 20
This is impossible because the last row indicates the need for a solution to the equation
0x + 0y + 0z = 20
and there is no such thing because 0 ̸= 20. This shows there is no solution to the three given
equations. When this happens, the system is called inconsistent. In this case it is very easy
to describe the solution set. The system has no solution.
Here is another example based on the use of row operations.
Example 7.1.17 Give the complete solution to the system of equations, 3x − y − 5z = 9,
y − 10z = 0, and −2x + y = −6.
The augmented matrix of this system is
 
3 −1 −5 9
 0 1 −10 0 
 

−2 1 0 −6
Replace the last row with 2 times the top row added to 3 times the bottom row combining
two row operations. This gives
 
3 −1 −5 9
 0 1 −10 0  .
 

0 1 −10 0
The entry, 3 in this sequence of row operations is called the pivot. It is used to create
zeros in the other places of the column. Next take −1 times the middle row and add to the
bottom. Here the 1 in the second row is the pivot.
 
3 −1 −5 9
 0 1 −10 0 
 

0 0 0 0
7.1. SYSTEMS OF EQUATIONS, ALGEBRAIC PROCEDURES 115

Take the middle row and add to the top and then divide the top row which results by 3.
 
1 0 −5 3
 0 1 −10 0  .
 

0 0 0 0

This is in reduced echelon form. The equations corresponding to this reduced echelon
form are y = 10z and x = 3 + 5z. Apparently z can equal any number. Lets call this number
t. 1 Therefore, the solution set of this system is x = 3 + 5t, y = 10t, and z = t where t is
completely arbitrary. The system has an infinite set of solutions which are given in the
above simple way. This is what it is all about, finding the solutions to the system.
There is some terminology connected to this which is useful. Recall how each column
corresponds to a variable in the original system of equations. The variables corresponding
to a pivot column are called basic variables. The other variables are called free variables.
In Example 7.1.17 there was one free variable, z, and two basic variables, x and y. In de-
scribing the solution to the system of equations, the free variables are assigned a parameter.
In Example 7.1.17 this parameter was t. Sometimes there are many free variables and in
these cases, you need to use many parameters. Here is another example.

Example 7.1.18 Find the solution to the system

x + 2y − z + w = 3
x+y−z+w = 1
x + 3y − z + w = 5

The augmented matrix is

 
1 2 −1 1 3
 1 1 −1 1 1  .
 

1 3 −1 1 5

Take −1 times the first row and add to the second. Then take −1 times the first row and
add to the third. This yields
 
1 2 −1 1 3
 0 −1 0 0 −2 
 

0 1 0 0 2

Now add the second row to the bottom row

 
1 2 −1 1 3
 0 −1 0 0 −2  (7.9)
 

0 0 0 0 0

This matrix is in echelon form and you see the basic variables are x and y while the free
variables are z and w. Assign s to z and t to w. Then the second row yields the equation,
y = 2 while the top equation yields the equation, x + 2y − s + t = 3 and so since y = 2, this
1 In this context t is called a parameter.
116 CHAPTER 7. SYSTEMS OF EQUATIONS

gives x + 4 − s + t = 3 showing that x = −1 + s − t, y = 2, z = s, and w = t. It is customary

to write this in the form    
x −1 + s − t
 y   2 
= . (7.10)
   

 z   s 
w t
This is another example of a system which has an infinite solution set but this time
the solution set depends on two parameters, not one. Most people find it less confusing
in the case of an infinite solution set to first place the augmented matrix in row reduced
echelon form rather than just echelon form before seeking to write down the description of
the solution. In the above, this means we don’t stop with the echelon form 7.9. Instead we
first place it in reduced echelon form as follows.
 
1 0 −1 1 −1
 0 1 0 0 2 .
 

0 0 0 0 0

Then the solution is y = 2 from the second row and x = −1 + z − w from the first. Thus
letting z = s and w = t, the solution is given in 7.10.
The number of free variables is always equal to the number of different parameters
used to describe the solution. If there are no free variables, then either there is no solution
as in the case where row operations yield an echelon form like
 
1 2 3
 0 4 −2 
 

0 0 1

or there is a unique solution as in the case where row operations yield an echelon form like
 
1 2 2 3
 0 4 3 −2  .
 

0 0 4 1

Also, sometimes there are free variables and no solution as in the following:
 
1 2 2 3
 0 4 3 −2  .
 

0 0 0 1

There are a lot of cases to consider but it is not necessary to make a major production of
this. Do row operations till you obtain a matrix in echelon form or reduced echelon form
and determine whether there is a solution. If there is, see if there are free variables. In this
case, there will be infinitely many solutions. Find them by assigning different parameters
to the free variables and obtain the solution. If there are no free variables, then there will
be a unique solution which is easily determined once the augmented matrix is in echelon
or row reduced echelon form. In every case, the process yields a straightforward way to
describe the solutions to the linear system. As indicated above, you are probably less likely
7.1. SYSTEMS OF EQUATIONS, ALGEBRAIC PROCEDURES 117

to become confused if you place the augmented matrix in row reduced echelon form rather
than just echelon form.
In summary,

Definition 7.1.19 A system of linear equations is a list of equations,

a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2
..
.
am1 x1 + am2 x2 + · · · + amn xn = bm

where ai j are numbers, and b j is a number. The above is a system of m equations in the
n variables, x1 , x2 · · · , xn . Nothing is said about the relative size of m and n. Written more
simply in terms of summation notation, the above can be written in the form
n
∑ ai j x j = fi , i = 1, 2, 3, · · · , m
j=1

It is desired to find (x1 , · · · , xn ) solving each of the equations listed.

As illustrated above, such a system of linear equations may have a unique solution, no
solution, or infinitely many solutions and these are the only three cases which can occur for
any linear system. Furthermore, you do exactly the same things to solve any linear system.
You write the augmented matrix and do row operations until you get a simpler system in
which it is possible to see the solution, usually obtaining a matrix in echelon or reduced
echelon form. All is based on the observation that the row operations do not change the
solution set. You can have more equations than variables, fewer equations than variables,
etc. It doesn’t matter. You always set up the augmented matrix and go to work on it.

Definition 7.1.20 A system of linear equations is called consistent if there exists a solution.
It is called inconsistent if there is no solution.

These are reasonable words to describe the situations of having or not having a solution.
If you think of each equation as a condition which must be satisfied by the variables, con-
sistent would mean there is some choice of variables which can satisfy all the conditions.
Inconsistent would mean there is no choice of the variables which can satisfy each of the
conditions.

7.1.3 Balancing Chemical Reactions

Consider the chemical reaction

SnO2 + H2 → Sn + H2 O

Here the elements involved are tin Sn oxygen O and Hydrogen H. Some chemical reaction
happens and you end up with some tin and some water. The question is, how much do you
start with and how much do you end up with.
The balance of mass requires that you have the same number of oxygen, tin, and hy-
drogen on both sides of the reaction. However, this does not happen in the above. For
118 CHAPTER 7. SYSTEMS OF EQUATIONS

example, there are two oxygen atoms on the left and only one on the right. The problem is
to find numbers x, y, z, w such that
xSnO2 + yH2 → zSn + wH2 O
and both sides have the same number of atoms of the various substances. You can do this
in a systematic way by setting up a system of equations which will require that this take
place. Thus you need
Sn : x=z
O : 2x = w
H : 2y = 2w
The augmented matrix for this system of equations is then
 
1 0 −1 0 0
 2 0 0 −1 0 
 

0 2 0 −2 0
Row reducing this yields  
1 0 0 − 21 0
 0 1 0 −1 0 
 

0 0 1 − 12 0
Thus you could let w = 2 and this would yield x = 1, y = 2, and z = 1. Hence, the description
of the reaction which has the same numbers of atoms on both sides would be
SnO2 + 2H2 → Sn + 2H2 O
You see that this preserves the total number of atoms and so the chemical equation is
balanced. Here is another example
Example 7.1.21 Potassium is denoted by K, oxygen by O, phosphorus by P and hydrogen
by H. The reaction is
KOH + H3 PO4 → K3 PO4 + H2 O
balance this equation.
You need to have
xKOH + yH3 PO4 → zK3 PO4 + wH2 O
Equations which preserve the total number of atoms of each element on both sides of the
equation are
K: x = 3z
O : x + 4y = 4z + w
H: x + 3y = 2w
P: y=z
The augmented matrix for this system is
 
1 0 −3 0 0
 1 4 −4 −1 0 
 
−2 0 
 
 1 3 0
0 1 −1 0 0
7.1. SYSTEMS OF EQUATIONS, ALGEBRAIC PROCEDURES 119

Then the row reduced echelon form is

 
1 0 0 −1 0
 0
 1 0 − 13 0 

− 13
 
 0 0 1 0 
0 0 0 0 0

You could let w = 3 and this yields x = 3, y = 1, z = 1. Then the balanced equation is

3KOH + 1H3 PO4 → 1K3 PO4 + 3H2 O

Note that this results in the same number of atoms on both sides.
Of course these numbers you are finding would typically be the number of moles of the
molecules on each side. Thus three moles of KOH added to one mole of H3 PO4 yields one
mole of K3 PO4 and three moles of H2 O, water.
Note that in this example, you have a row of zeros. This means that some of the in-
formation in computing the appropriate numbers was redundant. If this can happen with a
single reaction, think how much more it could happen if you were dealing with hundreds
of reactions. This aspect of the problem can be understood later in terms of the rank of a
matrix.
For an introduction to the chemical considerations mentioned here, there is a nice site on
the web [Link] where there
is a sample test and examples of chemical reactions. For names of the various elements
symbolized by the various letters, you can go to the site
[Link] Chemical elements Of course
these things are in standard chemistry books, but if you have not seen much chemistry, these
sites give a nice introduction to these concepts.

7.1.4 Dimensionless Variables∗

This section shows how solving systems of equations can be used to determine appropriate
dimensionless variables. It is only an introduction to this topic. I got this example from
[18]. This considers a specific example of a simple airplane wing shown below. We assume
for simplicity that it is just a flat plane at an angle to the wind which is blowing against it
with speed V as shown.

A
θ B

The angle is called the angle of incidence, B is the span of the wing and A is called the
chord. Denote by l the lift. Then this should depend on various quantities like θ ,V, B, A
and so forth. Here is a table which indicates various quantities on which it is reasonable to
120 CHAPTER 7. SYSTEMS OF EQUATIONS

expect l to depend.

Variable Symbol Units

chord A m
span B m
angle incidence θ m0 kg0 sec0
speed of wind V m sec−1
speed of sound V0 m sec−1
density of air ρ kgm−3
viscosity µ kg sec−1 m−1
lift l kg sec−2 m

Here m denotes meters, sec refers to seconds and kg refers to kilograms. All of these
are likely familiar except for µ. One can simply decree that these are the dimensions of
something called viscosity but it might be better to consider this a little more.
Viscosity is a measure of how much internal friction is experienced when the fluid
moves. It is roughly a measure of how “sticky” the fluid is. Consider a piece of area
parallel to the direction of motion of the fluid. To say that the viscosity is large is to say
that the tangential force applied to this area must be large in order to achieve a given change
in speed of the fluid in a direction normal to the tangential force. Thus

µ (area) (velocity gradient) = tangential force.

Hence  m 
(units on µ) m2 = kg sec−2 m
sec m
Thus the units on µ are kg sec−1 m−1 as claimed above.
Then one would think that you would want

l = f (A, B, θ ,V,V0 , ρ, µ)

However, this is very cumbersome because it depends on seven variables. Also, it doesn’t
make very good sense. It is likely that without much care, a change in the units such as
going from meters to feet would result in an incorrect value for l. The way to get around
this problem is to look for l as a function of dimensionless variables multiplied by some-
thing which has units of force. It is helpful because first of all, you will likely have fewer
independent variables and secondly, you could expect the formula to hold independent of
the way of specifying length, mass and so forth. One looks for

l = f (g1 , · · · , gk ) ρV 2 AB

where the units on ρV 2 AB are

kg  m 2 2 kg × m
m =
m3 sec sec2
which are the units of force. Each of these gi is of the form
x
Ax1 Bx2 θ x3 V x4 V0 5 ρ x6 µ x7 (7.11)
7.1. SYSTEMS OF EQUATIONS, ALGEBRAIC PROCEDURES 121

and each gi is independent of the dimensions. That is, this expression must not depend on
meters, kilograms, seconds, etc. Thus, placing in the units for each of these quantities, one
needs

x
x
mx1 mx2 mx4 sec−x4 mx5 sec−x5 kgm−3 6 kg sec−1 m−1 7 = m0 kg0 sec0

Notice that there are no units on θ because it is just the radian measure of an angle. Hence
its dimensions consist of length divided by length, thus it is dimensionless. Then this leads
to the following equations for the xi .

m: x1 + x2 + x4 + x5 − 3x6 − x7 = 0
sec : −x4 − x5 − x7 = 0
kg : x6 + x7 = 0

Then the augmented matrix for this system of equations is

 
1 1 0 1 1 −3 −1 0
 0 0 0 1 1 0 1 0 
 

0 0 0 0 0 1 1 0

The row reduced echelon form is then

 
1 1 0 0 0 0 1 0
 0 0 0 1 1 0 1 0 
 

0 0 0 0 0 1 1 0

and so the solutions are of the form

x1 = −x2 − x7 , x3 = x3 , x4 = −x5 − x7 , x6 = −x7

Thus, in terms of vectors, the solution is

   
x1 −x2 − x7

 x2  
  x2 

   
 x3   x3 
   

 x4 =
  −x5 − x7 

x5 x5
   
   
   
 x6   −x7 
x7 x7

Thus the free variables are x2 , x3 , x5 , x7 . By assigning values to these, we can obtain di-
mensionless variables by placing the values obtained for the xi in the formula 7.11. For
example, let x2 = 1 and all the rest of the free variables are 0. This yields

x1 = −1, x2 = 1, x3 = 0, x4 = 0, x5 = 0, x6 = 0, x7 = 0.

The dimensionless variable is then A−1 B1 . This is the ratio between the span and the chord.
It is called the aspect ratio, denoted as AR. Next let x3 = 1 and all others equal zero. This
122 CHAPTER 7. SYSTEMS OF EQUATIONS

gives for a dimensionless quantity the angle θ . Next let x5 = 1 and all others equal zero.
This gives
x1 = 0, x2 = 0, x3 = 0, x4 = −1, x5 = 1, x6 = 0, x7 = 0.
Then the dimensionless variable is V −1V01 . However, it is written as V /V0 . This is called
the Mach number M . Finally, let x7 = 1 and all the other free variables equal 0. Then
x1 = −1, x2 = 0, x3 = 0, x4 = −1, x5 = 0, x6 = −1, x7 = 1
then the dimensionless variable which results from this is A−1V −1 ρ −1 µ. It is customary to
write it as Re = (AV ρ) /µ. This one is called the Reynolds number. It is the one which
involves viscosity. Thus we would look for
l = f (Re, AR, θ , M ) kg × m/ sec2
This is quite interesting because it is easy to vary Re by simply adusting the velocity or A
but it is hard to vary things like µ or ρ. Note that all the quantities are easy to adjust. Now
this could be used, along with wind tunnel experiments to get a formula for the lift which
would be reasonable. Obviously, you could consider more variables and more complicated
situations in the same way.

7.2 MATLAB And Row Reduced Echelon Form

MATLAB will find the row reduced echelon form of a matrix and save you the trouble of
tedious computations. You open matlab. You will see >>. Then next to it you type the
following:
rref([1,2,3,4;2,5,6,10;3,2,0,-5])
Then press enter on your keyboard. It will give the following.
ans =
1 0 0 -3
0102
0011
In usual notation, this is the row reduced echelon form of the matrix
 Notice how you enter a row by placing commas between entries and

1 2 3 4
then when you start a new row, you put a ; to indicate it is a new row.
 2 5 6 10 
 
You do something similar for another matrix. You can also simply
3 2 0 −5 leave a space between the entries of a row and it will know what to do,
but you indicate a new row by using ;. The semicolon ; is also used to defer an operation.
MATLAB will know about it but won’t do anything.
In using MATLAB, you press shift enter to go to a new line. One thing might be helpful
to mention about MATLAB. It is very good at manipulating matrices and vectors and there
is distinctive notation used to accomplish this. For example say you type

x=[1,2,3]; y=[2,3,4]; x.*y

and then press “enter”. You will get 2,6,12. You would get an error if you wrote x*y.
Similarly, type
[2,4,6,8]./[1,2,3,4]
and press “enter”. This yields 2, 2, 2, 2. The expression [2,4,6,8]/[1,2,3,4] doesn’t make any
sense.
7.3. EXERCISES 123

7.3 Exercises
1. Find the point (x1 , y1 ) which lies on both lines, x + 3y = 1 and 4x − y = 3.
2. Solve Problem 1 graphically. That is, graph each line and see where they intersect.
3. Find the point of intersection of the two lines 3x + y = 3 and x + 2y = 1.
4. Solve Problem 3 graphically. That is, graph each line and see where they intersect.
5. Do the three lines, x + 2y = 1, 2x − y = 1, and 4x + 3y = 3 have a common point of
intersection? If so, find the point and if not, tell why they don’t have such a common
point of intersection.
6. Do the three planes, x + y − 3z = 2, 2x + y + z = 1, and 3x + 2y − 2z = 0 have a
common point of intersection? If so, find one and if not, tell why there is no such
point.
7. You have a system of k equations in two variables, k ≥ 2. Explain the geometric
significance of

(a) No solution.
(b) A unique solution.
(c) An infinite number of solutions.

8. Here is an augmented matrix in which ∗ denotes an arbitrary number and ■ denotes

a nonzero number. Determine whether the given augmented matrix is consistent. If
consistent, is the solution unique?
 
■ ∗ ∗ ∗ ∗ | ∗
 0 ■ ∗ ∗ 0 | ∗ 
 
 0 0 ■ ∗ ∗ | ∗ 
 

0 0 0 0 ■ | ∗

9. Here is an augmented matrix in which ∗ denotes an arbitrary number and ■ denotes

a nonzero number. Determine whether the given augmented matrix is consistent. If
consistent, is the solution unique?
 
■ ∗ ∗ | ∗
 0 ■ ∗ | ∗ 
 

0 0 ■ | ∗

10. Here is an augmented matrix in which ∗ denotes an arbitrary number and ■ denotes
a nonzero number. Determine whether the given augmented matrix is consistent. If
consistent, is the solution unique?
 
■ ∗ ∗ ∗ ∗ | ∗
 0 ■ 0 ∗ 0 | ∗ 
 
 0 0 0 ■ ∗ | ∗ 
 

0 0 0 0 ■ | ∗
124 CHAPTER 7. SYSTEMS OF EQUATIONS

11. Here is an augmented matrix in which ∗ denotes an arbitrary number and ■ denotes
a nonzero number. Determine whether the given augmented matrix is consistent. If
consistent, is the solution unique?
 
■ ∗ ∗ ∗ ∗ | ∗

 0 ■ ∗ ∗ 0 | ∗ 

0 0 ■ | 0
 
 0 0 
0 0 0 0 ∗ | ■

12. Suppose a system of equations has fewer equations than variables. Must such a
system be consistent? If so, explain why and if not, give an example which is not
consistent.
13. If a system of equations has more equations than variables, can it have a solution? If
so, give an example and if not, tell why not.
14. Find h such that !
2 h | 4
3 6 | 7
is the augmented matrix of an inconsistent matrix.
15. Find h such that !
1 h | 3
2 4 | 6
is the augmented matrix of a consistent matrix.
16. Find h such that !
1 1 | 4
3 h | 12
is the augmented matrix of a consistent matrix.
17. Choose h and k such that the augmented matrix shown has one solution. Then choose
h and k such that the system has no solutions. Finally, choose h and k such that the
system has infinitely many solutions.
!
1 h | 2
.
2 4 | k

18. Choose h and k such that the augmented matrix shown has one solution. Then choose
h and k such that the system has no solutions. Finally, choose h and k such that the
system has infinitely many solutions.
!
1 2 | 2
.
2 h | k
7.3. EXERCISES 125

19. Determine if the system is consistent. If so, is the solution unique?

x + 2y + z − w = 2
x−y+z+w = 1
2x + y − z = 1
4x + 2y + z = 5

20. Determine if the system is consistent. If so, is the solution unique?

x + 2y + z − w = 2
x−y+z+w = 0
2x + y − z = 1
4x + 2y + z = 3

21. Find the general solution of the system whose augmented matrix is
 
1 2 0 | 2
 1 3 4 | 2 .
 

1 0 2 | 1

22. Find the general solution of the system whose augmented matrix is
 
1 2 0 | 2
 2 0 1 | 1 .
 

3 2 1 | 3

23. Find the general solution of the system whose augmented matrix is
!
1 1 0 | 1
.
1 0 4 | 2

24. Find the general solution of the system whose augmented matrix is
 
1 0 2 1 1 | 2
 0 1 0 1 2 | 1 
.
 
|

 1 2 0 0 1 3 
1 0 1 0 2 | 2

25. Find the general solution of the system whose augmented matrix is
 
1 0 2 1 1 | 2
 0 1 0 1 2 | 1 
.
 
|

 0 2 0 0 1 3 
1 −1 2 2 2 | 0
126 CHAPTER 7. SYSTEMS OF EQUATIONS

26. Give the complete solution to the system of equations, 7x + 14y + 15z = 22, 2x +
4y + 3z = 5, and 3x + 6y + 10z = 13.
27. Give the complete solution to the system of equations, 3x − y + 4z = 6, y + 8z = 0,
and −2x + y = −4.

28. Give the complete solution to the system of equations, 9x − 2y + 4z = −17, 13x −
3y + 6z = −25, and −2x − z = 3.
29. Give the complete solution to the system of equations, 65x + 84y + 16z = 546, 81x +
105y + 20z = 682, and 84x + 110y + 21z = 713.

30. Give the complete solution to the system of equations, 8x + 2y + 3z = −3, 8x + 3y +

3z = −1, and 4x + y + 3z = −9.
31. Give the complete solution to the system of equations, −8x + 2y + 5z = 18, −8x +
3y + 5z = 13, and −4x + y + 5z = 19.

32. Give the complete solution to the system of equations, 3x − y − 2z = 3, y − 4z = 0,

and −2x + y = −2.
33. Give the complete solution to the system of equations, −9x+15y = 66, −11x+18y =
79 ,−x + y = 4, and z = 3.
34. Give the complete solution to the system of equations, −19x + 8y = −108, −71x +
30y = −404, −2x + y = −12, 4x + z = 14.
35. Consider the system −5x + 2y − z = 0 and −5x − 2y − z = 0. Both equations equal
zero and so −5x + 2y − z = −5x − 2y − z which is equivalent to y = 0. Thus x and
z can equal anything. But when x = 1, z = −4, and y = 0 are plugged in to the
equations, it doesn’t work. Why?

36. Four times the weight of Gaston is 150 pounds more than the weight of Ichabod.
Four times the weight of Ichabod is 660 pounds less than seventeen times the weight
of Gaston. Four times the weight of Gaston plus the weight of Siegfried equals 290
pounds. Brunhilde would balance all three of the others. Find the weights of the four
sisters.

37. The steady state temperature, u in a plate solves Laplace’s equation, ∆u = 0. One way
to approximate the solution which is often used is to divide the plate into a square
mesh and require the temperature at each node to equal the average of the temperature
at the four adjacent nodes. This procedure is justified by the mean value property of
harmonic functions. In the following picture, the numbers represent the observed
temperature at the indicated nodes. Your task is to find the temperature at the interior
nodes, indicated by x, y, z, and w. One of the equations is z = 41 (10 + 0 + w + x).
30 30
20 y w 0
20 x z 0

10 10
7.3. EXERCISES 127

38. Consider the following diagram of four circuits.

3Ω 20 volts 1 Ω

I2 I3
5 volts 5Ω 1Ω

2Ω 6Ω

I1 I4
10 volts 1Ω 3Ω
4Ω 2Ω

Those jagged places denote resistors and the numbers next to them give their resis-
tance in ohms, written as Ω. The breaks in the lines having one short line and one
long line denote a voltage source which causes the current to flow in the direction
which goes from the longer of the two lines toward the shorter along the unbroken
part of the circuit. The current in amps in the four circuits is denoted by I1 , I2 , I3 , I4
and it is understood that the motion is in the counter clockwise direction. If Ik ends
up being negative, then it just means the current flows in the clockwise direction.
Then Kirchhoff’s law states that
The sum of the resistance times the amps in the counter clockwise direction around
a loop equals the sum of the voltage sources in the same direction around the loop.
In the above diagram, the top left circuit should give the equation

2I2 − 2I1 + 5I2 − 5I3 + 3I2 = 5

For the circuit on the lower left, you should have

4I1 + I1 − I4 + 2I1 − 2I2 = −10

Write equations for each of the other two circuits and then give a solution to the
resulting system of equations. You might use a computer algebra system to find the
solution. It might be more convenient than doing it by hand.
39. Consider the following diagram of three circuits.
128 CHAPTER 7. SYSTEMS OF EQUATIONS

3Ω 12 volts 7 Ω

I1 I2
10 volts 5Ω 3Ω
2Ω 1Ω

I3
2Ω 4Ω
4Ω

Those jagged places denote resistors and the numbers next to them give their resis-
tance in ohms, written as Ω. The breaks in the lines having one short line and one
long line denote a voltage source which causes the current to flow in the direction
which goes from the longer of the two lines toward the shorter along the unbroken
part of the circuit. The current in amps in the four circuits is denoted by I1 , I2 , I3 and
it is understood that the motion is in the counter clockwise direction. If Ik ends up
being negative, then it just means the current flows in the clockwise direction. Then
Kirchhoff’s law states that
The sum of the resistance times the amps in the counter clockwise direction around
a loop equals the sum of the voltage sources in the same direction around the loop.
Find I1 , I2 , I3 .
40. Here are some chemical reactions. Balance them.

(a) KNO3 + H2CO3 → K2CO3 + HNO3

(b) Ba3 N2 + H2 O → Ba (OH)2 + NH3
(c) CaCl2 + Na3 PO4 → Ca3 (PO4 )2 + NaCl

41. In the section on dimensionless variables 119 it was observed that ρV 2 AB has the
units of force. Describe a systematic way to obtain such combinations of the vari-
ables which will yield something which has the units of force.
Chapter 8

Matrices

8.1 Addition And Scalar Multiplication Of Matrices

You have now solved systems of equations by writing them in terms of an augmented matrix
and then doing row operations on this augmented matrix. It turns out such rectangular
arrays of numbers are important from many other different points of view. Numbers are
also called scalars. In this book, numbers will generally be either real or complex numbers.
I will refer to the set of numbers as F sometimes when it is not important to worry about
whether the number is real or complex. Thus F can be either the real numbers, R or the
complex numbers C. However, most of the algebraic considerations hold for more general
fields of scalars.
A matrix is a rectangular array of numbers. Several of them are referred to as matrices.
For example, here is a matrix.
 
1 2 3 4
 5 2 8 7 
 

6 −9 1 2
The size or dimension of a matrix is defined as m × n where m is the number of rows and
n is the number of columns. The above matrix is a 3 × 4 matrix because there are three
rows and four columns. The  first row is (1 2 3 4) , the second row is (5 2 8 7) and so forth.
1
The first column is  5  . When specifying the size of a matrix, you always list the
 

6
number of rows before the number of columns. Also, you can remember the columns are
like columns in a Greek temple. They stand upright while the rows just lie there like rows
made by a tractor in a plowed field. Elements of the matrix are identified according to
position in the matrix. For example, 8 is in position 2, 3 because it is in the second row and
the third column. You might remember that you always list the rows before the columns
by using the phrase Rowman Catholic. The symbol, (ai j ) refers to a matrix. The entry in
the ith row and the jth column of this matrix is denoted by ai j . Using this notation on the
above matrix, a23 = 8, a32 = −9, a12 = 2, etc.
There are various operations which are done on matrices. Matrices can be added mul-
tiplied by a scalar, and multiplied by other matrices. To illustrate scalar multiplication,

129
130 CHAPTER 8. MATRICES

consider the following example in which a matrix is being multiplied by the scalar 3.
   
1 2 3 4 3 6 9 12
3  5 2 8 7  =  15 6 24 21  .
   

6 −9 1 2 18 −27 3 6

The new matrix is obtained by multiplying every entry of the original matrix by the given
scalar. If A is an m × n matrix, −A is defined to equal (−1) A.
Two matrices must be the same size to be added. The sum of two matrices is a matrix
which is obtained by adding the corresponding entries. Thus
     
1 2 −1 4 0 6
 3 4 + 2 8  =  5 12  .
     

5 2 6 −4 11 −2

Two matrices are equal exactly when they are the same size and the corresponding entries
are identical. Thus  
0 0 !
0 0
 0 0  ̸=
 
0 0
0 0
because they are different sizes. As noted above, you write (ci j ) for the matrix C whose
i jth entry is ci j . In doing arithmetic with matrices you must define what happens in terms
of the ci j sometimes called the entries of the matrix or the components of the matrix.
The above discussion stated for general matrices is given in the following definition.

Definition 8.1.1 (Scalar Multiplication) If A = (ai j ) and k is a scalar, then kA = (kai j ) .

! !
2 0 14 0
Example 8.1.2 7 = .
1 −4 7 −28

Definition 8.1.3 (Addition) If A = (ai j ) and B = (bi j ) are two m×n matrices. Then A+B =
C where
C = (ci j )
for ci j = ai j + bi j .

Example 8.1.4
! ! !
1 2 3 5 2 3 6 4 6
+ =
1 0 4 −6 2 1 −5 2 5

To save on notation, we will often use Ai j to refer to the i jth entry of the matrix A.

Definition 8.1.5 (The zero matrix) The m × n zero matrix is the m × n matrix having every
entry equal to zero. It is denoted by 0.
!
0 0 0
Example 8.1.6 The 2 × 3 zero matrix is .
0 0 0
8.1. ADDITION AND SCALAR MULTIPLICATION OF MATRICES 131

Note there are 2 × 3 zero matrices, 3 × 4 zero matrices, etc. In fact there is a zero matrix
for every size.

Definition 8.1.7 (Equality of matrices) Let A and B be two matrices. Then A = B means
that the two matrices are of the same size and for A = (ai j ) and B = (bi j ) , ai j = bi j for all
1 ≤ i ≤ m and 1 ≤ j ≤ n.

The following properties of matrices can be easily verified. You should do so. These
properties are called the vector space axioms.

• Commutative Law Of Addition.

A + B = B + A, (8.1)

• Associative Law for Addition.

(A + B) +C = A + (B +C) , (8.2)

• Existence of an Additive Identity

A + 0 = A, (8.3)

• Existence of an Additive Inverse

A + (−A) = 0, (8.4)

Also for α, β scalars, the following additional properties hold.

• Distributive law over Matrix Addition.

α (A + B) = αA + αB, (8.5)

• Distributive law over Scalar Addition

(α + β ) A = αA + β A, (8.6)

• Associative law for Scalar Multiplication

α (β A) = αβ (A) , (8.7)

• Rule for Multiplication by 1.

1A = A. (8.8)
132 CHAPTER 8. MATRICES

8.2 Multiplication of Matrices

As an example, consider the Commutative Law of Addition. Let A + B = C and B + A = D.
Why is D = C?
Ci j = Ai j + Bi j = Bi j + Ai j = Di j .
Therefore, C = D because the i jth entries are the same. Note that the conclusion follows
from the commutative law of addition of numbers.
From now on, we will typically write vectors as columns. Thus, when we write x ∈ Fn
we typically mean  
x1
 . 
x= . 
 . 
xn
We will also use the following convention.
 T  
x1 x1
 .     T
 .  = x1 ··· xn , x1 ··· xn =  ··· 
 
 . 
xn xn

The rules for adding and multiplying by a constant remain the same. To add, you add
corresponding entries and to multiply by a scalar, you multiply every entry by the scalar.
Consider the following system of equations:

x+y = 1
2x − y + z = 2
x+y = 1

Another way to write this is

       
1 1 0 1
x  2  + y  −1  + z  1  =  2 
       

1 1 0 1

That expression on the left is called a linear combination of the three vectors listed there
whenever x, y, z are numbers. Another way to write it is
    
1 1 0 x 1
 2 −1 1   y  =  2 
    

1 1 0 z 1
     
The rows of the above matrix are 1 1 0 , 2 −1 1 , 1 1 0 . The columns
     
1 1 0
of this matrix are  2  ,  −1  ,  1  . It is called a 3 × 3 matrix because it has
     

1 1 0
three rows and three columns. More generally, we have the following definition.
8.2. MULTIPLICATION OF MATRICES 133

Definition 8.2.1 An m × n matrix is a rectangular array of numbers which has m rows and
n columns. We write this as
 
A11 A12 · · · A1n
 A21 A22 · · · A2n 
 
A= .
 .. .. 
 ..

. . 
Am1 An2 · · · Amn

Thus the entry in the ith row and the jth column is denoted as Ai j . As suggested above,
  
A11 A12 · · · A1n x1
 A21 A22 · · · A2n   x2 
  
Ax=  .. .. ..   .. 
 

 . . .  . 
Am1 An2 · · · Amn xn
     
A11 A12 A1n
 A21   A22   A2n 
     
= x1  .  + x2  .  + · · · + xn  . 
    
 ..   ..   .. 


Am1 An2 Amn

Note that Ax is in Fm and the ith entry of this vector Ax is
n
Ai1 x1 + Ai2 x2 + · · · + Ain xn = ∑ Ai j x j .
j=1

In other words, the ith entry of Ax is the dot product of the ith row of A with the vector x.
Symbolically,
(Ax)i = ∑ Ai j x j (8.9)
j

We like to write x to denote an n × 1 matrix which is often called a vector. Then xT will
denote a 1 × n matrix or row vector.
Example 8.2.2  
! x1
1 2 1
 x2 
 
1 0 2
x3
! ! ! !
1 2 1 x1 + 2x2 + x3
= x1 + x2 + x3 =
1 0 2 x1 + 2x3
Note that if A is m × n then Ax is an m × 1 matrix provided x is n × 1. Thus A makes a
vector in Fn into a vector in Fm .
Example 8.2.3 Show the following:
    
1 −1 2 1 5
 3 2 1   2  =  10 
    

2 3 −3 3 −1
134 CHAPTER 8. MATRICES

Example 8.2.4 Write the system of equations

x + 2y − z = 2
x − 3y + z = 1

in the form Ax = b.

According to the above, this system can be written as

 
! x !
1 2 −1   2
 y =
1 −3 1 1
z

The following is the most fundamental observation about multiplying a matrix times a
vector.

Theorem 8.2.5 Let A be an m × n matrix and let x, y be two vectors in Fn with a, b two
scalars. Then
A (ax + by) = aAx + bAy

Proof: By the above definition and the way we add vectors,

(A (ax + by))i = ∑ Ai j (ax j + by j ) = a ∑ Ai j x j + b ∑ Ai j x j

j j j
= a (Ax)i + b (Ay)i = (aAx + bAy)i

Since the ith entries coincide, it follows that A (ax + by) = aAx + bAy as claimed. ■

Definition 8.2.6 Define some special vectors ei as follows:

1 in the ith position

z }| {
 T
ei ≡ 0 ··· 0 1 0 ··· 0

Thus in F3 , we would have

     
1 0 0
e1 =  0  , e2 =  1  , e3 =  0 
     

0 0 1

Observation 8.2.7 Let A be an m × n matrix. Then for ei ∈ Fn , Aei delivers the ith column
of A. To see this,
(Aei )k = ∑ Ak j (ei ) j = Aki
j

because (ei ) j = 0 unless j = i when it is 1. Thus, for k arbitrary, the kth entry of Aei is Aki .
Thus the result of multiplying by ei is
 T
A1i A2i ··· Ani
8.3. LINEAR TRANSFORMATIONS AND MATRICES 135

which is indeed the ith column. Another way to see this is to let
 
A = a1 · · · ai · · · an ,
 
0
 . 
 . 
  . 
Aei = a1 · · · ai · · · an  1  = 1ai = ai
 
 .. 
 
 . 
0

the ith column of A.

8.3 Linear Transformations and Matrices

We can also refer to a linear transformation as a linear function. These are defined as
follows.
Definition 8.3.1 Let T be a function defined on Fn which takes vectors in Fn to vectors in
Fm . This is written as T : Fn → Fm . It is a linear function or equivalently linear trans-
formation if it satisfies the following: For a, b scalars and x, y vectors in Fn it follows
that
T (ax + by) = aT x + bT y
In words: It goes across addition and you can factor out scalars. Then notice that an m × n
matrix A has the property that if x is in Fn then Ax is in Fm and by Theorem 8.2.5, if
T x ≡ Ax for A an m × n matrix, then it follows that T is a linear function.
The following definition defines a linear function and notes that matrix multiplication
gives an example of such a thing. The next theorem shows that this is the only way it can
happen.
Theorem 8.3.2 Let T be a linear transformation, T : Fn → Fm . Then there exists an m × n
matrix A such that for all x ∈ Fn , you have T x = Ax. This matrix is given by
 
T e1 · · · T en
 T
Proof: Let x be arbitrary and x = x1 ··· xn . Then

x = x1 e1 + · · · + xn en
It follows that, since T is linear,
 
x1
.. 
 
T x = T (x1 e1 + · · · + xn en ) = x1 T e1 + · · · + xn T en = T e1 ··· T en 
 . 

xn

and so the matrix which does what is claimed is the one whose ith column is T ei . That is
 
A x = T e1 · · · T en x ■
136 CHAPTER 8. MATRICES

8.4 Multiplication of Matrices

Say you have A an m × n matrix and B an n × p matrix. We want to define an m × p matrix
called AB such that
(AB) x = A (Bx) (8.10)
In other words, we want the linear transformation determined by AB to be the same as
first doing a linear transformation determined by B and then when this has been done,
do the linear transformation determined by A to what you got. In short, we want matrix
multiplication to correspond to composition of linear functions. Then 8.10 is satisfied if
and only if
((AB) x)i ≡ ∑ (AB)il xl = (A (Bx))i = ∑ Aik (Bx)k
l k
!
= ∑ Aik ∑ Bkl xl = ∑ ∑ Aik Bkl xl = ∑ ∑ Aik Bkl xl
k l k l l k

If this is to hold for all choices of x, then it must also hold for x = er . Thus the ith entry
of the rth column is (AB) er and using this, we obtain
!
(AB)ir = ∑ ∑ Aik Bkl (er )l = ∑ Aik Bkr
l k k

Thus if we have the requirement that matrix multiplication corresponds to composition of

the corresponding linear transformations, we are forced to conclude the following definition
for matrix multiplication.
Definition 8.4.1 Let A be m × n and B be n × p. Then AB is m × p and the irth entry of
(AB) is given by
(AB)ir ≡ ∑ Aik Bkr
k
That is, the irth entry is the dot product of the ith row of A with the jth column of B.
Note that from this definition, you must have the number of columns of A equal to
the number of rows of B in order to make any sense of the product. Indeed, this must be
so when you consider matrix multiplication in terms of linear transformations. A linear
transformation T : Fn → Fm is only defined on vectors in Fn .
For A and B matrices, in order to form the product, AB the number of columns of A
must equal the number of rows of B.
these must match!
(m × n)
d (n × p ) = m × p
Note the two outside numbers give the size of the product. Remember:
If the two middle numbers don’t match, you can’t multiply the matrices!
Example 8.4.2 Let
 
! 2 3
1 −1 2
A= , B =  −1 1 
 
3 −2 1
0 3
Then find AB. After this, find BA
8.4. MULTIPLICATION OF MATRICES 137

Consider first AB. It is the product of a 2 × 3 and a 3 × 2 matrix and so it is a 2 × 2

matrix. The top left corner is the dot product of the top row of A and the first column of B
and so forth. Be sure you can show the following.
 
! 11 −8 7
3 8
AB = , BA =  2 −1 −1 
 
8 10
9 −6 3
Note that this shows that matrix multiplication is not commutative. Indeed, it can result in
matrices of different size when you interchange the order. Here is a jucy little observation.
If you add the entries on the main diagonal of both matrices in the above, you get the same
number 13. This is the diagonal from upper left to lower right. You might wonder whether
this always happens or if this is just a fluke.
Although matrix multiplication is not commutative, it does have several very important
properties.
Proposition 8.4.3 If all multiplications and additions make sense, the following hold for
matrices A, B,C and a, b scalars.
A (aB + bC) = a (AB) + b (AC) (8.11)
(B +C) A = BA +CA (8.12)
A (BC) = (AB)C (8.13)
Proof: Using Definition 8.4.1,
(A (aB + bC))i j = ∑ Aik (aB + bC)k j
k


= ∑ Aik aBk j + bCk j
k
= a ∑ Aik Bk j + b ∑ AikCk j
k k
= a (AB)i j + b (AC)i j
= (a (AB) + b (AC))i j .
Thus A (B +C) = AB + AC as claimed. Formula 8.12 is entirely similar.
Formula 8.13 is the associative law of multiplication. Using Definition 8.4.1,
(A (BC))i j = ∑ Aik (BC)k j
k
= ∑ Aik ∑ BklCl j
k l
= ∑ (AB)il Cl j
l
= ((AB)C)i j .
This proves 8.13. ■
Note that the claim about the associative law happens because when you have functions
f , g, h such that it makes sense to take their composition in that order, we have f ◦ (g ◦ h) =
( f ◦ g)◦h and matrix multiplication corresponds to composition of the corresponding linear
functions. This is the real reason for the associative law.
138 CHAPTER 8. MATRICES

8.4.1 The Transpose

Another important operation on matrices is that of taking the transpose. The following
example shows what is meant by this operation, denoted by placing a T as an exponent on
the matrix.  T
1 4 !
1 3 2
 3 1  =
 
4 1 6
2 6
What happened? The first column became the first row and the second column became the
second row. Thus the 3 × 2 matrix became a 2 × 3 matrix. The number 3 was in the second
row and the first column and it ended up in the first row and second column. Here is the
definition.

Definition 8.4.4 Let A be an m × n matrix. Then AT denotes the n × m matrix which is

defined as follows.
AT i j = A ji

In words, the ith row becomes the ith column.

Example 8.4.5  
!T 1 3
1 2 −6
=  2 5 .
 
3 5 4
−6 4

The transpose of a matrix has the following important properties.

Lemma 8.4.6 Let A be an m × n matrix and let B be a n × p matrix. Then

(AB)T = BT AT (8.14)

and if α and β are scalars,

(αA + β B)T = αAT + β BT (8.15)

Proof: From the definition,

 
(AB)T = (AB) ji
ij
= ∑ A jk Bki
k
BT AT



= ∑ ik kj
k
BT AT i j


=

The proof of Formula 8.15 is left as an exercise and this proves the lemma. ■

Definition 8.4.7 An n × n matrix, A is said to be symmetric if A = AT . It is said to be skew

symmetric if A = −AT .
8.5. SOME EXAMPLES OF LINEAR FUNCTIONS ON Rn 139

Example 8.4.8 Let  

2 1 3
A= 1 5 −3  .
 

3 −3 7
Then A is symmetric.

Example 8.4.9 Let  

0 1 3
A =  −1 0 2 
 

−3 −2 0
Then A is skew symmetric.

8.5 Some Examples of Linear Functions on Rn

There are many examples of linear functions and we give a couple next.

8.5.1 Rotations in R2
Sometimes you need to find a matrix which represents a given linear transformation which
is described in geometrical terms. The idea is to produce a matrix which you can multiply
a vector by to get the same thing as some geometrical description. A good example of this
is the problem of rotation of vectors discussed above. Consider the problem of rotating
through an angle of θ .

Example 8.5.1 Determine the matrix which represents the linear transformation defined
by rotating every vector through an angle of θ .
! !
1 0
Let e1 ≡ and e2 ≡ . These identify the geometric vectors which point
0 1
along the positive x axis and positive y axis as shown.
e2
(−sin(θ ), cos(θ )) (cos(θ ), sin(θ ))
T (e1 )
T (e2 )
θ
θ
e1

From the above, you only need to find T e1 and T e2 , the first being the first column of
the desired matrix, A and the second being the second column. From the definition of the
140 CHAPTER 8. MATRICES

cos, sin the coordinates of T (e1 ) are as shown in the picture. The coordinates of T (e2 ) also
follow from simple trigonometry. Thus
! !
cos θ − sin θ
T e1 = , T e2 = .
sin θ cos θ

Therefore, from Theorem 8.3.2,

!
cos θ − sin θ
A=
sin θ cos θ

For those who prefer a more algebraic approach, the definition of (cos (θ ) , sin (θ )) is
as the x and y coordinates of the point (1, 0) . Now the point of the vector from (0, 0) to
(0, 1), e2 is exactly π/2 further along along the unit circle. Therefore, when it is rotated
through an angle of θ the x and y coordinates are given by

(x, y) = (cos (θ + π/2) , sin (θ + π/2)) = (− sin θ , cos θ ) .

Example 8.5.2 Find the matrix of the linear transformation which is obtained by first ro-
tating all vectors through an angle of φ and then through an angle θ . Thus you want the
linear transformation which rotates all angles through an angle of θ + φ .

Let Tθ +φ denote the linear transformation which rotates every vector through an angle
of θ + φ . Then to get Tθ +φ , you could first do Tφ and then do Tθ where Tφ is the linear
transformation which rotates through an angle of φ and Tθ is the linear transformation
which rotates through an angle of θ . Denoting the corresponding matrices by Aθ +φ , Aφ ,
and Aθ , you must have for every x

Aθ +φ x = Tθ +φ x = Tθ Tφ x = Aθ Aφ x.

Consequently, you must have

!
cos (θ + φ ) − sin (θ + φ )
Aθ +φ = = Aθ Aφ
sin (θ + φ ) cos (θ + φ )
! !
cos θ − sin θ cos φ − sin φ
= .
sin θ cos θ sin φ cos φ

You know how to multiply matrices. Do so to the pair on the right. This yields
!
cos (θ + φ ) − sin (θ + φ )
sin (θ + φ ) cos (θ + φ )
!
cos θ cos φ − sin θ sin φ − cos θ sin φ − sin θ cos φ
= .
sin θ cos φ + cos θ sin φ cos θ cos φ − sin θ sin φ

Don’t these look familiar? They are the usual trig. identities for the sum of two angles
derived here using linear algebra concepts.
You do not have to stop with two dimensions. You can consider rotations and other
geometric concepts in any number of dimensions. This is one of the major advantages
8.5. SOME EXAMPLES OF LINEAR FUNCTIONS ON Rn 141

of linear algebra. You can break down a difficult geometrical procedure into small steps,
each corresponding to multiplication by an appropriate matrix. Then by multiplying the
matrices, you can obtain a single matrix which can give you numerical information on the
results of applying the given sequence of simple procedures. That which you could never
visualize can still be understood to the extent of finding exact numerical answers. Another
example follows.

Example 8.5.3 Find the matrix of the linear transformation which is obtained by first ro-
tating all vectors through an angle of π/6 and then reflecting through the x axis.

As shown in Example 8.5.2, the matrix of the transformation which involves rotating
through an angle of π/6 is
!
1
√ !
cos (π/6) − sin (π/6) 2 3 − 12
= 1 1
√
sin (π/6) cos (π/6) 2 2 3

The matrix for the transformation which reflects all vectors through the x axis is
!
1 0
.
0 −1

Therefore, the matrix of the linear transformation which first rotates through π/6 and then
reflects through the x axis is
!
1
√ 1
!
1
√ 1
!
1 0 2 3 − 3 −
1 1
√2 = 2 √2 .
0 −1 2 2 3 − 12 − 12 3

8.5.2 Projections
In Physics it is important to consider the work done by a force field on an object. This
involves the concept of projection onto a vector. Suppose you want to find the projection of
a vector, v onto the given vector, u, denoted by P u (v) This is done using the dot product
as follows. v·u
P u (v) = u
u·u
Because of properties of the dot product, the map v → P u (v) is linear,
 
α v+ β w · u v·u w ·u
P u (α v+ β w) = u=α u+β u
u·u u·u u·u
= αP u (v) + β P u (w) .

Example 8.5.4 Let the projection map be defined above and let u = (1, 2, 3)T . Does this
linear transformation come from multiplication by a matrix? If so, what is the matrix?

You can find this matrix in the same way as in the previous example. Let ei denote the
vector in Rn which has a 1 in the ith position and a zero everywhere else. Thus a typical
vector, x = (x1 , · · · , xn )T can be written in a unique way as
n
x= ∑ x je j.
j=1
142 CHAPTER 8. MATRICES

From the way you multiply a matrix by a vector, it follows that P u (ei ) gives the ith column
of the desired matrix. Therefore, it is only necessary to find
 e ·u 
i
P u (ei ) ≡ u
u·u
For the given vector in the example, this implies the columns of the desired matrix are
     
1 1 1
1   2   3 
 2 ,  2 ,  2 .

14 14 14
3 3 3

Hence the matrix is  

1 2 3
1 
 2 4 6 .

14
3 6 9

8.5.3 Rotations About A Particular Vector

The problem is to find the matrix of the linear transformation which rotates all vectors
about a given unit vector u which is possibly not one of the coordinate vectors i, j, or k.
Suppose for |c| ̸= 1 p
u = (a, b, c) , a2 + b2 + c2 = 1.
First I will produce a matrix which maps u to k such that the right handed rotation
about k corresponds to the right handed rotation about u. Then I will rotate about k and
finally, I will multiply by the inverse of the first matrix to get the desired result.
To begin, find vectors w, v such that w × v = u. Let
 
b a
w = −√ ,√ ,0 .
a2 + b2 a2 + b2

w u

This vector is clearly perpendicular to u. Then v = (a, b, c) × w ≡ u × w. Thus from

the geometric description of the cross product, w × v = u. Computing the cross product
gives  
b a
v = (a, b, c) × − √ ,√ ,0
a2 + b2 a2 + b2
!
a b a2 b2
= −c p , −c p ,p +p
(a2 + b2 ) (a2 + b2 ) (a2 + b2 ) (a2 + b2 )
Now I want to have T w = i, T v = j, T u = k. What does this? It is the inverse of the
matrix which takes i to w, j to v, and k to u. This matrix is
8.5. SOME EXAMPLES OF LINEAR FUNCTIONS ON Rn 143

b
−q c
 
−√ a a
 a2 +b2 (a2 +b2 ) 
a
√ −q c
 
 b b .
 a2 +b2 (a2 +b2 ) 
 2 2

0 √a +b c
a2 +b2

Its inverse is

−q 1 1
 
b q a 0
 (a2 +b2 ) ( )
a2 +b2 
p
−q c a −q c b (a2 + b2 ) 
 

 (a2 +b2 ) (a2 +b2 ) 
a b c

Therefore, the matrix which does the rotating is

− √ 2b 2 − q c
 
a a  
 a +b (a +b )
2 2
 cos θ − sin θ 0
 √ a − c
 
b b  0 ·
  sin θ cos θ
q 
 a2 +b2 (a 2 +b 2 )
 2
√a +b
2
 0 0 1
0 2 2
c
a +b

−q 1 1
 
b q a 0
 (a2 +b2 ) ( )
a2 +b2 
p
−q c a −q c b (a2 + b2 ) 
 

 (a2 +b2 ) (a2 +b2 ) 
a b c
This yields a matrix whose columns are
b2 cos θ +c2 a2 cos θ +a4 +a2 b2
 
a2 +b2
−ba cos θ +cb2 sin θ +ca2 sin θ +c2 ab cos θ +ba3 +b3 a
,
 
a2 +b2

− (sin θ ) b − (cos θ ) ca + ca
−ba cos θ −ca2 sin θ −cb2 sin θ +c2 ab cos θ +ba3 +b3 a
 
a2 +b2
a2 cos θ +c2 b2 cos θ +a2 b2 +b4
,
 
a2 +b2

(sin θ ) a − (cos θ ) cb + cb
 
(sin θ ) b − (cos θ ) ca + ca
 − (sin θ ) a − (cos θ ) cb + cb 
 

a2 + b2 cos θ + c2

Using the assumption that u is a unit vector so that a2 + b2 + c2 = 1, it follows the

desired matrix has the following as columns.
   
cos θ − a2 cos θ + a2 −ba cos θ + ba − c sin θ
 −ba cos θ + ba + c sin θ  ,  −b2 cos θ + b2 + cos θ 
   

− (sin θ ) b − (cos θ ) ca + ca (sin θ ) a − (cos θ ) cb + cb

144 CHAPTER 8. MATRICES

 
(sin θ ) b − (cos θ ) ca + ca
 − (sin θ ) a − (cos θ ) cb + cb 
 

1 − c2 cos θ + c2

This was done under the assumption that |c| = ̸ 1. However, if this condition does not
hold, you can verify directly that the above still gives the correct answer.

8.6 The Inverse of a Matrix

8.6.1 The Identity And Inverses
There is a special matrix called I and referred to as the identity matrix. It is always a square
matrix, meaning the number of rows equals the number of columns and it has the property
that there are ones down the main diagonal and zeroes elsewhere. Here are some identity
matrices of various sizes.
 
  1 0 0 0
! 1 0 0
1 0   0 1 0 0 
 
(1) , , 0 1 0 , .

0 1  0 0 1 0 
0 0 1
0 0 0 1

The first is the 1 × 1 identity matrix, the second is the 2 × 2 identity matrix, the third is the
3 × 3 identity matrix, and the fourth is the 4 × 4 identity matrix. By extension, you can
likely see what the n × n identity matrix would be. It is so important that there is a special
symbol to denote the i jth entry of the identity matrix Ii j = δ i j where δ i j is the Kronecker
symbol defined by (
1 if i = j
δij =
0 if i ̸= j
It is called the identity matrix because it is a multiplicative identity in the following
sense.

Lemma 8.6.1 Suppose A is an m × n matrix and In is the n × n identity matrix. Then

AIn = A. If Im is the m × m identity matrix, it also follows that Im A = A.

Proof:
(AIn )i j = ∑ Aik δ k j = Ai j
k

and so AIn = A. The other case is left as an exercise for you. ■

Definition 8.6.2 An n × n matrix A has an inverse, A−1 if and only if AA−1 = A−1 A = I.
Such a matrix is called invertible.

It is very important to observe that the inverse of a matrix, if it exists, is unique. Another
way to think of this is that if it acts like the inverse, then it is the inverse.

Theorem 8.6.3 Suppose A−1 exists and AB = BA = I. Then B = A−1 .

8.6. THE INVERSE OF A MATRIX 145

Proof:
A−1 = A−1 I = A−1 (AB) = A−1 A B = IB = B. ■

Unlike ordinary multiplication of numbers, it can happen that A ̸= 0 but A may fail to
have an inverse. This is illustrated in the following example.
!
1 1
Example 8.6.4 Let A = . Does A have an inverse?
1 1

One might think A would have an inverse because it does not equal zero. However,
! ! !
1 1 −1 0
=
1 1 1 0

and if A−1 existed, this could not happen because you could write
! !! !!
0 −1 0 −1 −1
=A =A A =
0 0 1
! ! !
−1

−1 −1 −1
= A A =I = ,
1 1 1
a contradiction. Thus the answer is that A does not have an inverse.

! !
1 1 2 −1
Example 8.6.5 Let A = . Show is the inverse of A.
1 2 −1 1

To check this, multiply

! ! !
1 1 2 −1 1 0
=
1 2 −1 1 0 1

and ! ! !
2 −1 1 1 1 0
=
−1 1 1 2 0 1
showing that this matrix is indeed the inverse of A.

8.6.2 Finding The Inverse Of A Matrix

!
x z
In the last example, how would you find A−1 ? You wish to find a matrix such
y w
that ! ! !
1 1 x z 1 0
= .
1 2 y w 0 1
This requires the solution of the systems of equations,

x + y = 1, x + 2y = 0
146 CHAPTER 8. MATRICES

and
z + w = 0, z + 2w = 1.
Writing the augmented matrix for these two systems gives
!
1 1 | 1
(8.16)
1 2 | 0

for the first system and !

1 1 | 0
(8.17)
1 2 | 1
for the second. Lets solve the first system. Take (−1) times the first row and add to the
second to get !
1 1 | 1
0 1 | −1
Now take (−1) times the second row and add to the first to get
!
1 0 | 2
.
0 1 | −1

Putting in the variables, this says x = 2 and y = −1.

Now solve the second system, 8.17 to find z and w. Take (−1) times the first row and
add to the second to get !
1 1 | 0
.
0 1 | 1
Now take (−1) times the second row and add to the first to get
!
1 0 | −1
.
0 1 | 1

Putting in the variables, this says z = −1 and w = 1. Therefore, the inverse is

!
2 −1
.
−1 1

Didn’t the above seem rather repetitive? Note that exactly the same row operations
were used in both systems. In each case, the end result was something of the ! form (I|v)
x
where I is the identity and v gave a column of the inverse. In the above, , the first
y
!
z
column of the inverse was obtained first and then the second column .
w
To simplify this procedure, you could have written
!
1 1 | 1 0
1 2 | 0 1
8.6. THE INVERSE OF A MATRIX 147

and row reduced till you obtained

!
1 0 | 2 −1
0 1 | −1 1
and read off the inverse as the 2 × 2 matrix on the right side.
This is the reason for the following simple procedure for finding the inverse of a matrix.
This procedure is called the Gauss-Jordan procedure.
PROCEDURE 8.6.6 Suppose A is an n × n matrix. To find A−1 if it exists, form
the augmented n × 2n matrix
(A|I)
and then, if possible do row operations until you obtain an n × 2n matrix of the form
(I|B) . (8.18)
When this has been done, B = A−1 . If it is impossible to row reduce to a matrix of the form
(I|B) , then A has no inverse.
Actually, all this shows is how to find a right inverse if it exists. What has been shown
from the above discussion is that AB = I. Later, I will show that this right inverse is the
inverse. See Corollary 28.1.15 presented later. However, it is not hard to see that this should
be the case as follows.
The row operations are all reversible. If the row operation involves switching two rows,
the reverse row operation involves switching them again to get back to where you started.
If the row operation involves multiplying a row by a ̸= 0, then you would get back to where
you began by multiplying the row by 1/a. The third row operation involving addition of c
times row i to row j can be reversed by adding −c times row i to row j.
In the above procedure, a sequence of row operations applied to I yields B while the
same sequence of operations applied to A yields I. Therefore, the sequence of reverse row
operations in the opposite order applied to B will yield I and applied to I will yield A. That
is, there are row operations which provide
(B|I) → (I|A)
and as just explained, A must be a right inverse for B. Therefore, BA = I. Hence B is both
a right and a left inverse for A because AB = BA = I.
If it is impossible to row reduce (A|I) to get (I|B) , then in particular, it is impossible to
row reduce A to I and consequently impossible to do a sequence of row operations to I and
get A. Later it will be made clear that the only
! way this can happen is that it is possible to
C
row reduce A to a matrix of the form where 0 is a row of zeros. Then there will be
0
no solution to the system of equations represented by the augmented matrix
!
C 0
|
0 1
Using the reverse row operations in the opposite order on both matrices in the above, it
follows that there must exist a such that there is no solution to the system of equations
represented by (A|a). Hence A fails to have an inverse, because if it did, then there would
be a solution x to the equation Ax = a given by A−1 a.
148 CHAPTER 8. MATRICES
 
1 2 2
Example 8.6.7 Let A =  1 0 2 . Find A−1 if it exists.
 

3 1 −1
Set up the augmented matrix (A|I)
 
1 2 2 | 1 0 0
 1 0 2 | 0 1 0 
 

3 1 −1 | 0 0 1
Next take (−1) times the first row and add to the second followed by (−3) times the first
row added to the last. This yields
 
1 2 2 | 1 0 0
 0 −2 0 | −1 1 0  .
 

0 −5 −7 | −3 0 1
Then take 5 times the second row and add to -2 times the last row.
 
1 2 2 | 1 0 0
 0 −10 0 | −5 5 0 
 

0 0 14 | 1 5 −2
Next take the last row and add to (−7) times the top row. This yields
 
−7 −14 0 | −6 5 −2
 0 −10 0 | −5 5 0 .
 

0 0 14 | 1 5 −2
Now take (−7/5) times the second row and add to the top.
 
−7 0 0 | 1 −2 −2
 0 −10 0 | −5 5 0 .
 

0 0 14 | 1 5 −2
Finally divide the top row by -7, the second row by -10 and the bottom row by 14 which
yields  
1 0 0 | − 71 2
7
2
7
 
 
1
 0 1 0 | − 12 0 .
 
 2 
 1 5

1 
 0 0 1 | −7
14 14

Therefore, the inverse is  

− 71 2
7
2
7
 
 
1
− 12 0 
 

 2 
 1 5


14 14 − 17 
8.6. THE INVERSE OF A MATRIX 149
 
1 2 2
Example 8.6.8 Let A =  1 0 2 . Find A−1 if it exists.
 

2 2 4

Write the augmented matrix (A|I)

 
1 2 2 | 1 0 0
 1 0 2 | 0 1 0 
 

2 2 4 | 0 0 1

and proceed to do row operations attempting to obtain I|A−1 . Take (−1) times the top

row and add to the second. Then take (−2) times the top row and add to the bottom.
 
1 2 2 | 1 0 0
 0 −2 0 | −1 1 0 
 

0 −2 0 | −2 0 1

Next add (−1) times the second row to the bottom row.
 
1 2 2 | 1 0 0
 0 −2 0 | −1 1 0 
 

0 0 0 | −1 −1 1

At this point, you can see there will be no inverse because you have obtained a row of zeros
in the left half of the augmented matrix (A|I) . Thus there will be no way to obtain I on the
left.
 
1 0 1
Example 8.6.9 Let A =  1 −1 1 . Find A−1 if it exists.
 

1 1 −1

▶▶
Form the augmented matrix
 
1 0 1 | 1 0 0
 1 −1 1 | 0 1 0 .
 

1 1 −1 | 0 0 1

Now do row operations until the n × n matrix on the left becomes the identity matrix. This
yields after some computations,
1 1
 
1 0 0 | 0 2 2
 
 0 1 0 | 1 −1 0 
 

− 21 − 12
 
0 0 1 | 1
150 CHAPTER 8. MATRICES

and so the inverse of A is the matrix on the right,

1 1
 
0 2 2
 
 1 −1 0 .
 

1 − 12 − 12
 

Checking the answer is easy. Just multiply the matrices and see if it works.
1 1
 
  0  
2 2
1 0 1   1 0 0
 1 −1 1   1 −1 0  =  0 1 0  .
    

1 − 12 − 12
 
1 1 −1 0 0 1

Always check your answer because if you are like some of us, you will usually have made
a mistake.
Example 8.6.10 In this example, it is shown how to use the inverse of a matrix to find the
solution to a system of equations. Consider the following system of equations. Use the
inverse of a suitable matrix to give the solutions to this system.
 
x+z = 1
 x−y+z = 3 .
 

x+y−z = 2
The system of equations can be written in terms of matrices as
    
1 0 1 x 1
 1 −1 1   y  =  3  . (8.19)
    

1 1 −1 z 2

More simply, this is of the form Ax = b. Suppose you find the inverse of the matrix A−1 .
Then you could multiply both sides of this equation by A−1 to obtain
x = A−1 A x = A−1 (Ax) = A−1 b.

This gives the solution as x = A−1 b. Note that once you have found the inverse, you can
easily get the solution for different right hand sides without any effort. It is always just
A−1 b. In the given example, the inverse of the matrix is
 
1 1
0 2 2
 1 −1 0 
 

1 − 12 − 21
This was shown in Example 8.6.9. Therefore, from what was just explained, the solution
to the given system is
      
1 1 5
x 0 2 2 1 2
 y  =  1 −1 0   3  =  −2  .
      

z 1 − 12 − 21 2 − 23
8.7. MATLAB AND MATRIX ARITHMETIC 151

 T
What if the right side of 8.19 had been 0 1 3 ? What would be the solution to
    
1 0 1 x 0
 1 −1 1   y  =  1 ?
    

1 1 −1 z 3

By the above discussion, it is just

    
 
1 1
x 0 2 2 0 2
 y = 1 −1 0   1  =  −1  .
      

z 1 − 21 − 12 3 −2

This illustrates why once you have found the inverse of a given matrix, you can use it to
solve many different systems easily.

8.7 MATLAB And Matrix Arithmetic

To find the inverse of a square matrix in matlab, you open it and type the following. The
>> will already be there.
>>inv([1,2,3;5,2,7;8,2,1]) Then press enter and it will give the following:
ans =
-0.1667 0.0556 0.1111
0.7083 -0.3194 0.1111
-0.0833 0.1944 -0.1111
Note how it computed the inverse in decimals. If you want the answer in terms of
fractions, you do the following:
>>inv(sym([1,2,3;5,2,7;8,2,1])) Then press enter and it will give the following:
ans =
[ -1/6, 1/18, 1/9]
[ 17/24, -23/72, 1/9]
[ -1/12, 7/36, -1/9]
You can do other things as well. Say you have
>>A=[1,2,3;5,2,7;8,2,1];B=[3,2,-5;3,11,2;-3,-1,5];
C=[1,2;4,-3;7,3];D=[1,2,3;-3,2,1];
T
This defines some matrices. Then suppose you wanted to find A−1 DT + BC . You
would then type
transpose(inv(sym(A))*transpose(D)+B*C) or (inv(sym(A))*D’+B*C)’
and press enter. This gives
ans =
[ -427/18, 4421/72, 1007/36]
[ -257/18, -1703/72, 451/36]
In matlab, A’ means ĀT the conjugate transpose of A. Since everything is real here, this
reduces to the transpose.
To get to a new line in matlab, you need to press shift enter. Notice how a ; was placed
after the definition of A, B,C, D. This tells matlab that you have defined something but not
to say anything about it. If you don’t do this, then when you press return, it will list the
matrices and you don’t want to see that. You just want the answer. When you have done
152 CHAPTER 8. MATRICES

a computation in matlab, you ought to go to >> and type “clear all” and then enter. That
way, you can use the symbols again with different definition. If you don’t do the “clear all”
thing, it will go on thinking that A is what you defined earlier.

8.8 Exercises
1. Here are some matrices:
! !
1 2 3 3 −1 2
A = ,B = ,
2 1 7 −3 2 1
! ! !
1 2 −1 2 2
C = ,D = ,E = .
3 1 2 −3 3
Find if possible −3A, 3B − A, AC,CB, AE, EA. If it is not possible explain why.
2. Here are some matrices:
 
1 2 !
2 −5 2
A =  3 2 ,B = ,
 
−3 2 1
1 −1
! ! !
1 2 −1 1 1
C = ,D = ,E = .
5 0 4 −3 3
Find if possible −3A, 3B − A, AC,CA, AE, EA, BE, DE. If it is not possible explain
why.
3. Here are some matrices:
 
1 2 !
2 −5 2
A =  3 2 ,B = ,
 
−3 2 1
1 −1
! ! !
1 2 −1 1 1
C = ,D = ,E = .
5 0 4 −3 3

Find if possible −3AT , 3B − AT , AC,CA, AE, E T B, BE, DE, EE T , E T E. If it is not

possible explain why.
4. Here are some matrices:
 
1 2 !
2 −5 2
A =  3 2 ,B = ,
 
−3 2 1
1 −1
! ! !
1 2 −1 1
C = ,D = ,E = .
5 0 4 3
Find the following if possible and explain why it is not possible if this is the case.
AD, DA, DT B, DT BE, E T D, DE T .
8.8. EXERCISES 153
   
1 1 ! 1 1 −3
1 −1 −2
5. Let A =  −2 −1 , B = , and C =  −1 2 0 .
   
2 1 −2
1 2 −3 −1 0
Find if possible.

(a) AB
(b) BA
(c) AC
(d) CA
(e) CB
(f) BC

6. Suppose A and B are square matrices of the same size. Which of the following are
correct?

(a) (A − B)2 = A2 − 2AB + B2

(b) (AB)2 = A2 B2
(c) (A + B)2 = A2 + 2AB + B2
(d) (A + B)2 = A2 + AB + BA + B2
(e) A2 B2 = A (AB) B
(f) (A + B)3 = A3 + 3A2 B + 3AB2 + B3
(g) (A + B) (A − B) = A2 − B2
!
−1 −1
7. Let A = . Find all 2 × 2 matrices, B such that AB = 0.
3 3

8. Let x = (−1, −1, 1) and y = (0, 1, 2) . Find xT y and xy T if possible.

! !
1 2 1 2
9. Let A = ,B = . Is it possible to choose k such that AB = BA?
3 4 3 k
If so, what should k equal?
! !
1 2 1 2
10. Let A = ,B = . Is it possible to choose k such that AB = BA?
3 4 1 k
If so, what should k equal?
11. In 8.1 - 8.8 describe −A and 0.
12. Let A be an n × n matrix. Show A equals the sum of a symmetric and a skew sym-
metric matrix. (M is skew
symmetric if M = −M T . M is symmetric if M T = M.)
Hint: Show that 21 AT + A is symmetric and then consider using this as one of the
matrices.
13. Show every skew symmetric matrix has all zeros down the main diagonal. The main
diagonal consists of every entry of the matrix which is of the form aii . It runs from
the upper left down to the lower right.
154 CHAPTER 8. MATRICES

14. Suppose M is a 3 × 3 skew symmetric matrix. Show there exists a vector Ω such that
for all u ∈ R3
Mu = Ω × u
Hint: Explain why, since M is skew symmetric it is of the form
 
0 −ω 3 ω 2
M =  ω3 0 −ω 1 
 

−ω 2 ω 1 0

where the ω i are numbers. Then consider ω 1 i + ω 2 j + ω 3 k.

15. Using only the properties 8.1 - 8.8 show −A is unique.

16. Using only the properties 8.1 - 8.8 show 0 is unique.
17. Using only the properties 8.1 - 8.8 show 0A = 0. Here the 0 on the left is the scalar 0
and the 0 on the right is the zero for m × n matrices.

18. Using only the properties 8.1 - 8.8 and previous problems show (−1) A = −A.
19. Prove 8.15.
20. Prove that Im A = A where A is an m × n matrix.

21. Give an example of matrices, A, B,C such that B ̸= C, A ̸= 0, and yet AB = AC.
22. Suppose AB = AC and A is an invertible n × n matrix. Does it follow that B = C?
Explain why or why not. What if A were a non invertible n × n matrix?
23. Find your own examples:

(a) 2 × 2 matrices, A and B such that A ̸= 0, B ̸= 0 with AB ̸= BA.

(b) 2 × 2 matrices, A and B such that A ̸= 0, B ̸= 0, but AB = 0.
(c) 2 × 2 matrices, A, D, and C such that A ̸= 0,C ̸= D, but AC = AD.

24. Explain why if AB = AC and A−1 exists, then B = C.

25. Give an example of a matrix A such that A2 = I and yet A ̸= I and A ̸= −I.
26. Give an example of matrices, A, B such that neither A nor B equals zero and yet
AB = 0.
27. Give another example other than the one given in this section of two square matrices,
A and B such that AB ̸= BA.
28. Let !
2 1
A= .
−1 3

Find A−1 if possible. If A−1 does not exist, determine why.

8.8. EXERCISES 155

29. Let !
0 1
A= .
5 3

Find A−1 if possible. If A−1 does not exist, determine why.

30. Let !
2 1
A= .
3 0

Find A−1 if possible. If A−1 does not exist, determine why.

31. Let !
2 1
A= .
4 2

Find A−1 if possible. If A−1 does not exist, determine why.

!
a b
32. Let A be a 2 × 2 matrix which has an inverse. Say A = . Find a formula
c d
for A−1 in terms of a, b, c, d.
33. Let  
1 2 3
A= 2 1 4 .
 

1 0 2
Find A−1 if possible. If A−1 does not exist, determine why.

34. Let  
1 0 3
A= 2 3 4 .
 

1 0 2
Find A−1 if possible. If A−1 does not exist, determine why.
35. Let  
1 2 3
A= 2 1 4 .
 

4 5 10
Find A−1 if possible. If A−1 does not exist, determine why.
36. Let  
1 2 0 2
 1 1 2 0 
A=
 
−3 2 

 2 1
1 2 1 2
Find A−1 if possible. If A−1 does not exist, determine why.
156 CHAPTER 8. MATRICES

   
x1 − x2 + 2x3 x1
 2x3 + x1   x2 
37. Write   in the form A   where A is an appropriate matrix.
   
 3x3   x3 
3x4 + 3x2 + x1 x4
   
x1 + 3x2 + 2x3 x1
 2x3 + x1   x2 
38. Write   in the form A   where A is an appropriate matrix.
   
 6x3   x3 
x4 + 3x2 + x1 x4
   
x1 + x2 + x3 x1
 2x3 + x1 + x2   x2 
39. Write   in the form A   where A is an appropriate matrix.
   
 x3 − x1   x3 
3x4 + x1 x4

40. Using the inverse of the matrix, find the solution to the systems
         
1 0 3 x 1 1 0 3 x 2
 2 3 4  y  =  2 ,  2 3 4  y  =  1 
         

1 0 2 z 3 1 0 2 z 0
         
1 0 3 x 1 1 0 3 x 3
 2 3 4   y  =  0  ,  2 3 4   y  =  −1  .
         

1 0 2 z 1 1 0 2 z −2

Now give the solution in terms of a, b, and c to

    
1 0 3 x a
 2 3 4  y  =  b .
    

1 0 2 z c

41. Using the inverse of the matrix, find the solution to the systems
         
1 0 3 x 1 1 0 3 x 2
2 3 4 y = 2 , 2 3 4 =
   1 
y
         
     
1 0 2 z 3 1 0 2 z 0
         
1 0 3 x 1 1 0 3 x 3
 2 3 4   y  =  0  ,  2 3 4   y  =  −1  .
         

1 0 2 z 1 1 0 2 z −2

Now give the solution in terms of a, b, and c to

    
1 0 3 x a
 2 3 4  y   b . =
    

1 0 2 z c
8.8. EXERCISES 157

42. Using the inverse of the matrix, find the solution to the system
1 1 1
    
−1 2 2 2 x a
1 1 5 
 3
2 −2 −2  y   b   
= .
 
 −1 0
 
0 1  z   c 
−2 − 34 1
4
9
4 w d

43. Show that if A is an n × n invertible matrix and x is a n × 1 matrix such that Ax = b

for b an n × 1 matrix, then x = A−1 b.
44. Prove that if A−1 exists and Ax = 0 then x = 0.
45. Show that if A−1 exists for an n × n matrix, then it is unique. That is, if BA = I and
AB = I, then B = A−1 .

−1
T
46. Show that if A is an invertible n × n matrix, then so is AT and AT = A−1 .

47. Show (AB)−1 = B−1 A−1 by verifying that AB B−1 A−1 = I and

B−1 A−1 (AB) = I.

Hint: Use Problem 45.

48. Show that (ABC)−1 = C−1 B−1 A−1 by verifying that

(ABC) C−1 B−1 A−1 = I

and C−1 B−1 A−1 (ABC) = I. Hint: Use Problem 45.

−1
2
49. If A is invertible, show A2 = A−1 . Hint: Use Problem 45.

−1
50. If A is invertible, show A−1 = A. Hint: Use Problem 45.
51. Let A and be a real m × n matrix and let x ∈ Rn and y ∈ Rm . Show

(Ax, y)Rm = x,AT y Rn

where (·, ·)Rk denotes the dot product in Rk . In the notation above, A x · y = x· AT y.
Use the definition of matrix multiplication to do this.
52. Use the result of Problem 51 to verify directly that (AB)T = BT AT without making
any reference to subscripts.
53. Suppose A is an n × n matrix and for each j,
n
∑ Ai j < 1
i=1

Show that the infinite series ∑∞ k th

k=0 A converges in the sense that the i j entry of
n
the partial sums converge for each i j. Hint: Let R ≡ max j ∑i=1 Ai j . Thus R < 1.
Show that A2 i j ≤ R2 . Then generalize to show that (Am )i j ≤ Rm . Use this to


158 CHAPTER 8. MATRICES

show that the i jth entry of the partial sums is a Cauchy sequence. From calculus,
these converge by completeness of the real or complex numbers. Next show that
(I − A)−1 = ∑∞ k
k=0 A . The Leontief model in economics involves solving an equation
for x of the form
x = Ax + b, or (I − A) x = b
The vector Ax is called the intermediate demand and the vectors Ak x have economic
meaning. From the above,

x = Ib + Ab + A2 b + · · ·

The series is also called the Neuman series. It is important in functional analysis.
54. An elementary matrix is one which results from doing a row operation to the identity
matrix. Thus the elementary matrix E which results from adding a times the ith row
to the jth row would have aδ ik + δ jk as the jkth entry and all other rows would be
unchanged. That is δ rs provided r ̸= j. Show that multiplying this matrix on the
left of an appropriate sized matrix A results in doing the row operation to the matrix
A. You might also want to verify that the other elementary matrices have the same
effect, doing the row operation which resulted in the elementary matrix to A.

55. Let a be a fixed vector. The function Ta defined by Ta v = a + v has the effect of
translating all vectors by adding a. Show this is not a linear transformation. Explain
why it is not possible to realize Ta in R3 by multiplying by a 3 × 3 matrix.
56. In spite of Problem 55 we can represent both translations and rotations by matrix
multiplication at the expense of using higher dimensions. This is done by the homo-
geneous coordinates. I will illustrate in R3 where most interest in this is found. For
each vector v = (v1 , v2 , v3 )T , consider the vector in R4 (v1 , v2 , v3 , 1)T . What happens
when you do
  
1 0 0 a1 v1
 0 1 0 a  v 
2  2 
?

 
 0 0 1 a3   v3 
0 0 0 1 1
Describe how to consider both rotations and translations all at once by forming ap-
propriate 4 × 4 matrices.
Chapter 9

Subspaces Spans and Bases

The span of some vectors consists of all linear combinations of these vectors. A linear
combination of vectors is just a finite sum of scalars times vectors.
Definition 9.0.1 Let u1 , · · · , u p be some vectors in Fn . A linear combination of these


vectors is a sum of the following form:

p
∑ a k uk
k=1

That is, it is a sum of scalars times the vectors for some choice of scalars a1 , · · · , a p .
span (u1 , · · · , u p ) denotes the set of all linear combinations of these vectors.
Observation 9.0.2 Let u1 , · · · , u p be vectors in Fn . Form the n × p matrix

 
A ≡ u1 · · · u p

which has these vectors as columns. Then span (u1 , · · · , u p ) consists of all vectors which
are of the form
Ax for x ∈ F p .
Recall why this is so. A typical thing in what was just described is
 
x1
.. 
 
u1 · · · u p   .  = x1 u1 + · · · + xn u p

xp
In other words, a typical vector of the form Ax is a linear combination of the columns of A.
Thus we can write either span (u1 , · · · , u p ) or all Ax for x ∈ F p to denote the same thing.
Definition 9.0.3 The vectors Ax where x ∈ F p is also called the column space of A and
also Im (A) meaning image of A, also denoted as A (Fn ). Thus column space equals
span (u1 , · · · , u p ) where the ui are the columns of A.
What do you really mean when you say there is a solution x to a linear system of
equations Ax = b? You mean that b is in the span of the columns of A. After all, if A =
   T
u1 · · · u p , you are looking for x = x1 · · · x p such that x1 u1 + x2 u2 +
· · · + x p u p = Ax = b.

159
160 CHAPTER 9. SUBSPACES SPANS AND BASES

9.1 Subspaces
A subspace is a set of vectors with the property that linear combinations of these vectors
remain in the set. Geometrically, subspaces are like lines and planes which contain the
origin. More precisely, the following definition is the right way to think of this.

Definition 9.1.1 Let V be a nonempty collection of vectors in Fn . Then V is called a

subspace if whenever α, β are scalars and u, v are vectors in V, the linear combination
αu + β v is also in V .

There is no substitute for the above definition or equivalent algebraic definition! How-
ever, it is sometimes helpful to look at pictures at least initially. The following are four
subsets of R2 . The first is the shaded area between two lines which intersect at the origin,
the second is a line through the origin, the third is the union of two lines through the origin,
and the last is the region between two rays from the origin. Note that in the last, multipli-
cation of a vector in the set by a nonnegative scalar results in a vector in the set as does the
sum of two vectors in the set. However, multiplication by a negative scalar does not take a
vector in the set to another in the set.

not subspace subspace not subspace not subspace

Observe how the above definition indicates that the claims posted on the picture are
valid. Now here are the two main examples of subspaces.

Theorem 9.1.2 Let A be an m × n matrix. Then Im (A) is a subspace of Fm . Also let

ker (A) ≡ N (A) ≡ {x ∈ Fn such that Ax = 0}

Then ker (A) is a subspace of Fn .

Proof: Suppose Axi is in Im (A) and a, b are scalars. Does it follow that aAx1 + bAx2
is in Im (A)? The answer is yes because

aAx1 + bAx2 = A (ax1 + bx2 ) ∈ Im (A)

this because of the above properties of matrix multiplication. Note that A0 = 0 so 0 ∈

Im (A) and so Im (A) ̸= 0.
/
Now suppose x, y are both in N (A) and a, b are scalars. Does it follow that ax +
by ∈ N (A)? The answer is yes because

A (ax + by) = aAx + bAy = a0 + b0 = 0.

Thus the condition is satisfied. Of course N (A) ̸= 0/ because A0 = 0. ■

Subspaces are exactly those subsets of Fn which are themselves vector spaces. Recall
that a vector space is something which satisfies the vector space axioms on Page 55.
9.1. SUBSPACES 161

Proposition 9.1.3 Let V be a nonempty collection of vectors in Fn . Then V is a subspace

if and only if V is itself a vector space having the same operations as those defined on Fn .

Proof: Suppose first that V is a subspace. It is obvious all the algebraic laws hold on V
because it is a subset of Fn and they hold on Fn . Thus u + v = v + u along with the other
axioms. Does V contain 0? Yes because it contains 0u = 0. Are the operations defined
on V ? That is, when you add vectors of V do you get a vector in V ? When you multiply a
vector in V by a scalar, do you get a vector in V ? Yes. This is contained in the definition.
Does every vector in V have an additive inverse? Yes because − v = (−1) v which is given
to be in V provided v ∈ V .
Next suppose V is a vector space. Then by definition, it is closed with respect to linear
combinations. Hence it is a subspace. ■
There is a fundamental result in the case where m < n. In this case, the matrix A of the
linear transformation looks like the following.

Theorem 9.1.4 Let A be an m × n matrix where m < n. Then N (A) contains nonzero vec-
tors.

Proof: First consider the case where A is a 1 × n matrix for n > 1. Say
 
A = a1 · · · an

If a1 = 0, consider the vector x = e1 . If a1 ̸= 0, let

 
b
 1 
 
x=  .. 

 . 
1

where b is chosen to satisfy the equation

n
a1 b + ∑ ak = 0
k=2

Suppose now that the theorem is true for any m × n matrix with n > m and consider an
(m × 1) × n matrix A where n > m + 1. If the first column of A is 0, then you could let
x = e1 as above. If the first column is not the zero vector, then by doing row operations,
the equation Ax = 0 can be reduced to the equivalent system

A1 x = 0

where A1 is of the form !

1 aT
A1 =
0 B
162 CHAPTER 9. SUBSPACES SPANS AND BASES

where B is an m × (n − 1) matrix. Since n > m + 1, it follows that (n − 1) > m and so

by induction, there exists a nonzero vector y ∈ Fn−1 such that By = 0. Then consider the
vector !
b
x=
y
 
bT1
 . 
A1 x has for its top entry the expression b+aT y. Letting B =  .  th
 .  , the i entry of A1 x
bTm
T
for i > 1 is of the form bi y = 0. Thus if b is chosen to satisfy the equation b + aT y = 0,
then A1 x = 0.■
Now here is a very fundamental definition.

Definition 9.1.5 Let {u1 , · · · , ur } be vectors in F p . They are independent if and only if the
only solution to the system of equations
 
u1 · · · ur x = 0

is x = 0. In other words the vectors are independent means that whenever

r
∑ xi ui = 0
i=1

it follows that each xi = 0. The set of vectors is dependent if it is not independent. Thus
Theorem 9.1.4 says that if you have more than n vectors in Fn this set of vectors will be
dependent.

With this preparation, here is a major theorem.

Theorem 9.1.6 Suppose you have vectors {u1 , · · · , ur } and that this set of vectors is in-
dependent. Suppose also that there are vectors {v 1 , · · · , v s } and that each u j is a linear
combination of the vectors {v 1 , · · · , v s } . Then r ≤ s. A little less precisely, spanning sets
are at least as long as linearly independent sets.

Proof: Let ui = ∑sj=1 a ji v j . This is merely giving names to the scalars in the linear
combination which yields ui . Now suppose that s < r. Then if A is the matrix which has a ji
in the jth row and the ith column, it follows from Theorem 9.1.4 that there exists a vector
in Fr such that Ax = 0 but x ̸= 0. However, then
 
xi
r r s
.. 
 
u1 · · · ur   .  = ∑ xi ui = ∑ xi ∑ a ji v j

i=1 i=1 j=1
xr
s r s
= ∑ ∑ a ji xi v j = ∑ (Ax) j v j
j=1 i=1 j=1
s
= ∑ 0v j = 0
j=1
9.1. SUBSPACES 163

Which contradicts the assertion that the set of vectors

 {u1 , · · · , u
r } is linearly independent.
Indeed, there is a nonzero vector x for which u1 · · · ur x = 0. Thus we cannot
have s < r and so the only other possibility is that s ≥ r. ■
Definition 9.1.7 Let V be a subspace of Fn . Then {u1 , · · · , ur } is called a basis for V if
each ui ∈ V and span (u1 , · · · , ur ) = V and {u1 , · · · , ur } is linearly independent. In words,
{u1 , · · · , ur } spans and is independent.
Theorem 9.1.8 Let {u1 , · · · , ur } and {v 1 , · · · , v s } be bases for V . Then s = r.
Proof: From Theorem 9.1.6, r ≤ s because each ui is in the span of {v 1 , · · · , v s } and
{u1 , · · · , ur } is independent. Then also r ≥ s by the same reasoning. ■
Definition 9.1.9 Let V be a subspace of Fn . Then the dimension of V is the number of
vectors in a basis. This is well defined by Theorem 9.1.8.
Observation 9.1.10 The dimension of Fn is n. This is obvious because if x ∈ Fn , where
 T
x = x1 · · · xn , then x = ∑ni=1 xi ei which shows that {e1 , · · · , en } is a spanning
set. However, these vectors are clearly independent because if ∑i xi ei = 0, then
 T
0 = x1 · · · xn

and so each xi = 0. Thus {e1 , · · · , en } is also linearly independent.

The next lemma says that if you have a vector not in the span of a linearly independent
set, then you can add it in and the resulting longer list of vectors will still be linearly
independent.
Lemma 9.1.11 Suppose v ∈ / span (u1 , · · · , uk ) and {u1 , · · · , uk } is linearly independent.
Then {u1 , · · · , uk , v} is also linearly independent.
Proof: Suppose ∑ki=1 ci ui + dv = 0. It is required to verify that each ci = 0 and that d =
0. But if d ̸= 0, then you can solve for v as a linear combination of the vectors, {u1 , · · · , uk },
k c 
i
v =−∑ ui
i=1 d

contrary to assumption. Therefore, d = 0. But then ∑ki=1 ci ui = 0 and the linear indepen-
dence of {u1 , · · · , uk } implies each ci = 0 also. ■
It turns out that every subspace equals the span of some vectors. This is the content of
the next theorem.
Theorem 9.1.12 V is a nonzero subspace of Fn if and only if it has a basis.
Proof: Pick a nonzero vector of V, u1 . If V = span {u1 } , then stop. You have found
your basis. If V ̸= span (u1 ) , then there exists u2 a vector of V which is not a vector in
span (u1 ) . Consider span (u1 , u2 ) . By Lemma 9.1.11, {u1 , u2 } is linearly independent.
If V = span (u1 , u2 ) , stop. You have found a basis. Otherwise, pick u3 ∈ / span (u1 , u2 ) .
Continue this way until you obtain a basis. The process must stop after fewer than n + 1
iterations because if it didn’t, then there would be a linearly independent set of more than
n vectors which is impossible because there is a spanning set of n vectors from the above
observation. ■
The following is a fundamental result.
164 CHAPTER 9. SUBSPACES SPANS AND BASES

Theorem 9.1.13 If V is a subspace of Fn and the dimension of V is m, then m ≤ n and

also if {u1 , · · · , um } is an independent set of vectors of V , then this set of vectors is a
basis for V . Also, if you have a linearly independent set of vectors of V, {u1 , · · · , uk } for
k ≤ m = dim (V ) , there is a linearly independent set of vectors {u1 , · · · , uk , v k+1 , · · · v m }
which is a basis for V .

Proof: If the dimension of V is m, then it has a basis of m vectors. It follows m ≤

n because if not, you would have an independent set of vectors which is longer than a
spanning set of vectors {e1 , · · · , en } contrary to Theorem 9.1.6.
Next, if {u1 , · · · , um } is an independent set of vectors of V , then if it fails to span V, it
must be there is a vector w which is not in this span. But then by Lemma 9.1.11, you could
add w to the list of vectors and get an independent set of m + 1 vectors. However, the fact
that V is of dimension m means there is a spanning set having only m vectors and so this
contradicts Lemma 9.1.11. Thus {u1 , · · · , um } must be a spanning set.
Finally, if k = m, the vectors {u1 , · · · , uk } must span V since if not, you could add
another vector which is not in this list to the list and get an independent set which is longer
than a spanning set contrary to Theorem 9.1.6. Thus assume k < m. The set of vectors
{u1 , · · · , uk } cannot span V because if it did, the dimension of V would be k not m. Thus
there is a vector v k+1 not in this span. Then by Lemma 9.1.11, {u1 , · · · , uk , v k+1 } is
independent. If it spans V , stop. You have your basis. Otherwise, there is a v k+2 not
in the span and so you can add it in and get an independent set {u1 , · · · , uk , v k+1 , v k+2 }.
Continue this process till it stops. It must stop since otherwise, you would be able to get an
independent set of vectors larger than m which is the dimension of V, contrary to Theorem
9.1.6. ■

Definition 9.1.14 The rank of a matrix A is the dimension of Im (A) which is the same as
the column space of A.

Observation 9.1.15 When you have a matrix A and you do row operations to it. The
solutions to the system of equations having augmented matrix (A|0) are unchanged. This
was the entire basis for using row operations to solve systems of equations which was
presented earlier. Thus, if you can row reduce a matrix and obtain one for which it is clear
that all columns are in the span of certain columns and that these certain columns form an
independent set, then you have found the rank. You just need to count the number of these
special columns. Actually, the row reduced echelon form is designed to do this very thing.

Example 9.1.16 Let A =  

1 0 −5 −7 −3
 1 1 1 −1 1  (9.1)
 

0 1 6 6 4
Determine its rank.

The row reduced echelon form for the above matrix is

 
1 0 −5 −7 −3
 0 1 6 6 4  (9.2)
 

0 0 0 0 0
9.2. EXERCISES 165

and so its rank is 2 because every column is in the span of the first two columns. You can
think of the above as the row reduced version of several systems of equations, those which
have the following augmented matrices.
     
1 0 −5 1 0 −7 1 0 −3
 1 1 1  ,  1 1 −1  ,  1 1 1  ,
     

0 1 6 0 1 6 0 1 4

In each case, you can obtain the third column as a linear combination of the first two.
Thus the last three columns in 9.1 are linear combinations of the first two columns in 9.1.
Therefore, any linear combination of the columns of 9.1 can also be written as a linear
combination of the first two columns of 9.1. In other words, the span of the columns of 9.1
equals the span of the first two columns of 9.1. Also, from 9.2, we can see that the first two
columns of 9.1 are independent. Therefore, these columns are a basis for Im (A).
Similar considerations apply to determining whether some vectors are independent.
Remember the definition. To determine whether some vectors are independent, make them
the columns of a matrix A and determine the solution set to Ax = 0. If there is only the zero
solution, then the vectors are independent. If there are more solutions then these vectors
are not independent.

Theorem 9.1.17 Let A be an n × n matrix. Then A−1 exists if and only if the rank of A
equals n.

Proof: If the rank of A is n, then no column is a linear combination of the others

because, by definition, the columns are independent. In particular, one can row reduce
A to obtain I. Hence row reduction will do (A|I) → (I|B) and B is the inverse. Perhaps
a better way to see this is to note that the columns are independent because the span of
the columns has dimension n. Hence no column can be deleted and have the shorter list
still span the column space. Thus every vector in Fn is in the column space. It follows
for each b ∈ Fn , there exists x such that Ax = b. Thus A maps onto Fn . Considered as
a linear transformation, A is onto and it is also one to one because if Ax = b, Ax̂ = b,
then A (x − x̂) = 0 and so x − x̂ = 0 since otherwise, there would be a non trivial linear
combination of the columns of A which is 0 contrary to the observation that the columns are
independent.
Thus you can define a linear transformation, denoted as A−1 by A−1 (Ax) ≡ x
and A A−1 x = x. Then the matrix of A−1 , still denoted as A−1 is the desired inverse.
If A−1 exists, then it is obvious the columns of A are independent because a typical
linear combination of these columns is of the form Ax. If it equals 0 then A−1 (Ax) =
−1


A A x = x = 0. Thus the rank is n. ■

9.2 Exercises
1. Let {u1 , · · · , un } be vectors in Rn . The parallelepiped determined by these vectors

P (u1 , · · · , un )

is defined as
( )
n
P (u1 , · · · , un ) ≡ ∑ tk uk : tk ∈ [0, 1] for all k .
k=1
166 CHAPTER 9. SUBSPACES SPANS AND BASES

Now let A be an n × n matrix. Show that

{Ax : x ∈ P (u1 , · · · , un )}

is also a parallelepiped.

2. In the context of Problem 1, draw P (e1 , e2 ) where e1 , e2 are the standard basis vec-
tors for R2 . Thus e1 = (1, 0) , e2 = (0, 1) . Now suppose
!
1 1
E=
0 1

where E is the elementary matrix which takes the third row and adds to the first.
Draw
{Ex : x ∈ P (e1 , e2 )} .
In other words, draw the result of doing E to the vectors in P (e1 , e2 ). Next draw the
results of doing the other elementary matrices to P (e1 , e2 ). An elementary matrix is
one which is obtained from doing one of the row operations to the identity matrix.
3. Determine which matrices are in row reduced echelon form.
!  
1 2 0 1 1 0 0 0 5
(a)
(c)  0 0 1 2 0 4 
 
0 1 7

1 0 0 0
 0 0 0 0 1 3
(b)  0 0 1 2 
 

0 0 0 0

4. Row reduce the following matrices to obtain the row reduced echelon form. List the
pivot columns in the original matrix.
   
1 2 0 3 1 2 1 3
(a)  2 1 2 2  (c)  −3 2 1 0 
   

1 1 0 3 3 2 1 1
 
1 2 3
 2 1 −2 
(b) 
 

 3 0 0 
3 2 1

5. Find the rank of the following matrices. If the rank is r, identify r columns in the
original matrix which have the property that every other column may be written as
a linear combination of these. Also find a basis for column space of the matrices.
 
1 2 0
 3 2 1 
(a) 
 

 2 1 0 
0 2 1
9.2. EXERCISES 167

   
1 0 0 0 1 0 2 0 1 0
 4 1 1   0 3 2 6 0 5 4 
(b)  (d) 
   
 
 2 1 0   0 1 1 2 0 2 2 
0 2 0 0 2 1 4 0 3 2
   
0 1 0 2 1 2 2 0 1 0 2 1 1 2
 0 3 2 12 1 6 8   0 3 2 6 1 5 1 
(c)  (e) 
   
 
 0 1 1 5 0 2 3   0 1 1 2 0 2 1 
0 2 1 7 0 3 4 0 2 1 4 0 3 1

6. Suppose A is an m × n matrix. Explain why the rank of A is always no larger than

min (m, n) .
7. A matrix A is called a projection if A2 = A. Here is a matrix.
 
2 0 2
 1 1 2 
 

−1 0 −1
Show that this is a projection. Show that a vector in the column space of a projection
matrix is left unchanged by multiplication by A.
! ! !!
1 2 1
8. Let H denote span , , . Find the dimension of H and de-
2 4 3
termine a basis.
       
1 2 1 0
9. Let H denote span  2  ,  4  ,  3  ,  1  . Find the dimension of
       

0 0 1 1
H and determine a basis.
       
1 1 1 0
10. Let H denote span  2  ,  4  ,  3  ,  1  . Find the dimension of
       

0 0 1 1
H and determine a basis.
Let M = u = (u1 , u2 , u3 , u4 ) ∈ R4 : u3 = u1 = 0 . Is M a subspace? Explain.

11.
Let M = u = (u1 , u2 , u3 , u4 ) ∈ R4 : u3 ≥ u1 . Is M a subspace? Explain.

12.
Let w ∈ R4 and let M = u = (u1 , u2 , u3 , u4 ) ∈ R4 : w · u = 0 . Is M a subspace?

13.
Explain.
Let M = u = (u1 , u2 , u3 , u4 ) ∈ R4 : ui ≥ 0 for each i = 1, 2, 3, 4 . Is M a subspace?

14.
Explain.
15. Let w, w1 be given vectors in R4 and define
M = u = (u1 , u2 , u3 , u4 ) ∈ R4 : w · u = 0 and w1 · u = 0 .


Is M a subspace? Explain.
168 CHAPTER 9. SUBSPACES SPANS AND BASES

16. Let M = u = (u1 , u2 , u3 , u4 ) ∈ R4 : |u1 | ≤ 4 . Is M a subspace? Explain.



17. Let M = u = (u1 , u2 , u3 , u4 ) ∈ R4 : sin (u1 ) = 1 . Is M a subspace? Explain.



18. Suppose {x1 , · · · , xk } is a set of vectors from Fn . Show that span (x1 , · · · , xk ) con-
tains 0.
19. Prove the following theorem: If A, B are n × n matrices and if AB = I, then BA = I
and B = A−1 . Hint: First note that if AB = I, then it must be the case that A is onto.
Explain why this requires span (columns of A) = Fn . Now explain why, this requires
A to be one to one. Next explain why A (BA − I) = 0 and why the fact that A is one
to one implies BA = I.
20. Here are three vectors. Determine whether they are linearly independent or linearly
dependent.
 T  T  T
1 2 0 , 2 0 1 , 3 0 0

21. Here are three vectors. Determine whether they are linearly independent or linearly
dependent.
 T  T  T
4 2 0 , 2 2 1 , 0 2 2

22. Here are three vectors. Determine whether they are linearly independent or linearly
dependent.
 T  T  T
1 2 3 , 4 5 1 , 3 1 0

23. Here are four vectors. Determine whether they span R3 . Are these vectors linearly
independent?
 T  T  T  T
1 2 3 , 4 3 3 , 3 1 0 , 2 4 6

24. Here are four vectors. Determine whether they span R3 . Are these vectors linearly
independent?
 T  T  T  T
1 2 3 , 4 3 3 , 3 2 0 , 2 4 6

25. Determine whether the following vectors are a basis for R3 . If they are, explain why
they are and if they are not, give a reason and tell whether they span R3 .
 T  T  T  T
1 0 3 , 4 3 3 , 1 2 0 , 2 4 0

26. Determine whether the following vectors are a basis for R3 . If they are, explain why
they are and if they are not, give a reason and tell whether they span R3 .
 T  T  T
1 0 3 , 0 1 0 , 1 2 0
9.2. EXERCISES 169

27. Determine whether the following vectors are a basis for R3 . If they are, explain why
they are and if they are not, give a reason and tell whether they span R3 .
 T  T  T  T
1 0 3 , 0 1 0 , 1 2 0 , 0 0 0

28. Determine whether the following vectors are a basis for R3 . If they are, explain why
they are and if they are not, give a reason and tell whether they span R3 .
 T  T  T  T
1 0 3 , 0 1 0 , 1 1 3 , 0 0 0

29. Consider the vectors of the form

  
 2t + 3s
 

 s − t  : s,t ∈ R .
 
 
t +s
 

Is this set of vectors a subspace of R3 ? If so, explain why, give a basis for the sub-
space and find its dimension.

30. Consider the vectors of the form

  

 2t + 3s + u 

 
 s−t  
 : s,t, u ∈ R .
 




 t +s  


 
u

Is this set of vectors a subspace of R4 ? If so, explain why, give a basis for the sub-
space and find its dimension.
31. Consider the vectors of the form
  

 2t + u 

 
 + 3u
t
  
 : s,t, u, v ∈ R .



  t +s+v  


 

u

Is this set of vectors a subspace of R4 ? If so, explain why, give a basis for the sub-
space and find its dimension.
32. If you have 5 vectors in F5 and the vectors are linearly independent, can it always be
concluded they span F5 ? Explain.
33. If you have 6 vectors in F5 , is it possible they are linearly independent? Explain.

34. Suppose A is an m × n matrix and {w1 , · · · , wk } is a linearly independent set of

vectors in A (Fn ) ⊆ Fm . Now suppose A (z i ) = wi . Show {z 1 , · · · , z k } is also inde-
pendent.
170 CHAPTER 9. SUBSPACES SPANS AND BASES

35. Suppose V,W are subspaces of Fn . Show V ∩ W defined to be all vectors which are
in both V and W is a subspace also.
36. Suppose V and W both have dimension equal to 7 and they are subspaces of F10 .
What are the possibilities for the dimension of V ∩W ? Hint: Remember that a linear
independent set can be extended to form a basis.
37. Suppose V has dimension p and W has dimension q and they are each contained in
a subspace, U which has dimension equal to n where n > max (p, q) . What are the
possibilities for the dimension of V ∩W ? Hint: Remember that a linear independent
set can be extended to form a basis.
38. If b ̸= 0, can the solution set of Ax = b be a plane through the origin? Explain.
39. Suppose a system of equations has fewer equations than variables and you have found
a solution to this system of equations. Is it possible that your solution is the only one?
Explain.
40. Suppose a system of linear equations has a 2 × 4 augmented matrix and the last col-
umn is a pivot column. Could the system of linear equations be consistent? Explain.
41. Suppose the coefficient matrix of a system of n equations with n variables has the
property that every column is a pivot column. Does it follow that the system of
equations must have a solution? If so, must the solution be unique? Explain.
42. Suppose there is a unique solution to a system of linear equations. What must be true
of the pivot columns in the augmented matrix.
43. State whether each of the following sets of data are possible for the matrix equation
Ax = b. If possible, describe the solution set. That is, tell whether there exists a
unique solution no solution or infinitely many solutions.

(a) A is a 5 × 6 matrix, rank (A) = 4 and rank (A|b) = 4. Hint: This says b is in the
span of four of the columns. Thus the columns are not independent.
(b) A is a 3 × 4 matrix, rank (A) = 3 and rank (A|b) = 2.
(c) A is a 4 × 2 matrix, rank (A) = 4 and rank (A|b) = 4. Hint: This says b is in the
span of the columns and the columns must be independent.
(d) A is a 5 × 5 matrix, rank (A) = 4 and rank (A|b) = 5. Hint: This says b is not in
the span of the columns.
(e) A is a 4 × 2 matrix, rank (A) = 2 and rank (A|b) = 2.

44. Suppose A is an m × n matrix in which m ≤ n. Suppose also that the rank of A equals
m. Show that A maps Fn onto Fm . Hint: The vectors e1 , · · · , em occur as columns in
the row reduced echelon form for A.
45. Suppose A is an m × n matrix in which m ≥ n. Suppose also that the rank of A equals
n. Show that A is one to one. Hint: If not, there exists a vector x such that Ax = 0,
and this implies at least one column of A is a linear combination of the others. Show
this would require the column rank to be less than n.
46. Explain why an n × n matrix A is both one to one and onto if and only if its rank is n.
9.2. EXERCISES 171

47. Suppose A is an m × n matrix and B is an n × p matrix. Show that

dim (ker (AB)) ≤ dim (ker (A)) + dim (ker (B)) .

Hint: Consider the subspace, B (F p ) ∩ ker (A) and suppose a basis for this subspace
is
{w1 , · · · , wk } .
Now suppose {u1 , · · · , ur } is a basis for ker (B) . Let {z 1 , · · · , z k } be such that Bz i =
wi and argue that

ker (AB) ⊆ span (u1 , · · · , ur , z 1 , · · · , z k ) .

Here is how you do this. Suppose ABx = 0. Then Bx ∈ ker (A) ∩ B (F p ) and so
Bx = ∑ki=1 Bz i showing that
k
x− ∑ z i ∈ ker (B) .
i=1

48. Explain why Ax = 0 always has a solution even when A−1 does not exist.

(a) What can you conclude about A if the solution is unique?

(b) What can you conclude about A if the solution is not unique?

49. Let A be an n × n matrix and let x be a nonzero vector such that Ax = λ x for some
scalar λ . When this occurs, the vector x is called an eigenvector and the scalar λ
is called an eigenvalue. It turns out that not every number is an eigenvalue. Only
certain ones are. Why? Hint: Show that if Ax = λ x, then (A − λ I) x = 0. Explain
why this shows that (A − λ I) is not one to one and not onto.
n 2
o
50. Let A be an n × n matrix and consider the matrices I, A, A2 , · · · , An . Explain why
there exist scalars, ci not all zero such that

n2
∑ ci Ai = 0.
i=1

Then argue there exists a polynomial, p (λ ) of the form

λ m + dm−1 λ m−1 + · · · + d1 λ + d0

such that p (A) = 0 and if q (λ ) is another polynomial such that q (A) = 0, then q (λ )
is of the form p (λ ) l (λ ) for some polynomial, l (λ ) . This extra special polynomial,
p (λ ) is called the minimal polynomial. Hint: You might consider an n × n matrix
2
as a vector in Fn . What would be a basis for this set of matrices?
172 CHAPTER 9. SUBSPACES SPANS AND BASES
Chapter 10

Eigenvalues and Eigenvectors

10.1 Definition of Eigenvalues

The thing to always keep in mind is the following definition of eigenvalues and eigenvec-
tors. There are many ways to find them and in this chapter, I will present the standard way
to do this. It is also the very worst way.

Definition 10.1.1 Let A be an n × n matrix and let x ∈ Cn , λ ∈ C. Then x is an eigenvector

for the eigenvalue λ if and only if the following two conditions hold.

1. Ax = λ x
2. x ̸= 0. This is very important. By definition 0 is NEVER an eigenvector although
it can be an eigenvalue.

Now here is an important observation which really is just a re statement of the above
definition.

Theorem 10.1.2 Let A be an n×n matrix. The vector x is an eigenvector for the eigenvalue
λ if and only if (A − λ I)−1 does not exist.

Proof: If (A − λ I)−1 does not exist, then by Theorem 9.1.17 the columns of A − λ I
are not independent because its rank is less than n. Thus there exists x ̸= 0 such that
(A − λ I) x = 0 and so λ is an eigenvalue and x is an eigenvector which goes with λ .
Conversely, if (A − λ I) x = 0, and x ̸= 0, then the rank of (A − λ I) has no inverse be-
cause its rank is less than n. Indeed, some column is a linear combination of the others.
■
Now with this fundamental definition, I will present the worst way of finding eigenval-
ues and eigenvectors. It is very important because everyone cherishes it. Also, it gives an
introduction to the important topic of determinants which will be presented in more detail
later.

173
174 CHAPTER 10. EIGENVALUES AND EIGENVECTORS

10.2 An Introduction to Determinants

Here in this section, I will summarize the main properties of determinants without detailed
proofs. Proofs are presented later in the book. The idea is that you get used to using them
first.

10.2.1 Cofactors and 2 × 2 Determinants

Let A be an n × n matrix. The determinant of A, denoted as det (A) is a number. If the
matrix is a 2×2 matrix, this number is very easy to find.
!
a b
Definition 10.2.1 Let A = . Then det (A) ≡ ad − cb. The determinant is also
c d
often denoted by enclosing the matrix with two vertical lines. Thus
!
a b a b
det = .
c d c d
!
2 4
Example 10.2.2 Find det .
−1 6

From the definition this is just (2) (6) − (−1) (4) = 16.
Having defined what is meant by the determinant of a 2 × 2 matrix, what about a 3 × 3
matrix?

Definition 10.2.3 Suppose A is a 3 × 3 matrix. The i jth minor, denoted as minor(A)i j ,

is the determinant of the 2 × 2 matrix which results from deleting the ith row and the jth
column.

Example 10.2.4 Consider the matrix

 
1 2 3
 4 3 2 .
 

3 2 1

The (1, 2) minor is the determinant of the 2 × 2 matrix which results when you delete the
first row and the second column. This minor is therefore
!
4 2
det = −2.
3 1

The (2, 3) minor is the determinant of the 2 × 2 matrix which results when you delete the
second row and the third column. This minor is therefore
!
1 2
det = −4.
3 2
10.2. AN INTRODUCTION TO DETERMINANTS 175

Definition 10.2.5 Suppose A is a 3 × 3 matrix. The i jth cofactor is defined to be (−1)i+ j ×

i jth minor . In words, you multiply (−1)i+ j times the i jth minor to get the i jth cofactor.

The cofactors of a matrix are so important that special notation is appropriate when re-
ferring to them. The i jth cofactor of a matrix A will be denoted by cof (A)i j . It is also
convenient to refer to the cofactor of an entry of a matrix as follows. For ai j an entry of
the matrix, its cofactor is just cof (A)i j . Thus the cofactor of the i jth entry is just the i jth
cofactor.

Example 10.2.6 Consider the matrix

 
1 2 3
A= 4 3 2 .
 

3 2 1

The (1, 2) minor is the determinant of the 2 × 2 matrix which results when you delete the
first row and the second column. This minor is therefore
!
4 2
det = −2.
3 1

It follows !
4 2
cof (A)12 = (−1)1+2 det = (−1)1+2 (−2) = 2
3 1
The (2, 3) minor is the determinant of the 2 × 2 matrix which results when you delete the
second row and the third column. This minor is therefore
!
1 2
det = −4.
3 2

Therefore, !
2+3 1 2
cof (A)23 = (−1) det = (−1)2+3 (−4) = 4.
3 2
Similarly, !
2+2 1 3
cof (A)22 = (−1) det = −8.
3 1

Definition 10.2.7 The determinant of a 3 × 3 matrix A, is obtained by picking a row (col-

umn) and taking the product of each entry in that row (column) with its cofactor and adding
these. This process when applied to the ith row (column) is known as expanding the deter-
minant along the ith row (column).

Example 10.2.8 Find the determinant of

 
1 2 3
A= 4 3 2 .
 

3 2 1
176 CHAPTER 10. EIGENVALUES AND EIGENVECTORS

Here is how it is done by “expanding along the first column”.

cof(A)11 cof(A)21 cof(A)31
z }| !{ z }| !{ z }| !{
3 2 2 3 2 3
1(−1)1+1 det + 4(−1)2+1 det + 3(−1)3+1 det = 0.
2 1 2 1 3 2

This simply follows the rule in the above definition. We took the 1 in the first column and
multiplied it by its cofactor, the 4 in the first column and multiplied it by its cofactor, and
the 3 in the first column and multiplied it by its cofactor. Then we added these numbers
together.
You could also expand the determinant along the second row as follows.
cof(A)21 cof(A)22 cof(A)23
z }| !{ z }| !{ z }| !{
2 3 1 3 1 2
4(−1)2+1 det + 3(−1)2+2 det + 2(−1)2+3 det = 0.
2 1 3 1 3 2

Observe this gives the same number. You should try expanding along other rows and
columns. If you don’t make any mistakes, you will always get the same answer.
What about a 4 × 4 matrix? You know now how to find the determinant of a 3 × 3
matrix. The pattern is the same. In general, it is as described in the following definition.

Definition 10.2.9 Let A = (ai j ) be an n × n matrix and suppose the determinant of a

(n − 1) × (n − 1) matrix has been defined. Then a new matrix called the cofactor ma-
trix, cof (A) is defined by cof (A)i j = (ci j ) where to obtain ci j delete the ith row and the
jth column of A, take the determinant of the (n − 1) × (n − 1) matrix which results, (This is
called the i jth
minor of A. ) and then multiply this number by (−1)i+ j . Thus (−1)i+ j ×
the i jth minor equals the i jth cofactor. Then det (A) is given by ∑i Ai j ci j = ∑ j Ai j ci j . Any
of these expansions along a row or a column gives the same number.

You should regard the above claim that you always get the same answer by picking
any row or column with considerable skepticism. It is incredible and not at all obvious.
However, it requires a little effort to establish it. This is done in the section on the theory
of the determinant, Section 28 which is presented much later. This is summarized in the
following theorem whose conclusion is incredible.

Theorem 10.2.10 Expanding the n × n matrix along any row or column always gives the
same answer so the above definition is a good definition.
 
1 2 −1 1
 2 3 1 1 
Example 10.2.11 Expand det   along first column.
 
 1 1 0 0 
1 2 3 1

It is    
3 1 1 2 −1 1
1 det  1 0 0  − 2 det  1 0 0 
   

2 3 1 2 3 1
10.2. AN INTRODUCTION TO DETERMINANTS 177
   
2 −1 1 2 −1 1
+1 det  3 1 1  − 1 det  3 1 1 =0
   

2 3 1 1 0 0

10.2.2 The Determinant of a Triangular Matrix

Notwithstanding the difficulties involved in using the method of Laplace expansion, certain
types of matrices are very easy to deal with.
Definition 10.2.12 A matrix M, is upper triangular if Mi j = 0 whenever i > j. Thus such
a matrix equals zero below the main diagonal, the entries of the form Mii , as shown.
 
∗ ∗ ··· ∗
 0 ∗ . . . ... 
 
 
 .
 . .. ..

. . . ∗

 
0 ··· 0 ∗
A lower triangular matrix is defined similarly as a matrix for which all entries above the
main diagonal are equal to zero.
You should verify the following using the above theorem on Laplace expansion.
Corollary 10.2.13 Let M be an upper (lower) triangular matrix. Then det (M) is obtained
by taking the product of the entries on the main diagonal.
Example 10.2.14 Let
 
1 2 3 77
 0 2 6 7 
A=
 

 0 0 3 33.7 
0 0 0 −1
Find det (A) .
From the above corollary, it suffices to take the product of the diagonal elements. Thus
det (A) = 1 × 2 × 3 × (−1) = −6. Without using the corollary, you could expand along the
first column. This gives
   
2 6 7 2 3 77
1 det  0 3 33.7  + 0 (−1)2+1 det  0 3 33.7  +
   

0 0 −1 0 0 −1
   
2 3 77 2 3 77
0 (−1)3+1 det  2 6 7  + 0 (−1)4+1 det  2 6 7 
   

0 0 −1 0 3 33.7
and the only nonzero term in the expansion is
 
2 6 7
1 det  0 3 33.7  .
 

0 0 −1
178 CHAPTER 10. EIGENVALUES AND EIGENVECTORS

Now expand this along the first column to obtain

 ! ! 
3 33.7 6 7
 2 × det + 0 (−1)2+1 det
0 −1 −1

 0 
1× ! 
 6 7 
+0 (−1)3+1 det
 
3 33.7
!
3 33.7
= 1 × 2 × det
0 −1

Next expand this last determinant along the first column to obtain the above equals 1 ×
2 × 3 × (−1) = −6 which is just the product of the entries down the main diagonal of the
original matrix. It works this way in general.

10.2.3 Properties of Determinants

There are many properties satisfied by determinants. Some of these properties have to do
with row operations. Recall the row operations.

Definition 10.2.15 The row operations consist of the following

1. Switch two rows.

2. Multiply a row by a nonzero number.
3. Replace a row by a multiple of another row added to itself.

Theorem 10.2.16 Let A be an n × n matrix and let A1 be a matrix which results from
multiplying some row of A by a scalar c. Then c det (A) = det (A1 ).
! !
1 2 2 4
Example 10.2.17 Let A = , A1 = . det (A) = −2, det (A1 ) = −4.
3 4 3 4

Theorem 10.2.18 Let A be an n × n matrix and let A1 be a matrix which results from
switching two rows of A. Then det (A) = − det (A1 ) . Also, if one row of A is a multiple of
another row of A, then det (A) = 0.
! !
1 2 3 4
Example 10.2.19 Let A = and let A1 = . det A = −2, det (A1 ) = 2.
3 4 1 2

Theorem 10.2.20 Let A be an n × n matrix and let A1 be a matrix which results from
applying row operation 3. That is you replace some row by a multiple of another row
added to itself. Then det (A) = det (A1 ).
! !
1 2 1 2
Example 10.2.21 Let A = and let A1 = . Thus the second row of
3 4 4 6
A1 is one times the first row added to the second row. det (A) = −2 and det (A1 ) = −2.

Theorem 10.2.22 In Theorems 10.2.16 - 10.2.20 you can replace the word, “row” with
the word “column”.
10.2. AN INTRODUCTION TO DETERMINANTS 179

There are two other major properties of determinants which do not involve row opera-
tions.

Theorem 10.2.23 Let A and B be two n × n matrices. Then

det (AB) = det (A) det (B).

Also,
det (A) = det AT .

Example 10.2.24 Compare det (AB) and det (A) det (B) for
! !
1 2 3 2
A= ,B = .
−3 2 4 1

First ! ! !
1 2 3 2 11 4
AB = =
−3 2 4 1 −1 −4
!
11 4
and so det (AB) = det = −[Link]
−1 −4
! !
1 2 3 2
det (A) = det = 8, det (B) = det = −5.
−3 2 4 1

Thus det (A) det (B) = 8 × (−5) = −40.

10.2.4 Finding Determinants Using Row Operations

Theorems 10.2.20 - 10.2.22 can be used to find determinants using row operations. As
pointed out above, the method of Laplace expansion will not be practical for any matrix of
large size. Here is an example in which all the row operations are used.

Example 10.2.25 Find the determinant of the matrix

 
1 2 3 4
 5 1 2 3 
A=
 

 4 5 4 3 
2 2 −4 5

Replace the second row by (−5) times the first row added to it. Then replace the third
row by (−4) times the first row added to it. Finally, replace the fourth row by (−2) times
the first row added to it. This yields the matrix
 
1 2 3 4
 0 −9 −13 −17 
B=
 
 0 −3 −8 −13 


0 −2 −10 −3
180 CHAPTER 10. EIGENVALUES AND EIGENVECTORS

and from Theorem 10.2.20, it has the same determinant as A. Now using other row opera-
tions, det (B) = −1


3 det (C) where
 
1 2 3 4
 0 0 11 22 
C= .
 
 0 −3 −8 −13 
0 6 30 9

The second row was replaced by (−3) times the third row added to the second row. By
Theorem 10.2.20 this didn’t change the value of the determinant. Then the last row was
multiplied by (−3) . By Theorem 10.2.16 the resulting matrix has a determinant which is
(−3) times the determinant of the un-multiplied matrix. Therefore, we multiplied by −1/3
to retain the correct value. Now replace the last row with 2 times the third added to it.
This does not change the value of the determinant by Theorem 10.2.20. Finally switch
the third and second rows. This causes the determinant to be multiplied by (−1) . Thus
det (C) = − det (D) where
 
1 2 3 4
 0 −3 −8 −13 
D=
 

 0 0 11 22 
0 0 14 −17

You could do more row operations or you could note that this can be easily expanded along
the first column followed by expanding the 3×3 matrix which results along its first column.
Thus !
11 22
det (D) = 1 (−3) det = 1485
14 −17

and so det (C) = −1485 and det (A) = det (B) = −1

3 (−1485) = 495.

Example 10.2.26 Find the determinant of the matrix

 
1 2 3 2

 1 −3 2 1 

 
 2 1 2 5 
3 −4 1 2

Replace the second row by (−1) times the first row added to it. Next take −2 times the
first row and add to the third and finally take −3 times the first row and add to the last row.
This yields
 
1 2 3 2
 0 −5 −1 −1 
.
 
 0 −3 −4 1 


0 −10 −8 −4
By Theorem 10.2.20 this matrix has the same determinant as the original matrix. Remem-
ber you can work with the columns also. Take −5 times the last column and add to the
10.3. APPLICATIONS 181

second column. This yields

 
1 −8 3 2

 0 0 −1 −1 

−8 −4
 
 0 1 
0 10 −8 −4
By Theorem 10.2.22 this matrix has the same determinant as the original matrix. Now take
(−1) times the third row and add to the top row. This gives.
 
1 0 7 1

 0 0 −1 −1 

−8 −4
 
 0 1 
0 10 −8 −4

which by Theorem 10.2.20 has the same determinant as the original matrix. Lets expand
it now along the first column. This yields the following for the determinant of the original
matrix.  
0 −1 −1
det  −8 −4 1 
 

10 −8 −4
! !
−1 −1 −1 −1
which equals 8 det + 10 det = −82
−8 −4 −4 1
I suggest you do not try to be fancy in using row operations. That is, stick mostly to
the one which replaces a row or column with a multiple of another row or column added to
it. Also note there is no way to check your answer other than working the problem more
than one way. To be sure you have gotten it right you must do this. Unfortunately, this
process can go on and on when you keep getting different answers. This is a good example
of something for which you should use a computer algebra system.

10.3 Applications
10.3.1 A Formula For The Inverse
The definition of the determinant in terms of Laplace expansion along a row or column
also provides a way to give a formula for the inverse of a matrix. Recall the definition of
the inverse of a matrix in Definition 8.6.2 on Page 144. Also recall the definition of the
cofactor matrix given in Definition 10.2.9 on Page 176. This cofactor matrix was just the
matrix which results from replacing the i jth entry of the matrix with the i jth cofactor.
The following theorem says that to find the inverse, take the transpose of the cofactor
matrix and divide by the determinant. The transpose of the cofactor matrix is called the
adjugate or sometimes the classical adjoint of the matrix A. In other words, A−1 is equal
to one divided by the determinant of A times the adjugate matrix of A. This is what the
following theorem says with more precision. The proof is presented later in Section 27.2.1.
182 CHAPTER 10. EIGENVALUES AND EIGENVECTORS
 
Theorem 10.3.1 A−1 exists if and only if det(A) ̸= 0. If det(A) ̸= 0, then A−1 = a−1
ij
where
a−1 −1
i j = det(A) cof (A) ji

for cof (A)i j the i jth cofactor of A.

Example 10.3.2 Find the inverse of the matrix

 
1 2 3
A= 3 0 1 
 

1 2 1

First find the determinant of this matrix. Using Theorems 10.2.20 - 10.2.22 on Page
178, the determinant of this matrix equals the determinant of the matrix
 
1 2 3
 0 −6 −8 
 

0 0 −2

which equals 12. The cofactor matrix of A is

 
−2 −2 6
 4 −2 0 .
 

2 8 −6

Each entry of A was replaced by its cofactor. Therefore, from the above theorem, the
inverse of A should equal
 T  
−2 −2 6 −1/6 1/3 1/6
1 
 4 −2 0  =  −1/6 −1/6 2/3  .
  
12
2 8 −6 1/2 0 −1/2

Does it work? You should check to see if it does. When the matrices are multiplied
    
−1/6 1/3 1/6 1 2 3 1 0 0
 −1/6 −1/6 2/3   3 0 1 = 0 1 0 
    

1/2 0 −1/2 1 2 1 0 0 1

and so it is correct.

Example 10.3.3 Find the inverse of the matrix

 
1 1
0
 2 2

 1 1
− 12 

 −
A= 6 3 
 
 5 2 
 −
6 3 −1  2
10.3. APPLICATIONS 183

First find its determinant. This determinant is 16 . I will replace each entry in the above
matrix with the cofactor corresponding to the position of that entry. Then I will take the
transpose and multiply by 6. You should check that the result is as follows.
 T
1 1 1
6 3 6
1 1
6 − 13  .
 
3 6
− 16 1
6
1
6

This yields
 T  
1/6 1/3 1/6 1 2 −1
6  1/3 1/6 −1/3  =  2 1 1 
   

−1/6 1/6 1/6 1 −2 1

Always check your work.
    
1 2 −1 1/2 0 1/2 1 0 0
 2 1 1   −1/6 1/3 −1/2  =  0 1 0 
    

1 −2 1 −5/6 2/3 −1/2 0 0 1

and so we got it right. If the result of multiplying these matrices had been something other
than the identity matrix, you would know there was an error. When this happens, you
need to search for the mistake if you are interested in getting the right answer. A common
mistake is to forget to take the transpose of the cofactor matrix.

10.3.2 Finding Eigenvalues Using Determinants

Theorem 10.3.1 says that A−1 exists if and only if det (A) ̸= 0 when there is even a for-
mula for the inverse. Recall also that an eigenvector for λ is a nonzero vector x such that
Ax = λ x where λ is called an eigenvalue. Thus you have (A − λ I) x = 0 for x ̸= 0. If
(A − λ I)−1 were to exist, then you could multiply by it on the left and obtain x = 0 after
all. Therefore, it must be the case that det (A − λ I) = 0. This yields a polynomial of de-
gree n equal to 0. This polynomial is called the characteristic polynomial. For example,
consider  
1 −1 −1
 0 3 2 
 

0 −1 0
You need to have
   
1 −1 −1 1 0 0
det  0 3 2 −λ  0 1 0  = 0
   

0 −1 0 0 0 1

That on the left equals a polynomial of degree 3 which when factored yields

(1 − λ ) (λ − 1) (λ − 2)

Therefore, the possible eigenvalues are 1,1,2. Note how the 1 is listed twice. This is because
it occurs twice as a root of the characteristic polynomial. Also, if M −1 does not exist where
184 CHAPTER 10. EIGENVALUES AND EIGENVECTORS

M is an n × n matrix, then this means that the columns of M cannot be lineraly independent
since if they were, then by Theorem 11.5.2 M −1 would exist. Thus if A − λ I fails to have
an inverse as above, then the columns are not independent and so there exists a nonzero x
such that (A − λ I) x = 0. Thus we have the following proposition.

Proposition 10.3.4 The eigenvalues of an n × n matrix are the roots of det (A − λ I) = 0.

Corresponding to each of these λ is an eigenvector.

Note that if A = S−1 BS, then A, B have the same characteristic polynomial, hence the
same eigenvalues. (They might have different eigenvectors and usually will.) To see this,
note that from the properties of determinants

det (A − λ I) = det S−1 BS − λ S−1 IS = det S−1 (B − λ I) S

= det S−1 det (B − λ I) det (S) = det S−1 S det (B − λ I)

= det (I) det (B − λ I) = det (B − λ I) (10.1)

Chapter 11

Matrices and The Inner Product

Recall the inner product or dot product.

a · b ≡ ∑ ak bk
k

In more advanced contexts, this is usually written as ⟨a, b⟩ or often simply as (a, b) instead
of a · b. Also, the term “inner product” tends to be preferred over “dot product”. Thus, in
this chapter, we will adopt the notation (a, b) for the dot product. The first thing to consider
is the notion of the adjoint of a matrix.

Definition 11.0.1 Let A be an m×n matrix. Then its adjoint, denoted as A∗ is the transpose
of the conjugate of A. That is, you replace each entry of A with its complex conjugate and
take the transpose of what you got. Thus
 
!∗ −i 3
i 2 1+i
= 2 1+i 
 
3 1−i 1
1−i 1

In symbols, (A∗ )rs = Asr . Note that (A∗ )∗ = A.

The reason the adjoint is so important is the following proposition which in fact can
be used as a definition of the adjoint instead of the above explicit description in terms of
entries.

Proposition 11.0.2 Let A be an m × n matrix and let x ∈ Fm and y ∈ Fn . Then

(x,Ay) = (A∗ x, y)

Also, if B is an n × m matrix such that the above holds for all x ∈ Fm and y ∈ Fn , then
B = A∗ .

Proof: This follows directly from the definition of the inner product and the properties
of the complex conjugate which were reviewed in Section 2.3.

(x,Ay) = ∑ xk (Ay)k = ∑ xk ∑ Ak j y j = ∑ xk ∑ Ak j y j
k k j k j

185
186 CHAPTER 11. MATRICES AND THE INNER PRODUCT

= ∑ ∑ A∗jk xk y j = ∑ (A∗ x) j y j = (A∗ x, y)

j k j

Now suppose for all x, y (x,Ay) = (Bx, y) . Then you have (A∗ x, y) = (Bx, y) for all
x, y and so (A∗ x − Bx, y) = 0 for all x, y. In particular this holds for y = A∗ x − Bx.
Thus A∗ x − Bx = 0 for each x. Hence A∗ = B. To see this, note that (A∗ − B) e j says that
the jth column of A∗ − B is zero. ■
The last part of this argument deserves a little more emphasis. If you have an m × n
matrix M, then M is the zero matrix if and only if Mx = 0 for all x ∈ Fn . In other words,
to show something is zero, you show it sends every vector to 0. Equivalently, a matrix M
is not zero if and only if there is some vector x for which Mx ̸= 0.

11.1 Eigenvalues and Eigenvectors

Here I will consider eigenvectors and eigenvalues from a different point of view. Rather
than determinants, this approach depends on what is really fundamental about linear alge-
bra, linear independence. Consider the collection of n × n matrices consisting of complex
numbers Mn×n . Now consider the special matrices Ei j which has a 1 in the ith row and jth
column and zeros in all other positions. Then according to the way we add and multiply
2
matrices by scalars, Mn×n can be considered as Cn . For example, consider the case ! where
  1 2
n = 2. Instead of writing 1 2 3 4 , we bend it and write it as . Thus,
3 4
if ai j is the entry in i jth position of one of these matrices, then we can recover A by forming
the sum
∑ ∑ ai j Ei j (11.1)
i k
Thus these Ei j span the n × n matrices which, as just noted, can be considered as vectors
2
in Cn . Furthermore, these matrices Ei j are independent because if the above sum in 11.1
equals 0, then since ai j is the entry in the i jth position, it follows that ai j = 0. Thus these
matrices are a basis for Mn×n and the dimension of Mn×n = n2 which we already knew from
2
the above identification of Mn×n with Cn .
Define A0 ≡ I for any matrix A ∈ Mn×n . Consider the matrices
2
I, A, A2 , · · · , An
There are n2 + 1 of these matrices and so they can’t be independent. Hence there are scalars
c0 , · · · , cn2 not all zero such that
2
c0 I + c1 A + · · · + cn2 An = 0
In other words, there is a polynomial
q (λ ) = cm λ m + · · · + c1 λ + c0
such that
q (A) ≡ cm Am + · · · + c1 A + c0 I = 0 in Mn×n
Out of all such polynomials, let p̂ (λ ) be one which has the smallest degree. Denote by
p (λ ) the polynomial which results by dividing by the leading coefficient. Thus p (λ ) is the
monic polynomial of smallest degree which has p (A) = 0.
11.1. EIGENVALUES AND EIGENVECTORS 187

Definition 11.1.1 The minimum polynomial for an n × n matrix A is the polynomial which
has smallest degree and is monic such that p (A) = 0. In fact, it is unique and you might
think about why this is so using the division algorithm.

Now we can give the definition of eigenvalues and eigenvectors. Recall it is as follows.

Definition 11.1.2 Let A be an n × n matrix. A NONZERO VECTOR x is said to be an

eigenvector for A if there is some number λ such that

Ax = λ x

This number is called an eigenvalue.

It turns out that every n × n matrix has an eigenvalue and that in fact every root of the
minimum polynomial is an eigenvalue. Recall that by the fundamental theorem of algebra,
(See Section 2.9), the minimum polynomial has a root. In fact, we can completely factor
the minimum polynomial.

Proposition 11.1.3 Let p (λ ) be a monic polynomial of degree m ≥ 1 having complex co-

efficients. Then there are complex numbers µ 1 , · · · , µ m , possibly not all distince such that

p (λ ) = (λ − µ 1 ) (λ − µ 2 ) · · · (λ − µ m )

Proof: If m = 1, there is nothing to show. Suppose then that the Proposition is true for
some m ≥ 1 and suppose p (λ ) is a monic polynomial of degree m + 1.
From the fundamental theorem of algebra, there is a root to p (λ ). Denote this root as
µ 1 . From Lemma 2.8.2, the division algorithm,

p (λ ) = (λ − µ 1 ) k (λ ) + r (λ )

where the degree of r (λ ) is less than 1 or else r (λ ) = 0. Thus r (λ ) = r. However, if we

evaluate both sides at λ = µ 1 we get p (µ 1 ) = 0 = r and so

p (λ ) = (λ − µ 1 ) k (λ )

Now k (λ ) is also a monic polynomial which can be seen by comparing the leading co-
efficient of both sides and it has degree m. Therefore, by induction, there are m complex
numbers µ 2 , µ 3 , · · · , µ m+1 such that


p (λ ) = (λ − µ 1 ) k (λ ) = p (λ ) = (λ − µ 1 ) · · · λ − µ m+1

Thus the proposition holds for m = 1 and if it holds for m, then it also holds for m + 1.
Therefore, it is valid for any positive integer m. ■

Theorem 11.1.4 Let A be an n × n matrix. Let its minimum polynomial be p (λ ). Let

p (λ ) = (λ − µ 1 ) · · · (λ − µ m )

Then for each µ j , there is an eigenvector x j such that Ax j = µ j x j .

188 CHAPTER 11. MATRICES AND THE INNER PRODUCT

Proof: First note that IB = BI for any square matrix B. Next note that

0 = p (A) = (A − µ 1 I) · · · (A − µ m I) (11.2)

Also note that

(A − µI) (A − λ I) = A2 − (λ + µ) A + µλ I = (A − λ I) (A − µI)

Thus all the factors in the above product 11.2 can be interchanged and thereby placed in
any order in the product. We know that for any y,
 h    i
A − µ j I (A − µ 1 I) · · · A − µ j−1 I A − µ j+1 I · · · (A − µ m I) y = 0

However, there is some y j such that

  
(A − µ 1 I) · · · A − µ j−1 I A − µ j+1 I · · · (A − µ m I) y j ̸= 0

since otherwise, p (λ ) didn’t really have smallest degree. Then let

  
x j = (A − µ 1 I) · · · A − µ j−1 I A − µ j+1 I · · · (A − µ m I) y j ■

The minimum polynomial can be computed although it might seem a little tedious. In
the above discussion, the minimum polynomial is only known to have degree no more than
n2 . Actually it can be shown that the degree of the minimum polynomial is never more
than n although it might be less than n. We will show this later as part of the theory of the
determinant but in the meantime, one should go ahead and use it. Here is an example.

Example 11.1.5 Let !

2 1
A=
1 3
Find its minimum polynomial.
! !2
2 1 2 1
The matrices are I, , . These will end up being linearly depen-
1 3 1 3
! ! !
1 0 2 1 5 5
dent. They are , , . The polynomial is obtained by find-
0 1 1 3 5 10
ing a linear combination of these equal to 0. Lets make these into column vectors and use
row operations.
 
1 2 5
 0 1 5 
 
 
 0 1 5 
1 3 10
Now we row reduce this to get
 
1 0 −5

 0 1 5 

 
 0 0 0 
0 0 0
11.1. EIGENVALUES AND EIGENVECTORS 189

Thus, as explained earlier, the last column is −5 times the first added to 5 times the second.
Thus !2 ! !
2 1 1 0 2 1
= −5 +5
1 3 0 1 1 3
You can see from the row reduced echelon form that no smaller linear combination relating
the matrices I, A, A2 is possible. Hence the minimal polynomial is
λ 2 − 5λ + 5
The eigenvalues are therefore, the roots of this polynomial. They are
5 1√ 5 1√
+ 5, − 5
2 2 2 2
Now one can find eigenvectors associated
  with√ these.
  Consider the first of them. We
5 1
want a nonzero vector x such that A − 2 + 2 5 I x = 0. Thus we need consider the
augmented matrix
  √  
2 − 25 + 12 5 1 0
  √  
1 3 − 52 + 21 5 0

We row reduce this to obtain

1
√ !
1 2 − 12 5 0
0 0 0
Thus the eigenvectors are of the form
1
√ !
2 5 − 21
y ,y ∈ C
1
Example 11.1.6 Find the minimum polynomial for
 
0 −2 −2
A= 2 5 4 
 

−1 −2 −1

We look for linear combinations for A0 , A, A2 , A3 . These are the matrices, listed in
order of decreasing powers are
       
−6 −14 −14 −2 −6 −6 0 −2 −2 1 0 0
14 29 28 ,
  6 13 12  ,  2 5 4 , 0 1 0 
       

−7 −14 −13 −3 −6 −5 −1 −2 −1 0 0 1

We can arrange them as column vectors in C9 as done earlier, but it might be easier to
simply look at the entries in a single row or column. Lets pick the first column of each.
Thus the augmented matrix to solve would be
 
1 0 −2 −6 0
 0 2 6 14 0 
 

0 −1 −3 −7 0
190 CHAPTER 11. MATRICES AND THE INNER PRODUCT

Row reduce this.  

1 0 −2 −6 0
 0 1 3 7 0 
 

0 0 0 0 0
Thus column one of A2 equals −2 times column one of I added to three times column one
of A. Thus consider the polynomial λ 2 − 3λ + 2. This seems to work in so far as the first
column of A is concerned and there is no polynomial of smaller degree which will work.
Therefore, lets check to see if this sends A to 0. If it does, then it must be the minimal
polynomial. When you do the computations, you find that this indeed does send A to 0
and so it is the minimum polynomial. The eigenvalues are 1 and 2. You can now find the
eigenvectors for these using row operations.

11.2 Using Matlab

It is routine to find this polynomial and so it is not surprising that MATLAB is able to do
it for you. I recommend doing this rather than all the trouble just described. The syntax to
use is this:
>> A=[1,2,3;3,-3,1;2,7,1]; minpoly(A) Here you press enter. It gives:
1 1 -24 -69
These are the coefficients of the minimum polynomial which is
λ 3 + λ 2 − 24λ − 69
The matrix you entered was  
1 2 3
 3 −3 1 
 

2 7 1
You open MATLAB and you see >>. Then type in just what is above. The ; at the end after
entering the matrix says for MATLAB to know the matrix but not to rewrite it. You can of
course follow the same pattern to enter any square matrix you like. Then of course you are
faced with the problem of finding the roots of the polynomial. Sometimes you can’t do this
exactly. Neither can MATLAB. However, when the polynomial can be factored, MATLAB
can do it for you. Here is the syntax.
>> syms x

factor(xˆ2-3*x+2) (here you press enter and what results is:)

[x-1, x-2]
To get to a new line in MATLAB you press shift enter. You factored x2 − 3x + 2. You
can enter any polynomial you like, but sometimes they can’t be factored exactly. When this
happens, MATLAB will just return the original polynomial. This is its way of saying that
it has no idea how to do it.

11.3 Distance and Unitary Matrices

Some matrices preserve lengths of vectors. That is |Ux| = |x| for any x in Cn . Such a
matrix is called unitary. Actually, this is not the standard definition. The standard definition
11.4. SCHUR’S THEOREM 191

is given next. First recall that if you have two square matrices of the same size and one acts
like the inverse of the other on one side, then it will act like the inverse on the other side as
well. See Problem 19 on Page 168. The traditional definition of unitary is as follows.

Definition 11.3.1 Let U ∈ Mn×n . Then U is called unitary if U ∗U = UU ∗ = I. When U

consists entirely of real entries, a unitary matrix is called an orthogonal matrix.

Then the following proposition relates this to preservation of lengths of vectors.

Proposition 11.3.2 An n × n matrix U is unitary if and only if |Ux| = |x| for all vectors x.

Proof: First suppose the matrix U preserves all lengths. Since U preserves distances,
|Uu| = |u| for every u. Let u, v be arbitrary vectors in Cn and let θ ∈ C, |θ | = 1, and
θ (U ∗Uu − u, v) = |(U ∗Uu − u, v)|. Therefore from the axioms of the inner product,

|u|2 + |v|2 + 2 Re θ (u, v) = |θ u|2 + |v|2 + θ (u, v) + θ̄ (v, u)

= |θ u + v|2 = (U (θ u + v) ,U (θ u + v))

= (Uθ u,Uθ u) + (Uv,Uv) + (Uθ u,Uv) + (Uv,Uθ u)

= |θ u|2 + |v|2 + θ (U ∗Uu, v) + θ̄ (v,U ∗Uu)

= |u|2 + |v|2 + 2 Re θ (U ∗Uu, v)

and so, subtracting the ends, it follows that for all u, v,

0 = 2 Re θ (U ∗Uu − u, v) = 2 |(U ∗Uu − u, v)|

from the above choice of θ . Now let v = U ∗Uu − u. It follows that

U ∗Uu − u = (U ∗U − I) u = 0.

This is true for all u and so U ∗U = I. Thus it is also true that UU ∗ = I. One can use the
fact shown in Problem 19 on Page 168.
Conversely, if U ∗U = I, then

|Uu|2 = (Uu,Uu) = (U ∗Uu, u) = (u, u) = |u|2

Thus U preserves distance. ■

11.4 Schur’s Theorem

The most significant theorem about eigenvalues and eigenvectors in the space of n × n com-
plex matrices is Schur’s theorem. First is a simple version of the Gram Schmidt theorem.

Definition 11.4.1 A set of vectors in Fn , F = R or C, {x1 , · · · , xk } is called an orthonor-

mal set of vectors if (
1 if i = j
xTi x j = x∗i x j = δ i j ≡
0 if i ̸= j
192 CHAPTER 11. MATRICES AND THE INNER PRODUCT

Note this is the same as saying that (xi , x j ) = δ i j although here it will be slightly more
convenient to define the inner product differently. Indeed, we are really working with the
inner product ⟨x, y⟩ = x∗ y whereas the usual inner product is (x, y) = xT y. This alter-
nate version of the inner product is actually more convenient in matrix theory so we use it
here. The difference is that with this new version, the complex conjugate comes out of the
first entry rather than the second.

What does it mean to say that U ∗U = I which

 is thedefinition for U to be unitary? This
u1 T
.. 
 
says that for U = u1 · · · un , U ∗ = 

 .  and so from the way we multiply

un T
matrices in which the i j entry of the product is the product of the ith row of the matrix
th

on the left with the jth column of the matrix on the right, we have

u∗i u j = δ i j

in other words, the columns of U are orthonormal. From this simple observation, we get
the following important theorem.

Theorem 11.4.2 Let {u1 , · · · , un } be orthonormal. Then it is linearly independent.

Proof: We know from the above discussion that

 
U = u1 · · · un

is unitary. Thus if Ux = 0, you can multiply on the left on both sides with U ∗ and obtain
x = U ∗Ux = U ∗ 0 = 0. Thus, from the definition of linear independence, Definition 9.1.5,
it follows that the columns of U comprise an independent set of vectors. ■

Theorem 11.4.3 Let v 1 be a unit vector (|v 1 | = 1) in Fn , n > 1. Then there exist vectors

{v 2 , · · · , v n }

such that this set of vectors is an orthonormal set of vectors.

Proof: The equation for x, v 1 T x = 0 has a nonzero solution x by Theorem 9.1.4. Pick
such a solution and divide by its magnitude to get v 2 a unit vector such that v 1 T · v 2 = 0.
Now suppose v 1 , · · · , v k have been chosen such that {v 1 , · · · , v k } is an orthonormal set of
vectors. Then consider the equations

v j T x = 0 j = 1, 2, · · · , k

This amounts to the situation of Theorem 9.1.4 in which there are more variables than
equations. Therefore, by this theorem, there exists a nonzero x solving all these equations.
Divide by its magnitude and this gives v k+1 . Continue this way. At the last step, you obtain
v n and the resulting set is an orthonormalset. ■ 
Thus, as observed above, the matrix v 1 · · · v n is a unitary matrix. With this
preparation, here is Schur’s theorem. First is some terminology. An n × n matrix T is called
11.4. SCHUR’S THEOREM 193

upper triangular if it is of the form

 
∗ ··· ∗
 .. .. 

 . . 

0 ∗
meaning that all entries are zero below the main diagonal, consisting of those entries of the
form Tii .
Theorem 11.4.4 Let A be a real or complex n×n matrix. Then there exists a unitary matrix
U such that
U ∗ AU = T, (11.3)
where T is an upper triangular matrix. If A has all real entries and eigenvalues, then U
can be chosen to be orthogonal.
Proof: The theorem is clearly true if A is a 1 × 1 matrix. Just let U = 1 the 1 × 1
matrix which has 1 down the main diagonal and zeros elsewhere. Suppose it is true for
(n − 1) × (n − 1) matrices and let A be an n × n matrix. Then let v 1 be a unit eigenvector
for A. That is, there exists λ 1 such that
Av 1 = λ 1 v 1 , |v 1 | = 1.
By Theorem 11.4.3 there exists {v 1 , · · · , v n }, an orthonormal set in Cn . Let U0 be a matrix
whose ith column is v i . Then from the above, it follows U0 is unitary. Then from the way
you multiply matrices U0∗ AU0 is of the form
 ∗ 
v1
 ∗ 
 v2  
 .  λ 1 v 1 Av 2 · · · Av n =
 . 
 . 
v ∗n
 
λ1 ∗ ··· ∗
 0
 

 . 
 . 
 . A1 
0
where A1 is an n − 1 × n − 1 matrix. Now by induction there exists an (n − 1) × (n − 1)
unitary matrix U
e1 such that
Ue1∗ A1U
e1 = Tn−1 ,
an upper triangular matrix. Consider
!
1 0
U1 ≡
0 U
e1

From the way we multiply matrices, this is a unitary matrix and

! ! ! !
∗ ∗ 1 0 λ1 ∗ 1 0 λ1 ∗
U1 U0 AU0U1 = = ≡T
0 U e∗ 0 A1 0 Ue1 0 Tn−1
1
194 CHAPTER 11. MATRICES AND THE INNER PRODUCT

where T is upper triangular. Then let U = U0U1 . Both of the Ui are unitary and so U must
also be unitary. Indeed
U ∗U = (U0U1 )∗ U0U1 = U1∗U0∗U0U1 = U1∗U1 = I.
Then U ∗ AU = T.
If A is real having real eigenvalues, all of the above can be accomplished using the real
dot product and using real eigenvectors. Thus the unitary matrix can be assumed real. ■
The diagonal entries of T are each eigenvalues of A. This will become clear later when
we discuss the determinant and the characteristic polynomial. However, it is clear right
now that T and A have the same eigenvalues. If T x = λ x for nonzero x, then
U ∗ AUx = λU ∗Ux
U ∗ (AUx − λUx) = 0
Now multiply both sides by U and obtain that Ux is an eigenvector for A. It is nonzero
because U preserves lengths. Similar reasoning shows that every eigenvalue of A is an
eigenvalue of T. Thus one obtains the following important corollary.
Corollary 11.4.5 Let A be an n × n matrix. Then det (A) equals the product of the eigen-
values of A.
Proof: Let U ∗ AU = T where T is upper triangular. Then
product of eigenvalues of A = product of eigenvalues of T = det (T ) = det (A)
The reason for the last equality is that from Theorem 10.2.23,
det (T ) = det U T AU = det U T det (A) det (U) = det U T det (U) det (A)

Now U T U = I and so det U T det (U) = det (I) = 1. ■

The following result is about Hermitian matrices. These are those matrices for which
the upper triangular matrix in Schur’s theorem is actually a real diagonal matrix.
Definition 11.4.6 An n × n matrix A is Hermitian if A = A∗ . Thus a real symmetric matrix
is Hermitian but so is  
1 1−i 3
 1+i 2 i 
 

3 −i 1
In this book, we are mainly interested in real symmetric matrices.
The next theorem is the main result.
Theorem 11.4.7 If A is an n × n Hermitian matrix, there exists a unitary matrix U such
that
U ∗ AU = D (11.4)
where D is a real diagonal matrix. That is, D has nonzero entries only on the main diagonal
and these are real. Furthermore, the columns of U are an orthonormal basis of eigenvectors
for Cn . If A is real and symmetric, then U can be assumed to be a real orthogonal matrix
and the columns of U form an orthonormal basis for Rn . Furthermore, if A is an n × n
matrix and there is a unitary matrix U such that U ∗ AU = D where D is real and diagonal,
then A is Hermitian.
11.4. SCHUR’S THEOREM 195

Proof: From Schur’s theorem above, there exists U unitary (real and orthogonal if A is
real) such that
U ∗ AU = T
where T is an upper triangular matrix. Then from the rules for the transpose,

T ∗ = (U ∗ AU)∗ = U ∗ A∗U = U ∗ AU = T.

Thus T = T ∗ and T is upper triangular. This can only happen if T is really a diagonal
matrix having real entries on the main diagonal. (If i ̸= j, one of Ti j or T ji equals zero. But
Ti j = T ji and so they are both zero. Also Tii = Tii .)
Finally, let  
U = u1 u2 · · · un

where the ui denote the columns of U and

 
λ1 0
D=
 .. 
 . 

0 λn

The equation, U ∗ AU = D implies

 
AU = Au1 Au2 · · · Aun
 
= UD = λ 1 u1 λ 2 u2 · · · λ n un

where the entries denote the columns of AU and UD respectively. Therefore, Aui = λ i ui
and since the matrix is unitary, the i jth entry of U ∗U equals δ i j and so

δ i j = uTi u j = uTi u j = (ui , u j )

This proves the corollary because it shows the vectors {ui } form an orthonormal basis. In
case A is real and symmetric, simply ignore all complex conjugations in the above argu-
ment.
Finally suppose that U ∗ AU = D where D is real and diagonal. Thus D∗ = D. Then

A = UDU ∗

Thus A∗ = UD∗U ∗ = UDU ∗ = A. This last uses the fact that (AB)∗ = B∗ A∗ . ■

Example 11.4.8 Here is a symmetric matrix which has eigenvalues 6, −12, 18

 
1 −4 13
A =  −4 10 −4 
 

13 −4 1

Find a matrix U such that U T AU is a diagonal matrix.

196 CHAPTER 11. MATRICES AND THE INNER PRODUCT

From the above explanation the columns of this matrix U are eigenvectors of unit length
and in fact this is sufficient to obtain the matrix. After doing row operations and then
normalizing the vectors, you obtain
  √   √   √ 
1 1
1 −4 13 6 √6 6 6 6
 1   √   1√ 
 −4 10 −4   3 6  =  2 6  = 6  3 6 

1
√ √ 1
√
13 −4 1 6 6 6 6 6

  √   √   √ 
1 −4 13 − 21 2 6 2 − 12 2
 −4 10 −4   0 = 0  = −12  0
      
√ √ √

1 1
13 −4 1 2 2 −6 2 2 2
  √   √   √ 
1 1
1 −4 13 3 √3 6 3 3 3
 1 √   1√ 
 −4 10 −4   − 3 3  =  −6 3  = 18  − 3 3 
  
1
√ √ 1
√
13 −4 1 3 3 6 3 3 3
Thus the matrix of interest is

1
√ √ √ 
6 √6 − 12 2 1
3 3
1 1
√ 
U = 3 √6 0 −3 3 

1 1
√ 1
√
6 6 2 2 3 3

Then  √ √ √ T  
1
6 √6 − 12 2 1
3 √ 3 1 −4 13
1 1
3 √6 0 − 3 3   −4 10 −4 ·
   
√ √

1 1 1
6 6 2 2 3 3 13 −4 1

1
√ 1
√ 1
√   
6 √6 − 2 2 3 √ 3 6 0 0
1 1
6 0 − 3 = 0 −12 0
   
 3√ √ 3√   
1 1 1
6 6 2 2 3 3 0 0 18

11.5 Diagonalization
Theorem 11.4.7 is a special case of something known as diagonalization.

Definition 11.5.1 An n × n matrix A is diagonalizable if there exists an invertible matrix S

such that
S−1 AS = D
where D is a diagonal matrix.

The following theorem gives the condition under which a matrix is diagonalizable.

Theorem 11.5.2 An n × n matrix S is invertible if and only if its columns are linearly
independent.
11.5. DIAGONALIZATION 197

Proof: First note that if S is n × n and its columns are linearly independent, then these
columns must also span all of Fn since otherwise, there would be a vector v not in the
span and you could add it in to the list and get n + 1 vectors in an independent set. This is
contrary to Theorem 9.1.6. Thus Im (S) = Fn . Also, if Sx = Sy, then S (x − y) = 0 and
so
x = y (S is one to one.). Thus we can define S−1 y to be that vector such that S S−1 y = y.
Then S−1 is a linear transformation because of the following reasoning.
S S−1 (ax + by) = ax + by

S aS−1 x + bS−1 y = aS S−1 x + bS S−1 y

= ax + by
Thus, since S is one to one, as explained above, it follows that
S−1 (ax + by) = aS−1 x + bS−1 y
there is a matrix, still denoted as S−1 such that for any x ∈ Fn , S S−1 x =


Therefore,
SS−1 x −1 on
Page 168, S−1 S = I also. Alternatively,


−1

= x. −1
SS = I. By Problem 19
Hence
S = S. Hence for all x,S S S x = Sx and so S S−1 Sx − Ix = 0 and
−1


S S S = SS
so, since S is one to one, S−1 Sx = Ix for all x showing that S−1 S = I also. ■
Thus if the columns of a matrix are linearly independent, then the matrix is invertible.
On the other hand, if the matrix S is invertible, then if Sx = 0 one could multiply both
sides by S−1 and obtain x = 0 and so the columns of S are linearly independent.
Theorem 11.5.3 An n × n matrix is diagonalizable if and only if Fn has a basis of eigen-
vectors of A. Furthermore, you can take the matrix S described above, to be given as
 
S = s1 s2 · · · sn
where here the sk are the eigenvectors in the basis for Fn . If A is diagonalizable, the
eigenvalues of A are the diagonal entries of the diagonal matrix.
Proof: To say that A is diagonalizable, is to say that
 
λ1
S−1 AS = 
 .. 
 . 

λn
 
the λ i being elements of F. This is to say that for S = s1 · · · sn , sk being the kth
column,  
λ
    1
A s1 · · · sn = s1 · · · sn  .. 
 . 

λn
which is equivalent, from the way we multiply matrices, that
   
As1 · · · Asn = λ 1 s1 · · · λ n sn
which is equivalent to saying that the columns of S are eigenvectors and the diagonal matrix
has the eigenvectors down the main diagonal. Since S−1 is invertible, these eigenvectors
are a basis. Similarly, if there is a basis of eigenvectors, one can take them as the columns
of S and reverse the above steps, finally concluding that A is diagonalizable. ■
198 CHAPTER 11. MATRICES AND THE INNER PRODUCT

11.6 Approximations
11.6.1 Fredholm Alternative
First is a useful proposition which tells when there is a solution to a system of equations.

Ax = b (11.5)

It is based on the simple observation

 that
 the equation has a solution
 if and only ifthe row
reduced echelon form of A | b has no row of the form 0 · · · 0 ▼ .

Proposition 11.6.1 Let A be an m × n matrix and let b be an m × 1 column vector. Then

there exists a solution to 11.5 if and only if
 
rank A | b = rank (A) . (11.6)
 
Proof: Place A | b and A in row reduced echelon form, respectively B and C.
If the above condition on rank is true, then both B and C have the same number of nonzero
rows. In particular, you cannot have in B a row of the form
 
0 ··· 0 ▼

where ▼ ̸= 0. Therefore, there will exist a solution to the system 11.5.

Conversely, suppose there exists a solution. This means there cannot be such a row in B
described above. Therefore, B and C must have the same number of zero rows and so they
have the same number of nonzero rows. Therefore, the rank of the two matrices in 11.6 is
the same.
 say that b is
Another way to see this is as follows. To say there is a solution to 11.5 is to 
in the span of the columns of A which is to say that the rank of A is the rank of A | b
 
because to delete the vector b from the list of column vectors of A | b does not
change the rank. ■
There is a very useful version of Proposition 11.6.1 known as the Fredholm alterna-
tive.
The following definition is used to state the Fredholm alternative.

Definition 11.6.2 Let S ⊆ Rm . Then S⊥ ≡ {z ∈ Rm : z · s = 0 for every s ∈ S} . The funny

exponent, ⊥ is called “perp”.

Now note ( )
m
T
≡ z : AT z = 0 =


N A z: ∑ zk ak = 0
k=1

Here the ak are the rows of A because they are the columns of AT .

Lemma 11.6.3 Let A be a real m × n matrix, let x ∈ Rn and y ∈ Rm . Then

(Ax · y) = x·AT y


11.6. APPROXIMATIONS 199

Proof: This follows right away from the definition of the dot product and matrix multi-
plication.
(Ax · y) = ∑ Akl xl yk = ∑ AT lk xl yk = x · AT y . ■



k,l k,l

Now it is time to state the Fredholm alternative. The first version of this is the following
theorem.

and let b ∈ Rm . There exists a solution x to

Theorem 11.6.4 Let A be a real m × n matrix


T ⊥
the equation Ax = b if and only if b ∈ N A .


⊥
Proof: First suppose b ∈ N AT . Then this says that if AT x = 0, it follows that

b · x = xT b = 0.

In other words, on taking the transpose, if

xT A = 0T then xT b = 0.

Thus, if P is a product of elementary matrices such that PA is in row reduced echelon form,
then if PA has a row of zeros, in the kth position, obtained from the kth row of P times A,
then there is also a zero in the kth position of Pb. This is because the kth position
 in Pb is

th
just the k row of P times b. Thus the row reduced echelon forms of A and A | b
 
have the same number of zero rows. Thus rank A | b = rank (A). By Proposition
11.6.1, there exists
a solution x to the system Ax = b. It remains to prove the converse.
Let z ∈ N AT and suppose Ax = b. I need to verify b · z = 0. By Lemma 11.6.3,

b · z = Ax · z = x · AT z = x · 0 = 0 ■

This implies the following corollary which is also called the Fredholm alternative. The
“alternative” becomes more clear in this corollary.

Corollary 11.6.5 Let A be an m × n matrix. Then A maps Rn onto Rm if and only if the
only solution to AT x = 0 is x = 0.

⊥
Proof: If the only solution to AT x = 0 is x = 0, then N AT = {0} and so N AT =

Rm because every b ∈ Rm has the property that b · 0 = 0. Therefore, Ax = b has a solution

⊥
for any b ∈ Rm because the b for which there is a solution are those in N AT by Theorem
11.6.4. In other words, A maps Rn onto Rm .
Conversely if A is onto, then if AT x = 0, there exists y such that x = Ay and then
A Ay = 0 and so |Ay|2 = Ay · Ay = AT Ay · y = 0 · y = 0 and so x = Ay = 0. ■
T

Here is an amusing example.

Example 11.6.6 Let A be an m × n matrix in which m > n. Then A cannot map onto Rm .

The reason for this is that AT is an n × m where m > n and so in the augmented matrix

AT |0

there must be some free variables. Thus there exists a nonzero vector x such that AT x = 0.
Hence AT is not one to one and so A is not onto.
200 CHAPTER 11. MATRICES AND THE INNER PRODUCT

11.6.2 Least Squares

Suppose there is no solution to the system Ax = b. This happens often in applications
where you want to find the best solution. In other words, you want to find x such that for
all x̂,
|Ax − b| ≤ |Ax̂ − b|
It turns out that the solution to this problem is any solution x to

AT Ax = AT b

So this raises the question whether there is a solution to this last equation.
In order to present this material using notation which is common in more general
situations, we begin to denote the dot product x · y as (x, y). Thus, the property of
the transpose mentioned above about how it interacts with the dot product is written as
(Ax, y) = x,AT y .

Theorem 11.6.7 Let A be a real m × n matrix and let b ∈ Rm . Then there exists a solution
x to the system
AT Ax = AT b

T
Proof: First note that AT A = AT A. Thus, by the Fredholm alternative, it suf-

T ⊥
fices to verify that AT b is in N AT A . So suppose AT Az = 0. Does it follow that
z,AT b = 0? First note that N AT A = N (A) . To see this note that since any matrix times

the zero vector
is zero, the left side is at least as large as the right. But if AT Ax = 0, then
0 = AT
Ax, x = (Ax, Ax) = |Ax|2 so Ax = 0. Hence the two sets are the same. Thus
z,AT b = (Az, b) = (0, b) = 0. By Fredholm alternative, it follows there exists a solution
to the above equation. ■
Next we verify that any solution to this equation is a solution to the least squares prob-
lem of finding x such that Ax is as close as possible to b.

Theorem 11.6.8 |Ax − b| ≤ |Ax̂ − b| for all x̂ in Rn if and only if AT Ax = AT b.

Proof: x is such that Ax is as close as possible to bif and only if |A (x + tz) − b|2 is
minimized when t = 0 for any choice of z. This equals

(A x − b+ tAz, A x − b+ tAz)

Now, expanding this yields

|Ax − b|2 + 2t (Ax − b,Az) + t 2 |Az|2

If x solves the minimization problem, then taking a derivative and setting equal to 0 gives

0 = (Ax − b,Az) = AT (Ax − b) , z = AT Ax − AT b, z

for all z. In particular this holds for z = AT Ax − AT b. Hence AT Ax = AT b.

Conversely, if the equation holds, then 0 = (Ax − b,Az) = AT (Ax − b) , z and so for
any z,
|A (x + tz) − b|2 = |Ax − b|2 + t 2 |Az|2
which shows that the minimization property holds since you could let t = 1 in the above.
■
11.6. APPROXIMATIONS 201

Corollary 11.6.9 |Ax − b| ≤ |Ax̂ − b| for all x̂ ∈ Rn if and only if 0 = (Ax − b,Az) for
every z ∈ Rn .

Proof:This is the content of the above theorem because 0 = (Ax − b,Az) for every
z ∈ Rn if and only if AT Ax = AT b. ■
The corollary says that the vector Ax − b is perpendicular to the subspace Im (A) is the
same as saying that Ax is as close as possible to b. Note that this orthogonality condition
0 = (Ax − b,Az) is equivalent to saying 0 = (Ax − b,A x− Az) = (b − Ax,Az − Ax). Here
is a picture which illustrates the conclusion of this important theorem.
•b
A(Fn ) ≡ Im(A)

• •Ay
Ax

Next consider the problem of projection onto a subspace. Letting V be a subspace of

Rn , and letting b ∈ Rn , how do we find x ∈ V such that |b − x| ≤ |b − y| for every y ∈ V ?
The subspace has a basis, {v 1 , · · · , v m }, m ≤ n. Let
 
A = v1 · · · vm 0 · · · 0

Thus V is the column space of A, the span of the columns of A which is also Im (A). Thus
the question is to find Ay which is closer to b than any Az. Isn’t this just what was solved
above? Then the closest point to b in V will be x ≡ Ay. From the above explanation, y
must satisfy  
x u
AT Ay = AT b so b−Ay, Az =0

for all z ∈ Rn . In other words, you need x to be the point in V which satisfies (b − x, u) = 0
for all u in V . Note that since x is in V , a generic point of V is of the form u − x. Thus it
makes no difference whether we write (b − x, u) = 0 for all u in V or (b − x, u − x) = 0
for all u in V . The following picture illustrates what was just shown.
•b
A(Fn ) ≡ Im(A) ≡ V

• •u
x

Theorem 11.6.10 Let V be a finite dimensional subspace of Rn and let b ∈ Rn . Then there
exists a unique point of V which is closest to b out of all points of V . This point x is
characterized by the equation
(b − x, z) = 0 (11.7)
for all z ∈ V.
202 CHAPTER 11. MATRICES AND THE INNER PRODUCT

Proof: The existence of this point follows from Theorem 11.6.8. However, to empha-
size the uniqueness, suppose x, x̂ both are closest points. Then from the characterization
of these points in 11.7,

(b − x, x − x̂) = 0, (b − x̂, x − x̂) = 0

Then subtracting these yields (x − x̂, x − x̂) = 0 thus showing uniqueness. ■

11.6.3 Regression lines

In experimental work, one often wants to determine relationships which are not exactly
determined. For example, you might want to find whether a vaccine is effective. In terms
of linear equations, this amounts to there being no solution to a system of equations but
still needing to answer a question about the data.

Example 11.6.11 The least squares regression line is the line y = mx + b which approxi-
mates data points (xi , yi ) which typically come from some sort of experiment. It is desired
to choose m, b in such a way that the sum of the squares of the errors between the value
predicted by the line and the observed values is as small as possible. In other words, you
want to minimize ∑i (yi − (mxi + b))2 . Ideally, the sum would be zero and this would cor-
respond to the data points being on a straight line. This will never occur in any realistic
situation in which the data points come from experiments.

Suppose you are given points in xy plane

{(xi , yi )}ni=1

and you would like to find constants m and b such that the line y = mx + b goes through all
these points. Of course this will be impossible in general. Therefore, try to find m, b to get
as close as possible. The desired system is
   
y1 x1 1 ! !
 .   . .  m m
y =  ..  =  .. .. 
    ≡A
b b
yn xn 1

which is of the form y = Ax and it is desired to choose m and b to make

  2
! y1
m  . 
A − . 
b  . 
yn

as small as possible. According to Theorem 11.6.8, the best values for m and b occur as the
solution to    
! y1 x1 1
m  . 
AT A = AT  ..  , A =  ... ...  .
 
b    
yn xn 1
11.6. APPROXIMATIONS 203

Thus, after computing AT A, AT y

! ! !
∑ni=1 xi2 ∑ni=1 xi m ∑ni=1 xi yi
=
∑ni=1 xi n b ∑ni=1 yi

Solving this system of equations for m and b,

− (∑ni=1 xi ) (∑ni=1 yi ) + (∑ni=1 xi yi ) n

m= 2
∑ni=1 xi2 n − (∑ni=1 xi )

and
− (∑ni=1 xi ) ∑ni=1 xi yi + (∑ni=1 yi ) ∑ni=1 xi2
b= 2
.
∑ni=1 xi2 n − (∑ni=1 xi )

One could clearly do a least squares fit for curves of the form y = ax2 + bx + c in the
same way. In this case you want to solve as well as possible for a, b, and c the system
    
x12 x1 1 a y1
 . .. ..    . 
 
 .
 . . .   b  =  .. 
xn2 xn 1 c yn

and one would use the same technique as above. Many other similar problems are impor-
tant, including many in higher dimensions and they are all solved the same way.

Example 11.6.12 Find the least squares regression line for the data

(0, 1) , (2, 3) , (2, 4) , (3, 4) , (3, 5) , (4, 6) , (4, 5)

You would ideally want to solve the following system of equations

   
0 1 1

 2 1 


 3 

  !  
 2 1   4 
  m  
 3 1  = 4 
  b  
3 1 5
   
   
   
 4 1   6 
4 1 5

Of course there is no solution so you look for a least squares solution. You have AT A equals
!
58 18
18 7

and AT b is !
85
28
204 CHAPTER 11. MATRICES AND THE INNER PRODUCT

and so you need to solve

! ! !
58 18 m 85
=
18 7 b 28

The solution is: ! !

91
82 1. 109 8
47 =
41 1. 146 3
Thus the least squares line is
y = 1. 109 8x + 1. 146 3
If you graph these data points and the line, you will see how the line tries to do the impos-
sible by picking a route through the data points which minimizes the error which results.

0
0 2 4

11.6.4 Identifying the Closest Point

For V a finite dimensional subspace as in the above theorem, how can we identify the
closest point to b in Theorem 11.6.10? Suppose a basis for V is {v 1 , · · · , v m }. Then we
would have unique scalars ck (The ck are unique because the y is unique. )
m
y= ∑ ck vk
k=1

Therefore, taking inner products, it follows that for each j,

m
y, v j = ∑ ck (vk , v j ) (11.8)
k=1

Wouldn’t it be nice if (v k , v j ) = δ k j ? Recall that δ i j equals 1 if i = j and 0 if i ̸= j. If

this happens, the vectors {v 1
, · · · , v m } are called an orthonormal set of vectors. In this case,
you would have c j = y, v j because on the right, the sum would reduce to c j . In fact, if
you have a basis of vectors {v 1 , · · · , v m } , there always exists another basis {u1 , · · · , um }
which is orthonormal. Furthermore, there is an algorithm for finding this improved basis.
It is called the Gram Schmidt process.

Lemma 11.6.13 Let {v 1 , · · · , v n } be a linearly independent subset of R p , p ≥ n. Then

there exists orthonormal vectors {u1 , · · · , un } which have the property that for each k ≤ n,
span (v 1 , · · · , v k ) = span (u1 , · · · , uk ) .
11.6. APPROXIMATIONS 205

Proof: Let u1 ≡ v 1 / |v 1 | . Thus for k = 1, span (u1 ) = span (v 1 ) and {u1 } is an

orthonormal set. Now suppose for some k < n, u1 , · · · , uk have been chosen such that
(u j , ul ) = δ jl and span (v 1 , · · · , v k ) = span (u1 , · · · , uk ). Then define

v k+1 − ∑kj=1 (v k+1 , u j ) u j

uk+1 ≡ , (11.9)
v k+1 − ∑kj=1 (v k+1 , u j ) u j

where the denominator is not equal to zero because the v j form a basis, and so
v k+1 ∈
/ span (v 1 , · · · , v k ) = span (u1 , · · · , uk )
Thus by induction,
uk+1 ∈ span (u1 , · · · , uk , v k+1 ) = span (v 1 , · · · , v k , v k+1 ) .
Also, v k+1 ∈ span (u1 , · · · , uk , uk+1 ) which is seen easily by solving 11.9 for v k+1 , and it
follows
span (v 1 , · · · , v k , v k+1 ) = span (u1 , · · · , uk , uk+1 ) .
If l ≤ k, !
k
(uk+1 , ul ) = C (v k+1 , ul ) − ∑ (v k+1 , u j ) (u j , ul ) =
j=1
!
k
C (v k+1 , ul ) − ∑ (v k+1 , u j ) δ l j = C ((v k+1 , ul ) − (v k+1 , ul )) = 0.
j=1
 n
The vectors, u j j=1 , generated in this way are therefore orthonormal because each vector
has unit length. ■
Corollary 11.6.14 If you have a basis for R p ,

u1 , · · · , um , um+1 , · · · , u p
and {u1 , · · · , um } is orthonormal, then when the Gram Schmidt process is used on this
basis, it returns {u1 , · · · , um } . Thus it is always possible to extend an orthonormal set of
vectors to an orthonormal basis.
Proof:This follows right away from the algorithm. ■
Did we ever use the fact that all of this is taking place in R p ? No, this was never used at
all! In fact everything in the Gram Schmidt process holds if V is a subspace of an arbitrary
inner product space. You just need something which is a vector space which has an inner
product to have it all work out exactly the same. A vector space is something in which
you can add the “vectors” and multiply them by scalars in the usual way which we do for
vectors in Rn .
Now return to the stated problem which was to compute the closest point in V . This is
the content of the next theorem.
Theorem 11.6.15 Let V be an m dimensional subspace of R p having orthonormal basis
{u1 , · · · , um }. Let b ∈ R p and let y be the point of V closest to b. Then
m
y= ∑ (b, uk ) uk (11.10)
k=1
206 CHAPTER 11. MATRICES AND THE INNER PRODUCT

Proof:We only need to show that this satisfies the orthogonality condition 11.7. But
this is fairly obvious because, from properties of the inner product and y given above,
!

m m

y, u j = ∑ (b, uk ) uk , u j = ∑ (b, uk ) (uk , u j ) = b, u j
k=1 k=1


Thus b − y, u j = 0. Since this holds for every basis vector, it holds for every z ∈ V also.
!
m
m

(z, b − y) = ∑ z, u j u j , b − y = ∑ z, u j (u j , b − y) = 0
j=1 j=1

Therefore, the orthogonality condition holds for y given by the above formula and so y
equals the above sum in 11.10. ■
The sum in 11.10 is the Fourier series approximation to b. The scalars (b, uk ) are the
Fourier coefficients. Note that all this works any time you have a norm which comes from
an inner product, something which satisfies the same axioms as the dot product. That is,
|x| = (x, x) where (·, ·) satisfies the inner product axioms:
1. ( f , g) = (g, f )
2. (a f + bg, h) = a ( f , h) + b (g, h)
3. ( f , f ) ≥ 0 and equals 0 only if f = 0
The conjugate is placed on the (g, f ) to include the case of a complex inner product.
Just ignore it in the case where the scalars are real numbers.
Now we generalize these ideas more.
Theorem 11.6.16 Let V be a finite dimensional subspace of an inner product space X,
something with an inner product. (X is a nonempty set which satisfies the vector space
axioms. In addition it has an inner product satisfying the inner product axioms.) If b ∈ X,
there exists a unique y ∈ V such that |b − y| ≤ |b − z| for all z ∈ V . This point is character-
ized by (b − y, z) = 0 for all z ∈ V .
Proof: Letting t ∈ R,
|b − (y + tz)|2 = |b − y|2 − 2t (b − y, z) + t 2 |z|2
If y is closest to b then taking the derivative and setting t = 0, we must have (b − y, z) = 0.
Conversely, if this equals zero, let t = 1 and you have
|b − (y + z)|2 = |b − y|2 + |z|2
and so y solves the minimization property. It only remains to show the existence of such y
satisfying (b − y, z) = 0. However, using the Gram Schmidt process, there is an orthonormal
basis {u1 , · · · , un } whose span is V . Then all that remains is to verify that ∑ni=1 (b, ui ) ui
satisfies the orthogonality condition. Indeed,
!
n n
b − ∑ (b, ui ) ui , u j = (b, u j ) − ∑ (b, ui ) δ i j = 0
i=1 i=1

Since the ui are a basis, it follows that (b − y, z) = 0 for all z ∈ V . ■

Note that any time the norm comes from an inner product, something which satisfies
the properties of the dot product, all of this holds. You don’t need to be considering vectors
in Rn . It was only the axioms of the inner product which were used.
11.7. THE SINGULAR VALUE DECOMPOSITION∗ 207

11.6.5 Using MATLAB

You may notice that the Gram Schmidt process is pretty tedious but routine. Therefore, it
is not surprising that it can be automated using MATLAB. One way to do it would be to
use what is known as the QR factorization. Given a real m × n matrix A which has linearly
independent columns, you can always write it in the form A = QR where Q is an orthogonal
matrix and R is an upper triangular matrix in the sense that all entries are 0 below the main
diagonal. Actually, the computer algebra system doesn’t use the Gram Schmidt process. It
uses something called Householder reflections, but one obtains the same essentials although
sometimes a different set of vectors, the columns of Q being an orthonormal set. The span
of these columns will coincide with the span of the columns of A and in fact, the span of
the first k columns of A will be the span of the first k columns of Q just as in the Gram
Schmidt process. Here is the syntax:
>>A=[1,2,3;4,2,1;2,6,7;1,-4,2];[Q,R]=qr(A)
Then press enter and you get the following.

Q= R=
-0.2132 0.1756 -0.2593 0.9255 -4.6904 -3.8376 -4.9036
-0.8528 -0.1892 0.4862 -0.0244 0 6.7285 3.4453
-0.4264 0.6485 -0.5139 -0.3653 0 0 -5.2043
-0.2132 -0.7161 -0.6575 -0.0974 000
If you want to see something horrible, replace qr(A) with qr(sym(A)). This way it gives
the exact values. You can check your work by >>Q*Q’ and press enter. The Q’ means the
conjugate transpose in MATLAB. Since everything is real here, this is just the transpose.
There is so much more that could be discussed about the QR factorization, but this will
suffice here.
As to plotting data with a curve as in the least squares example, use the following
syntax.
x=-1:.1:5;
y=1.11*x+1.146;
plot(x,y,0,1,’*’,2,3,’*’,2,4,’*’,3,4,’*’,3,5,’*’,4,6,’*’,4,5,’*’)
In MATLAB, you press shift enter to get to a new line and you press enter to get it to
do something.

11.7 The Singular Value Decomposition∗

In this section, A will be an m × n matrix. To begin with, here is a simple lemma.
Lemma 11.7.1 Let A be an m × n matrix. Then A∗ A is Hermitian and all its eigenvalues
are nonnegative.
Proof:It is obvious that A∗ A is Hermitian because (A∗ A)∗ = A∗ (A∗ )∗ = A∗ A. Suppose
A∗ Ax= λ x. Then
λ |x|2 = (λ x, x) = (A∗ Ax, x) = (Ax,Ax) ≥ 0. ■
Definition 11.7.2 Let A be an m × n matrix. The singular values of A are the square roots
of the positive eigenvalues of A∗ A.
208 CHAPTER 11. MATRICES AND THE INNER PRODUCT

With this definition and lemma here is the main theorem on the singular value decom-
position.
Theorem 11.7.3 Let A be an m × n matrix. Then there exist unitary matrices, U and V of
the appropriate size such that !
∗ σ 0
U AV =
0 0
where σ is of the form  
σ1 0

σ = .. 
 . 

0 σk
for the σ i the singular values of A.
Proof: By the above lemma and Theorem 11.4.7 there exists an orthonormal basis,
{v i }ni=1 such that A∗ Av i = σ 2i v i where σ 2i > 0 for i = 1, · · · , k, (σ i > 0) and equals zero if
i > k. Thus for i > k, Av i = 0 because
(Av i , Av i ) = (A∗ Av i , v i ) = (0, v i ) = 0.
For i = 1, · · · , k, define ui ∈ Fm by
ui ≡ σ −1
i Av i .

Thus Av i = σ i ui . Now
   
(ui , u j ) = σ −1
i Av i , σ −1
j Av j = σ −1
i v i , σ −1 ∗
j A Av j
  σ
j
= σ −1 −1 2
i vi , σ j σ j v j = (v i , v j ) = δ i j .
σi
Thus {ui }ki=1 is an orthonormal set of vectors in Fm . Also,
AA∗ ui = AA∗ σ −1 −1 ∗ −1 2 2
i Av i = σ i AA Av i = σ i Aσ i v i = σ i ui .

Now extend {ui }ki=1 to an orthonormal basis for all of Fm , {ui }m

i=1 and let

U ≡ (u1 · · · um )
while V ≡ (v 1 · · · v n ) . Thus U is the matrix which has the ui as columns and V is defined
as the matrix which has the v i as columns. Then
 ∗   ∗ 
u1 u1
 .   . 
 ..   ..  !
    σ 0
U ∗ AV =  u∗k  A (v 1 · · · v n ) =  u∗k  (σ 1 u1 · · · σ k uk , 0 · · · 0) =
   
 .. 
 
 .. 
  0 0
 .   . 
u∗m u∗m
where σ is given in the statement of the theorem. ■
The singular value decomposition has as an immediate corollary the following interest-
ing result.
11.8. EXERCISES 209

Corollary 11.7.4 Let A be an m × n matrix. Then the rank of A and A∗ equals the number
of singular values.

Proof:Since V and U are unitary, it follows that

!
σ 0
rank (A) = rank (U ∗ AV ) = rank
0 0
= number of singular values.

Also since U,V are unitary,

rank (A∗ ) = rank (V ∗ A∗U) = rank (U ∗ AV )∗

!∗ !
σ 0
= rank = number of singular values. ■
0 0
This is based on the simple observation that for A an m × n matrix, the dimension of
Im (A) is the same as the dimension of Im (UAV ) if U,V are invertible matrices of the
right size. Indeed, Im (UAV ) = Im (UA) because V being invertible maps Fn onto Fn . The
dimension of Im (UA) and the dimension of Im (A) must be the same because U is one to
one. Thus if a basis for Im (A) is {a1 , · · · , ak } , columns of A, then a basis for UA will be
{Ua1 , · · · ,Uak } .

11.8 Exercises
1. Let {u1 , · · · , un } be a basis for Fn and define a mapping T : Fn → span (v 1 , · · · , v r )
as follows. !
n r
T ∑ a k uk ≡ ∑ ak vk
k=1 k=1

Explain why this is a linear transformation.

2. In the above problem, suppose v k = uk . Show that T v = v if v ∈ V ≡ span (u1 , · · · , ur ) .

Now show that T (T (x)) = T (x) and that |T x − T y| ≤ |x − y| .
3. Find the minimum polynomials for the following matrices and use to obtain the
eigenvalues of the matrix. The set of all eigenvectors associated with an eigenvalue
λ is called the eigenspace. Determine the eigenspaces for each of these matrices.
!
9 4
(a)
−20 −9
!
−3 −2
(b)
10 6
 
5 −2 2
(c)  2 0 1 
 

−4 2 −1
210 CHAPTER 11. MATRICES AND THE INNER PRODUCT
 
1 0 0
(d)  1 1 1 
 

1 0 2
 
2 0 1
(e)  1 1 1 
 

−1 0 0
 
6 −2 3
(f)  3 0 2 
 

−5 2 −2

4. Suppose you have p (λ ) is the minimum polynomial for a square n × n matrix A.

Show that this matrix is invertible if and only if the constant term of the minimum
polynomial is non zero. In this case, give a formula for A−1 in terms of powers of A.
Say
p (λ ) = λ n + an−1 λ n−1 + · · · + a1 λ + a0
Thus you need explain why a0 ̸= 0 if A−1 exists and then find a formula for A−1
when this is the case.
5. Find least squares solutions to the following systems of equations.
   
1 2 ! 1
 x
(a)  −1 1  = 1 
  
y
2 1 1
! ! !
1 1 x 1
(b) =
2 2 y 1
    
1 0 1 x 1
(c)  1 1 0   y  =  0 
    

2 1 1 z 2

6. Here are some matrices. Label according to whether they are symmetric, skew sym-
metric, or orthogonal. If the matrix is orthogonal, determine whether it is proper or
improper.
     
1 0 0 1 2 −3 0 −2 −3
√ √  
(a)  0 1/ 2 −1/ 2  (b)  2 1 4  (c)  2 0 −4 
   
√ √
0 1/ 2 1/ 2 −3 4 7 3 4 0

7. Show that every real matrix may be written as the sum of a skew symmetric

and a
symmetric matrix. Hint: If A is an n × n matrix, show that B ≡ 12 A − AT is skew
symmetric.
T
8. Let x be a vector in Rn and consider the matrix I − 2xx2 . Show this matrix is both
||x||
symmetric and orthogonal.
11.8. EXERCISES 211

9. For U an orthogonal matrix, explain why ||Ux|| = ||x|| for any vector x. Next ex-
plain why if U is an n × n matrix with the property that ||Ux|| = ||x|| for all vectors,
x, then U must be orthogonal. Thus the orthogonal matrices are exactly those which
preserve distance. This was done in general in the chapter for unitary matrices. Do
it here for the special case that the matrix is orthogonal. It will be simpler.
10. A quadratic form in three variables is an expression of the form a1 x2 + a2 y2 + a3 z2 +
a4 xy + a5 xz + a6 yz. Show that every such quadratic form may be written as
 
  x
x y z A y 
 

z
where A is a symmetric matrix.
11. Given a quadratic form in three variables, x, y, and z, show there exists an orthogonal
 T  T
matrix U and variables x′ , y′ , z′ such that x y z = U x′ y′ z′ with
the property that in terms of the new variables, the quadratic form is

2
2
2
λ 1 x′ + λ 2 y′ + λ 3 z′
where the numbers, λ 1 , λ 2 , and λ 3 are the eigenvalues of the matrix A in Problem
10.
12. If A is a symmetric invertible matrix, is it always the case that A−1 must be symmetric
also? How about Ak for k a positive integer? Explain.
13. If A, B are symmetric matrices, does it follow that AB is also symmetric?
14. Suppose A, B are symmetric and AB = BA. Does it follow that AB is symmetric?
15. Here are some matrices. What can you say about the eigenvalues of these matrices
just by looking at them?
   
0 0 0 0 −2 −3
(a)  0 0 −1  (c)  2 0 −4 
   

0 1 0 3 4 0
   
1 2 −3 1 2 3
(b)  2 1 4  (d)  0 2 3 
   

−3 4 7 0 0 2
 
c 0 0
16. Find the eigenvalues and eigenvectors of the matrix  0 0 −b  . Here b, c are
 

0 b 0
real numbers.
 
c 0 0
17. Find the eigenvalues and eigenvectors of the matrix  0 a −b . Here a, b, c
 

0 b a
are real numbers.
212 CHAPTER 11. MATRICES AND THE INNER PRODUCT

18. Find the eigenvalues and an orthonormal basis of eigenvectors for A.

 
11 −1 −4
A =  −1 11 −4  .
 

−4 −4 14

Hint: Two eigenvalues are 12 and 18.

19. Find the eigenvalues and an orthonormal basis of eigenvectors for A.
 
4 1 −2
A= 1 4 −2  .
 

−2 −2 7

Hint: One eigenvalue is 3.

20. Show that if A is a real symmetric matrix and λ and µ are two different eigenvalues,
then if x is an eigenvector for λ and y is an eigenvector for µ, then x · y = 0. Also all
eigenvalues are real. Supply reasons for each step in the following argument. First

λ xT x = (Ax)T x = xT Ax = xT Ax = xT λ x = λ xT x

and so λ = λ . This shows that all eigenvalues are real. It follows all the eigenvectors
are real. Why? Now let x, y, µ and λ be given as above.

λ (x · y) = λ x · y = Ax · y = x · A y = x· µy = µ (x · y) = µ (x · y)

and so
(λ − µ) (x · y) = 0.
Since λ ̸= µ, it follows x · y = 0.
21. Suppose U is an orthogonal n × n matrix. Explain why rank (U) = n.
22. Show that if A is an Hermitian matrix and λ and µ are two different eigenvalues, then
if x is an eigenvector for λ and y is an eigenvector for µ, then (x, y) = 0. Also all
eigenvalues are real. Supply reasons for each step in the following argument. First

λ (x, x) = (Ax, x) = (x,Ax) = (x,λ x) = λ (x, x)

and so λ = λ . This shows that all eigenvalues are real. Now let x, y, µ and λ be
given as above.

λ (x, y) = (λ x, y) = (Ax, y) = (x, Ay) = (x,µy) = µ (x, y) = µ (x, y)

and so (λ − µ) (x, y) = 0. Since λ ̸= µ, it follows (x, y) = 0.

23. Show that the eigenvalues and eigenvectors of a real matrix occur in conjugate pairs.
24. If a real matrix A has all real eigenvalues, does it follow that A must be symmetric.
If so, explain why and if not, give an example to the contrary.
11.8. EXERCISES 213

25. Suppose A is a 3 × 3 symmetric matrix and you have found two eigenvectors which
form an orthonormal set. Explain why their cross product is also an eigenvector.
26. Determine which of the following sets of vectors are orthonormal sets. Justify your
answer.

(a) {(1, 1) , (1, −1)}

n  o
√1 , √ −1
(b) 2 2
, (1, 0)
 1 2 2
−2 −1 2
2 −2 1


(c) 3 3 3 , 3 , 3 ,3 ,
, , 3, 3 , 3

27. Show that if {u1 , · · · , un } is an orthonormal set of vectors in Fn , then it is a basis.

Hint: It was shown earlier that this is a linearly independent set.
28. Fill in the missing entries to make the matrix orthogonal.
 −1 −1 1 
√ √ √
2 6 3
 
 
 √1 .
 2 
 √ 
6
3

29. Fill in the missing entries to make the matrix orthogonal.


2
√
2 1
√ 
3 2 6 2
 
 2 
 3 
 
0

30. Fill in the missing entries to make the matrix orthogonal.

 1 
− √2
3 5
 
 2 0 
 3 

4
√ 
15 5

31. Find the eigenvalues and an orthonormal basis of eigenvectors for A. Diagonalize A
by finding an orthogonal matrix U and a diagonal matrix D such that U T AU = D.
 
−1 1 1
A =  1 −1 1  .
 

1 1 −1
Hint: One eigenvalue is -2.
32. Find the eigenvalues and an orthonormal basis of eigenvectors for A. Diagonalize A
by finding an orthogonal matrix U and a diagonal matrix D such that U T AU = D.
 
17 −7 −4
A =  −7 17 −4  .
 

−4 −4 14
214 CHAPTER 11. MATRICES AND THE INNER PRODUCT

Hint: Two eigenvalues are 18 and 24.

33. Find the eigenvalues and an orthonormal basis of eigenvectors for A. Diagonalize A
by finding an orthogonal matrix U and a diagonal matrix D such that U T AU = D.
 
13 1 4
A =  1 13 4  .
 

4 4 10
Hint: Two eigenvalues are 12 and 18.
34. Find the eigenvalues and an orthonormal basis of eigenvectors for A. Diagonalize A
by finding an orthogonal matrix U and a diagonal matrix D such that U T AU = D.
√ √ √ 
− 53 1 8

15 6 5 15 5
√ √ √ 
 

1
 6 5 − 14 1
− 15 6 
A=

15 5 


8
√ 1
√ 7


15 5 − 15 6 15


Hint: The eigenvalues are −3, −2, 1.

35. Find the eigenvalues and an orthonormal basis of eigenvectors for A. Diagonalize A
by finding an orthogonal matrix U and a diagonal matrix D such that U T AU = D.
 
3 0 0
 0 3 1 
A=
 2 2 .

1 3
 
0 2 2

36. Find the eigenvalues and an orthonormal basis of eigenvectors for A. Diagonalize A
by finding an orthogonal matrix U and a diagonal matrix D such that U T AU = D.
 
2 0 0
A =  0 5 1 .
 

0 1 5

37. Find the eigenvalues and an orthonormal basis of eigenvectors for A. Diagonalize A
by finding an orthogonal matrix U and a diagonal matrix D such that U T AU = D.

4 1
√ √ 1
√ 
3 3 3 2 3 2
 

1
√ √ 1
√ 
3 2 1 −3 3 
 
A= 3
√ √
 
 1


3 2 − 13 3 5
3


Hint: The eigenvalues are 0, 2, 2 where 2 is listed twice because it is a root of multi-
plicity 2.
11.8. EXERCISES 215

38. Find the eigenvalues and an orthonormal basis of eigenvectors for A. Diagonalize A
by finding an orthogonal matrix U and a diagonal matrix D such that U T AU = D.

1
√ √ 1
√ √ 
1 6 3 2 6 3 6
√ √ √ √
 
1 3 1
 
6 3 2 12 2 6
 
A= 2 
√ √ √ √
 
 1 1 1


6 3 6 12 2 6 2


Hint: The eigenvalues are 2, 1, 0.

39. Find the eigenvalues and an orthonormal basis of eigenvectors for the matrix
 1 1
√ √ 7
√ √ 
3 6 3 2 − 18 3 6

 1 √3√2 √ √ 
 

3 1
 6 2 − 12 2 6 
 
 − 7 √3√6 √ √
 
1
18 − 12 2 6 −5 
6

Hint: The eigenvalues are 1, 2, −2.

40. Find the eigenvalues and an orthonormal basis of eigenvectors for the matrix
√ √ √ 
− 12 − 15 6 5 1

10 5

 − 1 √6√5 √ 
 

7 1
 5 5 −5 6  
 

1
√ √ 
 5
10 − 15 6 − 9
10


Hint: The eigenvalues are −1, 2, −1 where −1 is listed twice because it has multi-
plicity 2 as a zero of the characteristic equation.
41. Explain why a real matrix A is symmetric if and only if there exists an orthogonal
matrix U such that A = U T DU for D a diagonal matrix.
42. You are doing experiments and have obtained the ordered pairs,

(0, 1) , (1, 2) , (2, 3.5) , (3, 4)

Find m and b such that y = mx + b approximates these four points as well as possible.
Now do the same thing for y = ax2 + bx + c, finding a, b, and c to give the best
approximation.
43. Suppose you have several ordered triples, (xi , yi , zi ) . Describe how to find a polyno-
mial,
z = a + bx + cy + dxy + ex2 + f y2
216 CHAPTER 11. MATRICES AND THE INNER PRODUCT

for example giving the best fit to the given ordered triples. Is there any reason you
have to use a polynomial? Would similar approaches work for other combinations of
functions just as well?
44. Find an orthonormal basis for the spans of the following sets of vectors.

(a) (3, −4, 0) , (7, −1, 0) , (1, 7, 1).

(b) (3, 0, −4) , (11, 0, 2) , (1, 1, 7)
(c) (3, 0, −4) , (5, 0, 10) , (−7, 1, 1)

45. Using the Gram Schmidt process or the QR factorization, find an orthonormal basis
for the span of the vectors, (1, 2, 1) , (2, −1, 3) , and (1, 0, 0) .
46. Using the Gram Schmidt process or the QR factorization, find an orthonormal basis
for the span of the vectors, (1, 2, 1, 0) , (2, −1, 3, 1) , and (1, 0, 0, 1) .
47. The set, V ≡ {(x, y, z) : 2x + 3y − z = 0} is a subspace of R3 . Find an orthonormal
basis for this subspace.
48. The two level surfaces, 2x + 3y − z + w = 0 and 3x − y + z + 2w = 0 intersect in a
subspace of R4 , find a basis for this subspace. Next find an orthonormal basis for
this subspace.
49. Let A, B be a m × n matrices. Define an inner product on the set of m × n matrices by

(A, B)F ≡ trace (AB∗ ) .

Show this is an inner product satisfying all the inner product axioms. Recall for M an
n × n matrix, trace (M) ≡ ∑ni=1 Mii . The resulting norm, ||·||F is called the Frobenius
norm and it can be used to measure the distance between two matrices.

50. Let A be an m × n matrix. Show ||A||2F ≡ (A, A)F = ∑ j σ 2j where the σ j are the
singular values of A.
51. The trace of an n × n matrix M is defined as ∑i Mii . In other words it is the sum of the
entries on the main diagonal. If A, B are n×n matrices, show trace (AB) = trace (BA).
Now explain why if A = S−1 BS it follows trace (A) = trace (B). Hint: For the first
part, write these in terms of components of the matrices and it just falls out.
52. Using Problem 51 and Schur’s theorem, show that the trace of an n × n matrix equals
the sum of the eigenvalues.
53. If A is a general n × n matrix having possibly repeated eigenvalues, show there is a
sequence {Ak } of n × n matrices having distinct eigenvalues which has the property
that the i jth entry of Ak converges to the i jth entry of A for all i j. Hint: Use Schur’s
theorem.
Chapter 12

Vector Valued Functions

12.1 Vector Valued Functions

Vector valued functions have values in R p where p is an integer at least as large as 1. Here
are some examples.

Example 12.1.1 A rocket is launched from the rotating earth. You could define a function
having values in R3 as (r (t) , θ (t) , φ (t)) where r (t) is the distance of the center of mass
of the rocket from the center of the earth, θ (t) is the longitude, and φ (t) is the latitude of
the rocket.

Example 12.1.2 Let f (x, y) = sin xy, y3 + x, x4 . Then f is a function defined on R2 which

has values in R3 . For example, f (1, 2) = (sin 2, 9, 16).

As usual, D (f ) denotes the domain of the function f which is written in bold face be-
cause it will possibly have values in R p . When D (f ) is not specified, it will be understood
that the domain of f consists of those things for which f makes sense.
 √ 
Example 12.1.3 Let f (x, y, z) = x+y 2
z , 1 − x , y . Then D (f ) would consist of the set of
all (x, y, z) such that |x| ≤ 1 and z ̸= 0.

There are many ways to make new functions from old ones.

Definition 12.1.4 Let f , g be functions with values in R p . Let a, b be points of R (scalars).

Then af + bg is the name of a function whose domain is D (f ) ∩ D (g) which is defined as

(af + bg) (x) = af (x) + bg (x) .

f · g or (f , g) is the name of a function whose domain is D (f ) ∩ D (g) which is defined as

(f , g) (x) ≡ f · g (x) ≡ f (x) · g (x) .

If f and g have values in R3 , define a new function f × g by

f × g (t) ≡ f (t) × g (t) .

217
218 CHAPTER 12. VECTOR VALUED FUNCTIONS

If f : D (f ) → X and g : X → Y , then g ◦ f is the name of a function whose domain is

{x ∈ D (f ) : f (x) ∈ D (g)}

which is defined as
g ◦ f (x) ≡ g (f (x)) .
This is called the composition of the two functions.

You should note that f (x) is not a function. It is the value of the function at the point
x. The name of the function is f . Nevertheless, people often write f (x) to denote a
function and it does not cause too many problems in beginning courses. When this is done,
the variable, x should be considered as a generic variable free to be anything in D (f ). I
will use this slightly sloppy abuse of notation whenever convenient.

Example 12.1.5 Let f (t) ≡ (t, 1 + t, 2) and g (t) ≡ t 2 ,t,t . Then f · g is the name of the

function satisfying

f · g (t) = f (t) · g (t) = t 3 + t + t 2 + 2t = t 3 + t 2 + 3t

Note that in this case is was assumed the domains of the functions consisted of all of R
because this was the set on which the two both made sense. Also note that f and g map R
into R3 but f · g maps R into R.

Example 12.1.6 Suppose f (t) = 2t, 1 + t 2 and g : R2 → R is given by g (x, y) ≡ x + y.

Then g ◦ f : R → R and

g ◦ f (t) = g (f (t)) = g 2t, 1 + t 2 = 1 + 2t + t 2 .

12.2 Vector Fields

Some people find it useful to try and draw pictures to illustrate a vector valued function.
This can be a very useful idea in the case where the function takes points in D ⊆ R2 and
delivers a vector in R2 . For many points (x, y) ∈ D, you draw an arrow of the appropriate
length and direction with its tail at (x, y). The picture of all these arrows can give you an
understanding of what is happening. For example if the vector valued function gives the
velocity of a fluid at the point (x, y), the picture of these arrows can give an idea of the
motion of the fluid. When they are long the fluid is moving fast, when they are short, the
fluid is moving slowly. The direction of these arrows is an indication of the direction of
motion. The only sensible way to produce such a picture is with a computer. Otherwise,
it becomes a worthless exercise in busy work. Furthermore, it is of limited usefulness in
three dimensions because in three dimensions such pictures are too cluttered to convey
much insight.

Example 12.2.1 Draw a picture of the vector field (−x, y) which gives the velocity of a
fluid flowing in two dimensions.
12.2. VECTOR FIELDS 219

You can see how the arrows indicate the motion of this fluid.

Example 12.2.2 Draw a picture of the vector field (y, x) for the velocity of a fluid flowing
in two dimensions.

Here is another such example.

Example 12.2.3 Draw a picture of the vector field (y cos (x) + 1, x sin (y) − 1) for the ve-
locity of a fluid flowing in two dimensions.
220 CHAPTER 12. VECTOR VALUED FUNCTIONS

These pictures were drawn by maple. Note how they reveal both the direction and the
magnitude of the vectors. However, if you try to draw these by hand, you will mainly waste
time.

12.3 Exercises
1. Here are some vector valued functions.

f (x, y) = (x, y) , g (x, y) = (− (y − 1) , x) , h (x, y) = (x, −y) .

Now here are the graphs of some vector fields. Match the function with the vector
field.

 
xy
p
2. Find D (f ) for f (x, y, z, w) = zw , 6 − x 2 y2 .
 p 
1
3. Find D (f ) for f (x, y, z) = 1+x2 −y2
, 4 − (x2 + y2 + z2 ) .

1
4. For f (x, y, z) = (x, y, xy) , h (x, y, z) = y2 , −x, z and g (x, y, z) = 2



x , yz, x − 1, com-
pute the following.

(a) f × g
(b) g × f
(c) f · g
(d) f × g · h
(e) f × (g × h)
(f) (f × g) · (g × h)

5. Let f (x, y, z) = (y, z, x) and g (x, y, z) = x2 + y, z, x . Find g ◦ f (x, y, z).

6. Let f (x, y, z) = (x, z, yz) and g (x, y, z) = x, y, x2 − 1 . Find g ◦ f (x, y, z).

7. For f, g, h vector valued functions and k, l scalar valued functions, which of the
following make sense?

(a) f × g × h
(b) (k × g) × h
(c) (f · g) × h
(d) (f × g) · h
12.4. CONTINUOUS FUNCTIONS 221

(e) l g· k
(f) f × (g + h)
8. The Lotka Volterra system of differential equations, proposed in 1925 and 1926 by
Lotka and Volterra respectively, is intended to model the interaction of predators and
prey. An example of this situation is that of wolves and moose living on Isle Royal
in the middle of Lake Superior. In these equations x is the number of prey and y is
the number of predators. The equations are
x′ (t) = x (t) (a − by (t)) , y′ (t) = −y (t) (c − dx (t))
Written in terms of vectors,
x′ , y′ = (x (a − by) , −y (c − dx))

The parameters a, b, c, d depend on the problem. The differential equations are

saying that at a point (x, y), the population vector (x, y) moves in the direction of
(x (a − by) , −y (c − dx)). Here is the graph of the vector field which determines the
Lotka Volterra system in the case where all the parameters equal 1 which is graphed
near the point (1,1). What conclusions seem to be true based on the graph of this
vector field? What happens if you start with a population vector near the point (1,1)?
Remember these vectors in the plane determine the directions of motion of the pop-
ulation vector.

1.5

1.0

0.5

0.5 1.0 1.5

How did I know to graph the vector field near (1,1)?

12.4 Continuous Functions

What was done in one variable calculus for scalar functions is generalized here to include
the case of a vector valued function of possibly many variables.
Definition 12.4.1 A function f : D (f ) ⊆ R p → Rq is continuous at x ∈ D (f ) if for each
ε > 0 there exists δ > 0 such that whenever y ∈ D (f ) and
|y − x| < δ
it follows that
|f (x) − f (y)| < ε.
f is continuous if it is continuous at every point of D (f ).
Note the total similarity to the scalar valued case.
222 CHAPTER 12. VECTOR VALUED FUNCTIONS

12.4.1 Sufficient Conditions For Continuity

The next theorem is a fundamental result which allows less worry about the ε δ definition
of continuity.

Theorem 12.4.2 The following assertions are valid.

1. The function af +bg is continuous at x whenever f , g are continuous at x ∈ D (f )∩

D (g) and a, b ∈ R.
2. If f is continuous at x, f (x) ∈ D (g) ⊆ R p , and g is continuous at f (x) ,then g ◦ f
is continuous at x.
3. If f = ( f1 , · · · , fq ) : D (f ) → Rq , then f is continuous if and only if each fk is a
continuous real valued function.
4. The function f : R p → R, given by f (x) = |x| is continuous.

The proof of this theorem is in the last section of this chapter. Its conclusions are not
surprising. For example the first claim says that (af + bg) (y) is close to (af + bg) (x)
when y is close to x provided the same can be said about f and g. For the second claim,
if y is close to x, f (x) is close to f (y) and so by continuity of g at f (x), g (f (y)) is
close to g (f (x)). To see the third claim is likely, note that closeness in R p is the same as
closeness in each coordinate. The fourth claim is immediate from the triangle inequality.
For functions defined on Rn , there is a notion of polynomial just as there is for functions
defined on R.

Definition 12.4.3 Let α be an n dimensional multi-index. This means

α = (α 1 , · · · , α n )

where each α i is a natural number or zero. Also, let

n
|α| ≡ ∑ |α i |
i=1

The symbol xα means

xα ≡ x1α 1 x2α 2 · · · x3α n .
An n dimensional polynomial of degree m is a function of the form

p (x) = ∑ dα xα.
|α|≤m

where the dα are real numbers.

The above theorem implies that polynomials are all continuous.

12.5. LIMITS OF A FUNCTION 223

12.5 Limits Of A Function

As in the case of scalar valued functions of one variable, a concept closely related to con-
tinuity is that of the limit of a function. The notion of limit of a function makes sense at
points x, which are limit points of D (f ) and this concept is defined next.

Definition 12.5.1 Let A ⊆ Rm be a set. A point x, is a limit point of A if B (x, r) contains

infinitely many points of A for every r > 0.

Definition 12.5.2 Let f : D (f ) ⊆ R p → Rq be a function and let x be a limit point of

D (f ). Then
lim f (y) = L
y →x

if and only if the following condition holds. For all ε > 0 there exists δ > 0 such that if

0 < |y − x| < δ , and y ∈ D (f )

then,
|L − f (y)| < ε.

Theorem 12.5.3 If limy→x f (y) = L and limy→x f (y) = L1 , then L = L1 .

Proof: Let ε > 0 be given. There exists δ > 0 such that if 0 < |y − x| < δ and y ∈
D (f ), then
|f (y) − L| < ε, |f (y) − L1 | < ε.
Pick such a y. There exists one because x is a limit point of D (f ). Then

|L − L1 | ≤ |L − f (y)| + |f (y) − L1 | < ε + ε = 2ε.

Since ε > 0 was arbitrary, this shows L = L1 . ■

As in the case of functions of one variable, one can define what it means for limy→x f (x) =
±∞.

Definition 12.5.4 If f (x) ∈ R, limy→x f (x) = ∞ if for every number l, there exists δ > 0
such that whenever |y − x| < δ and y ∈ D (f ), then f (x) > l. limy→x f (x) = −∞ if for
every number l, there exists δ > 0 such that whenever |y − x| < δ and y ∈ D (f ), then
f (x) < l.

The following theorem is just like the one variable version of calculus.

Theorem 12.5.5 Suppose f : D (f ) → Rq . Then for x a limit point of D (f ),

lim f (y) = L (12.1)

y →x

if and only if
lim fk (y) = Lk (12.2)
y →x

where f (y) ≡ ( f1 (y) , · · · , f p (y)) and L ≡ (L1 , · · · , L p ). Suppose

lim f (y) = L, lim g (y) = K

y →x y →x
224 CHAPTER 12. VECTOR VALUED FUNCTIONS

where K, L ∈ Rq . Then if a, b ∈ R,

lim (af (y) + bg (y)) = aL + bK, (12.3)

y →x

lim f · g (y) = L · K (12.4)

y→x

In the case where q = 3 and limy→x f (y) = L and limy→x g (y) = K, then

lim f (y) × g (y) = L × K. (12.5)

y →x

If g is scalar valued with limy→x g (y) = K ̸= 0,

lim f (y) g (y) = LK. (12.6)

y →x

Also, if h is a continuous function defined near L, then

lim h ◦ f (y) = h (L) . (12.7)

y →x

Suppose limy→x f (y) = L. If |f (y) − b| ≤ r for all y sufficiently close to x, then |L − b| ≤

r also.

Proof: Suppose (12.1). Then letting ε > 0 be given there exists δ > 0 such that if
0 < |y − x| < δ , it follows

| fk (y) − Lk | ≤ |f (y) − L| < ε

which verifies (12.2).

Now suppose (12.2) holds. Then letting ε > 0 be given, there exists δ k such that if
0 < |y − x| < δ k , then
ε
| fk (y) − Lk | < √ .
p
Let 0 < δ < min (δ 1 , · · · , δ p ). Then if 0 < |y − x| < δ , it follows
!1/2
p
2
|f (y) − L| = ∑ | fk (y) − Lk |
k=1
!1/2
p
ε2
< ∑ p = ε.
k=1

Each of the remaining assertions follows immediately from the coordinate descriptions
of the various expressions and the first part. However, I will give a different argument for
these.
The proof of (12.3) is left for you. It is like a corresponding theorem for continuous
functions. Now (12.4) is to be verified. Let ε > 0 be given. Then by the triangle inequality,

|f · g (y) − L · K| ≤ |f · g (y) − f (y) · K| + |f (y) · K − L · K|

≤ |f (y)| |g (y) − K| + |K| |f (y) − L| .
12.5. LIMITS OF A FUNCTION 225

There exists δ 1 such that if 0 < |y − x| < δ 1 and y ∈ D (f ), then

|f (y) − L| < 1,

and so for such y, the triangle inequality implies, |f (y)| < 1 + |L|. Therefore, for 0 <
|y − x| < δ 1 ,

|f · g (y) − L · K| ≤ (1 + |K| + |L|) [|g (y) − K| + |f (y) − L|] . (12.8)

Now let 0 < δ 2 be such that if y ∈ D (f ) and 0 < |x − y| < δ 2 ,

ε ε
|f (y) − L| < , |g (y) − K| < .
2 (1 + |K| + |L|) 2 (1 + |K| + |L|)

Then letting 0 < δ ≤ min (δ 1 , δ 2 ), it follows from (12.8) that

|f · g (y) − L · K| < ε

and this proves (12.4).

Consider (12.5). Let δ 1 be as above. From the properties of the cross product,

|f (y) × g (y) − L × K| ≤ |f (y) × g (y) − f (y) × K| + |f (y) × K − L × K|

= |f (y) × (g (y) − K)| + |(f (y) − L) × K|

Now from the geometric description of the cross product,

≤ |f (y)| |g (y) − K| + |f (y) − L| |K|

Then if 0 < |y − x| < δ 1 , this is no larger than

(1 + |L|) |g (y) − K| + |f (y) − L| |K| ≤ (1 + |K| + |L|) [|g (y) − K| + |f (y) − L|]

and now the conclusion follows as before in the case of the dot product.
The proof of (12.6) is left to you.
Consider (12.7). Since h is continuous near L, it follows that for ε > 0 given, there
exists η > 0 such that if |y − L| < η, then

|h (y) −h (L)| < ε

Now since limy→x f (y) = L, there exists δ > 0 such that if 0 < |y − x| < δ , then

|f (y) −L| < η.

Therefore, if 0 < |y − x| < δ ,

|h (f (y)) −h (L)| < ε.

It only remains to verify the last assertion. Assume |f (y) − b| ≤ r. It is required to

show that |L − b| ≤ r. If this is not true, then |L − b| > r. Consider B (L, |L − b| − r).
Since L is the limit of f, it follows f (y) ∈ B (L, |L − b| − r) whenever y ∈ D (f ) is close
enough to x. Thus, by the triangle inequality,

|f (y) − L| < |L − b| − r
226 CHAPTER 12. VECTOR VALUED FUNCTIONS

and so

r < |L − b| − |f (y) − L| ≤ ||b − L| − |f (y) − L||

≤ |b − f (y)| ,

a contradiction to the assumption that |b − f (y)| ≤ r. ■

The relation between continuity and limits is as follows.

Theorem 12.5.6 For f : D (f ) → Rq and x ∈ D (f ) a limit point of D (f ), f is continuous

at x if and only if
lim f (y) = f (x) .
y →x

Proof: First suppose f is continuous at x a limit point of D (f ). Then for every ε > 0
there exists δ > 0 such that if |y − x| < δ and y ∈ D (f ), then |f (x) − f (y)| < ε. In
particular, this holds if 0 < |x − y| < δ and this is just the definition of the limit. Hence
f (x) = limy→x f (y).
Next suppose x is a limit point of D (f ) and limy→x f (y) = f (x). This means
that if ε > 0 there exists δ > 0 such that for 0 < |x − y| < δ and y ∈ D (f ), it follows
|f (y) − f (x)| < ε. However, if y = x, then |f (y) − f (x)| = |f (x) − f (x)| = 0 and so
whenever y ∈ D (f ) and |x − y| < δ , it follows |f (x) − f (y)| < ε, showing f is contin-
uous at x. ■
 2 
−9
Example 12.5.7 Find lim(x,y)→(3,1) xx−3 ,y .

2 −9
It is clear that lim(x,y)→(3,1) xx−3 = 6 and lim(x,y)→(3,1) y = 1. Therefore, this limit equals
(6, 1).

Example 12.5.8 Find lim(x,y)→(0,0) x2xy

+y2
.

First of all, observe the domain of the function is R2 \ {(0, 0)}, every point in R2 except
the origin. Therefore, (0, 0) is a limit point of the domain of the function so it might make
sense to take a limit. However, just as in the case of a function of one variable, the limit may
not exist. In fact, this is the case here. To see this, take points on the line y = 0. At these
points, the value of the function equals 0. Now consider points on the line y = x where the
value of the function equals 1/2. Since, arbitrarily close to (0, 0), there are points where
the function equals 1/2 and points where the function has the value 0, it follows there can
be no limit. Just take ε = 1/10 for example. You cannot be within 1/10 of 1/2 and also
within 1/10 of 0 at the same time.
Note it is necessary to rely on the definition of the limit much more than in the case of
a function of one variable and there are no easy ways to do limit problems for functions of
more than one variable. It is what it is and you will not deal with these concepts without
suffering and anguish.

12.6 Properties Of Continuous Functions

Functions of p variables have many of the same properties as functions of one variable.
First there is a version of the extreme value theorem generalizing the one dimensional case.
12.7. EXERCISES 227

Theorem 12.6.1 Let C be closed and bounded and let f : C → R be continuous. Then f
achieves its maximum and its minimum on C. This means there exist, x1 , x2 ∈ C such that
for all x ∈ C,
f (x1 ) ≤ f (x) ≤ f (x2 ) .

There is also the long technical theorem about sums and products of continuous func-
tions. These theorems are proved later in this chapter.

Theorem 12.6.2 The following assertions are valid.

1. The function af + bg is continuous at x when f , g are continuous at x ∈ D (f ) ∩

D (g) and a, b ∈ R.
2. If and f and g are each real valued functions continuous at x, then f g is continuous
at x. If, in addition to this, g (x) ̸= 0, then f /g is continuous at x.
3. If f is continuous at x, f (x) ∈ D (g) ⊆ R p , and g is continuous at f (x) , then g ◦ f
is continuous at x.
4. If f = ( f1 , · · · , fq ) : D (f ) → Rq , then f is continuous if and only if each fk is a
continuous real valued function.

5. The function f : R p → R, given by f (x) = |x| is continuous.

12.7 Exercises
 
t t
1. Let f (t) = t,t 2 + 1, t+1


and let g (t) = t + 1, 1, t 2 +1 . Find f · g.

2. Let f, g be given in the previous problem. Find f × g.

3. Let f (t) = t,t 2 ,t 3 , g (t) = 1,t 2 ,t 2 , and h (t) = (sint,t, 1). Find the time rate of

change of the box product of the vectors f, g, and h.

4. Let f (t) = (t, sint). Show f is continuous at every point t.

5. Suppose |f (x) − f (y)| ≤ K |x − y| where K is a constant. Show that f is every-

where continuous. Functions satisfying such an inequality are called Lipschitz func-
tions.
6. Suppose |f (x) − f (y)| ≤ K |x − y|α where K is a constant and α ∈ (0, 1). Show
that f is everywhere continuous. Functions like this are called Holder continuous.

7. Suppose f : R3 → R is given by f (x) = 3x1 x2 + 2x32 . Use Theorem 12.4.2 to verify

that f is continuous. Hint: You should first verify that the function π k : R3 → R
given by π k (x) = xk is a continuous function.
8. Show that if f : Rq → R is a polynomial then it is continuous.

9. State and prove a theorem which involves quotients of functions encountered in the
previous problem.
228 CHAPTER 12. VECTOR VALUED FUNCTIONS

10. Let
2x2 −y2
(
x2 +y2
if (x, y) ̸= (0, 0)
f (x, y) ≡ .
0 if (x, y) = (0, 0)
Find lim(x,y)→(0,0) f (x, y) if it exists. If it does not exist, tell why it does not exist.
Hint: Consider along the line y = x and along the line y = 0.
11. Find the following limits if possible
2−y 2
(a) lim(x,y)→(0,0) xx2 +y 2.

x(x2 −y2 )
(b) lim(x,y)→(0,0) = 0.
(x2 +y2 )
2
(x2 −y4 ) 2
(c) lim(x,y)→(0,0) 2 . Hint: Consider along y = 0 and along x = y .
(x2 +y4 )
 
1
(d) lim(x,y)→(0,0) x sin x2 +y 2 .

2 3 2 2 2 3
(e) lim(x,y)→(1,2) −2yx +8yx+34y+3y −18y +6x −13x−20−xy −x
−y2 +4y−5−x2 +2x
. Hint: It might help to
write this in terms of the variables (s,t) = (x − 1, y − 2) .

12. Suppose limx→0 f (x, 0) = 0 = limy→0 f (0, y). Does it follow that

lim f (x, y) = 0?
(x,y)→(0,0)

Prove or give counter example.

13. f : D ⊆ R p → Rq is Lipschitz continuous or just Lipschitz for short if there exists a
constant K such that
|f (x) − f (y)| ≤ K |x − y|
for all x, y ∈ D. Show every Lipschitz function is uniformly continuous which means
that given ε > 0 there exists δ > 0 independent of x such that if |x − y| < δ , then
|f (x) − f (y)| < ε.

14. If f is uniformly continuous, does it follow that |f | is also uniformly continuous?

If |f | is uniformly continuous does it follow that f is uniformly continuous? An-
swer the same questions with “uniformly continuous” replaced with “continuous”.
Explain why.
15. Let f be defined on the positive integers. Thus D ( f ) = N. Show that f is auto-
matically continuous at every point of D ( f ). Is it also uniformly continuous? What
does this mean about the concept of continuous functions being those which can be
graphed without taking the pencil off the paper?

16. Let
2
x2 − y4
f (x, y) = if (x, y) ̸= (0, 0)
(x2 + y4 )2
Show limt→0 f (tx,ty) = 1 for any choice of (x, y). Using Problem 11c, what does
this tell you about limits existing just because the limit along any line exists.
12.8. OPEN AND CLOSED SETS 229

17. Let f (x, y, z) = x2 y + sin (xyz). Does f achieve a maximum on the set

(x, y, z) : x2 + y2 + 2z2 ≤ 8 ?


Explain why.

18. Suppose x is defined to be a limit point of a set A if and only if for all r > 0, B (x, r)
contains a point of A different than x. Show this is equivalent to the above definition
of limit point.
19. Give an example of an infinite set of points in R3 which has no limit points. Show
that if D (f ) equals this set, then f is continuous. Show that more generally, if f is
any function for which D (f ) has no limit points, then f is continuous.
20. Let {xk }nk=1 be any finite set of points in R p . Show this set has no limit points.
21. Suppose S is any set of points such that every pair of points is at least as far apart as
1. Show S has no limit points.
x|)
22. Find limx→0 sin(|
|x| and prove your answer from the definition of limit.

23. Suppose g is a continuous vector valued function of one variable defined on [0, ∞).
Prove
lim g (|x|) = g (|x0 |) .
x→x0

12.8 Open And Closed Sets

Eventually, one must consider functions which are defined on subsets of Rn and their prop-
erties. The next definition will end up being quite important. It describe a type of subset of
Rn with the property that if x is in this set, then so is y whenever y is close enough to x.

Definition 12.8.1 Recall that for x, y ∈ Rn ,

!1/2
n
2
|x − y| = ∑ |xi − yi | .
i=1

Also let
B (x, r) ≡ {y ∈ Rn : |x − y| < r}
Let U ⊆ Rn . U is an open set if whenever x ∈ U, there exists r > 0 such that B (x, r) ⊆ U.
More generally, if U is any subset of Rn , x ∈ U is an interior point of U if there exists r > 0
such that x ∈ B (x, r) ⊆ U. In other words U is an open set exactly when every point of U
is an interior point of U.

If there is something called an open set, surely there should be something called a
closed set and here is the definition of one.

Definition 12.8.2 A subset, C, of Rn is called a closed set if Rn \ C is an open set. They

symbol Rn \C denotes everything in Rn which is not in C. It is also called the complement
of C. The symbol SC is a short way of writing Rn \ S.
230 CHAPTER 12. VECTOR VALUED FUNCTIONS

To illustrate this definition, consider the following picture.

x U
B(x, r)

You see in this picture how the edges are dotted. This is because an open set, can not in-
clude the edges or the set would fail to be open. For example, consider what would happen
if you picked a point out on the edge of U in the above picture. Every open ball centered
at that point would have in it some points which are outside U. Therefore, such a point
would violate the above definition. You also see the edges of B (x, r) dotted suggesting that
B (x, r) ought to be an open set. This is intuitively clear but does require a proof. This will
be done in the next theorem and will give examples of open sets. Also, you can see that if
x is close to the edge of U, you might have to take r to be very small.
It is roughly the case that open sets do not have their skins while closed sets do. Here
is a picture of a closed set, C.

C x
B(x, r)

Note that x ∈ / C and since Rn \C is open, there exists a ball, B (x, r) contained entirely
in R \C. If you look at Rn \C, what would be its skin? It can’t be in Rn \C and so it must
n

be in C. This is a rough heuristic explanation of what is going on with these definitions.

Also note that Rn and 0/ are both open and closed. Here is why. If x ∈ 0, / then there must
be a ball centered at x which is also contained in 0. / This must be considered to be true
because there is nothing in 0/ so there can be no example to show it false1 . Therefore, from
the definition, it follows 0/ is open. It is also closed because if x ∈
/ 0,
/ then B (x, 1) is also
contained in Rn \ 0/ = Rn . Therefore, 0/ is both open and closed. From this, it follows Rn is
also both open and closed.
1 To a mathematician, the statement: Whenever a pig is born with wings it can fly must be taken as true. We

do not consider biological or aerodynamic considerations in such statements. There is no such thing as a winged
pig and therefore, all winged pigs must be superb flyers since there can be no example of one which is not. On
the other hand we would also consider the statement: Whenever a pig is born with wings it cannot possibly fly,
as equally true. The point is, you can say anything you want about the elements of the empty set and no one can
gainsay your statement. Therefore, such statements are considered as true by default. You may say this is a very
strange way of thinking about truth and ultimately this is because mathematics is not about truth. It is more about
consistency and logic.
12.8. OPEN AND CLOSED SETS 231

Theorem 12.8.3 Let x ∈ Rn and let r ≥ 0. Then B (x, r) is an open set. Also,

D (x, r) ≡ {y ∈ Rn : |y − x| ≤ r}

is a closed set.

Proof: Suppose y ∈ B (x,r). It is necessary to show there exists r1 > 0 such that
B (y, r1 ) ⊆ B (x, r). Define r1 ≡ r − |x − y|. Then if |z − y| < r1 , it follows from the above
triangle inequality that

|z − x| = |z − y + y − x|
≤ |z − y| + |y − x|
< r1 + |y − x| = r − |x − y| + |y − x| = r.

Note that if r = 0 then B (x, r) = 0, / the empty set. This is because if y ∈ Rn , |x − y| ≥ 0

and so y ∈ / B (x, 0). Since 0/ has no points in it, it must be open because every point in it,
(There are none.) satisfies the desired property of being an interior point.
Now suppose y ∈ / D (x, r). Then |x − y| > r and defining δ ≡ |x − y| − r, it follows
that if z ∈ B (y, δ ), then by the triangle inequality,

|x − z| ≥ |x − y| − |y − z| > |x − y| − δ
= |x − y| − (|x − y| − r) = r

and this shows that B (y, δ ) ⊆ Rn \ D (x, r). Since y was an arbitrary point in Rn \ D (x, r),
it follows Rn \ D (x, r) is an open set which shows, from the definition, that D (x, r) is a
closed set as claimed. ■
A picture which is descriptive of the conclusion of the above theorem which also im-
plies the manner of proof is the following.

r r
r1 r1
x y x y

B(x, r) D(x, r)

Recall R2 consists of ordered pairs (x, y) such that x ∈ R and y ∈ R. R2 is also written
as R × R. In general, the following definition holds.

Definition 12.8.4 The Cartesian product of two sets A × B, means

{(a, b) : a ∈ A, b ∈ B} .
232 CHAPTER 12. VECTOR VALUED FUNCTIONS

If you have n sets A1 , A2 , · · · , An

n
∏ Ai = {(x1 , x2 , · · · , xn ) : each xi ∈ Ai } .
i=1

Now suppose A ⊆ Rm and B ⊆ Rn . Then if

(x, y) ∈ A × B, x = (x1 , · · · , xm ) and y = (y1 , · · · , yn ) ,

the following identification will be made.

(x, y) = (x1 , · · · , xm , y1 , · · · , yn ) ∈ Rn+m .

Similarly, starting with something in Rn+m , you can write it in the form (x, y) where
x ∈ Rm and y ∈ Rn . The following theorem has to do with the Cartesian product of two
closed sets or two open sets. Also here is an important definition.

Definition 12.8.5 A set, A ⊆ Rn is said to be bounded if there exist finite intervals, [ai , bi ]
such that
n
A ⊆ ∏ [ai , bi ] .
i=1

Theorem 12.8.6 Let U be an open set in Rm and let V be an open set in Rn . Then U ×V
is an open set in Rn+m . If C is a closed set in Rm and H is a closed set in Rn , then C × H
is a closed set in Rn+m . If C and H are bounded, then so is C × H.

Proof: Let (x, y) ∈ U ×V . Since U is open, there exists r1 > 0 such that B (x, r1 ) ⊆ U.
Similarly, there exists r2 > 0 such that B (y, r2 ) ⊆ V . Now

B ((x, y) , δ ) ≡
( )
m n
n+m 2 2 2
(s, t) ∈ R : ∑ |xk − sk | + ∑ yj −tj <δ
k=1 j=1

Therefore, if δ ≡ min (r1 , r2 ) and (s, t) ∈ B ((x, y) , δ ), then it follows that s ∈ B (x, r1 ) ⊆
U and that t ∈ B (y, r2 ) ⊆ V which shows that B ((x, y) , δ ) ⊆ U ×V . Hence U ×V is open
as claimed.
Next suppose (x, y) ∈ / C × H. It is necessary to show there exists δ > 0 such that
B ((x, y) , δ ) ⊆ Rn+m \ (C × H). Either x ∈ / C or y ∈ / H since otherwise (x, y) would be a
point of C ×H. Suppose therefore, that x ∈ / C. Since C is closed, there exists r > 0 such that
B (x, r) ⊆ Rm \C. Consider B ((x, y) , r). If (s, t) ∈ B ((x, y) , r) , it follows that s ∈ B (x, r)
which is contained in Rm \C. Therefore, B ((x, y) , r) ⊆ Rn+m \ (C × H) showing C × H is
closed. A similar argument holds if y ∈ / H.
If C is bounded, there exist [ai , bi ] such that C ⊆ ∏m i=1 [ai , bi ] and if H is bounded, H ⊆
m+n
∏i=m+1 [ai , bi ] for intervals [am+1 , bm+1 ] , · · · , [am+n , bm+n ]. Therefore, C ×H ⊆ ∏m+n i=1 [ai , bi ].
■
12.9. EXERCISES 233

12.9 Exercises
1. Let U = {(x, y, z) such that z > 0}. Determine whether U is open, closed or neither.

2. Let U = {(x, y, z) such that z ≥ 0} . Determine whether U is open, closed or neither.

n p o
3. Let U = (x, y, z) such that x2 + y2 + z2 < 1 . Determine whether U is open, closed
or neither.
n p o
4. Let U = (x, y, z) such that x2 + y2 + z2 ≤ 1 . Determine whether U is open, closed
or neither.

5. Show carefully that Rn is both open and closed.

6. Show that every open set in Rn is the union of open balls contained in it.
7. Show the intersection of any two open sets is an open set.

8. If S is a nonempty subset of R p , a point x is said to be a limit point of S if B (x, r)

contains infinitely many points of S for each r > 0. Show this is equivalent to saying
that B (x, r) contains a point of S different than x for each r > 0.
9. Closed sets were defined to be those sets which are complements of open sets. Show
that a set is closed if and only if it contains all its limit points.
234 CHAPTER 12. VECTOR VALUED FUNCTIONS
Chapter 13

Some Fundamentals∗

This section contains the proofs of the theorems which were stated without proof along
with some other significant topics which will be useful later. These topics are of funda-
mental significance but are difficult.

13.1 Combinations Of Continuous Functions

Theorem 13.1.1 The following assertions are valid.

1. The function af + bg is continuous at x when f , g are continuous at x ∈ D (f ) ∩

D (g) and a, b ∈ R.
2. If and f and g are each real valued functions continuous at x, then f g is continuous
at x. If, in addition to this, g (x) ̸= 0, then f /g is continuous at x.
3. If f is continuous at x, f (x) ∈ D (g) ⊆ R p , and g is continuous at f (x) , then g ◦ f
is continuous at x.
4. If f = ( f1 , · · · , fq ) : D (f ) → Rq , then f is continuous if and only if each fk is a
continuous real valued function.
5. The function f : R p → R, given by f (x) = |x| is continuous.

Proof: Begin with (1). Let ε > 0 be given. By assumption, there exist δ 1 > 0 such
ε
that whenever |x − y| < δ 1 , it follows |f (x) − f (y)| < 2(|a|+|b|+1) and there exists δ 2 > 0
ε
such that whenever |x − y| < δ 2 , it follows that |g (x) − g (y)| < 2(|a|+|b|+1) . Then let
0 < δ ≤ min (δ 1 , δ 2 ). If |x − y| < δ , then everything happens at once. Therefore, using
the triangle inequality

|af (x) + bf (x) − (ag (y) + bg (y))|

235
236 CHAPTER 13. SOME FUNDAMENTALS∗

≤ |a| |f (x) − f (y)| + |b| |g (x) − g (y)|

   
ε ε
< |a| + |b| < ε.
2 (|a| + |b| + 1) 2 (|a| + |b| + 1)
Now begin on (2). There exists δ 1 > 0 such that if |y − x| < δ 1 , then
| f (x) − f (y)| < 1
Therefore, for such y,
| f (y)| < 1 + | f (x)| .
It follows that for such y,
| f g (x) − f g (y)| ≤ | f (x) g (x) − g (x) f (y)| + |g (x) f (y) − f (y) g (y)|

≤ |g (x)| | f (x) − f (y)| + | f (y)| |g (x) − g (y)|

≤ (1 + |g (x)| + | f (y)|) [|g (x) − g (y)| + | f (x) − f (y)|] .
Now let ε > 0 be given. There exists δ 2 such that if |x − y| < δ 2 , then
ε
|g (x) − g (y)| < ,
2 (1 + |g (x)| + | f (y)|)
and there exists δ 3 such that if |x − y| < δ 3 , then
ε
| f (x) − f (y)| <
2 (1 + |g (x)| + | f (y)|)
Now let 0 < δ ≤ min (δ 1 , δ 2 , δ 3 ). Then if |x − y| < δ , all the above hold at once and
| f g (x) − f g (y)| ≤

(1 + |g (x)| + | f (y)|) [|g (x) − g (y)| + | f (x) − f (y)|]

 
ε ε
< (1 + |g (x)| + | f (y)|) + = ε.
2 (1 + |g (x)| + | f (y)|) 2 (1 + |g (x)| + | f (y)|)
This proves the first part of (2). To obtain the second part, let δ 1 be as described above and
let δ 0 > 0 be such that for |x − y| < δ 0 ,
|g (x) − g (y)| < |g (x)| /2
and so by the triangle inequality,
− |g (x)| /2 ≤ |g (y)| − |g (x)| ≤ |g (x)| /2
which implies |g (y)| ≥ |g (x)| /2, and |g (y)| < 3 |g (x)| /2.
Then if |x − y| < min (δ 0 , δ 1 ) ,
f (x) f (y) f (x) g (y) − f (y) g (x)
− =
g (x) g (y) g (x) g (y)
| f (x) g (y) − f (y) g (x)|
≤  2
|g(x)|
2
2 | f (x) g (y) − f (y) g (x)|
=
|g (x)|2
13.1. COMBINATIONS OF CONTINUOUS FUNCTIONS 237

2
≤ [| f (x) g (y) − f (y) g (y) + f (y) g (y) − f (y) g (x)|]
|g (x)|2
2
≤ [|g (y)| | f (x) − f (y)| + | f (y)| |g (y) − g (x)|]
|g (x)|2
 
2 3
≤ |g (x)| | f (x) − f (y)| + (1 + | f (x)|) |g (y) − g (x)|
|g (x)|2 2
2
≤ (1 + 2 | f (x)| + 2 |g (x)|) [| f (x) − f (y)| + |g (y) − g (x)|]
|g (x)|2
≡ M [| f (x) − f (y)| + |g (y) − g (x)|]

where
2
M≡ (1 + 2 | f (x)| + 2 |g (x)|)
|g (x)|2
Now let δ 2 be such that if |x − y| < δ 2 , then
ε
| f (x) − f (y)| < M −1
2
and let δ 3 be such that if |x − y| < δ 3 , then
ε
|g (y) − g (x)| < M −1 .
2
Then if 0 < δ ≤ min (δ 0 , δ 1 , δ 2 , δ 3 ), and |x − y| < δ , everything holds and

f (x) f (y)
− ≤ M [| f (x) − f (y)| + |g (y) − g (x)|]
g (x) g (y)
hε ε i
< M M −1 + M −1 = ε.
2 2
This completes the proof of the second part of (2). Note that in these proofs no effort is
made to find some sort of “best” δ . The problem is one which has a yes or a no answer.
Either it is or it is not continuous.
Now begin on (3). If f is continuous at x, f (x) ∈ D (g) ⊆ R p , and g is continuous at
f (x) , then g ◦ f is continuous at x. Let ε > 0 be given. Then there exists η > 0 such that
if |y − f (x)| < η and y ∈ D (g), it follows that |g (y) − g (f (x))| < ε. It follows from
continuity of f at x that there exists δ > 0 such that if |x − z| < δ and z ∈ D (f ), then
|f (z) − f (x)| < η. Then if |x − z| < δ and z ∈ D (g ◦ f ) ⊆ D (f ), all the above hold and
so
|g (f (z)) − g (f (x))| < ε.
This proves part (3).
Part (4) says: If f = ( f1 , · · · , fq ) : D (f ) → Rq , then f is continuous if and only if each
fk is a continuous real valued function. Then
!1/2
q
2
| fk (x) − fk (y)| ≤ |f (x) − f (y)| ≡ ∑ | fi (x) − fi (y)|
i=1
q
≤ ∑ | fi (x) − fi (y)| . (13.1)
i=1
238 CHAPTER 13. SOME FUNDAMENTALS∗

Suppose first that f is continuous at x. Then there exists δ > 0 such that if |x − y| < δ ,
then |f (x) − f (y)| < ε. The first part of the above inequality then shows that for each
k = 1, · · · , q, | fk (x) − fk (y)| < ε. This shows the only if part. Now suppose each function
fk is continuous. Then if ε > 0 is given, there exists δ k > 0 such that whenever |x − y| < δ k

| fk (x) − fk (y)| < ε/q.

Now let 0 < δ ≤ min (δ 1 , · · · , δ q ). For |x − y| < δ , the above inequality holds for all k and
so the last part of (13.1) implies
q q
ε
|f (x) − f (y)| ≤ ∑ | fi (x) − fi (y)| < ∑ = ε.
i=1 i=1 q

This proves part (4).

To verify part (5), let ε > 0 be given and let δ = ε. Then if |x − y| < δ , the triangle
inequality implies

| f (x) − f (y)| = ||x| − |y|| ≤ |x − y| < δ = ε.

This proves part (5) and completes the proof of the theorem. ■

13.2 The Nested Interval Lemma

First, here is the one dimensional nested interval lemma.

Lemma 13.2.1 Let Ik = [ak , bk ] be closed intervals, ak ≤ bk , such that Ik ⊇ Ik+1 for all k.
Then there exists a point c which is contained in all these intervals. If limk→∞ (bk − ak ) = 0,
then there is exactly one such point.

Proof: Note that the {ak } are an increasing sequence and that {bk } is a decreasing
sequence. Now note that if m < n, then

am ≤ an ≤ bn

while if m > n,
bn ≥ bm ≥ am .
It follows that am ≤ bn for any pair m, n. Therefore, each bn is an upper bound for all the
am and so if c ≡ sup {ak }, then for each n, it follows that c ≤ bn and so for all, an ≤ c ≤ bn
which shows that c is in all of these intervals.
If the condition on the lengths of the intervals holds, then if c, c′ are in all the intervals,
then if they are not equal, then eventually, for large enough k, they cannot both be contained
in [ak , bk ] since eventually bk − ak < |c − c′ |. This would be a contradiction. Hence c = c′ .
■

Definition 13.2.2 The diameter of a set S, is defined as

diam (S) ≡ sup {|x − y| : x, y ∈ S} .

Thus diam (S) is just a careful description of what you would think of as the diameter.
It measures how stretched out the set is.
Here is a multidimensional version of the nested interval lemma.
13.3. CONVERGENT SEQUENCES, SEQUENTIAL COMPACTNESS 239

p  k k
ai , bi ≡ x ∈ R p : xi ∈ aki , bki and suppose that for all
  
Lemma 13.2.3 Let Ik = ∏i=1
k = 1, 2, · · · ,
Ik ⊇ Ik+1 .
Then there exists a point c ∈ R p which is an element of every Ik . If limk→∞ diam (Ik ) = 0,
then the point c is unique.

Proof: For each i = 1, · · · , p, aki , bki ⊇ ak+1 , bk+1

   
i i and so, by Lemma 13.2.1, there
 k k
exists a point ci ∈ ai , bi for all k. Then letting c ≡ (c1 , · · · , c p ) it follows c ∈ Ik for all k.
If the condition on the diameters holds, then the lengths of the intervals limk→∞ aki , bki = 0
 

and so by the same lemma, each ci is unique. Hence c is unique. ■

I will sometimes refer to the above Cartesian product of closed intervals as an interval
to emphasize the analogy with one dimensions, and sometimes as a box.

13.3 Convergent Sequences, Sequential Compactness

A mapping
 f : {k, k + 1, k + 2, · · · } → R p is called a sequence. We usually write it in the
form a j where it is understood that a j ≡ f ( j). In the same way as for sequences of real
numbers, one can define what it means for convergence to take place.

Definition 13.3.1 A sequence, {ak } is said to converge to a if for every ε > 0 there exists
nε such that if n > nε , then |a − an | < ε. The usual notation for this is limn→∞ an = a
although it is often written as an → a.

One can also define a subsequence in the same way as in the case of real valued se-
quences.

Definition 13.3.2 ank is a subsequence of {an } if n1 < n2 < · · · .

The following theorem says the limit, if it exists, is unique.

Theorem 13.3.3 If a sequence, {an } converges to a and to b then a = b.

ε
Proof: There exists nε such that if n > nε then |an − a| < 2 and if n > nε , then
|an − b| < ε2 . Then pick such an n.
ε ε
|a − b| < |a − an | + |an − b| < + = ε.
2 2
Since ε is arbitrary, this proves the theorem. ■
The following is the definition of a Cauchy sequence in R p .

Definition 13.3.4 {an } is a Cauchy sequence if for all ε > 0, there exists nε such that
whenever n, m ≥ nε ,
|an −am | < ε.

A sequence is Cauchy, means the terms are “bunching up to each other” as m, n get
large.

Theorem 13.3.5 The set of terms in a Cauchy sequence in R p is bounded in the sense that
for all n, |an | < M for some M < ∞.
240 CHAPTER 13. SOME FUNDAMENTALS∗

Proof: Let ε = 1 in the definition of a Cauchy sequence and let n > n1 . Then from the
definition,
|an −an1 | < 1.
It follows that for all n > n1 ,
|an | < 1 + |an1 | .
Therefore, for all n,
n1
|an | ≤ 1 + |an1 | + ∑ |ak | . ■
k=1

Theorem 13.3.6 If a sequence {an } in R p converges, then the sequence is a Cauchy se-
quence. Also, if some subsequence of a Cauchy sequence converges, then the original
sequence converges.

Proof: Let ε > 0 be given and suppose an → a. Then from the definition of conver-
gence, there exists nε such that if n > nε , it follows that
ε
|an −a| <
2
Therefore, if m, n ≥ nε + 1, it follows that
ε ε
|an −am | ≤ |an −a| + |a − am | < + =ε
2 2
showing that, since ε > 0 is arbitrary, {an } is a Cauchy sequence. It remains to that the last
claim.  ∞
Suppose then that {an } is a Cauchy sequence and a = limk→∞ ank where ank k=1
is a subsequence. Let ε > 0 be given. Then there exists K such that if k, l ≥ K, then
|ak − al | < ε2 . Then if k > K, it follows nk > K because n1 , n2 , n3 , · · · is strictly increasing
as the subscript increases. Also, there exists K1 such that if k > K1 , ank − a < ε2 . Then
letting n > max (K, K1 ), pick k > max (K, K1 ). Then
ε ε
|a − an | ≤ a − ank + ank − an < + = ε.
2 2
Therefore, the sequence converges. ■

Definition 13.3.7 A set K in R p is said to be sequentially compact if every sequence in K

has a subsequence which converges to a point of K.
p
Theorem 13.3.8 If I0 = ∏i=1 [ai , bi ] where ai ≤ bi , then I0 is sequentially compact.
p
Proof: Leth {ak }∞ k=1i ⊆ I0 and consider
h alli sets of the form ∏i=1 [ci , di ] where [ci , di ]
ai +bi ai +bi p
equals either ai , 2 or [ci , di ] = 2 , bi . Thus there are 2 of these sets because
there are two choices for the ith slot for i = 1, · · · , p. Also, if x and y are two points in one
of these sets,
|xi − yi | ≤ 2−1 |bi − ai | .
13.4. CONTINUITY AND THE LIMIT OF A SEQUENCE 241

 1/2
p
diam (I0 ) = ∑i=1 |bi − ai |2 ,
!1/2
p
|x − y| = ∑ |xi − yi |2
i=1
!1/2
p
−1
≤2 ∑ |bi − ai | 2
≡ 2−1 diam (I0 ) .
i=1

In particular, since d ≡ (d1 , · · · , d p ) and c ≡ (c1 , · · · , c p ) are two such points,

!1/2
p
D1 ≡ ∑ |di − ci | 2
≤ 2−1 diam (I0 )
i=1

Denote by {J1 , · · · , J2 p } these sets determined above. Since the union of these sets equals
all of I0 ≡ I, it follows that for some Jk , the sequence, {ai } is contained in Jk for infinitely
many k. Let that one be called I1 . Next do for I1 what was done for I0 to get I2 ⊆ I1 such
that the diameter is half that of I1 and I2 contains {ak } for infinitely many values of k.
Continue in this way obtaining a nested sequence {Ik } such that Ik ⊇ Ik+1 , and if x, y ∈ Ik ,
then |x − y| ≤ 2−k diam (I0 ), and In contains {ak } for infinitely many values of k for each
n. Then by the nested interval lemma, there exists c such that c is contained in each Ik .
Pick an1 ∈ I1 . Next pick n2 > n1 such that an2 ∈ I2 . If an1 , · · · , ank have been chosen, let
ank+1 ∈ Ik+1 and nk+1 > nk . This can be done because in the construction, In contains {ak }
for infinitely many k. Thus the distance between ank and c is no larger than 2−k diam (I0 ),
and so limk→∞ ank = c ∈ I0 . ■

Corollary 13.3.9 Let K be a closed and bounded set of points in R p . Then K is sequentially
compact.
p
Proof: Since K is closed and bounded, there exists a closed rectangle, ∏k=1 [ak , bk ]
which contains K. Now let {x k } be a sequence of points in K. By Theorem 13.3.8, there
 p
exists a subsequence xnk such that xnk → x ∈ ∏k=1 [ak , bk ]. However, K is closed and
each of the points of the sequence is in K so x ∈ K. If not, then since KC is open, it would
follow that eventually xnk ∈ KC which is impossible. ■

Theorem 13.3.10 Every Cauchy sequence in R p converges.

Proof: Let {ak } be a Cauchy sequence. By Theorem 13.3.5, there exists some large
p
enough box ∏i=1 [ai , bi ] containing all the terms of {ak }. Therefore, by Theorem 13.3.8, a
p
subsequence converges to a point of ∏i=1 [ai , bi ]. By Theorem 13.3.6, the original sequence
converges. ■

13.4 Continuity And The Limit Of A Sequence

Just as in the case of a function of one variable, there is a very useful way of thinking of
continuity in terms of limits of sequences found in the following theorem. In words, it says
a function is continuous if it takes convergent sequences to convergent sequences whenever
possible.
242 CHAPTER 13. SOME FUNDAMENTALS∗

Theorem 13.4.1 A function f : D (f ) → Rq is continuous at x ∈ D (f ) if and only if, when-

ever xn → x with xn ∈ D (f ), it follows f (xn ) → f (x).

Proof: Suppose first that f is continuous at x and let xn → x. Let ε > 0 be given. By
continuity, there exists δ > 0 such that if |y − x| < δ , then |f (x) − f (y)| < ε. However,
there exists nδ such that if n ≥ nδ , then |xn −x| < δ , and so for all n this large,

|f (x) −f (xn )| < ε

which shows f (xn ) → f (x).

Now suppose the condition about taking convergent sequences to convergent sequences
holds at x. Suppose f fails to be continuous at x. Then there exists ε > 0 and xn ∈ D ( f )
such that |x − xn | < n1 , yet
|f (x) −f (xn )| ≥ ε.
But this is clearly a contradiction because, although xn → x, f (xn ) fails to converge to
f (x). It follows f must be continuous after all. ■

13.5 The Extreme Value Theorem And Uniform Continu-

ity
Definition 13.5.1 A function f having values in R p is said to be bounded if the set of
values of f is a bounded set.

Lemma 13.5.2 Let C ⊆ R p be closed and bounded and let f : C → Rs be continuous. Then
f is bounded.

Proof: Suppose not. Then since f is not bounded, there exists xn such that
s
f (xn ) ∈
/ ∏ (−n, n) ≡ Rn .
i=1

By Corollary 13.3.9, C is sequentially compact, and so there exists a subsequence xnk
which converges to x ∈ C. Now f (x) ∈ Rm for large enough m. Hence, by continuity of
f, it follows f (xn ) ∈ Rm for all n large enough, contradicting the construction. ■
Here is a proof of the extreme value theorem.

Theorem 13.5.3 Let C be closed and bounded and let f : C → R be continuous. Then f
achieves its maximum and its minimum on C. This means there exist x1 , x2 ∈ C such that
for all x ∈ C,
f (x1 ) ≤ f (x) ≤ f (x2 ) .

Proof: Let M = sup { f (x) : x ∈ C}. Then by Lemma 13.5.2, M is a finite number. Is
f (x2 ) = M for some x2 ? If not, you could consider the function

1
g (x) ≡
M − f (x)
13.6. CONVERGENCE OF FUNCTIONS 243

and g would be a continuous and unbounded function defined on C, contrary to Lemma

13.5.2. Therefore, there exists x2 ∈ C such that f (x2 ) = M. A similar argument applies to
show the existence of x1 ∈ C such that

f (x1 ) = inf { f (x) : x ∈ C} . ■

As in the case of a function of one variable, there is a concept of uniform continuity.

Definition 13.5.4 A function f : D (f ) → Rq is uniformly continuous if for every ε > 0

there exists δ > 0 such that whenever x, y are points of D (f ) such that |x − y| < δ , it
follows |f (x) − f (y)| < ε.

Theorem 13.5.5 Let f : K → Rq be continuous at every point of K where K is a closed

and bounded set in R p . Then f is uniformly continuous.
 
Proof: Suppose not. Then there exists ε > 0 and sequences x j and y j of points
in K such that
1
xj −y j <
j


but f (x j ) − f y j ≥ ε. Then by Corollary 13.3.9 on Page 241 which says K is sequen-
 
tially compact, there is a subsequence xnk of x j which converges to a point x ∈ K.
Then since xnk − y nk < 1k , it follows that y nk also converges to x. Therefore,


ε ≤ lim f xnk − f y nk = |f (x) − f (x)| = 0,
k→∞

a contradiction. Therefore, f is uniformly continuous as claimed. ■

13.6 Convergence of Functions

There are two kinds of convergence for a sequence of functions described in the next defi-
nition, pointwise convergence and uniform convergence. Of the two, uniform convergence
is far better and tends to be the kind of convergence most encountered in complex analy-
sis. Pointwise convergence is more often encounted in real analysis and necessitates much
more difficult theorems.

Definition 13.6.1 Let S ⊆ C p and let f n : S → Cq for n = 1, 2, · · · . Then {f n } is said to

converge pointwise to f on S if for all x ∈ S,

f n (x) → f (x)

for each x. The sequence is said to converge uniformly to f on S if

 
lim sup |f n (x) − f (x)| = 0
n→∞ x∈S

supx∈S |f n (x) − f (x)| is denoted as∥f n − f ∥∞ or just ∥f n − f ∥ for short.

∥·∥

is called the uniform norm.

244 CHAPTER 13. SOME FUNDAMENTALS∗

To illustrate the difference in the two types of convergence, here is a standard example.
Example 13.6.2 Let (
0 if x ∈ [0, 1)
f (x) ≡
1 if x = 1
Also let fn (x) ≡ xn for x ∈ [0, 1] . Then fn converges pointwise to f on [0, 1] but does not
converge uniformly to f on [0, 1].
Note how the target function is not continuous although each function in the sequence
is. The next theorem shows that this kind of loss of continuity never occurs when you have
uniform convergence. The theorem holds generally when S ⊆ X a normed linear space and
f, f n have values in Y another normed linear space. You should fill in the details to be sure
you understand this. You simply replace |·| with ∥·∥ for an appropriate norm.
Theorem 13.6.3 Let f n : S → Cq be continuous and let f n converge uniformly to f on S.
Then if f n is continuous at x ∈ S, it follows that f is also continuous at x.
Proof: Let ε > 0 be given. Let N be such that if n ≥ N, then
ε
sup |f n (y) − f (y)| ≡ ||f n − f ||∞ <
y ∈S 3
Pick such an n. Then by continuity of f n at x, there exists δ > 0 such that if |y − x| < δ ,
then |f n (y) − f n (x)| < ε3 . Then if |y − x| < δ , y ∈ S, then
|f (x) − f (y)| ≤ |f (x) − f n (x)| + |f n (x) − f n (y)| + |f n (y) − f (y)|
ε ε ε
< + + =ε
3 3 3
Thus f is continuous at x as claimed. ■

13.7 Root Test

The root test has to do with when a series of real or complex numbers converges. I am
assuming the reader has been exposed to infinite series. However, this that I am about to
explain is a little more general than what is usually seen in calculus. If you have a sequence
of real numbers {ak }∞ k=1 , if
An ≡ sup ak
k≥n
then the sequence {An } is decreasing. In the above, supk≥n ak means the least upper bound
of all ak for k ≥ n or if there is no upper bound, An is simply said to equal ∞. This is just
a formality to make it easy to give an easy discussion. Then, since {An } is a decreasing
sequence, there are two cases. One is that it is bounded below and the other case is that it
isn’t. In the first case, the sequence must converge to the greatest lower bound of the An
and in the second case, we say that the sequence converges to −∞. Then
!
lim sup an ≡ lim sup ak
n→∞ n→∞ k≥n

Thus, if lim supn→∞ < r, it follows that for all n large enough every ak < r. If lim supn→∞ ak >
r, it means there are infinitely many k such that ak > r.
13.8. CONVERGENCE OF SUMS 245

Theorem 13.7.1 Let ak ∈ F p , F is either R or C and consider ∑∞

k=1 ak . Then this series
converges absolutely if
lim sup |ak |1/k = r < 1.
k→∞

The series diverges spectacularly if lim supk→∞ |ak |1/k > 1 and if

lim sup |ak |1/k = 1,

k→∞

the test fails.

Proof: Suppose first that lim supk→∞ |ak |1/k = r < 1. Then letting R ∈ (r, 1) , it follows
from the definition of lim sup that for all k large enough,

|ak |1/k ≤ R

Hence there exists N such that if k ≥ N, then |ak | ≤ Rk . Let Mk = |ak | for k < N and let
Mk = Rk for k ≥ N. Then
N−1
∞
RN
∑ Mk ≤ ∑ |ak | + 1 − R < ∞
k=1 k=1

and so, by the Weierstrass M test applied to the series of constants, the series converges and
also converges absolutely. If

lim sup |ak |1/k = r > 1,

k→∞

then letting r > R > 1, it follows that for infinitely many k,

|ak | > Rk

and so there is a subsequence which is unbounded. In particular, the series cannot converge
1
and in fact diverges spectacularly. In case that the lim sup = 1, you can consider ∑∞ n=1 n
∞ 1
which diverges by calculus and ∑n=1 n2 which converges, also from calculus. However, the
lim sup equals 1 for both of these. ■
This is a major theorem because the lim sup always exists. As an important application,
here is a corollary.

Corollary 13.7.2 If ∑k ak converges, then lim supk→∞ |ak |1/k ≤ 1.

If the sequence has values in X a complete normed linear space, there is no change in
the conclusion or proof of the above theorem. You just replace |·| with ∥·∥ the symbol for
the norm.

13.8 Convergence of Sums

One can consider convergence of infinite series the same way as done in calculus.
246 CHAPTER 13. SOME FUNDAMENTALS∗

n
Definition 13.8.1 The symbol ∑∞ k=1 f k (x) means limn→∞ ∑k=1 f k (x) provided this limit
exists. This is called pointwise convergence of the infinite sum. Thus the infinite sum means
the limit of the sequence of partial sums. The infinite sum is said to converge uniformly if
the sequence of paritial sums converges uniformly.
Note how this theorem includes the case of ∑∞k=1 ak as a special case. Here the ak don’t
depend on x.
The following theorem is very useful. It tells how to recognize that an infinite sum is
converging or converging uniformly. First is a little lemma which reviews standard calcu-
lus.
Lemma 13.8.2 Suppose Mk ≥ 0 and ∑∞
k=1 Mk converges. Then
∞
lim
m→∞
∑ Mk = 0
k=m

Proof: By assumption, there is N such that if m ≥ N, then if n > m,

n m n
∑ Mk − ∑ Mk = ∑ Mk < ε/2
k=1 k=1 k=m+1

Then letting n → ∞, one can pass to a limit and conclude that

∞
∑ Mk < ε
k=m+1

It follows that for m > N, ∑∞

k=m Mk < ε. The part about passing to a limit follows from the
fact that n → ∑nk=m+1 Mk is an increasing sequence which is bounded above by ∑∞ k=1 Mk .
Therefore, it converges by completeness of R. ■
Theorem 13.8.3 For x ∈ S, if ∑∞ ∞
k=1 |f k (x)| < ∞, then ∑k=1 f k (x) converges pointwise. If
there exists Mk such that Mk ≥ |f k (x)| for all x ∈ S, then ∑∞
k=1 f k (x) converges uniformly.

Proof: Let m < n. Then

n m ∞
∑ f k (x) − ∑ f k (x) ≤ ∑ |f k (x)| < ε/2
k=1 k=1 k=m

whenever m is large enough due to the assumption that ∑∞ k=1 |f k (x)| < ∞. Thus the partial
sums are a Cauchy sequence and so the series converges pointwise.
If Mk ≥ |f k (x)| for all x ∈ S, then for M large enough,
n m ∞ ∞
∑ f k (x) − ∑ f k (x) ≤ ∑ |f k (x)| ≤ ∑ Mk < ε/2
k=1 k=1 k=m k=m

Thus, taking sup

n m
∑ f k (·) − ∑ f k (·) ≤ ε/2 < ε
k=1 k=1
and so the partial sums are uniformly Cauchy sequence. Hence they converge uniformly to
what is defined as ∑∞k=1 f k (x) for x ∈ S. ■
Some of the following exercises have been essentially done in the above discussion.
Try doing them yourself. There are also some new topics.
13.9. CONNECTED SETS 247

13.9 Connected Sets

Stated informally, connected sets are those which are in one piece. In order to define what
is meant by this, I will first consider what it means for a set to not be in one piece. This is
called separated. Connected sets are defined in terms of not being separated. This is why
theorems about connected sets sometimes seem a little tricky.

Definition 13.9.1 Let A be a nonempty subset Rn . Then A is defined to be the intersection

of all closed sets which contain A. This is called the closure of A. Note the whole space, Rn
is one such closed set which contains A.

Lemma 13.9.2 Let A be a nonempty set in Rn . Then A is a closed set and

A = A ∪ A′

where A′ denotes the set of limit points of A.

Proof: First of all, denote by C the set of closed sets which contain A. Then

A = ∩C

and this will be closed if its complement is open. However,

C
A = ∪ HC : H ∈ C .


Each H C is open and so the union of all these open sets must also be open. This is because
if x is in this union, then it is in at least one of them. Hence it is an interior point of that
one. But this implies it is an interior point of the union of them all which is an even larger
set. Thus A is closed.
The interesting part is the next claim. First note that from the definition, A ⊆ A so if
x ∈ A, then x ∈ A. Now consider y ∈ A′ but y ∈ / A. If y ∈
/ A, a closed set, then there exists
C
B (y, r) ⊆ A . Thus y cannot be a limit point of A, a contradiction. Therefore,

A ∪ A′ ⊆ A

Next suppose x ∈ A and suppose x ∈ / A. Then if B (x, r) contains no points of A different

than x, since x itself is not in A, it would follow that B (x,r) ∩ A = 0/ and so recalling that
open balls are open, B (x, r)C is a closed set containing A so from the definition, it also
contains A which is contrary to the assertion that x ∈ A. Hence if x ∈ / A, then x ∈ A′ and
so
A ∪ A′ ⊇ A ■
Now is a definition about what it means to not be connected. This is called separated.

Definition 13.9.3 A set, S in Rn , is separated if there exist sets A, B such that

S = A ∪ B, A, B ̸= 0,
/ and A ∩ B = B ∩ A = 0.
/

In this case, the sets A and B are said to separate S. A set is connected if it is not separated.
Remember A denotes the closure of the set A.
248 CHAPTER 13. SOME FUNDAMENTALS∗

Note that the concept of connected sets is defined in terms of what it is not. This makes
it somewhat difficult to understand. One of the most important theorems about connected
sets is the following.

Theorem 13.9.4 Suppose U is a set of connected sets and that there exists a point p which
is in all of these connected sets. Then K ≡ ∪U is connected.

Proof: Suppose
K = A∪B
where Ā ∩ B = B̄ ∩ A = 0,
/ A ̸= 0, / Let U ∈ U . Then
/ B ̸= 0.

U = (U ∩ A) ∪ (U ∩ B)

and this would separate U if both sets in the union are nonempty since the limit points of
U ∩ B are contained in the limit points of B. It follows that every set of U is contained
in one of A or B. Suppose then that some U ⊆ A. Then all U ∈ U must be contained in A
because if one is contained in B, this would violate the assumption that they all have a point
p in common. Thus K is connected after all because this requires B = 0. / Alternatively, p is
in one of these sets. Say p ∈ A. Then by the above argument every U must be in A because
if not, the above would be a separation of U. Thus B = 0. / ■
The intersection of connected sets is not necessarily connected as is shown by the fol-
lowing picture.

U
V

Theorem 13.9.5 Let f : X → Rm be continuous where X is connected. Then f (X) is also

connected.

Proof: To do this you show f (X) is not separated. Suppose to the contrary that f (X) =
A ∪ B where A and B separate f (X) . Then consider the sets f −1 (A) and f −1 (B) . If z
∈ f −1 (B) , then f (z) ∈ B and so f (z) is not a limit point of A. Therefore, there exists an
open set, U containing f (z) such that U ∩ A = 0. / But then, the continuity of f implies that
f −1 (U) is an open set containing z such that f −1 (U) ∩ f −1 (A) = 0. / Therefore, f −1 (B)
−1 −1
contains no limit points of f (A) . Similar reasoning implies f (A) contains no limit
points of f −1 (B). It follows that X is separated by f −1 (A) and f −1 (B) , contradicting the
assumption that X was connected. ■
An arbitrary set can be written as a union of maximal connected sets called connected
components. This is the concept of the next definition.

Definition 13.9.6 Let S be a set and let p ∈ S. Denote by Cp the union of all connected
subsets of S which contain p. This is called the connected component determined by p.
13.9. CONNECTED SETS 249

Theorem 13.9.7 Let Cp be a connected component of a set S . Then Cp is a connected set

and if Cp ∩Cq ̸= 0,
/ then Cp = Cq .

Proof: Let C denote the connected subsets of S which contain p. By Theorem 13.9.4,
∪C = Cp is connected. If x ∈ Cp ∩ Cq , then from Theorem 13.9.4, Cp ⊇ Cp ∪ Cq and so
Cp ⊇ Cq . The inclusion goes the other way by the same reason. ■
This shows the connected components of a set are equivalence classes and partition the
set.
A set, I is an interval in R if and only if whenever x, y ∈ I then (x, y) ⊆ I. The following
theorem is about the connected sets in R.

Theorem 13.9.8 A set C in R is connected if and only if C is an interval.

Proof: Let C be connected. If C consists of a single point, p, there is nothing to prove.

The interval is just [p, p] . Suppose p < q and p, q ∈ C. You need to show (p, q) ⊆ C. If

x ∈ (p, q) \C

let C ∩ (−∞, x) ≡ A, and C ∩ (x, ∞) ≡ B. Then C = A ∪ B and the sets A and B separate C
contrary to the assumption that C is connected.
Conversely, let I be an interval. Suppose I is separated by A and B. Pick x ∈ A and
y ∈ B. Suppose without loss of generality that x < y. Now define the set,

S ≡ {t ∈ [x, y] : [x,t] ⊆ A}

and let l be the least upper bound of S. Then l ∈ A so l ∈

/ B which implies l ∈ A. But if
l∈
/ B, then for some δ > 0,
(l, l + δ ) ∩ B = 0/
contradicting the definition of l as an upper bound for S. Therefore, l ∈ B which implies
l∈/ A after all, a contradiction. It follows I must be connected. ■
This yields a generalization of the intermediate value theorem from one variable calcu-
lus.

Corollary 13.9.9 Let E be a connected set in Rn and suppose f : E → R and that y ∈

( f (e1 ) , f (e2 )) where ei ∈ E. Then there exists e ∈ E such that f (e) = y.

Proof: From Theorem 13.9.5, f (E) is a connected subset of R. By Theorem 13.9.8

f (E) must be an interval. In particular, it must contain y. This proves the corollary. ■
The following theorem is a very useful description of the open sets in R.

Theorem 13.9.10 Let U be an open set in R. Then there exist countably many disjoint
open sets {(ai , bi )}∞
i=1 such that U = ∪i=1 (ai , bi ) .
∞

Proof: Let p ∈ U and let z ∈ C p , the connected component determined by p. Since U is

open, there exists, δ > 0 such that (z − δ , z + δ ) ⊆ U. It follows from Theorem 13.9.4 that

(z − δ , z + δ ) ⊆ C p .

This shows C p is open. By Theorem 13.9.8, this shows C p is an open interval, (a, b) where
a, b ∈ [−∞, ∞] . There are therefore at most countably many of these connected compo-
nents because each must contain a rational number and the rational numbers are countable.
Denote by {(ai , bi )}∞
i=1 the set of these connected components. ■
250 CHAPTER 13. SOME FUNDAMENTALS∗

Definition 13.9.11 A set E in Rn is arcwise connected if for any two points, p, q ∈ E, there
exists a closed interval, [a, b] and a continuous function, γ : [a, b] → E such that γ (a) = p
and γ (b) = q.

An example of an arcwise connected space would be any subset of Rn which is the

continuous image of an interval. Arcwise connected is not the same as connected. A well
known example is the following.
  
1
x, sin : x ∈ (0, 1] ∪ {(0, y) : y ∈ [−1, 1]} (13.2)
x

You can verify that this set of points in R2 is not arcwise connected but is connected.

Lemma 13.9.12 In Rn , B (z,r) is arcwise connected.

Proof: This is easy from the convexity of the set. If x, y ∈ B (z, r) , then let γ (t) =
x + t (y − x) for t ∈ [0, 1] .

∥x + t (y − x) − z∥ = ∥(1 − t) (x − z) + t (y − z)∥
≤ (1 − t) ∥x − z∥ + t ∥y − z∥
< (1 − t) r + tr = r

showing γ (t) stays in B (z, r).■

Proposition 13.9.13 If X ̸= 0/ is arcwise connected, then it is connected.

Proof: Let p ∈ X. Then by assumption, for any x ∈ X, there is an arc joining p and x.
This arc is connected because it is the continuous image of an interval which is connected.
Since x is arbitrary, every x is in a connected subset of X which contains p. Hence C p = X
and so X is connected. ■

Theorem 13.9.14 Let U be an open subset of a Rn . Then U is arcwise connected if and

only if U is connected. Also the connected components of an open set are open sets.

Proof: By Proposition 13.9.13 it is only necessary to verify that if U is connected and

open in the context of this theorem, then U is arcwise connected. Pick p ∈ U. Say x ∈ U
satisfies P if there exists a continuous function, γ : [a, b] → U such that γ (a) = p and
γ (b) = x.
A ≡ {x ∈ U such that x satisfies P.}
If x ∈ A, then Lemma 13.9.12 implies B (x, r) ⊆ U is arcwise connected for small
enough r. Thus letting y ∈ B (x, r) , there exist intervals, [a, b] and [c, d] and continuous
functions having values in U, γ, η such that γ (a) = p, γ (b) = x, η (c) = x, and η (d) = y.
Then let γ 1 : [a, b + d − c] → U be defined as
(
γ (t) if t ∈ [a, b]
γ 1 (t) ≡
η (t + c − b) if t ∈ [b, b + d − c]

Then it is clear that γ 1 is a continuous function mapping p to y and showing that B (x, r) ⊆
A. Therefore, A is open. A ̸= 0/ because since U is open there is an open set, B (p, δ )
containing p which is contained in U and is arcwise connected.
13.10. EXERCISES 251

Now consider B ≡ U \ A. I claim this is also open. If B is not open, there exists a
point z ∈ B such that every open set containing z is not contained in B. Therefore, letting
B (z, δ ) be such that z ∈ B (z, δ ) ⊆ U, there exist points of A contained in B (z, δ ) . But
then, a repeat of the above argument shows z ∈ A also. Hence B is open and so if B ̸= 0, /
then U = B ∪ A and so U is separated by the two sets B and A contradicting the assumption
that U is connected.
It remains to verify the connected components are open. Let z ∈ Cp where Cp is the
connected component determined by p. Then picking B (z, δ ) ⊆ U, Cp ∪ B (z, δ ) is con-
nected and contained in U and so it must also be contained in Cp . Thus z is an interior
point of Cp . ■
As an application, consider the following corollary.

Corollary 13.9.15 Let f : Ω → Z be continuous where Ω is a connected open set in Rn .

Then f must be a constant.

Proof: Suppose not. Then it achieves two different values, k and l ̸= k. Then Ω =
f −1 (l) ∪ f −1 ({m ∈ Z : m ̸= l}) and these are disjoint nonempty open sets which separate
Ω. To see they are open, note
  
1 1
f −1 ({m ∈ Z : m ̸= l}) = f −1 ∪m̸=l m − , m +
6 6

which is the inverse image of an open set while f −1 (l) = f −1 l − 16 , l + 16 also an open

set. ■

13.10 Exercises
1. Suppose {xn } is a sequence contained in a closed set C such that limn→∞ xn = x.
Show that x ∈ C. Hint: Recall that a set is closed if and only if the complement of
the set is open. That is if and only if Rn \C is open.
2. Show using Problem 1 and Theorem 13.3.8 that every closed and bounded set is
sequentially compact. Hint: If C is such a set, then C ⊆ I0 ≡ ∏ni=1 [ai , bi ]. Now if
{xn } is a sequence in C, it must also be a sequence in I0 . Apply Problem 1 and
Theorem 13.3.8.

3. Prove the extreme value theorem, a continuous function achieves its maximum and
minimum on any closed and bounded set C, using the result of Problem 2. Hint:
Suppose λ = sup { f (x) : x ∈ C}. Then there exists {xn } ⊆ C such that f (xn ) →
λ . Now select a convergent subsequence using Problem 2. Do the same for the
minimum.

4. Let C be a closed and bounded set and suppose f : C → Rm is continuous. Show that
f must also be uniformly continuous. This means: For every ε > 0 there exists δ >
0 such that whenever x, y ∈ C and |x − y| < δ , it follows |f (x) − f (y)| < ε. This
is a good time to review the definition of continuity so you will see the difference.
Hint: Suppose it is not so. Then there exists ε > 0 and {xk } and {y k } such that
|xk − y k | < 1k but |f (xk ) − f (y k )| ≥ ε. Now use Problem 2 to obtain a convergent
subsequence.
252 CHAPTER 13. SOME FUNDAMENTALS∗

5. From Problem 2 every closed and bounded set is sequentially compact. Are these the
only sets which are sequentially compact? Explain.
6. A set whose elements are open sets C is called an open cover of H if ∪C ⊇ H.
In other words, C is an open cover of H if every point of H is in at least one set
of C . Show that if C is an open cover of a closed and bounded set H then there
exists δ > 0 such that whenever x ∈ H, B (x, δ ) is contained in some set of C . This
number δ is called a Lebesgue number. Hint: If there is no Lebesgue number for
H, let H ⊆ I = ∏ni=1 [ai , bi ]. Use the process of chopping the intervals in half to
get a sequence of nested intervals, Ik contained in I where diam (Ik ) ≤ 2−k diam (I)
and there is no Lebesgue number for the open cover on Hk ≡ H ∩ Ik . Now use the
nested interval theorem to get c in all these Hk . For some r > 0 it follows B (c, r) is
contained in some open set of U. But for large k, it must be that Hk ⊆ B (c, r) which
contradicts the construction. You fill in the details.
7. A set is compact if for every open cover of the set, there exists a finite subset of
the open cover which also covers the set. Show every closed and bounded set in
R p is compact. Next show that if a set in R p is compact, then it must be closed and
bounded. This is called the Heine Borel theorem. Hint: To show closed and bounded
is compact, you might use the technique of chopping into small pieces of the above
problem.
8. Suppose S is a nonempty set in R p . Define

dist (x,S) ≡ inf {|x − y| : y ∈ S} .

Show that
|dist (x,S) − dist (y,S)| ≤ |x − y| .
Hint: Suppose dist (x, S) < dist (y, S). If these are equal there is nothing to show.
Explain why there exists z ∈ S such that |x − z| < dist (x,S) + ε. Now explain why

|dist (x,S) − dist (y,S)| = dist (y,S) − dist (x,S) ≤ |y − z| − (|x − z| − ε)

Now use the triangle inequality and observe that ε is arbitrary.

9. Suppose H is a closed set and H ⊆ U ⊆ R p , an open set. Show there exists a con-
tinuous function defined on R p , f such that f (R p ) ⊆ [0, 1], f (x) = 0 if x ∈
/ U and
f (x) = 1 if x ∈ H. Hint: Try something like

dist x,U C


,
dist (x,U C ) + dist (x, H)

where U C ≡ R p \U, a closed set. You need to explain why the denominator is never
equal to zero. The rest is supplied by Problem 8. This is a special case of a major
theorem called Urysohn’s lemma.
Chapter 14

Vector Valued Functions Of One

Variable

14.1 Limits Of A Vector Valued Function Of One Real

Variable
As in the case of a scalar valued function of one variable, the derivative is defined as
f (t0 + h) − f (t0 )
lim .
h→0 h
Thus the derivative of a function of one variable involves a limit. The following is the
definition of what is meant by a limit. The new topic is the case of one sided limits although
there is really nothing essentially new from what was done earlier. Here is the definition.

Definition 14.1.1 In the case where D (f ) is only assumed to satisfy D (f ) ⊇ (t,t + r),

lim f (s) = L
s→t+

if and only if for all ε > 0 there exists δ > 0 such that if

0 < s−t < δ,

then
|f (s) − L| < ε.
In the case where D (f ) is only assumed to satisfy D (f ) ⊇ (t − r,t),

lim f (s) = L
s→t−

if and only if for all ε > 0 there exists δ > 0 such that if

0 < t −s < δ,

then
|f (s) − L| < ε.

253
254 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

One can also consider limits as a variable “approaches” infinity. Of course nothing is
“close” to infinity and so this requires a slightly different definition.
lim f (t) = L
t→∞

if for every ε > 0 there exists l such that whenever t > l,

|f (t) − L| < ε (14.1)
and
lim f (t) = L
t→−∞
if for every ε > 0 there exists l such that whenever t < l, (14.1) holds.
Note that in all of this the definitions are identical to the case of scalar valued functions.
The only difference is that here |·| refers to the norm or length in R p where maybe p > 1.
Example 14.1.2 Let f (t) = cost, sint,t 2 + 1, ln (t) . Find limt→π/2 f (t) .

Use Theorem 12.5.5 on Page 223 and the continuity of the functions to write this limit
equals
 
lim cost, lim sint, lim t 2 + 1 , lim ln (t)


t→π/2 t→π/2 t→π/2 t→π/2
  2   
π π
= 0, 1, ln + 1 , ln .
4 2
Example 14.1.3 Let f (t) = sint 2


t ,t ,t + 1 . Find limt→0 f (t).

Recall that limt→0 sint

t = 1. Then from Theorem 12.5.5 on Page 223, limt→0 f (t) =
(1, 0, 1).

14.2 The Derivative And Integral

The following definition is on the derivative and integral of a vector valued function of one
variable.
Definition 14.2.1 The derivative of a function f ′ (t), is defined as the following limit when-
ever the limit exists. If the limit does not exist, then neither does f ′ (t).
f (t + h) − f (t)
lim ≡ f ′ (t)
h→0 h
As before,
f (s) − f (t)
f ′ (t) = lim .
s→t s−t
The function of h on the left is called the difference quotient just as it was for a scalar
valued function. If f (t) = ( f1 (t) , · · · , f p (t)) and ab fi (t) dt exists for each i = 1, · · · , p,
R

then ab f (t) dt is defined as the vector

R

Z b Z b 
f1 (t) dt, · · · , f p (t) dt .
a a

This is what is meant by saying f ∈ R ([a, b]).

14.2. THE DERIVATIVE AND INTEGRAL 255

Here is a simple proposition which is useful to have.

Proposition 14.2.2 Let a ≤ b, f = ( f1 , · · · , fn ) is vector valued and each fi is continuous,

then Z b
√ Zb
f (t) dt ≤ n |f (t)| dt.
a a

Proof: This follows from the following computation.

Z b Z b Z b 
f (t) dt = f1 (t) dt, · · · , fn (t) dt
a a a
!1/2 2 !1/2
n Z b 2 n Z b
= ∑ fi (t) dt ≤ ∑ | fi (t)| dt
i=1 a i=1 a

2 !1/2
b √
Z Z b 
≤ n max | fi (t)| dt = n max | fi (t)| dt
i a i a

√ Zb
≤ n |f (t)| dt ■
a

As in the case of a scalar valued function differentiability implies continuity but not the
other way around.

Theorem 14.2.3 If f ′ (t) exists, then f is continuous at t.

Proof: Suppose ε > 0 is given and choose δ 1 > 0 such that if |h| < δ 1 ,
f (t + h) − f (t)
− f ′ (t) < 1.
h

then for such h, the triangle inequality

 implies |f (t + h) − f (t)| < |h| + f ′ (t) |h| . Now
letting δ < min δ 1 , 1+|fε ′ (x)| it follows if |h| < δ , then |f (t + h) − f (t)| < ε. Letting
y = h + t, this shows that if |y − t| < δ , |f (y) − f (t)| < ε which proves f is continuous at
t. ■
As in the scalar case, there is a fundamental theorem of calculus.

Theorem 14.2.4 If f ∈ R ([a, b]) and if f is continuous at t ∈ (a, b), then

Z t 
d
f (s) ds = f (t) .
dt a

Proof: Say f (t) = ( f1 (t) , · · · , f p (t)). Then it follows

1 t+h 1 t
 Z t+h
1 t+h

1
Z Z Z
f (s) ds − f (s) ds = f1 (s) ds, · · · , f p (s) ds
h a h a h t h t

and limh→0 h1 tt+h fi (s) ds = fi (t) for each i = 1, · · · , p from the fundamental theorem of
R

calculus for scalar valued functions. Therefore,

Z t+h Z t
1 1
lim f (s) ds − f (s) ds = ( f1 (t) , · · · , f p (t)) = f (t) .■
h→0 h a h a
256 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

Example 14.2.5 Let f (x) = c where c is a constant. Find f ′ (x).

The difference quotient,

f (x + h) − f (x) c − c
= =0
h h
Therefore,
f (x + h) − f (x)
lim = lim 0 = 0
h→0 h h→0

Example 14.2.6 Let f (t) = (at, bt) where a, b are constants. Find f ′ (t).

From the above discussion this derivative is just the vector valued functions whose
components consist of the derivatives of the components of f . Thus f ′ (t) = (a, b).

14.2.1 Geometric And Physical Significance Of The Derivative

Suppose r is a vector valued function of a parameter t not necessarily time and consider
the following picture of the points traced out by r.

r(t + h)
r(t)

In this picture there are unit vectors in the direction of the vector from r (t) to r (t + h).
You can see that it is reasonable to suppose these unit vectors, if they converge, converge
to a unit vector T which is tangent to the curve at the point r (t). Now each of these unit
vectors is of the form
r (t + h) − r (t)
≡ T h.
|r (t + h) − r (t)|
Thus T h → T, a unit tangent vector to the curve at the point r (t). Therefore,

r (t + h) − r (t) |r (t + h) − r (t)| r (t + h) − r (t)

r ′ (t) ≡ lim = lim
h→0 h h→0 h |r (t + h) − r (t)|
|r (t + h) − r (t)|
= lim T h = r ′ (t) T.
h→0 h
In the case that t is time, the expression |r (t + h) − r (t)| is a good approximation for
the distance traveled by the object on the time interval [t,t + h]. The real distance would be
the length of the curve joining the two points but if h is very small, this is essentially equal
to |r (t + h) − r (t)| as suggested by the picture below.
14.2. THE DERIVATIVE AND INTEGRAL 257

r(t + h)

r(t)

Therefore, |r(t+h)−
h
r (t)|
gives for small h, the approximate distance travelled on the time
interval [t,t + h] divided by the length of time h. Therefore, this expression is really the
average speed of the object on this small time interval and so the limit as h → 0, deserves
to be called the instantaneous speed of the object. Thus |r ′ (t)| T represents the speed times
a unit direction vector T which defines the direction in which the object is moving. Thus
r ′ (t) is the velocity of the object. This is the physical significance of the derivative when
t is time. In general, r ′ (t) and T (t) are vectors tangent to the curve which point in the
direction of motion.
How do you go about computing r ′ (t)? Letting r (t) = (r1 (t) , · · · , rq (t)), the expres-
sion
r (t0 + h) − r (t0 )
(14.2)
h
is equal to  
r1 (t0 + h) − r1 (t0 ) rq (t0 + h) − rq (t0 )
,··· , .
h h
Then as h converges to 0, (14.2) converges to v ≡ (v1 , · · · , vq ) where vk = rk′ (t). This is
because of Theorem 12.5.5 on Page 223, which says that the term in (14.2) gets close to
a vector v if and only if all the coordinate functions of the term in (14.2) get close to the
corresponding coordinate functions of v.
In the case where t is time, this simply says the velocity vector equals the vector whose
components are the derivatives of the components of the displacement vector r (t).

Example 14.2.7 Let r (t) = sint,t 2 ,t + 1 for t ∈ [0, 5]. Find a tangent line to the curve

parameterized by r at the point r (2).

From the above discussion, a direction vector has the same direction as r ′ (2). There-
fore, it suffices to simply use r ′ (2) as a direction vector for the line. r ′ (2) = (cos 2, 4, 1).
Therefore, a parametric equation for the tangent line is

(sin 2, 4, 3) + t (cos 2, 4, 1) = (x, y, z) .

Example 14.2.8 Let r (t) = sint,t 2 ,t + 1 for t ∈ [0, 5]. Find the velocity vector when

t = 1.

From the above discussion, this is simply r ′ (1) = (cos 1, 2, 1).

14.2.2 Differentiation Rules

There are rules which relate the derivative to the various operations done with vectors such
as the dot product, the cross product, vector addition, and scalar multiplication.
258 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

Theorem 14.2.9 Let a, b ∈ R and suppose f ′ (t) and g ′ (t) exist. Then the following for-
mulas are obtained.
(af + bg)′ (t) = af ′ (t) + bg ′ (t) . (14.3)
(f · g)′ (t) = f ′ (t) · g (t) + f (t) · g ′ (t) (14.4)
If f, g have values in R3 , then

(f × g)′ (t) = f (t) × g ′ (t) + f ′ (t) × g (t) (14.5)

The formulas, (14.4), and (14.5) are referred to as the product rule.

Proof: The first formula is left for you to prove. Consider the second, (14.4).
f · g (t + h) − f g (t)
lim
h→0 h

f (t + h) · g (t + h) − f (t + h) · g (t) f (t + h) · g (t) − f (t) · g (t)

= lim +
h→0 h h
 
(g (t + h) − g (t)) (f (t + h) − f (t))
= lim f (t + h) · + · g (t)
h→0 h h
n n
(gk (t + h) − gk (t)) ( fk (t + h) − fk (t))
= lim ∑ fk (t + h) +∑ gk (t)
h→0 k=1 h k=1 h
n n
= ∑ fk (t) g′k (t) + ∑ fk′ (t) gk (t) = f ′ (t) · g (t) + f (t) · g′ (t) .
k=1 k=1

Formula (14.5) is left as an exercise which follows from the product rule and the definition
of the cross product. ■

Example 14.2.10 Let r (t) = t 2 , sint, cost and let p (t) = (t, ln (t + 1) , 2t). Simplify the

expression (r (t) × p (t))′ .

1
From (14.5) this equals(2t, cost, − sint)×(t, ln (t + 1) , 2t)+ t 2 , sint, cost × 1, t+1



,2 .

Example 14.2.11 Let r (t) = t 2 , sint, cost Find 0π r (t) dt.

R

This equals 0π t 2 dt, 0π sint dt, 0π cost dt = 31 π 3 , 2, 0 .

R R R


t
√ 
Example 14.2.12 An object has position r (t) = t 3 , 1+1 , t 2 + 2 kilometers where t is
given in hours. Find the velocity of the object in kilometers per hour when t = 1.

Recall the velocity at time t was r ′ (t). Therefore, find r ′ (t) and plug in t = 1 to find
the velocity.
! !
2 1 (1 + t) − t 1 2 1 1
′

−1/2 2
r (t) = 3t , , t +2 2t = 3t , ,p t
(1 + t)2 2 (1 + t)2 (t 2 + 2)

When t = 1, the velocity is

 
′ 1 1
r (1) = 3, , √ kilometers per hour.
4 3
14.2. THE DERIVATIVE AND INTEGRAL 259

Obviously, this can be continued. That is, you can consider the possibility of taking the
derivative of the derivative and then the derivative of that and so forth. The main thing to
consider about this is the notation, and it is exactly like it was in the case of a scalar valued
function presented earlier. Thus r ′′ (t) denotes the second derivative.
When you are given a vector valued function of one variable, sometimes it is possible
to give a simple description of the curve which results. Usually it is not possible to do this!

Example 14.2.13 Describe the curve which results from the vector valued function r (t) =
(cos 2t, sin 2t,t) where t ∈ R.

The first two components indicate that for r (t) = (x (t) , y (t) , z (t)), the pair, (x (t) , y (t))
traces out a circle. While it is doing so, z (t) is moving at a steady rate in the positive
direction. Therefore, the curve which results is a cork screw shaped thing called a helix.
As an application of the theorems for differentiating curves, here is an interesting ap-
plication. It is also a situation where the curve can be identified as something familiar.

Example 14.2.14 Sound waves have the angle of incidence equal to the angle of reflection.
Suppose you are in a large room and you make a sound. The sound waves spread out and
you would expect your sound to be inaudible very far away. But what if the room were
shaped so that the sound is reflected off the wall toward a single point, possibly far away
from you? Then you might have the interesting phenomenon of someone far away hearing
what you said quite clearly. How should the room be designed?

Suppose you are located at the point P 0 and the point where your sound is to be
reflected is P 1 . Consider a plane which contains the two points and let r (t) denote a
parametrization of the intersection of this plane with the walls of the room. Then the con-
dition that the angle of reflection equals the angle of incidence reduces to saying the angle
between P 0 − r (t) and −r ′ (t) equals the angle between P 1 − r (t) and r ′ (t). Draw a
picture to see this. Therefore,

(P 0 − r (t)) · (−r ′ (t)) (P 1 − r (t)) · (r ′ (t))

= .
|P 0 − r (t)| |r ′ (t)| |P 1 − r (t)| |r ′ (t)|

This reduces to
(r (t) − P 0 ) · (−r ′ (t)) (r (t) − P 1 ) · (r ′ (t))
= (14.6)
|r (t) − P 0 | |r (t) − P 1 |
Now
(r (t) − P 1 ) · (r ′ (t)) d
= |r (t) − P 1 |
|r (t) − P 1 | dt
and a similar formula holds for P 1 replaced with P 0 . This is because
p
|r (t) − P 1 | = (r (t) − P 1 ) · (r (t) − P 1 )

and so using the chain rule and product rule,

d 1
((r (t) − P 1 ) · (r (t) − P 1 ))−1/2 2 (r (t) − P 1 ) · r ′ (t)


|r (t) − P 1 | =
dt 2
(r (t) − P 1 ) · (r ′ (t))
= .
|r (t) − P 1 |
260 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

Therefore, from (14.6),

d d
(|r (t) − P 1 |) + (|r (t) − P 0 |) = 0
dt dt
showing that |r (t) − P 1 | + |r (t) − P 0 | = C for some constant [Link] implies the curve of
intersection of the plane with the room is an ellipse having P 0 and P 1 as the foci.

14.2.3 Leibniz’s Notation

dy
Leibniz’s notation also generalizes routinely. For example, dt = y ′ (t) with other similar
notations holding.

14.3 Exercises
1. Find the following limits if possible
 
(a) limx→0+ |x| x , sin x/x, cos x
 
x
(b) limx→0+ |x| , sec x, ex
 2 
−16
(c) limx→4 x x+4 , x + 7, tan5x4x
 
x x2 sin x2
(d) limx→∞ 1+x 2 , 1+x2 , x

2. Find
x2 − 4 2 x2 − 4
 
lim , x + 2x − 1, .
x→2 x+2 x−2
√ √
3. Prove from the definition that limx→a ( 3 x, x + 1) = ( 3 a, a + 1) for all a ∈ R. Hint:
You might want to use the formula for the difference of two cubes,

a3 − b3 = (a − b) a2 + ab + b2 .

4. Let  p 
r (t) = 4 + t 2 , t 2 + 1t 3 ,t 3

describe the position of an object in R3 as a function of t where t is measured in

seconds and r (t) is measured in meters. Is the velocity of this object ever equal to
zero? If so, find the value of t at which this occurs and the point in R3 at which the
velocity is zero.
5. Let r (t) = sin 2t,t 2 , 2t + 1 for t ∈ [0, 4]. Find a tangent line to the curve parame-

terized by r at the point r (2).

6. Let r (t) = t, sint 2 ,t + 1 for t ∈ [0, 5]. Find a tangent line to the curve parameterized

by r at the point r (2).

7. Let r (t) = sint,t 2 , cos t 2 for t ∈ [0, 5]. Find a tangent line to the curve parame-

terized by r at the point r (2).

14.4. LINE INTEGRALS 261

8. Let r (t) = sint, cos t 2 ,t + 1 for t ∈ [0, 5]. Find the velocity when t = 3.

9. Let r (t) = sint,t 2 ,t + 1 for t ∈ [0, 5]. Find the velocity when t = 3.

10. Let r (t) = t, ln t 2 + 1 ,t + 1 for t ∈ [0, 5]. Find the velocity when t = 3.

11. Suppose an object has position r (t) ∈ R3 where r is differentiable and suppose also
that |r (t)| = c where c is a constant.

(a) Show first that this condition does not require r (t) to be a constant. Hint: You
can do this either mathematically or by giving a physical example.
(b) Show that you can conclude that r ′ (t) · r (t) = 0. That is, the velocity is always
perpendicular to the displacement.

12. Prove (14.5) from the component description of the cross product.
13. Prove (14.5) from the formula (f × g)i = ε i jk f j gk .

14. Prove (14.5) directly from the definition of the derivative without considering com-
ponents.
15. A Bezier curve in R p is a vector valued function of the form
n  
n
y (t) = ∑ xk (1 − t)n−k t k
k=0 k

where here the nk are the binomial coefficients and xk are n + 1 points n . Show


in R
n n
(0) =
x0 , y (1) = xn , and find y ′ (0) and y ′ (1). Recall that 0 = n = 1 and



that y
n
n
n−1 = 1 = n. Curves of this sort are important in various computer programs.

16. Suppose r (t), s (t), and p (t) are three differentiable functions of t which have values
in R3 . Find a formula for (r (t) × s (t) · p (t))′ .
17. If r ′ (t) = 0 for all t ∈ (a, b), show that there exists a constant vector c such that
r (t) = c for all t ∈ (a, b).
Rb
18. If F ′ (t) = f (t) for all t ∈ (a, b) and F is continuous on [a, b], show that a f (t) dt =
F (b) − F (a).
19. Verify that if Ω × u = 0 for all u, then Ω = 0.

14.4 Line Integrals

The concept of the integral can be extended to functions which are not defined on an interval
of the real line but on some curve in Rn . This is done by defining things in such a way that
the more general concept reduces to the earlier notion. First it is necessary to consider what
is meant by arc length.
262 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

14.4.1 Arc Length And Orientations

The application of the integral considered here is the concept of the length of a curve.
Definition 14.4.1 C is a smooth curve in Rn if there exists an interval [a, b] ⊆ R and func-
tions xi : [a, b] → R such that the following conditions hold
1. xi is continuous on [a, b].
2. xi′ exists and is continuous and bounded on [a, b], with xi′ (a) defined as the derivative
from the right,
xi (a + h) − xi (a)
lim ,
h→0+ h
and xi′ (b) defined similarly as the derivative from the left.
3. For p (t) ≡ (x1 (t) , · · · , xn (t)), t → p (t) is one to one on (a, b).
 1/2
4. |p′ (t)| ≡ ∑ni=1 |xi′ (t)|2 ̸= 0 for all t ∈ [a, b].

5. C = ∪ {(x1 (t) , · · · , xn (t)) : t ∈ [a, b]}.

The functions xi (t), defined above are giving the coordinates of a point in Rn and the
list of these functions is called a parametrization for the smooth curve. Note the natural
direction of the interval also gives a direction for moving along the curve. Such a direction
is called an orientation. The integral is used to define what is meant by the length of such a
smooth curve. Consider such a smooth curve having parametrization (x1 , · · · , xn ). Forming
a partition of [a, b], a = t0 < · · · < tn = b and letting pi = ( x1 (ti ), · · · , xn (ti ) ), you could
consider the polygon formed by lines from p0 to p1 and from p1 to p2 and from p3 to p4
etc. to be an approximation to the curve C. The following picture illustrates what is meant
by this.
p3
p1
p2

p0
Now consider what happens when the partition is refined by including more points.
You can see from the following picture that the polygonal approximation would appear to
be even better and that as more points are added in the partition, the sum of the lengths
of the line segments seems to get close to something which deserves to be defined as the
length of the curve C.
p3
p1
p2
•

p0
Thus the length of the curve is approximated by
n
∑ |p (tk ) − p (tk−1 )| .
k=1
14.4. LINE INTEGRALS 263

Since the functions in the parametrization are differentiable, it is reasonable to expect this
to be close to
n
∑ p′ (tk−1 ) (tk − tk−1 )
k=1
Rb
which is seen to be a Riemannn sum for the integral a |p′ (t)| dt and it is this integral
which is defined as the length of the curve.
Definition 14.4.2 Let p (t), t ∈ [a, b] be a parametrization for a smooth curve. Then the
length of this curve is defined as ab |p′ (t)| dt.
R

Would the same length be obtained if another parametrization were used? This is a very
important question because the length of the curve should depend only on the curve itself
and not on the method used to trace out the curve. The answer to this question is that the
length of the curve does not depend on parametrization. The proof is somewhat technical
so is given in the last section of this chapter.
Does the definition of length given above correspond to the usual definition of length
in the case when the curve is a line segment? It is easy to see that it does so by considering
two points in Rn p and q. A parametrization for the line segment joining these two points
is
fi (t) ≡ t pi + (1 − t) qi , t ∈ [0, 1] .
Using the definition of length of a smooth curve just given, the length according to this
definition is !1/2
Z 1 n

0
∑ (pi − qi )2 dt = |p − q| .
i=1
Thus this new definition which is valid for smooth curves which may not be straight line
segments gives the usual length for straight line segments.
The proof that curve length is well defined for a smooth curve contains a result which
deserves to be stated as a corollary. It is proved in Lemma 14.6.6 on Page 272 but the proof
is mathematically fairly advanced so it is presented later.
Corollary 14.4.3 Let C be a smooth curve and let f : [a, b] → C and g : [c, d] → C be two
parameterizations satisfying (1) - (5). Then g −1 ◦ f is either strictly increasing or strictly
decreasing.
Definition 14.4.4 If g −1 ◦ f is increasing, then f and g are said to be equivalent parame-
terizations and this is written as f ∼ g. It is also said that the two parameterizations give
the same orientation for the curve when f ∼ g.
When the parameterizations are equivalent, they preserve the direction of motion along
the curve, and this also shows there are exactly two orientations of the curve since either
g −1 ◦ f is increasing or it is decreasing. This is not hard to believe. In simple language, the
message is that there are exactly two directions of motion along a curve. The difficulty is
in proving this is actually the case.
Lemma 14.4.5 The following hold for ∼.
f ∼ f; (14.7)
If f ∼ g then g ∼ f ; (14.8)
If f ∼ g and g ∼ h, then f ∼ h. (14.9)
264 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

Proof: Formula (14.7) is obvious because f −1 ◦ f (t) = t so it is clearly an increasing

function. If f ∼ g then f −1 ◦ g is increasing. Now g −1 ◦ f must also be increasing because
it is the inverse of f −1 ◦ g. This verifies (14.8). To see (14.9), f −1 ◦ h = f −1 ◦ g ◦

g −1 ◦ h and so since both of these functions are increasing, it follows f −1 ◦ h is also

increasing. ■
The symbol ∼ is called an equivalence relation. If C is such a smooth curve just de-
scribed, and if f : [a, b] → C is a parametrization of C, consider g (t) ≡ f ((a + b) − t), also
a parametrization of C. Now by Corollary 14.4.3, if h is a parametrization, then if f −1 ◦ h
is not increasing, it must be the case that g −1 ◦ h is increasing. Consequently, either h ∼ g
or h ∼ f. These parameterizations, h, which satisfy h ∼ f are called the equivalence class
determined by f and those h ∼ g are called the equivalence class determined by g. These
two classes are called orientations of C. They give the direction of motion on C. You see
that going from f to g corresponds to tracing out the curve in the opposite direction.
Sometimes people wonder why it is required, in the definition of a smooth curve that
p′ (t) ̸= 0. Imagine t is time and p (t) gives the location of a point in space. If p′ (t)
is allowed to equal zero, the point can stop and change directions abruptly, producing a
pointy place in C. Here is an example.

Example 14.4.6 Graph the curve t 3 ,t 2 for t ∈ [−1, 1].

In this case, t = x1/3 and so y = x2/3 . Thus the graph of this curve looks like the picture
below. Note the pointy place. Such a curve should not be considered smooth.

So what is the thing to remember from all this? First, there are certain conditions which
must be satisfied for a curve to be smooth. These are listed above. Next, if you have any
curve, there are two directions you can move over this curve, each called an orientation.
This is illustrated in the following picture.
q q

p p

Either you move from p to q or you move from q to p.

Definition 14.4.7 A curve C is piecewise smooth if there exist points on this curve, denoted
by p0 , p1 , · · · , pn such that, denoting Cpk−1 pk the part of the curve joining pk−1 and pk , it
follows Cpk−1 pk is a smooth curve and ∪nk=1Cpk−1 pk = C. In other words, it is piecewise
smooth if it consists of a finite number of smooth curves linked together.

Note that Example 14.4.6 is an example of a piecewise smooth curve although it is not
smooth.
14.4. LINE INTEGRALS 265

14.4.2 Line Integrals And Work

Let C be a smooth curve contained in R p . A curve C is an “oriented curve” if the only
parameterizations considered are those which lie in exactly one of the two equivalence
classes, each of which is called an “orientation”. In simple language, orientation specifies
a direction over which motion along the curve is to take place. Thus, it specifies the order in
which the points of C are encountered. The pair of concepts consisting of the set of points
making up the curve along with a direction of motion along the curve is called an oriented
curve.

Definition 14.4.8 Suppose F (x) ∈ R p is given for each x ∈ C where C is a smooth ori-
ented curve and suppose x → F (x) is continuous. The mapping x → F (x) is called a
vector field. In the case that F (x) is a force, it is called a force field.

Next the concept of work done by a force field F on an object as it moves along the
curve C, in the direction determined by the given orientation of the curve will be defined.
This is new. Earlier the work done by a force which acts on an object moving in a straight
line was discussed but here the object moves over a curve. In order to define what is meant
by the work, consider the following picture.

F (x(t))

x(t)

x(t + h)
In this picture, the work done by a constant force F on an object which moves from the
point x (t) to the point x (t + h) along the straight line shown would equal F ·(x (t + h) − x (t)).
It is reasonable to assume this would be a good approximation to the work done in moving
along the curve joining x (t) and x (t + h) provided h is small enough. Also, provided h is
small,
x (t + h) − x (t) ≈ x′ (t) h
where the wriggly equal sign indicates the two quantities are close. In the notation of
Leibniz, one writes dt for h and

dW = F (x (t)) · x′ (t) dt

or in other words,
dW
= F (x (t)) · x′ (t) .
dt
Defining the total work done by the force at t = 0, corresponding to the first endpoint of
the curve, to equal zero, the work would satisfy the following initial value problem.

dW
= F (x (t)) · x′ (t) , W (a) = 0.
dt
This motivates the following definition of work.
266 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

Definition 14.4.9 Let F (x) be given above. Then the work done by this force field on an
object moving over the curve C in the direction determined by the specified orientation is
defined as
Z Z b
F ·d R ≡ F (x (t)) · x′ (t) dt
C a
where the function x is one of the allowed parameterizations of C in the given orientation
of C. In other words, there is an interval [a, b] and as t goes from a to b, x (t) moves in the
direction determined from the given orientation of the curve.
R
Theorem 14.4.10 The symbol C F ·dR, is well defined in Rthe sense that every parametriza-
tion in the given orientation of C gives the same value for C F · dR.

Proof: Suppose g : [c, d] → C is another allowed parametrization. Thus g −1 ◦ f is an

increasing function φ . Then since φ is increasing, it follows from the change of variables
formula that
Z d Z b
F (g (s)) · g ′ (s) ds = F (g (φ (t))) · g ′ (φ (t)) φ ′ (t) dt
c a
Z b Z b
d
g g −1 ◦ f (t) dt = F (f (t)) · f ′ (t) dt.



= F (f (t)) · ■
a dt a
Regardless the physical interpretation of F, this is called the line integral. When F
is interpreted as a force, the line integral measures the extent to which the motion over
the curve in the indicated direction is aided by the force. If the net effect of the force on
the object is to impede rather than to aid the motion, this will show up as the work being
negative.
Does the concept of work as defined here coincide with the earlier concept of work
when the object moves over a straight line when acted on by a constant force? If it doesn’t,
then the above is not a good definition because it will contradict earlier and more basic
constructions. Math is not like religion which often abounds in apparent contradictions.
Let p and q be two points in Rn and suppose F is a constant force acting on an
object which moves from p to q along the straight line joining these points. Then the
work done is F · (q − p). Is the same thing obtained from the above definition? Let
x (t) ≡ p+t (q − p) ,t ∈ [0, 1] be a parametrization for this oriented curve, thestraight line
in the direction from p to q. Then x′ (t) = q − p and F (x (t)) = F. Therefore, the above
definition yields
Z 1
F · (q − p) dt = F · (q − p) .
0
Therefore, the new definition adds to but does not contradict the old one. Therefore, it is
not unreasonable to use this as the definition.

Example 14.4.11 Suppose for t ∈ [0, π] the position of an object is given by r (t) = ti +
cos (2t) j + sin (2t) k. Also suppose there is a force field defined on R3 , F (x, y, z) ≡ 2xyi +
2
R
x j + k. Find C F · dR where C is the curve traced out by this object which has the
orientation determined by the direction of increasing t.

To find this line integral use the above definition and write
Z Z π
2t (cos (2t)) ,t 2 ,1 · (1, −2 sin (2t) , 2 cos (2t)) dt


F · dR =
C 0
14.4. LINE INTEGRALS 267

In evaluating this replace the x in the formula for F with t, the y in the formula for F
with cos (2t) and the z in the formula for F with sin (2t) because these are the values of
these variables which correspond to the value of t. Taking the dot product, this equals the
following integral.
Z π
2t cos 2t − 2 (sin 2t)t 2 + 2 cos 2t dt = π 2


0

Example 14.4.12 Let C denote the oriented curve obtained by r (t) = t, sint,t 3 where

the orientation is determined by increasing t for t ∈ [0, 2]. Also let F = (x, y, xz + z). Find
R
C F ·dR.
You use the definition.
Z Z 2
t, sin (t) , (t + 1)t 3 · 1, cos (t) , 3t 2 dt



F · dR =
C 0
Z 2  1251 1
= t + sin (t) cos (t) + 3 (t + 1)t 5 dt = − cos2 (2) .
0 14 2
Suppose you have a curve specified by r (s) = (x (s) , y (s) , z (s)) and it has the property
that |r ′ (s)| = 1 for all s ∈ [0, b]. Then the length of this curve for s between 0 and s1
is 0 |r (s)| ds = 0s1 1ds = s1 . This parameter is therefore called arc length because the
R s1 ′ R

length of the curve up to s equals s. Now you can always change the parameter to be arc
length.
Proposition 14.4.13 Suppose C is an oriented smooth curve parameterized by r (t) for
t ∈ [a, b]. Then letting l denote the total length of C, there exists R (s), s ∈ [0, l] another
parametrization for this curve which preserves the orientation and such that R′ (s) = 1
so that s is arc length.
Rt ′ (τ)| dτ
Prove: Let φ (t) ≡ a |r ≡ s. Then s is an increasing function of t because
ds
= φ ′ (t) = r ′ (t) > 0.
dt
Now define R (s) ≡ r φ −1 (s) . Then

r ′ φ −1 (s)


′

−1
′
′ −1
R (s) = r φ (s) φ (s) = ′ −1

r φ (s)
 R 
and so R′ (s) = 1 as claimed. R (l) = r φ −1 (l) = r φ −1 ab |r ′ (τ)| dτ = r (b) and

R (0) = r φ −1 (0) = r (a) and R delivers the same set of points in the same order as r

because dsdt > 0. ■

The arc length parameter is just like any other parameter, in so far as considerations
of line integrals are concerned, because it was shown above that line integrals are inde-
pendent of parametrization. However, when things are defined in terms of the arc length
parametrization, it is clear they depend only on geometric properties of the curve itself and
for this reason, the arc length parametrization is important in differential geometry.
Definition 14.4.14 As to piecewise smooth curves, recall these are just smooth curves
joined together at a succession of points p1 , p2 , · · · , pm . If C is such a curve which goes
from p1 then to p2 then to p3 etc. one defines
Z Z Z Z
F ·d R ≡ F ·d R+ F ·d R + · · · + F ·dR
C Cp1 p2 Cp2 p3 Cp(n−1)n
268 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

14.4.3 Another Notation For Line Integrals

Definition 14.4.15 Let F (x, y, z) = (P (x,
R
y, z) , Q (x, y, z) , R (x, y, z)) and let C be an ori-
ented curve. Then another way to write C F ·dR is
Z
Pdx + Qdy + Rdz
C

This last is referred to as the integral of a differential form, Pdx + Qdy + Rdz. The
study of differential forms is important. Formally, d R = (dx, dy, dz) and so the integrand
in the above is formally F ·dR. Other occurrences of this notation are handled similarly in
2 or higher dimensions.

14.5 Exercises
2 √
 
1. Let r (t) = ln (t) , t2 , 2t for t ∈ [1, 2]. Find the length of this curve.

2 3/2


2. Let r (t) = 3t ,t,t for t ∈ [0, 1]. Find the length of this curve.

3. Let r (t) = (t, cos (3t) , sin (3t)) for t ∈ [0, 1]. Find the length of this curve.
4. Suppose for t ∈ [0, π] the position of an object is given by r (t) = ti + cos (2t) j +
sin (2t) k. Also suppose there is a force

field defined on R3 , which is given by the
2 2
R
formula F (x, y, z) ≡ 2xyi + x + 2zy j + y k. Find the work C F · dR where C is
the curve traced out by this object having the orientation determined by the direction
of increasing t.
5. In the following, a force field is specified followed by the parametrization of a curve.
Find the work.

(a) F = (x, y, z) , r (t) = t,t 2 ,t + 1 ,t ∈ [0, 1]

(b) F = (x − y, y + z, z) , r (t) = (cos (t) ,t, sin (t)) ,t ∈ [0, π]

(c) F = x2 , y2 , z + x , r (t) = t, 2t,t + t 2 ,t ∈ [0, 1]

(d) F = (z, y, x) , r (t) = t 2 , 2t,t ,t ∈ [0, 1]

6. The curve consists of straight line segments which go from (0, 0, 0) to (1, 1, 1) and
finally to (1, 2, 3). Find the work done if the force field is

(a) F = 2xy, x2 + 2y, 1

(b) F = yz2 , xz2 , 2xyz + 1

(c) F = (cos x, − sin y, 1)

(d) F = 2x sin y, x2 cos y, 1

7. ∗ Read ahead about the gradient in Definition 16.3.5 on Page 290. Show the vector
fields in the preceding problems are respectively

∇ x2 y + y2 + z , ∇ xyz2 + z , ∇ (sin x + cos y + z − 1)

14.5. EXERCISES 269

, and ∇ x2 sin y + z . Thus each of these vector fields is of the form ∇ f where f is a

function of three variables. For each f in the above, compute f (1, 2, 3) − f (0, 0, 0)
and compare with your solutions to the above line integrals. You should get the
same thing from f (1, 2, 3) − f (0, 0, 0) . This is not a coincidence and will be fully
discussed later. Such vector fields are called conservative.
8. Here is a vector field y, x + z2 , 2yz and here is the parametrization


R
of a curve C.
R (t) = (cos 2t, 2 sin 2t,t) where t goes from 0 to π/4. Find C F · dR.
9. If f and g are both increasing functions, show that f ◦ g is an increasing function
also. Assume anything you like about the domains of the functions.
10. Suppose for t ∈ [0, 3] the position of an object is given by r (t) = ti + tj + tk.
Also suppose there is a force field defined on R3 , F (x, y, z) ≡ yzi + xzj + xyk. Find
R
C F · dR where C is the curve traced out by this object which has the orientation de-
termined by the direction of increasing t. Repeat the problem for r (t) = ti+t 2 j +tk.
11. Suppose for t ∈ [0, 1] the position of an object is given by r (t) = ti + tj + tk.
Also suppose there is a force field defined on R3 , F (x, y, z) ≡ zi + xzj + xyk. Find
R
C F · dR where C is the curve traced out by this object which has the orientation de-
termined by the direction of increasing t. Repeat the problem for r (t) = ti+t 2 j +tk.
12. Let F (x, y, z) be a given force field and suppose it acts on an object having mass m
on a curve with parametrization, (x (t) , y (t) , z (t)) for t ∈ [a, b]. Show directly that
the work done equals the difference in the kinetic energy. Hint:
Z b
F (x (t) , y (t) , z (t)) · x′ (t) , y′ (t) , z′ (t) dt =


a
Z b
m x′′ (t) , y′′ (t) , z′′ (t) · x′ (t) , y′ (t) , z′ (t) dt,



a
etc.
13. Suppose for t ∈ [0, 2π] the position of an object is given by

r (t) = 2ti + cos (t) j + sin (t) k.

Also suppose there is a force field defined on R3 ,

F (x, y, z) ≡ 2xyi + x2 + 2zy j + y2 k.

R
Find the work C F · dR where C is the curve traced out by this object which has the
orientation determined by the direction of increasing t.
14. Here is a vector field y, x2 + z, 2yz and here is the parametrization


R
of a curve C.
R (t) = (cos 2t, 2 sin 2t,t) where t goes from 0 to π/4. Find C F · dR.
15. Suppose for t ∈ [0, 1] the position of an object is given by r (t) = ti + tj + tk.
Also suppose there is a force field defined on R3 , F (x, y, z) ≡ yzi + xzj + xyk. Find
R
C F · dR where C is the curve traced out by this object which has the orientation de-
termined by the direction of increasing t. Repeat the problem for r (t) = ti+t 2 j +tk.
You should get the same answer in this case. This is because the vector field happens
to be conservative. (More on this later.)
270 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

14.6 Independence Of Parametrization∗

Recall that if p (t) : t ∈ [a, b] was a parametrization of a smooth curve C, the length of
C is defined as ab |p′ (t)| dt. If some other parametrization were used to trace out C, would
R

the same answer be obtained? To answer this question in a satisfactory manner requires
some hard calculus.

14.6.1 Hard Calculus

Recall Theorem 13.4.1 about continuity and convergent sequences. It said roughly that
a function f is continuous at x if and only if whenever xk → x, then f (xk ) → f (x) .
Also recall the following Lemma from Volume 1, whose proof is summarized below for
convenience.

Lemma 14.6.1 Let φ : [a, b] → R be a continuous function and suppose φ is 1 − 1 on

(a, b). Then φ is either strictly increasing or strictly decreasing on [a, b]. Furthermore,
φ −1 is continuous.

Proof: First it is shown that φ is either strictly increasing or strictly decreasing on (a, b).
If φ is not strictly decreasing on (a, b), then there exists x1 < y1 , x1 , y1 ∈ (a, b) such that

(φ (y1 ) − φ (x1 )) (y1 − x1 ) > 0.

If for some other pair of points x2 < y2 with x2 , y2 ∈ (a, b), the above inequality does not
hold, then since φ is 1 − 1,

(φ (y2 ) − φ (x2 )) (y2 − x2 ) < 0.

Let xt ≡ tx1 + (1 − t) x2 and yt ≡ ty1 + (1 − t) y2 . It follows that xt < yt for all t ∈ [0, 1].
Now define
h (t) ≡ (φ (yt ) − φ (xt )) (yt − xt ) .
Then h (0) < 0, h (1) > 0 but by assumption, h (t) ̸= 0 for any t ∈ (0, 1) , a contradiction.
This property of being either strictly increasing or strictly decreasing on (a, b) carries
over to [a, b] by the continuity of φ .
It only remains to verify φ −1 is continuous. If not, there exists sn → s where sn and s
are points of φ ([a, b]) but φ −1 (sn ) − φ −1 (s) ≥ ε. By sequential compactness of [a, b] ,
there is a subsequence, still denoted by n, such that φ −1 (sn ) − t1 → 0. Thus sn → φ (t1 ),
so s = φ (t1 ), and t1 = φ −1 (s) , a contradiction. ■

Corollary 14.6.2 Let f : (a, b) → R be one to one and continuous. Then f (a, b) is an open
interval (c, d) and f −1 : (c, d) → (a, b) is continuous.
14.6. INDEPENDENCE OF PARAMETRIZATION∗ 271

Proof: Since f is either strictly increasing or strictly decreasing, it follows that f (a, b)
is an open interval (c, d). Assume f is decreasing. Now let x ∈ (a, b). Why is f −1 is contin-
uous at f (x)? Since f is decreasing, if f (x) < f (y), then y ≡ f −1 ( f (y)) < x ≡ f −1 ( f (x))
and so f −1 is also decreasing. Let ε > 0 be given. Let ε > η > 0 and (x − η, x + η) ⊆ (a, b).
Then f (x) ∈ ( f (x + η) , f (x − η)). Let

δ = min ( f (x) − f (x + η) , f (x − η) − f (x)) .

Then if | f (z) − f (x)| < δ , it follows

z ≡ f −1 ( f (z)) ∈ (x − η, x + η) ⊆ (x − ε, x + ε)

which implies
f −1 ( f (z)) − x = f −1 ( f (z)) − f −1 ( f (x)) < ε.
This proves the theorem in the case where f is strictly decreasing. The case where f is
increasing is similar. ■

Theorem 14.6.3 Let f : [a, b] → R be continuous and one to one. Suppose f ′ (x1 ) exists

′
for some x1 ∈ [a, b] and f ′ (x1 ) ̸= 0. Then f −1 ( f (x1 )) exists and is given by the formula

′
f −1 ( f (x1 )) = f ′ (x1 ) .
1

Proof: By Lemma 14.6.1 f is either strictly increasing or strictly decreasing and f −1 is

continuous on [a, b]. Therefore there exists η > 0 such that if 0 < | f (x1 ) − f (x)| < η, then

0 < |x1 − x| = f −1 ( f (x1 )) − f −1 ( f (x)) < δ

where δ is small enough that for 0 < |x1 − x| < δ ,

x − x1 1
− < ε.
f (x) − f (x1 ) f ′ (x1 )
It follows that if 0 < | f (x1 ) − f (x)| < η,

f −1 ( f (x)) − f −1 ( f (x1 )) 1 x − x1 1
− ′ = − <ε
f (x) − f (x1 ) f (x1 ) f (x) − f (x1 ) f ′ (x1 )
Therefore, since ε > 0 is arbitrary,

f −1 (y) − f −1 ( f (x1 )) 1
lim = ′ . ■
y→ f (x1 ) y − f (x1 ) f (x1 )
The following obvious corollary comes from the above by not bothering with end
points.

Corollary 14.6.4 Let f : (a, b) → R be continuous and one to one. Suppose f ′ (x1 ) exists

′
for some x1 ∈ (a, b) and f ′ (x1 ) ̸= 0. Then f −1 ( f (x1 )) exists and is given by the formula
′
f −1 ( f (x1 )) = f ′ (x1 ) .


1

Proof: From the definition of the derivative and continuity of f −1 ,

f −1 ( f (x)) − f −1 ( f (x1 )) x − x1 1
lim = lim = ′ . ■
f (x)→ f (x1 ) f (x) − f (x1 ) x→x1 f (x) − f (x1 ) f (x1 )
272 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE

14.6.2 Independence Of Parametrization

Theorem 14.6.5 Let φ : [a, b] → [c, d] be one to one and suppose φ ′ exists and is continuous
on [a, b]. Then if f is a continuous function defined on [c, d] which is Riemannn integrable1 ,
Z d Z b
f (s) ds = f (φ (t)) φ ′ (t) dt
c a

Proof: Let F ′ (s) = f (s). (For example, let F (s) = as f (r) dr.) Then the first integral
R

equals F (d) − F (c) by the fundamental theorem of calculus. Since φ is one to one, it
follows from Lemma 14.6.1 above that φ is either strictly increasing or strictly decreasing.
Suppose φ is strictly decreasing. Then φ (a) = d and φ (b) = c. Therefore, φ ′ ≤ 0 and the
second integral equals
Z b Z a
d
− f (φ (t)) φ ′ (t) dt = (F (φ (t))) dt = F (φ (a)) − F (φ (b)) = F (d) − F (c) .
a b dt
The case when φ is increasing is similar but easier. ■

Lemma 14.6.6 Let f : [a, b] → C, g : [c, d] → C be parameterizations of a smooth curve

which satisfy conditions (1) - (5). Then ϕ (t) ≡ g −1 ◦ f (t) is 1 − 1 on (a, b), continuous on
[a, b], and either strictly increasing or strictly decreasing on [a, b].

Proof: It is obvious φ is 1 − 1 on (a, b) from the conditions f and g satisfy. It only

remains to verify continuity on [a, b] because then the final claim follows from Lemma
14.6.1. If φ is not continuous on [a, b], then there exists a sequence, {tn } ⊆ [a, b] such
that tn → t but φ (tn ) fails to converge to φ (t). Therefore, for some ε > 0, there exists a
subsequence, still denoted by n such that |φ (tn ) − φ (t)| ≥ ε. By sequential compactness
of [c, d], (See Theorem 13.3.8 on Page 240.) there is a further subsequence, still denoted
by n, such that {φ (tn )} converges to a point s, of [c, d] which is not equal to φ (t). Thus
g −1 ◦ f (tn ) → s while tn → t. Therefore, the continuity of f and g imply f (tn ) → g (s) and
f (tn ) → f (t). Thus, g (s) = f (t), so s = g −1 ◦ f (t) = φ (t), a contradiction. Therefore, φ
is continuous as claimed. ■

Theorem 14.6.7 The length of a smooth curve is not dependent on which parametrization
is used.

Proof: Let C be the curve and suppose f : [a, b] → C and g : [c, d] → C both satisfy
conditions (1) - (5). Is it true that ab f ′ (t) dt = cd |g ′ (s)| ds?
R R

Let φ (t) ≡ g −1 ◦f (t) for t ∈ [a, b]. I want to show that φ is C1 on an interval of the form
[a + δ , b − δ ]. By the above lemma, φ is either strictly increasing or strictly decreasing on
[a, b]. Suppose for the sake of simplicity that it is strictly increasing. The decreasing case
is handled similarly.
Let s0 ∈ φ ([a + δ , b − δ ]) ⊂ (c, d). Then by assumption 4 for smooth curves, g′i (s0 ) ̸= 0
for some i. By continuity of g′i , it follows g′i (s) ̸= 0 for all s ∈ I where I is an open interval
contained in [c, d] which contains s0 . It follows from the mean value theorem that on this
interval gi is either strictly increasing or strictly decreasing. Therefore, J ≡ gi (I) is also an
open interval and you can define a differentiable function hi : J → I by

hi (gi (s)) = s.
1 Recall that all continuous functions of this sort are Riemann integrable.
14.6. INDEPENDENCE OF PARAMETRIZATION∗ 273

This implies that for s ∈ I,

1
h′i (gi (s)) = . (14.10)
g′i (s)
Now letting s = φ (t) for s ∈ I, it follows t ∈ J1 , an open interval. Also, for s and t related
this way, f (t) = g (s) and so in particular, for s ∈ I, gi (s) = fi (t) . Consequently,

s = hi (gi (s)) = hi ( fi (t)) = φ (t)

and so, for t ∈ J1 ,

fi′ (t)
φ ′ (t) = h′i ( fi (t)) fi′ (t) = h′i (gi (s)) fi′ (t) = ′ (14.11)
gi (φ (t))

which shows that φ ′ exists and is continuous on J1 , an open interval containing φ −1 (s0 ).
′
Since s0 is arbitrary, this shows
φ exists on [a + δ , b − δ ] and is continuous there.
Now f (t) = g◦ g ◦ f (t) = g (φ (t)), and it was just shown that φ ′ is a continuous
−1

function on [a − δ , b + δ ]. It follows from the chain rule, f ′ (t) = g ′ (φ (t)) φ ′ (t) and so, by
Theorem 14.6.5,
Z φ (b−δ ) Z b−δ Z b−δ
′ ′ ′
g (s) ds = g (φ (t)) φ (t) dt = f ′ (t) dt.
φ (a+δ ) a+δ a+δ

Now using the continuity of φ , g ′ , and f ′ on [a, b] and letting δ → 0+ in the above, yields
Z d Z b
g ′ (s) ds = f ′ (t) dt. ■
c a
274 CHAPTER 14. VECTOR VALUED FUNCTIONS OF ONE VARIABLE
Chapter 15

Motion On A Space Curve

15.1 Space Curves

A fly buzzing around the room, a person riding a roller coaster, and a satellite orbiting the
earth all have something in common. They are moving over some sort of curve in three
dimensions.
Denote by R (t) the position vector of the point on the curve which occurs at time t.
Assume that R′ , R′′ exist and are continuous. Thus R′ = v, the velocity and R′′ = a is
defined as the acceleration.
z
R(t)

x
Lemma 15.1.1 Define T (t) ≡ R′ (t) / R′ (t) . Then |T (t)| = 1 and if T ′ (t) ̸= 0, then
there exists a unit vector N (t) perpendicular to T (t) and a scalar valued function κ (t),
with T ′ (t) = κ (t) |v| N (t).

Proof: It follows from the definition that |T | = 1. Therefore, T · T = 1 and so, upon
differentiating both sides,

T ′ · T + T · T ′ = 2T ′ · T = 0.

Therefore, T ′ is perpendicular to T . Let N (t) T ′ ≡ T ′ . Note that if T ′ = 0, you could

let N (t) be any unit vector. Then letting κ (t) be defined such that T ′ ≡ κ (t) |v (t)|, it
follows
T ′ (t) = T ′ (t) N (t) = κ (t) |v (t)| N (t) . ■

Definition 15.1.2 The vector T (t) is called the unit tangent vector and the vector N (t) is
called the principal normal. The function κ (t) in the above lemma is called the curvature.
The radius of curvature is defined as ρ = 1/κ. The plane determined by the two vectors T

275
276 CHAPTER 15. MOTION ON A SPACE CURVE

and N in the case where T ′ ̸= 0 is called the osculating1 plane. It identifies a particular
plane which is in a sense tangent to this space curve.

The important thing about this is that it is possible to write the acceleration as the sum
of two vectors, one perpendicular to the direction of motion and the other in the direction
of motion.

Theorem 15.1.3 For R (t) the position vector of a space curve, the acceleration is given
by the formula
d |v|
a= T + κ |v|2 N ≡ aT T + aN N . (15.1)
dt
Furthermore, a2T + a2N = |a|2 .

Proof:
dv d d d |v| d |v|
R′ = (|v| T ) = T + |v| T ′ = T + |v|2 κN.


a= =
dt dt dt dt dt
This proves the first part.
For the second part,

|a|2 = (aT T + aN N ) · (aT T + aN N )

= a2T T · T + 2aN aT T · N + a2N N · N = a2T + a2N

because T · N = 0. ■
From 15.1 and the geometric properties of the cross product,

a × v = κ |v|2 N × v

Hence, using the geometric description of the cross product again using that the angle
between N and T is 90◦ ,
|a × v| |v × a|
|a × v| = κ |v|2 |v| , κ = 3
= (15.2)
|v| |v|3
Finally, it is good to point out that the curvature is a property of the curve itself, and
does not depend on the parametrization of the curve. If the curve is given by two different
vector valued functions R (t) and R (τ), then from the formula above for the curvature,

T ′ (t) d T dτ
dτ dt
dT
dτ
κ (t) = = d R dτ
= dR
≡ κ (τ) .
|v (t)| dτ dt dτ

From this, it is possible to give an important formula from physics. Suppose an object
orbits a point at constant speed v. In the above notation, |v| = v. What is the centripetal
acceleration of this object? You may know from a physics class that the answer is v2 /r
where r is the radius. This follows from the above quite easily. First, what is the curvature
of a circle of radius r? A parameterization of such a curve is

R (t) = (r cost, r sint)

1 To osculate means to kiss. Thus this plane could be called the kissing plane. However, that does not sound

formal enough so we call it the osculating plane.

15.1. SPACE CURVES 277

Thus using 15.2 and this parametrization,

i j k
v×a = −r sint r cost 0 = kr2
−r cost −r sint 0
Thus
r2 1
κ= =
r3 r
Since v is constant, it follows from 15.1 that
1 2 1
a= |v| N = v2 N
r r
2


Example 15.1.4 Let R (t) = cos (t) ,t,t for t ∈ [0, 3]. Find the speed, velocity, curva-
ture, and write the acceleration in terms of normal and tangential components.
First of all, v (t) = (− sint, 1, 2t) and so the speed is given by
q
|v| = sin2 (t) + 1 + 4t 2 .
Therefore, q 
d sin (t) cos (t) + 4t
aT = sin2 (t) + 1 + 4t 2 = p .
dt (2 + 4t 2 − cos2 t)
It remains to find aN . To do this, you can find the curvature first if you like.
a (t) = R′′ (t) = (− cost, 0, 2) .
Then
q
|(− cost, 0, 2) × (− sint, 1, 2t)| 4 + (−2 sin (t) + 2 (cos (t))t)2 + cos2 (t)
κ= q 3 = q 3
2
sin (t) + 1 + 4t 2 sin2 (t) + 1 + 4t 2

Then aN = κ |v|2
q
4 + (−2 sin (t) + 2 (cos (t))t)2 + cos2 (t)
sin2 (t) + 1 + 4t 2


= q 3
sin2 (t) + 1 + 4t 2
q
4 + (−2 sin (t) + 2 (cos (t))t)2 + cos2 (t)
= q .
sin2 (t) + 1 + 4t 2

You can observe the formula a2N + a2T = |a|2 holds. Indeed a2N + a2T =

q 2 !2
4 + (−2 sin (t) + 2 (cos (t))t)2 + cos2 (t) sin (t) cos (t) + 4t
 q  + p
2
sin (t) + 1 + 4t 2 (2 + 4t 2 − cos2 t)

4 + (−2 sint + 2 (cost)t)2 + cos2 t (sint cost + 4t)2

= + = cos2 t + 4 = |a|2
sin2 t + 1 + 4t 2 2 + 4t 2 − cos2 t
278 CHAPTER 15. MOTION ON A SPACE CURVE

15.1.1 Some Simple Techniques

Recall the formula for acceleration is

a = aT T + aN N (15.3)

where aT = d|dtv| and aN = κ |v|2 . Of course one way to find aT and aN is to just find
|v| , d|dtv| and κ and plug in. However, there is another way which might be easier. Take the
dot product of both sides with T. This gives,

a · T = aT T · T + aN N · T = aT .

Thus
a = (a · T ) T + aN N
and so
a − (a · T ) T = aN N (15.4)
and taking norms of both sides,

|a − (a · T ) T | = aN .

Also from (15.4),

a − (a · T ) T aN N
= = N.
|a − (a · T ) T | aN |N |
Also recall
|a × v|
κ= , a2T + a2N = |a|2
|v|3
This is usually easier than computing T ′ / T ′ . To illustrate the use of these simple obser-
vations, consider the example worked above which was fairly messy. I will make it easier
by selecting a value of t and by using the above simplifying techniques.

Example 15.1.5 Let R (t) = cos (t) ,t,t 2 for t ∈ [0, 3]. Find the speed, velocity, curva-

ture, and write the acceleration in terms of normal and tangential components when t = 0.
Also find N at the point where t = 0.

First I need to find the velocity and acceleration. Thus

v = (− sint, 1, 2t) , a = (− cost, 0, 2)

and consequently, T = √(− 2sint,1,2t) 2 . When t = 0, this reduces to

sin (t)+1+4t

v (0) = (0, 1, 0) , a = (−1, 0, 2) , |v (0)| = 1, T = (0, 1, 0) .

Then the tangential component of acceleration when t = 0 is

aT = (−1, 0, 2) · (0, 1, 0) = 0
√ √
Now |a|2 = 5 and so aN = 5 because a2T + a2N = |a|2 . Thus 5 = κ |v (0)|2 = κ · 1 = κ.
Next lets find N . From a = aT T + aN N it follows
√
(−1, 0, 2) = 0 · T + 5N
15.2. GEOMETRY OF SPACE CURVES∗ 279

and so
1
N = √ (−1, 0, 2) .
5
This was pretty easy.
Example 15.1.6 Find a formula for the curvature of the curve given by the graph of y =
f (x) for x ∈ [a, b]. Assume whatever you like about smoothness of f .
You need to write this as a parametric curve. This is most easily accomplished by letting
t = x. Thus a parametrization is (t, f (t) , 0) : t ∈ [a, b] . Then you can use the formula given
above. The acceleration is (0, f ′′ (t) , 0) and the velocity is (1, f ′ (t) , 0). Therefore,
a × v = 0, f ′′ (t) , 0 × 1, f ′ (t) , 0 = 0, 0, − f ′′ (t) .

Therefore, the curvature is given by

|a × v| | f ′′ (t)|
3
= 3/2 .
|v| ′
1 + f (t) 2

Sometimes curves do not come to you parametrically. This is unfortunate when it

occurs but you can sometimes find a parametric description of such curves. It should be
emphasized that it is only sometimes when you can actually find a parametrization. General
systems of nonlinear equations cannot be solved using algebra.
Example 15.1.7 Find a parametrization for the intersection of the surfaces
y + 3z = 2x2 + 4 and y + 2z = x + 1.
You need to solve for x and y in terms of x. This yields
z = 2x2 − x + 3, y = −4x2 + 3x − 5.
Therefore, letting t = x, the parametrization is
(x, y, z) = t, −4t 2 − 5 + 3t, −t + 3 + 2t 2 .

Example 15.1.8 Find a parametrization for the straight line joining (3, 2, 4) and (1, 10, 5).
(x, y, z) = (3, 2, 4) +t (−2, 8, 1) = (3 − 2t, 2 + 8t, 4 + t) where t ∈ [0, 1]. Note where this
came from. The vector (−2, 8, 1) is obtained from (1, 10, 5) − (3, 2, 4). Now you should
check to see this works.

15.2 Geometry Of Space Curves∗

If you are interested in more on space curves, you should read this section. Otherwise,
proceed to the exercises. Denote by R (s) the function which takes s to a point on this curve
where s is arc length. Thus R (s) equals the point on the curve which occurs when you have
traveled a distance of s along the curve from one end. This is known as the parametrization
of the curve in terms of arc length. Note also that it incorporates an orientation on the curve
because there are exactly two ends you could begin measuring length from. In this section,
assume anything about smoothness and continuity to make the following manipulations
valid. In particular, assume that R′ exists and is continuous.
280 CHAPTER 15. MOTION ON A SPACE CURVE

Lemma 15.2.1 Define T (s) ≡ R′ (s). Then |T (s)| = 1 and if T ′ (s) ̸= 0, then there exists
a unit vector N (s) perpendicular to T (s) and a scalar valued function κ (s) with T ′ (s) =
κ (s) N (s).
Proof: First, s = 0s R′ (r) dr because of the definition of arc length. Therefore, from
R

the fundamental theorem of calculus, 1 = R′ (s) = |T (s)|. Therefore, T · T = 1 and so

upon differentiating this on both sides, yields T ′ · T + T · T ′ = 0 which shows T · T ′ = 0.
Therefore, the vector T ′ is perpendicular to the vector T . In case T ′ (s) ̸= 0, let N (s) =
T ′ (s)
|T ′ (s)|
and so T ′ (s) = T ′ (s) N (s), showing the scalar valued function is κ (s) = T ′ (s) .
■
The radius of curvature is defined as ρ = κ1 . Thus at points where there is a lot of
curvature, the radius of curvature is small and at points where the curvature is small, the
radius of curvature is large. The plane determined by the two vectors T and N is called
the osculating plane. It identifies a particular plane which is in a sense tangent to this space
curve. In the case where T ′ (s) = 0 near the point of interest, T (s) equals a constant and
so the space curve is a straight line which it would be supposed has no curvature. Also, the
principal normal is undefined in this case. This makes sense because if there is no curving
going on, there is no special direction normal to the curve at such points which could be
distinguished from any other direction normal to the curve. In the case where T ′ (s) = 0,
κ (s) = 0 and the radius of curvature would be considered infinite.
Definition 15.2.2 The vector T (s) is called the unit tangent vector and the vector N (s) is
called the principal normal. The function κ (s) in the above lemma is called the curvature.
When T ′ (s) ̸= 0 so the principal normal is defined, the vector B (s) ≡ T (s) × N (s) is
called the binormal.
The binormal is normal to the osculating plane and B ′ tells how fast this vector changes.
Thus it measures the rate at which the curve twists.
Lemma 15.2.3 Let R (s) be a parametrization of a space curve with respect to arc length
and let the vectors T, N, and B be as defined above. Then B ′ = T × N ′ and there exists
a scalar function τ (s) such that B ′ = τN.
Proof: From the definition of B = T × N, and you can differentiate both sides and get
B ′ = T ′ × N + T × N ′ . Now recall that T ′ is a multiple called curvature multiplied by N
so the vectors T ′ and N have the same direction, so B ′ = T × N ′ . Therefore, B ′ is either
zero or is perpendicular to T. But also, from the definition of B, B is a unit vector and so
B (s) · B (s) = 1. Differentiating this, B ′ (s) · B (s) + B (s) · B ′ (s) = 0 showing that B ′ is
perpendicular to B also. Therefore, B ′ is a vector which is perpendicular to both vectors
T and B and since this is in three dimensions, B ′ must be some scalar multiple of N , and
this multiple is called τ. Thus B ′ = τN as claimed. ■
Lets go over this last claim a little more. The following situation is obtained. There
are two vectors T and B which are perpendicular to each other and both B ′ and N are
perpendicular to these two vectors, hence perpendicular to the plane determined by them.
Therefore, B ′ must be a multiple of N. Take a piece of paper, draw two unit vectors on it
which are perpendicular. Then you can see that any two vectors which are perpendicular to
this plane must be multiples of each other.
The scalar function τ is called the torsion. In case T ′ = 0, none of this is defined
because in this case there is not a well defined osculating plane. The conclusion of the
following theorem is called the Serret Frenet formulas.
15.2. GEOMETRY OF SPACE CURVES∗ 281

Theorem 15.2.4 (Serret Frenet) Let R (s) be the parametrization with respect to arc length
of a space curve and T (s) = R′ (s) is the unit tangent vector. Suppose T ′ (s) ̸= 0 so the
′ (s)
principal normal N (s) = |T T ′ (s)|
is defined. The binormal is the vector B ≡ T × N so
T, N, B forms a right handed system of unit vectors each of which is perpendicular to
every other. Then the following system of differential equations holds in R9 .

B ′ = τN, T ′ = κN, N ′ = −κT − τB

where κ is the curvature and is nonnegative and τ is the torsion.

Proof: κ ≥ 0 because κ = T ′ (s) . The first two equations are already established.
To get the third, note that B × T = N which follows because T, N, B is given to form a
right handed system of unit vectors each perpendicular to the others. (Use your right hand.)
Now take the derivative of this expression. thus

N ′ = B ′ × T + B × T ′ = τ N × T + κB × N.

Now recall again that T, N, B is a right hand system. Thus

N × T = −B, B × N = −T.

This establishes the Frenet Serret formulas. ■

This is an important example of a system of differential equations in R9 . It is a re-
markable result because it says that from knowledge of the two scalar functions τ and κ,
and initial values for B, T, and N when s = 0 you can obtain the binormal, unit tangent,
and principal normal vectors. It is just the solution of an initial value problem although
this is for a vector valued rather than scalar valued function. Having done this, you can
reconstruct theR entire space curve starting at some point R0 because R′ (s) = T (s) and so
R (s) = R0 + 0s T (r) dr.
The vectors B, T, and N are vectors which are functions of position on the space curve.
Often, especially in applications, you deal with a space curve which is parameterized by a
function of t where t is time. Thus a value of t would correspond to a point on this curve and
you could let B (t) , T (t) , and N (t) be the binormal, unit tangent, and principal normal at
this point of the curve. The following example is typical.

Example 15.2.5 Given the circular helix, R (t) = (a cost) i + (a sint) j + (bt) k, find the
arc length s (t), the unit tangent vector T (t), the principal normal N (t) , the binormal
B (t), the curvature κ (t), and the torsion, τ (t). Here t ∈ [0, T ].
R √  √ 
The arc length is s (t) = 0t a2 + b2 dr = a2 + b2 t. Now the tangent vector is
obtained using the chain rule as

dR dR dt 1 1
T= = =√ R′ (t) = √ ((−a sint) i + (a cost) j + bk)
ds dt ds 2
a +b 2 a + b2
2

The principal normal:

dT dT dt 1
= = ((−a cost) i + (−a sint) j + 0k)
ds dt ds a2 + b2
282 CHAPTER 15. MOTION ON A SPACE CURVE

and so
dT dT
N= / = − ((cost) i + (sint) j)
ds ds
The binormal:

i j k
1 1
B=√ −a sint a cost b =√ ((b sint) i− b costj + ak)
a2 + b2 a + b2
2
− cost − sint 0
r 2  2
dT a cost a sint a
Now the curvature κ (t) = ds = a2 +b2
+ a2 +b2
= a2 +b2
. Note the curvature
is constant in this example. The final task is to find the torsion. Recall that B ′ = τN where
the derivative on B is taken with respect to arc length. Therefore, remembering that t is a
function of s,
1 dt 1
B ′ (s) = √ ((b cost) i+ (b sint) j) = ((b cost) i+ (b sint) j)
a2 + b2 ds a2 + b2
= τ (− (cost) i − (sint) j) = τN

and it follows −b/ a2 + b2 = τ.

An important application of the usefulness of these ideas involves the decomposition

of the acceleration in terms of these vectors of an object moving over a space curve.

Corollary 15.2.6 Let R (t) be a space curve and denote by v (t) the velocity, v (t) = R′ (t),
let v (t) ≡ |v (t)| denote the speed, and let a (t) denote the acceleration. Then v = vT and
a = dv 2
dt T + κv N.

Proof: T = ddsR = ddtR ds dt dt

. Also, s = 0t v (r) dr and so ds
R
= v ds dt = v which implies
dt 1
ds = v . Therefore, T = v/v which implies v = vT as claimed.
Now the acceleration is just the derivative of the velocity and so by the Serrat Frenet
formulas,
dv dT dv dT dv
a = T +v = T +v v = T + v2 κN
dt dt dt ds dt
Note how this decomposes the acceleration into a component tangent to the curve and one
′ |T ′ |
which is normal to it. Also note that from the above, v T ′ T|T(t) 2
′ | = v κN and so v =κ
T ′ (t)
and N = |T ′ |
. ■

15.3 Exercises
1. Find a parametrization for the intersection of the planes 2x + y + 3z = −2 and 3x −
2y + z = −4.
2. Find a parametrization for the intersection of the plane 3x + y + z = −3 and the
circular cylinder x2 + y2 = 1.
3. Find a parametrization for the intersection of the plane 4x + 2y + 3z = 2 and the
elliptic cylinder x2 + 4z2 = 9.
15.3. EXERCISES 283

4. Find a parametrization for the straight line joining (1, 2, 1) and (−1, 4, 4).
5. Find a parametrization for the intersection of the surfaces 3y + 3z = 3x2 + 2 and
3y + 2z = 3.
6. Find a formula for the curvature of the curve y = sin x in the xy plane.
7. An object moves over the curve (t, et , at) where t ∈ R and a is a positive constant.
Find the value of t at which the normal component of acceleration is largest if there
is such a point.
8. Find a formula for the curvature of the space curve in R2 , (x (t) , y (t)).
9. An object moves over the helix, (cos 3t, sin 3t, 5t). Find the normal and tangential
components of the acceleration of this object as a function of t and write the acceler-
ation in the form aT T + aN N .
10. An object moves over the helix, (cost, sint,t). Find the normal and tangential com-
ponents of the acceleration of this object as a function of t and write the acceleration
in the form aT T + aN N .
11. An object moves in R3 according to the formula cos 3t, sin 3t,t 2 . Find the normal

and tangential components of the acceleration of this object as a function of t and

write the acceleration in the form aT T + aN N .
12. An object moves over the helix, (cost, sint, 2t). Find the osculating plane at the point
of the curve corresponding to t = π/4.
13. An object moves over a circle of radius r according to the formula

r (t) = (r cos (ωt) , r sin (ωt))

where v = rω. Show that the speed of the object is constant and equals to v. Tell
why aT = 0 and find aN , N.
14. Suppose |R (t)| = c where c is a constantR (t). Show the velocity, R′ (t) is always
perpendicular to R (t).
15. An object moves in three dimensions and the only force on the object is a central
force. This means that if r (t) is the position of the object, a (t) = k (r (t)) r (t) where
k is some function. Show that if this happens, then the motion of the object must be
in a plane. Hint: First argue that a × r = 0. Next show that (a × r) = (v × r)′ .
Therefore, (v × r)′ = 0. Explain why this requires v × r = c for some vector c
which does not depend on t. Then explain why c · r = 0. This implies the motion is
in a plane. Why? What are some examples of central forces?
√ 
16. Let R (t) = (cost) i + (cost) j + 2 sint k. Find the arc length, s as a function of
the parameter t, if t = 0 is taken to correspond to s = 0.
17. Let R (t) = 2i + (4t + 2) j + 4tk. Find the arc length, s as a function of the parameter
t, if t = 0 is taken to correspond to s = 0.
√
18. Let R (t) = e5t i+e−5t j +5 2tk. Find the arc length, s as a function of the parameter
t, if t = 0 is taken to correspond to s = 0.
284 CHAPTER 15. MOTION ON A SPACE CURVE

19. Consider the curve obtained from the graph of y = f (x). Find a formula for the
curvature.
20. Consider the curve in the plane y = ex . Find the point on this curve at which the
curvature is a maximum.

21. An object moves along the x axis toward (0, 0) and then along the curve y = x2 in
the direction of increasing x at constant speed. Is the force acting on the object a
continuous function? Explain. Is there any physically reasonable way to make this
force continuous by relaxing the requirement that the object move at constant speed?
If the curve were part of a railroad track, what would happen at the point where
x = 0?
22. An object of mass m moving over a space curve is acted on by a force F. Show the
work done by this force equals maT (length of the curve). In other words, it is only
the tangential component of the force which does work.
23. The edge of an elliptical skating rink represented in the following picture has a light
x2 y2
at its left end and satisfies the equation 900 + 256 = 1. (Distances measured in yards.)

T
z (x, y)

A hockey puck slides from the point T towards the center of the rink at the rate of 2
What is the speed of its shadow along the wall when z = 8? Hint:
yards per second.p
You need to find x′2 + y′2 at the instant described.
Chapter 16

Functions Of Many Variables

16.1 Review Of Limits

Recall the concept of limit of a function of many variables. When f : D (f ) → Rq one can
only consider in a meaningful way limits at limit points of the set D (f ).

Definition 16.1.1 Let A denote a nonempty subset of R p . A point x is said to be a limit

point of the set A if for every r > 0, B (x, r) contains infinitely many points of A.

Example 16.1.2 Let S denote the set (x, y, z) ∈ R3 : x, y, z are all in N . Which points are


limit points?

This set does not have any because any two of these points are at least as far apart as 1.
Therefore, if x is any point of R3 , B (x, 1/4) contains at most one point.

Example 16.1.3 Let U be an open set in R3 . Which points of U are limit points of U?

They all are. From the definition of U being open, if x ∈ U, There exists B (x, r) ⊆ U
for some r > 0. Now consider the line segment x + tr e1 where t ∈ [0, 1/2]. This describes
infinitely many points and they are all in B (x, r) because |x + tr e1 − x| = tr < r. Therefore,
every point of U is a limit point of U.
The case where U is open will be the one of most interest, but many other sets have
limit points.

Definition 16.1.4 Let f : D (f ) ⊆ R p → Rq where q, p ≥ 1 be a function and let x be a

limit point of D (f ). Then
lim f (y) = L
y →x

if and only if the following condition holds. For all ε > 0 there exists δ > 0 such that if

0 < |y − x| < δ and y ∈ D (f )

then,
|L − f (y)| < ε.

The condition that x must be a limit point of D (f ) if you are to take a limit at x is what
makes the limit well defined.

285
286 CHAPTER 16. FUNCTIONS OF MANY VARIABLES

Proposition 16.1.5 Let f : D (f ) ⊆ R p → Rq where q, p ≥ 1 be a function and let x be a

limit point of D (f ). Then if limy→x f (y) exists, it must be unique.

Proof: Suppose limy→x f (y) = L1 and limy→x f (y) = L2 . Then for ε > 0 given, let
δ i > 0 correspond to Li in the definition of the limit and let δ = min (δ 1 , δ 2 ). Since x is a
limit point, there exists y ∈ B (x, δ ) ∩ D (f ). Therefore,

|L1 − L2 | ≤ |L1 − f (y)| + |f (y) − L2 | < ε + ε = 2ε.

Since ε > 0 is arbitrary, this shows L1 = L2 . ■

The following theorem summarized many important interactions involving continuity.
Most of this theorem has been proved in Theorem 12.5.5 on Page 223.

Theorem 16.1.6 Suppose x is a limit point of D (f ) and

lim f (y) = L, lim g (y) = K

y →x y →x

where K and L are vectors in R p for p ≥ 1. Then if a, b ∈ R,

lim af (y) + bg (y) = aL + bK, (16.1)

y →x

lim f · g (y) = L · K (16.2)

y →x

Also, if h is a continuous function defined near L, then

lim h ◦ f (y) = h (L) . (16.3)

y →x

For a vector valued function

f (y) = ( f1 (y) , · · · , fq (y))T ,

limy→x f (y) =L = (L1 · · · , Lk )T if and only if

lim fk (y) = Lk (16.4)

y →x

for each k = 1, · · · , p.
In the case where f and g have values in R3

lim f (y) × g (y) = L × K. (16.5)

y →x

Also recall Theorem 12.5.6 on Page 226.

Theorem 16.1.7 For f : D (f ) → Rq and x ∈ D (f ) such that x is a limit point of D (f ),

it follows f is continuous at x if and only if limy→x f (y) = f (x).
16.2. EXERCISES 287

16.2 Exercises
1. Sketch the contour graph of the function of two variables f (x, y) = (x − 1)2 +(y − 2)2 .

2. Which of the following functions could correspond to the following contour graphs?
z = x2 + 3y2 , z = 3x2 + y2 , z = x2 − y2 , z = x + y.
4 4 4

2 2 2

0 0 0

-2 -2 -2

-4 -4 -4
-4 -2 0 2 4 -4 -2 0 2 4 -4 -2 0 2 4

3. Which of the following functions could correspond to the following contour graphs?
z = x2 − 3y2 , z = y2 + 3x2 , z = x − y, z = x + y.
4 4 4

2 2 2

0 0 0

-2 -2 -2

-4 -4 -4
-4 -2 0 2 4 -4 -2 0 2 4 -4 -2 0 2 4

4. Which of the following functions could correspond to the following contour graphs?
z = sin(x + y), z = x + y, z = (x + y)2 , z = x2 − y.
4 4 4

2 2 2

0 0 0

-2 -2 -2

-4 -4 -4
-4 -2 0 2 4 -4 -2 0 2 4 -4 -2 0 2 4

5. Find the following limits if they exist. If they do not exist, explain why.
−y 2 2
(a) lim(x,y)→(0,0) xx2 +y 2

3 +xy2 −x2 −2y2

(b) lim(x,y)→(0,0) 2x x2 +2y2
sin( x2 +y2 )
(c) lim(x,y)→(0,0) x2 +y2
sin(x2 +2y2 )
(d) lim(x,y)→(0,0) x2 +2y2
sin(x2 +2y2 )
(e) lim(x,y)→(0,0) 2x2 +y2
2
(x2 −y4 )
(f) lim(x,y)→(0,0) 2
(x2 +y4 )

6. Find the following limits if they exist. If they do not exist, tell why.
2
(x2 −y4 )
(a) lim(x,y)→(0,0) x 2
(x2 +y4 )
x sin(x2 +2y2 )
(b) lim(x,y)→(0,0) 2x2 +y2
xy
(c) lim(x,y)→(0,0) x2 +y2
288 CHAPTER 16. FUNCTIONS OF MANY VARIABLES

3 −3x2 +3x−1−y2 x+y2

(d) lim(x,y)→(1,0) x x2 −2x+1+y2

7. ∗ Suppose f is a function defined on a set D and that a ∈ D is not a limit point

of D. Show that if I define the notion of limit in the same way as above, then
limx→a f (x) = 5. Show that it is also the case that limx→a f (x) = 7. In other
words, the concept of limit is totally meaningless. This is why the insistence that the
point a be a limit point of D.

8. ∗ Show that the definition of continuity at a ∈ D (f ) is not dependent on a being a

limit point of D (f ). The concept of limit and the concept of continuity are related at
those points a which are limit points of the domain.

16.3 The Directional Derivative And Partial Derivatives

16.3.1 The Directional Derivative
The directional derivative is just what its name suggests. It is the derivative of a function in
a particular direction. The following picture illustrates the situation in the case of a function
of two variables.

v
In this picture, v ≡ (v1 , v2 ) is a unit vector in the xy plane and x0 ≡ (x0 , y0 ) is a point in
the xy plane. When (x, y) moves in the direction of v, this results in a change in z = f (x, y)
as shown in the picture. The directional derivative in this direction is defined as

f (x0 + tv1 , y0 + tv2 ) − f (x0 , y0 )

lim .
t→0 t
It tells how fast z is changing in this direction. If you looked at it from the side, you
would be getting the slope of the indicated tangent line. A simple example of this is a
person climbing a mountain. He could go various directions, some steeper than others. The
directional derivative is just a measure of the steepness in a given direction. This motivates
the following general definition of the directional derivative.

Definition 16.3.1 Let f : U → R where U is an open set in Rn and let v be a unit vector.
For x ∈ U, define the directional derivative of f in the direction v, at the point x as

f (x + tv) − f (x)
Dv f (x) ≡ lim .
t→0 t
16.3. THE DIRECTIONAL DERIVATIVE AND PARTIAL DERIVATIVES 289

Example 16.3.2 Find the directional derivative of the function f (x, y) = x2 y in the direc-
tion of i + j at the point (1, 2).
First you need a unit vector
 which has the same direction as the given vector. This
1 1
unit vector is v ≡ √2 , √2 . Then to find the directional derivative from the definition,
 2  
write the difference quotient described above. Thus f (x + tv) = 1 + √t 2 2 + √t 2 and
f (x) = 2. Therefore,
 2  
1 + √t 2 + √t −2
f (x + tv) − f (x) 2 2
= ,
t t
and to find the directional derivative, you takethe limit of this as t → 0. However,this dif-
√  √ √ 
1
ference quotient equals 4 2 10 + 4t 2 + t 2 and so, letting t → 0, Dv f (1, 2) = 25 2 .
There is something you must keep in mind about this. The direction vector must always
be a unit vector1 .

16.3.2 Partial Derivatives

There are some special unit vectors which come to mind immediately. These are the vectors
ei where
ei = (0, · · · , 0, 1, 0, · · · 0)T
and the 1 is in the ith position.
Thus in case of a function of two variables, the directional derivative in the direction
i = e1 is the slope of the indicated straight line in the following picture.

z = f (x, y)

y
e1
x
As in the case of a general directional derivative, you fix y and take the derivative of
the function x → f (x, y). More generally, even in situations which cannot be drawn, the
definition of a partial derivative is as follows.
Definition 16.3.3 Let U be an open subset of Rn and let f : U → R. Then letting x =
(x1 , · · · , xn )T be a typical element of Rn ,
∂f
(x) ≡ Dei f (x) .
∂ xi
1 Actually, there is a more general formulation of the notion of directional derivative known as the Gateaux

derivative in which the length of v is not one but it is not considered here.
290 CHAPTER 16. FUNCTIONS OF MANY VARIABLES

This is called the partial derivative of f . Thus,

∂f f (x+tei ) − f (x)
(x) ≡ lim
∂ xi t→0 t
f (x1 , · · · , xi + t, · · · xn ) − f (x1 , · · · , xi , · · · xn )
= lim ,
t→0 t
and to find the partial derivative, differentiate with respect to the variable of interest and
regard all the others as constants. Other notation for this partial derivative is fxi , f,i , or
Di f . If y = f (x), the partial derivative of f with respect to xi may also be denoted by ∂∂xyi
or yxi .
∂f ∂f ∂f
Example 16.3.4 Find ∂x , ∂y , and ∂z if f (x, y) = y sin x + x2 y + z.

From the definition above, ∂∂ xf = y cos x +2xy, ∂∂ yf = sin x +x2 , and ∂∂ zf = 1. Having taken
one partial derivative, there is no reason to stop doing it. Thus, one could take the partial
2
derivative with respect to y of the partial derivative with respect to x, denoted by ∂∂y∂fx or
fxy . In the above example,
∂2 f
= fxy = cos x + 2x.
∂ y∂ x
Also observe that
∂2 f
= fyx = cos x + 2x.
∂ x∂ y
Higher order partial derivatives are defined by analogy to the above. Thus in the above
example,
fyxx = − sin x + 2.
These partial derivatives, fxy are called mixed partial derivatives.
There is an interesting relationship between the directional derivatives and the partial
derivatives, provided the partial derivatives exist and are continuous.

Definition 16.3.5 Suppose f : U ⊆ Rn → R where U is an open set and the partial deriva-
tives of f all exist and are continuous on U. Under these conditions, define the gradient of
f denoted ∇ f (x) to be the vector

∇ f (x) = ( fx1 (x) , fx2 (x) , · · · , fxn (x))T .

Proposition 16.3.6 In the situation of Definition 16.3.5 and for v a unit vector

Dv f (x) = ∇ f (x) · v.

This proposition will be proved in a more general setting later. For now, you can use it
to compute directional derivatives.

Example 16.3.7 Find the directional derivative of the function f (x, y) = sin 2x2 + y3 at


 T
(1, 1) in the direction √12 , √12 .
16.3. THE DIRECTIONAL DERIVATIVE AND PARTIAL DERIVATIVES 291

First find the gradient.

T
∇ f (x, y) = 4x cos 2x2 + y3 , 3y2 cos 2x2 + y3


.
Therefore,
∇ f (1, 1) = (4 cos (3) , 3 cos (3))T
The directional derivative is therefore,
1 1 T 7 √
 
(4 cos (3) , 3 cos (3))T · √ , √ = (cos 3) 2.
2 2 2
Another important observation is that the gradient gives the direction in which the function
changes most rapidly. The following proposition will be proved later.
Proposition 16.3.8 In the situation of Definition 16.3.5, suppose ∇ f (x) ̸= 0. Then the
direction in which f increases most rapidly, that is the direction in which the directional
derivative is largest, is the direction of the gradient. Thus v = ∇ f (x) / |∇ f (x)| is the
unit vector which maximizes Dv f (x) and this maximum value is |∇ f (x)|. Similarly, v =
−∇ f (x) / |∇ f (x)| is the unit vector which minimizes Dv f (x) and this minimum value is
− |∇ f (x)|.
The concept of a directional derivative for a vector valued function is also easy to
define although the geometric significance expressed in pictures is not.
Definition 16.3.9 Let f : U → R p where U is an open set in Rn and let v be a unit vector.
For x ∈ U, define the directional derivative of f in the direction v, at the point x as
f (x + tv) − f (x)
Dv f (x) ≡ lim .
t→0 t

T
Example 16.3.10 Let f (x, y) = xy2 , yx . Find the directional derivative in the direction
(1, 2)T at the point (x, y).
 √ √ T
First, a unit vector in this direction is 1/ 5, 2/ 5 and from the definition, the
desired limit is
  √    √ 2   √    √  
x + t 1/ 5 y + t 2/ 5 2
− xy , x + t 1/ 5 y + t 2/ 5 − xy
lim
t→0 t
√ √ 4 √ 2 √ 1 √
 
4 4 1 4 2
= lim xy 5 + xt + 5y2 + ty + t 2 5, x 5 + y 5 + t
t→0 5 5 5 5 25 5 5 5
√ √ √ √
 
4 1 2 1
= xy 5 + 5y2 , x 5 + y 5 .
5 5 5 5
You see from this example and the above definition that all you have to do is to form
the vector which is obtained by replacing each component of the vector with its directional
derivative. In particular, you can take partial derivatives of vector valued functions and use
the same notation.
Example 16.3.11 Find the partial derivative with respect to x of the function f (x, y, z, w) =

T
xy2 , z sin (xy) , z3 x .

T
From the above definition, f x (x, y, z) = D1 f (x, y, z) = y2 , zy cos (xy) , z3 .
292 CHAPTER 16. FUNCTIONS OF MANY VARIABLES

16.4 Exercises
1. Find the directional derivative of f (x, y, z) = x2 y + z4 in the direction of the vector
(1, 3, −1) when (x, y, z) = (1, 1, 1).

2. Find the directional derivative of f (x, y, z) = sin x + y2 + z in the direction of the

vector (1, 2, −1) when (x, y, z) = (1, 1, 1).

3. Find the directional derivative of f (x, y, z) = ln x + y2 + z2 in the direction of the

vector (1, 1, −1) when (x, y, z) = (1, 1, 1).

4. Using the conclusion of Proposition 16.3.6, prove Proposition 16.3.8 from the ge-
ometric description of the dot product, the one which says the dot product is the
product of the lengths of the vectors and the cosine of the included angle which is no
larger than π.
5. Find the largest value of the directional derivative of f (x, y, z) = ln x + y2 + z2 at

the point (1, 1, 1).

6. Find the smallest value of the directional derivative of f (x, y, z) = x sin 4xy2 + z2 at

the point (1, 1, 1).

7. An ant falls to the top of a stove having temperature T (x, y) = x2 sin (x + y) at the
point (2, 3). In what direction should the ant go to minimize the temperature? In
what direction should he go to maximize the temperature?
8. Find the partial derivative with respect to y of the function

T
f (x, y, z, w) = y2 , z2 sin (xy) , z3 x .

9. Find the partial derivative with respect to x of the function

T
f (x, y, z, w) = wx, zx sin (xy) , z3 x .
∂f ∂f ∂f
10. Find ∂x , ∂y , and ∂z for f =

(a) x2 y2 z + w
(b) e2 + xy + z2
(c) sin z2 + cos (xy)

(d) ln x2 + y2 + 1 + ez

(e) sin (xyz) + cos (xy)

∂f ∂f ∂f
11. Find ∂x , ∂y , and ∂z for f =

(a) x2 y + cos (xy) + z3 y

2 +y2
(b) ex z sin (x + y)
 2 3
(c) z2 sin3 ex +y

(d) x2 cos sin tan z2 + y2

2 +z
(e) xy
16.5. MIXED PARTIAL DERIVATIVES 293

12. Suppose
2xy+6x3 +12xy2 +18yx2 +36y3 +sin(x3 )+tan(3y3 )
(
3x2 +6y2
if (x, y) ̸= (0, 0)
f (x, y) =
0 if (x, y) = (0, 0) .
∂f ∂f
Find ∂x (0, 0) and ∂y (0, 0).
13. Why must the vector in the definition of the directional derivative be a unit vector?
Hint: Suppose not. Would the directional derivative be a correct manifestation of
steepness?

16.5 Mixed Partial Derivatives

Under certain conditions the mixed partial derivatives will always be equal. This aston-
ishing fact may have been known to Euler in 1734.

Theorem 16.5.1 Suppose f : U ⊆ R2 → R where U is an open set on which fx , fy , fxy and

fyx exist. Then if fxy and fyx are continuous at the point (x, y) ∈ U, it follows

fxy (x, y) = fyx (x, y) .

Proof: Since U is open, there exists r > 0 such that B ((x, y) , r) ⊆ U. Now let |t| , |s| <
r/2 and consider
h(t) h(0)
1 z }| { z }| {
∆ (s,t) ≡ { f (x + t, y + s) − f (x + t, y) − ( f (x, y + s) − f (x, y))}. (16.6)
st
Note that (x + t, y + s) ∈ U because

1/2
|(x + t, y + s) − (x, y)| = |(t, s)| = t 2 + s2
 2 1/2
r r2 r
≤ + = √ < r.
4 4 2
As implied above, h (t) ≡ f (x + t, y + s) − f (x + t, y). Therefore, by the mean value theo-
rem from calculus and the (one variable) chain rule,
1 1
∆ (s,t) = (h (t) − h (0)) = h′ (αt)t
st st
1
= ( fx (x + αt, y + s) − fx (x + αt, y))
s
for some α ∈ (0, 1). Applying the mean value theorem again,

∆ (s,t) = fxy (x + αt, y + β s)

where α, β ∈ (0, 1).

If the terms f (x + t, y) and f (x, y + s) are interchanged in (16.6), ∆ (s,t) is also un-
changed and the above argument shows there exist γ, δ ∈ (0, 1) such that

∆ (s,t) = fyx (x + γt, y + δ s) .

294 CHAPTER 16. FUNCTIONS OF MANY VARIABLES

Letting (s,t) → (0, 0) and using the continuity of fxy and fyx at (x, y) ,

lim ∆ (s,t) = fxy (x, y) = fyx (x, y) . ■

(s,t)→(0,0)

The following is obtained from the above by simply fixing all the variables except for
the two of interest.

Corollary 16.5.2 Suppose U is an open subset of Rn and f : U → R has the property that
for two indices k, l, fxk , fxl , fxl xk , and fxk xl exist on U and fxk xl and fxl xk are both continuous
at x ∈ U. Then fxk xl (x) = fxl xk (x).

It is necessary to assume the mixed partial derivatives are continuous in order to assert
they are equal. The following is a well known example [3].

Example 16.5.3 Let

xy(x2 −y2 )
(
x2 +y2
if (x, y) ̸= (0, 0)
f (x, y) =
0 if (x, y) = (0, 0)

Here is a picture of the graph of this function. It looks innocuous but isn’t.

From the definition of partial derivatives it follows immediately that fx (0, 0) = fy (0, 0) =
0. Using the standard rules of differentiation, for (x, y) ̸= (0, 0),

x4 − y4 + 4x2 y2 x4 − y4 − 4x2 y2
fx = y , fy = x
(x2 + y2 )2 (x2 + y2 )2
Now
fx (0, y) − fx (0, 0) −y4
fxy (0, 0) ≡ lim = lim = −1
y→0 y y→0 (y2 )2

while
fy (x, 0) − fy (0, 0) x4
fyx (0, 0) ≡ lim = lim =1
x→0 x x→0 (x2 )2

showing that, although the mixed partial derivatives do exist at (0, 0), they are not equal
there.

16.6 Partial Differential Equations

Partial differential equations are equations which involve the partial derivatives of some
function. The most famous partial differential equations involve the Laplacian, named
after Laplace2 .
2 Laplace was a great physicist and mathematician of the 1700’s. He made fundamental contributions to me-

chanics and astronomy.

16.7. EXERCISES 295

Definition 16.6.1 Let u be a function of n variables. Then ∆u ≡ ∑nk=1 uxk xk . This is also
written as ∇2 u. The symbol ∆ or ∇2 is called the Laplacian. When ∆u = 0 the function u is
called [Link]’s equation is ∆u = 0. The heat equation is ut − ∆u = 0 and the
wave equation is utt − ∆u = 0.

Example 16.6.2 Find the Laplacian of u (x, y) = x2 − y2 .

uxx = 2 while uyy = −2. Therefore, ∆u = uxx + uyy = 2 − 2 = 0. Thus this function is
harmonic, ∆u = 0.

Example 16.6.3 Find ut − ∆u where u (t, x, y) = e−t cos x.

In this case, ut = −e−t cos x while uyy = 0 and uxx = −e−t cos x therefore, ut − ∆u = 0
and so u solves the heat equation ut − ∆u = 0.

Example 16.6.4 Let u (t, x) = sint cos x. Find utt − ∆u.

In this case, utt = − sint cos x while ∆u = − sint cos x. Therefore, u is a solution of the
wave equation utt − ∆u = 0.

16.7 Exercises
1. Find fx , fy , fz , fxy , fyx , fxz, fzx , fzy , fyz for the following. Verify the mixed partial deriva-
tives are equal.

(a) x2 y3 z4 + sin (xyz)

(b) sin (xyz) + x2 yz
(c) z ln x2 + y2 + 1
2 +y2 +z2
(d) ex
(e) tan (xyz)

2. Suppose f is a continuous function and f : U → R where U is an open set and

suppose that x ∈ U has the property that for all y near x, f (x) ≤ f (y). Prove that
if f has all of its partial derivatives at x, then fxi (x) = 0 for each xi . Hint: This is
just a repeat of the usual one variable theorem seen in beginning calculus. You just
do this one variable argument for each variable to get the conclusion.
3. As an important application of Problem 2 consider the following. Experiments are
done at n times, t1 ,t2 , · · · ,tn and at each time there results a collection of numerical
p
outcomes. Denote by {(ti , xi )}i=1 the set of all such pairs and try to find numbers a
and b such that the line x = at + b approximates these ordered pairs as well as pos-
p
sible in the sense that out of all choices of a and b, ∑i=1 (ati + b − xi )2 is as small
as possible. In other words, you want to minimize the function of two variables
p
f (a, b) ≡ ∑i=1 (ati + b − xi )2 . Find a formula for a and b in terms of the given or-
dered pairs. You will be finding the formula for the least squares regression line.

4. Show that if v (x, y) = u (αx, β y), then vx = αux and vy = β uy . State and prove a
generalization to any number of variables.
296 CHAPTER 16. FUNCTIONS OF MANY VARIABLES

5. Let f be a function which has continuous derivatives. Show that u (t, x) = f (x − ct)
solves the wave equation utt − c2 ∆u = 0. What about u (x,t) = f (x + ct)?
6. D’Alembert found a formula for the solution to the wave equation utt = c2 uxx along
with the initial conditions u (x, 0) = f (x) , ut (x, 0) = g (x). Here is how he did it. He
looked for a solution of the form u (x,t) = h (x + ct) + k (x − ct) and then found h and
k in terms of the given functions f and g. He ended up with something like
Z x+ct
1 1
u (x,t) = g (r) dr + ( f (x + ct) + f (x − ct)) .
2c x−ct 2
Fill in the details.
7. Determine which of the following functions satisfy Laplace’s equation.

(a) x3 − 3xy2
(b) 3x2 y − y3
(c) x3 − 3xy2 + 2x2 − 2y2
(d) 3x2 y − y3 + 4xy
(e) 3x2 − y3 + 4xy
(f) 3x2 y − y3 + 4y
(g) x3 − 3x2 y2 + 2x2 − 2y2
p
8. Show that z = x2 + y2 is a solution to x ∂∂ xz + y ∂∂ yz = z.

9. Show that if ∆u = λ u where u is a function of only x, then eλt u solves the heat
equation ut − ∆u = 0.

10. Show that if a, b are scalars and u, v are functions which satisfy Laplace’s equation
then au + bv also satisfies Laplace’s equation. Verify a similar statement for the heat
and wave equations.
1 −x2 /4c2 t
11. Show that u (x,t) = √
t
e solves the heat equation ut = c2 uxx .
Chapter 17

The Derivative Of A Function Of

Many Variables

17.1 The Derivative Of Functions Of One Variable

First consider the notion of the derivative of a function of one variable.

Observation 17.1.1 Suppose a function f of one variable has a derivative at x. Then

| f (x + h) − f (x) − f ′ (x) h|
lim = 0.
h→0 |h|
This observation follows from the definition of the derivative of a function of one variable,
namely
f (x + h) − f (x)
f ′ (x) ≡ lim .
h→0 h
Thus
| f (x + h) − f (x) − f ′ (x) h| f (x + h) − f (x)
lim = lim − f ′ (x) = 0
h→0 |h| h→0 h
Definition 17.1.2 A vector valued function of a vector v is called o (v) (referred to as
“little o of v”) if
o (v)
lim = 0. (17.1)
|v |→0 |v|

Thus for a function of one variable, the function f (x + h) − f (x) − f ′ (x) h is o (h).
When we say a function is o (h), it is used like an adjective. It is like saying the function is
white or black or green or fat or thin. The term is used very imprecisely. Thus in general,

o (v) = o (v) + o (v) , o (v) = 45 × o (v) , o (v) = o (v) − o (v) , etc.

When you add two functions with the property of the above definition, you get another one
having that same property. When you multiply by 45, the property is also retained, as it
is when you subtract two such functions. How could something so sloppy be useful? The
notation is useful precisely because it prevents you from obsessing over things which are
not relevant and should be ignored.

297
298 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

Theorem 17.1.3 Let f : (a, b) → R be a function of one variable. Then f ′ (x) exists if and
only if there exists p such that

f (x + h) − f (x) = ph + o (h) (17.2)

In this case, p = f ′ (x).

Proof: From the above observation it follows that if f ′ (x) does exist, then (17.2) holds.
Suppose then that (17.2) is true. Then

f (x + h) − f (x) o (h)
−p= .
h h
Taking a limit, you see that
f (x + h) − f (x)
p = lim
h→0 h
and that in fact this limit exists which shows that p = f ′ (x). ■
This theorem shows that one way to define f ′ (x) is as the number p, if there is one,
which has the property that

f (x + h) = f (x) + ph + o (h) .

You should think of p as the linear transformation resulting from multiplication by the 1 ×1
matrix (p).

Example 17.1.4 Let f (x) = x3 . Find f ′ (x).

f (x + h) = (x + h)3 = x3 + 3x2 h + 3xh2 + h3

= f (x) + 3x2 h + 3xh + h2 h.

Since 3xh + h2 h = o (h), it follows f ′ (x) = 3x2 .

Example 17.1.5 Let f (x) = sin (x). Find f ′ (x).

f (x + h) − f (x) = sin (x + h) − sin (x) = sin (x) cos (h) + cos (x) sin (h) − sin (x)
(cos (h) − 1)
= cos (x) sin (h) + sin (x) h
h
(sin (h) − h) (cos (h) − 1)
= cos (x) h + cos (x) h + sin x h.
h h
Now
(sin (h) − h) (cos (h) − 1)
cos (x) h + sin x h = o (h) . (17.3)
h h
Remember the fundamental limits which allowed you to find the derivative of sin (x) were

sin (h) cos (h) − 1

lim = 1, lim = 0. (17.4)
h→0 h h→0 h
17.2. EXERCISES 299

These same limits are what is needed to verify (17.3). !

u
How can you tell whether a function of two variables (u, v) is o ? In general,
v
there is no substitute for the definition, but you can often identify this property by observing
that the expression involves only “higher order terms”. These are terms like u2 v, uv, v4 , etc.
If you sum the exponents on the u and the v you get something larger than 1. For example,

vu 1 2 1 1p 2
u + v2 √


√ ≤ = u + v2
u2 + v2 2 u2 + v2 2

and this converges to 0 as (u, v) → (0, 0). This follows from the inequality |uv| ≤ 12 u2 + v2

which you can verify from (u − v)2 ≥ 0. Similar considerations apply in higher dimensions
also. In general, this is a hard question because it involves a limit of a function of many
variables. Furthermore, there is really no substitute for answering this question, because
its resolution involves the definition of whether a function is differentiable. That may be
why we spend most of our time on one dimensional considerations which involve taking
the partial derivatives. The following exercises should help give you an idea of how to
determine whether something is o.

17.2 Exercises
1. Determine which of the following functions are o (h).

(a) h2
(b) h sin (h)
(c) |h|3/2 ln (|h|)
(d) h2 x + yh3
(e) sin h2

(f) sin (h)

p 
(g) xh sin |h| + x5 h2
 
(h) exp −1/ |h|2

2. Here are some scalar valued functions of several variables. Determine which of these
functions are o (v). Here v is a vector in Rn , v = (v1 , · · · , vn ).

(a) v1 v2
(b) v2 sin (v1 )
(c) v21 + v2
(d) v2 sin (v1 + v2 )
(e) v1 (v1 + v2 + xv3 )
(f) (ev1 − 1 − v1 )
(g) (x · v) |v|
300 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

3. Here are some vector valued functions of v ∈ Rn . Determine which ones are o (v).
(a) (x · v) v
(b) sin (v1 ) v
(c) |(x · v)| |v|2/3
p

(d) |(x · v)| |v|1/2

p
 p  p 
(e) sin |x · v| − |x · v| ·
|v|−1/4
 
(f) exp −1/ |v|2
(g) v T Av where A is an n × n matrix.
4. Show that if f (x) = o (x), then f ′ (0) = 0.
5. Show that if limh→0 f (x) = 0 then x f (x) = o (x).
6. Show that if f ′ (0) exists and f (0) = 0, then f (|x| p ) = o (x) whenever p > 1.

17.3 The Derivative Of Functions Of Many Variables

The way of thinking about the derivative in Theorem 17.1.3 is exactly what is needed to
define the derivative of a function of n variables. Recall the following definition.
Definition 17.3.1 A function T which maps Rn to R p is called a linear transformation if
for every pair of scalars, a, b and vectors x, y ∈ Rn , it follows that T (ax + by) = aT (x) +
bT (y).
Recall that from the properties of matrix multiplication, if A is an p × n matrix, and if
x, y are vectors in Rn , then A (ax + by) = aA (x) + bA (y). Thus you can define a linear
transformation by multiplying by a matrix. Of course the simplest example is that of a 1 × 1
matrix or number. You can think of the number 3 as a linear transformation T mapping R to
R according to the rule T x = 3x. It satisfies the properties needed for a linear transformation
because 3 (ax + by) = a3x + b3y = aT x + bTy. The case of the derivative of a scalar valued
function of one variable is of this sort. You get a number for the derivative. However, you
can think of this number as a linear transformation. Of course it might not be worth the
fuss to think of it this way for a function of one variable but this is the way you must think
of it for a function of n variables.
Definition 17.3.2 Let f : U → R p where U is an open set in Rn for n, p ≥ 1 and let x ∈ U
be given. Then f is defined to be differentiable at x ∈ U if and only if there exists a linear
transformation T such that,
f (x + h) = f (x) + T h + o (h) . (17.5)
The derivative of the function f, denoted by Df (x), is this linear transformation. Thus
f (x + h) = f (x) + Df (x) h + o (h)
If h = x − x0 , this takes the form
f (x) = f (x0 ) + Df (x0 ) (x − x0 ) + o (x − x0 )
17.3. THE DERIVATIVE OF FUNCTIONS OF MANY VARIABLES 301

If you deleted the o (x − x0 ) term and considered the function of x given by what is
left, this is called the linear approximation to the function at the point x0 . In the case where
x ∈ R2 and f has values in R one can draw a picture to illustrate this.

Of course the first and most obvious question is whether the linear transformation is
unique. Otherwise, the definition of the derivative Df (x) would not be well defined.

Theorem 17.3.3 Suppose f is differentiable, as given above in (17.5). Then T is uniquely

determined. Furthermore, the matrix of T is the following p × n matrix
 
∂ f (x) ∂ f (x)
∂x · · · ∂ xn
1

where
∂f f (x+tei ) − f (x)
(x) ≡ lim ,
∂ xi h→0 t
the kth partial derivative of f .

Proof: Suppose T1 is another linear transformation which works. Thus, letting t be a

small positive real number,

f (x+th) = f (x) + T th + o (th)

f (x+th) = f (x) + T1th + o (th)

Now o (th) = o (t) and so, subtracting these yields

T th − T1th = o (t)

Divide both sides by t to obtain

o (t)
T h − T1 h =
t
It follows on letting t → 0 that T h = T1 h. Since h is arbitrary, this shows that T = T1 . Thus
the derivative is well defined. So what is the matrix of this linear transformation? From
Theorem 8.3.2, this is the matrix whose ith column is T ei . However, from the definition of
T, letting t ̸= 0,
f (x + tei ) − f (x) 1
= (T (tei ) + o (tei ))
t t
o (tei ) o (t)
= T (ei ) + = T (ei ) +
t t
Then letting t → 0, it follows that
∂f
T ei = (x)
∂ xi
302 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

Recall from theorem 8.3.2 this shows the matrix of the linear transformation is as claimed.
■
Other notations which are often used for this matrix or the linear transformation are
f ′ (x) , J (x), and even ∂∂ fx or dd fx . Also, the above definition can now be written in the form
p
∂ f (x)
f (x + v) = f (x) + ∑ v j + o (v)
j=1 ∂ x j

or  
∂ f (x) ∂ f (x)
f (x + v) − f (x) = ∂ x1 ··· ∂ xn
v + o (v)
Here is an example of a scalar valued nonlinear function.
√
Example 17.3.4 Suppose f (x, y) = xy. Find the approximate change in f if x goes from
1 to 1.01 and y goes from 4 to 3.99.

We can do this by noting that

f (1.01, 3.99) − f (1, 4) ≈ fx (1, 2) (.01) + fy (1, 2) (−.01)

1
= 1 (.01) + (−.01) = 7. 5 × 10−3 .
4
Of course the exact value is
√
(1.01) (3.99) − 4 = 7. 461 083 1 × 10−3 .
p

Notation 17.3.5 When f is a scalar valued function of n variables, the following is often
written to express the idea that a small change in f due to small changes in the variables
can be expressed in the form

d f (x) = fx1 (x) dx1 + · · · + fxn (x) dxn

where the small change in xi is denoted as dxi . As explained above, d f is the approximate
change in the function f . Sometimes d f is referred to as the differential of f .

Let f : U → Rq where U is an open subset of R p and f is differentiable. It was just

shown that  
∂ f (x)
f (x + v) = f (x) + ∂ x1 · · · ∂ ∂fx(x
p
)
v + o (v) .

Taking the ith coordinate of the above equation yields

p
∂ fi (x)
fi (x + v) = fi (x) + ∑ v j + o (v) ,
j=1 ∂ x j

and it follows that the term with a sum is nothing more than the ith component of J (x) v
where J (x) is the q × p matrix
 ∂f ∂ f1
· · · ∂∂ xf1p

1
∂ x1 ∂ x2
 ∂ f2 ∂ f2 ∂ f2 
 1 ∂ x2 · · · ∂ x p 
 ∂x 
 . .. .. ..  .
 .. . . . 
 
∂ fq ∂ fq ∂ fq
∂ x1 ∂ x2 ··· ∂ xp
17.3. THE DERIVATIVE OF FUNCTIONS OF MANY VARIABLES 303

Thus
f (x + v) = f (x) + J (x) v + o (v) , (17.6)
and to reiterate, the linear transformation which results by multiplication by this q × p
matrix is known as the derivative.
Sometimes x, y, z is written instead of x1 , x2 , and x3 . This is to save on notation and is
easier to write and to look at although it lacks generality. When this is done it is understood
that x = x1 , y = x2 , and z = x3 . Thus the derivative is the linear transformation determined
by  
f1x f1y f1z
 f2x f2y f2z  .
 

f3x f3y f3z

Example 17.3.6 Let A be a constant m × n matrix and consider f (x) = Ax. Find Df (x)
if it exists.

f (x + h) − f (x) = A (x + h) − A (x) = Ah = Ah + o (h) .

In fact in this case, o (h) = 0. Therefore, Df (x) = A. Note that this looks the same as the
case in one variable, f (x) = ax.

Example 17.3.7 Let f (x, y, z) = xy + z2 x. Find D f (x, y, z).

Consider f (x + h, y + k, z + l) − f (x, y, z). This is something which is easily computed

from the definition of the function. It equals

(x + h) (y + k) + (z + l)2 (x + h) − xy + z2 x

Multiply everything together and collect the terms. This yields

z2 + y h + xk + 2zxl + hk + +2zlh + l 2 x + l 2 h

It follows easily the last term at the end is o (h, k, l) and so the derivative of this

function

is
the linear transformation coming from multiplication by the matrix z2 + y , x, 2zx and
so this is the derivative. It follows from this and the description of the derivative in terms
of partial derivatives that

∂f ∂f ∂f
(x, y, z) = z2 + y, (x, y, z) = x, (x, y, z) = 2xz.
∂x ∂y ∂z
Of course you could compute these partial derivatives directly.
Given a function of many variables, how can you tell if it is differentiable? In other
words, when you make the linear approximation, how can you tell easily that what is left
over is o (v). Sometimes you have to go directly to the definition and verify it is differ-
entiable from the definition. For example, you may have seen the following important
example in one variable calculus.
(
x2 sin 1x if x ̸= 0


Example 17.3.8 Let f (x) = . Find D f (0).
0 if x = 0
304 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

 
1
f (h) − f (0) = 0h + h2 sin = o (h) ,
h
and so D f (0) = 0. If you find the derivative for x ̸= 0, it is totally useless information if
what you want is D f (0). This is because the derivative turns out to be discontinuous. Try
it. Find the derivative for x ̸= 0 and try to obtain D f (0) from it. You see, in this example
you had to revert to the definition to find the derivative.
It isn’t really too hard to use the definition even for more ordinary examples.
!
x2 y + y2
Example 17.3.9 Let f (x, y) = . Find Df (1, 2).
y3 x

First of all, note that the thing you are after is a 2 × 2 matrix.
!
6
f (1, 2) = .
8

Then
f (1 + h1 , 2 + h2 ) − f (1, 2)
! !
(1 + h1 )2 (2 + h2 ) + (2 + h2 )2 6
= −
(2 + h2 )3 (1 + h1 ) 8
!
5h2 + 4h1 + 2h1 h2 + 2h21 + h21 h2 + h22
=
8h1 + 12h2 + 12h1 h2 + 6h22 + 6h22 h1 + h32 + h32 h1
! ! !
4 5 h1 2h1 h2 + 2h21 + h21 h2 + h22
= +
8 12 h2 12h1 h2 + 6h22 + 6h22 h1 + h32 + h32 h1
! !
4 5 h1
= + o (h) .
8 12 h2
!
4 5
Therefore, the matrix of the derivative is .
8 12
!
x3 y + y2
Example 17.3.10 Let f (x, y) = . Find Df (x, y) .
xy2 + 1

You know that if there is a derivative, its standard matrix is of the form
! !
f1x (x, y) f1y (x, y) 3x2 y x3 + 2y
=
f2x (x, y) f2y (x, y) y2 2xy

Does it work? Is
! !
(x + u)3 (y + v) + (y + v)2 x3 y + y2
−
(x + u) (y + v)2 + 1 xy2 + 1
17.4. EXERCISES 305
! ! !
3x2 y x3 + 2y u u
− =o ?
y2 2xy v v
Doing the computations, it follows the left side of the equal sign is of the form
!
3x2 uv + 3xu2 y + 3xu2 v + u3 y + u3 v + v2
xv2 + 2uyv + uv2
!
u
This is o because it involves terms like uv, u2 v, etc. Each term being of degree 2 or
v
more.

17.4 Exercises
1. Use the definition of the derivative to find the 1 × 1 matrix which is the derivative of
the following functions.

(a) f (t) = t 2 + t.
(b) f (t) = t 3 .
(c) f (t) = t sin (t).
(d) f (t) = ln t 2 + 1 .

(e) f (t) = t |t|.

2. Show that if f is a real valued function defined on (a, b) and it achieves a local
maximum at x ∈ (a, b), then D f (x) = 0.
3. Use the above definition of the derivative to prove the product rule for functions of 1
variable.

4. Let f (x, y) = x sin (y). Compute the derivative directly from the definition.
5. Let f (x, y) = x2 sin (y). Compute the derivative directly from the definition.
!
x2 + y
6. Let f (x, y) = . Compute the derivative directly from the definition.
y2
!
x2 y
7. Let f (x, y) = . Compute the derivative directly from the definition.
x + y2
 
8. Let f (x, y) = xα yβ . Show D f (x, y) = αxα−1 yβ xα β yβ −1 .
!
x2 sin (y)
9. Let f (x, y) = . Find Df (x, y).
x2 + y
√ √
10. Let f (x, y) = x 3 y. Find the approximate change in f when (x, y) goes from (4, 8)
to (4.01, 7.99).
306 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

11. Suppose f is differentiable and g is also differentiable, g having values in R3 and f

having values in R. Find D ( f g) directly from the definition. Assume both functions
are defined on an open subset of Rn .
12. Show, using the above definition, that if f is differentiable, then so is t → f (t)n for
any positive integer and in fact the derivative of this function is n f (t)n−1 f ′ (t).
13. Suppose f is a scalar valued function of two variables which is differentiable. Show
that (x, y) → ( f (x, y))n is also differentiable and its derivative equals

n f (x, y)n−1 D f (x, y)

14. Let f (x, y) be defined on R2 as follows. f x, x2 = 1 if x ̸= 0. Define f (0, 0) = 0,

and f (x, y) = 0 if y ̸= x2 . Show that f is not continuous at (0, 0) but that

f (ha, hb) − f (0, 0)

lim =0
h→0 h
for (a, b) an arbitrary unit vector. Thus the directional derivative exists at (0, 0) in
every direction, but f is not even continuous there.

17.5 C1 Functions
Most of the time, there is an easier way to conclude that a derivative exists and to find it. It
involves the notion of a C1 function.

Definition 17.5.1 When f : U → R p for U an open subset of Rn and the vector valued
functions ∂∂ xfi are all continuous, (equivalently each ∂∂ xfij is continuous), the function is said
to be C1 (U). If all the partial derivatives up to order k exist and are continuous, then the
function is said to be Ck .

It turns out that for a C1 function, all you have to do is write the matrix described in
Theorem 17.3.3 and this will be the derivative. There is no question of existence for the
derivative for such functions. This is the importance of the next theorem.

Theorem 17.5.2 Suppose f : U → R p where U is an open set in Rn . Suppose also that all
partial derivatives of f exist on U and are continuous. Then f is differentiable at every
point of U.

Proof: If you fix all the variables but one, you can apply the fundamental theorem of
calculus as follows.
Z 1
∂f
f (x+vk ek ) − f (x) = (x + tvk ek ) vk dt. (17.7)
0 ∂ xk

Here is why. Let h (t) = f (x + tvk ek ). Then

h (t + h) − h (t) f (x + tvk ek + hvk ek ) − f (x + tvk ek )

= vk
h hvk
17.5. C1 FUNCTIONS 307

and so, taking the limit as h → 0 yields

∂f
h′ (t) = (x + tvk ek ) vk
∂ xk
Therefore,
Z 1 Z 1
∂f
f (x+vk ek ) − f (x) = h (1) − h (0) = h′ (t) dt = (x + tvk ek ) vk dt.
0 0 ∂ xk
Now I will use this observation to prove the theorem. Let v = (v1 , · · · , vn ) with |v|
sufficiently small. Thus v = ∑nk=1 vk ek . For the purposes of this argument, define
n
∑ vk ek ≡ 0.
k=n+1

Then with this convention,

! !!
n n n
f (x + v) − f (x) = ∑ f x+ ∑ vk ek − f x+ ∑ vk ek
i=1 k=i k=i+1
!
n Z 1 n
∂f
= ∑ x+ ∑ vk ek + tvi ei vi dt
i=1 0 ∂ xi k=i+1

! !
n Z 1 n
∂f ∂f
= ∑ x+ ∑ vk ek + tvi ei vi − (x) vi dt
i=1 0 ∂ xi k=i+1 ∂ xi
n Z 1 n
∂f ∂f
+∑ (x) vi dt = ∑ (x) vi
i=1 0 ∂ xi i=1 ∂ xi

! !
n Z 1 n
∂f ∂f
+∑ x+ ∑ vk ek + tvi ei − (x) vi dt
i=1 0 ∂ xi k=i+1 ∂ xi
n
∂f
= ∑ ∂ xi (x) vi + o (v)
i=1

and this shows f is differentiable at x.

Some explanation of the step to the last line is in order. The messy thing at the end is
o (v) because of the continuity of the partial derivatives. To see this, consider one term. By
Proposition 14.2.2,
! !
Z 1 n
∂f ∂f
x+ ∑ vk ek + tvi ei − (x) vi dt
0 ∂ xi k=i+1 ∂ xi
!
Z 1 n
√ ∂f ∂f
≤ p x+ ∑ vk ek + tvi ei − (x) dt |v|
0 ∂ xi k=i+1 ∂ xi

Thus, dividing by |v| and taking a limit as |v| → 0, this converges to 0 due to continuity
of the partial derivatives of f . The messy term is thus a finite sum of o (v) terms and is
therefore o (v). ■
Here is an example to illustrate.
308 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES
!
x2 y + y2
Example 17.5.3 Let f (x, y) = . Find Df (x, y).
y3 x

From Theorem 17.5.2 this function is differentiable because all possible partial deriva-
tives are continuous. Thus
!
2xy x2 + 2y
Df (x, y) = .
y3 3y2 x

In particular, !
4 5
Df (1, 2) = .
8 12
Here is another example.
 
x12 x2 + x22
Example 17.5.4 Let f (x1 , x2 , x3 ) =  x2 x1 + x3 . Find Df (x1 , x2 , x3 ).
 

sin (x1 x2 x3 )

All possible partial derivatives are continuous, so the function is differentiable. The
matrix for this derivative is therefore the following 3 × 3 matrix
 
2x1 x2 x12 + 2x2 0
x2 x1 1
 
 
x2 x3 cos (x1 x2 x3 ) x1 x3 cos (x1 x2 x3 ) x1 x2 cos (x1 x2 x3 )

Example 17.5.5 Suppose f (x, y, z) = xy + z2 . Find D f (1, 2, 3).

Taking the partial derivatives of f , fx = y, fy = x, fz = 2z. These are all continuous.

Therefore, the function has a derivative and fx (1, 2, 3) = 1, fy (1, 2, 3) = 2, and fz (1, 2, 3) =
6. Therefore, D f (1, 2, 3) is given by

D f (1, 2, 3) = (1, 2, 6) .

Also, for (x, y, z) close to (1, 2, 3),

f (x, y, z) ≈ f (1, 2, 3) + 1 (x − 1) + 2 (y − 2) + 6 (z − 3)
= 11 + 1 (x − 1) + 2 (y − 2) + 6 (z − 3) = −12 + x + 2y + 6z

When a function is differentiable at x0 , it follows the function must be continuous there.

This is the content of the following important lemma.

Lemma 17.5.6 Let f : U → Rq where U is an open subset of R p . If f is differentiable at

x, then f is continuous at x.

Proof: From the definition of what it means to be differentiable,

|f (y) − f (x)| = |Df (x) (y − x) + o (y − x)|

≤ |Df (x) (y − x)| + |y − x|
17.5. C1 FUNCTIONS 309

provided |y − x| is sufficiently small. Letting M denote the matrix of Df (x) ,

2 !2
|Df (x) (y − x)|2 = ∑ ∑ Mi j (y j − x j ) ≤ |y − x| ∑ ∑ Mi j
i j i j

and so, for y close enough to x, there exists a constant C such that

|f (y) − f (x)| ≤ C |y − x|

which shows that f is continuous at x. ■

There have been quite a few terms defined. First there was the concept of continuity.
Next the concept of partial or directional derivative. Next there was the concept of differ-
entiability and the derivative being a linear transformation determined by a certain matrix.
Finally, it was shown that if a function is C1 , then it has a derivative. To give a rough idea
of the relationships of these topics, here is a picture.

derivative
Partial derivatives Continuous
xy C1 |x| + |y|
x2 +y2

You might ask whether there are examples of functions which are differentiable but
not C1 . Of course there are. In fact, Example 17.3.8 is just such an example as explained
earlier. Then you should verify that f ′ (x) exists for all x ∈ R but f ′ fails to be continuous
at x = 0. Thus the function is differentiable at every point of R but fails to be C1 because
the derivative is not continuous at 0.

Example 17.5.7 Find an example of a function which is not differentiable at (0, 0) even
though both partial derivatives exist at this point and the function is continuous at this
point.

Here is a simple example.

(  
1
x sin xy if xy ̸= 0
f (x, y) ≡
0 if xy = 0

To see this works, note that f is defined everywhere and

| f (x, y)| ≤ |x|

so clearly f is continuous at (0, 0).

f (x, 0) − f (0, 0) 0 − 0
= =0
x x
and so fx (0, 0) = 0. Similarly,

f (0, y) − f (0, 0) 0 − 0
= =0
y y
310 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

and so fy (0, 0) = 0. Thus the partial derivatives exist. However, the function is not differ-
entiable at (0, 0) because  
1
x sin xy
lim
(x,y)→(0,0) |(x, y)|

does not even exist, much less equals 0. To see this, let x = y and let x → 0.

17.6 The Chain Rule

17.6.1 The Chain Rule For Functions Of One Variable
First recall the chain rule for a function of one variable. Consider the following picture.
g f
I→J→R

Here I and J are open intervals and it is assumed that g (I) ⊆ J. The chain rule says that if
f ′ (g (x)) exists and g′ (x) exists for x ∈ I, then the composition, f ◦ g also has a derivative
at x and
( f ◦ g)′ (x) = f ′ (g (x)) g′ (x) .
Recall that f ◦ g is the name of the function defined by f ◦ g (x) ≡ f (g (x)). In the notation
of this chapter, the chain rule is written as

D f (g (x)) Dg (x) = D ( f ◦ g) (x) . (17.8)

17.6.2 The Chain Rule For Functions Of Many Variables

Let U ⊆ Rn and V ⊆ R p be open sets and let f be a function defined on V having values in
Rq while g is a function defined on U such that g (U) ⊆ V as in the following picture.
g f
U → V → Rq

The chain rule says that if the linear transformations (matrices) on the left in (17.8) both
exist then the same formula holds in this more general case. Thus

Df (g (x)) Dg (x) = D (f ◦ g) (x)

Note this all makes sense because Df (g (x)) is a q × p matrix and Dg (x) is a p × n matrix.
Remember it is all right to do (q × p) (p × n). The middle numbers match. More precisely,

Theorem 17.6.1 (Chain rule) Let U be an open set in Rn , let V be an open set in R p , let
g : U → R p be such that g (U) ⊆ V , and let f : V → Rq . Suppose Dg (x) exists for some
x ∈ U and that Df (g (x)) exists. Then D (f ◦ g) (x) exists and furthermore,

D (f ◦ g) (x) = Df (g (x)) Dg (x) . (17.9)

In particular,
p
∂ (f ◦ g) (x) ∂ f (g (x)) ∂ gi (x)
=∑ . (17.10)
∂xj i=1 ∂ yi ∂xj
17.6. THE CHAIN RULE 311

There is an easy way to remember this in terms of the repeated index summation con-
vention presented earlier. Let y = g (x) and z = f (y). Then the above says

∂ z ∂ yi ∂z
= . (17.11)
∂ yi ∂ xk ∂ xk
Remember there is a sum on the repeated index. In particular, for each index r,

∂ zr ∂ yi ∂ zr
= .
∂ yi ∂ xk ∂ xk
The proof of this major theorem will be given later. It will include the chain rule for
functions of one variable as a special case. First here are some examples.

Example 17.6.2 Let f (u, v) = sin (uv) and let u (x, y,t) = t sin x + cos y and v (x, y,t, s) =
s tan x + y2 + ts. Letting z = f (u, v) where u, v are as just described, find ∂∂tz and ∂∂ xz .

∂z ∂z ∂u
From (17.11), ∂t = ∂ u ∂t + ∂∂ vz ∂∂tv = v cos (uv) sin (x) + us cos (uv) . Here y1 = u, y2 =
v,t = xk . Also,

∂z ∂z ∂u ∂z ∂v
= + = v cos (uv)t cos (x) + us sec2 (x) cos (uv) .
∂x ∂u ∂x ∂v ∂x
Clearly you can continue in this way, taking partial derivatives with respect to any of the
other variables.

Example 17.6.3 Let w = f (u1 , u2 ) = u2 sin (u1 ) and u1 = x2 y + z, u2 = sin (xy). Find
∂w ∂w ∂w
∂ x , ∂ y , and ∂ z .

The derivative of f is of the form (wx , wy , wz ) and so it suffices to find the derivative of
f using the chain rule. You need to find D f (u1 , u2 ) Dg (x, y, z) where
!
x2 y + z
g (x, y) = .
sin (xy)

Then !
2xy x2 1
Dg (x, y, z) = .
y cos (xy) x cos (xy) 0
Also D f (u1 , u2 ) = (u2 cos (u1 ) , sin (u1 )). Therefore, the derivative is

D f (u1 , u2 ) Dg (x, y, z)
!
2xy x2 1
= (u2 cos (u1 ) , sin (u1 ))
y cos (xy) x cos (xy) 0

= 2u2 (cos u1 ) xy + (sin u1 ) y cos xy, u2 (cos u1 ) x2

+ (sin u1 ) x cos xy, u2 cos u1 )
= (wx , wy , wz )
312 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

Thus
∂w
= 2u2 (cos u1 ) xy + (sin u1 ) y cos xy
∂x
= 2 (sin (xy)) cos x2 y + z xy

+ sin x2 y + z y cos xy.

Similarly, you can find the other partial derivatives of w in terms of substituting in for u1
and u2 in the above. Note
∂w ∂ w ∂ u1 ∂ w ∂ u2
= + .
∂x ∂ u1 ∂ x ∂ u2 ∂ x
In fact, in general if you have w = f (u1 , u2 ) and
!
u1 (x, y, z)
g (x, y, z) =
u2 (x, y, z)

then D ( f ◦ g) (x, y, z) is of the form

!
  u1x u1y u1z
wu1 wu2
u2x u2y u2z
 
= wu1 ux + wu2 u2x wu1 uy + wu2 u2y wu1 uz + wu2 u2z .

Example 17.6.4 Let w = f (u1 , u2 , u3 ) = u21 + u3 + u2 and

   
u1 x + 2yz
g (x, y, z) =  u2  =  x2 + y 
   

u3 z2 + x

∂w ∂w
Find ∂x and ∂z .

By the chain rule,

 
  u1x u1y u1z
(wx , wy , wz ) = wu1 wu2 wu3  u2x u2y u2z  =
 

u3x u3y u3z

wu1 u1x + wu2 u2x + wu3 u3x , wu1 u1y + wu2 u2y + wu3 u3y ,


wu1 u1z + wu2 u2z + wu3 u3z

Note the pattern,

wx = wu1 u1x + wu2 u2x + wu3 u3x ,

wy = wu1 u1y + wu2 u2y + wu3 u3y ,
wz = wu1 u1z + wu2 u2z + wu3 u3z .
17.6. THE CHAIN RULE 313

Therefore,

wx = 2u1 (1) + 1 (2x) + 1 (1) = 2 (x + 2yz) + 2x + 1 = 4x + 4yz + 1

and
wz = 2u1 (2y) + 1 (0) + 1 (2z) = 4 (x + 2yz) y + 2z = 4yx + 8y2 z + 2z.
Of course to find all the partial derivatives at once, you just use the chain rule. Thus you
would get
 
wx wy wz
 
  1 2z 2y
= 2u1 1 1  2x 1 0 
 

1 0 2z
 
= 2u1 + 2x + 1 4u1 z + 1 4u1 y + 2z
 
= 4x + 4yz + 1 4zx + 8yz2 + 1 4yx + 8y2 z + 2z
!
u21 + u2
Example 17.6.5 Let f (u1 , u2 ) = and
sin (u2 ) + u1
! !
u1 (x1 , x2 , x3 ) x1 x2 + x3
g (x1 , x2 , x3 ) = = .
u2 (x1 , x2 , x3 ) x22 + x1

Find D (f ◦ g) (x1 , x2 , x3 ).

To do this,
!
2u1 1
Df (u1 , u2 ) = ,
1 cos u2
!
x2 x1 1
Dg (x1 , x2 , x3 ) = .
1 2x2 0

Then !
2 (x1 x2 + x3 ) 1
Df (g (x1 , x2 , x3 )) =
cos x22 + x1


1
and so by the chain rule,

D (f ◦ g) (x1 , x2 , x3 )
Df (g (x)) Dg (x)
z }| !{z }| !{
2 (x1 x2 + x3 ) 1 x2 x1 1
=
cos x22 + x1


1 1 2x2 0
!
(2x1 x2 + 2x3 ) x2 + 1 (2x1 x2 + 2x3 ) x1 + 2x2 2x1 x2 + 2x3
=
x2 + cos x22 + x1 x1 + 2x2 cos x22 + x1




1
314 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

Therefore, in particular,
∂ f1 ◦ g
(x1 , x2 , x3 ) = (2x1 x2 + 2x3 ) x2 + 1,
∂ x1
∂ f2 ◦ g ∂ f2 ◦ g
(x1 , x2 , x3 ) = x1 + 2x2 cos x22 + x1 .



(x1 , x2 , x3 ) = 1,
∂ x3 ∂ x2
etc. ! !
z1 u21 + u2
In different notation, let = f (u1 , u2 ) = . Then
z2 sin (u2 ) + u1

∂ z1 ∂ z1 ∂ u1 ∂ z1 ∂ u2
= +
∂ x1 ∂ u1 ∂ x1 ∂ u2 ∂ x1
= 2u1 x2 + 1 = 2 (x1 x2 + x3 ) x2 + 1.

Example 17.6.6 Let

   
z1 u21 + u2 u3
f (u1 , u2 , u3 ) =  z2  =  u21 + u32 
   

ln 1 + u23


z3

and let    
u1 x1 + x22 + sin (x3 ) + cos (x4 )
g (x1 , x2 , x3 , x4 ) =  u2  =  x42 − x1 .
   

u3 2
x3 + x4
Find (f ◦ g)′ (x).
 
2u1 u3 u2
3u22
 
Df (u) =  2u 0
 1

2u3

0 0
(1+u23 )
Similarly,  
1 2x2 cos (x3 ) − sin (x4 )
Dg (x) =  −1 0 0 2x4 .
 

0 0 2x3 1
Then by the chain rule, D (f ◦ g) (x) = Df (u) Dg (x) where u = g (x) as described above.
Thus D (f ◦ g) (x) =
  
2u1 u3 u2 1 2x2 cos (x3 ) − sin (x4 )
 2u1 3u22
 
0   −1 0 0 2x4


2u3
 
0 0 0 0 2x 1
(1+u2 ) 3
3

 
2u1 − u3 4u1 x2 2u1 cos x3 + 2u2 x3 −2u1 sin x4 + 2u3 x4 + u2
 2u − 3u2 4u1 x2 2u1 cos x3 −2u1 sin x4 + 6u22 x4
= (17.12)

1 2 
u3 u3
0 0 4 1+u 2 x3 2 1+u2
3 3
17.7. EXERCISES 315

∂ z1
where each ui is given by the above formulas. Thus ∂ x1 equals

= 2 x1 + x22 + sin (x3 ) + cos (x4 ) − x32 + x4

2u1 − u3
= 2x1 + 2x22 + 2 sin x3 + 2 cos x4 − x32 − x4 .
∂ z2
while ∂ x4 equals

2
−2u1 sin x4 + 6u22 x4 = −2 x1 + x22 + sin (x3 ) + cos (x4 ) sin (x4 ) + 6 x42 − x1 x4 .

∂z ∂z
If you wanted ∂ x2 it would be the second column of the above matrix in (17.12). Thus ∂ x2
equals
 ∂ z1    

∂ x2 4u1 x2 4 x1 + x22 + sin (x3 ) + cos (x4 ) x2
∂ z2
 =  4u1 x2  =  4 x1 + x22 + sin (x3 ) + cos (x4 ) x2 
    

 ∂ x2
∂ z3
∂ x2
0 0

I hope that by now it is clear that all the information you could desire about various partial
derivatives is available and it all reduces to matrix multiplication and the consideration of
entries of the matrix obtained by multiplying the two derivatives.

17.7 Exercises
∂z
1. Let z = f (x1 , · · · , xn ) be as given and let xi = gi (t1 , · · · ,tm ) as given. Find ∂ti which
is indicated.
∂z
(a) z = x13 + x2 , x1 = sin (t1 ) + cos (t2 ) , x2 = t1t22 . Find ∂t1
∂z
(b) z = x1 x22 , x1 = t1t22t3 , x2 = t1t22 . Find ∂t1 .
∂z
(c) z = x1 x22 , x1 = t1t22t3 , x2 = t1t22 . Find ∂t1 .
∂z
(d) z = x1 x22 , x1 = t1t22t3 , x2 = t1t22 . Find ∂t3 .
∂z
(e) z = x12 x22 , x1 = t1t22t3 , x2 = t1t22 . Find ∂t2 .
∂z
(f) z = x12 x2 + x32 , x1 = t1t2 , x2 = t1t2t4 , x3 = sin (t3 ). Find ∂t2 .
∂z
(g) z = x12 x2 + x32 , x1 = t1t2 , x2 = t1t2t4 , x3 = sin (t3 ). Find ∂t3 .
∂z
(h) z = x12 x2 + x32 , x1 = t1t2 , x2 = t1t2t4 , x3 = sin (t3 ). Find ∂t1 .

2. Let z = f (y) = y21 + sin y2 + tan y3 and

 
x1 + x2
y = g (x) ≡  x22 − x1 + x2  .
 

x22 + x1 + sin x2

∂z
Find D ( f ◦ g) (x). Use to write ∂ xi for i = 1, 2.
316 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES
 
x1 + x4 + x3
3. Let z = f (y) = y21 + cot y2 + sin y3 and y = g (x) ≡  x22 − x1 + x2 . Find

 

x22 + x1 + sin x4
∂z
D ( f ◦ g) (x). Use to write ∂ xi for i = 1, 2, 3, 4.
 
x1 + x4 + x3
 x22 − x1 + x2 
4. Let z = f (y) = y21 + y22 + sin y3 + y4 and y = g (x) ≡ 


. Find
 
 x22 + x1 + sin x4 
x4 + x2
∂z
D ( f ◦ g) (x). Use to write ∂ xi for i = 1, 2, 3, 4.
5. Let !
y21 + sin y2 + tan y3
z = f (y) =
y21 y2 + y3
 
x1 + x2
∂ zk
and y = g (x) ≡  x22 − x1 + x2 . Find D (f ◦ g) (x). Use to write for
 
∂ xi
x22 + x1 + sin x2
!
z1x1 z1x2 z1x3
i = 1, 2 and k = 1, 2. Recall this will be of the form .
z2x1 z2x2 z2x3
 
y21 + sin y2 + tan y3
6. Let z = f (y) =  y21 y2 + y3  and
 
2 3


cos y1 + y2 y3
 
x1 + x4
y = g (x) ≡  x22 − x1 + x3  .
 

x32 + x1 + sin x2

∂ zk
Find D (f ◦ g) (x). Use to write ∂ xi for i = 1, 2, 3, 4 and k = 1, 2, 3.
7. Give a version of the chain rule which involves three functions f, g, h.

8. If f : U → V and f −1 : V → U for U,V open sets such that f, f −1 are both differen-
tiable, show that
det Df f −1 (y) det Df −1 (y) = 1

17.7.1 Related Rates Problems

Sometimes several variables are related and, given information about how one variable is
changing, you want to find how the others are changing.

Example 17.7.1 Bernoulli’s law states that in an incompressible fluid,

v2 P
+z+ =C
2g γ
17.7. EXERCISES 317

where C is a constant. Here v is the speed, P is the pressure, and z is the height above
some reference point. The constants g and γ are the acceleration of gravity and the weight
density of the fluid. Suppose measurements indicate that dv dz
dt = −3, and dt = 2. Find dt
dP

when v = 7 and z = 8 in terms of g and γ.

This is just an exercise in using the chain rule. Differentiate the two sides with respect
to t.
1 dv dz 1 dP
v + + = 0.
g dt dt γ dt
Then when v = 7 and z = 8, finding dP
dt involves nothing more than solving the following
for dP
dt .
7 1 dP
(−3) + 2 + =0
g γ dt
Thus  
dP 21
=γ −2
dt g
at this instant in time.
Example 17.7.2 In Bernoulli’s law above, each of v, z, and P are functions of (x, y, z), the
position of a point in the fluid. Find a formula for ∂∂ Px in terms of the partial derivatives of
the other variables.
This is an example of the chain rule. Differentiate both sides with respect to x.
v 1
vx + zx + Px = 0
g γ
and so  
vvx + zx g
Px = − γ
g
Example 17.7.3 Suppose a level curve is of the form f (x, y) = C and that near a point on
dy
this level curve y is a differentiable function of x. Find dx .
This is an example of the chain rule. Differentiate both sides with respect to x. This
gives
dy
fx + fy = 0.
dx
dy
Solving for dx gives
dy − fx (x, y)
= .
dx fy (x, y)
Example 17.7.4 Suppose a level surface is of the form f (x, y, z) = C. and that near a point
(x, y, z) on this level surface z is a C1 function of x and y. Find a formula for zx .
This is an example of the use of the chain rule. Differentiate both sides of the equation
with respect to x. Since yx = 0,
fx + fz zx = 0.
Then solving for zx ,
− fx (x, y, z)
zx =
fz (x, y, z)
318 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

Example 17.7.5 Polar coordinates are

x = r cos θ , y = r sin θ . (17.13)
Thus if f is a C1
scalar valued function you could ask to express fx in terms of the variables
r and θ . Do so.
This is an example of the chain rule. Abusing notation slightly, regard f as a function
of position in the plane. This position can be described with any set of coordinates. Thus
f (x, y) = f (r, θ ) and so
fx = fr rx + fθ θ x .
This will be done if you can find rx and θ x . However you must find these in terms of r and
θ , not in terms of x and y. Using the chain rule on the two equations for the transformation
in (17.13),
1 = rx cos θ − (r sin θ ) θ x , 0 = rx sin θ + (r cos θ ) θ x
Solving these using Cramer’s rule,
− sin (θ )
rx = cos (θ ) , θ x =
r
Hence fx in polar coordinates is
 
sin (θ )
fx = fr (r, θ ) cos (θ ) − fθ (r, θ )
r

17.7.2 The Derivative Of The Inverse Function

Example 17.7.6 Let f : U → V where U and V are open sets in Rn and f is one to one
and onto. Suppose also that f and f −1 are both differentiable. How are Df −1 and Df
related?
This can be done as follows. From the assumptions, x = f −1 (f (x)). Let Ix = x.
Then by Example 17.3.6 on Page 303 DI = I. By the chain rule,
I = DI = Df −1 (f (x)) (Df (x)) , I = DI = Df f −1 (y) Df −1 (y)

Letting y = f (x), the second yields

I = Df (x) Df −1 (f (x)) .
Therefore,
Df (x)−1 = Df −1 (f (x)) .
This is equivalent to

−1
Df f −1 (y) = Df −1 (y)
or
Df (x)−1 = Df −1 (y) , y = f (x) .
This is just like a similar situation for functions of one variable. Remember

′
f −1 ( f (x)) = 1/ f ′ (x) .
In terms of the repeated index summation convention, suppose y = f (x) so that x = f −1 (y).
Then the above can be written as
∂ xi ∂ yk
δij = (f (x)) (x) .
∂ yk ∂xj
17.7. EXERCISES 319

17.7.3 Proof Of The Chain Rule

As in the case of a function of one variable, it is important to consider the derivative of
a composition of two functions. As in the case of a function of one variable, this rule is
called the chain rule. Its proof depends on the following fundamental lemma. This proof
will include the one dimensional case. First let M be a matrix and v a vector of length 1.
Then ! 2 ! 2
|Mv|2 = ∑ ∑ Mi j v j ≤∑ ∑ Mi j <∞
i j i j

Here is the rough idea of the following lemma.

→0 as v →0 bounded
z }| {z }| {
|o (g (x + v) − g (x))| |o (g (x + v) − g (x))| |g (x + v) − g (x)|
=
|v| |g (x + v) − g (x)| |v|
Lemma 17.7.7 Let g : U → R p where U is an open set in Rn and suppose g has a deriva-
tive at x ∈ U. Then o (g (x + v) − g (x)) = o (v).

Proof: Let
|o(g (x+v )−g (x))|
(
|g (x+v )−g (x)| if g (x + v) − g (x) ̸= 0
H (v) ≡
0 if g (x + v) − g (x) = 0

Then limv→0 H (v) = 0 because of continuity of g at x and

|o (g (x + v) − g (x))| |g (x + v) − g (x)|
= H (v)
|v| |v|
Also
 
|g (x + v) − g (x)| |Dg (x) v| |o (v)| v |o (v)|
≤ + = Dg (x) +
|v| |v| |v| |v| |v|
which is bounded for small v. Therefore,
|o (g (x + v) − g (x))|
lim = 0. ■
v →0 |v|
Recall the notation f ◦ g (x) ≡ f (g (x)). Thus f ◦ g is the name of a function, and this
function is defined by what was just written. The following theorem is known as the chain
rule.

Theorem 17.7.8 (Chain rule) Let U be an open set in Rn , let V be an open set in R p , let
g : U → R p be such that g (U) ⊆ V , and let f : V → Rq . Suppose Dg (x) exists for some
x ∈ U and that Df (g (x)) exists. Then D (f ◦ g) (x) exists and furthermore,

D (f ◦ g) (x) = Df (g (x)) Dg (x) . (17.14)

In particular, If y = g (x) so yi = gi (x),

p
∂ (f ◦ g) (x) ∂ f (g (x)) ∂ gi (x)
=∑ . (17.15)
∂xj i=1 ∂ yi ∂xj
320 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

Proof: From the assumption that Df (g (x)) exists,

f (g (x + v)) = f (g (x)) + Df (g (x)) (g (x + v) − g (x)) + o (g (x + v) − g (x))

= f (g (x)) + Df (g (x)) (Dg (x) v + o (v)) + o (g (x + v) − g (x))

which by Lemma 17.7.7 equals

= f (g (x)) + Df (g (x)) Dg (x) v + Df (g (x)) o (v) + o (v)

= f (g (x)) + Df (g (x)) Dg (x) v + o (v)

and this shows

D (f ◦ g) (x) = Df (g (x)) Dg (x)
from the definition of the derivative and its uniqueness established in Theorem 17.3.3 on
Page 301. ■

17.8 Exercises
1. Suppose f : U → Rq and let x ∈ U and v be a unit vector. Show that Dv f (x) =
Df (x) v. Recall that

f (x + tv) − f (x)
Dv f (x) ≡ lim .
t→0 t
(
xy sin 1x if x ̸= 0


2. Let f (x, y) = . Find where f is differentiable and compute the
0 if x = 0
derivative at all these points.
3. Let (
x if |y| > |x|
f (x, y) = .
−x if |y| ≤ |x|
Show that f is continuous at (0, 0) and that the partial derivatives exist at (0, 0) but
the function is not differentiable at (0, 0).
4. Let !
x2 sin y + z3
f (x, y, z) = .
sin (x + y) + z3 cos x
Find Df (1, 2, 3).

5. Let !
x tan y + z3
f (x, y, z) = .
cos (x + y) + z3 cos x
Find Df (x, y, z).
17.8. EXERCISES 321

6. Let  
x sin y + z3
f (x, y, z) =  sin (x + y) + z3 cos x  .
 

x 5 + y2
Find Df (x, y, z).
7. Let  2 4 2
 (x −y ) if (x, y) ̸= (0, 0)
2
f (x, y) = (x2 +y4 ) .
1 if (x, y) = (0, 0)


Show that all directional derivatives of f exist at (0, 0), and are all equal to zero but
the function is not even continuous at (0, 0). Therefore, it is not differentiable. Why?
8. In the example of Problem 7 show that the partial derivatives exist but are not con-
tinuous.
2 y 2 2
x z
9. A certain building is shaped like the top half of the ellipsoid, 900 + 900 + 400 =1
determined by letting z ≥ 0. Here dimensions are measured in feet. The building
needs to be painted. The paint, when applied is about .005 feet thick. About how
many cubic feet of paint will be needed. Hint: This is going to replace the numbers,
900 and 400 with slightly larger numbers when the ellipsoid is fattened slightly by
the paint. The volume of the top half of the ellipsoid, x2 /a2 +y2 /b2 +z2 /c2 ≤ 1, z ≥ 0
is (2/3) πabc.
10. Suppose r 1 (t) = (cost, sint,t) , r 2 (t) = (t, 2t, 1), and r 3 (t) = (1,t, 1). Find the rate
of change with respect to t of the volume of the parallelepiped determined by these
three vectors when t = 1.
11. A trash compactor is compacting a rectangular block of trash. The width is changing
at the rate of −1 inches per second, the length is changing at the rate of −2 inches
per second and the height is changing at the rate of −3 inches per second. How fast
is the volume changing when the length is 20, the height is 10, and the width is 10?
12. A trash compactor is compacting a rectangular block of trash. The width is changing
at the rate of −2 inches per second, the length is changing at the rate of −1 inches
per second and the height is changing at the rate of −4 inches per second. How fast
is the surface area changing when the length is 20, the height is 10, and the width is
10?
13. The ideal gas law is PV = kT where k is a constant which depends on the number of
moles and on the gas being considered. If V is changing at the rate of 2 cubic cm.
per second and T is changing at the rate of 3 degrees Kelvin per second, how fast is
the pressure changing when T = 300 and V equals 400 cubic cm.?
14. Let S denote a level surface of the form f (x1 , x2 , x3 ) = C. Show that any smooth
curve in the level surface is perpendicular to the gradient.
15. Suppose f is a C1 function which maps U, an open subset of Rn one to one and onto
V , an open set in Rm such that the inverse map, f −1 is also C1 . What must be true of
m and n? Why? Hint: Consider Example 17.7.6 on Page 318. Also you can use the
fact that if A is an m × n matrix which maps Rn onto Rm , then m ≤ n.
322 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

16. Finish Example 17.7.5 by finding fy in terms of θ , r. Show that fy = sin (θ ) fr +

cos(θ )
r fθ .
17. ∗ Think of ∂x as a differential operator which takes functions and differentiates them
with respect to x. Thus ∂x f ≡ fx . In the context of Example 17.7.5, which is on polar
coordinates, and Problem 16, explain how
sin (θ )
∂x = cos (θ ) ∂r − ∂θ
r
cos (θ )
∂y = sin (θ ) ∂r + ∂θ
r
The Laplacian of a function u is defined as ∆u = uxx + uyy . Use the above observation
to give a formula ∆u in terms of r and θ . You should get urr + 1r ur + r12 uθ θ . This is
the formula for the Laplacian in polar coordinates.

17.9 The Gradient

Here we review the concept of the gradient and the directional derivative and prove the
formula for the directional derivative discussed earlier.
Let f : U → R where U is an open subset of Rn and suppose f is differentiable on U.
Thus if x ∈ U,
n
∂ f (x)
f (x + v) = f (x) + ∑ vi + o (v) . (17.16)
j=1 ∂ xi

Now we can prove the formula for the directional derivative in terms of the gradient.

Proposition 17.9.1 If f is differentiable at x and for v a unit vector

Dv f (x) = ∇ f (x) · v. (17.17)

Proof:
!
n
f (x+tv) − f (x) 1 ∂ f (x)
= f (x) + ∑ tvi + o (tv) − f (x)
t t j=1 ∂ xi
!
n n
1 ∂ f (x) ∂ f (x) o (tv)
= ∑ tvi + o (tv) = ∑ vi +
t j=1 ∂ xi j=1 ∂ xi t

Now limt→0 o(ttv) = 0 and so

n
f (x+tv) − f (x) ∂ f (x)
Dv f (x) = lim = ∑ vi = ∇ f (x) · v
t→0 t j=1 ∂ xi

as claimed. ■

Example 17.9.2 Let f (x, y, z) = x2 + sin (xy) + z. Find Dv f (1, 0, 1) where

 
1 1 1
v= √ ,√ ,√ .
3 3 3
17.9. THE GRADIENT 323

Note this vector which is given is already a unit vector. Therefore, from the above, it is
only necessary to find ∇ f (1, 0, 1) and take the dot product.

∇ f (x, y, z) = (2x + (cos xy) y, (cos xy) x, 1) .

Therefore, ∇ f (1, 0, 1) = (2, 1, 1). Therefore, the directional derivative is

4√
 
1 1 1
(2, 1, 1) · √ , √ , √ = 3.
3 3 3 3
Because of (17.17) it is easy to find the largest possible directional derivative and the
smallest possible directional derivative. That which follows is a more algebraic treatment
of an earlier result with the trigonometry removed.

Proposition 17.9.3 Let f : U → R be a differentiable function and let x ∈ U. Then

max {Dv f (x) : |v| = 1} = |∇ f (x)| (17.18)

and
min {Dv f (x) : |v| = 1} = − |∇ f (x)| . (17.19)
Furthermore, the maximum in (17.18) occurs when v = ∇ f (x) / |∇ f (x)| and the minimum
in (17.19) occurs when v = −∇ f (x) / |∇ f (x)|.

Proof: From (17.17) and the Cauchy Schwarz inequality,

|Dv f (x)| ≤ |∇ f (x)|

and so for any choice of v with |v| = 1,

− |∇ f (x)| ≤ Dv f (x) ≤ |∇ f (x)| .

The proposition is proved by noting that if v = −∇ f (x) / |∇ f (x)|, then

Dv f (x) = ∇ f (x) · (−∇ f (x) / |∇ f (x)|)

= − |∇ f (x)|2 / |∇ f (x)| = − |∇ f (x)|

while if v = ∇ f (x) / |∇ f (x)|, then

Dv f (x) = ∇ f (x) · (∇ f (x) / |∇ f (x)|)

= |∇ f (x)|2 / |∇ f (x)| = |∇ f (x)| . ■

For a different approach to the proposition, see Problem 7 which follows.

The conclusion of the above proposition is important in many physical models. For
example, consider some material which is at various temperatures depending on location.
Because it has cool places and hot places, it is expected that the heat will flow from the
hot places to the cool places. Consider a small surface having a unit normal n. Thus n is
a normal to this surface and has unit length. If it is desired to find the rate in calories per
second at which heat crosses this little surface in the direction of n it is defined as J · nA
where A is the area of the surface and J is called the heat flux. It is reasonable to suppose
the rate at which heat flows across this surface will be largest when n is in the direction of
greatest rate of decrease of the temperature. In other words, heat flows most readily in the
324 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

direction which involves the maximum rate of decrease in temperature. This expectation
will be realized by taking J = −K∇u where K is a positive scalar function which can
depend on a variety of things. The above relation between the heat flux and ∇u is usually
called the Fourier heat conduction law and the constant K is known as the coefficient of
thermal conductivity. It is a material property, different for iron than for aluminum. In
most applications, K is considered to be a constant but this is wrong. Experiments show
that this scalar should depend on temperature. Nevertheless, things get very difficult if this
dependence is allowed. The constant can depend on position in the material or even on
time.
An identical relationship is usually postulated for the flow of a diffusing species. In this
problem, something like a pollutant diffuses. It may be an insecticide in ground water for
example. Like heat, it tries to move from areas of high concentration toward areas of low
concentration. In this case J = −K∇c where c is the concentration of the diffusing species.
When applied to diffusion, this relationship is known as Fick’s law. Mathematically, it is
indistinguishable from the problem of heat flow.
Note the importance of the gradient in formulating these models.

17.10 The Gradient And Tangent Planes

The gradient has fundamental geometric significance illustrated by the following picture.

∇ f (x0 , y0 , z0 )
x′1 (t0 )

x′2 (s0 )

In this picture, the surface is a piece of a level surface of a function of three variables
f (x, y, z). Thus the surface is defined by f (x, y, z) = c or more completely as

{(x, y, z) : f (x, y, z) = c}

For example, if f (x, y, z) = x2 + y2 + z2 , this would be a piece of a sphere. There are two
smooth curves in this picture which lie in the surface having parameterizations, x1 (t) =
(x1 (t) , y1 (t) , z1 (t)) and x2 (s) = (x2 (s) , y2 (s) , z2 (s)) which intersect at the point (x0 , y0 , z0 )
on this surface.1 This intersection occurs when t = t0 and s = s0 . Since the points x1 (t) for
t in an interval lie in the level surface, it follows

f (x1 (t) , y1 (t) , z1 (t)) = c

1 Do there exist any smooth curves which lie in the level surface of f and pass through the point (x , y , z )? It
0 0 0
turns out there do if ∇ f (x0 , y0 , z0 ) ̸= 0 and if the function f , is C1 . However, this is a consequence of the implicit
function theorem, one of the greatest theorems in all mathematics and a topic for an advanced calculus class. An
elementary presentation is presented later.
17.10. THE GRADIENT AND TANGENT PLANES 325

for all t in some interval. Therefore, taking the derivative of both sides and using the chain
rule on the left,
∂f
(x1 (t) , y1 (t) , z1 (t)) x1′ (t) +
∂x
∂f ∂f
(x1 (t) , y1 (t) , z1 (t)) y′1 (t) + (x1 (t) , y1 (t) , z1 (t)) z′1 (t) = 0.
∂y ∂z
In terms of the gradient, this merely states
∇ f (x1 (t) , y1 (t) , z1 (t)) · x′1 (t) = 0.
Similarly,
∇ f (x2 (s) , y2 (s) , z2 (s)) · x′2 (s) = 0.
Letting s = s0 and t = t0 , it follows
∇ f (x0 , y0 , z0 ) · x′1 (t0 ) = 0, ∇ f (x0 , y0 , z0 ) · x′2 (s0 ) = 0.
It follows ∇ f (x0 , y0 , z0 ) is perpendicular to both the direction vectors of the two indicated
curves shown. Surely if things are as they should be, these two direction vectors would
determine a plane which deserves to be called the tangent plane to the level surface of f
at the point (x0 , y0 , z0 ) and that ∇ f (x0 , y0 , z0 ) is perpendicular to this tangent plane at the
point (x0 , y0 , z0 ).
Example 17.10.1 Find the equation of the tangent plane to the level surface
f (x, y, z) = 6
of the function f (x, y, z) = x2 + 2y2 + 3z2 at the point (1, 1, 1).
First note that (1, 1, 1) is a point on this level surface. To find the desired plane it
suffices to find the normal vector to the proposed plane. But ∇ f (x, y, z) = (2x, 4y, 6z) and so
∇ f (1, 1, 1) = (2, 4, 6). Therefore, from this problem, the equation of the plane is (2, 4, 6) ·
(x − 1, y − 1, z − 1) = 0 or in other words, 2x − 12 + 4y + 6z = 0.
√
3, 1, 4 is on both the surfaces, z = x2 + y2 and z = 8 −


Example 17.10.2 The point
x2 + y2 . Find the cosine of the angle between the two tangent planes at this point.

Recall this is the same as the angle between two normal vectors. Of course there is
some ambiguity here because if n is a normal vector, then so is −n and replacing n with
−n in the formula for the cosine of the angle will change the sign. We agree √to look for

the acute angle and its cosine rather than the obtuse angle. The normals are 2 3, 2, −1
√

and 2 3, 2, 1 . Therefore, the cosine of the angle desired is
√
2
2 3 + 4 − 1 15
= .
17 17
√

Example 17.10.3 The point 1, 3, 4 is on the surface z = x2 + y2 . Find the line perpen-
dicular to the surface at this point.
All that is needed is the direction vector of this line. The surface is the level surface
x2 + y2 − z = 0. The normal to this surface is given by the gradient at this point. Thus the
desired line is  √   √ 
1, 3, 4 + t 2, 2 3, −1 .
326 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES

17.11 Exercises
1. Find the gradient of f =

(a) x2 y + z3 at (1, 1, 2)
(b) z sin x2 y + 2x+y at (1, 1, 0)

(c) u ln x + y + z2 + w at (x, y, z, w, u) = (1, 1, 1, 1, 2)

(d) sin (xy) + z3 at (1, π, 1)

(e) ln x + y2 z

(f) z ln (4 + sin (xy)) at the point (0, π, 1)

 
1 1 √1
2. Find the directional derivatives of f at the indicated point in the direction 2, 2, 2 .

(a) x2 y + z3 at (1, 1, 1)
(b) z sin x2 y + 2x+y at (1, 1, 0)

(c) xy + z2 + 1 at (1, 2, 3)
(d) sin (xy) + z at (0, 1, 1)
(e) xy + z at (1, 1, 1).
(f) sin (sin (x + y)) + z at the point (1, 0, 1).

3. Find the directional derivatives of the given function at the indicated point in the
indicated direction.

(a) sin x2 + y + z2 at (0, π/2, 1) in direction of (1, 1, 2).

(b) x(x+y) + sin (zx) at (1, 0, 0) in the direction of (2, −1, 0).
(c) zsin(x) + y at (0, 1, 1) in the direction of (1, 1, 3).

4. Find the tangent plane to the indicated level surface at the indicated point.

(a) x2 y + z3 = 2 at (1, 1, 1)
(b) z sin x2 y + 2x+y = 2 sin 1 + 4 at (1, 1, 2)

(c) cos (x) + z sin (x + y) = 1 at −π, 3π

2 ,2
 √ 
5. The point 1, 1, 2 is a point on the level surface x2 + y2 + z2 = 4. Find the line
perpendicular to the surface at this point.
√ √ 
6. The level surfaces x2 + y2 + z2 = 4 and z + x2 + y2 = 4 have the point 22 , 22 , 1 in
the curve formed by the intersection of these surfaces. Find a direction vector for this
curve at this point. Hint: Recall the gradients of the two surfaces are perpendicular
to the corresponding surfaces at this point. A direction vector for the desired curve
should be perpendicular to both of these gradients.
17.11. EXERCISES 327

7. For v a unit vector, recall that Dv f (x) = ∇ f (x) · v. It was shown above that the
largest directional derivative is in the direction of the gradient and the smallest in the
direction of −∇ f . Establish the same result using the geometric description of the
dot product, the one which says the dot product is the product of the lengths of the
vectors times the cosine of the included angle.
 √ 
8. The point 1, 1, 2 is on the level surface x2 + y2 + z2 = 4 and the level surface
y2 + 2z2 = 5. Find an equation for the line tangent to the curve of intersection of
these two surfaces at this point.
9. ∗ In a slightly more general setting, suppose f1 (x, y, z) = 0 and f2 (x, y, z) = 0 are two
level surfaces which intersect in a curve which has parametrization, (x (t) , y (t) , z (t)).
Find a system of differential equations for (x(t), y(t), z(t)) where as t varies, the point
determined by (x(t), y(t), z(t)) moves over the curve.
328 CHAPTER 17. THE DERIVATIVE OF A FUNCTION OF MANY VARIABLES
Chapter 18

Optimization

18.1 Local Extrema

The following definition describes what is meant by a local maximum or local minimum.

Definition 18.1.1 Suppose f : D ( f ) → R where D ( f ) ⊆ Rn . A point x ∈ D ( f ) ⊆ Rn is

called a local minimum if f (x) ≤ f (y) for all y ∈ D ( f ) sufficiently close to x. A point
x ∈ D ( f ) is called a local maximum if f (x) ≥ f (y) for all y ∈ D ( f ) sufficiently close
to x. A local extremum is a point of D ( f ) which is either a local minimum or a local
maximum. The plural for extremum is extrema. The plural for minimum is minima and the
plural for maximum is maxima.

PROCEDURE 18.1.2 To find candidates for local extrema which are interior
points of D ( f ) where f is a differentiable function, you simply identify those points where
∇ f equals the zero vector.

To locate candidates for local extrema, for the function f , take ∇ f and find where this
vector equals 0.
Let v be any vector in Rn and suppose x is a local maximum (minimum) for f . Then
consider the real valued function of one variable, h (t) ≡ f (x + tv) for small |t|. Since
f has a local maximum (minimum), it follows that h is a differentiable function of the
single variable t for small t which has a local maximum (minimum) when t = 0. Therefore,
h′ (0) = 0.

h (∆t) − h (0) = f (x + ∆tv) − f (x)

= D f (x) ∆tv + o (∆t)

Now divide by ∆t and let ∆t → 0 to obtain

0 = h′ (0) = D f (x) v

and since v is arbitrary, it follows D f (x) = 0. However,

 
D f (x) = fx1 (x) · · · fxn (x)

and so ∇ f (x) = 0. This proves the following theorem.

329
330 CHAPTER 18. OPTIMIZATION

Theorem 18.1.3 Suppose U is an open set contained in D ( f ) such that f is differentiable

on U and suppose x ∈ U is a local minimum or local maximum for f . Then ∇ f (x) = 0.

Definition 18.1.4 A singular point for f is a point x where ∇ f (x) = 0. This is also called
a critical point. By analogy with the one variable case, a point where the gradient does not
exist will also be called a critical point.

Example 18.1.5 Find the critical points for the function f (x, y) ≡ xy − x − y for x, y > 0.

Note that here D ( f ) is an open set and so every point is an interior point. Where is the
gradient equal to zero? fx = y − 1 = 0, fy = x − 1 = 0, and so there is exactly one critical
point (1, 1).

Example 18.1.6 Find the volume of the smallest tetrahedron made up of the coordinate
planes in the first octant and a plane which is tangent to the sphere x2 + y2 + z2 = 4.
 q 
The normal to the sphere at a point (x0 , y0 , z0 ) is x0 , y0 , 4 − x02 − y20 and so the
equation of the tangent plane at this point is
q  q 
x0 (x − x0 ) + y0 (y − y0 ) + 4 − x02 − y20 z − 4 − x02 − y20 = 0

4 4 4
When x = y = 0, z = q . When z = 0 = y, x = x0 , and when z = x = 0, y = y0 .
(4−x02 −y20 )
Therefore, the function to minimize is
1 64
f (x, y) = p
6 xy (4 − x2 − y2 )

This is because in beginning calculus it was shown that the volume of a pyramid is 1/3 the
area of the base times the height. Therefore, you simply need to find the gradient of this
and set it equal to zero. Thus upon taking the partial derivatives, you need to have
−4 + 2x2 + y2
p = 0,
x2 y (−4 + x2 + y2 ) (4 − x2 − y2 )
and
−4 + x2 + 2y2
p = 0.
xy2 (−4 + x2 + y2 ) (4 − x2 − y2 )
Therefore, x2 + 2y2 = 4 and 2x2 + y2 = 4. Thus x = y and so x = y = √2 . It follows from
3
the equation for z that z = √2 also. How do you know this is not the largest tetrahedron?
3

Example 18.1.7 An open box is to contain 32 cubic feet. Find the dimensions which will
result in the least surface area.

Let the height of the box be z and the length and width be x and y respectively. Then
xyz = 32 and so z = 32/xy. The total area is xy + 2xz + 2yz and so in terms of the two
variables x and y, the area is A = xy + 64 64
y + x . To find best dimensions you note these must
result in a local minimum.
yx2 − 64 xy2 − 64
Ax = = 0, Ay = .
x2 y2
18.2. EXERCISES 331

Therefore, yx2 − 64 = 0 and xy2 − 64 = 0 so xy2 = yx2 . For sure the answer excludes the
case where any of the variables equals zero. Therefore, x = y and so x = 4 = y. Then z = 2
from the requirement that xyz = 32. How do you know this gives the least surface area?
Why is this not the largest surface area?

18.2 Exercises
1. Find the points where possible local minima or local maxima occur in the following
functions.

(a) x2 − 2x + 5 + y2 − 4y
(b) −xy + y2 − y + x
(c) 3x2 − 4xy + 2y2 − 2y + 2x
(d) cos (x) + sin (2y)
(e) x4 − 4x3 y + 6x2 y2 − 4xy3 + y4 + x2 − 2x
(f) y2 x2 − 2xy2 + y2

2. Find the volume of the largest box which can be inscribed in a sphere of radius a.
3. Find in terms of a, b, c the volume of the largest box which can be inscribed in the
2 2 2
ellipsoid ax2 + by2 + cz2 = 1.
4. Find three numbers which add to 36 whose product is as large as possible.
5. Find three numbers x, y, z such that x2 +y2 +z2 = 1 and x+y+z is as large as possible.
6. Find three numbers x, y, z such that x2 + y2 + z2 = 4 and xyz is as large as possible.
7. A feeding trough in the form of a trapezoid with equal base angles is made from a
long rectangular piece of metal of width 24 inches by bending up equal strips along
both sides. Find the base angles and the width of these strips which will maximize
the volume of the feeding trough.

8. An open box (no top) is to contain 40 cubic feet. The material for the bottom costs
twice as much as the material for the sides. Find the dimensions of the box which is
cheapest.
9. The function f (x, y) = 2x2 + y2 is defined on the disk x2 + y2 ≤ 1. Find its maximum
value.
10. Find the point on the surface z = x2 + y + 1 which is closest to (0, 0, 0).
11. Let L1 = (t, 2t, 3 − t) and L2 = (2s, s + 2, 4 − s) be two lines. Find a pair of points,
one on the first line and the other on the second such that these two points are closer
together than any other pair of points on the two lines.
332 CHAPTER 18. OPTIMIZATION

12. ∗ Let 
2
 −1 if y
= x , x ̸= 0

2
f (x, y) = y − x2 if y ̸= x2

0 if (x, y) = (0, 0)


Show that ∇ f (0, 0) = 0. Now show that if (a, b) is any nonzero unit vector, the
function t → f (ta,tb) has a local minimum of 0 when t = 0. Thus in every direction,
this function has a local minimum at (0, 0) but the function f does not have a local
minimum at (0, 0).

18.3 The Second Derivative Test

There is a version of the second derivative test in the case that the function and its first and
second partial derivatives are all continuous.
2
Definition 18.3.1 The matrix H (x) whose i jth entry at the point x is ∂ ∂xi ∂fx j (x) is called
the Hessian matrix. The eigenvalues of H (x) are the solutions λ to the equation
det (λ I − H (x)) = 0.

The following theorem says that if all the eigenvalues of the Hessian matrix at a critical
point are positive, then the critical point is a local minimum. If all the eigenvalues of the
Hessian matrix at a critical point are negative, then the critical point is a local maximum.
Finally, if some of the eigenvalues of the Hessian matrix at the critical point are positive and
some are negative then the critical point is a saddle point. The following picture illustrates
the situation.

Theorem 18.3.2 Let f : U → R for U an open set in Rn and let f be a C2 function and
suppose that at some x ∈ U, ∇ f (x) = 0. Also let µ and λ be respectively, the largest
and smallest eigenvalues of the matrix H (x). If λ > 0 then f has a local minimum at x.
If µ < 0 then f has a local maximum at x. If either λ or µ equals zero, the test fails. If
λ < 0 and µ > 0 there exists a direction in which when f is evaluated on the line through
the critical point having this direction, the resulting function of one variable has a local
minimum and there exists a direction in which when f is evaluated on the line through
the critical point having this direction, the resulting function of one variable has a local
maximum. This last case is called a saddle point.

Here is an example.

Example 18.3.3 Let f (x, y) = 10xy + y2 . Find the critical points and determine whether
they are local minima, local maxima or saddle points.
18.3. THE SECOND DERIVATIVE TEST 333

First ∇ 10xy + y2 = (10y, 10x + 2y) and so there is one critical point at the point (0, 0).

What is it? The Hessian matrix is

!
0 10
10 2

and the eigenvalues are of different signs. Therefore, the critical point (0, 0) is a saddle
point. Here is a graph drawn by Matlab.

Here is another example.

Example 18.3.4 Let f (x, y) = 2x4 − 4x3 + 14x2 + 12yx2 − 12yx − 12x + 2y2 + 4y + 2. Find
the critical points and determine whether they are local minima, local maxima, or saddle
points.

fx (x, y) = 8x3 − 12x2 + 28x + 24yx − 12y − 12 and 2

fy (x, y) = 12x − 12x + 4y + 4. The
1 1
points at which both fx and fy equal zero are 2 , − 4 , (0, −1), and (1, −1).
The Hessian matrix is
!
24x2 + 28 + 24y − 24x 24x − 12
24x − 12 4

and the thing to determine is the sign of its eigenvalues evaluated at the
! critical points.
16 0
First consider the point 21 , − 14 . The Hessian matrix is


and its eigenvalues
0 4
are 16, 4 showing that this is a local minimum. !
4 −12
Next consider (0, −1) at this point the Hessian matrix is and the
−12 4
eigenvalues are 16, −8. Therefore, this point is a saddle point. To determine this, find the
eigenvalues.
! !!
1 0 4 −12
det λ − = λ 2 − 8λ − 128 = (λ + 8) (λ − 16)
0 1 −12 4

so the eigenvalues are −8 and 16 as claimed. !

4 12
Finally consider the point (1, −1). At this point the Hessian is and the
12 4
eigenvalues are 16, −8 so this point is also a saddle point.
Below is a graph of this function which illustrates the behavior near saddle points.
334 CHAPTER 18. OPTIMIZATION

Or course sometimes the second derivative test is inadequate to determine what is going
on. This should be no surprise since this was the case even for a function of one variable.
For a function of two variables, a nice example is the Monkey saddle.
Example 18.3.5 Suppose f (x, y) = 6xy2 − 2x3 − 3y4 . Show that (0, 0) is a critical point
for which the second derivative test gives no information.
Before doing anything it might be interesting to look at the graph of this function of
two variables plotted using a computer algebra system.

This picture should indicate why this is called a monkey saddle. It is because the
monkey can sit in the saddle and have a place for his tail. Now to see (0, 0) is a critical point,
note that fx (0, 0) = fy (0, 0) = 0 because fx (x, y) = 6y2 − 6x2 , fy (x, y) = 12xy − 12y3 and
so (0, 0) is a critical point. So are (1, 1) and (1, −1). Now fxx (0, 0) = 0 and so are fxy (0, 0)
and fyy (0, 0). Therefore, the Hessian matrix is the zero matrix and clearly has only the zero
eigenvalue. Therefore, the second derivative test is totally useless at this point.
However, suppose you took x = t and y = t and evaluated this function on this line. This
reduces to h (t) = f (t,t) = 4t 3 − 3t 4 ), which is strictly increasing near t = 0. This shows
the critical point (0, 0) of f is neither a local max. nor a local min. Next let x = 0 and y = t.
Then p (t) ≡ f (0,t) = −3t 4 . Therefore, along the line, (0,t), f has a local maximum at
(0, 0).
Example 18.3.6 Find the critical points of the following function of three variables and
classify them as local minimums, local maximums or saddle points.

5 7 4 5 4 1
f (x, y, z) = x2 + 4x + 16 − xy − 4y − xz + 12z + y2 − zy + z2
6 3 3 6 3 3
First you need to locate the critical points. This involves taking the gradient.
 
5 2 7 4 5 2 4 1 2
∇ x + 4x + 16 − xy − 4y − xz + 12z + y − zy + z
6 3 3 6 3 3
 
5 7 4 7 5 4 4 4 2
= x + 4 − y − z, − x − 4 + y − z, − x + 12 − y + z
3 3 3 3 3 3 3 3 3
18.4. EXERCISES 335

Next you need to set the gradient equal to zero and solve the equations. This yields y =
5, x = 3, z = −2. Now to use the second derivative test, you assemble the Hessian matrix
which is  
5
3 − 73 − 43
 7 5
 −3 − 43  .

3
− 43 − 43 2
3
Note that in this simple example, the Hessian matrix is constant and so all that is left
is to consider the eigenvalues. Writing the characteristic equation and solving yields the
eigenvalues are 2, −2, 4. Thus the given point is a saddle point.

18.4 Exercises
1. Use the second derivative test on the critical points (1, 1), and (1, −1) for Example
18.3.5. The function is 6xy2 − 2x3 − 3x4 .
2. If H = H T and Hx = λ x while Hx = µx for λ ̸= µ, show that x · y = 0.
3. Show the points 21 , − 21


4 , (0, −4) , and (1, −4) are critical points of the following
function of two variables and classify them as local minima, local maxima or saddle
points.
f (x, y) = −x4 + 2x3 + 39x2 + 10yx2 − 10yx − 40x − y2 − 8y − 16.
4. Show the points 12 , − 12
53


, (0, −4) , and (1, −4) are critical points of the following
function of two variables and classify them according to whether they are local min-
ima, local maxima or saddle points.
f (x, y) = −3x4 + 6x3 + 37x2 + 10yx2 − 10yx − 40x − 3y2 − 24y − 48.
5. Show the points 21 , 20
37


, (0, 2) , and (1, 2) are critical points of the following function
of two variables and classify them according to whether they are local minima, local
maxima or saddle points.
f (x, y) = 5x4 − 10x3 + 17x2 − 6yx2 + 6yx − 12x + 5y2 − 20y + 20.
6. Show the points 21 , − 17


8 , (0, −2) , and (1, −2) are critical points of the following
function of two variables and classify them according to whether they are local min-
ima, local maxima or saddle points.
f (x, y) = 4x4 − 8x3 − 4yx2 + 4yx + 8x − 4x2 + 4y2 + 16y + 16.
7. Find the critical points of the following function of three variables and classify them
according to whether they are local minima, local maxima or saddle points.
f (x, y, z) = 31 x2 + 32 4 16 58 4 46 1 2 4 5 2
3 x + 3 − 3 yx − 3 y − 3 zx − 3 z + 3 y − 3 zy − 3 z .

8. Find the critical points of the following function of three variables and classify them
according to whether they are local minima, local maxima or saddle points.
f (x, y, z) = − 35 x2 + 23 x − 23 + 83 yx + 23 y + 14 28 5 2 14 8 2
3 zx − 3 z − 3 y + 3 zy − 3 z .

9. Find the critical points of the following function of three variables and classify them
according to whether they are local minima, local maxima or saddle points.
f (x, y, z) = − 11 2 40 56 8 10 4 22 11 2 4 5 2
3 x + 3 x − 3 + 3 yx + 3 y − 3 zx + 3 z − 3 y − 3 zy − 3 z .
336 CHAPTER 18. OPTIMIZATION

10. Find the critical points of the following function of three variables and classify them
according to whether they are local minima, local maxima or saddle points.
f (x, y, z) = − 32 x2 + 28 37 14 10 4 26 2 2 4 7 2
3 x + 3 + 3 yx + 3 y − 3 zx − 3 z − 3 y − 3 zy + 3 z .

11. ∗ Show that if f has a critical point and some eigenvalue of the Hessian matrix is
positive, then there exists a direction in which when f is evaluated on the line through
the critical point having this direction, the resulting function of one variable has a
local minimum. State and prove a similar result in the case where some eigenvalue
of the Hessian matrix is negative.
12. Suppose µ = 0 but there are negative eigenvalues of the Hessian at a critical point.
Show by giving examples that the second derivative tests fails.
13. Show that the points 12 , − 92 , (0, −5), and (1, −5) are critical points of the following

function of two variables and classify them as local minima, local maxima or saddle
points.
f (x, y) = 2x4 − 4x3 + 42x2 + 8yx2 − 8yx − 40x + 2y2 + 20y + 50.

14. Show that the points 1, − 11

2 , (0, −5), and (2, −5) are critical points of the follow-
ing function of two variables and classify them as local minima, local maxima or
saddle points.
f (x, y) = 4x4 − 16x3 − 4x2 − 4yx2 + 8yx + 40x + 4y2 + 40y + 100.

15. Show that the points 23 , 20

27


, (0, 0), and (3, 0) are critical points of the following
function of two variables and classify them as local minima, local maxima or saddle
points.
f (x, y) = 5x4 − 30x3 + 45x2 + 6yx2 − 18yx + 5y2 .
16. Find the critical points of the following function of three variables and classify them
as local minima, local maxima or saddle points.
10 2 44 64
f (x, y, z) = 3 x − 3 x+ 3 − 10 16 2 20 10 2 2 4 2
3 yx + 3 y + 3 zx − 3 z + 3 y + 3 zy + 3 z .

17. Find the critical points of the following function of three variables and classify them
as local minima, local maxima or saddle points.
f (x, y, z) = − 73 x2 − 146 83 16 4 14 94 7 2 14 8 2
3 x + 3 + 3 yx + 3 y − 3 zx + 3 z − 3 y − 3 zy + 3 z .

18. Find the critical points of the following function of three variables and classify them
as local minima, local maxima or saddle points.
f (x, y, z) = 32 x2 + 4x + 75 − 14 8 2 2 8 1 2
3 yx − 38y − 3 zx − 2z + 3 y − 3 zy − 3 z .

19. Find the critical points of the following function of three variables and classify them
as local minima, local maxima or saddle points.
f (x, y, z) = 4x2 − 30x + 510 − 2yx + 60y − 2zx − 70z + 4y2 − 2zy + 4z2 .

20. Show that the critical points of the following function are points of the form, (x, y, z) =
t, 2t 2 − 10t, −t 2 + 5t for t ∈ R and classify them as local minima, local maxima or

saddle points.
f (x, y, z) = − 61 x4 + 53 x3 − 25 2 10 2 50 19 2 95 5 2 10 1 2
6 x + 3 yx − 3 yx + 3 zx − 3 zx − 3 y − 3 zy − 6 z .
18.5. LAGRANGE MULTIPLIERS 337

21. Show that the critical points of the following function are
 
1
(0, −3, 0) , (2, −3, 0) , and 1, −3, −
3

and classify them as local minima, local maxima or saddle points.

f (x, y, z) = − 23 x4 + 6x3 − 6x2 + zx2 − 2zx − 2y2 − 12y − 18 − 23 z2 .

22. Show that the critical points of the function f (x, y, z) = −2yx2
− 6yx − 4zx2 − 12zx +
y2 +2yz. are points of the form, (x, y, z) = t, 2t 2 + 6t, −t 2 − 3t for t ∈ R and classify
them as local minima, local maxima or saddle points.

23. Show that the critical points of the function

1 1
f (x, y, z) = x4 − 4x3 + 8x2 − 3zx2 + 12zx + 2y2 + 4y + 2 + z2 .
2 2
are (0, −1, 0) , (4, −1, 0), and (2, −1, −12) and classify them as local minima, local
maxima or saddle points.

24. Suppose f (x, y), a function of two variables defined on all Rn has all directional
derivatives at (0, 0) and they are all equal to 0 there. Suppose also that for h (t) ≡
f (tu,tv) and (u, v) a unit vector, it follows that h′′ (0) > 0. By the one variable
second derivative test, this implies that along every straight line through (0, 0) the
function restricted to this line has a local minimum at (0, 0). Can it be concluded that
f has a local minimum at (0, 0). In other words, can you conclude a point is a local
minimum if it appears to be so along every straight line through the point? Hint:
Consider f (x, y) = x2 + y2 for (x, y) not on the curve y = x2 for x ̸= 0 and on this
curve, let f = −1.

18.5 Lagrange Multipliers

Lagrange multipliers are used to solve extremum problems for a function defined on a level
set of another function. This is the typical situation in optimization. You have a constraint
on the variables and subject to this constraint, you are trying to maximize of minimize some
function. It is the constraint which makes the problem interesting. For example, suppose
you want to maximize xy given that x + y = 4. Solve for one of the variables say y, in the
constraint equation x + y = 4 or x + y − 4 = 0 to find y = 4 − x. Then substitute this in to
the function you are trying to maximize and take a derivative. The difficulty comes when
you can’t solve for one of the variables in the constraint or perhaps you could, but it would
be inconvenient to do so.
In general, you want to maximize (minimize) f (x, y, z) subject to the constraint g (x, y, z) =
0. Just because you can’t algebraically solve for one of the variables, doesn’t mean the re-
lation does not define one of the variables in terms of the others. Say z = z (x, y) near a
point (x0 , y0 , z0 ) on the constraint surface where the maximum or minimum exists. Then
you could consider the unconstrained problem

(x, y) → f (x, y, z (x, y))

338 CHAPTER 18. OPTIMIZATION

and you would expect its partial derivatives to be 0 at the point of interest. By the chain
rule (never mind the mathematical questions on existence), at this special point,

fx + fz zx = 0, fy + fz zy = 0

By the process of implicit differentiation applied to g (x, y, z) = 0,

gx gy
zx = − , zy = −
gz gz
Thus,      
gx fz gy fz fz
fx = fz = gx , fy = fz = gy , fz = gz
gz gz gz gz gz
fz (x0 ,y0 ,z0 )
So letting λ = gz (x0 ,y0 ,z0 ) , it follows that at this point

∇ f (x0 , y0 , z0 ) = λ ∇g (x0 , y0 , z0 )

The situation in which it is x or y that is a function of the other variables is exactly similar.
Also, if there are more or fewer variables there is no difference in the argument. This λ is
called a Lagrange multiplier after Lagrange who considered such problems in the 1700’s.

Example 18.5.1 Maximize xyz subject to x2 + y2 + z2 = 27.

Here f (x, y, z) = xyz while g (x, y, z) = x2 + y2 + z2 − 27. Then ∇g (x, y, z) = (2x, 2y, 2z)
and ∇ f (x, y, z) = (yz, xz, xy). Then at the point which maximizes this function1 , (yz, xz, xy) =
λ (2x, 2y, 2z) . Therefore, each of 2λ x2 , 2λ y2 , 2λ z2 equals xyz. It follows that at any point
which maximizes xyz, |x| = |y| = |z|. Therefore, the only candidates for the point where the
maximum occurs are
(3, 3, 3) , (−3, −3, 3) (−3, 3, 3)
etc. The maximum occurs at (3, 3, 3) which can be verified by plugging in to the function
which is being maximized.
The method of Lagrange multipliers allows you to consider maximization of functions
defined on closed and bounded sets. Recall that any continuous function defined on a
closed and bounded set has a maximum and a minimum on the set. Candidates for the
extremum on the interior of the set can be located by setting the gradient equal to zero. The
consideration of the boundary can then sometimes be handled with the method of Lagrange
multipliers.

Example 18.5.2 Maximize f (x, y) = xy + y subject to the constraint, x2 + y2 ≤ 1.

Here I know there is a maximum because the set is the closed disk, a closed and bounded
set. Therefore, it is just a matter of finding it. Look for singular points on the interior of the
circle. ∇ f (x, y) = (y, x + 1) = (0, 0). There are no points on the interior of the circle where
the gradient equals zero. Therefore, the maximum occurs on the boundary of the circle.
That is, the problem reduces to maximizing xy + y subject to x2 + y2 = 1. From the above,

(y, x + 1) − λ (2x, 2y) = 0.

1 There exists such a point because the sphere is closed and bounded.
18.5. LAGRANGE MULTIPLIERS 339

Hence y2 − 2λ xy = 0 and x (x + 1) − 2λ xy = 0 so y2 = x (x + 1). Therefore from the con-

1
straint, x2 + x (x + 1)= 1 and  is x = −1, x = 2 . Then the candidates for a
 the solution
√ √
1 3 1 − 3
solution are (−1, 0) , 2, 2 , 2, 2 . Then
√ ! √ √ ! √
1 3 3 3 1 3 3 3
f (−1, 0) = 0, f , = ,f ,− =− .
2 2 4 2 2 4
√  √ 
3 3 1 3
It follows the maximum value of this function is 4 and it occurs at 2, 2 . The mini-
√  √ 
mum value is − 3 4 3 and it occurs at 12 , − 23 .

Example 18.5.3 Find candidates for the maximum and minimum values of the function
f (x, y) = xy − x2 on the set (x, y) : x2 + 2xy + y2 ≤ 4 .


First, the only point where ∇ f equals zero is (x, y) = (0, 0) and this is in the desired set.
In fact it is an interior point of this set. This takes care of the interior points. What about
those on the boundary x2 + 2xy + y2 = 4? The problem is to maximize xy − x2 subject
to the
constraint, x2 + 2xy + y2 = 4. The Lagrangian is xy − x2 − λ x2 + 2xy + y2 − 4 and this
yields the following system.

y − 2x − λ (2x + 2y) = 0
x − 2λ (x + y) = 0
x2 + 2xy + y2 = 4

From the first two equations,

(2 + 2λ ) x − (1 − 2λ ) y = 0, (1 − 2λ ) x − 2λ y = 0

Since not both x and y equal zero, it follows

!
2 + 2λ 2λ − 1
det =0
1 − 2λ −2λ

which yields λ = 1/8. Therefore, y = 3x. From the constraint equation x2 + 2x (3x) +
(3x)2 = 4 and so x = 21 or − 12 . Now since y = 3x, the points of interest on the boundary of
this set are    
1 3 1 3
, , and − , − . (18.1)
2 2 2 2
       2
1 3 1 3 1 1
f , = − =
2 2 2 2 2 2
      2
1 3 1 3 1 1
f − ,− = − − − − =
2 2 2 2 2 2
Thus the candidates for maximum and minimum are 12 , 23 , (0, 0), and − 12 , − 32 . There-

fore it appears that (0, 0) yields a minimum and either 12 , 32 or − 12 , − 32 yields a max-

imum. However, this is a little misleading. How do you even know a maximum or a
minimum exists? The set x2 + 2xy + y2 ≤ 4 is an unbounded set which lies between the
340 CHAPTER 18. OPTIMIZATION

two lines x + y = 2 and x + y = −2. In fact there is no minimum. For example, take
x = 100, y = −98. Then xy − x2 = x (y − x) = 100 (−98 − 100) which is a large negative
number much less than 0, the answer for the point (0, 0).
There are no magic bullets here. It was still required to solve a system of nonlinear
equations to get the answer. However, it does often help to do it this way.
A nice observation in the case that the function f , which you are trying to maximize,
and the function g, which defines the constraint, are functions of two or three variables is
the following.
At points of interest,
∇ f × ∇g = 0
This follows from the above because at these points,
∇ f = λ ∇g
so the angle between the two vectors ∇ f and ∇g is either 0 or π. Therefore, the sine of this
angle equals 0. By the geometric description of the cross product, this implies the cross
product equals 0. Here is an example.
Example 18.5.4 Minimize f (x, y) = xy − x2 on the set
(x, y) : x2 + 2xy + y2 = 4


Using the observation about the cross product, and letting f (x, y, z) = f (x, y) with a
similar convention for g, ∇ f = (y − 2x, x, 0) , ∇g = (2x + 2y, 2x + 2y, 0) and so
(y − 2x, x, 0) × (2x + 2y, 2x + 2y, 0)
= (0, 0, (y − 2x) (2x + 2y) − x (2x + 2y)) = 0
Thus there are two equations, x2 + 2xy 2 2 2

+ 1y 3=
4 and 4xy − 2y + 6x = 0. Solving these two
1 3
yields the points of interest − 2 , − 2 , 2 , 2 . Both give the same value for f a maximum.
The above generalizes to a general procedure which is described in the following major
Theorem. All correct proofs of this theorem will involve some appeal to the implicit func-
tion theorem or to fundamental existence theorems from differential equations. A complete
proof is very fascinating but it will not come cheap. Good advanced calculus books will
usually give a correct proof. If you are interested, there is a complete proof later. First here
is a simple definition explaining one of the terms in the statement of this theorem.
Definition 18.5.5 Let A be an m × n matrix. A submatrix is any matrix which can be
obtained from A by deleting some rows and some columns.
Theorem 18.5.6 Let U be an open subset of Rn and let f : U → R be a C1 function. Then
if x0 ∈ U, has the property that
gi (x0 ) = 0, i = 1, · · · , m, gi a C1 function, (18.2)
and x0 is either a local maximum or local minimum of f on the intersection of the level
sets just described, and if some m × m submatrix of
 
g1x1 (x0 ) g1x2 (x0 ) · · · g1xn (x0 )
 .. .. .. 
Dg (x0 ) ≡  . . .


gmx1 (x0 ) gmx2 (x0 ) · · · gmxn (x0 )
18.5. LAGRANGE MULTIPLIERS 341

has nonzero determinant, then there exist scalars, λ 1 , · · · , λ m such that

     
fx1 (x0 ) g1x1 (x0 ) gmx1 (x0 )
 .
..
 
 = λ1  .
..

+···+λm 
 .. 

     .

 (18.3)
fxn (x0 ) g1xn (x0 ) gmxn (x0 )

holds.

To help remember how to use 18.3, do the following. First write the Lagrangian,
m
L = f (x) − ∑ λ i gi (x)
i=1

and then proceed to take derivatives with respect to each of the components of x and also
derivatives with respect to each λ i and set all of these equations equal to 0. The formula
18.3 is what results from taking the derivatives of L with respect to the components of x.
When you take the derivatives with respect to the Lagrange multipliers, and set what results
equal to 0, you just pick up the constraint equations. This yields n + m equations for the
n + m unknowns x1 , · · · , xn , λ 1 , · · · , λ m . Then you proceed to look for solutions to these
equations. Of course these might be impossible to find using methods of algebra, but you
just do your best and hope it will work out.

Example 18.5.7 Minimize xyz subject to the constraints x2 + y2 + z2 = 4 and x − 2y = 0.

Form the Lagrangian,

L = xyz − λ x2 + y2 + z2 − 4 − µ (x − 2y)

and proceed to take derivatives with respect to every possible variable, leading to the fol-
lowing system of equations.

yz − 2λ x − µ = 0
xz − 2λ y + 2µ = 0
xy − 2λ z = 0
x2 + y2 + z2 = 4
x − 2y = 0

Now you have to find the solutions to this system of equations. In general, this could be
very hard or even impossible. If λ = 0, then from the third equation, either x or y must
equal 0. Therefore, from the first two equations, µ = 0 also. If µ = 0 and λ ̸= 0, then from
the first two equations, xyz = 2λ x2 and xyz = 2λ y2 and so either x = y or x = −y, which
requires that both x and y equal zero thanks to the last equation. But then from the fourth
equation, z = ±2 and now this contradicts the third equation. Thus µ and λ are either both
equal to zero or neither one is and the expression, xyz equals zero in thisqcase. However,
q I
know this is not the best value for a minimizer because I can take x = 2 35 , y = 35 , and
z = −1. This satisfies the constraints and the product of these numbers equals a negative
342 CHAPTER 18. OPTIMIZATION

number. Therefore, both µ and λ must be non zero. Now use the last equation eliminate x
and write the following system.

5y2 + z2 = 4
y2 − λ z = 0
yz − λ y + µ = 0
yz − 4λ y − µ = 0

From the last equation, µ = (yz − 4λ y). Substitute this into the third and get

5y2 + z2 = 4
y2 − λ z = 0
yz − λ y + yz − 4λ y = 0

y = 0 will not yield the minimum value from the above example. Therefore, divide the last
equation by y and solve for λ to get λ = (2/5) z. Now put this in the second equation to
conclude
5y2 + z2 = 4
,
y2 − (2/5) z2 = 0
a system which
 q Thus y2= 8/15 and z2 = 4/3. Therefore,
is easy to solve.  candidates for
q q q q q 
8 8 4 8 8 4
minima are 2 15 , 15 , ± 3 , and −2 15 , − 15 , ± 3 , a choice of 4 points to
check. Clearly the one which gives the smallest value is
r r r !
8 8 4
2 , ,−
15 15 3
 q q q 
8 8
or −2 15 , − 15 , − 43 and the minimum value of the function subject to the con-
√ √
straints is − 52 30 − 23 3.
You should rework this problem first solving the second easy constraint for x and then
producing a simpler problem involving only the variables y and z.

18.6 Exercises
1. Maximize x + y + z subject to the constraint x2 + y2 + z2 = 3.
2. Minimize 2x − y + z subject to the constraint 2x2 + y2 + z2 = 36.
3. Minimize x + 3y − z subject to the constraint 2x2 + y2 − 2z2 = 36 if possible. Note
there is no guaranty this function has either a maximum or a minimum. Determine
whether there exists a minimum also.
4. Find the dimensions of the largest rectangle which can be inscribed in a circle of
radius r.
2
x2
5. Maximize 2x + y subject to the condition that 4 + y9 ≤ 1.
18.6. EXERCISES 343

2
6. Maximize x + 2y subject to the condition that x2 + y9 ≤ 1.
2
7. Maximize x + y subject to the condition that x2 + y9 + z2 ≤ 1.
2
8. Minimize x + y + z subject to the condition that x2 + y9 + z2 ≤ 1.

9. Find the points on y2 x = 16 which are closest to (0, 0).

√
10. Find the points on 2y2 x = 1 which are closest to (0, 0).

11. Find points on xy = 4 farthest from (0, 0) if any exist. If none exist, tell why. What
does this say about the method of Lagrange multipliers?
12. A can is supposed to have a volume of 36π cubic centimeters. Find the dimensions
of the can which minimizes the surface area.
13. A can is supposed to have a volume of 36π cubic centimeters. The top and bottom of
the can are made of tin costing 4 cents per square centimeter and the sides of the can
are made of aluminum costing 5 cents per square centimeter. Find the dimensions of
the can which minimizes the cost.
14. Minimize and maximize ∑nj=1 x j subject to the constraint ∑nj=1 x2j = a2 . Your answer
should be some function of a which you may assume is a positive number.

15. Find the point (x, y, z) on the level surface 4x2 +y2 −z2 = 1which is closest to (0, 0, 0).
16. A curve is formed from the intersection of the plane, 2x + y + z = 3 and the cylinder
x2 + y2 = 4. Find the point on this curve which is closest to (0, 0, 0).
17. A curve is formed from the intersection of the plane, 2x + 3y + z = 3 and the sphere
x2 + y2 + z2 = 16. Find the point on this curve which is closest to (0, 0, 0).
18. Find the point on the plane, 2x + 3y + z = 4 which is closest to the point (1, 2, 3).
19. Let A = (Ai j ) be an n × n matrix which is symmetric. Thus Ai j = A ji and recall
(Ax)i = Ai j x j where as usual, sum over the repeated index. Show that ∂∂x (Ai j x j xi ) =
k
2Ai j x j . Show that when you use the method of Lagrange multipliers to maximize
the function Ai j x j xi subject to the constraint, ∑nj=1 x2j = 1, the value of λ which
corresponds to the maximum value of this functions is such that Ai j x j = λ xi . Thus
Ax = λ x. Thus λ is an eigenvalue of the matrix A.

20. Here are two lines.

x = (1 + 2t, 2 + t, 3 + t)T
and x = (2 + s, 1 + 2s, 1 + 3s)T . Find points p1 on the first line and p2 on the second
with the property that |p1 − p2 | is at least as small as the distance between any other
pair of points, one chosen on one line and the other on the other line.
21. ∗ Find points on the circle of radius r for the largest triangle which can be inscribed
in it.
22. Find the point on the intersection of z = x2 + y2 and x + y + z = 1 which is closest to
(0, 0, 0).
344 CHAPTER 18. OPTIMIZATION

23. Minimize xyz subject to the constraints x2 + y2 + z2 = r2 and x − y = 0.

24. Let n be a positive integer. Find n numbers whose sum is 8n and the sum of the
squares is as small as possible.
25. Find the point on the level surface 2x2 + xy + z2 = 16 which is closest to (0, 0, 0).
26. Find the point on x2 + y2 + z2 = 1 closest to the plane x + y + z = 10.
2
x2
27. Find the point on 4 + y9 + z2 = 1 closest to the plane x + y + z = 10.
28. Let x1 , · · · , x5 be 5 positive numbers. Maximize their product subject to the constraint
that
x1 + 2x2 + 3x3 + 4x4 + 5x5 = 300.

29. Let f (x1 , · · · , xn ) = x1n x2n−1 · · · xn1 . Then f achieves a maximum on the set S ≡
( )
n
x ∈ Rn : ∑ ixi = 1, each xi ≥ 0
i=1

If x ∈ S is the point where this maximum is achieved, find x1 /xn .

30. ∗ Let (x, y) be a point on the ellipse, x2 /a2 + y2 /b2 = 1 which is in the first quadrant.
Extend the tangent line through (x, y) till it intersects the x and y axes and let A (x, y)
denote the area of the triangle formed by this line and the two coordinate axes. Find
the minimum value of the area of this triangle as a function of a and b.
31. Maximize ∏ni=1 xi2
(≡ x12 × x22 × x32 × · · · × xn2 )

n
subject to the constraint, ∑ni=1 xi2 = r2 . Show that the maximum is r2 /n . Now
show from this that !1/n
n
2 1 n
∏ xi ≤ ∑ xi2
i=1 n i=1
and finally, conclude that if each number xi ≥ 0, then
!1/n
n
1 n
∏ xi ≤ ∑ xi
i=1 n i=1

and there exist values of the xi for which equality holds. This says the “geometric
mean” is always smaller than the arithmetic mean.
32. Maximize x2 y2 subject to the constraint

x2p y2q
+ = r2
p q
where p, q are real numbers larger than 1 which have the property that
1 1
+ =1
p q
18.7. PROOF OF THE SECOND DERIVATIVE TEST∗ 345

show that the maximum is achieved when x2p = y2q and equals r2 . Now conclude
that if x, y > 0, then
x p yq
xy ≤ +
p q
and there are values of x and y where this inequality is an equation.

33. The area of the ellipse x2 /aq

2 + y2 /b2 ≤ 1 is πab which is given to equal π. The

length of the ellipse is 02π a2 sin2 (t) + b2 cos2 (t)dt. Find a, b such that the ellipse
R

having this volume is as short as possible.

√
34. Consider the closed region in the xy plane which lies between the curve y = 1 − x2
and y = 0. Find the maximum and minimum values of the function x2 + x + y2 − y on
this region. Hint: First observe that there is a solution because the region is compact.
Next look for candidates for the extreme point on the interior. When this is done, look
for candidates on the boundary. Note that the boundary of the region does not come
as the level surface of a C1 function. The method does not apply to the corners of
this region, the points (1, 0) and (0, 1). Therefore, you need to consider these points
also.

18.7 Proof Of The Second Derivative Test∗

A version of the following theorem is due to Lagrange, about 1790.

Theorem 18.7.1 Suppose f has n + 1 derivatives on an interval (a, b) and let c ∈ (a, b) .
Then if x ∈ (a, b) , there exists ξ between c and x such that
n
f (k) (c) f (n+1) (ξ )
f (x) = f (c) + ∑ (x − c)k + (x − c)n+1 .
k=1 k! (n + 1)!

(In this formula, the symbol ∑0k=1 ak will denote the number 0.)

Proof: There exists K such that

!
n
f (k) (c)
f (x) − f (c) + ∑ (x − c)k + K (x − c)n+1 =0 (18.4)
k=1 k!

In fact,
f (k) (c)
 
− f (x) + f (c) + ∑nk=1 k! (x − c)k
K= n+1
.
(x − c)
Now define F (t) for t in the closed interval determined by x and c by
!
n
f (k) (t) k n+1
F (t) ≡ f (x) − f (t) + ∑ (x − t) + K (x − t) .
k=1 k!

The c in 18.4 got replaced by t.

346 CHAPTER 18. OPTIMIZATION

Therefore, F (c) = 0 by the way K was chosen and also F (x) = 0. Then F ′ (t) =
f (k) (t) f (k+1) (t)
!!
k−1 k
′ ∑nk=1 k! k (x − t) − ∑nk=1 k! (x − t)
− f (t) −
+K (n + 1) (x − t)n
n−1 f (k+1) (t) f (k+1) (t)
!!
k n k
∑ (x − t) − ∑ (x − t)
= − f ′ (t) − k=0 k! k=1 k!
+K (n + 1) (x − t)n
  
= − f ′ (t) − f ′ (t) − f (n+1) (t) (x − t)n + K (n + 1) (x − t)n

= − f ′ (t) + f ′ (t) − f (n+1) (t) (x − t)n + K (n + 1) (x − t)n

1
= − f (n+1) (t) (x − t)n + K (n + 1) (x − t)n
n!
By the mean value theorem or Rolle’s theorem, there exists ξ between x and c such that
F ′ (ξ ) = 0. Therefore,
1
− f (n+1) (ξ ) (x − ξ )n + K (n + 1) (x − ξ )n = 0
n!
and so
1
K (n + 1) = f (n+1) (ξ )
n!
f (n+1) (ξ )
K = ■
(n + 1)!
(n+1)
(ξ )
The term f (n+1)! (x − c)n+1 , is called the remainder and this particular form of the
remainder is called the Lagrange form of the remainder.
 2 
Definition 18.7.2 The matrix ∂ ∂xi ∂fx j (x) is called the Hessian matrix, denoted by H (x).

Now recall the Taylor formula with the Lagrange form of the remainder.

Theorem 18.7.3 Let h : (−δ , 1 + δ ) → R have m + 1 derivatives. Then there exists t ∈

(0, 1) such that
m (k)
h (0) h(m+1) (t)
h (1) = h (0) + ∑ + .
k=1 k! (m + 1)!

Now let f : U → R where U is an open subset of Rn . Suppose f ∈ C2 (U). Let x ∈ U

and let r > 0 be such that
B (x,r) ⊆ U.
Then for ||v|| < r consider
f (x+tv) − f (x) ≡ h (t)
for t ∈ [0, 1]. Then from Taylor’s theorem for the case where m = 2 and the chain rule,
using the repeated index summation convention and the chain rule,

∂f ∂2 f
h′ (t) = (x + tv) vi , h′′ (t) = (x + tv) vi v j .
∂ xi ∂ x j ∂ xi
18.7. PROOF OF THE SECOND DERIVATIVE TEST∗ 347

Thus
h′′ (t) = v T H (x + tv) v.
From Theorem 18.7.3 there exists t ∈ (0, 1) such that

∂f 1
f (x + v) = f (x) + (x) vi + v T H (x + tv) v
∂ xi 2
By the continuity of the second partial derivative
1
f (x + v) = f (x) + ∇ f (x) · v+ v T H (x) v+
2
1 T

v (H (x+tv) − H (x)) v (18.5)
2
where the last term satisfies

1 v T (H (x+tv) − H (x)) v


lim =0 (18.6)
|v |→0 2 |v|2

because of the continuity of the entries of H (x).

Theorem 18.7.4 Suppose x is a critical point for f . That is, suppose ∂∂ xfi (x) = 0 for each
i. Then if H (x) has all positive eigenvalues, x is a local minimum. If H (x) has all negative
eigenvalues, then x is a local maximum. If H (x) has a positive eigenvalue, then there exists
a direction in which f has a local minimum at x, while if H (x) has a negative eigenvalue,
there exists a direction in which f has a local maximum at x.

Proof: Since ∇ f (x) = 0, formula (18.5) implies

1 1
f (x + v) = f (x) + v T H (x) v+ v T (H (x+tv) − H (x)) v


(18.7)
2 2
and by continuity of the second derivatives, these mixed second derivatives are equal and so
H (x) is a symmetric matrix. Thus, by Theorem 11.4.7, H (x) has all real eigenvalues and
can be diagonalized with an orthogonal matrix U. Suppose first that H (x) has all positive
eigenvalues and that all are larger than δ 2 > 0.

uT H (x) u = uT UDU T u = (Uu)T D (Uu) ≥ δ 2 |Uu|2 = δ 2 |u|2

By continuity of H, if v is small enough,

1 1 δ2 2
f (x + v) ≥ f (x) + δ 2 |v|2 − δ 2 |v|2 = f (x) + |v| .
2 4 4
This shows the first claim of the theorem. The second claim follows from similar reasoning
or applying the above to − f .
Suppose H (x) has a positive eigenvalue λ 2 . Then let v be an eigenvector for this
eigenvalue. Then from (18.7), replacing v with sv and letting t depend on s,
1
f (x+sv) = f (x) + s2 v T H (x) v+
2
348 CHAPTER 18. OPTIMIZATION

1 2 T

s v (H (x+tsv) − H (x)) v
2
which implies
1 1
f (x) + s2 λ 2 |v|2 + s2 v T (H (x+tsv) − H (x)) v


f (x+sv) =
2 2
1 2 2 2
≥ f (x) + s λ |v|
4
whenever s is small enough. Thus in the direction v the function has a local minimum at
x. The assertion about the local maximum in some direction follows similarly. ■
Chapter 19

The Riemannn Integral On Rn

19.1 Methods For Double Integrals

This chapter is on the Riemannn integral for a function of n variables. It begins by in-
troducing the basic concepts and applications of the integral. The general considerations
including the definition of the integral and proofs of theorems are left till later. These are
very difficult topics and are likely better considered in the context of the Lebesgue integral.
Consider the following region which is labeled R.
y
y = t(x)
R

y = b(x)
x
a b
We will consider the following iterated integral which makes sense for any continuous
function f (x, y) .
Z b Z t(x)
f (x, y) dydx
a b(x)

It means just exactly what the notation suggests it does. You fix x and then you do the
inside integral
Z t(x)
f (x, y) dy
b(x)

This yields a function of x which will end up being continuous. You then do ab dx to this
R

continuous function.
What was it about the above region which made it possible to set up such an iterated
integral? It was just this: You have a curve on the top y = t (x) , and a curve on the bottom
y = b (x) for x ∈ [a, b]. You could have set up a similar iterated integral if you had a region
in which there was a curve on the left and a curve on the right for y in some interval. Here
is an example.

Example 19.1.1 Suppose t (x) = 4 − x2 , b (x) = 0 and a = −2, b = 2. Compute the iterated
integral described above for f (x, y) = xy + y.

349
350 CHAPTER 19. THE RIEMANNN INTEGRAL ON Rn

You should sketch the graphs of these functions. Filling in the limits as above, we
obtain Z 2 Z 4−x2 Z 2
1 2
2 256
(xy + y) dydx = x − 4 (x + 1) dx =
−2 0 −2 2 15
Of course one could do the iterated integral in the other
√ order for this example. In this
√ case,
you would be considering a curve on the left x = − 4 − y, a curve on the right x = 4 − y,
and y ∈ [0, 4] . Thus this iterated integral would be of the form
Z 4 Z √4−y Z 4 p 256
√ (xy + y) dxdy = 2y 4 − ydy =
0 − 4−y 0 15

Why should it be the case that these two iterated integrals are equal? This involves a
consideration of what you are computing when you do such an iterated integral. First note
that in the general example given above involving t (x) , b (x) , it would not have been at
all convenient to have done the iterated integral in the other order. So what is it you are
getting? Consider the first illustration where the region is between y = b (x) and y = t (x).
Consider the following picture

y
y = t(x)
R

y = b(x)
x
a b
For simplicity, we let the distance between the vertical lines be ∆x and the distance
between the horizontal lines be ∆y. We will only consider those rectangles which intersect
the region R. Thus we will have a = x0 < x1 < · · · < xn = b and in the vertical direction,
we will have
yim(i) < yi(m(i)+1) < · · · < yiM(i)
where m (i) is the largest such that yim(i) is no larger than b (xi ) and M (i) is the smallest
such that yiM(i) is as large as y (xi ) . Then the iterated integral should satisfy the following
R b R t(x)
approximate equalities a b(x)
f (x, y) dydx =

n Z xi Z t(x) n Z xi Z t(xi )
∑ f (x, y) dydx ≈ ∑ f (xi , y) dydx
i=1 xi−1 b(x) i=1 xi−1 b(xi )
n Z xi M(i)
≈ ∑ ∑ f (xi , yi j ) ∆ydx
i=1 xi−1 j=m(i)

n M(i)
= ∑ ∑ f (xi , yi j ) ∆y∆x
i=1 j=m(i)

where we can extend f to be 0 off the region R. We would expect these approximations
to improve as ∆x, ∆y converge to 0, provided that the bounday of R is sufficiently “thin”.
Thus the iterated integral ought to equal the number to which the “Riemannn sums” repre-
sented by the last expression converge as ∆x, ∆y → 0. That sum on the right is really just a
systematic way of taking the value of the function at a point of a rectangle which intersects
19.1. METHODS FOR DOUBLE INTEGRALS 351

R, multiplying by the area of the rectangle containing this point and adding them together.
It would have worked out similarly if we had been able to do the iterated integral in the
other order, provided the boundary of R is “thin” enough, a completely stupid considera-
tion which is not needed in the context of the Lebesgue integral. We would still have a sum
of values of the function times areas of little rectangles. This is why it is entirely reasonable
to expect the iterated integrals in two different orders to be equal. It is also why the iterated
integral is approximating something which we call the Riemannn integral.
Definition 19.1.2 Let R be a bounded region in the xy plane and let f be a bounded func-
tion
R
defined on R. We say f is Riemannn integrable if there exists a number, denoted by
R f dA and called the Riemannn integral such that if ε > 0 is given, then whenever one im-
poses a sufficiently fine mesh enclosing R and considers the finitely many rectangles which
intersect R, numbered as {Qi }m i=1 and a point (xi , yi ) ∈ Qi , it follows that
Z
f dA − ∑ f (xi , yi ) area (Qi ) < ε
R i
R
It is R f dA which is of interest. The iterated integral should always be considered as a
tool for computing this number.R
When this is kept in mind, things become less confusing.
Also, it is helpful to consider R f dA as a kind of a glorified sum. It means to take the value
of f at a point and multiply by a little chunk of area dA and then add these together, hence
the integral sign which is really just an elongated symbol for a sum.
The careful explanation of these ideas is contained later in a special chapter devoted to
the theory of the Riemannn integral. It is not for the faint of heart. It is only there for those
who have a compelling need to understand all the details.
Example 19.1.3 Let f (x, y) = x2 y + yx for (x, y) ∈ R where R is the triangular region de-
fined
R
to be in the first quadrant, below the line y = x and to the left of the line x = 4. Find
R f dA.

y
R

4 x
From the above discussion,
Z Z 4Z x
x2 y + yx dy dx


f dA =
R 0 0

The reason for this is that x goes from 0 to 4 and for each fixed x between 0 and 4, y goes
from 0 to the slanted line, y = x, the function being defined to be
0 for larger y. Thus y goes
from 0 to x. This explains the inside integral. Now 0x x2 y + yx dy = 21 x4 + 12 x3 and so
R

Z 4 
1 4 1 3 672
Z
f dA = x + x dx = .
R 0 2 2 5
What of integration in a different order? Lets put the integral with respect to y on the
outside and the integral with respect to x on the inside. Then
Z Z 4Z 4
x2 y + yx dx dy


f dA =
R 0 y
352 CHAPTER 19. THE RIEMANNN INTEGRAL ON Rn

For each y between 0 and 4, the variable x, goes from y to 4.

Z 4
88 1 1
x2 y + yx dx = y − y4 − y3


y 3 3 2
Now Z 4 
88 1 1 672
Z
f dA = y − y4 − y3 dy = .
R 0 3 3 2 5
Here is a similar example.

Example 19.1.4 Let f (x, y) = x2 y for (x, y) ∈ R where R is the triangular region defined
R
to
be in the first quadrant, below the line y = 2x and to the left of the line x = 4. Find R f dA.

y
R
4 x
Put the integral with respect to x on the outside first. Then
Z Z 4 Z 2x
x2 y dy dx


f dA =
R 0 0

because for each x ∈ [0, 4], y goes from 0 to 2x. Then

Z 2x
x2 y dy = 2x4


0

and so Z 4
2048
Z
2x4 dx =


f dA =
R 0 5
Now do the integral in the other order. Here the integral with respect to y will be on the
outside. What are the limits of this integral? Look at the triangle and note that x goes from
0 to 4 and so 2x = y goes from 0 to 8. Now for fixed y between 0 and 8, where does x go? It
goes from the x coordinate on the line y = 2x which corresponds to this y to 4. What is the
x coordinate on this line which goes with y? It is x = y/2. Therefore, the iterated integral is
Z 8Z 4
x2 y dx dy.


0 y/2

Now Z 4
64 1
x2 y dx = y − y4


y/2 3 24
and so Z 8 
64 1 4 2048
Z
f dA = y− y dy =
R 0 3 24 5
the same answer. R
A few observations are in order here. In finding S f dA there is no problem in setting
things up if S is a rectangle. However, if S is not a rectangle, the procedure always is
19.1. METHODS FOR DOUBLE INTEGRALS 353

agonizing. A good rule of thumb is that if what you do is easy it will be wrong. There
are no shortcuts! There are no quick fixes which require no thought! Pain and suffering
is inevitable and you must not expect it to be otherwise. Always draw a picture and then
begin agonizing over the correct limits. Even when you are careful you will make lots of
mistakes until you get used to the process.
Sometimes an integral can be evaluated in one order but not in another.

Example 19.1.5 For R as shown below, find R sin y2 dA.

R

4 x
Setting this up to have the integral with respect to y on the inside yields
Z 4Z 8
sin y2 dy dx.


0 2x

Unfortunately, there is no antiderivative in terms of elementary functions for sin y2 so

there is an immediate problem in evaluating the inside integral. It doesn’t work out so the
next step is to do the integration in another order and see if some progress can be made.
This yields
Z 8 Z y/2 Z 8
y
sin y2 dx dy = sin y2 dy



0 0 0 2
R8 y 2

1 1
and 0 2 sin y dy = − 4 cos 64 + 4 which you can verify by making the substitution, u =
y2 . Thus
1 1
Z
sin y2 dy = − cos 64 + .


R 4 4
This illustrates an important idea. The integral R sin y2 dA is defined as a number.
R

It is the unique number between all the upper sums and all the lower sums. Finding it is
another matter. In this case it was possible to find it using one order of integration but not
the other. The iterated integral in this other order also is defined as a number but it cannot be
found directly without interchanging the order of integration. Of course sometimes nothing
you try will work out.

19.1.1 Density And Mass

Consider a two dimensional material. Of course there is no such thing but a flat plate
might be modeled as one. The density ρ is a function of position and is defined as follows.
Consider a small chunk of area dA located at the point whose Cartesian coordinates are
(x, y). Then the mass of this small chunk of material is given by ρ (x, y) dA. Thus if the
material occupies a region in two dimensional space U, the total mass of this material
would be Z
ρ dA
U
354 CHAPTER 19. THE RIEMANNN INTEGRAL ON Rn

In other words you integrate the density to get the mass. Now by letting ρ depend on
position, you can include the case where the material is not homogeneous. Here is an
example.

Example 19.1.6 Let ρ (x, y) denote the density of the plane region determined by the curves
1 2
3 x + y = 2, x = 3y , and x = 9y. Find the total mass if ρ (x, y) = y.

You need to first draw a picture of the region R. A rough sketch follows.

(3, 1)
(1/3)x + y = 2
x = 3y2
(9/2, 1/2)

x = 9y
(0, 0)

This region is in two pieces, one having the graph of x = 9y on the bottom and the
graph of x = 3y2 on the top and another piece having the graph of x = 9y on the bottom and
the graph of 31 x + y = 2 on the top. Therefore, in setting up the integrals, with the integral
with respect to x on the outside, the double integral equals the following sum of iterated
integrals.

has x=3y2 on top has 13 x+y=2 on top

Z 3 Z √x/3
z }| { z }| {
Z 9 Z 2− 1 x
2 3
y dy dx + y dy dx
0 x/9 3 x/9

You notice it is not necessary to have a perfect picture, just one which is good enough to
figure out what the limits should be. The dividing line between the two cases is x = 3 and
this was shown in the picture. Now it is only a matter of evaluating the iterated integrals
which in this case is routine and gives 1.

19.2 Exercises
1. Evaluate the iterated integral and then write the iterated integral with the order of
integration reversed. 04 03y x dx dy.
R R

2. Evaluate the iterated integral and then write the iterated integral with the order of
integration reversed. 03 03y y dx dy.
R R

3. Evaluate the iterated integral and then write the iterated integral with the order of
integration reversed. 02 02y (x + 1) dx dy.
R R

4. Evaluate the iterated Rintegral and then write the iterated integral with the order of
integration reversed. 03 0y sin (x) dx dy.
R

5. Evaluate the iterated Rintegral and then write the iterated integral with the order of
integration reversed. 01 0y exp (y) dx dy.
R
19.3. METHODS FOR TRIPLE INTEGRALS 355

6. Let ρ (x, y) denote the density of the plane region closest to (0, 0) which is between
the curves x + 2y = 3, x = y2 , and x = 0. Find the total mass if ρ (x, y) = y. Set up the
integral in terms of dxdy and in terms of dydx.
7. Let ρ (x, y) denote the density of the plane region determined by the curves x + 2y =
3, x = y2 , and x = 4y. Find the total mass if ρ (x, y) = x. Set up the integral in terms
of dxdy and dydx.
8. Let ρ (x, y) denote the density of the plane region determined by the curves y =
2x, y = x, x + y = 3. Find the total mass if ρ (x, y) = y + 1. Set up the integrals in
terms of dxdy and dydx.
9. Let ρ (x, y) denote the density of the plane region determined by the curves y =
3x, y = x, 2x + y = 4. Find the total mass if ρ (x, y) = 1.
10. Let ρ (x, y) denote the density of the plane region determined by the curves y =
3x, y = x, x + y = 2. Find the total mass if ρ (x, y) = x + 1. Set up the integrals in
terms of dxdy and dydx.
11. Let ρ (x, y) denote the density of the plane region determined by the curves y =
5x, y = x, 5x + 2y = 10. Find the total mass if ρ (x, y) = 1. Set up the integrals in
terms of dxdy and dydx.
R 4 R 2 1 2y
12. Find 0 y/2 x e dx dy. You might need to interchange the order of integration.
x

R 8 R 4 1 3y
13. Find 0 y/2 x e dx dy.
x

R 13 π R 31 π sin y
14. Find 0 x y dy dx.
R 12 π R 1 π sin y
15. Find 0
2
x y dy dx.
R π R π sin y
16. Find 0 x y dy dx

17. ∗ Evaluate the iterated integral and then write the iterated integral with the order of
R3 Rx 2
integration reversed. −3 −x x dy dx
−y 2
Your answer for the iterated integral should be 30 −3 x dx dy + 0−3 −3
y 2
R R R R
x dx dy +
R3R3 2 R0 R3 2
0 y x dx dy + −3 −y x dx dy. This is a very interesting example which shows that
iterated integrals have a life of their own, not just as a method for evaluating double
integrals.
18. ∗ Evaluate the iterated integral and then write the iterated integral with the order of
R2 Rx 2
integration reversed. −2 −x x dy dx.

19.3 Methods For Triple Integrals

19.3.1 Definition Of The Integral
The integral of a function of three variables is similar to the integral of a function of two
variables. In this case, the term: “mesh” refers to a collection of little boxes which covers
a given region in R.
356 CHAPTER 19. THE RIEMANNN INTEGRAL ON Rn

Definition 19.3.1 Let R be a bounded region in the R3 and let f be a bounded function
R
defined on R. We say f is Riemannn integrable if there exists a number, denoted by R f dV
and called the Riemannn integral such that if ε > 0 is given, then whenever one imposes a
sufficiently fine mesh enclosing R and considers the finitely many boxes which intersect R,
numbered as {Qi }m i=1 and a point (xi , yi , zi ) ∈ Qi , it follows that

Z
f dV − ∑ f (xi , yi , zi ) volume (Qi ) < ε
R i

Of course one can continue generalizing to higher dimensions by analogy. By exactly

similar reasoning to the case of integrals of functions of two variables, we can consider
iterated integrals as a tool for finding the Riemannn integral of a function of three or more
variables.

19.3.2 Iterated Integrals

As before, the integral is often computed by using an R
iterated integral. In general it is
impossible to set up an iterated integral for finding E f dV for arbitrary regions, E but
when the region is sufficiently simple, one can make progress. Suppose the region E over
which the integral is to be taken is of the form E = {(x, y, z) : a (x, y) ≤ z ≤ b (x, y)} for
(x, y) ∈ R, a two dimensional region. This is illustrated in the following picture in which
the bottom surface is the graph of z = a (x, y) and the top is the graph of z = b (x, y).

x
R

Then Z Z Z b(x,y)
f dV = f (x, y, z) dzdA
E R a(x,y)
R b(x,y)
It might be helpful to think of dV = dzdA. Now a(x,y) f (x, y, z) dz is a function of x and
y and so you have reduced the triple integral to a double integral over R of this func-
19.3. METHODS FOR TRIPLE INTEGRALS 357

tion of x and y. Similar reasoning would apply if the region in R3 were of the form
{(x, y, z) : a (y, z) ≤ x ≤ b (y, z)} or {(x, y, z) : a (x, z) ≤ y ≤ b (x, z)}.
Example 19.3.2 Find the volume of the region E in the first octant between z = 1 − (x + y)
and z = 0.
In this case, R is the region shown.
z
y

R y
x
1 x
Thus the region E is between the plane z = 1 − (x + y) on the top, z = 0 on the bottom,
and over R shown above. Thus
Z Z 1−(x+y) Z 1 Z 1−x Z 1−(x+y)
1
Z
1dV = dzdA = dzdydx =
E R 0 0 0 0 6
Of course iterated integrals have a life of their own although this will not be explored
here. You can just write them down and go to work on them. Here are some examples.
R3RxRx
Example 19.3.3 Find 2 3 3y (x − y) dz dy dx.
x
(x − y) dz = x2 − 4xy + 3y2 . Next this must be integrated
R
The inside integral yields 3y
Rx 2
with respect to y to give 3 x − 4xy + 3y2 dy = −3x2 +18x−27. Finally the third integral

gives
Z 3Z xZ x Z 3
−3x2 + 18x − 27 dx = −1.


(x − y) dz dy dx =
2 3 3y 2
R π R 3y R y+z
Example 19.3.4 Find 0 0 0 cos (x + y) dx dz dy.
R y+z
The inside integral is 0 cos (x + y) dx = 2 cos z sin y cos y + 2 sin z cos2 y − sin z − sin y.
Now this has to be integrated.
Z 3y Z y+z
cos (x + y) dx dz
0 0
Z 3y
2 cos z sin y cos y + 2 sin z cos2 y − sin z − sin y dz


=
0

= −1 − 16 cos5 y + 20 cos3 y − 5 cos y − 3 (sin y) y + 2 cos2 y.

Finally, this last expression must be integrated from 0 to π. Thus
Z π Z 3y Z y+z
cos (x + y) dx dz dy
0 0 0
Z π 
= −1 − 16 cos5 y + 20 cos3 y − 5 cos y − 3 (sin y) y + 2 cos2 y dy = −3π
0
358 CHAPTER 19. THE RIEMANNN INTEGRAL ON Rn

R 2 R 3− 23 x R x2
Example 19.3.5 Here is an iterated integral: 0 0 0 dz dy dx. Write as an iterated
integral in the order dz dx dy.

The inside integral is just a function of x and y. (In fact, only a function of x.) The order
of the last two integrals must be interchanged. Thus the iterated integral which needs to be
done in a different order is
Z 2 Z 3− 3 x
2
f (x, y) dy dx.
0 0
As usual, it is important to draw a picture and then go from there.

3 − 23 x = y

2
Thus this double integral equals
Z 3Z 2 (3−y)
3
f (x, y) dx dy.
0 0

Now substituting in for f (x, y),

2 x2
3 (3−y)
Z 3Z Z
dz dx dy.
0 0 0

Example 19.3.6 Find the volume of the bounded region determined by 3y + 3z = 2, x =

16 − y2 , y = 0, x = 0.

In the yz plane, the first of the following pictures corresponds to x = 0.

y (0, 0, 0)
2
3

3y + 3z = 2
x = 16 − y2

2 z
3
R 2 R 2 −y
Therefore, the outside integrals taken with respect to z and y are of the form 03 03 dz dy,
and now for any choice of (y, z) in the above triangular region, x goes from 0 to 16 − y2 .
Therefore, the iterated integral is
2 2 −y Z 16−y2 860
Z Z
3 3
dx dz dy =
0 0 0 243
Example 19.3.7 Find the volume of the region determined by the intersection of the two
cylinders, x2 + y2 ≤ 1 and x2 + z2 ≤ 1.
19.4. EXERCISES 359

The first listed cylinder intersects the xy plane in the disk, x2 + y2 ≤ 1. What is the
volume
√ of the three dimensional
√ region which is between this disk and the two surfaces,
z = 1 − x2 and z = − 1 − x2 ? An iterated integral for the volume is
√ √
Z 1 Z 1−x2 Z 1−x2
16
√ √ dz dydx = .
−1 − 1−x2 − 1−x2 3

Note that I drew no picture of the three dimensional region. If you are interested, here it is.

One of the cylinders is parallel to the z axis, x2 + y2 ≤ 1 and the other is parallel to the
y axis, x2 + z2 ≤ 1. I did not need to be able to draw such a nice picture in order to work
this problem. This is the key to doing these. Draw pictures in two dimensions and reason
from the two dimensional pictures rather than attempt to wax artistic and consider all three
dimensions at once. These problems are hard enough without making them even harder by
attempting to be an artist.

19.4 Exercises
1. Find the following iterated integrals.
R 3 R 2z R z+1
(a) −1 0 y (x + y) dxdydz
R 1 R z R z2
(b) 0 0 y (y + z) dxdydz
R 3 R x R 3x−y
(c) 0 1 2 sin (x) dzdydx
R 1 R 2x R 2y
(d) 0 x y dzdydx
R 4 R 2x R x
(e) 2 2 2y dz dy dx
R 3 R 2−5x R 2−x−2y
(f) 0 0 0 2x dz dy dx
R 2 R 1−3x R 3−3x−2y
(g) 0 0 0 x dz dy dx
R π R 3y R y+z
(h) 0 0 0 cos (x + y) dx dz dy
R π R 4y R y+z
(i) 0 0 0 sin (x + y) dx dz dy

2. Fill in the missing limits.

R1RzRz R?R?R?
0 0 0 f (x, y, z) dx dy dz = ? ? ? f (x, y, z) dx dz dy,
360 CHAPTER 19. THE RIEMANNN INTEGRAL ON Rn

R 1 R z R 2z R?R?R?
0 0 0 f (x, y, z) dx dy dz = ? ? ? f (x, y, z) dy dz dx,
R1RzRz R?R?R?
0 0 0 f (x, y, z) dx dy dz = ? ? ? f (x, y, z) dz dy dx,
R 1 R √z R y+z
f (x, y, z) dx dy dz = ?? ?? ?? f (x, y, z) dx dz dy,
R R R
0 z/2 0
R6R6R4 R?R?R?
4 2 0 f (x, y, z) dx dy dz = ? ? ? f (x, y, z) dz dy dx.
3. Find the volume of R where R is the bounded region formed by the plane 51 x + y +
1
4 z = 1 and the planes x = 0, y = 0, z = 0.

4. Find the volume of R where R is the bounded region formed by the plane 15 x + 12 y +
1
4 z = 1 and the planes x = 0, y = 0, z = 0.

5. Find the volume of R where R is the bounded region formed by the plane 15 x + 13 y +
1
4 z = 1 and the planes x = 0, y = 0, z = 0.

6. Find the volume of the bounded region determined by 3y + z = 3, x = 4 − y2 , y =

0, x = 0.

7. Find the volume of the region bounded by x2 + y2 = 16, z = 3x, z = 0, and x ≥ 0.

8. Find the volume of R where R is the bounded region formed by the plane 14 x + 12 y +
1
4 z = 1 and the planes x = 0, y = 0, z = 0.
2
9. Here is an iterated integral: 03 03−x 0x dz dy dx. Write as an iterated integral in the
R R R

following orders: dz dx dy, dx dz dy, dx dy dz, dy dx dz, dy dz dx.

10. Find the volume of the bounded region determined by 2y + z = 3, x = 9 − y2 , y =

0, x = 0, z = 0.
11. Find the volume of the bounded region determined by y + 2z = 3, x = 9 − y2 , y =
0, x = 0.

12. Find the volume of the bounded region determined by y + z = 2, x = 3 − y2 , y = 0, x =

0.
13. Find the volume of the region bounded by x2 + y2 = 25, z = x, z = 0, and x ≥ 0.
250
Your answer should be 3 .

14. Find the volume of the region bounded by x2 + y2 = 9, z = 3x, z = 0, and x ≥ 0.

19.4.1 Mass And Density

As an example of the use of triple integrals, consider a solid occupying a set of points
U ⊆ R3R having density ρ. Thus ρ is a function of position and the total mass of the solid
equals U ρ dV . This is just like the two dimensional case. The mass of an infinitesimal
chunk of Rthe solid located at x would be ρ (x) dV and so the total mass is just the sum of
all these, U ρ (x) dV .

Example 19.4.1 Find the volume of R where R is the bounded region formed by the plane
1 1
5 x + y + 5 z = 1 and the planes x = 0, y = 0, z = 0.
19.4. EXERCISES 361

When z = 0, the plane becomes 15 x + y = 1. Thus the intersection of this plane with the
xy plane is this line shown in the following picture.

5
Therefore, the bounded region is between the triangle formed in the above picture by
the x axis, the y axis and the above line and the surface given by 51 x + y + 15 z = 1 or z =
5 1 − 15 x + y = 5 − x − 5y. Therefore, an iterated integral which yields the volume is

Z 5 Z 1− 1 x Z 5−x−5y
5 25
dz dy dx = .
0 0 0 6

Example 19.4.2 Find the mass of the bounded region R formed by the plane 13 x+ 31 y+ 15 z =
1 and the planes x = 0, y = 0, z = 0 if the density is ρ (x, y, z) = z.

This is done just like the previous example except in this case, there is a function to
integrate. Thus the answer is
Z 3 Z 3−x Z 5− 5 x− 5 y
3 3 75
z dz dy dx = .
0 0 0 8

Example 19.4.3 Find the total mass of the bounded solid determined by z = 9 − x2 − y2
and x, y, z ≥ 0 if the mass is given by ρ (x, y, z) = z

When z = 0 the surface z = 9 − x2 − y2 intersects the xy plane in a circle of radius 3

centered at (0, 0). Since x, y ≥ 0, it is only a quarter of a circle of interest, the part where
both these variables are nonnegative. For each (x, y) inside this quarter circle, z goes from
0 to 9 − x2 − y2 . Therefore, the iterated integral is of the form,
q
Z 3Z (9−x2 ) Z 9−x2 −y2 243
z dz dy dx = π
0 0 0 8
Example 19.4.4 Find the volume of the bounded region determined by x ≥ 0, y ≥ 0, z ≥ 0,
and 17 x + y + 14 z = 1, and x + 17 y + 14 z = 1.

When z = 0, the plane 17 x + y + 41 z = 1 intersects the xy plane in the line whose equation
1
is7x + y = 1, while the plane, x + 17 y + 14 z = 1 intersects the xy plane in the line whose
equation is x + 17 y = 1. Furthermore, the two planes intersect when x = y as can be seen
from the equations, x + 17 y = 1 − 4z and 17 x + y = 1 − 4z which imply x = y. Thus the two
362 CHAPTER 19. THE RIEMANNN INTEGRAL ON Rn

dimensional picture to look at is depicted in the following picture.

x + 71 y + 14 z = 1

y=x

R1 y + 71 x + 14 z = 1
R2

You see in this picture, the base of the region in the xy plane is the union of the two
triangles, R1 and R2 . For (x, y) ∈ R1 , z goes from 0 to what it needs to be to be on the plane,
1 1 1


7 x + y + z
4
= 1. Thus z goes from 0 to 4 1 − 7 x − y . Similarly, on R2 , z goes from 0 to
4 1 − 17 y − x . Therefore, the integral needed is
Z Z 4(1− 1 x−y) Z Z 4(1− 1 y−x)
7 7
dz dV + dz dV
R1 0 R2 0
R R
and now it only remains to consider R1 dV and R2 dV. The point of intersection of these
lines shown in the above picture is 87 , 78 and so an iterated integral is

Z 7/8 Z 1− x Z 4(1− 1 x−y) Z 7/8 Z 1− y Z 4(1− 1 y−x)

7 7 7 7 7
dz dy dx + dz dx dy =
0 x 0 0 y 0 6

19.5 Exercises
1. Find the volume of the region determined by the intersection of the two cylinders,
x2 + y2 ≤ 16 and y2 + z2 ≤ 16.
2. Find the volume of the region determined by the intersection of the two cylinders,
x2 + y2 ≤ 9 and y2 + z2 ≤ 9.
3. Find the volume of the region bounded by x2 + y2 = 4, z = 0, z = 5 − y
4. Find 02 06−2z 13−z (3 − z) cos y2 dy dx dz.
R R R

x 2
R 1 R 18−3z R 6−z
(6 − z) exp y2 dy dx dz.


5. Find 0 0 1x
3
R 2 R 24−4z R 6−z
(6 − z) exp x2 dx dy dz.


6. Find 0 0 1
4y
R 1 R 10−2z R 5−z sin x
7. Find 0 0 1 x dx dy dz.
2y
Hint: Interchange order of integration.
19.5. EXERCISES 363

8. Find the mass of the bounded region R formed by the plane 14 x + 12 y + 13 z = 1 and the
planes x = 0, y = 0, z = 0 if the density is ρ (x, y, z) = y
9. Find the mass of the bounded region R formed by the plane 12 x + 12 y + 14 z = 1 and the
planes x = 0, y = 0, z = 0 if the density is ρ (x, y, z) = z2
10. Find the mass of the bounded region R formed by the plane 14 x + 12 y + 14 z = 1 and the
planes x = 0, y = 0, z = 0 if the density is ρ (x, y, z) = y + z

11. Find the mass of the bounded region R formed by the plane 14 x + 12 y + 15 z = 1 and the
planes x = 0, y = 0, z = 0 if the density is ρ (x, y, z) = y
R 1 R 12−4z R 3−z sin x
12. Find 0 0 1 x dx dy dz.
4y

R 20 R 2 R 6−z sin x R 30 R 6− 15 y R 6−z sin x

13. Find 0 0 1 x dx dz dy + 20 0 1 x dx dz dy.
5y 5y

14. Find the volume of the bounded region determined by x ≥ 0, y ≥ 0, z ≥ 0, and 21 x +

y + 12 z = 1, and x + 12 y + 12 z = 1.

15. Find the volume of the bounded region determined by x ≥ 0, y ≥ 0, z ≥ 0, and 71 x +

y + 13 z = 1, and x + 17 y + 13 z = 1.
16. Find an iterated integral for the volume of the region between the graphs of z =
x2 + y2 and z = 2 (x + y).
17. Find the volume of the region which lies between z = x2 + y2 and the plane z = 4.

18. The base of a solid is the region in the xy plane between the curves y = x2 and y = 1.
The top of the solid is the plane z = 2 − x. Find the volume of the solid.
19. The base of a solid is in the xy plane and is bounded by the lines y = x, y = 1 − x, and
y = 0. The top of the solid is z = 3 − y. Find its volume.

20. The base of a solid is in the xy plane and is bounded by the lines x = 0, x = π, y = 0,
and y = sin x. The top of this solid is z = x. Find the volume of this solid.
364 CHAPTER 19. THE RIEMANNN INTEGRAL ON Rn
Chapter 20

The Integral In Other

Coordinates

20.1 Polar Coordinates

Recall the relation between the rectangular coordinates and polar coordinates is
! !
x r cos (θ )
x (r, θ ) ≡ = , r ≥ 0, θ ∈ [0, 2π)
y r sin (θ )

Now consider the part of grid obtained by fixing θ at various values and varying r and then
by fixing r at various values and varying θ .

The idea is that these lines obtained by fixing one or the other coordinate are very
close together, much closer than drawn and so we would expect the area of one of the
little curvy quadrilaterals to be close to the area of the parallelogram shown. Consider
this parallelogram. The two sides originating at the intersection of two of the grid lines as
shown are approximately equal to

xr (r, θ ) dr, xθ (r, θ ) dθ

where dr and dθ are the respective small changes in the variables r and θ . Thus the area
of one of those little curvy shapes should be approximately equal to

|xr (r, θ ) dr × xθ (r, θ ) dθ |

365
366 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

by the geometric description of the cross product. These vectors are extended as 0 in the
third component in order to take the cross product. This reduces to
!
cos (θ ) −r sin (θ )
dA = det drdθ = rdrdθ
sin (θ ) r cos (θ )

which is the increment of area in polar coordinates, taking the place of dxdy. The integral
is really about taking the value of the function integrated multiplied by dA and adding these
products. Here is an example.

Example 20.1.1 Find the area of a circle of radius a.

The variable r goes from 0 to a and the angle θ goes from 0 to 2π. Therefore, the area
is Z Z 2π Z a
dA = rdrdθ = πa2
D 0 0

Example 20.1.2 The density equals r. Find the total mass of a disk of radius a.

This is easy to do in polar coordinates. The disk involved has θ going from 0 to 2π and
r from 0 to 2. Therefore, the integral to work is just
dA
Z 2π Z a z }| { 2
rrdrdθ = πa3
0 0 3
Notice how in these examples the circular disk is really a rectangle [0, 2π] × [0, a]. This is
why polar coordinates are so useful. The next example was worked earlier from a different
point of view.

Example 20.1.3 Find the area of the inside of the cardioid r = 1 + cos θ , θ ∈ [0, 2π].

Here the integral is

Z 2π Z 1+cos(θ )
3
rdrdθ = π
0 0 2
To see how impossible this problem is in rectangular coordinates, draw the graph of the
cardioid.

-1

0 1 2
How would you go about setting this up in rectangular coordinates? It would be very
hard if not impossible, but is easy in polar coordinates. This is because in polar coordinates
the region integrated over is the region below the curve in the following picture.
20.2. EXERCISES 367

Example 20.1.4 Let R denote the inside of the cardioid r = 1 + cos θ for θ ∈ [0, 2π]. Find
Z
xdA
R

Here the convenient increment of area is rdrdθ and so the integral is

Z 2π Z 1+cos(θ )
xrdrdθ
0 0

Now you need to change x to the right coordinates. Thus the integral equals
Z 2π Z 1+cos(θ )
5
(r cos (θ )) rdrdθ = π
0 0 4
A case where this sort of problem occurs is when you find the mass of a plate given the
density.

Definition 20.1.5 Suppose a material occupies a region of the plane R. The density λ is a
nonnegative
R
function of position with the property that if B ⊆ R, then the mass of B is given
by B λ dA. In particular, this is true of B = R.

Example 20.1.6 Let R denote the inside of the polar curve r = 2 + sin θ . Let λ = 3 + x.
Find the total mass of R.

As above, this is
Z 2π Z 2+sin(θ )
27
(3 + r cos (θ )) rdrdθ = π
0 0 2

20.2 Exercises
1. Sketch a graph in polar coordinates of r = 2+sin (θ ) and find the area of the enclosed
region.
2. Sketch a graph in polar coordinates of r = sin (4θ ) and find the area of the region
enclosed. Hint: In this case, you need to worry and fuss about r < 0.
3. Suppose the density is λ (x, y) = 2 − x and the region is the interior of the cardioid
r = 1 + cos θ . Find the total mass.
4. Suppose the density is λ = 4 − x − y and find the mass of the plate which is between
the concentric circles r = 1 and r = 2.

5. Suppose the density is λ = 4 − x − y and find the mass of the plate which is inside
the polar graph of r = 1 + sin (θ ).
368 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

6. Suppose the density is 2 + x. Find the mass of the plate which is the inside of the
polar curve r = sin (2θ ). Hint: This is one of those fussy things with negative radius.
7. The area density of a plate is given by λ = 1 + x and the plate occupies the inside of
the cardioid r = 1 + cos θ . Find its mass.
8. The moment about R
the x axis of a plate with density λ occupying the region R is
defined
R
as m y = R yλ dA. The moment about the y axis of the same plate is mx =
R xλ dA. If λ = 2 − x, find the moments about the x and y axes of the plate inside
r = 2 + sin (θ ).
9. Using the above problem, find the moments about the x and y axes of a plate having
density 1 + x for the plate which is the inside of the cardioid r = 1 + cos θ .
10. Use the same plate as the above but this time, let the density be (2 + x + y). Find the
moments.
2 2
11. Let D = R(x, y) : x2 + y2 ≤ 25 . Find D e25x +25y dx dy. Hint: This is an integral of
 R

the form D f (x, y) dA. Write in polar coordinates and it will be fairly easy.
12. Let D = (x, y) :Rx2 + y2 ≤ 16 . Find D cos 9x2 + 9y2 dx [Link]: This is an inte-
 R

gral of the form D f (x, y) dA. Write in polar coordinates and it will be fairly easy.
13. Derive a formula for area between two polar graphs using the increment of area of
polar coordinates.
14. Use polar coordinates Rto evaluate the following integral. Here S is given in terms of
the polar coordinates. S sin 2x2 + 2y2 dV where r ≤ 2 and 0 ≤ θ ≤ 23 π.

2 2
15. Find S e2x +2y dV where S is given in terms of the polar coordinates r ≤ 2 and
R

0 ≤ θ ≤ π.
R y
16. Find dV where S is described in polar coordinates as 1 ≤ r ≤ 2 and 0 ≤ θ ≤ π/4.
Sx

2 
17. Find S xy + 1 dV where S is given in polar coordinates as 1 ≤ r ≤ 2 and 0 ≤
R

θ ≤ 61 π.
18. A right circular cone has a base of radius 2 and a height equal to 2. Use polar
coordinates to find its volume.
19. Now suppose in the above problem, it is not really a cone but instead z = 2 − 12 r2 .
Find its volume.

20.3 Cylindrical And Spherical Coordinates

Cylindrical coordinates are defined as follows.
   
x r cos (θ )
x (r, θ , z) ≡  y  =  r sin (θ )  ,
   

z z
r ≥ 0, θ ∈ [0, 2π), z ∈ R
20.3. CYLINDRICAL AND SPHERICAL COORDINATES 369

Spherical coordinates are a little harder. These are given by

   
x ρ sin (φ ) cos (θ )
x (ρ, θ , φ ) ≡  y  =  ρ sin (φ ) sin (θ )  ,
   

z ρ cos (φ )
ρ ≥ 0, θ ∈ [0, 2π), φ ∈ [0, π]

The following picture relates the various coordinates.

z

(ρ, φ , θ )
z1 (r, θ , z1 )
•(x1 , y1 , z1 )

φ ρ
y1
θ y
r

x1 (x1 , y1 , 0)
x
In this picture, ρ is the distance between the origin, the point whose Cartesian coor-
dinates are (0, 0, 0) and the point indicated by a dot and labelled as (x1 , y1 , z1 ), (r, θ , z1 ),
and (ρ, φ , θ ). The angle between the positive z axis and the line between the origin and
the point indicated by a dot is denoted by φ , and θ is the angle between the positive x
axis and the line joining the origin to the point (x1 , y1 , 0) as shown, while r is the length
of this line. Thus r = ρ sin (φ ) and is the usual polar coordinate while θ is the other polar
coordinate. Letting z1 denote the usual z coordinate of a point in three dimensions, like
the one shown as a dot, (r, θ , z1 ) are the cylindrical coordinates of the dotted point. The
spherical coordinates are determined by (ρ, φ , θ ). When ρ is specified, this indicates that
the point of interest is on some sphere of radius ρ which is centered at the origin. Then
when φ is given, the location of the point is narrowed down to a circle of “latitude” and
finally, θ determines which point is on this circle by specifying a circle of “longitude”. Let
φ ∈ [0, π], θ ∈ [0, 2π), and ρ ∈ [0, ∞). The picture shows how to relate these new coordinate
systems to Cartesian coordinates. Note that θ is the same in the two coordinate systems
and that ρ sin φ = r.

20.3.1 Volume and Integrals in Cylindrical Coordinates

The increment of three dimensional volume in cylindrical coordinates is dV = rdrdθ dz.
It is just a chunk of two dimensional area rdrdθ times the height dz which gives three
dimensional volume. Here is an example.

Example 20.3.1 Find the volume of the three dimensional region between the graphs of
z = 4 − 2y2 and z = 4x2 + 2y2 .
370 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

Where do the two surfaces intersect? This happens when 4x2 + 2y2 = 4 − 2y2 which is
the curve in the xy plane, x2 + y2 = 1. Thus (x, y) is on the inside of this circle while z goes
from 4x2 + 2y2 to 4 − 2y2 . Denoting the unit disk by D, the desired integral is
Z Z 4−2y2
dzdA
D 4x2 +2y2

I will use the dA which corresponds to polar coordinates so this will then be in cylindrical
coordinates. Thus the above equals
Z 2π Z 1 Z 4−2(r2 sin2 (θ ))
dzrdrdθ = 2π
0 0 4(r2 cos2 (θ ))+2(r2 sin2 (θ ))

Note this is really not much different than simply using polar coordinates to integrate the
difference of the two values of z This is
Z Z
2 2 2
4 − 4r2 dA



4 − 2y − 4x + 2y dA =
D D
Z 2π Z 1
4 − 4r2 rdrdθ = 2π


=
0 0

Here is another example.

Example 20.3.2 Find the volume of the three dimensional region between the graphs of
z = 0, z = x2 + y2 , and the cylinder (x − 1)2 + y2 = 1.
p

Consider the cylinder. It reduces to r2 = 2r cos θ or more simply r = 2 cos θ . This is

the graph of a circle having radius 1 and centered at (1, 0). Therefore, θ ∈ [−π/2, π/2]. It
follows that the cylindrical coordinate description of this volume is
Z π/2 Z 2 cos θ Z r
32
dzrdrdθ =
−π/2 0 0 9
20.3. CYLINDRICAL AND SPHERICAL COORDINATES 371

20.3.2 Volume And Integrals in Spherical Coordinates

What is the increment of volume in spherical coordinates? There are two ways to see what
this is, through art and through a systematic procedure. First consider art. Here is a picture.

ρdϕ dρ

ρ sin(ϕ)dθ

In the picture there are two concentric spheres formed by making ρ two different con-
stants and surfaces which correspond to θ assuming two different constants and φ assuming
two different constants. These intersecting surfaces form the little box in the picture. Here
is a more detailed blow up of the little box.

ρ sin(φ )dθ

dθ
ρdφ
dρ

dφ

has coordinates (ρ, φ , θ )

x y

What is the volume of this little box? Length ≈ ρdφ , width ≈ ρ sin (φ ) dθ , height ≈ dρ
and so the volume increment for spherical coordinates is

dV = ρ 2 sin (φ ) dρdθ dφ
Now what is really going on? Consider the dot in the picture of the little box. Fixing θ
and φ at their values at this point and differentiating with respect to ρ leads to a little vector
372 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

of the form  
sin (φ ) cos (θ )
 sin (φ ) sin (θ )  dρ
 

cos (φ )
which points out from the surface of the sphere. Next keeping ρ and θ constant and differ-
entiating only with respect to φ leads to an infinitesimal vector in the direction of a line of
longitude,  
ρ cos (φ ) cos (θ )
 ρ cos (φ ) sin (θ )  dφ
 

−ρ sin (φ )
and finally keeping ρ and φ constant and differentiating with respect to θ leads to the third
infinitesimal vector which points in the direction of a line of latitude.
 
−ρ sin (φ ) sin (θ )
 ρ sin (φ ) cos (θ )  dθ
 

0
To find the increment of volume, we just need to take the absolute value of the determi-
nant which has these vectors as columns, (Remember this is the absolute value of the box
product.) exactly as was the case for polar coordinates. This will also yield
dV = ρ 2 sin (φ ) dρdθ dφ .
However, in contrast to the drawing of pictures, this procedure is completely general
and will handle all curvilinear coordinate systems and in any dimension. This is discussed
more later.
2
R
Example 20.3.3 Find the volume of a ball, BR of radius R. Then find BR z dV where z is
the rectangular z coordinate of a point.
In this case, U = (0, R] × [0, π] × [0, 2π) and use spherical coordinates. Then this yields
a set in R3 which clearly differs from the ball of radius R only by a set having volume equal
to zero. It leaves out the point at the origin is all. Therefore, the volume of the ball is
Z Z
1 dV = ρ 2 sin φ dV
BR U
Z R Z π Z 2π
4
= ρ 2 sin φ dθ dφ dρ = R3 π.
0 0 0 3
The reason this was effortless, is that the ball, BR is realized as a box in terms of the
spherical coordinates. Remember what was pointed out earlier about setting up iterated
integrals over boxes.
As for the integral, it is no harder to set up. You know from the transformation equations
that z = ρ cos φ . Then you want
Z R Z π Z 2π
4
Z
zdV = (ρ cos (φ ))2 ρ 2 sin φ dθ dφ dρ = πR5
BR 0 0 0 15
This will be pretty easy also although somewhat more messy because the function you are
integrating is not just 1 as it is when you find the volume.
20.3. CYLINDRICAL AND SPHERICAL COORDINATES 373

Example 20.3.4 A cone is cut out of a ball of radius R as shown in the following picture,
the diagram on the left being a side view. The angle of the cone is π/3. Find the volume of
what is left.

π
3

Use spherical coordinates. This volume is then

1√
Z π Z 2π Z R
2
ρ 2 sin (φ ) dρdθ dφ = πR3 + 3πR3
π/6 0 0 3 3
Now change the example a little by cutting out a cone at the bottom which has an angle
of π/2 as shown. What is the volume of what is left?

This time you would have the volume equals

1√ 1√
Z 3π/4 Z 2π Z R
ρ 2 sin (φ ) dρdθ dφ = 2πR3 + 3πR3
π/6 0 0 3 3
Example 20.3.5 Next suppose the ball of radius R is a sort of an orange and you remove a
slice as shown in the picture. What is the volume of what is left? Assume the slice is formed
by the two half planes θ = 0 and θ = π/4.
374 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

Using spherical coordinates, this gives for the volume

Z π Z 2π Z R
7
ρ 2 sin (φ ) dρdθ dφ = πR3
0 π/4 0 6
Example 20.3.6 Now remove the same two cones as in the above examples along with the
same
R
slice and find the volume of what is left. Next, if R is the region just described, find
R xdV .

This time you need

7√ 7√
Z 3π/4 Z 2π Z R
ρ 2 sin (φ ) dρdθ dφ = 2πR3 + 3πR3
π/6 π/4 0 24 24
As to the integral, it equals
1 √ 4 √
Z 3π/4 Z 2π Z R 
(ρ sin (φ ) cos (θ )) ρ 2 sin (φ ) dρdθ dφ = − 2R 7π + 3 3 + 6
π/6 π/4 0 192
This is because, in terms of spherical coordinates, x = ρ sin (φ ) cos (θ ).
p
Example 20.3.7 Set up the integrals to findR the volume of the cone 0 ≤ z ≤ 4, z = x2 + y2 .
Next, if R is the region just described, find R zdV .

This is entirely the wrong coordinate system to use for this problem but it is a good
exercise. Here is a side view.

You need to figure out what ρ is as a function of φ which goes from 0 to π/4. You
should get
Z 2π Z π/4 Z 4 sec(φ )
64
ρ 2 sin (φ ) dρdφ dθ = π
0 0 0 3
R
As to R zdV, it equals
z
Z 2π Z π/4 Z 4 sec(φ ) z }| {
ρ cos (φ )ρ 2 sin (φ ) dρdφ dθ = 64π
0 0 0
20.3. CYLINDRICAL AND SPHERICAL COORDINATES 375

Example 20.3.8 Find the volume element for cylindrical coordinates.

In cylindrical coordinates,
   
x r cos θ
 y  =  r sin θ 
   

z z
Therefore, the Jacobian determinant is
 
cos θ −r sin θ 0
det  sin θ r cos θ 0  = r.
 

0 0 1
It follows the volume element in cylindrical coordinates is r dθ dr dz.
Example 20.3.9 In the cone of Example 20.3.7 set up the integrals for finding the volume
in cylindrical coordinates.
This is a better coordinate system for this example than spherical coordinates. This
time you should get
Z 2π Z 4 Z 4
64
rdzdrdθ = π
0 0 r 3
Example 20.3.10 This example uses spherical coordinates to verify an important conclu-
sion about gravitational force. Let the hollow sphere, H be defined by a2 < x2 +y2 +z2 < b2

and suppose this hollow sphere has constant density taken to equal 1. Now place a unit
 (0, 0, z0 ) where |z0 | ∈ [a,b] . Show that the force of gravity acting on this
mass at the point
R (z−z0 )
unit mass is αG H 3/2 dV k and then show that if |z0 | > b then the force
[x2 +y2 +(z−z0 )2 ]
of gravity acting on this point mass is the same as if the entire mass of the hollow sphere
were placed at the origin, while if |z0 | < a, the total force acting on the point mass from
gravity equals zero. Here G is the gravitation constant and α is the density. In particular,
this shows that the force a planet exerts on an object is as though the entire mass of the
planet were situated at its center1 .
Without loss of generality, assume z0 > 0. Let dV be a little chunk of material located
at the point (x, y, z) of H the hollow sphere. Then according to Newton’s law of gravity, the
force this small chunk of material exerts on the given point mass equals
xi + yj + (z − z0 ) k 1
  Gα dV =
|xi + yj + (z − z0 ) k| x2 + y2 + (z − z )2
0

1 This was shown by Newton in 1685 and allowed him to assert his law of gravitation applied to the planets as

though they were point masses. It was a major accomplishment.

376 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

1
(xi + yj + (z − z0 ) k)  3/2 Gα dV
2 2 2
x + y + (z − z0 )
Therefore, the total force is
1
Z
(xi + yj + (z − z0 ) k)  3/2 Gα dV.
H
x2 + y2 + (z − z0 )2

By the symmetry of the sphere, the i and j components will cancel out when the integral
is taken. This is because there is the same amount of stuff for negative x and y as there is
for positive x and y. Hence what remains is
(z − z0 )
Z
αGk h i3/2 dV
H
x2 + y2 + (z − z0 )2

as claimed. Now for the interesting part, the integral is evaluated. In spherical coordinates
this integral is.
(ρ cos φ − z0 ) ρ 2 sin φ
Z 2π Z b Z π

3/2 dφ dρ dθ . (20.1)
0 a 0 ρ 2 + z20 − 2ρz0 cos φ
Rewrite the inside integral and use integration by parts to obtain this inside integral equals
1 (2z0 ρ sin φ )
Z π
ρ 2 cos φ − ρz0


2z0
3/2 dφ =
0 ρ 2 + z20 − 2ρz0 cos φ

1  −ρ 2 − ρz0 ρ 2 − ρz0
−2 q + 2 q
2z0 ρ 2 + z20 + 2ρz0


ρ 2 + z20 − 2ρz0


sin
Z π
φ
− 2ρ 2 q
dφ .
 (20.2)
0 2 2
ρ + z0 − 2ρz0 cos φ
There are some cases to consider here.
First suppose z0 < a so the point is on the inside of the hollow sphere and it is always
the case that ρ > z0 . Then in this case, the two first terms reduce to

2ρ (ρ + z0 ) 2ρ (ρ − z0 ) 2ρ (ρ + z0 ) 2ρ (ρ − z0 )
q +q = + = 4ρ
(ρ + z0 ) ρ − z0
(ρ + z0 )2 (ρ − z0 )2

and so the expression in 20.2 equals

 
1  sin
Z π
φ
4ρ − 2ρ 2 q
dφ

2z0 0 2 2
ρ + z0 − 2ρz0 cos φ
 
1  1 π 2ρz0 sin φ
Z
= 4ρ − ρq
dφ

2z0 z0 0 2 2
ρ + z0 − 2ρz0 cos φ
20.4. EXERCISES 377
 
1 2ρ 2 2
1/2 π
= 4ρ − ρ + z0 − 2ρz0 cos φ |0
2z0 z0
 
1 2ρ
= 4ρ − [(ρ + z0 ) − (ρ − z0 )] = 0.
2z0 z0
Therefore, in this case the inner integral of 20.1 equals zero and so the original integral will
also be zero.
The other case is when z0 > b and so it is always the case that z0 > ρ. In this case the
first two terms of 20.2 are
2ρ (ρ + z0 ) 2ρ (ρ − z0 ) 2ρ (ρ + z0 ) 2ρ (ρ − z0 )
q +q = + = 0.
2 2 (ρ + z0 ) z0 − ρ
(ρ + z0 ) (ρ − z0 )

Therefore in this case, 20.2 equals

 
1  sin
Z π
φ
− 2ρ 2 q
dφ

2z0 0 ρ 2 + z20 − 2ρz0 cos φ
 
−ρ  π 2ρz0 sin φ
Z
=
dφ

2z20
q
0 2 2
ρ + z − 2ρz cos φ
0 0

which equals
−ρ  2 2
1/2 π 
ρ + z0 − 2ρz 0 cos φ |0
z20
−ρ 2ρ 2
= [(ρ + z0 ) − (z0 − ρ)] = − .
z20 z20

Thus the inner integral of 20.1 reduces to the above simple expression. Therefore, 20.1
equals
4 b3 − a3
Z 2π Z b  
2
− 2 ρ 2 dρ dθ = − π
0 a z0 3 z20
and so
(z − z0 )
Z
αGk
h i3/2 dV
H
x2 + y2 + (z − z0 )2
4 b3 − a3
 
total mass
= αGk − π = −kG .
3 z20 z20

20.4 Exercises
1. Find the volume of the region bounded by z = 0, x2 + (y − 2)2 = 4, and z =
p
x2 + y2 .
378 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

p
2. Find the volume of the region z ≥ 0, x2 + y2 ≤ 4, and z ≤ 4 − x 2 + y2 .
3. Find the volume of the region which is between the surfaces z = 5y2 + 9x2 and z =
9 − 4y2 .

4. Find the volume of the region which is between z = x2 + y2 and z = 5 − 4x. Hint:
You might want to change variables at some point.

 1ice cream in a sugar cone is described in spherical coordinates by ρ ∈ [0, 10] , φ ∈

5. The
0, 3 π , θ ∈ [0, 2π]. If the units are in centimeters, find the total volume in cubic
centimeters of this ice cream.
p
6. Find the volume between z = 3 − x2 − y2 and z = 2 (x2 + y2 ).
7. A ball of radius 3 is placed in a drill press and a hole of radius 2 is drilled out with the
center of the hole a diameter of the ball. What is the volume of the material which
remains?
8. Find the volume of the cone defined by z ∈ [0, 4] having angle π/2. Use spherical
coordinates.
p
9. A ball of radius 9 has density equal to x2 + y2 + z2 in rectangular coordinates. The
top of this ball is sliced off by a plane of the form z = 2. Write integrals for the mass
of what is left. In spherical coordinates and in cylindrical coordinates.
10. A ball of radius 4 has a cone taken out of the top which has an angle of π/2 and then
a cone taken out of the bottom which has an angle of π/3. Then a slice, θ ∈ [0, π/4]
is removed. What is the volume of what is left?
11. In Example 20.3.10 on Page 375 check out all the details by working the integrals to
be sure the steps are right.
12. What if the hollow sphere in Example 20.3.10 were in two dimensions and every-
thing, including Newton’s law still held? Would similar conclusions hold? Explain.
20.5. THE GENERAL PROCEDURE 379

13. Convert the following integrals into integrals involving cylindrical coordinates and
then evaluate them.
√
R 2 R 4−x2 R x
(a) −2 0 0 xydzdydx
√
R 1 R 1−y2 R x+y
(b) −1 √ 2 0 dzdxdy
− 1−y
√
R 1 R 1−x2 R 1
(c) 0 0 x dzdydx
√ √
R a R a2 −x2 R a2 −x2 −y2
(d) For a > 0, −a √ √ dzdydx
− a2 −x2 − a2 −x2 −y2
√ √
R 1 R 1−x2 R 4−x2 −y2
(e) −1 √ √ dzdydx
− 2
1−x − 2 2
4−x −y

14. Convert the following integrals into integrals involving spherical coordinates and
then evaluate them.
√ √
R a R a2 −x2 R a2 −x2 −y2
(a) −a √ √ dzdydx
− a2 −x2 − a2 −x2 −y2
√ √
R 1 R 1−x2 R 1−x2 −y2
(b) −1 0
√ dzdydx
− 1−x2 −y2
√ √
R √2 R 2−x2 R 4−x2 −y2
(c) √ √ √ dzdydx
− 2 − 2−x2 x2 +y2
√ √
R √3 R 3−x2 R 4−x2 −y2
(d) √ √ dzdydx
− 3 − 3−x2 1
√ √
R 1 R 1−x2 R 4−x2 −y2
(e) −1
√ √ dzdydx
− 2
1−x − 2 2
4−x −y

20.5 The General Procedure

As mentioned above, the fundamental concept of an integral is a sum of things of the form
f (x) dV where dV is an “infinitesimal” chunk of volume located at the point x. Up to
now, this infinitesimal chunk of volume has had the form of a box with sides dx1 , · · · , dxn
so dV = dx1 dx2 · · · dxn but its form is not important. It could just as well be an infinitesimal
parallelepiped for example. In what follows, this is what it will be.
First recall the definition of a parallelepiped.

Definition 20.5.1 Let u1 , · · · , u p be vectors in Rk . The parallelepiped determined by these

vectors will be denoted by P (u1 , · · · , u p ) and it is defined as
( )
p
P (u1 , · · · , u p ) ≡ ∑ s j u j : s j ∈ [0, 1] .
j=1

Now define the volume of this parallelepiped.

volume of P (u1 , · · · , u p ) ≡ (det (ui · u j ))1/2 .

380 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

The dot product is used to determine this volume of a parallelepiped spanned by the
given vectors and you should note that it is only the dot product that matters. Let

x = f1 (u1 , u2 , u3 ) , y = f2 (u1 , u2 , u3 ) , z = f3 (u1 , u2 , u3 ) (20.3)

where u ∈ U an open set in R3 and corresponding to such a u ∈ U there exists a unique point
(x, y, z) ∈ V as above. Suppose at the point u0 ∈ U, there is an infinitesimal box having sides
du1 , du2 , du3 . Then this little box would correspond to something in V . What? Consider
the mapping from U to V defined by
   
x f1 (u1 , u2 , u3 )
x =  y  =  f2 (u1 , u2 , u3 )  = f (u) (20.4)
   

z f3 (u1 , u2 , u3 )

which takes a point u in U and sends it to the point in V which is identified as (x, y, z)T ≡ x.
What happens to a point of the infinitesimal box? Such a point is of the form

(u01 + s1 du1 , u02 + s2 du2 , u03 + s3 du3 ) ,

where si ≥ 0 and ∑i si ≤ 1. Also, from the definition of the derivative,

f (u10 + s1 du1 , u20 + s2 du2 , u30 + s3 du3 ) − f (u01 , u02 , u03 ) =

   
s1 du1 s1 du1
Df (u10 , u20 , u30 )  s2 du2  + o  s2 du2 
   

s3 du3 s3 du3

where the last term may be taken equal to 0 because the vector (s1 du1 , s2 du2 , s3 du3 )T is
infinitesimal, meaning nothing precise, but conveying the idea that it is surpassingly small.
Therefore, a point of this infinitesimal box is sent to the vector
=Df (u10 ,u20 ,u30 )  
z
 }| { s1 du1
∂ x (u0 ) ∂ x (u0 ) ∂ x (u0 ) 
, ,  s2 du2  =

∂ u1 ∂ u2 ∂ u3
s3 du3

∂ x (u0 ) ∂ x (u0 ) ∂ x (u0 )

s1 du1 + s2 du2 + s3 du3 ,
∂ u1 ∂ u2 ∂ u3
a point of the infinitesimal parallelepiped determined by the vectors
 
∂ x (u10 , u20 , u30 ) ∂ x (u10 , u20 , u30 ) ∂ x (u10 , u20 , u30 )
du1 , du2 , du3 .
∂ u1 ∂ u2 ∂ u3

The situation is no different for general coordinate systems in any dimension. In gen-
eral, x = f (u) where u ∈ U, a subset of Rn and x is a point in V , a subset of n dimensional
space. Thus, letting the Cartesian coordinates of x be given by x = (x1 , · · · , xn )T , each xi
being a function of u, an infinitesimal box located at u0 corresponds tonan infinitesimal o
∂ x(u0 ) n
parallelepiped located at f (u0 ) which is determined by the n vectors ∂ ui dui i=1 .
20.5. THE GENERAL PROCEDURE 381

From Definition 20.5.1, the volume of this infinitesimal parallelepiped located at f (u0 ) is
given by
  1/2
∂ x (u0 ) ∂ x (u0 )
det dui · du j (20.5)
∂ ui ∂uj
in which there is no sum on the repeated index. Now in general, if there are n vectors in
Rn , {v 1 , · · · , v n } ,
det (v i · v j )1/2 = |det (v 1 , · · · , v n )| (20.6)
where this last matrix is the n × n matrix which has the ith column equal to v i . The reason
for this is that the matrix whose i jth entry is v i · v j is just the product of the two matrices,
 
v T1
 . 
 .  (v 1 , · · · , v n )
 . 
v Tn

where the first on the left is the matrix having the ith row equal to v Ti while the matrix on the
right is just the matrix having the ith column equal to v i . Therefore, since the determinant
of a matrix equals the determinant of its transpose,
  
v T1
 . 
 .  (v 1 , · · · , v n ) = det (v 1 , · · · , v n )2

det (v i · v j ) = det 
 .  
v Tn

and so taking square roots yields (20.6). Therefore, from the properties of determinants,
(20.5) equals
   
∂ x (u0 ) ∂ x (u0 ) ∂ x (u0 ) ∂ x (u0 )
det du1 , · · · , dun = det ,··· , du1 · · · dun
∂ u1 ∂ un ∂ u1 ∂ un

This is the infinitesimal chunk of volume corresponding to the point f (u0 ) in V .

Definition 20.5.2 Let x = f (u) be as described above. Then the symbol

∂ (x1 , · · · xn )
,
∂ (u1 , · · · , un )

called the Jacobian determinant, is defined by

 
∂ x (u0 ) ∂ x (u0 ) ∂ (x1 , · · · xn )
det ,··· , ≡ .
∂ u1 ∂ un ∂ (u1 , · · · , un )

Also, the symbol ∂∂(u(x1,···

,···xn )
,un ) du1 · · · dun is called the volume element or increment of vol-
1
ume, or increment of area.

This has given motivation for the following fundamental procedure often called the
change of variables formula which holds under fairly general conditions.
382 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

PROCEDURE 20.5.3 Suppose U is an open subset of Rn for n > 0 and suppose

f : U → f (U) is a C1 function which is one to one, x = f (u). 2 Then if h : f (U) → R,

∂ (x1 , · · · , xn )
Z Z
h (f (u)) dV = h (x) dV.
U ∂ (u1 , · · · , un ) f (U)

Example 20.5.4 Find the area of the region in R2 which is determined by the lines y =
2x, y = (1/2) x, x + y = 1, x + y = 3.

You might sketch this region. You will find it is an ugly quadrilateral. Let u = x+ y and
v = xy . The reason for this is that the given region corresponds to (u, v) ∈ [1, 3] × 12 , 2 , a


nice rectangle. Now we need to solve for x, y to obtain the Jacobian. A little computation
shows that
u uv
x= , y=
v+1 v+1
∂ (x,y)
Therefore, ∂ (u,v) is
 
1 u
− u
v+1 (v+1)2 =
det  v u .
v+1 (v+1)2
(v + 1)2

Therefore, the area of this quadrilateral is

Z 2 Z 3
u 4
dudv = .
2
1/2 1 (v + 1) 3

20.6 Exercises
1. Verify the three dimensional volume increment in spherical coordinates is

ρ 2 sin (φ ) dρdφ dθ .

2. Find the area of the bounded region R, determined by 5x + y = 1, 5x + y = 9, y = 2x,

and y = 5x.
3. Find the area of the bounded region R, determined by y + 2x = 6, y + 2x = 10, y = 3x,
and y = 4x.
4. A solid, R is determined by 3x + y = 2, 3x + y = 4, y = x, and y = 2x and the density
is ρ = x. Find the total mass of R.
5. A solid, R is determined by 4x +2y = 1, 4x + 2y = 9, y = x, and y = 6x and the density
is ρ = y. Find the total mass of R.
6. A solid, R is determined by 3x +y = 3, 3x +y = 10, y = 3x, and y = 5x and the density
is ρ = y−1 . Find the total mass of R.
2 This will cause non overlapping infinitesimal boxes in U to be mapped to non overlapping infinitesimal
parallelepipeds in V .
Also, in the context of the Riemann integral we should say more about the set U in any case the function
h. These conditions are mainly technical however, and since a mathematically respectable treatment will not be
attempted for this theorem in this part of the book, I think it best to give a memorable version of it which is
essentially correct in all examples of interest.
20.6. EXERCISES 383

7. Find a 2 × 2 matrix A which maps the equilateral triangle having vertices at

 √ 
(0, 0) , (1, 0) , and 1/2, 3/2

to the triangle having vertices at (0, 0) , (a, b), and (c, d) where (c, d) is not a multiple
of (a, b). Find the area of this last triangle by using the cross product. Next find the
area of this triangle using the√ change of variables formula and the fact that the area
of the equilateral triangle is 43 .

8. Find the volume of the region E, bounded by the ellipsoid, 41 x2 + y2 + z2 = 1.

9. Here are three vectors. (4, 1, 2)T , (5, 0, 2)T , and (3, 1, 3)T . These vectors determine a
parallelepiped, R, which is occupied by a solid having density ρ = x. Find the mass
of this solid.

10. Here are three vectors. (5, 1, 6)T , (6, 0, 6)T , and (4, 1, 7)T . These vectors determine a
parallelepiped, R, which is occupied by a solid having density ρ = y. Find the mass
of this solid.
11. Here are three vectors. (5, 2, 9)T , (6, 1, 9)T , and (4, 2, 10)T . These vectors determine
a parallelepiped, R, which is occupied by a solid having density ρ = y + x. Find the
mass of this solid.
12. Compute the volume of a sphere of radius R using cylindrical coordinates.
13. Fill in all details for the following argument that
1√
Z ∞
2
e−x dx = π.
0 2
2
Let I = 0∞ e−x dx. Then
R

1
Z ∞Z ∞ Z π/2 Z ∞
2 +y2 2
I2 = e−(x ) dx dy = re−r dr dθ = π
0 0 0 0 4
from which the result follows.
(x−µ)2
∞ √1
R −
14. Show that −∞ 2πσ
e 2σ 2 dx = 1. Here σ is a positive number called the standard
deviation and µ is a number called the mean.

√
15. Show using Problem 13 that Γ 21 = π. Recall Γ (α) ≡ 0∞ e−t t α−1 dt.
R

R 1 p−1
16. Let p, q > 0 and define B (p, q) = 0 x (1 − x)q−1 . Show that

Γ (p) Γ (q) = B (p, q) Γ (p + q) .

Hint: It is fairly routine if you start with the left side and proceed to change variables.
384 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

20.7 The Moment Of Inertia And Center Of Mass

The methods used to evaluate multiple integrals make possible the determination of centers
of mass and moments of inertia for solids. This leads to the following definition.

Definition 20.7.1 Let a solid occupy a region R such that its density is ρ (x) for x a point
in R and let L be a line. For x ∈ R, let l (x) be the distance from the point x to the line L.
The moment of inertia of the solid is defined as
Z
I= l (x)2 ρ (x) dV.
R

Letting (x,y,z) denote the Cartesian coordinates of the center of mass,

R R R
xρ (x) dV yρ (x) dV zρ (x) dV
x= RR ,y= RR , z = RR
R ρ (x) dV R ρ (x) dV R ρ (x) dV

where x, y, z are the Cartesian coordinates of the point at x.

The reason the moment of inertia is of interest has to do with the total kinetic energy
of a solid occupying the region R which is rotating about the line L. Suppose its angular
velocity is ω. Then the kinetic energy of an infinitesimal chunk of volume located at point
x is 12 ρ (x) (l (x) ω)2 dV . Then using an integral to add these up, it follows the total kinetic
energy is
1 1
Z
ρ (x) l (x)2 dV ω 2 = Iω 2
2 R 2
Thus in the consideration of a rotating body, the moment of inertia takes the place of mass
when angular velocity takes the place of speed.
As to the center of mass, its significance is that it gives the point at which the mass will
balance. See Volume 1 to see this explained with point masses. The only difference is that
here the sums need to be replaced with integrals.

Example 20.7.2 Let a solid occupy the three dimensional region R and suppose the density
is ρ. What is the moment of inertia of this solid about the z axis? What is the center of
mass?

Here the little masses would be of the form ρ (x) dV where x is a point
of R. Therefore,
the contribution of this mass to the moment of inertia would be x2 + y2 ρ (x) dV where
the Cartesian coordinates of the point x are (x, y, z). Then summing these up as an integral,
yields the following for the moment of inertia.
Z
x2 + y2 ρ (x) dV.


(20.7)
R

To find the center of mass, sum up rρ dV for the points in R and divide by the total
mass. In Cartesian coordinates, where r = (x, y, z), this means to sum up vectors of the
form (xρ dV, yρ dV, zρ dV ) and divide by the total mass. Thus the Cartesian coordinates of
the center of mass are
R R R  R
xρ dV R yρ dV R zρ dV rρ dV
RR , R , R ≡ RR .
R ρ dV R ρ dV R ρ dV R ρ dV

Here is a specific example.

20.8. EXERCISES 385

Example 20.7.3 Find the moment of inertia about the
z axis and center of mass of the solid
which occupies the region R defined by 9 − x2 + y2 ≥ z ≥ 0 if the density is ρ (x, y, z) =
p
x 2 + y2 .

p
This moment of inertia is R x2 + y2 x2 + y2 dV and the easiest way to find this
R

integral is to use cylindrical coordinates. Thus the answer is

Z 2π Z 3 Z 9−r2
8748
r3 r dz dr dθ = π.
0 0 0 35
To find the center of mass, note the x and y coordinates of the center of mass,
R R
xρ dV R yρ dV
RR , R
R ρ dV R ρ dV

both equal zero because the above shape is symmetric about the z axis and ρ is also sym-
metric in its values. Thus xρ dV will cancel with −xρ dV and a similar conclusion will
hold for the y coordinate. It only remains to find the z coordinate of the center of mass, z.
In polar coordinates, ρ = r and so,
R 2π R 3 R 9−r2
zr2 dz dr dθ
R
zρ dV 18
z= RR = R0 R0 R0 2 = .
2π 3 9−r 7
R ρ dV 0 0 0 r2 dz dr dθ

Thus the center of mass will be 0, 0, 18

7 .

20.8 Exercises
1. Let R denote the finite region bounded by z = 4 − x2 − y2 and the xy plane. Find zc ,
the z coordinate of the center of mass if the density σ is a constant.
2. Let R denote the finite region bounded by z = 4 − x2 − y2 and the xy plane. Find zc ,
the z coordinate of the center of mass if the density σ is equals σ (x, y, z) = z.

3. Find the mass and center of mass of the region between the surfaces z = −y2 + 8 and
z = 2x2 + y2 if the density equals σ = 1.
4. Find the mass and center of mass of the region between the surfaces z = −y2 + 8 and
z = 2x2 + y2 if the density equals σ (x, y, z) = x2 .
5. The two cylinders, x2 + y2 = 4 and y2 + z2 = 4 intersect in a region R. Find the mass
and center of mass if the density σ , is given by σ (x, y, z) = z2 .
6. The two cylinders, x2 + y2 = 4 and y2 + z2 = 4 intersect in a region R. Find the mass
and center of mass if the density σ , is given by σ (x, y, z) = 4 + z.
2
x2
7. Find the mass and center of mass of the set (x, y, z) such that 4 + y9 + z2 ≤ 1 if the
density is σ (x, y, z) = 4 + y + z.

8. Let R denote the finite region bounded by z = 9 − x2 − y2 and the xy plane. Find the
moment of inertia of this shape about the z axis given the density equals 1.
386 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES

9. Let R denote the finite region bounded by z = 9 − x2 − y2 and the xy plane. Find the
moment of inertia of this shape about the x axis given the density equals 1.
10. Let B be a solid ball of constant density and radius R. Find the moment of inertia
about a line through a diameter of the ball. You should get 52 R2 M where M is the
mass..
11. Let B be a solid ball of density σ = ρ where ρ is the distance to the center of the ball
which has radius R. Find the moment of inertia about a line through a diameter of
the ball. Write your answer in terms of the total mass and the radius as was done in
the constant density case.

12. Let C be a solid cylinder of constant density and radius R. Find the moment of inertia
about the axis of the cylinder
You should get 12 R2 M where M is the mass.
13. Let C be a solid cylinder of constant density and radius R and mass M and let B be a
solid ball of radius R and mass M. The cylinder and the ball are placed on the top of
an inclined plane and allowed to roll to the bottom. Which one will arrive first and
why?
14. A ball of radius 4 has a cone taken out of the top which has an angle of π/2 and then
a cone taken out of the bottom which has an angle of π/3. If the density is λ = ρ,
find the z component of the center of mass.

15. A ball of radius 4 has a cone taken out of the top which has an angle of π/2 and then
a cone taken out of the bottom which has an angle of π/3. If the density is λ = ρ,
find the moment of inertia about the z axis.
16. Suppose a solid of mass M occupying the region B has moment of inertia, Il about a
line, l which passes through the center of mass of M and let l1 be another line parallel
to l and at a distance of a from l. Then the parallel axis theorem states Il1 = Il + a2 M.
Prove the parallel axis theorem. Hint: Choose axes such that the z axis is l and l1
passes through the point (a, 0) in the xy plane.
17. ∗Using the parallel axis theorem find the moment of inertia of a solid ball of radius
R and mass M about an axis located at a distance of a from the center of the ball.
Your answer should be Ma2 + 25 MR2 .
18. Consider all axes in computing the moment of inertia of a solid. Will the smallest
possible moment of inertia always result from using an axis which goes through the
center of mass?

19. Find the moment of inertia of a solid thin rod of length l, mass M, and constant
density about an axis through the center of the rod perpendicular to the axis of the
1 2
rod. You should get 12 l M.
20. Using the parallel axis theorem, find the moment of inertia of a solid thin rod of
length l, mass M, and constant density about an axis through an end of the rod per-
pendicular to the axis of the rod. You should get 13 l 2 M.
20.8. EXERCISES 387

21. Let the angle between the z axis and the sides of a right circular cone be α. Also
assume the height of this cone is h. Find the z coordinate of the center of mass of this
cone in terms of α and h assuming the density is constant.
22. Let the angle between the z axis and the sides of a right circular cone be α. Also
assume the height of this cone is h. Assuming the density is σ = 1, find the moment
of inertia about the z axis in terms of α and h.
23. Let R denote the part of the solid ball, x2 + y2 + z2 ≤ R2 which lies in the first octant.
That is x, y, z ≥ 0. Find the coordinates of the center of mass if the density is constant.
Your answer for one of the coordinates for the center of mass should be (3/8) R.

24. Show that in general for L angular momentum,

dL
=Γ
dt
where Γ is the total torque,
Γ ≡ ∑ ri × F i
where F i is the force on the ith point mass.
388 CHAPTER 20. THE INTEGRAL IN OTHER COORDINATES
Chapter 21

The Integral on Two Dimensional

Surfaces In R3

A parametric surface is the image of a vector valued function of two variables. Earlier,
vector valued functions of one variable were considered in the study of space curves. Here
there are two independent variables. This is why the result could be expected to be a
surface. For example, you could have
   
r (s,t) = x y z = s + t cos (s) sin (s) ts

for (s,t) ∈ (0, 1) × (0, 1). Each value of (s,t) gives a point on this surface. The surface
is smooth if all the component functions are C1 and r s × rt (s,t) ̸= 0. This last condition
assures the existence of a well defined normal vector to the surface, namely r s × rt (s,t).
Recall from the material on space curves that rt , r s are both tangent to curves which lie in
this surface. If this cross product were 0, you would get points or creases in the surface.

21.1 The Two Dimensional Area In R3

Consider a function defined on a two dimensional surface. Imagine taking the value of
this function at a point, multiplying this value by the area of an infinitesimal chunk of area
located at this point and then adding these together. The only difference is that now you
need a two dimensional chunk of area rather than one dimensional.

Definition 21.1.1 Let u1 , u2 be vectors in R3 . The 2 dimensional parallelogram deter-

mined by these vectors will be denoted by P (u1 , u2 ) and it is defined as
( )
2
P (u1 , u2 ) ≡ ∑ s j u j : s j ∈ [0, 1] .
j=1

Then the area of this parallelogram is

area P (u1 , u2 ) ≡ |u1 × u2 | .

389
390 CHAPTER 21. THE INTEGRAL ON TWO DIMENSIONAL SURFACES IN R3

Suppose then that x = f (u) where u ∈ U, a subset of R2 and x is a point in V , a

subset of 3 dimensional space. Thus, letting the Cartesian coordinates of x be given by
x = (x1 , x2 , x3 )T , each xi being a function of u, an infinitesimal rectangle located at u0
corresponds to an infinitesimal parallelogram located at f (u0 ) which is determined by the
n o2
2 vectors ∂ f∂(uui 0 ) dui , each of which is tangent to the surface defined by x = f (u).
i=1
(No sum on the repeated index.)

f (dV )

du2
dV f u2 (u0 )du2
f u1 (u0 )du1
u0 du1

From Definition 21.1.1, the two dimensional volume of this infinitesimal parallelepiped
located at f (u0 ) is given by
∂ f (u0 ) ∂ f (u0 ) ∂ f (u0 ) ∂ f (u0 )
du1 × du2 = × du1 du2 (21.1)
∂ u1 ∂ u2 ∂ u1 ∂ u2
= f u1 × f u2 du1 du2 (21.2)
It might help to think of a lizard. The infinitesimal parallelepiped is like a very small
scale on a lizard. This is the essence of the idea. To define the area of the lizard sum up
areas of individual scales1 . If the scales are small enough, their sum would serve as a good
approximation to the area of the lizard.

This motivates the following fundamental procedure which I hope is extremely familiar
from the earlier material.
1 This beautiful lizard is a Sceloporus magister. It was photographed by C. Riley Nelson who is in the Zoology

department at Brigham Young University © 2004 in Kane Co. Utah. The lizard is a little less than one foot in
length.
21.1. THE TWO DIMENSIONAL AREA IN R3 391

PROCEDURE 21.1.2 Suppose U is a subset of R2 and suppose f : U → f (U) ⊆

R3 is a one
R
to one and C1 function. Then if h : f (U) → R, define the 2 dimensional surface
integral f (U) h (x) dA according to the following formula.
Z Z
h (x) dA ≡ h (f (u)) f u1 (u) × f u2 (u) du1 du2 .
f (U) U

Definition 21.1.3 It is customary to write f u1 (u) × f u2 (u) = ∂ ∂(x(u1 ,x,u2 ,x)3 ) because this new
1 2
notation generalizes to far more general situations for which the cross product is not de-
fined. For example, one can consider three dimensional surfaces in R8 .
Example 21.1.4 Consider the surface given by z = x2 for (x, y) ∈ [0, 1] × [0, 1] = U. Find
the surface area of this surface.

in the form x = f (u). This is

The first step in using the above is to write this surface
easy to do if you let u = (x, y). Then f (x, y) = x, y, x2 . If you like, let x = u1 and y = u2 .
What is ∂ (x∂1(x,y)
,x2 ,x3 )
= fx ×fy ?
 T  T
fx = 1 0 2x , fy = 0 1 0

and so  T  T p
fx ×fy = 1 0 2x × 0 1 0 = 1 + 4x2
√
and so the area element is 1 + 4x2 dx dy and the surface area is obtained by integrating
the function h (x) ≡ 1. Therefore, this area is
1√ √ 
Z 1Z 1p
1 
Z
dA = 1 + 4x2 dx dy = 5 − ln −2 + 5
f (U) 0 0 2 4
which can be obtained by using the trig. substitution, 2x = tan θ on the inside integral.
Note this all depends on being able to write the surface in the form, x = f (u) for
u ∈ U ⊆ R p . Surfaces obtained in this form are called parametrically defined surfaces.
These are best but sometimes you have some other description of a surface and in these
cases things can get pretty intractable. For example, you might have a level surface of the
form 3x2 +p4y4 + z6 = 10. In this case, you could solve for z using methods of algebra.
Thus
 z = 6 10 − 3x2 −4y4 and a parametric description of part of this level surface is
p
x, y, 6 10 − 3x2 − 4y4 for (x, y) ∈ U where U = (x, y) : 3x2 + 4y4 ≤ 10 . But what if


the level surface was something like

sin x2 + ln 7 + y2 sin x + sin (zx) ez = 11 sin (xyz)?

I really do not see how to use methods of algebra to solve for some variable in terms of the
others. It isn’t even clear to me whether there are any points (x, y, z) ∈ R3 satisfying this
particular relation. However, if a point satisfying this relation can be identified, the implicit
function theorem from advanced calculus can usually be used to assert one of the variables
is a function of the others, proving the existence of a parametrization at least locally. The
problem is, this theorem does not give the answer in terms of known functions so this is
not much help. Finding a parametric description of a surface is a hard problem and there
are no easy answers. This is a good example which illustrates the gulf between theory and
practice.
392 CHAPTER 21. THE INTEGRAL ON TWO DIMENSIONAL SURFACES IN R3

Example 21.1.5 Let U = [0, 12] × [0, 2π] and let f : U → R3 be given by

f (t, s) ≡ (2 cost + cos s, 2 sint + sin s,t)T

Find a double integral for the surface area. A graph of this surface is drawn below.

Then
 T  T
ft = −2 sint 2 cost 1 , fs = − sin s cos s 0

and  
− cos s
ft × fs =  − sin s
 

−2 sint cos s + 2 cost sin s
∂ (x1 ,x2 ,x3 )
and so ∂ (t,s) =
p
|f t × f s | = 5 − 4 sin2 t sin2 s − 8 sint sin s cost cos s − 4 cos2 t cos2 s.

Therefore, the desired integral giving the area is

Z 2π Z 12 p
5 − 4 sin2 t sin2 s − 8 sint sin s cost cos s − 4 cos2 t cos2 s dt ds.
0 0

If you really needed to find the number this equals, how would you go about finding it?
This is an interesting question and there is no single right answer. You should think about
this. Here is an example for which you will be able to find the integrals.

Example 21.1.6 Let U = [0, 2π] × [0, 2π] and for (t, s) ∈ U, let

f (t, s) = (2 cost + cost cos s, −2 sint − sint cos s, sin s)T .

Find the area of f (U). This is the surface of a donut shown below. The fancy name for
this shape is a torus.

1
0
-1
2
0 2
To find its area, 0
-2 -2
   
−2 sint − sint cos s − cost sin s
f t =  −2 cost − cost cos s  , f s =  sint sin s 
   

0 cos s
21.2. SURFACES OF THE FORM z = f (x, y) 393

and so |f t × f s | = (cos s + 2) so the area element is (cos s + 2) ds dt and the area is

Z 2π Z 2π
(cos s + 2) ds dt = 8π 2
0 0
Example 21.1.7 Let U = [0, 2π] × [0, 2π] and for (t, s) ∈ U, let
f (t, s) = (2 cost + cost cos s, −2 sint − sint cos s, sin s)T .
where h (x, y, z) = x2 .
R
Find f (U) h dV

Everything is the same as the preceding example except this time it is an integral of a
function. The area element is (cos s + 2) ds dt and so the integral called for is
2
Z 2π Z 2π z x on the surface

Z }| {
h dA = 2 cost + cost cos s (cos s + 2) ds dt = 22π 2
f (U) 0 0

21.2 Surfaces Of The Form z = f (x, y)

The special case where a surface is in the form z = f (x, y) , (x, y) ∈ U, yields a simple
formula which is used most often in this situation. You write the surface parametrically in
the form f (x, y) = (x, y, f (x, y))T such that (x, y) ∈ U. Then
   
1 0
fx =  0 , fy =  1 
   

fx fy
and q
fx ×fy = 1 + fy2 + fx2
so the area element is q
1 + fy2 + fx2 dx dy.
When the surface of interest comes in this simple form, people generally use this area
element directly rather than worrying about a parametrization and taking cross products.
In the case where the surface
q is of the form x = f (y, z) for (y, z) ∈ U, the area element is
obtained similarly and is 1 + fy2 + fz2 dy dz. I think you can guess what the area element
is if y = f (x, z).
There is also a simple geometric description of these area elements. Consider the sur-
face z = f (x, y). This is a level surface of the function of three variables z − f (x, y). In
fact the surface is simply z − f (x, y) = 0. Now consider the gradient of this function of
three variables. The gradient is perpendicular to the surface and the third component is
positive in this case. This gradient is (− fx , − fy , 1) and so the unit upward normal is just
√ 1 2 2 (− fx , − fy , 1). Now consider the following picture.
1+ fx + fy

θ
n k dA

θ dxdy
394 CHAPTER 21. THE INTEGRAL ON TWO DIMENSIONAL SURFACES IN R3

In this picture, you are looking at a chunk of area on the surface seen on edge and so it
seems reasonable to expect to have dx dy = dA cos θ . But it is easy to find cos θ from the
picture and the properties of the dot product.
n·k 1
cos θ = =q .
|n| |k| 1 + fx2 + fy2
q
Therefore, dA = 1 + fx2 + fy2 dx dy as claimed.
p
Example 21.2.1 Let z = x2 + y2 where (x, y) ∈ U for
U = (x, y) : x2 + y2 ≤ 4

R
Find S h dS where h (x, y, z) = x + z and S is the surface described as
 p 
x, y, x2 + y2

for (x, y) ∈ U.
√
Here you can see directly the angle in the above picture is π4 and so dA = 2 dx dy. If
q
you do not see this or if it is unclear, simply compute 1 + fx2 + fy2 and you will find it is
√
2. Therefore, using polar coordinates,
Z Z  p √
h dS = x + x2 + y2 2 dA
S U
√ Z 2π Z 2 16 √
= 2 (r cos θ + r) r dr dθ = 2π.
0 0 3
I have been purposely vague about precise mathematical conditions necessary for the
above procedures. This is because the precise mathematical conditions which are usually
cited are very technical and at the same time far too restrictive. The most general conditions
under which these sorts of procedures are valid include things like Lipschitz functions de-
fined on very general sets. These are functions satisfying a Lipschitz condition of the form
|f (x) − f (y)| ≤ K |x − y|. For example, y = |x| is Lipschitz continuous. This function
does not have a derivative at every point. So it is with Lipschitz functions. However, it
turns out these functions have derivatives at enough points to push everything through but
this requires considerations involving the Lebesgue integral.

21.3 MATLAB and Graphing Surfaces

I will illustrate with an example.
[s,t]=meshgrid(0:.02*pi:2*pi,0:.02*pi:pi);
[u,v]=meshgrid(0:.02*pi:2*pi,-1.4:.2:1.4);
hold on
surf(sin(t).*cos(s),sin(t).*sin(s),cos(t),’edgecolor’,’none’)
alpha .7
surf(.5*cos(u),.5*sin(u),v,’edgecolor’,’none’)
axis equal
21.4. PIECEWISE DEFINED SURFACES 395

This graphs two surfaces, a cylinder and a sphere. The .7 makes the sphere slightly
transparent. You can adjust this number to be anything between 0 and 1 depending on how
transparent you want it to be. If you just wanted to graph the sphere, you could forget
about the hold on and simply include the first of the two lines beginning with “surf”. You
should experiment with this. These are parametrically defined surfaces because this is
more general than a surface of the form z = f (x, y) and the integral is defined on these
more general kinds of surfaces. Click on the little curvy arrow on the top to allow rotating
the graph to see it from different angles.

21.4 Piecewise Defined Surfaces

As with curves, you might piece together surfaces. In this section is considered what hap-
pens on the place where the two surfaces intersect. First of all, we really don’t know how to
find the Riemann integral over arbitrary regions. We need to have the region be cylindrical
in either the u or the v direction. That is, u ∈ [a, b] and for each u, the variable v is between
T (u) and B (u). Alternatively, v ∈ [c, d] and for each v, the variable u is between L (v) and
R (v) where L (v) ≤ R (v). So what is meant by a piecewise smooth surface? Let

S ≡ S1 ∪ S2 ∪ · · · ∪ Sm

where Sk ≡ r k (Dk ) where Dk is one of the special regions just described and r k is one to
one and C1 on an open set Uk ⊇ Dk such that r u × r v ̸= 0. Then we assume that either
Sk ∩ S j = 0/ or their intersection is r k (lk ) = r j (l j ) where lk , l j are one of the four edges of
Dk and D j respectively. For example, say

Dk = {u ∈ [a, b] , v ∈ [B (u) , T (u)]}

and say lk is the top edge of Dk , {(u, T (u)) : u ∈ [a, b]}. Then from the definition, if f is
defined on S, and is 0 off Sk ∩ S j ,
Z Z b Z T (u)
f dS = f (u, v) |r ku × r kv | dvdu = 0
S a T (u)

Other situations are exactly similar. The point is, when you have a surface which is de-
fined piecewise as just described, you don’t need to bother with the curves of intersection
because the two dimensional iterated integral will be zero on these curves. The term for
this situation in the context of the Lebesgue integral is that the curve has measure zero. In
examples of interest, the situation is usually that surfaces intersect in sets of measure zero
and so as far as the integral is concerned, they are irrelevant.

21.5 Flux Integrals

These will be important in the next chapter. The idea is this. You have a surface S and a
field of unit normal vectors n on S. That is, for each
R
point of S there exists a unit normal.
There is also a vector field F and you want to find S F · ndS. There is really nothing new
here. You just need to compute the function F · n and then integrate it over the surface.
Here is an example.
396 CHAPTER 21. THE INTEGRAL ON TWO DIMENSIONAL SURFACES IN R3

Example 21.5.1 Let F (x, y, z) = (x, x + z, y) and let S be the hemisphere x2 +R y2 + z2 =

4, z ≥ 0. Let n be the unit normal to S which has nonnegative z component. Find S F · ndS.

First find the function

=n
z }| {
1 1 1 1
F · n ≡ (x, x + z, y) · (x, y, z) = x2 + (x + z) y + yz
2 2 2 2
This follows because the normal is of the form (2x, 2y, 2z) and then when you divide by its
length using the fact that x2 + y2 + z2 = 4, you obtain that n = (x, y, z) 12 as claimed. Next
it remains to choose a coordinate system for the surface and then to compute the integral.
A parametrization is

x = 2 sin φ cos θ , y = 2 sin φ sin θ , z = 2 cos φ

and the increment of surface area is then

   
−2 sin φ sin θ 2 cos φ cos θ
 2 sin φ cos θ  ×  2 cos φ sin θ  dθ dφ
   

0 −2 sin φ
 
−4 sin2 φ cos θ
=  −4 sin2 φ sin θ  dθ dφ = 4 sin φ dθ dφ
 

−4 sin φ cos φ

Therefore, since the hemisphere corresponds to θ ∈ [0, 2π] and φ ∈ [0, π/2], the integral to
work is
Z 2π Z π/2   
1 1
(2 sin φ cos θ )2 + (2 sin φ cos θ + 2 cos φ ) ·
0 0 2 2

1
(2 sin φ sin θ ) + (2 sin φ sin θ ) 2 cos φ 4 sin (φ ) dφ dθ
2

Doing the integration, this reduces to 16

3 π.
The important thing to notice is that there is no new mathematics here. That which is
new is the significance of a flux integral which will be discussed more in the next chapter.
In short, this integral often has the interpretation of a measure of how fast something is
crossing a surface.

21.6 Exercises
1. Find a parametrization for the intersection of the planes 4x + 2y + 4z = 3 and 6x −
2y = −1.
2. Find a parametrization for the intersection of the plane 3x + y + z = 1 and the circular
cylinder x2 + y2 = 1.
3. Find a parametrization for the intersection of the plane 3x + 2y + 4z = 4 and the
elliptic cylinder x2 + 4z2 = 16.
21.6. EXERCISES 397

4. Find a parametrization for the straight line joining (1, 3, 1) and (−2, 5, 3).
5. Find a parametrization for the intersection of the surfaces 4y + 3z = 3x2 + 2 and
3y + 2z = −x + 3.

6. Find the area of S if S is the part of the circular cylinder x2 + y2 = 4 which lies
between z = 0 and z = 2 + y.
7. Find the area of S if S is the part of the cone x2 + y2 = 16z2 between z = 0 and z = h.
8. Parametrizing the cylinder x2 + y2 = a2 by x = a cos v, y = a sin v, z = u, show that the
area element is dA = a du dv

9. Find the area enclosed by the limacon r = 2 + cos θ .

10. Find the surface area of the paraboloid z = h 1 − x2 − y2 between z = 0 and z = h.

Take a limit of this area as h decreases to 0.

R
11. Evaluate S (1 + x) dA where S is the part of the plane 4x + y + 3z = 12 which is in
the first octant.
12. Evaluate S (1 + x) dA where S is the part of the cylinder x2 + y2 = 9 between z = 0
R

and z = h.
where S is the hemisphere x2 + y2 + z2 = 4 between x = 0 and
R
13. Evaluate S (1 + x) dA
x = 2.

14. For (θ , α) ∈ [0, 2π] × [0, 2π] ,let

f (θ , α) ≡ (cos θ (4 + cos α) , − sin θ (4 + cos α) , sin α)T .

Find the area of f ([0, 2π] × [0, 2π]). Hint: Check whether f θ · f α = 0. This might
make the computations reasonable.
15. For (θ , α) ∈ [0, 2π] × [0, 2π], let

f (θ , α) ≡ (cos θ (3 + 2 cos α) , − sin θ (3 + 2 cos α) , 2 sin α)T , h (x) = cos α,

where
R
α is such that x = (cos θ (3 + 2 cos α) , − sin θ (3 + 2 cos α) , 2 sin α)T . Find
f ([0,2π]×[0,2π]) h dA. Hint: Check whether f θ · f α = 0. This might make the compu-
tations reasonable.
16. For (θ , α) ∈ [0, 2π] × [0, 2π], let

f (θ , α) ≡ (cos θ (4 + 3 cos α) , − sin θ (4 + 3 cos α) , 3 sin α)T , h (x) = cos2 θ ,

where
R
θ is such that x = (cos θ (4 + 3 cos α) , − sin θ (4 + 3 cos α) , 3 sin α)T . Find
f ([0,2π]×[0,2π]) h dA. Hint: Check whether f θ · f α = 0. This might make the compu-
tations reasonable.
17. In spherical coordinates, φ = c, ρ ∈ [0, R] determines a cone. Find the area of this
cone.
398 CHAPTER 21. THE INTEGRAL ON TWO DIMENSIONAL SURFACES IN R3

18. Let F = (x, y, z) and let S be the curved surface which comes from the intersection
of the plane zR = x with the paraboloid z = x2 + y2 . Find an iterated integral for the
flux integral S F · ndS where n is the field of unit normals which has negative z
component.

19. Let F = (x, 0, 0) and let S denote the surface which consists of the part of the sphere
x2 +y2 +z2 = 9 which lies between the planes z = 1 and z = 2. Find S F · ndS where
R

n is the unit normal to this surface which has positive z component.

20. In the situation of the above problem change the vector field to F = (0, 0, z) and do
the same problem.
21. Show that for a sphere of radius a parameterized with spherical coordinates so that

x = a sin φ cos θ , y = a sin φ sin θ , z = a cos φ

the increment of surface area is a2 sin φ dθ dφ . Use to show that the area of a sphere
of radius a is 4πa2 .
Chapter 22

Calculus Of Vector Fields

22.1 Divergence And Curl Of A Vector Field

Here the important concepts of divergence and curl are defined.

Definition 22.1.1 Let f : U → R p for U ⊆ R p denote a vector field. A scalar valued

function is called a scalar field. The function f is called a Ck vector field if the function f
is a Ck function. For a C1 vector field, as just described ∇ · f (x) ≡ div f (x) known as the
divergence, is defined as
p
∂ fi
∇ · f (x) ≡ div f (x) ≡ ∑ (x) .
i=1 ∂ xi

Using the repeated summation convention, this is often written as

fi,i (x) ≡ ∂i fi (x)

where the comma indicates a partial derivative is being taken with respect to the ith variable
and ∂i denotes differentiation with respect to the ith variable. In words, the divergence is
the sum of the ith derivative of the ith component function of f for all values of i. If p = 3,
the curl of the vector field yields another vector field and it is defined as follows.

(curl (f ) (x))i ≡ (∇ × f (x))i ≡ ε i jk ∂ j fk (x)

where here ∂ j means the partial derivative with respect to x j and the subscript of i in
(curl (f ) (x))i means the ith Cartesian component of the vector curl (f ) (x). Thus the curl
is evaluated by expanding the following determinant along the top row.

i j k
∂ ∂ ∂
∂x ∂y ∂z .
f1 (x, y, z) f2 (x, y, z) f3 (x, y, z)
Note the similarity with the cross product. Sometimes the curl is called rot. (Short for
rotation not decay.) Also
∇2 f ≡ ∇ · (∇ f ) .

399
400 CHAPTER 22. CALCULUS OF VECTOR FIELDS

This last symbol is important enough that it is given a name, the [Link] is also de-
noted by ∆. Thus ∇2 f = ∆ f . In addition for f a vector field, the symbol f · ∇ is defined as
a “differential operator” in the following way.
∂ g (x) ∂ g (x) ∂ g (x)
f · ∇ (g) ≡ f1 (x) + f2 (x) + · · · + f p (x) .
∂ x1 ∂ x2 ∂ xp

Thus f · ∇ takes vector fields and makes them into new vector fields.

This definition is in terms of a given coordinate system but later coordinate free defini-
tions of the curl and div are presented. For now, everything is defined in terms of a given
Cartesian coordinate system. The divergence and curl have profound physical significance
and this will be discussed later. For now it is important to understand their definition in
terms of coordinates. Be sure you understand that for f a vector field, div f is a scalar field
meaning it is a scalar valued function of three variables. For a scalar field f , ∇ f is a vector
field described earlier. For f a vector field having values in R3 , curl f is another vector
field.

Example 22.1.2 Let f (x) = xyi + (z − y) j + (sin (x) + z) k. Find div f and curl f .

First the divergence of f is

∂ (xy) ∂ (z − y) ∂ (sin (x) + z)
+ + = y + (−1) + 1 = y.
∂x ∂y ∂z
Now curl f is obtained by evaluating

i j k
∂ ∂ ∂
∂x ∂y ∂z =
xy z − y sin (x) + z
   
∂ ∂ ∂ ∂
i (sin (x) + z) − (z − y) − j (sin (x) + z) − (xy) +
∂y ∂z ∂x ∂z
 
∂ ∂
k (z − y) − (xy) = −i − cos (x) j − xk.
∂x ∂y

22.1.1 Vector Identities

There are many interesting identities which relate the gradient, divergence and curl.

Theorem 22.1.3 Assuming f, g are a C2 vector fields whenever necessary, the following
identities are valid.

1. ∇ · (∇ × f ) = 0
2. ∇ × ∇φ = 0
3. ∇ × (∇ × f ) = ∇ (∇ · f ) − ∇2 f where ∇2 f is a vector field whose ith component is
∇2 fi .
4. ∇ · (f × g) = g· (∇ × f ) − f · (∇ × g)
22.1. DIVERGENCE AND CURL OF A VECTOR FIELD 401

5. ∇ × (f × g) = (∇ · g) f − (∇ · f ) g+ (g·∇) f − (f ·∇) g

Proof: These are all easy to establish if you use the repeated index summation conven-
tion and the reduction identities.


∇ · (∇ × f ) = ∂i (∇ × f )i = ∂i ε i jk ∂ j fk = ε i jk ∂i (∂ j fk )
= ε jik ∂ j (∂i fk ) = −ε i jk ∂ j (∂i fk ) = −ε i jk ∂i (∂ j fk )
= −∇ · (∇ × f ) .

This establishes the first formula. The second formula is done similarly. Now consider the
third.

(∇ × (∇ × f ))i = ε i jk ∂ j (∇ × f )k = ε i jk ∂ j (ε krs ∂r fs )
=ε i jk
z}|{
= ε ki j ε krs ∂ j (∂r fs ) = (δ ir δ js − δ is δ jr ) ∂ j (∂r fs )
= ∂ j (∂i f j ) − ∂ j (∂ j fi ) = ∂i (∂ j f j ) − ∂ j (∂ j fi )
 
= ∇ (∇ · f ) − ∇2 f
i

This establishes the third identity.

Consider the fourth identity.

∇ · (f × g) = ∂i (f × g)i = ∂i ε i jk f j gk
= ε i jk (∂i f j ) gk + ε i jk f j (∂i gk )



= ε ki j ∂i f j gk − ε jik ∂i gk fk
= ∇ × f · g − ∇ × g · f.

This proves the fourth identity.

Consider the fifth.

(∇ × (f × g))i = ε i jk ∂ j (f × g)k = ε i jk ∂ j ε krs fr gs

= ε ki j ε krs ∂ j ( fr gs ) = (δ ir δ js − δ is δ jr ) ∂ j ( fr gs )
= ∂ j ( fi g j ) − ∂ j ( f j gi )
= (∂ j g j ) fi + g j ∂ j fi − (∂ j f j ) gi − f j (∂ j gi )
= ((∇ · g) f + (g · ∇) (f ) − (∇ · f ) g − (f · ∇) (g))i

and this establishes the fifth identity. ■

22.1.2 Vector Potentials

One of the above identities says ∇ · (∇ × f ) = 0. Suppose now ∇ · g = 0. Does it follow that
there exists f such that g = ∇ × f ? It turns out that this is usually the case and when such
an f exists, it is called a vector potential. Here is one way to do it, assuming everything
is defined so the following formulas make sense.
Z z Z z Z x T
f (x, y, z) = g2 (x, y,t) dt, − g1 (x, y,t) dt + g3 (t, y, 0) dt, 0 . (22.1)
0 0 0
402 CHAPTER 22. CALCULUS OF VECTOR FIELDS

In verifying this you need to use the following manipulation which will generally hold
under reasonable conditions but which has not been carefully shown yet.
Z b Z b
∂ ∂h
h (x,t) dt = (x,t) dt. (22.2)
∂x a a ∂x
The above formula seems plausible because the integral is a sort of a sum and the deriva-
tive of a sum is the sum of the derivatives. However, this sort of sloppy reasoning will
get you into all sorts of trouble. The formula involves the interchange of two limit opera-
tions, the integral and the limit of a difference quotient. Such an interchange can only be
accomplished through a theorem. The following gives the necessary result.
∂h
Lemma 22.1.4 Suppose h and ∂x are continuous on the rectangle R = [c, d] × [a, b]. Then
(22.2) holds.

Proof: Let ∆x be such that x, x + ∆x are both in [c, d]. By Theorem 13.5.5 on Page 243
there exists δ > 0 such that if |(x,t) − (x1 ,t1 )| < δ , then

∂h ∂h ε
(x,t) − (x1 ,t1 ) < .
∂x ∂x b−a

Let |∆x| < δ . Then

Z b Z b
h (x + ∆x,t) − h (x,t) ∂h
dt − (x,t) dt
a ∆x a ∂x
Z b h (x + ∆x,t) − h (x,t) ∂ h
≤ − (x,t) dt
a ∆x ∂x
Z b Z b
∂ h (x + θ t ∆x) ∂ h ε
= − (x,t) dt < dt = ε.
a ∂x ∂x a b−a

Here θ t is a number between 0 and 1 and going from the second to the third line is an
application of the mean value theorem. ■
The second formula of Theorem 22.1.3 states ∇ × ∇φ = 0. This suggests the following
question: Suppose ∇×f = 0, does it follow there exists φ , a scalar field such that ∇φ = f ?
The answer to this is often yes and a theorem will be given and proved after the presentation
of Stoke’s theorem. This scalar field φ , is called a scalar potential for f .

22.1.3 The Weak Maximum Principle

There is also a fundamental result having great significance which involves ∇2 called the
maximum principle. This principle says that if ∇2 u ≥ 0 on a bounded open set U, then u
achieves its maximum value on the boundary of U.

Theorem 22.1.5 Let U be a bounded open set in Rn and suppose

u ∈ C2 (U) ∩C U

such that ∇2 u ≥ 0 in U. Then letting ∂U = U \U, it follows that


max u (x) : x ∈ U = max {u (x) : x ∈ ∂U} .
22.2. EXERCISES 403

Proof: If this is not so, there exists x0 ∈ U such that

u (x0 ) > max {u (x) : x ∈ ∂U} ≡ M

Since U is bounded, there exists ε > 0 such that

n o
u (x0 ) > max u (x) + ε |x|2 : x ∈ ∂U .

Therefore, u (x) + ε |x|2 also has its maximum in U because for ε small enough,
n o
u (x0 ) + ε |x0 |2 > u (x0 ) > max u (x) + ε |x|2 : x ∈ ∂U

for all x ∈ ∂U.

Now let x1 be the point in U at which u (x) + ε |x|2 achieves its maximum. Asan
exercise you should show that ∇2 ( f + g) = ∇2 f +∇2 g and therefore, ∇2 u (x) + ε |x|2 =
∇2 u (x) + 2nε. (Why?) Therefore,

0 ≥ ∇2 u (x1 ) + 2nε ≥ 2nε,

a contradiction. ■

22.2 Exercises
1. Find div f and curl f where f is

T
(a) xyz, x2 + ln (xy) , sin x2 + z
(b) (sin x, sin y, sin z)T
(c) ( f (x) , g (y) , h (z))T
(d) (x − 2, y − 3, z − 6)T

T
(e) y2 , 2xy, cos z
(f) ( f (y, z) , g (x, z) , h (y, z))T

2. Prove formula (2) of Theorem 22.1.3.

3. Show that if u and v are C2 functions, then curl (u∇v) = ∇u × ∇v.
4. Simplify the expression f × (∇ × g) + g× (∇ × f ) + (f · ∇) g+ (g · ∇) f .
5. Simplify ∇ × (v × r) where r = (x, y, z)T = xi + yj + zk and v is a constant vector.
6. Discover a formula which simplifies ∇ · (v∇u).
7. Verify that ∇ · (u∇v) − ∇ · (v∇u) = u∇2 v − v∇2 u.
8. Verify that ∇2 (uv) = v∇2 u + 2 (∇u · ∇v) + u∇2 v.
9. Functions u, which satisfy ∇2 u = 0 are called harmonic functions. Show that the
following functions are harmonic where ever they are defined.
404 CHAPTER 22. CALCULUS OF VECTOR FIELDS

(a) 2xy
(b) x2 − y2
(c) sin x cosh y
(d) ln x2 + y2


p
(e) 1/ x2 + y2 + z2

10. Verify the formula given in (22.1) is a vector potential for g assuming that div g = 0.
11. Show that if ∇2 uk = 0 for each k = 1, 2, · · · , m, and ck is a constant, then
!
m
∇2 ∑ ck uk =0
k=1

also.
 
12. In Theorem 22.1.5, why is ∇2 ε |x|2 = 2nε?

13. Using Theorem 22.1.5, prove the following: Let f ∈ C (∂U) ( f is continuous on
∂U.) where U
is a bounded open set. Then there exists at most one solution u ∈
C2 (U) ∩ C U and ∇2 u = 0 in U with u = f on ∂U. Hint: Suppose there are two
solutions ui , i = 1, 2 and let w = u1 − u2 . Then use the maximum principle.
14. Suppose B is a vector field and ∇ × A = B. Thus A is a vector potential for B.
Show that A+∇φ is also a vector potential for B. Here φ is just a C2 scalar field.
Thus the vector potential is not unique.

22.3 The Divergence Theorem

The divergence theorem relates an integral over a set to one on the boundary of the set. It
is also called Gauss’s theorem.

Definition 22.3.1 A subset V of R3 is called cylindrical in the x direction if it is of the form

V = {(x, y, z) : φ (y, z) ≤ x ≤ ψ (y, z) for (y, z) ∈ D}

where D is a subset of the yz plane. V is cylindrical in the z direction if

V = {(x, y, z) : φ (x, y) ≤ z ≤ ψ (x, y) for (x, y) ∈ D}

where D is a subset of the xy plane, and V is cylindrical in the y direction if

V = {(x, y, z) : φ (x, z) ≤ y ≤ ψ (x, z) for (x, z) ∈ D}

where D is a subset of the xz plane. If V is cylindrical in the z direction, denote by ∂V the

boundary of V defined to be the points of the form (x, y, φ (x, y)) , (x, y, ψ (x, y)) for (x, y) ∈
D, along with points of the form (x, y, z) where (x, y) ∈ ∂ D and φ (x, y) ≤ z ≤ ψ (x, y). Points
on ∂ D are defined to be those for which every open ball contains points which are in D as
well as points which are not in D. A similar definition holds for ∂V in the case that V is
cylindrical in one of the other directions.
22.3. THE DIVERGENCE THEOREM 405

The following picture illustrates the above definition in the case of V cylindrical in the
z direction. Also labeled are the z components of the respective outer unit normals on the
sides and top and bottom.

1
nz = (1+ψ 2x +ψ 2y )1/2

z = ψ(x, y)
nz = 0

z = φ (x, y)
y
−1
x nz = (1+φ 2x +φ 2y )1/2

Of course, many three dimensional sets are cylindrical in each of the coordinate direc-
tions. For example, a ball or a rectangle or a tetrahedron are all cylindrical in each direction.
The following lemma allows the exchange of the volume integral of a partial derivative for
an area integral in which the derivative is replaced with multiplication by an appropriate
component of the unit exterior normal.

Lemma 22.3.2 Suppose V is cylindrical in the z direction and that φ and ψ are the func-
tions in the above definition. Assume φ and ψ are C1 functions and suppose F is a C1
function defined on V . Also, let n = (nx , ny , nz ) be the unit exterior normal to ∂V . Then

∂F
Z Z
(x, y, z) dV = Fnz dA.
V ∂z ∂V

Proof: From the fundamental theorem of calculus,

∂F ∂F
Z Z Z ψ(x,y)
(x, y, z) dV = (x, y, z) dz dx dy (22.3)
V ∂z D φ (x,y) ∂z
Z
= [F (x, y, ψ (x, y)) − F (x, y, φ (x, y))] dx dy
D

Now the unit exterior normal on the top of V , the surface (x, y, ψ (x, y)) is
1

q −ψ x , −ψ y , 1 .
ψ 2x + ψ 2y + 1

This follows from the observation that the top surface is the level surface z − ψ (x, y) = 0
and so the gradient of this function of three variables is perpendicular to the level surface.
It points in the correct direction because the z component is positive. Therefore, on the top
surface
1
nz = q
ψ 2x + ψ 2y + 1
406 CHAPTER 22. CALCULUS OF VECTOR FIELDS

Similarly, the unit normal to the surface on the bottom is

1

q φ x , φ y , −1
φ 2x + φ 2y + 1

and so on the bottom surface,

−1
nz = q
φ 2x + φ 2y + 1

Note that here the z component is negative because since it is the outer normal it must point
y) ∈ ∂ D and z ∈ [φ (x, y) , ψ (x, y)], nz = 0.
down. On the lateral surface, the one where (x, q
The area element on the top surface is dA = ψ 2x + ψ 2y + 1 dx dy while the area element
q
on the bottom surface is φ 2x + φ 2y + 1 dx dy. Therefore, the last expression in (22.3) is of
the form,
nz dA
z }| {z }| {
1
Z q
F (x, y, ψ (x, y)) q ψ 2x + ψ 2y + 1 dx dy+
D ψ 2x + ψ 2y + 1
nz
z }| {z dA
}| {
−1
Z q
F (x, y, φ (x, y))  q  φ 2x + φ 2y + 1 dx dy
D φ 2x + φ 2y + 1
Z
+ Fnz dA,
Lateral surface
theR last term equaling zero because on the lateral surface, nz = 0. Therefore, this reduces
to ∂V Fnz dA as claimed. ■
The following corollary is entirely similar to the above.

Corollary 22.3.3 If V is cylindrical in the y direction, then

∂F
Z Z
dV = Fny dA
V ∂y ∂V

and if V is cylindrical in the x direction, then

∂F
Z Z
dV = Fnx dA
V ∂x ∂V

With this corollary, here is a proof of the divergence theorem.

Theorem 22.3.4 Let V be cylindrical in each of the coordinate directions and let F be a
C1 vector field defined on V . Then
Z Z
∇ · F dV = F · n dA.
V ∂V
22.3. THE DIVERGENCE THEOREM 407

Proof: From the above lemma and corollary,

∂ F1 ∂ F2 ∂ F3
Z Z
∇ · F dV = + + dV
V V ∂x ∂y ∂y
Z
= (F1 nx + F2 ny + F3 nz ) dA
Z∂V
= F · n dA. ■
∂V

The divergence theorem holds for much more general regions than this. Suppose for
example you have a complicated region which is the union of finitely many disjoint regions
of the sort just described which are cylindrical in each of the coordinate directions. Then
the volume integral over the union of these would equal the sum of the integrals over the
disjoint regions. If the boundaries of two of these regions intersect, then the area integrals
will cancel out on the intersection because the unit exterior normals will point in opposite
directions. Therefore, the sum of the integrals over the boundaries of these disjoint regions
will reduce to an integral over the boundary of the union of these. Hence the divergence
theorem will continue to hold. For example, consider the following picture. If the diver-
gence theorem holds for each Vi in the following picture, then it holds for the union of these
two.

V1 V2

General formulations of the divergence theorem involve Hausdorff measures and the
Lebesgue integral, a better integral than the old fashioned Riemannn integral which has
been obsolete now for almost 100 years. When all is said and done, one finds that the
conclusion of the divergence theorem is usually true and the theorem can be used with
confidence.

Example 22.3.5 Let V = [0, 1] × [0, 1] × [0, 1]. That is, V is the cube in the first octant
having the lower left corner at (0, 0, 0) and the sides of length 1. Let F (x, y, z) = xi + yj +
zk. Find the flux integral in which n is the unit exterior normal.
Z
F · ndS
∂V

You can certainly inflict much suffering on yourself by breaking the surface up into 6
pieces corresponding to the 6 sides of the cube, finding a parametrization for each face and
adding up the appropriate flux integrals. For example, n = k on the top face and n = −k
on the bottom face. On the top face, a parametrization is (x, y, 1) : (x, y) ∈ [0, 1] × [0, 1]. The
area element is just dxdy. It is not really all that hard to do it this way but it is much easier
to use the divergence theorem. The above integral equals
Z Z
div (F ) dV = 3dV = 3.
V V
408 CHAPTER 22. CALCULUS OF VECTOR FIELDS

Example 22.3.6 This time, let V be the unit ball, (x, y, z) : x2 + y2 + z2 ≤ 1 and let F (x, y, z) =


x2 i + y j+ (z − 1) k. Find Z
F · ndS.
∂V

As in the above you could do this by brute force. A parametrization of the ∂V is

obtained as
x = sin φ cos θ , y = sin φ sin θ , z = cos φ
where (φ , θ ) ∈ (0, π) × (0, 2π]. Now this does not include all the ball but it includes all but
the point at the top and at the bottom. As far as the flux integral is concerned these points
contribute nothing to the integral so you can neglect them. Then you can grind away and
get the flux integral which is desired. However, it is so much easier to use the divergence
theorem! Using spherical coordinates,
Z Z Z
F · ndS = div (F ) dV = (2x + 1 + 1) dV
∂V V V
Z π Z 2π Z 1
8
= (2 + 2ρ sin (φ ) cos θ ) ρ 2 sin (φ ) dρdθ dφ = π
0 0 0 3

Example 22.3.7 Suppose V is an open set in R3 for which the divergence theorem holds.
Let F (x, y, z) = xi + yj + zk. Then show that
Z
F · ndS = 3 × volume(V ).
∂V

This follows from the divergence theorem.

Z Z Z
F · ndS = div (F ) dV = 3 dV = 3 × volume(V ).
∂V V V

The message of the divergence theorem is the relation between the volume integral and
an area integral. This is the exciting thing about this marvelous theorem. It is not its utility
as a method for evaluations of boring problems. This will be shown in the examples of its
use which follow.

22.3.1 Coordinate Free Concept Of Divergence

The divergence theorem also makes possible a coordinate free definition of the divergence.

Theorem 22.3.8 Let B (x, δ ) be the ball centered at x having radius δ and let F be a C1
vector field. Then letting v (B (x, δ )) denote the volume of B (x, δ ) given by
Z
dV,
B(x,δ )

it follows
1
Z
div F (x) = lim F · n dA. (22.4)
δ →0+ v (B (x, δ )) ∂ B(x,δ )
22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 409

Proof: The divergence theorem holds for balls because they are cylindrical in every
direction. Therefore,
1 1
Z Z
F · n dA = div F (y) dV.
v (B (x, δ )) ∂ B(x,δ ) v (B (x, δ )) B(x,δ )

Therefore, since div F (x) is a constant,

1
Z
div F (x) − F · n dA
v (B (x, δ )) ∂ B(x,δ )

1
Z
= div F (x) − div F (y) dV
v (B (x, δ )) B(x,δ )

1
Z
= (div F (x) − div F (y)) dV
v (B (x, δ )) B(x,δ )

1
Z
≤ |div F (x) − div F (y)| dV
v (B (x, δ )) B(x,δ )
1
Z
ε
≤ dV < ε
v (B (x, δ )) B(x,δ ) 2
whenever ε is small enough, due to the continuity of div F . Since ε is arbitrary, this shows
(22.4). ■
How is this definition independent of coordinates? It only involves geometrical notions
of volume and dot product. This is why. Imagine rotating the coordinate axes, keeping
all distances the same and expressing everything in terms of the new coordinates. The
divergence would still have the same value because of this theorem.

22.4 Some Applications Of The Divergence Theorem

22.4.1 Hydrostatic Pressure
Imagine a fluid which does not move which is acted on by an acceleration g. Of course the
acceleration is usually the acceleration of gravity. Also let the density of the fluid be ρ, a
function of position. What can be said about the pressure p in the fluid? Let B (x, ε) be a
small ball centered at the point x. Then the force the fluid exerts on this ball would equal
Z
− pn dA.
∂ B(x,ε)

Here n is the unit exterior normal at a small piece of ∂ B (x, ε) having area dA. By the
divergence theorem, (see Problem 1 on Page 426) this integral equals
Z
− ∇p dV.
B(x,ε)

Also the force acting on this small ball of fluid is

Z
ρg dV.
B(x,ε)
410 CHAPTER 22. CALCULUS OF VECTOR FIELDS

Since it is given that the fluid does not move, the sum of these forces must equal zero. Thus
Z Z
ρg dV = ∇p dV.
B(x,ε) B(x,ε)

Since this must hold for any ball in the fluid of any radius, it must be that

∇p = ρg. (22.5)

It turns out that the pressure in a lake at depth z is equal to 62.5z. This is easy to see
from (22.5). In this case, g = gk where g = 32 feet/sec2 . The weight of a cubic foot of
water is 62.5 pounds. Therefore, the mass in slugs of this water is 62.5/32. Since it is a
cubic foot, this is also the density of the water in slugs per cubic foot. Also, it is normally
assumed that water is incompressible1 . Therefore, this is the mass of water at any depth.
Therefore,
∂p ∂p ∂p 62.5
i+ j+ k= × 32k.
∂x ∂y ∂z 32
and so p does not depend on x and y and is only a function of z. It follows p (0) = 0, and
p′ (z) = 62.5. Therefore, p (x, y, z) = 62.5z. This establishes the claim. This is interesting
but (22.5) is more interesting because it does not require ρ to be constant.

22.4.2 Archimedes Law Of Buoyancy

Archimedes principle states that when a solid body is immersed in a fluid, the net force act-
ing on the body by the fluid is directly up and equals the total weight of the fluid displaced.
Denote the set of points in three dimensions occupied by the body as V . Then for dA
an increment of area on the surface of this body, the force acting on this increment of area
would equal −p dAn where n is the exterior unit normal. Therefore, since the fluid does
not move, Z Z Z
−pn dA = −∇p dV = ρg dV k
∂V V V
Which equals the total weight of the displaced fluid and you note the force is directed
upward as claimed. Here ρ is the density and (22.5) is being used. There is an interesting
point in the above explanation. Why does the second equation hold? Imagine that V were
filled with fluid. Then the equation follows from (22.5) because in this equation g = −gk.

22.4.3 Equations Of Heat And Diffusion

Let x be a point in three dimensional space and let (x1 , x2 , x3 ) be Cartesian coordinates of
this point. Let there be a three dimensional body having density ρ = ρ (x,t).
The heat flux J, in the body is defined as a vector which has the following property.
Z
Rate at which heat crosses S = J · n dA
S

where n is the unit normal in the desired direction. Thus if V is a three dimensional body,
Z
Rate at which heat leaves V = J · n dA
∂V
1 There is no such thing as an incompressible fluid but this doesn’t stop people from making this assumption.
22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 411

where n is the unit exterior normal.

Fourier’s law of heat conduction states that the heat flux J satisfies J = −k∇ (u) where
u is the temperature and k = k (u, x,t) is called the coefficient of thermal conductivity. This
changes depending on the material. It also can be shown by experiment to change with
temperature. This equation for the heat flux states that the heat flows from hot places
toward colder places in the direction of greatest rate of decrease in temperature. Let c (x,t)
denote the specific heat of Rthe material in the body. This means the amount of heat within
V is given by the formula V ρ (x,t) c (x,t) u (x,t) dV . Suppose also there are sources for
the heat within the material given by f (x,u,t). If f is positive, the heat is increasing
while if f is negative the heat is decreasing. For example such sources could result from a
chemical reaction taking place. Then the divergence theorem can be used to verify the
following equation for u. Such an equation is called a reaction diffusion equation.

∂
(ρ (x,t) c (x,t) u (x,t)) = ∇ · (k (u, x,t) ∇u (x,t)) + f (x, u,t) . (22.6)
∂t
Take an arbitrary V for which the divergence theorem holds. Then the time rate of
change of the heat in V is

d ∂ (ρ (x,t) c (x,t) u (x,t))

Z Z
ρ (x,t) c (x,t) u (x,t) dV = dV
dt V V ∂t
where, as in the preceding example, this is a physical derivation so the consideration of
hard mathematics is not necessary. Therefore, from the Fourier law of heat conduction,
d R
dt V ρ (x,t) c (x,t) u (x,t) dV =

rate at which heat enters

zZ }| {
∂ (ρ (x,t) c (x,t) u (x,t))
Z Z
dV = −J · n dA + f (x, u,t) dV
V ∂t ∂V V

Z Z Z
= k∇ (u) · n dA + f (x, u,t) dV = (∇ · (k∇ (u)) + f ) dV.
∂V V V
Since this holds for every sample volume V it must be the case that the above reaction
diffusion equation (22.6) holds. Note that more interesting equations can be obtained by
letting more of the quantities in the equation depend on temperature. However, the above
is a fairly hard equation and people usually assume the coefficient of thermal conductivity
depends only on x and that the reaction term f depends only on x and t and that ρ and c
are constant. Then it reduces to the much easier equation

∂ 1
u (x,t) = ∇ · (k (x) ∇u (x,t)) + f (x,t) . (22.7)
∂t ρc

This is often referred to as the heat equation. Sometimes there are modifications of this
in which k is not just a scalar but a matrix to account for different heat flow properties
in different directions. However, they are not much harder than the above. The major
mathematical difficulties result from allowing k to depend on temperature.
It is known that the heat equation is not correct even if the thermal conductivity did not
depend on u because it implies infinite speed of propagation of heat. However, this does
not prevent people from using it.
412 CHAPTER 22. CALCULUS OF VECTOR FIELDS

22.4.4 Balance Of Mass

Let y be a point in three dimensional space and let (y1 , y2 , y3 ) be Cartesian coordinates of
this point. Let V be a region in three dimensional space and suppose a fluid having density
ρ (y,t) and velocity, v (y,t) is flowing through
R
this region. Then the mass of fluid leaving
V per unit time is given by the area integralR ∂V ρ (y,t) v (y,t) · n dA while the total mass
of the fluid enclosed in V at a given time is V ρ (y,t) dV . Also suppose mass originates at
the rate f (y,t) per cubic unit per unit time within this fluid. Then the conclusion which can
be drawn through the use of the divergence theorem is the following fundamental equation
known as the mass balance equation.
∂ρ
+ ∇ · (ρv) = f (y,t) (22.8)
∂t
To see this is so, take an arbitrary V for which the divergence theorem holds. Then the
time rate of change of the mass in V is
∂ ρ (y,t)
Z Z
∂
ρ (y,t) dV = dV
∂t V V ∂t
where the derivative was taken under the integral sign with respect to t. (This is a physical
derivation and therefore, it is not necessary to fuss with the hard mathematics related to
the change of limit operations. You should expect this to be true under fairly general con-
ditions because the integral is a sort of sum and the derivative of a sum is the sum of the
∂ R
derivatives.) Therefore, the rate of change of mass ∂t V ρ (y,t) dV , equals
rate at which mass enters
z Z }| {
∂ ρ (y,t)
Z Z
dV = − ρ (y,t) v (y,t) · n dA + f (y,t) dV
V ∂t V
Z∂V
= − (∇ · (ρ (y,t) v (y,t)) + f (y,t)) dV.
V

Since this holds for every sample volume V it must be the case that the equation of
continuity holds. Again, there are interesting mathematical questions here which can be
explored but since it is a physical derivation, it is not necessary to dwell too much on them.
If all the functions involved are continuous, it is certainly true but it is true under far more
general conditions than that.
Also note this equation applies to many situations and f might depend on more than
just y and t. In particular, f might depend also on temperature and the density ρ. This
would be the case for example if you were considering the mass of some chemical and f
represented a chemical reaction. Mass balance is a general sort of equation valid in many
contexts.

22.4.5 Balance Of Momentum

This example is a little more substantial than the above. It concerns the balance of mo-
mentum for a continuum. To see a full description of all the physics involved, you should
consult a book on continuum mechanics. The situation is of a material in three dimensions
and it deforms and moves about in three dimensions. This means this material is not a rigid
body. Let B0 denote an open set identifying a chunk of this material at time t = 0 and let
Bt be an open set which identifies the same chunk of material at time t > 0.
22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 413

Let y (t, x) = (y1 (t, x) , y2 (t, x) , y3 (t, x)) denote the position with respect to Cartesian
coordinates at time t of the point whose position at time t = 0 is x = (x1 , x2 , x3 ). The
coordinates x are sometimes called the reference coordinates and sometimes the material
coordinates and sometimes the Lagrangian coordinates. The coordinates y are called the
Eulerian coordinates or sometimes the spacial coordinates and the function (t, x) → y (t, x)
is called the motion. Thus
y (0, x) = x. (22.9)
The derivative,
D2 y (t, x) ≡ Dx y (t, x)
is called the deformation gradient. Recall the notation means you fix t and consider the
function x → y (t, x), taking its derivative. Since it is a linear transformation, it is repre-
sented by the usual matrix, whose i jth entry is given by

∂ yi (t, x)
Fi j (x) = .
∂xj

Let ρ (t, y) denote the density of the material at time t at the point y and let ρ 0 (x) denote
the density of the material at the point x. Thus ρ 0 (x) = ρ (0, x) = ρ (0, y (0, x)). The first
task is to consider the relationship between ρ (t, y) and ρ 0 (x). The following picture is
useful to illustrate the ideas.
N
x y = y(t, x)
V0 Vt
y
n

Lemma 22.4.1 ρ 0 (x) = ρ (t, y (t, x)) det (F) and in any reasonable physical motion det (F) >
0.

Proof: Let V0 represent a small chunk of material at t = 0 and let Vt represent the same
chunk of material at time t. I will be a little sloppy and refer to V0 as the small chunk
of material at time t = 0 and Vt as the chunk of material at time t rather than an open set
representing the chunk of material. Then by the change of variables formula for multiple
integrals, Z Z
dV = |det (F)| dV.
Vt V0

If det (F) = 0 for some t the above formula shows that the chunk of material went from pos-
itive volume to zero volume and this is not physically possible. Therefore, it is impossible
that det (F) can equal zero. However, at t = 0, F = I, the identity because of 22.9. There-
fore, det (F) = 1 at t = 0 and if it is assumed t → det (F) is continuous it follows by the inter-
mediate value theorem that det (F) > 0 for all
t. ■
Of course it is not known for sure that this function is continuous but the above shows
why it is at least reasonable to expect det (F) > 0.
414 CHAPTER 22. CALCULUS OF VECTOR FIELDS

Now using the change of variables formula

Z Z
mass of Vt = ρ (t, y) dV (y) = ρ (t, y (t, x)) det (F) dV (x)
Vt V0
Z
= mass of V0 = ρ 0 (x) dV.
V0

Since V0 is arbitrary, it follows

ρ 0 (x) = ρ (t, y (t, x)) det (F)

as claimed. Note this shows that det (F) is a magnification factor for the density.
Now consider a small chunk of material, Vt at time t which corresponds to V0 at time
t = 0. The total linear momentum of this material at time t is
Z
ρ (t, y) v (t, y) dV
Vt

where v is the velocity. By Newton’s second law, the time rate of change of this linear
momentum should equal the total force acting on the chunk of material. In the following
derivation, dV (y) will indicate the integration is taking place with respect to the variable,
y. By Lemma 22.4.1 and the change of variables formula for multiple integrals
Z 
d
ρ (t, y) v (t, y) dV (y)
dt Vt
Z 
d
= ρ (t, y (t, x)) v (t, y (t, x)) det (F) dV (x)
dt V0

Z  Z  
d ∂ v ∂ v ∂ yi
= ρ 0 (x) v (t, y (t, x)) dV (x) = ρ 0 (x) + dV (x)
dt V0 V0 ∂t ∂ yi ∂t
 
1 ∂ v ∂ v ∂ yi
Z
= ρ 0 (x) + det (F) dV (x)
V0 det (F) ∂t ∂ yi ∂t
=ρ 0 (x)
Z z }| { 1 ∂v ∂v ∂y 
i
= ρ (t, y (t, x)) det (F) + det (F) dV (y)
V0 det (F) ∂t ∂ yi ∂t
 
∂ v ∂ v ∂ yi
Z
= ρ (t, y (t, x)) + det (F) dV (y)
V0 ∂t ∂ yi ∂t
 
∂ v ∂ v ∂ yi
Z Z
= ρ (t, y) + dV (y) = ρ (t, y) v̇ dV (y)
Vt ∂t ∂ yi ∂t Vt

where the dot on v indicates it is the total derivative. Having taken the derivative of the
total momentum, it is time to consider the total force acting on the chunk of material.
The force comes from two sources, a body force b and a force which acts on the bound-
ary of the chunk of material called a traction force. Typically, the body force is something
like gravity in which case, b = −gρk, assuming the Cartesian coordinate system has been
chosen in the usual manner. The traction force is of the form
Z
s (t, y, n) dA
∂Vt
22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 415

where n is the unit exterior normal. Thus the traction force depends on position, time, and
the orientation of the boundary of Vt . Cauchy showed the existence of a linear transfor-
mation T (t, y) such that T (t, y) n = s (t, y, n). It follows there is a matrix Ti j (t, y) such
that the ith component of s is given by si (t, y, n) = Ti j (t, y) n j . Cauchy also showed this
matrix is symmetric, Ti j = T ji . It is called the Cauchy stress. Using Newton’s second law to
equate the time derivative of the total linear momentum with the applied forces and using
the usual repeated index summation convention,
Z Z Z
ρ (t, y) v̇ dV (y) = b (t, y) dV (y) + ei Ti j (t, y) n j dA,
Vt Vt ∂ Bt

the sum taken over repeated indices. Here is where the divergence theorem is used. In
the last integral, the multiplication by n j is exchanged for the jth partial derivative and an
integral over Vt . Thus

ei ∂ (Ti j (t, y))

Z Z Z
ρ (t, y) v̇ dV (y) = b (t, y) dV (y) + dV (y) ,
Vt Vt Vt ∂yj

the sum taken over repeated indices. Since Vt was arbitrary, it follows

∂ (Ti j (t, y))

ρ (t, y) v̇ = b (t, y) + ei ≡ b (t, y) + div (T )
∂yj

where here div T is a vector whose ith component is given by

∂ Ti j
(div T )i = .
∂yj

The term ∂∂tv + ∂∂ yvi ∂∂tyi , is the total derivative with respect to t of the velocity v. Thus you
might see this written as
ρ v̇ = b + div (T ) .
The above formulation of the balance of momentum involves the spatial coordinates y
but people also like to formulate momentum balance in terms of the material coordinates
x. Of course this changes everything.
The momentum in terms of the material coordinates is
Z
ρ 0 (x) v (t, x) dV
V0

and so, since x does not depend on t,

Z  Z
d
ρ 0 (x) v (t, x) dV = ρ 0 (x) vt (t, x) dV.
dt V0 V0

As indicated earlier, this is a physical derivation, so the mathematical questions related to

interchange of limit operations are ignored. This must equal the total applied force. Thus
using the repeated index summation convention,
Z Z Z
ρ 0 (x) vt (t, x) dV = b0 (t, x) dV + ei Ti j n j dA, (22.10)
V0 V0 ∂Vt
416 CHAPTER 22. CALCULUS OF VECTOR FIELDS

the first term on the right being the contribution of the body force given per unit volume
in the material coordinates and the last term being the traction force discussed earlier. The
task is to write this last integral as one over ∂V0 . For y ∈ ∂Vt there is a unit outer normal n.
Here y = y (t, x) for x ∈ ∂V0 . Then define N to be the unit outer normal to V0 at the point
x. Near the point y ∈ ∂Vt the surface ∂Vt is given parametrically in the form y = y (s,t)
for (s,t) ∈ D ⊆ R2 and it can be assumed the unit normal to ∂Vt near this point is

y s (s,t) × yt (s,t)
n=
|y s (s,t) × yt (s,t)|

with the area element given by |y s (s,t) × yt (s,t)| ds dt. This is true for y ∈ Pt ⊆ ∂Vt , a
small piece of ∂Vt . Therefore, the last integral in 22.10 is the sum of integrals over small
pieces of the form Z
Ti j n j dA (22.11)
Pt

where Pt is parameterized by y (s,t), (s,t) ∈ D. Thus the integral in 22.11 is of the form
Z
Ti j (y (s,t)) (y s (s,t) × yt (s,t)) j ds dt.
D

By the chain rule this equals

 
∂ y ∂ xα ∂ y ∂ xβ
Z
Ti j (y (s,t)) × ds dt.
D ∂ xα ∂ s ∂ xβ ∂t j

Summation over repeated indices is used. Remember y = y (t, x) and it is always assumed
the mapping x → y (t, x) is one to one and so, since on the surface ∂Vt near y, the points
are functions of (s,t), it follows x is also a function of (s,t). Now by the properties of the
cross product, this last integral equals
 
∂ xα ∂ xβ ∂ y ∂y
Z
Ti j (x (s,t)) × ds dt (22.12)
D ∂ s ∂t ∂ xα ∂ xβ j

where here x (s,t) is the point of ∂V0 which corresponds with y (s,t) ∈ ∂Vt . Thus

Ti j (x (s,t)) = Ti j (y (s,t)) .

(Perhaps this is a slight abuse of notation because Ti j is defined on ∂Vt , not on ∂V0 , but it
avoids introducing extra symbols.) Next 22.12 equals

∂ xα ∂ xβ ∂ ya ∂ yb
Z
Ti j (x (s,t)) ε jab ds dt
D ∂ s ∂t ∂ xα ∂ xβ

∂ xα ∂ xβ ∂ ya ∂ yb
Z
= Ti j (x (s,t)) ε cab δ jc ds dt
D ∂ s ∂t ∂ xα ∂ xβ
=δ jc
z }| {
∂ xα ∂ xβ ∂ yc ∂ x p ∂ ya ∂ yb
Z
= Ti j (x (s,t)) ε cab ds dt
D ∂ s ∂t ∂ x p ∂ y j ∂ xα ∂ xβ
22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 417

=ε pαβ det(F)
}|
z {
∂ xα ∂ xβ ∂ x p ∂ yc ∂ ya ∂ yb
Z
= Ti j (x (s,t)) ε cab ds dt
D ∂ s ∂t ∂ y j ∂ x p ∂ xα ∂ xβ
∂ xα ∂ xβ ∂ x p
Z
= (det F) Ti j (x (s,t)) ε pαβ ds dt.
D ∂ s ∂t ∂ y j
∂ xp
Now ∂yj = Fp−1
j and also
∂ xα ∂ xβ
= (xs × xt ) p
ε pαβ
∂ s ∂t
so the result just obtained is of the form
Z
(det F) Fp−1
j Ti j (x (s,t)) (xs × xt ) p ds dt =
D
Z
(det F) Ti j (x (s,t)) F −T


jp
(xs × xt ) p ds dt.
D
This has transformed the integral over Pt to one over P0 , the part of ∂V0 which corresponds
with Pt . Thus the last integral is of the form
Z
det (F) T F −T


ip
N p dA
P0

Summing these up over the pieces of ∂Vt and ∂V0 , yields the last integral in 22.10 equals
Z
det (F) T F −T


ip
N p dA
∂V0

and so the balance of momentum in terms of the material coordinates becomes

Z Z Z
ei det (F) T F −T ip N p dA


ρ 0 (x) vt (t, x) dV = b0 (t, x) dV +
V0 V0 ∂V0

−T


The matrix det (F) T F ip
is called the Piola Kirchhoff stress S. An application of the
divergence theorem yields


Z Z Z ∂ det (F) T F −T ip
ρ 0 (x) vt (t, x) dV = b0 (t, x) dV + ei dV.
V0 V0 V0 ∂ xp

Since V0 is arbitrary, a balance law for momentum in terms of the material coordinates is
obtained


∂ det (F) T F −T ip
ρ 0 (x) vt (t, x) = b0 (t, x) + ei
∂ xp
= b0 (t, x) + div det (F) T F −T

= b0 (t, x) + div S. (22.13)

As just shown, the relation between the Cauchy stress and the Piola Kirchhoff stress is

S = det (F) T F −T ,


(22.14)
418 CHAPTER 22. CALCULUS OF VECTOR FIELDS

perhaps not the first thing you would think of.

The main purpose of this presentation is to show how the divergence theorem is used
in a significant way to obtain balance laws and to indicate a very interesting direction for
further study. To continue, one needs to specify T or S as an appropriate function of things
related to the motion y. Often the thing related to the motion is something called the strain
and such relationships are known as constitutive laws.

22.4.6 The Reynolds Transport Formula

The Reynolds transport formula is another interesting application of the divergence theorem
which is a generalization of the formula for taking the derivative under an integral.
Z b(t) Z b(t)
d ∂f
f (x,t) dx = (x,t) dx + f (b (t) ,t) b′ (t) − f (a (t) ,t) a′ (t)
dt a(t) a(t) ∂t
Of course there are difficult analytical questions connected with such a formal procedure,
but these can be easily justified with sufficient machinery involving the Lebesgue integral.
An elementary version of theorems necessary to justify this will be fussy and unpleasant so
I am going to emphasize the derivation of the formula without worrying about interchange
of limit considerations and whether the divergence theorem holds for the region of interest.
First is an interesting lemma about the determinant. A p × p matrix can be thought of
2
as a vector in C p . Just imagine stringing it out into one long list of numbers. In fact, a
2
way to give the norm of a matrix is just ∑i ∑ j Ai j ≡ ∥A∥2 . You might check to see that
this is the same as (trace (AA∗ ))1/2 = ∥A∥. It is called the Frobenius norm for a matrix.
Also recall that det maps p × p matrices to C. It makes sense to ask for the derivative of
2
det on the set of invertible matrices, an open subset of C p with the norm measured as just
described. This is because A → det (A) is continuous so the set where det (A) ̸= 0 would
be an open set. Recall that trace (AB) = trace (BA) whenever both products make sense.
Indeed,
trace (AB) = ∑ ∑ Ai j B ji = trace (BA)
i j

This next lemma is a very interesting observation about the determinant of a matrix
added to the identity.

Lemma 22.4.2 det (I +U) = 1 + trace (U) + o (U) where o (U) is defined in terms of the
Frobenius norm for p × p matrices.

Proof: This is obvious if p = 1 or 2. Assume true for n − 1. Then for U an n × n,

I +U =  
1 +U11 U12 ··· U1n
 U21 1 +U22 ··· U2n 
 
 .. .. .. .. 
. .
 
 . . 
Un1 ··· Un(n−1) 1 +Unn
Expand along the last column and use induction.
!
n−1
(1 +Unn ) 1 + ∑ Ukk + o (U) + o (U)
k=1
22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 419

That last term follows from observing that you have some Ukn times terms which have at
least one other factor involving some Un j . Simply expand the resulting cofactors along the
bottom row. Therefore, multiplying this out gives 1 + trace (U) + o (U) . ■
With this lemma, it is easy to find D det (F) whenever F is invertible.

det (F +U) = det F I + F −1U = det (F) det I + F −1U

= det (F) 1 + trace F −1U + o (U)

= det (F) + det (F) trace F −1U + o (U)

Therefore,
det (F +U) − det (F) = det (F) trace F −1U + o (U)

This proves the following.

Proposition 22.4.3 Let F −1 exist. Then D det (F) (U) = det (F) trace F −1U .

From this, suppose F (t) is a p × p matrix and all entries are differentiable. Then the
following describes dtd det (F) (t) .

Proposition 22.4.4 Let F (t) be a p × p matrix and all entries are differentiable. Then

d
det (F) (t) = det (F (t)) trace F −1 (t) F ′ (t)


dt
= det (F (t)) trace F ′ (t) F −1 (t)


(22.15)

The situation of interest is where x is the material coordinates and y the spacial co-
ordinates and y = h (t, x) with F = F (t, x) = D2 h (t, x) . I will write ∇y to indicate the
gradient with respect to the y variables and F ′ to indicate ∂t
∂
F (t, x). Note that h (t, x) = y
and so by the inverse function theorem, this defines x as a function of y, also as smooth as
h because it is always assumed det F > 0.
Now let Vt be h (t,V0 ) where V0 is an open set whose boundary is sufficient for using
the divergence theorem. Let f (y,t) be differentiable with as many derivatives as needed
to make the computations valid. The idea is to simplify

d
Z
f (t, y) dV (y)
dt Vt

This will involve the change of variables in which the Jacobian will be det (F) . It will not
be necessary to take the absolute value because det (F) ≤ 0 is not physically possible. Then,
it is fairly routine to justify the interchange of the derivative and the integral under suitable
assumptions. The best would be to use the dominated convergence theorem, but formally,
it is like saying the derivative of a sum is the sum of the derivatives. There is of course
the question whether the divergence theorem will continue to hold for Vt . This will end
up holding under typical assumptions normally used for assumptions that the divergence
theorem will hold for V0 . For example, if h (t, ·) is smooth and the boundary of V0 is
Lipschitz, all will be well, but this is an application of things like Rademacher’s theorem
and the area formula.
d d
Z Z
f (t, y) dV (y) = f (t, h (t, x)) det (F) dV (x) (22.16)
dt Vt dt V0
420 CHAPTER 22. CALCULUS OF VECTOR FIELDS
Z Z
∂ ∂
= f (·, h (·, x)) det (F) dV (x) + f (t, h (t, x)) (det (F)) dV (x)
V0 ∂t V0 ∂t

Z
∂
= f (t, h (t, x)) det (F) dV (x)
∂t
V0
Z
+ f (t, h (t, x)) trace F ′ F −1 det (F) dV (x)


V0

Z  
∂ ∂ f ∂ yi
= f (t, h (t, x)) + det (F) dV (x)
V0 ∂t ∂ yi ∂t
Z
f (t, h (t, x)) trace F ′ F −1 det (F) dV (x)


+
V0

∂ f ∂ yi
Z Z
∂
+ f (t, y) trace F ′ F −1 dV (y)


= f (t, y) dV (y) +
Vt ∂t Vt ∂ yi ∂t
∂
Now v = ∂t h (t, x) and also, as noted above, y = h (t, x) defines y as a function of x and
so trace F F −1 = ∂∂xvαi ∂∂xyαi . Hence the double sum ∂∂xvαi ∂∂xyαi is ∂∂ vyii = ∇y · v. The above then
′

gives
Z  
∂ f ∂ yi
Z
∂
f (t, y) dV (y) + + f (t, y) ∇y · v dV (y)
Vt ∂t Vt ∂ yi ∂t
Z Z
∂
= f (t, y) dV (y) + (D1 f (y, t) v + f (t, y) ∇y · v) dV (y) (22.17)
Vt ∂t Vt

Now consider the ith component of the second integral in the above. It is
Z
∇y fi (t, y) · v + f (t, y) ∇y · vdV (y)
Vt
Z
= ∇y · ( fi (t, y) v) dV (y)
Vt

At this point, use the divergence theorem to get

Z
= fi (t, y) v · ndA
∂Vt

Therefore, from 22.17 and 22.16,

d
Z Z Z
∂
f (t, y) dV (y) = f (t, y) dV (y) + f (t, y) v · ndA
dt Vt Vt ∂t ∂Vt

this is the Reynolds transport formula.

22.4.7 Frame Indifference

The proper formulation of constitutive laws involves more physical considerations such as
frame indifference in which it is required that the response of the system cannot depend
on the manner in which the Cartesian coordinate system for the spacial coordinates was
chosen.
22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 421

For Q (t) an orthogonal transformation, (see Problem 21 on Page 518) and

y ′ = q (t) + Q (t) y, n′ = Q(t)n

the new spacial coordinates are denoted by y ′ . Recall an orthogonal transformation is just
one which satisfies
Q (t)T Q (t) = Q (t) Q (t)T = I.
The stress has to do with the traction force area density produced by internal changes in
the body and has nothing to do with the way the body is observed. Therefore, it is required
that
T ′ n′ = QT n
Thus
T ′ Qn = QT n
Since this is true for any n normal to the boundary of any piece of the material considered,
it must be the case that
T ′ Q = QT
and so
T ′ = QT QT .
This is called frame indifference.
By 22.14, the Piola Kirchhoff stress S is related to T by

S = det (F) T F −T , F ≡ Dx y.

This stress involves the use of the material coordinates and a normal N to a piece of the
body in reference configuration. Thus SN gives the force on a part of ∂Vt per unit area on
∂V0 . Then for a different choice of spacial coordinates, y ′ = q (t) + Q (t) y,

−T
S′ = det F ′ T ′ F ′

but
F ′ = Dx y ′ = Q (t) Dx y = QF
and so frame indifference in terms of S is

S′ = det (F) QT QT (QF)−T = det (F) QT QT QF −T = QS

This principle of frame indifference is sometimes ignored and there are certainly inter-
esting mathematical models which have resulted from doing this, but such things cannot be
considered physically acceptable.
There are also many other physical properties which can be included, which require a
certain form for the constitutive equations. These considerations are outside the scope of
this book and require a considerable amount of linear algebra.
There are also balance laws for energy which you may study later but these are more
problematic than the balance laws for mass and momentum. However, the divergence
theorem is used in these also.
422 CHAPTER 22. CALCULUS OF VECTOR FIELDS

22.4.8 Bernoulli’s Principle

Consider a possibly moving fluid with constant density ρ and let P denote the pressure
in
R
this fluid. If B is a part of this fluid the force exerted on B by the rest of the fluid is
∂ B −PndA where n is the outer normal from B. Assume this is the only force which mat-
ters so for example there is no viscosity in the fluid. Thus the Cauchy stress in rectangular
coordinates should be  
−P 0 0
T =  0 −P 0  .
 

0 0 −P
Then div T = −∇P. Also suppose the only body force is from gravity, a force of the form
−ρgk, so from the balance of momentum

ρ v̇ = −ρgk − ∇P (x) . (22.18)

Now in all this, the coordinates are the spacial coordinates, and it is assumed they are
rectangular. Thus x = (x, y, z)T and v is the velocity while v̇ is the total derivative of
v = (v1 , v2 , v3 )T given by vt + vi v ,i . Take the dot product of both sides of 22.18 with v.
This yields
d dz d
(ρ/2) |v|2 = −ρg − P (x) .
dt dt dt
Therefore, !
d ρ |v|2
+ ρgz + P (x) = 0,
dt 2

so there is a constant C′ such that

ρ |v|2
+ ρgz + P (x) = C′
2
For convenience define γ to be the weight density of this fluid. Thus γ = ρg. Divide by γ.
Then
|v|2 P (x)
+z+ = C.
2g γ
This is Bernoulli’s2 principle. Note how, if you keep the height the same, then if you raise
|v|, it follows the pressure drops.
This is often used to explain the lift of an airplane wing. The top surface is curved,
which forces the air to go faster over the top of the wing, causing a drop in pressure which
creates lift. It is also used to explain the concept of a venturi tube in which the air loses
pressure due to being pinched which causes it to flow faster. In many of these applica-
tions, the assumptions used in which ρ is constant, and there is no other contribution to the
traction force on ∂ B than pressure, so in particular, there is no viscosity, are not correct.
However, it is hoped that the effects of these deviations from the ideal situation are small
enough that the conclusions are still roughly true. You can see how using balance of mo-
mentum can be used to consider more difficult situations. For example, you might have a
body force which is more involved than gravity.
2 There were many Bernoullis. This is Daniel Bernoulli. He seems to have been nicer than some of the others.

Daniel was actually a doctor who was interested in [Link] lived from 1700-1782.
22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 423

22.4.9 The Wave Equation

As an example of how the balance law of momentum is used to obtain an important equa-
tion of mathematical physics, suppose S = kF where k is a constant and F is the deforma-
tion gradient and let u ≡ y − x. Thus u is the displacement. Then from (22.13) you can
verify the following holds.

ρ 0 (x) utt (t, x) = b0 (t, x) + k∆u (t, x) (22.19)

In the case where ρ 0 is a constant and b0 = 0, this yields

utt − c∆u = 0.

The wave equation is utt − c∆u = 0 and so the above gives three wave equations, one for
each component.

22.4.10 A Negative Observation

Many of the above applications of the divergence theorem are based on the assumption that
matter is continuously distributed in a way that the above arguments are correct. In other
words, a continuum. However, there is no such thing as a continuum. It has been known
for some time now that matter is composed of atoms. It is not continuously distributed
through some region of space as it is in the above. Apologists for this contradiction with
reality sometimes say to consider enough of the material in question that it is reasonable
to think of it as a continuum. This mystical reasoning is then violated as soon as they
go from the integral form of the balance laws to the differential equations expressing the
traditional formulation of these laws. See Problem 10 below, for example. However, these
laws continue to be used and seem to lead to useful physical models which have value
in predicting the behavior of physical systems. This is what justifies their use, not any
fundamental truth.

22.4.11 Volumes Of Balls In Rn

Recall, B (x, r) denotes the set of all y ∈ Rn such that |y − x| < r. By the change of
variables formula for multiple integrals or simple geometric reasoning, all balls of radius r
have the same volume. Furthermore, simple reasoning or change of variables formula will
show that the volume of the ball of radius r equals α n rn where α n will denote the volume of
the unit ball in Rn . With the divergence theorem, it is now easy to give a simple relationship
between the surface area of the ball of radius r and the volume. By the divergence theorem,
x
Z Z
div xdx = x· dA
B(0,r) ∂ B(0,r) |x|
because the unit outward normal on ∂ B (0, r) is |x
x|
. Therefore, denoting A (∂ B) as the area
of ∂ B,
nα n rn = rA (∂ B (0, r))
and so
A (∂ B (0, r)) = nα n rn−1 .
You recall the surface area of S2 ≡ x ∈ R3 : |x| = r is given by 4πr2 while the volume


of the ball, B (0, r) is 34 πr3 . This follows the above pattern. You just take the derivative
424 CHAPTER 22. CALCULUS OF VECTOR FIELDS

with respect to the radius of the volume of the ball of radius r to get the area of the surface
of this ball. Let ω n denote the area of the sphere Sn−1 = {x ∈ Rn : |x| = 1}. I just showed
that ω n = nα n .
I want to find α n now and also to get a relationship between ω n and ω n−1 . Consider
the following picture of the ball of radius ρ seen on the side.

r
y
ρ Rn−1

Taking slices at height y as shown and using that these slices have n − 1 dimensional
area equal to α n−1 rn−1 , it follows
Z ρ
(n−1)/2
α nρ n = 2 α n−1 ρ 2 − y2 dy
0

In the integral, change variables, letting y = ρ cos θ . Then

Z π/2
α n ρ n = 2ρ n α n−1 sinn (θ ) dθ .
0

It follows that Z π/2

α n = 2α n−1 sinn (θ ) dθ . (22.20)
0
Consequently,
2nω n−1
Z π/2
ωn = sinn (θ ) dθ . (22.21)
n−1 0
This is a little messier than I would like.
Z π/2 Z π/2
sinn (θ ) dθ = − cos θ sinn−1 θ |0 cos2 θ sinn−2 θ
π/2
+ (n − 1)
0 0
Z π/2
1 − sin2 θ sinn−2 (θ ) dθ


= (n − 1)
0
Z π/2 Z π/2
= (n − 1) sinn−2 (θ ) dθ − (n − 1) sinn (θ ) dθ
0 0

Hence Z π/2 Z π/2

n sinn (θ ) dθ = (n − 1) sinn−2 (θ ) dθ (22.22)
0 0
and so (22.21) is of the form
Z π/2
ω n = 2ω n−1 sinn−2 (θ ) dθ . (22.23)
0

So what is α n explicitly? Clearly α 1 = 2 and α 2 = π.

22.4. SOME APPLICATIONS OF THE DIVERGENCE THEOREM 425

π n/2
Theorem 22.4.5 α n = Γ( n2 +1)
where Γ denotes the gamma function, defined for α > 0 by
Z ∞
Γ (α) ≡ e−t t α−1 dt.
0

Proof: Recall that Γ (α + 1) = αΓ (α). Now note the given formula holds if n = 1
because     √
1 1 1 π
Γ +1 = Γ = .
2 2 2 2

√
(I leave it as an exercise for you to verify that Γ 12 = π. This is also outlined in an
√
π
exercise in Volume 1.) Thus α 1 = 2 = √π/2 satisfying the formula. Now suppose this
formula holds for k ≤ n. Then from the induction hypothesis, (22.23), (22.22), (22.20) and
(22.21),
n
Z π/2 Z π/2
α n+1 = 2α n sinn+1 (θ ) dθ = 2α n sinn−1 (θ ) dθ
0 n+1 0
Γ n−2


n α n−1 π n/2 n 1/2 2 +1
= 2α n = π
Γ 2n + 1 n + 1


Γ n−1


n + 1 2α n−2 2 +1
n−2


π n/2 n 1/2 Γ 2 +1
= π
Γ n−2

n

Γ n−1


2 +1 2
n+1 2 +1
1 1 1 1
= 2π (n+1)/2 n−1

= π (n+1)/2 n+1

n−1


n+1 Γ 2 +1 2 Γ 2 +1
1 π (n+1)/2
= π (n+1)/2 n+1 n+1

=
. ■
Γ n+1


2 Γ 2 2 +1

22.4.12 Electrostatics
Coloumb’s law says that the electric field intensity at x of a charge q located at point x0 is
given by
q (x − x0 )
E=k
|x − x0 |3
where the electric field intensity is defined to be the force experienced by a unit positive
charge placed at the point x. Note that this is a vector and that its direction depends on the
sign of q. It points away from x0 if q is positive and points toward x0 if q is negative. The
constant k is a physical constant like the gravitation constant. It has been computed through
careful experiments similar to those used with the calculation of the gravitation constant.
The interesting thing about Coloumb’s law is that E is the gradient of a function. In
fact,  
1
E = ∇ qk .
|x − x0 |
The other thing which is significant about this is that in three dimensions and for x ̸= x0 ,
 
1
∇ · ∇ qk = ∇ · E = 0. (22.24)
|x − x0 |
426 CHAPTER 22. CALCULUS OF VECTOR FIELDS

This is left as an exercise for you to verify.

These observations will be used to derive a very important formula for the integral
Z
E · ndS
∂U

where E is the electric field intensity due to a charge, q located at the point x0 ∈ U, a
bounded open set for which the divergence theorem holds.
Let Uε denote the open set obtained by removing the open ball centered at x0 which
has radius ε where ε is small enough that the following picture is a correct representation
of the situation.

ε
x0 Uε
Bε

x−x0
Then on the boundary of Bε the unit outer normal to Uε is − |x−x | . Therefore, 0

q (x − x0 )
x − x0
Z Z
E · ndS = − k dS3
·
∂ Bε ∂ Bε |x − x0 | |x − x0 |
1 −kq
Z Z
= −kq 2
dS = dS
∂ Bε |x − x0 | ε 2 ∂ Bε
−kq
= 4πε 2 = −4πkq.
ε2
Therefore, from the divergence theorem and observation (22.24),
Z Z Z
−4πkq + E · ndS = E · ndS = ∇ · EdV = 0.
∂U ∂Uε Uε
R
It follows that 4πkq = ∂U E · ndS. If there are several charges located inside U, say
q1 , q2 , · · · , qn , then letting E i denote the electric field intensity of the ith charge and E
denoting the total resulting electric field intensity due to all these charges,
Z n Z n n
E · ndS = ∑ E i · ndS = ∑ 4πkqi = 4πk ∑ qi .
∂U i=1 ∂U i=1 i=1

This is known as Gauss’s law and it is the fundamental result in electrostatics.

22.5 Exercises
1. To prove the divergence theorem, it was shown first that the spacial partial deriva-
tive in the volume integral could be exchanged for multiplication by an appropriate
component of the exterior normal. This problem starts with the divergence theorem
and goes the Rother direction. Assuming the divergenceR theorem, holds for a region
V , show that ∂V nu dA = V ∇u dV . Note this implies V ∂∂ ux dV = ∂V n1 u dA.
R R
22.5. EXERCISES 427

2. Fick’s law for diffusion states the flux of a diffusing species, J is proportional to
the gradient of the concentration, c. Write this law getting the sign right for the
constant of proportionality and derive an equation similar to the heat equation for
the concentration, c. Typically, c is the concentration of some sort of pollutant or a
chemical.
3. Sometimes people consider diffusion in materials which are not homogeneous. This
means that J = −K∇c where K is a 3 × 3 matrix. Thus in terms of components,
Ji = − ∑ j Ki j ∂∂xcj . Here c is the concentration which means the amount of pollutant
or whatever is diffusing in a volume is obtained by integrating c over the volume.
Derive a formula for a nonhomogeneous model of diffusion based on the above.
R
4. Let V be such that the divergence theorem holds. Show that V ∇ · (u∇v) dV =
R ∂v ∂v
∂V u ∂ n dA where n is the exterior normal and ∂ n denotes the directional derivative
of v in the direction n.
5. Let V be such that the divergence theorem holds. Show that
Z  
∂u ∂v
Z  
2 2
v∇ u − u∇ v dV = v −u dA
V ∂V ∂n ∂n
∂u
where n is the exterior normal and ∂n is defined in Problem 4.

6. Let V be a ball and suppose ∇2 u = f in V while u = g on ∂V . Show that there

is at most one solution to this boundary value problem which is C2 in V and con-
tinuous on V with its boundary. Hint: You might consider w = u − v where u and
v are solutions to the problem. Then use the result of Problem 4 and the identity
w∇2 w = ∇ · (w∇w) − ∇w · ∇w to conclude ∇w = 0. Then show this implies w must
be a constant by considering h (t) = w (t x+ (1 − t) y) and showing h is a constant.
Alternatively, you might consider the maximum principle.
R
7. Show that ∂V ∇ × v · n dA = 0 where V is a region for which the divergence theorem
holds and v is a C2 vector field.
8. Let F (x, y, z) = (x, y, z) be a vector field
R
in R3 and let V be a three dimensional shape
and let n = (n1 , n2 , n3 ). Show that ∂V (xn1 + yn2 + zn3 ) dA = 3× volume of V .
9. Let F = xi+yj +zk and let V denote the tetrahedron formed by the planes, x = 0, y =
0, z = 0, and 31 x + 13 y + 15 z = 1. Verify the divergence theorem for this example.
10. Suppose f :RU → R is continuous where U is some open set and for all B ⊆ U where
B is a ball, B f (x) dV = 0. Show that this implies f (x) = 0 for all x ∈ U.
11. Let U denote the box centered at (0, 0, 0) with sides parallel to theR coordinate planes
which has width 4, length 2 and height 3. Find the flux integral ∂U F · ndS where
F = (x + 3, 2y, 3z). Hint: If you like, you might want to use the divergence theorem.
2 2

whose base is x + y ≤ 1 which has height 2 of the
12. Find the flux out of the cylinder
2
vector field F = xy, zy, z + x .
13. Find the flux out of the ball of radius 4 centered at 0 of the vector field F =
(x, zy, z + x).
428 CHAPTER 22. CALCULUS OF VECTOR FIELDS

14. Verify (22.19) from (22.13) and the assumption that S = kF.
15. Show that if uk , k = 1, 2, · · · , n each satisfies (22.7) with f = 0 then for any choice of
constants c1 , · · · , cn , so does ∑nk=1 ck uk .

16. Suppose k (x) = k, a constant and f = 0. Then in one dimension, the heat equation is
2
of the form ut = αuxx . Show that u (x,t) = e−αn t sin (nx) satisfies the heat equation3 .
17. Let U
R
be a three dimensional
R
region for which the divergence theorem holds. Show
that U ∇ × F dx = ∂U n × F dS where n is the unit outer normal.
18. In a linear, viscous, incompressible fluid, the Cauchy stress is of the form
 
vi, j (t, y) + v j,i (t, y)
Ti j (t, y) = λ − pδ i j
2

where p is the pressure, δ i j equals 0 if i ̸= j and 1 if i = j, and the comma followed

by an index indicates the partial derivative with respect to that variable and v is the
velocity. Thus vi, j = ∂∂ yvij . Also, p denotes the pressure. Show, using the balance of
mass equation that incompressible implies div v = 0. Next show that the balance of
momentum equation requires
 
λ ∂v ∂v λ
ρ v̇− ∆v = ρ + vi − ∆v = b − ∇p.
2 ∂t ∂ yi 2

This is the famous Navier Stokes equation for incompressible viscous linear flu-
ids. There are still open questions related to this equation, one of which is worth
$1,000,000 at this time.

3 Fourier, an officer in Napoleon’s army studied solutions to the heat equation back in 1813. He was interested
in heat flow in cannons. He sought to find solutions by adding up infinitely many solutions of this form. Actually,
it was a little more complicated because cannons are not one dimensional but it was the beginning of the study of
Fourier series, a topic which fascinated mathematicians for the next 150 years and motivated the development of
analysis.
Chapter 23

Stokes And Green’s Theorems

23.1 Green’s Theorem

Green’s theorem is an important theorem which relates line integrals to integrals over a
surface in the plane. It can be used to establish the seemingly more general Stoke’s theorem
but is interesting for it’s own sake. Historically, theorems like it were important in the
development of complex analysis. I will first establish Green’s theorem for regions of a
particular sort and then show that the theorem holds for many other regions also. Suppose
a region is of the form indicated in the following picture in which

U = {(x, y) : x ∈ (a, b) and y ∈ (b (x) ,t (x))}

= {(x, y) : y ∈ (c, d) and x ∈ (l (y) , r (y))} .

d y = t(x)

x = l(y) U x = r(y)

c y = b(x)
a b
I will refer to such a region as being convex in both the x and y directions.

Lemma 23.1.1 Let F (x, y) ≡ (P (x, y) , Q (x, y)) be a C1 vector field defined near U where
U is a region of the sort indicated in the above picture which is convex in both the x and
y directions. Suppose also that the functions r, l,t, and b in the above picture are all C1
functions and denote by ∂U the boundary of U oriented such that the direction of motion is
counter clockwise. (As you walk around U on ∂U, the points of U are on your left.) Then
Z  
∂Q ∂P
Z Z
Pdx + Qdy ≡ F ·d R = − dA. (23.1)
∂U ∂U U ∂x ∂y
Proof: First consider the right side of (23.1).
Z   Z d Z r(y) Z b Z t(x)
∂Q ∂P ∂Q ∂P
− dA = dxdy − dydx
U ∂x ∂y c l(y) ∂ x a b(x) ∂ y

429
430 CHAPTER 23. STOKES AND GREEN’S THEOREMS

Z d
= (Q (r (y) , y) − Q (l (y) , y)) dy
c
Z b
+ (P (x, b (x))) − P (x,t (x)) dx. (23.2)
a

Now consider the left side of (23.1). Denote by V the vertical parts of ∂U and by H the
horizontal parts. Z Z
F ·d R = ((0, Q) + (P, 0)) · dR
∂U ∂U

Z d Z
(0, Q (r (s) , s)) · r′ (s) , 1 ds + (0, Q (r (s) , s)) · (±1, 0) ds


=
c H
Z d Z b
′
(P (s, b (s)) , 0) · 1, b′ (s) ds



− (0, Q (l (s) , s)) · l (s) , 1 ds +
c a
Z Z b
(P (s,t (s)) , 0) · 1,t ′ (s) ds


+ (P (s, b (s)) , 0) · (0, ±1) ds −
V a
Z d Z d Z b Z b
= Q (r (s) , s) ds − Q (l (s) , s) ds + P (s, b (s)) ds − P (s,t (s)) ds
c c a a

which coincides with (23.2). ■

Corollary 23.1.2 Let everything be the same as in Lemma 23.1.1 but only assume the
functions r, l,t, and b are continuous and piecewise C1 functions. Then the conclusion this
lemma is still valid.

Proof: The details are left for you. All you have to do is to break up the various line
integrals into the sum of integrals over sub intervals on which the function of interest is C1 .
■
From this corollary, it follows (23.1) is valid for any triangle for example.
Now suppose (23.1) holds for U1 ,U2 , · · · ,Um and the open sets Uk have the property that
no two have nonempty intersection and their boundaries intersect only in a finite number of
piecewise smooth curves. Then (23.1) must hold for U ≡ ∪m i=1Ui , the union of these sets.
This is because  
∂Q ∂P
Z
− dA =
U ∂x ∂y

m  
∂Q ∂P
Z
= ∑ − dA
k=1 Uk ∂x ∂y
m Z Z
= ∑ F ·d R = F · dR
k=1 ∂Uk ∂U

because if Γ = ∂Uk ∩ ∂U j , then its orientation as a part of ∂Uk is opposite to its orientation
as a part of ∂U j and consequently the line integrals over Γ will cancel, points of Γ also not
being in ∂U. As an illustration, consider the following picture for two such Uk .
23.1. GREEN’S THEOREM 431

U1

U2

Similarly, if U ⊆ V and if also ∂U ⊆ V and both U and V are open sets for which (23.1)
holds, then the open set V \ (U ∪ ∂U) consisting of what is left in V after deleting U along
with its boundary also satisfies (23.1). Roughly speaking, you can drill holes in a region for
which (23.1) holds and get another region for which this continues to hold provided (23.1)
holds for the holes. To see why this is so, consider the following picture which typifies the
situation just described.

U V

Then Z  
∂Q ∂P
Z
F ·d R = − dA
∂V V ∂x ∂y
Z    
∂Q ∂P ∂Q ∂P
Z
= − dA + − dA
U ∂x ∂y V \U ∂x ∂y
 
∂Q ∂P
Z Z
= F ·dR + − dA
∂U V \U ∂x ∂y
and so  
∂Q ∂P
Z Z Z
− dA = F ·d R− F ·dR
V \U ∂x ∂y ∂V ∂U
which equals Z
F · dR
∂ (V \U)
where ∂V is oriented as shown in the picture. (If you walk around the region V \ U with
the area on the left, you get the indicated orientation for this curve.)
You can see that (23.1) is valid quite generally. This verifies the following theorem.
Theorem 23.1.3 (Green’s Theorem) Let U be an open set in the plane and let ∂U be
piecewise smooth and let F (x, y) = (P (x, y) , Q (x, y)) be a C1 vector field defined near U.
Then it is often1 the case that
Z  
∂Q ∂P
Z
F · dR = (x, y) − (x, y) dA.
∂U U ∂x ∂y
1 For a general version see the advanced calculus book by Apostol. The general versions involve the concept

of a rectifiable (finite length) Jordan curve.

432 CHAPTER 23. STOKES AND GREEN’S THEOREMS

Here is an alternate proof of Green’s theorem from the divergence theorem.

Theorem 23.1.4 (Green’s Theorem) Let U be an open set in the plane and let ∂U be
piecewise smooth and let F (x, y) = (P (x, y) , Q (x, y)) be a C1 vector field defined near U.
Then it is often the case that
Z  
∂Q ∂P
Z
F · dR = (x, y) − (x, y) dA.
∂U U ∂x ∂y
Proof: Suppose the divergence theorem holds for U. Consider the following picture.

(y′ , −x′ )
(x′ , y′ )

Since it is assumed that motion around U is counter clockwise, the tangent vector (x′ , y′ )
is as shown. The unit exterior normal is a multiple of

x′ , y′ , 0 × (0, 0, 1) = y′ , −x′ , 0 .

Use your right hand and the geometric description of the cross product to verify this. This
would be the case at all the points where the unit exterior normal exists.
Now let F (x, y) =q(Q (x, y) , −P (x, y)). Also note the area (length) element on the
bounding curve ∂U is (x′ )2 + (y′ )2 dt. Suppose the boundary of U consists of m smooth
curves, the ith of which is parameterized by (xi , yi ) with the parameter t ∈ [ai , bi ]. Then by
the divergence theorem,
Z Z Z
(Qx − Py ) dA = div (F ) dA = F · ndS
U U ∂U

m Z bi
= ∑ (Q (xi (t) , yi (t)) , −P (xi (t) , yi (t)))
i=1 ai
dS
z }| {
1
q
·q y′i , −xi
′
(xi′ )2 + (y′i )2 dt
(xi′ )2 + (y′i )2
m Z bi
(Q (xi (t) , yi (t)) , −P (xi (t) , yi (t))) · y′i , −xi′ dt


= ∑
i=1 ai
m Z bi Z
= ∑ Q (xi (t) , yi (t)) y′i (t) + P (xi (t) , yi (t)) xi′ (t) dt ≡ Pdx + Qdy
i=1 ai ∂U

This proves Green’s theorem from the divergence theorem. ■

23.1. GREEN’S THEOREM 433

Proposition 23.1.5 Let U be an open set in R2 for which Green’s theorem holds. Then
Z
Area of U = F ·dR
∂U

where F (x, y) = 12 (−y, x) , (0, x), or (−y, 0).

Proof: This follows immediately from Green’s theorem. ■
Example 23.1.6 Use Proposition 23.1.5 to find the area of the ellipse
x2 y2
+ ≤ 1.
a2 b2
You can parameterize the boundary of this ellipse as
x = a cost, y = b sint, t ∈ [0, 2π] .
Then from Proposition 23.1.5,
1 2π
Z
Area equals = (−b sint, a cost) · (−a sint, b cost) dt
2 0
1 2π
Z
= (ab) dt = πab.
2 0
R
Example 23.1.7 Find ∂U F ·dR where U is the set
(x, y) : x2 + 3y2 ≤ 9


and F (x, y) = (y, −x).

One way to do this is to parameterize the boundary of U and then compute the line
integral directly. It is easier to use Green’s theorem. The desired line integral equals
Z Z
((−1) − 1) dA = −2 dA.
U U
√ √
Now U is an ellipse having area equal to 3 3 and so the answer is −6 3.
R
Example 23.1.8 Find ∂U
F ·dR where U is the set {(x, y) : 2 ≤ x ≤ 4, 0 ≤ y ≤ 3} and
F (x, y) = x sin y, y3 cos x .
From Green’s theorem this line integral equals
Z 4Z 3
81 81
−y3 sin x − x cos y dydx =


cos 4 − 6 sin 3 − cos 2.
2 0 4 4
This is much easier than computing the line integral because you don’t have to break the
boundary in pieces and consider each separately.
R
Example 23.1.9 Find ∂U F ·dR where U is the set
{(x, y) : 2 ≤ x ≤ 4, x ≤ y ≤ 4}
and F (x, y) = (x sin y, y sin x).
From Green’s theorem, this line integral equals
Z 4Z 4
(y cos x − x cos y) dydx = 4 cos 2 − 8 cos 4 − 8 sin 2 − 4 sin 4.
2 x
434 CHAPTER 23. STOKES AND GREEN’S THEOREMS

23.2 Exercises
R
1. Find S xdS
p where S is the surface which results from the intersection of the cone
z = 2 − x2 + y2 with the cylinder x2 + y2 − 2x = 0.

2. Now let n be the unit normal to the above surface R

which has positive z component
and let F (x, y, z) = (x, y, z). Find the flux integral S F · ndS.
R
3. Find S zdSpwhere S is the surface which results from the intersection of the hemi-
sphere z = 4 − x2 − y2 with the cylinder x2 + y2 − 2x = 0.

R
4. In the situation of the above problem, find the flux integral S F · ndS where n is the
unit normal to the surface which has positive z component and F = (x, y, z).
5. Let x2 /a2 + y2 /b2 = 1 be an ellipse. Show using Green’s theorem that its area is πab.
6. A spherical storage tank having radius a is filled with water which weights 62.5
pounds per cubic foot. It is shown later that this implies that the pressure of the
water at depth z equals 62.5z. Find the total force acting on this storage tank.
p
7. Let n be the unit normal to the cone z = x2 + y2 which has negative z component
and let F = (x, 0, z) be a vector field. Let S be the part of this cone which lies
between the planes z = 1 and z = 2.

R
Find SF · ndS.
8. Let S be the surface z = 9 − x2 − y2 for x2 +Ry2 ≤ 9. Let n be the unit normal to S
which points up. Let F = (y, −x, z) and find S F · ndS.
23.3. STOKE’S THEOREM FROM GREEN’S THEOREM 435

9. Let S be the surface 3z = 9 − x2 − y2 for x2 +R y2 ≤ 9. Let n be the unit normal to S

which points up. Let F = (y, −x, z) and find S F · ndS.
10. For F = (x, y, z), S is the part of the cylinder x2 + y2 = 1 between the planes z = 1
and z = 3. Letting n be the unit normal which points away from the z axis, find
R
S F · ndS.
π
11. Let S be the part of the sphere of radius a which liesR
between the two cones φ = 4
and φ = π6 . Let F = (z, y, 0). Find the flux integral S F · ndS.
12. Let S be the part of a sphere of radius a above
R
the plane z = a2 , F = (2x, 1, 1) and let
n be the unit upward normal on S. Find S F · ndS.

13. In the above, problem, let C be the Rboundary of S oriented counter clockwise as
viewed from high on the z axis. Find C 2xdx + dy + dz.
14. Let S be the top half of a sphere of
R
radius a centered at 0 and let n be the unit outward
normal. Let F = (0, 0, z). Find S F · ndS.

15. Let D be a circleR in the plane which has radius 1 and let C be its counter clockwise
boundary. Find C ydx + xdy.
16. Let D be a circleR in the plane which has radius 1 and let C be its counter clockwise
boundary. Find C ydx − xdy.
R
17. Find C (x + y) dx where C is the square curve which goes from (0, 0) → (1, 0) →
(1, 1) → (0, 1) → (0, 0).
2
R
18. Find the line integral C (sin x + y) dx + y dy where C is the oriented square

(0, 0) → (1, 0) → (1, 1) → (0, 1) → (0, 0) .

19. Let P (x, y) = x2−y

+yR2
x
, Q (x, y) = x2 +y 2 . Show Qx − Py = 0. Let D be the unit disk.

Compute directly C Pdx + Qdy where C is the counter clockwise circle of radius 1
which bounds the unit disk. Why don’t you get 0 for the line integral?

20. Let F = 2y, ln 1 + y2 + x . Find C F ·dR where C is the curve consisting of line


R

segments,
(0, 0) → (1, 0) → (1, 1) → (0, 0) .

23.3 Stoke’s Theorem From Green’s Theorem

Stoke’s theorem is a generalization of Green’s theorem which relates the integral over a
surface to the integral around the boundary of the surface. These terms are a little different
from what occurs in R2 . To describe this, consider a sock. The surface is the sock and its
boundary will be the edge of the opening of the sock in which you place your foot. Another
way to think of this is to imagine a region in R2 of the sort discussed above for Green’s
theorem. Suppose it is on a sheet of rubber and the sheet of rubber is stretched in three
dimensions. The boundary of the resulting surface is the result of the stretching applied to
the boundary of the original region in R2 . Here is a picture describing the situation.
436 CHAPTER 23. STOKES AND GREEN’S THEOREMS

S
∂S

Recall the following definition of the curl of a vector field.

Definition 23.3.1 Let

F (x, y, z) = (F1 (x, y, z) , F2 (x, y, z) , F3 (x, y, z))

be a C1 vector field defined on an open set V in R3 . Then

i j k      
∂ ∂ ∂ ∂ F3 ∂ F2 ∂ F1 ∂ F3 ∂ F2 ∂ F1
∇×F ≡ ∂x ∂y ∂z ≡ − i+ − j+ − k.
∂y ∂z ∂z ∂x ∂x ∂y
F1 F2 F3

This is also called curl (F ) and written as indicated, ∇ × F .

The following lemma gives the fundamental identity which will be used in the proof of
Stoke’s theorem.

Lemma 23.3.2 Let R : U → V ⊆ R3 where U is an open subset of R2 and V is an open

subset of R3 . Suppose R is C2 and let F be a C1 vector field defined in V .

(Ru × Rv ) · (∇ × F ) (R (u, v)) = ((F ◦ R)u · Rv − (F ◦ R)v · Ru ) (u, v) . (23.3)

Proof: Start with the left side and let xi = Ri (u, v) for short.

∂ Fs
(Ru × Rv ) · (∇ × F ) (R (u, v)) = ε i jk x ju xkv ε irs
∂ xr
∂ Fs
= (δ jr δ ks − δ js δ kr ) x ju xkv
∂ xr
∂ Fk ∂ Fj
= x ju xkv − x ju xkv
∂xj ∂ xk
∂ (F ◦ R) ∂ (F ◦ R)
= Rv · − Ru ·
∂u ∂v
which proves (23.3). ■
The proof of Stoke’s theorem given next follows [11]. First, it is convenient to give a
definition.

Definition 23.3.3 A vector valued function R : U ⊆ Rm → Rn is said to be in Ck U, Rn if

it is the restriction to U of a vector valued function which is defined on Rm and is Ck . That

is, this function has continuous partial derivatives up to order k.
23.3. STOKE’S THEOREM FROM GREEN’S THEOREM 437

Theorem 23.3.4 (Stoke’s Theorem) Let U be any
region in R2 for which the conclusion
of Green’s theorem holds and let R ∈ C2 U, R3 be a one to one function satisfying
|(Ru × Rv ) (u, v)| ̸= 0 for all (u, v) ∈ U and let S denote the surface

S ≡ {R (u, v) : (u, v) ∈ U} ,
∂S ≡ {R (u, v) : (u, v) ∈ ∂U}

where the orientation on ∂ S is consistent with the counter clockwise orientation on ∂U (U

is on the left as you walk around ∂U). Then for F a C1 vector field defined near S,
Z Z
F · dR = curl (F ) · ndS
∂S S

where n is the normal to S defined by

Ru × Rv
n≡ .
|Ru × Rv |

Proof: Letting C be an oriented part of ∂U having parametrization,

r (t) ≡ (u (t) , v (t))

for t ∈ [α, β ] and letting R (C) denote the oriented part of ∂ S corresponding to C,
Z
F · dR
R(C)

Z β
F (R (u (t) , v (t))) · Ru u′ (t) + Rv v′ (t) dt


=
α
Z β
= F (R (u (t) , v (t))) Ru (u (t) , v (t)) u′ (t) dt
α
Z β
+ F (R (u (t) , v (t))) Rv (u (t) , v (t)) v′ (t) dt
α
Z
= ((F ◦ R) · Ru , (F ◦ R) · Rv ) · dr.
C

Since this holds for each such piece of ∂U, it follows

Z Z
F ·d R = ((F ◦ R) · Ru , (F ◦ R) · Rv ) · dr.
∂S ∂U

By the assumption that the conclusion of Green’s theorem holds for U, this equals
Z
[((F ◦ R) · Rv )u − ((F ◦ R) · Ru )v ] dA
ZU
= [(F ◦ R)u · Rv + (F ◦ R) · Rvu − (F ◦ R) · Ruv − (F ◦ R)v · Ru ] dA
ZU
= [(F ◦ R)u · Rv − (F ◦ R)v · Ru ] dA
U
438 CHAPTER 23. STOKES AND GREEN’S THEOREMS

the last step holding by equality of mixed partial derivatives, a result of the assumption that
R is C2 . Now by Lemma 23.3.2, this equals
Z
(Ru × Rv ) · (∇ × F ) dA
ZU
= ∇ × F · (Ru × Rv ) dA
ZU
= ∇ × F · ndS
S

(Ru ×Rv )
because dS = |(Ru × Rv )| dA and n = |(Ru ×Rv )| . Thus

(Ru × Rv )
(Ru × Rv ) dA = |(Ru × Rv )| dA
|(Ru × Rv )|
= ndS.

This proves Stoke’s theorem. ■

Note that there is no mention made in the final result that R is C2 . Therefore, it is not
surprising that versions of this theorem are valid in which this assumption is not present. It
is possible to obtain extremely general versions of Stoke’s theorem if you use the Lebesgue
integral.

23.3.1 The Normal and the Orientation

Stoke’s theorem as just presented needs no apology. However, it is helpful in applications
to have some additional geometric insight.
To begin with, suppose the surface S of interest is a parallelogram in R3 determined by
the two vectors a, b. Thus S = R (Q) where Q = [0, 1] × [0, 1] is the unit square and for
(u, v) ∈ Q,
R (u, v) ≡ ua + vb + p,
the point p being a corner of the parallelogram S. Then orient ∂ S consistent with the
counter clockwise orientation on ∂ Q. Thus, following this orientation on S you go from p
to p + a to p + a + b to p + b to p. Then Stoke’s theorem implies that with this orientation
on ∂ S, Z Z
F · dR = ∇ × F · nds
∂S S
where
n = Ru × Rv / |Ru × Rv | = a × b/ |a × b| .
Now recall a, b, a × b forms a right hand system.
p+a+b

S
p+a
b a
a×b
p
23.3. STOKE’S THEOREM FROM GREEN’S THEOREM 439

Thus, if you were walking around ∂ S in the direction of the orientation with your left
hand over the surface S, the normal vector a × b would be pointing in the direction of your
head.
More generally, if S is a surface which is not necessarily a parallelogram but is instead
as described in Theorem 23.3.4, you could consider a small rectangle Q contained in U
and orient the boundary of R (Q) consistent with the counter clockwise orientation on ∂ Q.
Then if Q is small enough, as you walk around ∂ R (Q) in the direction of the described
orientation with your left hand over R (Q), your head points roughly in the direction of
Ru × Rv .

∆v Rv (u0 )∆v
R(Q)
Q

u0 ∆u
Ru (u0 )∆u
As explained above, this is true of the tangent parallelogram, and by continuity of
Rv , Ru , the normals to the surface R (Q) Ru × Rv (u) for u ∈ Q will still point roughly in
the same direction as your head if you walk in the indicated direction over ∂ R (Q), meaning
the angle between the vector from your feet to your head and the vector Ru × Rv (u) is less
than π/2.
You can imagine filling U with such non-overlapping regions Qi . Then orienting
∂ R(Qi ) consistent with the counter clockwise orientation on Qi , and adding the resulting
line integrals, the line integrals over the common sides cancel as indicated in the following
picture and the result is the line integral over ∂ S.
Thus there is a simple relation between the field of normal vectors on S and the ori-
entation of ∂ S. It is simply this. If you walk along ∂ S in the direction mandated by the
orientation, with your left hand over the surface, the nearby normal vectors in Stoke’s the-
orem will point roughly in the direction of your head.

This also illustrates that you can define an orientation for ∂ S by specifying a field of
unit normal vectors for the surface, which varies continuously over the surface, and require
that the motion over the boundary of the surface is such that your head points roughly in
the direction of nearby normal vectors as you walk along the boundary with your left hand
over S. The existence of such a continuous field of normal vectors is what constitutes an
440 CHAPTER 23. STOKES AND GREEN’S THEOREMS

orientable surface.

23.3.2 The Mobeus Band

It turns out there are more general formulations of Stoke’s theorem than what is presented
above. However, it is always necessary for the surface S to be orientable. This means
it is possible to obtain a vector field of unit normals to the surface which is a continuous
function of position on S.
An example of a surface which is not orientable is the famous Mobeus band, obtained
by taking a long rectangular piece of paper and gluing the ends together after putting a twist
in it. Here is a picture of one.

There is something quite interesting about this Mobeus band and this is that it can be
written parametrically with a simple parameter domain. The picture above is a maple graph
of the parametrically defined surface
 θ
 x = 4 cos θ + v cos 2

R (θ , v) ≡ y = 4 sin θ + v cos θ2 , θ ∈ [0, 2π] , v ∈ [−1, 1] .

θ
z = v sin 2


An obvious question is why the normal vector R,θ × R,v / R,θ × R,v is not a continuous
function of position on S. You can see easily that it is a continuous function of both θ and
v. However, the map, R is not one to one. In fact, R (0, 0) = R (2π, 0). Therefore, near
this point on S, there are two different values for the above normal vector. In fact, a tedious
computation will show that this normal vector is

4 sin 21 θ cos θ − 12 v, 4 sin 12 θ sin θ + 21 v, −8 cos2 12 θ sin 12 θ − 8 cos3 12 θ + 4 cos 12 θ

D
where
v2
    
2 θ θ
D = 16 sin + + 4 sin v (sin θ − cos θ )
2 2 2
 !
1 2
       
3 2 θ 1 1 2 1
+ 4 cos cos θ sin θ + cos θ −
2 2 2 2 2

and you can verify that the denominator will not vanish. Letting v = 0 and θ = 0 and 2π
yields the two vectors (0, 0, −1) , (0, 0, 1) so there is a discontinuity. This is why I was
careful to say in the statement of Stoke’s theorem given above that R is one to one.
The Mobeus band has some usefulness. In old machine shops the equipment was run
by a belt which was given a twist to spread the surface wear on the belt over twice the area.
The above explanation shows that R,θ × R,v / R,θ × R,v fails to deliver an orientation
for the Mobeus band. However, this does not answer the question whether there is some
23.4. A GENERAL GREEN’S THEOREM 441

orientation for it other than this one. In fact there is none. You can see this by looking at the
first of the two pictures below or by making one and tracing it with a pencil. There is only
one side to the Mobeus band. An oriented surface must have two sides, one side identified
by the given unit normal which varies continuously over the surface and the other side
identified by the negative of this normal. The second picture below was taken by Ouyang
when he was at meetings in Paris and saw it at a museum.

23.4 A General Green’s Theorem

Now suppose U is a region in the uv plane for which Green’s theorem holds and that

V ≡ R (U)

where R is C2 U, R2 and is one to one, Ru × Rv ̸= 0. Here, to be specific, the u, v axes

are oriented as the x, y axes respectively.

y v

x u

Also let F (x, y, z) = (P (x, y) , Q (x, y) , 0) be a C1 vector field defined near V . Note that
F does not depend on z. Therefore,

∇ × F (x, y) = (Qx (x, y) − Px (x, y)) k.

You can check this from the definition. Also

!
x (u, v)
R (u, v) =
y (u, v)

and so Ru × Rv , the normal vector to V is

xu xv
yu yv
k
xu xv
yu yv
442 CHAPTER 23. STOKES AND GREEN’S THEOREMS

Suppose
xu xv
>0
yu yv
so the unit normal is then just k. Then Stoke’s theorem applied to this special case yields

xu xv
Z Z
F ·dR = (Qx (x (u, v) , y (u, v)) − Px (x (u, v) , y (u, v))) k · k dA
∂V U yu yv

Now by the change of variables formula, this equals

Z
= (Qx (x, y) − Px (x, y)) dA
V

This is just Green’s theorem for V . Thus if U is a region for which Green’s theorem holds
and if V is another region, V = R (U) , where |Ru × Rv | ̸= 0, R is one to one, and twice
continuously differentiable with Ru × Rv in the direction of k, then Green’s theorem holds
for V also.
This verifies the following theorem.
Theorem 23.4.1 (Green’s Theorem) Let V be an open set in the plane and let ∂V be piece-
wise smooth and let F (x, y) = (P (x, y) , Q (x, y)) be a C1 vector field defined near V. Then
if V is oriented counter clockwise, it is often2 the case that
Z  
∂Q ∂P
Z
F · dR = (x, y) − (x, y) dA. (23.4)
∂V V ∂x ∂y
In particular, if there exists U such as the simple convex in both directions case considered

earlier for which Green’s theorem holds, and V = R (U) where R : U → V is C2 U, R2
such that Rx × Ry ̸= 0 and Rx × Ry is in the direction of k, then 23.4 is valid where the
orientation around ∂V is consistent with the orientation around U.
This is a very general version of Green’s theorem which will include most of what will
be of interest.

23.4.1 Conservative Vector Fields

Definition 23.4.2 A vector field F defined in a three dimensional region is said to be con-
servative3 if for every piecewise smooth closed curve C, it follows C F · dR = 0.
R

Definition 23.4.3 Let (x, p1 , · · · , pn , y) be an ordered list of points in R p . Let

p (x, p1 , · · · , pn , y)
denote the piecewise smooth curve consisting of a straight line segment from x to p1 and
then the straight line segment from p1 to p2 · · · and finally the straight line segment from
pn to y. This is called a polygonal curve. An open set in R p , U, is said to be a region if
it has the property that for any two points x, y ∈ U, there exists a polygonal curve joining
the two points.
2 For a general version see the advanced calculus book by Apostol. This is presented in the next section also.

The general versions involve the concept of a rectifiable Jordan curve. You need to be able to take the area integral
and to take the line integral around the boundary.
3 There is no such thing as a liberal vector field.
23.4. A GENERAL GREEN’S THEOREM 443

Conservative vector fields are important because of the following theorem, sometimes
called the fundamental theorem for line integrals.

Theorem 23.4.4 Let U be a region in R p and let F : U → R p be a continuous vector field.

Then F is conservative if and only if there exists a scalar valued function of p variables φ
such that F = ∇φ . Furthermore, if C is an oriented curve which goes from x to y in U,
then Z
F · dR = φ (y) − φ (x) . (23.5)
C
Thus the line integral is path independent in this case. This function φ is called a scalar
potential for F .

Proof: To save space and fussing over things which are unimportant, denote by p (x0 , x)
a polygonal curve from x0 to x. Thus the orientation is such that it
goes from x0 to x. The curve p (x, x0 ) denotes the same set of points but in the opposite
order. Suppose first F is conservative. Fix x0 ∈ U and let
Z
φ (x) ≡ F · dR.
p(x0 ,x)

This is well defined because if q (x0 , x) is another polygonal curve joining x0 to x, Then
the curve obtained by following p (x0 , x) from x0 to x and then from x to x0 along
q (x, x0 ) is a closed piecewise smooth curve and so by assumption, the line integral along
this closed curve equals 0. However, this integral is just
Z Z Z Z
F · d R+ F ·d R = F · d R− F · dR
p(x0 ,x) q (x,x0 ) p(x0 ,x) q (x0 ,x)

which shows Z Z
F ·d R = F · dR
p(x0 ,x) q (x0 ,x)

and that φ is well defined. For small t,

R R
φ (x + tei ) − φ (x) p(x0 ,x+t ei ) F · d R− p(x0 ,x) F · dR
=
t R tR R
p(x0 ,x) F · d R+ p(x,x+t ei ) F · d R− p(x0 ,x) F · dR
= .
t
Since U is open, for small t, the ball of radius |t| centered at x is contained in U. There-
fore, the line segment from x to x + tei is also contained in U and so one can take
p (x, x + tei ) (s) = x + s (tei ) for s ∈ [0, 1]. Therefore, the above difference quotient re-
duces to
Z 1 Z 1
1
F (x + s (tei )) · tei ds = Fi (x + s (tei )) ds
t 0 0
= Fi (x + st (tei ))

by the mean value theorem for integrals. Here st is some number between 0 and 1. By
continuity of F, this converges to Fi (x) as t → 0. Therefore, ∇φ = F as claimed.
444 CHAPTER 23. STOKES AND GREEN’S THEOREMS

Conversely, if ∇φ = F, then if R : [a, b] → R p is any C1 curve joining x to y,

Z b Z b
F (R (t)) ·R′ (t) dt = ∇φ (R (t)) ·R′ (t) dt
a a
Z b
d
= (φ (R (t))) dt
dta
= φ (R (b)) − φ (R (a))
= φ (y) − φ (x)

and this verifies (23.5) in the case where the curve joining the two points is smooth. The
general case follows immediately from this by using this result on each of the pieces of the
piecewise smooth curve. For example if the curve goes from x to p and then from p to y,
the above would imply the integral over the curve from x to p is φ (p) − φ (x) while from p
to y the integral would yield φ (y) − φ (p). Adding these gives φ (y) − φ (x). The formula
(23.5) implies the line integral over any closed curve equals zero because the starting and
ending points of such a curve are the same. ■

Example 23.4.5 Let F (x, y, z) = (cos x − yz sin (xz) , cos

(xz) , −yx
R
sin (xz)). Let C be a piece-
wise smooth curve which goes from (π, 1, 1) to π2 , 3, 2 . Find C F · dR.

The specifics of the curve are not given so the problem is nonsense unless the vector
field is conservative. Therefore, it is reasonable to look for the function φ satisfying ∇φ =
F. Such a function satisfies
φ x = cos x − y (sin xz) z
and so, assuming φ exists,

φ (x, y, z) = sin x + y cos (xz) + ψ (y, z) .

I have to add in the most general thing possible, ψ (y, z) to ensure possible solutions are
not being thrown out. It wouldn’t be good at this point to only add in a constant since the
answer could involve a function of either or both of the other variables. Now from what
was just obtained,
φ y = cos (xz) + ψ y = cos xz
and so it is possible to take ψ y = 0. Consequently, φ , if it exists is of the form

φ (x, y, z) = sin x + y cos (xz) + ψ (z) .

Now differentiating this with respect to z gives

φ z = −yx sin (xz) + ψ z = −yx sin (xz)

and this shows ψ does not depend on z either. Therefore, it suffices to take ψ = 0 and

φ (x, y, z) = sin (x) + y cos (xz) .

Therefore, the desired line integral equals

π 
sin + 3 cos (π) − (sin (π) + cos (π)) = −1.
2
The above process for finding φ will not lead you astray in the case where there does
not exist a scalar potential. As an example, consider the following.
23.4. A GENERAL GREEN’S THEOREM 445

Example 23.4.6 Let F (x, y, z) = x, y2 x, z . Find a scalar potential for F if it exists.

2
If φ exists, then φ x = x and so φ = x2 + ψ (y, z). Then φ y = ψ y (y, z) = xy2 but this
is impossible because the left side depends only on y and z while the right side depends
also on x. Therefore, this vector field is not conservative and there does not exist a scalar
potential.
Definition 23.4.7 A set of points in three dimensional space V is simply connected if every
piecewise smooth closed curve C is the edge of a surface S which is contained entirely
within V in such a way that Stokes theorem holds for the surface S and its edge, C.

This is like a sock. The surface is the sock and the curve C goes around the opening of
the sock.
As an application of Stoke’s theorem, here is a useful theorem which gives a way to
check whether a vector field is conservative.
Theorem 23.4.8 For a three dimensional simply connected open set V and F a C1 vector
field defined in V , F is conservative if ∇ × F = 0 in V .
Proof: If ∇ × F = 0 then taking an arbitrary closed curve C, and letting S be a surface
bounded by C which is contained in V , Stoke’s theorem implies
Z Z
0= ∇ × F · n dA = F · dR.
S C

Thus F is conservative. ■
Example 23.4.9 Determine whether the vector field
4x3 + 2 cos x2 + z2 x, 1, 2 cos x2 + z2 z

is conservative.
Since this vector field is defined on all of R3 , it only remains to take its curl and see if
it is the zero vector.
i j k
∂x ∂y ∂z .
4x3 + 2 cos x2 + z2 x 2 2





1 2 cos x + z z
This is obviously equal to zero. Therefore, the given vector field is conservative. Can you
find a potential function for it?
Let φ be the potential function. Then φ z = 2 cos x2 + z2 z

and so φ (x, y, z) = sin x2 + z2 + g (x, y). Now taking the derivative of φ with respect to y,
2 + z2 . Taking


you see gy = 1 so g (x, y) = y + h (x). Hence φ (x, y, z) = y + g (x) + sin x
the derivative with respect to x, you get 4x3 + 2 cos x2 + z2 x = g′ (x) +
2x cos x2 + z2

and so it suffices to take g (x) = x4 . Hence φ (x, y, z) = y + x4 + sin x2 + z2 .

446 CHAPTER 23. STOKES AND GREEN’S THEOREMS

23.4.2 Some Terminology

If F = (P, Q, R) is a vector field. Then the statement that F is conservative is the same as
saying the differential form Pdx + Qdy + Rdz is exact. Some people like to say things in
terms of vector fields and some say it in

terms

of differential forms. In


Example 23.4.9, the
differential form 4x3 + 2 cos x2 + z2 x dx + dy + 2 cos x2 + z2 z dz is exact.


Chapter 24

Moving Coordinate Systems

24.1 The Acceleration In Polar Coordinates

I assume that by now, the reader has encountered Newton’s laws of motion, especially the
second law which gives the relationship, force equals mass times acceleration. Sometimes
you have information about forces which act not in the direction of the coordinate axes but
in some other direction. When this is the case, it is often useful to express things in terms
of different coordinates which are consistent with these directions. A good example of this
is the force exerted by the sun on a planet. This force is always directed toward the sun and
so the force vector changes as the planet moves. To discuss this, consider the following
simple diagram in which two unit vectors er and eθ are shown.
eθ • er
(r, θ )

The vector er = (cos θ , sin θ ) and the vector eθ = (− sin θ , cos θ ). Note that eθ · er =
0. You should convince yourself that the directions of these two perpendicular vectors
correspond to what is shown in the above picture. To help with this, note that er × eθ =
k if these vectors are considered as eθ = (− sin θ , cos θ , 0) , er = (cos θ , sin θ , 0)and so
(er , eθ , k) forms a right hand system, so if you see that er points away from the origin,
then it follows that eθ points in the direction shown.
These two vectors also have the following relationship

der deθ
eθ = , er = − . (24.1)
dθ dθ
Now consider the position vector from 0 of a point in the plane, r (t). Then if r (t) , θ (t)
are its polar coordinates at time t,

r (t) = r (t) er (θ (t))

where r (t) = |r (t)|. Thus r (t) is just the distance from the origin 0 to the point. What are

447
448 CHAPTER 24. MOVING COORDINATE SYSTEMS

the velocity and acceleration in terms of er and eθ ? Using the chain rule,
der der ′ deθ deθ ′
= θ (t) , = θ (t)
dt dθ dt dθ
and so from 24.1,
der deθ
= θ ′ (t) eθ , = −θ ′ (t) er (24.2)
dt dt
Using 24.2 as needed along with the product rule and the chain rule,
d
r ′ (t) = r′ (t) er + r (t) (er (θ (t)))
dt
= r′ (t) er + r (t) θ ′ (t) eθ .
Next consider the acceleration.
der d
r ′′ (t) = r′′ (t) er + r′ (t) + r′ (t) θ ′ (t) eθ + r (t) θ ′′ (t) eθ + r (t) θ ′ (t) (eθ )
dt dt
= r′′ (t) er + 2r′ (t) θ ′ (t) eθ + r (t) θ ′′ (t) eθ + r (t) θ ′ (t) (−er ) θ ′ (t)
 
r′′ (t) − r (t) θ ′ (t)2 er + 2r′ (t) θ ′ (t) + r (t) θ ′′ (t) eθ .


= (24.3)

This is a very profound formula. Consider the following examples.

Example 24.1.1 Suppose an object of mass m moves at a uniform speed v, around a circle
of radius R. Find the force acting on the object.
By Newton’s second law, the force acting on the object is mr ′′ . In this case, r (t) = R, a
constant and since the speed is constant, θ ′′ = 0. Therefore, the term in 24.3 corresponding
to eθ equals zero and mr ′′ = −Rθ ′ (t)2 er . The speed of the object is v and so it moves v/R
radians in unit time. Thus θ ′ (t) = v/R and so
 v 2 v2
mr ′′ = −mR er = −m er .
R R
This is the familiar formula for centripetal force from elementary physics, obtained as a
very special case of 24.3.
Example 24.1.2 A platform rotates at a constant speed in the counter clockwise direction
and an object of mass m moves from the center of the platform toward the edge at constant
speed along a line fixed in the rotating platform. What forces act on this object?
Let v denote the constant speed of the object moving toward the edge of the platform.
Then
r′ (t) = v, r′′ (t) = 0, θ ′′ (t) = 0,
while θ ′ (t) = ω, a positive constant. From 24.3
mr ′′ (t) = −mr (t) ω 2 er + m2vωeθ .
Thus the object experiences centripetal force from the first term and also a funny force from
the second term which is in the direction of rotation of the platform. You can observe this
by experiment if you like. Go to a playground and have someone spin one of those merry
go rounds while you ride it and move from the center toward the edge. The term 2mvωeθ
is called the Coriolis force.
24.2. PLANETARY MOTION 449

24.2 Planetary Motion

Suppose at each point of space, r is associated a force F (r) which a given object of mass m
will experience if its position vector is r. This is called a force field. a force field is a central
force field if F (r) = g (r) er . Thus in a central force field the force an object experiences
will always be directed toward or away from the origin, 0. The following simple lemma is
very interesting because it says that in a central force field objects must move in a plane.
Lemma 24.2.1 Suppose an object moves in three dimensions in such a way that the only
force acting on the object is a central force. Then the motion of the object is in a plane.
Proof: Let r (t) denote the position vector of the object. Then from the definition of a
central force and Newton’s second law,
mr ′′ = g (r) r.
Therefore,
′
mr ′′ × r = m r ′ × r = g (r) r × r + mr ′ × r ′ = 0 .
Therefore, (r ′ × r) = n, a constant vector and so r · n = r· (r ′ × r) = 0 showing that n is
a normal vector to a plane which contains r (t) for all t. ■
Kepler’s laws of planetary motion state, among other things, that planets move around
the sun along an ellipse. These laws, discovered by Kepler, were shown by Newton to be
consequences of his law of gravitation which states that the force acting on a mass m by a
mass M is given by
   
1 1
F = −GMm 3 r = −GMm 2 er
r r
where r is the distance between centers of mass and r is the position vector from M to
m. Here G is the gravitation constant. This is called an inverse square law. Gravity acts
according to this law and so does electrostatic force. The constant G, is very small when
usual units are used and it has been computed using a very delicate experiment. It is now
accepted to be
6.67 × 10−11 Newton meter2 /kilogram2 .
The experiment involved a light source shining on a mirror attached to a fiber from
which was suspended a long rod with two solid balls of equal mas at the ends which were
attracted by two larger masses. The gravitation force between the suspended balls and the
two large balls caused the fibre to twist ever so slightly and this twisting was measured
by observing the deflection of the light reflected from the mirror on a scale placed some
distance from the fibre. Part of the experiment must compute the necessary spring constant
of the fibre.
This constant was first measured successfully by Cavendish in 1798 in the manner
just described. The accelerations are extremely small so it took months to complete the
experiment. Also, the entire apparatus had to be shielded from any currents of air which
would of course render the results worthless. The measurement has been made repeatedly.
You should also note that it also depends on being able to show that the entire force can be
considered as acting between the centers of mass of the respective balls. However, this was
shown by Newton. If you have spherical coordinates which are curvilinear coordinates in
three dimensions, this is not too hard, but none of this was invented in Newton’s time.
In the following argument, M is the mass of the sun and m is the mass of the planet. (It
could also be a comet or an asteroid.)
450 CHAPTER 24. MOVING COORDINATE SYSTEMS

24.2.1 The Equal Area Rule, Kepler’s Second Law

An object moves in three dimensions in such a way that the only force acting on the object
is a central force. Then the object moves in a plane and the radius vector from the origin to
the object sweeps out area at a constant rate. This is the equal area rule. In the context of
planetary motion it is called Kepler’s second law.
Lemma 24.2.1 says the object moves in a plane. From the assumption that the force
field is a central force field, it follows from 24.3 that
2r′ (t) θ ′ (t) + r (t) θ ′′ (t) = 0
Multiply both sides of this equation by r. This yields

′
2rr′ θ ′ + r2 θ ′′ = r2 θ ′ = 0. (24.4)
Consequently,
r2 θ ′ = c (24.5)
for some constant C. Now consider the following picture.

dθ

In this picture, dθ is the indicated angle and the two lines determining this angle are
position vectors for the object at point t and point t + dt. The area of the sector, dA, is
essentially r2 dθ and so dA = 21 r2 dθ . Therefore,
dA 1 2 dθ c
= r = . (24.6)
dt 2 dt 2

24.2.2 Inverse Square Law, Kepler’s First Law

Consider the first of Kepler’s laws, the one which states that planets move along ellipses.
From Lemma 24.2.1, the motion is in a plane. Now from 24.3 and Newton’s second law,
 
r′′ (t) − r (t) θ ′ (t)2 er + 2r′ (t) θ ′ (t) + r (t) θ ′′ (t) eθ


   
GMm 1 1
= − er = −k 2 er
m r2 r
Thus k = GM and

1
r (t) − r (t) θ (t) = −k 2 , 2r′ (t) θ ′ (t) + r (t) θ ′′ (t) = 0.
′′ ′ 2
(24.7)
r

′
As in 24.4, r2 θ ′ = 0 and so there exists a constant c, such that

r2 θ ′ = c. (24.8)
24.2. PLANETARY MOTION 451

Now the other part of 24.7 and 24.8 implies

c2
   
′′ ′ 2 ′′ 1
r (t) − r (t) θ (t) = r (t) − r (t) 4 = −k 2 . (24.9)
r r

It is only r as a function of θ which is of interest. Using the chain rule,

dr dθ dr  c 
r′ = = (24.10)
dθ dt dθ r2
and so also
d2r
 
′′ dθ  c  dr
dr dθ
r = + (−2) (c) r−3
dθ 2 dt r2 dθ dθ dt
2
 2  2 
d r c 2
  dr c
= 2 r2
−2 (24.11)
dθ dθ r5

Using 24.11 and 24.10 in 24.9 yields

 2  2 
d 2 r  c 2
 2  
dr c c 1
2 r2
− 2 5
− r (t) 4
= −k .
dθ dθ r r r2

Now multiply both sides of this equation by r4 /c2 to obtain

 2
d2r dr 1 −kr2
− 2 − r = . (24.12)
dθ 2 dθ r c2

This is a nice differential equation for r as a function of θ but its solution is not clear. It
turns out to be convenient to define a new dependent variable, ρ ≡ r−1 so r = ρ −1 . Then
2
dρ d 2 r d2ρ

dr dρ
= (−1) ρ −2 , = 2ρ −3 + (−1) ρ −2 .
dθ dθ dθ 2 dθ dθ 2
Substituting this in to 24.12 yields
2 2
d2ρ −kρ −2
 
dρ −2 dρ
2ρ −3 + (−1) ρ −2 − 2 ρ ρ − ρ −1
=
dθ dθ 2 dθ c2

which simplifies to
d2ρ −kρ −2
2
(−1) ρ −2
− ρ −1 =
dθ c2
 2
dρ
since those two terms which involve dθ cancel. Now multiply both sides by −ρ 2 and
this yields
d2ρ k
2
+ρ = 2, (24.13)
dθ c
k
which is a much nicer differential equation. Let R = ρ − c2
. Then in terms of R, this
differential equation is
d2R
+ R = 0.
dθ 2
452 CHAPTER 24. MOVING COORDINATE SYSTEMS

dR
Multiply both sides by dθ . Then using the chain rule,
 2 !
1 d dR 2
+R = 0
2 dθ dθ

and so  2
dR
+ R2 = δ 2 (24.14)
dθ
for some δ > 0. Therefore, there exists an angle ψ = ψ (θ ) such that

dR
R = δ sin (ψ) , = δ cos (ψ)
dθ
1 dR 1


because 24.14 says δ dθ , δ R is a point on the unit circle. But differentiating, the first of
the above equations,
dR dψ
= δ cos (ψ) = δ cos (ψ)
dθ dθ
and so dψ
dθ = 1. Therefore, ψ = θ + φ . Choosing the coordinate system appropriately, you
can assume φ = 0. Therefore,

k 1 k
R=ρ− = − 2 = δ sin (θ )
c2 r c
and so, solving for r,

1 c2 /k pε
r=   = 2
=
k
+ δ sin θ 1 + (c /k) δ sin θ 1 + ε sin θ
c2

where
ε = c2 /k δ and p = c2 /kε.


(24.15)
Here all these constants are nonnegative.
Thus
r + εr sin θ = ε p
and so r = (ε p − εy). Then squaring both sides,

x2 + y2 = (ε p − εy)2 = ε 2 p2 − 2pε 2 y + ε 2 y2

And so
x2 + 1 − ε 2 y2 = ε 2 p2 − 2pε 2 y.


(24.16)
In case ε = 1, this reduces to the equation of a parabola. If ε < 1, this reduces to the
equation of an ellipse and if ε > 1, this is called a hyperbola. This proves that objects
which are acted on only by a force of the form given in the above example move along
hyperbolas, ellipses or circles. The case where ε = 0 corresponds to a circle. The constant
ε is called the eccentricity. This is called Kepler’s first law in the case of a planet.
24.2. PLANETARY MOTION 453

24.2.3 Kepler’s Third Law

Kepler’s third law involves the time it takes for the planet to orbit the sun. From 24.16 you
can complete the square and obtain
2
pε 2 p2 ε 4 ε 2 p2

2 2 2 2


x + 1−ε y+ = ε p + = ,
1 − ε2 (1 − ε 2 ) (1 − ε 2 )

and this yields

2 !
ε 2 p2 pε 2 ε 2 p2
  
2
x / + y+ / = 1. (24.17)
1 − ε2 1 − ε2 (1 − ε 2 )2

Now note this is the equation of an ellipse and that the diameter of this ellipse is
2ε p
≡ 2a. (24.18)
(1 − ε 2 )

This follows because

ε 2 p2 ε 2 p2
≥ .
(1 − ε 2 )2 1 − ε2
Now let T denote the time it takes for the planet to make one revolution about the sun. It
is left as an exercise for you to show that the area of an ellipse whose long axis is 2a and
whose short axis is 2b is πab. This is an exercise in trig. substitutions and is a little tedious
but routine. Using this formula, and 24.6 the following equation must hold.
area of ellipse
z }| {
εp εp c
π√ 2)
=T
1−ε 2 (1 − ε 2

Therefore,
2 πε 2 p2
T=
c (1 − ε 2 )3/2
and so
4π 2 ε 4 p4
T2 =
c2 (1 − ε 2 )3
Now using 24.15, recalling that k = GM, and 24.18,

4π 2 ε 4 p4 4π 2 (ε p)3 4π 2 a3 4π 2 a3
T2 = = = = .
kε p (1 − ε 2 )3 k (1 − ε 2 )3 k GM

Written more memorably, this has shown

3
4π 2

2 diameter of ellipse
T = . (24.19)
GM 2

This relationship is known as Kepler’s third law.

454 CHAPTER 24. MOVING COORDINATE SYSTEMS

24.3 The Angular Velocity Vector

Let (i (t) , j (t) , k (t)) be a right handed system of unit basis vectors. Thus k (t) = i (t) ×
j (t) and each vector has unit length. This represents a moving coordinate system. We
assume that i (t) , j (t) , k (t) are each continuous having continuous derivatives, as many as
needed for the following manipulations for t in some open interval. The various rules of
differentiation of vector valued functions will be used to show the existence of an angular
velocity vector.

Lemma 24.3.1 The following hold. Whenever r (t) , s (t) are two vectors from
{i (t) , j (t) , k (t)} ,
r (t) · s′ (t) = −r ′ (t) · s (t)
In particular, the case where r = s, implies r ′ (t) · r (t) = 0.

Proof: By assumption, r (t) · s (t) is either 0 for all t or 1 in case r = s. Therefore,

from the product rule,
r (t) · s′ (t) + r ′ (t) · s (t) = 0
which yields the desired result. ■
Then the fundamental result is the following major theorem which gives the existence
and uniqueness of the angular velocity vector.

Theorem 24.3.2 Let (i (t) , j (t) , k (t)) be a right handed orthogonal system of unit vectors
as explained above. Then there exists a unique vector Ω (t) , the angular velocity vector,
such that for r (t) any of the {i (t) , j (t) , k (t)} ,

r ′ (t) = Ω (t) × r (t)

Proof: First I will show that if this angular velocity vector Ω (t) exists, then it must
be of a certain form. This will prove uniqueness. After showing this, I will verify that it
does what it needs to do by simply checking that it does so. In all considerations, recall
that in the box product, the × and · can be switched. I will use this fact with no comment
in what follows. So suppose that such an angular velocity vector exists. Then i′ (t) =
Ω (t) × i (t) with a similar formula holding for the other vectors. Also note that since this
is a right handed system, i (t) × j (t) = k (t) , j (t) × k (t) = i (t) , and k (t) × i (t) = j (t) as
earlier. In addition, if you want the component of a vector v with respect to some r (t), it
is v · r (t) = vr (t). Thus

v = vi i (t) + v j j (t) + vk k (t) , vr = v · r (t) for each r (t) ∈ {i (t) , j (t) , k (t)}

Then
i′ (t) · j (t) = Ω (t) · i (t) × j (t) = Ω (t) · k (t) = Ω (t)k
Thus the component of Ω (t) in the direction k (t) is determined. Next,

i′ (t) · k (t) = Ω (t) · i (t) × k (t) = −Ω j (t)

and so the component in the direction j (t) is also determined. Next,

j ′ (t) · k (t) = Ω (t) × j (t) · k (t) = Ω (t) · (j (t) × k (t)) = Ωi (t)

24.4. ANGULAR VELOCITY VECTOR ON EARTH 455

so the component of Ω (t) in direction i (t) is determined. Thus, if there is such an angular
velocity vector, it must be of the form

Ω (t) ≡ j ′ (t) · k (t) i (t) − i′ (t) · k (t) j (t) + i′ (t) · j (t) k (t)

It only remains to verify that this vector works. Recall Lemma 24.3.1 which will be
used without comment in what follows. Does the above Ω (t) work?

Ω (t) × i (t) = i′ (t) · k (t) k (t)

 
=0
z }| {
+ i′ (t) · j (t) j (t) + i′ (t) · i (t) i (t)

 

= i′ (t)

j ′ (t) · k (t) k (t) + i′ (t) · j (t) (−i (t))

Ω (t) × j (t) =
j ′ (t) · k (t) k (t) + i (t) · j ′ (t) (i (t))



=
= j ′ (t)

and finally,

j ′ (t) · k (t) (−j (t)) − i′ (t) · k (t) i (t)

Ω (t) × k (t) =
j (t) · k′ (t) (j (t)) + i (t) · k′ (t) i (t)



=
= k′ (t)

Thus, this Ω (t) is the angular velocity vector and there is only one. Of course it might have
different descriptions but there can only be one and it is the vector just described. ■
This implies the following simple corollary.

Corollary 24.3.3 Let u (t) be a vector such that its components with respect to the basis
vectors i (t) , j (t) , k (t) are constant. Then u′ (t) = Ω (t) × u (t).

Proof: Say u (t) = ui i (t) + u j j (t) + uk k (t) . Then

u′ (t) = ui i′ (t) + u j j ′ (t) + uk k′ (t)

= ui Ω (t) × i (t) + u j Ω (t) × j (t)
+uk Ω (t) × k (t)

= Ω (t) × (ui i (t) + u j j (t) + uk k (t))

= Ω (t) × u (t) ■

24.4 Angular Velocity Vector on Earth

So how do you find the angular velocity vector? One way is to use the formula shown
above. However, in important cases, this angular velocity vector can be determined from
simple geometric reasoning. An obvious example concerns motion on the surface of the
456 CHAPTER 24. MOVING COORDINATE SYSTEMS

earth. Imagine you have a coordinate system fixed with the earth. Then it is actually
rotating through space because the earth is turning. However, to an observer on the surface
of the earth, these vectors are not moving and this observer wants to understand motion
in terms of these apparently fixed vectors. This is a very interesting problem which can
be understood relative to what was just discussed. In this, the motion of the earth through
space around the sun is not being considered because forces resulting from this motion are
negligible.
Imagine a point on the surface of the earth which is not moving relative to the earth.
Now consider unit vectors, one pointing South, one pointing East and one pointing directly
away from the center of the earth.

k
j

Denote the first as i (t), the second as j (t) , and the third as k (t). If you are standing on
the earth you will consider these vectors as fixed, but of course they are not. As the earth
turns, they change direction and so each is in reality a function of t. What is the description
of the angular velocity vector in this situation?
Let i∗ , j ∗ , k∗ , be the usual basis vectors fixed in space with k∗ pointing in the direction
of the north pole from the center of the earth and let i (t) , j (t) , k (t) be the unit vectors
described earlier with i (t) pointing South, j (t) pointing East, and k (t) pointing away
from the center of the earth at some point of the rotating earth’s surface p (t). (This means
that the components of p (t) are constant with respect to the vectors fixed with the earth. )
Letting R (t) be the position vector of the point p (t) , from the center of the earth, observe
that this is a typical vector having coordinates constant with respect to i (t) , j (t) , k (t) .
Also, since the earth rotates from West to East and the speed of a point on the surface of
the earth relative to an observer fixed in space is ω |R| sin φ where ω is the angular speed
of the earth about an axis through the poles and φ is the polar angle measured from the
positive z axis down as in spherical coordinates. It follows from the geometric definition of
the cross product that
R′ (t) = ωk∗ × R (t)
Therefore, the vector of Theorem 24.3.2 is Ω (t) = ωk∗ because it acts like it should for
vectors having components constant with respect to the vectors fixed with the earth. As
mentioned, you could let θ , ρ, φ each be a function of t and use the formula above along
with the chain rule to verify analytically that the angular velocity vector is what is claimed
above. That is, you would have θ (t) = ωt and the other spherical coordinates constant.
See Problem 12 on Page 464 below for a more analytical explanation.
24.5. CORIOLIS FORCE AND CENTRIPETAL FORCE 457

24.5 Coriolis Force and Centripetal Force

Let p (t) be a point which has constant components relative to the moving coordinate sys-
tem described above {i (t) , j (t) , k (t)}. For example, it could be a single point on the
rotating earth. Letting i∗ , j ∗ , k∗ be a typical rectangular coordinate system fixed in space
and let R (t) be the position vector of p (t) from the origin fixed in space. In the case of
the earth, think of the origin as the center of the earth. Thus the components of R (t) with
respect to the moving coordinate system are constants. Let r B (t) be the position vector
from this point p (t) to some other point.

r B (t) ≡ x (t) i (t) + y (t) j (t) + z (t) k (t)

The acceleration perceived by an observer moving with the moving coordinate system
would then be

r ′′B (t) ≡ aB (t) = x′′ (t) i (t) + y′′ (t) j (t) + z′′ (t) k (t)

and the perceived velocity would be r ′B (t) ≡ v B (t).

v B (t) ≡ x′ (t) i (t) + y′ (t) j (t) + z′ (t) k (t)

Let r (t) ≡ R (t)+r B (t) . Then, since R (t) has constant components relative to the moving
coordinate system,

v (t) = R′ (t) + r ′B (t) , r ′B (t) = v B (t) + x (t) i′ (t) + y (t) j ′ (t) + z (t) k′ (t)

= v B (t) + x (t) (Ω (t) ×i (t)) + y (t) (Ω (t) ×j (t)) + z (t) (Ω (t) ×k (t))
and so, from the last equation for r ′B (t) ,

v (t) = v B (t) + Ω (t) × r B (t) + Ω (t) × R (t)

= v B (t) + Ω (t) × r (t)

Now take a further derivative to find the total acceleration. Using what was just shown, it
equals

d 2 rB
a (t) = R′′ (t) + (t) = aB (t) + Ω (t) × v B (t) + Ω′ (t) × r (t) + Ω (t) × v (t)
dt 2
= aB (t) + (Ω (t) × v B (t)) + Ω′ (t) × r (t) + Ω (t) × (v B (t) + Ω (t) × r (t))

= aB (t) + 2 (Ω (t) × v B (t)) + Ω′ (t) × r (t) + Ω (t) × (Ω (t) × r (t))

= aB (t) + 2 (Ω (t) × v B (t)) + Ω′ (t) × r (t) + Ω (t) × (Ω (t) × r B (t))

+Ω (t) × (Ω (t) × R (t)) (24.20)

= aB (t) + 2 (Ω (t) × v B (t)) + Ω′ (t) × r B (t) + Ω (t) × (Ω (t) × r B (t))

+Ω (t) × R′ (t) + Ω′ (t) × R (t)

458 CHAPTER 24. MOVING COORDINATE SYSTEMS

= aB (t) + 2 (Ω (t) × v B (t)) + Ω′ (t) × r B (t)

d
+Ω (t) × (Ω (t) × r B (t)) + (Ω (t) × R (t))
dt
= aB (t) + 2 (Ω (t) × v B (t)) + Ω′ (t) × r B (t)

+Ω (t) × (Ω (t) × r B (t)) + R′′ (t)

Therefore,
d 2 rB
(t) = aB (t) + 2 (Ω (t) × v B (t)) + Ω′ (t) × r B (t) + Ω (t) × (Ω (t) × r B (t))


dt 2

where recall that aB (t) is the perceived acceleration relative to the moving coordinate sys-
tem. Solving for this yields

d 2 rB
(t) − 2 (Ω (t) × v B (t)) + Ω′ (t) × r B (t) + Ω (t) × (Ω (t) × r B (t)) = aB (t)



dt 2

The part of the acceleration on the left depending on the relative velocity is called the
Coriolis acceleration. The rest of it is sometimes called centrifugal acceleration. It is felt
by the observer by regarding the moving coordinates as fixed. On the earth, this force
is small enough to be neglected. However, when v B is large, one can get a significant
contribution from the Coriolis force.

24.6 Coriolis Force on the Rotating Earth

As shown above, on the rotating earth, Ω is a constant and so 24.20 reduces to

a = aB (t) + 2 (Ω (t) × v B (t)) + Ω (t) × (Ω (t) × r (t)) (24.21)

Since r B + R = r,

aB = a − Ω× (Ω × R) − 2Ω × v B − Ω× (Ω × r B ) . (24.22)

In this formula, you can totally ignore the term Ω× (Ω × r B ) because it is so small when-
ever you are considering motion near some point on the earth’s surface. To see this, note
seconds in a day
z }| {
ω (24) (3600) = 2π, and so ω = 7.2722 × 10−5 in radians per second. If you are using
seconds to measure time and feet to measure distance, this term is therefore, no larger than
 2
7.2722 × 10−5 |r B | .

Clearly this is not worth considering in the presence of the acceleration due to gravity which
is approximately 32 feet per second squared near the surface of the earth.
If the acceleration a is due to gravity, then

aB = a − Ω× (Ω × R) − 2Ω × v B =
≡g
z }| {
GM (R + r B )
− − Ω× (Ω × R) − 2Ω × v B ≡ g − 2Ω × v B .
|R + r B |3
24.6. CORIOLIS FORCE ON THE ROTATING EARTH 459

Note that
Ω× (Ω × R) = (Ω · R) Ω− |Ω|2 R
and so g, the acceleration relative to the moving coordinate system on the earth is not di-
rected exactly toward the center of the earth except at the poles and at the equator, although
the components of acceleration which are in other directions are very small when com-
pared with the acceleration due to the force of gravity and are often neglected. Therefore,
if the only force acting on an object is due to gravity, the following formula describes the
acceleration relative to a coordinate system moving with the earth’s surface.
aB = g−2 (Ω × v B )
While the vector Ω is quite small, if the relative velocity, v B is large, the Coriolis acceler-
ation could be significant. This is described in terms of the vectors i (t) , j (t) , k (t) next.
Letting (ρ, θ , φ ) be the usual spherical coordinates of the point p (t) on the surface
taken with respect to i∗ , j ∗ , k∗ the usual way with φ the polar angle, it follows the i∗ , j ∗ , k∗
coordinates of this point are  
ρ sin (φ ) cos (θ )
 ρ sin (φ ) sin (θ )  .
 

ρ cos (φ )
It follows,
i = cos (φ ) cos (θ ) i∗ + cos (φ ) sin (θ ) j ∗ − sin (φ ) k∗
j = − sin (θ ) i∗ + cos (θ ) j ∗ + 0k∗
and
k = sin (φ ) cos (θ ) i∗ + sin (φ ) sin (θ ) j ∗ + cos (φ ) k∗ .
It is necessary to obtain k∗ in terms of the vectors, i (t) , j (t) , k (t) because, as shown
earlier, ωk∗ is the angular velocity vector Ω. To simplify notation, I will suppress the
dependence of these vectors on t. Thus the following equation needs to be solved for a, b, c
to find k∗ = a i+ b j+ ck
k∗ i j k
z }| {
 z }| { z }| { z
 }| {
0 cos (φ ) cos (θ ) − sin (θ ) sin (φ ) cos (θ )
 0  = a cos (φ ) sin (θ )  + b cos (θ )  + c sin (φ ) sin (θ )  (24.23)
       

1 − sin (φ ) 0 cos (φ )

The solution is a = − sin (φ ) , b = 0, and c = cos (φ ) .

Now the Coriolis acceleration on the earth equals
k∗
 
z }| {
2 (Ω × v B ) = 2ω − sin (φ ) i+ 0 j+ cos (φ ) k × x′ i+ y′ j+ z′ k .
 

This equals
−y′ cos φ i+ x′ cos φ + z′ sin φ j − y′ sin φ k .




2ω (24.24)
Remember φ is fixed and pertains to the fixed point, p (t) on the earth’s surface. Therefore,
if the acceleration a is due to gravity,
aB = g−2ω −y′ cos φ i+ x′ cos φ + z′ sin φ j − y′ sin φ k




460 CHAPTER 24. MOVING COORDINATE SYSTEMS

where g = − GM(R+r3B ) −Ω× (Ω × R) as explained above. The term Ω× (Ω × R) is pretty

|R+r B |
small and so it will be neglected. However, the Coriolis force will not be neglected.

Example 24.6.1 Suppose a rock is dropped from a tall building. Where will it strike?

Assume a = −gk and the j component of aB is approximately

−2ω x′ cos φ + z′ sin φ .

The dominant term in this expression is clearly the second one because x′ will be small.
Also, the i and k contributions will be very small. Therefore, the following equation is
descriptive of the situation.
aB = −g k− 2z′ ω sin φ j.
z′ = −gt approximately. Therefore, considering the j component, this is

2gtω sin φ .

Two integrations give ωgt 3 /3 sin φ for the j component of the relative displacement at

time t.
This shows the rock does not fall directly towards the center of the earth as expected
but slightly to the east.

24.7 The Foucault Pendulum∗

In 1851 Foucault set a pendulum vibrating and observed the earth rotate out from under it.
It was a very long pendulum with a heavy weight at the end so that it would vibrate for a
long time without stopping1 . This is what allowed him to observe the earth rotate out from
under it. Clearly such a pendulum will take 24 hours for the plane of vibration to appear to
make one complete revolution at the north pole. It is also reasonable to expect that no such
observed rotation would take place on the equator. Is it possible to predict what will take
place at various latitudes?
Using 24.24, in 24.22,
aB = a − Ω× (Ω × R)
−2ω −y′ cos φ i+ x′ cos φ + z′ sin φ j − y′ sin φ k .





Neglecting the small term, Ω× (Ω × R) , this becomes

= −gk + T /m − 2ω −y′ cos φ i+ x′ cos φ + z′ sin φ j − y′ sin φ k






where T , the tension in the string of the pendulum, is directed towards the point at which
the pendulum is supported, and m is the mass of the weight at the end of the pendulum.
The pendulum can be thought of as the position vector from (0, 0, l) to the surface of the
sphere x2 + y2 + (z − l)2 = l 2 . Therefore,
x y l −z
T = −T i− T j+ T k
l l l
1 There is such a pendulum in the Eyring building at BYU and to keep people from touching it, there is a little

sign which says Warning! 1000 ohms. You certainly don’t want to encounter too many ohms! Most modern
Foucault pendulums have a mechanism which applies a periodic force to keep it vibrating.
24.7. THE FOUCAULT PENDULUM∗ 461

and consequently, the differential equations of relative motion are

x
x′′ = −T + 2ωy′ cos φ
ml
y
y′′ = −T − 2ω x′ cos φ + z′ sin φ


ml
and
l −z
z′′ = T − g + 2ωy′ sin φ .
ml
If the vibrations of the pendulum are small so that for practical purposes, z′′ = z = 0, the
last equation may be solved for T to get

gm − 2ωy′ sin (φ ) m = T.

Therefore, the first two equations become

x
x′′ = − gm − 2ωmy′ sin φ + 2ωy′ cos φ
ml
and
y
y′′ = − gm − 2ωmy′ sin φ − 2ω x′ cos φ + z′ sin φ .


ml
All terms of the form xy′ or y′ y can be neglected because it is assumed x and y remain small.
Also, the pendulum is assumed to be long with a heavy weight so that x′ and y′ are also
small. With these simplifying assumptions, the equations of motion become
x
x′′ + g = 2ωy′ cos φ
l
and
y
y′′ + g = −2ωx′ cos φ .
l
These equations are of the form

x′′ + a2 x = by′ , y′′ + a2 y = −bx′ (24.25)

where a2 = gl and b = 2ω cos φ . There are systematic ways to solve the above linear system
of ordinary differential equations, but for the purposes here, it is fairly tedious but routine
to verify that for each constant c,
√ ! √ !
b2 + 4a2 b2 + 4a2
   
bt bt
x = c sin sin t , y = c cos sin t (24.26)
2 2 2 2

yields a solution to 24.25 along with the initial conditions,

√
′ ′ c b2 + 4a2
x (0) = 0, y (0) = 0, x (0) = 0, y (0) = . (24.27)
2
It is clear from experiments with the pendulum that the earth does indeed rotate out from
under it causing the plane of vibration of the pendulum to appear to rotate. The purpose
of this discussion is not to establish this obvious fact but to predict how long it takes for
the plane of vibration to make one revolution. There will be some instant in time at which
462 CHAPTER 24. MOVING COORDINATE SYSTEMS

the pendulum will be vibrating in a plane determined by k and j. (Recall k points away
from the center of the earth and j points East. ) At this instant in time, defined as t = 0,
the conditions of 24.27 will hold for some value of c and so the solution to 24.25 having
these initial conditions will be those of 24.26. (Some interesting mathematical details are
being ignored here. Such initial value problems as 24.26 and 24.27 have only one solution
so if you have found one, then you have found the solution. This is a general fact shown in
differential equations courses. However, for the above system of equations see Problem 13
on Page 464 found below.) Writing these solutions differently,
!
! √ !
x (t) sin bt2 b2 + 4a2
=c
sin t
y (t) cos bt2 2

!
sin bt2
This is very interesting! The vector, c always has magnitude equal to |c| but
cos bt2

its direction changes very slowly because b is very small.

 √ The plane
 of vibration is deter-
b2 +4a2
mined by this vector and the vector k. The term sin 2 t changes relatively fast
and takes values between −1 and 1. This is what describes the actual observed vibrations
of the pendulum. Thus the plane of vibration will have made one complete revolution when
t = T for
bT
≡ 2π.
2
Therefore, the time it takes for the earth to turn out from under the pendulum is
4π 2π
T= = sec φ .
2ω cos φ ω
2π π
Since ω is the angular speed of the rotating earth, it follows ω = 24 = 12 in radians per
hour. Therefore, the above formula implies

T = 24 sec φ .

I think this is really amazing. You could determine latitude, not by taking readings with
instruments using the North star but by doing an experiment with a big pendulum. You
would set it vibrating, observe T in hours, and then solve the above equation for φ . Also
note the pendulum would not appear to change its plane of vibration at the equator because
limφ →π/2 sec φ = ∞.

24.8 Exercises
1. Find the length of the cardioid, r = 1 + cos θ , θ ∈ [0, 2π]. Hint: A parametrization
is x (θ ) = (1 + cos θ ) cos θ , y (θ ) = (1 + cos θ ) sin θ .
2. In general, show that theqlength of the curve given in polar coordinates by r =
f (θ ) , θ ∈ [a, b] equals ab f ′ (θ )2 + f (θ )2 dθ .
R

3. Using the above problem, find the lengths of graphs of the following polar curves.

(a) r = θ , θ ∈ [0, 3]
24.8. EXERCISES 463

(b) r = 2 cos θ , θ ∈ [−π/2, π/2]

(c) r = 1 + sin θ , θ ∈ [0, π/4]
(d) r = eθ , θ ∈ [0, 2]
(e) r = θ + 1, θ ∈ [0, 1]

4. Suppose the curve given in polar coordinates by r = f (θ ) for θ ∈ [a, b] is rotated

about the y axis. Find a formula for the resulting surface of revolution. You should
get
Z b q
2π f (θ ) cos (θ ) f ′ (θ )2 + f (θ )2 dθ
a

5. Using the result of the above problem, find the area of the surfaces obtained by
revolving the polar graphs about the y axis.

(a) r = θ sec (θ ) , θ ∈ [0, 2]

(b) r = 2 cos θ , θ ∈ [−π/2, π/2]
(c) r = eθ , θ ∈ [0, 2]
(d) r = (1 + θ ) sec (θ ) , θ ∈ [0, 1]

6. Suppose an object moves in such a way that r2 θ ′ is a constant. Show that the only
force acting on the object is a central force.
7. Explain why low pressure areas rotate counter clockwise in the Northern hemisphere
and clockwise in the Southern hemisphere. Hint: Note that from the point of view
of an observer fixed in space above the North pole, the low pressure area already
has a counter clockwise rotation because of the rotation of the earth and its spherical
shape. Now consider 24.5. In the low pressure area stuff will move toward the center
so r gets smaller. How are things different in the Southern hemisphere?
8. What are some physical assumptions which are made in the above derivation of Ke-
pler’s laws from Newton’s laws of motion?
9. The orbit of the earth is pretty nearly circular and the distance from the sun to the
earth is about 149 × 106 kilometers. Using 24.19 and the above value of the universal
gravitation constant, determine the mass of the sun. The earth goes around it in 365
days. (Actually it is 365.256 days.)
10. It is desired to place a satellite above the equator of the earth which will rotate about
the center of mass of the earth every 24 hours. Is it necessary that the orbit be
circular? What if you want the satellite to stay above the same point on the earth
at all times? If the orbit is to be circular and the satellite is to stay above the same
point, at what distance from the center of mass of the earth should the satellite be?
You may use that the mass of the earth is 5.98 × 1024 kilograms. Such a satellite is
called geosynchronous.
2
x2
11. Show directly that the area of the inside of an ellipse a2
+ by2 = 1 is πab. Hint: Solve
for y and consider the top half of the ellipse.
464 CHAPTER 24. MOVING COORDINATE SYSTEMS

12. Recall the formula derived above for the angular velocity vector

Ω (t) = j ′ (t) · k (t) i (t) − i′ (t) · k (t) j (t) + i′ (t) · j (t) k (t)

In the case of the rotating earth,

   
cos (ωt) cos φ − sin (ωt)
i (t) =  cos φ sin (ωt)  , j (t) =  cos (ωt)  ,
   

− sin φ 0
 
sin (φ ) cos (ωt)
k (t) =  sin (φ ) sin (ωt) 
 

cos (φ )

where column vectors are in terms of the fixed vectors i∗ , j ∗ , k∗ . Show directly that
Ω (t) = ωk∗ as claimed above.
13. Suppose you have
x′′ + a2 x = by′ , y′′ + a2 y = −bx′ (24.28)
and x (0) = x′ (0)
= y (0) = y′ (0)
= 0. Show that x (t) = y (t) = 0. Show this implies
there is only one solution to the initial value problem 24.26 and 24.27. Hint: If you
had two solutions to 24.26 and 24.27, x̃, ỹ and x̂, ŷ, consider x = x̂ − x̃ and y = ŷ − ỹ
and show x, y satisfies 24.28. To show the first part, multiply the first equation by x′
the second by y′ add and obtain the following using the product rule.

d  ′
2
2

x + y′ + a2 x2 + y2 = 0
dt
Thus the inside is a constant. From the initial condition, this constant can only be 0.
Chapter 25

Curvilinear Coordinates

25.1 Basis Vectors

In this chapter, I will use the repeated index summation convention unless stated otherwise.
Thus, a repeated index indicates a sum. Also, it is helpful in order to keep things straight
to always have the two repeated indices be on different levels. That is, I will write aij b j and
not ai j b j . The reason for this will become clear as the exposition proceeds.
The usual basis vectors are denoted by i, j, k and are as the following picture describes.
The vectors, i, j, k, are fixed. If v is a vector, there are unique scalars called
k
components such that v = v1 i+ v2 j+ v3 k . This is what it means that i, j, k is
i
j a basis. Review Section 9 at this time to see how this geometric notion relates
to the general concept of a basis in a vector space.
Now suppose e1 , e2 , e3 are three vectors which satisfy

e1 × e2 · e3 ̸= 0.

Recall this means the volume of the box spanned by the three vectors is not zero.
e3 Suppose e1 , e2 , e3 are as just described. Does it follow that they form
a basis? In other words, for any vector v, there are unique scalars vi such
e2 that v = vi ei . Of course this is the case because the box product is really
e1 the determinant of the matrix which has ei as the ith row (column). This is
the content of the following theorem.

Theorem 25.1.1 If e1 , e2 , e3 are three vectors, then they form a basis if

and only if
e1 × e2 · e3 ̸= 0.

This gives a simple geometric condition which determines whether a list of three vectors
forms a basis in R3 . One simply takes the box product. If the box product is not equal to
zero, then the vectors form a basis. If not, the list of three vectors does not form a basis.
p j p
 generalizes to R as follows. If ei = ai i j , then {ei }i=1 forms a basis if and
This condition
only if det aij ̸= 0.
These vectors may or may not be orthonormal. In any case, it is convenient to define
something called the dual basis.

465
466 CHAPTER 25. CURVILINEAR COORDINATES

p  p
Definition 25.1.2 Let {ei }i=1 form a basis for R p . Then ei i=1 is called the dual basis if
(
i i 1 if i = j
e ·ej = δ j ≡ . (25.1)
0 if i ̸= j
p  p
Theorem 25.1.3 If {ei }i=1 is a basis then ei i=1 is also a basis provided 25.1 holds.
Proof: Suppose
v = vi ei . (25.2)
Then taking the dot product of both sides of 25.2 with e j ,yields
v j = v · e j. (25.3)
Thus there is at most one choice of scalars v j such that v = v j e j and it is given by 25.3.
v − v · e j e j · ek = 0

p
and so, since {ei }i=1 is a basis,

v − v · e je j · w = 0

 p
for all vectors w. It follows v − v · e j e j = 0 and this shows ei i=1 is a basis. ■
In the above argument are obtained formulas for the components of a vector v, vi ,
with respect to the dual basis, found to be v j = v · e j . In the same way, one can find the
p
components of a vector with respect to the basis {ei }i=1 . Let v be any vector and let

v = v je j. (25.4)
Then taking the dot product of both sides of 25.4 with ei we see vi = ei · v.
Does there exist a dual basis and is it uniquely determined?
p p
is a basis for R p , then there exists a unique dual basis, e j j=1

Theorem 25.1.4 If {ei }i=1
satisfying
e j · ei = δ ij .
p
Proof: First I show the dual basis is unique. Suppose f j j=1 is another set of vectors


which satisfies f j · ei = δ ij . Then

f j = f j · ei ei = δ ij ei = e j .
 p
Note that from the definition, the dual basis to i j j=1 is just i j = i j . It remains to verify
the existence of the dual basis. Consider the matrix gi j ≡ ei · e j . This is called the metric
tensor. If the resulting matrix is denoted as G, does it follow that G−1 exists?  Suppose you
have ei · e j x j = 0. Then, since i is arbitrary, this implies e j x j = 0 and since e j is a basis,
this requires each x j to be zero. Thus G is invertible. Denote by gi j the i jth entry of this
inverse matrix. Consider e j ≡ g jk ek . Is this the dual basis as the notation implies?
e j · ei = g jk ek · ei = g jk gki = δ ij
so yes, it is indeed the dual basis. This has shown both existence and uniqueness of the
dual basis. ■
From this is a useful observation.
25.1. BASIS VECTORS 467
 
p
Proposition 25.1.5 {ei }i=1 is a basis for R p if and only if when ei = aij i j , det aij ̸= 0.

p
Proof: First suppose {ei }i=1 is a basis for R p . Letting Ai j ≡ aij , we need to show that
det (A) ̸= 0. This is equivalent to showing that A or AT is one to one. But

aij xi = 0 ⇒ aij xi i j = 0 ⇒ ei xi = 0 ⇒ xi = 0

so AT is one to one if and only if det (A) = det AT ̸= 0.

Conversely, suppose A has nonzero determinant. Why are the ek a basis? Suppose
xk ek = 0. Is each xk = 0? Then xk akj i j = 0 and so for each j, akj xk = 0 and since A has
nonzero determinant, xk = 0. ■
p
Summarizing what has been shown so far, we know that {ei }i=1 is a basis for R p if and
j
only if when ei = ai i j ,  
det aij ̸= 0. (25.5)
p p
is a basis, then there exists a unique dual basis, e j

If {ei }i=1 j=1
satisfying

e j · ei = δ ij , (25.6)

and that if v is any vector,

v = v je j, v = v je j. (25.7)
The components of v which have the index on the top are called the contravariant compo-
nents of the vector while the components which have the index on the bottom are called the
covariant components. In general vi ̸= v j ! We also have formulae for these components in
terms of the dot product.
v j = v · e j, v j = v · e j. (25.8)
As indicated above, define gi j ≡ ei · e j and gi j ≡ ei · e j . The next theorem describes the
process of raising or lowering an index.

Theorem 25.1.6 The following hold.

gi j e j = ei , gi j e j = ei , (25.9)

gi j v j = vi , gi j v j = vi , (25.10)
gi j g jk = δ ik , (25.11)
det (gi j ) > 0, det gi j > 0.


(25.12)

Proof: First,
ei = ei · e j e j = gi j e j
by 25.7 and 25.8. Similarly, by 25.7 and 25.8,

ei = ei · e j e j = gi j e j .

This verifies 25.9. To verify 25.10,

vi = ei · v = gi j e j · v = gi j v j .
468 CHAPTER 25. CURVILINEAR COORDINATES

The proof of the remaining formula in 25.10 is similar.

To verify 25.11,

gi j g jk = ei · e j e j · ek = ei · e j e j · ek = ei · ek = δ ik .

This shows the two determinants in 25.12 are non zero because the two matrices are in-
verses of each other. It only remains to verify that one of these is greater than zero. Letting
ei = aij i j = bij i j , we see that since i j = i j , aij = bij . Therefore,

ei · e j = ari ir · bkj ik = ari bkj δ kr = aki bkj = aki akj .

It follows that for G the matrix whose i jth entry is ei · e j , G = AAT where the ikth entry of
A is aki . Therefore, det (G) = det (A) det AT = det (A)2 > 0. It follows from 25.11 that if

H is the matrix whose i jth entry is gi j , then GH = I and so H = G−1 and

det (G) det G−1 = det gi j det (G) = 1.

Therefore, det G−1 >
0 also. ■

Note that det AAT ≥ 0 always, because the eigenvalues are nonnegative.
As noted above, we have the following definition.

Definition 25.1.7 The matrix (gi j ) = G is called the metric tensor.

25.2 Exercises
1. Let e1 = i + j, e2 = i − j, e3 = j + k. Find e1 , e2 , e3 , (gi j ) , gi j . If

v = i+2j + k, find vi and v j , the contravariant and covariant components of the

vector.
2. Let e1 = 2i + j, e2 = i − 2j, e3 = k. Find e1 , e2 , e3 , (gi j ) , gi j . If

v = 2 i− 2j + k, find vi and v j , the contravariant and covariant components of the

vector.
3. Suppose e1 , e2 , e3 have the property that ei · e j = 0 whenever i ̸= j. Show the same
is true of the dual basis and that in fact, ei is a multiple of ei .
4. Let e1 ,· · · , e3 be a basis for Rn and let v = vi ei = vi ei , w = w j e j = w j e j be two
vectors. Show
v · w = gi j vi w j = gi j vi w j .

5. Show if {ei }3i=1 is a basis in R3

e2 × e3 e1 × e2 e1 × e2
e1 = , e2 = , e3 = .
e2 × e3 · e1 e1 × e3 · e2 e1 × e2 · e3

6. Let {ei }ni=1 be a basis and define

ei
e∗i ≡ , e∗i ≡ ei |ei | .
|ei |

Show e∗i · e∗j = δ ij .

25.3. CURVILINEAR COORDINATES 469

7. If v is a vector, v∗i and v∗i , are defined by

v ≡ v∗i e∗i ≡ v∗i e∗i .

These are called the physical components of v. Show

vi
v∗i = , v∗i = vi |ei | ( No summation on i ).
|ei |

25.3 Curvilinear Coordinates

There are many ways to identify a point in n dimensional space with an ordered list of real
numbers. Some of these are spherical coordinates, cylindrical coordinates and rectangu-
lar coordinates and these particular examples are discussed earlier. I will denote by y the
a point in n dimensional space which I will go on writing as Rn .
rectangular coordinates of 
Thus y = y1 · · · yn . It follows there are equations which relate the rectangular co-
 
ordinates to some other coordinates x1 · · · xn . In spherical coordinates, these were
ρ, φ , θ where the geometric meaning of these were described earlier. However, completely
general systems are to be considered here, with certain stipulations. The idea is

yk = yk x1 , ..., xn , y = y x1 , ..., xn

 
Let x1 · · · xn ∈ D ⊆ Rn be an open set and let x → y x1 , ..., xn ≡ M x1 , ..., xn

satisfy
M is C2 , (25.13)
M is one to one. (25.14)
Letting x ∈ D, we can write
M (x) = M k (x) ik
where, as usual, ik are the standard basis vectors for Rn , ik being the vector in Rn which
has a one in the kth coordinate and a 0 in every other spot. Thus yk = M k (x) where this yk
refers to the kth rectangular coordinate of the point y as just described.
For a fixed x ∈ D, we can consider the space curves,

t → M (x + tik ) ≡ y (x + tik )

for t ∈ I, some open interval containing 0. Then for the point x,we let

∂M d ∂y
ek ≡ (x) ≡ (M (x + tik )) |t=0 ≡ k (x)
∂ xk dt ∂x
Denote this vector as ek (x) to emphasize its dependence on x. The following picture
illustrates the situation in R3 .
470 CHAPTER 25. CURVILINEAR COORDINATES

t → M (x01 ,t, x03 )

e3 e2 e1

t → M (t, x02 , x03 )

t → M (x01 , x02 ,t)

I want {ek }nk=1 to be a basis. Thus, from Proposition 25.1.5,

∂ Mi
 
det ≡ det (Dy (x)) ≡ det (D (M ) (x)) ̸= 0. (25.15)
∂ xk

Let
yi = M i (x) i = 1, · · · , n (25.16)
yi
so that the are the usual rectangular coordinates with respect to the usual basis vectors
{ik }nk=1 of the point y = M (x) . Letting x ≡ x1 , · · · , xn , it follows from the inverse

function theorem (See Chapter 26) that M (D) is open, and that 25.15, 25.13, and 25.14
imply the equations 25.16 define each xi as a C2 function of y ≡ y1 , · · · , yn . Thus, abusing

notation slightly, the equations 25.16 are equivalent to

xi = xi y1 , ..., yn , i = 1, · · · , n

where xi is a C2 function of the rectangular coordinates of a point y. It follows from the

material on the gradient described earlier,

∂ xk (y) j
∇xk (y) = i.
∂yj
Then
∂ x k s ∂ yr ∂ x k ∂ ys
∇xk (y) · e j = s
i · j ir = s j = δ kj
∂y ∂x ∂y ∂x
by the chain rule. Therefore, the dual basis is given by

ek (x) = ∇xk (y (x)) . (25.17)

Notice that it might be hard or even impossible to solve algebraically for xi in terms
of the y j . Thus the straight forward approach to finding ek by 25.17 might be impossible.
Also, this approach leads to an expression in terms of the y coordinates rather than the
desired x coordinates. Therefore, it is expedient to use another method to obtain these
vectors in terms of x. Indeed, this is the main idea in this chapter, doing everything in
terms of x rather than y. The vectors, ek (x) may always be found by using formula 25.9
and the result is in terms of the curvilinear coordinates x. Here is a familiar example.
25.3. CURVILINEAR COORDINATES 471

Example 25.3.1 D ≡ (0, ∞) × (0, π) × (0, 2π) and

  

y1 x1 sin x2 cos x3


 2   1
 y  =  x sin x2 sin x3 




y3 x1 cos x2

(We usually write this as

   
x ρ sin (φ ) cos (θ )
 y  =  ρ sin (φ ) sin (θ ) 
   

z ρ cos (φ )

where (ρ, φ , θ ) are the spherical coordinates. We are calling them x1 , x2 , and x3 to preserve
the notation just discussed.) Thus

e1 (x) = sin x2 cos x3 i1 + sin x2 sin x3 i2 + cos x2 i3 ,

e2 (x) = x1 cos x2 cos x3 i1

+x1 cos x2 sin x3 i2 − x1 sin x2 i3 ,

e3 (x) = −x1 sin x2 sin x3 i1 + x1 sin x2 cos x3 i2 + 0i3 .

It follows the metric tensor is

 
1 0 0

2
G =  0 x1 0  = (gi j ) = (ei · e j ) . (25.18)
 
1

2 2 2


0 0 x sin x

Therefore, by Theorem 25.1.6

G−1 = gi j

 
1 0 0
i j 1 −2



= e ,e =  0 x 0 .

1

−2 −2 2


0 0 x sin x
To obtain the dual basis, use Theorem 25.1.6 to write

e1 (x) = g1 j e j (x) = e1 (x)

−2
e2 (x) = g2 j e j (x) = x1 e2 (x)

−2 −2 2

e3 (x) = g3 j e j (x) = x1 sin x e3 (x) .
 
Note that ∂∂yyk ≡ ek (y) = ik = ik where, as described, y1 ··· yn are the rectan-
gular coordinates of the point in Rn .
472 CHAPTER 25. CURVILINEAR COORDINATES

25.4 Exercises
1. Let    
y1 x1 + 2x2
 2   2
 y  =  x + x3 


y3 x1 − 2x2
where the yi are the rectangular coordinates of the point. Find ei , ei , i = 1, 2, 3, and
i j


find (gi j ) (x) and g (x) .
2. Let y = y (x,t) where t signifies time and x ∈ U ⊆ Rm for U an open set, while
y ∈ Rn and suppose x is a function of t. Physically, this corresponds to an object
moving over a surface in Rn which may be changing as a function of t. The point
y = y (x (t) ,t) is the point in Rn corresponding to t. For example, consider the pen-
dulum

l
θ
•m

in which n = 2, l is fixed and y1 = l sin θ , y2 = l − l cos θ . Thus, in this simple exam-

ple, m = 1. If l were changing in a known way with respect to t, then this would be
of the form y = y (x,t) . In general, the kinetic energy is defined as

1
T ≡ mẏ · ẏ (∗)
2
where the dot on the top signifies differentiation with respect to t. Show

∂T ∂y
= m ẏ· k .
∂ ẋk ∂x
Hint: First show
∂y j ∂y
ẏ = ẋ + (∗∗)
∂xj ∂t
and so
∂ ẏ ∂y
= .
∂ ẋ j ∂xj
3. ↑ Show
∂ 2y r ∂ 2y
 
d ∂T ∂y
= m ÿ· + m ẏ· ẋ + m ẏ · .
dt ∂ ẋk ∂ xk ∂ xk ∂ xr ∂t∂ xk

4. ↑ Show  2
∂ 2y

∂T ∂ y r
= m ẏ· ẋ + .
∂ xk ∂ xr ∂ xk ∂t∂ xk
Hint: Use ∗ and ∗ ∗ .
25.4. EXERCISES 473

5. ↑ Now show from Newton’s second law ( mass times acceleration equals force ) that
for F the force,  
d ∂T ∂T ∂y ∂y
− k = m ÿ· k = F · k . (∗ ∗ ∗)
dt ∂ ẋk ∂x ∂x ∂x

6. ↑ In the example of the simple pendulum above,

!
l sin θ
y= = l sin θ i+ (l − l cos θ ) j.
l − l cos θ

Use ∗ ∗ ∗ to find a differential equation which describes the vibrations of the pendu-
lum in terms of θ . First write the kinetic energy and then consider the force acting
on the mass which is −mgj.
7. Of course, the idea is to write equations of motion in terms of the variables xk , instead
of the rectangular variables yk . Suppose y = y (x) and x is a function of t. Letting G
denote the metric tensor, show that the kinetic energy is of the form 21 mẋT Gx where
m is a point mass with m its mass.
8. The pendulum problem is fairly easy to do without the formalism developed. Now
consider the case where x = (ρ, θ , φ ) , spherical coordinates, and write differential
equations for ρ, θ , and φ to describe the motion of an object in terms of these coor-
dinates given a force, F.
9. Suppose the pendulum is not assumed to vibrate in a plane. Let it be suspended at
the origin and let φ be the angle between the negative z axis and the positive x axis
while θ is the angle between the projection of the position vector onto the xy plane
and the positive x axis in the usual way. Thus

x = ρ sin φ cos θ , y = ρ sin φ sin θ , z = −ρ cos φ

10. If there are many masses, mα , α = 1, · · · , R, the kinetic energy is the sum of the
kinetic energies of the individual masses. Thus,

1 R
T≡ ∑ mα |ẏα |2 .
2 α=1

Generalize the above problems to show that, assuming

y α = y α (x,t) ,
  R
d ∂T ∂T ∂ yα
− = ∑ Fα ·
dt ∂ ẋk ∂x k
α=1 ∂ xk
where F α is the force acting on mα .
11. Discuss the equivalence of these formulae with Newton’s second law, force equals
mass times acceleration. What is gained from the above so called Lagrangian for-
malism?
474 CHAPTER 25. CURVILINEAR COORDINATES

12. The double pendulum has two masses instead of only one.

l1
θ
m1

l2
φ
m2

Write differential equations for θ and φ to describe the motion of the double pendu-
lum.

25.5 Transformation of Coordinates.

How do we write ek (x) in terms of the vectors, e j (z) where z is some other type of
curvilinear coordinates? This is next.
Consider the following picture in which U is an open set in Rn , D and Db are open sets in
n 2
R , and M, N are C mappings which are one to one from D and D respectively. The only
b
reason for this is to ensure that the mixed partial derivatives are equal. We will suppose
that a point in U is identified by the curvilinear coordinates x in D and z in D.
b

U
M N

D Db
(x1 , x2 , x3 ) (z1 , z2 , z3 )
Thus M (x) = N (z) and so z = N −1 (M (x)) . The point in U will be denoted in
rectangular coordinates as y and we have y (x) = y (z) Now by the chain rule,

∂y ∂y ∂xj ∂xj
ei (z) = = = e j (x) (25.19)
∂ zi ∂ x j ∂ zi ∂ zi
Define the covariant and contravariant coordinates for the various curvilinear coordinates
in the obvious way. Thus,

v = vi (x) ei (x) = vi (x) ei (x) = v j (z) e j (z) = v j (z) e j (z) .

Then the following theorem tells how to transform the vectors and coordinates.

Theorem 25.5.1 The following transformation rules hold for pairs of curvilinear coordi-
nates.
∂xj ∂ zi j
vi (z) = v j (x) , vi (z) = v (x) , (25.20)
∂ zi ∂xj
∂xj ∂ zi j
ei (z) = e j (x) , ei (z) = e (x) , (25.21)
∂ zi ∂xj
25.6. DIFFERENTIATION AND CHRISTOFFEL SYMBOLS 475

∂ xr ∂ xs ij ∂ zi ∂ z j rs
gi j (z) = grs (x) , g (z) = g (x) . (25.22)
∂ zi ∂ z j ∂ xr ∂ xs
Proof: We already have shown the first part of 25.21 in 25.19. Then, from 25.19,

∂ zk
ei (z) = ei (z) · e j (x) e j (x) = ei (z) · ek (z) e j (x)
∂xj
∂ zk j ∂ zi j
= δ ik e (x) = e (x)
∂xj ∂xj
and this proves the second part of 25.21. Now to show 25.20,

∂xj ∂xj ∂xj

vi (z) = v · ei (z) = v· e j (x) = v · e j (x) = v j (x)
∂ zi ∂ zi ∂ zi
and
∂ zi j ∂ zi ∂ zi j
vi (z) = v · ei (z) = v · j
e (x) = j
v · e j (x) = v (x) .
∂x ∂x ∂xj
To verify 25.22,

∂ xr ∂ xs ∂ xr ∂ xs
gi j (z) = ei (z) · e j (z) = er (x) i
· es (x) j = grs (x) i j . ■
∂z ∂z ∂z ∂z

25.6 Differentiation and Christoffel Symbols

Let F : U → Rn be differentiable. We call F a vector field and it is used to model force,
velocity, acceleration, or any other vector quantity which may change from point to point
in U. Then ∂ F∂ x(xj ) is a vector and so there exist scalars, F,ij (x) and Fi, j (x) such that

∂ F (x) ∂ F (x)
j
= F,ij (x) ei (x) , = Fi, j (x) ei (x) (25.23)
∂x ∂xj
We will see how these scalars transform when the coordinates are changed.

Theorem 25.6.1 If x and z are curvilinear coordinates,

∂ xr ∂ z j ∂ xr ∂ xs
F,sr (x) = F,ij (z) , Fr,s (x) = Fi, j (z) . (25.24)
∂ zi ∂ x s ∂ zi ∂ z j
Proof:
∂ F (x) ∂ F (z) ∂ z j
F,sr (x) er (x) ≡ = ≡
∂ xs ∂ z j ∂ xs
∂zj ∂ z j ∂ xr
F,ij (z) ei (z)
s
= F,ij (z) s i er (x)
∂x ∂x ∂z
which shows the first formula of 25.23. To show the other formula,
∂ F (z) ∂ F (x) ∂ xs
Fi, j (z) ei (z) ≡ = ≡
∂zj ∂ xs ∂ z j
∂ xs ∂ xs ∂ xr
Fr,s (x) er (x) j
= Fr,s (x) j i ei (z) ,
∂z ∂z ∂z
476 CHAPTER 25. CURVILINEAR COORDINATES

and this shows the second formula for transforming these scalars. ■
Now F (x) = F i (x) ei (x) and so by the product rule,

∂F ∂ Fi ∂ ei (x)
= ei (x) + F i (x) . (25.25)
∂xj ∂xj ∂xj
( )
∂ ei (x) k
Now ∂xj
is a vector and so there exist scalars, such that
ij
( )
∂ ei (x) k
= ek (x) .
∂xj ij

Thus
∂ 2y
 
k
ek (x) =
ij ∂ x j ∂ xi
and so
( ) ( ) ( )
k k r ∂ 2y
r
ek (x) · e (x) = δ rk = = · er (x) (25.26)
ij ij ij ∂ x j ∂ xi
( )
∂F ∂ Fk r
Therefore, from 25.25, ∂xj
= ∂xj k
e (x) + F i (x) ek (x) which shows
ij
( )
∂ Fk k
F,kj (x) = + F i (x) . (25.27)
∂xj ij

This is sometimes called the covariant derivative.

Theorem 25.6.2 The Christoffel symbols of the second kind satisfy the following
( )
∂ ei (x) k
= ek (x) , (25.28)
∂xj ij
( )
∂ ei (x) i
=− ek (x) , (25.29)
∂xj kj
( ) ( )
k k
= , (25.30)
ij ji
( )
g jm ∂ gi j ∂ gk j ∂ gik
 
m
= + − . (25.31)
ik 2 ∂ xk ∂ xi ∂xj

Proof: Formula 25.28 is the definition of the Christoffel symbols. We verify 25.29 next.
To do so, note
ei (x) · ek (x) = δ ik .
25.6. DIFFERENTIATION AND CHRISTOFFEL SYMBOLS 477

Then from the product rule,

∂ ei (x) ∂ ek (x)
j
· ek (x) + ei (x) · = 0.
∂x ∂xj
Now from the definition,
( ) ( ) ( )
∂ ei (x) r r i
· ek (x) = −ei (x) · er (x) = − δ ir =− .
∂xj kj kj kj

But also, using the above,

( )
∂ ei (x) ∂ ei (x) i
= · ek (x) ek (x) = − ek (x) .
∂xj ∂xj kj

This verifies 25.29. Formula 25.30 follows from 25.26 and equality of mixed partial deriva-
tives.
It remains to show 25.31.
( ) ( )
∂ gi j ∂ ei ∂ej r r
= k · e j + ei · k = er · e j + ei · er .
∂ xk ∂x ∂x ik jk

Therefore,    
∂ gi j r r
= grj + gri . (25.32)
∂ xk ik jk
Switching i and k while remembering 25.30 yields
( ) ( )
∂ gk j r r
= gr j + grk . (25.33)
∂ xi ik ji

Now switching j and k in 25.32,

( ) ( )
∂ gik r r
= grk + gri . (25.34)
∂xj ij jk

Adding 25.32 to 25.33 and subtracting 25.34 yields

( )
∂ gi j ∂ gk j ∂ gik r
+ − =2 gr j .
∂ xk ∂ xi ∂xj ik

Now multiplying both sides by g jm and using the fact shown earlier in Theorem 25.1.6 that
gr j g jm = δ m
r , it follows
( )  
m ∂ gi j ∂ gk j ∂ gik
2 = g jm + −
ik ∂ xk ∂ xi ∂xj

which proves 25.31. ■

This is a very interesting formula because it shows the Christoffel symbols are com-
pletely determined by the metric tensor and its partial derivatives which illustrates the fun-
damental nature of the metric tensor. Note that the inner product is determined by this
metric tensor.
478 CHAPTER 25. CURVILINEAR COORDINATES

25.7 Gradients and Divergence

The purpose of this section is to express the gradient
and the divergence of a vector field in
general curvilinear coordinates. As before, y1 , ..., yn will denote the standard coordinates
with respect to the usual basis vectors. Thus

y ≡ yk ik , ek (y) = ik = ek (y) .

Let φ : U → R be a differentiable scalar function, sometimes called a “scalar field” in

this subject. Write φ (x) to denote the value of φ at the point whose coordinates are x. The
same convention is used for a vector field. Thus F (x) is the value of a vector field at the
point of U determined by the coordinates x. In the standard rectangular coordinates, the
gradient is well understood from earlier.
∂ φ (y) k ∂ φ (y) k
∇φ (y) = k
e (y) = i.
∂y ∂ yk
However, the idea is to express the gradient in arbitrary coordinates. Therefore, using the
chain rule, if the coordinates of the point of U are given as x,
∂ φ (x) ∂ xr k
∇φ (x) = ∇φ (y) = e (y) =
∂ xr ∂ yk

∂ φ (x) ∂ xr ∂ yk s ∂ φ (x) r s ∂ φ (x) r

e (x) = δ e (x) = e (x) .
∂ xr ∂ yk ∂ xs ∂ xr s ∂ xr
This shows the covariant components of ∇φ (x) are

∂ φ (x)
(∇φ (x))r = , (25.35)
∂ xr
Formally the same as in rectangular coordinates. To find the contravariant components,
“raise the index” in the usual way. Thus
∂ φ (x)
(∇φ (x))r = grk (x) (∇φ (x))k = grk (x) . (25.36)
∂ xk
What about the divergence of a vector field? The divergence of a vector field F defined
on U is a scalar field, div (F ) which from calculus is

∂ Fk
(y) = F,kk (y)
∂ yk
in terms of the usual rectangular coordinates y. The reason the above equation holds in
this case is that ek (y) is a constant and so the Christoffel symbols are zero. We want an
expression for the divergence in arbitrary coordinates. From Theorem 25.6.1,

∂ x s ∂ yi
F,ij (y) = F,sr (x)
∂ y j ∂ xr
From 25.27, ( ) !
∂ F r (x) r ∂ xs ∂ yi
= + F k (x) (x) .
∂ xs ks ∂ y j ∂ xr
25.7. GRADIENTS AND DIVERGENCE 479

Letting j = i yields
( ) !
∂ F r (x) r ∂ xs ∂ yi
div (F ) = + F k (x) (x)
∂ xs ks ∂ yi ∂ xr
( ) !
∂ F r (x) r
= + F k (x) (x) δ sr
∂ xs ks
( ) !
∂ F r (x) r
= + F k (x) (x) . (25.37)
∂ xr kr
( )
r
is simplified using the description of it in Theorem 25.6.2. Thus, from this theo-
kr
rem, ( )
g jr ∂ gr j ∂ gk j ∂ grk
 
r
= + −
rk 2 ∂ xk ∂ xr ∂xj
g jr
Now consider 2 times the last two terms in [·] . Relabeling the indices r and j in the second
term implies
g jr ∂ gk j g jr ∂ grk g jr ∂ gk j gr j ∂ g jk
− = − = 0.
2 ∂ xr 2 ∂xj 2 ∂ xr 2 ∂ xr
Therefore, ( )
r g jr ∂ gr j
= . (25.38)
rk 2 ∂ xk
Now recall g ≡ det (gi j ) = det (G) > 0 from Theorem 25.1.6. Also from the formula for the
inverse of a matrix and this theorem,

g jr = Ar j (det G)−1 = A jr (det G)−1

where Ar j is the r jth cofactor of the matrix (gi j ) . Also recall that
n
g= ∑ gr j Ar j no sum on j.
r=1

∂g
= Ar j . From 25.38,

Therefore, g is a function of the variables gr j and ∂ gr j
( )
r g jr ∂ gr j 1 ∂ gr j jr 1 ∂ g ∂ gr j 1 ∂g
= k
= k
A = k
=
rk 2 ∂x 2g ∂ x 2g ∂ gr j ∂ x 2g ∂ xk

and so from 25.37,

∂ F k (x)
div (F ) =
+
∂ xk
1 ∂ g (x) 1 ∂  i p 
+F k (x) = F (x) g (x) . (25.39)
2g (x) ∂ xk g (x) ∂ xi
p

This is the formula for the divergence of a vector field in general curvilinear coordinates.
Note that it uses the contravariant components of F .
480 CHAPTER 25. CURVILINEAR COORDINATES

The Laplacian of a scalar field is nothing more than the divergence of the gradient. In
symbols, ∆φ ≡ ∇ · ∇φ . From 25.39 and 25.36 it follows
 
1 ∂ ik ∂ φ (x) p
∆φ (x) = p g (x) g (x) . (25.40)
g (x) ∂ xi ∂ xk

We summarize the conclusions of this section in the following theorem.

Theorem 25.7.1 The following formulas hold for the gradient, divergence and Laplacian
in general curvilinear coordinates.

∂ φ (x)
(∇φ (x))r = , (25.41)
∂ xr
∂ φ (x)
(∇φ (x))r = grk (x) , (25.42)
∂ xk
1 ∂  i p 
div (F ) = p F (x) g (x) , (25.43)
g (x) ∂ xi
 
1 ∂ ik ∂ φ (x) p
∆φ (x) = p g (x) g (x) . (25.44)
g (x) ∂ xi ∂ xk

Example 25.7.2 Define curvilinear coordinates as follows

x = r cos θ , y = r sin θ

Find ∇2 f (r, θ ). That is, find the Laplacian in terms of these new variables r, θ .

First find the metric tensor. From the definition, this is

! !
1 0 −1 1 0
G= ,G =
0 r2 0 r−2

The contravariant components of the gradient are

! ! !
1 0 fr fr
=
0 r−2 fθ 1
f
r2 θ
√
Then also g = r. Therefore, using the formula,
   
1 1 1 1
∇2 f (u, v) = (r fr )r + r 2 fθ = (r fr )r + 2 fθ θ
r r θ r r

Notice how easy this is. It is anything but easy if you try to do it by brute force with none
of the machinery developed here.
25.8. EXERCISES 481

25.8 Exercises
1. Let y1 = x1 + 2x2 , y2 = x2 + 3x3 , y3 = x1 + x3 . Let

2
F (x) = x1 e1 (x) + x2 e2 (x) + x3 e (x) .
Find div (F ) (x) .
2. For the coordinates of the preceding problem, and φ a scalar field, find
(∇φ (x))3
in terms of the partial derivatives of φ taken with respect to the variables xi .
3. Let y1 = 7x1 + 2x2 , y2 = x2 + 3x3 , y3 = x1 + x3 . Let φ be a scalar field. Find ∇2 φ (x) .
4. Derive ∇2 u in cylindrical coordinates, r, θ , z, where u is a scalar field on R3 .
x = r cos θ , y = r sin θ , z = z.
p
5. ↑ Find all solutions to ∇2 u = 0 which depend only on r where r ≡ x2 + y2 .
6. Derive ∇2 u in spherical coordinates.
7. ↑Let u be a scalar field on R3 . Find all solutions to ∇2 u = 0 which depend only on
p
ρ ≡ x 2 + y2 + z2 .

8. The temperature, u, in a solid satisfies ∇2 u = 0 after a long time. Suppose in a long

pipe of inner radius 9 and outer radius 10 the exterior surface is held at 100◦ while
the inner surface is held at 200◦ find the temperature in the solid part of the pipe.
9. Show  
l ∂ ei l
= ·e .
ij ∂xj
Find the Christoffel symbols of the second kind for spherical coordinates in which
x1 = φ , x2 = θ , and x3 = ρ. Do the same for cylindrical coordinates letting x1 = r,
x2 = θ , x3 = z.
10. Show velocity can be expressed as v = vi (x) ei (x) , where

∂ ri dx j
  k
p dx
vi (x) = j
− r p (x)
∂ x dt ik dt
and ri (x) are the covariant components of the displacement vector,
r = ri (x) ei (x) .

11. ↑ Using problem 9 and 10, show the covariant components of velocity in spherical
coordinates are
dφ dθ dρ
v1 = ρ 2 , v2 = ρ 2 sin2 (φ ) , v3 = .
dt dt dt
Hint: First observe that if r is the position vector from the origin, then r = ρe3 so
r1 = 0 = r2 , and r3 = ρ. Now use 10.
482 CHAPTER 25. CURVILINEAR COORDINATES

25.9 Curl and Cross Products

In this section is the curl and cross product in general curvilinear coordinates in R3 . We
will always assume that for x a set of curvilinear coordinates,
 i
∂y
det >0 (25.45)
∂xj

Where the y i are the usual coordinates in which ek (y) = ik .

Theorem 25.9.1 Let 25.45 hold. Then

 i
∂y p
det j
= g (x) (25.46)
∂x

and
∂ xi
 
1
det =p . (25.47)
∂yj g (x)

Proof:
∂ yk
ei (x) = ik
∂ xi
and so
∂ yk ∂ yl ∂ yk ∂ yk
gi j (x) =
i k · il = .
∂ xi ∂xj ∂ xi ∂ x j
  k 2 √  k
Therefore, g = det (gi j (x)) = det ∂∂ yxi . By 25.45, g = det ∂∂ yxi as claimed. Now

∂ yk ∂ xi
= δ kr
∂ xi ∂ yr
and so
∂ xi
 
1
det =p .
∂ yr g (x)
This proves the theorem.
To get the curl and cross product in curvilinear coordinates, let ε i jk be the usual permu-
tation symbol. Thus,
ε 123 = 1
and when any two indices in ε i jk are switched, the sign changes. Thus

ε 132 = −1, ε 312 = 1, etc.

Now define
1
ε i jk (x) ≡ ε i jk p .
g (x)
Then for x and z satisfying 25.45,

∂ zr ∂ zs ∂ zt
 p r s t
i jk i jk ∂x ∂z ∂z ∂z
ε (x) i j k = ε det q
∂x ∂x ∂x ∂y ∂ xi ∂ x j ∂ xk
25.9. CURL AND CROSS PRODUCTS 483

∂ xp ∂ zi
   
rst
=ε det det = ε rst det (MN)
∂ yq ∂ xk
p ∂ zi
where N is the matrix whose pqth entry is ∂∂ xyq and M is the matrix whose ikth entry is ∂ xk
.
Therefore, from the definition of matrix multiplication and the chain rule, this equals
 i
rst ∂z
= ε det ≡ ε rst (z)
∂ yp
from the above discussion.
Now ε i jk (y) = ε i jk and for a vector field, F,
curl (F ) ≡ ε i jk (y) Fk, j (y) ei (y) .
Therefore, since we know how everything transforms assuming 25.45, it is routine to write
this in terms of x.
∂ yi ∂ y j ∂ yk ∂ x p ∂ xq ∂ xm
curl (F ) = ε rst (x) r s t
Fp,q (x) k j
em (x) i
∂x ∂x ∂x ∂y ∂y ∂y
q p
= ε rst (x) δ m
r δ s δ t Fp,q (x) em (x) = ε
mqp
(x) Fp,q (x) em (x) . (25.48)
More simplification is possible. Recalling the definition of Fp,q (x) ,
∂F ∂
≡ Fp,q (x) e p (x) = q [Fp (x) e p (x)]
∂ xq ∂x
( )
∂ Fp (x) p ∂ ep ∂ Fp (x) p r
= e (x) + Fp (x) q = e (x) − Fr (x) e p (x)
∂ xq ∂x ∂ xq pq
by Theorem 25.6.2. Therefore,
( )
∂ Fp (x) r
Fp,q (x) = − Fr (x)
∂ xq pq

and so
( )
∂ Fp (x) r
curl (F ) = ε mqp (x) em (x) − ε mqp (x) Fr (x) em (x) .
∂ xq pq
( ) ( )
r r
However, because = , the second term in this expression equals 0. To
pq qp
see this, ( ) ( ) ( )
mqp r mpq r mqp r
ε (x) =ε (x) = −ε (x) .
pq qp pq
Therefore, by 25.48,
∂ Fp (x)
curl (F ) = ε mqp (x)
em (x) . (25.49)
∂ xq
What about the cross product of two vector fields? Let F and G be two vector fields.
Then in terms of standard coordinates y,
F × G = ε i jk (y) Fj (y) Gk (y) ei (y)
484 CHAPTER 25. CURVILINEAR COORDINATES

∂ yi ∂ y j ∂ yk ∂ xp ∂ xq ∂ xl
= ε rst (x) Fp (x) G q (x) el (x)
∂ xr ∂ xs ∂ xt ∂yj ∂ yk ∂ yi
= ε rst (x) δ sp δ tq δ lr Fp (x) Gq (x) el (x) = ε l pq (x) Fp (x) Gq (x) el (x) . (25.50)
We summarize these results in the following theorem.

Theorem 25.9.2 Suppose x is a system of curvilinear coordinates in R3 such that

 i
∂y
det > 0.
∂xj

Let
1
ε i jk (x) ≡ ε i jk p .
g (x)
Then the following formulas for curl and cross product hold in this system of coordinates.

∂ Fp (x)
curl (F ) = ε mqp (x) em (x) ,
∂ xq
and
F × G = ε l pq (x) Fp (x) Gq (x) el (x) .
Chapter 26

Implicit Function Theorem*

The implicit function theorem is one of the greatest theorems in mathematics. There are
many versions of this theorem which are of far greater generality than the one given here.
The proof given here is like one found in one of Caratheodory’s books on the calculus
of variations. It is not as elegant as some of the others which are based on a contraction
mapping principle but it may be more accessible. However, it is an advanced topic. Don’t
waste your time with it unless you have first read and understood the material on rank and
determinants found in the chapter on the mathematical theory of determinants. You will
also need to use the extreme value theorem for a function of n variables and the chain rule
of multivariable calculus as well as everything about matrix multiplication.

Definition 26.0.1 Suppose U is an open set in Rn × Rm and (x, y) will denote a typical
point of Rn × Rm with x ∈ Rn and y ∈ Rm . Let f : U → R p be in C1 (U) . Then define
 
f1,x1 (x, y) · · · f1,xn (x, y)
 .. .. 
D1 f (x, y) ≡   . .
,

f p,x1 (x, y) · · · f p,xn (x, y)
 
f1,y1 (x, y) · · · f1,ym (x, y)
 .. .. 
D2 f (x, y) ≡   . .
.

f p,y1 (x, y) · · · f p,ym (x, y)

Theorem 26.0.2 (implicit function theorem) Suppose U is an open set in Rn × Rm . Let

f : U → Rn be in C1 (U) and suppose

f (x0 , y 0 ) = 0, D1 f (x0 , y 0 )−1 exists. (26.1)

Then there exist positive constants, δ , η, such that for every y ∈ B (y 0 , η) there exists a
unique x (y) ∈ B (x0 , δ ) such that

f (x (y) , y) = 0. (26.2)

Furthermore, the mapping, y → x (y) is in C1 (B (y 0 , η)).

485
486 CHAPTER 26. IMPLICIT FUNCTION THEOREM*

Proof: Let  
f1 (x, y)
f2 (x, y)
 
 
f (x, y) = 
 .. .

 . 
fn (x, y)
n
Define for x1 , · · · , xn ∈ B (x0 , δ ) and y ∈ B (y 0 , η) the following matrix.



f1,x1 x1 , y · · · f1,xn x1 , y


.. ..
J x1 , · · · , xn , y ≡ 

 
. (*)
 . . 
n n
fn,x1 (x , y) · · · fn,xn (x , y)

Then by the assumption of continuity of all the partial derivatives and the extreme value
theorem, there exists r > 0 and δ 0 , η 0 > 0 such that if δ ≤ δ 0 and η ≤ η 0 , it follows that
n
for all x1 , · · · , xn ∈ B (x0 , δ ) and y ∈ B (y 0 , η),

det J x1 , · · · , xn , y



> r > 0. (26.3)

and B (x0 , δ 0 )× B (y 0 , η 0 ) ⊆ U. By continuity of all the partial derivatives and the extreme
value theorem, it can also be assumed there exists a constant, K such that for all (x, y) ∈
B (x0 , δ 0 )× B (y 0 , η 0 ) and i = 1, 2, · · · , n, the ith row of D2 f (x, y) , given by D2 fi (x, y)
satisfies
|D2 fi (x, y)| < K, (26.4)
1 n

n th
and for all x , · · · , x ∈ B (x0 , δ 0 ) and y ∈ B (y 0 , η 0 ) the i row of the matrix,

−1
J x1 , · · · , xn , y

−1 
which equals eTi J x1 , · · · , xn , y satisfies

−1 
eTi J x1 , · · · , xn , y < K. (26.5)

(Recall that ei is the column vector consisting of all zeros except for a 1 in the ith position.)
To begin with it is shown that for a given y ∈ B (y 0 , η) there is at most one x ∈ B (x0 , δ )
such that f (x, y) = 0.
Pick y ∈ B (y 0 , η) and suppose there exist x, z ∈ B (x0 , δ ) such that f (x, y) = f (z, y) =
0. Consider fi and let
h (t) ≡ fi (x + t (z − x) , y) .
Then h (1) = h (0) and so by the mean value theorem, h′ (ti ) = 0 for some ti ∈ (0, 1) . There-
fore, from the chain rule and for this value of ti ,
n
∂
h′ (ti ) = ∑ ∂ x j fi (x + ti (z − x) , y) (z j − x j ) = 0. (26.6)
j=1

Then denote by xi the vector, x + ti (z − x) . It follows from 26.6 that

J x1 , · · · , xn , y (z − x) = 0


 487

and so from 26.3 z − x = 0. (The matrix, in the above is invertible since its determinant
is nonzero.) Now it will be shown that if η is chosen sufficiently small, then for all y ∈
B (y 0 , η) , there exists a unique x (y) ∈ B (x0 , δ ) such that f (x (y) , y) = 0.
Claim: If η is small enough, then the function, x → hy (x) ≡ |f (x, y)|2 achieves its
minimum value on B (x0 , δ ) at a point of B (x0 , δ ) . (The existence of a point in B (x0 , δ )
at which hy achieves its minimum follows from the extreme value theorem.)
Proof of claim: Suppose this is not the case. Then there exists a sequence η k → 0
and for some y k having |y k −y 0 | < η k , the minimum of hyk on B (x0 , δ ) occurs on a
point xk such that |x0 −xk | = δ . Now taking a subsequence, still denoted by k, it can
n assumed that xk → x witho|x − x0 | = δ and y k → y 0 . This follows from the fact that
be
x ∈ B (x0 , δ ) : |x − x0 | = δ is a closed and bounded set and is therefore sequentially
compact. Let ε > 0. Then for k large enough, the continuity of y → hy (x0 ) implies
hyk (x0 ) < ε because hy0 (x0 ) = 0 since f (x0 , y 0 ) = 0. Therefore, from the definition
of xk , it is also the case that hyk (xk ) < ε. Passing to the limit yields hy0 (x) ≤ ε. Since
ε > 0 is arbitrary, it follows that hy0 (x) = 0 which contradicts the first part of the argument
in which it was shown that for y ∈ B (y 0 , η) there is at most one point, x of B (x0 , δ ) where
f (x, y) = 0. Here two have been obtained, x0 and x. This proves the claim.
Choose η < η 0 and also small enough that the above claim holds and let x (y) denote
a point of B (x0 , δ ) at which the minimum of hy on B (x0 , δ ) is achieved. Since x (y) is an
interior point, you can consider hy (x (y) + tv) for |t| small and conclude this function of t
has a zero derivative at t = 0. Now
n
hy (x (y) + tv) = ∑ fi2 (x (y) + tv, y)
i=1

and so from the chain rule,

n n
d ∂ fi (x (y) + tv, y)
hy (x (y) + tv) = ∑ ∑ 2 fi (x (y) + tv, y) v j.
dt i=1 j=1 ∂xj
Therefore, letting t = 0, it is required that for every v,
n n
∂ fi (x (y) , y)
∑ ∑ 2 fi (x (y) , y) ∂xj
v j = 0.
i=1 j=1

In terms of matrices this reduces to

0 = 2f (x (y) , y)T D1 f (x (y) , y) v
for every vector v. Therefore,
0 = f (x (y) , y)T D1 f (x (y) , y)
From 26.3, it follows f (x (y) , y) = 0. This proves the existence of the function y → x (y)
such that f (x (y) , y) = 0 for all y ∈ B (y 0 , η) .
It remains to verify this function is a C1 function. To do this, let y 1 and y 2 be points of
B (y 0 , η) . Then as before, consider the ith component of f and consider the same argument
using the mean value theorem to write
0 = fi (x (y 1 ) , y 1 ) − fi (x (y 2 ) , y 2 )
= fi (x (y 1 ) , y 1 ) − fi (x (y 2 ) , y 1 ) + fi (x (y 2 ) , y 1 ) − fi (x (y 2 ) , y 2 ) (26.7)
= D1 fi xi , y 1 (x (y 1 ) − x (y 2 )) + D2 fi x (y 2 ) , y i (y 1 − y 2 ) .



488 CHAPTER 26. IMPLICIT FUNCTION THEOREM*

where y i is a point on the line segment
joining y 1 and y 2 . Thus from 26.4 and the Cauchy-
Schwarz inequality, D2 fi x (y 2 ) , y i (y 1 − y 2 ) ≤ K |y 1 − y 2 | . Therefore, defining the
symbol M y 1 , · · · , y n ≡ M denote the matrix having the ith row equal to

D2 fi x (y 2 ) , y i ,

it follows !1/2
2 2 √
|M (y 1 − y 2 )| ≤ ∑K |y 1 − y 2 | = mK |y 1 − y 2 | . (26.8)
i
Also, from 26.7,
J x1 , · · · , xn , y 1 (x (y 1 ) − x (y 2 )) = −M (y 1 − y 2 )


(26.9)
and so from 26.8, 26.5, |x (y 1 ) − x (y 2 )| =

−1
= J x1 , · · · , xn , y 1 M (y 1 − y 2 )
!1/2
n
−1 2
1
= ∑ eTi J n
x , · · · , x , y1 M (y 1 − y 2 )
i=1
!1/2 !1/2
n n √
2
2
≤ ∑ K 2 |M (y1 − y2 )| ≤ ∑ K2 mK |y 1 − y 2 |
i=1 i=1
√
= K 2 mn |y 1 − y 2 |
Now let y 2 = y, y 1 = y + hek for small h. Then M depends on h and
lim M (h) = D2 f (x (y) , y)
h→0

thanks to the continuity of y → x (y) just shown. Also,

x (y + hek ) − x (y)
−1
= −J x1 (h) , · · · , xn (h) , y + hek M (h) ek
h
Passing to a limit and using the formula for the inverse of a matrix in terms of the cofactor
matrix, and the continuity of y → x (y) shown above, this yields
∂x
= −D1 f (x (y) , y)−1 D2 fi (x (y) , y) ek
∂ yk
Then continuity of y → x (y) and the assumed continuity of the partial derivatives of f
shows that each partial derivative of y → x (y) exists and is continuous. ■
This implies the inverse function theorem given next.
Theorem 26.0.3 (inverse function theorem) Let x0 ∈ U, an open set in Rn , and let f :
U → Rn . Suppose
f is C1 (U) , and Df (x0 )−1 exists. (26.10)
Then there exist open sets W , and V such that
x0 ∈ W ⊆ U, (26.11)
f : W → V is one to one and onto, (26.12)
−1 1
f is C , (26.13)
26.1. MORE CONTINUOUS PARTIAL DERIVATIVES 489

Proof: Apply the implicit function theorem to the function

F (x, y) ≡ f (x) − y

where y 0 ≡ f (x0 ). Thus the function y → x (y) defined in that theorem is f −1 . Now let

W ≡ B (x0 , δ ) ∩ f −1 (B (y 0 , η))

and
V ≡ B (y 0 , η) .
This proves the theorem. ■

26.1 More Continuous Partial Derivatives

The implicit function theorem will now be improved slightly. If f is Ck , it follows that
the function which is implicitly defined is also Ck , not just C1 , meaning all mixed partial
derivatives of f up to order k are continuous. Since the inverse function theorem comes
as a case of the implicit function theorem, this shows that the inverse function also inherits
the property of being Ck . First some notation is convenient. Let α = (α 1 , · · · , α n ) where
each α i is a nonnegative integer. Then letting |α| = ∑i α i ,

∂ |α| f
Dα f (x) ≡ (x) , D0 f (x) ≡ f (x)
∂ α1 ∂ α2 · · · ∂ αn
Theorem 26.1.1 (implicit function theorem) Suppose U is an open set in Fn × Fm . Let
f : U → Fn be in Ck (U) and suppose

f (x0 , y 0 ) = 0, D1 f (x0 , y 0 )−1 ∈ L (Fn , Fn ) . (26.14)

Then there exist positive constants δ , η, such that for every y ∈ B (y 0 , η) there exists a
unique x (y) ∈ B (x0 , δ ) such that

f (x (y) , y) = 0. (26.15)

Furthermore, the mapping y → x (y) is in Ck (B (y 0 , η)).

Proof: From the implicit function theorem y → x (y) is C1 . It remains to show that it
is Ck for k > 1 assuming that f is Ck . From (26.15)

∂x ∂f
= −D1 f (x, y)−1 l .
∂ yl ∂y

Thus the following formula holds for q = 1 and |α| = q.

Dα x (y) = ∑ Mβ (x, y) Dβ f (x, y) (26.16)

|β |≤q

where Mβ is a matrix whose entries are differentiable functions of Dγ x for |γ| < q and
Dτ f (x, y) for |τ| ≤ q. This follows easily from the description of D1 f (x, y)−1 in terms
490 CHAPTER 26. IMPLICIT FUNCTION THEOREM*

of the cofactor matrix and the determinant of D1 f (x, y). Suppose (26.16) holds for |α| =
q < k. Then by induction, this yields x is Cq . Then

∂ Dα x (y) ∂ Mβ (x, y) β ∂ Dβ f (x, y)

= ∑ D f (x, y) + Mβ (x, y) .
∂ yp |β |≤|α|
∂ yp ∂ yp

∂ M (x,y )
By the chain rule β∂ y p is a matrix such that its entries are differentiable functions of
Dτ f (x, y) for |τ| ≤ q + 1 and Dγ x for |γ| < q + 1. It follows, since y p was arbitrary,
that for any |α| = q + 1, a formula like (26.16) holds with q being replaced by q + 1. By
induction, x is Ck . ■
As a simple corollary, this yields the inverse function theorem. You just let F (x, y) =
y − f (x) and apply the implicit function theorem.

Theorem 26.1.2 (inverse function theorem) Let x0 ∈ U ⊆ Fn and let f : U → Fn . Suppose

for k a positive integer,

f is Ck (U) , and Df (x0 )−1 ∈ L (Fn , Fn ). (26.17)

Then there exist open sets W , and V such that

x0 ∈ W ⊆ U, (26.18)

f : W → V is one to one and onto, (26.19)

f −1 is Ck . (26.20)

26.2 The Method Of Lagrange Multipliers

1. As an application of the implicit function theorem, consider the method of Lagrange
multipliers. Recall the problem is to maximize or minimize a function subject to
equality constraints. Let f : U → R be a C1 function where U ⊆ Rn and let

gi (x) = 0, i = 1, · · · , m (26.21)

be a collection of equality constraints with m < n. Now consider the system of

nonlinear equations

f (x) = a
gi (x) = 0, i = 1, · · · , m.

Recall x0 is a local maximum if f (x0 ) ≥ f (x) for all x near x0 which also sat-
isfies the constraints (26.21). A local minimum is defined similarly. Let F : U ×
R → Rm+1 be defined by
 
f (x) − a
 g1 (x) 
 
F (x,a) ≡  .. .
 (26.22)
 . 
gm (x)
26.2. THE METHOD OF LAGRANGE MULTIPLIERS 491

Now consider the m + 1 × n matrix

 
fx1 (x0 ) · · · fxn (x0 )
 g1x1 (x0 ) · · · g1xn (x0 )
 

 .. .. .
 
 . . 
gmx1 (x0 ) · · · gmxn (x0 )

If this matrix has rank m + 1 then some m + 1 × m + 1 submatrix has nonzero deter-
minant. It follows from the implicit function theorem, there exists m + 1 variables
xi1 , · · · , xim+1 such that the system

F (x,a) = 0 (26.23)

specifies these m + 1 variables as a function of the remaining n − (m + 1) variables

and a in an open set of Rn−m . Thus there is a solution (x,a) to 26.23 for some x
close to x0 whenever a is in some open interval. Therefore, x0 cannot be either a
local minimum or a local maximum. It follows that if x0 is either a local maximum
or a local minimum, then the above matrix must have rank less than m + 1. It follows
that some row is a linear combination of the others. Thus there exist m scalars,

λ 1, · · · , λ m,

and a scalar µ, not all zero such that

     
fx1 (x0 ) g1x1 (x0 ) gmx1 (x0 )
 .
..

 = λ1 
 ..  
+···+λm  .. 
.
µ   .   .  (26.24)
fxn (x0 ) g1xn (x0 ) gmxn (x0 )

If the rank of the matrix

 
g1x1 (x0 ) · · · gmx1 (x0 )
 .. .. 

 . .

 (26.25)
g1xn (x0 ) · · · gmxn (x0 )

is m, then we can choose µ = 1 because the columns span Rm . Thus there are scalars
λ i such that
     
fx1 (x0 ) g1x1 (x0 ) gmx1 (x0 )
 ..  
 = λ1  .. 
+···+λm 
 .. 

 .   .   .

 (26.26)
fxn (x0 ) g1xn (x0 ) gmxn (x0 )

at every point x0 which is either a local maximum or a local minimum. This proves
the following theorem.

Theorem 26.2.1 Let U be an open subset of Rn and let f : U → R be a C1 function. Then

if x0 ∈ U is either a local maximum or local minimum of f subject to the constraints 26.21,
then 26.24 must hold for some scalars µ, λ 1 , · · · , λ m not all equal to zero. If the rank of the
matrix in 26.25 is m, it follows 26.26 holds for some choice of the λ i .
492 CHAPTER 26. IMPLICIT FUNCTION THEOREM*

26.3 The Local Structure Of C1 Mappings∗

In linear algebra it is shown that every invertible matrix can be written as a product of
elementary matrices, those matrices which are obtained from doing a row operation to the
identity matrix. Two of the row operations produce a matrix which will change exactly one
entry of a vector when it is multiplied by the elementary matrix. The other row operation
involves switching two rows and this has the effect of switching two entries in a vector
when multiplied on the left by the elementary matrix. Thus, in terms of the effect on a
vector, the mapping determined by the given matrix can be considered as a composition of
mappings which either flip two entries of the vector or change exactly one. A similar local
result is available for nonlinear mappings. I found this interesting result in the advanced
calculus book by Rudin.

Definition 26.3.1 Let U be an open set in Rn and let G : U → Rn . Then G is called

primitive if it is of the form
 T
G (x) = x1 · · · α (x) · · · xn .

Thus, G is primitive if it only changes one of the variables. A function F : Rn → Rn is

called a flip if

F (x1 , · · · , xk , · · · , xl , · · · , xn ) = (x1 , · · · , xl , · · · , xk , · · · , xn )T .

Thus a function is a flip if it interchanges two coordinates. Also, for m = 1, 2, · · · , n, define

 T
Pm (x) ≡ x1 x2 · · · xm 0 · · · 0

It turns out that if h (0) = 0, Dh (0)−1 exists, and h is C1 on U, then h can be written
as a composition of primitive functions and flips. This is a very interesting application of
the inverse function theorem.

Theorem 26.3.2 Let h : U → Rn be a C1 function with h (0) = 0 Dh (0)−1 exists. Then

there is an open set V ⊆ U containing 0, flips F 1 , · · · , F n−1 , and primitive functions
Gn , Gn−1 , · · · , G1 such that for x ∈ V,

h (x) = F 1 ◦ · · · ◦ F n−1 ◦ Gn ◦ Gn−1 ◦ · · · ◦ G1 (x) .

The primitive function G j leaves xi unchanged for i ̸= j.

Proof: Let  T
h1 (x) ≡ h (x) = α 1 (x) · · · α n (x)

 T
Dh (0) e1 = α 1,1 (0) · · · α n,1 (0)

where α k,1 denotes ∂∂αx k . Since Dh (0) is one to one, the right side of this expression cannot
1
be zero. Hence there exists some k such that α k,1 (0) ̸= 0. Now define
 T
G1 (x) ≡ α k (x) x2 ··· xn
26.3. THE LOCAL STRUCTURE OF C1 MAPPINGS∗ 493

Then the matrix of DG1 (0) is of the form

 
α k,1 (0) · · · ··· α k,n (0)
0 1 0
 
 
 .
.. .. .. 
.
 
 . 
0 0 ··· 1

and its determinant equals α k,1 (0) ̸= 0. Therefore, by the inverse function theorem, there
exists an open set U1 , containing 0 and an open set V2 containing 0 such that G1 (U1 ) = V2
and G1 is one to one and onto, such that it and its inverse are both C1 . Let F 1 denote the
flip which interchanges xk with x1 . Now define
h2 (y) ≡ F 1 ◦ h1 ◦ G−1
1 (y)

Thus
h2 (G1 (x)) ≡ F 1 ◦ h1 (x) (26.27)
 T
= α k (x) · · · α 1 (x) · · · α n (x)

Therefore,  T
P1 h2 (G1 (x)) = α k (x) 0 ··· 0 .
Also  T
P1 (G1 (x)) = α k (x) 0 ··· 0

so P1 h2 (y) = P1 (y) for all y ∈ V2 . Also, h2 (0) = 0 and Dh2 (0)−1 exists because of the
definition of h2 above and the chain rule. Since F 21 = I, the identity map, it follows from
(26.27) that
h (x) = h1 (x) = F 1 ◦ h2 ◦ G1 (x) . (26.28)
Note that on an open set V2 ≡ G1 (U1 ) containing the origin, h2 leaves the first entry un-
changed. This is what P1 h2 (G1 (x)) = P1 (G1 (x)) says. In contrast, h1 = h left possibly
no entries unchanged.
Suppose then, that for m ≥ 2, hm leaves the first m − 1 entries unchanged,
Pm−1 hm (x) = Pm−1 (x) (26.29)

for all x ∈ Um , an open subset of U containing 0, and hm (0) = 0, Dhm (0)−1 exists. From
(26.29), hm (x) must be of the form
 T
hm (x) = x1 · · · xm−1 α 1 (x) · · · α n (x)

where these α k are different than the ones used earlier. Then
 T
Dhm (0) em = 0 · · · 0 α 1,m (0) · · · α n,m (0) ̸ 0
=

because Dhm (0)−1 exists. Therefore, there exists a k ≥ m such that α k,m (0) ̸= 0, not the
same k as before. Define
 T
Gm (x) ≡ x1 · · · xm−1 α k (x) · · · xn (26.30)
494 CHAPTER 26. IMPLICIT FUNCTION THEOREM*

so a change in Gm occurs only in the mth slot. Then Gm (0) = 0 and DGm (0)−1 exists
similar to the above. In fact

det (DGm (0)) = α k,m (0) .

Therefore, by the inverse function theorem, there exists an open set Vm+1 containing 0 such
that Vm+1 = Gm (Um ) with Gm and its inverse being one to one, continuous and onto. Let
F m be the flip which flips xm and xk . Then define hm+1 on Vm+1 by

hm+1 (y) = F m ◦ hm ◦ G−1

m (y) .

Thus for x ∈ Um ,
hm+1 (Gm (x)) = (F m ◦ hm ) (x) . (26.31)
and consequently, since F 2m = I,

F m ◦ hm+1 ◦ Gm (x) = hm (x) (26.32)

It follows

Pm hm+1 (Gm (x)) = Pm (F m ◦ hm ) (x)

 T
= x1 · · · xm−1 α k (x) 0 ··· 0

and  T
Pm (Gm (x)) = x1 ··· xm−1 α k (x) 0 ··· 0 .

Therefore, for y ∈ Vm+1 ,

Pm hm+1 (y) = Pm (y) .
As before, hm+1 (0) = 0 and Dhm+1 (0)−1 exists. Therefore, we can apply (26.32) repeat-
edly, obtaining the following:

h (x) = F 1 ◦ h2 ◦ G1 (x)
= F 1 ◦ F 2 ◦ h3 ◦ G2 ◦ G1 (x)
..
.
= F 1 ◦ · · · ◦ F n−1 ◦ hn ◦ Gn−1 ◦ · · · ◦ G1 (x)

where hn fixes the first n − 1 entries,

 T
Pn−1 hn (x) = Pn−1 (x) = x1 ··· xn−1 0 ,

and so hn (x) is a primitive mapping of the form

 T
hn (x) = x1 ··· xn−1 α (x) .

Therefore, define the primitive function Gn (x) to equal hn (x). ■

 Part II

Differential Equations

495
Chapter 27

Determinants

27.1 Basic Techniques And Properties

To begin with, the basic computations and properties of determinants are discussed. After
this, complete proofs are given for those who are interested.
Actually, determinants were studied before the modern theory of linear algebra. They
are very important in differential equations. Much of what was done earlier concerning
eigenvalues and eigenvectors could have been presented without determinants, but things
like the Wronskian are given to be certain determinants and so it is important to discuss
them.

27.1.1 Cofactors And 2 × 2 Determinants

Let A be an n × n matrix. The determinant of A, denoted as det (A) is a number. If the
matrix is a 2×2 matrix, this number is very easy to find.
!
a b
Definition 27.1.1 Let A = . Then det (A) ≡ ad − cb. The determinant is also
c d
often denoted by enclosing the matrix with two vertical lines. Thus
!
a b a b
det = .
c d c d
!
2 4
Example 27.1.2 Find det .
−1 6

From the definition this is just (2) (6) − (−1) (4) = 16.
Having defined what is meant by the determinant of a 2 × 2 matrix, what about a 3 × 3
matrix?

Definition 27.1.3 Suppose A is a 3 × 3 matrix. The i jth minor, denoted as minor(A)i j ,

is the determinant of the 2 × 2 matrix which results from deleting the ith row and the jth
column.

497
498 CHAPTER 27. DETERMINANTS

Example 27.1.4 Consider the matrix

 
1 2 3
 4 3 2 .
 

3 2 1

The (1, 2) minor is the determinant of the 2 × 2 matrix which results when you delete the
first row and the second column. This minor is therefore
!
4 2
det = −2.
3 1

The (2, 3) minor is the determinant of the 2 × 2 matrix which results when you delete the
second row and the third column. This minor is therefore
!
1 2
det = −4.
3 2

Definition 27.1.5 Suppose A is a 3 × 3 matrix. The i jth cofactor is defined to be (−1)i+ j ×

i j minor . In words, you multiply (−1)i+ j times the i jth minor to get the i jth cofactor.
th

The cofactors of a matrix are so important that special notation is appropriate when re-
ferring to them. The i jth cofactor of a matrix A will be denoted by cof (A)i j . It is also
convenient to refer to the cofactor of an entry of a matrix as follows. For ai j an entry of
the matrix, its cofactor is just cof (A)i j . Thus the cofactor of the i jth entry is just the i jth
cofactor.

Example 27.1.6 Consider the matrix

 
1 2 3
A= 4 3 2 .
 

3 2 1

The (1, 2) minor is the determinant of the 2 × 2 matrix which results when you delete the
first row and the second column. This minor is therefore
!
4 2
det = −2.
3 1

It follows !
1+2 4 2
cof (A)12 = (−1) det = (−1)1+2 (−2) = 2
3 1
The (2, 3) minor is the determinant of the 2 × 2 matrix which results when you delete the
second row and the third column. This minor is therefore
!
1 2
det = −4.
3 2
27.1. BASIC TECHNIQUES AND PROPERTIES 499

Therefore, !
2+3 1 2
cof (A)23 = (−1) det = (−1)2+3 (−4) = 4.
3 2
Similarly, !
2+2 1 3
cof (A)22 = (−1) det = −8.
3 1

Definition 27.1.7 The determinant of a 3 × 3 matrix A, is obtained by picking a row (col-

umn) and taking the product of each entry in that row (column) with its cofactor and adding
these up. This process when applied to the ith row (column) is known as expanding the de-
terminant along the ith row (column).

Example 27.1.8 Find the determinant of

 
1 2 3
A= 4 3 2 .
 

3 2 1

Here is how it is done by “expanding along the first column”.

cof(A)11 cof(A)21 cof(A)31
z }| { z }| { z }| {
3 2 2 3 2 3
1(−1)1+1 + 4(−1)2+1 + 3(−1)3+1 = 0.
2 1 2 1 3 2

You see, we just followed the rule in the above definition. We took the 1 in the first column
and multiplied it by its cofactor, the 4 in the first column and multiplied it by its cofactor,
and the 3 in the first column and multiplied it by its cofactor. Then we added these numbers
together.
You could also expand the determinant along the second row as follows.
cof(A)21 cof(A)22 cof(A)23
z }| { z }| { z }| {
2 3 1 3 1 2
4(−1)2+1 + 3(−1)2+2 + 2(−1)2+3 = 0.
2 1 3 1 3 2

Observe this gives the same number. You should try expanding along other rows and
columns. If you don’t make any mistakes, you will always get the same answer.
What about a 4 × 4 matrix? You know now how to find the determinant of a 3 × 3
matrix. The pattern is the same.

Definition 27.1.9 Suppose A is a 4 × 4 matrix. The i jth minor is the determinant of the
3 × 3 matrix you obtain when you delete the ith
row and the jth column. The i jth cofactor,
cof (A)i j is defined to be (−1)i+ j × i jth minor . In words, you multiply (−1)i+ j times the
i jth minor to get the i jth cofactor.

Definition 27.1.10 The determinant of a 4 × 4 matrix A, is obtained by picking a row (col-

umn) and taking the product of each entry in that row (column) with its cofactor and adding
these together. This process when applied to the ith row (column) is known as expanding
the determinant along the ith row (column).
500 CHAPTER 27. DETERMINANTS

Example 27.1.11 Find det (A) where

 
1 2 3 4
 5 4 2 3 
A=
 

 1 3 4 5 
3 4 3 2

As in the case of a 3 × 3 matrix, you can expand this along any row or column. Lets
pick the third column. det (A) =

5 4 3 1 2 4
1+3 2+3
3 (−1) 1 3 5 + 2 (−1) 1 3 5
3 4 2 3 4 2
1 2 4 1 2 4
3+3 4+3
+4 (−1) 5 4 3 + 3 (−1) 5 4 3 .
3 4 2 1 3 5

Now you know how to expand each of these 3 × 3 matrices along a row or a column. If you
do so, you will get −12 assuming you make no mistakes. You could expand this matrix
along any row or any column and assuming you make no mistakes, you will always get
the same thing which is defined to be the determinant of the matrix A. This method of
evaluating a determinant by expanding along a row or a column is called the method of
Laplace expansion.
Note that each of the four terms above involves three terms consisting of determinants
of 2×2 matrices and each of these will need 2 terms. Therefore, there will be 4×3×2 = 24
terms to evaluate in order to find the determinant using the method of Laplace expansion.
Suppose now you have a 10 × 10 matrix and you follow the above pattern for evaluating
determinants. By analogy to the above, there will be 10! = 3, 628 , 800 terms involved in
the evaluation of such a determinant by Laplace expansion along a row or column. This is
a lot of terms.
In addition to the difficulties just discussed, you should regard the above claim that you
always get the same answer by picking any row or column with considerable skepticism. It
is incredible and not at all obvious. However, it requires a little effort to establish it. This
is done in the section on the theory of the determinant.

Definition 27.1.12 Let A = (ai j ) be an n × n matrix and suppose the determinant of a

(n − 1) × (n − 1) matrix has been defined. Then a new matrix called the cofactor ma-
trix, cof (A) is defined by cof (A) = (ci j ) where to obtain ci j delete the ith row and the jth
column of A, take the determinant of the (n − 1) × (n − 1) matrix which results, (This is
called the i jth
minor of A. ) and then multiply this number by (−1)i+ j . Thus (−1)i+ j ×
the i jth minor equals the i jth cofactor. To make the formulas easier to remember, cof (A)i j
will denote the i jth entry of the cofactor matrix.

With this definition of the cofactor matrix, here is how to define the determinant of an
n × n matrix.
27.1. BASIC TECHNIQUES AND PROPERTIES 501

Definition 27.1.13 Let A be an n × n matrix where n ≥ 2 and suppose the determinant of

an (n − 1) × (n − 1) has been defined. Then
n n
det (A) = ∑ ai j cof (A)i j = ∑ ai j cof (A)i j . (27.1)
j=1 i=1

The first formula consists of expanding the determinant along the ith row and the second
expands the determinant along the jth column.
Theorem 27.1.14 Expanding the n × n matrix along any row or column always gives the
same answer so the above definition is a good definition.

27.1.2 The Determinant Of A Triangular Matrix

Notwithstanding the difficulties involved in using the method of Laplace expansion, certain
types of matrices are very easy to deal with.
Definition 27.1.15 A matrix M, is upper triangular if Mi j = 0 whenever i > j. Thus such
a matrix equals zero below the main diagonal, the entries of the form Mii , as shown.
 
∗ ∗ ··· ∗
 0 ∗ . . . ... 
 
 
 .
 . .. ..

 . . . ∗ 


0 ··· 0 ∗
A lower triangular matrix is defined similarly as a matrix for which all entries above the
main diagonal are equal to zero.
You should verify the following using the above theorem on Laplace expansion.
Corollary 27.1.16 Let M be an upper (lower) triangular matrix. Then det (M) is obtained
by taking the product of the entries on the main diagonal.
Example 27.1.17 Let
 
1 2 3 77
 0 2 6 7 
A=
 

 0 0 3 33.7 
0 0 0 −1
Find det (A) .
From the above corollary, it suffices to take the product of the diagonal elements. Thus
det (A) = 1 × 2 × 3 × (−1) = −6. Without using the corollary, you could expand along the
first column. This gives
2 6 7 2 3 77
2+1
1 0 3 33.7 + 0 (−1) 0 3 33.7
0 0 −1 0 0 −1
2 3 77 2 3 77
+0 (−1)3+1 2 6 7 + 0 (−1)4+1
2 6 7
0 0 −1 0 3 33.7
502 CHAPTER 27. DETERMINANTS

and the only nonzero term in the expansion is

2 6 7
1 0 3 33.7 .
0 0 −1

Now expand this along the first column to obtain

 
3 33.7 6 7
 2× + 0 (−1)2+1
−1 −1

 0 0 
1× 
 6 7 
+0 (−1)3+1
 
3 33.7
3 33.7
= 1×2×
0 −1

Next expand this last determinant along the first column to obtain the above equals

1 × 2 × 3 × (−1) = −6

which is just the product of the entries down the main diagonal of the original matrix. It
works this way in general.

27.1.3 Properties Of Determinants

There are many properties satisfied by determinants. Some of these properties have to do
with row operations. Recall the row operations.

Definition 27.1.18 The row operations consist of the following

1. Switch two rows.

2. Multiply a row by a nonzero number.
3. Replace a row by a multiple of another row added to itself.

Theorem 27.1.19 Let A be an n × n matrix and let A1 be a matrix which results from
multiplying some row of A by a scalar c. Then c det (A) = det (A1 ).
! !
1 2 2 4
Example 27.1.20 Let A = , A1 = . det (A) = −2, det (A1 ) = −4.
3 4 3 4

Theorem 27.1.21 Let A be an n × n matrix and let A1 be a matrix which results from
switching two rows of A. Then det (A) = − det (A1 ) . Also, if one row of A is a multiple of
another row of A, then det (A) = 0.
! !
1 2 3 4
Example 27.1.22 Let A = and let A1 = . det A = −2, det (A1 ) = 2.
3 4 1 2
27.1. BASIC TECHNIQUES AND PROPERTIES 503

Theorem 27.1.23 Let A be an n × n matrix and let A1 be a matrix which results from
applying row operation 3. That is you replace some row by a multiple of another row
added to itself. Then det (A) = det (A1 ).
! !
1 2 1 2
Example 27.1.24 Let A = and let A1 = . Thus the second row of
3 4 4 6
A1 is one times the first row added to the second row. det (A) = −2 and det (A1 ) = −2.

Theorem 27.1.25 In Theorems 27.1.19 - 27.1.23 you can replace the word, “row” with
the word “column”.

There are two other major properties of determinants which do not involve row opera-
tions.

Theorem 27.1.26 Let A and B be two n × n matrices. Then

det (AB) = det (A) det (B).
Also,
det (A) = det AT .

Example 27.1.27 Compare det (AB) and det (A) det (B) for
! !
1 2 3 2
A= ,B = .
−3 2 4 1

First ! ! !
1 2 3 2 11 4
AB = =
−3 2 4 1 −1 −4
and so !
11 4
det (AB) = det = −40.
−1 −4
Now ! !
1 2 3 2
det (A) = det = 8, det (B) = det = −5.
−3 2 4 1
Thus det (A) det (B) = 8 × (−5) = −40.

27.1.4 Finding Determinants Using Row Operations

Theorems 27.1.23 - 27.1.25 can be used to find determinants using row operations. As
pointed out above, the method of Laplace expansion will not be practical for any matrix of
large size. Here is an example in which all the row operations are used.

Example 27.1.28 Find the determinant of the matrix

 
1 2 3 4
 5 1 2 3 
A=
 

 4 5 4 3 
2 2 −4 5
504 CHAPTER 27. DETERMINANTS

Replace the second row by (−5) times the first row added to it. Then replace the third
row by (−4) times the first row added to it. Finally, replace the fourth row by (−2) times
the first row added to it. This yields the matrix
 
1 2 3 4
 0 −9 −13 −17 
B=
 
−3 −8 −13

 0 
0 −2 −10 −3

and from Theorem 27.1.23, it has the same determinant as A. Now using other row opera-
tions, det (B) = −1


3 det (C) where
 
1 2 3 4
 0 0 11 22 
C= .
 
 0 −3 −8 −13 
0 6 30 9

The second row was replaced by (−3) times the third row added to the second row. By
Theorem 27.1.23 this didn’t change the value of the determinant. Then the last row was
multiplied by (−3) . By Theorem 27.1.19 the resulting matrix has a determinant which is
(−3) times the determinant of the un-multiplied matrix. Therefore, we multiplied by −1/3
to retain the correct value. Now replace the last row with 2 times the third added to it.
This does not change the value of the determinant by Theorem 27.1.23. Finally switch
the third and second rows. This causes the determinant to be multiplied by (−1) . Thus
det (C) = − det (D) where
 
1 2 3 4
 0 −3 −8 −13 
D=
 

 0 0 11 22 
0 0 14 −17

You could do more row operations or you could note that this can be easily expanded along
the first column followed by expanding the 3×3 matrix which results along its first column.
Thus
11 22
det (D) = 1 (−3) = 1485
14 −17

and so det (C) = −1485 and det (A) = det (B) = −1

3 (−1485) = 495.

Example 27.1.29 Find the determinant of the matrix

 
1 2 3 2

 1 −3 2 1 

 
 2 1 2 5 
3 −4 1 2
27.2. APPLICATIONS 505

Replace the second row by (−1) times the first row added to it. Next take −2 times the
first row and add to the third and finally take −3 times the first row and add to the last row.
This yields
 
1 2 3 2
 0 −5 −1 −1 
.
 
 0 −3 −4 1 


0 −10 −8 −4
By Theorem 27.1.23 this matrix has the same determinant as the original matrix. Remem-
ber you can work with the columns also. Take −5 times the last column and add to the
second column. This yields
 
1 −8 3 2
 0 0 −1 −1 
 
 0 −8 −4 1 
 

0 10 −8 −4
By Theorem 27.1.25 this matrix has the same determinant as the original matrix. Now take
(−1) times the third row and add to the top row. This gives.
 
1 0 7 1

 0 0 −1 −1 

−8 −4
 
 0 1 
0 10 −8 −4

which by Theorem 27.1.23 has the same determinant as the original matrix. Lets expand
it now along the first column. This yields the following for the determinant of the original
matrix.  
0 −1 −1
det  −8 −4 1 
 

10 −8 −4
which equals ! !
−1 −1 −1 −1
8 det + 10 det = −82
−8 −4 −4 1
We suggest you do not try to be fancy in using row operations. That is, stick mostly to
the one which replaces a row or column with a multiple of another row or column added to
it. Also note there is no way to check your answer other than working the problem more
than one way. To be sure you have gotten it right you must do this.

27.2 Applications
27.2.1 A Formula For The Inverse
The definition of the determinant in terms of Laplace expansion along a row or column
also provides a way to give a formula for the inverse of a matrix. Recall the definition of
the inverse of a matrix in Definition 8.6.2 on Page 144. Also recall the definition of the
506 CHAPTER 27. DETERMINANTS

cofactor matrix given in Definition 27.1.12 on Page 500. This cofactor matrix was just the
matrix which results from replacing the i jth entry of the matrix with the i jth cofactor.
The following theorem says that to find the inverse, take the transpose of the cofactor
matrix and divide by the determinant. The transpose of the cofactor matrix is called the
adjugate or sometimes the classical adjoint of the matrix A. In other words, A−1 is equal
to one divided by the determinant of A times the adjugate matrix of A. This is what the
following theorem says with more precision.
 
Theorem 27.2.1 A−1 exists if and only if det(A) ̸= 0. If det(A) ̸= 0, then A−1 = a−1 ij
where
a−1 −1
i j = det(A) cof (A) ji

for cof (A)i j the i jth cofactor of A.

Example 27.2.2 Find the inverse of the matrix

 
1 2 3
A= 3 0 1 
 

1 2 1

First find the determinant of this matrix. Using Theorems 27.1.23 - 27.1.25 on Page
503, the determinant of this matrix equals the determinant of the matrix
 
1 2 3
 0 −6 −8 
 

0 0 −2

which equals 12. The cofactor matrix of A is

 
−2 −2 6
 4 −2 0 .
 

2 8 −6

Each entry of A was replaced by its cofactor. Therefore, from the above theorem, the
inverse of A should equal
 T  
−2 −2 6 −1/6 1/3 1/6
1 
 4 −2 0  =  −1/6 −1/6 2/3  .
  
12
2 8 −6 1/2 0 −1/2

Does it work? You should check to see if it does. When the matrices are multiplied
    
−1/6 1/3 1/6 1 2 3 1 0 0
 −1/6 −1/6 2/3   3 0 1 = 0 1 0 
    

1/2 0 −1/2 1 2 1 0 0 1

and so it is correct.
27.2. APPLICATIONS 507

Example 27.2.3 Find the inverse of the matrix

 
1 1
0
 2 2

 1 1
− 12

 − 
A= 6 3 
 
 5 2 
 −
6 3 − 12 

First find its determinant. This determinant is 16 . The inverse is therefore equal to
 T
1/3 −1/2 −1/6 −1/2 −1/6 1/3
−

 2/3 −1/2 −5/6 −1/2 −5/6 2/3


 

0 1/2 1/2 1/2 1/2 0 
6 − −  .
 
 2/3 −1/2 −5/6 −1/2 −5/6 2/3 
 
 
 0 1/2 1/2 1/2 1/2 0 
−
1/3 −1/2 −1/6 −1/2 −1/6 1/3
Expanding all the 2 × 2 determinants this yields
 T  
1/6 1/3 1/6 1 2 −1
6  1/3 1/6 −1/3  =  2 1 1 
   

−1/6 1/6 1/6 1 −2 1

Always check your work.
    
1 2 −1 1/2 0 1/2 1 0 0
 2 1 1   −1/6 1/3 −1/2  =  0 1 0 
    

1 −2 1 −5/6 2/3 −1/2 0 0 1

and so we got it right. If the result of multiplying these matrices had been something other
than the identity matrix, you would know there was an error. When this happens, you
need to search for the mistake if you are interested in getting the right answer. A common
mistake is to forget to take the transpose of the cofactor matrix.
Proof of Theorem 27.2.1: From the definition of the determinant in terms of expansion
along a column, and letting (air ) = A, if det (A) ̸= 0,
n
∑ air cof (A)ir det(A)−1 = det(A) det(A)−1 = 1.
i=1

Now consider
n
∑ air cof (A)ik det(A)−1
i=1
when k ̸= r. Replace the kth column with the rth column to obtain a matrix Bk whose
determinant equals zero by Theorem 27.1.21. However, expanding this matrix Bk along the
kth column yields
n
0 = det (Bk ) det (A)−1 = ∑ air cof (A)ik det (A)−1
i=1
508 CHAPTER 27. DETERMINANTS

Summarizing, (
n
−1 1 if r = k
∑ air cof (A)ik det (A) = δ rk ≡ .
i=1 0 if r ̸= k
Now
n n
∑ air cof (A)ik = ∑ air cof (A)Tki
i=1 i=1
T
which is the krth entry of cof (A) A. Therefore,

cof (A)T
A = I. (27.2)
det (A)

Using the other formula in Definition 27.1.13, and similar reasoning,

n
∑ ar j cof (A)k j det (A)−1 = δ rk
j=1

Now
n n
∑ ar j cof (A)k j = ∑ ar j cof (A)Tjk
j=1 j=1
T
which is the rkth entry of A cof (A) . Therefore,

cof (A)T
A = I, (27.3)
det (A)
 
and it follows from 27.2 and 27.3 that A−1 = a−1
i j , where

−1
a−1
i j = cof (A) ji det (A) .

In other words,
cof (A)T
A−1 = .
det (A)

Now suppose A−1 exists. Then by Theorem 27.1.26,

1 = det (I) = det AA−1 = det (A) det A−1

so det (A) ̸= 0. ■
This way of finding inverses is especially useful in the case where it is desired to find
the inverse of a matrix whose entries are functions.

Example 27.2.4 Suppose

 
et 0 0
A (t) =  0 cost sint 
 

0 − sint cost

Show that A (t)−1 exists and then find it.

27.2. APPLICATIONS 509

First note det (A (t)) = et ̸= 0 so A (t)−1 exists. The cofactor matrix is

 
1 0 0
C (t) =  0 et cost et sint 
 

0 −et sint et cost

and so the inverse is
 T  
1 0 0 e−t 0 0
1 t
 0 e cost et sint  =  0 cost − sint  .
  
et
0 −et sint et cost 0 sint cost

27.2.2 Finding Eigenvalues Using Determinants

It was shown in Theorem 27.2.1 that A−1 exists if and only if det (A) ̸= 0 when there is even
a formula for the inverse. Recall also that an eigenvector for λ is a nonzero vector x such
that Ax = λ x where λ is called an eigenvalue. Thus you have (A − λ I) x = 0 for x ̸= 0.
If (A − λ I)−1 were to exist, then you could multiply by it on the left and obtain x = 0
after all. Therefore, it must be the case that det (A − λ I) = 0. This yields a polynomial of
degree n equal to 0. This polynomial is called the characteristic polynomial. For example,
consider  
1 −1 −1
 0 3 2 
 

0 −1 0
You need to have
   
1 −1 −1 1 0 0
det  0 3 2 −λ  0 1 0  = 0
   

0 −1 0 0 0 1
That on the left equals a polynomial of degree 3 which when factored yields
(1 − λ ) (λ − 1) (λ − 2)
Therefore, the possible eigenvalues are 1,1,2. Note how the 1 is listed twice. This is because
it occurs twice as a root of the characteristic polynomial. Also, if M −1 does not exist where
M is an n × n matrix, then this means that the columns of M cannot be lineraly independent
since if they were, then by Theorem 11.5.2 M −1 would exist. Thus if A − λ I fails to have
an inverse as above, then the columns are not independent and so there exists a nonzero x
such that (A − λ I) x = 0. Thus we have the following proposition.
Proposition 27.2.5 The eigenvalues of an n × n matrix are the roots of det (A − λ I) = 0.
Corresponding to each of these λ is an eigenvector.
Note that if A = S−1 BS, then A, B have the same characteristic polynomial, hence the
same eigenvalues. (They might have different eigenvectors and usually will.) To see this,
note that from the properties of determinants
det (A − λ I) = det S−1 BS − λ S−1 IS = det S−1 (B − λ I) S

= det S−1 det (B − λ I) det (S) = det S−1 S det (B − λ I)

= det (I) det (B − λ I) = det (B − λ I) (27.4)

510 CHAPTER 27. DETERMINANTS

Definition 27.2.6 Let A be n × n. Then trace (A) ≡ ∑ni=1 Aii .

Proposition 27.2.7 Let A be m × n and let B be n × m. Then trace (AB) = trace (BA) . Also
for square matrices A, B, if A = S−1 BS, then trace (A) = trace (B). Also det (A) = det (B).

Proof: trace (AB) ≡ ∑i ∑ j Ai j B ji = ∑ j ∑i B ji Ai j ≡ trace (BA) . Now let A, B be as de-

scribed. Then

trace (A) = trace S−1 BS = trace (BS) S−1

= trace B SS−1 = trace (B)

As to the claim about the determinant, it follows from the properties of the determinant that

det (A) = det S−1 BS = det S−1 det (B) det (S)

= det (B) det S−1 S = det (B) ■

These two, the trace and the determinant are two of the so called principal invariants of a
3 × 3 matrix. The reason these are called invariants is that they are the same for A and B if
these two are related as described in the above proposition. In this case, the other principal
invariant is
1 1
(trace (A))2 − trace A2


2 2
It turns out these are related to the coefficients of the characteristic polynomial defined as

det (A − λ I)

and discussed below.

To see this last is also an invariant, the above proposition implies
1 1 1

2 1 
2 
(trace (A))2 − trace A2 trace S−1 BS − trace S−1 BS


=
2 2 2 2
1 1
(trace (B)) − trace S−1 BS S−1 BS
2



=
2 2
1 1
(trace (B)) − trace S−1 B2 S
2

=
2 2
1 2 1
(trace (B)) − trace B2


=
2 2
The physical reason these are important is that their invariance implies they do not change
when one uses a different coordinate system to describe points. That which is physi-
cally meaningful cannot depend on coordinate system because such coordinate systems
are purely artificial constructions used to identify points. Therefore, the principal invari-
ants are good for formulating physical laws. This is as far as we go here. To see much more
on these ideas, you should take a course on continuum mechanics. However, the trace and
determinant also have a very interesting relation to eigenvalues.

Theorem 27.2.8 The trace of a matrix is the sum of its eigenvalues listed according to
multiplicity as a root of the characteristic polynomial. Also, the determinant of the matrix
equals the product of its eigenvalues.
27.2. APPLICATIONS 511

Proof: Let A be an n × n matrix. By Schur’s theorem, there is unitary U such that

U ∗ AU = T

where T is upper triangular. The characteristic polynomial of T is

(λ − µ 1 ) (λ − µ 2 ) · · · (λ − µ n )

where µ 1 , · · · , µ n are the diagonal entries of T . From the above discussion 27.4, these
must also be the eigenvalues of A listed according to multiplicity since these two matrices
A, T have the same characteristic polynomial. By Proposition 27.2.7 A, T have the same
determinant, but since T is upper triangular, the product of its diagonal entries is the product
of the eigenvalues of A and this is the common value of the determinant of these two
matrices. ■

Example 27.2.9 Find the eigenvalues of the following matrix.

 
10 12 1
A =  −8 −9 0 
 

7 8 0

You take det (A − λ I) and after much fussing with details, you get the following for the
characteristic polynomial.
−X 3 + X 2 + X − 1
Thus the eigenvalues are the roots of this polynomial. These roots are 1, 1, −1 when listed
according to multiplicity. You can use the above Theorem 27.2.8 as a way to check whether
you likely have this right. Indeed, when you add these together, you get 1. When you take
the trace of the above matrix, you get 1. This is a little reassurance that you didn’t make a
mistake. Note that the determinant of the above matrix is −1 which also equals the product
of these eigenvalues.

27.2.3 Cramer’s Rule

This formula for the inverse also implies a famous procedure known as Cramer’s rule.
Cramer’s rule gives a formula for the solutions, x, to a system of equations, Ax = y in the
special case that A is a square matrix. Note this rule does not apply if you have a system of
equations in which there is a different number of equations than variables.
In case you are solving a system of equations, Ax = y for x, it follows that if A−1
exists,
x = A−1 A x = A−1 (Ax) = A−1 y

thus solving the system. Now in the case that A−1 exists, there is a formula for A−1 given
above. Using this formula,
n n
1
xi = ∑ a−1
ij yj = ∑
det (A)
cof (A) ji y j .
j=1 j=1
512 CHAPTER 27. DETERMINANTS

By the formula for the expansion of a determinant along a column,

 
∗ · · · y1 · · · ∗
1  . .. .. 
xi = det  .. . . ,

det (A) 
∗ · · · yn · · · ∗

where here the ith column of A is replaced with the column vector (y1 · · · ·, yn )T , and the
determinant of this modified matrix is taken and divided by det (A). This formula is known
as Cramer’s rule.
PROCEDURE 27.2.10 Suppose A is an n × n matrix and it is desired to solve
the system Ax = y, y = (y1 , · · · , yn )T for x = (x1 , · · · , xn )T . Then Cramer’s rule says
det Ai
xi =
det A
where Ai is obtained from A by replacing the ith column of A with the column
(y1 , · · · , yn )T .
Find x, y if     
1 2 1 x 1
 3 2 1  y  =  2 .
    

2 −3 2 z 3
 
1 2 1
The determinant of the matrix of coefficients,  3 2 1  is −14. From Cramer’s
 

2 −3 2
rule, to get x, you replace the first column of A with the right side of the equation and take
its determinant and divide by the determinant of A. Thus
1 2 1
2 2 1
3 −3 2 1
x= =
−14 2

Now to find y, z, you do something similar.

1 1 1 1 2 1
3 2 1 3 2 2
2 3 2 1 2 −3 3 11
y= =− , z= =
−14 7 −14 14

You see the pattern. For large systems Cramer’s rule is less than useful if you want to
find an answer. This is because to use it you must evaluate determinants. However, you
have no practical way to evaluate determinants for large matrices other than row operations
and if you are using row operations, you might just as well use them to solve the system to
begin with. It will be a lot less trouble. Nevertheless, there are situations in which Cramer’s
rule is useful.
27.3. MATLAB AND DETERMINANTS 513

Example 27.2.11 Solve for z if

    
1 0 0 x 1
t et sint   y  =  t 
 0 e cost
    

0 −et sint et cost z t2

You could do it by row operations but it might be easier in this case to use Cramer’s
rule because the matrix of coefficients does not consist of numbers but of functions. Thus

1 0 1
0 et cost t
0 −et sint t2
z= = t ((cost)t + sint) e−t .
1 0 0
0 et cost et sint
0 −et sint et cost

You end up doing this sort of thing sometimes in ordinary differential equations in the
method of variation of parameters.

27.3 MATLAB And Determinants

MATLAB can find determinants. Here is an example.
>> A=[1,3,2,4;-5,7,2,3;2,3,7,11;1,2,3,4]; det(A)
Then press enter and you get
ans =
-102.0000
To enter a complex number 1 + 2i for example, you type: complex(1,2). However,
when matlab gives the answer, it will write it in the usual form 1 + 2i. If you have ma-
trices in which there are complex entries, you can go ahead and let matlab do the tedious
computations for you.

27.4 The Cayley Hamilton Theorem∗

Definition 27.4.1 Let A be an n × n matrix. The characteristic polynomial is defined as

qA (t) ≡ det (tI − A)

and the solutions to qA (t) = 0 are called eigenvalues. For A a matrix and p (t) = t n +
an−1t n−1 + · · · + a1t + a0 , denote by p (A) the matrix defined by

p (A) ≡ An + an−1 An−1 + · · · + a1 A + a0 I.

The explanation for the last term is that A0 is interpreted as I, the identity matrix.

The Cayley Hamilton theorem states that every matrix satisfies its characteristic equa-
tion, that equation defined by qA (t) = 0. It is one of the most important theorems in linear
514 CHAPTER 27. DETERMINANTS

algebra1 . The proof in this section is not the most general proof, but works well when the
field of scalars is R or C. The following lemma will help with its proof.

Lemma 27.4.2 Suppose for all |λ | large enough,

A0 + A1 λ + · · · + Am λ m = 0,

where the Ai are n × n matrices. Then each Ai = 0.

Proof: Multiply by λ −m to obtain

A0 λ −m + A1 λ −m+1 + · · · + Am−1 λ −1 + Am = 0.

Now let |λ | → ∞ to obtain Am = 0. With this, multiply by λ to obtain

A0 λ −m+1 + A1 λ −m+2 + · · · + Am−1 = 0.

Now let |λ | → ∞ to obtain Am−1 = 0. Continue multiplying by λ and letting λ → ∞ to

obtain that all the Ai = 0. ■
With the lemma, here is a simple corollary.

Corollary 27.4.3 Let Ai and Bi be n × n matrices and suppose

A0 + A1 λ + · · · + Am λ m = B0 + B1 λ + · · · + Bm λ m

for all |λ | large enough. Then Ai = Bi for all i. If Ai = Bi for each Ai , Bi then one can
substitute an n × n matrix M for λ and the identity will continue to hold.

Proof: Subtract and use the result of the lemma. The last claim is obvious by matching
terms. ■
With this preparation, here is a relatively easy proof of the Cayley Hamilton theorem.

Theorem 27.4.4 Let A be an n × n matrix and let q (λ ) ≡ det (λ I − A) be the characteristic

polynomial. Then q (A) = 0.

Proof: Let C (λ ) equal the transpose of the cofactor matrix of (λ I − A) for |λ | large.
(If |λ | is large enough, then λ cannot be in the finite list of eigenvalues of A and so for such
λ , (λ I − A)−1 exists.) Therefore, by Theorem 28.1.14

C (λ ) = q (λ ) (λ I − A)−1 .

Say
q (λ ) = a0 + a1 λ + · · · + λ n
Note that each entry in C (λ ) is a polynomial in λ having degree no more than n − 1. For
example, you might have something like
 2 
λ − 6λ + 9 3−λ 0
C (λ ) =  2λ − 6 λ 2 − 3λ 0
 

2
λ −1 λ −1 λ − 3λ + 2
1 A special case was first proved by Hamilton in 1853. The general case was announced by Cayley some time

later and a proof was given by Frobenius in 1878.

27.4. THE CAYLEY HAMILTON THEOREM∗ 515
     
9 3 0 −6 −1 0 1 0 0
2
=  −6 0 0 +λ  2 −3 0 +λ  0 1 0 
    

−1 −1 2 1 1 −3 0 0 1
Therefore, collecting the terms in the general case,
C (λ ) = C0 +C1 λ + · · · +Cn−1 λ n−1
for C j some n × n matrix. Then
 
C (λ ) (λ I − A) = C0 +C1 λ + · · · +Cn−1 λ n−1 (λ I − A) = q (λ ) I

Then multiplying out the middle term, it follows that for all |λ | sufficiently large,
a0 I + a1 Iλ + · · · + Iλ n = C0 λ +C1 λ 2 + · · · +Cn−1 λ n
h i
− C0 A +C1 Aλ + · · · +Cn−1 Aλ n−1

= −C0 A + (C0 −C1 A) λ + (C1 −C2 A) λ 2 + · · · + (Cn−2 −Cn−1 A) λ n−1 +Cn−1 λ n

Then, using Corollary 27.4.3, one can replace λ on both sides with A. Then the right side
is seen to equal 0. Hence the left side, q (A) I is also equal to 0. ■
It is good to keep in mind the following example when considering the above proof of
the Cayley Hamilton theorem. If p (λ ) = q (λ ) for all λ or for all λ large enough where
p (λ ) , q (λ ) are polynomials having matrix coefficients, then it is not necessarily the case
that p (A) = q (A) for A a matrix of an appropriate size. Let
! ! !
1 0 0 0 0 1
E1 = , E2 = ,N =
0 0 0 1 0 0

Then a short computation shows that for all complex λ ,

 
(λ I + E1 ) (λ I + E2 ) = λ 2 + λ I = (λ I + E2 ) (λ I + E1 )

However,
(NI + E1 ) (NI + E2 ) ̸= (NI + E2 ) (NI + E1 )
The reason this can take place is that N fails to commute with Ei . Of course a scalar
commutes with any matrix so there was no difficulty in obtaining that the matrix equation
held for arbitrary λ , but this factored equation does not continue to hold if λ is replaced
by a matrix. In the above proof of the Cayley Hamilton theorem, this issue was avoided by
considering only polynomials which are of the form C0 +C1 λ +· · · in which the polynomial
identity held because the corresponding matrix coefficients were equal. However, you can
also argue that in the above proof, the Ci each commute with A.
Theorem 27.4.5 Let q (λ ) be the characteristic polynomial and p (λ ) the minimal poly-
nomial. Then there is a polynomial l (λ ) which could be a constant such that q (λ ) =
l (λ ) p (λ ).
Proof: By the division algorithm, q (λ ) = p (λ ) l (λ )+r (λ ) where the degree of r (λ ) is
less than the degree of p (λ ) or else r (λ ) = 0. But then, substituting in A, you get r (A) = 0
which is impossible if its degree is less than that of p (λ ). It follows that r (λ ) = 0 and so
the claim is established. p (λ ) “divides” q (λ ). ■
516 CHAPTER 27. DETERMINANTS

27.5 Exercises
1. Find the determinants of the following matrices.
 
1 2 3 375.)
(a)  3 2 2  (The answer is 31.)
   
1 2 3 2
0 9 8  1 3 2 3 
(c)  , (The answer is
   
4 3 2  4 1 5 0 
(b)  1 7 8 (The answer is
 
1 2 1 2
3 −9 3 −2.)

2. Find the following determinant by expanding along the first row and second column.

1 2 1
2 1 3
2 1 1

3. Find the following determinant by expanding along the first column and third row.

1 2 1
1 0 1
2 1 1

4. Find the following determinant by expanding along the second row and first column.

1 2 1
2 1 3
2 1 1

5. Compute the determinant by cofactor expansion. Pick the easiest row or column to
use.
1 0 0 1
2 1 1 0
0 0 0 2
2 1 3 1
6. Find the determinant using row operations.

1 2 1
2 3 2
−4 1 2

7. Find the determinant using row operations.

2 1 3
2 4 2
1 4 −5
27.5. EXERCISES 517

8. Find the determinant using row operations.

1 2 1 2
3 1 −2 3
−1 0 3 1
2 3 2 −2

9. Find the determinant using row operations.

1 4 1 2
3 2 −2 3
−1 0 3 3
2 1 2 −2

10. Verify an example of each property of determinants found in Theorems 27.1.23 -

27.1.25 for 2 × 2 matrices.

11. An operation is done to get from the first matrix to the second. Identify what was
done and tell how it will affect the value of the determinant.
! !
a b a c
,
c d b d

12. An operation is done to get from the first matrix to the second. Identify what was
done and tell how it will affect the value of the determinant.
! !
a b c d
,
c d a b

13. An operation is done to get from the first matrix to the second. Identify what was
done and tell how it will affect the value of the determinant.
! !
a b a b
,
c d a+c b+d

14. An operation is done to get from the first matrix to the second. Identify what was
done and tell how it will affect the value of the determinant.
! !
a b a b
,
c d 2c 2d

15. An operation is done to get from the first matrix to the second. Identify what was
done and tell how it will affect the value of the determinant.
! !
a b b a
,
c d d c
518 CHAPTER 27. DETERMINANTS

16. Let A be an r × r matrix and suppose there are r − 1 rows (columns) such that all rows
(columns) are linear combinations of these r − 1 rows (columns). Show det (A) = 0.
17. Show det (aA) = an det (A) where here A is an n × n matrix and a is a scalar.

18. Illustrate with an example of 2 × 2 matrices that the determinant of a product equals
the product of the determinants.
19. Is it true that det (A + B) = det (A) + det (B)? If this is so, explain why it is so and if
it is not so, give a counter example.
20. An n × n matrix is called nilpotent if for some positive integer, k it follows Ak = 0.
If A is a nilpotent matrix and k is the smallest possible integer such that Ak = 0, what
are the possible values of det (A)?
21. A matrix is said to be orthogonal if AT A = I. Thus the inverse of an orthogonal ma-
trix is just its transpose. What are the possible values of det (A) if A is an orthogonal
matrix?

22. Fill in the missing entries to make the matrix orthogonal as in Problem 21.
 √ 
−1
√ √1 12
 2 6 6 
 
√1 .
 

 2 
 √ 
6
3

23. Let A and B be two n × n matrices. A ∼ B (A is similar to B) means there exists an

invertible matrix S such that A = S−1 BS. Show that if A ∼ B, then B ∼ A. Show also
that A ∼ A and that if A ∼ B and B ∼ C, then A ∼ C.
24. In the context of Problem 23 show that if A ∼ B, then det (A) = det (B) .

25. Two n × n matrices, A and B, are similar if B = S−1 AS for some invertible n × n
matrix S. Show that if two matrices are similar, they have the same characteristic
polynomials. The characteristic polynomial of an n × n matrix M is the polynomial,
det (λ I − M) .

26. Tell whether the statement is true or false.

(a) If A is a 3 × 3 matrix with a zero determinant, then one column must be a

multiple of some other column.
(b) If any two columns of a square matrix are equal, then the determinant of the
matrix equals zero.
(c) For A and B two n × n matrices, det (A + B) = det (A) + det (B) .
(d) For A an n × n matrix, det (3A) = 3 det (A)
(e) If A−1 exists then det A−1 = det (A)−1 .

(f) If B is obtained by multiplying a single row of A by 4 then det (B) = 4 det (A) .
(g) For A an n × n matrix, det (−A) = (−1)n det (A) .
27.5. EXERCISES 519

(h) If A is a real n × n matrix, then det AT A ≥ 0.

(i) Cramer’s rule is useful for finding solutions to systems of linear equations in
which there is an infinite set of solutions.
(j) If Ak = 0 for some positive integer, k, then det (A) = 0.
(k) If Ax = 0 for some x ̸= 0, then det (A) = 0.

27. Use Cramer’s rule to find the solution to x + 2y = 1, 2x − y = 2.

28. Use Cramer’s rule to find the solution to x + 2y + z = 1, 2x − y − z = 2, x + z = 1.
29. Here is a matrix,  
1 2 3
 0 2 1 
 

3 1 0
Determine whether the matrix has an inverse by finding whether the determinant is
non zero. If the determinant is nonzero, find the inverse using the formula for the
inverse which involves the cofactor matrix.
30. Here is a matrix,  
1 2 0
 0 2 1 
 

3 1 1
Determine whether the matrix has an inverse by finding whether the determinant is
non zero. If the determinant is nonzero, find the inverse using the formula for the
inverse which involves the cofactor matrix.
31. Here is a matrix,  
1 3 3
 2 4 1 
 

0 1 1
Determine whether the matrix has an inverse by finding whether the determinant is
non zero. If the determinant is nonzero, find the inverse using the formula for the
inverse which involves the cofactor matrix.
32. Here is a matrix,  
1 2 3
 0 2 1 
 

2 6 7
Determine whether the matrix has an inverse by finding whether the determinant is
non zero. If the determinant is nonzero, find the inverse using the formula for the
inverse which involves the cofactor matrix.
33. Here is a matrix,  
1 0 3
 1 0 1 
 

3 1 0
520 CHAPTER 27. DETERMINANTS

Determine whether the matrix has an inverse by finding whether the determinant is
non zero. If the determinant is nonzero, find the inverse using the formula for the
inverse which involves the cofactor matrix.
34. Use the formula for the inverse in terms of the cofactor matrix to find if possible the
inverses of the matrices
   
! 1 2 3 1 2 1
1 1
, 0 2 1 , 2 3 0 .
   
1 2
4 1 1 0 1 2

If the inverse does not exist, explain why.

35. Here is a matrix,  
1 0 0
 0 cost − sint 
 

0 sint cost
Does there exist a value of t for which this matrix fails to have an inverse? Explain.
36. Here is a matrix,  
1 t t2
 0 1 2t 
 

t 0 2
Does there exist a value of t for which this matrix fails to have an inverse? Explain.
37. Here is a matrix,  
et cosht sinht
 t
 e sinht cosht 


et cosht sinht
Does there exist a value of t for which this matrix fails to have an inverse? Explain.
38. Show that if det (A) ̸= 0 for A an n × n matrix, it follows that if Ax = 0, then x = 0.
39. Suppose A, B are n × n matrices and that AB = I. Show that then BA = I. Hint: You
might do something like this: First explain why det (A) , det (B) are both nonzero.
Then (AB) A = A and then show BA (BA − I) = 0. From this use what is given to
conclude A (BA − I) = 0. Then use Problem 38.
40. Use the formula for the inverse in terms of the cofactor matrix to find the inverse of
the matrix  
et 0 0
A= 0 et cost et sint .
 
t t t
0 e cost − e sint e cost + e sint t

41. Find the inverse if it exists of the matrix

 
et cost sint
 t
 e − sint cost  .


et − cost − sint
27.5. EXERCISES 521

42. Here is a matrix,

 
et e−t cost e−t sint
−e−t cost − e−t sint −e−t sint + e−t cost 
 t
 e


et 2e−t sint −2e−t cost

Does there exist a value of t for which this matrix fails to have an inverse? Explain.
43. Suppose A is an upper triangular matrix. Show that A−1 exists if and only if all
elements of the main diagonal are non zero. Is it true that A−1 will also be upper
triangular? Explain. Is everything the same for lower triangular matrices?
44. If A, B, and C are each n × n matrices and ABC is invertible, why are each of A, B,
and C invertible.
!
a (t) b (t)
45. Let F (t) = det . Verify
c (t) d (t)
! !
′ a′ (t) b′ (t) a (t) b (t)
F (t) = det + det .
c (t) d (t) c′ (t) d ′ (t)

Now suppose  
a (t) b (t) c (t)
F (t) = det  d (t) e (t) f (t)  .
 

g (t) h (t) i (t)

Use Laplace expansion and the first part to verify F ′ (t) =
   
a′ (t) b′ (t) c′ (t) a (t) b (t) c (t)
det  d (t) e (t) f (t)  + det  d ′ (t) e′ (t) f ′ (t) 
   

g (t) h (t) i (t) g (t) h (t) i (t)

 
a (t) b (t) c (t)
+ det  d (t) e (t) f (t)  .
 

g′ (t) h′ (t) i′ (t)

Conjecture a general result valid for n × n matrices and explain why it will be true.
Can a similar thing be done with the columns?
46. Let Ly = y(n) + an−1 (x) y(n−1) + · · · + a1 (x) y′ + a0 (x) y where the ai are given con-
tinuous functions defined on a closed interval, (a, b) and y is some function which
has n derivatives so it makes sense to write Ly. Suppose Lyk = 0 for k = 1, 2, · · · , n.
The Wronskian of these functions, yi is defined as
 
y1 (x) ··· yn (x)
 y′ (x) ··· y′n (x) 
 1 
W (y1 , · · · , yn ) (x) ≡ det 
 .. .. 
. .

 
(n−1) (n−1)
y1 (x) · · · yn (x)
522 CHAPTER 27. DETERMINANTS

Show that for W (x) = W (y1 , · · · , yn ) (x) to save space,

 
y1 (x) ··· yn (x)
 y′1 (x) ··· y′n (x) 
W ′ (x) = det 
 
.. .. .
. .
 
 
(n) (n)
y1 (x) · · · yn (x)

Now use the differential equation, Ly = 0 which is satisfied by each of these func-
tions, yi and properties of determinants presented above to verify the differential
equation W ′ + an−1 (x)W = 0. Give an explicit solution of this linear differential
equation, Abel’s formula, and use your answer to verify that the Wronskian of these
solutions to the equation, Ly = 0 either vanishes identically on (a, b) or never. Hint:
To solve the differential equation, let A′ (x) = an−1 (x) and multiply both sides of the
differential equation by eA(x) and then argue the left side is the derivative of some-
thing.

47. Find the following determinants and the inverses of the given matrices. You might
use MATLAB to do this with no trouble.
   
2 2 + 2i 3 − 3i 10 2 + 6i 8 − 6i
(a) det  2 − 2i 5 1 − 7i  (b) det  2 − 6i 9 1 − 7i 
   

3 + 3i 1 + 7i 16 8 + 6i 1 + 7i 17

48. Find the eigenvalues and eigenvectors of the following matrices. List the eigenvalues
according to multiplicity as a root of the characteristic polyinomial.
 
4 7 5
(a)  −2 −4 −4 
 

1 3 4
 
1 1 2
(b)  0 0 −2 
 

0 1 3
 
−3 −7 −2
(c)  4 8 2 
 

−2 −3 1
 
4 6 3
(d)  −2 −3 −2 
 

1 2 2

49. The eigenspace for an eigenvalue λ is defined to be the span of all eigenvectors. If
the dimension of the eigenspace for each λ equals the multiplicity of the eigenvalue
as a root of the characteristic polynomial, then the matrix is said to be nondefective.
If, for any eigenvalue, the dimension of the eigenspace called geometric multiplicity
27.5. EXERCISES 523

is less than the algebraic multiplicity of the eigenvalue as a root of the characteris-
tic polynomial, then the matrix is called defective. It can be shown that A can be
diagonalized if and only if it is nondefective. See Theorem 11.5.3.
50. The typical situation is that an n × n matrix has n distinct eigenvalues. In this case,
the matrix is always nondefective. This comes from the following theorem which
you will show in this problem.

Theorem 27.5.1 Let A be an n × n matrix and let {µ 1 , · · · , µ k } be distinct eigenval-

ues corresponding to eigenvectors {x1 , · · · , xk }. Then this set of eigenvectors is a
linearly independent set.

Do the following. If not independent, then there exist scalars ai such that
l
∑ ai xi = 0
i=1

in which the ai are not all zero and l is as small as possible for this to take place.
Explain why al ̸= 0 and why l ≥ 2. Then multiply both sides on the left by A and
then both sides on the left by µ l . Subtract and obtain a contradiction of some sort,
having to do with l being as small as possible and all eigenvectors being nonzero.
524 CHAPTER 27. DETERMINANTS
Chapter 28

The Mathematical Theory Of

Determinants∗

28.0.1 The Function sgn

The following Lemma will be essential in the definition of the determinant.

Lemma 28.0.1 There exists a function, sgnn which maps each ordered list of numbers from
{1, · · · , n} to one of the three numbers, 0, 1, or −1 which also has the following properties.

sgnn (1, · · · , n) = 1 (28.1)

sgnn (i1 , · · · , p, · · · , q, · · · , in ) = − sgnn (i1 , · · · , q, · · · , p, · · · , in ) (28.2)

In words, the second property states that if two of the numbers are switched, the value of
the function is multiplied by −1. Also, in the case where n > 1 and {i1 , · · · , in } = {1, · · · , n}
so that every number from {1, · · · , n} appears in the ordered list, (i1 , · · · , in ) ,

sgnn (i1 , · · · , iθ −1 , n, iθ +1 , · · · , in ) ≡

(−1)n−θ sgnn−1 (i1 , · · · , iθ −1 , iθ +1 , · · · , in ) (28.3)

where n = iθ in the ordered list, (i1 , · · · , in ) .

Proof: Define sign (x) = 1 if x > 0, −1 if x < 0 and 0 if x = 0. If n = 1, there is only

one list and it is just the number 1. Thus one can define sgn1 (1) ≡ 1. For the general case
where n > 1, simply define
!
sgnn (i1 , · · · , in ) ≡ sign ∏ (is − ir )
r<s

This delivers either −1, 1, or 0 by definition. What about the other claims? Suppose you
switch i p with iq where p < q so two numbers in the ordered list (i1 , · · · , in ) are switched.

525
526 CHAPTER 28. THE MATHEMATICAL THEORY OF DETERMINANTS∗

Denote the new ordered list of numbers as ( j1 , · · · , jn ) . Thus j p = iq and jq = i p and if

/ {p, q} , jr = ir . See the following illustration
r∈

i1 i2 ··· ip ··· iq ··· in

1 2 p q n

i1 i2 iq ip in
··· ··· ···
1 2 p q n

j1 j2 ··· jp ··· jq ··· jn

1 2 p q n
Then !
sgnn ( j1 , · · · , jn ) ≡ sign ∏ ( js − jr )
r<s
 one of p,q

both p,q
z }| { neither p nor q
= sign (i p − iq ) ∏ (i j − iq ) ∏ (i p − i j ) (is − ir )
 
∏
p< j<q p< j<q r<s,r,s∈{p,q}
/

The last product consists of the product of terms which were in the un-switched product
∏r<s (is − ir ) so produces no change in sign, while the two products in the middle both
introduce q − p − 1 minus signs. Thus their product produces no change in sign. The first
factor is of opposite sign to the iq − i p which occured in sgnn (i1 , · · · , in ) . Therefore, this
switch introduced a minus sign and

sgnn ( j1 , · · · , jn ) = − sgnn (i1 , · · · , in )

Now consider the last claim. In computing sgnn (i1 , · · · , iθ −1 , n, iθ +1 , · · · , in ) there will
be the product of n − θ negative terms

(iθ +1 − n) · · · (in − n)

and the other terms in the product for computing sgnn (i1 , · · · , iθ −1 , n, iθ +1 , · · · , in ) are those
which are required to compute sgnn−1 (i1 , · · · , iθ −1 , iθ +1 , · · · , in ) multiplied by terms of the
form (n − i j ) which are nonnegative. It follows that

sgnn (i1 , · · · , iθ −1 , n, iθ +1 , · · · , in ) = (−1)n−θ sgnn−1 (i1 , · · · , iθ −1 , iθ +1 , · · · , in )

It is obvious that if there are repeats in the list the function gives 0. ■

Lemma 28.0.2 Every ordered list of distinct numbers from {1, 2, · · · , n} can be obtained
from every other such ordered list by a finite number of switches. Also, sgnn is unique.

Proof: This is obvious if n = 1 or 2. Suppose then that it is true for sets of n − 1

elements. Take two ordered lists of numbers, P1 , P2 . Make one switch in both to place n at
the end. Call the result P1n and P2n . Then using induction, there are finitely many switches
in P1n so that it will coincide with P2n . Now switch the n in what results to where it was in
P2 .
28.1. THE DETERMINANT 527

To see sgnn is unique, if there exist two functions, f and g both satisfying 28.1 and
28.2, you could start with f (1, · · · , n) = g (1, · · · , n) = 1 and applying the same sequence
of switches, eventually arrive at f (i1 , · · · , in ) = g (i1 , · · · , in ) . If any numbers are repeated,
then 28.2 gives both functions are equal to zero for that ordered list. ■

Definition 28.0.3 When you have an ordered list of distinct numbers from

{1, 2, · · · , n} ,

say
(i1 , · · · , in ) ,
this ordered list is called a permutation. The symbol for all such permutations is Sn . The
number sgnn (i1 , · · · , in ) is called the sign of the permutation.

A permutation can also be considered as a function from the set

{1, 2, · · · , n} to {1, 2, · · · , n}

as follows. Let f (k) = ik . Permutations are of fundamental importance in certain areas

of math. For example, it was by considering permutations that Galois was able to give a
criterion for solution of polynomial equations by radicals, but this is a different direction
than what is being attempted here.
In what follows sgn will often be used rather than sgnn because the context supplies the
appropriate n.

28.1 The Determinant

Definition 28.1.1 Let f be a function which has the set of ordered lists of numbers from
{1, · · · , n} as its domain. Define

∑ f (k1 · · · kn )
(k1 ,··· ,kn )

to be the sum of all the f (k1 · · · kn ) for all possible choices of ordered lists (k1 , · · · , kn ) of
numbers of {1, · · · , n} . For example,

∑ f (k1 , k2 ) = f (1, 2) + f (2, 1) + f (1, 1) + f (2, 2) .

(k1 ,k2 )

28.1.1 The Definition

Definition 28.1.2 Let (ai j ) = A denote an n × n matrix. The determinant of A, denoted by
det (A) is defined by

det (A) ≡ ∑ sgn (k1 , · · · , kn ) a1k1 · · · ankn

(k1 ,··· ,kn )

where the sum is taken over all ordered lists of numbers from {1, · · · , n}. Note it suffices
to take the sum over only those ordered lists in which there are no repeats because if there
are, sgn (k1 , · · · , kn ) = 0 and so that term contributes 0 to the sum.
528 CHAPTER 28. THE MATHEMATICAL THEORY OF DETERMINANTS∗

28.1.2 Permuting Rows Or Columns

Let A be an n × n matrix, A = (ai j ) and let (r1 , · · · , rn ) denote an ordered list of n numbers
from {1, · · · , n}. Let A (r1 , · · · , rn ) denote the matrix whose kth row is the rk row of the
matrix A. Thus

det (A (r1 , · · · , rn )) = ∑ sgn (k1 , · · · , kn ) ar1 k1 · · · arn kn (28.4)

(k1 ,··· ,kn )

and
A (1, · · · , n) = A.

Proposition 28.1.3 Let

(r1 , · · · , rn )
be an ordered list of numbers from {1, · · · , n}. Then

sgn (r1 , · · · , rn ) det (A)

= ∑ sgn (k1 , · · · , kn ) ar1 k1 · · · arn kn (28.5)

(k1 ,··· ,kn )
= det (A (r1 , · · · , rn )) . (28.6)

Proof: Let (1, · · · , n) = (1, · · · , r, · · · s, · · · , n) so r < s.

det (A (1, · · · , r, · · · , s, · · · , n)) = (28.7)

∑ sgn (k1 , · · · , kr , · · · , ks , · · · , kn ) a1k1 · · · arkr · · · asks · · · ankn ,

(k1 ,··· ,kn )

and renaming the variables, calling ks , kr and kr , ks , this equals

= ∑ sgn (k1 , · · · , ks , · · · , kr , · · · , kn ) a1k1 · · · arks · · · askr · · · ankn

(k1 ,··· ,kn )
 
These got switched
z }| {
= − sgn k1 , · · · , kr , · · · , ks , · · · , kn  a1k1 · · · askr · · · arks · · · ankn
 
∑
(k1 ,··· ,kn )

= − det (A (1, · · · , s, · · · , r, · · · , n)) . (28.8)

Consequently,
det (A (1, · · · , s, · · · , r, · · · , n)) =
− det (A (1, · · · , r, · · · , s, · · · , n)) = − det (A)
Now letting A (1, · · · , s, · · · , r, · · · , n) play the role of A, and continuing in this way, switch-
ing pairs of numbers,
det (A (r1 , · · · , rn )) = (−1) p det (A)
where it took p switches to obtain(r1 , · · · , rn ) from (1, · · · , n). By Lemma 28.0.1, this
implies
det (A (r1 , · · · , rn )) = (−1) p det (A) = sgn (r1 , · · · , rn ) det (A)
28.1. THE DETERMINANT 529

and proves the proposition in the case when there are no repeated numbers in the ordered
list, (r1 , · · · , rn ). However, if there is a repeat, say the rth row equals the sth row, then the
reasoning of 28.7 -28.8 shows that det A (r1 , · · · , rn ) = 0 and also sgn (r1 , · · · , rn ) = 0 so the
formula holds in this case also. ■

Observation 28.1.4 There are n! ordered lists of distinct numbers from {1, · · · , n} .

To see this, consider n slots placed in order. There are n choices for the first slot. For
each of these choices, there are n − 1 choices for the second. Thus there are n (n − 1) ways
to fill the first two slots. Then for each of these ways there are n − 2 choices left for the third
slot. Continuing this way, there are n! ordered lists of distinct numbers from {1, · · · , n} as
stated in the observation.

28.1.3 A Symmetric Definition

With the above, it is possible to give a more

symmetric description of the determinant from
which it will follow that det (A) = det AT .

Corollary 28.1.5 The following formula for det (A) is valid.

1
det (A) = ·
n!

∑ ∑ sgn (r1 , · · · , rn ) sgn (k1 , · · · , kn ) ar1 k1 · · · arn kn . (28.9)

(r1 ,··· ,rn ) (k1 ,··· ,kn )
 
And also det AT = det (A) where AT is the transpose of A. (Recall that for AT = aTij ,

aTij = a ji .)

Proof: From Proposition 28.1.3, if the ri are distinct,

det (A) = ∑ sgn (r1 , · · · , rn ) sgn (k1 , · · · , kn ) ar1 k1 · · · arn kn .

(k1 ,··· ,kn )

Summing over all ordered lists, (r1 , · · · , rn ) where the ri are distinct, (If the ri are not
distinct, sgn (r1 , · · · , rn ) = 0 and so there is no contribution to the sum.)

n! det (A) =

∑ ∑ sgn (r1 , · · · , rn ) sgn (k1 , · · · , kn ) ar1 k1 · · · arn kn .

(r1 ,··· ,rn ) (k1 ,··· ,kn )

This proves the corollary since the formula gives the same number for A as it does for AT .
■

28.1.4 The Alternating Property Of The Determinant

Corollary 28.1.6 If two rows or two columns in an n × n matrix A, are switched, the deter-
minant of the resulting matrix equals (−1) times the determinant of the original matrix. If
A is an n × n matrix in which two rows are equal or two columns are equal then det (A) = 0.
Suppose the ith row of A equals

(xa1 + yb1 , · · · , xan + ybn )

530 CHAPTER 28. THE MATHEMATICAL THEORY OF DETERMINANTS∗

Then
det (A) = x det (A1 ) + y det (A2 )
where the ith row of A1 is (a1 , · · · , an ) and the ith row of A2 is (b1 , · · · , bn ) , all other rows of
A1 and A2 coinciding with those of A. In other words, det is a linear function of each row
A. The same is true with the word “row” replaced with the word “column”.

Proof: By Proposition 28.1.3 when two rows are switched, the determinant of the re-
sulting matrix is (−1) times the determinant of the original matrix. By Corollary 28.1.5 the
same holds for columns because the columns of the matrix equal the rows of the transposed
matrix. Thus if A1 is the matrix obtained from A by switching two columns,

det (A) = det AT = − det AT1 = − det (A1 ) .

If A has two equal columns or two equal rows, then switching them results in the same
matrix. Therefore, det (A) = − det (A) and so det (A) = 0.
It remains to verify the last assertion.


det (A) ≡ ∑ sgn (k1 , · · · , kn ) a1k1 · · · xaki + ybki · · · ankn
(k1 ,··· ,kn )

=x ∑ sgn (k1 , · · · , kn ) a1k1 · · · aki · · · ankn

(k1 ,··· ,kn )

+y ∑ sgn (k1 , · · · , kn ) a1k1 · · · bki · · · ankn

(k1 ,··· ,kn )

≡ x det (A1 ) + y det (A2 ) .

The same is true of columns because det AT = det (A) and the rows of AT are the columns

of A. ■

28.1.5 Linear Combinations And Determinants

Linear combinations have been discussed already. However, here is a review and some new
terminology.

Definition 28.1.7 A vector w, is a linear combination of the vectors {v 1 , · · · , v r } if there

exists scalars, c1 , · · · cr such that w = ∑rk=1 ck v k . This is the same as saying

w ∈ span (v 1 , · · · , v r ) .

The following corollary is also of great use.

Corollary 28.1.8 Suppose A is an n × n matrix and some column (row) is a linear combi-
nation of r other columns (rows). Then det (A) = 0.
 
Proof: Let A = a1 · · · an be the columns of A and suppose the condition that
one column is a linear combination of r of the others is satisfied. Then by using Corollary
28.1. THE DETERMINANT 531

28.1.6 the determinant of A is zero if and only if the determinant of the matrix B, which has
this special column placed in the last position, equals zero. Thus an = ∑rk=1 ck ak and so
 
det (B) = det a1 · · · ar · · · an−1 ∑rk=1 ck ak .

By Corollary 28.1.6
r  
det (B) = ∑ ck det a1 ··· ar ··· an−1 ak = 0.
k=1

because
there are two equal columns. The case for rows follows from the fact that det (A) =
det AT . ■

28.1.6 The Determinant Of A Product

Recall the following definition of matrix multiplication.

Definition 28.1.9 If A and B are n × n matrices, A = (ai j ) and B = (bi j ), AB = (ci j ) where
n
ci j ≡ ∑ aik bk j .
k=1

One of the most important rules about determinants is that the determinant of a product
equals the product of the determinants.

Theorem 28.1.10 Let A and B be n × n matrices. Then

det (AB) = det (A) det (B) .

Proof: Let ci j be the i jth entry of AB. Then by Proposition 28.1.3,

det (AB) =

∑ sgn (k1 , · · · , kn ) c1k1 · · · cnkn

(k1 ,··· ,kn )
! !
= ∑ sgn (k1 , · · · , kn ) ∑ a1r1 br1 k1 ··· ∑ anrn brn kn
(k1 ,··· ,kn ) r1 rn

= ∑ ∑ sgn (k1 , · · · , kn ) br1 k1 · · · brn kn (a1r1 · · · anrn )

(r1 ··· ,rn ) (k1 ,··· ,kn )

= ∑ sgn (r1 · · · rn ) a1r1 · · · anrn det (B) = det (A) det (B) . ■
(r1 ··· ,rn )

28.1.7 Cofactor Expansions

Lemma 28.1.11 Suppose a matrix is of the form
!
A ∗
M= (28.10)
0 a
532 CHAPTER 28. THE MATHEMATICAL THEORY OF DETERMINANTS∗

or !
A 0
M= (28.11)
∗ a
where a is a number and A is an (n − 1) × (n − 1) matrix and ∗ denotes either a column
or a row having length n − 1 and the 0 denotes either a column or a row of length n − 1
consisting entirely of zeros. Then det (M) = a det (A) .
Proof: Denote M by (mi j ) . Thus in the first case, mnn = a and mni = 0 if i ̸= n while in
the second case, mnn = a and min = 0 if i ̸= n. From the definition of the determinant,
det (M) ≡ ∑ sgnn (k1 , · · · , kn ) m1k1 · · · mnkn
(k1 ,··· ,kn )

Letting θ denote the position of n in the ordered list, (k1 , · · · , kn ) then using Lemma 28.0.1,
det (M) equals
 
θ n−1
n−θ
∑ (−1) sgn n−1 k1 , · · · , k , k
θ −1 θ +1 , · · · , k n m1k1 · · · mnkn
(k1 ,··· ,kn )

Now suppose 28.11. Then if kn ̸= n, the term involving mnkn in the above expression equals
zero. Therefore, the only terms which survive are those for which θ = n or in other words,
those for which kn = n. Therefore, the above expression reduces to
a ∑ sgnn−1 (k1 , · · · kn−1 ) m1k1 · · · m(n−1)kn−1 = a det (A) .
(k1 ,··· ,kn−1 )

To get the assertion in the situation of 28.10 use Corollary 28.1.5 and 28.11 to write
!!
T AT 0
= a det AT = a det (A) .■



det (M) = det M = det
∗ a
In terms of the theory of determinants, arguably the most important idea is that of
Laplace expansion along a row or a column. This will follow from the above definition of
a determinant.
Definition 28.1.12 Let A = (ai j ) be an n × n matrix. Then a new matrix called the cofactor
matrix, cof (A) is defined by cof (A) = (ci j ) where to obtain ci j delete the ith row and the
jth column of A, take the determinant of the (n − 1) × (n − 1) matrix which results, (This
is called the i jth minor of A. ) and then multiply this number by (−1)i+ j . To make the
formulas easier to remember, cof (A)i j will denote the i jth entry of the cofactor matrix.
The following is the main result. Earlier this was given as a definition and the outra-
geous totally unjustified assertion was made that the same number would be obtained by
expanding the determinant along any row or column. The following theorem proves this
assertion.
Theorem 28.1.13 Let A be an n × n matrix where n ≥ 2. Then
n n
det (A) = ∑ ai j cof (A)i j = ∑ ai j cof (A)i j . (28.12)
j=1 i=1

The first formula consists of expanding the determinant along the ith row and the second
expands the determinant along the jth column.
28.1. THE DETERMINANT 533

Proof: Let (ai1 , · · · , ain ) be the ith row of A. Let B j be the matrix obtained from A by
leaving every row the same except the ith row which in B j equals

(0, · · · , 0, ai j , 0, · · · , 0) .

Then by Corollary 28.1.6,

n
det (A) = ∑ det (B j )
j=1

Denote by Ai j the (n − 1) × (n − 1) matrix obtained by deleting the ith row and the jth col-
i+ j i j


umn of A. Thus cof (A)i j ≡ (−1) det A . At this point, recall that from Proposition
28.1.3, when two rows or two columns in a matrix M, are switched, this results in multi-
plying the determinant of the old matrix by −1 to get the determinant of the new matrix.
Therefore, by Lemma 28.1.11,
!!
n− j n−i Ai j ∗
det (B j ) = (−1) (−1) det
0 ai j
!!
A i j ∗
= (−1)i+ j det = ai j cof (A)i j .
0 ai j

Therefore,
n
det (A) = ∑ ai j cof (A)i j
j=1

which is the formula for expanding det (A) along the ith row. Also,
n
det (A) = det AT = ∑ aTij cof AT



ij
j=1
n
= ∑ a ji cof (A) ji
j=1

which is the formula for expanding det (A) along the ith column. ■

28.1.8 Formula For The Inverse

Note that this gives an easy way to write a formula for the inverse of an n × n matrix.
 
Theorem 28.1.14 A−1 exists if and only if det(A) ̸= 0. If det(A) ̸= 0, then A−1 = a−1 ij
where
a−1 −1
i j = det(A) cof (A) ji

for cof (A)i j the i jth cofactor of A.

Proof: By Theorem 28.1.13 and letting (air ) = A, if det (A) ̸= 0,

n
∑ air cof (A)ir det(A)−1 = det(A) det(A)−1 = 1.
i=1
534 CHAPTER 28. THE MATHEMATICAL THEORY OF DETERMINANTS∗

Now consider
n
∑ air cof (A)ik det(A)−1
i=1

when k ̸= r. Replace the kth

column with the rth column to obtain a matrix Bk whose
determinant equals zero by Corollary 28.1.6. However, expanding this matrix along the kth
column yields
n
0 = det (Bk ) det (A)−1 = ∑ air cof (A)ik det (A)−1
i=1
Summarizing,
n
∑ air cof (A)ik det (A)−1 = δ rk .
i=1
Using the other formula in Theorem 28.1.13, and similar reasoning,
n
∑ ar j cof (A)k j det (A)−1 = δ rk
j=1
 
This proves that if det (A) ̸= 0, then A−1 exists with A−1 = a−1
i j , where

−1
a−1
i j = cof (A) ji det (A) .

Now suppose A−1 exists. Then by Theorem 28.1.10,

1 = det (I) = det AA−1 = det (A) det A−1

so det (A) ̸= 0. ■
The next corollary points out that if an n × n matrix A has a right or a left inverse, then
it has an inverse.

Corollary 28.1.15 Let A be an n × n matrix and suppose there exists an n × n matrix B

such that BA = I. Then A−1 exists and A−1 = B. Also, if there exists C an n × n matrix such
that AC = I, then A−1 exists and A−1 = C.

Proof: Since BA = I, Theorem 28.1.10 implies

det B det A = 1

and so det A ̸= 0. Therefore from Theorem 28.1.14, A−1 exists. Therefore,

A−1 = (BA) A−1 = B AA−1 = BI = B.

The case where CA = I is handled similarly. ■

The conclusion of this corollary is that left inverses, right inverses and inverses are all
the same in the context of n × n matrices.
Theorem 28.1.14 says that to find the inverse, take the transpose of the cofactor matrix
and divide by the determinant. The transpose of the cofactor matrix is called the adjugate
or sometimes the classical adjoint of the matrix A. It is an abomination to call it the adjoint
although you do sometimes see it referred to in this way. In words, A−1 is equal to one over
the determinant of A times the adjugate matrix of A.
28.1. THE DETERMINANT 535

28.1.9 Cramer’s Rule

In case you are solving a system of equations, Ax = y for x, it follows that if A−1 exists,

x = A−1 A x = A−1 (Ax) = A−1 y

thus solving the system. Now in the case that A−1 exists, there is a formula for A−1 given
above. Using this formula,
n n
1
xi = ∑ a−1
ij yj = ∑
det (A)
cof (A) ji y j .
j=1 j=1

By the formula for the expansion of a determinant along a column,

 
∗ · · · y1 · · · ∗
1  . .. .. 
xi = det  .. . . ,

det (A) 
∗ · · · yn · · · ∗

where here the ith column of A is replaced with the column vector (y1 · · · , yn )T , and the
determinant of this modified matrix is taken and divided by det (A). This formula is known
as Cramer’s rule.
536 CHAPTER 28. THE MATHEMATICAL THEORY OF DETERMINANTS∗
Chapter 29

First Order Scalar ODE

29.1 First Order Linear Equations

The homogeneous first order constant coefficient linear differential equation is a differential
equation of the form
y′ + ay = 0. (29.1)
It is arguably the most important differential equation in existence. Generalizations of
it include the entire subject of linear differential equations and even many of the most
important partial differential equations occurring in applications.
Here is how to find the solutions to this equation. Multiply both sides of the equation
by eat . Then use the product and chain rules to verify that

d at

eat y′ + ay =


e y = 0.
dt
Therefore, since the derivative of the function t → eat y (t) equals zero, it follows this func-
tion must equal some constant C. Consequently, yeat = C and so y (t) = Ce−at . This shows
that if there is a solution of the equation, y′ + ay = 0, then it must be of the form Ce−at
for some constant, C. You should verify that every function of the form, y (t) = Ce−at is
a solution of the above differential equation, showing that this yields all solutions. This
proves the following theorem.

Theorem 29.1.1 The solutions to the equation, y′ + ay = 0 for a a real number consist of
all functions of the form, Ce−at where C is some constant.

Example 29.1.2 Radioactive substances decay in the following way. The rate of decay is
proportional to the amount present. In other words, letting A (t) denote the amount of the
radioactive substance at time t, A (t) satisfies the following initial value problem.

A′ (t) = −k2 A (t) , A (0) = A0

where A0 is the initial amount of the substance. What is the solution to the initial value
problem?

537
538 CHAPTER 29. FIRST ORDER SCALAR ODE

Write the differential equation as A′ (t)+k2 A (t) = 0. From Theorem 29.1.1 the solution
is
2t
A (t) = Ce−k
and it only remains to find C. Letting t = 0, it follows A0 = A (0) = C. Thus A (t) =
A0 exp −k2t .

Now here is another problem which is a little harder because it has something extra
added in at the end.
Example 29.1.3 Find solutions to y′ = 2y + 1.
Here is how you do it:
1. Write as y′ − 2y = 1
R
2. Find an “Integrating Factor” (−2) dt = −2t. Note that I didn’t bother to add in the
arbitrary constant. This is because it does not matter. You don’t care about finding
all integrating factors. You just need one. Then an integrating factor is e−2t .
3. Multiply both sides of the equation by the integrating factor.
d −2t
e−2t y′ − 2y = e y (t) = e−2t (1)



dt
Note that the first equal sign follows from the product rule and the chain rule. This
is why we multiply by the integrating factor, to get the derivative of something
equal to something known.
4. Take antiderivatives of both sides.
1
Z
e−2t y (t) = e−2t dt = − e−2t +C
2
Thus
1
y (t) = − +Ce2t
2
This time you need to be sure to keep the constant of integration because it does
matter.
Note that by varying C you get different solutions to the differential equation. Now
here are graphs of a few of these solutions along with the slope field.
Note how the solutions follow the slope field. How
do you determine the “right value” of C? This involves
an
INITIAL CONDITION
An initial condition involves specifying a particular
point which is to lie on the graph of the solution to the
differential equation. Then you can see from the picture
that, having made this specification, the rest of the graph
should be determined by the need to follow the slope
field. When you have specified the initial condition as
well as the differential equation, the problem is called an
initial value problem.
29.1. FIRST ORDER LINEAR EQUATIONS 539

Example 29.1.4 Find the solution to the initial value problem

y′ = 2y + 1, y (1) = 2

From the above example, all solutions are of the form y = − 12 + Ce2t . It is now just
a matter of finding the value of C which will cause the given point (1, 2) , expressed by
saying that y (1) = 2, to lie on the graph of y. Thus you need to have 2 = − 12 +Ce2 . Then
you just need to solve this equation for C. This yields C = 2e52 . Therefore, the solution to
the initial value problem is
1 5
y = − + 2 e2t
2 2e
Note the use of the definite article. There is only one solution to this initial value problem
although there are infinitely many solutions to the differential equation, three of which were
graphed above. This uniqueness property will be discussed more later, but for now, you can
see roughly why this is. It comes from the need for the solution to follow the slope field,
so if you specify a point on the curve, you have essentially determined it.

Example 29.1.5 Find the solution to the initial value problem

2
y′ + 2ty = sin (t) e−t , y (0) = 3.
2
2tdt = t 2 . Integrating factor: exp t 2 = et .
R

1. Find the integrating factor.
2. Multiply both sides by the integrating factor.

d
exp t 2 y′ + 2ty = exp t 2 y = sin (t)





dt
R
3. Take of both sides.
exp t 2 y (t) = − cos (t) +C

4. Solve for y
y = exp −t 2 (C − cos (t))

5. Find C to satisfy the initial condition.

3 = C − 1, C = 4.

6. Place value of C you just found in the formula for y

y = exp −t 2 (4 − cos (t))

Now at this point, you should check and see if it works. It needs to solve both the initial
condition and the differential equation.

Example 29.1.6 Find the solutions to

y′ + a (t) y = b (t) .
R
1. Find integrating factor. A (t) +C ≡ a (t) . Integrating factor: exp (A (t))
540 CHAPTER 29. FIRST ORDER SCALAR ODE

2. Multiply by exp (A (t))

chain rule and product rule
z }| {
′

d
exp (A (t)) y + a (t) y = (exp (A (t)) y) = exp (A (t)) b (t)
dt
R R
3. Do to both sides. Pick F (t) ∈ exp (A (t)) b (t) dt.

exp (A (t)) y (t) = F (t) +C

y (t) = exp (−A (t)) F (t) +C exp (−A (t))

This proves the following theorem.

Theorem 29.1.7 The solutions to the equation, y′ + a (t) y = b (t) consist of all functions
of the form y (t) = e−A(t) F (t) + e−A(t)C where F (t) ∈ eA(t) b (t) dt and C is a constant,
R

A′ (t) = a (t).

Finally, here is a uniqueness theorem.

Theorem 29.1.8 If a (t) is a continuous function, there is at most one solution to the initial
value problem, y′ + a (t) y = b (t) , y (r) = y0 .

Proof: If there were two solutions y1 and y2 , then letting w = y1 − y2 , it follows w′ +

a (t) w = 0 and w (r) = 0. Then multiplying both sides of the differential equation by eA(t)
 ′
where A′ (t) = a (t) , it follows eA(t) w = 0 and so eA(t) w (t) = C for some constant, C.
However, w (r) = 0 and so this constant can only be 0. Hence w = 0 and so y1 = y2 . ■
Finally, consider the general linear initial value problem.

Definition 29.1.9 A linear differential equation is one which is of the form

y′ + a (t) y = b (t)

where a, b are continuous. The corresponding initial value problem is

y′ + a (t) y = b (t) , y (t0 ) = y0 .

Now here are the steps for solving the initial value problem.
R
1. Find the integrating factor a (t) dt ≡ A (t)+C. The integrating factor is exp (A (t)) =
eA(t) .
2. Multiply both sides by the integrating factor.
d
exp (A (t)) y′ (t) + a (t) y (t) = (exp (A (t)) y (t)) = exp (A (t)) b (t)


dt
Why is this so? It involves the chain rule and the product rule.
d
(exp (A (t)) y (t)) = exp (A (t)) A′ (t) y (t) + exp (A (t)) y′ (t)
dt
= exp (A (t)) a (t) y (t) + exp (A (t)) y′ (t)
= exp (A (t)) y′ (t) + a (t) y (t)


29.1. FIRST ORDER LINEAR EQUATIONS 541

Rt
3. Next do t0 to both sides.
Z t Z t
d
(exp (A (s)) y (s)) ds = exp (A (s)) b (s) ds
t0 ds t0

Then by the fundamental theorem of calculus,

Z t
exp (A (t)) y (t) − exp (A (t0 )) y (t0 ) = exp (A (s)) b (s) ds
t0

and so, you can solve for y (t) and get

Z t
y (t) = exp (−A (t)) exp (A (t0 )) y (t0 ) + exp (−A (t)) exp (A (s)) b (s) ds
t0
Z t
= exp (A (t0 ) − A (t)) y0 + exp (A (s) − A (t)) b (s) ds
t0

This shows that if the linear initial value problem has a solution, then it must be of the
above form. Hence there is at most one solution to the initial value problem. Does the
above formula actually give a solution to the initial value problem? Let y (t) be given by
that formula. Then
Z t0
y (t0 ) = exp (0) y0 + exp (A (s) − A (t)) b (s) ds = y0
t0

so the initial condition holds. Does it solve the differential equation? By the chain rule and
the fundamental theorem of calculus,
y′ (t) = −A′ (t) exp (A (t0 ) − A (t)) y0 + exp (−A (t)) exp (A (t)) b (t)


Z t
+ −A′ (t) exp (−A (t)) exp (A (s)) b (s) ds


t0

= (−a (t)) exp (A (t0 ) − A (t)) y0 + exp (−A (t)) exp (A (t)) b (t)
Z t
+ (−a (t)) exp (−A (t)) exp (A (s)) b (s) ds = −a (t) y (t) + b (t)
t0

so it also is a solution of the linear initial value problem.

Example 29.1.10 This example illustrates a different notation for differential equations.
Find the solutions to
xdy + (2xy − x sin x) dx = 0
The idea is you divide by dx and so the exact meaning is
xy′ + 2xy = x sin (x)
Then
′
y′ + 2y = sin x, e2x y = e2x sin x

1
Z
e2x y = e2x sin (x) dx = e2x (2 sin x − cos x) +C
5
1
y = (2 sin x − cos x) +Ce−2x
5
The reason for writing it this way is that sometimes you want to find x as a function of y
and this notation is neutral in terms of which variable is the independent variable.
542 CHAPTER 29. FIRST ORDER SCALAR ODE

Example 29.1.11 A radioactive substance decays in such a way that the rate of change of
the amount of the substance is a constant multiple of the amount present, the constant being
negative. Thus dA dt = −kA. There is a certain sample of decaying material. Measurements
are taken after 5 years and it is found that there is about 9/10 of the original amount
present. Find the half life of this material. The half life is the amount of time it takes for
half of it to have decayed.

From the equation, A = A0 e−kt . Then 10

9
A0 = A0 e−k(5) . Solving this for k yields − ln(.9)
5 =
k and so the amount of time to have half of what was started with is T given as a solution
to the following equation.
 
− ln(.9) 1 ln (.5)
− 5 (T )
e = , so T = = 32. 894
2 ln (.9) /5
This kind of thing is associated not just with radioactive material but with other chemi-
cals as well. They degrade over time according to such an equation.
Example 29.1.12 The ancient Babylonians were fascinated with the idea of compound in-
terest. They were interested in how long it would take an initial amount to double. One
can understand compound interest compounded continuously using the same kind of differ-
ential equation as the above only this time the constant is positive and is the interest rate.
Thus
dA
= kA
dt
If the interest rate is 20% per year compounded continuously, how long will it take for an
initial amount to double in size?
From the equation, A = A0 e.2t where A0 is the initial amount. Then you want to find T
such that 2A0 = A0 e.2T and so
ln 2
T= = 5.0 ln 2 = 3. 465 7
.2
If the rate is r per year and you have n years and the interest is compounded at the end
of each year rather than continuously, then the amount is given by the formula (1 + r)n =
A (n) . Anciently, they used this kind of thing because they did not have differential equa-
tions. If the interest rate is 20% compounded monthly, then the amount after n years is
.2 12n


A0 1 + 12 where A0 is the initial amount. If n = 3.5, a use of a calculator shows that

.2 12(3.5)
 
1+ = 2. 002 2
12
which is very similar to compounding the interest continuously. The rational for this for-
mula is that if it is compounded monthly, then the interest rate per month is .2/12. Each
successive month is called a payment period.
Example 29.1.13 A lake contains one million gallons of water. A gas tank starts to leak
upstream and contaminated water mixed with gasoline starts flowing into the lake at the
rate of 1000 gallons per month. This is mixed well due to large numbers of fish in the lake
and water flows out at the same rate. The amount of gasoline in the contaminated water
varies due to the demand for gas at the gas station and the concentration of gasoline in the
contaminated water is (1 + sin (t)) grams per gallon. Find a formula for the concentration
of gasoline in the lake in grams per gallon as a function of time in months after a long time.
29.1. FIRST ORDER LINEAR EQUATIONS 543

Let A be the amount of gas in the lake. Then

dA A 1
= (1 + sin (t)) × 1000 − 6 1000 = 1000 (1 + sin (t)) − 3 A
dt 10 10
Rather than worry with the stupid numbers, write this as
A′ + aA = b (1 + sin (t)) , A (0) = 0
Following the procedure for finding solutions to a linear equation,

′
eat A = b (1 + sin (t)) eat
Now it follows that, taking antiderivatives of both sides,
 at 
at e 2 2


e A=b 3 a sint − a cost + a + 1 +C
a +a
Since A (0) = 0, it follows that
 
1 2


0=b −a + a + 1 +C
a3 + a

and so C = − a3b+a a2 − a + 1 . Therefore,

a − a2 − 1 be−at


eat
 
−at 2 2


A = be a sint − a cost + a + 1 +
a3 + a a3 + a
Now placing in the formula the values of a and b and then simplifying the result it follows
that A equals

 
106 e−0.001t 1.0e0.001t − 0.001 e0.001t cost + 1.0 × 10−6 e0.001t sint − 0.999

Then, dividing by the number of gallons in the lake, this yields for the number of grams
per gallon
 
e−0.001t 1.0e0.001t − 0.001 e0.001t cost + 1.0 × 10−6 e0.001t sint − 0.999

After a long time, the terms having the negative exponential will disappear in the limit and
this yields for the number of grams per gallon the formula
1 − 0.001 cost + 1.0 × 10−6 sint
Note that this yields approximately 1 gram per gallon. Compare to the concentration of the
incoming water. The concentration of the incoming water oscillates about 1 and so does
the concentration of gas in the lake, although the oscillations are much much smaller. This
is due to the large number of gallons in the lake. You might have expected this but you
could not have predicted exact values without the differential equation.
Example 29.1.14 A pumpkin is launched 30◦ from the horizontal at a speed of 60 feet per
second. It is acted on by the force of gravity which delivers an acceleration which is 32
feet per second squared and an acceleration due to air resistance which we assume is .2
times the speed which acts in the opposite direction to the direction of motion. Describe
the position of the pumpkin as a function of time.
544 CHAPTER 29. FIRST ORDER SCALAR ODE

Let the initial position be at (0, 0)
the coordinates of the point be (x (t) , y (t)) .
√ and let
What is the initial velocity? It is 30 3, 30 . Then the acceleration is given by

x′′ (t) , y′′ (t) = −32 (0, 1) − .2 x′ (t) , y′ (t)

Thus y′′ + .2y′ = −32. Let’s solve for y′ .

′
e.2t y′ = (−32) e.2t
−32 .2t
e.2t y′ (t) =
e +C, so y′ (t) = −160 +Ce−.2t
.2
So what is C? When t = 0, we get C − 160 = 30 and so C = 190. Hence

y′ (t) = 190e−.2t − 160

y (t) = −160t − 950e−0.2t + D

What is D? When t = 0 we want y (0) = 0 and so D = 950. Thus

y (t) = −160t − 950e−0.2t + 950

As to x,

′
x′′ + .2x′ = 0 so x′ e.2t =0
√ −.2t
and so x′ (t) = Ce−.2t . To satisfy the initial condition, x′ (t) = 30 3e . Then
√
30 3 −.2t
x (t) = e +D
− (1/5)
√
What is D? to satisfy the initial condition for the position, D = 150 3 and so
√ √
x (t) = −150 3e−.2t + 150 3

The position of the pumpkin is

 √ √ 
(x (t) , y (t)) = −150 3e−.2t + 150 3, −160t − 950e−0.2t + 950

The following is a summary of the above discussion.

PROCEDURE 29.1.15 To solve the first order linear differential equation

y′ + a (t) y = f (t) ,

do the following:

1. Find A (t) ∈ a (t) dt. That is, find A (t) such that A′ (t) = a (t).
R

2. Multiply both sides by the integrating factor eA(t) .

3. The above step yields  ′
eA(t) y (t) = eA(t) f (t)
R
4. Do dt to both sides. Then choose the arbitrary constant to satisfy a given initial
condition.
29.2. BERNOULI EQUATIONS 545

29.2 Bernouli Equations

Some kinds of nonlinear equations can be changed to get a linear equation. An equation of
the form
y′ + a (t) y = b (t) yα
is called a Bernouli equation1 . The trick is to define a new variable, z = y1−α . Then yα z = y
and so z′ = (1 − α) y−α y′ which implies (1−α) 1
yα z′ = y′ . Then

1
yα z′ + a (t) yα z = b (t) yα
(1 − α)

and so
z′ + (1 − α) a (t) z = (1 − α) b (t) .
Now this is a linear equation for z. Solve it and then use the transformation to find y.

Example 29.2.1 Solve y′ + y = ty3 .

You let z = y−2 and make the above substitution. Thus zy3 = y and
1
z′ = (−2) y−3 y′ , y′ = − y3 z′
2
and so − 21 y3 z′ + y3 z = ty3 . Hence, cancelling the y3 , z′ − 2z = (−2)t. Then

d −2t

e z = −2te−2t
dt
and so
1
e−2t z = te−2t + e−2t +C
2
and so
1
y−2 = z = t + +Ce2t
2
and so
1
y2 = 1
.
t + +Ce2t
2
When you get this far, it is a good idea to check and see if it works. After all, this is
the point of the manipulations, to get the answer. If you get the answer, then if there is a
mistake, it is no longer terribly relevant.
!
d 1 8Ce2t + 4
2yy′ = = −
dt t + 12 +Ce2t (2t + 2Ce2t + 1)2

8Ce2t + 4
y′ = −
2y (2t + 2Ce2t + 1)2
1 This is named after Jacob Bernoulli (1654-1705), one of a whole family of Swiss mathematicians. Others

were Johann I and II Daniel, and Nicolaus.

546 CHAPTER 29. FIRST ORDER SCALAR ODE

Then
8Ce2t + 4
y′ + y = − +y
2y (2t + 2Ce2t + 1)2

2
8Ce2t + 4 2y2 2t + 2Ce2t + 1
= − +
2y (2t + 2Ce2t + 1)2 2y (2t + 2Ce2t + 1)2
 
1

2
2 2t + 2Ce2t + 1
8Ce2t + 4 1
t+ 2 +Ce2t
= − 2
+
2y (2t + 2Ce2t + 1) 2y (2t + 2Ce2t + 1)2
t t t 4 3
= 4 2
=
2 = y = ty
2t
y (2t + 2Ce + 1) 2t
y t +Ce + 2 1 y

so it appears to work.
The following procedure gives a summary of the above.

PROCEDURE 29.2.2 To solve the Bernouli equation

y′ + a (t) y = b (t) yα , α ̸= 1
do the following:
1. Change the variable. Let z = y1−α . Then z′ = (1 − α) y−α y′ , yα z = y.
2. Place in the equation.
1 α ′
y z + a (t) yα z = b (t) yα
1−α
3. Cancel the yα and solve the linear equation for z.

29.3 Separable Differential Equations, Stability

Separable differential equations also occur quite often in applications and they are fairly
easy to deal with. This section gives a discussion of these equations.
Definition 29.3.1 Separable differential equations are those which can be written in the
form
dy f (x)
= .
dx g (y)
The reason these are called separable is that if you formally cross multiply,
g (y) dy = f (x) dx
and the variables are “separated”. The x variables are on one side and the y variables are
on the other.
Proposition 29.3.2 If G′ (y) = g (y) and F ′ (x) = f (x) , then if the equation, F (x)−G (y) =
c specifies y as a differentiable function of x, then x → y (x) solves the separable differential
equation
dy f (x)
= . (29.2)
dx g (y)
29.3. SEPARABLE DIFFERENTIAL EQUATIONS, STABILITY 547

Proof: Differentiate both sides of F (x) − G (y) = c with respect to x. Using the chain
rule,
dy
F ′ (x) − G′ (y) = 0.
dx
dy
Therefore, since F ′ (x) = f (x) and G′ (y) = g (y) , f (x) = g (y) dx which is equivalent to
29.2. ■

Definition 29.3.3 The curves F (x) − G (y) = c for various values of c are called integral
curves or solution curves. It makes sense to think of these as giving a solution if, near a
point on the level curve, one variable is a function of the other.

Example 29.3.4 Find the solution to the initial value problem,

y2 y′ = x, y (0) = 1.

This is a separable equation and in fact, y2 dy = xdx so the solution to the differential
3 2
equation is of the form y3 − x2 = C and it only remains to find the constant C. To do this,
you use the initial condition. Letting x = 0, it follows 13 = C and so

y3 x 2 1
− =
3 2 3
The following picture shows how the integral curves follow the tangent field.

Sometimes, you can’t expect to solve for one of the variables in terms of the other.
In other words, the integral curve might not be a function of one variable. Here is a nice
example from [7].

Example 29.3.5 Find integral curves for the equation

x2
y′ =
(1 − y2 )

Separating variables, you get 1 − y2 dy = x2 dx and so the integral curves are of the

form
y3 x3
 
y− − =C
3 3
Here is a picture of a few of these integral curves along with the slope field.
548 CHAPTER 29. FIRST ORDER SCALAR ODE

-2

-4
-4 -2 0 2 4
I used MATLAB to graph the above. One thing might be helpful to mention about
MATLAB. It is very good at manipulating matrices and vectors and there is distinctive
notation used to accomplish this. For example say you type

x=[1,2,3]; y=[2,3,4]; x.*y

and then press “enter”. You will get 2,6,12. Of course you would get an error if you wrote
x*y. Similarly, type
[2,4,6,8]./[1,2,3,4]
and press “enter”. This yields 2, 2, 2, 2. Of course [2,4,6,8]/[1,2,3,4] doesn’t make any
sense.
You can get graphs of some integral curves in MATLAB by typing the following and
then “enter”. You don’t have to type it on two lines, but if you want to do so, to get to a
new line, you press “shift” and “enter”.

>> [x,y]=meshgrid(-4:.1:4,-4:.1:4);
z=y-(y.ˆ3/3+x.ˆ3/3);contour(x,y,z,[-.5,-1,-.3,1,2])

Example 29.3.6 What is the equation of a hanging chain?

Consider the following picture of a portion of this chain.

T (x)
T (x) sin θ
θ
T (x) cos θ

T0
ρl(x)g

In this picture, ρ denotes the density of the chain which is assumed to be constant and
g is the acceleration due to gravity. T (x) and T0 represent the magnitude of the tension in
29.3. SEPARABLE DIFFERENTIAL EQUATIONS, STABILITY 549

the chain at t and at 0 respectively, as shown. Let the bottom of the chain be at the origin
as shown. If this chain does not move, then all these forces acting on it must balance. In
particular,
T (x) sin θ = l (x) ρg, T (x) cos θ = T0 .
Therefore, dividing these yields
≡c
sin θ z }| {
= l (x) ρg/T0 .
cos θ
Now letting y (x) denote the y coordinate of the hanging chain corresponding to x,

sin θ
= tan θ = y′ (x) .
cos θ
Therefore, this yields
y′ (x) = cl (x) .
Now differentiating both sides of the differential equation,
q
y′′ (x) = cl ′ (x) = c 1 + y′ (x)2

and so
y′′ (x)
q = c.
1 + y′ (x)2
Let z (x) = y′ (x) so the above differential equation becomes

z′ (x)
√ = c.
1 + z2
′
√z (x) dx = cx + d. Change the variable in the antiderivative letting u = z (x)
R
Therefore, 2
1+z
and this yields

z′ (x) du
Z Z
√ dx = √ = sinh−1 (u) +C = sinh−1 (z (x)) +C.
1 + z2 1 + u2
Therefore, combining the constants of integration,

sinh−1 y′ (x) = cx + d

and so
y′ (x) = sinh (cx + d) .
Therefore,
1
y (x) =
cosh (cx + d) + k
c
where d and k are some constants and c = ρg/T0 . Curves of this sort are called catenaries.
Note these curves result from an assumption that the only forces acting on the chain are as
shown.
550 CHAPTER 29. FIRST ORDER SCALAR ODE

The next example has to do with population models. It was mentioned earlier. The
idea is that if there were infinite resources, population growth would satisfy the differential
equation
dy
= ky
dt
where k is a constant. However, resources are not infinite
and so k should be modified to
be consistent with this. Instead of k, one writes r 1 − Ky which will cause the population
growth to decrease as soon as y exceeds K. Of course the problem with this is that we are
not sure whether K itself is dependent on other factors not included in the model.

Example 29.3.7 The equation

dy  y
= r 1− y, r, K > 0
dt K
is called the logistic equation. It models population growth. You see that the right side is
equal to 0 at the two values y = K and y = 0.

This is a separable equation. Thus

dy

= rdt
1 − Ky y
R
Now you do to both sides. This requires partial fractions on the left.
1 1 1

= +
1 − Ky y K − y y

Therefore,
ln (y) − ln (K − y) = rt +C
if 0 < y < K. If y > K, you get

ln (y) − ln (y − K) = rt +C

Therefore, the integral curves are of the form

 
y
ln = rt +C
K −y
so changing the name of the constant C, it follows that for y < K, the integral curves are
described by the following function.

Cert
y=K , C>0
Cert + 1
In case y > K, these curves are described by

Cert
y=K , C>0
Cert − 1
What follows is a picture of the slope field along with some of these integral curves in case
r = 1 and K = 10.
29.3. SEPARABLE DIFFERENTIAL EQUATIONS, STABILITY 551

The bottom axis is the t axis. Note how all the integral
curves in the picture approach K as t increases. This is why
K is called a stable equilibrium point.

Definition 29.3.8 Consider the equation dy dt = f (y) . Then y0

is called an equilibrium point if f (y0 ) = 0. Note that the so-
lution to the initial value problem y′ = f (y) , y (t0 ) = y0 is just
y = y0 . An equilibrium point is stable if whenever y1 is close
enough to y0 , it follows that the solution to the initial value
problem y′ = f (y) , y (0) = y1 stays close to y0 for all t > 0. It is asymptotically stable if
whenever y1 is close enough to y0 , it follows that for y the solution to the initial value prob-
lem, y′ = f (y) , y (0) = y1 satisfies limt→∞ y (t) = y0 . The equilibrium point y0 is unstable
if there are initial conditions close to y0 but the solution does not stay close to y0 . That is,
there exists ε > 0 such that for any δ > 0 there is y1 with |y1 − y0 | < δ but the solution to
y′ = f (y) , y (0) = y1 has the property that for some t > 0, |y (t) − y0 | ≥ ε. An equilibrium
point y0 is semi-stable if it is stable from one side and unstable from the other.

Now observe that y = 0 is the solution which results if you begin with the initial con-
dition y (0) = 0. If there is nothing to start with, it can’t grow. However, if you have any
other positive number for y (0) , then you see that the solution curve approaches the stable
point K. You can see this, not just by looking at the picture but also by taking the limit as
t → ∞ in the above formulae.
One of the interesting things about this equation is that it is possible to determine K the
maximum capacity, by taking measurements at three equally spaced times. Suppose you
do so at times t, 2t, 3t and obtain y1 , y2 , y3 respectively. Assume you are in the region where
y < K. In an actual experiment, this is where you would be. Let λ ≡ ert . Then from the
above formula for y, you have the equations
   
KCλ = y1 (Cλ + 1) , KCλ 2 = y2 Cλ 2 + 1 , KCλ 3 = y3 Cλ 3 + 1

Then divide the second equation by λ and compare with the first. This shows that λ =
y2 /y1 . Next divide the top equation by Cλ and the last by Cλ 3 . This yields
   
1 1
K = y1 1 + = y3 1 +
Cλ Cλ 3
Now it becomes possible to solve for C. This yields

y3 y3 − y21 y22


C= 1 3
y1 y2 − y32 y3

Then substitute this in to the first equation. This obtains

! !
y31 y3 − y21 y22 y31 y3 − y21 y22

 
y2 y2
K = y1 +1
y1 y32 − y32 y3 y1 y1 y32 − y32 y3 y1

Then you can solve this for K. After some simplification, it yields

y22 y3 − y21 y3
=K
y22 − y1 y3
552 CHAPTER 29. FIRST ORDER SCALAR ODE

Note how the equilibrium point K was stable in the above example. There were only
two equilibrium points, K and 0. The equilibrium point 0 was unstable because if the
integral curve started near 0 but slightly positive, it tended to increase to K. Here is another
harder example. In this example, there are three equilibrium points.
Example 29.3.9 dy y

1 − Ky y, r > 0, 0 < T < K.


dt = −r 1 − T
This is a separable equation.
dy

= −rdt
1 − Ty 1 − Ky y

The partial fractions expansion is

 
1 1 K T 1
y

= − +
1− T 1 − Ky y K − T T −y K −y y
Therefore,
−1 1
K ln |T − y| + T ln |K − y| + ln |y| = −rt +C
K −T K −T
Consider the case where r = 1, T = 5, K = 10. Then you get
!
(10 − y) y
ln = −t +C
(5 − y)2
There are cases, depending on where y is. Suppose first that y ∈ (0, 10) . Then you get for
a different C
(10 − y) y
= Ce−t , C > 0
(5 − y)2
You could solve this for y if you like and get
1  √ 
y = −t −5 Ce−t + 1 + 5Ce−t + 5
Ce + 1
or  √
1 
y = −t 5 Ce−t + 1 + 5Ce−t + 5
Ce + 1
On the other hand, if y > 10, you get
(y − 10) y
= Ce−t
(5 − y)2
The following is a picture of some integral curves and the slope field. You see how the
equilibrium points 10 and 0 are both stable but the equilibrium point 5 is not.

dy
dt
→
← ←
→
y
T K
29.3. SEPARABLE DIFFERENTIAL EQUATIONS, STABILITY 553

Is there a systematic way to figure this out without doing lots of computer generated
pictures? The answer is yes! Furthermore, it is very easy to do. Consider the right side of
the equation. If you graph the function z = f (y) , you get something which looks like the
right side of the above.
Look at the graph. When y ∈ (0, T ) , you have the slope of the graph is negative and
so, from the equation, dy dt is negative and so t → y (t) is decreasing. (Remember calculus.)
If y ∈ (T, K) , then the graph is positive and so dy
dt is positive which requires that t → y (t)
is increasing. When y ∈ (K, ∞) , the graph is negative and so t → y (t) is decreasing. Thus
T is unstable, K is stable while 0 is also stable. I have not considered the case where
y < 0 because this is not too interesting in the example which typically describes y as a
population of something. However, you can see from the graph that if y < 0, then t → y (t)
is increasing.
In general, you can consider y′ = f (y) and the equilibrium points. The following picture
is descriptive of the situation. Such an equation is called autonomous because the function
on the right depends only on y and not on t.

Pieces of the graph of f

unstable
• •
stable

Proposition 29.3.10 Suppose f is continuous with continuous derivative and that f (y0 ) =
0, f ′ (y0 ) < 0. Then y0 is asymptotically stable.
Proof: By continuity of f ′ , there is δ > 0 such that for y ∈ (y0 − δ , y0 + δ ) = I, f ′ (y) ≤
−2η, η > 0. Thus
f (y) = f (y0 ) + f ′ (y0 ) (y − y0 ) + o (y − y0 )
= f ′ (y0 ) (y − y0 ) + o (y − y0 )
Then if y1 ∈ I, and if y (t) is the solution to the equation y′ = f (y) having this initial
condition, then
Z t
y (t) − y0 = y1 − y0 + f (y (s)) ds
0
Z t Z t
′
= y1 − y0 + f (y0 ) (y (s) − y0 ) ds + o (y (s) − y0 ) ds
0 0

We can also assume δ is small enough that |o (y − y0 )| < η |y − y0 | for y ∈ I. Say y1 > y0 .
Then by assumption, t → y (t) is decreasing since y′ = f (y) < 0 and so if y (t) fails to
converge to y0 , there would exist ε > 0 which is the limit of y (t) − y0 . Then
Z t Z t
ε+ − f ′ (y0 ) (y (s) − y0 ) ds ≤ y1 − y0 + η (y (s) − y0 )
0 0

Thus Z t
ε+ ηε ≤ y1 − y0
0
554 CHAPTER 29. FIRST ORDER SCALAR ODE

which is impossible because as t → ∞, the left side is unbounded. Similar reasoning shows
asymptotic stability if y1 < y0 . ■

PROCEDURE 29.3.11 To solve a separable equation which can be placed in

the form
f (y) dy + g (x) dx = 0
do the following:
R
1. Place an in front each term on the left and do what it seems to say.
2. You get F (y) + G (x) = C. This is the general solution. Now choose C to satisfy any
initial condition which may be given.
3. An equilibrium point for y′ = f (y) is a point y0 where f (y0 ) = 0. It is an asymptoti-
cally stable equilibrium if f ′ (y0 ) < 0 and unstable if f ′ (y0 ) > 0.

29.4 Homogeneous Equations

Sometimes equations can be made separable by changing the variables appropriately. This
occurs in the case of the so called homogeneous equations, those of the form
y
y′ = f .
x
When this sort of equation occurs, there is an easy trick which will allow you to consider a
separable equation.
You define a new variable,
y
u≡ .
x
Thus y = ux and so
y′ = u′ x + u = f (u) .
Thus
du
x = f (u) − u
dx
and so
du dx
= .
f (u) − u x
The variables have now been separated and you go to work on it in the usual way. This
method is due to Leibniz2 and dates from around 1691.

Example 29.4.1 Find the solutions of the equation

y2 + xy
y′ = .
x2
2 Gottfried Wilhelm (von) Leibniz, (1646-1716) is credited with Newton as being one of the inventors of

calculus. There was much controversy over who did it first. It is likely that Newton did it first, but Leibniz had
dy
superior notation. The notation dx for the derivative and the notation for integrals is due to him. Like many of
these men, he was interested in many other subjects besides mathematics, such as philosophy, theology, geology,
and medicine.
29.5. EXACT EQUATIONS 555

First note this is of the form

 y 2 y
y′ = + .
x x
y
Let u = x so y = xu. Then
u′ x + u = u2 + u
and so, separating the variables yields

du dx
=
u2 x
Hence
1
− = ln |x| +C
u
and so
y 1
=u=
x K − ln |x|
where K = −C. Hence
x
y (x) =
K − ln |x|

PROCEDURE 29.4.2 To solve a homogeneous equation, one which can be placed

in the form y
y′ = f ,
x
do the following:

1. Define a new variable v = y/x. Then y = xv and so y′ = v + xv′ .

2. Plug in to the equation.
dv
v + xv′ = f (v) , x = f (v) − v
dx
dv dx
=
f (v) − v x
R
This is separable. Place before each side and do what it says. Then choose the
constant of integration to satisfy any initial condition which may be present.

29.5 Exact Equations

Sometimes you have a differential equation of the form

M (x, y) dx + N (x, y) dy = 0

where Nx = My . In this happy situation, one can find a function of two variables f (x, y)
such that
fx (x, y) = M (x, y) , fy (x, y) = N (x, y) (29.3)
556 CHAPTER 29. FIRST ORDER SCALAR ODE

and the solution to the equation is of the form

f (x, y) = C (29.4)

where C is a constant. This function f is called a scalar potential or potential for short.
These equations are called exact. Why does ∗ yield a solution? Say the above relation
defines y as a function of x. Then using the chain rule,

dy
fx (x, y) + fy (x, y) =0
dx
and so

fx (x, y) dx + fy (x, y) dy = 0
M (x, y) dx + N (x, y) dy = 0

It is easy to see that if there exists a C2 function f with the property that fx = M, fy = N,
then Nx = My . This follows because My = fxy and Ny = fyx . By equality of mixed partial
derivatives, you need to have My = Nx . In fact, if this last condition holds, then there will
generally be such a potential function f (x, y).
Why is it that if Nx = My then there exists f with the properties described?
Let Z x
f (x, y) ≡ M (t, y) dt + N (0, y) .
0
Then fx (x, y) = M (x, y) , and formally differentiating across the integral,
Z x Z x
fy (x, y) = My (t, y) dt + N (0, y) = Nx (t, y) dt
0 0
= N (x, y) − N (0, y) + N (0, y) = N (x, y)
 R 
In general, this process of ∂∂y 0x M (t, y) dt = 0x My (t, y) dt has not been proved, but in
R

examples, it will be obviously true. Also, it is formally true when you think of the integral
as a sort of sum and use the fact that the derivative of a sum is the sum of the derivatives.

Example 29.5.1 Find the solutions to

(cos (x) + 2xy) dx + x2 dy = 0

You see that this is exact (2x = 2x). Then the f (x, y) satisfies fx (x, y) = cos (x) + 2xy
and so f (x, y) = sin (x) + x2 y + g (y). Then taking the partial derivative with respect to y,
it follows that x2 + g′ (y) = x2 and so is suffices to let g (y) = 0. Then the solutions to this
differential equation are
sin (x) + x2 y = C
where C is a constant which would be determined by some sort of an initial condition.


Example 29.5.2 In the above example, determine C if (x, y) = π2 , 0 is to be on the curve
which yields a solution to the differential equation.
29.6. THE INTEGRATING FACTOR 557

You need to have 1 = C because

 π   π 2
sin + ·0 =C
2 2
and so the solution in this case is sin (x) + x2 y = 1.
All of the examples of this sort of thing are similar. Exact equations are easy to solve.
Physically the solution to these equations is really a statement about the energy being con-
stant.
PROCEDURE 29.5.3 To solve an exact equation
M (x, y) dx + N (x, y) dy = 0
do the following:
1. Check to see if it really is exact by seeing if Nx = My . If it is, find a scalar potential
f (x, y) such that fx = M, fy = N.
2. The general solution is f (x, y) = C. Choose C to satisfy initial conditions.

29.6 The Integrating Factor

It turns out that theoretically, this is the most general method for solving equations
m (x, y) dx + n (x, y) dy = 0
I want to stress the word “theoretically” however. If the above equation is not exact, the
idea is to multiply by a function µ which will make it exact. Thus it would be sufficient to
have
(µm)y = (µn)x
The function µ is called an integrating factor. In other words, it is required that
µ y m + µmy = µ x n + µnx (29.5)
This is called a first order linear partial differential equation and we don’t know how to
solve them. However, we don’t need to find all solutions, just one which works. The idea
is to look for µ = µ (x) or µ = µ (y) . For us, if there is no such easy solution, the method
has failed. So what would happen if there is a solution µ = µ (x)? then you would have
my (x, y) − nx (x, y)
µ ′ (x) = µ (x)
n (x, y)
and so there will be such an integrating factor if
my (x, y) − nx (x, y)
n (x, y)
depends only on x. Similarly, there will be an integrating factor µ = µ (y) if
n x − my
m
depends only on y.
558 CHAPTER 29. FIRST ORDER SCALAR ODE

Example 29.6.1 Find the solutions to

2y3 + 2y dx + 3xy2 + x dy = 0

The equation is clearly not exact so we look for an integrating factor.

µ y 2y3 + 2y + µ 6y2 + 2 = µ x 3xy2 + x + µ 3y2 + 1

We look for one which depends on only one variable. Let’s try to find µ = µ (y) first. If
there is such a solution, then

µ ′ (y) 2y3 + 2y + µ 6y2 + 2 = µ 3y2 + 1

so it looks like there is such a solution.

−3y2 − 1
µ ′ (y) = µ
2y3 + 2y
Thus
dµ −3y2 − 1
= 3 dy (29.6)
µ 2y + 2y
Then
−3y2 − 1 1
Z
dy = − ln y3 + y


ln (µ) = 3
2y + 2y 2
p
An integrating factor would be 1/ y3 + y. This looks really ugly. Let’s try and find one
which depends only on x. Then

µ 6y2 + 2 − 3y2 + 1 = µ ′ (x) 3xy2 + x

µ 3y2 + 1 = µ ′ (x) 3xy2 + x

1
µ ′ (x) = µ (x)
x
Thus µ = x is also an integrating factor. Which would you rather use? Multiply by x. The
equation is now
2xy3 + 2yx dx + 3x2 y2 + x2 dy = 0

and it is an exact equation so you are in the situation of the preceding section. You find a
scalar potential. The manipulations explained in the last section yield x2 y3 + x2 y as a scalar
potential. Then the solutions are
x2 y3 + x2 y = C
All of these are the same. You begin with 29.5 and look for solutions. In particular you
look for solutions that depend on only one variable. If you can find one, then the problem
has been reduced to that of the preceding section. If you can’t find such a solution, then
you give up. Under general conditions, it can be proved that solutions exist but as usual in
mathematics, there is a big gap between knowing something exists and finding it. However,
29.6. THE INTEGRATING FACTOR 559

here is something nice which was discovered by Euler back in the 1700s. It is called Euler’s
identity along with the more famous one involving complex numbers.3

Lemma 29.6.2 A function M (x, y) is homogeneous of degree α if M (tx,ty) = t α M (x, y).

For such a function,
∂M ∂M
αM (x, y) = x (x, y) + y (x, y)
∂x ∂y
Proof: You use the chain rule to differentiate both sides of the equation

M (tx,ty) = t α M (x, y)

with respect to t. Thus

∂M ∂M
αt α−1 M (x, y) = x (tx,ty) + y (tx,ty)
∂x ∂y
Now let t = 1. ■
The reason this is pretty nice is that if you have the equation

M (x, y) dx + N (x, y) dy = 0

and both M and N are homogeneous of degree α, then

1
xM + yN
∂M
is an integrating factor. Here Mx = ∂x .
We verify this next. It is so if
   
N M
=
xM + yN x xM + yN y
By the quotient rule, this will be so if and only if

Nx (xM + yN) − N (M + xMx + yNx ) =

My (xM + yN) − M (xMy + N + yNy )

In both sides of the above equation, some terms cancel and it follows that the desired result
follows if and only if

xMNx − (NM + xMx N) = yNMy − (MN + yMNy )

3 Euler (1707-1783) (pronounced “oiler”) was a Swiss mathematician, a student of Johann Bernoulli. He is one

of the most important mathematicians to ever live. He wrote more mathematics than anyone else, some 530 books
and papers in all areas of the subject. His very unusual memory allowed him to continue doing mathematical
research even after he went blind in 1766. Many of the ideas in this book are due to him. Like many of the other
great mathematicians of his time Euler’s interests were not limited to mathematics. His work is also very important
in engineering and physics. A remarkable amount of notation is due to him or popularized by him. Included in this
list is the summation symbol Σ, e, π, i, and f (x). Like many of his time, he was a very religious man who believed
the Bible was inspired. He had incredible insight but like most of us, he made mistakes because he sometimes
neglected issues related to convergence. However, the need for this sort of thing was not well understood in his
time. Euler died in St. Petersburg.
560 CHAPTER 29. FIRST ORDER SCALAR ODE

and this happens if and only if

xMNx − xMx N = yNMy − yMNy

which happens if and only if

MxNx + MyNy = NyMy + NxMx

if and only if
M (xNx + yNy ) = N (yMy + xMx )
But this is true because by Euler’s identity, xNx + yNy = αN and yMy + xMx = αM so the
above is just αNM = αNM. Of course it is assumed that xM + yN ̸= 0 in the above.

Example 29.6.3 Find the integral curves for

x2 + xy dx + y2 + x2 dy = 0

Of course this can be written as a homogeneous equation and the technique for solving
these can be used. However, let’s use this new technique which says that an integrating
factor is
1 1
= 3
x (x + xy) + y (y + x ) x + 2x2 y + y3
2 2 2

Then multiplying by this yields an exact equation.

x2 + xy y2 + x2
dx + dy = 0
x3 + 2x2 y + y3 x3 + 2x2 y + y3
Unfortunately, it is too complicated for me to solve this conveniently. However, knowing
that it is exact allows the use of the formula derived in showing that if My = Nx then the
equation was exact. Thus the integral curves are of the form
Z x
M (t, y) dt + N (0, y)
0

t 2 + ty
Z x
1
= dt + =C
0 t 3 + 2t 2 y + y3 y
Now we consider an easier one.

Example 29.6.4 Find the integral curves for

xy + y2 dx + x2 dy = 0

The integrating factor is

1
xy (2x + y)
and so the equation to solve is
1 x
(x + y) dx + dy = 0
x (2x + y) y (2x + y)
29.6. THE INTEGRATING FACTOR 561

Then integrating the first term with respect to x, the scalar potential is of the form
1 1
f (x, y) = ln |x| − ln x + y + g (y)
2 2
Then differentiating with respect to y,
1 x
− + g′ (y) =
2 (2x + y) y (2x + y)
1
g′ (y) =
2y
and so g (y) = 12 ln |y| will work. Thus the integral curves are of the form
1 1 1
ln |x| − ln x + y + ln |y| = C
2 2 2
You could simplify this if desired.
PROCEDURE 29.6.5 To solve
M (x, y) dx + N (x, y) dy = 0
using an integrating factor, do the following:
1. Look for an integrating factor µ which is a function of x alone. You do this if
My − Nx
N
does not depend on y. In this case, you solve
 
My − Nx
µ ′ (x) = µ (x)
N
which is a separable equation. Solve and choose constant to satisfy initial condition.
If this doesn’t work,
2. Look for an integrating factor µ which is a function of y alone. You do this if
Nx − My
M
does not depend on x. In this case, you solve
Nx − My
µ ′ (y) = µ (y)
M
which is a separable equation. Solve and choose constant to satisfy initial condition.
3. If neither of these work, check to see if M, N are both homogeneous of the same
degree. If they are, you could use either the methods of homogeneous equations or
Euler’s formula for the integrating factor
1
.
xM + yN
4. If none of the above works, give up. You don’t know how to do it. The integrating
factor exists, but you don’t know how to find it.
562 CHAPTER 29. FIRST ORDER SCALAR ODE

29.7 The Case Where M, N Are Affine Linear

Something which often occurs is an equation of the form

(px + qy + r) dx + (αx + β y + γ) dy = 0

It doesn’t quite fit anything in the earlier discussion. It won’t be exact, homogeneous, or
separable or linear. However, one can massage it to get something which is homogeneous.
This is illustrated in some examples.

Example 29.7.1 Find the integral curves for

(x + 2y + 3) dx + (2x − y + 1) dy = 0

Of course the problem is those constants 3, 1 so it is reasonable to change variables. Let

u = x − a, v = y − b where we choose a, b in an auspicious manner to get the constants to
disappear. First, dx = du, dy = dv. Then in terms of the new variables,

(u + a + 2 (v + b) + 3) dx + (2 (u + a) − (v + b) + 1) dy = 0
(u + 2v + (a + 2b + 3)) dx + (2u − v + (2a − b + 1)) dy = 0

and we want
a + 2b + 3 = 0
2a − b + 1 = 0
Hence we should let a = −1 and b = −1. Then with this, the equations reduce to

(u + 2v) du + (2u − v) dv = 0

This is now a homogeneous equation, or we could use the integrating factor described
earlier, but, in this case, it is also an exact equation. A scalar potential is

u2 v2
+ 2uv − ,
2 2
and so the integral curves for the original equation would be
1 1
(x + 1)2 + 2 (x + 1) (y + 1) − (y + 1)2 = C
2 2
The example illustrates what to do in general. You just change the variables to remove
those constant terms and then obtain a homogeneous equation which can be solved by a
variety of methods.

Example 29.7.2 Find the integral curves for

(x + y + 2) dx + (2x − y + 4) dy = 0

As before, let u = x − a, v = y − b and write in terms of these new variables. Thus

(u + v + (a + b + 2)) du + (2u − v + (2a − b + 4)) dv = 0

29.7. THE CASE WHERE M, N ARE AFFINE LINEAR 563

Then you need

a+b+2 = 0
2a − b + 4 = 0
Thus a = −2, b = 0. The new equation then is

(u + v) du + (2u − v) dv = 0

This can be considered as a homogeneous equation.

dv (u + v) 1 + uv
= = v
du v − 2u u −2
v
Then let z = u and do the usual substitution. This yields

dz 1
−z2 + 3z + 1


u = (29.7)
du z − 2
and so, separating the variables,
2−z du
dz =
z2 − 3z − 1 u
Then after much work one obtains integral curves of the form
 1 √13
v 1√

3 26 1
ln − 13 − + ln √
1√
u 2 2 v 1 3 26 13
u + 2 13 − 2

1 1
+ ln q √ + ln q √ − ln |u| = C
v 1 3 v 1 3
u − 2 13 − 2 u + 2 13 − 2

Then you plug in what u, v are in terms of x, y.

Actually, it was real easy to do this. The computer algebra system did it for me. Here
is one which is not so ugly.

Example 29.7.3 Find the integral curve which contains the given ordered pair.

(6x − y − 4) dx = (y − 2x) dy, (2, 2)

The equation is
dy 6x − y − 4
=
dx y − 2x
Now let x = u + a, y = v + b. Then we choose a, b such that in terms of the new variables
the equation becomes homogeneous. Thus we need

6a − b − 4 = 0
b − 2a = 0

Thus we let a = 1, b = 2. Then the equation is

dv 6u − v
= (29.8)
du v − 2u
564 CHAPTER 29. FIRST ORDER SCALAR ODE

This is a homogeneous equation. Let z = uv . Then

6−z 1
uz′ = −z2 + z + 6


−z =
z−2 z−2
Separating the variables,
(2 − z) dz du
=
z2 − z − 6 u
This is easily solved,
 
4 1
C− ln |z + 2| + ln |z − 3| = ln |u|
5 5
The in terms of the original variables,
 
4 y−2 1 y−2
C= ln + 2 + ln − 3 + ln |x − 1|
5 x−1 5 x−1
Then to contain the ordered pair, you need
4 1 1
C= ln 2 + ln 3 = ln (48)
5 5 5

PROCEDURE 29.7.4 To solve affine linear equations of the form

(px + qy + r) dx + (αx + β y + γ) dy = 0,

do the following:

1. Change the variables u = x − a, v = y − b, plug in and choose a, b to make the

resulting equation homogeneous.
2. Solve the resulting homogeneous equation. Then substitute back in x − a for u and
y − b for v. Pick the constant to satisfy initial conditions.

29.8 Linear and Nonlinear Differential Equations

Recall initial value problems for linear differential equations are those of the form

y′ + p (t) y = q (t) , y (t0 ) = y0 (29.9)

where p (t) and q (t) are continuous functions of t. Then if t0 ∈ [a, b] , an interval, there
exists a unique solution to the initial value problem given above which is defined for all
t ∈ [a, b]. The following theorem which is really something of a review gives a proof.

Theorem 29.8.1 Let [a, b] be an interval containing t0 and let p (t) and q (t) be continuous
functions defined on [a, b] . Then there exists a unique solution to 29.9 valid for all t ∈ [a, b] .
Rt
Proof: Let P′ (t) = p (t) , P (t0 ) = 0. For example, let P (t) ≡ t0 p (s) ds. Then multiply
both sides of the differential equation by exp(P (t)). This yields

(y (t) exp (P (t)))′ = q (t) exp (P (t))

29.8. LINEAR AND NONLINEAR DIFFERENTIAL EQUATIONS 565

and so, integrating both sides from t0 to t,

Z t
y (t) exp (P (t)) − y0 = q (s) exp (P (s)) ds
t0

and so Z t
y (t) = exp (−P (t)) y0 + exp (−P (t)) q (s) exp (P (s)) ds
t0
which shows that if there is a solution to 29.9, then the above formula gives that solution.
Thus there is at most one solution. Also, you see the above formula makes perfect sense
on the whole interval. Since the steps are reversible, this shows y (t) given in the above
formula is a solution. You should provide the details. Use the fundamental theorem of
calculus. ■
It is not so simple for a nonlinear initial value problem of the form

y′ = f (t, y) , y (t0 ) = y0 .

Theorem 29.8.2 Let f and ∂∂ yf be continuous in some rectangle, a < t < b, c < y < d con-
taining the point (t0 , y0 ) . Then there exists a unique local solution to the initial value prob-
lem
y′ = f (t, y) , y (t0 ) = y0 .
This means there exists an interval, I such that t0 ∈ I ⊆ (a, b) and a unique function, y
defined on this interval which solves the above initial value problem on that interval.

A much more general theorem will be proved later. Also, in the above, it suffices to say
that f is continuous on the given rectangle and that for y, z ∈ [c, d] ,t ∈ [a, b] ,

| f (t, y) − f (t, z)| ≤ K |y − z|

for some K > 0. This is called a Lipschitz condition. For now, note that it is reasonable
to believe the conclusion of this theorem. Start with the point (t0 , y0 ) and follow the slope
field as illustrated in many of the above examples. The problem is, sometimes you can’t
extend the solution as far as you might like.

Example 29.8.3 Solve y′ = 1 + y2 , y (0) = 0.

This satisfies the conditions of Theorem 29.8.2. Therefore, there is a unique solution
to the above initial value problem defined on some interval containing 0. However, in this
case, we can solve the initial value problem and determine exactly what happens. The
equation is separable.
dy
= dt
1 + y2
and so arctan (y) = t +C. Then from the initial condition, C = 0. Therefore, the solution
to
the equation is y = tan (t) . Of course this function is defined on the interval − π2 , π2 . It is
impossible to extend it further because it has an asymptote at the two ends of this interval.
Theorem 29.8.2 does not say that the local solution can never be extended beyond
some small interval. Sometimes it can. It depends very much on the nonlinear equation.
For example, the initial value problem

y′ = 1 + y2 − εy3 , y (0) = y0
566 CHAPTER 29. FIRST ORDER SCALAR ODE

turns out to have a solution on R. Here ε is a small positive number. You might think about
why this is so. It is related to the fact that in this new equation, the extra term prevents y′
from becoming unbounded.
Also, you don’t know whether the interval of existence is symmetric about the point at
which the initial condition is given.

Example 29.8.4 Solve y′ = (y − 1)2 , y (0) = 0.

The equation is separable

dy
= dt
(y − 1)2
Then integrating and using the initial condition,
1
= t +1
1−y
Thus
t
y=
t +1
which makes sense on (−1, ∞).
The next one looks a lot like the above, but has a solution on the whole real line.

Example 29.8.5 Consider y′ = 1 − y2 , y (0) = 0.

You can verify that

e2t − 1
y=
e2t + 1
is the solution and it makes sense for all t.
Hopefully, this has demonstrated that all sorts of things can happen when you are con-
sidering nonlinear equations. However, it gets even worse.
If you assume less on f in the above theorem, you sometimes can get existence but not
uniqueness for the initial value problem. In the next example ∂∂ yf is not continuous near
(0, 0).

Example 29.8.6 Find the solutions to the initial value problem

y′ = y1/3 , y (0) = 0.
dy
The equation is separable so y1/3
= dt and so the solutions are of the form

3 2/3
y = t +C.
2

3/2
Letting C = 0 from the initial condition, one solution is y = 23 t for t > 0. However,
you can see that y = 0 is also a solution. Thus uniqueness is violated. Note there are two
solutions to the initial value problem and both exist and solve the initial value problem on
all of [0, ∞).
29.9. COMPUTER ALGEBRA METHODS 567

Observation 29.8.7 What are the main differences between linear and nonlinear equa-
tions? Linear initial value problems have an interval of existence which is the same as
the interval on which the functions in the equation are continuous. Nonlinear initial value
problems sometimes don’t. Solutions to linear initial value problems are unique. This is not
always true for nonlinear equations although if in the nonlinear equation, f and ∂ f /∂ y
are both continuous, then you at least get uniqueness as well as existence on some possibly
small interval of undetermined length.

29.9 Computer Algebra Methods

The above methods work very well except for when they don’t, which is the typical case.
One can use computer algebra systems to solve such equations. In this section, the use of
various systems will be discussed. The intent here is to give a reasonably simple way to
obtain these solutions, not to give all possible ways to use these systems. In this book, I
will be emphasizing MATLAB. However, other systems will be discussed in this section.
One very easy to use system which behaves a lot like MATLAB is Scientific Notebook,
which is actually based on mupad. I will mention its use also.

29.9.1 MATLAB
A frequently used computer algebra system is MATLAB. You can use this to find solutions
to the initial value problem. If you want commands to appear on separate lines, you use
“shift enter”.
The basic version of MATLAB is sufficient to do the numerical procedures discussed.
In order to do procedures which involve commands like “syms” you will need to have the
symbolic math toolbox also. In particular, you need this toolbox for the first example given
here in which “dsolve” is used, but not for the numerical procedures mentioned next.
Here is what you type to get MATLAB to compute the solution to

y′ = y − .01y2 , y (0) = 2.

After the >> you type the following:

syms y(t); y(t)=dsolve(diff(y,t)==y - .01*yˆ2, y(0)==2)

After typing in the above, you press enter and here is what results.

>>syms y(t); y(t)=dsolve(diff(y,t)==y-.01*yˆ2,y(0)==2)

y(t) =
100/(exp(log(49) - t)+1)

If you want a graph of this solution, this is also easy to get. After doing the above, type
in the following to the right of >>

ezplot(y(t),[0,3])

and then press “enter” to obtain the graph of the solution on the interval [0, 3].
568 CHAPTER 29. FIRST ORDER SCALAR ODE

Similarly, you can ask for numerical solutions in case you can’t find an analytical so-
lution. MATLAB can find these also. For example, if you wanted to solve on the interval
[0, 2] the initial value problem

y′ = y − .01y5 , y (0) = 1,

You would do the following: After >> you type

f=@(x,y) y-.01*yˆ5;

Next, type the following on a new line:

[x,y]=ode45(f,[0,2],1) (*)

and on the next new line,

plot(x,y)
and press “enter”. This will give a large table of values of x followed by values of y
which comes from using a suitable numerical method named ode45 and it will also plot the
solution.
If you don’t want to see this large table of values, simply
2 place a ; at the end of ∗. This will cause MATLAB to defer
displaying the table even though it knows about it.
If you placed ; at the end of ∗, and decide you would like
1.5 to see y (.5) for example, you ask for the table of values. This
is done by typing [x,y] after >> and then “enter” to see the
whole table and simply scroll down to find an entry in the
1
0 2 4 column for x which is close to .5. There is also another way
to find the values using the deval function.
Another thing which is pretty easy to do in MATLAB is to change the initial conditions
and graph the two solutions on the same set of axes. The above gives you a graph of y (x)
for x ∈ [0, 2]. It has defined the function y at least at many points. Now you can simply
define another solution with a different initial condition as follows.

[x1,y1]=ode45(f,[0,2],2)

and press return. This will define the function x1 → y1(x1). Then to graph both on the
same axes, you would type
plot(x,y,x1,y1)
and both will appear. You can do as many of these as you want of course. If you wanted to
do a lot of graphs all at once, you can also have this done. You would do the following:

>> f=@(t,x) [x-xˆ3];

hold on
for z=-2:.5:2
[t,x]=ode45(f,[0,4],z);
plot(t,x)
end
29.10. EXERCISES 569

Then press “enter” and you will get graphs of solutions for initial conditions

−2, −1.5, −1, −.5, · · · , 2

With the above, which is solving

x′ = x − x3 , x (0) = z

for various values of z, you get the following graph.

2 Note how this illustrates that there are three
equilibrium points −1, 0, 1 and that the first and
third are stable but 0 is not.
You can also do the following. After defining a
0 function, say h=@(t,y) [y-yˆ3], you do the follow-
ing:
sol=ode45(h,[0,7],3);
deval(sol,[1,2,3,4,5])
-2
then0 press “enter”.2You should get4 the values of y at the points 1, 2, 3, 4, 5. Remember that
to place on a new line, you use “shift enter”. You could also use any other symbol for
“sol”.
Another thing I have noticed when using MATLAB is that it sometimes puts the graph
behind the command window so you don’t see it till you shrink the command window.
To adjust the appearance of the graph which results, you go to the graph and click on
file and then export setup. You can make changes in a dialog box and do things like change
the thickness of the lines and the size of the font very easily. Then you can save it as an eps
file or several other kinds of files.
Also, when you are done, type >> clear all or close all and then “enter”. Then type
>> clf and “enter” to get rid of any graphs it may have done and press “enter”. To clear
the screen, type >>clc and then press “enter”. This is a very good idea because if you want
to do something else, you don’t want MATLAB to be confused about what you mean and
it will be confused if it can.

29.10 Exercises
Linear Equations

1. Find all solutions to the following linear equations. You may need to leave answers
in terms of integrals on some of them.
2
(a) y′ + 2ty = e−t (e) y′ + t−1
1
y= 1
(t−1)2
(b) y′ − ty = et (f) y′ + tan (t) y = cos (t)
(c) y′ + cos (t) y = cos (t) (g) y′ − tan (t) y = sec (t)
(d) y′ + ty = sin (t) (h) y′ − tan (t) y = sec2 (t)

2. In the above linear equations find the solution to the initial value problems when y (0)
equals the following numbers.
570 CHAPTER 29. FIRST ORDER SCALAR ODE

(a) 1 (e) −2
(b) 2 (f) 12
(c) 3 (g) −3
(d) 4 (h) −2

3. Solve the following initial value problem. ty′ − y = 1

t2
, y (1) = 2. Would it make any
sense to give the initial condition at t = 0?
4. Solve the following initial value problem. ty′ + y = 1t , y (−1) = 2. Would it make
any
R 1
sense to give the initial condition at t = 0? Hint: You need to remember that
t dt = ln |t| +C.
5. Solve the following initial value problems.

(a) ln (t) y′ + 1t y = ln (t) , y (2) = 3.

(b) ln (t) y′ − 1t y = ln2 (t) , y (2) = 3.
(c) y′ + tan (t) y = cos3 (t) , y (0) = 4.
(d) cosh (t) y′ + sinh (t) y = sinh (t) , y (0) = −4

6. You have the equation y′ + p (t) y = q (t) where P′ (t) = p (t) . Give a formula for all
solutions to this differential equation.
7. The height of an object at time t is y (t) . It falls from an airplane at 30,000 feet which
is traveling East at 500 miles per hour and is acted on by gravity which we will
assume has acceleration equal to 32 feet per second squared and air resistance which
we will suppose yields an acceleration equal to .1 times the speed of the falling object
opposite to the direction of motion. If its initial velocity is in the direction of motion
of the airplane, find a formula for the position of the object as a function of t in feet.
8. Solve the following differential equations. Give the general solution.

(a) x3 + y dx − xdy = 0

(b) ydx + (x − y) dy = 0 Hint: You might look for x as a function of y.

(c) y y2 − x dy = dx

(d) 2ydx = x2 − 1 (dx − dy)

di
(e) L dt + Ri = E sin (ωt) . Here L, R, E are positive constants. L symbolizes induc-
tance and R resistance while i is the current.

9. For compounding interest n times in one
year which has interest rate r per year, the
tn
amount after t years is given by A0 1 + nr . Show that
 r tn
lim 1 + = ert ,
n→∞ n
thus giving the same conclusion as mentioned in the chapter.

10. Consider the equation y′ + 2ty = t, y (0) = 32.76. Find limt→∞ y (t).
29.10. EXERCISES 571

11. Although the gas supply was shut off, the air in the building continued to circulate.
When the gas was shut off, the temperature in the building was 70 and after five
hours, the temperature had fallen to a chilly 50 degrees. If the outside temperature
was at 10 degrees, what is the constant in Newton’s law of cooling?
12. A radioactive substance decays according to how much is present. Thus the equation
is A′ = −kA. If after 40 years, there is 5/6 of the amount initially there still present,
what is the half life of this substance?
13. You have the following initial value problem y′ + y = sint, y (0) = y0 . Letting y be
the solution to this initial value problem, find a function u (t) which does not depend
on y0 and limt→∞ |y (t) − u (t)| = 0.
14. A pond which holds V cubic meters is being polluted at the rate of 10 + sin (2πt) kg
per year. The periodic source represents seasonal variability. The total volume of the
lake is constant because it loses 14 V cubic meters per year and gains the same. After
a long time, what is the average amount of pollutant in this lake in a year?
Bernouli Equations
15. Solve the following initial value problems involving Bernouli equations.

(a) y′ + 2xy = xy3 , y (1) = 2 (d) y′ − 2x3 y = x3 y−1 , y (1) = 1

(b) y′ + sin (t) y = sin (t) y2 , y (1) = 1 (e) y′ + y = x2 y−2 , y (1) = 1
(c) y′ + 2y = x2 y3 , y (1) = −1 (f) y′ + x3 y = x3 y−2 , y (1) = −1

p
16. Consider y′ = py − qy2 , y (0) = mq where p, q are positive and m > 1. Solve this
Bernouli equation and also find limt→∞ y (t).
17. Consider y′ = 3y − y3 , y (0) = 1. Solve this Bernouli equation and find limt→∞ y (t).
18. Find the solution to the Bernouli equation y′ = (cost + 1) y − y3 , y (0) = 1. Hint:
You may have to leave the solution in terms of an integral.
19. Actually the drag force of a small object moving through the air is proportional not
to the speed but to the square of the speed. Thus a falling object would satisfy the
following equation for downward velocity. v′ = g − kv2 . Here g is acceleration of
gravity in whatever units are desired. Find limt→∞ v (t) in terms of g, k. Hint: Look
at the equation.
20. A Riccati equation is like a Bernouli equation except you have an extra function
added in. These are of the form y′ = a (t) + b (t) y + c (t) y2 . If you have a solution,
1
y1 , show that y (t) = y1 (t) + v(t) will be another solution provided v satisfies a suit-
able first order linear equation. Thus the set of all such y will involve a constant
of integration and so can be regarded as a general solution to the Riccati equation.
These equations result in a very natural way when you consider y′ = f (t, y) and ap-
proximate f (t, y) by fixing t and approximating the resulting function of y with a
second order Taylor polynomial.
Separable Equations
21. Solve the following initial value problems involving separable equations. The or-
dered pair given is to be included in the solution curve.
572 CHAPTER 29. FIRST ORDER SCALAR ODE

(a) x2 dx + y2 + 1 dy = 0,

π π


(1, 1) (e) 0 = cos (y) dx + tan (x) dy, 2, 4
(b) xydx + y2 + 1 dy = 0,


(1, 1)
(f) xydx = y2 + 1 dy, (1, 1)


(c) xydx + y2 + 1 dy = 0,


(1, −1)
(d) ydx + y2 + 1 xdy = 0, (g) xydx = y2 − 1 dy, (2, 1)



(1, 2)

2
22. Find all integral curves of the equation yxdx + e−x dy = 0. Graph several.
1
23. Find all integral curves of the equation yxdx + ln y3 dy = 0. Graph several.
(1+x2 )

24. Give the integral curves to the equation v′ = g − kv2 mentioned above where g is
acceleration of gravity and k a positive constant.
25. You have a collection of hyperbolas x2 − y2 = C where each choice of C leads to a
different hyperbola. Find another collection of curves which intersect these at a right
angle. Hint: Say you have f (x, y) = C is one of these. If you are at a point where the
relation defines y as a function of x, and (x, y) is a point on one of these hyperbolas
dy
just mentioned, then dx should have a relation to the tangent line to x2 − y2 = C.
Since the two curves are  to be perpendicular,
 you should have the product of their
dy x
slopes equal to −1. Thus dx y = −1.

26. Generalize the above problem. Suppose you have a family of level curves f (x, y) = C
and you want another family of curves which is perpendicular to this family of curves
at every point of intersection. Find a differential equation which will express this
condition. Recall that two curves are perpendicular if the products of the slopes of
the tangent lines to the two curves equals −1.
27. Find and determine the stability of the equilibrium points for the following separable
equations.

y′ = y2 (y − 1) (e) y′ = ln 1 + y2


(a)
(b) y′ = (y + 1) (y − 1) (y + 2)
(f) y′ = e2y − 1
(c) y′ = sin (y)
2
(d) y′ = cos (y) (g) y′ = 1 − ey

28. The force on an object of mass m acted on by the earth having mass M is given
by Newton’s formula kmM/r2 where k is the gravitation constant first calculated by
Cavendish4 in 1798. Letting R be the radius of the earth and letting g denote the
acceleration of gravity on the earth’s surface, show that kM = R2 g. Now suppose a
large gun having its muzzle at the surface of the earth is fired away from the center
of the earth such that the projectile has velocity v0 . Explain why

dv R2 g
=−
dt (R + r)2
4 For about 100 years, since the time Newton claimed the existence of this gravitation constant, no one knew

what it was. Henry Cavendish did an extremely sensitive experiment in 1797-1798 to determine it. It involved
lead balls mirrors telescopes and a torsion balance. He was a chemist who also found ways to make hydrogen. He
did many other very precise experiments in physics and chemistry.
29.10. EXERCISES 573

where r is the distance to the surface of the earth and here v = v (t) the speed of the
projectile at time t when it is at a distance of r from the surface of the earth. Next
explain why
dv R2 g
v =−
dr (R + r)2
The two variables are v and r. Separate the variables and find the solution to this
differential equation given that the initial speed is v0 as stated above. Show that the
maximum distance from the surface of the earth is given by
!
Rg
R −1
Rg − 12 v20

provided that Rg > 21 v20 . What is the smallest value of v0 such that the projectile will
leave the earth and never return?
 
29. The Grompertz equation is dy dt = ry ln K
y . Find the solutions to this equation with
initial condition y (0) = y0 . Alsoidentify
 all equilibrium solutions and their stability.
Also verify the inequality ry ln Ky ≥ ry 1 − Ky for y ∈ [0, K]. Explain why for a

given initial condition y0 ∈ (0, K) , the solution to the Grompertz equation should be
at least as large as the solution to the logistic equation.
30. You have a population which satisfies the logistic equation y′ = ry 1 − Ky and the

initial condition is y (0) = αK where 0 < α < 1/2. How long will it take for the
population to double?

31. An equilibrium point is called semi-stable if it is stable from one side and not stable
from the other. Sketch the appearance of f (y) near y0 if y0 is a semi-stable equilib-
rium point. Here f (y0 ) = 0 and the differential equation is y′ = f (y).
32. Consider the differential equation y′ = a − y2 where a is a real number. Show that
there are no equilibrium solutions if a < 0 but there are two of them if a > 0 and only
one if a = 0. Discuss the stability of the two equilibrium points when a > 0. What
about stability of equilibrium when a = 0?
33. Do exactly the same problem when y′ = ay − y3 . This time show there are three
equilibrium points when a > 0 and only one if a < 0. Discuss the stability of these
points.

34. Do the same problem if y′ = ay − y2 . These three problems illustrate something

called bifurcation which is when the nature of the solutions changes dramatically
when some parameter changes.
Homogeneous Equations

35. Find the solution curve to the following differential equations which contains the
given point.

(a) y′ = 1
x(2x+y) (x + y)2 , (1, 1)
(b) y′ = − x(x−2y)
1
x2 − xy + 2y2 , (2, 0)


574 CHAPTER 29. FIRST ORDER SCALAR ODE

(c) y′ = 1
x2 + 4xy + y2 , (−1, 1)


4x2 +yx

y′ = − 3x21−xy x2 − 3xy + y2 , (1, 1)

(d)
y′ = x(y+5x)
1
x2 + 5xy + y2 , (−1, −1)


(e)
y′ = x(3y+2x)
1
x2 + 2xy + 3y2 , (−2, 3)


(f)
y′ = x(4y−x)
1
x2 − xy + 4y2 , (3, −2)


(g)

36. Find the solution curve to the following ODEs which contains the given point.

(a) y′ = x12 x2 + y2 + xy , (1, 1)

(b) y′ = x12 4x2 + y2 + xy , (2, 0)

(c) y′ = x12 x2 + 9y2 + xy , (3, 1)

(d) y′ = x12 4x2 + 2y2 + xy , (−1, 1)

37. Find the solution curve to the following ODEs which contains the given point.

(a) − (x + y) dx + (x + 2y) dy = 0, (1, 1)

(b) (x − y) dx + (x + 3y) dy = 0, (2, 1)
(c) (4x + y) dx + (x + 2y) dy = 0, (−1, 2)
(d) − (3x + y) dx + (x − y) dy = 0, (3, 2)
(e) (3x − 4y) dx + 4x − 34 y dy = 0, (3, 1)

(f) (−y) dx + (4y − x) dy = 0, (0, 2)

(g) −2x − 31 9



4 y dx + x − 4 y dy = 0, (−1, 2) .

Find all solutions to y′ + sin xy = 1. Hint: You might need to leave the answer in


38.
terms of integrals.
Solve: x2 dy + 4x2 − xy + 5y2 dx = 0, y (3) = −1.


39.

Solve: x2 dy + 7x2 − xy + 4y2 dx = 0, y (2) = −1.

40.

Solve: x2 dy + 6x2 − xy + 3y2 dx = 0, y (−1) = 1.

41.

Solve: x3 − 7x2 y − 5y3 dx + 7x3 + 5xy2 dy = 0, y (3) = −2.

42.
Exact Equations and Integrating Factor
43. Find the solution curve to the following ODEs which contain the given point. First
verify that the equation is exact.

(a) (2xy + 1) dx + x2 dy = 0, (1, 1)

(b) (2x sin y + 1) dx + x2 cos y dy = 0, 1, π2

(c) (2x sin y − sin x) dx + (cos y) x2 + 1 dy = 0, (0, 0)

 
y 1
(d) xy+1 dx + xy+1 (x + xy + 1) dy = 0, (1, 1)
(e) y2 cos xy2 + 1 dx + 2xy cos xy2 + 1 dy = 0, (1, 0)



29.10. EXERCISES 575

(f) y tan2 xy + 1 + y cos xy dx + x tan2 xy + 1 + x cos xy + 1 dy = 0,

(0, 1)

44. Find the solution curve to the following ODEs which contains the given point.

(a) 2y3 + 2 dx + 3xy2 dy = 0, (1, 1)

(b) 2y3 + 2y + 2 cos x2 dx + 3xy2 + x dy = 0, (1, 1)

(c) 2xy2 + y + 2xy cos x2 dx + 2 sin x2 + 3x2 y + 2x dy = 0, (2, 1)

 4

(d) 3y4 dx + 4xy3 + 5y x2
dy = 0, (1, 2)

(e) 5x4 y + 4x3 y3 dx + 3x5 + 5x4 y2 dy = 0, (1, 1)

(f) 8x4 y6 + 3x3 dx + 12x5 y5 + 3xy2 dy = 0, (−1, 2)

45. Explain why every separable ODE can be considered as an exact ODE.
46. Suppose you have a family of level curves f (x, y) = C where C is a constant. Also
suppose that f is a harmonic function. That is fxx + fyy = 0. Consider the problem
of finding another family of level curves such that each of these is perpendicular to
the original level curves f (x, y) = C at any point on both of them. Show that the
appropriate equation to solve is 0 = fy dx − fx dy. Verify that this is an exact equation.
Thus there exists g (x, y) such that the solutions are g (x, y) = C.
M, N Both Affine Linear
47. Find the integral curve for the following differential equation which contains the
given point. These are also exact so you could use either method.

(a) (2x + y − 3) dx + (x + y − 3) dy = 0, (1, 6)

(b) (y − x + 2) dx + ((x − y) − 2) dy = 0, (3, 2)
(c) (x + y − 3) dx + (x + 3y − 7) dy = 0, (2, 2)
(d) (2x + y − 8) dx + (x + y − 7) dy = 0, (−2, 1)
(e) (x + y − 2) dx + (x + 3y − 4) dy = 0 = 0, (4, 1)
(f) (y − 2x + 5) dx + (x + y + 2) dy = 0, (1, 1)
(g) (y − 4x + 3) dx + (x − 5y + 4) dy = 0, (2, 1)

48. Find the integral curves for the following differential equation.

(a) (2y − x) dx = (4x + y − 9) dy

(b) (5x + 4y − 13) dx = (8x + y − 10) dy
(c) (3x − 2y + 1) dx = (y − 4x − 3) dy
(d) (4y − 4x + 4) dx = (8x + y + 11) dy
(e) (2y − x − 3) dx = (4x + y + 21) dy
(f) (5y − 6x + 23) dx = (10x + y − 29) dy

An Assortment of Exercises
576 CHAPTER 29. FIRST ORDER SCALAR ODE

49. Solve: y′ + 3 cos (t) y = 4 (cost) e−3 sint , y (0) = 1.

50. Solve: y′ + tan (t) y = cos (t) , y (0) = −2.
51. Solve: x2 dy + 4x2 − xy + 3y2 dx = 0, y (2) = −2.

52. Solve: 72 y − 2x dx + x − 94 y dy = 0 which contains the point (x, y) = (1, 2) .

53. Solve: x2 dy + 3x2 − xy + 2y2 dx = 0, y (2) = −3.

54. Solve: x3 − 6x2 y − y3 dx + 6x3 + xy2 dy = 0, y (2) = −3. Graph the integral

curve.

55. Solve: (2y − 3x) dx+ 2x − 43 y dy = 0 which contains the point (x, y) = (1, 2) . Graph

the integral curve.

56. Solve: y′ + 5 cos (3t) y = 2e−(5/3) sin 3t cos 3t, y (0) = 2.
57. Solve: x2 dy + 5x2 − xy + 5y2 dx = 0, y (−2) = −2.

58. Solve: 3x + 19 9



4 y dx + −4x − 4 y dy = 0 which contains the point (x, y) = (1, 2) .

59. Solve: x3 − 3x2 y − y3 dx + 3x3 + xy2 dy = 0, y (3) = −1.

60. Solve: (y) dx + (x + 4y) dy = 0 which contains the point (x, y) = (1, 2) .
7
61. Solve: 5 t 6 y + y′ = −5t 6 et , y (1) = 1.

62. Solve: x2 dy + 6x2 − xy + 5y2 dx = 0, y (3) = 3.

63. Find the solutions to the equation y′ + y (3 cost) = 3 (cost) e−3 sint .
64. Solve: (y − 2x) dx + 29 y − x dy = 0 which contains the point (x, y) = (1, 2) .

65. Solve: x2 dy + 2x2 − xy + y2 dx = 0, y (2) = −1.

2
66. Find the solutions to the equation y′ + 2ty = tet .
67. Solve: 73 y − 2x dx + x − 43 y dy = 0 which contains the point (x, y) = (1, 2) .

68. Solve: y′ + tan (2t) y = cos 2t, y (0) = 2.

69. Find the general solution to the equation
 
1 3
y′ + 4x3 + x2 + 3x y = exp −x4 − x3 − x2 ln (x + 1)


3 2

70. Show that the following initial value problem fails to have a unique solution.

y′ = y1/(2n+1) , y (0) = 0, n a positive integer.

29.10. EXERCISES 577

71. Sometimes you have an equation of the form

y′′ = f y, y′

and you are looking for a function t → y (t) so the independent variable is missing.
These can be massaged into a first order equation as follows. Let v = y′ and then you
have
v′ = f (y, v)
dv dv dy dv
Now dt = dy dt = dy v. Thus we have

dv
v = f (y, v)
dy
which is now a first order differential equation. Use this technique to solve the fol-
lowing problems. This won’t always work. It is a gimmick which sometimes works.

(a) y′′ + 2y′ = 0, y (0) = 1, y′ (0) = 0

(b) y′′ = y′ (2y + 1) , y (0) = 0, y′ (0) = 1
(c) y′′ = 2yy′ , y (0) = 0, y′ (0) = 1
(d) y′′ = y′ 1 − 3y2 , y (0) = 1, y′ (0) = 0

(e) y′ y′′ = 2, y (0) = 1, y′ (0) = 2

(f) y′′ = 2y, y (0) = 1, y′ (0) = 2
(g) y′ y′′ + 3y = 0, y (0) = y′ (0) = 1
(h) 1 + 3t 2 y′′ +6ty′ − t32 = 0, y′ (1) = 1, y (1) = 2. Hint: This is not like the above

but dtd 1 + 3t 2 y′ gives the first two terms.

(i) yy′′ + (y′ )2 = 0. Give a general solution involving two constants of integration.
(j) y′′ + y (y′ )2 = 0. Give a general solution involving two constants of integration.
(k) y′′ y2 − 2y (y′ )2 = 0, Give a general solution involving two constants of integra-
tion.
(l) y′′ y3 − 3y′ y2 = 0,Give a general solution involving two constants of integration.
(m) 3 (y′ )2 y′′ y2 + 2y (y′ )4 = 0,Give a general solution involving two constants of
integration.

72. Explain how you would proceed to solve an equation of the form y′′ = f (t, y′ ) where
the function you are looking for is t → y (t) . How many independent constants would
you have in a general solution?
Computer Algebra Problems
73. Give a graph of the solution to the following initial value problem on the interval
[0, 5]. y′ = −y3 + 3y2 + 2, y (0) = 0.
74. Give a graph of the solution to the following initial value problem on the interval
[0, 5].y′ = −y3 + xy2 + 1, y (0) = 1.
578 CHAPTER 29. FIRST ORDER SCALAR ODE

75. Solve the following initial value problems and give a graph of each on [0, 3] on the
same axes. y′ = 10
1
y (5 − y) , y (0) = .3, y′ = 10
1
y (5 − y) , y (0) = .5, y′ = 10
1
y (5 − y) , y (0) =
−.3.
76. Give a graph of the solutions to the differential equation y′ = ty2 − (.1) y3 on the
interval [0, 5] which result from the initial conditions y (0) = 1, 0, 2, −3.

3/4
77. Give a graph of the solution to y′ = x y2 − xy3 + 1, y (0) = 0.
78. Use a computer algebra system to obtain a solution to the initial value problem

y3
y′ = , y (0) = 1
x3 + 8y3
You may have to obtain a numerical solution in terms of a graph. It is true that the
equation is homogeneous, but it might be too hard to carry out the computations.
Scientific notebook has trouble with this one.
79. Use a computer algebra system to obtain the graph of the solution to the initial value
problem x2 y′ = 4x2 + xy + y2 , y (4) = 1.
80. Find the solution to the following initial value problem, either a graph or a formula.
Then graph it
1
y′ = xy + sin (x) − y2 , y (0) = 1
10
81. When you use MATLAB or other computer algebra system to find a numerical solu-
tion to a differential equation, you are using a fairly sophisticated numerical method.
The most primitive method for obtaining numerical solutions to y′ = f (t, y) is called
Euler’s method. In this method, one has a step size h and partitions the time interval
into t0 < t1 < · · · < tn = T,t j+1 = t j + h. Then letting y0 be the initial condition,
Euler’s method goes like this. You iterate the following process.

k = f (ti , yi ) , yi+1 = yi + hk, ti+1 = ti + h

When you get to tn , you stop. Your solution consists of a function y which interpo-
lates the points (ti , yi ) meaning yi = y(ti ). You can easily get MATLAB to do this for
you. Here is the case of y′ = y, y (0) = 1.

f=@(t,y) y; h=.01; y(1)=1; t(1)=0;

hold on; for j=1:500;
k=f(t(j),y(j)); y(j+1)=y(j)+h*k; t(j+1)=t(j)+h;
end; plot(t,y); [t(501),y(501)]

The first line is defining the function f (t, y) = y. Thus the real solution is et . The
number y(501) is the Euler solution at 5. Compare with e5 .
82. Suppose you have the initial value problem y′ = y, y (0) = y0 . You know the solution
is et y0 . Consider the interval [0,t]. Consider for k ≤ n
t
yk+1 = yk + yk
n
29.10. EXERCISES 579

Show that this is the same as finding y1, · · · , yn where

yk+1 − yk
= yk
t/n
yk+1 −yk
n
In place of y′ (s) = y (s) , you have t/n = yk . Now show that yn = 1 + nt y0 .
What is the limit as n → ∞?
83. Suppose on an interval [a, a + h] , you have y′ (t) = f (t, y (t)) and z (t) = y (a) +
(t − a) f (a, y (a)) . Suppose also the solution y has bounded continuous second deriva-
tives. Show that |y (h) − z (h)| < Ch2 for some constant C. You will need to use
Taylor’s theorem. This is the local error for the Euler method.
580 CHAPTER 29. FIRST ORDER SCALAR ODE
Chapter 30

Laplace Transform Methods

30.1 Linear O.D.E. With Constant Coefficients

This method of Laplace1 transforms succeeds so well because of the algebraic technique of
partial fractions and the fact that the Laplace transform is a linear mapping. It works very
well to solve higher order initial value problems involving linear equations with constant
coefficients and also more generally first order systems. It is all about changing a differen-
tial equation into an algebraic equation, solving that one, and then extracting the solution
to the original differential equation from what was obtained.
This presentation will emphasize the algebraic procedures. The analytical questions are
not trivial and are given a discussion in Section 4.2.
For an initial value problem, you can often reduce to one which has initial condition
given at 0 by simply changing the independent variable. Also, this is where the initial
condition is typically given anyway so in this method, I will assume all the initial conditions
are given at 0.
Definition 30.1.1 Let f be a function defined on [0, ∞) which has exponential growth,
meaning that
| f (t)| ≤ Ceλt
for some real λ . Then the Laplace transform of f , denoted by L ( f ) is defined as
Z ∞
F (s) ≡ L f (s) = e−ts f (t) dt
0

for all s sufficiently large. It is customary to write this transform as F (s) or L f (s) and
the function as f (t) instead of f . In other words, t is considered a generic variable as is s
and you tell the difference by whether it is t or s. It is sloppy but convenient notation.
Lemma 30.1.2 L is a linear mapping in the sense that if f , g have exponential growth,
then for all s large enough and a, b scalars,
L (a f (t) + bg (t)) (s) = aL f (s) + bL g (s)
1 Pierre-Simon, marquis de Laplace (1749-1827) had interests in mathematics, physics, probability, and as-
tronomy. He wrote a major book called celestial mechanics. There is also the Laplacian named after him, and
Laplace’s equation in potential theory. The expansion of a determinant along a row or column is called Laplace
expansion. He was also involved in the development of the metric system. It is hard to overstate the importance
of his contributions to mathematics and the other subjects which interested him. [14]

581
582 CHAPTER 30. LAPLACE TRANSFORM METHODS

Proof: Let f , g be two functions having exponential growth. Then for s large enough,
Z ∞
L (a f (t) + bg (t)) ≡ e−ts (a f (t) + bg (t)) dt
0
Z ∞ Z ∞
= a e−ts f (t) dt + b e−ts g (t) dt = aL f (s) + bL g (s) ■
0 0

The usefulness of this method in solving differential equations, comes from the follow-
ing observation.

Z ∞ Z ∞
L x′ (t) = x′ (t) e−ts dt = x (t) e−st |∞
0 + se−st x (t) dt = −x (0) + sL x (s) .
0 0

In the following table, Γ (p + 1) denotes the gamma function

Z ∞
Γ (p + 1) = e−t t p dt
0

The function uc (t) denotes the step function which equals 1 for t > c and 0 for t < c.

1
uc (t)

c
The expression in Formula 20.) is defined as follows
Z
δ (t − c) f (t) dt = f (c)

It models an impulse and is sometimes called the Dirac delta function. There is no such
function but it is called this anyway. In the following, n will be a positive integer and
f ∗ g (t) ≡ 0t f (t − u) g (u) du. Also, F (s) will denote L { f (t)} the Laplace transform of
R

the function t → f (t).

30.1. LINEAR O.D.E. WITH CONSTANT COEFFICIENTS 583

Table of Laplace Transforms

f (t) F (s) f (t) F (s)
b
1.) 1 1/s 12.) eat sinh bt
(s−a)2 −b2
s−a
2.) eat 1/ (s − a) 13.) eat cosh bt
(s−a)2 −b2
n!
3.) t n sn+1 14) t n eat n!
(s−a)n+1
Γ(p+1)
4.) t p, p > −1 s p+1 15.) uc (t) e−cs
s
a
5.) sin at s2 +a2 16.) uc (t) f (t − c) −cs
e F (s)
s
6.) cos at s2 +a2 17.) ect f (t) F (s − c)
s+ib
7.) eibt 18.) f (ct) 1 s


s2 +b2 cF c
a
8.) sinh at s2 −a2 19.) f ∗ g (t) F (s) G (s)
s
9.) cosh at s2 −a2 20.) δ (t − c) e−cs
b
10.) eat sin bt ′
21.) f (t) sF (s) − f (0)
(s−a)2 +b2
11.) eat cos bt s−a n dnF
(s−a)2 +b2
22.) (−t) f (t) dsn (s)

You should verify the claims in this table. It is best if you do it yourself. The fun-
damental result in using Laplace transforms is this. If you have F (s) = G (s) then aside
from finitely many jumps on each bounded interval, it follows that f (t) = g (t) . Thus you
just go backwards in the table to find the desired functions. To see this shown, see Section
4.2 on Page 47. I will illustrate with a second order differential equation having constant
coefficients. Of course you can change to a first order system and this will be the emphasis
next, but you can also use the method directly. Note
Z ∞ Z ∞
′′ −st ′
y (t) e dt = y (t) e−st |∞
0 +s y′ (t) e−st dt
0 0
Z ∞
= −y′ (0) + s y′ (t) e−st dt
0
 Z ∞

= −y′ (0) + s y (t) e−st |∞
0 + s y (t) e−st dt
0
′ 2
= −y (0) − sy (0) + s Y (s) (30.1)

A similar formula holds for higher derivatives. You can also get this by iterating 21.

Example 30.1.3 Find all solutions to the equation y′′ − 2y′ + y = e−t .

From the table, first go to y′′ . This gives −y′ (0) − sy (0) + s2Y (s) then you go to the
next term which gives −2sY (s) + 2y (0) and finally, you get Y (s) from the y. On the right
you get from formula 2. 1/ (s + 1) . Therefore, you have

1
s2Y (s) − 2sY (s) +Y (s) − y′ (0) − sy (0) + 2y (0) =
s+1
1
s2 − 2s + 1 Y (s) = y′ (0) + (s − 2) y (0) +


s+1
584 CHAPTER 30. LAPLACE TRANSFORM METHODS

Thus we find the Laplace transform of the function desired.

1
1 s−2
Y (s) = y′ (0) + y (0) 2 + s+1
s2 − 2s + 1 s − 2s + 1 (s2 − 2s + 1)
1
1 s−2
= y′ (0) + y (0) 2 + s+1
s2 − 2s + 1 s − 2s + 1 (s2 − 2s + 1)
Now you go backwards in the table. This typically involves doing partial fractions to get
something which is in the table. It may be tedious, but is completely routine. You can also
get this from a computer algebra system. More on this later. Thus we need
     
1 s−2 1
y′ (0) L −1 2 + y (0) L −1 2 + L −1
s − 2s + 1 s − 2s + 1 (s + 2) (s2 − 2s + 1)

1
s+1 1 1 1
= + 2
−
(s2 − 2s + 1) 4 (s + 1) 2 (s − 1) 4 (s − 1)
Now you go backwards in the table to find that this comes from
1 −t 1 t 1 t
e + te − e .
4 2 4
Next consider the other two terms.
s−2 1 1
=− 2
+
s2 − 2s + 1 (s − 1) s − 1

These are in the table.  

s−2
L −1 = −tet + et
s2 − 2s + 1
 
−1 1
L = tet
s2 − 2s + 1
Therefore, our solution is

1 1 1
y′ (0)tet + y (0) −tet + et + e−t + tet − et
4 2 4
If you specify y′ (0) = y (0) = 1, then you will find the unique solution to the differential
equation with initial conditions. It is
1 3 1
y (t) = tet + et + e−t
2 4 4
You can check that this satisfies the initial conditions and the equation.
Another important formula mentioned in the above table is Formula 19. In this formula,
Z t Z t
f ∗ g (t) ≡ f (t − u) g (u) du = g (t − u) f (u) du
0 0

You can use change of variables to observe that the last equation is true so f ∗ g = g ∗ f .
Why is this formula so? It follows from the definition and interchanging the order of
integration.
30.1. LINEAR O.D.E. WITH CONSTANT COEFFICIENTS 585

Z ∞ Z ∞ Z t Z ∞Z t
e−st f ∗ g (t) dt = e−st f (t − u) g (u) dudt = e−st f (t − u) g (u) dudt
0 0 0 0 0
Z ∞Z t Z ∞ Z ∞
= e−s(t−u) f (t − u) e−su g (u) dudt = e−su g (u) e−s(t−u) f (t − u) dtdu
0 0 0 u
Z ∞ Z ∞
= e−su g (u) e−sv f (v) dvdu = G (s) F (s)
0 0
Now here is another example in which the right side of the equation is such that it will
be hard to find the Laplace transform.

Example 30.1.4 Solve the initial value problem y′′ + 5y′ + 6y = sin t 2 , y (0) = 1, y′ (0) =

0.

Using the initial conditions and taking the Laplace transform of both sides,

s2Y (s) − y′ (0) − sy (0) + 5sY (s) − 5y (0) + 6Y (s) = L sin t 2

Now solve for Y (s)

Y (s) s2 + 5s + 6 = 5 + s + L sin t 2

and so
5+s 1
L sin t 2



Y (s) = + (30.2)
s2 + 5s + 6 s2 + 5s + 6
Now
1 1 1
= − +
s2 + 5s + 6 s+3 s+2
5+s 2 3
= − +
s2 + 5s + 6 s+3 s+2
so going backwards in the table
1
= L −e−3t + L e−2t = L −e−3t + e−2t




s2 + 5s + 6
5+s
= L −2e−3t + 3e−2t


s2 + 5s + 6
Using the convolution formula, and taking inverse Laplace transforms by going backwards
in the table, it follows from 30.2
Z t 
y (t) = −2e−3t + 3e−2t + −e−3(t−u) + e−2(t−u) sin u2 du


0
Z t Z t
−3t −2t −3t 3u 2 −2t
e2u sin u2 du



= −2e + 3e −e e sin u du + 3
0 0

If you are interested in a finite time interval, there is no loss of generality in using this
method because any continuous function on [0, T ] can be considered the restriction to [0, T ]
of one having exponential growth.
586 CHAPTER 30. LAPLACE TRANSFORM METHODS

30.2 First Order Systems, Constant Coefficients

You want to find a matrix valued function Φ (t) such that
Φ′ (t) = AΦ (t) , Φ (0) = I, A is p × p (30.3)
Such a matrix is called a fundamental matrix. It turns out that if you can find Φ (t) , you
can always solve the first order system
x′ = Ax + f, x (0) = x0 (30.4)
I also want to have AΦ (t) = Φ (t) A.
What is meant by the above symbols? The idea is that Φ (t) is a matrix whose entries
are differentiable functions of t. The meaning of Φ′ (t) is the matrix whose entries are the
derivatives of the entries of Φ (t). For example, abusing notation slightly,
!′ !
t t2 1 2t
= .
sin (t) tan (t) cos (t) sec2 (t)
What are some properties of this derivative? Does the product rule hold for example?
Lemma 30.2.1 Suppose Φ (t) is m × n and Ψ (t) is n × p and these are differentiable ma-
trices. Then
(Φ (t) Ψ (t))′ = Φ′ (t) Ψ (t) + Φ (t) Ψ′ (t)
Proof: By definition,
!′
 ′
(Φ (t) Ψ (t))′ i j


= (Φ (t) Ψ (t))i j = ∑ Φ (t)ik Ψ (t)k j
k
= ∑ Φ (t)ik Ψ (t)k j + ∑ Φ (t)ik Ψ′ (t)k j
′
k k
Φ′ (t) Ψ (t) i j + Φ (t) Ψ′ (t) i j



=

and so the conclusion follows. ■

Now consider how to find the fundamental matrix Φ (t) to begin with. I will illustrate
with an example.
!
−1 2
Example 30.2.2 Let A = . Find the fundamental matrix.
−3 4

I want Φ′ (t) = AΦ (t) , Φ (0) = I. Take the Laplace transform of both sides. By this I
mean replace each entry of the matrix with its Laplace transform. Then if F (s) is the name
of the Laplace transform of Φ (t) ,
sF (s) − I = AF (s) so (sI − A) F (s) = I

and so F (s) = (sI − A)−1 . Now this is easy to find using the formula for the inverse pre-
sented earlier. Recall you took the transpose of the cofactor matrix and divided by the
determinant to get the inverse. See Theorem 27.2.1. In this example,
! !!−1 !
s−4 2
−1 1 0 −1 2 s 2 −3s+2 s 2 −3s+2
F (s) = (sI − A) = s − = 3 s+1
0 1 −3 4 − s2 −3s+2 s2 −3s+2
30.2. FIRST ORDER SYSTEMS, CONSTANT COEFFICIENTS 587

Note how the entries are all rational functions. This will ALWAYS happen no matter what
matrix you use and this follows from that method for finding the inverse in terms of the
transpose of the cofactor method. Since theoretically, by the fundamental theorem of alge-
bra, one can always factor a polynomial into a product of linear and irreducible quadratics
in the denominator of those rational functions, this process will ALWAYS work with the
caveat that one might not be able to actually carry out the factorization of the polynomials
in the denominator. However this shows that the fundamental matrix does exist and that
your ability to explicitly compute it is exactly as good as your ability to factor a polyno-
mial. In this case, I can take the inverse Laplace transform of that matrix on the right and
get !
3 2 2 2
s−1 − s−2 s−2 − s−1
F (s) = 3 3 3 2
s−1 − s−2 s−2 − s−1
!
3et − 2e2t 2e2t − 2et
Φ (t) =
3et − 3e2t 3e2t − 2et
Does it work?
! !
3et − 2e2t 2e2t − 2et 3et − 4e2t 4e2t − 2et
Dt =
3et − 3e2t 3e2t − 2et 3et − 6e2t 6e2t − 2et
! ! !
−1 2 3et − 2e2t 2e2t − 2et 3et − 4e2t 4e2t − 2et
=
−3 4 3et − 3e2t 3e2t − 2et 3et − 6e2t 6e2t − 2et
so yes, it solves the equation. Also Φ (0) = I. Thus this is indeed the fundamental matrix.

30.2.1 Some Technical Considerations∗

Now if F (s) = (sI − A)−1 = L (Φ (t)) , is Φ′ (t) = AΦ (t)? is Φ (0) = I? Here we are as-
suming that the entries of Φ (t) have exponential growth. Then multiplying through by
(sI − A) , Z ∞ Z ∞
I = (sI − A) e−st Φ (t) dt = (I − A/s) se−st Φ (t) dt (30.5)
0 0
Z ∞
(I − A/s) se−st (Φ (t) − Φ (0)) dt + (I − A/s) Φ (0) (30.6)
0

because 0∞ se−st dt = 1, this being true for all large enough s. Letting s → ∞, the first term
R

converges to 0. Here is roughly why this is so. Letting δ > 0, be so small that all entries of
Φ (t) are closer than ε to the entries of Φ (0) whenever t < δ ,
Z ∞ Z ∞  
se−st (Φ (t) − Φ (0)) dt ≤ se−st Ceλt +C dt
δ δ

where ∥A∥ will denote the maximum of the absolute values of all entries of A and it is
assumed that each of these is no more than Ceλt . Now that integral on the right can be
computed and it equals the following for large s
   
1 −st+λt −st ∞ 1 −sδ +λ δ −sδ
e s−e |δ = e s+e
−s + λ s−λ
588 CHAPTER 30. LAPLACE TRANSFORM METHODS

Letting s → ∞, this clearly converges to 0. Also,

Z δ Z δ
se−st (Φ (t) − Φ (0)) dt ≤ se−st εdt < ε
0 0

and so for all s large enough, ∥(I − A/s) 0∞ se−st (Φ (t) − Φ (0)) dt∥ < ε showing that this
R

does indeed converge to 0. The last term in 30.6 converges to Φ (0) as s → ∞ and so we do
indeed have Φ (0) = I.
What about Φ′ (t) = AΦ (t)? For large s,integrate by parts using Φ (0) = I to obtain
Z ∞ Z ∞
e−st Φ′ (t) dt = −I + se−st Φ (t) dt = −I + sF (s)
0 0
Z ∞
e−st AΦ (t) dt = AF (s)
0

Is −I + sF (s) = AF (s)? Yes because F (s) = (sI − A)−1 and so the Laplace transforms of
Φ′ (t) and AΦ (t) are the same. This means the two functions are the same because they are
both continuous, something which is shown later that the Laplace transform determines the
functions from which it comes. This has shown the following important theorem.

Theorem 30.2.3 Let A be a p × p matrix and suppose F (s) = (sI − A)−1 = L (Φ (t)) for
Φ (t) a matrix whose entries have exponential growth. Then Φ′ (t) = AΦ (t) , Φ (0) = I.
Conversely, if Φ′ (t) = AΦ (t) , Φ (0) = I, then L (Φ (t)) = (sI − A)−1 . Thus the fundamen-
tal matrix is unique.

As noted above, one can ALWAYS find from the table of Laplace transforms an explicit
solution Φ (t) whose Laplace transform is (sI − A)−1 provided you can factor the polyno-
mials in the denominators of the rational functions which are the entries of (sI − A)−1 . Such
factorizations always exist by the fundamental theorem of algebra and so the fundamental
matrix always exists. Thus your ability to find an explicit formula for such a fundamental
matrix is exactly as good as your ability to factor polynomials which occur as denominators
in the formula for (sI − A)−1 . Note that Φ (t) is unique, because if you have one then its
Laplace transform must be (sI − A)−1 .
One other item is of interest in these fundamental matrices and this is the group prop-
erty.

Theorem 30.2.4 The following hold:

Lemma 30.2.5 1. If Ψ′ (t) = AΨ (t) , Ψ (0) = 0, then Ψ (t) = 0.

2. If Φ (t) is the fundamental matrix for A then Φ (t) A = AΦ (t) .
3. Also if Φ (t) is the fundamental matrix, then Φ (t + u) = Φ (t) Φ (u) . In particular
Φ (t)−1 = Φ (−t).

Proof: 1. Consider the first claim. Letting G (s) be the Laplace transform, it follows
that
sG (s) − 0 = AG (s)
for all s large enough. This is impossible unless G (s) = 0. Therefore, L (0) = L (Ψ (t))
and so 0 = Ψ (t) from what is shown later about the Laplace transform determining the
function.
30.2. FIRST ORDER SYSTEMS, CONSTANT COEFFICIENTS 589

2. (AΦ (t) − Φ (t) A)′ = A2 Φ (t) − AΦ (t) A = A (AΦ (t) − Φ (t) A) , and also AΦ (0) −
Φ (0) A = A − A = 0 for from 1., it follows that AΦ (t) − Φ (t) A = 0.
3. Using 2., and letting t be the variable of differentiation,

(Φ (t + u) − Φ (t) Φ (u))′ = AΦ (t + u) − AΦ (t) Φ (u)

= A (Φ (t + u) − Φ (t) Φ (u))

Also Φ (0 + u) − Φ (0) Φ (u) = Φ (u) − Φ (u) = 0 so by part 1., it follows that

Φ (t + u) − Φ (t) Φ (u) = 0. ■

30.2.2 Solving a First Order System

If you can find the fundamental matrix, it is easy to solve a first order system.

x′ = Ax + f, x (0) = x0 (30.7)
Multiply on the left by Φ (−t) and permute A and Φ (t) as needed using Theorem 30.2.4.

Φ (−t) x′ − AΦ (−t) x = Φ (−t) f

(Φ (−t) x)′ = Φ (−t) f (t)

Now integrate and obtain
Z t
Φ (−t) x (t) − x0 = Φ (−u) f (u) du
0

Now multiply on left by Φ (t) to obtain

Z t
x (t) = Φ (t) x0 + Φ (t) Φ (−u) f (u) du
0
Z t
= Φ (t) x0 + Φ (t − u) f (u) du
0

Therefore, there is at most one solution to 30.7 and if there is one, then this is it.

Theorem 30.2.6 There exists a unique solution to 30.7 and it is given by

Z t
x (t) = Φ (t) x0 + Φ (t − u) f (u) du
0

Proof: I just showed there is at most one solution. It only remains to verify that the
above works. However, the formula can be written as
Z t
x (t) = Φ (t) x0 + Φ (t) Φ (−u) f (u) du
0

When t = 0 this yields x0 as it should. Now differentiate. Using the product rule,
I
Z t z }| {
′
x (t) = AΦ (t) x0 + AΦ (t) Φ (−u) f (u) du + Φ (t) Φ (−t)f (t)
0
= Ax (t) + f (t) . ■
590 CHAPTER 30. LAPLACE TRANSFORM METHODS

Example 30.2.7 Find the solution to

! ! !
′ −4 −3 cost 1
x = x+ , x (0) =
6 5 et 1

First find the fundamental matrix. One goes backwards in the table to find the following.

! !!−1
−1 1 0 −4 −3
(sI − A) = s −
0 1 6 5
 
3
− −s2s−5
+s+2 −s2 +s+2
=  1
1 s+ 2 
− − 6 3
− 16 s2 + 16 s+ 31 − 6 s2 + 16 s+ 31
1

Then using going backwards in the table and writing in terms of cosh and sinh,
1
1 !
e 2 t cosh 32 t − 3 sinh 32 t −2 sinh 23 t e 2 t


Φ (t) =
1 1
4 sinh 32 t e 2 t e 2 t cosh 32 t + 3 sinh 32 t

Then the solution is

1
1 ! !
e 2 t cosh 32 t − 3 sinh 32 t −2 sinh 32 t e 2 t


1
x (t) =
1 1
4 sinh 32 t e 2 t e2t cosh 32 t + 3 sinh 32 t


1
Z t
+ Φ (t − u) f (u) du
0

1
! Z !
e 2 t cosh 32 t − 5 sinh 32 t t cos s
x (t) = 1t + Φ (t − s) ds
e 2 cosh 32 t + 7 sinh 32 t es


0

Doing the integrations, one obtains

Z t
!
cos s
Φ (t − s) ds
0 es
1 −t

!
10 e 15e2t − 14e3t + 14 (cost) et + 8et sint − 15
= 1 −t
25e2t − 28e3t + 18 (cost) et + 6et sint − 15


− 10 e

It follows that x (t) =


1 
3 t
2e + 45 sint − 23 e−t − 75 e2t + cosh 32 t e 2 t

1
−5 sinh 23 t e 2 t + 75 (cost) et e−t
 
 

1t 
3 −t 3 5 t 14 2t 3
2 e − 5 sint − 2
e + 5 e + cosh 2 t e
2
 
 
3 1t 9 t −t
+7 sinh 2 t e 2 − 5 (cost) e e

Using the table as just described really is a pretty good way to solve these kinds of
equations, but there is a much easier way to do it. You let the computer algebra system do
the tedious work for you. Here is the general idea for a first order system. Be patient. I will
30.2. FIRST ORDER SYSTEMS, CONSTANT COEFFICIENTS 591

consider specific examples a little later. However, if you are looking for something which
will solve all first order systems in closed form using known elementary functions, then
you are looking for something which is not there. You can indeed speak of it in general
theoretical terms but the only problems which are completely solvable in closed form are
those for which you can exactly find the eigenvalues of the matrix. Unfortunately, this
involves solving polynomial equations and none of us can do these in general.

30.2.3 Using a Computer Algebra System

You want to solve
x′ = Ax + f, x (0) = x0
Use the Property 21. and take Laplace transforms of both sides. Thus

sX (s) − x0 = AX (s) + F (s)

where X (s) is the Laplace transform of x (t) and F (s) is the Laplace transform of f (t).
Then you can solve for X (s) , at least for large enough s so that (sI − A)−1 exists. Thus

(sI − A) X (s) = x0 + F (s)

Then
X (s) = (sI − A)−1 (x0 + F (s))
Note that there is even a formula for (sI − A)−1 . See Theorem 28.1.14. Thus you can
always find X (s). Then having done so, it is a matter of finding the function whose Laplace
transform gives X (s). By hand, you would consider each entry of X (s) and by using partial
fractions, you would go backwards in the table. It won’t always work. Sometimes you
won’t be able to factor the polynomials enough to carry this out and even when it does
work, it will be pretty tedious. This is why you should use Matlab or some computer
algebra system. Here is an example which can be done. The reason I know it will work out
is that I cooked it up to work out. I picked a matrix whose eigenvalues are known. I also
picked the forcing function to be something which will tend to make things work.

Example 30.2.8 Solve the following first order system.

     
2 2 −1 cost 1
x′ =  −1 0 1  x+  sint  , x (0) =  0 
     

−1 0 2 et 1

Following the above general procedure, the Laplace transform of the forcing function
is  s 
s2 +1
 1 
s2 +1
 
1
s−1
and so
   −1    s

s 0 0 2 2 −1 1 s2 +1
1
X (s) =  0 s 0  −  −1 0 1   0  + 
       
s2 +1

0 0 s −1 0 2 1 1
s−1
592 CHAPTER 30. LAPLACE TRANSFORM METHODS

Now I compute this.

 
1
−s4 + s3 + 2


−
 ( )(s−1)3
s2 +1 
1 s2 −3s+4
X (s) = 
 
(s−1)2 s3 −2s2 +s−2

 
1
s5 − 3s4 + 3s3 − 2s2 + s + 2


3 3
(s−1) (s −2s +s−2)
2

At this point, I use partial fractions and go backwards in the table or I ask a computer
algebra system to find the inverse Laplace transform. I recommend using the computer
algebra system. Thus
 
1 1 t 1 2 t 3 t
2 cost + 2 e − 2 t e + 2 te
x (t) =  12 et − 109
cost − 103
sint + 25 e2t − tet 
 
1 1 4 2t 1 2 t 1 t
5 cost − 10 sint + 5 e − 2 t e + 2 te

This is then the solution to the first order system. I used Scientific Notebook to do all of
these computations. However, one can also use Matlab. You will need Matlab and the
symbolic math toolbox installed for this to work.
>>syms s t; a=(enter initial vector here); b=(enter sI-A here); c=(enter f(t) here);
simplify(ilaplace(inv(b)*(a+laplace(c))))
I will use this to solve the above problem.
>> syms s t; a=[1;0;1]; b=[s-2 -2 1;1 s -1;1 0 s-2]; c=[cos(t);sin(t);exp(t)];
simplify(ilaplace(inv(b)*(a+laplace(c))))
Note the use of square brackets in entering the matrix. You must use these. You enter
one row at a time with a space between successive entries and a semicolon to indicate the
start of a new row. Then you press enter on your keyboard and it will produce the following:
cos(t)/2 + exp(t)/2 - (tˆ2*exp(t))/2 + (3*t*exp(t))/2
(2*exp(2*t))/5 - (9*cos(t))/10 + exp(t)/2 - (3*sin(t))/10 - t*exp(t)
(4*exp(2*t))/5 + cos(t)/5 - sin(t)/10 - (tˆ2*exp(t))/2 + (t*exp(t))/2
The advantage to using Scientific notebook is the result comes out looking a lot nicer
but you get the same thing either way. In fact Scientific notebook is based on mupad which
is part of the symbolic math toolbox in Matlab.

Example 30.2.9 Find the fundamental matrix of

 
−3 2 −1
A =  0 −1 1 
 

4 −4 3

and use to solve the initial value problem


  
ln t 2 + 1 1
′ 2


x = A x+  sin t  , x (0) =  1 
  

cos (t) 1

It will not be possible to give a closed form solution for this problem but we can write
it in terms of an integral if the fundamental matrix is found.
30.3. HOMOGENEOUS PARTICULAR AND GENERAL SOLUTIONS 593

    −1
1 0 0 −3 2 −1
Ψ (s) = s  0 1 0 − 0 −1 1 
    

0 0 1 4 −4 3
s−1 2 1
 
s2 +2s+1 s2 +2s+1
− s2 +2s+1
s2 −5
− −s3 −s42 +s+1 − −s3 −s s+3
− −s3 −s
 
=  2 +s+1 2 +s+1 
 1 3 
1 4 s+ 4
1 s2 − 1 − 1 21 1 1 s2 − 1
4 4 4s −4 4 4

Therefore,
 
−e−t (2t − 1) 2te−t −te−t
Φ (t) =  e − e−t − 2te−t 2e−t − et + 2te−t et − e−t − te−t 
 t 

4 sinht −4 sinht 2et − e−t

Then the solution is  
te−t − e−t (2t − 1)
x (t) =  et − te−t +
 
t
2e − e −t


Z t ln u2 + 1
+ Φ (t − u)  sin u2


 du

0
cos (u)
where Φ (t) is given above.
To find the fundamental matrix in Matlab, you would use the following syntax.
>> syms s t; b=[s+3 -2 1;0 s+1 -1;-4 4 s-3];
simplify(ilaplace(inv(b)))
Then you press enter and it gives the following:
[ -exp(-t)*(2*t - 1), 2*t*exp(-t), -t*exp(-t)]
[ -exp(-t)*(2*t - exp(2*t) + 1), exp(-t)*(2*t - exp(2*t) + 2), -exp(-t)*(t - exp(2*t) + 1)]
[ 2*exp(t) - 2*exp(-t), 2*exp(-t) - 2*exp(t), 2*exp(t) - exp(-t)]
which is just a messier version of what was obtained above using Scientific notebook.
Actually it might be a little easier to use the following syntax. You can adjust as needed.
>>syms s t; b=eye(3); c=[-3 2 -1;0 -1 1;4 -4 3];
simplify(ilaplace(inv(s*b-c)))
Here you just need to enter the matrix for c and the symbol eye(3) says it is a 3 × 3
identity matrix. Matlab then does the rest for you.

30.3 Homogeneous Particular and General Solutions

It is convenient to split things up into homogeneous problems and then look for particular
solutions. First of all, is a definition of the general solution to a homogeneous problem.
Definition 30.3.1 Let A be an n × n matrix. The general solution to the homogeneous
problem is defined to be all solutions to the equation
x′ = Ax
594 CHAPTER 30. LAPLACE TRANSFORM METHODS

Note how there is no initial condition. We just look for all solutions to the above
differential equation. The following theorem describes all of these solutions.

Theorem 30.3.2 The general solution to the homogeneous problem x′ = Ax consists of all
vectors of the form Φ (t) c where c is a vector in Fn and Φ (t) is the fundamental matrix of
A.

Proof: Let x be a solution to the equation. Then x (0) = c for some c. Consider Φ (t) c
and x (t) both solve x′ = Ax the first doing so because

Φ′ (t) c = AΦ (t) c

Thus Φ (t) c and x (t) both solve the same differential equation and have the same initial
condition. Therefore, these are the same and this shows that the set of solutions to x′ = Ax
consists of Φ (t) c for c ∈ Fn as claimed. ■

Example 30.3.3 Find the general solution to x′ = Ax where

 
4 1 2 1
 −3 0 −2 −1 
A=
 
−7 −3 −4 −3

 
8 4 6 5

According to the above theory, it suffices to find the fundamental matrix. The inverse
of sI − A is the matrix which has the following columns, beginning at the left and moving
toward the right:
   
s+2 1
s2 −2s+1 s2 −2s+1
 3   s−2 

 − s2 −2s+1  
, s2 −2s+1

,
 7s−13   3s−5 
 −s3 +4s2 −5s+2   −s3 +4s2 −5s+2 
1 6s−10 2s−3 6s−10
− 3s−5 (4s − 7) −s3 +4s 2 −5s+2 − 3s−5 −s3 +4s2 −5s+2
   
2 1
s2 −2s+1 s2 −2s+1
2 1
− s2 −2s+1 − s2 −2s+1
   
   
 2
, 
−8s+11 3s−5

 − −s3s +4s 2 −5s+2
 
  −s3 +4s2 −5s+2


6s−10 s2 +s−4
− −s3 +4s2 −5s+2 − −s3 +4s 2 −5s+2

This was done by a computer algebra system. Now take inverse Laplace transforms of this
to get the fundamental matrix Φ (t) =

et (3t + 1) tet 2tet tet

 

 −3tet −et (t − 1) −2tet −tet 

e − e2t − 6tet
t e − e2t − 2tet
t 2e − e2t − 4tet
t e − e2t − 2tet
t
 
 
2e2t − 2et + 6tet 2e2t − 2et + 2tet 2e2t − 2et + 4tet 2e2t − et + 2tet

therefore, the general solution is of the form Φ (t) c where c ∈ Fn . In other words, it is the
set of linear combinations of the columns of Φ (t). Since Φ (t)−1 = Φ (−t) , the columns
30.3. HOMOGENEOUS PARTICULAR AND GENERAL SOLUTIONS 595

are linearly independent and this shows that the dimension of the solution space is n if A is
n × n. In the above example, the dimension of the general solution is 4 because A is 4 × 4.
Now consider the general solution to

x′ = Ax + f

There is a very easy way to describe this. It is just the general solution to x′ = Ax added to
x p where x p is any particular solution to the above nonhomogeneous equation.

Theorem 30.3.4 The general solution to x′ = Ax + f consists of all solutions to this equa-
tion. It is of the form Φ (t) c + x p where x p is a particular solution meaning x′p = Ax p +f .

Proof: Anything of the form Φ (t) c + x p is a solution to x′ = Ax + f . It remains to

verify that this is the only way it can happen. Let z ′ = Az + f and consider (z − x p ) .
Then
(z − x p )′ = z ′ − x′p = A z + f − (Ax p + f ) = A (z − x p )
and so z − x p is a solution to x′ = Ax. Therefore, from Theorem 30.3.2, there exists c ∈ Fn
such that z (t) = Φ (t) c + x p (t). ■

Example 30.3.5 Find the general solution to

x′ = Ax + f

where    
2 −4 −2 et sint
A= 3 −4 −2  , f (t) =  e−t cost 
   

−3 10 6 t

First I will find the fundamental matrix using the following syntax.
>>syms s t; b=eye(3);
c=[2 -4 -2;3 -4 -2;-3 10 6];f=[exp(t)*sin(t);exp(-t)*cos(t);t];
simplify(ilaplace(inv(s*b-c)))
simplify(ilaplace(inv(s*b-c)*laplace(f)))
The first line starting with “simplify” will give the fundamental matrix and the second
will give a particular solution. The claim about the first was already considered. As to the
second, if x is a particular solution with zero initial condition,

sX (s) = AX (s) + F (s)

In the above syntax, the F (s) comes from laplace(f). Then

X (s) = (sI − A)−1 F (s)

and this involves inv(s*b-c)*laplace(f) in the above syntax. Then you do ilaplace to this
thing to get a particular solution. Try it. You will get a horrendous mess but Matlab has no
problem in doing it.
This has shown how to solve first order systems at least up to a suitable variation of
constants formula. There is one other topic which is sometimes useful and that is the
convolution integral and its relation to the Laplace transform.
596 CHAPTER 30. LAPLACE TRANSFORM METHODS

Theorem 30.3.6 Suppose F (s) is the Laplace transform of f (t) and G (s) is the Laplace
transform of g (t). Then F (s) G (s) is the Laplace transform of
Z t Z t
f (u) g (t − u) du = f (t − u) g (u) du ≡ f ∗ g (t)
0 0

Proof: To be rigorous, you really need to replace improper integrals with integrals over
a finite interval and then take a limit, but the idea is essentially as follows:
Z ∞ Z t Z ∞Z ∞
e−st f (t − u) g (u) dudt = e−st f (t − u) g (u) dtdu
0 0 0 u
Z ∞Z ∞
= e−s(t−u) f (t − u) e−su g (u) dtdu
0 u
Z ∞Z ∞
= e−sr f (r) e−su g (u) drdu
0 0
Z ∞ Z ∞ 
= e−su g (u) e−sr f (r) dr du
0 0
Z ∞ Z ∞
= e−sr f (r) dr e−su g (u) du
0 0
= F (s) G (s)

The other formula follows from changing the variable. ■

Note that F (s) could be a matrix and G (s) could be a vector. You simply need the
multiplication to make sense.

Example 30.3.7 Find a particular solution to

x′ (t) = Ax (t) +f (t)

where  
t
f (t) =  t
 

ln t 2 + 1

and  
−1 0 −6
A =  −2 1 −5 
 

1 0 4

There is no way you will find a decent closed
form solution to this in terms of elemen-
tary functions because of the horrible ln t 2 + 1 but this is not really a problem because
you can find a particular solution in terms of a convolution. You just need to find the fun-
damental matrix which is not hard. I will use the following to find the fundamental matrix.
syms s t; b=eye(3); c=[-1 0 -6;-2 1 -5;1 0 4];
simplify(ilaplace(inv(s*b-c)))
This yields for Φ (t)
 
−et (2et − 3) 0 −6et (et − 1)
 −et (t + et − 1) et −et (2t + 3et − 3) 
 

et (et − 1) 0 et (3et − 2)
30.4. HIGHER ORDER SCALAR LINEAR EQUATIONS 597

Then using the above theorem,x p (t) =

  
Z t −eu (2eu − 3) 0 −6eu (eu − 1) t −u
 −e (u + e − 1) e −e (2u + 3e − 3)    t − u
 u u u u u 


 du
0
 
eu (eu − 1) 0 eu (3eu − 2) ln (t − u)2 + 1

This gives a perfectly good description of a particular solution. Thus the general solution
is of the form
 
−et (2et − 3) 0 −6et (et − 1)
 −et (t + et − 1) et −et (2t + 3et − 3)  c + x p (t)
 

et (et − 1) 0 et (3et − 2)
Here c is an arbitrary vector in Fn . Note how this is essentially a return to the notion of the
variation of constants formula presented earlier.
Of course all of this depends on being able to say that if two functions have the same
Laplace transform, then they must in some sense be the same function. This will be dis-
cussed later when it will also be shown how to explicitly go backwards in the table and find
the original function given its Laplace transform.

30.4 Higher Order Scalar Linear Equations

Recall these are differential equations which are of the form
y(n) + an−1 (t) y(n−1) + · · · + a1 (t) y′ + a0 (t) y = f (t)
The notation y(k) means the kth derivative, and it is assumed that all given functions are
continuous. There is nothing new about these equations. They can all be studied as special
cases of first order systems.
The following is a procedure for changing one of these higher order linear equations
into a first order system which is the right way to study differential equations.
PROCEDURE 30.4.1 Consider the equation
y(n) + an−1 (t) y(n−1) + · · · + a1 (t) y′ + a0 (t) y = f
To write as a first order system, do the following.
   
x (1) y
 x (2)   y′
   

 . = .. 
 .   
 .   . 
x (n) y(n−1)

Then, suppressing the dependence on t,

 ′    
x (1) x (2) x (2)
 x (2)   x (3)   x (3) 
     
 .
..
  .  
 = . = .. 
  .   .
 
 
     
 x (n − 1)   x (n)   x (n) 
x (n) y(n) − (an−1 x (n) + · · · + a1 x (2) + a0 x (1)) + f
598 CHAPTER 30. LAPLACE TRANSFORM METHODS

In terms of matrices,
 ′  0 1 0 ··· 0
   
x (1) x (1) 0
 x (2)   .. .. 
   0 0 1 . .  x (2)  
  0 

..   
.. ..
..
   

.
 = .. .. .. 
.
+
.

   . . . . 0    
      
 x (n − 1)    0 ··· 0 0 1

 x (n − 1)   0 
x (n) −a0 −a1 ··· −an−2 −an−1 x (n) f

In case f = 0 so you have a homogeneous equation,

 ′  0 1 0 ··· 0
 
x (1) x (1)
 x (2)    . .. .. 
   0 0 1 .  x (2) 

.. ..

 =  ..
   

. . .. . .. . ..

.
 (30.8)
   . 0 
 
   
 x (n − 1)   0 ··· 0 0 1

 x (n − 1) 
x (n) −a0 −a1 · · · −an−2 −an−1 x (n)

it follows that in the above reduction to a first order system,

   
x (1) y
 x (2)   y′ 
   
 ..  
= .. 
. .
 
   
   (n−1) 
 x (n − 1)   y 
x (n) y(n−1)
where y is the solution to the higher order scalar equation and y is a solution to this scalar
higher order equation if and only if
 
y
 y′ 
 
 .. 
.
 
 
 (n−1) 
 y 
y (n−1)

is a solution to the above first order system.

When you have a fundamental matrix for A Φ (t), recall that the determinant of Φ (t)
is not zero because this matrix has an inverse, namely Φ (−t). In general, if you have
x′k = Axk for k ≤ p where A is a p × p matrix, you could form
 
Ψ (t) ≡ x1 x2 · · · x p (t) (30.9)
 
and Ψ′ = Ax1 Ax2 ··· Ax p = AΨ (t).

Theorem 30.4.2 Let Ψ (t) be as in 30.9 where x′k = Axk . Then Ψ (t)−1 exists for all t
if and only if Ψ (0)−1 exists and if this happens, then the fundamental matrix is Φ (t) =
Ψ (t) Ψ (0)−1 .
30.4. HIGHER ORDER SCALAR LINEAR EQUATIONS 599
 ′
Proof: ⇐Say Ψ (0)−1 exists. Then ΨΨ (0)−1 = (AΨΨ (0)) and so Φ (t) ≡ Ψ (t) Ψ (0)−1
is the fundamental matrix. Recall that there is only one and that it is invertible. Thus Ψ (t)−1
exists for all t.
⇒ If Ψ (t)−1 exists for all t, then this is true for t = 0. ■
The above says that if Ψ (t) is given by 30.9 then det (Ψ (t)) either vanishes for all t
or for no t. This determinant is called the Wronskian and this little observation is known
as the Wronskian alternative. Also note that the general solution is of the form Φ (t) c as
explained above. If Ψ (0) is invertible, this is Ψ (t) Ψ (0)−1 c but a generic c can be written
as Ψ (0)−1 Ψ (0) c and so the general solution is of the form Ψ (t) c exactly when Ψ (0)−1
exists.

Theorem 30.4.3 Consider the equation Ly ≡ y(n) +an−1 (t) y(n−1) +· · ·+a1 (t) y′ +a0 (t) y =
0 and suppose Lyk = 0 for k = 1, 2, · · · , n. Then every solution to Ly = 0 is of the form
∑nk=1 ck yk if and only if W (y1 , · · · , yn ) (t) ̸= 0 for some t. If this Wronskian condition holds
for some t, then it holds for all t. That is, the Wronskian vanishes identically or never.

In the case that W (y1 , · · · , yn ) (t) ̸= 0 for some t, we say that the general solution to
Ly = 0 consists of expressions of the form ∑nk=1 ck yk .
A useful way to recognize that you have the general solution in the case of second order
equations is as follows.

Proposition 30.4.4 Suppose yi , i = 1, 2 is a solution to

y′′ + p (t) y′ + q (t) y = 0

then the general solution to the equation is of the form

{c1 y1 + c2 y2 , c1 , c2 ∈ R}

if and only if y1 /y2 is nonconstant.

Proof: By the quotient rule,

y′ y1 − y′1 y2 W (y1 , y2 ) (t)

 
d y2
= 2 =
dt y1 (y1 )2 y21 (t)

and so the Wronskian is nonzero at some point if and only if y2 /y1 is not constant so that
the derivative of the quotient is not zero. ■
600 CHAPTER 30. LAPLACE TRANSFORM METHODS
Chapter 31

Numerical Solutions For Systems

You usually can’t factor the characteristic polynomial and so you usually can’t find explicit
solutions to the system

x′ (t) = Ax (t) + f (t) , x (0) = x0

Another serious difficulty is the case where f depends not just on t but also on x. This is
the case of nonlinear equations.
This is really just a more complicated problem than finding the integral when you are
unable to find an antiderivative in terms of known functions. In the simpler case of finding
integrals, there are numerical methods for determining the integral. It is no different in the
case of systems of ordinary differential equations.

31.1 A Few Numerical Methods

One way to obtain an approximate solution to a system of equations

y ′ = F (t, y) , y (0) = y 0

would be to replace the derivative with a difference quotient as follows:

y i − y i−1
= F (ti , y i ) , i ≥ 1, y 0 = x0
h
where here you have a uniform partition of [0, a] ,t0 < t1 < · · · < tn = a where h = ti − ti−1
and x0 is the given initial condition. I will leave it to you to see that there is a solution to
the above discrete problem. Then if you want, you could define a function y n (t) to be a
piecewise linear function which equals y i at ti . This is an example of a numerical solution.
The above method is called the Euler method. It isn’t as good as what a computer algebra
system will use. No one who is serious about getting numerical solutions will use this
method. A slightly better method is the improved Euler method.

PROCEDURE 31.1.1 To find a numerical solution to

y ′ = f (t, y) , y (0) = y 0

601
602 CHAPTER 31. NUMERICAL SOLUTIONS FOR SYSTEMS

using the improved Euler method, do the following:

h
k1 ≡ f (tk , y k ) , k2 ≡ f (tk + h, y k + k1 h) , tk = kh, y k+1 = y k + (k1 +k2 )
2
then there is some constant C such that |y (kh) − y k | < Ch2 .

It predicts what the slope should be at a point and then averages the two values to
advance another step.
This problem of getting solutions to first order systems of differential equations has
been studied extensively and a book like this is not the place to see a careful description of
the best methods. However, one of the very best was developed long before computers by
Runge and Kutta in 1901.

PROCEDURE 31.1.2 To find a numerical solution to

y ′ = f (t, y) , y (0) = y 0

using the Runge-Kutta algorithm, do the following: For tk = kh, k = 0, · · · ,

 
h h
k1 ≡ f (tk , y k ) , k2 ≡ f tk + , y k + k1 ,
2 2
 
h h
k3 ≡ f tk + , y k + k2 , k4 ≡ f (tk + h, y k + k3 h)
2 2
h
y k+1 = y k + (k1 + 2k2 + 2k3 + k4 )
6
then there is some constant C such that |y (kh) − y k | < Ch4 .

You can have MATLAB use the Runge-Kutta algorithm to numerically find a solution to
a system of ordinary differential equations. I will illustrate with a first order system which
comes from the Van der Pol equation. The exact equation studied is not too important at
this point. My intent is to illustrate the syntax used. Here it is:

f=@(t,x)[x(2),-((x(1)ˆ2-1)*x(2)+x(1))]; n=300; h=.05;

y(1,:)=[1,0]; t(1)=0;
hold on
for r=1:n
k1=f(t(r),y(r,:)); k2=f(t(r)+h/2,y(r,:)+k1*(h/2));
k3=f(t(r)+h/2,y(r,:)+k2*(h/2)); k4=f(t(r)+h,y(r,:)+k3*h);
y(r+1,:)=y(r,:)+(h/6)*(k1+2*k2+2*k3+k4); t(r+1)=t(r)+h;
end
plot(t,y(:,1),t,y(:,2))
interp1(t,[y(:,1),y(:,2)],[2,3,4,5,6])
Then press “enter”. This will compute the solution to the differential equation
!′ ! ! !
x y x (0) 1
=
, = .
− x2 − 1 y + x


y y (0) 0
31.2. USING MATLAB TO FIND SOLUTIONS 603

It will graph both components as functions of t, and it will give you a table of values at the
points t = 2, 3, 4, 5, 6.
To illustrate how the Runge Kutta algorithm works in comparison to the other two, con-
sider the initial value problem y′ = y, y (0) = 1. Then the three methods give the following
graphs.
The exact solution is y = et . In the picture,
the Euler method, improved Euler method, and
Runge-Kutta methods are used to solve the differ-
ential equation numerically. The diamonds on the
graph of y = et are from the Runge-Kutta method
and the other diamonds represent the result of us-
ing the other two methods. Note that the Runge-
Kutta points appear to be exactly right while the
other solutions stray from the true solution. When
one is evaluating numerical algorithms, it is in
general a good idea to experiment with them on
problems with a known answer. The improved Eu-
ler method is just what the words suggest but the
Runge Kutta method beats it easily. In all of these graphs, the step size was h = .2.

31.2 Using MATLAB to Find Solutions

A computer algebra system will use the best algorithms whenever possible.
!′ ! ! ! !
x t2 sin (x) x x (0) 0
= , =
y t +1 2 y y (0) 1
Then you would type in the following:

>>f=@(t,y)[-sin(y(2))*y(2)-y(2);(t+1)*y(1)+2*y(2)];
[t,x]=ode45(f,[0:.05:2],[0;1]); plot(t,x)
 T  T
The vector x y is denoted as y= y(1) y(2) . Then press “enter” and it will
graph these functions on [0, 2]. You should see two graphs, one for x (t) and one for y (t).
The first is 0 when t = 0 and the second is 1 when t = 0. In the second line, .05 is the
minimum step size for t. You can change this is you like.
If you want a table, you type in

>> s=ode45(f,[0,2],[0;1]);
deval(s,[0,.2,.4,.6,.8,1,1.2,1.4,1.6,1.8,2])

Then when you press ! “enter”, you get a table of column vectors which give the values of
y (1)
the vector at the specified values of t. You can also click on the data cursor icon
y (2)
on the top of the graph. Then place the little cross on a point of the curve which interests
you and left click. It will display the ordered pair on this point, a value for t and one for
604 CHAPTER 31. NUMERICAL SOLUTIONS FOR SYSTEMS

either y (1) or y (2). In the line which has ode45, if you type ode45(f,[0,2],[0;1]), then
MATLAB will decide on the step size for you.
Then, when you have what you want, you ought to type “clear all” and then “enter”
and then type clf and then “enter” to get rid of any figures. This is so you can do some-
thing else without closing MATLAB and starting it over again. MATLAB remembers the
functions which have been defined and so unless you do this, it may think you are referring
to something other than what you want if you do another computation without closing it
down.
You are not limited to systems which have two variables. For example, suppose you
wanted to get a solution to
 ′      
x −x3 + x − y x 0
 y  =  z − y + sin(z)  ,  y  (0) =  1 
       

z x z 1

Then you would enter something like the following.

>> f=@(t,y)[-y(1)ˆ3+y(1)-y(2);y(3)-y(2)+sin(y(3));y(1)];
[t,x]=ode45(f,[0:.05:2],[0;1;1]);plot(t,x)

Of course the solution to the initial value problem is a space curve. Suppose you wanted to
see the graph of this space curve. Try this

>> f=@(t,y)[-y(1)ˆ3+y(1)-sin(y(2));y(3)-y(2)+sin(y(3));y(1)];
[t,x]=ode45(f,[0:.05:30],[0;-1;1]);
plot3(x(:,1),x(:,2),x(:,3),’LineWidth’,2)

31.3 Stability of Equilibrium Points

Recall the simple case discussed earlier in which you are considering y′ = f (y) , y (a) = 0.
This is summarized in the following picture
unstable stable semi-stable
f ′ (a) > 0 f ′ (a) < 0 f ′ (a) = 0

a a a

The stability is determined by the sign of f ′ (a). However, if f ′ (a) = 0, then the equi-
librium point can be stable from one side and not from the other. In the example of the
above, it is stable from the left and unstable from the right. This is because if y is close
to a but less than a, the derivative y′ is positive so the solution to the differential equation
increases. If y is close to a but larger than a, then y′ > 0 and so the solution moves away
from a. Similar considerations show why the other two claims are so, stable if f ′ (a) < 0
and unstable if f ′ (a) > 0.
However, the situation is even more complicated when f ′ (a) = 0. You could have f
be strictly increasing through (a, 0) in which case, you would have that the equilibrium
31.3. STABILITY OF EQUILIBRIUM POINTS 605

point is unstable. Think f (y) = (y − a)3 . Then f has a 0 derivative at a but is increasing.
Similarly, you could have f decreasing through (a, 0) in which case, the equilibrium point
would be stable. The point is, anything can happen when f ′ (a) = 0.
You should regard f ′ (a) as an eigenvalue for the linear map x → f ′ (a) x. The eigen-
value is negative implies stability. The eigenvalue is positive implies not stable. The eigen-
value is 0 means anything can happen.
The situation is completely similar for nonlinear systems of equations.

Definition 31.3.1 Consider y ′ = f (y) , where we always assume f is C1 . A point a is

called an equilibrium point when f (a) = 0.

From the notion of differentiability,

f (a + y) = 0 + Df (a) y + o (y)

The situation which generalizes what happens with functions of one variable is as follows.
Let a be an equilibrium point for the differential equation y ′ = f (y). Thus f (a) = 0.

eigenvalues of Df (a) have negative real parts equilibrium is stable

some eigenvalue of Df (a) has positive real part equilibrium is unstable
some eigenvalue of Df (a) has zero real part you have no idea

So what exactly is meant by stable? It is the same as in the case of scalar valued
equations.

Definition 31.3.2 An equilibrium point a for y ′ = f (y) is stable if whenever the initial
condition y 0 is sufficiently close to a, it follows that the solution to the initial value problem
y ′ = f (y) , y (0) = y 0 will stay close to a. Also, a is asymptotically stable if whenever y 0
is close enough to a, then the solution of the initial value problem just described converges
to a as t → ∞.

In fact, one has a little more in case all eigenvalues are negative.

eigenvalues of Df (a) have negative real parts equilibrium is asymptotically stable

some eigenvalue of Df (a) has positive real part equilibrium is unstable
some eigenvalue of Df (a) has zero real part you have no idea
 T
Example 31.3.3 Consider the following system of equations for which 0 0 0 is
an equilibrium point. Determine whether this is a stable equilibrium.
 ′  
x xy − 12z − 5x
   2
 y  =  y − y − 2x − 6z 


z xy2 + x + 2z

You need to get the derivative of the right side. The matrix of this is
 
y−5 x −12
 −2 2y − 1 −6 
 

y2 + 1 2xy 2
606 CHAPTER 31. NUMERICAL SOLUTIONS FOR SYSTEMS

At the equilibrium point, you get

 
−5 0 −12
 −2 −1 −6 
 

1 0 2

The eigenvalues are −1, −2, −1 and so this equilibrium point is stable.

Example 31.3.4 The point (1, 0, 0)T is an equilibrium point of the following system
 ′  
x 11x + 11y − 12z + xy − 11
 y  = 6z − 7y − 6x + yz + 6
   

z 4x + 4y − 6z + xz − 4

Determine whether this point is stable.

You need to find the derivative. It equals

 
y + 11 x + 11 −12
 −6 z−7 y+6 
 

z+4 4 x−6

At the equilibrium point you get

 
11 12 −12
 −6 −7 6 
 

4 4 −5

Now you consider the eigenvalues for this matrix. In this case, there is a positive eigenvalue
and so the equilibrium point is unstable.
Of course there is a problem with this. How do you find the sign of the eigenvalues.
You don’t need to know the eigenvalues exactly, just their signs. However, MATLAB can
tell you the approximate eigenvalues. To find them in this case, you do the following.

A=[11 12 -12;-6 -7 6;4 4 -5];eig(A)

You enter the rows starting with the top row and then the next and so forth. You type
the numbers from left to right leaving a space between numbers or you can put a comma
between them. When you start a new row, you tell MATLAB this is the case by placing ;
there. Then type eig(A) and press enter. It will give you the eigenvalues.
 T
Example 31.3.5 1 1 0 is an equilibrium point for the differential equation
 ′  
x 7x + 14y − 3z + xy − 22
 y  =  z − 9y − 5x + yz + 14 
   

z z2 − 2z + 3x + 5y − 8

Determine its stability.

31.3. STABILITY OF EQUILIBRIUM POINTS 607

Find the derivative.  

y+7 x + 14 −3
 −5 z−9 y+1 
 

3 5 2z − 2
Now find this at the equilibrium point.
 
8 15 −3
 −5 −9 2 
 

3 5 −2

Next you need to consider the real parts of the eigenvalues. Use MATLAB. This gives the
eigenvalues are −1 + i, −1 − i, and −1 so they have negative real parts and this shows that
the equilibrium point is stable.
When the eigenvalues include one which has real part equal to 0 and none of them
having positive part larger than 0, then you really don’t know much. Nevertheless, there
is a way to consider this case also, but I do not plan to include it in this book. It involves
something called the center manifold and understanding it properly requires a little too
much hard mathematics. However, it is also true that in many cases of interest, the system
takes place in the plane and in this case, you can often figure out what is happening by
simply having MATLAB graph the space curves resulting from various initial conditions.
For example, consider !′ !
x −4y2
=
y 2x
If you graph the space curves which result from many different small initial conditions, you
will see that (0, 0) is a stable point although not asymptotically stable.
hold on
r=.1; f=@(t,x)[-4*x(2)ˆ3;2*x(1)];
for n=1:10
[t,x]=ode45(f,[0,40],[0,n*.1]);
plot(x(:,1),x(:,2),’LineWidth’,1.3)
end
You could modify this just a little and find a situation where (0, 0) is asymptotically
stable. !′ !
x −4y2
=
y 2x + .1y
hold on
r=.1; f=@(t,x)[-4*x(2)ˆ3;2*x(1)+.1*x(2)];
for n=1:3
[t,x]=ode45(f,[0,40],[0,n*.1]);
plot(x(:,1),x(:,2),’LineWidth’,1.3)
end
If you solve numerically and graph the solution using the above syntax, you will see
the solution spiral in towards (0, 0).
A great deal more can be said concerning stability and more generally the geometric
behavior of solutions to ordinary differential equations, especially for systems which have
608 CHAPTER 31. NUMERICAL SOLUTIONS FOR SYSTEMS

solutions in the plane. In the next section are some major results about these things. For
much more see a text on ordinary differential equations. My book has a good deal more
discussion. See [25].

31.4 Periodic Orbits, Poincare Bendixon Theorem

The fundamental result in this subject, at least in the plane, is the Poincare Bendixon theo-
rem.1

Definition 31.4.1 A periodic orbit is a set of points

{x (t, x0 ) ,t ≥ 0}

such that for some T > 0, x (t + T, x0 ) = x (t, x0 ) for all t ≥ 0. The number T is called a
period. Thus the point x (t, x0 ) goes around and around always returning to the point from
where it started.

Now the following is the Poincare Bendixon theorem which gives existence of periodic
orbits in the plane.

Theorem 31.4.2 Let D be the closure of a bounded region of the plane such that f is a
C1 function which has no zeros in D, and suppose that x (t, x0 ) stays in D for all t ≥ 0 if
x0 ∈ D, where this is the solution to

x′ = f (x) , x (0) = x0

Then letting Λ+ = ∪t≥0 x (t, x0 ) , it follows that Λ+ is either a periodic orbit or t → x (t, x0 )
spirals in toward a periodic orbit.

It is a plausible result. Say you have that every initial condition which starts off in a
bounded closed set stays in that set and there are no equilibrium points. Thus t → x (t, x0 )
just keeps moving. Then from this theorem, there must be a periodic orbit somewhere such
that either this function traces out a periodic orbit or it gets close to one. For example,
consider the system !′
!
x x + y − x x2 + 2y2
=
−x + y − y 2x2 + y2


y
From looking at the eigenvalues of the matrix in the almost linear system, you will see that
they are both positive. Hence every solution near (0, 0) but not equal to (0, 0) must fail to
remain near (0, 0). Also, you can see from the equations that the solutions cannot get very
large because the sign of x′ will change to oppose |x| getting large and a similar condition
happens for y′ . Therefore, there should exist a periodic orbit from the above theorem.
The following picture illustrates what happens for various initial conditions. Note how the
solutions spiral in toward a periodic orbit.
1 Ivar Otto Bendixson (1861-1935) was a Swedish mathematician. He is most famous for the Poincare

Bendixon theorem presented here. He also did work in topology.

31.5. EXERCISES 609

-1
-1 0 1
Definition 31.4.3 A saddle point x0 for x′ = f (x) is an equilibrium point (f (x0 ) = 0)
which is not stable, but which has the property that in in every set of the form
{x : r > |x − x0 | > 0}

there are points for which the solution having these as initial conditions converges to x0
as t → ∞. These saddle points occur for example if you have a negative and a positive
eigenvalue for Df (x0 ).
The following very interesting theorem can be obtained from the above.
Theorem 31.4.4 If you have a periodic orbit of a solution to an autonomous two dimen-
sional differential equation, x′ = f (x) , then it must go around some equilibrium point.
If there is only one equilibrium point inside the periodic orbit, then it cannot be a saddle
point.

31.5 Exercises
1. The Van der Pol equation describes nonlinear oscillations. It is
x′′ + x2 − 1 x′ + x = 0


(31.1)
Show that it has a non constant periodic solution. Do as follows. First write as a first
order system
x′ = y
′
y = − x2 − 1 y + x

Have MATLAB or some other computer algebra system give a graph of solutions for
the above system corresponding to various initial conditions in a way to show the
periodic solution. Try the following:

f=@(t,x)[x(2);-((x(1)ˆ2-1)*x(2)+x(1))];
d=pi/4; r=.3;
hold on
for n=1:9
[t,x]=ode45(f,[0:.05:10],[n*r*cos(n*d);n*r*sin(n*d)]);
plot(x(:,1),x(:,2),’LineWidth’,1.5)
end
610 CHAPTER 31. NUMERICAL SOLUTIONS FOR SYSTEMS

Yo should get something like the following.

-2

-4
-2 0 2
Chapter 32

Solutions Near a Regular

Singular Point

This chapter is on solutions to an equation of the form

x2 y′′ + xp (x) y′ + q (x) y = 0

where p (x) and q (x) can be expressed in terms of a power series centered at 0. Such
equations are said to have a regular singular point. This is very different because there is
generally no way to write such an equation in the form

y′′ + p (x) y′ + q (x) y = 0

where p (x) , q (x) are continuous near 0. Thus the initial value problem makes no sense.
Thus, none of the above theory applies to these equations and further analysis is needed.
Equations of this sort were found to be very important in the nineteenth century for various
reasons. For more on these topics, you can see my book [25]. What is here is a subset of
the contents of this book. Also, the most important example of this kind of equation is the
Bessel equation. Whole books are available on this which will develop more of the theory
than presented here or in my differential equations book. See [17].

32.1 The Euler Equations

The simplest equation to illustrate the concept of a regular singular point is the so called
Euler equation, sometimes called a Cauchy Euler equation.

Definition 32.1.1 A differential equation is called an Euler equation if it can be written in

the form
x2 y′′ + axy′ + by = 0.

Solving a Cauchy Euler equation is really easy. You look for a solution like y = xr
and try to choose r in such a way that it solves the equation. Plugging this in to the above
equation,
x2 r (r − 1) xr−2 + xarxr−1 + bxr = 0

611
612 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

This reduces to
xr (r (r − 1) + ar + b) = 0
and so you have to solve the equation

r (r − 1) + ar + b = 0

to find the values of r. If these values of r are different, say r1 ̸= r2 then the general solution
must be
C1 xr1 +C2 xr2
because the Wronskian of the two functions will be nonzero. I know this because the ratio
of the two functions is not a constant so Proposition 30.4.4 implies this gives the general
solution. The reason for this is that the quotient rule gives the numerator as ±1 times the
Wronskian.

Example 32.1.2 Find the general solution to x2 y′′ − 2xy′ + 2y = 0.

You plug in xr and look for r. Then as above this yields

r (r − 1) − 2r + 2 = r2 − 3r + 2 = 0

and so the two values of r are 1, 2. Therefore, the general solution to this equation is

C1 x +C2 x2 .

Of course there are three cases for solutions to the so called indicial equation

r (r − 1) + ar + b = 0

Either the zeros are distinct and real, distinct and complex or repeated. Consider the case
where they are distinct and complex next.

Example 32.1.3 Find the general solution to x2 y′′ + 3xy′ + 2y = 0.

This time you have

r2 + 2r + 2 = 0
and the solutions are r = −1 ± i. How do we interpret

x−1+i , x−1−i ?

It is real easy. You assume always that x > 0 since otherwise the leading coefficient could
vanish. Then
x−1+i = eln(x)(−1+i) = e− ln(x)+i ln(x)
and by Euler’s formula this equals
−1
x−1+i = eln(x ) (cos (ln (x)) + i sin (ln (x)))
1
= (cos (ln (x)) + i sin (ln (x)))
x
Corresponding to x−1−i we get something similar.
1
x−1−i = ((cos (ln (x)) − i sin (ln (x))))
x
32.1. THE EULER EQUATIONS 613

Adding these together and dividing by 2 to get the real part, the principle of superposition
implies
1
cos (ln (x))
x
is a solution. Then subtracting them and dividing by 2i you get
1
sin (ln (x))
x
is a solution. Hence anything of the form
1 1
C1 cos (ln (x)) +C2 sin (ln (x))
x x
is a solution. Is this the general solution? Of course. This follows because the ratio of the
two functions is not constant and this implies their Wronskian is nonzero. See Proposition
30.4.4.
In the general case, suppose the solutions of the indicial equation

r (r − 1) + ar + b = 0 (32.1)

are α ± iβ . Then the general solution for x > 0 is

C1 xα cos (β ln (x)) +C2 xα sin (β ln (x))

Finally consider the case where the zeros of the indicial equation are real and repeated.
Note I have included all cases because, since the coefficients of this equation are real, the
zeros come in conjugate pairs if they are not real. Suppose then that xr is a solution of

x2 y′′ + axy′ + by = 0

and that r is a repeated root. By the quadratic formula applied to the indicial equation 32.1,

− (a − 1)
r= (32.2)
2

Then if z (x) is another solution which is not a multiple of xr , you would have

z (x) = xr u (x)

The plug in to the equation and try to make it work.

x2 (xr u (x))′′ + ax (xr u (x))′ + bxr u (x) = 0

Then
x2 r (r − 1) xr−2 u + 2rxr−1 u′ + xr u′′ + ax rxr−1 u + xr u′ + bxr u = 0
 

Separate out the terms which multiply u

 
=0
z }| {
u r (r − 1) + ar + b xr = 0
 
614 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

All that is left is

xr+2 u′′ + 2rxr+1 + axr+1 u′ = 0

Thus, using 32.2,

xr+2 u′′ + (−a + 1) xr+1 + axr+1 u′ = 0

Therefore,
xu′′ + u′ = 0
and this is a first order linear equation for u′ . Thus, a nonzero solution to this is
1
u′ =
x
Therefore, if u = ln x, it follows that z (x) = u (x) xr is a solution to the Euler equation with
the repeated roots and so another solution is

z = xr ln (x)

Example 32.1.4 Find the general solution of the equation

x2 y′′ + 3xy′ + y = 0.

In this case the indicial equation is

r (r − 1) + 3r + 1 = r2 + 2r + 1 = 0

and there is a repeated zero, r = −1. Therefore, the general solution is

y = C1 x−1 +C2 ln (x) x−1 .

This is pretty easy isn’t it?

How would things be different if the equation was of the form

(x − a)2 y′′ + a (x − a) y′ + by = 0?

The answer is that is wouldn’t be any different. You could just define a new independent
variable t ≡ (x − a) and then the equation in terms of t becomes

t 2 z′′ + atz + bz = 0

where z (t) ≡ y (x) = y (t + a) . You can always reduce these sorts of equations to the case
where the singular point is at 0. However, you might not want to do this. If not, you look
for a solution in the form y = (x − a)r , plug in and determine the correct value of r. In the
case of real and distinct zeros you get

y = C1 (x − a)r1 +C2 (x − a)r2

In the case where r = α ± iβ you get

y = C1 (x − a)α cos (β ln (x − a)) +C2 (x − a)α sin (β ln (x − a))

for the general solution for x > a

In the case where r is a repeated zero, you get

y = C1 (x − a)r +C2 ln (x − a) (x − a)r .

32.2. SOME SIMPLE OBSERVATIONS ON POWER SERIES 615

32.2 Some Simple Observations on Power Series

This section is a review of a few facts about power series which should have been learned
in calculus. If you have not seen these things, which may well be the case given the way
calculus courses are systematically watered down, see my book Calculus of one and many
variables on my web page or my differential equations book [25].

Definition 32.2.1 A function f is analytic in some open set U if for each a ∈ U, f (x) =
k
∑∞
k=0 ak (x − a) for all x close enough to a. In other words, you can get the function near a
by a power series.
n
Theorem 32.2.2 Suppose f (x) = ∑∞
n=0 an (x − a) for x near a and suppose a0 ̸= 0. Then

1
f (x)−1 = + h (x)
a0
n
where h (x) = ∑∞
n=1 bn (x − a) so h (a) = 0.

Proof: It turns out that f (x)−1 has a power series representation near a and so f (a)−1 =
1/a0 . ■

Theorem 32.2.3 Suppose f (x) = ∑∞ n ∞ n

n=0 an x and g (x) = ∑n=0 bn x for x near 0. Then
f (x) g (x) also has a power series near 0 and in fact,
!
∞ n
f (x) g (x) = ∑ ∑ an−k bk xn . (32.3)
n=0 k=0

Proof: See the material on power series in my calculus book. However, it is quite
plausible.
! !
∞ ∞
∑ an x n ∑ bn xn
n=0 n=0

a0 + a1 x + a2 x 2 + · · · b0 + b1 x + b2 x2 + · · ·



=

Now formally multiply the two power series like they were polynomials and collect terms.
This will yield 32.3. ■

32.3 Regular Singular Points

First of all, here is the definition of what a regular singular point is.

Definition 32.3.1 A differential equation has a regular singular point at 0 if the equation
can be written in the form
x2 y′′ + xb (x) y′ + c (x) y = 0 (32.4)
where
∞ ∞
b (x) = ∑ bn xn , ∑ cn xn = c (x)
n=0 n=0
616 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

for all x near 0. Such functions are called analytic in this section. More generally, a differ-
ential equation
P (x) y′′ + Q (x) y′ + R (x) y = 0 (32.5)
where P, Q, R are analytic near a has a regular singular point at a if it can be written in the
form
(x − a)2 y′′ + (x − a) b (x) y′ + c (x) y = 0 (32.6)
where
∞ ∞
b (x) = ∑ bn (x − a)n , ∑ cn (x − a)n = c (x)
n=0 n=0

for all |x − a| small enough. The equation 32.5 has a singular point at a if P (a) = 0.

The following table emphasizes the similarities between the Euler equations and the
regular singular point equations. I have featured the point 0. If you are interested in another
point a, you just replace x with x − a everywhere it occurs.

Euler equation regular singular point

x2 y′′ + x (b0 + b1 x + · · · ) y′
form of equation x2 y′′ + xb0 y′ + c0 y = 0
+ (c0 + c1 x + · · · ) y = 0
indicial equation r (r − 1) + b0 r + c0 = 0 r (r − 1) + b0 r + c0 = 0
one solution y = xr y = xr ∑∞ k
k=0 ak x , a0 = 1.

Recognizing Regular Singular Points

How do you know a singular differential equation can be written a certain way? In
particular, how can you recognize a regular singular point when you see one? Suppose

P (x) y′′ + Q (x) y′ + R (x) y = 0

where all of P, Q, R are analytic functions near a. How can you tell if it has a regular
singular point at a? Here is how. It has a regular singular point at a if

Q (x)
lim (x − a) exists
x→a P (x)

R (x)
lim (x − a)2 exists
x→a P (x)
If these conditions hold, then by theorems in complex analysis it will be the case that
∞
Q (x)
(x − a) = ∑ bn (x − a)n ,
P (x) n=0

and
∞
R (x)
(x − a)2 = ∑ cn (x − a)n
P (x) n=0
for x near a. Indeed, equations of this form reduce to the form in 32.6 upon dividing by
P (x) and multiplying by (x − a)2 .
32.3. REGULAR SINGULAR POINTS 617

Example 32.3.2 Find the regular singular points of the equation and find the singular
points.
x3 (x − 2)2 (x − 1)2 y′′ + (x − 2) sin (x) y′ + (1 + x) y = 0

The singular points are 0, 2, 1. Let’s consider 0 first.

(x − 2) sin (x)
lim x
x→0 x3 (x − 2)2 (x − 1)2

does not exist. Therefore, 0 is not a regular singular point. I don’t have to check any further.
Now consider the singular point 2.

(x − 2) sin (x) 1
lim (x − 2) = sin 2
x→2 x3 (x − 2)2 (x − 1)2 8

and
1+x 3
lim (x − 2)2 =
x→2 x3 (x − 2)2 (x − 1)2 8
and so yes, 2 is a regular singular point. Now consider 1.

(x − 2) sin (x)
lim (x − 1)
x→1 x3 (x − 2)2 (x − 1)2

does not exist so 1 is not a regular singular point. Thus the above equation has only one
regular singular point and this is where x = 2.

Example 32.3.3 Find the regular singular points of

x sin (x) y′′ + 3 tan (x) y′ + 2y = 0

The singular points are 0, nπ where n is an integer. Let’s consider a point at nπ where
n ̸= 0. To be specific, let’s let n = 3

3 tan (x)
lim (x − 3π) =0
x→3π x sin (x)

Similarly the limit exists for other values of n. Now consider

2
lim (x − 3π)2 =0
x→3π x sin (x)

Similarly the limit exists for other values of n. What about 0?

3 tan (x)
lim x =3
x→0 x sin (x)

and
2
lim x2 =2
x→0 x sin (x)
so it appears all these singular points are regular singular points.
618 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

Example 32.3.4 Find the regular singular points of

x2 sin (x) y′′ + 3 tan (x) y′ + 2y = 0

Let’s look at x = 0 first. The equation has the same singular points.
3 tan (x)
lim x = undefined
x→0 x2 sin (x)
so 0 is not a regular singular point.
3 tan (x)
lim (x − 3π) =0
x→3π x2 sin (x)
and the situation is similar for other singular points nπ. Also
2
lim (x − 3π)2 =0
x→3π x2 sin (x)
with similar result for arbitrary nπ where n ̸= 0. Thus in this case 0 is not a regular singular
point but nπ is a regular singular point for all integers n ̸= 0.
In general, if you have an equation which has a regular singular point at a so that the
equation can be massaged to give something of the form

(x − a)2 y′′ + (x − a) b (x) y′ + c (x) y = 0

you could always define a new variable t ≡ (x − a) and letting z (t) = y (x) , you could
rewrite the equation in terms of t in the form

t 2 z′′ + tb (a + t) z′ + c (a + t) z = 0

and thereby reduce to the case where the regular singular point is at 0. Thus there is no
loss of generality in concentrating on the case where the regular singular point is at 0. In
addition, the most important examples are like this. Therefore, from now on, I will consider
this case. This just means you have all the series in terms of powers of x rather than the
more general powers of x − a.

32.4 Abel’s Formula

Suppose you have a differential equation

y′′ + p (x) y′ + q (x) y = 0

and you have two solutions to it y1 , y2 . Abel’s formula is a lovely little identity for the
Wronskian of these two functions. It gives another way to show that the Wronskian either
vanishes identically or not at all. From the equation,

y2 y′′1 + p (x) y2 y′1 + q (x) y2 y1 = 0

y1 y′′2 + p (x) y1 y′2 + q (x) y2 y1 = 0

Now subtract.
y1 y′′2 − y2 y′′1 + p (x) y1 y′2 − y2 y′1 = 0



(32.7)
32.5. FINDING THE SOLUTION 619

From the product rule,

′
y1 y′2 − y2 y′1 = y′′2 y1 + y′1 y′2 − y′′1 y2 + y′1 y′2 = y1 y′′2 − y2 y′′1

but y1 y′2 − y2 y′1 = W (y1 , y2 ) . Hence 32.7 is of the form

W ′ + p (x)W = 0

and so, from the theory of linear equations,

W (x) = Ce−P(x) where P′ (x) = p (x) .

This proves Abel’s formula.

Proposition 32.4.1 Let y1 , y2 be two solutions to y′′ + p (x) y′ + q (x) y = 0 for x in some
interval on which p (x) , q (x) are continuous. Then

W (y1 , y2 ) (x) = Ce−P(x) , P′ (x) = p (x) .

Note how this shows directly that the Wronskian either vanishes identically or not at
all. This also motivates the following procedure.

PROCEDURE 32.4.2 Suppose y is a known solution to

y′′ + p (x) y′ + q (x) y = 0 (32.8)

To find another solution z which solves

z′′ + p (x) z′ + q (x) z = 0

do the following: For

y z
W=
y′ z′
Find a nonzero solution W (x) of

W ′ (x) + p (x)W (x) = 0

Then solve for z in the equation

z′ y − zy′ = W

BE SURE THAT THE EQUATION IS IN THE FORM DESCRIBED IN THE

ABOVE PROCEDURE. THIS MEANS THE COEFFICIENT OF y′′ IS 1!

32.5 Finding the Solution

Suppose you have reduced the equation to

x2 y′′ + xp (x) y′ + q (x) y = 0 (32.9)

where each of p, q is analytic near 0. Then letting

p (x) = b0 + b1 x + · · ·
620 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

q (x) = c0 + c1 x + · · ·
you see that for small x the equation should be approximately equal to

x2 y′′ + xb0 y′ + c0 y = 0

which is an Euler equation. This would have a solution in the form xr where

r (r − 1) + b0 r + c0 = 0,

the indicial equation for the Euler equation, and so it is not unreasonable to look for a
solution to the equation in 32.9 which is of the form The values of r are called the exponents
of the singularity.
∞
xr ∑ ak xk , a0 ̸= 0.
k=0
You perturb the coefficients of the Euler equation to get 32.9 and so it is not unreasonable
to think you should look for a solution to 32.9 of the above form.

Example 32.5.1 Find the general solution to the equation

x2 y′′ + x 1 + x2 y′ − 2y = 0.

The associated Euler equation is of the form

x2 y′′ + xy′ − 2y = 0

and so the indicial equation is

r (r − 1) + r − 2 = 0 (32.10)
√ √
so r = 2, r = − 2. Then you would look for a solution in the form
∞ ∞
y = xr ∑ ak xk = ∑ ak xk+r
k=0 k=0
√
where r = ± 2. Plug in to the equation.
∞
x2 ∑ ak (k + r) (k + r − 1) xk+r−2
k=0

∞ ∞
+x 1 + x2 ∑ ak (k + r) xk+r−1 − 2 ∑ ak xk+r = 0


k=0 k=0
This simplifies to
∞ ∞
∑ ak (k + r) (k + r − 1) xk+r + ∑ ak (k + r) xk+r (32.11)
k=0 k=0

∞ ∞
+ ∑ ak (k + r) xk+r+2 − 2 ∑ ak xk+r = 0
k=0 k=0
The lowest order term is the xr term and it yields

a0 (r) (r − 1) + a0 (r) − 2a0 = 0

32.5. FINDING THE SOLUTION 621

but this is just a0 (r (r − 1) + r − 2) = 0. Since r is one of the zeros of 32.10, there is no

restriction on the choice of a0 . In fact, as discussed below, this lack of a requirement on
a0 is equivalent to finding the right value of r. Next consider the xr+1 terms. There are no
such terms in the third of the above sums just as there were no xr terms in this sum. Then

a1 ((1 + r) (r) + (1 + r) − 2) = 0

Now if r solves 32.10 then 1 + r does not do so because the two solutions to this equation
do not differ by an integer. Therefore, the above equation requires a1 = 0. At this point we
can give a recurrence relation for the other ak . To do this, change the variable of summation
in the third sum of 32.11 to obtain
∞ ∞
∑ ak (k + r) (k + r − 1) xk+r + ∑ ak (k + r) xk+r
k=0 k=0
∞ ∞
+ ∑ ak−2 (k − 2 + r) xk+r − 2 ∑ ak xk+r = 0
k=2 k=0
Thus for k ≥ 2,

ak [(k + r) (k + r − 1) + (k + r) − 2] + ak−2 (k − 2 + r) = 0

Hence for k ≥ 2,
−ak−2 (k − 2 + r) −ak−2 (k − 2 + r)
ak = =
[(k + r) (k + r − 1) + (k + r) − 2] [(k + r) (k + r − 1) + (k + r) − 2]
and we take a0 ̸= 0 while
√ a1 = 0. Now let’s find the√ first several terms of two independent
solutions, one for r = 2 and the other for r = − 2. Let a0 = 1 for simplicity. Then the
above recurrence relation shows that since a1 = 0 all the odd terms equal 0. Also
−r r
a2 = =−
[(2 + r) (2 + r − 1) + (2 + r) − 2] [(2 + r) (1 + r) + r]
while  
r
− − [(2+r)(1+r)+r] (4 − 2 + r) r 2+r
a4 = =
[(4 + r) (4 + r − 1) + (4 + r) − 2] [2 + 4r + r2 ] [14 + 8r + r2 ]
Continuing this way, you can get as many terms as you want. Now √ let’s put in the two
values of r to obtain the beginning of the two solutions. First let r = 2

√
  √ 
2
y1 (x) = x 2 1 + − h √  √  √ i  x2 +
2+ 2 2+1 + 2
 √ √  
2 2+ 2  4 
+ h √ ih √ i x ···
4 + 4 2 16 + 8 2
√
the solution which corresponds to r = − 2 is

√
  √ 
2
y2 (x) = x− 2 1 +  h √  √  √ ix +
2
2− 2 1− 2 − 2
622 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

√ 
√ −2 + 2 4
2h √ ih √ ix +···
4 − 4 2 16 − 8 2
Then the general solution is
C1 y1 +C2 y2
and this is valid for x > 0. Note that the ratio of the two solutions is not a constant so this
is indeed the general solution.

Generalities

For an equation
x2 y′′ + xp (x) y′ + q (x) y = 0
having a regular singular point at 0, one looks for solutions in the form
∞
y (x) = ∑ an xr+n (32.12)
n=0

where r is a constant which is to be determined, in such a way that a0 ̸= 0. It turns out that
such equations always have such solutions although solutions of this sort are not always
enough to obtain the general solution to the equation. The constant r is called the exponent
of the singularity because the solution is of the form

xr a0 + higher order terms.

Thus the behavior of the solution to the equation given above is like xr for x near the
singularity, 0.
If you require that 32.12 solves 32.9 and plug in, you obtain using Theorem 32.2.3
!
∞ ∞ n
∑ (r + n) (r + n − 1) an xn+r + + ∑ ∑ ak (k + r) bn−k xn+r
n=0 n=0 k=0
!
∞ n
+∑ ∑ cn−k ak xn+r = 0. (32.13)
n=0 k=0

Since a0 ̸= 0,
p (r) ≡ r (r − 1) + b0 r + c0 = 0 (32.14)
and this is called the indicial equation. (Note it is the indicial equation for the Euler equa-
tion which comes from deleting all the nonconstant terms in the power series for p (x) and
q (x).) Also the following equation must hold for n = 1, · · · .
n−1 n−1
p (n + r) an = − ∑ ak (k + r) bn−k − ∑ cn−k ak ≡ fn (ai , bi , ci ) (32.15)
k=0 k=0

These equations are all obtained by setting the coefficient of xn+r equal to 0.
There are various cases depending on the nature of the solutions to this indicial equa-
tion. I will always assume the zeros are real, but will consider the case when the zeros are
distinct and do not differ by an integer and the case when the zeros differ by a non negative
integer.
32.5. FINDING THE SOLUTION 623

It turns out that the nature of the problem changes according to which of these cases
holds. You can see why this is the case by looking at the equations 32.14 and 32.15. If r1 , r2
solve the indicial equation and r1 − r2 ̸= an integer, then with r in equation 32.15 replaced
by either r1 or r2 , for n = 1, · · · , p (n + r) ̸= 0 and so there is a unique solution to 32.15 for
each n ≥ 1 once a0 ̸= 0 has been chosen. Therefore, in this case that r1 − r2 ̸= an integer,
equation 32.4 has a general solution in the form
∞ ∞
C1 ∑ an xn+r1 +C2 ∑ bn xn+r2 , a0 , b0 ̸= 0.
n=0 n=0

It is obvious this is the general solution because the ratio of the two solutions is non con-
stant. As pointed out earlier, this requires their Wronskian to be nonzero.
On the other hand, if r1 − r2 = an integer, then there exists a unique solution to 32.15
for each n ≥ 1 if r is replaced by the larger of the two zeros r1 . Therefore, in this case there
is always a solution of the form
∞
y1 (x) = ∑ an xn+r1 , a0 = 1, (32.16)
n=0

but you might very well hit a snag when you attempt to find a solution of this form with
r1 replaced with the smaller of the two zeros r2 due to the possibility that for some m ≥ 1,
p (m + r2 ) = p (r1 ) = 0 without the right side of 32.15 vanishing. In the case when both
zeros are equal, there is only one solution of the form in 32.16 since there is always a
unique solution to 32.15 for n ≥ 1. Therefore, in the case when r1 − r2 = a non negative
integer either 0 or some positive integer, you must consider other solutions. I will use
Abel’s formula to find the second solution. The equation solved by these two solutions is

x2 y′′ + xp (x) y′ + q (x) y = 0

and dividing by x2 to place in the right form for using Abel’s formula, Proposition 32.4.1.
1 1
y′′ + p (x) y′ + 2 q (x) y = 0
x x
Thus letting y1 be the solution of the form in 32.16, and y2 another solution, Abel’s formula
gives
p (x)
Z
y′2 y1 − y2 y′1 = W ∈ e−P(x) dx, P′ (x) =
x
Thus, following Procedure 32.4.2
y′1 1
y′2 − y2 = W
y1 y1

Then using an integrating factor eln(1/|y1 |) = 1

|y1 |
 
d 1 1 1
y2 = W = ± 2W
dx |y1 | |y1 | y1 y1
The sign does not matter to whether you have a solution, so it suffices to let
 
d 1 1
y2 = 2 W
dx y1 y1
624 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

Taking antiderivatives, another solution is

1 −P
Z
y2 ∈ y1 e dx
y21

where P (x) ∈ x−1 p (x) dx. Thus

R

Z  
b0
P (x) ∈ + b1 + b2 x + · · · dx = b0 ln x + b1 x + b2 x2 /2 + · · ·
x
and so
−P (x) = ln x−b0 + k (x)
for k (x) some analytic function, k (0) = 0. Therefore,
−b0
e−P(x) = eln(x )+k(x) = x−b0 g (x)

for g (x) some analytic function, g (0) = 1. Therefore,

1  −b0
Z 
y2 ∈ y1 (x) 2
x g (x) dx, g (0) = 1. (32.17)
y1

Next it is good to understand y1 and r1 in terms of b0 . Consider the zeros to the indicial
equation,
r (r − 1) + b0 r + c0 = r2 − r + b0 r + c0 = 0.
It is given that r1 = r2 + m where m is a non negative integer. Thus the left side of the above
equals
(r − r2 ) (r − r2 − m) = r2 − 2rr2 − rm + r22 + r2 m
and so
−2r2 − m = b0 − 1
which implies
1 − b0 m
r2 = −
2 2
and hence
1 − b0 m
r1 = r2 + m = +
2 2
1−b0 +m ∞
y1 (x) = x 2
∑ an xn , a0 = 1 (32.18)
n=0

Now from Theorem 32.2.2 and looking at 32.18 y1 (x)−2 is of the form
1
= xb0 −1−m (1 + h (x))
x1−b0 +m (∑∞ n 2
n=0 an x )

where h (x) is analytic, h (0) = 0. Therefore, 32.17 is

Z  
y2 (x) ∈ y1 (x) xb0 −1−m (1 + h (x)) x−b0 g (x) dx
Z
y2 (x) ∈ y1 (x) x−1−m (1 + l (x)) dx, l (0) = 0 (32.19)
32.5. FINDING THE SOLUTION 625

Now suppose that m > 0. Then,

y2 (x) −x−m m−1 xn−m ∞
xn−m
= + ∑ An + Am ln (x) + ∑ An .
y1 (x) m n=1 n−m n=m+1 n−m
It follows
y1
!z { }|
∞ ∞
−m −1
y2 = Am ln (x) y1 + x + ∑ Bn x x ∑ an xn .
n r1
m n=1 n=0
An
Where Bn = n−m for n ̸= m. Therefore, y2 has the following form.
∞
y2 = Am ln (x) y1 + xr2 ∑ Cn xn .
n=0

If m = 0 so there is a repeated zero to the indicial equation then 32.19 implies

∞
y2 An n
= ln x + ∑ x + A0
y1 n=1 n

where A0 is a constant of integration. Thus, the second solution is of the form

∞
y2 = ln (x) y1 + xr2 ∑ Cn xn .
n=0

The following theorem summarizes the above discussion.

PROCEDURE 32.5.2 Let 32.4 be an equation with a regular singular point and
let r1 and r2 be real solutions of the indicial equation, 32.14 with r1 ≥ r2 . Then if r1 − r2 is
not equal to an integer, the general solution 32.4 may be written in the form :
∞ ∞
C1 ∑ an xn+r1 +C2 ∑ bn xn+r2
n=0 n=0

where we can have a0 = 1 and b0 = 1. If r1 = r2 = r then the general solution of 32.4 may
be obtained in the form
y1  y1 
z }| { z }| {
∞ ∞ ∞
C1 ∑ an xn+r +C2  n+r n+r 
 
ln (x) ∑ an x + ∑ Cn x 
n=0 n=0 n=0

where we may take a0 = 1. If r1 − r2 = m, a positive integer, then the general solution to

32.4 may be written as
y1  y1 
z }| !{ z }| !{
∞  ∞ ∞ 
C1 ∑ an xn+r1 +C2 k ln (x) ∑ an xn+r1 + xr2 ∑ Cn xn  ,
 
n=0  n=0 n=0 

where k may or may not equal zero and we may take a0 = 1.

This procedure indicates what one should look for in the various cases. There is more
discussion in [25].
626 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

32.6 The Bessel Equations

The Bessel differential equations are

x2 y′′ + xy′ + x2 − ν 2 y = 0

Obviously this has a regular singular point at 0 and the indicial equation is

r (r − 1) + r − ν 2 = r2 − ν 2 = 0

Thus the two indices of singularity are ±ν. There are various cases according to whether
ν is 0, not an integer, or an integer.

32.6.1 The Case where ν = 0

First consider the case where ν = 0. In this case, there exists a solution of the form
n
∑∞n=0 an x and it is required to find the constants an . Plugging into the equation one gets
∞ ∞ ∞
x2 ∑ an n (n − 1) xn−2 + x ∑ an nxn−1 + ∑ an xn+2 = 0
n=0 n=0 n=0

Then change the variable of summation in the last sum. This yields
∞ ∞ ∞
∑ an n (n − 1) xn + ∑ an nxn + ∑ an−2 xn = 0
n=0 n=0 n=2

It follows that there is no restriction on a0 , a1 but for n ≥ 2,

an (n (n − 1) + n) + an−2 = an n2 + an−2 = 0
a
Thus an = − n−2
n2
.
Taking a0 = 1, a1 = 0, it follows that all odd terms equal 0 and
−1 1 1 1 1 1
a2 = , a4 = 2 2 , a6 = − 2 2 2 , · · ·
4 2 4 2 4 6
The pattern is now fairly clear:
1
a2n = (−1)n
2n (n!)2

Then this solution is

∞
1
J0 (x) = ∑ (−1)k 2k (k!)2 x2k (32.20)
k=0
Then by Theorem 32.5.2, the general solution is of the form
!
∞
C1 J0 (x) +C2 ln (x) J0 (x) + ∑ Cn xn
n=0

for suitable choice of the Cn . Thus one is bounded near x = 0 and the other is unbounded
near x = 0. In fact, it is customary to let the second solution be a complicated linear
32.6. THE BESSEL EQUATIONS 627

combination of these two solutions. When this is done, the function which results is known
as Y0 (x). Then J0 (x) is the Bessel function of the first kind and the Y0 (x) is called the
Bessel function of the second kind. Here are graphs of these functions.

Y0 (x)

J0 (x)

32.6.2 The Case of ν Not an Integer

Next consider the case where ν is not an integer. This time, the series is of the form
∞
∑ an xn+ν
n=0

Substituting into the equation,

∞ ∞
x2 ∑ an (n + ν) (n + ν − 1) xn+ν−2 + x ∑ an (n + ν) xn+ν−1
n=0 n=0

∞ ∞
+ ∑ an xn+ν+2 − ∑ ν 2 an xn+ν = 0
n=0 n=0

Thus a little simplification yields

∞ ∞ ∞
∑ an (n + ν)2 xn+ν + ∑ an−2 xn+ν − ∑ ν 2 an xn+ν = 0
n=0 n=2 n=0

Then we need to have a1 = 0 but let a0 = 1. Then for n ≥ 2,

  −an−2 −an−2
an (n + ν)2 − ν 2 = −an−2 so an = 2
= (32.21)
(n + ν) − ν 2 n (n + 2ν)

Thus all the odd terms are 0 and the first several terms are as follows.
1 1 1
a0 = 1, a2 = − , a4 = ,···
2 (2 + 2ν) 2 (2 + 2ν) 4 (4 + 2ν)

The pattern seems clear at this point. Thus

(−1)n 1
a2n =
(2 · 4 · · · · · 2n) (2 + 2ν) (4 + 2ν) · · · (2n + 2ν)
628 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

(−1)n 1
=
22n n! (1 + ν) (2 + ν) · · · (n + ν)
That product (1 + ν) (2 + ν) · · · (n + ν) in the bottom will be denoted as (n + ν)n . Then
this reduces to
(−1)n
a2n = 2n
2 n! (n + ν)n
Thus a solution corresponding to ν is
∞
(−1)k
xν + ∑ 2k k! (k + ν)
x2k+ν
k=1 2 k

1
Then this is massaged a little more. It is multiplied by the constant Γ(ν+1)2ν . Recall that
the Gamma function satisfies
Γ (α) α = Γ (α + 1)
Applying this rule repeatedly in the above sum yields
∞
(−1)k  x 2k+ν
Jν (x) = ∑ k!Γ (k + ν + 1)
k=0 2

You can verify directly that

∞
(−1)k  x 2k−ν
J−ν (x) = ∑ k!Γ (k − ν + 1)
k=0 2

is also a solution to the Bessel equation. The definition of Γ (k − ν + 1) when the argu-
ment is negative is defined in terms of the property of the gamma function which was
responsible for making Jν (x) be a solution, Γ (α + 1) = αΓ (α) . Thus, for example, if
−ν + 1 < 0, Γ (−ν + 1) (−ν + 1) · · · (−ν + m) = Γ (−ν + 1 + m) where m is large enough
that −ν + 1 + m > 0. Since ν is not an integer, −ν + k is never zero so there is never a
difficulty in encountering something which does not make sense.
The Bessel function of the first kind Jν converges to 0 as x → 0+ while J−ν is un-
bounded as x → 0 + . Consequently, their ratio cannot be a constant and so the general
solution is obtained as linear combinations of these two solutions. Of course everything
changes if ν is a positive integer. In this case, the second solution fails to even make sense
because you could have k − ν = 0 and Γ (0) is not even defined.
In fact, what people tabulate is a linear combination of these two solutions
cos (πν) Jν (x) − J−ν (x)
Yν (x) ≡
sin (πν)
It is called the Weber function or the Neumann function. The main thing to notice here is
that it is unbounded as x → 0.

32.6.3 Case Where ν is an Integer

Let ν = m a positive integer. Then you still get one solution which is of the form
∞
(−1)k
xm + ∑ 2k
x2k+m
k=1 2 k! (k + m)k
32.6. THE BESSEL EQUATIONS 629

and multiplying by a constant as above, you can obtain

∞
(−1)k  x 2k+m ∞
(−1)k  x 2k+m
Jm (x) = ∑ k!Γ (k + m + 1) = ∑ k! (k + m)!
k=0 2 k=0 2

as one solution. In fact, you could consider simply replacing m with −m in the above, but
this will not work out. It won’t work out roughly because Γ (k − m + 1) = ±∞ for k +1 ≤ m.
Thus the sum will reduce to
∞
(−1)k  x 2k−m
∑
k=m k!Γ (k − m + 1) 2

Changing the variable of summation to k = j + m, this becomes

∞
(−1) j  x 2 j+m ∞
(−1) j  x 2 j+m
(−1)m ∑ = (−1)m ∑
j=0 ( j − m)!Γ ( j + 1) 2 j=0 Γ ( j − m + 1) j! 2
= (−1)m Jm (x)

so the new solution obtained by replacing m with −m is nothing more than (−1)m times
the old solution. It follows that there is no way to obtain the general solution as a linear
combination of these two. The second solution must involve a logarithmic term and will
therefore, be unbounded near 0. However, it is convenient to define

J−m (x) ≡ (−1)m Jm (x)

The way this is dealt with is to define the second solution as

Ym (x) ≡ lim Yν (x)

ν→m

because the limit does exist for all x > 0.

One other important consideration is easy to get which is that the solutions to Bessel’s
equation must oscillate about 0 like sines and cosines.

Proposition 32.6.1 Let y be a solution of the Bessel equation

x2 y′′ + xy′ + x2 − ν 2 y = 0

Then y has infinitely many zeros.

Proof: Change the independent variable to s where x = es . Thus, letting y (s) = y (x) ,
it will suffice to show that s → y (s) has infinitely many zeros. Doing the transformation
yields the following differential equation for s → y (s)

y′′ (s) + e2s − ν 2 y (s) = 0

Obviously for all s large enough, e2s − ν 2 > 1. Consider now the equation

z′′ + z = 0

The idea is to show that if a, b are successive zeros of z for a large enough that for s > a,
e2s − ν 2 > 1 it follows that y must have a zero in [a, b]. Since z has infinitely many zeros, it
follows that so does y.
630 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

Without loss of generality, assume z is positive on (a, b). If it isn’t, multiply by −1 to

make this happen. The solution z is a linear combination of sines and cosines. It can be
written in the form
z = A cos (s − φ )
and so z′ (a) > 0 and z′ (b) < 0.
If y has no zeros on [a, b] , then again, without loss of generality, let y be positive on
[a, b].
z′′ y − y′′ z + 1 − e2s − ν 2 yz = 0

Thus on the open interval (a, b) ,

W (y, z)′ = e2s − ν 2 − 1 yz > 0

where W (y, z) is the Wronskian. It follows from the mean value theorem that W (y, z) (a) <
W (y, z) (b). Then
y (a) 0 y (b) 0
′ ′
< ′ ′
y (a) z (a) y (b) z (b)
positive = y (a) z′ (a) < y (b) z′ (b) = negative,
a contradiction. ■
For the purposes of this book, this will suffice. The main message is that there are two
independent solutions, one bounded near 0 and the other unbounded as described above.
Both oscillate about 0 and have infinitely many zeros. In many applications, the unbounded
one is of no interest based on physical considerations.

32.7 Other Properties of Bessel Functions

Recall that for m a nonnegative integer,
∞
(−1)k  x 2k+m
Jm (x) = ∑ k! (k + m)!
k=0 2

and that if −m is negative,

J−m (x) = (−1)m Jm (x)
This Jm (x) was the bounded solution for the Bessel equation. Note that an infinite sum of
these functions is absolutely convergent. Indeed,

1 x m ∞ 1  x 2 k
∞  
1  x 2k+m
|Jm (x)| ≤ ∑ ≤ ∑ k! 2
k=0 k!m! 2 m! 2 k=0
1 x m
exp x2 /4


= (32.22)
m! 2
Therefore, it is permissible to sum the various series which result in what follows in any
order desired.
Now for t ̸= 0,
∞ ∞
xt 1  x l l − x 1  x k
e2 = ∑ l! t , e 2t = ∑ (−1)k t −k
l=0 2 k=0 k! 2
32.7. OTHER PROPERTIES OF BESSEL FUNCTIONS 631

We multiply these two series. This will involve many terms which can be added in any
order thanks to absolute convergence. To get t m for m ≥ 0, you need to multiply terms
l = m + k times the term for t −k in the second sum. Thus you get for this term
∞  x m+k 1  x k
1
tm ∑ (−1)k
k=0 (m + k)! 2 k! 2
∞
1  x 2k+m

= t m ∑ (−1)k = t m Jm (x)
k=0 k! (m + k)! 2

This gives the terms t m for m ≥ 0.

What of the terms involving m < 0? To get these terms, you need to have l − k = m so
you need k = l − m. Thus the sum which results for these terms is
∞
1  x l 1  x l−m ∞
(−1)l−m  x 2l−m
tm ∑ (−1)l−m = tm ∑
l=0 l! 2 (l − m)! 2 l=0 l! (l − m)! 2

∞
(−1)l  x 2l−m
= (−1)m t m ∑ = (−1)m t m J−m (x)
l=0 l! (l − m)! 2
Therefore, xt x
e 2 e− 2t = e(x/2)(t−1/t)
must equal the sum of t m terms for m ≥ 0 and the sum of t m terms for m < 0. It follows that
∞ ∞
e(x/2)(t−1/t) = J0 (x) + ∑ t m Jm (x) + ∑ (−1)m t −m Jm (x)
m=1 m=1
∞
t m + (−1)m t −m


= J0 (x) + ∑ Jm (x)
m=1

Now recall that J−m (x) = (−1)m Jm (x) and so

∞ ∞ ∞
e(x/2)(t−1/t) = J0 (x) + ∑ Jm (x)t m + ∑ t −m J−m (x) = ∑ t m Jm (x)
m=1 m=1 m=−∞

That is, Jm (x) is just the mth coefficient of the series for e(x/2)(t−1/t) . This has proved
the following interesting result on the generating function for Bessel equations.

Theorem 32.7.1 For m an integer and Jm (x) = (−1)m J−m (x) , we have the following gen-
erating function for these Bessel functions.
∞
e(x/2)(t−1/t) = ∑ t m Jm (x) (32.23)
m=−∞

In addition to this, there is an addition formula

∞
Jm (x + y) = ∑ Jm−k (x) Jk (y) (32.24)
k=−∞
632 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

Proof: It remains to obtain the above addition formula. This is remarkably easy to
obtain.
∞
e((x+y)/2)(t−1/t) = ∑ t m Jm (x + y)
m=−∞
((x+y)/2)(t−1/t) (x/2)(t−1/t) (y/2)(t−1/t)
e =e e
∞ ∞
= ∑ t l Jl (x) ∑ t k Jk (y)
l=−∞ k=−∞
and in this product, the tm term is the sum of products for which l + k = m. That is,
∞
Jm (x + y) = ∑ Jk (y) Jm−k (x)
k=−∞

This shows the addition formula. ■

Of course t was completely arbitrary as long as it is not zero. Thus let it equal eiθ in
32.23. Then from Euler’s identity, eiθ = (cos (θ ) + i sin (θ )) ,
∞  m
e(x/2)(2i sin θ ) = ∑ eiθ Jm (x)
m=−∞

Then using Euler’s identity again,

∞
cos (x sin (θ )) + i sin (x sin (θ )) = ∑ (cos (mθ ) + i sin (mθ )) Jm (x)
m=−∞

Equating real and imaginary parts,

∞
cos (x sin (θ )) = ∑ cos (mθ ) Jm (x)
m=−∞
∞
sin (x sin (θ )) = ∑ sin (mθ ) Jm (x)
m=−∞

Now recall from trig. identities,

cos (a) cos (b) + sin (a) sin (b) = cos (a − b)

multiply the top by cos (nθ ) and the bottom by sin (nθ ) and add. Thus
∞
cos (nθ − x sin (θ )) = ∑ cos (nθ − mθ ) Jm (x)
m=−∞

Because of the uniform convergence of the partial sums of Rthe above series which follows
from computations like those in 32.22, one can interchange 0π with the infinite summation.
This yields Z π
cos (nθ − x sin (θ )) dθ = πJn (x)
0
because, unless n = m, 0π cos (nθ − mθ ) dθ = 0. Therefore, this yields the very important
R

integral identity for Jn (x) ,

1
Z π
Jn (x) = cos (nθ − x sin (θ )) dθ (32.25)
π 0
32.8. EXERCISES 633

The interchange of the integral with the summation follows from noting that the sums
of the form ∑km=−k cos (nθ − mθ ) Jm (x) converge uniformly on [0, π] to the infinite sum
thanks to the 1/m! in the estimates of 32.22. Thus, from the fact that the integral is linear,
Z π ∞ Z π k
∑ cos (nθ − mθ ) Jm (x) dθ = lim ∑ cos (nθ − mθ ) Jm (x) dθ
0 m=−∞ 0 k→∞ m=−k

k Z π ∞ Z π
= lim ∑ cos (nθ − mθ ) Jm (x) dθ = ∑ cos (nθ − mθ ) Jm (x) dθ
k→∞ m=−k 0 m=−∞ 0

Theorem 32.7.2 Let n be a positive integer. Then

1
Z π
Jn (x) = cos (nθ − x sin (θ )) dθ
π 0

How do you compute Jn (x)? You can’t get it the usual way very conveniently because
the leading term vanishes at 0. This integral will give an easy way to do it. For example,
1
Z π
J4 (6) = cos (4θ − 6 sin (θ )) dθ = 0.357 64
π 0

I just did the integral numerically in Scientific Notebook and got the answer easily. One
can also produceRa graph of x → J4 (x) very easily in this software by graphing the function
of x given by π1 0π cos (4θ − x sin (θ )) dθ . To do this, you simply type the expression in
math mode and then select plot 2d. It has to work at it a little but will produce the graph. It
knows that the variable is x and acts accordingly. In the exercises is a problem on how to
do this in MATLAB. It is more elaborate.
There are whole books written on Bessel functions, [17].

32.8 Exercises
1. The Hermite equation is
y′′ − xy′ + ny = 0
Verify that if n = 0 or a positive integer, then this equation always has a polynomial
solution. These are called Hermite polynomials. Hint: This is easier than the case
of a regular singular point. Just look for a solution of the form y = ∑∞ n
n=0 an x and
choose the an in such a way that the series satisfies the equation using the fact that
you can differentiate a power series term by term. In this case, there should be two
solutions.
2. If you have two polynomial solutions to the Hermite equation above, pm (x) corre-
sponding to m in the equation and pn (x) corresponding to n in the equation, n ̸= m,
show that Z ∞
2
e−x pm (x) pn (x) dx = 0
−∞

3. The equation
1 − x2 y′′ − 2xy′ + n (n + 1) y = 0

is Legendre’s equation. Note that 0 is an ordinary point for this equation. Show that
for n a non-negative integer, this equation has polynomial solutions. Also explain
why this equation has a regular singular point at 1, −1.
634 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

4. In the above problem, suppose pk (x) and pl (x) are solutions, to the equations corre-
sponding to n = k, l respectively. Show that
Z 1
pk (x) pl (x) dx = 0
−1

Thus this gives an example of a collection of orthogonal polynomials.

5. The Legendre polynomials are given in the above problem but one multiplies by a
constant so that the result satisfies pn (1) = 1. The purpose

n of this problem is to find
the constant. Hint: Use the Leibniz formula on x2 − 1 = (x − 1)n (x + 1)n .

6. The equation 1 − x2 y′′ − xy′ + n2 y = 0 is called the Chebychev equation. Find

solutions to this equation. That is, specify a recurrence relation and two solutions.
Explain why there exist polynomial solutions to this equation. Hint: You just look
for power series solutions.
7. The equation 1 − x2 y′′ − 3xy′ + n (n + 2) y = 0 is also called the Chebychev equa-

tion. Find solutions to this equation. That is, specify a recurrence relation and two
solutions. Explain why there exist polynomial solutions to this equation. Hint: You
just look for power series solutions.
8. Specify two solutions to the following differential equation by determining a recur-
rence relation and then describing how to obtain two solutions. Hint: You just look
for power series solutions.

(a) y′′ x2 + 1 + 5xy′ + 2y = 0. (f) y′′ − 2x2 y′ − xy = 0.

(b) y′′ x2 + 1 + xy′ + 3y = 0.

(g) y′′ + x2 y′ + 2xy = 0.
(c) y′′ x2 + 1 + 7xy′ + 4y = 0.


(h) y′′ − 3x2 y′ − xy = 0.
(d) y′′ 1 − 3x2 + 6xy′ + 4y = 0.

(e) y′′ − 5x2 y′ − 4xy = 0. (i) y′′ + 2x2 y′ − 4xy = 0.

9. Find the solution to the initial value problem y′′ + sin (x) y′ + cos (3x) y = 0 along
with the initial conditions y (0) = 1, y′ (0) = −1. You just need to find the first terms
of the power series solution up to x4 .
10. Find the solution to the initial value problem y′′ + tan (2x) y′ + cos (3x) y = 0 along
with the initial conditions y (0) = −1, y′ (0) = 2. You just need to find the first terms
of the power series solution up to x4 .

11. Find the solution to the initial value problem y′′ + tan (5x) y′ + sec (3x) y = 0 along
with the initial conditions y (0) = −2, y′ (0) = 3. You just need to find the first terms
of the power series solution up to x4 .
12. Find the general solution to the following Euler equations.

(a) y′′ x2 − 3y′ x + 3y = 0. (d) y′′ x2 + 6y′ x + 6y = 0.

(b) y′′ x2 + 4y′ x − 4y = 0. (e) y′′ x2 + 4y′ x − 4y = 0.
(c) y′′ x2 + 2y′ x − 6y = 0. (f) y′′ x2 − 3y′ x + 4y = 0.
32.8. EXERCISES 635

(g) y′′ x2 + 5y′ x + 4y = 0. (l) y′′ x2 + 7y′ x + 10y = 0.

(h) y′′ x2 − 5y′ x + 9y = 0. (m) y′′ x2 + 7y′ x + 10y = 0.
(i) y′′ x2 − 3y′ x + 4y = 0. (n) y′′ x2 + 9y′ x + 32y = 0.
(j) y′′ x2 − 3y′ x + 4y = 0. (o) y′′ x2 + 11y′ x + 26y = 0.
(k) y′′ x2 − y′ x + y = 0. (p) y′′ x2 + 11y′ x + 34y = 0.

13. The hypergeometric equation is

x (1 − x) y′′ + (γ − (1 + α + β ) x) y′ − αβ y = 0

Show it has a regular singular point at 0 and that the roots of the indicial equation are
0 and 1 − γ.
14. In the above example, change the independent variable as follows: t = 1/x. Deter-
mine the equation which results in terms of t and show that the resulting equation
has a regular singular point at 0 and that the roots of the indicial equation are α, β .
Hint: You need to show that y′′ (x) = y′′ (t)t 4 + 2t 3 y′ (t) , y′ (x) = −t 2 y′ (t). When
you let t = 0, you are looking at the “point at infinity”. Thus you are showing that
the “point at infinity” is a regular singular point.
15. Consider the Bessel equation in which ν = 1/2. In this case, the roots of the indicial
equation differ by an integer. Nevertheless, there are two solutions, neither of which
involves a logarithm. Verify that for ν not an integer,
∞
(−1)k
x−ν + ∑ 2k k! (k − ν)
x2k−ν
k=1 2 k

does indeed yield a solution to the Bessel equation.

16. Show that for ν = 1/2, one solution to the Bessel equation is x−1/2 sin (x). What is
the other solution? Verify your answer. Show that one of these solutions is bounded
and in fact converges to 0 as x → 0+ while the other is unbounded as x → 0+.
17. Explain why in every case, if you have a general solution to the Bessel equation, one
of the solutions will be unbounded as x → 0 and the other must converge to 0 as
x → 0+.
18. The Laguerre differential equation is

xy′′ + (1 − x) y′ + my = 0

Show that when m is a nonnegative integer, there always exists a polynomial which is
a solution to this differential equation. Letting pk (x) , pl (x) be polynomial solutions
corresponding to m = k, l respectively, show that
Z ∞
e−x pk (x) pl (x) dx = 0, k ̸= l
0

19. Prove, Leibniz rule. !

n
(n) n
( f g) = ∑ f (k) g(n−k)
k=0 k
636 CHAPTER 32. SOLUTIONS NEAR A REGULAR SINGULAR POINT

20. Suppose you have any linear second order differential equation Ly = 0 in which
there is a general solution C1 y +C2 z such that W (y, x) ̸= 0 for x ∈ [a, b]. Show that if
y (x) = 0, then y′ (x) ̸= 0. Why does this show that given a zero of a nonzero solution
to the Bessel equation, or any other second order linear differential equation, there is
a next zero?
21. Consider the equation x3 y′′ + 2xy′ + y = 0. Explain why it does not have a regular
singular point at 0. Show that the only possible nonzero power series solution to this
has radius of convergence equal to 0. In fact there really isn’t any such series solution
to this problem.

22. Consider the Bessel function Jm (x) for m a positive integer. Recall the summation
formula.
∞
Jm (x + y) = ∑ Jm−k (x) Jk (y) ,
k=−∞
∞
(−1)k  x 2k+m
Jm (x) = ∑ k! (k + m)! 2 , Jm (x) = (−1)m J−m (x)
k=0

Explain why Jm is even if m is even and Jm is odd if m is odd. Next let m = 0 and see
what comes out of the summation formula. Then let y = −x to obtain an inequality
which shows that all the J√n are bounded. Show in particular that each Jn (x) has the
property that |Jn (x)| ≤ 1/ 2 if n > 0.

23. Use the integral formula for the Bessel function to graph J4 (x) for x ∈ [0, 20] . Here
is the syntax which will work for this. You put in the new lines.
hold on
for k=1:201 f=@(t,k)cos(4*t-((k-1)*.1)*sin(t));
y(k)=piˆ(-1)*integral(@(t)f(t,k),0,pi);
x(k)=(k-1)*.1; plot(x,y,’linewidth’,2) end.
Chapter 33

Boundary Value Problems,

Fourier Series

33.1 Boundary Value Problems

The initial value problem can always be formulated as
y ′ = Ay + f , y (a) = y 0 .
These are very nice problems because they always have a unique solution. A boundary
value problem is different. They don’t always have solutions.
Definition 33.1.1 A two-point boundary value problem is to find a solution y to a differen-
tial equation
y′′ + p (x) y′ + q (x) y = g (x) , x ∈ [a, b]
which also satisfies boundary conditions which are given at the two end points.
Examples of boundary values would be to give the value of y at the end points or the
value of y′ at the end points or some combination of y and y′ at the end points.
Example 33.1.2 Find the solutions to the equation y′′ + y = sin x, and boundary conditions
y (0) = 0, y (π) = 0
The general solution to the differential equation is easily seen to be A cos (x)+B sin (x)−
1
2 x cos (x) .You have find A, B such that the boundary conditions are satisfied. Substituting
t = 0 yields A = 0. Then you also need B sin (π) − 12 π cos (π) = 0 which is impossible.
Therefore, there is no solution to this boundary value problem.
This is a very significant issue because there are numerical methods for solving bound-
ary value problems. These methods will give you an answer even if there isn’t one, but if
it isn’t there, you won’t find it. Of course, just because you can’t find it does not neces-
sarily mean it isn’t there. This is the interesting thing about math. In the absence of good
existence and uniqueness theorems, you sometimes don’t know what you are getting.
′′
Example 33.1.3
π

Find the solutions to the equation y + y = sin x, and boundary conditions
y (0) = 0, y 2 = 0.

637
638 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

It is the same equation, but the end points are different. As in the above example, if
it has a solution, then it is of the form B sin x − 21 x cos (x) Now let x = π/2 and you find
B − 41 π0 = 0. Thus a solution to this boundary value problem is y = − 12 x cos (x)
In this example, there was exactly one solution. Next consider

Example 33.1.4 Find the solutions to the equation y′′ + y = sin x, and boundary conditions
π  π
y (0) = 0, y′ =
2 4
The general solution to the differential equation is easily seen to be A cos (x)+B sin (x)−
1
2 x cos (x) .You have find A, B such that the boundary conditions are satisfied. Substituting
t = 0 yields A = 0. Thus if there is a solution it is of the form y = B sin (x) − 12 x cos (x) .
Then y′ (x) = B cos x− 12 cos x+ 12 x sin x. Then you also need π4 = y′ π2 = B cos π2 +0+ π4

which happens for any value of B. Therefore, for any B, y = B sin (x) − 21 x cos (x) is a solu-
tion to this two point boundary value problem.
This is an example of a boundary value problem which has infinitely many solutions.
Notice how all three examples involved the same differential equation, just different bound-
ary conditions.
It turns out that for two point boundary value problems it is always this way. Either
there are no solutions, exactly one or there are infinitely many. This may look familiar.
Recall that it was this way for systems of linear equations. There are profound reasons why
this similarity takes place but they are not for a book like this.

33.2 Eigenvalue Problems

I suppose these are best discussed through the example which will be featured most promi-
nently.

Example 33.2.1 Find the values of λ such that there exist nonzero solutions to the
boundary value problem

y′′ + λ y = 0
y (0) = y (L) = 0

Along with any pair of boundary conditions which satisfy the conditions

y (0) y′ (0) = 0, y (L) y′ (L) = 0

Multiply by y and integrate from 0 to L.

Z L Z L
y′′ ydx + λ y2 dx = 0 (33.1)
0 0

Integrate by parts.
Z L Z L
′ ′ 2
y|L0 − y2 dx = 0


y y dx + λ
0 0
Consider now the boundary term. It equals 0 by assumption. Therefore,
Z L
2 Z L
− y′ dx + λ y2 dx = 0
0 0
33.2. EIGENVALUE PROBLEMS 639

If λ < 0, this equation could not be true and have y ̸= 0 because it would imply 0L y2 dx = 0
R

so y = 0. Therefore, for any such example, λ ≥ 0.

Case 1: Now consider some cases each of which have the property that yy′ equals 0 at
the end points of the interval [0, L]. First suppose y = 0 at the ends of the interval. To save
notation, write λ = µ 2 . Then you want

y′′ + µ 2 y = 0
y (0) = y (L) = 0

The solution to the differential equation is

C1 sin µx +C2 cos µx

Insert the boundary conditions. This yields

C2 = 0, C1 sin (µL) = 0

Therefore, for some nonnegative integer n, you must have µL = nπ. You can’t have n = 0
since then y = 0 and this is not allowed. Therefore, n is a positive integer and the eigenval-
ues are
n2 π 2
λ = 2 , n = 1, 2, · · ·
L
The corresponding eigenfunctions are
 nπ 
sin x , n = 1, 2, · · ·
L
Case 2: Next consider the case where y′ = 0 at the ends. Thus you want nonzero y and
λ such that

y′′ + µ 2 y = 0
y (0) = y′ (L) = 0
′

The solution to the differential equation is

y = C1 sin µx +C2 cos µx

Then
y′ = C1 µ cos µx −C2 µ sin µx
Insert the boundary conditions. At 0 this requires that

C1 µ = 0

At the right end point this requires

C2 µ sin (µL) = 0

One case is for µ = 0. This would result in an eigenfunction

y=1
640 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

which is a nonzero function. Of course any nonzero multiple of this is also an eigenfunc-
tion. If µ is not zero, then you need

µL = nπ, n = 1, 2, · · ·

so
n2 π 2
λ= , n = 0, 1, 2, · · ·
L2
The eigenfunctions in this case are
 nπ 
1, cos x , n = 1, 2, · · ·
L
Case 3: Next consider the case where y (0) = 0 and y′ (L) = [Link] you want nonzero
y and λ such that

y′′ + µ 2 y = 0
y (0) = y′ (L) = 0

In this case, you would have

y = C1 sin µx +C2 cos µx

and on inserting the left boundary condition, this requires that C2 = 0. Now consider the
right boundary condition. You can’t have µ = 0 in this case, because if you did, you would
have y = 0 which is not allowed. Hence you have

y′ (L) = C1 µ cos (µL) = 0

since µ ̸= 0, you must have

µL = (2n − 1) π for n = 1, 2, · · ·

Therefore, in this case the eigenvalues are

(2n − 1)2 π 2
λ= , n = 1, 2, · · ·
L2
and the eigenfunctions are
 
(2n − 1) π
sin x , n = 1, 2, · · ·
L

33.3 Fourier Series

A Fourier series is a series which is intended to somehow approximate a given periodic
function by an infinite sum of the form
∞   ∞  
kπ kπ
a0 + ∑ ak cos x + ∑ bk sin x
k=1 L k=1 L

First of all, what is a periodic function?

33.3. FOURIER SERIES 641

Definition 33.3.1 A function f : R → R is called periodic of period T if for all x ∈ R,

f (x + T ) = f (x) .

An example of a periodic function having period 2L is x → sin kπ kπ

L x and x → cos L x .
If you want to approximate a function with these periodic functions, then it is necessary
that it be periodic of period 2L. Otherwise it would not be reasonable to expect to be able
to approximate the function in any useful way with these periodic functions.
Before doing anything else, here are some important trig. identities.
1
sin a cos b = (sin (a + b) + sin (a − b)) (33.2)
2
1
cos a cos b = (cos (a − b) + cos (a + b)) (33.3)
2
1
sin a sin b = (cos (a − b) − cos (a + b)) (33.4)
2
These follow right away from the standard trig. identities for the sum of two angles. Here
is a lemma which gives an orthogonality condition.

Lemma 33.3.2 The following formulas hold. For m, n positive integers,

Z L
(
1  mπ  1  nπ  0 if m ̸= n
√ sin x √ sin x dx =
−L L L L L 1 if m = n

Z L
(
1  mπ  1  nπ  0 if m ̸= n
√ cos x √ cos x dx =
−L L L L L 1 if m = n
Z L  mπ   nπ 
sin x cos x dx = 0
−L L L
Proof: Consider the first of these formulas. From one of the above trig. identities,
Z L  mπ   nπ 
sin x sin x dx =
−L L L
Z L
1  mπ nπ    mπ nπ  
cos − x − cos + x dx
2 −L L L L L
If m ̸= n, this clearly integrates to 0. If m = n, you have

1 L
Z   
2n
1 − cos x dx = L
2 −L L

Thus Z L
1  nπ  1  nπ 
√ sin x √ sin x dx = 1
−L L L L L
The second formula works out the same way. Consider the third.
Z L  mπ   nπ 
sin x cos x dx =
−L L L
642 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

1 L   mπ nπ    mπ nπ  
Z
sin + x + sin − x dx
2 −L L L L L
It is easy to see that this integral is always 0 regardless the choice of m, n. ■
Now suppose you succeed in approximating f with a Fourier series in some meaningful
way.   ∞  
∞
1 1 kπ 1 kπ
f (x) ≈ a0 √ + ∑ ak √ cos x + ∑ bk √ sin x (33.5)
2L k=1 L L k=1 L L
What should be the formula for ak and bk ? Multiply both sides by √1L sin mπ


L x and then
integrate the resulting infinite sum by saying the integral of the sum is the sum of the
integrals. Since the sum involves a limit, this is nothing but a formal and highly speculative
piece of pseudo mathematical nonsense but we will not let a little thing like that get in the
way. Thus
Z L Z L
1  mπ  1  mπ 
f (x) √ sin x dx = a0 √ sin x dx+
−L L L −L 2L L

∞ Z L  
1 kπ 1  mπ 
∑ ak √ cos x √ sin x dx
k=1 −L L L L L
∞ Z L  
1 kπ 1  mπ 
+ ∑ bk √ sin x √ sin x dx
k=1 −L L L L L

All these integrals equal 0 but one and that is the one involving the sine and k = m. This is
by the above lemma. Therefore,
Z L Z L
1  mπ  1  mπ 
f (x) √ sin x dx = bm sin2 x dx = bm
−L L L L −L L
It seems likely therefore, that bm should be defined as
Z L
1  mπ 
bm = f (x) √ sin x dx (33.6)
−L L L

Next do the same thing after multiplying by √1L cos mπ

L x . Another use of the same lemma
implies that the appropriate choice for am is
Z L
1  mπ 
am = f (x) √ cos x dx (33.7)
−L L L
Finally integrate both sides of 33.5. This yields
Z L
1
f (x) √ dx = a0 (33.8)
−L 2L
and so the appropriate description of a0 is given above. Thus the Fourier series is of the
form
Z L ∞ Z L  mπ   1
1 1 1  mπ 
f (y) √ dy √ + ∑ f (y) √ cos y dy √ cos x
−L 2L 2L m=1 −L L L L L
∞ Z L  mπ   1
1  mπ 
+∑ f (y) √ sin y dy √ sin x
m=1 −L L L L L
33.4. MEAN SQUARE APPROXIMATION 643
√
Combining the L terms, this yields
Z L ∞  ZL  mπ  
1 1  mπ 
= f (y) dy + ∑ L −L f (y) cos y dy cos x
2L −L m=1 L L
 ZL
∞  mπ    mπ 
1
+∑ f (y) sin y dy sin x
m=1 L −L L L

This is so far completely speculative, but this was they often did things back in the
time when Fourier came up with the idea back in the early 1800s. Here is the definition
of the Fourier series in which we combine the various constant terms to make it easier to
remember.

Definition 33.3.3 Let f be a function defined on R which is 2L periodic and Riemann

integrable on every closed interval of length 2L. Then the Fourier series is defined as
∞   ∞  
kπ kπ
a0 + ∑ ak cos x + ∑ bk sin x
k=1 L k=1 L

where a0 , am , bm are given as

1 L 1 L  mπ 
Z Z
a0 = f (y) dy, am = f (y) cos y dy
2L −L L −L L
Z L
1  mπ 
bm = f (y) sin y dy
L −L L
We will refer to a0 , an , bn as Fourier coefficients.

33.4 Mean Square Approximation

When you have two functions defined on an interval [a, b], how do you measure the distance
between them? It turns out there are infinitely many ways to do this. One way is to say the
distance between f and g, denoted as ∥ f − g∥ is defined as

∥ f − g∥ = sup {| f (x) − g (x)| ,t ∈ [a, b]}

To say that two functions are close in this sense is to say that for each x you have f (x) close
to g (x). The two functions are said to be uniformly close if they are close in this norm.
This norm is also called the uniform norm.
This is a good way to define distance between functions, but it turns out that a more
useful way in many situations is the following. You define
Z b
1/2
∥ f − g∥ ≡ | f (x) − g (x)|2 dx
a

Then ∥ f − g∥ is called the mean square norm with the above definition. You should verify
that if two functions are close in the uniform norm, then they must be close in the mean
square norm, but not the other way around. Often the mean square norm is denoted as
| f − g|. So why is this a norm and what is meant by a norm? First here is a simple lemma.
644 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

Lemma 33.4.1 Suppose f , g are Riemann integrable functions. Define

Z b
( f , g) ≡ f (x) g (x) dx.
a

Then the following are satisfied.

( f , g) = (g, f )
(f, f) ≥ 0
For a, b real numbers,

(a f + bg, h) = a ( f , h) + b (g, h)
( f , ag + bh) = a ( f , g) + b ( f , h)

The following inequality called the Cauchy-Schwarz inequality holds.

|( f , g)| ≤ | f | |g| ≡ ( f , f )1/2 (g, g)1/2

where | f | denotes the mean square distance defined above.

Proof: All of the above are completely obvious except for the last one. As to that one,
note that from the first obvious properties, for t ∈ R

0 ≤ (t f + g,t f + g) = t 2 ( f , f ) + 2t ( f , g) + (g, g)

If ( f , f ) = 0 there is nothing to prove because you must have ( f , g) = 0 since otherwise,

the above inequality would be violated for suitable choice of t. It follows that the above is
a quadratic polynomial whose graph opens up and which has at most one real zero. Hence
by the quadratic formula,
4 ( f , g)2 − 4 ( f , f ) (g, g) ≤ 0
which reduces to the Cauchy-Schwarz inequality. ■
Now the mean square norm amounts to nothing more than | f | = ( f , f )1/2 .

Proposition 33.4.2 The mean square norm ∥ f ∥ = | f | = ( f , f )1/2 satisfies the following
axioms.

1. ∥ f ∥ ≥ 0
2. If a is a number, ∥a f ∥ = |a| ∥ f ∥
3. ∥ f + g∥ ≤ ∥ f ∥ + ∥g∥

Proof: The only one which is not completely obvious is the last. Then by the definition
of the norm and the properties of (·, ·) ,

∥ f + g∥2 ≡ ( f + g, f + g) = ∥ f ∥2 + ∥g∥2 + 2 ( f , g)

≤ ∥ f ∥2 + ∥g∥2 + 2 |( f , g)|
≤ ∥ f ∥2 + ∥g∥2 + 2 ∥ f ∥ ∥g∥
= (∥g∥ + ∥ f ∥)2
33.4. MEAN SQUARE APPROXIMATION 645

Now taking the square root of both sides yields the desired inequality. ■
The reason this is important is that if you have f close to g and h close to g, then you
have f close to h. Indeed,

∥ f − h∥ ≤ ∥ f − g∥ + ∥g − h∥

and if both of the terms on the right are small, then the term on 
the left is also.
There are 2n + 1 functions, √12L , √1L cos kπ √ sin jπ x for k, j ∈ 1, 2, · · · , n. De-

1
L x , L L
 
2n+1
note these functions as {φ k }k=1 to save on notation. It was shown above that φ k , φ j =
δ jk which is 1 if k = j and 0 if k ̸= j. Then for f a Riemann integrable function on [−L, L] ,
our problem is to choose α k to minimize
2
2n+1
f− ∑ α kφ k = (A + B, A + B)
k=1

where A = f − ∑2n+1 2n+1

k=1 ( f , φ k ) φ k , B = ∑k=1 (( f , φ k ) − α k ) φ k . Thus the above is

|A|2 + 2 (A, B) + |B|2 (*)

Consider the middle term.

!
2n+1    
f− ∑ ( f , φ k) φ k, φ j = f , φ j − ∑ ( f , φ k) φ k, φ j
k=1 k
  
= f,φ j − f,φ j = 0

and so the middle term of ∗ equals 0 because

!
2n+1 2n+1
f− ∑ ( f , φ k) φ k, ∑ ak φ k =0
k=1 k=1

for any choice of ak which includes the case of (A, B). Thus ∗ implies
2 2 2
2n+1 2n+1 2n+1
f− ∑ α kφ k = f− ∑ ( f , φ k) φ k + ∑ (( f , φ k ) − α k ) φ k
k=1 k=1 k=1

From the definition of the norm, the second term is

   
∑ (( f , φ k ) − α k ) f , φ j − α j φ k , φ j = ∑ (( f , φ k ) − α k )2
j,k k

Hence
2 2
2n+1 2n+1 2n+1
f− ∑ α kφ k = f− ∑ ( f , φ k) φ k + ∑ (( f , φ k ) − α k )2 (**)
k=1 k=1 k=1

which shows that the left side
is minimized exactly when α k = ( f , φ k ). It is clear then that
corresponding to √1L cos kπxL , you would have
Z L  
1 kπx
αk = f (x) √ cos dx
−L L L
646 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

and so, the term in the Fourier series which corresponds to this would be
Z L     
1 kπy 1 kπx
f (y) √ cos dy √ cos
−L L L L L
 ZL     
1 kπy kπx
= f (y) cos dy cos
L −L L L

which is exactly what was determined earlier.

In ∗∗, let each α k = 0. Then this equation implies
2n+1
| f |2 ≥ ∑ ( f , φ k )2
k=1

which is Bessel’s inequality. In particular, in the case of most interest here, this inequality
is
2  2
1 n
Z L Z L 
2 1 kπx
|f| ≥ f (x) dx + ∑ f (x) sin dx
2L −L L k=1 −L L
 2
1 n
Z L 
kπx
+ ∑ f (x) cos dx (**)
L k=1 −L L

It follows that the sequence of partial sums in the sum on the right in ∗∗ converges and
so
Z L  
kπ
lim f (x) cos x dx = 0
k→∞ −L L
Z L  
kπ
lim f (x) sin x dx = 0 (33.9)
k→∞ −L L

The two limits in 33.9 are special cases of the Riemann-Lebesgue lemma. These are the
considerations which make it possible to consider the pointwise convergence properties of
Fourier series. In particular, the following lemma is used.

Lemma 33.4.3 Suppose f is a Riemann integrable function defined on [−L, L] . Then

Z L   
1 π
lim f (x) sin k+ x dx = 0
k→∞ −L 2 L

Proof: It equals
Z L  πx   kπ  Z L   
 πx  kπ
lim f (x) cos sin x dx + f (x) sin cos x dx
k→∞ −L 2L L −L 2L L

and each of these converge to 0 thanks to 33.9. ■

Fourier series are really all about mean square convergence. If you are interested in
pointwise approximation with trig. polynomials, there are better ways to do it than with
Fourier series. However, the pointwise convergence is also very interesting and this is
discussed in the next section.
33.5. POINTWISE CONVERGENCE OF FOURIER SERIES 647

33.5 Pointwise Convergence of Fourier Series

For each x the Fourier series yields an infinite series. One wonders whether it converges
to f (x) . It is completely obvious that this is not necessarily the case. This is because the
Fourier series is completely unchanged if f is changed at any finite set of points. Therefore,
to obtain any sort of meaningful convergence, one must assume something about the func-
tion. The following is an elementary theorem which is a special case of a more substantial
real analysis result.

Definition 33.5.1 The one sided limits are

f (x+) ≡ lim f (x + h) , f (x−) ≡ lim f (x − h)

h→0+ h→0+

Theorem 33.5.2 Suppose f is a periodic function of period 2L such that f has only finitely
many jump discontinuities on the interval [−L, L). Suppose there exists a constant K such
that for all x,
| f (x+) − f (x + y)| < Ky
for all sufficiently small positive y. Also

| f (x−) − f (x − y)| < Ky

for all y sufficiently small. Then

∞   ∞  
f (x+) + f (x−) kπ kπ
= a0 + ∑ ak cos x + ∑ bk sin x
2 k=1 L k=1 L

In words, this says that the Fourier series converges to the midpoint of the jump. A
picture which represents a part of the graph of f is as follows.

You note that the dot is at the midpoint of the jump. The condition in the theorem is
there to rule out excessive steepness of the graph of the function. In fact, one can do a lot
better than what it says in this theorem. You should see [3] for two more general treatments
of this theorem.
One way to satisfy the condition on not having excessive steepness is to have f be
piecewise continuous such that if a, b are successive discontinuities, then redefining f on
[a, b] to equal f (a+) at the left and f (b−) at the right, the new function has a continuous
derivative on [a, b].
Theorem 33.5.2 is the convergence theorem. I am going to give a discussion of this
convergence theorem. If you are not interested in understanding why it works, ignore the
proof. It is included in case someone would be interested. This important theorem or one
like it was first proved in 1829 by Dirichlet.
648 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

33.5.1 Explanation of Pointwise Convergence Theorem

Proof of the convergence theorem: The convergence of sums has to do with the limit of
the sequence of partial sums. Let
n   n  
kπ kπ
Sn f (x) = a0 + ∑ ak cos x + ∑ bk sin x
k=1 L k=1 L

From the definition, this equals

Z L n Z L    
1 f (y) 1 kπ kπ
dy + ∑ f (y) cos y dy cos x
L −L 2 k=1 L −L L L
n Z L    
1 kπ kπ
+∑ f (y) sin y dy sin x
k=1 L −L L L
This simplifies to
Z L
1 f (y)
dy+
2 L −L
n Z L        
1 kπ kπ kπ kπ
∑L f (y) cos y cos x + f (y) sin y sin x dy
k=1 −L L L L L
which equals
Z L n Z L  
1 f (y) 1 kπ
dy + ∑ f (y) cos (x − y) dy
L −L 2 k=1 L −L L
Simplifying this a little more yields
Z L n  !
1 1 kπ
+ ∑ cos (x − y) f (y) dy
−L L 2 k=1 L
Z L
≡ Dn (x − y) f (y) dy
−L

Here Dn (t) is called the Dirichlet kernel. In order to consider the convergence of the partial
sums, it is necessary to study the properties of the Dirichlet kernel.

Lemma 33.5.3 The Dirichlet kernel is periodic of period 2L.

Z L Z L
Dn (t) dt = 2 Dn (t) dt = 1.
−L 0

There is also a formula for this kernel,

sin n + 21 πL t



Dn (t) = π


2L sin 2L t

Proof: As indicated above,

n  !
1 1 kπ
Dn (t) = + ∑ cos t
L 2 k=1 L
33.5. POINTWISE CONVERGENCE OF FOURIER SERIES 649

and so, it is obvious that Z L

Dn (t) dt = 1. (33.10)
−L
From the above formula, it follows that Dn (t) = Dn (−t) and Dn (x + 2L) = Dn (x). Since
Dn (t) = Dn (−t) ,
Z L Z L
Dn (t) dt = 2 Dn (t) dt
−L 0
It remains to find a formula. Use 33.2
n  !
π  1 1 π  π  kπ
sin t Dn (t) = sin t + ∑ sin t cos t
2L L 2 2L k=1 2L L

π  1 n      !
1 1 kπ π kπ π
= sin t + ∑ sin + t − sin − t
L 2 2L 2 k=1 L 2L L 2L

"  #
n    n 
1 π  1 π 1 π
= sin t + ∑ sin k+ t − ∑ sin k− t
2L 2L k=1 2 L k=1 2 L
"  #
n    n−1 
1 π  1 π 1 π
= sin t + ∑ sin k+ t − ∑ sin k+ t
2L 2L k=1 2 L k=0 2 L
  
1 1 π
= sin n+ t
2L 2 L
Thus the desired formula is
sin n + 21 πL t



Dn (t) = π

■
2L sin 2L t

Here is a graph of the first seven of these Dirichlet kernels, n ≥ 1 for L = π.

Next, it follows from the above that

2
Z L
Sn f (x) = Dn (x − y) f (y) dy.
1 −L

Change the variables. Let u = x − y. Then this re-

0
duces to Z L+x
-1 Dn (u) f (x − u) du
−L+x
-4 -2 0 2 4
Since Dn and f are both periodic of period 2L, this equals
Z L
Dn (y) f (x − y) dy
−L
RL
Therefore, since −L Dn (y) dy = 1,
Z L
f (x+) + f (x−) f (x+) + f (x−)
− Sn f (x) = − Dn (y) f (x − y) dy
2 2 −L
650 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

Z L 
f (x+) + f (x−)
= − f (x − y) Dn (y) dy
−L 2
Z L Z L
= ( f (x+) + f (x−)) Dn (y) dy − ( f (x − y) + f (x + y)) Dn (y) dy
0 0
Z L   
f (x+) − f (x + y) 1 π
≤ π

sin n+ y dy +
0 2L sin 2L y
2 L
Z L   
f (x−) − f (x − y) 1 π
π

sin n+ y dy
0 2L sin 2L y
2 L
Both of these converge to 0 thanks to Lemma 33.4.3. To use this lemma, it is only necessary
to verify that the functions

f (x−) − f (x − y) f (x+) − f (x + y)
y→ π

, y→ π


2L sin 2L y 2L sin 2L y

are each Riemann integrable on [−L, L].

I will show this now. Each is continuous except for finitely many points of discontinuity.
The only remaining issue is whether the functions are bounded as y → 0. However, there
exists a constant K such that

f (x+) − f (x + y) K |y|
π

≤ π


2L sin 2L y 2L sin 2L y

and this expression converges to K/π, so the function is Riemann integrable. The other
function is similar. ■

Example 33.5.4 Let f (x) = |x| for x ∈ [−1, 1) and let f be periodic of period 2. Find the
Fourier series of f .

Here you need L = 1. Then

Z 1
1 1
a0 = |x| dx =
2 −1 2
Z 1
2  k

ak = |x| cos (kπx) dx =
(−1) − 1
−1 π 2 k2
Note that ak = 0 if k is even and it equals −4/ π 2 k2 when k is odd.

Since the function is even, the bk = 0. Therefore, the Fourier series equals
∞
1 4
−∑ cos (2k − 1) πx
2 k=1 π 2 (2k − 1)2

Now here is the graph of the function between −1 and 1 along with the sum up to 2 in the
Fourier series. You will notice that after only three terms the Fourier series appears to be
very close to the function on the interval [−1, 1]. This also shows how the Fourier series
approximates the periodic extension of this function off this interval.
33.5. POINTWISE CONVERGENCE OF FOURIER SERIES 651

3 Notice that if you take x = 0 in the above, the

theorem on pointwise convergence of Fourier series
implies that the sum converges to the value of the
2 function which is 0. Therefore,
∞
1 4
1 =∑
2 k=1 π 2 (2k − 1)2

It follows that
0
-2 0 2 π2 ∞
1
=∑ 2
.
8 k=1 (2k − 1)

This is a remarkable assertion.

Now here is another example for which the Fourier series will have to struggle harder
to approximate the function.

Example 33.5.5 Let f (x) = 1 on (0, 2] and f (x) = −1 on (−2, 0] and f (x + 4) = f (x).

First note that L = 2. In this case, the function is odd and so all the ak = 0.

1 2
  Z 2  
kπx kπx
Z
bk = f (x) sin dx = sin dx
2 −2 2 0 2
 
Then bk = πk2
1 − (−1)k . Thus for k even, this is 0. For k odd, this is πk4
. It follows the
Fourier series is  
∞
4 (2k − 1) πx
∑ sin
k=1 π (2k − 1) 2
In the picture, is a graph of the addition of the
first four terms of the Fourier series along with part
1
of the function. Notice the way the Fourier series
is struggling to do the impossible, approximate uni-
0 formly a discontinuous function with one which is
very smooth. That little blip near the jump in the
function will never go away by taking more terms
-1 in the sum.
Note that if you take x = 1 the series must con-
-4 -2 0 2 4 verge to 1. Therefore,
∞
4
1= ∑ π (2k − 1) (−1)k−1
k=1

It follows that
π ∞
(−1)k−1
=∑
4 k=1 2k − 1
This is another remarkable assertion.
652 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

33.5.2 Mean Square Convergence

It is the case that if f is Riemann integrable and 2L periodic, then the Fourier series con-
verges to the function f in the mean square sense. That is
Z L
lim | f (x) − Sn f (x)|2 dx = 0
n→∞ −L

I will show this now, leaving out a few details which will be reasonable to believe. Suppose
that f is continuous and periodic with period 2L. The Cesaro means of f are defined as
follows.
1 n 1 L
Z
σ n f (x) ≡ ∑ Sk f (x) , S0 f (x) = a0 ≡ 2L −L f (x) dx
n + 1 k=0
Thus, from what was shown above,

1 n
Z L
σ n f (x) = ∑ Dk (x − y) f (y) dy
n + 1 k=0 −L
!
1 n
Z L
=
−L
∑ Dk (x − y) f (y) dy
n + 1 k=0

Then the Fejer kernel is

1 n
Fn (t) = ∑ Dk (t) (*)
n + 1 k=0
We compute this now. Recall that

sin n + 12 πL t



Dn (t) = π


2L sin 2L t

Thus
1 1 n  π    
2
π  1 π
sin t Fn (t) = ∑ sin 2L t sin k + 2 L t
2L 2L n + 1 k=0
1 n
    π     
1 1 π π 1 π
= ∑ cos k + t − t − cos t + k + t
2L (n + 1) 2 k=0 2 L 2L 2L 2 L
1 1 1 n h π  π i
= ∑ cos kt − cos t (k + 1)
2L n + 1 2 k=0 L L
1  π 
= 1 − cos t (n + 1)
4L (n + 1) L
Thus


1 1 − cos πL t (n + 1)
Fn (t) = (**)
sin2 2L


4L (n + 1) π
t
Here are graphs of Fn (t) for n = 1, 2, · · · , 7 for L = π. Notice how they are nonnegative
and are large on a small interval containing 0. As you increase n, the bump in the middle
gets taller.
33.5. POINTWISE CONVERGENCE OF FOURIER SERIES 653

There areR certain properties which are obvious.

L
1 First of all, −L Fn (t) dt = 1. This follows from ∗
and 33.10 which pertained to the Dirichlet kernel.
Another which is obvious from the above formula
0.5 ∗∗ is that Fn (t) ≥ 0. Finally, for any small δ > 0, if
|t| > δ then
0
-4 -2 0 2 4


1 1 − cos πL t (n + 1)
Fn (t) ≤ (***)
sin2 2L


4L (n + 1) π
δ

It follows from periodicity that for M ≥ maxx | f (x)|

Z L
| f (x) − σ n f (x)| = ( f (x) − f (t)) Fn (x − t) dt
−L
Z L
= ( f (x) − f (x − u)) Fn (u) du
−L
Z Z
≤ | f (x) − f (x − u)| Fn (u) du + 2M | f (x) − f (x − u)| Fn (u) du
L≥|u|≥δ |u|<δ

1 2
Z
≤ 2M 2

+ 2M | f (x) − f (x − u)| Fn (u) du
4L (n + 1) sin 2L π
δ |u|<δ

Now if ε > 0, there is δ > 0 such that if |u| < δ , then for all x, | f (x) − f (x − u)| < ε/2.
Thus for such a choice of δ and ∗ ∗ ∗,
1 2 ε
| f (x) − σ n f (x)| ≤ 2M 2

+
4L (n + 1) sin 2L δ
π 2

and so, if n is large enough, you get

| f (x) − σ n f (x)| < ε,

this for any x. Thus the convergence of σ n f (x) to f (x) is uniform. It follows that
Z L
lim | f (x) − σ n f (x)|2 dx = 0
n→∞ −L

This shows the following interesting result.

Proposition 33.5.6 If f is continuous and 2L periodic, then the Cesaro means converge
uniformly to f and also they converge to f in the mean square sense.

From this, it is not hard to establish that the Cesaro means converge in mean square to
any 2L periodic function f which is Riemann integrable on intervals of length 2L. To do
this, you argue that, given a Riemann integrable function which is 2L periodic, there exists
a continuous function which is close to it in the mean square norm. Then apply the above
proposition to this continuous function and get a Cesaro mean close to it in mean square
which is close to the original function in mean square sense.
654 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

The Cesaro means are trig polynomials of the form

n    
kπx kπx
a0 + ∑ ak cos + bk sin .
k=1 L L

One of these can be made as close as desired to f in the mean square sense. Hence the
corresponding Fourier series is even closer, by the above section on mean square approxi-
mation. Thus, for every ε > 0 there exists N such that if n > N, then
Z L
|Sn f (x) − f (x)|2 dx < ε
−L

which says the Fourier series converge in the mean square sense to f .
Note that the above proposition also shows an improved result about pointwise conver-
gence. The function f did not need to have any control on its derivative and yet the Cesaro
means converged uniformly to the function. If the function were piecewise continuous,
the Cesaro means would converge to the mid point of the jump with no condition on the
derivatives from left or right. This is easy to show but is as far as this will be taken here.
If you want uniform approximation using trigonometric series, you should not be using the
Fourier series. You should use the Cesaro means.

33.6 Integrating and Differentiating Fourier Series

Suppose that f is 2L periodic and piecewise continuous. This is defined next.

Definition 33.6.1 Let f be a bounded function defined on [a, b] . It is called piecewise con-
tinuous if there is a partition of [a, b] , {x0 , · · · , xn } and for each k, a continuous function gk
such that f (x) = gk (x) for all x ∈ (xk−1 , xk ).

It turns out that you can integrate a Fourier series term by term. This is generally true
but I will show it here for piecewise continuous 2L periodic functions. Let f be such a
function equal to a continuous function on [xi , xi+1 ] for i ≤ n. Then consider
Z x
G (x) ≡ ( f (t) − a0 ) dt
−L

where a0 is the Fourier coefficient

Z L
1
a0 = f (t) dt
2L −L

Thus G (−L) = G (L) = 0 and if we continue using G to denote the 2L periodic extension,
it follows from Theorem 33.5.2 that the Fourier series of G
∞  nπx  ∞  nπx 
A0 + ∑ An cos + ∑ Bn sin
n=1 L n=1 L

converges to G at every point. This is because | f (t) − a0 | is bounded by some M due to the
assumption that it is piecewise continuous and the observation that
Z x
|G (x) − G (x̂)| ≤ | f (t) − a0 | ≤ M |x − x̂|
x̂
33.6. INTEGRATING AND DIFFERENTIATING FOURIER SERIES 655

Then plugging in π to the Fourier series for G we get

∞ ∞
0 = A0 + ∑ An (−1)n , A0 = − ∑ An (−1)n (*)
n=1 n=1

Next consider An , n > 0.

Z LZ x  nπx 
LAn = ( f (t) − a0 ) dt cos dx
−L −L L
n−1 Z xk+1 Z xk Z x   nπx 
= ∑ ( f (t) − a0 ) dt + ( f (t) − a0 ) dt cos dx
k=0 xk −L xk L
n−1 Z xk+1 Z xk  nπx 
= ∑ ( f (t) − a0 ) dt cos dx
k=0 xk −L L
x
!
n−1 L nπx R x


nπR sin L xk ( f (t) − a0 ) dt|xk+1
k
+∑
− xxkk+1 L nπx


k=0 nπ sin L ( f (x) − a0 ) dx
n−1 Z xk+1  nπx  n−1
L  nπx 
k+1
= ∑ G (xk ) cos dx + ∑ sin (G (xk+1 ) − G (xk ))
k=0 xk L k=0 nπ L
n−1 Z xk+1
L  nπx 
−∑ sin ( f (x) − a0 ) dx
k=0 xk nπ L
Now do an integration on the first sum. This yields
n−1
L   nπx 
k+1
 nπx 
k
∑ G (xk ) sin − sin
nπ k=0 L L
L n−1  nπxk+1 
+ ∑ sin L (G (xk+1 ) − G (xk ))
nπ k=0
n−1 Z xk+1
L  nπx 
−∑ sin ( f (x) − a0 ) dx
k=0 xk nπ L

The sums simplify and the result one obtains is

n−1
L  nπx
k+1
  nπx 
k
∑ G (xk+1 ) sin − G (xk ) sin
nπ k=0 L L

n−1 Z xk+1
L  nπx 
−∑ sin ( f (x) − a0 ) dx
k=0 xk nπ L
The series telescopes and the result is 0 because G (L) = G (−L) = 0. Thus the result of it
all is
n−1 Z xk+1
L  nπx 
LAn = −∑ sin ( f (x) − a0 ) dx
k=0 xk nπ L
Z L
L  nπx 
= − sin ( f (x) − a0 ) dx
−L nπ L
656 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

Thus Z L
L 1  nπx  L
An = − sin ( f (x) − a0 ) dx = − bn
nπ L −L L nπ
Similar computations will show that for n > 0,
Z L
L 1  nπx  L
Bn = cos ( f (x) − a0 ) dx = an
nπ L −L L nπ
where an , bn are, respectively, the cosine and sine Fourier coefficients of f . Thus we have
from ∗,  An

Z x ∞
L 
G (x) = ( f (t) − a0 ) dt = − ∑ − bn (−1)n +
−L n=1 nπ
∞ ∞
L  nπx  L  nπx 
∑ − nπ bn cos L
+∑ an sin
n=1 n=1 nπ L
Hence
Z x ∞     nπx 
Lbn −nπL
( f (t) − a0 ) dt = ∑ nπ cos − cos
−L n=1 L L
∞
L  nπx 
+∑ an sin
n=1 nπ L

Thus
Z x Z x ∞ Z x  nπt 
f (t) dt = a0 dt + ∑ bn sin dt
−L −L n=1 −L L
∞ Z x  nπt 
+ ∑ an cos dt
n=1 −L L

This proves the following theorem.

Theorem 33.6.2 Let f be piecewise continuous and 2L periodic. Then for every x ∈
[−L, L] ,
Z x Z x ∞ Z x  nπt 
f (t) dt = a0 dt + ∑ bn sin dt
−L −L n=1 −L L
∞ Z x  nπt 
+ ∑ an cos dt
n=1 −L L

where a0 , ak , bk are the Fourier coefficients for f .

Note that there is nothing which says that the Fourier series of f converges to f ! This
is a wonderful result.
You can’t expect to be able to differentiate Fourier series. See the exercises. However,
there is something which can be said. Suppose for x ∈ [−L, L)
Z x
f (x) = f (−L) + f ′ (t) dt
−L
33.6. INTEGRATING AND DIFFERENTIATING FOURIER SERIES 657

and that f ′ is piecewise continuous and 2L periodic. Let f denote the 2L periodic extension
of the above f . Then let the formal Fourier series for f ′ be
∞  nπx  ∞  nπx 
a0 + ∑ an cos + ∑ bn sin
n=1 L n=1 L
Then by Theorem 33.6.2,
Z x Z x ∞
L  nπx 
f ′ (t) dt = a0 dt + ∑ an sin
−L −L n=1 nπ L
∞
L   nπx 
+ ∑ bn (−1)n − cos
n=1 nπ L
1 RL ′ 1
Then a0 ≡ 2L −L f (t) dt ( f (L) − f (−L)) = 0.
= 2L
1 L  nπt  1  nπt 
Z
an ≡ f ′ (t) cos dt = f (t) cos |L−L
L −L L L L
1 nπ L  nπt 
Z
+ f (t) sin dt
L L −L L
1 nπ L  nπt  nπ
Z
= f (t) sin dt = Bn
L L −L L L
where Bn is the Fourier coefficient for f (t) . Similarly,
1 L ′  nπt  1 nπ L  nπt  nπ
Z Z
bn = f (t) sin dt = − f (t) cos dt = − An
L −L L L L −L L L
where An is the nth cosine Fourier coefficient for f . Thus
Z x ∞
nπ L  nπx 
f ′ (t) dt = ∑ Bn sin
−L n=1 L nπ L
∞  nπ  L   nπx 
+ ∑ − An (−1)n − cos
n=1 L nπ L
∞  nπx  ∞  nπx  ∞
f (x) − f (−L) = ∑ Bn sin + ∑ An cos − ∑ An (−1)n
n=1 L n=1 L n=1
!
∞  nπx  ∞  nπx  ∞
n
f (x) = ∑ Bn sin + ∑ An cos + f (−L) − ∑ An (−1)
n=1 L n=1 L n=1
Thus that constant on the end is A0 . It follows that
∞  nπx  ∞  nπx 
f (x) = A0 + ∑ Bn sin + ∑ An cos
n=1 L n=1 L
and − nπ nπ
L An = bn , Bn L = an and so
∞  nπx  ∞  nπx 
f ′ (x) = ∑ an cos + ∑ bn sin
n=1 L n=1 L
∞ ∞
nπ  nπx   nπ   nπx 
= ∑ Bn cos + ∑ An − sin
n=1 L L n=1 L L
∞ ∞
d  nπx  d  nπx 
= ∑ Bn sin + ∑ An cos
n=1 dx L n=1 dx L
658 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

This proves the following.

Theorem 33.6.3 Let f denote the 2L periodic extension of the function f given on [−L, L)
by Z x
f (x) = f (−L) + f ′ (t) dt
−L
and suppose f ′ is 2L periodic and piecewise continuous. Then for each x ∈ [−L, L] ,
∞  nπx  ∞  nπx 
f (x) = A0 + ∑ Bn sin + ∑ An cos
n=1 L n=1 L

where the Ak , Bk are the Fourier coefficients of f and the Fourier series for f ′ is
∞  nπx  ∞
d d  nπx 
∑ Bn sin + ∑ An cos
n=1 dx L n=1 dx L

33.7 Odd and Even Extensions

Often, as in the above examples and in the applications which follow, the function you are
finding the Fourier series for is either even or odd. One way this often occurs is when the
function of interest is defined on an interval [0, L] and it is only its values on this interval
which are of interest. Then you could consider either the even or the odd extension of this
function to [−L, L] and then extend it to be a 2L periodic function. For example, consider
the following pictures.

−L L
−L L

The first of these is an even extension to [−L, L] and the second is an odd extension
to [−L, L]. In the first case where there is an even extension, the Fourier coefficients are
bk = 0
1 L 1 L
Z Z
a0 = f (x) dx = f (x) dx
2L −L L 0
1 L 2 L
   
kπx kπx
Z Z
ak = f (x) cos dx = f (x) cos dx
L −L L L 0 L
In the second case where you are dealing with the odd extension, each ak = 0 and

1 L 2 L
   
kπx kπx
Z Z
bk = f (x) sin dx = f (x) sin dx
L −L L L 0 L

Example 33.7.1 Let f (x) = x on [0, 1] . Find the Fourier series of its even extension.
33.8. EXERCISES 659

Its even extension is nothing more than the function of Example 33.5.4. This is
∞
1 4
−∑ cos (2k − 1) πx
2 k=1 π 2 (2k − 1)2

Example 33.7.2 Let f (x) = x on [0, 1] . Find the Fourier series of its odd extension which
is periodic of period 2.

This would be the function f (x) = x on (−1, 1) extended to be periodic of period 2.

Thus L = 1. Since it is an odd function, all the ak = 0 and from the above,
Z 1
2
bk = 2 x sin (kπx) dx = (−1)k+1
0 πk
Then the Fourier series is
∞
2
∑ πk (−1)k+1 sin (kπx)
k=1
The graph of the sum of the first five terms is given.

1
0
-1
-2 0 2
Note the difficulty in handling the jump with the little bump right before the disconti-
nuity. This illustrates that if you are only interested in the function on [0, 1] , it would be
better to use the even extension than the odd extension. However, in the applications, you
don’t get to choose.
Also, note that, unlike power series, Fourier series are attempting to approximate a
function on a whole interval, not just near a single point. This is much more interesting.
There is a general sort of problem called a Sturm-Liouville problem discussed in Prob-
lem 13. It turns out that there are general theorems about convergence of expansions in
terms of eigenfunctions to such problems [38]. However, you can often see that conver-
gence in the mean square sense will hold from observing that the Fourier series for the
eigenfunctions will converge because it is the restriction of the Fourier series of an even
or odd extension as discussed in this section. For many other considerations on Sturm-
Liouville problems, the old book by Ince [23] is very good. These problems have been
intensively studied since around 1830.

33.8 Exercises
1. Let f (x) be the even extension of sin x. Find the Fourier series and at x = π/2 write a
series which says that the Fourier series converges to the function at this point. Note
that here L = π and so nπ/L = n.
2. Let f (x) be the odd 2π periodic extension of y = x2 . Find what the sum converges to
at x = π/2. Again L = π and so nπ/L = π.
660 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

3. Let f (x) be the even 2π periodic extension of y = x2 . Find the Fourier coefficients
and obtain an interesting series by letting x = π.
4. In Example 33.5.5 the Fourier series was found for the function f which is 1 on [0, 2]
and −1 on (−2, 0).  
∞
4 (2k − 1) πx
∑ sin
k=1 π (2k − 1) 2
This function has a jump so it is not differentiable at 0,2,4, etc. However, it is dif-
ferentiable at most points, other than a few jumps. Furthermore, the Fourier series
converges to the function at these points. Can you differentiate the Fourier series
term by term and get something which converges to the derivative of the function?
What does this show about interchange of limits?
5. In one of the problems above, you found that the Fourier series for the 2π periodic
2 k
(−1)
extension of y = x2 is π3 + ∑∞
k=1 4 k2 cos (kx) . The derivative of this function, y =
2x is sure piecewise continuous. Find the Fourier series expansion for y = 2x without
any effort.

6. Find a Fourier series which converges to the 2π periodic extension of

π2
Z x  
1 1
t2 − dt = x3 − π 2 x.
−π 3 3 3

7. Suppose f is periodic with period 2L. Does it follow that f ′ is also periodic of period
2L? Explain.
8. Here are some boundary value problems. Find nonzero solutions if there are any or
determine that there are none.

2
(a) y′′ + 14 π 2 y = 0, (e) y′′ + 12 π y = 0,
y (0) = 0, y (2) = 0 y (0) = 0, y (1) = 0

2
(b) y′′ + 7π 5 y = 0, (f) y′′ + π 2 y = 0,
y (0) = 0, y 52 = 0


y (0) = 0, y (1) = 0

2
2
(c) y′′ + 5π 3 y = 0, (g) y′′ + 4π 2 y = 0,
y (0) = 0, y 32 = 0 y (0) = 0, y 22 = 0




2
(d) y′′ + 2π 7 y = 0, (h) y′′ + 25
9 2
π y = 0,
y (0) = 0, y 72 = 0 y (0) = 0, y 52 = 0

9. Here are some boundary value problems. Find nonzero solutions if there are any or
determine that there are none.

2
2
(a) y′′ + 11π
2 y = 0, (c) y′′ + 2π
7 y = 0,
′
y (0) = 0, y 22 = 0 ′
y (0) = 0, y 72 = 0

2
2
(b) y′′ + 4π
5 y = 0, (d) y′′ + 7π
3 y = 0,
y (0) = 0, y′ 25 = 0 y (0) = 0, y′ 32 = 0



33.8. EXERCISES 661

π 2 y (0) = 0, y′ 25 = 0
(e) y′′ + 13



y = 0,
y (0) = 0, y′ 13


2 =0

2
(g) y′′ + 5π
11 y = 0,

2
(f) y′′ + 2π ′
y (0) = 0, y 11


5 y = 0, 2 =0

10. In boundary value problems like the above, why is it that there is either no nonzero
solution or infinitely many?
11. In the study of buckling beams, you have an equation
y(4) (x) + λ y′′ (x) = 0, x ∈ [0, L]
along with boundary conditions like
y (0) = y′ (0) = 0, clamped at left end
y (L) = 0 = y′′ (L) , hinged at right end
where λ increases with the axial force and depends on geometrical and physical
properties of the beam. The idea is to find values of λ for which there is a nonzero
solution to the differential equation and the boundary conditions. Assume all bound-
ary conditions considered have y (0) = y (L) = 0 and at each end, either y′ or y′′ is
equal to 0. Thus one considers beams for which each end is either clamped or hinged.
Show that if λ is such that there exists a nonzero solution, then λ > 0. Hint: You
show this by multiplying the equation by y and integrating by parts.
12. Letting λ = δ 2 , in the above problem, show that there exist infinitely many values
for δ and corresponding nonzero solutions to the boundary value problem for the
following situation.

(a) y (0) = y′ (0) = 0, y (L) = y′ (L) = 0

(b) y (0) = y′′ (0) = 0, y (L) = y′ (L) = 0
(c) y (0) = y′′ (0) = 0, y (L) = y′′ (L) = 0

13. A Sturm-Liouville problem involves the differential equation for an unknown func-
tion of x which is denoted here by y,

′
p (x) y′ (x) + (λ q (x) + r (x)) y = 0, x ∈ [a, b]
and it is assumed that p (t) , q (t) ≥ 0 and are nonzero except for finitely many points
in [a, b] for any t along with boundary conditions,
C1 y (a) +C2 y′ (a) = 0
C3 y (b) +C4 y′ (b) = 0
where
C12 +C22 > 0, and C32 +C42 > 0.
There is an immense theory connected to these important problems. The constant λ
is called an eigenvalue. Show that if y is a solution to the above problem correspond-
ing toλ = λ 1 and if z is a solution corresponding to λ = λ 2 ̸= λ 1 , then
Z b
q (x) y (x) z (x) dx = 0. (33.11)
a
662 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

Hint: Do something like this:

′
p (x) y′ z + (λ 1 q (x) + r (x)) yz = 0,

′
p (x) z′ y + (λ 2 q (x) + r (x)) zy = 0.
Now subtract and either use integration by parts or show

′
′

′
p (x) y′ z − p (x) z′ y = p (x) y′ z − p (x) z′ y

and then integrate. Use the boundary conditions to show that y′ (a) z (a)−z′ (a) y (a) =
0 and y′ (b) z (b) − z′ (b) y (b) = 0. The formula 33.11 is called an orthogonality rela-
tion and it makes possible an expansion in terms of certain functions called eigen-
functions.

14. Here is a really nice result. Suppose you have y, z are both solutions of the differential
equation
′
p (x) y′ (x) + q (x) y (x) = 0
Show that p (x)W (y, z) (x) = C a constant. Here W (y, z) is the Wronskian.
(x)′
15. In the above problem, change the variables as follows. Let z (x) = p (x) yy(x) and
determine the equation which results for z. This kind of equation is called a Riccati
equation. In particular, show that
1 2
z′ + z + q (x) = 0
p (x)

This kind of equation is like a Bernouli equation with exponent 2, but with another
function added in. For more on this, see [29].
16. Suppose in the equation of Problem 14 you have two solutions u, v whose Wronskian
is nonzero so they are independent solutions. Suppose that a, b are consecutive zeros
of u and that p (x) > 0 on [a, b]. Show that v has exactly one zero in (a, b). This is
called the Sturm separation theorem. Hint: Use the result of the above mentioned
problem and argue that v (a) ̸= 0 and that you can assume that v (a) > 0 and that u is
positive on the open interval (a, b).
17. Letting [a, b] = [−π, π] , consider an example of a Sturm-Liouville problem which is
of the form
y′′ + λ y = 0, y (−π) = 0, y (π) = 0.
Show that if λ = n2 and yn (x) = sin (nx) for n a positive integer, then yn is a solution
to this regular Sturm-Liouville problem. In this case, q (x) = 1 and so from Problem
13, it must be the case that
Z π
sin (nx) sin (mx) dx = 0
−π

if n ̸= m. Show directly using integration by parts that the above equation is true.
33.8. EXERCISES 663

18. Sometimes one encounters an eigenvalue problem of the form

x2 y′′ + xy′ + λ x2 − n2 y = 0, C1 y (L) +C2 y′ (L) = 0

not both Ci equal zero and y bounded near 0. Discover an orthogonality relation
between this solution and one for which λ is changed to µ. Hint: You might divide
by x.
19. Let x → Jn (x) be a solution to the Bessel equation

x2 y′′ + xy′ + x2 − n2 y = 0

and suppose α is a positive number. Let z (x) ≡ Jn (αx). Find a differential equation
satisfied by z. You should show that it satisfies x2 z′′ + xz + α 2 x2 − n2 z = 0.

20. Let α, β be two zeros of the Bessel function Jn (x). It was shown in Proposition
32.6.1 on Page 629 that there are infinitely
  many of these zeros. Now consider the
two functions x → Jn αL x , x → Jn βL x . Show that

Z L α  β 
Jn x Jn x xdx = 0
0 L L

21. Consider
 
x2 y′′ + xy′ + δ 2 x2 − n2 y = 0, y (L) = 0, y bounded near 0

Show that there are only certain values of δ which work and they are of the form
δ 2 = (α/L)2 where α is some zero of a solution to Bessel’s equation.
22. Show that the only eigenvalues λ for

x2 y′′ + xy′ + λ x2 − n2 y = 0, y (L) = 0

are positive.
23. Recall that for n an integer, the general solution to Bessel’s equation is C1 Jn (x) +
C2Yn (x) where Yn is unbounded at 0. Using the above problem, characterize all
eigenvalues λ of the eigenvalue problem

x2 y′′ + xy′ + λ x2 − n2 y = 0, y (L) = 0, y bounded near 0.

and describe all solutions to this boundary value problem in terms of Bessel func-
√
tions. Hint: Rule out Yn to begin with. Then consider z λ x = y (x) for y a
solution to the above Sturm-Liouville equation.
24. A Sturm-Liouville eigenvalue problem involves the equation

′
p (x) y′ (x) + (λ q (x) + r (x)) y = 0, x ∈ (a, b)

The Liouville transformation is

Z x 1/2
1/4 q (s)
z = (p (x) q (x)) y, t = ds, c ∈ (a, b)
c p (s)
664 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

Then determine the equation solved by z. Hint: This is a little involved. First verify
that the left side reduces to
 r
d d  −1/4

1/4 dz q
p (pq) z + (pq) + (λ q + r) (pq)−1/4 z = 0
dt dx dt p

Next verify that the z′ (t) terms all cancel. That way, in the above, you can neglect
these terms in using the product rule. This leads to
!
d 1 −1/4 −5/4 d −1/4


− p q (pq) r (pq)
dx 4
dx + −1/4 3/4
z + z′′ + λ z = 0
p−1/4 q3/4 p q

Now argue that the equation is of the form

z′′ + (λ + m (t)) z = 0

where m (t) is a function which depends on p, q.

25. Consider the eigenvalue problem for Bessel’s equation,

x2 y′′ + xy′ + λ x2 − n2 y = 0, y (L) = 0

Show it can be written in self adjoint form as

n2
 

′
xy′ + λ x − y=0
x

Thus in this case, q (x) = x and r (x) = −n2 /x. What is the form of the equation if Li-
ouville’s transformation is applied to this Bessel eigenvalue problem? Hint: Just use
the specific description of what was obtained above and that r (x) = −n2 /x, p (x) =
q (x) = x, and so t = x. You should get something like

1 − 4n2
 
′′
z +λz+ z=0
4x2

26. In the above problem, let λ = 1 and let n = 1/2 and use to find the general solution
to the Bessel equation in which ν = 1/2. Show, using the above, that this general
solution is of the form
C1 x−1/2 cos x +C2 x−1/2 sin x.

27. Show that the polynomial q (x) of degree n which minimizes

Z 1
| f (x) − p (x)|2 dx
−1

out of all polynomials p of degree n is the nth partial sum of the Fourier series
taken with respect to the Legendre polynomials q (x) = Sn f (x) , where Sn f (x) ≡
R1
∑nk=0 ck pk (x) , ck = −1 qk (x) f (x) dx .
33.8. EXERCISES 665

28. Recall the normalized Legendre polynomials

√
2n + 1
qn (x) = √ pn (x)
2
which have the property that
Z 1
(
1 if j = k
q j (x) qk (x) dx = δ jk =
−1 ̸ k
0 if j =

If f is a Riemann integrable function, show that

Z 1
lim f (x) qn (x) dx = 0
n→∞ −1

29. Show that if f is any continous function on [−1, 1] , then the Fourier series in terms
of Legendre polynomials converges R1
to f in the mean square sense. This means that
for Sn f (x) ≡ ∑nk=0 ck pk (x) , ck = −1 qk (x) f (x) dx, it follows that
Z 1
lim | f (x) − Sn f (x)|2 dx = 0
n→∞ −1

30. It can be shown that there are no continuous, nonzero solutions to Legendre’s equa-
tion

′
1 − x2 y′ + λ y = 0
defined on [−1, 1] unless λ = n (n + 1) for n an integer. Use the above problem to
show this.
31. One of the applications of Fourier series is to obtain solutions to linear differential
equations which have a periodic right side. This is done by expanding the right side
which is a forcing function in a Fourier series, solving the simple equation which
corresponds to each term and then adding these solutions to obtain what is hoped
to be a representation of the solution. Find a particular solution for each of the
following. Let
y′′ + 3y = f (t) ,
where f (t) is the step function which is periodic of period 2 and equals −1 on
[−1, 0) and 1 on (0, 1]. Here are the steps. First find a Fourier series for f . Say
∑∞n=1 bn sin (nπx) . Then let yn be the solution to

y′′n + 3yn = sin (nπt)

and then hopefully, on neglecting mathematical issues, the solution to the original
problem is
∞
y (t) = ∑ bn yn (t)
n=1

32. Explain why the above procedure should give a particular solution if mathematical
issues related to interchange of limit operations are ignored.
666 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES

33. This problem is tedious but maybe it is better to do it all at once than to repeat
seemingly endless virtually identical problems. In this problem, a is positive and b is
a nonzero real number while n is a nonnegative integer. Find the real and imaginary
parts of a solution y to  nπt 
y′′ + 2ay′ + by = exp i
L
using the method of undetermined coefficients. Show that the real part is

2πL3 an sin πL nt − π 2 L2 n2 cos πL nt

L4 b2 + 4π 2 L2 a2 n2 − 2π 2 L2 bn2 + π 4 n4
and the imaginary part of the solution is

L4 b − π 2 L2 n2 sin πL nt − 2πL3 an cos πL nt

L4 b2 + 4π 2 L2 a2 n2 − 2π 2 L2 bn2 + π 4 n4
Explain why the real part is a particular solution to
 nπt 
y′′ + 2ay′ + by = cos
L
and the imaginary part is a particular solution to
 nπt 
y′′ + 2ay′ + by = sin
L
In case n is 0, a solution is 1/b.
34. Using the above problem, describe the solution after a long time to the equation

y′′ + 2y′ + 2y = f (t)

where f (t) is a periodic function which has the following Fourier series. Note that
the transient terms will disappear due to the fact that a = 1 is positive. Note that
with the above problem, you could do many other examples in which a and b are not
given as here.
1 nπt 1 nπt



(a) ∑∞ ∞
n=1 n2 cos 3 + ∑n=1 1+n2 sin 3 + 3
−n nπt 1 nπt



(b) ∑∞ ∞
n=1 e cos 2 + ∑n=1 n4 sin 2 + 1
1 nπt 1 nπt



(c) ∑∞ ∞
n=1 n3 cos 4 + ∑n=1 n3 +1 sin 4 − 2

35. Suppose you have an undamped equation

y′′ + 4y = f (t)

where f is periodic. Suppose in the Fourier expansion of f (t) there is a nonzero term
which is of the form b sin (2t) . Say it describes the transverse vibrations of a bridge
in the center. What will likely happen to this bridge?
33.8. EXERCISES 667

36. Consider the functions yn (x) = sin (nπx) on the interval [0, 2] . Show that these func-
tions satisfy 02 yn (x) ym (x) dx is 1 if n = m and zero if n ̸= m. Now consider using
R

them to expand the function f (x) = x in a Fourier series. Thus you would have
∞
∑ bn sin (nπx)
n=1

where Z 2
bn = x sin (nπx) dx
0
Graph the sum of the first seven terms in this Fourier series expansion along with
the function it is supposedly approximating. What does this tell you about being
able to approximate with orthogonal

functions? Now do the same problem with the
orthonormal functions sin n π2 x .
37. Recall that a sequence of functions defined on [a, b] { fn } converges to f in the mean
square sense if
Z b
lim | fn (x) − f (x)|2 dx = 0
n→∞ a
√
consider the function fn (x) for x ∈ [0, 1] defined as follows. fn (x) = n on (0, 1/n)
and fn (x) = 0 for x not on this interval. Show that limn→∞ fn (x) = 0 for each x
but
 fn fails to converge to 0 in the mean square sense. Now let fn (x) = 1 for x ∈
1, 1/2, 1/22 , · · · , 1/2n but it equals zero at all other points. Show that fn converges
to 0 in the mean square sense but not at every point.

38. Using Example 33.5.5 and the convergence theorem for Fourier series, explain why
∞  
4 (2k − 1) πα
1= ∑ sin for all α ∈ (0, 2) .
k=1 π (2k − 1) 2
668 CHAPTER 33. BOUNDARY VALUE PROBLEMS, FOURIER SERIES
Chapter 34

Some Partial Differential

Equations

34.1 Laplacian in Orthogonal Curvilinear

Coordinates
Recall the formula for the Laplacian in curvilinear coordinates
 
1 ∂ ik ∂ φ (x) p
∆φ (x) = p g (x) g (x)
g (x) ∂ xi ∂ xk

where g (x) was the determinant of the metric tensor. Using this, it was shown earlier that
the Laplacian in spherical coordinates can be obtained.

Example 34.1.1 Laplacian in spherical coordinates.

      
1 ∂ ∂f ∂ ρ sin φ ∂ f ∂ ρ ∂f
∆f = 2
ρ 2 sin φ + +
ρ sin φ
∂ρ ∂ρ ∂φ ρ ∂φ ∂ θ ρ sin φ ∂ θ

∂2 f
   
1 ∂ ∂f 1 ∂ ∂f 1
= 2 ρ2 + 2 sin (φ ) + 2 2
ρ ∂ρ ∂ρ ρ sin φ ∂ φ ∂φ ρ sin φ ∂ θ 2
Using the same machinery, one can obtain the Laplacian in sylindrical coordinates.

Example 34.1.2 Laplacian in cylindrical coordinates.

x = r cos θ
y = r sin θ
z=z

1 ∂2 f
      
1 ∂ ∂f ∂ 1∂f 1 ∂ ∂f
∆f = r + = r + 2
r ∂r ∂r ∂θ r ∂θ r ∂r ∂r r ∂θ2

669
670 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

34.2 Heat and Wave Equations

34.2.1 Heat Equation
Fourier’s law of heat conduction is that the heat flux J is proportional to the temperature
gradient ∇u where here u is the temperature. Specifically it says that

J = −k∇u

So what is the “heat flux”? Hopefully, you saw flux integrals in calculus but here is a short
review. If you have a surface S and a field of unit normals on S denoted as n, then the rate
at which the heat crosses S in the direction of n is
Z
J · ndS
S

where this is an integral over the surface. Now consider a ball B with boundary S in a heat
conducting material. Then the heat in B is given by
Z
ρcudV
B

where ρ is the density and c the specific heat. Then if no heat is being produced by some
chemical reaction for example, it follows that the time rate of change of the total heat in B
is equal to the rate at which heat flows into B. Thus
Z 
d
Z
ρcudV = − J · ndS
dt B S

where n is the outer normal from B. This is why there is a minus sign on the right. You
want the rate at which heat enters B. Then from the divergence theorem,
Z 
d
Z
ρcudV = − ∇ · J dV
dt B B

The integral is a sort of a sum, here over the spacial variables and so it makes sense to
formally take the time derivative into the integral1 and write, using the Fourier law of heat
conduction
∂ (ρcu)
Z Z
dV = ∇ · (k∇u) dV
B ∂t B
This must hold for any ball B and so the only way this could take place is to have
∂ (ρcu)
= ∇ · (k∇u)
∂t
We now let k, c, ρ all be constants and obtain
∂u k
= ∆u
∂t ρc
Of course these things are typically not constant, especially k but if we don’t assume this,
we can’t solve the equation.
1 This is horrible mathematics because it exchanges two limit operations. However, when modeling, one

doesn’t worry about rigorous math.

34.2. HEAT AND WAVE EQUATIONS 671

In one dimension, this reduces to

ut = α 2 uxx

and this is the equation in what follows. There are other issues besides the equation to
consider.
You have a rod of length L. The heat equation for the temperature u in the rod is

ut = α 2 uxx

In addition to this, there are boundary conditions given on u at the ends of the rod. For
example, you could have
u (0,t) = u (L,t) = 0
and there is also an initial temperature given

u (x, 0) = f (x)

Then the idea is to find the unknown function u (t, x) . Here t is time and x is the coordinate
of a point on the rod. The constant α 2 varies from material to material. It is different for
iron than for aluminum for example. Here you have x ∈ [0, L] and t > 0.
This is a rectangular shape and so it is reasonable to look for a nonzero solution to the
above partial differential equation and boundary condition in the form

u (x,t) = a (t) b (x)

Then
a ′ (t) b (x) = α 2 a (t) b ′′ (x)
One can separate the variables as follows.
a′ (t) b′′ (x)
2
= (34.1)
α a (t) b (x)
Both sides must equal to some constant c since otherwise they could not be equal. One way
to see this is to differentiate both sides with respect to t. Then

a (t) ′
 ′
a′ (t)

2
= 0 and so 2 = c,
α a (t) α a (t)
a constant. Consider the side involving x.

b ′′ (x) − cb (x) = 0, b (0) = b (L) = 0

Of course you can’t have b (x) = 0 since if it were 0, you would have u (x,t) = 0. Therefore,
2 2
from Example 33.2.1, −c = nLπ2 where n is a positive integer and
 nπx 
b (x) = sin
L
Of course there is such a function for each n a positive integer. Having picked such a
positive integer, 34.1 now forces a (t) to satisfy the equation

n2 π 2 α 2
a′ (t) + a (t) = 0
L2
672 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

Therefore,
2 2 2
− n π 2α t
a (t) = an e L

It follows that for each n, there exists a solution to the partial differential equation along
with the boundary conditions which is of the form
2 2 2  nπx 
−n π α t
un (x,t) = an e L2 sin
L
Now if you have solutions to the differential equation along with the boundary condition
and you add them together, you have another solution to these things. Therefore, it is not
unreasonable to hope that this would also be true for an infinite sum of such solutions.
Therefore, we look for a solution to the partial differential equation which is of the form
∞ 2 2 2  nπx 
−n π α t
u (x,t) = ∑ an e L2 sin
n=1 L
At least formally, such a thing would solve everything but the initial condition. Now you
choose an in such a way that when t = 0,
∞  nπx 
f (x) = ∑ an sin
n=1 L

for x ∈ [0, L] . This is now a Fourier series problem.

Another point of view is to look for eigenfunctions. b such that
b′′ (x) + λ b (x) = 0, b (0) = b (L) = 0
This is because if you had such an eigenfunction, you could replace the b′′ (x) with −λ b (x) .
2 2
From Example 33.2.1 on Page 638, λ = nLπ2 where n is a positive integer and
 nπx 
b (x) = sin
L
Denote by bn this eigenfunction. Then look for a solution to the whole problem which is in
the form
∞
u (x,t) = ∑ ak (t) bk (x)
k=1
Then proceeding formally,
∞ ∞ ∞  2 2 
k π
∑ a′k (t) bk (x) = α 2 ∑ ak (t) b′′k (x) = ∑ ak (t) − L2 α 2 bk (x)
k=1 k=1 k=1

It follows that you should have

k2 π 2 α 2
a′k (t) + ak (t) = 0
L2
so this results in
2 2 2
∞  
− k π 2α t kπx
u (x,t) = ∑ ak e L sin ,
k=1 L
the same as before. Then you just try and find the ak to satisfy the initial condition.
Here is a summary of the method. This method is general and will work for all the
examples discussed here.
34.2. HEAT AND WAVE EQUATIONS 673

PROCEDURE 34.2.1 To find the solution to an equation

ut = α 2 uxx , zero boundary conditions, Initial condition

you do the following.

1. First find eigenfunctions, nonzero solutions to

y′′ + λ 2 y = 0, boundary conditions

There will typically be infinitely many of these {yn (x)}∞

n=1 corresponding to eigen-
values λ n where limn→∞ λ n = ∞.
2. Your solution will then be of the form
∞
u (x,t) = ∑ bn (t) yn (x)
n=1

3. Choose bn (t) to satisfy the equation b′n (t) = −λ 2n bn (t) in order that the terms of the
sum satisfy the partial differential equation. Thus
 
bn (t) = bn exp −tλ 2n

Then the solution to the problem is

∞  
u (x,t) = ∑ bn exp −tλ 2n yn (x)
n=1

where bn is chosen such that ∑∞

n=1 bn yn (x) is the Fourier series expansion for the
initial condition.

Example 34.2.2 Find the solution to the initial boundary value problem

ut = .1uxx , u (0,t) = u (2,t) = 0

u (x, 0) = 1 − (1 − x)2

where (
x if x ∈ [0, 1]
f (x) =
1 − x if x ∈ [1, 2]

From the above discussion,

2 2
∞  
− .1k 2π t kπx
u (x,t) = ∑ ak e 2 sin
k=1 2

kπx


the eigenfunctions being sin 2 , and to satisfy the initial condition, you need

2 2   kπx 
Z
2
ak = 1 − (1 − x) sin dx
2 0 2
674 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

After some tedious computations, this yields

16  
ak = 3 3 1 − (−1)k
π k
32
Thus when k is even, this is 0 and when k is odd, it equals π 3 k3
. Thus
∞  
32 −
(2k−1)2 π 2
(.1)t (2k − 1) πx
u (x,t) = ∑ π 3 (2k − 1)3 e 4 sin
k=1 2

The next example has to do with the same equation but with one end insulated and
the other held at a temperature of 0. The physical modeling of this equation shows that to
consider an insulated boundary, say at L, you let ux (L,t) = 0.

Example 34.2.3 Solve the problem

ut = .1uxx , u (0,t) = ux (2,t) = 0

u (x, 0) = 1 − (1 − x)2

To do this, first look for eigenfunctions. Find solutions to

y′′ + λ y = 0, y (0) = 0, y′ (2) = 0

Then the eigenfunctions are in Example 33.2.1. They are

 
(2n − 1) πx
sin , n = 1, 2, · · ·
4
It follows that the solution desired is of the form
∞  
(2n − 1) πx
∑ bn (t) sin 4
n=1

and one needs  2

1 (2n − 1) π
b′n (t) = − bn (t)
10 4
so  2 !
1 (2n − 1) π
bn (t) = bn exp − t
10 4
Then the Fourier series expansion of the solution is
∞  2 !  
1 (2n − 1) π (2n − 1) πx
∑ bn exp − 10 4
t sin
4
n=1

where bn is an appropriate Fourier coefficient chosen to satisfy the initial condition. Thus
2 2
 
(2n − 1) πx 
Z 
bn = sin 1 − (1 − x)2 dx
2 0 4

32
= (2 (−1)n πn − (−1)n π + 4)
π 3 (2n − 1)3
34.2. HEAT AND WAVE EQUATIONS 675

Then the solution to this problem is

! 2  
∞ 
32 n n
1
− 10
(2n−1)π
t (2n − 1) πx
∑ (2 (−1) πn − (−1) π + 4) e 4
sin
n=1 π 3 (2n − 1)3 4
A graph of this function of two variables in which the sum is taken up to 8 for (t, x) ∈
[0, 12] × [0, 2] is:

0.5

0
0 5 1 2
10 0
34.2.2 The Wave Equation
The next example is of a different sort of equation, the wave equation. This equation is of
the form
utt = c2 uxx
It models the transverse displacements of a vibrating string. Here is a picture to discuss
why this is an appropriate equation. It is important to note that it is a string, not a beam.
This means that it cannot support itself in the sense that there is no internal stiffness. It is
also very important to note that the transverse displacements are assumed to be very small.
Thus the picture drawn below is blown up in the vertical direction.

T
θ

α
T x x + ∆x

Let ρ be the length density of this string which is assumed constant. This means that
the mass of the segment of string shown is just ρ (length of the segment of string) . Since
the transverse displacements are very small, this is essentially ρ∆x. The force acting on
the segment of string shown is T sin θ − T sin α, where T is the magnitude of the vector
T . We assume also that the magnitude of the tension in the string is also a constant due to
the assumption that the displacements are small. Let u (t, x) denote the vertical displace-
ment from horizontal. For ∆x small enough, the acceleration utt (t, x) should be essentially
constant on the interval [x, x + ∆x]. Then by Newton’s second law,
ρ∆xutt (t, x) = T (sin θ − sin α)
676 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

Since the displacement is very small, we can assume that there is really no difference in
replacing sin θ , sin α with tan θ , tan α respectively. But tan θ is just the slope of the tangent
line at (t, x + ∆x). Thus

ρ∆xutt (t, x) = T (ux (t, x + ∆x) − ux (t, x))

Divide by ∆x and let ∆x → 0 to obtain

T
ρutt = Tuxx , utt = uxx .
ρ
This is the wave equation for a vibrating string.
Since it is second order in t you need two initial conditions, one on the velocity and the
other on the displacement in order to get a unique solution. However, other than this, the
procedure is essentially the same.

Example 34.2.4 Find the solution to the initial boundary value problem

utt = α 2 uxx , u (0,t) = u (2,t) = 0,

u (x, 0) = 1 − (1 − x)2
ut (x, 0) = 0

The eigenfunctions are solutions to

y′′ (x) + λ y (x) = 0, y (0) = 0 = y (2)

This is discussed in Example 33.2.1. The eigenfunctions are

 nπx 
sin
2
2 2
and the eigenvalues are λ = n 4π .
Then you look for a solution to the equation with boundary conditions of the form
 nπx 
a (t) sin
2
Thus you need
 nπx  n2 π 2  nπx 
a′′ (t) sin = −α 2 a (t) sin
2 4 2
Hence
n2 π 2
a′′ + α 2
a=0
4
and so, since you know the general solution to this equation, it is
 nπ   nπ 
a (t) = an cos α t + bn sin α t
2 2
It follows that the solution to the full problem will be of the form
∞   nπ   nπ   nπx 
u (x,t) = ∑ an cos α t + bn sin α t sin
n=1 2 2 2
34.2. HEAT AND WAVE EQUATIONS 677

Now you need to find an and bn to get the initial conditions. Letting t = 0, you need to have
∞  nπx 
1 − (1 − x)2 = ∑ an sin
n=1 2

and this is something which was done earlier. You need

16
an = (1 − (−1)n )
π 3 n3
Next, what about bn ? Differentiate both sides. Thus
∞   nπ   nπ   nπ   nπ   nπx 
ut (x,t) = ∑ an −α sin α t + bn α cos α t sin
n=1 2 2 2 2 2

Of course this operation is complete garbage because it involves the interchange of limit
operations without any justification. However, we do it anyway. In fact it is all right. You
can do the formal manipulations and then you can rigorously verify that what you end up
with really is a solution to the problem in some sense. Now plug in t = 0. Then you need
∞  nπ   mπx 
0= ∑ α 2 sin 2
b n
n=0

Clearly you should take bn = 0. Therefore, the desired solution is

∞  ! 
32 (2n − 1) π nπx 
u (x,t) = ∑ 3
cos α t sin
3
n=1 π (2n − 1) 2 2

Let’s let α 2 = .09. Then the specific solution is

∞  ! 
32 (2n − 1) π nπx 
u (x,t) = ∑ cos .3 t sin
n=1 π 3 (2n − 1)3 2 2

Note that from calculus, the series makes perfect sense because in fact, it converges
absolutely.

Example 34.2.5 Solve the initial boundary value problem

utt = α 2 uxx , u (0,t) = u (4,t) = 0,

u (x, 0) = f (x)
ut (x, 0) = 0

where (
1 − (x − 2)2 on [1, 3]
f (x) =
0 on the rest of [0, 4]

By similar reasoning to the above example,

∞   nπ   nπ   nπx 
u (x,t) = ∑ an cos α t + bn sin α t sin
n=1 4 4 4
678 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

Then, as above, bn = 0 and an must be chosen such that

∞  nπx 
f (x) = ∑ an sin
n=1 4
Thus
2 3   nπx 
Z
an = 1 − (x − 2)2 sin dx
4 1 4
Then after doing the hard work, you end up with
4 cos 43 nπ + nπ sin 34 nπ − 4 cos 14 nπ + nπ sin 41 nπ
an = −16
n3 π 3
Then the solution is
!
∞ 4 cos 34 nπ + nπ sin 34 nπ − 4 cos 41 nπ + nπ sin 14 nπ
u (x,t) = ∑ −16 ·
n=1 n3 π 3
 nπ   nπx 
cos α t sin
4 4
Let α = .5 to give a specific example. Here is a graph of the function of two variables
in which the sum is taken up to n = 6. The t axis goes from 0 to 10 and if you fix t and
imagine a cross section, it will be x → u(x,t).

-1
0
5 4
10 0 2

34.3 Nonhomogeneous Problems

For the sake of completeness, here is a brief discussion of what can be done if you have a
nonhomogeneous equation of the form ut = auxx + f along with an initial condition
u (x, 0) = g (x)
and boundary conditions. As before, there are eigenfunctions yn satisfying the boundary
conditions and
y′′n = −λ 2n yn , lim λ n = ∞
n→∞
such that also (
Z L
1 if n = m
yn (x) ym (x) dx = δ nm =
0 0 if n ̸= m
34.3. NONHOMOGENEOUS PROBLEMS 679

and it is assumed that one can obtain a valid Fourier series expansion in terms of these
eigenfunctions of all the functions of interest. Note how, for the sake of simplicity, it is
assumed that Z L
y2n (x) dx = 1
0
You multiply by an appropriate constant to make it this way. Thus, if the eigenfunctions
are multiples of sin nπ


L x , you choose the multiple to satisfy the above equation. Let
∞
f (x,t) = ∑ fn (t) yn (x)
n=0

Thus it is desired to have

∞ ∞ ∞
∑ b′n (t) yn (x) = −a ∑ λ n bn (t) yn (x) + ∑ fn (t) yn (x)
n=0 n=0 n=0

and this is achieved if

b′n (t) = −aλ n bn (t) + fn (t)
which is a familiar equation, the solution being
Z t
bn (t) = e−aλ n t bn (0) + e−aλ n (t−s) fn (s) ds
0

Then the solution is

 ∞ Z t 
−aλ n t −aλ n (t−s)
u (x,t) = ∑ e bn (0) + e fn (s) ds yn (x)
n=0 0

where bn (0) needs to be chosen to satisfy the initial condition. Thus it is required that
Z L
bn (0) = g (u) yn (u) du
0

In what was done earlier, yn was typically something like (2/L)1/2 sin nπx


L . Then the
solution is
∞  Z L  Zt 
u (x,t) = ∑ e−aλ n t g (u) yn (u) du + e−aλ n (t−s) fn (s) ds yn (x)
n=0 0 0

Example 34.3.1 Find the solution to

ut (x,t) = uxx (x,t) + f (x,t)

u (0,t) = 0 = u (2,t)
u (x, 0) = x

where f (x,t) = xt.

First find the eigenfunctions and eigenvalues for the equation

y′′ + λ y = 0, y (0) = 0 = y (2)

680 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

2
You must have λ strictly positive. The eigenvalues are λ = nπ 2 , and the eigenfunctions
are sin nπx


2 . Also, there is a Fourier series expansion for f (x,t) as follows.
∞  nπx 
f (x,t) = ∑ fn (t) sin
n=1 2
where Z 2
2  nπx 
fn (t) = f (x,t) sin dx
2 0 2
Thus Z 2  nπx  1  n+1

fn (t) = (xt) sin dx = 4πnt (−1)
0 2 π 2 n2
Now the solution is
 nπ 2 R  
e−( 2 ) t L nπu


∞
0 u sin 2 du +  nπx 
u (x,t) = ∑ R t −( nπ )2 (t−s)  1    sin
n=0 e 2 4πns (−1)n+1 ds 2
0 π 2 n2

Once you know how to solve this kind of problem, it becomes routine, if long, to find
solutions to problems like this.

Example 34.3.2 Find the solution to the initial-boundary value problem

ut (x,t) = uxx (x,t) + f (x,t)

u (0,t) = 0, u (L,t) = g (t)
u (x, 0) = h (x)

In this case, you massage the problem to get one which is like one you do know how to
do which involves zero boundary conditions. Let
x
w (x,t) = u (x,t) − g (t)
L
then
x x x
wt = ut − g′ (t) = uxx + f − g′ (t) = wxx + f (x,t) − g′ (t)
L L L
w (0,t) = u (0,t) = 0, w (L,t) = u (L,t) − g (t) = 0
x x
w (x, 0) = u (x, 0) − g (0) = h (x) − g (0)
L L
and now you solve for w using the above procedure. There are seemingly endless variations
of this but all amount to the following.

PROCEDURE 34.3.3 To solve

ut = Au + f

nonzero boundary conditions

initial condition u (x, 0) = l (x)
34.3. NONHOMOGENEOUS PROBLEMS 681

You let w = u−k (x,t) where k is a known function chosen such that the boundary conditions
on w involve w or its partial x derivatives set equal to 0. Then adjust to consider the
equation solved for w which is of the form

wt = Aw + fˆ

zero boundary conditions

modified initial condition w (x, 0) = lˆ(x)
This is then of the right form which can be solved. Obtain eigenfunctions {yn }

Ayn = −λ 2n yn

Find the eigenfunction expansion for f in terms of these.

∞
∑ fn (t) yn (x)
n=0

Then you need

∞
w (x,t) = ∑ bn (t) yn (x)
n=0

where
b′n (t) = −λ 2n bn (t) + fn (t)
and bn (0) is an appropriate Fourier coefficient chosen to satisfy the initial condition. Find
w and then u (x,t) = w (x,t) + k (x,t).

In case the problem is second order in time, you do something similar except that the
differential equation for bn will now be second order in time and you will need to adjust
both bn (0) and b′n (0) to achieve appropriate initial conditions.

Example 34.3.4 Solve the following

utt = uxx , u (x, 0) = 0, ut (x, 0) = 0

u (0,t) = 0, u (L,t) = sint

Initially the string is at rest and then something starts moving the right side up and
down. What happens?
Following the procedure, let
x
w (x,t) = u (x,t) − sin (t)
L
this works because w has zero boundary conditions. Then
x x x
wtt = utt + sint = uxx + sint = wxx + sint
L L L
w (0,t) = w (L,t) = 0
x
w (x, 0) = 0, wt (x, 0) = − cos (t)
L
682 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

nπ


The eigenfunctions are sin . Then the expansion for (x/L) sin (t) is
L x
∞  Z L   nπ    nπ 
2 x
∑ sint sin x dx sin x
n=1 L 0 L L L
!
∞
(−1)n+1  nπ 
= ∑ 2 sin (t) sin x
n=1 πn L

then the solution is

∞  nπ 
w (x,t) = ∑ bn (t) sin x
n=1 L
where
n2 π 2 (−1)n+1
b′′n (t) = −
2
bn (t) + 2 sin (t) (*)
L πn
The Fourier series expansion for wt (x,t) = − Lx cos (t) in terms of these eigenfunctions is
∞  Z L   nπx    nπx 
2 x
∑ − cos (t) sin dx sin
n=1 L 0 L L L
∞  n  
(−1) nπx 
= ∑ 2 cost sin
n=1 πn L

Now the solution to ∗ is bn (t) =

  π  1 L   π  ′ sint
cos nt bn (0) + sin nt bn (0) + 2 (−1)n+1 L2 3 3
L πn L π n − πL2 n
Clearly the initial condition for w gives bn (0) = 0. It remains to find b′n (0) . The solu-
tion is
∞   
1 L   π  ′ sint nπ 
w (x,t) = ∑ sin nt bn (0) + 2 (−1)n+1 L2 3 3 sin x
n=1 π n L π n − πL2 n L

Now
∞    
π  ′ n+1 2 cost nπ 
wt (x,t) = ∑ cos nt bn (0) + 2 (−1) L sin x
n=1 L π 3 n3 − πL2 n L

and so the initial condition for wt requires

!
n+1 2 ∞ 
(−1)n
∞  
′ 2 (−1) L  nπ  nπx 
∑ bn (0) + π 3 n3 − πL2 n sin L x = ∑ 2 πn sin L
n=1 n=1

and so
(−1)n 2 (−1)n+1 L2 n
b′n (0) = 2 − 3 3 = 2 (−1)n π 2 2
πn π n − πL2 n π n − L2
Thus
sin π nt
!
∞
2 (−1)n L π 2 n2L−L2  nπ 
w (x,t) = ∑ sin x
n=1 +2 (−1)n+1 L2 π 3 n3sint
−πL2 n
L

Therefore, u (x,t) = w (x,t) + Lx sin (t) .

34.4. LAPLACE EQUATION 683

34.4 Laplace Equation

The Laplace equation is ∆u = 0. In two dimensions and in rectangular coordinates,
∆u = uxx + uyy = 0
Here u is a function of the two variables x, y. Note first that ∆ is a linear operator. That is,
for a, b scalars and u, v functions,
∆ (au + bv) = a∆u + b∆v
Because of this, if you have several solutions to the Laplace equation u1 , · · · , um , and if you
have scalars ci , then !
n n n
∆ ∑ ci ui = ∑ ci ∆ui = ∑ ci 0 = 0.
i=1 i=1 i=1

It is understood that the point (x, y) is contained in some region in the plane. One looks
for a solution to the equation which also satisfies boundary conditions on the boundary of
the region. When these conditions involve given values for the function u it is called the
Dirichlet problem. When it involves giving values for the normal derivative of u defined by
∇u · n for n the unit outer normal, it is called a Neuman problem. In this short introduction
this region will be either a circular disk or a rectangle. These are called boundary value
problems.

34.4.1 Rectangles
First consider the rectangle. Here is a typical problem. The boundary conditions are as
shown in the picture, zero on the top bottom and left side and f (y) on the right.

0
(0, b) (a, b)
0 f (y)

(0, 0) 0 (a, 0)
You can solve this the usual way. Look for eigenfunctions. These need to correspond
to the two opposite sides where the boundary condition is 0. Thus the eigenfunctions are
the nonzero solutions to
f ′′ (y) + λ f (y) = 0, f (0) = 0 = f (b)
It follows the eigenfunctions are
 nπ 
sin y , n = 1, 2, · · ·
b
2
and the eigenvalues are πb2 n2 , n = 1, 2, · · · . Next you need to find some g (x) such that
g (x) sin nπ


b y solves the boundary conditions and the equation. The boundary conditions
are automatic. Now consider the equation. You need
π2 2  π 
 nπ   
′′
g (x) sin y + g (x) − 2 n sin ny =0
b b b
684 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

Thus, you need

π2 2
g′′ (x) − n g (x) = 0
b2
You know the solution is nπ x nπ x
C1 e b +C2 e− b

Now it turns out that in this application, it is much more convenient to write the general
solution as  nπ   nπ 
an cosh x + bn sinh x
b b
This gives the same general solution. The above functions are linear combinations of the
known solutions and so things in the above form are solutions. Furthermore, the ratio of
the two solutions is not constant so their Wronskian does not vanish. Hence it is the general
solution. Now you try and get the solution to the boundary value problem in the form
∞   nπ   nπ   nπ 
u (x, y) = ∑ an cosh x + bn sinh x sin y
n=1 b b b

nπ


when x = 0, you get ∑∞
n=1 an sin b y = 0 and so each an = 0. When x = a, you need
∞  nπ   nπ 
f (y) = ∑ bn sinh a sin y
n=1 b b

Hence you need

 nπ  2 Z b  nπ  2
Z b  nπ 
bn sinh a = f (t) sin t dtbn = f (t) sin t dt
b sinh nπ


b b 0 b b a 0 b

Therefore, with this formula for bn

∞  nπ   nπ 
u (x, y) = ∑ bn sinh x sin y
n=1 b b

This shows how to solve a more general problem in which you have functions given on
the edges. You solve the problem for the situation in which there is something nonzero on
exactly one edge with 0 on the others and then you add these solutions together.

Example 34.4.1 Find the solution to the boundary value problem

uxx + uyy = 0

where the boundary conditions and rectangle are as expressed in the following picture.

1 − (x − 1)2
(0, 1) (2, 1)
0
sin(πy)

(0, 0) 0 (2, 0)
34.4. LAPLACE EQUATION 685

First find the solution which has sin y on the right and zero on the other edges. This was
done in the above. It is
∞
u1 (x, y) = ∑ bn sinh (nπx) sin (nπy)
n=1

where bn is as given above. Thus

nπ sin 1 (−1)n
Z 1
2 2
bn = sin (t) sin (nπt) dt =
sinh (nπa) 0 sinh nπa n2 π 2 − 1

Hence this partial solution is

∞
2 nπ sin 1 (−1)n
u1 (x, y) = ∑ sinh nπa sinh (nπx) sin (nπy)
n=1 n2 π 2 − 1

Next find the solution to the equation which has 1 − (x − 1)2 on the top and zero on the
other sides. This is just like what was done earlier except that you would switch a and b.
You find the eigenfunctions for the two opposite zero boundary conditions. These are
 nπ 
sin x , n = 1, 2, · · ·
2
n2 π 2
with eigenvalues 4 . Next you look for solutions to the equation which involve
 nπ 
a (y) sin x
2
Thus
π2 2
 nπ   
′′ π
a (y) sin x + a (y) − n sin nx = 0
2 4 2
Hence
π2 2
a′′ (y) −
n a (y) = 0
4
and so  nπ   nπ 
a (y) = an cosh y + bn sinh y
2 2
Then the general solution is
∞   nπ   nπ   nπ 
u2 (x, y) = ∑ an cosh y + bn sinh y sin x
n=1 2 2 2

When y = 0, you are supposed to get 0 for the boundary condition. Hence an = 0. When
y = b you need
∞   nπ   nπ 
1 − (1 − x)2 = ∑ bn sinh sin x
n=1 2 2
Therefore, you need

2 − 2 (−1)n
 nπ  Z 2   nπ 
bn sinh = 1 − (1 − s)2 sin s ds = 8
2 0 2 n3 π 3
686 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

Then this solution is of the form

!
∞
2 − 2 (−1)n 1  nπ   nπ 
u2 (x, y) = ∑ 8 sinh y sin x
sinh nπ


n=1 n3 π 3 2
2 2

∞    
32 1 (2n − 1) πy (2n − 1) π
= ∑ (2n − 1)3 π 3   sinh sin x
n=1 sinh (2n−1)π 2 2
2

Therefore, the solution to the boundary value problem is the sum of these two solutions.
∞
2 nπ sin 1 (−1)n
u (x, y) = ∑ sinh nπa sinh (nπx) sin (nπy) +
n=1 n2 π 2 − 1
∞    
32 (2n − 1) πy
1 (2n − 1) π
∑ 3 3
  sinh sin x
n=1 (2n − 1) π sinh
(2n−1)π 2 2
2

You can probably see how to consider given functions in place of 0 on the remaining
two sides.

34.4.2 Circular Disks

This is more interesting than the above because it is more often the case that you encounter
it in real situations. Most pipes are circular for example. The Laplacian in rectangular
coordinates is
∆ = uxx + uyy
However, rectangular coordinates are not natural for considering circles. For example,
the boundaries of a rectangle are obtained by letting one of the variables be constant. If
you want something like this to happen for a circular shape, you should consider polar
coordinates. For example, the boundary of a circular disk is obtained by letting r = c a
constant. Recall the relation between polar and rectangular coordinates. θ ∈ [0, 2π), r > 0,

x = r cos θ
y = r sin θ (34.2)

You have a scalar field u and it is a function of a point in two dimensional space. This
point can be described in terms of either polar coordinates or rectangular coordinates. Thus

u (x, y) = u (r, θ )

there (x, y) and (r, θ ) pertain to the same point in two dimensions. As discussed above in
Section 34.1, the Laplacian in polar coordinates is

1 ∂ 2u
 
1 1 1 ∂ ∂u
urr + ur + 2 uθ θ = r + 2
r r r ∂r ∂r r ∂θ2

Example 34.4.2 Find the solution to

1 1
∆u = urr + ur + 2 uθ θ = 0
r r
34.4. LAPLACE EQUATION 687

on the disc of radius R if on the boundary of this disk,

u (R, θ ) = f (θ )

where f (0) = f (2π) . This last condition is necessary because θ = 0 and θ = 2π corre-
spond to the same point on the boundary of this disk. Note how everything is in terms of the
variables r, θ and that in terms of these variables, the circular disk is actually a rectangle.

Use the method of separation of variables. Look for a solution to the equation which is
of the form R (r) Θ (θ ) .

r2 R′′ (r) Θ (θ ) + rR′ (r) Θ (θ ) + R (r) Θ′′ (θ ) = 0

So divide by RΘ. This leads to

R′′ R′ Θ′′
r2 +r + =0
R R Θ
Hence
Θ′′ R′′ R′
= −λ = r2 + r
Θ R R
for some constant λ . First consider Θ. You must have Θ (0) = Θ (2π) . Also

Θ′′ + λ Θ = 0

Multiply both sides by Θ and integrate. This leads to

Z 2π
2 Z 2π
Θ′ (θ ) Θ (θ ) |2π
0 − Θ′ dθ + λ Θ2 dθ = 0
0 0

The boundary terms disappear because you must also have Θ′ (2π) = Θ′ (0). Therefore,
to have a solution, it is necessary that λ ≥ 0. If λ = 0, you need to have Θ′ = 0 and
so Θ (θ ) = C a constant. Otherwise, you need λ = µ 2 , µ > 0. Then the solution to the
equation is
C1 cos µθ +C2 sin µθ
and you need to have Θ (0) = Θ (2π). Therefore, it is required that µ2π is an integer
multiple of 2π so µ = n for n an integer. Thus the eigenvalues are the nonnegative integers
and you get
Θn (θ ) = (an cos (nθ ) + bn sin (nθ )) , n = 0, 1, 2, · · ·
It follows that for each of these n,

r2 R′′n + rR′n − n2 Rn = 0

This is an Euler equation and you look for solutions in the form R (r) = rα . Then to
find α, you insert this into the equation.

r2 α (α − 1) rα−2 + rαrα−1 − n2 rα = 0

and so you get the indicial equation

α (α − 1) + α − n2 = (α − n) (α + n) = 0
688 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

Therefore, the solutions are of the form

cn rn + dn r−n

We can immediately conclude that dn = 0 because it makes no sense to have the solution to
the differential equation be unbounded as r → 0. Recall the theorem from calculus that on a
closed and bounded set, a continuous function achieves its maximum and minimum. If u is
going to be continuous, which we certainly expect it to be, then this cannot be harmonized
with dn ̸= 0. Thus this has found many solutions to the partial differential equation which
are of the form
rn (an cos (nθ ) + bn sin (nθ ))
The solution to the equation will then be an infinite sum of the functions of the above
form. Thus combining the cn with an and bn ,
∞
u (r, θ ) = ∑ rn (an cos (nθ ) + bn sin (nθ ))
n=0

If you want to achieve the boundary condition, then you need to have Rn an =
Z 2π Z 2π Z 2π
1 1 1
cos (nθ ) f (θ ) dθ , a0 = f (θ ) dθ , Rn bn = sin (nθ ) f (θ ) dθ
π 0 2π 0 π 0

If you like, you can simplify this and write an interesting formula for the solution to
this problem.
1 2π
Z
u (r, θ ) = f (θ ) dθ +
2π 0
1 ∞ rn
 Z 2π  Z 2π  
∑ Rn cos (nα) f (α) dα cos (nθ ) + sin (nα) f (α) dα sin (nθ )
π n=1 0 0

1 ∞ rn
Z 2π Z 2π
1
= f (α) dα + ∑ Rn f (α) cos (n (θ − α)) dα
2π 0 π n=1 0

In fact, it can be proved that the infinite sum and the integral can be interchanged. This
is thanks to the term (r/R)n which yields absolute convergence. There is no problem if it
were a finite sum and thanks to this term, the tail of the series is negligible. Thus one can
reduce to the finite sum case and make the interchange. Thus the above implies
!
rn
Z 2π ∞
1 1
u (r, θ ) = + ∑ n cos (n (θ − α)) f (α) dα
0 π 2 n=1 R

You can find a formula for this.

 r it 
rn reit
∞ ∞   
r it n Re
∑ n cos (nt) = Re ∑ e = Re = Re
n=1 R n=1 R 1 − Rr eit R − reit

Rr cos (t) − r2 Rr cos (t) − r2

= 2
=
(R − r cost) + r2 sin (t) 2 R2 − 2 (cost) Rr + r2
Then
1 ∞
rn 1 Rr cos (t) − r2 1 R2 − r2
+ ∑ n cos (nt) = + 2 2
=
2 n=1 R 2 R − 2 (cost) Rr + r 2 R − 2 (cost) Rr + r2
2
34.5. EXERCISES 689

Thus
R2 − r2
Z 2π  
1
u (r, θ ) = f (α) dα
2π 0 R2 − 2 (cos (θ − α)) Rr + r2
Note that this shows that if r = 0 so you are at the center, then
Z 2π
1
u (r, θ ) = f (α) dα
2π 0

so the value at the center is the average of the boundary values. This proves the following
fundamental result.

Theorem 34.4.3 The solution to the problem

1 1
∆u = urr + ur + 2 uθ θ = 0
r r
on the disc of radius R where on the boundary of this disk,

u (R, θ ) = f (θ ) , f (0) = f (2π)

is given by the formula

R2 − r2
Z 2π  
1
u (r, θ ) = f (α) dα
2π 0 R2 − 2 (cos (θ − α)) Rr + r2

34.5 Exercises
1. Solve the following initial boundary value problems.

(a) ut = uxx , u (x, 0) = 1, u (0,t) = 0, u (4,t) = 0

(b) ut = 2uxx , u (x, 0) = 1, ux (0,t) = 0, u (3,t) = 0
(c) ut = 3uxx , u (x, 0) = 1 − x, u (0,t) = 0, u (2,t) = 0
(d) ut = 4uxx , u (x, 0) = 1 − x, u (0,t) = 0, u (2,t) = 0
(e) ut = 5uxx , u (x, 0) = 1 − x, u (0,t) = 0, ux (1,t) = 0
(f) ut = uxx , u (x, 0) = x + 1, ux (0,t) = 0, u (2,t) = 0
(g) ut = 3uxx , u (x, 0) = x, ux (0,t) = 0, u (1,t) = 0
(h) ut = 3uxx , u (x, 0) = x2 , u (0,t) = 0, u (5,t) = 0
(i) ut = 4uxx , u (x, 0) = 1, u (0,t) = 0, ux (1,t) = 0
(j) ut = uxx , u (x, 0) = x, u (0,t) = 0, ux (4,t) = 0
(k) ut = 2uxx , u (x, 0) = 1, ux (0,t) = 0, ux (5,t) = 0
(l) ut = 2uxx , u (x, 0) = x, ux (0,t) = 0, ux (4,t) = 0
(m) ut = 2uxx , u (x, 0) = 1 − x, ux (0,t) = 0, ux (3,t) = 0

2. Find the solution to the initial boundary value problem

utt = 3uxx , u (0,t) = 0, u (5,t) = 0,

u (x, 0) = 3x (x − 5) , ut (x, 0) = x2
690 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

3. Find the solution to the initial boundary value problem

utt = 4uxx , u (0,t) = 0, u (5,t) = 0,

u (x, 0) = 3x (x − 5) , ut (x, 0) = x + 1

4. Find the solution to the initial boundary value problem

utt = 4uxx , u (0,t) = 0, u (2,t) = 0,

u (x, 0) = −x (x − 2) , ut (x, 0) = x2

5. Describe how to solve the initial boundary value problem

utt + 2ut = 2uxx , u (0,t) = 0, u (5,t) = 0,

u (x, 0) = −x (x − 5) , ut (x, 0) = x2

Hint: You might consider defining w = e2t u and see what equation is solved by w.
6. Find the solution to the initial boundary value problem

ut − 2u = uxx , u (0,t) = 0, u (5,t) = 0,

u (x, 0) = x

Hint: It is like before. You get eigenfunctions and match coefficients.

7. Find the solution to the initial boundary value problem

ut = uxx + (cos x) , u (0,t) = 0, u (2,t) = 0,

u (x, 0) = 1 − x

8. Find the solution to the initial boundary value problem

ut = 2uxx + (x − 1) , u (0,t) = 0, u (4,t) = 0,

u (x, 0) = 1

9. Find the solution to the initial boundary value problem

ut = 5uxx + (x − 1) , u (0,t) = 0, u (1,t) = 0,

u (x, 0) = x + 1

10. Find the solution to the initial boundary value problem

ut = 2uxx , u (0,t) = 0, u (5,t) = 0,

x for x ∈ 0, 45
 
u (x, 0) =
0 if x ∈ ( 54 , 5]

11. Find the solution to the initial boundary value problem

ut = 3uxx , u (0,t) = 0, u (3,t) = 0,

1 for x ∈ [0, 1]
u (x, 0) =
0 if x ∈ (1, 3]
34.5. EXERCISES 691

12. Find the solution to the initial boundary value problem

ut = 4uxx , u (0,t) = 0, u (2,t) = 0,

x for x ∈ 0, 23
 
u (x, 0) =
1 − 12 x if x ∈ ( 23 , 2]

13. Find the solution to the initial boundary value problem

ut = 5uxx , u (0,t) = 0, u (1,t) = 0,

x for x ∈ 0, 12
 
u (x, 0) =
1 − x if x ∈ ( 21 , 1]

14. Find the solution to the initial boundary value problem

ut = 3uxx , u (0,t) = 0, u (5,t) = 0,

x for x ∈ 0, 52
 
u (x, 0) =
5 − x if x ∈ ( 25 , 5]

15. Find the solution to the initial boundary value problem

ut = 2uxx , ux (0,t) = 0, u (2,t) = 0,

x for x ∈ 0, 21
 
u (x, 0) = 2 1 1
3 − 3 x if x ∈ ( 2 , 2]

16. Find the solution to the initial boundary value problem

ut = 5uxx , ux (0,t) = 0, u (1,t) = 0,

x for x ∈ 0, 21
 
u (x, 0) =
1 − x if x ∈ ( 12 , 1]

17. Find the solution to the initial boundary value problem

ut = 5uxx , ux (0,t) = 0, u (2,t) = 0,

x for x ∈ 0, 32
 
u (x, 0) =
1 − 21 x if x ∈ ( 23 , 2]

18. Consider the following initial boundary value problem,

ut = uxx , u (0,t) = 0, u (2,t) + ux (2,t) = 0,

u (x, 0) = f (x)

Determine the appropriate equation for the eigenfunctions and show that there exists
a sequence of strictly positive eigenvalues converging to ∞. Also explain why the
solution u if it exists, must have a limit limt→∞ u (x,t) = w (x) and that this limit
satisfies w (x) = 0.
692 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS

19. Consider the following initial boundary value problem,

ut = uxx , ux (0,t) = 0, u (2,t) + ux (2,t) = 0,

u (x, 0) = f (x)

Determine the appropriate equation for the eigenfunctions and show that there exists
a sequence of strictly positive eigenvalues converging to ∞. Also explain why the
solution u if it exists, must have a limit limt→∞ u (x,t) = w (x) and that this limit
satisfies w′′ (x) = w (x) = 0.
20. Consider the following initial boundary value problem,

ut = uxx , ux (0,t) = 0, ux (2,t) = 0,

u (x, 0) = f (x)

Determine the appropriate equation for the eigenfunctions and show that there exists
a sequence of strictly positive eigenvalues converging to R∞. Also explain why the
solution u if it exists, must have a limit limt→∞ u (x,t) = 21 02 f (x) dx.
21. Recall that on the circular disk of radius R centered at the origin, denoted here as DR
!
rn
Z 2π ∞
1 1
u (r, θ ) = + ∑ n cos (n (θ − α)) f (α) dα
0 π 2 n=1 R

gave the solution to ∆u = 0 and f (α) a given function on the boundary where f (0) =
f (2π). Show, using the divergence theorem from calculus that there is at most one
smooth solution to this problem. Then explain why
!
rn
Z 2π ∞
1 1
+ ∑ n cos (n (θ − α)) dα = 1
0 π 2 n=1 R

22. Recall that on a simple computation was done which showed that
!
1 1 ∞
rn 1 R2 − r2
+ ∑ n cos (n (θ − α)) =
π 2 n=1 R 2π R2 − 2 (cos (θ − α)) Rr + r2

Therefore,
R2 − r2
Z 2π
1
dα = 1
0 2π R2 − 2 (cos (θ − α)) Rr + r2
Explain why it is also the case that

1 R2 − r 2
≥0
2π R2 − 2 (cos (θ − α)) Rr + r2

and if |θ − α| ≥ δ > 0, then

1 R2 − r2
lim =0
r→R− 2π R − 2 (cos (θ − α)) Rr + r 2
2

uniformly for such α.

34.5. EXERCISES 693

23. The solution to Laplace’s equation on the disk DR which has boundary values f (α)
was derived and it is
R2 − r2
Z 2π
1
u (r, θ ) = f (α) dα
0 2π R2 − 2 (cos (θ − α)) Rr + r2

Show that
lim u (r, θ ) = f (θ )
r→R−

This shows how the boundary values are obtained.

24. Recall that u (r, θ ) =

1 ∞ rn
Z 2π Z 2π

1
f (θ ) dθ + ∑ Rn cos (nα) f (α) dα cos (nθ )
2π 0 π n=1 0

2π
Z  
+ sin (nα) f (α) dα sin (nθ )
0

Explain why if f is a 2π periodic continuous function, it follows that there is a

trigonometric polynomial which is uniformly close to f (θ ) for θ ∈ [0, 2π]. Hint:
From the above problem, convergence to f (θ ) as r → R− takes place. Note that
from the argument, this actually happens uniformly thanks to the uniform continuity
of f . Now argue that the tail ∑∞n=N of the above series is uniformly small if N is
large.
25. Let (
x if x ∈ [0, 1]
f (x) =
2 − x if x ∈ [1, 2]
Solve the following initial boundary value problems

(a) ut = a2 uxx , u (x, 0) = f (x) , u (0,t) = 0 = u (2,t)

(b) ut = a2 uxx , u (x, 0) = f (x) , ux (0,t) = 0 = u (2,t)
(c) ut = a2 uxx , u (x, 0) = f (x) , ux (0,t) = 0 = ux (2,t)
694 CHAPTER 34. SOME PARTIAL DIFFERENTIAL EQUATIONS
 Part III

Fundamentals of Complex
Analysis

695
Chapter 35

Analytic Functions

This part of the book is on the fundamentals of complex analysis. I will not try to give
theorems in greatest possible generality. My intent is to give a fairly rigorous presentation
of those parts of the subject which have the most interesting applications. I think that
sometimes, when one tries to give the greatest generality and precision, the fundamental
ideas are obscured. These are often very simple ideas and it is too bad when they are lost.
Complex analysis is quite different than real analysis. It is relatively free of pathology and
often has a much more algebraic flavor than real analysis. I am trying to emphasize these
things, many of which are very important in both pure and applied math.
The fundamental theorems of Chapter 13 are going to be needed here.

35.1 Cauchy Riemann Equations

Of interest are functions f : U → C where U is an open subset of R2 and we consider R2
to equal C where the ordered pair (x, y) is written as x + iy. It is customary to write ∂U
to denote the boundary of the open set U. This means U \U whenever U is open and it is
useful to think of it as the edge of U. Thus ∂ B is a circle if B is an open ball. This will be
used whenever convenient.
As noted earlier in Section 2.3, the complex numbers forms a field. That is, it acts just
like the real numbers. There is a multiplication and addition which satisfy the usual prop-
erties which we think numbers should satisfy. Recall from calculus the familiar formula

f (z + h) − f (z)
lim ≡ f ′ (z)
h→0 h
When functions of many variables were encountered earlier, it was necessary to present
this in another way in terms of little o notation or more directly as

|f (x + v) − f (x) − Df (x) v|
lim =0
|v |→0 |v|

We had to do it this way because one cannot divide by a vector. However, in the case where
z ∈ C, no such worry is necessary. The familiar calculus formula can be used because
indeed, you can divide by a nonzero complex number. This leads to the concept of an
analytic function which will be presented in what follows. We will see that these are just

697
698 CHAPTER 35. ANALYTIC FUNCTIONS

like long polynomials. In fact, this is the correct context for the study of power series.
Then from calculus, the next thing considered is the rational functions. The generalization
of this simple concept will be the meromorphic functions. Remarkable things are obtained
from these simple considerations. Surprising applications are available when this theory is
developed. I will demonstrate that these extravagant assertions are abundantly verified.
We will be considering line integrals and it will be assumed that the curves over which
the line integrals are taken are piecewise C1 . Actually, all that is needed is that these
curves have finite length but this is better considered in a book devoted primarily to the
mathematical theory.

35.2 The Cauchy Riemann Equations

These fundamental equations pertain to a complex valued function of a complex variable.
Recall the complex numbers should be considered as points in the plane. Thus a complex
number is of the form x + iy where i2 = −1. The complex conjugate is defined by

x + iy ≡ x − iy

and for z a complex number,

p
|z| ≡ (zz)1/2 = x 2 + y2 .

Thus when x + iy is considered an ordered pair (x, y) ∈ R2 the magnitude of a complex

number is nothing more than the usual norm of the ordered pair. Also for z = x + iy, w =
u + iv, q
|z − w| = (x − u)2 + (y − v)2

so in terms of all topological considerations, R2 is the same as C. Thus to say z → f (z) is

continuous, is the same as saying

(x, y) → u (x, y) , (x, y) → v (x, y)

are continuous where f (z) ≡ u (x, y) + iv (x, y) with u and v being called the real and imag-
inary parts of f . The only new thing is that writing an ordered pair (x, y) as x + iy with
the convention i2 = −1 makes C into a field. You should verify that for z, w two complex
numbers, |zw| = |z| |w| . Also z + w = z + w.
Now here is the definition of what it means for a function to be analytic.

Definition 35.2.1 Let U be an open subset of C (R2 ) and let f : U → C be a function.

Then f is said to be analytic on U if for every z ∈ U,

f (z + ∆z) − f (z)
lim ≡ f ′ (z)
∆z→0 ∆z
exists and is a continuous function of z ∈ U. For a function having values in C denote by
u (x, y) the real part of f and v (x, y) the imaginary part. Both u and v have real values and

f (x + iy) ≡ f (z) ≡ u (x, y) + iv (x, y)

35.2. THE CAUCHY RIEMANN EQUATIONS 699

All of the usual methods and formulas for finding the derivative which were discussed
in calculus hold with no change for a function of a complex variable. That is, you have
the product rule, chain rule, and quotient rule with no change. Also the differentiation of
polynomials is the same. The proofs of these theorems are exactly the same as in calculus.
Thus I will use the standard methods with no comment whenever convenient. The new
thing is a relationship between the partial derivatives of the real and imaginary parts known
as the Cauchy Riemann equations.

Proposition 35.2.2 Let U be an open subset of C . Then f : U → C is analytic if and only

if for
f (x + iy) ≡ u (x, y) + iv (x, y)
u (x, y) , v (x, y) being the real and imaginary parts of f , it follows

ux (x, y) = vy (x, y) , uy (x, y) = −vx (x, y)

and all these partial derivatives, ux , uy , vx , vy are continuous on U. (The above equations
are called the Cauchy Riemann equations.)

Proof: First suppose f is analytic. First let ∆z = ih and take the limit of the difference
quotient as h → 0 in the definition. Thus from the definition,

f (z + ih) − f (z)
f ′ (z) ≡ lim
h→0 ih
u (x, y + h) + iv (x, y + h) − (u (x, y) + iv (x, y))
= lim
h→0 ih
1
= lim (uy (x, y) + ivy (x, y)) = −iuy (x, y) + vy (x, y)
h→0 i

Next let ∆z = h and take the limit of the difference quotient as h → 0.

f (z + h) − f (z)
f ′ (z) ≡ lim
h→0 h
u (x + h, y) + iv (x + h, y) − (u (x, y) + iv (x, y))
= lim
h→0 h
= ux (x, y) + ivx (x, y) .

Therefore, equating real and imaginary parts,

ux = vy , vx = −uy (35.1)

and this yields the Cauchy Riemann equations. Since z → f ′ (z) is continuous, it follows
the real and imaginary parts of this function must also be continuous. Thus from the above
formulas for f ′ (z) , it follows from the continuity of z → f ′ (z) all the partial derivatives of
the real and imaginary parts are continuous.
Next suppose the Cauchy Riemann equations hold and these partial derivatives are all
continuous. For ∆z = h + ik,

f (z + ∆z) − f (z) = u (x + h, y + k) + iv (x + h, y + k) − (u (x, y) + iv (x, y))

= ux (x, y) h + uy (x, y) k + i (vx (x, y) h + vy (x, y) k) + o ((h, k))

700 CHAPTER 35. ANALYTIC FUNCTIONS

= ux (x, y) h + uy (x, y) k + i (vx (x, y) h + vy (x, y) k) + o (∆z)

This follows since C1 implies differentiable along with the definition of the norm (absolute
value) in C. By the Cauchy Riemann equations this equals

= ux (x, y) h − vx (x, y) k + i (vx (x, y) h + ux (x, y) k) + o (∆z)

= ux (x, y) (h + ik) + ivx (x, y) (h + ik) + o (∆z)
= ux (x, y) ∆z + ivx (x, y) ∆z + o (∆z)

Dividing by ∆z and taking a limit yields f ′ (z) exists and equals ux (x, y) + ivx (x, y) which
are assumed to be continuous. ■
For functions of a real variable, it is perfectly possible for the derivative to exist and not
be continuous. For example, consider
(
x2 sin 1x if x ̸= 0


f (x) ≡
0 if x = 0

You can verify that f ′ (x) exists for all x but at 0 this derivative is not continuous. This
will NEVER happen with functions of a complex variable. This will be shown later when
it is more convenient. For now make continuity of f ′ part of the requirement for f to be
analytic.

35.3 Contour Integrals

In the theory of functions of a complex variable, the most important results are those in-
volving contour integration. The most important tools in complex analysis are Cauchy’s
theorem in some form and Cauchy’s formula for an analytic function. These are statements
about certain contour integrals. Now a contour integral is just a sort of line integral. In what
follows, γ ∗ or Γ will denote the set of points on a curve and γ will denote a parametrization
of the given curve. Here is the definition. It should look familiar and resemble a corre-
sponding definition for line integrals presented earlier. In fact, these contour integrals are
just line integrals.

Definition 35.3.1 Let γ : [a, b] → C, t ∈ [a, b] be a parametrization for a smooth oriented

curve Γ, the direction of motion being increasing t ∈ [a, b] and let f be a complex valued
function defined on Γ. Then
Z Z b
f (z) dz ≡ f (γ (t)) γ ′ (t) dt
γ a

For a piecewise smooth curve γ going from z1 to z2 to · · · zm , and for γ ∗z(k−1)k the curve
joining zk−1 to zk ,
Z m Z
f (z) dz ≡ ∑ f (z) dz
γ k=1 γ z(k−1)k

Example 35.3.2 Let γ (t) = cos (t) + i sin (t) ,t ∈ [0, 1] and let f (z) = z2 . Find
R
γ f (t) dz.
35.3. CONTOUR INTEGRALS 701

It equals
Z 1
(cos (t) + i sin (t))2 (− sin (t) + i cos (t)) dt
0
Z 1
i cos3 t − 3 cos2 t sint − 3i cost sin2 t + sin3 t dt


=
0 
1 1 1
= cos 3 − + i sin 3
3 3 3
As claimed above, every contour integral reduces to a line integral. Say z = x + iy and
f (z) = u (x, y) + iv (x, y) as above and γ (t) = x (t) + iy (t) ,t ∈ [a, b] . Then from the above
definition,
Z Z b
(u (x (t) , y (t)) + iv (x (t) , y (t))) x′ (t) + iy′ (t) dt


f (z) dz =
γ a

Z b
u (x (t) , y (t)) x′ (t) − v (x (t) , y (t)) y′ (t)


=
a
+i v (x (t) , y (t)) x′ (t) + u (x (t) , y (t)) y′ (t) dt

Z Z
≡ u (x, y) dx − v (x, y) dy + i v (x, y) dx + u (x, y) dy
Γ Γ
which is indeed, just the sum of two line integrals. Thus all the theory of line integrals
applies. In particular, the contour integral is dependent only on the smooth curves and their
orientation. This yields most of the following lemma.

Lemma 35.3.3 Let f be defined and continuous on a piecewise smooth oriented curve Γ
contained in C having parametrization γ. Let the real and imaginary parts of f be denoted
by u and v respectively. Then
Z Z Z
f (z) dz = udx − vdy + i vdx + udy
γ Γ Γ

Also the following estimate is available.

Z
f (z) dz ≤ max (| f (z)| : z ∈ γ ∗ ) (length of γ ∗ )
γ

If fn is continuous and

lim (sup (| fn (z) − f (z)| : z ∈ Γ)) = 0 (35.2)

n→∞

then Z Z
lim fn (z) dz = f (z) dz (35.3)
n→∞ γ γ
R
Proof: It only remains to verify the estimate. γ f (z) dz is some complex number I so
let (
I
|I| if I ̸= 0
ω=
1 if I = 0
702 CHAPTER 35. ANALYTIC FUNCTIONS

R R
Thus |ω| = 1 and ω γ f (z) dz = γ f (z) dz . Then letting γ be a parametrization for a
smooth curve,
Z Z Z b Z b
f (z) dz = ω f (z) dz = ω f (γ (t)) γ ′ (t) dt ≤ | f (γ (t))| γ ′ (t) dt
γ γ a a
Z b
≤ max (| f (z)| : z ∈ γ ∗ ) γ ′ (t) dt
a

Now recall that this last integral is the definition of the length of γ ∗ . If the curve Γ is
piecewise C1 composed of smooth curves γ i , Then
Z m Z m Z
f (z) dz ≡ ∑ f (z) dz ≤ ∑ f (z) dz
Γ j=1 γ j j=1 γj
m  
≤ ∑ max (| f (z)| : z ∈ Γ) length of γ ∗j
j=1
= max (| f (z)| : z ∈ Γ) (length of Γ)

Consider the last claim. From Theorem 13.6.3, z → f (z) is continuous. Therefore, the
integral makes sense. Also from the estimate,
Z Z Z
f (z) dz − fn (z) dz = ( f (z) − fn (z)) dz
Γ Γ Γ
≤ max (| f (z) − fn (z)| : z ∈ γ ∗ ) (length of γ ∗ )

and by assumption, this last expression converges to 0 as n → ∞. This shows 35.3. ■

Observation 35.3.4 In the case that γ ∗ = [a, b] an interval on the real line,R the above
definition of the contour integral shows that if γ is oriented from a to b, then γ f (z) dz =
Rb R Ra
a f (z) dz and if γ is oriented from b to a, then γ f (z) dz = b f (z) dz where the notation
on the right signifies the usual Riemann integral.

Definition 35.3.5 If one reverses the order in which points of γ ∗ are encountered, then one
replaces γ with −γ in which, for γ : [a, b] → C, −γ (t) encounters the points of γ ∗ in the
opposite order, the definition of the contour integral shows that
Z Z
− f (z) dz = f (z) dz
γ −γ

You could get a parametrization for −γ as −γ (t) ≡ γ (b − t) for t ∈ [0, b − a] or if you

wanted to use the same interval, define −γ : [a, b] → C by −γ (t) ≡ γ (b + a − t) . A simple
closed piecewise C1 curve is one which has the first point encountered equal to the last
point encountered by the parametrization. We will only consider closed curves for which
Green’s theorem applies to the curve and its inside which will be denoted as Ui .

One other technical result is often useful. It involves interchanging the order of contour
integrals.
Recall the mean value theorem for integrals from calculus.
35.3. CONTOUR INTEGRALS 703

Lemma 35.3.6 Let f : [a, b] → R be continuous. Then there exists c ∈ (a, b) such that
Z b
f (c) (b − a) = f (x) dx
a
Rx
Proof: Let F (x) ≡ a f (t) dt. Then by the mean value theorem,

F (b) − F (a) = F ′ (c) (b − a)

for some c ∈ (a, b). But F ′ (x) = f (x) and so this proves the lemma. ■

Lemma 35.3.7 Let γ, η be parametrizations for two smooth curves, γ ([a, b]) and η ([c, d])
and let f : γ ∗ × η ∗ → R be continuous. Then
Z Z Z Z
f (z, w) dzdw = f (z, w) dwdz
η γ γ η

In other words, you can switch the contour integrals.

Proof: That on the left is by definition,

Z Z Z dZ b
f (z, w) dzdw = f (γ (t) , η (s)) γ ′ (t) η ′ (s) dtds
η γ c a

Let P be a partition for [a, b] and Q a partition for [c, d] . Then the above is
n m Z si Z t j
∑∑ f (γ (t) , η (s)) γ ′ (t) η ′ (s) dtds
i=1 j=1 si−1 t j−1
n m Z si
f (γ (tˆj ) , η (s)) γ ′ (tˆj ) t j − t j−1 η ′ (s) ds


= ∑∑
i=1 j=1 si−1
n m
∑ ∑ f (γ (tˆj ) , η (ŝi )) γ ′ (tˆj ) η ′ (ŝi )


= t j − t j−1 (si − si−1 )
i=1 j=1

by an application

of the mean value theorem for integrals from calculus. Here (tˆj , ŝi ) ∈
t j−1 ,t j × (si−1 , si ). Similarly,
Z Z m n
∑ ∑ f (γ (t˜j ) , η (s̃i )) γ ′ (t˜j ) η ′ (s̃i )


f (z, w) dwdz = t j − t j−1 (si − si−1 )
γ η j=1 i=1
n m
∑ ∑ f (γ (t˜j ) , η (s̃i )) γ ′ (t˜j ) η ′ (s̃i )


= t j − t j−1 (si − si−1 )
i=1 j=1


where (t˜j , s̃i ) ∈ t j−1 ,t j × (si−1 , si ). By uniform continuity, if ∥P∥ , ∥Q∥ are small enough,
then

f (γ (t˜j ) , η (s̃i )) γ ′ (t˜j ) η ′ (s̃i ) − f (γ (tˆj ) , η (ŝi )) γ ′ (tˆj ) η ′ (ŝi ) < ε

and so Z Z Z Z
f (z, w) dzdw − f (z, w) dwdz < ε (b − a) (d − c) .
η γ γ η
Since ε is arbitrary, the two contour integrals must be equal. ■
704 CHAPTER 35. ANALYTIC FUNCTIONS

Theorem 35.3.8 Let γ, η be two piecewise smooth oriented curves. Then if the oriented
parametrizations that go with γ ∗ are respectively γ 1 , γ 2 , · · · , γ n and the oriented parametriza-
tions that go with η are respectively η 1 , η 2 , · · · , η m , then if f : γ ∗ × η ∗ → C is continuous,
Z Z Z Z
f (z, w) dwdz = f (z, w) dzdw
γ η η γ

Proof: First suppose f has values in R. Then, starting with the left and using Lemma
35.3.7,
Z Z n Z m Z n m Z Z
f (z, w) dwdz ≡ ∑ ∑ f (z, w) dwdz = ∑∑ f (z, w) dwdz
γ η k=1 γ k l=1 η l k=1 l=1 γ k η l
n m Z Z m n Z Z
= ∑∑ f (z, w) dwdz = ∑∑ f (z, w) dwdz
k=1 l=1 η l γk l=1 k=1 η l γk
Z Z
= f (z, w) dzdw
η γ

In the general case, you simply apply this to the real and imaginary parts of f . ■
The main result is the Cauchy integral theorem which is presented next. First recall
Green’s theorem.

Theorem 35.3.9 (Green’s Theorem) Let V be an open set in the plane and let its boundary
Γ be piecewise smooth and let F (x, y) = (P (x, y) , Q (x, y)) be a C1 vector field defined near
V. Then if Γ is oriented counter clockwise, it is often1 the case that
Z  
∂Q ∂P
Z
F · dR = (x, y) − (x, y) dm2 . (35.4)
Γ V ∂x ∂y
In particular, if there exists U such as the simple convex in both directions case considered

earlier for which Green’s theorem holds, and V = R (U) where R : U → V is C2 U, R2
such that Rx × Ry ̸= 0 and Rx × Ry is in the direction of k, then 35.4 is valid where
the orientation around Γ is consistent with the orientation around U. Also, one can paste
together regions for which Green’s theorem holds to get another one for which Green’s
theorem holds.

Here are some examples of regions for which Green’s theorem holds:

U U Γ U Γ
Γ Γ

Recall that you determine the positive orientation for use with Green’s theorem as fol-
lows. You regard k as pointing out of the paper because the x axis points to the right and the
1 For a general version see the advanced calculus book by Apostol. This is presented in the next section also.

The general versions involve the concept of a rectifiable Jordan curve. You need to be able to take the area integral
and to take the line integral around the boundary.
35.4. CAUCHY INTEGRAL THEOREM 705

y axis points up. Then the motion is such that if your head points in the direction of k, your
left hand will be over the surface if you walk in the direction of the positive orientation.
These examples work for Green’s theorem and if you have a C2 mapping defined near
these regions, then if the resulting curves around images of U are oriented consistent with
the above orientations, then you have another example of a region and its boundary for
which Green’s theorem holds.

35.4 Cauchy Integral Theorem

With the above preparation, here is the Cauchy integral theorem. It is really very simple. It
involves the Cauchy Riemann equations and Green’s theorem.

Theorem 35.4.1 Let U be an open set and suppose U and its boundary Γ satisfy Green’s
theorem where Γ is suitably oriented for using Green’s theorem. Suppose also that f is
analytic on an open set containing U ∪ Γ. Then
Z
f (z) dz = 0.
Γ

Proof: From Lemma 35.3.3, the contour integral is

Z Z
udx − vdy + i vdx + udy
Γ Γ

By Green’s theorem, this equals

Z Z
(−vx − uy ) dA + (ux − vy ) dA = 0
U U

thanks to the Cauchy Riemann equations. ■

Now this yields an easy way to check orientation of a piecewise smooth simple closed
curve. Consider the following picture of a simple closed curve in which there is a hole on
its inside denoted as Hr , its boundary being Cr as shown.

Ur
a+
• ib
Hr
Cr

Theorem 35.4.2 Let Γ be a simple closed curve in C and let z ∈ U, the inside component
of ΓC . Then for γ a parametrization of Γ,
1 1
Z
n (γ, z) ≡ dw = ±1
2πi γ w−z
depending on the orientation of Γ. If z ∈
/ U ∪ Γ, the integral equals 0. n (γ, z)is called the
winding number.
706 CHAPTER 35. ANALYTIC FUNCTIONS

Proof: Denote by Γr the insider circle in the above picture having radius r oriented as
1
shown. Then f (w) = w−z has a derivative which is

−1
(w − z)2
a continuous function, and so its real and imaginary parts are continuous for w ̸= z. There-
fore, the function is analytic near Ur the open set bounded by the two curves Γr and Γ.
It follows from the Cauchy theorem that for γ an orientation on Γ as shown and γ̂ r an
orientation as shown on Γr ,
1 1
Z Z
dw + dw = 0
γ w−z γ̂ r w−z
Therefore, orienting Γr in the usual direction, a parametrization for this circle is

x = a + r cost, y = b + r sint,t ∈ [0, 2π]

Deote this parametrization by γ r . Then

1 1
Z Z
dw = dw
γ w−z γr w−z
and using the definition of the contour integral, the right side reduces to 2πi. Thus the
winding number is 1. Therefore, if Γ were oriented the opposite direction, you would get
−1 for the winding number. If z ∈ / U ∪ Γ, the function is analytic near U and so the Cauchy
integral theorem impliesR right away that the winding number is 0. ■
1 1
The expression 2πi γ w−z dw ≡ n (γ, z) is called the winding number. As explained, it
is either 1 or −1 depending on how the curve Γ is oriented. The winding number can be
defined with much more generality for any closed curve, simple or not. However, I will not
do so, choosing instead to emphasize the most basic ideas. The greater generality is needed
however, when you consider general versions of the Cauchy integral formula, a marvelous
representation theorem for an analytic function.

Definition 35.4.3 Given Γ a simple closed curve, the orientation is said to be positive if
the winding number is 1 and negative if the winding number is −1.

35.5 Primitives and Cauchy Goursat Theorem

In beginning calculus, the notion of an antiderivative was very important. It is similar for
functions of complex variables. The role of a primitive is also a lot like a potential in
computing line integrals.

Definition 35.5.1 A function F such that F ′ = f is called a primitive of f .

The following theorem shows that the primitive acts just like a potential, the difference
being that a primitive has complex, not real values. In calculus, in the context of a function
of one real variable, this is often called an antiderivative and every continuous function has
one thanks to the fundamental theorem of calculus. However, it will be shown below that
the situation is not at all the same for functions of a complex variable.
So what if a function has a primitive? Say F ′ (z) = f (z) where f is continuous.
35.5. PRIMITIVES AND CAUCHY GOURSAT THEOREM 707

Theorem 35.5.2 Suppose γ is a piecewise C1 curve. Let its endpoints be p and q with the
orientation of the curve from p to q. Suppose f : γ ∗ → C is continuous and has a primitive
F. Thus F ′ (z) = f (z) for some open set Ω ⊇ γ ∗ . Then
Z
f (z) dz = F (q) − F (p)
γ

Proof: Assume first that γ is a C1 curve defined on an interval [a, b]. Then by definition,
Z b Z b
d
Z
f (z) dz = f (γ (t)) γ ′ (t) dt =
(F (γ (t))) dt
γ a dt a
= F (γ (b)) − F (γ (a)) = F (q) − F (p)

Now in the general case, you have

γ j : [a j , b j ] → C

is C1 and γ j (b j ) = γ j+1 a j+1 , j ≤ m. Then

Z m Z m     
f (z) dz ≡ ∑ f (z) dz = ∑ F γ j (b j ) − F γ j (a j )
γ j=1 γ j j=1

where γ m (bm ) = q and γ 1 (a1 ) = p.

= F (q) − F (γ m (am )) + F γ m−1 (bm−1 ) − F γ m−1 (am−1 )




+ F γ m−1 (bm−1 ) − F γ m−1 (am−1 ) + · · · +
+ (F (γ 1 (b1 )) − F (p))


By assumption, this reduces to F (q) − F (p) because γ j (b j ) = γ j+1 a j+1 for each j. ■
The Cauchy Goursat theorem is the next big result. This is a major theorem which
does not depend on the derivative being continuous. Thus it will also provide the needed
generalization which involves not assuming that z → f ′ (z) is continuous.
If you have two points in C, z1 and z2 , you can consider γ (t) ≡ z1 + t (z2 − z1 ) for
t ∈ [0, 1] to obtain a continuous bounded variation curve from z1 to z2 . More generally, if
z1 , · · · , zm are points in C you can obtain a continuous bounded variation curve from z1 to
zm which consists of first going from z1 to z2 and then from z2 to z3 and so on, till in the
end one goes from zm−1 to zm . Denote this piecewise linear curve as γ (z1 , · · · , zm ) . Now let
T be a triangle with vertices z1 , z2 and z3 encountered in the counter clockwise direction as
shown.

z3

z1 z2
R R
Denote by ∂T f (z) dz, the expression, γ(z1 ,z2 ,z3 ,z1 ) f (z) dz. Consider the following pic-
ture.
708 CHAPTER 35. ANALYTIC FUNCTIONS

z3

T11 T
T21
T31 T41
z1 z2
Thus
Z 4 Z
f (z) dz = ∑ f (z) dz. (35.5)
∂T 1
k=1 ∂ Tk

On the “inside lines” the integrals cancel because there are two integrals going in opposite
directions for each of these inside lines. Recall the method for evaluating a line integral
with a C1 parametrization.

Theorem 35.5.3 (Cauchy Goursat) Let f : Ω → X, where Ω is an open subset of C and X

is a complex complete normed linear space, have the property that f ′ (z) exists for all z ∈ Ω
and let T be a triangle contained in Ω. Then
Z
f (w) dw = 0.
∂T

Proof: Suppose not. Then

Z
f (w) dw = α ̸= 0.
∂T

From 35.5 it follows

4 Z
α≤ ∑ f (w) dw
k=1 ∂ Tk1

and so for at least one of these Tk1 , denoted from now on as T1 ,

Z
α
f (w) dw ≥ .
∂ T1 4

Now let T1 play the same role as T . Subdivide as in the above picture, and obtain T2 such
that Z
α
f (w) dw ≥ 2 .
∂ T2 4
Continue in this way, obtaining a sequence of triangles,

Tk ⊇ Tk+1 , diam (Tk ) ≤ diam (T ) 2−k ,

and Z
α
f (w) dw ≥ .
∂ Tk 4k
35.6. FUNCTIONS DIFFERENTIABLE ON A DISK, ZEROS 709

Then let z ∈ ∩∞ ′
k=1 Tk and note that by assumption, f (z) exists. Therefore, for all k large
enough, Z Z
f (z) + f ′ (z) (w − z) + g (w) dw


f (w) dw =
∂ Tk ∂ Tk

where |g (w)| < ε |w − z| . Now observe that w → f (z) + f ′ (z) (w − z) has a primitive,
namely,
F (w) = f (z) w + f ′ (z) (w − z)2 /2.
Therefore, by Theorem 35.5.2,
Z Z
f (w) dw = g (w) dw.
∂ Tk ∂ Tk

From Theorem 35.3.3,

Z
α
≤ g (w) dw ≤ εdiam (Tk ) (length of ∂ Tk )
4k ∂ Tk
−k
≤ ε2 (length of T ) diam (T ) 2−k ,

and so
α ≤ ε (length of T ) diam (T ) .
R
Since ε is arbitrary, this shows α = 0, a contradiction. Thus ∂ T f (w) dw = 0 as claimed.
■
Note that no assumption of continuity of z → f ′ (z) was needed.
Obviously, there is a version of the above Cauchy Goursat theorem which is valid for
a rectangle. Indeed, apply the Cauchy Goursat theorem for the triangles obtained from a
diagonal of the rectangle. The diagonal will be oriented two different ways depending on
which triangle it is a part of.

Corollary 35.5.4 Let Ω be an open set on which f ′ (z) exists. Then if R is a rectangle
contained in Ω along with its inside, then orienting R either way results in
Z
f (z) dz = 0.
R

The following is a general version of the Cauchy integral theorem. If f ′ (z) exists on
the inside and if f is continuous on the boundary, then the integral over the bounding curve
is 0. Note how the closed curve is arbitrary, not just a triangle.

35.6 Functions Differentiable on a Disk, Zeros

It turns out that if a function has a derivative, then it has all of them, in contrast to functions
of a real variable.
710 CHAPTER 35. ANALYTIC FUNCTIONS

Theorem 35.6.1 (Morera2 ) Let Ω be an open set and let f ′ (z) exist for all z ∈ Ω. Let
D ≡ B (z0 , r) ⊆ Ω. Then there exists ε > 0 such that f has a primitive on B (z0 , r + ε).
Proof: Choose ε > 0 small enough that B (z0 , r + ε) ⊆ Ω. Then for w ∈ B (z0 , r + ε) ,
define Z
F (w) ≡ f (u) du.
γ(z0 ,w)
Then by the Cauchy Goursat theorem, and w ∈ B (z0 , r + ε) , it follows that for |h| small
enough,
F (w + h) − F (w) 1
Z
= f (u) du
h h γ(w,w+h)
1 1 1
Z Z
= f (w + th) hdt = f (w + th) dt
h 0 0
which converges to f (w) due to the continuity of f at w. ■
Consider the following picture where you have a large circle of radius R and a small
circle of radius r centered at z, a point on the inside of γ R . The Cauchy integral formula
gives f (z) in terms of the values of f on the large circle.

γR

•z
γr
Γ2 • z0 Γ1

Theorem 35.6.2 Let γ R be a positively oriented circle of radius R and let U be its inside.
Suppose f has a derivative on an open set containing Ui ∪ γ ∗R . Then if z ∈ U,
1 f (w)
Z
f (z) = dw
2πi γR w−z
Proof: Use −γ r for the orientation of the smaller circle. Then from the Cauchy integral
theorem above, if w → g (w) is analytic,
Z Z Z Z
0= g (w) dw + g (w) dw = g (w) dw − g (w) dw
γR −γ r γR γr

This results from using −γ r on the small circle so the small circle has the opposite orienta-
tion indicated in the picture. Now let
f (w)
g (w) =
w−z
2 Giancinto Morera 1856-1909. This theorem or one like it dates from around 1886
35.6. FUNCTIONS DIFFERENTIABLE ON A DISK, ZEROS 711

This has derivative outside the small disk and inside some open set containing the large
disk. Also
f (w) f (w)
Z Z
dw = dw
γR w − z γr w − z

Now, since γ r is oriented positively,

1 f (w) 1 f (w) − f (z)

Z Z
dw − f (z) = dw (35.6)
2πi γr w−z 2πi γr w−z

Since f ′ (z) exists,

f (w) − f (z) f (z) + f ′ (z) (w − z) + o (w − z) − f (z)

=
w−z w−z
o (w − z)
= f ′ (z) +
w−z
Now f ′ (z) is a constant and so it has a primitive, namely w → f ′ (z) w. Thus f ′ (z) dw =
R
γr
0. It follows that if r is sufficiently small, then

1 f (w) − f (z) 1 1
Z
dw ≤ 2πrε = ε
2πi γr w−z 2π r

Thus, as r → 0, the right term in 35.6 converges to 0. It follows that

1 f (w) 1 f (w) − f (z)

Z Z
dw = lim dw + f (z) = f (z) ■
2πi γR w−z r→0 2πi γr w−z

This is the Cauchy integral formula for a disk. This remarkable formula is sufficient to
show that if a function has a derivative, then it has infinitely many and in fact, the function
can be represented as a power series. When this is shown, it will be easy to give the general
Cauchy integral formula for an arbitrary piecewise smooth simple closed curve. Let z0 be
the center of the large circle.
In the situation of Theorem 35.6.2,
1 f (w) 1 1 f (w)
Z Z
f (z) = dw = z−z0 dw
2πi γR w − z0 − (z − z0 ) 2πi γR w − z0 1 − w−z
0

z−z0 |z−z0 |
Now w−z0 = R < 1 for all w ∈ γ ∗R . Therefore, the above equals
!
1 ∞
f (w) (z − z0 )k 1 ∞
(z − z0 )k
Z Z
∑ dw = ∑ (w − z )k+1 f (w) dw
2πi γ R k=0 (w − z0 )k+1 2πi γR k=0 0

If the partial sums of the above series converge uniformly on γ ∗R then by Lemma 35.3.3,
!
1 ∞
(z − z0 )k
Z

2πi γ R k=0∑ (w − z )k+1 f (w) dw

0
!
1 p
(z − z0 )k
Z
= lim ∑ k+1
f (w) dw
p→∞ 2πi γ
R k=0 (w − z0 )
712 CHAPTER 35. ANALYTIC FUNCTIONS

1 p (z − z0 )k
Z
= lim ∑ f (w) dw (35.7)
p→∞ 2πi
k=0 γR (w − z0 )k+1
Which by definition is
!
∞
1 1
Z
∑ k+1
f (w) dw (z − z0 )k
k=0 2πi γR (w − z0 )

It is assumed that f is continuous on Ui ∪ γ ∗R . Thus there is an upper bound for | f (w)|,

called M thanks to the extreme value theorem. Then for w ∈ γ ∗R ,
k+1 k
f (w) (z − z0 )k
 
1 |z − z0 | M |z − z0 |
≤ M <
(w − z0 )k+1 |z − z0 | R R R

and |z − z0 | /R < 1 so the right side is summable. Therefore, by Theorem 13.8.3, conver-
gence is indeed uniform on γ ∗R and so
!
∞ ∞
1 1
Z
f (z) = ∑ k+1
f (w) dw (z − z0 )k ≡ ∑ ak (z − z0 )k
k=0 2πi γ R (w − z0 ) k=0

This proves part of the next theorem which says, among other things, that when f has
one derivative on the interior of a disk, then it must have all derivatives.

Theorem 35.6.3 Suppose z0 ∈ U, an open set in C and f : U → X has a derivative for

each z ∈ U. Then if B (z0 , R) ⊆ U, then for each z ∈ B (z0 , R) ,
∞
f (z) = ∑ an (z − z0 )n . (35.8)
n=0

where
1 1
Z
an ≡ f (w) dw
2πi γR (w − z0 )n+1
and γ R is a positively oriented parametrization for the circle bounding B (z0 , R). Then

f (k) (z0 ) = k!ak, (35.9)

lim sup |an |1/n |z − z0 | < 1, (35.10)

n→∞
∞
f (k) (z) = ∑ n (n − 1) · · · (n − k + 1) an (z − z0 )n−k , (35.11)
n=k

Proof: 35.8 follows from the above argument. Now consider 35.10. The above argu-
ment based on the Cauchy integral formula for a disk shows that if R > |ẑ − z0 | > |z − z0 | ,
then
∞
f (ẑ) = ∑ an (ẑ − z0 )n
n=0
and so, by the root test, Theorem 13.7.1,

1 ≥ lim sup |an |1/n |ẑ − z0 | > lim sup |an |1/n |z − z0 |
n→∞ n→∞
35.6. FUNCTIONS DIFFERENTIABLE ON A DISK, ZEROS 713

Consider 35.11 which involves identifying the an in terms of the derivatives of f . This is
obvious if k = 0. Suppose it is true for k. Then for small h ∈ C,
1  (k) 
f (z + h) − f (k) (z)
h
1 ∞  
= ∑ n (n − 1) · · · (n − k + 1) an (z + h − z0 )n−k − (z − z0 )n−k
h n=k
 ! 
∞ n−k n − k j (z − z )(n−k)− j
1  ∑ h 0
= ∑ n (n − 1) · · · (n − k + 1) an  j=0 j

h n=k+1

n−k
− (z − z0 )
! !
∞ n−k
n−k
= ∑ n (n − 1) · · · (n − k + 1) an ∑ h j−1 (z − z0 )(n−k)− j
n=k+1 j=1 j
∞
= ∑ n (n − 1) · · · (n − k + 1) (n − k) an (z − z0 )(n−k)−1
n=k+1
! !
∞ n−k
n−k j−2 (n−k)− j
+h ∑ n (n − 1) · · · (n − k + 1) an ∑ h (z − z0 ) (35.12)
n=k+1 j=2 j
By what was shown earlier,
 1/n
lim sup n (n − 1) · · · (n − k + 1) |an | |z − z0 |n−k
n→∞

= lim sup |an |1/n |z − z0 | < 1 (35.13)

n→∞
Consider the part of 35.12 which multiplies h. Does the infinite series converge? Yes it
does. In fact it converges absolutely.
! !
∞ n−k
n−k j−2 (n−k)− j
∑ n (n − 1) · · · (n − k + 1) an ∑ j
h (z − z0 )
n=k+1 j=2
!
∞ n−k
n − k |h| j−2
≤ ∑ n (n − 1) · · · (n − k + 1) |an | |z − z0 |(n−2)−k ∑
n=k+1 j=2 j |z − z0 | j−2
∞  n−k
|h|
≤ ∑ n (n − 1) · · · (n − k + 1) |an | |z − z0 |(n−2)−k 1 +
n=k+1 |z − z0 |
For all h small enough, this series converges, the infinite sum being decreasing in |h|.
Indeed,
 n−k !1/n
(n−2)−k |h|
lim sup n (n − 1) · · · (n − k + 1) |an | |z − z0 | 1+
n→∞ |z − z0 |
 
|h|
= lim sup |an |1/n |z − z0 | 1 + <1
n→∞ |z − z0 |
if |h| is small enough. Thus we can take a limit as h → 0 in 35.12 and conclude that
∞
f (k+1) (z) = ∑ n (n − 1) · · · (n − k + 1) (n − k) an (z − z0 )n−(k+1) ■
n=k+1
714 CHAPTER 35. ANALYTIC FUNCTIONS

Corollary 35.6.4 Suppose f is continuous on ∂ B (z0 , r) and suppose that for all z ∈ B (z0 , r) ,

1 f (w)
Z
f (z) = dw,
2πi γ w−z

where γ is the positively oriented boundary of the circular disk, conveniently given as
γ (t) ≡ z0 + reit ,t ∈ [0, 2π] . Then f is analytic on B (z0 , r) and in fact has infinitely many
derivatives on B (z0 , r) .

Proof: This is just a repeat of the above arguments. You show that f (z) is given by a
power series for |z − z0 | < r and from this, the result follows. ■
The following is very different than what is expected in real analysis. It says that
uniform convergence tends to take with it differentiability.

Lemma 35.6.5 Let γ (t) = z0 + reit , for t ∈ [0, 2π], suppose fn → f uniformly on B (z0 , r),
and suppose
1 fn (w)
Z
fn (z) = dw (35.14)
2πi γ w − z
for z ∈ B (z0 , r) . Then
1 f (w)
Z
f (z) = dw, (35.15)
2πi γ w−z
implying that f is analytic on B (z0 , r) .

Proof: From 35.14 and the uniform convergence of fn to f on γ ([0, 2π]) , the integrals
in 35.14 converge to
1 f (w)
Z
dw.
2πi γ w − z
Therefore, the formula 35.15 follows. ■
Because of the above result, from now on, the term analytic will be used interchange-
ably with “has a derivative”. This has shown that if the function has one derivative on an
open set, then it has all of them. Now here is another version of Morera’s theorem.

Corollary 35.6.6 Let Ω be an open set and suppose that whenever

γ (z1 , z2 , z3 , z1 )

is a closed curve bounding a triangle T, which is contained in Ω, and f is a continuous

function defined on Ω, it follows that
Z
f (z) dz = 0,
γ(z1 ,z2 ,z3 ,z1 )

then f is analytic on Ω.

Proof: As in the proof of Morera’s theorem, let B (z0 , r) ⊆ Ω and use the given condition
to construct a primitive, F for f on B (z0 , r) . Then F is analytic and so by Theorem 35.6.3,
it follows that F and hence f have infinitely many derivatives, implying that f is analytic
on B (z0 , r) . Since z0 is arbitrary, this shows f is analytic on Ω. ■
The following observation is useful to keep in mind.
35.6. FUNCTIONS DIFFERENTIABLE ON A DISK, ZEROS 715

Observation 35.6.7 Suppose ∑∞ n

n=0 an h converges for |h| < r. Then

1 ∞
lim ∑ an hn = 0
h→0 hk n=k+1

To see this, note the expression is h ∑∞ n−(k+1) . Now the sum of the absolute values
n=k+1 an h
∞ n−(k+1)
is ∑n=k+1 |an | |h| and it converges because there exists ĥ, such that r > ĥ > |h| and
by the root test, Theorem 13.7.1, lim supn→∞ |an |1/n ĥ ≤ 1 so

lim sup |an |1/n |h| < 1

n→∞

Now applying this to the sum in question,

n−(k+1)
lim sup |an |1/n |h| n = lim sup |an |1/n |h| < 1
n→∞ n→∞

Also the sum decreases in |h| and so

∞ ∞
lim h ∑ an hn−(k+1) ≤ lim |h| ∑ |an | |h|n−(k+1) = 0
h→0 n=k+1 h→0 n=k+1

The tail of the series just described is sometimes referred to as “higher order terms”.

The following is a remarkable result about the zeros of an analytic function on a con-
nected open set. It turns out that if the set of zeros have a limit point, then the function
ends up being zero. It is an illustration of how analytic functions are a lot like polynomials
which have finitely many zeros unless they are identically zero.

Definition 35.6.8 Suppose f is an analytic function defined near a point, α where f (α) =
0. Thus α is a zero of the function f . The zero is of order m if f (z) = (z − α)m g (z) where
g is an analytic function which is not equal to zero at α.

Theorem 35.6.9 Let Ω be a connected open set (region) and let f : Ω → C be analytic.
Then the following are equivalent.

1. f (z) = 0 for all z ∈ Ω

2. There exists z0 ∈ Ω such that f (n) (z0 ) = 0 for all n.
3. There exists z0 ∈ Ω which is a limit point of the set,

Z ≡ {z ∈ Ω : f (z) = 0} .

Proof: It is clear the first condition implies the second two.

Suppose the third holds. Then for z near z0
∞
f (n) (z0 )
f (z) = ∑ (z − z0 )n
n=k n!

where k ≥ 1 since z0 is a zero of f . Suppose k < ∞. Then,

f (z) = (z − z0 )k g (z)
716 CHAPTER 35. ANALYTIC FUNCTIONS

where g (z0 ) ̸= 0. Letting zn → z0 where zn ∈ Z, zn ̸= z0 , it follows

0 = (zn − z0 )k g (zn )

which implies g (zn ) = 0. Then by continuity of g, we see that g (z0 ) = 0 also, contrary to
the choice of k. Therefore, k cannot be less than ∞ and so z0 is a point satisfying the second
condition, all derivatives at z0 are zero.
Now suppose the second condition and let
n o
S ≡ z ∈ Ω : f (n) (z) = 0 for all n .

It is clear that S is a closed set which by assumption is nonempty. However, this set is also
open. To see this, let z ∈ S. Then for all w close enough to z,
∞
f (k) (z)
f (w) = ∑ (w − z)k = 0.
k=0 k!

Thus f is identically equal to zero near z ∈ S. Therefore, all points near z are contained
in S also, showing that S is an open set. Now Ω = S ∪ (Ω \ S) , the union of two disjoint
open sets, S being nonempty. It follows the other open set, Ω \ S, must be empty because
Ω is connected. Therefore, the first condition is verified. This proves the theorem. (See the
following diagram.)
1.)
↙↗ ↘
2.) ←− 3.)
Note how radically different this is from the theory of functions of a real variable.
Consider, for example the function
(
x2 sin 1x if x ̸= 0


f (x) ≡
0 if x = 0

which has a derivative for all x ∈ R and for which 0 is a limit point of the set Z, even though
f is not identically equal to zero.
Here is a very important application called Euler’s formula. Recall that

ez ≡ ex (cos (y) + i sin (y)) (35.16)

zk
Is it also true that ez = ∑∞
k=0 k! ?

Theorem 35.6.10 (Euler’s Formula) Let z = x + iy. Then

∞
zk
ez = ∑ k! .
k=0

Proof: The Cauchy Riemann equations show that ez given by 35.16 is analytic. So is
zk
exp (z) ≡ ∑∞k=0 k! . In fact the power series converges for all z ∈ C. Furthermore the two
functions, ez and exp (z) agree on the real line which is a set which contains a limit point.
Therefore, they agree for all values of z ∈ C. ■
35.7. LIOUVILLE’S THEOREM 717

This formula shows the famous two identities,

eiπ = −1 and e2πi = 1.

This properties of zeros of an analytic function can be used to verify with no effort that
identities which hold for z real continue to hold for z complex and this can be done with no
effort.

35.7 Liouville’s Theorem

Now the following is the general Cauchy integral formula.

Theorem 35.7.1 Let U along with its boundary Γ satisfy satisfy Green’s theorem and let
f be analytic on an open set V containing U ∪ Γ and let γ be an orientation of Γ such that
Green’s theorem holds. Thus,
1 1
Z
n (γ, z) ≡ dw = 1
2πi γ w−z
Then if z ∈ U,
1 f (w)
Z
f (z) = dw
2πi γ w−z
Proof: Consider the function
(
f (w)− f (z)
w−z if w ̸= z
g (w) ≡ ′
(35.17)
f (z) if w = z

It remains to consider whether g′ (z) exists for z ∈ V . Then from the Theorem 35.6.3, we
can write f (z + h) as a power series in h whenever h is suitably small.
f (z+h)− f (z)
h − f ′ (z)
=
h
   
1 1 1 1
f ′ (z) h + f ′′ (z) h2 + f ′′′ (z) h3 + · · · − f ′ (z)
h h 2! 3!
  
1 1 1
= f ′ (z) + f ′′ (z) h + f ′′′ (z) h2 + · · · − f ′ (z)
h 2! 3!
1 ′′ 1 ′′′
= f (z) + f (z) h + higher order terms
2! 3!
1 ′′
Thus the limit of the difference quotient exists and is 2! f (z). It follows that

1 1 f (w) 1 f (z)
Z Z Z
0 = g (w) dw = dw − dw
2πi γ 2πi γ w − z 2πi γ w−z
1 f (w)
Z
= dw − f (z) ■
2πi γ w − z
The following is a spectacular application. It is Liouville’s theorem.
718 CHAPTER 35. ANALYTIC FUNCTIONS

Theorem 35.7.2 Suppose f is analytic on C and that | f (z)| is bounded for z ∈ C. Then f
is constant.

Proof: It was shown above that if γ r is a counter clockwise oriented parametrization

for the circle of radius r centered at z, then

1 f (w)
Z
f ′ (z) = dw if |z| < r
2πi γr (w − z)2

and so
1 1
f ′ (z) ≤ C2πr 2
2π r
where | f (z)| < C for all z and this is true for any r so let r → ∞ and you can conclude that
f ′ (z) = 0 for all z ∈ C. However, this shows that f (k) (z) = 0 for all z and for each k ≥ 1.
Thus the power series for f (z) , which exists by Theorem 35.6.3, is
∞
f (k) (0) k
f (z) = f (0) + ∑ z = f (0) . ■
k=1 k!

This leads right away to the shortest proof of the fundamental theorem of algebra.

Theorem 35.7.3 Let p (z) be a non constant polynomial with complex coefficients. Then
p (z) = 0 for some z ∈ C. That is, p (z) has a root in C.

Proof: Suppose not. Then 1/p (z) is analytic on C. Also, the leading order term dom-
inates the others and so 1/p (z) must be bounded. Indeed, lim|z|→∞ (1/ |p (z)|) = 0 and the
continuous function z → 1/ |p (z)| achieves a maximum on any bounded ball centered at
0 by the extreme value theorem. By Liouville’s theorem, this quotient must be constant.
However, by assumption, this does not take place. Hence there is a root of p (z). ■

35.8 Riemann Sphere

I do not wish to emphasize the Riemann sphere in this book but some mention of it is
appropriate. Consider the unit sphere, S2 given by (z − 1)2 + y2 + x2 = 1. Define a map
from the complex plane to the surface of this sphere as follows. Extend a line from the
point, p in the complex plane to the point (0, 0, 2) on the top of this sphere and let θ (p)
denote the point of this sphere which the line intersects. Define θ (∞) ≡ (0, 0, 2).

(0, 0, 2)
•

• • θ (p)
(0, 0, 1)
p
• C
−1
Then θ is sometimes called stereographic projection. The mapping θ is clearly con-
tinuous because it takes converging sequences, to converging sequences. Furthermore, it is
clear that θ −1 is also continuous. In terms of the extended complex plane C,
b consisting of
35.9. EXERCISES 719

C along with a point called ∞, a sequence, zn converges to ∞ if and only if θ zn converges to

(0, 0, 2) if and only if |zn | converges to ∞ in the usual manner from calculus and a sequence,
zn converges to z ∈ C if and only if θ (zn ) → θ (z) . This is interesting because of this last
part. It gives a meaning for a sequence of complex numbers to converge to something called
∞. To do this properly, we should define a metric on C b and word everything in terms of
this metric. However, it amounts to the same thing as saying what it means for sequences to
converge. Then, with this definition of what it means for a sequence of complex numbers
to converge to ∞, the usual definition of connected sets and separated sets is identical with
what was given earlier.

Definition 35.8.1 Let S ⊆ C b the extended complex plane in which this extra point ∞ has
been included as just described. Then S is separated if there exist A, B not both empty such
that S = A ∪ B, A ∩ B = 0/ and no point of A is a limit of any sequence of points of B while
no point of B is the limit of any sequence of points of A. If S is not separated, then it is
called connected.

Example 35.8.2 Consider the open set S ≡ {z ∈ C such that Im (z) > 0} . Then S ∪ {∞} ≡
b.
Sb is connected in C

It is obvious that S is connected in C because it is arcwise connected. Suppose Sb

= A ∪ B where these two new sets separate Sb in C
b . Then one of them, say B must contain
∞. Therefore, A is bounded since otherwise there would be a sequence of points of A
converging to ∞ which is assumed not to happen. Then S = A ∪ (B \ {∞}) and A, B \ {∞}
would separate S unless one is empty. If B \ {∞} = 0,/ then S would be bounded which is
not the case. Hence A = 0.
/ Thus Sb is connected.

Definition 35.8.3 Let S ⊆ C. It is said to be simply connected if the set is connected and
C \ S ∪ {∞} is connected in C
b . Written more compactly, S is simply connected means S is
connected and also C \S is connected in C.
b b

When looking at

a set S in C, how do you determine whether it is simply connected?
You consider θ SC in S2 and ask whether it is connected with the convention that if SC is
unbounded, you must include (0, 0, 2) in the image of θ .

Example 35.8.4 Consider the set S ≡ {z ∈ C such that |z| > 1} . This is a connected set,
b \S is not connected. On S2 it consists of a piece
but it is not simply connected because C
near the bottom of the sphere and the point (0, 0, 2) at the top.

Example 35.8.5 Consider S ≡ {z ∈ C such that |z| ≤ 1} . This connected set is simply con-
b \S corresponds to a connected set on S2 .
nected because C

35.9 Exercises
In the following exercises, the term “simple closed curve” will be used repeatedly. Assume
that such curves Γ have an inside Ui and an outside and that Green’s theorem applies for
Ui with its boundary Γ if the boundary is oriented appropriately. This can be proved, but is
not in this book. It is one of these things which is mainly of mathematical interest. In the
examples of interest, it is typically not an issue.
720 CHAPTER 35. ANALYTIC FUNCTIONS

1. Suppose you have U ⊆ C an open set and f : U → C is analytic but has only real
values. Find all possible f with these properties.
2. Suppose f is an entire function (analytic on C) and suppose Re f is never 0. Show
that f must be constant. Hint: Consider U = {(x, y) : Re f (x, y) > 0} ,V = {(x, y) : Re f (x, y) < 0} .
These are open and disjoint so one must be empty. If V is empty, consider 1/e f (z) .
Use Liouville’s theorem.
3. Suppose f : C → C is analytic. Suppose also there is an estimate


| f (z)| ≤ M 1 + |z|α , α > 0

Show that f must be a polynomial. Hint: Consider the formula for the derivative in
which γ r is positively oriented and a circle or radius r for r very large centered at 0,
n! f (w)
Z
f (n) (z) =
2πi γr (w − z)n+1
and pick large n. Then let r → ∞.
n z 2n+1
4. Define for z ∈ C sin z ≡ ∑∞ k=0 (−1) (2n+1)! . That is, you just replace x with z. Give a
z
similar definition for cos z, and e . Show that the series converges for sin z and that a
corresponding series converges for cos z. Then show that

eiz − e−iz eiz + e−iz

sin z = , cos z =
2i 2
Show that it is not longer true that the functions sin z, cos z must be bounded in abso-
lute value by 1. Hint: This is a very easy problem if you use the theorem about the
zeros of an analytic function, Theorem 35.6.9.
5. Verify the identities cos (z − w) = cos z cos w + sin z sin w and similar identities. Hint:
This is a very easy problem if you use the theorem about the zeros of an analytic
function, Theorem 35.6.9.
6. Consider the following contour in which the large semicircle has radius R and the
small one has radius r ≡ 1/R.

x
iz
The function z → ez is analytic on the curve and on its inside. Therefore, the con-
tour integral with respect to the givenR orientation is 0. Use this contour and the
Cauchy integral theorem to verify that 0∞ sinz z dz = π/2 where this improper integral
is defined as Z R
sin z
lim dz
R→∞ −1/R z

The function is actually not absolutely integrable and so the precise description of
its meaning just given is important. To do this, show that the integral over the large
35.9. EXERCISES 721

−z
circle of CR e z dz → 0 as R → ∞ and verify that you get something else like −π for
R

the integral over the small integral as r → 0.

7. A set U is star shaped if there exists a single point z0 ∈ U such that every segment
from z0 to z is contained in U. Now suppose that U is star shaped and f : U → C is
analytic. Show that f has a primitive on U.
8. Let U be what remains of C after (−∞, 0] is deleted. Explain why UR is star shaped.
Letting γ (1, z) be the straight line segment from 1 to z, let f (z) = γ(1,z) w1 dw. Ex-
plain why f ′ (z) = 1z , f (1) = 0. Now explain why f is analytic and why e f (z) = z for
all z ∈ U. Also formulate an assertion which says f (ez ) = z for suitable z. This f is
the principal logarithm, denoted log (z).
9. Explain why one could delete any ray starting at 0 and obtain a function f (z) which
is a primitive of 1/z.
10. For z ∈ C \ (−∞, 0], let arg (z) ≡ θ ∈ (−π, π) such that z = |z| eiθ . Show that

log (z) = ln |z| + i arg (z) .

11. Suppose f (z) = u (x, y) + iv (x, u) is analytic. Show that both u, v satisfy Laplace’s
equation, uxx + uyy = 0.
12. Suppose you have two complex numbers z = a + ib and w = x + iy. Show that the
dot product of the two vectors (a, b) · (x, y) is Re ((a + ib) (x − iy)) = Re (zw̄) .
13. ↑Suppose you have two curves t → z (t) and s → w (s) which intersect at some point
z0 corresponding to t = t0 and s = s0 . Show that the cosine of the angle θ between
these two curves at this point is
 
Re z′ (t0 ) w′ (s0 )
cos (θ ) =
|z′ (t0 )| |w′ (s0 )|

Now suppose z → f (z) is analytic. Thus there are two curves t → f (z (t)) and s →
f (w (s)) which intersect when t = t0 and s = s0 . Show that the angle between these
two new curves at their point of intersection is also θ . This shows that analytic
mappings preserve the angles between curves.
14. Suppose z = x + iy and f (z) = u (x, y) + iv (x, y) where f is analytic. Explain why
level curves of u and v intersect in right angles.
15. Let Γ be a simple closed piecewise C1 curve in C. Let γ be a parametrization of
Γ which has positive orientation. Thus n (γ, z) = 1 for all z inside Γ. Also suppose
f is an analytic function on a connected open set containing Γ and its inside Ui .
Suppose f is not identically zero and has no zeros on Γ. Explain why f has finitely
many zeros on the inside of Γ. A zero a has multiplicity m if f (z) = (z − a)m g (z)
where g (z) ̸= 0 on Ui . Let the zeros of f in Ui be {a1 , · · · , am } where there might be
repeated numbers in this list, zeros of multiplicity higher than 1. Show that

1 f ′ (z)
Z
m= dz (35.18)
2πi γ f (z)
722 CHAPTER 35. ANALYTIC FUNCTIONS

Thus you can count the zeros of an analytic function inside a simple closed curve by
doing an integral! Hint: First of all, m is finite since if not, Theorem 35.6.9 implies
that f (z) = 0 for all z since there would be a limit point or else a zero of infinite
order. Now argue that f (z) = ∏m k=1 (z − ak ) g (z) where g (z) is analytic and nonzero
′ (z)
on Ui . Use the product rule to simplify ff (z) . Then use the fact that n (γ, z) = 1.

16. Suppose now you have a piecewise C1 simple closed curve Γ and on Γ∗ , | f (z)| >
|g (z)| where f , g are analytic on an open set containing Γ∗ . Suppose also that f has
no zeros on Γ∗ . In particular, f is not identically 0. Let λ ∈ [0, 1].

(a) Verify that for λ ∈ [0, 1] , f + λ g has no zeros on Γ∗ .

( f ′ (z)+λ g′ (z)) f ′ (z)+µg′ (z)
(b) Verify that on Γ∗ , f (z)+λ g(z) − f (z)+µg(z) ≤ C |µ − λ | .

(c) Use Theorem 35.3.3 to show that for γ a positively oriented parametrization of
Γ, Z ′
1 f (z) + λ g′ (z)
λ→ dz
2πi γ f (z) + λ g (z)
is continuous.
(d) Now explain why this shows that the number of zeros of f + λ g on the inside
of Γ is the same as the number of zeros of f on the inside of Γ. This is a version
of Rouche’s theorem.

17. Give an extremely easy proof of the fundamental theorem of algebra as follows. Let
γ R be a parametrization of the circle centered at 0 having radius R which has positive
orientation so n (γ, z) = 1. Let p (z) be a polynomial an zn + an−1 zn−1 + · · · + a1 z + a0 .
Now explain why you can choose R so large that |an zn | > an−1 zn−1 + · · · + a1 z + a0
for all |z| ≥ R. Using Problem 16 above explain why all zeros of p (z) are inside γ ∗R
and why there are exactly n of them counted according to multiplicity.
18. The polynomial z5 + z4 − z3 − 3z2 − 5z + 1 = p (z) has no rational roots. You can
check this by applying the rational root theorem from algebra. However, it has five
complex roots. Also

z4 − z3 − 3z2 − 5z + 1 ≤ |z|4 + |z|3 + 3 |z|2 + 5 |z| + 1

By graphing, observe that x5 − x4 + x3 + 3x2 + 5x + 1 > 0 for all x ≥ 2.4. Explain

why the roots of p (z) are inside the circle |z| = 2.4.
19. This problem will feature the situation where the radius of the simple closed curve is
sufficiently small. The zero counting integral can be used to prove an open mapping
theorem for analytic functions. Suppose you have f (z) = f (z0 ) + φ (z)m for z ∈ V
an open set containing z0 and φ (z0 ) = 0, |φ ′ (z0 )| = 2r ̸= 0, and m ∈ N. Let C (a, ρ)
denote the positively oriented circle centered at a which has radius ρ.

(a) Explain why there exists δ > 0 such that if |z − z0 | = δ , then B (z0 , δ ) ⊆ V and
φ (z)
≥ r, |φ (z)| ≥ r |z − z0 | = rδ
z − z0
Therefore, if |w| < rδ , then if |z − z0 | = δ , |φ (z) − w| ̸= 0.
35.9. EXERCISES 723

φ (z)′
1 R r
(b) Use continuity of w → 2πi C(z0 ,δ ) φ (z)−w dz for |w| < δ 2 and Problem 15 to con-
clude that there exists ε < rδ such that if |w| < ε there is one zero for φ (z) − w
in B (z0 , δ ). In other words, φ (B (z0 , δ )) ⊇ B (0, ε) . Then also φ m (B (z0 , δ )) ⊇
B (0, ε m ). Hint: If you have
 w∈B
m th
(0, ε ) , then there are m m roots of w
equally spaced around B 0, |w|1/m . Thus these roots are on a circle of ra-
dius less than ε. Pick one. Call it ŵ. Then there exists z ∈ B (z0 , δ ) such that
φ (z) = ŵ. Then φ m (z) = w. Fill in details.
(c) Explain why f (B (z0 , δ )) ⊇ f (z0 )+B (0, ε m ) and why for w ∈ f (z0 )+B (0, ε m )
there are m different points in B (z0 , δ ) , z1 , · · · , zm such that f (z j ) = w.
20. ↑Let Ω be an open connected set. Let f : Ω → C be analytic. Suppose f (Ω) is not
a single point. Then pick z0 ∈ Ω. Explain why f (z) = f (z0 ) + (z − z0 )m g (z) for all
z ∈ V an open ball contained in Ω which contains z0 and g (z) ̸= 0 in V, g (z) analytic.
If this were not so, then z0 would be a zero of infinite order and by the theorem on
zeros, Theorem 35.6.9, f (z) = f (z0 ) for all z ∈ Ω which is assumed not to happen.
Thus, every z0 in Ω has this property that near z0 , f (z) = f (z0 ) + (z − z0 )m g (z) for
nonzero g (z). Now explain why f (z) = f (z0 )+φ m (z) where φ (z0 ) = 0 but φ ′ (z0 ) ̸=
0 and φ (z) is some analytic function. Thus from Problem 19 above, there is δ such
that f (Ω) ⊇ f (z0 ) + B (0, ε m ). Hence f (Ω) is open since each f (z0 ) is an interior
point of f (Ω). You only need to show that there is G (z) such that G (z)m = g (z) and
then φ (z) ≡ (z − z0 ) G (z) will work fine. When you have done this, Problem 19 will
yield a proof of the open mapping theorem which says that if f is analytic on Ω a
connected open set, then f (Ω) is either an open set or a single point. So here are
some steps for doing this.
g′ (z)
(a) Consider z → g(z) . It is analytic on the open ball V and so it has a primitive on
R g′ (w)
V . In fact, you could take h (z) ≡ γ(z0 ,z) g(w) dw.
 ′
(b) Let the primitive be h (z) . Then consider g (z) e−h(z) . Show this equals 0.
Then explain why this requires it to be constant. Explain why there is a + ib
such that g (z) = eh(z)+a+ib . Then use the primitive h (z) + a + ib instead of the
original one. Call it h (z). Then
g (z) = eh(z)
You can then complete the argument by letting g (z)1/m ≡ eh(z)/m and
G (z) ≡ (z − z0 ) g (z)1/m
(c) Show that this theorem is certainly not true when considering functions of a
real variable by considering f (x) = x2 .
21. If you have an open set U in C show that for all z ∈ U, |z| < sup {|w| : w ∈ U}. In
other words, z → |z| never achieves its maximum on any open set U ∈ C.
22. Let f be analytic on U and let B (z, r) ⊆ U. Let γ r be the positively oriented boundary
of B (z, r). Explain, using the Cauchy integral formula why
| f (z)| ≤ max {| f (w)| : w ∈ γ ∗r } ≡ mr
Show that if equality is achieved, then | f (w)| must be constantly equal to mr on γ ∗r .
724 CHAPTER 35. ANALYTIC FUNCTIONS

23. The maximum modulus theorem says that if Ω is a bounded connected open set and
f : Ω → C is analytic and f : Ω → C is continuous, then if | f | achieves its maximum
at any point of Ω then f is equal to a constant on Ω. Thus | f | achieves its maximum
on the boundary of Ω in every case. Hint: Suppose the maximum is achieved at
a point of Ω, z0 . Then let B (z0 , r) ⊆ Ω. Show that if f is constant on B (z0 , r) ,
then it equals this constant on all of Ω using Theorem 35.6.9. However, if it is not
constant, then from the open mapping theorem of Problem 20, f (B (z0 , r)) is an open
set. Then use Problem 21 above to obtain a contradiction. Alternatively, use Problem
22 to verify that the set where | f | achieves its maximum is both open and closed.
24. Let f : C → C be analytic with f ′ (z) ̸=!0 for all z. Say f (x + iy) = u (x, y) + iv (x, y).
u (x, y)
Thus the mapping (x, y) → is a C1 mapping of R2 to R2 . Show that at
v (x, y)
any point
ux uy
̸= 0
vx vy
Therefore, by the inverse function theorem, Theorem 26.0.3, this mapping is locally
one to one. However, the function does not need to be globally one to one. Give an
easy example using the complex exponential which shows this to be the case.

25. Let Γ be a simple closed piecewise C1 curve and let { fn } be a sequence of functions
which are analytic near Ui ∪ Γ∗ . Then if γ is a parametrization of Γ with n (γ, z) = 1
for z ∈ Ui , then
1 fn (w)
Z
fn (z) = dw
2πi γ w − z
This is by the Cauchy integral formula presented above. Suppose fn converges uni-
formly on Γ∗ to a continuous function f . Show that then, for z ∈ Ui , and f (z) defined
as
1 f (w)
Z
f (z) ≡ dw
2πi γ w − z
It follows that fn (z) → f (z) for each z ∈ Ui and also f is analytic on Ui . Hint: You
might use Theorem 35.3.3. This is very different than what happens with functions
of a real variable in which uniform convergence of polynomials pn to f does not nec-
essarily confer differentiability on f . For example, to approximate f , a continuous
function having no derivatives or even a very easy function like f (x) = |x − (1/2)|
for x ∈ [0, 1].

26. The Schwarz lemma is as follows: Suppose F : B (0, 1) → B (0, 1) , F is analytic, and
F (0) = 0. Then for all z ∈ B (0, 1) ,

|F (z)| ≤ |z| , (35.19)

and
F ′ (0) ≤ 1. (35.20)
If equality holds in 35.20 then there exists λ ∈ C with |λ | = 1 and

F (z) = λ z. (35.21)
35.9. EXERCISES 725

Prove the Schwarz lemma. Hint: Since F has a power series of the form ∑∞ k
k=1 ak z ,
it follows that F (z) /z equals an analytic function g (z) for all z ∈ B (0, 1). By the
maximum modulus theorem, Problem 23 above, applied to g (z) , if |z| < r < 1,

F reit


F (z) 1
≤ max ≤ .
z t∈[0,2π] r r

Explain why this implies

F (z)
|g (z)| = ≤1
z

Now explain why limz→0 F(z) ′ ′

z = F (0) = g (0) and so |F (0)| ≤ 1. It only remains
to verify that if |F (0)| = 1, then F (z) is just a rotation as described. If |F ′ (0)| = 1,
′

then the analytic function g (z) has the property that it achieves its maximum at an
interior point. Apply Problem 23 to conclude that g (z) must be a constant. Explain
F(z)
why this requires z = 1 for all z. Use this to conclude the proof.

27. Sketch an example of two differentiable functions defined on [0, 1] such that their
product is 0 but neither function is 0. Explain why this never happens for the set
of analytic functions defined on an open connected set. In other words, if you have
f g = 0 where f , g are analytic on D an open connected set, then either f = 0 or
g = 0. For those who like to classify algebraically, this says that the set of analytic
functions defined on an open connected set is an integral domain. It is clear that
this set of functions is a ring with the usual operations. The extra ingredient is this
observation that there are no nonzero zero divisors. Hint: To show this, consider
D \ f −1 (0) an open set. If f −1 (0) = D, then you are done. Otherwise, you have g is
0 on an open set. Now use Theorem 35.6.9.
1


28. For D ≡ {z ∈ C : |z| < 1} , consider the function sin 1−z . Show that this function
has infinitely many zeros in D. Thus there is a limit point to the set of zeros, but its
limit point is not in D. It is good to keep this example in mind when considering
Theorem 35.6.9.
726 CHAPTER 35. ANALYTIC FUNCTIONS
Chapter 36

Isolated Singularities and

Analytic Functions

36.1 Open Mapping Theorem for Complex Valued Func-

tions
The open mapping theorem is for an analytic function with values in C. It is even more sur-
prising result than the theorem about the zeros of an analytic function. The following proof
of this important theorem uses an interesting local representation of the analytic function.

Theorem 36.1.1 (Open mapping theorem) Let Ω be a region in C and suppose f : Ω → C

is analytic. Then f (Ω) is either a point or a region. In the case where f (Ω) is a region, it
follows that for each z0 ∈ Ω, there exists an open set V containing z0 and m ∈ N such that
for all z ∈ V,
f (z) = f (z0 ) + φ (z)m (36.1)
where φ : V → B (0, δ ) is one to one, analytic and onto, φ (z0 ) = 0, φ ′ (z) ̸= 0 on V and
φ −1 analytic on B (0, δ ) . If f is one to one then m = 1 for each z0 and f −1 : f (Ω) → Ω is
analytic.

Proof: Suppose f (Ω) is not a point. Then if z0 ∈ Ω it follows there exists r > 0 such
that f (z) ̸= f (z0 ) for all z ∈ B (z0 , r) \ {z0 } . Otherwise, z0 would be a limit point of the set,

{z ∈ Ω : f (z) − f (z0 ) = 0}

which would imply from Theorem 35.6.9 that f (z) = f (z0 ) for all z ∈ Ω. Therefore, mak-
ing r smaller if necessary and using the power series of f ,
?  m
f (z) = f (z0 ) + (z − z0 )m g (z) (= f (z0 ) + (z − z0 ) g (z)1/m )

for all z ∈ B (z0 , r) , where g (z) ̸= 0 on B (z0 , r) . As implied in the above formula, one
wonders if you can take the mth root of g (z) .

727
728 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

g′
gis an analytic function on B (z0 , r) and so by Morera’s theorem, Theorem 35.6.1, it
has a primitive on B (z0 , r) called h. Therefore by the product rule and the chain rule,
 ′    
ge−h = g′ e−h + g −e−h h′
    g′
= g′ e−h + g −e−h =0
g

and so there exists a constant, C = ea+ib such that on B (z0 , r) ,

ge−h = ea+ib .

Therefore,
g (z) = eh(z)+a+ib
and so, modifying h by adding in the constant, a + ib it is still a primitive of g′ /g and now
′ (z)
g (z) = eh(z) where h′ (z) = gg(z) on B (z0 , r) . Letting

h(z)
φ (z) = (z − z0 ) e m

implies formula 36.1 is valid on B (z0 , r) . Now φ (z0 ) = 0 but

h(z0 )
φ ′ (z0 ) = e m ̸= 0.

Shrinking r if necessary you can assume φ ′ (z) ̸= 0 on B (z0 , r). Is there an open set V
contained in B (z0 , r) such that φ maps V onto B (0, δ ) for some δ > 0?
Let φ (z) = u (x, y) + iv (x, y) where z = x + iy. Consider the mapping
! !
x u (x, y)
→
y v (x, y)

where u, v are C1 because φ is given to be analytic. The Jacobian of this map at (x, y) ∈
B (z0 , r) is
ux (x, y) uy (x, y) ux (x, y) −vx (x, y)
=
vx (x, y) vy (x, y) vx (x, y) ux (x, y)
2
= ux (x, y)2 + vx (x, y)2 = φ ′ (z) ̸= 0.
This follows from a use of the Cauchy Riemann equations. Also
! !
u (x0 , y0 ) 0
=
v (x0 , y0 ) 0

Therefore, by the inverse function theorem there exists an open set V, containing z0 and
δ > 0 such that (u, v)T maps V one to one onto B (0, δ ) . Thus φ is one to one onto B (0, δ )
as claimed. Applying the same argument to other points z of V and using the fact that
φ ′ (z) ̸= 0 at these points, it follows φ maps open sets to open sets. In other words, φ −1 is
continuous.
36.1. OPEN MAPPING THEOREM FOR COMPLEX VALUED FUNCTIONS 729

It also follows that φ m maps V onto B (0, δ m ) . Indeed,

|φ (z)|m = |φ (z)m | .
Therefore, the formula 36.1 implies that f maps the open set V, containing z0 to an open
set. This shows f (Ω) is an open set because z0 was arbitrary. It is connected because f is
continuous and Ω is connected. Thus f (Ω) is a region (open and connected). It remains to
verify that φ −1 is analytic on B (0, δ ) . Since φ −1 is continuous,
φ −1 (φ (z1 )) − φ −1 (φ (z)) z1 − z 1
lim = lim = ′ .
φ (z1 )→φ (z) φ (z1 ) − φ (z) z 1 →z φ (z1 ) − φ (z) φ (z)

Therefore, φ −1 is analytic as claimed.

It only remains to verify the assertion about the case where f is one to one. If m > 1,
2πi
then e m ̸= 1 and so for z1 ∈ V,
2πi
e m φ (z1 ) ̸= φ (z1 ) . (36.2)
2πi
But e m φ (z1 ) ∈ B (0, δ ) and so there exists z2 ̸= z1 (since φ is one to one) such that φ (z2 ) =
2πi
e m φ (z1 ) . But then
 2πi m
φ (z2 )m = e m φ (z1 ) = e2πi φ (z1 )m = φ (z1 )m

implying f (z2 ) = f (z1 ) contradicting the assumption that f is one to one. Thus m = 1
and f ′ (z) = φ ′ (z) ̸= 0 on V. Since f maps open sets to open sets, it follows that f −1 is
continuous and so

′ f −1 ( f (z1 )) − f −1 ( f (z))
f −1 ( f (z)) = lim
f (z1 )→ f (z) f (z1 ) − f (z)
z1 − z 1
= lim = ′ .■
z1 →z f (z1 ) − f (z) f (z)
You can dispense with the appeal to the inverse function theorem by using Problem 19 on
Page 722.
One does not have to look very far to find that this sort of thing does not hold for
functions mapping R to R. Take for example, the function f (x) = x2 . Then f (R) is neither
a point nor a region. In fact f (R) fails to be open.
Corollary 36.1.2 Suppose in the situation of Theorem 36.1.1 m > 1 for the local represen-
tation of f given in this theorem. Then there exists δ > 0 such that if w ∈ B ( f (z0 ) , δ ) =
f (V ) for V an open set containing z0 , then f −1 (w) consists of m distinct points in V. ( f is
m to one on V )
Proof: Let w ∈ B ( f (z0 ) , δ ) . n
Then w = fo(b z ∈ V. Thus f (b
z) where b z)m .
z) = f (z0 ) + φ (b
2kπi m
Consider the m distinct numbers, e m φ (b z) . Then each of these numbers is in B (0, δ )
k=1
and so since φ maps V one to one onto B (0, δ ) , there are m distinct numbers in V , {zk }m
k=1
2kπi
such that φ (zk ) = e m φ (b
z). Then
 2kπi m
f (zk ) = f (z0 ) + φ (zk )m = f (z0 ) + e m φ (bz)
= z)m = f (z0 ) + φ (b
f (z0 ) + e2kπi φ (b z)m = f (b
z) = w ■
730 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

Example 36.1.3 Consider the open connected set D ≡ R + i (a − π, a + π) . Then z → ez is

one to one and analytic on D. It maps D onto C \ l where l is the ray starting from 0 whose
angle is a. Therefore, it has an analytic inverse defined on C \ l. This is a branch of the
logarithm. It is of the form
log (z) = ln |z| + i arga (z)
where arga (z) is the angle in (a − π, a + π) with the property that

eln|z|+i arga (z) = z

We usually let a = 0 and then the inverse is what is usually called the logarithm and is
denoted by log . As in Problem 10 this is ln (|z|) + i arg (z) where arg (z) is the angle between
−π and π corresponding to z ∈ C \ (−∞, 0].

With the open mapping theorem, the maximum modulus theorem is fairly easy.

Theorem 36.1.4 Let Ω be an open connected, bounded set in C and let f : Ω → C be

analytic. Let ∂ Ω ≡ Ω̄ \ Ω. Then

max | f (z)| : z ∈ Ω̄ = max {| f (z)| : z ∈ ∂ Ω}

and if the maximum of | f (z)| is achieved at a point of Ω, then f is a constant.

Proof: Suppose f (Ω) is not a single point. That is, f is not constant. Then by the
open mapping theorem, f (Ω) is an open connected subset of C and so z → | f (z)| has no
maximum. Therefore, the maximum of | f (z)| for z ∈ Ω̄ is on ∂ Ω. If f (Ω) is a single point,
then the equation still holds. ■

36.2 Functions Analytic on an Annulus

First consider the definition of an annulus.

Definition 36.2.1 Define ann (a, r, R) ≡ {z : r < |z − a| < R} .

Thus ann (a, 0, R) would denote the punctured ball, B (a, R) \ {a} and when r > 0, the
annulus looks like the following.

• a

The annulus consists of the points between the two circles.

In the following picture, let there be two parametrizations γ R for the large circle and γ̂ r
for the small one with orientation as shown. There are also two line segments oriented as
shown which miss z ∈ ann (z0 , r, R) and constitute the intersection of the two simple closed
curves Γ1 , Γ2 . These two simple closed curves are oriented as shown. Thus each of Γi is
positively oriented. Let f be analytic near ann (z0 , r, R).
36.2. FUNCTIONS ANALYTIC ON AN ANNULUS 731

γR

γ̂ r •z
Γ2 • z0 Γ1

It follows from Theorem 35.7.1, that for z in the annulus,

1 f (w) 1 f (w)
Z Z
dw + dw = f (z)
2πi γR w−z 2πi γ̂ r w−z

This is because the contributions to the line integrals along those straight lines is 0 since
they cancel off because of opposite orientations. Let γ r be the opposite orientation from γ̂ r .
Then this reduces to
f (w) f (w)
Z Z
dw − dw = 2πi f (z)
γR w − z γr w − z

Thus Z 
1 f (w) f (w)
Z
f (z) = dw + dw
2πi γR w − z0 − (z − z0 )
γ r (z − z0 ) − (w − z0 )
"Z #
1 1 f (w) 1 f (w)
Z
= z−z0 dw + w−z dw
2πi γ R w − z0 1 − w−z γ r z − z0 1 − z−z 0
0 0

z−z0
Now note that for z in the annulus between the two circles and w ∈ γ ∗R , w−z0 < 1, and for
w−z0
w ∈ γ ∗r , z−z0 < 1. In fact, in each case, there is b < 1 such that

z − z0 w − z0
w ∈ γ ∗R , < b < 1, w ∈ γ ∗r , <b<1 (36.3)
w − z0 z − z0

Thus you can use the formula for the sum of an infinite geometric series and conclude
 R  n 
1 ∞ z−z0
1  γR f (w) ∑
w−z0 n=0 w−z0 dw
f (z) = 
w−z0
n 
2πi + f (w) 1 ∑∞
R
γr (z−z0 ) dw n=0 z−z0

Then from the uniform estimates of 36.3, one can conclude uniform convergence of the
partial sums for w ∈ γ ∗R or γ ∗r , and so by the Weierstrass M test, Theorem 13.8.3, one can
732 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

interchange the summation with the integral and write

!
∞
1 1
Z
f (z) = ∑ f (w) n+1
dw (z − z0 )n
n=0 2πi γ R (w − z0 )
∞  
1 1
Z
n
+∑ f (w) (w − z0 ) dw ,
n=0 2πi γr (z − z0 )n+1

both series converging absolutely. Thus there are an , bn ∈ X such that

∞ ∞
f (z) = ∑ an (z − z0 )n + ∑ bn (z − z0 )−n
n=0 n=1

This proves most of the following theorem.

Theorem 36.2.2 Let z ∈ ann (z0 , r, R) and let f : ann (z0 , r, R) → X be analytic near ann (z0 , r, R).
Then for any z ∈ ann (z0 , r, R) ,
∞ ∞
f (z) = ∑ an (z − z0 )n + ∑ bn (z − z0 )−n (36.4)
n=0 n=1

where
1 1
Z
an = f (w) dw
2πi γR (w − z0 )n+1
1
Z
bn = f (w) (w − z0 )n−1 dw
2πi γr

and both of these series in 36.4 converge absolutely. If
r < r̂ < R̂ < R, then convergence of
both series is absolute and uniform for z ∈ ann z0 , r̂, R̂ .

Proof: Consider the sum with the negative exponents. The other is similar. Let | f (w)| ≤
M on the closure of the annulus.
∞  
1
Z
−n n
∑ bn (z − z0 ) , bn = 2πi γ f (w) (w − z0 ) dw
n=1 r

Therefore, ∥bn ∥ ≤ 2πrMrn and |z − z0 | ≥ r̂ > r. Thus

q q
rn
∑ ∥bn ∥ |z − z0 |−n ≤ ∑ 2π r̂M r̂n < ε
n=p n=p

if p is large enough. Therefore, the partial sums are

 a uniformly Cauchy sequence so the
sum converges absolutely and uniformly on the set z : r̂ ≤ |z − z0 | ≤ R̂ . ■
Note that for arbitrary α with r ≤ α ≤ R,
1 1 1 1
Z Z
f (w) n+1
dw = f (w) dw (36.5)
2πi γR (w − z0 ) 2πi γα (w − z0 )n+1
This is a simple application of the Cauchy integral theorem applied to the union of two
simple closed curves of the sort used to prove Theorem 36.2.2. You consider the annulus
ann (z0 , α, R) and the following diagram.
36.2. FUNCTIONS ANALYTIC ON AN ANNULUS 733

γR

γ̂ α
Γ2 • z0 Γ1

The integrand is analytic on the inside of the two simple closed curves Γ1 and Γ2 .
Letting γ 1 and γ 2 be oriented parametrizations for these and using the argument that the
integrals over the straight lines cancel, this yields
1 1 1 1
Z Z
f (w) n+1
dw + f (w) dw = 0
2πi γR (w − z0 ) 2πi γ̂ α (w − z0 )n+1

and so, letting γ α ≡ −γ̂ α , yields formula 36.5.

Similar considerations apply to bn .

Corollary 36.2.3 The an and bn are uniquely determined.

Proof: Let α ∈ (r, R) and let γ α be a parametrization of the circle centered at z0 of

radius α which is counterclockwise. We have
∞ ∞
f (w) = ∑ an (w − z0 )n + ∑ bn (w − z0 )−n
n=0 n=1

for w in the annulus. Then for k ≥ 1,

∞ ∞
f (w) (w − z0 )k−1 = ∑ an (w − z0 )n+k−1 + ∑ bn (w − z0 )−n+k−1
n=0 n=1

By uniform convergence,
Z ∞ Z
f (w) (w − z0 )k−1 dw = ∑ an (w − z0 )n+k−1 dw
γα n=0 γα
∞ Z
+ ∑ bn (w − z0 )−n+k−1 dw
n=1 γα

Now in the sums, all integrals are 0 except the one when n = k in the second sum. Therefore,
Z Z
f (w) (w − z0 )k−1 dw = bk (w − z0 )−1 dw = 2πibk
γα γα
734 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

This shows that for any α, r < α < R,

1
Z
bk = f (w) (w − z0 )k−1 dw
2πi γα

Similar reasoning gives

1 1
Z
an = f (w) dw
2πi γα (w − z0 )n+1
and as explained above, nothing changes when α is changed. ■

Definition 36.2.4 For f near the closure of an annulus as just described, it follows that on
the annulus, f can be written as the sum of a power series and a series involving (z − z0 )
raised to negative powers. This is called the Laurent series. The series involving negative
powers of (z − z0 ) is called the principal part of the Laurent series.

Note that if f is analytic near z0 , but possibly not at z0 then the r in γ r can be taken as
small as desired.

36.3 Isolated Singularities

This is about the situation where the Laurent series of f has nonzero principal part. When
this occurs, we say that z0 is a singularity. The singularities are isolated if each is the center
of a ball such that f is analytic except for the center of the ball.

Definition 36.3.1 Let B′ (a, r) ≡ {z ∈ C such that 0 < |z − a| < r}. Thus this is the usual
ball without the center. A function is said to have an isolated singularity at the point a ∈ C
if f is analytic on B′ (a, r) for some r > 0.

It turns out isolated singularities can be neatly classified into three types, removable
singularities, poles, and essential singularities. The next theorem deals with the case of a
removable singularity.

Definition 36.3.2 An isolated singularity of f is said to be removable if there exists an

analytic function g analytic at a and near a such that f = g at all points near a.

Theorem 36.3.3 Let f : B′ (a, r) → X be analytic. Thus f has an isolated singularity at a.

Then a is a removable singularity if and only if

lim f (z) (z − a) = 0.
z→a

Thus the above limit occurs if and only if there exists a unique analytic function, g :
B (a, r) → X such that g = f on B′ (a, r) . In other words, you can re define f at a so
that the resulting function is analytic.

Proof: ⇒Let h (z) ≡ (z − a)2 f (z) , h (a) ≡ 0. Then h is analytic on B (a, r) because it is
easy to see that h′ (a) = 0. It follows h is given by a power series,
∞
h (z) = ∑ ak (z − a)k
k=2
36.3. ISOLATED SINGULARITIES 735

where a0 = a1 = 0 because of the observation above that h′ (a) = h (a) = 0. It follows that
for |z − a| > 0
∞
f (z) = ∑ ak (z − a)k−2 ≡ g (z) .
k=2

⇐The converse is obvious. ■

What of the other case where the singularity is not removable? This situation is dealt
with by the amazing Casorati Weierstrass theorem.

Theorem 36.3.4 (Casorati Weierstrass) Let a be an isolated singularity and suppose for
some r > 0, f (B′ (a, r)) is not dense in C. Then either a is a removable singularity or there
exist finitely many b1 , · · · , bM for some finite number, M such that for z near a,
M
bk
f (z) = g (z) + ∑ (36.6)
k=1 (z − a)k

where g (z) is analytic near a.

Such an a satisfying 36.6 is called a pole.

Proof: Suppose B (z0 , δ ) has no points of f (B′ (a, r)) . Such a ball must exist if f (B′ (a, r))
is not dense. Then for z ∈ B′ (a, r) , | f (z) − z0 | ≥ δ > 0. It follows from Theorem 36.3.3
1
that f (z)−z has a removable singularity at a. Hence, there exists h an analytic function such
0
that for z near a,
1
h (z) = . (36.7)
f (z) − z0
k 1
There are two cases. First suppose h (a) = 0. Then ∑∞ k=1 ak (z − a) = f (z)−z0 for z near
a. If all the ak = 0, this would be a contradiction because then the left side would equal zero
for z near a but the right side could not equal zero. Therefore, there is a first m such that
am ̸= 0. Hence there exists an analytic function, k (z) which is not equal to zero in some
ball, B (a, ε) such that
1
k (z) (z − a)m = .
f (z) − z0
Hence, taking both sides to the −1 power,
∞
1
f (z) − z0 = ∑ bk (z − a)k
(z − a)m k=0

and so 36.6 holds.

The other case is that h (a) ̸= 0. In this case, raise both sides of 36.7 to the −1 power
and obtain
f (z) − z0 = h (z)−1 ,
a function analytic near a. Therefore, the singularity is removable. ■
This theorem is the basis for the following definition which describes isolated singular-
ities.
736 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

36.4 Meromorphic Functions

In short, meromorphic functions have only isolated singularities and the singularities are
either poles or removable. Thus this collection of functions includes the analytic functions.
Analytic functions are all like polynomials. Meromorphic functions are all like rational
functions. This observation can be made much more precise but this is roughly the idea.
In fact, functions meromorphic on the Riemann sphere are rational functions but this is not
developed in this book. There is so much available in complex analysis that I don’t wish to
try and include it all.

Definition 36.4.1 Let a be an isolated singularity of f . When 36.6 holds for z near a, then
a is called a pole. The order of the pole in 36.6 is M. Essential singularities are those
which have infinitely many nonzero terms in the principal part of the Laurent series. When
a function f is analytic except for isolated singularities and the isolated singularities are
all poles, and there are finitely many of these poles in every compact set, the function is
called meromorphic.

Actually, if you insist only that the singularities are isolated and poles, then you can
prove that there are finitely many in any compact set so part of the above definition is
actually redundant as shown in the following lemma.

Lemma 36.4.2 If f has a pole at a, then limz→a | f (z)| = ∞. Also if f ∈ M (Ω) for Ω an
open set, then the poles cannot have a limit point in Ω and there are finitely many poles in
every B (0, R). For f ∈ M (Ω) , α is a pole if and only if limz→α | f (z)| = ∞. Also α is a
zero if and only if limz→α | f (z)| = 0.

Proof: We know by definition that

n
bk
f (z) = g (z) + ∑
k=1 (z − a)k

where bn ̸= 0. Thus by the triangle inequality,

!
n−1
|bn | |bk |
| f (z)| ≥ n − |g (z)| + ∑ k
|z − a| k=1 |z − a|
!!
|bn | n 1 n−1
=
|z − a|n
1 − |g (z)| |z − a| / |bn | + ∑ |bk | |z − a|n−k
|bn | k=1
|bn | 1
≥
|z − a|n 2

for |z − a| small enough. Thus the claim is verified.

Consider the second claim. Suppose α m is a pole and limm→∞ α m = α ∈ Ω and f is
analytic at α. Then from the first part, there exists β m such that |α m − β m | < 1/m but
| f (β m )| > m. Then β m → α but limm→∞ | f (β m )| does not exist. In particular, | f (β m )|
fails to converge to f (α) showing that f cannot be analytic at α. Thus it must be the case
that α is a pole. But now this is not allowed either because at a pole the function is analytic
on a deleted ball centered at the pole and it is assume here that limm→∞ α m = α. Thus there
are finitely many poles in every compact subset of Ω including B (0, R).
36.5. THE RESIDUE THEOREM 737

Finally, consider the last claim. It is obvious that α is a zero if and only if limz→α f (z) =
0. It was shown above that at poles limz→α | f (z)| = ∞. Then suppose the limit condition
holds. Why is α a pole? This happens because of the Casorati Weierstrass theorem, The-
orem 36.3.4. Every singularity is isolated for a meromorphic function by definition. Thus
there is a Laurent expansion for f near α. If the principal part is an infinite series, then
by this theorem, the values of f near α are dense in C and so limz→α | f (z)| does not even
exist. Therefore, this principal part must be a finite sum and so α is a pole. ■
What follows is the definition of something called a residue. This pertains to a singu-
larity which has a pole at an isolated singularity.

Definition 36.4.3 The residue of f at an isolated singularity α which is a pole, written

res ( f , α) is the coefficient of (z − α)−1 where
m
bk
f (z) = g (z) + ∑ .
k=1 (z − α)k

Thus res ( f , α) = b1 in the above.

36.5 The Residue Theorem

We have in mind finitely many poles enclosed by a simple closed, piecewise C1 curve as in
the following picture which shows the case of two poles.

γ0
γ1 γ2

a1 • a2 •

You have a simple closed curve, positively oriented. Say γ is a parametrization for this
curve. Then inside there are finitely many singularities {ak }nk=1 . Enclose each with a circle
oriented in the clockwise direction, parameterized by γ̂ k and connect them with straight
lines as shown. Then you have two simple closed curves which intersect in these finitely
many straight line segments. Orient them oppositely so that line integrals over the straight
line segments cancel and each of the two simple closed curves is oriented positively. Then
if f is analytic except at the points shown, the Cauchy integral theorem implies
Z n Z
f (z) dz + ∑ f (z) dz = 0
γ k=1 γ̂ k

Letting γ k ≡ −γ̂ k ,
Z n Z
f (z) dz = ∑ f (z) dz (36.8)
γ k=1 γ k
738 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

Now on the inside of γ k ,

Mk
bn
f (z) = gk (z) + ∑ (36.9)
n=1 (z − ak )n
Indeed, if n ̸= −1, (z − a)n
R
Thus γ 1 f (z) dz = b1 because all the other terms have primitives.
(z−a)n+1
has n+1 as a primitive. However,

b1
Z
dz = 2πib1
γk z − ak
bn
Definition 36.5.1 Suppose f (z) = g (z) + ∑M
n=1 (z−a)n for z near a. Then res ( f , a) ≡ b1 .
R
Using this notation, by analogy to the above, γk f (z) dz = 2πi res ( f , ak ) . Then from
36.8, Z n
f (z) dz = 2πi ∑ res ( f , ak )
γ k=1
In words, the contour integral is 2πi times the sum of the residues.
So is there a way to find the residues? The answer is yes.

PROCEDURE 36.5.2 Say you want to find res ( f , a) = b1 in

M
bn
f (z) = g (z) + ∑ , g analytic
n=1 (z − a)n

This is the case where you have a pole of order M at a. You would multiply by (z − a)M .
This would give
M
f (z) (z − a)M = g (z) (z − a)M + ∑ bn (z − a)M−n
n=1

Then you would take M − 1 derivatives and then take the limit as z → a. This would give
(M − 1)!b1 .

You can see from the formula that this will work and so there is no question that the
limit exists. Because of this, you could use L’Hospitals rule to formally find this limit. This
rule pertains only to real functions of a real variable so it is somewhat unjustified to use
it. However, since you know the limit exists in this case, you can pick a one dimensional
direction and apply L’Hospital to the real and imaginary parts to identify the limit which
is typically what needs to be done. It is a nice illustration of the difference between real
analysis which is characterized by pathology and complex analysis which is much more
agreeable. Difficult mathematical questions about whether something exists are often less
the issue in complex analysis.

36.6 Evaluation of Improper Integrals

You can use the above method of residues to evaluate obnoxious integrals of the form
Z R
p (x) p (x)
Z ∞
dx ≡ lim dx
−∞ q (x) R→∞ −R q (x)
36.6. EVALUATION OF IMPROPER INTEGRALS 739

provided the degree of p (x) is two less than the degree of q (x). This can be done by
using the contour γ R which goes from (−R, 0) to (R, 0) along the real line and then on the
semicircle of radius R from (R, 0) to (−R, 0).
y

x
Letting CR be the circular part of this contour, for large R,
p (z) CRk
Z
dz ≤ πR k+2
CR q (z) R
which converges to 0 as R → ∞. Therefore, it is only a matter of taking large enough R to
enclose all the roots of q (z) which are in the upper half plane, finding the residues at these
points and then computing the contour integral. Then you would let R → ∞ and the part
of the contour on the semicircle will disappear leaving the Cauchy principal value integral
which is desired. There are other situations which will work just as well. You simply need
to have the case where the integral over the curved part of the contour converges to 0 as
R → ∞.
Here is an easy example.
R∞ 1
Example 36.6.1 Find −∞ x2 +1 dx

You know from calculus that the answer is π. Lets use the method of residues to find
this. The function z21+1 has poles at i and −i. We don’t need to consider −i. It seems clear
that the pole at i is of order 1 and so all we have to do is take
x−i 1 1
lim = (x − i) =
1 + x2
z→i (x − i) (x + i) 2i

Then the integral equals 2πi 2i1 = π.

That one is easy. Now here is a genuinely obnoxious integral.

R∞ 1
Example 36.6.2 Find −∞ 1+x4 dx

It will have poles at the roots of 1 + x4 . These are

1 1 √ 1 1 √ 1 1 √ 1 1 √
       
− i 2, − + i 2, − − i 2, + i 2
2 2 2 2 2 2 2 2
Using the above contour, we only need consider
1 1 √ 1 1 √
   
− − i 2, + i 2
2 2 2 2
Since they are all distinct, the poles at these two will be of order 1. To find the residues at
these points, you would need to have
 
√ 
z − − 12 − 12 i 2
lim √
z→−( 12 + 12 i) 2 1 + z4
740 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

factoring 1 + x4 and computing the limit, you could get the answer. Applying L’Hospital’s
rule to identify the limit you know is there,
1 1 √
 
1
lim √ 3 = − i 2
z→−( 12 + 21 i) 2 4z 8 8

√
Similarly, the residue at 21 + 12 i 2 is
1 1 √
 
− + i 2
8 8
Then the contour integral is
1 1 √ 1 1 √ 1√
      
2πi − i 2 + 2πi − + i 2 = 2π
8 8 8 8 2
You might observe that this is a lot easier than doing the usual partial fractions and trig
substitutions etc. Now here is another tedious example.
R∞ x+2
Example 36.6.3 Find −∞ 2 dx
(x2 +1)(x2 +4)
The poles of interest are located at i, 2i. The pole at 2i is of order 2 and the one at i is
of order 1. In this case, the partial fractions expansion is
1 2 1 2 1 2
9x+ 9 3x+ 3 9x+ 9
− −
x2 + 1 (x2 + 4)2 x2 + 4
The pole at i would be
1 2 1 2



9z+ 9 (z − i) 9i+ 9 1 1
lim = = − i
z→i (z + i) (z − i) (i + i) 18 9
Now consider the pole at 2i by consideration of the next two terms in the partial fractions
expansion. You must multiply it by (x − 2i)2 , take the derivative and then take a limit as
x → 2i. Multiplying and taking the derivative yields
!
1 2
3 x + 3 1
Dx 2
=− (x + 4 − 2i)
(x + 2i) 3 (x + 2i)3
Then you have to take a limit as x → 2i which is
1
− i
48
Finally, consider the last term which has a pole of order 1.
1
x + 92 (x − 2i)


1 1
lim 9 = − i
x→2i (x − 2i) (x + 2i) 18 18
Then adding in the minus sign, we have the following for the integral.
      
1 1 1 1 1 5
2πi − i + 2πi − − i − 2πi − i = π
18 9 18 18 48 72
Sometimes you don’t blow up the curves and take limits. Sometimes the problem of
interest reduces directly to a complex integral over a closed curve. Here is an example of
this.
36.6. EVALUATION OF IMPROPER INTEGRALS 741

Example 36.6.4 The integral is

cos θ
Z π
dθ
0 2 + cos θ
This integrand is even and so it equals

1 cos θ
Z π
dθ .
2 −π 2 + cos θ

For z on the unit circle, z = eiθ , z = 1z and therefore, cos θ = 12 z + 1z . Thus dz = ieiθ dθ

and so dθ = dz iz . Note that this is done in order to get a complex integral which reduces
to the one of interest. It follows that a complex integral which reduces to the integral of
interest is
1 1


1 2 z+ z dz 1 z2 + 1
Z Z
1 1
= dz
2i γ z (4z + z2 + 1)


2i γ 2 + 2 z + z z
where γ is the unit circle oriented counter clockwise. Now the integrand has poles of order
1 at those points where z 4z + z2 + 1 = 0. These points are

√ √
0, −2 + 3, −2 − 3.

Only the first two are inside the unit circle. It is also clear the function has simple poles at
these points. Therefore,

z2 + 1
 
res ( f , 0) = lim z = 1.
z→0 z (4z + z2 + 1)
 √ 
res f , −2 + 3 =
  √  z2 + 1 2√
lim √ z − −2 + 3 2
=− 3.
z→−2+ 3 z (4z + z + 1) 3
It follows
cos θ 1 z2 + 1
Z π Z
dθ = dz
0 2 + cos θ 2i γ z (4z + z2 + 1)
2√
 
1
= 2πi 1 − 3
2i 3
2√
 
= π 1− 3 .
3

Other rational functions of the trig functions will work out by this method also.
Sometimes we have to be clever about which version of an analytic function that re-
duces to a real function we should use. The following is such an example.

Example 36.6.5 The integral here is

ln x
Z ∞
dx.
0 1 + x4
742 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

It is natural to try and use the contour in the following picture in which the small circle
has radius r and the large one has radius R.
y

x
However, this will create problems with the log since the usual version of the log is not
defined on the negative real axis. This difficulty may be eliminated by simply using another
branch of the logarithm as in Example 36.1.3. Leave out the ray from 0 along the negative
y axis and use this example to define L (z) on this set. Thus L (z) = ln |z| + i arg1 (z) where
arg1 (z) will be the angle θ , between − π2 and 3π iθ
2 such that z = |z| e . Then the function
L(z)
used is f (z) ≡ 1+z4 . Now the only singularities contained in this contour are

1√ 1√ 1√ 1√
2 + i 2, − 2+ i 2
2 2 2 2
 √ √ 
and the integrand f has simple poles at these points. Thus res f , 21 2 + 21 i 2 =
  √ √ 
z − 12 2 + 12 i 2 (ln |z| + i arg1 (z))
lim √
√
z→ 12 2+ 12 i 2 1 + z4
  √ √ 
(ln |z| + i arg1 (z)) + z − 12 2 + 12 i 2 (1/z)
= lim √
√
z→ 12 2+ 12 i 2 4z3
q 
1 1 π
ln + 2 +i4
2 1 √
 
1
=  √ √ 3 = 32 − 32 i 2π
4 12 2 + 21 i 2

Similarly
−1 √ 1√
 
res f , 2+ i 2 =
2 2
3 √ 3 √
2π + i 2π.
32 32
Of course it is necessary to consider the integral along the small semicircle of radius r. This
reduces to Z 0
ln |r| + it
rieit dt


it 4
π 1 + (re )

which clearly converges to zero as r → 0 because r ln r → 0. Therefore, taking the limit as

r → 0, Z −r
L (z) ln (−t) + iπ
Z
4
dz + lim dt+
large semicircle 1 + z r→0+ −R 1 + t4
3√ 3 √ 1√ 1 √
Z R  
lnt
lim dt = 2πi 2π + i 2π + 2π − i 2π .
r→0+ r 1 + t 4 32 32 32 32
36.6. EVALUATION OF IMPROPER INTEGRALS 743

R L(z)
Observing that large semicircle 1+z4 dz → 0 as R → ∞,
Z R Z 0 √
 
lnt 1 1 1
e (R) + 2 lim 4
dt + iπ 4
dt = − + i π2 2
r→0+ r 1+t −∞ 1 + t 8 4

where e (R) → 0 as R → ∞. From an earlier example this becomes

√ ! 
Z R √

lnt 2 1 1
e (R) + 2 lim 4
dt + iπ π = − + i π 2 2.
r→0+ r 1 + t 4 8 4

Now letting r → 0+ and R → ∞,

√ !
√
 
lnt 1 1 2
Z ∞
2 dt = − + i π 2 2 − iπ π
0 1 + t4 8 4 4
1 √
= − 2π 2 ,
8
and so
lnt 1√ 2
Z ∞
dt = −
2π ,
0 1 + t4
16
which is probably not the first thing you would thing of. You might try to imagine how this
could be obtained using elementary techniques.

Example 36.6.6 The Fresnel integrals are

Z ∞ Z ∞
cos x2 dx, sin x2 dx.



0 0

2
To evaluate these integrals we will consider f (z) = eiz on the curve  which
 goes from
1+i
√
the origin to the point r on the x axis and from this point to the point r 2
along a circle
of radius r, and from there back to the origin as illustrated in the following picture.
y

x
Thus the curve is shaped like a slice of pie. The angle is 45◦ . Denote by γ r the curved
part. Since f is analytic,
Z r Z r   1+i 2  
1+i
Z
2 2 i t √
0 = eiz dz + eix dx − e 2 √ dt
γr 0 0 2
Z r   Z r
1+i
Z
2 2
2
= eiz dz + e−t
eix dx −
√ dt
γr 0 0 2
Z r √  
π 1+i
Z
iz2 ix2
= e dz + e dx − √ + e (r)
γr 0 2 2
744 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

R ∞ −t 2 √
where e (r) → 0 as r → ∞. This used 0 e dt = 2π . Now examine the first of these
integrals.
Z Z π
2 4 it 2
eiz dz = ei(re ) rieit dt
γr 0
Z π
4 2 sin 2t
≤ r e−r dt
0
Z 1 −r2 u
r e
= √ du
2 0 1 − u2
Z −(3/2) Z 1
r r

1 r 1 1/2
= √ du + √ e−(r )
2 0 1 − u2 2 0 1 − u2
which converges to zero as r → ∞. Therefore, taking the limit as r → ∞,
√   Z∞
π 1+i 2
√ = eix dx
2 2 0

and so the Fresnel integrals are given by

Z ∞ √ Z ∞
π
sin x2 dx = √ = cos x2 dx.
0 2 2 0

The following example is one of the most interesting. By an auspicious choice of the
contour it is possible to obtain a very interesting formula for cot πz known as the Mittag
Leffler expansion of cot πz.
Example 36.6.7 Let γ N be the contour which goes from −N − 12 − Ni horizontally to N +
1 1 1
2 − Ni and from there, vertically to N + 2 + Ni and then horizontally to −N − 2 + Ni and
finally vertically to −N − 21 − Ni. Thus the contour is a large rectangle and the direction of
integration is in the counter clockwise direction.
(−N − 21 ) + Ni (N + 12 ) + Ni

(−N − 12 ) − Ni (N + 12 ) − Ni
Consider the following integral.
π cos πz
Z
IN ≡ dz
γN (α 2 − z2 ) sin πz
where α is not an integer. This will be used to verify the formula of Mittag Leffler,
∞
1 2 π cot πα
2
+∑ 2 2
= . (36.10)
α n=1 α − n α
36.6. EVALUATION OF IMPROPER INTEGRALS 745

It is left as an exercise to verify that cot πz is bounded on this contour and that therefore,
IN → 0 as N → ∞. Now compute the residues of the integrand at ±α and at n where
|n| < N + 12 for n an integer. These are the only singularities of the integrand in this contour
and therefore, IN can be obtained by using these. First consider the residue at ±α. These
are obviously poles of order 1 and so to get the one at α, you take

(z − α) π cos πz −π cos πz −π cos πα

lim 2 2
= lim =
z→α (α − z ) sin πz z→α (α + z) sin πz 2α sin πα
You get the same thing at −α. Next consider the residue at n. If you consider the power
series, you will see that this should also be a pole of order 1. Thus it is

(z − n) π cos πz π cos πz − (z − n) π 2 sin (πz)

lim = lim
z→n (α 2 − z2 ) sin πz z→n −2z sin πz + (α 2 − z2 ) π cos (πz)

π (−1)n 1
= = 2
(α 2 − n2 ) π (−1)n α − n2
Therefore, " #
N
1 π cot πα
0 = lim IN = lim 2πi ∑ 2 2
−
N→∞ N→∞ n=−N α − n α
which establishes the following formula of Mittag Leffler.
N
1 π cot πα
lim ∑ = .
N→∞ n=−N α 2 − n2 α

Writing this in a slightly nicer form, we obtain 36.10.

The next example illustrates the technique of a branch cut.
p−1
Example 36.6.8 For p ∈ (0, 1) , find 0∞ x1+x dx. This example illustrates the use of some-
R

thing called a branch cut. The idea is you need to pick a single determination of z p−1
which converges to x p−1 for x real and z getting close to x. It will make use of the following
contour. In this contour, the radius of the large circle is R and the radius of the small one
is r. The angle between the straight lines and the x axis is ε. Denote this contour by γ R,r,ε .

Choose a branch of the logarithm of the form log (z) = ln |z| + iA (z) where A (z) is the
angle of z in (0, 2π). Thus

z p−1 = e(p−1)(ln|z|+iA(z))
The straight lines, the one on top. reiε + t Reiε = z, t ∈ [0, 1].


746 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

Contour integral:

p−1 (p−1)iε
reiε + t Reiε
Z 1
e
Reiε dt
1 + reiε + t (Reiε ) f s
0
 
The one on the bottom: rei(2π−ε) + t Rei(2π−ε) = z, t ∈ [0, 1]
Contour integral:
  p−1
Z 1 rei(2π−ε) + t Rei(2π−ε) e(p−1)i(2π−ε)
−
Rei(2π−ε) dt
0 1 + rei(2π−ε) + t Rei(2π−ε)
The integral over the small circle: z = reit ,t ∈ [ε, 2π − ε]
Contour integral:
Z 2π−ε p−1 (p−1)it
r e
− rieit dt
ε 1 + reit
The integral over the large circle: z = Reit ,t ∈ [ε, 2π − ε]
Contour integral:
Z 2π−ε p−1 (p−1)it
R e
Rieit dt
ε 1 + Reit

z p−1
Z
2πieiπ(p−1) = dz
γ R,r,ε 1+z

The residue at −1 of the function is eiπ(p−1) and so the contour integral on the right equals
the sum of those other integrals above. Now let ε → 0. This yields

z p−1
Z
2πieiπ(p−1) = dz
γ R,r 1+z

where the integral on the right equals the sum

!
(r + tR) p−1 (r + tR) p−1 e(p−1)i(2π)
Z 1 Z 1
Rdt + − Rdt
0 1 + r + tR 0 1 + r + tR
E1 (r) E2 (R)
Z 2π p−1 (p−1)it Z 2π p−1 (p−1)it
r e it R e
+ rie dt + Rieit dt
0 1 + reit 0 1 + Reit
The last two integrals converge to 0 as r → 0 and R → ∞. This follows easily from the
form of the integrands. You can change the variable in the first two to write them as
Z R p−1 Z R p−1
x x
dx, −e(p−1)i(2π) dx
r 1+x r 1+x
Thus Z R p−1
x  
2πieiπ(p−1) = dx 1 − e(p−1)i(2π) + E1 (r) + E2 (R)
r 1+x
36.7. EXERCISES 747

where E (r) , E (R) converges to 0 as r → 0 and R → ∞. There is no hope of taking a limit

as R → ∞ while keeping r > 0 fixed, but if we let both variables converge at the same time,
then we could get something. Let r → 0+ and let R = 1/r.
Z 1/r p−1
x  
2πieiπ(p−1) = dx 1 − e(p−1)i(2π) + E (r, 1/r)
r 1+x
and now, as r → 0,which equals the sum of the two integrals over the straight lines added to
the integrals over the small circles which converge to 0 as r → 0 and R → ∞. Top straight
line converges as r → 0 to
(Rt) p−1
Z 1
Rdt
0 1 + tR
Bottom integral converges as r → 0 to

|tR| p−1 e(p−1)i(2π)

Z 1
− Rdt
0 1 + tR
Of course change variables letting x = tR and the two integrals which must be summed are
Z R p−1 Z R p−1
x (p−1)i(2π) x
dx, −e dx
0 1+x 0 1+x
Thus you get the following as R → ∞.
Z ∞ p−1 
x 
dx 1 − e2π(p−1)i = 2πieiπ(p−1)
0 1+x

Then what you get is

Z ∞ p−1
x 2πieiπ(p−1)
dx =
0 1+x 1 − e2π(p−1)i

−2πieiπ p −2πi −2πi

= = = −iπ p
1 − e2π pi (1 − e2π pi ) e−iπ p e − eπip
−2πi π
= =
(cos π p − i sin (π p)) − (cos π p + i sin (π p)) sin (pπ)
I think this is quite an amazing result.
Actually, people typically are a little more informal in the consideration of such inte-
grals. They regard the bottom side of the line x ≥ 0 as being associated with θ = 2π and
the top side being associated with θ = 0 and leave out the fuss with taking limits as ε → 0
and so forth.

36.7 Exercises
R ∞ cos x
1. Find the following improper integral. −∞ 1+x4 dx Hint: Use upper semicircle con-
∞ e
R ix
tour and consider instead −∞ 1+x4
dx. This is because the integral over the semicircle
will converge to 0 as R → ∞ if you have eiz but this won’t happen if you use cos z be-
cause cos z will be unbounded. Just write down and check and you will see why this
eiz
happens. Thus you should use 1+z 4 and take real part. I think the standard calculus
techniques will not work for this horrible integral.
748 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

cos(x)
dx. Hint: Do the same as above replacing cos x with eix .
R∞
2. Find −∞ 2
(1+x2 )
3. Consider the following contour.

The small semicircle has radius r and is centered at (1, 0). The large semicircle has
radius R and is centered at (0, 0). Use the method of residues to compute
 Z R Z r 
x x
lim lim dx + dx
r→0 R→∞ r 1 − x3 −R 1 − x
3

∞ x R
This is called the Cauchy principal value for −∞ 1−x3
dx. The integral makes no sense
in terms of a real honest integral. The function has Ra pole on the x axis. Another
instance of this was in Problem 6 on Page 720 where 0∞ sin (x) /xdx was determined
similarly. However, you can define such a Cauchy principal value. Rather than
belabor this issue, I will illustrate with this example. These principal value integrals
occur because of cancelation. They depend on a particular way of taking a limit.
They are not mathematically respectable but are certainly interesting. They are in that
general area of finding something by taking a certain kind of symmetric limit. Such
problems include the Lebesgue fundamental theorem of calculus with the symmetric
derivative.
R 2π cos(θ )
4. Find 0 1+sin2 (θ )
dθ .
R 2π dθ
5. Find 0 2−sin θ .
R π/2 dθ
6. Find −π/2 2−sin θ .

7. Suppose you have a function f (z) which is the quotient of two polynomials in which
the degree of the top is two less than the degree of the bottom and you consider the
contour.

Then define Z
f (z) eisz dz
γR
in which s is real and positive. Explain why the integral makes sense and why the
part of it on the semicircle converges to 0 as R → ∞. Use this to find
eisx
Z ∞
dx, k > 0.
−∞ k 2 + x2
36.7. EXERCISES 749

8. Show using methods from real analysis that for b ≥ 0,

Z ∞ √
2 π −b2
e−x cos (2bx) dx = e
0 2
√
2 2
Hint: Let F (b) ≡ 0∞ e−x cos (2bx) dx − 2π e−b . Then from Problem 13 on Page
R

383, F (0) = 0. Using the mean value theorem on difference quotients, explain why
Z ∞ √
′ −x2 π −b2
F (b) = −2xe sin (2bx) dx + 2b e
0 2
Z ∞ √ 
′ −x2 π −b2
F (b) = 2b e cos (2bx) dx + e
0 2
 √ √ 
π −b2 π −b2
= 2b F (b) + e + e
2 2
√ 2
= 2bF (b) + π2be−b

Now use the integrating factor method for solving linear differential equations from
beginning differential equations to solve the ordinary differential equation.

d  −b2  √ 2
e F (b) = π2be−2b
db
Then
2 1 2√ 1√
e−b F (b) − 0 = − e−2b π + π
2 2
1 2 1 √ −b2
F (b) = − e−b + πe =0
2 2
You fill in the details. This is meant to be a review of real variable techniques.
9. For b > 0, use the contour which goes from −a to a to a + ib to −a + ib to −a.
2
Then let a → ∞ and show that the integral of e−z over the vertical parts of this
contour converge to 0. Hint: You know from an earlier problem what happens
2 2
on the bottom part of the contour. Also for z = x + ib, e−z = e−(x −b +2ixb) =
2

2 2
eb e−x (cos (2xb) + i sin (2xb)) .
10. Consider the circle of radius 1 oriented counter clockwise. Evaluate
Z
z−6 cos (z) dz
γ

11. Consider the circle of radius 1 oriented counter clockwise. Evaluate

Z
z−7 cos (z) dz
γ

R ∞ 2+x2
12. Find 0 1+x4 dx.
R ∞ x1/3
13. Find 0 1+x2 dx
750 CHAPTER 36. ISOLATED SINGULARITIES AND ANALYTIC FUNCTIONS

14. Suppose f is an entire function and that it has no zeros. Show there must exist an
entire function g such that f (z) = eg(z) . Hint: Letting γ (0, z) be the line segment
′ (w) ′ (z)
which goes from 0 to z, let ĝ (z) ≡ γ(0,z) ff (w) dw. Then show that ĝ′ (z) = ff (z)
R
. Then
 ′ ′
e−ĝ(z) f (z) = e−ĝ(z) −ff(z)(z) f (z) + e−ĝ(z) f ′ (t) = 0. Now when you have an entire
function whose derivative is 0, it must be a constant. Modify ĝ (z) to make f (z) =
eg(z) .

15. Let f be an entire function with zeros {α 1 , · · · , α n } listed according to multiplicity.

Thus you might have repeats in this list. Show that there is an analytic function g (z)
such that for all z ∈ C,
n
f (z) = ∏ (z − α k ) eg(z)
k=1

Hint: You know f (z) = ∏nk=1 (z − α k ) h (z) where h (z) has no zeros. To see this, note
that near α 1 , f (z) = a1 (z − α 1 ) + a2 (z − α 1 )2 + · · · and so f (z) = (z − α 1 ) f1 (z)
where f1 (z) ̸= 0 at α 1 . Now do the same for f1 and continue till fn = h. Now use the
above problem.
Chapter 37

Some Fundamental Functions

and Transforms

37.1 Gamma Function

This chapter is on some fundamental ideas related to Fourier and Laplace transforms and
the Gamma function. The symbol a∞ f (t) dt will always mean
R

Z R
lim f (t) dt
R→∞ a

provided f is piecewise continuous

Ra
on [a, ∞) whenever the limit exists. It is the standard im-
proper integral from calculus. −∞ f (t) dt is defined similarly. However, if f is unbounded
at 0, the symbol will mean
Z R Z 1
lim f (t) dt + lim f (t) dt
R→∞ a+1 δ →0+ a+δ

or more simply, when f (t) ≥ 0,

Z a+δ −1
lim f (t) dt
δ →0 a+δ

First is a very important function defined in terms of an integral. Also recall that the value
of the Riemann integral does not depend on the value of the function at single points. All
this is more satisfactory if you do it in the context of the Lebesgue integral. Here it is
assumed that all functions are piecewise continuous having finitely many jumps in every
finite interval so there will be no difficulty in writing the Riemann integral.

Definition 37.1.1 The gamma function is defined by

Z ∞
Γ (α) ≡ e−t t α−1 dt
0

whenever α > 0.

Lemma 37.1.2 The integral is finite for each α > 0.

751
752 CHAPTER 37. SOME FUNDAMENTAL FUNCTIONS AND TRANSFORMS

Proof: By the monotone convergence theorem, for n ∈ N

Z n Z 1 Z n 
Γ (α) = lim e−t t α−1 ≤ lim sup t α−1 dt + Ce−t/2
n→∞ 1/n n→∞ 1/n 1
1  1 1

≤ + lim −2Ce− 2 n + 2Ce− 2 < ∞
α n→∞
The explanation for the constant is as follows. For t ≥ 1 and m a positive integer larger than
α,
t α−1 t n−1
< t/2
et/2 e
which converges to 0 as t → ∞ which is easily shown by an appeal to L’Hospital’s rule.
Hence
t α−1 e−t ≤ Cet/2 e−t = Ce−t/2 . ■

Proposition 37.1.3 For n a positive integer, n! = Γ (n + 1). In general, Γ (1) = 1, Γ (α + 1) =

αΓ (α)
R −1  −1

Proof: First of all, Γ (1) = limδ →0 δδ e−t dt = limδ →0 e−δ − e−(δ ) = 1. Next,
for α > 0,
Z −1
" Z −1
#
δ −1 δ
Γ (α + 1) = lim e−t t α dt = lim −e−t t α |δδ +α e−t t α−1 dt
δ →0 δ δ →0 δ

Z δ −1
!
−1
= lim e−δ δ − e−(δ ) δ
α −α
+α e−t t α−1 dt = αΓ (α)
δ →0 δ

Now it is defined that 0! = 1 and so Γ (1) = 0!. Suppose that Γ (n + 1) = n!, what of
Γ (n + 2)? Is it (n + 1)!? if so, then by induction, the proposition is established. From
what was just shown,

Γ (n + 2) = Γ (n + 1) (n + 1) = n! (n + 1) = (n + 1)!

and so this proves the proposition. ■

37.2 Laplace Transform

Everything holds for a much more general set of assumptions if you have a more modern
version of the integral. This is why I am using notation which corresponds to this more
general situation. All of the functions considered here are assumed piecewise continuous
with finitely many jumps in every finite interval. Then such a function f is said to be in
L1 ([0, ∞)) if Z ∞
| f (t)| dt < ∞
0

Similar usages of this symbol are defined synonomously. Sometimes I will just write L1
to indicate that the absolute value of the function is integrable. Here is the definition of a
Laplace transform.
37.2. LAPLACE TRANSFORM 753

Definition 37.2.1 A function φ has exponential growth on [0, ∞) if there are positive con-
stants λ ,C such thatR |φ (t)| ≤ Ceλt for all t. Then for s > λ , one defines the Laplace
transform L φ (s) ≡ 0∞ φ (t) e−st dt.

Theorem 37.2.2 If s is a complex number and Re s > λ where |φ (t)| ≤ Ceλt , and
Z ∞
f (s) ≡ e−st φ (t) dt
0

then for Re s > λ ,

f (s + h) − f (s)
Z ∞
lim ≡ f ′ (s) = (−t) e−st φ (t) dt
h→0 h 0

Thus s → f (s) is analytic on Re s > λ .

Proof: Let Re s > λ . s will be complex as will h.

Z ∞ −(s+h)t
− e−st e−ht − 1
 
e
Z ∞ Z ∞
−st −st
φ (t) dt + te φ (t) dt = e + t φ (t) dt
0 h 0 0 h

Then
!
e−ht − 1 1 ∞
k k k
+t = ∑ (−1) h t −1 +t
h h k=0
!
∞
k k−1 k
= ∑ (−1) h t +t
k=1
∞
= h ∑ (−1)k hk−2t k
k=2

Thus
e−ht − 1
 
+t ≤ |h|t 2 e|h|
h
and so
Z ∞ −(s+h)t
e − e−st
Z ∞ Z ∞
φ (t) dt + te−st φ (t) dt ≤ |h|t 2 e|h|t e− Re(s)t eλt dt
0 h 0 0

which clearly converges to 0 since for all |h| sufficiently small,

e|h|t e− Re(s)t eλt ≤ e−(Re(s)−(λ +ε))t

where ε is small enough that Re (s) > λ + ε. Thus the integral is finite for all |h| small
enough and it is multiplied by |h|. ■
This shows that f is analytic on Re (s) > λ . Hence it has all derivatives. In fact, you
can do a similar computation to the above and verify that
Z ∞
f (k)
(s) = (−t)k e−st φ (t) dt
0
754 CHAPTER 37. SOME FUNDAMENTAL FUNCTIONS AND TRANSFORMS

37.3 Fourier Transform

Definition 37.3.1 The Fourier transform is defined as follows for f ∈ L1 (R) meaning that
Z R
lim | f (t)| dt < ∞
R→∞ −R

f is piecewise continuous on [−R, R] . Then the Fourier transform is given by

1
Z ∞
F f (t) ≡ √ e−itx f (x) dx
2π −∞

The inverse Fourier transform is defined the same way except you delete the minus sign in
the complex exponential.
1
Z ∞
F −1 f (t) ≡ √ eitx f (x) dx
2π −∞

Does it deserve to be called the “inverse” Fourier transform? This question will be
explored somewhat below.
There is a very important improper integral involving sin (x) /x. You can show with a
little estimating that x → sin (x) /x is not in L1 (0, ∞) . Nevertheless, a lot can be said about
improper integrals involving this function.

Theorem 37.3.2 The following hold

R ∞ sin u π
1. 0 u du = 2
R ∞ sin(ru)
2. limr→∞ δ u du = 0 whenever δ > 0.

3. If f ∈ L1 (R) , then limr→∞

R
R sin (ru) f (u) du = 0. This is called the Riemann Lebesgue
lemma.

Proof: The first claim follows from Problem 6 on Page 720 above.
Now consider δ∞ sin(ru)
R R ∞ sin(ru) R δ sin(ru)
u du. It equals 0 u du − 0 u du which can be seen from
the definition of what the improper integral means. Also, you can change the variable. Let
ru = t so rdu = dt and the above reduces to
Z rδ
sin (t) 1 sin (t) sin (ru)
Z ∞ Z ∞
r dt − dt = du
0 t r 0 t δ u
Thus Z rδ
sin (t) sin (ru)
Z ∞
π
− dt = du
2 0 t δ u
and so limr→∞ δ∞ sin(ru)
R
u du = 0 from the first part.
Now consider the third claim, the Riemann Lebesgue lemma. For I an interval let
(
1 if t ∈ I
XI (t) ≡
0 if t ∈
/I
37.3. FOURIER TRANSFORM 755

Then for f ∈ L1 , let fR (t) ≡ X[−R,R] (t) f (t). Then for R large,
Z ∞ Z ∞ Z −R
| f (t) − fR (t)| dt = | f (t)| dt + | f (t)| dt < ε
−∞ R −∞

Now fR is Riemann integrable and so there is a step function s (t) = ∑ni=1 ai XIi (t) such that
|s (t)| ≤ | fR (t)| and
Z R Z ∞
| fR (t) − s (t)| dt = | fR (t) − s (t)| dt < ε
−R −∞

This follows from the definition of the Riemann integral as a limit of integrals of step
functions, details are left for you. Therefore,
Z ∞
|s (t) − f (t)| dt < 2ε
−∞

Now
Z ∞ Z ∞ Z ∞
f (t) sin (rt) dt ≤ |( f (t) − s (t)) sin (rt)| dt + s (t) sin (rt) dt
−∞ −∞ −∞
Z ∞
≤ 2ε + s (t) sin (rt) dt (37.1)
−∞
∞ R
It remains to verify that limr→∞ −∞ s (t) sin (rt) dt =R0. Since s (t) is a sum of scalars times
XI for I an interval, it suffices to verify that limr→∞ −∞ ∞
X[a,b] (t) sin (rt) dt = 0. However,
this integral is just
Z b
−1 1
sin (rt) dt = cos (rb) + cos (ra)
a r r
which clearly converges to 0 as r → ∞. Therefore, for r large enough, 37.1 implies
Z ∞
f (t) sin (rt) dt < 3ε
−∞

Since ε is arbitrary, this shows that 3. holds. ■

Definition 37.3.3 The following notation will be used assuming the limits exist.

lim g (x + r) ≡ g (x+) , lim g (x − r) ≡ g (x−)

r→0+ r→0+

Theorem 37.3.4 Suppose that g ∈ L1 (R) and that at some x, g is locally Holder continuous
from the right and from the left. This means there exist constants K, δ > 0 and r ∈ (0, 1]
such that for |x − y| < δ ,
|g (x+) − g (y)| < K |x − y|r (37.2)
for y > x and
|g (x−) − g (y)| < K |x − y|r (37.3)
for y < x. Then  
2 sin (ur) g (x − u) + g (x + u)
Z ∞
lim du
r→∞ π 0 u 2
g (x+) + g (x−)
= .
2
756 CHAPTER 37. SOME FUNDAMENTAL FUNCTIONS AND TRANSFORMS

R ∞ sin(ru)
Proof: As in the proof of Theorem 37.3.2, changing variables shows that π2 0 u du =
[Link],
 
2 ∞ sin (ur) g (x − u) + g (x + u) g (x+) + g (x−)
Z
du −
π 0 u 2 2
 
2 sin (ur) g (x − u) − g (x−) + g (x + u) − g (x+)
Z ∞
= du
π 0 u 2
 
2 g (x − u) − g (x−) g (x + u) − g (x+)
Z δ
= sin (ur) + du
0 π 2u 2u
 
2 ∞ sin (ur) g (x − u) − g (x−) g (x + u) − g (x+)
Z
+ + du (37.4)
π δ u 2 2
Second Integral: It equals
 
2 ∞ sin (ur) g (x − u) + g (x + u) g (x−) + g (x+)
Z
− du
π δ u 2 2

 
2 sin (ur) g (x − u) + g (x + u)
Z ∞
=
π δ u 2
 
2 sin (ur) g (x−) + g (x+)
Z ∞
− (37.5)
π δ u 2
From part 2 of Theorem 37.3.2,
2 sin (ur) g (x−) + g (x+)
Z ∞
lim du = 0
r→∞ π δ u 2
Thus consider the first integral in 37.4.
 
2 ∞ sin (ur) g (x − u) + g (x + u)
Z
du
π δ u 2
1 ∞ sin (ur) 1 ∞ sin (ur)
Z Z
= g (x − u) du + g (x + u) du
π δ u π δ u
Z −δ 
1 sin (ur) sin (ur)
Z ∞
= g (x + u) du + g (x + u) du
π −∞ u δ u
Now Z −δ Z −δ
sin (ur) g (x + u)
g (x + u) du = sin (ur) du
−∞ u −∞ u
g(x+u) 1 g(x+u)
and u ≤ δ |g (x + u)| for u < −δ . Thus u → u is in L1 ((−∞, −δ )) . Indeed,
Z −δ
g (x + u) 1 1
Z Z
du ≤ |g (x + u)| du = |g (y)| dy < ∞
−∞ u δ R δ R

It follows from the Riemann Lebesgue lemma

Z −δ
g (x + u) g (x + u)
Z ∞
lim sin (ur) du = lim sin (ur) du = 0
r→∞ −∞ u r→∞ δ u
37.3. FOURIER TRANSFORM 757

First Integral in 37.4: This converges to 0 as r → ∞ because of the Riemann Lebesgue

lemma. Indeed, for 0 ≤ u ≤ δ ,
g (x − u) − g (x−) 1
≤ K 1−r
2u u
which is integrable on [0, δ ]. The other quotient also is integrable by similar reasoning. ■
The next theorem justifies the terminology above which defines F −1 and calls it the
inverse Fourier transform. Roughly it says that the inverse Fourier transform of the Fourier
transform equals the mid point of the jump. Thus if the original function is continuous, it
restores the original value of this function. Surely this is what you would want by calling
something the inverse Fourier transform.
Now for certain special kinds of functions, the Fourier transform is indeed in L1 and
one can show that it maps this special kind of function to another function of the same
sort. This can be used as the basis for a general theory of Fourier transforms. However, the
following does indeed give adequate justification for the terminology that F −1 is called the
inverse Fourier transform.
Theorem 37.3.5 Let g ∈ L1 (R) and suppose g is locally Holder continuous from the right
and from the left at x as in 37.2 and 37.3. Then
Z R
1 g (x+) + g (x−)
Z ∞
lim eixt e−ity g (y) dydt = .
R→∞ 2π −R −∞ 2
Proof: Note that
Z R Z ∞ Z ∞ Z R
eixt e−ity g (y) dydt = e−ity g (y) dy eixt dt
−R −∞ −∞ −R
Z ∞ Z R
= e−ity g (y) eixt dydt
−∞ −R

One merely takes a constant outside the integral and then moves a constant inside an inte-
gral. Consider the following manipulations.
Z R
1
Z ∞
eixt e−ity g (y) dydt =
2π −R −∞
Z ∞Z R Z ∞Z R
1 1
eixt e−ity g (y) dtdy = ei(x−y)t g (y) dtdy
2π −∞ −R 2π −∞ −R
Z R Z R 
1 ∞
Z
= g (y) ei(x−y)t dt + e−i(x−y)t dt dy
2π −∞ 0 0
Z R 
1 ∞
Z
= g (y) 2 cos ((x − y)t) dt dy
2π −∞ 0

1 sin R (x − y) 1 sin Ry
Z ∞ Z ∞
= g (y) dy = g (x − y) dy
π −∞ x−y π −∞ y

1 ∞ sin Ry
Z
= (g (x − y) + g (x + y)) dy
π 0 y
 
2 ∞ g (x − y) + g (x + y) sin Ry
Z
= dy
π 0 2 y
758 CHAPTER 37. SOME FUNDAMENTAL FUNCTIONS AND TRANSFORMS

From Theorem 37.3.4,

1 R ixt ∞ −ity
Z Z
lim e e g (y) dydt
R→∞ 2π −R −∞
Z  
2 ∞ g (x − y) + g (x + y) sin Ry
= lim dy
R→∞ π 0 2 y
g (x+) + g (x−)
= .■
2

37.4 The Inversion of Laplace Transforms

How does the Fourier transform relate to the Laplace transform? This is considered next.
Recall that from Theorem 4.1.5 if g has exponential growth |g (t)| ≤ Ceλt , then if Re (s) >
λ , one can define L g (s) as
Z ∞
L g (s) ≡ e−su g (u) du
0

and also s → L g (s) is differentiable on Re (s) > λ in the sense that if h ∈ C and G (s) ≡
L g (s) , then
G (s + h) − G (s)
Z ∞
lim = G′ (s) = − ue−su g (u) du
h→0 h 0
Thus G is analytic and has all derivatives. Then the next theorem shows how to invert the
Laplace transform. It is another one of those results which says that you get the mid point
of the jump when you do a certain process. It is like what happens in Fourier series where
the Fourier series converges to the midpoint of the jump under suitable conditions and like
what was just shown for the inverse Laplace transform. For a fairly elementary discussion
of this kind of thing related to Fourier series, see the single variable advanced calculus book
on my web page.
The next theorem gives a more specific version of what is contained in Theorem 4.2.3
presented later. However, this theorem does assume a Holder continuity condition which
is not needed for Theorem 4.2.3. I think that it is usually the case that the needed Holder
condition will be available.
Theorem 37.4.1 Let g be a piecewise continuous function defined on (0, ∞) which has
exponential growth
|g (t)| ≤ Ceλt for some real λ
and is Holder continuous from the right and left as in 37.2 and 37.3. For Re (s) > λ
Z ∞
L g (s) ≡ e−su g (u) du
0

Then for any γ > λ ,

Z R
1 g (t+) + g (t−)
lim e(γ+iy)t L g (γ + iy) dy = (37.6)
R→∞ 2π −R 2
Proof: This follows from plugging in the formula for the Laplace transform of g and
then using the above. Thus
Z R
1
e(γ+iy)t L g (γ + iy) dy =
2π −R
37.5. THE BROMWICH INTEGRAL 759

1 R (γ+iy)t ∞ −(γ+iy)u
Z Z
e e g (u) dudy
2π −R −∞
Z R
1
Z ∞
= eγt eiyt e−(γ+iy)u g (u) dudy
2π −R −∞

1 R iyt ∞ −iyu −γu

Z Z
= eγt e e e g (u) dudy
2π −R −∞
Now apply Theorem 37.3.5 to conclude that
Z R
1
lim e(γ+iy)t L g (γ + iy) dy
R→∞ 2π −R
1 R iyt ∞ −iyu −γu
Z Z
= eγt lim e e e g (u) dudy
R→∞ 2π −R −∞
g (t+) e−γt+ + g (t−) e−γt− g (t+) + g (t−)
= eγt = .■
2 2
In particular, this shows that if L g (s) = L h (s) for all s large enough, both g, h having
exponential growth, then f , g must be equal except for jumps and in fact, at any point
where they are both Holder continuous from right and left, the mid point of their jumps is
the same.
This answers the question raised earlier about whether the Laplace transform method
even makes sense to use because it shows that if two functions have the same Laplace
transform, then they are the same function except at jumps where the midpoint of the jumps
coincide.
Next is a systematic way to invert the Laplace transform. It will be no harder than what
is usually done in standard differential equations courses but differs from this material in
being completely general.

37.5 The Bromwich Integral

First pick γ > λ and write the integral on the left in 37.6 as a contour integral. Thus
z = γ + iy and dz = idy and this is just the contour integral
1
Z γ+iR
eut L g (u) du
2πi γ−iR

where the contour is the straight line from γ − iR to γ + iR. Indeed, if you parametrize this
contour as z = γ + iy and use the procedures for evaluation of contour integrals, you get the
integral in 37.6. Then taking the limit as R → ∞ it is customary to write this limit as
1
Z γ+i∞
eut L g (u) du
2πi γ−i∞

This is called the Bromwich integral and as shown earlier it recovers the mid point of the
jump of g at t for every point t where g is Holder continuous from the right and from the
left. Remember t ≥ 0. Now u → eut L g (u) is analytic for Re (u) > η and in particular for
Re (u) ≥ γ therefore, all of the poles of u → L g (u) are contained in the set Re (u) < γ.
Indeed, in practice, u → L g (u) ends up being represented by a formula which is clearly a
meromorphic function, one which is analytic except for isolated poles.
So how do you compute this Bromwich integral? This is where the method of residues
is very useful. Consider the following contour.
760 CHAPTER 37. SOME FUNDAMENTAL FUNCTIONS AND TRANSFORMS
y x=γ
Let η R be the above contour oriented as shown. The radius of the
circular part is R. Let CR be the curved part. Then one can show that
x
under suitable assumptions
1
Z
lim eut F (u) du = 0 (37.7)
R→∞ 2πi CR

The needed condition is that for all |z| large enough,

C
|F (z)| ≤ , some α > 0. (37.8)
|z|α
Note that this assumption implies there are finitely many poles for F (z) because if w is a
pole, you have limz→w |F (z)| = ∞. Thus all poles are in some disk of suitable radius. Also
recall that poles have no limit point. Thus there are only finitely many in a suitably large
disk and this accounts for all of them.
Lemma 37.5.1 Let the contour be as shown and assume 37.8. Then the above limit in 37.7
exists.
Proof: Assume c ≥ 0 as shown and let θ be the angle between the positive x axis and a
point on CR . Let 0 < β < α. Then the contour integral over CR will be broken up into three
pieces, two pieces around the y axis
hπ c π  c i
θ ∈ − arcsin , + arcsin 1−β ,
2 R 2 R
 c 
3π  c  3π
− arcsin 1−β , + arcsin ,
2 R 2 R
and the third having
  c  3π  c 
π
θ∈ + arcsin 1−β , − arcsin 1−β
2 R 2 R
Then,
 
Z Z 3π −arcsin c  
2 R1−β
tz
e F (z) dz = π +arcsin

c
 e(R cos θ +iR sin θ )t F Reiθ Rieiθ dθ + (37.9)
CR 2 1−β
R

 
Z π +arcsin c  
2 R1−β
+ π −arcsin c
e(R cos θ +iR sin θ )t F Reiθ Rieiθ dθ
2 R( )
3π +arcsin c
Z
2 R( ) (R cos θ +iR sin θ )t
 
+ 3π −arcsin

c
e F Reiθ Rieiθ dθ
2 R1−β

Consider the last two integrals first. For large |z| , with z ∈ CR∗ , the sum of the absolute
values of these is no more than
 
π +arcsin c Z 3π +arcsin( c )
C R(cos θ )t C
Z
2 R1−β 2 R
eR(cos θ )t
Rdθ +  e Rdθ
π −arcsin c
2 (R) Rα 3π −arcsin
2
c Rα
R1−β
c
  c   c 
≤ CeR(cos( 2 −arcsin( R )))t arcsin 1−β + arcsin
π
R1−α
 R c  R c 
3π c
+CeR(cos( 2 +arcsin( R )))t arcsin 1−β + arcsin R1−α
R R
37.5. THE BROMWICH INTEGRAL 761

Now from trig. identities, cos π2 − arcsin (θ ) = θ , cos 3π

2 + arcsin (θ ) = θ , and so
the above reduces to
  c   c 
2Cect arcsin 1−β + arcsin R1−α
R R
which converges to 0 as R → ∞. Recall 0 < β < α. It remains to consider the integral in
37.9. For large |z| , the absolute value of this integral is no more than
3π −arcsin c
( )
  
C c
Z π +arcsin
2 R Rt cos β
eR(cos θ )t Rdθ ≤ Cπe 2 R1−β R1−α = CπR1−α e−cR
π +arcsin c
2 ( )
R
Rα
which converges to 0 as R → ∞. ■
Corollary 37.5.2 Let the contour be as shown and assume 37.8 for meromorphic F (s).
Then the above limit in 37.7 exists. Also f (t) , given by the Bromwich integral, is continuous
on (0, ∞) and its Laplace transform is F (s).
Proof: It only remains to verify continuity. Let R be so large that the above contour
η ∗R encloses all poles of F. Then for such large R, the contour integrals are not changing
because all the poles are enclosed. Thus
1 1
Z Z
f (tˆ) = lim eutˆF (u) du = eutˆF (u) du
R→∞ 2πi ηR 2πi ηR

1
Z
| f (tˆ) − f (t)| ≤ f (tˆ) − eutˆF (u) du
2πi ηR
1 1
Z Z
+ eutˆF (u) du − eut F (u) du
2πi ηR 2πi ηR

1
Z
+ eut F (u) du − f (t)
2πi ηR

1 1
Z Z
= eutˆF (u) du − eut F (u) du
2πi ηR 2πi ηR

Since η R is fixed, it follows that if |tˆ − t| is small enough, then | f (tˆ) − f (t)| is also small.
■
It follows from Lemma 37.5.1 that
1 γ+iR Z
f (t) ≡ lim eut F (u) du
R→∞ 2πi γ−iR
 
1 1
Z γ+iR Z
ut ut
= lim e F (u) du + e F (u) du
R→∞ 2πi γ−iR 2πi CR

1
2πi sum of residues of the poles of ezt F (z)


=
2πi
= sum of residues.
The following procedure shows how the Bromwich integral can be computed to obtain
an actual formula for a function. However, the integral itself will make sense and could be
numerically computed to solve for the inverse Laplace transform.
762 CHAPTER 37. SOME FUNDAMENTAL FUNCTIONS AND TRANSFORMS

PROCEDURE 37.5.3 Suppose F (s) is a Laplace transform and is meromor-

phic on C and satisfies 37.8. (This situation is quite typical) Then to compute the function
of t, f (t) whose Laplace transform gives F (s) , do the following. Find the sum of the
residues of ezt F (z) for Re z < γ where all the poles of F (z) have real part less than γ. This
yields the midpoint of the jump of f (t) at each t where f is Holder continuous from the
left and right. (Note there are no jumps by Corollary 37.5.2 so if f is Holder continuous at
every point, then f (t) is recovered.)
s
Example 37.5.4 Suppose F (s) = 2 . Find f (t) such that F (s) is the Laplace trans-
(s2 +1)
form of f (t).

There are two residues of this function, one at i and one at −i. At both of these points
the poles are of order two and so we find the residue at i by
!
d ets s (s − i)2 −iteit
res ( f , i) = lim =
s→i ds (s2 + 1)2 4

and the residue at −i is

!
d ets s (s + i)2 ite−it
res ( f , −i) = lim 2
=
s→−i ds (s2 + 1) 4

From the above procedure, the function f (t) is the sum of these.

ite−it −iteit 1
it e−it − eit


+ =
4 4 4
1
= it (cos (t) − i sint − (cost + i sint))
4
1
= t sint
2
You should verify that this actually works giving L ( f ) = s
2 .
(s2 +1)

Example 37.5.5 Find f (t) if F (s) , the Laplace transform is e−s /s.
est e−s
You need to compute the residues of s . The function equals

1 ∞ (−1)k (t − 1)k sk
∑ .
s k=0 k!

Thus the residue is 1. However, this fails to be the function whose Laplace transform is
F (s) . What is wrong? The problem with this is the failure of the estimate on F (s) to hold
for large s. Indeed, if s = −n, you would have en /n but it would need to be less than C/nα
which is not possible. The estimate requires F (s) → 0 as |s| → ∞ and this does not happen
here. You can verify directly that the function which works is u1 (t) which is 0 for t < 1
and 1 for t ≥ 1. Thus if the estimate does not hold, the procedure does not necessarily hold
either.
37.5. THE BROMWICH INTEGRAL 763

If Re p < γ for all p a pole of F (s) and if F (s) is meromorphic and satisfies the growth
condition 37.8, and if f (t) is defined by that Bromwich integral, is it true that F (s) is the
Laplace transform of f (t) for large s? Thus
Z R
1 1
Z γ+i∞
f (t) ≡ lim e(γ+iy)t F (γ + iy) dy = ezt F (z) dz
R→∞ 2π −R 2πi γ−i∞

The limit must exist because, as discussed above,

Z γ+iR 
1
Z
zt zt
lim e F (z) dz + e F (z) dz
R→∞ 2πi γ−iR CR

is eventually constant because the contour will have enclosed all poles of F (z), but as R
continues to increase, the integral over the curved part CR converges to 0. Let Re s be larger
than γ. One needs to consider

1
Z ∞ Z γ+i∞
L ( f ) (s) = e−st ezt F (z) dzdt
0 2πi γ−i∞
1
Z ∞ Z
−st
= e lim ezt F (z) dzdt
2πi 0 R→∞ η R

This equals Z r
1
Z
lim e−st lim ezt F (z) dzdt
r→∞ 2πi 0 R→∞ η R

Eventually, for all R large enough, the contour includes all of the finitely many poles of
F (z). There are only finitely many poles because of the estimate on F (z). Thus we can
pick R large enough that the limit on the inside equals the contour integral. Thus
1
Z ∞ Z
L ( f ) (s) = e−st ezt F (z) dzdt
2πi 0 ηR
Z r
1
Z
= lim e−st ezt F (z) dzdt
r→∞ 2πi 0 ηR

Interchanging the two contour integrals, Theorem 35.3.8,

Z r
1
Z
= lim e−(s−z)t F (z) dzdt
r→∞ 2πi ηR 0
!
1 1 e−(s−z)r
Z
= lim − F (z) dz
r→∞ 2πi ηR s−z s−z

1 F (z)
Z
= dz
2πi ηR s−z
Now this contour integral is not zero because F (z) is not analytic on the inside of η ∗R . Let
the orientation of η R be switched and call the new contour η̂ R . Then

1 F (z)
Z
L ( f ) (s) = dz
2πi η̂ R z−s
764 CHAPTER 37. SOME FUNDAMENTAL FUNCTIONS AND TRANSFORMS

Is this equal to F (s)? Consider a large circular contour of radius M where M > |s| and
orient it counter clockwise about s as shown in the following picture. Denote this oriented
curve as η M .
y

x=c
x
s

Then from the estimate assumed on F,

F (z) C 1
Z
dz ≤ α 2πM
ηM z−s M M − |s|

Now as M → ∞, this converges to 0. Therefore, from the usual Cauchy integral formula,
Z 
1 F (z) F (z)
Z
F (s) = dz + dz
2πi η̂ R z − s ηM z − s

Now take a limit of both sides as M → ∞ and you obtain

1 F (z) 1 F (z)
Z Z
F (s) = dz = dz
2πi η̂ R z−s 2πi ηR s−z

Thus this shows the following interesting proposition. This proposition shows conditions
under which a meromorphic function is the Laplace transform of a function which happens
to be given by the Bromwich integral and they are the conditions used earlier.

Proposition 37.5.6 If Re p < γ for all p a pole of F (s) and if F (s) is meromorphic and
satisfies the growth condition 37.8, and if f (t) is defined by the Bromwich integral, then
F (s) is the Laplace transform of f (t) for large s.

37.6 Exercises
2
1. Let F (s) = so it is the Laplace transform of some f (t). Use the method of
(s−1)2 +4
residues to determine f (t).
2. This problem is about finding the fundamental matrix for a system of ordinary dif-
ferential equations
Φ′ (t) = AΦ (t) , Φ (0) = I
having constant coefficients. Here A is an n × n matrix and I is the identity matrix.
A matrix, Φ (t) satisfying the above is called a fundamental matrix for A. In the
following, s will be large, larger than the magnitude of all poles of (sI − A)−1 .
R 
(·)
(a) Show that L 0 f (u) du (s) = 1s F (s) where F (s) ≡ L ( f ) (s)

(b) Show that L (I) = 1s I where I is the identity matrix.

37.6. EXERCISES 765

(c) Show that there exists an n × n matrix Φ (t) such that L (Φ) (s) = (sI − A)−1 .
Hint: From linear algebra
  cof (sI − A) ji
(sI − A)−1 =
ij det (sI − A)

Show that the i jth entry of (sI − A)−1 satisfies the conditions of Proposition
37.5.6 and so there exists Φ (t) such that L (Φ) (s) = (sI − A)−1 . By Corollary
37.5.2, this t → Φ (t) is continuous.
(d) Thus (sI − A) L (Φ) (s) = I. Then explain why I − 1s A L (Φ) (s) = 1s I =

L (I) and

1
L (Φ) (s) − L (AΦ) (s) = L (I)
s
Z (·) 
L (Φ) − L AΦ (u) du = L (I)
0

so Z t
Φ (t) − AΦ (u) du = I
0
and so Φ is a fundamental matrix.
(e) Next explain why Φ must be unique by showing that if Φ (t) is a fundamen-
tal matrix, then its Laplace transform must be (sI − A)−1 and use the theorem
which says that if the two continuous functions have the same Laplace trans-
form, then they are the same function.

3. In the situation of the above problem, show that there is one and only one solution to
the initial value problem

x′ (t) = Ax (t) + f (t) , x (0) = x0 ,t ≥ 0

and it is given by Z t
x (t) = Φ (t) x0 + Φ (t − u) f (u) du
0
R 
(·)
Hint: Verify that L 0 Φ (t − u) f (u) du (s) = L (Φ) (s) L (f ) (s) . Thus if x is
given by the variation of constants formula just listed, then

L (x) (s) = (sI − A)−1 x0 + (sI − A)−1 L (f ) (s)

(sI − A) L (x) (s) = x0 + L ( f )

Now divide by s and verify x (t) = x0 + 0t Ax (u) du + 0t f (u) du. You could also
R R

simply differentiate the variation of constants formula using chain rule and verify it
works.
766 CHAPTER 37. SOME FUNDAMENTAL FUNCTIONS AND TRANSFORMS
 Part IV

Probability and Statistics

767
Chapter 38

Probability

38.1 Improper Integrals

If f is Riemann integrable on [0, R] for each R, then
Z ∞ Z R
f (x) dx ≡ lim f (x) dx
0 R→∞ 0

if this limit exists. Otherwise the improper integral is not defined. If f is only Riemann
integrable on [δ , R] for each δ < R, then
Z ∞ Z R
f (x) dx ≡ lim f (x) dx
0 (δ ,R)→(0,∞) δ
R∞
provided this limit exists. This expression means: There exists I ≡ 0 f (x) dx such that for
each ε > 0 there is R0 and δ 0 such that if δ < δ 0 and R > R0 , then
Z R
f (x) dx − I < ε
δ
R0
Otherwise we don’t give a definition
R∞
of the improper integral. Integrals of the form −∞ f (x) dx
are defined similarly. As to −∞ f (x) dx, it equals
Z ∞ Z 0
f (x) dx + f (x) dx
0 −∞

provided these last two exist. As an application of polar coordinates, here is an important
theorem.
2 √ R ∞ −x2 √
Theorem 38.1.1 0∞ e−x dx = 12 π and −∞
R
e dx = π.
R R −x2 2 2
dx. Then IR IR = 0R 0R e−x e−y dx. Also
R R
Proof: Let IR ≡ 0 e

I ≡ lim IR
R→∞

769
770 CHAPTER 38. PROBABILITY

also exists. This is left as an exercise. Let DR be the quarter circle centered at (0, 0) with
2 2
radius R. Then using polar coordinates to write DR e−(x +y ) dx,
R

Z R Z π/2 Z RZ R
2 2 +y2
IR2 = e−r rdθ dr + √ e−(x ) dydx
0 0 0 R2 −x2

That second integral satisfies

Z RZ R Z RZ R
2 +y2 2 +R2 −x2
0 ≤ √ e−(x ) dydx ≤ √ e−(x ) dydx
0 R2 −x2 0 R2 −x2
Z RZ R
−R2 2
≤ e dydx = R2 e−R
0 0

which converges to 0 as R → ∞. Therefore,

2
π −e−r R π
Z R Z π/2
2
I 2 = lim e−r rdθ dr = lim | =
R→∞ 0 0 R→∞ 2 2 0 4
√ √
π
and so I = 2 .Then the other integral is obviously equal to π. ■
An alternative way to establish this integral is as follows.
Z x
2 Z x
2 2 2
F (x) ≡ e−t dt , F ′ (x) = 2 e−t dte−x
0 0
Z 1 Z 1
−x2 t 2 −x2 2 2
= 2x e dte = 2xe−x (t +1) dt
0 0

Z xZ 1 Z 1Z x
2 2 2 2
F (x) = 2ye−y (t +1) dtdy = 2ye−y (t +1) dydt
0 0 0 0
−y2 (t 2 +1)
Z 1
!
e
= − 2 |x
0 t +1 0
2 2
!
e−x (t +1)
Z 1
1 π
= 2
− 2 dt = − e (x)
0 1+t t +1 4

2
R 2
∞ −t 2
where |e (x)| < e−x . It follows on taking a limit that 0 e dt = π4 .
√
Corollary 38.1.2 Γ (1/2) = π
R ∞ −t −1/2
Proof: By definition it is 0 e t dt. Let t = u2 so dt = 2udu. Then, changing the
variables,
1√ √
Z ∞ Z ∞
−u2 −1 2
Γ (1/2) = e u 2udu = 2 e−u du = 2 π= π■
0 0 2
38.2. COMBINATIONS 771

38.2 Combinations
The fundamental problem is to find the number of ways of selecting a subset of k ≤ n
elements from a set having n elements. For example, consider the set S = {1, 2, 3} . How
many subsets having two elements are there? In this case, you can simply list them. Here
they are
{1, 2} , {1, 3} , {2, 3}
This seems easy enough, but what if you had a set of 52 things like a deck of cards and you
wanted the number of ways of picking a set of 5 things from it. Then it would be a little
harder. Here is some standard notation.
!
n
Definition 38.2.1 Let 0 ≤ k ≤ n. Then denotes the number of subsets of a set
k
having n elements which have k elements.

Here are some obvious assertions.

! ! !
n n n
= = 1, =n (38.1)
0 n 1

The first says there is one subset which has no elements in it. Of course it is the empty
set. The next says there is one subset of a set having n things which has n things in it. Of
course, this would be the whole set itself. The last says there are!n subsets which have a
n
single element of the set in them. Now to get a formula for , here is a lemma.
k

Lemma 38.2.2 Let n be a positive integer and let 1 ≤ k ≤ n. Then

! ! !
n+1 n n
= +
k k k−1

Proof: Letting 1 ≤ k ≤ n, suppose your set of n + 1 things is

{a1 , · · · , an , an+1 }

Here ai denotes the ith element of the set and this is just a list of the elements of the set.
Then there are two ways to select a set of k things from this !set depending on whether an+1
n
is in the set of k things. If it is, there are exactly ways to obtain such a set of
k−1
k things because it must be the number of ways of selecting the remaining k − 1 elements
from the first n elements in the set. The other case is where all
! k elements are selected from
n
the first n elements of the set. By definition, there are ways to do this. Thus
k
! ! !
n+1 n n
= +
k k−1 k

■
772 CHAPTER 38. PROBABILITY

Definition 38.2.3 Let 0! ≡ 1 and for n ∈ N, n! ≡ n (n − 1) (n − 2) · · · 1. This is called the

factorial symbol. We say n! as n factorial.
!
n
With this definition, it is easy to give a simple description of .
k

Theorem 38.2.4 Let 0 ≤ k ≤ n. Then

!
n n!
=
k k! (n − k)!

Proof: You see easily this is true if n = 1. In this case, the only possibilities for k are
0, 1 the the formula gives the right answer in either of these cases. Assume the formula
holds for n. Then by Lemma 38.2.2 and the induction hypothesis, if 1 ≤ k ≤ n
! ! !
n+1 n n
= +
k k−1 k

n! n!
= +
(k − 1)! (n − k + 1)! k! (n − k)!
kn! n! (n − k + 1)
= +
k! (n − k + 1)! k! (n − k)! (n − k + 1)
kn! (n − k + 1) n! (n + 1) n! (n + 1)!
= + = =
k! (n − k + 1)! k! (n − k + 1)! k! (n + 1 − k)! k! (n + 1 − k)!
and so this proves the formula in the case that 1 ≤ k ≤ n. If k = 0 or n + 1, the definition of
the factorial symbol and the obvious observations of 38.1 shows the formula holds in these
cases also. ■
Notice that ! !
n n
= .
k n−k

38.3 The Binomial Theorem

The Binomial theorem is one of the most useful and fundamental theorems in algebra. It is
easy to prove from the above using induction. Here it is.

Theorem 38.3.1 Let n ∈ N. Then

!
n
n n
(a + b) = ∑ an−k bk
k=0 k

Proof: In case n = 1, both sides reduce to a + b so it works in this case. Suppose now
it works for n. Then by induction,
!
n
n+1 n
(a + b) = (a + b) ∑ an−k bk
k=0 k
38.3. THE BINOMIAL THEOREM 773
! !
n n
n n
= ∑ an+1−k bk + ∑ an−k bk+1
k=0 k k=0 k
! !
n n+1
n n+1−k k n
= ∑ a b +∑ an+1−k bk
k=0 k k=1 k−1
! !
n n
n+1 n n+1−k k n
=a +∑ a b +∑ an+1−k bk + bn+1
k=1 k k=1 k−1
! !!
n
n+1 n n
=a +∑ + an+1−k bk + bn+1
k=1 k k−1
By Lemma 38.2.2 this reduces to
!
n
n+1 n+1
a +∑ an+1−k bk + bn+1
k=1 k
!
n+1
n+1
= ∑ an+1−k bk
k=0 k
which shows that when the formula holds for n it also holds for n + 1. ■
Another way to verify this important formula is as follows. For n a positive integer
(a + b)n must be of the form (a + b) (a + b) · · · (a + b) and it must consist of a sum of terms
of the form ak bn−k . How many are there for a given k? This involves the number of ways
to pick k factors in the product which contribute! a and the remaining factors contributing
n
b. Thus the coefficient of this term is . As to the case where k = 0, this means
k
all factors k n−k and this is
! contribute b and so there is only one way to obtain this term a b
n
. Thus the above product of terms reduces to
0
!
n
n
∑ k an−k bk
k=0

Example 38.3.2 Find the coefficient which multiplies x3 y6 in (x + y)9 .

By the binomial theorem, this is
!
9 9!
= = 84
3 3!6!

Thus (x + y)9 is the sum of terms ck xk yn−k and the ck which corresponds to k = 3 is 84.

8
Example 38.3.3 Find the constant coefficient of 2x + 3x−3 .

k
You have that this is the sum of constants times x8−k x−3 and so you need to have
8 − k − 3k = 0 so k = 2. It follows that this term is of the form
!
8
2 8! 6 2
(2x)6 3x−3 = 2 3 = 16 128
2 2!6!
774 CHAPTER 38. PROBABILITY

38.4 Exercises
1. Use the binomial theorem to expand or simplify the following.

(a) (x + y)5
(b) (x − y)5
(c) (x − y)4
(d) (x + h)3 − x3
(e) (x + h)4 − x4
 
(f) h−1 (x + h)5 − x5
 
(g) h−1 (x + h)6 − x6

2. Show that for a positive integer and x > 0, (1 + x)n ≥ 1 + nx.

!
n
3. Show that ∑nk=0 = 2n .
k

4. Approximate 100 (1. 005)12 . This would be the amount in the bank after one year if
interest is 6% compounded monthly.
5. Show that for k ≥ 1,
k factors
! z }| {
n n (n − 1) · · · (n − k + 1)
= .
k k!

38.5 Counting and Basic Probability

You do an experiment n times and there are two possible outcomes to this experiment
each time it occurs, a “success” having probability p, a positive number less than 1 and a
“failure” having probability (1 − p) . For example, you could have p be the probability of
getting a 4 when you roll a pair of fair dice. What would this probability be? Here is a table
of possible outcomes for the pair of dice.

1, 1 1, 2 1, 3 1, 4 1, 5 1, 6
2, 1 2, 2 2, 3 2, 4 2, 5 2, 6
3, 1 3, 2 3, 3 3, 4 3, 5 3, 6
4, 1 4, 2 4, 3 4, 4 4, 5 4, 6
5, 1 5, 2 5, 3 5, 4 5, 5 5, 6
6, 1 6, 2 6, 3 6, 4 6, 5 6, 6

The first number represents the one on the first die and the second represents the number
on the second die. (die is singular for dice) How many ways are there to get a 4? From
the table, there are exactly 3 ways, (3, 1) , (2, 2) , (1, 3). How many possible outcomes are
38.5. COUNTING AND BASIC PROBABILITY 775

there? There are 36. Thus if every outcome is as likely as any other, the probability of
rolling a 4 is 3/36 or 1/12.
Now in a succession of rolls of the dice, the probability of a particular outcome on roll
k is not affected by what happened on earlier rolls of the dice. Each time the dice are rolled,
the probability of rolling a four is 1/12 and the probability rolling a non four is 11/12.
What is the probability of rolling a 5 twice in a row? In this case there would be 362
possible outcomes and only 42 of them are favorable to rolling two fives in succession.
(Four possibilities for the first roll of the dice and for each of these, four for the second.)
Thus the probability of this occurring is
42 1
2
=
36 81
What about the probability of a five on the first roll and a non five on the second? This
probability is
4 32 8
· = .
36 36 81
You can determine this the same way by counting the ways favorable to the desired outcome
and dividing this by the number of possible outcomes.
4 · 32 8
2
=
36 81
Similarly, the probability of rolling a non five followed by a five would be
32 4 8
· =
36 36 81
More generally, the probability of getting k fives and n − k non fives in a particular order
would be  k  n−k
4 32
.
36 36
More generally, you have a situation where the probability of k success with probability
p and (n − k) failures happening with probability q ≡ (1 − p) in any particular order is
pk qn−k . What is the probability of having k successes in n trials? This is known as the
binomial distribution. How many ways can k success happen in n trials? It can ! happen
n
exactly the number of ways there are of selecting k of the n trials. There are ways
k
for this to happen. Therefore, since each of these has the same probability, pk qn−k , the
probability of k successes in n trials is
!
n
pk qn−k
k

This motivates the following definition of the binomial distribution and the idea of a
random variable.
Definition 38.5.1 Define a “random variable” X to be the number of successes, each hav-
ing probability p in n trials. Thus X has values 0,1,· · · , n. If the probability that X has
value k, written
P (X = k)
776 CHAPTER 38. PROBABILITY

is given by !
n
P (X = k) = pk qn−k
k
then X is said to have a binomial distribution.

Note that the sum

n
∑ P (X = k)
k=0

needs to equal 1 because the random variable must achieve one of the numbers 0, 1, 2, · · · , n.
This occurs by the binomial theorem,
!
n n
n
∑ P (X = k) = ∑ k pk qn−k = (p + q)n = 1n = 1.
k=0 k=0

There is a general principle of counting which should be mentioned. Suppose you have
m “positions” and n different things. How many ways are there to fill the m positions with
the n things? There are n choices for the first, and having filled this position, there are
n − 1 left to place in the second. Thus the number of ways to fill the first two positions
is n (n − 1) . Then, having filled these two, there are now n − 2 things left to place in the
third position and so there are n (n − 1) (n − 2) ways to fill the first three of these positions.
Continue this way till you run out of positions to fill. How many ways of filling them do
you obtain? You see that there are n (n − 1) (n − 2) · · · (n − m + 1) ways to do it. This is
called permutations of n things taken m at a time. See the exercise below.

Example 38.5.2 In a class of 12 students who are arranged in three rows of four students,
what is the probability that the particular four students, Eliphaz, Elihu, Zophar, and Bildad
will occupy the front four seats?

There are 4! ways for them to occupy these four seats in some order. There are 12 · 11 ·
10 · 9 ways to fill these seats in some order. Therefore, the probability is
4! 1
=
12 (11) (10) (9) 495

Of course, you don’t care about order in this problem so you could also do this in terms of
combinations of n things taken m at a time.
1 1
12!

=
4!8!
495

There is exactly one way to select these four students for the first four seats and then there
are 12!/ (4!8!) ways to fill these seats.

Example 38.5.3 In the above example involving 12 students, it is absolutely necessary for
disciplinary reasons that Eliphaz must not sit next to Elihu. If the students file in and sit
down randomly, what is the probability that Eliphaz ends up on the front right seat when
viewed by the teacher and is not sitting next to Elihu?
38.6. EXERCISES 777

There is one way to fill the front right seat with Eliphaz. Then there are 10 favorable
ways to fill the seat on the left side of Eliphaz with someone other than Eliphaz. There
are now 10 students left who can fill the remaining seats in any order because you have
used two. Thus there are 1 × 10 × 10 × 9 ways to have a favorable outcome. There are
12 × 11 × 10 × 9 ways for them to select seats at random. Therefore, the probability is
1 × 10 × 10 × 9 5
=
12 × 11 × 10 × 9 66

38.6 Exercises
1. Let k ≤ n where k and n are natural numbers. P (n, k) , permutations of n things taken
k at a time, is defined to be the number of different ways to form an ordered list of k
of the numbers {1, 2, · · · , n} . Show

n!
P (n, k) = .
(n − k)!

2. Now consider the word “mississippi”. By rearranging the letters, how many dis-
tinctly different words can you obtain? Note that for each list of these letters the four
different s are indistinguishable. There are therefore, 4! ways which are not really
different.
3. Using Problem 1, show the number of ways of selecting a set of k things from a set
n!
of n things is (n−k)!k! .

4. Prove by induction that n < 2n for all natural numbers n ≥ 1.

5. Prove by the binomial theorem and Problem 3 that the number of subsets of a given
finite set containing n elements is 2n .
6. Show that for p ∈ (0, 1) , ∑nk=0 nk kpk (1 − p)n−k = np.

7. Using the binomial theorem prove that for all n ∈ N,

1 n
   n+1
1
1+ ≤ 1+ .
n n+1
n
n·(n−1)···(n−k+1)
Hint: Show first that k = k! . By the binomial theorem,

k factors
z }| {
1 n n    k n
 
n 1 n · (n − 1) · · · (n − k + 1)
1+ =∑ =∑ .
n k=0 k n k=0 k!nk

n·(n−1)···(n−k+1)
Now consider the term k!nk
and note that a similar term occurs in the
1 n+1


binomial expansion for 1 + n+1 except that n is replaced with n + 1 wherever
this occurs. Argue the term got bigger and then note that in the binomial expansion
1 n+1


for 1 + n+1 , there are more terms.
778 CHAPTER 38. PROBABILITY

8. Let n be a natural number and let k1 + k2 + · · · kr = n where ki is a non negative

integer. The symbol  
n
k1 k2 · · · kr
denotes the number of ways of selecting r subsets of {1, · · · , n} which contain k1 , k2 · · · kr
elements in them. Find a formula for this number.
9. Is it ever the case that (a + b)n = an + bn for a and b positive real numbers?
√
10. Is it ever the case that a2 + b2 = a + b for a and b positive real numbers?
1
11. Is it ever the case that = 1x + 1y for x and y positive real numbers?
x+y
p
n
12. Derive a formula for the multinomial expansion, ∑k=1 ak which is analogous to
the binomial expansion. Hint: See Problem 8.
!
n
13. Let X be a binomial random variable. Thus P (X = k) = pk qn−k where p is
k
the probability of success and q = 1 − p is the probability of failure. The expected
value of X denoted as E (X) , is defined as
n
∑ kP (X = k) .
k=0

Show the expected value of X equals np.

14. The variance of the random variable in the above problem is defined as
n
σ2 ≡ ∑ (k − E (X))2 P (X = k)
k=0

Find σ 2 . You should get npq.

15. Find the probability of drawing from a shuffled deck of playing cards four hearts.
Hint: Use principles of counting to find the number of ways of drawing four hearts.
There are 13 of these. Now how many ways can you pull out four of them? Then
note there are 52 cards in all. How many ways can you pull out four cards from these.
16. Find the probability of obtaining 2 clubs and three spades from a shuffled deck of
cards.
17. A pond has N fish and 120 of these are marked fish. What is the probability in terms
of N of catching 10 fish, two of which are marked and 8 of which are unmarked?
18. Show that in general, for k ≤ m < N
! !
m N −m
k j k− j
∑ ! =1
j=0 N
k
38.7. GENERAL CONSIDERATIONS PROBABILITY 779

If X is a random variable having values in {0, 1, · · · , k} such that the probability that
X = j is given by the jth term of the above sum, then X is said to have a hyperge-
ometric distribution. Much much more can be said about this topic. Hint: If you
pick k things from N things m of which are marked and N − m unmarked, there are
various ways to do it determined by the value of j, the number of marked things out
of your sample of k things.
19. Suppose a pair of dice has one blue and the other one red. What is the probability
that when they are rolled the blue die delivers a strictly larger number than the red
die? Now what is the probability that either this happened or a 6 is rolled? What is
the probability that the blue is greater than the red and a 6 is rolled?

20. Recall the following table illustrating the possible outcomes of rolling a pair of dice.

1, 1 1, 2 1, 3 1, 4 1, 5 1, 6
2, 1 2, 2 2, 3 2, 4 2, 5 2, 6
3, 1 3, 2 3, 3 3, 4 3, 5 3, 6
4, 1 4, 2 4, 3 4, 4 4, 5 4, 6
5, 1 5, 2 5, 3 5, 4 5, 5 5, 6
6, 1 6, 2 6, 3 6, 4 6, 5 6, 6

Find the probability that you roll a 7 before you roll either a 3 or an 11. Hint: It can
happen in infinitely many distinct ways. You don’t roll either a 3 or an 11 for k rolls
and then on the kth roll you get a 7. Here k = 0, 1, 2, 3, · · · so you need to take a limit
of the partial sums associated with the different values of k and then take a limit. So
what is the probability of getting a 7 on try k + 1 and not getting either a 3 or an 11

k 1
before this? Argue it is 13
18 6.

21. Explain why in general, P (A ∪ B) = P (A) + P (B) − P (A ∩ B) where A, B are two

events such as in the above problem having the blue be larger than the red die or
rolling a 6.

22. Let X be the random variable which gives the number of heads when you flip a coin
6 times. Which value of X has the highest probability? What is the expected value
of X? What is the variance of X. For these last parts, see Problem 13 and 14 above.
23. You have a class of 12 students who will be seated in four rows consisting of three
students in each row. Jeroboam, Nadab, Baasha, and Elah must sit in the front for dis-
ciplinary reasons. Also, you absolutely must not have Baasha sitting next to Nadab
because Baasha is a thug who will attack Nadab. If Baasha is to sit on the front left
seat as viewed by the teacher, what is the probability that an acceptable outcome will
occur if the students take their seats completely at random?

38.7 General Considerations Probability

As mentioned earlier, X is a random variable if a probability is associated with X being
found in some set of possible values. The following examples have been discussed either in
the chapter or in the exercises. These are examples of discrete random variables because the
780 CHAPTER 38. PROBABILITY

random variable takes values in some subset of the integers. The following two examples
consider situations where X can only take finitely many values.
Example 38.7.1 Let an experiment be performed n times. Each time the experiment is
performed, the probability of a “success” is p and the probability of a “failure” is q, p +
q = 1. Then let X be the number of successes in the n experiments. The probability that
X = k, P (X = k) is !
n
pk qn−k
k
A distribution of this sort is called a binomial distribution.
Example 38.7.2 Let k ≤ m < N. If X is a random variable such P (X = j) , j ≤ k, is given
by ! !
m N −m
j k− j
P (X = j) ≡ !
N
k
this is called a hypergeometric distribution. This is when you have m marked fish and you
take a sample of k fish. Then X is the number of marked fish you get in your sample of k
fish. The probability it equals j is given by the above. Thus as explained in Problem 18 on
Page 778, ! !
m N −m
k j k− j
∑ ! =1
j=0 N
k
! !
m N −m
There are ways to get exactly j marked fish from a sample of k
j k− j
!
m
fish. You have ways to get j marked fish from the set of m marked fish and for
j
!
N −m
each of these, there are exactly ways to fill the set of k fish with non marked
k− j
fish. Thus ! ! !
k
m N −m N
∑ j =
j=0 k− j k
where the last is the total number of ways of selecting k fish from the N fish. Thus the
above claim is verified.
Now sometimes a random variable can take values from the set of all nonnegative in-
tegers. Suppose you have a binomial distribution in which the probability of a success is
extremely small and the number of trials is very large. Say pn = λ where n is large. Then
the probability of success in the n trials is
!  
λ k λ n−k

n
P (X = k) = 1−
k n n
38.7. GENERAL CONSIDERATIONS PROBABILITY 781

Thus, as n gets increasingly large and p correspondingly small,

λ n−k
 
n! 1
P (X = k) ≈ lim λk k 1−
n→∞ k! (n − k)! n n
λ n−k λ k −λ
 
n (n − 1) · · · (n − k + 1) k
= lim λ 1− = e
n→∞ k!nk n k!
Note that
∞
λ k −λ −λ
∞
λk
∑ k! e = e ∑ k! = e−λ eλ = 1.
k=0 k=0

Example 38.7.3 A random variable has Poisson distribution if for k a nonnegative integer,
λ k −λ
P (X = k) = e
k!
The sort of thing this models is the probability of being kicked by a mule k times in
some time interval of moderate length or the probability that k customers arrive at the check
out of a store in some 1 minute interval.
These random variables just discussed take values in a set of integers but often the
random variable takes values in the real numbers or Rn . When this is the case, you must
use an integral to determine the probability that the random variable is in some set. These
are called continuous random variables when you use a Riemann integral to determine the
probability that a random variable is in some set.
Example 38.7.4 Let (
x/2 if x ∈ [0, 2]
f (x) ≡
0 if x ∈
/ [0, 2]
R ∞
Thus −∞ f (x) dx = 1. Then f (x) is a distribution function for the random variable X if
P (X ∈ [a, b]) = ab f (x) dx. More generally, for all “suitable” sets F,
R

Z Z
P (X ∈ F) = f (x) dx ≡ XF (x) f (x) dx
F

where (
1 if x ∈ F
XF (x) ≡
0 if x ∈
/F
You really need the notions of measure spaces and Lebesgue integrals to do this right.
Now here is some terminology.
Definition 38.7.5 Two random variables X,Y are said to have the same distribution if for
all intervals I,
P (X ∈ I) = P (Y ∈ I)
Example 38.7.6 Let α > 0 and let f (x) ≡ Γ(α) 1
xα−1 e−x . Then the random variable X hav-
ing values in [0, ∞) has this as its distribution function if
Z b
f (x) dx = P (X ∈ [a, b]) .
a
R∞
Note that 0 f (x) dx = 1 from the definition of the gamma function.
782 CHAPTER 38. PROBABILITY

A modification of this density function gives the very important X 2 (r), chi squared,
distribution in the next example.

Example 38.7.7 Let r be a positive integer. The X 2 (r) distribution with r degrees of
freedom for a random variable having values in [0, ∞) is given by

1
f (x) = x(r/2)−1 e−x/2
Γ (r/2) 2r/2

Thus there are infinitely many of these, one for each r. I have no idea why they refer to r as
“degrees of freedom”.

It is necessary to verify that the above is a probability density. Thus it is necessary to

show that its integral is 1. This involves changing the variable. Let 2x = t so dx = 2dt then

1 1
Z ∞ Z ∞
x(r/2)−1 e−x/2 dx = (2t)(r/2)−1 e−t 2dt
0 Γ (r/2) 2r/2 0 Γ (r/2) 2r/2
1
Z ∞
= 2r/2 (t)(r/2)−1 e−t dt = 1
0 Γ (r/2) 2r/2

from the definition of Γ (r/2). The following picture gives the graphs of
Z x
1
Fr (x) ≡ t (r/2)−1 e−t/2 dt
0 Γ (r/2) 2r/2

for r = 2, 3, · · · , 8. Thus Fr (x) equals P (X ≤ x) where X is a X 2 (r) random variable.

1

0.5

0
0 5 10 15
As the number of degrees of freedom r increases, the graph becomes increasingly flat
near 0. This is good. Having many “degrees of freedom” is a fine thing because this
chi squared distribution can be used to estimate the variance of a normal distribution and
having the graph flat near 0 ends up meaning that you can be confident in a smaller upper
bound for the variance. This will be discussed more later. It turns out that having r large
is associated with having a large sample size. In other words, you are considering many
identically distributed random variables.

Example 38.7.8 Let X have values in R p . Then the density function of X will be f (x)
where Z Z
P (X ∈ A) = f (x) dV ≡ XA (x) f (x) dV
A
(
1 if x ∈ A
XA (x) =
0 if x ∈
/A
38.7. GENERAL CONSIDERATIONS PROBABILITY 783

where dV refers to the p dimensional volume. Thus the expression on the right is an integral
of a function of p variables. x = (x1 , · · · , x p ). We usually write dV as dx1 dx2 · · · dx p . Of
course the case p = 3 was discussed earlier and the higher dimensional case is exactly
similar. When it is desired to emphasize that X has values in R p it will be referred to as a
random vector and may be written in bold face.

The most important distribution is the normal distribution. It has two parameters and is
given as follows.

Example 38.7.9 Let µ, σ > 0 then a random variable X having values in R is normally
distributed if
Z b (x−µ)2
1 −
P (X ∈ (a, b)) = √ e 2σ 2 dx
2πσ a
The density function is then
1 (x−µ)2
−
√ e 2σ 2
2πσ
It is necessary to verify that this really is a density function. To do this, let
1 x−µ
y=
2 σ
Then, changing the variables in

1 (x−µ)2
Z ∞
−
√ e 2σ 2 dx
−∞ 2πσ
yields
1 y2 1 2√ 2
Z ∞ Z ∞ Z ∞
2
√ e− 2 dy = √ e−u 2du = √ e−u du
−∞ 2π −∞ 2π π 0

and from Theorem 38.1.1, this equals 1.

2
Often one reduces to the case that σ = 1 and µ = 0. Thus the density is √12π e−(x /2) .
I have heard people refer to random variables with this distribution as “standard normal
deviates”. Its graph is as follows.
0.4

0.2

0
-4 -2 0 2 4
You observe that if a random variable has this distribution defined by this probability
density function, it is very likely to assume a value between −2 and 2. The graph of
F (x) ≡ P (X ≤ x) for X a normally distributed random variable with µ = 0 and σ = 1
follows.
784 CHAPTER 38. PROBABILITY

0.5

0
-2 0 2
You can see that the probability that X ≤ 3 is very close to 1.

Example 38.7.10 The multivariate normal is as follows. The random variable X has
values in R p and its density function is of the form
1 −1 (x−m)∗ Σ−1 (x−m)
p/2 1/2
e 2
(2π) det (Σ)

Here Σ is the covariance matrix. It is a symmetric matrix with positive eigenvalues and
m ∈ R p is the mean. Thus
1
Z
−1 (x−m)∗ Σ−1 (x−m)
P (X ∈ A) = p/2 1/2
e 2 dx
A (2π) det (Σ)

You integrate over the set A the density function. Just as in the case of three dimensions, this
p
is easier said than done. However, if A has a simple form ∏k=1 (−∞, ak ], then P (X ∈ A) =
Z a1 Z a2 Z ap
1 −1 (x−m)∗ Σ−1 (x−m)
··· p/2 1/2
e 2 dx p · · · dx1
−∞ −∞ −∞ (2π) det (Σ)

Assuming there are no mathematical difficulties, the following is the definition of what
is meant by expectation.

Definition 38.7.11 Let X be a discrete random variable such that P (X = j) = f ( j). Then
if g is some function defined on the values of X, E (g (X)) ≡ ∑ j g ( j) f ( j) assuming the sum
makes sense. It is called the expected value of g (X) or simply the expectation of g (X). In
case X is a continuously
R
distributed random variable with density f (x) , the expectation of
g (X) is E (g (X)) ≡ g (x) f (x) dx, assuming the integral makes sense.

The two cases considered above are the discrete and continuously distributed cases for
random variables. However, this does not include all cases. To do this right, one needs
the notion of the Lebesgue integral and measure spaces and one defines exactly what a
random variable is, a measurable function defined on a measure space, instead of referring
to it vaguely in terms of the probability “it” has certain values or lies in some set called an
“event”. What does always happen is that, assuming everything makes sense,

E (aX + bY ) = aE (X) + bE (Y )

for two random variables X,Y and scalars a, b. Also, for any random variable X it may
or may not have a valid expectation, denoted as E (X) but in every case, it makes sense to
speak of P (X ∈ E) where E is some interval or more generally something called a Borel
set.
38.7. GENERAL CONSIDERATIONS PROBABILITY 785

Observation 38.7.12 In every case, if a, b are numbers, then if everything makes sense,
and X, X̂ are two random

variables having the same probability distribution, meaning that
P (X ∈ F) = P X̂ ∈ F for all F an interval, then



E aX + bX̂ = aE (X) + bE X̂

Suppose you had many random variables Xi each having the same distribution and the
collection of random variables independent, explained below. If you averaged Xi for all i,
what you would get is probably close to E (X). This is why taking the expectation is of
interest. I will give a brief explanation why this is so.
Where do independent random variables come from? In practice, you have independent
observations from an underlying probability space, meaning that it makes sense to ask for
the probability that a random variable is in suitable subsets of R or Rn . These observations
are independent in the sense that the outcome of an observation does not depend on the
outcome of the others. Then the numerical values are called independent random variables.
A more precise description is given below.
First, here is an important formula. I will be considering only the case of a continuous
distribution in explaining this inequality, but it all works in general. The inequality is called
the Chebychev inequality.

Proposition 38.7.13 Let X be a random variable. Then for ε > 0,

1
P (|g (X)| ≥ ε) ≤ E (|g (X)|)
ε

Proof: By definition of what is meant by a distribution function, if E ≡ |g|−1 ([ε, ∞)) ≡

{x : |g (x)| ≥ ε} , then on this set, |g (x)| /ε ≥ 1 and off this set, |g (x)| f (x) ≥ 0. Thus

1 1
Z Z
P (|g (X)| ≥ ε) = f (x) dx ≤ |g (x)| f (x) dx = E (|g (X)|) ■
E ε R ε
Now suppose you have Xi a random
 variable having distribution function f (x) and suppose
2 2
µ = E (X) , σ = E (X − µ) both exist. Suppose Xi , i = 1, ... all these random variables
are independent as in the next definition.

Definition 38.7.14 Let there be random variables X1 , ..., having well defined mean µ ≡
E (Xk ) and variance σ 2 = E (X − µ)2 . Then to say these are independent implies that
E ((Xi − µ) (X j − µ)) = E (Xi − µ) E (X j − µ) = 0 whenever i ̸= j. The more complete
meaning of independence is as follows: For each m,
m
P (Xi ∈ Ei for each i ≤ m) = ∏ P (Xi ∈ Ei ) .
i=1

Here the Ei can be considered intervals. The idea is that what happens in terms of proba-
bility involving each X j for j ̸= i does not affect probability involving Xi . Also, if you have
X1 , ..., independent, then if g is some continuous function, then g (X1 ) , ... will also be inde-
pendent. This is because g (Xi ) ∈ Ei if and only if Xi ∈ g−1 (Ei ). Then there is a significant
observation.
786 CHAPTER 38. PROBABILITY

Proposition 38.7.15 Suppose Xi for i = 1, ..., m are independent random variables and
E (Xi ) = µ while E (Xi − µ)2 = σ 2 . Then if Z ≡ m1 ∑m i=1 Xi is their average, then E (Z) =
 
µ and E (Z − µ)2 = σ 2 /m.

Proof: E (Z) = E m1 ∑m

1 m 1 m
i=1 Xi = m ∑i=1 E (Xi ) = m ∑i=1 µ = µ. Also, using the inde-
pendence of these random variables,
 !2 
m

2
 1
E (Z − µ) = E ∑ Xk − µ 
m k=1
 
m  !2
Xk − µ
= E ∑ 
k=1 m
!
1
= E ∑ (Xk − µ) (Xl − µ)
m2 k,l
1
= E ((Xk − µ) (Xl − µ))
m2 ∑
k,l

1 m  2
 σ2
= ∑ E (X k − µ) = ■
m2 k=1 m
Then it follows from this proposition and Proposition 38.7.13 the following important re-
sult which says that the average of observations of independent random variables which
have the same mean and same variance converges in probability to 0 as more and more
independent observations are taken. Actually much more can be said, but this is enough
here.
Proposition 38.7.16 Let Xi , i = 1..., be independent random variables with common mean
µ and common variance σ 2 then for Zm ≡ m1 ∑m k=1 Xk , the average of the first m,
 
lim P (Zm − µ)2 ≥ ε = 0
m→∞
Proof: This follows from the above propositions which imply
  1   1 2
P (Zm − µ)2 ≥ ε ≤ E (Zm − µ)2 = σ ■
ε εm
In words, this says that if you average independent observations (That is, the ith obser-
vation does not depend on the others. For example, you throw the marked fish back into the
lake and let them swim around before taking another observation.) then as you take more
and more of them, the probability that this average differs by very much from the true mean
becomes very small. This is a version of the law of large numbers. In words, the average is
probably close to the true mean if you average many independent observations.
Example 38.7.17 Let X have the hypergeometric distribution.
! !
m N −m
j k− j
P (X = j) = ! ,k ≥ 1
N
k
38.8. MOMENT GENERATING FUNCTIONS 787

where j ≤ k ≤ m < N. Find a formula for E (X).

This follows from a computation. The details are left to you.
! ! ! !
m N −m m N −m
j j
k j k− j k j k− j
E (X) ≡ ∑ ! =∑ !
j=0 N j=1 N
k k
! ! ! !
m m − 1 N − m m − 1 N − m
j
k j j−1 k− j k j−1 k− j
=∑ ! =m∑ !
j=1 N j=1 N
k k
! !
m−1 N − 1 − (m − 1)
k j−1 (k − 1) − ( j − 1)
=m∑ !
j=1 N N −1
k
k−1
! !
m−1 N − 1 − (m − 1)
mk k j−1 (k − 1) − ( j − 1)
= ∑ !
N j=1 N −1
k−1
! !
m−1 N − 1 − (m − 1)
mk k−1 j (k − 1) − j mk
= ∑ ! =
N j=0 N −1 N
k−1
Example 38.7.18 There are 100 fish in a pond and there are 30 fish which are marked.
You scoop up 20 fish with a large net and then throw them back after counting the number
of marked fish. If this is done repeatedly, then on average, about how many marked fish do
you expect to get?
From the formula, it would be (30)(20)
100 = 6. Note that if you did this a lot, you could
estimate how many fish are in the pond. You know there are 30 marked fish and so you
know m, k therefore, N should be such that your observed average is close to (30)(20)
N .
The expectations of most interest are E X k where k is a positive integer. These are

called the moments.

38.8 Moment Generating Functions

A very convenient gimmick for computing moments is the moment generating function.
Assuming there are no fussy mathematical issues, the moment generating function is
M (t) ≡ E etX


788 CHAPTER 38. PROBABILITY

Then, unless there are pathologies, you could write

M ′ (t) = E XetX , M ′′ (t) = E X 2 etX , etc.

Then you would simply let t = 0 and find various moments. The kth moment is E X k . In

all cases, you are using the fact that E is linear, either a sum or some sort of integral and
you interchange the derivative with the sum or integral. Of course the legitimacy of this
operation is in question, but in most cases of interest, there is no problem.
This will suffice for what is considered in this introduction, but the moment generating
function has some deficiencies. In particular, it might not exist.
A much better approach is the characteristic function

φ X (t) ≡ E eitX

because it always exists. It can be shown, although it won’t be attempted here, that the
distribution of the random variable is completely determined by the characteristic function.
However, you can see why this is so in case there is a continuous density function. Say
Z Z
eitx f (x) dx = eitx g (x) dx
R R

Then Z
eitx ( f (x) − g (x)) dx = 0
R
and by the Fourier inversion theorem adapted slightly, Theorem 37.3.5,
Z R
1
Z
−iyt
f (y) − g (y) = lim e eitx ( f (x) − g (x)) dxdt = 0
R→∞ 2π −R R

Thus it is not unreasonable to believe this assertion that if the two characteristic functions
coincide, then the densities are the same.
However, it is less trouble to use the moment generating functions because it does not
require fussing with complex numbers, and it can be shown that if two random variables
have the same moment generating function, then they have the same density, although it
has not been done in this book. Everything could be done just as well with the more general
characteristic functions.
The following definition includes the case where X is a random vector and gives the
above discussion as a special case.
 
Definition 38.8.1 Let X = X1 · · · X p be a random vector. The moment generating
function is defined as E et·X .

As mentioned above, if two random variables have the same moment generating func-
tion, then they will have the same distribution.

Example 38.8.2 Find the moment generating function for a binomial random variable X
and use to find some moments.

!
n
n
n
∑ etk pk qn−k = q + pet = M (t)
k=0 k
38.8. MOMENT GENERATING FUNCTIONS 789

Then M ′ (t) = npet (q + pet )n−1 . Then let t = 0 and you get M ′ (0) = np which is E (X).
Also
n−2
M ′′ (t) = npet q + pet q + npet

so E X 2 = np (q + np) .

Recall the following definition.

 
Definition 38.8.3 The variance of X is defined as E (X − E (X))2 . The mean is defined
as E (X).

This is a measure of how spread out the distribution is.

Example 38.8.4 Find the variance of X if X is a binomial random variable.

Note that in general,

   
E (X − E (X))2 = E X 2 − 2XE (X) + E (X)2

= E X 2 − 2E (X) E (X) + E (X)2

= E X 2 − E (X)2

Thus the variance of X for X a binomial random variable is np (q + np) − (np)2 = npq.

Example 38.8.5 Let X be normally distributed with parameters µ, σ 2 . Find the mean and
variance. In fact, show that the mean is µ and the variance is σ 2 . Determine the moment
generating function.

This will be done by using a moment generating function as above. For X normally
distributed,

Z∞ 1 (x−µ)2
tX −
M (t) ≡ E e = √ e 2σ 2 etx dx
−∞ 2πσ
 
1 1
Z ∞ 

2

=√ exp − 2 x − µ + tσ 2 − t 2 σ 4 + 2tσ 2 µ dx
2πσ −∞ 2σ
after simplification and completing the square. Thus this equals
   
1 2 2 1 1 
Z ∞


2 2
exp t σ + µt √ exp − 2 x − µ + tσ dx
2 2πσ −∞ 2σ

Change the variable in that integral on the right. Let

1
x − µ + tσ 2 = u



√
2σ

so √1 dx = du. Then the expression becomes

2σ

√
 
1 2 2 1
Z ∞
exp t σ + µt √ exp −u2 dx 2σ
2 2πσ −∞
 
1 2 2
= exp t σ + µt = M (t) (38.2)
2
790 CHAPTER 38. PROBABILITY

In this case there are no mathematical pathologies and so

1 2 σ 2 +µt
M ′ (t) = e 2 t tσ 2 + µ

so letting t = 0 yields E (X) = µ. Then also

1 2 σ 2 +µt
M ′′ (t) = e 2 t t 2 σ 4 + 2tσ 2 µ + σ 2 + µ 2

and so E X 2 = M ′′ (0) = σ 2 + µ 2 . Then the variance is E X 2 −E (X)2 = σ 2 + µ 2 − µ 2 =

σ 2 showing the identification of these parameters.

Example 38.8.6 Let X be a X 2 (r) distribution. Find the moment generating function
valid for t in some interval containing 0.

By definition, this is
1 1
Z ∞ ∞ Z
1
r/2
x(r/2)−1 e−x/2 etx dx = r/2
x(r/2)−1 e−x( 2 −t ) dx
0 Γ (r/2) 2 0 Γ (r/2) 2

so change the variable letting u = x 12 − t so du = 21 − t dx. Let |t| < 21 . Then the integral

is
 (r/2)−1
1 u 1
Z ∞
e−u du
0 Γ (r/2) 2r/2 (1/2 − t) (1/2 − t)
1 1 1
Z ∞
= r/2
ur/2−1 e−u du =
2r/2 (1/2 − t) Γ (r/2) 0 (1 − 2t)r/2

Now with this, you can find all the moments desired.

Proposition 38.8.7 Suppose X is normally distributed with mean µ and variance σ 2 . Then
X−µ
σ is normally distributed with mean 0 and variance 1.

Proof: This is real easy to do with the moment generating technique.

  
X −µ  t tµ   t   tµ 
E exp t = E exp X− = E exp X exp −
σ σ σ σ σ

Now t
σX and − tµ
σ are independent. (Check the definition.) Therefore, the above reduces
to
1 σ 2 t2
 t    tµ   
−t
 t 1 2
E exp X E exp − = eσ µe2 σ2 e σ µ = e 2 t
σ σ
which is the moment generating function of a random variable which is normally dis-
tributed with mean 0 and variance 1. ■
You might call X−µ
σ a standard normal deviate.

Corollary 38.8.8 Let X be normally distributed with mean µ and variance σ 2 . Then
 2
X−µ
σ is distributed as X 2 (1).
38.9. INDEPENDENCE AND CONDITIONAL PROBABILITY 791
 
Proof: From Proposition 38.8.7, X−µ σ is normally distributed with mean µ and vari-
 2
ance σ 2 . If f (t) is the density of X−µ
σ , then

2 !
Z x
√ X −µ √
  
X −µ
F (x) ≡ f (t) dt ≡ P <x =P − x< < x
0 σ σ

Z √x √ Z√
1 2 x
− 12 t 2 1 2
=√ √ e dt = √ e− 2 t dt
2π − x π 0
t2 √du .
change variables. Let 2 = u so tdt = du, dt = 2u
Then the above is
√ Z
2 x/2
√ √ u−1/2 e−u du
2 π 0
Then, taking the derivative will yield the density. This is

1 1  x −1/2 −x/2 1 1
√ e = √ √ √ e−x/2
π2 2 π 2 x
1
= x1/2−1 e−x/2
Γ (1/2) 21/2

because of Corollary 38.1.2, which is the density for X 2 (1) as claimed. ■

38.9 Independence and Conditional Probability

In the above, the concept of Probability that a random variable is in some set has been
considered. More generally, you have a set and a collection of subsets of this set and a
function which assigns a number between 0 and 1 to sets in this collection. This is the
probability function. The following has to do with conditional probability.

Definition 38.9.1 Let C be a collection of sets contained in some universal set U. These
could be intervals on the real line for example, and U could be R. Let P : C → [0, 1]. Thus
for A a set, P (A) ∈ [0, 1] . It satisfies the following conditions.

1. If Ai are disjoint sets in C , then P ∪ni=1 Ai = ∑ni=1 P (Ai ) . More generally, if you


∞
have infinitely many such disjoint sets, P (∪∞ i=1 Ai ) = ∑i=1 P (Ai ).

2. If A ∈ C , then P (A) + P (U \ A) = 1.

Then one defines the conditional probability as follows. If P (B) ̸= 0,

P (A ∩ B)
P (A|B) ≡
P (B)

The sets in C are called events.

792 CHAPTER 38. PROBABILITY

To do this right, you should be using σ algebras and measures on an abstract probability
space. However, these things are not discussed in this book.
The words used when you write P (A|B) are: probability of A given B. In other words,
if you are considering random variables, you know that X ∈ B where B is some possibly
smaller set than U. For example, you might know that a normally distributed random
variable is in [1, 5] and given this knowledge, the appropriate probability function would be
defined as
P (X ∈ A and X ∈ [1, 5])
P (X ∈ A|X ∈ [1, 5]) =
P (X ∈ [1, 5])
This really restricts the set U to B and A → P (A|B) is a probability function defined on B.
As indicated earlier, events A1 , · · · , An are said to be independent if
n
P (∩ni=1 Ai ) = ∏ P (Ai )
i=1

Note that if n = 2, and A1 , A2 are independent, this says that

P (A1 ∩ A2 ) P (A1 ) P (A2 )

P (A1 |A2 ) = = = P (A1 )
P (A2 ) P (A2 )

Example 38.9.2 You roll a die n times. Let Xk be the value on the die on the kth roll of the
die. Let Ak be the event that the value of Xk is in Sk where Sk is some set of numbers from 1
to 6. Then the value of X j for j ̸= k has absolutely no bearing on whether Xk is in Sk .
n
P (X1 ∈ S1 and X2 ∈ S2 · · · , and Xn ∈ Sn ) = ∏ P (Xk ∈ Sk ) .
k=1

Indeed, this follows from noting that if |Sk | is the number of outcomes in Sk then the expres-
sion on the left is
∏nk=1 |Sk | n
|Sk | n
= ∏ = ∏ P (Xk ∈ Sk ) (38.3)
6n k=1 6 k=1

This is a typical way of getting independent events. Just do experiments in which the
outcome of any experiment is totally unaffected by the outcome of all the others. This
also illustrates what it means for random variables to be independent. Recall the following
definition.

Definition 38.9.3 Let Xi , i = 1, 2, · · · , n be random variables. They are independent means

that for Ik a suitable set,
n
P (X1 ∈ I1 , · · · , Xn ∈ In ) = ∏ P (Xk ∈ Ik )
k=1

It follows that if the Xi are independent and if gi is a continuous function, then the gi (Xi )
are also independent. All this does is change the sets Ik in the above definition, but to do
this right, you need more mathematical machinery.

When you have a random vector X = (X1 , · · · , X p ) with density function f what does
it mean for the components of this random vector to be independent? It means that there
38.9. INDEPENDENCE AND CONDITIONAL PROBABILITY 793

p
are nonnegative functions xi → fi (xi ) such that f (x) = ∏i=1 fi (xi ) . Note how this gives
the conclusion of the above theorem.
Z
P (X1 ∈ I1 , · · · , X p ∈ I p ) = p
f (x) dx
∏i=1 Ii

Z Z
= ··· f1 (x1 ) · · · f p (x p ) dx p · · · dx1
I1 Ip
Z Z
= f1 (x1 ) dx1 · · · f p (x p ) dx p
I1 Ip
p
= ∏ P (Xi ∈ Ii )
i=1

In fact, this is a specialization of what always happens in every situation. Note that the
same argument shows that if these components are an independent set, then if you consider
(g1 (X1 ) , · · · , g p (X p ))
these would also be independent random variables. In this case,
P (g1 (X1 ) ∈ I1 , · · · , g p (X p ) ∈ I p )
= P X1 ∈ g−1 −1


1 (I1 ) , · · · , X p ∈ g p (I p )
Z
= p
f (x) dx
∏i=1 g−1
i (Ii )
Z Z
= ··· f1 (x1 ) · · · f p (x p ) dx p · · · dx1
g−1
1 (I1 ) g−1
p (I p )
Z Z
= f1 (x1 ) dx1 · · · f p (x p ) dx p
g−1
1 (I1 ) g−1
p (I p )
p p
−1


= ∏ P Xi ∈ g i (Ii ) = ∏ P (gi (Xi ) ∈ Ii )
i=1 i=1

This proves the following.

Proposition 38.9.4 Let X = (X1 , · · · , X p ) and suppose there is a density function f (x).
Then the components are independent random variables if f has the following form.
p
f (x) = ∏ fi (xi )
i=1
p
If these are independent random variables, then so are {gi (Xi )}i=1 whenever gi are con-
tinuous functions.
This is actually an equivalence so there is no loss of generality in taking it as the defini-
tion of independence. However, it gets much more involved because some random variables
are neither discrete nor continuous. Nevertheless, this kind of thing will hold if appropri-
ately generalized. To do this in full generality requires a much better mathematical theory
than any contemplated in this book. It will end up involving differentiation theory of some-
thing called a Radon measure. Having noted this, all of the techniques discussed here were
developed with nothing more than the Riemann integral in the early 1900’s.
Also recall the definition of mean and variance given above.
794 CHAPTER 38. PROBABILITY

Definition 38.9.5 Let X be a random  variable. Its mean is defined as µ ≡ E (X) . The
2
variance is defined as E (X − µ) . The mean is a weighted average. It is what you
would expect to see if you took many random samples from this distribution and averaged
them. (In fact there is a theorem which says this.) The variance is a description of how
spread out the probability density is. If the variance is small, then the random variable will
be close to µ with high probability and if it is large, then it is not as certain the random
variable is close to µ.

Now with this definition of mean and variance, why is the normal distribution so impor-
tant? It is because of the central limit theorem. Suppose E Xk2 < ∞ where Xk is a random

variable.
2


Theorem 38.9.6 Let {Xk }∞ k=1 be random variables satisfying E Xk < ∞, which are inde-
pendent
 and identically
 distributed with mean µ = E (Xk ) and positive variance 0 < σ 2 ≡
E (Xk − µ)2 . Let
n √
Xj − µ n (X̄ − µ)
Zn ≡ ∑ √ = (38.4)
j=1 σ n σ

where X̄ is the average of the Xk n1 ∑nk=1 Xk . Then for Z a normally distributed random
variable having mean 0 and variance 1,

lim P (Zn ∈ A) = P (Z ∈ A)
n→∞

where A is a suitable set.

Of course this begs the question: What are µ, σ ? Much that is done in statistics has to
do with determination of these or other parameters. They both give interesting information
if they can be estimated.
How does independence relate to moment generating functions?

Proposition 38.9.7 Let X k be a random vector with values in Rmk . Let

 
X = X1 · · · X p

and let the moment generating function for X exist

!!
p
t·X


M (t) ≡ M (t1 , · · · , t p ) ≡ E e = E exp ∑ tk · X k
k=1

Then the X k are independent if and only if

p
M (t) = ∏ M (0, · · · 0, tk , 0, · · · , 0) (38.5)
k=1

Proof: First suppose the X k are independent. Then the density function for X is of the
form
f (x) = f1 (x1 ) f2 (x2 ) · · · f p (x p )
38.9. INDEPENDENCE AND CONDITIONAL PROBABILITY 795

Therefore, Z Z Z
M (t) = m
··· f1 (x1 ) f2 (x2 )
Rm p R p−1 Rm1
!
p
· · · f p (x p ) exp ∑ tk · X k dx1 · · · dx p
k=1
Z Z Z p
= m
··· f1 (x1 ) f2 (x2 ) · · · f p (x p ) ∏ exp (tk · xk )
Rm p R p−1 Rm1 k=1
p Z p
=∏ m
fk (xk ) exp (tk · xk ) dxk = ∏ M (0, · · · 0, tk , 0, · · · , 0)
k=1 R k k=1
Conversely, suppose the other condition. Then
p
M (t) = ∏ M (0, · · · 0, tk , 0, · · · , 0) =
k=1
p Z Z Z
∏ mp m
··· f (x) exp (tk · xk )
k=1 R R p−1 Rm1

by Fubini’s theorem,
p Z Z Z
∏ m
exp (tk · xk ) · · · m
··· f (x) dx1 · · · dx j · · · dxk
k=1 R k R j Rm1

p Z
≡∏ m
exp (tk · xk ) fk (xk ) dxk (38.6)
k=1 R k
where fk (xk ) is called the marginal distribution and is obtained as
Z Zd Z
fk (xk ) ≡ ··· ··· f (x) dx1 · · · dx
dk · · · dxm
p
Rm p Rmk Rm1

where the hat indicates the thing is being omitted. Thus,

Z Z p
··· ∏ fk (xk ) dx1 · · · dx p = 1
Rm p Rm1 k=1
p
and with respect to the density E (t · X) yields 38.6. But, as noted above, if
∏k=1 fk (xk ) ,
two densities deliver the same moment generating function, then they are the same. Hence
the X k are independent because the density is the product of functions fk of the xk . ■
Proposition 38.9.8 Suppose {Xk }rk=1 are independent and each n (0, 1) , normal with 0
mean and variance 1. Then ∑rk=1 Xk2 is X 2 (r).
Proof: It follows from Corollary 38.8.8 that Xk2 is X 2 (1) . Then using independence,
!! !
r r r
2 2
= ∏ E tXk2



E exp t ∑ Xk = E ∏ exp tXk
k=1 k=1 k=1
r
1 1
= ∏ (1 − 2t)1/2 = (1 − 2t)r/2
k=1

which is the moment generating function of X 2 (r). ■

796 CHAPTER 38. PROBABILITY

 T
Corollary 38.9.9 Suppose X = X1 ··· Xn where X has a moment generating
function of the form
1 T At
M (t) = e 2 t
where A is real and symmetric having rank r ≤ n and eigenvalues 0 or 1. Then X T AX is
X 2 (r). (When r < n, this is a moment generating function of a random variable which is
said to be a singular multivariate normal. )

Proof: By Theorem
! 11.4.7 there is orthogonal U such that U T AU = D where D is of
I 0
the form where I is an r × r identity matrix. Then let Y = U T X. What is the
0 0
distribution of Y ?

E (exp (t · Y )) ≡ E exp t·U T X = E (exp (Ut · X))

   
1 T 1 T T
= exp (Ut) A (Ut) = exp − t U AUt
2 2
  r  
1 T 1 2
= exp t Dt = ∏ exp t
2 k=1 2k

Now exp (tYk ) = 1 and so Yk = 0 if k > r and otherwise, Yk is n (0, 1) , normal with mean 0
and variance 1. Thus Theorem 38.9.7 implies that these random variables are independent
and each n (0, 1). Hence by Proposition 38.9.8,
r
X T AX = Y T U T AUY = Y T DY = ∑ Yk2 which is X 2 (r) . ■
k=1
Chapter 39

Statistical Tests

In this chapter, are various tests for determining parameters and answering other questions
with a certain probability associated with the answers. This is all based on the notion of
random variables of various forms, called statistics, for which there is a known distribution.
The pattern is to compute the statistic which is based on a random sample and then to use
its known distribution to make statistical inferences. This is always what you do when
you know that the samples are coming from a probability distribution involving unknown
parameters.
For example, it is reasonable to believe that the weight of adult men in Arkansas is
normally distributed. However, you don’t know the mean µ and the variance σ 2 and these
are what you want because you want to know the probability that some man weighs between
140 and 180 pounds. You pick randomly 40 males and record their weights. These weights
are the values of independent random variables. Then, you estimate µ and σ 2 from this
sample, and things like an interval where you have a probability of .95 that the weight of
a person will lie in this interval. A hypothesis you might want to test for would be that the
average weight of men in Arkansas is the same as the average weight of men in Alabama.
If you are interested in something other than weight, you would adjust accordingly. You
could be interested in errors produced by a machine when it makes bolts for example. How
sure are you that some measurement is acceptable? If you were an insurrance company, you
would want to know with some confidence an interval containing the lifespan of a person
or an interval and probability associated with it which gives the number of accidents that
people age 17-30 will have. One could go on and on.
In addition to this, each application of these methods would need to be examined car-
fully to be sure that the assumptions on the underlying distribution are not unreasonable.
As suggested, there are two main forms these inferences take. One involves something
called a confidence interval and the other involves rejecting or accepting a given hypothesis,
called a null hypothesis. I admit to being prejudiced toward confidence intervals because
they deliver a straight forward affirmation that with a certain probability something happens
and involve less jargon. However, it is sometimes appropriate to test for a hypothesis
which is either true or false and you may be able to identify a probability that the so called
null hypothesis, that which is being tested, is false and this is also very useful. These
notions will be developed on specifice examples. I think this will make the ideas easier to
understand than to focus first on generalities laden with jargon. The statistics of interest in
the following will be those which have X 2 (r) , T, or F distributions. The first of these has

797
798 CHAPTER 39. STATISTICAL TESTS

already been considered. The last two are described below.

An interesting observation about all of this is that there is a gap between the theory and
the applications like those mentioned above. To really understand the mathematical theory,
you need much more advanced mathematics than what is encountered in this book. This
happens as soon as you start asking fundamental questions about what a random variable
is independent of some application or why certain limits exist and in what sense they exist.
Some of the most fundamental questions come from the Kolmogorov extension theorem
which has to do with measures defined on infinite products.

39.1 The Distribution of nS2 /σ 2

In all of this, Xk is a random variable and we assume X1 , X2 , ... are independent. For ex-
ample, you might have a large population of people and the weight of a person is normally
distributed. Then Xi would be the ith observation of a randomly selected person’s weight.
Definition 39.1.1 The symbol S2 denotes the sample variance of Xk , k = 1, · · · , n, which
2
is of the form n1 ∑nk=1 (Xk − X̄) where X̄ is the sample average of the random variables
X1 , · · · , Xn , X̄ ≡ 1n ∑ni=1 Xi .

When the sample is taken from a normal distribution having mean µ and variance σ 2 ,
it turns out that the random variable nS2 /σ 2 has a chi-squared distribution. When this
is shown, it becomes possible to estimate the variance along with a probability that the
variance is really in some interval called a confidence interval. One can also use this in
terms of a hypothesis test. For example, you might reject the hypothesis that the variance
is very large. This fact that nS2 /σ 2 is X 2 (n − 1) which is shown below is very significant
because the statistic nS2 /σ 2 does not involve µ. The following proposition makes this
possible. It is a statement about independence of the sample mean X̄ and the random
vector of deviations from the sample mean.
Proposition 39.1.2 Let Xk k = 1, 2, · · · , n be independent random variables all having a
normal distribution with mean µ and variance
 σ 2 . Let X̄ ≡ n1 ∑nk=1 Xk , called the sample
mean . Then X̄ and the random vector X1 − X̄ ··· Xn − X̄ are independent.

Proof: This is done most easily with the moment generating technique.
 n
  n n

E et X̄+∑k=1 tk (Xk −X̄) = E e(t−∑k=1 tk )X̄+∑k=1 tk Xk (39.1)
   n 
It is necessary to verify that this equals E et X̄ E e∑k=1 tk (Xk −X̄) . However, 39.1 equals
 1 n 1 n n

= E e( n t−∑k=1 n tk ) ∑ j=1 X j +∑k=1 tk Xk
 n 1 n 1 n

= E e∑ j=1 ( n t−∑k=1 n tk )X j +∑ j=1 t j X j

 n 1 n 1 
= E e∑ j=1 ( n t−∑k=1 n tk +t j )X j
! !!
n n
1 1
= E ∏ exp t − ∑ tk + t j X j
j=1 n k=1 n
39.1. THE DISTRIBUTION OF nS2 /σ 2 799

In that last term you have the product of continuous functions of independent random vari-
ables and so, by 38.2 which gives the moment generating function for a normally distributed
random variable, it equals
n  1 n 1 
∏E e( n t−∑k=1 n tk +t j )X j
j=1
 ! !2 
n n n
1 1 1 1 1
= ∏ exp  n t − ∑ n tk + t j µ + 2 σ 2 t − ∑ tk + t j
n k=1 n

j=1 k=1

 ! !2 
n n
1 1 1 1 1
= exp ∑ t − ∑ tk + t j µ + ∑ σ 2 t − ∑ tk + t j 
j n k=1 n j 2 n k=1 n

1 n 1


Simple algebra shows that ∑ j n t − ∑k=1 n tk + t j = t. Thus the above is
 !2 
n
1 1 1
= exp tµ + ∑ σ 2 t − ∑ tk + t j 
j 2 n k=1 n

Now − ∑nk=1 1n tk + t j = ∑nk=1 n1 (t j − tk ) so the above reduces to

 !2 
1 1 1
= exp tµ + σ 2 ∑ t+ (t j − tk ) 
2 j n ∑ k n

Consider the mixed term in that last summand above.

t 1 t 1
∑ 2 n ∑ n (t j − tk ) = 2 n ∑ ∑ n (t j − tk ) = 0
j k j k

Hence the above reduces to

  !2 
 2
1 1 1
exp tµ + σ 2 ∑ t + ∑ ∑ (t j − tk ) 
2 j n j k n
!
1 2 t2
 
1 21 2
= exp tµ + σ exp ∑ ∑ σ (t j − tk ) (39.2)
2 n j k 2 n

So you see, the moment generating function splits up the first factor depending only on t
and the second depending only on the tk .

!! !
  1 t 
E et X̄ = E exp tXk = E ∏ exp Xk
n∑k k n
 t  2
t µ+ 1 σ 2 t 1 2 t2

= ∏ E exp Xk = ∏ e n 2 n2 = etµ+ 2 σ n
k n k
800 CHAPTER 39. STATISTICAL TESTS

Thus the first term in 39.2 is the moment generating function of X̄. Some computations
show that the second term is the moment generating function of the vector
 
X1 − X̄ · · · Xn − X̄

Indeed, ! !!
n
1
E exp ∑ tk (Xk − X̄) = E exp ∑ tk Xk − ∑ tk ∑ X j
k=1 k k n j

!! !!
1 tk − t j
= E exp ∑ tk Xk − ∑ t j ∑ Xk = E exp ∑ ∑ Xk
k j n k j k n
!!   
tk − t j tk − t j
= ∏ E exp ∑ n Xk = ∏ ∏ E exp n Xk
j k j k
!!
1 tk − t j 2 2
   
tk − t j
= ∏ ∏ exp µ + σ
j k n 2 n
!
1 tk − t j 2 2
 
= exp ∑ ∑ σ
j k 2 n

Therefore, by Proposition 38.9.7, X̄ and this random vector are linearly independent. ■
The above proposition leads to something interesting, the distribution of nS2 /σ 2 . Let
the Xk be independent and normally distributed with mean µ and variance σ 2 . Then

1 n 2
S2 ≡ ∑ (Xk − X̄)
n k=1
2
The distribution of nS2 /σ 2 = ∑nk=1 (Xkσ−2X̄) will be considered. If we know this, then since
S2 is experimentally determined, it will follow that we could estimate σ 2 . First note that

Xk − X̄ = Xk − µ + µ − X̄

and so
2 2
(Xk − X̄) = (Xk − µ)2 − 2 (Xk − µ) (X̄ − µ) + (X̄ − µ)
n
2 ∑ (Xk − µ) (X̄ − µ) = 2n (X̄ − µ) (X̄ − µ)
k=1

Therefore,
n n
2 2
∑ (Xk − X̄) + n (X̄ − µ) = ∑ (Xk − µ)2
k=1 k=1

Then
2 2
n
(Xk − X̄) n (X̄ − µ) n
(Xk − µ)2
∑ σ2 + σ2 = ∑ σ2
k=1 k=1
39.1. THE DISTRIBUTION OF nS2 /σ 2 801

2  
From what was shown above, n(X̄−µ)σ2
and the vector X1 − X̄ ··· Xn − X̄ are inde-
pendent. From this, it follows that
2 2
n (X̄ − µ) n t (Xk − X̄)
,∑
σ2 k=1 σ2

are independent.
(Xk −µ)2
Using this and the known distribution of σ2
,
! !!
2 2
(Xk − µ)2
n
t (Xk − X̄) n
n (X̄ − µ)
E t∑ = E exp ∑ +t
k=1 σ2 k=1 σ 2 σ2
! !!
n 2 2
(Xk − X̄) n (X̄ − µ)
= E exp t ∑ exp t
k=1 σ2 σ2
By independence, this is
!! !!
n 2 2
(Xk − X̄) n (X̄ − µ)
= E exp t ∑ E exp t (39.3)
k=1 σ2 σ2
  
2
Of course the thing we want is E exp t ∑nk=1 (Xkσ−2X̄) , but the expression on the left
 2 
E t ∑nk=1 (Xkσ−µ)
2 , and the factor on the right are known or easy to find. Consider the
factor on the right.
2 1

2
n (X̄ − µ) ∑nk=1 (Xk − µ)
n n
t = t
σ2 σ2
n  !2
Xk − µ
= t ∑ √
k=1 nσ
 
What is the distribution of ∑nk=1 X√k −µ
nσ
? By independence, its moment generating func-
tion is !! !
n  n  
Xk − µ Xk − µ
E exp t ∑ √ = E ∏ exp t √
k=1 nσ k=1 nσ
n n 2
√
−1 Xk − µ
    
1 √t 1 2
= ∏ E exp t n = ∏e 2 n = e2t
k=1 σ k=1

so it is a normal distribution having variance 1 and mean 0. It follows from Corollary 38.8.8
that the square of this random variable is X 2 (1). Since we know the moment generating
function for chi squared distributions, it follows that we know all the terms in 39.3 except
for the one we want. It just a matter of filling in the expressions. Recall the moment
generating function for X 2 (r) is
1
(1 − 2t)r/2
802 CHAPTER 39. STATISTICAL TESTS

Thus from 39.3,

!! !!
2
n
(Xk − X̄) 1 n
(Xk − µ)2
E exp t ∑ = ∏ E exp t
k=1 σ2 (1 − 2t)1/2 k=1 σ2
!
n
(Xk − µ)2
= E t∑
k=1 σ2

This last term is the moment generating function of the sum of squares of standard normal
deviates and so its moment generating function is known by Proposition 38.9.8 equals
1 1
n/2 . Thus, dividing both sides by 1/2 we get
(1−2t) (1−2t)
!!
n 2
(Xk − X̄) 1
E exp t ∑ =
k=1 σ2 (1 − 2t)(n−1)/2

which shows that nS2 /σ 2 is distributed as X 2 (n − 1). This proves the following major
theorem.

Theorem 39.1.3 Suppose {X1 , · · · , Xn } are independent and they are normally distributed
2
with variance σ 2 . Let S2 = 1n ∑nk=1 (Xk − X̄) , called the sample variance, where X̄ =
n ∑k=1 Xk . Then the random variable nS /σ is distributed as X (n − 1).
1 n 2 2 2

This is really interesting. Note that we don’t know the mean and yet this allows an
estimation of the variance based on observations of the Xi .

39.1.1 Confidence Intervals for Variance

Here is the concept of a confidence interval for the variance of a normally distributed ran-
dom variable.

Example 39.1.4 Here are 11 numbers:

1, 2, 3, −3, 1.5, 0, −1, −2, −.7, −1.5, −2.2

These are independent samples taken from a normal distribution. Find a confidence inter-
val for the variance of this normal distruibution.

First, what in the world is a “confidence interval”.

Definition 39.1.5 An interval [a, b] is a .95 confidence interval for a parameter ν means
that the probability that ν lies in [a, b] is .95.

Of course, if the probability that the parameter lies in [a, b] is .9, then it would be a .9
confidence interval and so forth.
Now consider the above example of 11 numbers. The sample mean or average of
these numbers is −. 263 64. Then 11S2 for these numbers is 37. 065. This just follows
from a computation. Then from Theorem 39.1.3, 11S2 /σ 2 is a X 2 (10) random variable.
We find an interval such that the probability that such a X 2 (10) random variable is in
this interval. This is easy to do from tables. However, you can also use the distribution
39.1. THE DISTRIBUTION OF nS2 /σ 2 803

F (x) ≡ P (X ≤ x) . You can obtain your own table of this using MATLAB or you can use
the graph of this function using MATLAB. Here is an easy way to do it. I am sure there
are more elegant ways to obtain this graph but I am picking one which seems to minimize
the fussiness. MATLAB knows how to do numerical integration. The following tells it to
integrate up to n ∗ .05 and place a dot there at the point (n ∗ .05, y) where y is the integral
up to n ∗ .05.
>>hold on
r=10;
for n=1:1:1200
f=@(t)[1/(gamma(r/2)*2ˆ(r/2))*t.ˆ((r/2)-1).*exp(-t/2)];
y=integral(f,0,n*.05);
plot(n*.05,y,’.’,’Linewidth’,2,’color’,’black’)
end
This will produce a nice graph of F (x) ≡ P (X ≤ x) , called the probability distribution
function, and so you identify an interval for which the area under the curve is no more than
.95. Click on the icon on the tool bar for the graph which says: “data cursor”. This will give
you a little cross which you can move around and click on points of the graph and it will
tell you coordinates, an x coordinate and a y coordinate which is the probability that X ≤ x.
This allows you to avoid hunting for things in a table. In fact, MATLAB can essentially
produce the tables for you. In ancient times, we had to use tables and we even used tables
of trig. functions and logarithms. There was a whole set of specialized techniques which
are now obsolete which we suffered with long ago. Now of course, there is software which
can do all of it for you so it is important to understand what the software is doing.
If you have scientific notebook, it is even easier. In this case, all you have to do is type
in math mode Z x
1
t 4 e−t/2 dt
0 Γ (10/2) 210/2

and ask it to graph this function of x. It will do so. Then you click on the icon which lets
you identify coordinates just like you can do in MATLAB. The quality of the graph is not
as good as what you get in MATLAB, but it does the job quite well with less hassle. I like
looking at pictures better than rummaging through tables. However, if you like to look at
tables, try this. Matlab will make a table for you.
>> hold on
T=[]; r=10;
for n=1:1:100
f=@(t)[(gamma(r/2)*2ˆ(r/2))ˆ(-1)*t.ˆ((r/2)-1).*exp(-t/2)];
x=n*.5;
y=integral(f,0,x);
T=[T; x y];
end
T
I found this on line which is where I usually go for questions about MATLAB. It will
produce a table having two columns, one for x and the next for y which will be the proba-
bility up to x.
Here are two points on the graph: (3.45, .0312) and (21.85, .9841) . Thus the probability
that X ≤ 21.85 is .9841 and the probability that X ≤ 3.45 is .0312. It follows that the
probability that X ∈ [3.45, 21.85] is .9841 − .0312 = 0.952 9. Therefore, the probability
804 CHAPTER 39. STATISTICAL TESTS

that X = 11S2 /σ 2 is in [3.45, 21.85] is better than .95. Thus

(37. 065)
3.45 ≤ ≤ 21.85
σ2
so the probability is better than .95 that

1 σ2 1
≥ ≥
3.45 (37. 065) 21.85
In other words, the probability is better than .95 that

10. 743 ≥ σ 2 ≥ 1. 696 3

What if we only wanted to know with probability .7 where σ 2 is? Then one could get
a much smaller interval. Two points on the graph are (5.1, .1156) and (14.5, .8486). Then
the same process yields with probability better than .7

1 σ2 1
≥ ≥
5.1 (37. 065) 14.5

7. 267 6 ≥ σ 2 ≥ 2. 556 2
This is a much shorter interval but we can’t be as sure that the variance is in this interval. If
you say more about something, it is hardly surprising that you can’t be quite as sure about
your assertion.

PROCEDURE 39.1.6 To find a .95 confidence interval for the variance using a
random sample
X1 , X2 , · · · , Xn
from a normal distribution of mean µ and variance σ 2 do the following.

1. Using a table or graph, determine an interval [a, b] , a > 0 such that the probability
that a X 2 (n − 1) random variable is in [a, b] is at least .95.
2
2. Compute the sample mean X̄ ≡ 1n ∑nk=1 Xk and nS2 ≡ ∑nk=1 (Xk − X̄) .

3. The .95 confidence interval is determined by solving the following inequality for σ 2 .

nS2
a≤ ≤b
σ2

4. Thus the .95 confidence interval is

nS2 nS2
≥ σ2 ≥
a b

The same procedure is followed if you want some other probability than .95.

Incidentally, if you knew the mean µ you could replace the sample mean with this and
use a chi-squared distribution with one more degree of freedom which of course will result
in a better confidence interval. However, I don’t think you could have a good reason for
thinking you know the mean, so such observations are mainly theoretical at this point.
39.2. THE T AND F DISTRIBUTIONS 805

39.2 The T and F Distributions

These are really interesting. They both involve independent random variables which are
distributed as normal or X 2 distributions. These involve combinations of these other
random variables and the idea is to find the density of these combinations. It is a nice
application of the change of variables theorem.

39.2.1 The T Distribution

Here there are two independent random variables, W which is normally distributed with
mean 0 and variance 1 and V which is X 2 (r) . Thus, as explained above,

1 w2 1
Z
P ((V,W ) ∈ A) = √ e− 2 r/2
v(r/2)−1 e−v/2 dwdv
A 2π Γ (r/2) 2

thus (V,W ) ∈ (0, ∞) × (−∞, ∞). The idea is to find the probability density of the statistic
W
T=p
V /r

It is a random variable which has a known distribution. This involves changing the variable.
Let ! !
w u v
t = p , u = v, =r
v/r t w
This maps (0, ∞) × (−∞, ∞) one to one onto (0, ∞) × (−∞, ∞) as can be seen with a short
computation. Let the density function of (t, u) be f (t, u).

P ((t, u) ∈ U) = P (v, w) ∈ r −1 (U)

By the change of variables formula for multiple integrals if U is some open set in R2 ,
1 w2 1
Z Z
f (t, u) dudt = √ e− 2 v(r/2)−1 e−v/2 dwdu
U −1
r (U) 2π Γ (r/2) 2r/2
!
w
Z
= f p , v J (v, w) dwdu
r −1 (U) v/r

where  
1 0
J (v, w) = det  − 1 w q1
 = q1
2r 3 1v 1
( 1r v) rv
2 r

Thus !
w 1 1 w2 1
f p ,v q = √ e− 2 r/2
v(r/2)−1 e−v/2
v/r 1 2π Γ (r/2) 2
rv

Then ! r
w 1 − w2 1 1 (r/2)−1 −v/2
f p ,v = √ e 2 vv e
v/r 2π Γ (r/2) 2r/2 r
806 CHAPTER 39. STATISTICAL TESTS

Now it is necessary to invert the transformations and solve for v, w in terms of t, u.

w
t = p ,u = v
v/r
pu
So w = t r ,v = u. Thus
r
1 t2u 1 1 (r/2)−1 −u/2
f (t, u) = √ e− 2r uu e
2π Γ (r/2) 2r/2 r
r
1 − 1 t2u − 1 u 1 1 r− 1 1
= √ e 2r e 2
r/2
u 2 2
2π Γ (r/2) 2 r

Now this is the density for a random vector (T,U) and it is desired to find the density for
T. This means U can be anywhere in (0, ∞) and so to get this density we do the following
integral. r
1 1 1
Z ∞
1 2 1 1 1
√ r/2
e− 2r t u e− 2 u u 2 r− 2 du
2π r Γ (r/2) 2 0

Consider the integral. It is  2 

Z ∞
−u t2r + 12 1
u 2 (r−1) du
e
0
 2   2 
t
Change variables letting x = u 2r + 12 , dx = 2r
t
+ 12 du. Then it equals

! 1 (r−1)
2
x 1
Z ∞
e−x dx
t2 1 t2
0
2r + 2 2r + 12
! 1 r+ 1 Z
2 2
1 ∞ 1
= e−x x 2 (r−1) dx
t2 1
2r + 2
0

Let α − 1 = 12 (r − 1) . Then the above equals

! 1 r+ 1 ! 1 r+ 1 
2 2 2 2 
1 1 1 1
Γ (α) = Γ r+
t2 t2
2r + 21 2r +1 2
2 2

Therefore, the density function for T is

r   1 r+ 1  
1 1 1 2 2 2 1 1
= √ Γ r+
2π r Γ (r/2) 2r/2 (t 2 /r + 1) 2 2
1 r+ 1
1 Γ 21 r + 12
r
 
1 1 2 2
= √
π r Γ (r/2) (t 2 /r + 1)
Then
 1 r+ 1
1 Γ 12 r + 21
r

1 1 2 2
f (t) ≡ √ 2
π r Γ (r/2) (t /r + 1)
is the density for the T distribution. Here t ∈ R. Here is a graph of F (x) = P (X ≤ x) for X
distributed as a T distribution in which r = 10.
39.2. THE T AND F DISTRIBUTIONS 807

To get this graph I was tricky. I wanted to integrate from −∞ to some positive
Rx
point.
I used the fact that the Rdensity function is even. I didn’t want to consider −∞ f (t) dt so
instead considered .5 + 0x f (t) dt for x ≥ 0 and then this gave the right thing for positive x.
A similar adjustment took care of the graph for x < 0. In the syntax, you can pick r. I have
shown it for r = 10.
hold on
for n=1:1:1000
r=10;
a=((r*pi)ˆ(-1/2))*(gamma(.5*(r+1))/gamma(r/2));
f=@(t)[a*(((t.ˆ2)/r)+1).ˆ(-(.5*(r+1)))];
y=integral(f,0,n*.01);
hold on
plot(n*.01,y+.5,’.’,’Linewidth’,2,’color’,’black’)
plot(-n*.01,-y+.5,’.’,’Linewidth’,2,’color’,’black’)
end
If you wanted a table of x → P (X ≤ x), you can do the following.
hold on
T=[]; r=10;
for n=1:1:1500
a=((r*pi)ˆ(-1/2))*(gamma(.5*(r+1))/gamma(.5*r));
f=@(t)[a*(((t.ˆ2)/r)+1).ˆ(-.5*(r+1))];
x=-10+(n*.1);
y=integral(f,-10,x);
T=[T; x y];
end
T
This will produce a table for the T distribution with r = 10. You can follow the same
pattern to get a table for other values of r. Just change the statement r = 10 to r = 5 for
example. I started the integral at −10 because if x <R−10, P (X ≤ x) isRconsidered 0 due to
x x
round off error so there is no point in trying to take −∞ f (t) dt when −10 f (t) dt is going
to give the same thing as far as can be assertained. You might want to change where you
start the integral depending on r. The table should start at 0 and end at 1 or something close
to it.

39.2.2 Confidence Intervals for the Mean

Say you have r + 1 independent samples from a normal distribution with mean µ and vari-
ance σ 2 . Earlier a method was given which would allow one to give a confidence interval
for the variance. Now a method will be given for finding a confidence interval for the mean.
808 CHAPTER 39. STATISTICAL TESTS

This involves the T distribution. We will assume Xk , k ≤ r + 1 is an independent sample

from a normal distribution having mean µ and variance σ 2 .

Lemma 39.2.1 Let Xk be r + 1 independent random variables normally distributed with

mean µ and variance σ 2 . Then

1 r+1 Xk − µ
√ ∑
r + 1 k=1 σ

is normally distributed with mean 0 and variance 1.

Proof:
!! !
1 r+1 Xk − µ r+1 
t Xk − µ
E exp t √ ∑ =E ∏√
r + 1 k=1 σ k=1 r + 1 σ

Now recall that Xkσ−µ is normally distributed with mean 0 and variance 1. Therefore, by
independence, this equals
r+1
1 t2
   
1 2
∏ exp − = exp − t
k=1 2 r+1 2

which is the moment generating function for a normally distributed random variable with
mean 0 and variance 1. ■
Recall that (r + 1) S2 /σ 2 is X 2 (r) . By the above discussion of the T distribution,

√1 r+1 Xk −µ
r+1
∑k=1 σ
r
2
∑r+1
k=1 (Xk −X̄)
rσ 2

is a T random variable with r degrees of freedom discussed above. However, this expres-
sion simplifies quite a bit. It becomes
√ √
√ r √ r √ √
r+1
∑r+1
k=1 (Xk − µ) r+1
(r + 1) (X̄ − µ) r r + 1 (X̄ − µ)
q = q = q
2 2 2
∑r+1
k=1 (Xk − X̄) ∑r+1
k=1 (Xk − X̄) ∑r+1
k=1 (Xk − X̄)

Notice how the σ disappeared leaving only µ.

Example 39.2.2 Here are 11 numbers from an independent random sample of a normal
distribution having variance σ 2 and mean µ. Find a .95 confidence interval for the mean
µ. The numbers are
3, 4, 5, 6, 2, 3.5, 5, 4, 6, 2, 4.2

After some computations, we find X̄ = 4.0636 and 11S2 = 19. 245. Then the statistic
above is of the form √ √
10 11 (4. 063 6 − µ)
√
19. 245
Using the data cursor in the graph of the function F (x) = P (X ≤ x) for X a T random
variable with r = 10, we can find an interval corresponding to probability at least .95. A
39.2. THE T AND F DISTRIBUTIONS 809

point on this graph is (2.48, .9837). Another point is (−2.52, .01519) and so the probability
that X is in this interval is .9837 − .01519 = 0.968 51. Thus with probability at least .95
√ √
10 11 (4. 063 6 − µ)
−2.52 ≤ √ ≤ 2.48
19. 245
Of course you could arrange the interval to be symmetric about 0 because the distribution is
symmetric. I just used the data cursor to identify a couple of points. Thus with probability
at least .95,
1. 054 1 ≥ (µ − 4. 063 6) ≥ −1. 037 3
5. 117 7 ≥ µ ≥ 3. 026 3
Note that theoretically we could have used
X1 −µ
σ
r
2
∑r+1
k=1 (Xk −X̄)
rσ 2

√ √
but this would not give us such a√good result because instead of dividing by r r + 1 we
would end up dividing by only r. The interval would be much longer. Of course this is
not surprising. If you use more information, you should get better results. You might try
this to see what happens.

PROCEDURE 39.2.3 To find a .95 confidence interval for the mean of a normal
distribution having variance σ 2 and mean µ based on a random sample X1 , · · · , Xn , do the
following.

1. Using a table or graph, determine an interval [a, b] such that for X distributed as a
T random variable with r = n − 1, such that P (X ∈ [a, b]) ≥ .95.
2. Find the sample mean X̄ ≡ n1 ∑nk=1 Xk .
3. The .95 confidence interval for µ is determined by solving the following inequality
for µ. √ √
n − 1 n (X̄ − µ)
a≤ q ≤b
n 2
∑k=1 (Xk − X̄)

4. Thus the .95 confidence interval is

q q
2 2
a ∑nk=1 (Xk − X̄) b ∑nk=1 (Xk − X̄)
X̄ − √ √ ≥ µ ≥ X̄ − √ √
n−1 n n−1 n

If you want some other probability than .95, just find [a, b] associated with this other
probability for X ∈ [a, b] where X is X 2 (n − 1) and do the same thing.
810 CHAPTER 39. STATISTICAL TESTS

39.2.3 Testing For Two Different Means

Suppose you have two different normal distributions and you take samples from each. You
might ask whether the two means are different. This involves the general notion of hypoth-
esis testing. First of all, there is no way you will ever know that two means are exactly
equal based on random samples from the two, but you might be able to conclude that you
are very sure that the two are not equal. The following is some general terminology.

Definition 39.2.4 The hypothesis to be tested is called the null hypothesis, often denoted
as H0 . For example, you might have equality of two means be the null hypothesis. Rejection
of the hypothesis depends on whether some statistic, depending on the validity of H0 is in
a region for which we agree to reject the hypothesis. Usually this is done based on the
probability of the statistic being in this region. The set of values for which we don’t reject
the hypothesis is called the acceptance region.

Lemma 39.2.5 Let X,Y be independent random variables taken from two different normal
distributions, respectively n µ 1 , σ 21 and n µ 2 , σ 22 . Then X −Y is n µ 1 − µ 2 , σ 21 + σ 22 .

Proof: Let M (t) be the moment generating function of X −Y.

M (t) ≡ E (exp (t (X −Y ))) = E (exp (tX) exp (−tY ))

= E (exp (tX)) E (exp (−tY ))
1 2σ 2 1 2σ 2 1 2 2 2
= etµ 1 e 2 t 1 e−tµ 2 e 2 t 2 = et(µ 1 −µ 2 ) e 2 t (σ 1 +σ 2 )

which is the moment generating function of a random variable with distribution

n µ 1 − µ 2 , σ 21 + σ 22

■
that {Xk }r+1 r+1 2


Now suppose k=1 and {Yk }k=1 are two random samples taken from n µ 1 , σ 1
and n µ 2 , σ 22 respectively. Thus from Lemma 39.2.5,

Xk −Yk

is n µ 1 − µ 2 , σ 21 + σ 22 . Letting Zk ≡ Xk −Yk , then it follows as in Lemma 39.2.1

1 r+1 Zk − (µ 1 − µ 2 )
√ ∑ q 2 2 is n (0, 1)
r + 1 k=1 σ +σ 1 2

Then, just as before,

−(µ 1 −µ 2 )
r+1 Zk√
√1 ∑k=1
r+1 σ 21 +σ 22
r
2
∑r+1
k=1 (Zk −Z̄)
r(σ 21 +σ 22 )

is a T random variable with parameter equal to r, T (r). As before, we can simplify this to
obtain that the following is a T random variable.
√ √
r r + 1 (Z̄ − (µ 1 − µ 2 ))
q (39.4)
2
∑r+1
k=1 (Zk − Z̄)
39.2. THE T AND F DISTRIBUTIONS 811

Let H0 be the hypothesis that µ 1 = µ 2 . Then with this assumption, the above T (r) is
√ √
r r + 1Z̄
T≡q (39.5)
r+1 2
∑k=1 (Zk − Z̄)

Example 39.2.6 You have two random samples from normal distributions. The first {Xk }
is (2, 3, −2, −5, 7, 9) . The second {Yk } is (−2, −4, 1, 3, 4, 5) these taken in the order indi-
cated. Then the corresponding list of normal random variables Zk ≡ Xk −Yk is

(4, 7, −3, −8, 3, 4)

Lets agree to reject H0 that the two means are equal if T in 39.5 is either too large or too
small, meaning that T is in a region which is associated with small probability, thus the
imperative to reject the Hypothesis. Let a be such that P (T > a) < .05. Then reject H0 if T
is either larger than a or smaller than −a.
q
r+1 2
First find Z̄. It equals 1.1667. Next find ∑k=1 (Zk − Z̄) . It equals 12. 443. Thus T =
√ √
5 6 (1.1667)
= . 513 56
12. 443
Now we need to go to a table or use MATLAB or something to find out information about
T (5). From a table, P (T > 2.015) = .05 and so we do not reject H0 . In other words, we
“accept” the hypothesis that the two means are equal.

PROCEDURE 39.2.7 To test the hypothesis H0 that two means from two differ-
ent normal distributions are equal, do the following:

1. Take random samples (X
1 ,Y1 ) , (X2 ,Y2 ) , · · · , (Xr+1 ,Yr+1 ) where Xk is from n µ 1 , σ 22

and Yk is from n µ 2 , σ 22 .
2. Letting T (r) be a T random variable with parameter r, determine a such that

P (|T (r)| > a)

is smaller than .05. (You could pick any other number in (0, 1) here depending on
how sure you want to be that a rejection of H0 is warranted.)
3. Compute √ √
r r + 1Z̄
q
r+1 2
∑k=1 (Zk − Z̄)

where Zk = Xk −Yk and Z̄ is the average of the Zk .

4. Reject H0 if |T (r)| > a. Otherwise “Accept H0 ”.

Now what would be the outcome if we looked for a confidence interval.

Example 39.2.8 Find a .54 confidence interval for |µ 1 − µ 2 | in the above example.
812 CHAPTER 39. STATISTICAL TESTS

Here we use 39.4. From MATLAB, for T the T statistic with r = 5 being used here,

P (|T | < .8) = .54

Thus with probability .54 you get

√ √
r r + 1 (Z̄ − (µ 1 − µ 2 ))
−.8 < q < .8
r+1 2
∑k=1 (Zk − Z̄)
√ √
r r + 1 ((µ 1 − µ 2 ) − Z̄)
.8 > q > −.8
2
∑r+1
k=1 (Z k − Z̄)
Then √
30 ((µ 1 − µ 2 ) − 1.1667)
.8 > > −.8
12. 443
2. 984 1 > (µ 1 − µ 2 ) > −. 650 72
with probability .54. By replacing .54 with a smaller number, this could be changed and
we could conclude with a reasonable probability that µ 1 − µ 2 > 0. Thus not rejecting H0
isn’t really the same as saying that H0 is true.

39.2.4 The F Distribution

In this case, you have two independent random variables U,V which are respectively
X 2 (r1 ) and X 2 (r2 ). Thus the density for the random variable (U,V ) is
1 1
u(r1 /2)−1 e−u/2 v(r2 /2)−1 e−v/2 , u, v > 0
Γ (r1 /2) 2r1 /2 Γ (r2 /2) 2r2 /2
Here we consider the density function for
U/r1
F=
V /r2
Change the variables as done above.
ur2
f= ,k = v
vr1
inverting the transformations gives
! !
f vr1 u f
u= , v = k, =r
r2 v k
!
1
vr1 r2 − v2ur r2 1
J (u, v) = det 1 = r2
0 1 vr1
Then by the change of variables formula, and letting g denote the density for (F, K) ,
Z  
ur2
Z
g ( f , k) d f dk = g , v |J (u, v)| dudv
r (U) U vr1
39.2. THE T AND F DISTRIBUTIONS 813
 
ur2 1
Z
= g ,v r2 dudv
U vr 1 vr 1

However, the left side is

1 1
Z
/2
u(r1 /2)−1 e−u/2 v(r2 /2)−1 e−v/2 dudv
U Γ (r1 /2) 2r 1 Γ (r2 /2) 2r2 /2

because the probability that ( f , k) is in r (U) is the same as the probability that (u, v) is in
U. Thus
 
ur2 vr1 1 1
g ,v = u(r1 /2)−1 e−u/2 v(r2 /2)−1 e−v/2
vr1 r2 Γ (r1 /2) 2r1 /2 Γ (r2 /2) 2r2 /2

Now write in terms of f , k

 (r1 /2)−1 
f kr

kr1 1 f kr1 − 2r 1 1
g ( f , k) = e 2 k(r2 /2)−1 e−k/2
r2 Γ (r1 /2) 2r1 /2 r2 Γ (r2 /2) 2r2 /2

Of course k ∈ (0, ∞) and so if we want the density of F, all that is needed is to integrate the
above from 0 to ∞ with respect to k. Then this integral is
 r1 /2 Z 
r1 +r2
 
fr

1 1 r1 ∞
2 −1 − 2r1 + 12 k
f (r1 /2)−1 k e 2 dk
Γ (r1 /2) 2r1 /2 Γ (r2 /2) 2r2 /2 r2 0
 
f r1
Change the variable in the integral. Let u = 2r2 + 12 k so

du
dk =  
f r1
2r2 + 12

then the integral is

 
r1 +r2
2 −1
 
u du
Z ∞
  e−u  
f r1 1 f r1
0
2r2 + 2 2r2 + 12

Z ∞  r1 +r2 
1 −1
=  r1 +r2 u 2
e−u du
0

f r1 1 2
2r2 + 2
 
1 r1 + r2
=  r1 +r2 Γ

f r1 2 2
2r2 + 12

thus we end up with the following for the density for F.

 r1 /2 r1 +r2

1 1 r1 Γ
/2
f (r1 /2)−1 2
 r1 +r2
Γ (r1 /2) 2r 1 Γ (r2 /2) 2r2 /2 r2 
f r1 2
2r2 + 12
814 CHAPTER 39. STATISTICAL TESTS

 r1 /2
r1 +r2
r1
Γ 2 r2 f (r1 /2)−1
=  r1 +r2 , f > 0
Γ (r1 /2) Γ (r2 /2) f r

2
1
r2 + 1

Note that if r1 = r2 = r, this is much less ugly. It then reduces to

Γ (r) f (r/2)−1
r
Γ (r/2)2 ( f + 1)
You can probably see that this F distribution could be used to test the ratio of variances
coming from two normal densities and obtain a confidence interval for this ratio. If this
interval did not contain 1, then you could conclude that with a certain probability the two
variances are different. Here is a graph of F (x) ≡ P (X ≤ x) where X is an F random
variable with r = r1 = r2 = 10.
1

0.5

0
0 5 10

39.2.5 Confidence Intervals for the Ratio of Two Variances

Suppose you have two random samples of length r taken from two normal distributions
having variances σ 21 and σ 22 . The symbols which represent such normal distributions
are
2
n µ i , σ i . Let these be {X1 , · · · , Xr } from n µ 1 , σ 21 and {Y1 , · · · ,Yr } from n µ 2 , σ 22 . Let

r r
2 2
rS12 ≡ ∑ (Xk − X̄) , rS22 ≡ ∑ (Yk − Ȳ )
k=1 k=1

Then the rSi2 /σ 2i is X 2 (r − 1) . It follows from the above that

rS12 /σ 21 σ 2 rS2
2 2
= 22 12
rS2 /σ 2 σ 1 rS2

is distributed as an F random variable with the parameter equal to r − 1.

σ 22
Example 39.2.9 Find a .9 confidence interval for the ratio where you have two random
σ 21
samples,
of length 11 taken respectively from two normal distributions. n µ 1 , σ 21 and

n µ 2 , σ 22 . These samples are {−3, 2, −1, 0, 1, −2, .5, .4, −.2, −.5, .3} and
{−4, −7, 7, 10, 15, 5, −8, 11, 12, −12, −5} . You can see that the second sample is much
more spead out than the first. Thus, they should have different variances. Does the confi-
dence interval predict this?

Some computations show that 11S12 = 19.22 and 11S22 = 906.64. Now, from the graph
of F (x) ≡ P (X ≤ x) where X has F distribution with r1 = r2 = r = 10 given above, using
39.2. THE T AND F DISTRIBUTIONS 815

the data cursor, two ordered pairs on this curve are (.33, .047) and (3.01, .9516) . Thus the
associated probability for X in (.33, 3.01) is .9516 − .047 = 0.904 6. Thus

σ 22 19.22
.33 ≤ ≤ 3.01
σ 21 906.64

with probability larger than .9. Thus, with probability larger than .9,

σ 22
15. 567 ≤ ≤ 141. 99
σ 21

It is obvious from this that the ratio is much larger than 1 so, just as you might have guessed,
the two variances are very different. In fact, we could have asserted this with much higher
probability.
From the data cursor, we find the ordered pair (.16, .0038) . thus the probability that

σ 22 19.22
.16 ≤
σ 21 906.64

is no more than 1 − .0038 = 0.996 2. Therefore, the probability

−1
σ 22

19.22
.16 = 7. 547 5 ≤
906.64 σ 21

is at least .9962. There can be no doubt that the second variance is much larger than the
first. Of course we would have thought that, but in general, the samples might not exhibit
such extreme differences in how spread out they are.
If you believe that the two means are the same, then you can add one degree of freedom
to the X 2 distributions and replace rS2 with ∑rk=1 (Xk − µ)2 . This means you can get better
confidence intervals, but why should you believe this? I think that in general, you wouldn’t
know this, so I have emphasized the case where the means are not known and the sample
mean is used instead. There is seemingly no end to complicated tests on statistics which
can be used to draw conclusions about the parameters of underlying distributions. This
book is not the place to explore each and every such technical procedure. To do this, you
should see specialized texts on statistics.

PROCEDURE 39.2.10 Suppose you have two normal distributions

 
n µ, σ 2 , n µ̂, σ̂ 2

and two random samples {X1 , · · · , Xn } , {Y1 , · · · ,Yn } respectively from these two distribu-
2
tions. To find a .95 confidence interval for the ratio σσ̂ 2 , do the following.

1. For X a random variable distributed as an F distribution with r = r1 = r2 = n − 1,

determine an interval [a, b] such that P (X ∈ [a, b]) ≥ .95.
2. Determine the sample means X̄ ≡ ∑nk=1 Xk , Ȳ ≡ ∑nk=1 Yk . Then find
n n
2 2
nS2 ≡ ∑ (Xk − X̄) , nŜ2 ≡ ∑ (Yk − Ȳ )
k=1 k=1
816 CHAPTER 39. STATISTICAL TESTS

σ̂ 2
3. The confidence interval is determined by solving for the ratio σ2
in the inequality

σ̂ 2 nS2
a≤ ≤b
σ 2 nŜ2

4. Thus the confidence interval is

nŜ2 σ̂ 2 nŜ2
a 2
≤ 2
≤b 2
nS σ nS
or in other words,
2 2
∑nk=1 (Yk − Ȳ ) σ̂ 2 ∑nk=1 (Yk − Ȳ )
a 2
≤ ≤ b 2
∑nk=1 (Xk − X̄) σ2 ∑nk=1 (Xk − X̄)

You do the same thing if you want a different probability. Just identify a different inter-
val corresponding to the different probability and do the above.

39.3 Maximum Likelihood Estimates

These estimates give a simple way to estimate various parameters. Unlike the above mate-
rial on confidence intervals and hypothesis testing, you don’t get from these procedures a
confidence interval associated with a probability that the parameter is in this interval or a
direction to reject a hypothesis. You just get the best estimate for the parameter in terms of
maximizing likelihood. I think it is best to illustrate the technique using specific examples.

Example 39.3.1 You know a random variable is a binomial random variable. Thus
!
n
P (X = k) = pk qn−k
k

where q = 1 − p. Find the maximum likelihood estimate for p.

What you do is to write down the “likelihood” obtained by taking a sample X1 , · · · , Xm .

Then the likelihood is
m
L (p) ≡ ∏ pXk (1 − p)1−Xk
k=1

To find an estimate, you seek to pick p in order to maximize this likelihood. Obviously it
would be better to maximize ln (L (p)) which equals
m
ln (L (p)) = ∑ [Xk ln (p) + (1 − Xk ) ln (1 − p)]
k=1

Then from beginning calculus, we take a derivative with respect to p and set equal to 0 and
solve for p. This is the maximum likelihood estimate for p.
m
Xk −1
∑ + (1 − Xk ) =0
k=1 p 1 −p
39.3. MAXIMUM LIKELIHOOD ESTIMATES 817

Thus ! !
m
1 1 m m m
1
∑ Xk = ∑ 1 − Xk = − ∑ Xk
p k=1 1 − p k=1 1− p k=1 1− p
!
m
1 m
∑ Xk =
p (1 − p) k=1 1− p
and so !
m
1 1 m
∑ Xk = m, p = ∑ Xk
p k=1 m k=1
Surely this makes sense. Recall that p was the probability of a success in a Bernouli trial
and the random variable X is the sum of these successes in n trials. To emphasize that this
is an estimate, people will write
1 m
p̂ = ∑ Xk .
m k=1
The above trick in which you maximize ln (L) is typically used. It is generally a good
idea because the likelihood involves taking a product and when you take the ln of a product,
you end up with a sum which is a lot easier to work with than the original product.
Example 39.3.2 Find a maximum likelihood estimate for µ and σ based on a random
sample X1 , · · · , Xn taken from a normal distribution.
In this and other cases of random variables having a density function, f (x) , you choose
the parameters to maximize the likelihood ∏nk=1 f (Xk ) . Thus, in this case, you maximize
n  
1 1
L (µ, σ ) ≡ ∏ √ exp − 2 (Xk − µ)2
k=1 2πσ 2σ
√
You can delete the 1/ 2π. Then maximize the ln of this. Thus you want to maximize
n  
1 2
∑ − ln (σ ) + − (Xk − µ)
k=1 2σ 2

First take the partial with respect to µ. Cancelling out the σ 2 ,

n n
∑ (Xk − µ) = 0 = ∑ Xk − nµ
k=1 k=1

and so
1 n
µ̂ = ∑ Xk ≡ X̄
n k=1
which is the sample mean. Next take partial derivative with respect to σ
n
1 1
∑ − σ + σ 3 (Xk − µ)2 = 0
k=1

Thus, from the first part where µ̂ was found,

n
2
σ 2n = ∑ (Xk − X̄)
k=1
818 CHAPTER 39. STATISTICAL TESTS

and so
1 n 2
σ̂ 2 = ∑ (Xk − X̄) ≡ S2
n k=1
This is the maximum likelihood estimate for the variance.
It should
  be noticed that there is a problem with this. The estimate is biased. This means
2 2
that E σ̂ = E S ̸= σ 2 . To see this, recall that it was shown above that nS2 /σ 2 is a

X 2 (n − 1) random variable. We know the moment generating function of such a random

variable. It is
1
M (t) =
(1 − 2t)(n−1)/2
and so we can find the expectation of nS2 /σ 2 .
 
′ 2 1 1
M (t) = 1 1 n−
(1 − 2t) 2 n+ 2 2 2
 2

Now letting t = 0, you get E nS2 /σ 2 = n − 1. Thus E σS 2 = n−1


n ̸= 1. For this reason,
1 2
people often use n−1 ∑nk=1 (Xk − X̄) as an estimate for the variance.

39.4 Quadratic Forms

When you have a symmetric real matrix A = AT , a quadratic form is an expression of
the form xT Ax. What is considered here are two symmetric, real matrices A, B which are
n×n and
independent random variables X1 , · · · , Xn which have identical normal distribution
n 0, σ 2 . For example, it is a random sample from such a normal distribution. The question
of interest is whether X T AX and X T BX are independent random variables.
(Xk −µ)2
Why might this be of interest? Recall from Corollary 38.8.8, the distribution of σ2
2
is X 2 (1) and since these are independent, the distribution of ∑nk=1 (Xkσ−µ)
2 is X 2 (n). This
Xk −µ
is a quadratic form in the independent variables σ in which the symmetric matrix is just
I. It was important earlier to consider X̄ and the random vector
 
(X1 − X̄) · · · (Xn − X̄)

and it was shown, using the special form of the normal distribution that this random variable
and random vector are independent. This is what made it possible to determine the moment
generating function and distribution of nS2 /σ 2 which made possible a whole collection of
statistical tests and motivated the T and F distributions. However, what was really needed
2
were independence of the quadratic forms ∑nk=1 (Xkσ−2X̄) and X̄ 2 being independent. Thus
we have already been using quadratic forms evaluated at random samples of the normal
distribution.
The idea now is to just extend this to more general situations in which the symmetric
matrix is perhaps not I. To do this, I will first consider the moment generating function for
X T AX where

A is symmetric and X is a random vector whose components are distributed
as n 0, σ 2 . To save space let dx1 · · · dxn = d →−x
 n Z
1 T − 1 x·x
M (t) ≡ √ et x Ax e 2 σ 2 dx1 · · · dxn
2πσ R n
39.4. QUADRATIC FORMS 819
 n Z T
 n Z
1 σ 2 t x 2Ax − 12 x·x 1 − 1 2 xT (I−2σ 2 tA)x →
= √ e σ e σ2 d→ −
x = √ e2σ d−
x
2πσ Rn 2πσ Rn

Now let y ≡ U T x where U is an orthogonal matrix such that U T I − 2σ 2tA U = D, a

diagonal matrix having all positive diagonal entries. We can get such a thing whenever |t|
is small enough because then the expression I − 2σ 2tA will have all positive eigenvalues.
Now det (U) = ±1 because it is orthogonal. Changing variables to y and using the change
of variables formula,
 n Z
1 − 1 y T U T (I−2σ 2 tA)U y →
M (t) = √ e 2σ 2 d−
y
2πσ Rn
 n Z
1 − 1 y T D(t)y →
= √ e 2σ 2 d−y
2πσ R n

where D (t) is the diagonal matrix which has the positive eigenvalues λ 2k (t) down the diag-
onal. Then the above expression splits into factors of the form
1
Z
− 1 2 y2k λ 2k
√ e2σ dyk
2πσ R

So let u = λ k yk and this becomes

1 1 1
Z
− 1 2 u2
√ e 2σ du = .
2πσ λ k R λk
Hence,
!1/2
n n
1 1
M (t) = ∏ = ∏ 2
k=1 λ k k=1 λ k
 1/2
1 1
= = .
det (D (t)) det (I − 2σ 2tA)1/2
Remember the determinant is the product of the eigenvalues of the matrix and I − 2σ 2tA
and D are similar so they have the same eigenvalues, namely the diagonal entries of D. See
Corollary 11.4.5 to Schur’s theorem. This is stated as the following lemma.
Lemma 39.4.1 Let A be symmetric and let X1 , · · · , Xn be independent and n 0, σ 2 . Then



−1/2
the moment generating function of X T AX is M (t) = det I − 2σ 2tA for all |t| suffi-
ciently small.
Now suppose you have two symmetric matrices A, B and independent n 0, σ 2 random

variables X1 , · · · , Xn . Then there are two random variables X T AX, X T BX and we want
to determine when these two are independent. Then using similar reasoning to the above,
it follows that for |s| , |t| both small enough,
1
M (t, s) ≡ E exp tX T AX + sX T BX



=
det (I − 2σ 2tA − 2σ 2 sB)1/2
If AB = 0, then
I − 2σ 2tA I − 2σ 2 sB = I − 2σ 2tA − 2σ 2 sB + 4σ 4tsAB

= I − 2σ 2tA − 2σ 2 sB
820 CHAPTER 39. STATISTICAL TESTS

and so
1 1 1
M (t, s) = =
det (I − 2σ 2tA − 2σ 2 sB)1/2 det (I − 2σ 2tA)1/2 det (I − 2σ 2 sB)1/2
= M (t, 0) M (0, s)
which shows that the two quadratic forms X T AX, X T BX are independent.
In fact this is true the other direction. Suppose the two quadratic forms are independent.
Thus M (t, 0) M (0, s) = M (t, s). Then this requires
1 1 1
=
det (I − 2σ 2tA − 2σ 2 sB)1/2 det (I − 2σ 2tA)1/2 det (I − 2σ 2 sB)1/2
and so
det I − 2σ 2tA − 2σ 2 sB = det I − 2σ 2tA det I − 2σ 2 sB

= det I − 2σ 2tA − 2σ 2 sB + 4σ 4tsAB

This is to hold for all |t| , |s| small enough. However, if AB ̸= 0, the polynomial on the right
will be of degree 2n while the one on the left will be of degree n. Therefore, these cannot
be equal. The details follow.
From the definition of the determinant, the left side is

∑ sgn (r1 · · · rn ) δ 1r1 − 2σ 2tA1r1 − 2σ 2 sB1r1 · · · δ nr1 − 2σ 2tAnr1 − 2σ 2 sBnr1

r1 ···rn

This expression is of the form ∑ p+q≤n ak t p sq . However, similar reasoning gives

det I − 2σ 2tA − 2σ 2 sB + 4σ 4tsAB

is of the form ∑ p+q≤2n bk t p sq and if AB ̸= 0, there will be nonzero terms bt p sq where b ̸= 0

and p + q = 2n. These two polynomials cannot be equal if this happens. Say p̂ + q̂ = 2n
and the second has a term bt p̂ sq̂ . Then one of p̂, q̂ is larger than n. Say q̂ > n. Differentiate

∑ p+q≤n ak t p sq with respect to s q̂ + 1 times and the result causes det I − 2σ 2tA − 2σ 2 sB
2 2 4


to vanish but does not cause det I − 2σ tA − 2σ sB + 4σ tsAB to vanish. Hence AB = 0.
This proves the following very interesting result.
Proposition 39.4.2 Let X1 , · · · , Xn be a random independent sample from n 0, σ 2 and for

 T
X ≡ X1 · · · Xn

and A, B two symmetric real matrices, then X T AX and X T BX are independent if and
only if AB = 0.
Note that for A, B symmetric, AB = 0 if and only if BT AT = BA = 0.
If you have more than two of these, say Ak , k ≤ m the result would end up being similar
although you would need to have A j Ak = 0 whenever j ̸= k.
As an interesting observation, from linear algebra, this condition that the products give
0 implies that the matrices {Ak } are a commuting family of diagonalizable matrices and so
they are simultaneously diagonalizable, meaning that there exists a single invertible matrix
S such that S−1 Ak S = Dk where Dk is a diagonal matrix. However, more is assumed here in
saying that the product is 0. In particular, you can’t have a repeated nonzero matrix in the
list of matrices.
39.4. QUADRATIC FORMS 821

Corollary 39.4.3 Let X1 , · · · , Xn be a random independent sample from n 0, σ 2 and for

 T
X≡ X1 ··· Xn
m
and let {Ak }m
 T
k=1 be symmetric real matrices. Then the random variables X Ak X k=1
are independent if and only if Ak A j = 0 whenever k ̸= j.

Recall that for X1 , · · · , Xn independent random variables which are n µ, σ 2 , their sum


 2
X −µ
∑ni=1 iσ is X 2 (n). As noted, this is a quadratic form in the independent random
n on
variables Xiσ−µ . You just let the symmetric matrix A be the identity.
i=1
T
What about X σA2 X ? When will this be distributed as a X 2 (r) random variable? For
simplicity, assume the random variables are n 0, σ 2 . It was shown above that the moment

generating function for X T AX is


−1/2 
det I − 2σ 2tA

X T AX
Therefore, the moment generating function of σ2
is
T
t −1/2
     
t X 2AX

M (t) ≡ E e σ 2
= det I − 2σ 2 A = det (I − 2tA)−1/2
σ

Of course, it was shown some time ago that the moment generating function for X 2 (r) is
1 T
r/2 . Let U be an orthogonal matrix such that U AU = D, a diagonal matrix. Thus
(1−2t)

M (t) = det (I − 2tD)−1/2

Now if there is anything other than 1 or 0 on the diagonal of D then M (t) cannot possibly
1
be of the form r/2 . Lets consider why this is. Suppose the diagonal entries of D are
(1−2t)
d1 , · · · , dn . Then
!−1/2
n
M (t) = ∏ (1 − 2tdi )
i=1

If you have a factor (1 − 2tdi ) for some di ∈ / {0, 1} , then it simply does not have the right
form to be the moment generating function for X 2 (r). On the other hand, if each di is
either 0 or 1, then M (t) will have the right form and the r will be the number of eigenvalues
equal to 1, the rank of A.
Is there a simple way to describe this condition that X T AX is X 2 (r)? Yes there is.
The eigenvalues of the symmetric matrix A are either 1 or 0.

Lemma 39.4.4 Let A be a real symmetric matrix. Then A2 = A if and only if the eigenval-
ues of A are either 0 or 1.

Proof: Suppose the eigenvalues are 0 or 1. Since A is symmetric, there is an orthonor-

mal basis of eigenvectors. {v k }nk=1 . See Theorem 11.4.7 from the early material on linear
algebra. Then say Av k = λ v k either
λ is 0 or 1 so either Av k = v k or Av k = 0. In the
first case, A2 v k = Av k so A2 − A v k = 0. In the second case, A2 v k = A0 = 0 and so
822 CHAPTER 39. STATISTICAL TESTS

A2 − A v k = 0 − 0 = 0. Thus A2 − A = 0 because this matrix sends every vector in a

basis to 0.
Conversely, suppose A2 = A. Why are all eigenvalues 1 or 0? Say Av = λ v and say
λ ̸= 0. Then for each v an eigenvector, A2 v = Av = λ Av and so A (1 − λ ) v = 0. If λ ̸= 1,
then Av = 0 which is assumed not to be so. Hence λ = 1. Thus all eigenvalues are either
0 or 1. ■
This proves the following interesting theorem.

Theorem 39.4.5 Let X1 , · · · , Xn be independent and n 0, σ 2 . Let A be a real symmetric

 T T
matrix. Then for X = X1 · · · Xn , X σA2 X is X 2 (r) for some r ≤ n if and only if
A2 = A if and only if the eigenvalues of A are 0 or 1. In fact, r is the rank of A.

At this point, it might be good to recall that the distribution of

n 2
(Xk − X̄)
nS2 /σ 2 ≡ ∑ σ2
k=1

is X 2 (n − 1) where
1 n
X̄ = ∑ Xk
n k=1
In showing this, first there was some algebra.

X 2 (n)
z }| {
2
(Xk − µ)2
n n
((Xk − X̄) + (X̄ − µ))
∑ σ2 = ∑ σ2
k=1 k=1

After some simple manipulations,

 2
1
n
(Xk − X̄)
2 n
(X̄ − µ)
2
nS2 n
n ∑nj=1 (Xk − µ)
= ∑ 2
+∑ = 2 +∑
k=1 σ k=1 σ2 σ k=1 σ2
 2
X 2 (1)
!2 z }| {
n n
 n
nS2 2

1 (Xk − µ)  (Xk − µ)  + nS
= + ∑ ∑ = ∑ √

σ 2 k=1 n j=1 σ  j=1 nσ   σ2

Then it was proved that the two random variables at the end are independent. This was
done by using the special form of the normal distribution. Then from this, we obtained on
looking at the moment generating functions,
  !2 
 n/2   2  n
1 nS (Xk − µ) 
= E exp t 2 E exp t ∑ √
1 − 2t σ j=1 nσ
  2 
nS 1
= E exp t 2
σ (1 − 2t)1/2
39.4. QUADRATIC FORMS 823

and so the moment generating function for nS2 /σ 2 is

 (n−1)/2
1
M (t) =
1 − 2t
nS2
so σ2
is X 2 (n − 1). Notice how important independence was in doing this last step.
Also notice that all the terms in the above were quadratic forms. This hopefully mo-
tivates the following very interesting result and shows why it is interesting. It is going to
look a lot like what was done earlier with a difference. The independence of xT Bx, xT Cx
will be obtained directly from an assumption that X T BX is X 2 (r1 ).

Theorem 39.4.6 Let A, B,C be real symmetric n × n matrices. Let

 T
X = X1 · · · Xn

where {X1 , · · · , Xn } is a independent random sample from n 0, σ 2 . Let

xT Ax = xT Bx + xT Cx (39.6)

and suppose X T AX is X 2 (r) , X T BX is X 2 (r1 ) for r1 < r. Then the two random
variables on the right are independent and X T CX is X 2 (r − r1 ).

Proof: Since 39.6 is a statement about quadratic forms for arbitrary x, it ! follows that
I 0
A = B +C. Now there is an orthogonal matrix U such that U T AU = where I is
0 0
r × r for r the rank of A. This follows from Theorem 11.4.7 presented much earlier in the
material on linear algebra and the fact that X T AX is X 2 (r) which implies, from Theorem
39.4.5 the eigenvalues of A are 1 or 0. Therefore,
! ! !
I 0 P P12 Q Q12
= + (39.7)
0 0 P21 P22 Q21 Q22
! !
P P12 Q Q12
where P, Q are r × r matrices and U T BU = ,U T CU = .
P21 P22 Q21 Q22
!
I 0
Now multiply on both sides of 39.7 by . This yields
0 0
! ! !
P P12 Q Q12 I 0
+ =
P21 P22 Q21 Q22 0 0
! ! !
I 0 P 0 Q 0
= +
0 0 0 0 0 0
Thus P12 , P21 , P22 , Q12 , Q21 , Q22 are all 0 and
! ! !
T I 0 P 0 Q 0
U AU = = U T (B +C)U = + (39.8)
0 0 0 0 0 0
824 CHAPTER 39. STATISTICAL TESTS

Note that the symmetry of B,C implies P, Q are symmetric also. It is given that X T BX is
X 2 r1 which happens if and only if B2 = B thanks to Theorem 39.4.5. In other words, B

has eigenvalues either 0 or 1. It follows that P2 =!

P.
P 0
Now multiply on the left in 39.8 by .
0 0
! ! ! ! !
P 0 P2 0 PQ 0 P 0 PQ 0
= + = +
0 0 0 0 0 0 0 0 0 0
Thus, comparing the ends, PQ = QP = 0. It follows that BC = 0. By Corollary 39.4.3,
X T BX, X T CX are independent. It follows from this independence
E exp tX T AX = E exp tX T BX + tX T CX

= E exp tX T BX exp tX T CX

= E exp tX T BX E exp tX T CX

and so, by assumption,

1 1
E exp tX T CX



r/2
= r1 /2
(1 − 2t) (1 − 2t)
showing that
1
E exp tX T CX



=
(1 − 2t)(r−r1 )/2
which implies X T CX is X 2 (r − r1 ). ■
Something should be pointed out here. It is that xT Cx ≥ 0 and this follows from linear
algebra considerations. From the above argument,
! ! !
I 0 P 0 Q 0
= + , P2 = P
0 0 0 0 0 0
 
Thus P has eigenvalues 0 or 1. Then also, 0 ≤ (I − P)2 x, x = I − 2P + P2 x, x =

((I − P) x, x) and so I − P = Q has all nonnegative eigenvalues. Hence

!
T T Q 0
UT x ≥ 0


x Cx = x U
0 0
You can extend this to more than two quadratic forms on the right.
Corollary 39.4.7 Let A, {Ak }m k=1 be real symmetric n × n matrices. Let
 T
X = X1 · · · Xn

where {X1 , · · · , Xn } is a random sample from n 0, σ 2 . Let

m
xT A x = ∑ xT Ak x (39.9)
k=1

and suppose X T AX is X 2 (r) , X T Ak X is X 2 (rk ) for ∑m−1

k=1 rk < r. Then the random
m−1
variables X Ak X k=1 on the right are independent and X T Am X is X 2 r − ∑m−1
 T

k=1 rk .
39.5. LINEAR REGRESSION 825

Proof: Suppose the corollary is true for m − 1, where m − 1 ≥ 2.

A = A1 + (A2 + · · · + Am ) ≡ A1 + B.

Then doing the same argument as above, you find that A1 B = 0 and B2 = B since X T BX
is X 2 (r − r1 ). Now
B = A2 + · · · + Am
and there are only m − 1 in the sum on the right. By induction, Am is
!!
m−1
X 2 r − r1 − ∑ rk
k=2

and X T Ak X are independent for k between 2 and m because Ak A j = 0 for such k, j. Recall
Proposition 39.4.2. It only remains to verify that A1 Ak for 2 ≤ k ≤ m. You could do
the same argument in the form A = A2 + (A1 + A3 + · · · + Am ) and conclude that A1 Ak =
0 for 3 ≤ k ≤ m − 1. Then all that is left is A1 A2 . Just do the argument again for A =
A3 + (A1 + A2 + A4 + · · · + Am ) and conclude in particular that A1 A2 = 0. Thus all mixed
products are 0 and so the quadratic forms are independent. ■

Summary 39.4.8 The following are the main ideas in this section. In all of this,
 T
X = X1 · · · Xn

where the Xi are independent and n 0, σ 2 and the matrices are symmetric

1. For {Ak } symmetric matrices, X T Ak X are independent if and only if Ak A j = 0



for each k ̸= j.
X T AX
2. σ2
is X 2 (r) if and only if A2 = A and the rank of A is r.

k=1 x Ak x and X AX is X (r) , X Ak
X is X (rk ) where k ≤

T T
3. If xT A x = ∑m T 2 2

m − 1 and ∑k=1 rk < r, then X Am X is X r − ∑k=1 rk and the random variables

m−1 T 2 m−1
 T
X Ak X are independent.

39.5 Linear Regression

This will be an interesting and important application of the above theory of quadratic forms.
The idea is you have finitely many times ti ,t1 ≤ t2 ≤ · · · ≤ tn and there are independent
random variables X1 , · · · , Xn , corresponding to these ti . More generally, the ti are simply
real numbers, but often the interpretation is time. Assume the following condition.

Condition 39.5.1 The random variables Xi associated with time tk are

n α + β (tk − t¯) , σ 2

where t¯ ≡ n1 ∑nk=1 tk . There may be many Xi associated with a single tk but it is assumed
that they are independent and normally distributed with a mean which depends on the tk
but the variance is constant. Note that the tk might be repeated in the list. Typically they
are repeated because one is taking a sample larger than one for each tk .
826 CHAPTER 39. STATISTICAL TESTS

Should the above condition be assumed? I am not sure, from the point of view of
rigorous math, but this kind of thing is often assumed in experimental work and leads to
useful conclusions and is not unreasonable since it is just an assumption that the random
variables for each ti are normally distributed.
For example suppose someone is developing vaccines for antiplasmosis, a disease in
animals which causes anemia. You can measure anemia easily by keeping track of the
packed cell volume. He has 15 animals which are infected by the disease and every week,
he takes a small sample of blood from each and measures this packed cell volume using a
centrifuge and capillary tubes. These measurements yield the Xk . If he had another group
of animals say 20 which have been given a vaccine, how would he tell if the vaccine was
effective? He would look for differences in the two different values of β . If he has a confi-
dence interval for each β , the one for the vaccinated cows and the one for the unvaccinated
ones, he could possibly conclude that his vaccine was working. An ordinary least squares
approach would approximate the data with a straight line for each group of animals and
would give a slope β̂ based solely on geometric conditions. This may suggest that the vac-
cine is working, but to be sure the pictures mean something, he needs a confidence interval
or something similar involving a probability for the two parameters β , not just the estimate
β̂ . It would of course also be very interesting to estimate the variance. The machinery for
doing these estimates will be considered in this section. It is a very nice application of the
results of the last section in which the distribution of quadratic forms was considered.
 T
By independence, the probability density of the vector X = X1 · · · Xn is

(xk −(α+β (tk −t¯)))2

 n n
1 − 12
√ ∏e σ2
2πσ k=1

Here tk is the time which goes with Xk . Thus the tk may be repeated because for a given
ti , there are at least one Xk , maybe more. The reason for writing the mean as α + β (tk − t¯)
rather than more simply a + tb is that certain formulas come out looking much simpler if it
is written this way and the maximum likelihood estimates for α, β turn out to be X 2 (1).√
First consider the maximum likelihood estimates for α, β , σ 2 . Forget about the 2π
and work with ln of the expression.
n
1 (Xk − (α + β (tk − t¯)))2
n ln (σ ) + ∑ = L (σ , α, β ) (39.10)
k=1 2 σ2

Now take partial with respect to α and set equal to 0

n
∑ (Xk − (α + β (tk − t¯))) = 0
k=1

Next take partial with respect to β and set equal to 0. Denote by α̂, β̂ the solutions. These
are the maximum likelihood estimates.
n
∑ (Xk − (α + β (tk − t¯))) (tk − t¯) = 0
k=1

Thus
∑ Xk − nα = 0,
k
39.5. LINEAR REGRESSION 827

and so
1
α̂ = Xk ≡ X̄
n∑k
Then also
n
∑ Xk (tk − t¯) − β ∑ (tk − t¯)2 = 0
k=1 k
and so
∑nk=1 Xk (tk − t¯) ∑nk=1 (Xk − X̄) (tk − t¯)
β̂ = 2
= 2
∑k (tk − t¯) ∑k (tk − t¯)
because ∑nk=1 X̄ (tk − t¯) = 0. It remains to find the maximum likelihood estimate for σ 2 .
Using 39.10,
  2
n n Xk − α̂ + β̂ (tk − t¯)
−∑ =0
σ k=1 σ3
1 n   2
σ̂ 2 = ∑ Xk − α̂ + β̂ (tk − t¯)
n k=1
Now consider
n
(Xk − (α + β (tk − t¯)))2
∑ (39.11)
k=1 σ2
I will add in α̂ + β̂ (tk − t¯) and subtract it and then write this as a sum of quadratic forms.
First of all, note that it is the sum of the squares of independent random variables in n (0, 1)
and so it is X 2 (n). It equals
     2
n Xk − α̂ + β̂ (tk − t¯) + α̂ + β̂ (tk − t¯) − (α + β (tk − t¯))
∑ (39.12)
k=1 σ2
This will be expanded. I need to consider the mixed term in which I will use the above
descriptions of α̂ and β̂ .
     
∑ Xk − α̂ + β̂ (tk − t¯) α̂ + β̂ (tk − t¯) − (α + β (tk − t¯))
k
h  i h  i
= ∑ (Xk − X̄) − β̂ (tk − t¯) X̄ + β̂ (tk − t¯) − (α + β (tk − t¯))
k
First note that
∑ (Xk − X̄) X̄ = ∑ (Xk − X̄) α = ∑ β̂ (tk − t¯) X̄ = ∑ β̂ (tk − t¯) α = 0
k k k k
Thus the mixed term is
∑ (Xk − X̄) β̂ (tk − t¯) − β ∑ (Xk − X̄) (tk − t¯)
k k
2 2
−β̂ ∑ (tk − t¯) + β̂ β ∑ (tk − t¯)2
k k
   
= β̂ − β ∑ (Xk − X̄) (tk − t¯) + β̂ β − β̂ ∑ (tk − t¯)2
k k
   
= β̂ − β β̂ ∑ (t j − t¯)2 + β̂ β − β̂ ∑ (tk − t¯)2 = 0
j k
828 CHAPTER 39. STATISTICAL TESTS

It follows from the vanishing of the mixed term that 39.11 equals
n
(Xk − (α + β (tk − t¯)))2
∑ σ2
k=1
  2  
n Xk − α̂ + β̂ (tk − t¯) + α̂ + β̂ (tk − t¯) − (α + β (tk − t¯))2
= ∑
k=1 σ2

1   nσ̂ 2
= 2 ∑ α̂ + β̂ (tk − t¯) − (α + β (tk − t¯))2 + 2
σ k σ

1    2 nσ̂ 2
= ∑ (α̂ − α) + β̂ − β (t k − ¯
t ) + 2
σ2 k σ
Consider the mixed term in the first sum.
 
∑ (α̂ − α) β̂ − β (tk − t¯) = 0
k

Therefore, from 39.11,

(Xk − (α + β (tk − t¯)))2
n
∑ =
k=1 σ2
 2
n β̂ − β nσ̂ 2
2 2
(α̂ − α) + ∑ k(t − ¯
t ) + (39.13)
σ2 σ2 k σ2
At this point, we need to consider what we have.

Xi is n µ i , σ 2i and the Xi are independent for i ≤ n. Then ∑i ai Xi

Lemma 39.5.2 Suppose
is n ∑i ai µ i , ∑ a2i σ 2i .

Proof:Consider the moment generating function.

!! !
n
M (t) ≡ E exp t ∑ ai Xi =E ∏ exp (tai Xi )
i i=1
n n 1 2 2 2
= 2 t ai σ i etai µ i
∏ E (exp (tai Xi )) = ∏ e
i=1 i=1
1 22 2
= e (∑i ai σ ) et ∑i ai µ i ■
2t

   2

Thus α̂ = X̄ is n n1 ∑ni=1 (α + β (ti − tˆ)) , ∑i n12 σ 2 = n α, σn . Also (σα̂−α
√ = X̄−α
/ n)
√ is
(σ / n)
n (0, 1) and so the square root of the first term on the right in 39.13 is n (0, 1) so that first
term is X 2 (1). Similarly, consider the second term or rather its square root. This is

β̂ − β
  1/2  (39.14)
2
σ / ∑k (tk − t¯)
39.5. LINEAR REGRESSION 829

Consider the moment generating function for β̂ .

!!
∑n Xk (tk − t¯)
 
M (t) = E expt β̂ = E exp t k=1 2
∑i (ti − t¯)
!
n
t (tk − t¯)
= E ∏ exp X
2 k
k=1 ∑i (ti − t¯)
 
1 t 2 (tk −t¯)2
n
2 2 2
σ2
= ∏ exp 
 (∑i (ti −t¯) ) 
t(tk −t¯)

k=1 + 2 (α + β (tk − t¯))
∑i (ti −t¯)
!
1 2 σ2
= exp t + βt
2 ∑i (ti − t¯)2
 
σ2
So β̂ is n β , 2 . It follows that the random variable in 39.14 is n (0, 1) and so the
∑i (ti −t¯)
second term on the right in 39.13 is X 2 (1). The last term in 39.13 is

nσ̂ 2 1 n   2
= ∑ k X − α̂ + β̂ (t k − ¯
t )
σ2 σ 2 k=1

Thus we have
X 2 (1)
X 2 (n)
X 2 (1)
z { }|
z }| {  2
n
(Xk − (α + β (tk − t¯)))2
z
n
}| { β̂ − β
∑ = (α̂ − α)2 + ∑ (tk − t¯)2
k=1 σ2 σ2 σ2 k
n  2
1 
+ 2 ∑ Xk − α̂ + β̂ (tk − t¯) (39.15)
σ k=1

In fact, the terms on the right are quadratic forms in the variables Xk − (α + β (tk − t¯))
although it does not look like it. Consider the first term.
1 1
α̂ − α = Xk − α = ∑ (Xk − α)
n∑k n k
1
= (Xk − (α + β (ti − t¯)))
n∑k

This is squared and that is why this term is a quadratic form in the variables

Xk − (α + β (tk − t¯))

Note that the terms added in sum to 0. A similar trick will apply to the other terms. Consider
the second term.
2
∑nk=1 Xk (tk − t¯) ∑nk=1 Xk (tk − t¯) − β ∑k (tk − t¯)
β̂ − β = 2
− β = 2
∑k (tk − t¯) ∑k (tk − t¯)
830 CHAPTER 39. STATISTICAL TESTS

∑nk=1 (Xk − β (tk − t¯)) (tk − t¯) ∑nk=1 (Xk − (α + β (tk − t¯))) (tk − t¯)
= 2
= 2
∑k (tk − t¯) ∑k (tk − t¯)
Note that the terms added in which include α sum to 0. Thus this second term is a
constant times the square of the above which is a quadratic form in the variables Xk −
(α + β (tk − t¯)) .
Finally, consider the last term. Since all the other terms are quadratic forms in the
variables Xk − (α + β (tk − t¯)) , this one must also be so because it is equal to a linear com-
bination of these terms. Alternatively, you could verify this in a similar manner. However, I
will stop here and not wade in sorrow to massage the complicated expression into the right
form. This is what is needed for the following proposition.

Proposition 39.5.3 In 39.15 the various terms are chi-squared as indicated in the formula
and the last term is X 2 (n − 2). Also, the three terms on the right are independent random
variables.

Proof: This follows from Corollary 39.4.7 or Summary 39.4.8. ■

Something else should be noted which involves the maximum likelihood estimates.
1 1
α̂ + β̂ (ti − t¯) = ∑ Xk − β̂ ∑ tk + β̂ti
n k n k
≡ a + bti

Thus
∑nk=1 (Xk ) (tk − t¯) ∑k Xk tk − t¯ ∑k Xk
b = β̂ = 2
=
∑k (tk − t¯) ∑k tk2 − 2tk t¯ + t¯2
∑k Xk tk − t¯ ∑k Xk
=
∑k tk2 − nt¯2
Recall how the least squares line is a + bt where b =
− (∑nk=1 tk ) (∑nk=1 Xk ) + (∑nk=1 tk Xk ) n
2
∑nk=1 tk2 n − (∑nk=1 tk )

(∑nk=1 tk Xk ) n − nt¯ (∑nk=1 Xk )

=
∑nk=1 tk2 n − n2t¯2

(∑nk=1 tk Xk ) − t¯ (∑nk=1 Xk )
=
∑nk=1 tk2 − nt¯2

This is the same as β̂ . Similarly, more computations will show that a ≡ 1n ∑k Xk − n1 β̂ ∑k tk

will end up being the least squares estimate. Thus, if you have a confidence interval for α̂
and β̂ , this delivers a confidence interval for a, b.
Now suppose you want a confidence interval for σ 2 . It was shown above that
1 n   2
∑ Xk − α̂ + β̂ (t k − ¯
t )
σ 2 k=1

is X 2 (n − 2) and so you can use this statistic and a table or graph of the appropriate chi
∑n X j (t j −t¯)
squared distribution to obtain a confidence interval for σ 2 . Since β̂ = j=1 2 , the
∑ j (t j −t¯)
39.5. LINEAR REGRESSION 831

above expression is

1 n   2
2 ∑ Xk − α̂ + β̂ (tk − t¯)
σ k=1
! !!2
1 n ∑nj=1 X j (t j − t¯)
= ∑ Xk − X̄ + 2
(tk − t¯)
σ 2 k=1 ∑ j (t j − t¯)

Thus, to find a confidence interval for the variance, do the following.

PROCEDURE 39.5.4 In the above situation, to find a .95 confidence interval

for the variance which is unknown do this:

1. If there are n observations, n fairly large, certainly larger than 2, find an interval
[a, b] such that if V is a X 2 (n − 2) random variable P (V ∈ [a, b]) ≥ .95.
2. Find X̄ the sample mean and t¯ the average of the t values. Then fill in to find
! !!2
n ∑nj=1 X j (t j − t¯)
S≡ ∑ Xk − X̄ + 2
(tk − t¯)
k=1 ∑ j (t j − t¯)

3. Then the .95 confidence interval for σ 2 is determined by

S
a≤ ≤b
σ2
In other words, with probability .95, the variance σ 2 satisfies
S S
≤ σ2 ≤
b a

I think one is even more interested in β , the slope of the line for the mean. To find
a confidence interval for β , recall the T test. The T distribution was the distribution of
√W where V was X 2 (r) and W was n (0, 1). Do we have such random variables above?
V /r
 2

Recall it was shown above that β̂ is n β , σ ¯ 2 . Therefore,
∑i (ti −t )

β̂ − β
 q  is n (0, 1)
σ / ∑i (ti − t¯)2

Therefore,
 √β̂ −β 
σ / ∑i (ti −t¯)2
r   2
1
σ2 ∑nk=1 Xk − α̂ + β̂ (tk − t¯) / (n − 2)
832 CHAPTER 39. STATISTICAL TESTS

is a T random variable with r = n − 2. Simplifying the above gives

√
 n q
∑ j=1 X j (t j −t¯) 2
n−2 2 − β ∑i (ti − t¯)
∑ j (t j −t¯)
s   2
∑nj=1 X j (t j −t¯)
∑nk=1 Xk − X̄ + 2 (t k − t¯)
∑ j (t j −t¯)
√  
n − 2 ∑nj=1 X j (t j − t¯) − β ∑nj=1 (t j − t¯)2
= s   2
∑nj=1 X j (t j −t¯)
q
2
∑ j (t j − t¯) ∑nk=1 Xk − X̄ + 2 (t k − ¯
t )
∑ j (t j −t¯)

PROCEDURE 39.5.5 To find a .95 confidence interval for β , do the following.

1. If there are n observations, n fairly large, certainly larger than 2, find an interval
for the T distribution [a, b] such that if X is a random variable with this distribution,
P (X ∈ [a, b]) ≥ .95.
2. Compute t¯ the average t value and X̄ the sample mean. Then compute
v !!2
∑nj=1 X j (t j − t¯)
u
s u n
2t
S ≡ ∑ (t j − t¯) ∑ Xk − X̄ + ∑ (t − t¯)2 (tk − t¯)
j k=1 j j

√
and P ≡ n − 2 ∑nj=1 X j (t j − t¯) . Then the confidence interval is obtained by solving
the following inequality for β .
√
P − n − 2β ∑nj=1 (t j − t¯)2
a≤ ≤b
S
Then the confidence interval for β is

P − aS P − Sb
√ 2
≥β ≥ √
n
n − 2 ∑ j=1 (t j − t¯) n − 2 ∑nj=1 (t j − t¯)2

You can also find a confidence interval for α. Here you would use a T distribution
involving (σα̂−α
√ which was shown above to be n (0, 1) along with the distribution of the
/ n)
  2
X 2 (n − 2) random variable σ12 ∑nk=1 Xk − α̂ + β̂ (tk − t¯) in the description of the T
statistic. This is left to the interested reader. It is just like the above.

39.6 Goodness of Fit

In all of the above, it was about identifying parameters, usually for the normal distribution
which has two parameters µ and σ 2 . What if you wonder whether a given random sam-
ple is consistent with the sample coming from some probability distribution? What
then?
You have now abandoned the assumption that the sample comes from say n µ, σ 2 . This
question is of course much more speculative. However, there are methods for considering
39.6. GOODNESS OF FIT 833

it. These methods come from Pearson around 1900. Again, they involve massaging things
to use a known distribution, this time a chi-squared distribution.
To do this right, you should be using characteristic functions, but everything of interest
in this book will have a moment generating function and it is just less fussy to do everything
in terms of moment generating functions. However, the complex variable material in this
book is sufficient for you to do in terms of characteristic functions, except even then, there
are more advanced and theoretical theorems needed which involve much harder techniques.
These are related to convergence of the characteristic functions leading to convergence of
the distributions. Because of these considerations, I will give a discussion to make the
main result plausible based on moment generating functions. To see a full discussion of the
theory about to be presented, see [9]. The following is the situation of interest.

1. F (x) ≡ P (X ≤ x) so F is the distribution function of X which has a density f (x).

2. There are r disjoint intervals S1 , · · · , Sr whose union is R and pi ≡ P (X ∈ Si ).

3. For n large, there is a random sample X1 , · · · , Xn and Vi will denote the number of
these samples which end up in Si . Thus the expected number for Vi would be npi .
4. The determination whether it is reasonable to consider the Xk as coming from the
probability distribution F is dependent on consideration of
r
(Vk − npk )2
Q (n) ≡ ∑ npk
k=1

If this is small, then there isn’t much difference between the observed value Vi and
the expected value npi and it would be reasonable to think that the sample is from
the given probability distribution. On the other hand, if it is large, then it would not
be reasonable to consider the sample as coming from the given distribution.

Of course, the problem is in quantifying these issues and this involves the next major
proposition. First is a lemma about counting.

Lemma 39.6.1 The number of ways of selecting subsets having v1 , · · · , vr elements where
∑k vk = n, from a set having n elements is
n!
v1 !v2 ! · · · vr !
Also
n!
(a1 + a2 + · · · + ar )n = ∑ av11 av22 · · · avr r
v1 +···+vr =n v 1 !v2 ! · · · vr !

Proof: There is nothing to prove if r = 1. In case r = 2, it was shown earlier. Recall

n!
that v !(n−v = v1n!
!v2 ! is the number of ways to select a set having v1 elements and a set
1 1 )!
having v2 elements from a set having n elements. In general, the number of ways to obtain
subsets of size v1 , v2 , · · · , vr is the number of ways to select subsets of size v1 , v2 , · · · , vr−1
from a set of size n − vr times the number of ways to select the set of size n − vr which, by
induction is
(n − vr )! n! n!
= .
v1 !v2 ! · · · vr−1 ! (n − vr )!vr ! v1 !v2 ! · · · vr !
834 CHAPTER 39. STATISTICAL TESTS

As to the last assertion, (a1 + a2 + · · · + ar )n =

(a1 + a2 + · · · + ar ) (a1 + a2 + · · · + ar ) · · · (a1 + a2 + · · · + ar )

where there are n products. Thus this product equals a sum of terms of the form av11 av22 · · · avr r
where ∑k vk = n. How many are there for a given choice of exponents v1 , v2 , · · · , vr ? it is
the number of ways of picking v1 factors from the above product to go with av11 , v2 factors
to go with av22 and so forth. Thus the total number associated with a particular term of this
form is v1 !v2n!!···vr ! and this proves the second part. ■

Proposition 39.6.2 Let pk ,Vk be as described above where F (x) is a given distribution
function. Then if n is large, Q (n) is distributed approximately as X 2 (r − 1) where there
are r disjoint intervals covering R.

Proof: Assume the Xk are samples from F (x). Then the Vk have a multinomial distri-
bution. That is
n!
P (V1 = v1 ,V2 = v2 , · · · ,Vr = vr ) = pv1 · · · pvr r , ∑ vk = n.
v1 !v2 ! · · · vn ! 1 k

Indeed, the probability that any Xk is in Si is pi and so the probability that there are vi of
them in Si is as claimed above. Then consider the moment generating function
!
r
M (t1 , · · · ,tr ) ≡ E exp ∑ tkVk
k=1
n!
= ∑ pv11 · · · pvr r et1 v1 · · · etr vr
v !v
v1 +···+vr =n 1 2 ! · · · vn !

By Lemma 39.6.1 this equals
n

p1 et1 + · · · + pr etr (39.16)
Now consider the moment generating function of the vector
 
1 −np1 k −npk
V√
np1 · · · V√ npk .

Then ! !
np
Vk − npk − ∑k tk √npk tk
E ∑ tk √ =e kE
∑ √npk Vk
k npk k

From 39.16, this equals

np
 t
n
− ∑k tk √npk √1 √tr
Mn (t1 , · · · ,tr ) = e k p1 e np1
+ · · · + pr e npr

Taking ln of this, and simplifying a little,

 t

√ √1 √tr
ln (Mn ) = − ∑ tk pk n + n ln p1 e np1 + · · · + pr e npr
k
39.6. GOODNESS OF FIT 835

t
√k
npk
Now replace each e with the first few terms of its power series. Then the inside of ()
above becomes
  2    2 
√t1 √tr  
t1 np1 tr npr 1
p1 1 + √ +  + · · · + pr 1 + √ + +O
   
np1 2! npr 2! n3/2

1
That last term indicates that what is left over is just a lot of stuff times powers of n3/2
. Since
the sum of the pi is one, this yields ln (Mn ) =
  2 
t1
p1 √np
√  1 + √pnp
1 t1
1
+ 2!
1
+ · · · + √pnp
r tr

r 
− ∑ tk pk n + n ln  
t
2
 pr √nprr   
k 1
+ 2! + O n3/2

Now ln (1 + x) = 0 + x − 12 x2 + O x3 . Of course thex here

 is the material in the above
3 1


which comes after the 1. The O x terms are all O n3/2 and there are a few terms in
 2
the x2 which are not, which are included in ∑k √pnp
k tk
. I will retain these terms in the
k
following. Thus ln Mn =
  2  2 
t 1
p1 √np pr √tnp
r
√ √p1 t1 + + · · · + √pnp
1 r tr
+ r
− ∑ tk pk n + n 
 np1 2! 2! 
 2 r  
k − 12 ∑k √pnp
k tk
+ O n3/2 1
k

Of course this simplifies. When you multiply by the n you get ln (Mn ) =
 √ √ √
√ (t1 )2 (tr )2

√ n p1 t 1 + + · · · + n p r tr +
− ∑ tk p k n +   2 2   2 
1 pk tk 1
k − 2 n ∑k np √ + O n1/2
k
 !2 
(t1 )2 (tr )2 1
 
√ 1
= +···+ − ∑ p k tk +O 
2 2 2 k n1/2

Therefore,   !2 
1 √ √
Mn = exp  ∑ tk2 − ∑ pktk  eO(1/ n)
2 k k

For large n this is very close to

  !2 
1 √
Mn = exp  ∑ tk2 − ∑ pk tk 
2 k k

1 T
where t ∈ Rr . What is A?


which is of the form exp 2 t At
!2
√ √
∑ tk2 − ∑ p k tk = ∑ tk2 − ∑ pi p j tit j
k k k i, j
836 CHAPTER 39. STATISTICAL TESTS

and so  √ √ 
1 − p1 − p1 p2 · · · − p1 pr
√ √
 − p1 p2 1 − p2 ··· − p2 pr
 

A= .. .. .. .. 
. .
 
 . . 
√ √
− pr p1 − pr p2 · · · 1 − pr
 √ 
p1
 √ 
 p2  √ √ √

A = I −
 .. 
 p1 p2 ··· pr
 . 
√
pr

This is of the form I − aaT where aT a = |a|2 = 1. Now

I − aaT I − aaT = I − 2aaT + aaT aaT = I − 2aaT + aaT = I − aaT

Thus A2 = A and so by Lemma 39.4.4 the matrix A has only 0 and 1 as eigenvalues. Now
note that
I − aaT a = 0

and so there is a 0 eigenvalue. In fact multiples of this single eigenvector are the only ones
which deliver 0 as an eigenvalue. I show this now. If I − aaT b = 0, then b = aaT b

and so |b| = |a| aT b ≤ |a|2 |b| = |b| and so you must have (a · b) = aT b = |a| |b| which
means that the only eigenvectors b which have 0 as an eigenvalue are multiples of a. Recall
that this was the condition for equality in the Cauchy Schwarz inequality. Therefore, the
rank of A is r − 1. This is because A is symmetric so there is a basis of eigenvectors. By
Corollary 38.9.9, and the fact that if the moment generating functions converge, then so do
the random variables having the given moment generating function, it follows that for large
2
n, the distribution of ∑rk=1 (Vk −np
np
k)
is X 2 (r − 1) as claimed. ■
k

Example 39.6.3 A die is half of a pair of dice. It is cubic and has numbers from 1 to 6 on
the sides. They are suppose to come up with equal probability. Now you have a die and it
is rolled 60 times. The following table summarizes the outcomes.
1 2 3 4 5 6
6 5 9 10 23 7
Is it reasonable to conclude that the die is fair, doing what it is supposed to do by
giving the same probability to each possible outcome? Well, obviously not, but lets see how
to quantify this conclusion.

Let S1 = (−∞, 1], S2 = (1, 2], S3 = (2, 3], S4 = (3, 4], S5 = (4, 5], S6 = (5, ∞). Then the
pi are each 1/6. We compute the thing which will have a X 2 (5) distribution.

(6−10)2 2 2 !
r
(Vk − npk )2 10 + (5−10)
10 + (9−10)
10
∑ npk = 2 2 = 22
k=1 + (23−10) + (7−10)
10 10

Now from a table or using MATLAB and data cursor you find that the probability that a
X 2 (5) random variable is less than 14.35 is .98. However, here we have that the random
variable is 22 so I can say with probability .98 that the die is unfair.
39.6. GOODNESS OF FIT 837

Actually, there is a fly in the ointment. You know the random variable is not exactly
X 2 (5). After all, you only used a sample of 60 = n. The idea is to let n → ∞. Cramér
says that if n is large enough that the expected numbers npi ≥ 10 for each Si , then the
approximation will be good enough for ordinary applications. Of course you end up being
more sure if you take n larger. More information is typically better.
This process illustrates another example of Hypothesis testing. In this case, the “null
hypothesis” is that the die is fair and the density function is 1/6 for each outcome. The
above process indicates that we should reject the “null hypothesis” with probability .98.
This is a general notion in statistics called hypothesis testing. There is quite a bit of jargon
associated with this, but the main idea is illustrated by the above example. Here is another
example where it is not so clear.

Example 39.6.4 A die is half of a pair of dice. It is cubic and has numbers from 1 to 6 on
the sides. They are suppose to come up with equal probability. Now you have a die and it
is rolled 60 times. The following table summarizes the outcomes.
1 2 3 4 5 6
9 11 9 11 10 10
Is it reasonable to conclude that the die is fair, doing what it is supposed to do by giving
the same probability to each possible outcome?

We can do this the same way. Lets agree to reject the null hypothesis that the die
is fair if the statistic used above which measures discrepency is in a region x > a where
P (X ≤ a) = .6. The X 2 (5) variable is

1 1 1
+ + = .3
10 10 10
It is clearly not in a region associated with smaller than probability .4. In fact, from the
graph or a table, X 2 > .3, occurs with probability almost 1, certainly larger than .98.
Therefore, it is totally unsurprising that this random variable would be larger than .3. There-
fore, we don’t reject the hypothesis. It is reasonable to think that the die is fair.

PROCEDURE 39.6.5 A random sample {Xk }nk=1 is taken where n is large. To

test whether the sample is taken from a distribution function F (x) ≡ P (X ≤ x) , do the
following. Partition R into r disjoint intervals S1 , · · · , Sr such that P (X ∈ Si ) = pi > 0 and
assume n is large enough that npi ≥ 10 for each i. Letting Vk be the number of times some
Xi is in Sk form
r
(V − npk )2
D≡ ∑
k=1 npk

Since n is large, this is distributed as X 2 (r − 1) . The null hypothesis H0 is that F (x) is the
distribution for the sample. We reject H0 with probability .95 if D ≥ α where P (D ≤ α) ≥
.95.

The following example illustrates a situation which is more typical in which there are
parameters. The question is whether grades are normally distributed. Of course there are
two parameters µ and σ 2 and what you are asking is whether some choice of µ and σ 2 re-
sults in a normal distribution from which a random sample of test scores can be considered
drawn. The way to deal with this is to regard D in the above as X 2 ((r − 1) − s) where s is
838 CHAPTER 39. STATISTICAL TESTS

the number of unknown parameters and to replace each parameter with its maximum like-
lihood estimate. The proof of this is very technical and you can see it discussed in Cramér,
[9].

Example 39.6.6 A certain university assigns grades in calculus according to an assump-

tion that the grades on the final will be normally distributed. This is a great idea because
you don’t have to ask whether students have learned a well defined set of outcomes and
it removes the onus of having to tell the students that they didn’t learn anything, thus im-
proving course evaluations, which will please university administrators who regard the job
of the university as pleasing the customers. It also removes the responsibility of ensuring
that the exam is reasonable. The magic curve will take care of any problems. Here are
outcomes from a final exam. I am making these up of course.
The Sk (−∞, 50] (50,60] (60,70] (70,80] (80,90] >90
number in Sk 120 150 60 20 10 40
average grade in Sk 40 55 65 75 85 95
2
average grade in Sk 1369 3136 4489 5929 7396 9409

First we need to compute the maximum likelihood estimates. There are 400 exams.
!
1 40 × 120 + 55 × 150 + 65 × 60
X̄ = = 57.7 5
400 +75 × 20 + 85 × 10 + 95 × 40

Then the estimate for variance is

1 400 1 400 2
∑ (Xi − 56.5)2 = ∑ Xi − X̄ 2
400 k=1 400 k=1

1
To find the first term, 400 ∑400 2
k=1 Xi =
!
1 1369 × 120 + 3136 × 150 + 4489 × 60
= 3682. 3
400 +5929 × 20 + 7396 × 10 + 9409 × 40

Thus the sample variance is

3682. 3 − (57.7 5)2 = 347. 24

Now we can compute the pi .

Z 50 2
1 1 (x−56. 5)
p1 = √ √ e− 2 347. 24 dx = 0.363 61
2π 347. 24 −∞
Z 60 2
1 1 (x−56. 5)
p2 = √ √ e− 2 347. 24 dx = 0.210 88
2π 347. 24 50
Z 70 2
1 1 (x−56. 5)
p3 = √ √ e− 2 347. 24 dx = 0.191 12
2π 347. 24 60
Z 80 2
1 1 (x−56. 5)
p4 = √ √ e− 2 347. 24 dx = 0.130 75
2π 347. 24 70
39.7. CONTINGENCY TABLES 839

Z 90 2
1 1 (x−56. 5)
p5 = √ √ e− 2 347. 24 dx = 6. 752 6 × 10−2
2π 347. 24 80
2
1
Z ∞
1 (x−56. 5)
p6 = √ √ e− 2 347. 24 dx = 3. 610 8 × 10−2
2π 347. 24 90

Assemble D. After some computations one finds

D = 134. 96

Now since there are two parameters, D is X 2 (3) , (6 − 1)−2. If you use the data cursor
on a graph of the distribution function, you find the pair (15.3, .9984). Thus if the null
hypothesis is true that this sample is normally distributed, it would involve a probability
of less than .01. Therefore, the null hypothesis can be rejected with probability larger than
.99. This isn’t quite true of course. Clearly if you used more disjoint intervals, you should
be more sure that the approximation is good, so there is a little fuzziness in this goodness of
fit test. However, it does help to quantify the appearance that the distribution is not normal.
In the above example, this seems fairly clear just from looking at the scores, but this allows
you to give numbers to justify its lack of normality.
Incidentally, if you are not careful, you can get such a distribution of scores on a fi-
nal exam. All you need are faculty who wax creative rather than focussing on published
outcomes. One wonders whether it is reasonable to base assigning letter grades on an as-
sumption that the final exam scores are normally distributed, if the hypothesis that this is
so can be rejected with high probability according to the above procedure. However, I
think that it is often the case that people who follow these automatic procedures do not do
goodness of fit tests like that just described.

PROCEDURE 39.6.7 If the distribution depends on s parameters, modify Pro-

cedure 39.6.5 as follows. First replace each parameter with its maximum likelihood esti-
mate then do exactly the same thing to define D only this time, it is X 2 ((r − 1) − s).

39.7 Contingency Tables

Another application of Proposition 39.6.2 and its generalization to when the distribution
depends on parameters is to the notion of contingency tables. These can come in any size
including more than two dimensions, but I will give a simple example to illustrate and leave
to the reader whatever generalization is appropriate. Here is such a table.

A1 A2 A3
B1 p11 p12 p13
B2 p21 p22 p23

For example, you might be looking at the people in a city and B1 is the event that the
person is female and B2 the event that the person is male while A1 might be that the person
is a democrat, A2 the person is a republican and A3 the person is neither one. A given
person will be in exactly one of the Ai ∩ B j .
The numbers pi j are probabilities and ∑ j ∑i pi j = 1. Thus there is a random variable Z
and pi j is P (Z ∈ Ai ∩ B j ) = P (Ai ∩ B j ). Denote as p· j the marginal probability ∑i pi j and
840 CHAPTER 39. STATISTICAL TESTS

pi· the marginal probability ∑ j pi j . Thus p· j = P (Z ∈ A j ) = P (Ai ) and pi· = P (Z ∈ Bi ) =

P (Bi ).
The problem of interest is whether the events Ai and B j are independent. Is it the case
that
P (Ai ∩ B j ) = P (Ai ) P (B j )?
In other words, is pi j = p· j pi· ? If you knew each pi j this would be no problem but you have
no idea about pi j .
The null hypothesis will be that pi j = p· j pi· . Then the p· j , pi· are to be considered as
parameters. The first item is to find the maximum likelihood estimates for these based on
a random sample of size n Xi j , i = 1, 2 and j = 1, 2, 3. Here the sample has been indexed
according to which “cell” Bi ∩ A j contains the sample point. Then the likelihood based on
the null hypothesis is to maximize
2 3
X X
∏ ∏ p· ji j pi·i j
i=1 j=1

subject to the constraint that ∑i pi· = 1as usual, it works best to maximize the ln of the
above. Thus maximize
2 3
∑ ∑ (Xi j ln (p· j ) + Xi j ln (pi,· )) , ∑ pi· = 1
i=1 j=1 i

First consider the pi· . This amounts to maximizing

2
∑ Si ln (pi,· ) , ∑ pi· = 1
i=1 i

where Si ≡ ∑ j Xi j . Using the method of Lagrange multipliers, we need

   
S1 S2
p1· p2· = λ 1 1

S2 ∑i ∑ j Xi j
thus λ p1· = S1 , λ p2· = S2 . Thus λ + Sλ1 = 1 and so λ = 1 and so λ = n. Then
Si ∑ j Xi j
p̂i· = =
n n
where this is the maximum likelihood estimate for pi· . Similar reasoning shows that
∑i Xi j
p̂· j = .
n
Now form
(Xi j − n p̂i· p̂· j )2
D≡∑
i, j n p̂i· p̂· j

By what was explained above, this is X 2 ((2 × 3 − 1) − 3) . The reason there is a 3 there
rather than a 5 is that there are only 3 unknown parameters due to the fact that ∑i pi· =
1, ∑ j p· j = 1. In general, if the table is r × s, the above expression would be

X 2 ((rs − 1) − (r + s − 2))
This justifies the following proposition.
39.7. CONTINGENCY TABLES 841

Proposition 39.7.1 Let there be an r × s contingency table such that the random variable
is in exactly one of Bi ∩ A j for i = 1, · · · , r, j = 1, · · · , s. If P (Bi ∩ A j ) = P (Bi ) P (A j ) for all
i, j, then if a sample is taken of size n and Xi j is the observed number in Bi ∩ A j , then when
n is large,
(Xi j − n p̂i· p̂· j )2
D≡∑
i, j n p̂i· p̂· j

is distributed as X 2 ((rs − 1) − (r + s − 2)) = X 2 (rs − s − r + 1).

Assuming the null hypothesis that the events Bi and A j are independent, one can now
test this hypothesis by using a graph or table for X 2 (rs − s − r + 1) .

Example 39.7.2 You have a 3 × 2 contingency table, three rows and two columns. Also
the number in a random sample is 900. The numbers of observations found in the various
positions are illustrated in the following.
120 300
180 80
100 120
Determine whether the underlying contingency table has the property that the events
could be independent. If the probability is no more than .01 that the events are independent,
reject the null hypothesis. Otherwise conclude that the events might be independent.

In the above, n = 900. Now lets find the p̂.

420 260 220
p̂1· = , p̂2· = , p̂3· =
900 900 900
400 500
p̂·1 = , p̂·2 =
900 900
Now assemble D.
420 400



2 420

500


2
120 − 900 900 900 300 − 900 900 900
D= 420

400

+ 420

500


900 900 900 900 900 900

260

400


2 260

500


2
180 − 900 900 900 80 − 900 900 900
+ 260

400

+ 260

500


900 900 900 900 900 900
220

400 2


220

500


2
100 − 900 900 120 − 900
+ 220

400900

+ 220
900

500
900


900 900 900 900 900 900
Now compute this.
D = 107. 64
This is way too big to accept the null hypothesis. The events are not independent. The
statistic is distributed as X 2 (2) and a table gives probability 1 that the variable is less than
10. Yet D is larger than 100.
842 CHAPTER 39. STATISTICAL TESTS

Example 39.7.3 You have a 2 × 2 contingency table, three rows and two columns. Also
the number in a random sample is 1500. The numbers of observations found in the various
positions are illustrated in the following.
297 196
600 407
Determine whether the underlying contingency table has the property that the events
could be independent. If the probability is no more than .5 that the events are independent,
reject the null hypothesis. Otherwise conclude that the events might be independent.

First, what are the p̂?

493 1007 897 201
p̂1· = , p̂2· = , p̂·1 = , p̂·2 =
1500 1500 1500 500
Now find D.
493

897


2 493

201


2
297 − 1500 1500 1500 196 − 1500 1500 500
D= 493

897

+ 493

201


1500 1500 1500 1500 1500 500

1007

897


2 1007

201


2
600 − 1500 1500 407 − 1500
+ 493

2011500

+ 1500
1007

201500


1500 1500 500 1500 1500 500

D = 7. 623 7 × 10−2
This is distributed as X 2 ((4 − 1) − (2)) = X 2 (1) . From a table or graph, (.1, .248) is on
the graph of the distribution function. Therefore, since D is far smaller than .1, we don’t
reject the hypothesis that the sets are independent. Not being independent is indicated by
D being larger than some number a where the probability that a X 2 (1) random variable
is larger than a is very small, but this D is very small, so it is highly probable under the
null hypothesis that X 2 (1) > .1. This does not mean that the sets are independent. It
only means we don’t reject the possibility that they are. This is termed acceptance of
the hypothesis but you might fail to have independence even though you accept the
hypothesis.
Appendix A

The Theory Of The Riemannn

Integral∗

A.1 An Important Warning

If you read and understand this appendix on the Riemann integral you will become abnor-
mal if you are not already that way. You will laugh at atrocious puns. You will be unpopular
with well adjusted confident people, especially religious people who love to accept on faith
inconsistent decrees of authority figures. Furthermore, your confidence will be completely
shattered. Virtually nothing will be obvious to you ever again. Consider whether it would
be better to accept the superficial presentation given earlier than to attempt to acquire deep
understanding of the integral, risking your self esteem and confidence, before proceeding
further. This is only here for those who need explanations and are not content to accept on
faith. This chapter is one of the worst things I have seen and I don’t know how to improve
it without losing the rigor. I think it is a good illustration why, if you want to do integra-
tion, you should approach it as a Lebesgue integral. This is found in my book Calculus of
functions of real and complex variables or Calculus of One and Many Variables. It will be
much more abstract, but much less filled with mind numbing technicalities.

A.2 Basic Definition

The definition of the Riemannn integral of a function of n variables uses the following
definition.
∞
Definition A.2.1 For i = 1, · · · , n, let α ik k=−∞ be points on R which satisfy


lim α ik = ∞, lim α ik = −∞, α ik < α ik+1 . (1.1)

k→∞ k→−∞

843
844 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

For such sequences, define a grid on Rn denoted by G or F as the collection of boxes of

the form
n 
Q = ∏ α iji , α iji +1 .

(1.2)
i=1
If G is a grid, F is called a refinement of G if every box of G is the union of boxes of F .

Lemma A.2.2 If G and F are two grids, they have a common refinement, denoted here by
G ∨F.
n o∞
Proof: Let α ik k=−∞ be the sequences used to construct G and let β ik
 ∞
be the
 ik=−∞
sequence used to construct F . Now let γ k k=−∞ denote the union of α k k=−∞ and
 i ∞ ∞
n o∞
β ik . It is necessary to show that for each i these points can be arranged in order. To
k=−∞
do so, let γ i0 ≡ α i0 . Now if
γ i− j , · · · , γ i0 , · · · , γ ij
have been chosen such that they are in order and all distinct, let γ ij+1 be the first element of
 i ∞ n o∞
α k k=−∞ ∪ β ik (1.3)
k=−∞

which is larger than γ ij and let γ i−( j+1) be the last element of (1.3) which is strictly smaller
than γ i− j . The assumption (1.1) insures such a first and last element exists. Now let the grid
G ∨ F consist of boxes of the form
n h i
Q ≡ ∏ γ iji , γ iji +1 . ■
i=1

The Riemannn integral is only defined for functions f which are bounded and are equal
to zero off some bounded set D. In what follows f will always be such a function.

Definition A.2.3 Let f be a bounded function which equals zero off a bounded set D, and
let G be a grid. For Q ∈ G , define

MQ ( f ) ≡ sup { f (x) : x ∈ Q} , mQ ( f ) ≡ inf { f (x) : x ∈ Q} . (1.4)

Also define for Q a box, the volume of Q, denoted by v (Q) by

n n
v (Q) ≡ ∏ (bi − ai ) , Q ≡ ∏ [ai , bi ] .
i=1 i=1

Now define upper sums, UG ( f ) and lower sums, LG ( f ) with respect to the indicated grid,
by the formulas

UG ( f ) ≡ ∑ MQ ( f ) v (Q) , LG ( f ) ≡ ∑ mQ ( f ) v (Q) .
Q∈G Q∈G

A function of n variables is Riemannn integrable when there is a unique number between

all the upper and lower sums. This number is the value of the integral.

Note that in this definition, MQ ( f ) = mQ ( f ) = 0 for all but finitely many Q ∈ G so

there are no convergence questions to be considered here.
A.2. BASIC DEFINITION 845

Lemma A.2.4 If F is a refinement of G then

UG ( f ) ≥ UF ( f ) , LG ( f ) ≤ LF ( f ) .

Also if F and G are two grids,

LG ( f ) ≤ UF ( f ) .

Proof: For P ∈ G let Pb denote the set

{Q ∈ F : Q ⊆ P} .

Then P = ∪Pb and

LF ( f ) ≡ ∑ mQ ( f ) v (Q) = ∑ ∑ mQ ( f ) v (Q)
Q∈F P∈G Q∈Pb

≥ ∑ mP ( f ) ∑ v (Q) = ∑ mP ( f ) v (P) ≡ LG ( f ) .
P∈G Q∈Pb P∈G

Similarly, the other inequality for the upper sums is valid.

To verify the last assertion of the lemma, use Lemma A.2.2 to write

LG ( f ) ≤ LG ∨F ( f ) ≤ UG ∨F ( f ) ≤ UF ( f ) . ■

This lemma makes it possible to define the Riemannn integral.

Definition A.2.5 Define an upper and a lower integral as follows.

I ( f ) ≡ inf {UG ( f ) : G is a grid} ,

I ( f ) ≡ sup {LG ( f ) : G is a grid} .

Lemma A.2.6 I ( f ) ≥ I ( f ).

Proof: From Lemma A.2.4 it follows for any two grids G and F ,

LG ( f ) ≤ UF ( f ) .

Therefore, taking the supremum for all grids on the left in this inequality,

I ( f ) ≤ UF ( f )

for all grids F . Taking the infimum in this inequality, yields the conclusion of the lemma.
■

Definition A.2.7 A bounded function f which equals zero off a bounded set D, is said to be
Riemannn integrable, written as f ∈ R (Rn ) exactly when I ( f ) = I ( f ). In this case define
Z Z
f dV ≡ f dx = I ( f ) = I ( f ) .

As in the case of integration of functions of one variable, one obtains the Riemannn
criterion which is stated as the following theorem.
846 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

Theorem A.2.8 (Riemannn criterion) f ∈ R (Rn ) if and only if for all ε > 0 there exists a
grid G such that
UG ( f ) − LG ( f ) < ε.

Proof: If f ∈ R (Rn ), then I ( f ) = I ( f ) and so there exist grids G and F such that
ε  ε
UG ( f ) − LF ( f ) ≤ I ( f ) + − I ( f ) − = ε.
2 2
Then letting H = G ∨ F , Lemma A.2.4 implies

UH ( f ) − LH ( f ) ≤ UG ( f ) − LF ( f ) < ε.

Conversely, if for all ε > 0 there exists G such that

UG ( f ) − LG ( f ) < ε,

then
I ( f ) − I ( f ) ≤ UG ( f ) − LG ( f ) < ε.
Since ε > 0 is arbitrary, this proves the theorem. ■

A.3 Basic Properties

It is important to know that certain combinations of Riemannn integrable functions are
Riemannn integrable. The following theorem will include all the important cases.

Theorem A.3.1 Let f , g ∈ R (Rn ) and let φ : K → R be continuous where K is a compact

set in R2 containing f (Rn ) × g (Rn ). Also suppose that φ (0, 0) = 0. Then defining

h (x) ≡ φ ( f (x) , g (x)) ,

it follows that h is also in R (Rn ).

Proof: Let ε > 0 and let δ 1 > 0 be such that if (yi , zi ) , i = 1, 2 are points in K, such that
|z1 − z2 | ≤ δ 1 and |y1 − y2 | ≤ δ 1 , then

|φ (y1 , z1 ) − φ (y2 , z2 )| < ε.

Let 0 < δ < min (δ 1 , ε, 1). Let G be a grid with the property that for Q ∈ G , the diameter
of Q is less than δ and also for k = f , g,

UG (k) − LG (k) < δ 2 . (1.5)

Then defining for k = f , g,

Pk ≡ {Q ∈ G : MQ (k) − mQ (k) > δ } ,

it follows
δ2 > ∑ (MQ (k) − mQ (k)) v (Q) ≥
Q∈G
A.3. BASIC PROPERTIES 847

∑ (MQ (k) − mQ (k)) v (Q) ≥ δ ∑ v (Q)

Pk Pk

and so for k = f , g,
ε > δ > ∑ v (Q) . (1.6)
Pk

Suppose for k = f , g,
MQ (k) − mQ (k) ≤ δ .
Then if x1 , x2 ∈ Q,

| f (x1 ) − f (x2 )| < δ , and |g (x1 ) − g (x2 )| < δ .

Therefore,

|h (x1 ) − h (x2 )| ≡ |φ ( f (x1 ) , g (x1 )) − φ ( f (x2 ) , g (x2 ))| < ε

and it follows that

|MQ (h) − mQ (h)| ≤ ε.
Now let
S ≡ {Q ∈ G : 0 < MQ (k) − mQ (k) ≤ δ , k = f , g} .
Thus the union of the boxes in S is contained in some large box, R, which depends only
on f and g and also, from the assumption that φ (0, 0) = 0, MQ (h) − mQ (h) = 0, unless
Q ⊆ R. Then
UG (h) − LG (h) ≤ ∑ (MQ (h) − mQ (h)) v (Q) +
Q∈P f

∑ (MQ (h) − mQ (h)) v (Q) + ∑ δ v (Q) .

Q∈Pg Q∈S

Now since K is compact, it follows φ (K) is bounded and so there exists a constant C,
depending only on h and φ such that MQ (h) − mQ (h) < C. Therefore, the above inequality
implies
UG (h) − LG (h) ≤ C ∑ v (Q) +C ∑ v (Q) + ∑ δ v (Q) ,
Q∈P f Q∈Pg Q∈S

which by (1.6) implies

UG (h) − LG (h) ≤ 2Cε + δ v (R) ≤ 2Cε + εv (R) .

Since ε is arbitrary, the Riemannn criterion is satisfied and so h ∈ R (Rn ). ■

Corollary A.3.2 Let f , g ∈ R (Rn ) and let a, b ∈ R. Then a f + bg, f g, and | f | are all in
R (Rn ). Also, Z Z Z
(a f + bg) dx = a f dx + b g dx, (1.7)
Rn Rn Rn
and Z Z
| f | dx ≥ f dx . (1.8)
848 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

Proof: Each of the combinations of functions described above is Riemannn integrable

by Theorem A.3.1. For example, to see a f + bg ∈ R (Rn ) consider φ (y, z) ≡ ay + bz. This
is clearly a continuous function of (y, z) such that φ (0, 0) = 0. To obtain | f | ∈ R (Rn ), let
φ (y, z) ≡ |y|. It remains to verify the formulas. To do so, let G be a grid with the property
that for k = f , g, | f | and a f + bg,

UG (k) − LG (k) < ε. (1.9)

Consider (1.7). For each Q ∈ G pick a point in Q, xQ . Then

∑ k (xQ ) v (Q) ∈ [LG (k) , UG (k)]

Q∈G

and so
Z
k dx − ∑ k (xQ ) v (Q) < ε.
Q∈G

Consequently, since
∑ (a f + bg) (xQ ) v (Q)
Q∈G

=a ∑ f (xQ ) v (Q) + b ∑ g (xQ ) v (Q) ,

Q∈G Q∈G

it follows Z Z Z
(a f + bg) dx − a f dx − b g dx ≤

Z
(a f + bg) dx − ∑ (a f + bg) (xQ ) v (Q) +
Q∈G
Z Z
a ∑ f (xQ ) v (Q) − a f dx + b ∑ g (xQ ) v (Q) − b g dx
Q∈G Q∈G

≤ ε + |a| ε + |b| ε.
Since ε is arbitrary, this establishes (1.7) and shows the integral is linear.
It remains to establish the inequality (1.8). By (1.9), and the triangle inequality for
sums, Z
| f | dx + ε ≥ ∑ | f (xQ )| v (Q)
Q∈G
Z
≥ ∑ f (xQ ) v (Q) ≥ f dx − ε.
Q∈G

Then since ε is arbitrary, this establishes the desired inequality. ■

A.4. WHICH FUNCTIONS ARE INTEGRABLE? 849

A.4 Which Functions Are Integrable?

Which functions are in R (Rn )? As in the case of integrals of functions of one variable, this
is an important question. It turns out the Riemannn integrable functions are characterized
by being continuous except on a very small set. This has to do with Jordan content.
Definition A.4.1 A bounded set E, has Jordan content 0 or content 0 if for every ε > 0
there exists a grid G such that
∑ v (Q) < ε.
Q∩E̸=0/

This symbol says to sum the volumes of all boxes from G which have nonempty intersection
with E.
Next it is necessary to define the oscillation of a function.
Definition A.4.2 Let f be a function defined on Rn and let
ω f ,r (x) ≡ sup {| f (z) − f (y)| : z, y ∈ B (x,r)} .
This is called the oscillation of f on B (x,r). Note that this function of r is decreasing in r.
Define the oscillation of f as
ω f (x) ≡ lim ω f ,r (x) .
r→0+

 ω f ,r (x) decreases. It is also bounded below by 0,

Note that as r decreases, the function
so the limit must exist and equals inf ω f ,r (x) : r > 0 . (Why?) Then the following simple
lemma whose proof follows directly from the definition of continuity gives the reason for
this definition.
Lemma A.4.3 A function f is continuous at x if and only if ω f (x) = 0.
This concept of oscillation gives a way to define how discontinuous a function is at a
point. The discussion will depend on the following fundamental lemma which gives the
existence of something called the Lebesgue number.
Definition A.4.4 Let C be a set whose elements are sets of Rn and let K ⊆ Rn . The set C
is called a cover of K if every point of K is contained in some set of C. If the elements of C
are open sets, it is called an open cover.
Lemma A.4.5 Let K be sequentially compact and let C be an open cover of K. Then there
exists r > 0 such that whenever x ∈ K, B(x, r) is contained in some set of C.
Proof: Suppose this is not so. Then letting rn = 1/n, there exists xn ∈ K such that
B (xn , rn ) is not contained in any set of C. Since K is sequentially compact, there is a
subsequence, xnk which converges to a point x ∈ K. But there exists δ > 0 such that
B (x, δ ) ⊆ U for some
U ∈ C. Let k be so large that 1/k < δ /2 and xnk − x < δ /2 also.
Then if z ∈ B xnk , rnk , it follows
δ δ
|z − x| ≤ z − xnk + xnk − x < + =δ
2 2


and so B xnk , rnk ⊆ U contrary to supposition. Therefore, the desired number exists after
all. ■
850 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

Theorem A.4.6 Let f be a bounded function which equals zero off a bounded set and let
W denote the set of points where f fails to be continuous. Then f ∈ R (Rn ) if W has
content zero. That is, for all ε > 0 there exists a grid G such that

∑ v (Q) < ε (1.10)

Q∈GW

where
GW ≡ {Q ∈ G : Q ∩W ̸= 0}
/ .

Proof: Let W have content zero. Also let | f (x)| < C/2 for all x ∈ Rn , let ε > 0 be
given, and let G be a grid which satisfies (1.10). Since
f equals zero off some
bounded
set, there exists R such that f equals zero off of B 0, R2 . Thus W ⊆ B 0, R2 . Also note
that if G is a grid for which (1.10) holds, then this inequality continues to hold if G is
replaced with a refined grid. Therefore, you may assume the diameter of every box in G
which intersects B (0, R) is less than R3 and so all boxes of G which intersect the set where
f is nonzero are contained in B (0,R). Since W is bounded, GW contains only finitely many
boxes. Letting
n
Q ≡ ∏ [ai , bi ]
i=1
be one of these boxes, enlarge the box slightly as indicated in the following picture.

Q̃

The enlarged box is an open set of the form,

n
e ≡ ∏ (ai − η i , bi + η i )
Q
i=1

where η i is chosen small enough that if

n  
∏ ( bi + η i − (ai − η i )) ≡ v e ,
Q
i=1

and Gf
W denotes those Q for Q ∈ G which have nonempty intersection with W , then
e
 
∑ Qe < ε
v (1.11)
e
e Gf
Q∈ W

where Q is the box,

ee
n
∏ ((ai − 2η i ) , bi + 2η i ) .
i=1
For each x ∈ Rn , let rx < min (η 1 /2, · · · , η n /2) be such that

ω f ,rx (x) < ε + ω f (x) . (1.12)

A.4. WHICH FUNCTIONS ARE INTEGRABLE? 851

e ∩ B (x,rx ) such that x ∈ B (0,R) so that

Now let C denote all intersections of the form Q
C is an open cover of the compact set B (0,R). Let δ be a Lebesgue number for this open
cover of B (0,R) and let F be a refinement of G such that every box in F has diameter
less than δ . Now let F1 consist of those boxes of F which have nonempty intersection
with B (0,R/2). Thus all boxes of F1 are contained in B (0,R) and each one is contained in
e ∩ B (x,rx ), for which x ∈ W . Thus each
some set of C. Let CW be those open sets of C, Q
of these sets is contained in some Q where Q ∈ GW . Let FW be those sets of F1 which are
ee
subsets of some set of CW . Thus

∑ v (Q) < ε. (1.13)

Q∈FW

because each Q in FW is contained in a set Q described above and the sum of the volumes
ee
of these is less than ε by (1.11). Then

UF ( f ) − LF ( f ) = ∑ (MQ ( f ) − mQ ( f )) v (Q)
Q∈FW

+ ∑ (MQ ( f ) − mQ ( f )) v (Q) .
Q∈F1 \FW

If Q ∈ F1 \ FW , then Q must be a subset of some set of C \CW since it is not in any set of
CW . Say Q ⊆ Q f1 ∩ B (x,rx ) where x ∈
/ W . Therefore, from (1.12) and the observation that
x∈ / W , it follows ω f (x) = 0 and so

MQ ( f ) − mQ ( f ) ≤ ε.

Therefore, from (1.13) and the estimate on f ,

UF ( f ) − LF ( f ) ≤ ∑ Cv (Q) + ∑ εv (Q)
Q∈FW Q∈F1 \FW

≤ Cε + ε (2R)n ,
the estimate of the second sum coming from the fact that
n
B (0, R) ⊆ ∏ [−R, R] .
i=1

Since ε is arbitrary, this proves the theorem.1 ■

Definition A.4.7 A bounded set E is a Jordan set in Rn , also called a contented set in Rn
if XE ∈ R (Rn ). The symbol XE means
(
1 if x ∈ E
XE (x) =
0 if x ∈
/E
1 Infact one cannot do any better. It can be shown that if a function is Riemann integrable, then it must be
the case that for all ε > 0, (1.10) is satisfied for some grid G . This along with what was just shown is known as
Lebesgue’s theorem after Lebesgue who discovered it in the early years of the twentieth century. Actually, he also
invented a far superior integral which made the Riemann integral which is the topic of this appendix obsolete.
852 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

It is called the indicator function because it indicates whether x is in E according to

whether it equals 1. For a function f ∈ R (Rn ) and E a contented set, f XE ∈ R (Rn )
by Corollary A.3.2. Then Z Z
f dV ≡ f XE dV.
E

So what are examples of contented sets?

Theorem A.4.8 Suppose E is a bounded contented set in Rn and f , g : E → R are two

functions satisfying f (x) ≥ g (x) for all x ∈ E and f XE and gXE are both in R (Rn ).
Now define
P ≡ {(x,xn+1 ) : x ∈ E and g (x) ≤ xn+1 ≤ f (x)} .
Then P is a contented set in Rn+1 .

Proof: Let G be a grid such that for k = f XE , gXE ,

UG (k) − LG (k) < ε/4. (1.14)

Also let K ≥ ∑mj=1 vn (Q j ) where the Q j are the boxes which intersect E. Let {ai }∞
i=−∞ be a
sequence on R, ai < ai+1 for all i, which includes
ε
MQ j ( f XE ) + , MQ j ( f XE ) , MQ j (gXE ) ,
4mK
ε
mQ j ( f XE ) , mQ j (gXE ) , mQ j (gXE ) −
4mK
for all j = 1, · · · , m. Now define a grid on Rn+1 as follows.

G ′ ≡ {Q × [ai , ai+1 ] : Q ∈ G , i ∈ Z}

In words, this grid consists of all possible boxes of the form Q × [ai , ai+1 ] where Q ∈ G
and ai is a term
of the sequence just described. It is necessary to verify that for P ∈ G ′ ,
XP ∈ R Rn+1 . This is done by showing that UG ′ (XP ) − LG ′ (XP ) < ε and then noting
that ε > 0 was arbitrary. For G ′ just described, denote by Q′ a box in G ′ . Thus Q′ =
Q × [ai , ai+1 ] for some i.

UG ′ (XP ) − LG ′ (XP ) ≡ ∑ MQ′ (XP ) − mQ′ (XP ) vn+1 Q′

Q′ ∈G ′
∞ m 
= ∑ M
∑ Qj′ (XP ) − m ′
Qj (XP ) vn (Q j ) (ai+1 − ai )
i=−∞ j=1

and all sums are bounded because the functions f and g are given to be bounded. Therefore,
there are no limit considerations needed here. Thus

UG ′ (XP ) − LG ′ (XP ) =
m ∞  
∑ vn (Q j ) ∑ MQ j ×[ai ,ai+1 ] (XP ) − mQ j ×[ai ,ai+1 ] (XP ) (ai+1 − ai ) .
j=1 i=−∞

Consider the inside sum with the aid of the following picture.
A.4. WHICH FUNCTIONS ARE INTEGRABLE? 853

xn+1 = g(x) xn+1 = f (x)

x

Qj 0 0 0 0 0 0 0 0 0

mQ j (g) MQ j (g)
xn+1

In this picture, the little rectangles represent the boxes Q j × [ai , ai+1 ] for fixed j. The
part of P having x contained in Q j is between the two surfaces, xn+1 = g (x) and xn+1 =
f (x) and there is a zero placed in those boxes for which

MQ j ×[ai ,ai+1 ] (XP ) − mQ j ×[ai ,ai+1 ] (XP ) = 0.

You see, XP has either the value of 1 or the value of 0 depending on whether (x, y) is
contained in P. For the boxes shown with 0 in them, either all of the box is contained in P
or none of the box is contained in P. Either way,

MQ j ×[ai ,ai+1 ] (XP ) − mQ j ×[ai ,ai+1 ] (XP ) = 0

on these boxes. However, on the boxes intersected by the surfaces, the value of

MQ j ×[ai ,ai+1 ] (XP ) − mQ j ×[ai ,ai+1 ] (XP )

is 1 because there are points in this box which are not in P as well as points which are in P.
Because of the construction of G ′ which included all values of
ε
MQ j ( f XE ) + , MQ j ( f XE ) ,
4mK
MQ j (gXE ) , mQ j ( f XE ) , mQ j (gXE )

for all j = 1, · · · , m,
∞  
∑ MQ j ×[ai ,ai+1 ] (XP ) − mQ j ×[ai ,ai+1 ] (XP ) (ai+1 − ai ) ≤
i=−∞

∑ 1 (ai+1 − ai ) + ∑ 1 (ai+1 − ai ) (1.15)

n o n o
i:mQ j (gXE )≤ai <MQ j (gXE ) i:mQ j ( f XE )≤ai <MQ j ( f XE )

The first of the sums in (1.15) contains all possible terms for which

MQ j ×[ai ,ai+1 ] (XP ) − mQ j ×[ai ,ai+1 ] (XP )

might be 1 due to the graph of the bottom surface gXE while the second sum contains
all possible terms for which the expression might be 1 due to the graph of the top surface
f XE .
 ε   ε 
≤ MQ j (gXE ) + − mQ j (gXE ) + MQ j ( f XE ) + − mQ j ( f XE )
4mK 4mK
854 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

!−1
m
ε
= MQ j (gXE ) − mQ j (gXE ) + MQ j ( f XE ) − mQ j ( f XE ) +



2m ∑ v (Q j ) .
j=1

Therefore, by (1.14),
UG ′ (XP ) − LG ′ (XP ) ≤

m
MQ j (gXE ) − mQ j (gXE ) + MQ j ( f XE ) − mQ j ( f XE )



∑ vn (Q j )
j=1
!−1
m m
ε
+ ∑ v (Q j ) ∑ v (Q j )
j=1 2m j=1
ε
= UG ( f ) − LG ( f ) + UG (g) − LG (g) +
2
ε ε ε
< + + = ε.
4 4 2
Since ε > 0 is arbitrary, this proves the theorem. ■
Corollary A.4.9 Suppose f and g are continuous functions defined on E, a contented set
in Rn and that g (x) ≤ f (x) for all x ∈ E. Then
P ≡ {(x,xn+1 ) : x ∈ E and g (x) ≤ xn+1 ≤ f (x)}
is a contented set in Rn .
Proof: Since E is contented, meaning XE is integrable, it follows from Theorem A.4.6
the set of discontinuities of XE has Jordan content 0. But the set of discontinuities of XE
is ∂ E defined as those points x such that B (x, r) contains points of E and points of E C
for every r > 0. Extend f and g to equal 0 off E. Then the set of discontinuities of these
extended functions still denoted as f , g is ∂ E which has Jordan content 0. This reduces to
the situation of Theorem A.4.8. ■
As an example of how this can be applied, it is obvious a closed interval is a contented
set in R. Therefore, if f , g are two continuous functions with f (x) ≥ g (x) for x ∈ [a, b], it
follows from the above theorem or its corollary that the set
P1 ≡ {(x, y) : g (x) ≤ y ≤ f (x)}
is a contented set in R2 . Now using the theorem and corollary again, suppose f1 (x, y) ≥
g1 (x, y) for (x, y) ∈ P1 and f , g are continuous. Then the set
P2 ≡ {(x, y, z) : g1 (x, y) ≤ z ≤ f1 (x, y)}
is a contented set in R3 . Clearly you can continue this way obtaining examples of contented
sets. ■
Note that as a special case, it follows that every box is a contented set. Therefore, if Bi
is a box, functions of the form
m
∑ ai XBi
i=1
are integrable. These functions are called step functions.
The following theorem is analogous to the fact that in one dimension, when you inte-
grate over a point, the answer is 0.
A.4. WHICH FUNCTIONS ARE INTEGRABLE? 855

Theorem A.4.10 If a bounded set E, has Jordan content 0, then E is a Jordan (contented)
set and if f is any bounded function defined on E, then f XE ∈ R (Rn ) and
Z
f dV = 0.
E

Proof: Let m be a lower bound for f and let M be an upper bound. Let G be a grid with
ε
∑ v (Q) < .
Q∩E̸=0/ 1 + (M − m)

Then
εM
UG ( f XE ) ≤ ∑ Mv (Q) ≤
Q∩E̸=0/ 1 + (M − m)
and
εm
LG ( f XE ) ≥ ∑ mv (Q) ≥
Q∩E̸=0/ 1 + (M − m)
and so

UG ( f XE ) − LG ( f XE ) ≤ ∑ Mv (Q) − ∑ mv (Q)
Q∩E̸=0/ Q∩E̸=0/
ε (M − m)
= (M − m) ∑ v (Q) < < ε.
Q∩E̸=0/ 1 + (M − m)

This shows f XE ∈ R (Rn ). Now also,

Z
mε ≤ f XE dV ≤ Mε

and since ε is arbitrary, this shows

Z Z
f dV ≡ f XE dV = 0
E

Why is E contented? Let G be a grid for which

∑ v (Q) < ε
Q∩E̸=0/

Then for this grid,

UG (XE ) − LG (XE ) ≤ ∑ v (Q) < ε
Q∩E̸=0/

and this proves the theorem. ■

Corollary A.4.11 If f XEi ∈ R (Rn ) for i = 1, 2, · · · , r and for all i ̸= j, Ei ∩ E j is either the
empty set or a set of Jordan content 0, then letting F ≡ ∪ri=1 Ei , it follows f XF ∈ R (Rn )
and Z Z Z r
f XF dV ≡ f dV = ∑ f dV.
F i=1 Ei
856 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

Proof: This is true if r = 1. Suppose it is true for r. It will be shown that it is true
for r + 1. Let Fr = ∪ri=1 Ei and let Fr+1 be defined similarly. By the induction hypothesis,
f XFr ∈ R (Rn ). Also, since Fr is a finite union of the Ei , it follows that Fr ∩ Er+1 is either
empty or a set of Jordan content 0.

− f XFr ∩Er+1 + f XFr + f XEr+1 = f XFr+1

and by Theorem A.4.10 each function on the left is in R (Rn ) and the first one on the left
has integral equal to zero. Therefore,
Z Z Z
f XFr+1 dV = f XFr dV + f XEr+1 dV

which by induction equals

r Z Z r+1 Z
∑ f dV + f dV = ∑ f dV
i=1 Ei Er+1 i=1 Ei

and this proves the corollary. ■

In particular, for
n n
Q = ∏ [ai , bi ] , Q′ = ∏(ai , bi ]
i=1 i=1

both are contented sets and

Z Z
XQ dV = XQ′ dV = v (Q) . (1.16)
Q′

This is because
Q \ Q′ = ∪ni=1 ai × ∏(a j , b j ]
j̸=i

a finite union of sets of content 0. It is obvious XQ dV = v (Q) because you can use a grid
R

which has Q as one of the boxes and then the upper and lower sums are the same and equal
to v (Q). Therefore, the claim about the equality of the two integrals in (1.16) follows right
away from Corollary A.4.11. That XQ′ is integrable follows from

XQ′ = XQ − XQ\Q′

and each of the two functions on the right is integrable thanks to Theorem A.4.10.
In fact, here is an interesting version of the Riemannn criterion which depends on these
half open boxes.

Lemma A.4.12 Suppose f is a bounded function which equals zero off some bounded set.
Then f ∈ R (Rn ) if and only if for all ε > 0 there exists a grid G such that


∑ MQ′ ( f ) − mQ′ ( f ) v (Q) < ε. (1.17)
Q∈G

Proof: Since Q′ ⊆ Q,

MQ′ ( f ) − mQ′ ( f ) ≤ MQ ( f ) − mQ ( f )
A.5. ITERATED INTEGRALS 857

and therefore, the only if part of the equivalence is obvious.

Conversely, let G be a grid such that (1.17) holds with ε replaced with ε2 . It is necessary
to show that there is a grid such that (1.17) holds with no primes on the Q. Let F be a
refinement of G obtained by adding the points α ik + η k where η k ≤ η and is also chosen
so small that for each i = 1, · · · , n,

α ik + η k < α ik+1 .

You only need to have η k > 0 for the finitely many boxes of G which intersect the bounded
set where f is not zero. Then for
n 
Q ≡ ∏ α iki , α iki +1 ∈ G ,

i=1

Let
n 
b ≡ ∏ αi + ηk , αi

Q ki i ki +1
i=1

and denote by Gb the collection of these smaller boxes. For each set Q in G there is the
smaller set Qb along with n boxes, Bk , k = 1, · · · , n, one of whose sides is of length η k and
the remainder of whose sides are shorter than the diameter of Q such that the set
Q is the
union of Q b and these sets Bk . Now suppose f equals zero off the ball B 0, R . Then
2
without loss of generality, you may assume the diameter of every box in G which has
nonempty intersection with B (0,R) is smaller than R3 . (If this is not so, simply refine G
to make it so, such a refinement leaving (1.17) valid because refinements do not increase
the difference between upper and lower sums in this context either.) Suppose there are P
sets of G contained in B (0,R) (So these are the only sets of G which could have nonempty
intersection with the set where f is nonzero.) and suppose that for all x, | f (x)| < C/2.
Then  
∑ (MQ ( f ) − mQ ( f )) v (Q) ≤ ∑ MQb ( f ) − mQb ( f ) v (Q)
Q∈F b Gb
Q∈

+ ∑ (MQ ( f ) − mQ ( f )) v (Q)
Q∈F \Gb

The first term on the right of the inequality in the above is no larger than ε/2 because
MQb ( f ) − mQb ( f ) ≤ MQ′ ( f ) − mQ′ ( f ) for each Q. Therefore, the above is dominated by

≤ ε/2 +CPnRn−1 η < ε

whenever η is small enough. Since ε is arbitrary, f ∈ R (Rn ) as claimed. ■

A.5 Iterated Integrals

To evaluate an n dimensional Riemannn integral, one uses iterated integrals. Formally, an
iterated integral is defined as follows. For f a function defined on Rn+m ,

y → f (x, y)
858 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

is a function of y for each x ∈ Rn . Therefore, it might be possible to integrate this function

of y and write Z
f (x, y) dVy .
Rm
Now the result is clearly a function of x and so, it might be possible to integrate this and
write Z Z
f (x, y) dVy dVx .
Rn Rm
This symbol is called an iterated integral, because it involves the iteration of two lower
dimensional integrations. Under what conditions are the two iterated integrals equal to the
integral Z
f (z) dV ?
Rn+m

Definition A.5.1 Let G be a grid on Rn+m defined by the n + m sequences,

 i ∞
α k k=−∞ i = 1, · · · , n + m.

Let Gn be the grid on Rn obtained by considering only the first n of these sequences and
let Gm be the grid on Rm obtained by considering only the last m of the sequences. Thus a
typical box in Gm would be
n+m  i i 
∏ α ki , α ki +1 , ki ≥ n + 1
i=n+1

and a box in Gn would be of the form

n  i i 
∏ α ki , α ki +1 , ki ≤ n.
i=1

Lemma A.5.2 Let G , Gn , and Gm be the grids defined above. Then

G = {R × P : R ∈ Gn and P ∈ Gm } .

Proof: If Q ∈ G , then Q is clearly of this form. On the other hand, if R × P is one of the
sets described above, then from the above description of R and P, it follows R × P is one of
the sets of G . ■
Now let G be a grid on Rn+m and suppose

φ (z) = ∑ φ Q XQ′ (z) (1.18)

Q∈G

where φ Q equals zero for all but finitely many Q. Thus φ is a step function. Recall that for

n+m n+m
Q= ∏ [ai , bi ] , Q′ ≡ ∏ (ai , bi ]
i=1 i=1

The function
φ= ∑ φ Q XQ′
Q∈G
A.5. ITERATED INTEGRALS 859

is integrable because it is a finite sum of integrable functions, each function in the sum
being integrable because the set of discontinuities has Jordan content 0. (why?) Letting
(x, y) = z,
φ (z) = φ (x, y) = ∑ ∑ φ R×P XR′ ×P′ (x, y)
R∈Gn P∈Gm

= ∑ ∑ φ R×P XR′ (x) XP′ (y) . (1.19)

R∈Gn P∈Gm

For a function of two variables h, denote by h (·, y) the function x → h (x, y) and
h (x, ·) the function y → h (x, y). The following lemma is a preliminary version of Fubini’s
theorem.

Lemma A.5.3 Let φ be a step function as described in (1.18). Then

φ (x, ·) ∈ R (Rm ) , (1.20)

Z
φ (·, y) dVy ∈ R (Rn ) , (1.21)
Rm
and Z Z Z
φ (x, y) dVy dVx = φ (z) dV. (1.22)
Rn Rm Rn+m

Proof: To verify (1.20), note that φ (x, ·) is the step function

φ (x, y) = ∑ φ R×P XP′ (y) .

P∈Gm

Where x ∈ R′ and this is a finite sum of integrable functions because each has set of dis-
continuities with Jordan content 0. From the description in (1.19),
Z
φ (x, y) dVy = ∑ ∑ φ R×P XR′ (x) v (P)
Rm R∈Gn P∈Gm
!
= ∑ ∑ φ R×P v (P) XR′ (x) , (1.23)
R∈Gn P∈Gm

another step function. Therefore,

Z Z
φ (x, y) dVy dVx = ∑ ∑ φ R×P v (P) v (R)
Rn Rm R∈Gn P∈Gm
Z
= ∑ φ Q v (Q) = φ (z) dV. ■
Q∈G Rn+m

From (1.23),
Z  ( ! )
MR′ φ (·, y) dVy ≡ sup ∑ ∑ φ R×P v (P) XR′ (x) : x ∈ R′1
1 Rm R∈Gn P∈Gm

= ∑ φ R1 ×P v (P) (1.24)
P∈Gm
860 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

(·, y) dVy has the constant value given in (1.24) for x ∈ R′1 . Similarly,
R
because Rm φ
Z  ( ! )
mR′ φ (·, y) dVy ≡ inf ∑ ∑ φ R×P v (P) XR′ (x) : x ∈ R′1
1 Rm R∈Gn P∈Gm

= ∑ φ R1 ×P v (P) . (1.25)
P∈Gm

Theorem A.5.4 (Fubini) Let f ∈ R (Rn+m ) and suppose also that f (x, ·) ∈ R (Rm ) for
each x. Then Z
f (·, y) dVy ∈ R (Rn ) (1.26)
Rm
and Z Z Z
f (z) dV = f (x, y) dVy dVx . (1.27)
Rn+m Rn Rm

Proof: Let G be a grid such that UG ( f ) − LG ( f ) < ε and let Gn and Gm be as defined
above. Let

φ (z) ≡ ∑ MQ′ ( f ) XQ′ (z) , ψ (z) ≡ ∑ mQ′ ( f ) XQ′ (z) .

Q∈G Q∈G

Observe that MQ′ ( f ) ≤ MQ ( f ) and mQ′ ( f ) ≥ mQ ( f ). Then

Z Z
UG ( f ) ≥ φ dV, LG ( f ) ≤ ψ dV.

Also f (z) ∈ (ψ (z) , φ (z)) for all z. Thus from (1.24),

Z  Z 
MR′ f (·, y) dVy ≤ MR′ φ (·, y) dVy = ∑ MR′ ×P′ ( f ) v (P)
Rm Rm P∈Gm

and from (1.25),

Z  Z 
mR′ f (·, y) dVy ≥ mR′ ψ (·, y) dVy = ∑ mR′ ×P′ ( f ) v (P) .
Rm Rm P∈Gm

Therefore,
 Z  Z 
∑ MR′ f (·, y) dVy − mR′ f (·, y) dVy v (R) ≤
R∈Gn Rm Rm

∑ ∑ [MR′ ×P′ ( f ) − mR′ ×P′ ( f )] v (P) v (R) ≤ UG ( f ) − LG ( f ) < ε.

R∈Gn P∈Gm

f (·, y) dVy ∈ R (Rn ).

R
This shows, from Lemma A.4.12 and the Riemannn criterion, that Rm
It remains to verify (1.27). First note
Z
f (z) dV ∈ [LG ( f ) , UG ( f ) ] .
Rn+m

Next,
Z Z Z Z Z
LG ( f ) ≤ ψ dV = ψ dVy dVx ≤ f (x, y) dVy dVx
Rn+m Rn Rm Rn Rm
A.6. THE CHANGE OF VARIABLES FORMULA 861
Z Z Z
≤ φ (x, y) dVy dVx = φ dV ≤ UG ( f ) .
Rn Rm Rn+m
Therefore, Z Z Z
f (x, y) dVy dVx − f (z) dV ≤ ε
Rn Rm Rn+m

and since ε > 0 is arbitrary, this proves Fubini’s theorem2 . ■

Corollary A.5.5 Suppose E is a bounded contented set in Rn and let φ , ψ be continuous

functions defined on E such that φ (x) ≥ ψ (x). Also suppose f is a continuous bounded
function defined on the set

P ≡ {(x, y) : ψ (x) ≤ y ≤ φ (x)} ,

It follows f XP ∈ R R n+1


and
Z Z Z φ (x)
f dV = f (x, y) dy dVx .
P E ψ(x)

Proof: Since f is continuous, there is no problem in writing f (x, ·) X[ψ(x),φ (x)] (·) ∈
R R1 . Also, f XP ∈ R Rn+1 because P is contented thanks to Corollary A.4.9. There-

fore, by Fubini’s theorem

Z Z Z
f dV = f XP dy dVx
P Rn R
Z Z φ (x)
= f (x, y) dy dVx
E ψ(x)

proving the corollary. ■

Other versions of this corollary are immediate and should be obvious whenever en-
countered.

A.6 The Change Of Variables Formula

First recall Theorem 26.3.2 on Page 492 which is listed here for convenience.

Theorem A.6.1 Let h : U → Rn be a C1 function with h (0) = 0,Dh (0)−1 exists. Then
there exists an open set V ⊆ U containing 0 flips, F 1 , · · · , F n−1 , and primitive functions
Gn , Gn−1 , · · · , G1 such that for x ∈ V,

h (x) = F 1 ◦ · · · ◦ F n−1 ◦ Gn ◦ Gn−1 ◦ · · · ◦ G1 (x) .

Also recall Theorem 14.6.5 on Page 272.

Theorem A.6.2 Let φ : [a, b] → [c, d] be one to one and suppose φ ′ exists and is continuous
on [a, b]. Then if f is a continuous function defined on [a, b] ,
Z d Z b
f (s) ds = f (φ (t)) φ ′ (t) dt
c a
2 Actually, Fubini’s theorem usually refers to a much more profound result in the theory of Lebesgue integra-

tion.
862 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

The following is a simple corollary to this theorem.

Corollary A.6.3 Let φ : [a, b] → [c, d] be one to one and suppose φ ′ exists and is continuous
on [a, b]. Then if f is a continuous function defined on [a, b] ,
Z Z
X[a,b] φ −1 (x) f (x) dx = X[a,b] (t) f (φ (t)) φ ′ (t) dt


R R

Lemma A.6.4 Let h : V → Rn be a C1

function and suppose H is a compact subset of V .
Then there exists a constant C independent of x ∈ H such that
|Dh (x) v| ≤ C |v| .
Proof: Consider the compact set H × ∂ B (0, 1) ⊆ R2n . Let f : H × ∂ B (0, 1) → R be
given by f (x, v) = |Dh (x) v|. Then let C denote the maximum value of f . It follows that
for v ∈ Rn ,
v
Dh (x) ≤C
|v|
and so the desired formula follows when you multiply both sides by |v|. ■
Definition A.6.5 Let A be an open set. Write Ck (A; Rn ) to denote a Ck function whose

domain is A and whose range is in Rn . Let U be an open set in Rn . Then h ∈ Ck U; Rn
1 n

an open set V ⊇ U and a function g ∈ C (V ; R ) such that g = h on U.
if there exists
k
f ∈ C U means the same thing except that f has values in R. Also recall that x ∈ ∂U
means that every open set which contains x contains points of U and points of U C
Theorem
A.6.6 Let U be a bounded open set such that ∂U has zero content and let h ∈
C U; Rn be one to one and Dh (x)−1 exists for all x ∈ U. Then h (∂U) = ∂ (h (U)) and
∂ (h (U)) has zero content.
Proof: Let x ∈ ∂U and let g = h where g is a C1 function defined on an open set
containing U. By the inverse function theorem, g is locally one to one and an open mapping
near x.
Thus g (x) = h (x) and
is in an open set containing points of g (U) and points of
g U C . These points of g U C cannot equal any points of h (U) because g is one to one
locally. Thus h (x) ∈ ∂ (h (U)) and so h (∂U) ⊆ ∂ (h (U)). Now suppose y ∈ ∂ (h (U)).
By the inverse function theorem y cannot be in the open set h (U). Since y ∈ ∂ (h (U)),
every ball centered at y contains points of h (U) and so y ∈ h (U) \ h (U). Thus there
exists a sequence, {xn } ⊆ U such that h (xn ) → y. But then, by the continuity of h−1
which comes from the inverse function theorem, xn → h−1 (y) and so h−1 (y) ∈ / U but is
in U. Thus h−1 (y) ∈ ∂U. (Why?) Therefore, y ∈ h (∂U) , and this proves the two sets are
equal. It remains to verify the claim about content.
First let H denote a compact set whose interior contains U which is also in the interior
of the domain of g. Now since ∂U has content zero, it follows that for ε > 0 given, there
exists a grid G such that if G ′ are those boxes of G which have nonempty intersection with
∂U, then
∑ v (Q) < ε
Q∈G ′

and by refining the grid if necessary, no box of G has nonempty intersection with both U
and H C . Refining this grid still more, you can also assume that for all boxes in G ′ ,
li
<2
lj
A.6. THE CHANGE OF VARIABLES FORMULA 863

where li is the length of the ith side. (Thus the boxes are not too far from being cubes.)
Let C be the constant of Lemma A.6.4 applied to g on H.
Now consider one of these boxes, Q ∈ G ′ . If x, y ∈ Q, it follows from the chain rule
that Z 1
g (y) − g (x) = Dg (x+t (y − x)) (y − x) dt
0
By Lemma A.6.4 applied to H
Z 1
|g (y) − g (x)| ≤ |Dg (x+t (y − x)) (y − x)| dt
0
Z 1
≤ C |x − y| dt ≤ C diam (Q)
0
!1/2
n √
= C ∑ li2 ≤ C nL
i=1
√
Q. Thus diam (g (Q)) ≤ C nL and so g (Q) is
where L is the length of the longest side of √
contained in a cube having sides equal to C nL and volume equal to

Cn nn/2 Ln ≤ Cn nn/2 2n l1 l2 · · · ln = Cn nn/2 2n v (Q) .

Denoting by PQ this cube, it follows
h (∂U) ⊆ ∪Q∈G ′ v (PQ )
and
∑ ′ v (PQ ) ≤ Cn nn/2 2n ∑ ′ v (Q) < εCn nn/2 2n .
Q∈G Q∈G

Since ε > 0 is arbitrary, this shows h (∂U) has content zero as claimed. ■


Theorem A.6.7 Suppose f ∈ C U where U is a bounded open set with ∂U having content
0. Then f XU ∈ R (Rn ).
Proof: Let H be a compact set whose interior contains U which is also contained in the
domain of g where g is a continuous functions whose restriction to U equals f . Consider
gXU , a function whose set of discontinuities has content 0. Then gXU = f XU ∈ R (Rn ) as
claimed. This is by the big theorem which tells which functions are Riemannn integrable.
■
The symbol U − p is defined as {x − p : x ∈ U}. It merely slides U by the vector p.
The following lemma is obvious from the definition of the integral.
Lemma A.6.8 Let U be a bounded open set and let f XU ∈ R (Rn ). Then
Z Z
f (x + p) XU−p (x) dx = f (x) XU (x) dx

A few more lemmas are needed.

Lemma A.6.9 Let S be a nonempty subset of Rn . Define
f (x) ≡ dist (x, S) ≡ inf {|x − y| : y ∈ S} .
Then f is continuous.
864 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

Proof: Consider | f (x) − f (x1 )|and suppose without loss of generality that f (x1 ) ≥
f (x). Then choose y ∈ S such that f (x) + ε > |x − y|. Then

| f (x1 ) − f (x)| = f (x1 ) − f (x) ≤ f (x1 ) − |x − y| + ε

≤ |x1 − y| − |x − y| + ε
≤ |x − x1 | + |x − y| − |x − y| + ε
= |x − x1 | + ε.

Since ε is arbitrary, it follows that | f (x1 ) − f (x)| ≤ |x − x1 | and this proves the lemma.
■

Theorem A.6.10 (Urysohn’s lemma for Rn ) Let H be a closed subset of an open set U.
Then there exists a continuous function g : Rn → [0, 1] such that g (x) = 1 for all x ∈ H
and g (x) = 0 for all x ∈
/ U.

Proof: If x ∈/ C, a closed set, then dist (x,C) > 0 because there exists δ > 0 such that
B (x,δ ) ∩ C = 0.
/ This is
because, since C is closed, its complement is open. Therefore,
dist (x, H) + dist x,U C > 0 for all x ∈ Rn . Now define a continuous function g as

dist x,U C


g (x) ≡ .
dist (x, H) + dist (x,U C )

It is easy to see this verifies the conclusions of the theorem and this proves the theorem. ■

Definition A.6.11 Define spt( f ) (support of f ) to be the closure of the set {x : f (x) ̸= 0}.
If V is an open set, Cc (V ) will be the set of continuous functions f , defined on Rn having
spt( f ) ⊆ V .

Definition A.6.12 If K is a compact subset of an open set V , then K ≺ φ ≺ V if

φ ∈ Cc (V ), φ (K) = {1}, φ (Rn ) ⊆ [0, 1].

Also for φ ∈ Cc (Rn ), K ≺ φ if

φ (Rn ) ⊆ [0, 1] and φ (K) = 1.

and φ ≺ V if
φ (Rn ) ⊆ [0, 1] and spt(φ ) ⊆ V.

Theorem A.6.13 (Partition of unity) Let K be a compact subset of Rn and suppose

K ⊆ V = ∪ni=1Vi , Vi open and bounded.

Then there exist ψ i ≺ Vi with

n
∑ ψ i (x) = 1
i=1

for all x ∈ K.
A.6. THE CHANGE OF VARIABLES FORMULA 865

Proof: Let K1 = K \∪ni=2Vi . Thus K1 is compact because it is the intersection of a closed

set with a compact set and K1 ⊆ V1 . Let K1 ⊆ W1 ⊆ W 1 ⊆ V1 with W 1 compact. To obtain
W1 , use Theorem A.6.10 to get f such that K1 ≺ f ≺ V1 and let W1 ≡ {x : f (x) ̸= 0} . Thus
W1 ,V2 , · · ·Vn covers K and W 1 ⊆ V1 . Let K2 = K \ (∪ni=3Vi ∪ W1 ). Then K2 is compact
and K2 ⊆ V2 . Let K2 ⊆ W2 ⊆ W 2 ⊆ V2 W 2 compact. Continue this way finally obtaining
W1 , · · · ,Wn , K ⊆ W1 ∪ · · · ∪Wn , and W i ⊆ Vi ;W i compact. Now let W i ⊆ Ui ⊆ U i ⊆ Vi ,U i
compact.

Wi Ui Vi

By Theorem A.6.10, there exist functions φ i , γ such that U i ≺ φ i ≺ Vi , ∪ni=1W i ≺ γ ≺

∪ni=1Ui .
Define
(
γ(x)φ i (x)/ ∑nj=1 φ j (x) if ∑nj=1 φ j (x) ̸= 0,
ψ i (x) =
0 if ∑nj=1 φ j (x) = 0.

If x is such that ∑nj=1 φ j (x) = 0, then x ∈

/ ∪ni=1U i . Consequently γ(y) = 0 for all y near x
and so ψ i (y) = 0 for all y near x. Hence ψ i is continuous at such x. If ∑nj=1 φ j (x) ̸= 0, this
situation persists near x and so ψ i is continuous at such points. Therefore ψ i is continuous.
If x ∈ K, then γ(x) = 1 and so ∑nj=1 ψ j (x) = 1. Clearly 0 ≤ ψ i (x) ≤ 1 and spt(ψ j ) ⊆ V j .
■
The next lemma contains the main ideas. See [35] and [31] for similar proofs.

Lemma A.6.14 Let U be a bounded open set with ∂U having content 0. Also let
h ∈
C1 U; Rn be one to one on U with Dh (x)−1 exists for all x ∈ U. Let f ∈ C U be

nonnegative. Then
Z Z
Xh(U) (z) f (z) dVn = XU (x) f (h (x)) |det Dh (x)| dVn

Proof: Let ε > 0 be given. Then by Theorem A.6.7,

x → XU (x) f (h (x)) |det Dh (x)|

is Riemannn integrable. Therefore, there exists a grid G such that, letting

g (x) = XU (x) f (h (x)) |det Dh (x)| ,

LG (g) + ε > UG (g) .

Let K denote the union of the boxes Q of G which intersect U. Thus K is a compact subset
of V where V is a bounded open set containing U, and it is only the terms from these
boxes which contribute anything nonzero to the lower sum. By Theorem 26.3.2 on Page
492 which is stated above and the inverse function theorem, it follows that for p ∈ K, there
exists an open set contained in U which contains p, denoted as Op such that for x ∈ Op −p,

h (x + p) − h (p) = F 1 ◦ · · · ◦ F n−1 ◦ Gn ◦ · · · ◦ G1 (x)

866 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

where the Gi are primitive functions,and the F j are flips. Also h (O j ) is an open set.
q
Finitely many of these open sets O j j=1 cover K. Let the distinguished point for O j
be denoted by p j . Now refine G if necessary, such that the diameter of every cell of the
new G which intersects U is smaller than a Lebesgue number for this open cover. Denote
by G ′ those boxes of the new G which intersect U. Thus the union of these boxes of G ′
of G o
equals the set K and every box n ′ is contained in one of these O . By Theorem A.6.13,
j
there exists a partition of unity ψ j on h (K) such that ψ j ≺ h (O j ). Then
Z
LG (g) ≤ ∑′ XQ (x) f (h (x)) |det Dh (x)| dx
Q∈G
q Z  
= ∑ ∑ XQ (x) ψ j f (h (x)) |det Dh (x)| dx. (1.28)
Q∈G ′ j=1
 
Consider the term XQ (x) ψ j f (h (x)) |det Dh (x)| dx. By Lemma A.6.8 and Fubini’s
R

theorem this equals

Z Z  
XQ−p j (x) ψ j f (h (pi ) + F 1 ◦ · · · ◦ F n−1 ◦ Gn ◦ · · · ◦ G1 (x)) ·
Rn−1 R
|DF (Gn ◦ · · · ◦ G1 (x))| |DGn (Gn−1 ◦ · · · ◦ G1 (x))| ·
|DGn−1 (Gn−2 ◦ · · · ◦ G1 (x))| (1.29)
· · · |DG2 (G1 (x))| |DG1 (x)| dx1 dVn−1 . (1.30)

The vertical lines in the above signify the absolute value of the determinant of the matrix
on the inside. Here dVn−1 is with respect to the variables x2 , · · · , xn . Also F denotes
F 1 ◦ · · · ◦ F n−1 . Now
G1 (x) = (α (x) , x2 , · · · , xn )T
and is one to one. Therefore, fixing x2 , · · · , xn , x1 → α (x) is one to one. Also

|DG1 (x)| = |α x1 (x)|

Fixing x2 , · · · , xn , change the variable,

y1 = α (x1 , x2 , · · · , xn ) , dy1 = α x1 (x1 , x2 , · · · , xn ) dx1

Thus
x = (x1 , x2 , · · · , xn )T = G−1 −1
x′


1 (y1 , x2 , · · · , xn ) ≡ G1
Then in (1.30) you can use Corollary A.6.3 to write (1.30) as
Z Z

 
XQ−p j G−11 x′ ψjf
Rn−1 R
h (pi ) + F 1 ◦ · · · ◦ F n−1 ◦ Gn ◦ · · · ◦ G1 G−1 x′




1

· DF Gn ◦ · · · ◦ G1 G−1 x′ DGn Gn−1 ◦ · · · ◦ G1 G−1 x′

1 1 ·
DGn−1 Gn−2 ◦ · · · ◦ G1 G−1 x′ · · · DG2 G1 G−1 x′







1 1 dy1 dVn−1
which reduces to
Z

 
XQ−p j G−1 x′ ψ j f h (pi ) + F 1 ◦ · · · ◦ F n−1 ◦ Gn ◦ · · · ◦ G2 x′



1
Rn
A.6. THE CHANGE OF VARIABLES FORMULA 867

· DF Gn ◦ · · · ◦ G2 x′ DGn Gn−1 ◦ · · · ◦ G2 x′





·
Gn−2 ◦ · · · ◦ G2 x′ · · · DG2 x′ dVn .




DGn−1
Now use Fubini’s theorem again to make the inside integral taken with respect to x2 . Note
that the term |DG1 (x)| disappeared. Exactly the same process yields
Z Z

 
XQ−p j G−1 1 ◦ G −1
2 x ′′
ψ j f
Rn−1 R
h (pi ) + F 1 ◦ · · · ◦ F n−1 ◦ Gn ◦ · · · ◦ G3 x′′

· DF Gn ◦ · · · ◦ G3 x′′ DGn Gn−1 ◦ · · · ◦ G3 x′′

·
Gn−2 ◦ · · · ◦ G3 x′′ · · · dy2 dVn−1 .



DGn−1
Now F is just a composition of flips, so |DF (Gn ◦ · · · ◦ G3 (x′′ ))| = 1, and so this term can
be replaced with 1. Continuing this process, eventually yields an expression of the form
Z
 
XQ−p j G−1 1 ◦ · · · ◦ G −1
n−2 ◦ G −1
n−1 ◦ G −1
n ◦ F −1
(y) ψ j f (h (pi ) + y) dVn . (1.31)
Rn

Denoting by G−1 the expression, G−1 −1 −1 −1

1 ◦ · · · ◦ Gn−2 ◦ Gn−1 ◦ Gn ,

XQ−p j G−1 −1 −1 −1 −1


1 ◦ · · · ◦ Gn−2 ◦ Gn−1 ◦ Gn ◦ F (y) = 1

exactly when G−1 ◦ F −1 (y) ∈ Q − p j . Now recall that

h p j + x − h p j = F ◦ G (x)
and so the above holds exactly when
= h p j + G−1 ◦ F −1 (y) − h p j ∈ h p j + Q − p j − h p j





y


= h (Q) − h p j .
Thus (1.31) reduces to
Z  
Xh(Q)−h(p ) (y) ψ j f (h (pi ) + y) dVn
n j
ZR  
= Xh(Q) (z) ψ j f (z) dVn .
Rn

It follows from (1.28),

Z
UG (g) − ε ≤ LG (g) ≤ XU (x) f (h (x)) |det Dh (x)| dVn
Z
≤ ∑ XQ (x) f (h (x)) |det Dh (x)| dx
Q∈G ′
q Z  
= ∑′ ∑ XQ (x) ψ j f (h (x)) |det Dh (x)| dx
Q∈G j=1
q Z  
= ∑′ ∑ Xh(Q) (z) ψ j f (z) dVn
n
Q∈G j=1 R
Z Z
= ∑′ Xh(Q) (z) f (z) dVn = Xh(U) (z) f (z) dVn
Q∈G Rn
868 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗

which implies the inequality,

Z Z
XU (x) f (h (x)) |det Dh (x)| dVn ≤ Xh(U) (z) f (z) dVn

But now you can use the same information just derived to obtain equality.

x = h−1 (z)

and so from what was just done,

Z
XU (x) f (h (x)) |det Dh (x)| dVn

Z
= Xh−1 (h(U)) (x) f (h (x)) |det Dh (x)| dVn
Z
Xh(U) (z) f (z) det Dh h−1 (z) det Dh−1 (z) dVn


≥
Z
= Xh(U) (z) f (z) dVn

from the chain rule. In fact,

I = Dh h−1 (z) Dh−1 (z) ,

so

1 = det Dh h−1 (z) det Dh−1 (z) .

■

The change of variables theorem follows.

Theorem
A.6.15 Let U be a bounded open set with ∂U having content 0. Also
let h ∈
C1 U; Rn be one to one on U and Dh (x)−1 exists for all x ∈ U. Let f ∈ C U . Then
Z Z
Xh(U) (z) f (z) dz = XU (x) f (h (x)) |det Dh (x)| dx

| f |+ f | f |− f
Proof: You note that the formula holds for f + ≡ 2 and f − ≡ 2 . Now f =
f+− f− and so Z
Xh(U) (z) f (z) dz

Z Z
= Xh(U) (z) f + (z) dz − Xh(U) (z) f − (z) dz
Z Z
= XU (x) f + (h (x)) |det Dh (x)| dx − XU (x) f − (h (x)) |det Dh (x)| dx
Z
= XU (x) f (h (x)) |det Dh (x)| dx. ■
A.7. SOME OBSERVATIONS 869

A.7 Some Observations

Some of the above material is very technical. This is because it gives complete answers
to the fundamental questions on existence of the integral and related theoretical considera-
tions. However, most of the difficulties are artifacts. They should not even be considered!
It was realized early in the twentieth century that these difficulties occur because, from
the point of view of mathematics, this is not the right way to define an integral! Better
results are obtained much more easily using the Lebesgue integral. Many of the tech-
nicalities related to Jordan content disappear almost magically when the right integral is
used. However, the Lebesgue integral is more abstract than the Riemannn integral and it
is not traditional to consider it in a beginning calculus course. If you are interested in the
fundamental properties of the integral and the theory behind it, you should abandon the
Riemannn integral which is an antiquated relic and begin to study the integral of the last
century. An introduction to it is in [31]. Another very good source is [16]. This advanced
calculus text does everything in terms of the Lebesgue integral and never bothers to strug-
gle with the inferior Riemannn integral. A more general treatment is found in [26], [27],
[32], and [28]. There is also a still more general integral called the generalized Riemannn
integral. A recent book on this subject is [5]. It is far easier to define than the Lebesgue
integral but the convergence theorems are much harder to prove. An introduction is also in
[27].
870 APPENDIX A. THE THEORY OF THE RIEMANNN INTEGRAL∗
Appendix B

A Rigid Body Rotating About a

Point

Imagine a rigid body which is rotating about a point fixed in space. For example, you could
consider a bicycle wheel rotating about its axis which is held still. More generally, we let
the point about which the body rotates move also. In this case, the point is usually the
center of mass of the body. However, in this section, this point will be regarded as fixed.
Let B (t) denote the set of points in three dimensional space which the body occupies at
time t. We will refer to the points in three dimensional space occupied by the body at time
t = 0 as the material points of the body.
Recall Theorem 24.3.2 about the existence of the angular velocity vector. The idea is
that you have a material point x0 in the body and some right handed orthonormal system of
basis vectors {e1 (t) , e2 (t) , e3 (t)} which moves with the body such if x (t, x) is the vector
from x0 to the point where x is at time t, then x (t, x) = ae1 (t) + be2 (t) + ce3 (t) where
a, b, c are constants. Note that here it is assumed that x0 does not change. Thus it is not
moving through space. Then this theorem is summarized in the following lemma.
Lemma B.0.1 For a body which undergoes rigid body motion about a fixed point in three
dimensional space, if x (t, x) denotes the position vector of the point x at time t, from some
fixed point in the body, then there exists a time dependent vector ω (t) such that the velocity
of this point at time t, xt (t, x) is given by
xt (t, x) = ω (t) × x (t, x) .
In particular, letting x = ei , we see that e′i (t) = ω (t) × ei (t) .
Definition B.0.2 The vector, ω (t) whose existence is given by the above lemma is called
the angular velocity vector.
We are now ready to write the total angular momentum of the rigid body. In doing so,
we assume the density equals ρ (x) . Thus at time t the total angular momentum, Ω, would
be given by the three dimensional integral,
Z
Ω = x (t, x) × ρ (x) xt (t, x) dx
B(0)
Z
= ρ (x) x (t, x) × (ω (t) × x (t, x)) dx. (2.1)
B(0)

871
872 APPENDIX B. A RIGID BODY ROTATING ABOUT A POINT

In terms of the material basis, {e1 (t) , e2 (t) , e3 (t)} which is fixed with the body,

e1 (t) e2 (t) e3 (t)

(ω (t) × x (t, x)) = ω 1 (t) ω 2 (t) ω 3 (t)
x1 x2 x3

where the ω i are the components of ω taken with respect to {e1 (t) , e2 (t) , e3 (t)} and as
we observed earlier, {x1 , x2 , x3 } are the coordinates of the vector x (t, x) taken with respect
to the {e1 (t) , e2 (t) , e3 (t)} . To simplify the integrand in 2.1 that long cross product is
simplified.

Lemma B.0.3 Let a, b, c be three dimensional vectors. Then

a× (b × c) = (a · c) b− (a · b) c.

Proof: Let an orthonormal right handed coordinate system {e1 , e2 , e3 } be given. Then

a× (b × c) = ε i jk a j (b × c)k ei

= ε i jk ε kpq a j b p cq ei
= ε ki j ε kpq a j b p cq ei
= (δ ip δ jq − δ j p δ iq ) a j b p cq ei
= (a j bi c j − a j b j ci ) ei
= (a · c) b− (a · b) c. ■
Now simplify the integrand using this lemma.

x (t, x) × (ω (t) × x (t, x))

= (x (t, x) · x (t, x)) ω (t) − (x (t, x) · ω (t)) x (t, x) .

Writing x (t, x) and ω (t) in terms of the material coordinates,

ω (t) = ω 1 e1 (t) + ω 2 e2 (t) + ω 3 e3 (t) ,

x (t, x) = x1 e1 (t) + x2 e2 (t) + x3 e3 (t) ,

and so
x (t, x) × (ω (t) × x (t, x)) =
!
2
∑ |x| ω i ei (t) − ∑ ∑ x j ω j xi ei (t) . (2.2)
i i j

Thus, listing the components of x (t, x) × (ω (t) × x (t, x)) with respect to the material
basis yields the following in which x (t, x)×(ω (t) × x (t, x)) is written as a column vector.


x12 + x22 + x32 ω 1 − x12 ω 1 + x2 x1 ω 2 + x3 x1 ω 3


 2
 x1 + x22 + x32 ω 2 − x2 x1 ω 1 + x22 ω 2 + x3 x2 ω 3 




x12 + x22 + x32 ω 3 − x3 x1 ω 1 + x3 x2 ω 2 + x32 ω 3

 873

Written as a matrix, this is

  
x22 + x32 −x1 x2 −x1 x3 ω 1 (t)
 −x1 x2 x12 + x32 −x2 x3   ω 2 (t)  . (2.3)
  

−x1 x3 −x2 x3 x12 + x22 ω 3 (t)

Therefore, the components of angular momentum taken with respect to the material basis
are     
Ω1 (t) I11 I12 I13 ω 1 (t)
 Ω2 (t)  =  I21 I22 I23   ω 2 (t)  . (2.4)
    

Ω3 (t) I31 I32 I33 ω 3 (t)

Where
Z
!
Ikk = ∑ x2j ρ (x1 , x2 , x3 ) dx
B(0) j̸=k
Z
Ii j = − xi x j ρ (x1 , x2 , x3 ) dx, i ̸= j.
B(0)

Thus the matrix in 2.4 is symmetric. Because of the choice of coordinates, this matrix is
also time independent. It is called the moment of inertia tensor and the off diagonal terms
are called the products of inertia. Now recall that
Z
Ω= x (t, x) × ρ (x) xt (t, x) dx.
B(0)

Taking the time derivative on both sides, (We do not worry about mathematical details
related to differentiating under the integral sign here.)
Z
Ω′ = xt (t, x) × ρ (x) xt (t, x) dx
B(0)
d
Z
+ x (t, x) × (ρ (x) xt (t, x)) dx
B(0) dt
d
Z
= x (t, x) × (ρ (x) xt (t, x)) dx.
B(0) dt

Now from Newton’s second law, the force on the chunk of mass, ρ (x) dx at time t, denoted
here by F (x (t, x)) dx is just dtd (ρ (x) xt (t, x)) dx. Therefore,
Z
Γ (t) ≡ Ω′ (t) = x (t, x) × F (x (t, x)) dx
B(0)

which is the total torque acting on the body at time t. Note it has units of distance times
units of force. Now differentiate the angular momentum to find the torque, this in terms of
the moment of inertia tensor of 2.4 Ω = Iω. There is a slight complication due to the fact
that we have the angular momentum expressed in terms of a basis which is time dependent.
Therefore, when we take the derivative of this vector we must include this fact. From 2.4
we see
Ω (t) = ∑ ∑ Ii j ω j (t) ei (t) .
i j
874 APPENDIX B. A RIGID BODY ROTATING ABOUT A POINT

Now recall by Lemma B.0.1, e′i (t) = ω (t) × ei (t) . Therefore,

Γ (t) = Ω′ (t) =

∑ ∑ Ii j ω ′j (t) ei (t) + ∑ ∑ Ii j ω j (t) (ω (t) × ei (t)) . (2.5)

i j i j

This is called Euler’s equation for the torque. There are three equations hidden in the above
formula, one for each ei for i = 1, 2, and 3. If you want, you can write them down but there
is a simpler way to proceed. Recall the matrix, (Ii j ) is symmetric and real. Therefore, it
can be diagonalized by a unitary real matrix. See Theorem 11.4.7. If we let the columns of
this unitary matrix be the ei , it follows the moment of inertia tensor is a diagonal matrix,
diag (I1 , I2 , I3 ) and 2.5 becomes

Γ (t) = ∑ Ii ω ′i (t) ei (t) + ∑ Ii ω i (t) (ω (t) × ei (t))

i i

Writing the right side out, Γ (t) =

ω ×e1
 

I1 ω ′1 e1 + I2 ω ′2 e2 + I3 ω ′3 e3 + I1 ω 1 ω 3 e2 − ω 2 e3  +
z }| {

ω ×e2 ω ×e3
   
z }| { z }| {
I2 ω 2 ω 1 e3 − ω 3 e1  + I3 ω 3 ω 2 e1 − ω 1 e2 

and now, collecting terms, Γ (t) = Γ1 (t) e1 (t) + Γ2 (t) e2 (t) + Γ3 (t) e3 (t) where

Γ1 (t) = I1 ω ′1 + ω 3 ω 2 (I3 − I2 )
Γ2 (t) = I2 ω ′2 + ω 1 ω 3 (I1 − I3 )
Γ3 (t) = I3 ω ′3 + ω 1 ω 2 (I2 − I1 ) . (2.6)

These are called Euler’s equations for the torque. Although I invoked the theorem that
Hermitian or symmetric matrices can be diagonalized by a unitary transformation in order
to get axes with respect to which the moment of inertia tensor is diagonal, it is usually much
easier than this. Often there are symmetry considerations which make it obvious how to
choose these axes and when this is done 2.6 allows us to compute the torque which results
from a given angular velocity.

Example B.0.4 Consider a disc having negligible thickness and radius R with constant
density ρ taken with respect to area which spins around its center. How should we choose
the material bases to get a nice diagonal moment of inertia tensor?

Consider the following picture in which the vectors e1 (t) and e2 (t) are shown fixed
with the disc which is assumed to be rotating.
 875

e1 (t)

e2 (t)

We let e3 (t) = e1 (t) × e2 (t) so that we have a right handed orthonormal system of
basis vectors. We calculate the moment of inertia tensor first.
Z 2π Z R
1
Z
I11 ≡ ρ x22 dx = ρ (r sin θ )2 rdrdθ = R4 πρ
B(0) 0 0 4

By symmetry, we see that I22 = 14 R4 πρ also. Now

Z Z 2π Z R 1
x22 + x12 dx = ρ r3 drdθ = ρπR4 .


I33 ≡ ρ
B(0) 0 0 2

Now by symmetry considerations, I12 = 0 as are all the other off diagonal terms. Those that
have a 3 in the subscript are zero because we are assuming for the sake of simplicity that
the disc has negligible thickness. However, if we didn’t assume this we would still get zero
for these terms by the symmetry of the shape with respect to the other variable. Therefore,
the moment of inertia tensor is
 
1 4
4 ρπR 0 0
1 4
0 0 .
 
 4 ρπR
1 4
0 0 2 ρπR

It follows that for ω = ω 1 (t) e1 (t) + ω 2 (t) e2 (t) + ω 3 (t) e3 (t) we can find the Torque by
Euler’s equations.
 
1 1
Γ1 (t) = ρπR4 ω ′1 + ω 3 ω 2 ρπR4
4 4
 
1 1
Γ2 (t) = ρπR4 ω ′2 + ω 1 ω 3 − ρπR4
4 4
′
Γ3 (t) = I3 ω 3 . (2.7)

The physical interpretation of ω given above is that the term ω 3 (t) e3 (t) represents
the angular velocity about the axis determined by e3 (t) . Thus it is a measure of how fast
876 APPENDIX B. A RIGID BODY ROTATING ABOUT A POINT

and in what direction the disc is spinning about this axis. If the disc were spinning very
fast we would have ω 3 (t) very large. The other terms of angular velocity, ω 1 (t) e1 (t) +
ω 2 (t) e2 (t) , yield a vector which is in the plane determined by e1 (t) and e2 (t) and so it is a
measure of the angular velocity about this axis. If we assumed ω ′i (t) = 0 for each i = 1, 2, 3,
and ω 2 and ω 1 are moderate, note that we would still have substantial components of
torque, Γ2 (t) and Γ1 (t). Much more could be said about this problem and more examples
could be given but this much will suffice here.
Appendix C

Lagrangian Mechanics

Let y = y (x,t) where t signifies time and x ∈ U ⊆ Rm for U an open set, while y ∈ Rn
and suppose x is a function of t. Physically, this corresponds to an object moving over
a surface in Rn , its position being y (x,t). If we know about x (t) then we also know y.
More generally, we might have M masses, the position of mass α being y α . For example,
consider the pendulum in which there is only one mass.
in which n = 2, l is fixed and y1 = l sin θ , y2 = l − l cos θ . Thus, in this
simple example, m = 1 and x = θ . If l were changing in a known way
with respect to t, then this would be of the form y = y (x,t). We seek
l differential equations for x.
θ The kinetic energy is defined as
•m
1
T ≡ ∑ mα ẏ α ·ẏ α (∗)
2 α
where the dot on the top signifies differentiation with respect to t. Thus, from the chain
rule, T is a function of ẋ. The following lemma is an important observation.
Lemma C.0.1 The following formula holds.
∂T ∂ yα
= mα ẏ α · .
∂ ẋk ∑
α ∂ xk
Proof: From the chain rule,
∂ yα k ∂ yα
ẏ α = ∑ ẋ + (∗∗)
k ∂ xk ∂t
and so
∂ ẏ α ∂ yα
k
= .
∂ ẋ ∂ xk
Therefore,
∂T ∂ ẏ α ∂ yα
k
= ∑ mα ẏ α · k
= ∑ mα ẏ α · ■
∂ ẋ α ∂ ẋ α ∂ xk
It follows from the above and the product and chain rule that
 
d ∂T ∂ yα
= ∑ mα ÿ α · +
dt ∂ ẋk α ∂ xk

877
878 APPENDIX C. LAGRANGIAN MECHANICS

∂ 2y ∂ 2y
∑ mα ẏα · ∑ ∂ xr ∂ αxk ẋr + ∑ mα ẏα · ∂t∂ xαk . (∗ ∗ ∗)
α r α

Also from the product rule,

 
∂T ∂ ẏ α
= mα ẏ α ·
∂ xk ∑
α ∂ xk

But from ∗∗,

∂ ẏ α ∂ 2 yα r ∂ 2 yα
= ∑ ẋ + k
∂ xk k
r ∂x ∂x
r ∂ x ∂t
Thus
∂ 2 yα r ∂ 2 yα
 
∂T
∂ xk
= ∑ mα ẏα ·
∑ ∂ xk ∂ xr ẋ +
∂ xk ∂t
α r
∂ 2y ∂ 2y
 
= ∑ ∑ mα ẏ α · k α r ẋr + mα ẏ α · k α
α r ∂x ∂x ∂ x ∂t

From this and ∗ ∗ ∗,  

d ∂T ∂T ∂ yα
− = mα ÿ α ·
dt ∂ ẋk ∂ xk ∑
α ∂ xk
Now ÿ α denotes the acceleration of the α th mass and so, by Newton’s second law, if F is
the force acting on the object,
 
d ∂T ∂T ∂ yα
− k = ∑F α · (3.1)
dt ∂ ẋk ∂x α ∂ xk

This is a particularly agreeable formula in case F α = ∇Φα (y) + g α where g α is a force of

constraint which causes motion to remain in the surface x → y α (x) . Thus g α · ∂∂yxkα = 0.
In this special case, you have
 
d ∂T ∂T ∂ yα ∂
− k = ∑ ∇Φα (y) · = ∑ k (Φα (y))
dt ∂ ẋk ∂x α ∂ x k
α ∂ x

Let Φ denote the total potential energy so Φ = ∑α Φα .Now Φα (y) does not depend on ẋ,
only on x. Hence ∂ Φ∂αẋ(y) = 0. It follows that in this special case,
k
 
d ∂ (T − Φ) ∂ (T − Φ)
− = 0, (3.2)
dt ∂ ẋk ∂ xk

this for each k. This formula is due to Lagrange.1

1 Joseph Louis Lagrange (1736-1813) was born in Italy but lived much of his life in France which is where he

died. He made major contributions to analysis, number theory, and mechanics. His most famous work is likely
Mécanique analytique. He invented the method of variation of parameters used earlier. With Euler, he invented
the calculus of variations and also the method of Lagrange multipliers in order to include constraints. Lagrange
was also involved in the development of the metric system.
 879

Theorem C.0.2 Let y α (x,t) denote the position of an object of mass mα where x is a
function of t. Let the kinetic energy be defined by
1
T≡ mα ẏ α ·ẏ α .
2∑α

Let the mass mα be acted on by a force F α . Then Newton’s second law implies
 
d ∂T ∂T ∂ yα
− k = ∑F α · (3.3)
dt ∂ ẋk ∂x α ∂ xk

In case F α = ∇Φα + g α where g α is a force of constraint so the total force comes from
forces of constraint and the gradient of a potential function, then
 
d ∂ (T − Φ) ∂ (T − Φ)
− =0
dt ∂ ẋk ∂ xk
Also, the above 3.3 implies Newton’s second law.

Proof: The above derivation shows that Newton’s law implies the above two formulas.
On the other hand, if 3.3 holds, then in the case of one mass, the first part of the derivation
which depended only on the chain rule and product rule shows
 
d ∂T ∂T ∂y
− k = m ÿ· k
dt ∂ ẋk ∂x ∂x

Thus if 3.3 and there is no force of constraint, then F = mÿ which is Newton’s second law.
■

Example C.0.3 In the case of the simple pendulum, x = θ as shown in the picture and
! !
y1 l sin θ
=
y2 l − l cos θ

the force acting on weight being mg (−j) = ∇ −mgy2 . Find the equation of motion of

this pendulum.

! !
1 l cos (θ ) θ ′ l cos (θ ) θ ′ 1
2
T= m · = ml 2 θ ′
2 l sin (θ ) θ ′ l sin (θ ) θ ′ 2

2
Then Φ = −mg (l − l cos θ ) . T − Φ = 12 ml 2 θ ′ + mg (l − l cos θ ). Thus the equation of
motion of this pendulum is
d
ml 2 θ ′ − mgl (− sin (θ )) = 0


dt
so
g
θ ′′ + sin θ = 0
l
This is an equation which doesn’t have a simple analytic solution in terms of standard
calculus type functions.
880 APPENDIX C. LAGRANGIAN MECHANICS

Example C.0.4 In the above simple pendulum, suppose there is a friction force −k (y) ẏ
acting to impede the motion. What are equations of motion in this case?
The following is from the chain rule.
!
l cos θ
ẏ = θ′
l sin θ

Denote k (l sin θ , l − l cos θ ) as k (θ ) to save notation. Then it follows from 3.3 and the
previous example that
  
d ∂ 1 2 ′
2
ml θ + mg (l − l cos θ ) −
dt ∂ θ ′ 2
  ! !
∂ 1 2 ′
2 l cos θ l cos θ
ml θ + mg (l − l cos θ ) = −k (θ ) θ ′ ·
∂θ 2 l sin θ l sin θ
and so
d
ml 2 θ ′ + mgl sin (θ ) = −k (θ ) θ ′ l 2


dt
k (θ ) ′ g
θ ′′ + θ + sin (θ ) = 0
m l
This is another equation for which we don’t have a good way to obtain a simple analytic
solution.

C.1 The Spinning Top and the Euler Angles

This material is discussed in [8]. It is due to Lagrange. He was doing this kind of thing
in the second half of the 1700’s. However, he didn’t use pictures to illustrate what he was
doing. Here we consider the techniques for considering the motion of rigid bodies rotating
about a fixed point in space using Lagrangian mechanics. Earlier, it was done in a way
which made it convenient to find the torque given the angular velocity. Here we consider
a rigid body as a very large number of point masses which satisfy equations of constraint
which cause them to remain at a constant distance from each other. It follows we can
consider the motion using only three parameters. The ones we use are called the Euler
angles. To describe the Euler angles consider the following picture in which x1 , x2 and x3
are the usual coordinate axes fixed in space and the axes labeled with a superscript denote
other coordinate axes. Here is the picture.

x3 = x31 x31
x32
θ
x21 x22
φ θ
x2 x21
φ
x1
x11 x11 = x12
C.1. THE SPINNING TOP AND THE EULER ANGLES 881

x32 = x33

x23
ψ
x22
ψ
x12
x13

We obtain φ by rotating about the fixed x3 axis. Next we rotate about the x11 axis which
results from the first rotation. This gives θ . Finally, we rotate about the x32 axis by ψ.
This can realize any rotation about the origin in this manner. In practice one knows θ ′ , φ ′
and ψ ′ and you want to find a formula for the kinetic energy in terms of these quantities
because this will allow you to write a Lagrangian and obtain the equations of motion.
A little thought will show that a choice of these angles determines another right handed
orthogonal coordinate system, x13 , x23 , and x33 and that every such system is determined by a
suitable choice of the Euler angles. In the context of Lagrangian mechanics above, define
Gα (φ , θ , ψ) to be the point in space whose coordinates in x13 , x23 , and x33 are the same as the
coordinates of this point in x1 , x2 , and x3 and since the body is rigid, the constraints require
that G (φ , θ , ψ) ≡ (G1 (φ , θ , ψ) , · · · , GN (φ , θ , ψ)). Now recall Lemma B.0.1 listed here
for convenience.

Lemma C.1.1 For a body which undergoes rigid body motion about a fixed point in three
dimensional space, if we let x (t, x) denote the position vector of the point, x at time t,then
there exists a time dependent vector ω (t) such that the velocity of this point at time t,
xt (t, x) is given by
xt (t, x) = ω (t) × x (t, x) .
In particular, letting x = ei , we see that e′i (t) = ω (t) × ei (t) .

It follows from this lemma that the total kinetic energy of the rigid body is
1 1
Z Z
ρ (x) |xt (t, x)|2 dx = ρ (x) |ω (t) × x (t, x)|2 dx.
2 B(0) 2 B(0)

As discussed above, when the Euler angles change, this results in new coordinate axes that
come from rotating the original axes. If we let these new axes be fixed with the moving
body and call the new axes, x1 (t) , x2 (t) , and x3 (t) with ei (t) a unit vector in the positive
xi (t) direction, it follows the coordinates of x (t, x) with respect to these new axes are the
same as the coordinates of x with respect to x1 (0) , x2 (0) , and x3 (0) ,the axes at time t = 0.
We can compute |ω (t) × x (t, x)|2 as follows.

|ω (t) × x (t, x)|2 = ε i jk ω j (t) xk ε ipq ω p (t) xq

= δ j p δ kq − δ jq δ kp ω j (t) xk ω p (t) xq

= ω j (t) xk ω j (t) xk − ω j (t) xk ω k (t) x j

= |x|2 |ω|2 − (x · ω)2 .
882 APPENDIX C. LAGRANGIAN MECHANICS

Here ω i are the components of ω taken with respect to the ei (t) = ei (t) . Thus, as in Section
B, |ω (t) × x (t, x)|2 =  
I11 I12 I13
ω T (t)  I12 I22 I23  ω (t)
 

I13 I23 I33

Where
Z
!
Ikk = ∑ x2j ρ (x1 , x2 , x3 ) dx
B(0) j̸=k
Z
Ii j = − xi x j ρ (x1 , x2 , x3 ) dx, i ̸= j.
B(0)

As in this section, choose x1 (0) , x2 (0) and x3 (0) such that Ii j = 0 whenever i ̸= j. There-
fore, the kinetic energy in terms of the components of ω taken with respect to the axes,
xi (t) is seen to be
1 3
T = ∑ Ik ω k (t)2 .
2 k=1

Note that Ik is independent of t and the ω k are the components of ω taken with respect to the
axes, xi (t). While this is a nice formula, we want to relate it to the Euler angles because the
Euler angles have more geometric significance. Therefore, what we need to find is ω k (t) in
terms of the time derivatives of the Euler angles. Refering to the above picture of the Euler
angles, we see that φ ′ contributes a term, to the angular velocity vector which is of the
form (0, 0, φ ′ ) where these are the components taken with respect to x11 , x21 and x31 . Writing
this vector in terms of the axes, x12 , x22 and x32 , we get (0, φ ′ sin (θ ) , cos (θ ) φ ′ ) . Now to this
we add the angular
velocity vector contributed by θ ′ which with respect to the axes, x12 , x22
′
and x3 is θ , 0, 0 . Therefore, in terms of x1 , x2 and x32 , we
have the total angular velocity
2 2 2

vector resulting from θ and φ is θ ′ , φ ′ sin (θ ) , cos (θ ) φ ′ . Now we write this vector in
terms of the final coordinate system, x13 , x23 and x33 = x1 (t) , x2 (t) and
x3 (t) . This yields
cos (ψ) θ ′ + sin (ψ) sin (θ ) φ ′ , cos (ψ) sin (θ ) φ ′ − sin (ψ) θ ′ , cos (θ ) φ ′ . To this we must
add the contribution to the angular velocity from ψ ′ which in terms of this last system of
coordinate axes is just (0, 0, ψ ′ ) . Therefore, in terms of x1 (t) , x2 (t) and x3 (t) we have the
angular velocity is

ω = cos (ψ) θ ′ + sin (ψ) sin (θ ) φ ′ , cos (ψ) sin (θ ) φ ′ − sin (ψ) θ ′ , cos (θ ) φ ′ + ψ ′ .

Therefore, the kinetic energy is

1
2
T= I1 cos (ψ) θ ′ + sin (ψ) sin (θ ) φ ′ +
2

2
2 
I2 cos (ψ) sin (θ ) φ ′ − sin (ψ) θ ′ + I3 cos (θ ) φ ′ + ψ ′ . (3.4)

Now we will consider a spinning top or gyroscope. Consider the following picture.
There are two planes through the origin, one perpendicular to the x3 axis, and one perpen-
dicular to the x3 (t) axis. They intersect in the line of nodes shown in the picture. Also, in
the above discussion of the Euler angles, we see the x11 axis is in the plane perpendicular to
x3 and also is in the plane perpendicular to x33 = x3 (t) here. Therefore, φ is as shown in the
C.1. THE SPINNING TOP AND THE EULER ANGLES 883

picture and the other angles are as shown there as well. We see therefore, that φ ′ gives the
angular speed of the line of nodes as the axis, x3 (t) moves around the x3 axis. Thus φ ′ is a
measure of the speed the top or gyroscope moves around the fixed x3 axis.
x3

x3 (t)

x2

φ
x1

line of nodes

We will assume our top has the property that I1 = I2 . This would happen, for example if
the density is a constant and if the cross sections perpendicular to the x3 (t) axis are circles.
Then the potential energy of the top would be of the form Mgl cos θ where M is the total
mass, g is the acceleration of gravity, and l is the distance along the x3 (t) axis to the center
of mass. Then the Lagrangian is of the form
1 h 2
2
2 i
L = I1 sin (θ ) φ ′ + θ ′
2
1 
+ I3 cos2 θ (φ ′ )2 + 2 (cos θ ) φ ′ ψ ′ + (ψ ′ )2 − Mgl cos θ


2
and therefore, the equations of motion are

′
′
I1 sin2 (θ ) φ ′ + I3 cos2 (θ ) φ ′ + I3 cos (θ ) ψ ′ = 0 (3.5)

′
I3 cos (θ ) φ ′ + I3 ψ ′ = 0 (3.6)

2
I1 θ ′′ + φ ′ cos (θ ) sin (θ ) (I3 − I1 ) + I3 sin (θ ) φ ′ ψ ′ − Mgl sin (θ ) = 0 (3.7)
The conservation of energy yields
1 h 2
2
2 i I3  2
I1 sin (θ ) φ ′ + θ ′ + (cos θ ) (φ ′ ) + ψ ′ +
2 2
884 APPENDIX C. LAGRANGIAN MECHANICS

+Mgl cos θ = C. (3.8)

We can use the conservation of energy along with the equations of motion to gain
understanding of the spinning top. From 3.6 we see there is a constant, P such that
I3 cos (θ ) φ ′ + I3 ψ ′ = P and so

P − I3 cos (θ ) φ ′
ψ′ = (3.9)
I3

and from 3.5 there is a constant, Q such that I1 sin2 (θ ) φ ′ + I3 cos2 (θ ) φ ′ + I3 cos (θ ) ψ ′ =
Q. This along with 3.9 implies I1 sin2 (θ ) φ ′ + P cos (θ ) = Q and so we also have

Q − P cos (θ )
φ′ = . (3.10)
I1 sin2 (θ )

Therefore, from the conservation of energy,

1 h 2
2
2 i I3 2
I1 sin (θ ) φ ′ + θ ′ + P + Mgl cos θ = C
2 2
and using 3.10 to find φ ′ , and adjusting the constant,

2 (Q − P cos (θ ))2
I1 θ ′ + I1 + I3 P2 + 2Mgl cos θ = C
I1 sin2 (θ )
2
The expression, f (θ ) = I1 (Q−P cos(θ ))
I sin2 (θ )
+ I3 P2 + 2Mgl cos θ is concave up and has some
1
assmptotes. If C happens to equal the minimum value of f then we must have θ ′ = 0 and
so the top will circle around the x3 axis with θ a constant. Thus we would observe the angle
between the axis of the top and the x3 axis would be constant. If C is not the minimum value
of f then we will have θ changing between two values. This is called nutation. Also, from
3.10 we see that φ ′ is probably not zero. Thus the line of nodes moves around the x3 axis.
Even ψ ′ may change due to 3.9. If ψ ′ were known to be constant, then you could use 3.9
to conclude φ ′ = cosC θ .
Bibliography

[1] Apostol, T. M., Calculus second edition, Wiley, 1967.

[2] Apostol T. Calculus Volume II Second edition, Wiley 1969.
[3] Apostol, T. M., Mathematical Analysis, Addison Wesley Publishing Co., 1974.
[4] Baker, Roger, Linear Algebra, Rinton Press 2001.
[5] Bartle R.G., A Modern Theory of Integration, Grad. Studies in Math., Amer. Math.
Society, Providence, RI, 2000.
[6] Boas, M. Mathematical Methods in Physical Science, John Wiley and Sons, 1966.
[7] Boyce, W. and DiPrima, R. Elementary Differential Equations and Boundary Value
Problems, John Wiley and Sons, 2005.
[8] Bradbury T.C. Theoretical Mechanics, John Wiley and Sons, 1967.
[9] Cramér H., Mathematical Methods of Statistics, Princeton University Press, 1957.
[10] Chahal J. S. , Historical Perspective of Mathematics 2000 B.C. - 2000 A.D.
[11] Davis, H. and Snider, D. Introduction to Vector Analysis, William C. Brown 1997.
[12] D’Angelo, J. and West D. Mathematical Thinking Problem Solving and Proofs, Pren-
tice Hall 1997.
[13] Edwards C.H. Advanced Calculus of several Variables, Dover 1994.
[14] Eves, H. An Introduction To The History of Mathematics, Holt Rinehart and Winston
1976.
[15] Fitzpatrick P. M., Advanced Calculus a course in Mathematical Analysis, PWS Pub-
lishing Company 1996.
[16] Fleming W., Functions of Several Variables, Springer Verlag 1976.
[17] Gray, A. and Mathews, G. B. A Treatise on Bessel functions and Their Applications
to Physics, Macmillan, N.Y. 1952.
[18] Greenberg, M. Advanced Engineering Mathematics, Second edition, Prentice Hall,
1998
[19] Gurtin M. An introduction to continuum mechanics, Academic press 1981.

885
886 BIBLIOGRAPHY

[20] Hardy G., A Course Of Pure Mathematics, Tenth edition, Cambridge University Press
1992.
[21] Hog R. and Craig A., Introduction to Mathematical Statistics, third edition, Macmil-
lan Publishing co. 1970.

[22] Horn R. and Johnson C. matrix Analysis, Cambridge University Press, 1985.
[23] Ince, E.L. Ordinary Differential Equations, Dover 1956.
[24] Karlin S. and Taylor H. A First Course in Stochastic Processes, Academic Press,
1975.

[25] Kuttler K.L., Elementary Differential Equations, CRC Press 2018. 573 pages.
[26] Kuttler K. L., Basic Analysis, Rinton
[27] Kuttler K.L., Modern Analysis CRC Press 1998.

[28] Lang S. Real and Functional analysis third edition Springer Verlag 1993. Press, 2001.
[29] Leighton, W. An Introduction to the Theory of Differential Equations, McGraw Hill,
1952.
[30] Nobel B. and Daniel J. Applied Linear Algebra, Prentice Hall, 1977.

[31] Rudin, W., Principles of mathematical analysis, McGraw Hill third edition 1976
[32] Rudin W., Real and Complex Analysis, third edition, McGraw-Hill, 1987.
[33] Salas S. and Hille E., Calculus One and Several Variables, Wiley 1990.
[34] Sears and Zemansky, University Physics, Third edition, Addison Wesley 1963.

[35] Spivak M., Calculus On Manifolds, Benjamin 1965.

[36] Tierney John, Calculus and Analytic Geometry, fourth edition, Allyn and Bacon,
Boston, 1969.
[37] Widder, D. Advanced Calculus, second edition, Prentice Hall 1961.

[38] Yosida, [Link] on Differential and Integral Equations, Dover 1991.

Index

C1 , 306 balance of momentum, 412

Ck , 306 basic variables, 115
∆, 400 basis, 163
∩, 15 basis, 465
∪, 15 basis of eigenvectors
Rn , 53 diagonalizable, 197
∇2 , 400 beams
buckling, 661
Abel’s formula, 522 Bernoulli equation, 545
absolute value Bessel equation
complex number, 20 ν an integer, 628
acceptance region, 810 eigenvalues, 663
adjoint, 185 parameter not integer, 627
adjugate, 181, 506, 534 Bessel equations, 626
affine linear equations, 562 Bessel function
affine linear procedure, 564 integral identity, 632
agony, pain and suffering, 353 Bessel functions
analytic, 615, 616, 698 addition formula, 631
analytic functions generating function, 631
integral domain, 725 zeros, 663
uniform convergence, 724 Bezier curves, 261
analytic mappings biased estimate, 818
preservation of angles, 721 binomial distribution, 776, 780
angle between planes, 97 binomial theorem, 28, 772
angle between vectors, 78 binormal, 280
angular velocity, 93 boundary problem, 637
angular velocity vector, 454, 871 boundary value problem
annulus, 730 Sturm-Liouville , 661
anti-derivative, 42 bounded, 232
arc length, 262 box product, 89
arcwise connected, 250 Bromwich integral, 759
connected, 250 buckling beams
area of a parallelogram, 86 eigenvalues, 661
arithmetic mean, 344
asymptotically stable, 551 Cartesian coordinates, 54
augmented matrix, 107 Cartesian product, 231
autonomous, 553 Casorati Weierstrass theorem, 735
catenary, 549
back substitution, 107 Cauchy integral theorem, 709

887
888 INDEX

Cauchy principal value, 748 compound interest, 542

Cauchy Riemann equations, 699 computer algebra systems
Cauchy Schwarz inequality, 62, 76, 83 MATLAB, 567
Cauchy sequence, 239 conditional probability, 791
Cauchy stress, 415 confidence interval
Cauchy-Schwarz inequality, 644 mean, 808
Cavendish, 449, 572 ratio of variances, 814
Cayley Hamilton theorem, 513 confidence intervals, 802
Ceasaro means conjugate
convergence, 652 of a product, 29
uniform convergence, 653 connected, 247
center of mass, 92 extended complex plane, 719
central force, 283 open balls, 250
central force field, 449 connected component, 248
central limit theorem, 794 connected components, 248
centripetal force, 448 equivalence class, 249
chain rule, 319 equivalence relation, 249
change of variables formula, 381 open sets, 249
characteristic function, 788 connected set
characteristic polynomial, 183, 509, 513 continuous function, 251
chemical reactions continuous image, 248
balancing, 117 connected sets
chi-squared intersection, 248
moment generating function, 790 intervals, 249
sample variance and variance, 802 real line, 249
chi-squared distribution, 782 union, 248
circular helix, 281 conservation of mass, 412
classical adjoint, 181, 506 conservative, 269, 442
closed set, 229 consistent, 117
closure of a set, 247 constitutive laws, 418
limit points, 247 contented set, 851
coefficient of thermal conductivity, 324 continuity
cofactor, 175, 176, 499, 500, 532 limit of a sequence, 241
cofactor matrix, 176, 500 continuous function, 221
column space, 159 continuous functions
combinations, 771 properties, 226
compact, 252 contour integral, 700
complement, 229 converge, 239
completeness, 31 convergence
complex conjugate, 19 Fourier series, 647
complex numbers, 18 infinite series, 246
complex numbers midpoint of jump, 647
arithmetic, 18 pointwise, 243
roots, 22 uniform, 243
triangle inequality, 20 convolution integral, 595
component, 69, 84 Coordinates, 53
component of a force, 80 Coriolis acceleration
components of a matrix, 130 earth, 459
INDEX 889

Coriolis force, 448 differential equations

counting zeros, 722 homogeneous, 554
counting zeros of analytic function, 722 separable, 546
countour integrals differential equations, 473
definition, 700 differential equations of motion
Cramer’s rule, 512, 535 derivation, 879
critical point, 330 differentiation rules, 257
cross product, 86 dimension, 163
area of parallelogram, 86 directed line segment, 58
coordinate description, 87 direction vector, 58
geometric description, 86 directional derivative, 288
cross product directrix, 85
general curvilinear coordinates, 483 Dirichlet kernel, 648
curl, 399 distance formula, 60
curl distribution function, 803, 833
general curvilinear coordinates, 483 divergence, 399
curvature, 275, 280 divergence, 478
independence of parametrization, 276 general curvilinear coordinates, 479
divergence theorem, 404
D’Alembert, 296 donut, 392
De Moivre’s theorem, 21 dot product, 75
defective, 523 geometric description, 78
deformation gradient, 413 dual basis, 466
degrees of freedom, 782 dual basis, 470
density and mass, 353
dependent, 162 echelon form, 109
derivative, 300 eigenfunctions
has all derivatives, 712 Bessel functions, 663
derivative of a function, 254 eigenspace, 209
determinant, 527 eigenvalue, 187, 343
alternating property, 530 eigenvalues, 513
cofactor, 175, 498 eigenvector, 187
cofactor expansion, 532 Einstein summation convention, 95
expanding along row or column, 176, elementary matrix, 166
499 elementary operations, 105
expansion along row (column), 532 empty set, 16
matrix inverse formula, 181, 506, 534 entries of a matrix, 130
minor, 174, 497 equal area rule, 450
product, 179, 503, 531 equality of mixed partial derivatives, 294
row operations, 178, 502 equations of motion
transpose, 529 spherical coordinates, 880
diameter, 238 equilibrium point, 551
difference quotient, 254 stable, 551
differentiable, 297 Euler equation, 611
differential, 302 Euler’s equations, 874
differential equation Eulerian coordinates, 413
different notation, 541 even extension, 658
linear uniqueness, 540 events, 791
890 INDEX

exact equations, 556 Gauss Jordan method for inverses, 147

procedure, 557 Gauss’s theorem, 404
expansion general solution, 599
Legendre polynomials, 665 geometric mean, 344
exponential growth, 581 geometric multiplicity, 522
extended complex plane, 719 gradient, 290
separated sets, 719 gradient
contravariant components, 478
F distribution, 814 covariant components, 478
factorial, 772 Gram Schmidt process, 204
Fejer kernel, 652 Green’s theorem, 431, 432, 442, 704
Fick’s law, 324, 427 grids, 843
field axioms, 19
finding fundamental matrix hanging chain, 548
Laplace transforms, 592 harmonic, 295
first order systems heat equation, 295
Laplace transform, 591 derivation, 670
flip, 492 solutions, 672
focus, 66 Heine Borel theorem, 252
force, 67 Hermitian, 194
force field, 265, 449 Hessian matrix, 332, 346
Foucalt pendulum, 460 Holder continuous, 227
Fourier law of heat conduction, 324 homogeneous coordinates, 158
Fourier series, 206, 643 homogeneous equation
integrating term by term, 654 procedure, 555
pointwise convergence, 647 homogeneous equations, 554
term by term differentiation, 658 hypergeometric, 780
term by term integration, 656 hypergeometric distribution, 779
Fourier transform, 754 hypergeometric equation, 635
inverse, 754
Fredholm alternative, 199 imaginary part, 698
free variables, 115 implicit function theorem, 485, 489
Frenet Serret formulas, 281 inconsistent, 114, 117
Frobinius norm, 216 increment of volume
fundamental matrix, 586 increment of area, 381
fundamental theorem line integrals, 443 independence
fundamental theorem of algebra, 23, 26 moment generating functions, 794
fundamental theorem of algebra quadratic forms, 820
plausibility argument, 26 independent, 162
rigorous proof, 27 independent random variables
fundamental theorem of calculus, 38, 41 density, 793
functions of, 793
gamma function, 770 index
existence and convergence, 45, 751 lowering, 467
factorials, 752 raising, 467
properties, 46, 752 indicial equation, 612
Gauss Elimination, 117 initial condition, 538
Gauss elimination, 107, 108 initial value problem, 538
INDEX 891

inner product, 75 Laplace expansion, 500, 532

integral Laplace transform, 47, 581, 752
uniform convergence, 44 linear, 581
integral curves, 547 obvious properties, 48
integral domain, 725 step function, 582
integrating factor, 538, 544, 557 table, 582
Euler, 559 Laplacian, 294
procedure, 561 cylindrical coordinates, 669
intercepts, 99 polar coordinates, 322
interior point, 229 spherical coordinates, 669
intermediate value theorem, 249 Laplacian
intersection, 15 general curvilinear coordinates, 480
intervals Laurent series, 734
notation, 16 law of large numbers, 786
inverse leading entry, 108
left inverse, 534 least squares regression, 295
right inverse, 534 Lebesgue number, 252, 849
inverse Fourier transform, 754 Lebesgue’s theorem, 851
inverse function theorem, 488, 490 Legendre equation
inverses and determinants, 507, 533 eigenvalues, 665
invertible, 144 existence of bounded solutions, 665
isolated singularity, 734 Legendre polynomials
iterated integral, 349 expansion, 665
length of smooth curve, 263
Jacobian determinant, 381 Leontief model, 158
Jordan content, 849 level curves
Jordan set, 851 intersecting at right angles, 721
joule, 81 limit of a function, 223
limit point, 233, 285
Kepler’s first law, 450
limits and continuity, 226
Kepler’s laws, 450
line integral, 266
Kepler’s third law, 453
line of nodes, 882
kilogram, 92
linear and nonlinear equations
kinetic energy, 877
differences, 567
kinetic energy, 472
linear combination, 132, 159, 530
Kirchoff’s law, 127, 128
linear differential equations
Kroneker delta, 95
zeros of solutions, 636
Kroneker symbol, 144
linear equation
Lagrange procedure, 544
mechanics, 879 linear equations
Lagrange multipliers, 341, 490, 491 solution, 564
Lagrange remainder, 345, 346 linear regression
Lagrangian coordinates, 413 slope confidence interval, 832
Lagrangian formalism, 473 variance confidence interval, 831
lake pollution, 542 linear system
Laplace equation general solution, 594
circular disk, 686 linear transformations, 135
rectangles, 683 lines
892 INDEX

parametric equation, 58 moments, 788

Liouville motion, 413
transformation, 663 multi-index, 222
Liouville’s theorem, 717 multinomial expansion, 778
Lipschitz, 227, 228
lizards Navier, 428
surface area, 390 nested interval lemma, 32
local extremum, 329 Neuman series, 158
local maximum, 329 Newton, 70
local minimum, 329 Newton’s second law, 473
locally one to one, 724 nilpotent, 518
logistic equation, 550 nonhomogeneous problems
Lotka Volterra equations, 221 solutions, 680
lower sum, 844 nonlinear equation
no uniqueness, 566
main diagonal, 177, 501 nonlinear equations
mass balance, 412 local solutions, 565
material coordinates, 413 norm
math induction, 17 mean square, 643
mathematical induction, 17 uniform, 643
matrix, 88, 129 normal vector to plane, 97
identity, 144 null hypothesis, 810, 837
inverse, 144 nutation, 884
left inverse, 534
lower triangular, 177, 501 obnoxious integrals, 738
right inverse, 534 odd extension, 658
rotation about given vector, 142 one to one
self adjoint, 212 rank, 170
upper triangular, 177, 501 open cover, 252
matrix inverse open mapping theorem, 723, 727
finding it, 145, 147 open set, 229
maximum likelihood estimates, 816 order of a pole, 736
maximum modulus theorem, 724 order of a zero, 715
mean, 789 orientable, 440
mean square orientation, 265
poinwise, 667 positive, 706
mean square norm, 643 oriented curve, 265
meromorphic, 736 origin, 53
metric tensor, 466, 468 orthogonal matrix, 191, 518
metric tensor, 477 orthogonality conditions, 641
minimal polynomial, 171 orthonormal, 191
minimum polynomial, 187 osculating plane, 276, 280
minor, 174, 176, 499, 500, 532
parallelepiped
Mittag Leffler, 744
definition, 89
mixed partial derivatives, 293
volume, 89
moment generating function, 787, 788
parameter, 58
moment of a force, 91
parametric equation, 58
moment of inertia tensor, 873
parametrization, 262
INDEX 893

partial derivative, 290 principal invariants, 510

partial differential equations principal logarithm, 721
nonhomogeneous, 678 principal normal, 275, 280
partition, 33 principal part
partition of unity, 864 Laurent series, 734
periodic, 641 probability
permutation, 527 conditional, 791
permutation symbol, 95 properties, 791
permutations and combinations, 777 product of inertia, 873
permutations of n things taken m at a time, product rule
776 cross product, 258
perp, 198 dot product, 258
perpendicular, 79 projection, 167
piecewise continuous, 654 projection of a vector, 80
Piola Kirchhoff stress, 417 projection onto a subspace, 201
pivot, 114 pumpkin
pivot column, 109 flying, 543
pivot columns, 110
pivot position, 109 quadratic form
pivot positions, 110 distribution, 822
planes, 97 moment generating function, 819
Poincare Bendixon theorem, 608 quadratic forms
point at infinity, 635 independence, 821
pointwise summary of properties, 825
mean square, 667 quadratic formula, 23
pointwise convergence, 42
Poisson distribution, 781 radius of curvature, 275, 280
Poisson’s equation random variables
representation, 689 density, 793
polar form complex number, 21 independent, 792
pole, 735 rank, 164
polynomial, 25 existence of solutions, 198
addition, 25 rank and singular values, 208
degree, 25 real part, 698
division, 25 refinement of grids, 843
equality, 25 regular singular point, 615
multiplication, 25 Euler equation, 616
polynomial exponents of singularity, 620
leading term, 25 solution procedure, 625
monic, 25 regular singular points
polynomials Euler equations, 616
factoring, 23 finding them, 616
polynomials in n variables, 222 indicial equation, 620
position vector, 56, 67 removable singularity, 734
power series residue, 737
Cauchy product, 615 resultant, 69
of inverse, 615 Reynolds transport formula, 420
primitive, 492, 706 Riccati equation, 571, 662
894 INDEX

Riemann criterion, 845 smooth curve, 262

Riemann integrable, 351, 356 solving heat equation, 672
Riemann integral, 351, 356, 845 spacial coordinates, 413
Riemann sphere, 718 span, 159, 530
Riemann sums, 350 speed, 70
Riemann-Lebesgue lemma, 646 spherical coordinates, 471
right handed system, 85 stability from graphs, 553
right inverse, 147 stable, 551
ring, 725 stable equilibrium point, 551
root test, 244 standard normal deviate, 790
rot, 399 standard position, 68
Rouche’s theorem, 722 star shaped, 721
row operations, 108, 178, 502 statistic, 797, 805
row reduced echelon form, 109 statistics, 797
Runge-Kutta method stereographic projection, 718
syntax, 602 Stoke’s theorem, 437
Stokes, 428
saddle point, 332 Sturm
sample average, 798 separation theorem, 662
sample mean, 798 Sturm-Liouville problem, 661
sample variance, 798 subsequence, 239
scalar field, 399 subspace, 160
scalar multiplication, 55 support of a function, 864
scalar potential, 443 symmetric, 138, 153
scalar product, 75 symmetric form of a line, 59
scalars, 55, 129
Schur’s theorem, 192, 193 T distribution, 806
Schwarz lemma, 724 Taylor polynomial
second derivative test, 348 sine, 346
semi-stable Taylor’s formula, 345
equilibrium point, 573 term by term differentiation
separable differential equations, 546 Fourier series, 658
separable equations term by term integration
procedure, 554 Fourier series, 656
separated sets, 247 torque vector, 91
separation of variables, 672 torsion, 280, 281
sequence, 239 torus, 392
sequential compactness, 240 trace, 216
sequentially compact, 240 sum of eigenvalues, 216
set notation, 15 traces, 100
sgn, 525 transformation rules, 474
uniqueness, 527 transpose
sign of a permutation, 527 dot product, 198
simply connected, 719 triangle inequality, 63, 77
singular point, 330 complex numbers, 20
singular value decomposition, 208 trig. identities, 641
singular values, 207 two point boundary value problem, 637
skew symmetric, 138, 153
uniform continuity, 243
INDEX 895

uniform convergence, 42 zeros of analytic function

infinite sums, 246 counting them, 722
uniform norm, 243, 643
uniformly Cauchy sequence of functions, 44
uniformly close, 643
uniformly continuous, 228, 251
union, 15
unit tangent vector, 275, 280
unitary matrix, 191
upper sum, 844
Urysohn’s lemma, 252

variance, 789
meaning, 794
vector
scalar multiplication, 55
vector addition, 55
vector
contravariant components, 467
covariant components, 467
vector field, 265, 399
vector fields, 218
vector potential, 401
vector space axioms, 131
vector valued function
continuity, 222
derivative, 254
integral, 254
limit theorems, 223
vector valued functions, 217
vectors, 53, 67
velocity, 70
volume element, 381
volume increment, 381
volume of parallelepiped, 379
volume of unit ball in n dimensions, 424

wave equation, 295

derivation, 675
well ordered, 17
well ordering, 16
winding number
orientation, 706
simple closed curve, 706
work, 266
Wronskian, 521
Wronskian alternative, 599

zero matrix, 130