Dr.
Denis Ndanguza (2021)
ENE6161 MATHEMATICAL ANALYSIS AND MATRIX
THEORY
Chap 4. Numerical Differentiation
Dr. Denis Ndanguza
UNIVERSITY OF RWANDA
College of Science and Technology
School of Science
Department of Mathematics
January 30, 2021
Dr. Denis Ndanguza (2021)
Approximating the first derivatives
Suppose we have some blackbox function f (x) and we wish to calculate
f 0 (x) at some given x. Not surprisingly, we start with Taylor’s theorem:
1
f (x + h) = f (x) + f 0 (x)h + [f 00 (ξ)h2 ].
2
Rearranging we get
f (x + h) − f (x) 1 00
f 0 (x) = − [f (ξ)h].
h 2
Remember that ξ is between x and x + h, but its exact value is not
known. If we wish to calculate f 0 (x), we cannot evaluate f 00 (ξ), so we
approximate the derivative by dropping the last term.
Dr. Denis Ndanguza (2021)
Approximating the first derivatives
That is, we calculate (f (x + h) − f (x))/h as an approximation to f 0 (x). In
so doing, we have dropped the last term. If there is a finite bound on
f 00 (z) on the interval in question then the dropped term is bounded by a
constant times h. That is,
f (x + h) − f (x)
f 0 (x) = + Oh (1)
h
The error that we incur when we approximate f 0 (x) by calculating
(f (x + h) − f (x))/h is called truncation error. It has nothing to do with
the kind of error that you get when you do calculations with a computer
with limited precision; even if you worked in infinite precision, you would
still have truncation error.
Dr. Denis Ndanguza (2021)
First derivatives
The truncation error for this approximation is Oh. We may want a more
precise approximation. By now, you should know that any calculation
starts with Taylor’s Theorem:
f 00 (x) 2 f 000 (ξ1 ) 3
f (x + h) = f (x) + f 0 (x)h + h + h
2 3!
f 00 (x) 2 f 000 (ξ2 ) 3
f (x − h) = f (x) − f 0 (x)h + h − h
2 3!
By subtracting these two lines, we get
f 000 (ξ1 ) + f 000 (ξ2 ) 3
f (x + h) − f (x − h) = 2f 0 (x)h + h .
3!
Thus
f 000 (ξ1 ) + f 000 (ξ2 ) 3
2f 0 (x)h = f (x + h) − f (x − h) − h
3!
f (x + h) − f (x − h) f 000 (ξ1 ) + f 000 (ξ2 ) h2
f 0 (x) = −( )
2h 2 6
Dr. Denis Ndanguza (2021)
First derivatives
If f 000 (x) is continuous, then there is some ξ between ξ1 , ξ2 such that
f 000 (ξ1 ) + f 000 (ξ2 )
f 000 (ξ) = .
2
Assuming some uniform bound on f 000 (·), we get
f (x + h) − f (x − h)
f 0 (x) = + Oh2 (2)
2h
It is also called ”the central difference approximation of the first
derivative”.
Dr. Denis Ndanguza (2021)
Summary
These three formulas are summarized here for easy reference:
df f (x + h) − f (x)
Forward difference : =
dx h
df f (x) − f (x − h)
Backward difference : =
dx h
df f (x + h) − f (x − h)
Central difference : =
dx h
Dr. Denis Ndanguza (2021)
Example
The distance x of a runner from a fixed point is measured (in metres) at
intervals of half a second. The data obtained is
t 0.0 0.5 1.0 1.5 2.0
x 0.00 3.65 6.80 9.90 12.15
Use backward, forward and central differences to approximate the runner’s
velocity at times t = 0.5s and t = 1.25s.
Dr. Denis Ndanguza (2021)
Example
Use evaluations of f at x + h and x + 2h to approximate f 0 (x), assuming
f (x) is an analytic function, i.e one with infinitely many derivatives.
Dr. Denis Ndanguza (2021)
Solution
First use Taylor’s Theorem to expand f (x + h) and f (x + 2h), then
subtract to get some factor of f 0 (x):
4h2 00 8h3 000
f (x + 2h) = f (x) + 2hf 0 (x) + 2! f (x) + 3! f (x) + . . .
h2 00 h3 000
f (x + h) = f (x) + hf 0 (x) + 2! f (x) + 3! f (x) + . . .
3h2 00 7h3 000
f (x + 2h) − f (x + h) = hf 0 (x) + 2! f (x) + 3! f (x) + . . .
7h2 000
(f (x + 2h) − f (x + h))/h = f 0 (x) + 3h 00
2! f (x) + 3! f (x) + . . .
Thus
(f (x + 2h) − f (x + h))/h = f 0 (x) + Oh
Dr. Denis Ndanguza (2021)
Example
Show that
4f (x + h) − f (x + 2h) − 3f (x)
= f 0 (x) + Oh2 ,
2h
for f with sufficient number of derivatives
Dr. Denis Ndanguza (2021)
Solution
We only have to expand the appropriate terms with Taylor’s Theorem. As
before:
h2 00
f (x + h) = f (x) + hf 0 (x) + 2! f (x) + . . .
2
4f (x + h) = 4f (x) + 4hf 0 (x) + 4h 00
2! f (x) + . . .
2
f (x + 2h) = f (x) + 2hf 0 (x) + 4h 00
2! f (x) + . . .
−4h3
4f (x + h) − f (x + 2h) = 3f (x) + 2hf 0 (x) + 0f 00 (x) + 3!
−4h3 000
4f (x + h) − f (x + 2h) − 3f (x) = 2hf 0 (x) + 3! f (x) + . . .
−2h2 000
(4f (x + h) − f (x + 2h) − 3f (x))/2h = f 0 (x) + 3! f (x) + . . .
Dr. Denis Ndanguza (2021)
Approximating the Second Derivative
We want to approximate the second derivative of the function f (x).
f 00 (x) 2 f 000 (x) 3 f (4) (x) 4
f (x + h) = f (x) + f 0 (x)h + h + h + h + ...
2 3! 4!
f 00 (x) 2 f 000 (x) 3 f (4) (x) 4
f (x − h) = f (x) − f 0 (x)h + h − h + h − ...
2 3! 4!
Now add the two series to get
f (4) (x) 4 f (6) (x) 6
f (x + h) + f (x − h) = 2f (x) + h2 f 00 (x) + 2 h +2 h + ...
4! 6!
Then let
f (x + h) − 2f (x) + f (x − h) f (4) (x) 2 f (6) (x) 4
ψ(h) = = f 00 (x) + 2 h +2 h
h2 4! 6!
∞
X
00
= f (x) + b2k h2k .
k=1
Dr. Denis Ndanguza (2021)
Second derivative
Thus we can use Richardson Extrapolation on ψ(h) to get higher order
approximations.
This derivation also gives us the centered difference approximation to the
second derivative:
f (x + h) − 2f (x) + f (x − h)
f 00 (x) = + Oh2 . (3)
h2
Dr. Denis Ndanguza (2021)
Example
Values for f (x) = xe x are given in the following table. Use second
approximation formula to approximate f 00 (2.0).
x 1.8 1.9 2.0 2.1 2.2
f(x) 10.889365 12.703199 14.778112 17.148957 19.855030
1
f 00 (2.0) = [f (1.9) − 2f (2.0) + f (2.1)]
(0.1)2
1
= [12.703199 − 2(14.778112) + (17.148957)]
(0.1)2
=
Dr. Denis Ndanguza (2021)
Derivatives using Newton’s Difference table
x y0 ∆y0 ∆2 y0 ∆3 y0
x0 y0
∆y0 = y1 − y0
x1 y1 ∆2 y0 = ∆y1 − ∆y0
∆y1 = y2 − y1 ∆3 y0 = ∆2 y1 − ∆2 y0
x2 y2 ∆2 y1 = ∆y2 − ∆y1
∆y2 = y3 − y1
x3 y3
Dr. Denis Ndanguza (2021)
Forward and backward interpolation formula
Newton’s forward interpolation formula is
p(p − 1) 2 p(p − 1)(p − 2) 3
y = y0 + p∆y0 + ∆ y0 + ∆ y0 + . . . (4)
2! 3!
where p = (x−x0)
h .
Newton’s backward interpolation formula for the function f (x) is given by
p(p + 1) 2 p(p + 1)(p + 2) 3
y = yn + p∆yn + ∆ yn + ∆ yn + . . . (5)
2! 3!
(x−xn )
where p = h .
Dr. Denis Ndanguza (2021)
First derivatives
After differentiating both sides of (4) with respect to p, and at x = x0
with p = 0, we have the first derivative formula as follows:
dy 1 1 2 1 3 1 4
|x=x0 = ∆y0 − ∆ y0 + ∆ y0 − ∆ y0 + . . . (6)
dx h 2 3 4
The same for (5) at x = xn ,
dy 1 1 2 1 3 1 4
|x=xn = ∆yn + ∆ yn + ∆ yn + ∆ yn + . . . (7)
dx h 2 3 4
Dr. Denis Ndanguza (2021)
Second derivatives
Differentiating equations (4) and (5) with respect to x and putting p = 0,
d 2y
1 2 3 11 4
|x=x0 = 2 ∆ y0 − ∆ y0 + ∆ y0 − . . . (8)
dx 2 h 12
d 2y
1 11 4 5
|x=xn = 2 ∆2 yn + ∆3 yn + ∆ yn + ∆5 yn + . . . (9)
dx 2 h 12 6
Dr. Denis Ndanguza (2021)
Third derivatives
Similarly
d 3y d 2y
d
=
dx 3 dx dx 2
d 3y
1 3
3
|x=x0 = 3 ∆ y0 − ∆4 y0 + . . .
3
(10)
dx h 2
and for backward
d 3y
1 3 3 4
3
|x=xn = 3 ∆ yn + ∆ yn + . . . (11)
dx h 2
and so on.
Dr. Denis Ndanguza (2021)
Derivatives Using Stirling’s Interpolation Formula
Stirling’s central difference interpolation formula (taking x0 as the middle
value of the table) is given by:
p2 2 p(p 2 − 1) ∆3 y−1 + ∆3 y−2
∆y0 + ∆y−1
y ≡ y0 + p + ∆ y−1 +
2 2! 3! 2
2 2
p (p − 1) 4 x − x0
+ ∆ y−2 + . . . with p =
4! h
Dr. Denis Ndanguza (2021)
Derivatives
At x = x0 and p = 0;
1 ∆3 y−1 + ∆3 y−2
dy 1 ∆y0 + ∆y−1
]x=x0 = − +
dx h 2 6 2
1 1 ∆5 y−2 + ∆5 y−3
+ + ...
h 30 2
and the second derivative
d 2y
1 2 1 4 1 6
]x=x0 = ∆ y−1 − ∆ y−2 + ∆ y−3 − . . .
dx 2 h2 12 90
Dr. Denis Ndanguza (2021)
Example
Find the first, second and third derivatives of f (x) at x = 1.5 if
x 1.5 2.0 2.5 3.0 3.5 4.0
f(x) 3.375 7.00 13.625 24.00 38.875 59.00
Dr. Denis Ndanguza (2021)
Solution
Here
x0 = 1.5, y0 = 3.375, ∆y0 = 3.625, ∆2 y0 = 3.00, ∆3 y0 = 0.75, ∆4 y0 = 0
and h = 0.5
Now, from equation (6), we have:
dy 1 1 1 1
|x=x0 = f 0 (x0 ) = (∆y0 − ∆2 y0 + ∆3 y0 − ∆4 y0 + . . .)
dx h 2 3 4
0 1 1 1
f (1.5) = [3.625 − (3) + (0.750)] = 4.75
0.5 2 3
From equation (8);
d 2y 1 11
|x=x0 = f 00 (x0 ) = 2 (∆2 y0 − ∆3 y0 + ∆4 y0 − . . .)
dx 2 h 12
1
f 00 (1.5) = [3 − 0.75] = 9
(0.5)2
Dr. Denis Ndanguza (2021)
Solution...
From equation (10); we have:
d 3y 1 3
|x=x0 = f 000 (x0 ) = 3 (∆3 y0 − ∆4 y0 + . . .)
dx 3 h 2
1
f 000 (1.5) = (0.75) = 6
(0.5)3
Dr. Denis Ndanguza (2021)
Exercise
Given a polynomial with following data points:
x 1.0 1.1 1.2 1.3 1.4 1.5 1.6
f(x) 7.989 8.403 8.781 9.129 9.451 9.750 10.031
2
Find dy d y
dx and dx 2 at x = 1.1 and x = 1.5.
Derivatives has to be evaluated near the starting as well as towards the end
of the table, thereby constructing difference table for the function y = f (x)
Dr. Denis Ndanguza (2021)
Solution
x y = f (X ) ∆y0 ∆2 y 0 ∆3 y0 ∆4 y0 ∆5 y 0 ∆6 y 0
1.0 7.989
0.414
1.1 8.403 −0.036
0.378 0.006
1.2 8.781 −0.030 −0.002
0.348 0.004 0.001
1.3 9.129 −0.026 −0.001 0.002
0.322 0.003 0.003
1.4 9.451 −0.023 0.002
0.299 0.005
1.5 9.750 −0.018
0.281
1.6 10.031
Dr. Denis Ndanguza (2021)
Solution
To find the derivative at x = 1.1, taking x0 = 1.1 and applying the
relation:
dy 1 1 2 1 3 1 4 1 5
]x=x0 = ∆y0 − ∆ y0 + ∆ y0 − ∆ y0 + ∆ y0
dx h 2 3 4 5
From the table h = 0.1, ∆y0 = 0.378, ∆2 y0 = −0.03, ∆3 y0 =
0.004, ∆4 y0 = −0.001, ∆5 y0 = 0.003
dy 1 1 1 1 1
]x=x0 = 0.378 − (−0.03) + (0.004) − (−0.001) + (0.003)
dx h 2 3 4 5
= 3.952
Dr. Denis Ndanguza (2021)
Solution...
d 2y
1 2 3 11 4
]x=x0 = ∆ y0 − ∆ y0 + ∆ y0
dx 2 h2 12
1 11 5
= −0.03 − 0.004 + (−0.001) − (0.003)
(0.5)2 12 6
= −3.74
To find the derivative at x = 1.5, taking xn = 1.8 and applying the relation
dy 1 1 1 1 4
|x=xn = ∆yn + ∆2 yn + ∆3 yn + ∆ yn
dx h 2 3 4
Dr. Denis Ndanguza (2021)
Solution...
From table
h = 0.1, ∆yn = 0.299, ∆2 yn = −0.023, ∆3 yn = 0.003, ∆4 yn = −0.001
and ∆5 yn = 0.001. By substituting these values, we get
dy 1 1 1
|x=1.5 = 0.299 + (−0.023) + (0.003) +
dx 0.1 2 3
1 1 1
+ (−0.001) + (0.001)
0.1 4 5
= 2.8845
Dr. Denis Ndanguza (2021)
Solution...
Computing the second derivative,
d 2y
1 2 3 11 4 5 5
]x=xn = ∆ yn + ∆ yn + ∆ yn + ∆ yn
dx 2 h2 12 6
1 11 5
= −0.023 + 0.003 + (−0.001) + (0.001)
(0.1)2 12 6
= −2.0083
Dr. Denis Ndanguza (2021)
Exercise
The population in million of a certain town is shown in the following table:
Year 1951 1961 1971 1981 1991
Population 19.96 36.65 58.81 77.21 94.61
Find the rate of growth of the population in year 1951.
Dr. Denis Ndanguza (2021)