MATH 3321 Numerical Analysis I (Math3522 Numerical Methods) Chapter 0

Islamic University of Madinah

Faculty of Science
Mathematics Department

Numerical Analysis I (Numerical Methods)

Course References:
[1] Burden, R. and Fairs, J.D., Numerical analysis, 9th edition, Thomson Brooks Cole,
2010.

[2] Chapra, S.C. and Canale, R.P., Numerical Methods for Engineers: A MATLAB, 6
Edition McGraw- Hill, C (2010).

[3] Jacques, I. and Judd, C., Numerical analysis, Chapman and Hall, 1987.

[4] Cheney, W. and Kincaid, D.R., Numerical Mathematics and Computing, 7th edition,
Cengage Learning, 2013.

❖ Saudi Digital Library: [Link]

CHAPTER 0: Preliminaries
Chapter Objectives

❖ Understand the concepts of approximation of numbers.

❖ Know the techniques of chopping and rounding of numbers.

❖ Recognize the normalized decimal floating-point representation of numbers.

❖ Understand the absolute and the relative errors.

❖ Remember Taylor's theorem.

• Course Materials will be covered from the course Textbooks and These Lecture
Notes

1
MATH 3321 Numerical Analysis I (Math3522 Numerical Methods) Chapter 0

Chapter 0

Preliminaries
Round-Off Error and Computer Arithmetic

➢ Numerical systems:
1. Decimal (Base ten)
2. Binary (Base two)

Example:

(10.625)10 = 1 × 101 + 0 × 100 + 6 × 10−1 + 2 × 10−2 + 5 × 10−3

(1010.101)2
= 1 × 23 + 0 × 22 + 1 × 21 + 0 × 20 + 1 × 2−1 + 0 × 2−2
+ 1 × 2−3

Verify that
85
(10.625)10 = (1010.101)2 =
8

➢ Computers use:
1. Finite number of digits or bits (0’s and 1’s) to represent
numbers
2. Binary system
➢ Machine numbers: numbers that can be represented in a
computer.

2
MATH 3321 Numerical Analysis I (Math3522 Numerical Methods) Chapter 0

Some numbers cannot be represented exactly in any computer

(machine).

Example. Take the number 𝑥 = 1/10. Then

Decimal: 𝑥 = 10−1 = (0.1)10

Binary: 𝑥 = (0.00011001100. . . )2 (infinitely many digits!)

Thus, the number 𝑥 = 1/10 cannot be represented exactly on any

machine.

For simplicity, from now on, we assume that machine numbers are in
the normalized decimal floating-point form:

±0. 𝑑1 𝑑2 . . . 𝑑𝑘 × 10𝑛

called 𝒌-digit system with

0 ≤ 𝑑𝑖 ≤ 9 and 𝑑1 ≠ 0 (normalized)

We know that any real number 𝑟 can be written as

𝑟 = ±0. 𝑑1 𝑑2 𝑑3 . . . 𝑑𝑘 𝑑𝑘+1 . . . 𝑑∞ × 10𝑛

➢ The representation of 𝑟 in a 𝑘-digit machine is denoted by 𝑓𝑙(𝑟).

1. 𝑘-digit chopping: 𝑓𝑙(𝑟) = ±0. 𝑑1 𝑑2 𝑑3 . . . 𝑑𝑘 × 10𝑛
2. 𝑘-digit rounding: if 𝑑𝑘+1 ≥ 5 add 1 to 𝑑𝑘 .

Example. Determine the 5-digit (a) chopping and (b) rounding of the
number
𝑟 = 𝜋 = 3.14159265 …
3
 MATH 3321 Numerical Analysis I (Math3522 Numerical Methods) Chapter 0

Solution. First, normalize: 𝑟 = 0.314159265 … × 101 . Then

(a) 𝑓𝑙(𝑟) = 𝑓𝑙(0.314159265 … × 101 ) = 0.31415 × 101 = 3.1415

(b) 𝑓𝑙(𝑟) = 𝑓𝑙(0.314159265 … × 101 ) = 0.31416 × 101 = 3.1416.

➢ Finite-Digit Arithmetic:

On paper On computer
𝑥+𝑦 𝑥 ⊕ 𝑦 = 𝑓𝑙(𝑓𝑙(𝑥) + 𝑓𝑙(𝑦))
𝑥−𝑦 𝑥 ⊖ 𝑦 = 𝑓𝑙(𝑓𝑙(𝑥) − 𝑓𝑙(𝑦))
𝑥×𝑦 𝑥 ⊗ 𝑦 = 𝑓𝑙(𝑓𝑙(𝑥) × 𝑓𝑙(𝑦))
𝑥÷𝑦 𝑥 ⊘ 𝑦 = 𝑓𝑙(𝑓𝑙(𝑥) ÷ 𝑓𝑙(𝑦))

Example. Let 𝑥 = 1/7 and 𝑦 = 80/13. Use 3-digit rounding for

calculating 𝑥 + 𝑦, 𝑥 − 𝑦, 𝑥 × 𝑦, and 𝑥 ÷ 𝑦.

Solution. We have
1
𝑥= ≈ 0.142857 → 𝑓𝑙(𝑥) = 0.143
7
80
𝑦= ≈ 6.15385 = 0.615385 × 101 → 𝑓𝑙(𝑦) = 0.615 × 101 = 6.15
13

Therefore
𝑥 ⊕ 𝑦 = 𝑓𝑙(𝑓𝑙(𝑥) + 𝑓𝑙(𝑦)) = 𝑓𝑙(0.143 + 6.15) = 𝑓𝑙(6.293)
= 𝑓𝑙(0.6293 × 101 ) = 0.629 × 101 = 6.29

The rest are left as an exercise.

➢ Measuring errors: Suppose that 𝑝̃ approximates 𝑝.

4
MATH 3321 Numerical Analysis I (Math3522 Numerical Methods) Chapter 0

1. Absolute Error:
AE = |𝑝̃ − 𝑝|
2. Relative Error:
|𝑝̃ − 𝑝|
RE =
|𝑝|
Example. Determine the absolute and relative errors when
approximating 𝑝 by 𝑝̃.
(a) 𝑝 = 𝜋 and 𝑝̃ = 22/7.
22
22 | − 𝜋|
AE = | − 𝜋| = 0.0013 and RE = 7 = 0.0004
7 |𝜋|
(b) 𝑝 = 8! = 40320 and 𝑝̃ = 40000.
|40000 − 40320|
AE = |40000 − 40320| = 320 and RE = = 0.01
|40320|
(c) 𝑝 = 0.001 and 𝑝̃ = 0.01.
|0.01 − 0.001|
AE = |0.01 − 0.001| = 0.009 and RE = =9
|0.001|
Remark: The relative error is more meaningful than the absolute error.

Important. In finite-digit arithmetic AVOID:

1- Subtraction of nearly equal numbers

2- Division by small numbers

Example. Let 𝑥 = 2551.5052 = 0.25515052 × 104 , 𝑦 = 2551.4911 =

0.25514911 × 104 .

Exact arithmetic:
𝑥 − 𝑦 = 0.1410 × 10−1 ≅ 0.01.
Assume we are using 4-digit rounding.

Finite digit arithmetic:

5
MATH 3321 Numerical Analysis I (Math3522 Numerical Methods) Chapter 0

𝑥 ⊖ 𝑦 = 𝑓𝑙(0.2552 × 104 − 0.2551 × 104 ) = 𝑓𝑙(0.1000 × 101 ) = 1

Notice the relative error:
|(𝑥 ⊖ 𝑦) − (𝑥 − 𝑦)|
RE = ≅ 70 (Large!)
|(𝑥 − 𝑦)|
➢ Rates of Convergence
Let 𝛼𝑛 be a sequence such that 𝛼𝑛 → 𝛼. Notice this means
|𝛼𝑛 − 𝛼| → 0. Assume we found another sequence. 𝛽𝑛 such that
𝛽𝑛 → 0.

• If there is a constant 𝐾 such that

|𝛼𝑛 − 𝛼| ≤ 𝐾|𝛽𝑛 | (for all large 𝑛)
Then we say that 𝛼𝑛 converges to 𝛼 with a rate or order 𝑂(𝛽𝑛 ). In
this case we write
𝛼𝑛 = 𝛼 + 𝑂(𝛽𝑛 ).
Example. Find the rate of convergence of the following sequences as
𝑛 → ∞.
𝑛+1
(a) lim = 0.
𝑛→∞ 𝑛2

Solution. We have
𝑛+1 𝑛+𝑛 1
|𝛼𝑛 − 𝛼| = | − 0| ≤ | | = 2 | |.
𝑛2 𝑛2 𝑛
𝑛+1 1
2
= 0+𝑂( )
𝑛 𝑛
1 2
(b) lim (sin ) = 0.
𝑛→∞ 𝑛

Solution. Since |sin 𝜃| ≤ |𝜃|, we have

1 2 12 12 1
|𝛼𝑛 − 𝛼| = |(sin ) − 0| = |sin | ≤ | | = | 2 |.
𝑛 𝑛 𝑛 𝑛
6
MATH 3321 Numerical Analysis I (Math3522 Numerical Methods) Chapter 0

1 2 1
⟹ (sin ) = 𝑂 ( 2 )
𝑛 𝑛

5𝑛2 +4𝑛+2 1
Example. Let 𝛼𝑛 = . Show that 𝛼𝑛 = 5 + 𝑂 ( ).
𝑛2 𝑛

Solution. We have
5𝑛2 + 4𝑛 + 2 4𝑛 + 2 4𝑛 + 2𝑛 1
|𝛼𝑛 − 𝛼| = | − 5| = | | ≤ | | = 6 | |.
𝑛2 𝑛2 𝑛2 𝑛

7
MATH 3321 Numerical Analysis I (Math3522 Numerical Methods) Chapter 0

➢ Rates of Convergence (cont.)

We say that the function 𝐹(ℎ) converges to 𝐿 with a rate or order
𝑂(𝐺(ℎ)) as ℎ → 0 if

1. 𝐺(ℎ) → 0
2. There is a constant 𝐾 such that
|𝐹(ℎ) − 𝐿| ≤ 𝐾|𝐺(ℎ)| (for all small ℎ)
In this case, we write. 𝐹(ℎ) = 𝐿 + 𝑂(𝐺(ℎ)).
Remark. It helps to recall Taylor's theorem:
′(
𝑓 ′′ (𝑥0 ) 2
𝑓 (𝑛) (𝑥0 )
𝑓(𝑥) = 𝑓(𝑥0 ) + 𝑓 𝑥0 )(𝑥 − 𝑥0 ) + (𝑥 − 𝑥0 ) + ⋯ + (𝑥 − 𝑥0 )𝑛
⏟ 2! 𝑛!
𝑃𝑛 (𝑥)
(𝑛+1) (
𝑓 𝜉)
+ (𝑥 − 𝑥0 )𝑛+1
⏟(𝑛 + 1)!
𝑅𝑛 (𝑥)
where 𝜉 between 𝑥0 and 𝑥.
sin(ℎ)
Example. Show that = 1 + 𝑂(ℎ2 ).
ℎ

Solution. Taylor's Theorem with 𝑛 = 2 and 𝑥0 = 0 reads as:

′ (0)
𝑓′′ (0) 2
𝑓(3) (𝜉)
𝑓(𝑥) = 𝑓(0) + 𝑓 𝑥+ 𝑥 + 𝑥3 (𝜉 between 0 and 𝑥)
2! 3!
Let 𝑓(𝑥) = sin(𝑥), then
cos(𝜉) 3
sin(𝑥) = 𝑥 − 𝑥 (𝜉 between 0 and 𝑥).
6
Therefore (after setting 𝑥 = ℎ):

sin(ℎ) cos(𝜉) 2 1 sin(ℎ)

| − 1| = | ℎ | ≤ |ℎ2 | ⇒ = 1 + 𝑂(ℎ2 )
ℎ 6 6 ℎ
In other words:
sin(ℎ)
as ℎ → 0, approaches 1 about as fast as ℎ2 approaches 0.
ℎ
8