Math 4543: Numerical Methods

Lecture 2 — Measuring Errors & Sources of Error

Syed Rifat Raiyan

Lecturer
Department of Computer Science & Engineering
Islamic University of Technology, Dhaka, Bangladesh
Email: rifatraiyan@[Link]

Math 4543: Numerical Methods Lecture 2 SWE, IUT 1

Lecture Plan
The agenda for today

• Different types of errors

• How to measure them?

• Discuss about their interpretability

• Concept of Significant Digits

• Relationship between Relative Approximate Error and Significant Digits

• Different sources of errors

Math 4543: Numerical Methods Lecture 2 SWE, IUT 2

Errors
How do we deal with them

In any numerical analysis, errors will arise during the calculations.

To be able to deal with the issue of errors, we —
• Identify where the error is coming from
• Quantify the error
• Minimize the error as per our needs.

Let’s focus on identifying and quantifying the errors in this lecture.

Math 4543: Numerical Methods Lecture 2 SWE, IUT 3

True Error
What is True Error?

True error denoted by is the difference between the true value (also called the exact value) and
the approximate value.

The True value is also referred to as the Exact value or the Ground Truth value.

Math 4543: Numerical Methods Lecture 2 SWE, IUT 4

True Error
An example

Math 4543: Numerical Methods Lecture 2 SWE, IUT 5

True Error
An example

Math 4543: Numerical Methods Lecture 2 SWE, IUT 6

True Error
An example

Math 4543: Numerical Methods Lecture 2 SWE, IUT 7

True Error
But is it properly interpretable?

A low True Error doesn’t necessarily translate to a good approximation.

We need some scale invariance for proper interpretation!

Math 4543: Numerical Methods Lecture 2 SWE, IUT 8

Relative True Error
What is Relative True Error?

Relative true error is denoted by and is defined as the ratio between the true error and
the true value.

Now, we can interpret it as — what portion of the True value is different from the
Approximate value?

Math 4543: Numerical Methods Lecture 2 SWE, IUT 9

Relative True Error
Working with the same example

Now, try to find out the Relative True Error

for
using the same parameters and the
step size,

Spoiler: It should also be

Now we can interpret the error properly!

Math 4543: Numerical Methods Lecture 2 SWE, IUT 10

Approximate Error
What is Approximate Error?

Suppose, we don’t have the luxury of knowing the true value. When we are solving a problem
numerically, we will only have access to approximate values.

We need to know how to quantify error for such cases, and such an error is appropriately
called the approximate error.

Approximate error is denoted by , and is defined as the difference between the present
approximation and the previous approximation.

Math 4543: Numerical Methods Lecture 2 SWE, IUT 11

Approximate Error
An example

Math 4543: Numerical Methods Lecture 2 SWE, IUT 12

Approximate Error
An example

Math 4543: Numerical Methods Lecture 2 SWE, IUT 13

Approximate Error
Facing the same interpretability problem?

A low Approximate Error doesn’t necessarily translate to a good approximation.

We need some scale invariance for proper interpretation in this case as well!

Math 4543: Numerical Methods Lecture 2 SWE, IUT 14

Relative Approximate Error
What is Relative Approximate Error?

As observed in the previous lesson, approximate errors generally do not reflect how bad or
acceptable an error is. We hence define the relative approximate error, denoted by as the ratio
between the approximate error and the present approximation.

Now, we can interpret it as — what portion of the new approx. value is different from the
previous approx. value?

Math 4543: Numerical Methods Lecture 2 SWE, IUT 15

Relative Approximate Error
Working with the same example

Now, try to find out the Relative Approx. Error for

using the same step sizes,
and
Math 4543: Numerical Methods Lecture 2 SWE, IUT 16
Significant Digits
What are Significant Digits?

Significant figures (also known as the significant digits, precision or resolution) of a number in
positional notation are digits in the number that are reliable and necessary to indicate the
quantity of something. These are the digits that are known reliably in a reported number.

Thermal Expansion Coefficient of Steel =

=

If we were writing the value up to 2 decimal places, it would be written as .

Hence, the emphasis in engineering is to write the numbers mostly in the scientific format as it
represents the numbers with the proper number of significant digits.

Math 4543: Numerical Methods Lecture 2 SWE, IUT 17

Significant Digits
The rules that govern Significant Digits
• Non-zero digits are always significant.
• Zeros between non-zero digits are always significant.
• Leading zeros are never significant.
• Trailing zeros are only significant if the number contains a decimal point (or if willingly
considered to be significant).

Math 4543: Numerical Methods Lecture 2 SWE, IUT 18

Significant Digits
The rules that govern Significant Digits

Arbitrary precision

Math 4543: Numerical Methods Lecture 2 SWE, IUT 19

Mini Quiz
Practise identifying Significant Digits

Count the minimum number of significant digits and say which digits are significant.

81
26.2
0.007
5200.38
380.0
78800
78800.

Math 4543: Numerical Methods Lecture 2 SWE, IUT 20

Mini Quiz
Practise identifying Significant Digits (solutions)

Count the minimum number of significant digits and say which digits are significant.

81 — 2
26.2 — 3
0.007 — 1
5200.38 — 6
380.0 — 4
78800 — 3
78800. — 5

Math 4543: Numerical Methods Lecture 2 SWE, IUT 21

Significant Digits × | ∈𝒂 |
How do they relate?

Math 4543: Numerical Methods Lecture 2 SWE, IUT 22

Significant Digits × | ∈𝒂 |
An example

Math 4543: Numerical Methods Lecture 2 SWE, IUT 23

Significant Digits × | ∈𝒂 |
An example

Math 4543: Numerical Methods Lecture 2 SWE, IUT 23

Sources of Error
Types of errors in Numerical Analysis

Error in solving an engineering or science problem can arise due to several factors.

First, the error may be in the modeling technique. A mathematical model may be based on using
assumptions that are not acceptable.
Second, errors may arise from mistakes in programs themselves or in the measurement of
physical quantities.

In applications of numerical methods themselves, the two errors we focus on are inherent to the
subject, namely
• Round-off Error — caused by approximating number
• Truncation Error — caused by approximating mathematical processes or modeling technique

Math 4543: Numerical Methods Lecture 2 SWE, IUT 24

Round-off Error
What is Round-off Error?

Computers would need an infinite number of bits to store the numbers in binary!
But, registers have a finite number of bits. So we need to chop or round.

Math 4543: Numerical Methods Lecture 2 SWE, IUT 25

Truncation Error
What is Truncation Error?

Math 4543: Numerical Methods Lecture 2 SWE, IUT 26

Truncation Error
Examples (Chopping of terms in a series)

Think: How to reduce this?

Math 4543: Numerical Methods Lecture 2 SWE, IUT 27

Truncation Error
Examples (Using a finite step size instead of an infinitesimal value)

Math 4543: Numerical Methods Lecture 2 SWE, IUT 28

Truncation Error
Examples (Using a finite step size instead of an infinitesimal value)

Think: How to reduce this?

Math 4543: Numerical Methods Lecture 2 SWE, IUT 29
Mini Quiz
Theoretical vs Practical notion of infinitesimally small values

Is it a good idea to take a step size of ℎ = 10−20 to approximate the derivative 𝑓′(3)?
If so, why? If not, why not?

Math 4543: Numerical Methods Lecture 2 SWE, IUT 30

Truncation Error
Examples (Approximating Area under curve with Rectangles)

Think: How to reduce this?

Math 4543: Numerical Methods Lecture 2 SWE, IUT 31