• Mathematical models are always needed in solving engineering problems.

• Many times, these mathematical models are derived from engineering and science
principles (Newton’s 2nd Law for instance), while at other times the models may be
obtained from experimental data.
• Mathematical models generally result in need of using mathematical procedures that
include but are not limited to

(A) differentiation,
(B) nonlinear equations,
(C) simultaneous linear equations,
(D) curve fitting by interpolation or regression,
(E) integration,
(F) differential equations.
(G) Optimization

These mathematical procedures may be suitable to be solved exactly as you must have
experienced in the series of calculus courses you have taken, but in most cases, the
procedures can be solved approximately using numerical methods.

Since there is an approximation, errors always arise during calculations

1
Chapter 2: Error Analysis
– Significant Figures
– Accuracy & Precision
– Error Definitions
– Round-off Errors
– Truncation Errors
– Total Numerical Error
– Error Propagation
2
 Significant Figures
 The significant figures/digits carry meaning to the precision of a measurement.
 Consider three different measurements of the length of the table:
L1=3.2 m L2: 3.27 m L3: 3.270 m
 Number of significant figures for L1 is two, for L2 is three, and for L3 is four.
 First digit is the most significant digit, and the last digit is the least significant
digit in the measurement.
 There is always an error associated with each measurement:
L1=3.2 ± 0.2 m L2: 3.27 m ± 0.01 m L3: 3.270 ± 0.003 m
 Any digit beyond the error carrying digits is meaningless.
 Leading zeros are not significant. They are only used to show the location of
the decimal point (e. g., 0.00052 has only two significant digits). To avoid
confusion, use scientific notation (e.g., 5.2 x 10-4).
3
 Accuracy & Precision
 Accuracy refers to how closely a
computed or measured value
agrees with the true value.
 Inaccuracy (also called bias) is a
systematic deviation from the
truth.
 Precision refers to how closely
individual computed or measured
values agree with each other.
 Imprecision (also called
uncertainty) refers to the
accuracy and precision are
magnitude of the scatter. independent from each other.
4
 Error Definitions
In numerical methods both accuracy and precision is required for
a particular problem. We will use the collective term error to
represent both inaccuracy and imprecision in our predictions.
Numerical errors arise from the use of approximation to
represent exact mathematic operations or quantities.
Error is the difference between exact and numerical solution.
True error:
The relationship between exact and approximate values :

True value = approximation + error

or

Et = true value – approximation

5
  Here, error includes all factors contributing to the error. We used
the subscript ‘t’ to designate that this is the tRUE ERROR.
 To take into account different magnitudes in different
measurements, we prefer to normalize the error. Then, we define
the fractional relative (true)error:

fractional relative error = (true value-approximation)/(true value)

or the percent relative (true)error:

t = (true value-approximation)/(true value) x 100

Statement of “error” usually refers to “percent relative error”

6
So, we define true error as : what if we
don’t know
(true value  approximated value) the true
t  100% value?
(true value)

Approximate error:
 In most cases we don’t have the knowledge of the “true value”,
so we define approximate error
(approximate error )
a  100%
(approximate value)

 Approximate error can be defined in different ways depending

on the problem. For example, in iterative problems, error is
defined with respect to the previous calculation.
(current approx.  previous approx.)
a  100%
(current approx.)
7
 Round-off Errors
 Round-off errors result from the omission of the significant figures.
Base-10 (decimal) versus Base-2 (binary) system:
Base-10 Base-2
103 102 101 100 23 22 21 20
a b c d a b a b Positional
Notation

= ax103 + bx102 + cx101 + dx100 = ax23 + bx22 + ax21 + bx20

 Computers knows only two numbers (on/off states). So

computers can only store numbers in binary (base-2) system.
e.g. 100101 a bit (binary digit).
1 byte= 8 bits
computer uses 6 bits to store this number.
8
The Patriot missile defense system used during the Gulf War was also
rendered ineffective due to roundoff error (Skeel 1992, U.S. GAO 1992).
The system used an integer timing register which was incremented at
intervals of 0.1 s. However, the integers were converted to decimal
numbers by multiplying by the binary approximation of 0.1,

causing 9.537x10^-8 sec difference

As a result, after 100 hours ( ticks), an error of

had accumulated. And caused an error in determining the position of the

Scud as 687m. For the Patriot missile defense system, the target is
considered out of range if the shift was going to more than 137m. This
discrepancy caused the Patriot system to continuously recycle itself
instead of targeting properly. As a result, an Iraqi Scud missile could not
be targeted and was allowed to detonate on a barracks, killing 28 people.
[Link] 9
 Binary Representation of Numbers

How a Decimal Number is Represented

257.76  2 10 2  5 101  7 100  7 10 1  6 10 2

Base 2
 (1 23  0  22  1 21  1 20 ) 
(1011.0011) 2   1 2 3 4 

  ( 0  2  0  2  1  2  1  2 ) 10
 11.1875
10
Convert Base 10 Integer to binary representation

Table 1 Converting a base-10 integer to binary representation.

Quotient Remainder
11/2 5 1  a0
5/2 2 1  a1
2/2 1 0  a2
1/2 0 1  a3
Hence
(11)10  (a3 a 2 a1a0 ) 2
 (1011) 2

11
Fractional Decimal Number to Binary

Table 2. Converting a base-10 fraction to binary representation.

Number Number after Number before

decimal decimal
0.1875 2 0.375 0.375 0  a1
0.375  2 0.75 0.75 0  a2
0.75  2 1.5 0.5 1  a 3
0.5  2 1.0 0.0 1  a4

Hence

(0.1875)10  (0.a1a 2 a3a 4 ) 2

 (0.0011) 2

12
 Decimal Number to Binary

11.187510   ?.? 2
Since
(11)10  (1011) 2
and
(0.1875)10  (0.0011) 2

we have
(11.1875)10  (1011.0011) 2

13
 All Fractional Decimal Numbers Cannot be
Represented Exactly

(0.3)10  (??????)2
Table 3. Converting a base-10 fraction to approximate binary representation.
Number Number
Number after before
decimal Decimal
0.3 2 0.6 0.6 0  a1
0.6  2 1.2 0.2 1  a2
0.2  2 0.4 0.4 0  a 3
0.4  2 0.8 0.8 0  a4
0.8  2 1.6 0.6 1  a 5

(0.3)10  (a1a2 a3a4 a5 ) 2  (0.01001) 2  0.28125

14
Integer Representation:
 First bit is used to store the sign (0 for “+” and 1 for “-”);
remaining bits are used to store the number.

 In integer representation, numbers can be defined exactly but

only a limited range of numbers are allowed in a limited
memory. Also fractional quantities can not be represented.

Homework: Find the range of numbers (negative and positive) that

you can store in a 16-bit computer using integer representation. (-
32767 to 32767). 15
Floating Point Representation (FPR):
 FPR allows a much wider range of numbers than integer
representation.
 similar to the scientific notation.
 allows storing fractional quantities.

Mantissa (significand) exponent

mb e
base
a word
e.g. 0.015678  1.5678102 (base-10)

16
IEEE floating point representation standards:
32-bit (single precision) word format:

64-bit (double precision) word format:

 Mantissa takes only a limited number of significant digits 

round-off error
 increasing the number of digits (32-bit versus 64-bit) decreases
the roundoff error.
17
Range:
In FPR there is still a limit for the representation numbers but the
range is much bigger.
In 64-bit representiaton in IEEE format:
52 digits 11 digits
Max value= +1.111…1111 x 2 +(1111111111) = 1.7977 x 10+308
Min value= 1.000…0000 x 2 -(1111111111) = 2.2251 x 10 -308
Numbers larger than the max. value cannot be represented by
the computer  overflow error.
>> realmax
ans=
Any value bigger than this is set to infinity
1.7976931e+308

 Numbers smaller than the min. value cannot be represented.

There is a “hole” at zero.  underflow error.
>> realmin
ans= Any value smalller than this is set to zero
2.22500738e-308 18
Precision:
When 52 bits used for the mantissa in Double precision; This
corresponds to about 15-16 base-10 significant units.

>> pi
ans= 32-bit representation (single precision)
3.1416
>> formatlong
>> pi
ans= 64-bit representation (double precision)
3.141592653589793

Example: Assume you have a machine stores floating-point numbers in a

hypothetical 8-bit binary word (one digit for sign, four for exponent with sign,
three for mantissa). Represent (-13.9)10 in floating point binary format.

19
Quantization error : Chopping and Rounding:
Assume a computer that can store 7 significant digits:

Chopping Rounding

error error
4.2428576428...... 4.2428576428......
4.242857 4.242858

Rounding is a better choice since the sign of error can be either

positive and negative leading to smaller total numerical error.
Whereas error in chopping is always positive and adds up.
Rounding costs an extra processing to the computer, so most
computers just chops off the number.
Error associated with rounding/chopping  Quantization error
20
Machine epsilon:
As a result of quantization of numbers, there is a finite length of
interval between two numbers in floating point representation.
x

Machine epsilon (or machine precision) is the upper bound on

the relative error due to chopping/rounding in floating point
arithmetic.
x
 e.g., for a 64-bit
x representation, b=2, t=53
The machine epsilon can be computed by   =2-52
=2.22044.. x 10-16
  b1t >> eps
ans=
b=number base 2.2204460e-16
t= number of digits in mantissa 21
Finding Machine epsilon in Matlab:
epsilon = 1.0;
while (1.0 + epsilon) ~= 1.0
epsilon = epsilon/2
end
machineepsilon= epsilon*2

epsilon =
1.0
…
epsilon =
4.440892098500626e-016
epsilon =
2.220446049250313e-016
epsilon =
1.110223024625157e-016
machineepsilon =
2.220446049250313e-016

22
Arithmetic operations:
Besides the limitations of the computer for storage of numbers,
arithmetic operations of these numbers also contribute to the
round-off error.
Consider a hypotetical base-10 computer with 4-digit mantissa & 1-digit exponent:

1.345 + 0.03406 = 0.1345 x 101 + 0.003406 x 101 = 0.137906 x 101

chopped-off
during arithmetic operations, numbers are converted to be same exponents

Ex: a) Evaluate the polynomial

y  x 3  5 x 2  6 x  0.55
for x=2.73. Use 3-digit arithmetic with chopping. Evaluate the error.
b) If the function is expressed as
y  x( x  5)  6x  0.55
what is the percent relative error? Compare with part a. 23
 Truncation Errors
 Truncation errors result from using an approximations in place of exact
mathematical representations. truncation error is the error made
by truncating an infinite sum and approximating it by a finite sum. An example
is the approximation we used in the falling object in air problem:

 Truncation error was introduced here because

dv v v(ti 1 )  v(ti )
  the difference equation only approximates the
dt t ti 1  ti true value of the derivative

25
Taylor’s Theorem:
Taylor’s theorem states that if the function f and its (n+1)
drivatives are continous on an interval containing a and x, then
the values of the function at x is given by

f ( 2) (a) f ( n ) (a)
f ( x)  f (a)  f (a)( x  a) 
'
( x  a)  ... 
2
( x  a) n  Rn
th
2! n!
0 order 1st order term 2nd order term nth order term nth order
term reminder

Given the value of the function at a single point

(P1), and the value of all (first, second, and so on)
its derivatives at that single point, then one can
find the value of the function at any other point
(P1+h)
26
 exact solution
In other words, any smooth
function can be
approximated as a
polynomial of order n within
a given interval.
The error gets smaller as n
increases.

base point(a)=1

Use second order Taylor series expansion to approximate the

function
f ( x)  0.1x 4  0.15 x 3  0.5 x 2  0.25x  1.2

at x=1 around a=0. Calculate the truncation error from this

approximation.
27
Error in Taylor’s approximation:
Taylor’s theorem gives insight for estimating the truncation error
in the numerical approximation.
Suppose for base point f(xi ), we want to evaulate f(xi+1):
2 n
h:step size
f ( xi ) 2 f ( xi ) n
f ( xi 1 )  f ( xi )  f ( xi )h 
'
h  ...  h  Rn h  ( xi 1  xi )
2! n!
f n1 ( ) n1
  xi , xi 1 
truncation
where Rn  h
(n  1)! error

Here, Rn gives the exact determinaton of the error from the n-th
order approximation of the function.
We can estimate the order of the magnitude of the error in
terms of step size (h):
Step size (h) controls the magnitude
Rn  O(h n 1 ) of the error in the approximation!

28
Example: Finite difference approximation:
We can evaluate the truncation error for using finite difference
approximation in the “falling object in air” problem. Express
v(ti+1 ) in Taylor series:
v '' (ti ) 2 v n (ti ) n h  (ti 1  ti )
v(ti 1 )  v(ti )  v (ti )h 
'
h  ...  h  Rn
2! n!
Taylor series to n=1:
v(ti 1 )  v(ti )  v ' (ti )h  R1 R1  O(h 2 )
or
v(ti 1 )  v(ti ) R1
v ' (ti )  
h h
Truncation Error
finite difference app.

R1 O(h 2 ) Then the error associated with

Truncation error   O ( h) finite difference approximation
h h is in the order of h.
29
 Total Numerical Error
Total
Numerical = Round-off + Truncation
Error Error Error

Round-off errors can be minimized by increasing the number of

significant digits, by staying away from subtractive cancellations,
and by decreasing the number of computations.
Truncation errors can be reduced by decreasing the step size (h).
But, this may result in subtractive cancellation too!
 So, there is a trade-off between
truncation error and round-off error in
terms of step size (h).
Finally, remember that there is no single
systematic approach for evaluating numerical
error for different problems. We do it as much
as we can. 30
 Error Propagation
Error propagation concerns how an error on x is propagated to
the function f(x).

x=true v.
x  x  x  propagation f ( x  )  f ( x)  f ( x  )
xo= approx. v.
of error

Taylor expansion can be used to estimate the propagation of error.

Evaluate f(x) near f(xo):
f ( x)  f ( x  )  f ' ( x o )x  ...
expansion to
f ( x)  f ( x)  f ( x  )  f ' ( x o )x
1st order term

f ( x  )  f ' ( x o ) x

Given a measured value of x= 2.50.01, estimate the resulting

error in the function f(x)=x3. 31
Functions of more than one variable:

x  x  x  y  y  y  f ( x  , y o , z o ,..)
Propagation of
x  z  z  ...
error
error out
error in

Error propagation for functions of more than one variable can be

understood as the generalization of the case of functions with a
single variable:
f 2 f f
f ( x  , y o , z o ,..)  ( ) (x) 2  ( ) 2 (y ) 2  ( ) 2 (z ) 2  ...
x y z
Open channel flow formula for a rectangular channel is given by :

1 (bh)5 / 3 (Q=flow rate , n=roughness coff.,)

Q s (b=width, h=depth, s=slope)
n (b  2h) 2/3

Assume that b=20 m and h=0.3 m for the channel. If you know that
n=0.030±0.003 and s=0.0003±0.00003, what is the resulting error in calculation
of Q? 32
Condition and stability:
In mathematical computation, condition is the sensitivity of the
outputs to the input values. It is a measure of how much an
uncertainty is magnified by the computation.

Condition Error in the output

= used error
Number Error in the input propagation
formula
f ( x)  f ( x o ) f ' ( x o )( x  x o )
f ( xo ) f ( xo ) xo f ' ( xo )
C.N .   
(x  x )o
(x  x ) o
f ( xo )
xo xo
 For large values (>> 1) of C.N., a small uncertainty in the input
results in gross changes in the output, then we say “problem is
unstable” or “ill-conditioned”.

C.N.1 C.N.>>1 (ill-conditioned)

33