Complutense University of Madrid

———————————————————————
Scientific Computing Laboratory

Juan Jiménez
Héctor Garcı́a de Marina
Lı́a Garcı́a

August 2, 2025
2

El contenido de estos apuntes está bajo licencia Creative Commons Atribution-ShareAlike 4.0
[Link]
©Juan Jiménez DOG
Contents

1 Introduction to scientific software 15

1.1 Introduction to computers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
1.1.1 Computer description levels. . . . . . . . . . . . . . . . . . . . . . . . . . . 17
1.1.2 The Von Neumann’s computer model . . . . . . . . . . . . . . . . . . . . . 18
1.1.3 Binary coding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.2 Scientific software application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2 Introduction to Python Programming 25

2.1 A python’s development environment . . . . . . . . . . . . . . . . . . . . . . . . . . 25
2.1.1 Anaconda. A free software computing tools distribution . . . . . . . . . . . 26
2.1.2 Spyder . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.1.3 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
2.2 Arithmetical, relational and logical operations . . . . . . . . . . . . . . . . . . . . . 37
2.2.1 Arithmetic operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
2.2.2 Arithmetic operator precedence . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.2.3 Relational and logical operations. . . . . . . . . . . . . . . . . . . . . . . . . 40
2.3 Scripts in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.3.1 The Spyder’s text editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
2.3.2 Running a script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.4 Functions in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
2.5 Namespace and scope in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
2.6 Debugging . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
2.7 Flow control . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.7.1 Conditional flow. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.7.2 Loops . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.7.3 Recursive functions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.7.4 Algorithms and flow charts. . . . . . . . . . . . . . . . . . . . . . . . . . . . 72
2.8 Reading and writing data in Pyhton . . . . . . . . . . . . . . . . . . . . . . . . . . 76

3 Introduction to Numpy 81
3.1 Numpy: a Python library for scientific computing . . . . . . . . . . . . . . . . . . . 81
3.1.1 Vectors and matrices in Numpy. . . . . . . . . . . . . . . . . . . . . . . . . 83
3.2 Matrix Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.2.1 Functions included in Numpy . . . . . . . . . . . . . . . . . . . . . . . . . . 106
3.3 Vectorial operators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107
3.4 Kinds of matrices frequently used. . . . . . . . . . . . . . . . . . . . . . . . . . . . 117
3.5 Matrix factorization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120
3.5.1 LU factorization. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

3
4 CONTENTS

3.5.2 Cholesky factorisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

3.5.3 Diagonalisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
3.5.4 QR Factorisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
3.5.5 SVD Factorisation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

4 Introduction to matplotlib 133

4.1 2D Plotting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134
4.1.1 Working with multiple figures . . . . . . . . . . . . . . . . . . . . . . . . . . 136
4.1.2 Nonlinear axis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.1.3 Polar coordinates graph . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
4.1.4 Other graphical representations . . . . . . . . . . . . . . . . . . . . . . . . . 140
4.2 3D plots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.2.1 Surfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
4.3 Animations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146

5 Computer Arithmetics and Errors 149

5.1 Binary and denary representations . . . . . . . . . . . . . . . . . . . . . . . . . . . 149
5.2 Computer number representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150
5.2.1 Non-integer numbers: fixed-point and floating-point representations. . . . . 152
5.3 Numerical representations Errors. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5.3.1 round-off error . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 160
5.3.2 Underflow and Overflow errors . . . . . . . . . . . . . . . . . . . . . . . . . 163
5.4 Errors derived from arithmetic operation . . . . . . . . . . . . . . . . . . . . . . . . 163
5.4.1 Accumulation of round-off errors. . . . . . . . . . . . . . . . . . . . . . . . . 164
5.4.2 Catastrophic cancellation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 167
5.4.3 Overflow and underflow errors . . . . . . . . . . . . . . . . . . . . . . . . . 168

6 Methods for calculating the roots of a function 171

6.1 Roots of a function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
6.2 Local iterative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.2.1 Bisection Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.2.2 False position method or (Regula falsi ) . . . . . . . . . . . . . . . . . . . . . 174
6.2.3 Newton-Raphson method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.2.4 Secant method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180
6.2.5 Method of successive approximations or fixed point iteration . . . . . . . . 183
6.3 Python root finding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
6.3.1 Function fsolve . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
6.3.2 Polynomials roots . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193
6.4 Computing roots at maximum precision. . . . . . . . . . . . . . . . . . . . . . . . . 196

7 Linear equation systems 199

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199
7.2 Condition number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204
7.2.1 Direct methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.2.2 Triangular systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 205
7.2.3 Methods based on factorisations . . . . . . . . . . . . . . . . . . . . . . . . 207
7.2.4 The Gaussian elimination method. . . . . . . . . . . . . . . . . . . . . . . . 213
7.2.5 Gauss-Jordan and matrices in stepwise reduced form . . . . . . . . . . . . . 219
7.3 Iterative methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221
7.3.1 Jacobi’s method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222
CONTENTS 5

7.3.2 The Gauss-Seidel method . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

7.3.3 Weighted methods. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230
7.3.4 Convergence analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 233

8 Interpolation and function fitting 237

8.1 The Taylor’s polynomial. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238
8.2 Polynomial interpolation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 240
8.2.1 The Vandermonde’s matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
8.2.2 Lagrange Interpolating Polynomial . . . . . . . . . . . . . . . . . . . . . . . 243
8.2.3 Divided diferences. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 245
8.2.4 The Newton-Gregory polynomial . . . . . . . . . . . . . . . . . . . . . . . . 248
8.3 Piecewise interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 251
8.3.1 Cubic spline interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252
8.3.2 Python functions for piecewise interpolation . . . . . . . . . . . . . . . . . . 257
8.4 Least squared error method for polynomial data fitting . . . . . . . . . . . . . . . . 259
8.4.1 Least squares in Python. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
8.4.2 Analyzing goodness of fit using least squares method. . . . . . . . . . . . . 266
8.5 Bézier Curves . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 266

9 Numerical differentiation and integration 277

9.1 Numerical differentiation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
9.1.1 Numerical differentiation base on the interpolation polynomial. . . . . . . . 278
9.1.2 Numerical differentiation base on finite differences . . . . . . . . . . . . . . 279
9.2 Numerical Integration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
9.2.1 Trapezium formula . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
9.2.2 Simpson’s formulae . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
9.3 Differential equations. Initial value problems . . . . . . . . . . . . . . . . . . . . . 290
9.3.1 The Euler’s method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
9.3.2 The Runge-Kutta methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 297
6 CONTENTS
List of Figures

1.1 Computer level description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

1.2 Von Neumann’s model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.1 Anaconda-Navigator window, Spyder corresponding icon is shown in a red circle . 26

2.2 Anaconda-Navigator window, Environment tab is shown in red . . . . . . . . . . . 27
2.3 Anaconda-Navigator window, Environments tab . . . . . . . . . . . . . . . . . . . . 27
2.4 Anaconda-Navigator window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
2.5 Anaconda-Navigator window. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
2.6 Spyder: An integer development environment for Python . . . . . . . . . . . . . . . 30
2.7 Data types in Python . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
2.8 Text editor for Python included in Spyder . . . . . . . . . . . . . . . . . . . . . . . 45
2.9 Inclusion relationship among Python Namespaces . . . . . . . . . . . . . . . . . . . 53
2.10 A sintax error example (a closing bracket has been omited . . . . . . . . . . . . . . 55
2.11 Active Breackpoint pointed by a blue arrow and debug drop-down menu . . . . . . 56
2.12 program stop on a breakpoint and debugging tools . . . . . . . . . . . . . . . . . . 57
2.13 General outline of an if conditional flow structure. Terms enclosed in parenthesis
are optional . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.14 for structure General outline. Terms enclosed in parentheses are optional. . . . . . 64
2.15 General structure for a while loop. Terms enclosed in parentheses are optional. . . 69
2.16 Symbols used in flowcharts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
2.17 Flow chart for the prime numbers problem . . . . . . . . . . . . . . . . . . . . . . . 74

3.1 Graphic of vectors on the plane . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

3.2 Graphic of vector in the 3D space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83
3.3 Geometrical interpretation of the norm of a vector. . . . . . . . . . . . . . . . . . . 97
3.4 effect of the product of a vector by a scalar . . . . . . . . . . . . . . . . . . . . . . 108
3.5 A graphic representations of Vectors a = (1, 2), b = (−1, 1) and some vectors ob-
tained from a lineal combination of a and b. . . . . . . . . . . . . . . . . . . . . . . 109
3.6 Vectors a = (1, −2, 1), b = (2, 0 − 1)), c = (−1, 1, 1) and vector a − b + c; a graphic
representation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109
3.7 Graphic representation of vector a, in the canonical basis C and in B basis. . . . . 111
3.8 Linear transformation of vector = [1, 1]. D, factor 1.5/0.5 dilation/contraction. Ry
reflection on y-axis. Rθ θ = π/6 and θ = π/3 angles rotations around the origin. . 114
3.9 Quadratic forms obtained using the four diagonal matrices: |a11 | = |a22 | = 1,
a12 = a21 = 0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

4.1 Simple pyplot example. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

4.2 Different uses of plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

7
8 LIST OF FIGURES

4.3 Two graphs in one figure using subplot . . . . . . . . . . . . . . . . . . . . . . . . 137

4.4 Figure with text and labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.5 Graph of the function y = ex with axes on linear and logarithmic scale . . . . . . . 139
4.6 Spiral plot using polar coordinates . . . . . . . . . . . . . . . . . . . . . . . . . . . 140
4.7 Histogram of the number of cars for each 1000 people . . . . . . . . . . . . . . . . 142
4.8 3D plot using scatter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
4.9 Flat grid . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.10 Comparison between wireframe and surface . . . . . . . . . . . . . . . . . . . . . 146

5.1 Location of a non-machine number in relation to its rounding to a machine number

through truncation (xT ) and excess (xE ). When we round the number to the nearest
of the two, the error is always less than half the interval xE − xT . . . . . . . . . . . 161
5.2 Graphic illustration of precision change with the magnitude of represented numbers 163
5.3 Representable numbers and over/under-flows in the IEEE 754 simple precision stan-
dard. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163

6.1 An Example of Kepler’s equation is taking a = 40 and b = 2. . . . . . . . . . . . . 172

6.2 Bolzano’s theorem. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 173
6.3 Flowchart of the bisection method . . . . . . . . . . . . . . . . . . . . . . . . . . . 174
6.4 process of obtaining the root of a function by the bisection method. . . . . . . . . 175
6.5 Obtaining the line joining the extremes of an interval [a, b] containing a root of the
function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 176
6.6 Flowchart of false position method . . . . . . . . . . . . . . . . . . . . . . . . . . . 177
6.7 Process of obtaining the root of a function by the method of linear interpolation . 178
6.8 Tangent line to the function f (x) at the point x0 . . . . . . . . . . . . . . . . . . . . 179
6.9 Flowchart of Newton-Raphson method . . . . . . . . . . . . . . . . . . . . . . . . . 180
6.10 Process of obtaining the root of a function by Newton’s method . . . . . . . . . . . 181
6.11 Secant line to the function f (x) at points x0 and x1 . . . . . . . . . . . . . . . . . . 182
6.12 Flowchart of the secant method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183
6.13 process of obtaining the root of a function by the secant method √ . . . . . . . . . . 184
6.14 Obtaining the fixed point of the function graphically, g(x) = − ex . . . . . . . . . 185
6.15 Flowchart√of fixed point iteration. Root corresponds to f (x) = g(x) − x . . . . . . 187
6.16 g(x) = ± ex , only negative branch has a fixed point. . . . . . . . . . . . . . . . . 188
x 2
6.17 Fixed point√ iteration to compute the root of f (x) = e − x using the function
x
g(x) = − e . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 189
6.18 first iterations of obtaining the root of the function f (x) = ex − x2 by applying the
fixed point method on the function g(x) = ln(x2 ). . . . . . . . . . . . . . . . . . . 190
6.19 Fixed point iteration to compute the root of function f (x) = ex − x2 using the
function g(x) = ln(x2 ), iteration oscillates without converging to the solution. . . . 191
6.20 fixed point iteration to compute the root of function f (x) = ex − x2 using the
x
function g(x) = ex , the iteration diverges quickly. . . . . . . . . . . . . . . . . . . . 192
6.21 Plotting a polynomial using linspace . . . . . . . . . . . . . . . . . . . . . . . . . 195
6.22 Flow chart of bisection method with maximal precission . . . . . . . . . . . . . . . 197

7.1 Equation system with unique solution . . . . . . . . . . . . . . . . . . . . . . . . . 201

7.2 Equation system without solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202
7.3 Equation system with infinite solutions . . . . . . . . . . . . . . . . . . . . . . . . . 203
7.4 General flowchart of iterative methods for solving systems of equations. The function
M (x) is the one that specifies the method in each case. . . . . . . . . . . . . . . . 221
LIST OF FIGURES 9

7.5 evolution of the tolerance (modulus of the difference between two successive solu-
tions) for the same system solved by the Gauss-Seidel method and the Jacobi method230
7.6 Tolerance evolution for the same system using the Jacobi method (diverges) and the
weighted Jacobi method (converges). . . . . . . . . . . . . . . . . . . . . . . . . . . 236

8.1 A comparison among the results achieved using Taylor polynomials to approach the
logarithm. (2, 3, 5, 10, 20 degrees) . . . . . . . . . . . . . . . . . . . . . . . . . . . 241
8.2 Taylor polynomial for cosine and sine functions . . . . . . . . . . . . . . . . . . . . 242
8.3 Nine degree interpoling polynomial obtained using a set of ten data . . . . . . . . . 250
8.4 Zero-order and linear interpolation for the figure data 8.3 . . . . . . . . . . . . . . 251
8.5 Cubic Spline interpolation for the data represented in the figure . . . . . . . . . . . 258
8.6 0-degree least squared error polynomial . . . . . . . . . . . . . . . . . . . . . . . . 261
8.7 An example of least squares polynomial fit, using the function fit from Polynomial
and our function lse. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265
8.8 Comparison among the residuals resulting from a dataset least squares fitting, using
polynomials of degrees 1 to 4. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
8.9 Bézier curves traced between points P0 = (0, 0) y Pn = (2, 1), variating the control
points number and positions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 269
8.10 Equivalent curves, built from a three control point Bézier’s curve. . . . . . . . . . . 274
8.11 Bézier’s curve and its derivative with respect to the Berstein’s polynomial parameter
which defines the curve t ∈ [0, 1] . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
8.12 interpolating three point using two Bezier’s curves . . . . . . . . . . . . . . . . . . 275

9.1 Function derivative approximation error, using a a two-points differentiation for-

mula. Dependency on the difference h value . . . . . . . . . . . . . . . . . . . . . . 280
9.2 Comparison between the approximations achieved for a function derivative using
two point forward difference and two point central difference. . . . . . . . . . . . . 281
9.3 Graphical interpretation of trapezium rule. . . . . . . . . . . . . . . . . . . . . . . 284
9.4 Extended trapezium formula graphical interpretation . . . . . . . . . . . . . . . . 285
9.5 Graphical interpretation of Simpson’s 1/3 formula . . . . . . . . . . . . . . . . . . 286
9.6 Graphical interpretation of Simpson’s 3/8 formula . . . . . . . . . . . . . . . . . . 287
9.7 Circuito RC . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293
9.8 Comparison among the results obtained using Euler’s method with two different
integration steps h = 0.5 y h = 0.25, and the analytic solution for voltage Vo across
a capacitor. We took C = R = 1 and Vi = 10. . . . . . . . . . . . . . . . . . . . . . 295
9.9 A mass vertically hanging of a sprint with a damping resistance . . . . . . . . . . . 296
9.10 Results achieved using Euler’s method to solve the damped oscillator problem. . . 297
9.11 Result achieved using Euler’s method to solve the damped oscillator problem. K = 100298
9.12 Result achieved using Euler’s method to solve the damped oscillator problem. K =
100, h = 0.001 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
10 LIST OF FIGURES
List of Tables

2.1 Some methods of Python’s List . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.2 Arithmetic operators defined in python . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3 Relational operators defined in Python . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.4 Logical operators on values and variables . . . . . . . . . . . . . . . . . . . . . . . 41

3.1 frecuently used mathematical functions included in Numpy . . . . . . . . . . . . . 106

4.1 Point colors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

4.2 Some point markers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
4.3 Line types . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

5.1 2’s complement representation for a 4-bit register . . . . . . . . . . . . . . . . . . . 151

5.2 127-biased representation for a 8-bit exponent . . . . . . . . . . . . . . . . . . . . . 156
5.3 Comparison between the IEEE standards for floating point representation. (bs sign
bit, mi mantissa bit, ei exponent bit) . . . . . . . . . . . . . . . . . . . . . . . . . . 159

8.1 f (x) = erf (x) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

8.2 four data divided difference table . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246
8.3 Table of differences for a Newton-Gregory polynomial of four data . . . . . . . . . 249

9.1 Derivative formulae based on finite differences . . . . . . . . . . . . . . . . . . . . . 282

11
12 LIST OF TABLES
Preface

’Fair Lady!’ Said Frodo again

after e while. ’Tell me, if my
asking does not seem foolish, who
is Tom Bombadil?’
‘He is’ said Goldberry, staying her
swift movements and smiling.
(...) ‘he is as you have seen him’.
He is the Master of wood, water
and hill.
’Then all this strange land
belongs to him?’
‘No indeed!’ she answered, and
her smile faded. ‘That would
indeed be a burden,’ she added in
a low voice, as if to herself. ‘The
trees and the grasses and all
things growing or living in the
land belong each to themselves.
Tom Bombadil is the Master.

J.R.R Tolkien, The Lord of the

Rings.

These lecture notes cover the contents of the scientific computing lab: a first course in scientific
computing teaching during the first semester of the degree in physics. The aim is to introduce the
student to computer programming and numerical calculus, which are invaluable tools in scientific
research.
Chapters one to four present an introduction to Python programming, including some of the
most common used libraries for scientific computing. The remaining chapters describe classical
techniques to solve numerical problems.
Paragraphs enclosed in ’witches’,
 
paragraph ,

contain advanced material. They are optional for those interested in deepening their knowledge.
Almost every method described in these notes was developed, centuries ago, by the big ones:
Newton, Gauss, Lagrange, etc. But they are methods that are still usefull and, with the comming
up of digital computers, they are more interesting than ever. We can indeed quote the famous
sentence from The scholar Bernardo de Chartres:
“We are like dwarfs sitting on the shoulders of giants. We see more, and things that

13
14 LIST OF TABLES

are more distant, than they did, not because our sight is superior or because we are
taller than they, but because they raise us up, and by their great stature add to ours.”

The contents, examples, code, etc. of this notes, are the results of the effort of many people.
First, I would like to mention Manuel Prieto and Segundo Esteban, which prepared the slides for
Introducción al Cálculo Cientı́fico y Programación, the predecessor of the Scientific Computing
Laboratory in the old degree of Physics.
Second, I also want to thanks to my colleges, who have taught the subject during these years:
Rosa González Barras, Belén Rodrı́guez Fonseca, Maurizio Matessini, Pablo Zurita, Vicente
Carlos Ruı́z Martı́nez, Encarna Serrano, Carlos Garcı́a Sánchez, Jose Antonio Martı́n, Victo-
ria López López, Alberto del Barrio, Blanca Ayarzagüena, Javier Gómez Selles, Nacho Gómez
Pérez, Marta Ávalos, Iñaqui Hidalgo, Daviz sánchez, Juan Rodriguez, Marı́a Ramirez, Álvaro de
la Cámara, Marta Martı́n (I hope don’t forget anybody). Thank you very much for sharing so
many work hours.

Juan Jiménez, Lia Garcı́a y Héctor Garcı́a de Marina.

Chapter 1

Introduction to scientific software

”Begin at the beginning”, the

King said, gravely, ”and go on till
you came to the end; then stop”

Lewis Carroll, Alice in

Wonderland

Computers have become an essential tool in the daily work of every Scientific researcher. They
are used for doing tasks that could not be carried out without their help. We can point out the
following tasks, among others:
• Data acquisition from experimental devices.
• Experimental data analysis and processing.
• Scientific computing.
The first of these tasks is beyond the scope of this course. It aims to use the computer to get
data automatically from the sensors attached to an experimental device. Usually, data supplied
by a sensor (a thermometer, a manometer, a flow meter, an optical Camera, etc.) are converted
to voltages by some electronic device. Then, the voltages are digitalized —i.e., converted to a
sequence of zeros and ones— and stored in a computer for later analysis. Also, they can be
shown (monitored) on a computer screen. In many cases, the computer can also interact with the
experimental device: start or stop an experiment, control the experimental conditions, trigger an
alarm in case of error, etc.
In this way, the scientific researcher is released from getting the experimental data by himself.
A task that could sometimes be impossible to do. For instance, when he needs to measure many
variables simultaneously or when the measurements must be taken quickly. Besides, it is easy to
make mistakes when the measurements are manually taken.
Experimental data analysis and processing are fundamental tasks in scientific work. The com-
puters carry out such tasks efficiently and safely, working with data amounts that were impossible
to deal with fifty years ago. As we shall see later, a simple data sheet can save us many hours of
tedious calculations. Statistical analysis of experimental data, Estimation of experimental errors,
and regression of data to a mathematical function allows us to establish a law or at least find a
relationship among the data. All these are part of researchers’ daily work, virtually in any field of
science.
Lastly, computing . It can be said that scientific computing is the kernel of scientific work.
The researcher tries to explain the real world using a mathematical description. Such a description

15
16 CHAPTER 1. INTRO TO SCIENTIFIC SW.

usually takes the form of a mathematical model, which can be more or less complex. A model is
valid because it can reproduce the same results as the original experiment, which the model tries
to explain. If the model is good enough, it can be obtained by computing similar results from the
experiment. Then, the model becomes tested and can be used to forecast the system’s behaviour
under study in many different conditions.

1.1 Introduction to computers.

Everybody is somehow familiar with computers. We find them daily, and we depend on them in
such a way that our current lives would be unimaginable without them. In general, a computer
can be defined as a machine able to get instructions and data and use the instructions to perform
calculations with the data. The capacity to get instruction makes the computer a versatile tool;
according to the instruction received and the specific machine capacity, the computer can carry
out very different tasks. Among them, we can highlight the following:

• data processing

• data storing

• data transfer between the computer and the outside.

• Control of these operations just listed.

The computer is designed to perform general functions which are specified when the computer
is programmed. Programming is the way to define the tasks the computer will carry out.
Figure 1.1 shows a general computer layout described by levels. Each level lays and assumes
the previous level.

Aplication

High level languaje

operating System/Compiler

Computer Architecture

Hardware

Digital Circuit

Physical Level

Figure 1.1: Computer level description

1.1. INTRO TO COMPUTERS 17

1.1.1 Computer description levels.

1. Physical Level. It’s the ground of the computer hardware. Basic electronic components,
diodes, transistors, resistances, etc, make it up. In a modern computer, it is impossible to
split or even to watch such components: they are built directly in a semiconductor Crystal
and part of an electronic device Known as integrated circuit.
2. Digital Circuit. Physical level components are grouped, forming digital circuits. (In our
case, Integrated digital circuits). They work with only two voltage levels (V1 , V0 ). This allows
us to use them for defining logical relationships: V1 =true, V2 =false. This logical relationship
obtained using two voltage levels of digital circuits is, in turn, the basis for every computing
the computer can carry out.
3. Computer Hardware organisation. Digital circuits are grouped and organized to build
up the computer Hardware. Basic hardware modules are the central processing unit(CPU),
the memory unit and the input and output units. These components are connected by a bus,
which transfers data from one unit to another.
4. Computer Architecture. The architecture defines how the computer works. Thus, it is
highly related to the hardware organization of the system; bit architecture works at a higher
abstraction level. It defines how to get access to the memory register, arbitrate the use of
the buses with links to the different components and regulates the CPU work.
The primary language the computer works with is established according to its architecture.
This primary language is called Machine language. It still uses binary logical levels (zeros
and ones), so it is unsuitable for being understood by human beings. The computer uses
machine language to perform basic operations such as copying the content from one memory
register to another, adding the contents of two memory registers, etc.
Machine language is suitable for computers but not for human beings. Thus, manufacturers
supply the computer with a basic repertory of instructions that their machine can understand
and carry out, written in a more accessible language. It is known as assembler language. The
commands of this language are accessible to translate to one or several machine language
instructions. Writing code right in assembler language is tedious and prone to mistakes.
5. Compilers y operating Systems Compilers are special programs that allow us to translate
instructions written in a high-level language into machine language. The programmer writes
instructions in plain text, readable for a human being. Then, the compiler translates the
file’s contents into binary sentences that the machine can interpret.
Early computers were only able to run a program at a time. As new and more sophisticated
computers were built, it arises the idea of making programs which were able to take over
the basic tasks: Managing the information flow, dealing with peripheral devices, etc. These
programs are called operating systems. Modern computers load an operating system at the
boot time, which controls the running of the remaining programs. Some examples of oper-
ating systems are DOS (Disk Operating System), UNIX and the UNIX version for personal
computers LINUX.
6. High level languages High-level languages are intended to make the programmer’s work
easier, ignoring the hardware implementation details. They comprise a set of commands and
syntactic rules, allowing the programmer to describe computer instruction in plain text.
Generally, High-level languages can be divided into compiled and interpreted languages.
Compiled languages employ a compiler to traduce the command from the high-level lan-
guage to machine language. Some examples are C, C++ and FORTRAN. On the contrary,
18 CHAPTER 1. INTRO TO SCIENTIFIC SW.

interpreted languages do not translate to machine language. They use a second program
known as the interpreter. While a program in the interpreted language is running, the inter-
preter reads the program commands one by one and translates them to machine language.
Examples of interpreted languages are BASIC, Python or Java.
7. Programmes. A Programme is a piece of code intended to perform specific tasks. They are
available to the final users of the computer. Usually, programs are written using a high-level
language and are user-friendly, i.e. they have an interface that is easy to use.
There are many different kinds of programs, according to their purpose. We can find Internet
web browsers like Google Chrome, Mozilla or Bing, text editors like Word, Emacs or Latex.
E-mail clients (Mail User Agents) like Outlook or Mozilla Thunderbird. The list of available
programs would be endless.

1.1.2 The Von Neumann’s computer model

Modern computers generally follow the model proposed by Von Neumann. Figure 1.2 shows a
schematic view of Von Neumann’s model.

Control Unit (UC)

Prog. Count. (PC)
Stat. Reg (SR)

Arithmetic
Logic
UNIT
Registers (ALU)

Instr. Reg (IR)

Mem. Adr. Reg. (MAR)
Mem. Dat. Reg. (MDR)

CPU
Data Bus
Address Bus
Control Bus

Memory I/O)

Figure 1.2: Von Neumann’s model

Von Neuman’s model has been divided into three basic models and several interconnection
elements. The basic modules are:
• The Central Processing Unity. CPU This Unity is the kernel where the computer per-
forms operations.
1.1. INTRO TO COMPUTERS 19

Inside the CPU, it is possible to difference the following parts:

– The processing unit or data router comprises the Arithmetic Logic Unit (ALU), . The
ALU can perform the arithmetical and logical operations described in the program
instructions. They are built to perform arithmetic between integer numbers and the
basic Boole’s algebra logical operations (AND, OR, etc). Non-integer numbers are usu-
ally represented using a unique format called floating point representation. Operations
between floating point numbers are performed using a specific processor known as a
coprocessor. The processing speed is measured in millions of instructions per second
(MIPS) or millions of floating point operations per second (MFLOPS).
– The register bank: A set of registers for storing the data ALU is working with and the
results of ALU operations.
– The control unit (UC) or control route: It fetches the instructions from the main memory
and stores them in the instruction register. It also decoded the instructions, executed
them using the ALUs, and stored the results in the data register. Once the operation is
finished, the UC stores the conditions derived from the operation in the state register.
The memory data register holds the data read from the main memory or those ready to
be written there. Memory Address register holds the main memory address the ALU
is accessing for writing or reading. The programme counter, or the instruction pointer,
is a special register. It stores the current position at which the CPU is located inside a
program instructions sequence.

• Memory Unit It is the main or primary memory of the computer. It is divided into memory
blocks. Each memory block is identified by its address. The CPU has direct access to memory
blocks.
The elemental unit of digital information is the bit (0,1). Data storing capacity is gauged in
Bytes and multiples of Byte, represented as powers of two:

1 Byte = 8 bits
1 KB = 210 bits = 1024 B
1 M B = 220 bits = 1024 KB
1 GB = 230 bits
1 T B = 240 bits

• Input/Output Unit. This unit transfers information between the computer and the pe-
ripheral devices.

The interconnection elements are called Buses. We can define three: the data bus transfers
data between the CPU and the main memory or the input/output unity. The address bus is used
for transmitting a memory address or an input/output unit. The control bus for sending control
signals, such as the clock signal and the reading/writing control signal, among others.

1.1.3 Binary coding

Let’s see in more detail how a computer represents the information. As described before, the
electronic stuff represents the computer’s physical part. It works with two voltage levels, which
can be associated with two states —high and low— and, in turn, these levels can define two
20 CHAPTER 1. INTRO TO SCIENTIFIC SW.

symbols, 0 and 1. Usually, we employ 10 symbols (digits):

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, i.e. we use a decimal representation. Whenever we want to represent a
number greater than nine, we combine several symbols, gathering quantities in groups of ten
because we have exhausted the ten available digits. So, for instance, the number 16 represents six
units plus a group of ten units, and the number 462 represents two units plus six groups of ten
units, plus four groups of ten groups of 10 units. In mathematics, this is equal to using sums of
products of digits by powers of ten:

13024 = 1 × 104 + 3 × 103 + 0 × 102 + 2 × 101 + 4 × 100

If we look at the digits that compose the number from left to right, each one represents an
upper power of ten, i.e., each represents the number of groups of ten groups of groups ... of ten
groups of unities. This means we can represent amounts as significant as we wish, using only ten
symbols. We are used to this numerical representation, which is known as a decimal representation
or representation in base 10. But it is not the only one.
Returning to the computer’s numerical representation, we have only two digits: 0 and 1. If we
want to define a representation that resembles the base 10 representation, we must now gather the
quantities in groups of two. So, the number two is represented in base 2 as 10. To represent the
number three, we have a group of two plus one unit 11. In mathematics, this is equal to using
sums of products of digits by powers of two;

10110 = 1 × 24 + 0 × 23 + 1 × 22 + 1 × 21 + 0 × 20
This representation is known as binary coding. If we expand the binary encoding of a number
as powers of two, we obtain a direct way to obtain its decimal representation. For instance, If we
take the last example, calculate the power of two and add the results we obtain,

1 × 24 + 0 × 23 + 1 × 22 + 1 × 21 + 0 × 20 = 16 + 0 + 4 + 2 + 0 = 22
which is the representation on base 10 for the binary number 10110
For non-integer numbers, decimal and binary representations of a number extend straightfor-
wardly using the negative powers of 10 and 2, respectively. So,

835.41 = 8 × 102 + 3 × 101 + 5 × 100 + 4 × 10−1 + 1 × 10−2

and for a binary number,

101.01 = 1 × 22 + 0 × 21 + 1 × 20 + 0 × 2−1 + 1 × 2−2

Again, it is enough to compute the previous expression’s right-side term to obtain the decimal
representation for the number 101.01
How to convert a number representations from decimal to binary? One more time, the key is
to look at the number representation as digits multiplied by powers of two. Let us start with an
integer number. Suppose we have a number D, represented in decimal basis. We can expand it a
sum of powers of two. If we divide the number by two, we could represent it as:

D = 2 · C1 + R1
where C1 represents the division quotient and R1 the remaining after division. As we are
dividing by two, the remainder can only be zero or one. Suppose that we divide again the quotient
by two,

C1 = 2 · C2 + R2
1.1. INTRO TO COMPUTERS 21

If we replace the value we got for C1 in the initial equation, we get,

D = 2 · (2 · C2 + R2 ) + R1 = 22 · C2 + R2 · 21 + R1 · 20
If we divide the newly obtained quotient C2 by two and replace the value again,

C2 = 2 · C3 + R3
D = 22 · (2 · C3 + R3 ) + R2 · 21 + R1 · 20 = 23 · C3 + R3 · 22 + R2 · 21 + R1 · 20

After repeating the process n times, suppose we get a quotient Cn = 1. There is no point
in keeping on dividing because, after this point, any division we do will yield a 0 quotient and a
remainder equal to Cn . Therefore,

D = 1 · 2n + Rn · 2n−1 · · · + R3 · 22 + R2 · 21 + R1 · 20

The expression we have obtained fits the binary number 1Rn · · · R3 R2 R1 expansion in powers
of two.
For instance, we can compute the binary expression of 234 using the method already described.
We divide the number and the successive quotients by two until we get a quotient equal to one.
Then, we build the number binary representation, placing the remainders obtained from the con-
secutive divisions in order, from left to right, and adding a one on the left of the remainders.

Dividend Quotient ÷2 Remainder

234 117 0
117 58 1
58 29 0
29 14 1
14 7 0
7 3 1
3 1 1

Therefore, the binary representation of 234 is 11101010.

Suppose now, that we have non-integer number in denary representation as 0.d. If we multiply
the number by two:

E1 .d1 = 0.d · 2 (1.1)

Where E1 represents the integer part and d1 the decimal part of the computed number. We
can then represent 0.d as,

0.d = (E1 .d1 ) · 2−1 = E1 · 2−1 + 0.d1 · 2−1 (1.2)

If we multiply 0.d1 by two again,
E2 .d2 = 0.d1 · 2 (1.3)

0, d1 = E2 · 2−1 + 0, d2 · 2−1 (1.4)

and, after substituting in 1.2

0.d = E1 · 2−1 + E2 · 2−2 + 0, d2 · 2−2 (1.5)

22 CHAPTER 1. INTRO TO SCIENTIFIC SW.

When do we stop the consecutive multiplication process? Basically, we stop when the decimal
part becomes zero, [Link] = 0. However, it’s unlikely that we will reach exactly zero. It’s possible
that a number with an exact decimal representation does not have an exact binary representation.
To stop the process, we can choose to get a predefined number of decimals or continue until the
computed decimal part starts repeating. Since computers have limited register size, they can only
represent a limited number of decimal digits. Therefore, we often truncate the number, accepting
the error it introduces. Thus, we get the original number expansion in powers of two,

0.d · 2 = E1 · 2−1 + E2 · 2−2 + · · · + En · 2−3 + · · · (1.6)

Where values E1 . . . En are precisely the digits corresponding to the binary number represen-
tation: 0.E1 E2 · · · En . (It is trivial to check that they can only be 0 or 1).
Let us see an example of each case, computing the binary representation of the number 0.625
which has an exact binary representation and of the number 0.626 wit do not have it. In the second
case, we will compute an approximate representation, taking 8 decimals.

Decimal P. ×2 Integer P. Decimal P. ×2 Integer P

0,625 1,25 1 0,623 1,246 1
0,25 0,5 0 0,246 0,492 0
0,5 1,0 1 0,492 0,984 0
0,984 1,968 1
0,968 1,936 1
0,936 1,872 1
0.872 1.744 1
0.744 1.488 1

For building the binary representation for the first number, it is enough to take the computed
integer parts by order, from left to right, and add on the left a zero and the decimal point.
Therefore, the binary representation of 0.625 is 0.101. If we expand its value in powers of two, we
recover the original number in its decimal representation.
In the second case, the binary representation, taking nine decimals of the number 0.623, is
0.10011111. We can compute the error made by neglecting the remaining decimals, converting the
number back from binary to decimal,

0 · 20 + 1 · 2−1 + 0 · 2−2 + 0 · 2−3 + 1 · 2−4 + 1 · 2−5 + 1 · 2−6 + 1 · 2−7 + 1 · 2−8 = 0, 62109375

The error we made in the case was,

Error = 0, 623 − 0, 62109375 = 0, 00190625.

1.2 Scientific software application

Among the computer software applications, those devoted to scientific computing are worth men-
tioning due to their close connection with the subject.
It is possible to use high-level language to write programs and routines that allow solving
specific computation problems. Languages such as FORTRAN have been used for decades to do
it, and they are still used in many disciplines of science and engineering. However, many non-trivial
software and hardware computing details should force the scientist who writes their own programs
to be an expert in computer science. For this reason, specific software applications for scientific
1.2. SOFTWARE SCIENTIFIC APPS. 23

computing have been developed, allowing researchers to focus on solving their problems without
worrying about creating a suitable tool to solve them.
Some of these applications are ad-hoc and designed specifically for a particular scientific area.
Others are packages of specific functions intended to carry out computations efficiently. For ex-
ample, the software package SPPS is tailored for statistical computation, providing a high level of
efficiency for researchers.
There is a particularly interesting group of software packages that bridge the gap between high-
level languages and applications. These packages contain extensive function libraries that can be
used for a wide range of computations. They also allow the creation of specific programs using
their own languages. Among them, we can highlight Mathematica, Maple, Matlab, Octave, Scilab,
and Python.
24 CHAPTER 1. INTRO TO SCIENTIFIC SW.
Chapter 2

Introduction to Python
Programming

UN PARACAIDISTA SE
DESNUCA
Por darle una sorpresa y a
escondidas, su hacendosa, tierna e
inocente esposa le cambió a
última hora el paracaı́das de
reglamento por otro de punto,
amorosamente tejido por ella, de
malla ancha.

Julio Ceron. Diario ABC,

2.03.1986. p.55

This chapter presents a general introduction to programming. We will use Python, one of the
currently most used programming languages, along their pages. Guido van Rossum developed
Python in the late 1980s. It is a high-level, general-purpose programming language.
Why Python? Well, it has many useful modules that allow us to develop code for scientific
computing efficiently and reliably. Moreover, being an interpreted language facilitates focusing on
programming skills without bothering with the compiling process. We will use Python to learn
basic general programming methods and solve numerical problems.
This chapter is by no means exhaustive. It would be impossible in the case of a language like
Python. It is just an introduction to its use. Fortunately, Python has excellent online documenta-
tion and plenty of examples on the Web.

2.1 A python’s development environment

Before describing the programming language, we will introduce the tools we should use. It is
common for programming languages to provide an integrated development environment (IDE). An
IDE usually includes a graphic environment and tools, such as online help, a debugger, or a text
editor, that simplify code development and debugging. We are going to use an IDE specifically
designed for scientific computing. It’s not the only one available; neither is perhaps the best one,
but it is suitable to begin working with Python. We refer to Spyder. First, Let’s see how to get it

25
26 CHAPTER 2. INTRO TO PYTHON

and install it on a personal computer.

2.1.1 Anaconda. A free software computing tools distribution

Among the many ways you may follow to install Python and some of its development tools, We
have selected Anaconda because it is free software, easy to install, and includes Spyder. The easy
way to install Anaconda is by downloading it from its Web page:

[Link]

Figure 2.1: Anaconda-Navigator window, Spyder corresponding icon is shown in a red circle

Anaconda installers are available for Windows, Linux, and Mac. The installation process is
straightforward. The Anaconda webpage offers complete information on installation and perfor-
mance.
Once installed, it supplies an application anaconda-navigator to manage Anaconda. When we
start Anaconda-Navigator, we get a window like that shown in figure 2.1.
Anaconda-Navigator shows a set of icons. Anyone of them is intended to launch a different
application. Many of them are IDEs specifically designed to use Python. Others relate to other
programs devoted to data processing, graphics, or statistics. The work launch is written in some of
these icons. This means that the application has already been installed on our computer. Pressing
the left-hand button of the mouse over the word launch opens the application. In other cases,
the word install is Written over the icon. In these cases, the application is not installed on our
computer, but we can install it by pressing the left-hand button of the mouse over the word install.
We are going to use Spyder, which has already been installed. In figure 2.1, the Spyder icon
has been enclosed in a red circle.
Before starting with a Spyder description, we will use Anaconda to install a plug-in, allowing
us to use another of Phyton’s most currently used IDEs: Jupyter Notes. Jupyter Notes is an IDE
2.1. A PYTHON’S DEVELOPMENT ENVIRONMENT 27

Figure 2.2: Anaconda-Navigator window, Environment tab is shown in red

Figure 2.3: Anaconda-Navigator window, Environments tab

28 CHAPTER 2. INTRO TO PYTHON

Figure 2.4: Anaconda-Navigator window.

for Python code development. It is independent of Spyder. It is a handy tool because it allows us
to combine Python code with graphics and text. We will describe Jupyter Notes later on.

Spyder-notebook installation. To install the plugin that allows using Jupyter Notes from
Spyder, we press the mouse left button on the Environments icon (framed in red in figure 2.2).
Then, the Anaconda-Navigator window shows us Anaconda work environments (figure 2.3). If we
have just installed Anaconda, the only environment already available should be the base (root)
environment. We will focus on the right-side panel. The upper left corner has a drop-down
menu; we select the option all. Then, the panel shows us all the software packages available in
the Anaconda default channel. Those packages installed in the computer are ticked with a green
square label before the package name; packages available but not installed are ticked with a white
square.
There is a browser located in the upper right corner of the panel. We will write the word spyder
of spyder notebook. The panel now shows those software packages related to Spyder (figure 2.4).
We select the package spyder notebook by clicking the white square before the package name. A
new pair of push buttons appear on the panel’s lower side. Clicking on apply triggers the Spyder
notebook installation process. First, Anaconda launches a pop-up window and checks Spyder
Notebook software dependencies. The process may take some time, so be patient until it finishes
the checking. Once the checking process is over, we will press the apply button on the pop-up
window and let Anaconda install the software.

2.1.2 Spyder
Let’s focus on the Spyder description. As we said before, Spyder is an Integer Development
Environment specially designed to use Python for scientific computing. To start Spyder, we push
the launch button on the Spyder Icon, located in the Anaconda-Navigator window (figure 2.1).
Then, a new window opens, as shown in figure 2.6.
2.1. A PYTHON’S DEVELOPMENT ENVIRONMENT 29

Figure 2.5: Anaconda-Navigator window.

This (Spyder) window has three different panels1 . Besides, in the upper part of the window,
there is a toolbar and a second bar that contains drop-down menus. We are going to give a brief
description of the Spyder environment and, at the same time, some of Python programming’s basic
features. Anyway, we will not be exhaustive. An exhaustive description is far beyond the reach of
these notes. The best way to learn to deal with Spyder is by using it. Moreover, there is a vast
amount of available documentation.

The Python’s Enhance Terminal Returning to the three panels shown in figure 2.6, we will
focus first on the right-down one. This panel is a Python console. The console displays a symbol
In [1]: known as the prompt followed by a flicking vertical bar. The console allows the user to
interact with Python straight; i.e., Python can get direct instructions through the terminal, run
the instructions, and show and/or save in memory the achieved results. Let’s see an example. If
we write in the console:
In [1]: 2+2
and press the intro key; Python calculates the sum, shows the result using an output (echo) mark,
Out [1]:, gives back a new clean prompt, and stops waiting for a new command:
Out[1]: 4
In [2]:
In programming slang, it is usual to call the answer the computer shows on the screen an echo.
As we shall see later on, it is not usual for the computer to show us the result of the operations.
Most of the time, it just executes the commands, and when it finishes, it shows us the prompt
In[2]: again to tell us that it is ready for a new command. In this way, we can use Python as a
calculator. The syntax is, indeed, quite similar.
1 We describe here the default Spyder configuration. The user may change this default configuration according

to their preferences.
30 CHAPTER 2. INTRO TO PYTHON

Figure 2.6: Spyder: An integer development environment for Python

2.1.3 Variables
Nevertheless, handling Python as a calculator is hardly ever enjoyable. Once a command has been
executed after pressing the intro key, the computer keeps no memory of the operation. If we wish
to make a complex computation, splitting it into simpler operations, we should take note of the
partial results and copy them again on the console to be used again. Of course, a long time ago,
somebody found a good solution for this and other similar problems: using variables.
We can consider a variable as a computer memory region where a program has allocated specific
information: numbers, characters, etc. A variable fundamental characteristic is its name because
it permits one to identify it uni-vocally. We can take as a variable name whatever combination of
lowercase and uppercase letters and numbers, provided that the first character is always a letter.
When using Python2 , We may also use the symbol ” ”. For Python, uppercase and lowercase
letters are different symbols. Thus, if we choose variable names, such as Oack, OACK, and OaCk,
they represent different variables for Phyton.
The most straightforward method to create or use a variable is to assign the information we
want to contain. To do this, we use the assignment symbol , which coincides with the mathematical
symbol =. As will be seen later, programming assignment and mathematical equality are not the
same concept. To assign some piece of information to a variable, we write the variable name, then
the assignment symbol, and, eventually, the assigned information variable_1 = 18. If we write
this expression on the console and press the intro key, we get:

In [2]: variable_1 = 18
2 As we shall see later on, Python has a set of command names and keywords already defined. We should avoid

using these names or keywords as names for our variables. Otherwise, we could lose access to the corresponding
Python command or keyword.
2.1. A PYTHON’S DEVELOPMENT ENVIRONMENT 31

In [3]:

In this case, the computer doesn’t echo the result. Anyway, the variable has been saved in the
computer’s memory. We may ask Python to show it,

In [3]: variable_1
Out[3]: 18Ipython

In [4]: print(variable_1)
18

In [5]:

In the first case, we have written the name of our variable straightforwardly after the Ipython
prompt. In the second case, we use the Python function print(). Later on, we will see the
concept of function in Python in more detail. Meanwhile, it is enough to say that a function is a
programming object that takes an input variable, operates it and returns a result. In our case, the
function print() takes a variable whatsoever and prints in the Ipython console the content of the
variable.
We can also assign to a variable the result of an arithmetic operation,

In [5]: variable_2 = 2 * 5

To know which variables are saved in the computer memory, we can use some special Ipython
console commands:

In [5]: %whos
Variable Type Data/Info
-----------------------------------
variable_1 int 18
variable_2 int 10

In [6]: %who
variable_1 variable_2

In the first case, we used the Ipyhon command whos. Notice that we have written the symbol %
just before the command. This tells the computer we are introducing a console (Ipython) command,
not an ordinary Python command. The computer displays the variables we have defined until this
point, the variable Type, and the data contained. In the second case, we have used the command
who, which shows us the bare list of the variables saved in the computer memory.
We just mentioned the type of a variable. In some programming languages, it is necessary to
explicitly say the kind of information a variable will store before using it. The sort of information
stored defines the type of the variable. This helps to manage the computer memory more effi-
ciently, assigning memory zones according to the variable size. This process is known as variable
declaration. In Python, it is not necessary to declare variables. Python sets the type to a variable
when it is created. To Know the data type of a variable, we use the function type(). Let’s see
standard or built-in types in Python.
32 CHAPTER 2. INTRO TO PYTHON

Numeric. Data which represent numerical quantities. Inside the numeric data, Python defines
three different types:
Integer. It allows for representing whole numbers, positives, and negatives. The type is repre-
sented by the abbreviation int. A specific feature of integers in Python is that they have no size
limitation.

In [92]: a = 35
In [93]: type(a)
Out[93]: int

Float. Floating point numbers, i.e., approximated representations of real numbers. (See chapter
5). We use a point to split the integer and decimal parts of the number. It is also possible to write
a floating point number using scientific notation. This type is represented by the word float.
Notice that a = 3 creates an integer variable, but a=3. creates a real variable.

In [95]: b = -3.5
In [96]: type(b)
Out[96]: float
In [98]: c = 3e -4
In [99]: type(c)
Out[99]: float

Finally, it is possible in Python to deal with complex numbers. To define them, we write the
real part of the number, the addition symbol, and the imaginary part, followed by the symbol
j. Notice that Python considers a complex number’s real and imaginary parts as floating point
numbers even if you write them as integers.

In [100]: d = 2+3j
In [101]: type(d)
Out[101]: complex
In [102]: [Link] #real part
#of number d
Out[102]: 2.0
In [103]: [Link] #imaginary
#part of number d

As we will see later, it is widespread in programming to check if a condition fulfils (it is true)
or not (it is false). This leads to defining a type of variable that can take just two possible values:
True or False.

In [1]: D = True
In [2]: type(D)
Out[2]: bool

Sequence Type Every kind of variable included in Sequence Type, and also the types dictionary
and set, are Data structures. We can see them as containers that allow us to deal with several
(many) data using a single variable.
String This is the proper type to generate variables that contain text. They are built using
UNICODE3 characters enclosed in brackets.
3 UNICODE is a standar for computer character representation.
2.1. A PYTHON’S DEVELOPMENT ENVIRONMENT 33

Pyton-Data types

Numeric Dictionary Boolean Set Secuence Type text

Integer Float Complex List tuple string

Figure 2.7: Data types in Python

In [3]: tex1 = 'a'

In [4]: type(tex1)
Out[4]: str
In [5] tex2 ='perro'
In [6]: type(tex2)
Out[6]: str

A common property of all types belonging to Sequence Type is that they are indexable. The
indexation is a fundamental property that we frequently use in programming. The variable tex2
in the previous example is an array of characters, the letters that compound the work perro (Dog
in Spanish HA, HA). Using an index, Python, allows us to extract such characters individually.
The index represents the position the character we want to extract takes in the word. To extract a
character from a variable of type String, we write the variable’s name followed by the character’s
index enclosed in square brackets. It is important to note that Python begins to count the elements
of a sequence in zero.

In [7]: letra_1 = tex[0]

In [8]: print(letra_1)
p
In [9]: letra_4 = tex[3]
In [10]: print(letra_4)
r

Python also permits indexing a sequence beginning by the end. In this case, the sequence’s last
element takes the index -1, the next-to-last takes -2, and so on.
34 CHAPTER 2. INTRO TO PYTHON

In [40]: tex[-1]
Out[40]: 'o'
In [41]: tex[-3]
Out[41]: 'r'
Lists are ordered collections of data. The elements of a list can be data of whatever type, and
they do not need to be homogeneous. To create a list, write the elements that built it up, enclosed
in square brackets and separated by commas.

In [11]: L = ['carta', 23, 2.5, 1-2j]

In [12]: L
Out[12]: ['carta', 23, 2.5, (1-2j)]
In [13]: type(L)
Out[13]: list
We built a list in which the first element is a string, the second is an integer, the third a real
float type number, and the fourth a complex number. Due to their versatility, Lists are a very
useful type in Python. We can copy elements of a list into another variable using indexes and
assign a new value to an element of a list. Of course, in this case, we will lose the old value.

In [14]: L3 = L[3]
In [15] print(L3)
(1-2j)
In [16]: L[2] = 0.
In [17]:print(L)
['carta',0.0,2.5,(1-2j)]

As far as a list is nothing more than an ordered collection of variables, we can nest a list inside
another list,

In [14]: LL = [1, 'p',L,-0.7]

In [15]: print(LL)
[1, 'p', ['carta', 23, 2.5, (1-2j)], -0.7]

We have included the list L as the third element of a new list LL. We can extract an element
from the inner list using two indexes. The first one indicates the position of the list L inside the list
LL, and the second one indicates the position of the wanted element inside the list L. For instance,
if we want to get the word ’carta’,

In [16]: ctr = LL[2][0]

In [17]: print(ctr)
carta
We can change list data using the assignation symbol,

In [18]: LL[3] = 'J'

In [19]: print(LL)
[1, 'p', ['carta', 23, 2.5, (1-2j)], 'J']

An essential feature of Lists is their methods. The methods are special functions that allow us
to manipulate the lists in a straight way. Table 2.1 shows some of the most used list methods. To
use a method, we write the list name followed by a point and name of the method and, enclosed
in brackets, the parameters the method takes. If the method doesn’t use parameters, we write an
empty bracket after the method name but never leave it off. Let’s see some examples:
2.1. A PYTHON’S DEVELOPMENT ENVIRONMENT 35

append() adds a element to the end of a list [Link](-3)

copy() Returns a copy of a list L2 = [Link]()
clear() clears all elements from a list [Link]
count() Counts the elements of a list [Link]()
insert() Inserts an element at a specific position in a list [Link](i,-4)
pop() extracts the last element of a list. a =[Link]()
Adding an index, extracts the specific element a=[Link](i)

Table 2.1: Some methods of Python’s List

In [12]: L = [1, 'a',-0.5,'en',2.5,'a']

In [13]: print(L)
[1, 'a', -0.5, 'en', 2.5, 'a']
In [14]: [Link](-2)
In [15]: print(L)
[1, 'a', -0.5, 'en', 2.5, 'a', -2]
In [16]: [Link]()
Out[16]: -2
In [17]: print(L)
[1, 'a', -0.5, 'en', 2.5, 'a']
In [18]: [Link](3,'insrt')
In [19]: print(L)
[1, 'a', -0.5, 'insrt', 'en', 2.5, 'a']
In [20]: [Link](3)
Out[20]: 'insrt'
In [21]: print(L)
[1, 'a', -0.5, 'en', 2.5, 'a']
In [22]: [Link]('a')
Out[22]: 2
In [23]: [Link]('s')
Out[23]: 0

Tuples Tuples are similar to List. The main difference between them is that Tuples are im-
mutable, i.e., once created, their elements cannot be modified. To build a Tuple, we enclose its
elements into brackets separated by commas. The following example shows how Python throws an
error message when we try to modify a Tuple element once it has been built.
36 CHAPTER 2. INTRO TO PYTHON

In [41]: T = (0,'int',2.4,[1,4,3.5])
In [42]: T
Out[42]: (0, 'int', 2.4, [1, 4, 3.5])
In [43]: T = (0,'int',2.4,[1,4,3.5])
In [44]: print(T)
(0, 'int', 2.4, [1, 4, 3.5])
In [45]: type(T)
Out[45]: tuple
In [46]: T[3]
Out[46]: [1, 4, 3.5]
In [47]: T[0] = 12
Traceback (most recent call last):
Cell In[47], line 1
T[0] = 12
TypeError: 'tuple' object does not support item assignment

Dictionary. A dictionary is a structure in which data are associated with keys. The simplest
way to create them is by using a pair key: data separated by commas and enclosed in curly braces.
The symbol ’:’ associates each element with its key. A dictionary can contain whatever type of
variable, including lists and other dictionaries. To get access to data stored in a dictionary, we
write the name of the dictionary followed by the key of the element we want to access, enclosed in
square brackets.

In [63]: D = {'nombre':'Pepe','dı́a':27, 'mes':'Febrero', 'datos':[1,3.5,6,0.0]}

In [64]: D
Out[64]: {'nombre': 'Pepe', 'dı́a': 27, 'mes': 'Febrero', 'datos': [1, 3.5, 6, 0.0]}
In [65]: type(D)
Out[65]: dict

To add new elements, we write the name of the dictionary followed by the key we want to assign
to the element, enclosed in square brackets, and we use the assignation symbol to add the element
value. As in the case of lists, it is possible to nest one dictionary into another. To access the inner
dictionary, we use its key. To access data inside the inner dictionary, we use the inner dictionary
key followed by the data key. (see In[78]: in the example below). To eliminate an element from
a dictionary, we use the Python command del(). Similar to the Lists, there are quite a few
dictionary methods. Readers interested should be addressed to Python reference pages.

In [75]: D['nuevo'] = {'calle':'Atocha','num.':18,'Piso':'3D'}

In [76]: D
Out[76]:
{'nombre': 'Pepe',
'dı́a': 12,
'mes': 'Febrero',
'datos': [1, 3.5, 6, 0.0],
'nuevo': {'calle': 'Atocha', 'num.': 18, 'Piso': '3D'}}
In [77]: D['dı́a']
Out[77]: 12
In [78]: D['nuevo']['num.']
Out[78]: 18
2.2. ARITHMETICAL, RELATIONAL AND LOGICAL OPERATIONS 37

In [81]: del(D['dı́a'])
In [82]: D
Out[82]:
{'nombre': 'Pepe',
'mes': 'Febrero',
'datos': [1, 3.5, 6, 0.0],
'nuevo': {'calle': 'Atocha', 'num.': 18, 'Piso': '3D'}}

Set. Sets are inmutable non-repited data collections. To make a set in Python we writen its
elements separated by commas and enclosed in curly braces. If there are repited elements in its
definition, the set created will containt a single instace of the repited element.

In [85]: C= {'L','M','X','J','V'}
In [86]: C
Out[86]: {'J', 'L', 'M', 'V', 'X'}
In [87]: C= {'L','M','X','J','V','M'}
In [88]: C
Out[88]: {'J', 'L', 'M', 'V', 'X'}
In [89]: C[1] = 23
Traceback (most recent call last):
Cell In[89], line 1
C[1] = 23
TypeError: 'set' object does not support item assignment

2.2 Arithmetical, relational and logical operations

2.2.1 Arithmetic operations
Once we know how to create variables in Python, we will see how to perform basic arithmetic
operations. The syntax is pretty simple, and we can synthesize it as follows:

result = operand1 operator1

operand2 operator2 operand3 · · ·
operatorn−1 operandn

That is, it is enough to concatenate operands and operators and define a variable that gets the
result. For instance:
In [1]: a = 3
In [2]: b = 4
In [3]: c = 12.4
In [4]: d = a + b - c
In [5]: print(d)
-5.4
In this example, the operands are the variables a, b, c. the operator are the symbol + which
represents the addition operation and the symbol - which represents the substraction. d is the
variable that holds the results. In this case, the addition of the two first variables and the result
with the third one.
Python available arithmetics operators cover the usual arithmetic operations. Table 2.2 shows
the arithmetic operators defined in Python.
38 CHAPTER 2. INTRO TO PYTHON

Table 2.2: Arithmetic operators defined in python

Operation Symbol Uso notes

Addition + r=a+b
Substraction - r=a-b
Product * r=a*b
division / d=a/b
Calculate the division and round the cotient
Integer division // d=a//b
towards zero
Modullus % d=a% b Calculate the remainder after integer division
Power ** y=a ** b Rise a to the power of b: ab

In [92]: a = 5
In [93]: b = 3
In [94]: print(a+b)
8
In [95]: print(a-b)
2
In [96]: print(a*b)
15
In [97]: a = 5
In [98]: b = 3
In [99]: suma = a+b
In [100]: print(suma)
8
In [101]: dif = a-b
In [102]: print(dif)
2
In [103]: prod = a*b
In [104]: print(prod)
15
In [105]: div = a/b
In [106]: print(div)
1.6666666666666667
In [107]: div_ent = a//b
In [108]: print(div_ent)
1
In [109]: rem = a%b
In [110]: print(rem)
2
In [111]: pot = a**b
In [112]: print(pot)
125

2.2.2 Arithmetic operator precedence

The previous examples show a basic use of the seven arithmetic operators defined in Python.
By combining arithmetic operators, it is possible to build up complex expressions. For instance:

In[1]: R=5*3-6/3+2**3+2-4
2.2. ARITHMETICAL, RELATIONAL AND LOGICAL OPERATIONS 39

A question arises: in which order does Python carry out the operations involved in this expression?
To avoid ambiguities, Python—as any other programming language—establishes a precedence
order that allows knowing exactly in which order the operations will be carried out. Python
precedence order is:

1. First, it calculates the powers.

2. Then, products and divisions that share the same precedence degree.

3. Eventually, additions and subtractions are carried out.

Thus, in the example we have just shown, Python would calculate first,

2**3=8

then, the product and the division

5*3=15
6/3=2

eventually, it would sum up all intermedia results and save the final result in the variable R.

15-2+8-4=17
R=17

Using parentheses to modify the precedence order. When evaluating a complex expres-
sion, we need to modify the order of precedence in many cases. To do it, we use parentheses. The
rules of use are mainly two:

• Expresions enclosed in parenthesis have precedence over whatever other operation.

• When using nested paratheses (parenthesis enclosed in other parentheses). The results are
always obtained from the inner parenthesis to the outer one.

For instance,

In[1]: y=2+4/2
In[2] : print(y)
4
In[3]: y=(2+4)/2
In[4]: print(y)
3

In the first operation, the precedence order makes Python divide 4 between 2 and then add 2
to the result. In the second case, the Parenthesis has precedence; Python first adds 2 and 4 and
then divides the result between 2.
Using parentheses to alter the operator’s precedence allows building whatever mathematical
expression we wish. For example, to calculate the hypotenuse of a rectangular triangle using the
values of its catheti,
1
h = (c21 + c22 ) 2 ,
we may express this in Python as,

In[1]: h=(c1**2+c2**2)**(1/2)
40 CHAPTER 2. INTRO TO PYTHON

Or, as another example, the generic solution of a quadratic equation

1
−b ± (b2 − 4 · a · c) 2
x= ,
2·a
in this case, it is necessary to split the result into two expressions: one for the positive root,

In[2]: x=(-b+(b**2-4*a*c)**(1/2))/(2*a)

and the second one for the negative root.

In[3]: x=(-b-(b**2-4*a*c)**(1/2))/(2*a)

Caution is needed when building expressions that contain a large number of operations. Take
the example just shown: if we forget the last parathesis 2*a, Python would multiply by a the result
of the remaining operation instead of dividing it by a.

2.2.3 Relational and logical operations.

Although they are not equal, relational and logical operations are strongly related. As we have
seen in the case of arithmetic operations, relational and logic operations are built up using
operands—variables on which we perform the operations—and operators that indicate which op-
erations we carry out on the operand. The main difference with arithmetic operators is that both
relational and logic operations would only cast 1 (True) or 0 (False) as an operation result.

Relational Operators. Table 2.3 shows the relational operators available in Python. Their
result is always the truth or falsehood of the relationship the operator represents.

Table 2.3: Relational operators defined in Python

operation symbol example notes

The result is True if a is minor than b. Oth-
minor than < r=a<b
erwise the result is False
The result is True if a is greater than b. Oth-
greater than > r=a>b
erwise the result is False
The result is True if a is minor than b or equal
greater than or equal to >= r=a>=b
to b. Otherwise the result is False
The result is True if a is greater than b or
Less than or equal to <= r=a<=b
equal to b. Otherwise the result is False
The result is True is a is equal to b. Otherwise
equal to == *a==b
the result is False
The result is True if a is not equal to b. Oth-
not equal to != a!=b
erwise the result is False

It is important to note that the symbol double-equal == compares whether two variables are
equal. Please do not mistake it for the assignation symbol =.

Logical operator Python distinguishes three sets of logical operators. We will only present one
of them: Logical operators to compare variables.
Table 2.4 shows the logical operator defined in Python for variables. The result is always (1)
True or (0) False.
2.2. ARITHMETICAL, RELATIONAL AND LOGICAL OPERATIONS 41

Table 2.4: Logical operators on values and variables

operation symbol example notes

Logical operation and between variables a and
and and r=a and b
b
Logical operation or between the variables a
or or r=a or b
and b
not not r= not a a complement (if a is True then not a is Fase)

A logical operator works following the standard of Bool’s Algebra. So, the and operator follows
the true table of the and operation in Bool’s Algebra, the result is true (1) only if both its operands
are true, 4

Truth table for and operation

operand 1 operand 2 result
1 1 1
1 0 0
0 1 0
0 0 0

The operator or follows the true table of the boolean operation or, and the result is true if any
one of their operands is true or both of them are true.

Truth table for or operation

operand 1 operand 2 result
1 1 1
1 0 1
0 1 1
0 0 0

Let’s see how the logical operator in Python fullfil the truth tables,

4 In fact, Python takes any variable a true, whenever it is not zero.

42 CHAPTER 2. INTRO TO PYTHON

In [1]: a = 1
In [2]: b = 0
In [3]: c = 1
In [4]: d = 0

In [5]: a and b
Out[5]: 0

In [6]: a and c
Out[6]: 1

In [7]: b and d
Out[7]: 0

In [8]: a and d
Out[8]: 0

In [9]: a or b
Out[9]: 1

In [10]: a or c
Out[10]: 1

In [11]: a or d
Out[11]: 1

In [13]: b or d
Out[13]: 0

In [14]: not a
Out[14]: False

In [15]: not b
Out[15]: True

It is important to remark that logical operations in Python cast a value ̸= 0 if the result is true
or = 0 if the result is false, while relational operations and the complement (negation) cast boolean
values True or false. That is not a problem at all because Python manages these boolean values
as their integer counterpart 1 and 0. The fact that Python considers true any value not equal to
zero will help us to show how it carries out logical operations, get a lookout to the next examples,
2.2. ARITHMETICAL, RELATIONAL AND LOGICAL OPERATIONS 43

In [25]: vdo = 7.5

In [26]: fls = 0
In [27]: vdo2 = 1
In [28]: vdo3 = 18
In [29]: vdo and fls
Out[29]: 0
In [30]: vdo or fls
Out[30]: 7.5
In [31]: vdo and vdo2
Out[31]: 1
In [32]: vdo2 and vdo
Out[32]: 7.5
In [33]: vdo or vdo2
Out[33]: 7.5
In [34]: vdo2 or vod
Out[34]: 1

We defined four numeric variables in lines In[25] to In[26]. All of them are true for Python
except for fls, which is zero and, therefore, false. If we apply the operator and to variables vdo
and fls, the result is zero because one of the variables is zero. However, in line In[30], we see that
the result of the or operation carried out is 7.5, Which is the value of the first variable, i.e., vdo.
This result is related to Python’s way of performing logic operations. For an or operation to yield
true, it is enough that one of the operands be not equal to zero (true). The program begins trying
the first operand, and once it checks that it is true, it is not necessary to test the value of the
second operand because the result of the operation will be true, no matter which value the second
operand takes. So, once Python is sure the first operand is not equal to zero, it casts its value as
the true result of the operation.
By contrast, in lines In[31] and In[32,] we ask Python to evaluate the result of the and operation
applied to vdo and vdo2. Now, Python has to check that both variables fulfil the condition.
Although the first variable is not equal to zero, the result will only be true if the second variable
is not zero.
It’s worth noting that the result of this operation can change depending on the order of the
operators. In both cases (In[31] and [32]), the operation yields true, but Python assigns the value
of the second operand as the result because it is the last variable that Python has checked.
Lastly, we can combine logical operators with relational and arithmetic operators. The prece-
dence order is as follows,

1. Parenthesis ()

2. Aritmetical operators in their order of precedence

3. Relational operator, all have the same precedence order; thus, they are evaluated from left
to right.

4. and

5. or

Using parenthesis when several logical operations are linked is good practice to ensure their
correct use and to ease the code reading.
for instance,
44 CHAPTER 2. INTRO TO PYTHON

In [3]: a = 12.5
In [4]: b = 3.6
In [5]: c = -3,2
In [6]: d = 0.

In [15]: a>b or d>c

Out[15]: True

In [16]: (a>b) or (d>c)

Out[16]: True
In both cases, Python first computes the relational operators, i.e., it checks if a > b and c > d.
Then, it applies the logical operator or to the previous results. But, in the second case, it is easier
to understand the operation’s objective, thanks to the parentheses.
Let’s see another example where the parentheses are mandatory: the logical operation exclusive
or o XOR. It is an operation that compares two logical variables and yields true only if one of the
operands is true and the other is false. the XOR operation true table is as follows,

Truth table for xor operation

operand 1 operand 2 result
1 1 0
1 0 1
0 1 1
0 0 0

This operation is not defined in Python but can be easily implemented using the logical oper-
ations and, or y not:
XOR ≡(a or b) and not(a and b).
We left it as an exercise to check that such a combination of logical operations fulfils the XOR
true table and what happens if we remove the parenthesis.

2.3 Scripts in Python

So far, we have always dealt with Python using the command line. That is, we have constantly
introduced instructions using the command line. This method is rather inefficient because we need
to rewrite the commands again anytime we want to repeat the same operations.
With Python, you can effortlessly save a set of instructions in a text file and reuse them anytime
you need to carry out the same calculation. This is the usual way to work in Python and many
other programming environments. A Python code file is usually called a Script. Thus, a Python
Script is a text file that contains lines of valid Python commands. The file is saved using a name
and the extension .py. So a valid name would be, for example, [Link]. The file name
may contain numbers, letters, and special characters, such as the underline ( ). Using names that
include blanks or other special characters is not advisable as a general criterion because we may
have problems running them.

2.3.1 The Spyder’s text editor

We can use whatever text editor we want to write our programs, provided it generates text in
ASCII format. For instance, we can use the classical ’notepad’. Nevertheless, If we are working
with Spyder, we should use its text editor.
2.3. SCRIPTS IN PYTHON. 45

Figure 2.8: Text editor for Python included in Spyder

The Spyder’s text editor is located in the left panel of the IDE window (figure 2.6). Figure 2.8
shows a more detailed view.
The figure shows a panel with several tabs. Each one belongs to a different text file containing
a Python program. The file name is written in the tab. The file shown in the figure has the name
[Link]. It is a new file we have just opened to write a new program. To create a new Spyder
script, it is enough to click the left icon on top of the editor panel or unfold the File menu located
just above and select the option New file.
Looking at figure 2.8, we see how the code lines are numbered on the right side of the file how
the code lines are numbered. Besides, some code lines are included.
These lines, written by default in the text file, are not Python commands; they are comments,
i.e. lines that supply information on the file content to the reader. Usually, these lines contain
the program’s purpose, the type of variables it uses, the date of creation, the author, etc. Python
accepts two kinds of comments:

• Single comment lines. They start with a hash # symbol. Python considers the whole line a
comment and skips it.
 46 CHAPTER 2. INTRO TO PYTHON

• A sequence of lines enclosed into groups of three quotation marks """. They are used for
lengthier comments, which contain longer descriptions of the code performance.
[Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Sat Mar 16 20:05:44 2024
4

5 @author: abierto
6 """

In our case, Spyder has written a line comment showing the text codification (utf8 ) used to
store the file’s information. Below, Spyder creates a comment area –the second type of comment–
using the code lines 2 to 6. There, it writes the file creation date and the name of the file author.
Of course, this data can be changed by the user. This comment area created by Python is an
excellent place to add information about the file features, the type of variables defined, etc.
We will continue with the example, adding some code lines and then save the file with a new
name. To save a file, click the third icon from the left in the icon panel above the editor or unfold
the File menu and select Save as. We will save our file with the name [Link].
[Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Sat Mar 16 18:48:46 2024
4 Este Script es un simple ejemplo para mostrar el uso básico
5 de los Scripts de Python
6 This Script is a basic example just to show how to use Python's
7 Scripts
8 @author: abierto
9 """
10

11 #Primero creamos unas variables y mostramos su valor

12 #first we create some variables
13 a = [1,2,3.3,4,0,-8]
14 print(a)
15 b = 12
16 print(b)
17 c = [1,0,1,0]
18 print(c)
19 #Ahora realizamos algunas operaciones basicas con las variables
20 #Now we perform some basic operation with the variables created
21 a[3] = a[3] + 4.2
22 p = [Link]()
23 d = p+b-c[0]
24 #por ultimo mostramos los resultados de nuestras operaciones
25 #and eventually we show the resuts
26 print(a)
27 print(p)
28 print(d)

Focusing on the changes, we added some lines to the long comment explaining the file’s contents.
Besides, we have introduced some line comments now and then, describing what the code does in
2.4. FUNCTIONS IN PYTHON. 47

each step. It is easy to check that we have created two Lists a and c and an integer variable b.
Moreover, python prints these new variables on the console to show their values. Then, it adds
4, 5 to the current value of the fourth element of the list a, pops out the last values of this list and
saves it in a new variable p. On line 23, a new variable, d is created, adding pand b and subtracting
from the result the last element of the list c. Eventually, Python shows the values of variables a,
p and d.

2.3.2 Running a script

Once we have finished our code edition, we need to run it to obtain the results. Using the Spyder
environment, we run a script by clicking the green triangle icon at the top of the editor panel or
unfolding the Run menu and selecting the option Run.
The results are shown on the Ipython console,

In [4]: runfile('C:/Users/abierto/Documents/borrar/[Link]',
wdir='C:/Users/abierto/Documents/borrar')
[1, 2, 3.3, 4, 0, -8]
12
[1, 0, 1, 0]
[1, 2, 3.3, 8.2, 0]
-8
3

When we click the icon Run, we execute the command runfile(), including, enclosing in
parentheses, the name of the file we want to run. Then, Python shows the results of the print
command that we have included in our program. From the point of view of the results, there is no
difference between running the script or writing their lines one by one in the Ipython console. In
fact, Python has saved the variables we created running the script in the computer memory, and
we can use them if we introduce new commands in the console. For instance, we can add a new
element at the end of list c,

In [7]: [Link](31.2)
In [8]: print(c)
[1, 0, 1, 0, 31.2]

In [9]:

2.4 Functions in Python

In the previous section, we learned about using scripts to save our code and reuse it later. While
this is an improvement over repeatedly rewriting the same commands in the console, it still results
in code that is not very flexible and reusable. In this section, we will introduce functions, a
programming structure that enables our code to become a more powerful tool.
In Python, it is possible to define functions using the console. However, this is not advisable
because all work will be lost once Spyder is exited. takes in Python the following general structure,

def funcion_name(e1,e2,...):
function code
return, r1, r2, ...
 48 CHAPTER 2. INTRO TO PYTHON

The first line of a function definition is known as the function header. It always starts with
the keyword def. Following this, we write the name of the function. Valid function names follow
the same rules as file names. After the function name, we enclose input variables in parentheses,
separated by commas. These input variables are the variables that the function will use to perform
its operations. When the function does not need input variables, we leave the parentheses empty
but never omit them. Eventually, we write the symbol ’:’ to end the function header.
Beneath the header file comes the function body, that is, the code lines of the function. One
important feature of Python language is that any line belonging to the body function should be
indented. The function body lines should begin, leaving the same number of blank spaces from
the beginning of the line. One advantage of using the Spyder text editor is that it automatically
acknowledges the file header lines and indents the following lines. The last function line is usually
used to recover variables that contain the function operations’ relevant results. It starts with the
keyword return, followed by a list of variables separated by commas. These variables should be
defined in the function body.
Let’s look at an example function for solving quadratic equations.

ax2 + bx + c = 0

it has the following general solutions,

√
−b ± b2 − 4ac
x1,2 =
2a
Example [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Thu Mar 28 17:22:30 2024
4

5 @author: abierto
6 """
7

8 def soleq2(a,b,c):
9 """
10 Esta función calcula las soluciones de la ecuación general,
11 ax**2 + b*x + c = 0
12 Variables de entrada:
13 coeficientes a, b y c
14 Variables de salida:
15 xp Solucion para la raı́z cuadrada positiva
16 xm solucion para la raiz cuadrada negativa
17

18 this function calculates the solutions of the general equation

19 a*x**2 + b*x + c = 0
20 input variables:
21 a,b and c coeficients
22 output variables;
23 xp solution using positive square root
24 xm solutions using negative square root
25 """
26 xp = (-b+(b**2-4*a*c)**(1/2))/2/a
27 xm = (-b-(b**2-4*a*c)**(1/2))/2/a
28 return xp, xm
29
 2.4. FUNCTIONS IN PYTHON. 49

30 #aquı́ a~
nadimos un ejemplo de uso que no es parte de la funcion
31 #here we add a 'how to use' example that's not part of the function
32

33 solplus, solminus = soleq2(2, 3, 1)

34

35 #comprobacion
36 #checking
37 print('comprobación/cheking')
38 print('raı́z positiva/positive root:', 2*solplus**2+3*solplus+1)
39 print('raiz negativa/negative root:', 2*solminus**2+3*solminus+1)

We have opened a new script in the Spyder text editor to define our function. We have saved
the Script with the name eq seg [Link]. Obviously, you can save the file with whatever valid
name you like, but it is a good idea to find a name that gives us some hints in the script content
(quad [Link] should be a nice name for this script).
After briefly commenting on the file’s content, we define the function header on line number
9. It is interesting to look carefully at this definition: After writing the keyword def, we have
given a name to the function soleq2, and, next, we have defined the input variables, enclosed
in parentheses and separated by commas. These input variables will be the coefficients of the
quadratic equation we wish to solve. We close the function header, adding the final colon. Just
beneath the header, we have added a rather lengthy comment explaining the purpose of the function
and describing its input and output variables.
We have written the mathematical operations for the quadratic equation’s positive and negative
squared roots in lines 27 and 28, respectively. Check it! In the first case, we have saved the result
in variable xp and, in the second case, we have saved the result for the negative squared root in
variablexm. Lastly, line number 29 returns variables xp, xm with the operation results.
The function definition finishes here. The remaining code lines do not belong to the function,
so they are not indented. We have added them to show how to use the function soleq2 and check
if the results are correct.
In line 34, we show how to use (call) the already created function. First, we define two variables,
solplus and solminus for saving the function results. These variables, separated by commas, are
orderly associated with those returned by the function,i.e. the variables included in the function
code after the keyword return. Thus, Python will copy the content of variable xp in variable
solplus and the content of variable xm in variable solminus. It is unnecessary to say that we can
give whatever valid name we want to the variables we use to save the function results. After the
variables above, we write the assignation symbol followed by the function name. lastly, we write
values for input variables, a,b,c.
Lines 38 and 39 check that both obtained solutions fulfil the quadratic equation 2x2 +3x+1 = 0.
We have added a quoted text to the print command that helps to determine which result is up
to each solution.
If we click on the green triangle icon and run the script we have written, we get the following
result in the Ipython console,
In [21]: runfile('/home/juan/LCC_Python/codigos/introduccion/eq_seg_grado.py',
wdir='/home/juan/LCC_Python/codigos/introduccion')
comprobación/cheking
raı́z positiva/positive root: 0.0
raiz negativa/negative root: 0.0
It is important to note that a script is just a sequence of instructions that run in the console.
Thus, if we track the sequence of the example script, we first create the function defined in the
50 CHAPTER 2. INTRO TO PYTHON

script and save it in the computer memory. Hence, as long as we don’t quit Spyder, we can keep
on using it directly in the Ipython terminal,

In [12]: coef = [3,-2,2]

In [13]: sp, sm = soleq2(coef[0],coef[1],coef[2])
In [14]: print(sp)
(0.3333333333333334+0.7453559924999299j)
In [15]: print(sm)
(0.3333333333333333-0.7453559924999299j)
In [16]: print(solplus)
-0.5
In [17]: print(solminus)
-1.0

Notice that we have collected the quadratic equation coefficients in a Python list in this example.
Then, we call the function. The procedure is the same as before; we define output variables to save
the results sp, sn, write the assignation symbol and the function name and give values to the input
variables. These last have been written so that the function should identify the quadratic equation
a coefficient with the first element of the list, the second coefficient with the second element of the
list, etc.
Next, we use the command print to watch the output values of the output variables. In
this case, the equation does not have real solutions, and the results are two complex conjugated
numbers.
The last two lines, In[16] and In[17], are used to show the value of solplus and solminus.
these variables were created when we ran the script, but Python still holds them in the computer
memory.

2.5 Namespace and Scope in Python

We have seen that Python holds the created variables in the computer memory so that they can
be used later. Understanding how Python manages the variables saved in the computer memory
is essential. This takes us to the idea of Python namespaces. A namespace could be considered
a memory space reserved by Python, where programs and functions can access to find variables,
functions and, in general, programming objects.
Python defines four different namespaces. They are dynamically defined, meaning they last for
a limited time. They are created when necessary and destroyed when they are no longer needed.
Usually, several namespaces coexist. They are:

Built-in Namespace. This namespace stores all Python Built-in variables and functions. We
can access these variables whenever we have a Python terminal opened. We list their names by
writing the command

dir(__builtins__)

on the Ipython terminal. For instance, the function print belongs to this namespace. We can
access every function and variable stored in the Built-in namespace from whatever function or
program we write in Python.
 2.5. NAMESPACE AND SCOPE. 51

Global Namespace. We could say that this namespace contains all functions and variables
defined in a script. Nevertheless, we will introduce the importation concept to understand better
what we mean by this. Let’s return to the example above, where we wrote a script with a function
soleq2, allowing us to get a quadratic equation’s roots. We have seen that if we run the script in
Spyder, the function soleq2 is stored in the computer memory, and we can use it straightforwardly
on the Ipython terminal to solve any quadratic function.
But what if we want to use the function soleq2 in another script? Do we need to write down
the equation again in the new script? The answer is not. Python supplies a method to access
the functions and variables of one script from another. The method is known as importation; we
import one script into another using the command import. The following script shows an example,

Example [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Thu Mar 28 17:25:30 2024
4 Esto es un ejemplo para ver el comando import
5 This is an example on the use of the import command
6 @author: abierto
7 """
8 #importamos el script entero esto no siempre es una buena idea
9 #we import the whole script although not always it is a good idea
10 import eq_seg_grado
11 e = 2
12 f = -4
13 g = 1
14 solplus, solminus = eq_seg_grado.soleq2(e, f, g)
15

16 print('Example_import:', solplus)
17 print('Example_import:', solminus)
18

19 print('eq_seg_grado:', eq_seg_grado.solplus)
20 print('eqseg_grado:', eq_seg_grado.solminus)

Line number 10 on Example_import.py example, uses the command import to import the
content of script eq_seg_grado.py. Attention: we write the name of the file without the exten-
sion .py. When program line number 10 is executed, the script eq_seg_grado.py is thoroughly
executed, and a global namespace is created to save the variables and functions created by this
script.
In lines 11 − 13, we define three variables to be used as coefficients for a quadratic equation.
Later, in line 14, we call function soleq2 to solve the equation. But soleq2 belongs to the global
namespace created when the script eq_seg_grado was imported. So, to access it, we write:

eq_se_grado.soleq2

We indicate the namespace first and then the function name, separated by a point. This is how
we access the variables and functions defined in one script from another script.
Why is it convenient to define variables and functions in separated namespaces? One reason is
that it allows repeating the same variable and function names in different locations of our code,
avoiding them clashing accidentally. In the last example, in line 14, we saved the quadratic equation
solutions in variables solplus y solminus. But these variable names have already been used. We
said we were running it when we imported eq_seg_grado. If we review the code of this script we
52 CHAPTER 2. INTRO TO PYTHON

can see that in line 34

solplus, solminus = soleq2(2, 3, 1),
the program calls soleq2 and the results are saved in variables with identical names,
solplus y solminus. Although we are saving the solutions in both cases with the same names,
they belong to different namespaces. After running the script Example_import.py, we achieve the
following result,

In [1]: runfile('C:/Users/abierto/Documents/borrar/Example_import.py',
wdir='C:/Users/abierto/Documents/borrar')
raı́z positiva/positive root: 0.0
raiz negativa/negative root: 0.0
Example_import: 1.7071067811865475
Example_import: 0.2928932188134524
eq_seg_grado: -0.5
eqseg_grado: -1.0

In [2]:

Generally, a programming structure is created whenever we import a script. In Python, these

program structures are called modules. A script and the module generated when the script is
imported share the same name (except for the extension). To access the functions and variables
held in the module, it is enough to put the name of the module before the name of the function or
variable separated with a point.
Except for the functions and variables defined in the Built-in namespace, the remaining code
written in Python is intended to create modules. Usually, the modules are grouped into software
packages which enclose functions and programming objects devoted to specific tasks. In these
lecture notes, we will meet some of them, like numpy, oriented to scientific computing or matplotlib,
devoted to graphical representation.
As we have already said, this script is run when we import eq_seg_grado. Thus, the program
calculates first the solution of the quadratic function: x2 + 2x + 3 = 0 and shows the checking
of the solutions on the Ipython terminal. Once the importation is over, all created functions
and variables are available in the script’s global namespace. Then, we see the results of the two
quadratic equations solved on the Ipython terminal. First, the solutions of 2x2 + 3x + 1 = 0 and
then the solutions of x2 + 2x + 3 = 0. To print these last solutions, we use
eq_seg_grado.solplus
eq_seg_grado.solminus,
To indicate that, we want to print the variables held in the global namespace eq_seg_grado.

Local Namespace. This namespace is used by Python functions. Whenever a function is ex-
ecuted, a specific namespace (local) is created for the function. Every variable created by the
function is saved in its local namespace, and it is available for the function to use during execution.
Once the function ends, its associated local namespace is destroyed, and the variables are no longer
accessible.
Focus on function soleq2 code, we can see how the function creates variables a,b,c in its
header. In the function body, these variables are used as the coefficients of the quadratic function
to be solved. The solutions are saved in variables xp,xm. These five variables belong to the local
namespace of the function soleq2, and only this function code can access them.
When we call function soleq2, whether from another module or the command line, we use
the assignation symbol to copy the output variables and give values to the input variables. For
instance, recall the line number 14 of the Example_import.py module,
2.5. NAMESPACE AND SCOPE. 53

solplus, solminus = eq_seg_grado.soleq2(e, f, g)

Built-in Namespace

Global Namespace

Enclosing Namespace

Local Namespace

Functions inside functions

Functions

Modules

Python

Figure 2.9: Inclusion relationship among Python Namespaces

When the function is called, variables e,f,g, which belong to the global namespace of the
modules Example_import.py are copied in variables a,b,c, which belong to the function local
namespace. When the function at the end executes the command return, the values of variables
xp,xm, which belong to the local namespace of the function, are copied in variables
solplus, solminus
, which belong to the global namespace of module Example_import.py.

Enclosing namespace. Sometimes, we call a function from inside another function. So, our
function soleq2 calls to función print, which belongs to the Python Built-in namespace. When
function soleq2 is executed, it creates its own namespace and when inside it function, print is
executed this last, also creates its own local namespace inside the namespace of soleq2. The local
namespace of soleq2 encloses the namespace of print. In general, when a function calls another,
the namespace of the first function is considered the Enclosing namespace of the second function.
The reason for this name may be better understood by explaining the concept of variable scope.

Scope We have described the concept of namespace as a memory area linked to a Python module
or function. The namespaces make sure that not all variables or functions can be handled from
everywhere. The scope of a variable or a function defines which functions, modules, etc., can
manipulate such variable or function. The scopes are tightly related to namespaces. The rule known
as the “LEGB” rule (Local–>Enclosing–>Global->Built-in) establishes the namespace precedence
order. So, a function has access to all its local variables and function (Local Namespace), to all
ones belonging to the function that calls it, if any (Enclosingnamespace), to all ones belonging
 54 CHAPTER 2. INTRO TO PYTHON

to the module that calls the function (global namespace) to all builtin namespace variables and
functions. It is also true that if we follow the opposite direction from the command line, we can
only access built-in variables and variables created from the command line itself. From a module,
we have only access to variables and functions built-in or created from the module but not to
variables belonging to functions contained in the module. From a function, access is granted to
built-in variables, those belonging to the module that contains the function, those belonging to the
function that includes the function, if any, and those belonging to the function itself. However, a
function has no direct access to variables belonging to functions contained in or called from the
function. Figure 2.9 shows a schematic view of Python namespace nesting.

More on import command. The import command is widely used in Python programming.
We have already described how to use it to import a whole module. Nevertheless, it is possible to
use it in very many different ways,
1 import eq_sec_grado
2 import eq_sec_grado as sec
3 from eq_sec_grado import *
4 from eq_sec_grado import soleq2
5 from eq_sec_grado import soleq2 as cuqui
The module from which the import takes place is the same in every case, but not how the
import is carried out. So, in the line number 1 of the example, we import the module as a whole.
The method to access the variables and function of the module is by using its global namespace
eq_sec_grado.soleq2. In line 2, we are also importing the whole module, but now, we use an
’alias’, sec instead of the module name. This means that to access variables or functions belonging
to the module, we use the alias: sec.soleq2. In line 3, we are not importing the module as it
is. We are just importing all variables and functions included in the module. In this case, we
no longer need to put before the module name to access them. But we should be careful using
this importing method. If the script which has imported the module has variables or functions
whose names coincide with those of the variables or functions imported, the names will clash.
Only the last created ones will be accessible. In name 4, we only imported function soleq2 from
the module. We can use it in the destination script without putting it before the origin module
name. lastly, in line 5 we have imported only function soleq2 but this time we have assigned
it an alias. This means we have to use the alias instead of the proper function name when we
use it in the destination module. That is: we have to write x1,x2=cuqui(1,-3,2), instead of
x1,x2 = soleq2(1,-3,2).

2.6 Debugging
Whenever we write a program, we must check its performance; often, we must fix the errors. The
process that takes us from the original program version to the final one is known as code debugging.
We could differentiate two main groups of errors:

Syntax errors. Broadly speaking, these are spelling mistakes. We have written something
wrong, like a variable or a function name. Also, we make a syntax error if we do not follow the
language rules. The Spyder editor warns us in either case. If we ignore the editor signals and run
the program, we will get an error message in the Ipython console.
For example, figure 2.10 shows a script with several syntax errors. The editor has drawn a bold
red circle with a cross in the centre before the lines containing an error. If we put the mouse atop
the red circles, we get a message with some information on the error made.
2.6. DEBUGGING. 55

Figure 2.10: A sintax error example (a closing bracket has been omited

There is a second type of syntax error that the editor does not detect. It is related to the
properties of variables and functions. An error message will be cast on the Ipython console if we
try to run the program. For instance, if we write the two following code lines,
In [88]: a = 3
In [89]: [Link]()
Traceback (most recent call last):
Cell In[89], line 1
[Link]()
AttributeError:
'int' object has no attribute 'pop'
In [90]:
We get an error message telling us that we cannot extract an element from an integer variable,
as is the case of the variable a=3. For sure, you can only ’pop’ an element from a list.

Codding errors. Errors of this kind are usually more challenging to find out than Syntax errors.
The code runs correctly, but we do not get the expected results. The only way to find the bug is
to review the code line by line. Fortunately, there are specific programs that can carry out this
debugging task. They are called, well, yes, debuggers.
Thanks to its built-in debugger, the Spyder text editor allows us to run a program line by line,
watching the values the variables are taking, etc. To use the debugger, we need to define in the
56 CHAPTER 2. INTRO TO PYTHON

Figure 2.11: Active Breackpoint pointed by a blue arrow and debug drop-down menu

Spyder editor breakpoints, i.e., lines in which Python will stop the program, get into debugging
mode, and wait for the user’s instructions. Figure 2.11 shows the code of the quadratic equation
example, in which we have defined a breakpoint, pushing the mouse left button on top of the space
after the number of the line at which we want to define a breakpoint. Spyder marks the active
breakpoint, drawing a bold red circle in the space after the line number. It is also possible to set
or remove breakpoints, using the Set/Clear Breackpoint button located in the tab Debug on top of
the text editor.
Suppose after activating the breakpoint, we start the debugging mode by pressing the first of
the blue buttons located on top of the text editor (see figure 2.12). In that case, Spyder shows us
that it has stopped the program run at the line marked by the breakpoint red dot (line 35 in the
example). Besides, the Ipython console also shows that it has come into debugging mode.
IPdb [1]: !continue
> c:\users\abierto\documents\borrar\eq_seg_grado.py(35)<module>()
33 #here, we add a 'how to use' example that's not part of the function
34
2--> 35 solplus, solminus = soleq2(2, 3, 1)
36
37 #comprobacion

IPdb [2]:
Now, we can access Spyder debugger’s tools. Figure 2.12 shows the line where the program
execution has been stopped (marked with a green arrow) and some of these tools. It is the same
modules used in previous examples, but we have added a small error in line 28 this time. Spyder
allows us to run the program line by line, get into the function our program calls to and cover it
line by line, or continue the program execution till the next breakpoint or to the program end.
If we select the button “Run the current line,” the program will execute the code line marked
with the green arrow and stop in the following line. In each step, we can check the values the
2.7. FLOW CONTROL 57

Figure 2.12: program stop on a breakpoint and debugging tools

variables so far defined take by asking for them at the Ipython terminal. In the example shown in
the figure, the breakpoint is located in line 35, where the program calls to soleq function. If we
push the button “Run the current line” again, the debugger will execute the function and jump to
line 39. If we want to cover line by line the function soleq, then we may push the button “Get
into the function of the current line”. The performance of the remaining debugger tools is easy to
understand, looking at figure 2.12. If we get into the function soleq and execute its code line by
line, checking their results, we will realise that there is a bug in line 28 because the result cast by
this line is wrong. So we would be ready to review its code and discover the error. Can you find
it?

2.7 Flow control

In the previous section, we introduced how to write Python programs using scripts and functions.
In all the examples shown, the programs ran from the first line and went on, in an orderly manner,
line after line, until the program ended. These programs follow a linear flow because the results
of each line are obtained sequentially, one after another.
Many times, in situations we will describe later, we would like to modify the order in which
the program sentences are executed. Sometimes, we need to repeat a block of code several times,
while other times, we want a chunk of code to be executed only when a predefined condition is
fulfilled.
The control of the order a program follows when executing its sentences is known as flow control.
We will describe two main kinds of flow control: Conditional flow and loops.

2.7.1 Conditional flow.

We begin with a simple example of how and why conditioning the flow. Suppose we want to make
a program that gets a number as input and shows us a message on the screen if the number is
even.
 58 CHAPTER 2. INTRO TO PYTHON

We can use the operator modulus, \%. If we divide the number by two and the remainder is
zero, the number is even; otherwise, it is odd. We may use relational operators, specifically the
== operator, to check if the division by two remainder is zero. Eventually, we need some method
which allows the program to write a message only if the input number is even.

if-elif-else. The Python structure if supplies the method we need. But, let’s see first the code
of the example we are talking about,

numero [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Fri Apr 19 15:31:19 2024
5 Script para crear una función que comprueba si un número es par
6 Script to create a function tha checks if a number is even
7 @author: juan
8 """
9

10 def par(n):
11 '''
12 Esta función toma como entrada un número n. Si n es par devuelve un mensaje
13 indicando que el número es par.
14 This funtion takes an integer number n as imput. If n is even then
15 it shows a message indicating the number is even.
16 '''
17 if n%2 == 0:
18 print(str(n),' es un numero par')
19 print(str(n),'is an even number')
20 return()

The script generates a function par(a). In line 17, we find the definition of an if opening. It
starts with the keyword if followed by an expression that casts a logic result: true (1) or false (0).
This expression could be any valid combination of relational or logical expressions.
In our example, we are comparing the remainder of the division of the input number (n) by
two with zero. The if block opening line always makes up a condition and always ends with a
colon (:) symbol. The body of the if block comprises the code lines that follow the opening line,
which must be indented. The program executes these lines only if the condition defined in the
block opening line is met. In our case, the if block body comprised lines 18 and 19. Eventually,
in line 20, the command return() closes the function. Notice how this line is not indented like
the lines of the if body. It is instead aligned with the lines of the function body because it is not
part of the if block. Thus, if we introduce an odd number into the function, it will skip the lines
of the if block body and will go straight to line 20.
Below are some code lines that demonstrate the function par performance. To test it, we
imported the function to the IPython terminal and called it twice - first with the input variable
of 3 and then with the input variable of 8. When we passed 3, the condition was not met because
the number was odd. Therefore, the function skipped the conditional block and directly executed
the return command. Since we didn’t ask the function to return anything, it returned an empty
bracket. On the other hand, when we passed 8, the number was even, and it fulfilled the imposed
condition to execute the if block. As a result, the function printed the stated message in the
IPython terminal. Once the if block has been completed, the program executes the command
return, as in the even number case.
 2.7. FLOW CONTROL 59

In [2]: from numero_par import par

In [3]: par(3)
Out[3]: ()
In [4]: par(8)
8 es un numero par
8 is an even number
Out[4]: ()
We have seen the simplest conditional if structure. We could make it a bit more complex, also
asking for a message when the input number is odd. This means we want to include a disjunctive
in our program; if the number is even, the program should do one thing; if the number is odd, it
should do another one. To include this kind of disjunctive in an if structure, we use the keyword
else. We can see below our example after being modified,
numero par [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Fri Apr 19 15:31:19 2024
5 Script para crear una función que comprueba si un número es par
6 Script to create a function tha checks if a number is even
7 @author: juan
8 """
9

10 def par(n):
11 '''
12 Esta función toma como entrada un número n. Si n es par devuelve un mensaje
13 indicando que el número es par.
14 This funtion takes an integer number n as imput. If n is even then
15 it shows a message indicating the number is even.
16 '''
17 if n%2 == 0:
18 print(str(n),' es un numero par')
19 print(str(n),'is an even number')
20 return()
21

22 def par2(n):
23 '''
24 Esta función toma como entrada un número n y muestra un mensaje
25 indicando si el número es par o impar
26 This function takes an integer number n as input and shows a message
27 stating whether the number is even or odd
28 '''
29 if n%2 == 0:
30 print(str(n),' es un numero par')
31 print(str(n),'is an even number')
32 else:
33 print(str(n),' es un numero impar')
34 print(str(n),'is an odd number')
35 return()

We have added a second function called par2 to our script. The code is identical to the par
function until we reach the else keyword. At that point, the disjunctive is set. If the input
 60 CHAPTER 2. INTRO TO PYTHON

number is even, the program will execute the code lines between the if and else. Otherwise, it
will execute the indented lines beneath the else. In both cases, the program will execute line 35,
which does not belong to the conditional block.
The following code lines show the result of applying the function par2 to the numbers 3 and 8.

In [11]: from numero_par_mod import par2

In [12]: par2(3)
3 es un numero impar
3 is an odd number
Out[12]: ()

In [13]: par2(8)
8 es un numero par
8 is an even number
Out[13]: ()
The if structure allows to expand the number of possible elections even more, using the keyword
elif. As in the case of the keyword if, elif is followed by a logical expression which establishes a
condition. The following indented code lines will be executed if the condition is fulfilled. Otherwise,
the program will jump to the next line with a keyword: another elif, an else or straightforwardly
to the next non-indented line, exiting the conditional block. We are going to see how it works,
modifying our last example. Now we want the program to check when the input number is odd,
whether it is divisible by three,

numero div2 [Link]

1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Fri Apr 19 15:31:19 2024
5 Script para crear una función que comprueba si un número es par y si no lo es
6 comprueba si es divisible por tres
7 Script to create a function tha checks if a number is even
8 @author: juan
9 """
10

11

12 def parthree(n):
13 '''
14 Esta función toma como entrada un número n y muestra un mensaje
15 indicando si el número es par si no comprueva si es divisible por 3
16 This function takes an integer number n as input and checks if the
17 number is even. If not it checks if the number is divisble by 3
18 '''
19 if n%2 == 0:
20 print(str(n),' es un numero par')
21 print(str(n),'is an even number')
22 elif n%3 ==0:
23 print(str(n),' es un numero divisible por tres')
24 print(str(n),'is a number divisible by three')
25 else:
26 print(str(n),' es un numero impar pero no divisible por tres')
 2.7. FLOW CONTROL 61

27 print(str(n),'is an odd number but it not divisible by three')

28 return()

Structure if-elif-else
if condition
... code
...
(elif condition)
... code
...
(elif condition)
... code
...
.
(it could be as many elif blocks as needed)
.
(else)
... code
...

Figure 2.13: General outline of an if conditional flow structure. Terms enclosed in parenthesis are
optional

If we introduce now an even number in the function parthree, it executes the code beneath
the if and before the elif. It skips all the remaining code till the end of the conditional block. If
the introduced number is odd and divisible by three, the function skips the code beneath the if
and executes the code between the elif and the else. It skips the remaining code till the end of
the conditional block. Lastly, if the number is odd and is not divisible by three, it only executes
the code beneath the else.

Note that the program will execute the code belonging to the first condition met and skip the
remaining code till the end of the conditional block. For instance, if we introduce the number 6 in
our example function, the program will show the message “6 is an even number” because this is
the first condition it meets. But it never will show us the message ‘6 is a number divisible by three
because, once the program checks that the first condition is fulfilled (be even), the program jumps
to the end of the conditional structure without further checking. Figure 2.13 shows a complete
outline of the if structure. Terms enclosed in parentheses are optional and could or may not be
included in a specific program implementation. The following commands show the import and use
of function parthree, using different numbers as inputs.
 62 CHAPTER 2. INTRO TO PYTHON

In [18]: from numero_div2_div3 import parthree

In [19]: parthree(4)
4 es un numero par
4 is an even number
Out[19]: ()
In [20]: parthree(6)
6 es un numero par
6 is an even number
Out[20]: ()
In [21]: parthree(9)
9 es un numero divisible por tres
9 is a number divisible by three
Out[21]: ()
In [22]: parthree(7)
7 es un numero impar pero no divisible por tres
7 is an odd number but it not divisible by three
Out[22]: ()

Nested if structures In the previous example, we saw that the program could not detect if the
input number was both even and divisible by three due to the exclusive nature of the if statement.
We can solve this problem using nested if structures. It is a straightforward idea. We build an
if statement to check a specific condition. If the condition is fulfilled, inside the if statement,
we build another if statement to check a second condition, and so on as many times as needed.
Notice that in a nested if structure, the inner condition is checked only when the outer one is
met. We can modify our previous example, using now a nested if,

numero parytres [Link]

1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Fri Apr 19 15:31:19 2024
5 Script para crear una función que comprueba si un número es par
6 Si lo es, tiene un if anidado para probar si además es divisible entre tres
7 si no lo es comprueba si es divisible por 3.
8

9 Script to create a function tha checks if a number is even and if it is so,

10 checks if it is divisible by three to. else, the program checks if the number
11 can be divided by 3
12 @author: juan
13 """
14

15 def parthreenested(n):
16 '''
17 Esta función toma como entrada un número n. Si n es par devuelve un mensaje
18 indicando que el número es par.
19 Si no, comp
20 This funtion takes an integer number n as imput. If n is even then
21 it shows a message indicating the number is even.
22 '''
23 if n%2 == 0:
24

25 if n%3 == 0:
 2.7. FLOW CONTROL 63

26 print(str(n),' es un numero par y divisible entre 3')

27 print(str(n),'is an even & divisible by 3 number')
28 else:
29 print(str(n),' es un numero par')
30 print(str(n),'is an even number')
31 elif n%3 == 0:
32 print(str(n),' es un numero divisible entre 3')
33 print(str(n),'is an number divisible by 3')
34 else:
35 print(str(n),' es un numero no es divisible entre 2 ni entre 3')
36 print(str(n),'is not divisible by 2 neither by 3 ')
37

38 return()
39

We can see that we have added code lines 23 − 24. This added code defines a new if statement
defined inside the if structure that starts at line 20. The program will only check this new
condition if the number is even and it has met the condition imposed in line 20. If we focus on the
code structure, we can see that it has three indentation levels: The first one belongs to the code of
function parthreenested, the second one to the outer if structure and the last one to the nested
if. If we would add a new if inside the last one, we should indent the code one level more.

In [41]: from numero_parytres_nested import parthreenested as piubello

In [42]: piubello(12)
12 es un numero par y divisible entre 3
12 is an even & divisible by 3 number
Out[42]: ()

In [43]: piubello(5)
5 es un numero no es divisible entre 2 ni entre 3
5 is not divisible by 2 nor by 3
Out[43]: ()

In [44]: piubello(4)
4 es un numero par
4 is an even number
Out[44]: ()

In [45]: piubello(9)
9 es un numero divisible entre 3
9 is an number divisible by 3
Out[45]: ()

2.7.2 Loops
Sometimes, it is necessary to repeat an operation a number of times or till a particular condition is
fulfilled. High-level program languages have specific structures to repeat a chunk of code as many
times as needed. Each repetition is named an iteration. These structures receive the generic name
of loops. We will see two types of loops: for loops and while loops.
 64 CHAPTER 2. INTRO TO PYTHON

For loop structure

for index in [Values list]
...code
(condition: break)
...code
(condition: continue)
... code

Figure 2.14: for structure General outline. Terms enclosed in parentheses are optional.

For loops A for loop repeats the sentences inside the loop a predefined number of times. That
is, it performs a fixed number of iterations. A general outline of the for loop structure is depicted
in figure 2.14.
The loop starts with the keyword for followed by a variable we have called index. After,
we write the keyword in, a list of values and a colon :. The index (variable) will take all values
contained in the [list of values] one after another. The code contained in the for loop, from the line
just beneath the for, should be written using indented lines, in the same way that we seen for the
conditional structures. Inside the loop, this code will run as many times as items the list of values
has. Let’s have a look at some examples before describing the break and continue sentences.

numero [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Mon Apr 29 15:08:34 2024
5 Este modulo contiene algunos ejemplos del uso de los bucles for
6 This module contains some examples of for loops use
7 @author: juan
8 """
9

10 def sumalist(A):
11 """
12 Input A should be a list of numbers no mater how long
13 the function shows the values on the screen, sum them and save the result
14 in the returned variable y
15

16 La entrada A debe ser una lista de números the cualquier longitud.

17 La funcion muestra los valore de la lista, los suma y devuelve el resultado
18 en la variable y
19 """
20 y = 0 #definimos la variable en la que guardaremos el resultado de la suma
21 #define the variable to save the result of the sum
22 for i in A:
23 print(i)
24 y = y + i
25 return y
26

27 def sumaelementos(A,B):
28 """
 2.7. FLOW CONTROL 65

29 Inputs A and B shoudd be two list of numbers with the same length
30 The funtion calculates the sum of the elements of A and B that share the
31 same location inside their respective list i.e. The result is save
32 in list S in the same location: S[i] = A[i]+B[i]
33 """
34

35 if len(A) != len(B):
36 print('Las listas que me das no son igual de largas')
37 print('Sling your hook! The lists lengths are different. ')
38 return([]) #we return an empty list
39 else:
40 S = []
41 index = range(len(A)) #index to cover the elements of A&B
42 for j in index:
43 [Link](A[j] + B[j]) #adding the elements in the same location
44

45 return(S)
46

47 def firsttime(A,b):
48 """
49 esta función busca la primera ves que un número b aparece en una list
50 A y devuelve su posición dentro de la lista.
51 Si no aparece muestra un mensaje por pantalla
52 This program find the first time a number b apears in a list A
53 """
54 p = 0
55 for i in A:
56 if i==b:
57 break
58 p = p + 1 #p += 1
59 if p == len(A):
60 print('el número', b, ' no está en la lista')
61 print('number', b, 'is not included in the list')
62 else:
63 print('el número', b, 'ocupa en la lista la posición', p )
64 print('number', b, 'is located at position', p, 'in the list' )
65 return(p)
66

67 def buscapar(A):
68 """
69 Esta funcion busca los numeros pares de una lista de numeros y
70 crea una nueva lista solo con los pares contenidos en la lista inicial
71 """
72 B = [] #and empty list to save the even numbers
73 for i in A:
74 if i%2 !=0:
75 continue
76 [Link](i)
77 return B
78

79

80 def listadelistas(A,b):
 66 CHAPTER 2. INTRO TO PYTHON

81 """
82 Esta funcion busca las veces que aparece un número en una lista
83 bidimensional (una lista de listas de números)
84 Devuleve las veces que aparece el número y los indices de las posisiciones
85 en que aparece.
86 """
87 ind = [] #an empty list to save the list indexes where the number is.
88 counter = 0 #number of times the number is repeated in the list
89 for i in range(len(A)):
90 for j in range(len(A[i])):
91 if A[i][j] == b:
92 [Link]((i,j))
93 counter = counter+1
94

95 return(ind,counter)
96

97 def pairs(A,B):
98 """
99 A and B are two list of numbers of the same lenght. The function build a list
100 of pairs, taken one element from and one element from b with the condition that
101 they should be different. It doesn't the mater the order i.e (1,2) and (2,1)
102 are different a valid'
103

104 """
105 pares = [] #an empty list
106 for i in A:
107 for j in B:
108 if i != j:
109 [Link]((i,j))
110 return(pares)

The function sumalist takes a whatever length list of numbers as an input and returns the sum
of the number in the list as an output. Besides, it writes each number of the list in the Ipython
console. The for starts in line 22. The first time the program arrives at this line, it assigns to
variable i the first element of the list A. In line 23, the program shows the value of this element on
the screen. In line 24, it adds this value to variable y, and the program goes back to line 22 and
assigns to variable i the second value of the list A. This process repeats until the elements of the
list A have been exhausted. Then, the program finishes the for loop and runs line 25 returning
the value of variable y, which will have accumulated the sum of list A elements.
The function sumaelementos takes two lists of equal length as input variables and returns a
list with the sums of the elements located at the same position on the input lists. A conditional
structure protects the code so that if both input lists are not the same length, the program casts
a message on the screen and returns an empty list. If both lists have the same size, in line 41,
the program creates the variable index using the command range. Therefore, index will be a list
of numbers from 0 to length of the input lists−1. In line 42, the for loop stars, using this list
i in index. In line 43, the elements of the input lists, located at the position indicated by the
value of j, are summed, and the result is appended to the list S. When the loop has covered all
the elements of the input lists, the loop is over, and the program returns the value of the list S.
It is interesting to note the method for covering the elements of a data structure using the range
command because, in programming, it is a very commonly used method.
Before proceeding with the code examples, we will return to figure 2.14. There are two keywords
2.7. FLOW CONTROL 67

included in the general for structure layout that allows to interrupt or alter the execution of a for
block. Both should always be included in a conditional statement. The keyword break interrupts
the execution of the for loop that encloses it, and the program jumps to the following line after
the for block. The keyword continue skips every code line beneath it, belonging to the for block
and starts a new iteration. We will see how they work, resuming the study of the example code
included above.
The function firsttime searches for the first time that a number b appears in a list A, returns
the position the number is located at in the list and finishes the program execution. The for
loop starts at line 55. The if structure, enclosed in the for loop, checks if the element i of the
list meets the wanted number. if so, the program executes the break command, and the program
returns the position p of the number in the list. If the element i of the list A does not match
the wanted number, the program increments the value of variable p in 1 and goes back to the
beginning of the for loop and will try the next element of the list. If the wanted number is in
the list, the final value of p will be less than the list length, and the Ipython console will display
a message indicating that the number is in the list and the position the number has in the list. If
the program can not find the number in the list, the for loop is not interrupted, and the Ipython
console will display a message indicating that the number is not in the list 5 .

The function buscapar searches for the even numbers included in a list and builds a new list
with the even numbers contained in the original list. The program checks in line 71 if the number is
odd. If this is the case, the continue command makes the program skip line 73, avoiding including
the odd numbers in list B, which will only have even numbers when the program finishes.

Nested for loops. The for loops can be nested one into another, likewise to how we nested if
structures. Functions listadelistas and pairs show a couple of examples of how to do it.
The function listadelistas takes a bidimensional list – a list A in which each element is, in
turn, a list of numbers– as input and searches how many times a number also supplied to the
function as an input, is enclosed in the list. It performs this task using two nested for loops.
The first (outer) loop starts at line 86, defining an index i, which will cover the list. A second
(inner) for loop starts at line 87, creating a second index j. This second index will cover each list
of numbers included in list A. The inner for loop has a conditional structure inside that checks
if the number that takes position j inside the number list i meets the wanted value. If so, the
index (i,j) are saved as a tuple in list ind and the variable counter, which counts the times the
wanted number has been found in the list, is increased in one unit. The following code presents
an example of the function performance,

In [12]: LoL =[[1],[1,2,5],[2,-1,5,3,5],[4,3,2,0,5],[2,1]]

In [13]: from ejemplo_for import listadelistas

In [14]: indice, contador = listadelistas(LoL,5)

In [15]: indice
Out[15]: [(1, 2), (2, 2), (2, 4), (3, 4)]

In [16]: contador
Out[16]: 4
5 there are better ways to do this. This example is intended only to show the performance of the break statement.
 68 CHAPTER 2. INTRO TO PYTHON

The function shows us the times that number 4 is included in the list LoL. We leave you to try
to discover how the function pairs works. (if you have followed the reading till now, you should
find it out with your hands tied behind your backs).

List comprehension. Python has excellent resources for iterating over different data structures.
We will not present you many of them, but just one very useful and widely used by Python
programmers: List comprehesion. The idea is to build a list whose elements result from iterations
on several other lists. Its general structure is,

newlist = [expression for item in iterable if condition == True]

The new list is built from an operation (expression) carried out on any element (item) of another
(iterable) data structure. Moreover, it is possible to add a condition for the operation to be carried
out. The following Python script contains a couple of examples.

List comprehesion [Link]

1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Mon May 13 08:59:21 2024
5 Un par de funciones usando list comprehesion
6 A pair of function using list comprehesion
7 @author: juan
8 """
9

10

11 def lc_demo(A):
12 """
13 ESta función busca los numeros pares en una lista A y los guarda en
14 una nueva lista B
15 This function finds the odd number in list A, a created a new list
16 B with them
17 """
18 B = [i for i in A if i%2 != 0]
19 return B
20

21 def pair_cmh(A,B):
22 """
23 A and B are two list of numbers of the same lenght. The function build a list
24 of pairs, taken one element from and one element from b with the condition that
25 they should be different. It doesn't the mater the order i.e (1,2) and (2,1)
26 are different a valid'. But, but we will use a list comprehension
27

28 """
29 pares = [(i,j) for i in A for j in B if i != j]
30 return(pares)

The function lc_demo makes a list with the odd numbers contained in another list. It is easy
to understand the structure of the comprehension list created in line 18: List B is built up by the
elements i, included in list A which fulfil i\%2 != 0, that is, those elements that are not divisible
by two and, thus, odd numbers.
 2.7. FLOW CONTROL 69

while Loop structure

while condition:
... code
(condition: break)
...
(condition: continue)
...code

Figure 2.15: General structure for a while loop. Terms enclosed in parentheses are optional.

The second function, pair_cmh, is slightly more complex, but it allows us to appreciate the
possibilities of the method. The function creates a list of tuples. To do so, it uses a pair of for
structures, written in a way that is equivalent to two nested for statements. Hence, the function
compares each element, i, from the first list with any element, j, from the second one. If the
elements are different i != j, it saves the elements as a tuple (a,b) in the list pares.

While loop This kind of loop has the same purpose as the for loop, to repeat a chunk of code
several times. The main difference is the mechanism determining the number of iterations the
loop will take. For a while loop, the iterations are repeated an undefined number of times while
a condition imposed at the beginning of the loop is fulfilled. Figure 2.15 shows an outline of the
while loop structure. As in every Python programming structure we have been so far, the code
of the while loop should be indented. The following script shows some examples of how to use it.

examples [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Mon May 13 10:46:35 2024
5 Un par de ejemplos de uso de bucles while
6 A pair of loop while examples
7 @author: juan
8 """
9

10 def pmax(a,M):
11 """
12 This funtion optain the minimum exponent n such that a**n >= M
13 esta funci-on optiene el minimo esponte n tal que a**n >= M
14 """
15 n = 0
16 while a**n < M:
17 n = n+1
18 return n
19

20 def L_suma(A,B):
21 """
22 This function take tho list of list of numbers A and B and adds the numbers
23 located at the same position in both list. (both lists should have the
24 same dimensions)
25 Esta función toma dos listas A y B y crea una nueva lista con la suma de los
 70 CHAPTER 2. INTRO TO PYTHON

26 números que ocupan la misma posición (ambas listas deben tener las mismas
27 dimensiones)
28 """
29

30 i = 0 #indice de para recorrer las listas de números

31 #numbers to cover the list of number list
32 C =[]
33 n_lista = len(A)
34 while i < n_lista:
35 n_num = len(A[i])
36 j = 0 #indice para recorrer los numeros de cada lista de numeros
37 #index to cover the number of each number list
38 [Link]([])
39 while j < n_num:
40 C[i].append(A[i][j]+B[i][j])
41 j = j+1
42 i = i+1
43 return C

The function pmax takes two numbers as input and calculates the exponent you have to raise
the first number to obtain a result greater than the second one. The while loop code starts at
line 16, checking whether the first input variable raises to n is less than the second input variable.
When this condition is fulfilled, the program executes line 17 that increases in one unit the value
of n. Then, the program returns to line 16 and checks if the condition is met for a new value of
n. This process goes on until the condition is no longer met. At this moment, the program exits
the while loop and executes the return instruction, returning the last value of n the program
calculated.
A critical point to consider when programming a while is ensuring that the input to the loop
condition will change inside the loop. Otherwise, the program will get into an infinite loop and
never stop6 . The commands break and continue are identical to those described for the for
loops. For this reason, we will not insist on them any more.

Nested while loops. In a similar way as with the for loops, it is possible to nest while loops.
We have defined the function L_suma, in the above code example, which uses two nested while
loops to add the values of two lists located at the same position in each list. The lists should have
the same dimension, the same number of lists of numbers and the same number of numbers in each
list of numbers. We first define an index to cover the lists of numbers contained in the input lists
A and B. Then, a new empty list C is created. An outer while loop starts at line 34. This loop
will run while the index i is less than the number of list of numbers contained in the input lists.
Once the program gets into this outer loop, the program checks first of all how many numbers the
list i contains and appends a new empty list (of numbers) to list C. The program then gets into
the inner while loop, which covers the values of the lists of numbers i contained in the input lists,
adds these values between then and appends the result to the list C. Any time a list of numbers is
completed, j = n_num, the program jumps off the inner loop, increases in one unit the value of i,
checks how many numbers are in the new list of numbers, adds a new empty list to C and get again
into the inner loop. The process goes on till the lists of numbers are exhausted, i = n_lista;
when this condition is met, the program gets out the outer while loop and returns the value of C.
6 If you get into this situation for an error in the program design, please don‘t panic: press the board keys Control

+ C simultaneously and breathe normally.

 2.7. FLOW CONTROL 71

2.7.3 Recursive functions.

A recursive function is a function that calls itself. We have waited till now to speak of them
because, in some ways, they behave like a loop and need an output condition to stop their execution.
Speaking in general, they are difficult to deal with and tend to be very memory-consuming because
any time a function calls itself creates a new memory independent memory area. We are going to
see an example of a recursive function which allows for obtaining the Fibonacci series n term.

f0 = 0, f1 = 1, f2 = 1, f3 = 2, f4 = 3, f5 = 5, f6 = 8, · · · fi = fi−1 + fi−2 · · ·

The series starts with the terms 0 and 1, and later on, each term is the sum of the previous
two. We can straightforwardly cast this definition into code,

ejemplo [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Mon May 13 12:38:14 2024
5

6 @author: juan
7 """
8

9 def fibonacci(n):
10 """
11 Parameters
12 ----------
13 n : TYPE integer
14 DESCRIPTION. n is the fibonacci series term whose value we whant to
15 calculate
16 n: es el término de la serie de fibonacci cuyo valor queremos calcular
17

18 Returns
19 -------
20 s : TYPE integer
21 DESCRIPTION the value of fibonacci series term n
22

23 """
24 if n < 2:
25 #the value of the term is n itself
26 s = n
27 else:
28 s = fibonacci(n-1)+fibonacci(n-2)
29 return(s)
30

31

32 def fibo_series(n):
33 """
34 Parameters
35 ----------
36 n : TYPE integer
37 DESCRIPTION. Last number of the series whe want to build
38
 72 CHAPTER 2. INTRO TO PYTHON

39 Returns
40 -------
41 Fs: List with the n first fibonacci series terms.
42

43 """
44 Fs = [fibonacci(i) for i in range(n)]
45 return(Fs)

if n is less than or equal to two, the function fibonacci returns the corresponding term (1 or
0). If n is greater than two, the function calls itself with inputs n-1 and n-2, to calculate the value
of the nth term of the series by adding the values of the previous two. The function will be calling
itself till it arrives at n ≤ 2. From then on, it will be returning the values of the terms obtained in
each call till it gets the nth one.
the second function fibo_series, uses a list comprehension to create a list with the n first
terms of the Fibonacci series7 .

2.7.4 Algorithms and flow charts.

According to the Oxford Dictionary, an algorithm is defined as:
2. Mathematics and Computing. A procedure or set of rules used in calculation and
problem-solving; (in later use spec.) a precisely defined set of mathematical or logical
operations for the performance of a particular task.
The word algorithm is a transliteration of the Arab mathematician Al-Juarismi name (circa 780-
850 ac). Algorithms are fundamental in programming due to they are the previous step to program
creation.
Usually, we depart from a problem we have a precise statement for. For instance: Obtain the
first n prime numbers.
The next step would be to define an algorithm that allows us to solve the problem. It is
important to realise that the same problem can, in many cases, be solved by different methods.
Therefore, it is possible to design different algorithms to solve the same problem. An algorithm
for the odd numbers problem may be as follows,

• Take number 2 as the first prime number. (A prime number is such that is only divisible by
itself and by one)
• Cover all odd numbers from 3 till the number n of prime numbers required is completed.
• For each odd number, try to divide it between all prime numbers discovered so far. If the
number is not divisible by any of them, then the number is prime. save it and try the next
odd number. Otherwise, i.e. if it is divisible for one of them, break the process and try the
next odd number.

Sometimes understanding an algorithm is easier if we represent it graphically using a flow chart.

Flowcharts use well-defined symbols to represent the steps of an algorithm and arrows to indicate
the relationship among the steps; how the information flows from one step to another.
There is no rigid form to designing a flow chart; the degree of detail description used in the
algorithm depends on the programmer’s needs or the needs of the people the flow chart addresses.
7 If the number n is to large, it is possible that the maximum number of recursive function calls allowed by Python

be reached. In this case, the program casts an error. Besides, we could modify the function fibonacci to obtain
the list with the first n terms of the series, with less effort.
2.7. FLOW CONTROL 73

Algorithm start and end

procedures

Conditional flow

Figure 2.16: Symbols used in flowcharts

The basic idea is to use the chart to ease the algorithm’s understanding. There are several symbols
to indicate procedures, conditions, data storage, etc. Figure 2.16 shows the three most common
chart symbols.
We use an ellipse to indicate the beginning and end of an algorithm. To indicate a procedure,
for instance, to perform a computation, to assign a value to a variable, etc., we use a rectangle.
finally, we use a rhombus as a symbol to indicate a condition.
The symbols are related by arrows to establish the direction of the program flow. Rhombus
usually have two exit arrows marked with the words ’yes’ and ’not’ to indicate where the program
flows depending on whether the condition is met or not.
Finally, a loop is usually represented by an arrow which comes back to a previously covered
symbol.
Figure 2.17 shows a possible flow chart for the prime numbers problems. As you can see, it has
more information than the text version of the algorithm we gave above.
The arrows describing the information flow show that we must implement an outer loop until
the required number n of prime numbers is fulfilled. We will also need an inner loop if any odd
number we test is divisible by any of the prime numbers found so far.
There is a third condition which should break the checking when we find the first odd number
being a divisor for the odd number the program is checking.
From the chart flow, it is easy to obtain the programming structures needed to build a code
that allows us to solve the prime numbers problem. So, it seems reasonable to implement the outer
loop using a while loop, a for loop –that computes as many iterations as prime numbers have
been found so far– for the inner loop, a break to interrupt the checking process, etc.
Of course, it is possible to build a more detailed flow chart, even including explicitly part of the
code to be included in the program. For instance, we may indicate that we will use the command
\% to check if a number is divisible by another. Nevertheless, it is essential to note that you
should be careful to avoid an excess of details that make it difficult to understand the logic of the
algorithm described by the flow chart.
Eventually, we have to cast the algorithm into code, generating a computer program that allows
us to solve the problem. We need to identify the algorithm instruction with valid programming
structures: loops, conditionals, etc.
The following code shows two functions for calculating the n first prime numbers.
The first function uses a flag variable to detect when an even number is not divisible for any
prime number less than it. Try to understand how it works.
The second one uses a Python special for structure, which we have not spoken of so far. it
74 CHAPTER 2. INTRO TO PYTHON

Start

Insert how many prime numbers you wish to get (n)

save number 2 as the first prime number

Have you got the quantity of prime number required? End

Yes

Not

Take the next odd number

Take the first prime number of those already found

Is the odd number divisible by the prime one?

Yes

Add the odd number

to the found Not
primes list

Is the list of found prime numbers ended?

Yes

Not

Take the next number from the found prime numbers list

Figure 2.17: Flow chart for the prime numbers problem

 2.7. FLOW CONTROL 75

is the for - else structure. If we focus on code lines 37 to 41, we shall see a for structure in
which we check if the odd number we are checking is divisible by any of the previously found prime
numbers. In line 41, there is an else instruction. It is linked to the for loop and the program
will execute this instruction only if the for loop reaches its end. In our case, the program reaches
this else command only if the break instruction inside does not interrupt the for loop. In other
words, this happens only if the odd number we are checking is not divisible by any of the previously
found primes and, therefore, it is the next prime number. Then the program gets into the else
block and executes the single line it contains, saving the odd number in the list of prime numbers.
It is needless to say that the code within the ”else” statement should be properly indented.

[Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Tue May 14 11:22:29 2024
5 funcióne para obtener los primeros n numeros primos
6 En la primera empleamos un flag.
7 En la segunda empleamos la estructura for else
8 funtion to get the first n prime numbers
9 @author: juan
10 """
11

12 def numpr(n):
13 primos = [2] #introducimos 2 como el primer primo
14 #we take 2 as the first prime number
15 i = 0
16 impar = 3 #tomamos 3 como el rpier impar para probar
17 #we take 3 as the first odd number to test
18 while i<n:
19 pr = 1 #flag activation
20 for i in primos:
21 if impar%i == 0:
22 pr = 0 #flag deactivation
23 break
24 if pr ==1:
25 [Link](impar)
26

27 impar = impar + 2
28 return(primos)
29

30 def numprels(n):
31 primos = [2] #introducimos 2 como el primer primo
32 #we take 2 as the first prime number
33 i = 0
34 impar = 3 #tomamos 3 como el primer impar para probar
35 #we take 3 as the first odd number to test
36 while i<n:
37 for i in primos:
38 if impar%i == 0:
39 break
40 else:
41 [Link](impar)
 76 CHAPTER 2. INTRO TO PYTHON

42

43 impar = impar + 2
44 return(primos)
45

46

47

48

2.8 Reading and writing data in Pyhton

Storing data is important for working in multiple sessions and sharing results. Since when Python
shuts down, all variables in memory are lost, data must be stored in some other way. Sometimes
it is also necessary to read data that is stored in a file.

Text files A text file, many times saved with the .txt extension, is a file that only contains plain
text. To work with text files we need to use the open function, that returns a file object. It is
usually used with two arguments:
f=open(filename, mode)
f is the file object returned by the function, filename is a string with the name and path of
the text file to be opened and mode is another string indicating how the file is to be used. Some of
the most common modes are:
• ’r’ is the default mode. It indicates that the file is opened for reading only.
• ’w’ opens the file for writing to. If the file does not exist, it creates it.
• ’a’ is the append mode. Adds text to the end of the file. If the file does not exist, it creates
it as in ’w’ mode.
• ’r+’ opens a file (but does not create it) for reading and writing to it.
• ’w+’ opens a file for reading and writing, creates it if it does not exist. Discards what was
previously written to it.
• ’a+’ opens a file for reading and writing, creates it if it does not exist. Adds data to the end
of the file.
File type objects have methods that allow us to access and modify their content. Some of these
methods are:
• write which allows writing data to the file
• read reads all the data contained in the file and returns it
• readline reads a line (up to the end-of-line character) from the file
• readlines reads the whole file but returns a list where each line is one of the elements
In the following example you can see the code to use some of the above methods. You open a
file and write 5 lines to it. Then read the file. The file is reopened but in append mode, writing
a line to the end of the file. It is read but using the readlines method, so that what you get is
a list where each element is one of the lines. Finally the file is reopened and read, in the variable
line each line, and the loop is stopped when the readline method does not return anything.
2.8. READING AND WRITING DATA IN PYHTON 77

f=open('[Link]','w')
for i in range(5):
[Link](f"Line {i}\n")
[Link]()

f=open('[Link]','r')
print([Link]())
[Link]()

f=open('[Link]','a')
[Link]("Another line")
[Link]()

f=open('[Link]','r')
print([Link]())
[Link]()

f=open('[Link]','r')
while True:
line=[Link]()
if not line:
break
print(line)

[Link]()

Line 0
Line 1
Line 2
Line 3
Line 4

['Line 0\n', 'Line 1\n', 'Line 2\n', 'Line 3\n', 'Line 4\n', 'Another line']
Line 0

Line 1

Line 2

Line 3

Line 4

Another line

Working with numbers and arrays In many cases we will need to save and read numbers or
arrays. We could use the above methods to save the numbers or arrays to a file and read them
back. However, a better way to do this is to use the numpy package to directly save and read an
array.
78 CHAPTER 2. INTRO TO PYTHON

Within numpy we have the methods

savetxt and loadtxt to save arrays to text files or load them from text files.
To use savetxt we have to provide the following arguments: a string with the name of the file
and the variable containing the array to be saved. In addition, you can optionally provide more.
For example the format, under the key of fmt (in the example the format is a number (%), with
three decimal digits (.3) of type float (f). The type of delimiter between elements, a header etc.
You can consult the documentation to see them all.
The loadtxt method allows us to read an array from a file and store it directly into a numpy
array variable. To do this we simply provide the file where the array is stored and the method
will return the array. In the example you can see how a 2x3 array is stored in a text file, with a
header, how it is read using the open and read functions and how the array is read and stored in
a numpy array using the savetxt method.

import numpy as np

arr=[Link]([[0.3,4.3,3.5],[-1.23,0.032,-3.23]])
[Link]('[Link]', arr,fmt='%.3f',delimiter=' ',header='Col1 Col2 Col3')
f=open('[Link]','r')
print([Link]())
[Link]()
#Now we read it with loadtxt

m1= [Link]('[Link]')
print(m1)

# Col1 Col2 Col3

0.300 4.300 3.500
-1.230 0.032 -3.230

[[ 0.3 4.3 3.5 ]

[-1.23 0.032 -3.23 ]]

Data in CSV format There is a lot of scientific data that is stored in the Comma Separated
Values (CSV) file format. This is a text file that uses a comma to separate values. It is a very
useful format that allows large tables of data (numbers and text) to be stored in plain text. Each
line (row) of data is a data record, and each record consists of one or more fields, separated by
commas. It can also be opened with Microsoft Excel, and rows and columns can be displayed.
Python has its own csv module that can handle reading and writing the csv file. We will not
introduce this csv module here. Instead, we will use the numpy package to handle the csv file since
we will often read the csv file directly to a numpy array.
Using the same savetxt method of numpy from the previous section, we can indicate that the
delimiter character we want to use to separate the data is a comma. When reading the data with
loadtxt we will also have to indicate that the comma has been used to separate some columns
from others, as shown in the following example.

import numpy as np

# Create a random 20x4 matrix

2.8. READING AND WRITING DATA IN PYHTON 79

m2=[Link]([20,4])

#Save it as a csv file

[Link]('[Link]',m2, fmt='%.2f', delimiter=',')

#Read ir back with loadtxt

m3=[Link]('[Link]',delimiter=',')

print(m3)

[[0.08 0.91 0.36 0.19]

[0.61 0.22 0.7 0.66]
[0.77 0.57 0.77 0.4 ]
[0.58 0.32 0.89 0.78]
[0.63 0.09 0.97 0.78]
[0.35 0.7 0.5 0.96]
[0.65 0.78 0.04 0.04]
[0.96 0.31 0.45 0.26]
[0.75 0.51 0.09 0.87]
[0.93 0.13 0.8 0.61]
[0.67 0.99 0.83 0.76]
[0.21 0.72 0.56 0.46]
[0.81 0.82 0.67 0.88]
[0.12 0.01 0.02 0.55]
[0.88 0.08 0.95 0.7 ]
[0.74 0.84 0.56 0.3 ]
[0.77 0.73 0.67 0.89]
[0.44 0.23 0.67 0.18]
[0.61 0.4 0.06 0.1 ]
[0.55 0.35 0.89 0.11]]

∞ (
Ξi (t)
i=0

JSON files JSON stands for JavaScript Object Notation and is a widely used data file format,
independent of programming language. A JSON file usually ends with the extension ”.json”. We
will briefly explore how to handle JSON files in Python.
Text in JSON is stored as quoted strings containing values in key-value pairs inside {} braces.
It’s actually very similar to the dictionary we saw in Python. For example:

{
"id": "Frodo",
"tipo" : "Hobbit",
"estado":{
"posicion": [2.345,-5.985],
"velocidad lineal": 0.15,
"velocidad angular": 0,
80 CHAPTER 2. INTRO TO PYTHON

"baterias": 7.32
}
}

Although we could work directly with the text file and all the above functions, there are Python
libraries that provide methods to store, read and manipulate data in JSON format in a simple way.
One of them is the json library. Importing this library we have the dump method to save a Python
dictionary (uni in the example) in a json file. This process is called serialize. The dump must be
provided with the python object of type file (that’s why we open a file with open in write mode
’w’).
To read a JSON data file and save the content in a Python dictionary, we can use the load
method of the json library. Like dump we will have to provide it with the file object (now we will
open it in read mode ’r’) and it will return a dictionary object.

import json as js

robot= {
"id": "Frodo",
"tipo" : "Hobbit",
"estado":{
"posicion": [2.345,-5.985],
"velocidad lineal": 0.15,
"velocidad angular": 0,
"baterias": 7.32
}
}

[Link](robot,open('[Link]','w'))

mi_robot=[Link](open('[Link]','r'))

print(mi_robot)

{'id': 'Frodo', 'tipo': 'Hobbit', 'estado': {'posicion': [2.345, -5.985],

'velocidad lineal': 0.15, 'velocidad angular': 0, 'baterias': 7.32}}

Python is a widely used programming language for working with data. Its powerful libraries,
such as pandas, are suitable for handling large amounts of data.

)
Chapter 3

Introduction to Numpy

When the going gets tough, the

tough get going

Popular Witticism (US)

3.1 Numpy: a Python library for scientific computing

In chapter 2, we have seen how to import Python modules to a script or the Ipython console.
In this way, we can reuse the code available in the modules. Numpy is a Python library with
functions and variables specifically intended to perform scientific computing. Numpy is structured
in modules and submodules so that we can import the whole library, its submodules, or just
any of its functions into our scripts. The background of Numpy is built of algebraic objects and
operations. In this section, we will review some fundamental notions of linear algebra and how to
deal with them using Numpy. We will not provide formal definitions or demonstrations because
both will be studied in detail in algebra during the next term.

Matrices From a functional perspective, we will define a matrix as a table of numbers ordered
by rows and columns,
 √ 
1 2 3.5 0
A = −2 π −4.6 4
7 −19 2.8 0.6
Each horizontal line of numbers forms a matrix row, and each vertical line a matrix column. A
matrix with m rows and n columns is denoted as a matrix of order m × n. m and n are the matrix
dimensions, and they are always defined in the same order: first, the number of rows and then the
number of columns. So matrix A in the example above is a 3 × 4 matrix, and as it is built of real
numbers, we say that A ∈ R3×4 . The matrix order defines the size of the matrix.
Two matrices are equal if they have the same order and the entries located in both matrices
in the same place are equal. A matrix is square whenever it has the same number of rows and
columns. That is, it is an n × n matrix.
From now on, we will use capital letters A, B, · · · to name matrices. The expression Am×n
means that matrix A has dimensions m × n. We will refer to the entries of a matrix using the
same letter used to name the matrix but in lowercase and using subindexes to indicate the row

81
82 CHAPTER 3. INTRO TO NUMPY

and column the entry belongs to. The first subindex always represents the row, and the second is
the column. For instance, aij Represents the matrix A entry that belongs to row i and to column
j.
 √ 
1 2 3.5 0
A = −2 π −4.6 4  → a23 = −4.6
7 −19 2.8 0.6

We call a matrix formed by a single row a row vector. We call a matrix formed by a single
column a column vector. Whenever we speak of a vector without further specification, we consider
it a column vector.1 . To name vectors, we will use lowercase letters. To name the entries of a
vector, we will add a subindex to the letter representing the vector to show the row the entry
belongs to.


a1
 a2 
 .. 
 
.
a= 
 ai 
.
 
 .. 
an

We can establish a relationship between the plane points and two-dimensional vectors. To do
so, we use a Cartesian representation, in which the entries of a vector are the (x, y) coordinates of
the point of the plane they represent. Each vector is drawn using an arrow that starts at the origin
of the coordinates and ends at the point (x, y) represented by the vector. Figure 3.1 represents
vectors,
     
1 2 0
a= ,b = ,c =
2 −3 −2

2
a
b
1.5 c

0.5

−0.5

−1

−1.5

−2

−2.5

−3
−1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 3 3.5

Figure 3.1: Graphic of vectors on the plane

1 This identification of a vector with a column vector is by no means general. We introduce it because it simplifies

the exposition.
3.1. NUMPY: A MATH. PACK. 83

Likewise, we can associate vectors of dimension three with points in the 3D space. In this case,
the vector entries represent the coordinates (x, y, z) of the points in the space. Figure 3.2 shows a
graphic representation of vectors,

     
1 2 0
a = 2 , b = −3 , c = −2
1 −1 1

0
z

−1

−2

−3 2
−3
−2 0
−1
0
1 −2
2
3
y
x

Figure 3.2: Graphic of vector in the 3D space

Obviously, it is impossible to get a graphic representation for vectors of larger dimensions.

Nevertheless, many geometrical properties owned by 2D and 3D vectors can be applied to vectors
of whatever dimension.

3.1.1 Vectors and matrices in Numpy.

Matrices. One interesting feature of numpy is that it allows us to create matrices easily. There
are several methods to create them, but using the Numpy function array with a Python list with
the rows of the matrix we want to build is probably the easiest method. Each row should be, in
turn, a list of numbers, i.e., matrix entries. Of course, to build a matrix, all rows should have the
same number of entries. The following example shows how to build a matrix of two rows and three
columns.
84 CHAPTER 3. INTRO TO NUMPY

In [3]: import numpy as np

In [4]: A = [Link]([[1,2,3],[4,5,6]])

In [5]: print(A)
[[1 2 3]
[4 5 6]]

In [6]: L = [[1,2,3],[4,5,6]]

In [7]: B = [Link](L)

In [9]: print(L)
[[1, 2, 3], [4, 5, 6]]

In [10]: print(B)
[[1 2 3]
[4 5 6]]
First, we import Numpy; we do it using np as an alias. The reason is that np is shorter than
numpy, and it eases the calls to specific Numpy functions. We have built two identical matrices
a and B. In the first case, we have straightforwardly created the list needed to make the matrix
inside the call to the function array. In the second case, we first created a list L, and then we used
it as an input variable to the function array. Focusing on lines [9] and [10], we see how Python
can tell a list and a matrix apart. For a list, Python represents all entries in the same line. For
a matrix, each row is written in a different line. We can build matrices from lists and variables
already defined as long as we follow the criteria that each row be created from a list and all rows
have the same number of entries,
In [23]: a =1; b= 2; c =3

In [24]: d =[4,5,6]

In [25]: C = [Link]([[a,b,c],d])

In [26]: print(C)
[[1 2 3]
[4 5 6]]

In [27]: C = [Link]([[a,b,c],d,[7,8,9]])

In [28]: print(C)
[[1 2 3]
[4 5 6]
[7 8 9]]

Indexing As in algebra, in Numpy is also possible to refer any entry of a matrix using indexes
to indicate its position (row and column) in the matrix.
 
a11 a12 a13
A = a21 a22 a23 
a31 a32 a33
3.1. NUMPY: A MATH. PACK. 85

However, the Numpy criteria to refer to a specific entry inside a matrix has been borrowed
from Python Lists: we write the name of the matrix followed by the row and column indexes of
the entry we are interested in, enclosed in a square bracket and separated by a comma. The point
is that we count the rows and columns starting at 0. Thus, the first row of a matrix with
dimensions m × n is the row 0, and the last is the row m − 1. Likewise, its first column is column
0, and the last one is column n − 1,
In [31]: print(C)
[[1 2 3]
[4 5 6]
[7 8 9]]

In [32]: C[1,2]
Out[32]: 6

In [33]: C[0,0]
Out[33]: 1
Inside a matrix, Numpy can select single entries and whole submatrices. To do so, it uses the
colon : symbol as a special symbol. Using a fixed step, we use this symbol to cover a set of values
from a start (initial) value to a stop (final) one. The syntax is simple: start:stop:step. Beware!
Numpy will stop the count in the step stop-step. Besides, if we leave apart the step size, Numpy
will take a step equal to one. In this case, the expression is just start:stop

In [87]: D = [Link]([[1.,2.,3.],[4.,5.,6.],[-2,-3,0],[3,2,1]])

In [88]: print(D)
[[ 1. 2. 3.]
[ 4. 5. 6.]
[-2. -3. 0.]
[ 3. 2. 1.]]

In [90]: D[0:1,0:3]
Out[90]: array([[1., 2., 3.]])

In [91]: D[0:2,0:2]
Out[91]:
array([[1., 2.],
[4., 5.]])

In [92]: D[2:3,1:2]
Out[92]: array([[-3.]])

In [93]: D[3:4,0:3]
Out[93]: array([[3., 2., 1.]])

In [94]: D[0:4,0:3]
Out[94]:
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[-2., -3., 0.],
86 CHAPTER 3. INTRO TO NUMPY

[ 3., 2., 1.]])

In [95]: D[0:4,2:3]
Out[95]:
array([[3.],
[6.],
[0.],
[1.]])

Line [90] extracts a row vector from the first row of matrix D. Line [91] extracts a 2 × 2 matrix
using the four entries located on the left-up corner of the original, D, matrix. Line [92] extracts a
single matrix entry, but notice that it is a 1 × 1 matrix. Thus, this result is different than the result
achieved when we directly use the indexes of the entry: D[2,1]. Line [93] returns a row vector
with the matrix’s last row entries. Lastly, line [95] returns a column vector with the matrix’s last
column entries.

We have said that line [95] returns a column vector. Well, yes and no; let’s examine it in more
detail.

Vectors. When we introduced vectors in the previous section, we distinguished between row and
column vectors, defining them as matrices with a single row or column. Nevertheless, Numpy
follows another criterion that allows us to generalise the idea of matrix, linking it with the concept
of tensor. we do not get into details2 and simply say that a tensor is an algebraic object defined
by two parámeters, its order, and its dimension. So, a scalar a ∈ R is a tensor of order zero.
A vector b ∈ Rn is a tensor of order one and dimension n. A matrix A ∈ Rm×n is a tensor
of order two and dimensions m, n. A third order tensor, T ∈ Rn×m×l , etc. We can relate the
order with the minimum number of indexes we need to univocally define the tensor entry: for
a scalar, we don’t need an index at all; we have only a single entry, the scalar itself. For this
reason, we associate scalars with a zero-order tensor. For a vector, it is enough to use a single
index to define its entries, b = (bi ), i = 1, · · · , n, b ∈ Rn . For a matrix, we need two indexes,
A = (aij ), i = 1, . . . , n, j = 1, . . . , m, A ∈ Rn×m . For third-order tensors, we need three indexes,
T = (Tijk ), i = 1, · · · , n, j = 1, · · · m, k = 1, · · · , l, T ∈ Rn×m×l and so on. In Numpy, we can
define structures of whatever order and dimensions we want, but we will only use vectors (order
1) and matrices (order 2). Has, then, any sense to distinguish between row and column vector?
Only if we always consider vectors as matrices (order 2) and dimensions 1 × n (row vector) or
n × 1 (column vector). For Numpy, however, vectors an matrices are structures of different order.
Let’s see some examples of their differences. To appreciate it better, we can use the Numpy arrays
attribute shape, which gives us a tuple containing the dimensions of the array. The number of
entries of the tuple tells us the order of the array,

2A formal definition of tensors is far beyond the scope of these notes.

3.1. NUMPY: A MATH. PACK. 87

In [23]: D
Out[23]:
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[-2., -3., 0.],
[ 3., 2., 1.]])

In [24]: [Link]
Out[24]: (4, 3)

In [25]: D[1,1]
Out[25]: 5.0

In [26]: D[1,1].shape
Out[26]: ()

In [27]: D[1,1:2]
Out[27]: array([5.])

In [28]: D[1,1:2].shape
Out[28]: (1,)

In [29]: D[1:2,1:2]
Out[29]: array([[5.]])

In [30]: D[1:2,1:2].shape
Out[30]: (1, 1)

We begin with the same matrix D we use in previous examples. To get its dimensions, we use
[Link]. The result is a tuple of two entries because it is a matrix, and thus, its order is two. The
first entry gives us its column dimension, that is, its number of rows. The second entry is its row
dimension, that is, its number of columns.
In line [25] we extract the entry located at position [1, 1]. In line [26] when we try to get its
dimension, Numpy returns an empty tuple because it is a scalar and its order is zero. In line [27],
we ask Numpy that retrieve the entries of row 1 of the matrix D that fill the columns 1 to 1. Thus,
we are making reference to the same entry of the matrix as in the previous case. However, the
result is not exactly the same. Numpy retrieves an array with the selected entry. When we ask for
its dimensions in line [28] we get a tuple with a single entry (1,). Then, the order of the array is
one. It is a vector with a single entry.
Finally, in line [29], we ask Numpy to retrieve the entries of matrix D with fill the rows from 1
to 1 and the columns from 1 to 1. Thus, the result is now a matrix, We can see in line out [29]
that the number 5 is enclosed in two pair of square brackets. When in line [30], we ask for its
shape, we get a tuple with two entries –now the order of the array is two, because it is a matrix–
and their dimension are one row and one column as far as the matrix has a single entry.
We are going to complete our study on Numpy arrays, extracting larger parts of the same
matrix,
88 CHAPTER 3. INTRO TO NUMPY

In [9]: D
Out[9]:
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[-2., -3., 0.],
[ 3., 2., 1.]])

In [10]: D[1,:]
Out[10]: array([4., 5., 6.])

In [11]: D[:,1]
Out[11]: array([ 2., 5., -3., 2.])

In [12]: D[1:2,:]
Out[12]: array([[4., 5., 6.]])
In [13]: D[:,1:2]
Out[13]:
array([[ 2.],
[ 5.],
[-3.],
[ 2.]])
In the first case, line [10] we get the whole second row of matrix D. The result is a vector
and has order 1 and dimension 3. In line [11], we extract the first column of the matrix, and the
result is again a vector. This time the order is 1, and the dimension is 4. In line [12] we extract
all columns belonging to rows second to second. The result is a matrix because the order of the
extracted array is two. The dimensions are 1 for the rows and 3 for the columns. It is the most
similar to a ’row’ vector we can get using Numpy. Lastly, in line [13], we extract all the rows
belonging to columns second to second. the result is again a matrix because the array obtained
has order two, but the dimensions are now 4 for the rows and 1 for the columns, we could identify
it as a column vector of those described above.
Indeed, we can also extract a block matrix (submatrices), using indexes to define the rows and
columns we want to obtain from the original matrix,
In [20]: D
Out[20]:
array([[ 1., 2., 3.],
[ 4., 5., 6.],
[-2., -3., 0.],
[ 3., 2., 1.]])

In [21]: D[1:4,1:3]
Out[21]:
array([[ 5., 6.],
[-3., 0.],
[ 2., 1.]])

In [22]: D[1:3,0:2]
Out[22]:
array([[ 4., 5.],
[-2., -3.]])
3.2. MATRIX OPERATIONS 89

3.2 Matrix Operations

In this section we will define the most common mathematical operation for matrices. We are going
to beging for those operation that are carried out, entry by entry, between those entries which
occupy the same place in the operating matrices.

Addition. Addition of two matrix, the result is a new matrix obtained adding the entries which
ocuppy the same position in both matrices. It is denied only for matrix of the same dimensions,

C =A+B
cij = aij + bij

     
1 2 3 1 3 5 0 −1 −2
4 5 6 = 3 5 7 + 1 0 −1
7 8 9 5 7 9 2 1 0

Matrix addition fulfil the following properties,

1. Asociative: (A + B) + C = A + (B + C)

2. Conmutative: A + B = B + A

3. Identity element: On×m + An×m = Am×m The identity element On×m of the addition of
n × m matrices is the null matriz of this dimensions, —an only zeros matrix— .

4. Inverse: we get the addition inverse of a matrix changing the sings of all its entries Ainv = −A

In numpy, the symbol + also represents the matrix addition. Thus, you can use it to add two
matrices in the same way you use it to add two numbers,

In [0]: import numpy as np

In [1]: A = [Link]([[1,3,5],[2,4,6]])
In [2]: A
Out[2]:
array([[1, 3, 5],
[2, 4, 6]])

In [3]: B = [Link]([[3,-2,0],[1,-4,3]])

In [4]: A+B
Out[4]:
array([[4, 1, 5],
[3, 0, 9]])

We can use the Numpy command

[Link]((m,n)) to create a matrix, of whatever dimension, with all ist entries equal to zero.
m stands for the number of rows and n for the number of columns of the zero matrix we want to
create. If we use the command [Link](n) with a single value instead a tuple, then we will
obtain a vector built up of n zeros..
90 CHAPTER 3. INTRO TO NUMPY

In [375]: [Link](3)
Out[376]: array([0., 0., 0.])

In [377]: [Link]((3,1))
Out[378]:
array([[0.],
[0.],
[0.]])

In [379]: [Link]((1,3))
Out[380]: array([[0., 0., 0.]])

In [381]: [Link]((3,3))
Out[382]:
array([[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]])

In [383]: Z = [Link]((2,3))

In [384]: A+Z
Out[385]:
array([[1., 3., 5.],
[2., 4., 6.]])

In [386]: Aop = -A

In [387]: A+Aop
Out[388]:
array([[0, 0, 0],
[0, 0, 0]])

Entry-wise multiplication This is not a proper matrix (algebraic) operation. You can get the
entry-wise product of two matrices, A and B, with the same dimensions to obtain a new matrix
C which entries are ci,j = ai,j bi,j . This product, like the matrix addition, is conmutative and
asociative. In numpy the entry-wise multiplication symbol is the asterisk *,
3.2. MATRIX OPERATIONS 91

In [49]: A
Out[49]:
array([[1, 2],
[2, 0],
[3, 5]])

In [50]: B
Out[50]:
array([[-3, 1],
[ 0, 2],
[ 1, -4]])

In [51]: A*B
Out[51]:
array([[ -3, 2],
[ 0, 0],
[ 3, -20]])

Entry-wise division. It is very much alike the element-wise product. If we divide entry-wise
a matrix A by another matrix B, both of the same dimensions, we get a new matrix C which
entries are cij = aij /bij . For matrix entry-wise division, we use the same symbol as in the ordinary
number division.
In the following example, we divide the two matrix of the previous example, it is interesting to
see how Python warnings us of a division by zero, and assigns to the resulting entry the value inf.

In [52]: A/B
/tmp/ipykernel_10098/[Link]: RuntimeWarning: divide by zero encountered
in divide A/B
Out[52]:
array([[-0.33333333, 2. ],
[ inf, 0. ],
[ 3. , -1.25 ]])

Transposition. The transpose matrix AT of a matrix A is the matrix we obtain intechanging

its rows and columns.

A → AT
aij → aji
 
  1 2
1 −3 2
A= → AT = −3 7
2 7 −1
2 −1

In Numpy the traspose matrix is obtained adding a point an the letter T to the matrix we wish
to transpose, A.T. It only has sense for matrices. If we try to transpose a vector wi will optain the
same vector again.
92 CHAPTER 3. INTRO TO NUMPY

In [375]: A
Out[375]:
array([[1, 3, 5],
[2, 4, 6]])

In [376]: A.T
Out[376]:
array([[1, 2],
[3, 4],
[5, 6]])

In [377]: B = [Link]([1,2,3])

In [378]: B.T
Out[378]: array([1, 2, 3])

In [379]: C = [Link]([[1,2,3]])

In [380]: C
Out[380]: array([[1, 2, 3]])
In [381]: C.T
Out[381]:
array([[1],
[2],
[3]])
A square matrix is antisimetric if it is equal to its traspose matrix,

A = AT
aij = aji
 
1 3 −3
A = AT =  3 0 −2
−3 −2 4

A square matrix is antisimetric when it meets that A = AT . Any square matrix can be split in
the sum of two matrix, one simetric and another antisimetric
The simetric part can be defined as,
1
A + AT


AS =
2
An the antisimetric part as,
1
A − AT


AA =
2
For instance,
     
1 2 3 1 3 5 0 −1 −2
A = AS + AA → 4 5 6 = 3 5 7 + 1 0 −1
7 8 9 5 7 9 2 1 0
3.2. MATRIX OPERATIONS 93

The transpose of the addition of matrix meets,

(A + B)T = AT + B T

Product of a matrix and a scalar. The product of a matrix A for a number b is a matrix
with the same dimensions as A. We obtain the entry of this product multiplying the elemetns of
A by the number b,

C = b · A → cij = b · aij
   
1 −2 0 3 −6 0
3· =
2 3 −1 6 9 −3

In Python we also use the symbol * to represent the product of a matrix by a number.

In [391]: A
Out[391]:
array([[ 3, -5, -2, 1],
[ 2, 3, 4, 5]])

In [394]: A*5
Out[394]:
array([[ 15, -25, -10, 5],
[ 10, 15, 20, 25]])

Dot product of two vectors For two vectors of the same dimension, we define the the dot
product as,

n
X
a·b= ai bi
i=1
   
1 1
3 · −2 = 1 · 1 + 3 · (−2) + 4 · 0 = −5
4 0

The dot product result is always a number; we multiply the entry which take the same place
in both vectors and then, we sum the resulting products.

Matrix product The matrix product of a n × m matrix by a mtimesl matrix is a new matrix
which dimensions are n × l, We obtain the entries of the resulting matrix according with the
following expression,
m
X
P = A · B → aij = ait btj
t=1

So, we get the entry of the product matrix, which occupies the row i and the column j, by
multiplying in turn the entries of the row i of matrix A with the entries of the column j of matrix
B and adding up the resulting products.
Two matrix can be multiplied only if the number of columns in the first matrix dimension is
equal to the second matrix row dimension.
94 CHAPTER 3. INTRO TO NUMPY

We may understand better the matrix product mechanism if we consider the first matrix as a
group of row vectors,


A1 = a11 a12 · · · a1n  

a11 a12 · · · a1n
A2 = a21 a22 · · · a2n  a21 a22 · · · a2n 
→ A= . .. ..
 
..  ..
. . ··· .

am1 am2 · · · amn
Am = am1 am2 · · · amn
An the second matrix as a group of column vectors.

     
b11 b12   b11 b12 · · · b1n
 b21   b22  b1m  b21 b22 · · · b2n 
B1 =  . B2 =  .  · · · B3 =  b2m  → B= . .. ..
       
 ..   ..  ..  .. . ··· .
.b nm
bn1 bn2 bm1 bm2 · · · bmn
we may consider the consider each entry pij of the product matrix P = A · B as a escalar
product of the row vector Ai and the column vector Bj , pij = Ai · Bj . Now we can easily find out
some interesting properties of the matrix product.

1. Two matrices can be multiplyed only if the number of columns of the first matrix meet the
number of columns of the second one. Besides, the product matrix has so meny row as the
first matrix and so many columns as the second one.
2. Matrix produc it is not conmutative. In general, A · B ̸= B · A
3. (A · B)T = B T · AT

Numpy uses the symbol @ to represent the scalar product, the product of a matrix and a vector
and the matrix product. In any case, it is necessary that the inner dimensions of factor objects
meet. Next we show some matrix multiplication examples,
In [382]: a = [Link]([1,2,3,4])

In [383]: b = [Link]([-1,2,0,-3])

In [384]: a@b
Out[384]: -9

In [386]: A = [Link]([[3,-5,-2,1],[2,3,4,5]])

In [387]: A
Out[387]:
array([[ 3, -5, -2, 1],
[ 2, 3, 4, 5]])

In [388]: A@b
Out[388]: array([-16, -11])

In [389]: b@A.T
Out[389]: array([-16, -11])
3.2. MATRIX OPERATIONS 95

In [390]: b@A
Traceback (most recent call last):

Cell In[390], line 1

b@A

ValueError: matmul: Input operand 1 has a mismatch in its core dimension 0,

with gufunc signature (n?,k),(k,m?)->(n?,m?) (size 2 is different from 4)

In [398]: B
Out[398]:
array([[ 0, -1],
[-1, 0],
[ 2, 3],
[ 3, 4]])

In [399]: A@B
Out[399]:
array([[ 4, -5],
[20, 30]])

In [400]: B@A
Out[400]:
array([[-2, -3, -4, -5],
[-3, 5, 2, -1],
[12, -1, 8, 17],
[17, -3, 10, 23]])

In line In[384] we calcuted the scalar product of vectors a and b. In this case, the only require-
ment is that both vectors have the same dimension. In line In [388] we calculated the product of
matrix A and the vector b. Now the requirement is the dimension of the matrix (2×4), correspond-
ing to the number of columns, meets the the single dimension of the vector (4). In line [389], we
multiplied vector b with the transpose of matrix A. This operation can be carried out because the
single dimension of vector b is the same as the dimension of the transpose of the matrix (4 × 2),
which corresponds with the number of rows of the matrix. However, we cannot multiply vector b
by matrix A because the dimension of the vector does not meet the first dimension of the matrix
(2 × 4). In line In [399] we multiplied matrix A (2 × 4) by matix B (4 × 2). We can carry out the
operation because the “inner” dimensions of the matrices –number of columns of the first matrix
and number of rows of the second one–, meet. If we invert the order of the matrices (line In [400]),
we can still multiply the matrix because, in this particular case, the new “inner” dimensions also
meet. However, the result is entirely different

The identity matrix The identity matrix of dimension n × n is defined as:

(
ill = 1
In =
ikj = 0, k ̸= j

So, an indentity matrix has any entry off the main diagonal equal to 0 and all entries on the
nain diagonal equal to 1, for example,
96 CHAPTER 3. INTRO TO NUMPY

 
1 0 0
I3 = 0 1 0
0 0 1
The indentity matrix In is the identity element of the square n × n matrices product.

An×n · In = In · An×n
Besides,

An×m · Im = An×m
In · An×m = An×m

In Numpy, we can use the command eye(n) to build the identity matrix of dimension n × n.
In [381]: [Link](4)
Out[381]:
array([[1., 0., 0., 0.],
[0., 1., 0., 0.],
[0., 0., 1., 0.],
[0., 0., 0., 1.]])
An orthogonal matrix is a square matrix that fulfils,

AT · A = I

Vector norm The Euclidean length, module, norm two or just norm of a vector is defined as
follows,

n
! 21
√ √ q X
∥x∥2 = ∥x∥ = x·x= xT x = x21 + x22 + · · · x2n = x2i
i=1
The vector norm represents the usual method to measure its length. It has a direct geometrical
interpretation using Pithagoras’ theorem: the norm is the length of the segment that represents
the vector. Figure 3.3 shows such an interpretation for a bi-dimensional vector.
In [422]: a
Out[422]: array([1, 2, 3, 4])

In [425]: [Link](a)
Out[425]: 5.477225575051661
We can derive an alternative expression of the scalar product of two vectors, using the vector
norm

a · b = ∥a∥∥b∥ cos θ
where θ is the angle between the two vectors.

∞ 

i=0
3.2. MATRIX OPERATIONS 97

3
x2
|x|2=|x1|2+|x2|2
x
x2

0
x
1

−1
−0.5 0 0.5 1 1.5 2 2.5 3 3.5 4 4.5
x1

Figure 3.3: Geometrical interpretation of the norm of a vector.

Being the norm 2 the most common way to define a vector norm is by no means the only
possible definition,
Norm 1: defined as the sum of the absolute values of vector entries.

∥x∥1 = |x1 | + |x2 | · · · |xn |

P-Norm: It is a generalization of the norm 2

q n
! p1
X
∥x∥p = p |xp1 | + |xp2 | + · · · |xpn | = |xpi |
i=1

∞ Norm: The entry with the largest absolute value

∥x∥∞ = max {|xi |}

−∞ Norm: The entry with the smallest absolute value.

∥x∥−∞ = min {|xi |}

In Numpy the vector norm can be obtained using the command norm(v,p). The first variable
should be a vector and the second one represents the norm we want to calculate. If we leave out the
second variable, the function returns norm 2 by default. For norms ∞ y∞ we use the symbolinf
to represent the norm kind.
98 CHAPTER 3. INTRO TO NUMPY

In [422]: a
Out[422]: array([1, 2, 3, 4])

In [423]: [Link](a,1)
Out[423]: 10.0

In [424]: [Link](a,2)
Out[424]: 5.477225575051661

In [425]: [Link](a)
Out[425]: 5.477225575051661

In [426]: [Link](a,4)
Out[426]: 4.337613136533361

In [427]: [Link](a,[Link])
Out[427]: 4.0

In [428]: [Link](a,-[Link])
Out[428]: 1.0
In general, a norm can be define as functions Rn → R, which satisfies,

∥x∥ ≥ 0, ∥x∥ = 0 ⇒ x = 0
∥x + y∥ ≤ ∥x∥ + ∥y∥
∥αx∥ = |α|∥x∥, α ∈ R

Vectors u with ∥u∥ = 1 are called unitary vectors.

Two vectors a and b are orthogonal if their scalar product is zero, aT b = 0 ⇒ a⊥b. Besides, if
the vectors have norm one them, they are called orthonormal vectors. From the point of view of
the geometrical representation, the angle between two orthogonal vectors is a square angle.

Matrix’s trace. The trace of a square matrix is the sum of the entries located in the matrix’s
main diagonal.

n
X
T r(A) = aii
i=1
 
1 4 4
T r 2 −2 2 = 1 − 2 + 6 = 5
0 3 6

The trace of the sum of two square, same dimension matrices A and B is equal to the sum of
A trace plus B trace,
3.2. MATRIX OPERATIONS 99

tr(A + B) = tr(A) + tr(B)

For two matrices A of dimensions m × n and B of dimensions n × m is always true that,

tr(AB) = tr(BA)
In Numpy, the value of a matrix’s trace can be calculated using the function [Link](A). In
this case, the function is a method associated to any matrix and can be also expressed as [Link]()
In [197]: A = [Link]([[1,2,3],[3,-2,3],[0,2,-1]])

In [198]: A
Out[198]:
array([[ 1, 2, 3],
[ 3, -2, 3],
[ 0, 2, -1]])

In [199]: [Link](A)
Out[199]: -2

In [200]: [Link]()
Out[200]: -2

Matrix’s determinant. In Numpy, a matrix’s determinant can be calculated using the function
det, del submódulo linalng. We can calculate the determinant of matrix A of the previous
example,
In [203]: [Link](A)
Out[203]: 20.000000000000007

∞ (

i=0

The determinant of a matrix, A is usually represented as |A| and occasionally as det(A). To

define the determinant of a matrix, we need first to establish some previous definitions. First, if
we consider a scalar as a matrix of a single entry, the determinant of the matrix is this single entry.


A = a11 → |A| = a11
In linear algebra, a first minor Mij of a square matrix A is the determinant of the matrix
obtained by removing simultaneously row i and column j of A. For instance,

 
1 0 −2  
1 0
A = 3 −2 3 , M23 = det

0 6
0 6 5
   
1 −2 1 0
M32 = det , M33 = det ···
3 3 3 −2

We define the cofactor Cij of a matrix Aij entry, aij , using the first minor Mij obtained
removing the row and column aij belongs to.
100 CHAPTER 3. INTRO TO NUMPY

Cij = (−1)i+j Mij

Now, we can define the determinant of a n dimensional square matrix A, using the Laplace’s
formula,

n
X
|A| = aij Cij
j=1

or also as,

n
X
|A| = aij Cij
i=1

In the first case, we say that we develop the determinant along the row i. In the second one,
we say that we have developed the determinant along the column j
Laplace’s formula gets the determinant of a Matrix of dimensions n × n using the first minors
of a matrix row or column, i.e. the determinants of n matrix of order (n − 1) × (n − 1). In turn, we
can calculate each minor using Laplace’s formula and so on, till we arrive at matrices of dimension
2 × 2. We obtain the determinant of a 2 × 2 matrix developing it along its first row,

 
a11 a12
A=
a21 a22
2
X
|A| = a1j C1j = a11 C11 + a12 C12
j=1

= a11 (−1)1+1 |M11 | + a12 (−1)1+2 |M12 |

= −a11 a22 + a12 a21

And if we develop using the secon column,

 
a11 a12
A=
a21 a22
2
X
|A| = ai2 Ci2 = a12 C12 + a22 C22
j=1

= a12 (−1)1+2 |M12 | + a22 (−1)2+2 |M22 |

= −a12 a21 + a22 a12

For a matrix of arbitrary dimension we obtain the determinant using the Laplace’s formula
recursively,
 3.2. MATRIX OPERATIONS 101

n n
(n−1)×(n−1)
X X
|A| = aij Cij = aij (−1)i+j Mij
j=1 j=1
n−1 n−1
(n−1)×(n−1) (n−2)×(n−2)
X X
Mij = mlk Clk = mlk (−1)l+k Mlk
k=1 k=1
..
.
1×1
Mst = (−1)s+t mst

For instance, we can calculate the determinant of the following matrix,

 
1 0 −2
A = 3 −2 3
0 6 5
Developing by the first column entries,

−2 3 0 −2 0 −2
|A| =1 · (−1)2 · + 3 · (−1)3 · + 0 · (−1)4 ·
6 5 6 5 −2 −3
=1 · (−1)2 · [(−2) · 5 − 6 · 3] + 3 · (−1)3 · [0 · 5 − 6 · (−2)] + 0 · (−1)4 · [0 · 3 − (−2) · (−2)] = −64

We can now program a recursive function3 to calculate the determinant of any n×n dimensions.

[Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Thu May 23 14:50:48 2024
5 Este modulo implementa la funcion dumbdet que calcula el determinante de
6 una matriz empleando la formula de Laplace. La función es recursiva,
7 (se llama a si misma sucesivamente para calcular los cofactores necesarios).
8 Desarrolla siempre por los elementos de la primera columna.
9 (Es un prodigio de ineficiencia numerica, pero permite manejar bucles
10 y funciones recursivas, asi que supongo que puede ser útil para los que
11 empiezan a programar).
12 un posible ejercicio para ver lo malo que es el método, consiste ir
13 aumentando la dimension de la matriz y comparar lo que lo tarde en
14 calcular el determinante con lo que tarda la función de [Link]...
15 this module implements the function poordet which calculates a matrix
16 determinant using the Laplace's formulae. It is a recursive function,
17 (it calls itself on and on to calculate the cofactor nedded to get the
18 determinat). It always develod the formula using the elements of the matrix
19 first column.
20 (It is the most ineficient functionever writen, but may be an example of loops
21 and recursive functions for begginers)
22 @author: juan
23 """
24 import numpy as np
3 The method is not particularly efficient but it helps to show the use of recursive functions
 102 CHAPTER 3. INTRO TO NUMPY

25 def dumbdet(A):
26

27 #first we check the matrix is square

28 #primero comprobamos si la matriz es cuadrada
29 sz = [Link][0]
30 d = 0 #variable to save the result/ Variable para guardar el resultado
31

32 if sz != [Link][1]:
33 print('La matriz no es cuadrada')
34 print('The matrix is not square')
35 d = []
36 elif sz == 1:
37 return A[0,0]
38 else:
39

40 for i in range(0,sz):
41 N = [Link](A,i,0)
42 d = (-1)**(i+2)*A[i,0]*dumbdet(N[:,1:sz])+d
43

44

45 return d

Among determinants’ properties we will point out the following ones,

1. The determint of a escalar a by a matrix A is,
|a · A| = an · |A|

2. The determinant of a matrix traspose equals the determinant of the matrix,

|A| = AT

3. The determinant of a matrix product equals the product of their determinants,

|An×n · Bn×n | = |An×n | · |Bn×n |

A singular matrix is a square matrix whose determinant is zero.

The rank of a matrix A is the size of the largest submatrix, inside A, whose determinant is
non-zero. For instance,
 
1 2 3
A = 4 5 6 → |A| = 0
7 8 9
is a singular matrix, and its rank is 2,

1 2
= −3 ̸= 0 ⇒ r(A) = 2
4 5
For a non-singular square matrix, its rank meets its dimension.
In Numpy, we can obtain the rank of a matrix using the comand matrix_rank,
3.2. MATRIX OPERATIONS 103

In [395]: A
Out[395]:
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])

In [396]: [Link].matrix_rank(A)
Out[396]: 2

Inversion. For a square non-singular matrix A of dimension n there is a unique matrix A−1 of
dimension n that satisfies,
A · A−1 = In×n
Where In×n is the identity matrix of dimension n. The matrix A−1 is called the inverse matrix of A,
and we can calculate it in Numpy appliying the comands [Link](A), ó [Link].matrix_power(A,-1)
to the matrix A. In the second case, we are using the function [Link](A,n) which allows
us to calculate the result to raise a natrix to a power An .

In [409]: B = [Link](A)

In [410]: B
Out[410]:
array([[ 1.29166667, -0.58333333, -0.04166667],
[ 0.58333333, -0.16666667, -0.08333333],
[-0.48611111, 0.30555556, 0.06944444]])

In [411]: B @ A
Out[411]:
array([[ 1.00000000e+00, -5.55111512e-17, -1.11022302e-16],
[ 0.00000000e+00, 1.00000000e+00, -1.11022302e-16],
[ 0.00000000e+00, 0.00000000e+00, 1.00000000e+00]])

In [412]: [Link].matrix_power(A,-1)
Out[412]:
array([[ 1.29166667, -0.58333333, -0.04166667],
[ 0.58333333, -0.16666667, -0.08333333],
[-0.48611111, 0.30555556, 0.06944444]])

∞ (

i=0

A matrix inverse can be calculate using the following expression,

1
A−1 = [adj(A)]T
|A|
Where adj(A) is the adjugate matrix of A, which can be obtained replacing each entry aij of
A by its cofactor Cij . Next, We include the Python code of a function, inver, to calculate the
inverse of a matrix. The function inver, in turn, calls the function dumbdet. Therefore, we must
need to import the module determinante in the module where we will create the funtion inver
 104 CHAPTER 3. INTRO TO NUMPY

[Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Sat May 25 17:47:28 2024
4 Este módulo implementa la funcion inver, que calcula la inversa
5 de una matriz cuadrada empleando la expresión clásica
6 A^-1 = [adj(A)]^T*det(A)
7 EL determinante de la matriz se obtiene empleando la función
8 dumbdet incluida en el módulo determinante
9 @author: abierto
10 """
11 import numpy as np
12 from determinante import dumbdet
13 def inver(A):
14 """
15 Esta función devuelve la inversa de una matriz cuadrada A
16 """
17 # primero comprobamos que la matriz suministrada
18 #es cuadrada:
19 f = [Link][0]
20 #creamos una matriz de ceros del mismo tama~no
21 iA = [Link]((f,f))
22 if f != [Link][1]:
23 print('la matriz no es cuadrada, Campeón')
24 return();
25 else:
26 #calculamos el determinante de A, si es cero hemos terminado
27 dA=dumbdet(A)
28 if dA==0:
29 print('la matriz es singular, la pobre')
30 return()
31

32 if abs(dA) <= 10*[Link](float).eps:

33 print('cuidado determinante proximo a cero')
34

35

36 #Calculamos el cofactor de cada

37 #término de A mediante un doble bucle.
38 for i in range(f):
39 for j in range(f):
40 # Construimos el menor correspondiente al elemento (i,j)
41 m=[Link]([Link](A,i,0),j,1)
42

43 #calculamos el cofactor llamando a la función determinante

44 #lo ponemos ya en la posición que corresponderia a la matriz
45 # transpuesta de la adjunta.
46 iA[j,i]=(-1)**(i+j)*dumbdet(m)
47

48

49 #Terminamos la operacion dividiendo por el determinante de A

50 iA=iA/dA
 3.2. MATRIX OPERATIONS 105

51 return(iA)

Some properties relate with matrix inversion are,

1. Inverse of the the producto of two matrices,

(A · B)−1 = B −1 · A−1

2. Determinant of the inverse matrix,

A−1 = |A|−1

3. A matrix is ortogonal if its transpose is equal to its inverse,

A−1 = AT
106 CHAPTER 3. INTRO TO NUMPY

Table 3.1: frecuently used mathematical functions included in Numpy

Type Name variables funcion matemática

Trigonométricas coseno de un ángulo en radianes
cos y=cos(x)
Trigonometric Cosine of angle in radians
Trigonometricas seno de un ángulo en radianes
sin y=sin(x)
Trigonometric Sine of an angle in radians
Trigonométricas tangente de un ángulo en radianes
tan y=tan(x)
Trigonometric Tangent of an angle in radians
Trigonométricas ... y=arc...(x) inversa de una función trigonométrica
Trigonometric arcsin y=arcsin(x) Inverse of a trigonometric function
Exponencial
exp y=exp(x) ex
Exponential
Exponencial logaritmo natural
log y=log(x)
Exponential Natural logarithm
Exponencial logaritmo en base 10
log10 log10(x)
Exponential Basis 10 logarithm
Exponencial √
sqrt y=sqrt(x) x
Exponential
Redondeo redondeo hacia +∞
ceil y=ceil(x)
Rounding rounding towards +∞
Redondeo redondeo hacia −∞
floor y=floor(x)
Rounding rounding towards −∞
Redondeo redondeo al entero más próximo
rint y=rint(x)
Rounding rounding towards the nearest integer
Redondeo redondeo hacia 0
fix y=fix(x)
Rounding rounding towards 0
Redondeo resto de la división entera de y entre x
mod r=mod(x,y)
Rounding remainder after integer division
Módulos módulo de un vector x
norm y=norm(x)
Norms Norm of a vector
Módulos valor absoluto de x
abs y=abs(x)
Norms Absolute value of x
Módulos función signo; 1 si x > 0, -1 si x < 0, 0 si x=0
sign y=sign(x)
Norms sign function; 1 if x > 0, -1 if x < 0, 0 if x=0

3.2.1 Functions included in Numpy

Numpy includes a large number of mathematical functions. Many are intended to be applied to
arrays and simple variables. In the first case, the function is applied entry-wise, and the result is
an array with the same dimension as the original one.
Table 3.1 shows some examples of common mathematical functions. Not every function avail-
able in Numpy has been included. To achieve a complete view of available functions, it is advisable
to read Numpy documentation. These functions can be used straightforwardly. For instance, the
following code calculates the exponential values of a matrix entry,
3.3. VECTORIAL OPERATORS 107

In [1]: import numpy as np

In [2]: A = [Link]([[1,2],[2,0],[3,5]])

In [3]: A
Out[3]:
array([[1, 2],
[2, 0],
[3, 5]])

In [4]: [Link](A)
Out[4]:
array([[ 2.71828183, 7.3890561 ],
[ 7.3890561 , 1. ],
[ 20.08553692, 148.4131591 ]])

∞ 

i=0

3.3 Vectorial operators

In this section we will study the effect of matrix operation described in the previous section when
they are applied to vectors. We start considering the effect if the product of a vector by a scalar
α · a. The main effect is the change of the vector module,

   
a1 αa1 q q
α · a2  = αa2  → ||α · a|| = α2 a21 + α2 a22 + α2 a23 = |α| a21 + a22 + a23 = |α| · ||a||
a3 αa3

If α is a positive number and greater than one, we observe graphically that the result of the
product will be a vector larger than a and with the same direction and sense. Conversely, if α is
positive but less than one, the resulting vector will be shorter than a. Lastly if α is negative, a
change of sense with respect to a is added to the previously obtained results. Figure 3.4 shows
graphically and example of the product of a vector by a scalar.

Linear combination Combining vector addition with product by a scalar, we can use a set of
vectors to generate new vectors. This process is called linear combination.

c = α · a + β · b + · · · + θz
So, the vector c is the result of linear combination of vectors a, b, · · · , z. A set o vectors are
linearly independent if it is not possible to represent any of then as a combination of the others.

α · a + β · b + · · · + θz = 0 ⇒ α = β = · · · = θ = 0
It is possible to represent any vector of dimensions n as a lineal combination of n vectors linearly
independent.
Let’s take n = 2, any pair of no collinear vectors can generate all vectors of dimension 2, for
example,
108 CHAPTER 3. INTRO TO NUMPY

2
a2

a
−1
2*a
0.5*a
−2 −1*a

−3

−2 −1 0 1 2 3 4
a1

Figure 3.4: effect of the product of a vector by a scalar

     
x1 1 −1
=α +β
x2 2 1
Figure 3.5 shows these two vector and some other vectors obtained by combining them linearly.
If we take now n = 3, whatever three vectors not contained in the same plane can generate any
other vector of dimension three. For example,
       
x1 1 2 −1
x2  = α −2 + β  0  + γ  1 
x3 1 −1 1
Figure 3.6 shows these three vector and another one obtained by combining them linearly, taking
α = 1, β = −0.5 y γ = 1. It is easy to deduce using the figure thar any other three-dimensional
vector can be built using vectors a, b y c.

Vector spaces and vector space bases. The set of all n-dimensional vectors with the vector
addition and the product by scalars define a vector space of dimension n.
We have already seen that it is possible to obtain any vector of this n-dimensional space using
n linearly independent vectors belonging to the space. A set of n linearly independent vector in a
vector space of dimension n, is called a basis of the vector space. We can, in principle, find infinity
different basis for an n-dimensional vector space. Some of these bases have properties that make
them specially interesting,

Ortogonal bases. An ortogonal basis is built by vectors which are all ortogonal among
then. That is, the scalar product is bi · bj = 0, i ̸= j. Where bi is the i vector of the basis,
3.3. VECTORIAL OPERATORS 109

a+0.5b
2

0.5a+b a

1
b a−0.5b
x2

−1

−a−0.5b

−2
−a−0.5b

−3
−3 −2 −1 0 1 2 3
x1

Figure 3.5: A graphic representations of Vectors a = (1, 2), b = (−1, 1) and some vectors obtained
from a lineal combination of a and b.

a+0.5b+c
c
a

1
−1
0.5
−0.5
0
x3

0
b
−0.5
0.5
−1
−2 1
−1.5
−1
−0.5 1.5
0 x1
0.5
2
1
x2

Figure 3.6: Vectors a = (1, −2, 1), b = (2, 0 − 1)), c = (−1, 1, 1) and vector a − b + c; a graphic
representation
110 CHAPTER 3. INTRO TO NUMPY

B = b1 , b2 , · · · , bn .

Ortonormal bases. An ortogonal basis whose vectors, in addition, have norm 1 is called an
ortonormal basis. That is, bi · bj = 0, i ̸= j and bi · bj = 1, i = j. A particularly useful ortonormal
basis is the canonical basis, composed by the following vectors,
        

 1 0 0 0 


 0 1 0 0 
 
 ..  n  .. 
       
1 0 2 0 n−1
C = c =  ,c =  ,···c = . , c = .
  ..   ..     


 . . 1 0 

 
0 0 0 1
 

We can consider the components of any vector as the coefficients of the linear combination of
the canonical base vectors which represents the vector
       

a1
 1 0 0 0
 a2   0  1  0 
0
 
. .
     
0 0
 
a=  · · ·  = a1 ·   + a2 ·   + · · · + an−1 ·  ..  + an ·  .. 
        
. .
an−1   ..   .. 
       
1 0
an 0 0 0 1
We can also generalise and extend this result to any other basis, just grouping in a vector the
coeficients of linear combination of basis vectors which generates it. For example, we can build for
vector of dimension 3 the following basis,
      
 1 −1 1 
B = 2 ,  0  , −1
0 2 1
 

We can then represent a vector in the basis B as,

       
1 −1 1 α
B
α·  2 +β·  0  +γ·  −1 → a = β

0 2 1 γ
Where the superscript B , means that vector a components are defined in basis B.
So, for instance, vector,
 
1.125
aB = 0.375
0.75
Would have the following component in the canonical basis,
         
1.125 1 −1 1 1.5
aB = 0.375 → a = 1.125 · 2 + 0.375 ·  0  + 0.75 · −1 = 1.5
0.75 0 2 1 1.5
Figure 3.7, shows the relationship among the canonical basis vectors C, the basis B vectors, and
vector a, whose component have been represented in both bases.
We can use the matrix product to obtain any vector components in the canonical basis C from
the vector components in any other basis B. If we put together the basis B vector in a single
matrix B,
3.3. VECTORIAL OPERATORS 111

base B
base B
base B
base canónica
base canónica
b²=[−1;0;2]
base canónica
componente de a
2
componente de a
1.8 componente de a
componente de a
1.6 a3B=1.5 componente de a
componente de a
1.4
Vector a
1.2 b³=[1;−1;1] c³=[0;0;1]
1
z

a2B=0.375
a3B=0.75
0.8
a
0.6

0.4

0.2 c²=[0;1;0]
c¹=[1;0;0]
a2C=1.5
0 −1
−1
a1C=1.5 b¹=[1;2;0] −0.5
−0.5
0 0
0.5
a1B=1.125
1 0.5
1.5
1
2
2.5 1.5
x
y

Figure 3.7: Graphic representation of vector a, in the canonical basis C and in B basis.

 

 b11
  
b12

b1n

 b11 b12 ··· b1n
 b21 b22 ··· b2n
 

  b21   b22   b2n    
..
  
B = b1 =  b31  , b2 =  b32  , · · · bn =  ...  → B = 
        
 b31 b32 ··· .
      
  ..   ..     . ..

. . b(n−1)n   ..
  
. ··· b(n−1)n 

       
 
bn1 bn2 bnn
 
bn1 bn2 ··· bnn

Supose now that we vector a has the following components ins basis B,

aB

1
aB 
 2
aB =  . 
 .. 
aB
n

we can obtain the componets on the canonical base just multiplying matrix B for vector aB .
So in the example we just have seen,
 112 CHAPTER 3. INTRO TO NUMPY

     
1 −1 1 1.125 1.5
a = B · aB → a = 2 0 −1 · 0.375 = 1.5
0 2 1 0.75 1.5
Eventually, we can combine the matrix product and the inverse matrix to obtain the vector
components in any basis whatsoever, knowing the vector components is other arbitrary basis.
Suppose we have two bases B1 and B2 and a vector a. We can get the components of a in the
canonical basis from its components on basis B1 as, a = B1 · aB1 and from basis B2 as a = B2 · aB2 .
Using the inverse matrix we obtain,

a = B1 · aB1 ⇒ aB1 = B1−1 · a

a = B2 · aB2 ⇒ aB2 = B2−1 · a

,
and, after replace, we obtain,

aB1 = B1−1 · B2 · aB2

aB2 = B2−1 · B1 · aB1

The following code allows us to change a vector from one basis to aanother. Besides, if the
vector is in R3 , the program represents the vector and the old a the new bases.

cambio [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Fri Jun 7 15:40:36 2024
5 Vector basis change
6 aB1 : vector a coordinates represented in base b1
7 B1 Basis in which ab1 is represented. Matrix each column represents a vector of
8 the basis
9 B2 Basis in which we want to represent vector aB1
10 aB2: vector a coordinates in base B2
11 If base B2 is omited the program takes B2 as canonical
12 @author: juan
13 """
14 import numpy as np
15 import [Link] as plt
16

17 eps = [Link](np.float64).eps
18

19 def pintavec(v,ax,col='b'):
20 '''just to draw a 3D vector'''
21

22

23 [Link](0, 0, 0, v[0], v[1], v[2],color=col,length=0.1, normalize=True)

24 [Link]('equal')
25 def basisch(aB1,B1,B2=[Link](0)):
26 '''Cambio de base de B1 a B2
 3.3. VECTORIAL OPERATORS 113

27 Basis chage form B1 to B2

28 '''
29 if [Link] == ():
30 B2 = [Link]([Link][0])
31 if [Link](B1) < eps or [Link](B2) < eps:
32 print('los vectores de al menos una de las bases no sol linealmente independientes')
33 return([])
34 aB2 = [Link](B2)@B1@aB1
35 if [Link][0] == 3:
36 #draw the vector
37 ax = [Link]().add_subplot(projection='3d')
38 for i in B1.T:
39 pintavec(i,ax)
40 for i in B2.T:
41 pintavec(i,ax,'r')
42 pintavec(B1@aB1,ax,'k')
43 pintavec(B2@aB2,ax,'g')
44 ax.set_xlabel('x')
45 ax.set_ylabel('y')
46 ax.set_zlabel('z')
47 return(aB2)
48

49

Linear operators. In previous sections we sow that the product of a matrix A of dimensions
n×n by a vector b of dimension n cast a new vector c of dimension n as a result. We could consider
every matrix n × n as a linear operator, which transforms one vector into another. We define it
as a linear operator because the components of the resulting vector are linearly related with the
components of the operator (the original vector before the transformation). For example, in n = 3,
     
y1 a11 a12 a13 x1 y1 = a11 x1 + a12 x2 + a13 x3
y2  = a21 a22 a23  · x2  → y2 = a21 x1 + a22 x2 + a23 x3
y3 a31 a32 a33 x3 y3 = a31 x1 + a32 x2 + a33 x3
Some linear operator are particular interesting because they generate simple geometrical trans-
formations. Let’s see some examples for bi-dimensional vectors:
1. Dilation: Spans the module of a vector in a factor α > 1. Contraction: diminishes the
module of a vector in a factor 0 < α < 1. In both cases, the direction and sense of the original
vector is preserved.
       
α 0 α 0 a1 α · a1
R= →R·a= · =
0 α 0 α a2 α · a2
2. Reflection of a vector with respect to the x-axis, preserving its norm,
       
1 0 1 0 a a1
Rx = → Rx · a = · 1 =
0 −1 0 −1 a2 −a2
3. reflection of a vector with respect to the y-axis, preserving its norm,
       
−1 0 −1 0 a1 −a1
Ry = → Ry · a = · =
0 1 0 1 a2 a2
114 CHAPTER 3. INTRO TO NUMPY

4. reflection with respect to the origin: Invert the sense of the vector, preserving its norm and
direction.
       
−1 0 −1 0 a1 −a1
R= →R·a= · =
0 −1 0 −1 a2 −a2

It is equivalent to apply a refletion on x-axis and then a reflection on y-axis or vice versa,
R = Rx · Ry = Ry · Rx .
5. Rotation an angle θ around the origin,

       
cos(θ) −sin(θ) cos(θ) −sin(θ) a1 a1 cos(θ) − a2 sin(θ)
Rθ = → Rθ · a = · =
sin(θ) cos(θ) sin(θ) cos(θ) a2 a1 sin(θ) + a2 cos(θ)

Figure 3.8 shows the resulting

vectors of applying the linear transformations that we have just
described to the vector, a = 12 ,

2.5
D*a, α=1.5

2
Rθ*a,
θ=π/6
1.5 Ry*a
Rθ*a, a=[1;2]
θ=π/3
1

0.5
y

D*a, α=0.5

−0.5
R*a
Rx*a
−1

−1.5

−2
−3 −2 −1 0 1 2 3
x

Figure 3.8: Linear transformation of vector = [1, 1]. D, factor 1.5/0.5 dilation/contraction. Ry
reflection on y-axis. Rθ θ = π/6 and θ = π/3 angles rotations around the origin.

Matrix norm. A matrix norm can be defined from the effect of the matrix when operates on
a vector as a linear operator. In this case, we call these norms induced norms. For a matrix A
of dimensions m × n, y(m) = Am×n x(n) . The induced norm of A is defined using the norms of
the vectors x of its domain and the norms of the vectors y of its range, according to the following
expression,
3.3. VECTORIAL OPERATORS 115

∥y∥ ∥Ax∥
∥A∥ = max = max
x̸=0 ∥x∥ x̸=0 ∥x∥

It can be interpreted as the maximum factor a matrix A can enlarge any vector. Some times
the induced norm is defined using the unitary vector of its domain,

∥Ax∥ x
∥A∥ = max = max A = max ∥Ax∥
x̸=0 ∥x∥ x̸=0 ∥x∥ ∥x∥=1

Besides the induced norm already described, it is possible to define the following norms,
1. Norm 1: we add the entries of each column up and then take the maximun value of these
sums as a norm

m
X
∥Am,n ∥1 = max aij
j
i=1

2. Norm ∞: We add the entries of each row up and then take the maximum value of these
sums as the ∞ norm.

m
X
∥Am,n ∥∞ = max aij
i
j=1

3. Norm 2:it si defined as the largest of the matrix singular values. (See section 3.5.5).

∥Am,n ∥2 = σ1

[Link]’ norm. It is defined as the square root of the sum of the squared values of the
matrix entries,

v
um X
m
uX
∥Am,n ∥F = t a2ij
i=1 j=1

That can be also expressed in a more straightforward way as,

q
∥Am,n ∥F = tr(AT · A)

In Numpy is possible to calculate the different matrix norms, similarly to how we calculate the
norm of a vector, using the command norm(A,p). Where A is now a matrix and p specifies the
kind of norm we can calculate. In the case of a matrix, parameter p can take only the values, 1
(norm 1), 2 (norm 2), inf, (norm ∞), and 'fro' (frobenius’ norm). This last is the norm by
default and, thus, if the norm Numpy calculates if parameter p is omitted. The following example
shows the calculations of norms 1, 2, ∞, and Frobenius for the same matrix.
116 CHAPTER 3. INTRO TO NUMPY

In [11]: A = [Link]([[1,-1,3],[2,0,-2],[3,1,2]])

In [12]: A
Out[12]:
array([[ 1, -1, 3],
[ 2, 0, -2],
[ 3, 1, 2]])

In [13]: [Link](A)
Out[13]: 5.744562646538029

In [14]: [Link](A,'fro')
Out[14]: 5.744562646538029

In [15]: [Link](A,1)
Out[15]: 7.0

In [17]: [Link](A,inf)
Out[17]: 6.0

In [18]: [Link](A,2)
Out[18]: 4.552861506620628

Quadratic forms. We define the following operation between a square matrix A of dimensions
n × n an a vector x as a quadratic form,

α = xT · A · x, α ∈ R

The result is a scalar. For instance,

       
1 2 −1 1
1 2 −1 1
A = 2 0 2  , x = 2 → 1 2 3 · 2 0 2  · 2 = 21
3 2 −2 3 3 2 −2 3
In n = 2 dimension,
   

a a12 x1
α = x1 x2 · 11 · → x3 ≡ α = a11 x21 + (a12 + a21 )x1 x2 + a22 x22
a21 a22 x2
Depending of the sign of a11 y a12 we obtain a paraboloid or a hyperboloid equation. Figure
3.9 shows an example.
Briefly we will revise some quadratic forms properties,
1. A matrix A of dimensions n × n is positive definite is generates a quadratic form always
greater than zero for any non-null vector.

xT · A · x > 0, ∀x ̸= 0
2. A symmetric matrix is positive definite if all its eigenvalues (see section 3.5.3) are positive.
3. A non-symmetric matrix A is positive definite is it symmetric part As = (A + AT )/2 be so.

x · As · > 0, ∀x ̸= 0 ⇒ x · A· > 0, ∀x ̸= 0
3.4. KINDS OF MATRICES FREQUENTLY USED. 117

a11>0. a22>0 a11<0, a22>0

2 1

1.5 0.5

1 0
x3

x3
0.5 −0.5

0 −1
1 1
1 1
0 0
0 0
x2 −1 −1 x1 x2 −1 −1 x1

a11>0, a22<0 a11<0, a22<0

1 0

0.5 −0.5

0 −1
x3

x3

−0.5 −1.5

−1 −2
1 1
1 1
0 0
0 0
x2 −1 −1 x1 x2 −1 −1 x1

Figure 3.9: Quadratic forms obtained using the four diagonal matrices: |a11 | = |a22 | = 1, a12 =
a21 = 0

3.4 Kinds of matrices frequently used.

We define next some kinds of matrices frequently used in algebra, some of then have been already
introduced in previous sections; we put them together here for ease reference.
1 Ortogonal matrix: A matrix An×n is ortogonal whenever its inverse matrix is equal to its
transpose.

AT = A−1

ejemplo,
118 CHAPTER 3. INTRO TO NUMPY

   
1/3 2/3 2/3 1 0 0
A = 2/3 −2/3 1/3  → A · AT = AT · A = 0 1 0
2/3 1/3 −2/3 0 0 1
2. Symmetric Matrix: A matrix n×n is symmetric if it is equal to its transpose.

A = AT → aij = aji
ejemplo,
 
1 −2 3
A = −2 4 0
3 0 −5
3. Diagonal matrix: A matrix A is diagonal is only the entries of its main diagonal are different
from zero.
 
a11 0 · · · 0
 0 a22 · · · 0 
 .. ..  → aij = 0, ∀i ̸= j
 
. .
 . . . 0 
0 0 · · · ann
We can build a diagonal matrix departing from a list or an unidimensional array, using the
numpy command: diag. Besides, we can also use this command to get a matrix diagonal an put
it into an array,
In [105]: A = [1,2,3,4]

In [106]: a = [Link](A)

In [107]: a
Out[107]:
array([[1, 0, 0, 0],
[0, 2, 0, 0],
[0, 0, 3, 0],
[0, 0, 0, 4]])

In [108]: B = [Link]([[1,2,3],[4,5,6],[7,8,9]])

In [109]: B
Out[109]:
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])

In [110]: b = [Link](B)

In [111]: b
Out[111]: array([1, 5, 9])
4. Upper triangular matrix: a square matrix is upper triangular when every entry below its
main diagonal is zero, and if it is strictly upper triangular in all the diagonal entries are also zero.
3.4. KINDS OF MATRICES FREQUENTLY USED. 119

T RS → aij = 0, ∀i ≥ j
ET RS → aij = 0, ∀i > j

 
1 3 7
T RS = 0 2 −1
0 0 4
 
0 3 7
ET RS = 0 0 −1
0 0 0

5. Lower triangular matrix: A square matrix is lower triangular when every entry above its
main diagonal are zero, and it si strictly lower triangular if all the diagonal entries are also zero.

T RI → aij = 0, ∀i ≤ j
ET RI → aij = 0, ∀i < j

examples,

 
1 0 0
T RI = 3 2 0
7 −1 4
 
0 0 0
ET RI = 3 0 0
7 −1 0

We can use the numpy commands triu and tril to extract the upper and lower triangular
matrices from a given matrix,

In [112]: B
Out[112]:
array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])

In [113]: bup = [Link](B)

In [114]: bup
Out[114]:
array([[1, 2, 3],
[0, 5, 6],
[0, 0, 9]])
120 CHAPTER 3. INTRO TO NUMPY

In [115]: bdw = [Link](B)

In [116]: bdw
Out[116]:
array([[1, 0, 0],
[4, 5, 0],
[7, 8, 9]])
6. Positive definite matrix: A matrix An×n is positive definite is taking a non-null vector x
satisfy,

xT · A · x > 0, ∀x ̸= 0,
if,

xT · A · x ≥ 0, ∀x ̸= 0,
then matrix A is positive semi-definite.
7. A matrix is (strictly) diagonally dominant if, for every row of the matrix, the magnitude of
the diagonal entry in a row is (greater) greater than or equal to the sum of the magnitudes of all
the other (off-diagonal) entries in that row.
X
|aii | > |aij |, ∀i
j̸=i

example,

10 > 2 + 3
 
10 2 3 
A =  2 −5 1 →
 5>2+1
4 −2 8

8>4+2


3.5 Matrix factorization.

Factorizing a matrix consists on divide it into the product of two or more matrices. We choose the
resulting factor matrices so that they simplify or make certain matrix operations more robust. For
instance: matrix determinant calculation, obtaining inverse matrix , etc. Next, we will describe
most common matrix factorizations.

3.5.1 LU factorization.
We factorize a matrix into the product of a lower triangular matrix L by an upper triangular one
U ., A = L · U . For instance,
     
3 4 2 1 0 0 3 4 2
−8 −1 
2 0 1 = 2 /3 1 0 · 0 /3 /3
3 −3
3 2 1 1 /4 1 0 0 /4
A straightforward application is determinant calculation. Recall that a triangular matrix de-
terminant is the product of their diagonal entries.
So, from the previous example we obtain,
8 3
|A| = 6 ≡ |L| · |U | = 1 · 1 · 1 · 3 · (− ) · (− ) = 6
3 4
3.5. MATRIX FACTORIZATION. 121

The description of method to calculate a matrix LU factorization is far beyond the scope of this
notes. However, we do use the command lu, include in a Python library called Scipy, to compute
the result of a LU factorization. Scipy is a library very much alike to Numpy and similarly to
it, Scipy contains a large number of mathematical functions. In fact, Numpy and Scipy overlaps
a little and you can find certain functions in both libraries. The lu command belongs to Scipy
submodule called linalg and takes as input a square matrix A of dimension n × n, and returns as
outputs a permutation matrix P of dimension n × n we will describe later on, the lower triangula
matrix L, and the upper triangular matrix U . The three output matrices satisfy A = P LU .
Sometimes obtaining directly the LU factorization of a matrix, leads to numerically instable
calculations. This, in turn, affect the results precision. One way to overcome this problem is
to permute some rows of the matrix we intend to factorize and then, to factorize the permuted
version.
We can define the rows permutation of A matrix of dimensions n × m multiplying the matrix by
permutation matrices of dimensions n × n. We obtain these last permuting the rows of the identity
matrix In×n . When a permutation matrix multiplies another matrix by it left side, it permutes
the order of the matrix rows. By contrast, if the permutation matrix multiplies another matrix by
ts right, it permutes the order of the matrix columns. For instance, for matrix of dimensions 3 × n,

   
1 0 0 0 0 1
In×n = 0 1 0 → P1↔3 = 0 1 0
0 0 1 1 0 0

If we multiply P1↔3 by any other matrix A of dimensions 3 × n, the result is equivalent to

interchange row 1 and 3. For example,

     
0 0 1 1 2 5 3 3 6 2 1
P1↔3 · A = 0 1 0 · 4 2 3 0 = 4 2 3 0 = A1↔3
1 0 0 3 6 2 1 1 2 5 3

Coming back to LU factorization, we are going to calculate it for the following matrix,

 
3 4 2 5
2 0 1 −2
A=
3 2 1 8 


5 2 3 2

Scipy always use row permutation, when calculating an LU factorization, to obtain a result as
accurate as possible,
122 CHAPTER 3. INTRO TO NUMPY

In [213]: A = [Link]([[3,4,2,5],[2,0,1,-2],[3,2,1,8],[5,2,3,2]])

In [214]: A
Out[214]:
array([[ 3, 4, 2, 5],
[ 2, 0, 1, -2],
[ 3, 2, 1, 8],
[ 5, 2, 3, 2]])

In [215]: import scipy as sp

In [216]: [P,L,U] = [Link](A)

In [217]: P
Out[217]:
array([[0., 1., 0., 0.],
[0., 0., 0., 1.],
[0., 0., 1., 0.],
[1., 0., 0., 0.]])

In [218]: L
Out[218]:
array([[ 1. , 0. , 0. , 0. ],
[ 0.6 , 1. , 0. , 0. ],
[ 0.6 , 0.28571429, 1. , 0. ],
[ 0.4 , -0.28571429, 0.16666667, 1. ]])

In [219]: U
Out[219]:
array([[ 5. , 2. , 3. , 2. ],
[ 0. , 2.8 , 0.2 , 3.8 ],
[ 0. , 0. , -0.85714286, 5.71428571],
[ 0. , 0. , 0. , -2.66666667]])

In [220]: P @ L @ U
Out[220]:
array([[ 3., 4., 2., 5.],
[ 2., 0., 1., -2.],
[ 3., 2., 1., 8.],
[ 5., 2., 3., 2.]])

3.5.2 Cholesky factorisation

Given a square simetric positive definite matrix, it always possible to factorize it as A = L · LT .

Where L is a lower triangular matrix.
3.5. MATRIX FACTORIZATION. 123

     
a11 a12 ··· a1n L11 0 ··· 0 L11 L21 ··· Ln1
 a12 a22 ··· a2n   L21 L22 ··· 0   0 L22 ··· Ln2 
 .. .. ..  =  .. .. ..  ·  .. .. .. 
     
.. .. ..
 . . . .   . . . .   . . . . 
a1n a2n ··· ann Ln1 Ln2 ··· Lnn 0 0 ··· Lnn
Numpy has a function, included in the linalg submodule, called cholesky, to calculate the
cholesky factorization of a matrix.

In [224]: B
Out[224]:
array([[47, 28, 26, 45],
[28, 24, 16, 40],
[26, 16, 15, 22],
[45, 40, 22, 97]])

In [225]: L = [Link](B)

In [226]: L
Out[226]:
array([[ 6.8556546 , 0. , 0. , 0. ],
[ 4.08421976, 2.70539257, 0. , 0. ],
[ 3.79248978, 0.18874832, 0.76249285, 0. ],
[ 6.56392462, 4.87599823, -5.00195311, 2.2627417 ]])

In [227]: L @ L.T
Out[227]:
array([[47., 28., 26., 45.],
[28., 24., 16., 40.],
[26., 16., 15., 22.],
[45., 40., 22., 97.]])

If the matrix is not definite posive, then function cholesky returns an error message.

3.5.3 Diagonalisation
Eigenvectors and eigenvalues. We define an eigenvector or characteristic vector x of the n × n
matrix A as,

A · x = λ · x, x ̸= 0, λ ∈ C
Namely, the result of multiply matrx A for its eigenvector x is equal to multiply the number,
λ for the eigen vector. x and λ could be real or complex.
λ is call de eigenvalue or characteristic values of the matrix A, associated to the eigenvector x.
For example,‘
 
−2 0 0
A =  0 2 −1
0 0 3

It has an eigenvalue λ = 3 for the eigenvector x = [0, −3, 3]T ,

124 CHAPTER 3. INTRO TO NUMPY

       
−2 0 0 0 0 0
0 2 −1 · −3 = 3 · −3 = −9
0 0 3 3 3 9
There is not a single eigenvector associated to an eigenvalue, if x is an eigenvector of a matrix A,
associated to an eigenvalue λ, It is easy to check that any other vector that we build as α · x, α ∈ C
is also an eigenvector associated to λ,

A · x = λx ⇒ A · αx = λαx (3.1)
we define the set of all eigenvalues of a matrix A as the spectrum of A and we represented it
as Λ(A). The eigendecomposition of a matrix A is defined as,

A = X · D · X −1
Where D is a diagonal matrix built using the eigenvalues of A.
 
λ1 0 ··· 0
 0 λ2 ··· 0 
 .. ..
 

. . ··· 0 
0 0 · · · λn
The eigendescomposition does not always exist.

∞ (
Ξi (t)
i=0

In general, if two matrices A and B satisfy,

A = X · B · X −1
Their a similar matrices. The transformation that convert A into B is denoted a similarity
transformation. A matrix is diagonalisable when there is a similarity transformation that converts
it in a diagonal matrix formed by its eigenvalues.
Coming back to the previous example we can factorise matrix A as,
       
−2 0 0 1 0 0 −2 0 0 1 0 0
A =  0 2 −1 = 0 −1 −3 ·  0 2 0 · 0 −1 −1 
0 0 3 0 0 3 0 0 3 0 0 1/3
So, in this example the A eigenvectors would be λ1 = −2, λ2 = 2 and λ3 = 3. To study the
structure of matrix X, we rewrite the similarity relashionship as follows,

A = X · D · X −1 → A · X = X · D
Now, we can consider each column of matrix X as a vector,
 
λ1 0 ··· 0
0 λ2 ··· 0
A · X = X · D → A · (x1 |x2 | · · · |xn ) = (x1 |x2 | · · · |xn ) ·  . ..
 
 ..

. ··· 0
0 0 ··· λn
3.5. MATRIX FACTORIZATION. 125

It is easy to check that the product of matrix A by each column of matrix X satisfy,

A · x1 = λ 1 · x1
A · x2 = λ 2 · x2
..
.
A · xn = λ n · xn

But then, each column of matrix x must be an eigenvector of A because they satisfy,

A · xi = λ i · xi
Actually, each eigenvalue expand a subspace Eλ S of Cn . To apply matrix A to a vector in Eλ S
is identical to multiply the eigenvector by λ. Each subspace associated to an eigenvector is called
an eigensubspace.

Characteristic polynomial. The characteristic polynomial of a matrix A with dimensions n×n,

is defined as,

PA (z) = det(zI − A)
Where I is the n × n identity matrix . For instance, for a matrix with dimensions 2 × 2,

    
1 0 a a12 z − a11 −a12
PA (z) =det z · − 11 = =
0 1 a21 a22 −a21 z − a22

=(z − a11 ) · (z − a22 ) − a12 · a21 = z 2 − (a11 + a22 ) · z + a11 · a22 − a12 · a21

The eigenvalues λ of matrix A are the roots of the characteristic polynomial of matrix A.

pA (λ) = 0
The roots of a polynomial may be single of multiple that is the same root can be repeated
several times. So, the autovalues of a matrix can also be repeated.
Besides, the roots of a polynomial may be real or complex. For a matrix with real elements, if
it has complex eigenvalues their al always pairs complex conjugated numbers. That is, if λ = a + bi
is an eigenvalue then λ∗ = a − bi is also an eigenvalue.
Let’s have a look to some examples:
The matrix,
 
3 2
A=
−2 −2
has characteristic polynomial,

z−3 −2
PA (z) = = z2 − z − 2
2 z+2
If we equals the characteristic polynomial to zero and compute the roots of the resulting
quadratic equation, we obtain the eigenvalues of matrix A
126 CHAPTER 3. INTRO TO NUMPY

(
2 λ1 = 2
λ −λ−2=0→
λ2 = −1

An associated eigenvector to λ1 = 2 would be x1 = [2, −1]T ,

       
3 2 2 2 4
· =2· =
−2 −2 −1 −1 −2

An an associated vector to λ2 = −1 would be x2 = [1 − 2]T ,

       
3 2 1 1 −1
· = −1 · =
−2 −2 −2 −2 2
the matrix,
 
4 −1
B=
1 2
has charasteristic polynomial,

z−4 −1
PB (z) = = z 2 − 6z + 9
1 z−2
repeating the same procedure as in the previous example, we obtain the eigenvalues of matrix
B,
(
2 λ1 = 3
λ − 6λ + 9 = 0 →
λ2 = 3

In this case, we obtain a double root for the characteristic polynomial so, we get a single
eigenvector with algebraic multiplicity 2.
Matrix,
 
2 −1
C=
1 2
has characteristic polynomial,

z−2 1
PC (z) = = z 2 − 4z + 5
−1 z−2
So, its eigenvalues are two complex conjugate numbers.
(
λ1 = 2 + i
λ2 − 4λ + 5 = 0 →
λ2 = 2 − i

For an A matrix of dimension ntimesn be diagonalisable it is necessary that any eigenvalue

has a number of linearly independent eigenvector that meet its algebraic multiplicity. Matrix B in
the example showed above has a single eigenvector, x = [1, 1]T associated to its single eigenvalue
of multiplicity 2; thus, it is impossible to find another eigenvector linearly independent and B is
not a diagonalisable matrix.
The matrix,
3.5. MATRIX FACTORIZATION. 127

 
3 0 1
G = 0 3 0
0 0 1

has an eigenvalue λ1 = 3, with multiplicity 2, and an eigenvalue λ2 = 1 with multiplicity 1. It is

possible to find to linearly independent vectors for the eigen vector of multiplicity 2, for example:
x1 = [1, 0, 0]T y x2 = [0, 1, 0]T . A possible eigenvector for the second eigenvalues may be,
x3 = [−2, 0, 1]T . Thus, it is possible to diagonalise matrix G,
       
3 0 1 1 0 −2 3 0 0 1 0 2
G = 0 3 0 = X · D · X −1 = 0 1 0  · 0 3 0 · 0 1 0
0 0 1 0 1 1 0 0 1 0 1 1

Eigenvalues associated properties We include below, without proofs, some properties of a

matrix eigenvalues
1. the addition of the eigenvalues of a matrix equals the matrix trace.

n
X
tr(A) = λi
i=1

2. the product of the eigenvalues of a matrix equals the matrix determinant

n
Y
|A| = λi
i=1

3. Wen the eigenvector of a matrix A are orthogonal among them, then matrix a is orthogonally
diagonalisable

A = Q · D · Q−1 ; Q−1 = QT ⇒ A = Q · A · QT

Any symmetric matrix has real eigenvalues and is orthogonally diagonalisable. IN general, a
matrix is ortogonally diagonalizable if it is normal: A · AT = AT · A
4. The maximum absolute value of a n × n matrix A eigenvalues is called the spectral radius
of matrix A.

n
ρ(A) = max |λi |
i=1

Numpy includes, in the submodule linalg, the function eig to calculate the eigenvalues and
eigenvector of a matrix. Function eig takes a square matrix of arbitrary dimensions as input and
returns a vector with the eigenvalues of the input matrix and a square matrix, each of its column
is an eigenvector. We show next two different ways of calling function eig. In line In [281] we
computed the operation X · D · X −1 to check that the results coincides with matrix A.
128 CHAPTER 3. INTRO TO NUMPY

In [273]: A
Out[273]:
array([[ 3, 4, 2, 5],
[ 2, 0, 1, -2],
[ 3, 2, 1, 8],
[ 5, 2, 3, 2]])

In [274]: [autovalores, autovectores] = [Link](A)

In [275]: autovalores
Out[275]: array([10.71154715, -4.45608147, 0.70096017, -0.95642586])

In [276]: autovectores
Out[276]:
array([[-0.5482369 , -0.40905141, -0.34168564, 0.53836293],
[-0.05940265, 0.51767323, -0.46625182, -0.17480032],
[-0.63104339, -0.60944455, 0.78709433, -0.82325502],
[-0.54561146, 0.43962336, 0.2152735 , 0.04314365]])
In [277]: Results = [Link](A)
In [278]: Results
Out[278]:
EigResult(eigenvalues=array([10.71154715, -4.45608147, 0.70096017,
-0.95642586]), eigenvectors=array([[-0.5482369 , -0.40905141, -0.34168564, 0.53836293],
[-0.05940265, 0.51767323, -0.46625182, -0.17480032],
[-0.63104339, -0.60944455, 0.78709433, -0.82325502],
[-0.54561146, 0.43962336, 0.2152735 , 0.04314365]]))

In [279]: [Link]
Out[279]: array([10.71154715, -4.45608147, 0.70096017, -0.95642586])

In [280]: [Link]
Out[280]:
array([[-0.5482369 , -0.40905141, -0.34168564, 0.53836293],
[-0.05940265, 0.51767323, -0.46625182, -0.17480032],
[-0.63104339, -0.60944455, 0.78709433, -0.82325502],
[-0.54561146, 0.43962336, 0.2152735 , 0.04314365]])

In [281]: autovectores@[Link](autovalores)@[Link](autovectores)
Out[281]:
array([[ 3.00000000e+00, 4.00000000e+00, 2.00000000e+00,
5.00000000e+00],
[ 2.00000000e+00, -2.33146835e-15, 1.00000000e+00,
-2.00000000e+00],
[ 3.00000000e+00, 2.00000000e+00, 1.00000000e+00,
8.00000000e+00],
[ 5.00000000e+00, 2.00000000e+00, 3.00000000e+00,
2.00000000e+00]])
3.5. MATRIX FACTORIZATION. 129

3.5.4 QR Factorisation
The objetive of QR factorisation is to divide a matriz A in the product of two matrices; one of
then an orthogonal matrix Q and the other one an upper triangular matrix R.

A = Q · R, ← Q · QT = I
We are not going to get into details on how to calculate a QR factorisation. We can use a
Numpy function included in submodule linalg, [Q,R]=qr(A),

In [287]: A = [Link]([[1,-2,4],[2,5,3],[1,3,-3]])

In [288]: [Q,R] = [Link](A)

In [289]: Q@Q.T
Out[289]:
array([[1.00000000e+00, 2.56739074e-16, 8.32667268e-17],
[2.56739074e-16, 1.00000000e+00, 5.55111512e-17],
[8.32667268e-17, 5.55111512e-17, 1.00000000e+00]])

In [290]: R
Out[290]:
array([[-2.44948974, -4.4907312 , -2.85773803],
[ 0. , -4.22295315, 3.51254982],
[ 0. , 0. , -3.67359866]])

3.5.5 SVD Factorisation

Any matrix A of dimensions m × n can be factorised as the product of three matrices,

A=U ·S·VT
Where U is an ortogonal matrix of dimensions m × m, V is an ortogonal matrix of dimensions
n × n and S Is a diagonal matrix of dimensions m × n. Besides, the entries of S are positive or
zero and are ordered in non increasing order.
 
σ1 0 ··· 0
0 σ2 ··· 0
s= . .. .. ..  ; σ1 ≥ σ2 ≥ · · · ≥ σi ; i = min(m, n)
 
 .. . . .
0 0 ··· σi
The entries of the diagonal of matrix S, (σ1 , σ2 , · · · σi ), are konwn as the singular values of
matrix A. This is the reason why this factorisation is named Singular Value Descomposition, SVD.
We are not going to describe any algorithm to get the SVD factorisation of a matrix. In
Numpy, the function [U,s,VT]= svd(A), which belongs to the linalg submodule, allow us to get
straighforwardly the SVD matrix factorisation for a matrix A with arbitrary dimensions. It in
important to notice that Numpy returns a vector s with the singular values and the matrix V T ,
directly. To build the matrix S we need to create a matrix of dimensions m × n from vector S.
To do so, we first create a diagonal matrix with the entries of s and then, we pad the matrix with
rows or columns of zeros till complete the larger of the matrix dimension. If n > m we have to
add rows, if n < m columns. The following code shows and example for a non-square matrix
130 CHAPTER 3. INTRO TO NUMPY

In [93]: A = [Link]([[1,3,4],[2,3,2],[2,4,5],[3,2,3]])

In [94]: A
Out[94]:
array([[1, 3, 4],
[2, 3, 2],
[2, 4, 5],
[3, 2, 3]])

In [102]: U,s,VT = [Link](A)

In [105]: S[:[Link][0],:[Link][0]] = [Link](s)

In [106]: S
Out[106]:
array([[10.25447551, 0. , 0. ],
[ 0. , 1.90114896, 0. ],
[ 0. , 0. , 1.10966868],
[ 0. , 0. , 0. ]])

In [107]: U@S@VT
Out[107]:
array([[1., 3., 4.],
[2., 3., 2.],
[2., 4., 5.],
[3., 2., 3.]])
Matrix A has more rows that columns. For this reason, matrix S ends with a row of zeros.
Next, we include without demonstrations some SVD factorisation properties.
1. The rank of a matrix A coincides with the number of its non-zero singular values.
2. The induced 2-norm of a matrix A is equal to its largest singular value σ1 .
3. The Frobenius norm of matrix satisfies,
q
∥A∥F = σ12 + σ22 + · · · + σr2
4. The non-zero singular values of a matrix A are equal to the positive square root of the
non-zero eigen values of the matrices A · AT ó AT · A. (The non-zero eigenvalues of these matrix
are equal by obvious reasons),

σi2 = λi (A · AT ) = λi (AT · A)
5. The absolute value of a square n timesn matrix A determinant its equal to the product of
the matrix singular values.
n
Y
| det(A)| = σi
i=1

6. The condition number of a square n × n matrix A, which is defined as the product of the
A induced 2-norm by the induced 2-norm of the inverse of A, may be expressed as the quotient
between the largest singular value of A and its lower singular value,
1 σ1
k(A) = ∥A∥2 · ∥A−1 ∥2 = σ1 · =
σn σn
3.5. MATRIX FACTORIZATION. 131

The condition number of a matrix, is very useful property that allows us to estimate how stable
will be the operations performed using the matrix, i particular, those operations with involve
directly or indirectly the computing of the matrix inverse.
132 CHAPTER 3. INTRO TO NUMPY
Chapter 4

Introduction to matplotlib

El punto geométrico es invisible.

De modo que lo debemos definir
como un ente abstracto. Si
pensamos en él materialmente, el
punto se asemeja a un cero. Cero
que, sin embargo, oculta diversas
propiedades ≪humanas≫. Para
nuestra percepción este cero —el
punto geométrico—está ligado a
la mayor concisión. Habla, sin
duda, pero con la mayor reserva.

Vasili Kandinski, Punto y lı́nea

sobre plano

Visualising data is essential for understanding and conveying ideas and information in science
and engineering. Python has numerous graphics functions for displaying many types of graphics.
One of the most widely used data visualisation libraries is matplotlib to which we dedicate this
chapter.
We can import matplotlib as follows:

import matplotlib as mpl

[Link] is a collection of functions that make matplotlib work like MATLAB. Each
pyplot function makes some change to a figure: for example, it creates a figure, creates a plot area
in a figure, draws some lines in a plot area, decorates the plot with labels, etc.
In [Link] the figure is preserved across function calls, so that the plotting functions
are executed on the current axes.
To create graphs using Matplotlib we will need a figure. Each figure has a pair (or more) of
axes and an area which is where the points will be drawn in the coordinate system of your choice.
In addition we can put titles, legends etc...
The easiest way to create a figure is to use the figure method which will create a figure to
which we can later add axes, titles and more. Simply by calling figure we create an object of
type figure, although we can also optionally give it parameters such as size, background colour and
more.

133
134 CHAPTER 4. INTRO TO MATPLOTLIB

import [Link] as plt

fig = [Link](figsize=(2, 2), facecolor='lightskyblue',
layout='constrained')
fig =[Link]

4.1 2D Plotting
The easiest way to draw using [Link] is by using the plot method. To this method we
will pass two vectors with the x and y coordinates of the points we want to represent (obviously
we will have to make sure that they have the same length).
For example, if we generate coordinate vectors such as

x = (0, 2, −1, −2)

y = (0, 3, 2, −4)
and pass them to the pyplot method, it will draw the points (x, y) and join them with straight
lines, as can be seen in the figure 4.1.

import numpy as np
import [Link] as plt

x=[Link]([0, 2, -1, -2])

y=[Link]([0, 3, 2, -4])
[Link]
[Link](x,y)

Figure 4.1: Simple pyplot example.

In addition to the coordinates of the points (x, y), we can give the [Link] method a third
argument indicating the format (colour and shape) in which we want the points to be plotted and
optionally the line that joins them.
Some of the colours and formats supported are in the tables 4.1, 4.2 and 4.3. For more infor-
mation, see the help for [Link].
It is possible to combine one symbol of each type in the same text. So for example if we want
to represent the above data joined by a dotted line,
[Link](x,y,':')
4.1. 2D PLOTTING 135

Symbol Color
b blue
r red
g green
m magenta
c cyan
y yellow
k black
w white

Table 4.1: Point colors

Symbol Point format

. point
, pixel
o circle
s square
∧ down triangle
v up triangle
p pentagon
* star

Table 4.2: Some point markers

Symbol Line type

- solid
– dashed
-. dash and dot
: dotted

Table 4.3: Line types

136 CHAPTER 4. INTRO TO MATPLOTLIB

Figure 4.2: Different uses of plot

If we want it to paint only the dots without joining them with lines and in red,

[Link](x,y,'.r')

If we want it to paint the points represented by triangles with the vertex upwards, joined by a
continuous line and in black,

[Link](x,y,'-^k')

Figure 4.2 shows the results of the symbol combinations just described.

Scatter. We can also draw the points x, y, separately without joining them using the method
scatter(x,y).

4.1.1 Working with multiple figures

With [Link] you can draw different graphs in the same figure using the subplot
method. With this method we will create a figure where the different graphs will be arranged as
a table. To this method we will indicate by means of input parameters the number of rows, the
number of columns and the position where the next graph is going to be drawn.
4.1. 2D PLOTTING 137

Figure 4.3: Two graphs in one figure using subplot

In the following example, two graphs are created within the same figure in two rows and one
column. In the first one the cosine graph is drawn and in the second one the sine graph is drawn.
The result can be seen in figure 4.3.
import numpy as np
import [Link] as plt
t1 = [Link](0.0, 5.0, 0.1)
[Link](2,1,1)
[Link](t1,[Link](t1),'r.-')
[Link](2,1,2)
[Link](t1,[Link](t1),'b.-')
Text can be added anywhere on the drawing using the text method, indicating the coordinates
where the text is to be placed and the desired text. You can also put a text as the title of the
figure, using title, or by labelling the axes with xlabel, ylabel.
For example, the following code results in the following figure 4.4.
import numpy as np
import [Link] as plt
[Link]()
t=[Link](0.0,5,0.01)
y=[Link](-t) * [Link](2*[Link]*t)
[Link](t,y)
[Link]("Evolución de un sistema amortiguado")
[Link]('t(s)')
[Link]('x(m)')
[Link](2.5,0.8,'$x=e^{-t}\cos(2\pi \cdot t)$')
138 CHAPTER 4. INTRO TO MATPLOTLIB

Figure 4.4: Figure with text and labels

4.1.2 Nonlinear axis

[Link] has a method to define the scale of the axes, these are the methods xscale
and yscale. These methods can take as an argument the scale type of the axis, choose from the
following:
• ‘linear’ to define a linear scale
• ‘log’ to define a positive-axis logarithmic scale
• ‘symlog’ to define a logarithmic scale with a positive and negative span centred at zero (as
the values of the logarithm around zero tend to infinity there is a zone around zero where
the linear scale is considered)
• ‘logit’ logarithmic scale between 0 and 1.
Both xscale and yscale allow the scale to be defined by a function that we define.
The following code example shows an example of using the xscale and yscale methods to
generate the 4.5 figure.
As can be seen in the figure 4.5 the divisions of the logarithmic axis (y-axis in the second figure
and both in the third figure) are marked as powers of 10. As we are representing using the decimal
logarithm of the variable, the divisions correspond to the exponent of the power of 10 of each
division, log10 (10n ) = n.
Secondly, we have employed a new method: grid(). This method adds a grid to the graph so
that it is easier to see the values taken by the variables at each point on the graph. Calling the
grid() method without parameters toggles the visibility of the grid.

import numpy as np
import [Link] as plt
4.1. 2D PLOTTING 139

x=[Link](0,10,100)
y=[Link](x)
[Link]()

[Link](3,1,1)
[Link](x,y)
[Link]()
[Link]("Escala lineal")
[Link]("Escala lineal")
[Link]("$y=e^x$")

[Link](3,1,2)
[Link](x,y)
[Link]('log')
[Link]("Escala lineal")
[Link]("Escala logarı́tmica")
[Link]()

[Link](3,1,3)
[Link](x,y)
[Link]('log')
[Link]('log')
[Link]("Escala logarı́tmica")
[Link]("Escala logarı́tmica")
[Link]()

Figure 4.5: Graph of the function y = ex with axes on linear and logarithmic scale
140 CHAPTER 4. INTRO TO MATPLOTLIB

4.1.3 Polar coordinates graph

Using the polar method we can plot functions in polar coordinates. The first argument is an angle
in radians and the second the corresponding radius. The figure 4.6 shows the spiral,
√
r =2· θ (4.1)
For the angular interval [0, 8π].

import numpy as np
import [Link] as plt

theta=[Link](0,8*[Link],100)
radio=2*theta
[Link]()
[Link](theta,radio,'b.-')
[Link](visible=True)
[Link]("Espiral en polares")

Figure 4.6: Spiral plot using polar coordinates

4.1.4 Other graphical representations

With [Link] we have more options to make other types of graphical representations.

hist . This method allows you to draw the histogram of a collection of data. The histogram
represents how much data in a collection falls within a given interval. The hist method requires a
vector of data as input parameter. Unless otherwise specified, the data is grouped into 10 intervals.
4.2. 3D PLOTS 141

If you want to change the number of intervals you can specify it with the parameter bin. In the
following example the histogram of the car data per 1000 inhabitants is drawn using the hist
method with the default number of intervals, with 5 intervals and with 10 intervals. The data is
loaded from a file called [Link] and the resulting figure 4.7 is the resulting figure.

import numpy as np
import [Link] as plt

# Abrir el archivo en modo lectura

with open('[Link]', 'r') as file:
lineas=[Link]()
f=len(lineas)
n=[Link]([f])
i=0
for row in lineas:
n[i]=float(row)
i+=1

[Link]()

[Link](1,3,1)
[Link](n)
[Link](visible=True)
[Link]("10 intervalos")
[Link]("Coches por cada 1000 habitantes")
[Link]("Número de paises")
[Link](1,3,2)
[Link](n, bins=5)
[Link](visible=True)
[Link]("5 intervalos")
[Link]("Coches por cada 1000 habitantes")
[Link]("Número de paises")
[Link](1,3,3)
[Link](n,bins=20)
[Link](visible=True)
[Link]("20 intervalos")
[Link]("Coches por cada 1000 habitantes")
[Link]("Número de paises")

4.2 3D plots
In three dimensions it is possible to represent two types of graphics: points and curves, analogous
to those represented in two dimensions, and also surfaces in space.

subplots This method encapsulates everything needed to create a figure and a set of subfigures
contained in it. We can indicate many parameters such as the number of rows and columns
of the grid of subfigures, whether or not the axes are shared between them, etc. We can also
142 CHAPTER 4. INTRO TO MATPLOTLIB

Figure 4.7: Histogram of the number of cars for each 1000 people

indicate, by means of a dictionary, which type of projection we want to use in the axes created.
Concretely, by means of the following key-value pair, we are indicating that we want 3d axes
subplot kw=‘projection’: ‘3d’.

[Link](nrows=1, ncols=1, *, sharex=False,

sharey=False, squeeze=True, width_ratios=None, height_ratios=None,
subplot_kw=None, gridspec_kw=None, **fig_kw)

For example, we can represent the curve,

y = sin(2πx)
z = cos(2πx)

To do this, we select an interval of values for x ∈ (0, 2), and calculate the corresponding values
of y and z. Then, we plot them using the scatter method, Figure 4.8.

import [Link] as plt

import numpy as np

x=[Link](0,2,100)
y=[Link](2*[Link]*x)
z=[Link](2*[Link]*x)

fig,ax=[Link](subplot_kw={"projection": "3d"})
[Link](x,y,z)
4.2. 3D PLOTS 143

Figure 4.8: 3D plot using scatter

4.2.1 Surfaces
To draw a surface we will use the plotsurface method. As in the case of the scatter method, we
will have to previously create a figure and axes in 3 dimensions using subplots and indicating that
it is a 3D projection.
In order to draw a surface with plot surface we must generate a grid (Xm , Ym ) and at each of
the points of this grid calculate the value of the surface.
To define such a grid, two matrices are needed. One of them Xm contains the x coordinates of
the grid nodes and the other Ym the y coordinates. Elements occupying the same position in both
matrices represent - together - a point in the plane.
Matplotlib uses these matrices as adjacency matrices. Each node,

(xm (i, j), ym (i, j))

, will appear in the graph connected by an edge to each of its four neighbouring points,

(xm (i − 1, j), ym (i − 1, j))

,
(xm (i, j − 1), ym (i, j − 1))

,
(xm (i + 1, j), ym (i + 1, j))

,
(xm (i, j + 1), ym (i, j + 1))

Suppose we use the following matrices, Xm and Ym to define a lattice on which to draw a
surface,z.
144 CHAPTER 4. INTRO TO MATPLOTLIB

(0, 0) − (1, 0) − (2, 0) − (3, 0)

    | | | |
0 1 2 3 0 0 0 0
0 (0, 1) − (1, 1) − (2, 1) − (3, 1)
1 2 3 1 1 1 1 posiciones
Xm =  ,Y  −− −− −− −
→ | | | |
0 1 2 3 m 2 2 2 2 nodos
(0, 2) − (1, 2) − (2, 2) − (3, 2)
0 1 2 3 3 3 3 3
| | | |
(0, 3) − (1, 3) − (2, 3) − (3, 3)

Axes3D.plot_surface(X, Y, Z, *, norm=None, vmin=None, vmax=None,

lightsource=None, **kwargs)

The structure of the Xm and Ym matrices in the previous example is typical of the adjacency
matrices of a square lattice; the Xm matrix has repeated rows and the Ym matrix has repeated
columns. In the example the matrices are square and define a lattice of 4 × 4 nodes. In general,
we can define a rectangular lattice of timesn nodes. In this case the matrices used to define the
lattice would have dimension m × n.
To draw with Matplotlib surfaces we can first define the grid from two coordinate vectors using
the numpy method meshgrid. In the example we have just seen, we have used a grid covering the
interval, x ∈ [0, 3] and y ∈ [0, 3]. To define it we create the vectors,
x=[Link](0,4)

x
Out[23]: array([0, 1, 2, 3])

y=[Link](0,4)

y
Out[25]: array([0, 1, 2, 3])
Next, we use the method meshgrid to construct the two adjacency matrices. Numpy will take
care of repeating the necessary rows and columns,

[Xm,Ym]=[Link](x,y)
Xm
Out[20]:
array([[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3],
[0, 1, 2, 3]])

Ym
Out[21]:
array([[0, 0, 0, 0],
[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3]])

Once the adjacency matrices are constructed, we only need a matrix of values for Zm . If we define
for example,
4.2. 3D PLOTS 145

Figure 4.9: Flat grid

Zm=[Link]([Link](Xm))

Zm
Out[35]:
array([[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.],
[0., 0., 0., 0.]])

We could represent the flat grid of the figure 4.9, using the command plot wireframe(Xm,
Ym, Zm).
Now that we have seen how to construct a rectangular grid on which to build a surface, let’s
see how to draw it with an example. Suppose we want to draw the surface,

z = x3 + y 2

In the region of the plane, x ∈ [−1.5, 1.5], y ∈ [−2, 2].

As in the initial example, the first thing to do is to construct adjacency matrices that define a
grid in the region of interest,

x=[Link](-1.5,1.5,25)
y=[Link](-2,2,50)
[Xm,Ym]=[Link](x,y)

It is interesting to note that the region of interest is not square and that the adjacency matrices
are not square either (50 × 25). Also the points are not equally spaced on the two axes.
Next we obtain the z-coordinate matrix by applying the function to the grid points,

Zm=Xm**3+Ym**2
146 CHAPTER 4. INTRO TO MATPLOTLIB

(a) Function z = x3 + y 2 represented with

wireframe (b) Function z = x3 + y 2 represented with surf

Figure 4.10: Comparison between wireframe and surface

We can represent the surface using the wireframe method or the surface method. In the first
case the surface is drawn as a grid, figure 4.10a, and in the second as the colour-filled faces of the
grid, figure 4.10b.
We can also represent a surface with a color or contour map in 2 dimensions using the contourf
method or the pcolor method. In this case, subplot kw=“projection”: “3d” is not used because it
is not necessary to draw on the z-axis, since a contour or a colour is assigned to it.

import numpy as np
import [Link] as plt

x=[Link](1,100,1000)
y=[Link](1,100,1000)
[X,Y]=[Link](x,y)
Zm=X**2+Y**2
fig,(ax1,ax2)=[Link](nrows=1,ncols=2,figsize=(15,10))
[Link](X,Y,Zm)
[Link](X, Y, Zm)

4.3 Animations
Based on its plotting functionality, Matplotlib also provides an interface to generate animations
using the animation module. An animation is a sequence of frames where each frame corresponds
to a plot on a figure.
To create an animation in python using [Link] we will follow the following steps:
4.3. ANIMATIONS 147

1. Initialise the meta data for the animation.

2. Initialise the background of the animation.

3. Define the objects that are going to change in each frame.

Initialize the meta data. Initialise the meta data for the animation by creating a dictionary
of it and passing it to the method that will create the animation [Link] indicate both the
metadata and the number of frames per second with the parameter fps.

FFMpegWriter=[Link]["ffmpeg"]
metadata = dict(title="GIF Test", artist ="Me",
comment="A red circle following a blue sine wave")
writer = FFMpegWriter(fps=15, metadata=metadata)

Initialise the background of the animation , i.e. those elements that are not going to
change. We will do this by defining a figure and everything that will remain constant during the
animation, which is why we call it the background. In this case it is the sine curve. We also
initialise the red empty point, to which we give empty coordinates, so that Python knows that
these coordinates will be updated later. Since the labels don’t change, we also put them here.

fig =[Link]()
n=1000
x=[Link](0, 6*[Link],n)
y=[Link](x)
sine_line, =[Link](x,y,"b")
red_circle,=[Link]([],[],"ro",markersize=10)
[Link]("x")
[Link]("sin(x)")

Define the objects that are going to change in each frame. We will save a file, indicating
which figure we are going to save, the name of the file (movie [Link]) and the resolution of
the figure in dots per inch (dpi). In the for loop the figure is repeatedly updated to create
the movement, in each iteration the position of the red dot is updated (using set data). The
grab frame method captures these changes and displays them at the frames per second specified
above.

with [Link](fig,"movie_test.mp4",100):
for i in range(n):
x0=x[i]
y0=y[i]
red_circle.set_data([x0],[y0])
writer.grab_frame()
148 CHAPTER 4. INTRO TO MATPLOTLIB
Chapter 5

Computer Arithmetics and Errors

“What’s the use?” a few hundred

rivets chattered. “We’ve given -
we’ve given, and the sooner we
confess that we can’t keep the
ship together and go off our little
heads, the easier it will be. Not
rivet forged can stand this
strain”.
“No one rivet was ever meant to.
Share it among you”, the Steam
answered.

Rudyard Kipling, The Ship that

Found Herself

In chapter one, we introduced the binary representation of numbers and the conversion from
decimal to binary and from binary to decimal. In this chapter, we will get deep into how a computer
represents and operates numbers. We also discuss one of its direct consequences: the imprecision
of the results.

5.1 Binary and denary representations

Real numbers can be represented on a line extending from −∞ to ∞. Most do not have exact
numerical representations because they have infinite decimals.
The integer numbers 1 − 1, 2, −2, 3 − 3, · · · can be represented in scientific notation. However,
we rarely deal with numbers containing a large amount of digits. Usually, we take an approximate
value, written then in scientific notation, by multiplying the number by a power of 10. Let’s take
an example from chemistry: the number of atoms or molecules that make up a mole of a chemical
substance is defined as Avogadros’s number NA = 6.02214179 × 1023 . This number should be an
integer, but we represent it in scientific notation given just a few, nine in our example, of its 23
digits.
We truncate a number when we miss off digits past a certain point in the number. Then, we
approximate the truncated number by a rational number, i.e., a number that the quotient of two
integers can represent. 1/2, 3/4, cdots
Some fractional numbers reduce, in turn, to integer numbers 6/3. Others are numbers that

149
150 CHAPTER 5. C. ARITHM. AND ERRORS

need an infinity number of digits to be represented:

11/2 = (5.5)10 = 5 × 100 + 5 × 10−1

If we represent the same number in base 2 we obtain,

11/2 = (101.1)2 = 1 × 22 + 0 × 21 + 1 × 20 + 1 × 2−1

However, A rational number may or may not have a finite representation, depending on the
base used to represent it. For example, 1/10 = (0.1)10 has no finite representation in base 2.

1/10 = (0.0001100110011 · · · )2 = 1 × 2−4 + 1 × 2−5 + 0 × 2−6 + 0 × 2−7 + 1 × 2−8 + · · ·

For this case, the number representation is infinite but repetitive or periodic; following the first
zero after the decimal point, the sequence 0011 repeats endlessly. Rational numbers always have
a finite representation or a periodic infinite one.
The rational number 1/3 = (0.333 · · · )10 = (0.010101 · · · )2 has a periodic infinite representation
both in basis ten and in basis 2. However, it has a finite representation in base 3: 1/3 = (0.1)3 =
0 × 30 + 1 × 3−1 .
Usually, in ordinary life, we represent the number using base-ten. However, as we saw in
chapter 1, computers use a binary representation. The first consequence of this difference is that
we alter number precision when we change them from one representation to another. Thus, this
representation difference is, in fact, the first cause of errors that we must take into account when
we want to use a computer to perform scientific computing. In this chapter, we will find more
causes.

5.2 Computer number representation

positive integers. The most straightforward method for representing numbers would be using
the memory registers directly to store the numbers in binary format. Considering a computer with
16-bit size registers, we can save numbers between 0 and 216 − 1 = 65535. This maximum number
would be represented by filling each register position with a 1, so 16 ones in total. This number is
the largest one we can represent using 16 bits:
position 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
Our current representation system only accepts positive integer numbers. The lowest number
we can represent is zero, and the largest one –which will depend on the computer register size–,
would be 2n − 1. It’s clear that this representation system is insufficient.

Positive and negative integers. A first improvement would be to expand the system so that
it also admits negative numbers. One option would be to reserve a bit of the register to represent
the sign, using the remaining register to represent the (absolute value of the) number. Returning
to our example of a 16-bit register, the largest number representable would now be 215 − 1 and the
lowest −215 + 1. Zero would have a double representation, one with a plus sign and another with
a minus sign.
An alternative representation, frequently used by computers, is the 2’s complement represen-
tation. We will describe the representation for a generic n-bits register:
• A non-negative integer is stored using directly its binary representation. The range of repre-
sentable non-negative numbers is 0 ≤ x ≤ 2(n−1) − 1.
5.2. COMPUTER NUMBER REPRESENTATION 151

Decimal (4 bits) C2 = 2n − x n. represent-ado-ed

15 1111 (16 − 1) = 15 -1
14 1110 (16 − 2) = 14 -2
13 1101 (16 − 3) = 13 -3
12 1100 (16 − 4) = 12 -4
11 1011 (16 − 5) = 11 -5
10 1010 (16 − 6) = 10 -6
9 1001 (16 − 7) = 9 -7
8 1000 (16 − 8) = 8 -8
7 0111 7
6 0110 6
5 0101 5
4 0100 4
3 0011 3
2 0010 2
1 0001 1
0 0000 0

Table 5.1: 2’s complement representation for a 4-bit register

• A negative integer −x where 1 ≤ x ≤ 2(n−1) , is stored as the binary representation of the

positive integer 2n − x.
• The number built following the procedure just described is called the 2’s complement of x.
In general, we define the 2’s complement of an integer N < 2n in a binary representation of n
digits as:
C2N = 2n − N
Table 5.1 shows an example of the 2’s complement representation for a 4-bit register.
2’s complement representation has the following properties:
• For a positive integer, its 2’s complement representation fits its standard binary representa-
tion.
• We get a single representation for the number zero.
• The range of representable values in an N-bit representation is −2N −1 ≤ x ≤ 2N −1 − 1.
• We obtain the negative number by computing the 2’s complement of their binary represen-
tation and changing the sign of the result
A practical method for computing the 2’s complement of a binary number is to copy the number
from right to left till we reach the first 1. Once we have copied the first 1, we complement the
remaining number bits. This means changing the zeros to ones and the ones to zeros. For instance,
the 2’complemet of the number 2 (0010 in a -bit representation) will be 1110,

copy the right end bit

0010 −−−−−−−−−−−−−−−→ ···0
copy f irst bit=1
0010 −−−−−−−−−−−→ · · 10
complement bit 0
0010 −−−−−−−−−−−→ · 110
complement bit 0
0010 −−−−−−−−−−−→ 1110
152 CHAPTER 5. C. ARITHM. AND ERRORS

An important property of 2’s complement representation is that we can compute the difference
between two numbers just by adding the first number to the 2’s complement of the second one.
We can see it using the two numbers of the previous example: 0010 is the binary representation of
the number 2 if we use a 4-bit representation. it 2’s complement, 1110 represent the number −2.
If we add them 1 :

0 0 1 0
1 1 1 0
1 0 0 0 0
0 0 0 0

The result of the addition yields a number with its first four bits as 0. The bit with value one
cannot be accommodated in a 4-bit register. We refer to the result as an overflow of the register
size. This bit that can not be stored is discarded after the addition. Thus, the result will be zero, as
can be expected after adding 2+(−2). This is a fundamental motivation to use the 2’s complement
representation: we do not need any additional special circuits to compute the difference between
two numbers because we can represent it straightforwardly by adding the minuend plus the 2’s
complement of the subtrahend.

5.2.1 Non-integer numbers: fixed-point and floating-point representa-

tions.
We use the non-integer representation to approximately represent real numbers. As we have seen
before, periodic rational numbers and irrational numbers cannot be exactly represented using a
finite number of decimals. This is the reason why their representation is always an approximation.
In the case of non-periodic rational numbers, the representation will be exact or approximated
depending on the number itself, the size of the computer registers and the representation system
we use. The two most common representation systems are the fixed-point and the floating-point
representations.

fixed-point representation. To represent a number using a fixed-point representation, we di-

vide the register into three parts:

• We set aside one bit to store the number sign.

• We represent the integer part of the number using a fixed-size field of bits.

• We represent the decimal part of the number using also a fixed-size field of bits.

For instance, if we are using a 16-bit register we can take a bit to represent the number sign, seven
bits for the integer part and eight bits for the decimal part. following this register sharing, the
representation of the number −10111.0011101101 would be:

s integer p. decimal p.
1 0 0 1 0 1 1 1 0 0 1 1 1 0 1 1
It is interesting to notice how we have two extra bits to represent the integer part of the number
because the integer part has only five digits and we have seven bits to represent it. However, to
1 the addition of binary numbers is like the addition of decimal numbers. We add then, bitwise, from right to

left, and when we add 1 and 1, the result is 10 (base 2). So, the resulting bit is set to zero, and we carry on the 1
to the next left bit.
5.2. COMPUTER NUMBER REPRESENTATION 153

represent the decimal part of the number, we have ten digits and only eight bits to represent it.
So, the representation truncates the number cutting out the last two decimals.
If we associate each bit with a power of two, in increasing order from right to left, we can
straightforwardly convert the number from its binary representation to its decimal form.
s integer p. decimal p.
1 0 0 1 0 1 1 1 0 0 1 1 1 0 1 1
- 26 25 24 23 22 21 20 2−1 2−2 2−3 2−4 2−5 2−6 2−7 2−8

Then, the number represented will be,

− (0 · 26 + 0 · 25 + 1 · 24 + 0 · 23 + 1 · 22 + 1 · 21 + 1 · 20 + 0 · 2−1 + 0 · 2−2 + 1 · 2−3 + 1 · 2−4 +

+ 1 · 2−5 + 0 · 2−6 + 1 · 2−7 + 1 · −8) = −23.23046875
Similarly, we can calculate the largest representable number,
s p. entera/ integer p. p. decimal/decimal p.
0 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
26 25 24 23 22 21 20 2−1 2−2 2−3 2−4 2−5 2−6 2−7 2−8

1 · 26 + 1 · 25 + 1 · 24 + 1 · 23 + 1 · 22 + 1 · 21 + 1 · 20 + 1 · 2−1 + 1 · 2−2 + 1 · 2−3 + 1 · 2−4 +

+ 1 · 2−5 + 1 · 2−6 + 1 · 2−7 + 1 · 2−8 = 127.99609375
The lowest representable number,
s integer p. decimal p.
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
- 26 25 24 23 22 21 20 2−1 2−2 2−3 2−4 2−5 2−6 2−7 2−8

− (1 · 26 + 1 · 25 + 1 · 24 + 1 · 23 + 1 · 22 + 1 · 21 + 1 · 20 + 1 · 2−1 + 1 · 2−2 + 1 · 2−3 + 1 · 2−4 +

+ 1 · 2−5 + 1 · 2−6 + 1 · 2−7 + 1 · 2−8 ) = −127.99609375
The largest integer,
s integer p. decimal p.
0 1 1 1 1 1 1 1 0 0 0 0 0 0 0 0
26 25 24 23 22 21 20 2−1 2−2 2−3 2−4 2−5 2−6 2−7 2−8

1 · 26 + 1 · 25 + 1 · 24 + 1 · 23 + 1 · 22 + 1 · 21 + 1 · 20 + 0 · 2−1 + 0 · 2−2 + 0 · 2−3 + 0 · 2−4 +

+ 0 · 2−5 + 0 · 2−6 + 0 · 2−7 + 0 · 2−8 = 127
The positive decimal number closest to 1,

s integer p. decimal p.
0 0 0 0 0 0 0 0 1 1 1 1 1 1 1 1
26 25 24 23 22 21 20 2−1 2−2 2−3 2−4 2−5 2−6 2−7 2−8

0 · 26 + 0 · 25 + 0 · 24 + 0 · 23 + 0 · 22 + 0 · 21 + 0 · 20 + 1 · 2−1 + 1 · 2−2 + 1 · 2−3 + 1 · 2−4 +

+ 1 · 2−5 + 1 · 2−6 + 1 · 2−7 + 1 · 2−8 = 0.99609375
154 CHAPTER 5. C. ARITHM. AND ERRORS

Or the positive decimal number closest to 0,

s integer p. decimal p.
0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1
26 25 24 23 22 21 20 2−1 2−2 2−3 2−4 2−5 2−6 2−7 2−8

0 · 26 + 0 · 25 + 0 · 24 + 0 · 23 + 0 · 22 + 0 · 21 + 0 · 20 + 0 · 2−1 + 0 · 2−2 + 0 · 2−3 + 0 · 2−4 +

+ 0 · 2−5 + 0 · 2−6 + 0 · 2−7 + 1 · 2−8 = 0.00390625

The fixed point representation was widely used in the first computers. However, using the same
register size, it is possible to represent a much wider spectrum of numbers using the floating-point
representation.

floating-point representation. The floating-point representation of numbers in a computer is

similar to the numeric representation known as scientific notation. In scientific notation, number
are represented as the product of a quantity by a power of ten:

234.25 ≡ 2.345 × 102 , 0.000234 ≡ 2.34 × 10−5 , −56.238 ≡ −5.6238 × 101 .

We shift the decimal point till just a single figure remains at its left, i.e, only a figure is use to
represent the integer part of the number. We need to multiply the result for the power of ten that
allows us to recovery the original number.
Many calculator a scientific computing sofware show the computed results on the screen using
the scientific format. Usually they utilize the following notation to do it,

calculator
−5.3572 × 10−3 −−−−−−→ −5.3572e − 03
First, if the number is negative, we represent the sign. (Usually, if the number is positive we
omit the sign symbol). Then, the part of the number 5.3572 which, when multiplied by the power
of ten, retrieves the number represented, this part is called the mantissa of the number. Eventually,
we represent the value of the (10) power’s exponent, −3, preceded by the letter e, which stand for
exponent. Scientific notation always assumes that the exponent belongs to a power of ten. It is
clear that we have the number thoroughly described if we supply its sing, mantissa and exponent.

number scientific r. calculator r. sign mantissa exponent

−327.43 −3.2743 × 102 −3.2743e2 − 3.2743 2

The floating-point follows exactly the same representation that we just described for the scien-
tific notation. The only difference is that we now use powers of two, which are the proper powers
of binary representation, instead of powers of ten.
So, we would represent the binary number
−1101.00101 as −1.10100101 × 23 , and the binary number 0.001011101 as 1.011101 × 2−3 .
The term floating-point means that the mantissa decimal point does not split the integer part
of the number from its decimal part. We need to use the exponent value to split the integer and
decimal part of the number. So, for an exponent equal to zero, the mantissa decimal point is in the
correct position to represent the number. For an exponent equal to 1, we must move the decimal
point one position towards the left. For an exponent equal to −1, one position towards the right.
For exponent 2, two positions towards the left. For exponent −2, two positions towards the right.
And so on.
5.2. COMPUTER NUMBER REPRESENTATION 155

How can we represent floating points binary numbers using a computer register? A straight-
forward solution is to divide the register into three parts, One for the sign, another to store the
mantissa and the third part for the exponent. Suppose that, as in the case of the fixed point
representation, we have got 16-bit registers. We can divide the register in three zones. The first
one, with a single bit to save the number sign. A second one of, let say 11 bits, to save the number
mantissa and we reserve the remaining four bits to hold the exponent in binary representation.
We could, then, represent the number −1.10100101 × 23 as,

sign. mantissa exponent.

1 1 1 0 1 0 0 1 0 1 0 0 0 0 1 1
- 20 2−1 2−2 2−3 2−4 2−5 2−6 2−7 2−8 2−9 2−10 23 22 21 20

Podrı́amos entonces representar el número −1.10100101 × 23 como,

Although the representation just described suits well the represented number, it is clear that
we cannot represent with it a negative number like, for example, 1.011101 × 2−3 . Furthermore,
we have arbitrarily divided the 16-bit register; we could have chosen a different allocation of bits
between the mantissa and exponent. In fact, until the 1990s, each computer adopted its own
representation for floating-point numbers. As a consequence, the same programs run on different
computers yield different results.
already in 1974, John Palmer from INTEL company, promote the creation of a standard, that
is, of a floating-point representation which might be adopted for all computers manufactures. The
first draft of the standard, inspired in Kahan’s work for the IBM 7094 computer (1968), was
prepare by Willian Kahan (Intel), Jerome Coonen (Berkeley) and Harold Stone (Berkeley): K-C-S
document. The standardisation process was slow. The first version of the standard was no ready
till the year 1985. The result of these works was the current standard, known as the IEEE 754
standard. IEEE are the acronym of Institute of Electrical and Electronic Engineers. The Institute
is an association born in the United States in the XIX century, and it has evolved to become
an international. The IEEE is devoted, among other things, to stablish international industrial
standards on the electricity, electronics and computer fields. Let’s see in some detail the IEEE 754
standard.

the IEEE 745 standard. The standard was established thinking in register of two different
size. The first one uses 32 bits to save floating-point numbers. It is known as the simple precision
standard. The second one uses 64 bits and it is Known as the double precision standard.

Simple precision. We will start describing the simple precision standard. First we take a
bit to save the sign. If the bits is one, then the number is negative. If the bits is zero the number
is positive. We use 23 bits of the remaining 32, to represent the number mantissa and the other
last 8 bits to represent the exponent.
If we examine a binary number’s mantissa when using the floating-point representation, we
realize that it must always take a value equal to one. Therefore, we only need to save in the chunk
of the register reserved for the mantissa the decimal part, i.e., the digits located on the right of the
decimal point, and consider the leading 1, belonging to the integer part of the number, implicit.
For instance, if we have the binary number 1.010111 × 23 we only save the figure 010111. We Know
the leading 1 of integer part is there, but we do not represent it. We will call this kind of mantissa
a normalised mantissa, we shall see why later on.
We need a way to represent positive and negative exponents. The standard prescribes the use
of representation known as biased exponent representation. Using 8 bits, we can represent up to
28 = 256 different numbers. These numbers expand from [00000000] = 0 to [11111111] = 255.
156 CHAPTER 5. C. ARITHM. AND ERRORS

Exceso a 127
127 biased
Exponent’s bits base-ten equivalent represented exponent
00000000 0 0 − 127 = −127
00000001 1 1 − 127 = −126
00000010 2 2 − 127 = −125
.. .. ..
. . .
01111110 126 126 − 127 = −1
01111111 127 127 − 127 = 0
10000000 128 128 − 127 = 1
.. .. ..
. . .
11111110 254 254 − 127 = 127
11111111 255 255 − 127 = 128

Table 5.2: 127-biased representation for a 8-bit exponent

We divide our numbers into two halves and take the lower half to represent negative numbers,
using the largest number to represent the zero. Of course, we use the upper half to represent
positive numbers. It is enough to subtract the number reserved to represent the zero to recover
the real exponent of the number. So we can consider that our representation of the exponent is
biased by the number used to represent the zero.
For the simple precision standard, the first half of available numbers ranges from 0 to 127 and
the second half from 128 to 255. Then, the representation is biased by 127. Table 5.2 shows several
examples on how to calculate this biased representation.
In conclusion: the 8 bits of the simple precision standard allows us to represent all integer
exponents from −127 to 128.
We have almost ready the description of the IEEE standard. However, some essential details
about representing very large and very small numbers remain. Let’s start with the very large
numbers. What is the most significant number we could represent using the simple precision
standard? We could suppose that, in principle, the number with all its mantissa values set to one
(the most significant possible mantissa) and all its exponent values set to one (the largest possible
exponent). However, the standard was thought to protect the computer operations against overflow
errors (see section 5.3 below). That protection is carried out by reserving the largest possible value
of the exponent. Therefore, when a register holds an exponent equal to 128, this value is not
considered a proper number. It is necessary to see the mantissa to know what the register does
represent:

• If the mantissa bits are all zero, the register holds the representation of infinite ∞. The
standard specifies the results of some operations on infinite: 1/∞ = 0, arctan(∞) = π/2.

• On the contrary, if the content of the mantissa is not zero, the content of the register is
consider as non-numeric (NaN, acronym of Not a Number). Some computations, which lack
of mathematical sense like 0/ =, ∞ × 0, ∞ − ∞, yields this NaN kind of results.

Therefore, the standard limits to 127 the maximum exponent which can be use to represent
a finite number. We can build as an example the largest number we can represent, different to
infinity. This number should have the largest possible mantissa, i.e. a mantissa with all values
set to one, and the largest possible exponent, 127, using the biasing representation of the simple
5.2. COMPUTER NUMBER REPRESENTATION 157

precision standard. This exponent correspond to a binary exponent 127 + 127 = 254 ≡ [11111110].
Thus, the largest number for simple precision representation would be,
sig. ←mantisa, 23 bits → ← exponente, 8 bits →
0 11111111111111111111111 11111110

This number in base-ten will take the form, (1 · 20 + 1 · 2−1 + 1 · 2−2 + 1 · · · + 1 · 2−23 ) × 2127 ≈
3.402823466385289 × 1038 . Where we have write in bold letter the implicit 1, which is not present
in the matissa.
Let’s turn now to the smallest numbers, according to the method we are using to represent the
numbers, the number closest to zero we can build would be the number with the smallest mantissa
and the smallest exponent. That is, a mantissa with all its bits set to zero and also an exponent
with only zeros. This number will be, (1·20 )·2−127 (remember that the standard uses a normalised
mantissa, hence it has a implicit integer 1).
If the standard would not use a mantissa with an implicit one, we could represent smaller
numbers. On the other hand, using a normalised mantissa, it is impossible to represent the
number zero. The standard developers decided to add the following rule to allow representing
small numbers, including the zero: If the exponent bits of a number are all equal to zero, that is,
if the exponent of the number is -127, the standard considers that the number’s mantissa has not
got an integer 1 implicit. In addition, the exponent of the number is taken as -126. This special
numbers are called denormalised numbers. Let’s see some examples.

sig. ←mantisa, 23 bits → ← exponent., 8 bits →

0 10100000000000000000000 00000000
Notice how the exponent has been reduced one unit (-126) to calculate the decimal represen-
tation of the number.
The closest number to zero we can represent using denormalised number, will be that with the
minimum non-zero mantissa,
ssig. ←mantissa, 23 bits → ← exponent., 8 bits →
0 0000000000000000000001 00000000
Which represented in base-ten will be the number: (0 · 2−1 + 0 · · · + 1 · 2−23 ) × 2−126 = 2−149 ≈
1.401298464324817 × 10−45 . We can compare this number with the closest number to zero we can
represent using a normalised mantissa. This number has all zeros mantissa and the smallest possible
non-zero exponent (Recall; an all-zero exponent automaticaly implies a denormalised number),
sig. ←mantissa, 23 bits → ← exponent., 8 bits →
0 0000000000000000000000 00000001
If we now represent the number in denary system we obtain: (1 · 20 + 0 · 2−1 + 0 · · · + 0 · 2−23 ) ×
−126
2 = 2−126 ≈ 1.175494350822288 × 10−38 .
We will calculate, as a final example, the largest number we can represented using denormalised
numbers. In this case, the number should have the largest possible mantissa (all ones) and a all-
zeros exponent,

sig. ←mantissa, 23 bits → ← exponent., 8 bits →

—
0 11111111111111111111111 00000000
In base-ten representation the number would be: (1 · 2 + 1 · 2−2 + 1 · · · + 1 · 2−23 ) × 2−126 ≈
−1

1.175494210692441 × 10−38 . These last two represented number are really near one to another. In
fact, we have already calculate their difference, when we got the denormalised number closest to
zero, (1 · 20 + 0 · 2−1 + 0 · · · + 0 · 2−23 ) × 2−126 − (1 · 2−1 + 1 · 2−2 + 1 · · · + 1 · 2−23 ) × 2−126 =
(0 · 2−1 + 0 · · · + 1 · 2−23 ) × 2−126 = 2−149 ≈ 1.401298464324817 × 10−45 .
158 CHAPTER 5. C. ARITHM. AND ERRORS

Double precision The difference between the single and the double precision standards is
in the size of the registers we use to represent the numbers. When working with double precision,
we use 64-bits size registers. Therefore, the size of the register is double than the size we use for
simple precision. The standard is just fit to the larger size of the registers; we reserve a bit for the
number’s sign, 52 bits for the mantissa and 11 bits for the exponent.
In this case the exponent can store 211 = 2048 different numbers (from 0 to 2047). Very much
alike the simple precision case, we split the exponent numbers into two halves, representing the
values with a 1023 bias. Therefore, the an exponent equal to zero represent the value 0 − 1023 =
−1023 and the exponent 2047 represent the value, 2047 − 1023 = 1024.
Again, we use the largest exponent value 1024 to represent the infinite ∞ with an all-zeros
companying mantissa. Otherwise, if the companying mantissa has any non-zero bit, then the
register does not represent a number NaN (overflow error). As an example, We can obtain the
largest representable number using the double precision standard,
sig. ←mantissa, 52 bits → ← exponent., 11 bits →
0 111111111111111111111111111111111111111111111111111 11111111110
The value in base-ten of the represented number is: (1 · 20 + 1 · 2−1 + 1 · 2−2 + 1 · · · + 1 · 2−52 ) ×
1023
2 ≈ 1.797693134862316 × 10308
Lastly, numbers with an exponent -1023, that is, numbers with an all-zero exponent, represent
denormalised numbers. We remove the implicit one from the mantissa and increase the exponent
on one unit, which takes the value -1022. For instance, we can compute the closest to zero number
using the double precision representation,
sig. ←mantissa, 52 bits → ← exponent., 11 bits →
0 000000000000000000000000000000000000000000000000001 00000000000
The number represented takes in base-ten the value: (0·2−1 +0·2−2 +0 · · ·+1·2−52 )×2−1022 ≈
4.940656458412465 × 10−324
Table 5.3 summarises and compares the characteristics of both standards we have seen,
We will finish with some examples of number representation using the IEEE 754 standard:
1. Which is the base-ten representation of the following number represented in the simple
precision IEEE 754 standard?
sig. ←mantissa, 23 bits → ← exponent., 8 bits →
1 11000000000000000000000 01111100

• The bit sign is 1 therefore, it is a negative number.

• The exponent is 0 · 27 + 1 · 26 + 1 · 25 + 1 · 24 + 1 · 23 + 1 · 22 + 0 · 21 + 0 · 20 − 127 = 124 − 127 = −3

• looking at the exponent we realise that the number has a normalised mantissa, 1+1·2−1 +1·2−2 =
1.75

• Thus, the number in denary representation is: (−1)1 × (1.75) × 2−3 = −0, 21875

2. Which is the base-ten representation of the following number represented in the simple
precision IEEE 754 standard?
sig. ←mantissa, 23 bits → ← exponent., 8 bits →
0 01000000000000000000000 10000001

• The sign bit is 0, therefore is a positive number.

• The exponent is, 1 · 27 + 0 · 26 + 0 · 25 + 0 · 24 + 0 · 23 + 0 · 22 + 0 ·1 +1 · 20 − 127 = 129 − 127 = 2

• looking at the exponent we see that it is a normalised number, 1 · 20 + 1 · 2−2 = 0.25

5.2. COMPUTER NUMBER REPRESENTATION 159

Simple precision. 32-bits Register. Biased exponent a 127

Mantissa exponent. (8 bits) represented number
(23 bits) (-exceso)(-bias)
0 0 − 127 = −127 (−1)bs · (0 · 2−1 + · · · + 0 · 2−23 ) × 20−127 = 0
̸= 0 0 − 127 ≡ −126 (−1)bs · (m1 · 2−1 + · · · + m23 · 2−23 ) × 20−127≡−126
1 − 127 = −126
8 0
∀ hasta/until (−1)bs · (1 + m1 · 2−1 + · · · + m23 · 2−23 ) × 2(e8 ·2 +···+e1 ·2 )−127
254 − 127 = 127
0 255 − 127 = 128 (−1)bs · (0 · 2−1 + · · · + 0 · 2−23 ) × 2255−127 ≡ ∞
̸= 0 255 − 127 = 128 (−1)bs · (m1 · 2−1 + · · · + m23 · 2−23 ) × 2255−127 ≡ NaN

Double precision. 64-bits register. Biased exponent 1023

Mantissa exponent. (11 bits) represented number
(52 bits) (-exceso)(-bias)
0 0 − 1023 = −1023 (−1)bs · (0 · 2−1 + · · · + 0 · 2−52 ) × 20−1023 = 0
̸= 0 0 − 1023 ≡ −1022 (−1)bs · (m1 · 2−1 + · · · + m23 · 2−52 ) × 20−1023≡−1022
1 − 1023 = −1022
10 0
∀ hasta (−1)bs · (1 + m1 · 2−1 + · · · + m until
52 · 2
−52
) × 2(e11 ·2 +···+e1 ·2 )−1023
2046 − 1023 = 1023
0 2047 − 1023 = 1024 (−1)bs · (0 · 2−1 + · · · + 0 · 2−52 ) × 22047−1023 ≡ ∞
̸= 0 2047 − 1023 = 1024 (−1)bs · (m1 · 2−1 + · · · + m52 · 2−52 ) × 22047−1023 ≡ NaN

Table 5.3: Comparison between the IEEE standards for floating point representation. (bs sign bit,
mi mantissa bit, ei exponent bit)

• Then, the denary representation of the number is: (−1)0 × (0.25) × 22 = +5.0

3. Which is the simple precision standard IEEE 754 representation of the number
347.625?
• We change the number to binary representation, 347.625 = 101011011.101
• We get the floating point representation of the number, 1.01011011101 × 28
• mantissa: 01011011101 (normalised)
• exponent:
bias 127 binary 8 bits
8 −−−−−→ 127 + 8 = 135 −−−−−−−−→ 10000111
• sign: 0 (positive)
Then the representation of 347.625 is,
sig. ←mantissa, 23 bits → ← exponent., 8 bits →
0 0101101110100000000000 10000111
5
4. What is the standard IEEE 754 simple precision representation of the number?: 3

• 5
3 = 1.66666 · · ·
• We pass the integer part to binary: 1 = 1 · 20
• We pass the decimal part to binary:
0.666666 · · · × 2 = 1.333333 · · ·
160 CHAPTER 5. C. ARITHM. AND ERRORS

0.333333 · · · × 2 = 0.666666 · · ·
binario
from this point on the period c 10 repeats indefinitely: 1.666666 −−−−−→ 1.101010 · · ·
• Mantissa: the number representation in binary is: 1.101010 · · · , and the same floating-point
representation, 1.101010 · · · × 20 . The mantissa is, 10101010101010101010101
• Exponent:
exceso 127 binario 8 bits
0 −−−−−−−→ 127 + 0 = 127 −−−−−−−−−→ 01111111
• Sign: 0 (positive)
5
therefore, 3 binary representation is,
sig. ←mantisa, 23 bits → ← exponente, 8 bits →
0 10101010101010101010101 01111111

5.3 Numerical representations Errors.

As we have already stated before, whatever numerical representation we use with a computer is
subject to errors due to the computer register’s finite size. We are going to focus on the mistakes we
make when using the IEEE754 standard for floating point representation to represent real numbers.
First, there is only a finite set of numbers with an exact representation. These number are those
directly obtained from the values —ones and zeros— contained in the register and interpreted
according to the standard specifications. These numbers are called machine numbers.
A real number will not be a machine number if,
1. Once we have represented it in floating point format, the number exponent is out of the
admitted range for exponents: it is too large or too small.
2. Once we have represented it in floating point format, the number mantissa has got more
digits than bits can store the IEEE 754 standard mantissa.
If the exponent is beyond the admissible range, it lead to overflow or underflow. Overflow happens
when the exponent is too large, while underflow occurs when the exponent is too small. In the
first case (overflow) the computer will assign the number the value pm∞. In the second case,
(underflow) the computer will assign the number the value zero.
If a number mantissa size exceeds the size of a representable mantissa, it is cut off to a suitable
size to make it representable. In other words, we replace the number with a nearby machine
number. this process is know as rounding-off and the error we make as rounding error.

5.3.1 round-off error

Suppose we have a non-machine number x = (1.a1 a2 · · · a23 a24 · · · ) × 2exp .

Approximation by Truncation. If we want to represent the number using the simple precision
standard, we can only represent the first 23 bit of its mantissa. A possible solution is to truncate
the number, eliminating all mantissa bits beyond the 23rd, x ≈ xE = (1.a1 a2 · · · a23 + 2−23 ) × 2exp .
Because we have eliminated some bits, the machine number xT we have taken to approximate our
number is less than it.

Approximation by Excess. Another option is to approximate the number using the next upper
machine number. This is the same that eliminate all mantissa bits beyond the 23th and then, and
add bit to the 23th mantissa position value, x ≈ xE = (1.a1 a2 · · · a23 + 2−23 ) × 2exp . Notice that
now, we are using a machine number greater than the number we are approximating.
5.3. NUM. REP. ERRORS 161

Rounding. Whatever number admits an approximate representation using the standard format
will be comprised of two machine numbers: xT ≤ x ≤ xE .
We will follow, as a general criterion, to approximate each real number, either by truncation or
excess, always using the nearest number. Whenever we round a number, we are getting an error
that we can define as the absolute value of the difference between the actual valor of the number
and the machine number chosen to represent it. This error is called absolute error.

Error absoluto/absolute error = |x − xr |

Where xr = xT , if we rounded the number by truncation or xr = xE if we rounded the number
by excess.
We can obtain the interval between two consecutive machine numbers by subtracting the lower
from the larger.

xE − xT = (1.a1 a2 · · · a23 + 2−23 ) × 2exp − (1.a1 a2 · · · a23 ) × 2exp = 2−23 × 2exp

If we approximate any real number inside the interval (xT , xE ) by the nearest of this last two,
the absolute error we get is always less or as much equal to half the interval,
1 1
|x − xr | ≤ |xE − xT | = · 2−23 · 2exp
2 2
This result is graphically illustrated in figure 5.1

1 1
2 |xE − xT | 2 |xE − xT |

xT xE
x = (1.a1 a2 · · · a23 a24 · · · ) × 2exp

Figure 5.1: Location of a non-machine number in relation to its rounding to a machine number
through truncation (xT ) and excess (xE ). When we round the number to the nearest of the two,
the error is always less than half the interval xE − xT .

If we look at the previous result in more detail, we can see that is the product of three factors:
the term 21 comes from approximating a real number by its nearest machine number. The term
2exp depends on the size of the number. For large numbers, this factor will be large, for small
numbers, small. The last term is a fixed factor 2−23 and it is directly related with the mantissa
size used in the representation. Indeed, if we had used the double precision standard, the absolute
error we would have gotten would be limited as,
1 1
|x − xr | ≤|xE − xT | = · 2−52 · 2exp
2 2
That is, the only factor which changes is precisely the term related with the mantise size. This
term is known as the machine precision or the machine epsilon (eps) and it takes always the value
2 raises to minus (-) the mantissa number-of-bits. Therfore, we can generalise the upper limit of
the absolute error as,
1 1
|x − xr | ≤|xE − xT | = · eps · 2exp
2 2
The machine epsilon meaning is still easy to understand if we define relative error,

|x − xr | 1 |xE − xT | 1 eps · 2exp 1

Error relativo/Relative error = ≤ = · ≤ eps
|x| 2 |x| 2 (1.a1 a2 · · · a23 a24 · · · ) × 2exp 2
162 CHAPTER 5. C. ARITHM. AND ERRORS

The relative error weighs the value of the gotten error wit the magnitude of the represented
number. A simple example may help to better understand it meaning. Consider a representation
system that only allows as ti represent integer numbers. If we what to represent the numbers 1.5
and 1000000.5 their representations would be 1 and 1000000. In both cases we get an absolute
error equal to 0.5. But if we compare this results with the represented numbers, on the first case
the error is half the number represented while in the second case this error does not reach the
millionth part of the number.
For the representations we are studying, the relative error we get for whatever representable
number is always least than half the machine epsilon,

1
xr = x · (1 + δ); |δ| ≤ · eps
2
A last comment on the machine epsilon considering it from the machine precision point of view.
The difference between two consecutive machine numbers is tightly related to the epsilon. If we
have a machine number and we want to increase it by the least possible quantity, such quantity is
precisely the epsilon times two rises to the number exponent. The reason for this is the way we
add numbers when using the numbers floating point representation. Suppose we have a number
whatsoever represented using the simple precision standard,
sig. ←mantissa, 23 bits → ← exponent., 8 bits →
0 11110000000000000000000 10000000
The represented number has exponent 27 − 127 = 1. Suppose now that we want to add to
this number the quantity 2−22 Its representation in the standard should use an all-zeros mantissa
(recall the implicit 1) and an exponent −22 − 127 = 105. So it should be,
sig. ←mantissa, 23 bits → ← exponent., 8 bits →
0 00000000000000000000000 01101001
To add two numbers using scientific notation, we have to represent both numbers using the
same exponent and then directly add their mantissa. We cannot decrease the exponent of the larger
number to reach the exponent of the lower one because this means adding numbers to the mantissa
integer part. However, this is impossible because there are no more bits available to enlarge the
mantissa integer part. Therefore, the only solution available is to increase the exponent of the
lower number till it reaches the value of the larger number exponent and to diminish the value of
the mantissa, de-normalising it; that is, no longer considering the implicit 1. So, in our example,
we must represent 2−22 using a exponent equal to 1, 2−22 → 2−23 · 21 ,
sig. ←mantisa (desnorm.), 23 bits → ← exponente, 8 bits →
0 00000000000000000000001 100000000
We can now get the sum the numbers, just adding their mantissas.
sig. ←mantisa, 23 bits → ← exponente, 8 bits →
0 11110000000000000000001 10000000
What happen if we sum a smaller number, for example 2−23 ? When we try to represent the
number with exponent 1, the number will takes the form 2−23 → 2−24 · 21 . It is easy to see the
problem, with a 23-bits mantissa it is not possible to represent the number 2−24 because there is
not place for it. the mantissa would be equal to zero and, due to we are using a denormalised
representation, the resulting number is zero. Therefore, when wi try to add it to the other number
we obtain this same number again.
This lead us to realise that the precision is not equal for every representable number but it
depends on the number size. If we associate the precision with the distance between to consecutive
machine numbers we may write: precision = eps · 2exp and its values duplicates each time we
increase the exponent in one unit. Figure 5.2 depicts this phenomenon.
5.4. ERRORS DERIVED FROM ARITHMETIC OPERATION 163

Figure 5.2: Graphic illustration of precision change with the magnitude of represented numbers

5.3.2 Underflow and Overflow errors

In previous sections we have seen one of the computer limitations to represent numbers; a finite
mantissa makes that only a few numbers has an exact representation. We need to approximate
the remaining numbers, getting in the process round-off errors. In this section, we focus on the
limitations derived from the use of finite exponents, which the range of number we can represent
using a computer.
Figure 5.3 shows a schematic view of the representable numbers range. The lowest negative
representable number is define using a sign bit equal to 1, to indicate that it is a negative number,
an the largest mantissa and exponent which, according to the standard specification, still represent
a finite number. Any lowest negative number will produce an error technically known as negative
overflow error. The negative number closest to zero that we can represent will be that which
has a sign bit 1, the (denormalised) smallest possible mantissa and the lowest possible exponent.
Whatever number closer than it to zero will be represented as zero by the computer. In this case
we get an error known as negative underflow error.
Similar to how we have defined the lower negative number we can represent, we can also define
the largest positive representable number. The only difference with the lower number case will be
the sign bit, which now is equal to zero, to indicate that the represented number is positive. If
we try to represent a number greater than largest representable, we will get a positive overflow
error. Lastly, the positive number closest to zero we can represent is equal to the negative one
except for the sign bit, which must now be equal to zero. If we try to represent a smaller number
than the smallest representable, the computer cannot tell one from the other, producing a positive
underflow and taking zero as the representation of the number.

No Representable
No representable Negativos expresables Positivos expresables No representable
Non-representable Negative expressible Positive expressible Non-representable

−(2.0 − 2−23 ) · 2127 −2−23 · 2−126 0 2−23 · 2−126 (2.0 − 2−23 ) · 2127
Non-representable

Figure 5.3: Representable numbers and over/under-flows in the IEEE 754 simple precision stan-
dard.

5.4 Errors derived from arithmetic operation

So far, we have focused on the numerical representation problem. In this section we will introduce
a new kind of errors we are still more critical from the point of view of scientific computing; these
are errors derived from the arithmetic operations.
We partially introduced the problem in the previous section when we spoke about computer
precision and how we need to match the exponents of the addends, making them equal to the
largest one, before performing the addition in floating-point representation. Let see in more detail
some consequences of performing floating point arithmetic operations.
164 CHAPTER 5. C. ARITHM. AND ERRORS

5.4.1 Accumulation of round-off errors.

We start looking at the addition process of two base-ten numbers represented on floating-point
format. Suppose we want to add number 99.99 and 0.161. Besides, suppose we use the floating-
point representation with the following two limitation: the mantissa is four digit size and the
exponent has only two digits.
If we add the number as they are written above, without use the floating-point format the
result would be,

99.99 + 0.161 = 100.151

Suppose now that we represent the number following the floating-point format previously de-
scribed, 99.99 = 9, 999 × 101 , 0.161 = 1.610 × 10−1 . We will divided the addition process of this
two numbers in four steps which, schematically, shows the procedure the computer will carry out.
1. Alignment. This first step involves representing the smaller number using the exponent of
the larger one: We shift the smaller number digits rightwards so many positions as the difference
between the exponents of the numbers, and we equal the exponent of the smaller number to the
larger one.

1.610 × 10−1 → 0.016 × 101

As it can be expected, the rightwards shift of the mantissa digits causes the lost of the last
digits of the number. So we find here a first rounding error in the alignment procedure.
2. Operation. Once the addends have been aligned, we can carry out the operation. Whether
the operation is an addition and the number signs are equal or whether the operation is a sub-
traction and the signs are different, it is enough to add the mantissas. Otherwise, we subtract
them.
We must check after perform the computation if the mantissa has overflowed. In our example,
9.999 · 101
+ 0.016 · 101
10.015 · 101
We cannot use directly the result of the addends mantissas sum as the mantissa of the result
because it need an extra digit, i.e., this addition yields a result which overflows the size of the
mantissa.
3. Normalisation. If the mantissa has overflowed then, it is necessary to normalise it again,

10.015 × 101 = 1.0015 × 102

4. Rounding. The shifting of the digits makes that not all the digits can be accommodate
inside the mantissa. Therefore, it is necessary to round it (by truncation or excess).

1.0015 × 102 = 1.002 × 102

Eventually, we check if the operations we have carried out have produce an exponent overflow,
because in this case the number would be not representable: ∞ or 0.
5. Renormalisation. For number in binary representation is sometimes necessary to normalise
again the mantissa after carried out the rounding. Suppose as in the previous example that we have
a four-bit size mantissa, but now we numbers in binary representation, and that after performing
an addition operation it yield the result 11.111 × 21 . The mantissa has overflowed and then we
have to normalised it 11.111 × 21 → 1.1111 × 22 . We have to round the mantissa and, due to
the number is just on the interval half between two machine number, we round it by excess:
5.4. OP. ARITHM. ERRORS 165

1.1111 × 22 → 10.000 × 22 . The round-by-excess process has overflowed the mantissa one more
time and, thus, we have to nomarlise it again 10.000 × 22 → 1.000 × 23 .
If we analise the error we get after perform the addition in our previous example, In exact
arithmetic, the sum of the numbers yields 100.152. When we use floating-point representation
with a four-bit mantissa the result is 1.002 × 102 = 100.2. Then, the absolute error we get is,

|100.151 − 100.2| = 0.049

and the relative error,

100.151 − 100.2
≈ 0.000489
100.152
In general, it is possible to demonstrate that for any basic arithmetic operation ⊙ (addition,
subtraction, multiplication, division) and two machine numbers x, y, it fulfils that,

flotante(x ⊙ y) = (x ⊙ y) · (1 + δ) |δ| ≤ eps

This assertion is know as the fundamental axiom of floating-point arithmetic: the machine eps
is the upper bound of the relative error in any basic floating-point arithmetic operation performed
between to machine numbers.
The fundamental axiom of floating-point arithmetic sets an error upper bound. In practice,
when computing, we link many basic arithmetical operations. This operation linking will lead to
an accumulation of the errors we get in each basic operation.
for instance, suppose we want to carry out the operation, x · (y + z) in floating-point arithmetic.
We may describe the operation as follows,

flotante (x · (y + z)) = (x · flotante(y + z)) · (1 + δ1 )

= (x · (y + z)) · (1 + δ2 ) · (1 + δ1 )
≈ (x · (y + z)) · (1 + 2δ)

Where δ1 represents the relative error after performing the product and δ2 the error derived
from the addition. Both errors are bounded by the machine eps. (δ1 , δ2 ≤ eps). Notice that δ can
be approximated as,

(1 + δ1 ) · (1 + δ2 ) = 1 + δ1 δ2 + δ1 + δ2 ≈ 1 + 2δ, δ = max(δ1 , δ2 )
We may conclude that, in this case, the rounding error doubles the error of a simple arithmetical
operation. An, in general, the error tends to multiply with the number of linked operations.
In some situations, the rounding errors derived from an arithmetical operation are notable. For
example, when we add small quantities with large ones,

flotante(1.5 · 1038 + 1.0 · 100 ) = 1.5 · 1038 + 0

In this case, on the alignment process, it is necessary to shift the small number mantissa 38
decimal places to be able to sum it to the large one. Using any mantissa with less than 38 digits
is as much as converting the second addend into zero.
Another example is the lost of the associative property,
) (
x = 1.5 · 1038 (x + y) + 1 = 1
38
⇒ (x + y) + 1 ̸= x + (y + 1)
y = −1.5 · 10 x + (y + 1) = 0
166 CHAPTER 5. C. ARITHM. AND ERRORS

The results may be subject to very large errors when the arithmetical operation is the subtrac-
tion of two very similar quantities. Suppose we want to carry out the operation 100.1−99.35 = 0.75
using a floating-point representation with 4-digits mantissa and the numbers represented in base-10.
(100.1 = 1.001 · 102 , 99.35 = 9.935 · 101 )
1. Alignment

9.935 · 101 → 0.994 · 102

2. Operation
1.001 · 102
− 0.994 · 102
0.007 · 102
3. Normalisation

0.007 · 102 → 7.000 · 10−1

Steps 4 and 5 are not necessary in this example If we now calculate the absolute rounding error
we get,

|0.75 − 0.7| = 0.05

And the relative error,

0.75 − 0.7
≈ 0.0666
0.75
Hence, we have approximately a 6.7% error. In this case the problem arises during the alignment
process, because we lost a significant digit.
In a floating-point representation system, where we represent the number in base-β and the
mantissa size is p, if we perform differentiation using p digits, we can expect a relative rounding
error that can reach a maximum value of β − 1.
Let’s see an example for numbers written in base-10 (β = 10). Suppose we are using a four-
digits mantissa and we want to carry out the operation 1.000 · 100 − 9.999 · 10−1 . The exact result
would be, 1 − 0.9999 = 0.0001. Nevertheless, in the alignment process,

9.999 · 10−1 → 0.9999 · 100

If we round the number by excess the result of the substraction will be zero. If we round the
number by truncation,
1.000 · 100
− 0.999 · 100
0.001 · 100
and the relative error we get would be,

1.000 · 10−4 − 1.000 · 10−3 10−4 (10 − 1)

−4
= = 10 − 1 ⇒ β − 1
1.000 · 10 10−4
To alleviate these problems, the IEEE 754 standard establishes that arithmetical operation
should be performed always using two extra digits to save the intermediate results. These two
extra bits are called protection digit or guard digits. If we repeat the subtraction, 100.1 − 99.35,
but using now the two guard digits,
1. Alignment.

9.93500· = 101 → 0.99350 · 102

5.4. OP. ARITHM. ERRORS 167

2. Operation
1.00100 · 102
− 0.99350 · 102
0.00750 · 102
3. normalisation

0.00750 · 102 → 7.500 · 10−1

In this case we obtain the exact result. In general, using two guards bit to perform subtractions,
the rounding error is always less than the machine epsilon.

5.4.2 Catastrophic cancellation.

Catastrophic cancellation takes place when performing an arithmetical operation –typically the
subtraction– the more significant digits that are not affected by the rounding cancel between
them. The result will then contain digits that are indeed affected by the rounding. Let’s see an
example,
Suppose we can carry out the operation b2 − 4 · a · c using base-10 numbers and a five-digit
mantissa. Suppose also that we employ the numbers: b = 3.3357 · 100 , a = 1.2200 · 100 , c =
2.2800 · 100 . The exact result of the operation is,

b2 − 4 · a · c = 4.944 · 10−4 (5.1)

If we perform the operations in the requested representation,

b2 = 1.112689 · 101
4 · a · c = 1.112640 · 101
b2 − 4 · a · c = 5.0000 · 10−4

We get a relative error of around a 1%. We used two guard bits in the intermediate operations,
so the subtraction makes no mistake. The error is due to the rounding of previous results. Once
the subtraction is performed, the result only holds digits affected by the previous rounding because
the first (more significant) digits have been cancelled during the operation.
As a thumb rule, we have to avoid, when performing arithmetical operation, the subtraction of
of almost equal quantities. Let’s see another example, to clarify this idea. Suppose we whant to
carry out the operation,
p
y= x2 + 1 − 1

For small values of x, this operation implies a cancellation with the loss of significant digits. A
possible solution is to write the equation using an alternative form. If we multiply and divide by
the conjugate,

√ !
p x2 + 1 + 1 x2
y = ( x2 + 1 − 1) · √ =√
x2 + 1 + 1 x2 + 1 + 1
 168 CHAPTER 5. C. ARITHM. AND ERRORS

5.4.3 Overflow and underflow errors

When we spoke above the representable numbers finite rank, we showed how the IEEE 754 standard
deals with the numbers that overflows the representation boundaries. When we perform arithmetic
operations it is possible to arrive to situations where the operation result be a non-representable
number because its magnitude is too large (positive or negative overflow) or because its magnitude
is too small (positive or negative underflow). We will present an example that allows us to show
this phenomenon; the computing of a vector module,
v
u n
uX
||⃗v || = ||(v1 , v2 · · · vn || = t vi2
i=1

The following Python module includes two functions to calculate a vector norm. Let’s first
examine the function norma_naive.

norm [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Wed Jul 17 14:43:50 2024
5 This
6 @author: juan
7 """
8

9 import numpy as np
10

11 def norma_naive(x):
12 """
13 This program (naive) calculates the norm of a vector
14 Parameters
15 ----------
16 x : TYPE Real Vector
17 DESCRIPTION. imput vector whose norm we wamma get
18

19 Returns
20 -------
21 m: Type Real number
22 Description. The vector norm
23 """
24

25

26 m = 0.
27 for e in x:
28 m = m +e**2
29

30 m =[Link](m)
31 return m
32

33 def norma_robust(x):
34 """
35 This program calculates the norm of a vector
36 Parameters
 5.4. OP. ARITHM. ERRORS 169

37 ----------
38 x : TYPE Real Vector
39 DESCRIPTION. imput vector whose norm we wamma get
40

41 Returns
42 -------
43 m: Type Real number
44 Description. The vector norm
45 """
46 l =[Link][0]
47 if l==1:
48 m = [Link](x[0])
49 else:
50 mayor = 0
51 mscalado = -1
52 for e in x:
53 if e == 0:
54 continue
55 modxi = [Link](e)
56 if mayor<modxi:
57 mscalado = 1 + mscalado*(mayor/modxi)**2
58 mayor = modxi
59 else:
60 mscalado = mscalado+(modxi/mayor)**2
61 m = mayor*[Link](mscalado)
62 return m
63

norm_naive will generate an overflow error even for n=1, provide we introduce a number whose
square is greater that the largest representable number. If we introduce the number 21024/2 =
1.340780792994260e + 154, our function will return inf and we also get a warning from Python.

In [77]: x= [Link]([2**(1024/2)])

In [78]: m = norma(x)
/home/juan/LCC_Python/codigos/aritmetica_del_computador/norm_naive.py:28:
RuntimeWarning: overflow encountered in scalar power
m = m +e**2

In [79]: m
Out[79]: inf

So, the result we get is an overflow error. The problem takes place in the loop when it calculate
the square of the number,
 2
21024/2 = 21024 > (2 − 2−52 ) · 21023

once the programs produces the overflow, the remaining operation are invalid. Like in previous
cases, we can solve this kind of problems modifying the method we follow to compute the result.
A first step would be to make the norm equal to the absolute value of the number in case n=1.
This way we can correctly compute the norm of the number proposed in the previous example.
170 CHAPTER 5. C. ARITHM. AND ERRORS

However, if we can widen the rank of the vector for which we want to compute the norm, more
than this fix is needed. We can improve our code using a small trick: we divide the elements of
the vector for the largest element in absolute value, then calculate the module of the vector so
obtained, and eventually multiply again the result for the largest number.

v v
u n u n  2
uX
2
uX vi
t vi = | max vi | · t
i=1 i=1
| max vi |

The second function included in the python script above, norm_robust, computes the norm of
a vector using the procedure just described,
In [109]: x = [Link]([0,0,0,0,0,2**(1024/2),2**(1024/2)])

In [110]: x
Out[110]:
array([0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000,
0.00000000e+000, 1.34078079e+154, 1.34078079e+154])

In [111]: m = norma_naive(x)
/home/juan/LCC_Python/codigos/aritmetica_del_computador/norm_naive.py:28:
RuntimeWarning: overflow encountered in scalar power
m = m +e**2

In [112]: m
Out[112]: inf

In [113]: m = norma_robust(x)

In [114]: m
Out[114]: 1.8961503816218355e+154

If we apply this program to obtaint the norm of the vector,

x = [21024/2 21024/2 ]
We get the folloing result,

m = 1.8961503816218355 · 10154
Instead an overflow error (inf )
Chapter 6

Methods for calculating the roots

of a function

Sólo el tiempo muestra al hombre

justo mientras que podrı́as
conocer al perverso en un sólo dı́a

Sófocles. Edipo rey

6.1 Roots of a function

The roots of a real function f (x) : R → R are those values x = r for which f (r) = 0
Obtaining the roots of a function is of great interest because many scientific problems could be
reduced to the calculus of the root or roots of a function.
Obtaining the root of a function is trivial for those cases in which we know the inverse function
f −1 , (f (x) = y ⇒ f −1 (y) = x). In this case, r = f −1 (0). For instance,

f (x) = x2 − 4
f −1 (y) = ± y + 4 ⇒ r = f −1 (0) = ±2
p

Nevertheless, there are many cases where functions cannot be inverted. An example from
physics is Kepler’s equation for planetary orbits.

x − a sin(x) = b
We are given two known parameters, a and b, and we need to find the value of x. To solve
Kepler’s equation, we must calculate the roots of the function f (x) = x − a sin(x) − b. Figure 6.1
provides an example of this function. Since the inverse function is unknown, we can only obtain
an approximate root value using numerical methods.

Iterative methods This chapter solely focuses on iterative techniques. These methods begin
with an initial guess for the root, denoted as r0 , and then gradually refine the solution through a
series of iterative steps. With each step, the solution gets closer and closer to the actual value of

171
172 CHAPTER 6. ROOTS

y=x−40sin(x)−2 80

0 5 10 15 20 25 30 35 40
x

Figure 6.1: An Example of Kepler’s equation is taking a = 40 and b = 2.

the root. The successive approximations of the root are carried out based on the previous ones.

r0 → r1 → r2 → · · · → rk → · · ·
|f (r0 )| ≥ |f (r1 )| ≥ |f (r2 )| ≥ · · · ≥ |f (rk )| ≥ · · ·

Going from one approximate solution to the next is called iteration. Usually, the same mathe-
matical operations are repeated in each iteration.
The iterative process comes to a halt when the obtained solution is considered to be close
enough to the actual solution to be considered a good approximation. Typically, a value is set as
a convergence criterion, known as tolerance. The iterations continue until a value rn is obtained
that satisfies the criterion.
|f (rn )| ≤ (tol)
The algorithm used to obtain the root is then said to have converged in n iterations. On the
other hand, it is important to note that algorithms for calculating the root of a function do not
always converge. Sometimes it is not possible to get closer and closer to the value of the root either
because of the nature of the function or because the algorithm is not suitable for obtaining it.

Local search . A function can have any number of roots, even infinite ones, just think for
example of trigonometric functions such as cos(x). An important feature of the methods described
in this chapter is that they are only able to approximate one root. The root of the function to
6.2. LOCAL ITERATIVE METHODS 173

which the method converges depends on the initial value r0 with which the iterative search is
started 1 . This is why they are called local methods. If we want to find several (or all) the roots
of a given function, it is necessary to use the method on each of the roots separately, changing the
starting point each time.

6.2 Local iterative methods

6.2.1 Bisection Method
Bolzano’s theorem

Let f (x) be a continuous function defined in an interval [a, b]. Then, if f (a) · f (b) < 0
(therefore,f (a) < 0 and f (b) > 0 or f (a) > 0 and f (b) < 0), there exists at least a
point inside the interval c ∈ [a, b] such that f (c) = 0. (figure: 6.2)

0
y

Raíz=−0.703467422514226
−2

−4

−6
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
x

Figure 6.2: Bolzano’s theorem.

The well-known Bolzano’s theorem provides the simplest method of approximating the root of
a function: We start from an initial interval in which the theorem is satisfied; and we successively
limit the interval containing the root, reducing it by half in each iteration, so that in each new
interval Bolzano’s theorem is always satisfied.
1 Sometimes, as we will see later on, the algorithm is not given an initial value, but an interval in which to search

for the root

174 CHAPTER 6. ROOTS

Begin with [a, b]

such as
f (a) · f (b) < 0

a+b
Compute c = 2 , f (c)

Yes Convergence:
is |f (c)| ≤ tol?
finish

No

Yes b=c
is f (a) · f (c) < 0?
f (b) = f (c)

No
a=c
f (a) = f (c)

Figure 6.3: Flowchart of the bisection method

Figure 6.3 shows a flowchart corresponding to the bisection method. The starting point is an
interval [a, b] in which Bolzano’s theorem is satisfied, and which therefore contains at least one root.
It is interesting to note that Bolzano’s theorem is satisfied whenever the function is continuous on
the interval [a, b] and there are an odd number of roots. This is why it is important to choose the
interval [a, b] carefully. If there is more than one root, the algorithm may not converge.
Once we have the interval we calculate the midpoint c. The value taken by the function at c,
i.e. f (c), is then compared with the tolerance. If the value is less than the tolerance, the algorithm
has found an approximate value of the root with the required tolerance, so c is the root and no
further search is necessary. If, on the other hand, f (c) is above the required tolerance, we compare
its sign with the sign of the function at either end of the interval. In the flowchart we have chosen
the end a, but the algorithm would work equally well if we chose b. If the sign of f (c) coincides
with the sign that the function takes at the end of the chosen interval, c replaces the end, (we
make a = c and f (a) = f (c)) if on the contrary the sign is different, we make c replace the other
end of the interval (we make b = c and f (b) = f (c)). This process will be repeated until f (c) is
satisfied.
The process is shown graphically in figure 6.4, for a particular case. This is to obtain the root of
the function shown in figure 6.2, f (x) = ex − x2 . This function has a single root: r ≈ −0.7035. To
start the algorithm, an interval [a = −2, b = 2] has been chosen. Figure 6.4, shows three successive
iterations, and the final solution, which is obtained after eight iterations in this example, for which
a tolerance tol = 0.01 has been used. The sequence of graphs also shows the evolution of the search
interval, [−2, 2] → [−2, 0] → [−1, 0] → [−1, −0.5] · · · ; as well as the alternative change of the right
or left limit, to ensure that the root is always within the successive search intervals obtained.

6.2.2 False position method or (Regula falsi )

This method is an improvement on the previous one as it converges more quickly in general. The
idea is to modify the way in which we calculate the point c. In the case of the bisection method,
the criterion consisted of taking the midpoint of the interval containing the root at each iteration.
The linear interpolation method chooses as point c the point of intersection with the x-axis of the
straight line that passes through the points (a, f (a)) and (b, f (b)). That is, the line that cuts the
6.2. LOCAL ITERATIVE METHODS 175

6 6

4 4

2 2 c1

a a
0 0
y

y
b b

−2 −2

−4 −4

−6 −6
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
x x

(a) Starting interval (b) iteration 1

6 6

4 4

2 c 2 c
1 1
c3
a a a
2
0 0
y

b1 b b1 b
c2 c2
−2 −2

−4 −4

−6 −6
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
x x

(c) iteration 2 (d) iteration 3

6

2 c1
c3
a a2
0
y

R b1 b
c2
−2

−4

−6
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
x

(e) iteration 6: Root found

Figure 6.4: process of obtaining the root of a function by the bisection method.
176 CHAPTER 6. ROOTS

function f (x) at both limits of the interval containing the root we are looking for. The line through
both points can be constructed from them as,

f (a) − f (b)
y= · (x − b) + f (b)
a−b
The intersection point with the x-axis, will be the value for c, is obtained when y = 0,

f (a) − f (b)
0= · (x − b) + f (b)
a−b
and replacing cequivx in the above equation, we get

f (b)
c=b− · (b − a)
f (b) − f (a)
Figure 6.5 shows the position of point c computed with this method.

a
0
y

c b

−2

−4

−6
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
x

Figure 6.5: Obtaining the line joining the extremes of an interval [a, b] containing a root of the
function

Otherwise, the procedure is the same as in the case of the bisection method. We start with an
interval [a, b] containing a root, obtain the point c by the procedure described above and exchange
c with the end of the interval whose image f (a) or f (b) has the same sign as f (c). The procedure
is repeated iteratively until f (c) is less than the preset tolerance value.
The flowchart for the false position method is shown in figure 6.6. As can be seen, it is identical
to the bisection method except in the step where the value of c is obtained, where the calculation of
6.2. LOCAL ITERATIVE METHODS 177

Begin with [a, b]

being
f (a) · f (b) < 0

Compute
f (b)
c=b− · (b − a), f (c)
f (b)−f (a)

Yes It converges:
is |f (c)| ≤ tol?
finish

No

Yes b=c
is f (a) · f (c) < 0?
f (b) = f (c)

No
a=c
f (a) = f (c)

Figure 6.6: Flowchart of false position method

the midpoint of the search interval has been replaced by the calculation of the point of intersection
with the abscissa axis of the straight line joining the ends of the interval.
Figure 6.7 shows graphically the iterative process followed to obtain the root of a function in
an interval by means of the false position method. The same function and the same initial interval
have been used as in the case of bisection.
It is easy to see, however, that the intermediate points that the algorithm obtains until con-
verging to the root are different. In fact, the algorithm now uses only seven iterations to obtain
the root, using the same value for the tolerance, 0.01, as was used in the bisection method.
One final remark, it was said at the beginning that this method is an improvement on the
previous method of bisection. This is not always true. The bisection method has a constant
convergence rate, each iteration halving the search space. However, the convergence of the false
position method depends on the function f (x) and the relative position of the initial points (a, f (a))
and (b, f (b)) with respect to the root. This is why it is not always true that it converges faster
than the bisection method. On the other hand, the calculation of the successive values of the point
c requires more arithmetic operations in the false position method, making each iteration slower
than in the case of bisection.

6.2.3 Newton-Raphson method

Newton’s method is based on the expansion of a Taylor series function f (x) around a point x0 ,

1 1
f (x) ≈ f (x0 ) + f ′ (x0 )(x − x0 ) + f ′′ (x0 )(x − x0 )2 + · · · + f (n) (x0 )(x − x0 )n + · · ·
2 n!
It belongs to a family of methods widely used in numerical calculus. The idea in the case of
Newton’s method is to approximate the function for which one wishes to obtain the root, by means
of the first term of Taylor’s series. That is to say, to approximate locally f (x), in the environment
of x0 by the straight line,
f (x0 ) + f ′ (x0 )(x − x0 )
This line is precisely the tangent line to the curve f (x) at the point x0 (figure 6.8).
178 CHAPTER 6. ROOTS

6 6

4 4

2 2
c1

a a
0 0
y

y
b b1 b

−2 −2

−4 −4

−6 −6
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
x x

(a) Starting interval (b) iteration 1

6 6

4 4

2 2
c c
1 1
c2 c2
c3
a a
0 0
y

b2 b1 b b2 b1 b
b3

−2 −2

−4 −4

−6 −6
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5
x x

(c) iteration 2 (d) iteration 3

6

2
c1
R c2
c3
a
0
b3 b2 b1 b

−2

−4

−6
−2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 2.5

(e) iteration 6: Root found

Figure 6.7: Process of obtaining the root of a function by the method of linear interpolation
6.2. LOCAL ITERATIVE METHODS 179

12

10

6
y=f(x0)+f’(x 0)(x−x0)

4
y

0 x1 x0

−2

−4
−3 −2 −1 0 1 2 3
x
[h]

Figure 6.8: Tangent line to the function f (x) at the point x0 .

The method consists of obtaining the intersection of this tangent line with the abscissa axis,

0 = f (x0 ) + f ′ (x0 )(x − x0 )

and obtaining x,

f (x0 )
x = x0 −
f ′ (x0 )
Next, the function is evaluated at the point obtained xrightarrowf (x). As in the previous
methods, the value of f (x) is compared with a certain pre-set tolerance. If it is smaller, the value
of x is taken as the root of the function. If not, the algorithm is applied again, now using the
value of x just obtained as the starting point. Each calculation constitutes a new iteration and the
successive values obtained for x converge to the root,

f (x0 ) f (x1 ) f (xn−1 )

x0 → x1 = x0 − ′
→ x2 = x1 − ′ → · · · → xn = xn−1 − ′ → ···
f (x0 ) f (x1 ) f (xn−1 )

Figure 6.9 shows a flowchart for Newton’s method. Compared to the flowcharts of the previous
algorithms, Newton’s algorithm is somewhat simpler to implement. However, the value of the
function and its derivative must be evaluated at each iteration.
The calculation of the derivative is the weak point of this algorithm, since for values x0 close to
a local minimum or maximum we will obtain values of the derivative close to zero, which may cause
180 CHAPTER 6. ROOTS

Initial point x0

Compute
x = x0 − ff′(x 0)
(x0 ) , f (x)

Yes it converges:
is |f (x)| ≤ tol?
finish

No

x0 = x

Figure 6.9: Flowchart of Newton-Raphson method

an overflow error when calculating the point of intersection of the tangent line with the abscissa
axis or cause the algorithm to converge to a root far from the initial search point.
Figure 6.10 shows an example of obtaining the root of a function using Newton’s method. The
method is faster than the previous two, i.e., starting from a comparable distance to the root, it is
the one that converges in the fewest iterations.
In the example in the figure, the root has been obtained for the same function as in the examples
of the bisection and false position method. However, we have started at a point further away from
the root, so that the evolution of the algorithm can be better observed in the figure. In each of the
graphs that make up the figure, the steps of the algorithm can be observed: given the point xi ,
the tangent line to the function f (x) at the point is calculated and a new point xi+1 is obtained,
as the intersection of this tangent line with the abscissa axis.
In this example the algorithm converges in the five iterations shown in the figure, for the same
tolerance used in the previous methods, tol = 0.01. The starting point used was x0 = 2.5, so
it is outside the interval [−2, 2] and further away from the root than in the case of the previous
methods.

6.2.4 Secant method

The secant method could be considered a variant of Newton’s method in which the tangent line
to the point x0 is replaced by the secant line joining two points obtained in successive iterations.
The idea is to approximate the derivative of the function f at the point xn by the slope of a secant
line, i.e. a line that cuts the function at two points,

f (xn ) − f (xn−1 )
f ′ (xn ) ≈
xn − xn−1
The successive approximations to the root of the function are obtained in a similar way to
Newton’s method, by simply substituting the derivative of the function by its approximate value,

f (xn ) (xn − xn−1 ) · f (xn )

xn+1 = xn − ′
≈ xn −
f (xn ) f (xn ) − f (xn−1 )
To start the algorithm, two starting points must be used in this case. Figure 6.11 shows an
example.
The method could at this point be confused with false position method, but it has two important
differences: Firstly, the choice of the two initial points x0 and x1 , need not form an interval
6.2. LOCAL ITERATIVE METHODS 181

12 12

10 10

8 8

6 6

4 4
y

y
2 2

0 x1 x0 0 x1 x0

−2 −2

−4 −4
−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3
x x

(a) starting interval (b) iteration 1

12 12

10 10

8 8

6 6

4 4
y

2 2

x3
0 x2 x1 x0 0 x2 x1 x0

−2 −2

−4 −4
−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3
x x

(c) iteration 2 (d) iteration 3

12
12

10
10

8
8

6
6

4
y

4
y

2 2

x3 x4 x3 x
4
0 0 x x1 x0
x2 x1 x0 2

−2 Raíz
−2

−4 −4
−3 −2 −1 0 1 2 3 −3 −2 −1 0 1 2 3
x x

(e) iteration 4 (f) iteration 5: Root of the function

Figure 6.10: Process of obtaining the root of a function by Newton’s method

182 CHAPTER 6. ROOTS

x
0
0
x2 x1

−2

−4
y

−6

−8

−10
−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1
x

Figure 6.11: Secant line to the function f (x) at points x0 and x1 .

6.2. LOCAL ITERATIVE METHODS 183

containing the root. That is, they could both be located on the same side of the root. Secondly,
the points obtained are substituted in order, so that the new secant line is always constructed
from the last two points obtained, without paying attention to whether the value of the root is
contained between them. (The signs of the function at the points are not compared to see which
one is substituted, as in the case of the interpolation method).

Begin with two initial points x0 , x1

Compute
(x1 −x0 )·f (x1 )
x = x1 − f (x1 )−f (x0 ) , f (x)

Yes it converges:
is |f (x)| ≤ tol?
finish

No

x0 = x1
x1 = x

Figure 6.12: Flowchart of the secant method

Figure 6.12 shows a flow diagram for the secant method. The diagram is basically the same
as the one used for Newton’s method. The two fundamental differences are that now, instead of
evaluating the function and the derivative at each iteration, we calculate the value of the cut-off
point of the line passing through the last two points obtained (i.e. we use a secant line, which cuts
the curve at two points, instead of using a tangent line).
In addition, the value of the last two points obtained must be updated in each iteration: the
oldest point is discarded, the newly obtained point replaces the previous one and this one replaces
the one obtained two iterations before.
Figure 6.13 shows an example of obtaining a root by the secant method. The same function
has been used again as in the previous examples, taking as initial values, x0 = −2.5 and x1 = 0.5.
As in the previous algorithms described, the tolerance has been set to tol = 0.01. In this case, the
algorithm finds the root in five iterations. Each of the graphs that make up the figure 6.13, shows
the obtaining of a new point from the previous two.
In iteration 2, it can be seen how the new point is obtained from two points that are both
located to the right of the root, i.e. they do not form an interval containing the root. Here the
difference with the linear interpolation method is clearly shown. In fact, as already mentioned, the
secant method can be started by taking the first two points on either side of the root.
The method is, in principle, more efficient than bisection and false position, and less efficient
than Newton’s method.
The advantage of this method over Newton’s is that it avoids having to explicitly calculate
the derivative of the function for which the root is to be calculated. In some cases, obtaining the
analytical form of this derivative can be complex.

6.2.5 Method of successive approximations or fixed point iteration

The fixed point iteration is, as will be seen throughout this section, the simplest of all to program.
Unfortunately, it has the problem that we cannot apply it to all functions. There are cases in
which the method does not converge, so it is not possible to use it to find the root or roots of a
function.
184 CHAPTER 6. ROOTS

2 2

x0 x0
0 0
x x x x
2 1 2 1

−2 −2

−4 −4
y

y
−6 −6

−8 −8

−10 −10
−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 −3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1
x x

(a) intervalo inicial (b) iteración 1

2 2

x0 x3 x0 x3
0 0
x x x4 x x
2 1 2 1

−2 −2

−4 −4
y

−6 −6

−8 −8

−10 −10
−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 −3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1
x x

(c) iteracion 2 (d) iteracion 3

2 2

x0 x x x0 x
3 5
3
0 0 x4
x x x1 x2 x1
4 2

−2 −2

−4 −4
y

Raíz

−6 −6

−8 −8

−10 −10
−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 −3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1
x x

(e) iteracion 4 (f) iteracion 5: Raı́z de la función

Figure 6.13: process of obtaining the root of a function by the secant method
6.2. LOCAL ITERATIVE METHODS 185

Fixed point. xf is a fixed point of g(x) if it holds,

g(xf ) = xf

That is, the image of the fixed point xf is again the fixed point. So for example the function,
√
g(x) = − ex

Has a fixed point xf = −0.703467, because g(−0.703467) = −0.703467. The existence of a

fixed point can be obtained graphically, representing in the same graph the function y = g(x) and
the straight line y = x. If there is a point of intersection between both graphs,
√ it is a fixed point.
The figure 6.14, shows graphically the fixed point of the function g(x) = − ex of the previous
example. √
A function can have one or more fixed points or none at all. For example, the function y = ex
does not have any fixed point.

y=−sqrt(e x)
2 y=x
punto fijo

−1

−2

−3

−4

−5
−3 −2 −1 0 1 2 3

√
Figure 6.14: Obtaining the fixed point of the function graphically, g(x) = − ex

Attracting fixed point. SSuppose now that from the function g(x) we create the following
sequence,
xn+1 = g(xn )
That is, we start by taking an initial point x0 and from it we obtain the following values of the
sequence as,
186 CHAPTER 6. ROOTS

x0 → x1 = g(x0 ) → x2 = g(x1 ) = g (g(x0 )) → · · · → xn+1 = g(xn ) = g (g (· · · (g(x0 )))) → · · ·

We can say that fixed point xf of function g(x) is an attracting fixed point if the succession
xn+1 = g(xn ) converges to xf , if x0 is close enough to xf . How close x0 and xf have to be for
the series to converge is a delicate question. For example, the function g(x) = x2 has two fixed
points x = 0 and x = 1. The first is the limit of the sequence xn+1 = g(xn ) for any initial value
x0 contained in the open interval (−1, 1). The point x = 1 is unreachable for any sequence unless
the starting point is itself x0 = xf = 1.
There are some cases where it is possible, for certain functions, to know when one of your fixed
points is attractive,

Fixed point theorem. Let g(x) , be a continuous and derivable function in [a, b], if, ∀x ∈
[a, b] ⇒ g(x) ∈ [a, b], then g(x) has at least a fixed point in [a, b].
If, in addition, there is a positive constant k < 1 and it holds that |g ′ (x)| ≤ k, ∀x ∈ (a, b), then
g(x) has a unique fixed point in [a, b].
To prove the first part of the theorem, Bolzano’s theorem can be used. If it is satisfied that
g(a) = a or g(b) = b, then a or b would be the fixed point. Suppose this is not the case; then it
must be true that g(a) > a and that g(b) < b. Define f (x) = g(x) − x. This function, which is
continuous by construction, satisfies that f (a) = g(a) − a > 0 and f (b) = g(b) − b < 0. Then, there
is a xf ∈ [a, b] that holds f (xf ) = 0 and therefore f (xf ) = g(xf ) − xf = 0 ⇒ g(xf ) = xf . That is,
xf is a fixed point of g(x).
The second part of the theorem can be proved using the mean value theorem. If we assume
that there are two distinct fixed points xf 1 ̸= xf 2 in [a, b], according to the mean value theorem,
there exists a point ξ between xf 1 and xf 2 for which it is satisfied,

g(xf 1 ) − g(xf 2 )
= g ′ (ξ)
xf 1 − xf 2
Therefore,

|g(xf 1 ) − g(xf 2 )| = |xf 1 − xf 2 | · |g ′ (ξ)| ≤ |xf 1 − xf 2 | · k < |xf 1 − xf 2 |

But since we are dealing with fixed points |g(xf 1 ) − g(xf 2 )| = vertxf 1 − xf 2 |, this would lead
to the contradictory result

|xf 1 − xf 2 | = |g(xf 1 ) − g(xf 2 )| ≤ |xf 1 − xf 2 | · k < |xf 1 − xf 2 |

Unless, contrary to the initial hypothesis, it is satisfied that xf 1 = xf 2 . In which case, there
can only be a single fixed point in the interval [a, b] under the conditions imposed by the theorem.

Theorem of the attracting fixed point. 2 Let g(x), continuous and differentiable on an
interval [a, b], that holds, ∀x ∈ [a, b] ⇒ g(x) ∈ [a, b] and that |g ′ (x)| ≤ k, ∀x ∈ (a, b), with
0 < k < 1, then it holds that for any x0 , in [a, b], the succession xn+1 = g(xn ) converges to
the only fixed point of the interval [a, b]. The proof can again be obtained from the mean value
theorem. If we apply it to the initial value x0 and the fixed point xf , we obtain,

|g(x0 ) − g(xf )| = |x0 − xf | · |g ′ (ξ)| ≤ |x0 − xf | · k

2 There are several fixed point theorems defined in different mathematical contexts. A version reduced to functions

f (x) : R → R is given here.

6.2. LOCAL ITERATIVE METHODS 187

h
Initial point x0

Compute
x = g(x0 )

Yes it converges:
is |x − x0 | ≤ tol?
The end

No

x0 = x

Figure 6.15: Flowchart of fixed point iteration. Root corresponds to f (x) = g(x) − x

For the next iteration we will have,

|g(x1 ) − g(xf )| ≤ |x1 − xf | · k ≤ |x0 − xf | · k 2

since x1 = g(x0 ) y xf = g(xf ), because xf is a fixed point.
Using induction we can obtain the nth term of the sequence,

|g(xn ) − g(xf )| ≤ |xn−1 − xf | · k ≤ |xn−2 − xf | · k 2 ≤ · · · ≤ |x0 − xf | · k n

But,

lim k n = 0 ⇒ lim |xn − xf | ≤ lim |x0 − xf |k n = 0

n→∞ n→∞ n→∞

That is, the sequence xn+1 = g(xn ) converges to the fixed point xf .

Fixed point iteration. As we have already seen, obtaining a root of a function f (x), consists
of solving the equation f (x) = 0. Suppose we can decompose the function f (x) as the difference
of two terms, an auxiliary function, g(x), and the variable x itself.

f (x) = g(x) − x

Finding a root of f (x) is then equivalent to finding a fixed point of g(x).

f (x) = 0 → g(x) − x = 0 → g(x) = x

In general, starting from a given function f (x), it is possible to find different functions g(x)
that satisfy that f (x) = g(x) − x. We cannot guarantee that any of the decompositions we do will
generate a function g(x) that has a fixed point. Moreover, for functions that have more than one
root, it can happen that different decompositions of the function converge to different roots. If we
can find one that satisfies the conditions of the fixed point theorem just stated, in an environment
of a root of f (x), we can develop a method that iteratively obtains the values of the sequence
xn+1 = g(xn ), starting from an initial value x0 . The result will approximate the fixed point of
g(x), and therefore the root of f (x) as much as we want. It will suffice, as in the previous methods,
to define a value (tolerance), below which we consider that the value obtained is sufficiently close
to the root to be valid.
188 CHAPTER 6. ROOTS

4
y=−sqrt (e x)
y=x
3 x
y=sqrt(e )

0
y

−1

−2

−3

−4

−5
−3 −2 −1 0 1 2 3
x

√
Figure 6.16: g(x) = ± ex , only negative branch has a fixed point.

Figure 6.15 shows a flowchart of the fixed point iteration.

The idea is to carefully choose the initial point x0 , to ensure that it lies within the interval of
convergence of the fixed point. Then, we calculate the value of g(x0 ), the result will be a new value
x . We check the difference between the obtained point and the previous one and if it is less than
a certain tolerance, we consider that the method has converged, we stop iterating, and we return
the value of x obtained as a result. If not, we copy x into x0 and start the whole process again.
It is interesting to note that the algorithm converges when the difference between two consecutive
points is less than a certain value. According to the fixed point condition g(x0 ) = x0 , this distance
would be equivalent to that between f (x0 ) = g(x0 ) − x0 , the function for which we want to obtain
the root, and 0.
Let us look at an example. Suppose we want to calculate by the fixed point method the root
of the function y = ex − x2 . function y = ex − x2 , which we have used in the examples of the
previous methods.
First, we must obtain from it a new function that satisfies that f (x) = g(x) − x. We can do
this in several ways by clearing an ’x’ from the equation ex − x2 = 0. To illustrate the different
cases of convergence, we will clear x in three different ways.
 √

 x = ± ex

2

ex − x2 = 0 ⇒ x = ln(x ) = 2 · ln(|x|)
x
x = e



x
In our example we have obtained three different ways of clearing the variable x. The question
6.2. LOCAL ITERATIVE METHODS 189

that immediately arises is whether all the functions obtained have a fixed point and, if so, whether
it is possible to reach it iteratively.
√
In the first case, x = ± ex , we obtain the two branches of the square root. For the purposes of
our calculation, each of them constitutes a different function. If we draw them next to the straight
line y = x (figure 6.16), we observe that only the negative branch cuts it. Then it will be this
branch g(x) = −-sqrtex , which we will be able to use to obtain the root of the original function
by the fixed point method. The positive branch, as it does not cut the straight line y = x at any
point, is a function without a fixed point.

3 3

2 2

1 1

x1
0 0

−1 −1
y

y
−2 −2

−3 −3

x0 x0
−4 −4

−5 −5
−4 −3 −2 −1 0 1 2 3 −4 −3 −2 −1 0 1 2 3
x x

(a) startig interval (b) iteration 1

3 3

2 2

1 1

x1 x1
0 0
x3

−1 −1
y

x2 x2

−2 −2

−3 −3

x0 x0
−4 −4

−5 −5
−4 −3 −2 −1 0 1 2 3 −4 −3 −2 −1 0 1 2 3
x x

(c) iteration 2 (d) iteration 3

3 3

2 2

1 1

x1 x1
0 0
x3 x4 x3 x4

−1 −1
y

x2 x2

raíz
−2 −2

−3 −3

x0 x0
−4 −4

−5 −5
−4 −3 −2 −1 0 1 2 3 −4 −3 −2 −1 0 1 2 3
x x

(e) iteration 4 (f) iteration 5: Root of the function

x 2
Figure 6.17:
√ Fixed point iteration to compute the root of f (x) = e − x using the function
g(x) = − e x
190 CHAPTER 6. ROOTS

√
It is not difficult to show that the function g(x) = − ex satisfies the conditions of the fixed point
theorem described above for the interval (−∞, 0]. Then the fixed point iteration should converge for
any starting point x0 contained in that interval. In fact, for this function, the algorithm converges
from any starting point. (If we start at a positive point, the next point, x1 will be negative, and
therefore within the interval of convergence). This function is an example of the theorem providing
a sufficient, but not necessary condition for a fixed point to be attractive.
x 2
Figure 6.17 shows an √ example of the calculation of the root of the function f (x) = e − x using
the function g(x) = − ex , to obtain the fixed point. We have taken as a starting point x0 = 2.5,
a value outside the interval in which the theorem is satisfied. As can be seen in 6.17a. In spite
of this, the algorithm converges quickly, and after 5 iterations, 6.17f, has reached the fixed point
—and therefore the root sought—, with the imposed tolerance.
If we try to use the function g(x) = ln(x2 ) to obtain the root, we observe that the function
does not satisfy the theorem for any interval containing the root.
Figure 6.18 shows the function g(x), the line y = x and the evolution of the algorithm after
four evaluations. It is easy to deduce that the algorithm will jump from the positive branch to the
negative branch and from the negative branch back to the positive branch.
The function has a vertical asymptote at 0. If one starts from x0 = 0, x0 = 1 0 x0 = −1 the
algorithm does not converge, since the function diverges towards −∞. For the rest of the values,
the function oscillates between one branch and another. If in any of the oscillations it manages to
pass close enough to the fixed point, xn − xn− ≤ tol, the algorithm will have approximated the
root, although it cannot properly be said to converge.
Figure 6.19a, shows the evolution of the algorithm, taking x0 = −0.2 as the starting point. After
211 iterations the algorithm ’catches the root’. In this case the tolerance was set to tol = 0.01.

−2
y

−4

−6

−8

−10
−4 −3 −2 −1 0 1 2 3
x

Figure 6.18: first iterations of obtaining the root of the function f (x) = ex − x2 by applying the
fixed point method on the function g(x) = ln(x2 ).

The graph 6.19b shows an enlargement of 6.19a in which the values obtained for the last two
iterations can be seen in detail. The two horizontal dotted lines mark the textroot ± tol.
The iteration stops because the difference between the values obtained in the last two iterations
is lower than the tolerance. The value obtained in the penultimate iteration, which comes from the
6.2. LOCAL ITERATIVE METHODS 191

6
−0.62

4
−0.64

2
−0.66

0
−0.68
Punto fijo
−2
−0.7
y

y
2*tol
−4
−0.72

−6
Raíz
−0.74 obtenida

−8
−0.76

−10
−0.78

−12
−10 −5 0 5 −0.78 −0.76 −0.74 −0.72 −0.7 −0.68 −0.66 −0.64 −0.62 −0.6 −0.58
x x

(a) Evaluation of the Algorithm during 211 (b) Detailed view of the final iterations of
iterations the 6.19a
15

10

5
error

−5

−10
0 20 40 60 80 100 120 140 160 180 200
nº de iteración

(c) Error evaluation

Figure 6.19: Fixed point iteration to compute the root of function f (x) = ex −x2 using the function
g(x) = ln(x2 ), iteration oscillates without converging to the solution.

positive branch of the function g(x), falls very close to the fixed point. The last value obtained, in
fact, moves away from the value of the root, with respect to that obtained in the previous iteration,
but not enough to fall outside the limits of the band marked by the tolerance. As a result, the
termination condition is met and the algorithm stops.
If we decrease the value of the tolerance, we cannot guarantee that the algorithm will converge.
In fact, if we plot what would have been the next values that the solution of the algorithm would
have taken, had it not stopped, it is easy to see that they move further and further away from the
root. Again, we will have to wait for it to change branches and pass close to the fixed point again
for there to be another chance for the algorithm to catch the solution.
The graph 6.19c shows the evolution of the error as a function of the iteration number. As
can be seen, the error oscillates chaotically from one iteration to the next. In fact, the study of
sequences of the form xn+1 = g(xn ) is one of the starting points for the description and analysis
of so-called chaotic systems.
A simple, but very interesting one is that of the discrete logistic equation, xn+1 = R·(1−xn )·xn .
This equation shows very different behaviour, depending on the value of R and the initial value x0
with which we start iterating.
192 CHAPTER 6. ROOTS

4 4

3 3

2 2

1 1

x0 x0
0 0
y

y
x1
−1 −1

−2 −2

−3 −3

−4 −4

−5 −5
−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 −3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2
x x

(a) Starting interval (b) iteration 1

4 4

3 3

2 2

1 1

x x
0 0
0 0
x2 x2
y

x x
−1 1 −1 1
x3

−2 −2

−3 −3

−4 −4

−5 −5
−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 −3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2
x x

(c) iteration 2 (d) iteration 3

4 4

3 3

2 2

1 1

x0 x0
0 0
x2 x4 x2
y

x1 x1
−1 −1
x3 x3

−2 −2
x5
−3 −3

−4 −4

−5 −5
−3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2 −3 −2.5 −2 −1.5 −1 −0.5 0 0.5 1 1.5 2
x x

(e) iteration 4 (f) iteration 5: Root of the function

Figure 6.20: fixed point iteration to compute the root of function f (x) = ex − x2 using the function
x
g(x) = ex , the iteration diverges quickly.

x
Finally, if we use the function g(x) = ex , the fixed point theorem is not satisfied at any point.
In this case, the algorithm always diverges. Figure 6.20 shows the evolution of the fixed-point
iteration for this function. A starting point x0 = −0.745, very close to the value of the root, has
been chosen in order to observe the divergence of the solutions obtained with respect to the fixed
point. As can be seen, the value of xn moves further and further away from the root. The solution
oscillates between a value that gets closer and closer to zero and another that tends towards −∞. If
6.3. PYTHON ROOT FINDING 193

the number of iterations is allowed to increase sufficiently, there will come a time when an overflow
error will occur.
Unlike in the choice of g(x) = ln(x2 ), in this case, the algorithm does not oscillate between the
two branches. If we start on the right-hand branch, choosing a positive value for x0 , the algorithm
diverges, taking the solutions towards +∞. This is to be expected, since this branch has no fixed
point.

6.3 Python root finding

Python has its own methods for calculating roots of functions. Let’s take a look at two of those
methods. First, the fsolve function from the [Link] package calculates roots of all
kinds of functions. Then we will look at the specific functions for using polynomials from the
numpy package, including the roots function for finding the roots of a polynomial.

6.3.1 Function fsolve

fsolve returns the roots of the (non-linear) equations defined by f unc(x) = 0 given a starting
estimate.
If algorithm fails fsolve returns the last iteration result.
The syntax of the fsolve method in its simplest form is as follows:

import [Link] as opt

roots=[Link](fun,[x0])

where fun represents the name of the function to be evaluated, [x0] are the starting values of
the roots and roots are the calculated roots or fsolve.
An example of using fsolve to calculate the root of f (x) = ex − x2 starting from an initial
approximation of x0 = 2 would be as follows:

import numpy as np
import [Link] as opt

def fun(x):
y=[Link](x)-x**2
return y

roots=[Link](fun,-2)

The fsolve method has many possibilities to adjust the precision, the maximum number of
iterations, etc. For a more complete overview of its use, see the [Link] help.

6.3.2 Polynomials roots

Polynomials in NumPy can be created, manipulated, and even fitted using the convenience classes
of the [Link] package, introduced in NumPy 1.4. The Polynomial class provides the
standard Python numerical methods +, −, ∗, //, %, ∗∗ and others.
To create a polynomial we have to import the polynomial package and use the Polynomial class.
Coefficients are given in an array-like form in order of increasing degree, i.e., p1 = [1 4 3 2] give
y = 1 + 4x + 3x2 + 2x3 .
194 CHAPTER 6. ROOTS

from [Link] import Polynomial as P

p = P([1,4,3,2])

Polynomial y = 3x4 + 2x2 + 6x is represented by array, [0 6 2 0 3]. In general, polynomial

y(x) = a0 + a1 x + a2 x2 + · · · + an−1 xn−1 + an xn is represented by array [a0 a1 a2 · · · an−1 an ]. if
the polynomial does not have any of the monomials, the corresponding coefficient is array is 0.

from [Link] import Polynomial as P

p = P([0,6,2,0,3])

Method roots. This method returns the roots of a polynomial defined using the class Polynomial.
The sintax is: [Link]() where p is a polynomial defined as above. Let’s see an example. Given
the polynomial y(x) = x3 − 62 + 11x − 6 we define it and compute the roots in python as:

from [Link] import Polynomial as P

p=P([1,-6,11,-6])
print(p)
print("The roots are:\t",[Link]())

1.0 - 6.0·x + 11.0·x² - 6.0·x³

The roots are: [0.33333333 0.5 1. ]

The method roots returns the polynomial roots in a single vector. If the polynomial has
complex roots it returns its real part as in the case of the polynomial y(x) = x2 + 2x + 1

from [Link] import Polynomial as P

p=P([1,2,1])
print(p)
print("The roots are:\t",[Link]())

1.0 + 2.0·x + 1.0·x²

The roots are: [-1.00000001 -0.99999999]

The method fromroots. . This method could be considered the opposite of the previous one;
given a vector containing the roots of a polynomial, it returns the corresponding polynomial. For
example, if we define the vector of roots, [3, 2, 1] we can obtain the polynomial that has those roots.
In this case the polynomial is y(x) = x3 − 6x2 + 11x − 6.

from [Link] import Polynomial as P

p=[Link]([3,2,1])
print(p)

-6.0 + 11.0·x - 6.0·x² + 1.0·x³

Method linspace. Method linspace returns x, y values at equally spaced points in domain
of polynomial. If domain has been not defined [−1, 1] is set. This method is usefull to plot the
polynomial, as in the next example and figure 6.21.
6.3. PYTHON ROOT FINDING 195

Figure 6.21: Plotting a polynomial using linspace

from [Link] import Polynomial as P

#Polynomial p created and domain defined as [-2,2]
p=P([0,6,2,0,3],[-2,2])
dat=[Link]()
[Link]()
[Link]()
[Link](dat[0],dat[1])
[Link]('x')
[Link]('y')
p_string=str(p)
[Link](p_string)

Method deriv. Returns a polynomial instance of that is the derivative of the current polynomial.

Method integ. Returns a polynomial that is the integral of the current polynomial.

Method copy. Returns a copy of the current polynomial.

from [Link] import Polynomial as P

#Create a polynomial with roots 3, -2, 2

p3=[Link]([3,-2,2])
print(p3)
#Compute the polynomial that is its derivative
pdot=[Link]()
print("Derivative:", pdot)
#Compute the polynomial that is the integral
pint=[Link]()
print("Integral:", pint)
#Create polynomial p4 as a copy of pint
p4=[Link]()
print(p4)
196 CHAPTER 6. ROOTS

12.0 - 4.0·x - 3.0·x² + 1.0·x³

Derivative: -4.0 - 6.0·x + 3.0·x²
Integral: 0.0 + 12.0·x - 2.0·x² - 1.0·x³ + 0.25·x^4
0.0 + 12.0·x - 2.0·x² - 1.0·x³ + 0.25·x^4

6.4 Computing roots at maximum precision.

As we saw in section 5.3 all the numbers we use in a computer are subject to errors because of
the finite size of the registers used to store them. Any number we want to represent has to be
approximated by a machine number. Therefore, instead of looking for the approximation to the
root of a function with a given tolerance, we can find it with the maximum precision allowed by
the numerical representation used. This maximum precision is related to the computer’s epsilon,
since the distance between two consecutive numbers is the epsilon multiplied by two raised to the
exponent of the number. Thus, no matter how many iterations we make, the best approximation
we will be able to obtain of a root will be when the interval in which the root is sought is reduced
to eps and therefore it is not possible to reduce it further.
In Numpy the method spacing(a) returns the distance between a and the next nearest adjacent
number. So if we add a value less than spacing(x) to a number x, the result is the same number.
Note that for negative numbers x < 0 the distance returned by spacing(x) is also negative. There
should not be any representable number between x + spacing(x) and x for any finite x.

import numpy as np

[Link](1)
Out[9]: 2.220446049250313e-16

[Link](1e-10)
Out[10]: 1.2924697071141057e-26

[Link](1e20)
Out[11]: 16384.0

10+[Link](10)/2
Out[12]: 10.0
10==(10+[Link](10)/2)
Out[13]: True

1e20==([Link](1e20)/2)
Out[14]: True
[Link](-5)
Out[15]: -8.881784197001252e-16

-5+[Link](-5)
Out[16]: -5.000000000000001

-[Link](-5)
Out[17]: -4.999999999999999

Since between x and x + spacing(x) there is no machine number, we can use spacing to
establish the stop condition of the root search algorithms. We will stop searching for the root
6.4. COMPUTING ROOTS AT MAXIMUM PRECISION. 197

when the search interval (a, b) is such that there are no machine numbers between a and b, i.e.,
when |b − a| ≤ |[Link](a)|.
For the case of the bisection method the flowchart can be modified as shown in the figure 6.22

Begin with [a, b]

such as
f (a) · f (b) < 0

a+b
Compute c = 2 , f (c)

Yes it converges:
es |a − b| < eps(a)?
Then end

No

Yes b=c
es f (a) · f (c) < 0?
f (b) = f (c)

No
a=c
f (a) = f (c)

Figure 6.22: Flow chart of bisection method with maximal precission

Moreover, in the bisection method, the number of iterations required to achieve maximum
accuracy can be calculated as a function of the starting interval, since at each iteration the interval
is halved. If we call d the starting interval d = |a − b|, at iteration n the length of the interval will
d
be 2n−1 . The minimum possible interval will be reached when its length is that of [Link](a).
But as explained in the section 5.3 the value of eps is equal to 2 raised to the number of bits of
the mantissa. In the case of a double precision number the mantissa has 52 bits and therefore:

|d|
< 2−52
2n−1
And clearing and taking logarithms:

n > log2 (|d|) + 53

For methods that do not start from an interval but from an initial point, such as the Newton-
Raphson or the fixed-point method, the root can also be found using [Link](a), so that
maximum precision will be achieved when the interval between two solutions obtained in consec-
utive iterations is the size of [Link](a).
198 CHAPTER 6. ROOTS
Chapter 7

Linear equation systems

Después de cada guerra

alguien tiene que limpiar.
No se van a ordenar solas las
cosas,
digo yo.

Maria Wislawa Anna Szymborska

7.1 Introduction
A linear equation is an equation that establishes a linear relationship between two or more variables,
for example,

3x1 − 2x2 = 12 (7.1)

It is said to be a linear relationship, because the variables are related to each other only
through additions and products by constant coefficients. In particular, the above example can be
represented geometrically by a straight line.
The number of related variables in a linear equation determines the dimension of the equation.
The above example is a two-dimensional equation, since there are two variables. The number can
be arbitrarily large in general,

a1 x1 + a2 x2 + cdots + an xn = b

will be an n-dimensional equation.

As we have already pointed out above, a two-dimensional equation admits a straight line as geo-
metrical representation, a three-dimensional equation will admit a plane and for dimensions greater
than three each equation will represent a hyperplane of dimension n. Of course, for dimensions
greater than three, it is not possible to obtain a graphical representation of the equation.
Linear equations play a very important role in physics and, in general, in science and technology.
The reason is that they are the simplest mathematical approximation of the relationship between
physical quantities. For example, when we say that the force applied to a spring and the elongation
it undergoes are related by Hooke’s law, F = Kx, we are establishing a linear relationship between
the quantities force and elongation. Does this relationship always hold true? Certainly not. But

199
200 CHAPTER 7. SYSTEMS

it is reasonably true for small elongations and, based on this simple linear model of reality, a lot
of physics can be learned.
A system of linear equations is made up of several linear equations, which express different
linear relationships about the same variables. For example,

a11 x1 + a12 x2 = b1
a21 x1 + a22 x2 = b2

The values of the variables that simultaneously satisfy all the equations that make up the
system are called solutions of the system of equations. From the point of view of obtaining the
solutions, the variables are usually called unknowns, i.e. unknown values that we wish to obtain
or calculate.
A system of equations can have infinite solutions, it can have only one solution or it can have
no solution. In what follows, we will focus on systems of equations that have only one solution.
A first condition for a system of equations to have a unique solution is that the number of
unknowns present in the system coincides with the number of equations.
In general terms, we can say that we are going to study numerical methods to solve with a
computer systems of n equations with n unknowns,

a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2
···
an1 x1 + an2 x2 + · · · + ann xn = bn

One of the great advantages of systems of linear equations is that they can be expressed in
matrix product form,
A x b
z }| { z }| { z }| {
a11 x1 + a12 x2 + · · · + a1n xn = b1 
 
 a11 a12 ··· a1n x1 b1
 a21 a22 ··· a2n 
a21 x1 + a22 x2 + · · · + a2n xn = b2

   x 2   b2 
⇒ . .. ..  ·  ..  =  .. 
    
 .. ..
··· 
 . . .   .  .

an1 x1 + an2 x2 + · · · + ann xn = bn an1 an2 ··· ann xn bn


The matrix A is called the coefficient matrix of the system of equations, the vector x is the
vector of unknowns and the vector b is the vector of independent terms. To solve a system of
equations we could apply matrix algebra, as discussed in chapter 3:

A · x = b ⇒ x = A−1 · b
That is, it would be enough to invert the matrix of coefficients and multiply it on the left by
the vector of coefficients to obtain the vector of independent terms. From this we can deduce a
second condition for a system of equations to have a unique solution; its coefficient matrix must
have an inverse. Let us look at some simple examples.
We will first take a system of two equations with two unknowns,

4x1 + x2 = 6
3x1 − 2x2 = −1
7.1. INTRODUCTION 201

If we express the system as a product of matrices we obtain,

     
4 1 x1 6
· =
3 −2 x2 −1
and by inverting the coefficient matrix and multiplying it by the vector of independent terms
we arrive at the vector of solutions of the system,
   −1       
x1 4 1 6 2/11 1/11 6 1
= · = =
x2 3 −2 −1 3/11 −4/11 −1 2
In the example we have just seen, each equation corresponds to a line in the plane, in the figure
7.1 these lines have been represented graphically. The point where they intersect is precisely the
solution of the system.

10
3x1−2x2=−1
4x1+x2=6
8
(x1,x2)=(1,2)

4
2

2
x

−2

−4

−6
−1 −0.5 0 0.5 1 1.5 2 2.5 3
x1

Figure 7.1: Equation system with unique solution

Now suppose the following system, also of two equations with two unknowns,
4x1 + x2 = 6
1
2x1 + x2 = −1
2
202 CHAPTER 7. SYSTEMS

The system has no solution. Its coefficient matrix has zero determinant, so it is not invertible,

4 1
|A| = = 0 ⇒ ∄A−1
2 1/2

If we represent graphically the two equations of this system (figure 7.2) it is easy to understand
what happens, the lines are parallel, there is no point (x1 , x2 ) that belongs to the two lines, and
therefore the system has no solution.

10

4x1+x2=6
2x1+0.5x2= −1
5

0
x2

−5

−10

−15
−1 −0.5 0 0.5 1 1.5 2 2.5 3
x1

Figure 7.2: Equation system without solution

Two parallel lines are parallel because they have the same slope. This is reflected in the
coefficient matrix, where the rows are proportional; if we multiply the second row by two, we get
the first row.
Finally, the system,

4x1 + x2 = 6
1
2x1 + x2 = 3
2
7.1. INTRODUCTION 203

has infinite solutions. The reason is that the second equation is the same as the first equation
multiplied by two: that is, it represents exactly the same linear relationship between the variables
x1 and x2 , therefore, all the points on the line are solutions of the system. Again, the coefficient
matrix of the system has no inverse since its determinant is zero.

10
4x +x =6
1 2
2x1+0.5x2=3
8

4
2

2
x

−2

−4

−6
−1 −0.5 0 0.5 1 1.5 2 2.5 3
x1

Figure 7.3: Equation system with infinite solutions

For higher dimensional systems of equations, it is also true that the system has no unique
solution if the determinant of its coefficient matrix is zero. In all other cases, it is possible to
obtain the solution of the system by inverting the coefficient matrix and multiplying the result by
the vector of independent terms.
As soon as a system of equations has a sufficiently large dimension, inverting its coefficient
matrix becomes a costly or simply intractable problem.
From a numerical point of view, the inversion of a matrix often presents problems due to the
rounding error in the operations. For this reason, systems of equations are almost never solved by
inverting their matrix of coefficients. Throughout this chapter we will study two types of methods
for solving systems of equations. The first of these is known generically as direct methods, while
the second type is known as iterative methods.
204 CHAPTER 7. SYSTEMS

7.2 Condition number

In the introduction we have seen that for a system of equations to have a solution, its coefficient
matrix must be invertible. However, when we try to solve a system of equations numerically, using
a computer, we must first carefully examine the matrix of coefficients of the system. Let us look
at an example: the system,

4x1 + x2 = 6
2x1 + 0.4x2 = −1

Has the following solutions,

 
−8.5
x=
40

If we slightly change one of the coefficients,

4x1 + x2 = 6
2x1 + 0.49x2 = −1

The solutions become quite altered; they become about 10 times larger,
 
−98.5
x=
400

if we change again the same coefficient,

4x1 + x2 = 6
2x1 + 0.499x2 = −1

The solution is about 100 times larger,

 
−998.5
x=
4000

The reason for these changes is easy to understand intuitively; as we bring the coefficient closer
to 0.5, we are making the two linear equations more and more parallel, and small variations in the
slope greatly modify the position of the cut-off point.
When small variations in the matrix of coefficients generate large variations in the solutions
of the system, it is said that the system is ill-conditioned, in other words: that it is not a good
system to be solved numerically. The solutions obtained for an ill-conditioned system must always
be taken with considerable scepticism.
To estimate the good or bad conditioning of a system, we use the condition number, which we
defined in the chapter 3 in section 3.5.5, when talking about SVD factorisation. The condition
number of a matrix is the quotient of its largest and smallest singular values. The closer the
condition number is to 1, the better conditioned the matrix is, and the larger the condition number,
the worse conditioned it is.
In Python, we can use the function cond inside the linalg module in numpy to compute the
condition number of a matrix. We can apply to the last example coefficient matrix
7.2. CONDITION NUMBER 205

import numpy as np

A=[Link]([[4,1],[2,0.499]])

[Link](A)
Out[7]: 5312.250061755533

The number is quite far from one, which, in principle, indicates a ill-condition of the system.
Incidentally, we can calculate the factorisation svd of the coefficient matrix and divide the
largest singular value by the smallest to check that the result is the same as that given by the
function cond,

[U,S,Vt]=[Link](A)

S[0]/S[1]
Out[14]: 5312.250061755533

7.2.1 Direct methods

7.2.2 Triangular systems
We are going to begin the study of direct methods with the algorithms for solving the simplest
possible systems, those whose coefficient matrix is a diagonal, upper triangular or lower triangular
matrix.

Diagonal systems. A diagonal system is one whose coefficient matrix is a diagonal matrix.

A x b
z }| { z }| { z }| {
a11 x1 = b1 

 a11 0 ··· 0 x1 b1
a22 x2 = b2 
  0 a22 ··· 0   x 2   b2 
⇒ . .. ..  ·  ..  =  .. 
     
 .. ..
··· 
 . . .   .  .

ann xn = bn 0 0 ··· ann xn bn


Its resolution is trivial, just divide each independent term by the corresponding element of the
diagonal of the coefficient matrix,
bi
xi =
aii
To compute the solution it is enough to code a simple for loop,

import numpy as np

def diag_sys(A,b):
[f,c]=[Link](A)
x=[Link]([f,1])
for i in range(f):
x[i]=b[i]/A[i,i]
return x
206 CHAPTER 7. SYSTEMS

Lower triangular systems: method of progressive substitutions. A lower triangular sys-

tem of n equations with n unknowns will have the general form,
A x b
z }| { z }| { z }| {
a11 x1 = b1


 a11 0 ··· 0 x1 b1
 a21 a22 ··· 0 
a21 x1 + a22 x2 = b2

   x 2   b2 
⇒ . .. ..  ·  ..  =  .. 
    
 .. ..
··· 
 . . .   .  .

an1 x1 + an2 x2 + · · · + ann xn = bn an1 an2 ··· ann xn bn


The procedure to solve to hand is very simple, we clear the first unknown from the first equation,
b1
x1 =
a11
Next, we substitute this result in the second equation and clear x2
b2 − a21 x1
x2 =
a22
From each equation we obtain a component of the solution vector, substituting the solutions
obtained in the previous equations, so when we arrive at the equation i,
Pi−1
bi − j=1 aij xj
xi =
aii
If we repeat this same process until we reach the last equation of the system, n, we will have
obtained the complete solution.
The following code calculates the solution of a lower triangular system by progressive substi-
tutions,
import numpy as np

def progressive(A,b):
''' This function computes the solution of a lower equation system using
progressive substitution. It receives the coefficient matriz A and the
independent term vector b. It returns the vector solution x
'''
# Coefficient matrix size. Return error if not square
[f,c]=[Link](A)
if f!=c:
print("A is not square")
return
# To build the solution vector x
x=[Link](f)
x=[Link]()
for i in range(f):
'''The inner block subtracts to the independent term the previous solution
multiplied by the correspondent coefficient'''
for j in range(0,i):
x[i]-=x[j]*A[i,j]
'''Finaly divide the result by the diagonal coefficient'''
x[i]=(x[i])/A[i,i]
return x
7.2. CONDITION NUMBER 207

Upper triangular systems: method of backward substitutions. In this case, the general
system of n equations with n unknowns will have the general form,
A x b
z }| { z }| { z }| {
a11 x1 + a12 x2 + · · · + a1n xn = b1 

 a11 a12 ··· a1n x1 b1
 0 a22 ··· a2n 
a22 x2 + · · · + a2n xn = b2

   x 2   b2 
⇒ . .. ..  ·  ..  =  .. 
    
 .. ..
· · ·
 . . .   .  .

ann xn = bn 0 0 ··· ann xn bn


The method of solution is identical to that of a lower triangular system, except that now, we
start solving for the last equation,
bn
xn =
ann
And we keep substituting upwards,
bn−1 − a(n−1)n xn
xn−1 =
a(n−1)(n−1)
Pn
bi − j=i+1 aij xj
xi =
aii
The code to implement this method is similar to that of forward substitutions. It is left as an
exercise to build a Python function that calculates the solution of an upper triangular system by
the method of backward substitutions.

7.2.3 Methods based on factorisations

Since we know how to solve triangular systems, one way to solve more complex systems would be
to find methods to reduce them to triangular systems. In this way we avoid inverting the matrix
of coefficients and the possible problems of numerical stability derived from such an operation. If
we can express the matrix A as the product of diagonal or triangular matrices we can solve the
system using the above methods.
To factorise a matrix is to decompose it into the product of two or more matrices of some
canonical form.

LU factorisation . LU factorisation consists of factoring a matrix into the product of two, a

lower triangular L and an upper triangular U . The factorisation could include pivoting rows, to
achieve a numerically stable solution. In this case the LU factorisation takes the form,

PT · A = L · U

Where P represents a matrix of permutations. We can also look at it in another way, A =

P · L · U , since P is a permutation matrix and P −1 = P T .
Suppose we want to solve a system of n linear equations with n unknowns that we represent
generically in matrix form, as usual,
A·x=b
If we compute the LU factorisation of the coefficient matrix,

A→A=P ·L·U
We can transform our system of equations into an equivalent replacing A by its lu factorisation,
208 CHAPTER 7. SYSTEMS

A·x=b→P ·L·U ·x=b

In order to solve the system using progressive and reversible substitutions we will multiply both
terms of the equation by P −1 , remembering that P −1 = P T .

P −1 · P · L · U · x = P −1 · b → L · U · x = P T · b

The new system can be solved in two steps using backward and forward substitutions. To do
this, we associate the product U · x, to a vector of auxiliary unknowns which we will call z,
If we substitute our auxiliary vector z into the matrix expression of our system of equations,
z
z }| {
L · U · x = PT · b → L · z = PT · b
The resulting system is lower triangular, so we can solve it by forward substitutions, and thus
obtain the values of z. We can finally obtain the solution of the system through the definition of
z; U · x = z, it is an upper triangular system, which we can solve by backward substitutions.
Let us look at an example. Suppose we want to solve the system of linear equations,
     
1 3 2 x1 13
2 −1 1 · x2  =  3 
1 4 3 x3 18
First we should check that the coefficient matrix is well conditioned,

import numpy as np

A=[Link]([[1, 3, 2],[2, -1, 1],[1, 4, 3]])

cond=[Link](A)

print(cond)
24.382675394986972

It is not a large value, so we can consider that A is well conditioned. We compute the LU
factorisation of the coefficient matrix, for this we can use the lu method of the scipy linear algebra
package. This method returns the matrices P , L and U that satisfy that A = P · L · U

import numpy as np
import scipy as sc

A=[Link]([[1, 3, 2],[2, -1, 1],[1, 4, 3]])

b=[Link]([[13],[3],[18]])
cond=[Link](A)
P,L,U=[Link](A)

L: [[1. 0. 0. ]
[0.5 1. 0. ]
[0.5 0.77777778 1. ]]
U: [[ 2. -1. 1. ]
[ 0. 4.5 2.5 ]
7.2. CONDITION NUMBER 209

[ 0. 0. -0.44444444]]
P: [[0. 0. 1.]
[1. 0. 0.]
[0. 1. 0.]]
Next we must apply the transpose matrix of permutations to the vector of independent terms
of the system, in order to construct the equivalent system L · U · x = P T · b,
Pb=[Link](P).dot(b)
print("Pb: ",Pb)
Pb: [[ 3.]
[18.]
[13.]]
We use the matrix L obtained and the product bp = P T · b that we have just calculated, to
obtain, by progressive substitutions, the auxiliary vector z described above. To do this we use the
function progressive, whose code is included in the previous section,
z=progressive(L, bP)
print("z:",z)
z: [[ 3. ]
[16.5 ]
[-1.33333333]]
Finally, we can obtain the solution of the system by simply applying the method of regressive
substitutions to the matrix U and the auxiliary vector z that we have just obtained,1
x=regressive(U, z)
print("x:",x)
x: [[1.]
[2.]
[3.]]
To check that the solution is correct, it is enough to multiply the matrix of coefficients of the
original system by the result obtained for x and check that we obtain as a result the vector of
independent terms.
print([Link](x))
[[13.]
[ 3.]
[18.]]

∞ (
Ξi (t)
i=0

When solving a system of equations by factoring, we are only operating on the matrix of
coefficients, so that if we have to solve multiple times systems with the same matrix in which the
vector of independent terms varies, the calculations are minimal. These methods are very efficient.
In addition to factoring, there are other ways of factoring matrices that are useful for solving
systems of equations.
1 The function regressive has not been supplied. Its construction has been left as an exercise in the previous

section.
210 CHAPTER 7. SYSTEMS

Cholesky factorisation . The Cholesky factorisation allows to decompose a matrix A into the
product of a lower triangular matrix, by its transpose.

A = L · LT

For this, A must be symmetric and positive definite. A Matrix An×n is positive definite if given a
non-zero vector x it satisfies,
xT dotA · x > 0, ∀x ̸= 0

Therefore, in the particular case of a system whose coefficient matrix is symmetric and positive
definite, we could decompose it using the Cholesky factorisation and solve the system in a similar
way as we did with the LU factorisation, substituting A for the product LT ,

A · x = b → L · LT · x = b

We define the auxiliary vector z,

LT · x = z

We solve the system by progressive substitutions,

LT · z = b

and finally we obtain x by solving the system by backward substitutions,

LT · x = z

Next python code sequence shows the resolution of the system

     
2 5 1 x1 15
5 14 2 · x2  = 39
1 2 6 x3 23
To do so, we will use the cholesky function from the linalg package of scipy. By default,
this function returns the upper triangular matrix of the cholesky decomposition A = U T U . In
order to return the decomposition using the lower triangular one, A = L · LT , we will have to set
the parameter lower=True.
If we want to use directly the function cholesky without indicating lower=True, we will simply
have to take into account that LT = U and L = U T . Therefore, if the command cholesky is used
directly: A · x = b → U T · U T · x = b.

A=[Link]([[2., 5., 1.],[5., 14., 2.],[1., 2., 6.]])

b=[Link]([[15.],[39.],[23.]])
L=[Link](A,lower=True)
print("L: ",L)
z=progressive(L, b)
print("z: ",z)
LT=[Link](L)
print("LT: ",LT)
x=regressive(LT,z)
print("x: ", x)
print([Link](x))
7.2. CONDITION NUMBER 211

L: [[ 1.41421356 0. 0. ]
[ 3.53553391 1.22474487 0. ]
[ 0.70710678 -0.40824829 2.30940108]]
z: [[10.60660172]
[ 1.22474487]
[ 6.92820323]]
LT: [[ 1.41421356 3.53553391 0.70710678]
[ 0. 1.22474487 -0.40824829]
[ 0. 0. 2.30940108]]
x: [[1.]
[2.]
[3.]]
[[15.]
[39.]
[23.]]

QR factorisation . The QR factorisation decomposes a matrix into the product of an orthogonal

matrix Q by an upper triangular matrix R. A matrix Antimesn is orthogonal when its inverse
coincides with its transpose.
AT = A−1

If we obtain the QR factorisation of the coefficient matrix of a system,

A·x=b→Q·R·x=b

We can now solve the system in two steps. First, since Q is orthogonal, Q−1 = QT , we can
multiply by QT on both sides of the equality,

Q · R · x = b → QT · Q · R · x = QT · b → R · x = QT · b

But the resulting system is an upper triangular system, so we can solve it by backward substi-
tutions. Taking the same example we solved before by LU factorisation,
     
1 3 2 x1 13
2 −1 1 · x2  =  3 
1 4 3 x3 18

we can now solve it by QR factorisation. To do so, we apply the qr function included in the
linalg package of scipy, to the coefficient matrix of the system. Next we multiply QT by the
independent term vector and solve the R · x = QT · b using the regressive function.

# QR factorising
A=[Link]([[1, 3, 2],[2, -1, 1],[1, 4, 3]])
b=[Link]([[13],[3],[18]])
Q,R=[Link](A)
print("Q: ",Q)
print("R: ", R)
Qb=[Link](Q).dot(b)
x=regressive(R, Qb)
print("x: ",x)
212 CHAPTER 7. SYSTEMS

Q: [[-0.40824829 0.46369464 -0.78633365]

[-0.81649658 -0.5707011 0.08737041]
[-0.40824829 0.67770756 0.61159284]]
R: [[-2.44948974 -2.04124145 -2.85773803]
[ 0. 4.67261526 2.38981086]
[ 0. 0. 0.34948162]]
x: [[1.]
[2.]
[3.]]

SVD factorisation . The singular value decomposition (svd), decomposes any matrix into the
product of three matrices,
A = U = U sec : SV D, S = V T (7.2)
Where U and V are orthogonal matrices and S is the singular value diagonal matrix. If we calculate
the factorisation svd of the coefficient matrix of a system,

A=x=b→U =S =VT =x=b (7.3)

As in the case of QR factorisation, we can take advantage of the orthogonality of the U and V
matrices to simplify the system,

U · S · V T · x = b → UT · U · S · V T · x = UT · b → S · V T · x = UT · b
As in previous cases, we can create an auxiliary vector z,

V T dotx = z

so we solve the system first,

ST · z = U T · b
Since the matrix S is diagonal, this is a diagonal system which, as we have seen, is trivial to
solve.
Once z is known, we can obtain the solution of the original system, making use of the orthog-
onality of the matrix V ,

VT ·x=z →V ·VT ·x=V ·z →x=V ·z

Back to our example,
     
1 3 2 x1 13
2 −1 1 · x2  =  3 
1 4 3 x3 18
First we find the singular value decomposition (svd) of the coefficient matrix of the system
using the function svd from the linalg package of scipy. This function returns the orthogonal
matrices U and V T and a row vector with the eigenvalues of A ordered from largest to smallest.
The next step will be to construct the matrix S with the eigenvalues, for this we can use the
function diag of numpy: S=[Link](s) and construct the vector U T · b.
With the matrices already constructed, we solve the diagonal system S T · z = U T · b with
the function diag sys. Once z is known we calculate x knowing that the matrix V is orthogonal
(x = V · z).
7.2. CONDITION NUMBER 213

#SVD decomposition
A=[Link]([[1, 3, 2],[2, -1, 1],[1, 4, 3]])
b=[Link]([[13],[3],[18]])
U,s,V=[Link](A)
print("U: ",U)
print("V: ",V)
print("s: ",s)
S=[Link](s)
print("S: ",S)
Utb=[Link](U).dot(b)
z=diag_sys(S, Utb)
print("z: ",z)
x=[Link](V).dot(z)
print("x:",x)

U: [[-0.59080923 -0.00525033 -0.80679421]

[-0.04109702 -0.99848485 0.03659281]
[-0.80576392 0.0547762 0.58969829]]
V: [[-0.23386523 -0.78353016 -0.5756627 ]
[-0.79834737 0.49268931 -0.34626396]
[-0.55493111 -0.3785997 0.74075213]]
s: [6.32315973 2.43934396 0.25933002]
S: [[6.32315973 0. 0. ]
[0. 2.43934396 0. ]
[0. 0. 0.25933002]]
z: [[-3.52791365]
[-0.85176062]
[ 0.91012588]]
x: [[1.]
[2.]
[3.]]

7.2.4 The Gaussian elimination method.

The Gaussian elimination method consists of making all the elements below the diagonal of a
matrix zero. This is done by progressively replacing the rows of the matrix, except for the first
row, with suitable combinations of that row with the previous rows.
Let’s see what it consists of with an example. Let’s suppose we have the following matrix of
order 4 × 4,
 
3 4 2 5
2 0 1 −2
A= 3 2 1 8 


5 2 3 2
214 CHAPTER 7. SYSTEMS

If we replace the second row by the result of subtracting the first row multiplied by 2 and
divided by 3 we obtain the matrix,
   
3 4 2 5 3 4 2 5
2 0 1 −2 2 0 −2.6 −0.33 −5.33
A=
3 2 1
 → [2 0 1 − 2] − [3 4 2 5] → U1 =  
8 3 3 2 1 8 
5 2 3 2 5 2 3 2
Similarly, if we now replace the third row by the result of subtracting the first row multiplied
by 3 and dividing 3,
   
3 4 2 5 3 4 2 5
0 −2.6 −0.33 −5.33 3 0 −2.6 −0.33 −5.33
U1 = 
3
 → [3 2 1 8] − [3 4 2 5] → U1 =  
2 1 8  3 0 −2 −1 3 
5 2 3 2 5 2 3 2
Finally, if we replace the last row by the result of subtracting the first row multiplied by 5 and
divided by 3,
   
3 4 2 5 3 4 2 5
0 −2.6 −0.33 −5.33 5 0 −2.6 −0.33 −5.33
U1 = 
0
 → [5 2 3 2] − [3 4 2 5] → U1 =  
−2 −1 −3  3 0 −2 −1 −3 
5 2 3 2 0 −4, 66 −0.33 −6.33
The result we have obtained, after performing this transformation, is a new matrix U in which
all the elements of its first column, below the diagonal, are zeros.
We can proceed in a similar way to eliminate now the elements of the second column below
the diagonal. For them we replace the third row by the difference between it and the second row
multiplied by −2 and divided by −2.6.
 
3 4 2 5
 → [0 − 2 − 1 − 3] − −2 [0 − 2.6 − 0.33 − 5.33 ] →
0 −2.6 −0.33 −5.33
U1 = 

0 −2 −1 −3  −2.6
0 −4, 66 −0.33 −6.33
 
3 4 2 5
0 −2.6 −0.33 −5.33
U2 = 
 
0 0 −0.75 7 
0 −4.66 −0.33 −6.33
And substituting the last row by the difference between it and the second row multiplied by
−4.66 and divided by −2.6,
 
3 4 2 5
 → [0 − 4.66 − 0.33 − 6.33] − −4.66 [0 − 2.6 − 0.33 − 5.33 ] →
0 −2.6 −0.33 −5.33
U2 = 
0 −2 −1 −3  −2.6
0 −4, 66 −0.33 −6.33
 
3 4 2 5
0 −2.6 −0.33 −5.33
U2 = 
0

0 −0.75 7 
0 0 0.25 3
7.2. CONDITION NUMBER 215

In this way, the elements of the second column below the diagonal have been replaced by zeros.
A final step will lead us to an upper triangular matrix; we replace the last row by the difference
between it and the third row multiplied by 0.25 and divided by −0.75,

 
3 4 2 5
 → [0 0 0.25 3] − 0.25 [0 0 − 0.75 7 ] →
0 −2.6 −0.33 −5.33
U2 = 

0 0 −0.75 7  −0.75
0 0 0.25 3
 
3 4 2 5
0 −2.6 −0.33 −5.33
U3 = 
 =U
0 0 −0.75 7 
0 0 0 5.33
We can now, from the example, deduce a general procedure. To eliminate —convert to 0— the
element aij below the main diagonal, i > j:
1. Divide the row j elements by the diagonal element of the same row, ajj , eq. 7.4:
2. Multiply the previous result by the element aij , eq. 7.5
3. Finally, substitute initial matrix row i by the difference between it ant the previous result,
eq. 7.6.

 
0/ajj 0/ajj ··· ajj /ajj ajj+1 /ajj ··· (7.4)

 
aij · 0/ajj aij · 0/ajj ··· aij · ajj /ajj aij · ajj+1 /ajj ··· (7.5)

   
0 0 ··· aij aij+1 · · · − aij · 0/ajj aij · 0/ajj ··· aij · ajj /ajj
··· aij · ajj+1 /ajj
(7.6)
This procedure is applied iteratively starting at the a21 element of the matrix and moving the
computation downwards until the last row is reached and rightwards until the element before the
diagonal is reached in each row.
The following code applies the procedure described above to a matrix of any order,

import numpy as np
def eligauss(A):
'''This function obtains an upper triangular matrix, starting from a given
matrix, by applying the Gaussian elimination method.
It does not perform row piboting'''

#Matrix shape
[f,c]=[Link](A)
U=[Link]()
#For all the columns in A (except the last one)
for i in range(c-1):
# For all the rows below the diagonal
for j in range(i+1,f):
U[j,:]=U[j,:]-U[i,:]*U[j,i]/U[i,i]
return U
216 CHAPTER 7. SYSTEMS

The basic idea is to take advantage of the following properties of any system of linear equations;
1. A system of linear equations does not change even if the order of its equations is altered.
2. A system of linear equations does not change even if any of its equations is multiplied by a
non-zero constant.
3. A system of equations does not change if you replace any of its equations by a linear combi-
nation of it with another equation.
If we use the matrix representation of a system of equations, any of the changes described in
the above properties affect both the matrix of coefficients and the vector of independent terms,
e.g. given the system,
     
1 3 2 x1 13
2 −1 1 · x2  =  3 
1 4 3 x3 18
If I change the order of the second equation with the first equation I obtain the following
equivalent system,
     
2 −1 1 x1 3
1 3 2 · x2  = 13
1 4 3 x3 18
That is, the first row of the coefficient matrix is exchanged with the second, and the first element
of the vector of independent terms with the second. The vector of unknowns remains unchanged.
If we now substitute the second row, by the difference between it and the first row multiplied
by 0.5, we obtain again an equivalent system,
     
2 −1 1 x1 3
0 3.5 1.5 · x2  = 11.5
1 4 3 x3 18
We have just taken the first two steps in the Gaussian elimination process to convert the
coefficient matrix of the system into an upper triangular matrix: we have ’pivoted’ the first two
rows and then transformed the first element of the second row to zero, combining it with the
first row. We have also applied this same combination to the second element of the vector of
independent terms, so that the system obtained is equivalent to the original.
In order to work comfortably with the Gaussian elimination method, a matrix is usually con-
structed, known as an extended matrix, by adding the vector of independent terms to the coefficient
matrix as an additional column,

A, b → AM = (A|b)
In our example,
     
1 3 2 13 1 3 2 13
2 −1 1 ,  3  → 2 −1 1 3
1 4 3 18 1 4 3 18
We can apply directly to the extended matrix AM the Gaussian elimination program, eligauss.
If we apply eligauss to the extended matrix of the system of the example. To construct the
extended matrix we can use the concatenate method of numpy indicating that the axis on which
we want to concatenate A and b is 1 (columns):
AM=[Link]((A,b),axis=1
7.2. CONDITION NUMBER 217

A=[Link]([[1., 3., 2.],[2., -1., 1.],[1., 4., 3.]])

b=[Link]([[13.],[3.],[18.]])
# Extended matrix
AM=[Link]((A,b),axis=1)
print("AM: ",AM)
GA=eligauss(AM)
print("GA: ",GA)

AM: [[ 1. 3. 2. 13.]
[ 2. -1. 1. 3.]
[ 1. 4. 3. 18.]]
GA: [[ 1. 3. 2. 13. ]
[ 0. -7. -3. -23. ]
[ 0. 0. 0.57142857 1.71428571]]

The program has obtained as a result a new matrix in which the elements below the diagonal are
now zero. We can reconstruct, from the result obtained, a current equivalent system by separating
the last column of the result,
       
1 3 2 13 1 3 2 x1 13
0 −7 −3 −23  → 0 −7 −3  · x2  =  −23 
0 0 0.5714 1.7143 0 0 0.571 x3 1.7143
The system resulting from Gaussian elimination is upper triangular, so we can solve it directly
by backward substitutions,

RA=GA[:,0:3]
nb=GA[:,3]
print("RA: ",RA)
print("nb: ",nb)
x=regressive(RA, nb)
print("x: ",x)

RA: [[ 1. 3. 2. ]
[ 0. -7. -3. ]
[ 0. 0. 0.57142857]]
nb: [ 13. -23. 1.71428571]
x: [1. 2. 3.]

The eligauss program has a problem: what happens if the diagonal element is zero or a
relatively small value? If that happens, it is possible that the elimination process cannot be
completed or that the calculations are unstable. For this we will include row pivoting, i.e. we
will rearrange the rows of the matrix so that the element on the diagonal is larger than the same
element in the rows after it. Below, we include a modified version of eligauss that includes row
pivoting.

def eligaussp(A):
'''This function obtains an upper triangular matrix, starting from a given
matrix, by applying the Gaussian elimination method.
It includes row piboting. If the diagonal element is less than the same
element in next rows, rows are interchanged'''
218 CHAPTER 7. SYSTEMS

#Matrix shape
[f,c]=[Link](A)
U=[Link]()
#For all the columns in A (except the last one)
for i in range(c-1):
#Row pivoting
# Search the maximun in column i
maxcol= [Link](U[i,i])
index = i
for l in range(i,f):
if [Link](U[l,i])>maxcol:
maxcol=[Link](U[l,i])
index=l
# If we have found an element U[l,i] greater than U[i,i] we interchange
# row l with row i
if index!=i:
aux=[Link]([U[i,:]])
U[i,:]=U[index,:]
U[index,:]=aux[:]
# End of Row pivoting
# For all the rows below the diagonal
# print(U)
for j in range(i+1,f):
U[j,:]=U[j,:]-U[i,:]*U[j,i]/U[i,i]
return U

We can apply this function to our usual example and by separating the coefficient matrix and
the vector of independent terms in the extended matrix we can solve the system by backward
substitutions,
A=[Link]([[1., 3., 2.],[2., -1., 1.],[1., 4., 3.]])
b=[Link]([[13.],[3.],[18.]])
# Extended matrix
AM=[Link]((A,b),axis=1)
print("AM: ",AM)
GA=eligaussp(AM)
print("GA: ",GA)
RA=GA[:,0:3]
nb=GA[:,3]
print("RA: ",RA)
print("nb: ",nb)
x=regressive(RA, nb)
print("x: ",x)

AM: [[ 1. 3. 2. 13.]
[ 2. -1. 1. 3.]
[ 1. 4. 3. 18.]]
[[ 2. -1. 1. 3.]
[ 1. 3. 2. 13.]
[ 1. 4. 3. 18.]]
7.2. CONDITION NUMBER 219

[[ 2. -1. 1. 3. ]
[ 0. 4.5 2.5 16.5]
[ 0. 3.5 1.5 11.5]]
[[ 2. -1. 1. 3. ]
[ 0. 4.5 2.5 16.5 ]
[ 0. 0. -0.44444444 -1.33333333]]
GA: [[ 2. -1. 1. 3. ]
[ 0. 4.5 2.5 16.5 ]
[ 0. 0. -0.44444444 -1.33333333]]
RA: [[ 2. -1. 1. ]
[ 0. 4.5 2.5 ]
[ 0. 0. -0.44444444]]
nb: [ 3. 16.5 -1.33333333]
x: [1. 2. 3.]

7.2.5 Gauss-Jordan and matrices in stepwise reduced form

. The Gaussian elimination method allows us to obtain an upper triangular matrix from an
arbitrary matrix. Once this has been obtained, we could continue transforming the matrix in such
a way as to make all the elements located above its diagonal zero. To do this, we would simply
apply the same Gaussian elimination method, the difference being that we would now eliminate -
make zeros - the elements located above the main diagonal of the matrix, starting with the last
column and moving from bottom to top and from right to left. This process is known as Gauss-
Jordan elimination. For example, suppose we start from the matrix we have just obtained by
Gaussian elimination in the previous example,
 
1 4 3 18
GA = 0 −1 −1 −5
0 0 4 12

We would start by making the element ga23 , which is the one located above the last element
of the main diagonal, zero. To do this we would subtract the third column multiplied by −1 and
divided by 4 from the second column,
   
1 4 3 18 1 4 3 18
0 − 0 −1 − 0 −1 + 4/4 −5 + 12/4 = 0 −1 0 −2
0 0 4 12 0 0 4 12
Next, we would eliminate the element located in the same column, one row above. To do this
we subtract from the first the third multiplied by 3 and divided by 4,
   
1 − 0 4 − 0 3 − 3 · 4/4 18 − 12 · 3/4 1 4 0 9
 0 −1 0 −2  = 0 −1 0 −2
0 0 4 12 0 0 4 12
As we have reached the first column, we move on to remove the elements in the next column on
the left. In this example we only have one element above the diagonal. To eliminate it we subtract
from the first row the second row multiplied by 4 and divided by −1,
   
1 − 0 4 − 4 · (−1)/(−1) 0 9 − (−2) · 4/(−1) 1 0 0 1
 0 −1 0 −2  = 0 −1 0 −2
0 0 4 12 0 0 4 12
220 CHAPTER 7. SYSTEMS

If we now separate the coefficient matrix and the vector of independent terms into the resulting
expanded matrix, we obtain a diagonal system whose solution is trivial,
         
1 0 0 1 1 0 0 x1 1 1
0 −1 0 −2 → 0 −1 0 · x2  = −2 ⇒ x = 2
0 0 4 12 0 0 4 x3 12 3
The following program, gauss-jordan, adds the necessary lines of code to eligaussp to per-
form the complete Gauss-Jordan elimination of a matrix.

def gaussjordan(A):
'''This function implements gauss-jordan elimination to obtain a diagonal
matrix'''

#Matrix shape
[f,c]=[Link](A)
U=[Link]()
# Step 1: reduce matrix A to a triangular matrix
#For all the columns in A (except the last one)
for i in range(c-1):
#Row pivoting
# Search the maximun in column i
maxcol= [Link](U[i,i])
index = i
for l in range(i,f):
if [Link](U[l,i])>maxcol:
maxcol=[Link](U[l,i])
index=l
# If we have found an element U[l,i] greater than U[i,i] we interchange
# row l with row i
if index!=i:
aux=[Link]([U[i,:]])
U[i,:]=U[index,:]
U[index,:]=aux[:]
# End of Row pivoting
# For all the rows below the diagonal
# print(U)
for j in range(i+1,f):
U[j,:]=U[j,:]-U[i,:]*U[j,i]/U[i,i]
# Step 2: obtain the diagonal matrix
# For all the columns begining by the end
for i in range(c-2,-1,-1):
# For all the rows above the diagonal
for j in range(i-1,-1,-1):
U[j,:]=U[j,:]-U[i,:]*U[j,i]/U[i,i]
return U

If, after applying the Gauss-Jordan elimination to the extended matrix of a system, we divide
each row by the element occupying the main diagonal, we obtain the solutions of the system in the
last column. In our example,
7.3. ITERATIVE METHODS 221

 
1 0 0 1
0 1 0 2
0 0 1 3
The resulting matrix is said to be in row echelon form. It is left as an exercise to add the
necessary code to the above program to result in the row-reduced echelon form of the expanded
matrix of a system.

7.3 Iterative methods

Iterative methods are based on a different approach to the problem of solving systems of linear
equations. The idea in all of them is to find a method that, starting from an initial value for the
solution of the system, refines it progressively, getting closer and closer to the real solution. Figure
7.4, shows a general flow diagram for all iterative methods of solving systems.

Initial solution x(s) = x(0)

Compute
x(s+1) = M (x(s) )

Sı́ convergence:
es ∥x(s+1) − x(s) ∥ ≤ tol? end

No

x(s) = x(s+1)

Figure 7.4: General flowchart of iterative methods for solving systems of equations. The function
M (x) is the one that specifies the method in each case.

Following the flow chart, the first step is to propose a vector with solutions of the system. If
values close to the real solutions are known, these values are chosen as the initial solution. If, as
usual, you have no idea what the solutions are, it is most usual to start with the vector (0),
 
0
0
x(0) = .
 
 .. 
0

From the first solution, a second solution is calculated, following the specific specifications of the
method being used. In the flowchart, the method is represented in a generic way by the function
M (·). We will spend a good part of this section looking at some of the more common methods.
222 CHAPTER 7. SYSTEMS

Once two solutions are available, they are compared. To do this, the modulus of the difference
vector of the two solutions has been used. This modulus gives us a measure of how much the two
solutions resemble each other. If the difference is small enough, –less than a certain tolerance value
tol– we consider the solution to be good and the algorithm terminates. Otherwise, we copy the
last solution obtained into the penultimate one and repeat the whole process. The loop repeats
until the condition that the modulus of the difference of the difference of two successive solutions
is less than the set tolerance is fulfilled.
One question that immediately arises from the general scheme just introduced is whether the
process described converges; and, if it does, whether it converges to the correct solution of the
system.
The answer is that iterative systems do not always converge, that is, it is not guaranteed that
after a certain number of iterations the difference between two successive solutions will be less than
an arbitrary tolerance value. Convergence, as we will see later, will depend both on the system to
be solved and on the iterative method used. On the other hand, what is always true is that, if the
method converges, the successive solutions obtained approach the real solution of the system.

7.3.1 Jacobi’s method

.

Obtaining the algoritm. We will begin by introducing Jacobi’s iterative method, as it is the
most intuitive of all. To introduce it, we will use a simple example. Suppose we want to solve the
following system of equations,

3x1 + 3x2 = 6
3x1 + 4x2 = 7

Suppose we knew the value of x2 beforehand, to obtain x1 , it would then suffice to clear x1 ,
for example from the first equation, and substitute the known value of x2 ,
6 − 3x2
x1 =
3
In the same way, if we knew previously x1 , we could clear x2 , now in the second equation and
substitute the x1 known value.
7 − 3x1
x2 =
4
(0)
Jacobi’s method does this by assuming the known value of x2 , x2 = 0, and using it to obtain
(1)
a value of x1 ,
(0)
(1) 6 − 3x2 6−3·0
x1 = = =2
3 3
(0)
then x1 , x1 = 0 and with it obtains a new value for x2 ,
(0)
(1) 7 − 3x1 7−3·0
x2 = = = 1.75
4 4
(1) (1)
In the next step, we take the obtained values, x1 and x2 as a starting point to calculate new
values,
7.3. ITERATIVE METHODS 223

(1)
(2) 5 − 3x2 6 − 3 · 1.75
x1 = = = 0.25
3 3
(1)
(2) 7 − 3x1 7 − 3 · 1.67
x2 = = = 0.25
4 4
and in general,

(s)
(s+1) 6 − 3x2
x1 =
3
(s)
(s+1) 7 − 3x1
x2 =
4
If we repeat the same calculation ten times we get,

(10)
x1 = 0.7627
(10)
x2 = 0.7627

If we repeat it twenty times,

(20)
x1 = 0.9437
(20)
x2 = 0.9437

The exact solution of the system is x1 = 1, x2 = 1. As we increase the number of iterations,

we see how the solutions obtained get closer and closer to the real solution.
From the example, we can generalise Jacobi’s method for any system of n equations with n
unknowns,

a11 x1 + a12 x2 + · · · + a1n xn = b1

a21 x1 + a22 x2 + · · · + a2n xn = b2
···
an1 x1 + an2 x2 + · · · + ann xn = bn

The solution for the unknown xi in iteration s + 1 from the solution obtained in iteration s
takes the form,
P (s)
(s+1) bi − j̸=i aij xj
xi =
aii
Below, we include the code corresponding to the calculation of one iteration of Jacobi’s algo-
rithm. This code corresponds, for Jacobi’s method, to the function M (·) in the flowchart in figure
7.4.

# For all the equations

xs1=[Link]()
for i in range(nf):
224 CHAPTER 7. SYSTEMS

for j in range(i):
xs1[i]-=A[i,j]*xs[j]

for j in range(i+1,nf):
xs1[i]-=A[i,j]*xs[j]
xs1[i]=xs1[i]/A[i,i]

Matrix expression for Jacobi’s method . Jacobi’s method as described above can also be
expressed in matrix form. In python, the use of matrix form has the advantage of saving loops in
the calculation of the new solution from the previous one. If we express a general system of order
n in matrix form,
A x b
z }| { z }| { z }| {
a11 a12 ··· a1n x1 b1
 a21 a22 ··· a2n 
  x2   b2 
   
 .. .. ..  ·  ..  =  .. 

..
 . . . .   .  .
an1 an2 ··· ann xn bn
We could separate the left-hand expression of the system into three summands: a diagonal
matrix D, a strictly upper diagonal matrix U , and a strictly lower triangular matrix L,

 
D U L x b
z }| { z }| { z }| { z }| { 
z }| {
   
 a11 0 ··· 0 0 a12 ··· a1n 0 0 ··· 0  x1 b1
 0 a22 ··· 0  0 0 ··· a2n   a21 0 ··· 0     b2 
x
     
 2
 . .. .. +  .. .. ..  +  .. .. · . = . 
.. 
   
 . .. .. ..
 . . . . . . . .   . . . .   ..   .. 
 0 0 ··· ann 0 0 ··· 0 an1 an2 · · · 0  xn bn
 

If we pass the terms corresponding to the two triangular matrices to the right-hand side of the
equality,

D x
z
 }| { z }| {
a11 0 ··· 0 x1
 0 a22 ··· 0   x2 
 .. .. ..  ·  ..  =
   
..
 . . . .   . 
0 0 ··· ann xn
 
b U L x
z }| { z }| { z }| { z }| {
   
b1  0 a12 ··· a1n 0 0 ··· 0  x1
 b2  
  0 0 ··· a2n   a21 0 ··· 0   x2 
   

= . − . .. ..  +  .. .. · . 
.. 
 
 ..   .. ..
 .. . . .   . . . .   .. 
bn  0 0 ··· 0 an1 an2 · · · 0  xn
 

If we examine the rows of the resulting matrices on either side of the equal, it is easy to see
that each of them coincides with the expression corresponding to an iteration of Jacobi’s method,
7.3. ITERATIVE METHODS 225

simply by changing the values of the unknowns on the left of the equal by x(s+1) and on the right
by x(s) ,

D xs+1
z }| z }| {
{  (s+1) 
a11 0 ··· 0 x1
 (s+1) 
 0 a22 ··· 0  x2
 
.. .. ..  ·  =

.. .
.   .. 

 . . .  
0 0 ··· ann (s+1)
xn
xs
 
b U L
z }| { z }| { z }|  z }|s {
{
 
b1  0 a12 · · · a1n 0 0 ··· 0  x
 b2     1s 
  0 0 · · · a2n   a21 0 ··· 0   x2 
 
= . − . .. ..  +  .. .. · . 
.. 
 
 ..   .. ..
 .. . . .   . . . .   .. 
bn  0 0 ··· 0 an1 an2 · · · 0  xsn
 

and multiply both sides by the inverse of the matrix D2

xs+1 D −1
z }| { z }| {
 (s+1) −1
x1

a11 0 · · · 0
 (s+1)  
x2  0 a22 · · · 0 
 .  = .. .. . ..  ·

 .  
 .   . . . . . 
(s+1)
xn 0 0 · · · ann
xs
   
b U L
z }| { z }| { z }| { z }|s {

b 0 a · · · a 0 0 ··· 0  x 

 1 12 1n
  1s 

b 0 0 · · · a a 0 ··· 0   x2 
  
 2   2n 

 21

· ..  −  .. .. .. ..  +  .. .. .. ..  ·  . 

.   .. 
  . .
 .
   .
 . .   . .
 bn  0 0 ··· 0 an1 an2 · · · 0  xsn 
   

The final result is a matrix operation,

x(s+1) = D−1 · b − D−1 · (L + U ) · xs

which is equivalent to the code for one iteration and, therefore, to the function M (·) of Jacobi’s
method.
If we analyse the above equation, we observe that the term, D−1 · b is fixed. That is, it remains
the same in all the iterations we need to perform for the solution to converge. The second term
also has a fixed part, −D−1 (L + U ) This is a matrix of order n, equal to that of the system we
are trying to solve. This matrix, called the method matrix, is usually represented by the letter H
and, as we shall see later, it is directly related to the convergence of the method.
If we want to implement Jacobi’s method in python in matrix form, the most efficient way is
to first calculate the matrices D, L, U, f = D−1 −1b and H. Once calculated, we use the result
2 D is a diagonal matrix and its inverse is trivially calculated by simply changing each element of the diagonal

by its inverse,
226 CHAPTER 7. SYSTEMS

to approximate the solution iteratively. Let’s see with an example how to construct the matrices.
Suppose we have the following system,
     
4 2 −1 x1 5
3 −5 1  · x2  = −4
1 −1 6 x3 17
First, we calculate the matrix D, from the coefficient matrix of the system, using the diag
function of numpy. Applying diag to a matrix, we extract the elements of its diagonal into a
vector,
Applying diag to a vector constructs a matrix with the elements of the vector placed on the
diagonal of the matrix. The rest of the elements are zero.
If we apply diag twice on the matrix of coefficients of a system, we obtain directly the matrix
D,

A=[Link]([[4., 2.,-1.],[3., -5., 1.],[1., -1., 6.]])

[Link](A)
Out[1]: array([ 4., -5., 6.])

[Link]([Link](A))
Out[2]:
array([[ 4., 0., 0.],
[ 0., -5., 0.],
[ 0., 0., 6.]])

To compute the L and U matrices, we can use the numpy triu and tril which extract an upper
triangular matrix and a lower triangular matrix, respectively, from a given matrix.

[Link](A)
Out[3]:
array([[ 4., 2., -1.],
[ 0., -5., 1.],
[ 0., 0., 6.]])

[Link](A)
Out[4]:
array([[ 4., 0., 0.],
[ 3., -5., 0.],
[ 1., -1., 6.]])

The matrices L and U are strictly upper and lower triangular. We can obtain them by sub-
tracting the matrix D from the matrices we have just obtained,

[Link](A)-[Link]([Link](A))
Out[5]:
array([[ 0., 2., -1.],
[ 0., 0., 1.],
[ 0., 0., 0.]])

[Link](A)-[Link]([Link](A))
Out[6]:
7.3. ITERATIVE METHODS 227

array([[ 0., 0., 0.],

[ 3., 0., 0.],
[ 1., -1., 0.]])
We can also obtain them directly, using only the coefficient matrix,
[Link](A)
Out[7]:
array([[ 0., 2., -1.],
[ 0., 0., 1.],
[ 0., 0., 0.]])

[Link](A)
Out[8]:
array([[ 0., 0., 0.],
[ 3., 0., 0.],
[ 1., -1., 0.]])
We then construct the vector f
{f=([Link](A)).dot(b)}
, and the matrix of the system H
H=-([Link](A)).dot(U+L)
From the constructed matrices, the code to compute an iteration by Jacobi’s method would be
simply, xs1=f+[Link](xs).
The following code fragment brings together all the operations described,
def jacobi(A,b,x0,tol,itmax):
[nf,nc]=[Link](A)
xs=[Link]()
xs1=[Link]()
it=0
# Build the matrices D,U,f,H
D=[Link]([Link](A))
U=[Link](A)-D
L=[Link](A)-D
invD=[Link](D)
f=(invD).dot(b)
H=-[Link](L+U)
# First iteration
xs1=f+[Link](xs)
error=[Link](xs1-xs)
#Iteration counter
it+=1
'''From here would come the code needed to calculate the successive iterations
until the solution converges''

7.3.2 The Gauss-Seidel method

.
228 CHAPTER 7. SYSTEMS

Obtaining the algorithm . This method applies a simple and logical improvement to the Jacobi
method we have just studied. If we rewrite the general expression for calculating an iteration of
one of the components of the solution of a system by Jacobi’s method,
P (s)
(s+1) bi − j̸=i aij xj
xi =
aii
(s+1) (s)
We observe that to obtain the term xi
we use all the terms xj , jneqi of the previous
iteration. However, it is easy to realise that, as the algorithm calculates the components of each
(s+1)
iteration in order, when we calculate xi , we already have the result of all the previous compo-
(s+1)
nents of the solution for iteration s + 1. That is, we already know the result of xl , l < i. If the
algorithm converges, these solutions will be better than those obtained in the previous iteration.
(s+1)
If we use them to calculate xi , the result we obtain will be closer to the exact solution and
therefore the algorithm converges faster.
The idea, therefore, would be:
(s+1)
To calculate x1 we proceed in the same way as in Jacobi’s method.
Pn (s)
(s+1) b1 − j=2 a1j xj
x1 =
a11
(s+1) (s+1)
To calculate x2 we use the solution we have just obtained x1 and all the remaining
solutions from the previous iterations,
Pn (s)
(s+1) b2 − a21 xs+1
1 − j=3 a2j xj
x2 =
a22
(s+1) (s+1) (s+1)
To calculate x3 we use the two solutions we have just obtained x1 , x2 and all the
remaining solutions from the previous iterations,
Pn (s)
(s+1) b3 − a31 xs+1
1 − a32 xs+1
2 − j=4 a3j xj
x3 =
a33
And for any i component of the solution, we obtain the general expression of the Gauss-Seidel
method,
P (s+1) P (s)
(s+1) bi − j<i aij xj − j>i aij xj
xi =
aii
The following code implements an iteration of the Gauss-Seidel method and can be considered
as the function M (·), included in the flowchart in figure 7.4, for the Gauss-Seidel method.
Interestingly, the only change in the entire code from Jacobi’s method is the replacement of the
variable xs(j) with the variable xs1(j) at the end of the first nested for loop.

# For all the equations

xs1=[Link]()
for i in range(nf):
# For all the terms above x[i]
for j in range(i):
xs1[i]-=A[i,j]*xs1[j]
#For all the terms below x[i]
for j in range(i+1,nf):
7.3. ITERATIVE METHODS 229

xs1[i]-=A[i,j]*xs[j]
xs1[i]=xs1[i]/A[i,i]
error= [Link](xs1-xs)
xs=[Link]()
it+=1

Matrix form of the Gauss-Seidel method . Analogous to how we did for Jacobi’s method,
it is possible to obtain a matrix solution for the Gauss-Seidel method.
Suppose we perform the same decomposition into summands of the coefficient matrix as we did
for Jacobi’s method,

A · x = b → (D + L + U ) · x = b

In this case, the calculation of each component of the solution in an iteration involves the
components of the previous iteration that are below the one we want to calculate, therefore we
only have to pass to the right of the equality, the matrix U , which contains the coefficients that
multiply these components,

(D + L + U ) · x = b → (D + L) · x = b − U · x

Substituting x on each side of the equality for x(s+1) and x(s) and multiplying by the inverse
of D + U on both sides, we obtain the expression in matrix form of the calculation of an iteration
by the Gauss-Seidel method,

x(s+1) = (D + L)−1 · b − (D + L)−1 · U · x(s)

As we have done with Jacobi’s method, we can identify the fixed parts that do not change when
iterating: f = (D + L)−1 · b and the method matrix, that in this case is, H = −(D + L)−1 · U .
The following code fragment shows the construction of the matrices needed to implement the
Gauss-Seidel method,

[nf,nc]=[Link](A)
xs=[Link]()
xs1=[Link]()
error= [Link](xs1-xs)
it=0
D=[Link]([Link](A))
U=[Link](A)-D
L=[Link](A)-D
invD=[Link](D+L)
f=(invD).dot(b)
H=-[Link](U)
# First iteration
xs1=f+[Link](xs)
error=[Link](xs1-xs)
it+=1
'''From here would come the code needed to calculate the
successive iterations until the solution converges''
230 CHAPTER 7. SYSTEMS

In general, the Gauss-Seidel method is able to obtain solutions, for a given system and a
certain tolerance value, using fewer iterations than the Jacobi method. For example, if we apply
both methods to the solution of the system,
     
4 2 −1 x1 5
3 −5 1  · x2  = −4
1 −1 6 x3 17
Using in both cases a tolerance tol = 0.00001, the Jacobi method needs 23 iterations to achieve
a solution, while the Gauss-Seidel method needs only 13. The figure 7.5, shows the evolution of
the tolerance tol = ∥x(s+1) − x(s) ∥, as a function of the iteration number. In both cases the initial
value of the solution was taken as the vector (0, 0, 0)T .

4
Gauss−Seidel
Jacobi
3.5

−3 Detalle de las últimas iteraciones, (tol<0.00001)

2.5 x 10
tol.=||x(s+1)−x(s)||

2
tol.=||x(s+1)−x(s)||

1.5 2

1
1

0
0.5 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
nº de iteración

0
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
nº de iteración

Figure 7.5: evolution of the tolerance (modulus of the difference between two successive solutions)
for the same system solved by the Gauss-Seidel method and the Jacobi method

7.3.3 Weighted methods.

In weighted iterations we modify an iterative method so that at each iteration, the weighted average
of the results of two successive iterations is given as the solution,

x(s+1) = ω · x∗ + (1 − ω) · x(s)

Where x(∗) represents the value that would be obtained by applying one iteration of the method
to x(s) , i.e. it would be the value of x(s+1) if no weight were applied.
The parameter ω is called the relaxation factor. If

0<ω<1
7.3. ITERATIVE METHODS 231

this is a method of under-relaxation. Its use makes it possible to solve systems that do not converge
if the same method is used without relaxation. If ω > 1 the method is called over-relaxation, it
allows to accelerate the convergence with respect to the same method without relaxation. Finally,
if we make ω = 1, we recover the original method without relaxation.

The weighted Jacobi method. This is obtained by applying the relaxation method just de-
scribed to Jacobi’s method. The general expression for an iteration of the damped Jacobi method
would be,

x(∗)
z }| {
P (s)
(s+1) bi − j̸=i aij xj
xi =ω· +(1 − ω) · xsi
aii
To implement it, it would be sufficient to add to the code of Jacobi’s method a line including
the average between the two successive solutions,

error= [Link](xs1-xs)
it=0
while (error>tol)&(it<=itmax):
# For all the equations
xs1=[Link]()
for i in range(nf):
for j in range(i):
xs1[i]-=A[i,j]*xs[j]

for j in range(i+1,nf):
xs1[i]-=A[i,j]*xs[j]
xs1[i]=xs1[i]/A[i,i]
# Weighted iteration
xs1=w*xs1+(1-w)*xs
error= [Link](xs1-xs)
xs=[Link]()
it+=1

In matrix form, the general expression for the damped Jacobi method would be,

x(∗)
z }| {
(s+1) −1 −1 (s)
x =ω· D · b − D · (L + U ) · x +(1 − w)x(s)

Reordering the expression,

x(s+1) = ω · D−1 · b + (1 − w) · I − w · D−1 · (L + U ) · x(s)

 

We can easily identify the fixed term f = ω · D−1 · b and the method matrix

H = (1 − w) · I − w · D−1 · (L + U )

To implement the code of the damped Jacobi method, we must calculate the identity matrix
of the system size and modify the expressions of f and H,
232 CHAPTER 7. SYSTEMS

[nf,nc]=[Link](A)
xs=[Link]()
xs1=[Link]()
error= [Link](xs1-xs)
it=0
D=[Link]([Link](A))
U=[Link](A)-D
L=[Link](A)-D
invD=[Link](D)
I=[Link](nf)
f=w*(invD).dot(b)
H=(1-w)*I-w*[Link](L+U)
# First iteration
xs1=f+[Link](xs)
error=[Link](xs1-xs)
it+=1
'''From here would come the code needed to calculate the
successive iterations until the solution converges''

The SOR method . The SOR method – ‘successive over-relaxation’ – is obtained by applying
a weigth to the Gauss-Seidel method. Applying the same reasoning as the weighted Jacobi case,
the general expression for an iteration of the SOR method is,

x(∗)
z }| {
P (s+1) P (s)
(s+1) bi − j<i aij xj − j>i aij xj (s)
xi =ω· +(1 − ω) · xi
aii

As in the case of the damped Jacobi, to implement the SOR method it is sufficient to add a
line that calculates the average of two successive solutions,

# For all the equations

xs1=[Link]()
for i in range(nf):
# For all the terms above x[i]
for j in range(i):
xs1[i]-=A[i,j]*xs1[j]
#For all the terms below x[i]
for j in range(i+1,nf):
xs1[i]-=A[i,j]*xs[j]
xs1[i]=xs1[i]/A[i,i]
xs1=w*xs1+(1-w)*xs
error= [Link](xs1-xs)
xs=[Link]()
it+=1

In matrix form the expression for one iteration of the SOR method would be,
7.3. ITERATIVE METHODS 233

x(∗)
z }| {
−1 −1
x(s+1) = ω · (D + L) · b − (D + L) · U · x (s)
+(1 − ω) · x(s)

And after reordering,

x(s+1) = ω · (D + L)−1 · b + (1 − ω) · I − ω · (D + L)−1 · U · x(s)

 

Again, we can identify the
fixed term, f = ω · (D + L)−1 · b and the method matrix, H =

(1 − ω) · I − ω · (D + L)−1 · U .
The following code fragment shows how f and H are obtained for the SOR method,

[nf,nc]=[Link](A)
xs=[Link]()
xs1=[Link]()
error= [Link](xs1-xs)
it=0
D=[Link]([Link](A))
U=[Link](A)-D
L=[Link](A)-D
invD=[Link](D+L)
I=[Link](nf)
f=w*(invD).dot(b)
H=(1-w)*I-w*[Link](U)
#First iteration
xs1=f+[Link](xs)
error=[Link](xs1-xs)
it+=1
xs=[Link]()

7.3.4 Convergence analysis

In the introduction to iterative methods we left open the question of their convergence. Let us
analyse in more detail under what conditions we can ensure that an iterative method, applied to
a particular system of linear equations, converges.
First of all, we have to define what we mean by convergence. When an iterative method
converges, it does so asymptotically. That is, it would take an infinite number of iterations to
reach the exact solution.

x(0) → x(1) → x(2) → · · · → x(∞) = x

Logically, it is unfeasible to perform an infinite number of iterations. For this reason, the
solutions of iterative methods are always approximate; we perform a finite number of iterations
until a certain convergence condition is met. Since we do not know the exact solution, we impose
this condition between two successive iterations,

∥x(s+1) − x(s) ∥ ≤ C ⇒ ∥x(s+1) − x∥ = |e(s+1) |

Where e(s+1) would represent the real convergence error committed in iteration s + 1.
234 CHAPTER 7. SYSTEMS

Taking as a starting point the general expression for the calculation of an iteration in matrix
form,
x(s+1) = f + H · x(s)
We can express the convergence error as,

e(s+1) = x(s+1) − x = f + H · x(s) − x

But the exact solution, if it could be achieved, would comply,

x=f +H ·x

And if we substitute in the expression for the convergence error,

e(s+1) = f + H · x(s) − f − H · x
We finally arrive to the following expression, that relates the convergence errors between two
sucessive iterations,

e(s+1) = H · e(s)
For the error to decrease from iteration to iteration and for the method to converge, the method
matrix H must have norm-2 less than unity.
Suppose that a system of dimension n its method matrix H, has a set of n linearly independent
eigenvectors, w1 , w2 , · · · , wn , each one associated to its corresponding eigenvalue, λ1 , λ2 , · · · , λn .
The convergence error is also a vector of dimension n, so we can express it as a linear combination
of the n linearly independent eigenvectors of the matrix H. Suppose we do this for the convergence
error e(0) corresponding to the initial value of the solution x(0) ,

e(0) = α1 · w1 + α2 · w2 + · · · + αn · wn
If we use the equation deduced earlier for the error ratio between two successive iterations
and remembering that applying a matrix to an eigenvector is equivalent to multiplying it by the
corresponding eigenvalue: H · wi = λi · wi , we obtain for the error of convergence at iteration s,

e(1) = H · e(0) = α1 · λ1 · w1 + α2 · λ2 · w2 + · · · + αn · λn · wn
e(2) = H · e(1) = H 2 · e(0) = α1 · λ21 · w1 + α2 · λ22 · w2 + · · · + αn · λ2n · wn
..
.
e(s) = H · e(s−1) = H s · e(0) = α1 · λs1 · w1 + α2 · λs2 · w2 + · · · + αn · λsn · wn

For the error to tend to zero, e(s) → 0 as s increases, for any initial combination of values αi ,
i.e. for any initial approximation x(0) , it is necessary that all the eigenvalues of the method matrix
satisfy,

|λi | < 1
Therefore, the system converges if the spectral radius of the method matrix is less than unity.
3

ρ(H) < 1 ⇒ lim e(s) = 0

s→∞
3 The spectral radius of a matrix is the largest of its eigenvalues in absolute value.
7.3. ITERATIVE METHODS 235

Speed of convergence . For a sufficiently large number of iterations, the spectral radius of
the method matrix gives the speed of convergence. This is because the remaining error terms
associated with other smaller eigenvalues tend to zero faster. We can therefore make the following
approximation, where we are assuming that the eigenvalue λn is the spectral radius,

e(s) ≈ cn ρ(H)s wn = cn λsn wn

We can now calculate how many iterations it will take to reduce an initial error below a certain
value. This will depend on the initial solution and the spectral radius of the method matrix,

e(s)
e(s) ≈ ρ(H)s e(0) ⇒ ≈ ρ(H)s
e(0)
so for example if we want to reduce the initial error by m digits,

e(s) −m
≈ ρ(h)s ≤ 10−m ⇒ s ≥
e(0) log10 (ρ(H))
The method matrix therefore plays a fundamental role both in the convergence of the method
and in the speed (number of iterations) with which the method converges. Damped methods allow
the convergence matrix to be modified, thanks to the damping factor omega, making systems for
which it is not possible to find a solution by means of an iterative method converge.
As an example, the system,
     
1 2 −1 x1 2
2 −5 1  · x2  = −5
1 −1 3 x3 8
It does not converge if we try to solve it by the Jacobi method. However, it is possible to obtain
its solution using the Weighted Jacobi method. Figure 7.6 shows the evolution of the tolerance for
this system using both methods.
If we calculate the spectral radius of the method matrix, for the Jacobi method we would have,

import numpy as np
A=[Link]([[1,2,-1],[2,-5,1],[1,-1,3]])
D=[Link]([Link](A))
U=[Link](A)-D
L=[Link](A)-D
invD=[Link](D)
H=[Link](L+U)
eigenvalues,eigenvectors=[Link](H)
spectral_rad=[Link]([Link](eigenvalues))
print("Eigenvalues: ",eigenvalues)
print("Spectral radius: ", spectral_rad)

Eigenvalues: [ 0.11872445+1.05306845j 0.11872445-1.05306845j -0.2374489 +0.j ]

Spectral radius: 1.0597398959658624

The spectral radius is greater than unity and the method does not converge.
If we repeat the calculation for the weighted Jacobi method, with ω = 0.5,

D=[Link]([Link](A))
U=[Link](A)-D
236 CHAPTER 7. SYSTEMS

18

16 Jacobi amortiguado. ω = 0.5

Jacobi
14

12
tol.=||x(s+1)−x(s)||

10

0
0 5 10 15 20 25 30
nº de iteración

Figure 7.6: Tolerance evolution for the same system using the Jacobi method (diverges) and the
weighted Jacobi method (converges).

L=[Link](A)-D
invD=[Link](D)
w=0.5
I=[Link](3)
H=(1-w)*I-w*[Link](L+U)
eigenvalues,eigenvectors=[Link](H)
spectral_rad=[Link]([Link](eigenvalues))
print("Eigenvalues: ",eigenvalues)
print("Spectral radius: ", spectral_rad)

Eigenvalues: [0.44063777+0.52653422j 0.44063777-0.52653422j 0.61872445+0.j ]

Spectral radius: 0.6865857095839855

The spectral radius is now smaller than unity and the method converges.
Finally, it should be noted that any of the iterative methods described converges for a system
whose coefficient matrix is strictly diagonally dominant.
Chapter 8

Interpolation and function fitting

Digo que ya tú sabes que la

humildad es la basa y fundamento
de todas las virtudes, y que sin
ella no hay alguna que lo sea.
Ella allana inconvenientes, vence
dificultades, y es un medio que
siempre a gloriosos fines nos
conduce; de los enemigos hace
amigos, templa la cólera de los
airados y menoscaba la arrogancia
de los soberbios; es madre de la
modestia y hermana de la
templanza; en fin, con ella no
pueden atravesar triunfo que les
sea de provecho los vicios, porque
en su blandura y mansedumbre se
embotan y despuntan las flechas
de los pecados.

El coloquio de los perros. Miguel

de Cervantes

In this chapter, we will explore different methods of polynomial approximation. We can define
the problem in general terms as finding a polynomial to represent a function f (x).

f (x) ≈ p(x) = a0 + a1 · x + a2 · x2 + a3 · x3 + · · · + an · xn
To obtain the approximation, we can start from the equation that defines f (x), for instance,
the error function,
Z x
2 2
erf (x) = √ e−t dt
π 0
Nevertheless, it is possible that we only have some values of the function, such as when we only
have a data table.
To approximate a function using a polynomial has pros and cons.
Perhaps the main advantage, from a computing point of view, is that a polynomial is easy to
evaluate using a computer as it only involves simple arithmetical operations. Besides, Polynomial

237
238 CHAPTER 8. INTERP. & FUNCT FITTING

Table 8.1: f (x) = erf (x)

x f (x)
0.0 0.0000
0.1 0.1125
0.2 0.2227
0.3 0.3286
0.4 0.4284
0.5 0.5205

integration and derivation are operations that are easy to carry out, yielding other polynomials.
Among their drawbacks we have to remark the polynomial growing towards infinity or minus
infinity as the independent variable goes away zero. This can give rise in some cases to rounding
errors that are difficult to handle, making the approximation for non-increasing functions very
difficult.
We will study three different methods: first we will see the approximation using the Taylor
polynomial, very useful to approximate a function close to a point for which we know the function
value. Later, we will discus polynomial interpolation and finally we present mean square polynomial
fitting.
We may relate the use of one or another method to the available information on the function
we want to approximate and to the objectives we want to reach with the approximation.

8.1 The Taylor’s polynomial.

Suppose we have an function infinitely derivable around a point x0 . We define the function taylor
series expansion as,

1 1 1
f (x) = f (x0 )+f ′ (x0 )·(x−x0 )+ f ′′ (x0 )·(x−x0 )2 +· · ·+ f (n) (x0 )·(x−x0 )n + f (n+1) (z)·(x−x0 )n+1
2 n! (n + 1)!

Where z is an indeterminate point located between x and x0 . If we eliminate the last term the
function would be approximated by a degree n polynomial.

1 1
f (x) ≈ f (x0 ) + f ′ (x0 ) · (x − x0 ) + f ′′ (x0 ) · (x − x0 )2 + · · · + f (n) (x0 ) · (x − x0 )n
2 n!
The error we get when approximating a function by a degree n Taylor polynomial can be
obtained as,

1
e(x) = |f (x) − p(x)| = f (n+1) (z) · (x − x0 )n+1
(n + 1)!
Looking at the error function, it is easy to realize that decreases when the polynomial degree
increases and increases when the distance between x and x0 increases.
For instance, the Taylor polynomial of degree n for the exponential functions, around the point
x0 = 0 is,
n
1 1 X1
ex ≈ 1 + x + x2 + · · · + xn = xi
2 n! i=0
i!
 8.1. THE TAYLOR’S POLYNOMIAL. 239

and the Taylor polynomial for the logarithm, around the point x− == 1 is,
n
1 (−1)n+1 X (−1)i+1
log(x) ≈ (x − 1) − (x − 1)2 + · · · + (x − 1)n = (x − 1)i
2 n i=1
i
The existence of a general term for Taylor’s expansion of many essential functions makes it
highly interesting to approximate functions using a computer. For example, the following function,
written in Python, approximates the natural logarithm using Taylor’s polynomial of any degree
we want.
series de [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Fri Jul 12 15:02:07 2024
5

7 @author: juan
8 """
9 import numpy as np
10 from matplotlib import pyplot as pl
11 def taylorln(x,n):
12 """
13 Esta función aproxima el valor del logaritmo natural de un numero
14 empleando para ello un polinomio de Taylor de grado n desarrollado en
15 torno a x=1. Las variables de entrada son: x, valor para el que se desea
16 calcular el logaritmo. n Grado del polinomio que se empleará en el
17 cálculo. La variable de salida y es el logaritmo de x.
18 This fuction computes (approx) the value of the natural logarithm for a number
19 using a degree n Taylor's polynomial, at x0=1. input variables are: x, value to
20 calculate its logarithm. n degree of the polynomial useto aporaches the log.
21 the funtion returns the locarithm computed.
22

23 Parameters
24 ----------
25 x : Real
26

27 n : int
28 Taylor's polinomial Degree
29

30 Returns
31 -------
32 y : real
33 log(x)
34

35 """
36 y = 0
37 for i in range(1,n+1):
38 y = y+(-1)**(i+1)*(x-1)**i/i
39

40 return(y)
41

42
 240 CHAPTER 8. INTERP. & FUNCT FITTING

43 def taylorcum(fun,n,x):
44 """
45

46

47 Parameters
48 ----------
49 fun : takes the taylor series of degree n, described in input function fun
50 and calculates and draws the result for an array of points, x
51 DESCRIPTION.
52 n : TYPE int
53 DESCRIPTION.
54 Taylor's polynomial degree'
55 x : TYPE array of real numbers
56 DESCRIPTION. point to calculate the Taylor series
57

58 Returns
59 -------
60 y : Type real
61 DESCRIPTION: array of values computed
62

63 """
64

65 y =[Link]([fun(i,n) for i in x])

66 [Link](x,y)

The approximation works reasonably well for points inside the interval 0 < x < 2. Figure 8.1
shows the results achieved inside such interval, using Taylor’s polynomial with degrees 2, 5, 10 and
20. The blue line represent the logarithm values we get using the numpy function log.
Functions sin(x) and cos(x), are also easy to approach using Taylor’s polynomials. If we expand
around x0 = 0, the cosine series will have only even powers and the sine series will only have odd
powers,

n
X (−1)i
cos(x) ≈ x2i
i=0
(2i)!
n
X (−1)i 2i+1
sin(x) ≈ x
i=0
(2i + 1)!

Figures 8.2a and 8.2b show aproximations to sine and consine functions using Taylor’s polyno-
mials. We have use polynomias up to 8DWM1001 degree for the cosine function and up to degree
9 for the sine function. In both cases we have calculated the results inside the interval (−π, π).
When we compare these results with those yielded by Numpy functions cos and sin, we see that
the approximation is quite good for the higher degree polynomials we have used in each case.

8.2 Polynomial interpolation.

We define the interpolation as a process that, departing from a set of data pairs
(x0 , y0 ), (x1 , y1 ), · · · (xn , yn ), allows us to find a function f (x) such that yi = f (xi ) for all pairs
(xi , yi ) of the set. In the case we use a polynomial as the interpolating function f (x) ≡ p(x), we
denote it as polynomial Interpolation.
8.2. POLYNOMIAL INTERPOLATION 241

x2
x3
x5

0 x10
x20
log(x)

−1

−2
x

−3

−4

−5
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2
x

Figure 8.1: A comparison among the results achieved using Taylor polynomials to approach the
logarithm. (2, 3, 5, 10, 20 degrees)

The interpolation theorem. For any set of n + 1 pairs of data points

(x0 , y0 ), (x1 , y1 ), · · · (xn , yn ) , where no two xi are the same, there is one and only one polynomial
p(x) of degree n that interpolates these points, i.e, it satisfies that yi = p(xi ) for all pairs on the
dataset.
If we try to interpolate the points with a polynomial whose degree is less than n, it is possible
that we will not find one that fits all the pairs in the dataset. Conversely, if we try to use a
polynomial with a degree greater than n, it will not unique. Eventually, if we use a degree n
polynomial, this polynomial is always the same regardless method used to build it.

8.2.1 The Vandermonde’s matrix

Suppose we have a set A of data points.
For a polynomial of degree n,

p(x) = a0 + a1 x + a2 x2 + · · · + an xn
to go through all pairs in A, it must satisfy,

yi = a0 + a1 xi + a2 x2i + · · · + an xni , ∀(xi , yi ) ∈ A

242 CHAPTER 8. INTERP. & FUNCT FITTING

1 x3
x2 1 x5
x4 x7
x
6
0.8 x9
x
8 sin(x)
0.5 cos(x)
0.6

0.4

0 0.2

0
y

−0.2
−0.5
−0.4

−0.6

−1
−0.8

−1

−1.5
−4 −3 −2 −1 0 1 2 3 4 −4 −3 −2 −1 0 1 2 3 4
x

(a) polynomials 2, 4, 6 and 8 degrees (b) polynomials 2, 4, 6 and 8 degrees

Figure 8.2: Taylor polynomial for cosine and sine functions

x f (x)
x0 y0
x1 y1
x2 y2
.. ..
. .
xn yn

So, we would have a system of n linear equations, one for each pair of data points, where the
unknowns are the n + 1 coefficients ai of the polynomial.
for example, for the dataset,

x f (x)
1 2
2 1
3 −2

we get,

a0 + a1 · 1 + a2 · 12 = 2
a0 + a1 · 2 + a2 · 22 = 1
a0 + a1 · 3 + a2 · 32 = −2
which we can express in matrix form as,

12
     
1 1 a0 2
1 2 22  · a1  =  1 
1 3 32 a2 −2
And, in general, for n data pairs,

1 x0 x20 xn0
     
··· a0 y0
1 x1 x21 ··· xn1   a1   y1 
 .. .. .. ..  ·  ..  =  .. 
     
..
. . . . .  . .
1 xn x2n ··· xnn an yn
 8.2. POLYNOMIAL INTERPOLATION 243

The coefficient matrix of the resulting system is called the Vandermonde’s matrix. Its elements
are hte n first powers of the independent variable values, allocated by rows. It is easy to notice that
when the number of data increases the difference among the elements of a row will tend to increase
also. For this reason in most cases the Vandermonde’s matrix is a poor conditioned matrix and
so not suitable for solving the system numerically. For this reason, in practice,the interpolation
polynomial is computed using other alternative methods.

8.2.2 Lagrange Interpolating Polynomial

Departing from the values x0 , x1 , · · · , xn , We build the following set of n + 1 polynomial of degree
n
n
Y x − xk (x − x0 )(x − x1 ) · · · (x − xj−1 )(x − xj+1 ) · · · (x − xn )
lj (x) = =
xj − xk (xj − x0 )(xj − x1 ) · · · (xj − xj−1 )(xj − xj+1 ) · · · (xj − xn )
k=0
k̸=j

These polynomial exhibit an interesting property when evaluated at the points x0 , x1 , · · · , xn ,

we have used for building them.
(
1, i = j
lj (xi ) =
0, i ̸= j
From this polynomials we can build a interporlation polynomial using the codomain values
y0 , y1 · · · , yn corresponding to the domain values x0 , x1 , · · · , xn ,
n
X
p(x) = lj (x) · yj
j=0

It is ease to check that, using this method for building the interpolation polynomial, it passes
through all pairs of points xi , yi , because p(xi ) = yi .
The following python function lagrang calculates, at any point x, the Lagrange interpolating
polynomial value built from a set of pairs of points A ≡ {(xi , yi )}.

lgr [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Tue Jul 23 10:40:23 2024
5

6 @author: juan
7 A code to implement the Lagrange polynomial
8 """
9 import numpy as np
10

11 def lagrang(x,y,x1):
12 """
13 Function to calculate the Lagrange interpolating polynomial at point x1
14

15 Parameters
16 ----------
17 x : TYPE real np array
18 DESCRIPTION. Table Values x to be interpolated
 244 CHAPTER 8. INTERP. & FUNCT FITTING

19 y : TYPE real np array

20 DESCRIPTION. Table values y to be interpolated
21 x1 : TYPE real
22 DESCRIPTION. Point a which the polynomial is evaluated
23

24 Returns
25 -------
26 y1 : TYPE real
27 DESCRIPTION. Lagrange polynomial Value at x1. y1 = lagrang(x1)
28

29 """
30 y1 = 0
31 n = [Link][0]
32 for j in range(0,n):
33 lj = 1
34 for i in range(0,j):
35 lj = lj*(x1-x[i])/(x[j]-x[i])
36 for i in range(j+1,n):
37 lj = lj*(x1-x[i])/(x[j]-x[i])
38 y1 = y1 +lj*y[j]
39 return(y1)
40

41 def langrmult(x,y,x1):
42 """
43 Takes the funtion lagrang and calculates the values of the Lagrange
44 polinomial at any point in array x1
45

46 Parameters
47 ----------
48 x : TYPE real np array
49 DESCRIPTION. Table Values x to be interpolated
50 y : TYPE real np array
51 DESCRIPTION. Table values y to be interpolated
52 x1 : TYPE real
53 DESCRIPTION. array of point a which the polynomial is evaluated
54

55 Returns
56 -------
57 y1 : TYPE real
58 DESCRIPTION. Lagrange polynomial Valueat x1. y1 = PLagrange(x1)
59

60 """
61 yaux = [] #fast way to create an array same size as x
62 for i in x1:
63 [Link](lagrang(x,y,i))
64 y1 = [Link](yaux)
65 return(y1)
66

67
8.2. POLYNOMIAL INTERPOLATION 245

8.2.3 Divided diferences.

The two method we have studied so far, Vadermonde matrix and Lagrange polynomial, have a
common drawback. If we add a new pair of data (xn+1 , yn+1 ) to the existing data collection, it is
necessary to recalculate the interpolation polynomial from scratch.
The divided differences algorithm allows us to build the interpolation polynomial performing
less operations than in the case of Lagrange polynomial. Besides, the computing is carried out step-
wise, making use of all previous results when a new data pair is added to calculate the interpolation
polynomial.
We build the divided differences polynomial of degree n as follows,

pn (x) = a0 +(x−x0 )·a1 +(x−x0 )·(x−x1 )·a2 +· · ·+(x−x0 )·(x−x1 ) · · · (x−xn−2 )·(x−xn−1 )·an
Where (x0 , y0 ), (x1 , y1 ), · · · (xn , yn ), represent the dataset for which we want to calculate the
interpolation polynomial of degree n. If we evaluate the polynomial using the dataset we arrive to
a lower triangular system of linear equation, where the polynomial coefficients are the unknowns.

a0 = y0
a0 + (x1 − x0 )a1 = y1
a0 + (x2 − x0 )a1 + (x2 − x0 )(x2 − x1 )a2 = y2
···
a0 + (xn − x0 )a1 + · · · + (xn − x0 )(xn − x1 ) · · · (xn − xn−2 )(xn − xn−1 )an = yn
This system can be solved using the divided differences algorithm.
We define the first-order two-point divided difference (x0 , y0 ) y (x1 , y1 ) as,
y1 − y0
f [x0 , x1 ] =
x1 − x0
For three point, (x0 , y0 ), (x1 , y1 ) y (x2 , y2 ), we define the second-order divided difference as,

f [x1 , x2 ] − f [x0 , x1 ]
f [x0 , x1 , x2 ] =
x2 − x0
and eventually, we can define the i − order divide diference for i + 1 points as,

f [x1 , x2 , · · · , xi ] − f [x0 , x1 , · · · , xi−1 ]

f [x0 , x1 , · · · , xi ] =
xi − x0
We can now get the polynomial coefficients applying progressive substitutions to the lower
triangular system defined above, then, each coefficient may be related with a divided difference,

a0 = f [x0 ] = y0
a1 = f [x0 , x1 ]
..
.
ai = f [x0 , x1 , · · · , xi ]
..
.
an = f [x0 , x1 , · · · , xn ]
 246 CHAPTER 8. INTERP. & FUNCT FITTING

Therefore we can get the polynomial coefficients straightforwardly if we compute the divided
differences. Let’s see an example using the following four data set.

x 0 1 3 4
y 1 -1 2 3

Usually, we build the divided difference polynomial, using a table such as table 8.2. the first
column contains the values of variable x the second the values of the cero-order divided differences
(values of variable y). From the second one on, the following columns contain the divided differences
of the elements held in the previous column. These differences are calculated using the elements
located on consecutive rows. The table lost a row each time we advance a column. When we arrive
to the n − 1-order divided difference, we have a single value that depends on all initial data.

Table 8.2: four data divided difference table

xi yi f [xi , xi+1 ] f [xi , xi+1 , xi+2 ] f [xi , xi+1 , xi+2 , xi+3 ]

x0 =0 y0 = 1 f [x0 , x1 ] = −2 f [x0 , x1 , x2 ] = 7/6 f [x0 , x1 , x2 , x3 ] = −1/3
x1 =1 y1 = −1 f [x1 , x2 ] = 3/2 f [x1 , x2 , x3 ] = −1/6
x2 =3 y2 = 2 f [x2 , x3 ] = 1
x3 =4 y3 = 3

The divided differences polynomial coefficients are the elements held in the first row of the
table. So, for our example, we obtain the following interpolation polynomial,
7 1
p3 (x) = 1 − 2x + x(x − 1) − x(x − 1)(x − 3)
6 3
Notice that the interpolation polynomial, built using divided differences, it always represented
as a sum of binomials products (x − x0 )(x − x1 ) · · · and the coefficients computed belong to this
representation a not to the standard polynomial representation as a sum of variable x powers.
The following code implement the divided differences polynomial from a set of n data. We
must take into account the special form of the polynomial to calculate the value ot takes in an
specific x point. Function difdiv computes the polynomiañ coefficients departing from a set x, y
of data. Function evdif calculates the value of the polynomial in a point whatsoever, using the
computed coefficientes and the x values of the data set.

dif [Link]
1 #!/usr/bin/env python3
2 # -*- coding: utf-8 -*-
3 """
4 Created on Wed Jul 24 15:22:01 2024
5 This file define two funtion. one to calculate the divided differences
6 polynomial coefficients and the second one to evaluate the divided differences
7 polynomial any point
8 @author: juan
9 """
10 import numpy as np
11 import [Link] as pl
12 def difdiv(x,y):
13 """
14 This functions computes the coefficient of a the divided differences
15 polynomial using thw data arrays x,y
 8.2. POLYNOMIAL INTERPOLATION 247

16

17 Parameters
18 ----------
19 x : TYPE double
20 DESCRIPTION. numpy array with data x
21 y : TYPE double
22 DESCRIPTION. numpy array with data y
23

24 Returns
25 -------
26 a : TYPE double
27 DESCRIPTION: coefficientes of the polynomial
28 p_n(x)=a_0+(x-x_0)a_1+(x-x_0)(x-x_1)a_2+...
29 ...+(x-x_0)(x-x_1)...(x-x_{n-2})(x-x_{n-1})a_n
30 """
31 n = [Link][0] #number of data
32 #we use the y variable to inisialise the vector of coefficients
33 a = [Link]() #warning I need to copy to avoid overwrite the values of y
34 #we must compute n differences
35 for j in range(1,n): #start in 1 we alredy have the order zero differences
36 for i in range(j,n):
37 a[i] = (a[i]-y[i-1])/(x[i]-x[i-j])
38 y = [Link]()
39 return(a)
40

41 def evdif(a,x,xi):
42 """
43 Evaluates the divided diferences polynomial from points x and coefficients
44 y.
45

46 Parameters
47 ----------
48 y : TYPE double
49 DESCRIPTION. numpu arrray with divided differences polynomial coefficients
50 x : TYPE double
51 DESCRIPTION. numpy array with x data from the table to be interpolated
52 xi : TYPE double
53 DESCRIPTION. point to calculate the polynomial at. It can be also an array
54

55 Returns
56 -------
57 y1: TYPE: double
58 DESCRIPTION: computed value of the polynomial at x1
59 """
60 n = [Link][0]
61 yi = a[0] #copy the first coefficient into the result
62 for k in range(1,n):
63 #product of binomials to be multiplied for the coefficients
64 binprod = 1
65 for j in range(k):
66 binprod = binprod*(xi-x[j])
67 yi = yi +a[k]*binprod
 248 CHAPTER 8. INTERP. & FUNCT FITTING

68 return(yi)

8.2.4 The Newton-Gregory polynomial

This polynomial is a simplification of the divided difference polynomial for the case in which the
x data of the data set are equispaced and increasingly ordered.
For this case, it is much easier to compute the values of the differences. Think, for instance, in
the first-order differences, as far as the data are equispaced ll denominators are equal,

∆x ≡ xi − xi−1 = h
Concerning the numerators, they are computed as in the standard case of divided differences,

∆y0 = y1 − y0 , ∆y1 = y2 − y1 , · · · , ∆yi = yi+1 − yi , · · · , ∆yn−1 = yn − yn−1

Higher order differences can be computed recursively from the first-order differences because
the denominator of them all, h are equal.

∆2 y0 = ∆ (∆y0 ) = (y2 − y1 ) − (y1 − y0 ) = (y2 − 2y1 + y0 )

In this case, the difference denominator would be x2 − x0 = 2h, and the difference would take
the form,

∆2 y 0
f [x0 , x1 , x2 ] =
2h2
In genral, for the order-n differences we obtain,
   
n n
∆n y0 = yn − · yn−1 + · yn−2 − · · · + (−1)n · y0
1 2
Where we have used the binomial expression,
 
k k!
=
l l! · (k − l)!
And, eventually, we obtain the order-n divided difference just dividing by n! · hn .

∆n y0
f [x0 , x1 , · · · , xn ] =
n! · hn
Once we have got the differences we can write the divided differences polynomial as,

x − x0 (x − x1 ) · (x − x0 ) 2 (x − xn−1 ) · · · (x − x1 ) · (x − x0 ) n
pn (x) = y0 + ∆y0 + ∆ y0 + · · · + ∆ y0
h 2 · h2 n! · hn
This polynomial is known as the Newton-Gregory polynomial, and it could be considered as a
numerical approximation to the n-degree Taylor polynomial of the (possible) function associated
to the dataset.
In this case, we can build the table to obtain the polynomial coefficients, computing in each
column just the differences of the previous column. For example,
Once we have computed the differences, it is enough to divide by n! · hn the elements of the
table first row,
 8.2. POLYNOMIAL INTERPOLATION 249

Table 8.3: Table of differences for a Newton-Gregory polynomial of four data

xi yi ∆yi ∆2 yi ∆3 yi
x0 = 0 y0 = 1 −2 5 −7
x1 = 1 y1 = −1 3 −2
x2 = 2 y2 = 2 1
x3 = 3 y3 = 3

−2 5 −7
a0 = 1, a1 = , a2 = 2
, a3 =
1 2·1 6 · 13
The following code shows an example of Python implementation for the Newton-Gregory poly-
nomial

newton [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Thu Jul 25 20:12:49 2024
4 A gruby version of Newton-Gregory polynomial
5 @author: abierto
6 """
7 import numpy as np
8 from dif_div import evdif
9 def newgre(x,y,x1=0):
10 """
11

12

13 Parameters
14 ----------
15 x : TYPE numpy array of data
16 DESCRIPTION.
17 y : TYPE numpy array of data
18 DESCRIPTION.
19 x1 : TYPE a point to calculate the value the polynomial
20 takes
21 DESCRIPTION.
22 This function takes a data set, represented by the
23 x and y array and obtain the corresponding
24 newton-gregory [Link] it calculates the
25 value of the polypolimial at the point x1. if no x1
26 value is supplied then it takes x1 =0 by default.
27 The program returns the polynomial coeficients and
28 the value calculated at point x1
29

30 Returns
31 -------
32 a : TYPE numpy array of polynomial coeficients
33 DESCRIPTION.
34 y1: TYPE real value
35 DESCRIPTION. The value the polynomial takes at x1
36

37 """
 250 CHAPTER 8. INTERP. & FUNCT FITTING

38 n = [Link][0]
39 a = [Link]() #we start the coefficient with the 0-order differences,i.e.
40 #y values. remember use a copy to avoid overwrite the arrays
41 h = x[1] -x[0]
42 #here start the loop to calculate the differences
43 for j in range(1,n):
44 #for each iteration we calculate the differences
45 #of higher order. But we only need the first difference. So we restart
46 #the inner loop in the value j of the outer one
47 for i in range(j,n):
48 #now it is enough to divide for the distance h betwwen x point
49 #multiplied by order j of the diference
50 a[i] = (a[i]-y[i-1])/(j*h)
51 y = [Link]()
52 #now we calculate the value(s) of the polynomial using the
53 #function built to evaluate divided diferences polynomials
54 # if type(x1) != [Link]:
55 # x1 = [Link]([x1])
56 # y1 = []
57 # for i in x1:
58 # [Link](evdif(a,x,i))
59 # y1 = [Link](y1)
60 y1 = evdif(a,x,x1)
61 return(a,y1)

350
datos
polinomio de interpolación
300

250

200

150
y

100

50

−50
1 2 3 4 5 6 7 8 9 10
x

Figure 8.3: Nine degree interpoling polynomial obtained using a set of ten data
8.3. PIECEWISE INTERPOLATION 251

350 350
datos
datos
300 300 interpolación lineal
interpolación de orden cero

250 250

200 200

150 150
y

y
100 100

50 50

0 0

−50 −50
1 2 3 4 5 6 7 8 9 10 1 2 3 4 5 6 7 8 9 10
x x

(a) Zero-order interpolation (b) Linear interpolation

Figure 8.4: Zero-order and linear interpolation for the figure data 8.3

8.3 Piecewise interpolation

So far, we have seen how to interpolate a set of n + 1 data using a degree n polynomial. In many
case, in particular when the number of data is high, the results of such interpolation could be very
poor. The problem comes from the linear increasing of the polynomial degree with the number
of data. So, if we want to build the interpolation polynomial for a 11 data, we come up with a
10-degree polynomial. From a numerical point of view, this kinda polynomial are prone to cast
large errors due to the rounding process. On the other hand, depending on how data used to
compute the interpolation are distributed, the polynomial could take a too complicate shape that
hardly can be related with the information held in the dataset.
Figure 8.3 shows a interpolating polynomial of degree nine obtained from a set of ten data.
It is easy to realise, just for simple inspection, that there in not reason to justify the polynomial
curvature between points 1 and 2 or between the points 9 and 10.
For these reasons, in many cases it is better not to use the whole dataset to build a single
interpolation polynomial of maximum degree. Instead, a common practice is to divide the dataset
in several groups of data —usually they are gathered using interval of consecutive data– and build
several polynomial of lower degree, each one interpolating the data of a different group.
The degree of the polynomials should be related with the number of data allocated in each
interval.

Zero-order interpolation. If we get interval which only hold a single data pair X, y, then we get
zero-order interpolating polynomials, a0i = yi . The result is a stepwise interpolation which values
changes from one interval to the next, taken the data value defined for each interval according to
the dataset. Figure 8.4a shows the zero-order interpolation result for the same ten data of figure
8.3.

Linear interpolation. In this case, we divide the dataset in groups of two data. Each two
consecutive data are interpolated calculating the line that pass through them. The linear inter-
polation is very commonly used due to its computing simplicity. Figure 8.4b shows the result of
interpolating the same data utilised in previous examples, using linear interpolation.
Following the same procedure, we can increase the number of data include in each interval and
define quadratic interpolation, using second-degree polynomials and taking three points in each
252 CHAPTER 8. INTERP. & FUNCT FITTING

interval; cubic interpolation,using third-degree polynomial and four point intervals. etc.

8.3.1 Cubic spline interpolation

WE have seen how the n-degree interpolating polynomial for a set of n + 1 data, could present a
quite complex shape that does not represent well the information supplied by the dataset. The
piecewise interpolation that we have described so far, simplifies the shape of the interpolating curve
but have in turn the problem that it loses the continuity in the union between consecutive intervals.
It could be valuable to find interpolation methods that may be able to cope with both problems
simultaneously. A good approach to solve these problem is supplied by spline interpolation.
A spline is function built using a set of polynomials, any one of them defined in an interval. The
spline polynomial are connected in the ends of the intervals meeting certain continuity conditions.
Suposse we have a data table whatsoever,

x x0 x1 ··· xn
y y0 y1 ··· yn

To build a m order spline function S for interpolating the table data, we define intervals taking
two consecutive point onn the table as limits. Then, we define an m degree polynomial for each
interval.


 S0 (x), x ∈ [x0 , x1 ]

S1 (x), x ∈ [x1 , x2 ]






 ..
.


S=
 Si (x), x ∈ [xi , xi+1 ]


..



.





Sn−1 (x), x ∈ [xn−1 , xn ]


For s to be a order m spline function it should be continue and it should have m − 1 continue
derivatives in the interval [x0 , xn ] in which we want to interpolate the data.
To ensure the continuity of the polynomials that build S they must satisfy the following con-
dition at their ends,

Si (xi+1 ) = Si+1 (xi+1 ), (1 ≤ i ≤ n − 1)

Si′ (xi+1 ) = Si+1
′
(xi+1 ), (1 ≤ i ≤ n − 1)
Si′′ (xi+1 ) = Si+1
′′
(xi+1 ), (1 ≤ i ≤ n − 1)
..
.
Sim−1 (xi+1 ) = Si+1
m−1
(xi+1 ), (1 ≤ i ≤ n − 1)

That is, two consecutive polynomial belonging to the spline and their m − 1 first derivatives
must take the same values in the common end.
A straightforward consequence of the continuity conditions impose to spline functions is that
their successive derivatives S ′ . S ′′ , · · · are in turn order m − 1, m − 2 · · · spline functions too.
Besides, the continuity conditions supply (n − 1) · m equations that, together with the interpolation
8.3. PIECEWISE INTERPOLATION 253

conditions —each polynomial should pass through the two data point that defines the ends of its
definition interval—, sum up n · (m + 1) − (m − 1) equations. This number is insufficient for
obtaining the (m + 1 · n) parameters belonging to the the n polynomial of degree m used in the
spline interpolation. We need m − 1 equations more that are defined imposing to the polynomials
additional conditions.

Cubic Spline. Probably, the most used splines are those composed of third-degree polynomials.
We must need to determine (m + 1)ṅ = 4 · n coefficients to obtain all the polynomials that compose
the spline. The continuity plus the interpolation conditions supply 3 · (n − 1) + n + 1 = 4 · n − 2
equations. We need to impose two more conditions to the spline. Some frecuently used conditions
are,

1. natural Spline S ′′ (x0 ) = S ′′ (xn ) = 0

2. Spline with a known value for the derivatives at the ends of the interval S ′ (x0 ) = y0′ , S ′ (xn ) =
yn′

3. Periodic Spline,


 S(x0 ) = S(xn )

S ′ (x0 ) = S ′ (xn )
 ′′
S (x0 ) = S ′′ (xn )


Try to build a system of equations to obtain all the coefficients of all the polynomial at a time,
is an arduous task. There are too many parameters. We can address the problem starting with
S ′′ (x), which it is also a order-1 spline for the points we want to interpolate. If we define it as,

x − xi+1 x − xi
Si′′ (x) = −Mi + Mi+1 , i = 0, · · · , n − 1
hi hi
Where hi = xi+1 − xi stand for each interval width and where the value Mi = S ′′ (xi ) will be
the unknown we have to solve.
Now, we integrate two times this expression, Si′′ (x), to obtain,

(x − xi+1 )2 (x − xi )2
Si′ (x) = −Mi + Mi+1 + Ai , i = 0, · · · , n − 1
2 · hi 2 · hi
(x − xi+1 )3 (x − xi )3
Si (x) = −Mi + Mi+1 + Ai (x − xi ) + Bi , i = 0, · · · , n − 1
6 · hi 6 · hi

Let’s start imposing the interpolation conditions: the polynomial Si must to pass through the
point (xi , yi ),

(xi − xi+1 )3 Mi · h2i

Si (xi ) = −Mi + Bi = yi ⇒ Bi = yi − , i = 0, · · · , n − 1
6 · hi 6
Then, we impose the continuity of the spline in the points common to two polynomials: Poly-
nomial Si−1 must also pass through the point (xi , yi ),
254 CHAPTER 8. INTERP. & FUNCT FITTING

Bi−1
z }| {
(xi − xi−1 )3 Mi−1 · h2i−1
Si−1 (xi ) = Mi + Ai−1 (xi − xi−1 ) + yi−1 − = yi ⇒
6 · hi 6
yi − yi−1 Mi − Mi−1
⇒ Ai−1 = − · hi−1 , i = 1, · · · , n
hi−1 6
and thus,
yi+1 − yi Mi+1 − Mi
Ai = − · hi , i = 0, · · · , n − 1
hi 6
Thirdly, we impose that continuity condition to the derivatives, in the common end of two
consecutive polynomials,

(xi − xi+1 )2 (xi − xi )2 yi+1 − yi Mi+1 − Mi

Si′ (xi ) = −Mi + Mi+1 + − · hi , i = 0, · · · , n − 1
2 · hi 2 · hi hi 6
′ (xi − xi )2 (xi − xi−1 )2 yi − yi−1 Mi − Mi−1
Si−1 (xi ) = −Mi−1 + Mi + − · hi−1 , i = 1, · · · , n
2 · hi−1 2 · hi−1 hi−1 6
Si′ (xi ) = Si−1
′
(xi ), i = 1, · · · , n − 1 ⇒
hi yi+1 − yi Mi+1 − Mi hi−1 yi − yi−1 Mi − Mi−1
⇒ −Mi + − · hi = Mi + − · hi−1
2 hi 6 2 hi−1 6
If we group at one side the values Mi−1 , Mi , Mi+1 ,

 
yi+1 − yi yi − yi−1
hi−1 · Mi−1 + 2 · (hi−1 + hi ) · Mi + hi · Mi+1 = 6 · −
hi hi−1
i = 1, · · · , n − 1

We have in total M0 , · · · , Mn , n + 1 unknowns and the above expression only supply n − 1

equations. We need two more equations. If we impose the conditions for a natural spline for the
left end of the first polynomial and for the right end of the last one,

M0 = S ′′ (x0 ) = 0
Mn = S ′′ (xn ) = 0

With these last conditions and the expression obtained for the remaining Mi , we can build a
tridiagonal system of equations

 
2(h0 + h1 ) h1 0 0 ··· 0 0    
M1 b1

 h 1 2(h1 + h2 ) h 2 0 ··· 0 0 
  M2   b2 
 0 h 2 2(h2 + h3 ) h 3 ··· 0 0   
  M3   b3 

.. .. .. .. .. .. · =

 ..   ..   .. 


 . . . . . . .  .   . 
 0 0 0 0 ··· 2(hn−3 + hn−2 ) hn−2 
Mn−1 bn−1
0 0 0 0 ··· hn−2 2(hn−2 + hn−1 )

Where,
 8.3. PIECEWISE INTERPOLATION 255

 
yi+1 − yi yi − yi−1
bi = 6 · −
hi hi−1
So, we have a system of equations for which the coefficients matrix is tridiagonal and, besides,
it is a dominant diagonal matrix. For this reason, we can solve it with anyone of the methods
described in chapter 7. Once the system is solved, we get the values of Mi and the values of Ai
and Bi using the equations described above.
A last remark: we usually define the 3-degree polynomial, Si used for interpolating the values
inside the interval [xi , xi+1 ] as follows,

Si (x) = αi + βi (x − xi ) + γi (x − xi )2 + δi (x − xi )3 , x ∈ [xi , xi+1 ], (i = 0, 1, · · · , n − 1)

Where,

αi = yi
yi+1 − yi Mi · hi Mi+1 · hi
βi = − −
hi 3 6
Mi
γi =
2
Mi+1 − Mi
δi =
6 · hi
The following Python code computes the coefficients of the cubic spline defined using a dataset
and the value taken by the spline at any point.

cubic [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Wed Jul 31 11:01:10 2024
4 This script implemets some functions to compute Cubic spline
5 @author: abierto
6 """
7 import numpy as np
8

9 def spcubic(x,y):
10 """
11 Function to calculate the spline coefficients
12 Parameters
13 ----------
14 x : TYPE real np array
15 DESCRIPTION. Table Values x to be interpolated
16 y : TYPE real np array
17 DESCRIPTION. Table values y to be interpolated
18 Returns
19 -------
20 h: Type array of diferecences betwee x consecutive data
21 M,A,B: arrays with coefficientes for the polinomial that
22 that compose the spline expressed as differences
23 (see manual)
24 C : TYPE numpy array size [I,4]
 256 CHAPTER 8. INTERP. & FUNCT FITTING

25 I->intervals i.e number of data - 1

26 4->Each 3-degree polinomial has 4 coefficients
27 [Link] coeficients in standard polynomial representation
28 """
29 l = [Link][0] #number of data
30 dy = [Link](l-1)
31 h = [Link]()
32

33 for i in range(0,l-1):
34 h[i] = x[i+1] - x[i]
35 dy[i] = y[i+1]-y[i]
36 CSPp = [Link](l-2)
37 b = [Link]()
38 for i in range(l-2):
39 CSPp[i] = 2*(h[i+1]+h[i]) #ppal diagonal
40 b[i] = 6*(dy[i+1]/h[i+1]-dy[i]/h[i])
41 #building the system matrix (a pro. would use a spare matrix here)
42 CSP = [Link](CSPp)+[Link](h[1:-1],1)+[Link](h[1:-1],-1)
43 M = [Link](CSP,b)
44 M = [Link](M,0,0)
45 M= [Link](M,0)
46 A = [Link](l-1)
47 B = [Link](l-1)
48 for i in range(l-1):
49 A[i] = dy[i]/h[i] - (M[i+1]-M[i])*h[i]/6
50 B[i]=y[i]-M[i]*h[i]**2/6
51 C = [Link]([l-1,4])
52 for i in range(l-1):
53 C[i,0]=y[i];
54 C[i,1]=dy[i]/h[i]-M[i]*h[i]/3-M[i+1]*h[i]/6;
55 C[i,2]=M[i]/2;
56 C[i,3]=(M[i+1]-M[i])/(6*h[i]);
57 return M,A,B,C,h
58

59 def evspc(M,A,B,x,xi):
60 """
61 This function calculates the value of a interpolating spline in point xi
62

63 Parameters
64 ----------
65 M : TYPE nummpy array
66 DESCRIPTION. Coefficients M of the spline
67 A : TYPE numpy array cofficients A of the spline
68 DESCRIPTION.
69 B : TYPE
70 DESCRIPTION.
71 x : TYPE numpy array
72 DESCRIPTION. x data of the datatable interpolated
73 xi : TYPE double
74 DESCRIPTION. point to calculate the spline value
75

76 Returns
 8.3. PIECEWISE INTERPOLATION 257

77 -------
78 yi : TYPE double
79 DESCRIPTION. value calculate for the spline yi =s(xi)
80

81 """
82 l = [Link][0]
83 h = [Link](l-1)
84 for i in range(0,l-1):
85 h[i] = x[i+1] - x[i]
86 j = 0
87 while xi > x[j]:
88 j = j+1
89 if j > l - 1:
90 j = l - 1
91 elif j < 1:
92 j = 1
93 yi=-M[j-1]*(xi-x[j])**3/(6*h[j-1])+ M[j]*(xi-x[j-1])**3/(6*h[j-1])\
94 +A[j-1]*(xi-x[j-1])+B[j-1]
95 return yi

Figure 8.5 shows the result of interpolating the same data of figure 8.3 using a cubic spline. It
is ease to check how the interpolation polynomials which compose the spline cast a smooth result
in the points interpolated and how the curves shapes are reasonable smooth also in the intervals
between the data. They do not present variations difficult to explain using the data.

8.3.2 Python functions for piecewise interpolation

To carry out a piecewise interpolation using anyone of the method above described, we can use
the scipy sub-package [Link]. This package includes many intepolation methods;
we will only describe two of them:
The interp1d function. This function takes two arrays as inpunt variables with the ccordenates
x and y values belonging to the data we want to interpolate. In addition, the function may take
also a character string, as an input variable which defines the method we want to use to carry out
the interpolation. This last variable can take the following values:
1. ’nearest’. It interpolates the interval using the value yi corresponding to the value xi
nearest to the point at which we want to compute the interpolation. The result is stepwise
interpolation.
2. ’next’. It interpolates the interval using the value yi corresponding to the value xi which
closes the interval in which we want to compute the interpolation.
3. ’previous’ It interpolates the interval using the value yi corresponding to the value xi which
opens the interval in which we want to compute the interpolation.
4. ’linear’ It calculates a linear interpolation between the points at the ends of the interval
we want to interpolate. This is the default option.
interp1d returns a function that, in turn, takes as input variable an array with the points
we want to calculate the interpolation at, and returns an array we the computed results. The
following code shows how to use the function interp1d. To try it we have created two arrays x
and y which contain the dataset we want to interpolate. Besidas, we have created another array
xi with the point we want to compute the interpolation at.
 258 CHAPTER 8. INTERP. & FUNCT FITTING

350

datos
300 interp. splines cúbicos

250

200

150
y

100

50

−50
1 2 3 4 5 6 7 8 9 10
x

Figure 8.5: Cubic Spline interpolation for the data represented in the figure
8.3

ejemplo [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Fri Aug 2 16:55:04 2024
4 Ejemplo de uso de la función interp1d de Scipy
5 @author: abierto
6 """
7 import numpy as np
8 from [Link] import interp1d
9 from matplotlib import pyplot as pl
10 x = [Link]([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10.])
11 y = [Link]([-1.8143451, 20.914356, 26.714303, 61.129501,\
12 76.414728, 123.00032, 167.06809, 212.97832,\
13 258.67911, 320.53422])
14 #dibujamos los puntos
15 [Link](x,y,'o')
16 #creamos unos puntos sobre los que interpolar
17 xi = [Link](1.,10.,0.01)
18 #creamos la función interpoladora empleando inter1d
19 flin = interp1d(x,y,'linear')#puede omitirse linear el la opcion por defecto
20 #empleamos la funcion para obtener los valores de los puntos interpolados
21 yi = flin(xi)
22 [Link](xi,yi)
23

24 fnear = interp1d(x,y,'nearest')
 8.4. LEAST SQUARED ERROR METHOD FOR POLYNOMIAL DATA FITTING 259

25 yi = fnear(xi)
26 [Link](xi,yi)
27

28 fnext = interp1d(x,y,'next')
29 yi = fnext(xi)
30 [Link](xi,yi)
31 #and so on and so forth...

The second interpolating function, from

[Link] we are going to describe is CubicSpline. This function, as can be expected,
uses cubic splines to perform the interpolation of a dataset. In this cases, in addition to the x and
y data needed to define the spline, the function defines an input paramenter bc_type, which allows
us to stablish the boundary conditions in the ends of the spline. Some of the available options are:
bc_type='not__knot'. This is the default option. The first and second segments at an end of
the spline uses the same polynomial.
bc_type='natural'. Natural spline.
bc_type='periodic'. Periodic spline.
There are more options. Those interested can find them in scipy documentation. Its use is
similar to the use of interp1d. From the data we want to interpolate CubiSpline generates an
interpoling function which alows us to compute the spline value in the point we wish. For instance,
for natural splines we write,

spnat = CubicSpline(x,y,'natural')
yi = spnat(xi)

8.4 Least squared error method for polynomial data fitting

The interpolation methods we have described so far, try to find a polynomial of a function defined
using polynomials that passes through a set of data. In the case of Least squared error fitting, the
aim is to find the polynomial of a specific degree that better approximates a dataset.
Suppose we have a dataset holding m data,

x x1 x2 ··· xm
y y1 y2 ··· ym

We want to build a polynomial p(x) with degree n < m − 1 so that the values it takes for the
data p(xi ) are the nearest possible to the values yi of the data table.
First, we must clarify what means the nearest possible. One option is to measure the difference
yi − p(xi ) for each pair of data in the dataset. However, using the square of such difference
2
(yi − p(xi )) is more common. This quantity has the advantage, among others, of being always
positive, no matter whether the difference is positive or negative. Besides, it represents the square
distance between p(xi ) and yi . We can take the sum of these differences for all pairs in the dataset,
m
X 2
(yi − p(xi ))
i=1

as a measure of the distance from the polynomial to the data. In this way, the nearest possible
to the data would be the polynomial that minimised the sum of square differences we have just
defined. Indeed, this is the origin of the method’s name.
260 CHAPTER 8. INTERP. & FUNCT FITTING

The data we often want to fit a polynomial using the least square method are experimental
data. Depending on the environment and the method used for data acquisition, some data may
be more reliable than others. In this case, it is interesting that the polynomial passes closest to
the more reliable data. One way to achieve it is to add some weights, ωi to the square differences
according to the confidence that each data deserves,
m
X 2
ωi (yi − p(xi ))
i=1

We will multiply more reliable data by large values of ω and the less reliable by small values.
To see how to obtain the coefficients of a least square polynomial, we will start with the simplest
case; a 0-degree polynomial. In this case the polynomial is a simple constant value, defined by its
constant term p(x) = a0 . Then the fucntion to be minimised would be,
m
X 2
g(a0 ) = ωi (yi − a0 )
i=1

The minimum of this function must meet that its first derivative g ′ (a0 ) = 0 and its second
derivative g ′′ (a0 ) ≥ 0,

m Pm
i=1 ωi · yi
X
′
g (a0 ) = −2 ωi (yi − a0 ) = 0 ⇒ a0 = P m
i=1 i=1 ωi
m
X
g ′′ (a0 ) = 2 ωi ⇒ g ′′ (a0 ) ≥ 0
i=1

We obtain that the value for a0 is a mean of the data, weighted by the values ωi assigned to the
data. If we take wi = 1 ∀wi , we obtain the mean value of the data. This result is quite reasonable.
Approximating a set of values with a 0-degree polynomial is equivalent to considering that variable
y remains constant for every value of x. In this case, we may attribute the observed differences
among the values of the table to experimental random errors. Consequently, the best estimation
of y would be the mean value of the available data.
The nest step would be to approximate the data using a 1-degree polynomial, i.e., a straight
line, p(x) = a0 + a1 x. Inn this case, the sum of squared differences takes the form,
m
X 2
g(a0 , a1 ) = ωi (yi − a0 − a1 xi )
i=1

We have now two coefficient for computing the minimum. We get the minimum making the
function g(a0 , a1 ) partial derivatives equal to zero.

m
∂g X
= −2 ωi (yi − a0 − a1 xi ) = 0
∂a0 i=1
m
∂g X
= −2 ωi xi (yi − a0 − a1 xi ) = 0
∂a1 i=1

And after rearranging the previous equations,

8.4. LEAST SQUARED ERROR METHOD FOR POLYNOMIAL DATA FITTING 261

4
datos
a0=3.014
3.5

2.5

2
y

1.5

0.5

0
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1
x

Figure 8.6: 0-degree least squared error polynomial

m
! m
! m
X X X
ωi a0 + ωi xi a1 = ωi yi
i=1 i=1 i=1
m
! m
! m
X X X
ωi x i a0 + ωi x2i a1 = ωi x i yi
i=1 i=1 i=1

We get a linear system with two equations and two unknowns, which are the coefficients of the
least squares line.
We can now generalised this result for a n-degree polynomial p(x) = a0 +a1 x+a2 x2 +· · ·+an xn .
Function g would be now,
m
X 2
g(a0 , a1 , · · · , an ) = ωi (a0 + a1 xi + · · · + an xni − yi )
i=1
Again, we can get the polynomial coefficents making the partial derivatives equal to zero,

m
∂g(a0 , a1 , · · · , an ) X
=0⇒ ωi xji (a0 + a1 xi + · · · + an xni − yi ) = 0, j = 0, 1 · · · , n
∂aj i=1

If we rearrange the previous equation, we arrive to a linear system of n + 1 equations whose

unknowns are the coefficients of the least squares polynomial,
     
s0 s1 · · · sn a0 c0
 s1 s · · · s a
2 n+1   1   c1 
    
 .. .. · . = . 

. . . .   ..   .. 

.
sn sn+1 · · · s2n an cn
 262 CHAPTER 8. INTERP. & FUNCT FITTING

Where we have defined sj and cj as,

m
X
sj = ωi xji
i=1
m
X
cj = ωi xji yi
i=1

h following code calculates the coefficients of the least squares polynomial that approximate a
set of n data.

minimos [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Sat Aug 3 20:15:18 2024
4 Function to obtain the least squared error polynomial from a dataset
5 @author: abierto
6 """
7 import numpy as np
8 def lse(x,y,n,w='none'):
9 """
10 This function compute the least squared error polynomial of
11 degree n from a set of data x,y. Optionally it can use a
12 set of weights to perform the computation
13 Parameters
14 ----------
15 x : TYPE numpy array
16 DESCRIPTION. x data to be fitted
17 y : TYPE numpy array
18 DESCRIPTION. y data to be fitted
19 n : TYPE integer
20 DESCRIPTION. Polynomial degree
21 w : TYPE numpy array with the same length than x and y
22 DESCRIPTION.
23

24 Returns
25 -------
26 a: TYPE numpy array
27 DESCRITION. polynomial coefficients in increasing powers
28 a[0]+a[1]x+a[2]x**2+...+a[n]x**n
29

30 """
31 m = [Link][0]
32 #first we check that we have enought points
33 if m < n:
34 raise Warning('the degree is greater than the number of data')
35 #if there is no weight array generate an arry of ones
36 if type(w)==str:
37 w = [Link](m)
38 #we build the s elements of the system coefficient matrix
39 n = n+1 # for degree n I need n+1 coeficientes
40 s = [Link](2*n)
 8.4. LEAST SQUARED ERROR METHOD FOR POLYNOMIAL DATA FITTING 263

41 for j in range(2*n):
42 for i in range(m):
43 s[j] = s[j] + w[i]*x[i]**j
44 c = [Link](n)
45 for j in range(n):
46 for i in range(m):
47 c[j] = c[j] + w[i]*x[i]**j*y[i]
48 A = [Link]([n,n])
49 for i in range(n):
50 for j in range(n):
51 A[i,j] = s[i+j]
52 a =[Link](A,c)
53 return(a)
54

A last important remark: If we try to compute the least squared error polynomial of degree
m − 1 to approximate a set of m data, we will get the interpolating polynomial. In general, if we
increase the degree of the least squared polynomial, we increase the probability that the system
matrix we use to obtain the polynomial coefficients will be poorly conditioned.

8.4.1 Least squares in Python.

There are several ways to compute the Least squares polynomial for a set of data using Python.
We will describe only one of them. The numpy sub-package [Link] includes a class
called Polynomial (Mean the first capital letter!) allows us, among many other things, to create
polynomial as programming objects. One way to create a polynomial is to compute the least squares
polynomial from a dataset. The class Polynomial has a special function fit to accomplish it. This
command takes as inputs two arrays of data, one with the coordinates x and the other with the
coordinates y of the value we want to interpolate, plus a third variable for the polynomial degree.
Optionally, we can add an array of weights to balance the importance we give to each data.
Function [Link] returns a polynomial ’object’. This object contains all the informa-
tion on the polynomial it represents and it can be use straightforwardly as a function to compute
the value of the polynomial in any point. The following Python’s script generates a pair of vectors
and shows how to deal with the fit command.

ejemplo [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Sun Aug 4 16:00:12 2024
4 Example of least squares fit using the [Link] class Polynomial
5 @author: abierto
6 """
7 import numpy as np
8 from [Link] import Polynomial as py
9 from matplotlib import pyplot as pl
10 #we import the function included above to compare the result
11 #with the [Link] function
12 from minimos_cuadrados import lse
13

14 #define a data set to play with

15 x = [Link]([ 1., 2., 3., 4., 5., 6., 7., 8., 9., 10.])
 264 CHAPTER 8. INTERP. & FUNCT FITTING

16 y = [Link]([-1.8143451, 20.914356, 26.714303, 61.129501,\

17 350.414728, 123.00032, 167.06809, 212.97832,\
18 258.67911, 320.53422])
19

20 p3 = [Link](x,y,3) #we are not including weight

21 #we get the polynomial coefficients just to see them
22 a = [Link]().coef
23

24 #a set of point to evaluate the polynomial

25 xi = [Link](1.,10.,0.01)
26

27 yi = p3(xi)
28

29 #point y[4] (see graphics bellow) apears to be an outlayer we can

30 #use weights to atenuate its influence
31 w = [Link]([Link][0])
32 w[4] = 0.5 #we reduce the weight of the outlayer
33 p3w = [Link](x,y,3,w=w)
34 yiw = p3w(xi)
35

36 ################################################################
37 #we repeate the calculation using our own funtion lse
38 ################################################################
39 an = lse(x,y,3)
40 p = py(an) #we create a polynomial object using the coeficients
41 yin = p(xi)
42 #repeat the computing using the weight
43 aw = lse(x,y,3,w)
44 pw = py(aw)
45 yinw = pw(xi)
46

47 #drawing the results to compare

48 ax1 = [Link](2,1,1)
49 [Link](x,y,'o')
50 [Link](xi,yi)
51 [Link](xi,yiw)
52 [Link](['Data','[Link]',\
53 '[Link] (weights)'])
54 ax2 = [Link](2,1,2)
55 [Link](x,y,'o')
56 [Link](xi,yin)
57 [Link](xi,yinw)
58 [Link](['Data','lse','lse (weights)'])

We are going to review some important points of the previous code. In line 8, we import the
object Polynomial. We do it in the same way that we import whatever numpy submodule. We
have assigned to Polynomial the alias py. In line 12 we import the function lse. This is the
function we previously created to perform least squares fits and whose code we included above. In
lines 15 and 16 we create a dataset.
In line 20, we create a 3-degree least squares polynomial, p3, which fits the previous data. p3
holds all the information on the polynomial we have created. In line 22, we use the command
convert to obtain an array with the polynomial coefficients, ordered in ascending degree. p(x) =
8.4. LEAST SQUARED ERROR METHOD FOR POLYNOMIAL DATA FITTING 265

a0 + a1 x + a2 x2 + · · · + an xn . In line 27, we use p3 straightforwardly, as a function, to obtain the

values the polynomial takes at the points held by the array x_i.
We repeat the same computation in lines 29-34 but now use weights. We assign a value of 1 to
every weight except for the fifth data point, to which we assign a smaller weight.
In lines 39 to 45 we repeat the the same computation but this time using our own function,
lse
Lastly, we draw the results to compare, as can be seen in figure 8.7.

Data
300 [Link]
[Link] (weights)
200

100

0
2 4 6 8 10

Data
300 lse
lse (weights)
200

100

0
2 4 6 8 10

Figure 8.7: An example of least squares polynomial fit, using the function fit from Polynomial and
our function lse.

The figure clearly shows the effect the weights have on the regression results. When we attenuate
the influence of the fifth point, which appears to be quite an outlier, the curve approaches more
to the remaining points.
If we look at both graphics attentively, we will see that the Python function approximates
the polynomial more to the points than the function we have written. The difference comes from
how we introduce the weights, which are not the same as those used by Python. Function fit,
applies the weights directly to the difference between the data an the value taken by the polynomial
ωi |yi − p(xi )|, whike our function applies the weights to the squared difference ωi (yi − p(xi ))2 . If
we compare the coefficients of the polynomials computed by both methods,

In [47]: [Link]().coef
Out[47]: array([-40.71774313, 33.22583298, -1.61930914, 0.1854822 ])

In [48]: aw
Out[48]: array([-68.26797522, 56.50545799, -5.31193931, 0.34603134])
266 CHAPTER 8. INTERP. & FUNCT FITTING

We can see they are different. To get the same result, we must introduce the squared values of
our weight when we use our function,

In [49]: aw2 = lse(x,y,3,w**2)

In [50]: aw2
Out[50]: array([-40.71774313, 33.22583298, -1.61930914, 0.1854822 ])

This result teaches us a fundamental lesson: never use a numerical computing software package
without understanding its operations.

8.4.2 Analyzing goodness of fit using least squares method.

Suppose we have a dataset we have obtained as a result of an experiment. In many cases, least
squares data fitting aims to find a law that allows us to establish a mathematical relationship
between the independent and dependent variables. For instance, if we apply different stresses to a
spring and measure the elongation acquired by the spring, we expect to get a linear relationship
between stress and elongation as a first approximation: ∆x ∝ F . (Hooke’s law).
However, the results of an experiment hardly ever fit precisely into any law due to random
errors, which we cannot amend.
When we carry out a least squares fitting, we may use any polynomial of degrees 0 to m−1. From
the point of view of the error we make, considering only the available data, the best polynomial
would be the m − 1-degree polynomial because this is the interpolation polynomial, and, so, it
makes a zero error for every data. Nevertheless, if we are trying to fit experimental data, we are
including the experimental errors in the fitting.
As a consequence, for experimental data and assuming that the data only are affected by random
errors, we get the best fitting using the polynomial of lest degree for which the differences between
data and polynomial value yi − p(xi ) are randomly distributed. Such differences are usually called
residuals.
Figure 8.8 shows a least square fitting example, using polynomials of increasing degrees.
The residual also shows the data shape; they are not randomly distributed. In figure 8.8b, we
can see how a parabola better approximates the data but cannot follow the data up and down.
The residuals clearly show a smooth maximum and minimum, but they are not actually random.
In figure 8.8c, the polynomial fits better the data, and the residuals are randomly distributed. If
we compare them with the residuals presented in figure 8.8d, we see these last are smaller but
present essentially the same random distribution. We may conclude that the data approximation
using a fourth-degree polynomial does not add any useful information on the function shape the
data follow, and, worse than this, we are starting to include the data errors in the fitting.

∞ (
Ξi (t)
i=0

8.5 Bézier Curves

Bezier’s curves can connect points using curves with arbitrary shape. We define a Bezier’s curve
using a set of of n + 1 points, {⃗
p0 , p⃗2 , · · · , p⃗n } which are known as control points. The order of
the control points play a key role in the definition of the corresponding Bézier’s curve. The curve
pass through the end points p⃗0 and p⃗n . The remaining control points allow us to define the curve
8.5. BÉZIER CURVES 267

(a) Least squares line and residuals. (b) Least squares parabola and residuals.

(c) 3-degree polynomial and residuals (d) 4-degree polynomial and residuals

Figure 8.8: Comparison among the residuals resulting from a dataset least squares fitting, using
polynomials of degrees 1 to 4.
268 CHAPTER 8. INTERP. & FUNCT FITTING

shape, which has the interesting properties of being always tangent in point p⃗0 to the line that
connects p⃗0 with p⃗1 and tangent in point p⃗n to the line that connect p⃗n−1 with p⃗n .
To define the curve, we associate to the control point p⃗i , a function known as fusion function.
In the case of Bezier’s curves, we use as fusion function Berstein’s polynomials,
 
n n−i i
Bin (t) = (1 − t) t, i = 0, 1, . . . , n
i
The Berstein’s polynomials degree we use depends on the number of control points; for a set of
n + 1, we need polynomials of degree n. Variable t is a parameter which varies between 0 and 1.
The Bezier’s curve for a set of n + 1 control points, {⃗ p0 , p⃗2 , · · · , p⃗n }, can be defined using
Berstein’s polynomials, according to the following equation,
n n  
X X n n−i i
p⃗(t) = Bin (t) · p⃗i = (1 − t) t · p⃗i
i=0 i=0
i

The above expression yields p⃗0 as a result, if we take t = 0 and p⃗n for t = 1. For values of t
between t = 0 and t = 1, points p⃗(t) will describe a curve between p⃗0 y p⃗n .
Let’s see a basic example. Suppose we want to connect point p⃗0 = (0, 1) with point p⃗1 using a
Bezier’s curve. If we do not define any other control point, {⃗p0 = (0, 1), p⃗1 = (3, 1)}, The Bezier’s
curve we will obtain will be a line connection both points.
   
1 1 0 1 0
p⃗(t) = (1 − t) t · (0, 1) + (1 − t) t1 · (3, 1) = (1 − t) · (0, 1) + t · (3, 1)
0 1
If we split the components x and y of vector p⃗(t),

x = 3t
y=1

It is the equation of the horizontal segment that joints the points p⃗0 = (0, 1) y p⃗n = (3, 1)
If we add a control point more, for instance: p⃗1 = (1, 2) → {⃗
p0 = (0, 1), p⃗1 = (1, 2), p⃗2 = (3, 1)},
the resulting curve will be now a segment of a second degree polynomial on the variable t,

     
2 2 0 2 2 0
p⃗(t) = (1 − t) t · (0, 1) + (1 − t) t · (1, 2) + (1 − t) t2 · (3, 1) =
0 1 2
2
= (1 − t) · (0, 1) + 2 (1 − t) t · (1, 2) + t2 · (3, 1)

As we increase the number of control points, we also increase de degree of the polynomial used
to connect the first and last control points. The shape of this polynomial will change depending
on the intermediate control point we use.
Figure 8.9 shows an example in which we have built several Bézier’s curves using the same
initial and final controls points. Notice how the shape of the curve depends on the number and
position of the control points. If we orderly connect the control points using straight segments we
get a polygon know as control polygon. In figure 8.9 the control polygons have been represented
using dashed lines.
The following code includes functions bezier and bezier_sc to compute and draw Bezier’s
curves define from their control points.
 8.5. BÉZIER CURVES 269

Puntos de control
2.5 Curva de Bezier (2 puntos)
Puntos de control
Curva de Bezier (3 puntos)
Puntos de control
Curva de Bezier (4 puntos)
Puntos de control
Curva de Bezier (5 puntos)

1.5

0.5

0
0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

Figure 8.9: Bézier curves traced between points P0 = (0, 0) y Pn = (2, 1), variating the control
points number and positions

[Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Thu Aug 8 19:41:24 2024
4 This module defines a functions to generate Bezier Polynomials
5 bezier returns the value of the curve as a function of t
6 bezier_rc uses function bezier to obtain the values of the curve
7 y an array of point a draw the result
8 @author: abierto
9 """
10 import numpy as np
11 import [Link] as pl
12

13

14 def bezier(p,t):
15 """
16 Calculates de value of a bezier curve with control points
17 in the array p at value t
18

19 Parameters
20 ----------
21 p : TYPE numpy array
22 DESCRIPTION. Numpy array of dimension 2Xn where n is the
23 number of control points. first row contains x coordinate
24 and second row contain y coordinate, the curve pass through
 270 CHAPTER 8. INTERP. & FUNCT FITTING

25 the first a last control points

26 t : TYPE
27 DESCRIPTION. Berstein's polynomial parameter t belongs
28 to [0,1]
29

30 Returns
31 -------
32 ber : TYPE [Link]
33 DESCRIPTION. ber[0] x coordinate of the bezier's curve
34 at t. ber[1] y coordinate of the bezier's curve at t
35

36 """
37 n = [Link][1]
38 num = [Link](1,n).prod()
39 #we firt calculate all factorial needed
40 ftal = [Link]([Link](1.,n).cumprod(),0,1)
41 ber = [Link](2)
42 for i in range(n):
43 f = num/ftal[i]/ftal[n-1-i]
44 ber = ber + f*p[:,i]*(1-t)**(n-1-i)*t**i
45 return(ber)
46

47 def bezier_sc(p,t):
48 """
49 This function use funtion bezier to reproduce the bezier curve
50 with control point containing in p.
51 Parameters
52 ----------
53 p : TYPE numpy array
54 DESCRIPTION. Numpy array of dimension 2Xn where n is the
55 number of control points. first row contains x coordinate
56 and second row contain y coordinate, the curve pass through
57 the first a last control points
58 t : TYPE numpy array
59 DESCRIPTION. It should be an array of ordered values for
60 parameter t. To reproduce de whole Bezier's curve use
61 t = [Link](0,1+step,step) the smaller step the smoother
62 the result
63

64 Returns
65 -------
66 pt : TYPE numpy array
67 DESCRIPTION. a 2xm array with m the number of data in t
68 first row containts the x coordinates at the values of t
69 and second row the y coordinates at the values of t
70

71 """
72 pt = [Link]([bezier(p,i) for i in t]).T
73 [Link](p[0],p[1],'or')
74 [Link](p[0],p[1],'-.')
75 [Link](pt[0],pt[1])
76 return(pt)
 8.5. BÉZIER CURVES 271

77

78 def bzeq(p,n,step):
79 """
80 Computes Bezier's curves of higher degree, equivalent to a
81 given Bezier's curve
82

83 Parameters
84 ----------
85 p : TYPE numpy array
86 DESCRIPTION. 2xn array containing the control points of
87 the Bezier's curve to be derivate. (each column a point)
88 n : TYPE integer
89 DESCRIPTION. Degree of the equivalent Bezier's, it should
90 be greater than the number of points in p.
91 step : TYPE double
92 DESCRIPTION. a step to generate an ordered, array which
93 equispaciate values in [0,1].
94

95 Returns
96 -------
97 p : TYPE numpy array
98 DESCRIPTION. 2x(n-1) array containing the control points
99 of the equivalent Bezier's curve
100

101 """
102 c = [Link][1]
103 t = [Link](0,1+step,step)
104 bezier_sc(p, t) #draw the Bz curve
105 if c < n:
106 #adding one more control point and recalculating
107 # the control points
108 p = [Link](p,[[0],[0]],axis=1)
109 pp =[Link]()
110 for i in range(1,c+1):
111 pp[:,i] = p[:,i-1]*i/(c) + (1 -i/(c))*p[:,i]
112 #the function calls itself till the number of points
113 #equals n
114 p = bzeq(pp,n,step)
115 # and it returns the control points. In the meanwhile it
116 #has drawn all the equivalent curves with degrees between
117 # c and n
118 return(p)
119

120 def dbez(p):

121 """
122 Compute a Bezier's . Curve derivative, defined by control
123 points in p. It also draw de odograph and the derivatives
124 at some selected points and the Bezier's curve and
125 the derivative, as tangents vector, as some selected points.
126

127 Parameters
128 ----------
 272 CHAPTER 8. INTERP. & FUNCT FITTING

129 p : TYPE numpy array

130 DESCRIPTION. Control points of the Bezier's curve we
131 wanna derivate.
132

133 Returns
134 -------
135 d : TYPE numpy array
136 DESCRIPTION. Controls points of the derivative.
137

138 """
139 grado = [Link][1]-1
140 d = [Link]([2,grado])
141 #compute the control points of the derivativa
142 for i in range(grado):
143 d[:,i] = (grado-1)*(p[:,i+1]-p[:,i])
144 t = [Link](0,1+0.01,0.01)
145 #plot de odograph
146 [Link](1)
147 v = bezier_sc(d, t)
148 [Link]([Link](12),[Link](12),\
149 v[0,::9],v[1,::9],\
150 angles='xy', scale_units='xy', scale=1)
151

152 #plot the curve and its derivative. (the escale of the
153 #vector is arbitrary)
154 [Link](2)
155 ptos = bezier_sc(p,t)
156 [Link](ptos[0,::9],ptos[1,::9],\
157 v[0,::9],v[1,::9])
158 [Link]('equal')
159 return(d)

Upper degree equivalent curves. Given a Bezier’s curve, represented by a Berstein’s polyno-
mial of degree n, it is possible to find Berstein’s polynomials of higher degree, which also describes
this same Beezier’s curve. Indeed, this means an increase in the number of the control points.
Suposse we have a Bezier’s curve defined by four control points,

p⃗(t) = p⃗0 B03 (t) + p⃗1 B13 (t) + p⃗2 B23 (t) + p⃗3 B33 (t)

If we multiply this polynomial by t + (1 − t) ≡ 1, the polynomial remains the same an so it

represent the same Bezier’s curve too. However, we have now a polynomial one degree higher than
the initial one. If after multiplying we group terms like (1 − t)n−1 ti , we can compute the value of
the new control points,
8.5. BÉZIER CURVES 273

p⃗+
0 =p
⃗0
1 3
p⃗+
1 = p⃗0 + p⃗1
4 4
2 2
p⃗+
2 = p⃗2 + p⃗3
4 4
3 1
p⃗+
3 = p⃗2 + p⃗3
4 4
p⃗+
4 =p
⃗3

And, in general, we can compute the control points for the Bezier’s curve of degree n + 1
equivalent to a previous defined curve of degree n as,

i
p⃗+
i = αi p
⃗i−1 + (1 − αi )⃗
pi , αi =
n+1
Using the above equation, we can iteratively obtain the control points of a Bezier’s curve
equivalent to any other given of whatever degree. An interesting property is that, as we increase
the number of control points, these and the control polygon tend to converge to the Bezier’s curve.
Figure 8.10 shows an example using a Bézier’s curve built from three control points. As can be
seen, we always obtain the same curve, although we have increased the number of control points.
It is also easy to see in figure 8.10d that the control polygons also converge to the Bezier’s curve.
Function bzeq in [Link] module above include, allows us to calculate the Bezier’s curve
equivalent to any other given, using any number of control points.

Derivatives. To compute the derivatives of a Bezier’s curve with respect to parameter t is

particularly easy, using the control points. Suppose we have a degree n Bezier’s curve, defined
from control points p⃗i . It first derivative with respect to t will be a Bezier’s curve of degree n − 1,
whose control point we can compute as:

d⃗i = n (pi+1
⃗ − p⃗i )
The new Bézier curve we obtain is a hodograph. It represents the position of the tips of vectors
tangent in each point to the original Bézier curve and has a direct relationship with the speed at
which you can travel across the curve at any point.
Figure 8.11a shows a Bezier’s curve an the derivative vector at some points of the curve. Figure
8.11b show the hodograph corresponding to and the same derivative vectors of de curve presented
in figure 8.11a
Function dbez, belonging to the module [Link] included above, allows us to compute the
derivative of any Bezier’s curve.

Interpolation using Bézier’s curves We can use Bézier’s curve for interpolating a set of
points, {⃗ p0 , · · · p⃗m }. If we use a curve to interpolate every pair of consecutive points, p⃗i , p⃗i+1 , i =
1, · · · m − 1 we assure the continuity in the interpolated points because the curves have necessarily
to pass through them. As in the case of spline interpolation, we can also impose continuity
to the derivatives to get a smooth interpolating curve. In the case of Bèzier’s curves, this is
especially simple. If we call B to the n-degree Bezier’s curve built between the points p⃗i−1 , p⃗i
with control points, p⃗i−1 , b1 , ⃗b2 , · · · , ⃗bn−1 , p⃗i , and C to the s-degree Bezier’s curve built between
274 CHAPTER 8. INTERP. & FUNCT FITTING

3 3

2.5 2.5

2 2

1.5 1.5

1 1

0.5 0.5

0 0

−0.5 −0.5

−1 −1
1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3

(a) Original curve (3 points) (b) 4 to 6 points equivalent curves

3 3

2.5 2.5

2 2

1.5 1.5

1 1

0.5 0.5

0 0

−0.5 −0.5

−1 −1
1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.6 2.8 3

(c) 4 to 12 points Equivalent curves (d) 4 to 30 points equivalent curve

Figure 8.10: Equivalent curves, built from a three control point Bézier’s curve.

the points p⃗i , p⃗i+1 with control points, p⃗i , ⃗c1 , ⃗c2 , · · · , ⃗cs−1 , p⃗i+1 ; we can assure the continuity of the
first derivative in the point p⃗i just imposing that,
 
n · p⃗i − ⃗bn−1 = s · (⃗c1 − p⃗i )

This condition imposes a link between the second-last control point of the curve B and the
second control point of the curve C. However, it allows free election of the remaining control points
to build the curves as desired.
For instance we can freely choose all control points of curve B and use it to compute the point
⃗c1 ,
n+s n
p⃗i − ⃗bn−1 ⃗c1 =
s s
Figure 8.12 shows an example of interpolation in which we have applied the derivative continuity
condition we have described.

)
8.5. BÉZIER CURVES 275

3 2

2.5
1

2
0
1.5

−1
1

0.5 −2

0
−3

−0.5
−4
−1

−5
−1.5

−2 −6
1 1.5 2 2.5 3 3.5 4 −1 −0.5 0 0.5 1 1.5 2 2.5 3

(a) Bézier’s curve (4 points) (b) Derivative (hodograph)

Figure 8.11: Bézier’s curve and its derivative with respect to the Berstein’s polynomial parameter
which defines the curve t ∈ [0, 1]

4
6
3

2 4

1
2
0

−1
0
−2
p_i
pi
−3 −2

−4
−4
−5

−6
−2 0 2 4 6 8 −6
−2 0 2 4 6 8 10 12

(a) Interpolation using three control points (b) Interpolation using 3 and 4 control points
Bezier’s curves) Beziers’s curves

Figure 8.12: interpolating three point using two Bezier’s curves

276 CHAPTER 8. INTERP. & FUNCT FITTING
Chapter 9

Numerical differentiation and

integration

You never really understand a

person until you consider things
from his point of view...until you
climb into his skin and walk
around in it.

Harper Lee, To kill a mokingbird

Differentiation and, above all, integration are very common operations in numerical computing.
In many cases, obtaining the analytical expression for a function derivative or integral can be very
complex or even impossible. Besides, sometimes we have not an analytical expression for the
function we need to derivate of integrate, but only a set of numerical values of it. This is the case,
for example, when we are worming with experimental data. If we only have numerical values of
the function, we can only compute the derivative or integral numerically.
Physical systems are usually describe by differential equations. Most differential equations do
not have an analytical solution and thus, we can only obtain numerical solution for them.
In general, numerical differentiation approximates the value taken by the derivative of a function
at a specific point. Similarly, numerical integration approximates the value taken by the integral
of a function in an interval.

9.1 Numerical differentiation.

We start taking a set of pairs of points {xi , yi },

x x0 x1 ··· xn
y y0 y1 ··· yn

Which belong to a function y = f (x) whose analytical expression we may know or may not
know. The goal of numerical differentiation is to estimate the value of the function derivative
f ′ (x), in any of the points xi where the function value f (x) is known.
We can distinguish two different method to approximate the derivative:

277
278 CHAPTER 9. NUM. DIFF. AND INTEGR.

1. WE can derivate the polynomial that interpolates the point. So, we get a new polynomial
that approximate the derivative.

f (x) ≈ Pn (x) ⇒ f ′ (x) ≈ Pn′ (x)

2. Estimating the derivativa using a finite differences quotient obtained from the Taylor poli-
nomial expansion.
If we start from the definition of derivative,

f (x0 + h) − f (x0 ) f (x0 + h) − f (x0 )

f ′ (x0 ) = lim ≈
h→0 h h
We can associate this approach with the first-degree Taylor polynomial expansion of functions
f (x)

f (x) − f (x0 )
f (x) ≈ f (x0 ) + f ′ (x0 ) · (x − x0 ) ⇒ f ′ (x0 ) ≈
x − x0
If we take x − x0 = h both expressions meet.
In general, the numerical differentiation algorithms are unstable. Initial data errors due to
experimental factors or computer rounding off increase with the differentiation process. For this
reason, it is not possible to compute high-order derivatives, and the results achieved from numerical
differentiation must always be considered extremely carefully.

9.1.1 Numerical differentiation base on the interpolation polynomial.

The method involves deriving the polynomial Pn (x) using any of the methods described in chapter
8 and then evaluating the derivative polynomial Pn′ (x) at the desired point.
WE can obtain a specially simple expression for the derivative polynomial, in the case o equi-
spaced points interpolated by the newton-Gregory polynomial,

x − x0 (x − x1 ) · (x − x0 ) 2 (x − xn−1 ) · · · (x − x1 ) · (x − x0 ) n
pn (x) = y0 + ∆y0 + 2
∆ y0 + · · · + ∆ y0
h 2·h n! · hn
And after deriving, we obtain a new polynomial,

∆y0 ∆2 y0
p′n (x) = + [(x − x1 ) + (x − x0 )] +
h 2 · h2
∆3 y0
+ [(x − x1 )(x − x2 ) + (x − x0 )(x − x1 ) + (x − x0 )(x − x2 )] + · · · +
3! · h3
n−1
∆n y0 X (x − x0 )(x − x1 ) · · · (x − xn−1 )
+
n! · hn x − xk
k=0

To evaluate this polynomial at point x0 is particularly easy,

−h −h −2h −(n−1)h
∆y0 ∆2 y0 z }| { ∆n y0 z }| { z }| { z }| {
p′n (x0 ) = + (x 0 − x 1 ) + · · · + [(x 0 − x 1 ) (x 0 − x 2 ) · · · (x 0 − x n−1 )]
h 2 · h2 n! · hn
∆2 y0 ∆3 y0 ∆n y0

1
p′n (x0 ) = ∆y0 − + + ··· + (−1)n−1
h 2 3 n
9.1. NUMERICAL DIFFERENTIATION. 279

It is worth notice that the previous expression represent de derivative with increasing precision
as we add new higher-order differences. If we would only know two data, (x0 , y0 ) y (x1 , y1 ), we
could only calculate the first-order divided difference. In which case, our approximate derivative
computation at x0 would be,
1
p′1 (x0 ) = ∆y0
h
If we have three data, we could compute ∆2 y0 and add second term to our derivative estimation.

∆2 y 0
 
1
p′2 (x0 ) = ∆y0 −
h 2
And so on, precision is increasingly improving.
Let see the computing of the derivative at point x0 = 0.0 using the following data table,

xi yi ∆yi ∆2 yi ∆3 yi ∆4 y i
0.0 0.000 0.203 0.017 0.024 0.020
0.2 0.203 0.220 0.041 0.044
0.4 0.423 0.261 0.085
0.6 0.684 0.346
0.8 1.030

1. Using the two first points,

1
y ′ (0, 0) = p11 (0.0) = · 0.203 = 1.015
0.2
2. Using the three first points,
 
1 0.017
y ′ (0, 0) = p12 (0.0) = 0.203 − = 0.9725
0.2 2
3. Using the four first points,
 
1 0.017 0.024
y ′ (0, 0) = p13 (0.0) = 0.203 − + = 1.0125
0.2 2 3
4. Using the four available points,
 
1 0.017 0.024 0.020
y ′ (0, 0) = p14 (0.0) = 0.203 − + − = 0.9875
0.2 2 3 4

9.1.2 Numerical differentiation base on finite differences

As we explained in the introduction to the chapter, the idea is to take the Taylor’s expansion to
approximate a function derivative at a specific point. If we start with the simplest case, we can
approximate the derivative suppressing the limit computation in its definition

f (xk + h) − f (xk ) f (xk + h) − f (xk )

f ′ (xk ) = lim ≈
h→0 h h
The expression we have achieved is know as two-point forward difference formula. The error
we make due to the finite value we chose for h is called as truncation error. It is clear that from
a purely mathematical point of view, the approximation is better the smaller h is. However, this
280 CHAPTER 9. NUM. DIFF. AND INTEGR.

is not true from a numerical point of view. As we make smaller the value of h, the rounding-off
error increases due to the computer finite arithmetic. Then we have to take into account the sum
of both errors,
redondeo
truncating
z }| {
′ f (x + h) − f (x) z }| { 1
f (x) = + | C{z· h } + D · , C >> D
h h
truncamiento | {z }
rounding

The optimal value for h is that what minimised the rounding-off error plus the truncating error
sum. figure 9.1 shows schematically how truncating or rounding-off error dominates as we do grow
or decrease the value of h around its optimal value.

3.5
Dominan errores de redondeo Dominan errores de truncamiento
Rounding-off error dominate Truncating error dominate
3.0

2.5

2.0
error

1.5

1.0

0.5
valor óptimo
0.0 optimal value
0.4 0.6 0.8 1.0 1.2
h

Figure 9.1: Function derivative approximation error, using a a two-points differentiation formula.
Dependency on the difference h value

A we sow in the introducction to the present section, departing from a function Taylor’s expan-
sion it is possible to arrive to different numerical differentiation formulae and estimate the error
made. So, for instance, departing from the fist order Taylor’s polynomial,

h2 ′′ f (x + h) − f (x) h ′′
f (x + h) = f (x) + hf ′ (x) + f (z) ⇒ f ′ (x) = − f (z), x < z < x + h
2 h 2
The error we make is proportional to the interval h that we use. The constant of proportionality
depends on the second derivative of the function, f ′′ (z), at an undetermined point z ∈ (x, x + h).
We express this error as having a linear dependency on the value of h, stating that the error is of
order h, and we represent it as O(h).

f (x + h) − f (x)
f ′ (x) = + O(h)
h
To enhance the accuracy, we can calculate the value of the third-order Taylor polynomials at
two equally spaced points to the left and right of the point x. Then we find the difference between
these values and solve for the first derivative.
9.1. NUMERICAL DIFFERENTIATION. 281

f’(x) f’(x)

(f(x0+h)−f(x0−h)}/2h
(f(x0+h)−f(x0)}/h

f(x)
f(x)

x0 x0+h x0−h x0 x0+h

(a) Two points forward difference (b) Two point central difference

Figure 9.2: Comparison between the approximations achieved for a function derivative using two
point forward difference and two point central difference.


h2 ′′
′ h3 ′′′ 
f (x + h) = f (x) + hf (x) + f (x) + f (z) f (x + h) − f (x − h) h2 ′′′
2 3!

2 3
⇒ f ′ (x) = − f (z)
′ h ′′ h ′′′  2h 6
f (x − h) = f (x) − hf (x) + f (x) − f (z) 
2 3!
In this case, we get an error proportional to h square, thus,

f (x1 ) − f (x−1 )
f ′ (x0 ) = + O(h2 )
2h
Where we have taken x ≡ x0 , x + h ≡ x1 y x − h ≡ x−1 . This approximation is called two point
central difference. Figure 9.2a shows a comparison between an actual function derivative and its
approximation using a two point forward difference formula. Figure 9.2b Compares, for the same
function, the actual derivative with a two points central difference approximation. In this example,
we can see how the central approximation yields better results. However, the accuracy of a result
also depends on the higher-order derivative values of the function. Therefore, it is not possible to
claim that the central difference will always result in better outcomes.
With three points and the Tylor’s expansion we can approximate the derivative using a three
points forward difference formula,

h2 h3
 
4 · f (x1 ) = f (x0 ) + hf ′ (x0 ) + f ′′ (x0 ) + f ′′′ (z) 
2 3!

⇒
′ 2 ′′ 4h3 ′′′ 
f (x2 ) = f (x0 ) + 2hf (x0 ) + 2h 2f (x0 ) + f (z)


3
−f (x2 ) + 4f (x1 ) − 3f (x0 ) h2 ′′′
⇒ f ′ (x0 ) = − f (z)
2h 3
282 CHAPTER 9. NUM. DIFF. AND INTEGR.

In this case the also get a order h2 error but its two times the value achieved for the two point
central difference.
From Taylor’s expansion and using the appropriated number of points, it si possible to compute
approximation for the first and successive derivatives of a function. The procedure is similar to
the one we employed in the two and tree points differences. Table 9.1 shows some derivative
approximations using the most common finite differences formulae. To simplify notation, in every
(j)
case we have taken yi = f (xi ), yi = f (j) (xi ).

First derivative formulae Second derivative formulae

y1 −y0 y2 −2y1 +y0

y0′ = h + O(h) y0′′ = h2 + O(h)

y1 −y−1 y1 −2y0 +y−1

y0′ = 2h + O(h2 ) y0′′ = h2

−y2 +4y1 −3y0 −y3 +y2 −5y1 +2y0

y0′ = 2h + O(h2 ) y0′′ = h2 + O(h2 )

−y2 +8y1 −8y−1 +y−2 −y2 +16y1 −30y0 +16y−1 −y−2

y0′ = 12h + O(h4 ) y0′′ = 12h2 + O(h4 )

Third derivative formulae Fourth derivative formulae

y3 −3y2 +3y1 −y0 y4 −4y3 +6y2 −4y1 +y0

y0′′′ = h3 + O(h) y0iv = h4 + O(h)

y2 −2y1 +2yx−1 −y−2 y2 −4y1 +6y0 −4y−1 +y−2

y0′′′ = 2h3 + O(h2 ) y0iv = h4 + O(h2 )

Table 9.1: Derivative formulae based on finite differences

9.2 Numerical Integration.

Given an arbitrary function f (x) it is in many cases possible to obtain analitically is primitive
F (x) such that, f (x) = F ′ (x). In these cases, the function f (x) definite integral in an interval
[a, b] can be obtained from its primitive.
Z b
I(f ) = f (x)dx = F (x)|ba = F (b) − F (a)
a

There are many cases for which we do not know function F (x), and there are others where we do
not even know the function f (x) as, for example, when we only have a data table {xi , yi = f (xi )}
that represents the function. In these cases, it is possible to approximate the de definite integral
of a function f (x) in an interval [a, b] using available points and methods known as quadrature
formulae.
Z b n
X
I(f ) = f (x)dx ≈ Ai f (xi )
a i=0

A common technique to obtain Ai coefficients is to use an implicit method by interpolating

polynomial integration,
9.2. NUMERICAL INTEGRATION. 283

Z b Z b
I(f ) = f (x)dx ≈ Pn (x)dx
a a

To do this, we first takes the ends of the interval as the first and last data available, [a, b] ≡
[x0 , xn ].
So, for example, we using Lagrange’s polynomials defined in section 8.2.2,
n
X
p(x) = lj (x) · yj
j=0

We can compute the Ai coefficientes as,

 
Z b Z b Z xn Xn Z xn
I(f ) = f (x)dx ≈ Pn (x)dx =  lj (x) · yj dx ⇒ Aj =
 lj (x)dx
a a x0 j=0 x0

The family of numerical integration methods known as Newton-Cotes formulae, can be obtained
from the Newton-Gregory’s interpolation polynomial described in section 8.2.4 . Suppose we have
a function defined as a set of n + 1 equispaced points along an integrating interval {(xi , yi )}0,···n .
We can approximate the integral I(y) as,

xn xn  
x − x0 (x − x0 )(x − x1 ) 2 (x − x0 ) · · · (x − xn−1 ) n
Z Z
ydx ≈ y0 + ∆y0 + ∆ y 0 + · · · + ∆ y0
x0 x0 h 2!h2 n!hn

Newton-Cotes formulae are associated with the degree of the interpolation polynomial employed
to obtain the formula:
1. for n=1, we obtain the Trapezium rule.
Z x1
h
I(y) = ydx ≈ (y0 + y1 )
x0 2
2. for n = 2, we obtain the Simpson’s rule.
Z x2
h
I(y) = ydx ≈ (y0 + 4y1 + y2 )
x0 3
3. for n = 3, we obtain the Simpson’s 3/8 rule.
Z x3
3h
I(y) = ydx ≈ (y0 + 3y1 + 3y2 + y3 )
x0 8
We do not usually use interpolation polynomials with larger degrees due to rounding-off errors
and the local oscillations of such polynomials.

9.2.1 Trapezium formula

The trapezium formula uses only two points to obtain the function integral in the interval defined
by the points.

x1 x1 x1
∆y0 (x − x0 )2
 
x − x0 h
Z Z
I(y) = ydx ≈ y0 + ∆y0 dx = y0 x + = (y0 + y1 )
x0 x0 h h 2 x0 2
284 CHAPTER 9. NUM. DIFF. AND INTEGR.

Figure 9.3 shows graphically the results of approximating a function y = f (x) defined integral
using the trapezium rule. The integral fits the area of the trapezium formed by points (x0 , 0),
(x0 , y0 ), (x1 , y1 ) y (x1 , 0). This is the reason why the method is known as trapezium formula or
trapezium rule and the mathematical expression we get,
h
(y0 + y1 ),
I(y) =
2
which coincides with the expression to compute the area of the trapezium showed in the figure

y=f(x)

y_1

P1
y

y_0

x_0 x_1
x

Figure 9.3: Graphical interpretation of trapezium rule.

Extended (or composed) trapezium formula. Figure 9.3 allows us to observe the difference
between the area computed and the actual area comprised between the curve and the x axis. As
we have approximated the curve by a straight line (1-degree interpolation polynomial p1 ) in the
integration interval, the larger the integration interval and/or the variation of the function in the
interval, the larger the error we make.
One solution to this problem, if we know the analytic expression for the function we wish to
integrate or we have enough points, is to divide the integration interval into shorter intervals and
apply the trapezium formula in any of them.

xn n−1
X Z xi+1 n−1
h h
Z X
I(y) = ydx ≈ y(x)dx = (yi + yi+1 ) = (y0 + 2y1 + 2y2 + · · · + 2yn−1 + yn )
x0 i=0 xi i=0
2 2

Figure 9.4, shows the result obtained after applying the extended trapezium formula to the
same function of figure 9.3. In the present case, we have divided the integration interval into four
subintervals. It is evident from the figure that the approximation will progress as the number of
intervals increases and their size decreases.
9.2. NUMERICAL INTEGRATION. 285

y=f(x)

y4
y2
y1, y3
y

y0

x0 x1 x2 x3 x4
x

Figure 9.4: Extended trapezium formula graphical interpretation

9.2.2 Simpson’s formulae

The integral Simpson’s formulae (or rules) are two approximations to the definite integral of a
function, obtained from the Newton-Gregory’s interpolation polynomials of degree two (Simpson
1/3) and degree 3 (Simpson 3/8).
In the first case we need to know three equispaced values of the function in the integration
interval. In the second case we need to know four points.

Simpson’s 1/3 formula. The Simpson’s rule or Simpson’s 1/3 rule employs a 2-degree Newton-
Gregory’s interpolation polynomial to obtain the approximation of the integral,

x2 x2  
x − x0 (x − x0 ) · (x − x1 ) 2 h
Z Z
I(y) ≈ P2 (x)dx = y0 + ∆y0 + 2
∆ y0 dx = (y0 + 4y1 + y2 )
x0 x0 h 2h 3

Figure, 9.5 shows graphically the result of applying the Simpson’s method to the same function
used in previous examples. Again, the goodness of the approximation depends on the function
variation in the integration interval. The main difference with the trapezium method is that the
area is now bounded by the parabolic segment defined by the polynomial we have utilized.

Simpson’s 3/8 Formula. In this case, we use a Newton-Gregory’s polynomial of degree 3 to

obtain the approximation for the integral.
286 CHAPTER 9. NUM. DIFF. AND INTEGR.

y=f(x)

p2(x)

y_3
y_2

y_1

x_0 x_1 x_2

Figure 9.5: Graphical interpretation of Simpson’s 1/3 formula

Z x3
I(y) ≈ P2 (x)dx =
x
Z x0 3  
x − x0 (x − x0 ) · (x − x1 ) 2 (x − x0 ) · (x − x1 ) · (x − x2 ) 3
= y0 + ∆y0 + ∆ y0 + ∆ y 0 dx
x0 h 2h2 3!h3
3h
= (y0 + 3y1 + 3y2 + y3 )
8

Figure 9.6 shows the result of applying Simpson’s 3/8 formula to the same function utilized in
previous examples. In this case, the integral is exact because the example function is a three degree
polynomial and fit exactly the interpolating polynomial we have built to compute the integral.
Like the trapezium formula, we hardly ever apply Simpson’s methods to the whole integration
interval. Instead, we divide the integration interval into smaller subintervals and apply the method
to such subintervals. The results are known as extended Simpson’s methods. As in the case of the
trapezium formula, the extended Simpson’s methods improve the approximation computed for the
integral as much as the smaller the size of the subinterval we use.
So the extended Simpson 1/3 formula takes the form,

n n
2 −1
XZ x2i+2 2 −1
X h
I(y) ≈ P2 (x)dx = (y2i + 4y2i+1 + y2i+2 )
i=0 x2i i=0
3
h
= (y0 + 4y1 + 2y2 + 4y3 + 2y4 + · · · + 2yn−2 + 4yn−1 + yn )
3

Where we have divided the integration interval in n subintervals, the Simpson’s formula has
been applied to every two consecutive subintervals and, eventually we have summed the results
together.
 9.2. NUMERICAL INTEGRATION. 287

y=f(x)
2,5

p3(x)

y4
y2
y3

y1

x0 x1 x2 x3

Figure 9.6: Graphical interpretation of Simpson’s 3/8 formula

n n
3 −1
XZ x3i+3 3 −1
X 3h
I(y) ≈ P3 (x)dx = (y3i + 3y3i+1 + 3y3i+2 + y3i+3 )
i=0 x3i i=0
8
3h
= (y0 + 3y1 + 3y2 + 2y3 + 3y4 + 3y5 + 2y6 + · · · + 2yn−3 + 3yn−2 + 3yn−1 + yn )
8
In this case, we have also divided the interval in n subintervals but now we apply the Simpson’s
3/8 rule to each group of three successive subintervals.
The code included below compute an approximation to de definite integral of a function using
any one the three method we have described: Trapezium, Simpson’s or Simpson’s 3/8,

[Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Sun Aug 18 16:21:54 2024
4

5 @author: abierto
6 """
7 import numpy as np
8 from matplotlib import pyplot as pl
9 def integra(fun,met,inter,dib=False):
10 """
11 function numerical integration, using trapezium, Simpson's
12 1/3 or Simpson 3/8 methods
13
 288 CHAPTER 9. NUM. DIFF. AND INTEGR.

14 Parameters
15 ----------
16 fun : TYPE function
17 DESCRIPTION. Function to be integrated
18 met : TYPE char array
19 DESCRIPTION. method should be one of 'trap' 'simpson'
20 or 'simpson38'
21 inter : TYPE array [a,b]
22 DESCRIPTION. integration interval limits
23 dib : TYPE logical
24 DESCRIPTION. yes : draw the results no: skip the drawing
25

26 Returns
27 -------
28 intg: TYPE double
29 DESCRIPTION. Integral value
30 f1 = fun(inter[1])
31 h = inter[1] - inter[0]
32

33 """
34 if met == 'trap':
35 f0 = fun(inter[0])
36 f1 = fun(inter[1])
37 h = inter[1] - inter[0]
38 intg = h*(f0+f1)/2
39 if dib:
40 x = [Link](inter[0],inter[1],100)
41 y = [Link]([fun(i) for i in x])
42 ypol = f0 + (x - inter[0])*(f1-f0)/h
43 pl.fill_between(x,ypol,\
44 where=(inter[0]<=x)&(x<=inter[1]))
45 [Link](x,y,'b')
46 elif met == 'simpson':
47 pmedio = (inter[1] + inter[0])/2
48 f0 = fun(inter[0])
49 f1 = fun(pmedio)
50 f2 = fun(inter[1])
51 h = (inter[1] - inter[0])/2
52 intg = h*(f0+4*f1+f2)/3
53 if dib:
54 x = [Link](inter[0],inter[1],100)
55 y = [Link]([fun(i) for i in x])
56 ypol = f0+(x-inter[0])*(f1-f0)/h\
57 +(x-inter[0])*(x-pmedio)*(f2-2*f1+f0)/(2*h**2)
58 pl.fill_between(x,ypol,\
59 where=(inter[0]<=x)&(x<=inter[1]))
60 [Link](x,ypol,'k')
61 [Link](x,y,'b')
62 elif met == 'simpson38':
63 inter = [Link](inter[0],inter[1],4)
64 f = [Link]([fun(i) for i in inter])
65 h = inter[1]-inter[0]
 9.2. NUMERICAL INTEGRATION. 289

66 intg = 3*h*(f[0]+3*f[1]+3*f[2]+f[3])/8
67 if dib:
68 x = [Link](inter[0],inter[3],100)
69 y = [Link]([fun(i) for i in x])
70 ypol = f[0]+(x-inter[0])*(f[1]-f[0])/h\
71 +(x-inter[0])*(x-inter[1])\
72 *(f[2]-2*f[1]+f[0])/(2*h**2)\
73 +(x-inter[0])*(x-inter[1])*(x-inter[2])\
74 *(f[3]-3*f[2]+3*f[1]-f[0])/(6*h**3)
75 pl.fill_between(x,ypol,\
76 where=(inter[0]<=x)&(x<=inter[3]))
77 [Link](x,ypol,'k')
78 [Link](x,y,'b')
79 return(intg)
80

81 def trocea(fun,met,inter,div,dib=False):
82 """
83 This funtion divide a interval into the number of subintervals indicate
84 by div and calls function integra to obtain the integral in
85 any subinterval. Eventually sums all results to compute the
86 integral over the whole interval
87

88 Parameters
89 ----------
90 fun : TYPE function
91 DESCRIPTION. function to be integrate
92 met : TYPE character string
93 DESCRIPTION. Method it should be one of 'trap' 'simpson'
94 or 'simpson38'can be 'trap',
95 inter : TYPE array or list
96 DESCRIPTION. limits for the integration interval [a,b]
97 div : TYPE integer
98 DESCRIPTION. Number of subintervals
99 dib : TYPE bool
100 DESCRIPTION. True draw the result False does not draw
101

102 Returns
103 -------
104 total : TYPE double
105 DSCRIPTION Definite integral value of fun in inter
106

107 """
108 tramos = [Link](inter[0],inter[1],div+1)
109 total = 0
110 for i in [Link](div):
111 integ = integra(fun,met,[tramos[i],tramos[i+1]],dib)
112 total = total + integ
113 return(total)

This program holds two functions. The first one integra applies the method directly over
the integration interval. To obtain the extended methods, we can employ the second included
function trocea, which divides the initial integration interval into the number of subintervals we
290 CHAPTER 9. NUM. DIFF. AND INTEGR.

wish, applies the function integra to any subinterval and, eventually, add all the results together
to obtain the value of the integral in the wished interval.
Lastly, the Python module Scipy has several functions to compute a function’s definite integral.
It is located in the submodule integrate. The easiest way To compute the integral of a function is
to use the function [Link]. This function takes a funtion name and two variables which
represent the integration limits as imputs and returns the integral value in the interval introduced
and an estimation of the committed error. The following code shows a use example of the function
quad and compares the results with those of the function trocea whose code we have included
above.

In [56]: import [Link] as integ

In [57]: def probando(x):

...: """test funtion to be integrate"""
...: return((x-1)*x*(x-2))+6
...:

In [58]: [Link](probando,-5,2.5)
Out[58]: (-260.85937499999994, 3.3240810849038596e-12)

In [59]: trocea(probando,'simpson',[-5,2.5],5)
Out[59]: -260.859375

9.3 Differential equations. Initial value problems

The laws of physics are written as differential equations
Differential equations establish a mathematical relationship between a variable and its deriva-
tives with respect to another variable or other variables the first variable depends upon. We can
find the simplest example watching the one dimensional dynamics equations which link the second
derivative of a body position with the force acting on the body.

d2 x
m· =F
dt2
If the force applied is constant or we know explicitly how it evolves on time, we can integrate
the previous equation to straightforwardly obtain the position first derivative with respect to time
—the velocity—,

d2 x dx F (t)
Z
m · 2 = F → v(t) = = dt + v(0)
dt dt m
When we consider we know the body velocity v(0) at the initial time.
If we now integrate the expression we have got for the the velocity, we obtain the position as a
function of time,
Z Z 
dx F (t) F (t)
Z
v(t) = = dt + v(0) → x(t) = dt + v(0) dt + x(0)
dt m m
Were we suppose the initial position x(0) is known.
Perhaps the better known and most studied physical idealised system is the harmonic oscillator.
In this case, the system suffers a force that depends on the position and, if the oscillator is damped,
it suffer also a force that depends on the speed,
9.3. DIFFERENTIAL EQUATIONS. INITIAL VALUE PROBLEMS 291

d2 x dx
2
= −kx − µ
m
dt dt
In this last case, the expression we obtain is an ordinary differential equation (ODE), and it is
no longer so easy to solve and obtain an analytical expression for x(t). To obtain such a expression
we need to use specific methods to solve differential equations
The harmonic oscillator problem belongs to a problems family knows as initial value problems.
In general, to solve a first order initial value problem is to obtain the function x(t), that satisfies
the equation,
dx
x′ (t) ≡ = f (x(t), t), x(t0 )
dt
Where x(t0 ) represents a known initial value of the function x(t).
In many cases, differential equations describe physical phenomena that not have an analytical
solution. That is, there in no analytical function x(t) that satisfies de differential equation. In
these cases is still possible to compute numerical solutions using a computer. The initial value
problems reduces then to find a discretised version of the function x(t).
Developing techniques to numerically solve differential equations is perhaps one of the most
important fields of scientific computing. Here, we will describe only me most basic methods.
The methods we are going to introduce are essentially base on discretising the domain in which
we want to know the value of the function x(t). For instance, If we want to know the values of
the function in the interval t ∈ [a, b], we divide the interval in n subinterval, each one of size hi .
The methods qe are going to study give us an approximation of the function x(t), x0 , x2 · · · xn in
the n + 1 points t0 , t1 , ·, tn , where t0 = a, tn = b, y ti+1 − ti = hi . The value hi is known as the
integration step. Besides, we suppose that we know the value the function x(t) takes in the interval
initial side a, x(a) = xa .

9.3.1 The Euler’s method.

The Euler’s method can be obtained from the Taylor’s expansion of the function x, around the
known value (a, xa ). The idea is to start at the known value an compute iteratively the remaining
values x1 · · · until we arrive to the end of the interval b, at which we want to know the value
function x takes.
We can obtain the relationship between to consecutive values of the function from the Taylor’s
expansion as,

(ti+1 − ti )2 ′′ (ti+1 − ti )n (n)

x(ti+1 ) = x(ti ) + (ti+1 − ti )x′ (ti ) + x (ti ) + · · · + x (ti ) + · · ·
2 n!
As we are solving an initial condition problem, we know first the derivative of function x(t) at
the initial point x′ (t) = f (x(t), t). Therefore, we may substitute the derivatives of x by the funtion
f an its derivatives with respect to t.

(ti+1 − ti )2 ′ (ti+1 − ti )n (n−1)

x(ti+1 ) = x(ti ) + (ti+1 − ti )f (ti , xi ) + f (ti , xi ) + · · · + f (ti , xi ) + · · ·
2 n!
Where xi ≡ x(ti ). If we truncate the Taylor’s polynomial, leaving only the first order term and
set the integration step to a constant value, hi ≡ h = cte we arrive to the Euler’s method,

xi+1 = xi + h · f (ti , xi )
 292 CHAPTER 9. NUM. DIFF. AND INTEGR.

Departing from a known initial value, x0 , we can obtain the successive values using a simple
iterative algorithm,

x0 = x(a)
x1 = x0 + hf (a, x0 )
x2 = x1 + hf (a + h, x1 )
..
.
xi+1 = xi + hf (a + ih, xi )

The following code implements the Euler’s method using a function f (t) and an initial value
x0 as inputs,

Euler [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Wed Aug 21 11:08:42 2024
4 Euler's method for solving initial value problems
5 @author: abierto
6 """
7 import numpy as np
8 import [Link] as pl
9

10 def euler(fun,xa,a,b,h):
11 """
12 This function solve numerically the problem dx/dt = f(t,x)
13 form the intial condition xa along the interval [a,b] and
14 using an integration step h
15

16 Parameters
17 ----------
18 fun : TYPE function
19 DESCRIPTION. it is the funtion that describe the differential
20 equation to be solved it should be a funtion of t and x
21 f(t,x) also is there is not explicit dependence on t.
22 xa : TYPE numpy array
23 DESCRIPTION. initial condition
24 a : TYPE double
25 DESCRIPTION. initial value of t
26 b : TYPE double
27 DESCRIPTION. final value of t
28 h : TYPE double
29 DESCRIPTION. integration step
30

31 Returns
32 -------
33 x : TYPE numpy array
34 DESCRIPTION. intial value dx/dt = f(x) solution at points
35 a, a+h a+2h ... b
36 t. TYPE numpy array
37 DESCRIPTION. times x has been computed at
 9.3. DIFFERENTIAL EQUATIONS. INITIAL VALUE PROBLEMS 293

38 """
39 if a >= b:
40 raise ValueError('I need an increasing interval')
41

42 #to arrive to the final condition, we modify slighly the

43 #integration step. First we calculate the aproximate number
44 #of intervals between a and b of size h and round it towards
45 #infinite.
46 pt = int([Link]((b-a)/h))
47 #then, we modify the integration step to acomodate at
48 #the rounded number of interval. This is, of course, not
49 #necessary to implement the method, there are other
50 #posibilities to deal with the integration step and
51 #the limit b
52 hft = (b-a)/pt
53 t = [Link](a,b+hft,hft)
54 x = [Link]([[Link][0],pt+1])
55 x[:,0] = xa
56 for i in range(1,pt+1):
57 x[:,i] = x[:,i-1] + hft*fun(t[i-1],x[:,i-1])
58 [Link](t,x.T)
59 return(t,x)
60

61

The differential equation that models the charge and discharge of an electronic capacitor sup-
plies a simple example of an first order initial condition problem. Figure 9.7 shows a electronic
circuit composed of a resistor R in series with a capacitor C.

Figure 9.7: Circuito RC

The electric current that passes through a capacitor depends on the capacitor capacity and on
the variation on time of the voltage applied to it.

dVo
I=c
dt
the current through the resistor can be computed using Ohm’s law.
294 CHAPTER 9. NUM. DIFF. AND INTEGR.

VR = I · R
Additionally, the current flowing through the circuit is the same for both the resistor and the
capacitor. The voltage supplied to the circuit should equal the sum of the resistor and capacitor
voltage drop, expressed as Vi = Vo + VR . By substituting and solving,

dV0
Vi = Vo + VR → Vi = Vo + I · R → Vi = Vo + R · C
dt
And, after rearranging the result,

dVo Vi − Vo
=
dt R·C
We get a differential equation for the capacitor tension drop that could be solve as a initial
value problem. For this problem the function f (t, x) takes the form,

Vi − Vo
f (t, V0 ≡ x) =
R·C
In addition, we need to know an initial value Vo (0) for the voltage across the capacitor. If
we consider the capacitor is initially discharged, then Vo (0) = 0. For this problem we know the
analytic solution. We can express the voltage across the capacitor as a function fo time,
 
V0 (t) = Vi 1 − e−t/R·C

We can solve the capacitor problem using the function euler included above. WE only need
to define a Python funtion to represent the funtion f (x, t) for our initial value problem,

In [151]: def condensador(t,Vo,C=1.,R=1.,Vi=10.):

...: dVo = (Vi-Vo)/(R*C)
...: return(dVo)

Figure 9.8 graphically compares the results achieved using the function
euler(condensador,0,0,10,h) to compute the voltage across the capacitor after supplying a
constant input voltage Vi = 10V during ten seconds, using two different integration steps, h = 0.5s
ans h = 0.25s. We have also added the analytical solution to the problem to the graphic.
It’s interesting to observe how the voltage, denoted as V0 , increases until it reaches the value
Vi = 10V , which corresponds to the voltage supplied to the circuit. The time it takes for the
capacitor to reach this voltage and become fully charged depends on its capacitance and the value
of the resistor in the circuit.
As expected, when we diminish the integration step, the numerical solution approximates the
analytical solution better. However, as we saw for the case of numerical differentiation, there is
an optimal h. If we take the integration step value below this optimal value, the round-off errors
worsen the solution.

a Second-order problem. Let’s consider now a system with a mass hanging of a spring and with
a mechanical gadget (damper) that exerts a force opposite to the mass movement and proportional
to the mass speed. For this system the differential equation is as follows:

d2 y dy
m 2
=m·g−k·y−µ
dt dt
9.3. DIFFERENTIAL EQUATIONS. INITIAL VALUE PROBLEMS 295

10

6
V(volts)

2
h = 0.5

h = 0.25
0 y = 10(1 − e −t)

0 2 4 6 8 10
t(s.)

Figure 9.8: Comparison among the results obtained using Euler’s method with two different inte-
gration steps h = 0.5 y h = 0.25, and the analytic solution for voltage Vo across a capacitor. We
took C = R = 1 and Vi = 10.

We have a second-order initial values problem, with an equation that links the variable y its first
d2 y
derivative dy
dt and its second derivative dt2 . To solve this kinda problem, we rewrite the differential
equation using two first-order differential equations. one for the position y and the other for the
velocity vy = dx
dt . IN this way, we obtain two coupled first-order equations:

dy
= vy
dt
dvy k µ
=g− ·y− · vy
dt m m
This system can be solve using the Euler’s method, but we need to define a Python function
to represent the system of differential equations,

def amortiguador(t,y):
"""
Defines the set of differetial equations for a damping
oscilator

Parameters
----------
t : TYPE double
DESCRIPTION. time
296 CHAPTER 9. NUM. DIFF. AND INTEGR.

k µ

m
Figure 9.9: A mass vertically hanging of a sprint with a damping resistance
.

y : TYPE numpy array

DESCRIPTION. y[0] ->> position
y[1] ->> velocity

Returns
-------
dydt : TYPE numpy arra
DESCRIPTION. dydt[0] ->> velocity
dydt[1] ->> acceleration

"""
g = 9.8 #gravity accel.
k = 100. #spring constant
m = 2. #mass
mu = 0.5 #friction constant
dydt = [Link](2)
dydt[0]=y[1]
dydt[1]=g-(k/m)*y[0]-(mu/m)*y[1];
return(dydt)

Using the function just described

euler(’amortiguador’,[0 0],0,50,0.01) we compute the temporal evoluntion of position and
velocity for the mass attached to the spring. Figure 9.10 shows the result, the mass oscillates
around an equilibrium position until the damping mechanism, eventually, stops it.
If we change the values of the spring constant to k = 100 (kg/s2 ) and repeat the Euler’s
method, we arrive to the results showed figure 9.11. We can see in the graphic how the mass
oscillates with increasing oscillation amplitude. To analyse this unexpected result we are going to
repeat the Euler’s method computing, this time using an integration step one order of magnitude
less. We obtain for this new calculation the behaviour showed in figure 9.12. Notice how reducing
the integrations step reverts the situation and now, the solution of the problem is reasonable, the
mass oscillations diminish until, eventually, it stops. The behaviour observed in figure 9.11 is due
to Euler’s method integration error.
In practice, there are more precise (an complex) algorithms than Euler’s to solve initial value
problems. For instance, the family of Runge-Kutta methods.
9.3. DIFFERENTIAL EQUATIONS. INITIAL VALUE PROBLEMS 297

Figure 9.10: Results achieved using Euler’s method to solve the damped oscillator problem.

∞ (

i=0

9.3.2 The Runge-Kutta methods

If we try to use Taylor’s polynomials of higher degree than that used in Euler’s method, we have
the find the problem of computing the successive derivatives of the function f . So if we want to
use the Taylor’s second-degree polynomial, to compute the values of x, we have,

h2 ′
x(ti+1 ) = x(ti ) + h · f (ti , xi ) + f (ti , yi )
2
but,

∂f (t, x) ∂f (t, x)
f ′ (t, x) = + · f (t, x)
∂t ∂x
Taking out common factor h, we get,
  
h ∂f (ti , xi ) ∂f (ti , xi )
x(ti+1 ) = x(ti ) + h f (ti , xi ) + + · f (ti , xi )
2 ∂t ∂x
This result clearly shows that developing methods for solving initial value problems based on
Taylor’s expansion is challenging. In fact, it becomes increasingly complex as we employ Tailor’s
polynomials of a higher degree.
298 CHAPTER 9. NUM. DIFF. AND INTEGR.

Figure 9.11: Result achieved using Euler’s method to solve the damped oscillator problem. K = 100

From a practical point of view, the common practice is to search for approximations to successive
Taylor’s expansion terms rather than calculate them explicitly. These approximation are, in turn,
based on Taylor’s expansion for two variable functions.
The resulting methods are known in general as the Runge-Kutta methods. Let us see how to
do it for the case we have just shown of a second-degree Taylor’s polynomial.
First, we will find the first-degree Taylor expansion polynomial for the function f (t, x) around
the point ti , xi with respect to two variables,
 
∂f (ti , xi ) ∂f (ti , xi )
f (t, x) = f (ti , xi ) + (t − ti ) + (x − xi )
∂t ∂x
If we compare this result with the previous equation, we can establish a relationship between
the coefficients accompanying the partial derivatives.

h h
t − ti = → t = ti +
2 2
h h
x − xi = · f (ti , xi ) → x = xi + · f (ti , xi )
2 2
That is,
 
h ∂f (ti , xi ) ∂f (ti , xi ) h h
f (ti , xi ) + + · f (ti , xi ) = f (ti + , xi + · f (ti , xi ))
2 ∂t ∂x 2 2
And after substituting this result in our expression for function x(t) second-degree Taylor’s
polynomial expansion,
 9.3. DIFFERENTIAL EQUATIONS. INITIAL VALUE PROBLEMS 299

Figure 9.12: Result achieved using Euler’s method to solve the damped oscillator problem. K =
100, h = 0.001

h h
x(ti+1 ) = x(ti ) + h · f (ti +
, xi + f (ti , xi ))
2 2
Where xi ≡ x(ti ). This approximation paves the way for the first and simplest of Runge-
Kutta methods, the midpoint method. This name comes from evaluating the function f at a point
midway between ti and ti+1 = ti + h. We can write the midpoint method algorithm to compute
an initial value problem solution in a similar way to Euler’s method.

x0 = x(a)
h h
x1 = x0 + h · f (a + , x0 + f (a, x0 ))
2 2
..
.
h h
xi+1 = xi + h · f (ti + , xi + f (ti , xi ))
2 2

The following Python code implement the midpoint method,

midpoint [Link]
1 # -*- coding: utf-8 -*-
2 """
3 Created on Fri Aug 23 16:10:09 2024
4 The Runge-Kutta midpoint method
 300 CHAPTER 9. NUM. DIFF. AND INTEGR.

5 @author: abierto
6 """
7 import numpy as np
8 import [Link] as pl
9 def pmedio(fun,xa,a,b,h):
10 """
11 This funtion implements the second-order Runge-Kutta method
12 better known as the midpoint method
13

14 Parameters
15 ----------
16 fin : TYPE function
17 DESCRIPTION. a funtion describing the initial value problem
18 derivative to be integrate
19 xa : TYPE numpy array
20 DESCRIPTION. initial condition
21 a : TYPE double
22 DESCRIPTION. initial integratio point
23 b : TYPE double
24 DESCRIPTION. final integration point
25 h : TYPE double
26 DESCRIPTION. integration step
27

28 Returns
29 -------
30 x: TYPE numpy array
31 DESCRIPTION. solutions at points a, a+h, a+2h etc
32 t. TYPE numpy array
33 DESCRIPTION. times x has been computed at
34 """
35 if a >= b:
36 raise ValueError('I need an increasing interval')
37

38 #to arrive to the final condition, we modify slighly the

39 #integration step. First we calculate the aproximate number
40 #of intervals between a and b of size h and round it towards
41 #infinite.
42 pt = int([Link]((b-a)/h))
43 #then, we modify the integration step to acomodate at
44 #the rounded number of interval. This is, of course, not
45 #necessary to implement the method, there are other
46 #posibilities to deal with the integration step and
47 #the limit b
48 hft = (b-a)/pt
49 t = [Link](a,b+hft,hft)
50 x = [Link]([[Link][0],pt+1])
51 x[:,0] = xa
52 for i in range(1,pt+1):
53 x[:,i] = x[:,i-1] + hft*fun(t[i-1]+hft/2,x[:,i-1]\
54 +hft*fun(t[i-1],x[:,i-1])/2)
55 [Link](t,x.T)
 9.3. DIFFERENTIAL EQUATIONS. INITIAL VALUE PROBLEMS 301

56 return(t,x)

We leave the remaining Runge-Kutta method until you are grown up. But notice that it is one
of the most interesting topics in numerical methods.

)
302 CHAPTER 9. NUM. DIFF. AND INTEGR.
Alphabetic Index

ALU, 19 Gauss-Seidel’s method., 227

Attracting fixed point, 185 Gaussian elimination method, 213

bit, 19 Interpolation
Byte, 19 Divided diferences, 245
Polynomial, 240
Convergence analysis, 233
Conversion Jacobi’s method
binary to decimal, 20 Matrix expression, 224
CPU, 18 Jacobi’s method., 222

Data, analysis, 15 Loops, 63

Diferentiation
Two point central difference, 281 Nesting
Divided diferences, 245 Nested for, 67
Numpy, 81
Fijex point
Iteration, 187 Operations
Arithmétics, 37
Fixed point, 185
theorem, 186 Python, 25
Flow
Loops, 63 SOR method, 232
Functions Speed of convergence, 235
Functions included in Numpy, 106
Taylor polynomial for the logaritm function,
Gauss-Jordan 239
elimination, 219
Gauss-Seidel’s method Weighted Jacobi method, 231
Matrix form, 229 Weighted methods, 230

303
304 ALPHABETIC INDEX
Alphabetic Index

= assignment symbol, 30 Flow

2’s complement, 150 while loop, 69
substraction, 152 flow, 57
for loop, 64
Anaconda, 26 Conditional, 57
Arithmetic errors, 163 flow control, 57
Asignment Symbol, 30 functions, 47
Base 2, 19 IEEE 237 standard
Binary, 149 Doble precision, 158
binary addition, 152 IEEE 754
Bézier, 266 Closest Number to zero in simple
Bézier’s curves, 266 precision, 157
IEEE 754 Standard, 155
Catastrophic cancellation., 167 Simple precision, 155
Compiler, 17 import, 54
Computer Initial value problems., 290
hardware, 17 Integration
architecture, 17 Newton-Cotes formulae, 283
Conversion Trapezium rule, 283
decimal to binary. Integer numbers, 20 Integration
denary to binary, non-integer numbers, Composed trapezium formula, 284
21 Interpolation
Interpolation theorem, 241
Debugger, 54
Lagrange Polynomial, 243
Diferentiation
Splines, 252
Two point forward diference, 279
The Newton-Gregory polynomial, 248
Differential equation, 290
Intrpolation
Differentiation
Vandermonde’s matrix, 241
finite differences, 279
differentiation Lagrange Polynomial, 243
interpolation polynomial, 278 Least Squared Error, 259
Linear interpolation, 251
Equivalent Bezier’s curves, 273 List comprehesion, 68
Error Loops
syntax error., 54 while loop., 69
errors accumulation, 164
Exponent machine epsilon, 161
Biased, 155 Mantissa

305
306 ALPHABETIC INDEX

normalised, 155 Scientific computing, 15

Matrices Scientific Notation, 154
in Numpy, 83 Scripts, 44
Matrix Operations, 89 Splines, 252
Spyder, 28
Nesting SVD Factorisation, 129
Nested if, 62 systems, 199
Nested While, 70
Number representation Tailor’s polynomial
floating-point, 154 Sine and consine function expansion.,
Numeric Representation, 150 240
Numpy, 81 Taylor Polynomial
Indexing, 84 Series expansion for the exponential
function, 238
operating system, 17 Taylor’s polynomial, 238
Operations, 37 Text editor, 44
Relational and Logic , 40 The Newton-Gregory polynomial, 248
Operators Truncation, 149
Precedence, 38
Overflow, 163 Underflow, 163

Polinomial fitting, 259 Variable

Programming Namespace, 50
languages, 17 Scope, 50
Programmes, 18 Type, 31
Python, 25 Variable and function scope, 53
IDE, 25 Variables, 30
Vectors
RC circuit, 293 in Numpy, 83
residuals, 266 Von Neumann, 18
Roots, 171
RUnge-Kutta methods, 297 zero-order interpolation, 251