Modeling & Simulation of Control Systems

Assoc. Prof. Nguyen Hoai Nam

Department of Automation Engineering, School of Electrical and Electronic Engineering

Hanoi University of Science and Technology
Email: [Link]@[Link]
Website: [Link]

November 3, 2023

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 1 / 73

1. Introduction

Definitions
Model classification
Applications
Model evaluation
Procedure for modeling
Methods for building mathematical model

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 2 / 73

Definitions

System (object, plant): Manipulators, Wheeled mobile robots,

Drones, DC/AC Motors, Heating process ...
Output: Observable signals of interest.
Input: Manipulated external signals
Parameter: Physical coefficients.
Disturbance: Other external stimuli
Model: How is the output related to the input of the system
(mapping, function, relationship)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 3 / 73

Model Classification

Linear model (Continuous time - CT/Discrete-time - DT):

Differential equation: ẏ + ay = u (1)

Y (s)
Transfer function: G(s) = (2)
U(s)
State space model: ẋ = Ax + Bu (3)
Nonlinear model (CT/DT):

ẋ = f (x, u) (4)

Fuzzy logic inference system: Takagi-Sugeno-Kang

Neural network:

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 4 / 73

Applications

Simulations: Flight simulators, Nuclear power station training

simulators.
Simulate the system’s response to various input scenarios.
Predictions: Model predictive control.
Observer: Estimate the system state (Luenberger observer).
Control: Model based controller design

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 5 / 73

Model Validation

1. Does the model agree sufficiently well with the observed data?
2. Is the model good enough for the application?
3. Does the model describe the ”’true system”?

In principle, model validation is done by comparing the model’s

behavior with the system’s behavior.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 6 / 73

Procedure for modeling

Physical modeling: Break down the properties of the system to

subsystems whose behaviors are known (the laws of nature).
Identification: Use observations from the system to fit the model.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 7 / 73

Methods for building mathematical model

Time domain method: Transient response analysis

Frequency domain method
Parameter estimation method: Minimizing prediction errors

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 8 / 73

Chapter 2 Modeling

2.1 Models
2.2 Mathematical model
2.3 Theoretical model
2.4 Practical model
2.5 HIL Simulation

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 9 / 73

2.1 Models

Mathematical model: Differential/difference equation, transfer

function, state space model.
Theoretical model: Basic laws.
Practical model: Data and model

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 10 / 73

2.2 Mathematical model.

Differential equation:

SISO linear system

d ny d n−1 y dy d mu d m−1 u
+ a n−1 + ... + a1 + a0 y = + b m−1 + ...
dt n dt n−1 dt dt m dt m−1
du
+ b1 + b0 u
dt
(5)
m ≤ n, u : the input, y : output.
SISO nonlinear system:

d ny  d n−1 y dy d m u d m−1 u du 
= f , ..., , y, , , ..., ,u (6)
dt n dt n−1 dt dt m dt m−1 dt

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 11 / 73

2.2 Mathematical model.

Difference equation
y(n + k) =f (y(n − 1 + k), y(n − 2 + k, ..., y(k), u(m + k ), u(m − 1 + k), ..., u(k)). (7)

0 ≤ m < n, f : linear or nonlinear.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 12 / 73

2.2 Mathematical model.

Transfer function
Y (s)
G(s) = (8)
U(s)
Laplace transform
Z ∞
Y (s) = y(t)e−st dt (9)
0

Y (z)
G(z) = (10)
U(z)
Z transfrom:
∞
X
Y (z) = y (k )z −k (11)
0

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 13 / 73

2.2 Mathematical model.

State space model:

Continuous time system:

ẋ = Ax + Bu, A: square matrix (12)

Discrete time system:

x(k + 1) = Ax(k) + Bu(k ), A: square matrix (13)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 14 / 73

Neural network model

Artificial neuron (weight, bias, transfer function).

Layer: Neurons have the same input vector.
Neural network: Layers in series (feedback)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 15 / 73

Fuzzy models

Fuzzy set, membership function

Linguistic varible
IF-Then rule
Inference function
Defuzzification

Mamdani, Sugeno models.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 16 / 73

2.3 Physical (theoretical) model

Three phases of obtaining a mathematical model:

1 Problem is structured.
2 Formulate basic equations
3 Form state-space model

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 17 / 73

Structured problem

Divide the system into subsystems. Determine cause and effect,

important variables, interaction.
Understanding of the physical system, intuition for the physical
system.
Level of complexity, degree of approximation.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 18 / 73

Formulate basic equations

Relationships between variables and constants in the subsystems.

Laws of nature, basic physical equations (assumed to hold).
Certain approximations, and idealizations (point mass, ideal gas
...) to avoid complicated expressions.
Non-technical system, use new hypotheses and innovative
thinking.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 19 / 73

Formulate the state-space model

Suitable model for analysis and simulation.

State-space models for the subsystems together with instructions
for the interconnections.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 20 / 73

2.3 Theoretical (physical) model
Electrical circuits:

Ideal inductor (Law of inductance):

Z t
d 1
u(t) = L i(t) or i(t) = u(τ )dτ (14)
dt L 0
i: current (ampere-A), u: voltage (volt-V), L: inductance (henry-H)
Ideal capacitor
1 t
Z
d
i(t) = C u(t) or u(t) = i(τ )dτ (15)
dt C 0
C : capacitance (farad)
Linear resistor (Ohm’s law)
u(t) = Ri(t) (16)
R : resistance (ohm)
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 21 / 73
2.3 Theoretical (physical) model

Kirchhoff’s current law (1st Law): The current flowing into a node
(or a junction) must be equal to the current flowing out of it
(consequence of charge conservation).
Kirchhoff’s voltage law (2nd Law): In any complete loop within a
circuit, the sum of all voltages across components which supply
electrical energy must equal the sum of all voltages across the
other components in the same loop (consequence of both charge
conservation and the conservation of energy).

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 22 / 73

2.3 Theoretical (physical) model

Figure 1: RLC electrical circuit.

L: inductance (henry), R: resistance (ohm), C: capacitance (farad).

Differential equations:
1 t
Z
di
L + Ri + idτ =ei
dt C 0
(17)
1 t
Z
idτ =eo
C 0
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 23 / 73
2.3 Theoretical (physical) model

Applying Laplace transform:

1
LsI(s) + RI(s) + I(s) =Ei (s)
Cs (18)
1
I(s) =Eo (s)
Cs
Transfer function:
Eo 1
= 2
(19)
Ei LCs + RCs + 1
R 1 1
ëo + ėo + eo = ei (20)
L LC LC

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 24 / 73

2.3 Theoretical (physical) model

R 1 1
ëo + ėo + eo = ei (21)
L LC LC
1 R 1
Define x1 = eo , x2 = ėo . Then, ẋ1 = x2 , ẋ2 = − LC x1 − L x2 + LC ei .
State-space model:

ẋ = Ax + Bu, y = Cx (22)
   
0 1 0
A= 1 R ,B = 1 , and C = [0 1].
− LC −L LC

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 25 / 73

2.3 Theoretical (physical) model

Figure 2: Cascaded elements.

Eo
Find Ei .

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 26 / 73

2.3 Theoretical (physical) model

Figure 3: Operational amplifier (op amps)

eo = K (e2 − e1 ) (23)
K ≈ 105 ÷ 106 for dc signals and ac signals with small frequencies
(≤ 10Hz).

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 27 / 73

2.3 Theoretical (physical) model

Figure 4: Inverting operational amplifier

ei − e′ e ′ − eo
i1 = , i2 = (24)
R1 R2
Since e′ ≈ 0 and only a negligible current flows into the amplifier,
i1 = i2 .
R2
eo = − ei (25)
R1

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 28 / 73

2.3 Theoretical (physical) model

Figure 5: Non-inverting operational amplifier (left), equivalent circuit (right).

R2
eo = (1 + )ei (26)
R1

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 29 / 73

2.3 Theoretical (physical) model

Figure 6: First-order lag circuit using operational amplifier.

Eo R2 1
=− (27)
Ei R1 R2 Cs + 1

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 30 / 73

2.3 Theoretical (physical) model

Figure 7: Op. Amp. circuit used as a lead or lag compensator.

Eo Ts + 1 s + T1
= Kc α = Kc 1
(28)
Ei αTs + 1 s + αT
R4 C1 R2 R4
where T = R1 C1 , αT = R2 C2 , Kc = R3 C2 , Kc α = R1 R3 .

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 31 / 73

2.3 Theoretical (physical) model

Figure 8: Electronic PID controller.

Eo Ti
= Kp (1 + + Td s) (29)
Ei s
R4 (R1 C1 +R2 C2 ) 1
Proportional gain: Kp = R3 R1 C2 . Integral time: Ti = R1 C1 +R2 C2 .
R1 C1 R2 C2
Derivative time: Td = R1 C1 +R2 C2 .

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 32 / 73

2.3 Theoretical (physical) model

Mechanical systems:
Newton’s second law: X
ma = Fi (30)
P
m: mass, a: acceleration of the mass, Fi : the sum of forces Fi
acting on the mass.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 33 / 73

2.3 Theoretical (physical) model

Figure 9: Two springs in parallel.

F = k1 x + k2 x = keq x (31)
k1 , k2 > 0: spring constants (N/m)
F : spring force
keq = k1 + k2 (32)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 34 / 73

2.3 Theoretical (physical) model

Figure 10: Two springs in series.

F =k1 y = k2 (x − y ) (33)
k1 , k2 > 0: spring constants (N/m)
F : spring force
k1 k2
keq = (34)
k1 + k2

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 35 / 73

2.3 Theoretical (physical) model

A dashpot ( oil-filled damper ) is a device that provides viscous friction,

or damping, consisting of a piston and oil-filled cylinder.

Figure 11: Two dampers connected in parallel.

f = b1 (ẏ − ẋ) + b2 (ẏ − ẋ) = (b1 + b2 )(ẏ − ẋ) (35)

f = beq (ẏ − ẋ), beq = b1 + b2 , b1 , b2 viscous-friction coefficients (36)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 36 / 73

2.3 Theoretical (physical) model

Figure 12: Two dampers connected in series.

f = b1 (ż − ẋ) = b2 (ẏ − ż) (37)

b1 b2
f = beq (ẏ − ẋ), beq = (38)
b1 + b2

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 37 / 73

2.3 Theoretical (physical) model

Figure 13: Spring-mass-dashpot system mounted on a cart.

u, y : displacements of the cart and mass, resp. (relative to ground).

The cart and the spring-mass-dashpot system are standing still for
t < 0, the cart is moved at a constant speed at t = 0.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 38 / 73

2.3 Theoretical (physical) model

Newton’s second law: X

ma = F (39)
P
m: mass, a: acceleration of the mass, F : the sum of forces acting
on the mass.

d 2y dy du
2
= −b(
m − ) − k (y − u). (40)
dt dt dt
Differential equation:

d 2y dy du
m +b + ky = b + ku. (41)
dt 2 dt dt

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 39 / 73

2.3 Theoretical (physical) model

Transfer function:
Y (s) bs + k B(s)
G(s) = = 2
= (42)
U(s) ms + bs + k A(s)
U(s)
Denote X1 (s) = A(s) and X2 (s) = s U(s)
A(s) → ẋ1 = x2 .
k b 1
ms2 X1 + bsX1 + kX1 = U → ẋ2 = − m x1 − m x2 + m u.
Y (s) = (bs + k)X1 (s) → y = kx1 + bx2 .
State-space model (controllable canonical form):

ẋ = Ax + Bu, y = Cx (43)
   
0 1 0
where A = k b , B = 1 , C = [k b].
−m −m m

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 40 / 73

2.3 Theoretical (physical) model

Liquid-level system:
The flow through a short pipe connecting two tanks.

Resistance of liquid-level systems:

change in level difference  m 

R= , (44)
change in flow rate m3 /sec
Capacitance of liquid-level systems:

change in liquid stored  m3 

C= , (45)
change in head m

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 41 / 73

2.3 Theoretical (physical) model
Laminar flow:
Q = KH (46)
Q: steady-state liquid flow rate (m3 /sec), H: ss. head (m)
K : coefficient (m2 /sec)
dH H 1
Rlemina flow = = = (47)
dQ Q K

Figure 14: Liquid-level system

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 42 / 73

2.3 Theoretical (physical) model

Turbulent flow: √
Q=K H (48)
dH 2H
Rturbulent flow = = (49)
dQ Q

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 43 / 73

2.3 Theoretical (physical) model

Figure 15: Liquid-level system.

Q̄: ss. flow rate (before any change), (m3 /sec).

qi : small deviation of inflow rate from its ss. value (m3 /sec)
qo : small deviation of outflow rate from its ss. value (m3 /sec)
H̄: ss. head (m)
h: small deviation of head from its ss. value (m)
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 44 / 73
2.3 Theoretical (physical) model

The inflow minus outflow during the small time interval is equal to the
additional amount stored in the tank.

Cdh = (qi − qo )dt (50)

h
qo = , by definition of resistance (51)
R
dh
RC + h = Rqi , R is constant (52)
dt

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 45 / 73

2.3 Theoretical (physical) model

Figure 16: Liquid-level system with interaction.

Q̄: steady state flow rate

H̄1 : ss. liquid level of tank 1
H̄2 : ss. liquid level of tank 2

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 46 / 73

2.3 Theoretical (physical) model

Figure 17: Liquid-level system with interaction.

h1 − h2 dh1
= q1 , C1 = q − q1 (53)
R1 dt
h2 dh2
= q2 , C2 = q1 − q2 (54)
R2 dt
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 47 / 73
2.3 Theoretical (physical) model

Figure 18: Block diagram of the system.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 48 / 73

2.3 Theoretical (physical) model

Q2 1
= 2
(55)
Q R1 C1 R2 C2 s + (R1 C1 + R2 C2 + R2 C1 )s + 1

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 49 / 73

2.4 Practical model

Transient analysis
Linear regression

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 50 / 73

Transient analysis

Y (s)
G(s) = (56)
U(s)

Impulse response: u(t) = δ(t) or U(s) = 1.

G(s) = Y (s) or g(t) = y (t) (57)

Unit-step response: u(t) = 1(t) or U(s) = 1s .

G(s)
Y (s) = or y(t) = L−1 {G(s)/s} (58)
s

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 51 / 73

First-order system

Figure 19: A RC low-pass filter.

R: Resistance, C: capacitance.

Vout 1
= (59)
Vin 1 + RCs

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 52 / 73

First-order system

Figure 20: A thermal system and its block diagram

R: thermal resistance, C: thermal capacitance, θi : temperature

change of in-flowing liquid (o C), θ: temperature change of out-flowing
liquid (o C), hi : change in the heat input rate to the system.

Θ R
= (60)
Hi RCs + 1

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 53 / 73

First-order system

k
G(s) = (61)
Ts + 1
Unit-step response:
t
c(t) = k (1 − e− T ), for t ≥ 0. (62)

Unit-ramp response:
t
c(t) = k (t − T + Te− T ), for t ≥ 0. (63)

Unit-impulse response:

k −t
c(t) = e T, for t ≥ 0. (64)
T

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 54 / 73

Determine k and T

From the unit-step response

t
c(t) = k (1 − e− T ), for t ≥ 0. (65)

k = c(∞) (66)
c(T ) = k (1 − e−1 ) = 0.6321k. (67)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 55 / 73

Second-order system
Servo-system:
C 1 C K /J
Load: = Closed-loop TF: = (68)
T s(Js + b) R s2 + B
J
s + KJ

J: moment of inertia referred to the output shaft, B: viscous-friction coefficient referred to the output shaft, K : controller gain.

Figure 21: Servo system, block diagram, and simplified diagram.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 56 / 73
Second-order system

ωn2
G(s) = (69)
s2 + 2ζωn s + ωn2
ωn : undamped natural frequency, ζ: damping ratio of the system.

Underdamped system: 0 < ζ < 1.

Critically damped system: ζ = 1.
Overdamped system: ζ > 1.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 57 / 73

Unit-step response of the underdamped system.

ωn2
C(s) =
s(s2 + 2ζωn s + ωn2 )
1 s + 2ζωn p
= − 2 2
, ωd = ωn 1 − ζ 2 (damped nature frequency.).
s s + 2ζωn s + ωn
1 s + ζωn ζωn
= − 2
−
2
s (s + ζωn ) + ωd (s + ζωn )2 + ωd2
 ζ 
c(t) =1 − e−ζωn t cos(ωd t) + p sin(ωd t)
1 − ζ2
e−ζωn t p 
=1 − p 1 − ζ 2 cos(ωd t) + ζsin(ωd t)
1 − ζ2
p
e−ζωn t −1 1 − ζ2
=1 − p sin(ωd t + θ), θ = tan .
1 − ζ2 ζ
(70)
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 58 / 73
ζ=0

c(t) = 1 − cos(ωn t) (71)

The response becomes undamped and oscillations continue
indefinitely.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 59 / 73

Critically damped system: ζ = 1.

ωn2
C(s) = (72)
s(s + ωn )2
c(t) = 1 − e−ωn t (1 + ωn t) (73)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 60 / 73

Overdamped system: ζ > 1.

ωn2
C(s) = (74)
s(s2 + 2ζωn s + ωn2 )
ωn  e−s1 t e−s2 t 
c(t) = 1 + p − (75)
2 ζ 2 − 1 s1 s2
p p
s1 = (ζ + ζ 2 − 1)ωn , s2 = (ζ − ζ 2 − 1)ωn

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 61 / 73

Determine ζ and ωn

∂c(t) e−ζωn t h i
=p ζωn sin(ωd t + θ) − ωd cos(ωd t + θ)
∂t 1 − ζ2
=0
p
1 − ζ2
→ tan(ωd t + θ) =
ζ
ωd Tk =k π, k = 0, 1, 2, ...
(76)
kπ
Tk = p
ωn 1 − ζ 2
π
tp = T1 = p → ωn
ωn 1 − ζ 2
Mp =c(tp ) − 1
√−πζ
=e 1−ζ 2 →ζ
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 62 / 73
Linear regression

ŷ(t|θ) = θT ϕ(t) (77)

 T
θ = a1 a2 . . . an b1 b2 . . . bm . (78)
ϕ(t) =[−y (t − 1) − y (t − 2) . . . − y (t − n)
(79)
u(t − nb ) u(t − nb − 1) . . . u(t − nb − m + 1)]T
Prediction error:
ϵ(t, θ) = y (t) − ŷ (t|θ) (80)
Cost function:
N
1X 2
JN (θ) = ϵ (t, θ) (81)
N
t=1

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 63 / 73

AI based modeling methods

Fuzzy logic inference system

Neural network

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 64 / 73

Fuzzy logic inference system
Sugeno model:

R1 : IF χ is A1 THEN y = b1,1 x + b1,2 OR

R2 : IF χ is A2 THEN y = b2,1 x + b2,2 OR
...
Rn : IF χ is An THEN y = bn,1 x + bn,2

αi = gbellmf (x, [αi ; βi ; γi ]) is matching degree, χ is linguistic variable,

Ai is a value of χ, µAi (x) = gbellmf (x, [αi ; βi ; γi ]) is a membership
function, αi , βi , γi , bi,j are parameters.
Pn n
i=1 αi yi
X
yf = Pn (WTAVER) OR yf = αi yi (WTSUM) (82)
i=1 αi i=1

yi = bi,1 x + bi,2 .
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 65 / 73
Fuzzy logic inference system
Sugeno model:

R1 : IF χ1 is A1,1 AND χ2 is A2,1 THEN y = a1 x1 + b1 x2 + c1 OR

R2 : IF χ1 is A1,2 AND χ2 is A2,2 THEN y = a2 x1 + b2 x2 + c2 OR
...
Rn : IF χ1 is A1,n AND χ2 is A2,n THEN y = an x1 + bn x2 + cn

αi = αi,1 αi,2 is matching degree, χi is the i th linguistic variable, Aj,i is a

value of χj , µAj,i (x) is a membership function.
Pn n
αi yi X
yf = Pi=1
n (WTAVER) OR yf = αi yi (WTSUM) (83)
i=1 αi i=1

yi = ai x1 + bi x2 + ci .
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 66 / 73
Neural network

Figure 22: Neural model.

w weight, b bias, p input, n net input, f transfer function, a neuron

output.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 67 / 73

Neural network

Figure 23: Model with multiple-input.

R
X
n= w1,i pi + b, a = f (n) (84)
i=1

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 68 / 73

Artificial neuron

Figure 24: Simplified diagram

p =[p1,1 p1,2 . . . p1,i . . . p1,R ]T

W =[w1,1 w1,2 . . . w1,i . . . w1,R ] (85)
n =W p + b, a = f (n)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 69 / 73

Layer of neurons

Figure 25: One layer network

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 70 / 73

Layer of neurons

 
  b1
w1,1 w1,2 . . . w1,R−1 w1,R
 w2,1 w2,2 . . . w2,R−1 w2,R 

 b2

W= .
 
b=
 ..

(86)
. .. .. ..  .

 . . . .   
bS−1 
wS,1 wS,2 . . . wS,R−1 wS,R S×R
bS S×1

n =Wp + b
(87)
a =f(n)

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 71 / 73

 Figure 26: Simplified diagram.

Assoc. Prof. Nguyen Hoai Nam November 3, 2023 72 / 73

Multiple Layer Network

Figure 27: Two-layer network.

Network’s output:
n1 =W1 p + b1
a1 =f1 (n1 )
(88)
n2 =W2 p + b2
a2 =f2 (n2 )
Assoc. Prof. Nguyen Hoai Nam November 3, 2023 73 / 73