CHAPTER 2

(Part B)
NEURAL NETWORK FOR CONTROL
Outline
 Neural Network for Control:
Learning and coding, supervised learning,
unsupervised learning

Learning Outcome
 Able to differentiate between supervised and
unsupervised learning
 ANN Model

• Generally, an ANN is categorized by:

 Its pattern of connections between the neurons
(also called its architecture or model).
 the activation function used in the neurons.
 Its learning algorithm (which is the method of
determining its weights).
 ANN Model

• Generally, an ANN is categorized by:

 Its pattern of connections between the neurons (also
called its architecture or model).
 the activation function used in the neurons.
 Its learning algorithm (which is the method of
determining its weights).
 ANN Model (cont)

• Usually ANNs can be categorized into 2 models:

Feedforward
• All signals flow in one direction only, i.e. from lower
layers (input) to upper layers (output)

Feedback
• Signals from neurons in upper layers are fed back to
either its own or to neurons in lower layers.
 Learning in ANN
• In all of the neural paradigms, the application of an ANN
involves two phases:
1. Learning phase
2. Recall phase

• In the learning phase (usually offline) the ANN is trained until

it has learned its tasks (through the adaptation of its weights)
while the recall phase is used to solve the task.
• An ANN solves a task when its weights are adapted through a
learning phase.
• All neural networks have to be trained before they can be
used.
 Learning in ANN (cont)
• They are given training patterns and their weights are
adjusted iteratively until an error function is minimized.
• Once the ANN has been trained no more training is
needed.
• Two types of learning prevailed in ANNs:
1. Supervised learning:- learning with teacher signals or
targets
2. Unsupervised learning:- learning without the use of
teacher signals
Classification of Learning Algorithm
 Supervised Learning

• In supervised learning the training patterns are provided

to the ANN together with a teaching signal or target.
• The difference between the ANN output and the target is
the error signal.
• Initially the output of the ANN gives a large error during
the learning phase.
• The error is then minimized through continuous
adaptation of the weights to solve the problem through a
learning algorithm.
• In the end when the error becomes very small, the ANN is
assumed to have learned the task and training is stopped.
 Supervised Learning (cont)

• It can then be used to solve the task in the

recall phase.
 Unsupervised Learning
• In unsupervised learning, the ANN is trained without
teaching signals or targets.
• It is only supplied with examples of the input patterns
that it will solve eventually.
• The ANN usually has an auxilliary cost function which
needs to be minimized like an energy function, distance,
etc.
• Usually a neuron is designated as a “winner” from
similarities in the input patterns through competition.
• The weights of the ANN are modified where a cost
function is minimized.
 Unsupervised Learning (cont)

• At the end of the learning phase, the weights would have

been adapted in such a manner such that similar
patterns are clustered into a particular node.
 Simple Neural Networks
Introduction
• Also called primitive neural networks.
• Mainly used as pattern classifiers.
• Usually are single layer in architecture.
• Used in the 40s-60s for simple applications such as
• membership in a single-class (i.e. either “in’ or “out”).
 Examples of ANNs

• Several examples of these neural networks

 McCulloch-Pitts neuron ~ 1st artificial neuron
 Hebb net ~ 1st implementation of learning in neural
nets
 The Perceptron
 ADALINE and MADALINE
 The McCulloch-Pitts Neuron

• Uses only binary activation signals.

Example to solve an AND function

?
Example to solve an OR function

?
?
?
?
 The Hebb Neural Network
• Developed by Donald Hebb, a psychologist, in 1949.
• Actually he developed the first learning algorithm for
neural networks where weights are adapted iteratively.
• It can be used with patterns that are represented as
either binary or bipolar vectors.
• Has a number of limitations and cannot be used in real
world applications.
 The Perceptrons
• Developed by Frank Rosenblatt (1958).
• Its learning rule is superior than the Hebb learning rule.
• Has been proven by Rosenblatt that the weights can
converge on particular applications.
• However, the Perceptron does not work for nonlinear
applications as proven by Minsky and Papert (1969).
• Activation function used is the binary step function with
an arbitrary, but fixed threshold.
• Weights are adjusted by the Perceptron learning rule:
 Steps of Perceptron Algorithm
Step 1: Set up the NN model (which follows the problem to be
solved) Initialize weights and bias.
(For simplicity, set weights and bias to zero or randomize)
Set learning rate, α (0<α<1) and threshold (0<θ<1)
(For simplicity, α can be set to 1.)
Step 2: While stopping condition is false, do Steps 3-7.
Step 3: For each training pair u:t, do Steps 4-6.
Step 4: Set activations of input units: xi = ui .
Step 5: Compute response of output unit:θ
 Cont.
Step 6: Update weights and bias if an error occurred for this
pattern:

Step 7: Test stopping condition :

If no weights changed in Step 3, stop; else, continue.
 Exercise 1

Q1. The input to a single-input neuron is 2.0, its weight is

2.3 and its bias is -3.
i. What is the net input to the transfer function?
ii. What is the neuron output?

[Link] is the output of the neuron of Q1 if it has the

following transfer functions?
i. Hard limit
ii. Linear
iii. Log-sigmoid
Activation Function / Transfer Function
 Exercise 2
Q1. Given a two-input neuron with the following parameters: b=1.2, W= [3 2]
and p=[-5 6]T ,calculate the neuron output for the following transfer
functions:
i. A symmetrical hard limit transfer function
ii. A saturating linear transfer function
iii. A hyperbolic tangent sigmoid (tansig) transfer function

Q2. A single-layer neural network is to have six inputs and two outputs. The
outputs are to be limited to and continuous over the range 0 to 1. What can
you tell about the network architecture?
Specifically:
i. How many neurons are required?
ii. What are the dimensions of the weight matrix?
iii. What kind of transfer functions could be used?
iv. Is a bias required?
 Multilayer Neural Network
• Multilayer neural networks are feedforward ANN models which
are also referred to as multilayer perceptrons-MLP.
• The addition of a hidden layer of neurons in the perceptron
allows the solution of nonlinear problems many practical
applications (using the backpropagation algorithm).
• However, the difficulty of adaptation of the weights between the
hidden and input layers of the multilayer perceptrons have
dampen such architecture during the sixties.
• With the discovery of the backpropagation algorithm by
Rumelhart, Hinton and Williams in 1985, the adaptation of the
weights in the lower layers of multilayer neural networks are
now possible.
 Backpropagation (BP) Algorithm

• The BP algorithm is perhaps the most popular and

widely used neural paradigm.
• The BP algorithm is based on the generalized delta rule
proposed by Dave Rumelhart based at Stanford
University, California, U.S.A..
• The BP algorithm overcame the limitations of the
perceptron algorithm.
• Among the first applications of the BP algorithm is
speech synthesis called NETalk developed by Terence
Sejnowski.
 Learning Mode
• Before the BP can be used, it requires target patterns or
signals as it a supervised learning algorithm.
• Training patterns are obtained from the samples of the
types of inputs to be given to the multilayer neural
network and their answers are identified by the
researcher.
• Examples of training patterns are samples of handwritten
characters, process data, etc. following the tasks to be
solved.
 Learning Mode (cont)
• The configuration for training a neural network using the
BP algorithm is shown in the figure below in which the
training is done offline.
• The objective is to minimize the error between the target
and actual output and to find Δw.
 Learning Mode (cont)

• The error is calculated at every iteration and is

backpropagated through the layers of the ANN to adapt
the weights.
• The weights are adapted such that the error is
minimized.
• Once the error has reached a justified minimum value,
the training is stopped, and the neural network is
reconfigured in the recall mode to solve the task.
 The Generalized Delta Rule
(G.D.R.)
• The objective of the BP algorithm, like in other learning
algorithms, is to find the next value of the adaptation
weights (Δw) which is also known as the G.D.R..
• We consider the following ANN model:
 G.D.R. (cont)
• What we need is to obtain the following algorithm to
adapt the weights between the output (k) and hidden (j)
layers:

• where the weights are adapted as follows:

• Wkj (t+1) = Wkj(t) ΔWkj (t+1)

where t is the iteration number and δk is the error signal between

the output and hidden layers:
 G.D.R. (cont)

Adaptation between input (i) and hidden (j) layers :

η- learning rate
The new weight is thus:

and the error signal through layer j is:

Note that
 Steps of the BP Algorithm

Step 1: Obtain a set of training patterns.

Step 2: Set up neural network model:
No. of Input neurons, Hidden neurons, and
Output Neurons.
Step 3: Set learning rate η and momentum rate α
Step 4: Initialize all connection Wji, Wkj and bias
weights θj θk to random values.
Step 5: Set minimum error, Emin
 Steps of the BP Algorithm (cont)

Step 6: Start training by applying input patterns

one at a time and propagate through the
layers then calculate total error.
Step 7: Backpropagate error through output and
hidden layer and adapt weights.
Step 8: Backpropagate error through hidden and
input layer and adapt weights.
Step 9: Check it Error < Emin
If not repeat Steps 6-9. If yes stop training.
 Example - XOR
• For simplicity, the ANN model has only 4 neurons (2 inputs-i,
1 hidden-j and 1 output-k) and has no bias weights.
• The input neurons have linear functions and the hidden and
output neurons have sigmoid functions.
• The weights are initialised randomly.
• We train the ANN by providing the patterns #1 to #4 through
an iteration process until the error is minimized.
 Example - XOR

The ANN model with initial weights.

Training begins when the pattern#1 and its

target are provided to the ANN.
1st pattern: 0, 0 target : 0
(see slide 37)
 Exercises
1. Describe in detail the major
components of a neural network.
2. Explain supervised and unsupervised
training.
3. How does the recurrent (feedback)
network learn?
4. Differentiate single layer and multi layer
neural network.