Chapter 4: How a Machine Learns
• Understanding Layers
• Supervised Training
• Unsupervised Training
• Error Calculation
There are many different ways that a neural network can learn; however, every
learning algorithm involves the modification of the weight matrix, which holds the
weights for the connections between the neurons.
Learning Methods
Training is a very important process for a neural network. There are two forms of
training that can be employed, supervised and unsupervised. Supervised training in-
volves providing the neural network with training sets and the anticipated output. In
unsupervised training, the neural network is also provided with training sets, but not
with anticipated outputs.
Unsupervised Training
What does it mean to train a neural network without supervision? As previously
mentioned, the neural network is provided with training sets, which are collections of
defined input values. The unsupervised neural network is not provided with antici-
pated outputs.
Unsupervised training is typically used to train classification neural networks. A
classification neural network receives input patterns, which are presented to the in-
put neurons. These input patterns are then processed, causing a single neuron on the
output layer to fire. This firing neuron provides the classification for the pattern and
identifies to which group the pattern belongs.
Another common application for unsupervised training is data mining. In this
case, you have a large amount of data to be searched, but you may not know exactly
what you are looking for. You want the neural network to classify this data into several
groups. You do not want to dictate to the neural network ahead of time which input
pattern should be classified into which group. As the neural network trains, the input
patterns fall into groups with other inputs having similar characteristics. This allows
you to see which input patterns share similarities.
Unsupervised training is also a very common training technique for self-organiz-
ing maps (SOM), also called Kohonen neural networks.
Figure 4.1 illustrates the flow of information through an unsupervised training
algorithm.
Figure 4.1: Unsupervised training.
Start
Reset back to first
training pattern to try
again.
Any more Increase epoch
training patterns? No
counter by 1.
Yes
Epochs < 100? Yes
Adjust weights
based on Hebb's
rule or similar.
No
Stop
Supervised Training
The supervised training method is similar to the unsupervised training method,
in that training sets are provided. Just as with unsupervised training, these training
sets specify input signals to the neural network. The primary difference between su-
pervised and unsupervised training is that in supervised training the expected outputs
are provided. This allows the neural network to adjust the values in the weight matrix
based on the differences between the anticipated output and the actual output.
There are several popular supervised training algorithms. One of the most com-
mon is the backpropagation algorithm.
Figure 4.2 illustrates the flow of information through a supervised training algo-
rithm.
Figure 4.2: Supervised training.
Start
Reset back to first
training pattern to try
again.
Was this the
Any more Store weights as
No best error rate Yes
training patterns? best set.
yet?
Yes No
Adjust weights
based on Increase epoch
backpropagation counter by 1.
rule or similar,
calculate error.
Epochs < 100? Yes
No
Stop