A newborn baby has about 26,000,000,000 cells.
The human body is made up of trillions of cells. Cells of the nervous system, called
nerve cells or neurons, are specialized to carry "messages" through an
electrochemical process.
Neurons come in many different shapes and sizes. Some of the smallest neurons
have cell bodies that are only 4 microns wide. Some of the biggest neurons have
cell bodies that are 100 microns wide.
Typically, a neuron contains three important parts:
•a cell body that directs all activities of the neuron.
•dendrites (the part that looks like tree branches), which are short fibers that
receive messages from other neurons and relay those messages to the cell
body.
•axon, a long single fiber that transmits messages from the cell body to
dendrites of other neurons.
• The study of neurons, their
interconnections, and their role as the
brain’s elementary building blocks is one
of the most dynamic and important
research fields in modern biology.
• Between 1901 and 1991 approximately
ten percent of the Nobel Prizes for
Physiology and Medicine were awarded to
scientists who contributed to the
understanding of the brain.
• Artificial neural networks are an attempt
at modeling the information processing
capabilities of nervous systems.
• Artificial neural networks can be
considered as another approach to the
problem of computation.
What are Neural Networks used for?
There are two basic goals for neural network research:
Brain modelling: The biological goal of constructing models of how real brains work.
This can potentially help us understand the nature of perception, actions, learning and
memory, thought and intelligence and/or formulate medical solutions to brain
damaged patients
Artificial System Construction: The engineering goal of building efficient systems for
real world applications. This may make machines more powerful and intelligent, relieve
humans of tedious tasks, and may even improve upon human performance.
Warren McCulloch Walter Pitts
1943, "A Logical Calculus of the Ideas Immanent
in Nervous Activity". With Walter Pitts.
In: Bulletin of Mathematical Biophysics Vol 5, pp
115–133.
The McCulloch-Pitts Model of Neuron (1942 model)
The early model of an artificial neuron is introduced by Warren McCulloch and Walter
Pitts in 1943.
The main feature of their neuron model is that a
weighted sum of input signals is compared to a
threshold to determine the neuron output.
When the sum is greater than or equal to the threshold,
the output is 1.
When the sum is less than the threshold, the output is
0.
They demonstrated that networks of these neurons could, in principle, compute any
arithmetic or logical function.
Unlike biological networks, the parameters of their networks had to be designed, as no
training method was available.
However, the perceived connection between biology and digital computers generated a
great deal of interest
Implementation of basic logic gates with McCulloch-Pitts Model
Frank Rosenblatt
Rosenblatt, Frank (1958), The Perceptron: A Probabilistic
Model for Information Storage and Organization in the
Brain, Cornell Aeronautical Laboratory, Psychological
Review, v65, No. 6, pp. 386–408.
• The neurons in these networks were similar to those of McCulloch and Pitts.
• Rosenblatt’s key contribution was the introduction of a learning rule for training
perceptron networks to solve pattern recognition problems
• In addition to the variable weight values, the perceptron model added an extra
input that represents bias. Thus, the modified equation from is now as follows:
Bias is a measure of how easy it is to get the perceptron to output 1.
Perceptron learning rule
𝑊 𝑖 + 1 = 𝑊 𝑖 + ∆𝑊
∆𝑊 = 𝜂. 𝐸𝑟𝑟𝑜𝑟. 𝐼
𝐸𝑟𝑟𝑜𝑟 = 𝑇 − 𝑌
𝑏 𝑖 + 1 = 𝑏 + 𝜂𝐸
Hebb, D.O. (1949). The Organization of Behavior. New York: Wiley & Sons.
Perceptron convergence theorem
The Perceptron convergence theorem states that for any data set which is linearly separable
the Perceptron learning rule is guaranteed to find a solution in a finite number of steps.
XOR case