Structure of Biological neuron
In the context of soft computing and inspired by biological neural networks, a neuron consists of a
cell body (soma), dendrites (receiving inputs), an axon (transmitting signals), and synapses
(connections between neurons).
Soma (Cell Body):
This is the main part of the neuron where the nucleus and other essential cellular components are
located. It receives and integrates signals from other neurons.
Dendrites:
These are branched extensions that receive signals from other neurons, acting as input channels.
Axon:
This is a long, slender projection that transmits signals away from the cell body to other neurons or
target cells.
Synapses:
These are the junctions where the axon of one neuron connects to the dendrites of another neuron,
allowing for the transmission of signals.
Neuron Structure:
Dendrites: Branch-like extensions that receive signals from other neurons.
Cell Body (Soma): Contains the nucleus and other organelles, responsible for maintaining
the neuron's function.
Axon: A long, slender projection that carries electrical signals away from the cell body.
Axon Terminals: The end of the axon, where synapses are formed.
Synapse Structure and Function:
Synaptic Cleft: The small gap between the axon terminal of one neuron (presynaptic neuron)
and the dendrite or cell body of another neuron (postsynaptic neuron).
Presynaptic Neuron: The neuron that sends the signal.
Postsynaptic Neuron: The neuron that receives the signal.
Neurotransmitters: Chemical messengers released by the presynaptic neuron into the
synaptic cleft.
Receptors: Specialized proteins on the postsynaptic neuron's membrane that bind to
neurotransmitters.
Signal Transmission:
o An electrical signal (action potential) travels down the axon of the presynaptic
neuron.
o At the axon terminal, the electrical signal triggers the release of neurotransmitters
into the synaptic cleft.
o Neurotransmitters diffuse across the synaptic cleft and bind to receptors on the
postsynaptic neuron.
o Binding of neurotransmitters to receptors can cause the postsynaptic neuron to
become more or less likely to fire its own action potential.
Types of Synapses:
o Chemical Synapses: Use neurotransmitters to transmit signals across the synaptic
cleft.
o Electrical Synapses: Allow direct ionic flow between cells through gap junctions.
Biological Neuron vs Artificial Neuron: A
Comparison
Biological neurons and artificial neurons have some similarities and
differences in their structure and function. Here are some of the
main points of comparison:
Structure: Biological neurons have a complex and organic
structure, consisting of dendrites, soma, axon, and synapses.
Artificial neurons have a simple and mathematical structure,
consisting of inputs, weights, bias, and activation function.
Function: Biological neurons process and transmit electrical and
chemical signals, using action potentials and neurotransmitters.
Artificial neurons process and transmit numerical values, using
weighted sums and activation functions.
Learning: Biological neurons learn and adapt through synaptic
plasticity, changing the strength and number of synapses based on
experience and stimuli. Artificial neurons learn and adapt through
weight adjustment, changing the value and number of weights
based on error and feedback.
Efficiency: Biological neurons are highly efficient and parallel,
processing and transmitting signals at high speed and low energy
consumption. Artificial neurons are less efficient and sequential,
requiring more time and power to perform computations and
communications.
Artificial Neuron and its model.
An artificial neuron, a fundamental unit in artificial neural networks, is a simplified model of
a biological neuron, processing inputs, applying weights and biases, and producing an
output based on an activation function.
Here's a more detailed breakdown:
Key Components of an Artificial Neuron:
Inputs:
An artificial neuron receives signals (inputs) from other neurons or external sources.
Weights:
Each input connection has an associated weight, representing the strength of that
connection.
Bias:
A bias term is added to the weighted sum of inputs, allowing the neuron to activate even
when all inputs are zero.
Summation:
The weighted sum of inputs and the bias is calculated.
Activation Function:
A non-linear function is applied to the summed value to produce the neuron's
output. Common activation functions include sigmoid, ReLU, and tanh.
Output:
The neuron's output is a single value that is then passed on to other connected neurons.
How it Works:
1. Receive Inputs: The neuron receives inputs from other neurons or external data.
2. Apply Weights and Bias: Each input is multiplied by its corresponding weight, and
the bias is added to the sum.
3. Summation: The weighted sum of inputs and the bias is calculated.
4. Apply Activation Function: The activation function transforms the summed value
into the neuron's output.
5. Send Output: The neuron's output is then passed on to other neurons in the
network.
Types of Activation functions
Activation functions can be divided into three types:
1. Linear Activation Function
2. Binary Step Function
3. Non-linear Activation Functions
1. The linear activation function, often called the identity activation function, is proportional to
the input. The range of the linear activation function will be (-∞ to ∞)
Mathematically, it can be represented as: f(x) = x
2. Binary Step Activation Function
A threshold value determines whether a neuron should be activated or not activated
in a binary step activation [Link] activation function compares the input value
to a threshold value. If the input value is greater than the threshold value, the
neuron is activated.
Mathematically, the binary activation function can be represented as:
It cannot provide multi-value outputs — for example, it cannot be used for multi-
class classification problems.
Non-linear Activation Functions
The non-linear activation functions are the most-used activation functions. They
make it uncomplicated for an artificial neural network model to adapt to a variety of
data and to differentiate between the outputs.
1) Sigmoid function
2) ReLU function
3) Tanh function
The sigmoid function (S shaped function) :performs the following transform on
input x, producing an output value between 0 and 1:
-x -x -x
Range of e = [0, ∞) ,Range of 1+ e = [1, ∞) ,Range of 1/(1+ e ) = [0,1]
Rectified Linear Unit (ReLU)
ReLu is used widely among all other activation functions which brings non-linearity by
mapping negative values to 0 and leaving the positive value unchanged. It provides much
fast-learning during training and avoids the vanishing gradient problem. Compared to other
activation functions like Tanh and sigmoid it provides better performance.
The rectified linear unit activation function (or ReLU, for short) transforms output using the
following algorithm:
If the input value x is less than 0, return 0.
If the input value x is greater than or equal to 0, return the input value.
ReLU can be represented mathematically using the max() function:
F(x)=max(0,x)
Hyperbolic Tangent (Tanh)
Hyperbolic tangent function, which is also called tanh function and we use it same as the
sigmoid function and maps input values into a range of -1 to 1. THis activation function is
mostly used in the hidden layer of ANN.
Neural network architectures
There are three fundamental classes of ANN architectures:
1) Single layer feed forward architecture
2) Multilayer feed forward architecture
3) Recurrent networks architecture