Artificial Neural Network – In-Semester Answers

Prepared from Provided ANN Notes

February 2023 Paper (Artificial Neural Network – In-

sem)
Q1
(a) What is a neural network activation function? State its types. [5]

An activation function in a neural network is a non-linear transfer function applied to

the weighted sum of inputs and bias at a neuron to decide whether the neuron should fire
and to map the net input to a bounded output range such as [0, 1] or [−1, 1].[file:4] It in-
troduces non-linearity, enables the network to approximate complex decision boundaries,
keeps outputs within manageable ranges that stabilize learning, and ensures differentia-
bility needed for gradient–based learning like backpropagation.[file:4][file:3] The input to
an activation function is typically sum = ni=1 wi xi + b, where wi are synaptic weights,
P

xi inputs, and b bias.[file:4]

Common activation functions are:
1
• Logistic (unipolar sigmoid): f (sum) = ; monotonically increasing,
1 + e−sum
outputs in (0, 1), widely used in multilayer networks and backpropagation.[file:4]
esum − e−sum
• Hyperbolic tangent (bipolar sigmoid): f (sum) = tanh(sum) = ;
esum + e−sum
outputs in (−1, 1), zero–centred and often preferred for hidden layers.[file:4]

• Identity/Linear: f (x) = ax (special case f (x) = x); provides an unbounded

linear mapping, range (−∞, ∞), useful mainly in output layers for regression but
not suitable alone for deep non-linear modelling.[file:4]

• General “sigmoid” family: Any S–shaped, differentiable function such as logistic

and tanh that squashes large positive/negative inputs into a small bounded interval,
giving smooth gradients.[file:4][file:2]

• ReLU (Rectified Linear Unit): f (z) = max(0, z); outputs zero for negative
inputs and linear for positive inputs, giving sparse activations and efficient compu-
tation.[file:4][web:14]

1
Activation functions therefore control information flow, decide neuron firing, avoid satu-
ration of outputs, and enable multilayer networks to have greater representational power
than single–layer linear networks.[file:4][file:2]
Reference: [Link] (Unit 1, Section 1.6 Activation Functions) and
ANN Module 1 Lecture [Link] (Activation functions)[file:4][file:2]

(b) Explain architecture of Artificial Neural Network with a neat diagram.

[5]

An artificial neural network consists of many simple processing elements (artificial neu-
rons) arranged in layers and interconnected by weighted links to transform input patterns
into meaningful outputs.[file:4][file:1] A typical feed–forward ANN has three main layers:
an input layer that receives raw features, one or more hidden layers where weighted
sums plus activation functions perform non-linear processing, and an output layer that
produces the final decision or prediction.[file:4][file:1] Neurons in adjacent layers are usu-
ally fully connected, so every neuron in one layer sends its output (after activation) as
input to all neurons in the next layer, while the overall knowledge of the network is stored
in the distributed pattern of synaptic weights.[file:4]
In operation, an input vector is applied at the input layer; each hidden neuron com-
putes a weighted sum of its inputs plus a bias term and then passes this through a
non-linear activation function such as logistic or tanh.[file:4] The resulting hidden activa-
tions are again linearly combined and passed through activation functions at the output
layer, giving class labels, probabilities, or real–valued predictions depending on the prob-
lem.[file:4][file:1] Learning rules (e.g., perceptron learning or backpropagation) iteratively
adjust the weights so that the mapping from input layer to output layer approximates
the desired input–output relationship.[file:4]
[Insert Diagram Here – Source: [Link], Unit 1, Fig. 1.1.1 Artificial
Neural Network]
Reference: [Link] (Unit 1, Sections 1.1 and 1.5)[file:4]

(c) Explain Mc-Culloch & Pitts model with an example. [5]

The McCulloch–Pitts (MP) neuron is the first mathematical model of a biological neuron
and acts as a logical threshold element with multiple binary inputs and a single binary
output.[file:4][file:2] Each input xi ∈ {0, 1} is scaled by a weight wi , the neuron computes
P
the effective input u = i wi xi , and compares it with a threshold T ; the output is 1
(“fires”) if u ≥ T and 0 otherwise, using a step activation function.[file:4][file:2] Inputs
may be excitatory or inhibitory, and an inhibitory input can veto firing irrespective of
the total excitatory sum.[file:4]
For example, an AND function with two Boolean inputs x1 , x2 can be implemented
by choosing w1 = w2 = 1 and threshold T = 2; only when both inputs are 1 do we

2
get u = 2 ≥ T and output 1, otherwise the neuron outputs 0.[file:2] Similarly, an OR
function can be realised with w1 = w2 = 1 and T = 1, so any single active input
is sufficient to fire the neuron.[file:2] The MP model thus realises basic Boolean logic
but has important limitations: weights and thresholds are analytically chosen rather
than learned, the neuron output is strictly binary (discrete), and a single layer of such
neurons cannot model non–linearly separable problems like XOR.[file:4][file:2] Hence more
powerful neuron models (perceptron, Adaline) and multilayer networks were developed.
Reference: [Link] (Unit 1, Section 1.7) and ANN Module 1 Lecture [Link]
(McCulloch–Pitts model)[file:4][file:2]

Q2
(a) What are the main differences among the three models of artificial neuron,
namely, McCulloch-Pitts, Perceptron and Adaline? [5]

The McCulloch–Pitts neuron, perceptron, and Adaline are all single–neuron models but
differ in activation function, learning rule, and output representation.[file:4][file:2]

• McCulloch–Pitts (MP) neuron: Uses a hard threshold (step) function on the

weighted sum of binary inputs; weights and threshold are analytically selected
rather than learned, so no systematic learning algorithm is defined in the origi-
nal model.[file:4][file:2]

• Perceptron: Extends the MP neuron by introducing a supervised error–correction

learning rule (perceptron rule) that adjusts weights whenever the classification is
incorrect, but still uses a hard threshold (sign/step) to produce binary outputs for
linearly separable problems.[file:4][file:2]

• Adaline (Adaptive Linear Neuron): Uses a linear activation (identity) on the

net input and compares it externally with a desired output; weights are adapted to
minimize mean square error using the Widrow–Hoff (delta/LMS) rule based on the
continuous net output rather than thresholded output.[file:4][file:2]

Compared with perceptron learning, Adaline learning is based on gradient descent on a

well–defined quadratic cost surface, guaranteeing convergence to the least–squares solu-
tion when it exists, while perceptron convergence is guaranteed only for linearly separable
classes with a threshold output cost.[file:4] The MP neuron lacks such learning capability
altogether and serves mainly as a conceptual logical model.[file:4][file:2]
Reference: [Link] (Unit 1, Sections 1.7 McCulloch–Pitts, 1.8 Per-
ceptron, 1.8.3 Adaline) and ANN Module 1 Lecture [Link] (Adaline model)[file:4][file:2]

3
(b) Explain the structure and working of Biological Neural Network. [5]

A biological neuron (nerve cell) is an electrically excitable cell forming the basic building
block of the brain and nervous system, responsible for sensing, processing, and transmit-
ting information.[file:4][file:2] A typical neuron consists of a cell body (soma) contain-
ing the nucleus for protein synthesis, a tree–like set of dendrites that receive signals
from other neurons, and a single long axon that carries action potentials away from the
soma towards other neurons or effector organs.[file:4][file:2] At the terminal branches of
the axon are synaptic boutons, where neurotransmitters are released across synapses
(small gaps) to dendrites or soma of the next neuron, converting electrical signals to
chemical and back to electrical form.[file:4][file:2]
In a biological neural network, many billions of neurons are interconnected, and infor-
mation flows as sequences of brief electrical pulses called action potentials.[file:4] When
excitatory and inhibitory postsynaptic potentials arriving via dendrites change the mem-
brane voltage at the axon hillock sufficiently to cross a threshold, the neuron “fires” an
action potential that propagates along the axon and can excite or inhibit downstream
neurons at synapses.[file:4][file:2] Such networks are massively parallel, fault tolerant,
adaptive (through synaptic plasticity), and support higher functions like learning and
memory via modification of synaptic strengths over time.[file:4][file:3]
Reference: [Link] (Unit 1, Section 1.3) and ANN Module 1 Lecture [Link]
(Biological neuron structure and functions)[file:4][file:2]

4
(c) Differentiate between Biological Neural Network and Artificial Neural
Network. [5]

Biological Neural Network (BNN) Artificial Neural Network (ANN)

Contains about 1010 –1011 neurons with Contains comparatively few artificial
more than 1014 synapses, extremely neurons and weights, limited by compu-
large and dense.[file:4][file:2] tational resources.[file:4][file:2]
Built from living cells using elec- Built in software or hardware; signals
tro–chemical signaling (action poten- are real numbers propagated by arith-
tials, neurotransmitters). Signals are metic operations without chemical me-
pulses over time.[file:4][file:2] diators.[file:4][file:1]
Operates asynchronously; each neuron Operation is typically synchronous in
fires when its local membrane potential discrete time steps or layers during for-
crosses threshold.[file:4][file:2] ward and backward passes.[file:4][file:3]
Single neuron firing is slow (millisec- Arithmetic operations in digital hard-
onds); conduction speeds vary from ware are very fast; propagation is lim-
about 0.6 to 119 m/s.[file:2] ited mainly by processor and memory
speed.[file:2]
Highly complex, irregular connectivity; Often uses structured topologies (lay-
rich recurrent and lateral connections ered, feed–forward, simple recurrent,
form complex circuits.[file:4] convolutional) defined by the de-
signer.[file:1][file:3]
Very fault tolerant and self–repairing; Typically not inherently fault toler-
damage to some neurons rarely causes ant; failure of hardware units or severe
complete failure because of redun- weight corruption can stop correct op-
dancy.[file:2] eration unless extra redundancy is engi-
neered.[file:1][file:2]
Learning mechanism is biological synap- Learning is governed by explicit algo-
tic plasticity, still not completely under- rithms such as perceptron rule, delta
stood; learning and operation are tightly rule, backpropagation, Hebbian rule;
intertwined and continuous.[file:3][file:2] training and inference phases are often
separated.[file:4][file:3]
Energy–efficient: human brain uses Modern ANN implementations (e.g.,
about 20 W to perform massive paral- GPUs) consume far higher power for
lel computation.[file:2] large models.[file:2]
Can learn many diverse tasks and ex- Most ANNs are specialised for a sin-
hibit general intelligence and transfer gle task; changing the task usually re-
learning naturally.[file:2] quires retraining or redesigning the net-
work.[file:1][file:2]
Uses spiking temporal coding and com- Uses static activation functions on scalar
plex biophysical dynamics in neurons inputs (e.g., sigmoid, ReLU) and simple
and synapses.[file:4] weighted sums.[file:4][file:2]
5
 Reference: [Link] (Unit 1, comparison table) and ANN Module 1 Lecture Notes
(Major differences between BNN and ANN)[file:4][file:2]

Q3
(a) Explain Perceptron Learning Algorithm with an example. [5]

A single–layer perceptron is a feed–forward neuron that computes a weighted sum of its

inputs plus bias and passes it through a threshold (sign/step) activation to produce a
binary output.[file:4][file:2] Given input vector x = [x1 , x2 , . . . , xn ], weight vector w =
P
[w1 , . . . , wn ] and bias b, the net input is u = i wi xi + b and the output is typically
y = sgn(u) or a step function mapping to {0, 1}.[file:4][file:2]
Perceptron learning algorithm (supervised error–correction rule):[file:4][file:2]
1. Initialize all weights wi and bias b to small random values (or zero).

2. For each training pair (x, t), where t ∈ {0, 1} is the desired output, compute the
actual output y using the current weights and activation function.

3. If y = t, no change is made; if y ̸= t, update each weight and bias using

∆wi = η(t − y) xi , wi ← wi + ∆wi , ∆b = η(t − y), b ← b + ∆b

where η is the learning rate.[file:4][file:2]

4. Repeat pattern presentations (epochs) until all training patterns are correctly clas-
sified or a maximum number of iterations is reached.[file:4]
Implementing OR function: The OR function for two binary inputs has the truth
table (0, 0) → 0, (0, 1) → 1, (1, 0) → 1, (1, 1) → 1.[file:4][file:2]
• Choose initial weights w1 = w2 = 0, bias b = 0, learning rate η = 1.

• For pattern (0, 0) with target t = 0: u = 0, y = 0; since y = t, no update.

• For pattern (0, 1), t = 1: u = 0, so y = 0 ̸= 1; updates: ∆w1 = 1(1 − 0) · 0 =

0, ∆w2 = 1(1 − 0) · 1 = 1, ∆b = 1. New parameters: w1 = 0, w2 = 1, b = 1.[file:2]

• For pattern (1, 0), t = 1: u = w1 · 1 + w2 · 0 + b = 0 + 0 + 1 = 1, so y = 1 = t; no

change.

• For pattern (1, 1), t = 1: u = 0 · 1 + 1 · 1 + 1 = 2, y = 1 = t; no change.

After one epoch, the perceptron has learned weights w1 = 0, w2 = 1, b = 1, which correctly
classify all four OR patterns; geometrically, the decision boundary is a line separating
the class (0, 0) from the other three input points.[file:4][file:2]
Reference: [Link] (Unit 1, Sections 1.5 and 1.8) and ANN Module 1 Lecture Not
(Perceptron learning rule and OR example)[file:4][file:2]

6
(b) Explain the architecture of Multilayered neural network. [5]

A multilayer feed–forward neural network (multilayer perceptron, MLP) contains an input

layer, one or more hidden layers of adaptive neurons, and an output layer, with connec-
tions strictly moving from earlier to later layers (no cycles), hence “feed–forward”.[file:4][file:3]
Every hidden neuron receives inputs from all neurons in the previous layer and sends its
output to all neurons in the next layer, typically using sigmoid activation functions to
model non-linear decision surfaces.[file:4]
In a typical MLP, the input layer simply distributes the components of the input
vector to the next layer.[file:4][file:1] Each neuron in a hidden layer computes a weighted
sum of all its inputs plus a bias and passes it through a non-linear activation g(x), usu-
ally logistic or tanh, to produce its activation.[file:4] These hidden units act as feature
detectors that transform raw inputs into an internal representation suitable for solving
the task (e.g., classification or regression).[file:4][file:3] The output layer again computes
weighted sums of hidden activations, followed by an activation function (e.g., logistic for
probabilities, identity for regression), producing the network output.[file:4][file:3] MLPs
can approximate non–linearly separable mappings such as XOR and are trained by back-
propagation with gradient descent, which adjusts all interlayer weights to minimize an
error function over a set of training examples.[file:4][file:3]
[Insert Diagram Here – Source: [Link], Unit 1, Fig. 1.5.4/1.8.2
Multilayer Perceptron]
Reference: [Link] (Unit 1, Sections 1.5 and 1.8) and Module-
II [Link] (Unit 2, Backpropagation and multilayer networks)[file:4][file:3]

(c) Write and explain the steps of Back Propagation Learning algorithm. [5]

Backpropagation is a supervised learning algorithm for multilayer feed–forward neural

networks that uses gradient descent to minimize the total squared error between desired
and actual outputs.[file:4][file:3] It generalizes the Widrow–Hoff (delta) rule to networks
with one or more hidden layers by efficiently computing gradients of the error with respect
to all weights using repeated applications of the chain rule.[file:4][file:3]
A typical backpropagation learning procedure is:[file:4][file:3]

1. Initialization: Set all weights and biases in the network to small random values
(positive and negative) so that neurons start in a non-saturated region of their
activation functions.[file:4]

2. Forward pass: For a chosen training pair, apply the input vector to the input layer,
propagate activations layer by layer using weighted sums plus activation functions
to compute the network output.[file:4][file:3]

3. Error computation: Compute the error at each output neuron, usually using
squared error Ep = 21 k (dk − yk )2 for pattern p, where dk is desired and yk actual
P

7
 output.[file:4][file:3]

4. Backward pass: Propagate error signals backward:

• For each output neuron, compute its local gradient (delta) based on the deriva-
tive of the activation function and the error (dk − yk ).[file:4]
• For each hidden neuron, compute its delta as the derivative of its activation
multiplied by the weighted sum of deltas from the neurons it feeds into.[file:4][file:3]

5. Weight update: Adjust each weight wij proportional to the negative gradient
using
∂E
∆wij = −η = η δ j xi
∂wij
where η is learning rate, δj is local gradient at neuron j, and xi is the input to
weight wij .[file:4][file:3]

6. Iteration over patterns: Repeat forward and backward passes for all train-
ing pairs (one epoch), then cycle over epochs until the mapping error over the
training set falls below an acceptable threshold or a maximum epoch count is
reached.[file:4][file:3]

Backpropagation thus comprises a forward computation of outputs and a backward flow of

error information that systematically reduces the cost function and allows deep multilayer
networks to learn complex input–output mappings.[file:4][file:3]
Reference: [Link] (Unit 1, Backpropagation MLP and learning
rule) and Module-II [Link] (Unit 2, Section 2.7 Backpropagation)[file:4][file:3]

Q4
(a) Draw the architecture of Back Propagation Network and explain in detail.
[5]

A standard backpropagation network is a multilayer feed–forward architecture compris-

ing an input layer, one or more hidden layers, and an output layer, with differentiable
activation functions (commonly logistic or hyperbolic tangent) in the hidden and output
units.[file:4][file:3] Each neuron computes a weighted sum of its inputs plus a bias and
passes it through its activation; the same set of weights is used during forward and back-
ward passes, but reversed signal flow in the backward phase propagates error information
instead of activations.[file:4]
In the forward pass, input signals are propagated from the input layer through each
hidden layer to the output layer, computing activations at each stage.[file:3] In the back-
ward pass, starting from the output layer, error gradients are computed and propagated
backward through the hidden layers towards the input side, and all connection weights

8
and biases are updated according to the backpropagation learning rule.[file:4][file:3] The
network is trained with a large number of input–output pairs, iteratively adjusting weights
until the global error over the training set is acceptably low.[file:4] Typically, logistic or
tanh units are used in hidden layers, identity or logistic units in output layers, and gra-
dient descent (with optional momentum) is employed to speed up convergence and avoid
local minima.[file:4][file:3]
[Insert Diagram Here – Source: [Link], Unit 1, Fig. 1.5.6 and
Fig. 1.5.7 Backpropagation MLP]
Reference: [Link] (Unit 1, Section 1.5 Backpropagation MLP)
and Module-II [Link] (Unit 2, Section 2.7)[file:4][file:3]

(b) Differentiate between Feed forward and Feedback neural network. [5]

Feed–forward neural network Feedback / Recurrent neural net-

work
Connections form a directed acyclic Connections include cycles; outputs of
graph: information flows strictly from neurons can be fed back as inputs to
input layer through hidden layers to out- themselves or other neurons, forming
put layer, no cycles.[file:4][file:3] dynamic feedback loops.[file:3][file:2]
Has no internal state (memory) of pre- Possesses internal dynamic state; cur-
vious inputs beyond what is encoded in rent outputs depend on both present in-
weights; each input is processed inde- puts and past activations, giving mem-
pendently.[file:3] ory of sequences.[file:3]
Primarily used for static pattern recog- Used for time series prediction, speech
nition, function approximation, and recognition, natural language process-
classification/regression when data are ing, and any sequential data where tem-
not time–dependent.[file:4][file:3] poral context matters.[file:3]
Training is generally simpler; back- Training is more complex due to recur-
propagation with gradient descent rent connections; may suffer from van-
is straightforward and stable for ishing/exploding gradients and typically
shallow to moderately deep architec- uses backpropagation through time or
tures.[file:4][file:3] related variants.[file:3]
Network state converges immediately af- Network state may evolve over multi-
ter one forward pass for a given input; ple time steps until an equilibrium (fixed
no iterative settling required.[file:3] point or attractor) is reached, as in Hop-
field or BAM networks.[file:3]
Examples: single–layer perceptron, mul- Examples: Hopfield network, Elman
tilayer perceptron (MLP), radial basis network, Jordan network, simple RNN,
function (RBF) networks.[file:4][file:1] LSTM, GRU, BAM.[file:3][file:1]

Reference: Module-II [Link] (Unit 2, Sections 2.8 Feed Forward Neural Networks

9
and 2.8.1 Feedback Neural Networks) and ANN [Link] (Types of NN)[file:3][file:1]

(c) What is Error Correction and how to minimize these errors? [5]

Error–correction learning is a supervised learning paradigm in which adjustable param-

eters (weights, biases) of an ANN are updated so as to minimize the difference between
the desired output and the actual output produced by the network.[file:3] For a neuron
k, given an input vector xn at time n, desired output dk (n), and obtained output yk (n),
the instantaneous error is ek (n) = dk (n) − yk (n), and learning rules are designed to re-
duce a global error function, typically the mean square error E = 21 E{ k e2k (n)}.[file:3]
P

The error signal drives a control mechanism that adjusts the synaptic weights so that,
in a statistical sense, network outputs approximate desired outputs over the training
set.[file:3][file:4]
To minimize errors, gradient–based rules such as the delta rule, Widrow–Hoff LMS,
or backpropagation are used to move weights in the negative gradient direction of the error
surface.[file:4][file:3] For example, in the LMS rule applied to Adaline, the weight update
for pattern p is ∆wi = η(tp −op )xpi , directly proportional to the product of error and input,
ensuring that weights change most where error and input magnitude are large.[file:4][file:2]
Repeated presentation of many training examples causes the error function to converge
towards a minimum, thereby reducing classification or prediction errors; techniques like
appropriate learning rate selection, momentum, and sufficient training data further help
in minimizing residual errors and avoiding local minima.[file:3][file:4]
[Insert Diagram Here – Source: Module-II [Link], Fig. 2.4.1 Error correction learn-
ing block diagram]
Reference: Module-II [Link] (Unit 2, Section 2.4 Error Correction) and ANN-
[Link] (Unit 1, Adaline and LMS learning)[file:3][file:4]

March 2024 Paper (Artificial Neural Network – In-

sem)
Q1
(a) What is the role of activation function in neural network? Explain bipolar
Sigmoid function in detail. [5]

The role of the activation function is to introduce non-linearity into the neuron model
so that multilayer networks can approximate complex, non–linearly separable decision
boundaries and functions.[file:4] By squashing the weighted sum plus bias into a bounded
range, activation functions stabilise outputs, make gradients finite for learning, and con-
trol whether or not a neuron “fires” strongly for a given input pattern.[file:4][file:2] They

10
also ensure differentiability, which is essential for gradient–based training algorithms like
backpropagation.[file:4][file:3]
The bipolar sigmoid (hyperbolic tangent) function maps net input to the range
[−1, 1] and is defined as

esum − e−sum
f (sum) = tanh(sum) =
esum + e−sum

or equivalently as a scaled logistic function.[file:4][file:2] It is S–shaped (sigmoidal), con-

tinuous, monotonically increasing, differentiable everywhere, and approximately linear
around zero, which allows good gradient flow in intermediate regions.[file:4] Because its
output is zero–centred, it often leads to faster convergence than the unipolar logistic func-
tion when used in hidden layers; its derivative can be expressed in terms of its output as
f ′ (sum) = 1 − f 2 (sum), which simplifies backpropagation computations.[file:4][file:3]
Reference: [Link] (Unit 1, Sections 1.6.2 and 1.6.3) and ANN Module 1 Lecture
(Bipolar sigmoidal / TanH function)[file:4][file:2]

(b) Why is ReLU the most commonly used Activation Function? [5]

The Rectified Linear Unit (ReLU) activation f (z) = max(0, z) is widely used in modern
deep networks because it combines computational simplicity with favourable gradient
properties.[file:4][web:14] (Source: Internet) Unlike sigmoid and tanh, which saturate for
large |z| and produce very small derivatives leading to vanishing gradients, ReLU has a
constant derivative of 1 for z > 0, thus maintaining strong gradient flow in active regions
and enabling faster and more stable training of deep models.[web:14][web:24] (Source:
Internet) Its implementation is cheap (just a threshold at zero), making it highly efficient
on hardware accelerators.[web:14][web:24] (Source: Internet)
ReLU also produces sparse activations because many neurons output exactly zero
for negative inputs, which effectively prunes the network for a given sample and can reduce
overfitting and improve generalization.[web:16][web:24] (Source: Internet) Empirically,
ReLU–based networks converge faster and achieve better performance on large–scale
vision and language tasks compared with sigmoid/tanh networks, which is why ReLU
(and its variants such as Leaky ReLU and Parametric ReLU) has become the default
choice for hidden layers in deep learning architectures.[file:4][web:14] (Source: Internet)
Reference: [Link] (Unit 1, Section 1.6.3 Tanh and ReLU) and
GeeksforGeeks / other DL resources on ReLU vs Sigmoid/Tanh (Internet Source)[file:4][web:14]

(c) Explain architecture of Artificial Neural Network with a neat diagram. [5]

An artificial neural network consists of many simple processing elements (artificial neu-
rons) arranged in layers and interconnected by weighted links to transform input patterns
into meaningful outputs.[file:4][file:1] A typical feed–forward ANN has three main layers:

11
an input layer that receives raw features, one or more hidden layers where weighted
sums plus activation functions perform non-linear processing, and an output layer that
produces the final decision or prediction.[file:4][file:1] Neurons in adjacent layers are usu-
ally fully connected, so every neuron in one layer sends its output (after activation) as
input to all neurons in the next layer, while the overall knowledge of the network is stored
in the distributed pattern of synaptic weights.[file:4]
In operation, an input vector is applied at the input layer; each hidden neuron com-
putes a weighted sum of its inputs plus a bias term and then passes this through a
non-linear activation function such as logistic or tanh.[file:4] The resulting hidden activa-
tions are again linearly combined and passed through activation functions at the output
layer, giving class labels, probabilities, or real–valued predictions depending on the prob-
lem.[file:4][file:1] Learning rules (e.g., perceptron learning or backpropagation) iteratively
adjust the weights so that the mapping from input layer to output layer approximates
the desired input–output relationship.[file:4]
[Insert Diagram Here – Source: [Link], Unit 1, Fig. 1.1.1 Artificial
Neural Network]
Reference: [Link] (Unit 1, Sections 1.1 and 1.5)[file:4]

Q2
(a) Draw the structure of the biological neuron and explain working of the
same in brief. [5]

A biological neuron (nerve cell) is an electrically excitable cell forming the basic building
block of the brain and nervous system, responsible for sensing, processing, and transmit-
ting information.[file:4][file:2] A typical neuron consists of a cell body (soma) contain-
ing the nucleus for protein synthesis, a tree–like set of dendrites that receive signals
from other neurons, and a single long axon that carries action potentials away from the
soma towards other neurons or effector organs.[file:4][file:2] At the terminal branches of
the axon are synaptic boutons, where neurotransmitters are released across synapses
(small gaps) to dendrites or soma of the next neuron, converting electrical signals to
chemical and back to electrical form.[file:4][file:2]
In a biological neural network, many billions of neurons are interconnected, and infor-
mation flows as sequences of brief electrical pulses called action potentials.[file:4] When
excitatory and inhibitory postsynaptic potentials arriving via dendrites change the mem-
brane voltage at the axon hillock sufficiently to cross a threshold, the neuron “fires” an
action potential that propagates along the axon and can excite or inhibit downstream
neurons at synapses.[file:4][file:2] Such networks are massively parallel, fault tolerant,
adaptive (through synaptic plasticity), and support higher functions like learning and
memory via modification of synaptic strengths over time.[file:4][file:3]
Reference: [Link] (Unit 1, Section 1.3) and ANN Module 1 Lecture [Link]

12
(Biological neuron structure and functions)[file:4][file:2]
[Insert Diagram Here – Source: [Link], Unit 1, Fig. 1.3.1 Schematic
of biological neuron]

(b) Write an algorithm of ADALINE and focus on its upper bound with
largest Eigen Value of its correlation matrix. [10]

Adaline (Adaptive Linear Neuron) is a single–layer network that uses a linear activation
function and adapts its weights using the Widrow–Hoff (delta/LMS) rule to minimize
mean square error between actual and desired outputs.[file:4][file:2] Its architecture is
similar to a perceptron but learning operates on the continuous net output before thresh-
olding, which leads to a convex quadratic error surface.[file:4][file:2]
Adaline training algorithm (using bipolar patterns):[file:2][file:4]

1. Initialization: Set all weights wi and bias b to zero (or small random values) and
choose learning rate α (often initially 1 for hand computation).[file:2]

2. Pattern activation: For each bipolar training pair (s, t) with inputs si ∈ {−1, 1}
and target t ∈ {−1, 1}, assign xi = si for i = 1, . . . , n.[file:2]

3. Net input computation: Compute net input to Adaline as

n
X
net = wi xi + b.
i=1

4. Linear output: Because Adaline is linear, take y = net as the neuron output
before any external thresholding.[file:4]

5. Error calculation: Compute error e = t − y between desired target and linear

output.[file:2]

6. Weight and bias update: Adjust each weight and bias according to the delta
rule
wi ← wi + α e xi , b ← b + α e,

which performs gradient descent on the mean squared error surface.[file:4][file:2]

7. Stopping condition: Repeat steps 2–6 for all training pairs (an epoch) and iterate
epochs until there is no significant change in weights or the maximum weight change
is below a specified tolerance, indicating convergence.[file:2]

From an LMS theory viewpoint, if R is the input correlation matrix, the stability
and convergence speed of Adaline/LMS learning depend on the largest eigenvalue λmax
of R.[file:4] (Source: Internet for eigenvalue focus) To ensure convergence of the mean

13
square error for a fixed–step LMS algorithm, the learning rate α must satisfy an upper
bound of the form
2
0<α< ,
λmax
so that the weight–error dynamics are stable and oscillations are avoided.[web:21] (Source:
Internet) A smaller α yields slower but smoother convergence, while values approaching
the upper bound give faster learning at the risk of instability.[web:21] (Source: Internet)
Reference: ANN Module 1 Lecture [Link] (ADALINE model and training algo-
rithm) and [Link] (Unit 1, Section 1.8.3 ADALINE Network); eigen-
value bound concept from standard LMS theory (Internet Source)[file:2][file:4][web:21]

Q3
(a) What is Error Correction and how to minimize these errors? [5]

Error–correction learning is a supervised learning paradigm in which adjustable param-

eters (weights, biases) of an ANN are updated so as to minimize the difference between
the desired output and the actual output produced by the network.[file:3] For a neuron
k, given an input vector xn at time n, desired output dk (n), and obtained output yk (n),
the instantaneous error is ek (n) = dk (n) − yk (n), and learning rules are designed to re-
duce a global error function, typically the mean square error E = 12 E{ k e2k (n)}.[file:3]
P

The error signal drives a control mechanism that adjusts the synaptic weights so that,
in a statistical sense, network outputs approximate desired outputs over the training
set.[file:3][file:4]
To minimize errors, gradient–based rules such as the delta rule, Widrow–Hoff LMS,
or backpropagation are used to move weights in the negative gradient direction of the error
surface.[file:4][file:3] For example, in the LMS rule applied to Adaline, the weight update
for pattern p is ∆wi = η(tp −op )xpi , directly proportional to the product of error and input,
ensuring that weights change most where error and input magnitude are large.[file:4][file:2]
Repeated presentation of many training examples causes the error function to converge
towards a minimum, thereby reducing classification or prediction errors; techniques like
appropriate learning rate selection, momentum, and sufficient training data further help
in minimizing residual errors and avoiding local minima.[file:3][file:4]
[Insert Diagram Here – Source: Module-II [Link], Fig. 2.4.1 Error correction learn-
ing block diagram]
Reference: Module-II [Link] (Unit 2, Section 2.4 Error Correction) and ANN-
[Link] (Unit 1, Adaline and LMS learning)[file:3][file:4]

(b) Explain the architecture of Multilayered neural network. [5]

A multilayer feed–forward neural network (multilayer perceptron, MLP) contains an input

layer, one or more hidden layers of adaptive neurons, and an output layer, with connec-

14
tions strictly moving from earlier to later layers (no cycles), hence “feed–forward”.[file:4][file:3]
Every hidden neuron receives inputs from all neurons in the previous layer and sends its
output to all neurons in the next layer, typically using sigmoid activation functions to
model non-linear decision surfaces.[file:4]
In a typical MLP, the input layer simply distributes the components of the input
vector to the next layer.[file:4][file:1] Each neuron in a hidden layer computes a weighted
sum of all its inputs plus a bias and passes it through a non-linear activation g(x), usu-
ally logistic or tanh, to produce its activation.[file:4] These hidden units act as feature
detectors that transform raw inputs into an internal representation suitable for solving
the task (e.g., classification or regression).[file:4][file:3] The output layer again computes
weighted sums of hidden activations, followed by an activation function (e.g., logistic for
probabilities, identity for regression), producing the network output.[file:4][file:3] MLPs
can approximate non–linearly separable mappings such as XOR and are trained by back-
propagation with gradient descent, which adjusts all interlayer weights to minimize an
error function over a set of training examples.[file:4][file:3]
[Insert Diagram Here – Source: [Link], Unit 1, Fig. 1.5.4/1.8.2
Multilayer Perceptron]
Reference: [Link] (Unit 1, Sections 1.5 and 1.8) and Module-
II [Link] (Unit 2, Backpropagation and multilayer networks)[file:4][file:3]

(c) Define learning and memory. Explain learning algorithms in details. [5]

In neural networks, learning is the process of adapting synaptic weights and biases so
that the network can implement a desired input–output mapping, while memory refers
to the stored information represented by these learned connection strengths and network
states.[file:3] Biological brains implement associative memory, where patterns that occur
together become associated, and recalling one pattern can trigger the other; artificial
associative memories (auto– and hetero–associative networks) imitate this by mapping
input pattern vectors to stored output vectors.[file:3]
Learning algorithms can be broadly categorized as:[file:4][file:3]
• Supervised learning: Each input pattern is accompanied by a desired target
output; examples include perceptron learning, LMS (delta rule), backpropagation,
and correlation learning.[file:3][file:2] The goal is to minimize a cost function over
labeled data.

• Unsupervised learning: No explicit target outputs are provided; the network

discovers structure in input data through clustering or feature extraction, as in
Hebbian learning and competitive learning (e.g., Kohonen SOM).[file:3][file:4]

• Reinforcement learning: The network interacts with an environment and re-

ceives evaluative feedback (reward/penalty) rather than explicit target values, ad-
justing its parameters to maximize long–term reward.[file:3]

15
At a lower level, typical “basic learning laws” discussed in ANN include Hebbian rule
(weights increase when pre– and post–synaptic neurons fire together), perceptron rule
(supervised error–correction for threshold neurons), delta rule/LMS (gradient descent on
mean square error for linear units), and correlation/outstar rules for associative mem-
ories.[file:4][file:2] Together, these algorithms determine how experiences (training data)
are encoded into network memory as distributed weight patterns.
Reference: Module-II [Link] (Unit 2, Section 2.1 Learning and Memory, 2.2 Heb-
bian Learning, 2.3 Competitive Learning, 2.6 Supervised Learning) and ANN-Technical-
[Link] (Unit 1, Section 1.9 Basic Learning Laws)[file:3][file:4]

Q4
(a) What is the difference between Forward propagation and Backward Prop-
agation in Neural Networks? [5]

Forward propagation Backward propagation

Phase in which input data are applied Phase in which error signals (derivatives
at the input layer and activations are of loss) are propagated from output back
propagated layer by layer to compute through hidden layers to compute gradi-
the network output.[file:4][file:3] ents for all weights.[file:4][file:3]
Uses current weights and biases to Uses chain rule to combine local deriva-
perform weighted sums and activation tives and error terms; produces weight
functions only; no weight changes oc- updates according to a learning rule
cur.[file:4] such as gradient descent.[file:4][file:3]
P

Computes yk = f i w ik x i + b k at Computes local gradient δj at each neu-
each neuron, given inputs from previous ron (output or hidden) and then ∆wij =
layer.[file:4] ηδj xi for weight corrections, where η is
learning rate.[file:3]
Executed once per pattern (or batch) Executed after the forward pass and er-
before evaluating the error between ac- ror computation, and is repeated over
tual and desired outputs.[file:3] training epochs to iteratively reduce the
global error.[file:3]
Information flow direction: input → Information flow direction: output →
hidden → output.[file:3] hidden → input side (in terms of error
signals and gradients).[file:3]
Used both during training and inference Used only during training to adjust pa-
(testing) of the network.[file:4] rameters; not used during pure inference
once training is complete.[file:3]

Reference: Module-II [Link] (Unit 2, Backpropagation: forward and backward

pass) and [Link] (Backpropagation MLP description)[file:3][file:4]

16
(b) Explain the different types of Gradient Descent in detail. [5]

Gradient descent is an optimization method that adjusts network parameters in the neg-
ative direction of the gradient of a loss function to find its minimum.[file:3] In neural net-
works, different variants trade off computational cost and convergence behaviour:[web:17][web:21]
(Source: Internet)

• Batch (full) Gradient Descent: Computes the gradient of the loss using the
entire training dataset before each update; the parameter update is

w ← w − η∇w J(w)

where ∇w J(w) is computed over all samples.[web:17][web:21] (Source: Internet) It

has smooth, stable convergence but can be very slow and memory–intensive for
large datasets.[web:21] (Source: Internet)

• Stochastic Gradient Descent (SGD): Updates parameters using the gradient

computed from a single randomly chosen training sample (or very small subset)
at each step, giving noisy but fast updates that can help escape shallow local min-
ima.[web:17][web:23] (Source: Internet) It is computationally cheap per update and
suitable for online learning, but its loss curve is highly fluctuating.[web:21] (Source:
Internet)

• Mini–Batch Gradient Descent: Uses small batches (e.g., 32 or 64 examples)

to compute approximate gradients, combining advantages of batch and stochastic
methods.[web:17][web:25] (Source: Internet) It improves computational efficiency
via vectorized operations on hardware accelerators and leads to smoother and often
faster convergence than pure SGD while being less memory–demanding than full
batch.[web:21] (Source: Internet)

Modern deep learning typically relies on mini–batch gradient descent combined with
advanced optimizers (Momentum, RMSProp, Adam) that adapt learning rates and in-
corporate momentum terms, but the basic distinction between batch, stochastic, and
mini–batch remains fundamental.[web:17][web:21] (Source: Internet)
Reference: Module-II [Link] (Unit 2, Section 2.5 Gradient Descent Rules) and
GeeksforGeeks – “Different Variants of Gradient Descent” (Internet Source)[file:3][web:21]

(c) Write down Perceptron Learning Algorithm for OR function along with
calculation of each input vector. [5]

A single–layer perceptron is a feed–forward neuron that computes a weighted sum of its

inputs plus bias and passes it through a threshold (sign/step) activation to produce a
binary output.[file:4][file:2] Given input vector x = [x1 , x2 , . . . , xn ], weight vector w =

17
 P
[w1 , . . . , wn ] and bias b, the net input is u = i wi xi + b and the output is typically
y = sgn(u) or a step function mapping to {0, 1}.[file:4][file:2]
Perceptron learning algorithm (supervised error–correction rule):[file:4][file:2]

1. Initialize all weights wi and bias b to small random values (or zero).

2. For each training pair (x, t), where t ∈ {0, 1} is the desired output, compute the
actual output y using the current weights and activation function.

3. If y = t, no change is made; if y ̸= t, update each weight and bias using

∆wi = η(t − y) xi , wi ← wi + ∆wi , ∆b = η(t − y), b ← b + ∆b

where η is the learning rate.[file:4][file:2]

4. Repeat pattern presentations (epochs) until all training patterns are correctly clas-
sified or a maximum number of iterations is reached.[file:4]

Implementing OR function: The OR function for two binary inputs has the truth
table (0, 0) → 0, (0, 1) → 1, (1, 0) → 1, (1, 1) → 1.[file:4][file:2]

• Choose initial weights w1 = w2 = 0, bias b = 0, learning rate η = 1.

• For pattern (0, 0) with target t = 0: u = 0, y = 0; since y = t, no update.

• For pattern (0, 1), t = 1: u = 0, so y = 0 ̸= 1; updates: ∆w1 = 1(1 − 0) · 0 =

0, ∆w2 = 1(1 − 0) · 1 = 1, ∆b = 1. New parameters: w1 = 0, w2 = 1, b = 1.[file:2]

• For pattern (1, 0), t = 1: u = w1 · 1 + w2 · 0 + b = 0 + 0 + 1 = 1, so y = 1 = t; no

change.

• For pattern (1, 1), t = 1: u = 0 · 1 + 1 · 1 + 1 = 2, y = 1 = t; no change.

After one epoch, the perceptron has learned weights w1 = 0, w2 = 1, b = 1, which correctly
classify all four OR patterns; geometrically, the decision boundary is a line separating
the class (0, 0) from the other three input points.[file:4][file:2]
Reference: [Link] (Unit 1, Sections 1.5 and 1.8) and ANN Module 1 Lecture Not
(Perceptron learning rule and OR example)[file:4][file:2]

18
March 2025 Paper (Artificial Neural Network – In-
sem)
Q1
(a) What is neural network topology? Explain any three topologies in detail.
[6]

Neural network topology refers to the structural arrangement of neurons (nodes) and con-
nections (edges) in a network, including the number of layers, number of neurons per layer,
and the pattern of interconnections and signal flow.[file:1][file:4] Topology determines how
information flows from inputs to outputs and strongly influences the computational ca-
pabilities and learning behaviour of the network.[file:1]
Three important topologies are:

• Single–layer feed–forward network: Contains an input layer directly connected

to an output layer with no hidden layers; each output neuron receives weighted
connections from all inputs.[file:4] It can implement only linearly separable decision
boundaries using threshold or linear units (e.g., single–layer perceptron, Adaline)
and is suitable for simple classification or regression problems.[file:4][file:1]

• Multilayer feed–forward (Multilayer Perceptron) topology: Extends sin-

gle–layer networks by inserting one or more hidden layers between input and out-
put.[file:4][file:3] Every neuron in one layer is fully connected to neurons in the next
layer, with no cycles; hidden layers use non-linear activation functions, allowing
the network to approximate arbitrary continuous functions and solve non–linearly
separable problems such as XOR.[file:4][file:1]

• Recurrent (feedback) network topology: Neurons are connected with feedback

loops so that outputs of a neuron may be fed back to its own input or to other neu-
rons in the same or earlier layers, creating cycles in the connection graph.[file:3] Ex-
amples include Hopfield networks, Elman networks, and more general RNNs; they
possess internal state and are well–suited for time–dependent tasks like sequence
modelling and associative memory but are more complex to train.[file:3][file:1]

Other specialised topologies include convolutional networks (local receptive fields and
weight sharing for images), radial basis function networks, and self–organizing maps,
but feed–forward and recurrent topologies form the core building blocks of most ANN
architectures.[file:1][file:4]
Reference: ANN [Link] (Neural net architecture and topology of neural net-
work architecture) and [Link] (Unit 1, Section 1.5 Neural Net Archi-
tecture)[file:1][file:4]

19
(b) Explain briefly McCulloch Pitt’s (MP) artificial neuron model. Give its
limitations. [5]

The McCulloch–Pitts (MP) neuron is the first mathematical model of a biological neuron
and acts as a logical threshold element with multiple binary inputs and a single binary
output.[file:4][file:2] Each input xi ∈ {0, 1} is scaled by a weight wi , the neuron computes
P
the effective input u = i wi xi , and compares it with a threshold T ; the output is 1
(“fires”) if u ≥ T and 0 otherwise, using a step activation function.[file:4][file:2] Inputs
may be excitatory or inhibitory, and an inhibitory input can veto firing irrespective of
the total excitatory sum.[file:4]
For example, an AND function with two Boolean inputs x1 , x2 can be implemented
by choosing w1 = w2 = 1 and threshold T = 2; only when both inputs are 1 do we
get u = 2 ≥ T and output 1, otherwise the neuron outputs 0.[file:2] Similarly, an OR
function can be realised with w1 = w2 = 1 and T = 1, so any single active input
is sufficient to fire the neuron.[file:2] The MP model thus realises basic Boolean logic
but has important limitations: weights and thresholds are analytically chosen rather
than learned, the neuron output is strictly binary (discrete), and a single layer of such
neurons cannot model non–linearly separable problems like XOR.[file:4][file:2] Hence more
powerful neuron models (perceptron, Adaline) and multilayer networks were developed.
Reference: [Link] (Unit 1, Section 1.7) and ANN Module 1 Lecture [Link]
(McCulloch–Pitts model)[file:4][file:2]

20
(c) Distinguish between Biological Neural Network and Artificial Neural Net-
works. [4]

Biological Neural Network (BNN) Artificial Neural Network (ANN)

Contains about 1010 –1011 neurons with Contains comparatively few artificial
more than 1014 synapses, extremely neurons and weights, limited by compu-
large and dense.[file:4][file:2] tational resources.[file:4][file:2]
Built from living cells using elec- Built in software or hardware; signals
tro–chemical signaling (action poten- are real numbers propagated by arith-
tials, neurotransmitters). Signals are metic operations without chemical me-
pulses over time.[file:4][file:2] diators.[file:4][file:1]
Operates asynchronously; each neuron Operation is typically synchronous in
fires when its local membrane potential discrete time steps or layers during for-
crosses threshold.[file:4][file:2] ward and backward passes.[file:4][file:3]
Single neuron firing is slow (millisec- Arithmetic operations in digital hard-
onds); conduction speeds vary from ware are very fast; propagation is lim-
about 0.6 to 119 m/s.[file:2] ited mainly by processor and memory
speed.[file:2]
Highly complex, irregular connectivity; Often uses structured topologies (lay-
rich recurrent and lateral connections ered, feed–forward, simple recurrent,
form complex circuits.[file:4] convolutional) defined by the de-
signer.[file:1][file:3]
Very fault tolerant and self–repairing; Typically not inherently fault toler-
damage to some neurons rarely causes ant; failure of hardware units or severe
complete failure because of redun- weight corruption can stop correct op-
dancy.[file:2] eration unless extra redundancy is engi-
neered.[file:1][file:2]
Learning mechanism is biological synap- Learning is governed by explicit algo-
tic plasticity, still not completely under- rithms such as perceptron rule, delta
stood; learning and operation are tightly rule, backpropagation, Hebbian rule;
intertwined and continuous.[file:3][file:2] training and inference phases are often
separated.[file:4][file:3]
Energy–efficient: human brain uses Modern ANN implementations (e.g.,
about 20 W to perform massive paral- GPUs) consume far higher power for
lel computation.[file:2] large models.[file:2]
Can learn many diverse tasks and ex- Most ANNs are specialised for a sin-
hibit general intelligence and transfer gle task; changing the task usually re-
learning naturally.[file:2] quires retraining or redesigning the net-
work.[file:1][file:2]
Uses spiking temporal coding and com- Uses static activation functions on scalar
plex biophysical dynamics in neurons inputs (e.g., sigmoid, ReLU) and simple
and synapses.[file:4] weighted sums.[file:4][file:2]
21
 Reference: [Link] (Unit 1, comparison table) and ANN Module 1 Lecture Notes
(Major differences between BNN and ANN)[file:4][file:2]

Q2
(a) Discuss briefly the structure and function of a biological neuron. [7]

A biological neuron (nerve cell) is an electrically excitable cell forming the basic building
block of the brain and nervous system, responsible for sensing, processing, and transmit-
ting information.[file:4][file:2] A typical neuron consists of a cell body (soma) contain-
ing the nucleus for protein synthesis, a tree–like set of dendrites that receive signals
from other neurons, and a single long axon that carries action potentials away from the
soma towards other neurons or effector organs.[file:4][file:2] At the terminal branches of
the axon are synaptic boutons, where neurotransmitters are released across synapses
(small gaps) to dendrites or soma of the next neuron, converting electrical signals to
chemical and back to electrical form.[file:4][file:2]
In a biological neural network, many billions of neurons are interconnected, and infor-
mation flows as sequences of brief electrical pulses called action potentials.[file:4] When
excitatory and inhibitory postsynaptic potentials arriving via dendrites change the mem-
brane voltage at the axon hillock sufficiently to cross a threshold, the neuron “fires” an
action potential that propagates along the axon and can excite or inhibit downstream
neurons at synapses.[file:4][file:2] Such networks are massively parallel, fault tolerant,
adaptive (through synaptic plasticity), and support higher functions like learning and
memory via modification of synaptic strengths over time.[file:4][file:3]
Reference: [Link] (Unit 1, Section 1.3) and ANN Module 1 Lecture [Link]
(Biological neuron structure and functions)[file:4][file:2]

(b) Obtain the output of the neuron Y for the network shown in figure using
activation function as Binary Sigmoidal and Bipolar Sigmoidal. [8]

The exact numeric output depends on the specific weights, inputs, and bias values in the
given figure, which are not present in the shared resources; however, the procedure for
computing the neuron output using binary and bipolar sigmoidal activation functions is
as follows.[file:4][file:2]

1. Compute net input: For neuron Y with inputs xi , weights wi , and bias b, compute
X
net = wi xi + b.
i

2. Binary sigmoidal (unipolar) output: Use the logistic function

1
ybinary = f (net) = ,
1 + e−net

22
 which produces an output in (0, 1) suitable for representing probabilities or normal-
ized activations.[file:4][file:2]

3. Bipolar sigmoidal (tanh) output: Use the bipolar sigmoid (hyperbolic tangent)
function
enet − e−net
ybipolar = f (net) = tanh(net) = net ,
e + e−net
giving an output in (−1, 1).[file:4]

4. Substitute numeric values: Substitute the numeric values of wi , xi , and b from

the figure into the net expression and then evaluate both activation functions to
obtain the required outputs ybinary and ybipolar .[file:4][file:2]

Thus, once the diagram’s parameters are known, one first computes the scalar net
input and then applies each sigmoid function to get the corresponding neuron outputs
under binary and bipolar sigmoidal activations.[file:4][file:2]
Reference: [Link] (Unit 1, Sections 1.6.1–1.6.3 Activation Func-
tions) and ANN Module 1 Lecture [Link] (Binary and bipolar sigmoid definitions)[file:4][file:2]

Q3
(a) Write and explain Hebbian learning Algorithm. [6]

Hebbian learning is an unsupervised learning rule inspired by neurobiology stating that

“neurons that fire together, wire together”, i.e., if a presynaptic neuron i repeatedly helps
activate postsynaptic neuron j, the synapse between them is strengthened.[file:3][file:4]
Formally, if the activities of neurons on either side of a connection are correlated (syn-
chronous), the weight increases; if they are anti–correlated (asynchronous), the weight de-
creases, providing a local learning mechanism without an explicit error signal or teacher.[file:3]
A general Hebbian update at iteration p for weight wij is ∆wij (p) = F [yi (p), xj (p)],
where xj is input to neuron j and yi its output; a common special case is the activ-
ity product rule ∆wij (p) = α yi (p) xj (p), with learning rate α.[file:3] Because pure
Hebbian learning can cause unbounded weight growth, a “forgetting factor” is often
added: ∆wij (p) = αyi (p)xj (p) − ϕyi (p)wij (p), where ϕ is a small positive constant limit-
ing growth.[file:3] The Hebbian learning algorithm can be summarised as:

1. Initialize weights and thresholds to small random values.

2. For each input pattern, compute neuron outputs using the current weights and
activation functions.[file:3]

3. Update weights according to the Hebbian rule above for all active connections.

4. Repeat for many presentations until weights converge or desired properties (e.g.,
stable associative recall) are achieved.[file:3]

23
Hebbian learning underlies models like linear associators and is used to construct associa-
tive memory networks such as Hopfield and BAM via Hebbian weight matrices.[file:3][file:4]
Reference: Module-II [Link] (Unit 2, Section 2.2 Hebbian Learning) and ANN-
[Link] (Unit 1, Basic learning laws)[file:3][file:4]

(b) What is error correction learning? Explain in detail with diagram. [5]

Error–correction learning is a supervised learning paradigm in which adjustable param-

eters (weights, biases) of an ANN are updated so as to minimize the difference between
the desired output and the actual output produced by the network.[file:3] For a neuron
k, given an input vector xn at time n, desired output dk (n), and obtained output yk (n),
the instantaneous error is ek (n) = dk (n) − yk (n), and learning rules are designed to re-
duce a global error function, typically the mean square error E = 12 E{ k e2k (n)}.[file:3]
P

The error signal drives a control mechanism that adjusts the synaptic weights so that,
in a statistical sense, network outputs approximate desired outputs over the training
set.[file:3][file:4]
To minimize errors, gradient–based rules such as the delta rule, Widrow–Hoff LMS,
or backpropagation are used to move weights in the negative gradient direction of the error
surface.[file:4][file:3] For example, in the LMS rule applied to Adaline, the weight update
for pattern p is ∆wi = η(tp −op )xpi , directly proportional to the product of error and input,
ensuring that weights change most where error and input magnitude are large.[file:4][file:2]
Repeated presentation of many training examples causes the error function to converge
towards a minimum, thereby reducing classification or prediction errors; techniques like
appropriate learning rate selection, momentum, and sufficient training data further help
in minimizing residual errors and avoiding local minima.[file:3][file:4]
[Insert Diagram Here – Source: Module-II [Link], Fig. 2.4.1 Error correction learn-
ing block diagram]
Reference: Module-II [Link] (Unit 2, Section 2.4 Error Correction) and ANN-
[Link] (Unit 1, Adaline and LMS learning)[file:3][file:4]
[Insert Diagram Here – Source: Module-II [Link], Fig. 2.4.1 Error correction learn-
ing]

24
(c) Differentiate between Feed Forward and Feedback neural network. [4]

Feed–forward neural network Feedback / Recurrent neural net-

work
Connections form a directed acyclic Connections include cycles; outputs of
graph: information flows strictly from neurons can be fed back as inputs to
input layer through hidden layers to out- themselves or other neurons, forming
put layer, no cycles.[file:4][file:3] dynamic feedback loops.[file:3][file:2]
Has no internal state (memory) of pre- Possesses internal dynamic state; cur-
vious inputs beyond what is encoded in rent outputs depend on both present in-
weights; each input is processed inde- puts and past activations, giving mem-
pendently.[file:3] ory of sequences.[file:3]
Primarily used for static pattern recog- Used for time series prediction, speech
nition, function approximation, and recognition, natural language process-
classification/regression when data are ing, and any sequential data where tem-
not time–dependent.[file:4][file:3] poral context matters.[file:3]
Training is generally simpler; back- Training is more complex due to recur-
propagation with gradient descent rent connections; may suffer from van-
is straightforward and stable for ishing/exploding gradients and typically
shallow to moderately deep architec- uses backpropagation through time or
tures.[file:4][file:3] related variants.[file:3]
Network state converges immediately af- Network state may evolve over multi-
ter one forward pass for a given input; ple time steps until an equilibrium (fixed
no iterative settling required.[file:3] point or attractor) is reached, as in Hop-
field or BAM networks.[file:3]
Examples: single–layer perceptron, mul- Examples: Hopfield network, Elman
tilayer perceptron (MLP), radial basis network, Jordan network, simple RNN,
function (RBF) networks.[file:4][file:1] LSTM, GRU, BAM.[file:3][file:1]

Reference: Module-II [Link] (Unit 2, Sections 2.8 Feed Forward Neural Networks
and 2.8.1 Feedback Neural Networks) and ANN [Link] (Types of NN)[file:3][file:1]

Q4
(a) Draw the architecture of multilayer feed forward networks. Explain input
layer, hidden layer & output layer computations in multilayer feed forward
networks. [7]

A multilayer feed–forward network consists of an input layer, one or more hidden layers,
and an output layer, with weights only between adjacent layers and no recurrent connec-
tions.[file:4][file:3] Connections go strictly forward from input to output, so the network
can be represented as a layered directed acyclic graph; this is the standard architecture

25
used with backpropagation.[file:3][file:1]
Input layer computations: The input layer has one neuron per input attribute; it
usually performs no computation other than optionally appending a constant bias input
of value 1.[file:4][file:1] The feature vector x is simply presented at this layer and passed
to the first hidden layer as raw signals.
Hidden layer computations: Each hidden neuron j receives outputs xi from all
neurons in the previous layer, computes a net input
X
uj = wij xi + bj
i

and then applies a non-linear activation function yj = f (uj ), often logistic or tanh, to
produce its activation.[file:4][file:3] Hidden layers thus implement non-linear feature trans-
formations of the input space and allow the network to approximate complex functions
such as XOR or general decision boundaries.[file:4]
Output layer computations: Output neurons take hidden activations as inputs and
similarly compute weighted sums plus bias followed by an activation function appropriate
to the task (e.g., logistic/softmax for classification or identity for regression).[file:3][file:4]
The collection of outputs y forms the network’s response; during training, backpropa-
gation uses derivatives of the activation functions to adjust weights from both hidden
and output layers so that the mapping from input to output approximates the desired
targets.[file:3]
[Insert Diagram Here – Source: Module-II [Link], Unit 2, Fig. 2.8.1 Basic structure
of Feed Forward Neural Network]
Reference: Module-II [Link] (Unit 2, Sections 2.7 and 2.8) and ANN-Technical-
[Link] (Unit 1, Sections 1.5 and 1.8 on multilayer feed–forward networks)[file:3][file:4]

(b) Explain perceptron learning algorithm and implement OR function using

Perceptron network. [8]

A single–layer perceptron is a feed–forward neuron that computes a weighted sum of its

inputs plus bias and passes it through a threshold (sign/step) activation to produce a
binary output.[file:4][file:2] Given input vector x = [x1 , x2 , . . . , xn ], weight vector w =
P
[w1 , . . . , wn ] and bias b, the net input is u = i wi xi + b and the output is typically
y = sgn(u) or a step function mapping to {0, 1}.[file:4][file:2]
Perceptron learning algorithm (supervised error–correction rule):[file:4][file:2]

1. Initialize all weights wi and bias b to small random values (or zero).

2. For each training pair (x, t), where t ∈ {0, 1} is the desired output, compute the
actual output y using the current weights and activation function.

26
 3. If y = t, no change is made; if y ̸= t, update each weight and bias using

∆wi = η(t − y) xi , wi ← wi + ∆wi , ∆b = η(t − y), b ← b + ∆b

where η is the learning rate.[file:4][file:2]

4. Repeat pattern presentations (epochs) until all training patterns are correctly clas-
sified or a maximum number of iterations is reached.[file:4]

Implementing OR function: The OR function for two binary inputs has the truth
table (0, 0) → 0, (0, 1) → 1, (1, 0) → 1, (1, 1) → 1.[file:4][file:2]

• Choose initial weights w1 = w2 = 0, bias b = 0, learning rate η = 1.

• For pattern (0, 0) with target t = 0: u = 0, y = 0; since y = t, no update.

• For pattern (0, 1), t = 1: u = 0, so y = 0 ̸= 1; updates: ∆w1 = 1(1 − 0) · 0 =

0, ∆w2 = 1(1 − 0) · 1 = 1, ∆b = 1. New parameters: w1 = 0, w2 = 1, b = 1.[file:2]

• For pattern (1, 0), t = 1: u = w1 · 1 + w2 · 0 + b = 0 + 0 + 1 = 1, so y = 1 = t; no

change.

• For pattern (1, 1), t = 1: u = 0 · 1 + 1 · 1 + 1 = 2, y = 1 = t; no change.

After one epoch, the perceptron has learned weights w1 = 0, w2 = 1, b = 1, which correctly
classify all four OR patterns; geometrically, the decision boundary is a line separating
the class (0, 0) from the other three input points.[file:4][file:2]
Reference: [Link] (Unit 1, Sections 1.5 and 1.8) and ANN Module 1 Lecture Not
(Perceptron learning rule and OR example)[file:4][file:2]

27