Artificial Neural Network (ANN) – In-Semester Exam

Answers

February 2023 Paper (Artificial Neural Network – In-

sem)
Q1
(a) What is a neural network activation function? State its Types. [5]
An activation function (also called transfer function) is a mathematical function applied
to the net input of a neuron (weighted sum plus bias) to produce its output in a non-
linear and bounded fashion.[file:4] It decides whether a neuron should be activated or
not, helps to normalize the output in ranges like 0 to 1 or −1 to 1, and gives a multi-
layer network greater representational power than a single-layer network by introducing
non-linearity.[file:4] During backpropagation, the differentiable nature of the activation
function allows gradients to be computed and the loss function to be minimized using
gradient descent.[file:4]
Let the net input to the activation function be
n
X
sum = xi w i + b
i=1

where xi are inputs, wi are corresponding synaptic weights, and b is the bias.[file:4] The
activation function f (sum) squashes this value into a desired output range and introduces
nonlinearity so that complex, non-linearly separable mappings can be learned.[file:4]

Types of activation functions (with explanation):

1. Identity or Linear Activation Function
A linear activation function is a simple straight-line function in which the output
is directly proportional to the weighted sum of inputs, typically written as f (x) =
ax + c.[file:4] When a = 1 and c = 0, this becomes the identity function f (x) = x,
producing outputs in the range (−∞, +∞) and yielding a convex error surface
which can be optimized easily but without introducing nonlinearity, so it is not
useful alone in deep networks.[file:4]
2. Logistic (Unipolar Sigmoid) Function
The logistic sigmoid produces an S–shaped curve and maps real-valued inputs into
the interval (0, 1).[file:4] It is given by
1
f (sum) =
1 + e−sum

1
 and monotonically increases from near 0 for large negative inputs to near 1 for large
positive inputs, with a value of 0.5 at zero; its derivative is simple and is heavily
used in older backpropagation networks.[file:4]

3. Hyperbolic Tangent (Bipolar Sigmoid) Function

The hyperbolic tangent function is also sigmoidal but maps inputs into the range
(−1, 1) and is zero-centered.[file:4] It is usually written as

esum − e−sum
f (sum) = tanh(sum) =
esum + e−sum
and often provides better convergence than logistic sigmoid because its outputs are
symmetric around zero, reducing bias in gradient updates.[file:4]

4. General Sigmoid Family

In general, a sigmoid function is any smooth, S–shaped, real-valued, differentiable
function with a non-negative or non-positive first derivative, one local minimum
and one local maximum; sigmoid functions like logistic and hyperbolic tangent are
widely used to introduce nonlinearity and keep outputs bounded.[file:4]

5. Arc Tangent Activation Function

The arctangent activation uses f (sum) = π2 arctan(sum), mapping real inputs
smoothly into a finite interval similar to a squashing function.[file:4] It is mono-
tonic and differentiable, and can also be used in multilayer networks to increase
representational power when nonlinearity is required.[file:4]

6. ReLU (Rectified Linear Unit) and Related Variants

The ReLU function is defined as f (z) = max(0, z); it outputs zero for negative in-
puts and a linear response for positive inputs.[file:4] It is computationally efficient,
accelerates convergence compared to sigmoids because it avoids expensive exponen-
tial operations, and is widely used in deep and convolutional neural networks.[file:4]

Reference: [Link], Unit I, Section 1.6 Activation Functions (pages

1-18 to 1-23).[file:4]

(b) Explain architecture of Artificial Neural Network with a neat diagram.

[5]
An artificial neural network (ANN) is a computational system inspired by the structure
and processing method of the biological brain; it is composed of many simple processing
elements (artificial neurons) that are linked together according to a specific network
architecture.[file:4] A neural network transforms input vectors into meaningful outputs
by adjusting connection weights such that the overall mapping approximates the desired
input–output relationship.[file:4]
General layered architecture:[file:4][file:1]

• Input layer: This layer consists of source nodes which receive the external input
vector (features) and simply distribute these values to the next (hidden) layer with-
out performing computation; each input node corresponds to one component of the
input pattern.[file:4][file:1]

2
 • Hidden layer(s): One or more hidden layers of computation nodes process the
inputs by computing weighted sums of their inputs plus bias and then applying acti-
vation functions such as sigmoid or ReLU; these layers extract intermediate features
and enable the network to model complex, non-linear relationships.[file:4][file:1]
• Output layer: The output layer receives signals from the last hidden layer and
combines them using weights and activation functions to produce the final output
vector (e.g., class labels or predicted numeric values).[file:4][file:1]
Neurons in one layer are usually fully connected to neurons in the next layer, and
each connection has an associated weight which represents the strength of influence be-
tween neurons; learning algorithms adjust these weights so that the network maps inputs
to desired outputs with minimal error.[file:4] The architecture is typically feed-forward
(information flows from input to output without cycles), but recurrent architectures with
feedback connections also exist for sequential data; however, the basic multilayer feed-
forward architecture (input, hidden, output) is the most common for many pattern recog-
nition and regression tasks.[file:4][file:3]
Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-2 (Fig. 1.1.1
Artificial Neural Network)
[file:4]
Reference: [Link], Unit I, Sections 1.4 Introduction to Neural Net-
work and 1.5 Neural Net Architecture (pages 1-4 to 1-6 and 1-9 to 1-13); ANN [Link],
slides on ANN layers.[file:4][file:1]

(c) Explain Mc-Culloch & Pitts model with an example. [5]

The McCulloch–Pitts (M–P) model is the first mathematical model of a biological neuron
and forms the basic building block of early neural networks; it describes a neuron as a
logical threshold element with two possible output states (0 or 1).[file:4][file:2] The neuron
has N binary input channels and a single binary output channel; each input xi is either
active (1) or inactive (0), and the neuron computes a weighted sum of its inputs and
compares this effective input with a fixed threshold T .[file:4]
Mathematically, the model can be represented as
Xm
a = φ(u), u = w j xj − T
j=1

where wj are synaptic weights, xj are inputs and φ(·) is a step (Heaviside) activation
function defined as (
1 if u ≥ 0,
φ(u) =
0 if u < 0.
[file:4] The neuron “fires” (output 1) when its net input (weighted sum) exceeds the
threshold; otherwise it remains inactive (output 0).[file:4]
The M–P model operates under assumptions: neurons are binary devices, each neu-
ron has a fixed threshold, excitatory synapses all have identical positive weights, and
inhibitory inputs have veto power (any inhibitory input active forces the neuron output
to 0 regardless of excitatory input).[file:4][file:2] Because the activation function is a step
function taking only two values, the M–P neuron is often called a discrete neuron, and by
combining several such neurons, simple logical functions can be implemented.[file:4][file:2]

3
Example: Realization of AND function Consider a neuron with two binary inputs
x1 , x2 ∈ {0, 1} and weights w1 = w2 = 1 with threshold T = 2.[file:2]

• For input (x1 , x2 ) = (0, 0): net input u = 1 · 0 + 1 · 0 − 2 = −2 < 0, so output a = 0.

• For input (x1 , x2 ) = (0, 1): net input u = 0 + 1 − 2 = −1 < 0, output a = 0.

• For input (x1 , x2 ) = (1, 0): net input u = 1 + 0 − 2 = −1 < 0, output a = 0.

• For input (x1 , x2 ) = (1, 1): net input u = 1 + 1 − 2 = 0 ≥ 0, output a = 1.

Thus, the M–P neuron with these weights and threshold implements the logical AND
function, firing only when both inputs are 1.[file:2]

Limitations (also applicable wherever the model is discussed):

• Weights and thresholds are analytically determined and fixed; the M–P neuron
cannot learn from data by itself.[file:4]

• The model is too simplistic and only produces binary outputs with a step func-
tion, so it cannot handle graded responses or complex non-linearly separable prob-
lems; minimizing network size while implementing arbitrary logic is also very diffi-
cult.[file:4][file:2]

Reference: [Link], Unit I, Section 1.7 McCulloch and Pitts Model

(pages 1-23 to 1-25); ANN Module 1 Lecture [Link], McCulloch–Pitts model and
Boolean function examples.[file:4][file:2]

Q2
(a) Main differences among McCulloch–Pitts, Perceptron and Adaline models.
[5]
The following table summarizes the key differences among the three artificial neuron
models (number of comparison points > 5 as required):

4
Feature McCulloch–Pitts Perceptron Adaline (Adaptive
Neuron Linear Neuron)
Output type Strictly binary output Binary output (often Linear output (net in-
(0 or 1) using a hard {−1, 1} or {0, 1}) pro- put) used during learn-
step function as activa- duced by applying a ing; quantizer may map
tion.[file:4][file:2] threshold function to it to {−1, 1} for final
the weighted sum plus decision, but learning is
bias.[file:4] based on continuous net
value.[file:4][file:2]
Learning capa- No learning rule; Uses Perceptron learn- Uses delta rule /
bility weights and threshold ing rule that updates Least Mean Squares
are fixed and analyt- weights when classifica- (Widrow–Hoff) learning
ically chosen; cannot tion is incorrect; can procedure to minimize
adapt to data.[file:4] learn linearly separable mean squared error
patterns via supervised between desired and ac-
learning.[file:4][file:2] tual outputs; performs
gradient descent on er-
ror surface.[file:4][file:2]
Error measure No explicit error func- Classification error Explicit squared error
tion defined; purely log- (misclassified patterns) measure Ep = (tp −
ical model.[file:4] guides weight updates; op )2 for each pattern
no explicit MSE opti- and mean squared error
mization.[file:4] over all patterns; learn-
ing aims to minimize
this error.[file:4]
Activation func- Hard step function with Hard threshold (sign or Linear activation dur-
tion fixed threshold; discrete step) function over net ing learning; gradient is
neuron.[file:4] input; nonlinear but continuous, enabling ef-
non-differentiable; out- ficient optimization us-
put is binary.[file:4] ing calculus; quantizer
can be added only for fi-
nal decision.[file:4]
Problem types Models simple logical Can classify only lin- Performs linear dis-
operations and binary early separable patterns crimination and
decisions; mainly theo- such as AND, OR, regression; using
retical model of a neu- NAND, NOR; fails LMS, it approximates
ron.[file:4] on XOR and other continuous-valued tar-
non-linearly separable get functions and can
problems.[file:4] be used in adaptive
filters.[file:4][file:2]
Network usage Used historically to Forms the basis of Serves as basic building
show that networks of single-layer feed- block for adaptive linear
simple threshold units forward networks with systems and MADA-
can compute logical supervised learning; LINE (Multiple Ada-
functions; not used foundation for mul- line) networks; widely
directly for learning in tilayer perceptron used in adaptive signal
practice.[file:4] architectures.[file:4] processing.[file:4][file:2]

5
Reference: [Link], Unit I, Sections 1.7 McCulloch–Pitts, 1.8 Per-
ceptron, 1.8.3 Adaline Network (pages 1-23 to 1-31, 1-29 to 1-31, 1-30 to 1-31); ANN Module 1 Lecture N
Adaline and Madaline sections.[file:4][file:2]

(b) Explain the structure and working of Biological Neural Network. [5]
A biological neural network is the network of biological neurons present in the human
brain and nervous system; it is responsible for thinking, learning, memory and decision-
making.[file:4][file:2] The elementary building block is the neuron, which is an electrically
excitable cell that receives, processes and transmits information using bioelectrical and
biochemical processes.[file:4][file:2]

Structure of a biological neuron: A typical neuron consists of the following main

parts:[file:4][file:2]

• Cell body (soma): Contains the nucleus (genetic material DNA) and most of the
protein-synthesis machinery; it integrates incoming signals and generates outgoing
signals to the axon.[file:4][file:2]

• Dendrites: Tree-like branching fibers that emanate from the soma and form the
receptive zone of the neuron; they receive synaptic inputs (electrical signals) from
the axons of other neurons.[file:4][file:2]

• Axon: A single, long cable-like projection that carries nerve impulses (action po-
tentials) away from the soma toward other neurons or effector cells; it may extend
from centimeters up to more than a meter in length.[file:4][file:2]

• Axon hillock and terminals: The axon hillock is the region where the axon
emerges from the soma and where action potentials are typically initiated; axon
terminals (synaptic boutons) at the far end form synapses with target cells and
release neurotransmitters.[file:4][file:2]

• Synapse and synaptic cleft: The junction between the axon terminal of the
presynaptic neuron and the dendrite or soma of the postsynaptic neuron; neuro-
transmitters diffuse across the synaptic cleft to convey signals.[file:4][file:2]

Working (signal transmission) of a biological neural network:

• Signal reception and integration: Dendrites receive excitatory and inhibitory

postsynaptic potentials from many presynaptic neurons; the soma sums these inputs
spatially and temporally to form a membrane potential at the axon hillock.[file:4][file:2]

• Threshold and action potential: If the summed potential exceeds a thresh-

old, an all-or-nothing action potential (spike) is generated at the axon hillock and
propagated along the axon without attenuation.[file:4][file:2]

• Synaptic transmission: When the action potential reaches axon terminals, it

triggers the release of neurotransmitter molecules into the synaptic cleft; these
chemicals bind to receptors on the postsynaptic neuron, creating new postsynaptic
potentials and propagating information through the network.[file:4][file:2]

6
 • Network behavior: The brain, with around 1010 interconnected neurons and
a huge number of synapses, performs massively parallel processing; learning and
memory arise from long-term changes in synaptic strengths and connectivity pat-
terns.[file:4][file:2]

Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-4 (Fig. 1.3.1
Schematic of Biological Neuron)
[file:4]
Reference: [Link], Unit I, Section 1.3 Structure and Working of Bi-
ological Neural Network (pages 1-4 to 1-5); ANN Module 1 Lecture [Link], Biological
neuron structure and functions.[file:4][file:2]

(c) Differentiate between Biological Neural Network and Artificial Neural

Network. [5]
The following table presents the differences (with more than 6 comparison points) between
biological neural networks (BNN) and artificial neural networks (ANN):[file:4][file:2]

7
Aspect Biological Neural Net- Artificial Neural Net-
work (BNN) work (ANN)
Basic element Consists of biological neu- Consists of artificial neu-
rons (cells) with soma, den- rons (processing units) im-
drites, axon and synapses plemented as mathemati-
formed by biochemical cal functions or hardware
structures.[file:4][file:2] units with inputs, weights,
bias and activation func-
tion.[file:4][file:1]
Size and connec- Human brain has about 86 Typical ANNs have from
tivity billion neurons with more a few dozen to millions
than 1014 synapses; connec- of artificial neurons and
tivity is highly complex and weighted connections, usu-
irregular.[file:2] ally organized in regular
layers and simple topolo-
gies.[file:4][file:1]
Signal type and Signals are electrochemical Signals are numeric values
transport (action potentials and neu- (real numbers) propagated
rotransmitters), propagated synchronously according to
asynchronously with vari- algorithmic time steps; out-
able speeds; neurons fire in puts are continuous or dis-
spikes.[file:4][file:2] crete values.[file:4][file:2]
Processing speed Single biological neurons Individual arithmetic oper-
are relatively slow (millisec- ations are extremely fast
onds per spike), but mas- (nanoseconds), but paral-
sive parallelism yields high lelism is limited by hard-
overall processing capabil- ware architecture; ANNs
ity.[file:4][file:2] can still exploit SIMD/GPU
parallelism.[file:4][file:1]
Topology and Network topology is com- Topology is usually prede-
structure plex, three-dimensional, dy- fined (e.g., feed-forward, re-
namic and self-organizing; current) with fixed number
neurons can grow new of layers and connections;
connections or prune old only weights (and some-
ones.[file:2] times biases) are adjusted
during learning.[file:4][file:1]
Learning and Learning involves long-term Learning is defined by
adaptation synaptic plasticity, neu- explicit algorithms (e.g.,
rogenesis and biochemical Hebbian, perceptron rule,
changes; rules are not backpropagation, delta
yet fully understood and rule) that adjust weights
operate continuously.[file:2] based on error or cor-
relation; training and
inference phases are dis-
tinct.[file:4][file:2]
Fault tolerance Highly fault-tolerant; loss or Fault tolerance depends
and robustness damage of individual neu- on architecture and imple-
rons rarely destroys func- mentation; some robustness
tionality due to redundancy exists due to distributed
and distributed representa- storage in weights, but
8
tion.[file:2] hardware failures can
abruptly stop computa-
tion.[file:4][file:1]
Reference: [Link], Unit I, comparison between Biological NN and
Artificial NN (page 1-5); ANN Module 1 Lecture [Link], section “STATE THE MA-
JOR DIFFERENCES BETWEEN BIOLOGICAL AND ARTIFICIAL NEURAL NET-
WORKS”.[file:4][file:2]

Q3
(a) Explain Perceptron Learning Algorithm with an example. [5]
The perceptron is a single-layer feed-forward network whose neuron computes a weighted
sum of inputs plus bias and then applies a threshold function to produce a binary output;
the perceptron learning algorithm adjusts the weights so that all training patterns that
are linearly separable are correctly classified.[file:4][file:2]

Perceptron learning rule and algorithm: Let input vector be x = [x1 , x2 , . . . , xn ]T ,

weight vector w = [w1 , w2 , . . . , wn ]T , bias b and desired output d ∈ {−1, 1}.[file:4][file:2]

1. Initialization: Initialize all weights and bias to small random values (often zero
or small random numbers).[file:4][file:2]

2. Activation (forward computation): For each training pattern, compute

n
X
u= wi xi + b, y = sgn(u)
i=1

where sgn(u) = 1 if u ≥ 0 and −1 otherwise.[file:4]

3. Weight update: If the classification is correct (y = d), do nothing; if incorrect

(y ̸= d), update the weight and bias using

∆wi = η(d − y) xi , wi ← wi + ∆wi , b ← b + η(d − y)

where η is the learning rate.[file:4][file:2]

4. Iteration: Repeat steps over all patterns until no misclassification occurs (or until a
maximum number of epochs is reached); for linearly separable data, the perceptron
convergence theorem guarantees convergence in a finite number of iterations.[file:4]

Example: Implementing OR function (also used wherever an OR-example

perceptron is asked) Consider two-input OR function with inputs (x1 , x2 ) ∈ {0, 1}2
and desired output d ∈ {0, 1} given by the truth table:

x1 x 2 d
0 0 0
0 1 1
1 0 1
1 1 1

[file:2]

9
 We use bipolar encoding x′i = 2xi − 1 ∈ {−1, 1} and d′ = 2d − 1, so the patterns
become:
x′1 x′2 d′
−1 −1 −1
−1 1 1
1 −1 1
1 1 1
[file:2]
Initialize w1 = 0, w2 = 0, b = 0 and choose η = 1.[file:2]

• Pattern 1: x′ = [−1, −1], d′ = −1. Net input u = 0, y = sgn(0) = 1 (misclassified,

since desired is −1). Update:

∆w1 = 1(−1 − 1)(−1) = 2, ∆w2 = 2, ∆b = 1(−1 − 1) = −2,

giving w1 = 2, w2 = 2, b = −2.

• Pattern 2: x′ = [−1, 1], d′ = 1. Net input u = 2(−1) + 2(1) − 2 = −2, so y = −1

(misclassified). Update:

∆w1 = 1(1 − (−1))(−1) = −2, ∆w2 = 1(1 − (−1))(1) = 2, ∆b = 1(1 − (−1)) = 2,

giving w1 = 0, w2 = 4, b = 0.

• Pattern 3: x′ = [1, −1], d′ = 1. Net input u = 0(1) + 4(−1) + 0 = −4, y = −1

(misclassified). Update:

∆w1 = 1(1 − (−1))(1) = 2, ∆w2 = 1(1 − (−1))(−1) = −2, ∆b = 1(1 − (−1)) = 2,

giving w1 = 2, w2 = 2, b = 2.

• Pattern 4: x′ = [1, 1], d′ = 1. Net input u = 2(1) + 2(1) + 2 = 6 > 0, so y = 1

(correct); no update.

Repeating the patterns again will show that all are now classified correctly, so the
perceptron with learned weights w1 = 2, w2 = 2, b = 2 implements the OR function in
bipolar form.[file:2]
Reference: [Link], Unit I, Section 1.8 Perceptron and Perceptron
Learning Algorithm (pages 1-26 to 1-27); ANN Module 1 Lecture [Link], sections on
Perceptron learning rule and OR example.[file:4][file:2]

(b) Explain the architecture of Multilayered neural network. [5]

A multilayer neural network, often called a multilayer perceptron (MLP), consists of
multiple layers of adaptive units: an input layer, one or more hidden layers, and an
output layer; it forms a feed-forward architecture where signals propagate from input to
output without cycles.[file:4][file:3] Every neuron in one layer has directed connections to
neurons in the next layer and is associated with a set of weights and a bias; hidden and
output units apply nonlinear activation functions (commonly sigmoid or ReLU).[file:4]

10
Layers and computations in a multilayer network:

• Input layer: Contains source nodes which receive the external input vector;
this layer does not perform computation, only distributes inputs to the hidden
layer.[file:4][file:3]

• Hidden layer(s): Each hidden neuron computes

X
uj = wij xi + bj , yj = f (uj ),
i

where f is typically a sigmoid (logistic or hyperbolic tangent) function; these neu-

rons learn internal representations and can enable the network to solve non-linearly
separable problems such as XOR.[file:4][file:3]

• Output layer: Each output neuron similarly computes a weighted sum of hidden-
layer outputs plus bias and applies an activation function to generate the final
network outputs; for classification, softmax or sigmoids may be used, and for re-
gression, linear activations are common.[file:4]

In a typical three-layer MLP, the first layer acts as feature detectors hardwired to
input signals, the hidden layer combines these features into more abstract patterns, and
the output layer performs classification or function approximation.[file:4] Weights are
adjusted using supervised learning algorithms such as backpropagation, which computes
gradients of an error function with respect to all weights and updates them via gradient
descent to minimize overall error.[file:4][file:3]
Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-28 (Fig. 1.8.2
Multilayer Perceptron Model)
[file:4]
Reference: [Link], Unit I, Sections 1.5.2 Multi-Layer Feed Forward
Network and 1.8.2 Multilayer Perceptron (pages 1-12 to 1-13 and 1-28 to 1-29); Module-
II [Link], Unit II, Multilayer network architecture overview.[file:4][file:3]

(c) Write and explain the steps of Back Propagation Learning algorithm. [5]
Backpropagation is a supervised learning algorithm for training multilayer feed-forward
neural networks; it is a gradient descent method that minimizes the total squared error
between desired and actual outputs.[file:4][file:3] It generalizes the Widrow–Hoff error-
correction rule to multiple layers by propagating error gradients backward from output
to hidden layers.[file:4]

Backpropagation learning steps (for a typical MLP):

1. Initialization: Initialize all weights and biases to small random values (both pos-
itive and negative) to avoid saturation and symmetry.[file:3]

2. Forward pass (for each training pair):

• Apply the input vector x to the input layer.

11
 • For each hidden neuron, compute net input uj = i wij xi + bj and output
P
yj = f (uj ).
• For each output neuron k, compute uk =
P
j wjk yj + bk and output ok =
f (uk ).[file:4][file:3]
3. Error computation: Compare the network output with the desired target d and
compute an error measure, typically squared error for pattern p,
1X
Ep = (dk − ok )2 .
2 k

[file:4][file:3]
4. Backward pass (error backpropagation):
• For each output neuron k, compute its error term
δk = (dk − ok )f ′ (uk ).

• For each hidden neuron j, compute its error term using the errors of neurons
in the next layer X
δj = f ′ (uj ) wjk δk ,
k

where the sum is over all units in the layer above.[file:3]

5. Weight and bias update (gradient descent):
• Update weights from hidden to output layer:
∆wjk = η δk yj , wjk ← wjk + ∆wjk .

• Update weights from input to hidden layer:

∆wij = η δj xi , wij ← wij + ∆wij .

• Update biases similarly:

bk ← bk + η δk , bj ← bj + η δj .
[file:3]
6. Iteration over training set: Repeat forward and backward passes for all training
patterns; one complete pass over the training set is called an epoch.[file:3]
7. Stopping criterion: Continue training epochs until the total error over the train-
ing set is acceptably low or does not decrease significantly, or until a maximum
number of epochs is reached.[file:3]
Backpropagation training therefore consists of two phases: in the forward pass, the
input signals move from input to output, and in the backward pass, error signals prop-
agate back to adjust weights; this iterative process gradually reduces the mapping er-
ror.[file:4][file:3]
Reference: [Link], Unit I, Backpropagation Learning Rule (pages
1-15 to 1-17); Module-II [Link], Unit II, Section 2.7 Backpropagation and training
procedure (pages 2-20 to 2-22).[file:4][file:3]

12
Q4
(a) Draw the architecture of Back Propagation Network and explain in detail.
[5]
A backpropagation network is a multilayer feed-forward neural network trained using
the backpropagation algorithm; it usually consists of an input layer, one or more hidden
layers, and an output layer with differentiable nonlinear activation functions (e.g., logistic
or hyperbolic tangent) in hidden and output neurons.[file:4][file:3]

Architecture:

• Input layer: Composed of source nodes that pass the input vector to the network;
no computation is performed here.[file:3]

• Hidden layer(s): Each hidden neuron receives inputs from all units in the previous
layer; it computes a weighted sum plus bias and passes this through a nonlinear
differentiable activation function (sigmoid, tanh, etc.).[file:4][file:3]

• Output layer: Each output neuron receives signals from the last hidden layer,
computes weighted sums plus bias and applies an activation function to produce
final outputs.[file:4]

• Connections: Weights connect neurons only between adjacent layers; there are
no lateral or recurrent connections in the standard backpropagation network (it is
feed-forward during operation).[file:3]

In a typical two-hidden-layer example, the network is often denoted as a n1 −n2 −· · ·−

nL structure, where n1 is number of input units, intermediate numbers are hidden units
per layer, and the last is number of output units; all weights and biases in this structure
are trainable parameters updated by graded error signals flowing backwards.[file:4][file:3]
During training, an input pattern produces outputs via a forward pass; then backpropaga-
tion computes error gradients at the output layer and transports them backward through
hidden layers so that all connection weights are adapted to minimize overall squared
error.[file:4][file:3]
Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-16 (Fig. 1.5.7
Two-Layer Backpropagation MLP) and Module-II [Link], Page No: 2-20 (Fig. 2.7.1
Backpropagation Network)
[file:4][file:3]
Reference: [Link], Unit I, Backpropagation MLP diagrams (pages
1-15 to 1-17); Module-II [Link], Unit II, Section 2.7 Backpropagation (pages 2-20 to
2-22).[file:4][file:3]

(b) Differentiate between Feed forward and Feedback neural network. [5]
The differences between feed-forward and feedback (recurrent) neural networks are sum-
marized below:[file:4][file:3]

13
 Aspect Feed-forward Neural Feedback / Recurrent
Network Neural Network
Connection direc- Connections are unidirec- Connections may form
tion tional; signals flow strictly loops; outputs of neurons
from input layer to hid- can be fed back as inputs to
den layer(s) to output layer, the same or previous layers,
with no cycles.[file:3] permitting cycles.[file:3]
Information flow Only forward flow of in- Information can travel in
formation; no backward both directions; network
flow during normal opera- state evolves over time
tion (backprop is only dur- due to feedback connec-
ing training).[file:3] tions.[file:3]
Memory of past No internal dynamic mem- Has internal state and mem-
inputs ory of input sequence; out- ory; outputs depend on
put depends only on current current input and previ-
input (apart from trained ous internal states, suitable
weights).[file:3] for temporal and sequential
data.[file:3]
Dynamics Static input–output map- Dynamic system whose
ping; after input is ap- state may change until
plied, output is computed in equilibrium is reached;
a finite number of forward often used as content-
computations.[file:3] addressable memory or
temporal pattern recog-
nizer.[file:3]
Typical applica- Pattern classification, re- Time-series prediction,
tions gression, function approxi- language modeling, speech
mation where data are not recognition, associative
time-dependent (e.g., image memory, and problems
recognition, static pattern where sequence or context
recognition).[file:3] is important.[file:4][file:3]
Training complex- Training is generally sim- Training is more complex
ity pler; standard backpropa- due to recurrent connec-
gation is directly applicable tions; requires specialized
and stable for many prob- algorithms (e.g., backprop-
lems.[file:3] agation through time) and
careful handling of stabil-
ity.[file:3]

Reference: Module-II [Link], Unit II, Section 2.8 Feed Forward Neural Networks and
2.8.1 Feedback Neural Networks (pages 2-24 to 2-26); [Link], Unit I,
Neural net architecture overview.[file:3][file:4]

(c) What is Error Correction and how to minimize these errors? [5]
Error-correction learning is a supervised learning paradigm in which unknown ANN pa-
rameters (weights and biases) are adapted so as to minimize the difference between desired
outputs and actual network outputs; the error signal drives a control mechanism that up-
dates parameters to reduce an error function, usually mean squared error.[file:4][file:3] For

14
neuron k, with desired output dk (n) and actual output yk (n) at time n, the instantaneous
error is
ek (n) = dk (n) − yk (n),
and a commonly used error function over all output neurons is
1X
E= E[e2k (n)],
2 k

where E[·] denotes expectation.[file:3]

Minimizing errors (principles and practical methods):

• Gradient descent on error surface: The most common method is to apply

gradient descent, adjusting weights in the negative direction of the gradient of the
error function with respect to each weight:
∂E
wij (n + 1) = wij (n) − η ,
∂wij

where η is the learning rate; this gradually reduces mean squared error.[file:3]

• Use of local error signals: In multilayer networks, local error terms are computed
(e.g., in backpropagation) for each neuron, and weights are updated proportionally
to the product of local error and input to that weight (delta rule / Widrow–Hoff
rule).[file:4][file:3]

• Choice of learning rate and momentum: A small learning rate ensures stability
but may slow convergence, whereas a large learning rate may cause oscillations;
adding momentum combines current gradient with previous update to smooth and
accelerate convergence while helping avoid shallow local minima.[file:3] (Source:
Internet)[web:13]

• Appropriate network architecture and regularization: Selecting a suitable

number of hidden units and layers helps avoid underfitting (high bias error) and
overfitting (high variance error); regularization techniques (e.g., weight decay, early
stopping) further control error on unseen data. (Source: Internet)[web:51]

• Quality of training data: Reducing noise in training data, ensuring represen-

tative coverage of the input space, and proper normalization/scaling of inputs can
significantly reduce training and generalization errors.[file:3]

Thus, error-correction learning aims to minimize a well-defined error function, most

often mean squared error, by systematically updating weights and biases based on the
observed error between desired and actual outputs.[file:3]
Reference: Module-II [Link], Unit II, Section 2.4 Error Correction (pages 2-12 to
2-13); [Link], Unit II, Error Correction and Gradient Descent Rules
(syllabus and related text).[file:3][file:4]

15
March 2024 Paper (Artificial Neural Network – In-
sem)
Q1
(a) What is the role of activation function in neural network? Explain bipolar
Sigmoid function in detail. [5]
The activation function in a neural network maps the net input (weighted sum plus
bias) of a neuron to its output in a nonlinear and bounded way; it decides whether
the neuron should be activated and determines how strongly it contributes to the next
layer.[file:4] By normalizing outputs (e.g., between 0 and 1 or −1 and 1) and introducing
nonlinearity, activation functions allow multilayer networks to approximate complex, non-
linearly separable mappings, and they provide differentiable transformations required for
backpropagation learning.[file:4][file:2]

Role of activation function:

• Nonlinearity: Without activation functions, stacked linear layers collapse into a

single linear transformation; nonlinear activations enable networks to approximate
arbitrary nonlinear functions.[file:4][file:2]

• Normalization and bounded outputs: Many activation functions (e.g., sig-

moid, tanh) squash inputs into a limited range, improving numerical stability and
interpretability.[file:4]

• Enabling gradient-based learning: Differentiable activations provide gradients

used in backpropagation to update weights via gradient descent; good activations
help avoid issues like vanishing gradients.[file:4][file:3]

Bipolar sigmoid (hyperbolic tangent) function in detail: The bipolar sigmoid

function commonly used in neural networks is the hyperbolic tangent function, which
is an S-shaped, zero-centered squashing function mapping real inputs into the interval
(−1, 1).[file:4][file:2] Its general form is

esum − e−sum
f (sum) = tanh(sum) = .
esum + e−sum
Key properties:[file:4][file:2]

• Range: Output lies in (−1, 1); for large negative inputs it approaches −1, for
large positive inputs it approaches +1, and at zero input it yields 0, making it
zero-centered.

• Smoothness and differentiability: It is continuous and differentiable for all real

inputs; its derivative is
f ′ (sum) = 1 − f 2 (sum),
which is easily computed from its output and used in backpropagation.

16
 • Zero-centered output: Because outputs are centered around zero, the bipolar
sigmoid often leads to faster convergence than the standard logistic sigmoid, reduc-
ing bias in gradient updates.

• Use in multilayer networks: Hyperbolic tangent activations are widely used in

hidden layers of multilayer feed-forward networks, especially in older architectures,
to increase representational power and improve training behavior.[file:4][file:3]
Reference: [Link], Unit I, Sections 1.6 Activation Functions and
1.6.3 Tanh and ReLU (pages 1-18 to 1-22); ANN Module 1 Lecture [Link], Sigmoid
and bipolar sigmoid activation functions.[file:4][file:2]

(b) Why is ReLU the most commonly used Activation Function? [5]
The Rectified Linear Unit (ReLU) activation function is defined as

f (z) = max(0, z),

i.e., it outputs zero for negative inputs and z itself for positive inputs.[file:4] It has be-
come the most commonly used activation function in deep neural networks because of its
simplicity, computational efficiency, and favorable gradient properties.[file:4][web:48]

Reasons ReLU is widely used:

• Computational simplicity: ReLU involves only a comparison with zero, avoiding
expensive exponential operations required in sigmoid and tanh; this makes forward
and backward passes faster, which is crucial in very deep networks with millions of
parameters.[file:4][web:48] (Source: Internet)

• Nonlinearity with piecewise linear form: Although piecewise linear, ReLU is

still nonlinear and enables deep networks to learn complex decision boundaries and
non-linear mappings in a stable manner.[file:4][web:51] (Source: Internet)

• Mitigation of vanishing gradient problem: For positive inputs, the derivative

of ReLU is 1, so gradients do not vanish as easily as with sigmoid/tanh, which
saturate at extreme values; this allows gradients to propagate through many layers
and facilitates training of deep architectures.[web:48][web:46] (Source: Internet)

• Sparse activations: ReLU outputs exactly zero for negative inputs, so at any
given time many neurons are inactive; this sparsity can improve generalization and
reduce computational load, as only a subset of neurons are effectively contributing
to representations.[web:48][web:55] (Source: Internet)

• Empirical success in deep learning: Most successful deep CNNs and other deep
architectures (e.g., AlexNet and later models) have used ReLU or its variants as
the default hidden-layer activation, and extensive experiments show that networks
with ReLU typically converge faster and achieve better performance than those
using only sigmoid/tanh.[file:4][web:55] (Source: Internet)
Reference: [Link], Unit I, Section 1.6.3 Tanh and ReLU (page 1-22
to 1-23); GeeksforGeeks, “ReLU Activation Function in Deep Learning”; BuiltIn, “ReLU
Activation Function Explained”. (Source: Internet)[file:4][web:48][web:46]

17
(c) Explain architecture of Artificial Neural Network with a neat diagram. [5]
Same explanation as Q1(b), February 2023 Paper, to ensure consistency for the “archi-
tecture of ANN” topic.
An artificial neural network (ANN) is a computational system inspired by the struc-
ture and processing method of the biological brain; it is composed of many simple process-
ing elements (artificial neurons) that are linked together according to a specific network
architecture.[file:4] A neural network transforms input vectors into meaningful outputs
by adjusting connection weights such that the overall mapping approximates the desired
input–output relationship.[file:4]
General layered architecture:[file:4][file:1]

• Input layer: This layer consists of source nodes which receive the external input
vector (features) and simply distribute these values to the next (hidden) layer with-
out performing computation; each input node corresponds to one component of the
input pattern.[file:4][file:1]

• Hidden layer(s): One or more hidden layers of computation nodes process the
inputs by computing weighted sums of their inputs plus bias and then applying acti-
vation functions such as sigmoid or ReLU; these layers extract intermediate features
and enable the network to model complex, non-linear relationships.[file:4][file:1]

• Output layer: The output layer receives signals from the last hidden layer and
combines them using weights and activation functions to produce the final output
vector (e.g., class labels or predicted numeric values).[file:4][file:1]

Neurons in one layer are usually fully connected to neurons in the next layer, and
each connection has an associated weight which represents the strength of influence be-
tween neurons; learning algorithms adjust these weights so that the network maps inputs
to desired outputs with minimal error.[file:4] The architecture is typically feed-forward
(information flows from input to output without cycles), but recurrent architectures with
feedback connections also exist for sequential data; however, the basic multilayer feed-
forward architecture (input, hidden, output) is the most common for many pattern recog-
nition and regression tasks.[file:4][file:3]
Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-2 (Fig. 1.1.1
Artificial Neural Network)
[file:4]
Reference: [Link], Unit I, Sections 1.4 Introduction to Neural Net-
work and 1.5 Neural Net Architecture (pages 1-4 to 1-6 and 1-9 to 1-13); ANN [Link],
slides on ANN layers.[file:4][file:1]

Q2
(a) Draw the structure of the biological neuron and explain working of the
same in brief. [5]
Same explanation as Q2(b), February 2023 Paper, to keep the structure and working of
the biological neuron consistent.

18
 A biological neural network is the network of biological neurons present in the human
brain and nervous system; it is responsible for thinking, learning, memory and decision-
making.[file:4][file:2] The elementary building block is the neuron, which is an electrically
excitable cell that receives, processes and transmits information using bioelectrical and
biochemical processes.[file:4][file:2]

Structure of a biological neuron: A typical neuron consists of the following main

parts:[file:4][file:2]
• Cell body (soma): Contains the nucleus (genetic material DNA) and most of the
protein-synthesis machinery; it integrates incoming signals and generates outgoing
signals to the axon.[file:4][file:2]
• Dendrites: Tree-like branching fibers that emanate from the soma and form the
receptive zone of the neuron; they receive synaptic inputs (electrical signals) from
the axons of other neurons.[file:4][file:2]
• Axon: A single, long cable-like projection that carries nerve impulses (action po-
tentials) away from the soma toward other neurons or effector cells; it may extend
from centimeters up to more than a meter in length.[file:4][file:2]
• Axon hillock and terminals: The axon hillock is the region where the axon
emerges from the soma and where action potentials are typically initiated; axon
terminals (synaptic boutons) at the far end form synapses with target cells and
release neurotransmitters.[file:4][file:2]
• Synapse and synaptic cleft: The junction between the axon terminal of the
presynaptic neuron and the dendrite or soma of the postsynaptic neuron; neuro-
transmitters diffuse across the synaptic cleft to convey signals.[file:4][file:2]

Working (signal transmission) of a biological neuron:

• Signal reception and integration: Dendrites receive excitatory and inhibitory
postsynaptic potentials from many presynaptic neurons; the soma sums these inputs
spatially and temporally to form a membrane potential at the axon hillock.[file:4][file:2]
• Threshold and action potential: If the summed potential exceeds a thresh-
old, an all-or-nothing action potential (spike) is generated at the axon hillock and
propagated along the axon without attenuation.[file:4][file:2]
• Synaptic transmission: When the action potential reaches axon terminals, it
triggers the release of neurotransmitter molecules into the synaptic cleft; these
chemicals bind to receptors on the postsynaptic neuron, creating new postsynaptic
potentials and propagating information through the network.[file:4][file:2]

Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-4 (Fig. 1.3.1
Schematic of Biological Neuron)
[file:4]
Reference: [Link], Unit I, Section 1.3 Structure and Working of Bi-
ological Neural Network (pages 1-4 to 1-5); ANN Module 1 Lecture [Link], Biological
neuron structure and functions.[file:4][file:2]

19
(b) Write an algorithm of ADALINE and focus on its upper bound with
largest Eigen Value of its correlation matrix. [10]
ADALINE (Adaptive Linear Neuron) is an early single-layer artificial neural network
that uses a linear activation for learning and the delta (LMS) rule to minimize the mean
squared error between desired and actual outputs.[file:4][file:2] It is based on the idea that
for each training pattern, the error is computed as the difference between desired target
and net output, and weights are updated proportionally to the negative gradient of the
error surface.[file:4]

Standard ADALINE training algorithm: [file:4][file:2][web:54] (Source: Internet

where explicitly cited)
1. Initialization: Set all synaptic weights wi and bias b to small random values (often
zero for simplicity) and choose a learning rate η.[file:2][web:54] (Source: Internet)
2. For each bipolar training pair (s, t):
(a) Input activation: For each input unit, set xi = si , where si is the i-th
component of the bipolar input vector.[file:2][web:54] (Source: Internet)
(b) Net input computation: Compute the net input (before activation) to the
Adaline neuron: n
X
yin = wi xi + b.
i=1
[file:4][web:54] (Source: Internet)
(c) Output (for decision): Often a bipolar quantizer is used to generate out-
put y = sgn(yin ), but for learning, the continuous net value yin is used di-
rectly.[file:4]
(d) Error computation: Compute instantaneous error
e = t − yin ,
where t is desired bipolar target output.[file:4][web:54] (Source: Internet)
(e) Weight and bias update (delta rule): Update weights and bias using
(new) (old)
wi = wi + η e xi , b(new) = b(old) + η e.
[file:4][web:54] (Source: Internet)
3. Stopping condition: Repeat over all patterns (epochs) until the maximum weight
change is below a tolerance or the total mean squared error over all patterns falls
below a specified threshold.[file:2][web:54] (Source: Internet)
This procedure is equivalent to stochastic gradient descent on the mean squared error
(MSE) function, and is also known as the Widrow–Hoff or LMS rule; for a single-output
Adaline, the error for pattern p is
Ep = (tp − op )2 ,
and the weight update is
∆wi = η(tp − op )xp,i .
[file:4]

20
Upper bound on learning rate using largest eigenvalue of the correlation
matrix: For stable convergence of the LMS/Adaline algorithm, the learning rate η
must satisfy an upper bound related to the eigenvalues of the input correlation matrix
R.[web:47][web:54] (Source: Internet)

• Let x(n) denote the input vector at time n, and let the input correlation matrix be
defined as
R = E[x(n)xT (n)].

• Denote by λmax the largest eigenvalue of R.

• For the LMS/Adaline algorithm using steepest descent on a quadratic error surface,
a sufficient condition for convergence of weight updates is:
2
0<η< .
λmax

Intuitively, the learning rate must be small enough relative to the curvature of the
error surface; the curvature is characterized by the eigenvalues of the Hessian, which, for
quadratic error functions in linear adaptive systems, is proportional to the correlation
matrix R.[web:47] (Source: Internet) If η exceeds 2/λmax , the updates will overshoot and
the algorithm may diverge; if η is much smaller than this bound, convergence will be
stable but slower.[web:47][web:54] (Source: Internet)
Reference: [Link], Unit I, Section 1.8.3 ADALINE Network and
delta rule (pages 1-30 to 1-31); ANN Module 1 Lecture [Link], ADALINE model and
training algorithm; StudyGlance, “Adaptive Linear Neuron” (for explicit stepwise algo-
rithm and convergence condition). (Source: Internet)[file:4][file:2][web:54][web:47]

Q3
(a) What is Error Correction and how to minimize these errors? [5]
Same consistent explanation as Q4(c), February 2023 Paper.
Error-correction learning is a supervised learning paradigm in which unknown ANN
parameters (weights and biases) are adapted so as to minimize the difference between de-
sired outputs and actual network outputs; the error signal drives a control mechanism that
updates parameters to reduce an error function, usually mean squared error.[file:4][file:3]
For neuron k, with desired output dk (n) and actual output yk (n) at time n, the instan-
taneous error is
ek (n) = dk (n) − yk (n),
and a commonly used error function over all output neurons is
1X
E= E[e2k (n)],
2 k

where E[·] denotes expectation.[file:3]

21
Minimizing errors (principles and practical methods):

• Gradient descent on error surface: The most common method is to apply

gradient descent, adjusting weights in the negative direction of the gradient of the
error function with respect to each weight:
∂E
wij (n + 1) = wij (n) − η ,
∂wij

where η is the learning rate; this gradually reduces mean squared error.[file:3]

• Use of local error signals: In multilayer networks, local error terms are computed
(e.g., in backpropagation) for each neuron, and weights are updated proportionally
to the product of local error and input to that weight (delta rule / Widrow–Hoff
rule).[file:4][file:3]

• Choice of learning rate and momentum: A small learning rate ensures stability
but may slow convergence, whereas a large learning rate may cause oscillations;
adding momentum combines current gradient with previous update to smooth and
accelerate convergence while helping avoid shallow local minima.[file:3] (Source:
Internet)[web:13]

• Appropriate network architecture and regularization: Selecting a suitable

number of hidden units and layers helps avoid underfitting (high bias error) and
overfitting (high variance error); regularization techniques (e.g., weight decay, early
stopping) further control error on unseen data. (Source: Internet)[web:51]

• Quality of training data: Reducing noise in training data, ensuring represen-

tative coverage of the input space, and proper normalization/scaling of inputs can
significantly reduce training and generalization errors.[file:3]

Reference: Module-II [Link], Unit II, Section 2.4 Error Correction (pages 2-12 to
2-13); [Link], Unit II, Error Correction and Gradient Descent Rules
(syllabus and related text).[file:3][file:4]

(b) Explain the architecture of Multilayered neural network. [5]

Same as Q3(b), February 2023 Paper, for consistency of the “multilayered network ar-
chitecture” topic.
A multilayer neural network, often called a multilayer perceptron (MLP), consists
of multiple layers of adaptive units: an input layer, one or more hidden layers, and an
output layer; it forms a feed-forward architecture where signals propagate from input to
output without cycles.[file:4][file:3] Every neuron in one layer has directed connections to
neurons in the next layer and is associated with a set of weights and a bias; hidden and
output units apply nonlinear activation functions (commonly sigmoid or ReLU).[file:4]

Layers and computations in a multilayer network:

• Input layer: Contains source nodes which receive the external input vector;
this layer does not perform computation, only distributes inputs to the hidden
layer.[file:4][file:3]

22
 • Hidden layer(s): Each hidden neuron computes
X
uj = wij xi + bj , yj = f (uj ),
i

where f is typically a sigmoid (logistic or hyperbolic tangent) function; these neu-

rons learn internal representations and can enable the network to solve non-linearly
separable problems such as XOR.[file:4][file:3]

• Output layer: Each output neuron similarly computes a weighted sum of hidden-
layer outputs plus bias and applies an activation function to generate the final
network outputs; for classification, softmax or sigmoids may be used, and for re-
gression, linear activations are common.[file:4]

In a typical three-layer MLP, the first layer acts as feature detectors hardwired to
input signals, the hidden layer combines these features into more abstract patterns, and
the output layer performs classification or function approximation.[file:4] Weights are
adjusted using supervised learning algorithms such as backpropagation, which computes
gradients of an error function with respect to all weights and updates them via gradient
descent to minimize overall error.[file:4][file:3]
Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-28 (Fig. 1.8.2
Multilayer Perceptron Model)
[file:4]
Reference: [Link], Unit I, Sections 1.5.2 Multi-Layer Feed Forward
Network and 1.8.2 Multilayer Perceptron (pages 1-12 to 1-13 and 1-28 to 1-29); Module-
II [Link], Unit II, Multilayer network architecture overview.[file:4][file:3]

(c) Define learning and memory. Explain learning algorithms in details. [5]
Learning in neural networks can be considered as a process of adapting weights and bi-
ases so that the network performs a desired task (such as classification or prediction)
more accurately; memory refers to the storage of information in the strength of synaptic
connections and network structure.[file:3] One primary function of the brain is associative
memory, where learning is viewed as forming associations between related patterns; sim-
ilarly, in ANNs, learning rules modify connectivity matrices so that the network recalls
appropriate outputs when inputs are presented.[file:3]

Definitions:

• Learning: Learning is the process by which the free parameters (weights and
biases) of a neural network are adapted through a process of stimulation from the
environment, such that the network performs a specific task; a set of rules used to
modify these parameters is called a learning rule or learning algorithm.[file:4][file:3]

• Memory: Memory in ANNs is represented by distributed patterns of connection

weights; writing into memory corresponds to changing synaptic weights, whereas
reading (recall) corresponds to transforming input signals into stored output pat-
terns; associative memories (auto- and hetero-associative) exemplify this concept.[file:3]

23
Major categories of learning algorithms: According to ANN-Technical-InSem and
Module-II notes, learning rules fall into three broad categories:[file:4][file:3]

1. Supervised learning:

• The network is trained using input–output pairs where the desired output
(target) is provided by an external teacher (e.g., labeled data); the learning
algorithm adjusts weights to minimize error between actual and desired out-
puts.[file:3]
• Examples include Perceptron learning algorithm, delta rule (Widrow–Hoff/LMS),
and backpropagation; tasks such as classification and regression are typically
formulated as supervised learning.[file:4][file:3]

2. Unsupervised learning:

• Only input patterns are provided; no explicit target outputs are given; the
network must discover underlying structure, clusters or features in the data
on its own.[file:4][file:3]
• Competitive learning and Hebbian learning rules are classic examples; net-
works such as self-organizing maps (SOM) and clustering models operate under
unsupervised learning.[file:3]

3. Reinforcement learning:

• The network interacts with an environment and receives evaluative feedback

(rewards or penalties) rather than explicit target outputs; the goal is to learn
policies that maximize long-term reward.[file:4][file:2]
• Although not detailed as an algorithm in the given unit, reinforcement learning
bridges supervised and unsupervised paradigms and is widely used in control
and decision-making tasks. (Source: Internet)[web:51]

Examples of specific learning rules:

• Hebbian learning: An unsupervised rule stating that if neuron i repeatedly helps

excite neuron j, then the synaptic weight wij increases; weight update can be
written as
∆wij (p) = α yj (p) xi (p),
where α is learning rate, xi (p) input and yj (p) output at iteration p.[file:3]

• Perceptron learning rule: A supervised rule that modifies weights only when
the classification is incorrect, using ∆wi = η(d − y)xi ; it is guaranteed to converge
for linearly separable data.[file:4][file:2]

• Delta rule (LMS/Widrow–Hoff ): A supervised gradient-descent rule that min-

imizes mean squared error by updating weights proportionally to error times input,
∆wi = η(t − o)xi , where t is target, o is Adaline output.[file:4]

• Competitive learning: Units within a layer compete to respond to input; typ-

ically, the neuron with largest net input “wins” and updates its weights to move
closer to the current input pattern, thereby forming prototypes or clusters.[file:3]

24
Reference: [Link], Unit I, Section 1.9 Basic Learning Laws (pages 1-
31 to 1-33); Module-II [Link], Unit II, Sections 2.1 Learning and Memory, 2.2 Hebbian
Learning, 2.3 Competitive Learning, 2.6 Supervised Learning.[file:4][file:3]

Q4
(a) What is the difference between Forward propagation and Backward Prop-
agation in Neural Networks? [5]
Forward propagation and backward propagation are the two main phases of training in
multilayer neural networks; the former computes outputs from inputs, while the latter
propagates error gradients backward to update weights.[file:3]

Aspect Forward Propagation Backward Propagation

Direction of infor- Signals propagate from Error signals propagate
mation flow input layer through hidden from output layer back
layers to output layer; through hidden layers to-
no feedback during this ward input layer; used only
phase.[file:3] during training.[file:3]
Purpose To compute the network To compute gradients of er-
outputs for a given input us- ror with respect to weights
ing current weights and ac- and biases and adjust them
tivation functions.[file:3] to minimize error.[file:3]
Computation Each neuron computes net Uses chain rule of calcu-
input as weighted sum plus lus to compute local error
bias and applies activation terms; weight gradients are
function; outputs of one obtained by multiplying er-
layer become inputs to the ror terms with correspond-
next.[file:4] ing inputs and activation
derivatives.[file:3]
Use of target out- Does not use target out- Requires target outputs
puts puts; purely feed-forward to compute error (e.g.,
based on current parame- difference between desired
ters.[file:3] and actual outputs) and
propagate this error back-
ward.[file:3]
When applied Used during both training Used only during training
and testing/inference to ob- epochs to update network
tain predictions.[file:3] parameters; not used during
pure inference.[file:3]
Outcome Produces output vector and Produces weight and bias
intermediate activations; updates; determines how
determines current perfor- the network learns and im-
mance of the network.[file:3] proves over time.[file:3]

Reference: Module-II [Link], Unit II, Section 2.7 Backpropagation – Forward pass
and backward pass (pages 2-20 to 2-22).[file:3]

25
(b) Explain the different types of Gradient Descent in detail. [5]
Gradient descent is an optimization method used to minimize an objective function (e.g.,
loss function) by iteratively moving in the direction of negative gradient; in neural net-
works it is used to update weights to reduce training error.[file:3] There are three main
variants used in practice: batch (standard) gradient descent, stochastic gradient descent
(SGD), and mini-batch gradient descent.[web:13][web:17] (Source: Internet)

1. Batch (Standard) Gradient Descent

• Computes the gradient of the loss function with respect to parameters using the
entire training dataset at each update step.[web:49] (Source: Internet)

• Weight update rule:

θ ← θ − η∇θ J(θ),
where J(θ) is the cost accumulated over all training examples, θ are parameters
and η is learning rate.[web:13] (Source: Internet)

• Pros: Provides a stable and accurate estimate of the gradient and follows a smooth
path towards minima; suitable when dataset is small and fits in memory.[web:13]
(Source: Internet)

• Cons: Each update is computationally expensive for large datasets and can be
slow to converge since gradients are averaged over all examples.[web:49] (Source:
Internet)

2. Stochastic Gradient Descent (SGD)

• Updates parameters using the gradient computed from a single randomly chosen
training example at each iteration; thus, each update is fast but noisy.[web:13][web:17]
(Source: Internet)

• Update rule:
θ ← θ − η∇θ J(θ; x(i) , y (i) ),
where (x(i) , y (i) ) is a single sample.[web:13] (Source: Internet)

• Pros: Converges faster in practice on large datasets, particularly at the beginning of

training; can escape shallow local minima due to noise in updates.[web:17] (Source:
Internet)

• Cons: Path of convergence is noisy and may oscillate around minima instead of
converging smoothly; requires careful tuning of learning rate schedules.[web:49]
(Source: Internet)

3. Mini-Batch Gradient Descent

• Uses a small subset (mini-batch) of training examples (e.g., 32, 64, 128 samples) to
compute gradient at each update; it is a compromise between batch and stochastic
methods.[web:17][web:19] (Source: Internet)

26
 • Update rule:
m
1 X
θ ←θ−η ∇θ J(θ; x(j) , y (j) ),
m j=1
where m is batch size.[web:13] (Source: Internet)
• Pros: More efficient use of vectorized hardware (GPUs), smoother convergence than
pure SGD, and better scalability to large datasets; most deep learning frameworks
implement mini-batch training by default.[web:17][web:19] (Source: Internet)
• Cons: Requires selection of appropriate batch size; too small behaves like noisy
SGD, too large approximates batch gradient descent with high memory cost.[web:19]
(Source: Internet)
Reference: Module-II [Link], Unit II, Section 2.5 Gradient Descent Rules (concep-
tual description); TutorialsPoint and Baeldung articles on differences between GD, SGD
and Mini-Batch GD. (Source: Internet)[file:3][web:13][web:49][web:17]

(c) Write down Perceptron Learning Algorithm for OR function along with
calculation of each input vector. [5]
Same OR-function perceptron example as used previously in Q3(a), February 2023 Paper,
to maintain identical explanation for the perceptron OR example.
The perceptron learning algorithm for implementing the OR function proceeds as
follows.[file:4][file:2]

Truth table for OR (in bipolar form): Original OR truth table in {0, 1}:[file:2]
x1 x 2 d
0 0 0
0 1 1
1 0 1
1 1 1
Convert to bipolar form using x′i = 2xi − 1 ∈ {−1, 1} and d′ = 2d − 1:[file:2]
x′1 x′2 d′
−1 −1 −1
−1 1 1
1 −1 1
1 1 1

Algorithm and weight calculations: Initialize weights and bias: w1 = 0, w2 = 0, b =

0; choose learning rate η = 1.[file:2]
• Pattern 1: x′ = [−1, −1], d′ = −1.
u = w1 x′1 + w2 x′2 + b = 0, y = sgn(u) = 1 (misclassified).
Update:
∆w1 = 1(−1 − 1)(−1) = 2, ∆w2 = 2, ∆b = 1(−1 − 1) = −2,
so new weights: w1 = 2, w2 = 2, b = −2.

27
 • Pattern 2: x′ = [−1, 1], d′ = 1.
u = 2(−1) + 2(1) − 2 = −2, y = −1 (misclassified).
Update:
∆w1 = 1(1 − (−1))(−1) = −2, ∆w2 = 1(1 − (−1))(1) = 2, ∆b = 1(1 − (−1)) = 2,
giving w1 = 0, w2 = 4, b = 0.
• Pattern 3: x′ = [1, −1], d′ = 1.
u = 0(1) + 4(−1) + 0 = −4, y = −1 (misclassified).
Update:
∆w1 = 1(1 − (−1))(1) = 2, ∆w2 = 1(1 − (−1))(−1) = −2, ∆b = 1(1 − (−1)) = 2,
so w1 = 2, w2 = 2, b = 2.
• Pattern 4: x′ = [1, 1], d′ = 1.
u = 2(1) + 2(1) + 2 = 6, y = 1 (correct); no update.

When the four patterns are presented again, all will be classified correctly with weights
w1 = 2, w2 = 2, b = 2; thus the perceptron implements the OR function in bipolar
form.[file:2]
Reference: [Link], Unit I, Section 1.8 Perceptron Learning Algo-
rithm (pages 1-26 to 1-27); ANN Module 1 Lecture [Link], perceptron OR-example
calculations.[file:4][file:2]

March 2025 Paper (Artificial Neural Network – In-

sem)
Q1
(a) What is neural network topology? Explain any three topologies in detail.
[6]
Network topology in the context of neural networks refers to the arrangement of neu-
rons into layers and the pattern of connections between these neurons; it defines how
information flows through the network, how many layers exist, how many neurons are in
each layer, and which neurons are connected.[file:4][file:1] Different topologies are suited
to different tasks and determine the representational and computational capabilities of
the network.[file:4][file:3]

1. Single-Layer Feed-Forward Network

• Structure: Consists of an input layer and a single output layer; inputs are directly
connected to output neurons through weighted connections, with no hidden layers
and no feedback loops.[file:4][file:2]
• Working: Each output neuron computes a weighted sum of all inputs plus bias
and passes it through an activation function (e.g., step or sigmoid); such networks
can only represent linearly separable mappings and are used for simple classification
and regression tasks (e.g., perceptron, Adaline).[file:4]

28
2. Multilayer Feed-Forward (Multilayer Perceptron) Network

• Structure: Contains three or more layers: input layer, one or more hidden layers,
and an output layer; neurons in successive layers are usually fully connected, and
there are no cycles or feedback connections.[file:4][file:3]

• Working: Hidden-layer neurons compute nonlinear transformations of inputs,

enabling the network to approximate complex, non-linearly separable functions;
weights are typically trained using backpropagation, and activation functions such
as sigmoid, tanh or ReLU are used in hidden and output layers.[file:4][file:3]

• Applications: Pattern recognition, function approximation, time-independent clas-

sification and regression, where rich feature extraction is needed.[file:4]

3. Recurrent Neural Network (RNN) Topology

• Structure: RNNs have at least one feedback (recurrent) connection, forming di-
rected cycles; neurons can receive inputs not only from previous layers but also from
their own previous outputs or from peers in the same or next layer.[file:4][file:3]

• Working: At each time step, an RNN processes the current input and its inter-
nal state (memory) from the previous time step; this allows modeling sequences
and temporal dependencies, since outputs depend on both current input and past
history.[file:4][file:3]

• Applications: Natural language processing, speech recognition, time-series pre-

diction, and other problems involving sequential data.[file:4][file:3]

Other topologies, such as radial basis function networks, convolutional neural net-
works, Hopfield networks and self-organizing maps, also exist, but the above three (single-
layer feed-forward, multilayer feed-forward, and recurrent) cover the primary architectural
families discussed in the syllabus.[file:4][file:1]
Reference: [Link], Unit I, Section 1.5 Neural Net Architecture and
Topology (pages 1-9 to 1-13); ANN Module 1 Lecture [Link], “BRIEFLY EXPLAIN
THE BASIC BUILDING BLOCKS OF ARTIFICIAL NEURAL NETWORKS” (network
topology section).[file:4][file:2]

(b) Explain briefly McCulloch Pitt’s (MP) artificial neuron model. Give its
limitations. [5]
Same core explanation for M–P model as in February 2023 Q1(c), with explicit emphasis
on limitations.
The McCulloch–Pitts (M–P) artificial neuron is a logical threshold model of a neu-
ron; it has N binary inputs and one binary output and computes a weighted sum of
its inputs which is compared against a fixed threshold to decide whether the neuron
fires.[file:4][file:2]

29
Model description: Let inputs be x1 , x2 , . . . , xm ∈ {0, 1}, weights w1 , w2 , . . . , wm , and
threshold T .[file:4][file:2] The effective input (net input) is
m
X
u= wj xj − T,
j=1

and the output is given by the step activation function

(
1, u ≥ 0,
a = φ(u) =
0, u < 0.

[file:4] Inputs have either excitatory or inhibitory nature; any active inhibitory input
forces the neuron output to 0 regardless of the excitatory sum.[file:4][file:2] The neuron
is updated synchronously at discrete time steps, and because the activation function is
binary, the M–P neuron is called a discrete neuron.[file:4][file:2]

Limitations:

• No learning capability: Weights and threshold are chosen analytically; there is

no built-in learning rule to adapt them based on data, so the model cannot learn
from examples.[file:4]

• Binary output only: The model outputs only 0 or 1 and cannot represent graded
responses or approximate continuous functions.[file:4]

• Difficult network optimization: Designing minimal networks of M–P neurons

to implement complex logical functions is difficult, and the model does not scale
well to large, practical problems.[file:4][file:2]

Reference: [Link], Unit I, Section 1.7 McCulloch and Pitts Model

(pages 1-23 to 1-25); ANN Module 1 Lecture [Link], MCCULLOGH–PITTS MODEL
and its discussion.[file:4][file:2]

(c) Distinguish between Biological Neural Network and Artificial Neural Net-
works. [4]
Same BNN vs ANN table as in February 2023 Q2(c) to ensure consistency.

30
 Aspect Biological Neural Net- Artificial Neural Net-
work (BNN) work (ANN)
Basic element Consists of biological neu- Consists of artificial neu-
rons (cells) with soma, den- rons (processing units) im-
drites, axon and synapses plemented as mathemati-
formed by biochemical cal functions or hardware
structures.[file:4][file:2] units with inputs, weights,
bias and activation func-
tion.[file:4][file:1]
Size and connec- Human brain has about 86 Typical ANNs have from
tivity billion neurons with more a few dozen to millions
than 1014 synapses; connec- of artificial neurons and
tivity is highly complex and weighted connections, usu-
irregular.[file:2] ally organized in regular
layers and simple topolo-
gies.[file:4][file:1]
Signal type and Signals are electrochemical Signals are numeric values
transport (action potentials and neu- (real numbers) propagated
rotransmitters), propagated synchronously according to
asynchronously with vari- algorithmic time steps; out-
able speeds; neurons fire in puts are continuous or dis-
spikes.[file:4][file:2] crete values.[file:4][file:2]
Processing speed Single biological neurons Individual arithmetic oper-
are relatively slow (millisec- ations are extremely fast
onds per spike), but mas- (nanoseconds), but paral-
sive parallelism yields high lelism is limited by hard-
overall processing capabil- ware architecture; ANNs
ity.[file:4][file:2] can still exploit SIMD/GPU
parallelism.[file:4][file:1]
Topology and Network topology is com- Topology is usually prede-
structure plex, three-dimensional, dy- fined (e.g., feed-forward, re-
namic and self-organizing; current) with fixed number
neurons can grow new of layers and connections;
connections or prune old only weights (and some-
ones.[file:2] times biases) are adjusted
during learning.[file:4][file:1]
Learning and Learning involves long-term Learning is defined by
adaptation synaptic plasticity, neu- explicit algorithms (e.g.,
rogenesis and biochemical Hebbian, perceptron rule,
changes; rules are not backpropagation, delta
yet fully understood and rule) that adjust weights
operate continuously.[file:2] based on error or cor-
relation; training and
inference phases are dis-
tinct.[file:4][file:2]

Reference: [Link], Unit I, comparison between Biological NN and

Artificial NN (page 1-5); ANN Module 1 Lecture [Link], section “STATE THE MA-

31
JOR DIFFERENCES BETWEEN BIOLOGICAL AND ARTIFICIAL NEURAL NET-
WORKS”.[file:4][file:2]

Q2
(a) Discuss briefly the structure and function of a biological neuron. [7]
Same as the structure and working explanation given earlier for biological neuron, with
emphasis on both structure and function.
A biological neuron is an electrically excitable cell that serves as the basic information-
processing unit of the nervous system; its function is to receive, integrate and transmit
information to other neurons or effector cells.[file:4][file:2]

Structure:

• Soma (cell body): Contains nucleus and cytoplasmic organelles; performs metabolic
functions and integrates incoming signals.[file:4][file:2]

• Dendrites: Branch-like structures that receive inputs (postsynaptic potentials)

from other neurons via synapses; collectively form the receptive surface of the neu-
ron.[file:4][file:2]

• Axon: Single long fiber originating from axon hillock; carries action potentials
away from soma toward other neurons; may branch to form collaterals.[file:4][file:2]

• Axon hillock and initial segment: Specialized region where action potentials
are typically initiated when membrane potential exceeds threshold.[file:4][file:2]

• Axon terminals and synapses: Terminals form synapses with dendrites or soma
of postsynaptic neurons; neurotransmitter release at synapses is key for communi-
cation.[file:4][file:2]

Function:

• Signal reception: Dendrites and soma receive excitatory and inhibitory post-
synaptic potentials from thousands of synapses; these graded potentials reflect
weighted inputs from presynaptic neurons.[file:4][file:2]

• Integration and thresholding: Soma integrates all incoming signals; if the re-
sulting membrane potential at the axon hillock exceeds a threshold, an all-or-none
action potential is generated.[file:4][file:2]

• Conduction: The action potential propagates along the axon, often aided by
myelination, without decrement, ensuring that signals reach terminal branches.[file:4][file:2]

• Transmission: Arrival of the action potential at axon terminals triggers neuro-

transmitter release into synaptic cleft, converting the electrical signal into chemical
form; binding of neurotransmitter to postsynaptic receptors produces new electrical
responses in target cells.[file:4][file:2]

32
Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-4 (Fig. 1.3.1
Schematic of Biological Neuron)
[file:4]
Reference: [Link], Unit I, Section 1.3 Structure and Working of Bi-
ological Neural Network (pages 1-4 to 1-5); ANN Module 1 Lecture [Link], Biological
neuron structure and functions.[file:4][file:2]

(b) Obtain the output of the neuron Y for the network shown in figure using
activation function as Binary Sigmoidal and Bipolar Sigmoidal. [8]
The general procedure to obtain the output of a neuron for a given network using binary
sigmoidal and bipolar sigmoidal activation functions is as follows:[file:4][file:2]

1. Identify inputs and weights: From the given figure, determine the input values
xi to neuron Y and corresponding synaptic weights wi , as well as the bias b.

2. Compute net input: Calculate the net input (also called induced local field) to
neuron Y : X
v= wi xi + b.
i

3. Binary sigmoidal activation (unipolar sigmoid): Apply the binary sigmoidal

function
1
fbinary (v) = ,
1 + e−v
to obtain the output ybinary = fbinary (v), which lies in the range (0, 1).[file:2]

4. Bipolar sigmoidal activation (hyperbolic tangent): Apply the bipolar sig-

moidal function
ev − e−v
fbipolar (v) = tanh(v) = v ,
e + e−v
to obtain the output ybipolar = fbipolar (v), which lies in the range (−1, 1).[file:4][file:2]

Because the specific numeric values of inputs, weights and bias in the exam figure are
not provided in the given resources, the exact numerical outputs cannot be computed
here; however, by plugging those values into the formulas above, you can obtain the
binary sigmoidal and bipolar sigmoidal outputs for neuron Y step by step.[file:4][file:2]
Diagram placeholder:
sert Diagram Here – Source: March 2025 Exam Paper, Q2(b) Figure (Neuron Y
Network Diagram)

Reference: [Link], Unit I, Sections 1.6.2 Sigmoid and 1.6.3 Tanh

and ReLU (pages 1-21 to 1-22); ANN Module 1 Lecture [Link], Binary and bipolar
sigmoid activation function definitions.[file:4][file:2]

33
Q3
(a) Write and explain Hebbian learning Algorithm. [6]
Hebbian learning is an unsupervised learning rule based on the principle that the synaptic
weight between two neurons should be strengthened if they are activated simultaneously;
informally, “neurons that fire together, wire together.”[file:3] It provides a local learning
mechanism where weight updates depend only on the activities of pre- and post-synaptic
neurons.[file:3]

Hebb’s law and basic rule: Hebb’s law can be expressed as:[file:3]
• If two neurons on either side of a synaptic connection are activated synchronously,
the weight of that connection is increased.
• If they are activated asynchronously, the weight may be decreased (in generalized
forms).
In mathematical form, for neuron j with input xi (p) from neuron i and output yj (p)
at iteration p, the weight change can be written as
∆wij (p) = α yj (p) xi (p),
where α is the learning rate; this is called the activity product rule.[file:3]

Generalized Hebbian learning algorithm:

1. Initialization: Set initial synaptic weights wij (0) and thresholds to small random
values, e.g., in interval (0, 1).[file:3]
2. Activation (for each input pattern): For iteration p, compute neuron output
n
!
X
yj (p) = f wij (p) xi (p) − θj ,
i=1

where n is number of inputs and θj is threshold of neuron j.[file:3]

3. Weight update: Update weights using generalized activity product rule
wij (p + 1) = wij (p) + ∆wij (p),
where
∆wij (p) = α yj (p) xi (p) − β yj (p) wij (p),
with β possibly acting as a forgetting factor to limit unbounded growth; in simpler
form, without forgetting, ∆wij (p) = α yj (p) xi (p).[file:3]
4. Iteration: Increase iteration index p and repeat activation and weight update steps
for all input patterns, allowing the network to self-organize representations.[file:3]
Hebbian learning is useful in associative memory models (e.g., Hopfield network,
auto-associative memory) where patterns stored correspond to stable states defined by
strengthened synaptic connections; however, naive Hebbian learning without normaliza-
tion or forgetting can lead to unbounded growth of weights.[file:3]
Reference: Module-II [Link], Unit II, Section 2.2 Hebbian Learning (pages 2-6 to 2-
8); [Link], Unit II, Basic Learning Laws – Hebbian rule.[file:3][file:4]

34
(b) What is error correction learning? Explain in detail with diagram. [5]
Same conceptual explanation as earlier error-correction answers, with emphasis on block
diagram.
Error-correction learning is a supervised learning process in which the network ad-
justs its parameters (weights and biases) to minimize an error function derived from the
difference between desired outputs and obtained outputs; the learning rule modifies pa-
rameters proportionally to this error so that, over time, the network approximates the
target mapping in a statistical sense.[file:3]

Description: Consider neuron k in the output layer driven by input vector x(n) (pro-
duced by one or more hidden layers). Let dk (n) be the desired output and yk (n) be the
obtained output at time n.[file:3] The error signal is

ek (n) = dk (n) − yk (n).

This error is fed back to a training algorithm (controller) which computes adjustments
to weights wik to minimize a chosen error measure, usually mean squared error:[file:3]
1X
E= E[e2k (n)].
2 k

Weight updates often have the generic form

∆wik (n) = η ek (n) xi (n),

which is the basis of the delta rule; here, η is the learning rate, xi (n) is input to weight
wik , and ek (n) is error at neuron k.[file:3][file:4]

Minimization and block diagram: Error-correction learning uses gradient descent

on the error surface to iteratively refine weights; the process can be summarized in the
following block structure:[file:3]

• Input x: Presented to the adaptive system (neural network).

• Adaptive system (weights W ): Produces output y based on current weights.

• Desired output d: Provided by a teacher for supervised learning.

• Error computation: Error signal e = d − y is computed.

• Training algorithm: Uses error e to change parameters (weights, biases) accord-

ing to a learning rule (e.g., delta rule, backpropagation).

Diagram placeholder:
sert Diagram Here – Source: Module-II [Link], Page No: 2-12 (Fig. 2.4.1 Error
Correction Learning Block Diagram)
[file:3]
Reference: Module-II [Link], Unit II, Section 2.4 Error Correction (pages 2-12 to
2-13); [Link], Unit II, Error Correction concept.[file:3][file:4]

35
(c) Differentiate between Feed Forward and Feedback neural network. [4]
Same feed-forward vs feedback table as given earlier in February 2023 Q4(b) to maintain
identical explanation.

Aspect Feed-forward Neural Feedback / Recurrent

Network Neural Network
Connection direc- Connections are unidirec- Connections may form
tion tional; signals flow strictly loops; outputs of neurons
from input layer to hid- can be fed back as inputs to
den layer(s) to output layer, the same or previous layers,
with no cycles.[file:3] permitting cycles.[file:3]
Information flow Only forward flow of in- Information can travel in
formation; no backward both directions; network
flow during normal opera- state evolves over time
tion (backprop is only dur- due to feedback connec-
ing training).[file:3] tions.[file:3]
Memory of past No internal dynamic mem- Has internal state and mem-
inputs ory of input sequence; out- ory; outputs depend on
put depends only on current current input and previ-
input (apart from trained ous internal states, suitable
weights).[file:3] for temporal and sequential
data.[file:3]
Dynamics Static input–output map- Dynamic system whose
ping; after input is ap- state may change until
plied, output is computed in equilibrium is reached;
a finite number of forward often used as content-
computations.[file:3] addressable memory or
temporal pattern recog-
nizer.[file:3]
Typical applica- Pattern classification, re- Time-series prediction,
tions gression, function approxi- language modeling, speech
mation where data are not recognition, associative
time-dependent (e.g., image memory, and problems
recognition, static pattern where sequence or context
recognition).[file:3] is important.[file:4][file:3]
Training complex- Training is generally sim- Training is more complex
ity pler; standard backpropa- due to recurrent connec-
gation is directly applicable tions; requires specialized
and stable for many prob- algorithms (e.g., backprop-
lems.[file:3] agation through time) and
careful handling of stabil-
ity.[file:3]

Reference: Module-II [Link], Unit II, Section 2.8 Feed Forward Neural Networks and
2.8.1 Feedback Neural Networks (pages 2-24 to 2-26); [Link], Unit I,
Neural net architecture overview.[file:3][file:4]

36
Q4
(a) Draw the architecture of multilayer feed forward networks. Explain input
layer, hidden layer & output layer computations in multilayer feed forward
networks. [7]
A multilayer feed-forward network (multilayer perceptron) consists of an ordered sequence
of layers: input layer, one or more hidden layers, and an output layer; connections exist
only from a layer to the next higher layer, and there are no cycles.[file:4][file:3]

Input layer computations: The input layer contains one neuron per input feature;
these neurons simply distribute the input vector to the hidden layer without performing
computation.[file:4] If the input vector is x = [x1 , x2 , . . . , xn ]T , then the outputs of input
layer units are xi themselves.[file:3]

Hidden layer computations: For a hidden neuron j, receiving inputs from preced-
ing layer (which may be input or another hidden layer), the net input and output are
computed as:[file:4][file:3]
X
uj = wij xi + bj , yj = f (uj ),
i

where wij is weight from neuron i to j, bj is bias, and f (·) is a nonlinear activation
function such as logistic sigmoid or hyperbolic tangent.[file:4][file:3] The hidden layer
extracts features or intermediate representations, enabling the network to model non-
linear relationships.

Output layer computations: For output neuron k, receiving signals from last hidden
layer, we compute:[file:4]
X
uk = wjk yj + bk , ok = g(uk ),
j

where g(·) may be a sigmoid (for binary classification), softmax (for multi-class classifica-
tion) or linear function (for regression).[file:4][file:3] The output layer produces the final
network outputs corresponding to the task at hand.
During training, forward propagation computes these layer-wise transformations; back-
propagation then uses the difference between desired and actual outputs to adjust all
weights in the network.[file:3]
Diagram placeholder:
sert Diagram Here – Source: [Link], Page No: 1-28 (Fig. 1.8.2
Multilayer Perceptron Model)
[file:4]
Reference: [Link], Unit I, Sections 1.5.2 Multi-Layer Feed Forward
Network and 1.8.2 Multilayer Perceptron (pages 1-12 to 1-13 and 1-28 to 1-29); Module-
II [Link], Unit II, multilayer network computations.[file:4][file:3]

(b) Explain perceptron learning algorithm and implement OR function using

Perceptron network. [8]
Same perceptron learning algorithm and OR implementation as earlier answers to keep
consistency.

37
Perceptron learning algorithm: Given input vector x, weight vector w, bias b, and
target output d ∈ {−1, 1}, the perceptron learning rule is:[file:4][file:2]

1. Initialize w and b to small random values.

2. For each training pattern:

• Compute net input u =

P
i wi xi + b.
• Compute output y = sgn(u).
• If y ̸= d, update

wi ← wi + η(d − y)xi , b ← b + η(d − y),

otherwise leave weights unchanged.

3. Repeat until all patterns are correctly classified or a maximum number of epochs
is reached.

OR function implementation (with calculations): Using the bipolar truth table

and calculations already derived:

x′1 x′2 d′
−1 −1 −1
−1 1 1
1 −1 1
1 1 1
[file:2]
Initialize w1 = 0, w2 = 0, b = 0, learning rate η = 1.

• Pattern 1: [−1, −1], d′ = −1. u = 0 ⇒ y = 1 (error). Update:

∆w1 = 2, ∆w2 = 2, ∆b = −2 ⇒ w1 = 2, w2 = 2, b = −2.

• Pattern 2: [−1, 1], d′ = 1. u = −2 ⇒ y = −1 (error). Update:

∆w1 = −2, ∆w2 = 2, ∆b = 2 ⇒ w1 = 0, w2 = 4, b = 0.

• Pattern 3: [1, −1], d′ = 1. u = −4 ⇒ y = −1 (error). Update:

∆w1 = 2, ∆w2 = −2, ∆b = 2 ⇒ w1 = 2, w2 = 2, b = 2.

• Pattern 4: [1, 1], d′ = 1. u = 6 ⇒ y = 1 (correct); no update.

Hence, after training, perceptron parameters w1 = 2, w2 = 2, b = 2 implement the OR

function in bipolar form; mapping back to {0, 1} domain gives correct OR outputs for all
four input combinations.[file:2]
Reference: [Link], Unit I, Section 1.8 Perceptron and learning rule
(pages 1-26 to 1-27); ANN Module 1 Lecture [Link], perceptron OR function exam-
ple.[file:4][file:2]

38