Module–5
Artificial Neural Network
Notes / Study Material
April 30, 2026
Dept. of CSE (Data Science), GNDECB
1 Introduction to Artificial Neural Networks (ANN)
• Artificial Neural Networks (ANN) are computational models inspired by the struc-
ture and functioning of the human brain.
• They are designed to mimic how biological neurons process information and learn
from experience.
• An ANN consists of a large number of interconnected processing elements called
neurons.
• These neurons work collectively to solve complex problems and perform intelligent
tasks.
• The primary objective of ANN is to enable machines to learn patterns directly from
data.
• Unlike traditional programming, ANN does not require explicit instructions, as it
learns automatically from examples.
• This makes ANN suitable for problems where the relationship between input and
output is complex or not clearly defined.
• A neural network is organized into layers: input layer, hidden layer(s), and output
layer.
• The input layer receives raw data, which is processed through hidden layers.
• Hidden layers perform computations and extract important features from the data.
• The output layer produces the final result in the form of prediction or classification.
• Each neuron receives inputs, multiplies them with weights, adds a bias, and pro-
cesses the result using an activation function.
• The activation function introduces non-linearity, enabling the network to learn com-
plex relationships.
• Learning in ANN occurs by adjusting weights and biases based on the error in
prediction.
• This is achieved using algorithms such as backpropagation along with optimization
techniques like gradient descent.
• Depending on the type of data, ANN can learn through supervised, unsupervised,
or reinforcement learning.
• ANN is widely used in applications such as image recognition, speech processing,
medical diagnosis, and financial prediction.
• However, ANN requires large datasets and high computational resources.
• The internal working of ANN is often difficult to interpret, making it a black-box
model.
Dr. Veerendra Dakulagi 1
Dept. of CSE (Data Science), GNDECB
2 Biological Neurons
Figure 1: Biological Neuron
• A biological neuron is the basic unit of the nervous system.
• It has four main parts:
– Dendrites
– Cell body (Soma)
– Axon
– Synapse
• Dendrites: Receive input signals from other neurons.
• Cell body (Soma): Processes the input information.
• Axon: Sends output signals to other neurons.
• Synapse: Connection point between two neurons.
• A neuron sends signal only if input exceeds a threshold value.
• Signals are transmitted as electrical impulses (spikes).
• One neuron can connect to thousands of other neurons.
• Neurons work together to form a network.
Dr. Veerendra Dakulagi 2
Dept. of CSE (Data Science), GNDECB
• This network processes information and produces a response.
• Example: Brain processing, reflex action, sensory response.
3 Artificial Neurons
Figure 2: Artificial Neuron
• Artificial neuron is a model of biological neuron.
• Also called a node.
• It receives one or more input signals.
• Each input has an associated weight.
• Inputs are combined and processed.
• Output is sent to other neurons.
• Neurons are connected to form a network.
• Example: Used in neural networks for classification and prediction.
3.1 Simple Model of Artificial Neuron
• Developed by McCulloch and Pitts (1943).
• Receives inputs x1 , x2 , . . . , xn .
• Each input has weight w1 , w2 , . . . , wn .
• Calculates weighted sum: X
N et = xi w i
Dr. Veerendra Dakulagi 3
Dept. of CSE (Data Science), GNDECB
Figure 3: McCulloch & Pitts Neuron Mathematical Model
• Uses an activation function.
• Compares net value with threshold.
• If Net ≥ θ → output = 1.
• If Net < θ → output = 0.
• This is called a binary step function.
• Helps in decision making (yes/no type problems).
• Example: Logic gates (AND, OR).
4 Artificial Neural Network (ANN) Structure
• ANN is a model inspired by the human brain.
• It processes information using interconnected neurons.
• Neurons are arranged in layers.
• Main layers in ANN:
– Input layer
– Hidden layer(s)
– Output layer
• Input layer: Receives input data (x1 , x2 , . . . , xn ).
• Hidden layer: Processes data and learns patterns.
• Output layer: Produces final result.
• Each connection has a weight.
• Each neuron also has a bias.
Dr. Veerendra Dakulagi 4
Dept. of CSE (Data Science), GNDECB
Figure 4: Artificial Neural Network Structure
• Neuron computes weighted sum:
X
N et = xi w i + b
• Activation function converts input into output.
• If input exceeds threshold, neuron produces output.
• ANN processes multiple inputs in parallel.
• Used for solving complex problems.
• Example: Image recognition, speech recognition, prediction.
5 Activation Functions
• Activation function decides whether a neuron will fire or not.
• It converts input signal into output signal.
• Output is usually between 0 to 1 or -1 to 1.
• Helps in solving non-linear problems.
• Types: Linear and Non-linear functions.
Dr. Veerendra Dakulagi 5
Dept. of CSE (Data Science), GNDECB
6 Perceptron and Learning Theory
Figure 5: Perceptron Model
• Perceptron is the first neural network model.
• Developed by Frank Rosenblatt (1958).
• It is a linear classifier used for supervised learning.
• It learns by adjusting weights using training data.
• Based on McCulloch-Pitts neuron model.
• Used for simple classification problems.
• Example: AND, OR logic gates.
Perceptron Algorithm
• Initialize weights (w1 , w2 , . . . , wn ) and bias (b) randomly.
• For each input sample:
– Calculate weighted sum: X
N et = xi wi + b
– Apply activation function (step function):
Y = { 1 , if N et ≥ 00, if N et < 0
– Compare output with desired output.
– Calculate error:
Error = Ydesired − Y
Dr. Veerendra Dakulagi 6
Dept. of CSE (Data Science), GNDECB
– Update weights:
wi = wi + α × Error × xi
– Update bias:
b = b + α × Error
• Repeat for all inputs until error becomes zero.
• α = learning rate (small value).
XOR Problem
• XOR is a logical operation.
• Output is 1 if inputs are different.
• Output is 0 if inputs are same.
• Truth table:
– (0,0) → 0
– (0,1) → 1
– (1,0) → 1
– (1,1) → 0
• XOR is not linearly separable.
• Single-layer perceptron cannot solve XOR.
• Requires multi-layer neural network (MLP).
• Led to development of backpropagation.
• Example: Pattern classification problems.
7 Types of Artificial Neural Networks
7.1 1. Feed Forward Neural Network (FFNN)
• Data flows in one direction (input → output)
• No loops or feedback connections
• Simple and easy to understand model
• Used for basic classification and prediction tasks
• Works well with structured data
• Example: Handwritten digit recognition, spam detection
Dr. Veerendra Dakulagi 7
Dept. of CSE (Data Science), GNDECB
Figure 6: Model of a Feed Forward Neural Network
Figure 7: Model of a Fully Connected Neural Network
Dr. Veerendra Dakulagi 8
Dept. of CSE (Data Science), GNDECB
Figure 8: Model of a Multi-Layer Perceptron
2. Fully Connected Network
• Every neuron in one layer connects to all neurons in next layer
• Large number of weights and parameters
• Learns complex relationships in data
• High computational cost
• Commonly used inside deep learning models
• Example: Image classification, speech recognition
3. Multi-Layer Perceptron (MLP)
• Consists of input, hidden, and output layers
• Uses activation functions (ReLU, sigmoid, etc.)
• Can solve non-linear problems
• Trained using backpropagation algorithm
• Widely used supervised learning model
• Example: Stock prediction, medical diagnosis
Dr. Veerendra Dakulagi 9
Dept. of CSE (Data Science), GNDECB
Figure 9: Model of a Feedback Neural Network
4. Feedback Neural Network (Recurrent Neural Network - RNN)
• Has loops; output feeds back as input
• Can remember past information (has memory)
• Suitable for sequential data
• More complex than feedforward networks
• Used in time-dependent problems
• Example: Language translation, speech processing, time-series forecasting
8 Learning in Multi-Layer Perceptron (MLP)
• MLP is a type of feedforward neural network with input, hidden, and output layers.
• Each neuron in one layer is connected to neurons in the next layer.
• Input layer receives data (example: image pixels or numerical values).
• Hidden layers process the data and learn patterns.
• Output layer gives the final result (example: class label like ”cat” or ”dog”).
• MLP uses activation functions (like ReLU, sigmoid) to handle non-linear problems.
• It learns using a method called backpropagation.
Dr. Veerendra Dakulagi 10
Dept. of CSE (Data Science), GNDECB
• In backpropagation, error is calculated by comparing output with the correct an-
swer.
• The network adjusts weights and biases to reduce error step by step.
• Learning happens in two phases:
– Forward pass: input moves through the network to produce output
– Backward pass: error is sent back to update weights
• MLP can solve complex problems like XOR (which single-layer cannot solve).
• Example applications:
– Handwritten digit recognition
– Medical diagnosis prediction
– Stock price prediction
8.1 Algorithm-Learning in MLP (Backpropagation Algorithm)
• Input: (x1 , x2 , . . . , xn )
• Output: Y
• Learning rate: α
• Initialize weights and bias randomly.
Step 1: Forward Propagation
• Input Layer:
Ii = xi , Oi = xi
• Hidden Layer Net Input:
X
Ij = xi wij + θj
• Hidden Layer Output:
1
Oj =
1 + e−Ij
• Output Layer Net Input:
X
Ik = Oj wjk + θk
• Final Output:
1
Ok =
1 + e−Ik
Step 2: Error Calculation
• Calculate error at output:
Error = Odesired − Oactual
Dr. Veerendra Dakulagi 11
Dept. of CSE (Data Science), GNDECB
Step 3: Backward Propagation
• Error at Output Layer:
Errork = Ok (1 − Ok )(Odesired − Ok )
• Error at Hidden Layer:
X
Errorj = Oj (1 − Oj ) Errork wjk
Step 4: Update Weights and Bias
• Update Output Layer Weights:
∆wjk = α × Errork × Oj
wjk = wjk + ∆wjk
• Update Hidden Layer Weights:
∆wij = α × Errorj × Oi
wij = wij + ∆wij
• Update Bias:
∆θ = α × Error
Final Step
• Repeat all steps for all inputs.
• Continue until error becomes minimum.
9 Radial Basis Function Neural Network (RBFNN)
• RBFNN is a type of neural network with 3 layers: input, hidden, and output.
• Input layer receives data (example: sensor values, image pixels).
• Hidden layer uses radial basis (Gaussian) functions.
• Each hidden neuron checks how close input is to its center.
• If input is close to center → output is high; if far → output is low.
• Output layer combines all hidden neuron outputs using weights.
• RBFNN works based on distance instead of weighted sum like MLP.
• Learning is faster compared to many other neural networks.
• It is good for interpolation and function approximation.
Dr. Veerendra Dakulagi 12
Dept. of CSE (Data Science), GNDECB
Figure 10: Architecture of RBF Neural Network
• RBFNN can solve complex and non-linear problems easily.
• Example applications:
– Pattern recognition (face, speech)
– Time-series prediction
– Control systems
– Medical data classification
Working of RBFNN
• Step 1: Input is given to the network.
• Step 2: Hidden layer calculates distance from centers.
• Step 3: Gaussian function converts distance to output.
• Step 4: Output layer calculates weighted sum.
• Step 5: Final result is produced.
9.1 AlgorithmRBF Neural Network Algorithm
• Input: (x1 , x2 , . . . , xn )
• Output: Y
• Initialize weights between hidden and output layer.
Dr. Veerendra Dakulagi 13
Dept. of CSE (Data Science), GNDECB
Step 1: Input Layer
• Input is directly passed:
Ii = xi , Oi = xi
Step 2: Hidden Layer
• Each neuron has center cj and radius rj
• Compute output using Gaussian function:
(x−cj )2
−
r2
Hj (x) = e j
• Measures distance between input and center.
Step 3: Output Layer
• Compute weighted sum:
m
X
Y = wj Hj (x)
j=1
• Final output is linear combination of hidden outputs.
Step 4: Learning
• Adjust centers cj , radius rj , and weights wj .
• Reduce error between actual and desired output.
• Repeat until error is minimum.
10 Self-Organizing Feature Map (SOFM)
• SOFM is an unsupervised neural network (no target/output needed).
• Developed by Teuvo Kohonen.
• It groups similar data into clusters automatically.
• Maps high-dimensional data to low-dimensional (usually 2D map).
• Similar inputs are placed close to each other on the map.
• Used for pattern recognition and data clustering.
• Example: Customer segmentation, image grouping, speech patterns.
Dr. Veerendra Dakulagi 14
Dept. of CSE (Data Science), GNDECB
Figure 11: Network Architecture of Self-Organizing Feature Map
Working of SOFM
• Step 1: Input data is given to the network.
• Step 2: All output neurons compare the input.
• Step 3: Neurons compete (winner takes all).
• Step 4: Closest neuron is selected (winner neuron).
• Step 5: Update weights of winner neuron.
• Step 6: Repeat for all inputs until learning is complete.
Structure of SOFM
• Has two layers: input layer and output layer.
• No hidden layer.
• Each input connects to all output neurons.
• Output layer forms a map (grid structure).
• Uses Euclidean distance to find similarity.
SOFM Algorithm
• Initialize weights randomly.
• For each input:
– Calculate distance between input and weights.
Dr. Veerendra Dakulagi 15
Dept. of CSE (Data Science), GNDECB
– Select neuron with minimum distance (winner).
– Update weights of winner neuron.
• Repeat until weights stabilize.
• Final result: similar inputs form clusters.
11 Applications of Artificial Neural Networks (ANN)
• Used for solving complex real-world problems
• Works with noisy and incomplete data
• Performs pattern recognition like human brain
• Used in face recognition systems
• Used in emotion detection
• Used in self-driving cars
• Used in navigation systems
• Used in stock market prediction
• Used in sales forecasting
• Used in customer behavior analysis
• Used in fraud detection
• Used in risk analysis
• Used in medical diagnosis
• Used in image analysis
• Used in robotics and aerospace
12 Advantages and Disadvantages of ANN
Advantages:
• Can solve complex non-linear problems.
• Learns patterns and relationships in data.
• Works well with large datasets.
• Can handle noisy and incomplete data.
• Gives good prediction accuracy.
Disadvantages:
Dr. Veerendra Dakulagi 16
Dept. of CSE (Data Science), GNDECB
• Requires high computational power.
• Training takes more time.
• Difficult to understand (black-box model).
• Needs large amount of training data.
• Performance may be poor on small datasets.
13 Challenges of Artificial Neural Networks
• Training neural networks is time-consuming.
• Difficult to generalize real-world data.
• Selecting proper weights and parameters is hard.
• Requires large training dataset.
• Complex model design.
• Risk of overfitting (poor performance on new data).
• Needs high computational resources.
Dr. Veerendra Dakulagi 17
Dept. of CSE (Data Science), GNDECB
Module–5: Review Questions
5 Mark Questions
• Explain biological neuron with labeled diagram.
• Describe artificial neuron model with basic equation.
• Explain perceptron model and its components.
• Write short notes on perceptron learning rule.
• Describe Feed Forward Neural Network.
• Explain Fully Connected Neural Network.
• Write short notes on activation functions.
• Explain Radial Basis Function (RBF) concept.
• Describe Self-Organizing Feature Map (SOFM) concept.
• Write applications of Artificial Neural Networks.
• List advantages and disadvantages of ANN.
• Explain challenges in ANN.
• Describe XOR problem in neural networks.
• Explain binary and sigmoid activation functions.
10 Mark Questions
• Describe perceptron algorithm with learning steps.
• Explain different types of artificial neural networks in detail.
• Describe Multi-Layer Perceptron (MLP) with structure and working.
• Explain learning in MLP using backpropagation algorithm.
• Describe Radial Basis Function Neural Network with algorithm.
• Explain SOFM architecture and learning process.
• Discuss activation functions and their importance.
• Compare Feed Forward and Feedback Neural Networks.
• Discuss applications and limitations of ANN.
Dr. Veerendra Dakulagi 18