Artificial Intelligence and Machine Learning
Module -3
1) What is Perceptron
A Perceptron is the simplest type of artificial neuron and is the basic building
block of early Artificial Neural Networks. It was introduced by
Frank Rosenblatt in 1957.
1.1 How a Perceptron Works
A perceptron takes multiple inputs, assigns weights, adds a bias, and then applies a step function
to produce output.
Mathematical Representation
a) First, compute the weighted sum: z = w1x1 + w2x2 + w3x3 + ………….. + b
b) Then apply a step function:
_____________________________________________________________________________________________________________________
Compiled by : Narayana Maddala, Professor of Practice Page 1 of 6
1.2 Components of a Perceptron
1. Inputs (x₁, x₂, …) → features
2. Weights (w₁, w₂, …) → importance of inputs
3. Bias (b) → threshold adjustment
4. Activation Function → step function
5. Output (y) → binary result (0 or 1)
1.3 Example of Perceptron function
a) Input vector (with 6 elements)
X= [x1,x2,x3,x4,x5,x6] = [1, 0, 1, 1, 0, 1]
b) Weight Vector (6 weights)
w=[w1,w2,w3,w4,w5,w6] = [2, −1, 3, 0, −2, 1]
c) Bias (b) = -2
d) Compute Weighted Sum:
e) Apply Step Function
Output of Perceptron (y) = 1
1.4 Geometric Interpretation
• A perceptron creates a decision boundary
o In 2D → a straight line
o In 3D → a plane
_____________________________________________________________________________________________________________________
Compiled by : Narayana Maddala, Professor of Practice Page 2 of 6
2) Artificial Neural Network (ANN)
2.1 What is an Artificial Neural Network?
An Artificial Neural Network (ANN) is a computational model inspired by how the human
brain processes information. It is one of the core ideas in Machine Learning and Artificial
Intelligence.
An ANN consists of interconnected units called neurons (or nodes) that work together to learn
patterns from data.
Remember that the neurons (or nodes) are actually Mathematical Functions (like perceptrons).
2.2 Basic Structure of ANN
An ANN typically has three types of layers:
a) Input Layer
• Receives input data (features)
• Example: image pixels, temperature values
b) Hidden Layer(s)
• Perform computations and feature extraction
• Can be one or many layers (deep neural networks = more layers)
c) Output Layer
• Produces final result (prediction/classification)
2.3 Components of a Neuron
Each neuron has:
• Weights (w) → importance of each input
• Bias (b) → adjustment factor
• Activation Function → decides output
a) Computes the weighted sum: z = w1x1 + w2x2 + w3x3 + ………….. + b
b) Applies Activation Function
2.4 Activation Functions
Common activation functions:
• Sigmoid → outputs between 0 and 1
• ReLU (Rectified Linear Unit) → most widely used
• Tanh → outputs between -1 and 1
They introduce non-linearity, allowing ANN to learn complex patterns.
Sigmoid Function
The Sigmoid function (also called the logistic function) is a commonly used activation function
in Neural Networks.
_____________________________________________________________________________________________________________________
Compiled by : Narayana Maddala, Professor of Practice Page 3 of 6
Mathematically it is defined as :
Key Properties
• Output range: 0 to 1
• Smooth and continuous
• Differentiable (important for learning)
• S-shaped (curve)
• Converts output into probability-like value
• Acts like a soft version of the step function
Tanh
2.5 Structure and Function of Feedforward Neural Network
Forward Propagation
* Explain it as layer-by-layer propagation of input through matrix multiplication process.
_____________________________________________________________________________________________________________________
Compiled by : Narayana Maddala, Professor of Practice Page 4 of 6
* Input Layer is 0th Layer
* The output layer can be called Lth layer
* L-1 Hidden Layers
* ai is the Pre-activation vector and hi is the Activation Vector.
Mathematical Expression of Softmax Function
2.6 How ANN Learns (Training Process)
1. Forward Propagation
o Input passes through layers → output generated
_____________________________________________________________________________________________________________________
Compiled by : Narayana Maddala, Professor of Practice Page 5 of 6
2. Loss Calculation
o Compare predicted output with actual output
3. Backpropagation
o Errors are propagated backward
4. Weight Update
o Using optimization methods like gradient descent
_____________________________________________________________________________________________________________________
Compiled by : Narayana Maddala, Professor of Practice Page 6 of 6