MODULE 1

INTRODUCTION TO ARTIFICAL INTELLIGENCE AND MACHINE LEARNING

Introduction

In every aspect of life—whether it's conversations, grocery shopping, managing subscriptions, or navigating
traffic—we constantly deal with data. These are all forms of information. To make sense of this information, we
must analyze it. While small datasets can be managed using simple tools like Excel, larger and more complex
datasets require more advanced techniques. When the data is incomplete, vast, or needs to be interpreted in many
ways, traditional tools fall short.

To tackle such challenges, we now rely on data science and artificial intelligence (AI)—fields inspired by how the
human brain processes information. Data science involves manipulating and analyzing data to gain better insights
and extract meaningful information. Artificial intelligence enables machines to perform these operations using
computational power.

Within AI, a subset called machine learning (ML) focuses on teaching computers how to learn from data. In ML,
systems are trained to recognize patterns, make decisions, and even predict future outcomes, all based on historical
or input data. Together, these tools and techniques help us make informed decisions and automate complex
processes across industries.

Module 1

[Introduction to AI and Machine Learning: Basics of Machine Learning - types of Machine Learning systems-
challenges in ML- Supervised learning model example- regression models- Classification model example-
Logistic regression-unsupervised model example- K-means clustering. Artificial Neural Network- Perceptron-
Universal Approximation Theorem (statement only)- Multi-Layer Perceptron- Deep Neural Network-
demonstration of regression and classification problems using MLP.(Text-2)]

Artificial Intelligence (AI) is the field of study that focuses on enabling machines to perform tasks that typically
require human intelligence. These tasks include learning from experience, understanding language, recognizing
patterns, making decisions, and even solving problems. A vital part of AI is Machine Learning (ML), which
refers to the ability of machines to learn from data without being explicitly programmed for each task. ML systems
analyze past data, identify patterns, and use these insights to make predictions or decisions about new data.

There are different types of Machine Learning systems based on the nature of the learning task and the kind of
supervision involved. The three broad categories are supervised learning, unsupervised learning, and
reinforcement learning. In supervised learning, the model is trained on a labeled dataset—that is, input data is
paired with the correct output. In unsupervised learning, the model is given input data without any labeled
responses and must discover underlying patterns on its own. Reinforcement learning involves training a model to
make a sequence of decisions by rewarding it for good actions and penalizing it for poor ones.

Despite its potential, Machine Learning faces several challenges. One key issue is the quality and quantity of
data—insufficient, biased, or noisy data can lead to poor model performance.

Another challenge is overfitting, where a model learns the training data too well and fails to generalize to new
data. Interpretability is also a major concern, especially with complex models like deep neural networks, as it's often
difficult to understand how decisions are being made.

A basic example of supervised learning is a regression model, where the goal is to predict a continuous output
variable. For instance, predicting house prices based on square footage, location, and number of bedrooms can be
achieved using linear regression. In contrast, a classification model is used when the output variable is categorical.
For example, identifying whether an email is spam or not is a classification task, and one common algorithm used
here is logistic regression, which models the probability that a given input belongs to a certain class.

1
An example of unsupervised learning is K-means clustering, which groups data into clusters based on similarity.
In K-means, the algorithm tries to divide the data into K distinct clusters, where each data point belongs to the
cluster with the nearest mean. This is useful in market segmentation, image compression, and social network
analysis, where patterns or groupings in data are not predefined.

One of the most important advances in Machine Learning is the development of Artificial Neural Networks
(ANNs), which are computational models inspired by the structure and function of the human brain. The simplest
form of an ANN is the Perceptron, a model consisting of a single layer of nodes used for binary classification.
While the perceptron can solve simple linear problems, it is limited when it comes to complex or non-linear tasks.

The Universal Approximation Theorem states that a neural network with at least one hidden layer and sufficient
neurons can approximate any continuous function on a closed interval, given appropriate weights and activation
functions. This theorem provides the theoretical foundation for using neural networks to solve a wide range of
problems.
To overcome the limitations of the simple perceptron, more complex architectures like the Multi-Layer
Perceptron (MLP) have been developed. An MLP consists of an input layer, one or more hidden layers, and an
output layer. Each layer is made up of multiple neurons, and non-linear activation functions are used to enable the
network to model complex relationships in data.

When MLPs are expanded with more layers and larger datasets, they form Deep Neural Networks (DNNs), which
are capable of handling tasks such as image recognition, speech processing, and natural language understanding.
These deep architectures have led to major breakthroughs in AI over the past decade.
The practical application of an MLP can be demonstrated through both regression and classification problems.
For regression, an MLP can learn to predict values such as stock prices or temperatures by analyzing numerical
input features. For classification, it can be trained to recognize handwritten digits or detect spam emails, learning
to associate patterns in the input data with specific output classes.

Basics of Machine Learning

Machine Learning is a subset of Artificial Intelligence (AI) that enables systems to learn from data, identify
patterns, and make decisions with minimal human intervention. Rather than being explicitly programmed to
perform a task, ML algorithms use statistical methods to learn from data and improve performance over time.

Types of Machine Learning Systems

1. Supervised Learning
In supervised learning, the algorithm is trained on a labeled dataset, meaning the input data is paired with the
correct output.
Goal: Learn a function that maps inputs to desired outputs. Applications: Spam
detection, fraud detection, medical diagnosis.
◆ Examples:
Regression: Predicting continuous values (e.g., house prices).
Classification: Predicting discrete labels (e.g., email spam or not).
● Example 1: Fruit Classifier
` # Example 2: House Price Prediction
⬛
(Regression)

2. Unsupervised Learning
Here, the algorithm works with unlabeled data and tries to find hidden patterns or structures.
Goal: Discover the underlying structure in data.
Applications: Customer segmentation, anomaly detection, topic modeling.
2
◆ Examples:

• Clustering: Grouping similar data points (e.g., K-Means).

• Dimensionality Reduction: Reducing the number of variables (e.g., PCA).

Example 1: Customer Segmentation

Imagine a company has data about customers:
Age Income Shopping Amount
22 ₹30K ₹5000
45 ₹90K ₹25,000
30 ₹50K ₹7000
50 ₹1L ₹30,000
The company doesn't know anything else — no labels.
They use unsupervised learning to group customers into categories like:
Low-income low-spending
Middle-income moderate-spending
High-income high-spending
This helps them target marketing better.

3
 ● Common methods used in Unsupervised Learning:
’"´
◎
1. Clustering:
z\‘ Two Common Clustering Methods:
’
◆ K-Means Clustering:
You tell the computer how many clusters (groups) you want — say 3.
It finds 3 center points, then assigns every data point to the nearest center.
◆ Hierarchical Clustering:
It doesn’t need you to specify the number of groups.
It builds a tree of data points — grouping similar ones step by step.
You can cut the tree at any level to form clusters.
2. Dimensionality reduction:
Imagine you're filling out a student form:
Name
Father’s Name
Mother’s Name
School Name
Class
Age
Date of Birth
Now you realize:
Age and Date of Birth give almost the same info.
Father’s and Mother’s Name may not help in the analysis.
So you reduce the number of fields, keeping only the useful ones, like:
Name
Class
Age
’´" That’s dimensionality reduction — removing unnecessary or repeated info to keep the
◎
●
important stuff.
Real-world data often has hundreds or thousands of features (columns).
Dimensionality reduction helps to:
1. Remove noise
2. Make visualization easier
3. Speed up learning

3. Semi-Supervised Learning
A hybrid approach where the model is trained on a small amount of labeled data along with a large amount of
unlabeled data.
Goal: Improve learning accuracy when labeling data is expensive. Applications:
Medical imaging, speech recognition, web content classification. Applications of Semi-
Supervised Learning
Application How it's used
Medical Imaging Only a few X-rays are labeled by doctors, rest are unlabeled
Speech Recognition Only some audio clips have transcripts

4
 Web Content Classification A few web pages are manually labeled; model guesses the rest
Language Translation Limited labeled sentence pairs, huge unlabeled corpus

You’re at a fruit market and you label only 3 fruits:

But there are hundreds of other fruits with no labels.

You then let a child observe the shapes and colors of the other fruits.
Over time, the child starts recognizing and labeling the rest — based on the few examples
you gave.

Term Meaning
Labeled Data Data with correct answers (e.g., email = spam or not)
Unlabeled Data Data without any labels (e.g., just the email text)

Semi-Supervised Learning Model trained on a small amount of labeled data + a large amount of unlabeled
data

4. Reinforcement Learning
In this paradigm, an agent learns to make decisions by interacting with an environment, receiving
rewards or penalties for actions.
Goal: Learn a sequence of actions that maximize cumulative reward.
Applications: Robotics, game AI (e.g., AlphaGo), autonomous vehicles.
◆ Key Components:
1. Agent: Learner or decision maker.
2. Environment: Where the agent interacts.
3. Reward Signal: Feedback to guide learning.
🎮 What is Reinforcement Learning?
⬛ Simple Explanation:
Imagine you're teaching a dog a trick — say, to sit.
At first, the dog doesn’t know what to do.
When it accidentally sits, you give it a treat G
c̀ (reward).

5
 When it jumps instead, you give no treat.
Over time, the dog learns that sitting = reward — and does it more often.
"
’
´
●
◎ This is Reinforcement Learning:
Learning by trial and error with rewards and penalties.
C Video Games
You’re playing a game:
You move forward → gain coins ⬛
You fall into a pit → lose a life +
You finish a level → bonus points "´●
’
◎

,J In Machine Learning Terms:

u
Term Meaning
Agent The learner (e.g., dog, robot, AI player)
Environment The world it interacts with (e.g., house, game, road)
Action What the agent chooses to do (e.g., sit, jump, move
left)
Reward Positive or negative feedback (e.g., treat, penalty,
score)
Goal Learn which actions give the maximum long-term
reward

D Summary Table of All Learning Types

Learning Type What it Learns From Data Requirement Example

Supervised
Correct answers (labels) Lots of labeled data Spam detection
Learning
Unsupervised Customer
Hidden patterns Only input data, no labels
Learning grouping
Few labels + many Small labeled + large Language model
Semi-Supervised
unlabeled examples unlabeled data training
Reinforcement Rewards and No labels — only feedback Game AI, robot
Learning punishments after action walking

Challenges in Machine Learning

Machine Learning, while powerful, faces several real-world challenges:

1. Data-Related Challenges
• Insufficient Data: Not enough examples to train the model effectively.
• Poor Quality Data: Noisy, missing, or incorrect data can mislead the model.
• Imbalanced Data: One class dominates others, leading to biased models.

6
 • High Dimensionality: Too many features can lead to overfitting and complexity.

2. Model-Related Challenges
• Overfitting: Model learns noise instead of pattern (high accuracy on training data but poor
generalization).
• Underfitting: Model is too simple to capture underlying trends (poor performance).
• Model Selection: Choosing the right algorithm for a task can be tricky.
• Hyperparameter Tuning: Requires trial and error to find optimal settings.

3. Computational Challenges
• Scalability: Training large models on big datasets requires high computation power.
• Latency: In real-time applications, predictions need to be fast.
• Resource Limitations: Limited access to GPUs, memory, or storage.

4. Ethical and Interpretability Issues

• Bias and Fairness: ML models can perpetuate or amplify biases in training data.
• Interpretability: Black-box models like deep neural networks are hard to explain.
• Privacy Concerns: Use of sensitive data (e.g., health records) must ensure data protection.

5. Deployment and Maintenance

• Model Drift: Performance may degrade over time as data patterns change.
• Version Control: Managing updates to data, models, and code.
• Integration: Incorporating ML models into existing software systems.

Supervised learning model examples

There are two types of supervisory learning models in machine learning. They are
1. Regression models
2. Classification models
1. Regression models

⬛ Supervised Learning:

7
 You give the computer:
Input data
Correct output (label)
The computer learns the relationship and uses it to predict the output for new data.

⬛ Regression:
A regression model is used when you want to predict a number (a continuous value).

´"◎●’ Real-Life Example: House Price Prediction You're a

real estate agent.
You want to predict house prices based on: Size of the
house (in sqft)
Number of bedrooms You
have data like this:
Size (sqft) Bedrooms Price (₹)
1000 2 ₹50 lakhs
1500 3 ₹75 lakhs
200 4 ₹1 crore
This is your training data.
Now a customer comes and asks:
“What would be the price of an 1800 sqft, 3-bedroom house?” You give this to
0 your trained regression model, and it predicts:
⬛ ₹90 lakhs

⬛ How does the Regression Model work?

t
s
¹
²
Step 1: Understand the pattern It
learns from the data:
As size increases, price increases.
More bedrooms usually mean higher price. Step 2: Fit a
line or model
It tries to fit a mathematical formula that best matches the data:
Price = a × Size + b × Bedrooms + c
Where:
a, b, and c are values the model learns from the data. Step 3: Make
predictions
When new data comes in (like 1800 sqft, 3 bedrooms), it plugs into the formula and gives a predicted
price.

📉 Types of Regression Models in Supervised Learning

10
 Type of Regression What it does Best for

Linear Regression Fits a straight line When data has a simple trend

Multiple Linear Uses more than one feature When more than one input affects
Regression output

Polynomial Regression Fits a curved line When data has a curve

Decision Tree Uses rules and branches When data has if-else patterns
Regression

Random Forest Uses multiple trees, more accurate For complex and noisy data
Regression

Types of Regression Models in Supervised Learning

📘 1. Linear Regression

◆ Description:
Models the relationship between a single independent variable (X) and a dependent variable
(Y) using a straight line.
The equation is:
𝑌 = 𝑎𝑋 + 𝑏
where a is the slope and b is the intercept.
◆ When to Use:
When the relationship between variables is linear (i.e., increasing or decreasing in a straight line). Example:
Predicting house price based on its size (sqft).

Diagram Insight:
The red line is the "best-fit" line.

2. Multiple Linear Regression

◆ Description:
Models the relationship between two or more independent variables and a dependent variable. The equation
is:
𝑌 = 𝑎1𝑋1 + 𝑎2𝑋2 + ⋯ + 𝑏
◆ When to Use:
◆ When you want to include multiple factors in prediction.
◆ Example: Predicting house price based on size, number of bedrooms, and location.

11
 Diagram Insight:
Each dot represents a data point with more than one input variable (X₁, X₂...).
Harder to visualize in 2D, but the model combines all inputs to estimate Y.

📘 3. Polynomial Regression

◆ Description:

• Extends linear regression by adding powers of the input variable (X², X³, etc.).
• Captures curved patterns in data.

◆ Equation Example:
𝑌 = 𝑎𝑋2 + 𝑏𝑋 + 𝑐
◆ When to Use:

When data shows non-linear trends, like growth curves, speed vs. time, etc. Example:
Predicting population growth.

12
 Diagram Insight:
The orange curve shows a quadratic fit (degree 2).
Better fits curving patterns that a straight line would miss.

📘 4. Decision Tree Regression

◆ Description:
Breaks data into branches based on feature conditions (like a flowchart). Each
branch leads to a predicted output value.
◆ When to Use:
When the data has sudden jumps or clear thresholds. Handles
both linear and non-linear data.
◆ Example:
Predicting electricity usage:
If temperature < 20°C → Low usage If
20–30°C → Medium usage
If >30°C → High usage

13
 Diagram Insight:
You can see levels of outputs (5, 10, 15) based on value ranges.
Useful for interpretable, rule-based decisions.

🧠 Summary Table:

Regression Type Shape of Handles Multiple Suitable For

Model Inputs?
Linear Regression Straight Line + No Simple linear trends
Multiple Linear Straight Plane ⬛ Yes Multi-factor predictions
Regression
Polynomial Regression Curved Line ⬛ Yes Non-linear trends
Decision Tree Step-wise ⬛ Yes Rule-based or piecewise data
Regression patterns

2. Classification model example

A classification model is used in supervised learning when the output you are predicting is a category or class,
not a number.

"◎
●’´ It answers: “Which group does this data belong to?”

14
 ç Real-World Example: Email Spam Detection
v†.
Let’s say you want to build a system that decides whether an incoming email is
“Spam” or “Not Spam”.
) What the model sees (input features):
Does the email contain the word "free"?
Is there a suspicious link?
How many recipients are there?
Who is the sender?
These are called features (inputs).
◎●" What the model predicts (output label):
´’
⬛ Spam
⬛
ç
-
v
†
⬛ Not Spam

This is the class the model has to predict.

s¹²⬛t How Does the Classification Model Work?
Step-by-step:
1. Training:
You show the model many examples:
Email A → Spam
Email B → Not Spam
Email C → Spam
... (with inputs like keywords, links, etc.)
2. Learning:
The model finds patterns — for example:
Emails with “win money” are often spam.
Emails from known contacts are not.
3. Prediction:
When a new email arrives, the model uses what it learned and says:
“This email is likely spam.”

Most Common Classification Algorithms

Algorithm When to Use

Logistic Regression For binary choices (Yes/No, Spam/Not Spam)
Decision Trees When you want rule-based reasoning
K-Nearest Neighbors (KNN) When similar things should belong together
Random Forest When you want high accuracy from many decision
trees
Support Vector Machine For drawing a decision boundary
(SVM)
Naive Bayes For email, text, and language-based classification

15
 1. Logistic Regression

⬛ What is it?
Logistic Regression is a classification algorithm, not a regression algorithm (despite its name). It’s used to predict
categorical outcomes, mostly binary (like Yes/No, 0/1, True/False).

*
˛
C Why Not Linear Regression?
Linear regression predicts values from -∞ to +∞ like a number. For
classification, we need probabilities between 0 and 1.
Logistic Regression uses the Sigmoid function to squash output between 0 and 1.

◦ How it Works
O
☼
1. Linear Combination
First, it computes a linear combination of input features:
z = b0 + b1·x1 + b2·x2 + ... + bn·xn
2. Apply Sigmoid Function
Then, it passes z through the sigmoid function:
P(y) = 1 | x) = 1 / (1 + e^(−z))
This gives the probability that the input belongs to class 1.
3. Classification Rule
If the probability > 0.5 → predict class 1 If the
probability ≤ 0.5 → predict class 0 (This threshold can
be adjusted)
⬛#/ Training the Model: Log Loss
To train Logistic Regression, we use a loss function called Log Loss (or Cross Entropy):
Loss = −[y·log(p) + (1−y)·log(1−p)] y =
actual answer (0 or 1)
p = predicted probability
The algorithm uses gradient descent to adjust the weights (b0, b1, etc.) to reduce this loss — like a student
adjusting study habits to get better marks!

◦☼Gradient Descent?
O
Imagine you're blindfolded and trying to walk down a hill to reach the lowest point (the bottom of the hill = best
model).
At first, you guess some weights (b0, b1, etc.).
You calculate the error (how wrong your model is).
Then, you use gradient descent to take small steps downhill, adjusting the weights to reduce the error.
C˛* Gradient Descent = "Keep taking steps in the direction that reduces the error."
Each step makes the model slightly better, until you reach a point where it can’t get any better — that’s when
the model is "trained."

16
 ⬛ How Gradient Descent Works in Logistic Regression
#
Start with random weights
Use the weights to make predictions (using sigmoid function)
Calculate how wrong the predictions are (using a loss function) Adjust
the weights to make the predictions better
Repeat until the model becomes accurate
This loop happens many times — like practicing over and over until you get the answer right.

🌳 2. Decision Trees

⬛ When to Use:
When you want rule-based reasoning
When interpretability is important
Q Example:
•
˙
Loan Approval – A bank checks income, age, credit score.
[Credit Score > 700?]
/ \
Yes No
[Income > 50k?] Reject
/ \
Yes No
Approve Reject

3. K-Nearest Neighbors (KNN)

⬛ When to Use:
When "birds of a feather" logic applies (similar things group together) When
data has natural clusters
• Example:
Q̇
Classifying handwritten digits (0-9) using pixel similarity.

🌲 4. Random Forest

⬛ When to Use:
When you need high accuracy
When you have lots of features and data
Q̇ Example:
•
Predicting loan defaults using many customer details

📐 5. Support Vector Machine (SVM)

⬛ When to Use:
When you need a clear boundary between classes
Works well with high-dimensional data
• Example:
Q̇
Classifying tumors as malignant or benign based on gene expressions.

17
📧 6. Naive Bayes

⬛ When to Use:
For text classification (e.g., emails, reviews) When
independent features are assumed
Q̇ Example:
•
Email spam detection, sentiment analysis
‘z’ Based on Bayes’ Theorem and assumes each feature contributes independently to the outcome.

Unsupervised model example- K-means clustering.

˛ What is K-Means Clustering?

*
C
K-Means Clustering is an algorithm that automatically groups data into K clusters based on similarity
— without needing labels.
G Real-Life Analogy:
Imagine you're running a shopping mall. You have no idea who your customers are (no labels).
But you have data like:

Age

Spending pattern

You want to group customers:

Group 1: Young & High Spenders

Group 2: Middle-aged & Budget Conscious

Group 3: Senior Citizens

This is clustering. K-Means helps you find such groups.

How Does K-Means Work? (Step-by-Step)

⬛²
t
s
¹
Let’s say you want to make 3 clusters (K = 3) from your data.

Step 1: Choose the number of clusters (K)

E.g., K = 3 (you want to group into 3 parts)

Step 2: Randomly place 3 points — these are your initial centroids (cluster centers).
Step 3: Assign each data point to the nearest centroid
Each point now belongs to one of the 3 clusters.

18
 Step 4: Recalculate the centroids
For each cluster, calculate the mean position of all points in it — this becomes the new center.

Step 5: Repeat steps 3 & 4

Keep repeating until the centroids don’t change much.

Introduction to Artificial Neural Networks (ANNs)

An Artificial Neural Network (ANN) is a computational model inspired by the structure and functioning of the
biological nervous system, particularly the human brain. Just as the human brain is composed of billions of
neurons that communicate with each other to process information, ANNs consist of artificial neurons (also known
as nodes or units) that are interconnected in a network and work collectively to solve tasks.

20
Neural networks are capable of learning and identifying patterns directly from data without pre-defined rules. These
networks are built from several key components:
Neurons: The basic units that receive inputs, each neuron is governed by a threshold and an activation function.
Connections: Links between neurons that carry information, regulated by weights and biases.
Weights and Biases: These parameters determine the strength and influence of connections. During training, the
network adjusts these weights to minimize the prediction error. Biases are additional parameters added to the
neuron to shift the activation function, helping the model to better fit the data. Propagation Functions: Mechanisms
that help process and transfer data across layers of neurons.
Learning Rule: The method that adjusts weights and biases over time to improve accuracy.
ANNs are capable of learning from data. They are particularly useful for modeling complex relationships between
inputs and outputs and for discovering hidden patterns in data. Applications of ANNs include image recognition,
speech processing, medical diagnosis, and financial forecasting, among others.

Basic Structure of an ANN

A typical artificial neural network consists of three main types of layers:

⬛ a) Input Layer
This is the first layer of the network.

It receives the raw input data and passes it to the subsequent layers for processing.

Each neuron in this layer corresponds to one feature of the input dataset (e.g., in a student
performance prediction model: attendance rate, hours of study, and internal assessment
marks).

□ b) Hidden Layer(s)
These are the intermediate layers between the input and output layers.

They perform the core computations by processing the inputs through weighted
connections and activation functions.

The number of hidden layers and the number of neurons within each layer determine
the complexity and learning capacity of the network.

⬛ c) Output Layer
This layer provides the final output or prediction of the network.

The number of neurons in this layer corresponds to the nature of the task:

For binary classification, there is usually one neuron.

 For multi-class classification, there are multiple output neurons.

For regression, a single neuron with linear activation may be used.

˙„9•Illustrative Example: Predicting Student Performance

.
Let us consider a basic example where the objective is to predict whether a student will pass or fail
based on certain academic inputs.

➤ Inputs:
Attendance Percentage

Hours of Study per Week

Internal Assessment Marks

These three values are fed into the input layer of the neural network.

➤ Processing:
Each input is multiplied by a weight.

The weighted inputs are summed along with a bias.

The result is passed through an activation function (e.g., sigmoid or ReLU), which introduces
non-linearity and helps the network learn complex patterns.

➤ Output:
If the output value is close to 1, the student is predicted to pass.

If the value is close to 0, the student is predicted to fail.

 * Learning Patterns:
C˛
Over time, as the ANN is trained with more examples (student data with known pass/fail
outcomes), it learns patterns such as:

Higher hours of study and better internal marks increase the probability of passing.

Poor attendance may significantly reduce the chances of passing.

_
]H Summary of Core Concepts:

Concept Description
Artificial
Basic computational unit that mimics a biological neuron
Neuron
Layers Organized structure: Input, Hidden, and Output
Weights &
Parameters that guide the learning process
Biases
Activation Function that adds non-linearity and helps the network learn complex
Func. data
Adjusting weights based on data using optimization algorithms like
Learning
gradient descent

Perceptron
˛
C
* Perceptron
A Perceptron is a type of artificial neuron, introduced by Frank Rosenblatt in 1958. A perceptron takes several
inputs, applies weights to them, adds a bias, and then passes the result through an activation function to produce
an output. A Perceptron is the most fundamental unit of an artificial neural network, which is widely used in
machine learning and artificial intelligence. It is inspired by the functioning of a biological neuron and is used to
perform binary classification—that is, to decide whether something belongs to one class or another (e.g., yes/no,
true/false, safe/unsafe).

Q̇ Structure of a Perceptron
•
A perceptron includes:
Inputs: x₁, x₂, ..., xₙ
Weights: w₁, w₂, ..., wₙ Bias:
b
Weighted Sum: z = w₁·x₁ + w₂·x₂ + ... + wₙ·xₙ + b Activation
Function: Usually a step function

o Perceptron Equation
«
#
Weighted Sum:
 𝑧 = 𝑤₁ · 𝑥₁ + 𝑤₂ · 𝑥₂ + . . . + 𝑤ₙ · 𝑥ₙ + 𝑏
Output (Activation Function):
If z > 0, then Output = 1 Else,
Output = 0

H Real-Life Examples
_
]
1. University Admission
Inputs:
x₁ = Math marks x₂ =
English marks
If:
w₁·x₁ + w₂·x₂ + b > 0 → Admit (1) Otherwise →
Reject (0)
2. Fire Safety Alert
Inputs from sensors:
x₁ = Temperature x₂ =
Smoke level x₃ = Gas
detection Decision:
If w₁·x₁ + w₂·x₂ + w₃·x₃ + b > 0 → Unsafe (1) Else → Safe
(0)

’
⬛ Perceptron Learning Rule
When a prediction is wrong, we update the weights and bias: Weight Update
Rule:
wᵢ = wᵢ + α × (y_true − y_pred) × xᵢ Bias
Update:
b = b + α × (y_true − y_pred) Where:
α = learning rate y_true =
actual label
y_pred = predicted label

Ị Limitation
A single-layer perceptron can only classify linearly separable data (like AND, OR), but not problems like XOR,
which are non-linearly separable. This was addressed later using multi- layer perceptrons (MLPs).

⬛ Summary Table
Feature Description
Purpose Binary classification
Learning Weight adjustment based on error
Inspired by Biological neurons

Applications Safety systems, decision support, etc.

Limitation Cannot solve non-linear problems like XOR
Working of Neural Networks

Artificial Neural Networks (ANNs) are computational models inspired by the functioning of the human brain. They
learn from data through three key phases: Forward Propagation, Backpropagation, and Iteration.

1. Forward Propagation
(i) Linear Transformation
Each neuron receives inputs, multiplies them with weights, adds a bias, and computes a value denoted as z:
Equation:
𝑧 = 𝑤₁ · 𝑥₁ + 𝑤₂ · 𝑥₂ + … + 𝑤ₙ · 𝑥ₙ + 𝑏
Where:
x₁, x₂, ..., xₙ are input features
w₁, w₂, ..., wₙ are weights
b is the bias
z is the result passed to the activation function
(ii) Activation Function

To introduce non-linearity, z is passed through an activation function. Common

activation functions:
Sigmoid: 𝜎(𝑧) = 1 / (1 + 𝑒⁻ᶻ)
Tanh: 𝑡𝑎𝑛ℎ(𝑧) = (𝑒ᶻ − 𝑒⁻ᶻ) / (𝑒ᶻ + 𝑒⁻ᶻ)
ReLU: 𝑓(𝑧) = 𝑚𝑎𝑥(0, 𝑧)

2. Backpropagation
(i) Loss Calculation
The network calculates a loss to measure prediction error. Examples:
Mean Squared Error (MSE) (for regression):
𝑀𝑆𝐸 = (1/𝑛) · 𝛴(𝑦ᵢ − ŷᵢ)²
Cross-Entropy Loss (for classification): Loss = −Σ[y
· log(ŷ)]
Where:
yᵢ = actual output
ŷᵢ = predicted output
(ii) Gradient Calculation
The gradients of the loss with respect to weights and biases are computed using the chain rule of calculus.
(iii) Parameter Update
Weights and biases are updated using an optimization algorithm such as Stochastic Gradient Descent (SGD):
Update rules:
𝑤 = 𝑤 − 𝛼 · 𝜕𝐿𝑜𝑠𝑠/𝜕𝑤
𝑏 = 𝑏 − 𝛼 · 𝜕𝐿𝑜𝑠𝑠/𝜕𝑏
Where:
α = learning rate
∂Loss/∂w, ∂Loss/∂b = partial derivatives of the loss

3. Iteration (Training Process)

This cycle — forward propagation → loss computation → backpropagation → parameter update — is
repeated over multiple epochs (iterations through the dataset). Over time, the network learns to reduce the
loss and improve its predictions.

⬛ Conclusion
Neural networks improve by continuously adjusting internal parameters based on feedback from the loss function.
This enables them to solve various complex tasks such as:
Image and speech recognition Predictive
analytics
Natural language processing Risk
and safety modeling

Universal Approximation Theorem (UAT) – Statement only

“A feedforward neural network with a single hidden layer and a non-linear activation function (like sigmoid or
tanh) can approximate any continuous function defined on a bounded input domain (technically, a compact subset
of ℝⁿ), to any desired degree of accuracy, provided it has enough neurons.”
Q̇ What It Means – Intuition
•
Let’s interpret the components of the theorem in simpler terms:
Term Meaning
Feedforward network A neural network where information moves only forward — from
input to output (no loops).
Single hidden layer Just one intermediate layer between input and output layers.

Continuous function A function with no jumps or breaks (like a smooth curve).

Approximate The network output can get very close (within any tiny error) to
the actual function value.
Compact subset of ℝⁿ A finite region in n-dimensional space, like a cube or a closed
interval.
Non-linear activation A function that adds complexity — without it, the network would behave
function like a straight line.
🧠 Why It’s Important
It justifies the power of neural networks: Even with only one hidden layer, a network is theoretically capable of
learning any pattern or relationship in data — like predicting disease risk, modeling chemical reactions, or
identifying fire hazards.

This means neural networks are universal approximators — they can model anything (as long as it's continuous
and bounded).

🔄 Example Analogy
Think of a neural network as a toolbox of small building blocks (neurons). Imagine
you're trying to trace a curve (a function) using Lego blocks.

With enough small blocks (neurons) and the right arrangement (weights and biases), you can build a very
close copy of the curve. The more complex the curve, the more blocks you need — but you don’t need a
second layer of blocks to do it.

. Important Notes
'
The theorem doesn’t say the network will learn the function easily — just that it can, if given the right weights.

It doesn’t guarantee training success — learning the weights may be hard in practice.

More layers (deep learning) are often used in practice because they learn complex functions more efficiently
and with fewer neurons per layer.

⬛ Summary

Concept What It Tells Us

UAT Neural networks with one hidden layer can approximate any
continuous function
Practical Neural networks are flexible and powerful
implication
Limitation The theorem is about potential — not training efficiency

Multi-Layer Perceptron (MLP)

Q̇ What is a Multi-Layer Perceptron?
•
A Multi-Layer Perceptron (MLP) is a class of feedforward artificial neural networks that consists of
three or more layers: Input
layer
One or more hidden layers Output
layer
It extends the basic Perceptron model by introducing hidden layers, which enable it to learn and represent non-
linear relationships in data.
– Architecture of an MLP
'c‘.•⬛
1. Input Layer
Receives raw data features (x₁, x₂, ..., xₙ). Does not
perform any computation.
2. Hidden Layers
One or more intermediate layers where computation occurs.
Each neuron performs a weighted sum of inputs and applies a non-linear activation function.
Computation in each neuron:
𝑧 = 𝑤₁ · 𝑥₁ + 𝑤₂ · 𝑥₂ + . . . + 𝑤ₙ · 𝑥ₙ + 𝑏
𝑎 = 𝑓(𝑧)
Where:
wᵢ = weights
xᵢ = input values
b = bias
f(z) = activation function (e.g., ReLU, sigmoid, tanh)
a = activation/output of the neuron
3. Output Layer

Produces the final prediction.

For classification tasks, it often uses softmax or sigmoid activation.

◦ Working Mechanism
☼
O
⬛ Forward Propagation
Input data flows from the input layer to the output layer through hidden layers. Each layer
applies:
Linear transformation: 𝑧 = 𝑤·𝑥 + 𝑏
Activation function: 𝑎 = 𝑓(𝑧)
’ Backpropagation
⬛
The error between predicted and actual output is calculated using a loss function. Gradients
of the loss with respect to weights and biases are computed.
Parameters are updated using optimization algorithms (e.g., SGD, Adam).

« Activation Functions
«
o
#
Function Formula Purpose
ReLU f(z) = max(0, z) Fast and widely used
Sigmoid σ(z) = 1 / (1 + e⁻ᶻ) Squashes values between 0 and 1
Tanh tanh(z) = (eᶻ − e⁻ᶻ) / (eᶻ + e⁻ᶻ) Output between −1 and 1

H Example Use Cases

]_
Handwritten digit recognition (e.g., MNIST dataset) Fire
risk classification
Predicting patient outcomes based on symptoms Credit
risk evaluation

⬛ Key Features of MLP

Feature Description

Deep architecture One or more hidden layers

Non-linearity Enabled by activation functions
Universal Approximation Can model any continuous function
Trainable parameters Weights and biases learned from data
Supervised learning Requires labeled data for training

Ị Limitations
Requires large datasets and computational power Prone to
overfitting without regularization
May suffer from vanishing gradient problem in deep architectures

Deep Neural Network (DNN)

Q̇ What is a Deep Neural Network?

•
A Deep Neural Network (DNN) is an extension of a Multi-Layer Perceptron (MLP) that consists of multiple
hidden layers between the input and output layers. These deep architectures enable the model to learn hierarchical
representations of data, capturing complex patterns and dependencies.

– Structure of a DNN A
‘.•⬛
'c
typical DNN includes:
1. Input Layer:
Receives raw input features (x₁, x₂, ..., xₙ).
2. Multiple Hidden Layers:
Each layer contains neurons that perform computations using:
3. Linear transformation:
𝑧 = 𝑤₁ · 𝑥₁ + 𝑤₂ · 𝑥₂ + … + 𝑤ₙ · 𝑥ₙ + 𝑏
4. Non-linear activation function:
𝑎 = 𝑓(𝑧)
5. Output Layer:
Generates the final output. The type of activation function depends on the task (e.g., softmax
for multi-class classification, sigmoid for binary classification).

◦ Working Mechanism
O
☼
⬛ Forward Propagation
Data flows layer by layer from input to output. Each
neuron computes a value using:
z = w·x + b a
= f(z)
’⬛ Backpropagation and Learning
The network computes a loss to measure prediction error: For
regression: MSE = (1/n) Σ(yᵢ − ŷᵢ)²
For classification: Loss = −Σ[y·log(ŷ)]
Using gradient descent, weights and biases are updated to minimize the loss:
w = w − α · ∂Loss/∂w b
= b − α · ∂Loss/∂b
Deep networks can:
µ
Learn low-level features in early layers (e.g., edges in images) Learn high-
level abstractions in deeper layers (e.g., shapes, objects)
This makes them extremely powerful for:
• Image classification
• Speech recognition
• Language translation
• Fire hazard prediction
• Industrial process control

#« Common Activation Functions

o
Function Formula Output Range

ReLU f(z) = max(0, z) [0, ∞)

Sigmoid σ(z) = 1 / (1 + e⁻ᶻ) (0, 1)

Tanh tanh(z) = (eᶻ − e⁻ᶻ)/(eᶻ + e⁻ᶻ) (−1, 1)

⬛ Advantages of DNNs
• Can model very complex functions
• Enables automatic feature extraction
• Widely applicable to real-world problems

.
' Challenges and Limitations
Challenge Description
Overfitting High risk without proper regularization
Vanishing/Exploding Gradients Affects training in very deep networks
High computational cost Requires GPUs and large datasets
Interpretability Hard to understand what the model has learned

˛* Summary Table
C
Feature Deep Neural Network
Depth Multiple hidden layers
Learning Via forward and backward propagation
Advantage Learns complex hierarchical patterns
Limitation Computationally intensive, harder to interpret
30