Machine Learning

Introduction to Machine Learning

5th Semester [Link]
 Why Machine Learning?

• How does YouTube recommend videos?

• How does Google Maps predict traffic?
• How does Amazon suggest products?
• How does Gmail detect spam?
• In all these applications,
computers are not
explicitly programmed for
every situation. Instead,
they learn patterns from
data. This ability is called
Machine Learning.
What is Machine Learning?

• Machine Learning allows computers to

learn from data and improve
performance without being programmed
for every task.
 Traditional Programming vs ML

Machine Learning:
Traditional: Input + Input + Output
Rules → Output Data → Learning
Algorithm → Model
 Key Differences
Factor Traditional Programming Machine Learning

Instruction Method Explicit rules and logic Learns patterns from data

Data Handling Structured, predictable data Large, often unstructured data

Predictions may vary as model adapts to

Outcome Predictability Always the same result for same input
new data

Adjusts based on new data (self-

Flexibility Limited to predefined conditions
improving)

Development Process Linear: write, debug, deploy Iterative: train, evaluate, tune, retrain

Can be opaque, often needs explainable

Transparency Easy to trace and debug
AI

Problem Complexity Best for simple, well-defined tasks Best for complex, data-rich tasks
 • Marks = 95
Traditional • IF marks > 90
• Grade = A
Programming • Rules are written by programmers.
 • Student Data → ML
Machine Algorithm → Predict Grade
• The machine discovers
Learning rules automatically.
 What does • We do not write every rule
manually.
'Not Explicitly
• Instead, we provide data
Programmed' and the machine learns
Mean? patterns on its own.
 Real-Life Example

1 2 3
Spam Traditional: ML: Learn from
Detection: Write spam thousands of
rules manually. spam and non-
spam emails.
 • Arthur Samuel (1959):
• 'Ability to learn without being
explicitly programmed.'
• "Machine Learning is the field of
study that gives computers the
Definition ability to learn without being
explicitly programmed.“
of • Tom Mitchell Definition
Machine • "A computer program learns
from experience E with respect
Learning to task T and performance
measure P if its performance
improves with experience."
 • Task = Email Spam
Detection
Example: • Experience = Thousands of
Emails
• Performance = Accuracy
 Applications of
Field Application Machine Learning
Healthcare Cancer Detection

Agriculture Crop Disease Detection • Healthcare,

Finance Fraud Detection Agriculture, Finance,
E-commerce
Recommendation E-commerce,
System
Automobile Self-driving Cars Education,
Education
Student Performance Automobile, Social
Prediction
Media
Social Media Face Recognition
 Learning Paradigms
• A learning paradigm is the way a machine
learning model learns from data.
• Machine Learning is mainly divided into three
learning paradigms:
1. Supervised Learning
2. Unsupervised Learning
3. Reinforcement Learning
 Supervised Learning
Definition
• Supervised Learning is a type of machine learning
where the model learns from labelled data.
• Labelled Data means the correct output is already
known.
• The model learns the relationship between input
and output.
• Data contains labels.
• Examples: Spam Detection, Disease Prediction,
House Price Prediction
 Supervised Learning
Input Data + Correct Output
↓
Learning Algorithm
↓
Model
Supervised Learning
 Supervised Learning
• Machine learns:
Study Hours Result
2 Fail
4 Pass
6 Pass
1 Fail

• More study hours generally lead to passing.

• Now if a new student studies 5 hours, the
model predicts Pass.
 Supervised Learning
Its main features are:
• Each input has a known output
• Adjusts itself to reduce prediction errors
• Make accurate predictions on new data
• For example it recognizing handwritten digits
from trained data
 Types of Supervised Learning
• In machine learning, supervised learning is
broadly split into two main types based on the
nature of the target output variable you are
trying to predict:
– Classification
– Regression.
 Classification (Discrete Output)

• In classification tasks, the machine learning

model predicts a discrete label or category.
The output belongs to a specific, predefined
class.
• Output Example:
– "Spam" or "Not Spam",
– "Cat" or "Dog",
– "Risk Level: High/Medium/Low“
– Pass / Fail
 Classification (Discrete Output)

• Student Data
↓
• Classifier
↓
• Pass / Fail
 Types of Classification

• Binary Classification: Choosing between exactly

two classes (e.g., Yes/No).
• Multi-class Classification: Choosing from more
than two distinct classes (e.g., classifying an
image as a car, bike, or truck).
• Multi-label Classification: Assigning multiple
labels to a single input (e.g., tagging a news
article as both "Politics" and "Finance").
 Common Classification Algorithms:

• Logistic Regression (despite the name, it's

used for binary classification)
• Support Vector Machines (SVM)
• Decision Trees and Random Forests
• Naïve Bayes
• K-Nearest Neighbors (KNN)
 Regression (Continuous Output)

• Predicts continuous values.

• In regression tasks, the model predicts a
continuous numeric value. The output is a real
number that represents a quantity, measurement,
or scale.
• Output Example: Predicting a house price
(Rs.350,000), tomorrow's temperature (24.5°C),
or stock market prices.
• Core Objective: To find the relationship mapping
between input variables and a continuous output
mapping line or curve.
 Regression (Continuous Output)

Area + Bedrooms
↓
Regression
↓
House Price
 Common Regression Algorithms

• Linear Regression (Simple and Multiple)

• Polynomial Regression
• Ridge and Lasso Regression (Regularization
techniques)
• Support Vector Regression (SVR)
• Decision Trees / Random Forests (configured
for regression)
 Summary Comparison

Feature Classification Regression

Discrete / Categorical
Output Type Continuous (Numbers)
(Labels)
Map inputs to predefined Predict a specific numeric
Goal
classes value
Mean Squared Error (MSE),
Accuracy, Precision, Recall,
Evaluation Metrics 𝑅2 Score, Mean Absolute
F1-Score
Error (MAE)
Fraud detection (Fraud / Weather forecasting (Exact
Example Use Case
Safe) temperature)
 Advantages and Disadvantages

• Advantages
✓ High accuracy
✓ Easy evaluation
✓ Well-established algorithms
• Disadvantages
✗ Requires labelled data
✗ Labelling is expensive
 Advantages and Disadvantages
• Supervised learning delivers high accuracy
and clear results via mapped data, but
demands expensive manual labelling and
risks overfitting or inheriting human bias.
Overfitting (Memorizing vs. Learning)

• Overfitting happens when a model becomes too smart

for its own good. Instead of identifying the broad,
general rules of a dataset, it essentially memorizes the
specific training examples—including all the random
noise.
• The Problem: It scores a perfect 100% on the data it
already saw, but the moment you give it new, real-
world data, it fails miserably because it doesn't know
how to adapt.
• Analogy: It’s like a student who memorizes the exact
answers to a practice exam instead of studying the
actual formulas;
 Unsupervised Learning
• Unsupervised Learning learns from unlabelled data.
• The model finds hidden patterns automatically.
• Unsupervised machine learning is a type of machine
learning where algorithms learn from data that
has no pre-defined labels or categories.
• In contrast to supervised learning where the
training data is labeled (think "cat" pictures and
"dog" pictures), unsupervised learning algorithms
are tasked with finding hidden patterns or
structures within the data itself.
 Unsupervised Learning
• Unsupervised Learning is the exact opposite: the
AI has no teacher, no labels, and no correct
answers provided.
• You give the model raw data, and it has to figure
out the hidden patterns, structures, or groupings
entirely on its own.
 Unsupervised Learning
Input Data
↓
Learning Algorithm
↓
Hidden Patterns

• No correct answer is provided.

 How Unsupervised Machine Learning
Works?
• Unsupervised Machine Learning algorithms
explore data by looking for structures or patterns.
• The primary goal is to model the underlying
structure or distribution of the data to learn more
about the data.
• These algorithms are particularly useful for
exploratory data analysis, dimensionality reduction,
and discovering hidden patterns within data.
 Types of Unsupervised Learning
The Two Main Types (Outputs)
• Instead of Classification and Regression,
unsupervised learning is primarily split into:
1. Clustering (Grouping Data)
2. Association (Finding Hidden Rules)
 Clustering (Grouping Data)
• The model looks at the traits of your data points and
groups similar ones together into "clusters."
• How it works: Points inside a cluster are highly similar
to each other, but completely different from points in
other clusters.
• Real-World Examples:
– Customer Segmentation: A company inputs user shopping
habits, and the AI groups them into "bargain hunters,"
"impulse buyers," and "tech enthusiasts" so marketing can
target them differently.
– Document Clustering: Grouping thousands of news articles
by topic without knowing the topics beforehand.
• Common Algorithms: K-Means Clustering, Hierarchical
Clustering, DBSCAN.
 Example: Customer Segmentation
Suppose a shopping mall has customer data:

Customer Age Spending

A 22 High
B 24 High
C 60 Low
The machine automatically groups similar customers.

Customer Data
↓
Clustering
↓

Group A
Group B
Group C
 Association (Finding Hidden Rules)
• The model discovers fascinating, underlying
relationships and "if/then" rules between different
variables in a massive dataset.
• How it works: It identifies items or events that
frequently occur together.
 Real-World Examples
– Market Basket Analysis: Finding relationships.
Customers who buy bread often buy butter. (They
then place these items closer together in the store).
Used in supermarkets.
– Recommendation Engines: "Users who watched
Movie A also watched Movie B.“

• Common Algorithms: Apriori Algorithm, FP-

Growth.
 Real-Life Applications

• Customer Segmentation
• Market Basket Analysis
• Recommendation Systems
• Image Grouping
 Advantages

✓ No labeled data required

✓ Finds hidden patterns
 Disadvantages

✗ Results may be difficult to interpret

✗ Accuracy measurement is challenging
 Dimensionality Reduction
• Dimensionality Reduction falls under the category of
Unsupervised Learning.
• Sometimes you have too much data (e.g., a table with
500 columns/features). Unsupervised models can
compress this data down to just the 2 or 3 most
important columns without losing the core
information.
• Why use it? It makes data easier to visualize on a
graph and speeds up other machine learning models.
• Common Algorithm: Principal Component Analysis
(PCA).
 Summery
• Unsupervised learning acts like an autonomous
explorer—it requires zero expensive data
labeling, but its output can be unpredictable and
harder to evaluate since there is no "correct
answer" key.
 Reinforcement Learning
• Reinforcement Learning (RL) is a type of machine
learning in which an agent learns to make
decisions by interacting with an environment and
receiving rewards or penalties for its actions.
• At its core, Reinforcement Learning (RL) is a
branch of machine learning concerned with how
an agent should take actions in an environment to
maximize some notion of cumulative reward.
 Reinforcement Learning
• Think of it as training a dog: you give a treat
(positive reward) when it performs a trick correctly,
and no treat (or a mild correction) when it doesn’t.
• Over time, the dog learns which actions lead to the
best outcomes.
•
 The Core Elements of RL
• An RL system relies on a continuous loop of
interaction between the AI and its surroundings:
• Agent: The AI or decision-maker (e.g., a self-driving
car software, a chess-playing bot).
• Environment: The world the agent interacts with
(e.g., the city streets, the chessboard).
• State (S): The current situation or configuration of
the environment.
• Action (A): The moves or choices available to the
agent.
• Reward (R): The feedback from the environment
evaluating the action (can be positive or negative).
 The Core Elements of RL
1. Agent
• The agent is the learner or decision-maker.
• It observes the environment and takes
actions.
• Example
– In a self-driving car:
– Agent = Self-driving car
– In a game:
– Agent = Computer player
 The Core Elements of RL
2. Environment
• The environment is everything around the agent
with which it interacts.
• Example
– For a self-driving car:
– Roads
– Traffic signals
– Other vehicles

– For a chess game:

– Chessboard and pieces
 The Core Elements of RL
[Link] (S)
• A state represents the current situation of the
environment.
• The agent observes the state before making a decision.
• Example
– Robot Navigation:
– State 1 → Robot at Start
– State 2 → Robot in Middle
– State 3 → Robot near Goal
• For a self-driving car:
– Current position
– Speed
– Distance from other vehicles
 The Core Elements of RL
4. Action (A)
• An action is a move or decision taken by the agent.
• Example
– Robot:
– Move Left
– Move Right
– Move Forward
– Move Backward
• Chess AI:
– Move Pawn
– Move Knight
• Self-driving Car:
– Accelerate
– Brake
– Turn Left
– Turn Right
 The Core Elements of RL
• Reward (R)
• A reward is feedback received after taking an
action.
• It tells the agent whether the action was good or
bad.
• Example
– Robot Navigation:
– Reach Goal → +10
– Move Correctly → +1
– Hit Wall → -5
 Additional Components

• Policy (π)
• A policy is the strategy used by the agent to
choose actions.
Simply,
• Policy = Rule that tells the agent what action to
take in a given state.
• Example:
– State: Red Signal
– Policy: Stop
 Additional Components

• Value Function
• The value function estimates how good a state
is in terms of future rewards.
• Example
• State near the goal: Value = High
• State far from the goal: Value = Low
 Additional Components

• Q-Value (Action-Value Function)

• Q-value measures the usefulness of taking a
specific action in a specific state.
State Action Q-value
Near Goal Move Forward 10
2
Near Goal Move Backward
 General Idea
Agent
↓
Action
↓
Environment
↓
Reward/Penalty

• The goal is to maximize rewards.

 Key Concepts
• To succeed, an RL agent must balance how it learns and how
it acts:
1. Exploration vs. Exploitation
• This is the ultimate dilemma in RL:
• Exploration: Trying new, unfamiliar actions to see if they lead
to better rewards (gathering information).
• Exploitation: Using known information to take the actions
that have historically yielded the highest rewards.
• Analogy: Exploitation is going to your favorite restaurant
and ordering the dish you love. Exploration is trying a brand-
new restaurant because it might be even better, even though
it risks a bad meal.
 Example of Reinforcement Learning:
Training a Robot to Reach a Goal
• Imagine a robot is placed in a room and must
reach a charging station.
• Step 1: Initial State
– The robot does not know the correct path.
– Robot → ? → Charging Station
 Cont…
• Step 2: Actions
• The robot can perform actions such as:
• Move Left
• Move Right
• Move Forward
• Move Backward
 Cont…
• Step 3: Reward and Penalty
• If the robot moves closer to the charging station
→ Reward (+1)
• If it hits a wall → Penalty (-1)
• If it reaches the charging station → Large Reward
(+10)
• Correct Move → Reward +1
• Wrong Move → Penalty -1
• Reach Destination → Reward +10
 Cont…
• Step 4: Learning Process
• The robot repeatedly tries different paths.
Try Action
↓
Receive Reward
↓
Learn Better Action
↓
Try Again
• After many attempts, it learns the shortest and
safest path.
 Cont…

Reinforcement Learning Components

Component Example
Agent Robot
Environment Room
Action Move Left, Right, Forward, Backward
Reward +1, -1, +10
Goal Reach Charging Station
Advantages
✓ Learns from experience
✓ Suitable for dynamic environments
Disadvantages
✗ Requires large training time
✗ Computationally expensive
 Comparison of Learning Paradigms
Feature Supervised Unsupervised Reinforcement
Data Labeled Unlabeled Reward-based
Goal Prediction Pattern Discovery Decision Making
Feedback Immediate No Feedback Reward/Penalty
Customer
Example Spam Detection Robot Navigation
Segmentation
Decision Tree, SVM, Q-Learning, Deep Q-
Algorithms K-Means, DBSCAN
KNN Network
 Perspectives and Issues in Machine
Learning
• This topic explains how machine learning
views a learning problem (perspectives) and
the challenges/issues faced while developing
ML models.
 1. Perspectives of Machine Learning

• Machine Learning can be viewed from different

perspectives.
A. Learning as Pattern Recognition
– Machine learning identifies patterns from data and
uses them for prediction.
Student Marks Data
↓
Pattern Learning
↓
Pass/Fail Prediction
 Perspectives and Issues in Machine
Learning
• Applications
– Face Recognition
– Handwriting Recognition
– Speech Recognition
 Perspectives and Issues in Machine
Learning
B. Learning as Function Approximation
• The machine learns a mathematical relationship
between inputs and outputs.
• Example
– House Price Prediction
Area, Bedrooms
↓
Learning Function
↓
House Price
 Perspectives and Issues in Machine
Learning
• The model approximates the function:
–𝑌=𝑓 𝑋
• where
– X = Input Features
– Y = Output
 Perspectives and Issues in Machine
Learning
C. Learning as Optimization
• Machine learning tries to find the best model by minimizing
errors.
• Example
Training Data
↓
Model
↓
Error
↓
Optimization
• Goal:Minimize prediction error.
 Perspectives and Issues in Machine
Learning
D. Learning from Experience
• According to Tom Mitchell:
• A computer program learns from experience E
with respect to task T and performance measure
P if its performance improves with experience.
• Example
– Spam Detection
– Task = Identify spam emails
– Experience = Thousands of emails
– Performance = Accuracy
 Perspectives and Issues in Machine
Learning
[Link] in Machine Learning Several practical challenges
affect model performance.
Issue 1: Insufficient Training Data
• Machine learning requires a large amount of data.
Example
• Suppose we train a disease prediction model using only
20 patient records.
• The model may fail on new patients.
Less Data
↓
Poor Learning
↓
Poor Prediction
 Perspectives and Issues in Machine
Learning
• Issue 2: Poor Quality Data
• If data contains errors, noise, or missing
values, the model becomes inaccurate.
• Example
Age Salary
25 30000
? 40000
30 NULL
 Perspectives and Issues in Machine
Learning
Issue 3: Overfitting
• The model memorizes training data instead of learning
patterns.
Characteristics
• Very high training accuracy
• Poor testing accuracy
• Example
– Training Accuracy = 99%
– Testing Accuracy = 65%
Training Data
↓
Memorization
↓
Overfitting
 Perspectives and Issues in Machine
Learning
Issue 4: Underfitting
• The model is too simple and cannot capture
data patterns.
Characteristics
• Poor training accuracy
• Poor testing accuracy
Example
• Using a straight line to fit curved data.
 Perspectives and Issues in Machine
Learning
Issue 5: High Dimensionality
• Too many features increase complexity.
Example
• Medical dataset:
– 5000 Patients
– 1000 Features
• Problems:
• Increased computation
• Risk of overfitting
• This is known as the Curse of Dimensionality.
 Perspectives and Issues in Machine
Learning
Issue 6: Feature Selection
• Not all features are useful.
Example
• For house price prediction:
Useful:
– Area
– Location
– Bedrooms
Not Useful:
– House color
• Choosing irrelevant features reduces accuracy.
 Perspectives and Issues in Machine
Learning
Issue 7: Computational Complexity
• Large datasets require more:
• Memory
• Storage
• Processing power
Example
• Deep Learning models may require GPUs.
Issue 8: Data Imbalance
• One class may have significantly more samples
than another.
Example
• Cancer Dataset
Class Samples
Normal 950
Cancer 50

The model may become biased toward the majority class.

 Perspectives and Issues in Machine
Learning
Issue 9: Bias and Variance
High Bias
• Model too simple.
High Bias
↓
Underfitting
High Variance
• Model too complex.
High Variance
↓
Overfitting
Issue 10: Interpretability
• Some models act like a "Black Box."
Example
• Deep Neural Networks
Difficult to explain:
– Why a prediction was made
– Which features influenced the decision
This is a major issue in:
– Healthcare
– Finance
– Law
 applications of machine learning in
different fields
1. Healthcare
• Machine learning can analyse X-rays, CT scans and MRI images to
detect diseases such as cancer, pneumonia and brain tumours. It can
also predict whether a patient is at risk of diabetes or heart disease.
2. Agriculture
• Images of crop leaves can be classified to identify diseases. Machine-
learning models can also predict crop yield using rainfall, temperature,
soil condition and fertilizer information.
3. Manufacturing
• Sensors installed on machines collect vibration, temperature and sound
data. Machine learning analyses these signals to predict equipment
failure before breakdown occurs. This is called predictive maintenance.
4. Banking
• Banks use machine learning to detect unusual transactions. For
example, if a credit card is suddenly used in an unfamiliar location for a
large purchase, the system may identify it as possible fraud.
 applications of machine learning in
different fields
5. E-commerce
• Online platforms recommend products based on a customer’s
previous searches, purchases, ratings and browsing behaviour.
6. Transportation
• Machine learning is used in autonomous vehicles to identify
pedestrians, traffic signs, road boundaries and nearby vehicles. It
also helps navigation systems find the shortest or fastest route.
7. Education
• Learning platforms can analyse student performance and provide
suitable study materials. Machine learning can also identify
students who may require additional academic support.
 applications of machine learning in
different fields
Field Applications
Medical Science Disease Detection, Medical Imaging
Agriculture Crop Disease Detection, Yield Prediction
Automobile Self-Driving Cars, Driver Assistance
Mining Mineral Exploration, Safety Monitoring
Finance Fraud Detection, Loan Approval
Education Student Performance Prediction
E-Commerce Recommendation Systems
Social Media Face Recognition, Spam Detection
Cybersecurity Malware Detection
Weather Forecasting
Manufacturing Predictive Maintenance
Entertainment Movie Recommendations
Transportation Traffic Prediction
Smart Cities Traffic & Energy Management
 Common Machine-Learning Tasks
• Classification: Predicting a category, such as diseased
or healthy.
• Regression: Predicting a numerical value, such as
house price or crop yield.
• Clustering: Grouping similar customers, products or
data points.
• Recommendation: Suggesting products, movies,
music or courses.
• Anomaly detection: Identifying fraud, faults or
unusual behaviour.
• Forecasting: Predicting future sales, weather, demand
or energy consumption.
 Concept of Bias and Variance
• Bias and variance are two major sources of
error in a machine learning model. They
determine how well the model performs on
unseen data.
 Concept of Bias and Variance
1. Bias
• Bias is the error caused by making overly
simple assumptions about the data.
• A model with high bias cannot capture the
actual relationship between input and output.
• High bias leads to underfitting.
 Concept of Bias and Variance
Characteristics of high-bias models:
• Poor performance on training data
• Poor performance on testing data
• Model is too simple
• Important patterns are not learned
 Concept of Bias and Variance
2. Variance
• Variance is the error caused by the model
being too sensitive to changes in the training
data.
• A high-variance model learns the training data
in excessive detail, including noise and
random fluctuations.
• High variance leads to overfitting.
 Concept of Bias and Variance
Characteristics of high-variance models:
• Very good performance on training data
• Poor performance on testing data
• Model is too complex
• Small changes in training data produce large
changes in predictions
 Bias–Variance Trade-off
• Increasing model complexity generally:
– Decreases bias
– Increases variance
• Decreasing model complexity generally:
– Increases bias
– Decreases variance
• The objective is to find a proper balance where both
bias and variance are reasonably low.
– Total Error = Bias2 + Variance + Irreducible Error
Where:
– Bias2 represents error due to incorrect assumptions.
– Variance represents error due to sensitivity to training data.
– Irreducible error is caused by noise in the data and cannot
be completely removed.
 Example: Predicting the Temperature

• Suppose four weather models predict tomorrow’s actual temperature,

which is 30°C.
• High bias, low variance: The model repeatedly predicts around 24°C.
Predictions are consistent but far from the correct value.
• Low bias, high variance: The model predicts 20°C, 28°C, 35°C and 37°C.
The average may be close to 30°C, but predictions vary greatly.
• High bias, high variance: The model predicts 15°C, 20°C, 25°C and 35°C.
Predictions are both inconsistent and generally far from the correct value.
• Low bias, low variance: The model predicts 29°C, 30°C, 30°C and 31°C.
Predictions are consistent and close to the actual temperature.
• Therefore, a good machine learning model should have low bias and low
variance.
• How to Reduce Bias
• Use a more complex model
• Add useful features
• Reduce excessive regularization
• Train the model properly
• Use nonlinear algorithms when required
• How to Reduce Variance
• Increase the amount of training data
• Simplify the model
• Apply regularization
• Use cross-validation
• Use ensemble methods such as random forests
• Apply early stopping in neural networks
• In summary:
Bias indicates that the model is too simple, while
variance indicates that the model is too sensitive to
the training data. A good model maintains a balance
between the two.
 Thank You
• Questions & Discussion