1.
Meaning of Supervised Learning
Supervised learning is a type of machine learning in which a model learns from labelled
data.
Labelled data contains:
Input data
Correct output
The model studies the relationship between the input and the correct output. After learning
this relationship, it predicts the output for new data.
Simple example
Suppose we want to teach a machine to identify whether an email is spam.
Email message Correct label
“You have won a free prize” Spam
“Your meeting is scheduled at 10 AM” Not spam
“Claim your reward immediately” Spam
The email message is the input, and “Spam” or “Not spam” is the label.
The model learns from these examples and later classifies a new email.
2. Why Is It Called Supervised Learning?
It is called supervised learning because the machine learns under supervision.
The correct answers are already available during training.
It is similar to a teacher teaching students:
The teacher gives questions.
The teacher also provides correct answers.
The students learn the relationship.
Later, the students answer new questions independently.
In supervised learning:
Data acts as the question.
Labels act as the correct answer.
The machine-learning algorithm acts as the student.
3. Basic Components of Supervised Learning
3.1 Input Features
Features are the variables used by the model to make predictions.
For house-price prediction, features may include:
Area of the house
Number of bedrooms
Location
Age of the house
Number of floors
Features are commonly represented by the symbol:
X
For example:
X ={ Area, Bedrooms, Location }
3.2 Target or Label
The target is the output that the model must predict.
For house-price prediction:
Y =House Price
For email classification:
Y =Spam or Not Spam
Other names for target include:
Label
Output variable
Dependent variable
Response variable
3.3 Training Data
Training data is used to teach the model.
Example:
Area Bedrooms Price
1,000 sq. ft. 2 ₹40 lakh
Area Bedrooms Price
1,500 sq. ft. 3 ₹60 lakh
2,000 sq. ft. 4 ₹85 lakh
The model studies how area and bedrooms affect price.
3.4 Model
A model is the mathematical relationship learned from the data.
For example, a simple model may learn:
House Price=30,000 × Area
Real machine-learning models normally learn more complex relationships.
3.5 Prediction
After training, the model predicts the output for new input data.
Example:
New house area: 1,700 sq. ft.
Bedrooms: 3
Predicted price: ₹68 lakh
This predicted value is called:
Y^
The symbol Y^ means predicted output.
4. Main Types of Supervised Learning
Supervised learning is mainly divided into:
1. Classification
2. Regression
5. Classification
Classification is used when the output is a category or class.
Examples
Spam or not spam
Pass or fail
Fraud or genuine
Diseased or healthy
Cat, dog or bird
Positive, negative or neutral review
Example dataset
Study hours Attendance Result
2 50% Fail
5 75% Pass
8 90% Pass
The model predicts whether a student will pass or fail.
5.1 Binary Classification
Binary classification has only two possible classes.
Examples:
Yes or no
Fraud or genuine
Pass or fail
Disease or no disease
5.2 Multiclass Classification
Multiclass classification has more than two possible classes.
Examples:
Cat, dog or horse
Red, blue, green or yellow
Grade A, B, C or D
Normal, pneumonia or tuberculosis
5.3 Multilabel Classification
In multilabel classification, one data item may belong to multiple classes at the same time.
Example:
A photograph may contain:
Dog
Car
Person
Tree
One image can therefore have several labels.
6. Regression
Regression is used when the output is a continuous numerical value.
Examples
House-price prediction
Temperature prediction
Sales forecasting
Salary prediction
Rainfall prediction
Electricity-demand prediction
Example dataset
Advertising expenditure Sales
₹10,000 ₹60,000
₹20,000 ₹90,000
₹30,000 ₹1,25,000
The model learns the relationship between advertising expenditure and sales.
It can then predict sales for a new advertising budget.
7. Classification and Regression Comparison
Basis Classification Regression
Output Category Numerical value
Example Spam or not spam Predicting sales
Nature of target Discrete Continuous
Basis Classification Regression
Common measure Accuracy Mean Squared Error
Example Logistic Regression, Decision Linear Regression, Random Forest
algorithms Tree Regression
8. Steps in Supervised Learning
Step 1: Define the Problem
The first step is to clearly identify what the system must predict.
Examples:
Predict whether a bank customer will default.
Predict the selling price of a house.
Identify whether an image contains a dog.
Predict whether a patient has a disease.
The problem must specify:
Inputs
Expected output
Type of problem
Success criteria
Step 2: Collect Labelled Data
The quality of supervised learning depends heavily on labelled data.
Example for loan-default prediction:
Income Loan amount Credit score Default
₹50,000 ₹2 lakh 750 No
₹25,000 ₹5 lakh 550 Yes
₹80,000 ₹3 lakh 810 No
The “Default” column is the label.
Step 3: Clean the Data
Real-world data may contain:
Missing values
Incorrect values
Duplicate records
Inconsistent formats
Extreme values
Irrelevant features
Example:
Age Salary
28 ₹40,000
Missing ₹55,000
250 ₹60,000
Age 250 is likely an incorrect value.
Data cleaning improves the quality of learning.
Step 4: Select Features and Target
Features are selected based on their usefulness.
For predicting student results:
Features
Study hours
Attendance
Previous marks
Assignment scores
Target
Pass or fail
Not every available variable should be used. Irrelevant variables may reduce performance.
Step 5: Split the Data
The data is generally divided into:
Training data
Validation data
Test data
Example:
70% training
15% validation
15% test
Training data
Used to teach the model.
Validation data
Used to tune and compare models.
Test data
Used for final evaluation.
Step 6: Select an Algorithm
The algorithm depends on the problem.
Classification algorithms
Logistic Regression
Decision Tree
Random Forest
Support Vector Machine
K-Nearest Neighbours
Naive Bayes
Neural Networks
Regression algorithms
Linear Regression
Polynomial Regression
Decision Tree Regression
Random Forest Regression
Support Vector Regression
Neural Networks
Step 7: Train the Model
During training, the model:
1. Receives input data.
2. Makes predictions.
3. Compares predictions with correct outputs.
4. Calculates the error.
5. Adjusts itself to reduce the error.
This process may be repeated many times.
Step 8: Validate and Tune the Model
The model is tested on validation data.
The developer may change:
Learning rate
Number of trees
Tree depth
Number of neighbours
Regularisation strength
Number of neural-network layers
These settings are called hyperparameters.
Step 9: Test the Final Model
After selecting the best model, it is evaluated on test data.
The test data must not be used during training or tuning.
The test result provides an estimate of real-world performance.
Step 10: Deploy and Monitor
After testing, the model can be used in a real system.
Examples:
Fraud detection in a banking application
Disease prediction in a hospital system
Product recommendation on an e-commerce platform
Spam detection in email software
After deployment, the model should be monitored because real-world data may change.
9. How a Supervised Model Learns
Suppose the correct answer is:
Y =100
The model predicts:
Y^ =80
The error is:
Error=Y −Y^ Error=100−80=20
The model adjusts its parameters to reduce this error.
After adjustment, the next prediction may become:
Y^ =95
The error is now:
100−95=5
The model continues learning until the error becomes sufficiently small.
10. Loss Function
A loss function measures how wrong the model's prediction is.
Lower loss generally means better predictions.
For regression
Common loss functions include:
Mean Squared Error
Mean Absolute Error
Root Mean Squared Error
For classification
Common loss functions include:
Binary Cross-Entropy
Categorical Cross-Entropy
Hinge Loss
The training process tries to minimise the loss function.
11. Parameters and Hyperparameters
Parameters
Parameters are learned automatically from training data.
Examples:
Slope in linear regression
Intercept in linear regression
Feature weights in logistic regression
Split values in a decision tree
Hyperparameters
Hyperparameters are selected by the developer.
Examples:
Maximum depth of a tree
Number of trees
Number of neighbours
Learning rate
Number of hidden layers
Parameters Hyperparameters
Learned from data Selected before or during training
Adjusted automatically Tuned by the developer
Example: regression coefficient Example: tree depth
12. Important Supervised Learning Algorithms
12.1 Linear Regression
Linear regression predicts a numerical value.
It finds a linear relationship between input and output.
The basic equation is:
Y =mX + c
Where:
Y = predicted output
X = input
m = slope
c = intercept
Example
Predicting salary based on years of experience.
Experience Salary
1 year ₹25,000
3 years ₹40,000
5 years ₹60,000
Linear regression tries to draw the best-fitting line through the data.
Applications
Price prediction
Sales forecasting
Demand forecasting
Salary estimation
12.2 Logistic Regression
Despite its name, logistic regression is mainly used for classification.
It predicts the probability of a class.
Example:
P ( Default )=0.80
This means there is an 80% estimated probability of default.
A threshold is used:
Probability above 0.50: Default
Probability below 0.50: No default
Applications
Disease detection
Customer churn
Fraud detection
Pass or fail prediction
12.3 Decision Tree
A decision tree makes decisions using a series of questions.
Example:
Is attendance above 75%?
|
Yes No
| |
Study hours > 4? Fail
|
Yes No
| |
Pass Fail
Advantages
Easy to understand
Easy to explain
Works with numerical and categorical data
Requires limited data preparation
Limitations
Can overfit
Small changes in data may change the tree
12.4 Random Forest
Random Forest combines many decision trees.
Each tree makes a prediction, and the final answer is selected by:
Majority voting for classification
Averaging for regression
Example:
Tree Prediction
Tree 1 Pass
Tree 2 Pass
Tree 3 Fail
Tree 4 Pass
Final prediction: Pass
Advantages
Usually more accurate than one decision tree
Reduces overfitting
Handles complex data
Works for classification and regression
Limitation
It is more difficult to explain than a single decision tree.
12.5 K-Nearest Neighbours
K-Nearest Neighbours predicts an output based on the most similar nearby records.
Suppose K=3.
For a new student, the three nearest students have results:
Pass
Pass
Fail
The final prediction is Pass.
Advantages
Simple to understand
No complex training process
Useful for small datasets
Limitations
Slow for large datasets
Sensitive to feature scaling
Choice of K affects results
12.6 Support Vector Machine
Support Vector Machine finds the best boundary between classes.
For example, it may separate:
Fraudulent transactions
Genuine transactions
The best boundary is the one that creates the largest possible gap between the classes.
Advantages
Effective in high-dimensional data
Works well with clear class boundaries
Can model nonlinear relationships using kernels
Limitations
Can be slow for large datasets
Difficult to explain
Requires careful parameter tuning
12.7 Naive Bayes
Naive Bayes uses probability to classify data.
It is based on Bayes' theorem.
It assumes that the input features are independent of one another.
Applications
Spam detection
Text classification
Sentiment analysis
Document categorisation
Advantages
Fast
Works well with text
Requires relatively little training data
Limitation
Its independence assumption may not always be realistic.
12.8 Neural Networks
Neural networks are inspired by the structure of the human brain.
They consist of:
Input layer
Hidden layers
Output layer
Each connection has a weight.
During training, the network adjusts these weights to reduce prediction error.
Applications
Image recognition
Speech recognition
Medical diagnosis
Natural language processing
Complex prediction problems
Advantages
Learns complex patterns
Suitable for large datasets
High predictive power
Limitations
Requires more data
Requires greater computational power
Often difficult to explain
13. Evaluation of Classification Models
13.1 Confusion Matrix
A confusion matrix compares actual and predicted classes.
Predicted Positive Predicted Negative
Actual Positive True Positive False Negative
Predicted Positive Predicted Negative
Actual Negative False Positive True Negative
True Positive
The model predicts positive, and the actual result is positive.
Example: A sick patient is correctly identified as sick.
True Negative
The model predicts negative, and the actual result is negative.