Classification
Defining Classification
Classification is defined as the process of recognition, understanding, and
grouping of objects and ideas into preset categories a.k.a “sub-populations.”
Classification algorithms used in machine learning utilize input training data for the
purpose of predicting the likelihood or probability that the data that follows will fall
into one of the predetermined categories.
In short, classification is a form of “pattern recognition”. Here, classification
algorithms applied to the training data find the same pattern (similar number
sequences, words or sentiments, and the like) in future data sets.
Classification Real-World Applications
Email Spam Filtering - Classification algorithms are used to identify and filter out
unwanted spam emails by analyzing email content and characteristics.
Fraud Detection - Financial institutions use classification to identify fraudulent
transactions by analyzing patterns and anomalies in data, protecting against financial
losses.
Medical Diagnosis - Classification models assist in diagnosing diseases by analyzing
patient data such as symptoms and test results, aiding healthcare professionals in
making accurate and timely diagnoses.
Sentiment Analysis - Text classification is used to determine the emotional tone of
text, such as product reviews or customer feedback, identifying whether the
sentiment is positive, negative, or neutral.
Credit Risk Assessment - Banks and lenders employ classification to predict the
likelihood of a loan applicant defaulting, using factors like credit score and income.
Types of Classification
Binary Classification - This is the
simplest kind of classification. In binary
classification, the goal is to sort the data
into two distinct categories.
Multiclass Classification - Here, instead
of just two categories, the data needs to
be sorted into more than two
categories. The model picks the one that
best matches the input.
Multi-Label Classification - In multi-
label classification single piece of data
can belong to multiple categories at
once. Unlike multiclass classification
where each data point belongs to only one
class, multi-label classification allows
datapoints to belong to multiple
classes.
Activity – Determine if it is binary, multiclass or multilabel
• You’re building a system that detects whether an email is spam or not.
• You want a weather app to predict if tomorrow will be sunny, rainy, or snowy.
• You’re tagging a movie with all applicable genres, such as action, comedy, and drama.
• A hospital system checks whether a patient has diabetes based on lab results.
• You're designing a program that identifies handwritten digits from 0 to 9.
• A social media platform classifies a post as related to one or more topics like politics, health,
or sports.
• A bank uses a model to determine if a customer's loan application should be approved or
not.
• You're creating an app that identifies animals as either a dog, cat, rabbit, or parrot.
• A music app predicts all the genres that might apply to a given song, like pop, jazz, or rock.
• You’re analyzing customer reviews to classify the sentiment as positive, negative, or
neutral.
Classification Evaluation Metrics
A confusion matrix is a performance measurement tool for classification problems.
It compares the predicted labels from a model with the actual (true) labels to
evaluate how well the model is performing. It is mainly used for binary and multiclass
classification tasks.
Predicted: Positive Predicted: Negative
Actual: Positive True Positive (TP) False Negative (FN)
Actual: Negative False Positive (FP) True Negative (TN)
Classification Evaluation Metrics
Accuracy – we get accuracy by answering Accuracy =
the question “out of the predictions made
by the model, what percentage is
correct?”
Precision =
Precision – we get precision by answering
the question “out of all the YES
predictions, how many of them were
Sensitivity =
correct?”
Recall(Sensitivity) – it aims to answer the
question “how good was the model at
Specificity =
predicting REAL YES events?” which can
be considered as flip of the precision.
Recall(Specificity) – it aims to answer the
question “how good was the model at
predicting REAL NO events?”
Classification Evaluation Metrics
F1 Score – Sometimes used when dealing with imbalanced data set, meaning that
there are more of one class or label than there are of the other. It corresponds to the
harmonic mean of the precision and recall.
Precision = 2 x
Classification Evaluation Metrics
Example - Suppose you built a classifier to detect spam emails.
Predicted: Spam Predicted: Not Spam
Actual: Spam 70 (TP) 10 (FN)
Actual: Not Spam 5 (FP) 115 (TN)
Accuracy = Precision =
Recall(Sensitivity) = Recall(Specificity) =
F1 Score = 2 x
Logistic Regression
Logistic Regression is a supervised machine learning algorithm used for
classification problems. Unlike linear regression which predicts continuous values it
predicts the probability that an input belongs to a specific class. It is used for binary
classification where the output can be one of two possible categories such as
Yes/No, True/False or 0/1.
It uses sigmoid function to convert inputs into a probability value between 0 and 1.
Types Logistic Regression
Binomial Logistic Regression - This type is used when the dependent variable has
only two possible categories. Examples include Yes/No, Pass/Fail or 0/1.
Multinomial Logistic Regression - This is used when the dependent variable has
three or more possible categories that are not ordered. For example, classifying
animals into categories like "cat," "dog" or "sheep."
Ordinal Logistic Regression - This type applies when the dependent variable has
three or more categories with a natural order or ranking. Examples include ratings
like "low," "medium" and "high."
Assumptions of Logistic Regression
Independent observations - Each data point is assumed to be independent of the
others means there should be no correlation or dependence between the input
samples.
Binary dependent variables - It takes the assumption that the dependent variable
must be binary, means it can take only two values.
Linearity relationship between independent variables and log odds - The
model assumes a linear relationship between the independent variables and the log
odds of the dependent variable which means the predictors affect the log odds in a
linear way.
No outliers - The dataset should not contain extreme outliers as they can distort the
estimation of the logistic regression coefficients.
Large sample size - It requires a sufficiently large sample size to produce reliable
and stable results.
Understanding Sigmoid Function
1. The sigmoid function is an important part of logistic regression which is used to
convert the raw output of the model into a probability value between 0 and 1.
2. This function takes any real number and maps it into the range 0 to 1 forming an
"S" shaped curve called the sigmoid curve or logistic curve. Because probabilities
must lie between 0 and 1, the sigmoid function is perfect for this purpose.
3. In logistic regression, we use a threshold value usually 0.5 to decide the class label.
• If the sigmoid output is same or above the threshold, the input is classified as Class
1.
• If it is below the threshold, the input is classified as Class 0.
Understanding Sigmoid Function
Quiz
Quiz
1. Which of the following best defines classification in machine
learning?
a) Grouping data based on similarity without labels
b) Predicting continuous outcomes from input variables
c) Predicting categorical outcomes using labeled data
d) Ranking data in ascending or descending order
Quiz
2. A bank wants to decide if a loan application should be
approved or not. Which type of classification is being applied?
a) Multiclass Classification
b) Multi-label Classification
c) Binary Classification
d) Clustering
Quiz
[Link] a system classifies a product into multiple categories, what
type of classification is used?
a) Multiclass Classification
b) Multi-label Classification
c) Binary Classification
d) Clustering
Quiz
4. Accuracy answers the question:
a) Out of all predictions, what percentage was correct?
b) How good was the model at predicting real YES events?
c) How well the model predicts real NO events
d) Out of all predicted YES, how many were correct?
Quiz
[Link] measures:
a) Correct YES predictions out of all actual YES events
b) Correct NO predictions out of all predictions
c) Correct YES predictions out of all predicted YES
d) Correct predictions overall
Quiz
[Link] tool compares predicted labels to actual labels to
evaluate performance?
a) ROC Curve
b) R2 Score
c) Regression Line
d) Confusion Matrix
Quiz
7. Which function is used in logistic regression to map outputs
between 0 and 1?
a) Sigmoid function
b) Exponential function
c) Linear function
d) Hyperbolic tangent
Quiz
[Link] multinomial logistic regression, the dependent variable has:
a) Two possible categories
b) Multiple ordered categories
c) Three or more unordered categories
d) Continuous values
Quiz
9. Which of the following is not an assumption of logistic
regression?
a) Independent observations
b) Binary dependent variables
c) Linearity between independent variables and log odds
d) Dependent variable must be continuous
Quiz
[Link] the sigmoid output is 0.72 and the threshold is 0.5, how will
the model classify the input?
a) Class 1
b) Class 0
c) Neutral
d) Cannot be determined