Introduction to Classi cation

Think of classi cation as a system that helps a computer make decisions—just like
we do every day.
For example:

• When you get an email, Gmail decides: Is it spam or not spam?

• A bank decides: Will the customer repay a loan or default?

• YouTube predicts: Will you click this video or ignore it?

All these are classi cation problems.

In simple words:

👉 Classi cation means predicting a category or class for a given input.

Just like we classify fruits into apples, oranges, and bananas, in data science we
classify customers, emails, transactions, and so on.”

2. Types of Classi cation

“Most commonly, classi cation is binary, which means only two possible outcomes:

• Yes / No

• 1/0

• Click / No Click

• Fraud / Not Fraud

• Default / Paid-off

But classi cation is not limited to two categories. Some problems have three or
more classes.

Example:
When a customer’s contract ends, there may be three outcomes:

• 0 → Renew long-term

• 1 → Shift to month-to-month
fi
fi
fi
fi
fi
fi
fi
 • 2 → Leave the service (churn)

This is called multiclass classi cation.

Sometimes we break a multiclass problem into multiple smaller binary problems,

because binary problems are easier and more accurate.

This idea is used in many real-world models.”

3. Why Probability Is Important

“When we classify, it’s not enough to say default or paid.
Sometimes we want to know:

👉 How likely is the customer to default?

This likelihood is called the propensity score.

Models such as logistic regression, Naive Bayes, and LDA can give us:

• The predicted class (0 or 1)

• The probability of being in each class

To convert probabilities to nal class, we use a cutoff value.

Example:

• If probability > 0.5 → Predict 1

• Otherwise → Predict 0

Cutoff changes how strict we are:

• Higher cutoff → Fewer 1’s

• Lower cutoff → More 1’s”

4. Introducing Bayes’ Theorem

“Before learning Naive Bayes, we must understand Bayes’ theorem.

Let me explain Bayes theorem in a way you will never forget.

fi
fi
Imagine you are a doctor.
A patient comes with a symptom: headache.
You want to know:

👉 What is the probability that this headache is due to u?

There are two probabilities involved:

1. How common is u in general? (Prior probability)

2. How likely is a headache if the person already has u? (Conditional

probability)

Bayes’ theorem combines both to give:

👉 Probability of u given headache

Mathematically:

P(A|B) = P(B|A)*P(A)/P(B)

In words:

Posterior = (Likelihood × Prior) / Evidence

This simple formula is the heart of Naive Bayes.”

5. Prior Probability
“Now students often ask:

👉 Sir, what is prior probability and how do we calculate it?

Simple:

Prior probability =
How many records belong to the class / Total records

Example:

Out of 100 students:

fl
fl
fl
fl
 • 40 passed

• 60 failed

Then:

P(Pass) = 40/100 = 0.40

P(Fail) = 60/100 = 0.60

That’s it!
Prior = simple counting.”

6. Naive Bayes – Why “Naive”?

“Bayes theorem is perfect, but there is a problem.

Real-life data has many columns:

• age

• income

• marks

• attendance

• category

• gender
and so on.

To apply exact Bayes:

We would need to nd another record with exactly the same values.

This NEVER happens.

So Naive Bayes makes a simple assumption:

👉 All predictors are independent given the class.

fi
This assumption is not true in real life, but surprisingly, the algorithm works very
well!”

7. Naive Bayes – Very Simple Example with

Calculations
“Let us take the simplest possible example.

We want to predict Pass or Fail

based on two predictors:

1. Studied? (Yes / No)

2. Attendance (High / Low)

Here is the small dataset we will use:

Studied Attendance Result

Yes High Pass
Yes High Pass
No High Fail
Yes Low Pass
No Low Fail
No High Fail

Total = 6 students

Step 1: Prior Probabilities

Count Pass = 3
Count Fail = 3

P(Pass) = 3/6 = 0.5

P(Fail) = 3/6 = 0.5

Step 2: Conditional Probabilities

A. Predictor: Studied = Yes

Among Pass:

• Yes = 3 out of 3

P(Studied = Yes | Pass) = 3/3 = 1.0

Among Fail:

• Yes = 0 out of 3

P(Studied = Yes | Fail) = 0/3 = 0

B. Predictor: Attendance = High

Among Pass:

• High = 2 out of 3

P(High | Pass) = 2/3

Among Fail:

• High = 2 out of 3

P(High | Fail) = 2/3

Step 3: Predict for a NEW student

New student:

• Studied = Yes

• Attendance = High
We calculate probability of Pass and Fail.

Prediction

Since:

• Pass = 0.333

• Fail = 0

👉 Prediction: PASS

This example is perfect for understanding Naive Bayes because the math is simple
and clear.”
 8. Handling Numeric Predictors
“When predictors are numbers like age, marks, height, or income, Naive Bayes cannot simply
count.

So we do one of two things:

Method 1: Convert to categories (binning)

Example:

Marks:

• <40 = Low

• 40–70 = Medium

• 70 = High

Then Naive Bayes works as usual.

Method 2: Gaussian Naive Bayes

Assume numeric values follow the normal distribution.

For each class:

• nd mean

• nd standard deviation

• plug into the Gaussian formula

This method is used in Python and ML libraries.”

Example: Using Gaussian Naive Bayes for

Numeric Predictors
Problem

We want to predict whether a student will Pass or Fail based on Marks (a numeric
value).

Training Data
fi
fi
We take a very small dataset:

Marks Result
30 Fail
35 Fail
40 Fail
70 Pass
75 Pass
80 Pass

Total = 6 students

• Fail = 3

• Pass = 3

STEP 1: Calculate Prior Probabilities

P(Pass) = 3/6 = 0.5

P(Fail) = 3/6 = 0.5

STEP 2: Calculate Mean and Standard

Deviation
STEP 3: Predict for a NEW student
A new student has:

👉 Marks = 60

We calculate:

• Probability of getting marks = 60 if the student is Pass

• Probability of getting marks = 60 if the student is Fail

STEP 4: Calculate Probabilities

STEP 5: Multiply with Priors

FINAL DECISION
👉 Probability of Fail ≈ 0
👉 Probability of Pass ≈ 0.0006

Even though both numbers are small, Pass is much higher.

✔ Prediction: The student will PASS