INTRODUCTION TO DATA SCIENCE
(WEEK-4)
PROBABILITY AND STATISTICAL CONCEPT IN DATA
SCIENCE
Lecturer
Hira Faisal
Introduction to Bayes Theorem
Bayes Theorem
► Bayes' Theorem is a way to calculate the probability of an event
happening based on prior knowledge of conditions related to
the event.
► It's a fundamental concept in probability theory and is widely
used in statistics, machine learning, and data science.
The Formula
► The formula for Bayes' Theorem is:
Naming the Terms in the Theorem
► P(A∣B) is the posterior probability: the probability of event
A happening given that B is true.
► P(B∣A) is the likelihood: the probability of event B happening
given that A is true.
► P(A) is the prior probability: the initial probability of event
A happening without considering any evidence.
► P(B) is the marginal likelihood or evidence: the probability
of event B happening under all possible conditions (true and
false).
Problem 1
► Let’s work on a simple NLP problem with Bayes Theorem. By
using NLP, I can detect spam e-mails in my inbox.
► Assume that the word ‘offer’ occurs in 80% of the spam messages
in my account.
► Also, let’s assume ‘offer’ occurs in 10% of my desired e-mails.
► If 30% of the received e-mails are considered as a spam, and I will
receive a new message which contains ‘offer’,
► What is the probability that it is spam?
Solution with Bayes’ Equation
►
Calculations
Problem 2
► We want to solve one more example from a popular topic as
Covid-19.
► As you know, Covid-19 tests are common nowadays, but some
results of tests are not true.
► Let’s assume; a diagnostic test has 99% accuracy and 60% of all
people have Covid-19.
► If a patient tests positive, what is the probability that they actually
have the disease?
Problem 3
► You run an online store and want to determine that a visitor will make a
purchase based on whether they click on a specific promotional banner.
• You know that 10% of all visitors to your website end up making a
purchase.
• Among those who make a purchase, 40% click on the promotional
banner.
• However, only 20% of visitors who don’t make a purchase click on the
promotional banner.
P(Banner Click)= (P(Banner Click ∣ Purchase)×P(Purchase))
+
(P(Banner Click ∣ No Purchase)×P(No Purchase))
P(Banner Click)=(0.4×0.1)+(0.2×0.9)=0.04+0.18=0.22
Naive Bayes Classification
Naive Bayes Classification
► Naive Bayes Classification: A simplified version of Bayes
Theorem known as "Naive Bayes Classification" is employed.
► This variant makes certain simplifying assumptions about the
independence of features, which can significantly reduce
computation time and costs in machine learning tasks.
Naive Bayes Classifier
► Let’s see how the Naive Bayes Classifier can be applied as a classification algorithm.
► Consider a general example: X is a vector consisting of ‘n’ attributes, that is, X = {x1, x2,
x3, …, xn}.
Example: Predictively Classifying
Customers of a Bookstore
We have the following dataset from a bookstore:
Age Income Student Credit_Rating Buys_Book
Youth High No Fair No
Youth High No Excellent No
Middle_aged High No Fair Yes
Senior Medium No Fair Yes
Senior Low Yes Fair Yes
Senior Low Yes Excellent No
Middle_aged Low Yes Excellent Yes
Youth Medium No Fair No
Youth Low Yes Fair Yes
Senior Medium Yes Fair Yes
Youth Medium Yes Excellent Yes
Middle_aged Medium No Excellent Yes
Middle_aged High Yes Fair Yes
Senior Medium No Excellent No
We have attributes like age, income, student, and credit rating.
Our class, buys_book, has two outcomes: Yes or No.
Our goal is to classify based on the following attributes:
X = {age = youth, student = yes, income = medium, credit_rating = fair}.
Hence, P(buys_book = yes) = 9/14 = 0.643
P(buys_book = no) = 5/14 = 0.357
P(age = youth | buys_book = yes) = 2/9 = 0.222
P(age = youth | buys_book = no) =3/5 = 0.600
P(income = medium | buys_book = yes) = 4/9 = 0.444
P(income = medium | buys_book = no) = 2/5 = 0.400
P(student = yes | buys_book = yes) = 6/9 = 0.667
P(student = yes | buys_book = no) = 1/5 = 0.200
P(credit_rating = fair | buys_book = yes) = 6/9 = 0.667
P(credit_rating = fair | buys_book = no) = 2/5 = 0.400
Using the above-calculated probabilities, we have
P(X | buys_book = yes) = 0.222 x 0.444 x 0.667 x 0.667 = 0.044
Similarly,
P(X | buys_book = no) = 0.600 x 0.400 x 0.200 x 0.400 = 0.019
Which class does Ci provide the maximum P(X|Ci)*P(Ci)? We compute:
P(X | buys_book = yes)* P(buys_book = yes) = 0.044 x 0.643 = 0.028
P(X | buys_book = no)* P(buys_book = no) = 0.019 x 0.357 = 0.007
Comparing the above two, since 0.028 > 0.007, the Naive Bayes Classifier predicts
that the customer with the above-mentioned attributes will buy a book.
Estimate conditional probability of given each attributes for the species classes :
{M, H} using the given data in the table: