THIS PRESENTATION IS ABOUT

 Concept of Machine Learning

 Applications of Machine Learning
 Key elements of Machine Learning
 Types of Learning
 Statistical Description of Data and Data Visualization
 Introduction of Machine Learning algorithms
 Mathematical foundations
 Linear algebra,
 Statistical Learning
 Bayesian Method,
 The Naïve Bayes Classifier.
 Vectorization, Matrices and Vectors,
 T RADITIONAL A LGORITHM VS
M ACHINE L EARNING A LGORITHMS
Traditional Algorithms Data Program
Comput
er

Output

Machine Learning Algorithms Data Computer

Example/
Output

Program /
Model
 History Of Machine Learning

• Alan Turing creates the “Turing Test” to determine if a

1950 computer has real intelligence.

• To pass the test, a computer must be able to fool a human
into believing it is also human.

• Arthur Samuel wrote the first computer learning program.

1952 • The program was the game of checkers, and the computer
improved at the game the more it played

• Frank Rosenblatt designed the first neural network for

1957 computers (the perceptron), which simulate the thought
processes of the human brain.
 • The “nearest neighbor” algorithm was written, allowing
computers to begin using very basic pattern recognition.
1967 • This could be used to map a route for traveling salesmen,
starting at a random city but ensuring they visit all cities
during a short tour.

1979 • Students at Stanford University invent the “Stanford Cart”

which can navigate obstacles in a room on its own.

• Work on machine learning shifts from a knowledge-driven

approach to a data-driven approach.
1990s • Scientists begin creating programs for computers to
analyze large amounts of data and draw conclusions — or
“learn” — from the results.
1997 • IBM’s Deep Blue beats the world champion at chess.

• Geoffrey Hinton coins the term “deep learning” to

2006 explain new algorithms that let computers “see” and
distinguish objects and text in images and videos.

2011 • Deep neural network can learn to discover and

categorize objects much the way a cat does.

2012 • Google’s X Lab develops a machine learning algorithm that

is able to autonomously browse YouTube videos to identify
the videos that contain cats.
 • Facebook develops DeepFace, a software
2014 algorithm that is able to recognize or verify
individuals on photos to the same level as
humans can.

2015 • Amazon launches its own machine learning

platform.

2020 • ……………………
 Enables a machine to think

Statistical took to
explore data Artificial Intelligence
Brain like
functioning
ML
• Supervised • Artificial NN
Learning • Convolution NN
Deep • Recurrent NN
• Unsupervised Learning
Learning

• Semi-supervised
Learning
 Experience/ data
Problem /task

Model

Learner Reasoner

Answer/ performance
Background knowledge to help the system

Fig. Schematic diagram of a machine learning system

 W HAT IS M ACHINE L EARNING
 A branch of artificial intelligence, concerned with the
design and development of algorithms that allow computers
to evolve behaviors based on empirical data.

 As intelligence requires knowledge, it is necessary for the

computers to acquire knowledge.

Machine learning, a branch of artificial intelligence,

concerns the construction and study of systems that can
learn from data.
 M ACHINE L EARNING IS …

Machine learning is programming computers to optimize a

performance criterion using example data or past experience.
-- Ethem Alpaydin

The goal of machine learning is to develop methods that can

automatically detect patterns in data, and then to use the
uncovered patterns to predict future data or other outcomes of
interest.
-- Kevin P. Murphy

The field of pattern recognition is concerned with the

automatic discovery of regularities in data through the use of
computer algorithms and with the use of these regularities to
take actions.
-- Christopher M. Bishop
The unstructured growth of data
 Related Disciplines

decision game
theory theory
AI control
theory
information
biological theory
evolution
Machine
probability Learning
& philosophy
statistics
optimization
Data Mining statistical psychology
mechanics

computational
complexity
theory neurophysiology
 KEY ELEMENTS OF MACHINE
LEARNING

 Representation
 Evaluation
 Optimization
 KEY ELEMENTS OF MACHINE
LEARNING

 Representation
 How to represent knowledge?
 Examples:
o Decision trees
o Sets of rules
o Instances
o Graphical models
o Neural networks
o Support vector machines
o Model ensembles
 KEY ELEMENTS OF MACHINE
LEARNING

Evaluation:
 The way to evaluate candidate programs
(hypotheses).
 Examples
o Accuracy
o Precision and recall
o Squared error
o Likelihood
o Posterior probability
o Cost, margin
o Entropy k-l divergence
 KEY ELEMENTS OF MACHINE
LEARNING

 Optimization:
The way candidate programs are generated known
as the search process.
Example
 Combinatorial optimization Consists of finding an optimal
solution/object from a finite set of solutions/object.
 Convex optimization Many classes of convex optimization
problems admit polynomial-time algorithms, whereas
mathematical optimization is in general NP-hard.
 Constrained optimization Process of optimizing an
objective function with respect to some variables in the
presence of constraints on those variables.
T YPES OF LEARNING
 TYPES OF LEARNING

• Supervised (inductive) learning

– Training data includes desired outputs
• Unsupervised learning
– Training data does not include desired outputs
• Semi-supervised learning
– Training data includes a few desired outputs
• Reinforcement learning
– Rewards from sequence of actions
TYPES OF LEARNING
 SUPERVISED LEARNING

 Also called inductive learning

 Model relationships and dependencies between the
target prediction output and the input features such
that we can predict the output values for new data
based on those relationships which it learned from the
previous data sets.
 The training data includes both Inputs and
Labels(Targets)
 For example addition of two numbers a=5,b=6 result
=11, Inputs are 5,6 and Target is 11.
 SUPERVISED LEARNING:
CLASSIFICATION
 A classification problem is when the output variable is a category
or a group, such as “black” or “white” or “spam” and “no spam”.
 SUPERVISED LEARNING:
REGRESSION

A regression problem
is when the output Supervised learning
variable is a real
value, such as
“Rupees” or “height.”
 UNSUPERVISED LEARNING

 Training data does not include desired outputs.

 Here training data is not structured (contains noisy data,
unknown data and etc..)
 Example: A random articles from different pages
 UNSUPERVISED LEARNING
Association: An association rule learning problem is where you want to
discover rules that describe large portions of your data, such as “people that
buy X also tend to buy Y”.
Clustering: A clustering problem is where you want to discover the inherent
groupings in the data, such as grouping customers by purchasing behavior such
as “Blue color is mostly liked by boys.”
 SEMI-SUPERVISED LEARNING
• Training data does not include all desired outputs.
• Models are trained on a combination of labeled and
unlabeled data.
• Benefit: The process of labeling massive amounts of data
for supervised learning is often prohibitively time-
consuming and expensive.

Supervised Learning Semi-supervised learning

 REINFORCEMENT LEARNING
• It is a behavioral learning model.
• It receives feedback from the analysis of the data so the user is guided to the
best outcome.
 Input: The input should be an initial state from which the model will start
 Output: There are many possible output as there are variety of solution to a
particular problem
 Training: The training is based upon the input, The model will return a
state and the user will decide to reward or punish the model based on its
output.
• The model keeps continues to learn.
• The best solution is decided based on the maximum reward.
REINFORCEMENT VS SUPERVISED LEARNING
TYPES OF LEARNING
STATISTICAL D ESCRIPTION &
DATA V ISUALIZATION
ACTUALLY
EXTRACTING INFORMATION FROM DATA
 S TATISTICAL L EARNING
 Descriptive Statistical Analysis helps you to
understand your data and is a very important part of
Machine Learning.
 This is due to Machine Learning being all about
making predictions.
 On the other hand, statistics is all about drawing
conclusions from data, which is a necessary initial
step.

So, to make predictions you need to draw

conclusions first.
 S TATISTICAL L EARNING
Statistics is a branch of mathematics that deals
with collecting, interpreting, organization and
interpretation of data.
Within statistics, there are two main categories:
Descriptive Statistics: In Descriptive Statistics your are
describing, presenting, summarizing and organizing
your data (population), either through numerical
calculations or graphs or tables.

Inferential statistics: Inferential Statistics are produced

by more complex mathematical calculations, and
allow us to infer trends and make assumptions and
predictions about a population based on a study of a
sample taken from it.
 N ORMAL D ISTRIBUTION
A Normal Distribution is given if your data is
symmetrical, bell-shaped, centered and unimodal.
 C ENTRAL T ENDENCY
( MEAN , MODE , MEDIAN )
 In statistics we have to deal with the mean, mode
and the median. These are also called the
“Central Tendency”.
 The mean is simply the average.
 The mode is the value or category that occurs
most often within the data.
 The median is the “middle” value or midpoint
in your data.
S TATISTICAL D ESCRIPTION OF D ATA
 Mean: It is given by the total of the values of the samples
divided by the number of samples.

Notation : Let x1 , x2, ...xn are n observations of a variable

x. Then the mean of this variable,
n

x  x  ...  xn x i
x 1 2  i 1
n n

 Mean of 20, 30, 40 is (20+30+40)/3 = 30

 S TATISTICAL D ESCRIPTION OF
D ATA
 Median
Let x1 , x2, ...xn are n observations of a variable x.
Then the median of this variable is calculated as:
1. Sort the observations
2. If n is odd, the middle element is considered as median.
3. If n is even, average of the two middle values are taken as the median

If x = [23, 40, 6, 74, 38, 1, 70],

sorted_x = [1, 6, 23, 38, 40, 70, 74]

Median= 38
 S TATISTICAL D ESCRIPTION OF
D ATA
 Mode:

It represents the most common value in a data

set. Mode is most useful when you need to
understand clustering or number of ‘hits’.
 M EASURES OF VARIABILITY
The most popular variability measures are the range,
interquartile range (IQR), variance, and standard
deviation.
 The range describes the difference between the
largest and the smallest points in your data.
 The interquartile range (IQR) is a measure of statistical
dispersion between upper (75th) and lower (25th)
quartiles.
While the range measures where the beginning and end of your data point
are, the interquartile range is a measure of where the majority of the values
lie.
 VARIANCE AND S TANDARD D EVIATION
 The Standard Deviation and the Variance also
measure, like the Range and IQR, how spread apart our
data is (e.g the dispersion). Therefore they are both
derived from the mean.
 The variance is computed by finding the difference
between every data point and the mean, squaring
them, summing them up and then taking the average
of those numbers.
 Standard Deviation is used more often because it is
in the original unit. It is simply the square root of the
variance and because of that, it is returned to the
original unit of measurement.
VARIANCE AND S TANDARD
D EVIATION
 M ODALITY
The modality of a distribution is determined by the
number of peaks it contains. Most distributions have
only one peak but it is possible that you encounter
distributions with two or more peaks.
 S KEWNESS
Skewness is a measurement of the symmetry of a distribution. It
describes how much a distribution differs from a normal distribution,
either to the left or to the right. The skewness value can be either
positive, negative or zero.

Note that a perfect normal distribution would have a skewness of zero

because the mean equals the median.
S TATISTICAL F EATURES

A Box Plot
 D ATA V ISUALIZATION

 Data visualization is a very important part of data analysis.

You can use it to explore your data.
 If you understand your data well, you’ll have a better chance
to find some insights.
 Finally, when you find any insights, you can use
visualizations again to be able to share your findings with
other people.
 B ASIC V ISUALIZATION R ULES
 Before we look at some kinds of plots, we’ll introduce
some basic rules. Those rules help us make nice and
informative plots instead of confusing ones.
 The first step is to choose the appropriate plot type. If there are
various options, we can try to compare them, and choose the
one that fits our model the best.
 Second, when we choose your type of plot, one of the most
important things is to label your axis. If we don’t do this, the
plot is not informative enough.
 Third, we can add a title to make our plot more informative.
 Fourth, add labels for different categories when needed.
 Five, optionally we can add a text or an arrow at interesting
data points.
 Six, in some cases we can use some sizes and colors of the data
to make the plot more informative.
 T YPES OF V ISUALIZATIONS
 There are many types of visualizations. Some of the
most famous are:
 line plot,
 scatter plot,
 histogram,
 box plot,
 bar chart, and
 pie chart
 L INE P LOT
 a type of plot which displays information as a series of data points
called “markers” connected by straight lines.
 In this type of plot, we need the measurement points to be ordered
(typically by their x-axis values).
 This type of plot is often used to visualize a trend in data over
intervals of time - a time series.
 S CATTER PLOT
 This type of plot shows all individual data points.
 Here, they aren’t connected with lines. Each data point has the
value of the x-axis value and the value from the y-axis values.
 This type of plot can be used to display trends or correlations.
 H ISTOGRAM
An accurate representation of the distribution of numeric data.
 To create a histogram, first, we divide the entire range of values
into a series of intervals, and second, we count how many
values fall into each interval. The intervals are also called
bins.
 The bins are consecutive and non-overlapping intervals of a
variable. They must be adjacent and are often of equal size.
 B OX PLOT

 It is also called the box-and-whisker plot: a way to show

the distribution of values based on the five-number
summary: minimum, first quartile, median, third
quartile, and maximum.
 The minimum and the maximum are just the min and
max values from our data.
 The median is the value that separates the higher half of a
data from the lower half. It’s calculated by the following
steps: order your values, and find the middle one. In a
case when our count of values is even, we actually have 2
middle numbers, so the median here is calculated by
summing these 2 numbers and divide the sum by 2. For
example, if we have the numbers 1, 2, 5, 6, 8, 9, your
median will be (5 + 6) / 2 = 5.5.
 B OX PLOT

 The first quartile is the median of the data

values to the left of the median in our ordered
values. For example, if we have the numbers 1, 3, 4,
7, 8, 8, 9, the first quartile is the median from the 1,
3, 4 values, so it’s 3.
 The third quartile is the median of the data
values to the right of the median in our ordered
values. For example, if we use these numbers 1, 3,
4, 7, 8, 8, 9 again, the third quartile is the median
from the 8, 8, 9 values, so it’s 8.
 B OX PLOT
• IQR (Interquartile Range). The IQR approximates the amount of spread in
the middle 50% of the data. The formula is the third quartile - the first
quartile.
• This type of plot can also show outliers. An outlier is a data value that lies
outside the overall pattern. They are visualized as circles. When we have
outliers, the minimum and the maximum are visualized as the min and the
max values from the values which aren’t outliers. There are many ways to
identify what is an outlier. A commonly used rule says that a value is an
outlier if it’s less than the first quartile - 1.5 * IQR or high than the third
quartile + 1.5 * IQR.
 B AR CHART
 Represents categorical data with rectangular bars. Each
bar has a height corresponds to the value it represents.
 It’s useful when we want to compare a given numeric
value on different categories.
 P IE CHART
• A circular plot, divided into slices to show numerical
proportion. They are widely used in the business world.
• However, many experts recommend to avoid them.
• The main reason is that it’s difficult to compare the
sections of a given pie chart. Also, it’s difficult to compare
data across multiple pie charts. In many cases, they can be
replaced by a bar chart.
 S TATISTICAL D ESCRIPTION OF
D ATA AND D ATA V ISUALIZATION
 Statistics is the study of the collection,
analysis, interpretation, presentation, and
organization of data.
 Machine learning constructs algorithms
that can learn from data, especially for
prediction
 Statistical learning is branch of Statistics that
was born in response to Machine learning,
emphasizing statistical models and
assessment of uncertainty
S TATISTICAL D ESCRIPTION OF D ATA
Types of data: Understanding different types of data will
help you in choosing the different types of techniques that
you may use to get insights into the data.

1. Numerical

2. Categorical

3. Ordinal
 M ATHEMATICS IN M ACHINE
L EARNING
 Linear Algebra
(Matrix Operations, Projections, Factorization, Symmetric
Matrices, Vectorization)

 Probability Theory and Statistics

(Probability Rules & Axioms, Bayes’ Theorem, Random
Variables, Variance and Expectation, Conditional and Joint
Distributions, Standard Distributions.)

 Calculus
(Differential and Integral Calculus, Partial Derivatives)

 Algorithms and Complex Optimizations

(Binary Trees, Hashing, Heap, Stack)
I MPORTANCE OF M ATHEMATICS T OPICS
ON M ACHINE L EARNING

Linear Algebra (35%) Algorithm and Complexity (15%)

Multivariate Calculus (15%)

Others (10%)

Probability and Statistics (25%)

L INEAR A LGEBRA IN M ACHINE
L EARNING
 Linear algebra is a sub-field of mathematics concerned
with vectors, matrices, and linear transforms.
 It is a key foundation to the field of machine learning,
from notations used to describe the operation of
algorithms to the implementation of algorithms in code.
 Major Operations on:
Matrices
Vectors
STATISTICAL L EARNING :
BAYESIAN N ETWORK
 B AYESIAN N ETWORK
A simple graphical representation for a joint probability distribution.
• Nodes are random variables
• Directed edges between nodes reflect dependence

Syntax:
– a set of nodes, one per variable
– a directed, acyclic graph (link ≈ "directly influences")
– if there is a link from x to y, x is said to be a parent of y
– a conditional distribution for each node given its parents:
P (Xi | Parents (Xi ))

In the simplest case, conditional distribution represented as a conditional

probability table (CPT) giving the distribution over Xi for each
combination of parent values
 B AYESIAN N ETWORKS

Discrete Bayesian networks:

– Domain of each variable is finite
– Conditional probability distribution is a
CONDITIONAL PROBABILITY TABLE
B AYESIAN N ETWORK
R EPRESENTATION
 E XAMPLE : F IRE D IAGNOSIS
Construct the Bayesian Net (BN)
 Nodes are the random variables
 Directed arc from each variable in Pa(Xi ) to Xi
 Conditional Probability Table (CPT)
for each variable Xi : P(Xi | Pa(Xi ))

You want to diagnose whether there is a fire in a building ???

• You receive a noisy report about whether everyone is leaving the building
• If everyone is leaving, this may have been caused by a fire alarm
• If there is a fire alarm, it may have been caused by a fire or by tampering
• If there is a fire, there may be smoke
 H OW B AYESIAN N ETWORK W ORKS
Step 1: First choose the variables. In this case, all are Boolean:
 Tampering is true when the alarm has been tampered with
 Fire is true when there is a fire
 Alarm is true when there is an alarm
 Smoke is true when there is smoke
 Leaving is true if there are lots of people leaving the building
 Report is true if the sensor reports that lots of people are leaving the building

Step 2:Using the total ordering of variables:

Let’s say: Fire; Tampering; Alarm; Smoke; Leaving; Report.
 H OW B AYESIAN N ETWORK W ORKS
Step 3: Now choose the parents for each variable by
evaluating conditional independencies

 Fire is the first variable in the ordering, X1. It does not have parents.
 Tampering independent of fire (learning that one is true would not change
your beliefs about the probability of the other)
 Alarm depends on both Fire and Tampering: it could be caused by either
or both.
 Smoke is caused by Fire, and so is independent of Tampering and Alarm
given whether there is a Fire
 Leaving is caused by Alarm, and thus is independent of the other
variables given Alarm.
 Report is caused by Leaving, and thus is independent of the other
variables given Leaving.
B AYESIAN N ETWORK
 E XAMPLE 2: E ARTHQUAKE
E XAMPLE
I am at work, my neighbor John calls to say that my alarm went off,
neighbor Mary doesn’t call. Sometimes the alarm is set off by a minor
earthquake. Is there a burglar?
 E XAMPLE 2:
E ARTHQUAKE E XAMPLE
 F ULL J OINT D ISTRIBUTION
Conditional Probability
P(A^B)=P(A|B)* P(B)
P(A^B)=P(A|B)* P(A)
P(B|A)=P(A|B)*P(B)/ P(A)
 E XAMPLE 2: E ARTHQUAKE
E XAMPLE

Find the probability when John Call and Marry Calls and Alarm
Went Off and there is no burglary and no earthquake happened.
P(John, Marry , Alarm ,¬Burglary , ¬ Earthquake)
= P(J | A)* P(M , A)* P(A| ¬ B , ¬ E) * P(¬ B) *P(¬ E)
= 0.90 x 0.70 x 0.001 x 0.999 x 0.998
= 0.0006
NAÏVE BAYES C LASSIFIER
 N AÏVE B AYES C LASSIFIER
Reference Links
 [Link]
27a98ddd8e73

 [Link]
98a341224038
 Learning

supervised unsupervised

Eg: Fruit
Classification
Classifier-
with some Eg: Fruit
orange, apples, Seen lots of
known label Classifier-
bananas example but not
a proper label. “fruit with
soft skin”.
Like clustering “red
fruits”
 Supervised
learning
O/P variable
Draw is real or
conclusions continuous
like spam Classification Regression values like
or not, red marks or
or blue weight

Linear
Naïve Bayes
regression

Polynomial
Decision tree
regression

SVM
SVM
Regression
 W HAT I S N AIVE B AYES ?
 Naive Bayes is among one of the simplest, but most powerful
algorithms for classification based on Bayes' Theorem with an
assumption of independence among predictors.
 The Naive Bayes model is easy to build and particularly useful
for very large data sets. There are two parts to this algorithm:
 Naive
 Bayes
 The Naive Bayes classifier assumes that the presence of a
feature in a class is unrelated to any other feature.
 Even if these features depend on each other or upon the
existence of the other features, all of these properties
independently contribute to the probability that a particular
fruit is an apple or an orange or a banana, and that is why it is
known as "Naive."
 W HAT I S B AYES ' T HEOREM ?
 In statistics and probability theory, Bayes' theorem describes the probability
of an event, based on prior knowledge of conditions that might be related to
the event. It serves as a way to figure out conditional probability.
 Given a Hypothesis (H) and evidence (E), Bayes' Theorem states that the
relationship between the probability of the hypothesis before getting the
evidence, P(H), and the probability of the hypothesis after getting the
evidence, P(H|E), is :

 For this reason, P(H) is called the prior probability, while P(H|E) is called
the posterior probability. The factor that relates the two, P(E|H)/P(E), is
called the likelihood ratio. Using these terms, Bayes' theorem can be
rephrased as:
 "The posterior probability equals the prior probability times the likelihood
ratio."
 B AYES ' T HEOREM E XAMPLE
 Let's suppose we have a Deck of Cards and we wish to find out the
probability of the card we picked at random to being a king, given that it
is a face card. So, according to Bayes' Theorem, we can solve this
problem. First, we need to find out the probability:
 P(King) which is 4/52 as there are 4 Kings in a Deck of Cards.
 P(Face|King) is equal to 1 as all the Kings are face Cards.
 P(Face) is equal to 12/52 as there are 3 Face Cards in a Suit of 13 cards
and there are 4 Suits in total.
 E XAMPLE : B AYESIAN
T HEOREM
Doctor knows cold causes fever 50 percent of the time
Prob. patient having fever = 1/20
Prob. patient having cold=1/50000
If patient has fever, what is the probability that he or she
has cold??

P(C|F)=P(C)*P(F|C)/ P(F)

=(0.5*1/50000)/(1/20)
=0.0002
 Supervised
learning

O/P variable is real or

Classification Regression continuous values like
marks or weight
Draw conclusions like
spam or not, red or blue
Linear
Naïve Bayes
regression

Polynomial
Decision tree
regression

SVM
SVM
Regression
 W HAT I S N AIVE B AYES ?

Naive Bayes is among one of the simplest, but most powerful

algorithms for classification based on Bayes' Theorem with an
assumption of independence among predictors.

The Naive Bayes classifier assumes that the presence of a

feature in a class is unrelated to any other feature.

Very intuitive classification algorithm.

Naive

Naive
Bayes

• It makes the assumption that features

Bayes of a measurement are independent of
each other.

• Even if these features depend on each

Why Naïve other or upon the existence of the
other features, all of these properties
independently contribute to the
probability that a particular fruit is an
apple or an orange or a banana, and
that is why it is known as "Naive."
 Things we would like to do…..

Spam Classification
• Given an email, predict whether it is spam or
not

Medical Diagnosis
• Given a list of symptoms, predict whether a
patient has disease X or not

Weather
• Based on temperature, humidity, etc…
predict if it will rain tomorrow
Example-Air-Traffic Data

Cond. to next slide…

93
Example-Air-Traffic Data
Cond. from previous slide…

94
 Example-Air-Traffic Data
 In this database, there are four attributes
A = [ Day, Season, Fog, Rain]
with 20 tuples.
 The categories of classes are:
C= [On Time, Late, Very Late, Cancelled]

 Given this is the knowledge of data and classes, we are to find most likely
classification for any other unseen instance, for example:

 Classification technique eventually to map this tuple into an accurate class.

95
 Naïve Bayesian Classifier

 Example: With reference to the Air Traffic Dataset mentioned earlier, let
us tabulate all the posterior and prior probabilities as shown below.

96
Naïve Bayesian Classifier

97
 Naïve Bayesian Classifier

Instance:

Case1: Class = On Time : 0.70 × 0.64 × 0.14 × 0.29 × 0.07 = 0.0013

Case2: Class = Late : 0.10 × 0.50 × 1.0 × 0.50 × 0.50 = 0.0125

Case3: Class = Very Late : 0.15 × 1.0 × 0.67 × 0.33 × 0.67 = 0.0222

Case4: Class = Cancelled : 0.05 × 0.0 × 0.0 × 1.0 × 1.0 = 0.0000

Case3 is the strongest; Hence correct classification is Very Late

98
 Example: Game Prediction
Using NAÏVE Bayes'
To predict days suitable for a football match based on weather conditions
Smaller circle- low probability to play (P<0.5)
Big Circle- High probability to play (P>0.5)
Combining both the conditions
We get an OUTLOOK by combining both the data
Comparing the states when more
information is added

Naïve Bayes tries to understand

such interaction of probability
 Probabilistic classifier based on
Bayes’ Theorem

Takes independent assumptions

between the features.
Naïve Bayes
Classifier
Related dataset
 Attribute probability

Probability
of the class

0.60
Prior Probability
Example:
Predict the likelihood to play football
on ( Season =Winter, Sunny=No,
Windy= yes )
Try: Probability of match not being played?
 Applications of
Naïve Bayes Algorithm

Face recognition

Mail classification

Handwriting analysis

Salary prediction
 V ECTORIZATION ,
M ATRICES AND V ECTORS
 M ATRICES

Basic Operations on Matrices are as follows:

 Addition
 Subtraction
 Array Multiplication
 Matrix Multiplication
 Division
 V ECTORIZATION
 Vectors are a foundational element of linear algebra.
 Vectors are built from components, which are ordinary
numbers.
 Vectors can be considered as a list of numbers, and
vector algebra as operations performed on the numbers
in the list.
 Vectors are used throughout the field of machine
learning in the description of algorithms and processes
such as the target variable (y) when training an
algorithm.
 V ECTORIZATION
 Examples:
 For instance, a life insurance company might be
interesting in obtaining the vector of variables (blood
pressure, heart rate, height, weight, cholesterol level,
smoker, gender) to infer the life expectancy of a
potential customer.
 A farmer might be interested in determining the
ripeness of fruit based on (size, weight, spectral data).
An engineer might want to and dependencies in
(voltage, current) pairs.
 Likewise one might want to represent documents by a
vector of counts which describe the occurrence of
words. The latter is commonly referred to as bag of
words features.
 NAÏVE M ETHOD V / S V ECTORIZATION
Computing the sum of N elements:
 Naive way: Iterates over all the elements of the array
and stores the sum:
for i in range(n_rep):
loop_res=0
for elt in W:
Sum+=elt
 Vectorization: If W is our vector of interest, the sum of
its elements can be expressed as the dot product
: 1T W  Wi
for i in range(n_rep):
V=[Link]([Link])
Sum=[Link](W)
NAÏVE M ETHOD V / S V ECTORIZATION
V ECTORIZATION : C HALLENGES
 The scales and units of different coordinates may vary
widely.
 For instance, we could measure the height in
kilograms, pounds, grams, tons, stones, all of which
would amount to multiplicative changes.
 Likewise, when representing temperatures, we have a
full class of transformations, depending on whether we
represent them in terms of Celsius, Kelvin or Farenheit.
 If you think you are prepared for
Module 1

Try answering following questions:

 B RAIN T EASERS
1. What is Bayes Theorem and maximum posterior hypothesis?
2. Derive an equation for MAP hypothesis using Bayes Theorem.
3. Discuss the Naïve Bayes classifier.
4. Consider a football game between two rival teams Team 0 and
Team1. Suppose Team 0 wins 95% of the time and Team 1 wins the
remaining matches. Among the games won by Team 0, only 30% of
them come from playing on Team 1’s football field. On the other
hand 75% of the victories for Team 1 are obtained while playing at
home. If Team 1 is to host the next match between the two teams,
which team most likely emerge as the winner?
5. Define machine learning. Describe the steps in designing learning
system.
6. What do you mean by Hypothesis space, Instance space and Version
space?
 B RAIN T EASERS
7. The following Table gives dataset about stolen vehicles. Using Naïve
Bayes classifier classify the instance {Red, SUV, Domestic}

Color Type Origin Stolen

Red Sports Domestic Yes
Red Sports Domestic No
Red Sports Domestic Yes
Yellow Sports Domestic No
Yellow Sports Imported Yes
Yellow SUV Imported No
Yellow SUV Imported Yes
Yellow SUV Domestic No
Red SUV Imported No
Red Sports Imported Yes
ALL THE BEST!