SCE141 Machine Learning

[Link] First Semester SCE141

T.S. Kaveri, Assistant Professor

Department of Information Science and
Engineering
JSS S&TU, Mysuru- 570030
 Pre - Requisite
2

 Linear Algebra,
 Elementary Probability and Statistics,
 Knowledge of Programming in some Language.
 Course Outcomes
3

 After completing this course, students should be able to:

 CO1: Translate a business problem into a closely

related set of machine learning tasks.
 CO2: Select the appropriate algorithms and apply for
the machine learning tasks.
 CO3: Execute the machine learning tasks using the
Python language.
 CO4: Retrieve and assess the algorithm outcomes.
 CO5: Interpret and validate the outcomes
 Text Books
4

 1. Ethem Alpaydin (2014). Introduction to

Machine Learning, Third Edition, MIT
Press. The textbook website is
[Link]
 2. Pattern Recognition and Image Analysis, Earl Gose,
Richard Johnsonbaugh Steve Jos, Pearson 2015
 3. Understanding Machine Learning, Shai Shalev-
Shwartz and Shai Bendavid. Cambridge University
press. 2017. [SS-2017]
 Reference Books
5

 1. Machine Learning, Tom M. Mitchell, McGraw-

Hill Publishers, 1997.
 2. Pattern Recognition and Machine Learning,
Christopher M. Bishop, Springer Publishers, 2011.
 3. Kevin Murphy, Machine Learning: A Probabilistic
Perspective, MIT Press, 2012.
 4. Hal Daume III, A course in Machine Learning,
2015 (Most chapters freely available online).
 Assessment Weightage in Marks
6

 Class Test –I 10
 Quiz/Mini Projects/ Assignment/ seminars 10
 Class Test – II 10
 Quiz/Mini Projects/ Assignment/ seminars 10
 Class Test – III 10

 Total 50
 Question Paper Pattern
7

 Semester End Examination (SEE)

 Semester End Examination (SEE) is a written examination of three hours
duration of 100 marks with 50% weightage.

 Note:
 • The question paper consists of TWO parts PART- A and PART- B.

 • PART- A consists of Question Number 1-5 are compulsory

(ONE question from each unit)

 • PART-B consists of Question Number 6-15 will have internal choice.

(TWO question from each unit)

 • Each Question carries 10 marks and may consist of sub-questions.

 • Answer 10 full questions of 10 marks each
 UNIT – 1 Introduction & Bayesian Decision Theory:
1

 What Is Machine Learning? Examples of Machine

Learning Applications: Learning Associations,
Classification, Regression, unsupervised Learning,
Reinforcement Learning.
 Probability: Introduction, Probabilities of Events,
Random Variables, Joint Distributions and Densities
 Introduction to Bayesian Decision Theory,
Classification, Losses and Risks
MACHINE LEARNING – UNIT 1
Introduction
 Artificial Intelligence (AI)
 Machine Learning (ML)
 Deep Learning (DL)
 Data Science
Artificial Intelligence
 Artificial intelligence is intelligence demonstrated by
machines, as opposed to natural intelligence
displayed by animals including humans.
Machine Learning
 Machine Learning – Statistical Tool to explore the data.

Machine learning (ML) is a type of artificial intelligence

(AI) that allows software applications to become more
accurate at predicting outcomes without being explicitly
programmed to do so. Machine learning algorithms use
historical data as input to predict new output values.

If you are searching some item in amazon… next time…

without your request… your choice will be listed.
Variants of Machine Learning:
 Supervised
 Unsupervised

 Semi supervised

 Reinforcement Learning
Deep Learning
 It is the subset of ML, which mimic human brain.

 Three popular Deep Learning Techniques are:

 ANN – Artificial Neural Network
 CNN- Convolution Neural Network

 RNN- Recurrent Neural Network

Summary:

.
What is Machine Learning?
 Learning : is any process by which a system improves
performance from experience.
 Example: We call experienced doctor, Experienced
Teacher, Experienced driver

 Machine Learning is the study of algorithms that

• improve their performance P
• at some task T
• with experience E.
A well-defined learning task is given by <P,T,E>
 Defining the Learning Task
Improve on task T, with respect to performance metric P, based
on experience E
 T: Recognizing hand-written words
 P: Percentage of words correctly classified
 E: Database of human-labeled images of handwritten words

 T: Driving on four-lane highways using vision sensors

 P: Average distance traveled before a human-judged error

 E: A sequence of images and steering commands recorded

while observing a human driver.

 Features and classes
 Properties or attributes used to classify the objects are
called features.
 A collection of “similar” (not necessarily same) objects are
grouped together as one “class”.
 For example:

 All the above are classified as character T

 Classes are identified by a label.
 Most of the pattern recognition tasks are first done by
humans and automated later.
 Two broad types of classification
 Supervised classification

 Guided by the humans

 It is called supervised learning because the process of an
algorithm learning from the training dataset can be thought
of as a teacher supervising the learning process.
 We know the correct answers, the algorithm iteratively makes
predictions on the training data and is corrected by the
teacher.
 Classify the mails as spam or non spam based on redecided
parameters.
 Unsupervised classification
 Not guided by the humans.
 Unsupervised Classification is called clustering.
Another classifier :
Semi supervised learning

It makes use of a small number of labeled data

and a large number of unlabeled data to learn
 Reinforcement Learning
22

 Reinforcement learning is a machine learning

training method based on rewarding
desired behaviors and/or punishing
undesired ones.

 In general, a reinforcement learning agent is

able to perceive and interpret its environment,
take actions and learn through trial and error.
 Reinforcement – Learning from the
23
environment
Samples or patterns
 The individual items or objects or situations to be
classified will be referred as samples or patterns
or data.
 The set of data is called “Data Set”.
Training and Testing data
 Two types of data set in supervised classifier.
 Training set : 70 to 80% of the available data will be used for
training the system.
 In Supervised classification Training data is the data you use to
train an algorithm or machine learning model to predict the
outcome you design your model to predict.
 Testing set : around 20-30% will be used for testing the system. Test
data is used to measure the performance, such as accuracy or
efficiency, of the algorithm you are using to train the machine.
 Testing is the measure of quality of your algorithm.
 Many a times even after 80% testing, failures can be see during
testing, reason being not good representation of the test data in the
training set.
 Unsupervised classifier does not use training data
Statistical Decision Theory
 Decision theory, in statistics, a set of
quantitative methods for reaching optimal
decisions.
Example for Statistical Decision Theory
 Consider Hypothetical Basket ball Association:
 The prediction could be based on the difference
between the home team’s average number of points
per game (apg) and the visiting team’s ‘apg’ for
previous games.
 The training set consists of scores of previously played
games, with each home team is classified as winner or
loser
 Now the prediction problem is : given a game to be
played, predict the home team to be a winner or loser
using the feature ‘dapg’,
 Where dapg = Home team apg – Visiting team apg
Data set of games showing outcomes, differences between average
numbers of points scored and differences between winning
percentages for the participating teams in previous games
 The figure shown in the previous slide, lists 30 games and gives the
value of dapg for each game and tells whether the home team won or
lost.
 Notice that in this data set the team with the larger apg usually wins.

 For example in the 9th game the home team on average, scored 10.8
fewer points in previous games than the visiting team, on average and
also the home team lost.

 When the teams have about the same apg’s the outcome is less certain.
For example, in the 10th game , the home team on average scored 0.4
fewer points than the visiting team, on average, but the home team won
the match.

 Similarly 12th game, the home team had an apg 1.1. less than the
visiting team on average and the team lost.
Histogram of dapg
 Histogram is a convenient way to describe the data.
 To form a histogram, the data from a single class
are grouped into intervals.
 Over each interval rectangle is drawn, with height
proportional to number of data points falling in that
interval. In the example interval is chosen to have
width of two units.
 General observation is that, the prediction is not
accurate with single feature ‘dgpa’
Lost

Won
 Now observe the results on a scatterplot

 Each sample has a corresponding feature vector (dapg, dwp), which determines its position
in the plot.
 Note that the feature space can be classified into two decision regions by a straight line,
called a linear decision boundary. (refer line equation). Prediction of this line is logistic
regression.
 If the sample lies above the decision boundary, the home team would be classified as the
winner and it is below the decision boundary it is classified as loser.
 If the feature space cannot be perfectly separated by a straight
line, a more complex boundary might be used. (non-linear)

 Alternatively a simple decision boundary such as straight line

might be used even if it did not perfectly separate the classes,
provided that the error rates were acceptably low.
Simple illustration of Pattern
Classification
 A pattern/object can be identified by set of features.
 Collection of features for a pattern forms feature vector.

 Example : (in next slide)

 P1 and P2 are two patterns with 3 features, so 3 Dimensional
feature vector.
 There are two classes C1 and C2.
 P1 belongs to C1 and P2 belongs to C2
 Given P, a new pattern with feature vector, it has to be classified
into one of the class based on the similarity value.
 If d1 is the distance between (p and p1) and d2 is the distance
between (p and p2) then p will be classified into the class having
least difference.
Block diagram of Pattern recognition
and classification

Input to our pattern recognition system will be

feature vectors and output will be decision about
selecting the classes
 Having the model shown in previous slide, we can
use it for any type of recognition and classification.
 It can be
 speaker recognition
 Speech recognition

 Image classification

 Video recognition and so on…

 It is now very important to learn:
 Different techniques to extract the features
 Then in the second stage, different methods to
recognize the pattern and classify
◼ Some of them use statistical approach
◼ Few uses probabilistic model using mean and variance etc.
◼ Other methods are - neural network, deep neural networks
◼ Hyper box classifier
◼ Fuzzy measure
◼ And mixture of some of the above
When do we use Machine Learning?

 ML is used when:
• Human expertise does not exist (navigating on Mars)
• Humans can’t explain their expertise (speech
recognition)
• Models must be customized (personalized medicine)
• Models are based on huge amounts of data
(genomics)
EXAMPLES FOR MACHINE
LEARNING APPLICATIONS
 Learning Associations
42

 Basket analysis
Handwriting Recognition

43
License Plate Recognition

44
Biometric Recognition

45
Face Detection/Recognition

Detection

Matching

Recognition

46
Fingerprint Classification

Important step for speeding up identification

47
Autonomous Systems

Obstacle detection and avoidance

Object recognition

Image:NASA Mars [Link]

48
Medical Applications

Skin Cancer Detection Breast Cancer Detection

49
Land Cover Classification
(using aerial or satellite images)

Many applications including “precision” agriculture.

50
Applications of ML
 Learning associations  Face recognition
( Basket analysis)  Medical diagnosis
 Classification ( Credit
 Speech recognition
scoring)
 Biometrics
 Pattern Recognition(
Handwriting rec)  ML in NLP
 License plate  Autonomous system
recognition

51
Research avenues
 Deep learning
 Explainable AI (XAI)
 Reinforcement Learning
 Transfer learning
 Generative models
 Auto ML
 Privacy-preserving ML

52
Challenges in ML
 Data Quality
 Data Quantity
 Bias
 Interpretability
 Overfitting
 Scalability

53
PROBABILITY:
INTRODUCTION TO PROBABILITY
PROBABILITIES OF EVENTS
What is covered?
 Basics of Probability
 Combination
 Permutation
 Examples for the above
 Union
 Intersection
 Complement
 What is a probability
 Probability is the branch
of mathematics concerning numerical descriptions
of how likely an event is to occur

 The probability of an event is a number between 0

and 1, where, roughly speaking, 0 indicates that
the event is not going to happen and 1 indicates
event happens all the time.
 Experiment

 The term experiment is used in probability theory to

describe a process for which the outcome is not
known with certainty.

Example of experiments are:

Rolling a fair six sided die.
Randomly choosing 5 apples from a lot of 100 apples.
Event
 An event is an outcome of an experiment. It is
denoted by capital letter. Say E1,E2… or
A,B….and so on

 For example toss a coin, H and T are two events.

 The event consisting of all possible outcomes of a

statistical experiment is called the “Sample Space”.
Ex: { E1,E2…}
Examples
Sample Space of Tossing a coin = {H,T}
Tossing 2 Coins = {HH,HT,TH,TT}
Example

 The die toss:

 Simple events: Sample space:
1 E1
S ={E1, E2, E3, E4, E5, E6}
2 E2
S
3 E3 •E1 •E3
4 E4 •E5

5 E5 •E2 •E4 •E6

6 E6
 Frequency of an event
61

 Frequency of occurrence is measured by:

 (after the event has occurred)

Number of times A occurs f

=
n n

• If we let n get infinitely large,

f
P( A) = lim
n→ n
 The Probability of an Event

 The probability of an event A measures “how

often” A will occur. We write P(A).

 P(A) must be between 0 and 1.

 If event A can never occur, P(A) = 0. If event A always occurs
when the experiment is performed, P(A) =1.

 Then P(A) + P(not A) = 1.

 So P(not A) = 1-P(A)

 The sum of the probabilities for all simple events in S

equals 1.
Example 1

Toss a fair coin twice. What is the probability

of observing at least one head?

1st Coin 2nd Coin Ei

P(Ei)
H H 1/4
P(at least 1 head)
H H
T H 1/4
= P(E1) + P(E2) + P(E3)
H T 1/4
T 1/4 = 1/4 + 1/4 + 1/4 = 3/4
T H
T
Example 2
A bowl contains three color M&M’s, one red, one
blue and one green. A child selects two M&Ms at
random. What is the probability that at least one
is red?

1st M&M 2nd M&M Ei P(Ei)

m RB
m 1/6
m RG
1/6 P(at least 1 red)
m BR
m 1/6
m = P(RB) + P(BR)+
BG
1/6 P(RG) + P(GR)
m
m GB
1/6
m GR = 4/6 = 2/3
1/6
Example 3
The sample space of throwing a pair of dice is
 Example 3
Event Simple events Probability

Dice add to 3 (1,2),(2,1) 2/36

Dice add to 6 (1,5),(2,4),(3,3), 5/36
(4,2),(5,1)
Red die show 1 (1,1),(1,2),(1,3), 6/36
(1,4),(1,5),(1,6)
Green die show 1 (1,1),(2,1),(3,1), 6/36
(4,1),(5,1),(6,1)
Permutations

 The number of ways you can arrange

n distinct objects, taking them r at a
n!
time is r
P n
=
(n − r )!
where n!= n(n − 1)( n − 2)...( 2)(1) and 0! 1.
Example: How many 3-digit lock
combinations can we make from the
numbers 1, 2, 3, and 4?

4!
The order of the choice is
important!
P34 = = 4(3)( 2) = 24
1!
 Examples

Example: A lock consists of five parts and can be

assembled in any order. A quality control engineer wants
to test each order for efficiency of assembly. How many
orders are there?

The order of the choice is

important!
5!
P = = 5(4)(3)( 2)(1) = 120
5
5

0!
Combinations

 The number of distinct combinations of n

distinct objects that can be formed, taking
them r at a time is n!
Cr =
n

r!(n − r )!
Example: Three members of a 5-person committee
must be chosen to form a subcommittee. How many
different subcommittees could be formed?

5! 5(4)(3)( 2)1 5(4)

C35 = = = = 10
The order of 3!(5 − 3)! 3(2)(1)( 2)1 (2)1
the choice is
not
important!
Having 4 characters, how may 2 character words can be formed:
Permutation : P4,2= 12
Combination: C4,2 = 6

Remember Permutation is larger than combination

Summary:
 So formula for Permutation is : (order is relevant)

 Formula for Combination is: (Order is not relevant)

EVENT RELATIONS
 Special Events

The Null Event, is also called as empty event

represented by - 
 = { } = the event that contains no outcomes

The Entire Event, The Sample Space - S

S = the event that contains all outcomes
3 Basic Event relations

1. Union if you see the word or,

2. Intersection if you see the word and,
3. Complement if you see the word not.
Union

Let A and B be two events, then the union of A

and B is the event (denoted by AB) defined by:
A  B = {e| e belongs to A or e belongs to B}

AB

A B
The event A  B occurs if the event A occurs or
the event and B occurs or both occurs.

AB

A B
Intersection

Let A and B be two events, then the intersection

of A and B is the event (denoted by AB) defined
by:
A  B = {e| e belongs to A and e belongs to B}

AB

A B
The event A  B occurs if the event A occurs
and the event and B occurs .

AB

A B
 Complement

Let A be any event, then the complement of A

(denoted by A ) defined by:

A = {e| e does not belongs to A}

A
A
The event A occurs if the event A does not
occur

A
A
Mutually Exclusive

Two events A and B are called mutually

exclusive if:
A B =

A B
If two events A and B are mutually exclusive then:

1. They have no outcomes in common.

They can’t occur at the same time. The outcome of the
random experiment can not belong to both A and B.

A B
RULES OF PROBABILITY

ADDITIVE RULE
RULE FOR COMPLEMENTS
 Additive rule (General case)

P[A  B] = P[A] + P[B] – P[A  B]

or
P[A or B] = P[A] + P[B] – P[A and B]
The additive rule (Mutually exclusive events) if A  B = 

P[A  B] = P[A] + P[B]

i.e.
P[A or B] = P[A] + P[B]

if A  B = 
(A and B mutually exclusive)
 A B

A B

A B

When P[A] is added to P[B] the outcome in A  B are counted

twice
hence

P[A  B] = P[A] + P[B] – P[A  B]

 P  A  B = P  A + P  B − P  A  B

Example:
Bangalore and Manali are two of the cities competing for the National
university games. (There are also many others).

The organizers are narrowing the competition to the final 5 cities.

There is a 20% chance that Bangalore will be amongst the final 5.

There is a 35% chance that Manali will be amongst the final 5 and

an 8% chance that both Bangalore and Manali will be amongst the

final 5.

What is the probability that Bangalore or Manali will be amongst the

final 5.
Solution:
Let A = the event that Bangalore is amongst the final 5.
Let B = the event that Mohali is amongst the final 5.

Given P[A] = 0.20, P[B] = 0.35, and P[A  B] = 0.08

What is P[A  B]?

Note: “and” ≡ , “or” ≡  .

P  A  B = P  A + P  B − P  A  B

= 0.20 + 0.35 − 0.08 = 0.47

Find the probability of drawing an ace or a spade from a deck
of cards.
There are 52 cards in a deck; 13 are spades, 4 are aces.

Probability of a single card being spade is: 13/52 = 1/4.

Probability of drawing an Ace is : 4/52 = 1/13.

Probability of a single card being both Spade and Ace = 1/52.

Let A = Event of drawing a spade .

Let B = Event drawing Ace.

Given P[A] =1/4, P[B] =1/13, and P[A  B] = 1/52

P  A  B = P  A + P  B − P  A  B

P[A  B] = 1/4 + 1/13 – 1/52

Random Variables

Random variable takes a random value, which is real and can be finite
or infinite and it is generated out of random experiment.
The random value is generated out of a function.

Example: Let us consider an experiment of tossing two coins.

Then sample space is S= { HH, HT, TH, TT}

Given X as random variable with condition: number of heads.

X(HH) =2
X(HT) =1
X(TH) =1
X(TT) = 0
 Two types of random variables
 Discrete random variables
 Continuous random variable
Discrete random variables

 If the variable value is finite or infinite but

countable, then it is called discrete random
variable.

 Example of tossing two coins and to get the count of

number of heads is an example for discrete random
variable.

 Sample space of real values is fixed.

Continuous Random Variable

 If the random variable values lies between two certain fixed numbers then it is called
continuous random variable. The result can be finite or infinite.

 Sample space of real values is not fixed, but it is in a range.

 If X is the random value and it’s values lies between a and b then,

It is represented by : a <= X <= b

Example: Temperature, age, weight, height…etc. ranges between specific range.

Here the values for the sample space will be infinite
Probability distribution
 Frequency distribution is a listing of the observed
frequencies of all the output of an experiment that
actually occurred when experiment was done.
 Where as a probability distribution is a listing of
the probabilities of all possible outcomes that could
result if the experiment were done. (distribution with
expectations).
Broad classification of Probability
distribution

 Discrete probability distribution

 Binomial distribution
 Poisson distribution

 Continuous Probability distribution

 Normal distribution
Discrete Probability Distribution:
Binomial Distribution
 A binomial distribution can be thought of as
simply the probability of a SUCCESS or FAILURE
outcome in an experiment or survey that is
repeated multiple times. (When we have only two
possible outcomes)

 Example, a coin toss has only two possible

outcomes: heads or tails and taking a test could
have two possible outcomes: pass or fail.
 Assumptions of Binomial distribution
(It is also called as Bernoulli’s Distribution)
 Assumptions:
 Random experiment is performed repeatedly with a fixed and finite
number of trials. The number is denoted by ‘n’
 There are two mutually exclusive possible outcome on each trial,
which are know as “Success” and “Failure”. Success is denoted by ‘p’
and failure is denoted by ‘q’. and p+q=1 or q=1-p.
 The outcome of any give trail does not affect the outcomes of the
subsequent trail. That means all trials are independent.
 The probability of success and failure (p&q) remains constant for all
trials. If it does not remain constant then it is not binomial distribution.
For example tossing a coin the probability of getting head or
getting a red ball from a pool of colored balls, here every time after
the ball is taken out it is again replaced to the pool.
 With this assumption let see the formula
Formula for Binomial Distribution

OR

P(X=r) =

Where P is success and

q is failure
Binomial Distribution: Illustration with example

 Consider a pen manufacturing company

 10% of the pens are defective

 (i)Find the probability that exactly 2 pens are

defective in a box of 12
 So n=12,
 p=10% = 10/100 = 1/10
 q= (1-q) =90/100 = 9/10
 X=2
 Consider a pen manufacturing company
 10% of the pens are defective

 (i)Find the probability that at least 2 pens are defective

in a box of 12
 So n=12,
 p=10% = 10/100 = 1/10
 q= (1-q) =90/100 = 9/10
 X>=2
 P(X>=2) = 1- [P(X<2)]
 = 1-[P(X=0) +P(X=1)]
Binomial distribution: Another example

 If I toss a coin 20 times, what’s the

probability of getting exactly 10
heads?
 20  10 10
 (.5) (.5) = .176
 10 
The Binomial Distribution: another example

 Say 40% of the class is  n  x n− x

P ( x) =   p q
female.  x
 What is the probability 10  6 10−6
that 6 of the first 10 =  (.4 )(. 6 )
6
students walking in will
be female? = 210(.004096)(. 1296)
= .1115
 Continuous Probability Distributions
 When the random variable of interest can take any value in an interval, it is called
continuous random variable.
 Every continuous random variable has an infinite, uncountable number of possible
values (i.e., any value in an interval).

• Examples Temperature on a given day, Length, height, intensity of light falling on a

given region.
 The length of time it takes a truck driver to go from New York City to Miami.
 The depth of drilling to find oil.
 The weight of a truck in a truck-weighing station.
 The amount of water in a 12-ounce bottle.
For each of these, if the variable is X, then x>0 and less than some maximum value
possible, but it can take on any value within this range
 Continuous random variable differs from discrete random variable. Discrete
random variables can take on only a finite number of values or at most a
countable infinity of values.

 A continuous random variable is described by Probability density function.

This function is used to obtain the probability that the value of a continuous
random variable is in the given interval.
Continuous Uniform Distribution
 For Uniform distribution, f(x) is constant over the
possible value of x.
 Area looks like a rectangle.
 For the area in continuous distribution we need to
do integration of the function.
 However in this case it is the area of rectangle.
 Example to time taken to wash the cloths in a
washing machine. (for a standard condition)
CONTINUOUS DISTRIBUTIONS

The Uniform distribution from a to b

 1
 a xb
f ( x) = b − a
 0 otherwise
0.4
f ( x)
0.3

0.2


1 
0.1 
b−a 

0
0 5 10 x 15
a b
NORMAL DISTRIBUTION
 The most often used continuous probability distribution is the
normal distribution; it is also known as Gaussian distribution.
 Its graph called the normal curve is the bell-shaped curve.

 Such a curve approximately describes many phenomenon occur in

nature, industry and research.
 Physical measurement in areas such as meteorological
experiments, rainfall studies and measurement of
manufacturing parts are often more than adequately explained
with normal distribution.
NORMAL DISTRIBUTION Applications:

The normal (or Gaussian) distribution, is a very commonly used (occurring) function in the fields
of probability theory, and has wide applications in the fields of:

- Pattern Recognition;
- Machine Learning;
- Artificial Neural Networks and Soft computing;
- Digital Signal (image, sound , video etc.) processing
- Vibrations, Graphics etc.
Joint Distributions and Densities
 The joint random variables (x,y) signifies that ,
simultaneously, the first feature has the value x
and the second feature has the value y.

 If the random variables x and y are discrete,

the joint distribution function of the joint
random variable (x,y) is the probability of
P(x,y) that both x and y occur.
Joint distribution in continuous random
variable
 If x and y are continuous, then the probability density
function is used over the region R, where x and y is
applied is used.
 It is given by:

 Where the integral is taken over the region R. This

integral represents a volume in the xyp plane.
 Moments of Random Variables

Moments are very useful in statistics because they tell us much about our data.
 In mathematics, the moments of a function are quantitative measures related to the shape of the
function's graph.
 It gives information about the spread of data, skewedness and kurtosis.

 If the function is a probability distribution, then there are four commonly used moments in statistics
The first moment is the expected value - measure of center of the data
The second central moment is the variance - spread of our data about the mean
The third standardized moment is the skewness - the shape of the distribution
The fourth standardized moment is the kurtosis - measures the peakedness or flatness
of the distribution.
Computing Moments for population
Moment 3: To know the Skewness

In positive
Skewness,
Mean is >
median
and
Median>mo
de

And it is
reverse in
case of –ve
skewness
Moment 4 : To know the Kurtosis
D
Normal Distribution
 Consider an example of x values:
 4,5,5,6,6,6,7,7,8
 Mode, Median and mean all will be equal

 = Mode is 6
 = Median is 6
 = Mean is also 6
Positive Skew
 Consider an example of x values:
 5,5,5,6,6,7,8,9,10
 (It is an example for Normal Distribution)
 = Mode is 5(most repeated)
 = Median is 6 (middle element )
 = Mean is also 6.8 (average)
+ve skew
-ve skew
What Is Conditional Probability?

 Conditional probability is defined as the likelihood

of an event or outcome occurring, based on the
occurrence of a previous event or outcome.
 Conditional probability is calculated by multiplying
the probability of the preceding event by the
updated probability of the succeeding, or
conditional, event.
 Bayes' theorem is a mathematical formula used in
calculating conditional probability.
Definition

Suppose that we are interested in

computing the probability of event A and
we have been told event B has occurred.
Then the conditional probability of A given
B is defined to be:
P  A  B
P  A B  = if P  B  0
P  B

Illustrates that probability of A, given(|) probability of B occurring

 Rationale:
If we’re told that event B has occurred then the
sample space is restricted to B.

The event A can now only occur if the outcome is in of

A ∩ B. Hence the new probability of A in Bis:

A
P  A  B B
P  A B  =
P  B A∩B
An Example

Twenty – 20 World cup started:

For a specific married couple the probability that the

husband watches the match is 80%,
the probability that his wife watches the match is 65%,
while the probability that they both watch the match is
60%.

If the husband is watching the match, what is the

probability that his wife is also watching the match
Solution:

Let B = the event that the husband watches the

match
P[B]= 0.80

Let A = the event that his wife watches the

match
P[A]= 0.65 and
P[A ∩ B]= 0.60
P  A  B 0.60
P  A B  = = = 0.75
P  B 0.80
 Another example
 There are 100 Students in a class.
 40 Students likes Apple
 Consider this event as A, So probability of occurrence of A is 40/100 = 0.4
 30 Students likes Orange.
 Consider this event as B, So probability of occurrence of B is 30/100=0.3

 Remaining Students does not like either Apple nor Orange

 20 Students likes Both Apple and Orange, So probability of Both A and B occurring is = A
intersect B = 20/100 = 0.2

 What is the probability of A in B, means what is the probability that A is occurring given B
:
 P(A|B) = 0.2/0.3 = 0.67

P(A|B) indicates that A

occurring in the sample
40 20 30 space of B.

Here we are not

considering the entire
sample space of 100
students, but only 30
More Example Problem for Conditional Probability

Example : Calculating the conditional probability of rain given that the biometric
pressure is high.
Weather record shows that high barometric pressure (defined as being over 760 mm
of mercury) occurred on 160 of the 200 days in a data set, and it rained on 20 of the
160 days with high barometric pressure. If we let R denote the event “rain occurred”
and H the event “ High barometric pressure occurred” and use the frequentist approach
to define probabilities.
P(H) = 160/200 = 0.8
and P(R and H) = 20/200 = 0.10 (rain and high barometric pressure intersection)

We can obtain the probability of rain given high pressure, directly from the data.
P(R|H) = 20/160 = 0.10/0.80 = 0.125
Representing in conditional probability
P(R|H) = P(R and H)/P(H) = 0.10/0.8 = 0.125.
 Bayes Theorem

When the joint probability, P(A∩B), is hard to calculate or if the inverse or Bayes
probability, P(B|A), is easier to calculate then Bayes theorem can be applied.

Revisiting conditional probability

Suppose that we are interested in computing the probability of event A and we have been told
event B has occurred.
Then the conditional probability of A given B is defined to be:

P  A  B if P  B  0
P
 A B
= P  B

P[A  B]
Similarly, P[B|A] = if P[A] is not equal to 0
P[A]
 Original Sample space is the red coloured rectangular
box.
 What is the probability of A occurring given sample
space as B.
 Hence P(B) is in the denominator.
 And area in question is the intersection of A and B
 P  A  B
P  A B  =
P  B

and

From the above expressions, we can rewrite

P[A  B] = P[B].P[A|B]
and P[A  B] = P[A].P[B|A]
This can also be used to calculate P[A  B]

So
P[A  B] = P[B].P[A|B] = P[A].P[B|A]
or
P[B].P[A|B] = P[A].P[B|A]

P[A|B] = P[A].P[B|A] / P[B] - Bayes Rule

Bayes Theorem
Bayes Theorem:
 Example1:

 Compute : Probability in the deck of cards (52 excluding

jokers)

 Probability of (King/face)

 It is given by P(King/face) = P(face/King) * P(King)/ P(face)

= 1 * (4/52) / (12/52)
= 1/3
 Example2:

Cold (C) and not-cold (C’). Feature is fever (f).

Prior probability of a person having a cold, P(C) = 0.01.

Prob. of having a fever, given that a person has a cold is, P(f|C) = 0.4.
Overall prob. of fever P(f) = 0.02.

Then using Bayes Th., the Prob. that a person has a cold, given that she (or he) has a
fever is:
P(f|C) P(C ) 0.4∗0.01
P(C|f) = == = 0.2
P(f ) 0.02
Generalized Bayes Theorem
 Consider we have 3 classes A1, A2 and A3.
 Area under Red box is the sample space

 Consider they are mutually exclusive and

collectively exhaustive.
 Mutually exclusive means, if one event occurs then

another event cannot happen.

 Collectively exhaustive means, if we combine all the
probabilities, i.e P(A1), P(A2) and P(A3), it gives the sample
space, i.e the total rectangular red coloured space.
 Consider now another event B occurs over A1,A2
and A3.
 Some area of B is common with A1, and A2 and A3.
 It is as shown in the figure below:
 Portion common with A1 and B is shown by:
 Portion common with A2 and B is given by :
 Portion common with A3 and B is given by:

 Probability of B in total can be given by

 Remember :

 Equation from the previous slide:

 Replacing first in the second equation in this slide, we

will get:
Further simplified P(B)
Arriving at Generalized version of
Bayes theorem
Example 3: Problem on Bayes theorem
with 3 class case
What is being asked
 While solving problem based on Bayes theorem, we
need to split the given information carefully:
 Asked is:
 Note, the flip of what is asked will be always given:

 It is found in the following statement :

 What else is given:

 Represented by:
So.. Given Problem can be represented
as:
Example-4.
Given 1% of people have a certain genetic defect. (It means 99% don’t have genetic defect)
90% of tests on the genetic defected people, the defect/disease is found positive(true positives).
9.6% of the tests (on non diseased people) are false positives

If a person gets a positive test result,

what are the Probability that they actually have the genetic defect?

A = chance of having the genetic defect. That was given in the question as 1%. (P(A) = 0.01)
That also means the probability of not having the gene (~A) is 99%. (P(~A) = 0.99)
X = A positive test result.

P(A|X) = Probability of having the genetic defect given a positive test result. (To be computed)

P(X|A) = Chance of a positive test result given that the person actually has the genetic defect = 90%. (0.90)
p(X|~A) = Chance of a positive test if the person doesn’t have the genetic defect. That was given in the question as 9.6% (0.096)
Now we have all of the information, we need to put
into the equation:

P(A|X) = (.9 * .01) / (.9 * .01 + .096 * .99) =

0.0865 (8.65%).

The probability of having the faulty gene on the test is

8.65%.
Example - 5
Given the following statistics, what is the probability that a woman
has cancer if she has a positive mammogram result?
One percent of women over 50 have breast cancer.
Ninety percent of women who have breast cancer test
positive on mammograms.
Eight percent of women will have false positives.

Let women having cancer is W and ~W is women not having

cancer.
Positive test result is PT.
Solution for Example 5
What is asked: what is the probability that a woman has cancer if
she has a positive mammogram result?

 P(W)=0.01
 P(~W)=0.99
 P(PT|W)=0.9
 P(PT|~W)=0.08 Compute P(testing positive)
(0.9 * 0.01) / ((0.9 * 0.01) + (0.08 * 0.99) = 0.10.
 Example-6
A disease occurs in 0.5% of the population
(5% is 5/10% removing % (5/10)/100=0.005)

A diagnostic test gives a positive result in:

◦ 99% of people with the disease

◦ 5% of people without the disease (false positive)

A person receives a positive result

What is the probability of them having the disease, given a positive result?
 𝑃(𝑃𝑇|𝐷)×𝑃 𝐷
◦ 𝑃 𝑑𝑖𝑠𝑒𝑎𝑠𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑡𝑒𝑠𝑡 =
𝑃(𝑃𝑇|𝐷)×𝑃 𝐷 +𝑃 𝑃𝑇 ~𝐷 ×𝑃 ~𝐷

0.99×0.005
◦ =
0.99×0.005 + 0.05×0.995

Therefore:
0.99 × 0.005
𝑃 𝑑𝑖𝑠𝑒𝑎𝑠𝑒 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑡𝑒𝑠𝑡 = = 0.09
0.0547
𝑖. 𝑒. 9%

◦ We know:
𝑃 𝐷 = chance of having the disease
𝑃 ~𝐷 = chance of not having the disease

◦ 𝑃 𝑝𝑜𝑠𝑖𝑡𝑖𝑣𝑒 𝑡𝑒𝑠𝑡 𝑑𝑖𝑠𝑒𝑎𝑠𝑒 = 0.99

◦ 𝑃(𝑑𝑖𝑠𝑒𝑎𝑠𝑒) = 0.005
END OF UNIT 1