Course code: STAT-301
Course Title: Probability & Probability Distribution I
Topic: Normal distribution, Log Normal distribution
Week 14:
Probability Distributions:
Continuous Probability Distributions:-
If X is a continuous random variable, the probability that X takes on any one
particular value is generally zero, therefore we cannot define a probability
function in the same way as for a discrete random variable. In order to arrive at
a probability distribution for a continuous random variable we note that the
probability that lies between two different values is meaningful. ( )is called
the probability density function (p.d.f). The following are the properties of p.d.f;
(i) ( ) for all &
(ii)
f ( x)dx 1
A probability density function on a sample space is non-negative function
whose integral over the sample space is unity i.e.
Distribution Function:-
The Distribution function (D.F) of a continuous random variable is defined as
x
P( X x) F ( x)
f ( x)dx x
If ( ) is a continuous at the point and its first derivative exist then we have
dF ( x)
F ( x) f ( x)
dx
F ( y ) F ( x) for y x
( ) is non-decreasing function. Also we have ( ) ( )
Similarly we have
P( X x) f ( x)dx 1 F ( x)
x
P( x X y ) f (t )dt F ( y ) F ( x)
x
The distribution function is sometimes written as cumulative distribution
function (c.d.f).
Definition of Moments, Moment Generating Functions, Characteristic
Function and Measures of Location for Continuous Random Variable:-
We have to extend all the definitions and theorems of discrete probability
distribution to continuous probability distribution and we have to replace the
summation over the possible variations of the random variable by the
integration over the possible range of . let us take a random variable with
p.d.f ( ). Then we define;
r E ( X ) x
r r
f ( x)dx
M o (t ) E (etx ) e
tx
f ( x)dx
Provided the integral exists for real values of t
(t ) E (e ) e
itx itx
f ( x)dx
Mean deviation about the mean is given by
E ( x 1) ( x 1) . f ( x)dx
Median is defined as the solution of the equation;
m
2.F (m) 1 where F (m)
f ( x)dx
The mode is given by
d
f ( x) 0 or log f ( x) 0
dx
d 2 log f ( x)
f ( x) 0 or 0
dx 2
(7) Normal Distribution:
History:
The normal probability distribution, is considered the cornerstone of the modern
Statistical theory, was discovered by Abraham de Moivre (1667-1754) as the
limiting form of the binomial distribution by increasing n, the number of trials,
to a very large number for a fixed value of p. His paper remained unnoticed
until in 1924, when it was found in a library by Karl Pearson (1857-1936). The
name of Laplace (1749-1827) is also associated with the derivation of the
Normal distribution. The Normal distribution is also called the Gaussian
distribution in honor of the great German mathematician Gauss (1777-1855),
who also derived its equation mathematically as the probability distribution of
the errors of measurements. It was Karl Pearson who is 1893 called it the
Normal distribution and is best known by this name today.
Normal distribution as a limiting case:
The Normal distribution is another limiting form of the binomial distribution
under the following conditions
, the number of trials is indefinitely large i.e.
Neither nor the is very small
Normal distribution:
A random variable is defined to have a Normal distribution, if the probability
density function of X is given by
( )
( )
√
Normal distribution is a continuous distribution as Normal random variable
can assume each and every value between the given interval i.e. .
As is representing a Normal distribution, so is a Normal random variable.
Normal distribution has two parameters.
A random variable with mean and variance and following Normal
distribution is as ( ) ( ). And
is a Standard Normal distribution with mean 0 and unit variance (variance =1).
And we write it as
( )
The probability density function (p.d.f.) of standard normal variates is
( ) ( )
√
Normal distribution is important in Statistics and is often used in
the natural and social sciences to represent real-valued random variables whose
distributions are not known. Their importance is partly due to the central limit
theorem. It states that, under some conditions, the average of many samples
(observations) of a random variable with finite mean and variance is itself a
random variable—whose distribution converges to a normal distribution as the
number of samples increases. Therefore, physical quantities that are expected to
be the sum of many independent processes, such as measurement errors, often
have distributions that are nearly normal.
Moreover, Gaussian distributions have some unique properties that are valuable
in analytic studies. For instance, any linear combination of a fixed collection of
normal deviates is a normal deviate. Many results and methods, such as least
squares parameter fitting, can be derived analytically in explicit form when the
relevant variables are normally distributed.
A normal distribution is sometimes informally called a bell curve. However,
many other distributions are bell-shaped (such as the Cauchy, Student's t,
and logistic distributions)
Properties of Normal distribution:
( ) and the total area under the normal curve is unity.
The mean and variance of normal distributions are respectively.
The median and mode of the normal distribution are each equal to the mean of
normal distribution i.e. Normal distribution is a symmetrical and Unimodal.
Normal distribution is bell curved
The mean deviation of normal distribution is approximately .
For the normal distribution, all the odd order moments are zero.
No matter what are the values of are, area under the normal curve remains
in certain fixed proportions within a specified number of standard deviations on
either side of . For example
The normal curve approaches, but never touches the horizontal axis on either
side of .
Show that the total area under the normal curve is unity
∫ ( )
( )
∫ ( ) ∫
√
∫ ( ) ∫
√
∫ ( ) ∫
√
This is an even function
Now as we know that if function is an odd function such that
( ) ( ) ( ) ∫ ( )
Now as we know that if function is an even function such that
( ) ( ) ( ) ∫ ( ) ∫ ( )
∫ ( ) ∫
√
√ √ ( )
√ ( )
∫ ( ) ∫ √ ( )
√
∫ ( ) ∫ ( )
√
∫ ( ) ( ) ( ) ∫ ( ) ( )
∫ ( ) ( )
√
( ) ( ) ( ) ( ) ( ) √
∫ ( ) √
√
Mean & Variance of Normal distribution by direct method:
( ) ∫ ( )
( )
( ) ∫
√
( ) ∫ ( )
√
( ) ∫ ( )
√
( ) ∫ ∫
√ √
As we know that the first part of the integral is an even function and second part
of integral is an odd function and
∫ ( )
∫ ( ) ∫ ( )
So second integral would vanish as
∫
√
( ) ́ ( ) ( )
( ) ́ ∫ ( )
( )
( ) ∫
√
( ) ∫ ( )
√
( ) ∫ ( )
√
( ) ∫ ( )
√
( ) ∫ ∫ ∫
√ √ √
As we know that the first & third part of the integral are even functions and
second part of integral is an odd function and
∫ ( )
∫ ( ) ∫ ( )
So second integral would vanish as
∫
√
( ) ( ) ( ) ∫
√
( ) ∫ ( )
√
∫
√ √ ( )
√ ( )
√ ( )
∫ ∫ √ ( )
∫ √ ∫ ( )
∫ √ ∫ ( )
∫ √ ∫ ( )
∫ √ ∫ ( )
∫ ( ) ( ) ∫ ( ) ( )
∫ √ ( )
( ) ( ) ( ) ( ) ( )
( ) √ ( ) √
∫ √ √ √
Put this value in eq. (1)
( ) ( √ )
√
( ) ́
́ ( ́ )
( )
Similarly we can find ́ ́ by using direct method
( ) ́
( ) ́
Median of Normal distribution:
As we know that
( )
( )
( ) ∫ ( ) ∫
√
( )
∫
√
( )
∫
√
∫
√
Mode of Normal distribution:
Mode is the value of for which ( ) is maximum. Mode is the solution of
́( ) ́( )
( ( )) ( ( ))
( )
( )
√
( )
( ) ( )
√
( )
( ) ( ) ( )
√
( ) ( ) ( )
( ) ( )
( ( )) ( ( ) )
( ( )) ( )
( ( )) ( )
( ( ))
Mode is the solution of
( ( ))
( )
Normal distribution is symmetrical.
Question: Derive the Moment Generating Function, Cumulant
Generating Function, & Characteristic Function of Normal Distribution
and how the moments about mean can be generated using these functions.
1- Moment Generating Function:
( ) ( ) ∫ ( )
( )
( ) ∫
√
( )
( ) ∫
√
( ) ∫
√
( ) ∫
√
( )
( ) ∫
√
Adding & subtracting in the exponent
( )
( ) ∫
√
( )
( ) ∫
√
( )
( ) ∫
√
( )
∫ ( )
√
( )
∫ ( )
√
( )
How to generate mean moments by using M.g.f
Now, by differentiating the m.g.f ( ) ‘r’ times w.r.t ‘t’ and by letting equal
to zero.
| ( )| ( ) ́
We may get the moments about mean by using the relations between moments
about mean and moments about origin
́ ( ́ )
́ ́ ́ ( ́ )
́ ́ ́ ́ ( ́ ) ( ́ )
2- Mean Moment Generating Function:
As we know that by the relation between mean moment generating function and
moment generating function
( ) ( )
( )
( )
( )
( ) ( )
( )
( )
( )
Comparing the coefficients of in ( ) directly gives rth moments
about mean i.e. .
There is no odd order power, so all odd order moments are zero.
And the coefficients of are i.e.
( )
Coefficient of Skewness:
( )
( )
Coefficient of Kurtosis:
( )
( )
3- Cumulant Generating Function of Poisson distribution:
Taking the logarithm of moment generating function it becomes Cumulant
generating function
( ) ( ( ))
( ) ( )
( ) ( ( ))
( )
How to generate mean moments by using Cumulant generating function
( ) * +
(Cumulants) is the coefficient of in the expansion of ( ( ))
So, comparing the coefficients of in ( ) gives the Cumulants
Relation between moments & cumulants:
́
( )
4- Characteristic Function of Poisson Distribution:
( ) ( ) ∫ ( )
( )
( ) ∫
√
( )
How to generate mean moments by using Characteristic Function
Now, by differentiating the ( ) ‘r’ times w.r.t ‘t’ and by letting equal to zero
and then dividing by
| ( )| ( ) ́
Mean Deviation of Normal distribution:
(| |) ∫ | | ( )
( )
(| |) ∫ | |
√
(| |) ∫ | |
√
(| |) ∫ | |
√
As function is an even function
(| |) ∫
√
(| |) ∫
√
√
(| |) ∫
√
(| |) √ ∫
(| |) √
Evaluation of Distribution Function of Normal distribution when z is large
& positive:
( ) ( ) ∫
√
( ) ∫
√
( ) ∫ ∫
√ √
( ) ∫
√
Log Normal distribution:
If is ( ), then the r.v. is called lognormal random variable
(because its logarithm ( ) is a normal r.v.) , if ( ) is normally
distributed with mean and variance
The density function of is given by
( )
( )
( )
√
As is representing a Log Normal distribution, so
The pdf of normal distribution is
( )
( )
√
Making transformation ( ), , we get
( )
( )
( )
√
Log Normal distribution is a continuous distribution as random variable can
assume each and every value between the given interval i.e. . Log
Normal distribution has two parameters.
Log Normal distribution is positively skewed distribution.
If is sufficiently small or less than 0.2 then a log normal distribution similar
to the normal distribution can be constructed.
( ).
Direct Moment:
( ) ∫ ( )
( )
( )
( ) ∫
√
( )
( )
( )
( ) ∫ ( )
√
( )
( ) ∫
√
( )
( ) ∫
√
( ) ∫ * ( )+
√
( ) ∫ * ( )+
√
( ) ∫ * ( ( ) ( ) )+
√
( ) ∫ * ( )+
√
( ) ∫ * ( )+
√
( ) ∫ * ( )+
√
( ) ∫ * ( )+
√
( ) ∫ * ( ) +
√
( )
( ) ∫
√
( )
( )
√
( )
∫
√
( )
Moments about Origin:
( ) ́
( ) ́
( ) ́
( ) ́
Moments about Mean:
́ ( ́ )
( )
( )
́ ́ ́ ( ́ )
( )
( )
́ ́ ́ ́ ( ́ ) ( ́ )
( ) ( )
( )
The mgf of the r.v. cannot be expressed in a closed form and hence is not useful.