0% found this document useful (0 votes)
15 views33 pages

Joint and Marginal Distributions in Statistics

Chapter 4 discusses multiple random variables, focusing on joint and marginal distributions. It defines concepts such as joint probability mass functions (pmf), marginal distributions, and conditional distributions, providing examples including the calculation of joint probabilities and the use of continuous bivariate random vectors. The chapter emphasizes the importance of understanding the relationships between random variables through joint and marginal distributions.

Uploaded by

mintia766
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
15 views33 pages

Joint and Marginal Distributions in Statistics

Chapter 4 discusses multiple random variables, focusing on joint and marginal distributions. It defines concepts such as joint probability mass functions (pmf), marginal distributions, and conditional distributions, providing examples including the calculation of joint probabilities and the use of continuous bivariate random vectors. The chapter emphasizes the importance of understanding the relationships between random variables through joint and marginal distributions.

Uploaded by

mintia766
Copyright
© All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd

Mathematical Statistics 4-1

Chapter 4. Multiple Random Variables

4.1 Joint and Marginal Dist.s

• prob models that involve only one rv → univariate models

prob models that involve more than one rv → multivariate models

[Def. 4.1.1]    ⋯    -dimensional random vector



 

[Ex. 4.1.2] (Sample space for dice)


Toss two fair dice.
 ≡sum of the two dice
 ≡ |difference of the two dice|
Values of the joint pmf              → Table 4.1.1

[Def. 4.1.3]     discrete bivariate random vector


              jpmf of   

• Let  be any subset of  . Then   ∈ 


∈
   

•       
∈ 
       

•      : constants
                                 

• 
∈ 
        ∈    

[Thm. 4.1.6]        jpmf of  


⇒     
∈
      mpmf of 

    
∈
      mpmf of 
Mathematical Statistics 4-2

[Def. 4.1.10]     continuous bivariate random vector


     jpdf if for every  ⊂     ∈         

∞ ∞
•       
∞ ∞
         

• the marginal pdf of  and 



     ∞
      mpdf of 


     ∞
      mpdf of 

∞ ∞
• 
∞ ∞
       

[Ex. 4.1.11] (Calculating joint prob.s-Ⅰ)


                

(ⅰ) Check that    is indeed a joint pdf .


    ≥  ∀   
∞ ∞  
∞ ∞
             
 


(ⅱ) Consider     ≥   1

    ≥          

 

   
    
 

1
    

≡    
 

(ⅲ) the marginal pdf of 



     


     


 
     
    




   


Mathematical Statistics 4-3

[Ex. 4.1.12] (Calculating joint prob.s-Ⅱ)



        ∞ 
1

Consider     ≥  
         ≥            
 
1

   
    ≥            
 
  


 
        
  
 

(Note) It is almost always helpful to graph the sets of interest


in determining the appropriate limits of integration.

• joint cdf
       ≤   ≤  
 
continuous case :       ∞ ∞
      

    
     
 

4.2 Conditional Dist.s and Independence

[Def. 4.2.1]     discrete bivariate random vector with jpmf    
and mpmf      
the conditional pmf of  given that   
   
                 for any  st     
  
the conditional pmf of  given that   
   
                 for any  st     
  
Mathematical Statistics 4-4

• Check that      is a pmf.


  ∣   ≥  ∀

      
       
   

   .
  
 

•                  

[Ex. 4.2.2] (Calculating conditional prob.s)


the jpmf:
 
                       
 
 
            
 
⇒ the mpmf of 

              


                   


         

⇒ the conditional pmf of  given 
     
For               
     

    
        
    
 
For                   
 

 
        
 

For             

Mathematical Statistics 4-5

the conditional prob.s



                      


                

[Def. 4.2.3]     conti. bivariate random vector


   
             
  
   
             
  

         
∞
        

[Ex. 4.2.4] (Calculating conditional pdfs)


        ∞
Wish to compute the conditional pdf of  given   
∞ ∞
     ∞
       
         

     
∴          
        
   



  ≤

→  ∣    ∼ two parameter exponential dist.,
where   the location parameter
    the scale parameter

         
         →         

                 

 

            

(Note) the marginal dist. of  ∼     →     


Given the knowledge that   , the variability in  is
considerably reduced.
Mathematical Statistics 4-6

[Def. 4.2.5]  and  are indep if            ∀ 

•  and  are indep.


         
⇒             
     

         

[Ex. 4.2.6] (Checking independence-Ⅰ)



jpmf                  

 
                 
 

→ mpmf          

  
                 
  
∴  and  are not indep
 
∵       
       
  
        ⋅  
But the relationship (4.2.1) must hold for every choice
of  and  if  and  are to be independent.

[Lemma 4.2.7]  and  are indep iff ∃    and   


st            ∀ 
(p.f) Omit

[Ex. 4.2.8] (Checking independence-Ⅱ)



   
         
      


 
→ 
      
               

∴  and  are indep by Lemma  
Mathematical Statistics 4-7

•  and  are indep.


⇒       on the set     ∈ and ∈
where          and          

• the set where       is not a cross-product


⇒  and  are not indep.

(ex) Ex. 4.2.4 :        ∞

[Ex. 4.2.9] (Joint prob. model)


A student from an elementary school in Kansas City is
randomly selected, and  ≡ no of living parents of the student .
A retiree from Sun City is randomly selected, and
 ≡ no of living parents of the retiree .
⇒ It seems reasonable to assume that these two r.v.s are
indep.
∴                    

                    

[Thm. 4.2.10]  and  are indep.


a.  ∈  ∈    ∈   ∈ 
b.                     where
   is a ft only of  and    is a ft only of 

[Ex. 4.2.11] (Expectation of indep. variables)


  ∼ iid exp 
(ⅰ) From Thm. 4.2.10,
  ≥          ≥               
(ⅱ)                   

[Thm. 4.2.12]  and  are indep. with    and    .


Let       Then          
(p.f)
                           
Mathematical Statistics 4-8

[Ex. 4.2.13] (Mgf of a sum of normal variables)


 ∼       ∼      : indep. Let      

⇒          

  
 
 exp     exp    

 
 
 
  
    
 exp       

∴  ∼          ⇒ Thm. 4.2.14

4.3 Bivariate Transformation

•    : bivariate random vector with a known prob. dist.


Consider a new bivariate random vector    defined by
      and      

  ∈ for  ⊂   iff   ∈  where


              ∈
⇒    ∈      ∈  and the prob. dist. of    is
completely determined by the prob. dist. of   

•     discrete bivariate random vector


                   ∈   
∈ 
     

where             and        

[Ex. 4.3.1] (Dist. of the sum of Poisson variables)


 ∼      ∼     indep.
 
      
 
             ⋯      ⋯
  

Let          , i.e.               


Mathematical Statistics 4-9

(ⅰ) the jpmf of    :


      
                       ⋯      ⋯
      
(ⅱ) the mpmf of 

       
    
     
        
   

        ⋯
         

  
    
 

      
  
              ⋯

∴    ∼     

[Thm. 4.3.2]  ∼      ∼     indep.


⇒    ∼     
(p.f) Use mgf technique.
 
              

By Thm. 4.2.12,            

    

∴    ∼     

•     conti. random vector with jpdf     


Consider the jpdf of   
Let              
                   for some   ∈
Assume that the transformation       ,       defines
a one-to-one transformation of  onto  .
That is, for each   ∈ there is only one   ∈
s.t.             
           

           

 
 

 
    
  
     

 

: the Jacobian of the transformation
Mathematical Statistics 4-10

→ the role played by a derivative in the univariate case


(see p. 51, Thm. 2.1.5)

⇒                 ⋅∣ ∣ 


where ∣ ∣ is the absolute value of 

[Ex. 4.3.3] (Dist. of the product of beta variables)


 ∼       ∼         indep.
the jpdf of    :
           
               ⋅          
            
 

Let    and    
→   → 
 
    
 

 
 


   
   
       
   
   
  ∣∣
∴ the jpdf of  and  :
    
       
              
      

    
 
  
⋅


∴ the mpdf of  :
   
      
      
       



  
 
     

       
let    →      ie     
     

      
            
        

     

      
                
      

    
∴    ∼        ∵  

      
    
Mathematical Statistics 4-11

[Ex. 4.3.4] (Sum and difference of normal variables)


  ∼ iid    
 
 

         ∞    ∞ ∞    ∞

Let          
         

   →   ∞    ∞ ∞    ∞

   


  
   

 
 
 
  
    ∣ ∣  
     
   
   
∴                 ⋅∣ ∣

       
   
  exp        ⋅ 

  


 
   

 

 
   ⋅      
          
a ft of  alone a ft of  alone
⇒  and  are indep. by Lemma 4.2.7
Also,      ∼    
     ∼    

(Note) Sums and differences of indep. normal r.v.s are indep. normal
r.v.s regardless of the means of  and  , so long as
       .

[Thm. 4.3.5]  and  are indep.


   : a f.t only of      : a f.t only of 
⇒      and      are indep.

(p.f)
Assume  and  are continuous r.v.s.
Mathematical Statistics 4-12

For any  ∈  and  ∈  , define


       ≤  and        ≤ 
        ≤    ≤  

   ∈   ∈ 

   ∈    ∈  by Thm 


→             by p   
 

     
    ∈ ⋅   ∈ 
     
∴ By Lemma 4.2.7,  and  are indep.

• If we are interest in       only, then we can choose


      so that the resulting transformation from    to   
is one-to-one on  .

• In many situations, the transformation of interest is not


one-to-one. Just as Thm. 2.1.8 (p. 53) generalized the univariate
method to many-to-one f.t.s, the same can be done here.

      

        ⋅∣  ∣
 

→ See Ex. 4.3.6     one  to  one on      ⋯ 

[Ex. 4.3.6] (Dist. of the ratio of normal variables)


   ∼ iid    
Let       ∣ ∣  ← not one-to-one trans.
∵    →   
   
If we restrict consideration to either positive or negative
values of , then the transformation is one-to-one.
Let         
⇒ ∪∪ ≡    
        
partition of  
        
   ∈         
Mathematical Statistics 4-13

For                    →   


For                     →   
 
  
       


        
 
           ⋅∣ ∣

     


 
  

∴         ⋅∣  ∣    ⋅∣  ∣
 
 
 

  ∞    ∞      ∞

∴ the mpdf of  
 

  let    
    



 


   
    
∞ 
  
 



 




  
∞ 


  
 
 use exponential   

   
 
 
    

 
  ∞    ∞ → Cauchy dist
    

4.4 Hierarchical Models and Mixture Distributions

[Ex. 4.4.1] (Binomial-Poisson hierarchy)


An insect lays a large number of eggs, each surviving with
prob. . On the average, how many eggs will survive?

Let   no, of survivors,   no. of eggs laid


⇒    ∼     ← Hierarchical model
 ∼   
Mathematical Statistics 4-14

[Ex. 4.4.2] (Continuation of Ex. 4.4.1)



            


              

∞    
        
   

              if   


 
   

  multiple by 
    ∞       

 
 
   
let     
    ∞     

 
  
   
   

   
  ∼   

∴     

[Thm. 4.4.3]    any two r.v.s


⇒           provided that the expectations exit.
(p.f) For the conti. case :
                   

               

wrt  wrt   
wrt 

                

(Note) Ex. 4.4.1 →                ∵  ∣  ∼     

  ∵  ∼    

[Def. 4.4.4] A r.v.  is said to have a mixture dist. if the dist. of


 depends on a quantity that also has a dist.
In Ex. 4.4.1, the     dist. is a mixture dist. since it is
the result of combining a     with  ∼    .
Mathematical Statistics 4-15

[Ex. 4.4.5] (Generalization of Ex. 4.4.1)


There are a large number of mothers and one mother is
chosen at random.
  ∼    
   ∼   
 ∼   
⇒             

            
      ∵    ∼    

p. 166 (Ex) Above three-stage model can also be thought of as a


two-stage hierarchy by combining the last two stages.
   ∼   
 ∼   
               ∞ 
∞ ∞
 
       

  ∣      

  
∞
     
  
   
 

 ∞ 
   

      

 

use       

   

 
       
 

  
 
          
   
 
    
  
 
geometric dist

∼          

⇒ Three-stage hierarchy in Ex. 4.4.5 is equivalent to the two-stage
hierarchy   ∼     

 ∼        

 
Mathematical Statistics 4-16

• A generalization of a part of the previous model


⇒ Poisson-gamma mixture
   ∼   
 ∼    
⇒  ∼   (See Exercise 4.32)

• Aside from the advantage in aiding understanding, hierarchical


models can often make calculations easier.
(ex) Noncentral -distribution
 ∼     degree of freedom
  noncentrality parameter extremely messy
∞        expression
  
  ∣      
     


  
   and    
⇒  ∼  

Calculation    looks like quite a chore. However, if we examine


the pdf closely, we see that this is a mixture dist., made up of
central chi-squared densities and Poisson dist.s.
i.e. If we set up the hierarchy   ∼   , then the marginal
 ∼   

dist. of  is given by the above        . ∵           
 

∴                
    ∣     
 

   
∴ Similarly    can be calculated.

[Ex. 4.4.6] (Beta-binomial hierarchy)


 ∣ ∼    
 ∼    

⇒                 

Mathematical Statistics 4-17

[Thm. 4.4.7] (Conditional variance identity)


              ,
provided that the expectations exist.
(p.f)
              ∣  

                  

                       

            
Taking expectation yields
                       

             ⋆
Also,
                    

           

                    

           ⋆⋆

⋆  ⋆⋆ ⇒                   

   

[Ex. 4.4.8] (Continuation of Ex. 4.4.6)


We have          and since  ∼     
 
           

         
Also since    ∼                  
Then
Mathematical Statistics 4-18

   ∣         

     

          

 


              
 
           


 
        

∴                 

       

         

4.5 Covariance and Correlation

• Notation :                       

• Assume that     ∞      ∞.

• We discuss two numerical measures of the strength of a


relationship between two r.v.s, the covariance and correlation.

[Def. 4.5.1] the covariance of  and 


               
       by Thm  

[Def. 4.5.2] the correlation of  and 


   
   → the correlation coefficient
 

    
Mathematical Statistics 4-19

(Note) The sign of     gives information regarding the


relationship between  and  . But a given value of     does
not in itself give information about the strength of the relationship
between  and  . The correlation is always between -1 and 1, with
the values -1 and 1 indicating a perfect linear relationship between
 and  .

[Thm. 4.5.3]           


(p.f)             

           

      

[Ex. 4.5.4] (Correlation-I)


            


(ⅰ)     

             ∼    

 
∴       
 

   
(ⅱ)     

             

 


           ≤  ≤  

  
∴       

  

(ⅲ)       
    


⇒                 

    
    
  
Mathematical Statistics 4-20

[Thm. 4.5.5]  and  are indep. ⇒          



(p.f)                               

[Thm. 4.5.6]  and  : any two r.v.s,  and  : any two constants
⇒                      
If  and  are indep., then                .
(p.f) Omit.

[Thm. 4.5.7] a.   ≤  ≤ 


 ∣  ∣   iff ∃ 
 and           
If   , then    , and if    , then    .
(p.f)
a. Let               
⇒                           

           

Since    ≥            ≤ 


i.e.    ≤     ≤  
   
∴   ≤    ≤ 
 

b. ∣  ∣   iff the discriminant   i.e.    has a single root


               

iff                


⇒                
⇒          with           
   
and     ← the root of     


(Note) This proof can be shortened if we use the Cauchy-Schwarz


Inequality in Section 4.7. → See Ex. 4.7.4
Mathematical Statistics 4-21

[Ex. 4.5.8] (Correlation-II) → similar to Ex. 4.5.4

 
 ∼       ,  ∼    : indep.
 
Let      

             


∵                   

let       ∣ ∣  

   
  
 
          
 

∴                    

   
                            
   
             

               

                  


   

 
                  
 



∴    ← much stronger relationship than Ex. 4.5.4


(Note) Figure 4.5.1


→ In both cases (a) and (b), there is a linearly increasing
relationship between  and  , but the relationship is much
stronger in figure 4.5.1 (b).

• There may be a strong relationship between  and  , but if the


relationship is not linear, the correlation may be small.
Mathematical Statistics 4-22

[Ex. 4.5.9] (Correlation-Ⅲ)

 
 ∼       ,  ∼     : indep.
 
Let      

→                     

Since  ∼                 and since  and 
are indep.,           .
                  

         

            
∴   

[Def. 4.5.10]  ∞      ∞             : real numbers


the bivariate normal pdf with means  and  ,
variances  and  , and correlation 

 
    
→  
    


exp  

           

 
           

     
 
   
  ∞     ∞

Nice properties:
a  ∼     
b  ∼     
c   
d     ∼                 

where   : constant
Mathematical Statistics 4-23

4.6 Multivariate Distributions

• Notation :    ⋯   → multivariate random vector


   ⋯   → the sample

• For any  ⊂  ,
  ∈       
∈
if discrete random vector

 ⋯        ⋯    ⋯  
 
   ⋯ 
if conti random vector

•           
∈ 

∞ ∞
 
∞

∞
      

•   ⋯    
  ⋯ ∈ 
  

∞ ∞
 ⋯ ∞ ∞
    ⋯ 

  ⋯  
•    ⋯  ∣  ⋯    
  ⋯  

[Ex. 4.6.1] (Multivariate pdfs)



                           

   
(ⅰ)  
⋯                      
   







    


Mathematical Statistics 4-24

 




(ⅱ)            
 
 
 
 
 

 

             

     

    
     
    

∞ ∞
(ⅲ)       ∞ ∞
       

 
             
 







  

 
              
 
∞ ∞
     ∞ ∞
       

 
              
 
 



   



     


(ⅳ)    ∣     
   

 
               
     
  
   
           
   

  

 
 

    ∣               
  
 
       
          

 



               
 

    
   
 



 

Mathematical Statistics 4-25


[Def. 4.6.2]     positive integer    ⋯     ≤   ≤   

 

   ⋯   ∼ multinomial dist with  trials and cell probs   ⋯  



 
  
if  ⋯       ⋯    
 ⋯  

   
, 

where   nonnegative integer and   


[Ex. 4.6.3] (Multivariate pmf)


Consider tossing a six-sided die ten times. Suppose the die is

unbalanced so that  observing an          ⋯ 

Let   no. of times  comes up in the ten tosses.
Then  ⋯   ∼ multinomial dist with      
     
          

      

       

 

 

 







•   multinomial coefficient
 ⋯ 
the number of ways that  objects can be divided
into  groups with  in the first group,  in the
second group, …, and  in the  th group

[Thm. 4.6.4] (Multinomial Theorem)


← a generalization of the Binomial Thm. 3.2.2
   positive integer
 : the set of vectors    ⋯      is a nonnegative

integer and  


⇒ For any real numbers  ⋯  ,



  ⋯    
 ⋯ 
 ∈ 



 ⋯ 

Mathematical Statistics 4-26

• Consider some marginal and conditional dist.s for the multinomial


model.

(ⅰ) the marginal dist. of 


Let  be the set of vectors  ⋯    s.t.  is a nonnegative
 
integer and     .

 


       ⋯ 
⋯   ∈

 

 ⋯ 

         
 
⋯   ∈

 
 ⋯  ⋅ 
 ⋯           

 
      


  
   
 
   
∵ 
⋯   ∈
 


 ⋯        ⋯ 
 
   
≡ ∵  ⋯       
 ⋯      

∴  ∼     
∴ Similarly  ∼          ⋯ 

(ⅱ) the conditional pmf of  ⋯    given   


  
   ⋯ 
  ⋯    ⋯  
  ⋯   ∣      
    
      
    

 
   
 

 ⋯         ⋯ 
 
   
∼  dist
  
with    trials and cell probs   ⋯  
     

(ⅲ)           ← Exercise 4.39


Mathematical Statistics 4-27

[Def. 4.6.5]   ⋯    random vectors with   ⋯   


  ⋯   are mutually indep. random vectors

if, for every   ⋯    ,     ⋯     

    

 : one-dimensional ∀ 
→    ⋯    are mutually indep. r.v.s

[Thm. 4.6.6] (Generalization of Thm. 4.2.10)


 ⋯  are mutually indep. r.v.s
 ⋯  are real-valued f.t.s s.t.    is a f.t. only of  
    ⋯ 
⇒     ⋯         ⋯    

[Thm. 4.6.7] (Generalization of Thm. 4.2.12)


 ⋯  are mutually indep. r.v.s with mgfs   
Let     ⋯    Then the mgf  is        ⋯    .

If  ⋯  ∼ iid with    , then          

[Ex. 4.6.8] (Mgf of a sum of gamma variables)


 ∼ indep          ⋯ 
the mgf of     ⋯  
⇒        ⋯   
       ⋯   
      ⋯           
∴  ∼    ⋯     

[Corollary 4.6.9]  ⋯  are mutually indep. r.v.s with mgfs   
 ⋯  and  ⋯  are fixed constants.
Let       ⋯     . Then the mgf of  is
∑ 
        ⋯   
Mathematical Statistics 4-28

∑   
(p.f)            
∑ 
 
    
 ⋯
∑ 
    ⋯   

[Corollary 4.6.10]  ⋯  ∼ indep     


 ⋯  and  ⋯  are fixed constants. Then

   
  
 
     ∼ 

     

 
 

→ A linear combination of indep. normal r.v.s are


normally distributed.
(p.f)

  
 ∼      →      

    
∑ 
   
 
  
exp     ⋯ exp    
  
 ∑  
 exp ∑       
 
   
 
 
∴ ∼        
 

[Thm. 4.6.11] (Generalization of Lemma 4.2.7)


  ⋯   are mutually indep random vectors iff
∃         ⋯  st    ⋯          ⋯     

[Thm. 4.6.12] (Generalization of Thm. 4.3.5)


   ⋯   . are indep. random vectors.
    is a f.t. only of       ⋯ 
Then           ⋯  are mutually indep.
Mathematical Statistics 4-29

•    ⋯    random vector with   ⋯  


Let     ⋯       ⋯ 

⇒   ⋯    

    ⋯    ⋯   ⋯   ∣  ∣ ,
    ⋯          

 
 one  to  one on      ⋯  ⋅⋅⋅ 
   
⋮ ⋮ ⋮ ⋮
where   
⋮ ⋮ ⋮ ⋮
        
  ⋅⋅⋅ 
     

[Ex. 4.6.13] (Multivariate change of variable)


     with
       
                    ∞
Let                     .
→ one-to-one i.e.   
⇒                           

 
   
   
  
   
   
       
⇒                    

⇒         
   
    ∼  

  
∴     are mutually indep.
Mathematical Statistics 4-30

4.7 Inequality

4.7.1 Numerical Inequalities

 
[Lemma 4.7.1]                 
 
   
⇒      ≥  with equality iff   
 
(p.f) Omit.

[Thm. 4.7.2] (Hölder's Inequality)



     ≤      ≤          where     
(p.f)
(ⅰ)    ≤  ≤   and Thm  c p  
⇒    ≤    
   
(ⅱ) Take     
   in Lemma 4.7.1
         
         
⇒   
   
≥
                   

Take expectations of both sides.


   ∵
 
 
≥  
 
         

[Thm. 4.7.3] (Cauchy - Schwarz Inequality)


 
  ≤    ≤        
: a special case of Hölder's Inequality with     

[Ex. 4.7.4] (Covariance Inequality-I) ← Thm. 4.5.7


(ⅰ) Apply the Cauchy-Schwarz Inequality.
 
         ≤          
⇒     ≤  

⇒  ≤  ≤          ≤         


⇒   ≤  ≤ 
Mathematical Statistics 4-31

(ⅱ) In Lemma 4.7.1, equality is attained only if


        
       
∵       equality iff   
   
           
       
⇔  
         
⇔         
i.e. The correlation is ±  iff  and  are linearly related.

• Liapounov's Inequality ←  ≡  in Hölder's Inequality


       ≤               ∞

[Thm. 4.7.5] (Minkowski's Inequality)


         ≤                for  ≤   ∞
(p.f) Omit.

• A version of Hölder's Inequality


 

  ≤    
  
 
             
    

 
 

         → 
 
   ≤ 


4.7.2 Functional Inequalities

[Def. 4.7.6]    is convex


if         ≤            ∀       
   is concave if     is convex.

Convex [Link] lie below lines connecting any two points.


→ See Figure 4.7.1
convex [Link] → "hold water", bowl-shaped (ex)     
concave [Link] → "spil water" (ex)     log 
Mathematical Statistics 4-32

[Thm. 4.7.7]    is a convex f.t.


⇒      ≥    : Jensen’s Inequality
Equality holds iff, for every line    that is tangent to   
at                .
(p.f)
Let    be a tangent line to    at    
Write        for some  and .
By the convexity of ,    ≥   .
⇒     ≥                     
∵  is tangent at  

(Note) (ⅰ)       convex
⇒     ≥   

          convex



  ≥
  

(ⅱ)    is convex if ″   ≥   ∀
   is concave if ″   ≤   ∀
(ⅲ) If  is concave, then     ≤     .

[Ex. 4.7.8] (An inequality for means)


 ⋯   

      ⋯   : arithmetic mean

   ⋯   : geometric mean

   : harmonic mean
 
    
  ⋯ 
    
⇒  ≤  ≤ 
(p.f) Use Jensen’s Inquality.

Let  be a  with range  ⋯  and        .

Since log  is a concave f.t, Jensen's Inequality shows that
 log≤ log   .
Mathematical Statistics 4-33

    log 
 
 
(ⅰ) log   
 
log    log   ≤ log   log 
 
 

∴  ≤ 

   log  

 
≥  log      log
    
(ⅱ)  log   log   log 
    
  log 
∴  log  ≥ log  ⇒  ≤ 

•  : r.v. with finite mean  ,     a nondecreasing f.t.


⇒         ≥ 

∵         
       ⋅  ∞            ⋅ ∞    
≥       ⋅  ∞            ⋅  ∞    
         ∵  is nondecreasing and
    ⋅   ∞        ≤ 

[Thm. 4.7.9] (Covariance Inequality-Ⅱ)


  rv     and     any ft
st             exist
negative corr. between  and 
a.     nondecreasing and     nonincreasing
⇒          ≤             positive corr.
b.    and     both nondecreasing or both nonincreasing
⇒          ≥            

Common questions

Powered by AI

The method of transformations can simplify the study of Poisson distributed variables by reducing complex problems to simpler forms. For instance, when two independent Poisson variables X~Poisson(θ) and Y~Poisson(λ) are added, the resulting sum Z = X + Y is also Poisson distributed with parameter θ + λ . This property allows for the study of compound Poisson processes through relatively straightforward transformations, such as using moment-generating functions or leveraging properties of sums to easily infer about aggregates of Poisson-distributed events, thus simplifying analysis and solution derivation.

Holder's Inequality generalizes Cauchy-Schwarz Inequality by extending it to transformations and variables with different p-norms. While Cauchy-Schwarz is a specific case for p = q = 2, stating that |E[XY]| ≤ (E[X^2])^(1/2)(E[Y^2])^(1/2), Holder's Inequality covers a broader range by stating |E[XY]| <= (E[|X|^p])^(1/p) * (E[|Y|^q])^(1/q), where 1/p + 1/q = 1. This allows it to handle a wider range of mathematical situations involving sums of products, facilitating analysis under more general conditions.

Understanding the conditional distribution of random variables is crucial because it provides insights into how one variable behaves given specific information about another variable. This knowledge is important for predictive modeling and decision making in various fields such as finance, machine learning, and insurance where multi-variate dependencies must be modeled accurately . For instance, the conditional probability density function f(Y|X=x) helps in determining the probability distribution of Y given that X takes on a certain value, which can significantly refine predictions and analyses.

Mixture distributions are utilized in statistical modeling to represent populations that exhibit heterogeneity or are composed of distinct sub-populations, each having its own probability distribution. For example, combining a binomial distribution with a Poisson distribution yields a compound distribution used to model data where the mean of the binomial varies with a Poisson distribution . In practical terms, mixture models can be applied in scenarios like market segmentation, where the overall distribution of consumer preferences is seen as a composition of different underlying preference distributions.

Calculating the variance of composite random variables is fundamental to understanding the variability and reliability of statistical predictions. For two independent random variables X and Y, the variance of their sum is additive, Var(X+Y) = Var(X) + Var(Y). This additive property allows statisticians to account for sources of variation and gauge the uncertainty in composite measures, such as the collective risk in portfolio returns or aggregated error in sensor data fusion, which is integral to precision in forecasting, risk management, and decision-making models.

When two independent normal random variables X and Y are considered, their sum U = X + Y and difference V = X - Y are also independent if X and Y have the same variance . This is a remarkable property of normal distributions which does not extend to other distributions. Mathematically, this implies that the transformation maintaining the joint normality leads to a diagonal covariance matrix for (U, V), which simplifies the joint analysis, eases the calculation of joint probabilities, and enhances decomposition into independent components for further statistical inferences.

The correlation coefficient, denoted as ρ, measures the linear relationship between two random variables X and Y. It provides information about the direction and strength of the relationship. A positive correlation indicates that as one variable increases, the other tends to increase, whereas a negative correlation suggests that as one variable increases, the other tends to decrease. Values close to -1 or 1 indicate a strong linear relationship, while values near 0 indicate a weak linear relationship . However, the correlation does not imply causation and does not measure non-linear relationships.

If X and Y are independent, the joint PDF f(x, y) can be expressed as the product of their marginal PDFs, i.e., f(x, y) = f_X(x)f_Y(y) for all values of x and y . This means that knowing the value of one variable does not provide any information about the other. Independently, their conditional PDFs are also equal to the marginal PDFs, e.g., f(y|x) = f_Y(y). This property is essential for simplification in statistical computations involving independent random variables.

Jensen's Inequality states that for a convex function g, E[g(X)] ≥ g(E[X]). This inequality shows that the expected value of a convex function of a random variable is at least the convex function of the expected value of that variable. It provides essential insights into the behavior of random variables and expectations under convex transformations, indicating that convex transformations tend to amplify variability, while in a practical scenario, it can be used to derive bounds or estimate the likelihood of certain events when dealing with convex loss functions or utility functions.

Bivariate transformations are significant in statistical analysis because they allow the study of relationships between two random variables by transforming them into a new set of variables that might be easier to analyze or interpret . For instance, transformation can help in finding the distribution of a sum or a product of variables, which is a common requirement in many statistical problems. The probability distribution of the transformed variables is fully determined by the original joint distribution, enabling new insights and simplifications in handling complex data forms.

You might also like