Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Probability
Brian Caffo
Probability
Mathematical Biostatistics Boot Camp: Lecture 2, Probability
Random
variables
PMFs and
PDFs Brian Caffo
CDFs, survival
functions and
quantiles Department of Biostatistics
Johns Hopkins Bloomberg School of Public Health
Summary
Johns Hopkins University
August 29, 2012
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Table of contents
Probability
Brian Caffo
Probability
1 Probability
Random
variables
PMFs and
PDFs 2 Random variables
CDFs, survival
functions and
quantiles 3 PMFs and PDFs
Summary
4 CDFs, survival functions and quantiles
5 Summary
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Probability measures
Probability
Brian Caffo
Probability
Random
variables
PMFs and
A probability measure, P, is a real valued function from the collection of possible
PDFs events so that the following hold
CDFs, survival
functions and 1. For an event E ⊂ Ω, 0 ≤ P(E ) ≤ 1
quantiles
Summary
2. P(Ω) = 1
3. If E1 and E2 are mutually exclusive events P(E1 ∪ E2 ) = P(E1 ) + P(E2 ).
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Additivity
Probability
Brian Caffo
Probability Part 3 of the definition implies finite additivity
Random
variables n
X
PMFs and P(∪ni=1 Ai ) = P(Ai )
PDFs
i=1
CDFs, survival
functions and
quantiles where the {Ai } are mutually exclusive.
Summary This is usually extended to countable additivity
∞
X
P(∪∞
i=1 Ai ) = P(Ai )
i=1
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Note
Probability
Brian Caffo
Probability
Random
variables • P is defined on F a collection of subsets of Ω
PMFs and
PDFs
• Example Ω = {1, 2, 3} then
CDFs, survival
functions and
quantiles
F = {∅, {1}, {2}, {3}, {1, 2}, {1, 3}, {2, 3}, {1, 2, 3}} .
Summary
• When Ω is a continuous set, the definition gets much trickier. In this case we
assume that F is sufficiently rich so that any set that we’re interested in will
be in it.
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Consequences
Probability
Brian Caffo
Probability You should be able to prove all of the following:
Random
variables
• P(∅) = 0
PMFs and • P(E ) = 1 − P(E c )
PDFs
CDFs, survival • P(A ∪ B) = P(A) + P(B) − P(A ∩ B)
functions and
quantiles • if A ⊂ B then P(A) ≤ P(B)
Summary
• P (A ∪ B) = 1 − P(Ac ∩ B c )
• P(A ∩ B c ) = P(A) − P(A ∩ B)
Pn
• P(∪ni=1 Ei ) ≤ i=1 P(Ei )
• P(∪ni=1 Ei ) ≥ maxi P(Ei )
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Probability
Random
variables
Proof that P(E ) = 1 − P(E c )
PMFs and
PDFs
CDFs, survival
1 = P(Ω)
functions and
quantiles = P(E ∪ E c )
Summary = P(E ) + P(E c )
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability Pn
Proof that P(∪ni=1 Ei ) ≤ i=1 P(Ei )
Brian Caffo
Probability
Random P(E1 ∪ E2 ) = P(E1 ) + P(E2 ) − P(E1 ∩ E2 )
variables
PMFs and ≤ P(E1 ) + P(E2 )
PDFs
CDFs, survival
functions and
Assume the statement is true for n − 1 and consider n
quantiles
Summary P(∪ni=1 Ei ) ≤ P(En ) + P(∪n−1
i=1 Ei )
n−1
X
≤ P(En ) + P(Ei )
i=1
n
X
= P(Ei )
i=1
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Probability
Random
variables
PMFs and The National Sleep Foundation ([Link]) reports that around
PDFs
CDFs, survival
3% of the American population has sleep apnea. They also report that around 10%
functions and
quantiles
of the North American and European population has restless leg syndrome.
Summary
Similarly, they report that 58% of adults in the US experience insomnia. Does this
imply that 71% of people will have at least one sleep problems of these sorts?
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example continued
Probability
Brian Caffo Answer: No, the events are not mutually exclusive. To elaborate let:
Probability
Random
A1 = {Person has sleep apnea}
variables
A2 = {Person has RLS}
PMFs and
PDFs A3 = {Person has insomnia}
CDFs, survival
functions and
quantiles Then (work out the details for yourself)
Summary
P(A1 ∪ A2 ∪ A3 ) = P(A1 ) + P(A2 ) + P(A3 )
− P(A1 ∩ A2 ) − P(A1 ∩ A3 ) − P(A2 ∩ A3 )
+ P(A1 ∩ A2 ∩ A3 )
= .71 + Other stuff
where the “Other stuff” has to be less than 0
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Random variables
Probability
Brian Caffo
Probability
Random • A random variable is a numerical outcome of an experiment.
variables
PMFs and
• The random variables that we study will come in two varieties, discrete or
PDFs continuous.
CDFs, survival
functions and • Discrete random variable are random variables that take on only a countable
quantiles number of possibilities.
Summary
• P(X = k)
• Continuous random variable can take any value on the real line or some subset
of the real line.
• P(X ∈ A)
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Examples of variables that can be thought of as
Probability
Brian Caffo
random variables
Probability
Random
variables
PMFs and
PDFs
• The (0 − 1) outcome of the flip of a coin
CDFs, survival
functions and
quantiles • The outcome from the roll of a die
Summary • The BMI of a subject four years after a baseline measurement
• The hypertension status of a subject randomly drawn from a population
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
PMF
Probability
Brian Caffo
Probability
Random
variables
PMFs and A probability mass function evaluated at a value corresponds to the probability that
PDFs
a random variable takes that value. To be a valid pmf a function, p, must satisfy
CDFs, survival
functions and 1 p(x) ≥ 0 for all x
quantiles P
Summary 2 x p(x) = 1
The sum is taken over all of the possible values for x.
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Probability
Random
variables
Let X be the result of a coin flip where X = 0 represents tails and X = 1
PMFs and represents heads.
PDFs
p(x) = (1/2)x (1/2)1−x for x = 0, 1
CDFs, survival
functions and
quantiles Suppose that we do not know whether or not the coin is fair; Let θ be the
Summary probability of a head expressed as a proportion (between 0 and 1).
p(x) = θx (1 − θ)1−x for x = 0, 1
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Probability For the unfair coin
Random
variables
PMFs and
p(0) = 1 − θ and p(1) = θ
PDFs
CDFs, survival
functions and
so
quantiles
Summary
p(x) > 0 for x = 0, 1
and
p(0) + p(1) = θ + (1 − θ) = 1
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
PDF
Probability
Brian Caffo
Probability
Random
variables A probability density function (pdf), is a function associated with a continuous
PMFs and random variable
PDFs
CDFs, survival
Areas under pdfs correspond to probabilities for that random variable
functions and
quantiles
To be a valid pdf, a function f must satisfy
Summary
1 f (x) ≥ 0 for all x
R∞
2 −∞ f (x)dx = 1
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Assume that the time in years from diagnosis until death of persons with a specific
Probability
kind of cancer follows a density like
Random
variables (
e −x/5
PMFs and for x > 0
PDFs f (x) = 5
0 otherwise
CDFs, survival
functions and
quantiles
More compactly written: f (x) = 15 e −x/5 for x > 0.
Summary
Is this a valid density?
1 e raised to any power is always positive
2 Z ∞ Z ∞ ∞
f (x)dx = e −x/5 /5dx = −e −x/5 =1
0 0 0
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example continued
Probability
Brian Caffo
Probability
Random
variables
PMFs and
PDFs
CDFs, survival What’s the probability that a randomly selected person from this distribution
functions and
quantiles survives more than 6 years?
Summary
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example continued
Probability
Brian Caffo
Probability
Random What’s the probability that a randomly selected person from this distribution
variables
survives more than 6 years?
PMFs and
PDFs
∞
e −t/5 ∞
Z
dt = −e −t/5 = e −6/5 ≈ .301.
CDFs, survival
functions and P(X ≥ 6) =
quantiles 6 5 6
Summary
Approximation in R
pexp(6, 1/5, [Link] = FALSE)
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example continued
Probability
Brian Caffo
0.20
Probability
Random
0.15
variables
PMFs and
PDFs
density
0.10
CDFs, survival
functions and
quantiles
Summary
0.05
0.00
0 5 10 15 20
Survival time in years
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
CDF and survival function
Probability
Brian Caffo
Probability
• The cumulative distribution function (CDF) of a random variable X is
Random
variables defined as the function
PMFs and F (x) = P(X ≤ x)
PDFs
CDFs, survival • This definition applies regardless of whether X is discrete or continuous.
functions and
quantiles • The survival function of a random variable X is defined as
Summary
S(x) = P(X > x)
• Notice that S(x) = 1 − F (x)
• For continuous random variables, the PDF is the derivative of the CDF
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example continued
Probability
Brian Caffo
0.20
Probability
Random
0.15
variables
PMFs and
PDFs
density
0.10
CDFs, survival
functions and
quantiles
Summary
0.05
0.00
0 5 10 15 20
Survival time in years
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Probability
Random
variables
PMFs and
PDFs
CDFs, survival What are the survival function and CDF from the exponential density considered
functions and
quantiles before?
Summary
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Probability What are the survival function and CDF from the exponential density considered
Random
variables
before? Z ∞ −t/5
e ∞
PMFs and S(x) = dt = −e −t/5 = e −x/5
PDFs
x 5 x
CDFs, survival
functions and hence we know that
quantiles
F (x) = 1 − S(x) = 1 − e −x/5
Summary
Notice that we can recover the PDF by
d
f (x) = F 0 (x) = (1 − e −x/5 ) = e −x/5 /5
dx
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Quantiles
Probability
Brian Caffo
Probability
Random
variables
PMFs and • The αth quantile of a distribution with distribution function F is the point xα
PDFs
so that
CDFs, survival
functions and F (xα ) = α
quantiles
Summary • A percentile is simply a quantile with α expressed as a percent
• The median is the 50th percentile
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example continued
Probability
Brian Caffo
0.20
Probability
Random
0.15
variables
PMFs and
PDFs
density
0.10
CDFs, survival
functions and
quantiles
Summary
0.05
0.00
0 5 10 15 20
Survival time in years
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Probability
Random
variables
PMFs and
PDFs
CDFs, survival • What is the 25th percentile of the exponential survival distribution considered
functions and
quantiles before?
Summary
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Example
Probability
Brian Caffo
Probability • What is the 25th percentile of the exponential survival distribution considered
Random before?
variables
PMFs and
• We want to solve (for x)
PDFs
CDFs, survival
functions and
.25 = F (x)
quantiles
= 1 − e −x/5
Summary
resulting in the solution x = − log(.75) × 5 ≈ 1.44
• Therefore, 25% of the subjects from this population live less than 1.44 years
• R can approximate exponential quantiles for you
qexp(.25, 1/5)
Mathematical
Biostatistics
Boot Camp:
Lecture 2,
Probability models
Probability
Brian Caffo
Probability
Random
variables
• You might be wondering at this point “I’ve heard of a median before, it didn’t
PMFs and
require integration. Where’s the data?”
PDFs
• We’re referring to are population quantities. Therefore, the median being
CDFs, survival
functions and discussed is the population median.
quantiles
Summary
• A probability model connects the data to the population to the population
using assumptions.
• Therefore the median we’re discussing is the estimand, the sample median
will be the estimator