Probability I: Formula Sheet
Compiled from Course Notes provided earlier
1. Foundations of Probability
Axioms & Basic Properties
Let Ω be the sample space and A, B be events.
- Axioms:
1. P (A) ≥ 0
2. P (Ω) = 1
P∞
3. If A1 , A2 , . . . are disjoint, P (∪∞
i=1 Ai ) = i=1 P (Ai ).
- Complement Rule: P (Ac ) = 1 − P (A)
- Addition Rule: P (A ∪ B) = P (A) + P (B) − P (A ∩ B)
- Union Bound: P (∪ni=1 Ai ) ≤ ni=1 P (Ai )
P
Conditional Probability & Independence
P (A∩B)
- Definition: P (A|B) = P (B) , for P (B) > 0.
- Multiplication Rule: P (A ∩ B) = P (A|B)P (B).
P
- Total Probability: If {Ai } partition Ω, then P (B) = i P (B|Ai )P (Ai ).
- Bayes’ Theorem:
P (B|Ak )P (Ak )
P (Ak |B) = P
j P (B|Aj )P (Aj )
- Independence: A, B are independent iff P (A ∩ B) = P (A)P (B).
Counting
n!
- Permutations: n Pk = (n−k)! (Order matters)
- Combinations: nk = k!(n−k)!
n!
(Order does not matter)
1
2. Random Variables (General)
Definitions
- CDF: F (x) = P (X ≤ x). Properties: Non-decreasing, right-continuous, limits are 0 and 1.
- Expectation (Linearity): E[aX + bY ] = aE[X] + bE[Y ].
- Variance: V (X) = E[(X − µ)2 ] = E[X 2 ] − (E[X])2 .
p
- Standard Deviation: σX = V (X).
- Scaling Variance: V (aX + b) = a2 V (X).
3. Discrete Distributions
Let X be a discrete random variable with PMF p(x) = P (X = x).
Distribution PMF p(k) Mean E[X] Variance
V (X)
Bernoulli(p) pk (1 − p)1−k for k ∈ {0, 1} p p(1 − p)
(1 trial)
n
pk (1 − p)n−k
Binomial(n, p) k np np(1 − p)
(n trials, k successes) k = 0, . . . , n
1−p 1−p
Geometric(p) (1 − p)k p p p2
(Failures before 1st suc- k = 0, 1, . . .
cess)
k
Poisson(λ) e−λ λk! λ λ
(Rare events) k = 0, 1, . . .
k+r−1 r
k r(1−p) r(1−p)
Neg. Binomial(r, p) r−1 p (1 − p) p p2
(Failures before r-th k = 0, 1, . . .
success)
−K
(Kk )(Nn−k )
Hypergeometric N nK
N Complex
(n)
(N total, K success, n
drawn)
Note: For Geometric, if Y is the trial number of the 1st success (Y = X + 1), then P (Y = k) =
(1 − p)k−1 p for k ≥ 1, E[Y ] = 1/p, V (Y ) = (1 − p)/p2 .
2
4. Continuous Distributions
Rb
Let X be continuous with PDF f (x). P (a ≤ X ≤ b) = a f (x)dx.
Distribution PDF f (x) Mean Variance
1 a+b (b−a)2
Uniform(a, b) b−a for a ≤ x ≤ b 2 12
Exponential(λ) λe−λx for x > 0 1
λ
1
λ2
(x−µ)2
Normal(µ, σ 2 ) √1 e− 2σ 2 µ σ2
σ 2π
Gamma(α, β) 1 α−1 e−x/β αβ 2
β α Γ(α) x αβ
(Shape α, Scale β) x>0
α1 −1 (1 α1
Beta(α1 , α2 ) 1
B(α1 ,α2 ) x − x)α2 −1 α1 +α2 See Notes
0<x<1
1
Cauchy π(1+x2 )
Undefined Undefined
5. Joint Distributions & Independence
Properties
- Covariance: Cov(X, Y ) = E[(X − µX )(Y − µY )] = E[XY ] − E[X]E[Y ].
- Correlation: ρ(X, Y ) = Cov(X,Y )
σX σY , where −1 ≤ ρ ≤ 1.
P P P
- Variance of Sum: V ( Xi ) = V (Xi ) + 2 i<j Cov(Xi , Xj ).
- Independence implies: E[XY ] = E[X]E[Y ], Cov(X, Y ) = 0, and fX,Y (x, y) = fX (x)fY (y).
Transformations
If Y = g(X) is a transformation:
- One Variable: fY (y) = fX (g −1 (y)) d −1
dy g (y) .
1 y−b
- Linear Transformation: If Y = aX + b, fY (y) = |a| fX ( a ).
- Bivariate Normal: If (X, Y ) are Bivariate Normal, then X and Y are independent if and
only if ρ = 0.
3
6. Moment Generating Functions (MGF)
Definition: MX (t) = E[etX ]. Uniqueness: MGF uniquely determines the distribution.
- Properties:
(k)
- MX (0) = E[X k ] (k-th moment).
- If Y = aX + b, MY (t) = etb MX (at).
- If X, Y independent, MX+Y (t) = MX (t)MY (t).
- Common MGFs:
- Bernoulli: 1 − p + pet
- Binomial: (1 − p + pet )n
t −1)
- Poisson: eλ(e
λ
- Exponential: λ−t (for t < λ)
1 2 t2
- Normal: eµt+ 2 σ
- Gamma: (1 − βt)−α
7. Special Sampling Distributions
- Chi-square (χ2k ): Sum of squares of k independent standard normals.
k
X
Y = Zi2 ∼ χ2k , E[Y ] = k, V (Y ) = 2k
i=1
q
- t-distribution (tk ): Ratio of Standard Normal to χ2k /k.
Z
T =p ∼ tk
Y /k
- F-distribution (Fk1 ,k2 ): Ratio of two scaled Chi-squares.
X1 /k1
F = ∼ Fk1 ,k2
X2 /k2