Multinomial Distribution
1 Idea
The multinomial distribution generalizes the binomial distribution.
A binomial distribution counts outcomes when each trial has only two possible categories, such as
success and failure. A multinomial distribution counts outcomes when each trial has more than
two possible categories.
Main Idea
Suppose an experiment is repeated n independent times. Each trial can result in exactly one
of k mutually exclusive categories.
If the probability of category i is pi , then the random vector
(X1 , X2 , . . . , Xk )
counts how many times each category occurs.
Here:
Xi = number of outcomes of type i.
2 Definition
Multinomial Random Vector
Let n independent trials be performed. Each trial has one of k possible outcomes with proba-
bilities
p1 , p 2 , . . . , p k ,
where
p1 + p2 + · · · + pk = 1.
If Xi is the number of times outcome i occurs, then
(X1 , X2 , . . . , Xk )
has a multinomial distribution.
We write
(X1 , X2 , . . . , Xk ) ∼ Multinomial(n; p1 , p2 , . . . , pk ).
3 Conditions
Multinomial Conditions
1. The number of trials n is fixed.
2. Each trial has k possible categories.
1
3. The categories are mutually exclusive: one trial can produce only one category.
4. The trials are independent.
5. The category probabilities p1 , p2 , . . . , pk stay constant.
6. The counts x1 , x2 , . . . , xk are nonnegative integers and satisfy
x1 + x2 + · · · + xk = n.
4 Probability Mass Function
Suppose we want exactly
x1 outcomes of type 1, x2 outcomes of type 2, ..., xk outcomes of type k.
The multinomial probability mass function is
Multinomial PMF
n!
P (X1 = x1 , . . . , Xk = xk ) = px1 px2 · · · pxk k
x1 !x2 ! · · · xk ! 1 2
where
x1 + x2 + · · · + xk = n, p1 + p2 + · · · + pk = 1.
Meaning of Each Part
n!
x1 !x2 ! · · · xk !
counts how many different orders can produce the same category counts.
px1 1 px2 2 · · · pxk k
is the probability of one particular order with those counts.
5 Relation to Other Distributions
Connections
• When k = 2, the multinomial distribution becomes the binomial distribution.
• Each individual count Xi follows a binomial distribution:
Xi ∼ Binomial(n, pi ).
• The counts are not independent, because
X1 + X2 + · · · + Xk = n.
2
6 Mean and Variance
For a multinomial random vector
(X1 , X2 , . . . , Xk ) ∼ Multinomial(n; p1 , p2 , . . . , pk ),
each count Xi behaves like a binomial count when viewed by itself.
Mean and Variance
E(Xi ) = npi
Var(Xi ) = npi (1 − pi )
For two different categories i ̸= j, the counts move in opposite directions. If one category count is
larger, the others must share fewer remaining trials.
Covariance
Cov(Xi , Xj ) = −npi pj , i ̸= j.
7 Example 1: Rolling a Fair Die
Suppose a fair die is rolled 10 times. Then there are k = 6 categories:
1, 2, 3, 4, 5, 6.
For a fair die,
1
p1 = p2 = p3 = p4 = p5 = p6 = .
6
Find the probability of getting:
2 ones, 3 threes, 1 four, 4 sixes.
This means
x1 = 2, x2 = 0, x3 = 3, x4 = 1, x5 = 0, x6 = 4.
Check the total:
2 + 0 + 3 + 1 + 0 + 4 = 10.
Using the multinomial PMF,
( ) ( )0 ( )3 ( )1 ( )0 ( )4
10! 1 2 1 1 1 1 1
P =
2!0!3!1!0!4! 6 6 6 6 6 6
( )10
10! 1
= .
2!3!1!4! 6
Now compute the coefficient:
10! 3628800
= = 12600.
2!3!1!4! (2)(6)(1)(24)
Therefore,
( )10
1 12600
P = 12600 = ≈ 0.000208.
6 60466176
3
Answer
P ≈ 0.000208
8 Example 2: Email Classifier
A spam classifier places incoming emails into three categories:
S = Spam, P = Promotions, I = Important.
The probabilities are
P (S) = 0.5, P (P ) = 0.3, P (I) = 0.2.
A sample of 10 emails is taken. Find the probability that
4 are Spam, 3 are Promotions, 3 are Important.
Here:
n = 10, k = 3,
xS = 4, xP = 3, xI = 3.
Check the total:
4 + 3 + 3 = 10.
Using the multinomial PMF,
10!
P (XS = 4, XP = 3, XI = 3) = (0.5)4 (0.3)3 (0.2)3 .
4!3!3!
Compute the coefficient:
10! 3628800
= = 4200.
4!3!3! (24)(6)(6)
Then
P = 4200(0.5)4 (0.3)3 (0.2)3
= 4200(0.0625)(0.027)(0.008)
= 0.0567.
Answer
P = 0.0567
9 Summary
One-Box Summary
n!
P (X1 = x1 , . . . , Xk = xk ) = px1 1 px2 2 · · · pxk k
x1 !x2 ! · · · xk !
4
x1 + x2 + · · · + xk = n, p1 + p2 + · · · + pk = 1.
E(Xi ) = npi , Var(Xi ) = npi (1 − pi ).