Chapter 2

Basic Concepts of Probability

Notes, Worked Examples & Practice Questions — Master's Level

Covers: probability foundations · counting techniques · conditional probability & Bayes' Rule · random variables ·
expectation · variance · covariance & correlation
Contents

1. Basic Concepts of Probability

2. Probability Counting Techniques
3. Calculation of Probabilities
4. Conditional Probability, Independence, and Bayes' Rule
5. Random Variables and Probability Distributions
6. Expectation of Random Variables
7. Variance
8. Covariance and Correlation
9. Summary Formula Sheet
10. Practice Questions (64 questions, all sub-topics)
1. Basic Concepts of Probability

Probability theory provides the mathematical foundation for reasoning about uncertainty. A random experiment is any
process whose outcome cannot be predicted with certainty in advance, but whose set of possible outcomes is known.
1.1 Sample Space and Events
• Sample space (S): the set of all possible outcomes of a random experiment.
• Event (A): any subset of the sample space — a collection of outcomes we're interested in.
• Simple event: an event containing exactly one outcome. Compound event: contains two or more outcomes.
Worked Example
Experiment: toss a fair coin twice, recording the sequence of Heads (H) / Tails (T).
S = { HH, HT, TH, TT }
Event A = 'at least one Head' = {HH, HT, TH}. Event B = 'both the same' = {HH, TT}.
1.2 Axioms of Probability (Kolmogorov)
Any valid probability function P must satisfy three axioms:
• Axiom 1: P(A) ≥ 0 for every event A.
• Axiom 2: P(S) = 1 (something in the sample space must happen).
• Axiom 3: For mutually exclusive events A1, A2, A3, …: P(A1∪A2∪A3∪…) = P(A1)+P(A2)+P(A3)+…
Note: Every other probability rule — the complement rule, the addition rule, conditional probability — can be derived from these three
axioms. They are the starting point, not just a convenient checklist.
1.3 Approaches to Assigning Probability

Approach Definition Example

Classical P(A) = (favourable outcomes)/(total P(rolling a 4 on a fair die) = 1/6
outcomes), assuming all outcomes equally
likely
Empirical (relative P(A) ≈ (times A occurred)/(total trials), based P(defective item) estimated from a
frequency) on observed data sample of 500 inspected units
Subjective A degree of belief based on judgement, An analyst's estimate that a project
expertise, or partial information finishes on time

1.4 Complement and Addition Rules

Complement rule: P(Aᶜ) = 1 − P(A)
Addition rule (general): P(A∪B) = P(A) + P(B) − P(A∩B)
Addition rule (mutually exclusive A, B): P(A∪B) = P(A) + P(B), since P(A∩B) = 0
Worked Example
In a cohort of 200 Master's students, 120 are enrolled in a Statistics elective (A), 90 are enrolled in a Machine Learning
elective (B), and 50 are enrolled in both.
P(A) = 120/200 = 0.60, P(B) = 90/200 = 0.45, P(A∩B) = 50/200 = 0.25
P(A∪B) = 0.60 + 0.45 − 0.25 = 0.80
So 80% of the cohort takes at least one of the two electives, meaning 20% take neither.
1.5 Mutually Exclusive vs. Independent Events (preview)
These two ideas are often confused. Mutually exclusive events cannot both occur (P(A∩B)=0); independent events
can both occur, but the occurrence of one doesn't change the probability of the other. In fact, two events with
nonzero probability can never be both mutually exclusive and independent simultaneously — if A and B are mutually
exclusive, then knowing A occurred tells you B definitely did not, which is the opposite of independence.
Independence is covered formally in Section 4.
2. Probability Counting Techniques

When outcomes are equally likely, computing a classical probability reduces to counting: how many outcomes favour
the event, out of how many total outcomes. Counting techniques make this tractable for large sample spaces.
2.1 The Multiplication (Fundamental Counting) Principle
If a task consists of k steps, with n1 ways to do the first, n2 ways to do the second (regardless of the first), and so on,
the total number of ways to complete the whole task is:
n1 × n2 × n3 × … × nk
Worked Example
A 4-digit PIN uses digits 0–9, with repetition allowed.
Total PINs = 10 × 10 × 10 × 10 = 10,000
2.2 Permutations (order matters)
nPr = n! / (n−r)!
Used when arranging r items out of n distinct items, where the order of arrangement matters (e.g. 1st, 2nd, 3rd place;
a sequence of steps in a pipeline).
Worked Example
In how many ways can a lecturer award distinct 1st, 2nd and 3rd prizes to 3 students chosen from a class of 10?
10P3 = 10!/(10−3)! = 10×9×8 = 720
Permutations with repeated items
Number of distinct arrangements = n! / (n1! × n2! × … × nk!)
where n1, n2, … are the counts of each repeated item.
Worked Example
How many distinct arrangements are there of the letters in 'STATISTICS' (10 letters: S×3, T×3, A×1, I×2, C×1)?
10! / (3! × 3! × 1! × 2! × 1!) = 3,628,800 / 72 = 50,400
2.3 Combinations (order does not matter)
nCr = n! / [r!(n−r)!]
Used when selecting r items out of n distinct items where order doesn't matter (e.g. choosing a committee, a subset of
features, a hand of cards).
Worked Example
A project team of 5 is chosen (unordered) from 12 candidates.
12C5 = 12!/(5!×7!) = 792
Note: Confusing permutations and combinations is the single most common counting error. Ask: does swapping the order of two selected
items create a genuinely different outcome? If yes → permutation. If no → combination.
2.4 The Pigeonhole Principle
If n items are placed into m containers (n > m), at least one container must hold more than one item. A simple but
powerful proof technique.
Worked Example
Prove that among any 13 people, at least two share a birth month.
There are 12 possible birth months (containers) and 13 people (items). By the pigeonhole principle, at least one
month must contain at least 2 people.
3. Calculation of Probabilities

Combining counting techniques with the classical definition of probability lets us compute probabilities for
compound, real-world scenarios involving sampling without replacement, 'at least one' events, and combinatorial
selections.
3.1 Classical Probability via Counting
P(A) = (number of outcomes favouring A) / (total number of equally likely outcomes)
Worked Example
A committee of 5 is selected at random from 7 men and 6 women (13 people total). Find the probability the
committee has exactly 3 women.
P = [C(6,3) × C(7,2)] / C(13,5) = [20 × 21] / 1287 = 420/1287 = 0.326
3.2 'At Least One' Problems via the Complement
Directly computing P(at least one X) is often harder than computing its complement, P(no X at all), and subtracting
from 1.
P(at least one) = 1 − P(none)
Worked Example
20 applicants (12 men, 8 women) apply for 4 positions filled at random. Find the probability that at least one woman
is selected.
P(no women) = C(12,4)/C(20,4) = 495/4845 = 0.1022
P(at least one woman) = 1 − 0.1022 = 0.8978
3.3 Sampling Without Replacement
When items are drawn without replacement, probabilities change at each stage — this is naturally handled by
combinations (order-independent) or by multiplying successive conditional probabilities (order-dependent).
Worked Example
A box has 10 red, 8 blue, 6 green balls (24 total). Three are drawn without replacement. Find P(all three different
colours).
P = [C(10,1)×C(8,1)×C(6,1)] / C(24,3) = 480/2024 = 0.237
4. Conditional Probability, Independence, and Bayes' Rule

4.1 Conditional Probability

P(A|B) = P(A∩B) / P(B), provided P(B) > 0
P(A|B) is read 'the probability of A given that B has occurred' — it restricts the sample space to only those outcomes
in which B occurred.
Note: P(A|B) ≠ P(B|A) in general. E.g. P(rain | clouds) is high, but P(clouds | rain) is even higher (near 1) — these describe
different conditional relationships and are not interchangeable.
4.2 The Multiplication Rule
P(A∩B) = P(A|B)×P(B) = P(B|A)×P(A)
Worked Example
A batch of 20 components has 4 defective. Two are drawn without replacement. Find the probability both are
defective.
P(D1∩D2) = P(D1) × P(D2|D1) = (4/20) × (3/19) = 12/380 = 0.0316
4.3 Independence
Events A and B are independent if the occurrence of one does not change the probability of the other:

P(A|B) = P(A) ⟺ P(B|A) = P(B) ⟺ P(A∩B) = P(A)×P(B)

Worked Example
P(A)=0.5, P(B)=0.4, P(A∩B)=0.2. Are A and B independent?
P(A)×P(B) = 0.5×0.4 = 0.20 = P(A∩B) → A and B ARE independent
Note: Independence is a numerical coincidence you check with the multiplication test above — never assume it just because two events
seem unrelated in a real-world sense.
4.4 The Law of Total Probability
If A1, A2, …, An partition the sample space (mutually exclusive, collectively exhaustive), then for any event B:
P(B) = Σ P(B|Ai) × P(Ai)
Worked Example
60% of a factory's output comes from Machine 1 (2% defect rate) and 40% from Machine 2 (5% defect rate). Find the
overall probability a randomly selected item is defective.
P(D) = P(D|M1)P(M1) + P(D|M2)P(M2) = (0.02)(0.6) + (0.05)(0.4) = 0.012+0.020 = 0.032
4.5 Bayes' Rule
P(Ai|B) = [P(B|Ai)×P(Ai)] / Σ[P(B|Aj)×P(Aj)]
Bayes' Rule lets us 'reverse' a conditional probability — updating our belief about a cause (Ai) after observing an effect
(B).
Worked Example
Continuing the factory example: given that a randomly selected item is defective, find the probability it came from
Machine 2.
P(M2|D) = P(D|M2)P(M2) / P(D) = 0.020 / 0.032 = 0.625
Even though Machine 2 produces less output overall, its higher defect rate means most defective items actually come
from it.
Worked Example
Medical screening: 1% of a population has a disease. A test is 98% sensitive and 95% specific. Given a positive test,
find P(disease | positive).
P(Disease) = 0.01, P(No Disease) = 0.99
P(Pos|Disease) = 0.98, P(Pos|No Disease) = 1 − 0.95 = 0.05
P(Pos) = (0.98)(0.01) + (0.05)(0.99) = 0.0098 + 0.0495 = 0.0593
P(Disease|Pos) = 0.0098 / 0.0593 = 0.165 (only 16.5%!)
Despite a seemingly accurate test, most positive results are false positives because the disease is rare — this is the
base-rate effect, a critical consideration in screening-programme design.
5. Random Variables and Probability Distributions

5.1 Definitions
• A random variable (RV) X is a function that assigns a numerical value to each outcome of a random experiment.
• Discrete RV: takes a countable set of values (e.g. number of defects, number of clicks). Described by a
probability mass function (pmf).
• Continuous RV: takes any value in an interval (e.g. height, time, temperature). Described by a probability density
function (pdf).
5.2 Probability Mass Function (pmf)
For discrete X, the pmf P(X=x) must satisfy:
P(X=x) ≥ 0 for all x, and Σ P(X=x) = 1
Worked Example
A discrete RV X has pmf P(X=x) = c(5−x) for x=1,2,3,4. Find c.
Σ P(X=x) = c[(5-1)+(5-2)+(5-3)+(5-4)] = c(4+3+2+1) = 10c = 1 → c = 0.1
5.3 Probability Density Function (pdf)
For continuous X, the pdf f(x) must satisfy:
f(x) ≥ 0 for all x, and ∫f(x)dx = 1 (over the support)
Note that P(X=x) = 0 for any single value x — probability is only meaningful over an interval, obtained by integrating
f(x).
Worked Example
Verify that f(x) = 3x², 0≤x≤1, is a valid density, and find P(0.2 < X < 0.5).

∫₀¹ 3x² dx = [x³]₀¹ = 1 ✓ valid density

P(0.2<X<0.5) = ∫₀.₂⁰·⁵ 3x² dx = [x³]₀.₂⁰·⁵ = 0.125 − 0.008 = 0.117
5.4 Cumulative Distribution Function (CDF)
F(x) = P(X ≤ x)
Every valid CDF is non-decreasing, satisfies F(−∞)=0 and F(∞)=1, and is right-continuous. For continuous X, f(x) =
dF(x)/dx.
6. Expectation of Random Variables

6.1 Definition
Discrete: E[X] = Σ x·P(X=x) Continuous: E[X] = ∫ x·f(x) dx
E[X] (also written μ) is the long-run average value of X over many repetitions — the 'centre of mass' of the
distribution.
Worked Example
Find E[X] for a single fair die roll.
E[X] = Σ x(1/6) = (1+2+3+4+5+6)/6 = 3.5
6.2 Properties of Expectation
Linearity: E[aX + b] = aE[X] + b
Sum rule: E[X+Y] = E[X] + E[Y] (always true, even if X, Y are dependent)
Product rule (independent X, Y only): E[XY] = E[X]·E[Y]
6.3 The Law of the Unconscious Statistician (LOTUS)
E[g(X)] = Σ g(x)·P(X=x) (discrete) E[g(X)] = ∫ g(x)·f(x) dx (continuous)
This lets us find the expectation of any function of X (e.g. E[X²], needed for variance) without first deriving the
distribution of g(X).
Worked Example
A game costs $5 to play. A fair coin is tossed 3 times; the player wins $2 per Head. Find the expected net gain.
Number of heads H ~ counts from 3 tosses, E[H] = 3(0.5) = 1.5
Winnings = 2H, so E[Winnings] = 2(1.5) = $3.00
Expected net gain = E[Winnings] − Cost = 3.00 − 5.00 = −$2.00
The game is not fair — the player loses $2 on average per play.
7. Variance

7.1 Definition
Var(X) = E[(X−μ)²] = E[X²] − (E[X])²
Variance measures the average squared spread of X around its mean; the standard deviation is σ = √Var(X).
Worked Example
For the die-roll RV (E[X]=3.5): E[X²] = Σx²(1/6) = (1+4+9+16+25+36)/6 = 91/6 = 15.17
Var(X) = 15.17 − 3.5² = 15.17 − 12.25 = 2.92, σ = 1.71
7.2 Properties of Variance
Var(aX + b) = a²Var(X) (adding a constant b does not change spread; scaling by a scales variance by
a²)
Var(X+Y) = Var(X) + Var(Y) [only if X, Y are independent]
Var(X+Y) = Var(X) + Var(Y) + 2Cov(X,Y) [general case — see Section 8]
7.3 Chebyshev's Inequality
For ANY random variable with finite mean μ and variance σ² (no distributional assumptions needed):
P(|X−μ| ≥ kσ) ≤ 1/k² equivalently: P(|X−μ| < kσ) ≥ 1 − 1/k²
Worked Example
For k=2: P(|X−μ| ≥ 2σ) ≤ 1/4 = 0.25, so at least 75% of any distribution's probability lies within 2 standard
deviations of its mean — a distribution-free guarantee.
8. Covariance and Correlation

8.1 Covariance
Cov(X,Y) = E[(X−μx)(Y−μy)] = E[XY] − E[X]E[Y]
Covariance measures the direction of the linear relationship between X and Y: positive Cov means they tend to move
together; negative means they tend to move oppositely.
Note: If X and Y are independent, Cov(X,Y)=0. The converse is FALSE — e.g. if Y=X² and X is symmetric around 0, X and Y
are strongly (nonlinearly) related, yet Cov(X,Y)=0. Zero covariance only rules out LINEAR dependence.
8.2 Correlation Coefficient
ρ(X,Y) = Cov(X,Y) / (σx·σy), −1 ≤ ρ ≤ 1
Correlation is a scale-free (unitless) version of covariance, making the strength of linear association directly
comparable across different variable pairs.
Worked Example
Two stocks have Cov(X,Y)=12, Var(X)=25, Var(Y)=36. Find ρ.
ρ = 12 / (√25 × √36) = 12/(5×6) = 12/30 = 0.40
A moderate positive linear relationship — the two stocks' returns tend to move in the same direction, but far from
perfectly.
8.3 Variance of a Sum, and Portfolio Risk
Var(X+Y) = Var(X) + Var(Y) + 2Cov(X,Y)
This identity underlies diversification: combining two assets whose returns are negatively correlated (Cov(X,Y)<0)
produces a combined portfolio with LOWER variance than either asset alone.
Worked Example
A two-asset portfolio holds weight w in asset X and (1−w) in asset Y. Then:
Var(Portfolio) = w²Var(X) + (1−w)²Var(Y) + 2w(1−w)Cov(X,Y)
When Cov(X,Y) is strongly negative, the cross-term can outweigh the individual variances, producing a portfolio less
risky than either single asset — the mathematical basis for diversification.
9. Summary Formula Sheet

Concept Formula
Complement rule P(Aᶜ) = 1 − P(A)
Addition rule P(A∪B) = P(A) + P(B) − P(A∩B)
Permutations nPr = n!/(n−r)!
Combinations nCr = n!/[r!(n−r)!]
Conditional probability P(A|B) = P(A∩B)/P(B)
Multiplication rule P(A∩B) = P(A|B)P(B)
Independence test P(A∩B) = P(A)P(B)
Law of total probability P(B) = Σ P(B|Ai)P(Ai)
Bayes' Rule P(Ai|B) = P(B|Ai)P(Ai) / Σ P(B|Aj)P(Aj)
Expectation (discrete) E[X] = Σ x·P(X=x)
Linearity of expectation E[aX+b] = aE[X]+b
Variance Var(X) = E[X²] − (E[X])²
Variance of a scaled RV Var(aX+b) = a²Var(X)
Covariance Cov(X,Y) = E[XY] − E[X]E[Y]
Correlation ρ = Cov(X,Y) / (σxσy)
Variance of a sum Var(X+Y) = Var(X)+Var(Y)+2Cov(X,Y)
Chebyshev's inequality P(|X−μ| ≥ kσ) ≤ 1/k²
10. Practice Questions

64 questions across all 8 sub-topics of this chapter. Attempt all questions before checking with your instructor or
working group; no solutions are included so you can work through them independently.
10.1 Basic Concepts of Probability
1. Define the sample space and event space for the experiment of tossing three fair coins and recording the
sequence of heads/tails. List all elements.
2. Distinguish between the classical, empirical (relative frequency), and subjective approaches to probability, giving
one real-world example of each.
3. A quality control engineer inspects 500 items and finds 15 defective. Using the empirical approach, estimate
P(defective) and discuss under what conditions this estimate would be unreliable.
4. State and prove the addition rule for two events A and B: P(A∪B) = P(A) + P(B) − P(A∩B).
5. Using the axioms of probability, prove that P(Aᶜ) = 1 − P(A).
6. If A and B are mutually exclusive events with P(A) = 0.3 and P(B) = 0.25, find P(A∪B) and P(A∩B).
7. Explain why P(∅) = 0 follows from Kolmogorov's axioms rather than being assumed as a separate rule.
8. A survey of 1,000 data science graduates shows 620 work in tech, 310 work in finance, and 140 work in both.
Find the probability that a randomly selected graduate works in neither sector.

10.2 Probability Counting Techniques

9. State the multiplication principle and use it to find the number of distinct 4-digit PIN codes possible using digits
0–9 (repetition allowed).
10. In how many ways can a project team of 5 be selected from 8 statisticians and 6 computer scientists if the team
must include exactly 3 statisticians?
11. Distinguish between a permutation and a combination, giving the formula for each and one example where
confusing the two would produce the wrong answer.
12. How many distinct ways can the letters of the word 'STATISTICS' be arranged? (Account for repeated letters.)
13. A password must be 6 characters long, using uppercase letters and digits only, with at least one digit required.
How many valid passwords are possible?
14. Twelve candidates apply for 3 identical analyst positions. In how many ways can the positions be filled if order
does not matter?
15. Using the pigeonhole principle, prove that in any group of 13 people, at least two must share a birth month.
16. A round-robin tournament has 10 teams, each playing every other team exactly once. How many total matches
are played?
17. In how many distinct ways can 7 distinguishable books be arranged on a shelf if 2 specific books must always be
adjacent?

10.3 Calculation of Probabilities

18. A committee of 5 is chosen at random from 7 men and 6 women. Find the probability the committee has
exactly 3 women.
 19. A standard deck of 52 cards is shuffled. Find the probability that a 5-card hand contains exactly 2 aces.
20. In a lottery, 6 numbers are drawn without replacement from 49. Find the probability of matching all 6 numbers
with a single ticket.
21. A box contains 10 red, 8 blue, and 6 green balls. Three balls are drawn without replacement. Find the
probability that all three are different colours.
22. Twenty applicants (12 men, 8 women) apply for 4 positions, filled at random. Find the probability that at least
one woman is selected.
23. Using the complement rule, find the probability that at least two of 30 randomly selected people share a
birthday (state your assumptions clearly).
24. A fair six-sided die is rolled 4 times. Find the probability that the sequence of rolls is strictly increasing.
25. From a class of 25 Master's students, a group of 6 is selected at random to present a group project. Find the
probability that two specific students (A and B) are both selected.

10.4 Conditional Probability, Independence, and Bayes' Rule

26. Define conditional probability formally and explain, using an example, why P(A|B) ≠ P(B|A) in general.
27. Two events A and B satisfy P(A) = 0.5, P(B) = 0.4, P(A∩B) = 0.2. Determine whether A and B are
independent, justifying your answer algebraically.
28. A factory has two machines: M1 produces 60% of output with a 2% defect rate, and M2 produces 40% with a
5% defect rate. A randomly selected item is found defective. Use Bayes' Rule to find the probability it came
from M2.
29. Prove that if A and B are independent, then A and Bᶜ are also independent.
30. In a population, 1% have a rare disease. A test is 98% sensitive (true positive rate) and 95% specific (true
negative rate). Given a positive test result, find the probability the person actually has the disease, and discuss
the implication of this result for medical screening policy.
31. State the law of total probability and use it to find P(B) given P(B|A1) = 0.3, P(B|A2) = 0.6, P(A1) = 0.4,
P(A2) = 0.6, where A1 and A2 partition the sample space.
32. A data-breach investigation reveals three possible attack vectors, each with a different prior likelihood and a
different probability of leaving a certain digital signature. Set up (without necessarily solving numerically) a
Bayes' Rule framework to update the probability of each vector given the observed signature.
33. Show, with a numerical example, that pairwise independence of three events does NOT imply mutual
independence.
34. A student answers a multiple-choice question with 4 options. If they know the answer (probability 0.7 that they
do) they answer correctly for certain; if they don't know it, they guess randomly. Given that the student
answered correctly, find the probability they actually knew the answer.
35. In a Bayesian spam filter, the prior probability that an email is spam is 0.3. The word 'free' appears in 80% of
spam emails and 10% of legitimate emails. If an email contains 'free', find the posterior probability that it is
spam.

10.5 Random Variables and Probability Distributions

36. Distinguish between a discrete and a continuous random variable, giving two real-world examples of each.
 37. A discrete random variable X has pmf P(X=x) = c(5−x) for x = 1,2,3,4. Find c and verify that it is a valid pmf.
38. Define the cumulative distribution function (CDF) of a random variable and state three properties every valid
CDF must satisfy.
39. Given the pdf f(x) = 3x² for 0 ≤ x ≤ 1, verify that f is a valid density and find P(0.2 < X < 0.5).
40. Explain the difference between a probability mass function and a probability density function, and why P(X =
x) = 0 for any specific x when X is continuous.
41. A random variable X has CDF F(x) = 1 − e^(−2x) for x ≥ 0. Find the pdf of X and identify the distribution it
belongs to.
42. Two random variables X and Y have a joint pmf given in a table. Explain, step by step, how you would find the
marginal distribution of X from this table.
43. Define what it means for two random variables to be identically distributed versus independent, and explain
why the two concepts are different.

10.6 Expectation of Random Variables

44. Define E[X] for a discrete random variable and compute it for a single fair die roll.
45. Prove the linearity property E[aX + b] = aE[X] + b from first principles using the definition of expectation.
46. A game costs $5 to play. A fair coin is tossed 3 times, and the player wins $2 per head obtained. Find the
expected net gain or loss and advise whether the game is fair.
47. For a continuous random variable with pdf f(x) = 2x on [0,1], compute E[X] and E[X²].
48. State and justify the law of the unconscious statistician: E[g(X)] = Σ g(x)P(X=x) for discrete X.
49. An insurance policy pays out $10,000 with probability 0.02 and $0 otherwise. What premium should the insurer
charge to break even in expectation? Discuss why insurers charge more than this figure in practice.
50. If X and Y are independent random variables, prove that E[XY] = E[X]E[Y].

10.7 Variance
51. Define Var(X) and derive the computational formula Var(X) = E[X²] − (E[X])².
52. For the die-roll random variable above, compute Var(X) and its standard deviation.
53. Prove that Var(aX + b) = a²Var(X) for constants a and b.
54. Two investment strategies have the same expected return but different variances. Explain, in the context of
decision-making under uncertainty, why variance is used as a risk measure.
55. If X and Y are independent, prove that Var(X+Y) = Var(X) + Var(Y). Show, with a small example, why this
fails when X and Y are dependent.
56. A random variable X has E[X] = 4 and E[X²] = 20. Find Var(X) and the standard deviation.
57. Explain what Chebyshev's inequality tells us about the relationship between variance and the spread of a
random variable, and compute the bound it gives for P(|X−μ| ≥ 2σ).

10.8 Covariance and Correlation

58. Define Cov(X,Y) and derive the computational formula Cov(X,Y) = E[XY] − E[X]E[Y].
 59. Prove that if X and Y are independent, Cov(X,Y) = 0. Explain, with a concrete counterexample, why the
converse is not always true.
60. Define the Pearson correlation coefficient ρ in terms of covariance and standard deviations, and explain why −1
≤ ρ ≤ 1.
61. Two random variables representing daily returns on two stocks have Cov(X,Y) = 12, Var(X) = 25, Var(Y) = 36.
Compute the correlation coefficient and interpret its strength and direction.
62. Explain the difference between correlation and causation using a real-world example (e.g. ice-cream sales and
drowning incidents).
63. Show that Var(X+Y) = Var(X) + Var(Y) + 2Cov(X,Y), and use it to explain how diversification reduces
portfolio risk when Cov(X,Y) < 0.
64. Given a portfolio with two assets weighted w and (1−w), derive the formula for portfolio variance in terms of
Var(X), Var(Y), Cov(X,Y), and discuss how the optimal weight minimising risk depends on the correlation
between assets.

End of notes.