Probability & Statistics – Detailed Notes
1. Introduction
Probability and Statistics are branches of mathematics that deal with uncertainty, data
analysis, and decision-making.
• Probability measures the likelihood of an event occurring.
• Statistics involves collecting, organizing, analyzing, and interpreting data.
Importance in Computer Science:
• Designing algorithms and understanding their performance.
• Machine Learning and AI (e.g., predicting outcomes, classification).
• Networking (packet loss, reliability).
• Data analytics and business intelligence.
2. Probability
Definition:
Probability of an event 𝐸is the ratio of the number of favorable outcomes to the total
number of outcomes:
Number of favorable outcomes
𝑃(𝐸) =
Total number of outcomes
Key Concepts:
1. Experiment: A procedure that produces outcomes (e.g., rolling a die).
2. Sample Space (S): Set of all possible outcomes.
o Example: S = {1, 2, 3, 4, 5, 6} for a die.
3. Event (E): A subset of the sample space.
o Example: Rolling an even number → E = {2, 4, 6}.
4. Complement of an Event:
𝑃(𝐸 ′ ) = 1 − 𝑃(𝐸)
Rules of Probability:
• 0 ≤ 𝑃(𝐸) ≤ 1
• 𝑃(𝑆) = 1
• Addition Rule (for mutually exclusive events):
𝑃(𝐴 ∪ 𝐵) = 𝑃(𝐴) + 𝑃(𝐵)
• Multiplication Rule (for independent events):
𝑃(𝐴 ∩ 𝐵) = 𝑃(𝐴) ⋅ 𝑃(𝐵)
Example:
• Roll a die. Probability of getting a number greater than 4:
2 1
𝐸 = {5,6}, 𝑃(𝐸) = =
6 3
Conditional Probability:
𝑃(𝐴 ∩ 𝐵)
𝑃(𝐴 ∣ 𝐵) =
𝑃(𝐵)
• Example: Probability of drawing an Ace from a deck, given the card is red.
3. Random Variables
Definition: A random variable (RV) is a function that assigns a numerical value to each
outcome of a random experiment.
Types:
1. Discrete RV: Takes countable values (e.g., number of heads in 3 coin tosses).
2. Continuous RV: Takes any value in a range (e.g., height, weight).
Expected Value (Mean):
𝐸(𝑋) = ∑[𝑥 ⋅ 𝑃(𝑥)]
Variance and Standard Deviation:
𝑉𝑎𝑟(𝑋) = 𝐸[(𝑋 − 𝐸(𝑋))2 ], 𝜎 = √𝑉𝑎𝑟(𝑋)
Example:
Toss a fair coin 3 times, let X = number of heads.
• X can be {0,1,2,3}
• Probabilities: P(0)=1/8, P(1)=3/8, P(2)=3/8, P(3)=1/8
• E(X) = 0*(1/8) + 1*(3/8) + 2*(3/8) + 3*(1/8) = 1.5
4. Descriptive Statistics
Measures of Central Tendency:
• Mean: Average value
∑𝑥𝑖
𝑥ˉ =
𝑛
• Median: Middle value when data is sorted
• Mode: Most frequent value
Measures of Dispersion:
• Range: Difference between max and min
• Variance & Standard Deviation: Spread of data
Example:
Data = {2, 4, 6, 8, 10}
• Mean = 6
• Median = 6
• Mode = None
• Variance = 8, SD = √8 ≈ 2.83
5. Probability Distributions
Discrete Distributions:
1. Binomial Distribution: Number of successes in n independent trials.
𝑛
𝑃(𝑋 = 𝑘) = ( )𝑝𝑘 (1 − 𝑝)𝑛−𝑘
𝑘
2. Poisson Distribution: Number of events in a fixed interval of time/space.
𝜆𝑘 𝑒 −𝜆
𝑃(𝑋 = 𝑘) =
𝑘!
Continuous Distributions:
1. Normal Distribution: Bell-shaped curve, mean μ, standard deviation σ.
2. Uniform Distribution: All outcomes equally likely within a range.
Example:
• Probability of exactly 2 heads in 3 coin tosses: Binomial → P(X=2)=3/8
6. Applications in Computer Science
• Machine Learning: Predict outcomes, classification, regression.
• Network Reliability: Probability of packet loss or errors.
• Cryptography: Randomness and probabilistic algorithms.
• Data Analytics: Summarize datasets, detect anomalies.
• Simulation: Model real-world processes with random inputs.
7. Mini Exercises
1. A die is rolled twice. Find the probability of getting a sum of 7.
2. A deck of 52 cards: Probability of drawing a red king.
3. Random variable X = number of heads in 4 coin tosses. Find E(X) and Var(X).
4. Compute mean, median, mode, and standard deviation for data {5, 7, 3, 8, 10}.
5. Toss a coin 5 times. Probability of getting exactly 3 heads using binomial formula.