Probability as a Representation of Our State
of Knowledge
Probability is not only the long-run frequency of events but also a measure of uncertainty or
belief about an event based on available information.
Two Interpretations:
(a) Frequentist Interpretation
Probability is the relative frequency of an event in repeated trials.
Example: If a coin is tossed 1000 times and 500 heads occur,
P(H)=500/1000=0.5P(H) = 500/1000 = 0.5P(H)=500/1000=0.5
(b) Bayesian (Subjective) Interpretation
Probability reflects our degree of belief given current knowledge.
Example: If a student usually scores well, we may assign a high probability to passing
before results are declared.
Key Idea:
Probability changes when new information is obtained.
Example:
Before checking weather forecast: P(Rain)=0.5P(\text{Rain}) = 0.5P(Rain)=0.5
After forecast predicts rain: P(Rain)=0.8P(\text{Rain}) = 0.8P(Rain)=0.8
Thus, probability represents our state of knowledge.
2 Principle of Insufficient Reason
2️⃣
The Principle of Insufficient Reason states:
When there is no reason to favor one outcome over another, assign equal probabilities to all
possible outcomes.
Example 1: Fair Die
There are 6 faces.
P(1)=P(2)=...=P(6)=16P(1)=P(2)=...=P(6)=\frac{1}{6}P(1)=P(2)=...=P(6)=61
Example 2: Coin Toss
No bias known:
P(H)=P(T)=12P(H)=P(T)=\frac{1}{2}P(H)=P(T)=21
Limitation:
It assumes symmetry.
Not suitable when hidden bias exists.
3️⃣Estimating Probabilities by Simulation
Simulation uses repeated random experiments to estimate probability.
Steps:
1. Define experiment.
2. Perform large number of trials.
3. Count favorable outcomes.
4. Compute relative frequency.
P(A)≈Number of times A occursTotal trialsP(A) \approx \frac{\text{Number of times A
occurs}}{\text{Total trials}}P(A)≈Total trialsNumber of times A occurs
Example: Coin Toss Simulation
If 10,000 tosses produce 5030 heads:
P(H)≈5030/10000=0.503P(H) \approx 5030/10000 = 0.503P(H)≈5030/10000=0.503
As trials increase, estimate approaches theoretical value (Law of Large Numbers).
Applications:
Risk analysis
Reliability engineering
Monte Carlo methods
Machine learning
4️⃣Estimating Probabilities from Data
When real-world data is available, probability is estimated from observed frequency.
Formula:
P(A)=Number of favorable casesTotal observationsP(A) = \frac{\text{Number of favorable
cases}}{\text{Total observations}}P(A)=Total observationsNumber of favorable cases
Example 1:
Out of 200 products, 10 are defective:
P(Defective)=10/200=0.05P(\text{Defective}) = 10/200 = 0.05P(Defective)=10/200=0.05
Example 2:
In 100 students, 70 pass:
P(Pass)=70/100=0.7P(\text{Pass}) = 70/100 = 0.7P(Pass)=70/100=0.7
Difference from Simulation:
Simulation → Artificial experiment
Data-based → Real observations
5️⃣Basic Rules of Probability
(1) Non-negativity Rule
0≤P(A)≤10 \le P(A) \le 10≤P(A)≤1
(2) Complement Rule
Probability of event not occurring:
P(Ac)=1−P(A)P(A^c) = 1 - P(A)P(Ac)=1−P(A)
Example:
If P(Rain)=0.3P(\text{Rain}) = 0.3P(Rain)=0.3,
P(No Rain)=0.7P(\text{No Rain}) = 0.7P(No Rain)=0.7
(3) Addition Rule (General Case)
P(A∪B)=P(A)+P(B)−P(A∩B)P(A \cup B) = P(A) + P(B) - P(A \cap B)P(A∪B)=P(A)+P(B)
−P(A∩B)
Used when events overlap.
Example:
If:
P(A)=0.5,P(B)=0.4,P(A∩B)=0.2P(A)=0.5,\quad P(B)=0.4,\quad P(A \cap
B)=0.2P(A)=0.5,P(B)=0.4,P(A∩B)=0.2
Then:
P(A∪B)=0.5+0.4−0.2=0.7P(A \cup B)=0.5+0.4-0.2=0.7P(A∪B)=0.5+0.4−0.2=0.7
6️⃣Probability of Logical Disjunctions
Logical disjunction means:
A OR BA \text{ OR } BA OR B
In probability:
P(A∪B)P(A \cup B)P(A∪B)
This represents the probability that at least one event occurs.
Case 1: Mutually Exclusive Events
If A and B cannot occur together:
P(A∩B)=0P(A \cap B) = 0P(A∩B)=0
Then:
P(A∪B)=P(A)+P(B)P(A \cup B) = P(A) + P(B)P(A∪B)=P(A)+P(B)
Example:
Probability of getting 1 or 2 on die:
=16+16=26= \frac{1}{6} + \frac{1}{6} = \frac{2}{6}=61+61=62
Case 2: Non-Mutually Exclusive Events
If events overlap:
Use full addition rule.
Example:
In a class:
60% like Maths
50% like Physics
30% like both
P(M∪P)=0.6+0.5−0.3=0.8P(M \cup P)=0.6+0.5-0.3=0.8P(M∪P)=0.6+0.5−0.3=0.8
So, 80% like at least one subject.
7️⃣The Sum Rule (Conceptual Understanding)
The sum rule prevents double counting.
If overlap exists, subtract intersection.
Visual Understanding:
In Venn diagram:
Total area of both circles
Subtract overlapping region once
8️⃣Comparison Summary
Concept Meaning Formula
Complement Not A 1−P(A)1 - P(A)1−P(A)
Logical OR A or B P(A∪B)P(A \cup B)P(A∪B)
Mutually Exclusive No overlap P(A)+P(B)P(A)+P(B)P(A)+P(B)
P(A)+P(B)−P(A∩B)P(A)+P(B)-P(A \cap B)P(A)+P(B)
General Addition With overlap
−P(A∩B)
Simulation Random trials Relative frequency
Observed
Data Estimation Frequency ratio
data
9️⃣Practical Engineering Relevance
Quality control → Defect probability
Reliability analysis → Failure probability
AI & ML → Uncertainty modeling
Risk assessment → Event likelihood
Decision-making → Bayesian updating
🔟 Conclusion
Probability represents our uncertainty based on available knowledge.
It can be estimated through:
Theoretical reasoning
Simulation
Observed data
The addition rule and sum rule ensure correct computation of combined events, especially when
events overlap. Understanding these principles is fundamental in data science, engineering
analysis, and decision-making under uncertainty.