The classical approach to probability assumes all outcomes are equally likely, making it most suitable for theoretical scenarios such as fairness experiments, like tossing a fair coin or rolling a die. In these scenarios, the probability is calculated by dividing the number of favorable outcomes by the total number of possible outcomes . The empirical approach is based on actual experiments or historical data, making it more applicable in real-world situations where outcomes are not guaranteed to be equally likely. For instance, if an event has been observed multiple times, the empirical probability is determined by the ratio of the occurrence of the favorable outcomes to the total number of trials, as shown in the example of a coin tossed 100 times with 45 heads appearing .

Understanding disjoint (mutually exclusive) and independent events is crucial for correctly applying probability rules. Disjoint events cannot occur simultaneously, thus P(A ∩ B) = 0. For independent events, the occurrence of one does not affect the probability of the other, mathematically expressed as P(A ∩ B) = P(A) × P(B). The implications are vast in ensuring proper calculation of joint probabilities, crucial for predictions and probability distributions in various fields like data science and risk management—highlighting the need for precise distinction and application .

The axiomatic approach, introduced by Kolmogorov, is significant as it provides a formal foundation for probability theory using set theory. This method forms the basis of modern probability and ensures consistency and objectivity. It requires that any probability function P must satisfy certain axioms, such as P(A) ≥ 0 for any event A, and P(S) = 1 where S is the sample space. Moreover, for disjoint events A and B, P(A ∪ B) = P(A) + P(B). This approach moves away from subjective probability measures and emphasizes a structured, rigorous methodology.

The fundamental properties of probability involve non-negativity, normalization, and additivity. P(A) ≥ 0 for any event A, as probability is the ratio of favorable outcomes and cannot be negative . P(S) = 1, since the sample space S contains all possible outcomes, ensuring totality in events . The addition theorem P(A ∪ B) = P(A) + P(B) − P(A ∩ B) accounts for overlapping in events. This is illustrated with the example: Suppose P(A)=0.4, P(B)=0.5, and P(A ∩ B)=0.2, then P(A ∪ B) is calculated as P(A ∪ B) = 0.4 + 0.5 − 0.2 = 0.7, ensuring no double-counting of the intersection .

Bayes' Theorem is pivotal for updating probabilities when new information becomes available. This capacity is immensely useful in fields like medical diagnosis, where initial probability estimates (prior probabilities) are adjusted as new test results or evidence are assessed (likelihood and posterior probabilities). This yields a more refined probability of an underlying condition given the test results. The document simplifies this with a bag of balls: after observing a red ball, Bayes' Theorem helps determine the probability it came from a specific bag when multiple bags are involved – showing analogous application like determining disease probability using test results .

The Multiplication Theorem of Probability is essential for finding the joint probability of two events when they are independent. For independent events A and B, the theorem states that the probability of both events occurring is the product of their individual probabilities, i.e., P(A ∩ B) = P(A) × P(B). This theorem is commonly used in situations where events do not influence each other. For example, if P(A) = 0.6 and P(B|A) = 0.5, then using the theorem for independent events, we find P(A ∩ B) = 0.6 × 0.5 = 0.3 .

The complement rule states that the probability of an event not occurring is equal to one minus the probability of the event occurring: P(A') = 1 - P(A). This principle is useful for simplifying calculations by focusing on the complementary event. For instance, using the classical probability example from the document: a die is rolled, and we want to find the probability of not rolling a number less than 4. The probability of rolling a number less than 4 is 3/6 or 0.5 (favorable outcomes 1, 2, 3 out of total outcomes 1 to 6). Thus, the probability of not rolling a number less than 4 is P(A') = 1 - 0.5 = 0.5 .

In probability theory, the concept of a sample space is critical because it comprises all possible outcomes of an experiment. The probability of the entire sample space is always 1, reflecting the certainty that some outcome described by the sample space will occur. This is crucial for defining probabilities consistently across different experiments. The use of a sample space ensures that probabilities assigned to individual events add up coherently and relationship among events can be evaluated, such as disjoint events where P(A ∪ B) = P(A) + P(B) when A and B are disjoint, reflecting in the total probability not exceeding 1 .

Bayes' Theorem is crucial for determining the probability of an event based on prior knowledge of related conditions. Its application is widespread in various real-world scenarios such as medical diagnoses, spam filtering, and decision making under uncertainty. Bayes' Theorem updates the probability estimate for an event as evidence accumulates. An example from the document involves a disease affecting 1% of a population, where a test detects the disease 99% of the time if present, but also gives a false positive rate of 5%. Using Bayes' Theorem, one can calculate the probability that a person who tests positive actually has the disease, which is essential for understanding diagnostic test effectiveness .

The Addition Theorem of Probability is applicable for determining the likelihood of the occurrence of at least one of two events. For mutually exclusive events, where events A and B cannot happen simultaneously, the theorem simplifies to P(A ∪ B) = P(A) + P(B). For non-mutually exclusive events, the theorem considers the overlap, written as P(A ∪ B) = P(A) + P(B) − P(A ∩ B). For example, if P(A) = 0.4, P(B) = 0.5, and P(A ∩ B) = 0.2, the probability of either event A or B occurring is calculated as P(A ∪ B) = 0.4 + 0.5 − 0.2 = 0.7 .