Conditional probability
The probability of event E given that event F has occurred is
P(E | F) = P(E ∩ F) / P(F), provided P(F) ≠ 0
Knowing that F happened shrinks the sample space to F.
Example. A fair die is rolled. What is the probability of getting a 6, given that the number is even?
F = {2, 4, 6} and E ∩ F = {6}. So P(E | F) = (1/6)/(3/6) = 1/3.
Multiplication rule
Rearranging the definition:
P(E ∩ F) = P(F) P(E | F) = P(E) P(F | E)
Example. A bag has 5 red and 3 blue balls. Two are drawn one after the other without replacement. What is the probability that both are red?
P = (5/8) × (4/7) = 5/14.
Independent events
E and F are independent if the occurrence of one does not affect the probability of the other:
P(E ∩ F) = P(E) × P(F), or equivalently P(E | F) = P(E)
Independent is not the same as mutually exclusive. Mutually exclusive events (with non-zero probabilities) cannot both happen, so they are always dependent.
Theorem of total probability
If E₁, E₂, …, Eₙ partition the sample space (they are mutually exclusive and together cover every outcome), then for any event A:
P(A) = P(E₁)P(A | E₁) + P(E₂)P(A | E₂) + … + P(Eₙ)P(A | Eₙ)
Bayes' theorem
Bayes' theorem reverses a conditional probability. It finds the probability of a cause given an observed effect:
P(Eᵢ | A) = P(Eᵢ)P(A | Eᵢ) / Σⱼ P(Eⱼ)P(A | Eⱼ)
Example. A factory has two machines. Machine X makes 60% of the bolts, 2% of which are defective. Machine Y makes 40%, and 5% of its bolts are defective. A bolt chosen at random is defective. What is the probability that machine Y made it?
P(D) = 0.6 × 0.02 + 0.4 × 0.05 = 0.012 + 0.020 = 0.032
P(Y | D) = 0.020 / 0.032 = 5/8 = 0.625
Although Y makes fewer bolts, it is more likely to be responsible for a defective one.
Random variables
A random variable X assigns a real number to each outcome of a random experiment. Its probability distribution lists each value xᵢ with its probability pᵢ, where every pᵢ ≥ 0 and the pᵢ add up to 1.
Mean (expected value): E(X) = μ = Σ xᵢ pᵢ
Variance: Var(X) = Σ xᵢ² pᵢ − μ²
Standard deviation: σ = √Var(X)
Example. X is the number of heads when two fair coins are tossed.
x | 0 | 1 | 2 |
|---|---|---|---|
P(X = x) | 1/4 | 1/2 | 1/4 |
E(X) = 0 + 1/2 + 2/4 = 1. E(X²) = 0 + 1/2 + 4/4 = 3/2, so Var(X) = 3/2 − 1 = 1/2.
Bernoulli trials and the binomial distribution
Bernoulli trials are repeated trials that are independent, finite in number, have exactly two outcomes (success or failure), and have the same probability of success p each time.
The number of successes X in n Bernoulli trials follows the binomial distribution:
P(X = r) = ⁿCᵣ pʳ qⁿ⁻ʳ, where q = 1 − p, for r = 0, 1, …, n
Its mean is np and its variance is npq.
Example. A fair coin is tossed 6 times. What is the probability of exactly 4 heads?
P = ⁶C₄ (1/2)⁴(1/2)² = 15/64.