4.7 Conditional Probability & Independent Events

1. Conditional Probability

Notice the following difference in notation:

  • $P(A)$ means "probability of $A$"
  • $P(A|B)$ means "probability of $A$, given $B$"

Intuitively, we expect that "the probability that it will rain on some day" is inherently different than "the probability that it will rain on some day, given that this is a day in September".

In a more mathematical example, suppose that we pick a whole number in the range 1 to 100. Let $A = \text{"we pick 17"}$. Clearly $P(A) = \dfrac{1}{100}$. However, if we know the information $B = \text{"the number selected has two digits"}$, then $P(A|B) = \dfrac{1}{90}$ (because there are exactly 90 two-digit numbers in that range).

Formal Definition of $P(A|B)$

The conditional probability is formally given by the formula:

$P(A|B) = \dfrac{n(A \cap B)}{n(B)} \quad \text{or} \quad P(A|B) = \dfrac{P(A \cap B)}{P(B)}$

We will clarify the definition by using Venn diagrams and tables.

EXAMPLE 1: $P(A|B)$ in a Venn Diagram

S 100 A B 20 10 30 40
We know that $P(A) = \dfrac{30}{100}$. What about $P(A|B)$?
We start with the given event $B$; now the total number is not 100 (the size of the whole sample space), but only 40 (the size of $B$):
$P(A|B) = \dfrac{?}{40} \longleftarrow \text{given } B$
How many elements of $A$ are inside the given space $B$? Only 10. Therefore,
$P(A|B) = \dfrac{10}{40} = 0.25$
In fact, in the last result we applied the formula $P(A|B) = \dfrac{n(A \cap B)}{n(B)} = \dfrac{10}{40} = 0.25$.
If we divide both the numerator and the denominator by the TOTAL number of the sample space, we obtain the formal definition:
$P(A|B) = \dfrac{P(A \cap B)}{P(B)} = \dfrac{\dfrac{10}{100}}{\dfrac{40}{100}} = \mathbf{0.25}$
Similarly, we can obtain:
  • $P(B|A) = \dfrac{10}{30} \leftarrow \text{given } A$
  • $P(A'|B) = \dfrac{30}{40}$
  • $P(A|B') = \dfrac{20}{60}$
  • $P(A'|B') = \dfrac{40}{60}$

EXAMPLE 2: $P(A|B)$ in a Table

Perhaps it is much easier to observe the conditional probability in tables. Consider again the example of 200 people:

Male Female Total
Smoker 40 20 60
Non-smoker 80 60 140
Total 120 80 200

Observe the difference between the standard and conditional probabilities:

  • $P(\text{smoker}) = \text{the person is a smoker} = \mathbf{\dfrac{60}{200}}$
  • $P(\text{smoker}|\text{male}) = \text{the person is a smoker given it is male} = \mathbf{\dfrac{40}{120}} \longleftarrow \text{given male}$
  • $P(\text{male}|\text{smoker}) = \mathbf{\dfrac{40}{60}} \longleftarrow \text{given smoker}$
  • $P(\text{female}|\text{smoker}) = \mathbf{\dfrac{20}{60} \approx 0.33}$
  • $P(\text{non-smoker}|\text{female}) = \mathbf{\dfrac{60}{80} = 0.75}$

2. Independent Events

The events $A$ and $B$ are said to be independent if:

$P(A|B) = P(A)$

In other words, the event $B$ does not affect $A$; the probability of $A$ remains the same, whether $B$ is given or not! Similarly, in this case, it holds that $P(B|A) = P(B)$. That is, event $A$ does not affect $B$.

In this case, the definition $P(A|B) = \dfrac{P(A \cap B)}{P(B)}$ yields:
$P(A \cap B) = P(A|B) \cdot P(B) \implies \mathbf{P(A \cap B) = P(A) \cdot P(B)}$

To summarize, if A and B are independent:
(1)$P(A|B) = P(A)$
(2)$P(B|A) = P(B)$
(3)$P(A \cap B) = P(A) \cdot P(B)$

EXAMPLE 3: Proving Independence

S 120 A B 20 10 30 60
We can mathematically show in three different ways that $A$ and $B$ are independent in this specific diagram:
  • $P(A) = \dfrac{30}{120} = \dfrac{1}{4}$ and $P(A|B) = \dfrac{10}{40} = \dfrac{1}{4}$. Thus (1) holds.
  • $P(B) = \dfrac{40}{120} = \dfrac{1}{3}$ and $P(B|A) = \dfrac{10}{30} = \dfrac{1}{3}$. Thus (2) holds.
  • $P(A \cap B) = \dfrac{10}{120} = \dfrac{1}{12}$ and $P(A) \cdot P(B) = \dfrac{1}{4} \cdot \dfrac{1}{3} = \dfrac{1}{12}$. Thus (3) holds.

NOTICE: Mutually Exclusive vs. Independent

Many students confuse the terms Mutually Exclusive Events and Independent Events. Remember:

Mutually Exclusive Events means $A \cap B = \emptyset$.
This yields $P(A \cap B) = 0$.
$P(A \cup B) = P(A) + P(B) - P(A \cap B)$ holds in general.
Independent Events means $P(A \cap B) = P(A) \cdot P(B)$.
In particular for independent events, it is sometimes useful to combine the general addition rule with independence:
$P(A \cup B) = P(A) + P(B) - P(A) \cdot P(B)$
Sometimes we know beforehand that two events are independent. Thus, for their combination we can directly apply the formula $P(A \cap B) = P(A) \cdot P(B)$.
For example: We toss a die and a coin. Find the probability that the die shows a SIX and the coin shows a HEAD.
Let $A = \text{"the die shows a SIX"}$ $\implies P(A) = \dfrac{1}{6}$.
Let $B = \text{"the coin shows a HEAD"}$ $\implies P(B) = \dfrac{1}{2}$.
The events $A$ and $B$ are clearly independent, so for their combination it holds:
$P(A \cap B) = P(A) \cdot P(B) = \dfrac{1}{6} \cdot \dfrac{1}{2} = \mathbf{\dfrac{1}{12}}$

EXAMPLE 4

Let $P(A) = 0.4$ and $P(B) = 0.3$. Find $P(A \cup B)$ in the following distinct cases:

a) A and B are mutually exclusive
$P(A \cup B) = P(A) + P(B) = 0.4 + 0.3 = \mathbf{0.7}$
b) A and B are independent
$P(A \cup B) = P(A) + P(B) - P(A) \cdot P(B) = 0.4 + 0.3 - (0.4)(0.3) = \mathbf{0.58}$
c) Given that $P(A \cap B) = 0.2$
$P(A \cup B) = P(A) + P(B) - P(A \cap B) = 0.4 + 0.3 - 0.2 = \mathbf{0.5}$
d) Given that $P(A|B) = 0.2$
First find the intersection: $P(A|B) = \dfrac{P(A \cap B)}{P(B)} \implies P(A \cap B) = P(A|B)P(B) = (0.2)(0.3) = 0.06$.
Hence, $P(A \cup B) = P(A) + P(B) - P(A \cap B) = 0.4 + 0.3 - 0.06 = \mathbf{0.64}$

EXAMPLE 5

Let $A$ and $B$ be independent events with $P(A) = 0.4$ and $P(A \cup B) = 0.7$. Find $P(B)$.

For independent events it holds that: $P(A \cup B) = P(A) + P(B) - P(A) \cdot P(B)$
Substitute the known values: $\implies 0.7 = 0.4 + P(B) - 0.4P(B)$
Simplify the algebra: $\implies 0.3 = 0.6P(B)$
Final calculation: $\implies P(B) = \dfrac{0.3}{0.6} = \mathbf{0.5}$