Conditional Probability and Independence

Why this mat­ters#

Sup­pose you already know one thing has hap­pened. Does that change the chances of some­thing else? Very often it does, and that sim­ple idea is the heart of this chap­ter. We will build con­di­tional prob­a­bil­ity, get the mul­ti­pli­ca­tion rule out of it, and then see what it means for two events to be inde­pen­dent.

Con­di­tional prob­a­bil­ity#

If $F$ has already hap­pened ($P(F) \ne 0$), the prob­a­bil­ity of $E$ is

$$\displaystyle P(E \mid F) = \frac{P(E \cap F)}{P(F)}.$$

When all out­comes are equally likely, this is just $\displaystyle \tfrac{n(E \cap F)}{n(F)}$. Think of it this way: once you know the given event has occurred, $F$ becomes your whole sam­ple space.

A few prop­er­ties fol­low directly: $P(F \mid F) = 1$; $P(E' \mid F) = 1 - P(E \mid F)$; and $P((A \cup B) \mid F) = P(A \mid F) + P(B \mid F) - P((A \cap B) \mid F)$.

Mul­ti­pli­ca­tion rule#

Turn the def­i­n­i­tion around and you get

$$P(E \cap F) = P(F)\,P(E \mid F) = P(E)\,P(F \mid E),$$

and with three events, $P(E \cap F \cap G) = P(E)P(F \mid E)P(G \mid E \cap F)$. Each new fac­tor takes into account what has already hap­pened.

Inde­pen­dent events#

We call $E$ and $F$ inde­pen­dent when know­ing one tells you noth­ing new about the other, that is $P(E \mid F) = P(E)$. The same thing, writ­ten more use­fully:

$$P(E \cap F) = P(E)\,P(F).$$

Stu­dents often mix up inde­pen­dent and mutu­ally exclu­sive, so be care­ful. If two events with non-zero prob­a­bil­i­ties are mutu­ally exclu­sive, they can never be inde­pen­dent, because one hap­pen­ing rules the other out. Also, if $E$ and $F$ are inde­pen­dent, then so are $E$ and $F'$, and so are $E'$ and $F'$.

Prac­tice#

1. $P(E) = 0.6$, $P(F) = 0.3$, $P(E \cap F) = 0.2$. Find $P(E \mid F)$ and $P(F \mid E)$.
2. A coin is tossed three times. Find $P(\text{at least two heads} \mid \text{first toss is a head})$.
3. Two dice are thrown. Find $P(\text{sum} > 9 \mid \text{one die shows } 5)$.
4. A box has $4$ good and $2$ faulty bulbs. Two are drawn with­out replace­ment. Find $P(\text{both good})$ and $P(\text{one good, one faulty})$.
5. Are $E$ = "a card is a heart" and $F$ = "a card is a king" inde­pen­dent?
6. $P(A) = 0.3$, $P(B) = 0.4$, and $A$ and $B$ are inde­pen­dent. Find $P(A \cup B)$, $P(A' \cap B')$ and $P(A \mid B')$.
7. Three shoot­ers hit a tar­get with prob­a­bil­i­ties $\displaystyle \tfrac{1}{2}$, $\displaystyle \tfrac{1}{3}$ and $\displaystyle \tfrac{1}{4}$. Find the prob­a­bil­ity that the tar­get is hit at least once.
8. Show that if $E$ and $F$ are inde­pen­dent, then so are $E'$ and $F'$.
9. A die is thrown twice. $A$ = first throw even, $B$ = sec­ond throw odd, $C$ = sum odd. Are $A$ and $B$ inde­pen­dent? What about $A$ and $C$?
10. An urn has $3$ white and $5$ black balls. One ball is drawn, its colour is noted, and it goes back in along with $2$ more of the same colour. Then a sec­ond ball is drawn. Find $P(\text{second is white})$.

Answers#