Total Probability and Bayes' Theorem, Worked Solutions

Ques­tion 1: Prob­a­bil­ity of draw­ing a gold coin#

The prob­lem#

Two boxes: box A has $2$ gold and $3$ sil­ver coins; box B has $4$ gold and $1$ sil­ver. A box is picked at ran­dom and a coin drawn. Find $P(\text{gold})$.

Under­stand­ing the prob­lem#

There are two stages: first a box is cho­sen, then a coin from that box. The gold coin could come from either box, so both routes must be counted.

The idea#

The events $E_1$ = "box A cho­sen" and $E_2$ = "box B cho­sen" form a par­ti­tion. Use the the­o­rem of total prob­a­bil­ity:

$$P(G) = P(E_1)P(G \mid E_1) + P(E_2)P(G \mid E_2).$$

Step-by-step solu­tion#

Step 1. Pri­ors: a box is picked at ran­dom.

$$\displaystyle P(E_1) = P(E_2) = \frac12.$$

Step 2. Con­di­tional prob­a­bil­i­ties of gold. Box A has $2$ gold out of $5$; box B has $4$ gold out of $5$.

$$\displaystyle P(G \mid E_1) = \frac25, \qquad P(G \mid E_2) = \frac45.$$

Step 3. Apply total prob­a­bil­ity.

$$\displaystyle P(G) = \frac12 \cdot \frac25 + \frac12 \cdot \frac45 = \frac{2}{10} + \frac{4}{10} = \frac{6}{10} = \frac35.$$

Check­ing the answer#

With equal chances for the boxes, $P(G)$ is the aver­age of $\displaystyle \frac25$ and $\displaystyle \frac45$, which is $\displaystyle \frac35$.

Answer#

$\displaystyle P(\text{gold}) = \frac35$.

Ques­tion 2: Which box did the gold coin come from?#

The prob­lem#

In ques­tion 1 the coin is gold. Find the prob­a­bil­ity it came from box B.

Under­stand­ing the prob­lem#

Now you know the result (gold) and must rea­son back­wards to the cause (box B). You want $P(E_2 \mid G)$.

The idea#

Bayes' the­o­rem:

$$\displaystyle P(E_2 \mid G) = \frac{P(E_2)P(G \mid E_2)}{P(G)},$$

where the denom­i­na­tor was found in Ques­tion 1.

Step-by-step solu­tion#

Step 1. Numer­a­tor: the prob­a­bil­ity of the route "box B and gold".

$$\displaystyle P(E_2)P(G \mid E_2) = \frac12 \cdot \frac45 = \frac25.$$

Step 2. Denom­i­na­tor, from Ques­tion 1: $\displaystyle P(G) = \frac35$.

Step 3. Divide.

$$\displaystyle P(E_2 \mid G) = \frac{2/5}{3/5} = \frac23.$$

Check­ing the answer#

$\displaystyle P(E_1 \mid G) = \frac{\frac12 \cdot \frac25}{\frac35} = \frac13$, and $\displaystyle \frac13 + \frac23 = 1$. It also makes sense that box B, with more gold coins, is the like­lier source.

Answer#

$\displaystyle P(\text{box B} \mid \text{gold}) = \frac23$.

Ques­tion 3: Hos­tel or day stu­dent?#

The prob­lem#

$40\%$ of a col­lege's stu­dents live in hos­tels. $30\%$ of hos­tel stu­dents and $20\%$ of day stu­dents score an A grade. A stu­dent with an A is cho­sen at ran­dom. Find $P(\text{hostel})$.

Under­stand­ing the prob­lem#

The par­ti­tion is $H$ = "hos­tel stu­dent" and $D$ = "day stu­dent". The observed event is $A$ = "scores an A grade". You want $P(H \mid A)$.

The idea#

Use Bayes' the­o­rem with the two causes $H$ and $D$.

Step-by-step solu­tion#

Step 1. Pri­ors and con­di­tional prob­a­bil­i­ties.

$$P(H) = 0.4,\quad P(D) = 0.6,\quad P(A \mid H) = 0.3,\quad P(A \mid D) = 0.2.$$

Step 2. Prob­a­bil­ity of each route.

$$P(H)P(A \mid H) = 0.4 \times 0.3 = 0.12, \qquad P(D)P(A \mid D) = 0.6 \times 0.2 = 0.12.$$

Step 3. Total prob­a­bil­ity of an A grade.

$$P(A) = 0.12 + 0.12 = 0.24.$$

Step 4. Bayes' the­o­rem.

$$\displaystyle P(H \mid A) = \frac{0.12}{0.24} = \frac12.$$

Check­ing the answer#

Hos­tel stu­dents are fewer ($40\%$) but more likely to get an A ($30\%$ against $20\%$). The two effects bal­ance exactly, since both routes give $0.12$.

Answer#

$\displaystyle P(\text{hostel} \mid \text{A grade}) = \frac12$.

Ques­tion 4: Which plant made the faulty item?#

The prob­lem#

A fac­to­ry's two plants make $60\%$ and $40\%$ of its out­put; $3\%$ and $5\%$ of their items are faulty. A faulty item is found. Which plant is more likely to have made it?

Under­stand­ing the prob­lem#

The par­ti­tion is $E_1$ = "made in plant 1" and $E_2$ = "made in plant 2". The observed event is $F$ = "faulty". Com­pare $P(E_1 \mid F)$ and $P(E_2 \mid F)$.

The idea#

In Bayes' the­o­rem both pos­te­ri­ors have the same denom­i­na­tor $P(F)$, so you only need to com­pare the numer­a­tors $P(E_i)P(F \mid E_i)$. Still, com­pute the actual prob­a­bil­i­ties to be com­plete.

Step-by-step solu­tion#

Step 1. Numer­a­tors.

$$\begin{aligned} P(E_1)P(F \mid E_1) &= 0.6 \times 0.03 = 0.018, \\ P(E_2)P(F \mid E_2) &= 0.4 \times 0.05 = 0.020. \end{aligned}$$

Step 2. Total prob­a­bil­ity of a faulty item.

$$P(F) = 0.018 + 0.020 = 0.038.$$

Step 3. Pos­te­ri­ors.

$$\displaystyle P(E_1 \mid F) = \frac{0.018}{0.038} = \frac{9}{19}, \qquad P(E_2 \mid F) = \frac{0.020}{0.038} = \frac{10}{19}.$$

Step 4. Com­pare: $\displaystyle \frac{10}{19} > \frac{9}{19}$.

Check­ing the answer#

$\displaystyle \frac{9}{19} + \frac{10}{19} = 1$. Plant 2 makes less, but its fault rate is high enough to out­weigh that.

Answer#

Plant 2 is more likely: $\displaystyle P(\text{plant 2} \mid \text{faulty}) = \frac{10}{19}$ against $\displaystyle \frac{9}{19}$ for plant 1.

Ques­tion 5: A pos­i­tive test result#

The prob­lem#

A test for a con­di­tion has $98\%$ detec­tion and a $3\%$ false-pos­i­tive rate; the con­di­tion affects $2\%$ of peo­ple. Find the prob­a­bil­ity a per­son who tests pos­i­tive has it.

Under­stand­ing the prob­lem#

Key terms:

• "$98\%$ detec­tion" means $P(+ \mid C) = 0.98$, where $C$ = "has the con­di­tion".
• "$3\%$ false-pos­i­tive rate" means $P(+ \mid C') = 0.03$: a healthy per­son tests pos­i­tive $3\%$ of the time.
• "Affects $2\%$" means $P(C) = 0.02$, so $P(C') = 0.98$.

You want $P(C \mid +)$.

The idea#

Bayes' the­o­rem with the par­ti­tion $C$, $C'$.

Step-by-step solu­tion#

Step 1. Prob­a­bil­ity of a true pos­i­tive route.

$$P(C)P(+ \mid C) = 0.02 \times 0.98 = 0.0196.$$

Step 2. Prob­a­bil­ity of a false pos­i­tive route.

$$P(C')P(+ \mid C') = 0.98 \times 0.03 = 0.0294.$$

Step 3. Total prob­a­bil­ity of a pos­i­tive test.

$$P(+) = 0.0196 + 0.0294 = 0.049.$$

Step 4. Bayes' the­o­rem.

$$\displaystyle P(C \mid +) = \frac{0.0196}{0.049} = 0.4.$$

Check­ing the answer#

Think of $10\,000$ peo­ple: $200$ have the con­di­tion and about $196$ of them test pos­i­tive; of the $9800$ healthy peo­ple, $294$ test pos­i­tive. So $\displaystyle \frac{196}{196 + 294} = \frac{196}{490} = 0.4$. Same answer.

Answer#

$P(\text{condition} \mid \text{positive}) = 0.4$.

Com­mon mis­take to avoid#

Do not answer $0.98$. That is $P(+ \mid C)$, the reverse of what is asked.

Ques­tion 6: A report that a die showed one#

The prob­lem#

A woman tells the truth $4$ times in $5$. She reports that a die showed a one. Find the prob­a­bil­ity that it really did.

Under­stand­ing the prob­lem#

The par­ti­tion is $E_1$ = "the die showed one" and $E_2$ = "the die did not show one". The observed event is $R$ = "she reports a one". She reports a one either when it was a one and she tells the truth, or when it was not a one and she lies.

The idea#

Bayes' the­o­rem:

$$\displaystyle P(E_1 \mid R) = \frac{P(E_1)P(R \mid E_1)}{P(E_1)P(R \mid E_1) + P(E_2)P(R \mid E_2)}.$$

Step-by-step solu­tion#

Step 1. Pri­ors from a fair die.

$$\displaystyle P(E_1) = \frac16, \qquad P(E_2) = \frac56.$$

Step 2. Con­di­tional prob­a­bil­i­ties of the report.

$$\displaystyle P(R \mid E_1) = \frac45 \;(\text{she tells the truth}), \qquad P(R \mid E_2) = \frac15 \;(\text{she lies}).$$

Step 3. Prob­a­bil­ity of each route.

$$\displaystyle \frac16 \cdot \frac45 = \frac{4}{30}, \qquad \frac56 \cdot \frac15 = \frac{5}{30}.$$

Step 4. Bayes' the­o­rem.

$$\displaystyle P(E_1 \mid R) = \frac{4/30}{4/30 + 5/30} = \frac{4}{9}.$$

Check­ing the answer#

Even though she is usu­ally truth­ful, a one is rare ($\displaystyle \frac16$), so the lie route ($\displaystyle \frac{5}{30}$) is slightly more likely than the truth route ($\displaystyle \frac{4}{30}$). The answer below $\displaystyle \frac12$ makes sense.

Answer#

The prob­a­bil­ity that the die really showed a one is $\displaystyle \frac49$.

Ques­tion 7: Three coins, one shows heads#

The prob­lem#

Of three coins, one is two-headed, one is biased to show heads $75\%$ of the time, and one is fair. A coin is cho­sen at ran­dom and shows heads. Find the prob­a­bil­ity it is the two-headed coin.

Under­stand­ing the prob­lem#

The par­ti­tion is $E_1$ = two-headed coin, $E_2$ = biased coin, $E_3$ = fair coin, each cho­sen with prob­a­bil­ity $\displaystyle \frac13$. The observed event is $H$ = "heads". You want $P(E_1 \mid H)$.

The idea#

Bayes' the­o­rem with three causes. Because the pri­ors are all equal, they can­cel.

Step-by-step solu­tion#

Step 1. Con­di­tional prob­a­bil­i­ties of heads.

$$P(H \mid E_1) = 1,\qquad P(H \mid E_2) = 0.75,\qquad P(H \mid E_3) = 0.5.$$

Step 2. Bayes' the­o­rem.

$$\displaystyle P(E_1 \mid H) = \frac{\frac13 \cdot 1}{\frac13 \cdot 1 + \frac13 \cdot 0.75 + \frac13 \cdot 0.5}.$$

Step 3. Can­cel $\displaystyle \frac13$ and sim­plify.

$$\displaystyle P(E_1 \mid H) = \frac{1}{1 + 0.75 + 0.5} = \frac{1}{2.25} = \frac{4}{9}.$$

Check­ing the answer#

The other pos­te­ri­ors are $\displaystyle \frac{0.75}{2.25} = \frac13$ and $\displaystyle \frac{0.5}{2.25} = \frac29$. Then $\displaystyle \frac49 + \frac39 + \frac29 = 1$. Cor­rect.

Answer#

$\displaystyle P(\text{two-headed} \mid \text{heads}) = \frac49$.

Ques­tion 8: Why an accu­rate test can mis­lead#

The prob­lem#

Explain in one sen­tence why a test with high accu­racy can still give mostly false pos­i­tives.

Under­stand­ing the prob­lem#

You are asked for a rea­son, not a num­ber. Think about Exam­ple 3 of the les­son and Ques­tion 5 above.

The idea#

In Bayes' the­o­rem, the num­ber of true pos­i­tives depends on the prior $P(\text{disease})$. When that prior is tiny, the healthy group is huge, and even a small false-pos­i­tive rate applied to it gives many pos­i­tives.

Step-by-step solu­tion#

Step 1. Recall Exam­ple 3: $1\%$ preva­lence, $95\%$ detec­tion, $4\%$ false pos­i­tives. Out of $10\,000$ peo­ple, about $95$ true pos­i­tives and $396$ false pos­i­tives.

Step 2. So only about $\displaystyle \frac{95}{491} \approx 19\%$ of pos­i­tives are gen­uine, even though the test itself is accu­rate.

Step 3. The cause is the rar­ity of the con­di­tion, which makes the healthy group very large.

Check­ing the answer#

If the con­di­tion were com­mon (say $50\%$), the same test would give $\displaystyle P(\text{disease} \mid +) = \frac{0.475}{0.475 + 0.02} \approx 0.96$. So the prior really is what dri­ves the effect.

Answer#

When the con­di­tion is rare, the few true pos­i­tives are out­num­bered by the false pos­i­tives com­ing from the much larger healthy group.