Bayes’ Theorem

MATTotal Probability & Bayes’ Theorem

Bayes' theorem reverses conditioning: given that an effect has been observed, it finds the probability of each possible cause. With mutually exclusive, exhaustive causes $E_1,E_2,\dots,E_n$ and observed event $A$ , it updates the prior $P(E_i)$ to the posterior $P(E_i\mid A)$ .

It is a guaranteed board question through machine/defective, disease-test, and bag-of-balls scenarios.

Bayes' theorem

P(E_i \mid A) = \dfrac{P(E_i)\,P(A \mid E_i)}{\displaystyle\sum_{j=1}^{n} P(E_j)\,P(A \mid E_j)}

E_1,\dots,E_n

are mutually exclusive and exhaustive causes (a partition);

P(E_i)

= prior,

P(A\mid E_i)

= likelihood,

P(E_i\mid A)

= posterior; the denominator is

P(A)

Denominator = total probability of A

P(A) = \sum_{j=1}^{n} P(E_j)\,P(A \mid E_j)

The total probability of the observed event

A

; it is the sum of all numerator-type terms over every cause.

Two-cause form

P(E_1 \mid A) = \dfrac{P(E_1)\,P(A \mid E_1)}{P(E_1)\,P(A \mid E_1) + P(E_2)\,P(A \mid E_2)}

Common for two-bag or test (disease/no-disease) problems;

E_2 = E_1'

when there are only two causes.

Step 1: name the causes $E_i$ (which machine / which bag / disease vs no disease) and write their priors $P(E_i)$ , which must sum to $1$ .
Step 2: write the likelihood $P(A\mid E_i)$ = probability of the observed event under each cause (e.g. defective rate, test sensitivity).
Step 3: compute $P(A) = \sum P(E_i)\,P(A\mid E_i)$ (total probability), then divide the chosen term by it.
Posterior probabilities over all causes also sum to $1$ : $\sum_i P(E_i\mid A) = 1$ — a useful check.
Convert percentages to fractions or decimals (e.g. $50\% = 0.5$ ) before substituting.
In disease testing, sensitivity $= P(\text{positive}\mid \text{disease})$ and specificity $= P(\text{negative}\mid \text{no disease})$ , so $P(\text{positive}\mid \text{no disease}) = 1 - \text{specificity}$ .
A tree diagram (causes on the first branches, observed event on the second) makes the numerator one path and the denominator the sum of all relevant paths.

The four lines that earn full marks on any Bayes question

1Name the causes $E_1, E_2, \dots$ (the thing that happened *first* and is unobserved) and the evidence $A$ (the thing you *observed*). Getting these the wrong way round is the whole difficulty of the topic.
2Write the priors $P(E_i)$ — usually the percentages or the counts given in the stem.
3Write the likelihoods $P(A\mid E_i)$ — the success/defect/accuracy rates.
4Apply $P(E_k\mid A) = \dfrac{P(E_k)P(A\mid E_k)}{\sum_i P(E_i)P(A\mid E_i)}$ , where the denominator is just the total-probability sum.

Worked example · 4 marks

In a company

15\%

of employees are graduates and

85\%

are not.

80\%

of the graduates and

10\%

of the non-graduates hold administrative positions. An administrative employee is chosen at random. Find the probability that the employee is a graduate.

Causes: $E_1$ = graduate, $E_2$ = non-graduate. Evidence: $A$ = holds an administrative position.
Priors: $P(E_1) = 0.15$ , $P(E_2) = 0.85$ .
Likelihoods: $P(A\mid E_1) = 0.80$ , $P(A\mid E_2) = 0.10$ .
Total probability: $P(A) = 0.15\times0.80 + 0.85\times0.10 = 0.12 + 0.085 = 0.205$ .
Bayes: $P(E_1\mid A) = \dfrac{0.12}{0.205}$ .

Answer

P(E_1\mid A) = \dfrac{120}{205} = \dfrac{24}{41} \approx 0.585

. Note it is far below

0.80

— the small pool of graduates drags the posterior down, which is the point of the theorem.

Where the marks go

Confusing the likelihood $P(A\mid E_i)$ with the posterior $P(E_i\mid A)$ — Bayes' theorem exists precisely to convert one into the other.
Using only one cause's term as the denominator instead of the full sum $P(A) = \sum P(E_j)\,P(A\mid E_j)$ .
In test problems, plugging in specificity directly as $P(\text{positive}\mid\text{no disease})$ instead of $1 - \text{specificity}$ .
Forgetting that the priors must form a partition (mutually exclusive and exhaustive, summing to $1$ ) before applying the formula.

How the board asks it

Applicationthe machine/defective scenario6 mkAsked 2025
In a factory, machines $A$ , $B$ and $C$ manufacture $25\%$ , $35\%$ and $40\%$ of the total bolts. Of their outputs, $5\%$ , $4\%$ and $2\%$ respectively are defective. A bolt drawn at random is found to be defective. Find the probability that it was manufactured by machine $B$ .
Applicationthe bag-of-balls scenario4 mkAsked 2024
Bag $I$ contains $4$ red and $5$ black balls, while bag $II$ contains $3$ red and $6$ black balls. One bag is chosen at random and a ball is drawn from it which is found to be red. Find the probability that the ball was drawn from bag $II$ .
Applicationsensitivity and specificity as likelihoods
A test for a certain disease is $99\%$ accurate for those who have the disease and $99\%$ accurate for those who do not. If $0.5\%$ of the population actually has the disease and a randomly selected person tests positive, find the probability that the person actually has the disease.
Diagram / graphthe cause-then-event tree diagram
A man is known to speak the truth $4$ out of $5$ times. He throws a die and reports that it is a six. Draw a tree diagram showing the causes and the observed event, and hence find the probability that it is actually a six.
Applicationthe two-cause weighted-population form4 mkAsked 2023
An insurance company insures $2000$ scooter drivers and $4000$ car drivers. The probability of an accident is $0.01$ for a scooter driver and $0.03$ for a car driver. One insured person meets with an accident. Find the probability that the person is a scooter driver.

Practise this topic

PreviousTotal Probability Theorem NextRandom Variables & Probability Distributions (Mean, Variance)

Written for Sublevo. Question text quoted anywhere in these notes is the Council’s and carries its year and paper; the board’s own diagrams are not reproduced.