Total Probability Theorem

MATTotal Probability & Bayes’ Theorem

The total probability theorem computes $P(A)$ by splitting the sample space into mutually exclusive, exhaustive cases $E_1,E_2,\dots,E_n$ and adding up the contribution of $A$ through each. It is the forward, two-stage "tree" calculation (e.g.

choose a bag, then draw a ball) and is also the denominator that Bayes' theorem relies on.

Total probability theorem

P(A) = \sum_{i=1}^{n} P(E_i)\,P(A \mid E_i)

E_1,\dots,E_n

partition the sample space (mutually exclusive, exhaustive, each

P(E_i) > 0

);

P(E_i)

= probability of stage-one case

i

P(A\mid E_i)

= probability of

A

within that case.

Two-case version

P(A) = P(E_1)\,P(A \mid E_1) + P(E_2)\,P(A \mid E_2)

For exactly two cases (e.g. Bag I vs Bag II), with

P(E_1)+P(E_2)=1

Partition condition

E_i \cap E_j = \varnothing \ (i \ne j), \qquad \sum_{i=1}^{n} P(E_i) = 1

The cases must not overlap and must cover all possibilities; this is what makes the theorem valid.

Picture a tree: the first branches are the cases $E_i$ with probabilities $P(E_i)$ , the second branches are $A$ with probabilities $P(A\mid E_i)$ .
Each complete path contributes $P(E_i)\,P(A\mid E_i)$ ; sum the contributions of all paths leading to $A$ .
Choosing a bag or coin at random from $k$ equally likely options gives $P(E_i) = \dfrac{1}{k}$ for each.
If a fair coin (prior $\tfrac{1}{3}$ ) and a double-headed coin (prior $\tfrac{2}{3}$ ) are used, then $P(\text{heads}) = \dfrac{2}{3}\cdot 1 + \dfrac{1}{3}\cdot\dfrac{1}{2} = \dfrac{5}{6}$ .
The cases must be mutually exclusive and exhaustive; verify $\sum P(E_i) = 1$ before summing.
This theorem gives the unconditional $P(A)$ ; if the question instead asks "given $A$ , which case?" you continue with Bayes' theorem using this $P(A)$ as the denominator.
Conditional probabilities $P(A\mid E_i)$ come from the composition of each case (e.g. white balls $\div$ total balls in that bag).

Using the theorem on a two-stage experiment

1Check the first-stage events $E_1,\dots,E_n$ really do partition the sample space: they must be pairwise exclusive and exhaustive, so $\sum P(E_i) = 1$ .
2Draw the tree: one branch per cause, then one branch per outcome of the second stage.
3Multiply along each path to get $P(E_i)\,P(A\mid E_i)$ .
4Add the paths that end in $A$ : $P(A) = \sum_i P(E_i)\,P(A\mid E_i)$ .
5If the experiment changes the pot between stages (a ball transferred, a card not replaced), recompute the second-stage probabilities for each branch separately — they are no longer the same number.

Where the marks go

Adding the case probabilities $P(E_i)$ directly without multiplying by the conditional $P(A\mid E_i)$ on each branch.
Using cases that overlap or that do not cover all outcomes, so they fail to form a partition and the sum is wrong.
Forgetting the $\dfrac{1}{k}$ stage-one factor when a bag or coin is "chosen at random".
Confusing this forward calculation of $P(A)$ with the reverse question $P(E_i\mid A)$ , which needs Bayes' theorem.

How the board asks it

Applicationthe two-stage bag-and-ball tree
Bag A contains $4$ white and $5$ black balls; Bag B contains $6$ white and $3$ black balls. A bag is chosen at random and a ball is drawn from it. Find the probability that the ball drawn is white.
Applicationproduction percentages as the priors $P(E_i)$
In a factory, machines $A$ , $B$ and $C$ produce $25\%$ , $35\%$ and $40\%$ of the bolts, of which $5\%$ , $4\%$ and $2\%$ respectively are defective. A bolt is drawn at random from the total output. Find the probability that it is defective.
Numerical $P(A)=\sum P(E_i)\,P(A\mid E_i)$
A box contains $3$ coins: one is fair, one has heads on both faces and the third has tails on both faces. A coin is picked at random and tossed once. Calculate the probability of obtaining a head.
Applicationtransfer-of-ball staged experiment4 mk
An urn contains $5$ red and $3$ white balls. One ball is drawn at random and transferred to a second urn containing $4$ red and $4$ white balls; a ball is then drawn from the second urn. Find the probability that the ball drawn from the second urn is red.
Give reasonsthe exhaustiveness condition $\sum P(E_i)=1$
A student writes $P(A)=P(E_1)P(A\mid E_1)+P(E_2)P(A\mid E_2)$ for events with $P(E_1)=0.4$ and $P(E_2)=0.5$ . State, with reasons, whether $E_1$ and $E_2$ form a valid partition for applying the total probability theorem.

Practise this topic

PreviousIndependent Events NextBayes’ Theorem

Written for Sublevo. Question text quoted anywhere in these notes is the Council’s and carries its year and paper; the board’s own diagrams are not reproduced.