Independence and conditional independence

When considering independence and conditional independence, there are several well-known examples to show that neither implies the other. However, even though one can intuitively grasp that conditional independence does not imply independence tout-court, there is one special case that challenges intuition a little bit.

We want to show here that if two events $A$ and $B$ are independent conditional to a third event $C$ and also to its complement $C^C$ , then the two events are not necessarily independent, as it is shown in the following example.

Here event $A$ is coloured in blue, $B$ in red, and $C^c$ in yellow. Where the colours are superposed, so are the events. We have that

$\begin{gather*}P(A|C) = \frac{1}{3}, \ P(B|C)=1,\ P(A\cap B|C)=\frac{1}{3}\\ P(A|C^c)=\frac{1}{2}, \ P(B|C^c)=\frac{1}{3}, \ P(A\cap B|C^c)=\frac{1}{6},\end{gather*}$

so conditional to $C$ and $C^c$ , the events $A$ and $B$ are independent. However, $P(A)=\frac{5}{12}$ , $P(B)=\frac{2}{3}$ . Their product is $\frac{5}{18}$ , whereas $P(A\cap B)=\frac{1}{4}$ , so they are not independent.

It is interesting to notice that the two events are independent only if one of the following holds:

$P(A|C)=P(A|C^c)$ ,
$P(B|C)=P(B|C^c)$ ,
$P(C)\in\{0,1\}$ .

The latter is quite obvious, but the previous two ones tell us that in order for the two conditional independences to imply the overall independence, we need the probability of at least one of the two events to be constant across $C$ and $C^c$ .

Just a concluding remark: to have a nice representation of conditional independence, one can check the theory of graphical models (see for example ).

Independence and conditional independence

Bibliography

One thought on “Independence and conditional independence”

Leave a Reply Cancel reply