Bayes’ Theorem – Part 2

In the previous post, we have seen Bayes’ theorem and a way to use it in the framework of updating mathematical models. We concluded by foreshadowing multiple updates of a model (or a belief) following multiple experiments. So this is where we take off now.

Assume that we have two experiments E_1 and E_2. In the example we have seen in the first part, this could be “The first student I ask is in the program focused on Models” and “The second student I ask is in the program”. What we want to compute is our belief a posteriori, after both replies:

    \begin{align*}P(H|E_1, E_2) &= \frac{P(E_2|H, E_1)\cdot P(H|E_1)}{P(E_2|H, E_1)\cdot P(H|E_1)+P(E_2|H^c, E_1)\cdot P(H^c|E_1)} \\& = \frac{P(E_2|H, E_1)}{P(E_2|H, E_1)\cdot P(H|E_1)+P(E_2|H^c, E_1)\cdot P(H^c|E_1)}\\&\phantom{=}\cdot \frac{P(E_1|H)\cdot P(H)}{P(E_1|H)\cdot P(H) + P(E_1|H^c)P(H^c)}.\end{align*}

Continue reading “Bayes’ Theorem – Part 2”

Bayes’ Theorem – Part 1

In the first post, we discussed the importance of model updating, now we want to focus on the mathematics that comes into play. To do so, we will make use of Bayes’ theorem. In particular, we will need an alternative formulation of the (hopefully) well-known result.

First of all, we will briefly revise conditional probabilities.

Definition. Given two events A and B in a probability space, with P(B)\neq 0, the conditional probability of A given B is the quantity

    \[P(A|B)=\frac{P(A\cap B)}{P(B)}.\]

The idea is the following: we know that some event (B in the definition above) has happened, and we want to renormalise our notion of probability to the new reality we are living in, the one where B is certain, as it already happened. One can notice that the conditional probability with respect to B is itself a probability measure, and, in particular, satisfies all the properties of a probability. At the same time, it is worth noticing, with an example, that P(A|B), in general, is not equal to P(B|A). Among inmates in Italy, roughly 90% are male, so we can writeP(\text{male}\,|\,\text{inmate})=90\%. However, it is definitely false that the probability that an Italian male is an inmate, that is P(\text{inmate}\,|\,\text{male}), is 90%. At the same time, the two probabilities are connected with one another, as the following result shows.

Theorem (Bayes, probability form). Given two events with nonzero probability A and B, the following identity holds:

    \[P(B|A)=\frac{P(A|B)\cdot P(B)}{P(A)}.\]

Continue reading “Bayes’ Theorem – Part 1”