Making sense of Bayes’ Theorem

So today I was trying to help out my twin brother with understanding a statistics problem involving Bayes’ Theorem. Besides the fact that they always ask probability problems in the most confusing way possible, very few actually explain Bayes’ Theorem very well. So here I attempt to provide some additional explanation involving Bayes’ Theorem, hopefully some additional insight will be provided and Bayes’ Theorem will become a little bit easier for you.

First and foremost I would like to say that Bayes’ theorem is more than just a formula for just calculating probability. However, many explanations end with just the following equation:

Equation 1: The formula for Bayes’ Theorem. Image used for educational purposes only, pulled from data-flair.training.

First it will do to visualize probability as a space of all probabilities, and all we want to do is find a specific area within that space.

Now, how can we take Bayes’ Theorem and apply it to probability? With Bayes’ Theorem we have a hypothesis, and evidence. But the hypothesis being true is an event of probability, the inverse must also be taken into account so we also have the probability of the not hypothesis being true. This gives us two probability values.

P(H) = Probability of the Hypothesis
P(!H) = 1 – P(H) = Probability of the not Hypothesis

Now we can take a square with an area of one, and make a slice at some point x along its width so that we have two rectangles with widths corresponding to the probability of the hypothesis being true and the not hypothesis being true.

Then for both rectangles, for some amount of the height for both hypothesis and not hypothesis, the evidence will also be true. A slice can then be made at that height for our two rectangles to make four rectangles.

With Bayes’ Theorem you can calculate out of your new evidence, what is the probability of the hypothesis being true. With this in mind, out of your four blocks you only need the two where the evidence is true.

This is all illustrated in the figure below. The width of the square is the sum of the hypotheses, and the height is defined by the sum of evidence and not evidence.

Figure 1: Bayes’ Theorem visualized as blocks.

You then are given a formula that looks something like this:

Equation 1: Bayes’ Theorem, simplified. Image used for educational purposes only, pulled from data-flair.training

P(E | H) means probability of Evidence given the hypothesis. The result given by Bayes’ Theorem is the probability of the hypothesis given evidence, P(H | E).

Knowing what I described above, the Bayes’ Theorem can also be described as thus using the colors from Figure 1:

Blue divided by the sum of Blue and Yellow

Probability of picking Blue out of Blue and Yellow with the areas given.

Once you Understand Bayes’ Theorem there are some things to remember:

Differentiating between which is evidence and hypothesis is not of much value. This is because you are calculating the areas of rectangles and the multiplication property is commutative. Thus sometimes it is better to describe Bayes’ Theorem as Event A given Event B and be used for calculating dependent events.
Remember that the probability of the hypotheses given evidence is not the same for the hypothesis and not hypothesis.
Remember that with Bayes’ Theorem, one is not limited by only the variables described in Figure 1. Often you may be given other related variables such as P(!E | !H) or the dark gray square in Figure 1. One of the ways to recognize a problem will require Bayes’ Theorem is when you start reading things such as Event A given B. For example: “The probability of not having a beacon, given the object was not discovered is 90%.” A way to help solve the word problems like this is to write out what is given to you first, then use those variables to find what you need to use the formula.

Bayes’ Theorem is a very useful theorem that has a wide range of applications. Some applications involve neural networks and AI’s, it can be used to test how likely test results were pure chance or may actually be true.

Thank you for reading this far. I hope that my article has been helpful in explaining Bayes’ Theorem. Below you can find some other good sources on Bayes’ Theorem if you want to learn more or are still confused.

Related Links