Bayes Theorem

Bayes Theorem

Bayes Theorem (Thomas Bayes 1761) allows calculation of true probability based on a sample of prior knowledge and a base rate1. Seeing evidence should update prior beliefs, i.e. it should restrict the space of possibilities and the ratio you need to consider after that 2.

Think about a base rate for the Hypothesis P(H) (e.g. what is the ratio of librarians to farmers, say 1:20). Now consider seeing evidence (e.g. the person is meek and reserved), in the case where the hypothesis is true, let’s say that among librarians that would be 40% or 4 out of 10 librarians this is P(E|H)1. Among farmers the rate is lower say 1/10 this would be P(E|¬H). In order to find P(H|E) or the probability of the hypothesis given that we see the evidence you have to consider what is the base rate times the rate of seeing the evidence (P(H)P(E|H)) divided by the total group that fits the evidence P(H)P(E|H) and P(¬H)P(E|¬H).

\begin{equation} P(A|B)=\frac{P(B|A) * P(A)}{P(B|A)P(A)+P(\neg{A})P(B|\neg{A})} \end{equation}


\begin{equation} P(A|B)=\frac{P(B|A) * P(A)}{P(B)} \end{equation}

  • P(A) = Prior (base rate, probability that the hypothesis is true)
  • P(B) = Total probability of seeing the evidence
  • P(A|B) Posterior: Given Hypothesis, and some New Evidence, what is the probability that the hypothesis is true
  • P(B|A) = Likelihood: Probability that we would see this evidence give that the hypothesis is true

  1. | means given that, so P(E|H) is Probability of seeing the evidence given that hypothesis is true


    Lewis, M. The Undoing Project: A Friendship That Changed Our Minds. (W. W. Norton & Company, New York, 2016).
    3Blue1Brown. Bayes theorem. at (2019).