Bayes' Theorem

In my last post, I talked about Bayesian reasoning, but didn't really explain what it actually was. I was going to make a footnote explaining it, but realized it too big for a footnote, so I'm going to write a post explaining it.

I'll start with some terminology.

P(A) is the probability that something is true. For example, let's say I roll a fair die and A is "I roll an even number". Since the die is fair, all outcomes are equally likely, so P(A) = 3/6 = 1/2, since 2, 4, and 6 are even.

~A is not A. ~A then is "I roll an odd number". Also P(~A) = 1 - P(A), for any A. In this case, P(~A) = 1 - 1/2 = 1.

P(A|B) is read probability of A given B. Let's say B is "I roll a number greater or equal to 4". So, if I roll the die, see the number is greater or equal to 4, then the P(A|B) = 2/3, because 4 and 6 are even

Bayes' Theorem states that P(A|B) = P(B|A)*P(A) / ( P(B|A)*P(A) + P(B|~A)*P(~A)). So, if we want to know how likely A is after making some observation, all we have to know is how likely the observation is if A is true, how likely the observation is if A is false, and how likely A was before we made the observation.

If we're not interested in the exact value of P(A|B), but just whether P(A|B) is higher or lower than P(A), then all we need to know is whether P(B|A) is higher or lower than P(B|~A). If P(B|A) > P(B|~A) then P(A|B) > P(A). If P(B|A) < P(B|~A) then P(A|B) < P(A). And, if P(B|A) = P(B|~A) then P(A|B) = P(A).

Relating this back to the other post, I said that not observing evidence for a phenomenon makes that phenomenon less likely. Here's an example: Tigers don't exist. Evidence: There are no tigers in my house. A - Tigers exist. B - No tigers in my house. Now, if tigers do exist, it's very unlikely that they would be in my house. Not the right environment, needs some way to get in, etc., etc. So, P(B|A) = .99999999. But, if tigers don't exist, then it is absolutely impossible for tigers to be in my house. So, P(B|~A) = 1. 1 > .99999999, so P(A|B) < P(A). Of course, there's lot of other evidence and stronger evidence that tigers do exist, so P(A|B) is still very high.