Imagine you are on the battlefield and you are under attack by an enemy army containing 100 men. You know from intel that this army consists of 99 regular soldiers and 1 elite sniper, randomly distributed. One of these army members opens fire at you from 400 meters away and nearly hits you. If the elite sniper fired at you, there’s an 100% chance this would happen. However, the regulars are still good shots without sniper rifles, and could make the same shot about 0.1% of the time. What is the chance you were shot at by the sniper?
Thus enters Bayes Theorem, a way to assess probability.
Applying Bayes Theorem
Well we know that sniper is randomly distributed and there is only one of him. But we also know that he definitely could have made that shot, whereas it is unlikely that a regular made the shot. So we have a problem of which probability to use: 1%, or 100%, or something in between? We’ll use Bayes Theorem to decide.
Bayes Theorem looks like this:
But what does all that math-y stuff mean? Well, we have three things: We have H, which is our hypothesis (“A sniper has fired upon you”) and we have E our evidence (“The shot was made from 400 meters away”).
Lastly, we have the probability equation P(). P(H|E) is the probability of H given E, or what we want to solve for. This is the probability of our hypothesis being true given the evidence we have. In this scenario, it is the probability that a sniper fired at you given what we know about the shot made.
P(E|H) is the other way around, the probability of E given H, or the probability of the evidence assuming that our hypothesis is true. In this scenario, it is the probability that such a shot was made assuming you were fired upon by a sniper.
The last two P(H) and P(E) are the individual probabilities. P(H) is the probability of E assuming nothing else, or the probability that a sniper is firing upon you, assuming nothing else. P(E) is the probability that such a shot could be made at you, assuming nothing else.
Calculating P(H), P(E), and P(E|H)
So what do we need to know for Bayes Theorem? We want to solve for P(H|E), and to do we need values for P(E|H), P(E), and P(H).
First lets get P(H) and P(E|H) since they’re easy. What is P(H)? What is the probability that the sniper is firing upon you? Well since the sniper is randomly distributed among an army of 100, the probability the sniper is attacking you assuming nothing else is 1%. Therefore P(H) = 1% = 0.01.
Now what is P(E|H)? The symbols look complicated but it is an easy question: What is the probability the shot was made assuming it came from the sniper? We know from the story that this is 100%. P(E|H) = 100% = 1.
Now for the hard one: what is P(E), or what is the probability that such a shot could be made? Well the single sniper could have made that shot, but regulars can make the same shot 2% of the time. The way we solve this is to add two quantities: the first is the percentage of the army the sniper are multiplied by the percentage of times the sniper could make the shot, and the second quantity is the percentage of the army the regulars are multiplied by the percentage of times the regular could make the shot. P(E) = [(1%)(100%) + (99%)(0.1%)] = [(0.01)(1) + (0.99)(0.001)] = [0.01 + 0.00099] = 0.01099
Answering the Hypothetical With The Formula
Now we can put it together:
So via Bayes Theorem, we know that despite the rarity of snipers, there is a 91% chance you were just attacked by the sniper. You better call in reinforcements! Congratulations on using probability effectively!
So let’s sharpen our Bayes Theorem usage by another, more relevant, example. This one is the classic example for mammograms, that I have adapted from Eliezer S. Yudkowsky, along with pretty much everyone else who has written an explanation of Bayes Theorem. It goes like this:
- 1% of women have breast cancer.
- If a woman has breast cancer, a mammogram will detect the breast cancer 80% of the time and fail (false negative) 20% of the time.
- If a woman does not have breast cancer, a mammogram will return a false positive 9.6% of the time.
Now we ask the key question: If you get a positive result from the mammogram, do you have cancer?
Well again, we determine our hypothesis, our evidence, and find values for P(H), P(E), and P(E|H). In this example, our hypothesis is I have cancer and our evidence is I got a positive result from a mammogram.
Since only 1% of the population has cancer, P(H) = 1% = 0.01.
Since the mammogram will return a positive result 80% of the time if I have cancer, P(E|H) = 80% = 0.8
Since a mammogram returns a positive result 80% of the time if there is cancer and 9.6% of the time if there isn’t cancer, P(E) = [(0.01)(0.8) + (0.99)(0.096)] = [0.008 + 0.09504] = 0.10304. This is the percentage of all potential mammograms that return positive, regardless of whether or not the person has cancer.
Therefore, if you get a positive mammogram result, there is only a 7.76% chance you have cancer!
The Counterintuitive Lesson
This result at first seems rather weird. After all, isn’t this supposed to be an accurate test? Well then, why would it not really do much to tell me if I have cancer or not? Why is it that if I test positive for cancer there is still a 92.24% chance I don’t have cancer.
That’s because the rate of people who actually have cancer is so low that the chance of a false positive over a very large population greatly skews the test.
Therefore Bayes Theorem is teaching us some important lessons:
- Background evidence is important: as seen in the sniper example, the fact of the shot greatly outweighs the rarity of the sniper
- Tests can be wrong: tests can return false positives and false negatives.
- There is a difference between the event and a test: having cancer is different than testing positive for cancer.
- False results greatly skew our data: If you’re searching for something really rare and find it, it becomes more likely you got a false positive or false negative. (This is exactly why tests need to be replicable.)
- Everything we know is still a test: Since it is impossible to have all of the information, we can only make our best guess using Bayes Theorem.
Bayes Theorem, Ghosts, Gullibility, and Replication
Now that we have an understanding of Bayes Theorem and the important philosophy behind it, we can apply it to an important point. Consider the number of people who have gone hunting for ghosts. This represents people preforming a test for ghosts. While we will never be able to figure out P(H), the probability that ghosts actually exist, and therefore cannot apply Bayes Theorem to get an actual probability of ghosts, we can apply the general philosophy here.
Since ghost hunting is a test, it also can return false positives at some rate. Someone can think they see a ghost when there really is no ghost at all. (Ghost hunting can also give false negatives at some rate as well.) Now, hypothetically imagine that we have gone testing for ghosts and have gotten a positive result. Should we assume ghosts are real?
Well, no. Perhaps the tests for ghosts are more like the mammogram example. Given that thousands of other tests for ghosts have failed for whatever reason, we have to reason to assume that we just got a false positive until more testing can be done. We would have to repeat our test a few more times, as well as have other, unrelated, unbiased people replicate our test before we have conclusive results. We just simply won’t know until we get more background information and evidence. Our knowledge is always incomplete.
The strong reality of false positives means that we need to adopt some skepticism. We shouldn’t believe in ghosts or ESP just because one test shows they exist, especially if hundreds of other tests show they don’t exist. Eliezer Yudkowsky’s essays “I Defy the Data” and “Your Strength as a Rationalist” elaborate on this.
Lastly, consider another example: We have a hypothetical friend named Bob who tells the truth 99% of the time. We both write down a number at random between 1 and 1000. I hand him my number and, without ever looking at either number, ask him if they match. He says “Yes, they match”. Should we assume Bob is telling the truth here, since he always tells the truth 99% of the time? Bayes Theorem says that we shouldn’t be so recklessly naïve. We should ask to see both slips of paper for ourselves, and get very suspicious when he refuses.
And also, we can block spam with it.
This is not meant to be an exhaustive explanation of Bayes Theorem, but a simple intuitive guide to get you going and to make a few points. If you want, here are a lot more references to fully get a grasp of Bayes Theorem, in order from simple to complicated:
I now blog at EverydayUtilitarian.com. I hope you'll join me at my new blog! This page has been left as an archive.