In Enlightenment Now, while discussing how reason gets hijacked by values, Steven Pinker cites a classic study by another famous psychologist, Philip E. Tetlock, on whether experts’ predictions can actually be trusted.
Tetlock is a leading expert in the study of forecasting. A few years ago, his popular-science book The Fox and the Hedgehog was published in China, and all his major academic contributions are reflected in that book.
The title The Fox and the Hedgehog comes from an English proverb: “The fox knows many things; the hedgehog knows one big thing.” In Tetlock’s view, the experts who like predicting the future—stock analysts, music analysts, economists forecasting whether housing prices will rise or fall, and so on—can be neatly divided into “foxes” and “hedgehogs.”
Hedgehog experts “know one big thing.” Their minds contain a preselected Big Idea, a grand value system they believe in beyond all doubt: left-wing values versus right-wing values, optimism versus pessimism, for example. When judging how anything will develop, hedgehog experts start from this Big Idea. And the accuracy of their predictions is often wildly off. They simply force the complicated world into an overly simplified grand idea because that is what they wish were true.
Fox experts “know many things.” They have no grand idea prepared in advance; they take each issue on its own terms and analyze each situation specifically. They use many analytical tools, choosing among them according to the particular problem they face. They gather as much information as possible from as many sources as possible, and they change their way of thinking when the question changes. Fox experts are far more accurate at predicting the future than hedgehog experts.
Hedgehog experts love using “-isms” to analyze “problems”—a classic case of values hijacking reason. Fox experts “study the problem, not the ideology,” allowing reason to rise above values. The result is that, when it comes to predicting the future, reason wins by a mile.
The forecasting tool fox experts use most often is Bayesian reasoning.
I won’t list the formula for Bayesian reasoning here. Once you understand the principle, it is already enormously helpful for estimating the probability of events in everyday life.
Put simply, Bayesian reasoning means first learning an event’s base rate—the probability of it happening in the future—and then adjusting that base rate up or down in light of new evidence. The adjusted probability is our estimate of the event’s likelihood.
Let me give you an example.
On 2015-01-07, the world was stunned by the Charlie Hebdo massacre. Terrorists stormed the headquarters of the French satirical magazine Charlie Hebdo, killing 12 people.
Soon after the event, a hedgehog expert declared that another terrorist attack was bound to happen in Western Europe by the end of March. But this prediction was really emotional: the massacre sharply raised the expert’s pessimistic expectations.
So how did Tetlock, a fox, estimate the probability of a terrorist attack?
Tetlock used Bayesian reasoning.
First, he looked up a list of terrorist attacks in Western Europe over the previous five years on Wikipedia. Dividing the total by five, he got 1.2 terrorist attacks per year. That base rate was his starting point.
Next, he began adjusting the base rate in light of recent factors. One important fact was that, since the Arab Spring in 2011, the counterterrorism situation had changed significantly, so Tetlock decided to use data from 2010 onward. That changed the number of terrorist attacks per year to 2.5.
Then, since the Charlie Hebdo attack, ISIS recruitment activity had increased—a reason to raise the probability—but security measures had also been strengthened, a reason to lower it. Balancing these two factors, increasing the probability by roughly one-fifth seemed reasonable. The resulting forecast was 0.8 attacks per year.
And from the day that earlier expert made his prediction until the end of March, there were still 79 days, so:
(69/365)*1.8=0.34
This meant that the probability of a terrorist attack in Western Europe by the end of March was about one-third.
That was a forecast made using Bayesian reasoning.
Of course, the result of any single forecast may not be accurate. But looking at the combined results of a large body of research, forecasts made with Bayesian reasoning are much more accurate than emotional forecasts.
First estimate the base rate from historical data, then adjust it in light of new facts—that Bayesian way of thinking is well worth learning.
Explanation
- This article in the “New Psychology Knowledge Mini-Course” series was a short popular-science essay written at the time to accompany content related to Zhichao Wei: New Psychology Knowledge Course. It was published on the now-closed platform Fantu, and the original link no longer exists.
Leave a Reply