In yesterday’s afternoon reads, I mentioned a Boston Globe story “That guy who called the big one? Don’t listen to him.”
While it ostensibly looks at the track record of Nouriel Roubini, it isn’t really about him –its really about outliers and mean reversion in forecasting.
“How can someone with the insight to be so right about a major event be so wrong about so many other ones? According to a recent study, it’s simple: The people who successfully predict extreme events, and are duly garlanded with accolades, big book sales, and lucrative speaking engagements, don’t do so because their judgment is so sharp. They do it because it’s so bad . . .
[Oxford economist Jerker Denrell and Christina Fang of New York University] are the latest in a long line of researchers dismantling the notion that predictions are really worth anything. The most notable work in the field is “Expert Political Judgment” by Philip Tetlock of the University of Pennsylvania. Tetlock analyzed more than 80,000 political predictions ventured by supposed experts over two decades to see how well they fared as a group.
The answer: badly. The experts did about as well as chance. And the more in-demand the expert, the bolder, and thus the less accurate, the predictions. Research by a handful of others, Denrell included, suggests the same goes for economic forecasters. An accurate prediction — of an extreme event or even a series of nonextreme ones — can beget overconfidence, which can lead to making bolder and bolder bets, and thus, more and more errors . . .
There’s no great, complex explanation for why people who get one big thing right get most everything else wrong, argues Denrell. It’s simple: Those who correctly predict extreme events tend to have a greater tendency to make extreme predictions; and those who make extreme predictions tend to spend most of the time being wrong — on account of most of their predictions being, well, pretty extreme. There are few occurrences so out of the ordinary that someone, somewhere won’t have seen them coming, even if that person has seldom been right about anything else.”
Fascinating stuff, worth reading in full . . .
>
Previously:
The Folly of Forecasting (June 2005)
The Illusory World of Economic Forecasting (September 19th, 2006)
Guru Time of Year (January 2nd, 2011)
Source:
That guy who called the big one? Don’t listen to him.
Joe Keohane
Boston Globe, January 9, 2011
http://www.boston.com/bostonglobe/ideas/articles/2011/01/09/that_guy_who_called_the_big_one_dont_listen_to_him
What's been said:
Discussions found on the web: