Every now and again, I read something that simply makes my head spin. This morning’s NYT article on prediction markets was just such a head-spinning column. It reached such a bizarre conclusion that it begs for comment.
Longtime readers are familiar with my views on prediction markets. I believe they have some value, when applied correctly. Where prediction markets excel is in acting as a collective real time polling mechanism for their participants. The closer the collective group is to the broader population, the more accurate these markets tend to be. Where they do poorly is when the collective attempt to “pool their ignorance” and forecast the random or the unknowable.
There is strength in any mechanism that can factor a large pool of participants collective experience and knowledge, discount several possibilities into the most likely outcome. These market mechanisms are hardly the Oracles of Delphi their supporters make them out to be. It is only by din of Humans’ even worse forecasting abilities (with experts leading the way down) that these markets garner so much respectability in the first place. Blame fuzzy thinking for placing too much credibility on a mechanism with so mixed a track record. There is also a subtle but important distinction between forecasting the future versus discounting various outcomes.
Which leads me to today’s NYT column. In a bizarre twist of logic, the massive failure of the prediction markets in the US 2006 mid-term elections somehow gets credit for being right. They were not a day late and a dollar short, they were completely, totally and incontrovertibly wrong.
Except in the NYTimes.
Consider the following: As of 11:50 p.m. on the evening of Election day, with the voting completed (except for Hawaii) and the majority of the ballots counted, Intrade gave the Republicans an 85 percent chance of retaining the Senate. Of course, we know that’s not at all what happened. If that’s not a forecasting failure, then what is?
If forecasting the results of an election held yesterday is considered prescience, then sign me up — and find me a bookie who will take my Superbowl bets on the Monday after the game. I suspect I will at least cover the spread.
To reiterate my views, these markets — thinly traded, easily manipulated, poorly administered — do have some real value. However, they have hardly been "remarkably clairvoyant," as the Times describes them. Consider these recent acts of "clairvoyance:"
GOP Retention of Senate
Michael Jackson Trial Results
Morgan Stanley CEO Purcell resignation
Howard Dean’s Iowa Primary
pre-Election 2004
Election day trading frenzy
There are some interesting explanations for some of these errors: Pooling the collective ignorance of millions does not produce Wisdom. The Michael Jackson Trial and Morgan Stanley’s CEO are perfect examples of that.
As to the prediction market’s failure of the Howard Dean primary and the 2006 mid-term elections, I’ve been toying with a theory. Since the political affiliation of Wall Street tends toward Republican, and since there is a big overlap between Wall Street and the participants of the prediction markets, the collective had an inherent bias built into it. Remember, where these markets excel is when they act as a realtime polling mechanism of their particpants. If the pool has a bias, the outcome may very well too. Hence, these widescale failures.
~~~
Bottom line: While having value, prediction markets are subject to error and bad outcomes. Some of this is due to their relative thinness of the markets; some is due to the inherent biases of the participants, and their failure to parallel the population at large.
They should be considered with the recognition of their fallibility.
>
Source:
Odds Are, They’ll Know ’08 Winner
DAVID LEONHARDT
NYT, February 14, 2007
Economix
http://www.nytimes.com/2007/02/14/business/14leonhardt.html
Under the spell of Surowiecki’s book, without having read past the first few pages.
Keneth Arrow’s works have proved it, no consensus without an axiom of dictatorship.
After this you may doubt on “Vox populi,vox Dei”
Barring a very few observers/predictors who are usually right (think Krugman), the Times has long ago stopped being a bastion of sound thinking. Or reporting. Anyone seen Judith Miller?
Nice post. From my dealings with tradesports and other prediction market sites, they lag more than they lead. They’re definitely not God’s gift to prognosticators that many stories lead you to believe. I like to track the sports games on tradesports simply because you can trade contracts throughout the actual games, which is so much different than traditional sports betting. But in general I think they give very broad indications on where things stand.
The futures are still interesting to look at though.
The way a predictive market could be accurate is if the participants have some valid knowledge of the topic – who’s the better team, who’s the stronger company, etc. In something like an election, the only way it could be accurate is if it the participants make up a near-perfect mirror sample of the population that will do the electing – clearly NOT the case here, or in any predictive “market” I’ve seen. Pollsters have a far greater shot at success (accuracy) in this arena.
I don’t think you understand what these prediction markets really imply. You mentioned several events that had a ‘market’-implied chance of happening of ~80%, but ultimately closed at 0%. It doesn’t mean that just because they were at 80% they would go to 100%; it meant that people gave a 80% chance of happening based on information available at that time.
Now because you picked a few contracts that the ‘market’ predicted wrong you are concluding that those markets are flawed or have no use. I disagree. What you should do is take for example a sample of 100 contracts that trade in the 80% range and track how they end. If you find that far less than 80 of them end at 100 then perhaps you have a point. See what I mean?
Taking up for Barry here in reply to Rubens: he did not say these markets have no use; he was merely STUNNED by the article he sited, which trumpeted this particular failure to predict correctly as a resounding triumph of predictive markets. His problem was with the reporting, not with the concept of the predictive market. At least that’s how I read him.
Ruben –that was an 85% chance of occurence AFTER THE EVENT WAS OVER – and they still got it wrong .
That hardly qualifies as “remarkably clairvoyant”
from http://en.wikipedia.org/wiki/List_of_Latin_phrases_%28P%E2%80%93Z%29
…the source is usually given as the monk Alcuin, who advised Charlemagne that nec audiendi qui solent dicere vox populi vox Dei quum tumultuositas vulgi semper insaniae proxima sit, meaning “And those people should not be listened to who keep saying, ‘The voice of the people [is] the voice of God,’ since the riotousness of the crowd is always very close to madness.” (Works, Letter 164)
This post attempts to create controversy, where frankly, there is none. Take Ritholz’s comment seriously:
“These market mechanisms are hardly the Oracles of Delphi their supporters make them out to be. It is only by din of Humans’ even worse forecasting abilities (with experts leading the way down) that these markets garner so much respectability in the first place.”
This is precisely the point! If markets do worse than the alternative then I choose the market. And sure, some things are hard to forecast, but I still would prefer the thing that does better.
And that is one of the main implications of recent research on prediction markets. Ritholz ends up echoing precisely the findings he claims to be countering.
Yes, markets are fallible. But I still prefer the least fallible forecasting insitution. And in many domains, that appears to be prediction markets.
If you can find prediction market prices that you think are wrong, you should expect to make money by trading in those markets. And doing so will increase their accuracy.
Some good comments have been made in defense of prediction markets, but Barry’s objection is really a minor, nearly semantic distinction, and one that is colored by his basically correct yet irrelevant opinion that PMs are still mainly domain of blogger “weenies”, gamblers and academics as opposed to serious finance guys.
It is true that most of what prediction markets do is aggregate current information, “discounting” more than “predicting” as Barry would say. Even when traders extrapolate trends and make synthetic judgments about the future, they are relying on present/past information.
Ritholtz’s beef is more about how some authors (including perhaps a strawman or two) write about PMs, as opposed to the subject itself. He would agree that sufficiently liquid PMs perform better than experts on questions where there is actually dispersed information to be aggregated.
Often, not to say that this happened here, proponents of maligned or unpopular positions tend towards hyperbole.