Why Prediction Markets Fail

Over the years, I have been critical of prediction and futures markets. In particular, the specific ways certain parties misuse them (i.e., politics).

However, I am a big believer that markets can generate valuable economic and investing data that can be quite helpful when handled appropriately. In my own work, we rely on the 10 year interest rate as part of our overall economic modeling. On the equity side, we use various trend component for indices in addition to volume, money flow, short interest and institutional ownership for sectors and specific companies.

In each of these cases, we are relying on markets that are deep, diverse and trading in dollar volumes measured in trillions. This is part of my criticism about future markets: They are thin, trading volumes are anemic, the dollar amounts at risk are pitifully small. Thus, these markets are subject to failure at times.

Indeed, the excuse making for the failure of the futures market began almost as soon as the New Hampshire primary ended (see Why the Online Prediction Markets Blew New Hampshire, Prediction markets are forecasting tools of convenience that feed on advanced indicators, etc.)

None of these really looked at what makes futures markets work, how they can fail, and what their strengths and weaknesses are. This morning, I want to delve into some the problems these markets can have, why they fail, and what value they do offer.

First, Let’s start by reviewing some of the more spectacular prediction market failures:

• Iowa Primary 2004 (Howard Dean vs everyone else)

• 2006 GOP Senate Retention

• New Hampshire Primary 2008 (Obama vs Hillary)

In the area of politics, I am in agreement with Dan Gross, who wrote “these are less futures markets than immediate-past markets.” In other words, bettors essentially aggregate polling data that is already out — rather than forecasting the unknown future.

When it comes to politics, its not as if the Iowa electronic market or Intrade gamblers have any special insight or inside knowledge. They don’t “know” anything — as individuals or collectively — that the rest of the public (insiders included) don’t already know. They’re all reading the same newspapers, blogs, polls, etc., and responding to whatever broad narrative happens to be coming out that day or week. They respond just as any other focus group would, off of the same information the electorate has.

I sometimes think of the political futures markets as a focus group unto themselves. Here’s where things get really interesting:  When the group is something less representative of the target market, they get it wrong with alarming frequency. Indeed, the closer the traders are as a group to the target decision makers/voters,  the better their track record.

Consider smaller “electorates” beyond the political events mentioned above. Think of the Michael Jackson Trial, or the Morgan Stanley CEO Purcell resignation betting — these much smaller groups of 12 jurors, or a handful of MS directors don’t lend themselves very well to focus groups or polling.

Hence, the lower success rate.

Back to political futures: I suspect the traders at Intrade in 2006 missed the GOP Senate loss, because as a group, they themselves skewed away from the rest of the nation. The demographics of Intrade traders are likely higher income, better educated, leaning more GOP than the USA as a whole. Hence, the variation of this focus group away from the larger electorate led to the bad forecasts there.

We see this especially in close elections: When they are similar to the voting electorate, they get it right; as they differ, their error rate goes up. This commonality to where they fail is one that’s worth exploring further by some suitable academic (Robin? Justin?)

A more academic way to analyze these issues is to look at what makes markets work. Lets go to Legg Mason’s Michael Mauboussin:

“The “wisdom of crowds” is a colloquial way of describing the market as a complex system. The work on wisdom of crowds shows that when certain conditions are met—diversity, aggregation, and incentives — markets tend to be efficient. Conversely, when one or more of those conditions  are violated, markets can and do become inefficient (i.e., price is no longer an unbiased reflection of value).

For a host of social psychological and sociological reasons, diversity is the most likely condition to be violated. Here, too, market observers have recognized the importance of diversity. But what’s essential to recognize is the relationship between diversity breakdowns and asset price performance is nonlinear. Diversity can be on the decline as the asset price rises, which makes the market fragile. Like a rubber band that’s stretched, the tension builds and the inevitable snapback is often painful.”

Hence, we see more potential sources of market failure as these conditions  are violated:

1) An insufficient amount of incentives;

2) A lack of diversity of ideas;

We’ve discussed both of these, albeit in less than academic terms: The small thinly traded markets means the money at risk is rather small — measured in $1000s, not trillions of dollars. Hence, the incentive is minimal — or perhaps non-monetary. Maybe this attracts a somewhat more politically passionate — and less objective — group of traders.

That can lead to a lack of intellectual diversity. As Mauboussin discussed, that is the most likely source of market failure. And as we review above, this is particularly acute when it comes to smaller groups not paralleling the demographics and or politics of Futures markets’ traders.

~~~

In some ways, Futures markets are similar to many Wall Street economists: at times, they merely extrapolate the current trend forward. Hence, they can be right for most of a run — an economic expansion or stock bull market, etc.

However, this process leads them to completely miss major turning point, or an unexpected shift or event. I suspect this factor also played into the New Hampshire 2008 failure…

>

Sources:
Mauboussin on Strategy: Fat Tails and Nonlinearity
Diversity Breakdowns and Invisible Vulnerability.
Legg Mason Capital Management, DEC-20-2007
http://www.leggmason.com/individualinvestors/documents/insights/D4114-FatTailsNonlinearityLMIS.pdf

Bad Bet
Why were the political futures markets so wrong about Obama and Clinton?
Daniel Gross
Slate, Tuesday, Jan. 8, 2008, at 10:12 PM ET
http://www.slate.com/id/2181745/