Prediction Markets Redux

Its like waving a red flag in front of a bull. And I knew it would engender a response  . . . I just was hoping for a more intellectually honest one — one that might stimulate a dialogue on the value and inherent limitations of prediction markets. 

Last month, I mentioned that the InTrade Prediction Market got not one but two high profile events wildly wrong: The first was the Michael Jackson verdict — Not Guilty on all counts; The Second was the Morgan Stanley CEO Purcell resignation before June 30, 2005.

And the betting crowd didn’t just barely miss, either — they totally $*%t the bed.

My pointing this out caused a fit of pique amongst the prediction market crowd. Now that the hyper-ventilating has subsided, lets revisit (again) some of the readily identifiable problems with endemic to these markets.

1) The inherently unknowable is, by definition (duh) inherently unknowable. The further into the future one peers, the less accuracy one can expect on forecasting singular events. Call this the rule of the unknown future.

2) As we saw in both the Jackson and Purcell cases (as well as Howard Dean’s Iowa Primary), the crowd cannot create wisdom simply by pooling their collective ignorance.  Indeed, the less an event is dependent upon the actual behavior of crowds, the less likely it is that prediction markets will forecast a particular outcome correctly.

3) I use markets as a forecasting tool in my own work. The collective behavior of market participants can often provide insight into future market behavior. In other words, the present behavior of investors can sometimes be used to predict the future behavior of these same investors. The key is to use markets that are liquid, widely followed, with significant money at stake.

Example: Fed Futures have a very respectable track record of predicting Fed interest rate tightening and loosening. Not perfect, but good. That’s because fixed income managers use it to hedge their portfolios. They also have an idea of whether they will be buying or selling bonds, and therefore they collectively impact bond prices, yield, and therefore the Fed. There’s a feedback loop involved.

Speaking of the 30 trillion dollar U.S. Bond Market, low interest rates (set by collective bond traders and investors — not the Fed) have a good track record predicting future economic activity. Anyone who ignores the Long Bond’s low rates, and the implication of slowing economic activity, is inviting disaster.

Again, not perfect, but respectable. The depth, amount of money involved, and liquidity of this market is what makes its “predictions” worth following.

4) Where futures market do particularly well is where they track the behavior, beliefs and perspectives of their collective participants. For example, I do not believe political futures predict anything — instead, they are more like an alternative polling method that consolidates and synthesizes all the pre-existing tracking polls.

That’s why the last election cycle, as we saw in the Primaries, the Iowa Electronic Markets  totally missed Dean’s meltdown, and underestimated Kerry.  Further, on election day 2005, as soon as exit polls in Florida gave Kerry a lead, the futures traders did what I believe is now incontrovertible: they aped the polls!

As the data shows, by 5 pm, the Futures Markets were PREDICTING:

President.Kerry2004 trading at 67.0 by 67.9
President.GWBush2004 trading at 32.2 by 33.0

If that’s not prima facie proof that prediction traders merely track polls . . .

As Slate’s Dan Gross noted:

“The furious and seemingly irrational Election Day market action stands as evidence that the traders are more poll-followers than poll-beaters.”

Somehow, this intraday swing gets omitted from all of the charts mustered to show how prescient prediction markets are.

That hardly demonstrates collective wisdom or forecasting acumen. I will reiterate that what makes these markets so susceptible to failure: too thin, too few participants, illiquid, with too little real money actually at stake.

Even The Capital Markets Get it Wrong

“The Stock Market has predicted 11 of the past 4 recessions”

That old joke is quite telling: Despite the liquidity and massive
dollar amounts involved, even the Stock and Bond markets are quite
fallible as predictors. 

After the 2000 crash, the market had 5 substantial rallies
which many economists proclaimed as a sign of an imminent recovery.
They were wrong. Similarly, some people were insisting the first half
of the year’s weakness as a sign of an imminent recession (wrong again).

If these enormous, liquid, widely followed markets forecast wrong
(or were wrongly interpreted by the experts) — how accurate can we
expect the thinly traded futures markets to be?\

Less than Honorable Criticism
One final note: I am in favor of accountability. I have said repeatedly that forecasting is folly; When I do make forecasts, my working assumption is that they will be wrong

As a strategist, I like to wargame scenarios, taking them into whatever dark crevices they may lead. This leads me to, on occasion, I often stake out positions that are not mainstream. Offbeat, but intellectually honest.

Which brings us back to “fair & honest" criticism: Last year, A discussion I had in Business Week last year before the elections is now being used to (lets call it) inelegantly critique my criticism of prediction markets.

After a long discussion about some of the historical quantitative data that typically corresponded to an incumbent defeat, I was asked who I thought would win:

“What makes the 2004 election such a challenge to forecast is that we have never seen a Presidential term with a burst market bubble, a recession, a major terrorist attack on U.S. soil, a big tax cut, and not one, but two, wars. So without an analogous comparable, making a prediction with a high degree of confidence becomes quite problematic — it’s just a crapshoot.”

That’s the context of my “prediction” — I called it a crapshoot. Hardly pounding the table, certainly not the same as putting on a 10 million dollars position (see the whole thing for yourself here)

Yet somehow, this article was called the basis for my “humbling” and was mustered as an example of the superiority of prediction markets.

As criticism, I find it wanting.

Here’s my criticism:  I make this challenge to Chris Masse & company:

I believe the political futures traders don’t lead, but merely ape the polls. If that’s not the case, please explain (in a 1000 words or less) why once the exit polls were released as of 5 pm on election day 2004, Kerry futures shot up to 67 while G.W. Bush were trading at a mere 32.2.

I’ll pick the best one three and post them on The Big Picture.

Print Friendly, PDF & Email

What's been said:

Discussions found on the web:
  1. The Nattering Naybob commented on Jul 25


    Forget all that stuff, I will give you 3 words, a product name, a name and an acronym.

    Poultry, Eggs, Vaccines, Tamiflu (oseltamivir and zanamivir), Donald Rumsfeld and H5N1.

    You can take them all to the bank. If you want the latest, please come visit.

  2. Chris. F. Masse commented on Jul 25

    Hello Barry Ritholtz,

    Congrats for this long, detailled and combative piece.

    I have of course another interpretation of the facts. And I’ll take your challenge.

    In some time, I’ll have my response ready.

    In the meantime, I wish to you and to your readers a great summer holidays.

    Best regards,

    Chris. F. Masse

  3. RW commented on Jul 25

    Excellent piece Barry. Even when both the quantity and quality of information is good, forecasting the behavior of complex social systems is a nontrivial pursuit. In the case of prediction markets and other attempts to graft market-based technologies upon the deliberative process the question has to be asked: When is a market not a market? When there is inadequate information, no stakes on the table, no actual transaction, and no feedback loop.

  4. royce commented on Jul 25

    Have proponents of the predictive markets claimed total accuracy? It seems to me that you can make the case that the most efficient markets provide a more consistently reliable predictive capability than any given individual. That’s no small thing.

  5. common sense commented on Jul 25

    I think that there is no question that markets can provide better allocation of resources and even predict. The Delpjhi method was a play on this though it used experts.

    However the worship of markets is not related to the practical structuring of society. It is worship. There is a quasi religious cult associated with this. It is very powerful in the right which believes that it is possible to manufacture reality through thought. Thus if we wish well and the media gives good news we will have success in Iraq and the economy. It is no coincidence that many who believe this quite literally believe that other opinions are treason and thus not protected by the Constitution.

    So while you are discussing the powers and limits of markets and dealing them with as a tool like calculus (which can measure some things, be used predict some things, be used to help build some things and which can be improperly applied to create messes as well as incorrectly calculated) they are dealing with markets as a religious icon. It is like arguing about the power of some god/dess or force with a member of a “primitive” society. Entirely different mental frameworks are being used.

  6. Mapping Strategy commented on Jul 30

    Prediction or Information Aggregation?

    Catching up after one of the busiest weeks in recent memory… I’ve wanted to post regarding Barry Ritholtz’ most recent sage comments on prediction markets but haven’t had enough cycles until now… Ah, Saturday mornings in the summer… cool breeze o…

  7. Mapping Strategy commented on Jul 30

    Prediction or Information Aggregation?

    Catching up after one of the busiest weeks in recent memory… I’ve wanted to post regarding Barry Ritholtz’ most recent sage comments on prediction markets but haven’t had enough cycles until now… Ah, Saturday mornings in the summer… cool breeze o…

  8. Mabelle commented on May 8

    People Refinance because of Changing Reasons

    A large amount of mortgage refinancers have tapped their home equity in exchange of cash in the first quarter of this year. An analysis that was released yesterday said that this quarter has the lasgest proportion compared to any other quarter over the past 15 years.According to the latest quarterly review of loans owned by Freddie Mac, 88% of the people who refinanced their houses applied for loans that are 5% higher than their original balances. On the other hand, more than 50% took out loans that have higher interest rates than what they previously paid.

    Refinancing activity increases as interest rates increase. People refinance to get out og adjustable-rate loans from second or third mortgages and home equity loans. This allows them to switch to fixed-rate loans.

    Freddie Mac reported that homeowners have pulled $59.6 billion in equity out of their homes in the first three months. This is lower than the $70.9 billion that was pulled out in 2005’s last quarter.

    By Mabelle Sese
    Miami Real Estate Florida

Posted Under