At the risk of starting up another controversy, I must point out that the InTrade prediction market got this about as wrong as you possibly could:
>
Jackson-Guilty of AT LEAST 1 of 2 counts of administering an intoxicating agent to a minor
click for larger chart
Source: InTrade
On average, the Alcohol issue was an over 50% likelihood for nearly the entire length of the contract, but briefly dropped towards 20% for a few weeks last month.
>
Jackson – Guilty of AT LEAST 1 of 7 counts alleging lewd or lascivious acts against a minor
click for larger chart
Source: InTrade
>
The Lewd issue started out at 40%, spend almost a year near 60%, bottomed at 35% before climbing back towards 60%.
Okay, Chris — before I say anything else, I will wait for your deconstruction of this — where and how did it go wrong, and why?
>>
UPDATE: June 15, 2005 6:26am
Not only did the crowds get it wrong, but as the brilliant Daily Show reveals, the experts did just as poorly. (warning: hysterical video)
>>
Hello Barry Ritholtz,
There are two ways to assess the predictive power of the prediction markets. Number one, you judge their predictions against the outcomes of the events in question. Number two, you compare the prediction markets’ accuracy against other institutions’ performance.
#1. As you demonstrated, the two MJ futures markets got it wrong. They got an F grade.
#2. Who were the legal experts who got it spot on? Which poll foretold us of MJ being exonerated? Delphi method anyone? Think tank? Focus group? Group of experts? A comparative analysis would bring us light on this.
Now that I have declared defeat on point #1 (point #2 is up in the air, but I suspect Barry won’t be forthcoming with any competitive institution that got it spot on), I should mitigate with two arguments.
A/ Scholars are rather interested in analyzing long series. They would love to have a set of 1,000 court futures markets. They would draw conclusion only out of long series. That’s this kind of study that IEM put forward to prove that prediction markets are more predictive than polls — OVERALL. (16 years of experience and counting.)
B/ Not all prediction markets are created equal. Some events are more difficult to predict than others. Events whose outcomes are determined in a conclave by sequestrated jury members are among the hardest to divine. “A market can’t aggregate information unless there is some way for traders to get access to it.” (Quote from Matt Einbinder, PhD student from the University of Virginia, in an e-mail to me.) On that one, the burden is on us to refine the kind of bets that exchanges float.
So, Barry Ritholtz, you will certainly celebrate your small victory with a cup of champagne tonight —a genuine French champagne, of course. Here’s a French proverb for you:
“Une hirondelle ne fait pas le printemps.”
“Spotting one single swallow does not prove that the spring is here yet.”
The trick is: think long series!
One reminder: the Martha Stewart futures market was predictive!
One admission: the papacy futures markets were not bright either. One hit; two misses.
Micro-prediction: the 2012-Olympics futures markets will come up as a great victory. Paris, France, is in —big time. On July 6th, I will drink champagne till I can’t stand on my two legs. Hope you will share joy with my people and with the prediction markets crowd.
Your minute is up!
Best regards to you and to all your readers,
Chris. F. Masse
PS: To be fair with everyone, the main prediction exchanges are:
– TradeSports.com (= InTrade.com)
– BetFair.com
– Betdaq.com
– SpreadFair.com
Excellent and informative information, Chris —
thanks for the clarification!
It couldn’t have been “investors” betting on emotion could it?
One scholar, Paul C. Tetlock, has shown that TradeSports/InTrade traders are rational when it comes to operating on financial futures markets, and a bit less when it comes to operating on sports futures markets.
http://www.mccombs.utexas.edu/faculty/Paul.Tetlock/papers/Tetlock-Efficient_Info_Markets-01_02.pdf
No idea what’s up for celebrity futures markets.
To come back on Michael Jackson, I would love to have recorded what the legal pundits were prognosticating on cable TVs. That comparison would be interesting.
Also, please note, and I’m playing advocate here of course, please note that it would have been better to have the 9 individual counts separately, i.e. 9 separate MJ futures markets.
Best regards,
Chris. F. Masse
When do markets work?
Barry Ritholtz is celebrating Michael Jackson’s acquittal more than most, because it corroborates his view that prediction markets can be wrong; Intrade bettors were backing a guilty verdict on the charge of supplying alcohol to minors.However, Barry i…
One reason the markets got it wrong could be the lack of useful information. For instance:
1) Journalists covering the trial only reported the highlights based on what they thought was important, and that testimony wasn’t necessarily important to the 12 decision makers that created the result, the jury. The alternative was to read the daily transcripts, but getting those probably would have required not-insignificant transaction costs, discouraging their use by market participants (people who presumably weren’t investing thousands and thousands in each transaction).
2) We don’t know that much about the jurors themselves, or how they would interact to make decisions in a group setting. Without knowledge of the process involved with this specific jury, the market was left with generalizations about what an ‘average’ jury will do.
3) The market participants didn’t get to see the witnesses on the stand and so weren’t able to get any sense of the witnesses’ credibility. That’s the really important metric in any trial, and it was filtered through the lens of a series of talking heads on TV.
In short, all available information was not being included in the market, which would have reduced its efficiency, no?
Royce:
– spot-on analysis!
Haven’t we been warned that there are rules for determining which issues are well handled in betting markets and which aren’t (Royce’s point, more or less). Did we have reason to expect before the fact that betting markets would get the Jackson outcome right?
One possible answer to Mr. Masse’s question about othere predictive institutions which made similar guesses about the outcome of the Jackson trial, against which to measure betting markets – the DA’s office ought to be expert in assessing the quality of evidence and the odds of winning. The DA’s office had the advantage of being able to mashal the evidence to support SPECIFIC charges, and the ability to bring several related charges, in an effort to improve their odds. The “experts” in the DA’s office failed, and failed abjectly.
what prediction markets do best is predict the conventional wisdom (not necessarily the same as what will be in actuality) say the outcome will be. If you turned into MSNBC’s Abrams Report before the verdict you’d see that most of the legal analysts were leaning more toward a conviction of some sort than a preiction of an outright acquital.
PM Common Sense, Part II
As I’ve noted before, prediction markets are not a panacea. They do some things much better than any existing alternatives. In other cases they simply aggregate stupidity more quickly and efficiently, as Big Picture points out with regards to the Micha…
PM Common Sense, Part II
As I’ve noted before, prediction markets are not a panacea. They do some things much better than any existing alternatives. In other cases they simply aggregate stupidity more quickly and efficiently, as Barry Ritholtz points out over at Big Picture wi…
As i posted in my own site:-
Where I do definitely depart from the path of ‘betting on everything’ was after reading the section “The Prediction Markets Chalk Another One Up” devoted to the betting market on the outcome of the trial of Michael Jackson. Now here was a man, brought before a jury on serious criminal charges, where if found guilty he may have served up to twenty-five years in jail, on charges which would disgust most people of what may be termed ‘normal’ behaviour patterns and morals! The fact that the man is a ‘Celebrity’ and has been known in the past to behave as a ‘kook’ (as I believe the term applies) is irrelevant, the man was in court on a long list of charges regarding abuse of children! So what do you bunch do? You start up a betting pool on whether he is found guilty, and if he is guilty, how long does he get to serve in prison? What sort of of people actually place MONEY on the outcome of a morals trial where the victim was a child? Are there no limits to your sacred “Freedom of Expression” rules? I have worked with Americans, have known and corresponded with many more, and I have to admit that this betting market, for me at least, is the most cynical, disgraceful, debased, disparaging, negative and unpleasant item I have had knowledge of in a very long time!”
Why Prediction Markets Fail
Over the years, I have been critical of prediction and futures markets. In particular, the specific ways certain parties misuse them (i.e., politics). However, I am a big believer that markets can generate valuable economic and investing data that can …