The Wyatt Earp Effect

Bob Seawright is the Chief Investment & Information Officer for Madison Avenue Securities, a broker-dealer and investment advisory firm headquartered in San Diego, California. Its from his blog, Above the Market. Enjoy.

~~~

I recently wrote about luck and skill in investing and argued that we tend to underestimate luck’s role pretty dramatically. I opened with the story of the 2006 TradingMarkets/Playboy 2006 Stock Picking Contest that was won by Playboy’s Miss May of 1998 (Deanna Brooks, shown above). She won out over many professionals with lots of experience and vast resources who spent pretty much all day, every day studying the markets. That result provided still further evidence for what we should already know – market success (however defined), especially over the relatively short run, is more a matter of luck than of skill.

Investment performance data supports this idea unequivocally. As Charley Ellis has pointed out, “research on the performance of institutional portfolios shows that after risk adjustment, 24% of funds fall significantly short of their chosen market benchmark and have negative alpha, 75% of funds roughly match the market and have zero alpha, and well under 1% achieve superior results after costs — a number not statistically significantly different from zero.” My conclusion was thus predictable and straightforward.

“As [Nate] Silver emphasizes in The Signal and the Noise, we readily overestimate the degree of predictability in complex systems [and t]he experts we see in the media are much too sure of themselves (I wrote about this problem in our industry from a slightly different angle…). Much of what we attribute to skill is actually luck. Invest accordingly.”

That led to the following (predictable) response after Jason Zweig of the Journal tweeted out my piece.

Let’s think about that for a bit.

It’s important to note at the outset that whenever tremendous investment streaks or returns are claimed, there is considerable reason (see here, for example) to doubt the factual basis for the claim. Sometimes the problem is mathematical, sometimes it’s a matter of a faulty memory, and sometimes people are simply dishonest. In the “professional” space, it is commonplace to see survivorship bias as well as phony measurement and benchmarking (for example, asset-weighted performance typically tells a very different story than more traditional performance measures).  That’s why, in a discussion of the likelihood of getting “heads” 100 coin-flips in a row (we should expect it to happen once in 79 million million million million million — that’s 79 with 30 zeros after it — fair sets of tosses), the probabilities favor a loaded coin. Barry dealt with this dishonesty among the pros earlier in the year, noting how common it is to hear from “Pinocchio traders.”

Secondly, we need to recall that, generally speaking, we suck at math and are even worse at probabilities.  We look for patterns to convince ourselves that we have found a “secret sauce” that justifies our apparent or would-be success when dumb luck is at least as likely to be at work. In this regard, we are dumber than rats – literally (as I have argued before).

In numerous studies (most prominently those by Edwards and Estes, as reported by Philip Tetlock in Expert Political Judgment), the stated task for observers was predicting which side of a “T-maze” held food for a subject rat.  Unbeknownst both to observers and the rat, the maze was rigged such that the food was randomly placed (no pattern), but 60 percent of the time on one side and 40 percent of the time on the other.

The rat quickly “gets it” and waits at the “60 percent side” every time and is thus correct 60 percent of the time.  Human observers, however, kept looking for patterns and chose sides in rough proportion to recent results.  As a consequence, the humans were right only 52 percent of the time. They (we!) are much dumber than rats.  Overall, we insist upon rejecting probabilistic strategies that accept the inevitability of randomness and error and upon rejecting the idea that randomness is a crucial component of our success (on account of self-serving bias, randomness is seen as only being behind our failures).

All of which leads to the question at hand. How many years of outperformance in a row would it take for me to be convinced that such investment success was a matter of skill?

Bill Miller of Legg Mason famously beat the S&P 500 for 15 straight years from 1991-2005. During that time, he was the poster boy of investment skill. Michael Mauboussin calculated the odds against that happening randomly as exceptionally long indeed. Miller himself was much less self-congratulatory. “As for the so-called streak, that’s an accident of the calendar. If the year ended on different months it wouldn’t be there and at some point the mathematics will hit us. We’ve been lucky. Well, maybe it’s not 100% luck — maybe 95% luck.” As Mauboussin points out, such streaks indicate skill, but luck is heavily involved. Indeed, in the five years after the streak ended, Miller lost 9 per cent annually and ranked dead last out of the 840 funds in the same category. He lost 55 percent in 2008. That said, he’s hot again now.

After the streak, maybe Miller lost his edge. Maybe he simply made a few foolish decisions (like doubling-down on financials at the wrong time). Maybe he had personal issues. Maybe he lost key staff. But maybe he was luckier during the streak than we’d like to think. I suspect it’s some combination of factors. It’s easy to falsify the Efficient Markets Hypothesis. But it is still really, really hard to beat the market. This whole question of investment skill is a vexing one, as even Warren Buffet (the best available argument for investment skill) acknowledges.

More to the point, the odds of such a streak happening are not actually all that long when the question is examined properly. As Leonard Mlodinow has explained, the proper question, given the number of mutual funds that have existed in the modern era, concerns the odds that any of them would have beaten the market over any 15-year period of time due to chance alone? That answer is an extremely surprising one: almost three out of four. The reason is what statisticians sometimes call the Wyatt Earp Effect. Earp is famous largely for one simple reason: he quite remarkably survived a lot of duels. We only calculate the odds in these highly improbable situations when we already know what happened and are surprised by it. Thus, in terms of predictive value, these instances don’t mean very much at all.

Football Perspective has a wonderful series about such questions, asking What Are the Odds of That? about a number of odd sequences. Here’s the key takeaway:

Either way, asking ‘what are the odds of that’ is not the right question…. It’s asking “how likely is it that we’ll see some crazy statistical outliers over the course of three games?” [like the roulette wheel at the Rio in Las Vegas that landed on 19 seven consecutive times]. The answer to that, as always, is, “very.”

There is a famous story, almost surely apocryphal, that makes the rounds with some regularity in our industry. As the story goes, a new, struggling advisor buys a mailing list of 10,000 potential prospects. He then writes two separate newsletters about the same, high variance stock. One newsletter advises a leveraged short of the stock and is sent to half of the list. The other suggests a leveraged buy and is sent to the other 5,000 prospects. A month later, the prospects that got the newsletter touting the wrong side of the trade are discarded. The 5,000 that were on the right side are split into two groups and the same process is repeated with a different stock. He does the same thing again two more times. At the end of four months, the advisor has a list of 625 prospects who can’t believe their luck in finding the next Michael Burry (or Seth Klarman or Warren Buffett). Of course, they will also spend the next ten years wondering why, after they transferred their life savings, the magic disappeared.

No number of consecutive market-beating years will convince me of a manager’s investment skill in and of itself. There are way too many potential false positives in the data for that. Long-term outperformance is indicative of skill, surely, but it isn’t dispositive, as Miller’s post-streak performance suggests. I’m interested in long-term performance, yes, but I’m more focused on a manager’s investment process (more here).

As I routinely emphasize, the best performers in all probabilistic fields dwell on process. This is true for great value investors, great poker players, and great baseball players. A great hitter focuses upon a good approach, his mechanics, being selective and hitting the ball hard. If he does that – maintains a good process – he will make outs sometimes (even when he hits the ball hard) but the hits will take care of themselves. Maintaining good process is really hard to do psychologically, emotionally, and organizationally. But it is absolutely imperative for investment success. And it’s more important than how many consecutive years anybody has beaten the market.