Barron’s Mike Santoli looks at one of our favorite pet peeves: The use and abuse of data. His column this week, The Limits of History, notes that this has become especially prevalent of late:
"People who try to handicap the markets for a living practice the art of the plausible. Many trudge from conference room to lunch table to banquet hall lugging PowerPoint decks full of unobjectionable statistical touchstones for commission-wielding clients. At times of investor confusion and market dissonance, such as now, their art is often reduced to carving a slice out of economic history that ratifies their existing outlook."
Mike then proceeds to look at a "dog’s breakfast of the kinds of historical analogies making the rounds." What is especially amusing about his list is the number of "Every time X happens, Y has occurred" that collectively produce all manners of mutually exclusive results. Since "X" occurred we will definitely have/avoid a recession; Stocks are undervalued/overvalued; Markets must rally/fall.
What is an investor to do? Whenever you are confronted with an "incontrovertible proof" based on historical data, prior to taking any action, I suggest asking yourself this short list of questions:
• Do we have enough historical examples? Is the data sample statistically significant?
• Causation or Correlation? Does "X" cause "Y" to occur? Or, are we presented with two things that may have the same underlying causes? Is there even interaction between X & Y?
• Coincidence? How possible is it that these two items are utterly unrelated (i.e., proof-we-are-clueless Superbowl indicator).
• Look for differentiating elements in different time periods: What factors are similar? What factors are different?
• Compare interest rates, inflation, dividend yield, P/E contraction or expansion, sentiment, overall market trend, business cycles — across different eras. Might that account for potentially different outcomes?
• Any recent market environmental changes (regulation shift, financial innovation, etc.) have an impact? What might these specific changes do to the data? Consider Decimalization, ETFs, online trading, change in dividend tax, etc.
• Subjective versus Objective measures: Are the factors under discussion hard numerical data, squishy or somewhere in between? Percentage of stocks over 200 day moving average is objective; I find some chart pattern readings subjective. Earnings at time have been rather subjective; official inflation measures somewhere in the middle.
• Consider things in terms of probabilities, not outcomes: Assume a causative factor resulted in a specific event (X –> Y) 7 out of 9 times. The most you can say is that when "X" occurred in the past, it has resulted in "Y" approximately 78% of the time.
• There is a difference between historical occurrence and future likelihood. In the example above, this does not necessarily even mean that since "X" has just occurred, there is a 78% that "Y" will happen. Consider: was the first X/Y occurrence really a 100% or zero? Did the second one become 100% or 50%, then next a 66% or 33%?
• Contextualize data: Sometimes a single data point — even a mean or median — only tells half a story. Any data point can be trending or reversing. Going higher, lower, topping, bottoming. Each of these may have differing implications for what comes next. Inflation is high, but coming down. Gold is high — and going up. It helps to think of data not as a still photograph, but as a frame in an ongoing film.
I’m sure there are more — that short list is off the top of my head.
Any others? Please make your suggestions below. If we get enough good ones, I’ll try to massage this into a more formal column.
The Limits of History
Barron’s, MONDAY, JANUARY 28, 2008