Statistical Mistakes Traders Make

Doc Steenbarger has an interesting discussion about one the most
common mistakes traders make: They confuse descriptive statistics with inferential ones:

"Much of the information that traders work with is descriptive in nature. Consider these statements:

* The market trendline is up;
* Advancing stocks are ahead of declining stocks by 2:1;
* We are making new weekly highs in the market;
* Volatility is at a new monthly low;
* The market made a breakout from the trading range.

All of these are descriptive. They take a sample from a price or indicator series and describe characteristics of that sample.

The intermediate step between observation of descriptive stats and a hypothesis of inferential ones is backtesting:

"Descriptive statistics can lead us to the formulation of hypotheses, but they cannot provide tests of those hypotheses.
That is the role of inferential statistics. To test a hypothesis, we
must evaluate multiple samples and verify the existence of suspected

quantitative, system trader trades patterns that have been tested with
inferential statistics. The discretionary trader trades descriptive
hypotheses that he/she validates with updated, real-time readings of
market conditions. Is the discretionary trader justified in doing that?
The same inferential tests that inform us of the validity of trading
systems, when applied to the trader’s trading results, will answer that

There are numberous ways to backtest theories and trading hypotheses:  Trader DNA evaluate trader performance metrics; On a broader level, firms like Clarifi can backtest entire quantitative market strategies.

Regardless of the level of analysis you may be working with, this leads to an obvious but overlooked question:  Whta have you done to qualify or test your trading or investing strategy?

That is one of the reasons I respect pure value investors: It may be somewhat boring and require  immense patience, but it has been thoroughly vetted and tested as an investment methodology.



The Most Common Mistake Traders Make 
Brett Steenbarger    
TraderFeed, Thursday, September 14, 2006             

Print Friendly, PDF & Email

What's been said:

Discussions found on the web:
  1. Eclectic commented on Sep 17

    Zat so?…… well…..

    “Crest has been shown to be an effective decay preventative dentifrice, when used in a conscientiously applied program of oral hygiene and regular professional care.”

  2. blam commented on Sep 17

    Just read an interesting article by Ben Inker on J Grantham web site about P/E inflation.

    Since 1988, operating earnings have consistently grown to exceed GAAP net earnings, averaging 20 % since 2002. This is statistically impossible in an honest world. It gets worse. Forward operating earnings estimates average 13.5 % vs the realized 3.5 %.

    Abby Cohen has stated she thinks the market is 6 – 8 % undervalued and will be made up by Christmas with an additional increase of 12 % over the next 12 months.

    The Federal Reserve remains in bubble support mode. The record low risk premium indicate that risk is a quaint idea who’s time has passed.

    Although I think the market is +20% overvalued, it doesn’t matter. If the wall street bubble makers want to drive the market up 15 %, they will. I think the market may be the next bubble, replacing housing.

    It appears to be a dangerous time to be too bearish.

    My 2 centavos.

  3. flap commented on Sep 17

    when the weatherman says that there is a 90% chance of rain, he isn’t saying it is certain that there will be rain. what that 90% means, is that when barometric conditions have been the same as they are currently, that has resulted in rain about 90% of the time.
    you can either take your umbrella, or not.

  4. Bluzer commented on Sep 17

    There are good traders and not-so-good traders. And the difference between them is NOT the employment of descriptive vs inferential statistics. Because, even as you describe it, there really is no difference between them. A descriptive statistic has, implicit in it, backtesting. When I say I think the market will rise the next couple of weeks because the winners-to-losers ratio was 4-to-1 on above average volume this week I don’t have to back-test using gobs of data on a multi-processing mainframe.

    “The same inferential tests that inform us of the validity of trading systems, when applied to the trader’s trading results, will answer that question.”
    This seems to be saying that if the trade worked out it was a good inferential system. If not it wasn’t. Profound.

  5. Bluzer commented on Sep 17

    “What have you done to qualify or test your trading or investing strategy?”

    A more appropriate question would be “If you have a trading or investment strategy what have you done to qualify or test the same” And I would emphasize the former part over the latter.

  6. permabull commented on Sep 17

    Although very dated, Karl Popper’s writings are a very lucid expanation of why we ought to state our theories clearly and how we ought to go about testing them.

    Even better are his two more lucid and gripping books on how all this applies to us – The Open Society and its Enemies, and The Poverty of Historicism.

    The excellent summary that heads this post also leads to the problem Popper identified of inductive versus deductive reasoning. In this case, as it applies to investment.

    Popper’s preference for inductive reasoning is a preference most older investors have. It comes from closely examining individual companies and constructing a portfolio, one company at a time.

    At the end, we may make some general observations about the commonalities of the successful, and indeed unsuccessful, components of the portfolio.

    Yet it is only in the last 30 years that it has been thought prudent to bypass building a portfolio inductively; and, instead, to begin with a general portfolio view, and – from that – deduce the companies that will form the portfolio.

    Both methods will lead to the formation of hypotheses which must then be continually tested.

    However, older investors will think of modern deductive portfolio theory as an ultimately flawed attempt to simplify and neaten the usually tedious and ungainly task of finding good companies at fair value – wherever they may reside – and doing so, one company at a time.

    It is probably the case that almost every good equities investor spends most time answering one simple inductive question :-

    If I add or delete this company, will the portfolio’s future total returns be diminuished or increased?

    To ask the question is to guarantee long hours spent analysing company returns and very few hours spent analysing the broader economic environment.

    As there are not enough hours, at some point everyone will have to choose between the inductive and deductive paths.

  7. JGarcia commented on Sep 17


    I’d like to hear your read on Cycle work in this regard. We always hear/read alot of predictions by the cycle guys. They definately catch “some” moves. You don’t ever hear about all the ones they miss, or more importantly why they were wrong.

    Any personal thoughts on this Barry?



    You should be able to figure that out on your own through the voluminous commentary placed on this site. There is a robust Google search attached, and an extensive keywork tags system built in.

    Don’t give me more work, figure it out yourself!

  8. ari5000 commented on Sep 17

    Just wanted to add one thing about this rally as the markets near their yearly highs.

    Hedge funds run $1.2 trillion and they are “all in” at the moment.

    So the only way they win is if nobody sells — or better yet — guys like us start buying. Well — most of us are not buying. So these hedge funds are somehow going to have to sell these tech stocks to someone — most likely each other.

    It’s VERY easy to buy and show a gain.

    In this new world with 7000 hedge funds (+ mutual funds) all crowded together — it’s a very different thing to sell out at the current highs and book those gains.

    We shall certainly see what we shall see next week.

  9. Eclectic commented on Sep 17

    My focus is on P/E multiples. It seems to me they’re under a sustained period of compression.

    I don’t have to know why… I just have to know it’s happening.

    If you can tell me what the collective market will decide to award as far as broad market P/Es, then I can tell you what the market will do.

    That’s the key… for me… to attempt to think through what the psychological reasons for a range of P/Es are.

    I know they’ve seen 9-12s before, even as low as 6 or 7, and I know those appeared like phantoms out of nowhere.

    Is there anybody here who claims to have some reasonable basis for estimating P/Es?

  10. permabull commented on Sep 17


    The majority of equities hedge funds are both long and short a collection of stocks; usually using some form of derivative exposure as well.

    The net exposure of an equities hedge fund is often 1% to 10% of the gross exposure and the relevant risk management team will reduce the gross exposure as the net exposure rises.

    The returns to equities hedge fund managers come from correctly analysing variations of arbitrage, usually by country or sector.

    Given the foregoing, most equities hedge funds are unconcerned about the maket’s general direction; and very concerned about the internal arbitrage direction.


    Using the ROIC would probably be a better measure.

    However, it’s probably true that , over the years, the main influence on a comapany’s P/E – outside of the E itself – is inflation.

    Many, many years ago a simple measure of this was the Rule of 20, whereby the current headline inflation rate was added to the current P/E.

    When that number went over 20, the market was said to be overvalued; when it went under 16, the market was said to be undervalued.

    Outside of the contractionary earnings period during recessions, that old rule did a reasonable job of aligning relative value between periods of high – and low – inflation.

    Although simple, that old rule was almost the only one that allowed a like for like comparison of values between the high inflation 70’s/part of 80’s and the low inflation 90’s/00’s.

    It would have interesting if someone had done that exercise for ROIC – but they never did.

  11. S commented on Sep 17

    A private equity club deal led by Blackstone is taking Freescale private at a trailing multiple of 19x. Consensus ’07 earnings value it at 21x.

    Semiconductors are a notoriously cyclical business. And almost from the moment a new generation chip is introduced, it’s average selling price starts declining. Both factors can create big variance in cash flows.

    If you assume the private equity guys have IRR hurdle rates of around 30%-40%, it’s hard for me to see how the numbers work without expecting significant multiple expansion on the exit.

    Private equity is the new bubble asset class.

  12. PC commented on Sep 17

    You wrote:
    “What have you done to qualify or test your trading or investing strategy?”

    Backtesting is not of much use, in my humble opinion. There is just too much temptation to curve fit.

    I am a discretionary trader within a set of clearly defined rules. The best way to qualify my trading methodology is to trade it real time for extended periods. If it makes money, your method is good. If it doesn’t, keep fine tuning it until it does.

  13. Barry Ritholtz commented on Sep 17

    How can you discern when the conditions are shifting, and/or the cycle is turning — and not a specific problem with your methodology?

  14. JGarcia commented on Sep 17

    I think the bubble is in calling bubbles.

    N’uff already.

  15. diva commented on Sep 17

    Excellent comment J Garcia!

  16. jj commented on Oct 1

    Good lessons for all ::::

    #1 there is a huge gap between market analysis and trading markets to make money.

    #2 There is no relationship between being “right” and making money.

    #3 While markets are not predictable people are.

    #4 Anything can happen in the markets so how worthwhile is a market opinion?

    #5 Having a definable game plan and following it will overcome poor analysis.

    #6 I know some very rich traders but I have yet to meet a rich analyst.

    #7 You should never give out market advice because readers don’t need your bad advice and they will ignore your good advice so don’t give them any advice.

    #8 A correct market opinion does not answer the questions of how and when do I place a bet, when do I know I am wrong, how big is my bet in terms of dollar or percent risk, and most important how do I manage my trades when they are working.

    #9 Some very smart people think the stock market is going up. Some other very smart people think the stock market is going down. Since I don’t have a clue what the stock market is going to do I totally agree with both opinions.

    #10 Managing the money and more importantly managing the trade is more important than being “right”.

    #11 A good trade is a trade which was entered and exited following one’s rules regardless of the dollar outcome be it a gain or a loss.

    #12 Most newsletters offer both sides regarding market direction. Whichever way the market goes will then be highlighted in subsequent newsletters as if the writer new what was coming.

    #13 The more negative email you receive regarding a market opinion the more you should bet.

    #14 If you receive emails endorsing your view you might want to re-think your opinion.

    #15 You learn very little “watching” someone else trade and you might very well harm yourself as a trader by following the advice of others. Be your own man or in one case lady!

    #16 Keeping a trading diary on a daily basis will teach you how you think. Be honest and don’t edit your diary in hindsight. Again trading is not about right and wrong but it is about doing and not doing

  17. A Dash of Insight commented on May 7

    Mistakes Market Strategists Make

    Please consider some data. Readers should try as hard as possible to put aside their pre-conceived notions. Pretend you are looking at it for the first time. In many cases, readers probably will be doing exactly that! This is a

Posted Under