Paul Farrell (who’s been on an angry tear lately) states what we have all known for quite sometime: Economists make awful forecasters. Farrell calls it "the illusory world of economic illusionists."
We’ve discussed this issue several times in the past: 1) in general with The Folly of Forecasting; and B) specifically, with The Mystery of the Awful Economists.
William Sherden did research on the accuracy of leading forecasters over a few decades, and his findings were summarized in his book "The Fortune Sellers: The Big Business of Buying and Selling Predictions." And his conclusions are still accurate:
"They are timeless. The political influence on predictions is basic human nature. I see no way that economic forecasting can improve since it is trying to do the impossible."
Here are Sherden’s top 10 findings in The Fortune Sellers:
1. The forecasting skill of economists is on average about as good as guessing. In fact, predictions by the politically driven Council of Economic Advisors, Federal Reserve Board and Congressional Budget Office were often worse than guessing.
2. Economists cannot predict the turning points in the economy. Of 48 predictions made by economists, 46 missed the turning points.
3. Economic forecasting accuracy declines with longer lead times.
4. No economic forecasters consistently lead the pack in accuracy.
5. No economic ideology consistently produces superior forecasts.
6. No economic forecaster has consistently higher forecasting skills predicting any particular economic statistic.
7. Consensus forecasts do not improve accuracy (although the press loves them).
8. Psychological bias affects forecasters and their forecasts. Some economists are naturally optimistic and bullish, others are consistently pessimistic bears.
9. Increased sophistication provides no improvement in forecasting accuracy. Remember the Long-Term Capital Management hedge fund? Two brilliant Nobel Economists backed by Wall Street’s elite nearly sabotaged the world economy.
10. Finally, Sherden says there’s no evidence that economic forecasting has improved in recent decades. In fact, forecasting appears to be deteriorating as partisan politics, Wall Street gaming and unpredictable global events invent new illusions.
The one caveat I would add to this is I have found ECRI to be consistently useful as an economic analysts, along with ISI and NDR. They help discern objective reality, rather than make forecasts per se.
Good stuff, Paul.
>
Source:
America’s ‘Hollywood economy’
Paul B. Farrell, MarketWatch
Marketwatch, 7:23 PM ET Sep 18, 2006
http://tinyurl.com/qxc7a
I recently read Sheridan’s “The Fortune Sellers.” I think he overstates his case a little bit, but not nearly as much as most forecasters.
If, as you noted, “forecasters” spent more time on possibilities then on time sensitive predictions their input would be much more useful. Why on earth people think they can make time accurate predictions on systems with intelligent agents and feedback loops is beyond me.
Although your list covers many of Mr. Sheridan’s points, I would still recommend the book to anyone who bases money decisions on ANY type of forecasting.
He has a whole chapter where he shreds the practice of forecasting company earnings. By implication, it could be taken that companies that consistently hit their numbers are suspect with regards to ‘gaming’ the system.
Russell
“ECONOMISTS BELIEVE THE COMING DECADE WILL BE A GOLDEN ERA. Many economists, and the Japanese government as well, say the classic theory of business cycle no longer applies to Japan, which has minimized instability factors and learned to drive slowly but steadily when necessary.” – Japan Times, December 26, 1989
Barry, anyone who tries to time the market is a forecaster. Going into cash because you think there will be a deep recession next year is the result of forecasting. Saying, “I’m not forecasting, but I see a good chance of a slowdown in the economy next year” is forecasting. You work in a business based around forecasting. The only people who aren’t forecasting right now are the people who have set allocations among asset classes and don’t vary them.
Conversely, anyone buying a stock is making a forecast — that the stock price will go up.
But are you forecasting when you take an umbrella to work? I prefer to think of that as a probability play — we know when we see these types of clouds, and when the barometric pressure does that, rain is a higher probability event than normal.
So too with equities: My approach is of a risk/reward basis: How much risk am I taking for how much potential reward? Its less forecasting than managing probabilitities. If stocks and markets have certian conditions, sometines we see more risk for less reward.
The ironic thing is that much of my gains this year came on the long side. . .
The only fault I can find with Economists is that their predications assume that the market will act rationally.
Anyone who thinks that the CEA, the CBO, and Fed economists are all “politically driven” has not studied these institutions nor carefully compared their forecasts.
The CEA is controlled by one party — indeed one person.
The CBO is consistently under bipartisan control (even when both houses are held by the same party) with a record of picking directors who resist political pressure. The Fed is an institution that is not under partisan control.
It was probably just a careless line, but the author is not an expert in any of the fields studied. That makes it wise to stick to the facts rather than making causal inferences.
~~~
BR: I agree about the carelessness — more int he grammatical structure —
I read that as the politically driven CEA, but did not carry it over to the CBO or the Fed.
The ironic thing is that much of my gains this year came on the long side. . .
Shorting just didn’t work so far this year… I wonder how many shorts will have to throw in the proverbial towel before it starts working again.
Small Investor Chronicles
Has anyone bothered to look at the performance of an economist of the Austrian persuasion? Just curious.
There are no Austrian economists, only people uncredentialed in economics who call themselves Austrians.
“Increased sophistication provides no improvement in forecasting accuracy. Remember the Long-Term Capital Management hedge fund? Two brilliant Nobel Economists backed by Wall Street’s elite nearly sabotaged the world economy.”
Bringing up LTCM in the context of forecasting economic numbers is somewhat odd. Does the guy really know what he is talking about? My first guess would be that those who are able to beat the market consensus (Not impossible if you don’t have to bet every month, only when you have a strong signal) don’t particuarly want to publish their sophisticated methods in the first place.
Regarding Self professed Austrians/Austrian leaning commentators: Not that the ABCT is wrong, but those who gravitate towards it tend to both be perma-bears and afraid of math so they’ll call 46 out of the last 2 turning points.
While it is good to realize there are charlatans out there, this book probably goes too far. Hopefully this book will trick more decently intelligent people into not trying to figure out specific parts of the world. The efficient market hypothesis and theories like this generate a lot of alpha for the rest of us.
Barry,
The best compliment I can pay you is that, in my opinion, you’re one of a few I pay attention to that I trust to tell me the truth.
That’s all… just the truth… I don’t have to like it, and I may not agree, but it seems to have truth as its core intention.
Hat’s off.
You may be right or wrong… you make make or lose a trillion (I hope you make a trillion), but I think you’ve got your head screwed on right.
Bringing up LTCM in the context of forecasting economic numbers is somewhat odd.
Why is placing bets on forecasts (I believe they shorted 10-year notes and went long on Ruble) is exactly the topic of the quality of the forecasts.
Economic forecasters produce faulty findings because their methods are faulty. Instead of working hands-on and observing the real trends on the street, they browse the endless lagging indicators produced either by the government or the industries. Any meaningful information gets averaged-out in the process and only smooth curves are left.
“Economics is the art of predicting the past”
–source unknown
“The one caveat I would add to this is I have found ECRI to be consistently useful as an economic analysts, along with ISI and NDR. They help discern objective reality, rather than make forecasts per se.”
What do these initials (ECRI, ISI and NDR) stand for?
A good review of Sherdan’s book. Bottom line is that the book has good points, but the man makes an incredible amount of unexplainable untruths.
http://www.findarticles.com/p/articles/mi_m1094/is_n4_v33/ai_21222068/pg_1
What’s the definition of an earnings surprise? If the analyst was right, he’d not be surprised. So an earnings surprise is really an analyst mistake! Imagine all the brokerage firm ads that would be pulled from CNBC if they told it like it really was. “GM came out with earnings today, and there were 40 analyst mistakes on that one stock alone. This stock has more sell ratings on it than any other Dow stock. It’s also up more than any other Dow stock as well.” Thank goodness for the delusions of those who follow analysts forecasts. It’s what gets a stock’s price down to an area where we can buy it, and up to a level where we can sell it.
OkieLawyer,
ECRI stands for Economic Cycle Research Institute (http://www.businesscycle.com/) (clickable in Barry’s post), ISI stands for International Strategy & Investment (http://www.isigrp.com/corp/index.jsp), and NDR stands for Ned Davis Research (http://www.neddavisresearch.com/public/container/index.jsp).
It’s not surprising to hear that economsits don’t make good forecasters in the traditional sense. Not only do they have to deal with an uncertain future, with all the revisions to data they have to treat uncertainty about the past. Furthermore, stats are only sometimes sparsely available. Consider that Ben Bernanke has decided to hide M3. Economists should be up in arms: They aren’t.
International acitivity adds another layer of complexity assuming it has any impact on domestic performance. International stats are often few and far in between, different from domestic stats in design and timed differently in terms of announcement dates.
How can economists possibly know what planet they’re on? Fortunately, they only have to try their best to be more accurate than the next guy. Don’t they?
Another approach to investing that obviates the so-called forecast is called trend following. By definition, trend followers don’t make forecasts. They simply seek to understand the current instance and invest accordingly. When the instance changes, so does the position.
History <------<< Instance <-----<< Future
Then again, there is an even larger industry of market forecasters, to whom the same criticisms could be extended with perfect ease. Limiting the critique to economists seems a bit small-minded when swimming in a sea of equally feckless forecasters.