Black Friday Forecasts Still Suck

Holiday Sales Forecasts Still Stink
Just say no to unreliable methodologies.
Bloomberg, November 22, 2017

It’s that time of year when we all gather to give thanks for our blessings, overindulge in various foodstuffs, and, of course, go shopping.

Shopmas is also the time of the year when I call out the National Retail Federation for their bogus retail forecasts; they return the favor by calling me a Grinch. Sales forecasts drawn from the NRF’s annual survey have been consistently bad — rarely even coming close to actual holiday retail sales numbers as reported by the Census Bureau — and the methodology is unreliable.

Our annual dance looks something like this:

1. An NRF survey of shoppers asks them what they plan to spend this holiday season and what they spent last year;

2. The net difference between those two numbers gets shared by the media as the gain (or loss) for the upcoming holiday shopping season;

3. Headline writers will get it totally wrong, as much of the media reports the surveys as if they were hard sales data;

4. The forecasts inevitably are proven wrong; and

5. Almost no one recalls this when the actually sales data comes out in January.

The NRF did make a minor adjustment to their approach this year, now forecasting a range for holiday sales gains. And so:

The National Retail Federation announced today that it expects holiday retail sales in November and December — excluding automobiles, gasoline and restaurants — to increase between 3.6 and 4 percent for a total of $678.75 billion to $682 billion, up from $655.8 billion last year.

Predicting a range instead of a single specific number, while not an improvement on the underlying methodology, does increase the likelihood that the NRF’s prediction will be correct. It is much easier to hit a range of almost half a percent than to nail an exact number.

Well played, NRF. You are a worthy opponent.

As it turns out, the way the NRF forecasts seasonal sales results is pretty awful. Relying on a survey of shoppers instead of using real sales data is deeply problematic.

Why? Well, there are quite a few reasons:

• People’s memories are surprisingly unreliable. The way to learn what you spent last year during the holidays is to look at your bank accounts and credit card statements. Guessing that number is simply untrustworthy. 1

• As individuals, we really do not know what we are going to do in the future. People are simply terrible at predicting their own behaviors. This has been repeatedly validated by academic literature. 2

• When we survey people, we learn how they are feeling at that very moment — and little else. Trying to forecast future behavior this way is an inherently problematic methodology. Incidentally, this also helps to explain why “things” tend to not make us all that much happier. People often expect that a shiny new house/car/boat/gadget/bauble will increase their total life satisfaction; instead, their predictions fail, because they eventually become used to it.

• The last way we know that this survey methodology is unreliable is to look at its track record. It has been uniformly awful.

Note it is not just this one retail trade group that relies on a flawed methodology, but others as well. Even Gallup, which should know better, does the same thing.

Why make a big deal about this? Well, facts matter, especially to investors. Relying on overly optimistic (or excessively pessimistic) forecasts spells trouble for people who put capital at risk.

We can make an educated guess as to what sales data for 2017 will look like, but not much more than that. Despite what you might have read, we barely have any idea of what holiday sales will be like, or what the Black Friday retail numbers will be.

Happy Shopmas, everybody. Grinch out.

___________

1. How good is your memory? It is neither precise nor accurate — a polite way of saying it is not all that good. Quick, what did you spend last year on holiday gifts? Unless you have an uncanny sense of recall, or you actually tallied up last year’s spending, “I don’t know” is the only accurate answer. My pet thesis is that you’ve been trained to respond with an answer — any answer — other than “I don’t know” or “I don’t remember.” Most of us suffer from the erroneous belief that admitting we don’t know the answer to a question is somehow a sign of ignorance or stupidity. So most people instead make up a ballpark number that sounds somewhat reasonable.

2. Not only is it not peer-reviewed by other statisticians, this year, the details of the methodology are not in the press release, and seem to not be publicly available on the NRF website.

Originally: Holiday Sales Forecasts Still Stink