"The U.S. Commerce Department said Friday that new home sales rose 2.8% in July after falling 4% in June."
>
That was how most of the MSM covered Friday’s New Home Sales.
The problem is, it is not correct.
First, let’s start with the actual data release, via Commerce:
Sales of new one-family houses in July 2007 were at a
seasonally adjusted annual rate of 870,000, according to estimates
released jointly today by the U.S. Census Bureau and the Department of
Housing and Urban Development.This is 2.8 percent (±12.0%)* above the revised June rate of 846,000
and is 10.2 percent (±12.3%)* below the July 2006 estimate of 969,000.
That seems pretty straight forward — except the way it was reported ignored the statistical reality.
Commerce noted what the margin of error and statistical significance was. They included this small caveat about the actual data:
Estimated average relative standard errors of the preliminary data are shown in the tables. Whenever a statement such as “2.5 percent (±3.2%) above” appears in the text, this indicates the range (-0.7 to +5.7 percent) in which the actual percent change is likely to have occurred. All ranges given for percent changes are 90-percent confidence intervals and account only for sampling variability. If a range does not contain zero, the change is statistically significant. If it does contain zero, the change is not statistically significant; that is, it is uncertain whether there was an increase or decrease.
So the correct answer to the question "What were New Home Sales in July 2007" is as follows:
There was no statistically significant change from June to July. According to the Department of Commerce, the range was -9.2% to +14.8%.
There was no statistically significant change on a year-over-year basis, either. Commerce reported a range from -22.5% to +2.1%.
This is not how it gets reported.
I am not sure if it is a case of innumeracy or of the media wouldn’t have a story about New Home Sales otherwise.
As the Commerce Department itself reported in the footnotes, Friday’s New Home Sales were statistically meaningless.
Even the nearby chart has the illusion of precision
Existing Home sales were out today, and may come in for the same treatment later this weekend.
Note: This is before we even factor in the cancellation factor after the jump:
>
UPDATE August 27, 2007 2:21pm
I see that Northern Trust’s Paul Kasriel comments:
Gain in New Home Sales Is Inconsistent with Reports from Home Builders
Today’s report that suggests a recovery in sales of new homes is not anywhere close. At the same time, the increase in sales and price are suspect because the financial press has a number of stories everyday about home builders reporting significant declines in sales and earnings, a plethora of incentives to move sales, cancellations of contracts, and so on. Cancellations of contracts to purchase homes are not reflected in this report. It is reasonable to assume that excluding cancellations leads to overestimating sales of new homes and underestimating inventories of unsold homes. Also, the home builders (see chart 4) survey for August showed the second lowest reading in the history of series. We need to see reports of future months and watch out for revisions of estimates of home sales.
>
Sources:
NEW RESIDENTIAL SALES IN JULY 2007 (PDF)
AUGUST 24, 2007 AT 10:00 A.M. EDT
http://www.census.gov/const/newressales.pdf
How does the Census Bureau handle cancelled sales contracts?
http://www.census.gov/const/www/salescancellations.html
Gain in New Home Sales Is Inconsistent with Reports from Home Builders
Northern Trust Global Economic Research
August 24, 2007
http://tinyurl.com/2rnbcb
Note: This is before we even factor in the cancellation factor:
How does the Census Bureau handle cancelled sales contracts in the published estimates of New Home Sales?
The Census Bureau does not make adjustments to the new home sales figures to account for cancellations of sales contracts.
The Survey of Construction (SOC) is the instrument used to collect
all data on housing starts, completions, and sales. This survey usually
begins by sampling a building permit authorization, which is then
tracked to find out when the housing unit starts, completes, and sells.
When the owner or builder of a housing unit authorized by a permit is
interviewed, one of the questions asked is whether the house is being
built for sale. If it is, we then ask if the house has been sold
(contract signed or earnest money exchanged). If the respondent reports
that the unit has been sold, the survey does not follow up in
subsequent months to find out if it is still sold or if the sale was
cancelled. The house is removed from the "for sale" inventory and
counted as sold for that month. If the house it is not yet started or
under construction, it will be followed up until completion and then it
will be dropped from the survey. Since we discontinue asking about the
sale of the house after we collect a sale date, we never know if the
sales contract is cancelled or if the house is ever resold.Therefore, the eventual purchase by a subsequent buyer is not
counted in the survey; the same housing unit cannot be sold twice.As a result of our methodology, if conditions are worsening in the marketplace and cancellations are high, sales would be temporarily overestimated.
When conditions improve and these cancelled sales materialize as actual
sales, our sales would then be underestimated since we did not allow
the cases with cancelled sales to re-enter the survey. In the long run,
cancellations do not cause the survey to overestimate or underestimate
sales. (emphasis added)
To be fair, Barry, the vast majority of the 90% confidence interval is in positive territory here (assuming we’ve sampled properly in the first place, which is another story…), so I can see why most people would be calling this a gain.
Perhaps more interesting is how it’s called a gain when it’s simply better than the month before by a tiny margin, but staggeringly worse than the previous year? This aspect was glossed over in all of the coverage which I was unprivileged enough to see. There is a degree of logical inconsistency or outright dishonesty at work.
I think the real story is that, ultimately, until the evidence comes in as to how the unwind progresses through the next few layers of mortgages issued in the entire 04 – 06 window, and any regulatory / banking / governmental action is decided, it will be hard to call this one way or another.
But I guess “Housing: Who the knows?” doesn’t make for as good of a headline to attract advertisers.
They have to print something and even though you are right…Joe Sixpack could care less.
Reporters often suffer from double penetration from both their bosses and their audience. Both seem to prefer the rosier scenario at all costs.
This is basic stuff:
If a range does not contain zero, the change is statistically significant. If it does contain zero, the change is not statistically significant; that is, it is uncertain whether there was an increase or decrease.
silly rabbit…..like it made one bit of difference to the chinese, I mean market…
Ciao
MS
Hi Barry,
no where else to really post this, but I wanted to say you are having a very nice interview on Bloomberg Radio. I wish CNBC could give you that type of respect. You are able to explain your view points very nicely without all the Kudlow RAH! RAH! RAH! that normally interupts you.
Cheers!
Just take a look at the not seasonally adjusted figures:
2006: July 83,000 homes sold
2007: June 77,000 homes sold
2007: July 74,000 homes sold (a decrease both M/M and Y/Y.)
I have a hard time believing the seasonal adjustments they do.
I am in the market to buy a house but am taking my time waiting for an opportunity. I regularly walk through the neighborhood and take note of the for sale signs. About a month ago on a Monday morning I noticed that three houses that had been on the market suddenly sold (one of them came back on the market). Considering there had been very few sales recently this surprised me. I think what happened is that people started to realize that the mortgage window was closing and they rushed out and bought a house.
Now that mortgages have become a lot harder to get the coming months sales reports should be very interesting. I know that the builders are getting increasingly desperate. Lennar sent me an email offering a mortgage with a 4.5% rate for the first year and a 5.5% rate for the second year. Aren’t teaser rates how we got into this problem in the first place?
“If a range does not contain zero, the change is statistically significant. If it does contain zero, the change is not statistically significant; that is, it is uncertain whether there was an increase or decrease.”
Barry, I’m not disagreeing. I’m just saying I can see why people without a background would be easily confused by this report. I agree that, when you look at all of the data, it’s either inconclusive (month to month) or, in fact, depressing (year to year).
I’m merely pointing out why the perception is the way it is, which ultimately is what can drive the disconnect here. What people believe and what is actually happening are not necessarily in alignment. I sense opportunity.
This is a great piece of analysis how media can shape the facts without taking the statistical meanings into consideration.
Is the real question how to interpret the US Commerce Dep. data accurately or how to guess the media’s interpretation and market’s response to it ?
I can see why something like margin of error got left out. For people to understand it, they would have to understand the theory of random sampling. What news article is going to have space for that? Saying that an increase is “not statistically significant” or even something more concrete like “we cannot rule out chance sampling error as the reason for the apparent 2.8% change” would only confuse most readers. Maybe online news stories could link to a statistical reference so readers can educate themselves?
the statistical opaqeness is just exactly what the commerce dept. wants.
They report….you decide……
And if you believe that….
Ciao
MS
WSJ article link regarding temp worker employment numbers. Not very encouraging.
http://online.wsj.com/article/SB118817263054609344.html?mod=hpp_us_whats_news
The problem Mr. Flibble is that the generalpopulation is completely ignorant of sampling theory and that includes a great many college grads. What is also not covered is when sampling there are at times errors of commission which goes right over peoples heads, especially in electoral polls when you get that oddball survey.
One of the most interesting things to me is that in the news media we have a “credit crunch”, and this in turn is how average folks have heard the news. As if it’s only a temporary problem that makes normal credit hard to get. It’s not yet understood in the broad public we had a credit bubble, and speculative excess….and what happens after a bubble.
It’s not reporters, or advertisers, or Joe SixPack or some vast MSM conspiracy that demands specific numbers where none are available: It’s the markets that demand them.
These numbers are reported first and foremost by the financial news wires: Bloomberg, Reuters, Dow Jones, Market News Service, and yes even MarketWatch. Their audience? The trading desk, which needs a hard number so that it can instantly judge whether the news is bullish or bearish. Nuance is not needed. I’ve beaten my head against the wall for 13 years about this. My head is flat.
The problem comes when usually boring reports like new-home sales suddenly reach the front page. The analysis that was fine for the trading desk is not useful for a mainstream audience, but story doesn’t get rewritten for a mass audience.
For what it’s worth, I’ve included these caveats in every story I’ve written on new-home sales for years:
“The government cautions that its housing data are subject to large sampling and other statistical errors. Large revisions are common. The standard error of 12% is so high, in fact, that the government cannot be sure in most months whether sales rose or fell.
“It can take up to five months for a trend in sales to emerge. New-home sales have averaged 867,000 per month over the past five months, compared with 861,000 in the five months ending in June. The five-month sales average is now down 19% from last July’s 1.07 million pace.”
The issue of sampling error is especially important in the housing data, but it also has an impact on the employment numbers, GDP figures, and almost every other report you see. The nonfarm payrolls numbers, for instance, have a confidence interval of about 100,000. And yet the market reacts violently when the reported number is off by 25,000 from the expected number.
Remember people: These numbers are just educated estimates!!! They are not gospel (not that the gospel is any more reliable than the NAR….)
If you assume the data are normally distributed, the probability of an increase from June to July is 65% based on the data. The probability of a decrease from last July to this July is 91%. This assumes they are giving the 90% confidence interval, and not the 90% confidence of not including zero (in other words, assuming the 90% is a two-sided test rather than one-sided).
So there is a measurable difference. It just isn’t statistically significant at the chosen level of confidence. Choosing a lower level of confidence (80%) would make the yearly change statistically significant. A much lower level of confidence (30%) makes the monthly change statistically significant.
i sense a big rally coming. 60 minute chart on the spx and the bkx are getting in tune and have reached oversold levels. if bkx breaks out or if gs breaks 180.5 watch out for the squeeze.
jkw gets it right. The standard assumption in these cases is a normal distribution and even if the range includes zero the probability that the change is positive or negative depends on the actual number. It’s significantly more likely that the y-o-y number is actually negative than it is that the m-o-m number is actually positive.
“i sense a big rally coming. 60 minute chart on the spx and the bkx are getting in tune and have reached oversold levels. if bkx breaks out or if gs breaks 180.5 watch out for the squeeze.”
– – – – –
I had already got my afternoon coffee as well in anticipation of sitting back and watching the 3pm buy program action. SPY volumes are pretty low today in comparison though so might not be much action as some of the previous days.
lower volumes are easier to move with program trading. we’ll see if the gods can defy gravity again.
Best comment by halbhh
…. what happens after a bubble?
If that were to be talked about more I think the average Joe would be scaling back their home buying.
Barry
Seems to me that there is another very misleading aspect of the numbers. It has to do with the inconsistency of reporting the changes to the previous month. When it is revised upward, it makes the report. If it is revised downward, no news. Even more importantly is the distortion to the current number. Since we always talk about month to month changes. If the previous month gets revised downward, that will make the current month look better. To be a true comparison one would have to compare last months unrevised number to the current month.
so much for a rally. bkx actually broke its triangle to the downside! here comes the next leg down.
Calculated Risk has an interesting analysis about positive bias in the Census Bureau’s reports during a market downturn:
http://calculatedrisk.blogspot.com/2007/08/new-home-sales-revisions.html
Unless I am forgetting something, the other thing to keep in mind when interpreting this (frequentist) 90% confidence interval (CI) is the sample size. That is to say, for a given alpha level (in this case, 0.1) and a certain effect size (in this case, 870,000), there is some sample size above which the resultant CI will no longer include 0.
A nice graph showing both the point estimates and the CIs for the past few years would be more informative (IMHO), but I know far too many Ph.D. academics who either don’t do this, or do a half-assed job of it, and so it is difficult to fault the press/MSM as well. (For those of you with access to a good university library, see also Reese, A., 2007, “Bah! Bar charts”, Significance, Vol. 4, Iss. 1, pp. 41-44.)
for what it is worth, 3+-12 % is by all measures and common sense = 0+-whatever %, or “we really have no idea what happened”. no scientist would give it more than a glancing thought.
especially since +-12% at 90% confidence implies (roughly) +-6% at 65% confidence, or in other words, a sample size of 250 (??!!) who the heck is going to trust a sample that small over the huge population it is supposed to represent ??
and remember that systematic uncertainties of the sampling method are NOT included.(hence the “revisions” afterwards)
those monthly numbers are USELESS. wait for (+-5% accuracy at 90% confidence) numbers half-yearly to get something reasonably useful.
But DO NOT TRY TO EXPLAIN THIS TO THE MEDIA (and many economists). do yourselves a favor, and fuggedaboudit : it is no use tilting against the “statistics ignorance” windmill hovering over society. Can’t be done. Believe me, I have tried, hard, for years. (in this blog as well). Painful. Save yourselves, it is too late for me.
I reviewed the same report and came to the same conclusion. The media goes for the simple headline.
Barry,
July New Home Sales were Jason Giambi like… juiced.
Still down 10% YOY and subject to downward revision as contracts cancel.
August New & Existing Home Sales (released in Sept) will be Jose Canseco like…
yielding a false positive for a housing recovery.
The head fake will even fool the Fed. Why?
Anyone locked in before Jumbo money went from 6% to 8% in early August, panicked and pulled the trigger.
September numbers (due in Oct) or post 8% numbers will be a disaster. As the % drop from the juiced August numbers will be significant.
Add in disappointing economic news and Q3 reporting misses, and solvency issues, Oct 31st, the Fed lowers.
The Nattering Naybob