One of the more interesting issues I observed last week (albeit from a distance, margarita in hand) was the pre-NFP excitement generated by the ADP National Employment pre-release. I wrote before the NFP release that I had little confidence in the ADP report.
Boy, did I understate the case. There are many, many things that the private sector does better than governments or academia (probably most!); objectively compiling complex data and releasing it at no cost to users is apparently not one of them.
IMHO, the ADP report’s main utility is not to investors, but rather as a PR score brought to its parent co. Other than that, it turns out to be just another piece of self-promotional random PR garbage.
What I found surprisingly revealing was the
absurdist reaction to the ADP data prior to NFP by Wall Street. What in hell were those Economists thinking about? Why toss out a perfectly serviceable model, and raise Payroll expectations — merely based upon an ADP data point and its rather unimpressive track record?
Let’s be brutally honest here: ADP is a for-profit firm that processes
payrolls for corporate employers. From what I gather, they do that
very well. But as an econometric firm, they are in way over their
heads. (I am not familiar enough with Macroeconomic Advisers to opine on their complicity in all this). The data ADP has been releasing has served its purposes well:
Its a nice little piece of publicity that generates exposure and
cheap marketing for the firm. Indeed, if you go to the website that was set up for this report — ADP Employment Report.com — it looks much more like an advertisement:
>
>
To me, this looks like marketing — and not a serious bit of economic research.
But enough opinion. The Big Picture is about interpreting data, and for that we will go straight to the evidence: the actual data comparing BLS Non-Farm Payroll Reports with the ADP National Employment Report for 2006 (January through June):
>
| Employment Report | ADP (forecast) Private Sector Only |
BLS (actual) All NFP |
Correlation |
| January 2006 | 162k | 154k | 92.6% |
| February 2006 | 342k | 200k | 58.4% |
| March 2006 | 133k | 175k | 131.5% |
| April 2006 | 178k | 126k | 70.8% |
| May 2006 | 121k | 75k | 62.0% |
| June 2006 | 368k | 121k | 32.9% |
>
Note that the BLS measures both private and government job creation, while ADP only measures private employers. That means the overestimates by ADP are off by an even bigger factor, while the underestimates are more accurate than they appear. The most recent miss is particularly huge — 25% of June’s new job creation was governmental. So the actual number of private sector BLS jobs was 90,000 versus the forecast of 368,000. That a miss of about 300%.
This is somewhat of an Apples and Oranges comparison, as the ADP measure does not include government (Federal, State and Local) employees. But since ADP is touting this as a forcast of the BLS data, it is incumbent upon them to make the appropriate adjustments — not the reader.
Several other factors are likely contributors to the ADP forecasting errors:
• BLS sample is considerably larger, covering ~400 thousand establishments with total employment of ~45 million; ADP NER covers roughly a bit more than half as many establishments (226k) with about a thirds as many workers — 14 million.
• BLS employs a Birth/death adjustment; ADP does npot attempt to measure new business creation, using their customer data. Ironically, this may make ADP even m roe inaccurate, as the Birth/Death adjustment has shown a strong upwards bias;
Bloomberg noted the gaffe in article Friday. Here’s the most amusing line from their report on the ADP miss:
"Joel Prakken, chairman of the St. Louis-based forecasting firm Macroeconomic Advisers LLC, which produces the report jointly with ADP, couldn’t immediately be reached for comment."
One of the committments I made to myself when I started speaking in public was to frequently say "I was wrong." I have spit out mea culpas on a regular basis. As soon as I become "unavailable for comment" after being way wrong on something is when you should immeidately stop reading this blog.
Bottomline: As far as the ADP report being a legitimate data source, they are no better than other self promotional private reports announcements, like the Monster Jobs report or just about anything out of the Conference Board. All are transparent pieces of PR puffery. None has a methodology that is time tested or particularly valuable. Each methodology varies so much from the BLS process that it has almost zero value as a predictor of the actual NFP report.
The one exception seems to be the National Association of Realtors; At least they release their data, which seems to have some general value; of course, you
can ignore most of what comes out of their chief economist for the same
reasons.
>
>
Sources:
ADP National Employment Report FAQ
http://www.adpemploymentreport.com/ner_faq.aspx
ADP Job Survey Loses Luster Among Economists After June Miss
Joe Richter
Bloomberg, July 7, 2006 15:09 EDT
http://tinyurl.com/oaamn
Bureau of Labor Statistics, (PDF)
BLS report, March 5, 2004
http://www.bls.gov/cps/ces_cps_trends.pdf
“One of the committments I made to myself when I started speaking in public was to frequently say “I was wrong.” I have spit out mea culpas on a regular basis. As soon as I become “unavailable for comment” after being way wrong on something is when you should immeidately stop reading this blog.”
Thank you.
Macroeconomic Advisers is definitely a firm that knows its econometrics. It was started by Larry Meyer, former Fed Governor and long-time Wash. U. – St. Louis professor. It obviously makes a lot of mistakes when forecasting, but it does large-scale econometric modeling as well as anyone out there. Maybe they sold out on this one, though.
Here is a link for folks who want to read how BLS does thier two different survey’s:
http://www.bls.gov/sae/790meth.htm
Um, correlation? 131.5%? Time for a jaunt over to the statistics textbook.
not so sure i would beat the drum on nar data utility. the existing home sale data needs a very close inspection. for example, in 2005, nar said existing home sales (sf & condo) in florida were 546,400 units. the florida association of realtors said existing sf home sales were 248,565 (condo na). for 1q2006, nar says existing home sales (sf & condo) were 469,600 (saar), but far reports existing home sales of 61,158 (sf & condo), which if annualized is only about half of the nar figure.
an inquiry was made to nar about the wide discrepency and here is a portion of the response:
“…the difference between NAR’s estimates and FARs is that we benchmark our estimates to the Census, so in theory, we include for-sale-by-owner, non-MLS listings and MLS listings. (We went through a complete rebenchmark of our data in 2004 based on the 2000 Census.) The FAR numbers include only listings on the various MLSs across the state.”
i may be alone with concerns about nar existing home sales data, but it is a bit hard to accept that 48 percent of existing home sales in florida occur outside the listing service. particularly given the grip that nar has maintained over housing transactions (lobbying, legislations, etc.)
the nar existing sales data is used to calculate the month’s inventory of unsold homes. if i understand nar’s methodolgy correctly, they take the current month’s existing home sales and convert them to a saar. the saar figure is then divided by the inventory of unsold homes to get the month’s figure. however, where nar gets the inventory number or denominator from is unknown. an inquiry to nar about the source of the inventory has gone unanswered.
should anyone be aware of the source for the inventory data, please post. after examining the data for florida, the concern is that the inventory overhang may be larger than currently reported.
You spend most of your time on ADP, but I think your best criticism in this case is aimed at forecasters who fled their own models and jumped on ADP data. Getting the wrong answer is not a crime. Writing endlessly about “our model” and then abandoning the model when Macroeconomic Advisors offers another one is not a show of virtue.
But don’t tar everybody. Greenwich came out before the release sounding skeptical about the link between ADP and payroll data. So did others.
A look at the chart – ADP data against BLS private hires – shows that during the backcast period, there was a far better match prior to 2004 than since. Seeing that, you could guess that the 90% or so correlation ADP advertises between ADP data and BLS private hires depends on the period under consideration. My suspect that, given that Macro Advisors is involved, the model will be updated periodically, and misses like that in June (biggest on record, I believe) will be addressed. Till then, we will see fewer forecast changes in response to ADP.
Barry: Going on a vacation should make you less cranky, not more.
Despite the big miss this month, the ADP employment report has provided a more accurate forecast of monthly payrolls than any other method.
It beats the consensus 65% of the time. Vegas would like those odds. But that means the consensus is more accurate 35% of the time, like in June.
Let’s be brutally honest: You are not qualified to make a judgment on quality of the economics of this report. Macroeconomics Advisors is a well-regarded economics firm, with employees who are trained in economics and statistics; you are a trader who goes on TV and has a blog.
Finally, i guess you didn’t see that Joel Prakken did comment to Bloomberg in its updated story. I think you owe him an apology for your below-the-belt attack on his integrity.
I’ll give you 5 minutes to comment about this post. If you don’t reply, I’ll take that to mean that Ritholtz was “unavailable to comment.”
But I’ll still read your blog, no matter how off base you get.
Ah Grasshopper; “The Tao of the market where silence is golden”.
I wonder what Guru would be the qualified one to uncover such unknowable truths?
Who will bear such high responsibility?
Afterall, we are all so small and stupid and below knowing such deep analysis.
I’ll let Barry speak to his capabilities, but he must be as meek and stupid as the rest of us, right?
Bahhhhhh.
I don’t think I’ve ever seen correlation used like that. Normally correlation between two data series measures how well the two series match each other. A high correlation means that the two series go up and down by similar amounts at similar times. You can’t have a correlation between two data points.
“As far as the ADP report being a legitimate data source, they are no better than other self promotional private reports announcements.
Barry, i think you’re a bit off on your conclusion. Several houses applauded the report saying it’s good to have another tool in the bag to help with their internal forecasts. What’s wrong with ADP getting some PR from this piece? After all, they are the largest payroll processor in the country with over 18% of workers served. In addition Macroeconomic Advisors are the ones that filters and crunches the data, ADP just provides it. Further, who’s to say the BLS model is the most accurate to compare to?
Such data points as a poster mentioned earlier are more accurate in some cycles than others. Look underneath the sheets at the methodology and you’ll see where it’s useful to rely on more versus not. It’s what the brokerage houses stated which I agree with.
Well, ADP doesn’t do payrolls well either, so I can’t say I have any confidence in these bozos. They screwed up a payroll for us 6 months ago (gave an employee a $10K raise for a pay period, which I luckily caught) and botched the tax deduction and we are still waiting for our refund from them. They won’t give it to us — they have to get it from the state, they say. Never mind that they made the error…. Once we get our refund, sayonara, ADP.