Redux: Household versus Establishment Surveys

One last item:

"The Labor Department’s payrolls report is also at odds with its own survey of households, which is used to calculate the unemployment rate. The household survey showed employment grew by 387,000 in June, in line with ADP’s figures.

The enormous disparity of recent months coupled with the ADP data supporting the household-data implications raises important questions about the reliability of all three employment counts,” said David Resler, chief economist at Nomura Securities International in New York."    –Bloomberg

Umm, no — that’s simply incorrect (and Bloomberg and Nomura Securities ought to know better).

For the record: The Labor Department’s Establishment Survey (aka NonFarm Payroll report) number of 121k WAS NOT AT ODDS with its own Household Survey of 387k. These two reports measure two very different things. The numbers can be off by a few 100k — and still be consistent with each other. 

As we have discussed all too many times, the Household survey measures:

– Agriculture and related employment;
– Uncompensated Workers;
– Unpaid Family Employees;
– Part Time Workers;
– Workers absent without pay from their jobs;
Self employed, Work-at-home Contractors;

— none of which are counted in the Establishment (Non-Farm Payroll) Survey.   

In fact, the BLS specifically looked at and compared the two data series back in 2004. Once they made an adjustment so both surveys were counting the same thing, the huge gap disappeared.

I seriously challenge the expertise of any economist that fails to recognize the different data series.

This is something I would hope that mainstream economic reporters would understand — even if they are not economists (i.e., Bloomberg!). Any reporter that dutifully repeats this tripe has been punk’d.  If this is your beat, and you are not familiar with these two survey methodologies, than you better get up to speed quickly. Especially if you are going to get dissed by some economist trying to cover his tracks on
a regular basis.

Here are the graphs from the BLS report:

1994 -2004
click for larger graphics

Bls_hs_19942004

2001-2004

Bls_hp_200104

 

Note:  This isn’t a new development — these are from 2004.

>

Sources:
ADP Job Survey Loses Luster Among Economists After June Miss
Joe Richter
Bloomberg, July 7, 2006 15:09 EDT
http://www.bloomberg.com/apps/news?pid=20601087&sid=aB3TRqqC9acI&refer=worldwide_news

Bureau of Labor Statistics, (PDF)
BLS report, March 5, 2004
http://www.bls.gov/cps/ces_cps_trends.pdf

BLS on Payroll vs. Household Survey
The Big Picture, March 14, 2004
http://bigpicture.typepad.com/comments/2004/03/bls_on_payroll_.html

Print Friendly, PDF & Email

What's been said:

Discussions found on the web:
  1. pf commented on Jul 8

    Looks like the difference after the adjustment is roughly about 10,000 jobs per period …. job growth appears to be , for both households and payrolls , between 112k -133k / month from 1994 thru 2004 …. almost the same for Clinton and Bush ?…. am I reading this correctly?

    BR: No, its not a read of monthly employment gains — its total employment.

    150,000 (in thousands) is ~150 million employed.

  2. jm commented on Jul 8

    Barry, you might want to add a link to ADP’s methodology description at “http://www.adpemploymentreport.com/PDF/ADPNERMethodology.pdf”

    The discrepancy between the BLS and ADP numbers is extremely interesting, as the historical correlations shown in that methodology document are very good, and ADP tout their report as being earlier than the BLS report to detect changes in trend.

  3. donna commented on Jul 8

    So I guess a lot more people are self-employeed contractors now, huh?

    Yeah, me too, and I haven’t worked in over a year (by choice, fortunately – I imagine a lot of those other “self-employed contractors” aren’t so lucky.

    It’s a lot easier to explain being self-employed than that big gap in your resume. ;^)

  4. Barry Ritholtz commented on Jul 8

    No, thats not a monthly measure, its total employment — 150,000 (in thousands) is ~150 million employed.

  5. Mark commented on Jul 8

    Barry-

    Geezuz. If IT WERE THE SAME THING wouldn’t they 1) call it the same thing and 2) have it only done ONCE by the same group? Don’t these people ever THINK?

  6. Brian commented on Jul 8

    Yeah, I used to have a real job. Now I’m a self-employed contractor. I deal with the giant gap in the resume by calling myself a consultant. Impressive no? I’ve been selling some junk on Ebay and have considered calling myself a Cyber Collectable Coordinator.

    Ebay seems like it should be a good bear market stock doesn’t it?

  7. A Dash of Insight commented on Jul 8

    Redux: Household versus Establishment

    While Barry Ritholtz’s blog is a big favorite for lots of us, I do not think that David Malpass, Chief Global Eocnomist for Bear Stearns, is going to drop by to comment on the employment numbers. His work in the

  8. whipsaw commented on Jul 8

    I don’t have much doubt that ebay will pick up sell listings as things degrade, but what about the buy side? Failed auctions are just failed auctions.

    My rule of thumb if you don’t know what else to do:

    Good Liquor Store+Good Location=Profit!

  9. JRG commented on Jul 8

    Personally, until convinced otherwise, I would hang my hat on the ADP numbers….. which are based on actual payroll processed numbers (for a large portion of small companies).
    BLS (the govt) has always had a terrible lag time in even getting new small companies into their survey data base.
    This is a chronic problem for govt agencies.
    I’ll go with private sector data over govt data any day.
    Prove me wrong!
    (I’m willing to listen)

  10. Cherry commented on Jul 8

    Prove you wrong? ADP is ALWAYS wrong. They think some part time job paying 7.00hr should count on these figures.

    Give me a break JRG.

  11. JRG commented on Jul 9

    Dear cherry,
    a job is a job.
    What definition does BLS use?
    (I’m going to guess that anything over 24 hrs per week counts)
    My point is that BLS is asking companies the same basic questions/collecting the same data that ADP is getting. However, I still say ADP has better real-time info (by 6 to 9 months). That is the way it has always been. Prove to me that the govt has gotten that much better than they have traditionally been [say I based on my experience on the Hill on a Senator’s staff in the 80’s, and visiting with my friends who were political appointees under Clinton in the 90’s, plus other points of reference including the Survey Research Center at the Uof Mich]
    If you have never worked in DC, or in a Federal Agency, you have a lot to be amazed about.

  12. cm commented on Jul 9

    Donna: “It’s a lot easier to explain being self-employed than that big gap in your resume. ;^)”

    You can still report different things in the CPS survey and on your resume; they will not be cross-checked.

    I’d say it is more an issue of individual pride and keeping up appearances to oneself than that. Which is probably the strongest force that there is.

  13. cm commented on Jul 9

    Brian: Same thing I said to donna. On your resume you can do window dressing, but that has no bearing on your actual status unless you participate in the CPS and tell them the same thing, and not according to the legal definition. I’m not suggesting you are or should be lying on your resume, but Legalese and vernacular English are two different languages where words don’t necessarily mean the same thing.

  14. Steven commented on Jul 9

    Liquor store in Detroit
    More risk –> greater reward

    I know a guy with a fast food joint there that rakes it in

  15. drey commented on Jul 9

    my definition of ‘self employed consultant’ would be a well educated individual with an uncertain income and zero benefits/health insurance.

    Don’t get me wrong, I’ve been there – and probably will be again.

  16. Barry Ritholtz commented on Jul 9

    There are many things the private sector does much better than governement or academia; This isn’t one of them.

    Here are the numbers comparing the ADP Report (Private Sector Only) with the BLS data (actual):

    Month ADP BLS Correlation
    January
    162k 154k 92.6%
    February
    342k 200k 58.4%
    March
    133k 175k 131.5%
    April
    178k 126k 70.8%
    May
    121k 75k 62.0%
    June 368k 121k 32.9%

    Any questions?

    I’ll have some more on this tomorrow . . .

  17. Blissex commented on Jul 9

    «‘self employed consultant’ would be a well educated individual with an uncertain income and zero benefits/health insurance.»

    In the IT business nowadays the canonical definition of consultant is ”unemployed white middle aged guy who still has some white shirts, a briefcase and an old laptop”. :-)

  18. me commented on Jul 9

    I only posted part of this article, but this is the nature of “the consultant”.

    The tech worker’s contract to nowhere

    Never heard of it? Well, you’re missing out on the most wonderful thing since child labor. Agencies and employers use the contract-to-hire scam to pull in top-rung permanent workers with the promise that if they prove themselves worthy during the contract period — it’s described as a sort of probation — there’s a good chance they’ll be hired. That good chance is next to no chance, but a worker who isn’t wise to the scheme will sweat blood, volunteer unpaid overtime, and carry a company-issued cell phone to earn that upgrade to permanence.

    A contract extension is held out as being one giant step closer to full employment. It’s actually a sure sign that there is no permanent position, but so what? The employer managed to sidestep having to renegotiate the contract because, ho-ho, a contract-to-hire doesn’t realize he or she is a temp. Benefits? Taxes? Paid overtime? Contract pay scale? That’s for chumps.

    http://www.infoworld.com/article/06/06/28/79610_27OPcurve_1.html

  19. cm commented on Jul 9

    me: I read the whole article; good points.

    I have previously been railing against incompetence in HR and staffing organizations (and won’t generally take that back), but lately I had an eye-opening experience in an interview roundtable where I asked the staffing rep directly what’s the deal with “we get too few resumes”, and the response was quite clearly, while not directly stated, that they are using very tight “filters” (that much I knew) browsing through internal and external resume databases, but at the behest of hiring managers who don’t want to “waste their time” screening too many candidates. Then the very same managers turn around and complain how difficult it is to find good people, pushing the onus back to HR.

  20. Brian commented on Jul 9

    I’ve been wondering if there’s some trick to getting your resume through the filters. Or if a person even looks at most of them. Seems like HR types look at potential employees as modular parts to fit into a prexisting machine. If you don’t have whatever quantitative specs they’re looking for forget about it.

    On the other hand I have a friend who is a nurse. She gets cold calls and junk mail begging her to work. So, I guess a few million of us should go to nursing school.

  21. cm commented on Jul 9

    Brian: I don’t know how exactly those filters work, but I can honestly not imagine any method other than looking for keyword and keyword combinations (and, importantly, *rejecting* certain keywords) to select from a large pool of resumes without any other information.

    So, at an abstract level, you have to use the right keywords in your text, and the problem is that you don’t know what the “other side” is looking for. (Assuming a level of honest trying on the latter.)

    I looked for a job in the US at a time when they were desperate for any warm body, and that certainly helped. I got few, but exclusively relevant calls (I wanted to change industry, my “relevant experience” was an RA job in grad school, and my more extensive industrial experience was only very generally applicable). I went to exactly one interview, and they made sure to let me know right on the spot I will get an offer. I’m not sure that today I would be considered at my experience profile I had back then (note, not “level”, but “profile”).

    And there is nothing malicious about the filtering — nobody can reasonably review thousands of candidates. But there is no excuse for not selecting whichever 10 out of the thousands every week, after all, they are “looking for people”.

  22. JRG commented on Jul 9

    I just accidentally bumped into this link and info:
    “The ADP National Employment Report for June, reported yesterday, was correctly calculated. There is no truth to rumors that the ADP National Employment Report released yesterday, signaling a June increase of 368 thousand in private nonfarm payrolls, was erroneously calculated. Macroeconomic Advisers stands behind the calculations. Over the 69 months in which the index is available, from 2001 to May 2006, the standard error relative to the “as was reported” BLS preliminary estimate (for private nonfarm employment) is 91 thousand…….”
    http://www.macroadvisers.com/csx3/csxPage.asp?APPNAME=Macroecon&LABEL=CSXGETPAGE&CLASS=Macroecon.pclsMain&METHOD=pageMain

    [Meyer’s MPI

    Laurence H. Meyer’s Monetary Policy Insights analyzes the strategic link between the US economic outlook and the policy actions of the Federal Reserve. MPI clients benefit from timely web-based briefings and direct personal attention from Dr. Meyer, an award-winning forecaster and former member of the Federal Reserve Board of Governors]

    Barry – do you know much about these guys?

  23. dloberk commented on Jul 10

    A good analysis of this from an economist at Northern Trust who understands it correctly states in referring to the Establishment Survey “there is a steady deceleration of hiring in place” and concludes that “the June employment report clearly indicates that hiring has slowed significantly in the U.S. economy. The weak tone of this report offers “cover” for the FOMC to pause, assuming that the July employment report, to be published on August 4 (prior to the August 8 FOMC), does not have payrolls increasing in the range forecast by ADP/Macroeconomic Advisors for today’s June report.”

    Link to analysis: http://www.northerntrust.com/pws/jsp/display2.jsp?XML=pages/nt/0601/1138283678319_6.xml&TYPE=interior

  24. JoshK commented on Jul 10

    Does anyone know the collection methodology for the BLS #’s? My gut is to trust ADP #’s b/c they come out of their payroll system and I suspect that the BLS #’s have a large dependency on people with phones and clipboards. But maybe someone here knows more?

  25. jrg commented on Jul 10

    JoshK
    check this site out:
    http://www.bls.gov/sae/790meth.htm

    Here is a some of the ‘scary’ part:
    “BLS cooperates with State employment security agencies in the Current Employment Statistics (CES) survey to collect data each month on employment, hours, and earnings from a sample of nonfarm establishments (including government). ”
    I’m still reading……

  26. jrg commented on Jul 10

    more excerpts:
    “Total employment in nonfarm establishments from the CES or payroll survey is not directly comparable with the Bureau’s estimates of the number of persons employed in nonagricultural industries obtained from the monthly household survey. (See chapter 1 for a description of the Current Population Survey, or household survey.) The two surveys have differences in concept and scope and employ different collection and estimating techniques.”
    and
    “All firms with 250 employees or more are asked to participate in the survey, as well as a sample of smaller firms. Together, they comprise the largest monthly employer survey in existence, with a sample of over 350,000 establishments. Despite the voluntary nature of the survey, numerous establishments have reported regularly for many years.”

    http://www.bls.gov/sae/790meth.htm

    (you really must read all about their ‘methodology’…… the more I’ve read, the more I believe the ADP numbers are more accurate over-all)

  27. JoshK commented on Jul 10

    JRG,

    Thanks. This just sounds abusrd, I’m going to bet on ADP, too.

    -J

  28. A Dash of Insight commented on Jul 10

    Redux: Household versus Establishment

    While Barry Ritholtz’s blog is a big favorite for lots of us, I do not think that David Malpass, Chief Global Eocnomist for Bear Stearns, is going to drop by to comment on the employment numbers. His work in the

  29. whipsaw commented on Jul 10

    “Over the 69 months in which the index is available, from 2001 to May 2006, the standard error relative to the “as was reported” BLS preliminary estimate (for private nonfarm employment) is 91 thousand…….”

    91000? That isn’t ‘standard error’ that’s Wild Ass Guess. Somebody is grossly wrong and I’ll go with Barry on this- ADP needs to quit putting out the puff jobs.

  30. szara commented on Jul 11

    It’s kind of ill informed to describe the BLS NFP survey as a bunch of people with clipboards. Every employer that provides unemployment insurance (ie damn near everybody) has to provide them with a monthly employee count. They draw an enormous sample from these records to do the NFP data. People gripe about the birth/death adjustments, but they seem to do a credible job, and they tinker with it every year to fine tune it. With several months lag, they also publish the 100% count, so you can check their how close their initial estimates were. It’s really hard to find a more solid economic data source. The initial release is preliminary, though, and is often revised substantially.

    On the other hand, people shouldn’t be so quick to bash Macroeconomic Advisors, the firm that puts the ADP data together. It’s former Fed governer Laurence Meyer’s firm. He has a long standing reputation as an excellent data cruncher and forecaster. I don’t know how active a role he plays in the firm now, though.

    My guess is that the “true” payroll growth was close to the 155k consensus, and that next month’s revision of the BLS data will show that.

Posted Under