Household versus Establishment Surveys, part 42

For some reason, there seems to be this ongoing wishful thinking that somehow, this economy is creating far more jobs than we are measuring.

I chalk it up to Projection:  When one’s own situation is personally improving, there is the temptation to believe everyone else is too. But that simply is not the case. The reality of the Zero Sum Game in trading markets, nature, and society as a whole, simply makes that impossible. For every leopard that eats dinner, an antelope dies; For each Amaranth that blows up, someone else is on the other side of the trade improving their performance.

Far too many people have a genteel misunderstanding of what zero sum is when applied to investing and trading, to economics and even information.

Which leads us to the Household Survey of Employment: Despite Greenspan, the BLS and other authorities repeated discussions why this is a less reliable number than Establishment Survey (NFP), it simply refuses to go away: The differences between the Household and the Establishment (NFP) surveys is a meme that’s long past its use by date. Its resurrection is like Jason in Friday the 13th or Freddy in Nightmare on Elm Street:  this one simply won’t stay dead and buried.

A quick review is in order. As we have discussed all too many times, the Household survey measures:

– Agriculture and related employment;
– Uncompensated Workers;
– Part Time Workers;
– Unpaid Family Employees;
– Workers absent without pay from their jobs;
Self employed, Work-at-home Contractors;

— none of which are counted in the Establishment (Non-Farm Payroll) Survey. In fact, the BLS specifically looked at and compared the two data
series. Once they made an adjustment so both surveys were
counting the same thing, any gap between the two disappeared.

Over the past few years, the Household survey has become the last refuge of bullish employment scoundrels. So rather than having a fit of pique when I see the same old meme getting circulated AGAIN, I decided to have a closer look at the smaller, less reliable, not payroll tax driven but self-reported Household survey.

Let’s put all that messy "factual stuff" aside for the moment. How did the Establishment NFP survey show but 51k new jobs, while the Household survey show 271k job creations?

A sharp-eyed friend directed  me to the “Explanatory Note” BLS releases with the Household Survey. Specifically, I scroll down under “-8-“, paragraph 3 “Reliability of Estimates.” This discusses the “confidence interval” a/k/a/ as the margin of error:

The confidence interval for the monthly change in total employment from the household survey is on the order of plus or minus 430,000. 

Suppose the estimate of total employment increases by 100,000 from one month to the next.  The 90-percent confidence interval on themonthly change would range from -330,000 to 530,000 (100,000 +/- 430,000).

These figures do not mean that the sample results are off by these magnitudes, but rather that there is about a 90-percent chance that the "true" over-the-month change lies within this interval. Since this range includes values of less than zero, we could not say with confidence that employment had, in fact, increased.

You read that correctly: The margin of error is 163% of the reported monthly Household survey.  This means that we a have a 90% degree of confidence that the “true over-the-month change lies within this interval" in between from -159k to +701k. 

Or as BLS specifically states: "Since this range includes values of less than zero, we could not say  with confidence that employment had, in fact increased."

Case closed.

Print Friendly, PDF & Email

What's been said:

Discussions found on the web:
  1. Leisa commented on Oct 10

    Barry…I would love to see this type of analysis on the reliability of the Manufacturing Conditions Survey that the Fed does. I know this is a repeat, but I think it is relevant….this is from the Richmond Fed’s manufacturing conditions.

    About the Manufacturing Conditions Survey
    Each month, the Manufacturing Survey is sent electronically or by mail to about 220 contacts whose firm type, firm size and location collectively match the profile of overall manufacturing in the District. In a typical month, approximately 100 contacts respond to the survey. Respondents provide information on current activity, including shipments, new orders, order backlogs, and inventories. In addition, manufacturers inform us about employment conditions, prices and their expectations of business activity for the next 6 months. I would surmise that the complexion of the respondents from one period to the next could significantly alter the presentation of the material….then one must ask, if the underlying information is not subject to data integrity with respect to either the acquiring and/or reporting of such information, why bother?

  2. KL2005 commented on Oct 10

    Zero sum game – how negative – HOW WRONG

    There have been enormous gains by all members of our country. You are getting to be comical.

  3. Barry Ritholtz commented on Oct 10

    Trading in the markets are a zero sum game.

    Amaranth loses 6 billion dollars, and that means the folks on the other side of the trade gained 6 Billion dollars. Its a simple concept fot those who trade professionally.

    Now, if you want to discuss the gains to overall society, we can all agree the overall pie has increased significantly. However, it remains a zero sum game as to how those gains are distributed.

    This is a very basic concept mathematical, and if you think it is comical, well, best of luck to ya — I’ll be waiting on the other side of your trades!

  4. grodge commented on Oct 10

    Afterall, BR, with all due respect you did state:

    The reality of the Zero Sum Game in trading markets, nature, and society as a whole, simply makes that impossible.

    On a given trade, or even event such as the Amaranth implosion, I can see your thesis; but the market seems to be telling us something with the higher volume and market strength.

    I truly truly enjoy and value your technical analysis. What are your calculations on the recent strength in various sectors?

  5. Morgan commented on Oct 10

    Sorry Barry, but I disagree with you, Greenspan, and anyone else touting the establishment numbers as a cumulative measure. The household survey is a better measure of cumulative change. It is a smaller sample, and subject to far more error month-to-month, but if you’re looking at jobs created over the last three years, it’s by far the better number. Why anyone would believe that a number known to have a cyclical bias beats an unbiased random sample of households over the long run is beyond me.

    Over time, the two do tend to even out, because, as I said, the relationship is cyclical. Self-employment grows when establishment hiring is slow, and shrinks when establishment hiring picks up.

    As a check, look at the NIPA tables – proprietors’ income (net of adjustments) increased from 8.17 to 9.09% of national income from Q1 2000 through Q3 2005. That’s $100 billion more going to proprietors than would have been the case if the proportion stayed the same (it’s now down to 8.63%, which may indicate that the cyclical reversal is already underway).

    If proprietors in business for, on average, 2.5 years net $25K (mean), that’s roughly 4 million new proprietors over 5 1/2 years. Four million people mostly not counted by the establishment survey – which is (order of magnitude) about the same as the cumulative gap between the two surveys.

    The test of employment bulls’ honesty will be whether they start to deny that cumulatively, the household survey is a better measure of job creation should the cycle reverse.

  6. Q-Ball commented on Oct 10

    Not exactly case closed.

    The month to month number is not accurate enough to guarantee a gain but if the results truly do fluctuation between that range based on a random sample then one would expect the results for different months to jump all over the place with some months being very negative and some months being very positive.

    Keep in mind that the higher end of the range was over 700K new jobs so maybe the results are just too low. While that seems pretty unlikely what does seem likely is that the high range of error is due simply to small sample size.

    As that data piles up over the years the sample size increases and should work to create a much more reliable number in a smoothed average of all the month-to-month reports.

    Unless there is something inherently biased with the sample itself such that it is repeatedly surveying only the strongest pockets of the economy and leaving out the weakest, then the cumulative affect of all the samples would add to reliability and the fact that it so handily beats the NFP survey every month says this is not due to confidence interval sample error.

    It doesn’t make it right, but the difference is not due to random fluctuation error. It is either measuring different things, or measuring an entirely different demography or sample. Its not a sampling error when you add it up over 3 years.

  7. oldprof commented on Oct 10

    Barry –
    If your sharp-eyed friend would read the footnotes to the payroll survey, she would see that the 90% confidence interval there is +/- 100K. Q-Ball is correct in pointing out that the estimate could be either high or low.

    This is why the benchmarking process, the closest we come to actually counting all of the jobs, is important.

  8. lurker commented on Oct 10

    Please don’t forget LUCK. Some of us, two actually, are lucky enough to be born George Soros and Warren Buffett. Don’t be Fooled By Randomness.

  9. alexd commented on Oct 10


    I hope that your comment was tongue in cheek. Since both Soros and Buffett created their accumulation of wealth.

    Now Paris Hilton………….

  10. tt commented on Oct 10

    Zero Sum Game

    GOOG ipo raises $1.67 B , for an initial market cap of $23B in 2004 …… if there’s Zero Sum here , please let me know who lost the $23 Billion

  11. Darin commented on Oct 10

    I think that whether or not we are operating in a ‘zero sum game’ is exactly what is in question. The recent boom in commodities was an market reaction to an influx of large amounts of inflation. The even more recent correction puts the possibility of any zero sum game in question for now. With the tremendous imbalance in the US current account, and the at least temporary disconnect between market realities, e.g. the persistantly ‘strong’ dollar’ vis a vis the debt burnden, shows that there is at least a lag time if not an indefinit stay between market issues. The question is never so absolutist as whether or not the markets are or are not zero sum, but rather what is the delay between cause and effect–that is where we all can really make money…

  12. me commented on Oct 10

    “Our free market capitalist economy a zero sum game? ”

    If it isn’t, tell that to these IBM engineers:
    “IBM is quietly laying off about 400 U.S.-based engineers who have
    been working on the development of components for one of the
    technology giant’s most important hardware products, according to
    sources familiar with the company’s plans.

    The cuts are taking place at IBM engineering facilities around the
    country, including sites located in Austin, Tex., Burlington, Vt.,
    San Jose, Calif., Raleigh, N.C., and Rochester, Minn. The cuts have
    also touched IBM locations in Poughkeepsie and Fishkill, N.Y

    If you are naive enough to think it it not zero sum why does IBM say :
    “The memo doesn’t specify whether the work performed at the affected facilities will be moved elsewhere. However, IBM has publicly stated its intention to invest $6 billion over the next three years”

    Those of us that used to work at IBM know damn well where those jobs went.

    And if it is sooo great, maybe you can convince theirs retired Delta pilot:

    “Jim Cochran expected bad news when he got the big white envelope from Delta Air Lines. What he found inside was worse.

    The retired Boeing 767 captain said he expected his $2,460-a-month pension to shrink by about 70 percent after Delta last summer won bankruptcy court approval to terminate its pilot pension plan.

    His benefit had already dropped by more than $3,000 when Delta cut part of the program after flying into Chapter 11 a little more than a year ago.

    But Cochran saw two things ”

    The two things he found were that his pension was reduced to zero and now, he owes delta $907 a month for medical insurance. Now keep in mind, the execs and other employees did not those thier pension

    As we all know, elections are about pocketbook issues. So maybe you can explain if things are so great why the republicans are in such deep do do?

  13. vfsv commented on Oct 10

    The employment picture becomes a little clearer when you deal with actual numbers.

    We track them at:

    In short, Santa Clara county residents with jobs peaked Dec’99 at 944K.

    The current figure is 784K.

    This is -160K jobs since the boom but the cheerleaders pronounce it a “recovery.”

    Not unlike the BLS, the mechanism is a bright headline every month which is quietly revised down in subsequent months. Month after month we hear about job growth, yet today’s figure is lower than Jan’03.


    To see “What’s really happening,” visit:

  14. Jim Bergsten commented on Oct 10

    Two observations:

    1. It ISN’T an zero-sum game. It’s a negitive sum game (as there is the overhead of commissions, spread, and so on). Then again, these overheads get put somewhere so I guess it IS a zero-sum game. But, wait, the Fed prints more money, so it isn’t, but wait, aside from comets and meteors, the sum total of physical assets isn’t growing, so it is. I’m confused.

    2. Confusion aside, one cannot (or at least shouldn’t) point at exceptions to disprove the rule.

    3. I’m equally rusty on statistics, but any survey that has a 430% margin of error ought to be thrown in the trash (thereby adding to entropy, making it a negitive-sum game).

    I need a nap!

  15. traderb commented on Oct 10

    “Trading in the markets are a zero sum game…..Its a simple concept fot those who trade professionally.”

    Barry, your above quote is not correct (I am a professional trader fyi). As asset values go up, money/value/something is created that is NOT to the cost of someone else. Rising asset values create wealth.

    Lets say I start a website…lets call it, ummm, i merely provide a service, but make no money, then Google comes in to buy me, and hey presto I have made $1.7bn. NOBODY has lost $1.7bn though. That valuation has merely been created as it is the present value of future cashflows (whether you agree with that discounting or not!).

    Great site btw, the best blog out there, even if I disagree with you here!

  16. jkw commented on Oct 10

    The market is a zero-sum game if you include the companies issuing the shares. They are short their own stock for every share they issue. But they have no margin requirements, never have to cover their short position, and don’t actually lose money when the price goes up. At worst they miss the profit opportunity of buying back shares when their stock was cheap and reissuing them when their stock became expensive. Including the companies seems rather pointless.

    Futures and options are zero-sum. But the market’s position is always net long equities (once you ignore the companies’ short positions), so net wealth increases when prices rise and decreases when prices fall.

    Society as a whole has always gotten wealthier over long periods of time (and most short periods of time too). The distribution of wealth matters. To have a healthy economy, you want a small upper class that people will work hard to join and a large consumer class that does most the work and consumes what is produced. A lower class is not necessary. The consumer class in America is shrinking now as the lower class is growing. This can only lead to recession.

  17. KL2005 commented on Oct 10

    basic economics courses teach that trading comodities is a ZERO sum game. An ounce of gold remains an ounce of gold and money is simply traded.

    Trading stock IS VERY DIFFERENT. A company that produces 500 widgets at the beginning of the year may be able to produce 10,000 widgets at the end of the year. Everybody can make money people who sell half way through the year and people who buy BOTH. It is not a ZERO sum game and you are not only comical and negative but dangerous as an adviser.

  18. Will commented on Oct 10

    jkw and KL2005 hit it on the head, commodities are zero sum.

    Real wealth is created in the equities market, if you don’t understand what a share of stock represents, I can see how you would think it is a zero sum game, but i sure as hell can’t see how you can act as a responsible fiduciary-

  19. Jim Bergsten commented on Oct 10

    The sudden image of a bucket of poop has inspired this comment…

    Suppose I poop into a bucket, label it modern art, and sell it for a million bucks.

    From MY perspective, I’ve created wealth, one million bucks of wealth out of, well, poop.

    It’s not a zero sum game to ME (or to my investors who fed me ExLax to boost my creativity).

    But, some schmuck PAID me $1M for that poop. He may well FEEL enriched (especially if he turns around and sells it for $2M), but for the moment he’s out $1M. From a CASH perspective, no NET wealth was created (and if we tack on taxes, commissions, legal fees, audit fees, due dilligence fees and so on, well, still no NET wealth was created).

    Wealth is DISTRIBUTED in the equities market, not CREATED.

    Now, one can argue, the world is one bucket of poop richer. But the poop didn’t, ummm, appear out of nowhere either.

    I think if you look at a piece of the economy, however large, you can conclude there is a net gain or loss. If you look at the whole thing, it’s zero minus entropy. Has to be — that’s why a 5 cent candy bar is now 75 cents. An ounce of gold, or an acre in Manhattan don’t change, their PRICE changes, thus the CURRENCY changes value, NOT the hard asset.

    If it’s any consolation, investors neither own nor care about the whole economy, they care about their returns, and for them it is certainly possible to create wealth (theirs) at the expense (realized or unrealized) of others.

    And, to save you the trouble, yeah, it’s a whole lot of poop.

  20. Will commented on Oct 10

    horrible example Jim. companies take resources, and convert them into more valuable products/services, which make life easier, allow technological advances, allow us to type like monkeys all day rather till a hectare, etc.

    now, it doesn’t sound like you understand that a share of stock is a contingent claim on a company, ie equity, nor do you seem to understand that productivity creates wealth, not redistributes it-

  21. Jim Bergsten commented on Oct 10

    Well, sure Will. That’s why I said it was all poop.

    Even so, you’re looking at the few points of light in the overall darkness.

    More poop.

  22. Will commented on Oct 10

    let’s use a real life example:

    GOOG gaps up this morning and is instantly 2.2% higher than yesterday’s close. The shareholders are wealthier, but who’s poorer? the wealth was redistributed right?

    likewise, company ABC gaps down 10%, the shareholders are worse off, but who’s richer now?

  23. alexd commented on Oct 10

    More than one game is going on.

    I buy something, I sell it for more money. Therefore I have more money. I sell it for less, I have less. The buyer may or may not have more or less. Companies are tough a lot of good faith is going on there. We hope that the company does well so that our per share value goes up. But it only is worth what the market offers us, so that too is dependent on what the other side of the trade thinks the price might become. Unless the comany is being bought out or it passes on substantial and hopefully increasing dividends we are at the mercy of the perception of what might happen to the percieved value of the person who is on the other side of the trade. If the company is selling below book or is holding vast amounts of cash then the game is a bit different. Then it has something that can be quantified. I do realize that ingeneuity and productivity are very important, but they are tough to pin down.

    But if I am a manufacturer and I am buying a commodity to change into a value added product then there is no loss. I am buying stuff to make into more sophisticated stuff. Which hopefully will be sold at a decent profit.

  24. samuel commented on Oct 11

    I think a lot of this money that is paying for all these stock purchases etc is from a massive debt bubble that seems to be hidden from most market observers who refuse to acknowledge it. It’s about 42 trillion dollars right now.

    Japan figured out that it is indeed a zero sum game. The money was created through a debt bubble and then transferred to other market participants via the stock and real estate markets, though the future obligations for the most part still exist. Someone still has to pay those debt obligations unless they are liquidated through bankruptcy, in which case the creditors lose out.

    Same situation now in the US, we are creating massive future debt obligations so we can party in the stock market and real estate markets. Eventually, that money will have to be paid back or the creditors will lose that money, and most of the creditors for the mortgage market are savers from East Asia.

  25. Jim Bergsten commented on Oct 11

    Say GOOG gaps up 2.2%, and I happen to own GOOG. Am I 2.2% “richer”? On paper, perhaps, but not in “reality” unless I sell (and get that price). Nor am I “poorer” if my stock gaps down 10% unless I sell.

    So, my answer to Will is nobody “on the sidelines” is richer or poorer (yet), because no wealth can be redistributed without equities changing hands.

    An even better example: I form a company, price the stock at .001 cent and buy a few million shares for a few thousand bucks. I then have my Board (me) reprice the stock at $10/share. Am I now a multi-millionare? Maybe I can con a bank into thinking so with my fat financial statement, but without an arms-length buyer…

    The point is, this whole “zero-sum” thing only comes into place when equities change hands, otherwise it’s all just numbers.

    Samuel’s excellent point is that borrowing is in fact a bad and unregulated way to create money (I lend $1 to you, you lend to Joe, Joe lends to Fred, Fred lends to me, there’s now $4 on the books where there was arguably at most $1 to begin with). But since it creates virtual money, it dilutes the value, just as if the Fed had printed a whole bunch.

    The troubling question here is the issue of creation of value out of “nothing.” Say I discover a cure for cancer tomorrow. Have I created value? Well, sure, from a “good for humanity” point of view. But have I created MARKET value? Well, maybe. What if I give the cure away? What if I THROW the cure away?

    I think this strays from the simpler issue of whether the equities markets, taken as a whole are a zero-sum (or worse) game. At the risk of copping out by dropping names, “both” Adam Smiths seemed to think so. So did Jesse Livermore. So should you.

Posted Under