Inflation Data: Accuracy versus Precision (as if either matters)

One of the more interesting issues of any modelling or statistical work is whether they have the attributes of Accuracy or Precision.

Accuracy_and_precisionhighaccuracylowpre
Accuracy is defined as the "degree of conformity of a measured/calculated quantity to its actual value." I often refer to this as how closely a measure represents  reality.

Accuracy_and_precisionhighprecisionlowac
Precision is defined as "the degree to which further measurements or calculations will show the same or similar results." A precise measure will be reproducible or repeatable.

Ideally, any economic measurement should aim for both qualities: they will be
accurate in that they reflect what reality looks like, and they will be precise, with measurements close to and tightly
clustered around "a known value."

Which brings us to the Bureau of Labor Statistics: they are contemplating a change in the consumer-price index (July data released tomorrow) which may have a big impact on how the latest inflation data gets interpreted. BLS is  considering publishing the index (and subindexes) "to three decimal places instead of one . . . [to] greatly reduce the frequency with which rounding produces a misleading inflation rate."

Let’s take a moment to consider the humor of this statement:

On the one hand, it is an admirable goal on the part of the econ wonks who do the heavy lifting and produce the initial data for CPI and PPI. I have spoken with many of the economists and statisticians at BLS, Commerce Department, and DoL. I have yet to meet anyone there who wasn’t intelligent,  hardworking  and diligent in the pursuit of their craft.

The problem comes about after they produce their numbers. It is not the precision or accuracy of the data; rather, it is the bias of the model and the subsequent torture of that data where the inaccuracies and imprecisions are generated.

Accuracy and precision have become almost
irrelevant, as it ultimately gets fed into economic machinary that
is so biased towards showing a minimal level of inflation — regardless of what is happening in the real world.

Example: Yesterday’s WSJ had a front page article on commodity prices:

"Shortages and high prices for raw materials are fueling a new and
unusual wave of acquisitions and deals. Steelmakers are buying iron-ore
mines, airplane manufacturers are striking long-term deals for
titanium, and the world’s second-largest tire maker is cultivating
rubber trees.

This return to a type of vertical integration that has been out of
favor for decades signals a new phase of industry consolidation. Having
bulked up by acquiring rivals, manufacturers are turning their
deal-making prowess to raw materials providers in hopes of ensuring
adequate supplies and controlling costs."

And yet, inflation is said to remain tame "in the core rate." An entire wave of M&A and vertical integration in several industries is underway due to soaring commodity prices, and yet, we continue to hear from officials how contained inflation is. Once you consider the absurdity of that, you will realize why 3 decimal accuracy is a noble effort in a wasted cause.

Precise data fed in, inflation ex-inflation out; it is a classic example of torturing the data long enough to get it to confess to whatever crime you want.

For example, the BLS surveys prices on a monthly basis about 80,000 items in 200 categories. The results of that survey then gets converted to series of index numbers. That’s how the  CPI and PPI index and core get created.

The WSJ noted that:

"Because of rounding, these inflation figures can be misleading — in two ways. One is that the percentage change is rounded. For example, an increase of 0.249% would be rounded down to 0.2%, while an increase of 0.251% would be rounded up to 0.3%. The difference between 0.2% and 0.3% seems large, but without rounding the difference is trivial. Economists, however, can adjust for that problem by calculating the percentage changes in the indexes themselves.

The second, more serious way the figures can be misleading results from the fact the BLS rounds the indexes as well before publishing them. Suppose the index for one month is 198.945, and then rounded down to 198.9, and the index for the next month is 199.355, and then rounded up to 199.4. The change in the rounded numbers is 0.251%, which rounds up to 0.3%, but the change in the unrounded numbers is only 0.206%, which rounds down to 0.2%.

The difference between 0.2% and 0.3% can have a huge impact on the market."

Accuracy_and_precision_1This precision difference is analogous to sterilizing the scalpel for an incompetant surgeon. No matter how clean his surgical instruments are, if he kills the patient, it is irrelevant. As long as the pateint doesn’t die from an infection caused by dirty scalpels, the person who preps the tools has done their job.

That is similar to the situation the BLS statistical analysts find themselves in. They want to make sure that the inflation statistics — nonsensical though they might be — are neither imprecise nor inaccurate due to anything they did.

The net of this is that, assuming this change gets incorporated in the BLS data surveying methodology, we will have the luxury of more precise measurements to be ignored by policy makers.

Inflation? What Inflation?

P1af464_vertic_20060813190036
After all, if the focus is remains obsessed on the core rate of inflation, if the Fed and other policy makers ignore long term underlying inflationary trends — if they only consider those items not going up in pricehow much does it really matter if the inflation data is accurate to one two or three decimal places?

>

The problem lies not with the dirty scalpel, but the incompetent surgeons . . .

>

>

>


Source:
Key Inflation Index May Get Greater Precision
GREG IP
WSJ, August 14, 2006; Page A2
http://online.wsj.com/article/SB115552131969034794.html

Accuracy and precision
http://en.wikipedia.org/wiki/Accuracy

A Hot Commodities Market Spurs Buying Spree by Manufacturers
TIMOTHY AEPPEL
WSJ, August 14, 2006; Page A1
http://online.wsj.com/article/SB115552274982134821.html

Print Friendly, PDF & Email

What's been said:

Discussions found on the web:
  1. Royce commented on Aug 15

    Barry, I get that you don’t like the government’s inflation numbers. So what do you think it should be? 5%? 10% 15%?

  2. spencer commented on Aug 15

    Good comments.

    This will lead to people claiming or thinking the data has a degree of precision that does not exist. There is no economic significance to a difference of 0.1 in the reported inflation rate.

    If the brokerage houses want to take traders that bet on the data to the clearners why should the government interfer.

    Is there a problem with decision makers like the fed making a bad decision because of a difference of 0.1 in the data for one month — expecially since it is corrected the next month? I do not think so.

  3. 23 commented on Aug 15

    Can’t we just get 10 digits after the decimal?

    I mean, the difference between the 1% “core” inflation and the 5% “true” inflation is only 3.58395875% when you don’t round it off.

  4. ss commented on Aug 15

    You make fair points on the models Barry.

    I would point out on commodities, that they’re not immune from all the slowdown talk we read on your blog…how much copper, cement, wood, etc is used in the housing industry? In addition, many argue that speulators in the futures market inflate these very prices during tight demand, low supply times. What happens to the commodity cost side when demand softens and (gasp) new supply comes in?

  5. PeterB commented on Aug 15

    A falling PPI actually confirms a slowing economy. How can THAT be good for equities?

  6. T commented on Aug 15

    Goldilocks my friend. Now where’s the three bears (or maybe one?)

  7. trendwatcher commented on Aug 15

    Barry. Loved your target graphics for accuracy vs. precsion. Reminds me of the story of two snipers shooting at each other. The first one is absolutely precise with a cluster of shots all missing his target’s head by exactly 1 inch. The second shooter is reality based and he learns from each shot getting closer and close to the real target till he hits it dead on.

    That said, I would prefer to see an extra digit added on to any metric that currently is reported to only a single significant digit. I always aim for using 2 or 3 significant digits in the work I do based on the recommendations of an excellent book by Ehrenberg (Data Reduction). That way, effective surgeons will have a clean scalpel to work with.

  8. ss commented on Aug 15

    Another favorite old saw for the inflation bears has been the coming collapse in the dollar…caused by everyone (who hates us) pulling all their investments away from us and the Dollar. Well today’s data suggest something different — June Net Foreign Security Purchases…reported at $75.1B, above consensus or $65.0B and above last months revised $63.6B (down from $69.6B).

    Some grizzle for the dollar bears to chew on…

  9. konman commented on Aug 15

    I wonder why every time the market gets worked up about inflation a suddenly good number appears. Maybe it is becasue the Gov is simply managing its liabilities? All the while we see reports from truckers – man on the street interviews – about the shortage of diesal fuel and the residual impact on the supply chain. Inflation is everywhere for anyone paying attension and yet we have to be treated to a parade of commentary and statistics which defy logic. When does the charade end? The obfuscation on the war on terror (maybe profiling) and the economy are amazing; the government must think everyone is an idiot.

  10. Leisa commented on Aug 15

    PeterB…I agree. I’m reminded of Marty Zweig’s 3 characteristics of a bear market that I posted here within the last month or so…one of the 3 requirements were falling prices. Remember the business inventory build number released last week. I think one reduces prices to get rid of excess inventory.

  11. Michael C. commented on Aug 15

    >>>A falling PPI actually confirms a slowing economy. How can THAT be good for equities?<<< I don't think that's necessarily true. Just like strong inflation lately hasn't been confirmation of a strengthening economy. As for equities, people were selling on fear of inflation. So, with the tame and weak inflation numbers, the buy back begins...

  12. Michael C. commented on Aug 15

    Also, let’s see the the CPI tomorrow. Per Tony C on Realmoney, there is a larger auto factor and no rent in the PPI.

  13. Fullcarry commented on Aug 15

    Do exploding Dell laptops cause hedonic adjustments lower in the inflation indices?

  14. Chief Tomahawk commented on Aug 15

    The U.S. is so dependent on attracting foreign investment to fund it’s spending that it’s practically a matter of national security. To keep the engine going there’s plenty of incentive to try to present figures which make the best possible case. To lose foreign investment would trigger a downward spiral of cascading proportions.

  15. BDG123 commented on Aug 15

    Let me turn that statement around on you. My belief is that certain foreign economis are so dependent on the American consumer that they must continue to lend to them as a matter of national security. To keep the engine going there’s plent of incentives to try to present figures which make the best possible case. To lose the American consumer would trigger a downward spiral of cascading proportions.

  16. M.Z. Forrest commented on Aug 15

    I wouldn’t get so excited SS. Net foreign flows into the stock market have gone from 21.2 billion in Jan 06 to (3.971) billion in June 06. That is six months of declining flows. (See here.)

    On the other side of the coin, the last month where the U.S. redeemed more foreign bonds and stocks than it purchased was back in March 2002. From May to June of this year we did decrease our purchases of foreign secuities, buying only 9.6 billion net in June versus 20.5 billion net in May.

  17. tjofpa commented on Aug 15

    Bingo on the Bill Cara article… mad rush to buy in the first 30 secs the stocks nobody wanted the day before;
    then mostly RED candles for the next 4 1/2 hours of trading.
    Won’t this little diddy in the PPI report have the effect of reducing tomorrow’s CPI calc;
    “a 1.8% increase in wholesale residential electric power prices”
    because in the bizarro world of Gov’t econ models rising util bills REDUCE the owner’s equiv rent calc?

  18. Bill Cara commented on Aug 15

    Inflation data: precise, accurate or neither?, Tues., Aug. 15, 2006, 11:06 AM

    “Bernard” wrote to ask what’s up with the government’s proposed change to CPI reporting, as linked to this article….

  19. JDamon commented on Aug 15

    While I respect the intelligence of the posters on this blog, I have to say that most who are bearish here will NEVER be bullish and that no matter what “data” is presented, they continue to try to turn the “data” to their favor or just label it inaccurate. I true intelectual can learn from their mistakes. Can perma-bears?

    I want to see some of the frequent posters post a POSITIVE comment on either the US economy or the Stock/Bond market (if they can).

    ~~~

    BR: What is your basis for that unsupported conclusion? You read the blog during a kate cycle peruiod when most of Wall Street is uber Bullish, and thats the basis of your conclusion?

    Come back in 6 months and see if there’s some bullishness when everyone else is panicking

  20. snook commented on Aug 15

    Excuse me. Has anyone priced hotel rooms in Carmel, CA this week(end)?

  21. kckid816 commented on Aug 15

    JDamon,
    How can you post something positive when the signs are pointing to bad things?

    Put down the kool aid.
    I know things maybe nice for you but for people the don’t make 6 figures a year the economy isn’t so great.

  22. Bob_in_MA commented on Aug 15

    Barry,

    Your use of anecdotal data (commodity prices) to buttress your argument against official inflation numbers is severely flawed. The sharp rise in commodity prices DOES show up in official data, look at the PPI for crude goods. There then is a somewhat smaller, delayed rise in the PPI of intermediate goods, and yet a smaller rise in the PPI for finished goods….

    The crux of your argument, today’s version at least, seems to be that the PPI for crude goods should be the headline inflation number, and presumably the Fed should target that.

    Why? Is that something the Fed can really influence?

    You throw up these posts, making half an argument here, taking a graph from there, etc., but you do a very poor job of working into a coherent argument. Maybe that’s just the nature of blogs.

    The official inflation data may well be profroundly flawed, but you have yet to make the point.

    BR Bob,

    Have a look at today’s PPI report for more on the same issues.

    There are ZERO ANECDOTAL DATAPOINTS mentioned in PPI — energy, chemicals, metals — all appreciably higher.

    The key data driving PPI down? Light turck prices, down 3.1% due to high energy prices

  23. Craig commented on Aug 15

    “Another favorite old saw for the inflation bears has been the coming collapse in the dollar…caused by everyone (who hates us) pulling all their investments away from us and the Dollar.”

    Do you suppose foreign investors could be as emotionally unstable as the above statement would indicate if they invested in the U.S. (the supposed object of said hatred) in the first place?

    SS, I know you are a diehard bull cheerleader and it’s your money to lose, but it might be better to look deeper, and longer term at these things than to emotion.

  24. ss commented on Aug 15

    MZ….see the trees from the Forrest (sorry, couldn’t resist).

    The data I posted was Net Foreign Security Purchases…reported at $75.1B, above consensus…not a reading on US Equities. My point is relating to the (supposed) inflation pressure that was coming from a collapsing dollar…creating higher rates, and inflation….all wrong….oh, and despite a typical DBG post, there is no conspiracy.

    JDamon…the “half empty crowd” is a cerebral, yet stubborn crew.

    As far as me?…if the curve inverts (hard), and banks collapse, and the much hyped “flation bubble” rises from the blogosphere to reality, you’ll have my attention…and my repositioning of my portfolios. I “hear” your sound arguments…but I don’t “see” it unfolding that way.

  25. Barry Ritholtz commented on Aug 15

    Bob,

    Hardly anecdotal data. My argument is that whether the data is to one or three decimals, its irrelevvant due tot he obsessive focus on the Core data.

    Is it your position there is no or little inflation?

  26. Craig commented on Aug 15

    Positive statement: You could make some decent money if you follow the advice of some of these bears.
    They are making it on the up and down tape.

    That’s positive isn’t it?

    Guys, it’s a market, it goes UP and it goes DOWN. Neither is good or bad, they just are. It would be mindnumbingly simple if all anyone had to do was buy and hold and the market went up automatically.

    Not to worry. As soon as the market bottoms and the winter is over, the bears will emerge from hibernation and become bulls again. Ahhhhh….positive.

  27. babycondor commented on Aug 15

    Re: “torturing the data long enough to get it to confess to whatever crime you want.”

    Isn’t that what this blog is all about?

    Look, there is inflation. The question is, how much, and is it contained? For now, the FOMC (who decides how inflation is measured–deal with it!) has cast its vote. Inflation is contained, at least until September.

  28. S commented on Aug 15

    FullCarry:

    Dell stock moves up on news of a 4 million recall? Sounds like someone ringing a bell saying the stock has found a bottom? Or maybe its just getting bid up in a rising market. Not sure, but I’m a buyer of Dell today with a tight stop.

    Does this market feel like a submerged beachball to anyone else besides me?

  29. Blissex commented on Aug 15

    «Guys, it’s a market, it goes UP and it goes DOWN. Neither is good or bad, they just are. It would be mindnumbingly simple if all anyone had to do was buy and hold and the market went up automatically.»

    Oh man, but it has been going up and up again for well over 10 years now, and whatever ”corrections” have happened have been short and mild (even the excessively insane NASDAQ valuations of early 2000 have only regressed to merely insane early 1999 ones).

    According to a PIMCO man, this is all thanks to the ”Greenspan Put”:

    http:/WWW.beearly.com/pdfFiles/PIMCO092005.pdf
    http://WWW.PIMCO.com/LeftNav/Late+Breaking+Commentary/FF/2005/FF+September+2005.htm

    «Bluntly put, the Fed has a credibility problem, because the
    markets know – because Mr. Greenspan has taught us! – that the Fed’s asset price bubble policy is asymmetric:
    • Deny that you can see them when they are inflating, tightening against them only if you can justify tightening on the basis of conventional inflation-pressure models and data.
    • Ease vigorously and purposefully when bubbles confirm their existence by blowing up.»

    And let me add: the denying extends to conventional ”inflation” too. Greenspan has essentially outlawed the down half of the market cycle, with fantastic effects:

    «[6] I attempted to put a value on the Greenspan Put in the February 2000 Fed Focus, “Me and Morgan le Fay”, writing that without the Put, the P/E for the S+P 500 should be 18, not 32.»

    Nearly doubling the P/E across the S&P500, and in 2000? The power of monetary policy is truly impressive. I hope Citigroup, Merrill or Goldman build Greenspan a statue of solid gold :-).

    Now the big question is whether Bernanke too is a generous fellow that gives the whole market a free put, and the merry times can continue for a while yet.

    Greenspan could afford his generous present because any wider inflationary impact would end up in China and India, as after 1995 their integration in the USA trading system added a colossal amount of spare capacity to it; and indeed China and India have been exporting deflation and low priced services and good, and importing inflation, jobs and capital from the USA.

    But how much spare capacity is left in China and India and in commodity exporting countries? Not a whole lot it would seem. So Bernanke’s Put is going to have a bigger USA cost.

    Now Berkanke has to decide what to do. He can decide to do a Volcker Squeeze or a to do a Greenspan Put.

    Is Bernanke a good patriot and Republican? :-)

    With a difficult war or two going on, does he feel it is right to undermine the financing of the war and civilian morale, and raise the risk of Democrats getting control of the House with a squeeze?

    We don’t know yet.

  30. ss commented on Aug 15

    Look…when Greenspan flooded the market with liquidity (1% rates) it was on the backs of what he described as a potential Japan type deflation/depression…the CP market was frozen, and a credit crash was in the crosshairs. As soon as he SAID this the market reacted, and rates plunged, because he got it. Suddenly GM was able to sell bonds, and the corporate market bottomed, and corporate spreads steadily narrowed…all the way to the very healthy balance sheets we see now. Consumers were also able to refinance their balance sheets. Many find fault with Greenspan. I don’t…I see a guy who changed the way to appproach challenges in Capital market crises, and wasn’t afraid to act.

    Bernanke, has already caught flack (some deserved thanks to Barteromo), yet I think he gets it too…he’s willing to “look out” and respect the fact that he has quintupled rates…and it takes TIME to feel those effects. (When was the last time we quintupled rateds, btw?!). He has learned from both Volker and Greenspan, and I believe that he’s humble enough to turn on a dime if he’s wrong. He’s already shown me that he’s changed a bit from his writings on targeting inflation.

    Quite a hornets nest I stirred up, eh?

  31. BDG123 commented on Aug 15

    Not really. I still think you are stupid. I AM JOKING. I couldn’t resist as you left yourself wide open. You have to understand, my sense of humor is like that of Lewis Black. Very cynical and derogatory.

    I don’t think you are stupid, you are just presenting your views which are stupid. lol. I AM JOKING!

  32. tjofpa commented on Aug 15

    4 highest vol DJIA stocks on my board all have RED candles. XOM, WMT, GE, HD

    BELLY UP TO THE BAR. We’ll see what happens after tomorrow’s low CPI Gap higher.

    On a positive note; Go NVDA

  33. D_Rumsfeld commented on Aug 15

    The real question is whether we believe the numbers published to the degree of precision published right now.
    Barry alluded to the problem that we’re taking a weighted composite average of some 80,000 thousand metrics and then running it through a bunch of models with unknown uncertainties to get a single number for “inflation”.

    Maybe if they put some error bars around the CPI number, we could see if providing three decimal places is justified. It is likely that the CPI provided is really plus or minus 25% or so, meaning that 2 significant digits is the way to go.

  34. diva commented on Aug 16

    You guys are SOOO lost (in those dang trees – HA!)
    Gold is a bit over its non-inflationary indication level of $350 – 400/oz = inflation (especially with all of the volatility gold has been having)
    Yep: the inflation is real
    Yep: inflation will continue UP (even as ‘measured’ by the dingbat govt folks and their silly lagging stats)
    1, 2, 3 decimal places = a great BIG joke
    Darda is a buddy of mine, but he has been wrong all along about the Fed raising rates. The Fed has been raising rates for months (years) now…… and has only managed to increase inflation. Ya gotta luv it!
    (or weep)
    Careful out there….. cause those silly lagging stats are gonna continue UP!
    (and no ‘slower economy’ will make a diddly squat bit of difference). I have no idea what Bernanke is smoking.
    Doesn’t every single intelligent person on the planet finally understand that inflation is always and ONLY evidence of poor monetary policy???
    I thought everyone understood that by now.
    I really find it hard to believe that everyone in the mass media is still ignorant of that very basic fact, but one would sure be convinced of their ignorance if you read their stuff.
    Unbelievable

  35. Measure for Measure commented on Aug 16

    ———- An entire wave of M&A and vertical integration in several industries is underway due to soaring commodity prices, and yet, we continue to hear from officials how contained inflation is. Once you consider the absurdity of that, you will realize why 3 decimal accuracy is a noble effort in a wasted cause.

    Why is that absurd? Copper prices rise, apparel prices fall. Price indices are weighted on the basis of final goods demand, which means that clothing matters more than raw metals. Heck, we’d expect big relative price changes in a market economy: that’s how production adjusts to scarcity in one part of the economy or the other.

    Maybe you’re saying that metals et al are leading indicators of inflation. That may be so, but I’d like to see some evidence. (Eg, which predicts CPI changes better, lagged PPI changes or lagged CPI changes? I’m guessing the latter. )

    Tangent:
    Actually though, if I was a big inflation hawk, I’d be turning my attention to the declining dollar.

  36. Tom commented on Sep 14

    Are soft commodities more closely linked to inflation and thus a good hedge?

Posted Under