Much of investing relates to mathematics and the application of statistics. Markets are statistical data generating machines, and that data can be sliced and diced in a myriad of ways. We always pay close attention whenever we see an interesting application — or misapplication — of quantitative data that may be instructive or applicable to investing.
So I was particularly intrigued by a study in today’s NYTime’s OP-ED page that purported to look at the impact of steroids on the performance of Baseball players, based on the Mitchell Report. They asked the question: "In a complex team sport like baseball, do the drugs make a difference sufficient to be detected in the players’ performance records?"
Their conclusion? The authors of More Juice, Less Punch
found that Steroids, Human Growth Hormone and the like do not have a net benefit to major league players. Based on their review of pre- and post- steroidal usage, the overall impact on players stats was de minimus.
I remain unconvinced.
Ever since Freakonomics became a runaway economics best seller, there seems to be increasing attempts by "rogue economists" and others to discover the hidden, counter-intuitive side of everything. This column seems to be of that genre. They would have been better served if they were channeling the statistical approach of Moneyball, instead.
When you come across broad attempts to explain complex systems, your inner mathematician should always be concerned that the methodology employed is sound, any initial assumptions made are justified, and the analytical steps taken are well supported.
In the present case, I suspect they are not. Consider the following statistical and analytical issues:
1. The authors of the Times Op-Ed looked at 48 batters and 23 pitchers named in the Mitchell Report; This may be too small a sample to draw any valid conclusion.
2. For pitchers, they studied ERA. Is the main impact pitching advantage of Juice the impact on ERA? That stat is a function of many things — intelligence, pitch selection, opposing batter research, etc. — not just physical power.
The authors ignored many other stats that might be more telling as to the impact of ‘roids: Consider strike outs, average pitch speed, average number of pitches thrown per game, total games pitched. These data points would have been quite instructive as to the impact of performance enhancing drugs (PED) on issues such as strength and durability, even injury recovery.
3. For Hitters, they examined batting averages, home runs and slugging percentages. The same durability issues were overlooked — games played and missed, total at bats, swings with ball contact, distance traveled of hit balls, etc.
And what about speed — why not consider stolen bases? We know lots of runners and cyclers have been accused of using PEDs — isn’t this a valid data point to consider?
4. Dates: What were the Before & After dates? It appears that by drawing the line at the date of accusation, lots of PED usage will have taken place in the BEFORE data set. If the performance gains of the AFTER group, began in actuality during the BEFORE, the entire statistical conclusion becomes indeterminate.
5. No control group: All players begin to show statistical deterioration as they age, get worn down, injured, etc. How can we tell what their stats would have been looked had they not been juiced?
Rather than comparing pre-accusation and post-accusation stats,
perhaps a better comparison would have been to look at the group of players who used PEDs versus those who didn’t as their careers wound down. How do the two groups compare in their mid 30s? Late 30s? Early 40s?
Note that even this grouping may be flawed, because of the self-selection factor of those who chose to use the drugs in the first place (more injury prone, weaker, slower, etc).
6. False Accusations: Are any of the players accused in the Mitchell Report not guilty of using PEDs? I have no idea, but its a valid possibility. How might their false positives impact the author’s conclusions regarding stats?
I don’t know what the total impact of Steroids and Human Growth Hormone were on baseball player’s performance — but based upon the above, neither do Professors Jonathan Cole and Stephan Stigler.
~~~
One last thought: Why hasn’t Baseball Commissioner Bud Selig resigned or been fired?
Shouldn’t he — like Merrill Lynch’s O’Neal and Citigroup’s Prince — fall on his sword? This happened on his watch, and he apparently was asleep at the wheel. For this gross incompetency, Selig should be tossed aside like a used syringe.
>
Source:
More Juice, Less Punch
JONATHAN R. COLE and STEPHEN M. STIGLER
NYT, December 22, 2007
http://www.nytimes.com/2007/12/22/opinion/22cole.html
INDEPENDENT INVESTIGATION INTO THE ILLEGAL USE OF STEROIDS AND OTHER
PERFORMANCE ENHANCING SUBSTANCES BY PLAYERS IN MAJOR LEAGUE BASEBALL
GEORGE J. MITCHELL
DLA PIPER US LLP, December 13, 2007
http://assets.espn.go.com/media/pdf/071213/mitchell_report.pdf
Barry – thanks for this. The recent spate of longer and deeper posts is very helpful and much appreciated.
The book to read here is of course Moneyball for the background on SABRMetrics but the guys over at Baseball Prospectus would have been the folks to collaborate with as they could have had the huge database and toolkit to investigate the problem in depth. But of course no right thinking academic would ask an amateur to help frame and investigate such a problem :).
Nontheless the use of math and analysis in the real world is powerful but the catch, is as you imply, the intitial framing of the problem and the translation of that problem into the proper “model”. Back in the day we called that model specification error.
If you’d like to see multiple interesting examples where the math is real, the problems are real but the drama is well-done try NUMB3RS. Outstanding and on-line now :)
~~~
BR: Thanks for the kind words — As I read the column, all I could think was they were going for a Freakonomics spin, when they should have been thinking Moneyball.
I should shoot this over to Michael Lewis . . .
“For this gross incompetency, Selig should be tossed aside like a used syringe.”
That’s a good line.
It could be that the sport sampling is not relevant enough,when selecting base ball only as a game.
From my personal studies maths as applied to the markets are quiet useful when the markets are « gamed »
Great post. I’m a cricket fan {and we have our scandals ( match fixes, and even a purported murder)} so can’t comment on the specifics. You should publish a similar sceptical post on stock market stats and stuff like the “Santa Claus” rally, “January effect”.
When you tackle the predictive power of analyst upgrades and downgrades, be aware of the post dating scandal there – as discussed in the WSJ,Barrons the analyst went and modified Thompson’s I/B/E/S database. A recent article is at:
http://dealbook.blogs.nytimes.com/2007/03/07/are-some-stock-analysts-rewriting-history/
The money quote for me is:
”
Mr. [Daniel Gross writes. But “Wall Street executives—stock analysts among them — have shown that there’s virtually nothing they won’t do, and nobody they won’t corrupt, to advance their own careers and portfolios.”
”
-K
Baseball players normally peak in their late 20s.
Which hitters have had the most improbable late-career success?
They’re nearly all from recent years.
Remember all those overachieving, underpriced players in Moneyball?
All juiced.
Billy Beane: “Power is something that can be acquired.”
Now we know how.
One of the main advantages of taking human growth hormone is a faster recovery time. For pitchers this would be like taking an extra day rest without having to lose their ‘edge’. I can’t image a comparison of stats could reflect this effect if suddenly a 23 year old and a 40 year old recover at the same rate.
I did a study before the drug problems of when in their careers baseball players’s performance peaks. The best measure of how well someone hits is to add (or multiply) their on-base percentage and their slugging average.
I looked at these stats for most of the well known Hall of Famers, about fifty. For these players about 92% of them had their best years between the ages of 26 and 32. There were only a few outliers and those by at most two years.
So, some of these guys, Barry Bonds most notably, having their best years in their late thirties had to be on the juice and thus their performances had been enhanced regardless of what the authorative NYT prints.
I always thought drugs were the choice of criminals, addicts and rock musicians alike. Funny how sports has gotten so superficial.
Fascinating comments – thanks guys. The one about applying this to market, stock and, dare I say it, company analysis is really a critical point. Let me reiterate “model spec” error – no finance guy I’ve known or worked with understands how business works.
BR – try it. Better yet also read the postscript to the 2nd ed trade paperback where he talks about the widespread attacks on the industry. Let us know if he responds :) !
Based on their review of pre- and post- steroidal usage, the overall impact on players stats was de minimus.
Just a quibble, but it’s de minimis.
The preposition de takes the ablative.
Yeah, I know, it’s a dead language, but you don’t have to rub it in…..
Can’t help thinking that the old ‘reserve clause’ in player contracts served a purpose. Crazy contracts and the dumbing down of the average fan has probably led to incentivizing ‘bad behavior’ as much as anything and the owners are complicit. Gotta put ‘meat in the seats’ and the easiest way has been long ball. Pitchers need to defend themselves and juiced up as well. Baseball, where individual statistics are everything, has been damaged since stats can no longer be used to judge players of one generation with another. The law on unintended consequences is at work here.
And to think, It used to be a game. Unfortunately (or not) I am old enough to remember when players in all sports had an off season job to make ends meet.
Unusual performances given players’ ages. This is a better statistical analysis.
I guess if the juice doesn’t help the players, the bribes that guys like that get paid to write blatant falsehoods don’t impact their opinions, either? Maybe they can write a piece on how all op ed’s are completely unbiased and that the writer’s personal finances have never been impacted by their opinions.
They are either stupid (unlikely), dishonest, or really never played a sport. Try going up against a guy who is juiced when you aren’t and then writing that column.
I look forward to their opinion on why doping and juice don’t affect track and field. Or how being a roid monster has no positive impact on your ability to play outside linebacker in the NFL.
Oooo…I think I hear santa coming down my chimney! I believe!
400 ft homers on a wrist swing were not unusual for Maguire at his peak. Look at the swings of the juiced up modern players vs a guy like Mantle. Night and day. Some of these guys have no business carrying around the kind of numbers they do.
Haha! Concluding that PEDs have no impact on performance is like conlcuding that wacky mortgages had no impact on the housing bubble.
Here’s the whole key on why the analysis is meaningless “…of PED usage will have taken place in the BEFORE data set…”
PEDs have been pervasive in baseball since the 70s. Just that then it was a few lines to improve reaction time.
Swinging a bat to hit a ball is a tough skill to master. And taking steroids doesn’t give you skill. I saw plenty of bloated “big guys” when I played baseball who could not hit a damn thing. That said, if you are a Barry Bonds, and you have that skill already, there is no doubt that steroids will help your hitting. To say otherwise is foolish.
It’s a game of inches.
Barry Bonds, Mark McGwire and Sammy Sosa would have hit a lot more 320 ft warning track outs vs 340 ft homers in the late 90’s without PED’s. The average Schmo on ‘Roids would have converted some 290 ft outs to 260 ft outs with no material benefit to the batting average. My estimate.
Likewise a Clemens 95 mph fastball may strike a guy out, but the same 40 year old arm throwing a non-enhanced 91 mph is just another batting practice pitcher.
Perhaps lots of guys were using PED’s but only the stars’ stats were converted to superstars’ stats and thus noticed by us mortals.
What if Griffey or Frank Thomas had used PED’s? or Greg Maddox, etc? To say that the effect on the sport was “de minimis” is a huge leap and not even worthy of consideration.
Selig should not only be fired– he should be beaten while naked with a six foot wet noodle by a cross-dressing Larry Craig and pummeled with a thousand used sanitary napkins by Rosie dressed in her League of Their Own baseball uniform at Madison Square Garden with prime time time coverage on every network.
(BTW, Davis X. Machina is the best name I’ve ever seen! and good call on the Latin, you are my hero.)
re: points 2 and 3, the modern hardcore baseball statheads have generated some newer metrics (OPS, xFIP) you’d never find on the back of a baseball card. So these are definitely significant points to be made.
For instance, ERA is objectively a poor measurement regarding a pitcher’s ability because of all the external factors (luck, team defense, weather, ballpark dimensions, etc etc).
Some of the most interesting work going on in the whole baseball-statistical field is trying to come up with solid measurements and quantifications for pitching and defensive ability. These are very hard problems, but the rewards of being able to scout talent with a greater success rate than the traditional “eyeball” or imperfect ERA-type stat metrics are obvious.
This a classic quant-idiot savant trap that Nassim Taleb talks about. Simple logic is thrown out in favor of highly complex methods.
Yes, please fire Bud Selig…he must belong to the same country club as Easy Al. From steroids to the bad MLB Extra Innings exclusive deal with DirecTV, Selig has proven incompetent.
As Vito pointed out, one of the biggest benefits of HGH is a faster recovery time, including time to recover from injuries. It might be interested to note the number of days spent on the DL vs. players of similar age. (But this is probably too small a sample to generate any significant stats.)
This seems more inline with the Wages of Wins tripe than Moneyball. The WoW guys are Economists who ignored almost all of the basketball metrics community when they came up with their worthless metric. Now when that community asks questions about their methods the WoW guys don’t engage it because its not ‘Peer reviewed’. The irony there is that the book they put out isn’t peer reviewed either.
Sorry for the rant, but economists overreaching has become a pet peeve of mine.
I have always said that it is the ball that was juiced. The stats just took a jump overnight from 1993 to 1994 when baseballs reputation took a big hit from the strike.
Steroids use should have led to a more gradual increase since it is unlikely all players started using at the same time and reached maximal benefit at the same time.
Not saying steroids were not part of players being stronger, but they had to spend time in the gym, and while some players benefited from more durability, there were an equal number who probably shortened their careers with tendon and ligament ruptures which plague steroid users.
The hypocrisy among the sports writers and some players who were in the locker rooms daily and today are expressing their outrage is pathetic, since none of them had the courage to break the story, and they surely knew what was going on. Of course, if they wrote about it or talked about it (players) it was career suicide. From the players view, taking the moral road when others were using may also have been career suicide.
Baseball facilitated the usage of steroids
by not doing anything to stop it, players did not complain since salaries were escalating, and both saw steroids usage as beneficial, as was the juiced ball which MLB wants to keep secret since this would tarnish the stats of all players, and not just the few juicers who got caught.
Does anyone know of an analysis of exactly what the players who used PED’s did wrong? What baseball regulations were broke and over what years? What federal laws were broken if/any and over what years? Is it just an ethical issue?
Thanks
What a great post BR.
Your analysis is spot on and it gives great insight into why you are successful in the world of finance.
I find no fault with your analysis, but in the end, I can’t help thinking the NYT conclusion is correct.
If an equal portion of all players (by position) was on the juice, then the effects would offset. Better power hitters would be offset by better pitchers. The net effect on statistics might very well be null, despite the flaws in the NYT study.
Excellent post!
And Selig’s job is safe because the MLB doesn’t have public shareholders or a public share price to worry about. That and he can easily hide behind the players.
I wonder if some of the writers will boycott Selig the first time he comes up for the HOF.
Great blog and article! Now you are getting to something much more senstive than our money, baseball. The “unusual performance for age” theory does not hold water, IMO. Too many exceptions exist for the statistical comparison to be valid. Some players are able to adapt their aging bodies for continued success. Some through drug use (steroid stud Bonds), but others through the knuckleball (Charlie Hough) or superb mechanics and conditioning (Nolan Ryan). I know all the “I see everything in black and white” statisticians will not like this, but all the average statistic tells us is what outcome is likely. It has no relevance for individual performance. A better measure for steroid might be an explanation of muscle mass being added in your 4th and 5th decade of life!
As for Selig, he is only around because he is a brown-noser doing exactly what the powerful owner want. Once he quits that, he like any other commish, he is gone. Selig is much more like a pusher than a baseball commissioner.
Thanks for getting into this. I find Freakonomics really annoying.
I haven’t searched your blog for Malcolm Gladwell but I hope you’ll consider poking some holes in that hot air balloon as well.
Disclosure: I used to work for Bud Selig.
Mr. Selig was “chosen” as commissioner by the owners because they couldn’t come up with a traditional commissioner, remember? He is as close to a boy scout as exists in the owners club. Trust me, they could have done worse.
So, he could/should have done better, but he basically does the bidding of the owners.
A perfect example of a “non-independent” board of directors.