Thursday, February 23, 2006 |
Overestimating The Fog
But it's the others in the BPro dogpile that have worked up the hackles of Matt Welch and others around the Halosphere. Take this Griddle interview with Rob Neyer, where Neyer once more dismisses the Halos, saying, "I don't find the AL West at all difficult; on paper the A's are easily the best team in that division, and they might be the best team in the major leagues." Or, how about ESPN's power rankings (good luck if you're running Firefox on that one, by the way), which rate the Angels' offseason a tepid 11th overall in the majors?
I bring this up in part because it's part and parcel of a theme that Matt Welch brought up in his Hardball Times Annual 2006 essay, "Getting With The Program". Matt argues that the sabermetricists have done themselves no favors by being so forlornly, consistently, pig-headedly wrong in their predictions, year in and year out, of the A's taking the division, league, and even World Series title.
In 2002, the year the Anaheim Angels would go on to win the World Series after racking up the best Pythagorean win-loss record in baseball, each and every one of the 11 Prospectus writers came to the exact same preseason conclusion: the team from Orange County would finish dead last.Now, forget that the "productive out" correlates poorly to runs scored (something Larry Mahnken blasted in Hardball Times two years ago). It isn't even really something the Angels care about per se, as evidenced in Jeff Angus' Management By Baseball interview with Mike Scioscia last year, when Scioscia said that the two most important stats they care about are hitting with RISP and hitting with RISP and two out (RISP2) -- which is to say, they care about hitting in the clutch, and doing so in extremis.Such are the many joys and delicious ironies of being both an avid Angels fan and consumer of sabermetric analysis. Year after year, Mike Scioscia's boys and their distasteful "productive outs" are picked to stumble; year after year, the widely scrutinized moves of Oakland's Billy Beane are picked to pay off with division titles, MVP Awards, even World Series rings.
The "Beaniacs", as some would call them, have for years said there's no such thing as clutch hitting as a statistically verifiable skill, or maybe more accurately, if there is, it requires a phalanx of statistical caveats so deep even an insurance company actuary couldn't find it. This got started by the research of Dick Cramer, and while there has been endless research on the topic, the distance between particular players -- Gary Sheffield, for instance, whom some say has this skill -- and the general mass of hitters is still quite great, and so to my knowledge, "hitting in the clutch" as a skill remains as solid as a mirage.
Or perhaps I should say, it remains as solid as the fog Bill James recently wrote about. In a thumb-in-the-nose at his most avid followers, a piece he wrote for the Baseball Research Journal entitled "Underestimating The Fog", James argues that Dick Cramer's discovery that clutch hitting is a chimera is as accurate as Columbus' claim to have discovered a new route to India:
Cramer was using random data as proof of nothingness and I did the same, many times, and many other people also have done the same. But I'm saying now that's not right; random data proves nothing and it cannot be used as proof of nothingness.This is a very disturbing paragraph. He may be right about that, but I see no functional difference between the two. James here suggests that there may be clutch hitting, but we cannot yet prove it. Well, I reply, so what? We also cannot mathematically prove the existance of God, mermaids, and unicorns; does this now mean we get to (or have to) believe in all three? James, who is unquestionably a smart guy, here asks his audience to prove a negative, something that is logically impossible.Why? Because whenever you do a study, if your study completely fails, you will get random data. Therefore, when you get random data, all you may conclude is that your study has failed. Cramer's study may have failed to identify clutch hitters because clutch hitters don't exist as he concluded or it may have failed to identify clutch hitters because the method doesn't work as I now believe. We don't know. All we can say is that the study has failed.
Worse, in pushing for a distinction between "the study has failed" and "there is no statistical proof of X as a skill" (where X may be "clutch hitting" or any of a dozen other similar issues within baseball), James has opened the door to all the snake oil peddlers like Buster Olney to tout their wares. If Nate Silver, Keith Woolner, Keith Davenport and the like over at Baseball Prospectus have, from time to time, been the victim of overzealous jacket blurbs (PECOTA, "deadly accurate"? What, it's a weapon of mass instruction?), at the same time at least they can say whether a player's future performance fit within the error bands the projection gave. It's a flawed system, to be sure, but player projection is notoriously difficult. But to surrender some knowledge for unverifiable speculation -- and make no mistake, in the absence of proof, that is what the notion of RISP- and RISP2-hitting-as-skills are -- amounts to creationism, the "intelligent design" of baseball.
Update 5/21/2014: Corrected the link for the Bill James essay, no longer available as a PDF.
: 2/23/2006 12:24:00 AM2. Agree w/ you about the fallibility of the RISP(1 & 2) data - Hopefully Joe Maddon took those methods to TB and Dino Ebell will guide Scioscia toward the light...
: 2/23/2006 01:41:00 AM
James seems to be saying that there is a lot of unknown unaccounted for in current baseball statistics (unquestionably true) and that just because some people have shown using certain methodologies that you get nothing new, that does not mean that using other methodologies not yet figured out nothing new will be learned.
As a matter of fact, this is something I have hated about the SABR ideas since I first heard about them: they are always so sure everything they do is perfect and if it doesn't conform to current ideas, it is wrong. Well, keeping such a closed mind about these things will never allow the field to move forward properly. Go ahead and look at clutch hitting data with skepticism, but don't believe it can never be proven.
Neither does clutch hitting claim to explain the whole of baseball. Both are willful rejections of well-known and tested models.
The Angels didn't spend this offseason looking for someone to make productive outs; they knew they needed more offense, especially power but also OBP, from 1B, DH & CF. Even though they didn't acquire anybody, they managed to make improvements in this regard.
This is probably the most important point here: the Angels may say they care mostly about RISP and RISP2 hitting, but they're not acting much on it.
Also, Creationism itself is completely compatible with all scientific thoeries currently. Biblical creationism (the stuff that Carl Everett believes in) is not. Not to get too far off topic, but this is an issue that many Evolutionists (those that believe in the religion of evolution) and Intelligent Designists don't seem to understand. Creationism does not have anything to do with evolution. Saying you believe one does not exclude you from being able to believe the other. One explains the origin of the universe (and life) and the other describes how the current species have come to exist from original life. Evolution cannot possibly explain the origin of life or the universe itself, while Creation explains ONLY those things.
Basically, Creationism is not a willful rejection of a tested model: it is an explanation for a model that cannot possibly be tested at this time. And if you want to say that it doesn't change your argument (ie clutch hitting still cannot be tested at this time), then either accept that it has an unknowable impact on the game (it is one of the intangibles that aren't yet fully accounted for) or try to come up with another method that can attempt to measure it. If the latter fails again, you try a new method. You cannot say UNEQUIVOCALLY that something is impossible. If it were easy to find equations to explain everything, they would have already found the Unified Equation that explains everything in the universe.
: 2/23/2006 09:28:00 AM... in order to be considered properly by scientists for research, a hypothesis
must make testable, repeatable predictions. As far as my research has
discovered, Intelligent Design doesn't make any predictions, simply declarations. If it doesn't make testable predictions, it's nothing more than an idea (aka a hypothesis) and definitely doesn't elevate to the level of "theory", a term with a very rigorous definition in the scientific
community. From this standpoint, Intelligent Design and the Flying Spaghetti Monster are on equal footing in the view of science and neither should be taught in a science course worthy of the name.
Josh, to explain Rob's lack of differentiation, I grew up in Arkansas at a time when folks who believed in "Intelligent Design" called themselves "Creationists." The current theory is simply a rehash of the stuff they tried to get through the state legislature in the 70's and 80's. So around here, one term equals the other, just different decades. We figure folks who believe that God created everything, but maybe the Bible isn't literal, are just called "Christians."
As for clutch hitting, it's like porn. I can't define it, but I know it when I see it. ;-) (And it's fun, too.)
: 2/23/2006 12:45:00 PMAs far as Creationism being a testable hypothesis and scientific theory, I never said that. But just because at the moment there is no testable hypothesis does not mean that eventually there will not be methods to do so. What outer space is made up of was not a testable hypothesis for a long time, but that doesn't mean it wasn't a sceintific question that couldn't eventually be tested.
The question of Creationism is one similar to the clutch hitting one: just because you can't think of how it can eventually be tested, doesn't mean it is untestable or that it doesn't exist. All it means is that we do not know at this time and we cannot test it yet. (And if you are wondering how you could possibly test Creationism, well, you could invent time travel and go to the beginning of time and observe, but that's not yet a method available to us).
And as for the cold fusion analogy: just because we don't know if cold fusion can exist or if we will ever figure out how to make it work does not mean that it will forever be impossible and that we shouldn't keep trying to figure it out.
But how could there ever be such? What testable hypothesis do you arrange to divine an all-knowing creator? And then, how was that creator created? It's unscientific silliness.
And as for the cold fusion analogy: just because we don't know if cold fusion can exist or if we will ever figure out how to make it work does not mean that it will forever be impossible and that we shouldn't keep trying to figure it out.
But there's a huge difference between investigating cold fusion and planning on driving to Phoenix in a car with a cold fusion engine in it. That latter is the position the Angels have placed themselves in.
And just because you haven't figured out how clutch hitting works doesn't mean that someone somewhere hasn't. You may think that that person would want to publish said results, but if freeing that information up will hurt you, then you won't do it. Just because you don't have the data in front of you saying that RISP matters does not mean taht someone else hasn't figured it out and just doesn't want to share it. With the cold fusion analogy: just because you don't know how to create cold fusion and you can't see it, it doesn't mean that someone somewhere isn't already driving to Phoenix with a cold fusion engine.
I have my own belief about clutch hitting, which is that it involves some unique skills, but cannot be quantified. Basically, I believe that players react differently to "clutch" situations based upon their personality, experiences, and everything that makes us human. Some have the ability to focus better than others, and better than they might be able to in non-clutch situations.
Thus, it's not quite accurate to say that there are clutch hitters, but rather that some players might have a particular skill set that is more likely to produce success under specific conditions. E.g., a good-contact, fastball hitter who tends to hit the ball where the fielders aren't when there are runners on base.
: 2/23/2006 04:36:00 PMNewer› ‹Older
 |
Post a Comment
Note: Only a member of this blog may post a comment.