Proceeds from the ads below will be donated to the Bob Wuesthoff scholarship fund.

Tuesday, January 04, 2005

The Dodgers' Ticking Rule 5 Clock, Meta, And Sabermetric Mal-de-Mer

Tom has some good points here about the Rule 5 clock ticking in 2005 for some of its better draftees, which, translated, means the Dodgers will almost certainly start making some trades. This, I think, comes as no surprise considering the overheated free agency market this year, but I wanted to make a brief comment regarding this bit:
Now, I think Rob McMillin is an excellent blogger, but his analysis of that trade was as offensive to me as a Myrow/Saenz platoon at third base ever could have been to him. I mean, to me the notion of using year-old PECOTA projections as one's primary source of data is amazingly objectionable. That doesn't mean it's wrong, but it does mean that it was very difficult for me to take his arguments seriously. So I got out my torches and rallied the townspeople, looked up Dr. Frankenstein's address on mapquest, and got ready to bring it. But the more I wrote, the less comfortable I felt writing, not because I felt I was wrong but because I sensed that I was looking at the data to find the things that matched up with what I wanted to say, and even though I looked for everything and tried to balance the arguments to evaluate them I still couldn't eliminate the bias, and that was the vary same trait that I had gotten me so riled up about Rob's write-up. There's a line between opinion and analysis, and though I'm not sure how fine that line is I know I was too close to encroachment for my comfort.
While I take the left-handed compliment with all the cold enthusiasm it was delivered, I defend my actions as follows:
  1. PECOTA is what we have to get to VORP, a single number indicating offensive player performance.
  2. Year-old projections, like it or not, are the only complete data we have in many cases; and rather than try to cast an incomplete or constantly changing set of ZiPS projections into a single number (such as EqA or OBP, both of which certainly have their own problems relative to VORP), I'll take the straightforward if, yes, somewhat superannuated data.
That is to say, a good bit of the writing found herein lately attempts to answer the question, "good trade, yes or no?" in a reasonably short timespan. Recognizing that this is constrained by the ability to project player performance -- something others have worked at much longer and harder than I have -- is the first step toward publishing something. I imagine I could cook up my own half-baked player performance projection system, one that might even be a quarter as good as some of the systems developed by others with a geekier bent than myself, thus freeing me from waiting on the geniuses at Baseball Prospectus to crank out a new issue of PECOTA. However, heeding Bill James' cry that the amateurs could contribute best if they depart the floor, I do so and pass on. Moreover, I take to heart another Jamesian passage, quoted by Rich, from the last of his Baseball Abstract series:
James discusses how the relationship with his readers changed over the years from a “virtual love fest” to one in which he was getting “more and more letters that irritate the living hell out of me. People have started assuming that I am a goddamn public utility or something. I get letters from people telling me that I do this well but that I shouldn’t do that and I should do more of that and less of this and try some of the other. If they irritate me enough, I write back “Dear Jackass: I am not your employee. It is not my function to write about what you are interested in. I write about what I am interested in. If you want to read it, read it. If you don’t, don’t. But DON’T TELL ME WHAT TO WRITE ABOUT.”

James says that he only wrote about 30 such letters the last year but was concerned about how many more he would need to write in the future. “I think that whenever a writer finds that he is beginning to dislike his own readers, it’s a very clear sign that he’s heading down the wrong road.”

I should note that I'm far from there now. But -- well, let me say I understand James's dilemma.

I think you need to brush up a bit on these metrics. VORP is very simple:

((runs/PA) - (replacement position runs/PA))*PA

Take your pick of RC formulas - I prefer lwts, that's just me - and plug it in (park adjust, of course). You can take any player's stats from BP to find what BP deems replacement level for his position. This takes a minute or two. You can then create a spreadsheet for this data, and whenever you want to analyze a player, plug in the player's stats. The same is true of EqA, except it's expressed as a ratio normalized to .260.

ZiPS ain't perfect, but it will be pretty close. If you put a player's ZiPS into your spreadsheet, you can get VORP.

Furthermore, VORP is playing time dependent. I think it's pretty difficult to analyze a player based on a number that depends in large part on the roughly arbitrary playing time projections.

But much, much more important is the fact that PECOTA is a system built on the accumulation of data. It can be vaguely useful to check out an old PECOTA projection for the projected trajectory curve of a player's future, although normally that won't differ much from the typical age curve, but the data derived from the player's last season will always be much more useful than year-old PECOTA projections.

I guess my concern is that you made a claim to authority based on PECOTA projections, and I don't think you made it clear that those were last year's PECOTA projections. Somebody who doesn't subscribe to BP wouldn't understand that; someone who did subscribe to BP could just look up the data themselves, so a PECOTA-centric analysis doesn't add a lot of value to them. I guess my point is, if you're not willing to critically analyze your own analysis of a trade, you really shouldn't be comparing the trade to prison rape. Maybe that's just me.
Perhaps what I should have said was "PECOTA is what we have to get to projected VORP".

As to the "VORP is very simple" comment, you of all people should know that the "replacement position runs" term in the equasion above is the tricky bit to calculate. It's like saying, "magic goes here". One beef I've always had with VORP is that the mechanism for determining what, exactly, a replacement player looks like isn't well documented, and I suspect may not match the real world very closely. That dig taken, I still think VORP is probably the best offensive/pitching metric around.

As to whether my analysis of the trade -- as opposed to my earlier, rather emotional response -- was based on old data, well, upon re-reading it, I plead guilty to confusing the reading audience, for I indeed failed to mention that I was using old PECOTA data. Even so, I stand by it; it's what I have that I can readily use.

However -- what I really don't get is this:

>>I guess my point is, if you're not willing to critically analyze your own analysis of a trade, you really shouldn't be comparing the trade to prison rape.<<

You will notice the latter (emotional name-calling) came before the former (actual analysis). I'm not sure how you get me being unwilling to look critically at my own analysis, because I did exactly that. Has my initial reaction to the trade colored your subsequent judgement of what I wrote about it? I think it has, and that's unfortunate.

I'm certainly open to discussing improved methods of using, say, ZiPS projections for analyzing these things, but if you're going to sneer at me in print for making honest mistakes or omissions, or for simply being human, then I'm not really interested in that kind of discourse. For that, I could wander over to the Dodger Fan Forum.

Post a Comment

Newer›  ‹Older
This page is powered by Blogger. Isn't yours?

WWW 6-4-2