Go straight to the Projection Update!
Go to the previous projection evaluation update
It's June 22nd and we're rapidly approaching the halfway point of the season. As I did at the beginning of the month, I thought it would be kind of fun to see how my projections have done so far, compared to some of the more high profile names out there. I will issue the words of caution here that it is just June, and lots can happen to the stats of various players.
I also decided to post some more projections of players who have, during the season, managed to obtain a position where they are currently getting some playing time. This happens usually when someone gets injured, but sometimes they just win the job. On the same page, I've also finally added the projections for the hitters lucky enough to play their home games in Coors. Yes, I now have the projections up for all of the Colorado regulars. These projections do not use any of the current season performance (minors or majors) as a basis. They appear the same as if they were posted before the season started. They can all be found here.
I'll also add that I won't use any of these projections when evaluating how well the projections have done (neither now, nor at the end of the season). Obviously if they're not up by the time the season is underway, it's hard to fairly include them in the analysis.
As of games of June 21st, I decided I would take a quick look at how my projections are doing compared to others. I'm going to do a quick comparison of mine, with those of Stats, Inc. (as published in their yearly handbook), CBS Sportsline and Diamond Mind Baseball (A computer simulation game). In this update, I've now added the projections from Baseball Prospectus. I am simply going to compare OPS at this point (though when the season's over, I'll get much more in depth and maybe use other sources as well).
What I did was keep track of the projected OPS for each player from each source, and use the least squares method to obtain a correlation coefficient between the projected OPS for each source and the actual current OPS. (For all of you who don't speak math: I looked at what was projected and used a valid statistical technique to compare it with what has happened). I have decided that from here until the end of the season, I'll only do the comparison for the top 170 players in At Bats plus walks. This will make it easier to compare how much the projections improve as the sample increases. The cutoff happened to hit right at 200 At Bats plus Walks. The results:
| Source | Corr. Coeff. | Average Error |
|---|---|---|
| Vörös | 0.681 | 0.087 |
| Stats Inc. | 0.659 | 0.091 |
| CBS Sportsline | 0.613 | 0.091 |
| Diamond Mind Baseball | 0.620 | 0.091 |
| Baseball Prospectus | 0.660 | 0.087 |
The first thing I'd like to state is that I think all of these are pretty good, and better than I'd thought we'd get this early in the season. Now, I'll state very clearly here that it's early, and that those aren't very big differences (they shouldn't be as the projections were all reasonably close to each other). I'll also state that CBS Sportsline's projections pretty clearly look to be just simple estimations based on fairly recent performance without taking into account minor league numbers. That fact makes it pretty difficult for their's to compete (and frankly I don't think they much care about their projections anyway). I'll also note that while CBS Sportsline is doing almost as well as DMB, some of their more dubious projections are just below the cutoff currently (Homer Bush, J.D. Drew and Mark Quinn). Including the next 10 players would drop their correlation quite a bit.
But I will say that even at this early stage, I'm very pleased. I know both Stats, Inc., Baseball Prospectus and DMB put a whole bunch of effort into their projections, and I was pretty sure that for this first year they'd spank mine pretty hard. That still could very well happen, but for the time being I'm extremely happy to simply be holding my own with them.
It's worth noting that the average projected OPS of the group for each source was .814 (me), .805 (Stats), .819 (CBS), .817 (DMB) and .817 (BP). Whereas the actual average OPS of the group is .850. I'm not sure whether this is due to increased offense this year, or shaping, or whatever. If I had to guess, I'd say it's partially increased offense, and partially that there's a minor bias due to the fact that hitting well early on probably increases playing time.
As the year goes on, I'll check back here and make more comments. Thanks for reading along.
Vörös McCracken