During my first couple weeks writing for FanDuel, I discussed the most common contextual factor that fantasy players consider: park effects. As I discussed then, park effects are extremely important for fantasy players – daily fantasy players in particular – to consider when selecting their teams. They are not, however, the only contextual factor that’s worth considering. After all, there are plenty of variables that distinguish one baseball game from another. Throughout the season, I’ll be going over a few more of these contextual factors – some you may have considered before, others maybe not. This week, I’m going to be talking about umpires. I’ll be going over the methodology I use to evaluate umpires and then we’ll go over some of the results of the system.
While the umpire is almost always overlooked by fantasy players, he is a part of the game (and an important one at that). Further, the quality of umpire differs from game-to-game on any given day. Umpires are human beings and, therefore, are prone to differences in judgment. Each umpire has his own unique strike zone and, as such, results for both the pitcher and the hitter will be different given identical conditions aside from a different umpire.
What do umpires affect? Because an umpire’s effect on a game is limited to the strike zone he calls, the only things he has any meaningful control over are the events that are directly related to these strike zone calls: strikeouts and walks. Strikeouts are a category unto themselves in most all 5×5 or larger roto leagues and are one of just four categories in FanDuel’sMLB 35k Salary Cap format. While walks aren’t a category in FanDuel games, they do have an effect on runs allowed, which is a category, and by extension wins.
How do we capture the effects of umpires? To capture the effects of umpires, I decided it would be best to create a full projection system dedicated strictly to umpires. I began this work shortly after my work on CAPS (Context Adjusted Pitching Statistics) at The Hardball Times a couple years ago and finally came up with a finished product that I liked this past off-season.
Projection systems are a very long and complicated process that would need a series of articles just to scrape the surface of them, but I’ll gloss over the important points so you have an idea what went into these umpire projections.
In almost any true baseball projection system you see, there will be three main components: regression to the mean, multiple season’s worth of data and weighting of that data, and an age curve.
Regression to the mean Regression to the mean is a complicated topic if you haven’t heard of it before, but there’s a good primer on it here. The basics of regression to the mean state that because we’re working with a finite amount of data, that data doesn’t represent with 100% accuracy the true ability of the umpire (or pitcher or hitter or whoever).
For example, a pitcher may post a 1.00 ERA in one game and a 9.00 in the next. That 1.00 ERA after the first game is only a sample of the pitcher’s talent – we can’t assume that he’ll always post a 1.00 ERA based on that one data point. The more data we get, the more certain we can be about the player’s talent, but because we will never have an infinite amount of data, there will always be some form of random variation present.
Regression to the mean accounts for this in my system by taking the umpire’s observed data and combining it with league average data proportionately based upon the size of our sample. The more batters the umpire has seen, the more his observed data counts in the system.
Also, because different stats “stabilize” at different rates, I’ve used Russell Carleton’s split-half correlation method, Harry Pavlidis’s continuation of the work, and Tom Tango‘s suggestion to test the correlations at different intervals to determine how much regression is needed for each stat.
Weighting of past seasons This one is pretty easy to understand. In general, the more data we have, the better, especially when stats take a little while to stabilize like they do for umpires. I ran an analysis to see the optimal number of years of data and what proportion of each to use. After all, last year is going to be more meaningful than the year before since it is more recent, and eventually we start to lose accuracy by using data that is too old, so this all has to be accounted for.
Aging An umpire’s age doesn’t matter much since his skills aren’t physical in the way that a player’s are (aside from maybe eyesight), but I hypothesized that an umpire’s experience would matter more than age. From what I’ve looked at so far, though, I haven’t been able to find much of an experience curve for umpires. I’m going to continue looking into some things, but for now my system doesn’t have any age or experience effects incorporated.
Park Factors Parks will, of course, affect things, so all numbers have been park adjusted. I also accounted for park effects when figuring out the proper amount of regression and weighting.
Data source The umpire data I used for my system comes from Retrosheet.
The information used here was obtained free of charge from and is copyrighted by Retrosheet. Interested parties may contact Retrosheet at “www.retrosheet.org”.
So how much impact do umpires actually have? So how much of an effect does an umpire have on a player’s statistics? Let’s turn to the system to find out.
I’ve retroactively created umpire projections for the past five seasons, and the largest strikeout projection over that period was Scott Barry projected to inflate strikeouts by 12% in 2007. On the opposite end of the spectrum, Kevin Causey was projected to deflate strikeouts by 12% in 2009.
The effects for walks are even larger. Over the past five seasons, the most extreme projections we see are Derryl Cousins projected to inflate unintentional walks by 17% in 2006 and Doug Eddings projected to deflate unintentional walks by 19% in 2007.
Focusing only on this season, we still see highly significant results with Gary Darling projected to inflate stikeouts by 7% and Bill Hohn projected to deflate stikeouts by 9%. For unintentional walks, we see Paul Schrieber projected to inflate them by 11% and Tony Randazzo projected to deflate them by 11%.
Applying this to FanDuel As crazy as it may seem, actually coming up with these projections is the easy part. The hard part is applying them to actual daily games, mainly because of the absence of necessary data (or at least the difficulty in getting the data in certain situations).
You see, it does us no good to know that Gary Darling is the best umpire for strikeouts this season if we don’t know what game Gary Darling is involved in. My good friend Eriq Gardner at Fantasy Ball Junkie discussed the difficulty in getting umpire data earlier this season.
Eriq said that “MLB is rather careful about revealing umpire assignments much in advance of a game. In fact, legend has it that big-time Vegas bookies actually trail umpires on their travels in order to glean that information advantage.” Crazy, but given how large these effects can be, I can see why they’d do it.
While this isn’t great news for us, it’s not the end of the world. You can usually find out who is umpiring a game ten-to-twenty minutes before the first pitch by checking a site like ESPN or Yahoo! Additionally, this is only really a problem for the first game of a series.
Umpires work in crews, and once the crew for a series is established, it’s easy to figure out who will be calling balls and strikes in subsequent games. Umpires follow a clockwise rotation around the diamond, so whoever is the first base umpire in the first game of a series will be the home plate umpire (the guy we care about) in the second game of the series. Likewise, whoever is at second base during the first game of the series will be at home during the third game of the series.