Modeling strike zones with neural networks

In the 2016 Hardball Times Annual I wrote about evaluating umpire consistency. The analysis implements an idea Tom Tango originally blogged about. Here on my own blog I’m going to go a little more in depth on the underlying methodology and potential improvements. Along the way I’ll also relay some R tips I’ve picked up that are useful for sabermetric analysis, particularly on large datasets like PITCHf/x.

[If you haven’t already read the THT Annual, I strongly encourage you to pick up a copy (available on Amazon). Besides the background on my own work there are 300+ pages of great analysis and research.]

There are many methods one might use to model an umpire’s strike zone. In my THT piece I discussed some of the key drawbacks of neural networks, which include:

  • Opacity: Neural networks are black box models, and there is no “plain english” interpretation of the differences between the specifications of two different networks.
  • Non-Determinism: There is no closed-form method for deriving a neural network model; they are trained using iterative methods (typically with an element of randomization) and depending on implementation two training runs with the same data may not yield identical models.
  • Overfitting: If not carefully constructed, neural networks can be prone to overfitting their training data.

On the plus side, neural networks offer several advantages:

  • Flexibility: Neural networks are extremely flexible as applied to binary classification problems like calling balls and strikes; they assume no particular shape to the target distribution.
  • Robustness: Unlike binning methods for modeling a strike zone, in which the zone is divided into bins and actual frequencies are observed for each bin, a neural network model gracefully handles sparse data and introduces no discontinuities that need to be smoothed out.

To create the neural network models of strike zones, I used R’s neuralnet package.

As discussed in my THT Annual piece, one of the first things I did was model individual strike zones for every batter. This was to establish the top and bottom of their typical individual strike zones: e.g. Jose Altuve and Giancarlo Stanton aren’t going to have the same zone, and the problems with PITCHf/x’s sz_top and sz_bot fields are well documented.

In the remainder of this post I’m going to work through an example of modeling an individual batter strike zone in R. With that knowledge you can train your own strike zone models for batters, pitchers, umpires – whatever. I will assume some basic knowledge of R concepts, as well as access to PITCHf/x data from Gameday inside your R environment (for the quickest way to get up and running with PITCHf/x analysis in R, try pitchRx).

First, if you haven’t already, install the neuralnet package:

> install.packages(“neuralnet”)
> library(neuralnet)

Let’s assume we have every 2014 pitch loaded for Mike Trout in a data frame df, with horizontal pitch location in column px, vertical pitch location in column pz, and the pitch outcome (in typical Gameday notation) in column des.

To evaluate a strike zone we need to look at called pitches only. To do that we’ll subset df:

> df_c <- df[df$des==‘Called Strike’ || df$des==‘Ball’ || df$des==‘Ball in Dirt’,]

Next, we’ll convert the ball/strike call into binary data to train the neural network:

> df_c$call <- ifelse(df$des==‘Called Strike’,1,0)

The neuralnet package makes the last step very easy:

> m_bat <- neuralnet(call ~ px + pz, data=df_c, hidden=4, linear.output=FALSE)

(Note that the THT Annual has a discussion of how I arrived at using 4 hidden layers.)

These last few commands illustrate my favorite feature of R: its syntax for working with vectors is extremely concise. In three lines of code we filtered out unnecessary data, transformed a text variable to numeric, and trained a neural network.

One pedagogical comment: although the interactive R shell is extremely cool and useful and is how I’m presenting this tutorial, and even though you can save and reload your command history between R sessions, the best practice is to do your work in script files (with comments!) and run the script from the R shell when you want to see the results. You do this with the source command:

> source(“batter_strike_zone.R”)

Trust me, the sooner you get into this habit the happier you will be: it’s no fun having to sort through your command history to figure out what you were trying to do weeks, months or years ago.

Getting back to the topic at hand, now we have a neural network model of a strike zone for Mike Trout. To find the estimated probability of a pitch over the plate around belt-high (say px=0, pz=2.5) being called a strike, use compute:

> x <- compute(m_bat, data.frame(px=0, pz=2.5))

The variable x now contains an object with a variety of information about the neural network. To get the estimated strike probability, which is what we care about, access the net.result attribute:

> x$net.result
          [,1]
[1,] 0.9842061

That’s it for training and using a neural network model of a batter strike zone! In my next post on this topic, I will show how I use these models to estimate the top and bottom of each batter’s strike zone.

Advertisements

5 thoughts on “Modeling strike zones with neural networks

  1. Peter,

    Absolutely loved your analysis in this year’s HBT. One of my favorites in the book. However, I did have one question regarding the visualizations depicted. I’ve been trying to wrap my head around the strike zone visuals and have one particular question of DeMuth’s strike zone model on pg.344 with LHH. Within the explanation it states that “the lefty zone is visibly different from the righty zone. The difference in strike calls between lefties and righties is well-documented, and in this example it is obvious: the lefty zone extends well beyond the righty zone on the left side and ends well short of the righty zone on the right side.” Wouldn’t those two statements be reversed if this visualization is from a catcher’s perspective? So that the lefty zone extends well beyond the righty zone on the right side… and vice versa?

    Any information would be greatly appreciated! Keep up the great work!

    Thanks

    Like

  2. Thanks for the kind words and for taking the time to comment!

    I definitely could have explained that better – my reference to sides in that paragraph is pretty ambiguous.

    What I should have said is that the left side of the lefty strike zone (outside for a lefty hitter) extends further from the center of the plate than the right side of the righty strike zone (outside for a righty hitter). I.e. an outside pitch to a lefty is more likely to be called a strike than an outside pitch to a righty, and an inside pitch to a lefty is less likely to be called a strike than an inside pitch to a righty.

    Does that clear it up?

    Like

    1. I believe it does. So in essence the outside zone for the lefty is depicted by the -1 on the x axis and the outside zone for the righty is depicted by the 1 on the x axis even though they are essentially both 1 foot outside the center of the plate, relative to the batter?

      Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s