The basic framework for doing so seems to be:
- Determine a projection of the player's value for the life of the contract. Value is expressed in Wins Above Replacement and the player's decline is often expressed at a rate of 0.5 wins per year.
- Determine the "dollar amount per free agent win" in the first year, then account for inflation in this amount for the following years by using a 5% rate.
- By multiplying projected WAR by the projected dollar amount per WAR, determine the value that this player provides, expressed in a dollar amount.
- Compare this figure to the actual annual salary the player will be receiving in order to determine an opinion on the contract.
Below is Dave's chart that shows this methodology used to analyze Reye's contract.
| Year | WAR | $/WAR | Value |
|---|---|---|---|
| 2012 | 3.53 | $5.00 | $17.67 |
| 2013 | 3.37 | $5.25 | $17.67 |
| 2014 | 3.20 | $5.51 | $17.67 |
| 2015 | 3.05 | $5.79 | $17.67 |
| 2016 | 2.91 | $6.08 | $17.67 |
| 2017 | 2.77 | $6.38 | $17.67 |
Without saying anything about the rather arbitrary annual decline/inflation rates, the second step is where I believe this model truly fails. The "dollar amount per free agent win", also written as $/WAR, is the average amount that all teams spent per WAR on the free agent market. In other words, this is an aggregate figure resulting from hundreds of free agent contracts awarded by many different teams.
The problem here lies in the fact that no two teams are in the same financial and/or competitive position. But why is that a problem?
Chapter 5.2 of Baseball Prospectus's Baseball Between the Numbers illustrates a concept that can help begin to explain why this type of valuation is unsound. After confirming that reaching the postseason results in great financial gains, Nate Silver finds that not all marginal wins are of equal value, demonstrated by the density curve below (because of this financial incentive to reach October, a nearly identical curve can be extrapolated to show the marginal value of an additional win, expressed in monetary terms).
Consider this scenario: a 80-win team has about a 1.8% chance of making the postseason. If it adds a free agent who it believes will produce one additional win for the team (a 1 WAR player), it becomes an 81-win team and now has a 2.9% chance at reaching the postseason. The marginal postseason probability added is 1.1 percent: this player does not significantly affect this team's chances of receiving the huge financial benefits that are married with a postseason appearance. Now consider a second scenario. This same one-win player is signed by an 89-win team, thus making them a 90-win team. This move raises this team's chances of making the playoffs by over 11.4 percent.
Which team does this player provide more value to? Quite clearly, the expected marginal revenue produced by this 1 WAR free agent is much greater for the second club. Since this player means more to the 89-win team, they would be willing to offer him a greater wage than the 80-win team would. This is what makes the FanGraphs model fall short. By assuming that each marginal win is worth a universal dollar amount (Dave used $5 million/WAR), the model states that revenue is a linear function of wins. This, however, is far from reality as linearity implies that context is not a factor. In this specific case, the FanGraphs methodology claims that both teams would be "rational" to offer this player a yearly salary of $5 million. Again, because of context, the offers from each team are not going to be equal, so two vastly different contracts could both be rational.
Of course, the amount of games a team can expect to win in an upcoming season is not the only matter that defines context. Other factors, such as market size, positional needs, new ballparks, and upcoming minor league talent all help define context. Each is taken into consideration by front offices when deciding how to value free agents. Again, a rigid $/WAR model is inherently incapable of realistically capturing the effects of such factors.
The $/WAR calculations that FanGraphs carries out tell us useful information about what the entire free agent market looks like as a whole--it shows that the average amount spent on each additional win is rising. But because it chooses to ignore a multitude of variables, I find it to be a futile tool for analyzing free agent contracts on an individual basis. To ignore context in such a way that strict $/WAR models do is a lazy and irresponsible excuse for analysis.
Wonderful piece. I have been worried with Cameron's discussion of $/WAR, and this does a very good job explaining the problems with it. Good work.
ReplyDeleteThanks, I'm glad you like it. I've seen lots of people with the same sentiment on this topic so I thought I'd try to articulate some common frustrations.
ReplyDelete