Looking to get WAR levels in OOTP more in line with MLB

[teddy]

Hi all, I've simmed through 2019 a couple of times, and I've noticed on the macro level that pitching WAR and hitting WAR don't reflect a typical real MLB season as much as I'd like.

For example:

In OOTP, the 30th ranked pitcher posts around 4.8 to 5.0 WAR. In real life, it's usually around 3.5 to 3.8 WAR. For hitters, the 30th ranked player posts a WAR around 4.0. In real life, the 30th ranked hitter is good for 4.5 WAR.

I have a feeling there are settings I could adjust to get the WAR dispersal more to my liking.

Could anyone help me out? I appreciate any input.

Thanks!

[cody8200]

WAR isn't calculated the exact same way in OOTP that it is IRL. In the game, it is partially calculated via VORP. That might have something to do with the discrepancies you are seeing.

[teddy]

I gotcha. I'll accept that the OOTP formula for WAR is different, but the replacement-level SPs should probably still be performing better, because as it looks like right now, there are too many elite-WAR pitchers relative to hitters. It appears out of balance.

Is there anything I can modify to "raise the baseline" of the replacement-level pitcher so that the difference in performance between the elite and replacement pitcher is a little narrower?

[The Wolf]

Ask Markus to start using standard WAR instead of his own custom version.

[cody8200]

It is the fielding model that won't support it in OOTP (if I remember correctly). Also, there are multiple versions of WAR in the wild. In fact, baseball reference says this -

"There is no one way to determine WAR. There are hundreds of steps to make this calculation, and dozens of places where reasonable people can disagree on the best way to implement a particular part of the framework. We have taken the utmost care and study at each step in the process, and believe all of our choices are well reasoned and defensible. But WAR is necessarily an approximation and will never be as precise or accurate as one would like."

That all being said, I'm on board with Teddy's request to make replacement level pitching more realistic in terms of WAR, I'm just not sure how to do it.

[teddy]

Yep, it's just the hitting-pitching value proportion that I was hoping to smooth out. If it's not really possible, no big deal. This game is amazing, and I can just value higher-end SPs more when I work on my rosters.

[Curve Ball Dave]

WAR is a theoretical concept, not an actual descriptive stat so it's far more art than science. That's why there's so many different ways to calculate it and it should never be taken literally. It's just one tool in the box among many and not always the most useful.

[r0nster]

I agree with Dave ..... I dont use too many of todays stats to determine who I want in varrious positions anyways

[RchW]

Originally Posted by teddy

Hi all, I've simmed through 2019 a couple of times, and I've noticed on the macro level that pitching WAR and hitting WAR don't reflect a typical real MLB season as much as I'd like.

For example:

In OOTP, the 30th ranked pitcher posts around 4.8 to 5.0 WAR. In real life, it's usually around 3.5 to 3.8 WAR. For hitters, the 30th ranked player posts a WAR around 4.0. In real life, the 30th ranked hitter is good for 4.5 WAR.

I have a feeling there are settings I could adjust to get the WAR dispersal more to my liking.

Could anyone help me out? I appreciate any input.

Thanks!

The difference you are seeing with pitching is most likely due to injuries IRL. It also may be influenced by a wider talent distribution. At the default setting starting pitchers do not get injured enough. Now that may have been fixed, I don't have enough playing time to see.

On the batting side it may be due to a wider talent distribution. I think it is very difficult to replicate real life "opportunity performance" by certain players in a computer sim. Markus cannot control the distribution of base out states that occur each year. Those base out states produce results that may be distributed slightly differently in-game vs real life.

[The Wolf]

Originally Posted by Curve Ball Dave

WAR is a theoretical concept, not an actual descriptive stat so it's far more art than science. That's why there's so many different ways to calculate it and it should never be taken literally. It's just one tool in the box among many and not always the most useful.

All Markus has to do is select one version, and then use it. I suggest that he use Baseball Reference's version myself, but he needs to just pick one of the standard ones and go with it

[teddy]

Neato. :-) Thanks, that's very thought-provoking.

[jaa36]

I've been interested in this issue since WAR was added to OOTP a few versions ago. I agree with the OP that it appears likely to be related to the replacement level being lower than it should be for pitchers, resulting in pitchers having artificially high WAR values. Another possibility is that the "rate" at which pitchers accumulate WAR for performing above replacement level is higher than it should be.

Fangraphs and BP recently agreed that the replacement level would be set such that for a full slate MLB season (30 teams, 162 games = 2430 total games played) all players would accumulate 1000 WAR. On Fangraphs, in both 2012 and 2013, position players earned exactly 570 WAR and pitchers earned exactly 430 WAR; it's probably defined somewhere that 57% of all available WAR goes to position players.

I may run some tests to see what the numbers would look like in a full season of OOTP- whether the spread of WAR replicates real-life MLB. My guess is that it's closer to a 50-50 split. I also wonder if OOTP automatically calculates the replacement level based on the league's wOBA and FIP, or if there is just a default baseline replacement level. The latter would mean that WAR could get pretty skewed in very high or low run environments.

[The Wolf]

Originally Posted by jaa36

Fangraphs and BP recently agreed that the replacement level would be set such that for a full slate MLB season (30 teams, 162 games = 2430 total games played) all players would accumulate 1000 WAR. On Fangraphs, in both 2012 and 2013, position players earned exactly 570 WAR and pitchers earned exactly 430 WAR; it's probably defined somewhere that 57% of all available WAR goes to position players.

I was really pleased to see that happen. WAR needs to be a standardized stat.

[jaa36]

I did a quick test- set up a historical sim of 2013, set the injury rate to realistic modern day, and looked at the WAR output over the season. It was very different than Fangraphs WAR...

Position players accumulated 510 WAR, a little lower than the 570 in real life. However, pitchers accumulated 742 WAR, which is significantly higher than the 430 in real life.

In comparing the list of top pitchers ranked by WAR in real life vs OOTP, the OOTP pitchers were consistently credited with about 60% more WAR than their real-life counterparts. (FWIW, 742/430=1.72.) This was a pretty consistent pattern down the list, suggesting to me that the problem is not that the replacement level is set too low, but rather that pitchers are receiving proportionately too many WAR per inning pitched.

The hitters were actually pretty close in OOTP and real life. The top 20 OOTP hitters tended to have WARs about 10% higher than real life, though lower down the list they tended to have about 10% less WAR than real life.

Anyway, this does give pretty good evidence that OOTP WAR consistently overrates pitchers. For me, when considering the top players in the league, it means that I would probably consider a pitcher's actual value to be only half to two-thirds of his reported WAR. Said more scientifically, to better replicate the spread of WAR in real life, you would have to divide pitcher WAR by 1.92: (742/430)/(510/570).

This would also be a good methodology to figure out whether OOTP WAR underrates catchers, though the mechanism there (at least in OOTP 14) seemed to be systematically low zone ratings. I may look at that sometime.

[jaa36]

Another quick test, looking at UZR calculations for the same season.

I summed up all the UZRs for each position. Of course, at each position, net UZR should by definition be zero.

C: -125
1B: +7.2
2B: +20.2
3B: +31.8
SS: +17.5
LF: +0.6
CF: +17.2
RF: +3.9
P: +4.8
Total: -21.8

The overall total is not so bad: -21.8 runs off across a total of 43,740 games started. (That's 162 games x 30 teams x 9 positions = 43,740.)

But obviously the spread of UZR from position to position is hugely problematic. Essentially, the "average" catcher is being recorded as over four runs (nearly half a win) "worse" than average. The rest of the positions are all within a run of each other, but catcher is a huge outlier.

So, as I suspected, catchers are being unfairly penalized in OOTP WAR, and a full season of catching should be rewarded with about an additional 0.5 WAR. For me, this has implications for end of season awards and for hall of fame; I'd be more likely to select a 6.0-WAR catcher than a 6.4-WAR first baseman for the MVP.

Perhaps the devs can fix some of these issues this year, or at least for OOTP 16...

[The Wolf]

This is good and valuable work. Well done.

[RchW]

Originally Posted by jaa36

I did a quick test- set up a historical sim of 2013, set the injury rate to realistic modern day, and looked at the WAR output over the season. It was very different than Fangraphs WAR...

Position players accumulated 510 WAR, a little lower than the 570 in real life. However, pitchers accumulated 742 WAR, which is significantly higher than the 430 in real life.

In comparing the list of top pitchers ranked by WAR in real life vs OOTP, the OOTP pitchers were consistently credited with about 60% more WAR than their real-life counterparts. (FWIW, 742/430=1.72.) This was a pretty consistent pattern down the list, suggesting to me that the problem is not that the replacement level is set too low, but rather that pitchers are receiving proportionately too many WAR per inning pitched.

The hitters were actually pretty close in OOTP and real life. The top 20 OOTP hitters tended to have WARs about 10% higher than real life, though lower down the list they tended to have about 10% less WAR than real life.

Anyway, this does give pretty good evidence that OOTP WAR consistently overrates pitchers. For me, when considering the top players in the league, it means that I would probably consider a pitcher's actual value to be only half to two-thirds of his reported WAR. Said more scientifically, to better replicate the spread of WAR in real life, you would have to divide pitcher WAR by 1.92: (742/430)/(510/570).

This would also be a good methodology to figure out whether OOTP WAR underrates catchers, though the mechanism there (at least in OOTP 14) seemed to be systematically low zone ratings. I may look at that sometime.

Nice work. I feel somewhat embarrassed that I never checked this before. My MLB fictional based on 2009-2012 MLB output shows 554 WAR for batters and 534 for Pitchers. I do have more dominant starting pitchers by design. If I reduced SP stamina would that reduce pitcher WAR or are you saying the calculation is wrong?

[jaa36]

Originally Posted by RchW

Nice work. I feel somewhat embarrassed that I never checked this before. My MLB fictional based on 2009-2012 MLB output shows 554 WAR for batters and 534 for Pitchers. I do have more dominant starting pitchers by design. If I reduced SP stamina would that reduce pitcher WAR or are you saying the calculation is wrong?

I'm really not sure what effect reducing SP stamina would have. I think it's likely that it would reduce the WAR that goes to SP, and likely the difference would go to the RP with the overall pitching WAR approximately the same.

As I said a few posts ago, it would be interesting to find out if the WAR ratio for pitchers to hitters would remain the same in a league with an abnormally high or low run environment. By the way WAR is described on Fangraphs, the run environment really shouldn't matter; pitchers should accumulate 43% of the available WAR regardless of the average runs scored per game. I may check how run environment affects things in the upcoming days, unless someone beats me to the punch.

[Pacoheadley]

Originally Posted by jaa36

Another quick test, looking at UZR calculations for the same season.

I summed up all the UZRs for each position. Of course, at each position, net UZR should by definition be zero.

C: -125
1B: +7.2
2B: +20.2
3B: +31.8
SS: +17.5
LF: +0.6
CF: +17.2
RF: +3.9
P: +4.8
Total: -21.8

The overall total is not so bad: -21.8 runs off across a total of 43,740 games started. (That's 162 games x 30 teams x 9 positions = 43,740.)

But obviously the spread of UZR from position to position is hugely problematic. Essentially, the "average" catcher is being recorded as over four runs (nearly half a win) "worse" than average. The rest of the positions are all within a run of each other, but catcher is a huge outlier.

So, as I suspected, catchers are being unfairly penalized in OOTP WAR, and a full season of catching should be rewarded with about an additional 0.5 WAR. For me, this has implications for end of season awards and for hall of fame; I'd be more likely to select a 6.0-WAR catcher than a 6.4-WAR first baseman for the MVP.

Perhaps the devs can fix some of these issues this year, or at least for OOTP 16...

Thank you for adding up the UZR totals. I noticed the catcher problem a year or two back and meant to get data on it, but never got around to it.

And people, we know that WAR can be calculated in multiple ways... However, it would be nice to have OOTP conform to one of the 2 by far most popular ones, either Fangraphs WAR, or BB Reference WAR.

[teddy]

Originally Posted by jaa36

Anyway, this does give pretty good evidence that OOTP WAR consistently overrates pitchers. For me, when considering the top players in the league, it means that I would probably consider a pitcher's actual value to be only half to two-thirds of his reported WAR. Said more scientifically, to better replicate the spread of WAR in real life, you would have to divide pitcher WAR by 1.92: (742/430)/(510/570).

Correct me if I'm wrong guys, but I thought high pitcher WAR values could be corrected if the replacement-level pitchers were better. To put it another way, replacement-level hitters would have to fare a bit worse. Maybe the answer lies in weakening the output of the replacement-level hitter? Is there a way to modify that by a certain degree in the settings?

Thanks for all the ideas in this post!