Does the engine limit "outliers"?

[HoustonGM]

I've played about six years into the future now, starting with a 2022 live start, and while each year, the leaguewide totals are in-line with real life and what I'd expect, I've noticed that at least when it comes to pitchers, it seems the ERA leaders average worse ERAs than they do IRL. This is why I asked about a yearly leaderboard overview (https://forums.ootpdevelopments.com/...d.php?t=338700) as I wanted a quick easy reference. I went through, though, and here are the ERA leaders each year (AL/NL, with the league ERA in parentheses):

2022: 2.30/2.61 (3.77/3.97)
2023: 3.08/2.16 (4.04/4.14)
2024: 2.97/2.79 (4.16/4.31)
2025: 3.01/2.42 (4.15/4.31)
2026: 3.17/3.05 (4.22/4.20)
2027: 2.59/2.91 (4.13/4.10)
2028: 3.24/3.14 (4.14/4.09)

The league leader each year has an ERA that's roughly 70% of the league total, or to put another way, the ERA+ leader is generally in the 130-150 range.

Now the last 7 years of real life:

2015: 2.45/1.66 (4.01/3.90)
2016: 3.00/2.13 (4.20/4.16)
2017: 2.25/2.31 (4.37/4.34)
2018: 1.89/1.70 (4.27/4.02)
2019: 2.32/2.50 (4.60/4.38)
2020: 1.63/1.73 (4.42/4.47)
2021: 2.84/2.43 (4.32/4.20)

In real life, the league tends to have an ERA around half of what the leaguewide average is - an ERA+ in the 180-220 range.

My gut feeling is that there's some part of the engine that keeps the leaguewide stats accurate based on the modifiers has a side effect of normalizing individual performances, making truly standout seasons rarer. I haven't observed the same effect on the offensive side of things, so I'm not really sure, but I thought I'd bring it up for discussion as it's lessened the realism a bit for me to consistently have Cy Young winners with ERA's in the low 3s!

[jcard]

Originally Posted by HoustonGM

I've played about six years into the future now, starting with a 2022 live start, and while each year, the leaguewide totals are in-line with real life and what I'd expect, I've noticed that at least when it comes to pitchers, it seems the ERA leaders average worse ERAs than they do IRL. This is why I asked about a yearly leaderboard overview (https://forums.ootpdevelopments.com/...d.php?t=338700) as I wanted a quick easy reference. I went through, though, and here are the ERA leaders each year (AL/NL, with the league ERA in parentheses):

2022: 2.30/2.61 (3.77/3.97)
2023: 3.08/2.16 (4.04/4.14)
2024: 2.97/2.79 (4.16/4.31)
2025: 3.01/2.42 (4.15/4.31)
2026: 3.17/3.05 (4.22/4.20)
2027: 2.59/2.91 (4.13/4.10)
2028: 3.24/3.14 (4.14/4.09)

The league leader each year has an ERA that's roughly 70% of the league total, or to put another way, the ERA+ leader is generally in the 130-150 range.

Now the last 7 years of real life:

2015: 2.45/1.66 (4.01/3.90)
2016: 3.00/2.13 (4.20/4.16)
2017: 2.25/2.31 (4.37/4.34)
2018: 1.89/1.70 (4.27/4.02)
2019: 2.32/2.50 (4.60/4.38)
2020: 1.63/1.73 (4.42/4.47)
2021: 2.84/2.43 (4.32/4.20)

In real life, the league tends to have an ERA around half of what the leaguewide average is - an ERA+ in the 180-220 range.

My gut feeling is that there's some part of the engine that keeps the leaguewide stats accurate based on the modifiers has a side effect of normalizing individual performances, making truly standout seasons rarer. I haven't observed the same effect on the offensive side of things, so I'm not really sure, but I thought I'd bring it up for discussion as it's lessened the realism a bit for me to consistently have Cy Young winners with ERA's in the low 3s!

While I wonÂ’t go into detail or theories at the moment, multiple simulations of the 2022 MLB season, I have been asking this and similar questions about the results (including some aspects of offensive statistics, and the comparisons between a typical 2022 MLB sim and similar default fictional league modeled on the 2022 MLB). The harder one looks, the more one wonders how much of the claim to realism is staked on hitting league-total targets- and how many other components of simulacrum is either directly or indirectly sacrificed at the altar of this idol.

[Brad K]

The developers wear the randomness purity ring. There has been no admitted effort to control outliers except some real life seasonal home run totals are considered outliers and the players rated for fewer HRs that year. jcard is right in saying league totals is what matters to the devs.

Another example is the adjust/weaken function for low AB/IP players. The production taken from the weakened players in the game is distributed among all players. It should be distributed among the pool of players eligible for weaken/adjust. After all, some low AB/IP players over perform and some under perform, in both cases due to low sample size and chance. So the under performers should get the performance taken away from the over performers.

[HoustonGM]

Originally Posted by Brad K

The developers wear the randomness purity ring. There has been no admitted effort to control outliers except some real life seasonal home run totals are considered outliers and the players rated for fewer HRs that year. jcard is right in saying league totals is what matters to the devs.

Another example is the adjust/weaken function for low AB/IP players. The production taken from the weakened players in the game is distributed among all players. It should be distributed among the pool of players eligible for weaken/adjust. After all, some low AB/IP players over perform and some under perform, in both cases due to low sample size and chance. So the under performers should get the performance taken away from the over performers.

Those functions are for historical play though, right? My observation is about current+future play

[Syd Thrift]

Originally Posted by HoustonGM

Those functions are for historical play though, right? My observation is about current+future play

ERA is tough since it’s a third order statistic. What do K, BB, HR, and BABIP rates look like? It’s possible - not saying that’s what happened, just that I don’t know - that you’ve got a top tier Stuff guy who plays in front of a bad defense, for example. I know defense has a pretty big impact on the game at the moment, perhaps too large of an impact vs. real life (also, IRL guys who strike out a lot of batters have a small but not insubstantial boost to their BABIPs - IIRC Nolan Ryan’s was around 10 points lower than it should have been for his career - and I’m not 100% sure OOTP models that).

FWIW, I’ve seen the devs and beta team talk about the issue that the top level starting pitchers are never quite as good in OOTP as they are in real life or how they could be in fictional saves. I think there are hypotheses being explored but I’m not privy to the reasons.

[WyattGOAT]

Noticed this as well in regards to pitcher ERA. Batter "outliers" seem fine in my sims as there are usually a few 45+ HR guys

[Syd Thrift]

and how many other components of simulacrum is either directly or indirectly sacrificed at the altar of this idol.

I'll take Hyperbole for $1200, Alex.

The league totals are indeed what the game tries to hit with the league totals modifiers. That is why they are called league totals modifiers. Otherwise, if great players aren't great enough, especially in fictional leagues (although again I think people have seen this issue with historical leagues as well and I can definitely say that in fictional/historical, pitchers don't have 300+ win careers so much, which I'll go into in a separate paragraph), that's its own issue to get into and no amount of whinging about "the altar of purity" is going to make the devs improve on this.

I agree too that fictional *hitters* do not seem to have this issue. I had a guy hit 65 HRs in a fictional/historical league in 1962 and I've seen hitters flirt with and go over .400, hit a crap ton of RBIs in some eras, and in 23 get over 100 steals in a season (which has some issues attached to managing games but that's another matter).

The real question here is why does this happen to pitchers? I have a few theories:

- Pitchers get injured more than regular players, which is proper, but I've observed this thing when simming out lots of (fictional) seasons where the effect of that is that basically any pitcher who is at all effective in the league and who therefore gets a lot of starts suffers through a very bad injury sooner or later and this curtails their effectiveness. I'm not sure if this would show up in a league after only 7ish years but it certainly could, I think. In my dynasty that I simmed from 1946 to 1968 before taking over in 1969 (again, fictional players) the lifetime leader in wins finished with 239. He was all but unplayable his final year (1969) because he simply could not start twice in a row without getting hurt some time. I can understand that happening to some guys. It probably shouldn't happen to literally every good pitcher in a league.

- I feel like in individual years, especially in the 60s and 70s, the best pitchers don't pitch enough from season to season. OK, a guy has 10 years of dominance and then falls off. If he's playing in that era, 10 years of dominance all by itself should be 200+ wins. A huge, huge part of this issue is that the game goes to a 5 man rotation for most of the 60s when it clearly should be a 4-man... but I'm getting into the weeds here. Somehow or some way, those top players should really be ridden kind of hard. My sense is that a lot of them drop off due to injury, which, see above.

- The AI may be too aggressive in taking out good starting pitchers, although that would tend to keep ERAs low rather than high.

- Fielding may be unbalanced, which adds another mitigating factor into something like ERA, although to me it seems like that should also be a way to have guys have really, really good seasons (by ERA) that are based on more than just devastating Stuff/Movement/Control numbers. Jim Palmer immediately springs to mind historically.

Honestly I'm all but convinced that the first item I mentioned is the biggest factor. I look back on the Year of the Pitcher in my dynasty (which, again, I didn't sim personally) and what I see are the actual Cy Youngs going to relief pitchers but also that there's not only no Denny McClain 30 game winner type but there aren't even 20 game winners. Once I took over, I "alleviate" this by resetting injuries on a few pitchers in the league but of course this is not something I should have to do.

I also think that it would make sense that you don't see this effect as much with historical leagues because recalc washes away a lot if not all of this effect.

[Syd Thrift]

Sorry for the double post but I wanted to take a look at the league leaders in my league in the 60s for ERA (this was all compiled in 22 but I think the issue is one that carries over):

My league is in the middle of 1970 now, fyi...

1960: AL: 2.92 in a league with a 3.93 (135), NL: 2.73 in 3.87 (142) (also, both guys are still in the league; the AL leader is, surprisingly, still an effective pitcher with a 191-130 record and career 124 ERA+, whereas the NL guy is barely kicking on, was left exposed in the expansion draft, and carries a career 104 ERA+ and 183-161 career record)

1961: AL 2.36 vs 3.93 (167) NL 3.42 vs 4.06 (119) (the NL guy is still kicking along as a league average pitcher with a 164-136 record and a 112 ERA+ and also exactly that kind of "too little usage" thing for the prime of his career - he led the *league* with 18 wins in 1964 for instance. The AL guy retired long enough ago to get into the HOF; he finished 203-133 with a 122)

1962: AL 2.39 in 3.90 (163) NL 2.44 in 4.07 (167) (the AL guy also was exposed to expansion and his career is basically done with a 78-111 record and a 90 ERA+; the NL guy is the pitcher I alluded to in my wall o' text above and finished his career 239-149 with a 129 ERA+)

1963: AL 2.14 in 3.53 (165) NL 2.14 in 3.51 (the AL guy was converted to relief before I took the league over; he's now, at age 30, pretty washed up with a 56-47 record with a 106, and it does look like he missed a whole bunch of time to injury between '63 and '69 before re-emerged as a full-time reliever, which he's been kind of meh at this year and also he's Wrecked and still missed the first 6 weeks of the season; the NL guy retired after a bad 1969 and, you guessed it, got hurt for large chunks of his career, and in the end was converted to a not very good middle reliever)

1964: 1.86 in 3.37 (181) NL 2.15 in 3.60 (the AL guy had a decent run of injury luck but has played exactly 18 games since 1968 and is currently missing the entire year with a torn UCL... also 17 of those 18 games played were at the beginning of 1968 and I know it's the beginning because he made the All-Star Game; the NL guy is... literally the first guy in this sample to not have been hugely and adversely effected by injuries. He's just affected by having a weird skillset that causes him to get pulled from games because he's not really that good - career record of 150-162, 101 ERA+)

1965: AL 2.16 in 3.47 NL 1.83 in 3.61 (208) (the AL guy is the same dude as last year; the NL guy is the same guy from 1963)

I'll stop there. I looked at the histories to give you an idea of what I think is happening. Basically this entire league is of guys who came up, looked lights out for a few years, got hurt, and then either never came back or came back as a mediocre pitcher. The results I pulled look a little better in terms of ERA+ than the other sample but then 2 of the years are expansion years and two of the others are directly post expansion. You just see an awful lot of guys not playing as much as their non-fictional counterparts did in seasons (which is partially settings but again, partially injuries) and then not having the same length of dominance as real-life guys.

[HoustonGM]

Originally Posted by Syd Thrift

ERA is tough since it’s a third order statistic. What do K, BB, HR, and BABIP rates look like? It’s possible - not saying that’s what happened, just that I don’t know - that you’ve got a top tier Stuff guy who plays in front of a bad defense, for example. I know defense has a pretty big impact on the game at the moment, perhaps too large of an impact vs. real life (also, IRL guys who strike out a lot of batters have a small but not insubstantial boost to their BABIPs - IIRC Nolan Ryan’s was around 10 points lower than it should have been for his career - and I’m not 100% sure OOTP models that).

FWIW, I’ve seen the devs and beta team talk about the issue that the top level starting pitchers are never quite as good in OOTP as they are in real life or how they could be in fictional saves. I think there are hypotheses being explored but I’m not privy to the reasons.

Defense and such could explain why a particular pitcher may not have an ERA that matches his stuff - but it can't explain that year after year, consistently, the pitchers with the best ERA aren't as good as they are IRL, in comparison to the league average. The leaguewide totals for the component stats largely seem accurate. It is true that these years I've simmed since have had a lower strikeout rate leaguewide than recent IRL years - but it's on par with the 2015-2018 levels which were still part of the years I assessed ERA.

I could see what you're saying about injuries, but that, at least logically, would be seen more in career numbers than year by year... there are still pitchers each year that stay healthy the full season, you'd think that even if injuries disproportionately harm pitcher longevity, the ones that are healthy and at the top of the league would still prevent runs at an elite rate?

At least I'm not crazy and others have noticed similar! As mentioned, it doesn't seem to occur with hitters, in either counting stats like HRs or rate stats like OPS+ - those league leaders always seem realistically above the league average.

[jcard]

Originally Posted by Syd Thrift

I'll take Hyperbole for $1200, Alex.

The league totals are indeed what the game tries to hit with the league totals modifiers. That is why they are called league totals modifiers. Otherwise, if great players aren't great enough, especially in fictional leagues (although again I think people have seen this issue with historical leagues as well and I can definitely say that in fictional/historical, pitchers don't have 300+ win careers so much, which I'll go into in a separate paragraph), that's its own issue to get into and no amount of whinging about "the altar of purity" is going to make the devs improve on this..

It is unfortunate that you would characterize any of the posts in this thread as *whinging*(?), but I suppose hyperbole can take many forms.

[Syd Thrift]

Originally Posted by HoustonGM

Defense and such could explain why a particular pitcher may not have an ERA that matches his stuff - but it can't explain that year after year, consistently, the pitchers with the best ERA aren't as good as they are IRL, in comparison to the league average. The leaguewide totals for the component stats largely seem accurate. It is true that these years I've simmed since have had a lower strikeout rate leaguewide than recent IRL years - but it's on par with the 2015-2018 levels which were still part of the years I assessed ERA.

I could see what you're saying about injuries, but that, at least logically, would be seen more in career numbers than year by year... there are still pitchers each year that stay healthy the full season, you'd think that even if injuries disproportionately harm pitcher longevity, the ones that are healthy and at the top of the league would still prevent runs at an elite rate?

At least I'm not crazy and others have noticed similar! As mentioned, it doesn't seem to occur with hitters, in either counting stats like HRs or rate stats like OPS+ - those league leaders always seem realistically above the league average.

My theory is that the injury issue hits the best pitchers the most often because they’re in there pitching the most (and probably the players the AI is pitching while exhausted the most). I guess one way to test this would be to turn injuries entirely off, maybe…

[Rain King]

Outliers (by the examples being used in the OP) are going to be largely about rating distribution.

It will also matter whether you are using game generated players or the MLB Quickstart which uses manually create players. I doubt the Quickstart is adhering to the same distribution curve that the game has programmed into it.

[MathBandit]

Originally Posted by Syd Thrift

ERA is tough since it’s a third order statistic. What do K, BB, HR, and BABIP rates look like? It’s possible - not saying that’s what happened, just that I don’t know - that you’ve got a top tier Stuff guy who plays in front of a bad defense, for example. I know defense has a pretty big impact on the game at the moment, perhaps too large of an impact vs. real life (also, IRL guys who strike out a lot of batters have a small but not insubstantial boost to their BABIPs - IIRC Nolan Ryan’s was around 10 points lower than it should have been for his career - and I’m not 100% sure OOTP models that).

I know this is somewhat off-topic, but I really do think that now that we've basically all figured out that RBIs aren't a useful metric and most people seem to have understood that Batting Average is either obsolete or at minimum not nearly as important as a catch-all "Batting Champion" metric as we used to think, that ERA is absolutely going to be the next domino to fall, and that by 2030 people will think it's absolutely wild that anyone used to judge a pitcher's performance by it.

[HoustonGM]

Originally Posted by Rain King

Outliers (by the examples being used in the OP) are going to be largely about rating distribution.

It will also matter whether you are using game generated players or the MLB Quickstart which uses manually create players. I doubt the Quickstart is adhering to the same distribution curve that the game has programmed into it.

The issue could definitely be with the rating distribution. Here's a positional breakdown of all 70+ overalls in this particular league (with 100% scouting accuracy):

C 3
1B 2
2B 6
3B 4
SS 1
LF 2
CF 4
RF 2

SP 4
RP 9

[Lukas Berger]

The engine doesn't limit outliers at all.

But talking about the quickstart the ratings distribution there is indeed such that it's likely to limit outliers.

Because the base ratings are initially based on ZIPS projections. And you're not really projecting anyone to have a terrible season. Anyone that is projected to have a bad season is just not going to be good enough to even play in MLB in the quickstart.

So as opposed to other game modes, like historical for example, where you'll get the guys who actually did have terrible seasons folded into the game and still in MLB, in the quickstart, you don't have those lower level players. Plus, given that the quickstart is designed to be used with dozens of other real life leagues, the ratings baseline for MLB level guys has to be higher in order for those leagues to coexist and to get enough of a ratings spread in the other leagues.

So long story short, at MLB level the quickstart does have more highly rated players in the middle and at the bottom of the distribution and that will naturally tend to limit outlier season for players at the top.

That's just to be expected based on what the quickstart is and how its modelled.

[Syd Thrift]

Originally Posted by Lukas Berger

The engine doesn't limit outliers at all.

But talking about the quickstart the ratings distribution there is indeed such that it's likely to limit outliers.

Because the base ratings are initially based on ZIPS projections. And you're not really projecting anyone to have a terrible season. Anyone that is projected to have a bad season is just not going to be good enough to even play in MLB in the quickstart.

So as opposed to other game modes, like historical for example, where you'll get the guys who actually did have terrible seasons folded into the game and still in MLB, in the quickstart, you don't have those lower level players. Plus, given that the quickstart is designed to be used with dozens of other real life leagues, the ratings baseline for MLB level guys has to be higher in order for those leagues to coexist and to get enough of a ratings spread in the other leagues.

So long story short, at MLB level the quickstart does have more highly rated players in the middle and at the bottom of the distribution and that will naturally tend to limit outlier season for players at the top.

That's just to be expected based on what the quickstart is and how its modelled.

OK, but there is clearly an effect on starting pitching with leagues that have been left to fictional development several years in. Can we get an acknowledgement? I swear I've seen beta testers talk about how y'all have been debating how to handle this issue.

[Syd Thrift]

Originally Posted by HoustonGM

The issue could definitely be with the rating distribution. Here's a positional breakdown of all 70+ overalls in this particular league (with 100% scouting accuracy):

C 3
1B 2
2B 6
3B 4
SS 1
LF 2
CF 4
RF 2

SP 4
RP 9

Well, right, but *why* are these 70+ guys so rare, relatively speaking?

[Lukas Berger]

Originally Posted by Syd Thrift

OK, but there is clearly an effect on starting pitching with leagues that have been left to fictional development several years in. Can we get an acknowledgement? I swear I've seen beta testers talk about how y'all have been debating how to handle this issue.

What the beta testers are talking about has to do with historical play, not fictional and not really so much to do with outliers in general but with certain specific pitchers who tended to consistently outpitch their FIP related stats in real life not getting that 'extra' boost in OOTP because it's not actually reflected in the stats we use to model them. Mariano Rivera or Catfish Hunter etc.

As far as fictional play, if there are any issues with outliers not happening enough there, this is the first I've heard of it. I'm not aware of any issues there (Doesn't mean there aren't any, of course).

[Lukas Berger]

Originally Posted by Syd Thrift

Well, right, but *why* are these 70+ guys so rare, relatively speaking?

Well, you can only go so high at the top end of the scale before things start to get a little crazy and you get unintended consequences.

And besides, I doubt there are more guys than this you'd actually give 70+ ratings to in MLB now (which would be two standard deviations or more above average).

If anything, those listed numbers might be too many highly rated guys, not too few, if you compare to real life top of the scale rated players and how many real life guys you'd actually hang that sort of grade on.

[Syd Thrift]

Originally Posted by Lukas Berger

Well, you can only go so high at the top end of the scale before things start to get a little crazy and you get unintended consequences.

And besides, I doubt there are more guys than this you'd actually give 70+ ratings to in MLB now (which would be two standard deviations or more above average).

If anything, those listed numbers might be too many highly rated guys, not too few, if you compare to real life top of the scale rated players and how many real life guys you'd actually hang that sort of grade on.

OK, to this though... if there are 150 starting pitchers in the major leagues at any time, I'd expect a lot more than 2 of them to be 2 standard deviations above the norm in terms of raw ability. If memory serves, in normal distribution 2 standard deviations cover roughly 95% of the population (really 90%, with the other 5% being above and 5% being below) so there ought to be somewhere around 8 or 9 starting pitchers rather than 4. Small sample size theater in this case but like I said it's observable that there are flat-out fewer great starting pitchers in any league that relies on the development engine for a few years. As noted, I have my theories.