A Possible Issue With Pitcher Talent Distribution

[Craig Scarborough]

Anyway, here's what I've tested. Taking a standard major league - 2009 teams, fictional league. Nothing out of the ordinary. I auto-simmed 2010 to 2030. In my gameplay with multiple versions of OOTP, I've seen what I perceived to be an issue with talent distribution and decided to test it (finally). While I don't think this is the be-all end-all as far as testing methodologies, I think it does shed some light on what I'm seeing. Bottom line - I'm seeing really great pitchers, some good pitchers and then a whole heck of a lot of horrible pitchers. There doesn't seem to be a nice in-between, more indicative of RL pitching. Anyway, those were my thoughts, and I decided to test it out.

While this is by no means scientific - if I were looking to get published, I would use much more exhaustive techniques - I do think this shows what I perceive to be a problem. I put starters who pitched 150 innings or more in a single year into ERA "buckets". The first being ERA < 3.00, then < 4.00, < 5.00, etc. I then took 2009 data and did the same with real players. Being that I've read that some people create long histories before taking on a team because of perceived issues with fictional players generated at league creation, I decided to not only test the years of 2010-2015 but 2025-2029 as well. Here's what I found:

REAL 2009: 11 < 3, 33 < 4, 36 < 5, 10 < 6
AVG OOTP 2010-2016: 4 < 3, 22 < 4, 43 < 5, 21 < 6, 10 > 6
AVG OOTP 2025-2029: 1 < 3, 30 < 4, 50 < 5, 22 < 6, 4 > 6

So - what do I see? Oddly, I'm seeing less extreme awesome pitching than I expected (11 with less than 3.00 ERA in real life vs. an average of 4 and 1 in the simulation). Beyond that, I'm seeing exactly what I surmised. While there's an even distribution between 3.XX ERAs and 4.XX ERAs in real life, in OOTP there are a great many more 4.XX ERAs than 3.XX (1.67x to almost double). Beyond that, there's a 2.5x uptick in horrible pitchers - guys that have an era of 5.00 or more. There's a slight uptick in pitcher's who pitch 150 innings+ in the OOTP sim - I can deal with that. While it seems at the macro level everything is just fine (league ERA and .AVG are in-line with what we expect) the distribution of ERAs is much flatter in real life than it is in the simulation. Therefore, in real life, there are a great many more serviceable pitchers than in OOTP.

In my OOTP sims, an average of ~30% of the starters have an ERA above 5.00. In real life, this is about 11%.

[damientheomen3]

Originally Posted by Craig Scarborough

Anyway, here's what I've tested. Taking a standard major league - 2009 teams, fictional league. Nothing out of the ordinary. I auto-simmed 2010 to 2030. In my gameplay with multiple versions of OOTP, I've seen what I perceived to be an issue with talent distribution and decided to test it (finally). While I don't think this is the be-all end-all as far as testing methodologies, I think it does shed some light on what I'm seeing. Bottom line - I'm seeing really great pitchers, some good pitchers and then a whole heck of a lot of horrible pitchers. There doesn't seem to be a nice in-between, more indicative of RL pitching. Anyway, those were my thoughts, and I decided to test it out.

While this is by no means scientific - if I were looking to get published, I would use much more exhaustive techniques - I do think this shows what I perceive to be a problem. I put starters who pitched 150 innings or more in a single year into ERA "buckets". The first being ERA < 3.00, then < 4.00, < 5.00, etc. I then took 2009 data and did the same with real players. Being that I've read that some people create long histories before taking on a team because of perceived issues with fictional players generated at league creation, I decided to not only test the years of 2010-2015 but 2025-2029 as well. Here's what I found:

REAL 2009: 11 < 3, 33 < 4, 36 < 5, 10 < 6
AVG OOTP 2010-2016: 4 < 3, 22 < 4, 43 < 5, 21 < 6, 10 > 6
AVG OOTP 2025-2029: 1 < 3, 30 < 4, 50 < 5, 22 < 6, 4 > 6

So - what do I see? Oddly, I'm seeing less extreme awesome pitching than I expected (11 with less than 3.00 ERA in real life vs. an average of 4 and 1 in the simulation). Beyond that, I'm seeing exactly what I surmised. While there's an even distribution between 3.XX ERAs and 4.XX ERAs in real life, in OOTP there are a great many more 4.XX ERAs than 3.XX (1.67x to almost double). Beyond that, there's a 2.5x uptick in horrible pitchers - guys that have an era of 5.00 or more. There's a slight uptick in pitcher's who pitch 150 innings+ in the OOTP sim - I can deal with that. While it seems at the macro level everything is just fine (league ERA and .AVG are in-line with what we expect) the distribution of ERAs is much flatter in real life than it is in the simulation. Therefore, in real life, there are a great many more serviceable pitchers than in OOTP.

In my OOTP sims, an average of ~30% of the starters have an ERA above 5.00. In real life, this is about 11%.

The AI definitely skews this though, as a pitcher, unless getting absolutely dominated, will stay in a rotation for the AI with an ERA of 5+ while in real life those guys wouldn't be allowed to pitch 150+ innings too often.

[injury log]

There's a methodological issue here though; the pitchers in real life who actually have the opportunity to pitch 150 innings are going to be the ones who have been pitching well, and thus normally the guys who have benefited from good luck; if a real life guy has an ERA of 7.50 after 100 innings, he won't be given a chance to throw 50 more innings. A pitcher with a 4.00 ERA in real life might actually be a 'true' 5.00 ERA pitcher in ability, but may have just been lucky for a few starts. That guy will keep playing. The 'true 5.00 ERA' starter who is unlucky, and posts a 6.00 ERA, gets cut or demoted. In OOTP, that isn't necessarily the case; if the guy has decent ratings, and you use ratings as part of your AI eval, this guy might continue to play even with bad results.

To put it another way, say you have 20 pitchers who are 'true 4.50 ERA' guys. After 100 innings

* In real life, half of these guys will be 'lucky' and put up an ERA better than 4.50. They'll all keep their jobs. Because they've been lucky, while you'd expect regression to their normal 4.50 ERA, most will end up with ERAs better than 4.50 over 150 IP, because of their good luck in the first two thirds of the year. Now, half of these guys will be 'unlucky' and post ERAs worse than 4.50. A lot of these guys won't live to see 150 IP in the Majors; they'll be released, demoted to the bullpen or shipped to AAA. They don't show up in your results.

* In OOTP, on the other hand, whether these 20 guys are lucky or not, they still are rated like no. 4 starters. If the AI is taking that into account, the AI is going to keep sending these guys out every fifth day. They'll all reach the 150 IP threshold, good luck or no. The ones who were 'unlucky' are going to end up in your 5+ or even 6+ ERA bins. This isn't revealing much about the distribution of pitcher ability; it's revealing more about how the AI is selecting who should play.

You might notice there's a larger number of pitchers reaching 150 IP in your OOTP sim than in real life. That needs to be accounted for.

[Craig Scarborough]

Originally Posted by injury log

There's a methodological issue here though; the pitchers in real life who actually have the opportunity to pitch 150 innings are going to be the ones who have been pitching well, and thus normally the guys who have benefited from good luck; if a real life guy has an ERA of 7.50 after 100 innings, he won't be given a chance to throw 50 more innings. A pitcher with a 4.00 ERA in real life might actually be a 'true' 5.00 ERA pitcher in ability, but may have just been lucky for a few starts. That guy will keep playing. The 'true 5.00 ERA' starter who is unlucky, and posts a 6.00 ERA, gets cut or demoted. In OOTP, that isn't necessarily the case; if the guy has decent ratings, and you use ratings as part of your AI eval, this guy might continue to play even with bad results.

To put it another way, say you have 20 pitchers who are 'true 4.50 ERA' guys. After 100 innings

* In real life, half of these guys will be 'lucky' and put up an ERA better than 4.50. They'll all keep their jobs. Because they've been lucky, while you'd expect regression to their normal 4.50 ERA, most will end up with ERAs better than 4.50 over 150 IP, because of their good luck in the first two thirds of the year. Now, half of these guys will be 'unlucky' and post ERAs worse than 4.50. A lot of these guys won't live to see 150 IP in the Majors; they'll be released, demoted to the bullpen or shipped to AAA. They don't show up in your results.

* In OOTP, on the other hand, whether these 20 guys are lucky or not, they still are rated like no. 4 starters. If the AI is taking that into account, the AI is going to keep sending these guys out every fifth day. They'll all reach the 150 IP threshold, good luck or no. The ones who were 'unlucky' are going to end up in your 5+ or even 6+ ERA bins. This isn't revealing much about the distribution of pitcher ability; it's revealing more about how the AI is selecting who should play.

You might notice there's a larger number of pitchers reaching 150 IP in your OOTP sim than in real life. That needs to be accounted for.

A few things:

#1 OOTP averaged about 10% more pitchers who threw 150 innings per year (approximately 10 pitchers). If I take them out the 10 worst pitchers from my findings, the distribution is still out of whack.

#2 While I agree some of this may be an AI issue because the AI keeps trotting out these same horrible pitchers, where are these guys who are actually "good" being kept if the OOTP distribution is the same as real life baseball? I assume the AI is putting the best guys out there based on their ratings. Therefore, these must be the best pitchers in the league - it doesn't make sense why then they would consistently be worse than the worst pitchers in real life.

#3 Are you saying then, the OOTP AI defaults are not the way to go to get an experience most like real life?

#4 If the OOTP AI is out of whack like this, wouldn't we also see this occurring from batters as well (i.e. trotting out hitters that are consistently worse than league average)?

[injury log]

Originally Posted by Craig Scarborough

#1 OOTP averaged about 10% more pitchers who threw 150 innings per year (approximately 10 pitchers). If I take them out the 10 worst pitchers from my findings, the distribution is still out of whack.

There are a few possible causes; you might not have enough injuries, for example, or the AI might be overusing some pitchers.

Originally Posted by Craig Scarborough

#2 While I agree some of this may be an AI issue because the AI keeps trotting out these same horrible pitchers, where are these guys who are actually "good" being kept if the OOTP distribution is the same as real life baseball? I assume the AI is putting the best guys out there based on their ratings. Therefore, these must be the best pitchers in the league - it doesn't make sense why then they would consistently be worse than the worst pitchers in real life.

These are the 'good' pitchers; they're just having bad luck. Or at least that's a possibility I can't discount from the data you've presented.

When I first read your post, my instinctive response was to reject the premise outright; my impression is that real life teams pay a premium for 'serviceable' starting pitching, precisely because it's in such short supply. Jason Marquis gets a generous contract in the offseason, for example, even though he isn't very good. Now, my intuition here might be far off base, but I'd need to see different data to be convinced that OOTP's talent distribution is far from that of real life baseball.

Originally Posted by Craig Scarborough

#3 Are you saying then, the OOTP AI defaults are not the way to go to get an experience most like real life?

Well, it would be remarkable coincidence if they were. There are two considerations that need to be balanced in the game. Suppose, in real life, someone just past prospect age - say Clay Buchholz or Phil Hughes - puts up an ERA of 6.00 in his first 60 innings. BOS or NYY would likely send the guy to AAA to figure things out, or perhaps send the guy to the pen. Either way, he's not getting 150 innings this year. There are a lot of factors that would go into these real life decisions; what effect the pitcher's performance might have on his psychology, for example.

Now, in OOTP, with the AI eval settings, you have to balance two considerations: will the AI continue to use the player (taking into account his ratings predict he should be better than his performance), and what will the AI want in exchange for the player in trade? If stats are weighted too heavily, the AI will give away these players too cheaply, and if ratings are weighted too heavily, players who are not performing well will still see a lot of playing time. I'd be all in favour of having separate settings which determine who the AI chooses to play, and how much value the AI attaches to players when contemplating transactions, but we don't have that in the game yet.

Originally Posted by Craig Scarborough

#4 If the OOTP AI is out of whack like this, wouldn't we also see this occurring from batters as well (i.e. trotting out hitters that are consistently worse than league average)?

Possibly, though I'd expect real life managers treat pitchers differently from batters. I'd certainly need to look at data to say anything for sure, but I wouldn't be at all surprised to discover that there is more turnover among pitchers in real life than among batters.

[Jaxxvain]

I have always thought that OOTP has too many really bad players (hitters and pitchers). Perhaps it is because these players are getting "unlucky" but are still rated fairly well.

In real life it seems like there are a handful of studs, a bit higher amount of "good" players, a boatload of mediocre players and then a few (2 or 3 per team) of AAAA types that go up and down, don't play all that much types.

In OOTP is seems like there are a handful of studs (bit more than real life), a bunch of "good" players (less than real life), some mediocre players (severely lacking here) and a HUGE amount of AAAA players (who get significant play time).

I don't play with the modifiers much, since I can't really grasp how to, but I see this in most leagues I play in, both online and solo.

[Craig Scarborough]

Originally Posted by injury log

There are a few possible causes; you might not have enough injuries, for example, or the AI might be overusing some pitchers.



These are the 'good' pitchers; they're just having bad luck. Or at least that's a possibility I can't discount from the data you've presented.

This is why, with the OOTP data, I took a 5-year average. Wouldn't this "luck" have evened out because of this? Using the 5-year average there are consistently more "bad" pitchers used in OOTP than in real life. If I'm using a 5-year average, wouldn't there be the same amount of pitchers who receive "good" luck?

Luck, by definition, is centered on the player's real value, otherwise it's not luck at all.

[Craig Scarborough]

Originally Posted by injury log

Now, my intuition here might be far off base, but I'd need to see different data to be convinced that OOTP's talent distribution is far from that of real life baseball.

What data would you be looking for here? I'm a little exasperated that presenting a 5-year average of OOTP data that consistently shows a different distribution than real life is not convincing.

You throw "luck" in there as an explanation, but if it was due to luck wouldn't I then see some years where the pitchers distributions are generally better than real life major leagues? I'm not.

Help me see what I'm missing.

[CoachDirty]

Originally Posted by Craig Scarborough

A few things:

#1 OOTP averaged about 10% more pitchers who threw 150 innings per year (approximately 10 pitchers). If I take them out the 10 worst pitchers from my findings, the distribution is still out of whack.

#2 While I agree some of this may be an AI issue because the AI keeps trotting out these same horrible pitchers, where are these guys who are actually "good" being kept if the OOTP distribution is the same as real life baseball? I assume the AI is putting the best guys out there based on their ratings. Therefore, these must be the best pitchers in the league - it doesn't make sense why then they would consistently be worse than the worst pitchers in real life.

#3 Are you saying then, the OOTP AI defaults are not the way to go to get an experience most like real life?

#4 If the OOTP AI is out of whack like this, wouldn't we also see this occurring from batters as well (i.e. trotting out hitters that are consistently worse than league average)?

First of all, I'm not saying your research is wrong, just commenting on point #2. I would guess that in real Baseball, if a pitcher is struggling (ERA 6.00+ after 75-100 innings) they will move him out of the rotation and try someone else. They may think/scout the original pitcher as being better, but they will try the one the perceive as the lesser pitcher, simply because the original isn't getting the job done. So in real life, neither "bad pitcher" gets to 150 innings.

The AI, on the other hand, depending on how you have set your evaluation modifiers, will generally play who it perceives as the best player, even as he continues to get killed.


Again, I'm not disputing your original conjecture, just pointing out what might be a flaw with your logic in #2.

[Craig Scarborough]

Originally Posted by CoachDirty

First of all, I'm not saying your research is wrong, just commenting on point #2. I would guess that in real Baseball, if a pitcher is struggling (ERA 6.00+ after 75-100 innings) they will move him out of the rotation and try someone else. They may think/scout the original pitcher as being better, but they will try the one the perceive as the lesser pitcher, simply because the original isn't getting the job done. So in real life, neither "bad pitcher" gets to 150 innings.

The AI, on the other hand, depending on how you have set your evaluation modifiers, will generally play who it perceives as the best player, even as he continues to get killed.


Again, I'm not disputing your original conjecture, just pointing out what might be a flaw with your logic in #2.

Well, in a no scout or coach league, wouldn't the AI have an honest evaluation of the pitcher? Meaning - since the AI sees the actual ratings for those pitchers, aren't they then the best he has to offer?

I'm all for changing the modifiers if that would give me a more true to life experience (if that's what you're getting at).

[CoachDirty]

Originally Posted by Craig Scarborough

Well, in a no scout or coach league, wouldn't the AI have an honest evaluation of the pitcher? Meaning - since the AI sees the actual ratings for those pitchers, aren't they then the best he has to offer?

I'm all for changing the modifiers if that would give me a more true to life experience (if that's what you're getting at).

I believe yes (though Marcus may correct me on that), and that's my point. The AI will continue to use the "best it has to offer" even though that player is getting pounded. By doing so, that allows him to get to 150 innings. Where in real life, said pitcher would/may be pulled and a "worse pitcher" used, but neither of them would get 150 innings, and thereby not appear in the data.

Again, I'm not saying your original premise is wrong. In fact, I think the point about their only being a ten pitcher difference is strong evidence in your favor.

[Craig Scarborough]

Originally Posted by CoachDirty

I believe yes (though Marcus may correct me on that), and that's my point. The AI will continue to use the "best it has to offer" even though that player is getting pounded. By doing so, that allows him to get to 150 innings. Where in real life, said pitcher would/may be pulled and a "worse pitcher" used, but neither of them would get 150 innings, and thereby not appear in the data.

Again, I'm not saying your original premise is wrong. In fact, I think the point about their only being a ten pitcher difference is strong evidence in your favor.

Ah, I see what you're saying now. You're right, trotting the same pitcher out there may skew the number of 150 inning pitchers out there. However, I knew no other way of shaking down the list so I'm not including every Tom, Dick and Harry.

I may try this test again with different AI parameters to see if that will solve it. As others have said, I'm concerned that there just isn't an even distribution of talent in an OOTP environment.

[billethius]

One other thing to note on the low end is that 2009 was somewhat of an anomoly with regards to the number of pitchers with an ERA under 3. Qucikly browsing baseball-reference, I see that there were 7 in 2008, 1 in 2007, and 2 in 2006.

[damientheomen3]

Originally Posted by Craig Scarborough

What data would you be looking for here? I'm a little exasperated that presenting a 5-year average of OOTP data that consistently shows a different distribution than real life is not convincing.

You throw "luck" in there as an explanation, but if it was due to luck wouldn't I then see some years where the pitchers distributions are generally better than real life major leagues? I'm not.

Help me see what I'm missing.

If you don't edit the league totals from time to time (to create random spikes in offense/pitching) then no, you won't see this in the league averages.

[Craig Scarborough]

Originally Posted by damientheomen3

If you don't edit the league totals from time to time (to create random spikes in offense/pitching) then no, you won't see this in the league averages.

I don't understand what you're saying here. I'm saying that if the reason for having more poor pitchers pitch is due to random luck (meaning, they are actually not awful but performing that way due to luck) wouldn't I see the opposite occur just as often. The fact is, I'm not - so I'm discounting luck as a reason for this.

[RchW]

Originally Posted by Jaxxvain

I have always thought that OOTP has too many really bad players (hitters and pitchers). Perhaps it is because these players are getting "unlucky" but are still rated fairly well.

In real life it seems like there are a handful of studs, a bit higher amount of "good" players, a boatload of mediocre players and then a few (2 or 3 per team) of AAAA types that go up and down, don't play all that much types.

In OOTP is seems like there are a handful of studs (bit more than real life), a bunch of "good" players (less than real life), some mediocre players (severely lacking here) and a HUGE amount of AAAA players (who get significant play time).

I don't play with the modifiers much, since I can't really grasp how to, but I see this in most leagues I play in, both online and solo.

I don't disagree but would put it another way. There are a huge number of declining players. This is the nut and bolts in the game that mimics, with varying degrees of success, the talent distribution IRL. The boatload of mediocre players you note above are declining or have declined leaving the good players and the studs to dominate.

It's another reason you can dominate solo leagues. How often have you seen the AI shell out big bucks long term to someone you know has 2-3 good years left? I'm sure most of us turn over our Fictional rosters way more than the AI.

[RchW]

Originally Posted by damientheomen3

If you don't edit the league totals from time to time (to create random spikes in offense/pitching) then no, you won't see this in the league averages.

Hmmm... I edit league total modifiers (not the totals themselves) to mitigate random spikes. I know you can do what you say, however in my experience not editing LTM's will result in random spikes due to injuries varying draft classes and talent distribution. This is desirable from time to time.

[RchW]

Originally Posted by Craig Scarborough

Ah, I see what you're saying now. You're right, trotting the same pitcher out there may skew the number of 150 inning pitchers out there. However, I knew no other way of shaking down the list so I'm not including every Tom, Dick and Harry.

I may try this test again with different AI parameters to see if that will solve it. As others have said, I'm concerned that there just isn't an even distribution of talent in an OOTP environment.

This may muddy the waters more. I can't speak for v11 as I have yet to put any serious time into it but the defensive part of the player creation model in all previous versions could have a big effect on pitching. I'm going to sound like a broken record here as I've been whining about this for 3 years.

Simply put, back up infielders (with the possible exception of 1B) in real life are usually better than average defensively. IRL, there is no possible way you would take a weak hitter/weak fielder over a weak/hitter good/excellent fielder. In the OF you see this from time to time if the hitter is too good to pass up. You can hide an OF occasionally.

In OOTP (don't know about v11) the opposite is true. the distribution of defensive skills for poor hitting IF appears to be random when it should favor fielding.

I wonder if average or below average pitchers get even worse results because of the flawed defensive talent distribution?

[injury log]

Originally Posted by Craig Scarborough

This is why, with the OOTP data, I took a 5-year average. Wouldn't this "luck" have evened out because of this? Using the 5-year average there are consistently more "bad" pitchers used in OOTP than in real life. If I'm using a 5-year average, wouldn't there be the same amount of pitchers who receive "good" luck?

Luck, by definition, is centered on the player's real value, otherwise it's not luck at all.

But the 'unlucky' pitchers aren't showing up in your real life average at all, because they don't reach your 150 IP threshold - it doesn't matter if you average over two years or five years.

I'm interested in this question now, so when I have a chance I'll try to suggest a different methodology that might be used to investigate this.