OOTP Developments Forums - View Single Post - Ratings vs Results

chazzycat · 03-06-2019, 06:00 PM

DonkeyKongSr - this was inspired by our chat on Discord yesterday.

I exported the data available from the mini-whale perfect league, each dot representing a player's entire career - but with a minimum 1,000 PA threshold. Total PA represented in the sample is around 1.6 million.

First off, as I suspected, the relationship between Contact Rating and BABIP becomes much clearer with this large of a sample:

It's not the strongest relationship, but certainly there. However the next graph is more enlightening:

As you can see, the Contact rating has a much stronger relationship with batting average than it does with BABIP.

This is important because it lends some credence to the theory that strikeouts are not separated from other outs in the game's calculation until after it's been decided whether it will be an out or not. To back up for a moment, there are two competing theories that could explain how contact and "avoid K" ratings work in the game. It's basically just a question of the order-of-operations. Here are the two theories:

1) The game calculates first whether a player will strike out, by the avoid K rating. For all the PAs where the player doesn't strike out, walk, HBP etc., then those fall into a "BIP" bucket. Only then would the game apply the BABIP calculation to determine if it's a hit or not (determined by the CON rating, luck, defense, etc.).

2) The game calculates first whether a player will get out or not. After it makes that determination, it then determines a strikeout vs. other types of outs based on the "Avoid K" rating. So having a high "Avoid K" rating in effect just moves K's to groundouts or flyouts.

If theory #1 was true, then we would expect to see the CON rating have a very strong relationship with BABIP, significantly stronger than it does with AVG. AVG includes strikeouts, which don't apply to BABIP. So removing the strikeouts from the equation should show a stronger relationship between CON & BABIP, if that theory was true.

But, instead we see the opposite. To me, this alone strongly implies that it's theory #2 which is correct.

I didn't stop there though. We can also compare the "avoid K" rating to real results for more direct observations:

Ouch. Strike two. If theory #1 was correct, we should also see a much stronger relationship between avoid K and AVG. This is close to zero relationship. However, it would be going too far to claim zero correlation. It is worth noting that there may be a very small effect here.

Recall the graph above where CON and AVG have a strong relationship, with an R-squared value of .54. This is roughly ten times as strong as the relationship AVG has with Avoid K's. So we can crudely estimate that upgrading the CON rating is about 10X more effective per point than upgrading the Avoid K rating, when it comes to batting averages.

Moving on, let's see how Avoid K rating effects overall offensive production. wRC+ is the best stat to neutralize park effects etc., so that is what I will use.

Again, there is pretty much no relationship here. It appears as though strikeouts simply do not factor into overall output whatsoever.

For anyone thinking "ratings don't matter!"...well yes, they do. That's definitely not the point I am making here. For reference, here are how the other ratings correlate with wRC+:

Compared to Avoid K, the other batter ratings show very strong correlation with overall results. Of course if we looked at more precise stats for each rating (HR rate for POW, BB rate for eye, etc.) we would see MUCH stronger correlations. But it's reassuring that we still see significant correlations once the results have been watered down to a single number. And we can make logical decisions based on the R-squared values here. For example, the EYE rating has more effect on overall output than the POW rating. That may come as a surprise.

Note: I didn't include "GAP" in the graphs above because it's very confusing. For what it's worth the R-squared is .33. However I suspect that is mostly due to the fact that high-GAP players also have high CON ratings. I certainly would not conclude that GAP is more important than EYE or POW. It's hard to say for sure.

I want to say one last thing with regards to "Avoid K": just because it doesn't impact wRC+, doesn't make it useless. That stat (nor any others) don't include the benefit of moving runners over, which could potentially be quite important. Unfortunately, that is impossible to quantify.

Any feedback is appreciated!

03-06-2019, 06:00 PM	#1
chazzycat All Star Starter Join Date: Dec 2014 Posts: 1,685	Ratings vs Results - Avoid K Rating DonkeyKongSr - this was inspired by our chat on Discord yesterday. I exported the data available from the mini-whale perfect league, each dot representing a player's entire career - but with a minimum 1,000 PA threshold. Total PA represented in the sample is around 1.6 million. First off, as I suspected, the relationship between Contact Rating and BABIP becomes much clearer with this large of a sample: It's not the strongest relationship, but certainly there. However the next graph is more enlightening: As you can see, the Contact rating has a much stronger relationship with batting average than it does with BABIP. This is important because it lends some credence to the theory that strikeouts are not separated from other outs in the game's calculation until after it's been decided whether it will be an out or not. To back up for a moment, there are two competing theories that could explain how contact and "avoid K" ratings work in the game. It's basically just a question of the order-of-operations. Here are the two theories: 1) The game calculates first whether a player will strike out, by the avoid K rating. For all the PAs where the player doesn't strike out, walk, HBP etc., then those fall into a "BIP" bucket. Only then would the game apply the BABIP calculation to determine if it's a hit or not (determined by the CON rating, luck, defense, etc.). 2) The game calculates first whether a player will get out or not. After it makes that determination, it then determines a strikeout vs. other types of outs based on the "Avoid K" rating. So having a high "Avoid K" rating in effect just moves K's to groundouts or flyouts. If theory #1 was true, then we would expect to see the CON rating have a very strong relationship with BABIP, significantly stronger than it does with AVG. AVG includes strikeouts, which don't apply to BABIP. So removing the strikeouts from the equation should show a stronger relationship between CON & BABIP, if that theory was true. But, instead we see the opposite. To me, this alone strongly implies that it's theory #2 which is correct. I didn't stop there though. We can also compare the "avoid K" rating to real results for more direct observations: Ouch. Strike two. If theory #1 was correct, we should also see a much stronger relationship between avoid K and AVG. This is close to zero relationship. However, it would be going too far to claim zero correlation. It is worth noting that there may be a very small effect here. Recall the graph above where CON and AVG have a strong relationship, with an R-squared value of .54. This is roughly ten times as strong as the relationship AVG has with Avoid K's. So we can crudely estimate that upgrading the CON rating is about 10X more effective per point than upgrading the Avoid K rating, when it comes to batting averages. Moving on, let's see how Avoid K rating effects overall offensive production. wRC+ is the best stat to neutralize park effects etc., so that is what I will use. Again, there is pretty much no relationship here. It appears as though strikeouts simply do not factor into overall output whatsoever. For anyone thinking "ratings don't matter!"...well yes, they do. That's definitely not the point I am making here. For reference, here are how the other ratings correlate with wRC+: Compared to Avoid K, the other batter ratings show very strong correlation with overall results. Of course if we looked at more precise stats for each rating (HR rate for POW, BB rate for eye, etc.) we would see MUCH stronger correlations. But it's reassuring that we still see significant correlations once the results have been watered down to a single number. And we can make logical decisions based on the R-squared values here. For example, the EYE rating has more effect on overall output than the POW rating. That may come as a surprise. Note: I didn't include "GAP" in the graphs above because it's very confusing. For what it's worth the R-squared is .33. However I suspect that is mostly due to the fact that high-GAP players also have high CON ratings. I certainly would not conclude that GAP is more important than EYE or POW. It's hard to say for sure. I want to say one last thing with regards to "Avoid K": just because it doesn't impact wRC+, doesn't make it useless. That stat (nor any others) don't include the benefit of moving runners over, which could potentially be quite important. Unfortunately, that is impossible to quantify. Any feedback is appreciated! Last edited by chazzycat; 03-06-2019 at 06:19 PM.