Catcher Influence

[RonCo]

Originally Posted by Eumel

a ZR of -30 isn't really *that* bad either an can conceivably be compensated with great batting...

Just to be clear, I think it's probably true that non-catchers should be more heavily penalized. But from what I see here it seems that the basic structure of catcher defense is pretty well done.

Don't know if you saw it, but I did a mega-opus on OOTP defense some time back that gave me some comfort about defense on the whole, though as in any simluation there are always uncertainties. Its still out on the FOBL site someplace. I can see if I can find a link sometime.

Again saying that non-catchers probably need to be penalized more, I think there is some chance here that Markus has the structure right, and the numbers "right enough" that a false truism of baseball is having a coming out party--that being that the general public and baseball as a whole tend to vastly over-rate the influence of a catcher's defensive contribution. (Please note, this is _not_ saying catchers make no contribution at all), and that a big-stick behind the plate is probably worth some pretty poor defense.

If true, perhaps baseball itself will be changing its stripes on this matter. This could be a similar transition that the shortstop position has gone through over the past 30-40 years, but the difference is that (1) it's easier to find guys who _want_ to play shortstop, and (2) when teams get a good hitter, they often don't want to burden them with injury risk that catchers have, (3) neither the team, nor the player really like the idea of the shortened careers that can come with the physical beating it takes to play catcher.

One thing that I wonder about regarding catchers and OOTP is that I get the sense that the injury model is very light on catchers. I haven't done a full study on it, but I've seen several pieces of data that suggest the injury risk of catching is not overly great. This is something that swings the balance of these decisions, too...even if the defensive model is too lenient, do you want to risk a good-hitting OF behind the plate when he will almost certainly get dinged up? If the injury rate isn't great enough then the answer is more likely to be "yes."

Anyway, the bottom line, as GMLoophole said, is that from an overall perspective it can be said that purely defensive catchers are essentially useless in OOTP. The correlary, as I'll paraphrase from RchW, is that it's fairly obvious from the Economists' way teams buy and sell them, that purely defensive catchers are essentially worthless in real life, also.

I and other keep talking about Mike Piazza, but he's a HoFer...so look at guys today that fit the mold. AJ Pierzynski is a poor defensive catcher with a sub-par arm, but has had a long career because his offensive value is at or just below league average. Jorge Posada is a below average defensive catcher whose RTO% has consistently been cruddy, but he's good for a 130 or so OPS+ on average. Of course, you would love to have the Carlos Ruiz's and Ivan Rodgriguez's of the world who will give you production bot offensively and defensively, but it's fairly rare these days to find a purely defensive catcher who holds onto the position for very long (and when you do, I suspect you find a pretty bad team, or a team who has decided to save $ and just go with what they've got for now).

Anyway...blah, blah, blah.

Originally Posted by Eumel

I may look at this again, but back then I didn't really have the impression that this research could have any impact on how fielding is modelled in the game as the powers that be didn't really seem to care...

Markus does care, but this is admittedly a very complex situation, and I find that sometimes he would prefer to work on things that area a bit more clear-cut.

[Eumel]

Originally Posted by RonCo

Just to be clear, I think it's probably true that non-catchers should be more heavily penalized. But from what I see here it seems that the basic structure of catcher defense is pretty well done.

Yes, I agree with this. I would like to see a non-linear rating scale, though. Not just for catchers, but also for other positions, where very low ratings are penalized more. In other words, the value of a rating point at the lower end of the scale needs to be bigger than at the top end. I think it's pretty clear that the value of catching skills is often overestimated, but we are talking about the differences between skilled catchers there. The overall ZR span between a weak regular catcher and a strong one also seemed a tad too small in OOTP, but that's a minor quibble. If an emergency catcher ended up at -50 a year, then I'd be happy. I'm certainly not lobbying for a gamecalling rating or anything like that.

Don't know if you saw it, but I did a mega-opus on OOTP defense some time back that gave me some comfort about defense on the whole, though as in any simluation there are always uncertainties. Its still out on the FOBL site someplace. I can see if I can find a link sometime.

Yeah, I remember that and found it an interesting read. I've been lurking around on here for years....

Anyway, the bottom line, as GMLoophole said, is that from an overall perspective it can be said that purely defensive catchers are essentially useless in OOTP. The correlary, as I'll paraphrase from RchW, is that it's fairly obvious from the Economists' way teams buy and sell them, that purely defensive catchers are essentially worthless in real life, also.

Yes, but as it stands now, even average offensive catchers aren't worth much in OOTP. This may be a little over the top. Other than that I agree.

Markus does care, but this is admittedly a very complex situation, and I find that sometimes he would prefer to work on things that area a bit more clear-cut.

I'd be happy with some number tweaking here. Have to take the plunge into OOTP12 soon anyway

[injury log]

Originally Posted by RonCo

I took a four-team league and turned off development and set all ratings to "100" everywhere possible based on the roster export (which is nearly everything). I turned off fatigue and injuries, and I forced the AI to not make any changes.

Unfortunately, no one is running four team leagues in which everyone has average basestealing ratings and 1/4 of the teams are fielding incompetent catchers. Already from your data the 50% CS rates of the average defensive catchers should stand out as a problem. Those absurdly high CS% ratings don't occur in typical leagues, and nor do the absurdly bad defensive stats of incompetent catchers - your results are nothing at all like what you would see if you put your CF in as a regular catcher in a 30 team league with typical talent distribution. Instead you'll see a CS% rate around 25%, and PB rates in line with those of the other bad fielding catchers in the league.

[RonCo]

Originally Posted by injury log

Unfortunately, no one is running four team leagues in which everyone has average basestealing ratings and 1/4 of the teams are fielding incompetent catchers. Already from your data the 50% CS rates of the average defensive catchers should stand out as a problem. Those absurdly high CS% ratings don't occur in typical leagues, and nor do the absurdly bad defensive stats of incompetent catchers - your results are nothing at all like what you would see if you put your CF in as a regular catcher in a 30 team league with typical talent distribution. Instead you'll see a CS% rate around 25%, and PB rates in line with those of the other bad fielding catchers in the league.

Sigh.

[Tycobbler]

A catcher is supposed to help on the defensive end. Calling pitches, throwing out stealing runners, ability to prevent passed balls and wild pitches, chemistry with the pitcher, and, of course, hitting.

Those factors working in conjunction influence the game and make the pitcher look good.

[RonCo]

Originally Posted by injury log

Unfortunately, no one is running four team leagues in which everyone has average basestealing ratings and 1/4 of the teams are fielding incompetent catchers. Already from your data the 50% CS rates of the average defensive catchers should stand out as a problem. Those absurdly high CS% ratings don't occur in typical leagues, and nor do the absurdly bad defensive stats of incompetent catchers - your results are nothing at all like what you would see if you put your CF in as a regular catcher in a 30 team league with typical talent distribution. Instead you'll see a CS% rate around 25%, and PB rates in line with those of the other bad fielding catchers in the league.

After doing some additional testing this evening using my little 4-team league, I think the problem may be related to whether the player being used at "C" actually has his position changed to "C" or has it remain at "CF" (or whatever).

When I put a CF in the C slot, and leave his position as CF, he appears to perform considerably worse than he does if I change his position to "C."

If this is the case, online leagues should probably put a house rule in place that says if a position player is put in as a C, that player is not allowed to have their position set to "C" until the player's rating is 2 or three or whatever...

[Eumel]

Hm, this make a lot of sense. I remember that I set the positions to C so I could filter out the stats more easily. I had one guy with -20 or so (and constantly, for several sims) for no reason that I could explain. Maybe I forgot to change the position for him? -20 over 1100 innings is still not quite enough in my eyes, but certainly better than the other results.

[RonCo]

Yes, I agree that the penalty should probably be a bit more, but at this point I'm mostly just looking for the cause of the discrepancy between your results and mine, and I am pretty sure that's it. I'll post some data if I get a little chance to write it up. Bottom line, though, it appears that if you change the position of a non-C to C the player performs like a low-end C--and low-end Cs can sometimes perform very well just due to sample size (100 SB attempts is really a small sample size that can mask skills). But if you leave their position as "CF" or whatever, then there's a deeper penalty to pay.

[ELDoro]

Its pretty much a myth that MLB catchers don't call games anymore. Many believe this because they see the catcher looking into the dugout after every pitch and getting signs.

Most of those signs are garbage so when the bench is really calling a pickoff play, defense bunt coverage, ect that it doesnt stand out that they are doing something different.

I would actually go in the opposite direction and say that most MLB catchers DO call pitches.

[Eumel]

I've just tried a sim without setting the position to catcher. The results aren't what we would have expected. Again, standard fictional league, no injuries, no player development, no morale, etc.

1133 Inn., ZR -2.3, RTO% 35.3, 16 PB, 16 E
1107.2 Inn., ZR -5.7, RTO% 30.3, 11 PB, 14 E
1117.2 Inn., ZR -7.1, RTO% 25.5, 13 PB, 13 E
1162.2 Inn., ZR -9.5, RTO% 25.5, 11 PB, 11 E
1211.1 Inn., ZR -12.9, RTO% 24.8, 14 PB. 16 E

So that was not it... these guys all have arm and catching skill ratings of 1 or 2 out of 20 and no experience at catcher during the whole season, as player development was disabled. The RTO% are still *way* too high.

[RonCo]

Actually, those neither surprise me, would I have predicted them. How are they in comparison to the rest of the league? I'm going to publish a table of expected "statistical penalties" for using Cs with low ratings and Cs of other positions. My guess based on my data is that if the average RTO% of a C who is actually a non-catcher is 27% (which this looks to be), then the RTO% of the average C in that league is probably 36% or so.

If you change the non-C catchers to the "C" position, my guess is that their RTO% will rise to about 31% on average.

So, yes, I think Markus needs to make the penalty higher...but at least I'm pretty sure I understand the fundamental way the game is appying its penalty structure.

[Eumel]

These guys are the worst catchers in the league (which is a good thing).

The others have ZR between -1.8 (rated 11/20) and +11 (rated 19/20). The average RTO% is more like 38%, because there are a few outliers above 40%. The top guy has 45%.

This season seems to be a bit of an anomaly though because there are lots of good catchers around -- half of the other starters are rated 18/20 or better overall. There's one guy with 11/20, the worst regular catcher. He ended up with 33%.

[Eumel]

Here's another season, this time the numbers look a bit better:

POS	Name	        C	PA	wOBA	VORP	G	GS	TC	E	ZR	EFF	SBA	RTO	RTO%	IP	PB	CERA
C	Soto	        -	588	.249	-8.1	141	141	1147	21	-20.2	.838	153	35	22.9	1243.0	22	4.04
C	Alvarado	-	586	.328	30.2	141	141	1070	29	-16.4	.944	173	35	20.2	1258.0	17	4.78
C	Villanueva	-	562	.313	23.0	141	141	1202	28	-10.9	1.002	189	49	25.9	1255.1	26	4.22
C	Medina	        -	566	.295	18.9	139	139	959	22	-10.5	.929	113	28	24.8	1197.2	15	4.34
C	Vélez	        8	636	.355	43.8	142	142	967	14	-10.1	1.015	144	31	21.5	1246.1	14	4.23
C	Webster		-	584	.320	32.8	141	141	957	16	-7.5	.930	174	50	28.7	1192.2	13	3.76
C	Medina		-	555	.271	5.3	142	142	1115	18	-7.2	.904	153	45	29.4	1165.1	12	4.15
C	Dunne		-	593	.299	21.9	142	142	1083	21	-5.1	1.062	153	39	25.5	1226.0	13	3.58
C	Lee	        19	487	.279	8.4	124	120	886	7	-2.3	1.029	61	17	27.9	1070.2	11	4.07
C	Ortíz	        18	205	.286	.3	45	40	322	5	-2.2	1.045	38	8	21.1	352.2	3	5.41
C	Green	        -	564	.268	2.9	141	141	1150	14	-1.7	.945	147	47	32.0	1241.0	6	3.75
C	Romano		14	215	.296	6.6	56	42	394	4	-1.5	.919	36	11	30.6	419.1	2	3.63
C	Hensley		12	129	.207	-3.7	33	24	153	0	-1.4	1.045	24	8	33.3	236.2	8	3.69
C	Richardson	13	194	.315	7.3	55	40	367	5	-1.1	1.045	33	8	24.2	393.1	3	5.17
C	Dye	        10	187	.309	8.3	58	40	321	3	-0.9	.990	19	6	31.6	382.2	4	4.73
C	Hogan		15	24	.233	-0.9	11	4	47	0	-0.9	1.045	7	1	14.3	41.2	1	3.02
C	García		10	112	.275	1.1	58	20	229	4	-0.8	1.045	35	9	25.7	230.1	1	4.30
C	Kippersluis	14	95	.334	5.8	19	1	49	1	-0.7	.696	1	0	.0	47.2	0	1.70
C	Alexander	15	94	.322	6.6	11	1	22	0	-0.5	1.045	3	0	.0	26.2	0	5.40
C	Johnson		15	98	.310	7.3	11	1	25	0	-0.2	1.045	0	0	.0	25.2	1	1.05
C	Fields	        10	102	.334	6.3	34	20	209	1	-0.2	1.045	8	3	37.5	219.0	3	2.96
C	Caldwell	16	107	.237	-2.4	26	20	176	1	-0.1	1.045	17	5	29.4	182.0	1	4.05
C	Wilson		16	89	.355	7.3	20	16	121	2	-0.1	1.045	24	7	29.2	147.1	1	4.64
C	Shannon		19	53	.223	-0.8	11	1	21	0	-0.0	.000	2	1	50.0	25.1	1	4.26
C	McKeag		19	39	.193	-1.4	2	1	11	0	0.0	.000	0	0	.0	12.1	0	2.92
C	Jefferson	5	126	.249	-8.2	57	19	201	2	+0.2	1.045	21	7	33.3	255.0	2	5.29
C	Smith	        10	59	.223	-6.9	11	5	59	0	+0.3	1.045	2	1	50.0	57.0	0	2.21
C	Gómez		14	36	.359	3.1	14	1	41	0	+0.6	1.045	1	1	100.0	37.1	0	2.89
C	Simmons		15	490	.243	-8.0	127	122	953	5	+0.6	1.096	97	28	28.9	1074.1	9	4.85
C	Carpegiani	18	94	.319	3.9	29	17	171	1	+0.7	1.045	13	5	38.5	167.0	1	3.50
C	Tannehill	15	129	.339	10.3	52	19	245	1	+0.9	1.045	22	8	36.4	239.2	1	2.48
C	Carlson		11	144	.260	-6.6	34	24	222	1	+1.1	1.045	13	6	46.2	235.1	2	2.79
C	Vivekanand	11	115	.266	.4	34	22	184	1	+1.5	1.045	16	7	43.8	217.1	0	2.98
C	Knobelsdorff 	17	138	.257	-6.4	39	20	181	1	+1.6	1.175	18	7	38.9	198.1	1	4.45
C	White	        16	482	.293	12.0	124	122	918	10	+2.9	1.014	95	36	37.9	1072.2	9	5.31
C	Evans	        16	137	.332	8.1	58	19	242	0	+3.0	1.144	22	9	40.9	244.0	1	3.91
C	House		8	565	.247	-3.7	144	143	928	15	+3.6	1.045	159	55	34.6	1186.0	10	4.97
C	Maldonado	20	528	.262	4.7	143	138	1079	6	+3.8	1.045	77	29	37.7	1230.2	7	4.40
C	Davis	        13	507	.312	19.5	129	122	962	7	+4.5	1.119	100	35	35.0	1108.2	6	4.33
C	Baker	        20	534	.233	-7.1	140	138	859	7	+7.7	1.091	50	25	50.0	1201.1	8	4.70

The zero-rating guys were all random bench players.

Edit: and I did not set their positions to C during the simulation

[Chicagofan76]

Originally Posted by ELDoro

Its pretty much a myth that MLB catchers don't call games anymore. Many believe this because they see the catcher looking into the dugout after every pitch and getting signs.

Most of those signs are garbage so when the bench is really calling a pickoff play, defense bunt coverage, ect that it doesnt stand out that they are doing something different.

I would actually go in the opposite direction and say that most MLB catchers DO call pitches.

Perhaps, but Pierzynski, Soto & Koyie Hill do not, and those are the oly 1's i can truly speak of. There is an article out there from last season where the Chicago Tribune interviewed both PC's and they both said they primarily call balls & strikes, and that Ozzie would call def. No word of what Senile Lou was calling.

[RonCo]

Executive Summary:

1) When used as a catcher, a player's set position influences Zone Rating, EFF, RTO% and apparently their pitching staff's BABIP. (i.e. changing the role from "CF" to "C" is enough to make a performance change, regardless of ratings).
2) When a non-catcher has their role set to "C," they appear to perform like a lower-graded catcher, which can oftentimes result in good on-field performances relative to known-good catchers.
3) Reduced C ratings influences Errors, DP, F%, ZR, EFF, RTO%, and possibly pitcher's BABIP.
4) Neither changing the role to "C" nor adjusting ratings caused much obvious change in the catcher's PB or WP allowed. This seems to be a basic flaw in the catcher defensive algorithm.

Another interesting tangential thing I learned: When you shut off development, fielding experience continues to accrue. This means fielding skill continued to get better over the five years I ran this study. Not surprisingly, BABIP steadily got lower (.335 -> .325 -> .318 -> .317 ->.318).

Details:

Using my dinky little 4-team league with all dev, injuries, coaching, scouts, and whatever set off and ratings set to 100 and ballparks neutral, and AI unable to make any changes, I ran a total of five seasons of data:

Year 1: No changes to base league. Catchers are all "normal"
Year 2: Swap CF and Catcher for one team. No ratings changes (all players rated 100 everywhere -- compare to year 1 to test if position change makes a difference)
Year 3: Change CF Position to "C" (still no ratings changes -- compare to year 1 to test if position change makes the CF look just like the C looked before
Year 4: Change CF back to "CF" and reduce catcher ratings to 0 Exp, 5 ability, 5 arm -- compare to 1 & 2 above to see the impact of both rating and poaition changes.
Year 5: Put Real "C" back to C and "CF" to CF, then adjust catcher ratings down to (0,5,5) -- compare to others to confirm ratings impact is consistent.

Note that in every case the KC catcher (first in the list) was the one that was adjusted.

Look, IL ... I understand that THE ACTUAL RESULTS OF THIS TEST WILL NOT GIVE YOU THE ACTUAL RESULTS OF A "REAL LEAGUE." That's not the point, nor has it ever been. The point is to study the magnitude of the performance difference between catchers in equal environments, and the only way to know that for certain is to ACTUALLY HOLD EVERYTHING CONSTANT THAT YOU CAN HOLD CONSTANT, and VARY ONLY THE THINGS YOU WANT TO STUDY.

What this means is that this test can tell us the relative performance drop you should see (relative to the performance level of actual catchers in the league). So if you know the performance of actual C-rated catchers in your league, you could then determine about what a non-C should perform at in your actual league. Pretty slick, eh? Well...I think it is, anyway. YMMV.

So, let's look at the results:

Here are runs #1 and #2:

#1
Baseline - All players rated 100 everywhere																						
	Team	POS	Exp	Abil	Arm	G	IP	TC	E	DP	PCT	RNG	ZR	EFF	SBA	RTO	RTO%	PB	ER	CERA	WP	BABIP
Peters	KC*	C	100	100	100	162	1455.1	1284	17	5	0.987	7.84	10	.950	133	64	48.1	8	734	4.54	79	0.33
Parker	Mesa	C	100	100	100	162	1427.1	1279	11	3	0.991	8	12.8	1.000	105	69	57.1	18	723	4.56	113	0.331
Petson	NO	C	100	100	100	162	1437.1	1282	13	5	0.99	7.95	9.1	.974	109	56	56	14	800	5.01	114	0.34
Stepens	Por	C	100	100	100	162	1454	1312	6	8	0.995	8.08	16	1.065	106	58	54.7	16	810	5.01	97	0.34

#2
Insert CF into C role (no rating changes)																						
	Team	POS	Exp	Abil	Arm	G	IP	TC	E	DP	PCT	RNG	ZR	EFF	SBA	RTO	RTO%	PB	ER	CERA	WP	BABIP
Lewis	KC*	CF	100	100	100	162	1439	1315	15	5	0.989	8.13	-3.8	.922	110	43	39.1	15	716	4.48	112	0.339
Parker	Mesa	C	100	100	100	162	1456.1	1276	3	2	0.998	7.87	15.3	1.056	82	52	63.4	19	672	4.15	90	0.318
Petson	NO	C	100	100	100	162	1447.1	1231	8	6	0.994	7.61	9.9	1.024	117	53	45.3	8	677	4.21	113	0.324
Stepens	Por	C	100	100	100	162	1452.1	1254	6	5	0.995	7.73	2.6	1.022	94	36	38.2	13	732	4.54	90	0.321

A few things to notice here. The CF in the C role makes about twice the number of errors, and has a Fielding % hit. His ZR drops about 12-13 points, and his EFF is a pretty major downward trend. Also, the CF's RTO drops considerably.

Of course, Portland's C dropped RTO%, too. But if we compare the average "C" with the "CF" we can say the "CF"'s RTO% value drops from 49% avg to 39%, about a 20% drop overall.

In addition, note a few other things--BABIP increases for the CF, but drops overall. WP vary all over the place, and PB don't seem to change at all. This will be a common theme. I could make an argument that a non-C in the role makes WP a little more likely, but not much if any. PB do not seem to change.

Overall, this says that there is a definite penalty to changing the player's position to CF, even if his actual ratings have not changed at all. But it seems that the impact could be bigger.

Now let's look at run #3. For this one we leave the CF in the role, but change his position to "C." If all things are equal, this means the league should look very similar to run #1.:

#3
Change CF position to C (no rating Changes)																						
	Team	POS	Exp	Abil	Arm	G	IP	TC	E	DP	PCT	RNG	ZR	EFF	SBA	RTO	RTO%	PB	ER	CERA	WP	BABIP
Lewis	KC*	C	100	100	100	162	1440.2	1250	12	5	0.99	7.73	11.3	1.049	105	49	46.7	9	745	4.65	116	0.315
Parker	Mesa	C	100	100	100	162	1445	1269	11	6	0.991	7.84	9.1	.979	104	53	51	11	704	4.38	94	0.32
Petson	NO	C	100	100	100	162	1449	1277	10	4	0.992	7.87	2.7	.981	132	49	37.1	7	710	4.41	101	0.313
Stepens	Por	C	100	100	100	162	1436.1	1289	11	3	0.991	8.01	3.8	.991	86	41	47.1	19	689	4.32	113	0.322

Indeed, there's very little between runs #1 and #3 to say the differences are beyond any random variance you owuld expect of evenly-matched teams and players. This run looks pretty similar to run #1, as expected.

So, let's have some fun with ratings. We'll change our test team's catcher position back to CF, and reduce his ratings to the 0,5,5 levels noted. This case is closer to a "real league" where a non-C would be placed into the role with zero experience and small C ratings. We would hope to see deeper penalties in this case than we saw in run #2.

#4
Change back to CF (Reduce C Ratings)																						
	Team	POS	Exp	Abil	Arm	G	IP	TC	E	DP	PCT	RNG	ZR	EFF	SBA	RTO	RTO%	PB	ER	CERA	WP	BABIP
Lewis	KC*	CF	0	5	5	162	1461.1	1258	25	3	0.98	7.59	-8.8	.849	195	62	31.8	20	772	4.75	105	0.323
Parker	Mesa	C	100	100	100	162	1471.1	1269	9	5	0.993	7.65	10.9	1.043	106	52	49.1	17	714	4.37	95	0.316
Petson	NO	C	100	100	100	162	1476.1	1276	6	4	0.995	7.74	11.8	.982	124	62	50	14	714	4.35	125	0.316
Stepens	Por	C	100	100	100	162	1469.1	1348	11	7	0.992	8.19	12.4	1.062	110	53	48.2	16	719	4.4	102	0.315

Indeed, we see that the combination of poor ratings and playing out of position cause larger penalties than merely playing a guy out of position. RTO%, for example drops from 49% to 31%, nearly a 40% total drop relative to average. In addition, we also see that runners attempt to steal about 80% more often due to the rag-arm behind the plate. Errors are nearly double, double plays are down (but very rare, still) and the gap between Fielding %, ZR, and EFF are all considerable greater. I note again that BABIP for our test team is noticeably higher than the rest of the teams. This makes me thing that a non-C actually does result in poorer pitcher performance...something I don't like, but I guess I can deal with, as the data we have about C performance in real life is all built around players who can actually catch.

Note again that I don't see much indication of WP or PB influence. These vary widely across all runs, probably due to their relative rarity (?). Dunno.

Learning 1: the penalty for non-C catchers appears to be cumulative, and comprised of a penalty for the position difference + the penalty for poor ratings.
Learning 2: this is the worst case, as it should be.
Learning 3: that said, I would tend to agree that the data posted in the other thread about the topic of the use of non-C, "emergency catchers" suggest that the penalty should be even deeper than it is.

Let's finish up by confirming Learning #1 by putting the original C back into place, but lowering his ratings to 0,5,5. We would expect poor performance, but not as poor as run #4.

#5
Put "Real" C back and change ratings down																						
	Team	POS	Exp	Abil	Arm	G	IP	TC	E	DP	PCT	RNG	ZR	EFF	SBA	RTO	RTO%	PB	ER	CERA	WP	BABIP
Peters	KC*	C	0	5	5	162	1459.2	1263	18	3	0.986	7.68	0.1	.966	153	58	37.9	21	735	4.53	120	0.321
Parker	Mesa	C	100	100	100	162	1467.2	1283	13	10	0.99	7.79	8.2	1.008	106	50	47.2	13	751	4.61	109	0.318
Petson	NO	C	100	100	100	162	1458	1271	9	5	0.993	7.79	0.7	1.020	138	49	35.5	20	700	4.32	115	0.316
Stepens	Por	C	100	100	100	162	1461.2	1275	14	6	0.989	7.76	10	1.006	103	50	48.5	8	681	4.19	108	0.317

And, indeed we see exactly what we would expect, poor performance due to ratings, but not as poor as when we have taxes of both ratings and position.

SUMMARY/SUGGESTIONS

1) The bottom line here is that there is a penalty for playing a non-C out of position, and as noted above those non-Cs will generally, but not always (due to randomness), be at the bottom of the league in defensive measures and value. But I think there is no doubt based on this data and the data posted about the real capability of true-life MLB "emergency catchers," that the penalty should be considerably harsher.
2) In addition, something should probably be done to remove the ability of the human owner to just change the role to "C" and, therefore, remove roughly half the penalty by magic.
3) In the meantime, Online leagues should immediately put in place a house rule that forbids owners from changing the role of non-catchers to "C" if they are worried about the use of this as a "cheat."

As I noted above, I'll post a table of cumulative expected penalties for ratings and non-C tax, probably tomorrow sometime.

[injury log]

Originally Posted by RonCo

Look, IL ... I understand that THE ACTUAL RESULTS OF THIS TEST WILL NOT GIVE YOU THE ACTUAL RESULTS OF A "REAL LEAGUE." That's not the point, nor has it ever been. The point is to study the magnitude of the performance difference between catchers in equal environments, and the only way to know that for certain is to ACTUALLY HOLD EVERYTHING CONSTANT THAT YOU CAN HOLD CONSTANT, and VARY ONLY THE THINGS YOU WANT TO STUDY.

What this means is that this test can tell us the relative performance drop you should see (relative to the performance level of actual catchers in the league). So if you know the performance of actual C-rated catchers in your league, you could then determine about what a non-C should perform at in your actual league. Pretty slick, eh? Well...I think it is, anyway. YMMV.

Thank you for using capital letters to clarify your points, which I would not have understood otherwise.

There are a few reasons I don't find your study especially illuminating:

* when the catcher you're studying drops 10 percentage points in RTO%, you are consistently assuming that whatever edits you made caused that drop. I imagine you noticed that several of your completely unchanged catchers also lost 10 percentage points in RTO% from one year to the next - Stepens even dropped 17 percentage points from year 1 to year 2. There needs to be some consideration of margin of error here to know if your results are statistically significant, or if your conclusions are just post hoc ergo propter hoc rationalizations;

* When all of your baserunners have 100/100 Speed/Steal ratings, then all of your basestealers are guys who would attempt 4 steals a year in a normal league. So yes, it may be possible that OOTP correctly models how a bad catcher would fare (relative to good catchers) against guys who would never steal bases in the first place. What really matters is how a bad catcher would do against guys who would actually attempt steals;

* When you have a small league, then if statistical output is controlled by league totals, you'll see a much clearer difference between good and bad performances. If say you have a league which should average 25 HR per player, and there are only two batters, one with a 190/200 Power, and one with a 10/200 Power, then the first hitter will likely get 49 HR, the second hitter 1 HR. If instead you have one player with a 10/200 Power and 29 players with 190/200 power, there just aren't as many HRs to go around. The big power hitters then only get about 26 HR per season. That is, the weak hitter is suddenly much closer to the rest of the league. I imagine it's possible that something similar occurs when there is just one incompetent catcher in a league of 29 otherwise good catchers; his statistical performance becomes closer to league average, and is thus less of a penalty than in a 4-team league.

* If 67-70% is the break-even rate for stealing, and if even the incompetent catcher in your study is throwing out more than 27-30% of baserunners, then other teams are just shooting themselves in both feet by running more often against him, even if his RTO% is lower than that of other catchers. When I've looked at this issue in normal leagues, I find that an outfielder will throw out roughly 25-26% of baserunners. That is so close to the break-even RTO rate that it barely hurts a team to have that kind of performance, and the gain other teams get for running more often is negligible.


In any case, a few weeks ago I ran a two season test in a standard 30 team fictional league without controlling much of anything. I'll just paste my post to the beta forum:

I moved my CF to catcher (his ratings are horrible there). In his first season his defensive stats looked like this:

100 games, 8 PB, 10 E, .989 FPCT, .950 EFF, 35 CS, 27.3 CS%, 3.20 CERA

Now that's not very good, but the PB total was nowhere near the worst figure in the league, the CS% was also not close to the worst total in the league, and the CERA was one of the best figures in the league. This is an outfielder playing catcher for the first time. I've kept him there, and in his second season his line reads:

136 G, 11 PB, 16 E, .988 FPCT, .918 EFF, 49 CS, 24.5 CS%, 3.34 CERA

Again, that's pretty bad, but it's certainly tolerable from a guy who hits about .200 points of OPS higher than a typical catcher. Real life teams place an enormous premium on catcher defense, and would never, ever consider moving some random CF to catcher. There need to be far more serious consequences than the defensive line above for playing someone who is defensively completely incompetent behind the plate. I could see a strong justification for having a negative effect on CERA when a non-catcher plays at catcher, for example, and a non-catcher should probably throw out about 12-15% of basestealers. The PB total is also way too low.

Of course I wouldn't read much into the CERA figures above, since I did not account at all for defense, home park, etc; I only mentioned them to demonstrate that if any CERA penalty exists in the game at the moment for playing a CF at catcher, it is not a severe penalty. I don't see much reason to dismiss the PB and RTO% figures, however, in such a large league; any oddities in the distribution of Speed/Steal ratings, pitcher Hold ratings, team Strategy settings, etc, will largely average out.

Finally, I'd point out that it's the IF Error rating which is supposed to affect a catcher's FPCT, and not any of his catching ratings.

[RonCo]

Originally Posted by injury log

There are a few reasons I don't find your study especially illuminating:

* when the catcher you're studying drops 10 percentage points in RTO%, you are consistently assuming that whatever edits you made caused that drop. I imagine you noticed that several of your completely unchanged catchers also lost 10 percentage points in RTO% from one year to the next

Yes, one of the problems with studying catcher performance is that with low sample sizes (like 100-150 SBA) you do get a lot of random variation. If you plot the RTO% of catchers versus their ratings you'll see a very wide scatter in any one year. This is why very poorly rated catchers can and do end up at the top of the leader boards in various stats at times.

However, it is worth noting that in all cases that I've run (and I've done this same study three times), the gaps between the non-C and the _average_ of the real-C's has remained pretty consistent. There's always room for error, of course. But I'm growing more and more comfortable with the data as time goes--comfortable enough to post it publicly, anyway, and think it's almost certainly in the right ballpark.

- Stepens even dropped 17 percentage points from year 1 to year 2. There needs to be some consideration of margin of error here to know if your results are statistically significant, or if your conclusions are just post hoc ergo propter hoc rationalizations;

Yes, Stepens did drop 17 points in a year. Then he bounced back the following year to his norm of ~50%, which is in the ballpark of the league average.

* When all of your baserunners have 100/100 Speed/Steal ratings, then all of your basestealers are guys who would attempt 4 steals a year in a normal league. So yes, it may be possible that OOTP correctly models how a bad catcher would fare (relative to good catchers) against guys who would never steal bases in the first place. What really matters is how a bad catcher would do against guys who would actually attempt steals;

I'll plan to do a trial run with all speeds set to 200 or whatever in a short while. One would expect the RTO% of all catchers to get considerably worse, but that a gap will still exist. I completely agree that the gap is not big enough, which I've stated several times. It's certainly possible that the gap will get smaller, but it's also possible the gap will get bigger or stay the same. Either way, I'm very confident there will be a gap, and that the gap will be accentuated when the player at the "C" position is set to "CF" or some other role.

* When you have a small league, then if statistical output is controlled by league totals, you'll see a much clearer difference between good and bad performances. If say you have a league which should average 25 HR per player, and there are only two batters, one with a 190/200 Power, and one with a 10/200 Power, then the first hitter will likely get 49 HR, the second hitter 1 HR. If instead you have one player with a 10/200 Power and 29 players with 190/200 power, there just aren't as many HRs to go around. The big power hitters then only get about 26 HR per season. That is, the weak hitter is suddenly much closer to the rest of the league. I imagine it's possible that something similar occurs when there is just one incompetent catcher in a league of 29 otherwise good catchers; his statistical performance becomes closer to league average, and is thus less of a penalty than in a 4-team league.

Unless Markus dramatically changed the structure of the results algorithms, this is not a proper representation of how league totals work. League totals don't constrain the number of a stat in any league, but instead define the ratios used to calculate any one at-bat. The actual statistical output of a league will almost never actually match the league totals. The final statistical output of a league will be the result of a mixing of the league totals and the actual ratings of the players in the league, with each player's performance created on its own merits (and not directly influenced by those around him ... Duane Kuiper will not hit many HR whether he players with a bunch of Duane Kuipers or with a bunch of Barry Bondses...but a league of DK's will not hit many homers regardless of what the league totals are, whereas a league of BBs will either hit a bunch or very few, depending on how skewed you make the league totals).

* If 67-70% is the break-even rate for stealing, and if even the incompetent catcher in your study is throwing out more than 27-30% of baserunners, then other teams are just shooting themselves in both feet by running more often against him, even if his RTO% is lower than that of other catchers. When I've looked at this issue in normal leagues, I find that an outfielder will throw out roughly 25-26% of baserunners. That is so close to the break-even RTO rate that it barely hurts a team to have that kind of performance, and the gain other teams get for running more often is negligible.

This is why the CAP LETTERS in my earlier posts are important. I agree totally with you, but this fact does nothing to reflect upon the accuracy of the study, nor does it have anything to do with the actual findings. In fact, the findings support your concern that the gap between non-C and C (or even poor Cs and good Cs) is almost certainly not great enough. I even say that in the post. I'm finding it interesting that you're arguing so heavily against me, when the study I've put forward supports your concern and my recommendations are--I think--along a similar path as yours. If you have other recommendations (other than "don't read RonCo's reports," I suppose) I would be interested in hearing them.

That said, I think that even when a non-C's performance is cranked way down, you'll still find situations where a great bat is worth the pain.

In any case, a few weeks ago I ran a two season test in a standard 30 team fictional league without controlling much of anything. I'll just paste my post to the beta forum:



Of course I wouldn't read much into the CERA figures above, since I did not account at all for defense, home park, etc; I only mentioned them to demonstrate that if any CERA penalty exists in the game at the moment for playing a CF at catcher, it is not a severe penalty. I don't see much reason to dismiss the PB and RTO% figures, however, in such a large league; any oddities in the distribution of Speed/Steal ratings, pitcher Hold ratings, team Strategy settings, etc, will largely average out.

I wouldn't put any stock in CERA ratings of an "open" league because it's impossible to know the ratings matches that created them.

It's possible to use CERA in a league that controls all ratings, though. In fact, in my tightly controlled test environment CERA = ERA.

It's also in error to say that the large "open" league will have all ratings even out. There are too many selection bias issues at play to say that, and in fact in real baseball sabermetric studies this mix-and-match problem is the root of much gnashing of teeth. Certainly, they *could* even out, but it is far more likely that the opposite is the case--stats get skewed by the fact that some hitters face aces more often than others, and some hitters hit in pitchers parks and all that. Add to that manager's human natures (or in OOTP you also have the usage patterns of players with varied ratings highly dependent upon the in-game AI algorithms). Us OOTP guys are lucky, though, because we can hold the ratings constant, which allows us to remove the in-game AI algorithms from the variables list, and actually study the calculations that create stats. You suggest that you know this, but your methods and commentary suggest you're not using this knowledge.

Regardless, I think my study augments your findings and agrees with its fundamental point of view--that being the gap between catchers and non-catchers is not big enough. My study's purpose is to attempt to figure out why that is--where the gaps are created. This is the only way to be able to figure out what to do to fix the problem.

Finally, I'd point out that it's the IF Error rating which is supposed to affect a catcher's FPCT, and not any of his catching ratings.

I make no complete statement about that either way, as I'm not studying infield ratings. It's definitely possible that they do influence F%. I can say, however, that based on this study at least one of "C ability" or "C arm" also appears to strongly influence a catcher's error rate and fielding %. (Note that while all players in my study had 100 IF ratings everywhere, the C-ARM-C-ABILITY adjustments consistently cause errors to rise and F% to drop).

My first guess as to why this might be is that poor C-Arm ratings result in more errant throws to bases. This is truly a very wild guess, though. I have no information to back it up. However, a thought experiment is that when Arm Rating drops, SBA rose by ~ 80% and errors rose by a similar rate (nearly double). The fairly close relationship of these numbers support, but do not prove, my guess.

If my first guess turns out to be incorrect, though, my second guess (which is probably untrue, but has been known to happen to the best of us) is that Markus either isn't quite right about his algorithm, or has a bug that he's not found.

[CBL-Commish]

Don't want to go too far off topic, but this is related to an issue I've long thought might be lurking in OOTP: Whether or not there is enough penalty in very low fielding ratings across the board, not just for catchers. I think that some people have done studies a little like RonCo's here regarding really bad shortstops or center fielders, but I'm not sure how comprehensive they are.

I've never quite gotten a really good warm and fuzzy feeling that, for example, there's an appropriate penalty for playing a big, fat, 36-year-old first baseman at shortstop. This also has a potential impact on universes with widely varying levels of leagues with players moving around all the time.

[RonCo]

Originally Posted by CBL-Commish

Don't want to go too far off topic, but this is related to an issue I've long thought might be lurking in OOTP: Whether or not there is enough penalty in very low fielding ratings across the board, not just for catchers. I think that some people have done studies a little like RonCo's here regarding really bad shortstops or center fielders, but I'm not sure how comprehensive they are.

I've never quite gotten a really good warm and fuzzy feeling that, for example, there's an appropriate penalty for playing a big, fat, 36-year-old first baseman at shortstop. This also has a potential impact on universes with widely varying levels of leagues with players moving around all the time.

This doesn't get directly to what happens if a 1-rated slug gets put into the SS slot, but it does give the ranges of performance in what I'll call an "average" (*) online league:

Defensive Performance v2010 (I think) converted to wins:

Full League, by team and position:
theFOBL.com : Forums - View Single Post - OOTP Defense: A Running Lab Diary
Min/Max, by position
theFOBL.com : Forums - View Single Post - OOTP Defense: A Running Lab Diary

Obviously, there's a bunch more in that thread. And just as obviously, it's based on a previous version, so it's posisble the defensive scheme has changed a bit. My guess is the base structure is the same, though.

* I note that the FOBL will never be average in any way.

[Eumel]

Just to make sure: the catcher's values in both links are without the running game, right?