Catcher Influence

[RonCo]

Correct. It's all batted-ball data.

[RonCo]

Here were the C detail data. Note that there is considerable difference between C capability, but that difference is only expressed over a very few batted balls. If I remember correctly, I wasn't able to pull bunt data out of the game logs, so I can't say which of these batted balls are bunts vs. choppers in front of the plate or whatever.

theFOBL.com : Forums - View Single Post - OOTP Defense: A Running Lab Diary

[injury log]

You're attacking a straw man. I'm not disputing that catchers with bad ratings perform worse than catchers with good ratings. Of course they do, and that's what I've seen every time I've looked at this issue as well. What I took issue with were the conclusions you've drawn from your study, in particular here:

Originally Posted by RonCo

Stolen Bases Against

Team	SBA	RTO	RTO%
Mesa	103	52	50.5
NO	127	61	48
KC	124	54	43.5
Por	153	17	11.1

Again, the order matches expectation, and the non-catcher is off-the-charts bad.

Originally Posted by RonCo

So, while I think the defensive penalty for using a position player as a catcher could be made a bit worse, I don't think it's a flaw of catastrophic levels).

Anyone seeing this data, and the conclusions you've drawn from it, would think that OOTP is modeling incompetent catcher defense reasonably well, and that only minor adjustments are required. Your data is nothing like what I've seen when I've used an incompetent catcher in a typical league, something I've done on several occasions (you might recall we discussed this question at length a few years ago: http://www.ootpdevelopments.com/boar...ew-limits.html). Now I don't know precisely why your setup produces results so dramatically different from what I've observed, but every time I've suggested that your methodology does not produce results relevant to a normal league setup you've dismissed my comments with exceedingly patronizing replies. There are important differences between your setup and the setup any normal user will experience, and those differences are very important if we want to determine the scale of the problem.

My position is this: yes, bad catchers perform worse than good catchers. No, the drop in performance is not nearly steep enough in any normal league setup. I am not 'arguing heavily against you', and I'm not sure why you interpret my position in that way. I simply don't agree with your assessment of the magnitude of the problem.

As for the other points, I'd agree that any increase in Error rate for bad catchers arises because throwing errors are likely more common for catchers than are errors on other play types - that seems the most plausible explanation. Further, if your finding is correct that setting a player's position to C changes his performance, that is a very helpful observation which I will test and report to Markus.

[endgame]

Originally Posted by injury log

Further, if your finding is correct that setting a player's position to C changes his performance, that is a very helpful observation which I will test and report to Markus.

FWIW, on RonCo's first mention that the position change produced different results, I immediately added the link to a PT already in place addressing the use AI position labeling. Of course it couldn't hurt to provide a second entry specifically pointing at the C position.

[OldFatGuy]

And once again folks that know more about stats want claim to know everything about baseball.

Pitchers don't have any control over balls in play.

Catchers have no influence over pitchers.

Truly, funny funny stuff.

Actually though, I take back the first sentence, because they really apparently don't know much about stats either.

Because you see, when results from a billion different outcomes with a billion different control factors happen to NOT prove that pitchers have control over babip or catchers have control over pitchers only means those results and your tests didn't prove it.

Yet it's always taken to mean, by those spreading the gospel of the day, that by not proving it, ergo, it's been proven false. Which it HAS NOT.

Just because some statistical analysis done on some of those results didn't prove a correlation is there, that is NOT proof that there is no correlation.

So, one is technically free to believe in whatever one wants. If challenged for "back-up" or "proof", well, there is no more statistical back-up or proof for the assertion that there is no catcher influence than there is for statistical back-up or proof that there is.

There is, however, the experience of thousands and thousands of pitchers and catchers who have played the game. For whatever that's worth.

[RonCo]

IL,

1) We agree that the gap between emergency catchers and "real" catchers is too slight.

2) I think we agree that the gap should be be expanded so that emergency catchers perform within the parameters of the data shown in the earlier thread linked to in this thread.

3) I think we agree that the algorithm should be changed to not allow a non-C who has their position changed to a C automatically field better than he would if he retained his non-C position.

Before I did my work with my strange little league, I would have generally agreed with 1) and strongly agreed with 2), but I would have no idea that the impact of 3) even existed. In fact, I suggest there was no other way to identify that bug (and yes, I think it's a bug). I can confirm problems with a full-out league, but to troubleshoot root causes I find it is almost always valuable to run fix-ratings leagues.

I admit I get worn out arguing about why using fixed-rating leagues have value in studying problems. I apologize that my demeanor was out of line.

I admit also that I've read the other thread, though, and don't actually see what was patronizing about my responses in that thread. In fact, I thought we had a pretty fruitful discussion, and I thought I learned something from you in that thread.

Ron

[speels]

Originally Posted by OldFatGuy

And once again folks that know more about stats want claim to know everything about baseball.

Pitchers don't have any control over balls in play.

Catchers have no influence over pitchers.

Truly, funny funny stuff.

Actually though, I take back the first sentence, because they really apparently don't know much about stats either.

Because you see, when results from a billion different outcomes with a billion different control factors happen to NOT prove that pitchers have control over babip or catchers have control over pitchers only means those results and your tests didn't prove it.

Yet it's always taken to mean, by those spreading the gospel of the day, that by not proving it, ergo, it's been proven false. Which it HAS NOT.

Just because some statistical analysis done on some of those results didn't prove a correlation is there, that is NOT proof that there is no correlation.

So, one is technically free to believe in whatever one wants. If challenged for "back-up" or "proof", well, there is no more statistical back-up or proof for the assertion that there is no catcher influence than there is for statistical back-up or proof that there is.

There is, however, the experience of thousands and thousands of pitchers and catchers who have played the game. For whatever that's worth.

Having played sports all my life and playing baseball for a good amount, I have to agree. When I am on the hill and I have X catcher behind the plate and I know that he is going to block 99% of the balls I throw in the dirt, I gain confidence. That, in turn, helps me pitch better if I have a pitch, that when on, works best if it bounces at the plate.

Now does this mean my stats will be better? No, because I can't control what the batters are doing. Maybe they came in batting .500 in there last 6 games and are seeing the ball really well, or maybe they are in a slump and can't see the ball at all. But, my confidence will be high and anyone who has ever played sports, or really done anything knows that if you have good confidence things seem to go better for you.

So I don't think that this is something that can be put in the game, because there are too many variables that can affect it that are out of a pitchers control.

[RonCo]

Originally Posted by OldFatGuy

Yet it's always taken to mean, by those spreading the gospel of the day, that by not proving it, ergo, it's been proven false. Which it HAS NOT.

This is not true. The proper line, which was even in the Woolner link provided earlier, is: "it’s more accurate to say that if there is a true game-calling ability, it lies below the threshold of detection. "

So true saber guys will not say something like this is proven false.

In context of the game implementation, this suggests that the influence of catchers on pitcher performance should either not be implemented, or should be implemented at such a small rate that it does not influence the statistical output of pitchers. If I were Markus, I would take the easy way out, and not implement it, but Markus thinks in ways that Markus thinks. Either way works.

[statfreak]

Originally Posted by RonCo

I admit I get worn out arguing about why using fixed-rating leagues has value in studying problems.

FWIW, using fixed-rating leagues is the only logical way to pinpoint a problem. I'd be skeptical of any study that didn't have any such controls in place.

[ike121212]

Originally Posted by OldFatGuy

And once again folks that know more about stats want claim to know everything about baseball.

Two things to keep in mind:

These studies always stop short of saying they prove anything. Woolner's article is compelling and well presented. It's hard to argue with his conclusion, even if you don't agree with it.

The study deals with Major League Pitchers and Catchers, safe to say a pretty skilled group. His single season threshold is 1,000 pa and his career threshold is 10,000 pa's. Hitting those marks requires an even higher level of skill and, obviously, experience.

[RonCo]

Originally Posted by ike121212

Two things to keep in mind:

These studies always stop short of saying they prove anything. Woolner's article is compelling and well presented. It's hard to argue with his conclusion, even if you don't agree with it.

The study deals with Major League Pitchers and Catchers, safe to say a pretty skilled group. His single season threshold is 1,000 pa and his career threshold is 10,000 pa's. Hitting those marks requires an even higher level of skill and, obviously, experience.

Yes, that study is well-presented and the conclusion is logical. It seems to leave very little wiggle-room for the possibility that catchers have game-calling skills that differentiate themselves. Very little is not none, of course.

It would be interesting to hear what influence folks who are not swayed by the study would expect game-calling skills to have, and how it influences the game results. Do they think it will put the count in their favor (hence making it harder for the hitter to get hits?). Do they think it helps pitchers avoid walks? Keep them from hitting batters? Improve their command? Reduce runs the opponent scores? All of the above?

Given these expected impacts, then, would they still hold onto the claim of game-calling ability if it doesn't show up in the performance measures? And, if it doesn't show up in the game results, how does it make a difference?

As I said up-stream, there was a tangible benefit to the Braves signing Eddie Perez. That tangible benefit was that it kept Gregg Maddux happy. Maddux didn't perform any better with Perez than he did with other catchers, but he performed as a Brave. Definite value. But that value appears to have been based on human factors, not based on anything to do with quantifiable performance factors.

[RonCo]

Originally Posted by speels

Having played sports all my life and playing baseball for a good amount, I have to agree. When I am on the hill and I have X catcher behind the plate and I know that he is going to block 99% of the balls I throw in the dirt, I gain confidence. That, in turn, helps me pitch better if I have a pitch, that when on, works best if it bounces at the plate.

Now does this mean my stats will be better? No, because I can't control what the batters are doing. Maybe they came in batting .500 in there last 6 games and are seeing the ball really well, or maybe they are in a slump and can't see the ball at all. But, my confidence will be high and anyone who has ever played sports, or really done anything knows that if you have good confidence things seem to go better for you.

So I don't think that this is something that can be put in the game, because there are too many variables that can affect it that are out of a pitchers control.

Those factors you mentioned can be teased out of the data when you use enough data points. This means that if you actually pitch better because you trust your catcher, then your stats over time should reflect it.

In this case, your catcher has a quantifiable skill--he reduces passed balls or wild pitches. These are things that will definitely improve a pitcher's ability to win (influence stats), and will endear a catcher to a pitcher. They are also skills that have a tendency to transfer from year-to-year in stats. Controlling passed balls and wild pitches, the ability to influence the opposition's running game, and the ability to field choppers and bunts, seem to comprise the bulk of the catcher's defensive value (and all can be observed and measured in statistical output).

I don't think they have anything to do with game-calling skills, however.

[injury log]

Originally Posted by RonCo

I admit also that I've read the other thread, though, and don't actually see what was patronizing about my responses in that thread. In fact, I thought we had a pretty fruitful discussion, and I thought I learned something from you in that thread.

No, it was your dismissive "Sigh" and your emphatic use of capital letters in two posts in this thread that I found patronizing. But since we've had many disagreements in the past, and you have been, without fail, diplomatic and civil in your replies, I don't really mind. I probably shouldn't have said anything.

And since we appear to largely agree on the main points here and have arrived at some constructive conclusions, I don't think I have much more to say. I do appreciate your systematic analysis of the issue, and hopefully it helps to make the game better.

[injury log]

Originally Posted by RonCo

It would be interesting to hear what influence folks who are not swayed by the study would expect game-calling skills to have, and how it influences the game results. Do they think it will put the count in their favor (hence making it harder for the hitter to get hits?). Do they think it helps pitchers avoid walks? Keep them from hitting batters? Improve their command? Reduce runs the opponent scores? All of the above?

One effect that has gone unmentioned so far: commentators often talk about a catcher's ability to 'frame' a pitch - i.e. to turn a ball into a strike. I have no idea if that ability is just an invention of modern baseball commentary or a genuine phenomenon, but if it is a genuine effect, I would imagine incompetent catchers would be worse at it than good ones.

I also don't know if any study has been made of the success of various pitch sequences. My guess is that by the time catchers reach MLB, they've all learned more or less the same things about pitch sequencing, which would explain why there is no great observable game calling effect in big league baseball.

Originally Posted by OldFatGuy

And once again folks that know more about stats want claim to know everything about baseball.

In my experience, people in the sabermetric community who analyze these issues using statistics are almost always very careful to qualify any conclusions they draw. Your impression, that this community claims "to know everything about baseball", couldn't be more different from my impression. Indeed, if they thought they knew everything about baseball already, they'd be doing something else.

You certainly can draw conclusions from statistical analysis. You just need to be very careful about the scope of the conclusions you draw. That you can't observe an effect in the data is not proof that there is no effect, and I'm sure most sabermetric analysts are well aware of that fact.

Originally Posted by OldFatGuy

Just because some statistical analysis done on some of those results didn't prove a correlation is there, that is NOT proof that there is no correlation.

So, one is technically free to believe in whatever one wants. If challenged for "back-up" or "proof", well, there is no more statistical back-up or proof for the assertion that there is no catcher influence than there is for statistical back-up or proof that there is.

I suppose anyone's free to believe anything they want if they don't care about actual data. But if some effect genuinely exists in baseball, I'd expect it to show up somehow in statistics.

There is, however, the experience of thousands and thousands of pitchers and catchers who have played the game. For whatever that's worth.

I think that's worth a lot, and I'm sure many of my impressions of how real baseball work come from hearing interviews and such with pitchers and players. But I don't simply take on faith what baseball players claim to be true. And if, for example, a catcher influences pitching results, that should show up somehow in results. If it doesn't, then either the data wasn't analyzed correctly or the effect is too small to observe. Since the main question we're discussing on this forum is how to model real baseball in a simulation, effects that are too small to observe in real life probably don't warrant inclusion in OOTP.

And for what it's worth, most people in the sabermetric community agree that pitchers *do* have some influence on balls in play, just that the influence is reasonably small, probably smaller than many people believe. I doubt you'd find anyone these days who would agree that pitchers don't have any influence on balls in play.

[CBL-Commish]

Originally Posted by RonCo

This doesn't get directly to what happens if a 1-rated slug gets put into the SS slot, but it does give the ranges of performance in what I'll call an "average" (*) online league:

Defensive Performance v2010 (I think) converted to wins:

Full League, by team and position:
theFOBL.com : Forums - View Single Post - OOTP Defense: A Running Lab Diary
Min/Max, by position
theFOBL.com : Forums - View Single Post - OOTP Defense: A Running Lab Diary

Obviously, there's a bunch more in that thread. And just as obviously, it's based on a previous version, so it's posisble the defensive scheme has changed a bit. My guess is the base structure is the same, though.

* I note that the FOBL will never be average in any way.

Thanks, appreciate the response. I did a quick test with a 4-team league, not taking the time to set all ratings to equal, and it indicated that the difference between an excellent shortstop and a poor left-handed first baseman playing shortstop was on the order of 100 runs per 1350 innings.

That passes the sniff test, seems reasonable. Of course further, more comprehensive testing could confirm whether that's typical.

[Eumel]

Originally Posted by CBL-Commish

Thanks, appreciate the response. I did a quick test with a 4-team league, not taking the time to set all ratings to equal, and it indicated that the difference between an excellent shortstop and a poor left-handed first baseman playing shortstop was on the order of 100 runs per 1350 innings.

That passes the sniff test, seems reasonable. Of course further, more comprehensive testing could confirm whether that's typical.

That's interesting. The short test I did in OOTP 11 back then was more to the tune of 20-30 runs or so.

[CBL-Commish]

Originally Posted by Eumel

That's interesting. The short test I did in OOTP 11 back then was more to the tune of 20-30 runs or so.

Hmmm... I guess further testing is warranted. Your results don't even come close to what they should be. There's a bigger spread than that between actual MLB shortstops. The difference between Ozzie playing short and Kent Hrbek playing short should be ~100 runs.

[RonCo]

I'm not sure 100 runs sounds quite right. My gut reaction would be to expect the number to be more like 50-70, personally (realizing that's subjective). That's 5-7 wins. (My study on OOTP defense previously found about 3-wins delta in shortstop defensive performance from top to bottom, which is probably about reasonable. You can argue it a bit, of course).

The two quick tests together suggest that difference is between 3-10 wins, so if we take the mid-point, it's 6.5 wins, which is in the zone, but may be just a little high.

[Eumel]

Just to be clear: I did not run any serious tests on this. Because I was curious I took Jason Giambi and let him play at shortstop all year, resulting in a ZR somewhere between -20 and -30. If the delta for actual shortstops is 3 wins, then the worst of these guys should be around -15 or so. Giambi wasn't much worse for me.

[CBL-Commish]

Originally Posted by RonCo

I'm not sure 100 runs sounds quite right. My gut reaction would be to expect the number to be more like 50-70, personally (realizing that's subjective). That's 5-7 wins. (My study on OOTP defense previously found about 3-wins delta in shortstop defensive performance from top to bottom, which is probably about reasonable. You can argue it a bit, of course).

The two quick tests together suggest that difference is between 3-10 wins, so if we take the mid-point, it's 6.5 wins, which is in the zone, but may be just a little high.

My thinking was that we occasionally see a shortstop at +30 or even a bit more, like Everett or Ozzie. And we sometimes see one near -30, like Jeter.

That's 60 runs, or six wins, right there. To me replacing Jeter with Kent Hrbek would have to be another 30-40 runs (plausibly more), making the Everett-Hrbek difference around 100.