Does defense matter in OOTP6?

[dougaiton]

Orignally I started just with adjustment to SS range, then gradually added the change to other positions. I'll just show results from the most extreme case, changing all IF & OF range and arm ratings.

Yup, Zweilblumen and Draven, you're both right. Time to pay more attention for me!

[draven085]

Originally posted by dougaiton
Yes, in evaluating the role of defense, but not the individuals, which is what I'm getting at here. If the team remains constant, then the player's RF should change, because other players are making up the outs slack. In these cases, despite the individual's lack of range, he is still making up the same percentage of outs as he would have otherwise. If the rest of the team remained good, then they would capture a greater proportion of chances, increasing their RF, while the player's RF would go down because he wouldn't be converting his share each time round.

i.e.
Team A records 90 outs. Say there's 90 chances, they are distributed equally and every player bar x makes 100% of the chances that come to them into outs. Now say player x makes none of the chances - that leaves 10 outs left to be rescattered. Player x makes his out the second time, but he's left with only 1 of 10 chances, and more importantly for RF, 1 putout compared with every other team members 11. The other players put-out rates go up respectively, to more like 12%. Turn that into rf, and we're there. Because the balls player x doesn't get to are re-distributed across the team, the rest of the side will have higher put-outs, assists etc. numbers, and x will have less. Or, another way of thinking, the balls that x doesn't get to won't get hit back to him when the outs are redistributed.

Well, that made no sense.

Unless I misunderstood GMO, but I presumed that in each test only one player was changed.

I understand where you're coming from. I still think it's important to remember that defensive statistics can be very misleading.

Let's consider your Frank Thomas issue. You say Thomas ranks in the middle of the pack in range factor and turns the 2nd most double plays in the league. On the surface he appears to have played a respectable defensive game at shortstop. I would argue (and its impossible for me to know for sure without your league data) that Thomas' fielding statistics are a direct result of his defensive ineptitude at shortstop.

Tom Tippett of Diamond Mind Baseball has found that the difference between average defensive players and those at either end of the defensive spectrum is typically in the neighborhood of 40 plays per year. All things being equal, the best defensive players in a given year will make 40 more plays than an average defender, while the worst will make 40 less. Let's assume that Thomas is making 40 plays less than an average shortstop per year and thus putting 40 more runners on first base. By failing to make those 40 plays Thomas probably loses 40 total chances off his season total.

However, those extra 40 baserunners provide Thomas with an opportunity to save his totals. They provide his team with 40 additional chances to turn a double play. I don't have any numbers on distribution of balls in play in front of me, but it's safe to assume that Thomas would indeed end up handling a fair share of those chances as double play opportunities. On a double play a middle infielder has a good chance at recording both an assist and a putout on the play, therefore artificially increasing his range factor.

For example, the leadoff hitter in an inning hits a ground ball up the middle that the rangeless Thomas can not make a play on. An average shortstop makes the play and records an assist on the 6-3 putout. With Thomas at shortstop the batter reaches first with a single. The next batter hit a hard groundball to the secondbaseman for a tailor-made 6-4-3 double play. Thomas records a putout and an assist on the play. This obviously won't happen every time, it is merely an example. More baserunners will also afford middle infielders more opportunities for force plays at second base and could result in the middle infielders taking putouts that would typically go to the firstbaseman.

My point is that extra base runners change the dynamic of the defense. I believe that Bill James discusses this as problem in regards to poor defensive teams in Win Shares.

I think you can still make a case that non-rated players should be much worse than they are in OOTP, I just wouldn't use range factor to do it. The statistic is simply far too misleading.

[dougaiton]

No, you're right. The Frank Thomas example was just a bit of fun - the real analysis was the BABIP stuff at the start. I just thought I'd put it in an easily accessible manner.

[dougaiton]

Namely:

Transplanted RF at SS no skill - Doug's badBABIP = .276, avg RF = 4.71 avg (3 year)
SS with all skill removed - .271, avg RF = 4.79 (3 year)

1 range = .287 .279

All players with no range, out of positon: 0.298, 0.292, 0.301
All players range set to 1: 0.308, 0.298, 0.300
All players range set to 50: 0.281, 0.278, 0.278
All players range set to 100: 0.266, 0.267, 0.265

For a definition of Doug'sbadBABIP, check out the start!

[dougaiton]

Originally posted by draven085
A lot of interesting stuff that I should have thought about while making a purely rhetorical point!

I forgot to say thanks for such an eloquent answer!

[draven085]

Originally posted by dougaiton
I forgot to say thanks for such an eloquent answer!

Why can't it be my job to sit around, dig through box scores or play-by-play data, and see if what I wrote actually makes sense?

[Specs]

Range factors say almost nothing about a player's defensive value, even in relation to the league. Juan Samuel had a higher range factor than the national league by a good half a play per game, and he's usually regarded as a terrible defender. There's a reason Steve Jeltz ever played major league baseball, and his name is Juan Samuel.

On the other hand, Mark Belanger may have been the best defensive shortstop of all time. In any case, he and Brooks Robinson formed an impenatrable left side of the infield which resulted in Jim Palmer's hall of fame career and Money Store advertisements. Yet Belanger's range factor relative to the league was only a quarter of a play per game better, and twice in his prime (at ages 26 and 28) it was worse!

On the contrary, Samuel had been turned into an outfielder after six years of range factors above the league average and fielding percentages at or near it. His replacement, Tom Herr, had a similar range factor behind basically the same staff the next season, but if you find me one board member who says that Tom Herr hasn't gone down in history as a great defensive second baseman, I'll bend over for both of you.

[Specs]

Since wins go up 12.4 from 0 to 100, can we say that 37.2 more win shares are to be given to the defense of the excellent team versus the defense of the terrible team? Using a standard distribution, over seven positions because the catcher seemed to remain constant, 2.4 of those should go to the first baseman, 8.9 to the second baseman, 6.2 to the third baseman, 9.0 to the shortstop, 2.9 to the leftfielder, 4.5 to the centerfielder, and 3.3 to the rightfielder. All of these numbers point to the conclusion that the difference between a perfect defensive team and a terrible one is still a VERY GOOD defensive team.

[obaslg]

I'm very interested in these studies - thanks for posting.

I think I already reported this somewhere:
I ran three fictional, 36-team, no-injury, no-fatigue, DH, default stadium leagues to test v.6. One thing I did was run a regression analysis of the range and arm ratings versus the total chances.

Focusing on shortstops, with a pool of about 80 players over the three leagues (I had a minimum games played), I found no statistically significant correlation. Same for catchers. Please note that this doesn't mean that defense has no impact (which I doubt for obvious reasons), but it's surprising that the impact wasn't measurable in that analysis.

For perspective by comparison to hitting ratings, the adjusted R squared* for batting ratings versus homers was .9, and the adjusted R squared for the fielding analysis was less than .1.

*Basically, this refers to the amount of the variance in the final output (homers in one case, and total chances in the other) attributable to the tested factors (batting ratings in once case, and range and arm in the other).

I never ran the regression fielding test on v.5, so I don't know how it differs.

My analysis partner and I are now doing a more extensive analysis of a sort that I did do with v.5. We create a 36-team league as described above, lock all the depth charts to maximize playing time, and then for each position change the player on each team to a 10 range and then to a 90, with multiple runs of each.

So far we've only done SS. We did ten runs with all SS at 10 and then ten with all at 90. Then we compared the total league offense.

OK - now's the part when I piss you off. We guard these analyses pretty closely as part of our online competive edge, so I won't tell you exactly what we found, but I can say that there was a significant effect, but less than in v.5 - almost exactly half as much, actually, assuming 10=E and 90=A.

Sorry I'm not giving more detail - I just wanted the OOTP team to know that we have found significant, though reduced, effect.

[Big Train]

Originally posted by rasnell
This whole thread is very interesting to those of us who go for the best defense up the middle and greatly sacrifice offense. If it turns out this is a very flawed strategy -- at least under OOTP 6 -- that's a major discovery.

It would be a rather disappointing one for me, I know that real teams used to often sacrifice offence for good defense [Especially before the current "rocketball era"]. I have always thought that SS and 2B defense may be a bit more important in OOTP [4&5] than in real life but I kind of like this because it penalizes teams who insist on playing a slugger with an "E" rating at SS as opposed to a light hitter with an "A" rating which I feel is basically realistic and adds strategy to the game.

[Specs]

Well in the 1920s second base was a completely unimportant position defensively, usually reserved for the worst player on the team, but that's not the case any more, is it?

[Big Train]

Originally posted by Specs
Well in the 1920s second base was a completely unimportant position defensively, usually reserved for the worst player on the team, but that's not the case any more, is it?

um...Rogers Hornsby, Charlie Gehringer, Frankie Frisch? Frisch and Gehringer were known as good defensively, Hornsby I've most often heard to described as "adequate" [but a .424 average does make up for a little less range ) but probalby all were the best or one of the best on their teams for most of their career.

[enuttage]

Just wanted to drop in and say thanks for boiling things down for those of us who have less math-oriented brains (or failed to bother taking calculus, physics and the like).

This is fascinating stuff, and something I'd love to see more of, both on the fielding end of things, and elsewhere in the game.

Keep up the good work fellas!

-E

p.s. This takes me back. Those two names are using neural paths that haven't been open in awhile.

Originally posted by Specs
Juan Samuel

Originally posted by Specs
Steve Jeltz

[scotto313]

Originally posted by obaslg
We guard these analyses pretty closely as part of our online competive edge, so I won't tell you exactly what we found, but I can say that there was a significant effect, but less than in v.5 - almost exactly half as much, actually, assuming 10=E and 90=A.

Which leagues are you in?

[dougaiton]

Originally posted by obaslg

OK - now's the part when I piss you off. We guard these analyses pretty closely as part of our online competive edge, so I won't tell you exactly what we found, but I can say that there was a significant effect, but less than in v.5 - almost exactly half as much, actually, assuming 10=E and 90=A.

To be honest, I'm of the opinion that if you put as much time in as you no doubt do into testing, you can pretty much do whatever you like with it! Thanks for the info.

[Skipaway]

Originally posted by Specs
Range factors say almost nothing about a player's defensive value, even in relation to the league.

However, that's not the case in studies like this. When everything else is controled, range factors tells you a lot.

[Skipaway]

Originally posted by Specs
Since wins go up 12.4 from 0 to 100, can we say that 37.2 more win shares are to be given to the defense of the excellent team versus the defense of the terrible team? Using a standard distribution, over seven positions because the catcher seemed to remain constant, 2.4 of those should go to the first baseman, 8.9 to the second baseman, 6.2 to the third baseman, 9.0 to the shortstop, 2.9 to the leftfielder, 4.5 to the centerfielder, and 3.3 to the rightfielder. All of these numbers point to the conclusion that the difference between a perfect defensive team and a terrible one is still a VERY GOOD defensive team.

I don't think that's exactly correct. While not really employing replacement level ideas, Win Shares do use cut off points. Contributions below certain points would be rewarded nothing in Win Shares.

Therefore, you can't really compare difference in Win Shares to absolute values in Win Shares.

[gmo]

Did some further testing with the league of mine descibed earlier in the thread...

Set some guys in positions where they had no range rating. Actually it was just removing the range and arm ratings from their position. Within a week or so (perhaps after one game?), the fielding percentage reappeared for those players with the same value as before the range was removed taking the fielding percentage with it. The range rating however did not return during the season. But when I reset the league after running a season the players had gotten a minimal range rating (low single digits on 1-100 scale). If it is the resetting giving the rating since the player is set in the lineup at a position, maybe it would not happen in a career league. But it doesn't really matter because in testing I saw no difference between the cases when the players had no rating because I bothered to re-remove it and when they had say a 3/100 rating. Arm rating never emerged spontaneously.

About the testing... With all other players on all teams made average (50 ratings), what would happen if on one team 2B, SS, CF were changed to terrible fielders but excellent hitters? For those three positions on one team I set their range and arm to no rating and all their hitting ratings to 90. I also tested with the hitting ratings set to 70. Going the other direction, I set the 2B, SS, CF to 99 range and arm and their batting ratings to 10, then to 30.

So there are 4 test cases listed below, and for each case I simmed 5 seasons.

2B, SS, CF: zero/minimal range & arm, 90 hitting ratings
2B, SS, CF: zero/minimal range & arm, 70 hitting ratings
2B, SS, CF: 99 range & arm ratings, 30 hitting ratings
2B, SS, CF: 99 range & arm ratings, 10 hitting ratings

Before I get into the results, think about what you would expect. If you had the choice at SS of a guy who hit like A-Rod or better but was the worst fielder at that position in professional ball or a guy who was the very best possible fielder at the position but hit like a Single A scrub, who would you pick? Given the choice I assume most everybody would if they had the chance choose best-hitter/worst-fielder over worst-hitter/best-fielder.

Now some data supporting that position.

Wins (finish among 10 teams)
No glove, 90 bat - 102(1st), 104(1st), 104(1st), 97(1st), 105(1st)
No glove, 70 bat - 94(1st), 99(1st), 91(2nd), 86(4th), 84(3rd)
99 glove, 30 bat - 69(9th), 68(10th), 75(10th), 68(10th), 80(7th)
99 glove, 10 bat - 69(10th), 54(10th), 52(10th), 62(10th), 56(10th)

More stats shown as 5-season averages

                   Wins  DP   BABIP**
No glove, 90 bat  102.4  149  .303
No glove, 70 bat   90.8  143  .307
99 glove, 30 bat   72.0  141  .284
99 glove, 10 bat   58.6  145  .283

**calculated as pitching staff (Hits-HR)/(Inn*3+Hits-HR-SO)
**also subtracting DPs from bottom term adds about .010 to each value

On the teams where all the players are average the SS,CF,2B bat 1st,2nd,9th. That doesn't really matter much since everybody's hitting ratings are the same. On the team with improved hitting SS,CF,2B they batted 2nd,3rd,4th, and when their hitting was worsened they batted 7th,8th,9th.

I don't like playing with ratings on, and I avoided mostly looking at the batting stats of the adjusted hitters. But surely someone with all 90 hitting ratings is best-in-league caliber. All 70s, I figure there's a handful or two of those guys in a typical league. I assume that having three such improved players in a batting order otherwise totally average is some amount more than 3 times better than having just one. For instance three 90 hitters was worth apparently ~21 extra wins (102 minus an average 81), but I do not think every such player would be worth 7 extra wins. It would seem the more better hitters you have, the more they help each other and the team overall produce more. I don't feel like testing any more right now, so that will have to stay opinion. Any other takes?

Back to the issue of putting players at positions they have no business playing. Analogous to the above experiment, would the Yankees be even better if they got Vlad Guerrero and played him at 2nd? What if they put Frank Thomas there? Presumably either of those guys would be the worst 2B in the majors & minors (a no or 1 rating in the game). But would a more athletic Vlad be better than Thomas? I can see this being a very difficult problem to tackle in programming the game. "Fixing" this is some way would likely put something else out of whack. Maybe future versions will see some sort of athleticism ratings that influence fielding ability. Maybe somehow variable penalty points (big for poor 1B playing CF, small for good 3B playing 2B, etc) that could result in negative defensive ratings if playing a position where not rated could be incorporated. Or maybe not. Any other ideas?

Should much time even be spent addressing it? It seems the defensive system is quite good now, except for this issue. Online leagues can make rules against taking unfair advantage, and solo players can choose not to exploit the system. I guess the issues with the CPU not using players at their optimal defensive positions could make this more important. Guess it could depend on the possible solutions that are conceived.

[jbmagic]

in an online league i am in..the commish made a rule where you can put a player at a certain position if only they have a range rating for that position...

[dougaiton]

Originally posted by gmo

Should much time even be spent addressing it? It seems the defensive system is quite good now, except for this issue. Online leagues can make rules against taking unfair advantage, and solo players can choose not to exploit the system. I guess the issues with the CPU not using players at their optimal defensive positions could make this more important. Guess it could depend on the possible solutions that are conceived.

I think I've made my views clear on this issue, but I suppose I need to defend why it is important.

Obviously, house rules can be made to deal with this. But house rules can be made to deal with a trade AI, house rules can be made to deal with the waiver system. I suppose my views are 'house rules' are fine for in-game options that reflect the gameworld, but not the simulation. We could ask people to remove their relievers (and remove the AIs relievers) after tehy pitched their 50th pitch in extra innings, but its far from ideal.

I know I'm in the minority here, but most solo leagues I play have very small rosters, 15 or 20. One of the challenges in OOTP was what happens when a player gets injured? I would set up depth charts for uncovered position with terrible defensive players behind them. Now it appears those terrible defensive players would perform adequately.

Secondly, the star rating misjudges those of a low defensive value. There is no visible benefit in my tests to signing a player who has a 10 rating at short, yet he will recieve kudos in the AI evaluation of him because of it.

Thirdly, the CPU does play players out of place, especially with pinch-hitting/substitution logic. That's something that house rules can't deal with.

But I think the most important thing is the way in which the game simulates baseball. If pitchers left in an OF spot always hit .240 with 10 homers, or if batters left as a starter were in the region of a 5.00 ERA and with 100ks, this would be seen as a serious problem. To me, leaving anyone in a position should be a disaster.

How would I deal with it? Just like on-line leagues, use the defensive scale. Correct me if I'm wrong but it runs from SS-2B-CF all the way down to 1B-LF. Anyone trying to play down the scale can do using the normal system. Anyone not gets killed. It's not perfect, but it makes a lot more sense than presuming ML ability (even if very bad ML ability) at any position for any player.