DIPS, Defense, and 1974: A Case Study

[old timer]

I went through all of the teams and modified many of the players defensive ratings and was pleased to see that the replays were noticeably improved. Pitchers like Hunter, P. Niekro, etc. who never did as well in the sims, now performed much closer to RL. Instead of showing individual stats, I thought it might be interesting to show how well the teams did in the standings compared to both RL and to an unmodded set of sims, even though I realize that without the rest of the data, the standings alone can be rather misleading - I'll give some examples of this below.

The 2 pictures at the bottom show the wins each team had in RL, the avg. wins over 10 season replays, the number of times each team took 1st place and the final columns have the number of wins in each of the 10 seasons.

It's important to note that although the first set of 10 sims (the unmodded ones) appear at times to be better, upon closer analysis, they are not. For example, you'll notice that the Yankees did much poorer in the defensive mod sim compared to RL and compared to the unmodded sim. This can be explained in large part by roster differences. Rudy May and Dick Tidrow who both performed well for NY in RL weren't on the team in the sims. They had a combined 40 starts in RL and did much better than the two who pitched in the sims. The unmodded sim had a record that was closer to RL, but for the wrong reasons. Their defense was too good (defensive ratings were too high), which made up for weaker pitching (the two missing pitchers).

As another example, look at the Angels. They do WAY better in both sims compared to RL. Perhaps part of the reason the Angels did so poor in RL was simply bad luck. If you look at the Angels Pythagorean W-L for that year, it was 77-85, 9 games better than the RL record and closer to the sims. Furthermore, two players (3B&1B) that didn't start for the real Angels, performed much better than their real counterparts who did start.

In other words, once you fix the defensive ratings and account for roster and starting lineup differences as well as injuries (I had injuries off), the performances are actually pretty good. Although I'm not personally that into accuracy (I like more variation), it would be interesting to see how true to life things would play out if trades and player usage were taken into account as well.

I did many other sims besides the ones listed here and in the unmodded ones I occasionally seen things like the Padres and Cubs winning the World Series, while that never happened in over 40 sims with the modded teams. Still, if you look through the (modded) standings, you'll see some seasons working out in a rather implausible way.

One last thing I'd like to add. I didn't have to spend too much time modifying the defensive ratings and since I know so little about the players, I pretty much just went off of the basic defensive stats. In other words, it looks like the game should be able to be improved to rate players better and this in turn will help make the results more realistic. Someone else with more knowledge and more time could do a much better job I'm sure. Improving the defensive ratings will benefit all historical simmers, not just those few of us who like replaying a single season, so hopefully Markus will be able to make some improvements here.

[old timer]

In my very long post above, I forgot to list one very important stat. The team BABIP for each of the (modded) teams was much closer to RL. For example, the A's, Dodgers, Braves and Orioles all benefited from a much lower team BABIP (compared to the unmodded sims). This was reflected in their individual pitching stats being much closer to RL and, of course, in their improvements in the standings.

[pstrickert]

Did you change the LTMs at all before simming? Or did you leave them all at default (1.000)?

[old timer]

Originally Posted by pstrickert

Did you change the LTMs at all before simming? Or did you leave them all at default (1.000)?

I did change them using SteveP's suggestion. However, the fielding pcts. were way off (too high), so I made them all 1.300 which made the league error totals spot on. I didn't bother to try and find individual fielding position error totals.

[pstrickert]

The next step is up to Markus, then. He needs to change the formula for fielding ratings. Once he does that, we're good to go!

[Rondell Tate]

How do we account for something as basic as fielders' arms? I note the best-throwing catchers (Bench, for instance, Tom Haller, Jim Sundberg) get mediocre imports (probably because nobody attempts to steal on them ... Bench a 10/20 arm?).
I haven't checked if the best-throwing RF have the same problem, but it wouldn't surprise me. OF assists are an awful stat.

[Spritze]

Originally Posted by Rondell Tate

How do we account for something as basic as fielders' arms? I note the best-throwing catchers (Bench, for instance, Tom Haller, Jim Sundberg) get mediocre imports (probably because nobody attempts to steal on them ... Bench a 10/20 arm?).
I haven't checked if the best-throwing RF have the same problem, but it wouldn't surprise me. OF assists are an awful stat.

Spritze Defensive C Arm Ratings
Bench 2.52
Haller 1.85
Sundberg 3.07

Major league average C arm rating is set at 1.00, rating is 100% based on the available Lahman stats so it can be done correctly with the right formula.

OF arm examples... Clemente 3.83, Ichiro Suzuki 1.85, major league average is pegged at 1.00, still just looking at available Lahman stats in the right light.

I am totally confidant Markus can come up with a better fielding rating formula than the current fairly inaccurate one.

[RonCo]

This seems like really nice work.

[Spritze]

Oddly enough C arm and OF arm are two of the easiest things to get right (well not right exactly since everyone has an opinion so perhaps very acceptable would be a better term) for historical players.

[pstrickert]

Spritze, do you have any thoughts about fielding ratings in OOTP? How can we improve them? What stats would produce more representative ratings? What do you think of the DefEff stat? Is it simply 1-BABIP for the team?

[Spritze]

Originally Posted by pstrickert

Spritze, do you have any thoughts about fielding ratings in OOTP? How can we improve them? What stats would produce more representative ratings? What do you think of the DefEff stat? Is it simply 1-BABIP for the team?

Yes, fielding ratings can be improved. I think rather than add new stats of some kind one should see what can be done with the stats available in Lahman. Lahman is sort of the stats standard for OOTP and perhaps one should see what can be done with that data for a start.

My job for the last 26 years as a database consultant has been to take the old adage garbage in/garbage out and change it to garbage in/useful info out for a wide range of companies and government agencies. This training gives me some extra tools to see info perhaps a bit differently than some.

If one takes the Lahman files as the garbage in part of the equation and fielding ratings as the good stuff out it is real clear that there is plenty of ignored historical data that is available in the standard Lahman db. The first thing to note is that the Lahman folk do not mean to hide useful info and so they don't. It is there, hanging out, having a beer. Hence the title line of this post. The companies I work with have collected the data they need too. They just don't always see it.

For instance one poster in this thread thought OF assists were not a terribly good indicator of OF Arm rating. OF assists by themselves make this statement totally true! But there is other info in the Lahman that if combined with OF assists results in ratings for OF arms that can match real life pretty darn good. The top 50 players in OF arm can (arguably of course) be the players one would think ought to be there.

The other half of the equation is the effect fielding ratings have in OOTP. It is easy to see there is not enough of a spread between the good and bad fielders. A good arm in RF needs to keep the runner from trying for third or home. I play out every game and don't notice any difference in this play whether the fielder in right has an arm rating of 25 or 125. I am sure there is a difference but it is miniscule. I saw one of Ichiro's first games, the game against the A's where he threw out the runner at third with a spot on laser throw and noted no one tried to take an extra base on him, ever, for what seemed like a year but was actually probably only a month or two. Still that sort of thing should be modeled. Mazeroski on the DP, should be modeled.

And I don't mean modeled outside the game by some extra db shenanigans but inside the game, as just another formula.

I am guessing here but given how spot-on the batting and pitching is in OOTP I am thinking Markus just hasn't got around to spot-oning fielding yet.

[swampdragon]

Do you have specific formulae for these ratings that you play around with? I'd love to see them. Is there a way to put them in the Spritze db? Maybe as an attachment?

[Rondell Tate]

Originally Posted by Spritze

Spritze Defensive C Arm Ratings
Bench 2.52
Haller 1.85
Sundberg 3.07

Major league average C arm rating is set at 1.00, rating is 100% based on the available Lahman stats so it can be done correctly with the right formula.

OF arm examples... Clemente 3.83, Ichiro Suzuki 1.85, major league average is pegged at 1.00, still just looking at available Lahman stats in the right light.

I am totally confidant Markus can come up with a better fielding rating formula than the current fairly inaccurate one.

So I'm using the wrong DB? Or my scout's a moron?
Bench arm 10/20
Haller arm 9/20
Sundberg arm 12/20

[pstrickert]

I posted this before, but I'll do it again here. Even if league BABIP in OOTP is close to real life numbers, the individual team BABIP often departs crazily from real life. IMO, this demonstrates a significant flaw in OOTP.

1974 Import - 1 year recalc - INJ off

BABIP (points +/- league average)

real life vs. sim

Baltimore: -10 ....... +9
Oakland: -13 ....... -1
New York (A): +1 ....... -21
Los Angeles: -20 ...... -3
Atlanta: -15 ..... +3
Chicago (NL): +26 ...... -9

[Spritze]

Originally Posted by swampdragon

Do you have specific formulae for these ratings that you play around with? I'd love to see them. Is there a way to put them in the Spritze db? Maybe as an attachment?

Yes I have the formulas, they are included in my personal OOTP db, but there is no way currently to get them into the game other than hand entry or export/change/import. I'll add a (good idea!) csv to the next Spritze db that includes them (results and formulas) and also try to get them posted weekend after the upcoming one. I'm out of town as of yesterday. I actually notice a great improvement in historical play when I do the export/change/import thing. Fielding stuff seems to take a big step forward. It's just a pain in the a**.

[Spritze]

Originally Posted by Rondell Tate

So I'm using the wrong DB? Or my scout's a moron?

I'd vote for moron AND the rating formula in OOTP doesn't take the right Lahman fields into account for catchers arms I'll bet. Possibly because all three of these fellas might have a 20/20 rated catcher arm?

[swampdragon]

Originally Posted by Spritze

Yes I have the formulas, they are included in my personal OOTP db, but there is no way currently to get them into the game other than hand entry or export/change/import. I'll add a (good idea!) csv to the next Spritze db that includes them (results and formulas) and also try to get them posted weekend after the upcoming one. I'm out of town as of yesterday. I actually notice a great improvement in historical play when I do the export/change/import thing. Fielding stuff seems to take a big step forward. It's just a pain in the a**.

Great! I'm looking forward to seeing them. Thank you.

[Rondell Tate]

Originally Posted by Spritze

I'd vote for moron AND the rating formula in OOTP doesn't take the right Lahman fields into account for catchers arms I'll bet. Possibly because all three of these fellas might have a 20/20 rated catcher arm?

I'd vote for 20/20 - 18/20 - 20/20 (never saw Haller that I can remember, but he's generally rated a small step behind the other two ...)
In 1968, Bench had 102 assists because it took a while for the league to learn not to run on him. By 1972, there were half as many attempts and the successful steal rate had shot up from less than 50 to 56, largely because if you weren't a track star, you just didn't try running on him ... so I can see where it's hard to automatically import those numbers and have them make sense.
Ditto OF, where I had the privilege of watching Andre Dawson and Jesse Barfield in the outfield, and nobody ran on them. Tim Raines and George Bell, guys tested their arms all the time ... in 1983, Raines had 23 OF assists and Dawson 6. Six! Again, as with Bench, when Dawson had the ball, you just held your ground ... so to hear that you've gotten good imports on that is very, very impressive.

[swampdragon]

Originally Posted by pstrickert

I posted this before, but I'll do it again here. Even if league BABIP in OOTP is close to real life numbers, the individual team BABIP often departs crazily from real life. IMO, this demonstrates a significant flaw in OOTP.

1974 Import - 1 year recalc - INJ off

BABIP (points +/- league average)

real life vs. sim

Baltimore: -10 ....... +9
Oakland: -13 ....... -1
New York (A): +1 ....... -21
Los Angeles: -20 ...... -3
Atlanta: -15 ..... +3
Chicago (NL): +26 ...... -9

So in general there's a 20 point spread. That's an acceptable amount. If they all looked like Baltimore's, I'd think we were very close to getting it right.

[Spritze]

Originally Posted by Rondell Tate

In 1968, Bench had 102 assists because it took a while for the league to learn not to run on him. By 1972, there were half as many attempts and the successful steal rate had shot up from less than 50 to 56, largely because if you weren't a track star, you just didn't try running on him ... so I can see where it's hard to automatically import those numbers and have them make sense.

Good example, here is how it works, very roughly. The 102 assists were the Arm test Bench passed, proved he had an arm, the 50 assists the next year are fairly inconsequential but the 140 games caught are not inconsequential nor is his age. What I do is spot the career mid point of his arm and his age at that point and the high point of his arm and that age. Bench's career mid point arm came out to be the square root of 2.52.

That is why the average C arm is pegged at 1. If you square the average arm of 1 you still get 1 but a .9 arm goes to a .81 for instance, this puts more separation between the C arm ratings and allows the good arms to be gooderer.

From there you let his age and the number of games played at catcher determine if his arm gets weaker rather than pegging arm to the number of assists. As you noted, the number of assists lessens because the opponents get smarter, not because Bench's arm gets weaker.

That gives you C arm strengths that make sense. OF's are even more important to be done this way because OF assists for a good armed RF'er might equal zero, but his arm should not track that number in any way. The number of assists in this case is inconsequential. The number of games played in RF is not.