Historical Accuracy Overrated?

[f.montoya]

I've enjoyed this thread and just thought I'd drop in for a word or two or three...

I'd like to state that the accuracy that beta testers had been applauding had most everything to do with long term sims. I personally have, and continue to be amazed at how long term sims produce very near to life stat outputs for hall of famers we all all expect them from. Of course real life injuries, playing time, manager decision making, trades, lineup placements, environment(ballparks, game situations, etc.) cannot be replicated and thus this leaves all those variables that can wreak havoc on "accuracy". Now keeping in mind that all these variables are part of the equation I still got the following stat output for some of the players you mentioned...

                                                                                                                                                                                                                                                                                                                                                                    <table x:str="" style="border-collapse: collapse; width: 408pt;" border="0" cellpadding="0" cellspacing="0" width="544"><col style="width: 117pt;" width="156"> <col style="width: 72pt;" width="96"> <col style="width: 54pt;" span="2" width="72"> <col style="width: 57pt;" width="76"> <col style="width: 54pt;" width="72"> <tbody><tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt; width: 117pt;" height="18" width="156">
</td> <td style="width: 72pt;" width="96">
</td> <td style="width: 54pt;" width="72">ERA</td> <td style="width: 54pt;" width="72">WHIP</td> <td style="width: 57pt;" width="76">K/9</td> <td style="width: 54pt;" width="72">
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">Moose Haas</td> <td>Real Life</td> <td class="xl22" x:num="4.01" align="right">4.01 </td> <td class="xl22" x:num="1.31" align="right">1.31 </td> <td class="xl22" x:num="4.63" align="right">4.63 </td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>OOTP 2007</td> <td class="xl22" x:num="3.85" align="right">3.85 </td> <td class="xl22" x:num="1.26" align="right">1.26 </td> <td class="xl22" x:num="4.19" align="right">4.19 </td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td>
</td> <td>
</td> <td>
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td>
</td> <td>
</td> <td>
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td>ERA</td> <td>WHIP</td> <td>K/9</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">Pete Vuckovich</td> <td>Real Life</td> <td class="xl22" x:num="3.66" align="right">3.66 </td> <td class="xl22" x:num="1.37" align="right">1.37 </td> <td class="xl22" x:num="5.45" align="right">5.45 </td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>OOTP 2007</td> <td class="xl22" x:num="3.56" align="right">3.56 </td> <td class="xl22" x:num="1.35" align="right">1.35 </td> <td class="xl22" x:num="5.5" align="right">5.50 </td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td class="xl22">
</td> <td class="xl22">
</td> <td class="xl22">
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td>
</td> <td>
</td> <td>
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td>
</td> <td>
</td> <td>
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td>HR</td> <td>AVG</td> <td>K</td> <td>Games</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">Robin Yount</td> <td>Real Life</td> <td class="xl24" x:num="251" align="right">251 </td> <td class="xl23" x:num="0.28499999999999998" align="right">0.285 </td> <td class="xl24" x:num="1350" align="right">1350 </td> <td x:num="" align="right">2856</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>OOTP</td> <td class="xl24" x:num="254" align="right">254 </td> <td class="xl23" x:num="0.27500000000000002" align="right">0.275 </td> <td class="xl24" x:num="1490" align="right">1490 </td> <td x:num="" align="right">2868</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td class="xl24">
</td> <td class="xl23">
</td> <td class="xl24">
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td class="xl24">
</td> <td class="xl23">
</td> <td class="xl24">
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td>HR</td> <td>AVG</td> <td>K</td> <td>Games</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">Jim Gantner</td> <td>Real Life</td> <td class="xl24" x:num="47" align="right">47 </td> <td class="xl23" x:num="0.27400000000000002" align="right">0.274 </td> <td class="xl24" x:num="501" align="right">501 </td> <td x:num="" align="right">1801</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>OOTP</td> <td class="xl24" x:num="22" align="right">22 </td> <td class="xl23" x:num="0.26700000000000002" align="right">0.267 </td> <td class="xl24" x:num="266" align="right">266 </td> <td x:num="" align="right">968</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td class="xl24">
</td> <td class="xl23">
</td> <td class="xl24">
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td class="xl24">
</td> <td class="xl23">
</td> <td class="xl24">
</td> <td>
</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td>HR</td> <td>AVG</td> <td>K</td> <td>Games</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">Ed Romero</td> <td>Real Life</td> <td class="xl24" x:num="8" align="right">8 </td> <td class="xl23" x:num="0.247" align="right">0.247 </td> <td class="xl24" x:num="159" align="right">159 </td> <td x:num="" align="right">730</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>OOTP</td> <td class="xl24" x:num="4" align="right">4 </td> <td class="xl23" x:num="0.22800000000000001" align="right">0.228 </td> <td class="xl24" x:num="66" align="right">66 </td> <td x:num="" align="right">457</td> </tr> <tr style="height: 13.5pt;" height="18"> <td style="height: 13.5pt;" height="18">
</td> <td>
</td> <td class="xl24">
</td> <td class="xl23">
</td> <td class="xl24">
</td> <td>
</td> </tr> </tbody></table>

The above was just a regular Lahman import of the 1982 season and simmed through 1996, thus I only included players in the chart above that would have played more than half, or all of their careers in OOTP. I don't see any stat above worth complaining about.

[f.montoya]

By the way, Coaches were on. Scouts OFF. Recalc was OFF, completely.

[RonCo]

Originally Posted by pstrickert

I'm rather pleased with the consistency from one sim to the next. I didn't expect that. I wonder, however, if the Lahman DB import works right for pitchers. I'd welcome any thoughts on that subject. RonCo?

Trial runs should be fairly consistent overall, but individual players can and will vary a bit just through the mathematics of random chance.

There is no doubt that the Lahman->OOTP import is not quite right for pitchers, given the new development curves. I've already spoken about that earlier. Markus has a TTed item on that, and I expect him to work on it.

A point on the "resulting stats" issue: I'm assuming your historical sims are not being played in a neutral, modern MLB setting. They are being played in with era-specific settings, and almost certainly in environments that are better for pitchers (and worse for hitters) than the calculation indicates.

I know a lot of folks use that "Resulting Stats" line to level set expectations, but, personally, I ignore it. Stats are the result of the rating/talent pool overlayed with ballparks and league totals. I don't think the "resulting stats" calculation is keyed to the league's parks or totals, so it's almost always going to be off--unless you play in all neutral parks and a modern day setting. For example, I import 1955, and look at Sandy Koufax...he is expected to K 86 and walk 84. I then change the league totals to 0 BB and double the league's Ks, then look at Koufax again and see that his "resulting" BB/K numbers have not changed.

The "resulting" numbers are still useful in that they give you a sense of ranking between players. And if you are baseball-smart and understand the nuances of how offense and pitching varied across the eras, you can mentally translate modern/neutral into 1955/Brooklyn. But overall, using "resulting stats" as a waypoint for your expectations is fraught with danger.

[dougaiton]

I see satchel pointed out the Moose Haas problem - he had a great season in everything but giving up hits (), which meant he gave up a ton of runs. OOTP discounts that and sees a 3.00-3.50 ERA type pitcher.

The other big thing is historical stadiums. Essentially, Brooks Robinson was the example I always used for my '69 league. Brooks had 10-15 homers a year power, but Memorial Stadium made that more like 20-25 homer power. OOTP imports a 20-25 homer player as it doesn't account for parks, and then sticks him in Memorial Stadium, where he becomes a 30-35 homer player!

[RonCo]

Originally Posted by dougaiton

The other big thing is historical stadiums. Essentially, Brooks Robinson was the example I always used for my '69 league. Brooks had 10-15 homers a year power, but Memorial Stadium made that more like 20-25 homer power. OOTP imports a 20-25 homer player as it doesn't account for parks, and then sticks him in Memorial Stadium, where he becomes a 30-35 homer player!

This is a good point. Setting ratings off the Lahman DB is essentially baking in park effect, since the stats were generated in parks the players actually played in. When a historical simmer then places the players in "accurate" parks, they are essentially doubling the park effect...assuming, of course, that the players are playing in the parks where they played in real life (and the models are close to right).

Now, you could decide to make all ballparks neutral, but then you're effectively sending players from place to place, but having them carry their park effects with them--since they are imported off straight stats, there's really no way to get around that. It's a built-in modifier that adds a small off-set that will be noticeable if you dig deep into the details. Neutral parks would also result in a player putting up equal stats regardless of ballpark. A casual glance at the stats would probably reveal them to be pretty good, though.

Ultimately, this is one of those areas where historical simmers are caught between a rock and a hard place. They want ballpark effects to be accurate, and they want to be able to play whatif/trade games, but they also want "realistic" stats. Since the import process is driven by stats created and influenced by ballparks, this is actually an impossible design box unless the import process is given enough intelligence to derate ratings based on park and statistical environment. That's doable, but would require fairly complex logic. I would be surprised if Markus would consider biting that off in a patch release--and maybe not even in 2008. That's for him to say, of course. I have no real idea what his code base is like.

[Ksyrup]

Originally Posted by RonCo

Ultimately, this is one of those areas where historical simmers are caught between a rock and a hard place. They want ballpark effects to be accurate, and they want to be able to play whatif/trade games, but they also want "realistic" stats. Since the import process is driven by stats created and influenced by ballparks, this is actually an impossible design box unless the import process is given enough intelligence to derate ratings based on park and statistical environment. That's doable, but would require fairly complex logic. I would be surprised if Markus would consider biting that off in a patch release--and maybe not even in 2008. That's for him to say, of course. I have no real idea what his code base is like.

Good point. That seems like something that probably can't be effectively remedied at any point in the near future. I think we just need to pick our poison and live with it. If I was just simming out a long stretch of history, I'd probably tend to go with the neutral ballparks, but if I was playing "in-game," I'd probably go with differing ballpark effects to add to the strategy of the game.

[ctorg]

Originally Posted by RonCo

This is a good point. Setting ratings off the Lahman DB is essentially baking in park effect, since the stats were generated in parks the players actually played in. When a historical simmer then places the players in "accurate" parks, they are essentially doubling the park effect...assuming, of course, that the players are playing in the parks where they played in real life (and the models are close to right).

Now, you could decide to make all ballparks neutral, but then you're effectively sending players from place to place, but having them carry their park effects with them--since they are imported off straight stats, there's really no way to get around that. It's a built-in modifier that adds a small off-set that will be noticeable if you dig deep into the details. Neutral parks would also result in a player putting up equal stats regardless of ballpark. A casual glance at the stats would probably reveal them to be pretty good, though.

Ultimately, this is one of those areas where historical simmers are caught between a rock and a hard place. They want ballpark effects to be accurate, and they want to be able to play whatif/trade games, but they also want "realistic" stats. Since the import process is driven by stats created and influenced by ballparks, this is actually an impossible design box unless the import process is given enough intelligence to derate ratings based on park and statistical environment. That's doable, but would require fairly complex logic. I would be surprised if Markus would consider biting that off in a patch release--and maybe not even in 2008. That's for him to say, of course. I have no real idea what his code base is like.

Actually, Markus has said that players imported from Lahman have their stats adjusted up or down for park factors, so guys who played in hitter's parks would have their batting ratings reduced and vice versa, so it's not too too bad.

[RonCo]

Originally Posted by ctorg

Actually, Markus has said that players imported from Lahman have their stats adjusted up or down for park factors, so guys who played in hitter's parks would have their batting ratings reduced and vice versa, so it's not too too bad.

Hmm...shows what I know.

[BaseballMan]

I wonder how accurate do the stats need to be?
You know what is acceptable to me may not be acceptable to others.
So i dont know how much more Markus can do with historical accuracy.
I really dont know how any game can do it. I think the idea of replaying baseball seasons in a game with accurate results is almost impossible. There is always gonna be a player that didnt perform like he did in real life. Unless you just want the team stats to be accurate.
But even there would you be satisfied if Gehrig hit 63 hrs while Ruth hit 39 for the 27 Yankees but yet they still won 108 games? I am just satisfied that the sats are in the ballpark but yet still allow for what if scenarios without turning a Menodza into a career .350 hitter 95 % of the time.

[f.montoya]

Originally Posted by BaseballMan

I wonder how accurate do the stats need to be?
You know what is acceptable to me may not be acceptable to others.
So i dont know how much more Markus can do with historical accuracy.
I really dont know how any game can do it. I think the idea of replaying baseball seasons in a game with accurate results is almost impossible. There is always gonna be a player that didnt perform like he did in real life. Unless you just want the team stats to be accurate. but even there would you be satisfied if Gehrig hit 63 hrs while Ruth hit 39 for the 27 Yankees but yet they still won 108 games? I am just satisfied that the sats are in the ballpark but yet still allow for what if scenarios without turning a Menodza into a career .350 hitter 95 % of the time.

Perfect statement. Thanks.

"What? No. I don't think about that a whole lot. I just open the morning paper and look to make sure I'm above the Mendoza line"

- George Brett -

[f.montoya]

If this guy hits .300, we have a bug. Otherwise...

[pstrickert]

Originally Posted by dougaiton

I see satchel pointed out the Moose Haas problem - he had a great season in everything but giving up hits (), which meant he gave up a ton of runs. OOTP discounts that and sees a 3.00-3.50 ERA type pitcher.

I'll bet a lot of bad pitchers, during contract negotiations, would like to argue that they had a great season, except for giving up a lot of hits and a lot of runs.

[Zitofan75]

Originally Posted by f.montoya

If this guy hits .300, we have a bug. Otherwise...

I have to say it makes me nostalgic for late 70s early 80s baseball, where shortstops were expected to be defensive wizards and not Home Run hitters.

[Syd Thrift]

Originally Posted by pstrickert

I'll bet a lot of bad pitchers, during contract negotiations, would like to argue that they had a great season, except for giving up a lot of hits and a lot of runs.

Except that pitchers don't really give up hits. They walk batters, they strike batters out, they give up long drives by throwing the ball down the middle of the plate... and that's about it. Pitchers have very little control over how many hits they give up on balls in play in real life, which is accurately modeled in OOTP. Ironically, it's *not* accurately modeled in DMB, which creates situations like these where people ignorant of OOTP think that Moose Haas (or Jeff Ballard or Dave Fleming... or heck, maybe even Chien-Ming Wang) is simmed poorly when in fact it's DMB that's simming poorly.

[Russ]

Originally Posted by Syd Thrift

Except that pitchers don't really give up hits. They walk batters, they strike batters out, they give up long drives by throwing the ball down the middle of the plate... and that's about it. Pitchers have very little control over how many hits they give up on balls in play in real life, which is accurately modeled in OOTP. Ironically, it's *not* accurately modeled in DMB, which creates situations like these where people ignorant of OOTP think that Moose Haas (or Jeff Ballard or Dave Fleming... or heck, maybe even Chien-Ming Wang) is simmed poorly when in fact it's DMB that's simming poorly.

I vehemently disagree with BABIP, however I will not pollute this thread.

[swampdragon]

Originally Posted by Syd Thrift

Except that pitchers don't really give up hits. They walk batters, they strike batters out, they give up long drives by throwing the ball down the middle of the plate... and that's about it. Pitchers have very little control over how many hits they give up on balls in play in real life, which is accurately modeled in OOTP. Ironically, it's *not* accurately modeled in DMB, which creates situations like these where people ignorant of OOTP think that Moose Haas (or Jeff Ballard or Dave Fleming... or heck, maybe even Chien-Ming Wang) is simmed poorly when in fact it's DMB that's simming poorly.

The problem is that the worst thing the game does is import accurate defensive ratings. DIPS depends on accurate defensive models to get realistic results. In order to do this, you'd have to base a good percentage of defensive rating on team BABIP, essentially giving the team as a whole an extra pool of defensive points and then dividing those among the positions. I don't get the feeling the game does this. So until the defensive ratings catch up to the offensive ones, a DIPS model will necessarily be less accurate than the older models.

[Syd Thrift]

Originally Posted by swampdragon

The problem is that the worst thing the game does is import accurate defensive ratings. DIPS depends on accurate defensive models to get realistic results. In order to do this, you'd have to base a good percentage of defensive rating on team BABIP, essentially giving the team as a whole an extra pool of defensive points and then dividing those among the positions. I don't get the feeling the game does this. So until the defensive ratings catch up to the offensive ones, a DIPS model will necessarily be less accurate than the older models.

True. However, since the vast, vast majority of pitchers don't get big ERA bumps or drops due to having good/bad fielders behind them, it still works better than the alternative if you look at it the right way.

I'm definitely not saying that the game is perfect here. Personally I'd like to see it find some way to create Fielding Win Shares and go off of those for ratings - not perfect either, not by a longshot, but so far they seem to be the most accurate way of tossing known fielding stats together and coming out with something that makes sense.

[swampdragon]

Originally Posted by Syd Thrift

True. However, since the vast, vast majority of pitchers don't get big ERA bumps or drops due to having good/bad fielders behind them, it still works better than the alternative if you look at it the right way.

I'm definitely not saying that the game is perfect here. Personally I'd like to see it find some way to create Fielding Win Shares and go off of those for ratings - not perfect either, not by a longshot, but so far they seem to be the most accurate way of tossing known fielding stats together and coming out with something that makes sense.

I can think of a number of ways to do a team-adjusted defensive system that would probably make sense, whether you use win shares or park-adjusted team BABIP. I just can't think of how you would import the ratings you came up with from Lahman.

[The Wolf]

Originally Posted by Zitofan75

I have to say it makes me nostalgic for late 70s early 80s baseball, where shortstops were expected to be defensive wizards and not Home Run hitters.

Analysis showed that a high SLG/OPS SS was much more valuable to a team than a Mendoza Line hitter of a glove wizard was. Blame Bil James.

[pstrickert]

Originally Posted by ctorg

Actually, Markus has said that players imported from Lahman have their stats adjusted up or down for park factors, so guys who played in hitter's parks would have their batting ratings reduced and vice versa, so it's not too too bad.

Park Factors can be broken down into 8 different categories: AVG Overall, AVG LHB, AVG RHB, Doubles, Triples, HR Overall, HR LHB, HR RHB. The Lahman import assigns the same Park Factor to each of the 8 categories. If we were able to input the specific Park Factors for each category, wouldn't we likely get more accurate stats? (For example, County Stadium in 1982 had a Park Factor of 94. However, the PF for HR Overall was 74. That seems like a significant difference to me.)