OOTP Developments Forums - View Single Post

Elendil · 06-02-2007, 02:14 PM

What follows is a comparison of OOTP and MLB statistical output in certain fundamental areas of the game engine. I was going to do a blog on this during the beta process but never had the time. I've re-run the sim using patch 2.

I ran a 32-year sim on the default MLB-style quickstart, using regular MLB rules except that trading was turned off to speed up sim time. I exported the stats for the last 11 seasons to compare how OOTP models baseball compared to real-life MLB over the 1995-2005 period.

The first area of comparison is correlations among different aspects of hitter performance. The goal of this comparison is to figure out whether OOTP correctly models real-life hitter types. For instance, if OOTP tends to create fast power hitters with high BABIP and slow contact hitters with low BABIP, that would be wrong. In real life, power hitters tend to strike out and walk a lot, and are usually slow. Fast guys tend to hit fewer homeruns and have higher BABIP.

So the first thing I've done is to get Pearson correlations on a bunch of hitting stats for 1995-2005 MLB, each player-season being a single "observation," to use statistical language. I've only included player-seasons with at least 400 plate appearances, to eliminate "cup of coffee" players whose performances will vary widely because of the sample size problem.

Here's a guide to the abbreviations of the individual statistical variables:

avg=batting average
babip=batting average on balls in play
kab=strikeouts per at-bat
hrab=home runs per at-bat
bbpa=walks per plate appearance
doab=doubles per at-bat
trab=triples per at-bat
hppa=hit by pitches per plate appearance
sbattg=stolen base attempts per game played

Here's the table of MLB correlations. A variable's correlation with itself is of course 1. The number underneath each correlation is the statistical significance level. Numbers under 0.05 indicate that the correlation is statistically significant from zero at the 95% confidence level.

Code:

         |      avg    babip      kab     hrab     bbpa     doab     trab
----------+---------------------------------------------------------------
      avg |   1.0000 
          |
          |
    babip |   0.7792   1.0000 
          |   0.0000
          |
      kab |  -0.3503   0.0820   1.0000 
          |   0.0000   0.0001
          |
     hrab |   0.2330  -0.0094   0.4258   1.0000 
          |   0.0000   0.6511   0.0000
          |
     bbpa |   0.1252   0.0966   0.2980   0.4521   1.0000 
          |   0.0000   0.0000   0.0000   0.0000
          |
     doab |   0.4015   0.3271  -0.0304   0.2185   0.1182   1.0000 
          |   0.0000   0.0000   0.1443   0.0000   0.0000
          |
     trab |   0.0706   0.1696  -0.1271  -0.3049  -0.1337  -0.1517   1.0000 
          |   0.0007   0.0000   0.0000   0.0000   0.0000   0.0000
          |
     hppa |   0.0028   0.0232   0.0842   0.0776  -0.0105   0.0789  -0.0525 
          |   0.8910   0.2645   0.0000   0.0002   0.6139   0.0001   0.0116
          |
   sbattg |   0.0820   0.1633  -0.1497  -0.3107  -0.0310  -0.2057   0.4291 
          |   0.0001   0.0000   0.0000   0.0000   0.1364   0.0000   0.0000
          |

          |     hppa   sbattg
----------+------------------
     hppa |   1.0000 
          |
          |
   sbattg |  -0.0161   1.0000 
          |   0.4379

So we find some interesting things here. Of course batting average is strongly positively associated with BABIP, and negatively associated with strikeouts per at-bat (failure to put balls in play). Nothing surprising there.

More interesting findings:

1) Stolen base attempts, a good proxy for speed, is modestly positively associated with BABIP (fast guys get more hits on balls in play, because they can beat out throws more easily), modestly negatively associated with strikeouts, fairly strongly negatively associated with home runs, modestly negatively associated with doubles, and strongly positively associated with triples.

2) Home run power is strongly positively associated with strikeouts and walks (thus, those two are correlated as well, more weakly). Think Ken Phelps. There's also some positive correlation between HR's and doubles.

3) BABIP is somewhat positively associated with doubles, and more weakly, triples. Doubles hitters have apparently learned how to hit it where they ain't.

4) Hit by pitches really aren't very correlated with anything else.

Conclusion: There are, roughly speaking, two types of hitter: slow power hitters who strike out and walk a lot, and fast contact hitters who have high BABIP, presumably mostly with singles, don't strike out or walk much, and hit triples. Of course, most players will not fit clearly into either type, because these are just general tendencies, not strict categories.

How does OOTP compare to real life? Here's the same table based on the years 2028-2038 from the OOTP sim described earlier, same plate appearance threshold:

Code:

          |      avg    babip      kab     hrab     bbpa     doab     trab
----------+---------------------------------------------------------------
      avg |   1.0000 
          |
          |
    babip |   0.8141   1.0000 
          |   0.0000
          |
      kab |  -0.3858   0.0592   1.0000 
          |   0.0000   0.0035
          |
     hrab |   0.1661  -0.2119   0.0615   1.0000 
          |   0.0000   0.0000   0.0024
          |
     bbpa |  -0.0294  -0.0488   0.0081   0.0607   1.0000 
          |   0.1466   0.0160   0.6901   0.0027
          |
     doab |   0.2617   0.2737  -0.1103  -0.1076   0.0049   1.0000 
          |   0.0000   0.0000   0.0000   0.0000   0.8077
          |
     trab |   0.0935   0.1453  -0.0553  -0.1743  -0.1080   0.1405   1.0000 
          |   0.0000   0.0000   0.0063   0.0000   0.0000   0.0000
          |
    hbppa |   0.0162   0.0201   0.0149   0.0089  -0.1190   0.0277   0.0263 
          |   0.4243   0.3224   0.4635   0.6602   0.0000   0.1712   0.1936
          |
   sbattg |   0.0560   0.1407  -0.0125  -0.2044  -0.0551  -0.0864   0.4296 
          |   0.0057   0.0000   0.5376   0.0000   0.0065   0.0000   0.0000
          |

          |    hbppa   sbattg
----------+------------------
    hbppa |   1.0000 
          |
          |
   sbattg |   0.0764   1.0000 
          |   0.0002

OOTP gets some things basically perfect. The relationship of batting average to BABIP and strikeouts is uncanny. Speed also has the expected relationships with BABIP, homers, and triples. The big deficiency is with the correlation of HR's to walks and strikeouts. This is way too small in OOTP. Also, doubles and triples are modestly positively correlated in OOTP because of the Gap Power concept, but in MLB doubles and triples are negatively correlated. Also, doubles and homers are essentially uncorrelated in OOTP, when they need to be positively correlated.

How to fix these issues? Player creation seems to be the place. You could set up a player creation algorithm that makes high-power guys also have good eye and bad avoid k's (and lower-power guys have bad eye and good avoid k's). Markus did say that he would work on this for 2008; it's a complex issue.

Next up: correlations among pitcher skills, OOTP vs. MLB.

06-02-2007, 02:14 PM	#1
Elendil Hall Of Famer Join Date: Dec 2003 Location: the dynasty forum Posts: 2,318	How OOTP 2007 Models Real Baseball What follows is a comparison of OOTP and MLB statistical output in certain fundamental areas of the game engine. I was going to do a blog on this during the beta process but never had the time. I've re-run the sim using patch 2. I ran a 32-year sim on the default MLB-style quickstart, using regular MLB rules except that trading was turned off to speed up sim time. I exported the stats for the last 11 seasons to compare how OOTP models baseball compared to real-life MLB over the 1995-2005 period. The first area of comparison is correlations among different aspects of hitter performance. The goal of this comparison is to figure out whether OOTP correctly models real-life hitter types. For instance, if OOTP tends to create fast power hitters with high BABIP and slow contact hitters with low BABIP, that would be wrong. In real life, power hitters tend to strike out and walk a lot, and are usually slow. Fast guys tend to hit fewer homeruns and have higher BABIP. So the first thing I've done is to get Pearson correlations on a bunch of hitting stats for 1995-2005 MLB, each player-season being a single "observation," to use statistical language. I've only included player-seasons with at least 400 plate appearances, to eliminate "cup of coffee" players whose performances will vary widely because of the sample size problem. Here's a guide to the abbreviations of the individual statistical variables: avg=batting average babip=batting average on balls in play kab=strikeouts per at-bat hrab=home runs per at-bat bbpa=walks per plate appearance doab=doubles per at-bat trab=triples per at-bat hppa=hit by pitches per plate appearance sbattg=stolen base attempts per game played Here's the table of MLB correlations. A variable's correlation with itself is of course 1. The number underneath each correlation is the statistical significance level. Numbers under 0.05 indicate that the correlation is statistically significant from zero at the 95% confidence level. Code: \| avg babip kab hrab bbpa doab trab ----------+--------------------------------------------------------------- avg \| 1.0000 \| \| babip \| 0.7792 1.0000 \| 0.0000 \| kab \| -0.3503 0.0820 1.0000 \| 0.0000 0.0001 \| hrab \| 0.2330 -0.0094 0.4258 1.0000 \| 0.0000 0.6511 0.0000 \| bbpa \| 0.1252 0.0966 0.2980 0.4521 1.0000 \| 0.0000 0.0000 0.0000 0.0000 \| doab \| 0.4015 0.3271 -0.0304 0.2185 0.1182 1.0000 \| 0.0000 0.0000 0.1443 0.0000 0.0000 \| trab \| 0.0706 0.1696 -0.1271 -0.3049 -0.1337 -0.1517 1.0000 \| 0.0007 0.0000 0.0000 0.0000 0.0000 0.0000 \| hppa \| 0.0028 0.0232 0.0842 0.0776 -0.0105 0.0789 -0.0525 \| 0.8910 0.2645 0.0000 0.0002 0.6139 0.0001 0.0116 \| sbattg \| 0.0820 0.1633 -0.1497 -0.3107 -0.0310 -0.2057 0.4291 \| 0.0001 0.0000 0.0000 0.0000 0.1364 0.0000 0.0000 \| \| hppa sbattg ----------+------------------ hppa \| 1.0000 \| \| sbattg \| -0.0161 1.0000 \| 0.4379 So we find some interesting things here. Of course batting average is strongly positively associated with BABIP, and negatively associated with strikeouts per at-bat (failure to put balls in play). Nothing surprising there. More interesting findings: 1) Stolen base attempts, a good proxy for speed, is modestly positively associated with BABIP (fast guys get more hits on balls in play, because they can beat out throws more easily), modestly negatively associated with strikeouts, fairly strongly negatively associated with home runs, modestly negatively associated with doubles, and strongly positively associated with triples. 2) Home run power is strongly positively associated with strikeouts and walks (thus, those two are correlated as well, more weakly). Think Ken Phelps. There's also some positive correlation between HR's and doubles. 3) BABIP is somewhat positively associated with doubles, and more weakly, triples. Doubles hitters have apparently learned how to hit it where they ain't. 4) Hit by pitches really aren't very correlated with anything else. Conclusion: There are, roughly speaking, two types of hitter: slow power hitters who strike out and walk a lot, and fast contact hitters who have high BABIP, presumably mostly with singles, don't strike out or walk much, and hit triples. Of course, most players will not fit clearly into either type, because these are just general tendencies, not strict categories. How does OOTP compare to real life? Here's the same table based on the years 2028-2038 from the OOTP sim described earlier, same plate appearance threshold: Code: \| avg babip kab hrab bbpa doab trab ----------+--------------------------------------------------------------- avg \| 1.0000 \| \| babip \| 0.8141 1.0000 \| 0.0000 \| kab \| -0.3858 0.0592 1.0000 \| 0.0000 0.0035 \| hrab \| 0.1661 -0.2119 0.0615 1.0000 \| 0.0000 0.0000 0.0024 \| bbpa \| -0.0294 -0.0488 0.0081 0.0607 1.0000 \| 0.1466 0.0160 0.6901 0.0027 \| doab \| 0.2617 0.2737 -0.1103 -0.1076 0.0049 1.0000 \| 0.0000 0.0000 0.0000 0.0000 0.8077 \| trab \| 0.0935 0.1453 -0.0553 -0.1743 -0.1080 0.1405 1.0000 \| 0.0000 0.0000 0.0063 0.0000 0.0000 0.0000 \| hbppa \| 0.0162 0.0201 0.0149 0.0089 -0.1190 0.0277 0.0263 \| 0.4243 0.3224 0.4635 0.6602 0.0000 0.1712 0.1936 \| sbattg \| 0.0560 0.1407 -0.0125 -0.2044 -0.0551 -0.0864 0.4296 \| 0.0057 0.0000 0.5376 0.0000 0.0065 0.0000 0.0000 \| \| hbppa sbattg ----------+------------------ hbppa \| 1.0000 \| \| sbattg \| 0.0764 1.0000 \| 0.0002 OOTP gets some things basically perfect. The relationship of batting average to BABIP and strikeouts is uncanny. Speed also has the expected relationships with BABIP, homers, and triples. The big deficiency is with the correlation of HR's to walks and strikeouts. This is way too small in OOTP. Also, doubles and triples are modestly positively correlated in OOTP because of the Gap Power concept, but in MLB doubles and triples are negatively correlated. Also, doubles and homers are essentially uncorrelated in OOTP, when they need to be positively correlated. How to fix these issues? Player creation seems to be the place. You could set up a player creation algorithm that makes high-power guys also have good eye and bad avoid k's (and lower-power guys have bad eye and good avoid k's). Markus did say that he would work on this for 2008; it's a complex issue. Next up: correlations among pitcher skills, OOTP vs. MLB. __________________ Heaven is kicking back with a double Talisker and a churchwarden stuffed with latakia. Last edited by Elendil; 06-03-2007 at 01:02 PM.