How OOTP 2007 Models Real Baseball

[Elendil]

Originally Posted by jdawgsenior

Elendil, fantastic work, as always.

It's been my impression that even in this version of OOTP, the best pitchers in any given league or dynasty will consistently have seasonal ERAs below 2. I've seen pitchers with double-digit strings of seasons of ERAs ranging from, say, 1.10 to 1.80 in generally neutral park and era settings. Of course I think there may be a need for some tweaking there. Can we take this problem - if it's in fact a problem - and trace it back to any of the factors you discussed at the beginning of this thread? My first guess is that pitcher K consistency is part of the problem, but beyond that, I'm not sure of any other connections.

Now that I think of it, these pitchers I'm referring to consistently seem to have very high K/BB ratios - generally 8:1 or higher over a long period of time. I get the impression that strikeouts and walks should be directly related to a small degree, and I don't see that as often as I'd like. Do you have time to test that at all?

Well, in 1995-2005 MLB, the correlation between K's and walks is a very modest (but statistically significant) 0.07. In the OOTP test I ran it's 0.10. So OOTP seems to capture that positive correlation, if it does exist. However, I think the problem is likely to be too much K consistency and especially too much BABIP consistency, which could be a result of qualifying DIPS a little too much.

When Markus was coding OOTP6, he found that if he fully implemented DIPS (gave pitchers no control over BABIP at all), no pitchers had ERAs under 3.00. So he gave some pitchers some control over BABIP, initially through the Stuff rating, which created a negative correlation between K/IP and BABIP. Now, it turns out he was right to want to give pitchers some control over BABIP, but I think he's given them too much control now. The problem might be that BABIP is in 2007 controlled through GB/FB ratio (GB pitchers have higher BABIP). However, some data have shown that OOTP has wider variation in GB% ratings compared to real-life MLB, and of course, GB% ratings don't change each year.

So I think the solution for 2008 is twofold: 1) compress the variance of GB% ratings in player creation; 2) allow the GB% rating to vary a little from year to year.

[Markus Heinsohn]

Good stuff we're getting closer and closer each year to make the game as realistic as possible...

[Elendil]

Originally Posted by Elendil

Well, those tables show simple regressions of the kind y=a+bx, where "y" is "this year's stat," "a" is the constant, "b" is a coefficient, and "x" is "last year's stat." So after I run those regressions for the MLB sample, I run the same regressions on the OOTP sample, and after each one I do a Hausman test to determine whether the coefficient ("b") in each OOTP-sample regression is equal to the coefficient in the corresponding MLB-sample regression. (Hope that's clear enough.)

So the Hausman test itself is actually quite simple. You take the coefficient and its standard error and use a t-statistic to determine whether you can reject, at a particular level of confidence, the null hypothesis that the coefficient is equal to some number (in this case, the coefficient derived from the equivalent MLB-sample regression). So a Hausman test is really just a generalization of the more usual test of statistical significance done in a regression, i.e., that b=0.

LOL! I kept calling those tests "Hausman tests," which are something completely different. They are "Wald tests." I must have Hausman selection models on the mind from some other stuff I'm doing. Sorry for leading you astray, Injury Log!

[injury log]

Originally Posted by Elendil

LOL! I kept calling those tests "Hausman tests," which are something completely different. They are "Wald tests." I must have Hausman selection models on the mind from some other stuff I'm doing. Sorry for leading you astray, Injury Log!

All cool, Elendil- I won't complain if OOTP leads me to learn a few new things. Luckily, I've found some good info on Wald tests, so I should be able to work out what's going on- looks straightforward enough. Thanks for the clarification.

[Mattingly06]

Quick question, what method does OOTP use to model minor leaguers?

I noticed that a few of my youngsters didn't too well in the minors but performed good once I called them up. The opposite should happen imo, they should earn perform well in the minors earning themselves a shot at the bigs and then struggle a bit at first due to tougher competition. Is the current model too reliant on ratings and not enough stats which is a reflection of a players ability to put those ratings to good use? Just a bit curious.

[Elendil]

Originally Posted by Mattingly06

Quick question, what method does OOTP use to model minor leaguers?

I noticed that a few of my youngsters didn't too well in the minors but performed good once I called them up. The opposite should happen imo, they should earn perform well in the minors earning themselves a shot at the bigs and then struggle a bit at first due to tougher competition. Is the current model too reliant on ratings and not enough stats which is a reflection of a players ability to put those ratings to good use? Just a bit curious.

I've no idea, but it would be interesting to test. You could regress a player's career batting/pitching stats at the major league level on their career minor league stats, ideally adjusted for the ages at which they played in the minors. Then compare OOTP and MLB. Would be a lot of work, but it can be done.

[RonCo]

Originally Posted by Mattingly06

Quick question, what method does OOTP use to model minor leaguers?

I noticed that a few of my youngsters didn't too well in the minors but performed good once I called them up. The opposite should happen imo, they should earn perform well in the minors earning themselves a shot at the bigs and then struggle a bit at first due to tougher competition. Is the current model too reliant on ratings and not enough stats which is a reflection of a players ability to put those ratings to good use? Just a bit curious.

The same method as it uses to model major leaguers. Maybe I don't understand your question. Note that minor league league totals can me modified now, so you have some control of minor league stats across a league, now.

[RonCo]

Stats are the result of ratings. The game, to my knowledge, does not use a player's current stats as a generator of any future stats (with the exception that a play who is performing well apparently has a small increase of a chance to get a potential boost).

[RonCo]

And personally, I'm 100% fine with stats being a result of ratings.

[Mattingly06]

Originally Posted by RonCo

The same method as it uses to model major leaguers. Maybe I don't understand your question. Note that minor league league totals can me modified now, so you have some control of minor league stats across a league, now.

I would like to know if I can use Minor League Equivalencies (MLE) to predict how well a player will do in the majors. I dont' know if this is possible at the moment because I see a number of people on my team who struggle in the minors, then do well in the majors (this may be caused by the morale system- struggling becuase he's mad at me thinking he should be up with the big club).

[RonCo]

You should be able to use real minor league equivalency equations, adjusting for ballparks, and then using the in-game difference between "MLE" for each level as another adjustment...though maybe you'll need to adjust for League Totals differences, too. Have to think about that a little.

Or you could just use the guy's ratings, which are essentially what the use of MLE on real players are attempting to devine. Kinda.

[Elendil]

Originally Posted by RonCo

Stats are the result of ratings. The game, to my knowledge, does not use a player's current stats as a generator of any future stats (with the exception that a play who is performing well apparently has a small increase of a chance to get a potential boost).

Agreed. Maybe I didn't understand Mattingly's question, but I thought he was talking about how realistically OOTP "manages" the minor leagues, so that minor league stats will be a good indicator of future major league performance (as good as they are in real life). Not sure what part of the game engine would affect that, maybe roster AI.

[His Own Bad Self]

Originally Posted by Elendil

Well, in 1995-2005 MLB, the correlation between K's and walks is a very modest (but statistically significant) 0.07. In the OOTP test I ran it's 0.10. So OOTP seems to capture that positive correlation, if it does exist. However, I think the problem is likely to be too much K consistency and especially too much BABIP consistency, which could be a result of qualifying DIPS a little too much.

When Markus was coding OOTP6, he found that if he fully implemented DIPS (gave pitchers no control over BABIP at all), no pitchers had ERAs under 3.00. So he gave some pitchers some control over BABIP, initially through the Stuff rating, which created a negative correlation between K/IP and BABIP. Now, it turns out he was right to want to give pitchers some control over BABIP, but I think he's given them too much control now. The problem might be that BABIP is in 2007 controlled through GB/FB ratio (GB pitchers have higher BABIP). However, some data have shown that OOTP has wider variation in GB% ratings compared to real-life MLB, and of course, GB% ratings don't change each year.

So I think the solution for 2008 is twofold: 1) compress the variance of GB% ratings in player creation; 2) allow the GB% rating to vary a little from year to year.

I follow you, and I agree. Much appreciated - thanks again.

[voxpoptart]

Terrific research in this post, and I'm delighted to see OOTP's modeling so well-done. (As far as one exception goes, I'm actually happy enough with OOTP creating a positive correlation of doubles and triples - to me logic says they should be positively correlated, even if real life disagrees rather firmly - but I can see the purist point of view.)

In terms of year-to-year consistency for batting average and doubles, I have an idea: wouldn't it be simple to solve the problem _and_ re-create a little of life's extra unpredictability by giving each player a small, random, invisible modifier to their BABIP and Gap rating each year? THe invisible modifier would warp each year's BA and 2B figures, that is, withoutchanging the underlying skill involved ... and thus without otherwise changing the balance of the game.

[tanglewood]

Perhaps the reason why some stats were more consistent in OOTP than MLB is because you turned off player trading? I'm thinking particulary of BABIP for pitchers here. Less player movement means that the pitching staff will generally have a similar fielding unit behind them year on year, more player movement would mix up those units and create a bit more inconsistency perhaps. I know that OOTP isn't very far off at all with that stat but just thinking aloud as to one reason why it might seem so in your sim but the engine may be bang on.

[Elendil]

Originally Posted by tanglewood

Perhaps the reason why some stats were more consistent in OOTP than MLB is because you turned off player trading? I'm thinking particulary of BABIP for pitchers here. Less player movement means that the pitching staff will generally have a similar fielding unit behind them year on year, more player movement would mix up those units and create a bit more inconsistency perhaps. I know that OOTP isn't very far off at all with that stat but just thinking aloud as to one reason why it might seem so in your sim but the engine may be bang on.

Sure, that might factor in a bit. Players did still move around in the sim, of course, due to waivers and free agency, but less often than in real life.

[RonCo]

My gut feeling is that consistency of pitcher's BABIP is mostly due to OOTP fielding being more consistent than MLBs, but that's really just a semi-educated hunch.

[ctorg]

This is great stuff.

Elendil, did you use Stata to do the analysis? I've used that program a bunch and it looks just like the output.

[Raidergoo]

Originally Posted by Elendil

I use Stata. I actually have the "do" file written up for anyone else who wants to process their own data in Stata.

Originally Posted by ctorg

This is great stuff.

Elendil, did you use Stata to do the analysis? I've used that program a bunch and it looks just like the output.

Yes, he did. What was truly impressive is that he did this for just about every build he had access to.

[Eugene Church]

Originally Posted by Mattingly06

Quick question, what method does OOTP use to model minor leaguers?

I noticed that a few of my youngsters didn't too well in the minors but performed good once I called them up. The opposite should happen imo, they should earn perform well in the minors earning themselves a shot at the bigs and then struggle a bit at first due to tougher competition. Is the current model too reliant on ratings and not enough stats which is a reflection of a players ability to put those ratings to good use? Just a bit curious.

This has always been a problem in OOTPB.

Players do better in the majors than they do in the minors.

The game promotes on ratings, rather than performance, which is not realistic.

I just did a 30-year sim in V2007 and this continues to be the case.

That is why I never use the minor leagues. The stats are not realistic.