Historical Accuracy Overrated?
 

I know a lot of people rave about the historical results for this version of OOTP. I agree the game is improved in that respect over 2006. However, I still think it leaves a lot to be desired. I'm surprised more people have not noticed the flaws in the historical aspect of the game.

Here's what I decided to do. I imported the 1982 MLB season from the Lahman DB. I turned Coaching off, Scouts off, Financials off, Amateur Draft off, Morale off, Personality off. I disabled player development. I kept injuries on and trading on. I set the Ratings Recalc to 1 year. I turned off the option to auto adjust league total modifiers after each season. (I'm not sure how those last two options would matter anyhow, since I only simmed the 1982 season to the playoffs.)

I decided to focus on 8 players for the Milwaukee Brewers: Moose Haas, Pete Vuckovich, Mike Caldwell, Rollie Fingers, Gorman Thomas, Jim Gantner, Robin Yount, and Ed Romero. I then simmed the 1982 season and looked at the stats for these 8 players. Then I did everything all over again. I imported the 1982 MLB season from Lahman DB and set the options exactly as I had before. I did this five different times.

What would you expect to happen? Maybe I'm wrong, but I expected the ratings and stats to be reasonably consistent from one sim to the next. Here are the results:

MOOSE HAAS
Real Life: 193.1 IP, 11-8, 4.47
Sim 1: 255.2, 14-12, 2.68
Sim 2: 247.2, 18-12, 3.96
Sim 3: 259.2, 15-13, 3.71
Sim 4: 253.2, 20-8, 3.55
Sim 5: 246, 22-5, 2.74

Haas received a 4-1/2 star rating each and every time.

COMMENT: Notice that his ERA in every case was lower than his RL ERA. Notice also that his ERA ranged from 2.68 to 3.96. That's not very consistent. Notice also that he pitched a lot more innings in the sims than IRL.

PETE VUCKOVICH
Real Life: 223.2 IP, 18-6, 3.34 (Cy Young Award)
Sim 1: 144.2, 4-10, 4.42
Sim 2: 249.1, 20-10, 3.72
Sim 3: 94, 1-6, 5.74 (he lost his starting role due to ineffectiveness, not injury)
Sim 4: 194.2, 16-6, 3.10
Sim 5: 231.1, 16-7, 2.68

Vuckovich received a 1 star rating every time, except for Sim 2 in which he received a 1-1/2 star rating. Who knows why?

COMMENT: I have no idea how to explain the radically different results from one sim to the next.


MIKE CALDWELL
Real Life: 258 IP, 17-13, 3.91
Sim 1: 235.1, 16-11, 4.28
Sim 2: 148.1, 9-4, 3.64
Sim 3: 191.1, 14-7, 3.53
Sim 4: 216.1, 13-9, 4.24
Sim 5: 200, 18-3, 2.92

Caldwell received a 1-1/2 star rating in the first and fourth sim. In the other three sims, he received only 1 star.

COMMENT: For a pitcher with such low ratings, he actually performed fairly well. In fact, his Sim 5 record was Cy Young quality.


ROLLIE FINGERS
Real Life: 79.2 IP, 5-6, 29 SV, 2.60
Sim 1: 62.1, 6-5, 33, 3.03
Sim 2: 50.1, 4-4, 33, 2.86
Sim 3: 57.1, 6-6, 31, 1.88
Sim 4: 65.1, 3-6, 42, 2.89
Sim 5: 61.1, 4-5, 49, 3.52

Fingers received a 4-star rating for every sim, except the first in which he received 4-1/2 stars.

COMMENT: His ERAs ranged from 1.88 to 3.52. Now a historical simmer might see a career ERA near 2.60 and conclude that OOTP does a great job producing historically accurate results. I look at the numbers and think it's a crapshoot on a single season basis.


GORMAN THOMAS
Real Life: 567 AB, 39 HR, 112 RBI, .245 BA, .343 OBA, .506 SLG
Sim 1: 576, 35, 106, .236, .330, .464
Sim 2: 549, 37, 96, .260, .348, .492
Sim 3: 186, 12, 26, .237, .313, .462 (CEI)
Sim 4: 519, 25, 73, .245, .328, .443
Sim 5: 586, 29, 83, .239, .307, .423

Thomas received 4 stars in the first, second, and fourth sims, 4-1/2 stars in the other two.

COMMENT: BA looked reasonable, but his power numbers were lower than expected.


JIM GANTNER
Real Life: 447 AB, 4 HR, 43 RBI, .295 BA, .335 OBA, .369 SLG
Sim 1: 387, 2, 36, .274, .320, .331
Sim 2: 669, 8, 61, .294, .331, .366
Sim 3: 681, 7, 78, .289, .316, .376
Sim 4: 549, 1, 67, .330, .353, .413
Sim 5: 677, 2, 57, .257, .288, .316

Gantner received a 3-1/2 star rating for every sim, except for Sim 4 when he received 4 stars.

COMMENT: BA ranged from .257 to .330. That's quite uneven.


ROBIN YOUNT
Real Life: 635, 29, 114, .331, .379, .578
Sim 1: 684, 8, 83, .289, .334, .418
Sim 2: 656, 18, 111, .360, .401, 546
Sim 3: 593, 18, 90, .309, .352, .501
Sim 4: 625, 14, 85, .317, .372, .474
Sim 5: 613, 21, 115, .297, .359, .475

Yount received a 5-star rating every time.

COMMENT: He batted below average 4 times out of 5 (14-43 points below average). He batted 29 points above his RL average the other time. Go figure. His power production was well below his real life numbers.


ED ROMERO
Real Life: 144, 1, 7, .250, .289, .326
Sim 1: 517, 12, 53, .242, .284, .360
Sim 2: 559, 5, 61, .272, .307, .372
Sim 3: 232, 5, 23, .246, .275, .375
Sim 4: 590, 11, 76, .275, .314, .398
Sim 5: 433, 10, 78, .330, .362, .462

Romero was rated 1 star every time.

COMMENT: For some reason, the OOTP AI loved Ed Romero. He got a lot of playing time at 2B (with Jim Gantner, for some reason, being assigned to be the starting 3Bman). Don't ask me why. His BA ranged from .242 to .330. Nothing consistent about that. He also showed a lot more power than IRL.

I deliberately gave this thread a provocative title. I think OOTP's a great game. I'm pleased with the vast improvements. But I cringe a bit when I hear people boast about the historical accuracy of the sim results. Maybe the sim totals for the great players work out OK in the end. But I'm not seeing such good results when I look at things up close.

Thoughts? Theories? Suggestions?

Try it with a three year recalc. I'm using that and getting great results.

It's a well-known problem that OOTP doesn't do well with the first year in a sim (some testers have been trying, to little avail, to get this changed - perhaps it can't be for some reasons unclear to me). I bet if you simmed 1981 and then 1982 the 1982 results would be much closer.

pstrikert, you want to read this thread

http://www.ootpdevelopments.com/boar...d.php?t=145092

I ran the same sim but only with re-calc set to three years and auto adjust league total modifiers on. Here's what I got for those players.


MOOSE HAAS
Real Life: 193.1 IP, 11-8, 4.47
Sim 1: 236.1, 13-13, 3.69
Sim 2: 263.0, 17-9, 3.76
Sim 3: 269.0, 19-11, 2.98
Sim 4: 276.2, 19-11, 3.68


PETE VUCKOVICH
Real Life: 223.2 IP, 18-6, 3.34 (Cy Young Award)
Sim 1: 216.0, 13-10, 4.71
Sim 2: 211.2, 14-12, 4.04
Sim 3: 257.1, 18-7, 2.62
Sim 4: 253.1, 19-10, 3.77


MIKE CALDWELL
Real Life: 258 IP, 17-13, 3.91
Sim 1: 83.1, 4-2, 4.10
Sim 2: 119.2, 8-6, 4.44
Sim 3: 87.0, 5-6, 5.90
Sim 4: 125.1, 6-7, 5.67


ROLLIE FINGERS
Real Life: 79.2 IP, 5-6, 29 SV, 2.60
Sim 1: 56.1, 3-5, 39, 2.72
Sim 2: 62.2, 6-5, 44, 2.73
Sim 3: 67.2, 5-3, 41, 2.66
Sim 4: 69.1, 4-6, 45, 2.99


GORMAN THOMAS
Real Life: 567 AB, 39 HR, 112 RBI, .245 BA, .343 OBA, .506 SLG
Sim 1: 533, 25, 82, .227, .299, .428
Sim 2: 577, 36, 111, .248, .319, .499
Sim 3: 597, 39, 111, .240, .310, .486
Sim 4: 546, 31, 84, .216, .314, .416


JIM GANTNER
Real Life: 447 AB, 4 HR, 43 RBI, .295 BA, .335 OBA, .369 SLG
Sim 1: 469, 5, 52, .269, .324, .365
Sim 2: 493, 6, 56, .292, .319, .369
Sim 3: 498, 15, 73, .281, .307, .442
Sim 4: 643, 6, 62, .264, .297, .344


ROBIN YOUNT
Real Life: 635, 29, 114, .331, .379, .578
Sim 1: 406, 12, 67, .298, .345, .443 (injured for 12 months on July 24th)
Sim 2: 635, 22, 100, .312, .360, .498
Sim 3: 179, 7, 26, .374, .426, .564 (injured for 7 months on May 27th)
Sim 4: 650, 9, 74, .286, .354, .411


ED ROMERO
Real Life: 144, 1, 7, .250, .289, .326
Sim 1: 542, 4, 58, .297, .325, .375
Sim 2: 278, 4, 34, .309, .351, .388
Sim 3: 152, 0, 13, .263, .282, .309
Sim 4: 456, 2, 52, .250, .279, .322

That doesn't look any better than using 1-year recalc IMO. Then again, what should it matter for a single season? Recalc is supposed to be put into effect AFTER the season, right?

I've heard that before. Here's the concern, though: Let's say the ratings for individual players for that first year in a sim may not be reliable. Even so, wouldn't you expect there to be some consistency from one sim to the next? If Pete Vuckovich is rated a 1-star pitcher, shouldn't his season stats look like the stats of a 1-star pitcher? Sometimes, they do (4-10, 4.42). Other times, they look more like a 4-star or 5-star pitcher (16-7, 2.68). Should Robin Yount, a 5-star player, hit .289 in one sim and .360 the next? Let's say I sim 1981 and then 1982. If Robin Yount then receives a 5-star rating for the 1982 season (as he likely would!), why should I not expect his BA to be .289? Or .360? Why should I expect it to be closer to his RL BA of .331?

I've seen that. Again, it's not the career totals that bother me. Everything "evens out" in the end. It's the individual season totals that bother me. More often than not, they don't look very accurate.

If you want a game that simulates with extreme accuracy a single season, I highly recommend Diamond Mind Baseball or Strat-O-Matic.

Its silly to expect innings pitched/at bats etc to be the same (or perhaps even close to the same) because these are determined by game situations that wont be the same as they were during the real season. It should be enough that the players are accuratly recreated.

As noted in a prior post, there are other games which are specifically designed to faithfully recreate the results of a single season. OOTP is an open-ended career simulator, so I don't how one can reasonably expect it to faithfully recreate the results of a single season given the huge number of potential variables at work. By your own account you left injuries and trading on; those two items alone offer up a lot of variables.

In addition, the computer knows a lot more about its team, especially in a first season, than GMs know in real life. I was semi-working on a 1982 replay and the game likes to stick Matt Sinatro of the Braves in A-ball. IRL Sinatro got 81 at-bats. Problem was, he hit like .161 in those 81 at-bats. In long-term replays I see teams give bad players that many ABs due to not having anybody else ready or thinking that a player is better than he really is (due to bad scouting or a sudden downturn), but not so much in single-year replays.

Again, DMB and SOM do a fantastic job with this sort of thing. You can even play out a season with real rosters, lineups, and pitching rotations as actually played. They don't do career mode, which is the point of OOTP.

I own DMB -- it has its own shortcomings, though. I don't expect OOTP to be as accurate as DMB, but I expect it to be more accurate than what I've seen so far.

But how do you expect it to account for all the variables?

I included the IPs to show that the ERA was not due to a limited number of IPs. Same with ABs. I included them to put the HR and RBI totals in perspective.

Okay, I did it again but this time starting in 1981 (as suggested above). Here's what I got.

MOOSE HAAS
Real Life: 193.1 IP, 11-8, 4.47
Sim 1: 241.0, 13-11, 3.21
Sim 2: 243.0, 11-14, 3.85
Sim 3: 142.2, 10-5, 3.79


PETE VUCKOVICH
Real Life: 223.2 IP, 18-6, 3.34 (Cy Young Award)
Sim 1: 220.0, 6-15, 5.23
Sim 2: 245.2, 17-16, 4.21
Sim 3: 172.1, 9-11, 3.86


MIKE CALDWELL
Real Life: 258 IP, 17-13, 3.91
Sim 1: 109.2, 9-5, 3.61
Sim 2: 76.2, 4-3, 5.87
Sim 3: No stats. Got released at the beginning of '82 and didn't sign with anyone.


ROLLIE FINGERS
Real Life: 79.2 IP, 5-6, 29 SV, 2.60
Sim 1: 75.1, 7-3, 13, 1.79
Sim 2: 115.2, 13-1, 2, 1.01 (???)
Sim 3: 63.1, 4-3, 6, 3.55


GORMAN THOMAS
Real Life: 567 AB, 39 HR, 112 RBI, .245 BA, .343 OBA, .506 SLG
Sim 1: 517, 22, 74, .209, .302, .377
Sim 2: 591, 41, 112, .247, .323, .486
Sim 3: 505, 27, 91, .220, .312, .420


JIM GANTNER
Real Life: 447 AB, 4 HR, 43 RBI, .295 BA, .335 OBA, .369 SLG
Sim 1: 594, 0, 13, .246, .316, .261
Sim 2: 671, 9, 78, .310, .350, .408
Sim 3: 570, 6, 63, .328, .363, .447


ROBIN YOUNT
Real Life: 635, 29, 114, .331, .379, .578
Sim 1: 348, 9, 54, .322, .386, .489 (injured at the end of 1981, out for 10 months)
Sim 2: 505, 11, 74, .267, .313, .412
Sim 3: 531, 18, 89, .328, .371, .501


ED ROMERO
Real Life: 144, 1, 7, .250, .289, .326
Sim 1: 316, 1, 23, .212, .247, .256
Sim 2: 351, 3, 22, .239, .272, .308
Sim 3: 305, 1, 22, .243, .276, .302


The stuff for Fingers was kind of weird.

I tried to turn off any options that could influence the outcome. (See my first post.) I left injuries on, though. I thought a few minor injuries (along with fatigue) could prevent everyone in the starting lineup from playing all 162 games. Long-term injuries rarely impacted the 8 players I chose to document.

Did the AI make him a starter? I've seen that happen before, I believe, with 3-year recalc for the 1974 season. Fingers' position was listed as CL, but in the pitching chart he was listed as the fifth starter in the rotation. Weird.

His position was listed as a closer but I forgot to check my if he started any games and I've since deleted all of the test leagues I ran. Still, the numbers for him over those three test sims were pretty strange.

But what about all the players those eight guys played against? You can't look at just those eight players in isolation; their stats are generated by their game interactions with all the other players in the league.