OOTP and DMB (Historical Accuracy)

pstrickert · 06-24-2008, 02:33 PM

Which is more accurate, OOTP or DMB? I suppose we would have to agree, first, about the definition of "accuracy." Fat chance, huh? Nevertheless, I would like to suggest that the historical accuracy of DMB, though highly regarded, may be a bit overblown. (I'm not putting down DMB, though. It is a great game in its own right. I simply hope to separate myth from fact.)

Garlon posted some OOTP numbers and, I hope, others will post their results, too. What about DMB? I perused the DMB message board for replay reports. I looked specifically for end-of-year stats, so that I could compare them to RL stats. I could post several examples, but this one should suffice (I added RL stats in brackets]:

I just bought the 1998 Season Disk and the first thing I did when I received it was sim the whole season for the heck of it. What I saw afterwards still has me speechless:

Mark McGwire - .355 Avg, 91 HR, 179 RBI, 180 RS, 145 BB
91 HR???? Should I be awed or appalled?

Here'e the other shocker, David Wells went 26-1 for the Yankees with a 2.22 ERA. He could have easily won 30 but the Yanks bullpen blew several leads for him. In reality, Wells was 18-4 witha 3.49 ERA in 1998.

McGwire hit #61 on 7/28 in the bottom of the ninth to win the game. Then hit #62 on 7/30. That's July 30th. He broke the record before August!
And he never hit more than 3 homers in a game. And he only did that twice.

I looked at this Splits. His numbers against Lefties are almost exactly the same as Real-Life. The only difference being that he was intentionally walked by LH 11 times in Real-Life and only 1 time in DMB.

It was against Right-handers where he put up numbers nowhere near Real-Life. In 378 DMB at-bats (as opposed to 379 in Real-Life):

0.397 Avg vs. .314 Real-life
0.522 OBP vs. .474
1.008 SPC vs. .794
150 Hits vs. 119
21 2B vs. 17
70 HR vs. 55
102 BB vs. 113
12 IBB vs. 17
2 HB vs. 4

And McGwire hits Right-handed....

I'm just shocked that a player could perform that far over his head in a given replay. 83 points higher on the Batting Average and of his 31 extra hits, 15 were homers? Wow. If you look at what he did in real-life, of his 152 hits, 70 wree homers. So its not unbelievable that if he did get 31 more hits against righties than in real-life, that 15 would be homers.

Other numbers from this replay:

Sammy Sosa .310 76 HR 164 RBI 138 RS [RL: .308 66 HR 158 RBI 134 RS]
Greg Vaughn .284 60 HR 141 RBI 123 RS [RL: .272 50 HR 119 RBI 112 RS]
Ken Griffey Jr. .302 58 HR 152 RBI 140 RS (Hey! Its close to real-life) [RL: .284 56 HR 146 RBI 120 RS]
Jose Canseco .254 57 HR 134 RBI 118 RS [RL: .237 46 HR 107 RBI 98 RS]
Andres Galarraga .313 56 HR 165 RBI 118 RS [RL: .305 44 HR 121 RBI 103 RS]
Juan Gonzalez .337 43 HR 170 RBI 130 RS [RL: .318 45 HR 157 RBI 110 RS]
Vinny Castilla .356 47 HR 174 RBI 135 RS [RL: .319 46 HR 144 RBI 108 RS]

Tony Gwynn won the NL batting title with a MLB leading .368 avg (47 pts higher than in real-life). He crushed LH at a .389 clip (.323 in real-life). Interesting since he's a Lefty. Castilla finished second with a .356, and then McGwire was third.

As in real-life, Bernie Williams won the AL batting title. This time with a .346 average (instead of .339). Scott Brosius came in second with a .344 average [RL: .300]. That's Scotty Brosius, not Wade Boggs.

and in a pedestrian year compared to these others...
Barry Bonds .304 39 HR 152 RBI 126 RS [RL: .303 37 HR 122 RBI 120 RS]
(13 triples as opposed to 7 in real-life.)

jsatterly Re: 1998 Replay - McGwire hits 91 HR??? #1 [-]

I'm no mathematician, but that seems to be a real outlier. If you simmed it again, you'd probably be surprised at how much it varies. McGuire might hit 40 the next time. I personally would like to see fewer unusual performances (i.e. an algorithm with less variation), but I have no idea how that would affect the game. Any math geniuses out there who could speculate?

Viva los Spurs Re: 1998 Replay - McGwire hits 91 HR??? #2 [-]

I just ran a quickie sim of the '98 season (real-life numbers in parentheses):

McGwire .333/.504/.770 (.299/.470/.705), 66 HR (70), 172 hits (152), 28 2B (20)
His numbers vs. lefties were pretty close .257/.473/.563 (.254/.473/.563 real-life) but he killed righties at .362/.516/.850 (.314/.474/.794 real-life).

timz Re: 1998 Replay - McGwire hits 91 HR??? #3 [-]

Quote:
--------------------------------------------------------------------------------
His numbers vs. lefties were pretty close .257/.473/.563 (.254/.473/.563 real-life)...
--------------------------------------------------------------------------------
Unless you mistyped, "pretty close" doesn't really begin to cover it.

aglenn99 Re: 1998 Replay - McGwire hits 91 HR??? #4 [-]

Maybe such skewed numbers are due to the fact that its a simulation and there's no having to worry about release points and arm angles. Yet should that account for a Righthanded batter performing that much better than Real-life against Right-handed pitching? (.400 vs. .300?)

I have a Mathematics degree (and I play DMB what a coincidence :-) ) so I can't really make too much out of this one replay. And Yes, assuming that DMB is purely a statistical replay, if I sim this season again 100 times, I may never see these kind of results again. And I should expect to see similar results in the opposite direction in at least one of those 100 sims.
Its still shocking to see those kind of numbers when you do see them. Mainly because I think of DMB as simulating Baseball as opposed to Baseball Stats. I play the game expecting it to behave as real-life. Which I have to remind myself its a computer game, not an alternate reality :-)
Maybe McGwire hit that many homers in DMB because the Computer Manager didn't pitch around him. Maybe in real-life, McGwire would have hit 91 homers if he was pitched to and not pitched around when first-base was open. Hmmm

Maybe the game needs to add some kind of "Approaching Milestone Pressure Factor" to the game. Then as guys are approaching 60 homers, or 300 wins, 500 homers, 44-game hit streak, etc. they could go into a cold streak. Its an idea. (Luke and the gang, thoughts?)

Or maybe make the Computer Manager smart enough not to pitch to the guy with 60 homers. There's a thought. "Hey, its June 1st and Rod Carew is hitting .420. Maybe I should walk him?" But now we're talking AI ;->
(Anyone sim the season where Bonds hit 75 homers? I'd like to see what his totals come out to be on average.)

But enough of beating a dead horse...Anyone pickup on my mention that David Wells went 26-1?

(In the history of MLB, the closest guy with 20+ wins to touch that winning percentage is Ron Guidry in 1978 when he went 25-3. Which is a .893 winning percentage. Can you image how big Wells' head would have been if he really went 26-1 in a season?)

timz Re: 1998 Replay - McGwire hits 91 HR??? #5 [-]

Quote:
--------------------------------------------------------------------------------
Maybe the game needs to add some kind of "Approaching Milestone Pressure Factor" to the game. Then as guys are approaching 60 homers, or 300 wins, 500 homers, 44-game hit streak, etc. they could go into a cold streak. Its an idea. (Luke and the gang, thoughts?)
--------------------------------------------------------------------------------
Hugely bad idea, IMO, though Strat-O-Matic uses it.

If you want a "baseball" sim instead of a "stat" sim you can simply use the projection disks instead of the season disks. By basing a player's performance on more than a single season of stats you set a baseline that doesn't lend itself quite so easily to outrageous performances. Player splits are much more realistic, too, with the larger sample size.

aglenn99 Re: 1998 Replay - McGwire hits 91 HR??? #6 [-]

Yeah, upon further reflection that "Add an approaching milestone factor" thing is a bad idea. Not everyone has a harder time as they approach a milestone. So how would you know who would get a negative factor and who would get a positive factor? So scratch that.

But I still stick with my other point that the Computer AI should be smart enough to realize that someone has 60 homers in August and thus that guy should be pitched around.

This is not the only replay to result in real "head scratchers." Sift through the DMB Replay message board for more examples. The point is: DMB's reputation for historical replay accuracy may be a bit overblown. It's very good, but not without flaws. As for OOTP, it has improved greatly in the "historical replay" department (though it has its flaws, too). OOTP's reputation for historical replay accuracy may, in fact, be underrated. If so, it's about time we corrected the myths. What do y'all think?

swampdragon · 06-24-2008, 02:57 PM

Before this version, I would say that DMB was well ahead of OOTP in terms of historical accuracy. You're certainly right that some of the DMB season disks can yield some weird results, and I think its accuracy level does tend to be a little overstated. This year's OOTP has a better feel to the game engine, and people are reporting very accurate results. At this point, the only thing I'm sure DMB does better is pbp.

skunkle333 · 06-25-2008, 10:45 AM

I haven't replayed 1959 or 2005 yet so I can't speak fully about OOTP9's accuracy but while playing 2008 I'm not seeing any radical or strange results albeit it obviously isn't a real season replay as such.

For some time now my actual play with DMB was replaying fav World Series and creating round robin tournaments with fav WSer teams etc. So full season accuracy isn't a real biggie with me like it used to be. I'm just looking for a solid degree of realism for my short replays. Just hope I can find a way to do WSeries replays in OOTP9 without playing through a whole season. Maybe some kind of exhibition setup.

So to speak to the issue of this thread on a small sample size scale as such I can just say this: In all the single games I've played with OOTP9 (maybe 25 or so pitch by pitch) I haven't seen anything that was out of kelter for a single game. Which I guess isn't saying much...or is it?

Had a great game recently with the WSox and Cubs...Zambrano against Buehrle and it was very realistic.

I'll report back later on my 59 and 2k5 full season replays.

OOTP9...pitch by pitch...with pics and sounds...fantastic...

bbastyr · 06-25-2008, 03:27 PM

s

swampdragon · 06-25-2008, 03:50 PM

Quote:

Originally Posted by bbastyr

s a DMB player, I'm very impressed with OOTPs engine. DMB's play by play is still at another level, though, and has a lot more variety.

It's not a matter of the OOTP pbp being bad. Remember that DMB has been focusing on that for about ten years now. DMB's pbp is outstanding by comparison with any sports sim I've ever played. The OOTP pbp looks OK if you compare it to Strat or APBA, and it gets better every year.

pstrickert · 06-25-2008, 04:42 PM

The DMB PbP is definitely NOT overrated! It's excellent! The OOTP PbP has improved, IMO, by leaps and bounds -- but more can (and hopefully will) be done to improve it!

bbastyr · 06-26-2008, 01:04 AM

OOTP's pbp is quite good. It's passed BBW as second favorite in-game pbp. With DMB, you still get a better feel for a ball game. OOTP is definitely closing the gap, though. More variety, a little more color, and park specific PBP is all that's needed.

06-24-2008, 02:33 PM	#1
pstrickert Hall Of Famer Join Date: Dec 2005 Posts: 15,741	OOTP and DMB (Historical Accuracy) Which is more accurate, OOTP or DMB? I suppose we would have to agree, first, about the definition of "accuracy." Fat chance, huh? Nevertheless, I would like to suggest that the historical accuracy of DMB, though highly regarded, may be a bit overblown. (I'm not putting down DMB, though. It is a great game in its own right. I simply hope to separate myth from fact.) Garlon posted some OOTP numbers and, I hope, others will post their results, too. What about DMB? I perused the DMB message board for replay reports. I looked specifically for end-of-year stats, so that I could compare them to RL stats. I could post several examples, but this one should suffice (I added RL stats in brackets]: I just bought the 1998 Season Disk and the first thing I did when I received it was sim the whole season for the heck of it. What I saw afterwards still has me speechless: Mark McGwire - .355 Avg, 91 HR, 179 RBI, 180 RS, 145 BB 91 HR???? Should I be awed or appalled? Here'e the other shocker, David Wells went 26-1 for the Yankees with a 2.22 ERA. He could have easily won 30 but the Yanks bullpen blew several leads for him. In reality, Wells was 18-4 witha 3.49 ERA in 1998. McGwire hit #61 on 7/28 in the bottom of the ninth to win the game. Then hit #62 on 7/30. That's July 30th. He broke the record before August! And he never hit more than 3 homers in a game. And he only did that twice. I looked at this Splits. His numbers against Lefties are almost exactly the same as Real-Life. The only difference being that he was intentionally walked by LH 11 times in Real-Life and only 1 time in DMB. It was against Right-handers where he put up numbers nowhere near Real-Life. In 378 DMB at-bats (as opposed to 379 in Real-Life): 0.397 Avg vs. .314 Real-life 0.522 OBP vs. .474 1.008 SPC vs. .794 150 Hits vs. 119 21 2B vs. 17 70 HR vs. 55 102 BB vs. 113 12 IBB vs. 17 2 HB vs. 4 And McGwire hits Right-handed.... I'm just shocked that a player could perform that far over his head in a given replay. 83 points higher on the Batting Average and of his 31 extra hits, 15 were homers? Wow. If you look at what he did in real-life, of his 152 hits, 70 wree homers. So its not unbelievable that if he did get 31 more hits against righties than in real-life, that 15 would be homers. Other numbers from this replay: Sammy Sosa .310 76 HR 164 RBI 138 RS [RL: .308 66 HR 158 RBI 134 RS] Greg Vaughn .284 60 HR 141 RBI 123 RS [RL: .272 50 HR 119 RBI 112 RS] Ken Griffey Jr. .302 58 HR 152 RBI 140 RS (Hey! Its close to real-life) [RL: .284 56 HR 146 RBI 120 RS] Jose Canseco .254 57 HR 134 RBI 118 RS [RL: .237 46 HR 107 RBI 98 RS] Andres Galarraga .313 56 HR 165 RBI 118 RS [RL: .305 44 HR 121 RBI 103 RS] Juan Gonzalez .337 43 HR 170 RBI 130 RS [RL: .318 45 HR 157 RBI 110 RS] Vinny Castilla .356 47 HR 174 RBI 135 RS [RL: .319 46 HR 144 RBI 108 RS] Tony Gwynn won the NL batting title with a MLB leading .368 avg (47 pts higher than in real-life). He crushed LH at a .389 clip (.323 in real-life). Interesting since he's a Lefty. Castilla finished second with a .356, and then McGwire was third. As in real-life, Bernie Williams won the AL batting title. This time with a .346 average (instead of .339). Scott Brosius came in second with a .344 average [RL: .300]. That's Scotty Brosius, not Wade Boggs. and in a pedestrian year compared to these others... Barry Bonds .304 39 HR 152 RBI 126 RS [RL: .303 37 HR 122 RBI 120 RS] (13 triples as opposed to 7 in real-life.) jsatterly Re: 1998 Replay - McGwire hits 91 HR??? #1 [-] I'm no mathematician, but that seems to be a real outlier. If you simmed it again, you'd probably be surprised at how much it varies. McGuire might hit 40 the next time. I personally would like to see fewer unusual performances (i.e. an algorithm with less variation), but I have no idea how that would affect the game. Any math geniuses out there who could speculate? Viva los Spurs Re: 1998 Replay - McGwire hits 91 HR??? #2 [-] I just ran a quickie sim of the '98 season (real-life numbers in parentheses): McGwire .333/.504/.770 (.299/.470/.705), 66 HR (70), 172 hits (152), 28 2B (20) His numbers vs. lefties were pretty close .257/.473/.563 (.254/.473/.563 real-life) but he killed righties at .362/.516/.850 (.314/.474/.794 real-life). timz Re: 1998 Replay - McGwire hits 91 HR??? #3 [-] Quote: -------------------------------------------------------------------------------- His numbers vs. lefties were pretty close .257/.473/.563 (.254/.473/.563 real-life)... -------------------------------------------------------------------------------- Unless you mistyped, "pretty close" doesn't really begin to cover it. aglenn99 Re: 1998 Replay - McGwire hits 91 HR??? #4 [-] Maybe such skewed numbers are due to the fact that its a simulation and there's no having to worry about release points and arm angles. Yet should that account for a Righthanded batter performing that much better than Real-life against Right-handed pitching? (.400 vs. .300?) I have a Mathematics degree (and I play DMB what a coincidence :-) ) so I can't really make too much out of this one replay. And Yes, assuming that DMB is purely a statistical replay, if I sim this season again 100 times, I may never see these kind of results again. And I should expect to see similar results in the opposite direction in at least one of those 100 sims. Its still shocking to see those kind of numbers when you do see them. Mainly because I think of DMB as simulating Baseball as opposed to Baseball Stats. I play the game expecting it to behave as real-life. Which I have to remind myself its a computer game, not an alternate reality :-) Maybe McGwire hit that many homers in DMB because the Computer Manager didn't pitch around him. Maybe in real-life, McGwire would have hit 91 homers if he was pitched to and not pitched around when first-base was open. Hmmm Maybe the game needs to add some kind of "Approaching Milestone Pressure Factor" to the game. Then as guys are approaching 60 homers, or 300 wins, 500 homers, 44-game hit streak, etc. they could go into a cold streak. Its an idea. (Luke and the gang, thoughts?) Or maybe make the Computer Manager smart enough not to pitch to the guy with 60 homers. There's a thought. "Hey, its June 1st and Rod Carew is hitting .420. Maybe I should walk him?" But now we're talking AI ;-> (Anyone sim the season where Bonds hit 75 homers? I'd like to see what his totals come out to be on average.) But enough of beating a dead horse...Anyone pickup on my mention that David Wells went 26-1? (In the history of MLB, the closest guy with 20+ wins to touch that winning percentage is Ron Guidry in 1978 when he went 25-3. Which is a .893 winning percentage. Can you image how big Wells' head would have been if he really went 26-1 in a season?) timz Re: 1998 Replay - McGwire hits 91 HR??? #5 [-] Quote: -------------------------------------------------------------------------------- Maybe the game needs to add some kind of "Approaching Milestone Pressure Factor" to the game. Then as guys are approaching 60 homers, or 300 wins, 500 homers, 44-game hit streak, etc. they could go into a cold streak. Its an idea. (Luke and the gang, thoughts?) -------------------------------------------------------------------------------- Hugely bad idea, IMO, though Strat-O-Matic uses it. If you want a "baseball" sim instead of a "stat" sim you can simply use the projection disks instead of the season disks. By basing a player's performance on more than a single season of stats you set a baseline that doesn't lend itself quite so easily to outrageous performances. Player splits are much more realistic, too, with the larger sample size. aglenn99 Re: 1998 Replay - McGwire hits 91 HR??? #6 [-] Yeah, upon further reflection that "Add an approaching milestone factor" thing is a bad idea. Not everyone has a harder time as they approach a milestone. So how would you know who would get a negative factor and who would get a positive factor? So scratch that. But I still stick with my other point that the Computer AI should be smart enough to realize that someone has 60 homers in August and thus that guy should be pitched around. This is not the only replay to result in real "head scratchers." Sift through the DMB Replay message board for more examples. The point is: DMB's reputation for historical replay accuracy may be a bit overblown. It's very good, but not without flaws. As for OOTP, it has improved greatly in the "historical replay" department (though it has its flaws, too). OOTP's reputation for historical replay accuracy may, in fact, be underrated. If so, it's about time we corrected the myths. What do y'all think?

06-24-2008, 02:57 PM	#2
swampdragon Hall Of Famer Join Date: May 2002 Location: The Lonely Mountain Posts: 2,509	Before this version, I would say that DMB was well ahead of OOTP in terms of historical accuracy. You're certainly right that some of the DMB season disks can yield some weird results, and I think its accuracy level does tend to be a little overstated. This year's OOTP has a better feel to the game engine, and people are reporting very accurate results. At this point, the only thing I'm sure DMB does better is pbp. __________________ “Of all tyrannies, a tyranny sincerely exercised for the good of its victims may be the most oppressive. It would be better to live under robber barons than under omnipotent moral busybodies." -- C.S. Lewis

06-25-2008, 10:45 AM	#3
skunkle333 All Star Reserve Join Date: May 2006 Location: St. Louis metro area...IL side. Posts: 643	I haven't replayed 1959 or 2005 yet so I can't speak fully about OOTP9's accuracy but while playing 2008 I'm not seeing any radical or strange results albeit it obviously isn't a real season replay as such. For some time now my actual play with DMB was replaying fav World Series and creating round robin tournaments with fav WSer teams etc. So full season accuracy isn't a real biggie with me like it used to be. I'm just looking for a solid degree of realism for my short replays. Just hope I can find a way to do WSeries replays in OOTP9 without playing through a whole season. Maybe some kind of exhibition setup. So to speak to the issue of this thread on a small sample size scale as such I can just say this: In all the single games I've played with OOTP9 (maybe 25 or so pitch by pitch) I haven't seen anything that was out of kelter for a single game. Which I guess isn't saying much...or is it? *Had a great game recently with the WSox and Cubs...Zambrano against Buehrle and it was very realistic.* *I'll report back later on my 59 and 2k5 full season replays.* *OOTP9...pitch by pitch...with pics and sounds...fantastic...*

06-25-2008, 03:27 PM	#4
bbastyr Minors (Rookie Ball) Join Date: Jun 2008 Posts: 22	s Last edited by bbastyr; 09-05-2020 at 12:15 AM.

06-25-2008, 04:42 PM	#6
pstrickert Hall Of Famer Join Date: Dec 2005 Posts: 15,741	The DMB PbP is definitely NOT overrated! It's excellent! The OOTP PbP has improved, IMO, by leaps and bounds -- but more can (and hopefully will) be done to improve it!

06-26-2008, 01:04 AM	#7
bbastyr Minors (Rookie Ball) Join Date: Jun 2008 Posts: 22	OOTP's pbp is quite good. It's passed BBW as second favorite in-game pbp. With DMB, you still get a better feel for a ball game. OOTP is definitely closing the gap, though. More variety, a little more color, and park specific PBP is all that's needed.