Definitely Something Wrong with the Simulation

[Otrex]

Thank you for running that - it's a bit of a head-scratcher for me then. Can you confirm my initial results, even if we have to chalk it up to a highly-improbable circumstance? It would, at least, demonstrate that the results I claimed were what I indeed experienced.

Also, do you see any settings I am using which are of concern to you? Earlier it was expressed to me not to change the "adjust" and "weaken" settings, even if to make them tougher standards. It seems strange, but I am willing to try it if it creates a more authentic experience.

[joefromchicago]

Originally Posted by Otrex

My background is in economics and I've done a bit of work in data regression models

With that kind of background, I'm surprised you didn't hit upon the obvious method of testing your hypothesis. With OOTP, you can play any team against any other, which means you can play the same team against itself. The solution, then, is to set up a league where every member is the same team. It doesn't matter which one. Let's say you pick the 2010 Oakland A's, a team that finished with a record of 81-81. Fill your league with identical copies of the 2010 A's. Turn off injuries and trades, put every team in the same stadium, set the same team strategy for each -- in short, make each team absolutely identical in every respect. Now take one of those versions of the A's and see what happens. That strikes me as the only way to truly test whether the AI favors the human player.

[Otrex]

My hypothesis was that it likely had something to do with a combination of factors which are not necessarily relevant to the play of a historical season with historical players on the correct teams as you suggest I try (perhaps settings, or perhaps something else). I don't know of any way to create a twin season with a historical league with random debut players.

Further, your example would still not be a laboratory-correct experiment because of x-variables such as injuries, weather, and such which I don't believe are set before the season starts. Because of this, I know of no way to simulate the exact same season twice without something drastic.

[joefromchicago]

Originally Posted by Otrex

My hypothesis was that it likely had something to do with a combination of factors which are not necessarily relevant to the play of a historical season

No, your hypothesis was that the AI favors human players. In particular, you said: "There is DEFINITELY a human-controlled player bias when it comes to winning games - and a big one."

Originally Posted by Otrex

I don't know of any way to create a twin season with a historical league with random debut players.

Why would you want to?

Originally Posted by Otrex

Further, your example would still not be a laboratory-correct experiment because of x-variables such as injuries, weather, and such which I don't believe are set before the season starts

As I said above, you should turn injuries off. As for weather, it doesn't affect play, except for rain delays, and those will even out over the course of the season.

Originally Posted by Otrex

Because of this, I know of no way to simulate the exact same season twice without something drastic.

Why would you want to simulate the exact same season? You're testing one variable - human vs. non-human. Under my set-up, you control for all the other variables. That's what you want, isn't it?

[Otrex]

Yes, and I stand behind my original hypothesis. In the games I've played of OotP16, there is definitely a bias towards the human player as my results demonstrate.

If I have to add "In the games I've played on my computer it's been my experience that..." to everything I've said so that you don't mistake it for a global statement, we'll be here all day typing. I assumed that people would normally assume that to be the case, and understand that I cannot control for what your settings may be. I spelled out precisely how many seasons I've played AND my results. If you want to stare at one sentence out of several and draw a conclusion about what I was trying to state, that's your problem.

Are my results in the 95% confidence window? I'm not sure. But again, I qualified my statement with how I came to that conclusion. Some people chose to inquire and see if they could figure out why this was happening, and some chose to mock. Again, that's a concern for those folks.

I'll yield the floor to you hereafter on this point.

[joefromchicago]

Originally Posted by Otrex

Yes, and I stand behind my original hypothesis. In the games I've played of OotP16, there is definitely a bias towards the human player as my results demonstrate.

No, your results don't demonstrate that. Even you are walking that claim back.

Originally Posted by Otrex

If I have to add "In the games I've played on my computer it's been my experience that..." to everything I've said so that you don't mistake it for a global statement, we'll be here all day typing.

I didn't mistake that for a global statement, it was a global statement. You said something was wrong with the game, not with your computer.

Originally Posted by Otrex

Are my results in the 95% confidence window? I'm not sure.

I gave you a method for determining whether your claims were valid or not. You can either test your claims using that method or you can continue to backtrack. Your choice.

[Lou Gehrig]

Originally Posted by Otrex

Yes, and I stand behind my original hypothesis. In the games I've played of OotP16, there is definitely a bias towards the human player as my results demonstrate.

Originally Posted by joefromchicago

No, your results don't demonstrate that. Even you are walking that claim back.

Looks like this is going into extra innings!

[byzeil]

I hope my analysis makes sense, I put it together in a hurry (during commercials breaks watching the Walking Dead).

The first screenshot is the OP's history as GM-MGR.

The 2nd one is AJAX leading to the OP taking over. They were very bad winning 54/56/70 games then they win 83 and 90 under the OP. The 3 seasons before the OP took over they were improving (+14 games the year before the OP took over). The 13 and 7 game improvements under the OP were not out of line of normal improvement of a team. The 7 game improvement translated to moving from 17GB to winning it all but that means the teams ahead of them really fell apart as much or more than AJAX improved.

So it looks to me the team was improving before they were taken over and it just continued. So nothing unusual here.

The 3rd screenshot shows GYPSY before and through the OP taking over in '99. This one is tougher to explain. First year under OP no improvement then a huge jump. That huge jump is dramatic but doesn't prove a human bias because the 1st year the OP ran the team they won 69 games, it was the second year that saw the improvement.

I had a team of mine in an online league that made a jump almost identical to that and it was because my prospects made the jump and I made a couple good FA moves.

So, I only see one year that is very odd looking and that isn't enough to prove a human bias to me especially when you see AJAX's normal improvement and Ottawa's progression.

I hate to use the 'small sample size' thing but to me that is what the OP was looking at. Only one season makes me raise an eyebrow but it is only 1 season.

I fully believe that if there was the obvious human bias the OP seems to think there are many people in these forums would be raising all sorts of...heck. The veterans here are not shy about challenging the developers. I've played since 2008 and see no human bias and the league file the OP posted doesn't change at all my opinions on that.

[gads]

Originally Posted by byzeil

I hope my analysis makes sense, I put it together in a hurry (during commercials breaks watching the Walking Dead).

The first screenshot is the OP's history as GM-MGR.

The 2nd one is AJAX leading to the OP taking over. They were very bad winning 54/56/70 games then they win 83 and 90 under the OP. The 3 seasons before the OP took over they were improving (+14 games the year before the OP took over). The 13 and 7 game improvements under the OP were not out of line of normal improvement of a team. The 7 game improvement translated to moving from 17GB to winning it all but that means the teams ahead of them really fell apart as much or more than AJAX improved.

So it looks to me the team was improving before they were taken over and it just continued. So nothing unusual here.

The 3rd screenshot shows GYPSY before and through the OP taking over in '99. This one is tougher to explain. First year under OP no improvement then a huge jump. That huge jump is dramatic but doesn't prove a human bias because the 1st year the OP ran the team they won 69 games, it was the second year that saw the improvement.

I had a team of mine in an online league that made a jump almost identical to that and it was because my prospects made the jump and I made a couple good FA moves.

So, I only see one year that is very odd looking and that isn't enough to prove a human bias to me especially when you see AJAX's normal improvement and Ottawa's progression.

I hate to use the 'small sample size' thing but to me that is what the OP was looking at. Only one season makes me raise an eyebrow but it is only 1 season.

Thanks for proving that he was indeed one.

[greenOak]

This comic is grossly unfair to frequentist statisticians, but it sure seems to apply to OP.

https://xkcd.com/1132/

[NCBeachBum]

Looking at what little we can see there, I notice that the year of the big jump, OP's pitcher BABIP dropped by 24 points, and his ERA dropped 1.27 runs. His BA, for what very little this is worth, was almost identical. Can one of you check the players used in those seasons and see what led to the significantly better defense? That combined with the pretty significant payroll drop causes me to wonder if some bad, old players with large contracts left, and this led to youngsters with better gloves getting more playing time.

[anson283]

Yeah something is wrong with the simulation....the phillies won the world series in 2017

[dcordash]

I haven't sounded of on this board in years, but I don't care for judgemental bullies. Otrex genuinely feels there is an issue with his experience with ootp that doesn't make sense. I've seldom witnessed anyone on these boards articulate an unexpected behavior as well as he did without ranting on and taking shots at the code or software logic. He humbly admitted he likely had some settings jacked up. He patiently continued to address every single suggestion that wasn't blatant hateful sarcasm with respect and consideration.
The only mistake I can see he made was mentioning his economics degree and some understanding of statistics and probability. This really seemed to threaten a few on here that are likey less schooled. Maybe they took his declaration of having no less than a couple brain cells to rub together as pretentiousness. I don't know but, Christ, give him a <snip> break!

[majesty95]

I think the issue was that he made the bold claim that there was "definitely" a problem and didn't seem open to random variance as an explanation. He felt something was amiss and decided it wasn't the game but maybe his setting. Still clinging to the thought that there was an issue.

After looking at the seasons posted by byzeil, I don't see anything that strange. Yeah, we won a lot at Montreal and the turnaround at Ajaz was quick. However, as was mentioned a few times, it could very well be that he took over a couple of teams at just the right time. Probably not statistically probable but not definitively "off" either.

Had he played out another 10-20 seasons playing as a different team every 3-4 seasons he probably would have noted his run end and realize it was just random luck. But he seemed set in his thinking there was an error caused in some way. Its understandable. I want to blame game very often myself as I know they are imperfect and its easier to explain that than a rare feat happening (for or against you). I think that is probably what rubbed some the wrong way.

I think we all can try to be more tolerant and less judgmental in our dealing with others. However, I've also seen far worse responses by long-time members of various communities. At the end of the day, its all just a bunch of faceless dudes sharing opinions on an anonymous message board. No harm or foul either way.

[ToTheBackstop]

Originally Posted by joefromchicago

As for weather, it doesn't affect play, except for rain delays, and those will even out over the course of the season.

To exclude weather as a factor, one could just switch all teams to play inside a domed environment -- no precipitation, no wind, weather doesn't come into play at all.

Originally Posted by joefromchicago

Why would you want to simulate the exact same season? You're testing one variable - human vs. non-human. Under my set-up, you control for all the other variables. That's what you want, isn't it?

For sake of argument, should someone want to simulate the exact same season, wouldn't it be as straightforward as:

a) at beginning of season X, make a backup of your league
b) sim out season X, get your outcome
c) restore your backup; now you're back at beginning of season X again
d) sim out season X again, get outcome #2
e) rinse, repeat, get enough outcome samples in order to make a meaningful declaration

?

[RchW]

Originally Posted by ToTheBackstop

For sake of argument, should someone want to simulate the exact same season, wouldn't it be as straightforward as:

a) at beginning of season X, make a backup of your league
b) sim out season X, get your outcome
c) restore your backup; now you're back at beginning of season X again
d) sim out season X again, get outcome #2
e) rinse, repeat, get enough outcome samples in order to make a meaningful declaration

?

I've been doing that for years to set up proper LT and LTM. Disabling development turning off injuries suspensions weather makes it a great way to get an uncluttered set of results.

Essentially this is what autocalc does.

[gads]

Originally Posted by ToTheBackstop

To exclude weather as a factor, one could just switch all teams to play inside a domed environment -- no precipitation, no wind, weather doesn't come into play at all.



For sake of argument, should someone want to simulate the exact same season, wouldn't it be as straightforward as:

a) at beginning of season X, make a backup of your league
b) sim out season X, get your outcome
c) restore your backup; now you're back at beginning of season X again
d) sim out season X again, get outcome #2
e) rinse, repeat, get enough outcome samples in order to make a meaningful declaration

?

This is what I do except that I use Windows task manager to exit as reloading the game is much faster than restoring a backup for me. I have a backup in case something goes wrong but I don't use it to resimulate a season. They should provide an option to disable automatic saving when exiting the game to make things easier for us.

[OldFatGuy]

Originally Posted by Otrex

I've recently purchased OotP16, and I set up a fictional 16 team league that plays according to 1993 stats with modern 2014 financials. It also uses the random debut feature and allows any player from 1901 to 1981.

There is DEFINITELY a human-controlled player bias when it comes to winning games - and a big one. Here's how I know:

I've played 5 seasons so far, but I have yet to actually input any manual changes. Every single season I've played, I've let the A.I. do everything. I simply run the whole season and watch.

In my first 3 seasons, my team rattled off a high of 118 wins. My team won two world series during that time.

Okay, maybe I got lucky and received a superstar team by chance? I tested that theory too.

In season 4, I switched managing positions to the WORST team in the league, who had just completed a season with 120 losses. That very next season, with 90% the same lineup, they rattled off 85 wins and made the playoffs. The next season, they won the World Series. My previous team was down in the middle of the pack within 2 seasons, no longer benefiting from my golden "non-touch".

Keep in mind I have yet to input a single instruction to any of the teams I've managed.

Something is definitely wrong here. Anyone have any input on this? It's not terribly fun to get a constant assist from the AI - imagine when I start tweaking things to improve them further!


__________________________________________________ _______

On a side note, some of the stats generated are a bit strange too. My final season had a certain "Ray Jablonski" lead the league with 52 homeruns, despite hitting 21 as his career best (in the 50s, I believe).

If you want to test your theory out, here's a suggestion. Make a league, a 30 team league would be good, and save it as a quickstart, single season league.


Then create a new league from that quick start, select a team to own (and do nothing with) and play out the season. Then, create a new league from the quickstart again, and pick a different team.


Repeat this 30 times (once for all 30 teams) and then total up your wins and losses from all 30 times and if it's significantly over .500, then you may have discovered something. I suspect it won't be though, as I've played OOTP for literally thousands and thousands of hours and I've never experienced it, but I admit my experience with 16 is minimal as I usually just play around with a new version until the last patch comes out and then I get serious with it.

[Markus Heinsohn]

No, there is nothing wrong with the simulation. Of course not

So, in order to shut down future weird conspiracy threads:
Sample size determination - Wikipedia, the free encyclopedia

Cheers!