Study: Market Size Bias

[CBeisbol]

A recent thread contended that the game was biased towards teams with large market sizes.

As evidence for this the user cited a game they lost on 3 consecutive passed balls with the bases loaded with 2 outs in the ninth inning and some other anecdotal evidence.

It seems very obvious that certain big city teams have inherent advantages and do crazy stuff to win games even with ****ty players. I get that this happens in baseball, but not like it does here, not even close. The last game that broke my back was 3 passed balls in a row with bases loaded and 2 outs in the 9th to lose by one

Not a compelling case

But, i decided to take a look



For this study I:

Created a new Real World League.

Noted the 25 man rosters for the Yankees and the Rays.

Confirmed that the Yankees had a larger market size and budget than the Rays (Huge, Extreme, 94, $262,000,000 to Small, Pathetic, 55, $100,000,000).

Turned off injuries, suspensions and trades.

Simmed 50 seasons. Recorded the wins for the Yankees and Rays

In commissioner mode, traded all 25 players on the active roster for the Yankees with the 25 players on Tampa's active roster.

Simmed 50 seasons.

RESULTS
In the original sim (Yankees players on the Yankees, Rays players on the Rays) both teams won 90 games.

In the altered sim (Rays players on the Yankees, Yankees players on the Rays) the Yankees (Rays players) won 90 games, and the Rays (Yankees players) won 89 games

There was a slight (1 game) difference in the test. This does not confirm the original claim of bias.

First, a 1 game difference could be explained by random variation. Fifty games may not have been a large enough sample

Second, the results are only in whole numbers. There are no fractions of games. So 89 wins could have been 88.5 wins to 89.4 wins. As well, 90 wins could have been 89.5 wins to 90.4 wins.

[ALB123]

Originally Posted by CBeisbol

Simmed 50 seasons.

RESULTS
In the original sim (Yankees players on the Yankees, Rays players on the Rays) both teams won 90 games.

In the altered sim (Rays players on the Yankees, Yankees players on the Rays) the Yankees (Rays players) won 90 games, and the Rays (Yankees players) won 89 games

There was a slight (1 game) difference in the test. This does not confirm the original claim of bias.

First, a 1 game difference could be explained by random variation. Fifty games may not have been a large enough sample

Second, the results are only in whole numbers. There are no fractions of games. So 89 wins could have been 88.5 wins to 89.4 wins. As well, 90 wins could have been 89.5 wins to 90.4 wins.

Very interesting. I appreciate you doing this for the OOTP community. A personal regret is that I never took the Statistics 101 course in college. So, my opinions are based solely on my gut, but I know many OOTP Forum Users do understand the subject of statistics. So...

I think 50 seasons is a decent total where we can start saying there is truth in these numbers. When I read the first set of results, where you mentioned both teams won 90 games, were you saying that each team averaged 90 wins over the course of 50 seasons? Or, are you stating that for 50 simulated seasons each time won 90 games every single time? If you meant the average number of wins over 50 seasons was 90 & 90, did you happen to hold onto the data so we could see what the highest number of wins, in one season, was for each team as well as the lowest number of wins, in one season, for each team?

I hate asking someone for more work to be done, however, I think you've proved NYY & Rays are pretty even. So, maybe you should pick a last-place skilled team to go against the Yankees. That way, when you do switch the entire team rosters you'll more clearly see if Market Size is affecting the outcome. Does that make sense?

We know a crummy team isn't going to finish with 90 wins, but if that crummy team roster now plays as the NYYs, will there be a noticeable, undeniable difference in their performance over 50 seasons? Perhaps I am showing my naiveté in this post??

[CBeisbol]

Using the simulation, you only get averages. That's why I said 90 could be 89.5 to 90.4. it's only reported as a single number

The Rays and Yankees didn't play each other, they played a full season including interleague

[too_on_too_out]

50 seasons is just another way of saying a sample size of 50. Seems low.

[BBGiovanni]

Originally Posted by CBeisbol

A recent thread contended that the game was biased towards teams with large market sizes.

I don't want to start whatever discussion led to this, but... so is MLB and has been forever? There's a reason only 20% of ticket revenue goes to the road team. Anyway, thanks for posting this.



FWIW in the season I just played through, I don't think I made any changes to either team, and the Rays won the East by a handful of games over the Yankees and my Mets beat the Rays in game 7 of the World Series after the Rays beat the Yanks in 6 games in the ALCS. Just for the sake of another data point.

[CBeisbol]

Originally Posted by BBGiovanni

I don't want to start whatever discussion led to this, but... so is MLB and has been forever? There's a reason only 20% of ticket revenue goes to the road team. Anyway, thanks for posting this.



FWIW in the season I just played through, I don't think I made any changes to either team, and the Rays won the East by a handful of games over the Yankees and my Mets beat the Rays in game 7 of the World Series after the Rays beat the Yanks in 6 games in the ALCS. Just for the sake of another data point.

There's that bias, sure

But that's not what I was responding to

What I was responding to was a claim that the OOTP sim engine favored big budget teams

https://forums.ootpdevelopments.com/...d.php?t=316874

[ALB123]

Originally Posted by CBeisbol

Using the simulation, you only get averages. That's why I said 90 could be 89.5 to 90.4. it's only reported as a single number

The Rays and Yankees didn't play each other, they played a full season including interleague

OMG. I had no idea... What's one more post where I make myself look stupid? LOL Thank you for educating me once again, CBeisbol.

[ALB123]

Oh...I forgot to ask a question I have...

What exactly determines Market Size? I am so shocked to learn that my New York Yankees only have a "Very Big" market size. The California Angels & Houston Astros both have "Huge" market sizes. While Philadelphia takes the top prize with an "Astronomical" market size.

Does it have to do with the size of the land that constitutes the majority of their fanbase? Like, is the greater Philadelphia region bigger than NY's 5 boroughs, so it gets a higher market size rating?

[darkcloud4579]

You all are overthinking market size in this game. It's been the same since the old days without a shred of an update. The only thing it seems to impact is the cashflow, but there are lots of ways for you to fix that so you don't have to let the game throw money around unchecked.

I built a tool some years ago to help you calculate market size based on city, but it's out of date and I haven't had a chance to fix it. Still, it's only about money and not performance or anything else. So if you neutralize market sizes [which I like to do by making tv contracts national] then you don't have to deal with it. Alternatively, you're going to find yourself making a lot of manual changes to get the market sizes accurate with IRL cities, because it's never been improved.

[quillenl]

Market Size changes are fairly random. There are a few storylines that are supposed to affect it, but I think only the Charity one which decreases Market Size currently works. The effect of Market Size on your team is partially on Ticket Sales and a bit more heavily on your Media Revenue as well as Merchandising (don't recall how heavily it affected Merchandising).

I've been told that assignment of Market Size, but my personal observational bias has always had me feeling otherwise. I've always played the Yankees or a local small town team I created as an expansion team. More recently I have run through various careers as a GM of the Vancouver Canadians (maybe 5 or so. I've started as Low as Tiny with the Yankees once or twice, and a handful of Below Average to Average... but 75% the time the Market Size has been blue. I am in 1914 right now playing the Greenville Fighting Amish as an Average Market Size and checked the Yankees for giggles and they are ranked as HUGE. Long Story short, my biased observation has always indicated that Market's have some initial range they may generate at similar to players so you are a lot more likely to fall into a large Market with a team like the Yankees than you will be starting up with a team like Vancouver or Greenville, SC.

Now, here is the ironic part if you read this far. I've asked about Market Size and how it changes to the Devs... (others have as well) and I have never seen them address it. Truth is no one really knows but many have a strong opinion. I use "bias" in my opinion a lot because frankly it wouldn't make sense to bias the weigth the markets in this game on cities. I can tell you I seldom see the Mets as an Astronomical Franchise so my bias would lean towards the Market not actually being based on the assigned City in the settings. That makes no sense because it would indicate it pulls from the Team name or an underlying value (like Team ID) we don't pay much attention to normally. I 100% see it biased for certain franchises... but I also am 100% certain that the only sensible way for them to have programmed it (in my opinion) would have been random if it is not based on cities.

[Syd Thrift]

If there is any correlation I would be extremely surprised if they baked in a “Yankees fix” into the game. That would quite frankly be orders of magnitude harder than even having a dev bump related to market size, which itself is harder to implement than not having any of that. When in doubt, assume programmers took the easiest way out. It’s a rule of thumb that usually works.

That being said there *is* one factor that could push things and that’s the development budget. I remember it was added a few years ago to to give teams another place to spend their money. IIRC OOTPD have insisted it only constitutes a small difference but it’s *a* difference, and generally speaking if you have more money in general you have more money to drop into your development budget.

[Syd Thrift]

Originally Posted by darkcloud4579

You all are overthinking market size in this game. It's been the same since the old days without a shred of an update. The only thing it seems to impact is the cashflow, but there are lots of ways for you to fix that so you don't have to let the game throw money around unchecked.

I built a tool some years ago to help you calculate market size based on city, but it's out of date and I haven't had a chance to fix it. Still, it's only about money and not performance or anything else. So if you neutralize market sizes [which I like to do by making tv contracts national] then you don't have to deal with it. Alternatively, you're going to find yourself making a lot of manual changes to get the market sizes accurate with IRL cities, because it's never been improved.

IIRC someone complained that big market teams won too much in their leagues and I guess we’re correlation mining right now.

[CBeisbol]

Originally Posted by Syd Thrift

IIRC someone complained that big market teams won too much in their leagues

You don't. Well, I'm sure someone has said that but I was responding to a poster who claimed that the *sim engine* was biased for big market teams.

Originally Posted by CBeisbol

A recent thread contended that the game was biased towards teams with large market sizes.

As evidence for this the user cited a game they lost on 3 consecutive passed balls with the bases loaded with 2 outs in the ninth inning and some other anecdotal evidence.

Not a compelling case

But, i decided to take a look

[ALB123]

Originally Posted by darkcloud4579

You all are overthinking market size in this game. It's been the same since the old days without a shred of an update.

Well, I only have 6 months experience with OOTP - this is the only version I know. So okay, at least now I don't really need to care about it. It just seems kind of silly to have a "feature" in OOTP that really doesn't do much at all. Especially if it's a feature that's been around for a decade-plus and has hardly changed at all. Thanks for letting me know.

[quillenl]

Originally Posted by ALB123

Well, I only have 6 months experience with OOTP - this is the only version I know. So okay, at least now I don't really need to care about it. It just seems kind of silly to have a "feature" in OOTP that really doesn't do much at all. Especially if it's a feature that's been around for a decade-plus and has hardly changed at all. Thanks for letting me know.

It is a feature that DOES do much. Left to its own devices you will have as big a gap between teams in your league as that seen in real life MLB. Fan Interest and Fan Loyalty also affect finances but those two can and will change with long term winning. Market Size will change on it's own and can result in a huge difference in payroll.

You should definitely care about it as in solo leagues it will be an important way of giving yourself a handicap against the AI and /or understanding why your team can only reach 50% of the payroll of one of the AI teams. In Online leagues it can set you at a huge disadvantage if not addressed in other rules which may or may not be something you desire in a competitive environment.

[ThePretender]

I don't know if this proves your point. If someone says the game is biased to big market teams, what they're referring to is that those teams have the payroll to keep their stars and spend more and all the benefits that come with that. So these teams are likely to win more because they can spend more.

Doesn't mean they have an advantage with identical rosters, IMO.

[CBeisbol]

Originally Posted by ThePretender

I don't know if this proves your point. If someone says the game is biased to big market teams, what they're referring to is that those teams have the payroll to keep their stars and spend more and all the benefits that come with that. So these teams are likely to win more because they can spend more.

Doesn't mean they have an advantage with identical rosters, IMO.

Nope
That's not what the person to whom I was responding meant

Originally Posted by CBeisbol

A recent thread contended that the game was biased towards teams with large market sizes.

As evidence for this the user cited a game they lost on 3 consecutive passed balls with the bases loaded with 2 outs in the ninth inning and some other anecdotal evidence.

Not a compelling case

But, i decided to take a look

[Déjà Bru]

I would like to see this promising thread continued. I am once again faced with the dilemma in my single-player league as to what to do with Market Sizes. I do not want franchises rewarded or punished by this randomly assigned variable that changes on its own over time. My solution in the past has been to edit every team to "Average" Market Size (and needing to visit the topic annually to correct drifting) but, would the consensus be that Market Size is immaterial to results even though someone said it affects revenue?

[Rain King]

It absolutely affects nearly every part of revenue. It does now, however, affect any specific results on the field outside of the quality of players the teams are able to acquire due to their amounts of revenue. That was quite a silly premise.

[Déjà Bru]

Originally Posted by Rain King

It absolutely affects nearly every part of revenue. It does now, however, affect any specific results on the field outside of the quality of players the teams are able to acquire due to their amounts of revenue. That was quite a silly premise.

Good to know, thanks. Even that is more than I want; all results should be based on decisions, strategy, and luck. Not on a randomly assigned background variable like market size. I know in the MLB game, they try to make it more realistic by using this parameter, but playing fictionally, especially in the dead ball era, I just don't want it. Sticking to across the board "average" market sizes, then.