AI Evaluation Settings

[BarneyRubble]

Originally Posted by Dr Naysay

Just to be clear you mean the AI player evaluation settings back to default?

Trade settings, the player AI evaluation settings you are using look good to me.

[PSUColonel]

This is interesting and not something I had previously considered. (Leaving trade difficulty at default and favoring prospects instead). To what degree I’d want prospects favored is a question I have. Another is AI evaluation. With settings such as these, I think ratings would almost HAVE to be the overriding factor. Either my 55/20/15/10 or even the default 65/20/10/5.

The trading AI going off of stats more than ratings might create a big problem. I would need to o a little testing to see, but my gut tells me this type of approach requires a ratings heavy AI evaluation.

[KBLover]

Originally Posted by PSUColonel

This is interesting and not something I had previously considered. (Leaving trade difficulty at default and favoring prospects instead). To what degree I’d want prospects favored is a question I have. Another is AI evaluation. With settings such as these, I think ratings would almost HAVE to be the overriding factor. Either my 55/20/15/10 or even the default 65/20/10/5.

The trading AI going off of stats more than ratings might create a big problem. I would need to o a little testing to see, but my gut tells me this type of approach requires a ratings heavy AI evaluation.

Does this impact AI trades with each other? I know trade difficulty doesn't but the Favor Prospects slider?

I turned it up and I'm seeing more 20-30 POT prospects for 45-55 established players that aren't older or on high contracts. Well, some are but I don't mind the salary dump deals. It's the trading a 55 rated, 28 year old with a 5 million contract for a 25 POT prospect.

This is with ratings now 55% of the evaluation.

So I'm wondering if this is causing the AI teams to value that 25 POT prospect too much because they all "favor" prospects. So prospects become more valuable and non-prospects (the "players" and "veteran" classes) less so.

Sure their scouts might be off, but on High scouting accuracy, I highly doubt any scout will rate a 55 major league level player as similar to a 25 POT prospect.

[PSUColonel]

Yea, I have always gone neutral on the "favor prospect vs veterans" setting, and gone with the most difficult trading setting. I know people might not think it's always realistic, but the user is going up against AI, and needs to be as difficult as possible IMO. Players will be able to "game" the AI if you don't do this. I know it's not perfect, but I think it's the best we can do in terms of what we (the users) can control.

Same with the AI evaluation weight. The AI simply needs all the help it can get, and a ratings heavy evaluation definitely helps in terms of challenge.

I know this forum is full of "stat heads", and that's fine...but the bottom line is, if you want the greatest challenge possible, you need the toughest trade difficulty, a ratings heavy AI evaluation, and likely more TCR. I'm not sure you need 200 TCR, but probably more than 100. Maybe 150? which is 75% of the maximum allowable randomness.

[MizzouRah]

I'm with you PSU.. for my real game that I just started about a week ago with the Mariners.. I put trade difficulty to hardest, left the other one neutral and I using default AI eval. for MLB live start. (I also like scouting reports incorporating stats)

I've tested other settings and I've come to the conclusion for how I play and what I'm wanting in terms of a challenge, those seem to work best for me.

I do love the yearly discussion though!!

[KBLover]

Originally Posted by PSUColonel

know people might not think it's always realistic, but the user is going up against AI, and needs to be as difficult as possible IMO. Players will be able to "game" the AI if you don't do this. I know it's not perfect, but I think it's the best we can do in terms of what we (the users) can control.

I'm more concerned with seeing how/if I can influence AI trades beyond the evaluation settings.

I don't worry about what I do. I'm in control of the deals I will or won't make, including not gaming the system even if I can.

But can I stop the AI teams from gaming each other?

[PSUColonel]

trading difficulty only really affects the human vs AI trades

[KBLover]

Originally Posted by PSUColonel

trading difficulty only really affects the human vs AI trades

A shame. I knew the actual difficulty did but was hoping the Favoring slider would work on the AI.

Since it doesn't, that's my new great wish in terms of new features for OOTP - AI, sliders/settings to impact AI trade valuations when dealing with non-human controlled teams.

Most of the teams most non-online league universes are likely AI controlled after all.

[oomm]

Since I initiated a thread on this last year am going with 25s across the board. However, am currently just managing. Also have finances turned off and trades set to virtually never. Not a good sample size yet.

[PSUColonel]

Originally Posted by KBLover

The only injury he suffered in 2030 was a mild calf strain, day-to-day, that lasted a week. He's suffered just 5 injuries in his career, only one, a fractured thumb in 2028, made him miss time (4 weeks).

Scouting reports on him attached. Why OSA is saying he's a 65 makes no sense.

I still say there's a bug/glitch with this guy. But I guess this is "working as designed" but it makes no sense.

So far, he's not been picked up out of free agency. Yet. I'm sure it will happen, just to irk me




Yes, I think that's what happens. To me, the last year stats "eat" the current year.

My 30/30/20/20 became 30/0/50/20, imo, which would explain why folks say this makes sense, though you will not get me to understand how this is a 65 player like OSA says.


Edit: Attached my development modifiers

Any thoughts on instances like these since the latest patch?

[Grim Madder]

Maybe I missed the answer to this in the thread.

I have just set up a stats-only game. I plan on using 25/25/25/25. I have put the trade difficulty to the highest and left the veteran/prospect level balanced. I have scouting on low.

Does anyone think there is a better way to set this up?

Thanks

[Sweed]

Originally Posted by Grim Madder

Maybe I missed the answer to this in the thread.

I have just set up a stats-only game. I plan on using 25/25/25/25. I have put the trade difficulty to the highest and left the veteran/prospect level balanced. I have scouting on low.

Does anyone think there is a better way to set this up?

Thanks

Other than the one's in this thread that have questioned the 4-25's or have stated they use other settings and why?

I think the question is "is this a reasonable way to set up?". And to that, I think, most would say "yes".

From my point of view I love the 4-25's and have stated so in this thread and others many times.

Stats only, I think is very unrealistic as one is working without information they would have in real life. Having said that I do understand why some like to play that way. That's cool, OOTP is a "play it your way" game

Your trade sliders are, I think, what a good number of folks choose. I prefer two clicks above neutral on the difficulty which IIRC is 8\10? I don't spend time trying to fleece the AI and I find this setting to work well... for me.

I too like "middle" for favor prospects\vets. I think most still prefer favor prospects. I would argue OOTP, under the hood, has finetuned this over the last few versions where favoring prospects is no longer a "must".

[PSUColonel]

Originally Posted by Sweed

Other than the one's in this thread that have questioned the 4-25's or have stated they use other settings and why?

I think the question is "is this a reasonable way to set up?". And to that, I think, most would say "yes".

From my point of view I love the 4-25's and have stated so in this thread and others many times.

Stats only, I think is very unrealistic as one is working without information they would have in real life. Having said that I do understand why some like to play that way. That's cool, OOTP is a "play it your way" game

Your trade sliders are, I think, what a good number of folks choose. I prefer two clicks above neutral on the difficulty which IIRC is 8\10? I don't spend time trying to fleece the AI and I find this setting to work well... for me.

I too like "middle" for favor prospects\vets. I think most still prefer favor prospects. I would argue OOTP, under the hood, has finetuned this over the last few versions where favoring prospects is no longer a "must".

I can respect this point of view, however, I am the type of player who wants a challenge, without having to constantly be "policing" myself. Therefore I am still going to stick with higher ratings. (over 50)

My question to those who also prefer this style of AI evaluation: Which of the following do you think yield the best results?:

1. 55/20/15/10

2. 55/25/15/5

3. 65/20/10/5 (default setting)


I realize it may not seem like a big difference...but the "two years ago" slot is a significant change I think between ⅔ & 1. Obviously there's also a significant change between the default (3) and the other two.

[Coldfront]

Assuming the goal is: Create a game that handles General Manager settings realistically.

From a math perspective, this is almost an unsolvable problem.

Each default input value generated in the settings, if user-modified, may seem like a small tweak, but I bet (and game programmers can call me out if I am off ) each change made in any of these settings can create a exponential impact with computer/AI behavior over time.

Each of these aspects have different distributions of possible values, meaning a single step change in Injury Frequency may not have the same weighted impact to the AI vs using one step change in the usage of Stats and AI-League totals.

Ultimately the game has to do the math with weights on the coefficients for all of these inputs. Any one change in the 22 settings will create a constantly expanding change in all of the other settings over time. Moreover, any one change which has a high weighting on statistics will create a HUGE and ultimately random change in the league universe as time goes on, and there's really no way to know the "true" effect of any one change within any one simulation (career).

These coefficient weights are almost certainly not linear, but are exponential with a larger magnitude of impact over time (small changes will show in a current season, for example, if a change is made to the Injury Frequency settings, but each year will be essentially effect^2 (effect size * effect size) from year to year.

The default settings are the most "known" by the base algorithm and weighted by all of the stats and rules the OOTB team has tweaked over time. Any one "adjustment" to any one setting will generate unknowable changes in future AI vs AI trading behavior within the random universe, especially with random injuries and off field incidents included.

For example, just try simulating a set of 10 years seasons, making NO changes to the settings. The distribution of MVP awards, World Series wins by any one team, home-run champs or even simple win/loss percentages will vary by team over time, with that variance increasing for every season after 2022.

An that will happen even if the current stats, in-game rules, meteorology, ball-park physics and player movement/talent/eval algos PERFECTLY reflect the how baseball actually works.

Even with the same GM's, biases will be introduced, some will game the AI and some will be terrible, but any one GM can drift the wrong way at the wrong time and make a decision that impacts 1/30th of the universe at any one point in time.

Said another way, let's say there are 30 players with perfect 5 star talent, and we completely "know" the 5 star ratings are right, and each one plays for a major league team.

Let's also assume there's a 10% chance any one of them will experience a season ending injury before June 1st in the first season in the sim.

3 of the 30 would get injured early enough to significantly damage their team's chances of winning the division.

What if all 3 happened in the same division in a simulation?
What if all 3 happened to the Yankees 3 simulations in a row?
What if all 3 happened to the best 3 teams in the game?
Or the worst 3?

Then, how does each team's GM handle these issues in terms of timing, how aggressively to start stocking up or start a fire-sale, etc...then that gets compounded by how each GM would hande the same situation at the trade-deadline.

Even with Monte-Carlo after Monte-Carlo simulation which could be done 1000 times per minute, it's almost guaranteed the ultimate distribution of outcomes would be almost randomly distributed by the time you get to 2052...unless there are systematic programming biases that have a statistically significant impact given the same starting conditions in each career.

In other words, it's remarkable this game gets it this good already. That's why I always start with the best known weights (the defaults) and create biases that make me, as a single user, enjoy the game more. That's what it's ultimately all about, right?

#ButterFlyEffect
#NoFreeLunch

Mike

Edit: Been playing the game and have purchased close to every OOTP version since the original launch...been playing since the original text PC sim games of the late 80's...

[PSUColonel]

Originally Posted by Coldfront

Assuming the goal is: Create a game that handles General Manager settings realistically.

From a math perspective, this is almost an unsolvable problem.

Each default input value generated in the settings, if user-modified, may seem like a small tweak, but I bet (and game programmers can call me out if I am off ) each change made in any of these settings can create a exponential impact with computer/AI behavior over time.

Each of these aspects have different distributions of possible values, meaning a single step change in Injury Frequency may not have the same weighted impact to the AI vs using one step change in the usage of Stats and AI-League totals.

Ultimately the game has to do the math with weights on the coefficients for all of these inputs. Any one change in the 22 settings will create a constantly expanding change in all of the other settings over time. Moreover, any one change which has a high weighting on statistics will create a HUGE and ultimately random change in the league universe as time goes on, and there's really no way to know the "true" effect of any one change within any one simulation (career).

These coefficient weights are almost certainly not linear, but are exponential with a larger magnitude of impact over time (small changes will show in a current season, for example, if a change is made to the Injury Frequency settings, but each year will be essentially effect^2 (effect size * effect size) from year to year.

The default settings are the most "known" by the base algorithm and weighted by all of the stats and rules the OOTB team has tweaked over time. Any one "adjustment" to any one setting will generate unknowable changes in future AI vs AI trading behavior within the random universe, especially with random injuries and off field incidents included.

For example, just try simulating a set of 10 years seasons, making NO changes to the settings. The distribution of MVP awards, World Series wins by any one team, home-run champs or even simple win/loss percentages will vary by team over time, with that variance increasing for every season after 2022.

An that will happen even if the current stats, in-game rules, meteorology, ball-park physics and player movement/talent/eval algos PERFECTLY reflect the how baseball actually works.

Even with the same GM's, biases will be introduced, some will game the AI and some will be terrible, but any one GM can drift the wrong way at the wrong time and make a decision that impacts 1/30th of the universe at any one point in time.

Said another way, let's say there are 30 players with perfect 5 star talent, and we completely "know" the 5 star ratings are right, and each one plays for a major league team.

Let's also assume there's a 10% chance any one of them will experience a season ending injury before June 1st in the first season in the sim.

3 of the 30 would get injured early enough to significantly damage their team's chances of winning the division.

What if all 3 happened in the same division in a simulation?
What if all 3 happened to the Yankees 3 simulations in a row?
What if all 3 happened to the best 3 teams in the game?
Or the worst 3?

Then, how does each team's GM handle these issues in terms of timing, how aggressively to start stocking up or start a fire-sale, etc...then that gets compounded by how each GM would hande the same situation at the trade-deadline.

Even with Monte-Carlo after Monte-Carlo simulation which could be done 1000 times per minute, it's almost guaranteed the ultimate distribution of outcomes would be almost randomly distributed by the time you get to 2052...unless there are systematic programming biases that have a statistically significant impact given the same starting conditions in each career.

In other words, it's remarkable this game gets it this good already. That's why I always start with the best known weights (the defaults) and create biases that make me, as a single user, enjoy the game more. That's what it's ultimately all about, right?

#ButterFlyEffect
#NoFreeLunch

Mike

Edit: Been playing the game and have purchased close to every OOTP version since the original launch...been playing since the original text PC sim games of the late 80's...

I can't really disagree with you on this. I DO however think, stats need to be a bit more of the equation...as the defaults for fictional leagues are very different form the MLB setup (30/50/15/5). Which is why I have finally decided on using 55/25/15/5. It sort of follows both formulas, but definitely favors ratings by more than half, which is what I think it MUST do, in order to help the AI.

[BarneyRubble]

If you use a setting that favors 'ratings', it only helps the AI when their team has a good scout and spends money on scouting. Otherwise, It hurts the AI. Even more so if you have scouting accuracy set to low. AI GMs and Managers use the 'ratings' THEIR scout provide. If you have 'ratings' set too high, the scouts become the most important thing in the game. Bad scouts=bad GMs/Managers. If I was OOTPD, I would replace the word 'ratings' with 'scout input' in the 'AI Player Evaluation' setting. Where else would a manager or GM get the 'ratings' from?

It appears to me, both scouting Accuracy and Incorporate Stats affect AI player evaluation.

Why OOTP doesn't use 30/50/15/5 as defaults in every 'start' is beyond me.

If one gets sick of dealing with GMs that have inferior scouting, yet rely on it, they can always crank up the trade difficulty to 'try' and help them.

[PSUColonel]

Originally Posted by BarneyRubble

If you use a setting that favors 'ratings', it only helps the AI when their team has a good scout and spends money on scouting. Otherwise, It hurts the AI. Even more so if you have scouting accuracy set to low. AI GMs and Managers use the 'ratings' THEIR scout provide. If you have 'ratings' set too high, the scouts become the most important thing in the game. Bad scouts=bad GMs/Managers. If I was OOTPD, I would replace the word 'ratings' with 'scout input' in the 'AI Player Evaluation' setting. Where else would a manager or GM get the 'ratings' from?

It appears to me, both scouting Accuracy and Incorporate Stats affect AI player evaluation.

Why OOTP doesn't use 30/50/15/5 as defaults in every 'start' is beyond me.

If one gets sick of dealing with GMs that have inferior scouting, yet rely on it, they can always crank up the trade difficulty to 'try' and help them.

This is also true, which is why I think 65 is also a bit too high for ratings...a 55-45 split to me seems pretty reasonable.

[Pelican]

Originally Posted by Coldfront

Assuming the goal is: Create a game that handles General Manager settings realistically.

From a math perspective, this is almost an unsolvable problem.

Each default input value generated in the settings, if user-modified, may seem like a small tweak, but I bet (and game programmers can call me out if I am off ) each change made in any of these settings can create a exponential impact with computer/AI behavior over time.

Each of these aspects have different distributions of possible values, meaning a single step change in Injury Frequency may not have the same weighted impact to the AI vs using one step change in the usage of Stats and AI-League totals.

Ultimately the game has to do the math with weights on the coefficients for all of these inputs. Any one change in the 22 settings will create a constantly expanding change in all of the other settings over time. Moreover, any one change which has a high weighting on statistics will create a HUGE and ultimately random change in the league universe as time goes on, and there's really no way to know the "true" effect of any one change within any one simulation (career).

These coefficient weights are almost certainly not linear, but are exponential with a larger magnitude of impact over time (small changes will show in a current season, for example, if a change is made to the Injury Frequency settings, but each year will be essentially effect^2 (effect size * effect size) from year to year.

The default settings are the most "known" by the base algorithm and weighted by all of the stats and rules the OOTB team has tweaked over time. Any one "adjustment" to any one setting will generate unknowable changes in future AI vs AI trading behavior within the random universe, especially with random injuries and off field incidents included.

For example, just try simulating a set of 10 years seasons, making NO changes to the settings. The distribution of MVP awards, World Series wins by any one team, home-run champs or even simple win/loss percentages will vary by team over time, with that variance increasing for every season after 2022.

An that will happen even if the current stats, in-game rules, meteorology, ball-park physics and player movement/talent/eval algos PERFECTLY reflect the how baseball actually works.

Even with the same GM's, biases will be introduced, some will game the AI and some will be terrible, but any one GM can drift the wrong way at the wrong time and make a decision that impacts 1/30th of the universe at any one point in time.

Said another way, let's say there are 30 players with perfect 5 star talent, and we completely "know" the 5 star ratings are right, and each one plays for a major league team.

Let's also assume there's a 10% chance any one of them will experience a season ending injury before June 1st in the first season in the sim.

3 of the 30 would get injured early enough to significantly damage their team's chances of winning the division.

What if all 3 happened in the same division in a simulation?
What if all 3 happened to the Yankees 3 simulations in a row?
What if all 3 happened to the best 3 teams in the game?
Or the worst 3?

Then, how does each team's GM handle these issues in terms of timing, how aggressively to start stocking up or start a fire-sale, etc...then that gets compounded by how each GM would hande the same situation at the trade-deadline.

Even with Monte-Carlo after Monte-Carlo simulation which could be done 1000 times per minute, it's almost guaranteed the ultimate distribution of outcomes would be almost randomly distributed by the time you get to 2052...unless there are systematic programming biases that have a statistically significant impact given the same starting conditions in each career.

In other words, it's remarkable this game gets it this good already. That's why I always start with the best known weights (the defaults) and create biases that make me, as a single user, enjoy the game more. That's what it's ultimately all about, right?

#ButterFlyEffect
#NoFreeLunch

Mike

Edit: Been playing the game and have purchased close to every OOTP version since the original launch...been playing since the original text PC sim games of the late 80's...

I haven’t been around that long; but this has to rank among the finest debut posts (albeit by a long-time user) of all time! Thorough and fascinating! And somehow comprehensible to a struggling “C” student in math and statistics.

[LansdowneSt]

Originally Posted by Pelican

I haven’t been around that long; but this has to rank among the finest debut posts (albeit by a long-time user) of all time! Thorough and fascinating! And somehow comprehensible to a struggling “C” student in math and statistics.

It's one of the challenges the game has - giving us so many things to tweak but us not appreciating the compounding impact of those changes to the engine over time. An all-default run might begin to evolve as indicated in that post but reviewing those results for possible debugging is still an important step in QA/QC-ing the game.

It is also completely different than a post where the users says: "I only changed these several things/modifiers, simmed for 50 years and things look bonkers." Those several things might be like Manfred saying, "I don't know why stats starting to differ from historic norms, but we did change the core of the baseball, loosen the steams, enforce a ban on foreign substances for pitchers, change how we muddy the balls, adjust the humidor, move second base, ban select defensive shifts, move some fences in, etc. (you get the point). It doesn't mean that something isn't buggy in the code - but man it's just harder to tell when you change any default - let alone multiple ones. As I read more about various experiences players have, it seems, once you adjust you may need to constantly readjust, or one needs to accept the world as it evolves - just as we accept baseball isn't the same as it was for past generations irl.

But a thanks to everyone that comes on to give their recommendations, advice and shared experience as each instance provides a data point for users wishing to tailor their OOTP game experience.

[Pelican]

[Just lost a post when this site timed out, forced me to log in again, after five minutes…Will try to recreate.]

Tying to wrap my head around the many and varied ideas and approaches here - all of which are appreciated. This is a good thread.

I now understand better that over-reliance on ratings depends too much on scouting. I confess I was at 50% ratings because it was easy to compare across players, as in a list, without even bothering with stats. Yeah, I was lazy there.

So, I am beginning to see the logic in reducing that 50 to 35 or even 25, and placing more reliance on stats. Question: when the second slash is “this year”, does that mean the season I am currently playing, as in stats to date in OOTP? And if I played the previous season in OOTP, would the third slash be those (OOTP; not IRL) stats? And would the fourth slash be the (IRL, if I didn’t play that season in OOTP) stats from two years before?

I ask this because, in my 2021 Season sim, I could bump the second slash up to 50% for “this year”, i.e. current 2022 Season stats in my sim. The third slash would be 2021 stats from the sim. Since I started in 2021, the final slash would be IRL 2020 stats, such as they are, for sixty games, with many players missing time, small sample size. For that unique three-year period, I would probably go 35/50/15/0.
For other non-pandemic years, I would land on maybe 35/35/20/10.

I now understand the logic behind the first (ratings) number for the 25/25/25/25 camp. What I don’t get is why you would value the past three seasons (or the current season and previous two) equally. That is going to hurt young players who went through growing pains but are now playing at a high - and sustainable - level. Likewise it will reward aging stars whose numbers have been declining steadily. Maybe it’s a kind of “recency bias”, but I think it’s fair to ask “what have you done for me lately?”. Two years ago is ancient history in baseball.