AI Evaluation Settings

[BarneyRubble]

As a 19 year veteran of OOTP and one who wanted to rid the game of what I call 'abnormalities', such as players suddenly not playing because their ratings dropped (not their production) or players getting huge salaries based on one monster year, I'll take a stab at why '2 years ago' matters.

Let's take the two examples MathBandit gives, Player A is 8/4/2 WAR vs Player B with 2/4/8 WAR. Let's start on the day that OOTP 'rolls' current year to 'last year'. Player A hasn't put up 8 WAR yet, so he is at 0/4/2. Player B is at 0/2/4. Suddenly things look totally different. When the season is over and 'current' rolls to 'last year', Player A is 0/8/4 and Player B is 0/2/4. Huge difference. This fog of war exists unless you crank up the ratings. Which some people prefer and that's OK.

I don't know when 'current year' is reset (I wish I did), 'current year' is very dependent on how deep into the season the Player is.

The scouts and GMs have their own values and not many scouts or GMs value 'current' and '2 year ago' equally. I like the fog of war and hated winning 100+ games a year because scouting was way overpowered. That doesn't mean it's for everyone. In fact, it's fairly close to being stats only.

I found it's not good to overvalue anything, including 'current year'. The 'current year' is only a valid data point between the all-star break and whenever OOTP 'resets' the 'current year'. There are really only three dates that would make sense, 1) off season begins, 2) Jan 1 and 3) pre-season begins. I would personally do it when the pre-season begins for several reasons.

Some people use 67 as 'current season'. Well that 50 HR year is worth '67' until the day it's magically considered '20'? Sadly, I believe they may be using ratings more than they realize because they value 'current year' at 67 when there are NO STATS for the 'current year'.

One of the issues is not knowing exactly how the mechanics work. That said, I found 4x25 to be the most realistic. Of course, OOTP might have made changes in 23 and I'm on here in a few weeks saying 25/25/25/25 isn't any good now.

[uruguru]

To me the issue is that WAR is highly variable so there's still significant uncertainty whether someone with 8/4/2 WAR will have a better next season that someone with 2/4/8 WAR. The OOTP sim engine has a lot of variability so it's risky to put too much weight on last season's WAR as opposed to the average of the past 3 seasons where some of that variability will be smoothed out.

[PSUColonel]

Another year, another great discussion for sure! .


I must admit I desperatley want to use 25/25/25/25 for so many of the reasons that have been articulated. For one, it allows the personality of each staff member to truly come into play. It also gives a better overall "body of work" perspective on each player.

Here is a question I have for those using these numbers: Do you not get an advantage over the AI when you can see the scouted grades drop, yet the AI cannot? (or not as clearly, as it's only 25 percent of the equation)

I think (as I have mentioned in previous posts & threads) this is where the challenge vs realism issue rears it's ugly head. The fact is...we all WANT both. Reality is...we CAN'T HAVE both.

Some will say, "well if you have ratings & stats split 50/50 that should do it". The truth however is that it doesn't seem to work that way. It has literally been my quest I think to try to find this middle ground which doesn't exist.

So my argument for 55 ratings is still there because 1. It helps the AI with trading, waivers and contract decisions. Is it realistic to see a player who is performing well suddenly place on waivers or cut? No...it isn't...but I contend you the human player can see the ratings dip and now weigh it more heavily than the AI...which gives the players a massive advantage. The only way for this not to happen is to not use ratings at all, and go stats only.

This is why 55 to me, is the best number. It allows for stats to be a pretty big part of the equation without blinding the AI to the facts. My biggest question however now, is the debate between Sweet & MathBandit....should the weights be 55/15/15/15...which at least looks at each year equally and in premise is a bit like 25/25/25/25 with the obvious exception of ratings....or go with the declining scale of 55/20/15/10?

People did use 40/30/20/10 for years, and seemed to like it as they felt any combination of the two had a lot of weight, and it was still mostly stats over ratings. I do think 25/25/25/25 is better than 40/30/20/10 simply because I feel if you use anything less than 55 for ratings, you might as well look at the entire body of work.

[PSUColonel]

One more thing to mention...scouts do look at players and can tell if they have progresses or regressed in their physical toolsets. To me, that is what is represented in the scouted (ratings portion) or the equation.

And truthfully, with 25/25/25/25 some GMs will act more like a 55/15/15/15...but they are all different, so the weights get mixed around quite a bit depending on the staff member''s profile.

[Sweed]

Originally Posted by PSUColonel

Another year, another great discussion for sure! .

Here is a question I have for those using these numbers: Do you not get an advantage over the AI when you can see the scouted grades drop, yet the AI cannot? (or not as clearly, as it's only 25 percent of the equation)

Yes, it is a good discussion like every year.

The short answer to your question is.... No.

When my scout drops one of my players grades how do I know he is correct?
There are times I see a player's ratings take a decrease but his stats do not. I don't bench or think about trading this guy until I see proof of the decrease. What if I did to only later find out my scout is wrong? I am the same way when the scout says the ratings have increased. Oh yeah, show me. If you can't back it up with stats come back when you can.


Isn't the AI using the 4- 25's the same? IE it's not placing as much weight on the ratings so when they do change it also waits for some confirmation from the stats? Isn't this a good thing and doesn't it actually help the AI?

What if ratings increase with the 55 you use but the stats don't follow? Will the AI extend this guy without any statistical proof? Would a real life GM extend a player based on "the scout said"? I know I won't. Is knowing the scout may be wrong and waiting on stats an advantage to the human player when using 55?

What if the scout telling the AI the ratings increased or decreased is wrong? Then the 55 rating has it making bad decisions based on bad information instead of waiting for verification through stats that come from game performance. In this situation isn't relying more on skills than stats a disadvantage to the AI? IMHO the knife cuts both ways.

As I said earlier I do not see the "sweeteners" in my trades via the AI asking for those types of players. I do see the AI ask for stars or my top prospects in deals that would be fleecing me if I accepted.

Can I add "sweeteners" one by one to fleece the AI? Sure, but that is under my control and in the end, why would I? And for the guy that "can't help himself" or thinks it's "a perfectly good way to win since the game allows it"?
Well, that player is always going to find a way and won't be bound by either a 55 or 25 evaluation number. He simply won't care.

Viva La 25's

[PSUColonel]

OK Sweet...you've convinced me to give 25x4 another shot. When the new patch comes out, I plan to start a new save, and will give it another ride.


EDIT: Let me ask, how many of you use overall/potential ratings? Sometimes I wonder if that's a big part of the issue...as pitchers' will be highly valued even if their potential ceiling might only be a 30 or 35.

It might also be more realistic and challenging to get rid of these two ratings while going with 25/25/25/25

[SirMichaelJordan]

I’ve tried many combinations over the years, but I always end up back at 25/25/25/25


The main reason I put weight on two years ago stats is that the current year stats doesn’t become the current year stats until a certain amount of PA/AB.

But this could be outdated information, so I may be wrong.

Also, I play with very Hard trading and heavily favor prospects.

Some issues people have using heavily favor prospects, I do not have, and I think it may be because of the AI settings.

The interesting part about using 25’s is how it interacts with “incorporated stats” where the scout now have their own personality AI eval settings. But there is little information on how that feature actually works. i don't really see any changes when it's on or off.

[MizzouRah]

They should probably have an option to turn AI evaluation settings OFF and just go by the actual scout ratings.

That being said, I've settled on 50/25/15/10. I also incorporate stats into scouting reports.

[BarneyRubble]

Originally Posted by Sweed

Viva La 25's

^^This x1000^^

I tested different settings 2 or 3 years ago until I was blue in the face. I have no idea how accurate (or relevant) that testing is as far as OOTP 23 is concerned. One extremely important thing I learned (thanks Markus), and I assume is still accurate, is scouts will ALWAYS use ratings to some extent.

I consider the first number (25) as the MINIMUM weight a scout will use ratings for evaluation. If a journeyman player has a sufficient amount of 'stats', those 'stats' will be used. If not (draftees for instance), the scouts use ratings.

From my observations (it's impossible to know without OOTP telling me), if no sufficient data exists for a 'current year', 'last year' or '2 years ago', ratings are used. For example, when 'current year' is 'reset' (becomes last year), the effective weight is 50/0/25/25 (assuming last and 2 years ago are present). A draftee with no stats would be 100/0/0/0. The bottom line is, the OOTP world makes a lot more sense with 25/25/25/25.

It would be nice to know when 'current year' becomes 'last year', but with 25/25/25/25, it doesn't really matter I guess.

I'll say this, I've played since OOTP 4 and when I went to 25/25/25/25, it became a brand new game. But not everyone has the same expectations and that truly is the beauty of OOTP baseball.

[Sweed]

Originally Posted by PSUColonel

OK Sweet...you've convinced me to give 25x4 another shot. When the new patch comes out, I plan to start a new save, and will give it another ride.


EDIT: Let me ask, how many of you use overall/potential ratings? Sometimes I wonder if that's a big part of the issue...as pitchers' will be highly valued even if their potential ceiling might only be a 30 or 35.

It might also be more realistic and challenging to get rid of these two ratings while going with 25/25/25/25

I do use overall and potential. I hate the stars and find them useless. Overall and potential are slightly better than the stars IMHO. They are nothing more than a "quick guide" to get my attention. After that I know I have to look at the players skills\tools to get an idea of what I actually have.

Having said that I never limit myself to looking at players that have xx+ in overall\potential. Many good players can be found by filtering for one or two tools\skills that I value. Many can also be eliminated by not being good enough in one category. A pitcher with a 3 in control (2-8 scale) pot. or overall is not going to make it onto any of my shortlists. I will not have this player in my organization.

Yes, on occasion I will find later my scout miss judged one of these Ps and they ended up in the league and doing just fine and, that's ok. I'll live with one or two and not wasting my time on the many that won't cut it on my team. In the final analysis I hate giving up walks and I filter out P's that are below my standard in control.

At the end of the day I could live without Ov\Pot but I see no reason to not have them. If I did go without I would simply load up views, sort by what I value, and then note the skills in the neighboring columns. It's something I already do in my league that has Ov\Pot when I "drill down" into the player's actual tools, abilities, and personality attributes. This is how I fill out the lower portions of my draft board and why I can draft 20-25\80 players late and have some of those guys eventually see time in the league.

I think the AI does evaluate in a similar fashion. It certainly doesn't just look at Ov\Pot. IOW all 55's are not alike to me or the AI. I can offer a 55 OV for a 55 Ov at like positions and be laughed out of the building. I can also make that same offer and have the AI jump at it and have me thinking "did I just get screwed?"

So, my opinion is having or not having the Ov\Pot makes no real difference other than maybe making me take more time to find the information I would have found anyway with Ov\Pot. YMMV

[BarneyRubble]

Originally Posted by PSUColonel

EDIT: Let me ask, how many of you use overall/potential ratings? Sometimes I wonder if that's a big part of the issue...as pitchers' will be highly valued even if their potential ceiling might only be a 30 or 35.

It might also be more realistic and challenging to get rid of these two ratings while going with 25/25/25/25

I use OVR/POT, but I don't view them as end-all be-all. However, they are still fairly accurate. Ratings are by no means ignored. I consider '25' to be the minimum amount ratings will used. If you are looking at a 20 year old with zero to little stats, the OVR/POT means A LOT. If it's a 30 year old then you take a deeper dive.

[Sweed]

Originally Posted by BarneyRubble

^^This x1000^^

One extremely important thing I learned (thanks Markus), and I assume is still accurate, is scouts will ALWAYS use ratings to some extent.

I was thinking I had seen Markus or Matt mention this too, at one time, and almost said the same in one of my replies. Being only about 90% I was remembering correctly and unable, well more like didn't feel like searching, I decided to leave it out.

[BarneyRubble]

Originally Posted by SirMichaelJordan

I’ve tried many combinations over the years, but I always end up back at 25/25/25/25

I believe it was you who mentioned 25/25/25/25 and I tried it. I haven't been off it since.

Thank you

[SirMichaelJordan]

Yup, ratings are used if there aren't enough stats to go off.

[PSUColonel]

Originally Posted by MizzouRah

They should probably have an option to turn AI evaluation settings OFF and just go by the actual scout ratings.

That being said, I've settled on 50/25/15/10. (Except 55 ratings)I also incorporate stats into scouting reports.

Yep, this is what I have been using/recommending as it still puts more weight on last year & two years ago than the default does.

[PSUColonel]

Originally Posted by Sweed

I was thinking I had seen Markus or Matt mention this too, at one time, and almost said the same in one of my replies. Being only about 90% I was remembering correctly and unable, well more like didn't feel like searching, I decided to leave it out.

They did…this is a well known fact. Stats are always used on some level…even for those who “think” they’re playing stats only…they really aren’t in the true sense of the words.

[Jabn]

Originally Posted by PSUColonel

I’ve never tried it, but I wonder how 55/15/15/15 might function?

Ill give it a shot

[italyprof]

Originally Posted by Sweed

I'll post three quotes from last years thread that I linked. https://forums.ootpdevelopments.com/...light=settings
I think you'll get the gist but may want to go to that thread for the full context to make sense of it all. IE one user wondered if going with all 25's amounted to "turning off" the ratings evaluation. So, you'll see me saying things like "Level out to me does not mean turns off." Reading through last years thread will put this in context.

Also to be clear the whole 4- 25's was not my idea. SMJ gets credit for that. I just tried it, liked it, and have continued to use it







I know from reading more recent posts from PSU that he has moved off of the 4- 25's but, this post still sums up the reasoning of using 4- 25's.

Intuitively, if one plays 3 or 5-year recalc as I usually do, then 25x4 seems like the best setting, since the player's talent/performance is spread out over several seasons in a sort of average, and these settings allow the AI to perceive that fact, no?

[Bluenoser]

What do you all do with a brand new fictional league that has no stats?

Do you adjust it each year for the first 3?

[BarneyRubble]

Originally Posted by Bluenoser

What do you all do with a brand new fictional league that has no stats?

Do you adjust it each year for the first 3?

If there are no stats, don't worry about it. Don't get hung up on the settings'. Not every player is scouted exactly according to the 'settings'. A player with no stats will be scouted by ratings. As he accumulates stats, they'll be incorporated per the 'settings'. This prevents an all-star veteran from suddenly loosing his job. I'll give you two examples that made me search for a solution.

JJ Hardy, coming off of a very good season and making the all-star team, shows up to pre-season and is no longer the starting SS. In fact, he gets almost no playing time in spring training. He is now backing up a scrub at SS. Apparently the scout was spying on him during the holidays and saw that he couldn't field or hit anymore. The same year, I noticed Adrian Gonzalez was in AAA. He had hit 30+ HRs the year before. But the scout said he was no good anymore and LA spent huge money on another 1B and sent Adrian to the minors.

Does the problem still exists? I don't know. I went to 25/25/25/25 and haven't looked back. It forces the AI to use performance (stats) as an indication of talent. Now, JJ Hardy might (or might not) loose his job before the season is over because his 'current year' statistics dropped as well as his ratings.

If a player with little to no stats (for whatever reason, draftee, international, fictional league, etc...) is scouted, the ratings are used. When looking at the 'settings', think of that first number (the ratings number) as the minimum number a scout will use. The setting really doesn't have an impact if a player has no stats.