Question about Ratings, AI evaluation, and scouting reports.

[Matches]

I have a question about what ratings are displayed on a player's profile page.

Boston sends me a trade offer for a relief pitcher they have in the minors. His page says his OV is 59 out of 80. That seems okay to me and all they want is my lousy backup catcher. But just to be sure, because I've seen instances where a guy is displayed as being a certain rating and then I turn around and he's absolute garbage, I send my scout to evaluate him. He comes back to me telling me his rating is a 20.

But when I open the player's page up again, it still says 59.

I advanced a day and now it says he's a 64. Scouted him again, the result came back that he's a 46. But now his profile says he's a 41.

Why is there a discrepancy between what my scout reports and what his profile page says?

Also, on the Game setup screen, there is a checkbox that says "Overall rating based on AI evaluation, not pure ratings." This box is checked. If I uncheck it, his rating on his profile page shoots up to a 77. If I re-check the box, and change the AI evaluation weights to 100% ratings, should this not make him a 77 on his profile? Intuitively, perhaps, but no, now he's a 67.

All of this begs the question, shouldn't the Overall Rating on the profile page match what my scout says? After all, it says right there at the top "Ratings Scouted by:" If the rating on this and every other screen ISN'T the opinion of my scout, then who's is it? In the AI Player Evaluation Options (AI Options tab under Game Setup), what exactly is this controlling? Is it controlling how scouts evaluate players? How auto-managers select lineups and pitching rotations?

[Biggio509]

A couple of things.

1. Your scout has bad info on players not on your team. The more the player is scouted the more accurate the info is. This seems a little unusual but it seems like the scouting report did not update the screen the first time. Almost always a guy's scouting changes after being in your org. Unless he is a well scouted veteran.

2. The use AI evaluation instead of pure ratings changes the way current overall is calculated. Normally the AI uses scouted ratings, real ratings with some blur factor based on several things. When you click AI evaluation it uses your AI evaluation settings by defualt that formula rates ratings 40% of the total, current stats 40%, last year's stats 15%, two years ago stats 5%. So a guy with low stated ratings but good stats will get a higher current overall if you use AI evaluation. Note exactly how each individual rating is weighted is somewhat black box for pitchers stuff seems to have the most effect on rating, movement second, and control third. I am not sure how the weights for batting ratings work and fielding is counted in there somehow too.

"This box is checked. If I uncheck it, his rating on his profile page shoots up to a 77. If I re-check the box, and change the AI evaluation weights to 100% ratings, should this not make him a 77 on his profile? Intuitively, perhaps, but no, now he's a 67."

No because he was rated at 64 with pure ratings on that is the same as 100% ratings in AI evaluation. Likely he bumped up to 67 when the next scouting report was done after you tweaked AI evaluation because you had more info on him.

You can tweak AI evaluation to be all ratings, defualt, or even all stats or somewhere in between. AI evaluation does have some impact on trades and I believe AI promotion and demotion. However, it has zero effect on lineup AI.

I can't 100% for sure if there is no bug with the scouting reports though. Your email and the profile should say the same thing since it should be the same scouting. This does seem to be changing awfully quickly with a couple of scouting reports. I am assuming the guy is not in system so the info is going to be skewed more but I have never seen it that erratic.

"How auto-managers select lineups and pitching rotations? "

Good question only Markus and maybe a few others know. That is pretty black box but I am 100% sure AI evaluation has 0 to do with it nor does what your current rating of a player says. It is a completely different AI from trade and promotions/demotions.

[Matches]

Thanks. By tweaking the AI Evaluation percentages I was able to confirm that it does not have any effect on lineup selection. And I have noticed that in most cases the new scouting rating does get duplicated in the overall rating.

I guess the one question I have still is -- do the AI Evaluation% values affect the overall rating that your scout will assign to a player?

When I tweak those settings in the Game setup menu, the ratings for players change immediately. Do scouts use the weights defined on that screen to make their evaluations? It seems like this must be the case, if the ratings on a screen are supposed to be the same as what the scout has most recently determined. I guess if a change is made to the weights, the rating changes based on the last evaluation?

Also, I am getting Potential Ratings that are higher than Current Ratings. Isn't the checkbox in the Game Setup screen supposed to remedy that, and if so, how? (The help screen for this is broken).

[Biggio509]

AI evaluation will affect the current rating. That is formed by your scout's opinion of a players rating and if you check AI evaluation stats. So tweaking that will immediately change current rating even if the scout still has the same opinion of ratings. The current will use the new AI evaluation. Assuming you are using AI evaluation and not just ratings.

"Also, I am getting Potential Ratings that are higher than Current Ratings. Isn't the checkbox in the Game Setup screen supposed to remedy that, and if so, how? "

No potential > current means the player should get better. Potential < current means the player should get worse over time. Then there is aging that does not affect what potential shows but decreases ratings over time after a certain point. I would prefer that aging lowers potential but that is not how it is coded.

I think what you may be thinking about is a patch that allowed potential to display < current. At one time if potential < current it showed potential = current. No you can tell your scout thinks the guy is trending downward.

NB: The potential > current means a player should get better or the reverse only applies when you use only ratings not when you use AI evaluation. On any one rating potential > current rating means your scout thinks he has room to improve in that area. Potential rating < Current means your scout thinks the guy is declining in that area.

[Matches]

Sorry, I mistyped. I am seeing Potential < Actual, which seems odd. Even with the item that says "Show Potential < Actual" marked "No, adjust." The numbers do change, but I still see Potential numbers being < Actual. I have a 25 year old outfielder who won the MVP being shown as having a Potential rating of 28. This is, quite simply, absurd.

Strangely, I changed the setting to YES to see what would happen. Then I changed it back (to compare the two) and now it seems to be working properly. Perhaps this is a bug.

Nevertheless, I'm concerned about what possible justification there could be for a scout to think an 25 year old MVP's potential is that of a barely replacement-level guy. This has been a long time beef of mine in that the scouts/ratings in this game do not adjust their opinion of someone's potential for strong performances by young players. This MVP candidate has a Current Rating of 74. That's fine, since the rating takes last year's performance into effect as well as "ratings". (My AI evaluation is set as Ratings: 20%, Current Year Stats: 32%, Last Year Stats: 38%, 2 Years Ago Stats: 10%). But potential rating should take into account his age and his current performance. It makes no sense to say that his potential is "27" when he just hit .352 - 32 -111. What am I to think of a system where a 25-year old guy with a .347 batting average and 47 homers over 259 games is considered to have a potential of 26/80? Especially if the AI evaluation is set to consider ratings to be only 20% of the total picture.

I also note that his individual skill ratings do not show anything worse in "Potential" than his "Current" ratings. So how does

Contact: 17
Gap: 9
Power: 14
Eye: 10
Avoid K's: 19

...translate into 62 as a Current Rating (with Overall rating based on pure ratings and not AI Evaluation) but those same Potential Ratings translate to a 27?

[Fyrestorm3]

I believe (though I could be wrong) that overall potential can be lower than overall ratings, and checking that box just ensures that Power, Contact, etc. won't have lower potential ratings. I think it has to do with how the AI evaluates players - the more weight you give the current year's stats, the more you'll see it happen.

Granted, I've always had some players with better Actual than Potential ratings, so it could just be a bug that I'm experiencing as well.

[Matches]

It's definitely buggy because while I was able to get the POT rating to stop being lower than the ACT for a bit, it's now reverted back. Changing this setting does not change the ratings for any of the hitters on the team and only one of the pitchers changed from a 75/72 to 75/75.

[Biggio509]

Potential is based on ratings alone. If you use AI evaluation for current rating there is absolutely no relation to potential and current rating. Any correlation at that point is broken. So a guy might be an overachiever with your AI evaluation, the scout could be wrong, or he has taking a talent hit and is trending downward. The only time current and potential ratings are correlated in any way is when you use ratings only and not AI evaluation.

Still potential > actual for any rating, eg. con, means that rating should get better. Well if your scout is right. So if contact is 10 and potential is 15 you can expect the guy to get better unless he is an older playing and aging effects are dragging him down. If Con is 12 and potential con is 10 then the guy should get worse.

Once you click AI evaluation the potential ratings and your current ratings are calculated differently. Potential is also ratings only. AI evaluation only affects current. So potential will always reflect what your scout thinks player X's rating will be not what player X's stats look like. Only current will look at his stats over the 3 years and use that as part of the rating. Once you turn on AI evaluation you potential and current ratings are calculated in two different ways. It like comparing inches to centimeters but in this case no one really knows the conversion.

[Matches]

Thank you for that explanation. With that being the case I would strongly suggest revising this in future iterations of the game.

First, a quote:

Director Josef: “No one exceeds his potential.”
Investigator: “If he did?”
Director Josef: “It means that we did not accurately gage his potential in the first place.”

I think it doesn't make much sense for POT ratings to not take into account things that have already happened. Basically I have to imagine that this guy is telling me "Okay well I know this guy just hit .350-30-120, but I don't care, he pretty much sucks." Why continue to insist that someone's potential is somehow LESS than what he's been shown to be capable of?

[endgame]

Disclaimer: I'm not certain of anything.

The system isn't what it appears to be; potential doesn't work like it infers it would. A POT rating simply implies a driving direction for the player. A POT>ACT means the player, if he enjoys typical movement in ratings, will rise toward it, the farther away the faster the movement 'may' take place, i.e. a 23yr old with a 34/80; the closer, the more likely stagnant or slower the movement, i.e. a 28yr old with a 54/59. Although it happens, it's uncommon to see a 36yr old with a wide disparity between ACT/POT. Essentially, he has arrived.

Many factors are playing into the comparison including, with scouting, how that player is measured against the league or his position. Keeping in mind that POT is a 'driving' factor influencing the 'direction' of the player's growth in that big sea of also-growing players, it wouldn't be atypical to see a 24/80 player plateau into a 59/69.

ACT>POT simply implies the player is over-performing, given the comparative factors at work in his evaluation. He's likely to level off, but then again, he may not. The wind is blowing toward him, so to speak, and all the comparative factors will determine whether he continues to step forward or if the gust is strong enough to force him back toward the POT. Consider POT gravity, if you will.

[RchW]

Originally Posted by endgame

Disclaimer: I'm not certain of anything.

The system isn't what it appears to be; potential doesn't work like it infers it would. A POT rating simply implies a driving direction for the player. A POT>ACT means the player, if he enjoys typical movement in ratings, will rise toward it, the farther away the faster the movement 'may' take place, i.e. a 23yr old with a 34/80; the closer, the more likely stagnant or slower the movement, i.e. a 28yr old with a 54/59. Although it happens, it's uncommon to see a 36yr old with a wide disparity between ACT/POT. Essentially, he has arrived.

Many factors are playing into the comparison including, with scouting, how that player is measured against the league or his position. Keeping in mind that POT is a 'driving' factor influencing the 'direction' of the player's growth in that big sea of also-growing players, it wouldn't be atypical to see a 24/80 player plateau into a 59/69.

ACT>POT simply implies the player is over-performing, given the comparative factors at work in his evaluation. He's likely to level off, but then again, he may not. The wind is blowing toward him, so to speak, and all the comparative factors will determine whether he continues to step forward or if the gust is strong enough to force him back toward the POT. Consider POT gravity, if you will.

I agree with most of what you say except for the last part about ACT>POT meaning over-performing. A players actual ratings, whether fogged by scouting or not will predict his results subject to all the vagaries of Baseball (bad luck, bad team, competition etc). So a 75/45 ACT/POT player should perform at 75 not 45. I think the ACT/POT should vary more than it has in past versions and v12 seems to show this more often. It's unrealistic IMO for many players to carry a high POT value into their late 30's.

If you consider RL baseball, many many players clearly perform above their potential especially after 30. Exhibit A is Jose Bautista, a special case for sure but clearly someone who's potential cannot be considered close to his current ability/performance.

An ex-Blue Jay Shaun Marcum is still fooling hitters in 2011. Never a huge talent anyway it's impossible for me to see his potential exceeding his current performance. He could be gone in 3-years or he could be Jamey Moyer II but neither result comes from potential.

Derek Jeter might be a case where a player carries POT=ACT well into his 30's. I'd argue that's no longer the case even with the nice rebound made this year. Some of the angst in St.Louis over Albert Pujols next contract is the real fear that he may have peaked and the next team (Cardinals included) may be paying for the last 10 years plus the next 10 years.

The problem with POT in the game is that it "seems" to be too much info and maybe too definitive. I'd suggest that POT itself be a range but don't know if that's practical.

[Biggio509]

One thing I wish is that aging affects affected potential. You would start to see when a guy's age is showing. In the past this has not been the case. A guy's actuals drop but his potential stays high. So a lot of guys who can no longer perform at the former level are still rated high potential but the aging factor is stronger. I am not sure why Markus just did not make aging effects lower potential ratings to cause the downward movement. Maybe the potential trending does not work fast enough?

[RonCo]

This is a confusing topic because it intertwines base base dynamics with a very wonky scouting algorithm and a somewhat inconsistent aging algorithm.

I do think the aging algorithm can and does influence potential--but it's like any other element of the game, open to randomness. Somewhere along the line, though, development (meaning base rating change) seems to shift away from being driven toward potential (as it is for kids) to being one that fades regardless of potential. This actually makes me (purely) guess that ratings movement has two components--one that is completely driven by age, and one that is an adjustment based on distance from potential. This model would result in players who develop faster when the gap between ratings and potential is high, as well as age more slowly as the difference grows...both behaviors that we see in the game.

Again, just guessing on that.

Endgame has it basically right, though, although I agree that it's not really a guy over-performing.

I stand by my point of view that this process would be far better, and far easier to understand, if the development model was revamped to go away from the "potential" model in the base process--and leave the determination of potential to the scouting algorithm only. (This actually fits that very good quote above).

[Matches]

If you consider RL baseball, many many players clearly perform above their potential especially after 30. Exhibit A is Jose Bautista, a special case for sure but clearly someone who's potential cannot be considered close to his current ability/performance.

An ex-Blue Jay Shaun Marcum is still fooling hitters in 2011. Never a huge talent anyway it's impossible for me to see his potential exceeding his current performance. He could be gone in 3-years or he could be Jamey Moyer II but neither result comes from potential.

To me, "potential" means the level of ability a player is likely to have at his peak. It's somewhat irrelevant for players past their peak, but as soon as Bautista hits 54 homers, its nuts to say that he doesn't have the potential to hit 54 homers. It may not be "likely", but the potential is obviously there unless he loses an arm or a leg, or his steroid dealer gets thrown in jail . Bautista didn't exceed his potential (no one does!), he just exceeded what we THOUGHT his potential was. Not only that, but you could argue that he's done it again this year! (No one thought he had a .300 season in him, but now that he might have one, we have to acknowledge that what we thought about his potential was wrong, and adjust accordingly)

Regardless of his actual, behind-the-scenes abilities in the game, if a player has that kind of season, that should absolutely become the basis for his new scouted "potential", colored somewhat by what the scout thinks is "likely."

Several years ago I also suggested a "range" for potential -- a foggy number that would get narrower and narrower over time. You wouldn't see top prospects with a range of 79, but "44-80". Basically saying, "At the least, we expect this kid to be a serviceable major leaguer, but he could be a star." As they age and scouts get more information on them, AND as their actual performance indicates (which apparently is not taken into effect now), that number would shift and get narrower, to represent the kind of real-life certainty you gain over time about what you expect from a player. So if after last year a scout would say Bautista's actual OV is 71, his "potential" would be, for example, 65-71. That's a scout saying "I think he's absolutely peaked." The 65 comes from the scouts ability to see that his underlying talents are not that good (comes from ratings, the way it does now). The 71 comes from his knowledge that he IS a 71, therefore it's nonsense to say he can't BE at least a 71.

Granted, I could theoretically do this now, and take a scout's POT rating as a consideration of the lowest expectation of the scout, but first of all, think of how absurd that is... Second, if I'm going to do that, and tell myself, that if POT < ACT, then this is just the scout telling me that the guy should not be doing what he's doing, it should at least be tweaked a little to allow for past performance to be a factor. Don't tell me a guy's a POT 26 when he's hitting .300 with 30 homers at age 25. I just simmed another season, and his stats are

2013 .343-15-60
2014 .352-32-111
2015 .345-29-126

26 years old, and the scout now has changed his mind about his potential. It's no longer a 26. He now thinks he's a 27.

And again I raise the question, how can CON-GAP-HR-EYE-K fundamentals of 18-9-16-10-19 translate into a 60 ACT (with ratings only) and those same ratings make him a 28 POT? This seems like a bug, or are there just a ton of outfielders out there with crazy high potential, making his relative ratings seem like 27?

[robc]

Originally Posted by Matches

To me, "potential" means the level of ability a player is likely to have at his peak. It's somewhat irrelevant for players past their peak, but as soon as Bautista hits 54 homers, its nuts to say that he doesn't have the potential to hit 54 homers.

In game terms, it's not entirely true. His power rating potential may mean he should only hit 30 hrs in a year, but do to some the randomness of the outcomes he hit drastically more HRs then he statistically should. So you can look at it as he has the potential to hit 54 HRs because it is mathmatically possible, but I think it is more common to discuss it in terms of what he is likely to hit.

[RonCo]

This is the ultimate problem. The term "potential" means something different in game than it means to any normal human being or follower of a sport. 80% of the time, that works out okay because the game's method manages to work kinda like a human being thinks, and us humans are notoriously great at finding patterns we expect where none actually exist (or where a different one exists). But a very large 20% of the time it just doesn't make any sense to us folks with wetware between our ears.

[Biggio509]

1. Potential does in a way mark the maximum rating your scout thinks. The player may never reach that and the model may cause ratings to increase when POT > CUR but still under that system this is the maximum your scout thinks a developing player can reach. When it does not make sense is when a player is declining. Then potential is where the scout thinks he is going, which is also what it is when the player is improving. Potential is what your scout thinks the player will be in the future. So for a young growing player it is maximum that your scout thinks he will achieve. For declining players it is saying this guy is having a "career" year. A lot of players have one or two really abnormal years.

When I think of Pot > Actual I think of a couple of 3B older prospects the Astros have had. Chris Truby and Chris Johnson both preferred well above what the scouts thought their potential was in their rookie year. Truby reverted back to his normal and I do think in was in the Big leagues after his rookie year. Johnson's hitting was terrible this year and he moved to AAA. In both cases the hitters were above their potential in game terms.

2. With scouting on the emphasis is on what your scout thinks. Even if potential were a maximum it is not an accurate measure with scouting on. You will notice OSA reports do not report on current ability nor potential ratings. So even if potential is a maximum, your scout maybe off on maximum. We should also note say a 20 PH does not mean 54 HRs exactly or any other exact number of HRs. It means he could be a 50 to 70 guy. So no potential precisely pinpoints the number of hits, walks, HRs, doubles, or anything else. The game is a stochastic system not deterministic. All a rating says is what range of stats a guy should be not a guy with 20 CH hitting should hit .400. It means he could hit .400 but he also could hit in the .360's. An example is in 1871 historical sometimes Ross Barnes hits near his historical .428 that year sometimes he hits in the .380's when you play the 1871 season. The CH only gives an expected range.

3. Again note if you are using AI evaluation then current rating has no relation to potential. It is calculated by AI evaluation. Potential is based solely on ratings not stats while AI evaluation brings stats in.

[Matches]

Originally Posted by robc

In game terms, it's not entirely true. His power rating potential may mean he should only hit 30 hrs in a year, but do to some the randomness of the outcomes he hit drastically more HRs then he statistically should. So you can look at it as he has the potential to hit 54 HRs because it is mathmatically possible, but I think it is more common to discuss it in terms of what he is likely to hit.

That's not necessarily the case -- his power rating potential has nothing to do with how he actually performs. It's has actua rating that determines how many he "should" hit, and randomness ensures that maybe he hits 54 a year. Once he's done that, though, his potential should be re-evaluated to take that into account. He can hit 54 homers in a season. It's well within his potential because he just did it!

If OV ratings based on AI evaluation are turned on, they should also be turned on for Potential ratings as well. The whole reason I like to turn on AI evaluation is to make it more difficult -- the ratings I see are based on a combination of what the scout sees, plus what players have done in the past. It makes it tougher because if you have a good scout, you can see a player falling off the cliff because his ratings plummet. By blurring the scouts view with a player's performance, it makes it harder to decide whether or not something is a fluke or not. But having the OV rating be pure scout ratings takes the point out of it because I can see that the scout thinks the guy actually stinks.

Sam Shaw now has 4 seasons under his belt, averages .335-32-124 over 162 games. I hired a new scout, who's rated "Outstanding". He ALSO thinks this guy's potential is 29. How can two scouts get a guy so horrifically wrong?

[endgame]

Okay, so to the over-performing idea, which I'll conclude based on the following quotes, betray a different perspective and show I'm clearly mistaken. I'm confident these posts were public, and can't positively say nothing has changed since they were made, other than in historical contexts where- and those directly involved can speak to it -I believe POT will change to equal OVR if it is less than OVR under certain conditions.

But reading Markus' comments here- snipped, but you can access the original -potential is relative to the league's potential without stats, and overall is actual or scouted rating compared to league/position with stats. So while potential can be compared to player's OVR rating, it's just as tied to league potential, stats excluded. Now, as I say, because I haven't lived in scouting this version, I can't attest that v12 falls exactly in line with the comments below, all made in v10. Regardless, FWIW:

Originally Posted by Markus Heinsohn

...The next patch will introduce some changes to the calculation of the potential overall value (read: silver star rating). In the past this value displayed how a player's potential ratings did match up to the average overall rating on the league. But since the average overall rating had stats included in the calculations, results were strange at times. So, it is now relative to the average potential rating of the league. Another change is the the potential was adjusted to the averall current rating (gold stars) if the current rating was higher than the potential. This was removed.

So, in a nutshell:

- Overall (gold) stars are always relative to the league average ratings (at the player's position), including stats evaluation

- Potential (silver) stars are always relative to the league average potential ratings (at the player's position), EXcluding stats evaluation

Originally Posted by Markus Heinsohn

1) Overall stars always use stats in the equation (the way it's set up in the AI evaluation settings), Potential stars do not, they are 100% ratings.

2) Both are relative to the league average.

3) In some cases with the new pitching system, the overall rating is calculated using the current role of the pitcher, where as his potential rating is calculated the projected role in the future. So, when a pitcher is currently a decent reliever, but projects to a below-average starter, the overall rating may be lower than his potential rating.

So, nothing is broken, it works as designed.

[RonCo]

Yes, I'm sure it's working as designed. But it's designed in a wonky way.