Scouting Level Hardly Matters

[greenOak]

Something I came across just poking around the game. I looked at a draft class' potential under 100% accurate ratings and then compared it when scouted with various accuracy. All results are from OOTP scouting. All players have "Average" level for accuracy of most recent scouting report - NOTE THAT THIS IS DIFFERENT FROM SCOUTING ACCURACY UNDER GAME SETTINGS WHICH IS STATIC. All potential ratings are on the 20-80 scale. I'll show the mean error (absolute value) on each the settings.

Very High: 2.204
High: 2.251
Normal: 2.336
Low: 2.407
Very Low: 2.467

A lot of 20/20 players are rated 20/20 on every scouting level, so if we throw those players out of the pool and do the same analysis I get:

Very High: 3.810
High: 3.932
Normal: 4.110
Low: 4.281
Very Low: 4.416

Basically the difference in accuracy between Very High and Very Low is about a sixth of the difference between Very High and 100%. Moreover, the maximum error for the levels (VH to VL) are 22,22,23,23,24. The number of players who are off by double digits are 85,84,90,97,103. There are 1746 players in the sample.

Basically, there is little difference in scouting levels (at least for the amateur draft). This is annoying to me because I would rather play with scouting more accurate than normal, but even on very high it is hardly noticeable. My guess is there are some people who would rather the ratings be less accurate than normal who will be disappointed by very low as well.

[endgame]

Interesting, but as I'm digesting this I'll have to ask if you might pare that down to exactly what is concluded. Is this an OSA (you said OOTP) assessment then? For the simpler guy like me, what's your contention?

[greenOak]

Yes this was just the OSA (OOTP) assessment on this years amateur draft. I'm doing more tests and so far it holds true for team scouts too. There's some other interesting findings regarding scouting that I'll post in the near future.

[greenOak]

Finding #2: Your scouts will tend to underestimate a players potential in the amateur draft. This effect is bigger the more the scout favors ability over tools. OSA does not show this bias. Here is a chart summarizing some of my findings. All team scouts have "low" intel while OSA has "average". Global scouting accuracy is on normal.

[greenOak]

Finding #3: The players OSA tends to overrate will be the same players your scout tends to overrate and vice versa. Moreover, this correlation is stronger the more your scout favors tools over ability.

This table shows R^2 between the OSA scout error and the teams scout error. In addition are two scatter plots for the two teams whose scouting errors show the greatest and least correlations.

[endgame]

Thanks for the presentations. Looks like you've find a niche of interest and as a result you also have mine. I'll inquire of something on occasion, if that's acceptable to you.

I may not be asking this in the correct terminology in your system of study, but when I identify a single point on your OSA vs Team compositions, how many incidents are represented on or at that point/interval? Is this a single player?

And while I'm considering potential contributors, have you plans to chart the influence of monetary influx or the lack of it? That's also in your wheelhouse, no?

[greenOak]

Yeah each point represents a player in the draft pool. For example, locate the dot in the bottom-right of the Cincinnati scatter plot. For this player, OSA rated his potential 12 points above his actual potential, while the Cincinnati scout rated him 37 points worse than his actual potential. This was obviously an extreme example.

[endgame]

Originally Posted by greenOak

Yeah each point represents a player in the draft pool. For example, locate the dot in the bottom-right of the Cincinnati scatter plot. For this player, OSA rated his potential 12 points above his actual potential, while the Cincinnati scout rated him 37 points worse than his actual potential. This was obviously an extreme example.

Thank you. Outliers and extremes, especially the number and nature of appearances, are great considerations of balance, equalizers, and play.

[stealofhome]

Thanks for doing this work! Did you only look at overall potential? What about for various skills? Can you post the raw data?

[andyhdz]

Conclusion: scouting accuracy setting doesn't make much of a difference so don't bother. Pick a scout with highly favor tools to get the best result.

[SirMichaelJordan]

Keep in mine that OSA is always up to date on their scouting and will always be on an average scouting level.

It would be interesting to see this info under your team's scout.

[endgame]

Originally Posted by andyhdz

Conclusion: scouting accuracy setting doesn't make much of a difference so don't bother. Pick a scout with highly favor tools to get the best result.

an average of best results. While I need to re-read and digest all of the above, I doubt it will change my personal immersion with scouting, as I am all about the exceptions and possibilities.

I think the best results by your definition- I am assuming here -are the more guarded and certain performance from College players, and the high risk, higher potential returns still come from HS, as goes my historical choices.

[greenOak]

I wouldn’t go as far as saying favor tools is better than favor ability. You just need to add 8-9 points on potential if you have a scout that strongly favours ability. Once you do that they seem to be as accurate as a strongly favor tools scout.

[greenOak]

Never mind, disregard what I just said. Highly favor tools scouts are more accurate even when you remove the bias from favor ability scouts. IMPORTANT: This only applies to the amateur draft. Scouts of any type do not seem to underestimate potential (or overall) of players in the MLB or MiLB. I’ll post a more detailed summary later.

[greenOak]

Another finding: The amateur draft budget effects the dynamic scouting level of your scouting reports. I've only tested a few teams, but the results seem clear. To do this, I took over a team on May 5, let the AI handle everything, and simmed until the draft. Then I would CTRL+ALT+DEL force quit out, load the game again, and pick a different team.

The Indians had a measley 330K invested in amateur scouting, and by the time of the amateur draft they had the accuracy of their scouting reports had the following distribution:

High: 112
Average: 148
Low: 315
Very Low: 505

Meanwhile, the Reds had 4.41M invested in amateur scouting and they got the following distribution:

High: 336
Average: 744

Keep in mind there is still give or take to this relationship. So far, the Diamondbacks got the best distribution with 554 high accuracy reports and 526 average accuracy reports despite only investing 1.716M in scouting. Nevertheless, the trend is clear. Of the teams I have tested so far, Baltimore was the only other team to have low and very low accuracy reports and they had the second lowest amateur scouting budget (1.24M).

The difference in accuracy between a High accuracy report and an average accuracy report is small but noticeable. The mean error for high accuracy points so far has been about 0.25 less than the mean error for average accuracy reports.

[Roy Tucker]

So scouting does matter then? Just have to allocate the budget to it

Sent from my Pixel 3a using Tapatalk

[naturaldopamine]

Thanks for this information. So what is the point in 'requesting scouting reports'?? It seems like it doesn't make much difference