Scouting Bugs

turtle4499 · 04-08-2024, 06:51 PM

Scouting appears to give extremely inaccurate ratings for players with two systematic errors that lead to overrating certain players well beyond previous games.

Systematic error 1:

The error rate for a scouts measurement does not appear to properly account for the new rating range changes. Previously the ratings where linear with the 1-100 scale that is no longer true. Scouting measurement does not seem to account for this, which leads to high skewed scouting reports for major league level players a ± 20 in 0-550 has dramatically different range in 1-100 depending on the players actual rating. For a player with a rating in the 90 range it is 82-104. This leads to systematic overrating of players. It is extremely problematic on players with stat ratings above 500 as they get extremely exaggerated.

Code:

Rating	1-100	20-80
30	4	20
50	6	25
100	12	25
150	18	30
200	24	35
250	29	35
300	35	40
350	43	45
400	50	50
410	54	50
420	58	55
430	62	55
440	66	60
450	70	60
460	74	65
470	78	65
480	82	70
490	86	75
500	90	75
510	97	80
520	104	80
530	111	85
540	118	90
550	125	95

Systematic error 2:

I am not sure what triggered the following change but comparing 100 scouted clones with fixed equal hitting potentials (83 1-100) there appears to be a dramatic shift in the correlation between scouting ratings. In 23 there was an r^2 of .64 in 25 the r^2 is .98. This leads to a dramatic decrease in the accuracy of scouting reports due to increasing variance at the overall level. Low correlation reduces variance at the player overall level. I assume this is a bug related to the scale changes.

Further it does not appear that the max values for EYE and HR power are scouting the same as the max values for K% BB% and BABIP.

Code:

HT P	K P	GAP P	POW P	EYE P
88	88	88	100	101
88	89	88	99	99
87	87	87	97	98
87	87	87	96	97
86	87	85	94	97
87	85	86	91	94
86	86	86	92	94
86	87	87	93	93
86	85	85	89	92
86	85	85	89	92
86	86	87	94	92
86	85	86	92	92
85	86	85	91	91
85	86	85	91	91
85	85	85	92	90
84	85	85	90	90
85	85	85	89	90
85	85	85	90	89
85	84	84	89	89
83	84	85	89	89
84	85	84	88	89
84	85	84	88	89
87	84	84	89	89
85	84	85	89	89
85	85	85	89	89
84	86	86	88	89
85	84	83	89	88
85	84	83	87	88
83	83	84	87	87
84	84	85	87	87
83	83	84	87	87
85	84	84	89	87
83	84	85	87	86
82	83	83	85	86
83	83	83	85	86
83	84	83	86	86
82	81	81	82	85
83	83	82	85	85
83	83	83	85	85
83	83	82	85	85
82	82	81	83	85
83	83	84	86	85
82	82	83	84	84
81	82	82	83	84
82	82	82	84	83
83	83	82	83	83
81	82	81	82	83
83	84	83	84	83
82	82	81	84	83
82	82	82	83	83
82	81	82	83	83
81	82	80	82	83
82	82	84	85	83
81	82	82	83	83
81	81	83	81	82
81	81	81	81	82
81	81	81	81	82
81	81	82	83	82
81	81	82	82	82
83	82	83	83	82
82	81	81	82	82
81	83	82	83	82
79	81	81	81	81
81	80	80	81	81
80	79	80	79	81
79	79	79	81	80
79	77	77	78	79
79	80	80	79	79
79	80	79	79	79
80	80	79	79	78
79	79	77	78	78
77	77	77	76	77
77	78	77	77	77
77	77	77	77	77
77	77	75	76	77
76	77	77	77	76
77	76	76	75	76
77	75	75	76	76
76	77	75	75	75
77	76	76	76	75
77	75	75	75	74
75	75	74	74	74
74	73	72	73	74
75	74	74	75	74
75	75	74	74	73
74	76	75	74	73
74	72	71	72	72
73	73	72	72	71
72	72	71	71	71
71	73	71	71	71
73	73	71	71	71
72	72	72	71	70
73	73	71	70	70
71	71	69	71	69
71	71	69	70	69
71	70	68	69	68
69	71	69	67	68
68	67	67	67	67
71	70	69	69	67
71	70	67	67	67

The pow and eye ones are correct and give the correct avg when the underlying ratings are regressed in the others are too low.

Just as a side comment, I do not think it is good for the game or justifiable that the range between 40-50 rating is the same as between 50-75. (20-80)

I know these ratings are not directly populating overall but since they seem highly related I assume that it is the same scale range. A 40 FV player should be around 0 WAR a 75 FV player should be around 6 war. The scale range in here is the opposite of what you would expect by normal measures. There should be half the range between 40-50 as between 50-75 not the same. I know this leads to a more accurate number of players but it is way to far from expectations to be reasonable. A 40 rating in any stat is more or less in unplayable territory now.

locuspc · 04-09-2024, 12:00 AM

Wouldn't it be surprising if scouting reports on identical clones didn't correlate? Why do you think it indicates a problem? They're scouting the same exact player, shouldn't they end up with similar scouting reports?

turtle4499 · 04-09-2024, 12:48 AM

Quote:

Originally Posted by locuspc

Wouldn't it be surprising if scouting reports on identical clones didn't correlate? Why do you think it indicates a problem? They're scouting the same exact player, shouldn't they end up with similar scouting reports?

Every one of the 100 players is the exact same. Every one has the same exact rating of 480 for potential.

If my scout saw one players eye as 67 he almost certainly saw there HR power as 65-68. That means that its not the ratings that are correlated that the scouts error and direction of error are correlated. 98% correlated you can use that information to back calculate actual players stats trivially if a player has a single rating that goes above 80. You know that most of his other ratings are also being falsely scouted above their real value. You can use that to estimate with high precision the players real stats across the board.

Lukas Berger · 04-10-2024, 07:25 AM

It's on our list to tweak some things with scouting.

turtle4499 · 04-10-2024, 11:13 AM

I think there needs to be a serious repass of the pitch scouted ratings as well. I know that it doesn't show the stuff rating per pitch directly historically, but the information is being used by the computer to set pitch development levels. Several of them falsely return that the pitch is fully developed, they are not well correlated to actual pitch stuff anymore either.

Fastballs where the most obvious offender the pitch rating is closer to the previous stuff calculation and not very close to the current stuff calculation. 100 mph with 1 point shows up as a 60 despite giving only 1 stuff.

Is there a plan to do a retune on pitch stuff? I think the current one is problematic and causing the issues with too few relievers developing. There doesn't seem to be enough boost to relievers to their primary pitch rating to ensure that relievers have higher stuff then starters. The int'l free agent relievers look correct and reasonable but younger ones get developed like that. It ends up causing relievers as a whole to have k% then starters.

I can make another bug report with some more details but I don't like putting those type of in game formulas in the main forum.

04-08-2024, 06:51 PM	#1
turtle4499 Minors (Single A) Join Date: Apr 2022 Posts: 51	Scouting Bugs Scouting appears to give extremely inaccurate ratings for players with two systematic errors that lead to overrating certain players well beyond previous games. Systematic error 1: The error rate for a scouts measurement does not appear to properly account for the new rating range changes. Previously the ratings where linear with the 1-100 scale that is no longer true. Scouting measurement does not seem to account for this, which leads to high skewed scouting reports for major league level players a ± 20 in 0-550 has dramatically different range in 1-100 depending on the players actual rating. For a player with a rating in the 90 range it is 82-104. This leads to systematic overrating of players. It is extremely problematic on players with stat ratings above 500 as they get extremely exaggerated. Code: Rating 1-100 20-80 30 4 20 50 6 25 100 12 25 150 18 30 200 24 35 250 29 35 300 35 40 350 43 45 400 50 50 410 54 50 420 58 55 430 62 55 440 66 60 450 70 60 460 74 65 470 78 65 480 82 70 490 86 75 500 90 75 510 97 80 520 104 80 530 111 85 540 118 90 550 125 95 Systematic error 2: I am not sure what triggered the following change but comparing 100 scouted clones with fixed equal hitting potentials (83 1-100) there appears to be a dramatic shift in the correlation between scouting ratings. In 23 there was an r^2 of .64 in 25 the r^2 is .98. This leads to a dramatic decrease in the accuracy of scouting reports due to increasing variance at the overall level. Low correlation reduces variance at the player overall level. I assume this is a bug related to the scale changes. Further it does not appear that the max values for EYE and HR power are scouting the same as the max values for K% BB% and BABIP. Code: HT P K P GAP P POW P EYE P 88 88 88 100 101 88 89 88 99 99 87 87 87 97 98 87 87 87 96 97 86 87 85 94 97 87 85 86 91 94 86 86 86 92 94 86 87 87 93 93 86 85 85 89 92 86 85 85 89 92 86 86 87 94 92 86 85 86 92 92 85 86 85 91 91 85 86 85 91 91 85 85 85 92 90 84 85 85 90 90 85 85 85 89 90 85 85 85 90 89 85 84 84 89 89 83 84 85 89 89 84 85 84 88 89 84 85 84 88 89 87 84 84 89 89 85 84 85 89 89 85 85 85 89 89 84 86 86 88 89 85 84 83 89 88 85 84 83 87 88 83 83 84 87 87 84 84 85 87 87 83 83 84 87 87 85 84 84 89 87 83 84 85 87 86 82 83 83 85 86 83 83 83 85 86 83 84 83 86 86 82 81 81 82 85 83 83 82 85 85 83 83 83 85 85 83 83 82 85 85 82 82 81 83 85 83 83 84 86 85 82 82 83 84 84 81 82 82 83 84 82 82 82 84 83 83 83 82 83 83 81 82 81 82 83 83 84 83 84 83 82 82 81 84 83 82 82 82 83 83 82 81 82 83 83 81 82 80 82 83 82 82 84 85 83 81 82 82 83 83 81 81 83 81 82 81 81 81 81 82 81 81 81 81 82 81 81 82 83 82 81 81 82 82 82 83 82 83 83 82 82 81 81 82 82 81 83 82 83 82 79 81 81 81 81 81 80 80 81 81 80 79 80 79 81 79 79 79 81 80 79 77 77 78 79 79 80 80 79 79 79 80 79 79 79 80 80 79 79 78 79 79 77 78 78 77 77 77 76 77 77 78 77 77 77 77 77 77 77 77 77 77 75 76 77 76 77 77 77 76 77 76 76 75 76 77 75 75 76 76 76 77 75 75 75 77 76 76 76 75 77 75 75 75 74 75 75 74 74 74 74 73 72 73 74 75 74 74 75 74 75 75 74 74 73 74 76 75 74 73 74 72 71 72 72 73 73 72 72 71 72 72 71 71 71 71 73 71 71 71 73 73 71 71 71 72 72 72 71 70 73 73 71 70 70 71 71 69 71 69 71 71 69 70 69 71 70 68 69 68 69 71 69 67 68 68 67 67 67 67 71 70 69 69 67 71 70 67 67 67 The pow and eye ones are correct and give the correct avg when the underlying ratings are regressed in the others are too low. Just as a side comment, I do not think it is good for the game or justifiable that the range between 40-50 rating is the same as between 50-75. (20-80) I know these ratings are not directly populating overall but since they seem highly related I assume that it is the same scale range. A 40 FV player should be around 0 WAR a 75 FV player should be around 6 war. The scale range in here is the opposite of what you would expect by normal measures. There should be half the range between 40-50 as between 50-75 not the same. I know this leads to a more accurate number of players but it is way to far from expectations to be reasonable. A 40 rating in any stat is more or less in unplayable territory now.

04-10-2024, 07:25 AM	#4
Lukas Berger OOTP Developments Join Date: Aug 2007 Location: Nice, Côte d'Azur, France Posts: 21,133	It's on our list to tweak some things with scouting. __________________ lukas@ootpdevelopments.com PreOrder Out of the Park Baseball 26! Need to upload files for us to check out? Instructions can be found here

04-09-2024, 12:00 AM	#2
locuspc Minors (Triple A) Join Date: Apr 2022 Posts: 298	Wouldn't it be surprising if scouting reports on identical clones didn't correlate? Why do you think it indicates a problem? They're scouting the same exact player, shouldn't they end up with similar scouting reports?

04-10-2024, 11:13 AM	#5
turtle4499 Minors (Single A) Join Date: Apr 2022 Posts: 51	I think there needs to be a serious repass of the pitch scouted ratings as well. I know that it doesn't show the stuff rating per pitch directly historically, but the information is being used by the computer to set pitch development levels. Several of them falsely return that the pitch is fully developed, they are not well correlated to actual pitch stuff anymore either. Fastballs where the most obvious offender the pitch rating is closer to the previous stuff calculation and not very close to the current stuff calculation. 100 mph with 1 point shows up as a 60 despite giving only 1 stuff. Is there a plan to do a retune on pitch stuff? I think the current one is problematic and causing the issues with too few relievers developing. There doesn't seem to be enough boost to relievers to their primary pitch rating to ensure that relievers have higher stuff then starters. The int'l free agent relievers look correct and reasonable but younger ones get developed like that. It ends up causing relievers as a whole to have k% then starters. I can make another bug report with some more details but I don't like putting those type of in game formulas in the main forum.