View Single Post
Old 01-12-2020, 01:04 AM   #65
TomVeal
All Star Starter
 
Join Date: Jul 2013
Location: Spanaway, Washington
Posts: 1,181
I ran a small experiment: downloaded the USBL quickstart, set Talent Change Randomness to 1, turned off injuries and suspensions, and created a Legendary scout to go head-to-head against OSA. I noted each one's top ten pitching and batting prospects from the 2018 draft pool, then simmed 12 years to see how they panned out.

With TCR at 1, Potential ratings should have been very good predictors. (Scouting was set at "Normal", so the predictions were bound to be imperfect, but they shouldn't have been terribly off for a scout with perfect accuracy.)

The results were mixed. The scout and OSA overlapped strongly in their pitching predictions; eight of their top ten were the same. But the two on which the scout differed from TSA were the two best performers by a wide margin. They had career WAR/rWar of 36.4/35.7 and 35.8/38.4, respectively. The best OSA-picked performer was 25.4/25.9.

Position players were another story. Among the scout's picks, the best career WAR was 17.8, the second best a mere 8.4. Two guys never made the Majors, and one finished with negative WAR (-0.2).

The OSA identified three prospects who did far better: WAR's of 36.8, 31.5 and 30.5. It also had some busts: one who didn't reach the Majors and an 0.2 career WAR.

In this small sample, OSA held its own against what should have been overwhelming competition. If I have time, I'll experiment further to see whether that result was a fluke.
TomVeal is offline   Reply With Quote