Do You Use Ratings?

[Skevich]

I am tempted to turn them off, as they are a distraction, but am unsure how to tell if a a slyer in the amateur draft is a good pick. If you do not use scouting ratings, how do you decide how much potential young talent has?

[Hrycaj]

I am a stats only player. So maybe I can help. I’m sure there are plenty of other ways to go about things so don’t take my method as gospel. First choose what type scout you want. That is important because you want to understand the scouting reports you get from said scout. Here is a snippet from the manual.

The Scouting Director's preference when it comes to projecting the future potential of players. The projection attribute has five settings: Highly Favor Ability, Favor Ability, Neutral, Favor Tools, and Highly Favor Tools. A Scouting Director that favors ability will base players' potential ratings on what the player can already do and project what he thinks the player will become in the future. The director that favors tools will base the players' potential ratings on the maximum peak the player could achieve. In theory, a scout that favors tools will find more superstars, while a scout that favors ability will find more serviceable major league players.

Next really determine which stats are important to you for hitters and pitchers. Once I identify the players statistically, I next read the scouting reports. When I get stats I like, combined with reports I like, then they make my board. I also take into consideration the competition level. It is a lot of reading and digging but I like that sort of thing so I don’t mind. My scouting in game is set to low or normal I think, for what it is worth. My scout is never 100% and injuries (mine are set to realistic modern day regardless of the era I play in) can derail development. My TCR is maxed out as well.

Finally, and probably the easiest way to do things is to simply follow the mock draft. That thing is way too accurate for its own good in my opinion. It serves a purpose though. Some players understandably do not want to get bogged down reading hundreds of scouting reports so the mock streamlines things for those players.

This stuff you probably already know but HS players take longer to develop giving them a greater boom/bust ratio. I will take college players with lesser stats than other guys if the competition level is different. A .270 hitting college outfielder from a great competitive level is probably better than a .320 hitting guy from a community college.

Sorry, I keep adding things but I could talk about this all day. Don’t be afraid to screw up a few drafts. It takes time to be comfortable with your scouts reports combined with your stat evals. After a while though you just get a feel for it.

[Pelican]

I use ratings, with high scouting accuracy, because I am not a “fog of war” person. That said, the younger the player, the less the experience, the less useful any ratings will be. I use a 0-100 scale for ability and potential [I have no illusion that higher numbers mean more accuracy. I’m just a decimal guy, so that scale makes more sense to me than 20-80.], and a 40/30/20/10 mix [ratings/this year/last year/two years ago] for the evaluations.

For high school guys, those numbers are all low, and the potential is pure conjecture. I tend to prefer college guys, with much more of a track record, and an injury history, so they can be projected (and the best can skip the low minors). I have learned to do a manual recalc before the start of each new season. If I spy a guy doing exceptionally well (or exceptionally poorly) in the minors, I dispatch my scout to find out why. To spice things up, so that the ratings are not too predictive, I intend to increase the TCR figure above the default of 100.

[PSUColonel]

Originally Posted by Hrycaj

Finally, and probably the easiest way to do things is to simply follow the mock draft. That thing is way too accurate for its own good in my opinion. It serves a purpose though. Some players understandably do not want to get bogged down reading hundreds of scouting reports so the mock streamlines things for those players.

If you ask me, this is a huge problem, as the mock draft is a dead giveaway (similar to when you used to be able to order the players without stars or the salary demand, or WAR stat) beasically if you follow the mock draft (lack of variance as you noted) you will be drafting the best players from best to worst.

I’m not saying the first round mock draft should be all over the map, it should realistically contain the perceived best players available…but not in order.

[Anonymous Ghost]

Originally Posted by PSUColonel

If you ask me, this is a huge problem, as the mock draft is a dead giveaway (similar to when you used to be able to order the players without stars or the salary demand, or WAR stat) beasically if you follow the mock draft (lack of variance as you noted) you will be drafting the best players from best to worst.

I’m not saying the first round mock draft should be all over the map, it should realistically contain the perceived best players available…but not in order.

I thought that the draft used OSA ratings, which are subject to measurement error?

Anyway, to answer the OP's question, I do use ratings in combination with stats. I hire the best scouting director that I can and lean heavily into their ratings, particularly with respect to drafting prospects.

[PSUColonel]

Originally Posted by Anonymous Ghost

I thought that the draft used OSA ratings, which are subject to measurement error?

Anyway, to answer the OP's question, I do use ratings in combination with stats. I hire the best scouting director that I can and lean heavily into their ratings, particularly with respect to drafting prospects.

Yes, it does use OSA ratings…but what he was saying is there’s not much variance in the mock draft.

[Hrycaj]

After reading some of the dialogue I guess the issue (for me) isn’t in the fact that the mock draft tells you essentially who to draft. I think of it as what all the scouting services do irl as most of the masses have never seen these kids ever play. Yet when it is draft day we have these extensive mocks telling us who the best players are. So I’m okay there.

My issue though is that these mock guys are the ONLY guys that pan out. There are simply not enough busts in the early rounds and hidden gems later. This of course is my perception of things but I have tested this pretty extensively and posted stuff in the beta forums about it. It’s my problem alone though as I have learned that OOTP players don’t really want to see their top picks bust at the rate they do in real life. I imagine the devs have to balance reality with fun.

I don’t want to get too off track though. The OP was simply asking how to go about evaluating talent in a stats only environment and I tried to answer that as clearly as I could.

[MathBandit]

How many years does it take in a new Standard Game for the draft classes to be fully engine-generated with fictional players?

I can try and run a few tests today with various TCR numbers to see how well the first round picks pan out. And is it players picked in the first by the AI, or do those differ significantly from the mock list (and so I should be tracking Mock 1st rounders, instead of actual 1st rounders)?

[PSUColonel]

Originally Posted by Hrycaj

After reading some of the dialogue I guess the issue (for me) isn’t in the fact that the mock draft tells you essentially who to draft. I think of it as what all the scouting services do irl as most of the masses have never seen these kids ever play. Yet when it is draft day we have these extensive mocks telling us who the best players are. So I’m okay there.

My issue though is that these mock guys are the ONLY guys that pan out. There are simply not enough busts in the early rounds and hidden gems later. This of course is my perception of things but I have tested this pretty extensively and posted stuff in the beta forums about it. It’s my problem alone though as I have learned that OOTP players don’t really want to see their top picks bust at the rate they do in real life. I imagine the devs have to balance reality with fun.

I don’t want to get too off track though. The OP was simply asking how to go about evaluating talent in a stats only environment and I tried to answer that as clearly as I could.

yes, but there is no way for the player to simulate reality if he/she wants to. The fact there are very few first round busts is an issue IMHO.

This is something that should probably be adjustable if hey won't go the realistic route.

[Hrycaj]

Originally Posted by PSUColonel

yes, but there is no way for the player to simulate reality if he/she wants to. The fact there are very few first round busts is an issue IMHO.

This is something that should probably be adjustable if hey won't go the realistic route.

I agree with you. I just understand we are in the minority.

[Syd Thrift]

The mock draft uses OSA ratings, which are less accurate than each team’s own individual scout ratings,c even if the scout is awful.

Personally I agree that there are waaay too few busts in MLB but I also remember that the setup used to be a bit more realistic but they got a toooon of complaints that the draft was too chaotic and that they’d draft a guy with a 70 OVR who’d quickly turn into a 35, like by the end of the season. To me, the “unrealistic” part of that is that if a team re-evaluates a player, it should at best use Bayesian logic to evaluate the latest ratings against priors (so if a guy suddenly looks like a 45 OVR but you have, say, 6 or 7 reports throughout college (which we don’t really get and which is its own issue, but I digress), that 70 OVR assessment might only drop to a 65 because that’s one bad evaluation weighted against a bunch of good ones) and perhaps that initial rating should be more or less set in stone until they turn 25 or so.

IME turning development up drastically reduces busts, even with TCR ramped up to 150. That makes some sense as well (I will say that there are a decent number of late bloomers in the draft with TCR bumped up); a lot of those bust types are players who simply didn’t progress (most notably, projected SPs who never develop the 3rd quality pitch).

To the original question I play with ratings off except for “other” ratings because I don’t think the game does a great job of encapsulating the differences in, say, infield arm in-game via stats or the Mad Libs style scouting reports. There too I tried converting some of the reports in the old Scouting Reports handbooks into the game but it’s a bit of an effort.

[PSUColonel]

Originally Posted by Syd Thrift

The mock draft uses OSA ratings, which are less accurate than each team’s own individual scout ratings,c even if the scout is awful.

Personally I agree that there are waaay too few busts in MLB but I also remember that the setup used to be a bit more realistic but they got a toooon of complaints that the draft was too chaotic and that they’d draft a guy with a 70 OVR who’d quickly turn into a 35, like by the end of the season. To me, the “unrealistic” part of that is that if a team re-evaluates a player, it should at best use Bayesian logic to evaluate the latest ratings against priors (so if a guy suddenly looks like a 45 OVR but you have, say, 6 or 7 reports throughout college (which we don’t really get and which is its own issue, but I digress), that 70 OVR assessment might only drop to a 65 because that’s one bad evaluation weighted against a bunch of good ones) and perhaps that initial rating should be more or less set in stone until they turn 25 or so.

Or until they have a certain amount of playing time/at bats and it should likely depend on the levels where the playing time takes place (AAA, AA etc)

[Syd Thrift]

Originally Posted by PSUColonel

Or until they have a certain amount of playing time/at bats and it should likely depend on the levels where the playing time takes place (AAA, AA etc)

Even then I feel like guys like Dustin Ackley and Jesus Montero (or arguably Kyle Lewis) keep getting chances until it becomes abundantly clear that they’re no longer prospects at all.

[PSUColonel]

Originally Posted by Syd Thrift

Even then I feel like guys like Dustin Ackley and Jesus Montero (or arguably Kyle Lewis) keep getting chances until it becomes abundantly clear that they’re no longer prospects at all.

Yea…Injury Log & I were trying to suggest a new scouting system years ago that was tied more to playing time and at bats as well as pathways and shouting strategy. For me, the idea was if a playing, he can’t be scoured very well.

[progen]

"To the original question I play with ratings off except for “other” ratings because I don’t think the game does a great job of encapsulating the differences in, say, infield arm in-game via stats or the Mad Libs style scouting reports. There too I tried converting some of the reports in the old Scouting Reports handbooks into the game but it’s a bit of an effort."

I agree with Syd, I say I'm a "stats only" guy, but like Syd, I do use "other" ratings as well. Also, I never look at the Mock Draft, nor do I look at the email that comes out 30 days before the draft with the top prospects, or the email that comes out the day of the draft. I just feel that makes the draft much more challenging and fun for me.

My scouting is on normal, but as an expansion team in its first year, I don't have the greatest scout. And I generally look to drafting college players in the first few rounds, that will help my Nashville expansion team much quicker than a HS draft pick would.

[Syd Thrift]

To me the mock draft is... fine, especially if you have the scouting accuracy set to the lowest level. IMO the big scouting issues aren't so much with draftees or young prospects (although please note the lack of type errors) but with vets. I still have to turn off potentials because for vets they're basically current ratings (in fact, if a player does take a hit, sometimes that's reflected in their POTs before their current ratings go down so it's sort of cheating) except when I turn them on on Draft Day, but ranking guys by a pretty inaccurate metric isn't the worst thing in the world IMO, especially since your own scout should have a better, if more or less uniquely tailored, rating.

[Charlie Hough]

There's no right or wrong way to go about it, but I've always found the idea of ignoring or turning off the ratings to be rather strange. When you do this, you are literally removing a major aspect of how players are scouted and evaluated in the real world. Scouting reports begin with scouts rating players in specific key abilities and categories for each position, and OOTP's ratings reflect that.

Also, ignoring or turning off ratings seems like you're shutting off something that could be seen as a reflection of the mental and sometimes unspoken ways that general managers, managers and coaches evaluate players. Even if they're not explicitly looking at ratings in a scouting report or quantifying a player's abilities numerically, they're still mentally evaluating them in a qualitative way, and OOTP's ratings can seen as a reflection of that.

Without ratings, the game seems a bit less realistic or immersive to me. You're making yourself deliberately and significantly blind to real-life measures of players' abilities, potential, and what you can reasonably expect of them, which is something a real-life GM or manager would likely never do.

As an added wrinkle, you're also ignoring the key indicators that the game engine uses in combination with league totals to generate player performance. I'm not among them, but for a lot of baseball simulationists, having clear visibility and transparency into the sim's mechanics and how it generates outcomes is crucial. It goes back to tabletop baseball sims and being able to see each player's individual card and the numbers that drive results.

Again, there is no right or wrong approach. If you prefer the challenge of stats-only or have other reasons why you prefer not to use ratings, that's certainly okay. But I've always seen a lot of good reasons to use the ratings.

[PSUColonel]

Originally Posted by Charlie Hough

There's no right or wrong way to go about it, but I've always found the idea of ignoring or turning off the ratings to be rather strange. When you do this, you are literally removing a major aspect of how players are scouted and evaluated in the real world. Scouting reports begin with scouts rating players in specific key abilities and categories for each position, and OOTP's ratings reflect that.

Also, ignoring or turning off ratings seems like you're shutting off something that could be seen as a reflection of the mental and sometimes unspoken ways that general managers, managers and coaches evaluate players. Even if they're not explicitly looking at ratings in a scouting report or quantifying a player's abilities numerically, they're still mentally evaluating them in a qualitative way, and OOTP's ratings can seen as a reflection of that.

Without ratings, the game seems a bit less realistic or immersive to me. You're making yourself deliberately and significantly blind to real-life measures of players' abilities and potential, and what you can reasonably expect of them, which is something a real-life GM or manager would likely never do.

As an added wrinkle, you're also ignoring the key indicators that the game engine uses in combination with league totals to generate player performance. I'm not among them, but for a lot of baseball simulationists, having clear visibility and transparency into the sim's mechanics and how it generates outcomes is crucial. It goes back to tabletop baseball sims and being able to see each player's individual card and the numbers that drive results.

Again, there is no right or wrong approach. If you prefer the challenge of stats-only or have other reasons why you prefer not to use ratings, that's certainly okay. But I've always seen a lot of good reasons to use the ratings.

Agreed 100%. I don't like the idea of turning off ratings at all. They could may be a bit less predictive as they are in OOTP, but I do believe in using them for the very reasons you outlined here.

[Syd Thrift]

Originally Posted by Charlie Hough

There's no right or wrong way to go about it, but I've always found the idea of ignoring or turning off the ratings to be rather strange. When you do this, you are literally removing a major aspect of how players are scouted and evaluated in the real world. Scouting reports begin with scouts rating players in specific key abilities and categories for each position, and OOTP's ratings reflect that.

Also, ignoring or turning off ratings seems like you're shutting off something that could be seen as a reflection of the mental and sometimes unspoken ways that general managers, managers and coaches evaluate players. Even if they're not explicitly looking at ratings in a scouting report or quantifying a player's abilities numerically, they're still mentally evaluating them in a qualitative way, and OOTP's ratings can seen as a reflection of that.

Without ratings, the game seems a bit less realistic or immersive to me. You're making yourself deliberately and significantly blind to real-life measures of players' abilities and potential, and what you can reasonably expect of them, which is something a real-life GM or manager would likely never do.

As an added wrinkle, you're also ignoring the key indicators that the game engine uses in combination with league totals to generate player performance. I'm not among them, but for a lot of baseball simulationists, having clear visibility and transparency into the sim's mechanics and how it generates outcomes is crucial. It goes back to tabletop baseball sims and being able to see each player's individual card and the numbers that drive results.

Again, there is no right or wrong approach. If you prefer the challenge of stats-only or have other reasons why you prefer not to use ratings, that's certainly okay. But I've always seen a lot of good reasons to use the ratings.

But the way the game does it, and TBF the way all games do this, isn't the way it's done in the real world. Scouts look at a player for several games (or review tape, whatever) and base their ratings on what they see. It's not literally "oh, he went 15 for 30 when I saw him play, therefore he must have a contact of 80" but it's all based on watching guys. This may seem like nitpicking but consider:

- You can't say a guy's ratings lowered in the offseason if you didn't look at him. At most you can assume an older player's ratings dropped (or a younger player's ratings went up).

- If a player is sitting on the bench, watching him in BP is only going to tell you so much. It might tell you nothing. Lots of players look nice in BP but aren't good in game situations and lots of players work on things in BP that makes them look subpar. The same goes with spring training and even to some extent the minor leagues.

- Even when you're taking into account things like bat speed or launch angle or LD%, humans, scouts included, are going to be very, very invested in their priors, to the point that IRL a scout will often be the *last* person, not the first, to tell you that a player had an unexpected drop in their "ratings". This is because we don't actually get to look at ratings IRL and can only guess at them by watching players. There's an old Bayes demonstration involving tossing a beanbag over your shoulder, hearing where it is in relation to the mark you're tyring to hit, and then guessing where the mark is. That's what scouts are doing, basically, with "ratings".

- And of course IRL there are all kinds of stuff that can affect "ratings", some of which are modeled in the game and some of which are not. For instance, mechanical issues could be fixable and could "unlock" a guy who's got bad control but good control "potential".

- On the flip side, humans looooove to ascribe meaning to any and all things, and so we will start with a player who is supposed to be good but isn't performing, decide that something has to be wrong, and work our way backwards. This worked pretty well on the savannah but when it comes to stuff like scouting baseball players, often times we wind up blaming mechanics or work ethic or some kind of minor injury that lots of players get when the reality is that the player just wasn't that talented in the first place. It's very, very hard and maybe impossible to separate the signal from the noise here.

I feel that I'm not eliminating a thing that's a vital part of the game, I'm eliminating a thing that kinda-sorta simulates what happens IRL but not super well (and that's not really a knock on OOTP; again, no other sports sim does this either).

[PSUColonel]

I am going to try very low scouting accuracy and 200 TCR for the first time. I really don’t know that it’s going to make anything better, but at this point I guess I’m willing to try anything that might make the game more challenging. Hopefully it won’t be too unrealistic though.

I am also using current & potential ratings (20-80) with trading difficulty as hard as possible. AI Evaluation is 65/20/10/5, so I figure this is probably as challenging as I can get things without robbing my game of realism.