The Transparency & Reliability of Scouts' Abilities

[thbroman]

I have had an unusual experience recently that I thought might be interesting to share & get comment on.

Lately I have begun a new fictional league using my customary setting based the early 1970s. Typically when I start such a league, I let it run forward for 10 or 12 years, to generate a bit of an historical record and then I take over one of the weaker and smaller-market teams. It usually takes 4 or 5 years to clear the existing contracts, find a good scout with high ratings, and start building a winner through drafting. Trust me when I say that I have a lot of experience doing this, since I've owned every version of OOTP since 2 or 3!

This year, things have gone quite differently. Nine years into my "dynasty," my team is still mired at the bottom of the standings and my minor-league system is rated 23rd out of the 24 teams in the league. This is the case even though my recently retired scout had "outstanding" and "LEGENDARY" ratings across the board. I should explain that I don't use any player ratings in my games except fielding and speed, having been converted years ago to Malleus Dei's "Let's play stats only!" crusade. So when I draft, I depend exclusively on the scouts' ratings and the draftees' records in college and high school. As I said above, this approach has served me well enough in the past.

So what has happened in this most recent league? One idea that occurs to me is that the ratings shown for scouts and coaches are not 100% reliable. The usually are, by this line of thinking, so that most of the time a coach or scout who is said to be outstanding at what he does really is. (This would explain why most of the time my way of doings things has been moderately successful, because most of the time my scouts are as good as they're supposed to be) If this is really the case, then I applaud this bit of "fog of war" in the game, although I think we should be told about it, so we can learn to factor in a bit of uncertainty when hiring scouts and coaches and renewing their contracts. There's nothing unrealistic in Markus revealing this bit of the engine's secrets, because in doing so he would only be enhancing the game's realism.

There's a second and possibly more interesting reason for the lack of success, which is that my "outstanding" and "LEGENDARY" scout also stongly favored tools instead of ability and this preference, combined with the invisibility of ratings, may have led my scout's recommendations to be less useful than they might otherwise have been. Does this make sense? I'm not sure.

Finally, there's a third possibility which I do not think is in the game, but which maybe ought to be. This is that scouts and coaches have the same personalities that players have, and sometimes they slack off and don't work very hard. One thing I've always liked about Football Manager I that when hiring scouts and coaches for your clubs you have to take into consideration their work habits as well as their pure ratings. I think OOTP could use a dose of this kind of reality somewhere along the line, too.

Well, there it is. Sorry to be so long-winded, but I thought the assembled multitude might find it interesting to read about this, and I know I would benefit from hearing your thoughts on what might be going on. Meanwhile, my woe-begotten club now has a new scout, who, if not "outstanding" and "LEGENDARY" in all of his ratings, is at least "excellent." So we all have high hopes for the future.

[Déjà Bru]

Interesting ideas but I doubt to some extent their grounding in fact. Might lead to a good discussion, though.

I would think that MH would have announced the concept of fuzzy (i.e., potentially inaccurate) coach and scout ratings. That would be a major change and he would be uncharacteristically remiss in not informing the community of it. Same thing for introducing coach and scout personalities, albeit a very intriguing prospect.

One quibble: Am I reading this wrong or do you contradict yourself in this sentence?

I don't use any player ratings in my games except fielding and speed, having been converted years ago to Malleus Dei's "Let's play stats only!" crusade. So when I draft, I depend exclusively on the scouts' ratings and the draftees' records in college and high school.

Or do you mean scout's recommendations at draft time? And, if so, if you have ratings turned off for the most part except for fielding and speed, does THAT affect the ability of scouts to give good recommendations? I don't know the answer to that, myself.

You know, MH keeps talking (as he always does) about improved AI. Maybe it's especially true this time, with this version. Perhaps you are merely being outplayed, for now, by the AI including drafting the best players?

EDIT: A major thought just hit me (and hurt my head, by the way). What if what you are saying about fuzzy coaching and scouting ratings is true and is part of the campaign to improve the AI in this game? Hmmm. Conspiracy theorists, fire away!

[thbroman]

No, I don't let the scouts recommend which players to draft each round - maybe that's my problem eh? Although possibly not, since that other scouting thread has been talking about what appear to be odd recommendations when the scout is asked his opinions. I misspoke by talking about the scout's ratings. I should have said I use his comments.

It's certainly possible that improvements to the a.i. might be responsible for what I've experienced, and if so, I can have no complaints. On the other hand, when you're always drafting in the 1,2 or 3 positions each year, you would think that I would land on one or two stars somewhere along the way. . . .

And as to your last thought, I really would like to see coaching and scouting made a little fuzzier. Life has few certainties, and baseball even fewer!

[Déjà Bru]

Originally Posted by thbroman

And as to your last thought, I really would like to see coaching and scouting made a little fuzzier. Life has few certainties, and baseball even fewer!

You know, I was being facetious with that to some extent. Yet, it is possible if you think about it. What makes you weaker makes the AI stronger. Hmmm.

[Déjà Bru]

I'm surprised this thread has not gotten more play. I thought we raised an intriguing possibility here.

[Left-handed Badger]

I find scouts are unreliable. Legendary= Out of touch old **** who may be mired in senility. Outstanding and Excellent=Full of himself, and who may not have a working relationship with reality. The best scouts seem to gravitate near average, to me. Though, I've come to rely mostly on stats.

[BoSoxFan2]

Originally Posted by thbroman

There's a second and possibly more interesting reason for the lack of success, which is that my "outstanding" and "LEGENDARY" scout also stongly favored tools instead of ability and this preference, combined with the invisibility of ratings, may have led my scout's recommendations to be less useful than they might otherwise have been. Does this make sense? I'm not sure.

Just to throw another thing out there- is there any chance the scouting budget to accuracy metric has been adjusted this year? I'm seeing different results as well for my farm systems- I have found that using scouting recommendations usually gives me more late round sleepers though. One last thing- do you find that the scouting recommendations have a tendency to be hitter focused and/or need-based? When I let the draft auto-complete I often end up with 8 pitchers and 32 batters.

[injury log]

There was a lot of discussion on the beta forum about improving the way coaches and scouts work, and especially how their attributes are presented to the user, but there wasn't time to do anything about it, despite some interesting proposals. So if you're wondering whether scout/coach attributes are now 'fuzzy', the answer is no, they are no more 'fuzzy' than when the text-based ratings were first introduced back in v12 or v13 or whenever that was. Hopefully some interesting changes will be made in v15 or v16, since I think it's an area of the game that is substantially underdeveloped.

[VanillaGorilla]

Originally Posted by Déjà Bru

I'm surprised this thread has not gotten more play. I thought we raised an intriguing possibility here.

One of the many great things about this board is that it is a place where people share their experiences having played the game in so very many different ways.

I have no idea what all went into the scouting rework. The topic of draft poolee's potentials was thoroughly hammered out, some time ago. We know those have changed.

I have played through 70+ seasons in v14, 90% of those I have been merely a spectator. What has jumped out at me, compared to v13 leagues where I played the same way, is the frequency of dynasties appearing that are completely AI controlled.

Teams string together multiple pennants and division wins much more than I witnessed in v13.

Shortly before v14 was announced to have a completely new scouting engine, a thought had occurred to me to make the scouting more realistic. Since it was in the Road to Release thatscouting was being overhauled, I didn't suggest anything because Markus had come up with his own avenue for improvement...whatever that would be.

The thought was this:

Allowing the user to select "unknown" for a scouting setting.

No team IRL knows how good their scouting (or minor league coaches) are until years down the road when they reap the fruits of their labors.

Why should a user know how good his scouting is in comparison to other teams simply by looking at a profile rating of a scout and the number of dollars spent relative to other teams?

Sure, the more money you spend, the better you will be, all things being equal. But, scouting is more than spending. And it is more than reputation. It is many things put together...a culture, if you will. The only way to say if one has a better scouting team than others is to see what guys look like 5 years down the road. You don't know in 2013 how good your scouts are in 2013. You know how good they were in 2008, though.

My idea with the "unknown" setting would change nothing about the previous scouting system (or the current one, for that matter) other than the fact that you would not know if you were receiving scouting reports with high accuracy or low accuracy, or somewhere in between.

There would be a fluidity, yet a tendency for stasis in the accuracy. Meaning that a high accuracy team wouldn't flip flop to a low accuracy in one dice roll, very often. It would usually stay the same, moving up or down a notch as the most likely result other than staying the same.

You would get better results by spending more money, hiring better personnel, but all the money and legendary statuses would have you at a disadvantage if your scout culture gave you a low accuracy as your starting point. A team with a high accuracy from its culture could spend less and get better results....but you would not know which team was better, at the time.

My league is a random debut and I use a AAA system, and that is it. Players that can play, play right away. There is very little that actually gets developed in the minors.

So, when I see teams winning 10 divisions in a row, or going to 6 WS in a row, all the while getting poorer draft picks, I wonder what keeps them going so strong for so long as compared to my v13 experience.

The thought has crossed my mind that MAYBE something like the unknown scouting idea has been put in place, and those teams with the better scouts have made better trades and signed better Free Agents than the other teams over the course of a particular set of years.

This would add to the fog of war in a way that would be, in theory, very simple to code, yet would offer a radically different experience than if the user knows how good his scouts are compared to his competition.

ADD: posted before I read InjuryLog's post. OK, so this isn't in the game. I still like the idea.

[injury log]

Originally Posted by VanillaGorilla

This would add to the fog of war in a way that would be, in theory, very simple to code, yet would offer a radically different experience than if the user knows how good his scouts are compared to his competition.

You can sort of achieve what you're looking for just by making all coach ratings invisible. Coaches and scouts don't really have ratings, at least how OOTP models them, in real life anyway, so my dream coach/scout system does away with ratings entirely. Instead, coaches and scouts would have 'personalities' that might mesh more or less well with your team. So some scouts might drool over the toolshed athletes, and others over the scrappy grinders. And some managers might be fiery and strict, which might be good for a young and possibly undisciplined team, and some might be laid back, which might be better for veterans. And then of course managers could have strategic tendencies - some play small ball, some play for the big inning - and when hiring a manager, you could be hiring his strategic philosophy at the same time.

Anyway, that's what I'd like to see happen with the OOTP Personnel system, since right now hiring coaches is a boring task of chasing down the guy with the best numbers.

[Koprnkc]

I would like a list of the scouts discoveries, along with if the scout pick was made. Then, I could click on the player(s), review their stats and such, and judge for myself if that scout is any good or not.

Granted, other factors determine a players development, but, if a scout has a bunch of busts and guys in low A with no future, then I want to know.

Same token, if he is hitting on more prospects then missing, I would prefer to know that.

Just a thought.

[Déjà Bru]

Originally Posted by injury log

There was a lot of discussion on the beta forum about improving the way coaches and scouts work, and especially how their attributes are presented to the user, but there wasn't time to do anything about it, despite some interesting proposals. So if you're wondering whether scout/coach attributes are now 'fuzzy', the answer is no, they are no more 'fuzzy' than when the text-based ratings were first introduced back in v12 or v13 or whenever that was. Hopefully some interesting changes will be made in v15 or v16, since I think it's an area of the game that is substantially underdeveloped.

Thanks for debunking the "conspiracy"!

Originally Posted by injury log

You can sort of achieve what you're looking for just by making all coach ratings invisible. Coaches and scouts don't really have ratings, at least how OOTP models them, in real life anyway, so my dream coach/scout system does away with ratings entirely. Instead, coaches and scouts would have 'personalities' that might mesh more or less well with your team. So some scouts might drool over the toolshed athletes, and others over the scrappy grinders. And some managers might be fiery and strict, which might be good for a young and possibly undisciplined team, and some might be laid back, which might be better for veterans. And then of course managers could have strategic tendencies - some play small ball, some play for the big inning - and when hiring a manager, you could be hiring his strategic philosophy at the same time.

Anyway, that's what I'd like to see happen with the OOTP Personnel system, since right now hiring coaches is a boring task of chasing down the guy with the best numbers.

Definitely, definitely.

[VanillaGorilla]

Originally Posted by injury log

You can sort of achieve what you're looking for just by making all coach ratings invisible..

Yes...sort of.

One BIG plus to having an "unknown" scouting accuracy as I describe is that you will instantly cut down the ability to exploit the AI in trades.

You can still take all the steps you want to improve your scouting (hiring better personnel, allocating more resources), but in the end, you have no idea if your reports are better than the team with whom you are considering to make a trade. This cuts down the user's incentive to make trades (since it is more likely he will get the short end of a deal) and if the number of trades are reduced, so are the number of trades which exploit the AI.

You will still know that you have a better idea of your own player's worth (all other things being equal) than those on another team, but other than that you won't be able to just go around and finagle your way through the league and achieve dominating talent because you know you have superior information than your trade partners....unless you happen to actually have the most accurate scouting in the league, which you will not know for several years.

ADD: I also like your personality modeled you outlined.