Does OOTP Player Development Needs to Be Fixed?

[injury log]

Originally Posted by RonCo

I decided that I need to respond to this, just because it's an argument that comes up several times and is not correct. It is, in fact, quite normal to use very small sample sizes to adequately analyze entire populations.

I have never said that the issue is sample size. I said the exact opposite, post 136:

Originally Posted by injury log

The issue isn't sample size; Tango's samples are huge. The issue is sample bias, and in the case of Tango's studies, the bias is extreme.

I don't know how I can say that more clearly.

If you want to apply Tango to the same population he did, the data is good. For big league players, OOTP should give results similar to Tango- I agree there. If you're extending it to a different population- minor leaguers, say- there's no reason to expect anything more than superficial similarity with Tango data. To use your polling example, it's akin to polling only those with >$10million annual incomes and hoping to get a read on how an election will play out. By looking only at Major Leaguers, you're only looking at guys who a priori have developed well enough to play in the Majors. That's a pretty extreme sample bias if you're hoping for data that will apply to all minor leaguers.

[injury log]

Originally Posted by RonCo

There certainly exists a chance that people who make the majors develop differently than people who do not, but realize there also exists a chance that all baseball players do develop in the same basic fashion. Since we have no other data to fall back on, we get to use our human intuition on what we think is the right bet. Personally, if I were forced to bet my house on which was more correct, I would bet on the data we've got as being right because as you look at the breakout of skill acquisition and degradation, it just makes sense in the big picture. Again, though...that last is my opinion.

It's here where we disagree. Not sure I'd stake the house on it, but I just don't see any good reason to think this should be true- it seems more natural to me to assume that the top 1%, in achievement, in a population have probably developed differently from the rest of that population.

[spark240]

Originally Posted by Ambermonk

I have scouting off, so supposedly I see the correct potential for all of my players.

I think this is more of a conceptual problem than anything else. A lot of people seem to have the idea that, with scouts off, "potential" is an actual, manifest attribute rather than a hypothetical. It's as if injuries were inflicted by an injury rating that actually scheduled injuries to occur on a predetermined timeline, and that by un-hiding ratings, you could see when so-and-so was going to get hurt. Of course that isn't how injuries work, and, in fact, it isn't how talent-and-potential works in OOTP. Even with scouts off (ratings un-hidden), potential is always theoretical until (unless) it is attained, just like the possibility of an injury is theoretical, until the guy is in fact hurt.

[RonCo]

Originally Posted by injury log

I have never said that the issue is sample size. I said the exact opposite, post 136:



I don't know how I can say that more clearly.

If you want to apply Tango to the same population he did, the data is good. For big league players, OOTP should give results similar to Tango- I agree there. If you're extending it to a different population- minor leaguers, say- there's no reason to expect anything more than superficial similarity with Tango data. To use your polling example, it's akin to polling only those with >$10million annual incomes and hoping to get a read on how an election will play out. By looking only at Major Leaguers, you're only looking at guys who a priori have developed well enough to play in the Majors. That's a pretty extreme sample bias if you're hoping for data that will apply to all minor leaguers.

OK. I missed your 136. I completely disagree with your analogy.

We're talking about essentially all physical skills. Speed, for example. It is not stretching too far to think that a major league baseball player's speed fades any more slowly than a AAA or semi-pro player's speed, or a football player's speed, or my speed. In fact, I would postulate that would be the case--were I to have continued to train as an athlete throughout my life, I'm guessing my speed would grow and peak and fade in a curve that looks like a majority of people's. I may be much slower than other people, but I would guess my peak years wouldn't be far from the "average" person. Similarly, strength. Or quickness ... two things it takes to create batspeed, which equates to home run power. I would guess I would peak in the range of 27 give or take, and that my peak would not be good enough to hit 30 homers in the majors. I see no reason to think normal, physically capable people wouldn't age in similar fashions (average, and standard deviations applicable of course).

The fact that the aging curves have not changed in any substantial fashion over the 100 years of pro baseball is an indicator (but not a proof) that my hypothesis is correct.

[RonCo]

Another way of looking at this:

All people have the ability to run. If all people trained to run every day of their lives, they would all run faster and faster as they grew older and stronger until they peaked, then their speed would eventually fade. Those who are the fastest may well make the Olympic Games, but I see no reason to think that the pool of people who make the Olympics would have their speed peak at a different age than the pool of people who did not make the Olympics.

[injury log]

Originally Posted by RonCo

OK. I missed your 136. I completely disagree with your analogy.

We're talking about essentially all physical skills. Speed, for example.

If only it were that simple. If I recall correctly, Tango is looking at changes in statistical performance, not in 100 yard dash times.

The problems with sampling bias I don't think can be ignored. Tango only considers pitchers who throw at least ~60 innings (250 batters faced) in two consecutive seasons. To even pitch that much in the Majors in one season, a pitcher must have already developed; anyone who doesn't develop isn't included in the study. That's why I don't take it on faith that Tango curves have any relevance to minor leaguers.

Further, if a pitcher's control regresses so much from one year to the next that he is no longer a competent big league pitcher, he's not in the study either; he won't be allowed to pitch the required 60 IP in the second season.

I'll quote Tango himself on the subject:

"Be careful!

Seriously, how the selective sampling issues are dealt with will severely impact the results.

You can also consider taking pitchers with at least 5 consecutive years of 250 PAs. But, there's selective sampling there too. If a pitcher managed to get that much playing time, then chances are his performances did not deteriorate as much as others did. "

from
www.tangotiger.net/adjacentPitching.html

[deadringer]

Originally Posted by RonCo

We're talking about essentially all physical skills. Speed, for example. It is not stretching too far to think that a major league baseball player's speed fades any more slowly than a AAA or semi-pro player's speed, or a football player's speed, or my speed. In fact, I would postulate that would be the case--were I to have continued to train as an athlete throughout my life, I'm guessing my speed would grow and peak and fade in a curve that looks like a majority of people's. I may be much slower than other people, but I would guess my peak years wouldn't be far from the "average" person. Similarly, strength. Or quickness ... two things it takes to create batspeed, which equates to home run power. I would guess I would peak in the range of 27 give or take, and that my peak would not be good enough to hit 30 homers in the majors. I see no reason to think normal, physically capable people wouldn't age in similar fashions (average, and standard deviations applicable of course).

This is what I have the most issues with concerning OOTP's dev engine. A player's potential should be based on the the player's raw physical tools and to a lesser degree his mental tools. A players "current" skills should be based on the players training and experience and again to a lesser degree his mental tools.

The only way a player's potential should drop drastically is if he has an injury that changes his bodies core abilities or if he has a complete mental breakdown. I'm not saying that potential shouldn't change at all over time but the amount that it does should be much, much more subtle than it does in OOTP. In real life I doubt very much that a scout will come and tell a GM one day that a player has gone from a blue chip prospect to a junker incapable of making the majors overnight.

It's much more realistic that the players current "skills" just don't reach their potential and the teams give up on the prospect. Maybe the player was lazy, stupid, or just not mentally tough enough to take advantage of the skills that he was given.

I'd much prefer to see a system where potential is only listed until either a player reaches a certain age, or a certain major league service period. Having a potential rating for a veteran seems silly. We all know that once a player has reached his physical peak (at whatever age that player happens to peak) that his potential is going to go down. I just don't think we should also know when and how fast he's going to plummet. In real life teams find that out the hard way.

The point is that whether your tracking a players development from prospect to peak to retirement, it's the physical tools that determine the potential. Players are ready for the majors when they've matured into the physical skill set they were born with and they retire when those same physical skills have degraded too far for their knowledge, experience, and "learned" skills to keep them effective.

[RonCo]

As I said...even Tango himself would not view this as the bible. But it's certainly true to it's extent. Also, the following statements are true:

• The places where Tango philosophy is most fully "implemented" (major league hitters) are providing results that OOTP users are most content with.
• While the current model does well across all players in the majors, it fails miserably on several case-by-case basis studies
• Users who are focused on the minor leagues are among the most dissatisfied
• Many, many players are getting absolutely destroyed in the minor leagues using the current development model

I've never said Tango's research is perfect, and I don't think I've ever attempted to dodge the possible issues sampling can cause. But the system today is bad. It is "acceptable" only because it is probably better than any system that has been in existence previously. But it's bad.

We have a set of data that is founded in truth for a large number of human beings. If I were Markus I would have no issues with taking a week or so and implementing a Tango driven system. I have yet to see a data-driven argument that suggests this would be a bad decision for Markus to take. If you've got one, please bring it. I'm all eyes.

This is my fault, though. I take full responsibility for the fact that I am obviously lacking in some fundamental capability of thinking, because I just don't get it.

[RonCo]

Originally Posted by deadringer

This is what I have the most issues with concerning OOTP's dev engine. A player's potential should be based on the the player's raw physical tools and to a lesser degree his mental tools. A players "current" skills should be based on the players training and experience and again to a lesser degree his mental tools.

The only way a player's potential should drop drastically is if he has an injury that changes his bodies core abilities or if he has a complete mental breakdown. I'm not saying that potential shouldn't change at all over time but the amount that it does should be much, much more subtle than it does in OOTP. In real life I doubt very much that a scout will come and tell a GM one day that a player has gone from a blue chip prospect to a junker incapable of making the majors overnight.

It's much more realistic that the players current "skills" just don't reach their potential and the teams give up on the prospect. Maybe the player was lazy, stupid, or just not mentally tough enough to take advantage of the skills that he was given.

I'd much prefer to see a system where potential is only listed until either a player reaches a certain age, or a certain major league service period. Having a potential rating for a veteran seems silly. We all know that once a player has reached his physical peak (at whatever age that player happens to peak) that his potential is going to go down. I just don't think we should also know when and how fast he's going to plummet. In real life teams find that out the hard way.

The point is that whether your tracking a players development from prospect to peak to retirement, it's the physical tools that determine the potential. Players are ready for the majors when they've matured into the physical skill set they were born with and they retire when those same physical skills have degraded too far for their knowledge, experience, and "learned" skills to keep them effective.

I think if we were sitting in a room together we would agree on about everything. Markus is constrained a bit by the need to support the historical simmers, hence ratings need to eventually boil down to values that can be calculated with standard Lahman database values. So a rating scheme based on pure physical abilities would require some pretty major surgery and pretty major design work. A more simple approach is to leverage development and ratings on the Tango (or better?) scheme, then, perhaps use knowledge of physical growth to make adjustments.

If I were guessing at what research will eventually show is that the Tango methodology is fundamentally correct, but that rather than one set of Tango values, there might be different Tango curves based on a few different body types. (heavy, light, tall, short would be my first thought, but admittedly I'm making stuff up here)

[The Wolf]

Originally Posted by RonCo

I think if we were sitting in a room together we would agree on about everything.

Sure. After about three days of arguing about it first.

[RonCo]

That would be the fun part.

[deadringer]

Originally Posted by RonCo

I think if we were sitting in a room together we would agree on about everything. Markus is constrained a bit by the need to support the historical simmers, hence ratings need to eventually boil down to values that can be calculated with standard Lahman database values. So a rating scheme based on pure physical abilities would require some pretty major surgery and pretty major design work. A more simple approach is to leverage development and ratings on the Tango (or better?) scheme, then, perhaps use knowledge of physical growth to make adjustments.

If I were guessing at what research will eventually show is that the Tango methodology is fundamentally correct, but that rather than one set of Tango values, there might be different Tango curves based on a few different body types. (heavy, light, tall, short would be my first thought, but admittedly I'm making stuff up here)

I'm certainly not expecting Markus to completely overhaul his dev engine to incorporate physical tools but it shouldn't take that much work to make his engine more realistic.

I don't believe that people suddenly develop much more raw talent at all. They may very well grow and physically mature at different speeds and at different ages and some of these "late bloomer" types may very well account for the Mike Piazza's of the world.

Personally I believe that a player should be "born" into OOTP with a potential level at each of the ratings that the engine uses. I believe that those potential ratings should not fluctuate (+/-)more than maybe 15-20% unless a major injury occurs. I think instead that the players actual (or "current") ratings should only meet those potentials if they are developed properly.

Right now, players potential can be completely decimated by playing them at the wrong level, the amount of their playing time, major injuries, or worse yet, completely at random. I just don't buy into this being realistic.

I do believe that the speed at which they develop and the percent of their potential their current ratings end up should fluctuate a lot. It should grow based on their personality ratings, the level of competition they face, the amount of playing time and experience they acquire, and the level of coaching and instruction they receive. This would allow for quite a bit of randomness and yet fairly realistic curves.

This would also be very easy for historical simmers too, because the ratings of those potentials and the speed of those development curves are already plotted out statistically through the player's history.

With a system like this, you could still end up with realistic talent distribution and plenty of randomness happening but it would "feel" more realistic along the way.

[Kelric]

I was going to stay out of this discussion despite agreeing with many here about the need to re-model the development, ah, model, but then this happened tonight...

I'm chugging along in my online league, fighting for first place in the division. Both of my division rivals make trades to strengthen themselves for a push at the AL East to topple me. I'm in negotiations for a trade of my own, but I refuse to trade my 3/9/6 (on a 1-10 scale), 4-star Potential ace pitching prospect. I can't move the guy, he'll anchor my rotation for the next fifteen years. He's 18 years old, has yet to be injured (this is his first season of pro ball), has a 10 Intelligence and a 10 Work Ethic and has this line:

14-3, 2.36 ERA, 152 IP in 19 games (all starts). He has a 1.13 WHIP, a .223 OAVG and a .272 BABIP. The Triple-A averages are a 3.40 ERA, 1.33 WHIP, .250 OAVG and .270 BABIP. So, to me at least, his success is clearly because he's just a good pitcher - that BABIP is as close to average as you are going to get, so there is no flukiness there thanks to balls luckily dropping into gloves.

What happens tonight? Talent Drop. He's down to 2.5 stars and his Stuff has dropped to a 2, with the Movement and Control not dropping noticeably (still 9 and 6, respectively), but they have to have moved somewhat.

Where is the injury or the poor performance? No offense to you here Markus, but you seem to be the only person in this thread who has never experienced this phenomenon in the game. Perfectly healthy player, chugging along, is performing well... BAM, he's not going to be as good as we thought he would be yesterday. Why? No idea. No explanation from the stats, as far as I can see.

(His player page may not have updated yet, the HTML reports tend to take a while to upload after each sim.)

He isn't nearly as valuable to me, as I already have mid-level starting pitching. I wanted another ace. Now I can't even trade him for the player I was interested in.

I know it isn't a large drop (I've seen much worse), but this has happened throughout my six years of playing your games. Don't get me wrong, I love your games and consider them the best baseball gaming I've ever done, but there are definitely still some things wrong that have never changed since I started back with OOTP4. Mysterious Potential drops is the big one.

Just sayin'.

[Syd Thrift]

Some of the talent drops are random. Isn't much of this thread about how we prefer random chance to something in the stats?

As for a guy suddenly losing it like that, it's rare but it happens. Steve Blass and Mark Wohlers both lost any semblance of control seemingly overnight. I also recall the end of being a top-rated pitcher coming relatively quickly for Steve Carlton as well, although he kept on pitching for several years after he was done.

[deadringer]

Originally Posted by Syd Thrift

Some of the talent drops are random. Isn't much of this thread about how we prefer random chance to something in the stats?

As for a guy suddenly losing it like that, it's rare but it happens. Steve Blass and Mark Wohlers both lost any semblance of control seemingly overnight. I also recall the end of being a top-rated pitcher coming relatively quickly for Steve Carlton as well, although he kept on pitching for several years after he was done.

With Mark Wohlers you could at least argue that it was health related. It may not be an "injury" but didn't he attribute his fall to anxiety or some other mental health issue?

[conception]

Originally Posted by deadringer

With Mark Wohlers you could at least argue that it was health related. It may not be an "injury" but didn't he attribute his fall to anxiety or some other mental health issue?

If I had to pitch and I couldn't throw a strike I would have had anxiety too. There is always something, however the game doesn't have an "emotionally disabled list."

But if you are talking about pitchers fizzling out, Rick Ankiel would definitely be one. It would be sweet if the game generated guys like Ankiel who could either be a pitcher or hitter. A lot of guys drafted have potential for both but the organization picks one to run with. Some guys switch after a year or two.

[The Wolf]

Well, they did say that Wohlers had came down with Steve Blass disease.

[Kelric]

Originally Posted by Syd Thrift

As for a guy suddenly losing it like that, it's rare but it happens. Steve Blass and Mark Wohlers both lost any semblance of control seemingly overnight. I also recall the end of being a top-rated pitcher coming relatively quickly for Steve Carlton as well, although he kept on pitching for several years after he was done.

Rare in big league baseball, sure. How that translates to many OOTP owners seeing it more than once a season, I have no idea. Mayhap we have different definitions of 'rare.'

BTW, Steve Carlton was very good at age 37 (Cy Young Award) and almost as good at 38. Then he was average for two years and below average for two more. Perfect decline, really.

I never would have said something in this thread if Markus hadn't twice said that a situation like I just described DOES NOT HAPPEN without injury or bad performance. Neither was going on here, as far as I can tell.

A player not living up to his Potential does not bother me. I'm perfectly fine with a player busting (since it happens so often in reality), I just think that the way it is displayed is probably not the best. The best solution I can think of that does not involve re-writing the development model is to simply never tell us with numbers if a player loses Potential. Using my pitcher as the example, have him keep his 3/9/6 Potentials on my screen (but lose them in the background) and just never develop his ratings beyond the 2/9/5 he'll probably wind up being. I'll sit there wondering why the guy can't put it together, but the game engine will still be happy. It's just as with a real life example of, say, Matt Clement. For years teams have thought if he could just put all of his talent together he'd be a really good pitcher. It hasn't happened. He kept the tantalizing pitch movement and occasional control (his Potential was always high), but it never 'clicked' for him (his Ratings never improved).

With the apparent return of the text-based Scouting Report, this seems a lot more possible. Someone else in this thread mentioned doing something similar, where the text would just read "Matt Clement still has a good curveball, but his other pitches aren't developing as we had hoped. He's probably a back-of-the-rotation starter at best."

That makes a lot more sense to me than drafting Clement as a 4 star prospect with, say, 6/7/7 Potentials and having him working his way through the minor leagues with no visible problem and then all of a sudden the next day the game says he's a 4/5/5 instead. He'll still have dropped to a 4/5/5 behind the scenes, but to me he'll still be a 6/7/7 who just never developed his ratings all the way.

[jarmenia]

Originally Posted by Kelric

Rare in big league baseball, sure. How that translates to many OOTP owners seeing it more than once a season, I have no idea. Mayhap we have different definitions of 'rare.'

BTW, Steve Carlton was very good at age 37 (Cy Young Award) and almost as good at 38. Then he was average for two years and below average for two more. Perfect decline, really.

I never would have said something in this thread if Markus hadn't twice said that a situation like I just described DOES NOT HAPPEN without injury or bad performance. Neither was going on here, as far as I can tell.

A player not living up to his Potential does not bother me. I'm perfectly fine with a player busting (since it happens so often in reality), I just think that the way it is displayed is probably not the best. The best solution I can think of that does not involve re-writing the development model is to simply never tell us with numbers if a player loses Potential. Using my pitcher as the example, have him keep his 3/9/6 Potentials on my screen (but lose them in the background) and just never develop his ratings beyond the 2/9/5 he'll probably wind up being. I'll sit there wondering why the guy can't put it together, but the game engine will still be happy. It's just as with a real life example of, say, Matt Clement. For years teams have thought if he could just put all of his talent together he'd be a really good pitcher. It hasn't happened. He kept the tantalizing pitch movement and occasional control (his Potential was always high), but it never 'clicked' for him (his Ratings never improved).

With the apparent return of the text-based Scouting Report, this seems a lot more possible. Someone else in this thread mentioned doing something similar, where the text would just read "Matt Clement still has a good curveball, but his other pitches aren't developing as we had hoped. He's probably a back-of-the-rotation starter at best."

That makes a lot more sense to me than drafting Clement as a 4 star prospect with, say, 6/7/7 Potentials and having him working his way through the minor leagues with no visible problem and then all of a sudden the next day the game says he's a 4/5/5 instead. He'll still have dropped to a 4/5/5 behind the scenes, but to me he'll still be a 6/7/7 who just never developed his ratings all the way.

This is basically the suggestion I wanted to make to Markus but since he didn't respond to the email Steve had sent him and copied me on I decided to start this thread.

[rwd59]

Originally Posted by Syd Thrift

Some of the talent drops are random. Isn't much of this thread about how we prefer random chance to something in the stats?

As for a guy suddenly losing it like that, it's rare but it happens. Steve Blass and Mark Wohlers both lost any semblance of control seemingly overnight. I also recall the end of being a top-rated pitcher coming relatively quickly for Steve Carlton as well, although he kept on pitching for several years after he was done.

Random is cool but the drops are too dramatic when there are not injuries. Wohlers and Blass all of the sudden couldn't throw strikes ( Wohlers also had an underlying injury) and I wouldn't mind seeing those type drops occasionally. They just happen too often and too dramatically in the game. Carlton was around 40 years old when he flamed out, something not neccesarily unexpected.

Oops, should have finished reading the thread before I responded as others already said what I just said.