The Hidden Rating Question (again)

[Long_Long_Name]

Originally Posted by Steve Kuffrey

Concur!!!

Didn't you once say they existed? I think I remember that from a few years ago.

[dougaiton]

One does exist, for sure. Markus said there was a hidden DIPS rating for pitchers.

[mcgsports]

They must exist based on all the seasons that
I have run
The .690 and .960 OPS are perfect example
As proof imagine that you imported
Ted Williams and he retires with 10000 atbats!
If he has .960 OPS you think life is ok
But if he has .690 OPS and no injurys and
No rating changes then you know the game must be
broken
Randomness can lead to one season of .690 OPS
for superstar but not a full career
I dont want to do the math now and also
we might have slightly different assumptions
but basically if you have two guys with identical
ratings and that many atbats then its nearly impossible for OPS
To difference by more than 250 points!!!
Not saying Markus is lying but I think he avert
the issue somewhat perhaps by defining hidden ratings
Different than I would
I believe there is no hidden average or homer rating
But there is hidden fatigue and streak and clutch and whatnot
that if not tweaked properly could lead to
a 3000 game streak if you will where a player
over performs his ratings many times more
often than truly random chance
Would allow!!!!!!

[mrbill]

Remember kids, don't do drugs.

But, previous stats being a part of the die roll would be an EXCEPTIONAL change from how OOTP is assumed to work, I believe. Thats a very unstable situation AND has no foundation in baseball stats, I believe. You represent that positive reinforcement as talent and ratings changes, and the "player development" model is responsible for that, not the game engine.

Ratings are the ratings, they're expectation and of course subject to all the externals (pitchers, parks, league settings). But, that's how a player will perform over infinite AB.

If you don't like knowing *exactly* how a player is expected to perform, turn ratings to a 1-5 scale or turn them off completely.

But, if you have 1000 players, of course you're going to see some outperform over their whole careers. If you assume a normal distribution and fairly high variance though, this is perfectly acceptable under the assumption its ratings only with no hidden modifiers.

Markus I believe has said there's some sort of hidden modifier, but its likely fairly inconsequential even in extreme cases, or we'd be able to notice the effect. I'm lead to believe that the modifiers are possible, but they still don't have more effect than the noise of randomness.

[dodgecharger1968]

I think team chemistry affects numbers more than anyone ever wants to admit. If one guy's hitting 3 and he's always got a guy on base, and another guy is hitting 4 with nobody else (decent) ahead of him or behind him, who'll get pitched around, who'll get RBI chances, who'll let things like clutch ratings come into play? And that's the tip of the iceberg. Forget about ballparks; does one guy have a crappy pitching staff he never gets to hit against, or does the other have a great pitching staff he never faces?

On the other hand, I do wanna know what's up with that scouting report stuff with "is well-liked and a lot of fun in the clubhouse" or "the only thing holding him back is his attitude", or the rebuke "my mood isn't that bad yet". There's something hidden, whether it has a big effect or not.

[RonCo]

Markus's post in the link in post #1 of this thread said only that there was no hidden accuracy rating. He didn't say there were no hidden ratings in the game. Given what I've seen in various studies of the game, I would be surprised if hidden ratings did not exist.

[mcgsports]

Originally Posted by mrbill

Remember kids, don't do drugs.

But, previous stats being a part of the die roll would be an EXCEPTIONAL change from how OOTP is assumed to work, I believe. Thats a very unstable situation AND has no foundation in baseball stats, I believe.

Are you talking to me?
Are you talking to me??!?
Thats not what I said
I did not say that Markus folds
The Sim stats back into the ratings
I said an underlying rating in
My opinion from a large
But anecdotal dataset would
I could do more research
But I'm pretty sure so
Expressor variable!
Also be by far the most likely
Far that the data shows
There is an underlying rating
So the ratings we see dont
Explain all the variation

[mrbill]

Lots of people believed that "Bunt for Hit" was a big impact with their anectodal evidence, but even though I did 120,000 AB that showed at best a .002 bonus for a 5 rating over a 1, people probably still think it does something significant.

Without data, I'm going to always play the game as though hidden modifiers didn't exist. The amount of times people believe someone's an outperformer then their luck goes away is more often from when people continues to outperform in the long run, in my experience.

It would take 10 seasons worth of data for tons and tons of players to even begin to prove hidden modifiers. You'd have to start with the theory that there is no such thing as hidden modifiers. Have hundreds of exactly the same rating players. Then test to see if the distribution of averages in a replay league shows a non-gaussian distribution that implies there's actually something separating players with the same ratings around different mean season performances. A good Bayesian framework and a LOT of data would give you information on if and how much hidden ratings exist.

[RonCo]

Data always trumps opinion.

I suspect that if there are hidden ratings, they regulate the development path players take--talent and ratings change rate and longevity--rather than directly influencing performance in any given time period. My opinion on the existence of these can change with the wind since I haven't seen anything to prove their existence. At present, I admit I'm leaning toward something existing in that area (development) due to some partial data I've taken on v5 development ... but *shurg* ... I've been wrong in this area before.
I'm the first to admit that my opinion is essentially a pure guess. And even if I'm right, I woudl have no way to predict what such a hidden value might be, or which players might be affected by it.

Originally Posted by mrbill

Lots of people believed that "Bunt for Hit" was a big impact with their anectodal evidence, but even though I did 120,000 AB that showed at best a .002 bonus for a 5 rating over a 1, people probably still think it does something significant.

Without data, I'm going to always play the game as though hidden modifiers didn't exist. The amount of times people believe someone's an outperformer then their luck goes away is more often from when people continues to outperform in the long run, in my experience.

It would take 10 seasons worth of data for tons and tons of players to even begin to prove hidden modifiers. You'd have to start with the theory that there is no such thing as hidden modifiers. Have hundreds of exactly the same rating players. Then test to see if the distribution of averages in a replay league shows a non-gaussian distribution that implies there's actually something separating players with the same ratings around different mean season performances. A good Bayesian framework and a LOT of data would give you information on if and how much hidden ratings exist.

[Malleus Dei]

Originally Posted by Bobble

R.O.U.S.'s? I don't think they exist.

Then what's that chewing on your arm?

[Malleus Dei]

Originally Posted by RonCo

Data always trumps opinion.

Optimist.

[dodgecharger1968]

Originally Posted by mrbill

Lots of people believed that "Bunt for Hit" was a big impact with their anectodal evidence, but even though I did 120,000 AB that showed at best a .002 bonus for a 5 rating over a 1, people probably still think it does something significant.

Link?

[Eckstein 4 Prez]

I think there are hidden ratings for players in the game. I'm not exactly staying up nights worrying about it, though.

[dodgecharger1968]

Originally Posted by mrbill

Lots of people believed that "Bunt for Hit" was a big impact with their anectodal evidence, but even though I did 120,000 AB that showed at best a .002 bonus for a 5 rating over a 1, people probably still think it does something significant.

That test on the "Power of Bunt for Hit", the one where all your findings were wacky and inconclusive? You've gotta be kidding me. You didn't even have a conclusion. You mostly wanted to know how the rating correlated with speed!

I appreciate the effort, but I may need more of a walkthrough of the significance of your various tests. I am curious to see whether my anecdotal observations hold up to the scrutiny of testing, but I'm still waiting.

By the way, maybe you should focus only on the leadoff hitter, or the top couple. Either way, it's your test. If you're running it to convince yourself, and you have, more power to you. If you want to convince others, keep going.

[mrbill]

Wacky and inconclusive? No conclusion? I started with the hypothesis that 5 BfH should produce about .020 average boost over 1 BfH, at least that's a hard number someone in that thread suggested. I started to get tests that showed a positive correlation. As I did more tests, that correlation got weaker. My "conclusions" were more running summary of what I thought as I did each test. My last conclusion was that in all the tests I did, any difference was probably more noise than an actual BfH effect. "Wacky and inconclusive" is random variance, the heart and soul of OOTP.

I wanted to try and find a scenario where bunt for hit produced a meaningful improvement in average. But, understanding that the effect of "Avoiding Ks" depends on the "Stuff" of the opposing pitcher, I had to vary other factors than just BfH, and the obvious first choice is speed. Maybe BfH talent is more effective when fast or slow. Of course, I ended my test with "I didn't find evidence, what other tests can I do that might highlight how much of an effect is has?"

Now, I could run tests where only the leadoff hitter is different from the rest of the players, but I'd have to do so many seasons to get enough data, that its not worth it to me.

Even if players lower in the batting order bunt half as often for hits, if the rating has a meaningful effect, I think it should show up over 10 seasons.

The real test I want to do is play out games myself, and just make one particular hitter BfH EVERY time at the plate. Using the AI slider may not increase bunting overall enough to notice the effect, if you work off the assumption the effect is still there.

I'm not saying its worthless, but my tests were centered around how an online league player has to use the game, so maybe solo players who control the strategy can make better use of the skill.

Or, maybe its like the "Clutch" rating, and the effects are nebulous at best.

And that ties back into hidden ratings.... in that it's probably hard to find a test that'll highlight the effects without TONS of work on someones behalf, assuming they aren't chasing a development hidden rating or a 0.5% effect hidden rating.

Does anyone know what consistency does? I think that one might play more of a role than people assume.

[RonCo]

Clutch has a correlation to success in close games/situations, but consistency has no correlation to anything I've been able to find. The study I did looked at data from five seasons of an online league I'm in.

[RonCo]

Dola,

The study was on hitters only, BTW.

[mrbill]

Originally Posted by RonCo

Clutch has a correlation to success in close games/situations, but consistency has no correlation to anything I've been able to find. The study I did looked at data from five seasons of an online league I'm in.

Would be interested in that data.

My guess is that most ratings have some impact, but that it could be hard to set up a controlled-enough situation to observe the potential magnitude of the impact.

[dodgecharger1968]

I remember somebody once saying that they found players with more inconsistency win more weekly/monthly awards when they examined records from their sololeagues. I certainly feel like there is a fairly strong effect from this label from my own experience, too. What I'd really like to know about is injury rating. Does it correlate to anything? At all?

[RonCo]

Originally Posted by mrbill

Would be interested in that data.

My guess is that most ratings have some impact, but that it could be hard to set up a controlled-enough situation to observe the potential magnitude of the impact.

Hmm....looked on my hard drive and can't find my study. I'll see if I can recreate it tomorrow. I ran it through some statistitcal software I don't have access to right now.