[mrbill]

Lots of people believed that "Bunt for Hit" was a big impact with their anectodal evidence, but even though I did 120,000 AB that showed at best a .002 bonus for a 5 rating over a 1, people probably still think it does something significant.

Without data, I'm going to always play the game as though hidden modifiers didn't exist. The amount of times people believe someone's an outperformer then their luck goes away is more often from when people continues to outperform in the long run, in my experience.

It would take 10 seasons worth of data for tons and tons of players to even begin to prove hidden modifiers. You'd have to start with the theory that there is no such thing as hidden modifiers. Have hundreds of exactly the same rating players. Then test to see if the distribution of averages in a replay league shows a non-gaussian distribution that implies there's actually something separating players with the same ratings around different mean season performances. A good Bayesian framework and a LOT of data would give you information on if and how much hidden ratings exist.