Clutch stat

[Syd Thrift]

Nah, the issue here is that the map is not the field, and that was even more true in the days before pitchFX than it is now. Pitch framing always existed, we just weren't able to detect it, and the people who relied on their own intuition that was based on thousands of hours of watching and playing the game turned out not to be wrong. I'm not saying that the stat folks were wrong to stick to their guns, either; however, one thing that Baseball Prospectus for example has done a bunch of in the past 10-15 years is dial back the "us vs the world" mentality and start assuming that people who see the game differently from them arent idiots. In turn, of course, baseball has largely embraced the statistical revolution.

Just to be clear, this isn't about tone either. One of the big issues with conducting an experiment in the social sciences is that it is very, very hard to control for confounding variables. Often you don't even know what might have been confounding before it sneaks up on you. Baseball viewed from this perspective has much the same issue and we need to be a whole hell of a lot less arrogant about what statistics and the like represent.

[RonCo]

I think we're saying the same thing, but maybe I'm wrong.

[BPS]

Some people do better in stressful situations than others do. Lots of empirical evidence supports this claim. The difference in performance in stressful situations is generally linked to come combination of biological, social, and psychological factors. It is also consistent with my observations.

It seems plausible, then, that some might be more likely to do better in stressful baseball situations than others do. Baseball players are human and not machines, and so they are affected by stress.

This might not be fancy thinking, but it is my starting point.

However, the outcome of an at bat has a high degree of randomness and even over a very long time randomness might overwhelm the clutch ability of a baseball hitter. That is, if batting average in a stressful situation is a function of batter skill, clutch ability, and randomness you have: BA(stressful) = F(skill, clutch ability, luck), where luck has a mean of zero but a very high variance.

Among the problems with using measures like batting average to determine clutch hitting ability is identifying who a potential clutch hitter is. For instance, someone might have a high BA(stressful) in a certain year, but this might be due to simple luck. Showing that such a player, in a later year, has a low BA(stressful) might only reveal luck abandoned him. It might say little about whether clutch hitting exists and/or is repeatable.

Let's say, however, you do have some (as yet unknown) way to determining who the clutch hitters are. That is, you know that player X has a positive clutch ability. However, luck might flop around so much that you can't tell, by looking at BA(stressful) of player X that the player actually is a clutch hitter! The variance of luck might be so high that luck cancels out (and achieves is mean value of zero) only with a very high number of at bats.

In principle, though, it might be possible to discover evidence that player X was a clutch hitter if you had enough at bats to consider. Perhaps 100,000 at bats when luck finally cancels out and you have revealed to you the existence of true clutch hitting.

Of course, the problem might be that clutch hitting is itself something that exists for only a relatively small time for certain players. Or, it might be that the level of the clutch hitting skill varies over time and might even disappear only to reappear.

So, clutching hitting skill might exists but might be hard, or perhaps impossible, to detect by simple investigation of batting averages. Yet if you think you have something that indicates someone might be a clutch hitter (although maybe this is only luck that makes this appear to be the case) it might make sense to use them in important at bats.

[italyprof]

Originally Posted by PSUColonel

I would argue offensive stats with runners in scoring position.

The sample size is always too small to be meaningful and the same player has vastly differebt numbers WRISP from month to month, year to year. Been shown time and time again. As several people here keep pointing out.

[Syd Thrift]

Originally Posted by RonCo

I think we're saying the same thing, but maybe I'm wrong.

I think it's similar, but if I can paraphrase, you're saying that the people who said that pitch framing probably wasn't a thing due to the lack of evidence were correct, and my issue is a pretty minor quibble, which is that I'm admitting that they *were* wrong, although I think I agree with you that they followed a good process. That being said, they were still, at least according to the current evidence, factually wrong, and in order to move forward there was a need to understand why the old evidence was incomplete. It doesn't mean that Bill James et al were bad, per se, but tey were wrong and there are reasons that they were wrong, reasons we can and ought to apply to future situations.

The thing that irks me about aspects of the sabermetric community is that there's no sense of skepticism. To me, skepticism isn't about proclaiming to the world that Bigfoot probably doesn't exist or that clutch is a load of crap in baseball, it is to me anyway personal and introspective. It's about having the humility to understand there is always a chance that you're wrong about everything. It's about understanding that other people are smarter than you are. especially when it comes to specialized subjects that you did not choose to specialize in, such as climate science or sabermetrics, and that you should both take the actual positions that these people have studied to heart but at the same time not make additional suppositions about what they said.

I'm not saying you're guilty of any of this; you just came into the middle of a conversation and were like "whoa guys, what's all the hubbub"? You've done a ton of great work with getting the stats to match up with real life in this game, and I felt you deserved an (admittedly long-winded) explanation.

[RonCo]

What part of:

Originally Posted by RonCo

In this light, all the people running around saying differentiation due to catcher defense didn't exist were wrong, despite all the studies around that were unable to see it. It did exist. It just didn't exist in the datasets and methods that were being studied.

Says that I think they were correct?

[RonCo]

Originally Posted by Syd Thrift

The thing that irks me about aspects of the sabermetric community is that there's no sense of skepticism.

People who are real statisticians and truer scientists (for the lack of a better term) are always quite skeptical. It's what allows them to study situations as they do.

If they were not skeptics, then those who were looking for catcher defense would have stopped at the first trials. But instead, they kept searching, kept questioning. Don't confuse those on either side of a debate who are mostly trying prove the others as being stupid with representing the values of those who are trying to determine truth.

That's why I'm asking the hard-core "no clutch exists" folks in this thread why they care so much as to say with 100% certainty that it does not. All you can say with certainty is that we cannot see it using today's methods.

Similarly, if you think true sabermetricians have no skepticism, I ask you what value you get out of that position, since it's so demonstrably wrong as to not really need defense.

But, yes, a lot of people have studied clutch hitting and not found proof of it. At present that's enough for me to say that if I were king for a day I would leave it out of my baseball sim.

Bottom line, I think: either I'm not communicating well, or you're not reading my stuff right, or both.

[Syd Thrift]

Yeah, like I said I think we're pretty close. I agree with you that "true" sabermetricians are probably pretty skeptical. The people over at BP seem to make a point of saying that they aren't sure and that they're constantly recalibrating to get closer to the truth. The same goes for the folks at Fangraphs and, well, all those folks who work for the teams now are interested in winning, not lining up with the orthodoxy. My animus was primarily at the people who kind of hang on the coattails of those people, who get all smug about clutch or whatever without actually understanding what it is the studies say or for that matter what it is the sabermetricians say about the data themselves. And then that made me get on a hobby horse because it's eerily similar to the attitude I see expressed by self-proclaimed skeptics, one of whom apparently decided to challenge me on my own skepticism by asking how closely I cleave to the orthodoxy (which, as it happens, I do, but not because orthodoxy is inherently good, but because that's where the evidence is right now).

Anyway, I think we were speaking mostly sideways at each other rather than against and if I misinterpreted your post, I apologize.

[RonCo]

There are "fanboys" on each side--people who make fun of old school baseball folks for the fun of it, and people who make fun of the geeks for the same reason. Shrug.

[BPS]

It would be easy to produce a baseball simulation in which some players were "clutch hitters" but those who "played" the simulation wouldn't be able to see it by a statistical analysis of the output of the simulation. The major role of randomness makes that possible.

In short, the lack of statistical evidence of clutch hitting can be consistent both with clutch hitting AND the lack of clutch hitting.

[RonCo]

Originally Posted by BPS

It would be easy to produce a baseball simulation in which some players were "clutch hitters" but those who "played" the simulation wouldn't be able to see it by a statistical analysis of the output of the simulation. The major role of randomness makes that possible.

I think this is unlikely to be true. Or, if it is, OOTP is not that game. A simulation has some very specific ways to create it's output. If clutch were put in as an algorithm, it would be relatively easy to find it.

I already have, for example, a script that will go through every game log and register pitcher, hitter, and count and tabulate every result (GB/FB/base hit/walk/HR/K) by inning and ballpark. I could take a few hours and expand that to include score and runners on base. Another hour or so and I can put ratings onto the players.

Next thing you know, I can find "classic" clutch situations and tabulate deviation from norm. At that point it's merely a matter of running a couple hundred sims and crunching some numbers and I could find about anything that's in the code.

Finding these things in software like OOTP requires work hours, but they aren't particularly hard to find if you want to.

[RonCo]

Now, that said. It's always possible Markus has put clutch in right now and it hasn't been found merely because folks like me haven't put the work in to find it. If that's the case, I personally would be unhappy, but the world will keep going.

[BPS]

Originally Posted by RonCo

I think this is unlikely to be true.

It would be very easy to do.

The relevant statistical test would be concerned with whether you have more high performing players than would be predicted by random chance alone. The problem in running such a test is you wouldn't know the exact nature of the randomness and not knowing that you'd not know what the "right" number of high performing players would be. Further, as one can presume clutch ability is normally distributed with a mean of zero, clutch behavior would be nicely hidden in the data. If you knew for a fact that the distribution of high/low performing players should be, say, normally distributed, then you'd likely see a distribution that was just a bit flatter than a normal distribution (if you did have clutch ability) but it would likely be such a small difference that it would not achieve statistical significance. But you would be unlikely to know the true nature of the distributions involved.

This above approach (of looking at the difference between the actual number of high performing players and that predicted by pure randomness) the right approach to take instead of what appears to be the current (misdirected) focus on whether individual players are clutch players. But with real world data you also don't know the nature of randomness necessary to run the statistical test.

Originally Posted by RonCo

Or, if it is, OOTP is not that game. A simulation has some very specific ways to create it's output. If clutch were put in as an algorithm, it would be relatively easy to find it.

I think it's pretty difficult to pull out a signal from noise when you don't know the process generating the noise.

[RonCo]

I'm not planning to argue the field of mathematical statistics tonight, but simplifying greatly I suppose...and not to trivialize the work, but the only real impediment to finding the existence of a factor is to have a properly constructed test (not impossible to do with OOTP), and then take enough data (also not hard with a computer sim).

[BPS]

Originally Posted by RonCo

I'm not planning to argue the field of mathematical statistics tonight, but simplifying greatly I suppose...and not to trivialize the work, but the only real impediment to finding the existence of a factor is to have a properly constructed test (not impossible to do with OOTP), and then take enough data (also not hard with a computer sim).

There are at least 3 things being discussed:
(1) can one discover any hidden algorithms in OOTP by running enough sims? I'm not sure this is possible but maybe.
(2) is it possible to produce a baseball sim that has clutch hitting but no one can use the output of this sim to provide statistically significant evidence for its existence (and you can't simply run as many sims as you'd like...just like in the real world!). This would be easy to do.
(3) have current studies using real baseball data proven that a clutch hitting skill does not exist? The answer, I think, is no, in part because of what #2 shows. Current studies also fail because they are poorly designed.

I am most interested in #2 and #3. The importance of #2 is that it shows something can exist without generating statistical evidence of its existence. Simple empiricism (only things that have empirical evidence for them exist) does not always hold.

[RonCo]

I'm interested in how you would do #2. I'm often too being educated.

Originally Posted by BPS

There are at least 3 things being discussed:
(1) can one discover any hidden algorithms in OOTP by running enough sims? I'm not sure this is possible but maybe.
(2) is it possible to produce a baseball sim that has clutch hitting but no one can use the output of this sim to provide statistically significant evidence for its existence (and you can't simply run as many sims as you'd like...just like in the real world!). This would be easy to do.
(3) have current studies using real baseball data proven that a clutch hitting skill does not exist? The answer, I think, is no, in part because of what #2 shows. Current studies also fail because they are poorly designed.

I am most interested in #2 and #3. The importance of #2 is that it shows something can exist without generating statistical evidence of its existence. Simple empiricism (only things that have empirical evidence for them exist) does not always hold.

Sent from my SM-G950U using Tapatalk

[Syd Thrift]

Originally Posted by BPS

I think it's pretty difficult to pull out a signal from noise when you don't know the process generating the noise.

FWIW I'm almost positive that Markus implemented a hidden "clutch" rating or flag, my evidence being that he's said as much. He's also said that it's a small effect and I don't think he ever defined what he meant by it (if I had to guess it's probably less "late inning pressure situations" and more "LIPS in the playoffs", but all that is is a guess with no further evidence than the intuition that there don't seem to be any clutch-related effects during the regular season, at least not that I can spot).

[NoOne]

ronco : i was speaking from an omniscient standpoint.. the human body is a finite thing. it can be quantified even if we cannot at the moment. i'm including the things you mention as far as the unknowns about the human body - even though we don't have it 100% mapped out, yet.

this isn't an argument, because i'm 99% sure we agree on teh basics about "clutch". just going to clarify what i mean, because i think it was inclusive of what you were saying for the most part.

even the things you mention are limited per person based on their dna, diet, workout regimen etc etc etc including all that we don't udnerstand. even psychosomatic stuff.

when i say 100% yield, i mean getting the absolute most out of your body and talent = 100% and includes all the nuances you allude to. a human (potentially 'anything' unless you have tech that conrols/quantifies/manipulates things down to the subatomic particle or even further) can never reach 100%, let alone go over that 100%.

skepticism within the laws of reality or context of a human is great. even though we need to be skeptical, we know things like a peretual motion machine isn't possible due to laws of thermodynamics. a gentleman i was an acquantance with passed away recently and he had a petent on a machine that supposedly generated more electricity than the work provided by the motor... it was just a big scam. his whole life was selling this scam to people, lol. a modern snake oil salesman. i bet any scientist would agree with what i wrote in teh previous paragraph about a human being and never reaching 100% of their potential in anything, and it's not about a lack of effort at all.

i don't think fight or flight type adrenalin responses are completely out of our control, either. heck, you can meditate or use a sensory deprevation chamber and reduce your heart rate significantly, or listen to some death metal and run yourself through some mental gymnastics to go nuts like you're about to fight someone right befor an AB.

plus, all the emphatamines these guys take thinking it helps in that regard, lol. they didn't have nearly the effect, if any, on the game as steroids have. that was the wonder drug before steroids, and like a corked bat, probably did nothing or possibly redcued output in many cases. being all jittery at bat can't help much.

lol, don't get me started on kreotene and other gnc supplements proven to do nothing, ever, lol.. make your urine expensive and burn out your liver is about all that stuff does.

understanding the corked bat is a matter of physics. function of acceration * mass. either it increases force or it doesn't... no mystery there during any time baseball has been played, yet it's a generally accepted myth that it is true. it takes X newtons of force to move Ygrams of the bat at Z accelleration... it always adds up to the same number of newtons delivered by the bat relative to the person swinging. the excess speed is inherently offset by the loss of mass - it has to be, no ifs or buts. it's an equation that stays in balance.

the only possible benefit is additional bat control if it's a bit lighter than allowed. that could increase contact rates and how pure you hit it. you aren't swinging harder or faster relative to newtons, you are just centering the ball better, if anything. still relative to overall talent (=dna, effort etc)

everythign in the universe works on cause and effect and adheres to specific and repeatable rules even if we don't know them, yet. even so, >100% is not possible. you can hope to approach 100% as a human, but never can we exceed it.

[NoOne]

Originally Posted by Syd Thrift

FWIW I'm almost positive that Markus implemented a hidden "clutch" rating or flag, my evidence being that he's said as much. He's also said that it's a small effect and I don't think he ever defined what he meant by it (if I had to guess it's probably less "late inning pressure situations" and more "LIPS in the playoffs", but all that is is a guess with no further evidence than the intuition that there don't seem to be any clutch-related effects during the regular season, at least not that I can spot).

i've seen enough ~similar quality guys seemingly have too large of a difference in results (~10 years type sample, not 1 season). i have a couple examples on my current ootp18 team right now. in the video game i wouldn't be surprised if there's some unseen ratings that affect how much they guy gets ot of their ratings on a consistent basis.

some guys are WAY more volatile -- within the same ilk of quality or very similarly rated.

without a lot of study and effort i won't say that with confidence. same reasons you allude to. i'm easily fooled by randomness as anyone else is.

[RonCo]

Originally Posted by Syd Thrift

FWIW I'm almost positive that Markus implemented a hidden "clutch" rating or flag, my evidence being that he's said as much. He's also said that it's a small effect and I don't think he ever defined what he meant by it (if I had to guess it's probably less "late inning pressure situations" and more "LIPS in the playoffs", but all that is is a guess with no further evidence than the intuition that there don't seem to be any clutch-related effects during the regular season, at least not that I can spot).

Hmmm...then that's a shame.