Clutch stat

[Syd Thrift]

I'm almost positive that "playoff LIPS" style clutch has gone unstudied, largely because the sample sizes are tiny. I'm pretty meh about that either way.

[BPS]

Originally Posted by RonCo

I'm interested in how you would do #2. I'm often too being educated.

Sent from my SM-G950U using Tapatalk

Batters, pitchers, and fielders all have clutch abilities.

But, you could simply add a variable "clutch ability" for each batter, C, that affected outcomes only in pressure situations. C would be distributed over batters with a mean zero and a "relatively small" variance.

If you ran a series of seasons where players typically played only a small number of years, and you did this one time, the combination of randomness and small sample size would make it very difficult to determine (as an outsider who didn't know whether C existed or not) whether players had clutch abilities or not.

As I said, it is very simple to add clutch ability and very difficult to determine whether clutch ability existed just looking at the data.

[RonCo]

Originally Posted by BPS

Batters, pitchers, and fielders all have clutch abilities.

But, you could simply add a variable "clutch ability" for each batter, C, that affected outcomes only in pressure situations. C would be distributed over batters with a mean zero and a "relatively small" variance.

If you ran a series of seasons where players typically played only a small number of years, and you did this one time, the combination of randomness and small sample size would make it very difficult to determine (as an outsider who didn't know whether C existed or not) whether players had clutch abilities or not.

As I said, it is very simple to add clutch ability and very difficult to determine whether clutch ability existed just looking at the data.

This would be discoverable in something like OOTP merely by setting all players other skills to neutral and running a hundred seasons or so, then doing a study like I described. It takes work, but is able to be discovered.

But, yes, by definition if you make the variance you code extremely tiny, then you can implement anything and have it hard to feel in everyday play. This is the scenario by which its impossible to prove something doesn't exist without very large samples. All you can say in real life is that the impact is so small it's hard to find without more sample. The key here is that it is so small it is not affecting the outcome in any perceivable way. Operationally, some people would then say it does not exist, which is fair enough even if it's wrong.

Discounting, perhaps, some more advanced deep-learning algorithms, my view is that if you code it in software, I will eventually be able to find it in results if I care to spend the time looking.

I can, for example, see influences of managers and coaches in OOTP data, though I haven't gone the extra step to quantify them. It's hard work. Tedious and somewhat complex. But easily doable with enough motivation.

[RonCo]

Originally Posted by BPS

Batters, pitchers, and fielders all have clutch abilities.

But, you could simply add a variable "clutch ability" for each batter, C, that affected outcomes only in pressure situations. C would be distributed over batters with a mean zero and a "relatively small" variance.

If you ran a series of seasons where players typically played only a small number of years, and you did this one time, the combination of randomness and small sample size would make it very difficult to determine (as an outsider who didn't know whether C existed or not) whether players had clutch abilities or not.

As I said, it is very simple to add clutch ability and very difficult to determine whether clutch ability existed just looking at the data.

This "clutch situations" idea has been the core of much of the studies done in real life data and generally found to not be helpful in finding clutch performance. Hence all that one has been able to say is that if clutch performance exists it's either too small to be found in the results or is not manifested in clutch situations as we have defined them in the past.

[BPS]

Originally Posted by RonCo

This would be discoverable in something like OOTP merely by setting all players other skills to neutral and running a hundred seasons or so, then doing a study like I described. It takes work, but is able to be discovered.

But, yes, by definition if you make the variance you code extremely tiny, then you can implement anything and have it hard to feel in everyday play. This is the scenario by which its impossible to prove something doesn't exist without very large samples. All you can say in real life is that the impact is so small it's hard to find without more sample. The key here is that it is so small it is not affecting the outcome in any perceivable way. Operationally, some people would then say it does not exist, which is fair enough even if it's wrong.

Discounting, perhaps, some more advanced deep-learning algorithms, my view is that if you code it in software, I will eventually be able to find it in results if I care to spend the time looking.

I can, for example, see influences of managers and coaches in OOTP data, though I haven't gone the extra step to quantify them. It's hard work. Tedious and somewhat complex. But easily doable with enough motivation.

You're returning you YOUR concern: can you do something to discover the model OOTP uses to generate data.

MY concern is the following:
You have a set of data generated by a sim. You cannot modify the sim or run it multiple times. You have the data you have.

It might be impossible to determine which of two models generated the data: (1) a model with no clutch ability OR (2) a model with clutch ability. The data generated by these two models could be so similar (in a small sample setting) that you can't use the data to reveal what model generated it

This is relevant to the discussion of clutch hitting in the real world: you have data and the data very likely does not permit you to see if real world players have clutch abilities or not.

In the real world, the magnitude of clutch ability is not huge. But it is possibly large enough it makes sense for managers to try to use clutch players in appropriate situations. But the data might make the magnitude of clutch hitting appear smaller "than it really is" because the relatively large effect of randomness can make it hard to see clutch hitting ability.

[BPS]

Originally Posted by RonCo

This "clutch situations" idea has been the core of much of the studies done in real life data and generally found to not be helpful in finding clutch performance. Hence all that one has been able to say is that if clutch performance exists it's either too small to be found in the results or is not manifested in clutch situations as we have defined them in the past.

Speaking narrowly about the study linked to at the beginning of this thread: this is a poorly conceived study that fails to give insight into the existence of clutch hitting. I've already indicated above my concerns about this study.

[RonCo]

By definition, any piece of data that is too small to determine the root of the outcome means you are unable to determine what skills are in place in the model. Assuming a 1-week sim, you would be unable to determine if, say, Power, was in the model. So, sure, it's easy to create a model in which small sample sizes hide an impact.

I think that's different from your original statement.

Or, if it's not different from your original statement then your original statement was so elementary that it's not particularly valuable in the conversation to my way of seeing things. Shrug.

[RonCo]

Originally Posted by BPS

Speaking narrowly about the study linked to at the beginning of this thread: this is a poorly conceived study that fails to give insight into the existence of clutch hitting. I've already indicated above my concerns about this study.

The history of this quest is littered with several expeditions.

Simply googling clutch hitting will provide many, many studies...some better conceived than others.

[RonCo]

Another way of saying this is that if you code a "clutch rating" and it shows up in the data enough to give you a feel of its existence, then it's effect has been coded too strongly.

The best way to code "clutch rating" in our world, then, might be to present it to the end user but not have it in any code...letting the human need to make patterns where none appear to exist do the rest.

[BPS]

Originally Posted by RonCo

By definition, any piece of data that is too small to determine the root of the outcome means you are unable to determine what skills are in place in the model. Assuming a 1-week sim, you would be unable to determine if, say, Power, was in the model. So, sure, it's easy to create a model in which small sample sizes hide an impact.

I think that's different from your original statement.

Or, if it's not different from your original statement then your original statement was so elementary that it's not particularly valuable in the conversation to my way of seeing things. Shrug.

You have two models, M1 and M2 that have important differences. You want to determine which model generated the data you have. But if M1 and M2 generate very similar data, the data can't speak about which model generated it. This is an important statistical point, and directly relevant to the existence of clutch hitting in the real world.

I've said this a couple of times. But it seems you are interested in other things. That's okay. People are interested in different things.

[RonCo]

Originally Posted by BPS

But if M1 and M2 generate very similar data, the data can't speak about which model generated it.

This is not correct with regard to software models. With software models as we know them, it's always possible to determine which model created which data set as long as you take enough data.

If you force yourself to study only a playoff series worth of data, though, then of course you will not be able to see anything in the two data sets. This is a trivial case, though, since you can say that regardless of whether we're talking about clutch hitting or power.

I've said this a couple of times. But it seems you are interested in other things. That's okay. People are interested in different things.

I don't understand your statement. You said it's easy to code "clutch" in the game in such a way as to be undiscoverable in the data. I don't think that's true unless you artificially constrain yourself to small data sets.

If you change your statement to "it's possible to code a game with clutch rating and then give you such a small sample set that you can't find it," I'll agree, but say: so what? That's like saying 1+1 = 2 to me.

Regardless, I don't think we're doing much of value here, so ...

[BPS]

Originally Posted by RonCo

This is not correct with regard to software models. With software models as we know them, it's always possible to determine which model created which data set as long as you take enough data.

I have been talking about statistical issues involved in model selection when the data you have is limited and given to you by someone else.

You want to talk about discovering the model within software when you have the ability to generate as much data as you'd like.

These are very different things. Only the first is relevant to thinking about the existence of clutch hitting in the real world.

[RonCo]

Originally Posted by BPS

I have been talking about statistical issues involved in model selection when the data you have is limited and given to you by someone else.

You want to talk about discovering the model within software when you have the ability to generate as much data as you'd like.

These are very different things. Only the first is relevant to thinking about the existence of clutch hitting in the real world.

Then, yes, by definition, it's always possible to code something that is too small to find in a small sample set.

[RonCo]

Your proposal, though, did not limit my ability to take as much data as I wanted.

Originally Posted by BPS

It would be easy to produce a baseball simulation in which some players were "clutch hitters" but those who "played" the simulation wouldn't be able to see it by a statistical analysis of the output of the simulation. The major role of randomness makes that possible.

[BPS]

Originally Posted by RonCo

Then, yes, by definition, it's always possible to code something that is too small to find in a small sample set.

Obviously true, but I think we can agree that, as an isolated statement, it is not particularly interesting.

Originally Posted by RonCo

Your proposal, though, did not limit my ability to take as much data as I wanted.

You can, of course, do whatever you want. But what you (hypothetically) propose doing has little to do with the argument I have been making.

[RonCo]

I was proposing that if you produced a baseball simulation in which some players were "clutch hitters" that I would be able to play the simulation and see it by a statistical analysis of the output of the simulation.

[BPS]

Originally Posted by RonCo

I was proposing that if you produced a baseball simulation in which some players were "clutch hitters" that I would be able to play the simulation and see it by a statistical analysis of the output of the simulation.

Maybe; maybe not. But whether you can has nothing to do with what I've been arguing.

[NoOne]

in a simulation it'd simply be a factor applied - like anything else.

maybe i'ts randomly between 80-1.00 or if they prescribe to the idea a person can over-produce not due to small sample randomness - .80-1.05 etc... (>1 at high end) @ any distribution curve to their hearts content - skewed or normal etc.

relative to situation their contact would be adjusted by this factor. *or power, stuff, control etc... all or some blah blah minutiea blah blah.

technically, something like that would be applied at any moment if realistic. no person is 100% consistent each time up even in seemingly perfect health. even normal biology ebbs and flows without injury

-- the only thing that wouldn't be realistic is a factor greater than max potential for that player (within dynamics of ootp -- ie you can go 5-10pts over over max potential for any particular rating, so that's techinically the "max potential" relative to ootp mechanics and not the value you see for "potential" in the editor -- lol which isn't rational either.. either it is a ceiling or it is not.. silly to have it go over.. jsut make the last 5-10 tougher to maintain and it remains more in tune with logic)

"max" aboive isn't about the ">max" 201 - 250 range.. but, rather relative to each player's potential

[Syd Thrift]

So, I would really like to see someone perform that BP study on a single OOTP universe. You don't get to run Monte Carlo-style simulations, you don't get to expand the schedule to 300 games a year, you don't even get to count late-inning stats from the 1880s, you just get the same body of data to try and determine whether or not clutch exists. Intuitively, people who don't remember Markus' conversation don't think it exists in the game, which I find a bit funny but also a bit interesting. It may be that folks just assume that the game will operate the same way they think real baseball operates (i.e. without clutch) and so their intuitions are colored by it, but it may also be that they're not intuitively seeing it the same way they don't see it express itself in real-world data.

I think it would be an interesting way to go, not because it would in any way prove that clutch exists in the real world, merely that the contrasting hypothesis that clutch exists and the fact that it's hard to impossible to see evidence for it with basic statistics can co-exist. Or there may be a marked difference you *can* actually see from player to player or observable player type to player type. Or Markus said he'd add clutch but then removed it when people raised a stink over it (which I don't think he ever said he did; what he said IIRC when people raised said stink - I think RonCo may have been one of the people who did so - was that the effect was small).