A Post Regarding Park Factors, Neutralizing OPS, and Something That Just Doesn't Work Right

[Buane]

Disclaimer: this post is really long and probably pretty dull, but there are pretty graphs and one big honkin' strange discovery at the end.


Hello OOTPdev forums. I'd like to share some data with you that I've collected and analyzed over the last 24 hours. But first some explanation: I've been playing OOTP since version 6. I'm a member of several online leagues, as well as Commissioner of an online league since the release of OOTP9. For one of the leagues that I'm in I've developed "Power Ranking" charts using MS Excel. These charts compare teams to the rest of the league in the three main areas of the game: pitching, hitting, and defense. It's a fun way for people to look at areas of their team that they need to improve in, as well as where certain areas of their team stand relative to the rest of the league. Now, over the seasons I've refined my rankings more and more. I've added formulas to help neutralize park effects, creating ballpark-adjusted FIP, OPS and Defensive numbers to (in my biased opinion) great success.

Anyway, fast forward to yesterday. I decided to run some tests on Park Factors. When I'd first started creating my ballpark-adjusted charts, I had to guess at the factors. The OOTP manual clearly stated that ballpark factors were not simple percentages - which is to say, a park with a HR factor of 1.100 does not simply yield 10% more HRs. I accepted this, and simply made educated guesses at the factor's effects based on logic and anecdotal evidence. Well, now it was time to get serious. I was going to finally run a series of actual tests to determine just how many points I should adjust a team's OPS if that team played in an extreme hitter's park.

So I set up a league to test park factors in. I created a two team league, with two teams that were exact duplicates of each other. Every hitter in the league had the same ratings, the same speed, the same defensive ability. Every pitcher in the league the same ratings, the same pitches, the same velocity. The players all even had the same birthdate. I turned off injuries, I turned off player development, I turned off player fatigue. I turned team strategies all the way down, so I could minimize things like stolen bases, hit-and-runs, and anything else that could affect the numbers I was trying to gauge too much. I set the schedule to 200 games; a nice round number with which to play in. I put both teams in domed stadiums so wind and weather would not play a part in the outcomes. I basically created a test tube league, controlled as best as I could.

The only thing I would change would be the park factors. I ran one season with all park factors at 1.000. I'd copy the league totals to my excel spreadsheet, end the ootpx process through windows, reload my league at the beginning of the season, change the park factors for both teams' parks, and run another season. This was the fastest way I could think to run the 100+ seasons that I'd need to run to mine all the data I'd need.

I started with the HR factor. Starting at 1.000 I increased it to 1.1 for the second sim, then 1.2 for the next, then 1.3 and 1.4. Then I went the other way, decreasing the HR factor to .9, .8, .7, and .6. Then, I'd fill in the middle of my data, setting the HR factor to 1.05, 1.15, 1.25, etc. You get the idea. After running seventeen sims, I had league data for seventeen different HR factor settings.

Now, ideally you'd want thousands, if not tens of thousands of games of data for each different factor. So my results won't be the most accurate results in the world, but they should tell a pretty decent story, and I still should be able to pull some meaningful numbers and conclusions from them.

Anyway, my HR data looked like this:



Charting Batters Faced per Home Run for the entire league, this was what my numbers came up with. Note, this is not a chart of total HRs, but rather a chart of HR rate. That's important to keep in mind.

Now, the first thing you'll notice is that the trend line is not a straight line. As the HR factor increased more and more, the effect on HR rate became less and less. It was approaching a point of diminished returns. Meanwhile, when the HR factor shrunk, it continued at about a constant pace. I won't bore you with the mathematics, but the formula I derived for normalizing a park's HR rate numbers was (1/((HRF^2)+(HRF)))*2 with HRF being the Home Run Factor. Let's apply this formula to a hypothetical park with a 1.100 HR factor. Plugging 1.1 into the formula gives us 0.865800866, meaning we'd have to multiply the homerun rate in that park by about .86 to "neutralize" them, or show about how much more infrequent that same team would hit HRs in a park with a HR factor of 1.000.

Given that my test league's HR rate when I tested a 1.1 HR factor were 24.5 BF/HR, and 28.9 BF/HR when I tested a 1.0 HR factor - and 24.5 is about 85% of 28.9 - I was pretty damn pleased with my formula's apparent accuracy.

But that's not all. I dug a little deeper into my HR data to analyze the effect it seemed to have on OBP.



Hmm. Changes in HR factor also seemed to have an effect on OBP. But the data seemed a little wild, likely due to random BABIP fluctuations within each season. So I normalized hit totals with a standard BABIP, and constructed a new chart for normalized OBP.



Not perfect, but looks a bit cleaner. The point is there's a definite correlation here between HR rate and OBP. Now, obviously, the effect that more HRs have on OBP is that the Batting Average goes up. Here's that same chart with neutralized AVG (nAVG) graphed on it.



The change in nOBP is almost wholly due to the change in nAVG. However, AVG is not a component of OPS, and since I'm trying to create a park-neutralized OPS, I don't care about AVG (I just didn't want anyone thinking I was saying that more HRs were causing more BBs).

Anyway, unlike the HR factor's effect on HRs, the effect on OBP was practically linear. I felt fine assigning a linear function that adjusted team OBP based on the HR park factor. I (roughly) determined that every .001 change in the HR park factor was equal to a .00004 change in OBP. So a team with a .350 OBP in a park with a 1.1 HR factor should have their OBP dropped to .346.

Having completed my analysis of the HR factor, I moved on to doubles. Once again, I graphed the rate of doubles compared to the different park factors.



An interesting graph. The effect is very smooth on the period 1.200 to .800. But at both 1.250 and .750 things seem to go haywire. This may be a game engine issue, or just a small sample size issue. Still, it's smooth enough for me. After a few calculations, I settle on the formula of 1+((1-DF)*0.8) to adjust a team's double totals, where DF is the park's Doubles Factor.

At this point I get lazy. I decide that instead of running tests for triples at this point, I'll just duplicate my doubles adjustment. Maybe I'll run a test for the Triples factor at a later point, but for now I just want to check out the Average park factor results.

Now, at this point, I think it's safe to say that while my data doesn't have the grandest scope, or the greatest sample size to be as accurate as it could be, it's been pretty good at showing trends. It's also allowed me to make some pretty decently accurate formulas for Park Neutralizing purposes.

I'm bringing this up because I want to emphasize that while the data's not perfect, it's not wrong either.


So with that in mind, here's where I discover that the AVG park factor is completely and totally not working correctly. At all.

Now, I wasn't exactly sure what increasing the AVG park factor would affect. Would it make BABIP go up? Would it increase singles? Would it just increase all hits? I didn't know what to look for in the data. So, first I took a look at the effect on BABIP.



As you can see, there's almost no correlation between changing the AVG ballpark factor and the resulting BABIP. The chart just shows normal BABIP fluctuation. So, I decide that effect that AVG has must be in Batting Average itself. Knowing that I've already done for the other factors, I graph a BABIP-neutralized Batting Average (nAVG) and graph it against the changes in the AVG park factor.



Ok, so when I see this, I'm convinced I did something wrong. I check over the math, double check the results...nope, it all looks ok. But this graph shows that changing the AVG ballpark factor has no effect on batting average at all? How can that be true? Begrudgingly, I run 17 entirely new tests on the AVG factor, convinced my original data was wrong. Of course, I get perfectly similar results from the second set of 17 seasons.

Changing the AVG factor in a ballpark has absolutely, positively, no effect on Batting Average. Not even a slight, casual effect. Just none whatsoever.

But the fun doesn't end there. Checking out the same AVG data, I see another pattern.



Ok, so two questions arise at this point. One: why does changing a park's AVG affect how many doubles are hit? Two: why the heck is this graph sloping the wrong way?

So now we come to the big finish. If my data is correct, and I have every reason to believe it is, then changing a ballpark's AVG modifier has absolutely no effect on Batting Average, and instead has a reverse-effect on Doubles. So if you set your ballpark to a 1.300 AVG, your team won't hit for a higher average but will instead hit a lot fewer doubles. I can't explain it, but it's very clearly what the data is saying. These aren't subtle trend lines either...they're quite pronounced.

Going a little further, it's not even as if the AVG factor is simply behaving like the 2B factor, which is what I guessed at first. While the data I have from increasing a park's 2B factor shows that increasing doubles has no effect on that park's triples, the data I have here shows that decreasing a park's AVG boosts not only doubles, but triples too.



This is a graph of PAs per Triple. The Red points are graphed against changes in the 2B Park Factor. The Blue points are graphed against changes in the AVG Park Factor. As you can see, there's really no trend that suggests increasing a park's doubles also increases its triples. They seem to operate separately. However, changes in a park's AVG factor very clearly do affect both doubles and triples (in reverse, it's worth mentioning again).



Ok. Sheesh. That's a lot of words, graphs, and incorrect usages of affect/effect. Sorry, but it's late. If you got tired of reading and just skipped to the end, I'll try to sum up:

I collected a lot of data regarding park factors, gathered in a special test-league environment. My data, which proved to be pretty accurate when testing Homeruns, Doubles, and most other statistics, strongly suggests that changing a park's AVG factor does not affect batting average in that park in any way whatsoever. Changing a park's AVG factor does however affect doubles and triples, bizarrely, and - perhaps even more bizarrely - in a negative relationship.

[Athletics17]

Have you been researching 24hrs straight.

[Buane]

Originally Posted by Athletics17

Have you been researching 24hrs straight.

No but that post seemed to take about that long.

[William Foster]

Nice post very informative and probably valuable to the development team if they ever see it. I did a graph a while ago with OOTP data but I never finished it or planned to make it public.

[Questdog]

Originally Posted by Buane

No but that post seemed to take about that long.

Interesting stuff and it seems to me that not enough of this is done by the ones who should be doing it.....

Anyway, a few questions.

Ball park factors for Average are usuaully between .950 and 1.050. Anything outside this range would be considered an extreme bias, though some parks have managed to best these extremes. But no parks have ever come close to hitting .600 or 1.400. Is it possible that these extreme factors are messing up the curve in your study? I mean, I would not hold it agianst the game engine to not be able to model a Ball Park factor of 1.400 for Batting Average. For that to happen in real life, a .250 hitter everywhere else should be a .350 hitter in this park. And a factor of .600 would mean a .300 hitter only makes it to .180.....

But I will admit that I have wondered about this issue my self, having seen data in my leagues that seemed odd.

Also, is it possible that your model player is skewing the results somehow? You haven't told us what his ratings are, however I can think of an explanation for the flattening of the curve at higher ball factors for homers, for example, if the model player has a high power rating. Boosting his rating will eventually max it out and any further boosting would be for naught. This wouldn't mean that the model is broken however.....

Not saying, but just wondering.....

[Buane]

Originally Posted by Questdog

Interesting stuff and it seems to me that not enough of this is done by the ones who should be doing it.....

Anyway, a few questions.

Ball park factors for Average are usuaully between .950 and 1.050. Anything outside this range would be considered an extreme bias, though some parks have managed to best these extremes. But no parks have ever come close to hitting .600 or 1.400. Is it possible that these extreme factors are messing up the curve in your study? I mean, I would not hold it agianst the game engine to not be able to model a Ball Park factor of 1.400 for Batting Average. For that to happen in real life, a .250 hitter everywhere else should be a .350 hitter in this park. And a factor of .600 would mean a .300 hitter only makes it to .180.....

Well, I may be wrong, but Ballpark Factors in OOTP and Ballpark Factors in Real Life are not the same thing. You can go to ESPN.com and check out the ballpark factors for any park this year, or in previous years, and it will give you a straight % number.

In OOTP, however, that is not the case. From the Manual:

Originally Posted by The Manual

So, for example, if your AVG Overall factor is 1.100, you can expect that if you had identical players in this park and a neutral park, the player in the park with the 1.100 AVG Overall factor would have a slightly higher average. The modifiers are not straight percentages. So, a 2.000 doesn't mean you will do "twice as well."

So a factor of 1.4 in OOTP does not mean 140% better, and .6 does not mean 60%. That was one of my motivations for performing these tests to begin with - to figure out just what the effects were so I could better design a ballpark-neutral power ranking.

Originally Posted by Questdog

Also, is it possible that your model player is skewing the results somehow? You haven't told us what his ratings are, however I can think of an explanation for the flattening of the curve at higher ball factors for homers, for example, if the model player has a high power rating. Boosting his rating will eventually max it out and any further boosting would be for naught. This wouldn't mean that the model is broken however.....

Not saying, but just wondering.....

Good question. All the batters in my league had ratings of 120/250 (That equates to a 61 on the 1-100 scale) in all areas, including speed, and their defensive abilities. Defensively, they were all given 200 experience at their position. I also played with a DH. Pitchers were given 120 ratings in Fastball, Curveball, Slider, and Changeup. Pitchers all threw 95-97 mph, were rated with 120 control, 130 movement, and 50% groundball.

All the players were average in every area, and in no area did anybody stand out.

[Questdog]

Originally Posted by Buane

Well, I may be wrong, but Ballpark Factors in OOTP and Ballpark Factors in Real Life are not the same thing. You can go to ESPN.com and check out the ballpark factors for any park this year, or in previous years, and it will give you a straight % number.

In OOTP, however, that is not the case. From the Manual:



So a factor of 1.4 in OOTP does not mean 140% better, and .6 does not mean 60%. That was one of my motivations for performing these tests to begin with - to figure out just what the effects were so I could better design a ballpark-neutral power ranking.

Well, that may be, but the park factors imported for historical replays are identical to what you'd expect from a conventional perspective. If they intended park factors to work differently, then you'd think they'd modify the park factors to correspond to the altered perspective.

[Buane]

Originally Posted by Questdog

Well, that may be, but the park factors imported for historical replays are identical to what you'd expect from a conventional perspective. If they intended park factors to work differently, then you'd think they'd modify the park factors to correspond to the altered perspective.

Good point. Nowhere in my tests did any park factor of 1.2 correspond to 120% of a league total when compared to a park factor of 1.0. This could very well be a case of one hand not knowing what the other is doing, since the manual pretty clearly states ootp's park factors aren't based on percentage (a statement my data backs up, again) while they certainly are here in the real world.

[Buane]

A thought occurred to me this morning: maybe the AVG factor for a ballpark works relative to the rest of the league? Maybe because there are only two ballparks in the league, and I set both of them to the same AVG factor, it considers them both to have an AVG factor of 1.0? It wasn't a wonderful theory (since obviously doubles, triples, and HR park factors don't work like that), but it wasn't hard to test out.

I ran my same test league, except I turned one ballpark up to 1.4 AVG and the other ballpark I left at 1.0. If nothing else, this should at least show some difference from my tests where I left both AVG factors at 1.0. It, of course, did not. The league's BABIP-neutralized AVG (nAVG) was exactly the same for this new test as it was for all the previous tests. So my new theory was incorrect.

This new test did, however, bring to light another issue. VORP is obviously being calculated incorrectly. Now, obviously, if the AVG factor is broken like I claim it is, then VORP is being awarded incorrectly based on the Average that different players maintain in different AVG-factor ballparks. I don't need to point that out.

However, remember, this is a league with two teams, and two stadiums. That means on my 200-game schedule, 100 games are played in one stadium, and 100 games in the other. Both players on both teams play in exactly the same environments relative to the entire season - in the case of this latest test, all players played the same number of games in the 1.4 AVG environment and the same number of games in the 1.0 AVG environment. So even if the AVG factor was working correctly, VORP should have been awarded equally to players from both teams, as there was no advantage or disadvantage on either side.

Well, not quite. Here's a scatter chart of each player's season stats from this test season, charted by VORP (x-axis) and OPS (y-axis).



As you can see, despite extremely similar OPS numbers from players on both teams, the difference in VORP was incredibly vast. Players with similar OPS numbers were being given 40-50 points more in VORP despite having no actual advantage because both teams played all their games in the same places.

This seems awfully broken to me. A player's VORP is being adjusted based only on his home park, and not the parks he plays in for the rest of the season?

[TribeFanInNC]

Interesting work Buane. I'm having a hard time digesting all of it while at work, but you're right I liked all the graphs.

This is right up RonCo's wheelhouse. Hopefully he'll see this and add to the conversation.

[CaptainObvious]

Can you get graphs of the following?
2B, 3B factors vs. PA/Hits
AVG factor vs. PA/singles, PA/doubles, PA/triples


My theory is that AVG affects just the likelihood of singles vs. other hits, rather than hits vs. PA (which is what you would think it would affect).

[Buane]

Originally Posted by CaptainObvious

Can you get graphs of the following?
2B, 3B factors vs. PA/Hits
AVG factor vs. PA/singles, PA/doubles, PA/triples


My theory is that AVG affects just the likelihood of singles vs. other hits, rather than hits vs. PA (which is what you would think it would affect).

This is an awesome theory, I'll crunch the numbers.

[Questdog]

Originally Posted by Buane

This seems awfully broken to me. A player's VORP is being adjusted based only on his home park, and not the parks he plays in for the rest of the season?

I like your work, but I disagree on the VORP issue. I don't know that VORP is being calcualted correctly overall, but to be upset that it is only being adjusted for the home park is unfair. By definition, Park Factors are relative to one another and valid Park Factors must add up to average 1.000 for a league. In real life, you could not have all parks in the league being 1.200 factors for HRs, for example. Now, granted, it could be that all parks are positively affecting home runs compared to the IDEAL neutral stadium, but this is not how park factors are calculated, and an IDEAL park factor is next to unknowable. Also, even if you fixed your test to make one park 1.200 and the other .800, the VORP calculations would be incorrect; however, in a normal league enviroment such a situation could never occur and to expect the VORP calculation to assume that the road parks are not close to 1.000 is unreasonable.

It's early for me and I'm afraid I am not lucent yet.....

[Buane]

Originally Posted by Questdog

I like your work, but I disagree on the VORP issue. I don't know that VORP is being calcualted correctly overall, but to be upset that it is only being adjusted for the home park is unfair. By definition, Park Factors are relative to one another and valid Park Factors must add up to average 1.000 for a league. In real life, you could not have all parks in the league being 1.200 factors for HRs, for example. Now, granted, it could be that all parks are positively affecting home runs compared to the IDEAL neutral stadium, but this is not how park factors are calculated, and an IDEAL park factor is next to unknowable. Also, even if you fixed your test to make one park 1.200 and the other .800, the VORP calculations would be incorrect; however, in a normal league enviroment such a situation could never occur and to expect the VORP calculation to assume that the road parks are not close to 1.000 is unreasonable.

It's early for me and I'm afraid I am not lucent yet.....

True, perhaps "awfully broken" was a bit strong. More like, potentially inaccurate. Obviously my test league doesn't mirror anything close to an average league. But that doesn't mean it still can't cause an issue in a real life.

For example, I'm Commissioner of an OOTP league. In one of our divisions last season there were five teams, four of whom played in pretty significant hitters ballparks. Our schedule is unbalanced, so that inter-divison play is more common - 23 games against division foes versus 14 games against intra-division foes.

Therefore, the team in that division who didn't play in a hitter-friendly ballpark would have had its batters VORP numbers artificially inflated, since the game only credited them for playing in their home park - when in reality, they were playing a great number of games in some significant hitters parks. Conversely, that team's pitchers would have had their VORP numbers artificially deflated. The game was judging them based on their normal home park when they were actually pitching lots of games in hitters paradises.

So is it broken in my league to the extent of 50-60 VORP per player? Of course not, but seeing this data it will be impossible for me to think that there are teams out there whose players aren't getting inaccurate statistics.

Originally Posted by CaptainObvious

My theory is that AVG affects just the likelihood of singles vs. other hits, rather than hits vs. PA (which is what you would think it would affect).

This turned out to be an awesome theory. Here are some numbers, all measured against a change in Park AVG:

First, Plate Appearances per Double:



Next, Triples:



Ok, so it's pretty clear raising AVG has an effect on doubles and triples - namely, they become rarer. So what about singles? I graphed two sets of data: the first is raw singles numbers, and the other is BABIP-neutralized singles numbers (n1B):



I guess I really didn't need to neutralize the singles numbers, since they turned out to show practically identical trends. You'll notice that unlike the doubles and triples graphs, this graph has a positive slope. As AVG increases, doubles and triples become rarer, and singles become more common. Sounds like your theory is starting to hold water.

Here I graphed Plate Appearances per XBH. Despite HR numbers not really showing any sign of being affected by AVG changes, I wanted to see the numbers with them included as well.



So it's pretty clear. Extra-base hits are becoming less frequent as AVG increases. No doubt about it. Finally, I went for the jugular, and graphed PA/XBH as well as Plate Appearances per BABIP-neutralized Hits.



As you can plainly see, Extra Base Hits become less frequent as AVG-factor rises, and more frequent as AVG-factor drops, while Hits overall remain exactly the same.

I believe we've reached the crux of this issue, gentlemen.

[Questdog]

I hope that this is not how the park factors are intended to work......

[Jamee999]

Surely 2B, 3B and HR should be more correlated with the park than singles? How many singles do you get that just stay in the park?

[jasonn29tn14]

If the AVG factor isn't being used to determine batting averages there would be no way to simulate parks with large foul territory.

[Buane]

Also keep in mind parks with large outfields mean there's more ground to cover for outfielders, and more room for balls to drop in for base hits. There's every reason why the AVG Park Modifier should exist in OOTP, the problem is it's just not working correctly.

[risp2out]

Could one or more of you very smart gentlemen opine on the relatioship between park factors and league total modifiers. As Questdog mentioned earlier, park factors IRL are descriptive, not prescriptive - that is, they reflect how the league totals were distributed. Now, in OOTP, you have league totals saying "I want x number of home runs hit by everyone this season." Then, YOU set certain park factors saying "I want a factor of 1.2 homeruns hit in park Y and 1.0 in park Z, but let's not change those league totals." This seems problematic to me, but I'm not the brightest bulb in the lamp when it comes to topics statistical. Anyway, I'm interested to learn more about this.

[Questdog]

Originally Posted by Buane

Also keep in mind parks with large outfields mean there's more ground to cover for outfielders, and more room for balls to drop in for base hits. There's every reason why the AVG Park Modifier should exist in OOTP, the problem is it's just not working correctly.

I seems obvious that the AVG factor of a park MUST affect the BAPIP rating of the hitter.....

If it does not, then I don't see any way of explaining an alternative approach, since there seems to be no way to directly affect a batting average, except through the hitters BABIP rating, unless CONTACT rating is used for calculations in the game (though it be a calculated rating).....