I thought the mlb roster set was using ZiPS?

[Neags23]

This isn't a complaint about the rosters at all. Overall, most of the opening day rosters look great as far as what players belong where. And the overall general ratings are okay. I think the roster team should be commended!

But I'm confused. I thought the mlb roster set used the ZiPS projection system for player ratings. From the newsletter:

Our brand new 2018 roster sets include all Opening Day MLB rosters as well as the complete minor league system down to rookie leagues, including independent leagues. All Major League (and over a thousand minor league) player ratings are based on Dan Szymborski’s popular ZiPS player projection system.

But none of the players I see have projected stats that match ZiPS. Some aren't really that close. Attached are a couple of examples, but literally none of them I've seen match.

Trout's ZiPS projected slash line: .290/.421/.590
Trout's OOTP projected slash line: .305/.442/.631

Julio Teheran's ZiPS projected ERA, FIP, and WAR: 4.35/4.58/1.7
Julio Teheran's OOTP projected ERA, FIP, and WAR: 4.09/4.26/2.5

What am I missing?

[Rain King]

I believe they use ZIPS as a base, but aren't entering the stats directly into the game. The rosters are created in a program outside of the game and then imported, so there are likely going to be small differences when looking in the editor. They also make some adjustments to try to get the teams to use the players as close to real-life as possible.

[RchW]

I'm no projections expert but 5% to 7% off does not seem way off to me. I see articles explaining projection misses all the time on the web.

[Neags23]

Originally Posted by RchW

I'm no projections expert but 5% to 7% off does not seem way off to me. I see articles explaining projection misses all the time on the web.

That's not really my point. All projection systems are generally within 5% to 7% of each other. I'm more asking why the game is advertised as using ZiPS when it seems clear that it's really not, or if there is an explanation of like "We started with ZiPS, then put our own spin on them." In which case... why mention ZiPS at all?

And yes, in these two examples, they're acceptably close. There are lots of players that are not.

Andrew Triggs, for example, has ZiPS projections of 4.38/4.18 & 1.4. OOTP projects as 3.51/3.62 & 4.5. That's definitely not "based" on ZiPS. It's not even close. And if you look through players, you'll find a lot like that.

[Lukas Berger]

Originally Posted by Neags23

That's not really my point. All projection systems are generally within 5% to 7% of each other. I'm more asking why the game is advertised as using ZiPS when it seems clear that it's really not, or if there is an explanation of like "We started with ZiPS, then put our own spin on them." In which case... why mention ZiPS at all?

And yes, in these two examples, they're acceptably close. There are lots of players that are not.

Andrew Triggs, for example, has ZiPS projections of 4.38/4.18 & 1.4. OOTP projects as 3.51/3.62 & 4.5. That's definitely not "based" on ZiPS. It's not even close. And if you look through players, you'll find a lot like that.

It is based on ZIPS, but then we make changes to get players to fit on the rosters, start on the right levels and play in their right roles.

f.e. Triggs just based on his ZIPS projections and nothing else started off in AA in OOTP iirc.

So we use ZIPS as the base, but it's not purely ZIPS, because if we used nothing but pure ZIPS projections, the rosters would look incredibly weird and just not be right. We'll also sometimes make changes if guys consistently over/under-perform in our test seasons.

Another factor is that I believe the ZIPS projections we used we got directly from Dan and they were an earlier version that what you see on Fangraphs f.e. So a lot of the changes are just updates that Dan made to his projections after we'd already imported them into OOTP.

Also, as mentioned above, the ratings also come out slightly different just because of how they're imported into the game as well. That's especially true for pitchers.

Then there's the issue that the pitcher's projections are influenced by the gb% and how we calculate that is quite a bit different than how some other places do. So the out of the box pitcher projections just won't match 100% perfectly.

[Neags23]

Originally Posted by Lukas Berger

It is based on ZIPS, but then we make changes to get players to fit on the rosters, start on the right levels and play in their right roles.

....

Thank you for the explanation! Would this roster creation program be accessible by customers, say if I wanted to make a roster set that does closely match ZiPS or PECOTA or some other projection system?

It seems to me that simulating the current season would be just a little bit off if there are a number of players like Mr. Triggs who have inflated ratings just so the AI is comfortable putting them on a ML roster.

I'm sure on the whole some players are inflated while others are deflated and everything works out on a macro level. I'm interested in more micro level matchups, though, that get thrown pretty out of whack with such discrepancies. And as I'm sure anyone on the roster team knows, going through the editor for each player to try to get accurate ratings from projections is a very labor-intensive task.

[Neags23]

Or even better, could a roster set be released that does have the original ZiPS projection, regardless of where the AI puts them? I'd rather rearrange rosters than have to input stats to get appropriate ratings.

[NoOne]

it would require your Stats and AI settings to be perfectly aligned with 2018 results too.. which is impossible to project accurately.

so even if ratings are perfectly callibrated, resutls will depend on settings that control statistical environment in game too.

[Lukas Berger]

Originally Posted by NoOne

it would require your Stats and AI settings to be perfectly aligned with 2018 results too.. which is impossible to project accurately.

so even if ratings are perfectly callibrated, resutls will depend on settings that control statistical environment in game too.

Yeah exactly.

I wouldn't say the current season will be any more or less off though, since ZIPS is just a projection, and there's nothing to say the ZIPS projections are more accurate than the hybrid results we're using. Most likely they'll be more accurate in some cases and less accurate in others.

Plus we don't have rosters with the original ZIPS ratings anyway.

So even just from a practical standpoint I don't really see any way we could do this other than just going through and hand-importing the ZIPS ratings again (which would take too long and have too little interest I think to make it worth it. Might be a good idea for a mod though).

Even then, in that case the results will still not be accurate since the players ZIPS doesn't rate will still be in the roster set with their non-ZIPS ratings anyway and while Triggs is in AA it's possible someone pretty random like Trey Cochran-Gill might get called up instead of him at some point in the season, especially once the ai starts giving out ratings increases and decreases.