Starters v Relievers

[RonCo]

I got interested in the theory presented elsewhere on this board that OOTP does not create relief pitchers who are as good as starters...or visa versa--the term "good" being defined by me here as based on talents and ratings rather than actual statistical output. So I extracted data from three leagues--two v6.12 leagues, and one v5.14d league. All three of these leagues are active on-line leagues. One v6 league is fictional, as is the v5 league. The second v6 league uses real players with rookies imported from outside rosters.

The data I took are snapshots of all pitchers at their current status.

Using this data I ran correlation studies to determine if there were a relationship between a pitcher's role/endurance rating, and their primary pitching ratings and talents.

Here is what I found:

League 1: V6

Correlations: PRat OvStuff, PRat OvControl, PRat Movement, PRat OvEndurance

              PRat OvStuff  PRat OvContr  PRat Movemen
PRat OvEndur         0.028         0.012         0.025
                     0.304         0.653         0.358  <- P-value


Correlations: PTal Stuff, PTal Control, PTal Move, PRat OvEndurance 

                PTal Stuff  PTal Control     PTal Move
PRat OvEndur         0.034         0.045         0.061
                     0.204         0.093         0.024  <- P-value

Cell Contents: Pearson correlation
P-Value

The general rule of thumb says that if the p-value is less than .05, than a relationship exists, and that the relationship's strength is the Pearson correlation. A Pearson correlation of 1.0 is very strong. A Pearson correlation approaching 0 is very weak.

This data says that in this v6 league, there is no correlation between a pitcher's endurance rating and his current overall Stuff, Control, and Movement performance ratings. The data shows a very, very weak correlation between a pitcher's endurance rating and his Movement Talent, but no other correlations with Talent were identified.

This says that there is not much, if any, discrepancy in the quality of starters and relievers in this league.

League 2: v6

Correlations: PRat OvStuff, PRat OvControl, PRat Movement, PRat OvEndurance

              PRat OvStuff  PRat OvContr  PRat Movemen
PRat OvEndur        -0.062         0.084        -0.039
                     0.045         0.007         0.212

Correlations: PRat OvEndurance, PTal Stuff, PTal Control, PTal Move 

              PRat OvEndur    PTal Stuff  PTal Control
              	PTal Stuff  PTal Control  PTal Move
PRat OvEndur        -0.017         0.115         0.021
                     0.581         0.000         0.502

Cell Contents: Pearson correlation
P-Value

This data shows that for this league, there is a relationship between pitcher's endurance rating and their current ratings for Stuff and Control. It finds no relationship between endurance and the current Movement ratings. Note, however, that the magnitude of the relationship is very small--below .1, which is essentially negligible. Of particular note to this study, the correlation of endurance to Stuff is inverse, meaning that relievers in this league (players with lower endurance) tend to have better Stuff ratings than starting pitchers (who have higher endurance).

The data also shows there is a small correlation between a pitcher's endurance rating and his control rating.

I should note, also, that this league imports rookies from an established roster set. Hence the player's starting point is not defined by the OOTP engine, but is instead influenced by human perception. This alone would cast some question on the data.

League 3: V5

Correlations: PRat RHits, PRat RD, PRat RHR, PRat RBB, PRat RSo(80), ...

                PRat RHits       PRat RD      PRat RHR      PRat RBB	Prat RSo

Dur Start           -0.057        -0.018        -0.032        -0.032	-0.032
                     0.053         0.553         0.271         0.280	0.271

Dur Rel              0.028         0.008         0.013         0.010	-.027
                     0.347         0.793         0.659         0.733	-.362


Correlations: PTal Hits, PTal D, PTal HR(90), PTal BB, PTal SO, Dur Start, ... 

Dur Start         -0.034        0.010        0.004       -0.020       -0.000
                   0.255        0.726        0.890        0.489        0.987

Dur Rel            0.031        0.007        0.030        0.031        0.022
                   0.301        0.815        0.305        0.291        0.465

Cell Contents: Pearson correlation
P-Value

This data shows that for this v5 league, no statistically significant correlations exist whatsoever between a pitchers Duration Ratings and his current performance ratings or talents (all p-values are greater than .05). I was a little concerned about the fact that OOTP v5 provides separate duration ratings, so I also ran the v5 league through a correlation with "Role."

Correlations: PRat Runs, PRat OvHits, PRat OvD(70), PRat OvHR, PRat OvBB, ...

                 PRat Runs   PRat OvHits  PRat OvD(70)     PRat OvHR	Prat OvBB
Role(100)            0.074         0.061         0.018         0.033	0.049
                     0.012         0.037         0.540         0.264	0.099

Correlations: PTal Hits, PTal D, PTal HR(90), PTal BB, PTal SO, Role(100) 

               PTal Hits       PTal D  PTal HR(90)      PTal BB      PTal SO
Role(100)          0.042       -0.006        0.024        0.041       -0.006
                   0.156        0.841        0.410        0.166        0.844

The re-run confirmed that there was no correlation between a pitcher's role and their pitching talents. It found a statistically significant correlation between a pitcher's role and his current performance ratings for "Avoid Runs" and "Avoid Hits." However, these correlations were very small, with magnitudes below .080.

SUMMARY:

The data is pretty conclusive that neither OOTP 5 of OOTP 6 created relief pitchers and starting pitchers of different quality when measured by ratings and talents. This strongly suggests to me that if a league is seeing performance variance (relievers perform worse than starters, or starters perform worse than relievers), that the issue is related to something other than player quality--perhaps random variance, or OOTP in-game AI flaws, or perhaps even a figment of selective human memory (i.e. the reported occurrences are unique instances, and the theory isn't actually true when looked at across an entire league).

[andymac]

For anyone who subscribes to Baseball Prospectus, there is an article today about how major league closers very rarely were minor league closers (or relievers at all for that matter). I think this type of thing probably extends to major league middle relievers as well. Should I start a thread campaigning for the removal of "pitcher distinctions"???

[kageboshi]

Originally Posted by andymac

For anyone who subscribes to Baseball Prospectus, there is an article today about how major league closers very rarely were minor league closers (or relievers at all for that matter). I think this type of thing probably extends to major league middle relievers as well. Should I start a thread campaigning for the removal of "pitcher distinctions"???

I'm all for that, the thing I dislike the most about version 6.5 is the fact that the whole starter-reliever thing is kind of unrealistic. There has to be a way to make it so that relievers end up with better ERA's ( and peripherals ) than starters in general. Also... ( I don't remember if they fixed this in ver 6.5, but I know it was this way in 6.0 ) sometimes AI undervalued a lot pitchers with decent endurance(let's say 65 ) which they had labeled as MR ... I remember I could get a bunch of aces for cheap just because in their teams they were labeled as MR's even though they had a decent enough endurance to start games.

[RonCo]

I've done some work, but haven't had a chance to write it up, that shows OOTP creates relievers that are as good or better ratings wise than starters (as noted above), but that these relievers don't perform as well in-game (which confirms the complaint of those who are complaining). This tells me that either there's a hard-coded bias against relievrs (which I doubt seriously), or that the AI is using them badly (which I suspect is the case merely from reading the logs and watching the AI work).

If my findings are true, this makes me more firmly in the camp that says if Markus can improve the AI, he will resolve the complaints.

[andymac]

Originally Posted by RonCo

If my findings are true, this makes me more firmly in the camp that says if Markus can improve the AI, he will resolve the complaints.

Well, I think that would still leave the problem of endurance. Relief pitchers aren't relief pitchers because they can't throw 100 pitches. It is other reasons that lead to them being relievers.

[witwerk8267]

This is a good thread.

[jarmenia]

Originally Posted by RonCo

SUMMARY:

The data is pretty conclusive that neither OOTP 5 of OOTP 6 created relief pitchers and starting pitchers of different quality when measured by ratings and talents. This strongly suggests to me that if a league is seeing performance variance (relievers perform worse than starters, or starters perform worse than relievers), that the issue is related to something other than player quality--perhaps random variance, or OOTP in-game AI flaws, or perhaps even a figment of selective human memory (i.e. the reported occurrences are unique instances, and the theory isn't actually true when looked at across an entire league).

All you've proven is that there is no correlation between skill and endurance. Try running the same test comparing starters and MR.

[draven085]

Originally Posted by andymac

For anyone who subscribes to Baseball Prospectus, there is an article today about how major league closers very rarely were minor league closers (or relievers at all for that matter). I think this type of thing probably extends to major league middle relievers as well. Should I start a thread campaigning for the removal of "pitcher distinctions"???

See post #6 in this thread. RMPL, meat!

[RonCo]

Originally Posted by jarmenia

All you've proven is that there is no correlation between skill and endurance. Try running the same test comparing starters and MR.

The "role" study at the very end of my post is a v5 study of SP/MR/CL, which proves no correlation between them and ratings.

I didn't do that in v6 because there is only 1 endurance rating, so I assumed (perhaps wrongly) since a majority of starters are going to be 6 endurance and above and a majority of reliever will be lower endurance, hence the correlation on endurance was enough. I should probably go back and do it specifically on role for v6, also, though I would be surprised if it showed a difference. Still, I've been surprised before.

[RonCo]

Originally Posted by andymac

Well, I think that would still leave the problem of endurance. Relief pitchers aren't relief pitchers because they can't throw 100 pitches. It is other reasons that lead to them being relievers.

I agree. A model based on endurance is flawed and should be changed. But the model is the model.

[RonCo]

Originally Posted by jarmenia

All you've proven is that there is no correlation between skill and endurance. Try running the same test comparing starters and MR.

Data for League 1, above: (v6.12), based on role rather than duration

Correlations: PTal Stuff, PTal Control, PTal Move, Role(100) 

                PTal Stuff  PTal Control     PTal Move

Role(100)            0.014        -0.028        -0.033 
                     0.595         0.298         0.219 <- p-value

Since the p-value is > .05, this study finds no relationship between a pitcher's role and his talent--meaning that for this league, relievers as a group are no worse or better than starters. This league uses players generated by the OOTP engine.

[RonCo]

Data for League 2 above: (v6.12) based on role rather than duration

Correlations: PTal Stuff, PTal Control, PTal Move, Role(100) 

                PTal Stuff  PTal Control     PTal Move
Role(100)            0.047        -0.045         0.044
                     0.129         0.150         0.154 <- p-value

Again, the p-values are greater than .05, hence no relationship. These p-values are _closer_ to the cut-off than those of League 1, which makes me wonder if the fact that League 2 imports rookies from third-party roster makers makes a difference here. I expect that it does.

Bottom line, based on these three leagues, I would challenge the statement that OOTP does not create enough good relief pitchers relative to starters.

The fact that relief pitchers' ERAs do not show the same dip in some leagues as seen in the real MLB world suggests something faulty in the game engine or in the AI. My bet is on the AI, but that is again merely an opinion framed from years of reading logs and gnashing my teeth over odd decisions that I wouldn't have made in various situations.

[RonCo]

Dola again,

I should note that there exists a possibility that three total leagues is not a large enough sample size to make any qualitative statement, but the total span is roughly 1,000 pitchers/league. Roughly 3,000 total (or 2,000 if you want to discount the league the imports its players).

[tangotiger]

Hi,

I'm the author of The Book, and I did the study on the starter/relief. If someone has any questions, I'd be happy to answer them.

Tom

[blgoblue2]

Are you from myspace?

[GForce]

Originally Posted by tangotiger

Hi,

I'm the author of The Book, and I did the study on the starter/relief. If someone has any questions, I'd be happy to answer them.

Tom

Great to see you here, Tom. Been a couple years since we've talked, but your knowledge should prove an asset to the community.

GH

[Gastric ReFlux]

I'm gonna bump this back up, I collected these numbers from the sample reports we got:

Team		Starters	Bullpen
Fresno		3.26		2.98
Plano		3.35		4.07
Okla City	3.14		4.91
KC		4.24		5.23

Houston		3.47		4.67
Montgomery	4.39		4.16
Oakland		3.71		4.40
Indianapolis	4.23		4.28

Virginia Beach	4.50		4.02
El Paso		4.89 		4.08
Chicago 	4.84		4.47
Jax		5.71		4.57

Arlington	4.06		4.94
LA		4.69		4.43
Denver		5.19		4.50
Tucson		4.42		4.96

Does anyone know where to find numbers from actual major league splits like these?

[Stu]

Originally Posted by Gastric ReFlux

I'm gonna bump this back up, I collected these numbers from the sample reports we got:


Does anyone know where to find numbers from actual major league splits like these?

On ESPN's team stats you can filter them "as a Starter" and "as a Reliever"

http://sports.espn.go.com/mlb/stats/...28&season=2005

[Gastric ReFlux]

Thank you, just glancing at that it appears that relief pitching ERAs should be slightly lower than starting pitching ERAs.

Relief pitching 2005, low 2.80, high 5.40, middle (15th) 3.98
Starting 2005, low 3.46, high 6.00, middle (15th) 4.20

[Stu]

For 2005:

EDIT - Added a diff column that shows starter - reliever

Only 4 teams had a significantly better starters ERA than Bullpen ERA. (arizon, florida, boston, atlanta)

team	starter	reliever	diff
kc	6.00	4.70	1.30
tb	5.62	4.99	0.63
tex	5.04	4.83	0.21
ari	4.58	5.40	-0.82
cin	5.38	4.75	0.63
fla	3.81	4.85	-1.04
col	5.30	4.80	0.50
bal	4.82	4.10	0.72
bos	4.56	5.15	-0.59
pit	4.63	4.06	0.57
phi	4.20	4.24	-0.04
nyy	4.59	4.37	0.22
lad	4.36	4.42	-0.06
sf	4.52	3.98	0.54
atl	3.65	4.74	-1.09
tor	4.20	3.81	0.39
sd	4.49	3.49	1.00
chc	4.17	4.24	-0.07
was	4.03	3.55	0.48
mil	4.02	3.87	0.15
det	4.85	3.78	1.07
sea	4.91	3.60	1.31
oak	3.82	3.39	0.43
nym	3.71	3.86	-0.15
laa	3.75	3.52	0.23
hou	3.46	3.63	-0.17
min	3.93	3.20	0.73
stl	3.61	3.17	0.44
chw	3.75	3.23	0.52
cle	3.96	2.80	1.16