How OOTP 2007 Models Real Baseball - Page 2

RonCo · 06-03-2007, 01:28 PM

At the end of the day, Elendil's consistency studies are effectively measuring the rate of _ratings_ change, which is only indirectly a result of talent change.

Elendil · 06-03-2007, 02:20 PM

Quote:

Originally Posted by RonCo

I should mention that I think Elendil's commentary about needing more talent changes in certain areas is not quite right. Yes, it could be made to work that way, but in order to get less consistency, we need more _ratings_ changes, not talent changes. In the current OOTP model, ratings follow talent, so perhaps that's saying that ratings need to grow or fade faster than they do in the current OOTP model. "Inconsistency" does not specifically mean the variance must necessarily be both positive and negative.

The consternation about talent changes in other areas of the forum are that OOTP talents go down a whole lot, and up almost never. So this dynamic is not being used to induce inconsistency. We worked hard on the basic growing and fading of each skill in v2007, and I think that work is showing well. But v2008 can be made better by addressing the year-to-year consistency question, and I suggest that fine-tuning the growth process at the rating level, and improving the fielding model are the design items that will be most useful in this area. Oscillating talent will not do much to affect year-to-year stats consistency unless the ratings development rate would be made much, much more rapid, which would in turn cause many more issues.

Well, I agree that the real issue is getting ratings to change more frequently in particular areas. Currently, ratings rarely go down if potentials don't. That may be the thing to change.

However, in defense of the potential change as a general phenomenon in OOTP, I can see treating injury-free potential drops as simply updates in what we know about a player's "true" potential, rather than changes in that true potential. The problem with using the God's-eye view for potential is that when you play with scouts off, as I do, you'll have access to the God's-eye view, which is unrealistic. (Of course, you could play with potentials disabled, but that would be perhaps overly challenging.)

RonCo · 06-03-2007, 02:47 PM

Quote:

Originally Posted by Elendil

Well, I agree that the real issue is getting ratings to change more frequently in particular areas. Currently, ratings rarely go down if potentials don't. That may be the thing to change.

However, in defense of the potential change as a general phenomenon in OOTP, I can see treating injury-free potential drops as simply updates in what we know about a player's "true" potential, rather than changes in that true potential. The problem with using the God's-eye view for potential is that when you play with scouts off, as I do, you'll have access to the God's-eye view, which is unrealistic. (Of course, you could play with potentials disabled, but that would be perhaps overly challenging.)

With ratings off, you _do_ have a God's eye view. But OOTP's God changes its mind very often. My supposition is that Real Life's God doesn't change its mind often at all, just that some people don't happen to utilize all of their capabilities. An interesting side question: Does anyone actually reach their true potential? Probably not. Heck, I'm fairly sure that most superstars of the past would have said they could have been better than they ended up being.

Your approach to viewing injury-free talent drops as updates includes a pseudo-scouting function into the fundamental game engine, but one in which the scouts are always right about current status. Again, it works as a game engine, but it is not a very good model of reality.

Elendil · 06-03-2007, 02:51 PM

Quote:

Originally Posted by RonCo

Your approach to viewing injury-free talent drops as updates includes a pseudo-scouting function into the fundamental game engine, but one in which the scouts are always right about current status. Again, it works as a game engine, but it is not a very good model of reality.

But it might be the best we can do. If God didn't change his mind very often, then the only way to get a realistic perspective on players would be to use scouts, a part of the game that will probably always fail to appeal to some who see it as busy-work.

RonCo · 06-03-2007, 03:09 PM

Quote:

Originally Posted by Elendil

But it might be the best we can do. If God didn't change his mind very often, then the only way to get a realistic perspective on players would be to use scouts, a part of the game that will probably always fail to appeal to some who see it as busy-work.

We're on the same overall page as far as goals are concerned. If talents were made less volatile, we would need a more randomized ratings development algorithm--which I suggest we need anyway--and a process by which development to peak rating was not a guarantee. Put both of these into the game and you don't need to rely upon scouts totally--though a nice simple implementation of scouting's influence would still be great.

Elendil · 06-03-2007, 03:29 PM

Quote:

Originally Posted by RonCo

We're on the same overall page as far as goals are concerned. If talents were made less volatile, we would need a more randomized ratings development algorithm--which I suggest we need anyway--and a process by which development to peak rating was not a guarantee. Put both of these into the game and you don't need to rely upon scouts totally--though a nice simple implementation of scouting's influence would still be great.

Yup, more random ratings changes in the appropriate areas would do the trick, and I'd be comfortable with the God's-eye view of potential under those circumstances.

sporr · 06-03-2007, 04:14 PM

Nice work!

Zeyes · 06-03-2007, 04:38 PM

I'm not sure why introducing more inconsistency into the results necessarily requires an adjustment to how ratings are changed. That may be easiest to achieve, admittedly (by overlaying a sort-of random walk process on the regular ratings development mechanism for each player?), but couldn't the same effect also be achieved by introducing more variance into the "translation" from ratings to events as games are played out by the engine?

RonCo · 06-03-2007, 04:50 PM

I suppose the game engine itself could be made more random, but I would be reluctant to change the Log5 process currently employed, as that's been pretty well proven out.

LASpirit · 06-03-2007, 05:59 PM

Quote:

Originally Posted by RonCo

But v2008 can be made better by addressing the year-to-year consistency question, and I suggest that fine-tuning the growth process at the rating level, and improving the fielding model are the design items that will be most useful in this area.

Has anyone performed even an elementary comparison between OOTP and MLB fielding performances?

RonCo · 06-03-2007, 06:32 PM

Yes, indeed. I'll see if I can dig out some of the stuff I've done in the past.

LASpirit · 06-03-2007, 07:29 PM

RonCo, thanks. My primary reason for asking is that the sim league I'm involved with has created our own lahman style database using a weighted combination of prev year MLB, prev year sim, and upcoming year projections to create player models for OOTP. Hitting and Pitching stats are fairly easy using this method, but fielding are a bit more difficult. We'd like to get as accurate a model for the fielding as possible. To do this, we need to understand the strong points and limitations of the OOTP fielding performance. Any input you have would be very helpful. Thx.

kroney37 · 06-03-2007, 08:23 PM

What program were you using Elendil? (Or for that matter, anyone else that's done work here). I've been using SPSS, but I'm always interested in different stats packages.

RonCo · 06-03-2007, 08:48 PM

I use Minitab for running statistics analysis--and Excel, of course.

Elendil · 06-03-2007, 09:22 PM

I use Stata. I actually have the "do" file written up for anyone else who wants to process their own data in Stata.

kroney37 · 06-03-2007, 11:44 PM

Darn... I was hoping for more SPSS people - doing some basic stuff with stolen bases & ratings and looking for people more skilled than I.

darkcloud4579 · 06-04-2007, 02:11 AM

Looks very good. Great work.

His Own Bad Self · 06-04-2007, 04:27 AM

Elendil, fantastic work, as always.

It's been my impression that even in this version of OOTP, the best pitchers in any given league or dynasty will consistently have seasonal ERAs below 2. I've seen pitchers with double-digit strings of seasons of ERAs ranging from, say, 1.10 to 1.80 in generally neutral park and era settings. Of course I think there may be a need for some tweaking there. Can we take this problem - if it's in fact a problem - and trace it back to any of the factors you discussed at the beginning of this thread? My first guess is that pitcher K consistency is part of the problem, but beyond that, I'm not sure of any other connections.

Now that I think of it, these pitchers I'm referring to consistently seem to have very high K/BB ratios - generally 8:1 or higher over a long period of time. I get the impression that strikeouts and walks should be directly related to a small degree, and I don't see that as often as I'd like. Do you have time to test that at all?

StyxNCa · 06-04-2007, 09:15 AM

All you're formulas, etc is a language foreign to me. I'm totally lost.

RonCo · 06-04-2007, 09:21 AM

Quote:

Originally Posted by LASpirit

RonCo, thanks. My primary reason for asking is that the sim league I'm involved with has created our own lahman style database using a weighted combination of prev year MLB, prev year sim, and upcoming year projections to create player models for OOTP. Hitting and Pitching stats are fairly easy using this method, but fielding are a bit more difficult. We'd like to get as accurate a model for the fielding as possible. To do this, we need to understand the strong points and limitations of the OOTP fielding performance. Any input you have would be very helpful. Thx.

Here's all I've been able to dig up that's presentable. It's a bit out of date (from a build in late Feb), but if you consider it a "worst case" match, it's maybe useful.

A few notes...

1) I think the data spans 8 or 9 OOTP seasons
2) OOTP defenders were generally a little too good at the major league level at this point. I know Markus made some changes to try to bring this closer to "true." All I've compared in this data is fielding %. If I can find my Range Factor comparison, I'll post it. Some of the other guys did really nice studies of OOTP pbp to make sure that fielding chances by position were realistic, too.
3) The MLB baseline is, I think, the 2006 MLB season, but I have not gone back to verify this. Sorry for my sloppiness, but i'm running out of time this morning.
4) I do not have the minor league baselines.
5) I also have a runchart graphic that shows the model gets closer to "true" after the first 5 or so seasons, but I'm above my limit for posting attachments.

6) Level 0 is the MLB equivalent.

Code:

Level	pos	tc	a	po	er	ip	g	gs	e	dp	tp	pb	sba	rto	ipf	pct	RTO pct	MLB Pct
0	1	132156	95090	31729	0	576844	200237	64514	5337	6014	2	0	0	0	8470	0.9596		0.958
0	2	482567	24681	455365	279575	579911	68098	64575	2521	3235	0	2806	40785	12824	1139	0.9948	0.3144	0.992
0	3	631129	36242	591307	0	579600	69756	64550	3580	60555	38	0	0	0	1168	0.9943		0.994
0	4	322866	188481	129914	0	580252	72871	64637	4471	52417	38	0	0	0	1506	0.9862		0.982
0	5	181120	133628	40177	0	580061	72328	64620	7315	11018	18	0	0	0	1466	0.9596		0.954
0	6	305794	197573	101161	0	577594	73202	64324	7060	48188	8	0	0	0	1588	0.9769		0.972
0	7	122644	4097	116479	0	580096	71069	64624	2068	557	1	0	0	0	1692	0.9831		0.981
0	8	156848	3343	151894	0	579040	71994	64509	1611	653	4	0	0	0	1830	0.9897		0.9875
0	9	127636	4847	120881	0	579928	70597	64603	1908	683	3	0	0	0	1760	0.9851		0.981
1	1	90092	65052	21405	0	391539	140483	43760	3635	4041	1	0	0	0	4908	0.9597		
1	2	341254	16706	322862	191283	392956	46182	43760	1686	2117	0	1865	26422	8362	657	0.9951	0.3165	
1	3	423684	24529	396891	0	392948	46594	43760	2264	39466	24	0	0	0	681	0.9947		
1	4	215586	126288	86490	0	392882	48287	43760	2808	34077	24	0	0	0	879	0.9870		
1	5	120278	88859	26851	0	392879	48359	43760	4568	7028	15	0	0	0	886	0.9620		
1	6	203219	131812	67047	0	392864	49148	43760	4360	31682	5	0	0	0	935	0.9785		
1	7	81365	2671	77441	0	392843	47883	43760	1253	347	1	0	0	0	996	0.9846	
1	8	103242	2155	100115	0	392811	48798	43760	972	415	1	0	0	0	1092	0.9906	
1	9	83433	3145	79147	0	392807	47819	43760	1141	439	1	0	0	0	1104	0.9863	
2	1	8334	6000	2023	0	38401	12421	4308	311	364	0	0	0	0	625	0.9627	
2	2	34493	1605	32728	18584	38799	4464	4317	160	235	0	207	2816	921	80	0.9954	0.3271
2	3	40028	2302	37503	0	38356	4702	4268	223	3938	2	0	0	0	88	0.9944	
2	4	20712	12019	8311	0	38471	4806	4281	382	3455	3	0	0	0	115	0.9816	
2	5	11624	8476	2648	0	38682	4768	4304	500	756	1	0	0	0	92	0.9570	
2	6	19176	12262	6439	0	37840	4735	4205	475	3057	0	0	0	0	114	0.9752	
2	7	8223	273	7808	0	38808	4627	4319	142	47	0	0	0	0	111	0.9827	
2	8	10601	257	10225	0	38680	4644	4306	119	56	1	0	0	0	130	0.9888	
2	9	8543	328	8069	0	38811	4645	4319	146	48	1	0	0	0	102	0.9829	
3	1	8135	5807	1995	0	37259	12477	4201	333	370	0	0	0	0	582	0.9591	
3	2	30302	1594	28554	18628	37481	4386	4190	154	210	0	189	2910	908	71	0.9949	0.3120
3	3	39977	2201	37533	0	37071	4491	4143	243	4069	4	0	0	0	77	0.9939	
3	4	20757	11988	8492	0	37675	4837	4213	277	3562	4	0	0	0	105	0.9867	
3	5	11860	8770	2559	0	37722	4816	4220	531	767	1	0	0	0	93	0.9552	
3	6	19879	12732	6622	0	36741	4809	4107	525	3218	1	0	0	0	122	0.9736	
3	7	8205	307	7751	0	37615	4598	4208	147	42	0	0	0	0	127	0.9821	
3	8	10476	227	10148	0	37369	4618	4181	101	49	1	0	0	0	130	0.9904	
3	9	8949	357	8437	0	37744	4585	4222	155	52	0	0	0	0	117	0.9827	
4	1	16889	12072	4116	0	73906	23912	8250	701	833	1	0	0	0	1305	0.9585	
4	2	54010	3278	50388	35675	74784	8791	8317	344	467	0	365	5764	1833	208	0.9936	0.3180
4	3	84634	4748	79349	0	75229	9217	8365	537	8861	7	0	0	0	213	0.9937	
4	4	43773	25283	17879	0	74843	9689	8334	611	7655	6	0	0	0	251	0.9860	
4	5	24785	18336	5431	0	74556	9443	8304	1018	1716	1	0	0	0	239	0.9589	
4	6	41987	26964	13944	0	74081	9621	8237	1079	6821	2	0	0	0	271	0.9743	
4	7	16539	583	15621	0	74607	9099	8300	335	87	0	0	0	0	283	0.9797	
4	8	21739	490	20955	0	74462	9192	8288	294	99	1	0	0	0	329	0.9865	
4	9	18026	702	16998	0	74485	8910	8283	326	94	0	0	0	0	265	0.9819	
5	1	3490	2493	866	0	14877	4479	1653	131	159	0	0	0	0	370	0.9625	
5	2	9703	596	9042	6209	14811	1745	1638	65	92	0	63	1135	329	47	0.9933	0.2899
5	3	17256	955	16176	0	14718	1822	1642	125	1707	1	0	0	0	40	0.9928	
5	4	8914	5233	3541	0	15038	2038	1666	140	1522	1	0	0	0	68	0.9843	
5	5	5173	3808	1153	0	15048	1943	1666	212	318	0	0	0	0	59	0.9590	
5	6	8832	5711	2889	0	15049	1967	1667	232	1406	0	0	0	0	60	0.9737	
5	7	3491	105	3314	0	15015	1879	1662	72	17	0	0	0	0	63	0.9794	
5	8	4416	89	4292	0	14768	1842	1633	35	15	0	0	0	0	52	0.9921	
5	9	3455	149	3250	0	14777	1760	1636	56	21	1	0	0	0	73	0.9838	
6	1	5216	3666	1324	0	20862	6465	2342	226	247	0	0	0	0	680	0.9567	
6	2	12805	902	11791	9196	21080	2530	2353	112	114	0	117	1738	471	76	0.9913	0.2710
6	3	25550	1507	23855	0	21278	2930	2372	188	2514	0	0	0	0	69	0.9926	
6	4	13124	7670	5201	0	21343	3214	2383	253	2146	0	0	0	0	88	0.9807	
6	5	7400	5379	1535	0	21174	2999	2366	486	433	0	0	0	0	97	0.9343	
6	6	12701	8092	4220	0	21019	2922	2348	389	2004	0	0	0	0	86	0.9694	
6	7	4821	158	4544	0	21208	2983	2375	119	17	0	0	0	0	112	0.9753	
6	8	6374	125	6159	0	20950	2900	2341	90	19	0	0	0	0	97	0.9859	
6	9	5230	166	4980	0	21304	2878	2383	84	29	0	0	0	0	99	0.9839

06-03-2007, 07:29 PM	#32
LASpirit Bat Boy Join Date: Nov 2006 Posts: 14	RonCo, thanks. My primary reason for asking is that the sim league I'm involved with has created our own lahman style database using a weighted combination of prev year MLB, prev year sim, and upcoming year projections to create player models for OOTP. Hitting and Pitching stats are fairly easy using this method, but fielding are a bit more difficult. We'd like to get as accurate a model for the fielding as possible. To do this, we need to understand the strong points and limitations of the OOTP fielding performance. Any input you have would be very helpful. Thx. __________________ Continental Baseball Association Federal League West Champs: 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006 Federal League Pennants: 1999, 2000, 2001 CBA Champions: 1999, 2000, 2001

06-03-2007, 09:22 PM	#35
Elendil Hall Of Famer Join Date: Dec 2003 Location: the dynasty forum Posts: 2,318	I use Stata. I actually have the "do" file written up for anyone else who wants to process their own data in Stata. __________________ Heaven is kicking back with a double Talisker and a churchwarden stuffed with latakia.

06-04-2007, 02:11 AM	#37
darkcloud4579 Hall Of Famer Join Date: Jun 2003 Posts: 8,689	Looks very good. Great work. __________________ ------ My Mods OOTP Advanced Stats & DFS Suite Managerial Strategy Pack Competitive Balance Tax Calculator Major League Women's Baseball (OOTP24) quickstart Indian Premier League \| 300+ years of baseball quickstart \| Expatriate League quickstart \| Off-Field Injuries Update \| Women's Name File for OOTP \| ---- Dynasty classics: Centurion comes to OOTP5 \| DC Moneyball Dynasty (2004)

06-03-2007, 01:28 PM	#21
RonCo Hall Of Famer Join Date: Aug 2003 Posts: 10,383	At the end of the day, Elendil's consistency studies are effectively measuring the rate of _ratings_ change, which is only indirectly a result of talent change.

06-03-2007, 04:14 PM	#27
sporr Global Moderator Join Date: Dec 2001 Location: Muscatine, IA Posts: 8,277	Nice work!

06-03-2007, 04:38 PM	#28
Zeyes All Star Reserve Join Date: Jun 2006 Posts: 818	I'm not sure why introducing more inconsistency into the results necessarily requires an adjustment to how ratings are changed. That may be easiest to achieve, admittedly (by overlaying a sort-of random walk process on the regular ratings development mechanism for each player?), but couldn't the same effect also be achieved by introducing more variance into the "translation" from ratings to events as games are played out by the engine?

06-03-2007, 04:50 PM	#29
RonCo Hall Of Famer Join Date: Aug 2003 Posts: 10,383	I suppose the game engine itself could be made more random, but I would be reluctant to change the Log5 process currently employed, as that's been pretty well proven out.

06-03-2007, 06:32 PM	#31
RonCo Hall Of Famer Join Date: Aug 2003 Posts: 10,383	Yes, indeed. I'll see if I can dig out some of the stuff I've done in the past.

06-03-2007, 08:23 PM	#33
kroney37 Minors (Double A) Join Date: Mar 2006 Posts: 132	What program were you using Elendil? (Or for that matter, anyone else that's done work here). I've been using SPSS, but I'm always interested in different stats packages.

06-03-2007, 08:48 PM	#34
RonCo Hall Of Famer Join Date: Aug 2003 Posts: 10,383	I use Minitab for running statistics analysis--and Excel, of course.

06-03-2007, 11:44 PM	#36
kroney37 Minors (Double A) Join Date: Mar 2006 Posts: 132	Darn... I was hoping for more SPSS people - doing some basic stuff with stolen bases & ratings and looking for people more skilled than I.

06-04-2007, 04:27 AM	#38
His Own Bad Self All Star Starter Join Date: Jan 2006 Location: Phoenix, AZ Posts: 1,221	Elendil, fantastic work, as always. It's been my impression that even in this version of OOTP, the best pitchers in any given league or dynasty will consistently have seasonal ERAs below 2. I've seen pitchers with double-digit strings of seasons of ERAs ranging from, say, 1.10 to 1.80 in generally neutral park and era settings. Of course I think there may be a need for some tweaking there. Can we take this problem - if it's in fact a problem - and trace it back to any of the factors you discussed at the beginning of this thread? My first guess is that pitcher K consistency is part of the problem, but beyond that, I'm not sure of any other connections. Now that I think of it, these pitchers I'm referring to consistently seem to have very high K/BB ratios - generally 8:1 or higher over a long period of time. I get the impression that strikeouts and walks should be directly related to a small degree, and I don't see that as often as I'd like. Do you have time to test that at all?

06-04-2007, 09:15 AM	#39
StyxNCa Hall Of Famer Join Date: Dec 2004 Location: Victoria, Texas Posts: 3,136	All you're formulas, etc is a language foreign to me. I'm totally lost.