OOTP6 Defense Study

[kq76]

So I wanted to redo Jason Moyer's defense study [ http://www.ootpdevelopments.com/boar...ead.php?t=2310 ], but for OOTP6. I did not copy his method exactly however, so these results cannot really be compared to his. I did not change all ratings to average and I did not have a control group (no changes) and experimental groups (changing one rating). I don't think changing everything to average is best because we don't really know what average is and we want the effects of non-average players as well. To be more exact, testing against a league with many teams of different players might be best, but that would take much longer to test. And if you did that your results would depend on how good your team was. I also did not do a control group and then experiment groups for each extreme because that is twice as much work and you would think the low extreme experiment groups would be below the control and the high above the control and that the range of the low to the high shouldn't be too far off from the single group that has one team high and one low though that should have a slightly higher range. In summary, I'm not trying to measure the exact effects because I don't have the necessary time nor the interest in knowing that. I'm just trying to determine whether there probably is an effect and if so, approximately how much. My method:

-create custom lg of 2 teams
-options disabled:
--minors
--coaches
--trades
--injuries
--fatigue
--development (replay mode), then press "run computer mngr on all teams", or else OOTP will crash
--waivers
-delete FAs
-sim pre-season
-copy lg
-using importer, replace lg B's tm Y with lg A's tm X, so lg B teams are the same
-using lg B, name 1 team high, 1 low
-create backup A
-edit a rating in high (e.g. SS range to 100 for all players with SS) and the opposite for low (e.g. SS range to 1 for all players with SS)
-create backup X
-sim 10 seasons (restoring backup X after each), noting records
first sim check stats to make sure players you edited played the most, if not fix the problem

If anyone has any suggestions on how to improve the method feel free to bring them up. If they greatly increase the time to conduct the study I probably won't try them, but I encourage others to. I can provide my A backup for anyone interested.

The results were:

	Rng				Arm
	SS	2B	CF	C	C
1	71	86	80	93	88
2	86	71	85	89	75
3	84	93	67	81	80
4	93	89	86	85	79
5	88	88	69	69	92
6	87	90	88	74	80
7	89	91	92	80	71
8	90	87	86	72	93
9	82	83	80	85	78
10	80	71	89	81	83
W	85.0	84.9	82.2	80.9	81.9
L	77.0	77.1	79.8	81.1	80.1
D	8.0	7.8	2.4	-0.2	1.8
%	.525	.524	.507	.499	.506

I'm no statistician so I can't give you the possible +/- error, but I imagine the wins are +/- 2 to 3. I find it discouraging that there appears to be so little CF range affect, but maybe there's a great deal of error involved in the results. I also find it discouraging that 2 teams that are otherwise completely identical, except for one rating change on both teams results in one team going 69 and the other 93, or put another way, at least 24 games back 8 times out of 50.

EDIT: I realized waivers need to be disabled as well.

[gmo]

Interesting stuff.

All I can speak to is the range in win totals. Similar work I've done suggests there is a lot of noise in the system (and makes me think the Rangers really can hang on this year!) as there tends to be quite a range in results for equal teams. I was testing home field advantage with a 6-team league of identical players, where 3 teams always played at home and 3 always played away. Can't recall offhand what varied between the runs, but I did 3 50-year runs for that setup. The standard deviation value for wins for any group of the 150 team seasons for one of the three team sets in a run was somewhere between 6-7. Your 93 wins would be almost 2 standard deviations away from the expected mean of 81, so based on my data you'd only expect about 3 such occurrences in your 50 seasons. But is it right to apply that logic here? Would always be interesting to see more data.