View Single Post
Old 05-12-2005, 11:57 PM   #10
Jayzone
All Star Reserve
 
Jayzone's Avatar
 
Join Date: Apr 2004
Posts: 706
Really not knocking off what you were trying to do but as a word of caution. (thanks for all the efforts on ERA regression testing, but I didn't really have the time to examine the data) There are a few things that I want to comment on R^2 used as a measure of model fitness.

First, R^2 is not always a good comparative statistics between models. For a model with higher R^2, it could still have larger (worse) mean error square value. (mean error sqaure is generally a reasonable measure of the overall quality of the fit) This means that a model with higher R^2 would not necessarily be a better fit model.

Second, it is especially unfair to compare R^2 between multiple regression models. For mutiple regression, adding an additional regressor (x variable, so to speak) to the model will always increase R^2 regradless of whether or not the additional regressor contributes to the model. So, comparaing R^2 value is pretty much useless in helping to select the best model in multiple regression. For multiple regression, there are a few tests that you can perform to measure the fitness and adequacy of the model.

Third, the variance in actual stats is inherent large (and the correlation between ratings and stats is never close to perfect 1). So, unless the model is very sophisticated or very complete, the value of any measure of model adequacy will never be very good. For simple regression (ie. single rating to single rate stats), this inherent lagrge variance will still very likely keep R^2 to be low. So, we should probably be satisfy with a reasonable R^2 value. As I said before, the acutal stats is never a result of certain mathematical formula (like y = a*x), but rather it's a result from complex random events.

Last, you were performing regression directly on ERA and I am performing regression on elemental rate stats. Even disregarding what I said about R^2 above, it would probably also be unfair and not very sensible to compare R^2 values betwen them because we are not really doing the same thing here.

Hope this provides somes thing for think about.
__________________
Rating Translator v0.93b (for OOTP 6/6.5) - released/updated on 03/13/05.
- The function of this utility is to translate both the player's ability and talent ratings into stats (expected performance).

Note: Rating Translator is now also hosted at jazzrack's CLB - OOTP Download.

Click here for the discussion on "How computer GM may be made smarter by having built-in rating translation ability"

Last edited by Jayzone; 05-12-2005 at 11:58 PM.
Jayzone is offline   Reply With Quote