Defense & AI Evaluation

PSUColonel · 01-04-2017, 01:21 PM

For all the talk and debate we have here regarding the various AI evaluation settings people use, I thought I would ask something I am not sure has been discussed.

For arguments sake, lets take two different and popular evaluation settings.

50/30/15/5 & 40/30/20/10

OK. Now, my question is this...when the AI looks at stats, does it also include defensive metrics in those numbers, or is defense solely judged on ratings? So if you were to have a player like CF Peter Bourjos, who is an absolute stud defensively, but not so hot with the bat, would the 50/30/15/5 settings favor him?

If the AI looks at WAR (which I assume it does) the defense would be baked in though correct? I am wondering if a more ratings heavy setting is better to measure defense though.

Thoughts?

NoOne · 01-04-2017, 04:34 PM

i'm not sure if it uses defense, no way to know really... at least without alot of data and a spreadsheet.

anyway my guess would be that it takes a rating and uses stats to assess lower or higher then once done with each "new" and unseen rating it calculates the new overall from those values - just as it always does.

so you see the ratings as your scout guesses them, but the overall is calculated with modification of those ratings based on the most directly related statistics that are available. which stats? don't let anyone tell you they know for sure....some linked stats are quite obvious, but may include things we don't expect, too.

so however defense is incorporated into Overall, it can feasibly be modified by statistical results just as the other factors...

which stats they use in what weight is completely speculation without real effort. we can be in the ballpark on a few things, i'm sure, but nothign to be overconfident about.

PSUColonel · 01-05-2017, 12:07 AM

This would be interesting to know, as the answer would certainly play a role in what AI evaluation settings I'd choose to use. If defensive is more or less calculated using just ratings, then obviously ratings become that much more important, but if WAR, ZR, or some other combination of defensive metrics are used, then the less need for ratings weight.

NoOne · 01-06-2017, 04:30 PM

i don't see how you draw that conclusion.

if stats affect it, it is being applied just as it is to all other ratings. if it isn't, it's a small portion of the total pie, nonetheless. (the difference between the pure rating and stats-modified version that is used to caclulate the ai-eval Overall, and not the total effect of defense. so, a very small difference relative to all that goes into calculating an AI-eval OVerall rating)

as a human you'd be able to notice a lack of integrity in the Overall that the AI could not... ie you could see the great defensive ratings and the possibly poor results from a small sample and know the plaer is slightly underrated due to this context, all other factors ignored / staying the same. (stats lowered his defense, which lowered overall relative to actual ratings)

i have this ffeeling that you think the stats are more telling than the ratings for defense (or anythign for that matter)? if so, this is not right. it's as if you want to incorporate the stats to "fix" the bad defensive rating (baed on how you wrote it)? the defensive rating isn't bad... the results are just a poor sample.

(excludes things we cannot see that the game might or might not model... like a 7' tall catcher with 80/80 arm/ability may not be as good as a 6' catcher with the same ratings... we don't know for sure what's going on without investigation... i.e. height visibly affects 1b defensive rating, but maybe other physical aspects are unseen influences.)

In those very specific types of situations, ai-eval will likely give a more accurate depection of actualy ability... but otherwise it's just stirring the water and making it more difficult to see reality (reality in the video game - ha!).

that is what incorporating stats does... it muddies the water... any other perception of it isn't quite right.... that doesn't mean i am cheerleading 100% ratings... never have... human gm's make stupid decisions based on bad samples all the time.

as with most occupations maybe 5-10% are excellent at their job - truly intuitive about what they do.... the rest are derivative thinkers that are more akin to monkeys than an intelligent human being. they just copy others and draw baseless conclusions with bad info and feelings. some are actually completely incompetent despite being hired as a gm (and rehired).

Matt Arnold · 01-06-2017, 05:02 PM

We do incorporate defense into the stats values.

PSUColonel · 01-06-2017, 05:04 PM

OK...thank you. It was unclear to me.

NoOne · 01-09-2017, 11:25 AM

are there missing posts here or is this just strikingly similar to another thread?

the sites been quite funky lately - main reason i ask... i but i remember being happy about learning that tidbit about defense being included with certainty. (not just us speculating about something)

01-04-2017, 01:21 PM	#1
PSUColonel Hall Of Famer Join Date: Feb 2002 Posts: 12,990 Blog Entries: 37	Defense & AI Evaluation For all the talk and debate we have here regarding the various AI evaluation settings people use, I thought I would ask something I am not sure has been discussed. For arguments sake, lets take two different and popular evaluation settings. 50/30/15/5 & 40/30/20/10 OK. Now, my question is this...when the AI looks at stats, does it also include defensive metrics in those numbers, or is defense solely judged on ratings? So if you were to have a player like CF Peter Bourjos, who is an absolute stud defensively, but not so hot with the bat, would the 50/30/15/5 settings favor him? If the AI looks at WAR (which I assume it does) the defense would be baked in though correct? I am wondering if a more ratings heavy setting is better to measure defense though. Thoughts?

01-04-2017, 04:34 PM	#2
NoOne Hall Of Famer Join Date: Apr 2015 Posts: 7,167	i'm not sure if it uses defense, no way to know really... at least without alot of data and a spreadsheet. anyway my guess would be that it takes a rating and uses stats to assess lower or higher then once done with each "new" and unseen rating it calculates the new overall from those values - just as it always does. so you see the ratings as your scout guesses them, but the overall is calculated with modification of those ratings based on the most directly related statistics that are available. which stats? don't let anyone tell you they know for sure....some linked stats are quite obvious, but may include things we don't expect, too. so however defense is incorporated into Overall, it can feasibly be modified by statistical results just as the other factors... which stats they use in what weight is completely speculation without real effort. we can be in the ballpark on a few things, i'm sure, but nothign to be overconfident about. Last edited by NoOne; 01-04-2017 at 04:36 PM.

01-05-2017, 12:07 AM	#3
PSUColonel Hall Of Famer Join Date: Feb 2002 Posts: 12,990 Blog Entries: 37	This would be interesting to know, as the answer would certainly play a role in what AI evaluation settings I'd choose to use. If defensive is more or less calculated using just ratings, then obviously ratings become that much more important, but if WAR, ZR, or some other combination of defensive metrics are used, then the less need for ratings weight.

01-06-2017, 04:30 PM	#4
NoOne Hall Of Famer Join Date: Apr 2015 Posts: 7,167	i don't see how you draw that conclusion. if stats affect it, it is being applied just as it is to all other ratings. if it isn't, it's a small portion of the total pie, nonetheless. (the difference between the pure rating and stats-modified version that is used to caclulate the ai-eval Overall, and not the total effect of defense. so, a very small difference relative to all that goes into calculating an AI-eval OVerall rating) as a human you'd be able to notice a lack of integrity in the Overall that the AI could not... ie you could see the great defensive ratings and the possibly poor results from a small sample and know the plaer is slightly underrated due to this context, all other factors ignored / staying the same. (stats lowered his defense, which lowered overall relative to actual ratings) i have this ffeeling that you think the stats are more telling than the ratings for defense (or anythign for that matter)? if so, this is not right. it's as if you want to incorporate the stats to "fix" the bad defensive rating (baed on how you wrote it)? the defensive rating isn't bad... the results are just a poor sample. (excludes things we cannot see that the game might or might not model... like a 7' tall catcher with 80/80 arm/ability may not be as good as a 6' catcher with the same ratings... we don't know for sure what's going on without investigation... i.e. height visibly affects 1b defensive rating, but maybe other physical aspects are unseen influences.) In those very specific types of situations, ai-eval will likely give a more accurate depection of actualy ability... but otherwise it's just stirring the water and making it more difficult to see reality (reality in the video game - ha!). that is what incorporating stats does... it muddies the water... any other perception of it isn't quite right.... that doesn't mean i am cheerleading 100% ratings... never have... human gm's make stupid decisions based on bad samples all the time. as with most occupations maybe 5-10% are excellent at their job - truly intuitive about what they do.... the rest are derivative thinkers that are more akin to monkeys than an intelligent human being. they just copy others and draw baseless conclusions with bad info and feelings. some are actually completely incompetent despite being hired as a gm (and rehired).

01-06-2017, 05:02 PM	#5
Matt Arnold OOTP Developer Join Date: Jun 2009 Location: Here and there Posts: 14,149	We do incorporate defense into the stats values.

01-06-2017, 05:04 PM	#6
PSUColonel Hall Of Famer Join Date: Feb 2002 Posts: 12,990 Blog Entries: 37	OK...thank you. It was unclear to me.

01-09-2017, 11:25 AM	#7
NoOne Hall Of Famer Join Date: Apr 2015 Posts: 7,167	are there missing posts here or is this just strikingly similar to another thread? the sites been quite funky lately - main reason i ask... i but i remember being happy about learning that tidbit about defense being included with certainty. (not just us speculating about something)