The draft and OSA scouting

[Spracks]

I should probably not say anything, since my Geno Smith/Matt Barkley analogy was destroyed. But.....

To say that Mel Kiper had 0 correct 1st round predictions is a little misleading. He correctly predicted 26 players who would be taken in the first round, although exactly when they would be drafted was a little off (although I think he still had 6 picks that were exactly right, doesn't really matter).

He was able to correctly assess 26 players who were first round talents. Again, comparing that with the OSA, who only agrees with my scout on 2 players who should be taken in the first round. The difference between my scout and the OSA is so ridiculously huge it is impossible to make a logical decision. It's not like I can go watch the players in the OOTP universe with my own eyes to determine which scout is correct. I have one scouting source telling me a guy is the second coming of Mickey Mantle, and another scouting source telling me he couldn't hit the pinata at my sister's birthday party, how am I supposed to tell which is correct without going in the editor? That magnitude of disagreement does not happen in real life.

What is worse, is that the OSA is labeled as less accurate than your own scouting staff, and only to be used as a second opinion. But, I am finding when I look up the numbers in the editor, that the OSA has the correct evaluation of potential on a vast majority of players. I now find myself trusting the OSA ratings much more than the ratings of my own scout, which is definitely not how it is supposed to work.

[Wrigleyviller]

Originally Posted by Spracks

I should probably not say anything, since my Geno Smith/Matt Barkley analogy was destroyed. But.....

To say that Mel Kiper had 0 correct 1st round predictions is a little misleading. He correctly predicted 26 players who would be taken in the first round, although exactly when they would be drafted was a little off (although I think he still had 6 picks that were exactly right, doesn't really matter).

He was able to correctly assess 26 players who were first round talents. Again, comparing that with the OSA, who only agrees with my scout on 2 players who should be taken in the first round. The difference between my scout and the OSA is so ridiculously huge it is impossible to make a logical decision. It's not like I can go watch the players in the OOTP universe with my own eyes to determine which scout is correct. I have one scouting source telling me a guy is the second coming of Mickey Mantle, and another scouting source telling me he couldn't hit the pinata at my sister's birthday party, how am I supposed to tell which is correct without going in the editor? That magnitude of disagreement does not happen in real life.

What is worse, is that the OSA is labeled as less accurate than your own scouting staff, and only to be used as a second opinion. But, I am finding when I look up the numbers in the editor, that the OSA has the correct evaluation of potential on a vast majority of players. I now find myself trusting the OSA ratings much more than the ratings of my own scout, which is definitely not how it is supposed to work.

Agreed - this is a huge problem, and it extends beyond the draft. I recently looked at the top prospects report, and the difference between my scout's opinion and OSA's is completely insane. The number one overall ranked prospect by OSA is a one-star prospect according to my scout. The top ten prospects top out at 2.5 stars for my scout. That's nuts - no scout on earth thinks Jurickson Profar or Oscar Tavares or Dylan Bundy or whomever is a bad prospect. This is even worse than the draft, where at least we're talking about evaluating amateurs and opinions will naturally differ more. Now we're talking about the top prospects in all of baseball, players who have destroyed AA and AAA and have a track record of professional success and my scout still summarizes them as unlikely to ever make the majors.

[flair1239]

Just a quick opinion.

I believe in real life there is more volatility in scouting for baseball, than football.

For one, football players do not get drafted right out of high school. So at the minimum scouts get to see them against relatively tougher competition for at least two years.

In baseball, at least two factors are at work. First, many players are drafted right out of high school. Even when you take into account elite summer traveling leagues and what not, most of the players drafted have not consistently faced elite competition... or at least faced other players of similar ability level.

Those players that do go on to college, (many times because they were not satisfied with their draft position), face competition where the truly elite players have already been conscripted into minor league ball.

To summarize, players drafted into the NFL have for the most part already been facing a higher level of competition relative to baseball players. Therefore the football scouts have a better idea of what they are buying. Whereas in baseball, I would think that the first time a player with "elite" potential consistently faces others at that level would be in AA ball?

In football a lot of the flameouts of talented players happen in college. In baseball because of the way they handle talent acquisition, those flameouts happen in the minor leagues.

[Gern44]

Originally Posted by Spracks

To say that Mel Kiper had 0 correct 1st round predictions is a little misleading.

Because I enjoy Kiper Bashing, I’ll emphasize that he did in fact correctly predict the draft slot of exactly 0 first round prospects.

You are, however, correct in that that is misleading. But that doesnt serve my discussion, so we'll conveniently overlook that point.

The Geno/Barkley example is key because it highlights the extreme variances in opinions. Which is what we are getting with OSA/Team Scouts.

It is realistic to have varying degrees of opinions on these prospects. But it seems that the degree of variance, and the frequency of variances, are too great right now.

[Lukas Berger]

Originally Posted by Gern44

The Geno/Barkley example is key because it highlights the extreme variances in opinions. Which is what we are getting with OSA/Team Scouts.

Yup. And as the post above alludes to, that variance is increased greatly in baseball. Other than 5-6 guys generally agreed to be elite, there's really no consensus among top draft guys and even less correlation between whatever consensus there is and the actual draft, when it happens.

Right now, I can give you 100+ names for this years Rule 4 draft that I've seen mocked or ranked as first round players. Even then, I'll bet a couple guys slip into the first round that no one predicted would be there. It's the same every year.

[NerdPowah]

I've noticed that at first your scout and OSA's opinion on players are usually on the opposite end of the spectrum. But as the seasons go by, they both seem to be pretty on-par with each other.

[BIG17EASY]

Originally Posted by NerdPowah

I've noticed that at first your scout and OSA's opinion on players are usually on the opposite end of the spectrum. But as the seasons go by, they both seem to be pretty on-par with each other.

I think this is because the more a player plays, the more the OSA and your scout have to go on to make their evaluations, so generally the two will come closer together once there are many years worth of performance to evaluate.

[da commish]

Originally Posted by Gern44

Because I enjoy Kiper Bashing, I’ll emphasize that he did in fact correctly predict the draft slot of exactly 0 first round prospects.

Kiper picked 8 first round draft slots correctly, including 4 of the first 5.

[Lukas Berger]

Originally Posted by da commish

Kiper picked 8 first round draft slots correctly, including 4 of the first 5.

Pretty sure he got 6 right, not including any of the top 3 but getting 4 and 5 right.

Based on this, his final mock draft.

[da commish]

Originally Posted by lukasberger

Pretty sure he got 6 right, not including any of the top 3 but getting 4 and 5 right.

Based on this, his final mock draft.

That wasn't his latest mock. He updated it Thursday afternoon, but I think you could only get it if you were an ESPN Insider. He had Fisher #1 and Joeckel #2 in his latest. He also was correct on #4, 5, 11, 14, 26, and 27.

[JMDurron]

For the record, I don't actually hate Mel Kiper (although I do hate ESPN's college football "coverage"), I just thought it would be the easiest comparison for many posters to relate to.

The point remains that there should be massive variance between internal evaluations and external evaluations of draft prospects, AND massive variance between the evaluations of prospects on Draft Day and the likelihood that those prospects will ever become real MLB players.

EDIT - As an aside, I don't believe that Mr. Kiper has gotten a single Patriots draft pick (in any round) right for the last 11 years running.

[dkgo]

Originally Posted by JMDurron

For the record, I don't actually hate Mel Kiper (although I do hate ESPN's college football "coverage"), I just thought it would be the easiest comparison for many posters to relate to.

The point remains that there should be massive variance between internal evaluations and external evaluations of draft prospects, AND massive variance between the evaluations of prospects on Draft Day and the likelihood that those prospects will ever become real MLB players.

EDIT - As an aside, I don't believe that Mr. Kiper has gotten a single Patriots draft pick (in any round) right for the last 11 years running.

its nearly impossible to exactly predict a late first round pick especially when he doesnt account for trades that will happen on draft day. later than that is a complete guess. in general anyone he has as a top 20 player, with the possible exception of QBs towards the end of that range which are very need dependant, is going to get picked in the first round or early second.

i agree with what OP is saying here. it seems crazy that of the ten highest rated osa prospects for the draft (who every other team seems to like) my scout doesn't have any of them over 2 stars. my scout usually only has five to ten prospects over four stars out of the entire draft. i almost always just go with osa now especially for starting pitchers and over the past couple seasons those are the players that are putting up good minor league numbers and getting moved up while my scout slowly upgrades his ratings.

[Gern44]

Originally Posted by da commish

Kiper picked 8 first round draft slots correctly, including 4 of the first 5.

I'm not an ESPN Insider, so I can't speak for which revision of Kiper's Mock I last saw. Clearly, he updated it as the draft drew closer.

I googled it to try to find it, but failed. However, that brought up an interesting point. What makes Kiper (or any Draftnik) change their draft opinions so much day to day? The players haven't played a down in months. The combine is far in the rearview mirror.

These guys change their mocks based on what they are hearing from the Teams and from the Teams Scouts, not based on their own personal evaluations. As the draft draws closer, its not longer a matter of the Draftniks evaluations of players, its what they hear NFL teams are going to do.

Kiper rated Geno Smith as the 1 or 2 QB, then just before the draft, dropped him to a day 2 pick (according to an sbnation.com article published April 26th). Geno Smith didnt get any better or worse as a player. But Kiper heard rumblings of him dropping... of NFL Teams not liking him as much as he thought they did. So he dropped in the mock. Thats all fine and good... its expected and its Kiper's job.

Based on that, the only point in time that you can fairly say that these guys have relatively unbiased rankings based solely on evaluations is probably right after the combine, before NFL Teams start to drop hints as to what they plan to do. After that point, the Draftniks are just trying to be as 'right as possible'. Evaluations no longer matter.

Therefore, after that point in time, the Kiper/OSA comparison is inaccurate. But I'd say that during the College Season, all the rankings and ratings the guys like Kiper do are certainly comparable to the OSA ratings.

[soadeathdealer]

Originally Posted by JMDurron

For the record, I don't actually hate Mel Kiper (although I do hate ESPN's college football "coverage"), I just thought it would be the easiest comparison for many posters to relate to.

The point remains that there should be massive variance between internal evaluations and external evaluations of draft prospects, AND massive variance between the evaluations of prospects on Draft Day and the likelihood that those prospects will ever become real MLB players.

EDIT - As an aside, I don't believe that Mr. Kiper has gotten a single Patriots draft pick (in any round) right for the last 11 years running.

Just want to point out that patriots trade there picks at draft time a lot so that really narrows anybody's chances of predicting there pick lol

Sent from my ADR6350 using Tapatalk 2

[JMDurron]

Originally Posted by Gern44

These guys change their mocks based on what they are hearing from the Teams and from the Teams Scouts, not based on their own personal evaluations.

I read an article a couple of years ago that emphasized how important Mel Kiper is to another set of people that you've overlooked here. Player Agents. It's in the best interests of almost every player being drafted to have mock drafts out there showing them to be going as high as possible, and that info comes to the draft prognosticators from those agents, who have more of an interest in tipping the hands of the teams than the teams themselves generally do.

Originally Posted by soadeathdealer

Just want to point out that patriots trade there picks at draft time a lot so that really narrows anybody's chances of predicting there pick lol

Sent from my ADR6350 using Tapatalk 2

Sure, that is obviously a factor. That's also part of why their drafts tend to be graded so harshly after the fact - I think the draftniks get frustrated that the Patriots never do what they can predict they will do, making their predictions reliably useless for that particular team.

[Cras]

Mock drafts are horse.

They are made by people with inside information, and they get inside information based on confidentiality, so they will mix in the information they know and they put out false information knowingly so that they don't burn their sources, which in that business is all they have.

And if they were actually able to just rely solely on their scouting abilities, they would not be reporters, they would be scouts, or GMs. More money to be made being a big league GM than some dirt bag scoop artist.

And the NFL is a different beast since you can trade pics, so there is much more mis-information going around out there. From what I have read, the MLB Rule 4 draft is more about GMs calling around and getting a feel for who is going to draft who so they know if their guy is going to be available. The NFL draft, while there is that aspect, there is also the blowing smoke up everyone's you know what so that they can drum up trades, there is more of a concept of value of a draft pic, and you will not pic a guy you like too early because you know that is under valuing the pick, so you look to trade down so you can get your guy at the right spot worth of his value. You don't have that in baseball. You want a guy, you are gonna have to take him in the spots you have available.

[Shadow Knows]

NFL teams don't just draft on talent, they draft on immediate position need and scheme suitability, so Mel Kiper's attempts to divine who'll be taken with which pick is entirely separate from his own assessments of players' platonic talent level. OSA's analogue isn't Mel Kiper, it's the organizations that run the NFL combine. Its function is to grade players, not forecast who will draft them.

Even if one essentially fanwanks that OSA is actually supposed to be a stand-in for Baseball America or something, it can't even be said to portray that correctly. Because baseball teams don't draft for immediate position need or employ wildly divergent personnel schemes, the eventual draft order correlates quite well with the aforementioned platonic talent rankings. This is evident by looking at BA's pre-draft rankings.

When 99% of the world rates Johnny Fastball in the top 20, Grizzled Scout will not see the 96-MPH heater and sharp slider everyone else sees and think "garbage." He may rate Johnny 33rd overall, but he will not peg him as a 33rd-round talent. There isn't unanimity, but there *is* consensus. The game should capture that.

[Canuck]

The issue isn't predicting which exact spot a player will go. It's about OSA and your scout having 25 of the top 30 prospects different. That just doesn't happen.

[Bobble]

Originally Posted by Shadow Knows

NFL teams don't just draft on talent, they draft on immediate position need and scheme suitability, so Mel Kiper's attempts to divine who'll be taken with which pick is entirely separate from his own assessments of players' platonic talent level. OSA's analogue isn't Mel Kiper, it's the organizations that run the NFL combine. Its function is to grade players, not forecast who will draft them.

Even if one essentially fanwanks that OSA is actually supposed to be a stand-in for Baseball America or something, it can't even be said to portray that correctly. Because baseball teams don't draft for immediate position need or employ wildly divergent personnel schemes, the eventual draft order correlates quite well with the aforementioned platonic talent rankings. This is evident by looking at BA's pre-draft rankings.

When 99% of the world rates Johnny Fastball in the top 20, Grizzled Scout will not see the 96-MPH heater and sharp slider everyone else sees and think "garbage." He may rate Johnny 33rd overall, but he will not peg him as a 33rd-round talent. There isn't unanimity, but there *is* consensus. The game should capture that.

This. Plus, I like the use of "platonic" here.

[ezpkns34]

I get why the NFL draft is being used in the analogy (b/c of it being so much more popular than the MLB draft), but there is an actual MLB draft that is accompanied by MLB mock drafts each year as well that may prove be at least as helpful as the NFL comparison

Now, I don't follow MLB drafts at all, so I simply picked up a 2 of the top results for 2012 mlb mock drafts:
SI's & mymlbdraft's

I would assume we can equate SI's w/ a Kiper mock & the other with a walterfootball type of mock. Again, I don't follow the MLB draft, so if there are other well knock MLB mock drafters, feel free to add in how their info compared to that I'm posting below

Those 2 mock drafts only had 4 different players on their mock drafts, while they both agreed on 27 players being 1st round talents

Of the 27 players they agreed should/would be 1st round picks, 23 of them were taken in the first round (with the other 4 being taken in the Supplemental 1st)

Of SI's other 4 projected 1st rounders, 2 went in the first & 2 went in the Supplemental 1st

Of mymlbdraft's other 4 projected 1st rounders, 2 went in the first, 1 went in the Supplemental 1st & the other went in the 2nd


So, at least according to this isolated case, it would appear that there's somewhat of a consensus on what constitutes a 1st round talent among MLB draft analysts. Maybe 2012 was a fluke season for MLB mock drafters (I don't care enough to dig up multiple previous seasons' worth of data), but I'll assume that it's a safe bet that typically at least 20 of the first round picks would be widely considered first round talents by most evaluators