Question about Master.csv vs. MiLBMaster.csv

royhobbs · 11-18-2017, 09:13 AM

Hi all:

I need help with something. I'm at my wit's end trying to create a truly "Master" CSV combining the Master.csv and MiLBMaster.csv with one instance for each player.

I'm trying to set up a fictional league with real players starting in 1901. I begin with all the players who were in the AL and NL in 1901, but I release them all and replace the actual teams with fictional ones who then draft the players.

Because I do not want the actual minor league structure, I've been manually importing players from the MiLBMaster.csv at year's end to feed the league with new players. The way I do this is to import all those players who debuted the next year from a text file that I create. So, for example, at the end of the 1901 season, I import all players from MiLBMaster.csv with a debut year of 1902.

So far so good.

But I am having two problems.

1) There are players in Master.csv who do not appear in MiLBMaster.csv at all. These are players who never played in the minors. Examples include Bob Feller, Ichiro Suzuki and Dave Winfield.

2) Then there are players who skipped the minors but then went to the minors later in their career. Examples include John Olerud, who played in Boston's minors in 2005. These players appear in both Master.csv and MiLBMaster.csv but their debut year in Master.csv is earlier than their debut year in MiLBMaster.csv. I would want to import John Olerud from Master.csv for his 1989 debut but I wouldn't want to re-import him from MiLBMaster.csv when it came time to import players for 2005.

What I would like is one CSV to rule them all that has one instance of each player with their earliest debut year, be it in the Majors or Minors.

I've tried a number of things including importing both CSVs into one MS Access database, trying to match IDs from the two .CSVs and deduplicating in MS Access, but I can't forge what would be, for me, a truly Master CSV. I was able to remove a lot of duplicates by matching the field "playerid" in Master.csv with the field "BBrefMLBid" in MiLBMaster.csv, but I wasn't entirely successful.

After deduplicating, I searched for players who had the same name, birth day, birth month and birth city and encountered a number of apparent duplicates who have different birth years in the two CSVs, but are clearly the same player. I can't figure out how to embed a screenshot, but example players whose birth years differ in the two CSVs but nevertheless appear to be the same player are Bill Birrer, Bill Clymer and Bill Fagan.

Here's a Dropbox link to a screenshot from my database. Under "Source," "Minors" indicates they came from MiLBMaster.csv whereas "Master" indicates they came from Master.csv.

Does anyone have any suggestions on how to create a truly Master.csv to use for manually importing players — one that (a) uses the Master.csv entry for players who aren't in MiLBMaster.csv, (b) uses the MiLBMaster.csv entry for players who aren't in Master.csv, and (c) uses the earlier debut year for players who are in both?

Again, maybe I'm misunderstanding these CSVs. The Spritze db, which is a phenomenal product, seems to do this, as it's one list from which I can query a particular year's debuts. Maybe the solution is just to use that, but I would like, if possible, to create my own from the two CSVs that came with the game.

Perhaps someone has a list of players who never played in the Minors they would be willing to share?

Any help is really appreciated as I'm at a major roadblock at this point.

Given that my ramble above is likely unclear, I'm totally willing to provide any additional details as needed.

Thank you!

-Roy

Spritze · 11-18-2017, 10:31 AM

Quote:

Originally Posted by royhobbs

players whose birth years differ in the two CSVs but nevertheless appear to be the same player are Bill Birrer, Bill Clymer and Bill Fagan.

These players have been adjusted in the databases. Thanks for reporting them. If you find more please report them in the historical bug report thread. I'll be back next week and fix any more you find.

Using the mlb id's and the milb id's are the way to go so you have that correct. Once combined just use the MIN command on the debut field.

11-18-2017, 09:13 AM	#1
royhobbs Bat Boy Join Date: Oct 2014 Posts: 6	Question about Master.csv vs. MiLBMaster.csv Hi all: I need help with something. I'm at my wit's end trying to create a truly "Master" CSV combining the Master.csv and MiLBMaster.csv with one instance for each player. I'm trying to set up a fictional league with real players starting in 1901. I begin with all the players who were in the AL and NL in 1901, but I release them all and replace the actual teams with fictional ones who then draft the players. Because I do not want the actual minor league structure, I've been manually importing players from the MiLBMaster.csv at year's end to feed the league with new players. The way I do this is to import all those players who debuted the next year from a text file that I create. So, for example, at the end of the 1901 season, I import all players from MiLBMaster.csv with a debut year of 1902. So far so good. But I am having two problems. 1) There are players in Master.csv who do not appear in MiLBMaster.csv at all. These are players who never played in the minors. Examples include Bob Feller, Ichiro Suzuki and Dave Winfield. 2) Then there are players who skipped the minors but then went to the minors later in their career. Examples include John Olerud, who played in Boston's minors in 2005. These players appear in both Master.csv and MiLBMaster.csv but their debut year in Master.csv is earlier than their debut year in MiLBMaster.csv. I would want to import John Olerud from Master.csv for his 1989 debut but I wouldn't want to re-import him from MiLBMaster.csv when it came time to import players for 2005. What I would like is one CSV to rule them all that has one instance of each player with their earliest debut year, be it in the Majors or Minors. I've tried a number of things including importing both CSVs into one MS Access database, trying to match IDs from the two .CSVs and deduplicating in MS Access, but I can't forge what would be, for me, a truly Master CSV. I was able to remove a lot of duplicates by matching the field "playerid" in Master.csv with the field "BBrefMLBid" in MiLBMaster.csv, but I wasn't entirely successful. After deduplicating, I searched for players who had the same name, birth day, birth month and birth city and encountered a number of apparent duplicates who have different birth years in the two CSVs, but are clearly the same player. I can't figure out how to embed a screenshot, but example players whose birth years differ in the two CSVs but nevertheless appear to be the same player are Bill Birrer, Bill Clymer and Bill Fagan. Here's a Dropbox link to a screenshot from my database. Under "Source," "Minors" indicates they came from MiLBMaster.csv whereas "Master" indicates they came from Master.csv. Does anyone have any suggestions on how to create a truly Master.csv to use for manually importing players — one that (a) uses the Master.csv entry for players who aren't in MiLBMaster.csv, (b) uses the MiLBMaster.csv entry for players who aren't in Master.csv, and (c) uses the earlier debut year for players who are in both? Again, maybe I'm misunderstanding these CSVs. The Spritze db, which is a phenomenal product, seems to do this, as it's one list from which I can query a particular year's debuts. Maybe the solution is just to use that, but I would like, if possible, to create my own from the two CSVs that came with the game. Perhaps someone has a list of players who never played in the Minors they would be willing to share? Any help is really appreciated as I'm at a major roadblock at this point. Given that my ramble above is likely unclear, I'm totally willing to provide any additional details as needed. Thank you! -Roy