What's the difference in the DBs

[Bristolduke]

I noticed there is a .odb database and a set of lahman-like .csv files in the Stats folder. The manual seems to imply that if you want lehman or lehamn normalized use the .obd file, ortherwise you can use 3rd party.

My question is if I understand the manual correctly, what are the .csv files for?

Sorry if this was asked before, I tried to look it up in both the OOTP9 and X threads.

Thanks
Bill

[Spritze]

ODB is the compiled version of the CSV files you see.

[Nukester]

Originally Posted by Spritze

ODB is the compiled version of the CSV files you see.

The .ODB also contains correct pitches for pitchers, but the .CSV files do not, right ? Ive tried using a 3rd party database, and some of the best pitchers in real life imported with bad individual pitch ratings, while I did not have that issue when using the .ODB

[Bristolduke]

Based on another thread, the csv files don't support neutralized stats but the odb does.

Correct?

If true, then there must be some additional information the odb utilizes.

[Spritze]

I do not believe that historically correct pitches currently exist in either the odb nor in the csv's. Fictional pitches for historical pitchers exist in the odb.

The ONLY neutered stats used in the game come from the odb.

[Nukester]

Originally Posted by Spritze

I do not believe that historically correct pitches currently exist in either the odb nor in the csv's. Fictional pitches for historical pitchers exist in the odb.

But using the .ODB is the only way to ensure that premier pitchers import with high pitch ratings, if not necessarily "historically correct" pitches, correct ? Ive started a bunch of leagues in 1903, and when using the .ODB someone like Rube Waddell always imports with high pitch ratings, while starting the league with a 3rd party .CSV file is a crapshoot as to whether his pitch ratings will be high or not.

So basically if you use the .ODB, you get good pitch ratings for good pitchers, BUT you lose out on the 3rd party extras like complete SB/CS/K data or "missing years" (like during the war) for players. If you use good 3rd party .CSV files (like Gambo or Garlon/Spritze) you get the 3rd party extras like complete SB/CS/K data and "missing years", BUT pitch ratings could be all over the map. Am I correct here?

[Gambo]

The data within the databases generates the ratings. So if you use the .odb or the .csv with the same data then the ratings will be similar. The main difference between the .odb and .csv is simply the speed for importing data.

[Nukester]

Originally Posted by Gambo

The data within the databases generates the ratings. So if you use the .odb or the .csv with the same data then the ratings will be similar. The main difference between the .odb and .csv is simply the speed for importing data.

Im still not understanding something then.

If:

1 - I use the .ODB, its the same as using the plain vanilla Lahman DB, with missing data for SB/CS/K's and players like Dimaggio missing years, except its optimized for speed, and it has fictional pitch ratings for players, to ensure good pitchers get good pitch ratings

2 - I delete the .ODB and replace the default .CSV files with a Garlon or Gambo database, the missing data for SB/CS/K's and missing years will be in there (at least with the Garlon DB..Not very familiar with the Gambo DB), but since the .CSV files dont contain any sort of pitch ratings (real or fictional), the game makes up the ratings, which could be all over the place (Ive had Waddell import with a highest pitch rating of 4 on a 1-10 scale a few times in this case)

So if you want to ensure that the new pitching model is going to work for a league, without having to look at all of the pitch ratings for all pitchers, you must use the .ODB file. Using a 3rd party DB might or might not work, but you would have to look at everyone to see if the pitch ratings are high enough (no way Waddell should have a max pitch rating of 4 on a 1-10 scale). Am I right?

[Gambo]

Before I answer the questions. The .odb database is the .csv files merged together for quicker access. Essentially it is somewhere between a .zip (.rar) file and an access database. There is no difference in data between the .odb and the .csv file.

1) Yes. The .odb is the lahman database with adjustments made by OOTP developers to make it more accurate. It contains both the real stats and neutralized stats within it. It can be used to any style.

2) Yes and No.
a) I'm not certain what the garlon db contains but the Gambo database has all the same data as the .odb with minor league seasons, negro leaguers and some minor leaguers added in. So yes the data will be there.
b) The provided master.csv file (which is the same file included in the .odb file) provided in game does have pitch and fielding ratings. My database uses an adjusted master.csv file based on the provided master.csv for this year so it does include the same ratings the .odb and the original .csv includes. BUT...
IMPORTANT : the "real" pitch ratings don't exist yet and no matter what database you use it will not import the individual pitch ratings. This function does not exist yet in OOTPX as it was not completed upon release. So no matter what database you use Phil Niekro will not throw a knuckler. The game will always produce fictional pitch ratings.

So the answer to your final question is I really don't think it makes a difference. I just imported 1907 using my database (.csv) and the .odb database. In mine for Waddell he had 3 pitches rated 6-6-6 and the league had very few higher than that with the max at 8. When I did the .odb database he had 3 pitches rated 8-7-6 and he actually had the only pitches rated above 6.

[Nukester]

I know the "historically correct" pitches arent in the .ODB. Im questioning the individual pitch ratings when importing with the .ODB vs a 3rd party

Since the .CSV files contain no individual pitch data, there is no way for the game to create good fictional pitches for good players UNLESS the pitches are created based on the pitching stats (which I know is in the .CSV files). However, from what Im reading, the game does not create the fictional pitches based on the real life stats. That is why the .ODB contains some sort of fictional pitch ratings. As I said, sometimes the pitches come out ok using the 3rd party DB's, but sometimes they dont. Ive had Waddell import with individual pitch ratings around 7 or 8 and sometimes Ive had Waddell import with individual pitch ratings around 4 using the same 3rd party .CSV files

So we have to pick the lesser of two evils....either missing SB/CS/K data and missing years but with good individual pitch ratings (using the .ODB), or having the missing data and years filled in but at the expense of the game possibly importing bad pitch ratings for good players (using the Garlon DB which estimates the missing SB/CS/K data for more precise offensive ratings)

[Bristolduke]

The ONLY neutered stats used in the game come from the odb.

Spritze

Before I answer the questions. The .odb database is the .csv files merged together for quicker access. Essentially it is somewhere between a .zip (.rar) file and an access database. There is no difference in data between the .odb and the .csv file.

Gambo

I know we have two conversations going here, so I want to try and clarify what is in the files (irrespective of pitcher ratings).

These two quotes "appear" to contradict each other. One states that neutralized stats are ONLY possible if you select the odb. The other implies that you can get neutralized stats from the csv as well it will just take longer (I've run the option of csv and neutralized stats for 1919 and the difference in speed was minimal).

Let me put some perspective in my question.

I have the Gambo, Spritze and Garlon-Spritz DBs. I also have Lehman 5.6 csv (and obviously I have the Stats folder contents.). I can't see into the odb file to make any changes (which is OK as long as understand how it was built and the source data). Because...


I have two projects I'm doing. First, I want to ensure I get the right picture for the right player. So, I have changed the name some of the players so they are unique and done the same for the photo pack.

Second, I want the option to run baseball histories (and possibly on-line leagues) with the following options.

1 Real life stats -real life seasons- recalc
2 Real life stats - real life seasons - career stats no recalc
3 Neutralized stats -real life seasons- recalc
4 Neutralized stats - real life seasons - career stats no recalc
5 Real life stats -randomized seasons- recalc
6 Real life stats - randomized seasons - career stats no recalc
7 Neutralized stats -randomized seasons- recalc
8 Neutralized stats - randomized seasons - career stats no recalc

These 8 options could be with or without negro/japanese/etc players.

Obviously I want my photo work to apply to all scenarios.

I have excel spreadsheets that perform the name changes (I've also completed the photo name changes.) and performs the randomization. What I don't have is an accurate definition of what is in the stats folder files. I have the 3rd party descrition of their databases.

The "manual" implies the only way to get the lahman real and reutraized is via the odb. Which led to the question about what is in the csv files. As I stated, I can't change the odb, but can I get the same data so I can make the changes. And how?

(I suspect once I understand Nukester's question/answers - I can incorporate his concerns in my excel file utilities as well).

Thanks for the help.

[Nukester]

Sorry to hijack the thread Bristolduke. I figured this was a good place for my question since you were discussing differences in the DB's and that was what my question pertained to.

I guess in a nutshell Im trying to understand how individual pitch ratings are created when using a 3rd party .CSV set. From what Ive seen they are just randomly created. Otherwise why have a special "fictional" individual pitch rating function within the .ODB? If they are not randomly created, I think the individual pitch rating creation routine needs some fine tuning, as I have seen individual pitch ratings all over the place when not using the .ODB. As Ive said, sometimes it works, sometimes it doesnt. I know with the .ODB we dont have to worry about that. With .CSV files we do. Sure 1907 might have imported well, but what about all subsequent years? I know with the .ODB we dont have to worry about that. With the .CSV files it looks like we do.

[Gambo]

To clarify a bit about the individual pitch ratings in the .odb database.

From what has been explained on the board, the individual pitch rating were not put in to the database upon release of the game as they were incomplete. So the .odb database does not have individual pitch ratings. So in terms of the pitch ratings there is, at this point, no difference between the .odb and .csv files.

The individual pitch rating are not really ratings at all but rather a number between 1-5 determining frequency. A fictional example of this if (if the ratings were even there, which they're not) is Rube Waddell is rated for four pitches based on frequency. Fastball, Curve, Slider and Changeup. He throws a fastball most often so this is 1, Curve second so 2, Slider 3 and Changeup 4. The game than assigns the fastball as his best pitch when calculating the rating BASED ON STATISTICS, so for Fastball lets say a 95 of 100. The curveball as his 2nd 75, Slider 60, Changeup 50.

This system is evident in the master.csv provided in game. You'll also see that many pitches aren't even rated such as circlechanges and knucklers though the column is there. However, as of right now the game ignores all this and creates random ratings for random pitches based on their usage. Starters will have at least 3 pitches to ensure the pitching model works.

BTW, Bristolduke my database has no duplicate names and I am "slowly"working on a similar project for pictues based updating each year for team changes using my databases name structure. I am about 1/4 through this but I dont expect to be able to finish it any time soon.

[Bristolduke]

No problem Nukester. If we can understand how it's done, we can create our databases appropritely.

My guess is that the odb is a "pre-compiled" set for every year (real life and neutralized). It is so much larger in size than any other file. So the reason you "think" it produces "consistent" ratings is because it produces "identical" ratings every time. The csv files won't do that since it runs thru the generation algorithms (which have some randomness to it). Thus you will get "similar" ratings (in all areas not just the pitches), but you do indeed get some variability. I don't think anyone would necessarily say the odb is the "most accuate" ratings but just a "pre-compiled" or representative set.

BTW, Bristolduke my database has no duplicate names and I am "slowly"working on a similar project for pictues based updating each year for team changes using my databases name structure. I am about 1/4 through this but I dont expect to be able to finish it any time soon.

I used you db as the original master, but found a couple of errors. Also, I was looking at first names only and realized you changed the last name as well (particularly in Jrs). I was concerned that if Jr was the last name in the text the game might think that his last name was Jr (I saw this somewhere else, not sure it was true with X), so I was taking the safe route by changing only first name (e.g Ken(Jr) Griffey). I also wanted it to work on the web, so all are lowercase and underscore (vice space).

So do you know the answer the 8 scenario question?

[Gambo]

1) Just to clarify something. The .odb, the included .csv and at least I know my .csv files have the individual pitch ratings, though they are not accurate in any database.

This is all moot however as the game DOES NOT implemenent this feature at all.

This is supposed to be fixed in a patch but it came to light too late for the last patch.

I am personally waiting for this to be patched before starting a league since we have no idea how it will work with ANY database.

2) I really dont know the answer to the # 8 scenario.

3) Having the Jr. at the end of the last name is fine as long as you dont use the "." or an "_".
Ken_Griffey Jr.jpg is fine but Ken_Griffey_Jr.jpg or Ken_Griffey Jr..jpg will not work.

Also, in game as long as no . or _ is used in the game then it will read just fine. It treats it the same way as it does names like Bruce Dal Canton or Dave Von Ohlen

[Bristolduke]

The question around the 8 scenarios is If I use the included csv files and I tell the game to use neutralized stats, can it create that database?

or

I can only get neutralized data from 3rd party and the odb file.

[Gambo]

Any database which has batting,fielding,pitching and batting2,fielding2 and pitching2 will access the *2 files when creating ratings using the neutralized option. As long as those 3 files are "neutralized" statistics then they will produce those ratings.

[Nukester]

1) Just to clarify something. The .odb, the included .csv and at least I know my .csv files have the individual pitch ratings, though they are not accurate in any database.

This is all moot however as the game DOES NOT implemenent this feature at all.

hahaha Im getting more confused as we go on. The CSV files have individual pitch ratings ? Im not infront of the game now, but Ill have to check that out. Im pretty sure I looked through the default .CSV files and there was nothing in there about individual pitches. Are they contained within the Pitching.CSV file? If they are in fact in there, then I guess Ive solved my problem, as Ill just run some queries to join the 3rd party database to the default .CSV file to add on the individual pitch ratings to the 3rd party DB. However, doing that isnt going to matter either as its not used at all (until a patch hopefully fixes this).

So what we are left with is that the individual pitch ratings are created by looking at the real life stats afterall, no matter what database we are using ?

[Bristolduke]

Thanks Gambo. That's the answer I was hoping for.

Nukester - I am looking at the game. The pitch ratings are in the master.csv

[Nukester]

Originally Posted by Bristolduke

Thanks Gambo. That's the answer I was hoping for.

Nukester - I am looking at the game. The pitch ratings are in the master.csv

Thanks everyone. Im not sure how I could have missed them, unless I only looked in the pitching.csv file and not master.csv. Ill run a few queries to match up the data I want. Very cool. Again, thanks a bunch. I was about to give up on this version and go back to an older version. Seems like a whole lot of work just to get some pitchers to be both starters and relievers IMHO