Uploading a Lahman Year..taking forever

[bababui]

The first time I uploaded a historical roster it took about 75 minutes...the next time I tried it was 2 hours plus with only 19% of players downloaded. Anyone else experience this? Any ideas on whats wrong?

[JimG]

+1

I tried to start in 2006 with Lahman and it got to 54% after 8 hours so I gave up since I needed to use my computer.

[redmarkYankees]

Doesn't take more than a couple of minutes for me, and I've done lots already testing certain things. It really does seem that some people are having big problems not experienced by others - hardware setup, anti-virus, other processes? Don't know.

[JimG]

Im starting to think that Ootp isnt a game as much as a database application and so you should think about building a scripting langauge for it.

Instead of download a roster database I want to download an entire script that loads up the game, builds a 16-team universe with the correct minor league teams, and loads the 1927 data from Lahamn including all the rookies on the right teams. Then I can just leave the computer overnight and I dont have to agonize over whether Im doing something wrong that I cant undo.

Like Catobase is this somethign you could support in next years version?

[Bluenoser]

Originally Posted by JimG

+1

I tried to start in 2006 with Lahman and it got to 54% after 8 hours so I gave up since I needed to use my computer.

Lahman doesn't have 2006 in it since the 2006 season hasn't been completed yet.

[edm]

not sure why it's so slow ofr you. i've imported about ten seasons so far, the last being 2005, and each took less than a minute or two.

[JimG]

Originally Posted by BruceM

Lahman doesn't have 2006 in it since the 2006 season hasn't been completed yet.

Right but you can set it up to load 2005 stats and start the game in 2006. Unless you shouldnt be able to do that and thats my bug.

[redmarkYankees]

Originally Posted by JimG

Right but you can set it up to load 2005 stats and start the game in 2006. Unless you shouldnt be able to do that and thats my bug.

No, I've done exactly that with Lahman and it took only a minute or so.

[bababui]

The Arod/Garlon DB works just fine. So all is good once again in the universe.

[marc420]

OK, take this with a grain of salt as I haven't tried this myself. But this might be the sort of case where an anti-virus program might really slow things down.

If the game is doing a lot of modifying of files, especially opening then closing then opening then closing, etc, the anti-virus software may step in each time and do a little check to make sure everything's kosher ... which in turn would really slow things down.

Again, take this with a grain of salt. I don't work here, I haven't loaded in a Lahman database, and I didn't even stay at a Holiday Inn Express last night.

But with some people saying its fast and some people saying its really slow, that might be a guess I might take.

A completely unrelated case where an anti-virus program does massively slow things down is disk de-fragmentation. I know from experience its several orders of magnitude slower with the anti-virus enabled than disabled. And its because the anti-virus software keeps jumping in and checking every change.

Attach all required disclaimers about why anti-virus is very important and why I don't shut them down lightly. But its something someone might think of trying ... after making sure they are either off-line or not doing any interactions with the internet while doing that test.

[Treen007]

lahman db makes my game crash... I dont know why, the Arod/Garlon db works fine.

[Jeremy Weimer]

It seems as though the Lahman DB is causing problems for a lot of people but other DB's like Arod/Garlon and Ankit work just fine. It's weird because when you get down to it they're all basically the same thing. Just a few player tweaks/players added and other minor things.

I'm in no way trying to discredit any of the DB makers but Ankit and Arod/Garlon both used the Lahman database as their starting point. So that's all I'm trying to say. It's jus weird that the Lahman one is causing so many people problems but the other two seem to work fine.

[JimG]

Turned off my antivirus software. This time I left for 2 hours and came back and it was froze at 40%.

[Marc Duffy]

What Lahman DB is this.

if it's 5.3, there might be a problem with it that Sean Lahman has been alerted to.

If you are using 5.3, can you try 5.1 or 5.2 and do the same stuff

[disposableheros]

double check and make sure you are using the csv version

[redmarkYankees]

Originally Posted by Marc Duffy

What Lahman DB is this.

if it's 5.3, there might be a problem with it that Sean Lahman has been alerted to.

If you are using 5.3, can you try 5.1 or 5.2 and do the same stuff

Sorry Marc, but as long as he's using the csv version, it's not a problem with the database. I can import a league in about a minute, as can others. There seem to be problems for some users, not other.

Getting a fictional 1871 start with historical rookies importing correctly is another matter entirely .

[Marc Duffy]

Ok, well there is a problem with the database that is for sure (it becomes more apparent on the mac) and it's to do with some of the line endings in the file. We have alerted Mr Lahman about this and I am thinking, depending on the machine, it might also be a problem on the PC...hence me asking (for sanity checking purposes) for people with problems to try an earlier version

[redmarkYankees]

Do you know which file(s) in the Lahman database has problems with the EOL characters? That can be changed very easily to allow those having trouble to test with a fixed DB.

[redmarkYankees]

nm, got it.

Batting.csv and Fielding.csv lack carriage return characters, having only linefeeds. Some users systems presumably have setups or other software installed that allows them to cope with that; others don't.

For anyone who wants to try fixing that to get a speedy Lahman import and isn't afraid to run vbscript given by some bloke on the internet - copy and paste the following into Notepad:



set oFS = createobject("scripting.filesystemobject")
set oFILE1 = oFS.GetFile("Batting.csv")
oFILE1.Name = "Batting.bak"
set oFILE2 = oFS.GetFile("Fielding.csv")
oFILE2.Name = "Fielding.bak"
set oTR1 = oFS.OpenTextFile("Batting.bak")
set oTW1 = oFS.CreateTextFile("Batting.csv")
set oTR2 = oFS.OpenTextFile("Fielding.bak")
set oTW2 = oFS.CreateTextFile("Fielding.csv")
Do
strTemp=oTR1.ReadLine
oTW1.WriteLine(strTemp)
Loop Until oTR1.AtEndOfStream
Do
strTemp=oTR2.ReadLine
oTW2.WriteLine(strTemp)
Loop Until oTR2.AtEndOfStream
set oFILE1 = Nothing
set oFILE2 = Nothing
set oTR1 = Nothing
set oTR2 = Nothing
set oTW1 = Nothing
set oTW2 = Nothing
set oFS = Nothing
Msgbox "Finished"



Save this as "whatever.vbs", in the same folder as your Lahman database. Double click the file.

That creates versions of the Batting.csv and Fielding.csv with proper end-of-line CR/LF combos; if that is the reason for slow loading/failure to load, this should fix it. (Temporary fix, obviously, for those who want to try). (All it does is read the line and writes it back to a new file, but doing it with vbscript means it gets a vbCrLf end of line combo).

[bababui]

Thanks RY....lets see how this goes after the new patch tomorrow.