Write Report to Disk Problems

Cobby · 09-01-2017, 04:15 PM

I like to get player data into external spreadsheets where I can manipulate the data to my heart's content. To do this, I set up various custom views and filters, then use the "Report" dropdown to "Write Report to Disk". This puts the data into a browser window and from there I can easily copy and paste into a spreadsheet. Very easy and simple.

But, I've recently hit a snag doing this and I wonder if anyone has any ideas how to get around it.

It appears that when the source data grows to a certain point, the player screen gets broken down into pages. Now when I "Write Report to Disk" I only get 19 players at a time. If I'm examining a list of say 4000 or so players it's no longer practical to write the report to disk and pop it into a spreadsheet. (I'd have to do it over 200 times to get all the data 19 players at a time)

The thing is, this happens based on the size of the *source* data, not what it gets filtered down to. So even if I use filters to get the list whittled down to say 200 players, if the source data is large enough (not sure what the limit is) it will still break the 200 players down into pages and I'd still have to write, copy and paste over 10 times to get the data into the spreadsheet.

My first thought as a work-around would be to break the source data into a couple of chunks - then I could do the copy and paste operation twice. Not ideal, but I could live with it. However, I only have two checkboxes to manipulate the size of the source data. One is "Include Retired Players" and the other is "Include Minor Leaguers". In both cases I still get active major leaguers, so there's no way to break the source data into two mutually exclusive chunks. We would need a third check box "include active major leaguers", instead of having them always included my default.

I think we really do need that third checkbox anyway regardless of my particular problem so that we can easily exclude active major leaguers look at minor leaguers only or retired players only.

By the way, another way I thought of to get around my problem was to go to the league history - retired players section. That way I could get retired players from there and the rest from the normal players list. (my two mutually exclusive chunks) But, that doesn't work either. This is because, oddly, there are more retired players showing up on that list than there are when I check the retired players checkbox on the normal player's screen. By quite a bit - I assume it must be including retired players that never made the majors on that list or something. Whatever the case, the source limit is exceeded and that screen is always broken up into pages, making it useless for exporting.

Any suggestions?

cavebutter · 09-07-2017, 02:47 PM

It's been a while since I have been able to even load up OOTP, so working from memory here, but I believe what you want to do is set up a data dump rather than writing to disk. In one of the menus, you can set up which tables you want to dump and in what format (I believe you can choose between MySQL, Access, and CSV, and the last time I checked, Access didn't work).

You can open the csv's in Excel, or if you have the program, the knowledge, and the time, set up a database in Access to manipulate the data to your heart's content.

The major drawback to this method is that you don't get the advanced statistics - those you would have to calculate on your own. A fun challenge.

NoOne · 09-07-2017, 09:06 PM

if familiar with excel, you can make a output worksheet and use links to the exported files as data worksheets.

type all formulas in the output worksheet by referencing cells from the data worksheet -- either type them or go click on them etc (useful to know syntax).

Now, once you have everythign either simply "=worksheet3.a1" etc to be visible or some advanced stat each and every time you update those external links to the exported files, everythign is updated.

even withoutknowledge, it's simply point-and-click on the cells involved. you only have to setup the first row. *if 2 subleagues, you'll want to also highlight the second row before "filling" as explained below.

to add years, you simply highlight the row and then "fill" down (mouse-over bottom-right of selection until cursor turns to "+" like symbol, then click and drag down in this case). must match orientation of the data worksheets. years ascendending. can sort anyway you want after that point, but keep a "cnt" column to reverse and change, so you can add years easily in future. type "1" in first row and "fill" down.. .it will count up for you.

a cumulative total row can be kept at the bottom.. simply insert entire rows within the table above that row and re-fill down to bottom row. all fixed. either expand on-demand or make a huge table and let it fill up. want to start at a different year? drag that row to the top and re-fill... start earlier? drag the top row down that many years and fill up to top. re-fill down to bottom, if necessary.

the best use of a spreadsheet invovled setting it up 1 time... then using it without as little effort as possible.

i googled a macro that could add a button to the toolbars which updates external links to all files. definitely useful too.

You make the basic shell, and the data will simply accrue. make charts and graphs etc... they will auto-update with the table you create (whether you make grid around it or not, it's a "table" to a spreadsheet program)

Cobby · 09-10-2017, 12:09 AM

Yeah, I'll probably have to use the csv files (or access) in cases where the data set is too large. Too bad, since my method is so easy and I'll set up for it now.

Perhaps the developers will help me out in a future version (*hint* *hint*).

Drstrangelove · 09-10-2017, 10:39 PM

I just did it with 6,374 players (free agents), and I don't see the problem.

1) I open the free agent canned screen
2) select write to disk
3) a long report comes up
4) I select all the data (mouse clicks)
5) control c
6) open excel
7) select paste special
8) choose text

Voila!

No page breaks. All the data shows up as expected.

I'd probably use the other methods if I needed to do this often (I don't), but I can't replicate the problem. Now I have seen something like that, but I had to use print along the way to get there. (I don't think you did that.) Anyway, maybe this will spark a clue.

Cobby · 09-11-2017, 08:52 PM

I'm good at 6235 players, but by 9081 it's broken into pages. So the breakpoint is somewhere between those two numbers - 8192 maybe? A nice power-of-two number?

NoOne · 09-12-2017, 06:23 PM

i had an idea...

use an > / =< age that splits 9k+ down to a size that does not switch to 'pages'

wherever the demarcation line is, that's how many slices you'd need and be able to keep doing it your current way... ~12k before you need a 3rd split etc (well 3rds - <x, between x and y, then >y etc)... or whatever it may be.

Cap Heresy · 09-12-2017, 06:45 PM

I'm getting bleary from reading the manual, but I think I saw something about a "scrollbar" option that impacts what is exported/reported.

Cobby · 09-12-2017, 07:11 PM

Quote:

Originally Posted by Cap Heresy

I'm getting bleary from reading the manual, but I think I saw something about a "scrollbar" option that impacts what is exported/reported.

Yes, there is a scrollbar option (under file->settings) and I have it set to scrollbar already. Apparently it only works until the source data reaches a certain size and then reverts to paged...

Cobby · 09-12-2017, 07:16 PM

Quote:

Originally Posted by NoOne

i had an idea...

use an > / =< age that splits 9k+ down to a size that does not switch to 'pages'

wherever the demarcation line is, that's how many slices you'd need and be able to keep doing it your current way... ~12k before you need a 3rd split etc (well 3rds - <x, between x and y, then >y etc)... or whatever it may be.

The trouble is, for some reason, whether or not it gets broken into pages depends on the size of the source data not on the size of what the data is filtered down to. Here is an example where I filtered my 9000+ players down to 591 and it's still broken up into pages.

NoOne · 09-13-2017, 01:15 AM

export will be easy enough. exclude headers either on export or during import to excel/calc program. no different than copy/paste.. you'll select the delimiter the same way etc in calc at least.

may need to change some links. just make sure the imported data has its own worksheet(s). update external data links when necessary. import file as a worksheet option, if i recall - something like that. re-do some linked fields on your existing spreadsheet and never worry about it again.

Cobby · 09-13-2017, 09:56 AM

Quote:

Originally Posted by NoOne

export will be easy enough. exclude headers either on export or during import to excel/calc program. no different than copy/paste.. you'll select the delimiter the same way etc in calc at least.

may need to change some links. just make sure the imported data has its own worksheet(s). update external data links when necessary. import file as a worksheet option, if i recall - something like that. re-do some linked fields on your existing spreadsheet and never worry about it again.

Wait a minute. I'm not sure if I get what you're saying. When the data is broken into pages like that, I only get 19 lines at a time when I select "write data to disk". This makes getting the data out very tedious at best, or completely impractical at worst. Am I missing something?

NoOne · 09-15-2017, 11:35 AM

i'm sayingn use the export features instead of 'write to disk'

one-time setup, then no big deal. plus, i think more info is given in an export, although you may need to calculate certain stats from more basic ones in some instances.

at least 2 areas you have access to these options. main league settings in the available option drop down... look for 'export' or related verbage. also, global settings->database tab in available options with less clutter than the league settings menu. (where you can 'purge' db etc). choose settings for export is in same areas.

09-01-2017, 04:15 PM	#1
Cobby Minors (Triple A) Join Date: May 2016 Posts: 252	Write Report to Disk Problems I like to get player data into external spreadsheets where I can manipulate the data to my heart's content. To do this, I set up various custom views and filters, then use the "Report" dropdown to "Write Report to Disk". This puts the data into a browser window and from there I can easily copy and paste into a spreadsheet. Very easy and simple. But, I've recently hit a snag doing this and I wonder if anyone has any ideas how to get around it. It appears that when the source data grows to a certain point, the player screen gets broken down into pages. Now when I "Write Report to Disk" I only get 19 players at a time. If I'm examining a list of say 4000 or so players it's no longer practical to write the report to disk and pop it into a spreadsheet. (I'd have to do it over 200 times to get all the data 19 players at a time) The thing is, this happens based on the size of the source data, not what it gets filtered down to. So even if I use filters to get the list whittled down to say 200 players, if the source data is large enough (not sure what the limit is) it will still break the 200 players down into pages and I'd still have to write, copy and paste over 10 times to get the data into the spreadsheet. My first thought as a work-around would be to break the source data into a couple of chunks - then I could do the copy and paste operation twice. Not ideal, but I could live with it. However, I only have two checkboxes to manipulate the size of the source data. One is "Include Retired Players" and the other is "Include Minor Leaguers". In both cases I still get active major leaguers, so there's no way to break the source data into two mutually exclusive chunks. We would need a third check box "include active major leaguers", instead of having them always included my default. I think we really do need that third checkbox anyway regardless of my particular problem so that we can easily exclude active major leaguers look at minor leaguers only or retired players only. By the way, another way I thought of to get around my problem was to go to the league history - retired players section. That way I could get retired players from there and the rest from the normal players list. (my two mutually exclusive chunks) But, that doesn't work either. This is because, oddly, there are more retired players showing up on that list than there are when I check the retired players checkbox on the normal player's screen. By quite a bit - I assume it must be including retired players that never made the majors on that list or something. Whatever the case, the source limit is exceeded and that screen is always broken up into pages, making it useless for exporting. Any suggestions?

09-07-2017, 02:47 PM	#2
cavebutter All Star Starter Join Date: Dec 2005 Location: Los Angeles Posts: 1,164	It's been a while since I have been able to even load up OOTP, so working from memory here, but I believe what you want to do is set up a data dump rather than writing to disk. In one of the menus, you can set up which tables you want to dump and in what format (I believe you can choose between MySQL, Access, and CSV, and the last time I checked, Access didn't work). You can open the csv's in Excel, or if you have the program, the knowledge, and the time, set up a database in Access to manipulate the data to your heart's content. The major drawback to this method is that you don't get the advanced statistics - those you would have to calculate on your own. A fun challenge. __________________ MySQL, MyStruggle - A self-indulgent blog about my attempts to roll my own MySQL Database with OOTP Logo Gallery

09-07-2017, 09:06 PM	#3
NoOne Hall Of Famer Join Date: Apr 2015 Posts: 7,258 Infractions: 1/0 (0)	if familiar with excel, you can make a output worksheet and use links to the exported files as data worksheets. type all formulas in the output worksheet by referencing cells from the data worksheet -- either type them or go click on them etc (useful to know syntax). Now, once you have everythign either simply "=worksheet3.a1" etc to be visible or some advanced stat each and every time you update those external links to the exported files, everythign is updated. even withoutknowledge, it's simply point-and-click on the cells involved. you only have to setup the first row. if 2 subleagues, you'll want to also highlight the second row before "filling" as explained below. to add years, you simply highlight the row and then "fill" down (mouse-over bottom-right of selection until cursor turns to "+" like symbol, then click and drag down in this case). must match orientation of the data worksheets. years ascendending. can sort anyway you want after that point, but keep a "cnt" column to reverse and change, so you can add years easily in future. type "1" in first row and "fill" down.. .it will count up for you. a cumulative total row can be kept at the bottom.. simply insert entire rows within the table above that row and re-fill down to bottom row. all fixed. either expand on-demand or make a huge table and let it fill up. want to start at a different year? drag that row to the top and re-fill... start earlier? drag the top row down that many years and fill up to top. re-fill down to bottom, if necessary. the best use of a spreadsheet invovled setting it up 1 time... then using it without as little effort as possible. i googled a macro that could add a button to the toolbars which updates external links to all files. definitely useful too. You make the basic shell, and the data will simply accrue. make charts and graphs etc... they will auto-update with the table you create (whether you make grid around it or not, it's a "table" to a spreadsheet program) Last edited by NoOne; 09-07-2017 at 09:08 PM.*

09-10-2017, 10:39 PM	#5
Drstrangelove Major Leagues Join Date: Jun 2014 Posts: 346	I just did it with 6,374 players (free agents), and I don't see the problem. 1) I open the free agent canned screen 2) select write to disk 3) a long report comes up 4) I select all the data (mouse clicks) 5) control c 6) open excel 7) select paste special 8) choose text Voila! No page breaks. All the data shows up as expected. I'd probably use the other methods if I needed to do this often (I don't), but I can't replicate the problem. Now I have seen something like that, but I had to use print along the way to get there. (I don't think you did that.) Anyway, maybe this will spark a clue. Last edited by Drstrangelove; 09-10-2017 at 10:42 PM.

09-11-2017, 08:52 PM	#6
Cobby Minors (Triple A) Join Date: May 2016 Posts: 252	I'm good at 6235 players, but by 9081 it's broken into pages. So the breakpoint is somewhere between those two numbers - 8192 maybe? A nice power-of-two number? Attached Images

09-10-2017, 12:09 AM	#4
Cobby Minors (Triple A) Join Date: May 2016 Posts: 252	Yeah, I'll probably have to use the csv files (or access) in cases where the data set is too large. Too bad, since my method is so easy and I'll set up for it now. Perhaps the developers will help me out in a future version (hint hint).

09-12-2017, 06:23 PM	#7
NoOne Hall Of Famer Join Date: Apr 2015 Posts: 7,258 Infractions: 1/0 (0)	i had an idea... use an > / =< age that splits 9k+ down to a size that does not switch to 'pages' wherever the demarcation line is, that's how many slices you'd need and be able to keep doing it your current way... ~12k before you need a 3rd split etc (well 3rds - <x, between x and y, then >y etc)... or whatever it may be.

09-12-2017, 06:45 PM	#8
Cap Heresy Minors (Single A) Join Date: Sep 2014 Posts: 65	I'm getting bleary from reading the manual, but I think I saw something about a "scrollbar" option that impacts what is exported/reported.

09-13-2017, 01:15 AM	#11
NoOne Hall Of Famer Join Date: Apr 2015 Posts: 7,258 Infractions: 1/0 (0)	export will be easy enough. exclude headers either on export or during import to excel/calc program. no different than copy/paste.. you'll select the delimiter the same way etc in calc at least. may need to change some links. just make sure the imported data has its own worksheet(s). update external data links when necessary. import file as a worksheet option, if i recall - something like that. re-do some linked fields on your existing spreadsheet and never worry about it again.

09-15-2017, 11:35 AM	#13
NoOne Hall Of Famer Join Date: Apr 2015 Posts: 7,258 Infractions: 1/0 (0)	i'm sayingn use the export features instead of 'write to disk' one-time setup, then no big deal. plus, i think more info is given in an export, although you may need to calculate certain stats from more basic ones in some instances. at least 2 areas you have access to these options. main league settings in the available option drop down... look for 'export' or related verbage. also, global settings->database tab in available options with less clutter than the league settings menu. (where you can 'purge' db etc). choose settings for export is in same areas.