CSV to HTML Box Score Converter

[drummerJ99]

Originally Posted by daigle91

I will implement an option for that in the config.txt file for version 0.5,

; single_team_output
; 0=off (default)
; 1=on

single_team_output=1


; teamID
; (0, 1, 2 ...etc.)

teamID=4



The converter's code is due for some optimization as well, which will definitely help.

If you have a sample large database I could test/benchmark (mine are typically small, with a few teams), that would be helpful for me to pinpoint any bottlenecks and improve overall performance. Would just need the following CSVs only,

league_data.csv
player_master.csv
team_data.csv
schedules.csv
player_skater_stats_rs.csv
boxscore_summary.csv
boxscore_skater_summary.csv
boxscore_goalie_summary.csv
boxscore_period_scoring_summary.csv
boxscore_period_penalties_summary.csv

e-mail: mrmikecrawford@rogers.com

I’ll email you those tonight. I had to increase the size again due to my database for the league I’m in getting bigger..for some reason. So guessing that adds to processing time of html files.


Sent from my iPhone using Tapatalk

[daigle91]

Originally Posted by drummerJ99

I’ll email you those tonight. I had to increase the size again due to my database for the league I’m in getting bigger..for some reason. So guessing that adds to processing time of html files.


Sent from my iPhone using Tapatalk

The big change from version 0.3 was,

- show cumulative season goal and assist totals for players on each goal scored.

which will definitely affect performance as it is very CPU intensive upfront before the HTML boxscores are built, and is more noticeable as the database grows in size. As a comparison I will run your database under version 0.3 just to see how much of a performance hit this new feature adds, and go from there.

[drummerJ99]

Originally Posted by daigle91

The big change from version 0.3 was,

- show cumulative season goal and assist totals for players on each goal scored.

which will definitely affect performance as it is very CPU intensive upfront before the HTML boxscores are built, and is more noticeable as the database grows in size. As comparison I will run your database under version 0.3 just to see how much of a performance hit this new feature adds, and go from there.

That makes sense. I really like that feature though, especially for my league that sims 7 days. Easier to see a players goal total at each point the game happens. So I can deal with longer processing times.


Sent from my iPhone using Tapatalk

[drummerJ99]

Originally Posted by daigle91

I will implement an option for that in the config.txt file for version 0.5,

; single_team_output
; 0=off (default)
; 1=on

single_team_output=1


; teamID
; (0, 1, 2 ...etc.)

teamID=4



The converter's code is due for some optimization as well, which will definitely help.

If you have a sample large database I could test/benchmark (mine are typically small, with a few teams), that would be helpful for me to pinpoint any bottlenecks and improve overall performance. Would just need the following CSVs only,

league_data.csv
player_master.csv
team_data.csv
schedules.csv
player_skater_stats_rs.csv
boxscore_summary.csv
boxscore_skater_summary.csv
boxscore_goalie_summary.csv
boxscore_period_scoring_summary.csv
boxscore_period_penalties_summary.csv

e-mail: mrmikecrawford@rogers.com

Ran out of time to send those yesterday, so just sent them. My boxscore_skater_summary.csv is 67MB. Everything else is under 5MB.

[daigle91]

Originally Posted by drummerJ99

Ran out of time to send those yesterday, so just sent them. My boxscore_skater_summary.csv is 67MB. Everything else is under 5MB.

This will be very helpful, many thanks.

[daigle91]

Originally Posted by drummerJ99

Ran out of time to send those yesterday, so just sent them. My boxscore_skater_summary.csv is 67MB. Everything else is under 5MB.

Just did a test on your,
Global Hockey League ( '02-03 Pre-Season )

I am getting approximately 3 mins. for pre-processing before the HTML box score files start to be created, then just under 12 secs. average per box score conversion (over the 56 game pre-season schedule). Should be able to shave down these times with some optimizing for sure.

If you get a chance I would be curious to see what kind of times you are getting in this scenario for comparison.

[drummerJ99]

Originally Posted by daigle91

Just did a test on your,
Global Hockey League ( '02-03 Pre-Season )

I am getting approximately 3 mins. for pre-processing before the HTML box score files start to be created, then just under 12 secs. average per box score conversion (over the 56 game pre-season schedule). Should be able to shave down these times with some optimizing for sure.

If you get a chance I would be curious to see what kind of times you are getting in this scenario for comparison.

We are in regular season, if that matters. But after tomorrow’s sim which will be 7 days, I’ll time how long it takes. I’m pretty sure it’s longer than that. Which is kinda weird as I have a decent pc. But I also like to multitask using both my monitors. So that could be slowing it down.


Sent from my iPhone using Tapatalk

[daigle91]

Originally Posted by drummerJ99

We are in regular season, if that matters. But after tomorrow’s sim which will be 7 days, I’ll time how long it takes. I’m pretty sure it’s longer than that. Which is kinda weird as I have a decent pc. But I also like to multitask using both my monitors. So that could be slowing it down.


Sent from my iPhone using Tapatalk

Which batch mode are you using in the config.txt file?

[drummerJ99]

Originally Posted by daigle91

Which batch mode are you using in the config.txt file?

The 7 day option. As we do 7 day sims. And then the box score without images.


Sent from my iPhone using Tapatalk

[daigle91]

Originally Posted by drummerJ99

A suggestion if possible to speed things up a bit. Would it be possible to convert just one team's boxscores to HTML?

Of course, that probably won't help or be possible once you start adding more HTML outputs, like standings and stats. idk, just a thought.

I was able to implement this just now, and it dramatically speeds things up. I chose to focus on the London TowerGuards who had played 7 pre-season games (over the span of 9 days) and the conversion process (not including pre-processing) went from 11 mins to 1.5 mins.

Originally Posted by drummerJ99

Of course, that probably won't help or be possible once you start adding more HTML outputs, like standings and stats. idk, just a thought.

These will still work as they will get calculated independent of any HTML box score filtering.

I think what I will do is release version 0.45b that will specifically just be for optimizations only, that will include this one team filter option.

[daigle91]

Originally Posted by drummerJ99

We are in regular season, if that matters. But after tomorrow’s sim which will be 7 days, I’ll time how long it takes. I’m pretty sure it’s longer than that. Which is kinda weird as I have a decent pc. But I also like to multitask using both my monitors. So that could be slowing it down.


Sent from my iPhone using Tapatalk

Just tested your regular season, and pre-processing took 31 mins. I will narrow it down and see where I can optimize the code.

[drummerJ99]

Originally Posted by daigle91

Just tested your regular season, and pre-processing took 31 mins. I will narrow it down and see where I can optimize the code.

Yeah I just tried the latest sim and it took 10:23 before it started producing the html files and then 2 more minutes to finish up. The config I'm using is: ouput_style=0, batch=1.



My PC specs: I5-10400, 32gb ram, 1660 super.

[daigle91]

Originally Posted by drummerJ99

Yeah I just tried the latest sim and it took 10:23 before it started producing the html files and then 2 more minutes to finish up. The config I'm using is: ouput_style=0, batch=1.



My PC specs: I5-10400, 32gb ram, 1660 super.

Some great news, I have pre-processing down to 4 mins. (from 31 mins., almost 8X faster!) and each boxscore conversion down to 5 secs. (from 12 secs, almost 2.5X faster!) on my PC. This will translate extremely well to your PC based on your provided benchmarks and PC specifications, as my PC is from 2009. Your time of 10:23 may very well drop below the 1 minute mark

Many thanks again for letting me run these tests on your database.

[daigle91]

Originally Posted by drummerJ99

Yeah I just tried the latest sim and it took 10:23 before it started producing the html files and then 2 more minutes to finish up. The config I'm using is: ouput_style=0, batch=1.



My PC specs: I5-10400, 32gb ram, 1660 super.

Optimization update... I have pre-processing down to less than 1 min. (from 31 mins.) on my PC. With this latest tweak I was able to lower memory consumption a little bit as well.
Should have version 0.45b available soon

[drummerJ99]

Originally Posted by daigle91

Optimization update... I have pre-processing down to less than 1 min. (from 31 mins.) on my PC. With this latest tweak I was able to lower memory consumption a little bit as well.
Should have version 0.45b available soon

Awesome work! Looking forward to it.


Sent from my iPhone using Tapatalk

[daigle91]

version 0.45b is now available for download.

[drummerJ99]

Awesome work as always. Drastically decreased the times for me. Plus using the single-team option makes it even quicker. Awesome work.

[Lurker]

Thanks to all in community to get something that should have been in the game from the beginning. Just my two cents but its appalling that a text based game lacks these fundamental reporting mechanisms.

[Tiger Fan]

Originally Posted by Lurker

Thanks to all in community to get something that should have been in the game from the beginning. Just my two cents but its appalling that a text based game lacks these fundamental reporting mechanisms.

I agree with you completely on both points. Many thanks to daigle91 for creating and continuing to improve and add to an easy to use and much needed program but I agree it is ridiculous that after 10 versions the game still has no way to output game summaries, standings or stats for the average, non-computer programming specialist, user.

[daigle91]

Many thanks drummerJ99, Lurker, Tiger Fan.

Looking forward to version 0.5b,

- standings (league, conference, division etc.)
- leader boards.
- 2 new box score styles.
- expand the schedule/results HTML page to show 'game programs' for upcoming matches that includes season series stats between teams, as well as the option of displaying each team's full length jersey in high resolution.
- ???

If anyone has any other ideas they would like to see implemented in this version, would love to hear them