Ghost player problem, related CSV info accuracy issues and some thoughts

[Jayzone]

I recently have looked into CSV export/import function closerly and I found something that may have related to the "ghost player" problem. Ghost player problem, as have been brought up by quite a few people in the past, is where you can find a particular player in the league via search player function. However, you can not find the location of the player anywhere in the leauge (team roster, FA, etc.). So, the player seems to exist but doesn't really exist - like a ghost. Ghost players are not fictional/phantom. They did actually exist in the league for a time. They become ghost players when they exit the leauge (retire via old-age, long-time on FA, injury, etc.) and they are not properly removed from active player list (or variable/flag is not set properly).

After thoroughly going to my 30-year simmed result for my planned new league-wide rating level study, I have discovered some players are listed as FA in the CSV roster export file, but they are actually ghost players. That is, I can find them via search function but they are nowhere in the league. (Some of them are even 50 something year-old!) This is further verified from the fact the those players are no longer listed in the "Players" list in ALMANAC. What ALMANAC player list does is to list the actual/correct active player list. This is unlike CSV export (roster) which is usually an inacurrate active player list. (Players who are listed in FA section of the CSV file are actually no longer in the league.)

The accuracy difference between the two is very strange. From the point of view of OOTP program, they should be of similar/same function - to provide an active player list. I susupect they are done with different programming approaches.

One thing about ghost players is that those ghost players seems be the ones that does NOT accumulate enough major league time (for example, career minor leaguer with zero major AB/IP). In another word, those players won't enter the "retired players" database even if they exit the league properly. This has lead me to suspect that the programming routine of player retirement/removal may be different between players with major league experience and players with no major league experience/no enought major league experience. I would even go as far to say the routine is faulty/broken so that ghost player problem exits.

Another thing about ghost players is that ghost players are not permanent but rather temporary. Even though OOTP program are still confused on the active/inactive status of ghost players, OOTP program knows they can recycle that player's ID. (This indicates that the player retirement routine may be just imcomplete.) So, when the need of player ID arises, that ghost player will finally disappear and are replaced by new players. "Permanent" ghost players are the ones with high ID number (close to largest ID). The need of those ID never arise, so those ghost players appears to be permanent. I suspect that ghost players may be more common than not. If you chceck players with high player ID in the league that has som history, you should be able to see a few.

As for CSV issues, they are two fold. First, CSV roster export does not generate correct active player list. (FA list inflates with ghost players) This is directly related to the ghost problem discussed above. However, even though ghost player problem exits, CSV should be designed in such way that a correct active player list can be still generated. This has actually been done. Just look at the correct active player list generated in ALMANAC. Second, CSV batting rating, pitching rating, and general info covers more players than they should. That is, the files goes from player ID#1 to player ID#last. As we all know that not every player ID# always represents a active player. CSV export, IMO, should just contain info on acitve players. I have alreay made a thread on this. (No one appears to have any thought on that topic, however.) For more info, follow the link below.

http://www.ootpdevelopments.com/boar...ad.php?t=69058

I konw that I probably have not explained things extremely clear so far. However, if you have any question, don't hesitate to ask here. I will try to explain more clearly if I can. I just hope Markus can address this soon. The ghost player problem affect gameplay itself. CSV problem affect its usefulness. It has became really hard to do a study with intended accuracy using CSV export function. (I want the info gnereated with CSV only for active players and the list needs to be accurate)

[dougaiton]

Well, that's excellent analysis of the problem.

Thanks for your hard work!

[DaveHorn]

Nice work Jayzone. I'd love to see this problem fixed. These problems (ghost player at least, haven't checked CSV) also existed in v5 and continually annoy me.

[Jayzone]

dougaiton and DaveHorn,

Thanks. Well, to be honest, I didn't expect any replies like this - ones with praises in them! I have to confess that I was writing in ranting mood, so I thought maybe this thread will be less well received or outright ignored. You have truly made me feel much better now. Thanks again.

****************

Now, I have collected more info on this ghost player problem and I will share it here. The info is obtained from the 2-league, 12-team league with 30 years of simmed history. This is basically the same setup as the league in my previous league-wide rating level study for v6.01. ( http://www.ootpdevelopments.com/boar...ad.php?t=64539 ) The 'headcount' of ghost players is not done physically by searching for them, but rather it's done by finding players that showed up in CSV roster files but not in ALMANAC files. The relationship between inacurrate CSV info and ghost players was established and explained in the previous post. The following is a table for ghost player composition. The explanations of the entries are at the bottom.

(1) Table for ghost player composition

# of players (headcount)

                 Generated Ghost Percentage(%)
Position players      2043    90    4.405 %
Pitchers              1693   110    6.497 % 
All                   3736   200    5.353 %


# of PYA (player yearly appearance)

                 Generated Ghost Percentage(%)
Position players     20285   989    4.876 %
Pitchers             16946  1332    7.860 %
All                  37231  2321    6.234 %


Note: (1) 'Generated' represents league generated players from initial league
      creation and all draftees until year 29 (The draftees of current/last
      year do not count as they haven't had the chance to become a
      ghost player - they are new to the league and are with zero pro year.)

      (2) 'Ghost' represents ghost players that are players that are supposed
      to have exited the league but not properly removed from database.

      (3) 'Percentage' is just percentage ratio.

      (4) 'PYA' stands for player yearly appearance. This is defined as how
      many times that a particular player has show up (as in year count) in
      the whole group data. So, for example, a player with 10 years in league
      (from draft to retirement) will have 10 player yearly apperances in
      the data.

The next part is the the comparison between permanent/temporary ghost. As discussed in the previous post, all ghost players are temporary. However, if theeir player IDs are never recycled, they will apears as 'permanent'. Below is the comparsion between still-existing ghost players (after 30 years, considered as 'permanent ghost players here') and all ghost players ever existed. The explanations of the entries are at the bottom.

(2) Table for Permanent/Temporary ghost player composition

Ghost player number count after 30 years

                 Ghost Total Percentage(%)
Position players    31    90   34.444 %
Pitchers            47   110   42.727 %
All                 78   200   39.000 %

Note: (1) 'Ghost' is for ghost player number count after 30-year sim.
      ('permanent' ghost players)

      (2) 'Total' is for total ghotst player number count from duting 30-year
      sim period. ('all' ghost players)

As it can be clearly seen, the portion of ghost players can not be considered insignificant. Perhaps this warrants that this is a serious problem that needs to be addressed.

While fixing the ghost player problem may be an important issue, the CSV info accuracy should be of even higher priority in my opinion. I seems to be in the minority group who view CSV as an essential and indispensable tool. However, it is a really good tool when you plan to do some studies regarding player ratings or just use CSV for global rating editing/tweaking in general.

Well, that's it for the ghost player issue. Thanks for reading.

[Officespace99]

I have a very similar problem w/ an online league I am commish for. We have players that are phantom duplicates of players. The previous commish attempted to use the utility to copy players between league files to replace existing players, and although the new ones were created properly, the old ones still exist 'in the shadows'

Now my phantoms cant be found on a search, but show up in the career leaders and also get development changes. If there is anyway to 'turn' these players off so they dont show up I'd be eternally gratefull!

[Jayzone]

I have not encountered the situation where ghost players do not show up in the search. Well, maybe I was just not aware of that. In my limited playing time of OOTP, I have not seen the ghost player gets development change. Also, my assumption is that ghost players are career minor leaguer (see my previous posts) so they should never appear on career leader board. Players with sufficient playing time in the majors will be properly removed upon exiting the league because the player will be physically transferred from Players.dat to retired.dat. Anyhow, this is my first time to hear this. Don't have any comment and can't help you here. Sorry.

p.s. By the way, I am curious about something. Does the situation happen with the league has already been attempted fixing with utility or the league without the "touch" of utility. If it's the former case, it may be the utility that makes ghost players gone from searchable to un-searchable.

[Officespace99]

The problem was that the utility didnt properly overwrite the old players, and instead created new players w/ new numbers, and just marked a flag somewhere on the old players so they arent visible or assigned to any team. I am guessing eventually they will retire, but I'd really like to be able to remove them from players.dat somehow.

[Jayzone]

So the the problem that you described exists only in the league that was modified by utility? Well, this is a very different ghost player issue than the one that I discussed in the thread. (The cause is different.) Well, to be honest, what you did (with utility) seems to make the matter worse (ghost players have gone from not interfering with the league to interfering with the league). The ghost player issue originally only plagues CSV report and the player search function. However, I do feel for you. Sorry that I can't be more any help. Well, I did not solve the ghost player problem anyway, just speculated the cause. Hope this does get addressed in the future. (and CSV issue, of course!)

[jouiskc]

Hi there,

I think this might be a related problem, but if it isn't please excuse a rookie's ignorance.

For my very first league I am running a historical league playing God so to speak. But I am checking up frequently on the real Hall of Famers to see how they are doing in the "fantasy" world. When checking up on the 1946 Brooklyn Dodgers, I noticed that Gil Hodges was not on the roster(he had played substantially in '44 and '45). Thinking that I had missed him being traded and/or released I did a search on him. He was not listed under any team. So I went back and checked the previous years almanacs, and I found him listed, but when you pull up his HTML report, it is blank. It is like the program has for some reason deleted him and his past stats from the database.

This is the only instance that I can find of this occuring and I am currently in the 27th season of the sim.

Any ideas as to why this would happen? Or should I just consider this a isolated database flake-out?

Thanks for you input.
Jeff

[Jayzone]

Originally Posted by jouiskc

...
So I went back and checked the previous years almanacs, and I found him listed, but when you pull up his HTML report, it is blank. It is like the program has for some reason deleted him and his past stats from the database.
...

jouiskc,

I am no expert in ghost players issue, but I will try to give some inputs. First, forgive me for a such obvious question: do you have the "Include Player Report" enabled in ALMANAC setting? OOTP has that disabled as default. If it's not on, then you won't see players' reports in ALMANAC, they were simply not made. Since Gil Hodges has significant playing time in 44 and 45, he should be in ALMANAC for those years (since ALMANAC is made at the end of the year). I doubt OOTP has the capability to erase a player's past, it does not simulate "Witness Protection Program". Then, more questions. Second, where can you find Gil Hodges in 46? Can you find him in (1) player search function or (2) retired player database? (The retired player database is accessed through "Hall of Fame" in General History/ALMANAC screen) If you can find him in retired player database, then it's obivious that he has retired (possible due to injury, age, etc.) If you can find him in search function but not on any team's roster or FA list, then you have a ghost player on your hand. However, my hypothesis on ghost player is that ghost players are players without ML experience since I have yet to encounter ghost players with significant ML playing time. So, if it's the second situation, it will be new to me. And then, if you truly can not find him anywhere. Well then I really have not idea what's going on.

Hope this helps a bit.

[jouiskc]

Jayzone,

You were correct in your analysis as to why Gil Hodges was not appearing in the almanac, because I did not have player reports included in the almanac settings. Unfortunately, I could not find him in any section of the current year (1946), including the retired players area. I am just going to assume that there was a database error of some kind and that this is an isolated incident. But I do appreciate your help.

Thanks
Jeff

[DaveHorn]

Excellent follow up Jayzone! I have serious doubts this would get fixed in already released versions of ootp. We can certainly hope v7 and the database rewrite would fix the problem though.

I'd be eternally grateful with a compromise and just find out a way to delete these ghosts. I've searched and searched and haven't found a way to just completely delete a player and all his records from ootp.

[Jayzone]

Dave Horn,

Thanks. However, I don't know the way to delete ghost players. Sorry.

**************************************

jouiskc,

I am actually intrigued by your situation. There some things that you can check/test to confirm that Gill is no longer in the active player database. Note that You will only have the most accurate test result if you are still in the '46 season. First, confirm that Gill can't be found from 'player search' function (don't search for Gil manually, let OOTP do it). Second, output CSV files (player general information CSV and roster CSV). Confirm that Gil is not in either of those (just use search function in the spreadsheet). If you still can't find the name of 'Gil Hodges' in any of those places, then I am pretty sure that Gil is no longer in the active player database. You can do the above if you want/choose to. Thanks.

[Officespace99]

I wonder if Pierre might be able to adapt his career editor to allow for the editing or deletion of players like these. I am sure there is some field in the player record that keeps them from showing up in his editor. Maybe this could be used to trigger the editor to display the 'deleted' players under a separate heading, or even give us the option of deleting them. Just a thought that would help any of us w/ ghost players.