Cities.txt Editor

LeafsFan2005NC · 06-26-2006, 05:25 PM

Hi, all. Just thought I'd let y'all know that I'm currently working and a program that will let us edit the cities.txt file much easier. This one will, of course, take a bit longer to create, especially since it's not a "must have" like the schedule converter was.

At the moment, I've got it to load up the nations.txt file so a nation can actually be chosen and it will load the list of cities for any given nation, but I'm trying to get it to do that a little faster. After that comes editing the file, backing up the old file (this will be handled automatically) and then saving the new, edited file.

Hopefully after I get all that worked out, I'll code it so we can more easily add new nations to the nations.txt file as well as edit the file more easily.

Raidergoo · 06-26-2006, 05:36 PM

I'd like to have the ability to quickly edit all cities of a certain province to a new nation.

Example:
make southern US states and their cities join a Confederate States of America.

I want to sort by state, then renumber the nation code.

Example:
make ex-Soviet republic and their cities join the USSR.

I want to sort by province/region, have the old nation name become the province/region, and then renumber the nation code.

Morozovsk,29289,227,Russia,48.35,41.83
Morshansk,43311,227,Russia,53.45,41.81
Mortka,3698 ,227,Russia,59.33, 66.02
Moryakovskiy Zaton,4745,227,Russia, 56.7,84.65
Mosalsk,4320,227, Russia ,54.49,34.98
Moscow,10381222,227,Russia,55.75, 37.62

Using Excel is a hassle due to the size and sorting.

redsox45 · 06-26-2006, 07:03 PM

This sounds good.

It's nice to be able to edit the files, but you need to know what you're doing. Still, there's nothing like having an editor.

Will

LeafsFan2005NC · 06-26-2006, 10:45 PM

Indeed I hope to be able to do this. Editing the nations.txt file is pretty simple since it's a small file, but the cities.txt file is a monster. Like already pointed out, Excel won't load it because it's so huge.

I do have the listing of cities pretty much figured out and it works pretty well. It takes a bit for the US cities to load since there's about 15,000 of them, but it's still pretty quick.

I'll see about the sorting deal. It shouldn't be too hard once I figure it out. But, I'll keep everyone abreast as to my progress.

EDIT: Well, I've got sorting (ascending and descending) down pat, plus editing (no find/replace yet) and saving of the new file. I've also managed to add the ability to add or delete a city. Also when you initially start the program the first time, it will back up the nations.txt and cities.txt automatically.

LeafsFan2005NC · 06-28-2006, 04:19 PM

Well, I've managed to get the nations.txt file to load. Apparently, even though there are 15 fields for each country, the majority of the countries don't even include those fields, meaning Excel wants to complain when you try to load it, as well as my program. I have manually edited the nations.txt file to make sure each line has 14 commas.

I've added a find/replace function to the cities, but will try to make it quicker. Since the US has about 15,000 cities alone, searching for one city takes a little bit, but I will see about adding so only one column is searched at a time.

Raidergoo · 06-28-2006, 08:06 PM

Quote:

Originally Posted by LeafsFan2005NC

I've added a find/replace function to the cities, but will try to make it quicker. Since the US has about 15,000 cities alone, searching for one city takes a little bit, but I will see about adding so only one column is searched at a time.

What? You aren't using my mod that adds 14,000 more US cities? For shame!

LeafsFan2005NC · 06-28-2006, 09:36 PM

Quote:

Originally Posted by Raidergoo

What? You aren't using my mod that adds 14,000 more US cities? For shame!

You're serious? The cool thing is, I searched through the US cities and they actually put in my local town, about fifteen minutes from my house.

On the program front, I've added editing of the nations.txt file and it does work. I added the CSA to my nations.txt file and then went back and deleted it.

Now I will see if an edited nations.txt will load up in the cities.txt editor and, if all goes well, should be at a point where y'all can test it out for me.

Raidergoo · 06-28-2006, 09:56 PM

Quote:

Originally Posted by LeafsFan2005NC

You're serious?

http://www.allsimbaseball.com/index....d=18&Itemid=37

Look for the file for 14,000 additional small towns.

Bob Zupcic · 06-28-2006, 10:16 PM

Quote:

Originally Posted by Raidergoo

http://www.allsimbaseball.com/index....d=18&Itemid=37

Look for the file for 14,000 additional small towns.

Something's wacky with this mod. It only lists five towns in Maine, and of those, several are already in the original database.

A number of the populations are wildly off. Somerville, Massachusetts, is listed with a population of 863. Somerville is essentially part of Boston; there are more than 863 people in Davis Square alone. Same with Northampton, Massachusetts, which is listed as having a population of 161. I'd guess the population of Northampton to be something like 29,000.

Not that I'm complaining. Mods are always a good thing. But how were the numbers calculated for this database?

LeafsFan2005NC · 06-29-2006, 01:48 AM

I'm guessing they searched for census data or some sort. Must have been really old census data if towns are showing up with so low numbers.

My biggest beef, and this seems to be in any baseball sim, is that Brooklyn isn't included at all. Now, granted, Brooklyn is part of New York City these days, but how many Brooklynites do you know that consider themselves New Yorkers?

On the programming front, the editor works. I just need to add a few lines here and there and figure out why it wants to complain when I want it to backup the file I'm editing, but maybe I'll have a good working version for someone to play with tomorrow sometime. I've tired my brain figuring it out tonight, so maybe tomorrow.

Bob Zupcic · 06-29-2006, 02:05 AM

Quote:

Originally Posted by LeafsFan2005NC

Must have been really old census data if towns are showing up with so low numbers.

You ain't kidding. In order to find a census when Northampton, Massachusetts, had a population of 161, you'd have to go back to about 1660.

Carplos · 06-29-2006, 03:32 AM

There latitude and longitudes are also way off.

Raidergoo · 06-29-2006, 07:27 AM

http://www.census.gov/tiger/tms/gazetteer/places2k.txt

This is the data source, from the last census.

EDIT

What is up with Maine? I will have to find my drafts.

EDIT 2

What happened to Northampton, MA? Now I am mad.

Bob Zupcic · 06-29-2006, 12:49 PM

Quote:

Originally Posted by Raidergoo

What is up with Maine? I will have to find my drafts.

What happened to Northampton, MA? Now I am mad.

Unfortunately, these aren't the only things that got messed up.

Elko, Nevada's listed population: 923.
Elko's actual population: abut 17,000.

Rutland, Vermont's listed population: 753.
Rutland's actual population: about 17,000.

Brevard, North Carolina's listed population: 162.
Bervard's actual population: about 7,000.

And so on. These are just places I happen to know well and was easily able to see that things weren't quite right. Maybe the database somehow got corrupted? Sorry to be the bearer of bad news. But we're all detail-oriented folks around here. That's why we love this game so much, right?

Raidergoo · 06-30-2006, 07:16 AM

I have a theory. I may have zipped and emailed the wrong file. I think this old data may have been from a different source at the Census dept.

Those cities you are pointing out ( I think ) are in multiple counties. The data that I was using listed their populations in each county, so I had to merge them and then delete them. I gave that up as too cumbersome and inaccurate when a different data file was shown to me.

For the life of me, I cant find the file I created from the places.txt Census data. It was a rather simple thing; I saved the file to C, imported it into Excel, used a custom space delimiting function to sort data into columns, deleted useless data like FIPS code, land and water area, then sorted by population, then chopped off the file at 1000 population size.

I think I will have to do it all over again.

Carplos · 06-30-2006, 08:06 AM

I haven't gone state by state to look at how many duplicates there were (I went back to the original file), but there was nearly two dozen in Idaho alone. Something in your process screwed up Lat/Long because the Idaho cities in it would've been around Ohio.

(I'm honestly not trying to knock all the work you did for it, just trying to help figure out what happened...)

06-26-2006, 05:25 PM	#1
LeafsFan2005NC Minors (Double A) Join Date: Mar 2005 Posts: 108	Cities.txt Editor Hi, all. Just thought I'd let y'all know that I'm currently working and a program that will let us edit the cities.txt file much easier. This one will, of course, take a bit longer to create, especially since it's not a "must have" like the schedule converter was. At the moment, I've got it to load up the nations.txt file so a nation can actually be chosen and it will load the list of cities for any given nation, but I'm trying to get it to do that a little faster. After that comes editing the file, backing up the old file (this will be handled automatically) and then saving the new, edited file. Hopefully after I get all that worked out, I'll code it so we can more easily add new nations to the nations.txt file as well as edit the file more easily. __________________ Download my OOTP 6 to OOTP 2006 Schedule Converter: http://www.ootpdevelopments.com/boar...d.php?t=121664

06-26-2006, 07:03 PM	#3
redsox45 Hall Of Famer Join Date: Dec 2001 Location: Machias, Maine Posts: 4,573 Blog Entries: 1	This sounds good. It's nice to be able to edit the files, but you need to know what you're doing. Still, there's nothing like having an editor. Will __________________ Top Five Books I Ever Read: 1. Murder of Roger Ackroyd -- Agatha Christie 2. Birds of Prey -- Wilbur Smith 3. King Solomon's Mines - H. Rider Haggard 4. Comstock Lode -- Louis L'Amour 5. Andersonville -- McKinley Kantor

06-26-2006, 10:45 PM	#4
LeafsFan2005NC Minors (Double A) Join Date: Mar 2005 Posts: 108	Indeed I hope to be able to do this. Editing the nations.txt file is pretty simple since it's a small file, but the cities.txt file is a monster. Like already pointed out, Excel won't load it because it's so huge. I do have the listing of cities pretty much figured out and it works pretty well. It takes a bit for the US cities to load since there's about 15,000 of them, but it's still pretty quick. I'll see about the sorting deal. It shouldn't be too hard once I figure it out. But, I'll keep everyone abreast as to my progress. EDIT: Well, I've got sorting (ascending and descending) down pat, plus editing (no find/replace yet) and saving of the new file. I've also managed to add the ability to add or delete a city. Also when you initially start the program the first time, it will back up the nations.txt and cities.txt automatically. __________________ Download my OOTP 6 to OOTP 2006 Schedule Converter: http://www.ootpdevelopments.com/boar...d.php?t=121664 Last edited by LeafsFan2005NC; 06-27-2006 at 03:48 PM.

06-28-2006, 04:19 PM	#5
LeafsFan2005NC Minors (Double A) Join Date: Mar 2005 Posts: 108	Well, I've managed to get the nations.txt file to load. Apparently, even though there are 15 fields for each country, the majority of the countries don't even include those fields, meaning Excel wants to complain when you try to load it, as well as my program. I have manually edited the nations.txt file to make sure each line has 14 commas. I've added a find/replace function to the cities, but will try to make it quicker. Since the US has about 15,000 cities alone, searching for one city takes a little bit, but I will see about adding so only one column is searched at a time. __________________ Download my OOTP 6 to OOTP 2006 Schedule Converter: http://www.ootpdevelopments.com/boar...d.php?t=121664

06-29-2006, 01:48 AM	#10
LeafsFan2005NC Minors (Double A) Join Date: Mar 2005 Posts: 108	I'm guessing they searched for census data or some sort. Must have been really old census data if towns are showing up with so low numbers. My biggest beef, and this seems to be in any baseball sim, is that Brooklyn isn't included at all. Now, granted, Brooklyn is part of New York City these days, but how many Brooklynites do you know that consider themselves New Yorkers? On the programming front, the editor works. I just need to add a few lines here and there and figure out why it wants to complain when I want it to backup the file I'm editing, but maybe I'll have a good working version for someone to play with tomorrow sometime. I've tired my brain figuring it out tonight, so maybe tomorrow. __________________ Download my OOTP 6 to OOTP 2006 Schedule Converter: http://www.ootpdevelopments.com/boar...d.php?t=121664

06-26-2006, 05:36 PM	#2
Raidergoo Hall Of Famer Join Date: Mar 2003 Posts: 9,004	I'd like to have the ability to quickly edit all cities of a certain province to a new nation. Example: make southern US states and their cities join a Confederate States of America. I want to sort by state, then renumber the nation code. Example: make ex-Soviet republic and their cities join the USSR. I want to sort by province/region, have the old nation name become the province/region, and then renumber the nation code. Morozovsk,29289,227,Russia,48.35,41.83 Morshansk,43311,227,Russia,53.45,41.81 Mortka,3698 ,227,Russia,59.33, 66.02 Moryakovskiy Zaton,4745,227,Russia, 56.7,84.65 Mosalsk,4320,227, Russia ,54.49,34.98 Moscow,10381222,227,Russia,55.75, 37.62 Using Excel is a hassle due to the size and sorting.

06-29-2006, 03:32 AM	#12
Carplos Hall Of Famer Join Date: Feb 2002 Location: Idaho Posts: 2,688 Infractions: 1/0 (0)	There latitude and longitudes are also way off.

06-29-2006, 07:27 AM	#13
Raidergoo Hall Of Famer Join Date: Mar 2003 Posts: 9,004	http://www.census.gov/tiger/tms/gazetteer/places2k.txt This is the data source, from the last census. EDIT What is up with Maine? I will have to find my drafts. EDIT 2 What happened to Northampton, MA? Now I am mad. Last edited by Raidergoo; 06-29-2006 at 07:31 AM.

06-30-2006, 07:16 AM	#15
Raidergoo Hall Of Famer Join Date: Mar 2003 Posts: 9,004	I have a theory. I may have zipped and emailed the wrong file. I think this old data may have been from a different source at the Census dept. Those cities you are pointing out ( I think ) are in multiple counties. The data that I was using listed their populations in each county, so I had to merge them and then delete them. I gave that up as too cumbersome and inaccurate when a different data file was shown to me. For the life of me, I cant find the file I created from the places.txt Census data. It was a rather simple thing; I saved the file to C, imported it into Excel, used a custom space delimiting function to sort data into columns, deleted useless data like FIPS code, land and water area, then sorted by population, then chopped off the file at 1000 population size. I think I will have to do it all over again.

06-30-2006, 08:06 AM	#16
Carplos Hall Of Famer Join Date: Feb 2002 Location: Idaho Posts: 2,688 Infractions: 1/0 (0)	I haven't gone state by state to look at how many duplicates there were (I went back to the original file), but there was nearly two dozen in Idaho alone. Something in your process screwed up Lat/Long because the Idaho cities in it would've been around Ohio. (I'm honestly not trying to knock all the work you did for it, just trying to help figure out what happened...)