Page MenuHomeWildfire Games

rename maps/ file names to use underscores (instead of spaces and capitalization)
ClosedPublic

Authored by Nescio on Nov 15 2017, 1:34 PM.

Details

Reviewers
Stan
elexis
bb
Group Reviewers
Restricted Owners Package(Owns No Changed Paths)
Commits
rP23719: Remove all spaces, caps, parentheses and hyphens from map file names.
Summary

File names in the public folder typically use underscores; occassionally capitalization or CamelCase is used. Many of the maps files are different in this respect: many include spaces (" ") in their file names, which is not only inconsistent, but also inconvenient when using a command line.

The situation currently is:
maps/random/: all file names use underscores and no capitalization
maps/scenarios/: about half use spaces and capitalization, others underscores, a few camel case
maps/skirmishes/: most use spaces and capitalization, a few underscores

[Update] To implement a consistent renaming, run:


which does the following:

  • removes hyphens
  • removes parentheses
  • replaces spaces with underscores
  • replaces uppercase with lowercase

And also deletes the following files:

  • scenarios/Flight_demo.* because the _2 version looks much better
  • scenarios/Height Map *.png because unused in game, unclear licence, and no reason to keep them, and *.pmp and .xml because they don't seem to have added value either
  • scenarios/Sandbox - Ptolemies.* because the 2 version looks much better
Test Plan

Run the script and afterwards fix the following five lines:

scenarios/cinema_demo.xml: line 105
scenarios/triggers_demo.xml: line 68
scenarios/units_demo.xml: line 43
skirmishes/gallic_fields_3.xml: line 97
tutorials/introductory_tutorial.xml: line 87

Check if everything works after renaming.

Diff Detail

Repository
rP 0 A.D. Public Repository
Branch
/ps/trunk
Lint
Lint OK
Unit
No Unit Test Coverage
Build Status
Buildable 5260
Build 8963: Vulcan BuildJenkins
Build 8962: arc lint + arc unit

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
Nescio edited the summary of this revision. (Show Details)Feb 28 2018, 2:54 PM
Nescio added a comment.EditedFeb 28 2018, 2:59 PM

Build failure - The Moirai have given mortals hearts that can endure.

Or instead, run:


and manually correct afterwards:

  • scenarios/cinema_demo.xml: line 106
  • scenarios/triggers_demo.xml: line 69
  • scenarios/units_demo.xml: line 44
  • skirmishes/gallic_fields_(3).xml: line 98
  • tutorials/introductory_tutorial.xml: line 88
Nescio edited the summary of this revision. (Show Details)Feb 28 2018, 3:00 PM
vladislavbelov added inline comments.
binaries/data/mods/public/maps/skirmishes/Gallic Fields (3).xml
98

We need to backslash braces ((, )) on Linux and macOS too. May be we need to remove it too? @Itms

Nescio edited the summary of this revision. (Show Details)Feb 28 2018, 3:26 PM
Itms added inline comments.Mar 3 2018, 4:50 PM
binaries/data/mods/public/maps/skirmishes/Gallic Fields (3).xml
98

Braces are not nice either, I agree. But I'm not sure what would be a good naming convention for the number of players on skirmishes. Maybe gallic_fields_3p.xml?

Itms added inline comments.Mar 11 2018, 7:11 PM
binaries/data/mods/public/maps/skirmishes/Gallic Fields (3).xml
98

Got another idea, we could write gallic_fields-3.xml.

Currently all skirmish files include the number of players within parentheses in the file name, e.g. Acropolis Bay (2).xml Is this actually necessary? Most maps have just one version.
If those skirmish player numbers were removed, file names would be shorter and simultaneously more consistent with other maps, which don't include player numbers in their name either. There are just two cases to distinguish different versions of the same map:

scenarios/Azure Coast.xml
scenarios/Azure Coast(2).xml
scenarios/Azure Coast(4).xml
skirmishes/Corinthian Isthmus (2).xml
skirmishes/Corinthian Isthmus (4).xml
binaries/data/mods/public/maps/skirmishes/Gallic Fields (3).xml
98

Although I agree with removing parentheses and other special characters, I'm not sure replacing (3) with -3, _3, _3p, _3x, or something similar would be an improvement.

It might be better to omit the player number altogether.

Itms added a comment.Mar 12 2018, 2:42 PM
In D1042#56753, @Nescio wrote:

Currently all skirmish files include the number of players within parentheses in the file name, e.g. Acropolis Bay (2).xml Is this actually necessary? Most maps have just one version.

Since we do have maps with the issue, we still need to find something, so we might as well use that for all maps (the aim was to improve consistency after all).

Yes, the aim of this patch is to improve map files naming consistency and to remove inconvenient white spaces. However, random maps and scenarios do not include player numbers in the file name, only skirmishes do; furthermore, other data, such as map size, aren't included in the file name either. Removing the player numbers from the file names would thus improve naming consistency and also allow the removal of parentheses ().
Nevertheless, we still need to find a proper way to differentiate the "Azure Coast" scenarios and the "Corinthian Isthmus" skirmishes. Any suggestions, anyone?

Nescio added a comment.EditedJan 3 2019, 10:37 AM

Spaces and special characters in file names are still problematic, so it would be nice if we could simply use snake case for all map files.

To recapitulize, I believe the script:

  • replaces spaces with underscores
  • replaces hyphens with underscores
  • removes underscores when followed by an underscore
  • replaces capitals with lower case

Not yet decided:

  • what to do with _(2) and similar

Not yet decided:
what to do with _(2) and similar

It sounds easier to remove the teamdescriptions such (2) and (3v3), remove one of the map variants if there are two variants with identical pmp file (#5361), rename legitimate variants to have the difference not be reflected by the number of teams but an artistic title.
But perhaps there was a reason I didn't notice for the teamnumbers being part of the filename.

camel case

You mean lower case?

Oops, I meant snake case (azure_coast instead of Azure Coast or AzureCoast).

Nescio added a comment.EditedJan 5 2019, 9:00 PM

There are currently several sets of files which are distinguished by a number in their file name:

  • "scenarios/Azure Coast", " Coast(2)", and "Coast(4)"
  • "scenarios/Flight_demo" and "Flight_demo_2"
  • "scenarios/Sandbox - Ptolemies" and " Ptolemies 2"
  • "skirmishes/Corinthian Isthmus (2)" and " (4)"
  • "skirmishes/Cycladic Archipelago (2)" and " (3)"
  • "skirmishes/Greek Acropolis (2)" and " (4)"
  • "skirmishes/Median Oasis (2)" and " (4)"
  • "skirmishes/Neareastern Badlands (2)" and " (4)"

However, I don't know if there are more differences than just the number of players. How does one check?

Also, what to do with the "scenarios/Height Map *.png" files?

Stan added a subscriber: Stan.Jan 5 2019, 9:08 PM
In D1042#69408, @Nescio wrote:

However, I don't know if there are more differences than just the number of players. How does one check?
Also, what to do with the "scenarios/Height Map *.png" files?

Well you could open the maps in Atlas. But I believe that's the main difference.

Nescio added a comment.Jan 5 2019, 9:55 PM

Well you could open the maps in Atlas.

The Azure Coast scenarios seem to use the same map, so perhaps it's unnecessary to keep three versions.

The skirmishers are all different; the (4) versions are larger than the (2) versions, have a different layout (e.g. river vs no water) etc. So perhaps we should rename the (2)s to "small" and the (4)s to "large" if we want to get rid of the numbers.

Nescio edited the summary of this revision. (Show Details)Jan 26 2019, 1:11 PM

Still to be decided:
1 what to do with the scenarios/Height Map *.png files
2 what to do with the three Azure Coast scenarios (identical maps, different player numbers)
3 what to do with the two Flight_demo scenarios (different maps, similar function)
4 what to do with the two Sandbox - Ptolemies scenarios (different maps, similar function; all other factions have only one sandbox)
5 how to differentiate similary named skirmishes, e.g. Corinthian Isthmus (different maps, similar names)

How about the following?

  • 2, 3, 4: keep only one version, delete the others
  • 5: rename them to _small and _large
Itms added a comment.Feb 7 2019, 10:50 PM
In D1042#71075, @Nescio wrote:

Still to be decided:
1 what to do with the scenarios/Height Map *.png files
2 what to do with the three Azure Coast scenarios (identical maps, different player numbers)
3 what to do with the two Flight_demo scenarios (different maps, similar function)
4 what to do with the two Sandbox - Ptolemies scenarios (different maps, similar function; all other factions have only one sandbox)
5 how to differentiate similary named skirmishes, e.g. Corinthian Isthmus (different maps, similar names)

How about the following?

  • 2, 3, 4: keep only one version, delete the others
  • 5: rename them to _small and _large

I think 2) should be kept: it's nice, for skirmishes, to be able to choose depending on the number of players, instead of saying that Azure Coast is only for two players.

For 3 and 4 there should be merges indeed, but let's make sure we keep all the interesting things from both merged maps each time.

For 5, I would rather find a unique and appealing name allowing to discriminate the maps easily. Small/Large breaks immersion quite a bit.

elexis added a comment.Feb 8 2019, 8:24 PM

I think 2) should be kept: it's nice, for skirmishes, to be able to choose depending on the number of players, instead of saying that Azure Coast is only for two players.

The problem with that is that this
(a) it means it needs a copy of the terrain file for each instance (we have the file three times). For Azure Coast this case it's only 1.5mb, but other maps use 10mb
(b) if we already have the copy, why not add so much art to it, that the maps become an actually unique experience. For Azure Coast, one can say that the map terrain is so amazing and unique that one wants to play it in multiple scenarios. Notice that the entities are really unique (players start at the shore with a dock and a tower).
(c) if the argument is true in general, then it would apply to the other maps as well. So then we would be inviting to copy around many maps and just differ the playercount. But we don't want so many copied files, and we don't want to get the same map multiple times when we browse through the mapselection list (but rather get a map variant choice after having selected a map, if it's essentially the same map). Also there are random maps that can load *pmp files since a23, so I would encourage future devs to implement such map variants using that.
I'd be ok with it being a exemption though.

Probably agree with 3-5 although I didn't look at how most maps look. The first flight demo map looks bad and doesn't seem to have anything that the second one doesn't have.

For 5, I would rather find a unique and appealing name allowing to discriminate the maps easily. Small/Large breaks immersion quite a bit.

Well, I'm open for suggestions, of course. *_large and *_small are probably less problematic placeholders than * (2) and * (4). Also, this patch is about renaming the file names. The map names as displayed in game can be different and don't have to be changed.

1 what to do with the scenarios/Height Map *.png files

Something else, there are quite a few demo scenarios, e.g. Combat_demo and Demo_Trading. Perhaps it would be better to rename them so they all start with demo_ ? (Cf. sandbox maps.) Could be done in a separate patch.

Nescio edited the summary of this revision. (Show Details)Mar 17 2019, 10:02 PM
Nescio edited the summary of this revision. (Show Details)
Nescio edited the summary of this revision. (Show Details)Mar 17 2019, 10:20 PM

Updated script:

I'd give this a look but I'm kind of lost on the consensus that was attained here?

Stan added a comment.Apr 20 2019, 2:14 PM

Well if you listen to the art document there should be no spaces in any art files. I think maps fall in that category.

See the summary.
I believe there is consensus to rename the map files to 'snake_case', but that no decision has been made on what to do with the player numbers in map file names.

Stan added a comment.Apr 20 2019, 4:20 PM

Demo maps already have a tag in their XML files so we could omit the demo_ alltogether.
As for player numbers, either we could support subfolders that would contain the number of players like maps/scenarios/3/gallic_fields.xml. But we can't remove the number of players for all maps because some have the same name.

Actually I think removing demo and Demo from file names won't be an improvement, because it would result in potentially confusing map file names, e.g. combat, multiplayer, territory, etc.

Also, what to do with the height maps?

Stan added a comment.Apr 20 2019, 5:36 PM

Well heightmaps should have the same name as the map with a different extension ?

That's not currently the case, though.

Stan added a comment.Apr 20 2019, 6:08 PM

@elexis Do you have an opinion on renaming heightmaps to match their map name ? I guess the only trouble would be what to do in case two maps use the same heightmaps.

opinion on renaming heightmaps

I thought you mean the heightmaps that were used: rmgen, but they already have that naming pattern. But you mean those scenario/skirmish png copies that are not used by the game? Perhaps just move to art_source? Just adding a comment and link to the XML file instead of a copy of the source? (And maybe a reference to the commit IDs so that people can click from one to the other to find the source)

Yes, I mean the height maps in the maps/scenarios/ folder:

  • Height Map 03.png
  • Height Map Import Demo Image.png
  • Height Map Import Demo Image 512x512.png
  • Height Map Import Demo Image - Fractal.png
  • Height Map Import - Four Oases.png
  • Height Map Import - Oasis 12.png
Nescio added a reviewer: Restricted Owners Package.Jul 6 2019, 4:35 PM

1 what to do with the scenarios/Height Map *.png files

Height Map 03.png
Height Map Import Demo Image.png
Height Map Import Demo Image 512x512.png
Height Map Import Demo Image - Fractal.png
Height Map Import - Four Oases.png
Height Map Import - Oasis 12.png

The question is which purpose do they serve.
If they're in the public.zip, I suppose players dont actually see the files?
Can post copies of the files in some place that people look up, wiki page, forum post, trac ticket, and or phabricator page and or commit to art repository.

2 what to do with the three Azure Coast scenarios (identical maps, different player numbers)

Vladislav said parentheses need to be escaped, parentheses look ugly, Itms mapname-3p.xml proposal seemed sane.

3 what to do with the two Flight_demo scenarios (different maps, similar function)

The first one looks like it may be removed since the second one has everything that the first one has plus more?

4 what to do with the two Sandbox - Ptolemies scenarios (different maps, similar function; all other factions have only one sandbox)

From what I gather those are copies of existing maps plus a bunch of entities.
The first one comes from rP13909, the other from rP14388.
That's one useless megabyte I suppose, so delete the one you like less?
The task is to show all units, so perhaps one can also check which one is more up to date / complete.

5 how to differentiate similary named skirmishes, e.g. Corinthian Isthmus (different maps, similar names)

Technically one can play the 2v2 map was a 1v1 map by setting to unassigned (plus people were asking to have skirmish entities be deleted for unassigned players)?
Otherwise same as 2, and if someone wants to improve a map, they are free to?

Updated script:


Which removes Flight_demo.* and Sandbox - Ptolemies.* (because their second version looks much better), takes care of Azure Coast* scenarios, then performs the earlier agreed-upon map moves (removes hyphens and parentheses, replaces spaces with underscores, converts upper case into lower case).

what to do with the scenarios/Height Map *.png files

The question is which purpose do they serve.
If they're in the public.zip, I suppose players dont actually see the files?
Can post copies of the files in some place that people look up, wiki page, forum post, trac ticket, and or phabricator page and or commit to art repository.

I don't know what their purpose is. Where should they be moved to exactly? Or do you think they should simply be deleted?

Nescio edited the summary of this revision. (Show Details)Jul 17 2019, 10:53 AM
Nescio edited the summary of this revision. (Show Details)
Nescio edited the test plan for this revision. (Show Details)
Nescio edited the summary of this revision. (Show Details)Jul 17 2019, 10:55 AM
In D1042#87050, @Nescio wrote:

what to do with the scenarios/Height Map *.png files

I don't know what their purpose is. Where should they be moved to exactly? Or do you think they should simply be deleted?

The purpose can only be to load the heightmap in atlas again, or to document the heightmaps used, no?
Hiding them in public.zip will make the former one impossible, no? And the latter one doesn't require distribution in public.zip but can also be documented elsewhere.

Now where I look at them again, the licensing wasn't clear, I remember I asked the author as well.
Speaks more for erasure than to keep it to me.
If one wants such an example heightmap, it should at least be in a folder that the user can access without unzipping from public.zip.
There already is some online documentation on how to import heightmaps and where to obtain data sources from.
If one wants to add it offline, I don't disagree with that in advance, but the currently committed files don't seem to achieve that objective.

Agreed, let's delete those pngs then; updated script:

Nescio edited the summary of this revision. (Show Details)Jul 17 2019, 3:35 PM
Nescio edited the test plan for this revision. (Show Details)
Nescio edited the summary of this revision. (Show Details)
In D1042#75465, @Nescio wrote:

Yes, I mean the height maps in the maps/scenarios/ folder:

  • Height Map 03.png

rP12792 That's great britain

  • Height Map Import Demo Image.png

rP12381, rP12598 That's 1000x1000 greece, this might have been used for the 8 player greece scenario map. There is an according map in the commit.

  • Height Map Import Demo Image 512x512.png

rP12454 That's greece

  • Height Map Import Demo Image - Fractal.png

rP12515 512*512

  • Height Map Import - Four Oases.png

rP13764 256*256, poor quality

  • Height Map Import - Oasis 12.png

rP12792 256*256, poor quality. This looks like it might be skirmishes/Team Oasis (2v2)

Those files shouldn't just be deleted because noone will look at them in the public.zip, there was also a hidden purpoe for them.
They left us the information that there was something to audit, because the data clearly comes from some author, so the license is missing, so it might be a copyright problem.
This reminded me that I already asked the author and read through IRC logs to find the data source - but without success.
This was done in rP21915 / D1646 deleting the heightmap from rP15219 and its map.

So if one wants to touch these files, one should figure out whether they need to be deleted for copyright reasons and whether there is a map that was derived from it that needs to go with it (refs rP20362).

I looked through the thumbnails but Im not sure I see something that resembles the heightmaps.
One might find more in the revision history.
On the other hand, there might also be more maps that might have used unlicensed heightmaps that didnt have the pngs committed.
meh. It sounds like the solution will be some kind of delete, but the question will be what the reason will be, and which maps will be affected.

If I understand you correctly, you suggest those heightmaps to be deleted, but want to know on what grounds exactly?

I don't want anything, but files should serve a purpose, otherwise nuked. There are two possible purpose identified (1) atlas use for players and (2) documenting the heightmaps used for developers.
In both cases it doesn't seem to be right to have it in public.zip, since atlas doesn't provide that file to be selected (afaik) and (2) also doesn't mean that it has to be distributed in public.zip.
If we want to document the heightmap source for developers, it can be commented in the license file or in the mapfile.
Deleting the heightmaps would however cover up the fact that there might be a possible copyright infringement, so we should check if we need to delete not only the heightmap images but also maps using those licensed data.
For example that Greece heightmap looks like its used for the 8 player scenario greece map, is that now an illegal map or not?
If one could find out that its a specific NASA SRTM version, one can prove that the map is legitimate.
(Or one can delete it and create it from scratch, or delete it and do some other maps instead, or prepare a legal defense based on the four factors of Fair Use copyright exemption)

Those height maps aren't used, their copyright is unclear, and none of their commit messages says anything about them.
I just opened the three maps/scenarios/Height Map Import - Demo * files and they're all three blank maps without any entities. I'm not sure what they demonstrate; if one wants to know how to import height maps I suppose they could have a look at e.g. maps/random/hellas.js. So perhaps delete all those Height Map scenario files (png, xml, and pmp)?

Updated script

now deletes all those scenarios/Height Map * files (png, pmp, and xml), because they don't seem to add anything and their copyright is apparently unclear.

Nescio edited the summary of this revision. (Show Details)Jul 24 2019, 3:31 PM

I would /will have to compare the images against the maps, look up the commits, check forum posts on copyright complaints (I remember there was at least one),
because the heightmap images may have been also used for scenario maps.

For example that Greece heightmap looks like its used for the 8 player scenario greece map, is that now an illegal map or not?

I meant the three Azure Coast scenario maps. But seems like the image would have to have been mirrored, rotated by 90°, cropped and then still the terrain modified.

Now I'm confused. What has the Cote d'Azur to do with Greece?

Nescio edited the test plan for this revision. (Show Details)Aug 31 2019, 10:21 AM
Stan added a comment.Aug 31 2019, 11:02 AM

Maybe we could split the heightmap discussion in another diff and just commit the map name fixes.

It was actually @elexis' suggestion to remove those scenario height maps.

Stan edited reviewers, added: Stan; removed: elexis.Nov 26 2019, 11:47 AM
Stan added a comment.Nov 26 2019, 12:02 PM

Okay so I tried the script in the WSL (Windows Subsystem for Linux) As I expected it totally failed because Windows disks are not case sensitive.

Nescio added a subscriber: elexis.

Then let's hope another team member (@elexis?) will eventually find time to review this.

@elexis, it would be great if you could continue your review of this patch at some point. Spaces etc. in map file names still cause problems with Vulcan when uploading patches to phabricator.

@Itms, could you finish the review of this patch? I'm asking you because you participated earlier in the discussion and helped formulate the rename script.
This patch is from 2017 yet still relevant. I initially hoped it could make it in for A23, which didn't happen, but it would still be a significant improvement if map file names could be fixed before A24 is released.

bb accepted this revision.May 31 2020, 9:05 PM
bb added a subscriber: bb.

The name of Flight_demo_2 should have been adapted.

scenario's with parenthesis should get a treatment too

Not removing the heightmap demos or the ptol sandbox, they should get an own revision.

binaries/data/mods/public/maps/skirmishes/Gallic Fields (3).xml
98

We should avoid hyphens and parenthesis. Going for _3p.
(Considering https://trac.wildfiregames.com/wiki/ArtFileNamingConventions nothing proposed is 100% correct, discussion on irc today lead to this choice)

This revision is now accepted and ready to land.May 31 2020, 9:05 PM

(It's accepted but not closed.)

Stan added a comment.Jun 1 2020, 12:29 PM

Presumably because @bb wrote Differential Revision: D1042 https://code.wildfiregames.com/rP23719 instead of Differential Revision: https://code.wildfiregames.com/D1042. We'll wai for his input, but I think you can close it manually.

bb added a comment.Jun 1 2020, 4:22 PM

The revision should have closed, even though I wrote D1042 and not the full link (there are plenty examples in the commit history where the shorthand does work). However phab also didn't add an edge here, so seems to be some hickup in phab. @Nescio can you close it manually?

Nescio closed this revision.Jun 1 2020, 5:37 PM
This revision was landed with ongoing or failed builds.Jun 2 2020, 4:47 AM