Back to the original diff again.

Removed changes to LOGMESSAGERENDER.

rP28162 reverts this patch again as it had some unintended consequences. We'll try again with a different one to fix #5563.

In D5314#226863, @Itms wrote:

LOGMESSAGERENDER is, contrary to LOGMESSAGE, supposed to be displayed to the user, not only in the logs. So it would probably make sense to keep translating them.

Update the text used as context.

In D5309#226820, @Itms wrote:

Yes it should have no impact on the actual translation, but I would like to make sure tinygettext doesn't choke on strange unicode delimiters when parsing pot files.

In D5309#226798, @Itms wrote:

Does tinygettext properly handle the unicode whitespace delimiters or should we patch it as well?

Rebase against current trunk and skipping of the "debug" translation as well.

In D5308#226795, @Itms wrote:

Did the empty strings actually appear in Transifex?

Simplify regex

In D5311#226673, @Stan wrote:

Just to be sure, we can't currently use STUN for LAN/WAN games, can we?

In D5309#226621, @Stan wrote:

If we're looking at filenames with tabs shouldn't we look for any whitespace?

Also enclose file names containing tabs, as that's what gettext does as well. I also opened a pull request for Babel (https://github.com/python-babel/babel/pull/1105), so they do it if necessary. If this pull request gets accepted running updateTemplates.py with upcoming versions of Babel wouldn't need the special handling for enclosing the file name in our code. As the pull requests isn't accepted yet and for backwards compatibility enclosing file names ourselves still makes sense.

Fixes that not just the white spaces is be enclosed by U+2068 and U+2069. but the whole file name. The specification wasn't entirely clear about that, but gettext itself does enclose the whole file name.

In D5309#226602, @Stan wrote:

Hmm okay. We currently have some issues with identical msgids (Bad translations getting picked up), does that make any difference if they are in the same file with your change?

In D5309#226593, @Stan wrote:

Does that mean any change to those jsons will break all the translations?

Simplify the regex a bit. \s already matches \n.

Remove empty strings not just for XML files, but for every file type.

In D5308#226528, @Stan wrote:

A good question is where are such strings generated? Anyway sounds good to ignore them

In D5286#226423, @Stan wrote:

This should be checked under the minimum 1024x768 resolution we currently support.

In D5286#225319, @sera wrote:

For instance single-player and multiplayer replay are the same, just a different default for one filter.

@Stan: Any additional thoughts about this?

In D5304#226385, @Stan wrote:

Maybe you could sort to put title Names first ?

Reverse order when sorting translators for the first time, to prefer all-lowercase ("john doe") over all-uppercase ("JOHN DOE").

Removed the manual second rule for name ordering and made it robust against a changing order of translators when the order of the list of translators returned by Transifex changes.

In D5304#226358, @Stan wrote:

Should cover most cases. Not sure wether we prefer having FIRSTNAME Lastname over firstname lastname. Other than that looks good.

In D5303#226362, @Stan wrote:

Do we really care about Dennis lint for long translations?

Fix usage of isupper().

Updated copyright year.

In D5301#226286, @Itms wrote:

Hello Dunedan, after the git migration, the translation scripts will be run on Windows as part of the nightly build generation. So the Docker image for translations will be dropped. See 2f38a3d67e.

In D5301#226283, @Stan wrote:

What are the new features we need?

Update copyright year.

In D5300#226221, @bb wrote:

See the reason for the current (yes ugly) list https://trac.wildfiregames.com/ticket/3827

By the looks of things this diff reopens the loophole.

In D5270#226194, @Stan wrote:

Well, if you tested the --without-lobby flag

These patches don't seem to work on Debian and derivatives, as the wheels get installed into libraries/source/spidermonkey/mozjs-91.13.1/build-debug/_virtualenvs/common/lib/python3.12/dist-packages, but _disable_pip_outdated_warning() in virtualenv.py expects them in libraries/source/spidermonkey/mozjs-91.13.1/build-debug/_virtualenvs/common/lib/python3.12/site-packages (as that's what sysconfig.get_path(…) returns).

@Stan: I got confused why you are set as author and I am as reviewer of this diff, while it should be the other way around. Then I remembered how that happened.
Anyway, from my perspective this is fine and I'd like to commit it. What about you? Any additional thoughts?

In D5299#226177, @Stan wrote:

True. I noticed recently that some translator names are anonymized as hashes, do we have a way to handle those cases? In theory one would have to check manually during the releases. Hopefully that will not cause useless changes.

In D5299#226172, @Stan wrote:

Looks good on paper, but since the translation will no longer be in the git repo but a svn repo, I wonder if it will not break stuff.

In D5298#226171, @Stan wrote:

Regex looks okay but are we sure it handles all the transifex cases ?

In D5290#225504, @Stan wrote:

IIRC @Itms touched those files for the Gitea migration, he might have a look.

In rP28127#62214, @Stan wrote:

Sometimes we I release I have to patch a lot of occurences for all mainstream languages. A bit annoying but you gotta do what you can

In rP28127#62209, @Stan wrote:

Looks like it's fixed @Dunedan :)

In D5293#225609, @Dunedan wrote:

Sure. Since Babel 2.12 no functional changes have been made to the JavaScript-lexer anymore, but in contrast to the latest version Babel 2.12 still supports Python 3.7. We need support for Python 3.7 because the updateTemplates.py script is run as part of CI in a Docker container based on Debian Buster, which ships with Python 3.7.

In D5293#225603, @sera wrote:

The summary doesn't say why exactly 2.12 and looking at https://babel.pocoo.org/en/latest/changelog.html I don't see an obvious reason either. Can you please clarify?

In D5293#225601, @Stan wrote:

One think I'm not sure about is the current babel version in the dockerfile. In theory it should be latest, but I don't remember if we use pip or the distro provided one.

I made some progress. I was indeed missing something and that was that there was a change to binaries/data/mods/public/l10n/public-tutorials.pot. With that I was able to figure out that the JavaScript-lexer (source/tools/i18n/extractors/jslexer.py) used by updateTemplates.py was just missing handling for the hex notation for unicode strings. That's something which got added upstream a while ago (https://github.com/python-babel/babel/pull/877) and easy for us to pull in as well. However, that only fixes the templates for Transifex, not how the strings are displayed in 0 A.D..

In rP28109#62171, @Stan wrote:

The translation pulling script is running in a docker.

https://trac.wildfiregames.com/browser/ps/trunk/build/jenkins/dockerfiles/translations.Dockerfile

You can see the commands here, maybe it's the checkdiff or cleanTranslationFiles that messes it up
https://trac.wildfiregames.com/browser/ps/trunk/build/jenkins/pipelines/docker-translations.Jenkinsfile

In rP28109#62171, @Stan wrote:

maybe it's the checkdiff or cleanTranslationFiles that messes it up

In rP28109#62164, @Stan wrote:

Sounds like an encoding problem?

In rP28109#62162, @Stan wrote:

We update the pot when we update the translations, but transifex pulls strings from the repo every four hours or so IIRC.

In rP28109#62160, @Dunedan wrote:

Todays push of updated of the strings to Transifex broke the one touched by this commit. It's now shown as Welcome to the 0xa0A.D. tutorial. there. So we need another round of fixing that.

Todays push of updated of the strings to Transifex broke the one touched by this commit. It's now shown as Welcome to the 0xa0A.D. tutorial. there. So we need another round of fixing that.