Page MenuHomeWildfire Games

Add commandline flag to set source and execution charsets to utf-8
ClosedPublic

Authored by Itms on Mar 23 2018, 5:36 PM.

Details

Reviewers
Inari
Silier
Group Reviewers
Restricted Owners Package(Owns No Changed Paths)
Commits
rP23383: Set source and executable character sets to UTF-8 on Windows, removes a…
Trac Tickets
#5094
Summary

Add /utf-8 as a commandline compiler flag. Only supported in VS2015, so can't make it work for VS2013.

Sets execution and source charsets to utf-8, preventing issues with Non-English Locale settings (e.g. Japanese or Chinese) on Windows. See #5094 for further info.

Test Plan

Update workspaces and see if /utf-8 pops up in Project properties -> C/C++ -> Command Line.

If non-English locale, test if building works and does not throw any errors like "illegal escape sequence" or "newline in constant"

Diff Detail

Repository
rP 0 A.D. Public Repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Inari created this revision.Mar 23 2018, 5:36 PM
Owners added a subscriber: Restricted Owners Package.Mar 23 2018, 5:36 PM
Inari edited the summary of this revision. (Show Details)Mar 23 2018, 5:41 PM
Inari changed the visibility from "Public (No Login Required)" to "No One".
Inari updated the Trac tickets for this revision.
Inari updated this revision to Diff 6264.Mar 23 2018, 5:46 PM
Inari changed the visibility from "No One" to "Inari (Inari)".
Inari changed the edit policy from "All Users" to "Inari (Inari)".
Inari edited the summary of this revision. (Show Details)Mar 23 2018, 5:55 PM
Inari changed the visibility from "Inari (Inari)" to "Public (No Login Required)".
Inari changed the edit policy from "Inari (Inari)" to "All Users".
Inari updated the Trac tickets for this revision.
vladislavbelov added reviewers: Itms, Restricted Owners Package.Mar 27 2018, 4:53 PM
wraitii resigned from this revision.Apr 20 2019, 4:43 PM
wraitii removed a reviewer: wraitii.
Stan added a subscriber: Stan.Apr 20 2019, 5:46 PM

@Itms anything wrong with that patch ?

Stan added a reviewer: Silier.Sep 24 2019, 4:18 PM
Silier requested changes to this revision.Oct 31 2019, 6:11 PM

Sorry it took so long.
Tests failed with this patch applied while they pass without it.

      <failure file="F:\Jenkins\workspace\0adtest\source\scriptinterface\tests\test_ScriptConversions.h" line="69" type="failedAssertEquals" >Error: Expected (std::string(source) == std::string(expected)), found (&quot;\&quot;\\u0442\\u0435\\u0441\\u0442\&quot;&quot; != &quot;\&quot;\\xD1\\u201A\\xD0\\xB5\\xD1\\x81\\xD1\\u201A\&quot;&quot;)
Error: Expected (std::string(source) == std::string(expected)), found (&quot;\&quot;\\u0442\\x00\\u0441\\u0442\&quot;&quot; != &quot;\&quot;\\xD1\\x00\\xD0\\xB5\\xD1\\x81\\xD1\\u201A\&quot;&quot;)</failure>
    </testcase>
This revision now requires changes to proceed.Oct 31 2019, 6:11 PM
Itms commandeered this revision.Dec 30 2019, 6:24 PM
Itms edited reviewers, added: Inari; removed: Itms.

Thanks again for the patch Inari. I am commandeering it to fix the tests as well. I will commit this just after D2524.

@Angen As you can see the current tests are testing that Windows does something specific and different from all other platforms; the change by Inari removes the discrepancy.

Itms updated this revision to Diff 10829.Dec 30 2019, 6:26 PM

Tests fixed.

I removed the comment in the lua file but I can add it back, or change it, if you feel like this needs a comment.

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/vs2015-differential/915/display/redirect

Build failure - The Moirai have given mortals hearts that can endure.

Link to build: https://jenkins.wildfiregames.com/job/docker-differential/1431/display/redirect

Successful build - Chance fights ever on the side of the prudent.

Link to build: https://jenkins.wildfiregames.com/job/macos-differential/9/display/redirect

Build failure - The Moirai have given mortals hearts that can endure.

Link to build: https://jenkins.wildfiregames.com/job/docker-differential/1432/display/redirect

Successful build - Chance fights ever on the side of the prudent.

Linter detected issues:
Executing section Source...

source/scriptinterface/tests/test_ScriptConversions.h
|  31| class·TestScriptConversions·:·public·CxxTest::TestSuite
|    | [MAJOR] CPPCheckBear (syntaxError):
|    | Code 'classTestScriptConversions:' is invalid C code. Use --std or --language to configure the language.
Executing section JS...
Executing section cli...

Link to build: https://jenkins.wildfiregames.com/job/docker-differential/1433/display/redirect

Silier resigned from this revision.Dec 31 2019, 2:40 PM

I can confirm it is building on windows with non english local settings, but I cannot test build with Chinese or Japanese locals. Maybe ask gameboy if he can try to build this one.

This revision was not accepted when it landed; it landed in state Needs Review.Jan 12 2020, 9:20 PM
This revision was automatically updated to reflect the committed changes.