Page MenuHomeWildfire Games

Migrate to premake5
ClosedPublic

Authored by Itms on Jan 15 2017, 7:44 PM.

Details

Summary

This is a revision for testing https://github.com/na-Itms/0ad/tree/premake5.
Review will be eased by using the github branch. Also, this revision doesn't include the new premake files, in order to avoid cluttering Phabricator...

This revision is mainly aimed at automatically testing that the premake5 support doesn't break anything with premake4. It is also a small proof-of-concept of how we can integrate build system changes in the new Phabricator workflow.

Test Plan

Test on *nix, Mac OSX, Windows (I put one reviewer by system ?)

Diff Detail

Repository
rP 0 A.D. Public Repository
Lint
Automatic diff as part of commit; lint not applicable.
Unit
Automatic diff as part of commit; unit tests not applicable.

Event Timeline

Itms updated this revision to Diff 248.Jan 15 2017, 7:44 PM
Itms retitled this revision from to Migrate to premake5 (part 1 - support premake5).
Itms updated this object.
Itms edited the test plan for this revision. (Show Details)
Itms added reviewers: leper, wraitii, Yves.
Itms set the repository for this revision to rP 0 A.D. Public Repository.
Itms updated the Trac tickets for this revision.
Owners added a subscriber: Restricted Owners Package.Jan 15 2017, 7:44 PM
wraitii requested changes to this revision.Jan 15 2017, 7:54 PM
wraitii edited edge metadata.

I get this error calling update-workspaces.sh --premake5

==== Building lua-lib (release) ====
==== Building zip-lib (release) ====
==== Building zlib-lib (release) ====
==== Building curl-lib (release) ====
asyn-thread.c
base64.c
In file included from ../../contrib/curl/lib/base64.c:26:
In file included from ../../contrib/curl/lib/urldata.h:82:
In file included from ../../contrib/curl/lib/cookie.h:26:
../../contrib/curl/include/curl/curl.h:34:10: fatal error: 'curlbuild.h' file not found
#include "curlbuild.h"       /* libcurl build definitions */
         ^
In file included from ../../contrib/curl/lib/asyn-thread.c:60:
In file included from ../../contrib/curl/lib/urldata.h:82:
In file included from ../../contrib/curl/lib/cookie.h:26:
../../contrib/curl/include/curl/curl.h:34:10: fatal error: 'curlbuild.h' file not found
#include "curlbuild.h"       /* libcurl build definitions */
         ^
1 error generated.
1 error generated.
make[1]: *** [obj/Release/curl-lib/asyn-thread.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: *** [obj/Release/curl-lib/base64.o] Error 1
make: *** [curl-lib] Error 2
ERROR: Premake build failed

Do we need to recompile curl though? we already do from the build scripts.

This revision now requires changes to proceed.Jan 15 2017, 7:54 PM
leper added inline comments.Jan 15 2017, 9:00 PM
build/workspaces/update-workspaces.sh
127 ↗(On Diff #248)

You can just have `"$premake_version/bin/release/$premake_version".

That said I'm not sure if we should actually support both systems in parallel.

Vulcan added a subscriber: Vulcan.Jan 15 2017, 9:52 PM

Build is green

Updating workspaces.
Build (release)...
Build (debug)...
Running debug tests...
Running cxxtest tests (302 tests)..............................................................................................................................................................................................................................................................................................................OK!

http://jw:8080/job/phabricator/179/ for more details.

Itms changed the visibility from "All Users" to "Public (No Login Required)".Jan 23 2017, 12:27 PM
Itms updated this revision to Diff 342.Jan 25 2017, 7:12 PM
Itms edited edge metadata.

I fixed a number of issues I stumbled upon on Unix (including big issues with parallel builds that are more hacked away than fixed). That makes me more convinced that premake5 shouldn't become the default just now on non-Windows. However, as we want to move to VS2015 as soon as possible, I would agree on making it the default on Windows (which is the platform with the higher support and the least amount of bugs in premake5 AFAICS).

I didn't test my changes still work on Windows but it should, I'll check that tonight.

For reviewing, the git branch has been updated with new commits without a squashing, so you can review the novelties.

Build is green

Updating workspaces.
Build (release)...
Build (debug)...
Running debug tests...
Running cxxtest tests (302 tests)..............................................................................................................................................................................................................................................................................................................OK!

http://jw:8080/job/phabricator/245/ for more details.

leper edited edge metadata.Jan 26 2017, 12:38 AM

I'd gladly take the NASM removal before the rest of this diff.

Any work being done on upstreaming some of the modules (cxxtest, ...)? During the 4.4betaSomething and 5.0alpha (or maybe even beta) something days upstream seemed quite responsible and happy about contributions.

Itms added a comment.Jan 26 2017, 1:22 AM
In D72#3265, @leper wrote:

I'd gladly take the NASM removal before the rest of this diff.

Sure! I created D97 (and I could play with dependent revisions this way). I'll rebase this one tomorrow.

Any work being done on upstreaming some of the modules (cxxtest, ...)? During the 4.4betaSomething and 5.0alpha (or maybe even beta) something days upstream seemed quite responsible and happy about contributions.

As far as I understand https://github.com/premake/premake-core/wiki/Sharing-Your-Module, the way to go would be having a GitHub repository (0ad/premake-cxxtest) and sharing it with the community, but it wouldn't be included in premake releases. It would be nice to do that I think, at least for getting feedback from premake devs, but I don't think our current implementation is generic enough to serve as a full-fledged CxxTest premake module. The first things I'd rather do would be sending PRs upstream to address the hack I made in the cxxtest module (and other little things I spotted).

Sure! I created D97 (and I could play with dependent revisions this way). I'll rebase this one tomorrow.

Thanks, that feature might sound useful for a few things I do have lying around. Does this result in the dependency patches being applied first, or just to link diffs?

As far as I understand https://github.com/premake/premake-core/wiki/Sharing-Your-Module, the way to go would be having a GitHub repository (0ad/premake-cxxtest) and sharing it with the community, but it wouldn't be included in premake releases. It would be nice to do that I think, at least for getting feedback from premake devs, but I don't think our current implementation is generic enough to serve as a full-fledged CxxTest premake module. The first things I'd rather do would be sending PRs upstream to address the hack I made in the cxxtest module (and other little things I spotted).

Ah, I guess it might still be useful to get it into a state where others could use it (and we could get distros to include that so those don't have to use a bundled binary (or compile it) just to build the game. (Sort of a long-term plan would be to require premake as a build dependency on non-win32 instead of bundling it.)

wraitii requested changes to this revision.Jan 27 2017, 1:15 PM

Still getting the same curl build error on OSX with Premake5 mode, premake4 works.

This revision now requires changes to proceed.Jan 27 2017, 1:15 PM
Itms updated this revision to Diff 511.Feb 11 2017, 5:14 PM
Itms edited edge metadata.

Here is an updated version!

I rebased, fixed a couple issues and made premake5 the default on Windows (support for VS2015 itself requires some work but I already did the most difficult part of it on a branch of mine).

I also fixed wraitii's issue, which comes from a gitignore problem for which I warned upstream: https://github.com/premake/premake-core/issues/693

@wraitii : I get an issue with Objective-C when building on my Hackintosh after a successful premake run. Can you see if it's easily fixable? If it isn't I can take a look. I don't think it comes from clang, but from my dev environment itself, so I'd like to have this tested on a real Mac.

Build is green

Updating workspaces.
Build (release)...
Build (debug)...
Running debug tests...
Running cxxtest tests (302 tests)..............................................................................................................................................................................................................................................................................................................OK!

http://jw:8080/job/phabricator/349/ for more details.

wraitii added a comment.EditedApr 30 2017, 11:58 AM

Works under OSX with the following changes:
-PCH (see inline)
-Most "includedirs" calls need to be changed to "sysincludedirs". Premake5, unlike Premake4, uses includedirs for user-header only and sysincludedir for regular header search path (on Xcode at least). So all the include using "<>" end up failing. Not sure if that affects other platforms but it seems sensible overall.
-Collada fails to link in debug mode (it's looking for the non-debug library) - might just be a config issue.

I think the only include that needs to remain an includedirs instead of a sysincludedirs is the include to the PCH header and the include to source_root.

Edit: what's your use of pig-config exactly? I notice it links against my Brew-SDL2 instead of the currently linked one for example.

Edit: oh yeah and Atlas fails to compile, seems to be a linking issue, will check.

build/premake/premake5.lua
453 ↗(On Diff #511)

Xcode seems to actually expect "../"..pch_dir.."precompiled.h" . This sets a header param in Xcode which is relative to the workspace directory, not premake (whereas the linking is). Not sure why it works under Premake4 tbh :p

This revision now requires changes to proceed.Apr 30 2017, 11:58 AM

Found the atlas issue: it's related to linking system frameworks which has changed under premake5.

build/premake/extern_libs5.lua
103 ↗(On Diff #511)

This is no longer true (see https://github.com/premake/premake-core/issues/196 ). I'm not sure we need special code or not since I don't think we have external-libs that are frameworks (as far as I can tell).

build/premake/premake5.lua
1005 ↗(On Diff #511)

This needs to be

links { "ApplicationServices.framework", "Cocoa.framework", "CoreFoundation.framework" }
1255 ↗(On Diff #511)

This needs to be

linkoptions { "CoreServices.framework" }
1255 ↗(On Diff #511)

sorry I mean "links" here.

echotangoecho added a subscriber: echotangoecho.EditedMay 14 2017, 10:51 AM

Would be nice if we could move to VS2015 soon and use the full range of C++11 features, good patch!

build/premake/premake5.lua
278 ↗(On Diff #511)

-std=c++11?

Would be nice if we could move to VS2015 soon and use the full range of C++11 features, good patch!

We won't support full C++11 only with VS2015, we need to drop old gcc too.

Would be nice if we could move to VS2015 soon and use the full range of C++11 features, good patch!

We won't support full C++11 only with VS2015, we need to drop old gcc too.

Actually we did already, see also https://trac.wildfiregames.com/wiki/CppSupport

vladislavbelov added a comment.EditedMay 14 2017, 12:30 PM

Would be nice if we could move to VS2015 soon and use the full range of C++11 features, good patch!

We won't support full C++11 only with VS2015, we need to drop old gcc too.

Actually we did already, see also https://trac.wildfiregames.com/wiki/CppSupport

No, we don't, we still support 4.8.1. There was a fix, because this version doesn't support default/delete ctors/dtors in headers.

Itms updated this revision to Diff 2078.May 21 2017, 11:24 AM
Itms retitled this revision from Migrate to premake5 (part 1 - support premake5) to Migrate to premake5.
Itms edited the test plan for this revision. (Show Details)
Itms removed a subscriber: Restricted Owners Package.

Import the changes from the github branch. I also convinced myself we should switch to premake5 and keep an option to use the legacy premake4, instead of supporting premake5 and making it the default later.

I'd still be in favor of using the --premake4 option in CI scripts so that developers can discover possible bugs without having our review process disrupted.

@wraitii, can you summarize what are the remaining issues on OSX? Make sure you don't use your brew hacks at all, just the regular build system ?

Build has FAILED

Updating workspaces.

Link to build: http://jw:8080/job/phabricator/1282/
See console output for more information: http://jw:8080/job/phabricator/1282/console

I'd still be in favor of using the --premake4 option in CI scripts so that developers can discover possible bugs without having our review process disrupted.

Sounds reasonable.

Could you check what that build failure was, and also look into why those files aren't detected as mostly copies? (They were detected as such in some previous iteration.)

build/workspaces/update-workspaces.sh
110 ↗(On Diff #2078)

Judging by some code above this looks like it will break.

130 ↗(On Diff #2078)

Is this currently used by someone? (@Yves?) If yes, why not do that for premake5 too?

135 ↗(On Diff #2078)

Do we still care about xcode3? Or can we just drop this now instead of later?

Itms added a comment.May 21 2017, 6:33 PM
In D72#21403, @leper wrote:

Could you check what that build failure was, and also look into why those files aren't detected as mostly copies? (They were detected as such in some previous iteration.)

The failure was obviously because I don't attach premake5 to that differential, so making premake5 the default in it has a downside ? And I can't pass --premake4 to the CI scripts yet because the argument isn't implemented ?
The failing copy detection might be me not using svn cp (but I don't remember doing so in a previous iteration, my memory might be failing)

build/workspaces/update-workspaces.sh
110 ↗(On Diff #2078)

I can't see which code but I did want to install a BSD and test things so that might be a good opportunity.

130 ↗(On Diff #2078)

Premake 5 still doesn't have support for codeblocks (even through official and semi-official modules). We could either drop the support or just wait for a module, letting codeblocks users use premake 4.

135 ↗(On Diff #2078)

No idea. I suppose we can drop it but I don't know whether the situation "have xcode3, can't update to xcode4" is frequent.

leper added inline comments.May 21 2017, 6:56 PM
build/workspaces/clean-workspaces.sh
48 ↗(On Diff #2078)

This part, there is no .bsd makefile here.

I'll check the OSX version sometimes next week.

build/workspaces/update-workspaces.sh
135 ↗(On Diff #2078)

We're technically on Xcode 8 now, so xCode 3 should be considered "extremely legacy" and can safely be dropped imo.

Created a PR on your GitHub for the required OSX changes, which are few. However wx-widgets won't link because wx-config returns broken linker flags (tested with brew AND with the provided libs, which BTW we need to update to 3.0.3). Specifically the "-framework" flags don't really work anymore, so we need to figure out something here, maybe a regex to remove them.

wraitii requested changes to this revision.Jun 24 2017, 11:42 AM

administration

This revision now requires changes to proceed.Jun 24 2017, 11:42 AM
Itms updated this revision to Diff 3996.Oct 28 2017, 6:55 PM

Update to the latest version and include the planned change to the Jenkins script which should theoretically work.

Owners added a subscriber: Restricted Owners Package.Oct 28 2017, 6:55 PM
wraitii accepted this revision.Oct 28 2017, 7:05 PM

Tested on OSX with Xcode and clang and both worked.

This revision is now accepted and ready to land.Oct 28 2017, 7:05 PM

Build is green

Updating workspaces...
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDAnimation]’:
FCollada/FCDocument/FCDLibrary.cpp:149:30:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
  const T* cptr = ((const FCDLibrary<T>*)l1)->GetEntity(0);
           ^
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDAnimationClip]’:
FCollada/FCDocument/FCDLibrary.cpp:150:34:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDCamera]’:
FCollada/FCDocument/FCDLibrary.cpp:151:27:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDController]’:
FCollada/FCDocument/FCDLibrary.cpp:152:31:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDEffect]’:
FCollada/FCDocument/FCDLibrary.cpp:153:27:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDEmitter]’:
FCollada/FCDocument/FCDLibrary.cpp:154:28:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDForceField]’:
FCollada/FCDocument/FCDLibrary.cpp:155:31:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDGeometry]’:
FCollada/FCDocument/FCDLibrary.cpp:156:29:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDImage]’:
FCollada/FCDocument/FCDLibrary.cpp:157:26:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDLight]’:
FCollada/FCDocument/FCDLibrary.cpp:158:26:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDMaterial]’:
FCollada/FCDocument/FCDLibrary.cpp:159:29:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDSceneNode]’:
FCollada/FCDocument/FCDLibrary.cpp:160:30:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDPhysicsModel]’:
FCollada/FCDocument/FCDLibrary.cpp:161:33:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ set but not used [-Wunused-but-set-variable]
FCollada/FCDocument/FCDLibrary.cpp: In instantiation of ‘void LibraryExport() [with T = FCDPhysicsMaterial]’:
FCollada/FCDocument/FCDLibrary.cpp:162:36:   required from here
FCollada/FCDocument/FCDLibrary.cpp:141:11: warning: variable ‘cptr’ s

http://jenkins-master:8080/job/phabricator/2176/ for more details.

Executing section Default...
Executing section Source...
Executing section JS...

http://jenkins-master:8080/job/phabricator_lint/640/ for more details.

leper added inline comments.Oct 29 2017, 3:44 AM
build/workspaces/update-workspaces.sh
55 ↗(On Diff #3996)

Move the --collada part up here?

Itms updated this revision to Diff 4011.Oct 29 2017, 12:36 PM

Follow leper's suggestion.
This version is identical to https://github.com/na-Itms/0ad/tree/premake5 now that rP20366 has propagated to the git mirror.

Itms marked an inline comment as done.Oct 29 2017, 12:38 PM
Itms added inline comments.
build/workspaces/update-workspaces.sh
55 ↗(On Diff #3996)

Yessir!

Build is green

Updating workspaces...
../../contrib/luashim/luashim.c: In function ‘shimInitialize’:
../../contrib/luashim/luashim.c:854:32: warning: multi-character character constant [-Wmultichar]
  const Node* n = findNode(reg, 'SHIM');
                                ^
In file included from ../../contrib/curl/include/curl/curl.h:2523:0,
                 from ../../src/host/curl_utils.h:15,
                 from ../../src/host/curl_utils.c:8:
../../src/host/curl_utils.c: In function ‘curlRequest’:
../../contrib/curl/include/curl/typecheck-gcc.h:56:9: warning: call to ‘_curl_easy_setopt_err_write_callback’ declared with attribute warning: curl_easy_setopt expects a curl_write_callback argument for this option
         _curl_easy_setopt_err_write_callback();                               \
         ^
../../src/host/curl_utils.c:175:2: note: in expansion of macro ‘curl_easy_setopt’
  curl_easy_setopt(curl, CURLOPT_WRITEFUNCTION, curlWriteCallback);
  ^
../../contrib/curl/include/curl/typecheck-gcc.h:71:9: warning: call to ‘_curl_easy_setopt_err_progress_cb’ declared with attribute warning: curl_easy_setopt expects a curl_progress_callback argument for this option
         _curl_easy_setopt_err_progress_cb();                                  \
         ^
../../src/host/curl_utils.c:181:3: note: in expansion of macro ‘curl_easy_setopt’
   curl_easy_setopt(curl, CURLOPT_PROGRESSFUNCTION, curlProgressCallback);
   ^
../../src/host/premake.c: In function ‘premake_init’:
../../src/host/premake.c:194:36: warning: multi-character character constant [-Wmultichar]
  lua_rawseti(L, LUA_REGISTRYINDEX, 'SHIM');
                                    ^
Build (release)...
Build (debug)...
Running release tests...
Running cxxtest tests (307 tests)...................................................................................................................................................................................................................................................................................................................OK!
Running debug tests...
Running cxxtest tests (307 tests)...................................................................................................................................................................................................................................................................................................................OK!
Checking XML files...

http://jenkins-master:8080/job/phabricator/2185/ for more details.

Executing section Default...
Executing section Source...
Executing section JS...

http://jenkins-master:8080/job/phabricator_lint/642/ for more details.

This revision was automatically updated to reflect the committed changes.
Itms marked an inline comment as done.
Silier added a subscriber: Silier.Jan 14 2019, 5:33 PM
Silier added inline comments.
ps/trunk/build/premake/premake5.lua
1453

this makes collada as start project, could be pyrogenesis start project pls ? it is annoying