This includes an upstream PR for fixing macOS build on recent macOS (more recent than the Jenkins slave), and a new patch for non-aarch64 arm (typically raspberry pi).
Fixes current issues reported on rP23305.
Details
Details
- Reviewers
Krinkle Stan wraitii - Commits
- rP24245: Fix recent macOS builds.
- Trac Tickets
- #4549
Test on macOS and Raspberry Pi, and check that the new va_list initializer works on other compilers (no issue on MSVC).
Diff Detail
Diff Detail
- Repository
- rP 0 A.D. Public Repository
- Branch
- /ps/trunk
- Lint
No Linters Available - Unit
No Unit Test Coverage - Build Status
Buildable 12685 Build 24580: Vulcan Build Jenkins Build 24579: Vulcan Build (macOS) Jenkins Build 24578: Vulcan Build (Windows) Jenkins Build 24577: arc lint + arc unit
Event Timeline
Comment Actions
Build failure - The Moirai have given mortals hearts that can endure.
Link to build: https://jenkins.wildfiregames.com/job/docker-differential/1534/display/redirect
Comment Actions
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/vs2015-differential/1016/display/redirect
Comment Actions
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/macos-differential/112/display/redirect
Comment Actions
This fixes NVTT build however I still get for some images:
Warning: Cannot convert string "vlines2" to type Pixmap tex_dds.cpp(418): Function call failed: return value was -120102 (Invalid/unsupported texture format) Function call failed: return value was -120102 (Invalid/unsupported texture format) Location: tex_dds.cpp:418 (decode_pf)
Comment Actions
Thanks. Just to be sure, did you clear the whole cache first?
If yes I would need a stack trace for both the warning (add a debug_break to it) and the error. I would like to know which image triggers the crash (add some debug_printf before it tries to decode the dds). Thank you ?
Comment Actions
It looks like it is the fonts it does not like
WARNING: Loading "fonts/sans-bold-16.png"
WARNING: lama
tex_dds.cpp(420): Function call failed: return value was -120102 (Invalid/unsupported texture format)
Function call failed: return value was -120102 (Invalid/unsupported texture format)
Location: tex_dds.cpp:420 (decode_pf)
Call stack:
(0x526584) 0ad/binaries/system/pyrogenesis() [0x526584]
(0x4dbc18) 0ad/binaries/system/pyrogenesis() [0x4dbc18]
(0x4dcf20) 0ad/binaries/system/pyrogenesis() [0x4dcf20]
(0x4dbe90) 0ad/binaries/system/pyrogenesis() [0x4dbe90]
(0x542f40) 0ad/binaries/system/pyrogenesis() [0x542f40]
(0x540bfc) 0ad/binaries/system/pyrogenesis() [0x540bfc]
(0x51d52c) 0ad/binaries/system/pyrogenesis() [0x51d52c]
(0x51fb20) 0ad/binaries/system/pyrogenesis() [0x51fb20]
(0x51b200) 0ad/binaries/system/pyrogenesis() [0x51b200]
(0x34c4fc) 0ad/binaries/system/pyrogenesis() [0x34c4fc]
(0x353828) 0ad/binaries/system/pyrogenesis() [0x353828]
(0x348df8) 0ad/binaries/system/pyrogenesis() [0x348df8]
(0x32a1e8) 0ad/binaries/system/pyrogenesis() [0x32a1e8]
(0x346b60) 0ad/binaries/system/pyrogenesis() [0x346b60]
(0x4d0398) 0ad/binaries/system/pyrogenesis() [0x4d0398]
(0x4c5f54) 0ad/binaries/system/pyrogenesis() [0x4c5f54]
errno = 0 (No error reported here)
OS error = ?
Warning: Cannot convert string "vlines2" to type Pixmap
Sleeping until debugger attaches.
Please wait.
GNU gdb (Raspbian 8.2.1-2) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "arm-linux-gnueabihf".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 2682
[New LWP 2684]
[New LWP 2687]
[New LWP 2688]
[New LWP 2689]
[New LWP 2690]
[New LWP 2691]
[New LWP 2692]
[New LWP 2693]
[New LWP 2694]
[New LWP 2700]
[New LWP 2701]
[New LWP 2702]
[New LWP 2703]
[New LWP 2704]
[New LWP 2707]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/arm-linux-gnueabihf/libthread_db.so.1".
__GI___nanosleep (remaining=0xbeed50b4, requested_time=0xbeed50b4) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
28 ../sysdeps/unix/sysv/linux/nanosleep.c: Aucun fichier ou dossier de ce type.
(gdb) info stack
#0 __GI___nanosleep (remaining=0xbeed50b4, requested_time=0xbeed50b4) at ../sysdeps/unix/sysv/linux/nanosleep.c:28
#1 __GI___nanosleep (requested_time=requested_time@entry=0xbeed50b4, remaining=remaining@entry=0xbeed50b4) at ../sysdeps/unix/sysv/linux/nanosleep.c:25
#2 0xb5a41578 in __sleep (seconds=0, seconds@entry=3) at ../sysdeps/posix/sleep.c:55
#3 0x00526d80 in udbg_launch_debugger () at ../../../source/lib/sysdep/os/unix/udbg.cpp:85
#4 0x00529368 in try_gui_display_error (no_continue=false, allow_suppress=true, manual_break=true, text=0x61a6ec L"\x7273752f\x6e69622f\x656d782f\x67617373e\x6f65672d\x7274656dy\x30303578")
at ../../../source/lib/sysdep/os/unix/unix.cpp:175
#5 sys_display_error (
text=text@entry=0xa527f000 L"Function call failed: return value was -120102 (Invalid/unsupported texture format)\r\nLocation: tex_dds.cpp:420 (decode_pf)\r\n\r\nCall stack:\r\n\r\n(0x526584) 0ad/binaries/system/pyrogenesis() [0x526584]\n(0x"..., flags=flags@entry=6) at ../../../source/lib/sysdep/os/unix/unix.cpp:216
#6 0x004dd058 in CallDisplayError (flags=6,
text=0xa527f000 L"Function call failed: return value was -120102 (Invalid/unsupported texture format)\r\nLocation: tex_dds.cpp:420 (decode_pf)\r\n\r\nCall stack:\r\n\r\n(0x526584) 0ad/binaries/system/pyrogenesis() [0x526584]\n(0x"...) at ../../../source/lib/debug.cpp:383
#7 debug_DisplayError (description=<optimized out>, description@entry=0xbeed5f04 L"Function call failed: return value was -120102 (Invalid/unsupported texture format)", flags=6, flags@entry=4, context=context@entry=0xbeed53c0,
lastFuncToSkip=0x612da8 L"debug_OnError", pathname=<optimized out>, pathname@entry=0x61d8a8 L"../../../source/lib/tex/tex_dds.cpp", line=line@entry=420, func=<optimized out>,
func@entry=0x61d858 <decode_pf(DDS_PIXELFORMAT const*, unsigned int&, unsigned int&)::__func__> "decode_pf", suppress=suppress@entry=0x697dcc <decode_pf(DDS_PIXELFORMAT const*, unsigned int&, unsigned int&)::suppress__>)
at ../../../source/lib/debug.cpp:474
#8 0x004dbe90 in debug_OnError (err=-120102, suppress=0x697dcc <decode_pf(DDS_PIXELFORMAT const*, unsigned int&, unsigned int&)::suppress__>, file=0x61d8a8 L"../../../source/lib/tex/tex_dds.cpp", line=line@entry=420,
func=0x61d858 <decode_pf(DDS_PIXELFORMAT const*, unsigned int&, unsigned int&)::__func__> "decode_pf") at ../../../source/lib/debug.cpp:551
#9 0x00542f40 in decode_pf (flags=<error reading variable>, bpp=<error reading variable>, pf=<optimized out>) at ../../../source/lib/tex/tex_dds.cpp:420
#10 decode_sd (flags=<error reading variable>, bpp=<error reading variable>, h=<error reading variable>, w=<error reading variable>, sd=<optimized out>) at ../../../source/lib/tex/tex_dds.cpp:518
#11 TexCodecDds::decode (this=<optimized out>, data=<optimized out>, t=t@entry=0xb1b7ea38) at ../../../source/lib/tex/tex_dds.cpp:614
#12 0x00540bfc in Tex::decode (this=this@entry=0xb1b7ea38, Data=std::shared_ptr<unsigned char> (use count 2, weak count 0) = {...}, DataSize=<optimized out>) at ../../../source/lib/tex/tex.cpp:736
#13 0x0051d52c in OglTex_reload (ot=0xb1b7ea38, vfs=..., pathname=..., h=1114129) at ../../../source/lib/res/graphics/ogl_tex.cpp:472
#14 0x0051fb20 in call_init_and_reload (type=0x69421c <V_OglTex>, type=0x69421c <V_OglTex>, init_args=0xbeed68d0, pathname=..., vfs=std::shared_ptr<struct IVFS> (use count 4, weak count 0) = {...}, hd=0xb1b7ea00, h=<optimized out>)
at ../../../source/lib/res/h_mgr.cpp:438
#15 alloc_new_handle (init_args=0xbeed68d0, flags=<optimized out>, key=3986027790, pathname=..., vfs=std::shared_ptr<struct IVFS> (use count 4, weak count 0) = {...}, type=0x69421c <V_OglTex>) at ../../../source/lib/res/h_mgr.cpp:481
#16 h_alloc (type=0x69421c <V_OglTex>, vfs=std::shared_ptr<struct IVFS> (use count 4, weak count 0) = {...}, pathname=..., flags=<optimized out>) at ../../../source/lib/res/h_mgr.cpp:518
#17 0x0051b200 in ogl_tex_load (vfs=std::shared_ptr<struct IVFS> (use count 4, weak count 0) = {...}, pathname=..., flags=flags@entry=17) at ../../../source/lib/res/graphics/ogl_tex.cpp:544
#18 0x0034c4fc in CTextureManagerImpl::LoadTexture (this=this@entry=0xd67da0, texture=std::shared_ptr<class CTexture> (use count 4, weak count 4) = {...}, path=...) at ../../../source/graphics/TextureManager.cpp:179
#19 0x00353828 in CTextureManagerImpl::TryLoadingCached (this=0xd67da0, texture=std::shared_ptr<class CTexture> (use count 4, weak count 4) = {...}) at ../../../source/graphics/TextureManager.cpp:267
#20 0x00348df8 in CTexture::TryLoad (this=0x1825da8) at ../../../source/graphics/TextureManager.cpp:566
#21 CTexture::GetHandle (this=0x1825da8) at ../../../source/graphics/TextureManager.cpp:553
#22 0x0032a1e8 in CShaderProgram::BindTexture (this=0x1712ba8, id=..., tex=...) at /usr/include/c++/8/bits/shared_ptr_base.h:1018
#23 0x00346b60 in CTextRenderer::Render (this=this@entry=0xbeed7658) at /usr/include/c++/8/ext/atomicity.h:96
#24 0x004d0398 in CGUIText::Draw (this=0x116e528, pGUI=..., DefaultColor=..., pos=..., z=<optimized out>, clipping=...) at ../../../source/gui/CGUIText.cpp:462
#25 0x004c5f54 in CText::Draw (this=0x116d950) at ../../../source/gui/ObjectTypes/CText.cpp:230
#26 0x0047ec14 in IGUIObject::RecurseObject<>(bool (IGUIObject::*)() const, void (IGUIObject::*)()) (this=0x116d950, isRestricted=(bool (IGUIObject::*)(const class IGUIObject * const)) 0x48f928 <IGUIObject::IsHidden() const>,
callbackFunction=&virtual table offset 36) at /usr/include/c++/8/bits/stl_iterator.h:804
#27 0x0047ebc4 in IGUIObject::RecurseObject<>(bool (IGUIObject::*)() const, void (IGUIObject::*)()) (this=<optimized out>, isRestricted=(bool (IGUIObject::*)(const class IGUIObject * const)) 0x48f928 <IGUIObject::IsHidden() const>,
callbackFunction=&virtual table offset 36) at /usr/include/c++/8/bits/stl_iterator.h:783
#28 0x0047ebc4 in IGUIObject::RecurseObject<>(bool (IGUIObject::*)() const, void (IGUIObject::*)()) (this=<optimized out>, isRestricted=(bool (IGUIObject::*)(const class IGUIObject * const)) 0x48f928 <IGUIObject::IsHidden() const>,
callbackFunction=&virtual table offset 36) at /usr/include/c++/8/bits/stl_iterator.h:783
#29 0x0047ebc4 in IGUIObject::RecurseObject<>(bool (IGUIObject::*)() const, void (IGUIObject::*)()) (this=<optimized out>, isRestricted=(bool (IGUIObject::*)(const class IGUIObject * const)) 0x48f928 <IGUIObject::IsHidden() const>,
callbackFunction=&virtual table offset 36) at /usr/include/c++/8/bits/stl_iterator.h:783
#30 0x0047ebc4 in IGUIObject::RecurseObject<>(bool (IGUIObject::*)() const, void (IGUIObject::*)()) (this=<optimized out>, isRestricted=(bool (IGUIObject::*)(const class IGUIObject * const)) 0x48f928 <IGUIObject::IsHidden() const>,
callbackFunction=&virtual table offset 36) at /usr/include/c++/8/bits/stl_iterator.h:783
#31 0x0047ebc4 in IGUIObject::RecurseObject<>(bool (IGUIObject::*)() const, void (IGUIObject::*)()) (this=<optimized out>, isRestricted=(bool (IGUIObject::*)(const class IGUIObject * const)) 0x48f928 <IGUIObject::IsHidden() const>,
callbackFunction=&virtual table offset 36) at /usr/include/c++/8/bits/stl_iterator.h:783
#32 0x00474068 in IGUIObject::RecurseObject<>(bool (IGUIObject::*)() const, void (IGUIObject::*)()) (callbackFunction=&virtual table offset 36, isRestricted=<optimized out>, this=<optimized out>)
at /usr/include/c++/8/bits/stl_iterator.h:783
#33 CGUI::Draw (this=<optimized out>) at ../../../source/gui/CGUI.cpp:291
#34 0x0045ad0c in CGUIManager::Draw (this=0x1138728) at /usr/include/c++/8/bits/shared_ptr_base.h:1018
#35 0x0021e7a0 in Render () at ../../../source/ps/GameSetup/GameSetup.cpp:252
#36 0x0003a6c4 in Frame () at ../../../source/main.cpp:416
#37 0x0003db50 in RunGameOrAtlas (argc=argc@entry=2, argv=argv@entry=0xbeed8324) at ../../../source/main.cpp:638
#38 0x000272c4 in main (argc=2, argv=0xbeed8324) at ../../../source/main.cpp:684When building
CMake Warning (dev) at /usr/share/cmake-3.13/Modules/FindOpenGL.cmake:270 (message):
Policy CMP0072 is not set: FindOpenGL prefers GLVND by default when
available. Run "cmake --help-policy CMP0072" for policy details. Use the
cmake_policy command to set the policy and suppress this warning.
FindOpenGL found both a legacy GL library:
OPENGL_gl_LIBRARY: /usr/lib/arm-linux-gnueabihf/libGL.so
and GLVND libraries for OpenGL and GLX:
OPENGL_opengl_LIBRARY: /usr/lib/arm-linux-gnueabihf/libOpenGL.so
OPENGL_glx_LIBRARY: /usr/lib/arm-linux-gnueabihf/libGLX.so
OpenGL_GL_PREFERENCE has not been set to "GLVND" or "LEGACY", so for
compatibility with CMake 3.10 and below the legacy GL library will be used.
Call Stack (most recent call first):
src/CMakeLists.txt:18 (INCLUDE)
This warning is for project developers. Use -Wno-dev to suppress it.
/media/pi/844df88a-3cff-4625-be92-09d7028b5f931/0ad/libraries/source/nvtt/src/src/nvcore/StrLib.cpp:567:6: warning: ‘void nv::swap(nv::StringBuilder&, nv::StringBuilder&)’ has not been declared within ‘nv’
void nv::swap(StringBuilder & a, StringBuilder & b) {
^~
In file included from /media/pi/844df88a-3cff-4625-be92-09d7028b5f931/0ad/libraries/source/nvtt/src/src/nvcore/StrLib.cpp:3:
/media/pi/844df88a-3cff-4625-be92-09d7028b5f931/0ad/libraries/source/nvtt/src/src/nvcore/StrLib.h:167:21: note: only here as a ‘friend’
friend void swap(StringBuilder & a, StringBuilder & b);
^~~~
/media/pi/844df88a-3cff-4625-be92-09d7028b5f931/0ad/libraries/source/nvtt/src/src/nvcore/StrLib.cpp:770:6: warning: ‘void nv::swap(nv::String&, nv::String&)’ has not been declared within ‘nv’
void nv::swap(String & a, String & b) {
^~
In file included from /media/pi/844df88a-3cff-4625-be92-09d7028b5f931/0ad/libraries/source/nvtt/src/src/nvcore/StrLib.cpp:3:
/media/pi/844df88a-3cff-4625-be92-09d7028b5f931/0ad/libraries/source/nvtt/src/src/nvcore/StrLib.h:356:21: note: only here as a ‘friend’
friend void swap(String & a, String & b);
^~~~
Comment Actions
False alert. after running make clean clean workspaces, and rebuilding everything overnight, then cleaning the cache, it works now. There are only the compilation warning but I can live with those.
| libraries/source/nvtt/src/src/nvcore/Debug.cpp | ||
|---|---|---|
| 1031–1032 | It looks like va_list empty_va_list = {};
messageHandler->log(lines[i], empty_va_list);builds on GCC. I have yet to test it on my rpi emulator. | |
Comment Actions
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/docker-differential/1712/display/redirect
Comment Actions
Ricotz told today on IRC it still doesn't work on ARM64. See
Stan`, https://code.wildfiregames.com/D2563 seems to work for armhf, but not completely for arm64 - https://launchpadlibrarian.net/467121489/buildlog_ubuntu-bionic-arm64.0ad_0.0.24~r23516-0ubuntu1~18.04~wfg1_BUILDING.txt.gz
Comment Actions
FYI (and ricotz's), this was reported upstream but not fixed yet: https://github.com/castano/nvidia-texture-tools/issues/298
Comment Actions
I've confirmed that the nvtt issue still exists on latest trunk for me (using macOS 10.14.6), and re-confirmed that with this patch applied, it builds nicely without issue.
Comment Actions
I guess this can be committed, it fixes my issue with my RPI3 and RPI4, and a ticket can be created until they fix the ARM64 upstream.
Comment Actions
Can you include https://github.com/castano/nvidia-texture-tools/pull/309/commits/831d8e1a9173b1fb7e01f10fc1d33f42fe2da0fd as well? reported by @ricotz on IRC today. This way the launchpad could probably build all the packages here https://launchpad.net/~wfg/+archive/ubuntu/0ad.dev/+packages
Comment Actions
I further confirm that this fixes the issue on OSX. Can't test the AArch64 fix though.
Comment Actions
With the fix above
I kept the va_list_change by @Itms so it's just one change + the files.Comment Actions
ricotz asked today on IRC if we could make the va_list fix specific to the platforms it's broken on aka arm aarch64
#if NV_CPU_ARM || NV_CPU_AARCH64 || NV_OS_DARWIN || NV_OS_IOS for (uint i = 0; i < lines.count(); i++) { va_list empty_va_list = {}; messageHandler->log(lines[i], empty_va_list); delete lines[i]; } #else for (uint i = 0; i < lines.count(); i++) { messageHandler->log(lines[i], NULL); delete lines[i]; } #endif
Can also confirm the diff above works on aarch64
Comment Actions
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/docker-differential/2145/display/redirect
| libraries/source/nvtt/src/src/nvcore/Debug.cpp | ||
|---|---|---|
| 1026 | From irc today [10:07:44] ricotz: Stan, I assume it will build with the proposed patch [10:08:32] Stan: ricotz, sure, you can download it manually from the interface as well if you want to make sure, was merely inquiring if the fix now looked okay to you? [10:11:01] ricotz: Stan, yeah, personally I would reduce the #if-#endif to the required changes [10:11:17] Stan: Alright thanks. [10:11:24] ricotz: is there no upstream feedback? [10:12:03] Stan: Like us they didn't have a release for a very long time [10:12:13] Stan: and even then, we still have to patch it heavily [10:12:21] ricotz: I guess there is no alternative to nvtt? [10:13:03] Stan: I guess not, to use DirectDrawSurface files. [10:13:03] Stan: Maybe one day will switch from s3tc to something else like linkmauve suggested [10:13:14] Stan: but it has been a huge standard in games for a long time [10:13:48] Stan: I wonder what spring and irrlicht use [10:14:04] ricotz: I don't mean changing the compression but the library who provides that functionality [10:14:45] Stan: Looks like spring just use bits of the code https://github.com/spring/spring/blob/16a88b7ee2f1c5f6e66bc7d07becd549573f1d17/rts/Rendering/Textures/nv_dds.h [10:16:12] Stan: Irrlicht have their own implementation https://github.com/zaki/irrlicht/blob/1b0e050d90f280507ff248fc85897efe6737a13c/source/Irrlicht/CImageLoaderDDS.cpp | |
Comment Actions
Also:
[10:28:26] Angen: Stan: is build with --system-nvtt fixable ? [10:28:39] Stan: Not really? [10:29:01] Stan: It's up to the package maintainers to patch their NVTT copy [10:29:08] Stan: hence the system [10:29:28] Stan: so unless NVTT decides to make a new release with the fixes [10:29:35] Stan: I guess we're a bit stuck [10:30:51] Stan: ricotz can correct me if I say something wrong ^^ [10:31:26] ricotz: correct [10:31:50] ricotz: dropping the --system-nvtt option would that more clear [10:31:55] ricotz: *make
Comment Actions
Make the preprocessor change minimal.
I think it is up to the package maintainer to choose between the bundled version and the upstream version. The important thing IMO is to have with-system-nvtt disabled by default.
Comment Actions
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/docker-differential/2160/display/redirect
Comment Actions
Build failure - The Moirai have given mortals hearts that can endure.
Link to build: https://jenkins.wildfiregames.com/job/docker-differential/2185/display/redirect
Comment Actions
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/docker-differential/2186/display/redirect
Comment Actions
Build failure - The Moirai have given mortals hearts that can endure.
Link to build: https://jenkins.wildfiregames.com/job/macos-differential/770/display/redirect
Comment Actions
Build failure - The Moirai have given mortals hearts that can endure.
Link to build: https://jenkins.wildfiregames.com/job/macos-differential/772/display/redirect
Comment Actions
Build failure - The Moirai have given mortals hearts that can endure.
Link to build: https://jenkins.wildfiregames.com/job/macos-differential/776/display/redirect
Comment Actions
For what it's worth, I can no longer reproduce any kind of build failure with macOS 10.15 on latest trunk. It seems the GnuTLS, ICU and other libs changes have made this go away?
Let me know what I should do to test this propertly. I'm able to compile cleanly from libs to pyro without any patches on latest trunk, no errors. Although there is still https://trac.wildfiregames.com/ticket/5777 which means the game is unplayable.
Comment Actions
Successful build - Chance fights ever on the side of the prudent.
Link to build: https://jenkins.wildfiregames.com/job/docker-differential/2830/display/redirect