Page MenuHomeWildfire Games

Fix compilation in Clang/Linux (undefined reference to atomic)
Needs ReviewPublic

Authored by nwtour on Mar 18 2021, 2:57 PM.
This revision needs review, but there are no reviewers specified.

Details

Reviewers
None
Summary

Compilation with clang-11 failed (gcc build successfull)

libboost_atomic = 1.67
$ clang -v
clang version 11.0.0
Selected GCC installation: /usr/bin/../lib/gcc/i586-alt-linux/8

Related commit: https://github.com/0ad/0ad/commit/5431ed81d546233fad3d150cc349cece95a5033d

Linking pyrogenesis
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::queue<_ENetEvent>::queue(unsigned int)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::queue<_ENetPacket*>::queue(unsigned int)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::queue<_ENetEvent>::~queue()':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::queue<_ENetPacket*>::~queue()':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o):/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: more undefined references to `__atomic_load' follow
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::detail::freelist_stack<boost::lockfree::queue<_ENetEvent>::node, std::allocator<boost::lockfree::queue<_ENetEvent>::node> >::deallocate_impl_unsafe(boost::lockfree::queue<_ENetEvent>::node*)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::queue<_ENetEvent>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<_ENetEvent>::node, std::allocator<boost::lockfree::queue<_ENetEvent>::node> >::allocate_impl<false>()':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:288: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::detail::freelist_stack<boost::lockfree::queue<_ENetPacket*>::node, std::allocator<boost::lockfree::queue<_ENetPacket*>::node> >::~freelist_stack()':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::detail::freelist_stack<boost::lockfree::queue<_ENetPacket*>::node, std::allocator<boost::lockfree::queue<_ENetPacket*>::node> >::deallocate_impl_unsafe(boost::lockfree::queue<_ENetPacket*>::node*)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::queue<_ENetPacket*>::node* boost::lockfree::detail::freelist_stack<boost::lockfree::queue<_ENetPacket*>::node, std::allocator<boost::lockfree::queue<_ENetPacket*>::node> >::allocate_impl<false>()':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:288: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `bool boost::lockfree::queue<_ENetEvent>::unsynchronized_pop<_ENetEvent>(_ENetEvent&)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `bool boost::lockfree::queue<_ENetPacket*>::unsynchronized_pop<_ENetPacket*>(_ENetPacket*&)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `bool boost::lockfree::queue<_ENetEvent>::do_push<false>(_ENetEvent const&)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:320: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:288: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:320: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `bool boost::lockfree::queue<_ENetPacket*>::pop<_ENetPacket*>(_ENetPacket*&)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:320: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:288: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::detail::freelist_stack<boost::lockfree::queue<_ENetPacket*>::node, std::allocator<boost::lockfree::queue<_ENetPacket*>::node> >::deallocate_impl(boost::lockfree::queue<_ENetPacket*>::node*)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:288: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `bool boost::lockfree::queue<_ENetEvent>::pop<_ENetEvent>(_ENetEvent&)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:320: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:288: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `boost::lockfree::detail::freelist_stack<boost::lockfree::queue<_ENetEvent>::node, std::allocator<boost::lockfree::queue<_ENetEvent>::node> >::deallocate_impl(boost::lockfree::queue<_ENetEvent>::node*)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:288: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: ../../../binaries/system/libnetwork.a(NetSession.o): in function `bool boost::lockfree::queue<_ENetPacket*>::do_push<false>(_ENetPacket* const&)':
/usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:239: undefined reference to `__atomic_store'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:320: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:250: undefined reference to `__atomic_load'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:288: undefined reference to `__atomic_compare_exchange'
/usr/bin/ld.default: /usr/bin/../lib/gcc/i586-alt-linux/8/../../../../include/c++/8/atomic:320: undefined reference to `__atomic_compare_exchange'
clang-11: error: linker command failed with exit code 1 (use -v to see invocation)
Test Plan

Expected: for Windows and Makos should not have effect.
GСС and Сlang for Linux have build successfully

Diff Detail

Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

nwtour requested review of this revision.Mar 18 2021, 2:57 PM
nwtour created this revision.
Stan set the repository for this revision to rP 0 A.D. Public Repository.Mar 18 2021, 3:07 PM
Stan added a subscriber: Stan.

You need to set the repository to get autobuilds.

I'm a little confused why this is necessary. Clang 11 should provide a modern enough standard library that atomic operations are available, and I believe boost::lockfree should default to using that.

It seems to be compiling against the libstdc++ library instead of libc++ which might be the problem ?

nwtour added a comment.EditedMar 18 2021, 5:56 PM

As I understand the problem, some distributions pack libatomic into a separate library. At the same time, the GCC knows that it is standard and the Clang needs to be indicated -latomic.

The bottom block is running on my computer (/usr/include/boost/lockfree/detail/atomic.hpp):

#if defined(BOOST_LOCKFREE_NO_HDR_ATOMIC) || defined(BOOST_LOCKFREE_FORCE_BOOST_ATOMIC)
#include <boost/atomic.hpp>
#else
#include <atomic>
#endif

List of distros: https://pkgs.org/download/libatomic

How a similar problem was solved in open projects

https://github.com/android/ndk/issues/104
https://github.com/scylladb/seastar/issues/530

Here is the deep investigation of why Clang itself behaves
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230888

Thanks, that makes the issue a bit clearer.

I would rather not add this unconditionally however, on the off-chance it's a problem on some particular distributions, which seems like it could happen (& it's going to be difficult to test all linux distributions)
I can see two fixes:

  • Conditionally link against this in setup_all_libs itself, depending on some architecture flag
  • Add a test and conditionally link, see D2671/rP23787 (if that's not too difficult, that's probably my preferred option).

Also adding --with-system-atomic might be a choice.

Also adding --with-system-atomic might be a choice.

I did not understand this phrase. Is it planned "not a system atomic"?
package libatomic1 and file /usr/lib/libatomic.so.1 created from the same archive as libctdc++
Just the GCC links it automatically and Clang considers optional.
Option -latomic has no goal to pick up a custom atomic.

I did not understand this phrase. Is it planned "not a system atomic"?

Boost has own implementation under BOOST_LOCKFREE_FORCE_BOOST_ATOMIC define.

package libatomic1 and file /usr/lib/libatomic.so.1 created from the same archive as libctdc++
Just the GCC links it automatically and Clang considers optional.
Option -latomic has no goal to pick up a custom atomic.

-latomic picks the system one, GCC does it silently.

In general I expect that linking boost automatically adds all needed dependencies. The network subproject shouldn't know about other libraries. It uses boost, and not libatomic directly.

Boost has own implementation under BOOST_LOCKFREE_FORCE_BOOST_ATOMIC define.

Boost makes complex calculations which version to use depending on the compiler versions. Do you want me to force his choice?

I make an experiment and build project with CXX=g++
Saved NetSession.o
I tried to build a project with CXX=clang++
"network" project build successfully and received an error at the linking "pyrogenesis" stage
replaced NetSession.o (version of GCC)
and linking with Clang was successful
means file NetSession.o is already compiled like this (option -latomic in "pyrogenesys link stage" should not affect on boost in "network stage")

The network subproject shouldn't know about other libraries. It uses boost, and not libatomic directly.

$ grep -R atomic source/network/*
source/network/NetSession.h:#include <atomic>
source/network/NetSession.h:	// Wrapper around enet stats - those are atomic as the code is lock-free.
source/network/NetSession.h:	std::atomic<u32> m_LastReceivedTime;
source/network/NetSession.h:	std::atomic<u32> m_MeanRTT;
source/network/NetSession.h:	std::atomic<bool> m_LoopRunning;
source/network/NetSession.h:	std::atomic<bool> m_ShouldShutdown;

Boost makes complex calculations which version to use depending on the compiler versions. Do you want me to force his choice?

I only suggested a way. I do not insist to use it. AFAICS it needs a workaround in any case to fix the issue, so I don't mind.

$ grep -R atomic source/network/*
source/network/NetSession.h:#include <atomic>
source/network/NetSession.h:	// Wrapper around enet stats - those are atomic as the code is lock-free.
source/network/NetSession.h:	std::atomic<u32> m_LastReceivedTime;
source/network/NetSession.h:	std::atomic<u32> m_MeanRTT;
source/network/NetSession.h:	std::atomic<bool> m_LoopRunning;
source/network/NetSession.h:	std::atomic<bool> m_ShouldShutdown;

It's not the libatomic interface, it's the part of STL. And in theory we shouldn't care how it's implemented.

Would it suffice to link libatomic when we link boost? I'm not sure why this only happens on boost code when we use std::atomic (thus <atomic>) elsewhere.