Page MenuHomeWildfire Games

implement Travis code integration testing
AbandonedPublic

Authored by andy5995 on Aug 19 2018, 12:21 PM.

Details

Summary

This patch will allow developers and team members to view the output of building 0 A.D. using multiple compilers and versions. In it's present form, this patch will perform test-builds of 0 A.D. on Ubuntu trusty using gcc 4.8, 5, 6, 7, and 8; clang 5 and 6; osx High Sierra (not yet working).

Once this patch is merged into the svn repo, any time the GitHub repo is updated with new code from svn, the Travis test will be triggered for the latest code.

Test Plan

To test this patch, Travis must be told about the .travis.yml file. To enable, an admin of the 0 A.D. GitHub repo must create an account on https://travis-ci.org , add the 0 A.D. repo by syncing, then toggle the enable/disable switch. Once that's done, when a branch is created with these files, a travis build will trigger.

I have already set this up on a branch on my fork, and the output can be seen at https://travis-ci.org/andy5995/0ad

I submitted a pull request with this patch. Normally pull requests are not done through the 0 A.D. GitHub repo, but to test this, it must be tested on the GitHub repo. It does NOT have to be merged into the master branch to be tested. The Travis service can test any branch and any pull request.

Diff Detail

Repository
rP 0 A.D. Public Repository
Lint
Lint Skipped
Unit
Unit Tests Skipped

Event Timeline

andy5995 created this revision.Aug 19 2018, 12:21 PM
andy5995 edited the test plan for this revision. (Show Details)Aug 19 2018, 12:29 PM
andy5995 edited the test plan for this revision. (Show Details)

Based on @wraitii 's comment in this forum thread, I've removed the osx test from the PR on GitHub.

andy5995 edited the test plan for this revision. (Show Details)Aug 22 2018, 2:01 PM
andy5995 added reviewers: s0600204, vladislavbelov, lyv.
lyv resigned from this revision.Aug 26 2018, 4:11 PM

This is just my opinion.
I am not sure about doing this through a third-party when an infrastructure that makes this possible inhouse is already present.
Also, in terms of review there isnt much I can say here other than “it works” (not saying it does atm). The decisions arent mine to take.

andy5995 edited reviewers, added: elexis; removed: lyv.Aug 26 2018, 6:45 PM
Imarok added a reviewer: Itms.Aug 28 2018, 7:09 PM
Imarok added a subscriber: Imarok.

I think Itms has the best overview about our CI and such things.

In D1617#64692, @Imarok wrote:

I think Itms has the best overview about our CI and such things.

Oh, ok. Thanks, @Imarok

Hi @andy5995, thanks for the proposal!

I believe Travis would be a nice extra thing to have. Notably,

  • It would allow us to test on different GCC versions (which should be possible on our current setup but not done yet)
  • It would allow us to test on macOS (which we can't do for free ourselves)
  • In the same vein as the previous bullet point, it gives us the Travis computing power for free, since we are an OSS project.

The downsides of using Travis are:

  • We can't integrate it with Phabricator/SVN, so we can't test patches with it, only commits that are mirrored to the GitHub/GitLab repo.
  • We are not using our own resources, and depend on a third-party which might bail out, or decide that even open-source projects should pay.

So in a nutshell I'd be happy to have this, but it would be better to figure out how to make macOS work. You should rely on our build-osx-libs.sh, which is supposed to prepare libraries correctly (and which should be tested by the CI). You may use brew to install build tools like cmake, pkgconfig, etc.

And apart from Travis, are you familiar with CI and/or with Jenkins? I would gladly get some help on this platform, which is our preferred way of testing patches, and I could give you suggestions about needed contributions.

Itms requested changes to this revision.Aug 31 2018, 4:18 PM
This revision now requires changes to proceed.Aug 31 2018, 4:18 PM

but it would be better to figure out how to make macOS work

I tried. Travis apparently has a 50-minute limit on tests. It terminates the build.

And apart from Travis, are you familiar with CI and/or with Jenkins? I would gladly get some help on this platform, which is our preferred way of testing patches, and I could give you suggestions about needed contributions.

No, I don't have experience working with those. Thank you for the offer to contribute in that area. I will get back in touch with you if I'm ready to do that. I have some other tasks I'm working at right now.

Stan added a subscriber: Stan.Sep 4 2018, 6:10 PM

Apparently you have to email them to get longer builds. https://github.com/travis-ci/travis-ci/issues/9605

Itms added a comment.Sep 4 2018, 6:28 PM

I tried. Travis apparently has a 50-minute limit on tests. It terminates the build.

Would it be possible to make Travis pre-generate and cache a development environment where the libraries have been downloaded and built? This doesn't need to be done each time. The rest of the build process will take less than 50 minutes. At first glance this part of the documentation would be the way to go.

No, I don't have experience working with those. Thank you for the offer to contribute in that area. I will get back in touch with you if I'm ready to do that. I have some other tasks I'm working at right now.

No problem ?

In D1617#64738, @Itms wrote:

I tried. Travis apparently has a 50-minute limit on tests. It terminates the build.

Would it be possible to make Travis pre-generate and cache a development environment where the libraries have been downloaded and built? This doesn't need to be done each time. The rest of the build process will take less than 50 minutes. At first glance this part of the documentation would be the way to go.

Based on that link you gave me, it looks like it may very well be possible. However I'm not sure it's recommended in this case. See the section https://docs.travis-ci.com/user/caching/#things-not-to-cache

From reading that page, it would take me hours and days to figure out how to cache what's needed.

@Stan

Apparently you have to email them to get longer builds. https://github.com/travis-ci/travis-ci/issues/9605

Nice investigative work. A special request like that would have more impact coming from a 0 A.D. maintainer though.

imo, It would be best if trac:5231 were resolved before continuing with this. I already got some errors when building. Everything known should be ruled out to reduce troubleshooting time later.

Krinkle added a comment.EditedJun 16 2019, 11:54 PM

While I support Travis CI, as mentioned it will be limited to daily post-merge builds from the GitHub mirror.

I think if we ask MacStadium there's also a chance they might sponsor us a Mac mini or macOS VM for 0AD development. These could be connected to from the existing Jenkins server using the SSH plugin and then used as a Jenkins node.


EDIT: Filed a Trac ticket for the general objective of macOS testing at https://trac.wildfiregames.com/ticket/5595.

Krinkle abandoned this revision.Sep 19 2020, 3:47 AM