Issue 6220 - add travis-ci configuration for adblockpluscore

only linux and osx
# This part is for emscripten branch.

[sergei, 2018-01-11 21:48:12]

However it is not working right now because precompiled emscripten is linked
with libraries which are incompatible with ones currently available by default
on travis servers.

[sergei, 2018-01-11 22:04:10]

https://codereview.adblockplus.org/29663688/diff/29663689/.travis.yml
File .travis.yml (right):

https://codereview.adblockplus.org/29663688/diff/29663689/.travis.yml#newcode28
.travis.yml:28: - pip3 install -q --user meson
we cannot install meson via apt-get because the version is too old there. The
one which is installed via pip is 0.44.
Here I again think about having it in third_party, see #6200.

https://codereview.adblockplus.org/29663688/diff/29663689/.travis/prepare-nin...
File .travis/prepare-ninja.sh (right):

https://codereview.adblockplus.org/29663688/diff/29663689/.travis/prepare-nin...
.travis/prepare-ninja.sh:9: wget
https://github.com/ninja-build/ninja/releases/download/v1.8.2/ninja-linux.zip -O
third_party/ninja-build.zip
It's again #6200, the version which is installed via apt-get by default is too
old (1.3.4 in case of travis-ci), meson requires at least 1.5 or newer.

[hub, 2018-01-12 14:36:30]

On 2018/01/11 21:48:12, sergei wrote:
> However it is not working right now because precompiled emscripten is linked
> with libraries which are incompatible with ones currently available by default
> on travis servers.

One common workaround with Travis failure to provide recent Linux is to run CI
inside a Docker container. I have done that in the past, and it is quite solid
and somewhat documented.

[kzar, 2018-01-15 12:01:11]

I don't have time (and am probably not the best person) to review this, but
don't want to block it either. Mind having a look in my stead Tristan?

[tlucas, 2018-01-17 12:41:40]

Hey there, sorry for the late reply.

Please find the comments below.

https://codereview.adblockplus.org/29663688/diff/29663689/.travis.yml
File .travis.yml (right):

https://codereview.adblockplus.org/29663688/diff/29663689/.travis.yml#newcode40
.travis.yml:40: email: false
Do we not want to be notified about failed builds?
Also, FWIW we could add some sugar and notify #adblockplus about succeeded
builds ;)

https://codereview.adblockplus.org/29663688/diff/29663689/.travis/prepare-nin...
File .travis/prepare-ninja.sh (right):

https://codereview.adblockplus.org/29663688/diff/29663689/.travis/prepare-nin...
.travis/prepare-ninja.sh:11: unzip third_party/ninja-build.zip -d
third_party/ninja
Please help me out here - can we be sure, that "unzip" is always given? If not,
you might want to add it to the "install"-section in .travis.yml

[sergei, 2018-01-17 14:26:26]

https://codereview.adblockplus.org/29663688/diff/29663689/.travis.yml
File .travis.yml (right):

https://codereview.adblockplus.org/29663688/diff/29663689/.travis.yml#newcode40
.travis.yml:40: email: false
On 2018/01/17 12:41:39, tlucas wrote:
> Do we not want to be notified about failed builds?
> Also, FWIW we could add some sugar and notify #adblockplus about succeeded
> builds ;)

See
https://codereview.adblockplus.org/29663680/diff/29663681/.travis.yml#newcode22.

https://codereview.adblockplus.org/29663688/diff/29663689/.travis/prepare-nin...
File .travis/prepare-ninja.sh (right):

https://codereview.adblockplus.org/29663688/diff/29663689/.travis/prepare-nin...
.travis/prepare-ninja.sh:11: unzip third_party/ninja-build.zip -d
third_party/ninja
On 2018/01/17 12:41:39, tlucas wrote:
> Please help me out here - can we be sure, that "unzip" is always given? If
not,
> you might want to add it to the "install"-section in .travis.yml

As far as I know travis-ci servers usually include a plenty of already installed
tools which can be absent on a fresh linux, in our case they are at least wget,
unzip and nvm. Since we are targeting only travis-ci here, I think we should not
include those tools because otherwise we would have to include a long list of
tools (just imagine how difficult to follow this convention with android
builds).

[tlucas, 2018-01-17 16:39:03]

On 2018/01/17 14:26:26, sergei wrote:
> https://codereview.adblockplus.org/29663688/diff/29663689/.travis.yml
> File .travis.yml (right):
> 
>
https://codereview.adblockplus.org/29663688/diff/29663689/.travis.yml#newcode40
> .travis.yml:40: email: false
> On 2018/01/17 12:41:39, tlucas wrote:
> > Do we not want to be notified about failed builds?
> > Also, FWIW we could add some sugar and notify #adblockplus about succeeded
> > builds ;)
> 
> See
>
https://codereview.adblockplus.org/29663680/diff/29663681/.travis.yml#newcode22.
> 
>
https://codereview.adblockplus.org/29663688/diff/29663689/.travis/prepare-nin...
> File .travis/prepare-ninja.sh (right):
> 
>
https://codereview.adblockplus.org/29663688/diff/29663689/.travis/prepare-nin...
> .travis/prepare-ninja.sh:11: unzip third_party/ninja-build.zip -d
> third_party/ninja
> On 2018/01/17 12:41:39, tlucas wrote:
> > Please help me out here - can we be sure, that "unzip" is always given? If
> not,
> > you might want to add it to the "install"-section in .travis.yml
> 
> As far as I know travis-ci servers usually include a plenty of already
installed
> tools which can be absent on a fresh linux, in our case they are at least
wget,
> unzip and nvm. Since we are targeting only travis-ci here, I think we should
not
> include those tools because otherwise we would have to include a long list of
> tools (just imagine how difficult to follow this convention with android
> builds).

Well - at least it's not mentioned in their travis-ci's official environment
description:

https://docs.travis-ci.com/user/reference/trusty/

However - in case you verified it being available, then LGTM :)

[sergei, 2018-01-26 16:34:33]

I have added docker config.

Since regular init and some other system parts are not supposed to run inside
docker, I have based it on https://github.com/phusion/baseimage-docker.git. In
addition to that I'm running tests under a regular user in order to be sure that
we are not trying to harm anything.

I have put all travis-ci related files into .travis and extracted normal steps
into bash scripts, so they are either called by travis or by our docker-run.sh.
The call flow for the docker is the following:

travis calls .travis/prepare-docker.sh
  build phusion/baseimage-docker
  build adblockpluscore-test-image
    install packages and configure the environment
    RUN bash ${TRAVIS_BUILD_DIR}/.travis/docker-run.sh
      bash .travis/travis-install.sh
      bash .travis/travis-before_script.sh
      bash .travis/travis-script.sh

The docker container mocks travis environment but only parts which are required.
I hope that travis team finally gets a recent OS running
(https://github.com/travis-ci/travis-ci/issues/5821) and we just drop this
intermediate layer, remove most of if [[ os == linux ]] continue to use
.travis/travis-*.sh almost without touching the rest.

[Oleksandr, 2018-02-23 10:19:31]

LGTM. Is there support for Windows containers on Travis? That would be pretty
useful as well.

[sergei, 2018-02-23 10:27:09]

On 2018/02/23 10:19:31, Oleksandr wrote:
> LGTM. Is there support for Windows containers on Travis? That would be pretty
> useful as well.

FYI:
For Windows I have already some progress for appveyor. The main issue is that
there is no prebuilt emscripten available for Windows, building of it it takes
up to few hours and definitely more than 20 GB. Nevertheless soon I plan to
create a codereview with configuration for appveyor which is building only
native version (C++ emscripten code but without generation of JS). I find it
very important because a lot of development is going on Windows and it seems the
best option to debug almost everything quickly enough.

[hub, 2018-02-24 18:35:43]

Why can't we make a Docker image with all the requirements that would be fetched
from Docker hub and use it? This would significantly shorten the build time.

[sergei, 2018-02-27 16:17:23]

On 2018/02/24 18:35:43, hub wrote:
> Why can't we make a Docker image with all the requirements that would be
fetched
> from Docker hub and use it? This would significantly shorten the build time.

No objections here, see hub ticket 8475, however JIC it's not a stopper here.

I plan to land it soon, so could you please let me your concerns if there are
some, even if it's already landed.

[hub, 2018-02-27 16:54:21]

On 2018/02/27 16:17:23, sergei wrote:
> On 2018/02/24 18:35:43, hub wrote:
> > Why can't we make a Docker image with all the requirements that would be
> fetched
> > from Docker hub and use it? This would significantly shorten the build time.
> 
> No objections here, see hub ticket 8475, however JIC it's not a stopper here.
> 
> I plan to land it soon, so could you please let me your concerns if there are
> some, even if it's already landed.

Fair enough. LGTM

[sergei, 2018-03-01 10:56:01]

https://codereview.adblockplus.org/29663688/diff/29712568/.travis.yml
File .travis.yml (right):

https://codereview.adblockplus.org/29663688/diff/29712568/.travis.yml#newcode23
.travis.yml:23: - if [[ "$TRAVIS_OS_NAME" == "osx" ]]; then export
PATH=/usr/local/opt/python/libexec/bin:${PATH}; fi
It turned out that meanwhile the configuration stopped working on osx because of
the error:

Error downloading URL
'https://github.com/kripken/emscripten/archive/1.37.3.tar.gz': <urlopen error
[SSL: TLSV1_ALERT_PROTOCOL_VERSION] tlsv1 alert protocol version (_ssl.c:590)>

Basically a newer version of openssl is installed while installing of python3
and in addition to that a newer python using it is also already installed, the
everything one has to do is to let the system find it and not use the versions
shipped with the system.

[hub, 2018-03-01 16:12:05]

LGTM