UAT on Travis CI
• Mark Eschbach
I wrote a small suite of user end to end tests and now is time to start expirimenting with placing it in Travis CI. The first step is to get a single run through Travis CI which fails in the manner I expect. A long term goal of the system would be to run this suite against UAT instances of our systems to verify our MVP works as expected whenever there are changes in our application, but realistically I think this larger build pipeline is something I’ll have to resolve later as it doesn’t come out of the box with Travis CI. We’ve got an iOS client and a web application hitting an API. Ideally these tests would ensure they all integrate and opreate as expected. These are high level tests intended to ensure the integration, but where I draw the lines is probably best left for more details later.
First step in the process is to get a basic Travis deployment descriptor setup. Since we are testing iOS code we will need to use an OSX image, usually the closed source wait for these isn’t too bad! Begining with the example I used from SwiftyPapertrail because it was a simple exmaple the .travis.yml
file should look like the following:
language: objective-c
os: osx
osx_image: xcode8.2
before_install: .travis/before_install
script: .travis/script
I originally placed all of the Travis related files in the root of the repository, however this got messy as we continued to add more hooks. I began expirimenting with using the directory .travis
to house all of the Travis CI related scripts and it’s worked pretty well. In the long term I will probably convert our other repositories to do the same. With the UAT system I don’t have any need to customize the installation of dependencies yet so I’ll drop that and move on to the meat and potatoes: .travis/script
. This script is responsible for wiring the environment against the tests and running them. Now to verify it’s failure! Passed on the backlog of 300 or so open source projects waiting to build the results will probably take a while.
That came back quicker than I thought. I really dislike installing dependencies outside of a working directory as I work on a lot of projects. The global installs of Bundler and NPM result in different behavior than those who use the builds system and removes the first rule, which is to have repeateable builds. Simple solution to this for node is to add ./node_modules/.bin
to the path. I’m using using something like rbenv would work also…but I haven’t looked into that yet. Well, there are quiet a few things I’ll have to change to get iOS up and running, including making the target platform tunable. The WebDriverAgent
used to actually run the tests is having a lot of trouble booting up so I’ll have to return to the issue later.
In the mean time to deliver some value I’m pushing out the critical web tests which will start to give us some coverage on the system. I’ve done this using Cucumber’s tagging system. I did get hung up on command line execution where I forgot the @
symbol in front of the tags. We’re using something like cucumber --tags @critical --tags ~@wip --tags ~@ios
to drive the web based tests. The idea being in the future I’ll remove the exclusion of @ios
so we can get our tets up and runnning.