The Chronicles of Circle CI - Headless Chrome

With my level of patience, I would say running an automated end to end test in circle CI ( v1.0, yes, I am still living in the yesteryears) is quite a nerve wrecking experience. Our requirement was simple - to be able to run end-to-end tests using protractor wrapper over selenium running chrome headless.

Out of the box syndrome

The first trial run was setting up the end-to-end execution in native circleCI environment. CircleCI v1.0 comes shipped with Ubuntu 14.04, Google-Chrome v54, and Java1.8  &  1.7. The default Java version under CircleCI is set to v1.7.

Versioning 

Driver

We started the trial run by fixing the versions in native CircleCI environment, finalising on protractor v5.2.2, and upgrading webdriver-manager, selenium, chrome-driver and gecko ( FireFox) drivers. The final fixed versions looked something like this:

• protractor v5.2.2
• seleniumv3.8.1
• chrome-driverv2.34

Update the drivers using webdriver-manager from node_modules/protractor/bin 

webdriver-manager update --gecko false --versions.standalone 3.8.1 --versions.chrome 2.34

Fix the java version required by selenium, in CircleCI machine setup section of circle.yml

machine:
  java:
    version: oraclejdk8        

Chrome

Google chrome needs to be updated, as, headless support was introduced from v59. google-chrome-stable is available on a 3rd Party Repository, which can be installed by adding the PPA or installing the .deb package. We chose the .deb approach.

apt-get update && apt-get -y install libxss1 libappindicator1 libindicator7
curl -L -o google-chrome.deb https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
dpkg -i ./google-chrome*.deb
apt-get install -yf

The dpkg -i ./google-chrome*.deb errors out due to unmet dependencies. apt-get intall -yf is a required step as it does the real installation by pulling up chrome's dependencies.
Protractor configuration needs to be tweaked for chrome to run in headless mode.

"chromeOptions" : { 
    "args": [
      'headless'
    ]
  }

With the changed driver versions, the webdriver-manager is required to be manually started

webdriver-manager start --versions.standalone 3.8.1 --versions.chrome 2.34 &

This starts the selenium server in the background, and the end-to-end testing can begin.

protractor ./protractor_conf.js 

Docker to the rescue

The above approach did help us in triggering the end-to-end tests in CircleCI, but with more GUI heavy frontend, chrome headless crashes with session not available or chrome not available error. This was due to the limited size of  /dev/shm that was given to the CircleCI docker. Spinning up a docker image was one of the solutions which we stumbled upon. The docker image had just one dedicated purpose; to run end-to-end tests. It also helped us in having a uniform environment to run end-to-end tests across environments.

FROM openjdk:8

ARG NODE_ENV

# replace shell with bash so we can source files
RUN rm /bin/sh && ln -s /bin/bash /bin/sh

# update the repository sources list
# and install dependencies
RUN apt-get update \
    && apt-get install ssh git ca-certificates curl build-essential -y \
    && apt-get -y clean autoclean \
    && apt-get -y autoremove \
 && rm -rf /var/lib/apt/lists/*

# nvm environment variables
ENV NVM_DIR /usr/local/nvm
ENV NODE_VERSION 6.10.1

# install nvm
# https://github.com/creationix/nvm#install-script
RUN curl -o- https://raw.githubusercontent.com/creationix/nvm/v0.31.2/install.sh | bash

RUN ls /usr/local/nvm
# install node and npm
RUN source $NVM_DIR/nvm.sh \
    && nvm install $NODE_VERSION \
    && nvm alias default $NODE_VERSION \
    && nvm use default

# add node and npm to path so the commands are available
ENV NODE_PATH $NVM_DIR/v$NODE_VERSION/lib/node_modules
ENV PATH $NVM_DIR/versions/node/v$NODE_VERSION/bin:$PATH

# confirm installation
RUN node -v && npm -v
ADD ./ ./
RUN sh ./setup_chrome.sh
RUN npm install
CMD ["/bin/bash", "./node_modules/.bin/protractor ./protractor_conf.js"]

We used a prebuilt image for Java 1.8 (openjdk:8) which made the original machine setup in CircleCI (circle.yml) obsolete. We also tucked away chrome installation into a bash script.
The docker image now could be built and run as an independent unit and solved the shared memory ( /dev/shm) issue faced during earlier trials by mounting a local folder as the shared memory location.
The protractor configuration need to be updated to support running of headless chrome under the root user from the docker context.

"chromeOptions" : { 
    "args": [
      'headless',  '--no-sandbox'
    ]
  }

Build and spin up the docker Image.

docker build -t app/e2e:latest .
docker run -v $HOME/tmp-shm:/dev/shm -i -t app/e2e

The Lost Artefacts

One thing which was available in the native CircleCI setup was the ability to save test outcomes, and build artefacts, but by running tests under docker the artefacts and test results lived and died within the dockers context. Volume mounting seemed like a viable option, but it din't achieve the desired result. After digging around a lot we figured out that CircleCI doesn't support folder/volume mounting. The document around mounting folders in CircleCI v2.0 proved useful in our quest.
And, finally an acceptable build candidate to run end-to-end test was conceived.

docker build -t app/e2e:latest .
docker run -v $HOME/tmp-shm:/dev/shm --name app-e2e -i -t app/e2e

docker cp app-e2e:/output ${CIRCLE_TEST_REPORTS}/

-- A special shoutout to Smriti Tuteja who survived along with me searching and documenting the process.