In client projects, I push for as much testing as possible. In Drupal 8 we finally have PHPUnit, which makes life grand and simpler. We use Kernel tests to run through some basic integration tests using a minimally bootstrapped database. However, we need Behat to cover behavior and functional testing on an existing site for speed and sanity reasons.

For our client project, we host our repository on GitHub and our environments on Platform.sh. Work for sprints and hotfixes are done on their own branch and handled via pull request. With Platform.sh that gives us a unique environment for each pull request, above and beyond our development, stage, and production.

There is one slight problem, however. Platform.sh environment URLs are a bit random, especially for pull request environments. There is a pattern: branch-randomness-project. It used to be easier to guess, but they made a change in Q3 2016 to better support GitFlow (yay!). It changes a bit when you're using pull requests. The URL does not contain a branch name, but the PR number.

Luckily, there is the Platform.sh CLI tool which provides a way to look up URLs and other goodies. After the Platform.sh blog Backup and Forget, I realized the same idea can be used for running Behat against our dynamic environments effectively.

Configuring CircleCI

It is not hard to get set up, there are just a few steps required. 

Add API token to environment variables

In order to use the Platform.sh CLI tool, you will need an API token added to your build environments. Follow the instructions on the Platform.sh documentation page https://docs.platform.sh/gettingstarted/cli/api-tokens.html#obtaining-a-token to configure a token.

Configure your project's environment variables. You will want to add a variable named PLATFORMSH_CLI_TOKEN that contains your token. The CLI will automatically detect this environment variable and use it when communicating with the Platform.sh API.

Install Platform CLI and get the URL

We are using CircleCI 2.0 and their circleci/php:7.1-browsers image. For our setup, I created a script called prepare-behat.sh in our .circleci directory. I'll break the script down into different chunks and then provide it in its entirety.

The first step is to download and install the Platform.sh CLI tool, which is simple enough.

curl -sS https://platform.sh/cli/installer | php

Next, we determine what environment to get a URL for. If the build is for a regular branch, then we can use the branch name. However, if it is a pull request we have to extract the pull request number from the CIRCLE_PULL_REQUEST environment variable.  CircleCI provides a CIRCLE_PR_NUMBER environment variable, but only when the pull request is from a forked repository. We do not fork the repository and make pull requests off of branches from the project repository.

if [ -z "${CIRCLE_PULL_REQUEST}" ]; then
  PLATFORMSH_ENVIRONMENT=${CIRCLE_BRANCH}
else
  PR_NUMBER="$(echo ${CIRCLE_PULL_REQUEST} | grep / | cut -d/ -f7-)"
  PLATFORMSH_ENVIRONMENT="pr-${PR_NUMBER}"
fi

Now that we have our environment, we can get the URL from Platform.sh! I used the pipe option so it displayed values instead of attempting to launch the URL, and modify the output so we use the first URL returned. We export it to a variable.

BEHAT_URL=$(~/.platformsh/bin/platform url --pipe -p 3eelsfv6keojw -e ${PLATFORMSH_ENVIRONMENT} | head -n 1 | tail -n 1)

Behat will read the BEHAT_PARAMS environment variable to override configuration. We'll pipe our environment URL into the Behat configuration.

export BEHAT_PARAMS="{\"extensions\" : {\"Behat\\\MinkExtension\" : {\"base_url\" : \"${BEHAT_URL}\"}}}"

All together it looks something like this

#!/usr/bin/env bash
curl -sS https://platform.sh/cli/installer | php
if [ -z "${CIRCLE_PULL_REQUEST}" ]; then
  PLATFORMSH_ENVIRONMENT=${CIRCLE_BRANCH}
else
  PR_NUMBER="$(echo ${CIRCLE_PULL_REQUEST} | grep / | cut -d/ -f7-)"
  PLATFORMSH_ENVIRONMENT="pr-${PR_NUMBER}"
fi
BEHAT_URL=$(~/.platformsh/bin/platform url --pipe -p projectabc123 -e ${PLATFORMSH_ENVIRONMENT} | head -n 1 | tail -n 1)
export BEHAT_PARAMS="{\"extensions\" : {\"Behat\\\MinkExtension\" : {\"base_url\" : \"${BEHAT_URL}\"}}}"

Make sure you commit the script as executable.

Start PhantomJS and run tests!

All that is left now is to add steps to boot up PhantomJS (or other headless browsers), prime the environment variables, and run Behat!

      - run:
          name: Start PhantomJS
          command: phantomjs --webdriver=4444
          background: true
      - run:
          name: Get environment URL and run tests
          command: |
            . .circleci/prepare-behat.sh
            ./bin/behat -c behat.defaults.yml --debug
            ./bin/behat -c behat.defaults.yml

I keep a debug call in so that I can see the output, such as the URL chosen. Just in case things get squirrely.

Testing Goodness

We now have tests running. With some caveats.

The environment will be deployed regardless if testing fails, kind of avoiding the concept of CI/CD.

The workaround would be to kill the GitHub integration and use the environment:push command in a workflow. In this example we'd set up the following flow:

• Run Unit and Kernel tests via PHPUnit
• Push to Platform.sh
• Run Behat tests

There's more to it, like ensuring the environment is active or not. But the command would look something like

platform environment:push -p projectabc123 -e ${PLATFORMSH_ENVIRONMENT}

Sometimes the Behat tests can run before the environment is deployed

The first option is to Implement full on CI/CD flow with the above. Or, in our case, define a flow which runs the PHPUnit stuff first. That happens to take longer than a deploy, so we're pretty much set. Unless the next issue happens.

Sometimes GitHub sends events before the merge commit is updated on a PR, causing the environment to miss a build. But this only seems to happen after a PR receives a lot of commits.

Then we're kind of stuck. We have to make a comment on the pull request to trigger a new event and then a build. This causes Platform.sh to kick over but not the tests.

Final notes

There's more which can be done. I run the preparation script in the same step as our tests to ensure variables are available. I really could export them to $BASH_ENV so they are sourced in later steps. I'm sure there are workarounds to my caveats. But we were able to integrate a continuous integration flow within a few hours and start shipping more reliable code.