Wanna test? Here’s how


[2018-04-13 – update – changed instructions to include ChromeDriver]
[2016-06-12 – update – changed github repo]

I have been writing about testing for some time now. I have even spoken about it at the EMEA Adobe Summit!

So this week, for the first time since starting this blog, let me address your friendly marketer instead of you. The only ask for you, dear developer, is to print this, head over to her desk, then work through it with her. Thank you!

Note: this is for Windows machines. I am 100% sure you can easily show your friendly marketer how to do all of this on her Mac if she has one. If not, read the Mac Edition.


First of all, your friendly marketer has to install a couple of things:

  • phantomJS
  • Chrome Canary
  • git
  • Maven
  • selenium webdriver
  • PhantomJSDriver
  • ChromeDriver
  • selenium Java bindings
  • a JDK

Let’s do them, one by one.

Install a JDK

Go to the Java SE download site and download the JDK. You’ll have to accept a license agreement. If your system can handle it, select the 64bit version.

Run the installer, and you should end up with something like this.

JDK folder
You can reveal and copy the full path in Explorer if you click into the address bar, so do that now.

We now need to add an “environment variable” to her system. The easiest way to do that on Windows 10 is to tap Start, then type “environment”, then select the top entry. Click the “Environment Variables” button in the dialog that pops up, then the “New” button at the bottom, where the system variables are.

Name the variable “JAVA_HOME”, and paste your path.

Attention! Windows is funny with spaces, so you should replace “Program Files” with “Progra~1”. That’s a really old trick from the 90s, I believe.

Copy the folder from here
JAVA_HOME system variable
Good. Onwards.

Install Git

Go to the Git site and click the “Downloads for Windows” button. That should automatically trigger the download of an installer.

I keep all the defaults, including “Use Git from the Windows Command Prompt”, just because it makes my life easier. YMMV.

As a test, I would then open a Command Prompt and type

git --version

I should get some version information back.

Success: git –version

Install Maven

Download Maven from the Apache Maven Project download page. You’ll need the ZIP file.

You can unzip this to anywhere on your machine. I usually create a folder called “dev” on my C: drive, then unzip into that. I choose a short folder name, because this folder has to be added to the PATH variable later.

Unpacked Maven folder structure
We need to add the path to the bin folder to the PATH variable, so go back into the settings, environment variables, find PATH in the System variables, edit it, and append the path to the maven bin folder.

Maven in the PATH variable
To test whether you’re good, open a new command prompt, type

mvn --version

You should get some version information back.

Success: mvn –version

Install PhantomJS

Download PhantomJS from the PhantomJS download page. Get the ZIP file.

Like with Maven, unzip this into your dev folder.

Unpacked phantomJS folder structure
Again, you need to add the bin folder to your PATH system environment variable, but by know, you know how to do that.

Want to test? Open a command prompt, type

phantomjs --version

Success: phantomjs –version


Install Chrome Canary & ChromeDriver

Download and install Chrome Canary. While technically, you could use your regular Chrome for the tests, I would use Canary as a fresh, non-customised, raw testing tool.

Download the latest version of ChromeDriver (2.37 at the time of this writing). You’ll find the ZIP file if you follow the “Latest Release” link on the downloads page.

Like with the other tools above, unzip this into your dev folder. Like before, you need to add the folder to your PATH system environment variable. Since this is just a single file, you’ll add the dev folder to the path, this time.

If you have done it correctly, typing

ChromeDriver --version

should lead to this:

ChromeDriver –version result

Download the Tests

We’re done with installations now, it is time to download the test framework.

In your command prompt, change to the dev folder. Type

cd \dev

to do so.

Once there, type

git clone https://github.com/janexner/site-infrastructure-tests

and hit enter. Watch git download stuff.

git clone
Git will download everything you need from github, where I host the prototype implementation. You’ll end up with a folder structure under your dev folder that should look like this:

Project Folder Structure


In your command prompt, run your first ever test simply by typing

mvn test -Dwebdriver.chrome.driver=c:\dev\chromedriver.exe

Running mvn test
You’ll see a lot of activity in that window. The first time round, Maven needs to download some stuff, build the test framework, and then run the tests. Subsequent test runs will be faster, I promise!

In my case, the test is not successful:

Failure: mvn test
My guess would be that something in my test description file is wrong, or that I have an error in my prototype somewhere.

By the time you read this, I will have fixed that 😉 [update: now actually fixed]

A successful test run would look like this:

Success: mvn test

What next?

The test framework is now setup and working. From here, you can start building your own tests.

The best way to do that is to take it slow. Start with an empty test description file and add tests, one at a time.

Here’s a simple file you could start with, just replace the URL with one you want to test.

    "name": jan-exner.de",
    "pagesToTest": [
            "pageURL": "http://www.jan-exner.de/",
            "jQueryLoaded": true,

All it does is test whether jQuery is actually loaded on that page.

The other tests that you can add are documented on github.

In order to test with your file, you need to tell the framework the file name. You can do that like so:

If your test file is called “mytests.json”, you’d run

mvn test -Dwebdriver.chrome.driver=c:\dev\chromedriver.exe -Dtest.description.file=mytests.json


By the way: editing a JSON file with Notepad is not cool. I suggest you use a better editor. Which one works for you is very much a question of personal taste. Some editors that you might want to try:

And then?

When you have built a test file that works for you, I suggest you automate.

Now when I say “you” in this post, I mean either you, the developer, or you, the friendly marketer.

In the “What next?” chapter, the “you” is likely the friendly marketer or an analytics team member. But now we’re back to “you”, the developer.

I am counting on you (yes, you!) to integrate the test framework into your usual integration or regression tests. These should run every time before you go live with a new version of your site!

9 thoughts on “Wanna test? Here’s how

  1. Can this tool be used with gradle instead of maven. If so what are the parameters you would use in the gradle build file ?


  2. I’m fairly new to this tool as in i only started using it today! Its going to make my life so much easier over the coming months when the company i’m with finishes the implementation of a new and fully fledged data layer (the current one is more of a place holder and doesn’t contain a lot of data).

    This method covers the data layer in terms of what’s available when the page loads. Do you plan to extend this technique, or is there already a way, to be able to emulate events that push data to the data layer. say, for example, we have some sort of AJAX component on the site that returns data and fires a direct call rule after the user interacts with it. can we some how emulate that behaviour within the test description?

    I noticed that there is a way to inject javascript into the page so that’s a start, but how would we then wait for that interaction to return and populate the data layer before checking to see if the “dataLayerElementExists” or “dataLayerElementValue” equates to the correct value?


    1. Hi Finbar,

      There are probably different ways of doing this.

      The way I have currently implemented DTMEventBasedRuleHasRunTestCase.java injects the action into PhantomJS first, then waits 1 second, then injects the Javascript for the test.

      I’m not sure how to do it in pure JS, to be honest.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.