TDD for Analytics, please

Let me throw out an idea here. Given that you are a developer, I hope you can a) appreciate the idea and b) help me develop it into something that works.

The idea is to use test-driven development for Analytics or maybe for digital marketing tagging in general.

Motivation

I recently attended a conference in Berlin where we discussed “agile” in the context of analytics. The participants had a lot of ideas and were trying hard to grasp the meaning of “agile” in our part of the world.

Unlike the other discussion rounds I attended, this one was more like poking around in the dark, and I think we all felt it.

The truth is: we are so far away from being “agile” in analytics that noone really has a clue even where to start! Most people do not know what “agile” means in this context. And it must be difficult for a marketer to see why they should work “agile”, whereas for you, it is almost obvious by now.

So, could we pick just one aspect of “agile” and start with that?

Funny enough, we sort of have already. I am not the only one who says that the implementation phase is NOT key. Consultants everywhere use the “crawl, walk, run” analogy to describe how an organisation should approach new stuff.

Starting with a small item (plan, deploy, learn, understand, use) is making it easier for everyone to do things and limits the risks.

And starting with something small is almost like working on a story, isn’t it?! Or at least it could be, which is why I think this might be the easiest way into the whole “agile” world.

Plan

So the idea would be to take it a step further and formalise the way new functionality is introduced into the analytics setup (or the overall digital marketing deployment setup).

Let’s use “I want to know the impact of my paid campaigns on newsletter signups” as a requirement and acceptance test case (if we did ATDD).

We would add that test to our analytics framework.

If we ran it, the test would obviously (hopefully!) fail. Good!

Then we would break it down into smaller (unit) tests, such as “when I am passing a tracking code on the URL of a page, the Javascript code shall take the tracking code and pass it into the analytics system'” and “if a tracking code is passed into the system, a Processing Rule will assign the tracking code into the campaign variable”.

Now both these rules will also fail, of course, but at this level, a developer (you!) can do something about it and provide the code to fix test 1. Your friendly marketer or an analytics administrator can fix test 2.

This way, the big, overall acceptance test will eventually pass.

Infrastructure

Up to this point, nothing has helped you.

For this idea to truly make a difference, there must be automated unit tests, built into a framework. That’s a job for the vendors, of course.

But the moment this unit test framework exists, you will have two important advantages:

  1. no more releases that accidentally break something
  2. explicit documentation of requirements in the system

There are a lot of challenges and some great opportunities here. Off the top of my head, and to open a discussion:

  • Instead of writing “Tech Specs” or “Solution Design References”, consultants could essentially sell acceptance tests or unit tests. They’d be documentation and goal specification in one.
  • Tests can be extended as technology evolves. Think about the new Visitor ID Service. If you created a test before migrating over, you’d be a lot less stressed out about it, right? Plus all your existing tests would still be there, alerting you if the change broke anything.
  • If you think about the processing of hits, you can see that there are multiple points that tests could hook into. The simpler tests could look at (or live in) the doPlugins method in the s_code.js file, whereas more involved scenarios would have tests that looked at the processed data, maybe using the Reporting API.
  • Of course the backend would have to be tweaked to enable testing in real time. Maybe the new “Live Stream” feature could help with that.

All of this is useful and makes sense from the point of view of a developer.

The one big thing that the marketer gets out of it is the knowledge that everything will be fine because the developers know what to do and they won’t break anything.

Eventually the marketer should also realise that by working with small, contained stories, she gets things more quickly and she will know up front what she’ll get.

What next?

I think there are a lot of unknowns. Apart from the changes to the backend to enable all of this, there a surely some conceptual issues I have overlooked or maybe the whole thing is utter nonsense. But I think there is something to it, and I’d love to hear your opinion and suggestions!

About

German expat living in Switzerland (formerly UK and France). Consultant and member of the Multi-Solutions Group at Adobe, working with the Digital Marketing Suite. Father of 4 girls.

Tagged with: , , ,
Posted in Automation, Integration, Opinion, TDD
9 comments on “TDD for Analytics, please
  1. […] will tell you to do small chunks, then clean up after yourself and do other chunks. Be nimble, agile. Implementation defines what your marketer can see, but the implementation phase is not key. It is […]

    Like

  2. Hi Jan! Thank you for the article. Very good and important points. Makes a lot of sense for me as a developer. I wonder, have you seen anything like that implemented in the wild?

    Like

    • Jan Exner says:

      Thanks, Eugene.

      I haven’t seen anything like this in the wild, although there are some tools (e.g. ObservePoint) which help with testing.

      None of them do anything close to TDD, and no tool out there comes with the necessary framework to implement TDD, as far as I know.

      Hope that’ll change one day…

      Like

  3. […] time ago I wrote an article in which I dreamt of having some sort of test-driven development capability in online marketing […]

    Like

  4. […] we really need TDD, so things like this don’t break […]

    Like

  5. […] post is a sequel to TDD for Analytics, please and TDD with DTM and Adobe Analytics. While those two were mainly a rant, speculation, and some […]

    Like

  6. […] habe letztes Jahr angefangen, über test-driven development in der Webanalyse zu schwafeln. Im Dezember habe ich dann Butter bei die Fische getan und zwei Artikel gepostet, in […]

    Like

  7. […] have written a couple of articles about the subject, and I am likely going to discuss my ideas at the Adobe […]

    Like

  8. […] still dreaming of being able to TDD analytics/target/etc., but there is currently no tool that allows me to do […]

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 1,398 other followers

%d bloggers like this: