Using Analytics Live Stream

I have wanted to write and post this article for ages now. The thing that always held me back: setting up OAuth. But the EMEA Summit is upon us, and with it a lab session about the Analytics Live Stream, and I am going to be part of that, so I have to.

The bad news first: Analytics Live Stream is available for customers who have subscribed to Analytics Premium. If you’re on a SiteCatalyst or Analytics Standard contract, you do not have access to this. It is a cool feature, though, so you might want to read on anyway.

What? Why?

Analytics Live Stream is a feature very similar to Data Feeds. It spits out raw data, hit per hit, containing whatever has been tracked.

Analytics Live Stream differs from Data Feeds in that it is not using FTP or sending files, or sending them once a day or even every half hour or so. No, it sends each hit the moment it arrives.

The hits go through Processing Rules and VISTA, the visitor profile is applied, but that is it. They are pretty much the same data that you sent in.

Why would you do that? Well, the biggest reason is to integrate with Adobe Analytics. I am not talking BI or DWH integrations, those might not even need real-time data. I am thinking about interactive displays or mechanisms that do something when data arrives.

The example we see most in the field are dashboards or big screen displays showing off what the current status is.

For a newspaper, this might be the number of articles currently being viewed, and obviously which ones. For a bank, it might be the number of online banking logins. A retailer might want to see orders as they happen, or maybe (if they’re a merchandiser) what products people are looking at right now.

There’s a use case for pretty much everyone, including, of course, your friendly marketer. Talk to her, ask her whether she’d like to see some real-time data on some impressive screen.

How?

Right, you want to know how. Of course.

I am presuming here that your friendly marketer has already done the setup with someone from Adobe. As a result of that, you should have a client ID and client secret, the URL for requesting a token, and the URL of the Analytics Live Stream API itself.

The authentication is pretty simple in terms of dependencies; all we need is Base64 encoding and Jackson.

Let’s start with the simplest possible thing you’d do: getting an access token. You will need that token for all calls against Analytics Live Stream.

The result is JSON, so let’s parse it:

If I call those two methods, I’ll see something like this:

I also get some grief from slf4j, but that is just me being sloppy and it is irrelevant in this context. Also: now you get a feeling for how long ago I actually developed for a job, and that I learned Java by trial and error.

You probably noticed that the code references an instance of the Credentials class. That class holds client ID, client secret and those two URLs we need.

Note that for you, the second URL will be different!

Getting Data

Now that we have a token, we can request some actual data.

I want to keep this simple, as simple as possible. I trust you to be able to take this skeleton and build on top of it. That’s what you do, isn’t it?

This time, we’re going to connect to the Live Stream. That’s what we need the token for in the first place.

The documentation says the results come back gzip’d, so we’ll need a GZIPInputStream on top of our connection. To make it easier to read, let’s add a BufferedReader on top of that.

This really simple example opens a connection using the token, piles the reader on top, then just reads from the Analytics Live Stream until the connection goes down or someone kills the program.

Pretty simple, isn’t it?

The result looks something like this:

Again, the results are in JSON and need to be parsed. See the hitIdHigh & hitIdLow values? You can use those to deduplicate your data.

Notes

I have of course only scratched the surface, there is a lot more to it.

You will want to actually process the data that comes back. You will have to respond to a multitude of error cases, like randomly dropped connections or duplicate or out-of-order records.

If you want to know more about what the data coming back looks like, check out the Request Details documentation.

I just wanted to show this really short example to show you that using the Analytics Live Stream is really not rocket science.

And, given what you could do with it, you might want to discuss with your friendly marketer and build something spectacular! (If you do, let us know!)

Btw: if you want to try any of the above, feel free to join me at the Adobe EMEA Summit at the London ExCel on April 29th. I’m running a lab session in room CS 1 at 4.10pm.

Also: if you want to see what you could do with Analytics Live Stream, check out this link: http://adobefirehose.mediarain.com/#/dashboard

This is live data from adobe.com.