One Report Suite or Two?

The question was raised by Antti Koski: if I have an app and a mobile site, do I track those separately or into a single report suite?

I felt the question warrented more than just a reply on his comment, so I promised Antti to do my best and answer it in an article as soon as possible.

Let me start by framing the question, give it some context.

One or Two Report Suites?

Here is the setup Antti described: he has an app that he wants to track. He would like to use SiteCatalyst Reports & Analytics for that, or maybe Mobile Services (given he posted his comment on an article of the mini series on Mobile) (Did you know that he wrote an article himself about the subject? He did).

He also has a related mobile web site, which he also wants to track.

So far so good.

On his wishlist, he has one more big thing: ability to follow users across both site and app. The idea is to find out whether there is cross-over and how much. Do people routinely use both? Or do they choose at some point and then stick with their decision? How soon? Can we see what it is that makes them choose?

Then he has some operational questions: would segmenting between site and app work well? Or would people (analysts and business users) dread the need to constantly segment for one or the other? Would it slow them down?

How about “variables”? Essentially having twice as many sounds good, no?

So? One or Two?

There is a very simple answer to this question, actually a single word: depends…

Ha, and that was the world-record for build-up of tension via long introduction combined with most disappointing anti-climax since the invention of the anti-climax.

I am mean.

And I’m loving it!

I think I should be mean much more often! It is a thrilling experience, to be honest. Liberating. Relieves a lot of tension.

Except you’re starting to think I’m a right tool by now, aren’t you?

And since I’m not, I’ll stop fooling about now and answer the question.

It depends!

The answer above is spot on, of course, it really does depend. The least I can do then is to tell you what aspects you need to look at and what awaits you either way.

Goals

Let’s start with what I think is the number one point to consider: what are the goals of the app versus the site?

Are they exactly the same in terms of what you want people to do with them? Are there differences? Do you want people to do one thing in the app but something different on the mobile site?

If the goals are indeed the same (aside: why do you have both?), then you should consider tracking them into one report suite, maybe. But if the goals are different enough, then you definitely shouldn’t!

The reason is that all data in a report suite should help your friendly marketer make decisions. Anything that diverts her from that is bad and ugly and stinks. That also applies to data that would be relevant for someone else, or even herself but in a different context. Focus!

Access

The other big question is: who is looking at the data?

There are two aspects to this question: a) the aspect I just mentioned above — focus. Don’t think I have to elaborate more here. The other aspect is b) confidentiality of data or access rights.

Who is responsible for the app? Who needs to see and analyse that app data? Is it a problem if those people see the site data as well?

If it is, then I’d suggest separating the data into two report suites.

The usual example for this is about country-specific sites. Should the marketing people in the UK see the data for the French web site? How about a partner in the UK who works with the British colleagues? What if that partner is competing with a partner who works with the French?

Online v Offline

If you want to enable offline tracking for your app, you will have to either track into separate report suites or add timestamps to the implementation on your mobile site.

This might not be a show stopper or even a hurdle.

If I understand Antti correctly, his company is starting from scratch. But for an existing site with existing tracking on it, a switch to timestamp enabled is not trivial due to the fact that you need to essentially switch the live implementation at the exact moment when our colleagues switch on the timestamped setting in the back end.

I don’t even know whether that can be synchronised somehow, except via “Gentlement, check your watches!”

Cross-“site” Analysis

Antti was wondering whether he could analyse how users switched between using the app versus the site by tracking them into one report suite.

He sure won’t be able to do it if he doesn’t, but there is more to this, and Antti hints at it “Especially if we would save unique loginID to an eVar.”

Tracking visitors across multiple sites is almost easy, as in: as long as we can set and read one cookie on all sites involved, and/or we can hand a visitor ID between them, all tracking requests can be sent with the same visitor ID, which as you know means Adobe Analytics will treat all hits as coming from the same visitor. The reports will reflect that.

Between a mobile web site and an app, the challenge is exactly the passing of visitor IDs. It is pretty much impossible to do it for anonymous visitors in the direction site to app (except for Android, where you could abuse campaign tracking).

It is theoretically possible from app to site, if you get the user to at least once and preferably early to open the site within your app. If they do, you can pass the visitor ID (see “Hybrid Apps” by my colleague Carl Sandquist).

It is also theoretically possible if your visitors routinely log into app and site. If they do, you can generate and sync your own visitor IDs and use those for tracking.

This method would even work across devices, but it falls flat if visitors don’t log in. The challenge then is to find a reason why they should. The finance industry is a good example where this question is often asked and seldom answered elegantly.

Note that traffic before first login on any given device is “lost” in the sense that it will belong to a short, orphan visit. If you routinely attract new visitors through campaigns, this might well be a real issue.

Antti’s solution of putting a user ID into an eVar does work, slightly limited.

You can’t work with the built-in Visits and Visitors metric if you want to look at cross-usage. You need to set your own Success Events and use those.

Having written that: you need to do that anyway for any useful analytics.

The issue is more about breaking data down: any sort of user-based analysis needs to start on that eVar and then drill down. That works perfectly fine in Discover Ad Hoc, but you’re limited to one break down in Reports & Analytics, so beware.

75 vs 150

On the number of “variables” — I think that should not be an issue.

If you track both into one report suite, they should be so similar that the “variables” will overlap almost 100% anyway.

Global Report Suite

Now this is a related topic, but not quite the same.

The country example I mentioned before could nicely be amended with a global report suite — for the global marketing team who obviously have the right to see data for all countries.

When I write “Global” Report Suite, I mean one that gets data via multi-suite tracking, i.e. the app or site sends hits to more than one report suite. That implies “secondary server calls” and therefore a 50% rise of cost on that volume of calls.

The alternative is a “Rollup” Report Suite. Those are essentially copying data from the underlying report suites once a day and then show the sum. No deduplication of visitors here! It is exactly the same as pulling the same report in two report suites, then adding up the numbers.

It only makes sense if you know there is no cross-over. Therefore I dismiss it in the context of Antti’s question.

Also: Mobile Services work perfectly on a “Global” but not on a “Rollup”.

And one more thing: if you’re thinking about tracking into a Global Report Suite and you’re worried about only having 75 “variables” (you shouldn’t, but who am I…): use Context Data and Processing Rules. You can assign some data in the Global but leave some out. And you can assign data into different variables and still be consistent!

Conclusion

The tl;dr of this posting is: two show stoppers for combining tracking: a) both have to have the same goals and b) any doubts about who can access what data rule out the single-report suite approach. Then there are also some technical considerations. Cross-over tracking is not easy, currently.

Antti: I think I am speaking for everyone when I say: please let us know what decision you made and why!

Looking forward to that posting!