A Short History of Processing a Hit

In the article about Modifying Data Server-Side, we gave a brief list of the steps that every bit of data has to go through before your friendly marketer will see it in her reports.

Let’s expand on that and also mention the different points where you can add information or retrieve it.

Let’s start in your realm, the page or app.

On the Page

This is where you primarily enter information into Adobe Analytics. You “capture” data.

How you do that is up to you, of course, and it depends on the environment as well as the goals. To drop a few keywords:

• You assign values to “Variables” for basic tracking
• You add code that will help Tracking Links & Actions
• You use Context Data and maybe a Data Layer
• You can use Dynamic Variables to keep the URL shorter and observe DRY
• Tag Management is a good idea, and it will hopefully make life simpler for the analysts and yourself

This stage ends when you call the s.t() or s.tl() method and the core Javascript code takes over.

In Transit

The next step is for the core code to call the s.doPlugins callback which allows you to automate things that should happen on every hit.

The data is eventually being sent to “the cloud”. To be exact, it goes to one of the collection servers somewhere in the world.

At this point, you can very easily see what happens, which makes this the perfect spot to debug by looking at the URL (do I have to repeat that I use Charles for that? Or did I mention that before?).

This is where your — the developer’s — involvement with putting data into the system usually stops. But don’t stop reading, because it makes sense for you to know how your marketer can influence & change (and potentially mess up!) data later.

There will also be some points where you can get data out of the system for further use, and your friendly marketer might ask you to do that.

In the Cloud

Once the data arrives on Adobe servers, it goes through a number of processing steps. You might remember this diagram:

There is actually more to it.

1. Collection server resolves any Dynamic Variables
2. Collection server does an IP-based geo-lookup
3. Collection server passes data on
4. System applies Processing Rules
5. System applies VISTA Rules

At this point the data in the hit has been modified to an extent, but the view is still hit-based, i.e. so far the system hasn’t applied any prior knowlege.

The upcoming “Firehose” feature that allows you to receive raw hit data in realtime sits at this point. In other words: Firehose will give you a stream of hit-level data that has gone through all the steps we have seen so far.

6. System applies the “Visitor Profile”, which basically means it adds data it collected in the past, like eVar values. The visitor profile is matched based on the “visitor ID“.
7. System applies Marketing Channel Processing Rules (for the Marketing Channel reports)

So right now the hits have been enriched with data that is tied to the visitor via the visitor ID.

This is the point where data gets split: it goes into Data Warehouse (which means that Data Feeds contain what we have now), plus it is sent into a further processing queue for Adobe Analytics.

8. System passes data into report servers
9. Report servers process and store data for easy access in Adobe Analytics
10. Friendly marketer navigates to a report
11. System retrieves data
12. System passes data through SAINT and displays report

The reports that your friendly marketer pulls are like a pre-defined view on aspects of the data that you have sent into the system. They are designed to help her find answers and make decisions.

And there is one more thing that we need to mention, one more thing you can do to help her: you can use the Reporting API to pull data for her, and you can make fancy displays or dashboards for her. The data that you pull out of the Reporting API has gone through all 12 steps.

Don’t forget to use the all new version 1.4 for added goodness.

*phew*, done.

If you are at the EMEA Summit in London in two weeks, you might want to come and see Bret Gundersen and myself at 11.20 on Thursday when we’ll talk about Processing Rules. Bret will also show a slide that’ll sum up a lot of what I wrote above.

Looking forward to seeing you there!