This week we want to show an example of how to pull a report out of Adobe Analytics using the Reporting API. We’ll start with a simple report, then add metrics, a subrelation and a filter in the next post. We’ll also apply a segment.
The Report
Most Reporting API projects do not start with a full brief. Your friendly marketer will likely walk up to your desk or send you an email stating “we need an automated way to pull these campaign reports. Adobe say you can use the Reporting API. Make it so.”
Our advice is to stop right here.
Bad or incomplete specifications are an issue at the best of times. In this case add a cultural rift, misunderstandings of capabilities plus a distinct hunch that the marketer himself might not know what exactly it is he wants automated.
So walk over to his desk and sit down. Quiz him. Don’t stop until you know that he does actually want you to programmatically pull data and what he wants to do with it.
Then, and only then, have him log into SiteCatalyst and show you the actual report he wants. Or, if he is so inclined, have him show you in Report Builder.
The Reporting API
The Reporting API gives you access to the aggregated reports that SiteCatalyst and Report Builder show as well. There is no hidden trick, it just pulls the exact same data. If you run your report against the Reporting API at a specific time and your marketer pulls his at the same time, your numbers will match exactly.
The report in SiteCatalyst or Report Builder is therefore a really good prototype that’ll help you a) define your report and b) debug really easily.
Let’s take this Campaign report as an example:
![[Screenshot]](https://webanalyticsfordevelopers.files.wordpress.com/2013/05/130510-api-campaign-report.png)
So let’s build it!
We’ll use Java for the examples, mostly because it translates fairly easily to a .NET language, but also because accessing the APIs using PHP is easier. So if you’re using PHP, you should not have too much trouble translating the code examples below.
We’ll use SOAP this time round. A REST-ful example will follow.
Setup
You’ll need a couple of libraries. Versions change and technologies move on, of course. At the time of this writing, we are using the following JARs:
- axis
- wsdl4j
- jaxrpc
- saaj
- xercesImpl
- wss4j
- axis-ant
- xmlsec
- commons-codec
- commons-discovery
- commons-httpclient
- serializer
- xalan
- resolver
You will also need the WSDL file that describes the Reporting API interface. Go and download it from the SiteCatalyst UI. You can find it under Admin > Admin Console > Company and then behind the Web Services link.
![[Screenshot]](https://webanalyticsfordevelopers.files.wordpress.com/2013/05/130510-api-wsdl-file.png)
Authentication
You can find your username and password (called “secret”) in the SiteCatalyst UI under Admin > Admin Console > User Management, then click Edit Users, find yourself and click your login. Or your friendly marketer can find it for you.
![[Screenshot]](https://webanalyticsfordevelopers.files.wordpress.com/2013/05/130510-api-username-secret.png)
You need to instruct axis to use it by generating a client-side descriptor file, usually named client_deploy.wsdd, like so:
<?xml version="1.0" encoding="UTF-8"?>
<deployment xmlns="http://xml.apache.org/axis/wsdd/" xmlns:java="http://xml.apache.org/axis/wsdd/providers/java">
<transport name="http" pivot="java:org.apache.axis.transport.http.HTTPSender"/>
<globalConfiguration>
<requestFlow>
<handler type="java:org.apache.ws.axis.security.WSDoAllSender">
<parameter name="action" value="UsernameToken" />
<parameter name="user" value="USERNAME" />
<parameter name="passwordCallbackClass" value="YOUR.FQDN.HERE.PWCallback" />
<parameter name="addUTElements" value="Nonce Created" />
<parameter name="mustUnderstand" value="true" />
<parameter name="passwordType" value="PasswordDigest" />
</handler>
</requestFlow>
</globalConfiguration>
</deployment>
For axis to use this file, pass it to your JVM as a paramter: -Daxis.ClientConfigFile=client_deploy.wsdd
You also must provide a callback class and method like this:
@Override
public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
for (int i = 0; i < callbacks.length; i++) {
if (callbacks[i] instanceof WSPasswordCallback) {
WSPasswordCallback pc = (WSPasswordCallback) callbacks[i];
// set the password given a user name
if (AxisTest.USERNAME.equals(pc.getIdentifer())) {
pc.setPassword(generateAuthKey());
}
} else {
throw new UnsupportedCallbackException(callbacks[i], "Unrecognized Callback");
}
}
}
The generateAuthKey() method simply passes back the password.
In order to test whether your authentication works, I’d suggest you call the Company.GetTokenCount method, which returns the number of tokens you have left for the month.
If that works, move on to the next step – building the report.
Report Definition
This is the most important part: you need to tell the Reporting API which report you want.
With the report you got from your friendly marketer, this step is relatively straight-forward. All we need to do is instantiate an object of the reportDescription class and populate it’s members accordingly.
// Create a report description
ReportDescription reportDesc = new ReportDescription();
reportDesc.setReportSuiteID("<RSID>");
ReportDefinitionLocale locale = ReportDefinitionLocale.en_US;
reportDesc.setLocale(locale);
reportDesc.setDateTo("2013/04/01");
reportDesc.setDateFrom("2013/04/30");
ReportDefinitionMetric[] metrics = new ReportDefinitionMetric[1];
metrics[0] = new ReportDefinitionMetric();
metrics[0].setId("visits");
reportDesc.setMetrics(metrics);
ReportDefinitionElement[] elements = new ReportDefinitionElement[1];
elements[0] = new ReportDefinitionElement();
elements[0].setId("tracking_code");
reportDesc.setElements(elements);
The setMetrics method allows you to specify the metric(s) you need while the setElements method is used to specify the dimension(s).
Make sure you use the actual report suite ID instead of “RSID” above. Your friendly marketer should be able to tell you, or you can find it yourself.
“Run” the Report
Now all you need to do is to “queue” that report so the system can run it.
The Reporting API has three methods for queueing: Report.QueueRanked, Report.QueueTrended and Report.QueueOvertime. They do different things.
Report.QueueRanked creates ranked reports like the one in the screenshot above, basically listing dimensions in descending order of whatever the metric is. A “most popular pages” report is a good example.
Report.QueueTrended creates reports that show you how up to five dimensions develop over time for a specific metric. An example would be a report that shows how marketing channels evolve.
![[Screenshot]](https://webanalyticsfordevelopers.files.wordpress.com/2013/05/130510-api-trended-mc-report.png)
Report.QueueOvertime creates a simple overview report of multiple metrics, similar to the “Key Metrics” report in SiteCatalyst.
![[Screenshot]](https://webanalyticsfordevelopers.files.wordpress.com/2013/05/130510-api-overtime-report.png)
- A ranked report can have up to 9 metrics and up to two dimensions. Those must either both be traffic variables or conversion variables, but not a mix of both.
- A trended report can accept one single metric.
- An overtime report takes no dimension at all but multiple metrics.
Next you wait for the processing to be done. It is impossible to predict in general how long it’ll take, so you can either run a couple of tests, or just have your program go into a sleep loop and check the status of the report using Report.GetStatus every second or so.
You should look for either “done” or “failed” as a response. We think it’s obvious what those mean.
Retrieve the Data
When the waiting is over, you can retrieve the resulting data using the Report.GetReport method. You’ll get back a reportResponse object which carries status as well as the data in a nested report object. Again, you should check that the status is “done”.
Now is where the printout from your friendly marketer really is needed: the data structure that you get back will be a bit complex, depending on how complex your reportDefinition is, of course. Use the original report to figure out where in the structure you can find the data that you need.
As an example, a result for the report definition above might look like this:
![[Screenshot]](https://webanalyticsfordevelopers.files.wordpress.com/2013/05/130524-api-vs-report.png)
reportDescription to make it more interesting.
And that’s pretty much it. Happy coding!
Web Analytics for Developers 
Hi Jan,
Thanks for sharing and the level of detial it is great. Can you please shed some light on token usage when using the reporting API? How many per report are used if that is how they are consumed. Also is this the best way to extract data from Adobe Analytics in order for it to be displayed on plasma screens for general consumption?
LikeLike
Tokens: I did not mention them because since May 23rd they don’t matter anymore. Access to the Reporting API is now free. It might be throttled, but it doesn’t cost money anymore. The current guidance from Engineering is to run small requests multiple times rather than single big ones.
Plasma screens: yes, this is perfect for building custom screens or dashboards. Best uses cases that we’ve seen: C-level office dashboard, and information screen at reception.
LikeLike
Thanks Jan
LikeLike
Thank you SO MUCH for this Jan.! This article is so extremely useful I’d say you should sell it to Adobe. I got myself up and running end to end in eclipse in just a couple of hours, and I started without axis –never worked with it before. The only thing that would make this article perfect is letting people know what kind of a callback this is. Is it an Axis callback? Posting the full java class here would be so useful.!
@Override
public void handle(Callback[] callbacks) throws IOException,
LikeLike
Ok, got that figured out. Here it is for all your other readers Jan.
For anybody else reading, I went through quite a bit to reproduce Jan’s work. I learned a lot. This is an older blog entry though, so let me throw in a couple of updates. First and most importantly, this: java:org.apache.ws.axis.security.WSDoAllSender is available in the version of wss4j that Jan shows in his screen shot. I couldn’t get anything working in any newer versions, I say stick with that one unless you know exactly how Axis works.
Here’s my version of the callback code.
import javax.security.auth.callback.Callback;
import javax.security.auth.callback.CallbackHandler;
import org.apache.ws.security.*;
import java.io.IOException;
import javax.security.auth.callback.UnsupportedCallbackException;
public class TjCallback implements CallbackHandler {
public void handle(Callback[] callbacks) throws IOException, UnsupportedCallbackException {
for (int i = 0; i < callbacks.length; i++) {
if (callbacks[i] instanceof WSPasswordCallback) {
WSPasswordCallback pc = (WSPasswordCallback) callbacks[i];
pc.setPassword("your password");
System.out.println("pause");
} else {
throw new UnsupportedCallbackException(callbacks[i], "Unrecognized Callback");
}
}
}
}
Also, I couldn't get the wsdd thing working right. I saw another thread that used a different invocation using the wsdd file as an engine configuration. That seemed to work for me.
try {
EngineConfiguration config = new FileProvider("/Users/xyz/Documents/workspace/SiteCatalyst/clientdeploy.wsdd");
AdobeAnalyticsServiceBindingStub stub = (AdobeAnalyticsServiceBindingStub) new AdobeAnalyticsServiceLocator(config).getAdobeAnalyticsServicePort();
stub.setPassword("secret:secret");
Report_queue_item[] items = stub.reportGetQueue();
System.out.println("hello world");
} catch (ServiceException | RemoteException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Final note:
Jan, much respect for figuring all this out. And many many thanks for posting. I don't think I could have done it in less than a month, instead it took me two days. I'm pretty good with Hadoop and HDFS and such, and a 3rd party called Domo. LMK if you ever need to do any brainstorming. Find me here: tonyandjaime.com
tony
LikeLiked by 1 person
Thanks a lot, Tony, for the praise, and for your follow-up!
Re selling it to Adobe: I work at Adobe, so that’ll be difficult 😉
LikeLike
Hey Jan. Man this is so crazy. These seemed to work fine…
(this)
Report_queue_item[] items = stub.reportGetQueue();
(and)
Report_suite_props[] rsProps = stub.reportSuiteGetProps(rsid);
Prop[] props = rsProps[0].getProps();
for (int i = 0; i < props.length; i++){
System.out.println("Properties: ID=" + props[i].getId() + "\tName=" + props[i].getName());
}
But when I try to run.. (which is almost the same thing)
Report_suite_events[] rsEvents = stub.reportSuiteGetEvents(rsid);
Event[] events = rsEvents[0].getEvents();
for (int i = 0; i Events: ID=” + events[i].getId() + “\tName=” + events[i].getName());
}
I get a deserializer error…
About to throw this 0
Feb 16, 2016 3:59:23 PM org.apache.axis.client.Call invoke
SEVERE: Exception:
org.xml.sax.SAXException
java.lang.IllegalArgumentException
at org.apache.axis.encoding.ser.SimpleDeserializer.onEndElement(SimpleDeserializer.java:176)
at org.apache.axis.encoding.DeserializerImpl.endElement(DeserializerImpl.java:502)
at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)
at org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:171)
at org.apache.axis.message.MessageElement.publishToHandler(MessageElement.java:1141)
at org.apache.axis.message.RPCElement.deserialize(RPCElement.java:236)
at org.apache.axis.message.RPCElement.getParams(RPCElement.java:384)
at org.apache.axis.client.Call.invoke(Call.java:2467)
at org.apache.axis.client.Call.invoke(Call.java:2366)
at org.apache.axis.client.Call.invoke(Call.java:1812)
at com.adobe.marketing.analytics.AdobeAnalyticsServiceBindingStub.reportSuiteGetEvents(AdobeAnalyticsServiceBindingStub.java:6925)
at com.tj.Main.main(Main.java:58)
Caused by: java.lang.IllegalArgumentException
at com.adobe.marketing.analytics.Event_visibility_enum.fromValue(Event_visibility_enum.java:31)
at com.adobe.marketing.analytics.Event_visibility_enum.fromString(Event_visibility_enum.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.axis.encoding.ser.EnumDeserializer.makeValue(EnumDeserializer.java:53)
at org.apache.axis.encoding.ser.SimpleDeserializer.onEndElement(SimpleDeserializer.java:172)
… 11 more
About to throw this 6!
AxisFault
faultCode: {http://schemas.xmlsoap.org/soap/envelope/}Server.userException
faultSubcode:
faultString: org.xml.sax.SAXException
java.lang.IllegalArgumentException
faultActor:
faultNode:
faultDetail:
{http://xml.apache.org/axis/}stackTrace:org.xml.sax.SAXException
java.lang.IllegalArgumentException
at org.apache.axis.encoding.ser.SimpleDeserializer.onEndElement(SimpleDeserializer.java:176)
at org.apache.axis.encoding.DeserializerImpl.endElement(DeserializerImpl.java:502)
at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1087)
at org.apache.axis.message.SAX2EventRecorder.replay(SAX2EventRecorder.java:171)
at org.apache.axis.message.MessageElement.publishToHandler(MessageElement.java:1141)
at org.apache.axis.message.RPCElement.deserialize(RPCElement.java:236)
at org.apache.axis.message.RPCElement.getParams(RPCElement.java:384)
at org.apache.axis.client.Call.invoke(Call.java:2467)
at org.apache.axis.client.Call.invoke(Call.java:2366)
at org.apache.axis.client.Call.invoke(Call.java:1812)
at com.adobe.marketing.analytics.AdobeAnalyticsServiceBindingStub.reportSuiteGetEvents(AdobeAnalyticsServiceBindingStub.java:6925)
at com.tj.Main.main(Main.java:58)
Caused by: java.lang.IllegalArgumentException
at com.adobe.marketing.analytics.Event_visibility_enum.fromValue(Event_visibility_enum.java:31)
at com.adobe.marketing.analytics.Event_visibility_enum.fromString(Event_visibility_enum.java:36)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.axis.encoding.ser.EnumDeserializer.makeValue(EnumDeserializer.java:53)
at org.apache.axis.encoding.ser.SimpleDeserializer.onEndElement(SimpleDeserializer.java:172)
… 11 more
{http://xml.apache.org/axis/}hostname:Atom
I modified the code to say this to know where it was failing. Pretty clear there is something wrong with my Axis somewhere.
try {
System.out.println(“About to throw this 0”);
java.lang.Object _resp = _call.invoke(new java.lang.Object[] {rsid_list});
System.out.println(“About to throw this 1”);
if (_resp instanceof java.rmi.RemoteException) {
System.out.println(“About to throw this 2 !”);
throw (java.rmi.RemoteException)_resp;
}
else {
extractAttachments(_call);
System.out.println(“About to throw this 3 !”);
try {
System.out.println(“About to throw this 4 !”);
return (com.adobe.marketing.analytics.Report_suite_events[]) _resp;
} catch (java.lang.Exception _exception) {
System.out.println(“About to throw this 5 !”);
return (com.adobe.marketing.analytics.Report_suite_events[]) org.apache.axis.utils.JavaUtils.convert(_resp, com.adobe.marketing.analytics.Report_suite_events[].class);
}
}
} catch (org.apache.axis.AxisFault axisFaultException) {
System.out.println(“About to throw this 6!”);
throw axisFaultException;
}
And when I run a report,
ReportDescription desc = new ReportDescription();
//ReportDefinitionLocale locale = ReportDefinitionLocale.en_US;
desc.setReportSuiteID(“myreportsuite”);
desc.setDateTo(“01-15-2016”);
desc.setDateTo(“02-15-2016”);
ReportDescriptionMetric[] metrics = new ReportDescriptionMetric[1];
metrics[0] = new ReportDescriptionMetric();
metrics[0].setId(“40”);
ReportQueueResponse queResponse = stub.reportQueue(desc);
System.out.println(“Report Queue: ” + queResponse.getReportID());
I get…
faultDetail:
{http://xml.apache.org/axis/}stackTrace:org.w3c.dom.DOMException: WRONG_DOCUMENT_ERR: A node is used in a different document than the one that created it.
at org.apache.xerces.dom.ParentNode.internalInsertBefore(Unknown Source)
at org.apache.xerces.dom.ParentNode.insertBefore(Unknown Source)
at org.apache.xerces.dom.NodeImpl.appendChild(Unknown Source)
at org.apache.axis.message.SOAPFaultBuilder.onEndChild(SOAPFaultBuilder.java:305)
at org.apache.axis.encoding.DeserializationContext.endElement(DeserializationContext.java:1090)
Seriously feels like an Axis thing.
I’ve been banging my head against the wall. Tried migrating to axis 2, but they don’t support rpc style declarations. Went back to 1.5.1, then back to 1.5.3 for both wsdl4j and wss4j. Must have built those classes and checked my classpath in eclipse a dozen times.
And what makes it worse is the eclipse TCP Monitor won’t work because this is all on https dev. Figured at least there was some XML coming back that wasn’t right that I could verify.
Any suggestions? Also, there used to be a method for getMetrics, what happened to it? I can’t seem to find all my metrics.
Tony Fraser from NYC.
LikeLike
Hi Tony,
I’m afraid I ditched AXIS and SOAP the moment I learned how to access the Reporting API in a RESTful way. Haven’t touched the old code since then.
Can you switch to REST at all? Might be easier…
LikeLike