Rich-text Reply

How to Analyze Your A/B Test Results with Google Analytics

conversionxl 12-18-14

How to Analyze Your A/B Test Results with Google Analytics

[ Edited ]

Built-in Google Analytics integration is not foolproof. Sometimes the data is not passed on, there’s a 20% to 50% discrepancy – somewhere somehow part of the data gets lost. There could be numerous reasons for that, anything from how the scripts are loaded, in which order to script timeouts and other issues. I’ve dealt with a lot of different problems over the years.

My good friend Ton Wesseling taught me this “trick” that I now use for every test: sending an event to Google Analytics each time a variation is loaded.


All you need to do is add one line to the test Global Javascript (executed for all variations), plus a line of event tracking code as the last line for each test variation.


So this is the line you should add in the Global Experiment Javascript console:


window.ga=window.ga||function(){(window.ga.q=window.ga.q||[]).push(arguments);};window.ga.l=+new Date();


This makes sure that the GA tracker gets all the information once it loads.

Here’s where you do it in Optimizely. First open up the Settings while editing a test:

 

1

And now choose Experiment Javascript. Add the code there:

global.jpg

And now you need to add a line of event tracking code at the end of each variation (including Original). You need to just change the Experiment ID number and the name of the Variation:

 

 

window.ga('send', 'event', 'Optimizely', 'exp-2207684569', 'Variation1', {'nonInteraction': 1});

 

So what the code does is send an event to GA where the event category is Optimizely, action is Experiment ID (you can get that from your URL while editing a test) and label is Variation1 (can also be Original, Variation 2 etc). Non-interaction means that no engagement is recorded. Otherwise your bounce rate for experiment pages would be 0%.

 

Here’s where you add the code in Optimizely:

cxl-test-1024x593.jpg

 

Now you’re able to create segments in Google Analytics for each of the variations.

 

Segment setup:

originalopt

 

Create separate segments for each variation, and apply them onto any report that you want. So you could see something like this:

test33-1024x365.jpg

Illustrative data only.

 

Same thing can be of course done with Custom Dimensions. Just make sure data consistency is there – compare thank you page visits, revenue numbers etc between your Optimizely result panel and GA custom dimension or event based report”.

 

Read more here:

How to Analyze Your A/B Test Results with Google Analytics

Founder of ConversionXL and Markitekt
andreamoro 12-24-14
 

Re: How to Analyze Your A/B Test Results with Google Analytics

There is actually another better way to do this integration, that would save you the necessity to add javascript code to every experiment.

 

The code below is all you need. Didn't test with GA tracking code at the bottom of the page though, but I will very soon.

 

/* _optimizely_evaluate=force */
setTimeout(function(){
experimentId = <Numeric ID from the URL of the experiment editor>;
if (typeof(optimizely) != "undefined" &amp;&amp;
optimizely.variationMap.hasOwnProperty(experimentId)) {
window._gaq = window._gaq || [];
_gaq.push(['_setAccount', '<Google Analytics web property ID>']);
_gaq.push(['_trackEvent', 'Optimizely', optimizely.data.experiments[experimentId].name, optimizely.variationNamesMap[experimentId], 1, true]);
}},1000);
/* _optimizely_evaluate=safe */

nolanmargo 12-24-14
 

Re: How to Analyze Your A/B Test Results with Google Analytics

[ Edited ]

I'd be interested to hear how you diagnosed and pinpointed the data discrepancies. Would you mind elaborating on this point?