Rich-text Reply

Cracking down on revenue tracking

thomasvoh 12-09-14

Cracking down on revenue tracking

Hey all,

 

I browsed around the forum on some old posts and it seems like the feature to remove outliers hasn't been built in yet. My question is: has anyone had any luck in attempting to get a more accurate revenue figure? As it stands, the figure jumps around a lot for any test that I run. I've tried to create a segment to limit my audience, but it's not perfect, because it applies per session, not per user and so there's a lot of overlap between variations.

 

Any suggestions would be helpful!

 

Thanks

Brian_Abad 12-09-14
 

Re: Cracking down on revenue tracking

Hey @thomasvoh ,

 

You're correct - the Revenue goal is the only goal that is not de-duplicated, meaning that in the scenario in which a user were to refresh their page after firing the Revenue goal it will trigger the goal again and record double the Revenue amount within the Results page.

 

One workaround could be to wrap the Revenue API call in logic that checks for the absence of a particular cookie, and if the cookie is not there --> fire the Revenue goal and drop the cookie that we are checking for. Here's an example set-up of how this would look:

 

window.optimizely = window.optimizely || [];
if (document.cookie.indexOf('trigger_revenue=true') === -1) {
	optimizely.push(['trackEvent', 'eventName', {'revenue': valueInCents}]);
	document.cookie = "trigger_revenue=true";
}

 

The above code will only work within the same session, meaning if the user closes their browser window they will lose the cookie that we are checking for. If you'd like for the cookie we are checking for to emulate the 10 year expiration date that Optimizely cookies exhibit, you could use the following modified code:

 

function setCookie(c_name,value,exdays,c_domain) {
    c_domain = (typeof c_domain === "undefined") ? "" : "domain=" + c_domain + ";";
    var exdate=new Date();
    exdate.setDate(exdate.getDate() + exdays);
    var c_value=escape(value) + ((exdays==null) ? "" : "; expires="+exdate.toUTCString());
    document.cookie=c_name + "=" + c_value + ";" + c_domain + "path=/";
}

window.optimizely = window.optimizely || [];
if (document.cookie.indexOf('trigger_revenue=true') === -1) {
	optimizely.push(['trackEvent', 'eventName', {'revenue': valueInCents}]);
	setCookie ('trigger_revenue', 'true', 3650);
}

So in this set-up, once a user triggers the Revenue goal, they will no longer be able to trigger it again to account for times in which they may accidentally re-load the page, or for whatever reason the Revenue goal fires more than once.

 

It's important to note that if you were to utilize this code, if the same user were to make 2 separate, authentic purchases Optimizely would only record the first purchase because of how this is set-up. So while you are removing some of the cases you do not want to measure, you are also taking away the cases in which multiple purchases were truly made.

 

Let me know if you have any other quesitons as I know this is a good amount of information to take in!

---
Brian Abad
Manager, Technical Support
Customer Success
Optimizely
thomasvoh 12-10-14
 

Re: Cracking down on revenue tracking

[ Edited ]

Hey Brian,

 

Thanks for your detailed response. My concern was less about the revenue figure increasing for a single user and more about finding a way to exclude users who have hit a certain revenue figure from the results. Do you have insights on that?

Brian_Abad 12-10-14
 

Re: Cracking down on revenue tracking

Thanks for the clarification! In this case I'm not sure of a way to do that within Optimizely. Could you expand a bit more on the problem you're facing? Perhaps there's an alternative solution someone in the community has in mind with a bit more context on what's going on.

Thanks!

---
Brian Abad
Manager, Technical Support
Customer Success
Optimizely
thomasvoh 12-11-14
 

Re: Cracking down on revenue tracking

Certainly, here's an example scenario:

We launch a test with original + variation 1, and now we have 10 users who have seen each variation and have each made a payment.

In terms of cumulative revenue:

10 of 10 users in original have paid $10.
9 of 10 users in variation 1 have paid $10, 1 of 10 users in variation 1 has paid $10000.

The results say that variation 1 is a clear revenue winner with however much increase in baseline revenue. If we could somehow remove anyone with a cumulative revenue greater than a certain amount, say $100, as a segment in the results page, we could get a more accurate figure without the high-paying users skewing the data.
Brian_Abad 12-11-14
 

Re: Cracking down on revenue tracking

Thanks for the added context, @thomasvoh .

 

While I wouldn't suggest as best practice to limit the data that comes in - though in this case I understand you'd rather not have the outliers reported - here is some code for the Revenue goal that could implement to measure the goal as desired:

 

window.optimizely = window.optimizely || [];

//only fire the Revenue goal if the value in cents is less than or equal to 100 dollars
if (valueInCents*100 <= 100*100){
	window.optimizely.push(['trackEvent', 'eventName', {'revenue': valueInCents}]);
};

 

In this case, the revenue amount is passed as the variable `valueInCents`. If this variable is less than or equal to $100 (100 cents * 100) then we fire off the Revenue goal.

 

More on the Revenue goal can be found here: https://help.optimizely.com/hc/en-us/articles/200039865-Revenue-tracking-goals.

 

Hope this helps!

---
Brian Abad
Manager, Technical Support
Customer Success
Optimizely
thomasvoh 01-14-15
 

Re: Cracking down on revenue tracking

Hi again Brian,

 

Like you said, I don't think it would be a good idea to limit our test to just those who paid under a certain amount. The idea was to see if we could create a segmentation that would allow us to see if the results differed between those who paid more and those who paid less. In other words, this would give us a way to control for high and low paying users to get a more reliable figure for measuring revenue.