Improve Confidence of Revenue per Visitor by removing Outliers

Status: Great Idea!
by FalkoM ‎05-08-2014 May 8, 2014



Currently the Revenue per Visitor is just a simple average value. It should be possible to remove outliers in this calculation to get more reliable results.


Currently we have to let run tests longer than they need, just to be sure about the RpV results.

The possibility of removing outliers would be simple feature with big leverage.


Hope you can do this soon!


Kind regards


- Falko

Status: Great Idea!
Level 2

by Philippe
‎06-03-2014 June 3, 2014

Absolutely!! And other testing platforms have been offering this for years... 

Level 1
by alphanumerritt
‎06-27-2014 June 27, 2014

I third that request! 


The revenue tracking isn't reliable in our business without outlier treatment. Only 2 ways to do it currently and neither is very good:


1. Request the raw data from support and then treat outliers manually using any number of methods. They've been kind enough to provide it before, but it's a pain and someone is going to tell me no at some point.

2. Write logic into the revenue collection script that normalizes (or excludes) revenue if it exceeds a threshold (E.g. 3x the StDev of AOV). Problems: What if I test on a subset of products? That would change AOV and outlier treatment would be weakened. What if AOV changes regularly? Requires constant updates.

by abastian
‎06-30-2014 June 30, 2014

I was just going to request this same enhancement.


Like commenters above, I have to request the raw data and rerun statistical significance calcuations.


We had this functionality in Adobe Target. We could actually see the actual row data behind revenue calculations, including what was excluded when we chose to exclude outliers. This was very helpful.



Level 2
by Optimizely
‎07-29-2014 July 29, 2014
Status changed to: Great Idea!
by abastian
‎10-03-2014 October 3, 2014

Another comment to say that we really, really, really need this enhancement. Our most recent test had revenue skewed by four high dollar transactions that went throught the same variation. It gets tiresome to have do our own calculations with the raw data.



Level 2
by jonathan123
‎10-07-2014 October 7, 2014

Any news on this? It's the reason I came to the support area today.


Level 1
by daishi446
‎02-09-2015 February 9, 2015

+1 on this. Lets get it built into optimizely. Also usefull would be having more then one revenue variable. Its why I am in support area today is to research this issue. Looks like I will need to write some code to handle this on our side Smiley Sad

Level 1
by dboroi1
‎08-15-2017 August 15, 2017

Any update on this? Big need. 

Level 2