Currently the Revenue per Visitor is just a simple average value. It should be possible to remove outliers in this calculation to get more reliable results.
Currently we have to let run tests longer than they need, just to be sure about the RpV results.
The possibility of removing outliers would be simple feature with big leverage.
The revenue tracking isn't reliable in our business without outlier treatment. Only 2 ways to do it currently and neither is very good:
1. Request the raw data from support and then treat outliers manually using any number of methods. They've been kind enough to provide it before, but it's a pain and someone is going to tell me no at some point.
2. Write logic into the revenue collection script that normalizes (or excludes) revenue if it exceeds a threshold (E.g. 3x the StDev of AOV). Problems: What if I test on a subset of products? That would change AOV and outlier treatment would be weakened. What if AOV changes regularly? Requires constant updates.
I was just going to request this same enhancement.
Like commenters above, I have to request the raw data and rerun statistical significance calcuations.
We had this functionality in Adobe Target. We could actually see the actual row data behind revenue calculations, including what was excluded when we chose to exclude outliers. This was very helpful.
Another comment to say that we really, really, really need this enhancement. Our most recent test had revenue skewed by four high dollar transactions that went throught the same variation. It gets tiresome to have do our own calculations with the raw data.
+1 on this. Lets get it built into optimizely. Also usefull would be having more then one revenue variable. Its why I am in support area today is to research this issue. Looks like I will need to write some code to handle this on our side
