Rich-text Reply

Total revenue - using the Optimizely calculator

jencajenca 12-01-14
Accepted Solution

Total revenue - using the Optimizely calculator


We use the Optimizely calculator ( when pondering on how long to keep running a test. The total revenue is somewhat of a different type of goal - is the sample size calculator working as well on this type of goal?


Best regards!


Hudson 12-02-14

Re: Total revenue - using the Optimizely calculator

[ Edited ]

Hi Jenny,


You're right to consider Revenue as a different type of goal!


The short answer is that the sample size calulator is not designed to work with Revenue goals. Try using a conversion goal closely tied to revenue, for example completed purchase, as your goal for estimating sample size. 


The longer answer:

The Sample Size Calculator is designed to function for binary goals - goals that have a conversion rate. (In this context, 'binary' just means that we're tracking whether or not a user triggered a given goal condition; the conversion rate is % of visitors that triggered the goal / all visitors). Revenue, as a continous variable, does not have a conversion rate, and would require a custom calculation based on the variance of your site's average revenue/visitor calculation in order to calculate sample size. 


An example of a binary goal that would work for estimating sample size would be completed purchase; use the baseline percentage of visitors that normally complete a purchase as the baseline conversion rate % input in the calculator. 


Would you find it useful to use a purchase conversion rate as your experiment's primary goal? (You can still report on revenue later, but would be using the conversion rate to ultimately declare whether a test variation has won or lost.)


This approach is extremely common; in addition to the statistical benefit, in most cases, the behavior you're actively seeking to influence is whether or not they complete a purchase, rather than  the size of purchase, in which case purchase conversion rate is a more precise KPI. 


Try plugging in some of your historical conversion rate values into the calculator.


Happy testing!



jencajenca 12-09-14

Re: Total revenue - using the Optimizely calculator

Thanks Hudson, that was a clever work-around! We messure completed purchases anyway so I think one single calculator is well enough for us.