Rich-text Reply

Total revenue - using the Optimizely calculator

jencajenca 12-01-14
Accepted Solution

Total revenue - using the Optimizely calculator

Hello!

We use the Optimizely calculator (https://www.optimizely.com/resources/sample-size-calculator) when pondering on how long to keep running a test. The total revenue is somewhat of a different type of goal - is the sample size calculator working as well on this type of goal?

 

Best regards!

//Jenny

Hudson 12-02-14
 

Re: Total revenue - using the Optimizely calculator

[ Edited ]

Hi Jenny,

 

You're right to consider Revenue as a different type of goal!

 

The short answer is that the sample size calulator is not designed to work with Revenue goals. Try using a conversion goal closely tied to revenue, for example completed purchase, as your goal for estimating sample size. 

 

The longer answer:

The Sample Size Calculator is designed to function for binary goals - goals that have a conversion rate. (In this context, 'binary' just means that we're tracking whether or not a user triggered a given goal condition; the conversion rate is % of visitors that triggered the goal / all visitors). Revenue, as a continous variable, does not have a conversion rate, and would require a custom calculation based on the variance of your site's average revenue/visitor calculation in order to calculate sample size. 

 

An example of a binary goal that would work for estimating sample size would be completed purchase; use the baseline percentage of visitors that normally complete a purchase as the baseline conversion rate % input in the calculator. 

 

Would you find it useful to use a purchase conversion rate as your experiment's primary goal? (You can still report on revenue later, but would be using the conversion rate to ultimately declare whether a test variation has won or lost.)

 

This approach is extremely common; in addition to the statistical benefit, in most cases, the behavior you're actively seeking to influence is whether or not they complete a purchase, rather than  the size of purchase, in which case purchase conversion rate is a more precise KPI. 

 

Try plugging in some of your historical conversion rate values into the calculator.

 

Happy testing!

 

 

Optimizely
jencajenca 12-09-14
 

Re: Total revenue - using the Optimizely calculator

Thanks Hudson, that was a clever work-around! We messure completed purchases anyway so I think one single calculator is well enough for us.

 

Cheers!