Rich-text Reply

Original Variation & Sample Size Calculator

ange_t 05-03-16

Original Variation & Sample Size Calculator

Hi All!


I've created my tests and am getting ready to launch them however I am a little confused around how the variations work based off the sample size calculator.


For example, in one of my tests I am testing out adding a CTA button to a page which previously didn't have a CTA button. I'd like to send 90% of my traffic to this page and only 10% to the original page given that I'm fairly certain my conversation rate will increase from adding this button. However when I input my baseline conversion into the sampe size calculator it is telling me that I need 8500 per variation. So does this mean that the test time will be dragged out based on waiting for my small 10% variation to reach that 8500 mark? If that is the case is there a way for me to exclude the original altogether from the experiment? 



Level 1

Re: Original Variation & Sample Size Calculator

Hi there,

Optimizely's Sample Size Calculator gives you an estimate based on a 50/50 traffic allocation. With a 10/90 split you might need to wait for more than 8500 users. However, Stats Engine takes many factors into account. Therefore, if the conversion rate (improvement from the baseline) is higher than expected, you may see a winner sooner than the time it takes to acquire 8500 visitors.

If you'd like to only have one variation, you can delete the default "Variation 1" and code your changes in the "Original" variation. To avoid confusion, you can rename this.

Please note, with only one variation, you will no longer be running an A/B test (there is no baseline) and so all goals will show up as "Winners". Hope this helps!

Amy Herbertson
Customer Success