Rich-text Reply

limited testing

cturanli 05-06-14
Accepted Solution

limited testing

Hi there,

 

I want to A/B test but want to limit test to 20K or less people. Is this possible?

 

We know statistically more people doesn't mean better data. So, it's ok for us to test on limited data for some cases.

 

Thanks,

cturanli

Level 1

tfootdale 05-06-14
 

Re: limited testing

[ Edited ]
Highlighted
Kathryn 05-06-14
 

Re: limited testing

Hi @Cturanli,

There isn't a way to automatically stop testing once X amount of visitors have been tested, but as @tfootdale mentioned, you can use your traffic allocation to limit how many visitors see your experiment. For example, if you know that your site gets 100k unique visitors per month, you could set your test up so that 10% of your site's visitors see the original variations and 10% see the variation, which after a month of testing would results in roughly 20k visitors having been tested. Once you've reached the desired number of visitors, you can then pause the test.

Hope this helps,
Kathryn
Optimizely
tfootdale 05-06-14
 

Re: limited testing

Also, if you are only testing one page, look at the amount of traffic for that page rather than the entire site. Optimizely will send you an email when it looks like you will go over your plan allowance.