Statistics: Power in relation to type of test, statistical assumptions

[#1]

Hi fellow testers,

What is the approach to find out the amount of visitors for MVT test, or an ABCD test?

In articles, like the ones shown below, I find information about significance and how to deal with power for an A/B test, but not for other types of tests.

So, eg. could I use the 'A/B test sample size calculator' in the context of an MVT or ABCD test? (just use the outcome for each cell, variant?)

Kind regards,

Jeroen

See articles:

Interpreting More Complex Results

and

How many visitors do I need for A/B test?

[#2]

Hi Jeroen,

A tool that might be able to help you out is:

http://www.houseofkaizen.com/conversion-rate-optimisation/resources/calculators/test-length-estimate...

If you fill in each section and hit go it will tell you approx how many days you will need to test for based on a rough estimate of the traffic that will hit the page you are testing.

Hope that helps

[#3]

thx David

[#4 · Accepted solution]

Hi Jeroen, 

You can use Optimizely's Sample Size Calculator to understand how to calculate the sample size of each variation.

The logic is the same for an A/B test as for MVTs and A/B/n tests; you'll have more variations and therefore more total traffic, but the calculation is the same per variation. 

Happy testing!

Hudson