Rich-text Reply

Mutually exclusive experiments and traffic allocation

jordanhorowitz 06-08-16

Mutually exclusive experiments and traffic allocation

I recently asked a related question, and just wanted to follow up with it.

 

So I know that to run two mutually exclusive experiments, both of them have to be running at 100% traffic allocation. Say we start one experiment at 50% traffic allocation. We then want to make it mutually exclusive with another experiment. Can we raise the allocation to 100% and then run the two experiments simultaneously, or does the first have to be at 100% from the beginning?

robertchan 06-08-16
 

Re: Mutually exclusive experiments and traffic allocation

It is not recommended to adjust allocation once your experiment has started. In short, your data may get skewed, therefore, you'll want to duplicate the experiment and start it again with the traffic allocation you'd like to set it to.
Robert Chan

Experimentation Hero
CouchPsycho 06-08-16
 

Re: Mutually exclusive experiments and traffic allocation

Confirmed! I ran into trouble after setting one variant to 0%. Why? Because already bucketed users keep coming performing tracked actions, so the conversion rate got better continuously while i got more and more confused. After i experienced the same within another experiment, i began to understand.



"the essence of the creative act is to see the familiar as strange." (anonymous)