Rich-text Reply

allocation percentage convergence

fooyourhomesuit 10-09-16

allocation percentage convergence

Hi, I was curious how quickly the allocation percentage converges. Suppose I have an experiment with variant A of 10% & variant B of 90%. Suppose I got a perfect distribution after 100 samples of A == 10, B == 90. Let's say I change allocation of A to 50% & B to 50%. Does that mean the next 80 samples will be put in the B bucket so it will converge to A == 90 & B == 90? Thanks.
CouchPsycho 10-09-16
 

Re: allocation percentage convergence

Hi,

to keep in short: No. As far as i know, the percentages are the probabilities for a user being assigned to a bucket. This assignment has no memory regarding other users.

But keep in mind that changing allocation values might have an undesired influence on your results. Whenever changing them it is a good idea to copy your experiment and starrt the new one, as long as you do not fear to confuse a larger set of users by showing them something new.

Kind regards

Michael



"the essence of the creative act is to see the familiar as strange." (anonymous)
LightYear 10-10-16
 

Re: allocation percentage convergence

It is not a good idea to put 100% of the experimental units to Bucket B for a paricular time peirod, which means that something happen during that time period could become very influential for the results, and there would be no way to tease it out.