Deciding traffic distribution
So my team and I are discussing when it is best to do accerlated impact vs manual. There seem to be differing opinions on what is best: using manual to obtain statistical significance and splitting distribution 50/50 versus running an experiment on accerlate impact, not worrying abotu statistical significance and just obtaining conversions.
Solved! Go to Solution.
If you want to know what is best long-term, then go with 50-50 so you have better chance of not implementing bad changes into your baseline.
If your issue is time-sensitive (like testing a blog post headline that is only going to generate traffic for a few hours or a few days), then you're probably not going to get statistical significance anyway (unless one variation really really sucks), go with the Bandit approach.
Analytics and Testing Guru