0 Likes

Hide losing variations in Results

Status: New
by ‎09-01-2016 September 1, 2016 - edited ‎09-01-2016 September 1, 2016

We need a way to hide losing variations from the results of experiments with several variations.

 

When running an experiment with multiple variations, especially a multivariate test, it is common to pause variations along the way as they become obvious losers. This frees up traffic percentages and allows the better performing variations to potential converge sooner.

 

Unfortunately, there doesn't appear to be any effective way to reflect this in the Results. In fact, depending on the trends of certain metrics, the paused variations can appear to become winners. This can happen when frequency of event drops for the baseline (original) and all remaining variations. Since the number of unique visitors for the paused variations doesn't change, any goal that decreases in frequency will automatically increase in conversion percentage. 

 

While it is possible (albiet tedious) to manually click individual variations out of the graphs/charts, even this function can not be saved in a view and it must be performed individually for each goal. 

 

As an example, here's a selection from the Results page in which all of the apparent winning goals were paused due to lace of performance:

 

Losing Variations.png

Level 7