Rich-text Reply

Preview Multiple Experiments

anthonygreco 10-13-14

Preview Multiple Experiments

Say I have Experiment A (3 variations and the original) that focuses on a different header and Experiment B (1 variation and te original) that focuses on inner content.


Experiment A is running across several pages and Experiment B is only on one page but falls within the same url structure as Experiment A, therefore they should both run simulataneously.


This is precisely the intent, but I'm having a couple issues with this plan of action.


The first, is that I can't see both of the tests running together unless at least one of them is live allowing me to preview the other with the Optimizely footer widget. When I load my previewed Experiment B I see there's an arrow with a little popup that says "Stage Experiment" next to my Experiment A; is that effectively the same as previewing? I don't want to guess and turn out to find that "StageExperiment" is equivalent as Start Experiment.


My second question is in regard to quantifying the data. Will the data still be as quantifieable in this fashion? In other words, I know that the data being collected for each individual test should remain intact, but what about the variations of variations? For instance, what if we wanted to see what did best between all possible variations:


Experiment A Original with Experiment B Original
Experiment A Variation 1 with Experiment B Original
Experiment A Variation 2 with Experiment B Original
Experiment A Variation 3 with Experiment B Original
Experiment A Original with Experiment B Variation 1
Experiment A Variation 1 with Experiment B Variation 1
Experiment A Variation 2 with Experiment B Variation 1
Experiment A Variation 3 with Experiment B Variation 1


Any input and/or advice is appreciated!

Amanda 10-13-14

Re: Preview Multiple Experiments

For your first question, the stage button will not start your experiment live. It will just allow you to run that experiment within the preview itself. You can then toggle between experiment variations.
Khattaab 10-15-14

Re: Preview Multiple Experiments

@anthonygreco the best way to capture data on combinations of variaitons is to run a multivariate test (MVT). However, MVTs require substantially more traffic than simple A/B/n tests.


If you do not have sufficient traffic for an MVT, the cleanest way to run this combination is through a single multi-page experiment where you manually activate (found in the Options menu under Activation Mode) Experiment B on a given page within your user path. For example, say you have a 3-step process where Experiment A (header) will be applied to all steps. You can then activate Experiment B (text/image) on page 2 of the process for all variations 1-3; page 3 loads would be an appropriate goal. This format ensures that anytime Experiment A Variation 1-3 is being shown os page 2, Experiment B is as well. The only limitation is that you will be unable to track Experiment A Orginal with Experiment B Variation 1; all other combinations would be tracked.


Please refer to the following links for additioinal context on the available activation modes:
Knowledge Base


Khattaab Khan
Director, Experience Optimization | BVAccel
Level 5