Rich-text Reply

How to reflect Original's changes to Variation

Eram 12-19-16

How to reflect Original's changes to Variation

We are going to add a change to our website. I was wondering how I can reflect that change to the test variation without reruning the test.

 

The change is a development change and it is not easy to add it to the Variation in Optimizely.

Level 2

RyanC 12-20-16
 

Re: How to reflect Original's changes to Variation

Not sure if I understood your question correctly, but I think a redirection test be an option in your case.
Ryan
------------------------------------------------
The best solution is the simplest.
Level 10
JasonDahlin 12-20-16
 

Re: How to reflect Original's changes to Variation

When the development change goes live, if it is on the same page as your experiment, the experiment will automatically pick up the change. (Note: this change may alter your page in such a way that you need to re-code your experiment - you should definitely test the experiment on the modified page before the change goes to production).

If the development change is not on the same page as your experiment but you want to make this change to the experiment's page, you will need to either add it to the experiment (which you said will be difficult) or have the developers also make the change to the page where the experiment is running. In the latter case, be sure to re-test your experiment before the change goes to production.

Lastly - if this change creates some sort of fundamental/major change to your page (anything more significant than minor text changes), you should consider re-running the experiment (by creating a clone of the existing experiment, pausing the current experiment, and starting the clone).
--Jason Dahlin
Analytics and Testing Guru Smiley Happy
Eram 12-21-16
 

Re: How to reflect Original's changes to Variation

Thank you @JasonDahlin

 

The change is happening in the same page. I thought the experiment wouldn't pick up the development changes as I previously had an experience that changing a banner photo did not reflect on the experiment page. But I can try it again.

Level 2
JDahlinANF 12-21-16
 

Re: How to reflect Original's changes to Variation

[ Edited ]

disregard - posted as wrong user

JasonDahlin 12-21-16
 

Re: How to reflect Original's changes to Variation

Just FYI...

 

Optimizely's code runs on the page that is delivered to the end user.  Whenever your developers change the page, the experiment will automatically pick up whatever changes they have made.

 

In the example where a DEV change was made but you were not seeing it in the test variation, I'm theorizing that your experiment may have been removing or altering the HTML element that the DEVs had changed (e.g., if they changed the HERO spot on the homepage and your experiment also modifed the same HERO spot on the homepage, your test variation would not reflect the DEV changes because you replaced that spot with your own code.  

 

I've seen this happen before where an experiment was designed to modify the CTA in a HERO (making the button say "Apply Now" instead of "Submit").  The way the experiment was coded, rather than replacing only the text on that one specific button, it actually replaced the entire HERO with exactly the same HTML as the original (except the button text).  In this scenario, if the DEVs modified the background image of the HERO, since that entire spot was being replaced by the expderiment, the experiment maintained the appearance of the original HERO's background image.

 
 
--Jason Dahlin
Analytics and Testing Guru Smiley Happy
robertchan 12-21-16
 

Re: How to reflect Original's changes to Variation

As @JasonDahlin had mentioned, changes made by the developers will automatically update in Optimizely. One thing to note though is that visitors get bucketed once they hit your experiment after you've started it. So if you're already half into your experiment then you'll have an issue with changes by your developers skewing your results if their changes impact the original user experience.
Robert Chan