What is your workflow for X?
When we were using Classic we had a project for each of our environments: QA and Production. We would develop an experiment in the QA project and test it. Once it passed testing we would just duplicate the experiment into the production project and fire it up. With Optimizely X this workflow no longer really works. The new concept of "pages" become a mess as each page is duplicated along with the experiement causing me to now have pages Home Page, Home Page (1), and event Home Page (1)(2). Optimizely recommeneded to me that, with X, we should no longer use multiple projects and just use the Share Link feature to send experiments to QA for testing. However, doing it this way removes all other experiments so it isn't an accurate representation of a true production environment. I have another question about that specifically here: Other Forum Question
So what is the best (or even good enough) way to test an experiment before making it public? Thanks!
1) We build most of our experiments on Production for this reason since that is where they're most likely to be running and it's less of a headache to shuffle things around b/w pages.
2) Use the same project for both environments. Add a page where you look for substring "test_param=true" and then add your production page and that test param page to the experiment. You'll then be able to activate your test on the staging page by adding the query param.
3) Use the same project for both environments. Separate and duplicate your pages w/ tags [staging] and [prod]. Then when you're ready to launch just swap out the pages.
- Tom Fuertes | CTO @ CROmetrics / LinkedIn
"Most Impactful Use of Personalization" and "Experience of the Year" Optie award winner.
Thanks for taking the time to reply! I really hope Optimizely helps fix this.
I have the same problem too. I spoke the Optly-X PM last week and they acknowledged the problem and are investigating a fix - but no committed date.
One of the support engineers (Rocky) gave me a good workaround that works for me.
Our environments are named qa.site.com, dev.site.com, www.site.com. When you create your page defintions if you use a regular expression that will capture all 3 of these url's then you wont have the problem when you duplicate experiments. The RegEx below works on our "/goto" page on all three environments.
Hope that helps.
You are correct that you would be unable to test across multiple experiments with the Preview Tool in Optimizely X. One option available would be to build and launch the experiment on your production site, but use a Test Cookie as an Audience Condition. This would limit who could enter the experiment and allow you to test across multiple experiments. Since the experiment is already on the on the production site, once everything has been approved you could simple remove the Test Cookie and select the appropriate Audience Conditions. The other option is to continue to test in Optimizely Classic, as you are already accustomed to the work flow."
As an ecommerce site we want to run the same test on listing pages, but due to the nature of the feature we are testing the CTA content needs to differ. I can't duplicate tests and the custom changes we made and then edit them. It's very annoying.
All of these workarounds are deal breakers on our conversion to X.
1- New features exists in QA before they are launched in production, so the experiment needs to be tested in QA while DEV still has time to modify their code if needed to accommodate using Optimizely to turn the feature on or off.
2- we cannot use URL parameters (they don't work when you are testing a page that requires a form submit and the page cannot be re-loaded after the submit). The similar approach, setting a test cookie, is a PITA for non-technical users
3- Creating the experiment twice means that we are not actually testing the production version of the experiment...
Ideal workflow mimics what we use for Classic:
1- Create the experiment and add the QA pages
2- Once QA is complete, duplicate the experiment, remove the QA pages, add the Production pages.
TLDR; Manually creating the production version of the experiment rather than duplicating the QA version *will* lead to errors.
Analytics and Testing Guru
And when QA is ready, we usually duplicate the experiment, remove the Test Cookie filter in the audience tab and the experiment would be good to go live.
More info here: https://help.optimizely.com/QA_Campaigns_and_Experiments/Set_a_test_cookie%3A_Preview_a_live_campaig...
So that when
The best solution is the simplest.
@santhony7- You wrote: "The new concept of "pages" become a mess as each page is duplicated along with the experiement causing me to now have pages Home Page, Home Page (1), and event Home Page (1)(2)".
This is entirely not my experience with Optimizely X.
I just created a new experiment "POC-1" with a new page definition "All Pages". I duplicated the experiment creating "Copy of POC-1". When I click into the experiment and select "Pages" in the left-navigation, I see that the experiment "Copy of POC-1" is using "All Pages". It did not duplicate the page definition as you illustrated.
Analytics and Testing Guru