Rich-text Reply

What is your workflow for X?

santhony7 03-02-17

What is your workflow for X?

When we were using Classic we had a project for each of our environments: QA and Production. We would develop an experiment in the QA project and test it. Once it passed testing we would just duplicate the experiment into the production project and fire it up. With Optimizely X this workflow no longer really works. The new concept of "pages" become a mess as each page is duplicated along with the experiement causing me to now have pages Home Page, Home Page (1), and event Home Page (1)(2). Optimizely recommeneded to me that, with X, we should no longer use multiple projects and just use the Share Link feature to send experiments to QA for testing. However, doing it this way removes all other experiments so it isn't an accurate representation of a true production environment. I have another question about that specifically here: Other Forum Question

So what is the best (or even good enough) way to test an experiment before making it public? Thanks!

Anthony Smith
tomfuertes 03-02-17
 

Re: What is your workflow for X?

Optimizely is always making improvements and I've heard about something on their roadmap to solve this. That said the current 3 ways we tackle this are:

1) We build most of our experiments on Production for this reason since that is where they're most likely to be running and it's less of a headache to shuffle things around b/w pages.

2) Use the same project for both environments. Add a page where you look for substring "test_param=true" and then add your production page and that test param page to the experiment. You'll then be able to activate your test on the staging page by adding the query param.

3) Use the same project for both environments. Separate and duplicate your pages w/ tags [staging] and [prod]. Then when you're ready to launch just swap out the pages.

-Tom
____
- Tom Fuertes | CTO @ CROmetrics / LinkedIn
"Most Impactful Use of Personalization" and "Experience of the Year" Optie award winner.


santhony7 03-02-17
 

Re: What is your workflow for X?

The problem with this is that when I go to swap out pages it wipes out all custom JS and CSS. So when I go to change pages I have to copy the JS and CSS to a text editor, go swap out the page, and then re-add the custom JS and CSS. Not to mention....doesn't your library of pages look messy? I am guessing you have "Staging Home Page" and "Prod Home Page" or something like that?

Thanks for taking the time to reply! I really hope Optimizely helps fix this.
Anthony Smith
Rhys 03-05-17
 

Re: What is your workflow for X?

[ Edited ]

Hi Anthony,

 

I have the same problem too. I spoke the Optly-X PM last week and they acknowledged the problem and are investigating a fix - but no committed date.

 

One of the support engineers (Rocky) gave me a good workaround that works for me.

 

Our environments are named qa.site.com, dev.site.com, www.site.com. When you create your page defintions if you use a regular expression that will capture all 3 of these url's then you wont have the problem when you duplicate experiments. The RegEx below works on our "/goto" page on all three environments.

 

^(?=.*\b.site.com\b)(?=.*\bgoto).*$  

 

Hope that helps.

 

Rhys.

Level 2
santhony7 03-10-17
 

Re: What is your workflow for X?

What Optimizely Support came up with:

You are correct that you would be unable to test across multiple experiments with the Preview Tool in Optimizely X. One option available would be to build and launch the experiment on your production site, but use a Test Cookie as an Audience Condition. This would limit who could enter the experiment and allow you to test across multiple experiments. Since the experiment is already on the on the production site, once everything has been approved you could simple remove the Test Cookie and select the appropriate Audience Conditions. The other option is to continue to test in Optimizely Classic, as you are already accustomed to the work flow."
Anthony Smith
Rhys 03-10-17
 

Re: What is your workflow for X?

Well... the support team obviously don't work in the real world:-). A/B testing new feature releases mean you need to build your experiments on a Dev/QA environment before the code gets to production. The suggestion works for simple A:B tests of existing site content but not necessarily for new features that are not yet in production.
Level 2
santhony7 03-10-17
 

Re: What is your workflow for X?

Exactly what I wrote when I replied to them. :-)
Anthony Smith
emilystewart a week ago
 

Re: What is your workflow for X?

I'm please this has been raised by other people. I posted a new topic in the community (before I saw this) last week - https://community.optimizely.com/t5/Using-Optimizely/Duplicating-tests-in-Optimizely-changing-pages/...

As an ecommerce site we want to run the same test on listing pages, but due to the nature of the feature we are testing the CTA content needs to differ. I can't duplicate tests and the custom changes we made and then edit them. It's very annoying.
JasonDahlin a week ago
 

Re: What is your workflow for X?

[ Edited ]

All of these workarounds are deal breakers on our conversion to X.

 

1- New features exists in QA before they are launched in production, so the experiment needs to be tested in QA while DEV still has time to modify their code if needed to accommodate using Optimizely to turn the feature on or off.

2- we cannot use URL parameters (they don't work when you are testing a page that requires a form submit and the page cannot be re-loaded after the submit).  The similar approach, setting a test cookie, is a PITA for non-technical users

3- Creating the experiment twice means that we are not actually testing the production version of the experiment...

 

Ideal workflow mimics what we use for Classic:

1- Create the experiment and add the QA pages

2- Once QA is complete, duplicate the experiment, remove the QA pages, add the Production pages.

 

TLDR; Manually creating the production version of the experiment rather than duplicating the QA version *will* lead to errors.

--Jason Dahlin
Analytics and Testing Guru Smiley Happy
RyanC a week ago
 

Re: What is your workflow for X?

We usually use the "Test Cookie" method for QA and add that into the audience. With Optimizely X this is much easier because you can join and reuse multiple audiences together, where as Classic we had to constantly modify audiences for each experiment.

And when QA is ready, we usually duplicate the experiment, remove the Test Cookie filter in the audience tab and the experiment would be good to go live.

More info here: https://help.optimizely.com/QA_Campaigns_and_Experiments/Set_a_test_cookie%3A_Preview_a_live_campaig...

So that when
Ryan
------------------------------------------------
The best solution is the simplest.
Level 11
JasonDahlin a week ago
 

Re: What is your workflow for X?

@santhony7- You wrote: "The new concept of "pages" become a mess as each page is duplicated along with the experiement causing me to now have pages Home Page, Home Page (1), and event Home Page (1)(2)".

 

This is entirely not my experience with Optimizely X.

I just created a new experiment "POC-1" with a new page definition "All Pages".  I duplicated the experiment creating "Copy of POC-1".  When I click into the experiment and select "Pages" in the left-navigation, I see that the experiment "Copy of POC-1" is using "All Pages".  It did not duplicate the page definition as you illustrated.

 

Capture.PNG

--Jason Dahlin
Analytics and Testing Guru Smiley Happy