Rich-text Reply

Split Testing Checkout Process

jamesmerr 10-08-14

Split Testing Checkout Process

Hello everyone - long time user, first time poster.


I'm in the process of creating a split test that will alter the checkout experience on our site.  We're just about done with the required changes on the site and I want to start creating the experiment.  


Here's how it works - when we want to trigger the test checkout experience, we modify some of the HTML attributes of the 'checkout button' on a particular product page.  Then when a user clicks the checkout button, they funnel into the new version.  A couple concerns:


1. all of the products on our site follow a URL structure of{productName}.  what i'd like to do is write a regex that will evaluate a URL to make sure it's a product page and then modify the HTML attributes if they should get the variation experience.  Is this possible? Is there an easier way that I'm missing?


2. once a user gets assigned a particular experience (original vs. variation), I want that experience to be stuck with them for at least the length of their session.  for example some users will click checkout, make it part way through the checkout process, go back and do more browsing, and then re-enter the checkout process again.  I want to make sure they still see the same version they did the first time rather than re-entering the experiement and potentially getting the OTHER variation. Is the variation a user sees stored in session state or will it change within the same session?


thanks in advance!



adzeds 10-08-14

Re: Split Testing Checkout Process


In answer to your questions.

1 - Could you simply do your matching on a sub-string basis if all your URLs are the same structure:

2 - When a visitor is bucketed in a test they will continue to see that version whenever they come back to the site. This is true so long as they are using the same browser as before!
David Shaw
Level 11
jamesmerr 10-09-14

Re: Split Testing Checkout Process

Hi David - great suggestion, thanks. Couple follow up questions. I'm in the process of creating the experiment in our test environment.

1. When I initially create the experiment, it wants me to specify a page...if I enter, we will throw a 404 because our directory structure requires a product to be specified. Should I just enter one eg. and then change URL targeting to be on substring match once the experiment is created?

2. On the variation page, I need to inject some JS to get the modified experience. I ONLY want that JS to fire on the variation page, not the original. I looked in optimizely and I could only find options to inject Global JS (which fires on the original and variation pages). Is there a way to drop some JS on only the variation pages?