Rich-text Reply

Setting up goals for different workflows

alanho 11-02-15
Accepted Solution

Setting up goals for different workflows

[ Edited ]

Hi, I'm running a A/B experiment with two separate pages (and workflows) for the first time.

And I'm having a hard time understand how to set up goals for such an experiment.

 

Assuming my Original workflow consists of a single page form with a single Submit button, and I'd like to test this against a Variation #1 which breaks up the form fields in two progressive pages.

 

How can I define the goals so that I can compare the 'Submit' button clicks between Original and Variation #1?

 

At the same time, I would also like to monitor the number of clicks within the first page of variation #1 (index.php?var=1).

 

(I have tried using the Multi-Page Experiment Type, however I'm still baffled on how that applies in my scenario whereby I have differential number of pages in each workflow and how I can aggregate my conversion rates of each variation using Goals.)

 

Any clues?

Experiment.png

- Alan Ho
Level 3

robertchan 11-02-15
 

Re: Setting up goals for different workflows

I would set up two separate experiments placing tracking on the submit button for both experiments. Assuming the submit button's selectors are the same, after you set up one experiment, you can just use the same goal for the other one. For the "Original," you'll simply have one variant within one A/B experiment that tracks clicks on the submit button. For the other experiment, you'll want to create a multipage experiment. Your first page will have tracking on clicks for the next button (just my recommendation), and your second page will have tracking on the clicks for the submit button. The tracking on the submit button should match the one on your first experiment. At the end of running both experiments, compare the conversion rate of both experiments on the submit button.

I'd also recommend setting both experiments to have tracking via GA to determine bounce rate and amount of time users spent on each page.
Robert Chan

Experimentation Hero
Amanda 11-02-15
 

Re: Setting up goals for different workflows

@alanho - Can you just clarify a couple things: 

 

1. Do you have this set up as a redirect test (it looks like it based on your screenshot)? I.E are you testing 1 URL against a totally separate URL? Or, did you change the functionality of the form just within the variation code?

 

2. In the variation #1, do the "next" buttons and the final "submit" button have the same or different selectors?

 

From what I've learned so far, you should not need a multi page experiment and you also should be able to do this with just 1 experiment. 


@robertchan is right in that if the selector for the "Submit" button is the same, you can just add a goal for this single button and Optimizely will automatically separate clicks into the Original vs Variation. However, if the button in the variation #1 lives on a different URL than where the experiment is running, you will need to specify this in the goal set-up (check out this article for details: https://help.optimizely.com/hc/en-us/articles/200039905-Click-goals#redirect). Also, you'll just want to make sure that your goal is not tracking clicks on the "next" button in the variation instead of the final submit. This is where my question #2 comes into play. 

 

Looking forward to hearing back from you so we can get this sorted out.

 

 

 

Optimizely
alanho 11-03-15
 

Re: Setting up goals for different workflows

@Amanda Thanks for the quick reply.

 

Here're my responses (in blue) to your questions.

 

  1. Do you have this set up as a redirect test (it looks like it based on your screenshot)?
    Yes, this is correct. Variation #1 is hosted on a page (with query parameters attached) that has a different UI (less a field) and functionality (less the validation for the removed field)
  2. In the variation #1, do the "next" buttons and the final "submit" button have the same or different selectors?
    Yes, they would have the same selectors.

 

 

From what I am hearing, it seems that I would need the following set up:

 

  • A redirect experiment with Variation #1 pointing to the challenger URL
  • Specify the goal to target the submit button (on page2.php) <- Do I need to also add in the URL for the original page here as well? Else how can I compare the conversion rates between the two?

Is my assumption above correct? Have I lacked out anything?

- Alan Ho
Level 3
Amanda 11-03-15
 

Re: Setting up goals for different workflows

@alanho you are right. 

 

1. Set up your experiment so that variation #1 redirects to the page with the query parameters attached. 

 

2. Click the goals UI and add a new goal that tracks clicks on the Submit button. In the goal targeting section, you will need to be sure to add the URL where the button exists in the original *and* add the URL where the submit button exists in variation #2.  (feel free to edit the URL match types like you would for URL targeting) 

 

3. Just test to make sure the event is not triggering on the Next button by accident. If it is, you will need to use a more specific selector. Here's an article that might help: http://www.w3schools.com/css/css_attribute_selectors.asp

 

 Screen Shot 2015-11-03 at 11.13.34 AM.png

 

Let me know if that works for you! 

Optimizely
Amanda 11-03-15
 

Re: Setting up goals for different workflows

@alanho - based on your post here https://community.optimizely.com/t5/Using-Optimizely/Tracking-conditional-custom-event/m-p/13101#M41... it sounds like you might have solved this with custom events. With a custom event, you just need to add this goal to your experiment and it will be captured wherever it's triggered within your site code (ie on form submit of either variation). 

Optimizely