Rich-text Reply

Advice sought on how to measure the removal of an opportunity to unconvert

ahrensj 07-12-16

Advice sought on how to measure the removal of an opportunity to unconvert

My brain is fried Smiley Frustrated I am struggling to see a way to measure a change that I have been asked to test. Can anyone help?

 

On step 1 of our quote and buy funnel visitors are auto-opted in to marketing (pre-quote stage)

They are given the opportunity to opt out

Unsurprisingly a lot visitors do opt-out

 

On step 3 (optional step) of our funnel (post quote)

And step 7 of our funnel (post purchase)

Visitors are again given the opportunity to opt-out

 

We believe that many visitors are opting out as they have been given the opportunity to opt out too early in the funnel before they are invested in the brand/product.

We would like to test the impact on conversion for step 3 and step 7 if we were to remove the marketing question from step 1 of the funnel.

 

What I am struggling with is how to measure this in Optimizely, I am not sure it’s possible.

 

I would like to measure the clicks on the checkboxes for each step in the funnel separately, however the checkboxes are identical across the funnel so when I create the click goal on the original it tracks the goal on all of the steps.

Is there any way to segment the results so that I can see which page the goal was clicked on. Or can you think of a way that I can measure the impact of removing the questions in step 1.

 

Thanks in advance for any suggestions no matter how small.

Jackie

Level 2

AllisonR 07-13-16
 

Re: Advice sought on how to measure the removal of an opportunity to unconvert

Hi Jackie,

 

Thanks for reaching out about this! Depending on how you want to be able to view your results there are potentially two ways to set this up. 

 

Page-specific click goals: 

By default a click goal is targeted to the same page(s) as the experiment. This can be changed however in the Goal Targeting (described here). Even though the checkbox looks the same in steps 3 and 7, if these steps are on different URLs, you could set up one click goal for each page. You would end up with one goal, for example 'Opt-Out step 3' targeted to an exact match on the step 3 URL, and a similar goal for 'Opt-Out step 7' targeted to the URL of this step. So on your results page you would have a conversion rate of visitors who opted out on step 3 and one for visitors who opted out on step 7.

 

Dimensions (involves some JavaScript knowledge): 

At the time the visitor clicks the opt-out checkbox, capture the step they are on in a dimension value. Details about setting up dimensions are here. The dimension could be named 'opted-out' for example and the possible values would be 3 or 7. This way you can filter your results by visitors who opted out at each step. An example of setting the dimension would be like this: 

  

optimizely.push(['setDimensionValue', 'opted-out', '3']);

 

Does this help? 

 

Allison
Sr. Technical Support Engineer
Optimizely | EMEA
Optimizely
ahrensj 07-21-16
 

Re: Advice sought on how to measure the removal of an opportunity to unconvert

Hi Allison

 

I have gone down the Page-specific click goals and all seems well, however if I click on the checkbox twice that fires off 2 events in the console (currently testing in the developer console) does this mean that the goal will be counted twice?

 

Many thanks

Jackie

Level 2

Re: Advice sought on how to measure the removal of an opportunity to unconvert

Hi,

 

Allison's advice is correct, but I wanted to offer an alternative option as well.

 

If you are or have the help of a developer, I would recommend you use custom events to track this action.

 

One of the problems you have is that, just because someone clicks the link, does not mean they have opted out. It sounds like its possible for them to opt-back in on the same step. So tracking clicks here wouldn't really give you a very accurate answer.

 

One way you could overcome this would be using cookies and custom events. 

 

You can start loading step 1 of your flow by placing an opt-in cookie. If the user clicks the opt-out button, you can overwrite that cookie with and opt-out version of the cookie (And repeat depending how many times they take the action).

 

You can continue this on the other step that the user has the option as well. 

 

This way, once the user gets to the end of the flow they have the most up-to-date cookie availible (their opt-in/out status). When they finally complete your flow/transact/buy/whatever you fire the custom event. 

 

If opt-in cookie exists, fire "custom event - Opt-In",

If opt-out cookie exists, fire "custom event - Opt-out", 

 

You don't necessarily need both of these, since one of them implies the other, but you might as well just to make sure the tracking is right!

 

There are always many ways to track things, so hopefully this provides another option for you. At WiderFunnel (where I work) we have found combining cookies and custom events to be extremely powerful for tracking things unique situations. It makes it possible to track pretty much anything.

 

 

 

In response to your question to Allison, the answer is "yes, and no"! Optimizely has the ability to filter the results by "unique" or "totals". If you are looking at the unique goal (the default view) then even if you click it twice it will only record it once, because you are one person. In the "totals" view then it will show 2 because it is counting the absolute number of times that the link is clicked.

 

Michael St Laurent Smiley Happy

Optimization Strategist, WiderFunnel

 

Mike St Laurent - Optimization Strategist, WiderFunnel
JasonDahlin 07-21-16
 

Re: Advice sought on how to measure the removal of an opportunity to unconvert

To track "opt outs", only send the event n the user is clicking on a checked checkbox.  Don't bind to an unchecked checkbox.  e.g.,

 

$('input#OptOutBoxHere').on('click',function(){
  if ($('input#OptOutBoxHere').is(':checked')) {
    //user opted in
  } else {
    //user opted out
  }
})
--Jason Dahlin
Analytics and Testing Guru Smiley Happy
ahrensj 07-25-16
 

Re: Advice sought on how to measure the removal of an opportunity to unconvert

Thanks very much, I will speak to my developer and check out the various suggestions, much appreciated.

 

Level 2
ahrensj 07-25-16
 

Re: Advice sought on how to measure the removal of an opportunity to unconvert

Thanks very much, I will speak to my developer and check out the various suggestions, much appreciated.

 

Level 2