Rich-text Reply

We would like to test a specific element across multiple pages ( but there may be a catch)

Lisa_Norris 08-13-14
Accepted Solution

We would like to test a specific element across multiple pages ( but there may be a catch)

We would like to conduct a simple test in our Ecommerce site 

The hypothisis is that if we change the wording from "Add to cart " to "BUY NOW" that this stronger call to action will increase revenue.

 

The issue we are having is applying this change site wide to all current buttons that say "add to cart"

 

we have read all the instruction on URL targeting but still cannot seem to get it to work.......

 

One possibility that may be an issue is how our URL structure works ( and this is the possibile catch)

 

our category pages are as follows

 

http://www.qispackaging.com.au/Product/paper-bags-coloured.aspx

 

and our Product pages are as follows

 

http://www.qispackaging.com.au/Product/paper-bags-coloured/coloured-bags-with-paper-twist-handle.asp...

 

BUT .....the add to cart buttons only appear on the Product pages and NOT the category pages.

 

In URL Targeting we have selected http://www.qispackaging.com.au/Product/     and the match validator say that it works ...

 

any help/ideas/suggestions/  would be greatly appreciated

 

Thanks Guys !!

 

 

Amanda 08-13-14
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

I assume that you got it working because when I visit http://www.qispackaging.com.au/Product/paper-bags-coloured/coloured-bags-with-paper-twist-handle.asp... I am actually seeing the "Buy Now" CTA Smiley Happy.

 

However, although it works, the targeting set-up is not ideal. The reason being that a substring match to http://www.qispackaging.com.au/Product/  includes pages that don't have the "Add to cart" button, thus inflating your visitor count (the experiment is running on pages it doesn't need to). This will skew your results.

 

Here's how I suggest setting up targeting: 

 

Use a Regular Expression Match:   .*qispackaging.com.au\/Product\/.*\/.*

 

This will ensure that the experiment runs on any page that follows the URL structure:[anything]ispackaging.com.au/Product/[anything]/[anything]. But, it will not run on the category pages due to the missing content detailing the specific product (the final  /[anything] ). For additional help on Regular Expressions, I like to use http://rubular.com/

 

Product page example:

Screen Shot 2014-08-12 at 7.07.27 PM.png

 

 

 

Category page example: 

Screen Shot 2014-08-12 at 7.07.39 PM.png

 

 

 

Let me know if you have any questions! 

Optimizely
Lisa_Norris 08-13-14
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

Hi Amanda, Thank you so much for your help. You are a legend. Thanks for the Regex string - I need to learn a bit more about this but I followed your instructions. We have turned the test back on after changing the targeting. If you have time to double check us, that would be so appreciated. We are really new to testing.

Cheers
Lisa
Amanda 08-13-14
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

[ Edited ]

It looks good to me! I double checked your targeting and confirmed it's running on the correct pages. The button text also changes on the "read more" pages as well which seems correct. 

 

However, there is at least one URL that doesn't follow the structure as outlined above: http://www.qispackaging.com.au/Product/Heat-Sealers.aspx. I recommend just adding this URL manually to the targeting as in the screenshot below. You can do the same if there are any others like this. 

 

For reference:

If you have dev resources or can add snippets of code to your page, you should check out the custom tag targeting. This will make it easy for you to isolate page types. 

 

Since you're new to Optimizely and A/B testing, I strongly recommend the Academy as a place to start. There are some wonderful tutorials and hands-on activities. 

 

Screen Shot 2014-08-12 at 9.18.20 PM.png

Optimizely
Lisa_Norris 08-13-14
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

Thanks Amanda,

great advice , your a gem !!
Lisa_Norris 08-13-14
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

Amanda,

Just a quick question , out of all the categories that we have , how did you find that particular one was not working ??
adzeds 08-13-14
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

Regex and targetting can often be a bit of a minefield.

 

I always find it useful to pull out a report from Google Analytics with all traffic'd pages on the site and check my rules against that list just to be sure we are not targetting the wrong pages as this can often skew your results.

David Shaw
Level 11
Amanda 08-13-14
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

@Lisa_Norris - I just visted your homepage and did a spot check to confirm each of the pages were working as expected based on the URL structure you provided in the intial post. 

Optimizely
Gina123 03-10-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

Hi @Amanda 

 

We're having a similar problem. Can you please explain how including pages that don't have the element we're testing will skew results? Can we still run the test and trust the results?

 

We don't have a lot of experience writing regular expressions. We're hoping to launch the test in next few hours.

 

Thanks,

Gina

Level 2
Amanda 03-10-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

@Gina123  - It will skew your results because your visitor count for the experiment will increase every time a unique visitor hits a page targeted by your experiment. So, if your visitors are landing on pages that do not have the element, then the visitor count increases, but the conversion rate will not increase since the visitor is not technically seeing any changes to the page. Does that make sense?

 

I'm sure that I can help point you in the right direction for regular expressions or alternate targeting structure. Can you post an example of the pages for which you are hoping to run an experiment?

Optimizely
Gina123 03-10-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

@Amanda Is it okay if the conversion rate doesn't increase? In other words, would this issue equally affect all variations and therefore cancel itself out?

Level 2
Gina123 03-10-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

@Amanda Or, would it be okay to only compare the Unique Conversions instead of the Conversion Rates?

Level 2
Amanda 03-10-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

Hi @Gina123 , as long as you recognize that unique visitors will be high, then you are okay. You are right in that the effect will be the equal for all variations. Let me know if you have additional questions

Optimizely
Gina123 03-10-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

Thank you @Amanda !

Level 2
adzeds 03-11-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

@Gina123  & @Amanda 

 

I would suggest that this is a much riskier way of testing and I certainly would not recommend taking this approach without knowing what traffic levels you get and how that traffic is distributed across the pages that will be affected by this test.

 

I do feel that custom targetting is one of the areas that is lacking in Optimizley and I often look at ways I can get around it on a test by test basis.

David Shaw
Level 11
Gina123 03-12-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

@adzeds @Amanda 

 

My main problem was that the element I'm testing only appears on some of the pages I'm URL targeting. It would be difficult to identify all of the regular expressions where the element does and does not appear.

 

Optimizely helped me focus the test using Conditional Activation. So now the test only tracks visits to pages where the element-in-question appears. Problem solved!

Level 2
Amanda 03-12-15
 

Re: We would like to test a specific element across multiple pages ( but there may be a catch)

So glad to hear it!
Optimizely