Rich-text Reply

Testing Symmetry of headlines

DCKevin 09-13-14

Testing Symmetry of headlines

[ Edited ]

I just read the blog post on the importance of symmetry.  In email marketing, we have always explained to our clients the importance of the landing page and the email message having a similar look and feel. If not, the experience can be jarring or confusing to the customer. They may wonder if they made it to the right place, which can make them unsure or less trusting of the page they landed on, for example. 

 

The article, of course, talks about symmetry much more specifically than just look and feel. It tested the importance of the headline of the ad matching the headline on the landing page. I would like to run a similar headline test on a Facebook ad campaign we are about to start for our client. There will be many versions of these Facebook ads, so I want to make sure I am setting up the test correctly. 

 

1. There will be one landing page. 

 

2. I will pass a parameter like ?version=1 in the URL for each unique headline on Facebook.  

 

3. On the landing page, I will create a variation to match each Facebook headline and use Exact Match for each with the appropriate parameter.  50% of visitors to the site from the ads will see the default and the other 50% will see the variation.

 

Am I setting this up correctly? This should test each headline against the default to test the importance of symmetry. It should also help determine which ad with which headline has the winning combination, right?

 

Thanks, 

 

Kevin

Level 2

michaelwei 09-15-14
 

Re: Testing Symmetry of headlines

Hi Kevin,

 

You're definitely on the right track. There are actually two ways to do this, one requires some coding but is more streamlined, the other is very simple but requires multiple experiments.

 

1. Using the version parameter you described, you can code a Javascript switch that looks at this value and changes the headline accordingly. This way, you can use one experiment to handle as many versions as you need. Conceptually, this is what was described in the blog post. The downside is that it groups all versions together. In order to see each version separately, you'll have to make a Segment or custom dimension for each version and then filter them on the results page.

 

2. You set up a different experiment for each version, and use Audience targeting to look at the version parameter. This will require one experiment for each version, but you'll be able to see each version's conversion lift separately.

 

Which one sounds better to you?

Optimizely
DCKevin 09-15-14
 

Re: Testing Symmetry of headlines

Hello, Michael.

Can you insert JavaScript with Optimizely? If not, the simpler version may be a better solution for the client depending on their comfort level with this. This will be their first A/B test.

- Kevin
Level 2
michaelwei 09-15-14
 

Re: Testing Symmetry of headlines

Yep, you can definitely insert JavaScript with Optimizely! In fact, everything you do in the Editor is actually converted into JavaScript. On the Editor, if you open up the <edit code> tab at the bottom right of the window, you can see and edit/add JavaScript directly.
Optimizely
DCKevin 09-15-14
 

Re: Testing Symmetry of headlines

Ack! Well, I guess I knew that. :-) I just wasn't thinking of using the tool that way. I will talk with the client about this. Thank you for your help.

- Kevin
Level 2
Amanda 09-15-14
 

Re: Testing Symmetry of headlines

Awesome, this is a great conversation and the tips highlighted will be useful to many Optiverse members. @DCKevin - if you do decide to add JavaScript to one variation, you should check out the example provided in our support article here. It provides a breakdown of how to use information from the query string parameter to personalize the headline. 

 

Please let us know what you end up deciding to do with the test.

Optimizely