Rich-text Reply

[Email] Subject Line AB Test: Calling out content type

Allison 07-21-14

[Email] Subject Line AB Test: Calling out content type

[ Edited ]

We created an email test with the hypothesis that leads are more likely to click if the subject line highlighted the piece of content included.


Conversion: clicks


Control:   Why You’re Crazy to Spend on SEM But Not A/B Testing
Variation:   [Ebook] Why You’re Crazy to Spend on SEM But Not A/B Testing


The variation received a 35.3% increase in conversions, with > 99.9% confidence rate!

Optimizely

Amanda 07-21-14
 

Re: Testing Idea for Email Subject Line: Content type

This is an incredible result! Why do you think that including the content signal [Ebook] into the subject line had such a dramatic effect?
Optimizely
adzeds 07-22-14
 

Re: Testing Idea for Email Subject Line: Content type

That is an impressive result..

I have recently carried our some email subject line testing and got some interesting results.

Did you find that the increase came in open-rate or click-through rate?

David Shaw
Level 11

Re: Testing Idea for Email Subject Line: Content type

Action-oriented email subject lines do regularly lead to higher open rates. Obviously, a higher open rate means more eyeballs on your content and a higher chance more people will click. However, higher email opens don't necessarily cause higher email clicks, and in edge cases, it is possible to record an email click without recording an open. The keys to guiding the user through the email interaction experience are a) action- or specific data oriented subject lines (e.g. "Save $35" is better than "Great Savings"), b) clear alignment between the email subject line and content, and c) a clear CTA above the fold in the email copy.
Harrison Krat
Solutions Architect | Optimizely, Inc.
harrison@optimizely.com
 
Allison 07-29-14
 

Re: Testing Idea for Email Subject Line: Content type

Hey all,

 

The conversion rate here was clicks on email. Interestingly enough, open rates only increased 3.3%, and without >80% statistical power (therefore not statistically significant).

 

Our audience loves content and a large portion of our leads are in "awareness/education" stage. I also think that letting our leads know that we are providing them with a valuable resource up front is exciting, instead of just using a catchy subject header to increase open rates.

 

I do agree with Harrison that aligning subject/CTA is important for conversions. Numbers in subject headers are also successful as Harrison also pointed out: we ran an A/B test using "how to increase sales 43%" a subject header, and received an 18% increase in opens.

Optimizely