[Email] Subject Line AB Test: Calling out content type
We created an email test with the hypothesis that leads are more likely to click if the subject line highlighted the piece of content included.
Control: Why You’re Crazy to Spend on SEM But Not A/B Testing
Variation: [Ebook] Why You’re Crazy to Spend on SEM But Not A/B Testing
The variation received a 35.3% increase in conversions, with > 99.9% confidence rate!
I have recently carried our some email subject line testing and got some interesting results.
Did you find that the increase came in open-rate or click-through rate?
Solutions Architect | Optimizely, Inc.
The conversion rate here was clicks on email. Interestingly enough, open rates only increased 3.3%, and without >80% statistical power (therefore not statistically significant).
Our audience loves content and a large portion of our leads are in "awareness/education" stage. I also think that letting our leads know that we are providing them with a valuable resource up front is exciting, instead of just using a catchy subject header to increase open rates.
I do agree with Harrison that aligning subject/CTA is important for conversions. Numbers in subject headers are also successful as Harrison also pointed out: we ran an A/B test using "how to increase sales 43%" a subject header, and received an 18% increase in opens.