Rich-text Reply

One of our winning tests using Optimizely :)

Alix 04-22-14
Accepted Solution

One of our winning tests using Optimizely :)

[ Edited ]

We've been using Optimizely since 2011 and after attending Opticon last week, it put my mind to a test we'd set live just a week before - a test that was out of our comfort zones.

 

We have a German side to our business, but they focussed more on retail bookings than driving conversions through their SEM pages, so we wanted to jump in and start driving up the booking rate across our German sites. The challenges were, firstly, the language barrier - we needed to ensure we weren't stripping any key messaging out of the page, but also the business model. Based on the data we'd collected, German customers prefer to research products and pricing online before going into a travel agents to finally book.

 

Having looked at the pages, we came up with a priority list of iterative tests we wanted to run to drive up bookings. The first thing the team noted was that the same pages were being used across both organic and paid search traffic, so there was a ton of content on the page, loads of images, really pronounced navigation, so in short, lots of distractions for a PPC page. Our hypothesis was as follows:

 

By removing the noise and distractions, the pages' purpose will be clearer to customers and therefore we believe customers will be more likely to progress through the booking flow and purchase, rather than be put off by information overload. 

 

Using Optimizely, we created an A/B test variation for the landing page with the majority of the SEO content, navbar, external links, etc removed. We left the form, the main image and key USPs, footer, header and 'amend booking' link.

 

You can see the original page here: http://goo.gl/wFKzCG

And the test page here: http://goo.gl/2dn8Uf (as you can see, the test is for the purposes of validating the concept rather than testing a polished re-design). 

 

The test has now reached a stastically significent result and the variation page has generated +49.19% booking rate vs. the control, as well as +89.10% search-through rate!!

 

It was great to drive such success on our first iterative test on the German site, and it further cemented that split-testing is an essential part of the CRO process. Running this kind of test meant that we didn't need to invest any developer time before we had proof of concept - I could jump in, make the changes, and go live within an hour (including testing time). Now we can build those changes in with full confidence in the uplift value. Smiley Happy

Level 2

zemaniac 04-22-14
 

Re: One of our winning tests using Optimizely :)

Great story! Thanks for sharing.
Alessio Romito
Project Manager at 21DIAMONDS GmbH
Level 2
ShanaR 04-22-14
 

Re: One of our winning tests using Optimizely :)

[ Edited ]

Alix, thanks for sharing, and congratulations on this success!

Stripping away most of the content from your landing page is what we call a 'fearless' test. I love that you included your challenges and the process around forming a hypothesis for the experiment.

If you run any follow-up experiments that build off this win, be sure to let us know!

Shana Rusonis
Content Marketing Manager, Optimizely
shana@optimizely.com
Optimizely
HeatherW 04-23-14
 

Re: One of our winning tests using Optimizely :)

This is incredible! Thank you so much for sharing Alix!
Optimizely
Valentin 04-23-14
 

Re: One of our winning tests using Optimizely :)

This is what testing is all about!
Level 2
Jonathan 04-26-14
 

Re: One of our winning tests using Optimizely :)

Loved it. Thanks for sharing Alix.

It looks like all this extra info DID raise the anxiety level. I like the "old school" trick of removing the menu as well as the bread crumbs.
Level 2
Mike_in_SF 04-28-14
 

Re: One of our winning tests using Optimizely :)

Great results!

We tested a similar solution (not quite a 'fearless' test as @ShanaR mentions below) but one where we removed breadcrumbs, header 'noise', stripped every single CTA except for buy now (customer has a trial option, but we wanted to see what would happen...) and our conversion rate went up by ~50%

Sometimes as marketers we get in our own way with all the great stuff we have to say! Taking a 'fearless' approach sometimes helps remind us to Keep It Simple Stupid (K.I.S.S.)

Keep on testing!
- Mike
Jonathan 04-29-14
 

Re: One of our winning tests using Optimizely :)

I'm very intruiged. I'll see what I can apply to this one.

http://www.restorationsos.com/water-damage/services.asp

 

I'm always suspect removing the two bottom paragraphs will do damage. since you're leaving no room for the customer to understand who you are but I might have to try.

 

What do you think?

Level 2
Alix 04-30-14
 

Re: One of our winning tests using Optimizely :)

Hi Jonathan,

 

By all means trial the removal of the content - in this case, you may want to take a more cautious approach and split-test removing only certain sections in an A/B, as the site isn't especially content-heavy.

 

It might be worth split-testing changing the order of the content. Looking at the site you posted up, I wasn't 100% sure why I'd need water removed/dried until I saw the sections at the bottom of the page, specifically the 'Assisting With' and 'What you need to do. Now!'.

 

Once I read those, I had a much firmer grasp of the services provided, so potentially it's worth trialling those sections higher up the page nearer to the 'Free Onsite Analysis' form to persuade more people to complete it (if that's your KPI). 

 

Hope that's helpful. Smiley Happy

 

 

Level 2
Alix 04-30-14
 

Re: One of our winning tests using Optimizely :)

Thanks for taking the time to leave comments!

We were really pleased with the results of the split-tests and are in the process of rolling out the changes we made and setting up the next test.

For those of you who have had time to take a look at the links I've posted to the original and test variations of the page, you might have a fair idea of another quick and dirty test we're going to run - it's a split-testing favourite; the CTA button.

You'll notice that currently the button is grey, which customers usually associate with disabled, or discouraged interaction so we're going to trial different button colours in order to test what will drive more form searches.

This test is pretty much challenge-free, but to be on the safe side, we've done some research on the German market. Colours have different symbolism in different countries and we wanted to make sure that we didn't test a colour that symbolised 'confidence' in Britain, but 'uncertainty' in Germany! For example, some sources suggested that the colour yellow symbolised envy, and that grey was associated with pessimism.

If people are interested, I can post-up the particulars of the test when we launch. Smiley Happy

Level 2
Jonathan 04-30-14
 

Re: One of our winning tests using Optimizely :)

There is one thing with the type of user my business accommodates that makes the difference. If they land on my site/landing page - they KNOW why.

 

I'm in the emergency industry so they don't browse to my site at 3am out of boredom. For them, there is no need to an explanation why they need the service.

 

Based on that, would you still get rid of the text?

Level 2
Alix 04-30-14
 

Re: One of our winning tests using Optimizely :)

I probably should have asked for clarity on the source of your traffic - my bad. Smiley Happy

Along those lines, what are your KPIs? Do you want to drive up more phone calls whether they come directly from ringing the number on your site or filling in the form, or is there more value in customers who fill out the call-back form, for example.

Who are your audience? Are you B2B or B2C? Do you get many repeat customers hitting this page or is it all new business?

If I assume the page is for PPC traffic, B2C and for new customers, I personally feel like I'd need all that information to buy into the service, particularly as I have no experience of your product offering. Trialling a re-order of content could potentially yield uplift if you've not tested this before.

Side note - there are two typo's I've noticed on this page. I.e. 'satisfation' should be 'satisfaction' in both the link in your bullet list and the penultimate section title at the bottom, and 'expirence' should be 'experience' in the last section. Only little things, but can help add more integrity to the page.


Level 2
Jonathan 04-30-14
 

Re: One of our winning tests using Optimizely :)

I don't have a specific prefrence as far as phone calls or form submissions. Both are great. I wanted to accomodate the user and give him the feeling we're easy to work with that's why I'm giving both option. Thank for the heads up on the typos. I'll fix it.

 

I am B2C. With 0 repeat customers. This is all new business. Yes, the page is PPC. I see this page for so long now, I have no fresh eyes anymore and it's hard to judge it.

Level 2