One of my favorite tests run through Optimizely was our Mobile Theme test.
Our client had been toying with the idea of a mobile theme, but wasn't sure if they wanted to make the full investment. Rather than putting together extensive mockups and building out the full site, we launched an out-of-the-box mobile theme with light modifications for proof of concept. This cut down development hours drastically, yet gave the client a sense of whether or not a mobile theme would improve conversion rates. It wasn't necessarily the pretties thing, but it did the trick.
Our Hypothesis: The functionality of a mobile theme will increase revenue per visitor for Mobile visitors.
The test was simple enough to set up. We used Optimizely's redirect feature to add a parameter to 50% of the traffic to send them back to the desktop site, and of course only targeted mobile phones.
After just over a month, both Revenue Per Visitor and Conversion Rate goals had reached statistical confidence with a 37.5% lift in Conversion Rate with 99% chance to beat baseline and 28.1% lift in Revenue per Visitor with 100% chance to beat baseline.
The client was thrilled (needless to say), and we quickly got to work on mocking up a more aesthetically pleasing and on-brand experience for their customers.
Has anyone else ever tried a quick "proof of concept" test? They are risky to run, but can definitely pay off!
Here are images of the mobile version of the desktop theme, followed by the mobile theme for references!
