Voices.com Realizes a 33% Increase in Sign Ups
Optimizely Voices.com Case Study
We've been around the testing block a few times, most famously for a case study on the Conversion Rate Experts blog where we realized a 400% increase in sign-ups. That story was also covered on the Google Website Optimizer blog.
In searching for a new tool, we settled upon Optimizely because it came highly recommended.
Why Optimization Matters
As I've emphasized with our team, we’re resisting being a company-centric organization. Rather, we continually strive to being customer-centric. One way we can stay focused on our customers is to look at our website through their eyes. Then, when we make changes, it's because customers have voted with their clicks. As an organization, we want to challenge the status quo, eliminate guess work and make data-driven decisions.
Rather than jumping right into split testing, I learned at the Website Optimization conference about the importance of variance testing. We needed to determine the natural variation in sign-ups on any given day. By knowing this number, the “variance,” we can eliminate false positives and know with confidence that the number is statistically significant. As you can see from the variance test, we needed a test improvement greater than 0.4% for us to be confident that the test was indeed accurate and it should be considered conclusive.
The current version of our home page features ten (10) professional voice actors, five (5) male voices and five (5) female voices. We originally designed the home page this way because we hypothesized that by featuring voice talent on our home page, we would show visitors that we have “real talent" and “real people,” and thus increase conversions.
With our intense focus on using the main page of the website to inform visitors and lead them towards signing up for either a free or paid account, we must be willing to challenge the status quo. The hypothesis was by removing the “Featured Voice Talent” on our home page, we will focus on visitors on the website’s value (is this right for them) and encourage more sign-ups.
The Home Page testing originally ran for 10 days with 11,258 visitors participating. A follow-up test ran for 5 days with 6,463. We set-up “goals” in Optimizely to identify a successful outcome as a sign-up for any level of service with Voices.com.
Both tests resulted conclusively that by hiding the Featured Voice Talent, we are able to acquire more sign-ups. One test achieved a 28% increase and the follow-up test yielded a 33% increase, as displayed in the screenshot below.
For the voice talent on the website who may have previously gained some visibility by being featured, they now benefit by having more clients such as video producers, creative directors and advertising executives sign-up, search for and ultimately hire more voice actors - not just the few people previously on the home page. Now, the Featured Voice Talent are showcased as a series of Top 100 lists.
Throughout my decade of experience, I'm continuously reminded that you should test all your changes, big and small. These results in particular taught me that:
We should create a hypothesis for each test
We should test everything
We should be confident in our changes, because they were scientifically proven to increase conversions