OptiCon 2014: Is This Normal? Common Challenges Testing Programs Face

by HeatherW ‎05-05-2014 May 5, 2014

You are doing it by the book. After a thoughtful search, you choose and implement Optimizely. You've built organizational buy--in for testing, and your colleagues are now sending you their hypotheses. You have even run your first set of tests and perhaps found a winner. Your business is growing and you are data driven now ... right?

For many organizations, it's not that easy. The establishment of needle moving optimization programs inside of often complex organizations with federated departments who all feel at least partial ownership of testing is challenging and full of growing pains and existential crises along the way.

In this discussion, we will get real and offer practical perspective on the common and healthy optimization program challenges many organizations face. Example scenarios include:

  1. We've run a bunch of quick, easy tests and nothing is really moving the needle. Where's the lift?
  2. We have more ideas than we could ever possibly test. How do we prioritize?
  3. We launched a test that broke stuff on our site! I thought building tests was easy?
  4. We said that we should test everything but now realize that is impossible. Does anyone really test everything? Is this program working? How do we keep score?
  5. We've had a few winners and now our bosses want to know the annual growth we can expect from them. How do we answer that?

Ryan Garner, Co--Founder of Clearhead, and Jessica Vasbinder, Senior Director at Warner Music Group, discuss what healthy and winning optimization programs really look like based on the experiences of great brands like Warner Music Group, FreshDirect, Blu Dot and more.



Ryan Garner, Co--Founder of Clearhead

Jessica Vasbinder, Senior Director at Warner Music Group




Download the Presentation Slides