Habits of a Highly Successful Optimization Professional

by Optimizely ‎10-19-2015 October 19, 2015 - edited ‎10-20-2015 October 20, 2015

We’ve hit a big milestone this week….the Optiverse Community has reached 4,000 members!  Each community member has unique expertise and a slightly different process for staying organized, sourcing new ideas, checking data, and researching new industry trends. However, there are some commonalities in habits that will help ensure success down the road. 

 

We'd love your help in surfacing these commonalities.  

 

What is one thing that you do without fail every single week? How does this help you become a better Growth Manager? 

 

We're giving away swag to celebrate our 4,000th member! 3 lucky partipants of this discussion will be sent a swag bag to share with your team. We'll announce the winner via email and in Optiverse during the week of November 2, 2015. 

 

For additional inspiration, I would encourage you to check out a prior contest that asked "What are the must-have skills to be an effective Growth Marketer?" We turned all of these valuable responses into a blog post

 

 

 

 

 

 

Comments
by
‎10-20-2015 October 20, 2015

Always question everything you're currently doing to see what you can improve. I think that's something in general that applies greatly to A/B testing/CRO and that would also make you a great growth manager.

Level 10
by Pauline
‎10-20-2015 October 20, 2015

- Open Optimizely blog once a week 

- Open conversion XL blog once a week

- Block time on my agenda to look at my competitor websites and pay attention to new tests and features (2hrs/week)

- Send a bi-weekly update to the company: I use it as a chekpoint to make sure that everything is documented and archived + it raised awareness in the organization about A/B test

- Meeting with my analyst to talk about optimization oppportunities and recent results

- Read our Lab user reports

 

Everyday: question everything, be curious, listen to your customers, take risks.

 

Level 2
by Optimizely
‎10-20-2015 October 20, 2015

Awesome. Thanks @MartijnSch and @Pauline

Optimizely
by smarkman
‎10-21-2015 October 21, 2015

- Look at the previous days final revenue numbers.

- review analytics to see where the revenue came from.

- look at the days meetings

- see what marketing initiatives need to get done

- do work

- check out various news outlets to catch up on the news

Level 1
by BryanClayton
‎10-21-2015 October 21, 2015

Do customer support

 

As our team and revenues has scaled over the past 2 years, I have found it difficult to stay as close to our end users as I once could.

 

To combat this, I hold myself accountable to man our in customer support chat for 1 hour everyday. 7 days a week.

 

This daily first hand interaction with our users allows me to stay grounded in our customer's logic.

 

This has been instrumental in guiding our product development strategy.

 

I can confidently explain to our engineers why a feature is important, and critical having the customer conversations, as well as data to back it up.

 

I have a mantra, "Listen to your customers, or you will have none

Level 1
by
‎10-21-2015 October 21, 2015

Bi-Weekly Workflow

 

  • Send 1 bi-weekly testing roadmap report and  1 analytical report to design, edit, product, web, and business development departments.
     
  • Meeting with the heads of the following departments to go over experiments and elicit ideation: design, product, web, and business development.

  • Manage traffic usage to determine how to distribute traffic allocation for experiments.

  • Manage testing backlog to move items into development based on priority and level of significance.

 


 

Daily Workflow

 

  • Reconcile results with running experiments, and update testing roadmap accordingly

  • Develop experiments based on priority and level of significance. Work is also done with development team and Optimizely's support team when an issue with code comes up toward the end of working on an experiment.
     
  • Analyze competitors with product managers to ideate or manipulate existing experiments.
     
  • Build quarterly PowerPoint that is distributed to all departments at the end of each quarter.
     
  • Prepare content for weekly meetings with various departments.
     
  • Cross-check whether code that was developed can be applied to new experiments.
Level 14
by Optimizely
‎10-21-2015 October 21, 2015

@BryanClayton - As a former support rep, I LOVE your mantra. And more power to you for manning support at least once a week!

 

@robertchan - This is phenomenal. It sounds like you have great experience to contribute in the following places as well. What tool do you use to manage your testing backlog? - How do you prioritize tests? - Getting the organization to take part in generating test ideas (we'll be writing a blog with these comments, so we'd love to hear from you!)

Optimizely
by cpeterson
‎10-21-2015 October 21, 2015

Every week I do a full check on our analytics and make sure things are tracking properly and we're not tanking something important. 

 

This isn't a quick task ... it means I need to analyze data to make sure metrics are reporting back correctly. This is an important QA check. I've heard of a number of horror stories where something breaks a key metric (conversion events for instance) and the yearly report is messed up.

 

It also means I need to look at all of our individual analysis tools. Optimizely of course, but also google analytics, Hot Jar, adwords/ppc metrics, Hubspot metrics, etc...

 

 

I use all these tools individually  -- usually multiple times per week. However, the purpose of this check is different. It's to validate our reporting and course, rather than analyze the data which is what I usually do.

 

This helps me become a better growth manager b/c it ensures I know what we're tracking and what we're not. When we start new projects that involve special tracking or analysis, this make sure that it's implemented and implemented correctly.

Level 1
by Kevin
‎10-21-2015 October 21, 2015

The one thing I do without fail every single week is review the KPIs and any campaigns related to them.  What worked? What didn't work? What looks like it's getting better? What looks like it's loosing steam?  This has helped me become more focused on goals in everything I do in my day to day.

 

Also, the other thing I do without fail is learn! No matter what, I'm trying to take in new information or learn from past mistakes so I can continue to improve.

Level 1
by
‎10-21-2015 October 21, 2015

@Amanda thanks! I would love to follow up in those categories!

Level 14
by EH
‎10-21-2015 October 21, 2015

 

  • Tracking the data day to day to see if there are trends based on weekday, weekend, and night behavior.  The day traffic usually reflects business contacts while evening/weekend is hobbyists and students.  
  • Engaging with other departments for project updates, current test trends, and testing ideas.  There's a lot of aspects of our websites that I will never know all the intricacies. This interaction is very valuable to me and helps me make wise decisions when creating new ideas or analyzing data.
  • Checking the Optiverse Community and Knowledge base once a week for new ideas and increase my understanding of optimization fundamentals.  There's always something new to learn.

 

EH
Level 2
by maddy
‎10-23-2015 October 23, 2015

Each week on Friday we send out our "Testing Weekly" where we inform the management and all other stakeholder about the latest test results and started tests in that week. 

 

And of course I never fail to check my twitter stream for the newest information and updates from my network.

Level 2
by Optimizely
‎10-23-2015 October 23, 2015

Thanks @EH - glad to hear htat you are getting value from the Optiverse! 

 

@maddy - I would love for you to post a new discussion in the "Strategy and Culture" board that talks about a "Testing weekly" email, what content you include, and what the learnings have been. This would be very valuable to others.  For reference, @Pauline hosted a recent User Group in SF where Hotwire, Autodesk, and Atlassian shared similar strategies. You can check out the slides here

Optimizely
by
‎10-26-2015 October 26, 2015

Ok, my weekly habit seems to be different:

 

Every weekend i focus on the main actor in our work; users, real people, which are out there, in the real world.

 

I tend to observe them in the city, their habits, how they react to different influences: shop promos, people in the street, sounds....

 

From my point of view, growth is dependant on understanding people. To develop great testing ideas you need a lot of triggers to make your mind work, to be creative. Just by sitting at your desk in your office, you miss a great opportunity.

Level 9
by Optimizely
‎10-26-2015 October 26, 2015

@CouchPsycho - This is very interesting. Can you tell us a little more about how you use these learnings from the real world to apply to your online job? Is there something in particular you're looking out for? Was there one observation that made a big difference for one of your tests? Thanks for sharing! 

Optimizely
by MJBeisch Level 2
‎10-27-2015 October 27, 2015

Every week I make sure to collate all of my experiment data. Both for regular maintenance and organazation purposes, but also to do macro analysis. I have a big spreadsheet that I use to track experiment result data for every experiment I run. I track things like revenue generation, experiment location, theme, affected product vertical(s), etc. Working across multiple clients/sites gives me exposure to lots of different experiment result data. This helps me to prepare quarterly status/health reports on each client's testing programs, but also allows me the opportunity to easily look at all of that experiment data as a whole. There's a lot of experiment crossover, so I can see how successful a change is on a meta level, ie:

  • This has worked 7 out of 8 times tested
  • This only seems to work on sites with an upscale, classy look/product
  • This works on the product page 40% of the time, but works on the cart page 80% of the time

It really helps me to optimize my optimization. I can adjust each client's experiment prioritization list based on data collected across all clients. Definitely useful for me working as an individual, but also an important thing to consider for teams; make sure your team is cross-communicating.

Level 2
by lhendron
‎10-27-2015 October 27, 2015

Everyweek I have a 2 hour scheduled meeting with our design and technical lead. We start the meeting by reviewing how the previous weeks tests faired. We review conversion, revenue and each triggered event and come up with a plan for next week.
Then we review the trending analytics and come up with a new round of tests to put in the queue. This process has worked very nicely to help us develop a rhythm of constant testing and analyzing.

 

Level 1
by Shlomo
‎10-27-2015 October 27, 2015

Daily Work Flow

- Get Starbucks grande blonde roast (This is integral to CRO) 

- Check on running tests

- Segement results to find opportunities for personalization

- Identify underperforming pages, audiences, devices in GA

- Look at heatmap to identify specific problems 

- Develop hypothesis

- Run a user test to validate hypothesis 

- Start again 

 

Level 2
by Optimizely
‎10-27-2015 October 27, 2015

@Shlomo - Well said! Coffee is fuel : )

 

Thansk for sharing. @MJBeisch and @lhendron, I loved learning about your processes as well. 

 

Data and analytics seem to be one of the major themes of these responses -- great habits to have!

Optimizely
by
‎10-28-2015 October 28, 2015

To make a long story short:

 

I usually look at shop windows:

 

- What are they offering?

- What is the average price?

- How many products are there?

- What are there slogans?

- How long do people stay in front of it?

- Do they talk?

- How do they behave?

 

Then i try, it does not work always, to find analogies between real world and our website....

 

Try it out....it works!

Level 9
by glvzLIFT
‎10-30-2015 October 30, 2015
Another great discussion @Amanda. @BryanClayton's daily practice is a phenomenal way to stay grounded and in touch with customers. @MJBelsch's collation exercise is inspiring, I'd be interested in any tools & techniques he's found effective for collating.
 
A weekly practice I recently put in place is to update the daily visitor and baseline success rate in the segments I am targeting and the flows that I'm optimizing. There are two key benefits I've drawn from this practice.
  1. For the tests I have running, I get a clearer idea of where they are in terms of statistical significance
  2. For the tests in development, I can do a gut check of whether or not the variants we're considering could possibly generate the lift we'd need in order to see significance.

 

Level 2
by scleveland
‎10-30-2015 October 30, 2015

Our team is simply set up with a designer, developer and our supervisor who helps guide us and help with analytics.

 

We meet each week to go over any mockups that our designer has finished to turn over to the developer, we also go over the details of the test.  Hypothesis, Goals, Traffic, etc. 

 

We then go over analytics so we can see where to focus our efforts, then come up with test ideas and prioritize what to work on next.

 

We use Red Booth to keep track of where we keep track of the status of tests we are working on.  We also use a document to keep track of tests we want to do.

 

Since we started these weekly meetings, our testing culture has gotten a lot stronger and more focussed.

 
 
 
 
 
Level 2
by brushbill
‎11-12-2015 November 12, 2015 - edited ‎11-15-2015 November 15, 2015

Consistently challenge(test) yourself with something you've never done before.

Level 2