Rich-text Reply

Running 20+ Experiments // How Do You Display Results & Insights?

Mike_in_SF 05-07-15

Running 20+ Experiments // How Do You Display Results & Insights?

Hey there Optiverse,

 

We are running ~20 experiments simultaneously on one of our client's websites. Providing access to the 'results' tab for each experiment was sufficient for that client when we only have 2 or 3 experiments running, but now things are getting way too hard to track using that 'results' tab.

 

Additionally, a few of our experiments are MTV experiments on a single URL, making a bigger mess of the results analysis for the client.

 

In a perfect world, I'd love to build a dashboard that shows the following for every URL:

  • Overall page URL conversion rate, visits and conversions
  • Individual experiment results (CR, visits, conversions) for each page URL

Any suggestions on where to start? There don't seem to be any plug-and-play solutions (looked at dashboarding tools like Dashzen, Cyfe, etc) and it seems that my request is somewhat custom. A node.js developer suggested that I use node or ruby, load all the data from Optimizely's API into a SQL database and then just build out a series of queries to grab data and display it in the way I outlined above. 

 

Thoughts?

- Mike

Re: Running 20+ Experiments // How Do You Display Results & Insights?

Hi @Mike_in_SF,

 

You're on the right path - our Results API is a very powerful way to capture the essential pieces of the Results Page for a large set of experiments in one shot.  There are then a nearly limitless number of ways to leverage that information - one has been suggested via Google Sheets here in our Community.

 

Another option is our Executive Summary.  If you've got enough experiments running with enogh visitor volume, you can pull a spreadsheet on demand that collects all your experiment results on one tab, and highlights winning and losing goals on two others.  You can view the avaolability of that feature by going to your Home Page -> Settings Tab -> Labs subtab.

Harrison Krat
Solutions Architect | Optimizely, Inc.
harrison@optimizely.com
 

Re: Running 20+ Experiments // How Do You Display Results & Insights?

I had no idea that Executive Summary tab was there, that's awesome!

Re: Running 20+ Experiments // How Do You Display Results & Insights?

Hi Harrison, I like the Exec summary. What is "enough experiments and enough visitor volume" ?

Re: Running 20+ Experiments // How Do You Display Results & Insights?

Hi @JoniLindgren,

 

Grest question.  Here are the current criteria:

  1. The project for which you want to run the report must be one of the first 500 active projects for the account.
  2. If you have multiple projects on your account, any project with a name like UAT, Dev, Staging, or QA will not generate the report (you're unlikely to meet the below visitor volume thresholds here anyway).
  3. The number of started experiments (all time) is more than 5.
  4. The number of experiments with over 10,000 visitors is more than 1/5th of the total experiments.
  5. The number of experiments with under 1,000 visitors is less than 1/2 of the total experiments.

Please keep in mind the adoption of this report is growing steadily over time and we are closely watching usage patterns, so these criteria are subject to change at any time.

 

Once you generate experiment velocity and visitor volume that gets you in these ranges, please generate a report and take a look.  Then if you have any questions, you can reach out to our Support team here.

 

Thanks!

Harrison Krat
Solutions Architect | Optimizely, Inc.
harrison@optimizely.com