Share your Idea
Sort by:
New Idea

Integrate Sample Size Calculater into tests

Status: Done
by Betabrand ‎07-24-2014 July 24, 2014

I just read your article on how long to run a test.

It would be great if you folks would integrate your Sample Size Calculator directly into tests.  Since the number of variations and daily visitors are known by the test, if the initial parameters are entered, the test results page should be able to give you a real-time estimate of how much longer you should run your tests.
 
Status: Done

With our new Stats Engine, we now provide a real-time estimate of how much longer you'll have to wait (in number of visitors) before your test calls a winner or loser, assuming the observed conversion rates were to hold. Read more about Stats Engine in our FAQ

Archive Content Retrieval

Status: New
by ‎03-03-2016 March 3, 2016

Having a lot of experiments in archive, it would be great to be able to access the editor for experiments without having to actually unarchive the experiments. 

Minor suggestion:

I often find myself wanting to rename experiments, so the naming can be consistent across experiments, when I see them together with others on the dashboard. It's a minor annoyance to have to slowly load to edit each one just to change the names to be consistent across experiments.

 

So,  a nice minor improvement, would be to add a 'rename' option on clicking on an experiment in the dashboard, so you can rename without going to edit the experiment.

Status: Done

Hypothesis Tab

Status: Done
by Julita ‎10-31-2014 October 31, 2014

Hi! Smiley Very Happy

 

I think it would be very nice if the Experiment Edit Page would include Hypothesis Tab.

Then the editor would be able to write the hypothesis or give a short description about the experiment.

This feature would help with giving a better idea about the experiment, especially if a few people are working on it.

 

It would be useful to grab it later from the API as well.

 

Attached are 2 pictures of how it could look like.

 

hypothesis2.png
hypothesis3.png
Status: Done

Make the report grafics downloadable

Status: New
by Sven ‎06-30-2015 June 30, 2015

Hi,

 

we use screenshots from the results page in our reports and it would save a lot of time if we could just download your realy nice looking grafics.

 

Kr,

Sven

In Optimizely Classic, it was very easy to click to see total vs unique data on each metric in the results page. Now with Optimizely X, you either have to create two metrics (one unique and one total), or go back in and edit a metric to change it to total in order to see the total data. This is time-consuming and frustrating vs. the easy way this was done in Classic. Our experiments are unlike the 'standard' ones which only have a few metrics. We have many that are based on increasing engagement on the page, and as such we need to measure how many different things are affected or changing. We have lots of metrics we are tracking. And for many of these, if they are click-based, we need to see the totals. It takes a lot of time now to go in and pull this data. Smiley Sad

Community Workshop Idea

Status: New
by IanHo ‎04-21-2015 April 21, 2015

I think it would be great if there were more hands-on events. I would love if I could go to San Francisco for a hack-a-thon style optimization event. Less time would be on speakers, more time on collaborating in teams.

 

It could be impleneted many different ways. Here's one way:

 

The competitors at the event could break off into different segments: lead gen, e-commerce, etc. In each segment everyone would be given the same website to "optimize". (The site that's picked could be a random raffle for each segment, every competitor gets to put 1 entry) People then get organized into teams of 3. They then have a set amount of time to setup 3 A/B tests on 3 different pages of the website. Each team presents their tests and pre-selected judges will give out different awards to different teams.

 

Possible awards:

Most Likely to Succeed

Best UX

Best UI

Most Creative Technical Setup

 

At the end, the owner of the website for the experiment can get access to any of the experiments they want to run. As part of entry into the content, you have to agree to run at least 1 experiment and share the results. Once the results are significant, the reults are shared with Optimizely who then shares it with all paricipants.

Status: New

Optimizely includes tablets within mobile devices, however if you check with google analytics you will see 3 types of devices which are desktop, tablet and mobile and there's areason for that. We can't target only mobile devices without doing extra work with optimizely and it's just a non-reliable workaround. This is a problem if you have a mobile first strategy like us. We start with the smallest screens, which are the smartphones and continue to grow to tablet and then finally desktop. How can I target these seperately. Currently we can't, but we should.

Status: Done

It would be awesome to have a visual representation to differentiate between whether you are targeting a positive vs. negative attribute. Maybe just have one tab be a different color, as well as along the left nav once the targeting is set.

Status: Done
We made it easier to see and toggle between positive and negative targeting conditions with our new audiences launch. I hope it helps! You can learn more here: http://blog.optimizely.com/2014/07/22/create-personalized-experiences-audiences/

QA Testing Ball

Status: New
by ‎08-30-2017 August 30, 2017

The Optimizely X - QA BALL -- frequently gets in the way when I resize my page.  Can this be made smaller? Get rid of the "Welcome to Optimizely QA" and intro copy.  (Maybe have a "hide this" check box).  

We end up doing an unecessary amount of custom events to get around timing of elements being in the dom and/or needing to track something other than clicks and pageviews. Building out the goals config further to something more generic that could config selector, action, bind time (domReady or window.load), and/or the option to delegate the selector so that you don't have to waitForElement poll to bind the handlers would be great.

 

// configurable in goals modal
var selector = $('#config-selector').val(); // '#form' var action = $('#config-action').val(); // 'submit' var ready = $('#config-ready').val(); // true
// psuedocode for binding
$(ready ? document : window)[ready ? 'ready' : 'load'](function () { $(selector)[action](function () { window.optimizely.push(['trackEvent', '{slug}']); }); }) // var delegateTo = $('#config-delegate'); // optional (string) // would be a bit more difficult than coding in the optiverse // code editor before coffee lends itself... I have faith though!

 

Status: Maybe One Day

Optimizely App

Status: Maybe One Day
by jerryaus ‎02-18-2015 February 18, 2015

Optimizely is a great tool that I like more and more every day.

 

However I use my mobile phone a lot for work. I use apps many times of day to get updates about data for our apps and websites.

 

However it's very difficult to access Optimizely on mobile. The website is not adaptive or responsive and for that reason I can only check my results or even work on some experiments on Desktop.

 

It would be great if the users could have an Optimizely app where we could at least check the results.

 

Cheers

Status: Maybe One Day

Add the option for audiences to be joined by an AND instead of the current ANY (OR) for targeting.

 

As default, ANY audience that matches will be included in the experiment.  However, you may want to use multiple audiences to avoid having to create weird hybrids.

 

Example:  A campaign targeted to mobile phones

 

Audience 1:  All mobile devices

Audience 2:  query param = your campaign

 

If you had an "AND", we could use these two audiences (thereby being able to always have a reusable mobile device audience).  However, at present, we would have to create a 3rd audience that has both the mobile targeting AND the query param.  

Status: Great Idea!

Folders on Dashboard

Status: Done
by salismat ‎07-03-2014 July 3, 2014

Please, please please can you put some folders on the dashboard? I don't want to have to implement a new code snippet every tme I want to put things in a folder. Really simple idea but would be fantastically useful. 

 

Once you have run over 30 tests you want to be able to quickly find the ones you are working on creating and the ones that have completed. Its a waste of time to scrolling up and down the page. This is particularly acute when you are trying to present your wizzy new tool to the Marketing Director and you can't find the things you need.

 

Thank you

Matt

Status: Done

Significance calculator

Status: New
by StefanF ‎07-10-2015 July 10, 2015

I think it would be really great if you can develop a significance calculator where we can input our numbers for example from Omniture and see if the results are significance.

 

So I can easy see if the order number uplift in Omniture are significance. At the moment I use another online tool (http://www.konversionskraft.de/tools) to get this numbers, but in my point of view I would say it make sense that Optimizely also provide such a tool.

 

 

Additional Options on "New" Results Page


The new results page is a great step-up visually from the old results page, but a few basic features are missing that would be nice to have (in priority order):

 

1. Adjust time displayed.  Similar to Google Analytics, it would be great if I can easily adjust the time period that shows up to correct for any implementation issues, site issues, etc.

2. Email the report.  I swear this was demoed at Opticon and is probably in the pipeline, but I can't find it anywhere. 

3. Comments: Before I send the results to leadership, I'd like to add comments at the top (key findings) as well as comments with each graph.
4. Option to Add Images from Test: To make this report even more compelling, it would be great to be able to add images of each variation tested.

Ability to Pause Experiment on Results Page

Status: Done
by Allie ‎04-18-2014 April 18, 2014
I would love to be able to pause an experiment from the results page instead of needing to hit "Edit Experiment" and waiting for the full page to load.
Status: Done

Statistical Test Choice & Power Analysis

Status: Done
by jweinstein Level 2 ‎04-16-2014 April 16, 2014 - edited ‎04-16-2014 April 16, 2014

Choosing Statistical Test: 

It would be helpful if there were different statistical tests to choose from when looking at the results. We generally use two-tail tests, so we take the numbers of visitors and conversions and run the results in R.

 

Power Analysis: 

This would  tell the user what kind of effect size he or she could expect to be able to observe given the traffic so far (or how many additional visitors will be needed to reach a certain confidence level).

Status: Done

With our new Stats Engine, Optimizely now uses sequential hypothesis testing with false discovery rate controls to calculate statistical significance. This means that the p-value that you see is a valid measure of statistical significance at all times that your experiment is running.

 

As for power, there is no longer a need to consciously set statistical power. Instead the specificity of your results depends only on how long you are willing to wait. The sequential test we implemented is a test of power one, which means it will detect a non-zero effect size always, if you wait for enough samples. Waiting longer on any test gives you more chance to detect a winner or loser, if one exists. Plus, instead of the effect size you can expect to detect with power given your traffic so far, we now give an in-product estimate of how much more traffic you need to call your test significant given the currently observed effect size stays the same.

Goal descriptions

Status: Great Idea!
by openjck ‎10-10-2014 October 10, 2014

For experiment results to be readable, goal names need to be short. Unfortunately, it can be difficult to describe a goal accurately in just a few words.

 

For example, I am currently setting up a goal that tracks wiki edit submissions, but only on certain pages. The name "Edit submission" is misleading, because not all submissions count. A longer, more descriptive goal name would be better, except that this would make the results page difficult to read—long names cannot be easily skimmed and when a goal name is long, scrollbars appear and part of the name is hidden.

 

An optional goal description would allow an experiment creator to share additional detail without hurting readability.

Status: Great Idea!
Other Ways the Optiverse Can Help
Knowledge Base
Your reference guide for support topics, FAQs, feature documentation, and optimization strategies.
Academy
Learn optimization essentials at your own pace and practice your skills through interactive, step-by-step courses.
Support
View your ticket history or file a support ticket with our team of helpful experts.
​Agencies and Optimizely Partners
Access the Optimizely Agency Portal, an exclusive site for agencies and partners to view Optimizely resources and register client leads.