In the experiments list, it would be nice to have a expansion button to display current percentages for variants of the experiment, and also change them.
Current behavior of:
clicking on experiment -> scroll down to variants -> change percentage -> scroll to bottom -> save
is way too much work.
In Optimizely Classic, it was very easy to click to see total vs unique data on each metric in the results page. Now with Optimizely X, you either have to create two metrics (one unique and one total), or go back in and edit a metric to change it to total in order to see the total data. This is time-consuming and frustrating vs. the easy way this was done in Classic. Our experiments are unlike the 'standard' ones which only have a few metrics. We have many that are based on increasing engagement on the page, and as such we need to measure how many different things are affected or changing. We have lots of metrics we are tracking. And for many of these, if they are click-based, we need to see the totals. It takes a lot of time now to go in and pull this data.
I think it would be very helpful if there was a way that we could measure time on page for each Variation. This would allow us to measure engagement in a different way and it would be awesome if it could be measured directly through optimizely.
An example, we have a better more informative product page on our site comparing it against our standard page. If i could set Time on page as a goal and measure it directly through optimizely that would save my needing to dig through GA or another analytics tool.
I think it would be really great if you can develop a significance calculator where we can input our numbers for example from Omniture and see if the results are significance.
So I can easy see if the order number uplift in Omniture are significance. At the moment I use another online tool (http://www.konversionskraft.de/tools) to get this numbers, but in my point of view I would say it make sense that Optimizely also provide such a tool.
I think it would be great if there were more hands-on events. I would love if I could go to San Francisco for a hack-a-thon style optimization event. Less time would be on speakers, more time on collaborating in teams.
It could be impleneted many different ways. Here's one way:
The competitors at the event could break off into different segments: lead gen, e-commerce, etc. In each segment everyone would be given the same website to "optimize". (The site that's picked could be a random raffle for each segment, every competitor gets to put 1 entry) People then get organized into teams of 3. They then have a set amount of time to setup 3 A/B tests on 3 different pages of the website. Each team presents their tests and pre-selected judges will give out different awards to different teams.
Most Likely to Succeed
Most Creative Technical Setup
At the end, the owner of the website for the experiment can get access to any of the experiments they want to run. As part of entry into the content, you have to agree to run at least 1 experiment and share the results. Once the results are significant, the reults are shared with Optimizely who then shares it with all paricipants.
We are planning to update our sites to AMP (Accelerated Mobile Pages )
we want to compare our Bootstrap version vs AMP version
and searching around I did not found anything about it.
Google MAP allows to run a/b test, but just between AMP pages, but does not allow a normal website vs AMP
It could a great idea if you figure out how to run that kind experiments.
Why don't you create templates marketplace or template gallery
such as Google Analytics ( https://analytics.google.com/analytics/gallery/#)
another good example is Codecanyon
this could be the categories or tags or whatever
- Mobile App Experiment
- E-commerce Experiment
- Lead Generation Experiment
So the user just need to import some settings from the gallery and has up and running an experiment
in minutes not hours, days or weeks
The current Saved Goal implementation can be painful when trying to add several saved goals at once. A simple, but not complete, solution would be to just make the lightboxes larger so less scrolling is necessary. The current implementation results in many cases where there are two vertical scrollbars within the frame, which is frustrating to deal with.
An easier implementation could involve a bulleted list of goals where we can just check off all of goals we want to add at once.
Sometimes in my job I have to use Adobe Target. Optimizely exceeds this tool in every category but there are a couple of features that Adobe offers that would be awesome to see Optimizely roll out (In a better way of course!).
Multi convert goals
Is there a way to allow (as an option) for goals to "always convert" rather than limiting one conversion/per visitor/per goal? We run into cases where it would be beneficial to have Optimizely allow a user to convert 2+ times on the same goal.
A scenario is that we have several forms with varying thank-you pages. We track visits to each individual thank-you page, but we also want to have a goal that is the sum of all these (ex. Visits to - All Thank-You Pages). This way we can have a quicker snapshot of a variation across the board. If a user comes to the site and ends up filling out two of our different forms we want to record a sum total of two, but currently Optimizely would always records this as one.
For example, Adobe Target has a dropdown feature that deafaults to "Count Once" but also allows the tool to force an "Always record conversion".
Another cool feature that I really like is the idea of segmenting goal data based off of another goal. For example, lets pretend there are two goals; the end goal is a "transaction conversion" and there is a micro-conversion, such as "Watched Marketing Video".
The metric I want to see is "Transaction Conversion" of users who converted on goal: "Watched Marketing Video". Then alternatively be able to compare that to a goal that says "Transaction Conversion" of users who did NOT convert on the goal "Watched Marketing Video".
Making it easier to tell if the video is either helping or hurting conversions. By chaining goals together it is helpful to assess the value of an interaction with different elements on a page.
I realize that goals like this can be set up by setting cookies and custom events but it would be sweet if Optimizely was able to create a feature to do it for us (Preferribly by filtering results). Something that would look kind of like this:
So we can sort the Macro Conversion by micro conversions. This would make Optimizely immensely more powerful of a tool using data that Optimizely should already have!
When we create custom dimension it should allow us to choose data type of it like string , integer, date etc.
If custom dimension is created using "date" or "Integer" data type then when we create audience using that dimesion as condition then in condition we should be able to select "Between" rather than only "has"
if it is going to possible then please make it possible for dimensions which are going to be created using REST API
Putting aside for a moment, the restrictiveness of having only 10 potential segments and having to define them before and experiment has started, it would be exceptionally helpfull to be able to combine multiple segments.
Visitors who have seen a certain page, using a particular browser.
Visitors who have coverted 1 or more specific goals 1 or more times.
As a fully developed concept, this feature would allow for inclusion or exclusion, along with compound AND/OR logic.
Obviously, a segment can be as complicated as we choose, but we are limited to 10 at any one time and most segmentation ideas occur during the analysis of an experiment, not always during the planning stage.
The ability to combine segments that are already created would expand the pool and give us the freedom to apply new segmentation as inspiration hits during the course of an experiment.
It would be awesome if we could have new/returning visitors as an audience segmentation that is created by default and automatically applied to all tests (similar to the different browser types).
These two groups can have wildly different behaviour, and it is not ideal to have to add multiple audiences for every test to capture these basic audience types.
Edit: Would also be really helpful if mobile/tablet/desktop were included in there as well by default!