Optimizely has this Engagement goal set as a default goal in every experiment. It measures any click on any element. Other than this you have no idea of the industry standard bounce rate and they are quite different in nature. It is unthinkable for me to not measure the bounce rate as a primary KPI for a landing page which Optimizely should be designed for. Engagement is such a vague term. Getting accurate bounce rate during A/B tests especially if you use redirects to other domains or subdomains is such a difficult task and misleading.
Man, just give us the bounce rate.
One thing I feel is lacking (and some times frustrating) is the ability to look back at experiment variations.
After I have archived a completed experiement, I'll often what to reference it for ideas or learnings. The problem is that production code changes may of occured since the experiement was turned off, and the variation design no longer renders correctly.
It would be nice if Optimizely could take a snap shot of the variation just as a visual refernece to the experiment's variations.
For me, the most painful day-in-day-out part of using Optimizely right now is waiting for the iframe of my site to load before I can begin using the editor. Would it be possible to lazy load the experiment page and make the test configuration options and variation code immediately available for interaction?
IMHO, this would be a tremendous improvement. Thanks!
Add the option for audiences to be joined by an AND instead of the current ANY (OR) for targeting.
As default, ANY audience that matches will be included in the experiment. However, you may want to use multiple audiences to avoid having to create weird hybrids.
Example: A campaign targeted to mobile phones
Audience 1: All mobile devices
Audience 2: query param = your campaign
If you had an "AND", we could use these two audiences (thereby being able to always have a reusable mobile device audience). However, at present, we would have to create a 3rd audience that has both the mobile targeting AND the query param.
For experiment results to be readable, goal names need to be short. Unfortunately, it can be difficult to describe a goal accurately in just a few words.
For example, I am currently setting up a goal that tracks wiki edit submissions, but only on certain pages. The name "Edit submission" is misleading, because not all submissions count. A longer, more descriptive goal name would be better, except that this would make the results page difficult to read—long names cannot be easily skimmed and when a goal name is long, scrollbars appear and part of the name is hidden.
An optional goal description would allow an experiment creator to share additional detail without hurting readability.
I manage multiple separate Optimizely accounts for each of my CRO clients. Whenever I get a notification email (ie. a visitor account overage alert) there's no way for me to tell which Optimizely account the overage is for, based on the content of the email alone. Same thing with billing invoices. I can eventually figure it out if look into the account settings and line up the relevant data (visitor count, invoice date, etc.) but it would be really great if account names were placed into emails and on invoices.
Our team would love to have access to BlueKai's data universe, however, being an SME, we're unfortunately priced-out of their Data Activation System. Allowing Optimizely customers to opt-into BlueKai's segmentation, on a pay-as-you-go model - think MediaMath's Terminal one CPM pricing - would enable us to test & route advanced personalization on an as needed basis...
Currently the Revenue per Visitor is just a simple average value. It should be possible to remove outliers in this calculation to get more reliable results.
Currently we have to let run tests longer than they need, just to be sure about the RpV results.
The possibility of removing outliers would be simple feature with big leverage.
Hope you can do this soon!
It would be great to be able to see the conversion rate for multiple goals in the results page, with the ability to create goal "clusters".
A lot of our clients have more than one conversion on their site, e.g. sign up, contact us, and chat -- they all become leads.
- goal 1
- goal 2
- goal 3
- goal 4
- goal 5
- cluster goal 1 (combination of CR for goals 1-3)
This would save a lot of manual Excel work!
First, I have to say I love the tool and I have found only one issue. I have already sent feedback about this, but since it´s not fixed I’ll give more pressure and write my idea to all.
IP filtering is very frustrating, I mean not so easy and quick thing to do especially when we could have about 50-100 address to filter and 500 character limit is going to be also issue for us. Couldn’t you just make X amount of fields (like in Excel) and to those we could copy & paste our IP-addresses and forget those regular expressions (don’t forget, but as default I don’t want to use those). So like this:
Add IP to filter:
+ Click this and add new ip to filter.
Or easier and faster way would be just to import .csv file with all the ip addresses and Optimizely could automatically add those to IP filtering. And make it possible to add at least 100-200 IP-addresses to exclude.
It would be very helpful if it would be possible to share the results link with a custom starting/ending date.
It for example could record every changes made to the GUI in the URL. Therefore it would allow to share links and get people to see exactly what you are looking at.
For now whenever the link is shared, it allows only the standard date(when the experiment has been started).
It can be changed only after the link is loaded.
It looks like this:
It is not too good solution, because if we want to share it with people and our goal is to point out a specific period of time, we would have to tell them to do so manually.
This is an example of what I have in mind:
For example instead of 11th September, it would be 1st October.
For those of us who like dashboards, it would be great to be able to fetch our accounts' monthly users and monitor this along other metrics rather than keep going to the account details page.
It would help us plan or mitigate overusage.
A couple of improvements for ListTargeting:
1- value in a cookie (vs value of a cookie)
-- cookies are a valuable commodity - we cannot arbitrarily create new ones whenever we want, so instead I've started storing valuable information inside of a JSON formatted cookie. In the following example, I'd like List Targeting to be able to detect "ABC123" inside the cookie where the contents are:
2- use localStorage and sessionStorage
-- similar to the above... I'd like to look for a value inside of localStorage or sessionStorage
window.localStorage.eid = 'ABC123'
This would drastically improve the flexibility of using List Targeting.
In addition to the request of amitch5903 I miss also an option to sort variations on coloms like Visitors, Conversions, Conversion Rate, Improvement and Chance to Beat Baseline.
Especially by running MVT tests with a lot of variations you want to sort variations on highest to lowest Conversion Rate, but at least so important is to sort from lowest to highest for example. Another example is to sort on Chance to Beat Baseline to check which variations have the most chance to become winners, or even important losers.
In addition to better sorting the result page it would be great to have an option to exclude traffic for example the worst performing bottom top 10 MVT combinations in an experiment to get faster significant results for the top 10 winners.
Was wondering if anyone else has wanted to see the vertical view for tablet? Currently we are only able to see the horizontal view. My main concern was since the site we are working with has finicky responsive states, I wanted to ensure it worked properly in the vertical, narrower view for tablet.
It would be awesome to have a goal funnel within Optimizely for experiments with page view goals that tracks the visitor flow within the experimental conditions.
I am looking at my analytics program in which I have configured the experiments. But there is always the possibility of data skew as visitors from outside the experiment do end up on the goal page in the analytics software.
Would be nice to be able to do exclusion for URL targeting of goals. There's already exclusion on experiment URL targeting, so why not here as well?
Even though I find myself on the Results page almost every day, it would be great to have a "Notifications Center" where I could create automatic alerts when my tests pass certain thresholds. A bonus would be making text alerts an option as well.