Allow For Full Raw Data Export
Currently, when we want to match 1-to-1 between Optimizely test results and our database to do additional analysis, we have to request the raw data file from our Optimizely Team. It would be great if I could export a .csv of the full visitor-level results and avoid this extra step in the future!
It would be great to be able to post notes regarding certain Goals or dates within the 'View Results' section. Currently, we are at the mercy of the Goal Name and other Optimizely terms. Being able to input notes/details about certain goals will help other teammembers understand what each change is affecting without having to figure it out in QA.
Also, the ability to add annotations as certain events may want to be noted within the results. (i.e. - release went out, reasons for traffic shfts, etc)
It would be great to have an option to make experiments mutually exclusive without writing complex JS. A simple option within targeting would be helpful.
For example, if a user is entered into experiment 1, then don't allow the user into experiment 2. Or, if a user is included in any live experiments, then don't allow the user in the current experiment.
It would be great to be able to pick OR conditions between different types of targeting, for example, "visitors who come to this URL OR have this cookie". It would also be good to allow for AND conditions within a category, for example, "visitors who have the query parameter utm_source AND utm_campaign in the URL."
It's frustrating to have to use a new email address for every Optimizely account you get added to. Ideally I would like to use my own work email address across every account. So when I log in with that email I see a list of the accounts I've been added to in a splash screen and can then dive into each of these accounts. Google Analytics has a similar screen on log in.
There are likely to be a number of use cases for been a user on multiple accounts:
- agency with multiple clients each with unique account
- multinational with different accounts/business units per market but centralised testing team
- strategy or development freelancer who does ad-hoc consulting on different accounts
It would be nice if the colors assigned to variations were consistent between goals on the results page. Here are some screenshots of 2 different goals for an experiment, demonstrating the current inconsistency:
On the first goal, the "Original" variation is orange, but on the second goal it is blue. These shots were taken after initial page load, no configuration changes or any other page manipulation was performed.
I could have sworn that these colors were consistent, but I had a client point this out to me today. Went back and looked at the result pages of several other experiments and sure enough, the colors change from goal to goal. I guess I never noticed it because I'm usually looking at the numbers and the order of the variations is consistent, but this does appear to cause some confusion when doing "at-a-glance" monitoring or if using screenshots of a goal graphs to assemble a report.
I just read your article on how long to run a test.
I often find myself wanting to rename experiments, so the naming can be consistent across experiments, when I see them together with others on the dashboard. It's a minor annoyance to have to slowly load to edit each one just to change the names to be consistent across experiments.
So, a nice minor improvement, would be to add a 'rename' option on clicking on an experiment in the dashboard, so you can rename without going to edit the experiment.
I think it would be very nice if the Experiment Edit Page would include Hypothesis Tab.
Then the editor would be able to write the hypothesis or give a short description about the experiment.
This feature would help with giving a better idea about the experiment, especially if a few people are working on it.
It would be useful to grab it later from the API as well.
Attached are 2 pictures of how it could look like.
Optimizely includes tablets within mobile devices, however if you check with google analytics you will see 3 types of devices which are desktop, tablet and mobile and there's areason for that. We can't target only mobile devices without doing extra work with optimizely and it's just a non-reliable workaround. This is a problem if you have a mobile first strategy like us. We start with the smallest screens, which are the smartphones and continue to grow to tablet and then finally desktop. How can I target these seperately. Currently we can't, but we should.
It would be awesome to have a visual representation to differentiate between whether you are targeting a positive vs. negative attribute. Maybe just have one tab be a different color, as well as along the left nav once the targeting is set.
Please, please please can you put some folders on the dashboard? I don't want to have to implement a new code snippet every tme I want to put things in a folder. Really simple idea but would be fantastically useful.
Once you have run over 30 tests you want to be able to quickly find the ones you are working on creating and the ones that have completed. Its a waste of time to scrolling up and down the page. This is particularly acute when you are trying to present your wizzy new tool to the Marketing Director and you can't find the things you need.
Choosing Statistical Test:
It would be helpful if there were different statistical tests to choose from when looking at the results. We generally use two-tail tests, so we take the numbers of visitors and conversions and run the results in R.
This would tell the user what kind of effect size he or she could expect to be able to observe given the traffic so far (or how many additional visitors will be needed to reach a certain confidence level).
Need the ability to archive goals that are outdated or that shouldn't be used for one reason or another. Deleting the goals would be great, except that compromises any past test results that utilized the deleted goal. As it is, I've got a bunch of garbage goals in my goal list that just muddy the water when I need to pick/reuse goals for tests.
Before the Audiences release, you were able to target based on new vs returning visitors. I know the previous implementation was not ideal, but I would really love to see this general functionality come back in a more robust way.
It would be nice if the Global JS/CSS sections of the Optimizely editor were code syntax highlighted and supported code editing functions just like the Variation Code box does!