Tools to help me figure out what to test (qualaroo etc.)
I have seen tools like Qualaroo that can provide exit surveys and such...
1. Has anyone used such tools with success to help you figure out what you should be a/b testing?
2. are there any other tools out there that can help me learn what I should be testing?
I'm an ecommerce site.. So our goal is of course a higher conversion rate and more revenue.
1. A bit, it can provide you with valuable user feedback, although you really have to ask the right questions to the people on your site, otherwise it's going to be a bit useless. But in our case we tested it to validate the questions or suggestions users provided us with.
2. Have you tried using CrazyEgg? It provides you with heat map on where people click on pages. Based on that I was able to set up a couple of times A/B tests to change the way where people click on the page and where there focus is / you want it to be. Probably in your case closer to your CTA's to buy something.
2. I would certainly consider getting a heatmap tool as suggested by Martijnsch. If you have a tool like Qualaroo and a heatmap tool and Google Analytics you should be in a good position to analyse your visitors.
Development & Optimization Manager
It can be helpful for adding some more tests to your plan based on their responses though.
Usertesting.com and Crazyegg.com
With CrazyEgg you can get many heatmaps and figure out where users are clicking on each page. In addition you can segment and see what appeals to different demographics and use that information to target your changes to your highest revenue producing users.
With UserTesting.com you can have actual users run through your site with goals and see where they have issues or get hung up. If you are testing for someone else, like a client or you need proof for your boss, one amazing thing about user testing is you can run a few tests, say 5 - 10 then ID the biggest pain points. Use that to run some AB testing, and when the pain points are removed you can run a few more UserTesting walkthroughs and show with no data, through an easy to watch video - how much the user experience has improved with your test.
Director of Performance Marketing
@eamonhoolihan posted a very interesting article in the discussion here that highlights some tools to better understand your visitors. I thought it might be interesting given the conversation and intital conversation. Here's a link to the article: http://www.conversion-rate-experts.com/understanding-your-visitors/
@eamonhoolihan - can you provide some additional information on the specific tools you frequently use to come up with A/B testing ideas or an example of what you found out based on the "homework" you did pre-test?
I think all the tools mentioned in this thread have merits, with their strengths and weaknesses depending on your specific circumstances.
UserTesting.com is valuable for generating a breadth of testing ideas and, although it's qualitative feedback, in one recent project we discovered a handful of UX issues that almost all testers reported. Once these were brought to light, our next step was to further investigate or validate these findings with quantitative data e.g. analytics, heatmaps, form analytics or surveys.
Specifically, for the project in question, insights from UserTesting.com led to a complete change in the client's underlying business model, not merely a landing page or the user journey. So user testing had value in this case, which I'm not sure would have been the case with the quantitative tools.