Rich-text Reply

Tools to help me figure out what to test (qualaroo etc.)

dfoley 06-13-14

Tools to help me figure out what to test (qualaroo etc.)

I have seen tools like Qualaroo that can provide exit surveys and such... 


1. Has anyone used such tools with success to help you figure out what you should be a/b testing?


2. are there any other tools out there that can help me learn what I should be testing?


I'm an ecommerce site.. So our goal is of course a higher conversion rate and more revenue.

Level 2

MartijnSch 06-13-14

Re: Tools to help me figure out what to test (qualaroo etc.)


1. A bit, it can provide you with valuable user feedback, although you really have to ask the right questions to the people on your site, otherwise it's going to be a bit useless. But in our case we tested it to validate the questions or suggestions users provided us with.

2. Have you tried using CrazyEgg? It provides you with heat map on where people click on pages. Based on that I was able to set up a couple of times A/B tests to change the way where people click on the page and where there focus is / you want it to be. Probably in your case closer to your CTA's to buy something.
adzeds 06-13-14

Re: Tools to help me figure out what to test (qualaroo etc.)

1. Those tools are extremely powerful for helping to develop hypothesis for A/B testing. They allow you to establish what the visitors pain points are without having to guess using the data in Google Analytics. They can provide the basis for FAQ/Help sections as well as identifying problems in your websites logic that can be testing. I have written about their value on eConsultancy in the past:

2. I would certainly consider getting a heatmap tool as suggested by Martijnsch. If you have a tool like Qualaroo and a heatmap tool and Google Analytics you should be in a good position to analyse your visitors.
David Shaw
Level 11

Re: Tools to help me figure out what to test (qualaroo etc.)

I agree. Definitely get CrazyEgg on your site. Not only does this tool help you come up with test ideas, but with the Optimizely integration you can load heatmaps on your variations. This really comes in handy if your test changes a lot of design elements on the page.
Bryan Daniels
Development & Optimization Manager
Vertical Nerve
MeganBush 06-13-14

Re: Tools to help me figure out what to test (qualaroo etc.)

I've found the biggest pain points in the shopping funnel through User Testing. is super easy to use. You still have to come up with questions and buying scenarios, but the user provides feedback throughout the whole process. It makes it clear where issues in language or pain points in layout are so you can prioritize what areas of the site need help.
Megan Bush
MartijnSch 06-15-14

Re: Tools to help me figure out what to test (qualaroo etc.)

Hi Megan, I don't want to go completely off topic with this conversation/ question. But how many people do you ask these questions on I've tried a couple of people with the free test they did a while ago. I was impressed by the quality but in general I'm not a huge fan of doing such 'qualitative' tests as it could get your focus based on quantitive stats really off the hook as in the end it's 'just' somebodies opinion (although I see the value of course) and could not be reliable for your whole audience.
adzeds 06-16-14

Re: Tools to help me figure out what to test (qualaroo etc.)

I would use for usability analysis but would not rely too heavily on the feedback for determining user pain points/motivation drop off as they are not 'true' use cases.

It can be helpful for adding some more tests to your plan based on their responses though.
David Shaw
Level 11
SpencerPad 06-16-14

Re: Tools to help me figure out what to test (qualaroo etc.)

The other two tools I would recommend (and have been mentioned here): and

With CrazyEgg you can get many heatmaps and figure out where users are clicking on each page. In addition you can segment and see what appeals to different demographics and use that information to target your changes to your highest revenue producing users.

With you can have actual users run through your site with goals and see where they have issues or get hung up. If you are testing for someone else, like a client or you need proof for your boss, one amazing thing about user testing is you can run a few tests, say 5 - 10 then ID the biggest pain points. Use that to run some AB testing, and when the pain points are removed you can run a few more UserTesting walkthroughs and show with no data, through an easy to watch video - how much the user experience has improved with your test.
Spencer Padway
Director of Performance Marketing
Amanda 06-17-14

Re: Tools to help me figure out what to test (qualaroo etc.)

[ Edited ]

Hey everyone!


@eamonhoolihan posted a very interesting article in the discussion here that highlights some tools to better understand your visitors. I thought it might be interesting given the conversation and intital conversation.  Here's a link to the article:


@eamonhoolihan - can you provide some additional information on the specific tools you frequently use to come up with A/B testing ideas or an example of what you found out based on the "homework" you did pre-test? 


Re: Tools to help me figure out what to test (qualaroo etc.)

I think all the tools mentioned in this thread have merits, with their strengths and weaknesses depending on your specific circumstances. is valuable for generating a breadth of testing ideas and, although it's qualitative feedback, in one recent project we discovered a handful of UX issues that almost all testers reported. Once these were brought to light, our next step was to further investigate or validate these findings with quantitative data e.g. analytics, heatmaps, form analytics or surveys.


Specifically, for the project in question, insights from led to a complete change in the client's underlying business model, not merely a landing page or the user journey. So user testing had value in this case, which I'm not sure would have been the case with the quantitative tools.