Rich-text Reply

In your testing program, what data sources could you not live without? Heatmapping, analytics etc.

leepearso 07-29-15

In your testing program, what data sources could you not live without? Heatmapping, analytics etc.

[ Edited ]

Interested to hear what data sources people are using to inform future testing and how much testing comes from this type of data instead of influence around the business or common sense?

 

What couldn't you live without analytics, heatmapping etc ? 

igerber 07-29-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

We use a combination of web analytics and heatmapping to inform most experiments. I'd especially highlight heatmapping as once we know what pages to test, knowing what to look at in particular is so helpful.

 

Heatmapping (and also scrollmapping) shows us where to focus CTAs, what sections are important and what aren't.

Level 2
leepearso 07-29-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

What programmes do you use for heatmapping or analytics?

Does anyone have experience of crazyegg?
igerber 07-29-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

Yes - we specifically use CrazyEgg for our heatmapping and Adobe Analytics for analytics
Level 2
cschroeder 07-29-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

Google Analytics: which can help me understand more deeply how each variation within a test is working.

 

Crazy Egg! (Heatmapping): Optimizely helps me understand which test variation won, and how the goals performed but by adding heatmapping I can more readily see WHY they won. Did the user interact with the part of the site I thought they would? Etc.

If you don't know, just test it!
ShaneHale 07-29-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

Hands down it is the Adobe Omniture integration. Makes for easy analysis for understanding the different variations and the rest of the visitors journey on the site.
MartijnSch 07-30-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

Google Analytics is an absolute must, we can't live without and send millions of events to it on a daily basis.
Heatmapping, we're using an open source/inhouse solution which saves every click on The Next Web so we can use it for segmentation but also have access to the raw data.
Case 07-30-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

Great open question!

 

We use CrazyEgg and GA for both analysis and insights, as well as an integration with our internal datawarehouse and Tableau, which allows us to tag users who saw an experiment and apply custom business events that Optimizely results and GA can't capture. For us this is very valuable. 

Level 2
jfx1026 07-31-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

We use quantitative data almost exclusively. I'd love to get more quantitative, but it's hard to make time for that piece. Google Analytics, Optimizely, crazy egg (heat mapping, scroll mapping) and sales data our our main sources.

The data does lead our testing ideas. Some of our ideas come from reactions to the business landscape–aka common sense or market intuition.

I don't think we could live without any of our tools at this point. We always need more data.
http://johnfreeborn.com
Level 3
Zsolt 07-31-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

Session recording has been a huge eye opener for us - especially from a user experience point of view, then it's very useful to see how people interact with your experiments.

A bunch of tools out there have this feature - I warmly recommend to everyone to give it a try.

Level 5

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

We use Hotjar a good bit for session recording, Verifyapp for preference and label tests, UserTesting.com for user tests, GA for engagement data.
adzeds 08-03-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

User session recordings provide visibility into your users that you cannot get with other forms of analytics...

Try it once and you will continue to use it forever!
David Shaw
Level 11
leepearso 08-03-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

Thanks for the input really interesting insight as to what other teams are using.

AdamA 08-03-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

In our own Optiverse testing program (KB, Academy, and Community), we use a combination of analytics, heatmapping, user testing, and qualitative feedback from our Qualaroo surveys.

 

User Testing and Qualaroo help paint a picture for us of where you may be struggling, or common confusion points, which we can then use analytics and heatmapping to investigate from a quantitative perspective.

 

Whenever we're able, we like to user test before and after to make sure that the human experience matches the data we saw, especially since on many of the Optiverse properties, there isn't a straightforward conversion funnel.

Optimizely

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

I can't live without Google Analytics. I use it for all my analytics tasks, including analysing the A/B tests. Our GA setup isn't the average setup anymore. We send of lot of extra data to GA so we can understand our users better.
jrichardson 08-06-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

We use a combination of analytics, user testing, and qualitative feedback from surveys.

 

We typically are using user testing before and after a test to uncover the "why" behind a result or determine our hypothesis. We also try to get real customer interviews as much as we can to vet out concepts and ideas with real customers. 

ishaan 08-07-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

SessionCam is really good
Level 2
George 08-07-15
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

Great question. Love seeing all the different tools people use. I'd like to add/second the following:

Mixpanel
Google Analytics
Tune
Accomplice.io
Validately
Mechanical Turk (both for qualitative surveys, interviews, etc. to sending traffic to tests)
CrazyEgg
VerifyApp

And of course, if you really want to get into it, custom datawarehouses using RedShift with Tableau/Looker. Using Optimizely's SDKs to get the experiment/variation data, exporting data from other sources (like Mixpanel) and importing it all into the data warehouse.
Level 2
Lucek 05-24-16
 

Re: In your testing program, what data sources could you not live without? Heatmapping, analytics et

I use an integration that we recently created at Survicate (in fact, I'm its first user : ). It allows me to target website surveys at certain variants of experiments so I know what people are saying about a variant. We have moderate traffic so it helps me get ideas what to improve and how people react to experiments way before I get enough qualitative data to see trends. If you're interested in trying it out you can find details here and get 2 months of free trial: https://survicate.com/blog/optimizely-integration/
Level 1