Rich-text Reply

Finding Data Driven Ideas

Joe-Appleton 10-12-16

Finding Data Driven Ideas

I’ve been using Optimizely for a few months now and have run many experiments without achieving any results with statistical significance. My issue is that I am creating experiments using ideas off the top of my head that feel logical. I know that I need to be using actual data to find a problem that needs to be solved rather then running tests that feel logical. However, I am finding this particularly difficult to analyse the data as I only have classic Google Analytics in place with a limited budget so unable to explore third party data providers. To make it more difficult I am only using Optimizely on PPC landing pages, no further down the conversion funnel then the initial landing page. Could you help me in where exactly I should be looking in GA or any alternative approaches as I don’t have heat maps or user testing available to me? Thanks in advance.
PPC Marketing Manager at Enterprise Holdings
robertchan 10-12-16

Re: Finding Data Driven Ideas

I would recommend creating custom reports within GA. Instead of just approaching experiments with what feels logical, you could work with your historical data and correlate it with where you see conversions on your site struggling. For example, if you notice that people are leaving a certain page at a rate higher than the average of others, you could analyze time spent on pages in GA and determine what pages need testing. You should also make use of GA's custom dimensions. Such dimensions can help you segment users and build a path for them to convert accordingly. Optimizely offers a great way to integrate custom dimensions from GA with reporting so I think that that could really help you.
Robert Chan

Experimentation Hero
c5wales 10-12-16

Re: Finding Data Driven Ideas

Joe - It's important to first understand the 'macro' conversion you want your user to complete. During their experience they'll have to complete multiple 'micro' conversions along the way, whether it's clicking on a cta, adding an item to a shopping cart, or filling out a form. These micro conversion points on your site are where I'd initially focus.

If you set up a goal in GA and have your 'macro' conversion as goal completion, you can see where users are falling off in your funnel. You can focus on the pages, or 'micro' conversions where this drop-off is highest. I hope this helps.
Chad Wales
Product Development Manager
American Medical Association
Level 2
juliofarfan 10-12-16

Re: Finding Data Driven Ideas

What has worked for us the last 2 years is using PIE framework by Widerfunnel in order to priorize tests according to potential, importance & ease.
Just creat a spreadsheet first column is the test idea, then potential (1-10), then importance (1-10), then ease (1-10) sum the 3 variables and youll get a score, then sort ideas from top to low.

Test idea: Changing the "Make payment" button at checkout for a different layout with images that blink
Potential: Its potential is hide as its an element that triggers sales (10)
Importantce: Its importante because without it sales wont ocurr (10)
Ease: Its quite easy but images will need to be worked as well as the layout (7)
Total PIE Score: 27
maddy 10-13-16

Re: Finding Data Driven Ideas

Hi Joe,

great that you started optimizing and testing your optimization ideas!

No worries, Google Analytics (GA) is more then enough for getting relevant data that fuels your optimization efforts. Look at the landingpage you want to optimize and deep dive into the metrics. What is the "bounce rate" like, what is the "time on page" - what thesis could you come up with, why the users are bouncing or not spending much time on the page?
Even better, look holisticly at the hole funnel you want to optimize. Maybe it is not the landingpage that is the "problem" but a step further down in the funnel is causing problems - friction - for the user.
There are many walkthroughs on how do to a funnel analysis with GA on youtube and google. You'll find them Smiley Happy

Have fun digging in your data and finding the "sweet spot" where you next experiment has not only the chance of becoming significant (test bold changes!) but at the same time influence the user in a positive way so you get more conversions.

Level 2

Re: Finding Data Driven Ideas

Thanks everyone for your help on this, very useful!

I will spend much more time in GA digging further into the data, I will also review the funnel and understand the users journey beyond just the landing page.

Thanks again guys - Joe
PPC Marketing Manager at Enterprise Holdings
turker 10-13-16

Re: Finding Data Driven Ideas

Within a data-driven perspective we're using user feedbacks as qualitative and for sure GA for quantitative data.
Level 2
Christof 10-13-16

Re: Finding Data Driven Ideas

We have remote usability tests conducted by an external partner.
Level 2
Zsolt 10-13-16

Re: Finding Data Driven Ideas

[ Edited ]

I'd invest the $29 into HotJar - it's totally worth it. Heatmaps and average fold is useful but the recorded user sessions is a goldmine of opportunities. The learnings are invaluable when you can see how people interact with your site/product/design.

P.S. - I'm not affiliated with HotJar.

Level 5
aweitsman 10-13-16

Re: Finding Data Driven Ideas

We had a similar problem not too long ago, as the tagging for all of the pages on my company's site was inconsistent, with different tags and structures set up over the course of several years. So what we did (until our tagging was more uniform) was look at things from a more holistic level.

The best way to start an experiment is to determine your overall goals and KPIs before you even consider the "how" of your experiment - it's really the "what" for your users that you need to understand. If the only area that you can control is the landing pages, then what are you trying to point them towards? Are there pages on your site (that you can track with GA) that seem to be getting more or less love from visitors? Is the messaging on your landing pages consistent with the rest of your site? When you look at the stats from the rest of your pages, that might help you get a better idea of how to tweak the landing pages with Optimizely tests.

Also, depending on the scale of your business, there are some heatmapping products that you can implement for free to get at least a sample size of visitors. If you can apply the code across your whole site, that might help you get even more insight into how everything fits together.

Lastly, do you have tracking codes on the links in your experiments? Those can feed back into GA so you can get more insight into the results of your tests, which can give even more data to help you find what works. Good luck!
MartijnSch 10-14-16

Re: Finding Data Driven Ideas

Enrich your Google Analytics data, define the micro conversions on your page so you make sure what kind of actions people perform before they fill in a form or do what you would like your users to do. By doing this you know what leads up to a conversion and then you can also test these interactions as they're likely to have a bigger impact on the tests that you run.

Besides that there are many ways to test on users what they actually think about the pages and what kind of things they find annoying when the land on them. So coming up with good ideas/ hypothesis based on their behaviour might also give you good insights.

In our case we usually only have 5-10% max of our tests being inconclusive. So you saying that most of yours don't reach it is a big issue if you go further this route.

Re: Finding Data Driven Ideas

I absolutely second that. We signed up for HotJar a few months back and the insights are absolutely priceless. I wish there was an integration between the 2 platforms though.


ps. also not affiliated with HotJar

Performance Marketing Manager @ Blacklane
Founder @ Stereofox