Rich-text Reply

A/B test an entire website?

csam00031 10-18-16
Accepted Solution

A/B test an entire website?

How do you suggest and do you think it's ok (from an SEO standpoint) to A/B test an ENTIRE website?

Has this been done before?
mminshew 10-18-16

Re: A/B test an entire website?

[ Edited ]

Just to clarify? Are you thinking of testing two totally different websites against each other?Unless the sites are EXTREMELY simple i would say that it would be really difficult to properly measure the total impact of the test. You may find some baseline metrics that look better (i.e. revenue) but it will be hard to test what exactly about the site made the difference. It could be that the simple change increases your metrics for a short time but long term your customer base might not like the change or maybe it was only the cta that made a difference but since your testing such a massive point of data how could you tell. I think it would be extremely difficult to tell if measuring a whole site side by side exactly what caused a win/loss.

I would strongly suggest breaking the test into several large scale multi variable tests.Pick the things you think up front that could have the greatest impact, I.e try a redirect test for the homepage or try a different cart style. Then see which one wins and try to break down what exactly caused the win. Then try the next part of your funnel. That way you see exactly what changes won and why and you have real solid data for each individual step, Maybe that fancy CTA was all the difference and you saved yourself a ton of time by just testing that single piece. Just my suggestions.


(EDIT) I will say kudo's for the idea though not sure if it would get you were you wanna go but man thats thinking outside the box!

Michael Minshew,
Learning something new every day.

Level 4
JasonDahlin 10-19-16

Re: A/B test an entire website?

If you are making global changes across the entire site, such as replacing the entire top-navigation with a new design, or similar, this is absolutely possible and is not at all uncommon and Google has stated that they do a good job of recognizing AB Tests and will not penalize you for the efforts.


If you are talking about running two entirely different versions of the website, you have many considerations on how to redirect users to the correct page on the new site.  Do the URLs match between the sites but share one singular difference (such as a subdomain?)  if so, you can do a redirect from old to new.  If the URLs don't match, youi'll need to consider *who* you want to redirect over to the new site.  Do you want 50% of all traffic coming to your old site to go to the homepage of the new site?  What if someone lands on a product page?  Should they be sent to the new site's homepage?  If your redirect is set up only on your homepage, you probably don't want to send 50% of all homepage traffic, you'll want to send only 50% of people who enter your site on the homepage (it would probably be bad to redirect someone who entered the old site on a marketing landing page and has been on your old site for 30 minutes who happens to finally click a link to your homepage).

--Jason Dahlin
Analytics and Testing Guru Smiley Happy
robertchan 10-19-16

Re: A/B test an entire website?

From a SEO standpoint, you'll be fine as Jason Dahlin mentioned, Google supports testing. In terms of testing an entire website, that will be dependent on your conversion metric and how you're attempting to gain insight. For example, if you're seeing whether an entirely new layout would contribute to a greater number of subscriptions then you could track the CTR on subscription buttons on the new layout versus the old. The issue with this is that it will be difficult to pinpoint what exactly's causing the subscriptions to increase or decrease on a local level.
Robert Chan