Rich-text Reply

Exclude search engines from experiments

dharkness 05-06-16

Exclude search engines from experiments

Is it possible to exclude search engines from our experiments--not just the results but the actual experiment itself?


We are redesigning our site page-by-page and using Optimizely to make sure we see basic metric improvements before going live. We do this by redirecting to the same URL with a query parameter added that tells the backend to render the new layout. We don't want search engines to index both versions as if they are different pages, but we cannot use the canonical as it contains the same query parameter on the redesigned pages.


Why? Some of the new designs span multiple page types as a cohesive unit, and we want each user to remain in the same bucket across those pages without having to redirect on each page load. The code that generates all URLs copies the query parameter if found which means all redesigned pages have the query parameter in their canonical URL. Since one of the major reasons for the redesign is to improve SEO metrics like bounce rate, time on site and pages per session, it would suck to get dinged for duplicate content (only the layout is changing) at the same time!


Or am I attacking this problem the wrong way? Thanks in advance.

DavidS 05-09-16

Re: Exclude search engines from experiments

Hi dharkness,


Thanks for reaching out!

I am not aware of anything built within Optimizely that would allow you to exclude search engines bots from accessing your experiment. An alternative that I would see is to set up a redirect experiment (the new design would be the redirect) and then set the canonical for this URL. 


Another way may be to set up a Javascript condition that would check for the visitor's user agent and check if this user agents belongs to a search engine. If it does not, then you would active the experiment. You could use Manual Activation to activate the experiment when the visitor is not a user agent. 


I hope this is helpful. Feel more than free to reply back with any additional questions at all.