- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Separating bots from humans
Separating bots from humans
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
How do I separate googlebots from human visitors for my tests? I want to make sure the original version shows to all traffic and the variations show to all except for googlebot (and possibly other spiders), this will make sure our seo efforts are not harmed with the tests. Or please let me know if the bots are excluded by default.
Re: Separating bots from humans
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
My first question to you, is whtat is in your Robots.txt file? That is the place to control what bots come to your site. If you can successfully block the robots - then your test will be clean. You should be able granually control to a file path, if you desire more specific bot management. More bot info can be found here: https://support.google.com/webmasters/answer/6062608?hl=en
baconipsum.com - because everyone need
a little bacon in their text!
Optimizely X Web Foundation and Technical Foundations Certified.
Re: Separating bots from humans
[ Edited ]- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Perhaps this has changed recently, (not my area of expertise, so do your own research to confirm this is still true), but as of a few years ago bots and spiders do not run javascript and would be automatically ignorant of any changes that your experiments may be making.
EDIT: yeah - it's changed. Google now executes javascript, follows recirdects, reads dynamic content, etc..
http://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157
Re: Separating bots from humans
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Re: Separating bots from humans
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Re: Separating bots from humans
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Google bots can be identified by their user agents as described on Google's article linked below:
https://support.google.com/webmasters/answer/1061943?hl=en
With this information, a custom javascript audience condition, which is available on our enterprise plans, can be used to exclude visitors with a certain user agent. Here is an example on how the custom javascript would appear:
navigator.userAgent.indexOf('Googlebot') < 0
It appears that you are using our free Starter plan which doesn't include the "Custom Javascript" audience condition. I recommend using the code provided above and wrap your variation code around an if statement to prevent any variation code to appear for Google's bots.
Here is a mock up of the code:
if (navigator.userAgent.indexOf('Googlebot') < 0) {
// insert your variation code
}
Here is a link to our knowledge base that will walk you through on how to access the variation code:
https://help.optimizely.com/hc/en-us/articles/200039835-The-Code-Editor-Edit-Code-and-Variation-Code