Rich-text Reply

Experiment not showing to all the traffic

augusteu 07-05-16

Experiment not showing to all the traffic

Hi! 

 

I've started using Optimizely about a month ago and have already set up a couple of experiments. One problem that I have noticed across all the experiments is that they do not seem to be shown to all the site visitors even though I allocate traffic to 100% and split it 50-50 between two variations. 

 

The reason that makes me think that is because the number of visitors who see an experiment on a certain page is always lower than the traffic I see on GA on the same page/date range/audience. Let's say I run an experiment and Desktop only - the number of Desktop users that actually see an experiment is at least half of the total traffic that this page gets. I know Optimizely might count visits in a slightly different way, but it can't be that dramatic! 

 

Can it be something with URL trageting match? Does it make a difference if I use Simple match vs. Substring match if I only run an experiment on one page which might sometimes contain different query strings?

 

Since I am quite new to Optimizely, I'm sure I might be missing something! If anyone had similar problems or has an idea where to start troubleshooting, I would love to hear from you. 

Level 1

CouchPsycho 07-05-16
 

Re: Experiment not showing to all the traffic

Just a thought and please do not feel insulted in any way: Are you comparing the right metric in analytics, which means "Users" not pageviews or sessions?



"the essence of the creative act is to see the familiar as strange." (anonymous)
robertchan 07-05-16
 

Re: Experiment not showing to all the traffic

There will be a difference if you're using simple match since that would correspond your experiment to one specific link as opposed to using substring match which is what it sounds like you should be using. In addition, if the discrepancy in your GA report and Optimizely is extreme, then the metric you're using for your experiment is likely causing the issue was @CouchPsycho suggested. Finally, I would highly recommend reading about and launching the Optimizely log to see what exactly is causing your issue (please specifically read the "The Optimizely Log" section). Here's the link to the article: https://help.optimizely.com/QA_and_Troubleshoot/The_console,_data_object,_and_Optimizely_log
Robert Chan

Experimentation Hero