Rich-text Reply

Testing in production with only one user

jencajenca 06-23-14

Testing in production with only one user

Hi!

 

I am trying to run a test (with one variation) in production for one user only (me!) but am failing miserably. This is my procedure:

 

1) Turn on 100% on variation in traffic allocation

2) Under targeting - I go to "match this IP address" and put in my ip (according to http://whatismyipaddress.com/)

3) Turn on experiment

 

It is not running for me.

 

I have also tried the cookie version, according to Optimizely's guidelines - same result.

 

Finally, I read that if I run the experiment with the query parameter "optimizely_x<number>=<variation_number>" set, that I should get the cookie set and next time not have to use the url parameter but this was not working for me either.

 

Any ideas how I should test this in a safe way before going public?

 

Thanks!

MeganBush 06-23-14
 

Re: Testing in production with only one user

Hello,

Its difficult to troubleshoot without all of the information, but lets try it anyways!

Did the variation show when you used the query parameter you specified?

When you go to a URL within the targeteding conditions, open your console, type this and press enter:
optimizely.activeExperiments

If this returns nothing, the experiment isn't running and I would double check your targeting conditions (both URL and IP conditions).

If this returns the appropriate Experiment ID, then type this and press enter:
optimizely.variationNamesMap

This will return what variation you're looped into. If it is the wrong one, clear your cookies and start again. If it is the right one and you're not seeing the changes, it could be something in the edit code isn't working properly.

Let me know what you find!
Megan Bush
jencajenca 06-23-14
 

Re: Testing in production with only one user

Thanks for the quick reply, Megan!

Ok, I started a new test that did nothing but a small css change. Thereafter I put the variation up to 100% again and put in the ip address like this: 12.34.567.890 and matched it exact in the match ip part.

I checked what optimizely contained via the console. No active tests on the page where the url hits! Although I can see that if I open allExperiments, the test project I just created is there (conditions: Object { type="url", values=[1]}, Object { type="ip", values=[1]}, Object { only_first_time=true, type="visitor", value="all"}, where the ip value array contains the following: Object { value="12.34.567.890", match="exact" and the url array contains the startpage).

What more is, I turned off the ip matching and redid the test - now it shows perfectly on the page. So I can conclude it is not a problem with the url nor the script.

I might add that the ip has been changed to a dummy text in both places, if it wasn't obvious Smiley Happy

Also, I am working from home when checking this so my IP address has not been excluded when the project was set up, if that matters (perhaps only matters for goals but still).

So, is there something odd going on with the ip check? It seems pretty straight forward when I read how other people has used it.

Any help is much appreciated!
Amanda 06-24-14
 

Re: Testing in production with only one user

Hey there! The test cookie version is definitely the best way to go about testing your experiment. @MeganBush 's advice for checking whether the experiment is actually running is spot on as well. 

 

It sounds like you are doing everything right for IP targeting, but I'd be happy to take a closer look at the implementation for you. Can you add the experiment ID to the post here?

 

 

Optimizely
jencajenca 06-24-14
 

Re: Testing in production with only one user

Hi!

I tried the cookie solution again, didn't get it to work a few weeks ago but this time it worked OK and thanks to Megan, it was easy to check that it was switched on.

Although Amanda, I am very curious as to how to improve the ip test, would be delighted if you could have a look at it - 1361258280 is the ID.

Best regards!
Amanda 06-25-14
 

Re: Testing in production with only one user

I took a look at your test and also set up an experiment of my own to try out the IP targeting. I was able to get it working just fine -- my suspicion is that there may just be a mismatch between your browser settings and what our geo.js file is picking up on. 

 

Here is how you can confirm that all the IPs are the same:

 

  1. Visit http://www.whatsmyip.org/ to get your IP
  2. Visit your experiment page and search the network traffic for geo.js event. See the attached. If you take a look at the request URL and enter it into the browser, you'll see a JSON that looks like optimizelyGeo = {   }; and the ip will be contained here. 
  3. There should be a match between the 2 items above and your experiment targeting settings to work. 

 

I recommend sticking with the test cookie, as it's definitely the easiest solution once you get the hang of it and create a bookmark for it. 

 

Let me know if you have any questions. 

 

Screen Shot 2014-06-24 at 6.34.08 PM.png

Optimizely

Re: Testing in production with only one user

[ Edited ]

Hej Megan - just to drop in and say thanks for that awesome tip with the developer console, great stuff.