Rich-text Reply

Bucket based on query parameter

andreasbalzer 05-16-16

Bucket based on query parameter

Hi,

My goal: Does an audience A behave different than an audience B and comparing them to each other.

 

I have several questions:

  1. Is it possible in results to compare segments to each other? Like get a table with Variation 1 for segment 1, V2 for S1, V1 for S2, V2 for S2, with like Improvement calculatation.
  2. Is it possible to bucket users into variations based on presence of a query parameter? One of my audiences comes from an external advertisement and has a parameter in the URL. Maybe I could force them into a variation, much like a redirect experiment.
  3. Would it be possible to take 100% of those users that have the parameter but lets say 10% of those who don't have it?

 

I am thinking of two experiments E1 and E2.

E1 targets 10% of all traffic and has two variations. V1 is the Original, V2 is the Redirect to the url having the parameter. V2 gets 0% of traffic in Traffic Allocation, because in my case these people come from an external advertisement source. E2 has only one variation V3. Both experiments share the same goals. I might be able to fetch both "audiences" in one result view. Right?

 

Would welcome any feedback and suggestions.

 

Thanks,

Andreas

Andreas
Mario 05-17-16
 

Re: Bucket based on query parameter

[ Edited ]

Hi there,

Thanks for reaching out to the community.

 

Regarding your questions:

1. You can segment your results for Segment1 (V1+V2) and Segment2 (V1+V2) and manually compare both results with each other.
2. By default, it is not possible to bucket a visitor based on the query parameter into a specific variation. Instead you could use a query parameter audience condition to include or exclude visitors from a specific experiment.
3. No, this is not possible (by default).

 

Regarding E1 and E2:
It is not possible to merge the results of E1 and E2 into one results page. Maybe you are able to figure out a way to set up the 3 variations within the same experiment, instead.

 

I hope this is helpful.

 

Best,
Mario

Technical Support Engineer
EMEA | Amsterdam
Optimizely
JasonDahlin 05-17-16
 

Re: Bucket based on query parameter

[ Edited ]

@andreasbalzer - Hi Andreas,

You have several questions wtih varying degrees of interpaly between them.  Let;s break it out into each separate entity.  Before you can examine things in combination, first we need to make sure we have all the parts.

 

Q1- Does an audience A behave different than an audience B and comparing them to each other

A1- Create an experiment that does nothing (only an "A" variation).  If the variation code, check to see if the user has a value in localStorage or in a certain cookie.  If no value exists, check the query string for the parameter you are looking for.  If the parameter exists, add them to a segment (aka dimension) indicating such and create the value in localStorage/cookie.  if the parameter does not exist, add them to a different segment indicating such and create the value in localStorage/cookie.  With this in place, users will only be assigned a segement on their first page hit.  From here, you can evaluate whether the two audiences behave differently or not.

 

Q2- Is it possible in results to compare segments to each other?

A2- Not side by side.  You select the segment/dimension and note which values are of interest, then select a different segment/dimension and make note of those values, then compare them (e.g., copy the results into Excel)

 

Q3- Is it possible to bucket users into variations based on presence of a query parameter?

A3- Yes, but... you really need to know what you are doing.  You could accomplish this by editing the optimizelyBuckets cookie.  However, I strongly suggest that there may be a different approach that gives you the same results.

Consider the answer to Q1.  You could create a 50-50 experiment where the code from Q1 is placed into both variations and also used to make changes to the page in the B variation.

 

Q4- Would it be possible to take 100% of those users that have the parameter but lets say 10% of those who don't have it

A4- Building off of #3 - instead of placing all 100% of the users that don't have the parameter into the "No Param" segment/dimension, you could randomly give 10% the value you are using in the B variation to alter the page.

 

 

Something like this:

/* -------------------------------------- */
/* this part goes in both the A_control and B_variation code block (or in experiment JS, if your plan supports it) */ 
/* evaluate the entry page for anyone without a dimension already set */
if (type localStorage.optDimension == 'undefined') {
	if (window.location.search.indexOf('myParam=something') != -1 ) {
		/* put 100% of these users into the "Something" segment */
		localStorage.optDimension = "Something";
	} else {
		/* put 10% of these users into "Not Something A" */
		/* put the rest into "Not Something B" */
		var rand = Math.floor(Math.random() * (10 - 1)) + 1;
		if (rand == 1) {
			localStorage.optDimension = "Not Something A";
		} else {
			localStorage.optDimension = "Not Something B";
		}
	}
        /* push the dimension to optimizely so you can read it in results */
optimizely.push(['addToSegment', 'myParam', localStorage.optDimension]); } /* -------------------------------------- */ /* this part goes only in the B_variation */ /* with localStorage.optDimension set, we can use it to modify the pages in the B variation */ if (localStorage.optDimension == "Something" || localStorage.optDimension == "Not Something A") { /* code goes here that modifies the page for the experiment */ }


1- Users will be randomly allocated to A_control and B_variation
2- Users in the A group will be placed into Dimensions, but otherwise not affected by the experiment
3- Users in the B group will be placed into Dimensions, and only users in the "Something" and "Not Something A" dimensions are affected by the experiment

You can use "Not Something B" as a control group to gauge "margin of error" for your common metrics
You can use "Not Something A" and "Something" to compare the experiment on each of the two segments.
If you wanted to add a 4th segment, one that combines "Not Something A" and "Something", you could do that too, but you couldn't use the same segment/dimension, you would need a different Dimension (a dimension can only hold one value at a time)

 

 

--Jason Dahlin
Analytics and Testing Guru Smiley Happy
Experimentation Hero