Rich-text Reply

Using Optimizely to track and report experiments, but not run them

ronl 08-25-16

Using Optimizely to track and report experiments, but not run them

Our goal is to run Split URL tests on our single page application that we will execute manually and use Optimizely only for tracking and reporting purposes.  Is this possible with the current API?

 

Here's an example:

 

Our control is: splittest.com/control and our variation is splittest.com/variationA.  We want to configure a test in Optimizely purely for custom goal tracking and reporting purposes only.  For example, upon a visitor viewing /control we will handle the decision on which variation the visitor will see, bucket them to that variation, and change state to that variation (either /control or /variationA) manually.  Once on the variation, we will want to track events to the Optimizely Experiment manually so we can use Optimizely's reports to evaluate performance.

 

Thanks,

Ron.

Level 1

CouchPsycho 08-26-16
 

Re: Using Optimizely to track and report experiments, but not run them

Hi,

if i got it right, the bucketing is done by any other application, a load balance for example?

If yes:

  • Set up an experiment which targets both urls
  • Create a custom dimension two seperate both groups
  • Create a variation with 100% traffic allocation
  • Within the variation, check which url is active and push its value to the custom dimension via API
  • Create your click-goals within the editor, possibly using the advanced input-field for selectors
  • Create you event goals
  • Be sure to fire those events in both variations via API
  • Use custom views in the results view to compare both segments
  • do some manual calculation afterwards ;-)

Hope this gives you an idea. This would be my approach, but there might be better ones :-)

 

Kind regards

 

Michael




"the essence of the creative act is to see the familiar as strange." (anonymous)
inigam 08-26-16
 

Re: Using Optimizely to track and report experiments, but not run them

Hey

 

I am assuming you already have these pages live

splittest.com/control and

splittest.com/variationA

 

and

a) Want to send visitors to these pages (or say experiences )using Optimizely but 'manually'.

b) Want to track Goals for each of two.

 

of what I understood by "we will handle the decision", I believe you would want to use manual or conditional activation.

 

Setup your Experiment

- You can create an Exp and use the 'Redirect to new page' in your 'variation tab' > set it to splittest.com/variationA. (your control will be Original leading users to /control)

 

- set the test Activation Mode as 'Manual', it will generate a code like this

 

window.optimizely.push(["activate", 69XXXX7XXX]);

 

Now you can activate the test when you want.

(if you don't use this, then test will run on page load of /control and optimizely will send users to /control or /variationA as per your traffic allocation)

 

- if you want, instead of Optimizely to randomly show users control/variationA, you can even choose which variation a user should see /control OR /variationA. use something like this ( check for any changes to this as I used it a while back)

 

window.optimizely.push(["bucketVisitor", selectedExperiment, selectedVariation]);

 

Goal Tracking

if I am reading your question correctly

- Simply set up your goals as you would. This test will give your Goals separated for /control (Original) and /VariationA (your variation)

- You can use dimensions to track whn users are on control and variationA, it usrely gives you to slice and dice your results data, but I think that may not be needed because any goal in will be reported by under its experiement variations and if say Goal A doesnt exist in control, it will be zero for control anyway.

 

PS - Consider using integration with your analytics. I would encourage to do so.

 

Hope that helps

 

Regards

IN

 

Regards,
IN


Level 6
ronl 08-26-16
 

Re: Using Optimizely to track and report experiments, but not run them

Hi Inigam, thank you for your response.

 

What you describe makes a lot of sense. For clarity, when I say "we will handle the decision" ourselves, we don't want Optimizely to handle the distribution of traffic at all (not even through conditional activation) as our application is a single page application.  It looks like the manual activation is the way to go for us. What is not yet fully clear to me is whether or not we have to associate events triggered on a variation specifically to that variation, and how we should do this.

 

I see in the API Function Calls there is a "bucketVisitor" method.  Am I correct to assume that execution sequence should be as follows?

 

1. Visitor visits on /control

2. Our internal code decides whether the visitor will see /control or /variationA

3. Once the visitor has been distributed and is shown /control or /varationA, call Optimizely to manually activate the test

4. Once the test is activated call bucketVisitor to associate the visitor to the appropriate variation in Optimizely

5. Use existing analytics integration (eg. Segment) to track events and goals

 

 

Will this result in us be able to report on the experiment in Optimizely effectively?

Level 1