Rich-text Reply

Overlapping experiments goals

niklongstone 05-11-16

Overlapping experiments goals

HI, I've 2 experiments running on the same page.

 

The experiment one fire the event "myCustomEvent" on a button click.

The experiment two fire the event "myCustomEvent" in certain conditions.

 

So I have same goals name for 2 different experiments, but triggered in different conditions.

 

I hope that optimizely fires and tracks the custom events by experiment.

So, for example the goal of the custom event of the experiment one should not affect the experiment two.

 

To make it more clear, I suppose optimizely will listen for something like (eventName, experimentId) instead of only (eventName). 


Thanks

Re: Overlapping experiments goals

Hi there,

 

Thanks for posting your question here! You're definitely right in that Optimizely attributes conversions not just to a certain event but also for a certain experiment. Optimizely keeps track of which experiments/variations a visitor is in (in cookies, see here), and this information is sent to our servers. When a visitor converts for a goal, that conversion is only attributed to experiments/variations that that visitor has seen, even if that goal is also used in another experiment.

 

Does that answer your question?

 

Best wishes,

Nils

JasonDahlin 05-12-16
 

Re: Overlapping experiments goals

@niklongstone - 

 

Goals have nothing to do with an experiment

Consider: You have an experiment on your homepage.  You have a goal that tracks "Order Confirmation" page views.  The Order Confirmation page is not part of your experiment, but everyone who sees the Order Confirmation page counts as someone who met that goal.

 

Unless you set up a goal so that it tracks something that is only available in an experiment

Consider: Your experiment adds a button to the homepage and you want to track how often someone clicks on that button, so you set up a goal that is specific to that button.

 

You say you have two goals with the same name.  Do you mean that you have 2 ways of meeting the same goal or did you really create two different goals and give them the same name?  If so, how do you plan on knowing which is which?  Change one of them immediately.

 

The following statement is not correct

"To make it more clear, I suppose optimizely will listen for something like (eventName, experimentId) instead of only (eventName). "  A goal is a goal, it is independent of any experiment.  It is measured in terms of how many people who are in an experiment met the goal.  You do not have to be in an experiment to meet a goal.

 

 

--Jason Dahlin
Analytics and Testing Guru Smiley Happy
JasonDahlin 05-12-16
 

Re: Overlapping experiments goals

@Nils_van_Kleef posted while I was writing up my reply.

If you read our replies quickly, it will seem as if we are contradicting each other.

If you read them carefully, you will see that Nils explained how the reporting aspects work while I was speaking to how goals are met.

 

The difference being that in the example of you have only a homepage experiment and an Order Confirmation goal, you will have zero insight into how many times your Order Confirmation goal is met in total.  The Optimizely dashboard only shows users who are in an experiment - so if someone placed an order without having seen the homepage, they would not show up in the dashboard for the homepage experiment.  Similarly, if that same user sees the homepage *after* they have met the goal, their conversion will not show up because they had not seen the experiment at the time of the goal.

--Jason Dahlin
Analytics and Testing Guru Smiley Happy

Re: Overlapping experiments goals

@JasonDahlin I found the reply button a little bit faster than you Smiley Wink You're totally right! Thanks for writing this up. Experiments and goals are separate and they're only linked (visitor converting for a certain goal for a certain experiment/variation) when you link both together in Optimizely.

Nils

Re: Overlapping experiments goals

I saw the contradiction, I'll try to be more clear

Let's say:
I've an homepage with 2 experiments running on it: Experiment A and Experiment B.
I've one goal called "Order Confirmation" triggered with a custom event.

The Exp A, has the Order Confirmation goal active.
The Exp B, has the Order Confirmation goal active as well, but it's triggered only when the user performs some other actions before.

The user visit the Homepage, and he fits in both the experiments.

He triggers the "Order Confirmation" custom event as defined in the Exp A.

Will the Exp B report this event as well because is listening on same Goal ?

As you described in your reply, I think yes.


JasonDahlin 05-12-16
 

Re: Overlapping experiments goals

@niklongstone - Yes.  No matter how the goal is met, it is the same goal.  So a user who saw both experiments who triggered the goal in a way that is specific to Experiment A would also show a convesion for that goal in for Experiment B.

--Jason Dahlin
Analytics and Testing Guru Smiley Happy