Rich-text Reply

event deduplication question

jhk 01-15-19

event deduplication question

Hi,

I have an experiment with the following variations:

 

V1: Show a call to action button on page 1. If you click it, a new site opens in a new tab and a conversion is counted by an event

 

V2: Show the same call to action button on page 1. If you click it, you get to page 2 on which there are 3 buttons. If you click one of them a new site opens and the same event is triggered and a conversion is counted.

 

So I would expect V1 to outperform V2 as the number of clicks to the conversion is less and the call to action for click one is the same.

 

Surprisingly it is vice versa. V2 shows a better performance in the test.

 

So I wonder if the reason is that in V2 the user may return later to the browser-tab of page 2 and click another button and by that two conversions are counted. The documentation states that events are deduplicated but I don't know how to verify that.

 

Jan

 

 

jhk
Level 2

JasonDahlin 01-15-19
 

Re: event deduplication question

[ Edited ]

Events are either "triggered" or "not triggered".
As soon as a user converts, they have converted.
If they convert again, they are still only "converted".
No matter how many times one user converts, they only count as one conversion.
If all of these clicks trigger the same event, whether they click on any or all only count as one conversion.

(Conversion rate is "the number of users that converted"/"the number of users in the experiment".
No consideration is given for how often a user converts.
(You could set up a Custom Attribute and in your own code count how often someone converts and push that conversion number to Optimizely so then you could look at how many of your users had 1 conversion, 2 conversions, 3 conversions, etc.)

 

Is V2P1 identical to V1P1?  You may want to check to make sure that your event is not recording on V2P1's CTA click.

--Jason Dahlin
Analytics and Testing Guru Smiley Happy


Experimentation Hero
jhk 01-16-19
 

Re: event deduplication question

Hi Jason,
the user experience on V2P1 is identical as on V1P1. The event is only tracked on V1. Otherwise the results would be exactly equal.
Jan
jhk
Level 2
JasonDahlin 01-16-19
 

Re: event deduplication question

Hi @jhk,

Thanks for confirming that P1 is identical across variations (not just identical CTAs, but the entire page).

Can you clarify how you are recording the conversion? Is it based on the new page loading and *not* the click of the CTA that leads it there?  If so, then is it possible that users are able to reach that page through other links on the site other than the page you are testing? (e.g., a users may be exposed to P1 in either variation but find a different link on the site that leads them to the page where the conversion happens)?

 

How significantly is V2 outperforming V1?

--Jason Dahlin
Analytics and Testing Guru Smiley Happy


Experimentation Hero