Rich-text Reply

Significance calculation for third-party metrics?

hv 02-04-16

Significance calculation for third-party metrics?

[ Edited ]

Hello!

 

I got some metrics from a third-party tool that are a relevant goal for a running test (number of app installs). Now these numbers are not available within Optimizely, so I have to do the math myself. For this I found the VWO Testing Significance calculator to be helpfull which gives me p-value and significance (yes/no). But actually I also would need the difference interval (like we have it on the Optimizely results page) - or to get those numbers somehow imported into Optimizely.

 

Any ideas? Experiences?

 

Harry

 

 

A/B Split Test Significance Calculator

https://vwo.com/ab-split-test-significance-calculator/

hv
Level 2

Re: Significance calculation for third-party metrics?

Hi Harry,

 

Thanks for your question! You can have a look at this page for tracking offline conversion events with Optimizely: https://help.optimizely.com/hc/en-us/articles/200040195-Tracking-offline-conversion-events-with-Opti...

 

A caveat is that you'll need to add .the user_id to offline conversions, otherwise Optimizely will count them as anonymous (new) visits. This might skew your current experiment data since it could add a lot of new visitors that haven't converted on any of the other goals. You can also add this data anonymously to a new experiment to make sure this data and your current experiment data don't cause duplicate visitors.

 

Does this answer your question?

 

Best,

Nils

 

hv 02-05-16
 

Re: Significance calculation for third-party metrics?

I hope it does. Thanks!

hv
Level 2
hv 02-05-16
 

Re: Significance calculation for third-party metrics?

Great that optimizely offers this possibility! Still I'm not completely sure how to proceed in my case here: If I add to the current experiment I have the problem with those new anonymous users. But if I add them to a new experiment I dont have the visitors count there - so I'll have a 100% conversion rate, right? Or can I count up the visits somehow to the numbers of the actual experiment?
hv
Level 2

Re: Significance calculation for third-party metrics?

Hi hv, what you could do here is add other visitors by sending as many extra conversion events you need to get there. You should be able to add another event name than that goal, so that it doesn't count as a conversion for the goal you're looking to measure. I hope that makes sense!

 

Best, Nils