Rich-text Reply

A/B test: need help with choosing a winner

kimwie 10-04-14

A/B test: need help with choosing a winner

 

Can someone help me with the following dilemma?

 

 

I have performed a simple A/B test with 2 variants. Variant A shows a decrease in the main KPI (CTR (Clickouts to webshops) which directly impacts our bottom line) but an increase in customer satisfaction. Variant B has increased CTR but decreased customer satisfaction.

 

How would you proceed in choosing a 'winner' that will be deployed on our website? Thanks in advance!

 

Regards Kim

Level 1

MartijnSch 10-06-14
 

Re: A/B test: need help with choosing a winner

Hi Kim, are you able to share the numbers. I think that will make a more interesting case to see what influences what exactly so at least I can make a better assumption. For now I would say go with a decrease of click outs. But if the lifetime of the user won't be influenced but only their satisfaction then I would hint at more revenue.
adzeds 10-06-14
 

Re: A/B test: need help with choosing a winner

Could do with a little more data on this one.

Does customer satisfaction have a major affect on your performance?
David Shaw
Level 11