Rich-text Reply

Similar goals, different results

eranfiliba 06-27-14
Accepted Solution

Similar goals, different results

I've set up an A/B test trying different texts on 'Place the Order' button on the last part of the checkout page. This is the last button the visitor needs to click in order to complete her order. 

 

Two of the goals I'm following are 'the number of visitors who visit the checkout complete page' which is the page that shows an order is complete, and 'number of people who click the complete order button'. 

 

Although the sample size is not that big yet, the results on both those goals are very different. Variations show positive improvement on one goal, while the same variation set shows negative on the other goal. 

 

I've checked goal settings, they seem to be accurately set. Can anyone offer any insight on such an issue?

 

Thanks,

 

Eran

zemaniac 06-27-14
 

Re: Similar goals, different results

Is it possible for a customer to see the "checkout complete page" afterwards, like by clicking on a confirmation email or a link in the "order history page"?

I know that the results are de-duplicated, but maybe you have people getting to that page from past orders?
Alessio Romito
Project Manager at 21DIAMONDS GmbH
Level 2
eranfiliba 06-27-14
 

Re: Similar goals, different results

Maybe. Users can't go to the checkout complete page from another place in the site, one way for them to be able to see that page again would be to refresh it. But the conversion numbers are 3x different between the two goals.

Even if users refreshed the page, it should be at least 3 times, so that doesn't really make sense.
zemaniac 06-27-14
 

Re: Similar goals, different results

Refresh wouldn't duplicate the results... there must be somebody getting to that page without clicking the button...
Alessio Romito
Project Manager at 21DIAMONDS GmbH
Level 2
Josh 06-27-14
 

Re: Similar goals, different results

Can you clarify which goal is higher?

If the click is higher then maybe people are clicking the button before they've entered in all the required fields. This could register as a "click" even if the form wasn't valid.

If the visit numbers are higher I agree with Zemaniac; users must be getting there from some other source... This isn't a very likely candidate.. but depending on your implementation maybe users are pressing enter and it isn't registering as a click to the button.
Level 2
eranfiliba 06-28-14
 

Re: Similar goals, different results

The checkout complete page goal has a higher conversion. But there isn't any other way the users can see that page from somewhere else. And if they did, the difference shouldn't have been 3x. 

 

Our checkout complete page url structure is /checkout/complete/(order id) - so I used substring of /checkout/complete/ for targeting. Is that properly set? 

zemaniac 06-30-14
 

Re: Similar goals, different results

Sounds weird... I'm out of options here :\

 

any other /checkout/complete/ page that could go in the substring match?

Alessio Romito
Project Manager at 21DIAMONDS GmbH
Level 2
eranfiliba 06-30-14
 

Re: Similar goals, different results

Problem solved: There is another 'complete order' button that appears after the customer fills all the required information. That was the cause of discrepancy Smiley Happy