Experiment Goals don't match data in my CMS
I am using Optimizely to optimise an Ecomm store on Big Commerce.
I have a pageview goal set-up for the completed order page. This is to track "Completed Purchases" as a goal.
The data is coming through, however the information reported in Optimizely is different to the number of orders I'm seeing in the Big Commerce CMS.
Optimizely is reporting less orders than are actually coming through.
I'm tracking Pageviews on the "finishorder.php" page with a simple match.
Any ideas why there is a discrepancy?
Is it 1-2 orders of 100s?
My initial thought is that Optimizely is not bucketing all your visitors due to your targeting conditions so it wont track those sales.
Once we know how many orders are affected we can try and narrow down the issue.
From a look:
Sunday 20th: Optimizely tracked 0 conversions, CMS reported 4.
Saturday 19th: Optimizely tracked 4 conversions, CMS reported 8
Friday 18th: Optimizely tracked 3 conversions, CMS reported 4
Thursday 17th: Optimizely tracked 2 conversions, CMS reported 4
Wednesday 16th: Optimizely tracked 2 conversions, CMS reported 4
Admittedly these are small discrepancies, but still they are discrepancies and it's annoying. Could this be an issue with PayPal order completions and the way the user is passed back from PayPal to the confirmation page?
This is a multi page test, with home, product, category and checkout page mods.
The targeting for the category pages is using simple match targeting for various categories. It covers a large range of the top traffic categories, however not all of them.
If traffic is entering by some of these less popular pages then going on to convert I imagine it's never triggering the Optimizely test and subsequently not being counted for the purposes of the test.
"My initial thought is that Optimizely is not bucketing all your visitors due to your targeting conditions so it wont track those sales."
David's suggestion to compare the data in Google Analytics is a great one and should help you clarify if this is indeed the case. Should you notice anything unusual, of course feel free to come back here and discuss.
I've done a bit more digging into this, and it appears this is not the issue.
I am seeing however that only payments made via credit card are coming through into Optimizely. Payments processed via PayPal don't get recorded as a conversion.
I'll be trying to fix this today, so will follow up with my solution.
That's interesting. Is it possible that visitors going via Paypal are not returned to your site after confirming the payment, thus resulting in the log calls not being correctly fired to the Optimizely servers? Otherwise, might it be that the redirect URL reached by confirming an order via Paypal falls outside of your targeting conditions.
I'd be interested to hear what you found!
Don't they have to click a link to return to the merchants site in a lot of cases, or wait 10 secs to be redirected? It may be that they are closing the browser before heading to your success page.