For an ecommerce site, should the primary goal be revenue for every test?
The ultimate goal for an ecommerce site is, of course, to get more money (more orders, higher average order value).
However... When testing a page, such as the homepage, is the primary goal in an A/B still revenue?
Is it purely luck that one homepage would increase revenues over another homepage?
Or, is the primary goal engagement?
What are your A/B testing goals for ecommerce sites?
For an e-commerce website revenue is certainly a key metric and the primary site metric, however that does not mean it is the primary metric for every test.
Your hypothesis for each test should accurately define the primary metric for the test. So in a homepage test you might actually be looking to reduce bounce rate and increase engagement. But you should always track your primary site goal in every test!
Add To Basket Conversion Rate
Average Order Value
There is soo much to optimize on an e-commerce website!
Each experiment is unique, as is each site. You never really want to have a single goal for an experiment, rather a set of goals. Then, your goal analysis is weighted depending on what data point the goal is tracking and how it relates to the site as a whole. Everything is contextual and there's no "one size fits all" goal.I have a strong focus on e-com testing and I generally have 2 types of goals, broad funnel and specific action.
Broad funnel goals are goals that are generally applicable to every e-com experiment, things like:
- Product pageviews
- Add-to-cart rage
- Checkout pageviews
- Order conversion
These goals are re-usable from experiment to experiment. They are things that are almost always important to the overall health of an e-com site; something you would collate together and build a quarterly/yearly report out of. You want to make sure whatever you're testing doesn't negatively affect these goals, but they may not always be what you're directly trying to affect.
Specific action goals are data points that have a direct correlation to the changes that an experiment is making. These are usually the type of goal that you set up specifically for that experiment and don't end up re-using (or use infrequently). Let's take your example of a home page experiment. Maybe you're testing hero image content replacement on a slide-show carousel. Those images each link to a promoted category or product. The measure of success for that experiment would be clicks on those links; whichever variation gets more clicks is likely the most successful.
However, just because a user clicks that link doesn't mean they're going to follow through with purchase; depending on site scale, there many other decision points that they encounter before checkout, none of which the experiment was affecting. The experiment changes are probably too far removed from the point of transaction to say for certainty whether they directly affect order conversion or revenue. You would have to run an experiment like that for months on end and get hundreds of thousands (maybe millions) of unique visitors for those goals to reach confidence.
So in summary, goal analysis is prioritized based on distance from the point of change.