Rich-text Reply

A/B Split test Postage

elburnso 06-08-16
Accepted Solution

A/B Split test Postage



I have been asked to setup an A/B split test on the rate of shipping. We have a magento enterprise website and I know how to change the shipping rates but I don't know how I would go about setting up this test. The shipping rates are taken from an excel spreadsheet uploaded into magento but how do I setup the test so that some customers are offered free shipping or a different postage price instead?


Is this possible to do through optimizely? Any help is greatly appreciated?





Website Manager
Level 4

CouchPsycho 06-09-16

Re: A/B Split test Postage

Based on the information you provided there is little hope. If you modify the rates in the frontend, your background-processes will not be informed and your customers will be charged the full-amount. If possible, you might use Universal Analytics integration with eCommerce fully used to get an idea, which bills must be modified... but there remains an email confirmation, i suppose.

Without your IT / developers you will not make you way through this problem...sorry to say that.
Kind regards


beckyhall 06-09-16

Re: A/B Split test Postage

Optimizely is a frontend solution, so your key challenge will be finding a way to change the real shipping rates (and not just the appearence of them) from the frontend.


The most simple version would probably be introducing a banner or some other form of advertisement with a free shipping (or reduced shipping) coupon code into your variations. Then in your admin, you are just setting up that coupon code to the discount you want. Obviously this is not the exact experience you are going for, but it could still provide insight into your user's shipping preferences.


A more complex version might be setting up those coupon codes in the admin, but instead of advertising them, automatically apply them to sessions in your variation. So if you want to offer free shipping to people in the B variation, you would have JavaScript on the cart page (or where ever your promotion code input is) that would prevent the default action on the "proceed to checkout" button, submit the promotion code form with the free shipping code, and then take the user to checkout. This experience might be closer to what you're looking for, but you'll definitely need to spend some time thinking through edge cases and may find that it won't work for you at all.


Also, I believe the shipping calculations for Magento are typically done in a php template, so you might be able to rig up a redirect test where B visitors are being served an entirely different template with the different shipping rules.


Keep in mind, though, that all of this is just brainstorming, and you will definitely need to discuss budget and approach with your team to find the best solution for you Smiley Happy

elburnso 06-09-16

Re: A/B Split test Postage

Thanks for the feedback. I thought that would be the case might try out the coupon code idea though didn't think.


Thank you


Website Manager
Level 4