- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page
Ask the Expert: Tom Fuertes, CTO of CROmetrics
Ask the Expert: Tom Fuertes, CTO of CROmetrics
[ Edited ]- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Respond to this post with questions you may have, bounce ideas around, and get to know the expert!
Hi Optiverse! I’m Tom Fuertes, CTO of CROmetrics, where I specialize in solving complex problems with elegant solutions, data-driven experimentation and analytics, and crafting new connections between client optimization and company success.
You can ask questions until noon CST on Friday, March 24 at which point this session will be closed. Check out the post here for guidelines and additional information.
- Tom Fuertes | CTO @ CROmetrics / LinkedIn
"Most Impactful Use of Personalization" and "Experience of the Year" Optie award winner.
Re: Ask the Expert: Tom Fuertes, CTO of CROmetrics
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
@tomfuertes has been an amazing Community member answering more than five questions in the last week alone!
Tom: many clients I want to work with want to know when they should run a full-factorial multivariate test (MVT) vs running back-to-back a/b/n tests. Do you have any best practices you follow when deciding which type of test to run?
Thanks!
Re: Ask the Expert: Tom Fuertes, CTO of CROmetrics
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
@jason-gsel: "many clients I want to work with want to know when they should run a full-factorial multivariate test (MVT) vs running back-to-back a/b/n tests. Do you have any best practices you follow when deciding which type of test to run?"
Great question! We prioritize based on a lot of factors. Our decision tree is roughly as follows, but the end result is that we run very few MVT's and almost always run A/B/n's.
- How long will 3 vs 4 variations take to reach significance? We use Optimizely's Sample Size Calculator to get a good feel for how long adding an additional variation will take to the length of the test. If you run an A/B/C (3 variations) test its usually much faster than a AB/AC MVT (4 variations).
- Do the variations have the same KPI? If so then an A/B/n make more sense so you can learn the most about how each variation affects the KPI.
- Do the likelihood of interactions heavily overlap? If not it might make sense to run then as an A/B/n instead of an MVT so you get more information on how each one affects the main KPI more quickly.
- Do you have a solid hypothesis backed by data? If not MVT's are a great way to learn directionally how elements on your site overlap. They're great for "exploratory testing".
Hope that helps!
- Tom Fuertes | CTO @ CROmetrics / LinkedIn
"Most Impactful Use of Personalization" and "Experience of the Year" Optie award winner.
Re: Ask the Expert: Tom Fuertes, CTO of CROmetrics
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Re: Ask the Expert: Tom Fuertes, CTO of CROmetrics
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
@mstrong - great question! The dataLayer definitely makes it easier. You'll need custom code and conditional activation to accomplish this. Something like below should work:
function (activate) { // populate an array with sku's to look for var skus = '12345,67890'.split(','); // poll for utag_data and activate if sku in cart (function poll() { var utag_data = window.utag_data; if (!utag_data) return setTimeout(poll, 50); var product_sku = utag_data.product_sku || []; var shouldActivate = false; product_sku.forEach(function (sku) { if (skus.indexOf(sku)>=0) { shouldActivate = true; } }); if (shouldActivate) activate(); })(); }
Here's the docs on conditional activation: https://help.optimizely.com/Build_Campaigns_and_Experiments/Activation_Mode%3A_Activating_an_experim...
- Tom Fuertes | CTO @ CROmetrics / LinkedIn
"Most Impactful Use of Personalization" and "Experience of the Year" Optie award winner.
Re: Ask the Expert: Tom Fuertes, CTO of CROmetrics
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Tom, thank you so much - that's working right out of the gate! Some people are going to flip around here when I show them!
Mike
Re: Ask the Expert: Tom Fuertes, CTO of CROmetrics
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report

- Tom Fuertes | CTO @ CROmetrics / LinkedIn
"Most Impactful Use of Personalization" and "Experience of the Year" Optie award winner.
Re: Ask the Expert: Tom Fuertes, CTO of CROmetrics
- Mark as New
- Bookmark
- Subscribe
- Subscribe to RSS Feed
- Permalink
- Email to a Friend
- Report
Thanks for doing this. One thing I've always wondered: Would you rather fight 100 duck-sized horses or 1 horse-sized duck?
Thanks,
Tom
Optimizing Sales/Growth for eCommerce Clients through Multivariate & A/B Testing, Personalization & Analytics @ Clearhead: The Digital Optimization Company
Connect with me on Linkedin

