Rich-text Reply

Optimizely Performance Impact

dbose 06-02-16

Optimizely Performance Impact

My employer (based out of APAC region) is a big user of Optimizely. It's great product to quickly learn about our Audience. Thank you !

 

However, recently we have observed some performance impact around Optimizely loader.

cdn.optimizely.com is taking a long time to load (https://drive.google.com/file/d/0Bxc-qJFEX0VGTFFNalN2a1BxNVE/view?usp=sharing) over a network speed of 1.5 Mbps, while blocking at the pre-render phase.

 

Interestingly we are not running any experiment targeting the current URL. This is unacceptable and at the least, we need to detect quickly (may be via REST API  and caching the result in Redis/Memcaced ?) whether we should download the master script depending on any experiment configured to target the current URL.

 

What's the solution designer's best practice for this ?

 

 

Level 1

AllisonR 06-03-16
 

Re: Optimizely Performance Impact

I definitely understand the concern you may have about page load time. Optimizely does everything it can to ensure its effects are minimal if at all detectable. We use 2 CDN Providers, Akamai and Edgecast, to serve the Optimizely snippet to users based off of proximity and efficiency. More information about how the snippet is delivered to your site is hereHowever there are a few measures you can take to ensure your site is never slowed down by Optimizely:

 

Load Optimizely asynchronously

 

By default Optimizely uses a synchronous snippet to prevent flickering and has invested heavily in a balanced CDN system to make sure the impact on the page is minimized. However you can also load the snippet asynchronously to allow all the elements on your page to load simultaneously instead of waiting for Optimizely to finish loading.

A major drawback to this is that it can lead to flashing on the page (where a visitor might see the original content before the variation loads) which is why this is not recommended for optimal performance. More information on the pros and cons of asynchronous loading is here.

 

Self-host the snippet

 

If you prefer not to rely on our CDN for delivery of your Optimizely snippet, you also can self-host this on your own domain. The drawback to this is that you would need to download the snippet from Optimizely and update the self-hosted snippet file every time you change the settings for your project or make changes to an experiment.

You can take a look at our CDN uptime history here:http://status.optimizely.com/499047/history for an idea of what to expect with a default implementation.

 

Decrease your snippet size

 

Another suggestion is to decrease the size of the Optimizely snippet. The snippet includes the logic for all of your active, paused, and draft experiments. So, the more active, paused, and draft experiments you have in a specific project, the bigger your snippet size will be which can lead to increased loading times. A best practice is to keep your snippet size below 60KB. There are a number of ways to go about this that do not affect running experiments:

  • Archive paused/draft experiments that will no longer run.When you archive an experiment, the experiment logic is no longer included in the snippet which reduces the size as a result. More information on how to do this is here.

  • Adjust the jQuery settings of the Optimizely snippet. if you are already loading jQuery on your site, you can disable jQuery from loading in the snippet and this will reduce the snippet size pretty significantly. This can be done in the project settings.

  • Exclude paused and draft experiments from the snippet - the force variation query parameter QA tool will not work on these experiments if this setting is enabled

  • When you're thinking about running multiple experiments:

    1. Keep the amount of jQuery in the Edit Code box as low as possible. If you're planning to test extensive changes, you may want to use a redirect test instead of making the changes in the Editor.
    2. Consider whether you can use substring targeting, multivariate tests, or multi-page tests instead of running multiple experiments at one time.

 

Hope this helps!

 

Best,

Allison
Sr. Technical Support Engineer
Optimizely | EMEA

Allison
Sr. Technical Support Engineer
Optimizely | EMEA
Optimizely
dbose 06-23-16
 

Re: Optimizely Performance Impact

[ Edited ]

Thanks Allison

 

On deeper investigation it looks like Personalization is taking an heavy troll on A/B Testing. Is there a way to decouple Personalzation from Testing at script level ?

 

This is a scenario/implementation where a media giant's layouts (CMS etc.) are shared across several assets, unfairly penalizing everybody even though Personalization stuff targets a small subset (subscription flow) of URLs. 

Level 1