Rich-text Reply

Optimizely X unacceptable render blocking evaluation time

puglisij 06-14-17

Optimizely X unacceptable render blocking evaluation time

We're looking to move to Optimizely X in hopes of improving the load time on our site. 

In our experience, Optimizely demonstrates an unacceptable level of performance during the critical rendering path, where optimizely must be loaded synchronously to avoid the dreadful FOUC. 

 

During our test, we created 3 small experiments in X.  One was running on the homepage, another was paused, and another not started.  This results in a snippet which is about 230kb minified and around 70kb gzipped. Quite large as is.   In our runs we noticed this adds an average of 500ms of render blocking time before the first paint (which we confirmed by removing optimizely from the page) on pages where there aren't even any experiments running.  As you know, this can have a major impact on revenue generating websites. 

 

This is Not an improvement over Classic.  While we like some of the features X advertises (e.g.  asynchronously loaded experiments)  we were really hoping for better performance when our pages are initially loading.  

 

What could possibly be running during this phase, when there are no experiments that would necessitate this excessive load time? We find this completely unacceptable. 

Level 2

JasonDahlin 06-14-17
 

Re: Optimizely X unacceptable render blocking evaluation time

What does the timing inside the request look like?
(DNS lookup, Initial Connection, SSL, Waiting, Downloading, etc.)
Hitting a few different sites that are using Optimizely X that I know have only 1 or 2 experiments set up, loading the un-cached snippet 10 times each:
1- snippets were all between 75 and 100 KB
2- DNS Lookup: avg 80ms
3- Initial Connection: 25ms
4- Waiting: 75ms
5- Content Download: 120-350ms

Total load time was 300-550 ms.
The longest section of the timing is the download speed.

You could try testing to see if running a <link rel="dns-prefetch" href="//www.optimizely.com"> would help, but if your snippet is at the top, it probably won't.
--Jason Dahlin
Analytics and Testing Guru Smiley Happy


Experimentation Hero
RyanC 06-15-17
 

Re: Optimizely X unacceptable render blocking evaluation time

Some points to confirm: 

  • Is 'Exclude draft and paused experiments' checked under settings?
  • Does the running experiment include a number of images?
  • What's the amount of 'implementation' pages?
  • Any Project javascript?
  • When setting an Experiment, Optimizely X allows the user to select to load Asynchronous.
  • Are you running no, trimmed or full jQuery?
Ryan
------------------------------------------------
The best solution is the simplest.

https://www.linkedin.com/in/ryan-camilleri
Experimentation Hero
puglisij 06-15-17
 

Re: Optimizely X unacceptable render blocking evaluation time

Hello,

 

  • Yes we've excluded draft and paused experiments. 
  • No, there are no images in the experiment.
  • We only have 4 pages under 'Implementation'
  • We do have other javascript on the page but it's not the issue. 
  • Using an asynchronous experiment doesn't seem to help. It seems to be evaluation of the core optimizely library that's the issue.
  • We have our own jQuery on the page so we've excluded it from the Optimizely bundle.

 

Thanks!

 

Level 2
JasonDahlin 06-16-17
 

Re: Optimizely X unacceptable render blocking evaluation time

@puglisij- Can you clarify if you are asking about the time spent blocking (which I asked for you to provide a breakdown of the timing you are seeing) or are you asking about "what could possibly be running during this phase, when there are no experiments that would necessitate this excessive load time". 

 

If you are asking about "processing time" (which I presume means 'after the snippet loads but before the browser is able to proceed due to processing the snippet which it just loaded'), can you share the details of how you determined it is taking 500ms to process?  That would truly be excessive and should be examined in detail.

 

 

--Jason Dahlin
Analytics and Testing Guru Smiley Happy


Experimentation Hero
puglisij 06-16-17
 

Re: Optimizely X unacceptable render blocking evaluation time

Hello. Sorry, I missed your question and thanks for the reply.

I'm asking about the time spent blocking / processing after the browser has loaded/received the optimizely script. 

Here is a screenshot as an example:

oz-blocking.png

This timeline shows roughly 560 ms total, compiling and processing the Optimizely snippet.

 

A lot of time spent on "initialize" is one common factor. 

 

 

 

Level 2