Optimizely X unacceptable render blocking evaluation time
We're looking to move to Optimizely X in hopes of improving the load time on our site.
In our experience, Optimizely demonstrates an unacceptable level of performance during the critical rendering path, where optimizely must be loaded synchronously to avoid the dreadful FOUC.
During our test, we created 3 small experiments in X. One was running on the homepage, another was paused, and another not started. This results in a snippet which is about 230kb minified and around 70kb gzipped. Quite large as is. In our runs we noticed this adds an average of 500ms of render blocking time before the first paint (which we confirmed by removing optimizely from the page) on pages where there aren't even any experiments running. As you know, this can have a major impact on revenue generating websites.
This is Not an improvement over Classic. While we like some of the features X advertises (e.g. asynchronously loaded experiments) we were really hoping for better performance when our pages are initially loading.
What could possibly be running during this phase, when there are no experiments that would necessitate this excessive load time? We find this completely unacceptable.
(DNS lookup, Initial Connection, SSL, Waiting, Downloading, etc.)
Hitting a few different sites that are using Optimizely X that I know have only 1 or 2 experiments set up, loading the un-cached snippet 10 times each:
1- snippets were all between 75 and 100 KB
2- DNS Lookup: avg 80ms
3- Initial Connection: 25ms
4- Waiting: 75ms
5- Content Download: 120-350ms
Total load time was 300-550 ms.
The longest section of the timing is the download speed.
You could try testing to see if running a <link rel="dns-prefetch" href="//www.optimizely.com"> would help, but if your snippet is at the top, it probably won't.
Analytics and Testing Guru
Some points to confirm:
- Is 'Exclude draft and paused experiments' checked under settings?
- Does the running experiment include a number of images?
- What's the amount of 'implementation' pages?
- When setting an Experiment, Optimizely X allows the user to select to load Asynchronous.
- Are you running no, trimmed or full jQuery?
The best solution is the simplest.
- Yes we've excluded draft and paused experiments.
- No, there are no images in the experiment.
- We only have 4 pages under 'Implementation'
- Using an asynchronous experiment doesn't seem to help. It seems to be evaluation of the core optimizely library that's the issue.
- We have our own jQuery on the page so we've excluded it from the Optimizely bundle.
@puglisij- Can you clarify if you are asking about the time spent blocking (which I asked for you to provide a breakdown of the timing you are seeing) or are you asking about "what could possibly be running during this phase, when there are no experiments that would necessitate this excessive load time".
If you are asking about "processing time" (which I presume means 'after the snippet loads but before the browser is able to proceed due to processing the snippet which it just loaded'), can you share the details of how you determined it is taking 500ms to process? That would truly be excessive and should be examined in detail.
Analytics and Testing Guru
Hello. Sorry, I missed your question and thanks for the reply.
I'm asking about the time spent blocking / processing after the browser has loaded/received the optimizely script.
Here is a screenshot as an example:
This timeline shows roughly 560 ms total, compiling and processing the Optimizely snippet.
A lot of time spent on "initialize" is one common factor.