Wordpress Headline Testing Plugin - Understanding Results
We have just installed the Optimizely headline testing wordpress plugin. The plugin is very easy to use, however, we're not sure what exactly it's testing.
The results show lifts in 'engagement', described as any clicks on the experiment page. But I'm not sure how clicks on the page the visitor is on relates to the performance of the headline? The variations do not appear to follow the content object accross the site to measure CTR on the headline, so I'm just not sure what exactly the headline experiment is actually testing for?
One way to think of "Engagement" is the opposite of bounces. What it's measuring is how many people perform at least one action on the page, which means they have not bounced. It does not make distinctions between what is being clicked.
To track clicks on a specific element, such as a link, then you should use click goals. More info here: https://help.optimizely.com/hc/en-us/articles/2000
Hello @AboveAverage !
My name is Brad and I am the developer that created the plugin as a Hackweek project. When a test is created its URL Targeting is set to the entire domain. This way any page the headline might be included on will be changed.
However, this is not ideal since people are included in the experiment that may not have even seen the new headline. They are then considered a conversion if they are included in the experiment, and they visit the article page. By default we add two goals, one for engagement like you mention above and one for views to article page. The views to article page is the goal you should want to pay attention to and not the engagement goal.
We have an enhancement request for the API to allow for conditional activation. What this means is that we will be able to activate the experiment only if the user sees the headline instead of including everyone in the experiment no matter if they see it or not.
I hope that answers your question and please let me know if you have any other questions or suggestions on how we can make the plugin better!