Rich-text Reply

Combine two test result

Eram 03-08-17

Combine two test result

I was wondering if I can combine two tests to get to the result sooner?

 

Each of these pages are redirecting to another URL for Variation#1 but the change is similar, so I was wondering if there is any way to add up the result on both pages?

 

Level 2

JasonDahlin 03-08-17
 

Re: Combine two test result

If the experiments are similar, you can run it as a single experiment, but you would need to start over.  If you want to presume that the visits in each experiment are unique (no duplication) you can combine the results of the two experiments in any of several non-Optimizely calculators to gauge the effectiveness.0

If you want to start over, the new experiment would be:
1- URL Targeting for each of the two URLs you want to redirect
2- In the edit-code block for Variation B, you will write code that does the redirect so that it redirects to the correct new URL depending on which URL they are entering.

 

Something like this:

/* _optimizely_redirect=http://www.thisdoesntreallymatter.com */
var _optly={redir:document.createElement("a")};

if (window.location.pathname == "/the_path_of_the_first_page") {
	_optly.redir.href="https://www.example.com/the_full_path_of_the_first_URLs_redirected_page";
	_optly.cur=window.location.search;
	if (_optly.cur) {_optly.redir.search=_optly.redir.search ? _optly.cur + "&" + _optly.redir.search.slice(1) : _optly.cur;}
	window.location.replace(_optly.redir.href);
} else if (window.location.pathname == "/the_path_of_the_second_page") {
	_optly.redir.href="https://www.example.com/the_full_path_of_the_second_URLs_redirected_page";
	_optly.cur=window.location.search;
	if (_optly.cur) {_optly.redir.search=_optly.redir.search ? _optly.cur + "&" + _optly.redir.search.slice(1) : _optly.cur;}
	window.location.replace(_optly.redir.href);
}
--Jason Dahlin
Analytics and Testing Guru Smiley Happy
Eram 03-08-17
 

Re: Combine two test result

Thank you Jason.

Which URL should I start the Experiment with? (the first URL that is asked in a pop up when you want to start an Experiment?)

Level 2
JasonDahlin 03-09-17
 

Re: Combine two test result

Since you will not be using the WYSIWYG editor for this experiment, you can put whatever URL you want to in there (I like to use my homepage URL).

 

--Jason Dahlin
Analytics and Testing Guru Smiley Happy
Eram 03-10-17
 

Re: Combine two test result

Thanks. I've started the test. Is there any way to check if everything is working?

 

I can see the original and variations here:

https://online.scu.edu.au/ppc/law/?optimizely_x8287591517=0

https://online.scu.edu.au/ppc/law-2/?optimizely_x8287591517=1

https://online.scu.edu.au/ppc/mbl/?optimizely_x8287591517=0

https://online.scu.edu.au/ppc/mbl-2/?optimizely_x8287591517=1

 

And so far Optimizely is showing visits to Original and Variation pages, but Hotjar is not showing any recording from the Variation versions yet.

Level 2
JasonDahlin 03-10-17
 

Re: Combine two test result

I'm not familar with the HotJar integration, so cannot comment on that part, but the experiment is working!

 

Stays on the original:

https://online.scu.edu.au/ppc/law/?optimizely_x8287591517=0

https://online.scu.edu.au/ppc/mbl/?optimizely_x8287591517=0

 

Redirects to the new:

https://online.scu.edu.au/ppc/law/?optimizely_x8287591517=1

https://online.scu.edu.au/ppc/mbl/?optimizely_x8287591517=1

 

I noticed your experiment is targeting the entire website.  This is fine, it works, but it will make it much more difficult for your experiment to achieve statistical significance when it includes lots of users who do not actually participate.  You should change your URL targeting to include only the two original URLs that you are testing.  Make it use "Simple Match" for these:
https://online.scu.edu.au/ppc/law

https://online.scu.edu.au/ppc/mbl

--Jason Dahlin
Analytics and Testing Guru Smiley Happy
Eram 03-14-17
 

Re: Combine two test result

The URL targeting was exactly the problem, so I added those and it is working perfectly.

 

Thanks a lot for your help Smiley Happy

Level 2
Eram 04-18-17
 

Re: Combine two test result

Hi Jason,

 

I've started a new test similar to what you helped me before on these three pages:

 

https://studyonline.rmit.edu.au/ppc/master-of-hr-management/?optimizely_x8310616427=0

https://studyonline.rmit.edu.au/ppc/master-of-marketing/?optimizely_x8310616427=0

https://studyonline.rmit.edu.au/ppc/master-supply-chain-logistics-management/?optimizely_x8310616427=0

 

Which redirects the pages above to :

https://studyonline.rmit.edu.au/ppc/master-of-hr-management-4

https://studyonline.rmit.edu.au/ppc/master-of-marketing-3

https://studyonline.rmit.edu.au/ppc/master-supply-chain-logistics-management-3

 

The problem that I'm facing again is I'm not seeing consistency between Optimizely, Google Analytics, and Hotjar's data.

Optimizely is showing much more visits to Variation#1 pages than Analytics and Hotjar.

 

Have I set anything wrong again this time?

 

Thanks for your help.

 

Level 2
JasonDahlin 04-19-17
 

Re: Combine two test result

Hi @Eram,

The only change I see that you should make to the experiment code is that it will not redirect users in the test group if their URL does not contain the trailing "/".  That is, this will redirect:
https://studyonline.rmit.edu.au/ppc/master-of-hr-management/

But this will not

https://studyonline.rmit.edu.au/ppc/master-of-hr-management

 

Here's an easy remedy (there may be more efficient ways of doing this, but this is nice and legible, making it suitable for re-use):

var _optly={redir:document.createElement("a")};

if (window.location.pathname == "/ppc/master-of-hr-management" || window.location.pathname == "/ppc/master-of-hr-management/") {
	_optly.redir.href="https://studyonline.rmit.edu.au/ppc/master-of-hr-management-4";
	_optly.cur=window.location.search;
	if (_optly.cur) {_optly.redir.search=_optly.redir.search ? _optly.cur + "&" + _optly.redir.search.slice(1) : _optly.cur;}
	window.location.replace(_optly.redir.href);
} else if (window.location.pathname == "/ppc/master-of-marketing" || window.location.pathname == "/ppc/master-of-marketing/") {
	_optly.redir.href="https://studyonline.rmit.edu.au/ppc/master-of-marketing-3";
	_optly.cur=window.location.search;
	if (_optly.cur) {_optly.redir.search=_optly.redir.search ? _optly.cur + "&" + _optly.redir.search.slice(1) : _optly.cur;}
	window.location.replace(_optly.redir.href);
} else if (window.location.pathname == "/ppc/master-supply-chain-logistics-management" || window.location.pathname == "/ppc/master-supply-chain-logistics-management/") {
	_optly.redir.href="https://studyonline.rmit.edu.au/ppc/master-supply-chain-logistics-management-3";
	_optly.cur=window.location.search;
	if (_optly.cur) {_optly.redir.search=_optly.redir.search ? _optly.cur + "&" + _optly.redir.search.slice(1) : _optly.cur;}
	window.location.replace(_optly.redir.href);
}

Looking through the waterfall of network requests, GA, HotJar, DoubleClick, and even LiveChat are being called before the redirect occurs.  To help eliminate unneeded network calls try adding this line above the redirect stuff:

/* _optimizely_evaluate=force */

and add this line at the end of it:

/* _optimizely_evaluate=safe */

However, you said that GA and HotJar are *not* picking up as many visitors as Optimizely is... if anything, I would think GA would be tracking both the hit to the original page and the redirected-to page.  Some of this can be a difference in how GA and HotJar and Optimizely define "Visitors" (Optimizely's is "unique" and GA, if I recall correctly, it's been awhile since I've used them, Visitors really means 'Visits').

--Jason Dahlin
Analytics and Testing Guru Smiley Happy
Eram 04-21-17
 

Re: Combine two test result

Thanks for you help Jason.

 

I've added the change to the code as you suggested and restarted a new test. Hopefully it works fine this time.

 

I received the notification below when I started the test. Could this be an issue?

"Your experiment's targeting url settings don't match https://studyonline.rmit.edu.au"

Level 2
JasonDahlin 04-21-17
 

Re: Combine two test result

Hi @Eram,

That message is a non-issue.  It only means that your editor used your homepage, but you altered the URL targeting so that pages other than your homepage are affected.  Almost every experiment I run sees that message.

--Jason Dahlin
Analytics and Testing Guru Smiley Happy
Eram 3w ago
 

Re: Combine two test result

Hi Jason,

 

I was wondering if there is any way to check the reasult of each landing page seperately?

 

Thanks,

Eram

Level 2

Re: Combine two test result

Perhaps you would be better off with 3 separate experiments?

Another option would be to create a custom dimension that records a different value based on which page they see.  You could then use this as a segment in your experiment results.

--Jason Dahlin
Analytics and Testing Guru Smiley Happy