Rich-text Reply

Testing New Sports Navigation Sitewide

Wowie 03-22-16

Testing New Sports Navigation Sitewide

Hi guys,

 

Our business is very new to using Optimizely. It was only recently implemented on our site after we had a new registration form go out 6 weeks ago that dropped conversion by 25% - against our advice of course...

 

In a few weeks we are looking to launch a new sports nav which is planned to take over the current navigation at crownbet.com.au/sports. The new nav can be viewed here; https://www.dropbox.com/s/212qt7mewc8vq4o/new-sports-navigation.png?dl=0

 

given what happened with registration form they are very keen to roll it out by way of testing. So my main question is; what is the best way to build/launch/use Optimizely to facilitate this test so we can understand the impact? Is it URL redirect? So it would be built on somthing like; crownbet.com.au/sports/2. This would need to be displayed across all pages and markets that sit under the 'sports' URL's

 

Some questions our devs had were;

  1. How should it be built so Optimizely can be setup to test?
  2. How will the test be setup when the nav needs to be across all sports pages?
  3. What are the best targeting options?
  4. If it's a winning test how do we then roll it out in production (I assume our devs should be able to anser this)?

Any advice or expereince doing similar greatly appreciated.

 

Cheers

Level 2

JDahlinANF 03-22-16
 

Re: Testing New Sports Navigation Sitewide

[ Edited ]

Hi @Wowie,

Can you annotate the dropbox screen shot to show all of the new items?  There may be more to it than what I describe below.

 

If the only difference is the grid near the top of the page (the stuff between "All Sports" and "Upcoming Sport"), then this could be made part of the default presentation of the page and you could use Optimizely to inject CSS that hides it for the Control version of the pages in your experiment:

/* _optimizely_evaluate=force */
$('head').append('<style id="opt_123456789">#box_nav_ver2 {display:none !important;}</style>');
/* _optimizely_evaluate=safe */

(Placing an ID on the style that you are inserting is my own personal preference and totally not required.  I like to do this so I can easily QA the experiment by seeing if my <style> has been added to the page or not.)

 

For URL Targeting, you can use substring match on "crownbet.com.au/sports" which would make it run on every page containing that text in the URL.

 

For roll-out:

1- if the experiment wins, you simply end the test

2- if the experiment loses, DEV removes the new stuff (if this takes much effort, they can temporarily modify the site's CSS to hide the new nav just as you did in the experiment)

Wowie 03-22-16
 

Re: Testing New Sports Navigation Sitewide

Hi @nap0leon

 

Thanks for taking the time to answer this - really apprecaited.

 

I should have mentioned this, but how would it work when a users navigates down to the next level of the menu? An example would be; user clicks soccer grid button (see dropbox), this clicks through to the soccer sub-page with a soccer only menu in the same style, but different menu items.

 

Thanks

Level 2
JDahlinANF 03-22-16
 

Re: Testing New Sports Navigation Sitewide

@Wowie - Presuming the appropriate drop-down box has been created for each page, as long as the ID of the wrapper, regardless of content (whichever sub-menu is appropriate for that page), the same code would work across all pages.

 

Since we're talking design stage of how to do, I would suggest they build it like this:

<div id="box_nav_ver2" class="whatever is needed for this navigations color scheme and sizing on this page">
  <!-- The HTML for this pages navigation goes here -->
</div>

Each page would have it's own HTML that is the correct drop-down navigation for that page (the sub categories for that page's grid).  The ID used on this would be the same for every page, the classes can be altered to set anything specific for this page... perhaps each sport has a different color scheme, items in the grid, etc.

 

From a testing perspective, you would run the code that hides the div on all pages under /sport by using "Substring Match" which means anything containing "crownbet.com.au/sports" in the URL would be included, such as "crownbet.com.au/sports-soccer", "crownbet.com.au/sports/boxing", and "crownbet.com.au/sports-baseball/MLB/RedSox", etc.

 

 

 

 

Wowie 03-23-16
 

Re: Testing New Sports Navigation Sitewide

Thanks @nap0leon. This has helped greatly.

Level 2
Wowie 05-05-16
 

Re: Testing New Sports Navigation Sitewide

Hi again guys - we are about two weeks away from going live with this test, which is exciting.

 

I'd love to hear from the community here how you would measure the success of a navigation change like this?

 

The way I am looking to do it is as follows:

 

With the end goal being bets placed using number of bets placed conversion - thinking being if the new nav makes it easier to find the page/market a users is after, conversion will go up.

 

Will will also be looking in Google Analytics for time spent on page, page value, exit rate etc.

 

Obviously many of our repeat users are “trained” to use the old nav and a change could throw them. To counter this we will use a segment for “New Users”, so we can look at them in isolation, I think they’re a better measure of this UX change. One problem we are going to encounter is the login/logged out state so we may just target Chrome browser instead.

 

You can view the current sports nav here https://crownbet.com.au/sports/next-up and view the new nav here https://invis.io/MY6DW0RAN

 

Would love your thoughts and/or suggestions.

 

Cheers

Chris

 

 

 

Level 2