Rich-text Reply

Testing with a popup form

rcarrollinogen 07-02-14

Testing with a popup form

Hey, has anyone implemented a test using a popup form? I'm wondering how compatible Optimizely is with elements such as pop ups?



Test A: Homepage with normal embedded lead gen form

Test B: Homepage with popup lead gen form


Would love to hear insight, best practices or tips on this kind of test and how it could be implemented from a technical standpoint.

nolanmargo 07-02-14

Re: Testing with a popup form

Hey Carrollinogen - we've recently run a series of lead capture tests - using a modal lightbox technique. We identified a candidate page that receives thousands of visitors daily, through an inherited SEO tactic, that represented roughly 45% of our daily visitor volume. Despite the high number of visits, the sessions were purely transitory, at times a >97% bounce rate, all while providing ZERO economic value.

We quickly drafted up 6 variations, and called them on the highly trafficked page and were surprising to see each day our email address list pile & pile up. We now have a steady-stream of 'leads' pulling through. Even without optimizing our email series, this new cohort of 'submitters' is now 5x more likely to head down our conversion funnel and discovery more about our product.

Since the initial test, we ran a couple more variations, one that 'delayed' the modal display by 10 seconds, which flopped and another, which removed a restrictive opt-in statement that actually continued to work as well as the original (this is the one we are currently running).

Be wary of offer testing (i.e. discounting) for those, who come from less qualified sources OR like in our case, exhibit user behaviors that may not lead to conversion, even WITH an offer ... search out your 'qualified' visitors and target them with your discounts as incentive for an email submission Smiley Wink

Hope that helps - PM if you'd like some examples &OR techniques.
adzeds 07-03-14

Re: Testing with a popup form

Optimizely is perfectly capable of running this sort of test.

We ran something similar last month and were able to generate a substantial increase in newsletter signups so I would very much recommend giving it a try.

If you need some help with coding it into Optimizely then feel free to drop me a message and I can advise with some code and implementation techniques that I have tried and tested.
David Shaw
Level 11
Khattaab 07-03-14

Re: Testing with a popup form

Great question @rcarrollinogen. It's important to avoid introducing added friction to the user experience when requesting personal information. In my experience, normal embedded forms perform better than pop-ups. A great way to draw attention to a form field is to layer it on top of an image, ideally near a face or on the side of the page that the face is looking. User intuition is to lock eyes with a face, so if that is where the user begins evaluating content, layering the form field adjacent to the image and leading it with a bright color and compelling value proposition (to answer "Why should I complete this form?") is a best practice.

Khattaab Khan
Director, Experience Optimization | BVAccel
Level 5

Re: Testing with a popup form

Thanks everyone for the feedback so far! Good stuff here.

Re: Testing with a popup form

We ran a somewhat similar test and found that the lightbox version did not perform as well as having all of the fields on the page. Our situation was a little unique though: we were testing a two-stage form process where the user submits their email address and from there are presented with the rest of the fields required for full registration to our site.

In the control version the rest of the fields were presented on a separate page from the original email only form; obviously that isn't ideal. So our test was to present the rest of the fields as either a lightbox or on the same page directly underneath the original email field. Both the lightbox and the on-page versions outperformed the original in terms of full registrations by a significant margin. Between the two though the on-page version did much better than the light box. It resulted in a ~60% increase in full reg conversions.

I think for your situation a lot will depend on how many fields your form contains and how you're presenting the form on the page itself. Finding ways to highlight the form on the page may work better than presenting it in a light box.

Re: Testing with a popup form

Thanks Brian, good insight there and I will consider all these factors when testing!
Amanda 08-22-14

Re: Testing with a popup form

@rcarrollinogen , @nolanmargo , @brian_deloach and @Khattaab  :


@gmoptimizer had a question about dropdown and flyout menus that I think you would all have great advice for based on the expertise you provided here! I moved the discussion out of this thread in an effort to give it as much exposure to the whole community as possible. Take a look at his question and let us know what you think.