Rich-text Reply

Mobile Walkthrough: From 0 to Experiment

Pam 10-07-15

Mobile Walkthrough: From 0 to Experiment

[ Edited ]

Hi Optiverse,


I’m Pam, the Mobile Solutions Architect Lead at Optimizely. I hosted an online workshop called “Mobile Walkthrough: From 0 to Experiment” as part of our hands-on Optimizely Workshop series today. We covered a few important points about mobile apps, including:


  • The biggest challenges in mobile apps today
  • How A/B testing can help mobile apps be more agile
  • How to set up an Optimizely mobile experiment
  • Best practices in designing mobile experiments
  • How A/B testing fits in with my app development cycle


Mobile apps face a few unique challenges. Due to the high technical demands and slow release process of mobile apps, it’s very difficult to be agile. Furthermore, mobile apps face serious challenges in long-term retention. One study shows that on average, mobile apps lose nearly 80% of new users within the first three days of install.

Screen Shot 2015-10-07 at 7.43.56 AM.png


How do mobile apps address these pain points? One tool in your arsenal is A/B testing, which can help you measure the effectiveness of new features in phased rollouts, make data-driven decisions about what to build next, and optimizely your app design based the metrics that are most important for mobile.


Screen Shot 2015-10-07 at 7.42.05 AM.png


Registration, which marks one of the first moment when a user engages with your app, presents great opportunities for A/B testing. If you’re a music app, what kind of experience encourages more visitors to choose and play their first song? If you’re a game app, how do you prompt visitors to create a character? Usage and Retention presents another great opportunity for A/B testing. How do you help users continue to find value in your app, over time?


Many of the tests that Optimizely mobile customers run on their apps are designed around these two key metrics. The top five areas where our mobile customers test are:


  1. The onboarding experience (registration)
  2. Product detail screens (registration, usage & retention)
  3. Promotions and upsell opportunities (usage & retention)
  4. The homescreen (usage & retention)
  5. Registration flow and checkout flow (registration, usage & retention)


My workshop today walks through the entire process of creating a mobile experiment in Optimizely. Ultimately, each step in your A/B testing timeline should be integrated with your full app development timeline so your development process is more agile and the decisions you make are backed by data.


Screen Shot 2015-10-07 at 7.36.52 AM.png 


Here are a few other best practices in mobile testing:


  • Know what tests you want to run at the beginning of your sprint.
  • Have a developer involved in the process.
  • Use phased rollouts for new features.
  • Use live variables and code blocks.
  • Integrate Optimizely with your analytics platform.


That’s it! Have any questions? Feel free to chat with me here in Optiverse.


Here's the recording


bartman4 10-09-15

Re: Mobile Walkthrough: From 0 to Experiment

I clicked the link in the email I received to watch the recording of this webinar and this is the page it led me to...I do not see the recording. Where is it?

Level 1
Amanda 10-09-15

Re: Mobile Walkthrough: From 0 to Experiment

@bartman4 - The recording is added now. Sorry for the confusion! Let us know if you have any questions. 

JennyD 10-13-15

Re: Mobile Walkthrough: From 0 to Experiment

Being able to revisit the webinar is extremely valuable but the audio is pretty spotty. 

Level 1
Yeesheen 10-13-15

Re: Mobile Walkthrough: From 0 to Experiment

Hi @JennyD,

We noticed that there were issues with the audio quality in the recording too. Apologies for any frustration this may have caused! We're troubleshooting our process to improve on this in future Optimizely Workshops.

Pam 11-24-15

Re: Mobile Walkthrough: From 0 to Experiment

Hi @JennyD,


The recording has been re-recorded with updated audio!  Feel free to check it out!