Faces of Optiverse: Hudson Arnold, a Strategic Optimization expert at Optimizely

by Optimizely ‎12-02-2014 December 2, 2014 - edited ‎12-03-2014 December 3, 2014

Hey Optiverse! Hudson is one of our Strategic Optimization Consultants here at Optimizely, and I'm sure you seen him participating and lending advice in Optiverse disucssions. Hudson is fantastic -- he genuinely wants to help empower others, and his expertise in analytics and optimization is exceptional. I've asked Hudson a few questions and posted his answers here so you can all get to know him a little better. Please feel free to ask any questions and bounce ideas around. 
Name: Hudson Arnold
Title: Strategic Optimization Consultant
  • Can you tell us a little about your path to end up working in Conversion Rate Optimization? 
At university in Santa Cruz, I majored in economics. Within economics, my primary concentrations were in econometrics and information systems. Along with classes in game theory, marketing, business strategy, I feel very strongly that my education had a very direct effect on my career today. 
The second half of the story is the past 3 years working in web analytics in an agency environment. I've 'grown up' professionally with marketing tech, particularly tools like Google Analytics. Always one to be pushing my own personal development and wanting to stay on the cutting edge, I started exploring testing techniques and technologies summer of 2012. Optimizely was still pretty new at that point, but already had a great reputation with my tech team. (We were lucky enough to have technical leads who are now at Apple and really knew their stuff). I started pitching, designing, project managing, and reporting on tests on a project basis for the agency's clients at that point. 
As my agency was relatively small and new to testing, it was pretty much on my shoulders to 'figure it out' and create a process that worked. And I'll tell you what, I loved doing it. I had never dreamed of a professional role that was so multi-disciplinary, in fields I was deeply interested;  design, communication, U/X, software development, statistics, all wrapped in empiricism. Not only that, as an analyst working mostly in reporting and pattern recognition, testing with Optimizely helped bring the power of action to those most invested in what the data was saying. I was able to produce a tangible, visible impact on what I had previously been stuck only reporting on.
Fast forward a couple years -  I had continued my development as an all-around digital marketer with a strong data bent (working in SEO, developing for responsive design, creative testing, mobile app measurement, messaging strategy, interactive banners, etc.), but still primarily focusing on web analytics. 
I was an unabashed Optimizely advocate throughout, and since joining the team earlier in 2014 I've been lucky to work with a huge variety of organizations at different market capitalizations and verticals on developing superior optimization strategies. It's very much a professional dream to come to work here every day. 
  • What do you think are the top 3 skills required to be effective at designing and executing your testing program (from a strategic point of view)?
This is a tough question, as a good strategist needs to be multidisciplinary at their heart, as I mentioned above. But, I'll give it a try. 
1. Understanding of core web tech: Strategy is about intelligently deciding, 'what should we do'. To a large extent, knowing 'what to do' is shaped by 'what can we do'. Understanding what's possible is critical to deciding what's optimal. Great strategists don't need to be able to write code - one could even argue that over-investing in development skills could make it harder to be great at strategy - but they do need to understand the vagaries of analytics software, the core architecture of the web, different applications of HTML, CSS, and JS, as well how their testing tool delivers experiences and records data. 
2. U/X: This one's a no-brainer; when we talk about A/B testing, most of the time we're talking about rigorously applying changes to U/X in order to influence behavior. But to be good, really good, at U/X, a great strategist should maintain some baseline understanding of behavioral psychology, principles of design, statistics, and always be on the hunt for what's brilliant, cool, or interesting in this rapidly evolving world of digital experience curation. 
3. Project management: This is a similarly faceted skill set (I'm kind of cheating through the '3 skills' brief at this point). To me this means, building meaningful project timelines, gathering technical and design requirements, garnering approval from stakeholders, creating thorough and useful documentation, fostering effective communication between team-mates, and generally just owning whatever it takes to get the job done right. Not all strategic leads will be managing projects as part of their day to day role, but I urge every strategic lead to get their hands dirty at least once in a while with a 'soup to nuts' commitment to getting a test built. Its an edifying exercise to say the least. 
  • On a similar note, you’ve gotten compliments about your contributions to strategic discussions in the Optiverse and you’ve helped many customers build out successful testing programs.   Can you share your secrets for effective strategic analysis?  
Wow, another great question. This is tough to answer in a general sense without getting too abstract, but I'll try to at least partially address it. (And maybe start thinking about a book!)
The easiest way to put it is that effective strategic action and analysis starts with effective prioritization. Prioritization allows one to focus on solving the right problems at the right time.
To unpack further, at any point in time, there are probably a range of business problems you can tackle. There exists any huge number of possible actions you can take per problem. How do you decide what to do? This is the strategist's dilemma. 
To effectively prioritize your work as a testing lead:
1. Define your goals
2. Define your problems and opportunities as they pertain to those goals. These can be actual test ideas, they can be organizational, they can be technical. 
3. Define your possible courses of action to approach those problems and opportunities. Again, its important you remain flexible; this is where the 3 key skills I mentioned above really come into play. 
4. Define a criteria (for example estimated value vs. estimated effort) for ranking courses of action. 
5. Select a course of action and execute. 
6. Repeat
Now, this isn't necessarily something you'll write down on paper every time you make a decision. No one  has time for that! Nor is it exhaustive or even reflective of a deep situational awareness that is the mark of great strategy. 
However, its a decent starting point when thinking about how to solve a given problem. 
Prioritization leads to focus, which leads to effective execution. 
  • Are there any tools/blogs out there you can’t live without?
Optimizely's Conversion Rate Optimism is fantastic. This is both a shameless plug and completely honest. It's great.
Avinash Kaushik's 'Occam's Razor'. If you're not already reading it, you need to be. 
For you Google Analytics pros out there, I really like the Excellent Analytics open source Excel plugin. It allows you to use the GA API in a GUI interface, no code required, and get your data directly into excel. If you're a power user, huge time save. 
UserTesting.com has an amazing service which I use extensively, crowd-sourcing demographically correct walk-throughs of your site. 
Eyequant is a brilliant new tool that algorithmically creates heat maps for your site, no actual behavioral data required. It actually works really well.
  • Where do you see conversion rate 2 years from now? Do you think it will evolve from A/B testing? How does the increasing dominance of mobile play a part?  
Over the next two years, we're going to see more of what we've seen on the web and in computing over the past 10 and 20 years; democratization of technology that was previously bleeding age into mass adoption. 
Specifically, I'd like to share a few related trends I'm excited about that are going to dramatically upgrade how CROs research, create, and learn from their optimization programs. I'm proud to say that Optimizely is driving these trends. They're currently barely on the horizon of most CRO's to-do list right now, but in 2015 we'll start to see adoption of what will be part of an evolving standard in 2016. 
1. Cross-device experiences:
  • As you alluded to, and to no-one's surprise, organizations are about to get dramatically better at connecting with their customers on mobile devices, or more appropriately, regardless of the device and application. Optimizely's recent launch of our iOS testing product is leading this charge. The pipes are being put in place to allow much easier implementation of creative, seamless cross-device experiences. 
2. Personalization: 
  • We’re nearing a tipping point in ‘personalization’ that is very exciting to me personally, which I’m sure that those of you who have experience with web analytics can relate to. There’s been a paradigm of using segmentation to unearth insights about how different cohorts of your audience interact with your brand online, differentiated by Demographics, Behavioral, and Contextual cues; until very recently, it was extremely difficult to make this data actionable. Intuitively, we all want to make sure our ‘50 year old male power user’ is getting a different experience than ‘teenage girl social media expert’. Once again, Optimizely is leading the democratization of technology by taking the guesswork and legwork from delivering personalization; we’re working on reporting that will ‘automagically’ discover segments of your audience that behave statistically significantly differently from the average, and allow you to instantly create audiences that you can target to; even today, with new features like List Targeting, Audiences, and integration with 3rd party data data management platforms (DMP) like BlueKai, Tealium, and DemandWare DemandBase, personalization is rapidly moving from ‘Nice to Have’ to ‘Playing Stakes’.

3. Machine Learning powered Everything:
  • This is the one that's really going to take people by surprise. We're just starting to hear about exciting, fanciful use cases of 'Big Data' powering machine learning from companies like IBM (Watson) and Google (search, driverless cars, etc.). Online, everyone is exploring what they can learn from the immense data sets provided by their analytics platform, digital advertising providers, A/B testing, app usage, CRM provider, email, etc. The current state of the industry is, if you're lucky, to have dedicated analysts conduct laborious querying, reporting, analysis, and maybe modeling exercises to find something useful from this data, then fed to your design or marketing or engineering team to hopefully make something happen. Imagine if your platform automatically discovers and displays the most interesting data, and provides a seamless process for turning that into action? This is actually much closer than you might expect. Optimizely is uniquely positioned as a platform that powers both experience creation and analytics reporting, so look for more automated machine-learning powered experiment analysis and experiment creation from us in the (short) future.
  • What’s your favorite vacation spot you’ve travelled to? Why?
Believe it or not, I think this question may be the hardest one here Smiley Happy
I've been fortunate enough to travel to a number of countries from new Zealand and Australia to Colombia to much of Western Europe. 
The cop-out answer is that whatever place I'm going to next is my favorite place to travel to. 
If I had to go anywhere right now, it would have to be Italy for its amazing culture, country, food, and history. And I'm always excited to find something new in my home of Northern California. 
Thank you. I hope you find my story and background interesting and useful as you work on your own 'testing journey'. 
Happy Testing!

by keith_lovgren Level 2
‎12-04-2014 December 4, 2014

I really enjoyed this interview!


Your 3 part answer to this question really resonated with me -


What do you think are the top 3 skills required to be effective at designing and executing your testing program (from a strategic point of view)?


In particular this part stood out to me -


''But to be good, really good, at U/X, a great strategist should maintain some baseline understanding of behavioral psychology, principles of design, statistics, and always be on the hunt for what's brilliant, cool, or interesting in this rapidly evolving world of digital experience curation."


I hope you do follow through with writing a book on CRO. I'd definetly preorder some copies for our office!