Rich-text Reply

How are your teams setup?

leepearso 08-03-15

How are your teams setup?

[ Edited ]

I've come across examples of teams that are setup in a huge variety of ways so would like to hear about how everyone else is setup and what you are finding as working or not working.

 

Has your setup changed much and how do you intergrate with other teams around the business?

 

In our case we are responsible for merchandising the site and trading online, but also have ownership of the tool and run testing with this. We link into specific trading teams, UX, design and analytics and try and take data from all of these. We're currently exploring agile teams as our current setup isnt optimal and there's some overlap, for example analytics, UX and development will roll out tests on our actual platform which also has a/b testing functionality.

 

What is everyone else doing?

 

 

Amanda 08-03-15
 

Re: How are your teams setup?

@leepearso - Thanks for posting. I am sure the Optiverse will have some incredible insight here. I also wanted to point out this amazing resource @ShanaR wrote: https://blog.optimizely.com/2015/06/15/optimization-benchmark-report/

 

It's a benchmark report that highlights the top benefits and key metrics around team size, pacing, and prioritization. Check it out and let us know what you find most helpful. 

Optimizely
Pauline 08-03-15
 

Re: How are your teams setup?

Hello!

 

We recently changed the way our testing team is structured:

 

Before:

- We had one team dedicated to A/B testing. The team had 1 product manager + dev + designers+ QA + analytics. It was great to implement the testing framework, create the testing culture within the company, have strong optimization advocates and experts in the organization, build a great testing velocity, define prioritization and best practices.

It was entirely product driven (with inputs from everyone in the company).

 

Now:

- Each product team is responsible for A/B testing in their product area. Let's say we have a Car team, this team is responsible for the entire product development, including A/B testing.

It is still very much product driven and has the support of a full team including dev and designers.

This new structure will help develop a culture of test & learn/optimization among all team members and give clear ownership to each team from ideation phase to deployment.

 

 

Hope it helps!

Product Manager
Level 2
MartijnSch 08-04-15
 

Re: How are your teams setup?

Our structure hasn't really changed in the past years but as our team has grown the roles have changed a bit making them more specialized. Currently our analytics & insights/research team takes care of A/B testing. They're a team of 2,5 FTE in the marketing team of The Next Web. Consisting of 1 fulltime web analyst, 1 data reseacher, 1 PPC Specialist/ Web Analyst. Our web analyst takes care of building and managing the team to set up the tests that we'd like to run. I assist the team whenever needed with support on all fronts.

if needed they can also be supported by the development team to build tests but for now that is not needed that often.
Zsolt 08-04-15
 

Re: How are your teams setup?

For us it looks like this - all of marketing has ideas, we get together and refine a hypothesis, marketing director approves and gives the thumbs up, and then I go in and create the test. The only (and I've been told rare) advantage we have is that the person who sets up the test is a designer who can code - which makes us very efficient.
Level 5
jrichardson 08-07-15
 

Re: How are your teams setup?

We sit in ecommerce and are a centralized team that supports the ecommerce product teams. We have 2 front-end and 1 analyst on our AB Team. Each product team has a product owner who defines the feature roadmap and discovery efforts, UX, Designer, dedicated analyst and dev team.

 

Our setup hasnt changed yet. We are starting to rely more on the product teams for analytics support and test roadmaps as our goal as a team is to help the product owners validate the priority of their new ideas and the best UX for roadmap items. 

 

We handle all the strategy around the test idea, hypothesis generation and complete delivery - including analysis. 

 

Hope this helps!

 

leepearso 08-09-15
 

Re: How are your teams setup?

Loads of different set ups which is really interesting to read about.

Amanda i did have a read through the blog again lots of insight. Most interesting is that whilst it tends to sit with marketing its also sits with a lot of other places within various businesses.

How do you all handle personalisation and optimisation. Do your same teams look both or do you handle them in different ways?

Re: How are your teams setup?

Our testing team is made up of three primary members and we support A/B testing, CRO and landing page creation for the company. Two of our team members have design and coding skills which allows us to create most of our tests ourselves, and two members have extensive experience with testing strategy and idea generation.

 

It's interesting to hear that many teams don't seem to have a dedicated CRO role on their team. I agree that dev, design and analytics support is critical but also having someone with a history of optimization experience is also beneficial.

SH13 08-13-15
 

Re: How are your teams setup?

We have a central digital services team that owns analytics, SEO, testing, and a few special digital products involving content marketing. We work under the corporate branch of our business and assist upwards of 25-30 brands. Our analytics, SEO, and testing team is really only 2 people (3 if we have an intern). I personally manage Optimizely right now, consulting with individual marketing teams to help craft tests. Typically they reach out to us and submit through a form that we've created internally. This is how we queue jobs for review. The longterm plan is to put more power into the hands of the individual teams (and alleviate some of our execution workload), by training them up on the platform to handle most of their simple tests. I've already started this with some of our power users, so they create tests and ping me when they've set them up. I go in and review to make sure nothing will break the site, and push it live. This is a much better synergy for us.

As for development and design, we have other internal teams that we coordinate with when necessary. For most design needs, the brand teams coordinate that creative before reaching out to us with their test. I usually work directly with development if needed. So far we haven't really needed them, although some more complicated tests (like pop-ups) will require their help.
Level 2
tscala 08-14-15
 

Re: How are your teams setup?

[ Edited ]

Hello! Our B2B media company's CRO team is about five people, a mix product management, audience and analytics, and design. We use a simple framework (https://community.optimizely.com/t5/Optiverse-What-s-New/Have-questions-about-prioritizing-your-road... ) to catalog, prioritize and track tests in Trello, and one dedicated resource for HTML, CSS and JavaScript to develop and implement the tests.

Level 2
Abel_Ramos 08-18-15
 

Re: How are your teams setup?

[ Edited ]

Hi There,

We are also structured very much like Pauline outlined.

We have product teams that are responsible for their areas of the site. As an example, I lead the Ecommerce team and everything related to testing falls to us. We have both designers and developers who work on our team that assist with the implementation.

All the teams then meet biweekly to discuss results and determine how tests being run on one area of the site can be translated to other areas. We also meet to discuss site wide testing ideas. This meeting also includes our Analytics teams who helps drives ideas from what they are seeing.

adzeds 08-19-15
 

Re: How are your teams setup?

We currently have a strange approach for my team...

In that we are not a team, it is just me!

I am the product owner and the conversion optimizer!

It is my responsibility to draw up the product strategy, research, plan tests, build tests, report on tests and push through any dev work required post testing!

Obviously this is not a very efficient system because the product owner side of my role takes up a lot of my time now, so it may be time to build out a team structure!
David Shaw
Level 11
leepearso 08-19-15
 

Re: How are your teams setup?

[ Edited ]

David do you class (or would you class) your conversion optimizer role as covering anything that aids conversion? - Do you cover onsite personalisation or is it more geared toward site improvement, UX, Design where your product owner position plays more of a roll?

Also if you were building that team out, how would your product owner link with the CRO person?


adzeds 08-19-15
 

Re: How are your teams setup?

[ Edited ]

I would class the conversion optimizer role as anything that aids conversion across the board!

In building out a team the product owner and the CRO role would be tightly linked.

I am currently in the process of implementing a data-driven decision making process for running our product development lifecycle.

The product owner will own the business strategy and product growth strategy and takes direction from all areas of the business to decide what changes should be considered for our product (website/app) and then all of these would be tested to make them quantifiable. We then decide what gets built into the product and prioritized based on this data.

It will make our product development much more efficient and ensure that everything that gets added to the product has a quantifiable benefit and will not lead to degradation to the product over time.

David Shaw
Level 11
JeremyBrad 09-30-15
 

Re: How are your teams setup?

Over the years we've really changed the way we approach (and value) a/b testing. Traditionally we used simple in-house a/b testing tools and each Product Manager was responsible for a/b testing their own products whenever big changes were made. Now we tend to test ideas much faster than our release cycles would allow and we’re more inclined to test even small changes before building them out on our site. Because testing has become so much more involved, it hasn’t been feasible for every product manager to specialize in conversion testing. In order to help the agile delivery teams get the most out of their products, we’ve formed a CRO department/team that supports all the product teams with their conversion goals. Product Managers meet with the CRO team to describe their conversion goals, explain why they’re important, offer experiment ideas, and agree upon importnat KPIs. From there the CRO team prioritizes the requests based on level of effort and potential impact for the business overall. This has been an ad hoc process for awhile now, but we’re trying to make it more structured going forward. Ideally the CRO team will support the Product Mangers much like the UX and Design teams have traditionally done.

 

Hope this is helpful.

 

-Jeremy

CRO Specialist at HomeAdvisor

Jeremy Bradford
Kristen_S 10-09-15
 

Re: How are your teams setup?

Our web team is responsible for analytics, SEO, UX/UI, and testing (qualitative and quantitative) in addition to maintaining the website and creating content in support of Marketing. We have 1 manager, 4 web designers, and a project manager. Testing ideas are generated within our team as well as from the various marketing teams. Our web team is responsible for developing those ideas into an actionable test strategy, implementing the tests, and then analyzing the results so that we can report back and provide recommendations on how to move forward.
"Ease of use may be invisible, but its absence sure isn't." - IBM
robertchan 10-27-15
 

Re: How are your teams setup?

Our company is structured to have all departments work together. I think the key components of a successful testing environment are connecting results of experiments with corresponding departments, and ideating together from that point. On a week to week basis, I will meet with the following departments: audience development, design, product management, and web development. Each department plays its own role in the process of testing. Audience development reconciles all experiments running to ensure that the proper goals are being reached while design sees that the designs constructed for tests meet company branding protocol. Finally, product management and web development go hand in hand. Product management provides support in ideation while web development checks to see that all development within experiments are feasible to eventually be implemented and assists with any coding issues.
Robert Chan

scleveland 11-09-15
 

Re: How are your teams setup?

We have a smaller team that takes care of all the A/B testing. 

 

Personally, i'm officially the point person for our A/B testing program.  I take care of all the set up, coding, timing, reporting, etc for all the tests.  Our Design Lead and I work together on getting the look the way want it and if i'm able to code it like she wants it. 

 

Our suprvisor, helps guide us with the Analytics side of things.  She helps us focus on what we should test and how to test it.

 

The 3 of us meet weekly going over any new mockups our designer has completed, and tests that haven't started yet but are ready to go, go over results of currents, and brainstorm ideas for upcoming tests.

 

For Q/A, I do not typically do any, being that I'm doing the coding.  That way I don't get blinders on and miss anything.  I typically put it back on our designer to do that, but on larger tests we include our supervisor on that.

 

It seems to work quite well doing testing this way.  Each of us has our own role, but we all are there to help each other out.