Rich-text Reply

Best Practices for Sharing Results Internally?

JohnH 01-06-15

Best Practices for Sharing Results Internally?

[ Edited ]

How frequently and in what format do you all share your testing results internally? It seems like adding in layers of transperancy would be helpful to building support & excitement about the testing program. 

 

I was thinking a monthly email, but wasn't sure what metrics would be the most interesting to share. 

 

What do you all think / What's your process???

Level 2

greg 01-07-15
 

Re: Best Practices for Sharing Results Internally?

[ Edited ]

I'm an independent consultant, so my way of working is a bit different from internal teams.

 

I track all experiments in Trello, with different lists that correspond with the stage of the experiment (eg, ideas, to-do, in development, in review, live, etc). I add things like screenshots, notes, results, lessons learned, etc.

 

The Trello board is shared with the clients so they can always check the status of things, pending items, and past results.

 

Once a week we review the Trello board and address any pending items, questions, or issues.

 

So I don't produce any fancy report document. I just keep the Trello board updated and easy to review, meaning don't fill it with unimportant content.

Level 2
Pauline 01-12-15
 

Re: Best Practices for Sharing Results Internally?

[ Edited ]

I run Optimization at Hotwire and I have different ways to community results and updates:

 

1) Bi-weekly emails with 3 highlights per email and a table with new experiments that were started in the past 2 weeks and new results. I try to keep it as short and concise as I can, but provide each experiment as a link to a wiki page with more details. This email is sent to a distribution dedicated to Optimization. People can sign up for it (currently 1/3 of the company is receiving my emails)

 

2) Update wiki whenever I start a new experiment or I get results. People can "follow me" on the wiki and receive emails when I post updates

 

3) Bi-weekly demos with product and UX teams: demo of the new A/B tests the team worked on + slides with results of A/B tests. I try to gamify this demo as much as I can

  • For new experiments, people have to guess which version is A or B and what is the difference
  • For new results, people have to do a hands up vote for winner vs loser

Hope this helps! 

Product Manager
Level 2
IanHo 04-21-15
 

Re: Best Practices for Sharing Results Internally?

Maybe because I'm in marketing, I always think about what's in it for them. What do they care about? That answer will depend on who you plan to send the email out to. The CMO will likely be interested in different stats then your backend developer.

If you plan to do a company wide blast of one email, then maybe just focus on a major company wide goal: revenue goals, number of new leads or user engagement goals

In our company, we have a company wide goal of increasing EPC or earnings-per-click. That's total revenue/# of visits to our website.

Here's one of the emails I sent out to my team. I sent out results for channel "y" which is 1 segment of our site's visitors.

-------
Hi all,

Here are some summary stats of the results of lead quality efforts filtered by channel y%. Although limited data, the rollout is continuing to look good.

Today 11/7 : 0-11am (vs. 10/31 : 0-11am)

4.16% Conversion Rate +43.94% increase
$45.25 rev/sale - with ZC revenue +9.19% increase
*important to note that this is rev/sale NOT rev/submit
$1.88 EPC – with ZC revenue +56.67% increase

If interested in my more in-depth analysis you can take a look here:
[link to my evernote was here]

Sarah Montgomery - Marketing Project Manager
Level 2