Strategies for Success: A/B Testing Key Principles

Every business wants to grow and become more profitable year after year. But many don’t realise growth can come from optimising existing traffic an following some AB testing key principles.

Instead, when companies look for more customers they usually turn to increasing digital marketing spend to acquire more traffic and hope conversion remains the same.

This tactic only works if you already have a high converting site. Rather than growing traffic (which introduces extra cost) you should be focussed on running a successful AB testing plan.

Here’s why you need to start testing.

Key principles of AB testing

Before you become an expert, you’re going to need to understand the basic principles of what makes a successful AB testing strategy. For me these are:

Resources – Making sure you have the right tools in place.
Insight - Becoming obsessed with data and customer feedback.
Process - Following 6 repeatable steps.
Results - Sharing learning far and wide.

Let's break these down in to more detail:

1 - Resources

Since we’re talking about AB testing specifically here, you first need decide which testing tool is best for your business. When I’ve been shopping for testing tools in the past I tend to ask these simple questions:

• How easy is this tool going to be to implement and use?
• How will data/results be collected and will they integrate with my existing analytics software?
• How much is this going to cost me each year?

Good place to start out – Monetate, Optimizely & Webtrends Optimize. All of these companies have very simple UIs and are more than capable of handling the complex tests when you’re ready. As an added bonus, personalised campaigns can also be delivered - but we'll save that for another blog post.

If you’ve been in the game for a while then you’re probably going to want to look at – Adobe Target, Coveo & Maxymiser.

So, you’ve got an idea of what to look for in a testing tool. The next step is to start thinking about where your inspiration will come from for generating test ideas.

Your answer lies in reviewing your customer data and feedback. I’ll cover this in the next section but here are a few ways in which you can gather this invaluable information.

1. Get in touch directly with your customers - create an automated email campaign that gets sent after every purchase. Linking through to a feedback form or survey.
2. Feedback forms – Ever seen pop-ups on sites asking for your feedback? This is a great way to capture lots of information relatively quickly. Just make sure not to display this every time the same customer comes back to the site.
3. Surveys – SurveyMonkey comes highly recommended and is highly customisable. The key here is to keep it short and easy to complete by mostly using multiple choice questions. Don’t forget a text field for added insight.

All of the methods above are a great way to capture this insight.

2 - Insight

It's one thing being able to collect data. It's another skill all together being able to turn that data in to actionable insight.

A good place to start here is finding out what customers are saying about your site, journey or product by presenting a simple feedback form. Or turn to your website analytics platform to see where customers are spending the most (or least) amount of time.

When in logged in to your analtytics, start by looking at your most visited landing pages - as the more visits a page is receiving, the quicker it will be to generate a statistically significant test result.

Landing page tests are great ways to understand how you can entice customers to enter the top of your funnel or journey. But if you really want to boost conversion rates, you need to find out where customers are exiting your site and coming up with creative ways to keep customers in the flow.

3 - Process

1. Understand current customer and business challenges, use insight to create solid hypotheses.
2. Capture test ideas from your closest stakeholders or customer facing teams, they usually have a great idea of the current problems being faced.
3. Prioritise which test you're going to take forward and build - I like to keep things simple and use the tried and tested effort vs impact scale (more on this in future posts).
4. Develop your test within your chosen testing tool, and preview the experience across different devices, so you know everything is going to work as expected.
5. Launch the test - depending on how many variants your are running will detremine how long to run this for. But the key is to make sure you leave it running for enough time to reach a high confidence and statistically significant result.
6. Analyse the results and share far and wide - don't over complicate this, export the data from your testing tool and drop it in to a Control vs Variant table. Adding in the main KPIs you were tracking and comapre the percentage uplift of each metric vs control. Share your findings by following the format below.

4 - Sharing Results

Time to talk about one of the most important parts of AB testing. Results sharing.

Doing this in the right way will not only allow you to validate your original hypothesis (why you ran this test), but also give you an opportunity to create a knowledge bank of previous learnings.

Every time you have a statistically significant test result, whether that be positive or negative you need to have one aim:

Inspire others to want to get involved with your testing programme.

How do you do this? Simple. An email sent to everyone (yes everyone) in your business outlining what you tested and how your key metrics changed. I've always found, that the jauntier your update, the better, there’s a lot of emails sent/received in a day and this one needs to stand out.

Here’s an example of how to share test results over email.

Subject: (Make this is as intriguing as possible)

Hi Team -

We’ve recently concluded a test on the homepage after being able to generate a statisically significant test result, resulting in +15% improvement in conversion.

Who did we target?
(Short description of what type of customers were exposed to the test)

What did we test?
(Give a visual summary and short explanation of what what you changed vs control - A vs B)

Why did we run the test?
(This section is best reserved for your hypothesis)

Key Results
(Include a concise view of how your KPIs were affected vs control and how confident you are in your results)

Sign-off
(Be sure to thank everyone involved in making the test a success, from your dev team to designer, and don't forget to encourage your recipients to reach out with additional test ideas so you can to your backlog.)

An email like the one above is an effective way to share a ‘short but sweet’ set of results, but you’re going to need more detail to really demonstrate you know what you’re talking about.

You’re going to need to create a highly detailed summary report, which explains what you tested, why you tested and the full results beyond your headline metric changes - be sure to sign up to my newsletter to find out exactly how to do this.

Conclusion

There’s been a lot to take in here. Each of the 4 principles could have easily become a separate blog post I plan to expand on each point over time.

As an added bonus, I've always found these personal qualities helpful when it comes to increasing conversion.

Ask yourself if these sound like you?

Be curious - continuously ask questions, especially why.
Be critical - never settle, everything can be improved.
Be creative - solving problems for your customers requires an open mind to finding the right solution.

Lastly, you should always share your test results. Regardless of whether or not the test had a positive or negative impact on your main KPI. Trust me, everyone will be interested in finding out what you’ve learnt.

Like what you read? Sign up below for more content like this.