A/B testing is a fantastic method for figuring out the best online promotional and marketing strategies for your business. It can be used to test everything from website copy to sales emails to search ads. And the advantages A/B testing provide are enough to offset the additional time it takes.
Well-planned A/B testing can make a huge difference in the effectiveness of your marketing efforts. Narrowing down the most effective elements of a promotion, and then combining them, can obviously make your marketing efforts much more profitable and successful.
How Do You Plan an A/B Test?
The first thing to do when planning an A/B test is to figure out what you want to test. Are you running an on-site test, or an off-site test? If you’re running an on-site test, you’ll want to think of all the sales-related pieces of your website, and then figure out which elements you want to test.
With off-site tests, you’re probably testing either an ad, or a sales email. Testing ad copy to see which ad brings in more converting visitors can help you focus your advertising efforts. Once you know your ad is converting as well as possible, it’s easier to justify spending more money on it. The same goes for emails. If you send out two versions to your list (randomly selecting which half gets which email), and then track which one converts better, you can send only that version the next time.
Once you know what you’ll test, make a list of all the variables you’ll test. For example, if you’ve decided to test your call to action, you might test:
- the location of the call to action
- the exact text used
- the button color or surrounding space
It’s a process, and it’s common for multiple A/B tests to be carried out prior to making a final decision or final change.
Make sure that before you start testing you have a clear idea of the results you’re looking for. You should already know your baseline result, which is the results you’re currently getting. You want to test option A and B against each other, but you also want to know that whichever one does better in the test is also doing better than your current results. Alternatively, you can use A as your control (leaving it whatever you’re currently using) and then use something new for B.
Tests need to be run simultaneously to account for any variations in timing. You can’t test one variation today and the other one tomorrow, because you can’t factor in any variables that might have changed between today and tomorrow. Instead, you need to split the traffic seeing your variations at the same time.
A/B Testing Can Improve Your Bottom Line
Accurate A/B tests can make a huge difference to your bottom line. By using controlled tests and gathering empirical data, you can figure out exactly which marketing strategies work best for your company and your product. When you figure that one variation might work two, three, or even four times better than another, the idea that you would conduct promotions without testing starts to seem a bit ludicrous.
A/B testing, done consistently, can improve your bottom line substantially. If you know what works and what doesn’t, and have evidence to back it up, it’s easier to make decisions, and you can often craft more effective marketing materials from the outset. Just remember to keep testing regularly, since the effectiveness of anything can change over time.
Things to Test
You can test virtually anything in your marketing materials: headlines, calls to action, body copy, images, etc. If you can change it, you can test it. But that doesn’t mean you should necessarily spend months testing every little thing. Instead, focus on the things that are most likely to have a big impact.
On your website, this likely includes:
- the headline
- your call to action
- any graphic you use in direct correlation to your sales efforts
- the sales copy or product descriptions
In an email, it probably includes the same. In an ad, especially a text ad (like a search ad), you have fewer things to change, and so likely you’ll test either the main headline or the offer itself.
Testing different offers is also important. Just make sure that you have methods in place to ensure that each person is always offered the same promotion. For example, if a free gift is offered to group A, and a discount is offered to group B, then you want to make sure that group A always contains the same visitors, as does group B.
You can also test things in conjunction with each other. For example, you might want to test newsletter A with landing page A, and newsletter B with landing page B. And then, later, you may want to test newsletter A with landing page B, and vice versa. This can give you a more concrete result if you’re getting mixed results, or if your results are very close.
Devoting Time to Testing
A/B testing is not an overnight project. Depending on the amount of traffic you get, you might want to run tests for anywhere from a few days to a couple of weeks. And you’ll only want to run one test at a time for the most accurate results.
Giving a test insufficient time can mean skewed results, as you don’t get a large enough group of visitors to be statistically accurate. Running a test for too long can also give skewed results, though, since there are more variables you can’t control over a longer period. Make sure that you stay abreast of anything that might effect your test results, so that you can account for any statistic anomalies when reviewing your results. If you’re in doubt, it’s perfectly reasonable to retest.
Considering the impact A/B testing can have on your bottom line, though, it’s worth taking a few weeks to properly conduct tests. Test one variable at a time, and give each test sufficient time to run.
Can I Test More Than One Thing At a Time?
There are two approaches to this question. First of all, let’s say you just want to test your headline, but you have three possible variations. In that case, running a single test and splitting your visitors (or recipients in the case of an email) into three groups instead of two is reasonable, and would likely still be considered an A/B test. This is more efficient than running three separate tests (A vs. B, B vs. C, and A vs. C). You may want to give your test an extra couple of days to run, so that you still have enough results to base any conclusions on.
Testing more than one thing at a time, such as headline and call to action, is a multi-variate test, and is more complicated to run. There are plenty of resources out there for multi-variate testing, but we won’t be covering that when talking about A/B testing.
If you’re looking to get started with A/B testing, you can start by learning how to a/b test in Google Analytics.
About the Author: Cameron Chapman is a freelance designer, blogger, and the author of Internet Famous: A Practical Guide to Becoming an Online Celebrity.
Comments (20)