Why A/B Testing Can Be a Waste of Time (And What to Do Instead)

A/B testing is useful. To a point.

For you?

Probably useless.

Here’s why, and what you should do instead.

When A/B Testing Doesn’t Make $en$e

The lure of a single A/B test skyrocketing conversions 10% overnight is powerful.

But it’s also mostly a myth.

The problem? Most fail. Or at the very least, fail to deliver significant benefits.

They also have a few inherent problems, such as:

• You’re testing your own assumptions. So allowing bias to distort your A/B tests is both common and frequent.
• They take a long time to see results (minimum of 3-4 weeks). That opportunity cost could be better spent somewhere else (like the three tips below).
• Results are going to be completely dependent on a large sample size. So it might take awhile to attain true statistical significance (see above).

But don’t take my word for it.

Peep Laja from ConversionXL weighs in on this topic too, suggesting that you’re better off just forgetting about A/B testing all together until you reach a specific number of monthly conversions.

And that magic number is…

1000. Minimum.

Wait, there’s more!

To put faith in the results, every individual test should have at least 250 conversions (minimum) too.

Simply put, that rules out A LOT of people reading this. Or it should.

But fear not, because you can STILL greatly increase low conversions. You just need to turn your attention towards capitalizing on the big wins that happen on your site.

Avoid the tunnel vision of A/B testing minutia, and focus on improving how people flow through your website, converting at each step, to raise sitewide conversions across the board.

Here’s how.

Step #1. Where are Your Visitors Coming From?

The quickest way to increase conversions is to understand who’s coming to your website, why they’re coming, and what they’re looking for.

Cool. How?

By starting with where they’re coming from and how they got there.

Sounds catchy and overcomplicated. That’s why consultants get paid the big bucks. 🙂 To tell you what you already know.

Fortunately, it’s pretty simple.

All you’re doing is piecing together bits of basic information from your site analytics to make assumptions or hypotheses about your web traffic.

Let’s start with the easiest thing you probably already know: traffic channels. Start by pulling up results for the past 30 days, and the past six months to a year to rule out any sudden spikes or changes.

In the example above, you’ll notice that A LOT of this website’s traffic is coming through Search, both Organic and Paid, and Direct.

(For now, let’s ignore Direct. The TL;DR version is that Direct tends to be overstated with visits that are actually from emails or social – while those are understated proportionally.)

The fact that this site receives a TON of visits through Search and Direct, but very little from other sources, begins to hint at how people are finding them.

But to really make any educated guesses, we need to see where this traffic is going. Because that tells you A LOT about who’s coming to these pages, and why.

Here’s how to find that.

Step #2. Where are Visitors Going, and Why?

In Google Analytics, go find your ‘Top Content’ to see which pages are the most popular on your site.

You can then add a secondary layer of context by seeing which traffic channels are sending people to those top pages.

You can also look at Google Analytics’ fun User Flow for a simple visual representation. Please note that this is hardly scientific. But at this stage you’re just looking for a push (or shove) in the right direction.

The details in the image are a little difficult to read without context (and I had to hide some sensitive client data), so I’ll extrapolate. When you compare the top traffic channels with the most popular starting pages on your site, you can begin to put people in buckets.

OK, so what does this all mean?

In this case, it looks like:

• There are a lot of visitors from Paid Search going directly to a campaign landing page. Makes sense.
• Otherwise, there are a lot of visitors going to the Home and other ‘branded’ pages (from both Direct and Search)

Not exactly a super-sleuth so far. Benedict Cumberbatch probably wouldn’t approve of my deduction skills.

But this begins to give you enough basic information so far to see how your site should evolve or change to increase the amount of people opting-in (or purchasing).

The objective is to do a better job of giving these people what they want.

Here’s how to find that out.

Step #3. What Should You Show these Visitors?

HubSpot has popularized the S-out of the Buyer’s Journey (in a good way). It looks something like:

Essentially, you have:

• Awareness: New peeps who don’t know who you are, but like something you’re saying.
• Information: Peeps who are digging your stuff, and beginning to understand why they might need what you offer.
• Evaluation: Those who’re actively researching and comparing alternatives for solutions to their problems.
• Purchase: Obvi!

The people coming to your website fall in one of these various buckets. Ideally, you’d be able to tell which ones are falling under Awareness vs. Evaluation, so that you could react accordingly to give them different info.

The key to figuring out which stage they’re in, is to look at where they’re coming from.

Google’s Customer Journey to Online Purchase Path tool can help you figure this out.

For example, B2B customers for small-ish companies (under 10k monthly transactions) are influenced by the following channels:

Display ads, Organic Search, Generic Paid Search and Referrals are the left-most ‘assisting’ options, so they’re going to coincide with stuff that happens earlier in the ‘Buyer’s Journey’.

Whereas Email, Direct, and Branded Paid Search are most often the ‘last-touch’ interaction, which means these peeps are commonly closer to purchasing when they arrive at your site.

Awesome. Now you have a place to start.

Step #4. Redesign Pages & CTAs to Target these Visitors

Inbound marketing is all about showing the right stuff to the right people at the right time.

Sounds nice in theory, but how do you do it?

Using all of the research we’ve learned so far, you can begin to update how people find certain things on your website, and then flow through these offers to eventually become customers.

Here are a few examples.

Top of the Funnel Examples

Let’s say that you have a high percentage of organic search visitors going straight to blog content first that focuses on some long-tail content.

Based on what we’ve looked at so far, those people are most likely at the top of the funnel, lacking need awareness for your products and services at this time.

It makes no sense then to hit them up with a direct product or service related CTA. Instead, you should appeal to what brought them there; providing information and education related to the original problem they set out to solve.

For example, on a recent Unbounce blog post about email marketing, they feature this interesting, interactive inline call-to-action that’s aligned with solving email marketing for customers.

You’ll even notice a similar example on another blog, where email subscription and webinar CTA’s are used primarily on blog posts (again – instead of having ‘Free Trial’ plastered on the sidebar).

Middle of the Funnel Examples

The middle of the funnel is that critical bridge between getting people who are initially interested in a topic or problem, to find out more about why they need you, and how you can help them.

It’s challenging because it’s a delicate balancing act, trying to educate while also demonstrating how they can (no, should!) be doing better.

WordStream does this brilliantly with their AdWords grader, featured directly on their homepage as the primary CTA:

Try to leave or go somewhere else, and their exit intent pop-up reiterates the same messaging one last time to get you to click. (You can even combine this approach with Facebook custom audiences to retarget visitors with the same message or value prop.)

Once people have navigated from the homepage into some of the more ‘branded’ pages (like Tour, Features, Services, etc.), you can begin to alter the messaging to be a little more explicit and direct.

Bottom of the Funnel Examples

When people finally hit the bottom of the funnel, you know they’re ready to take action.

Earlier we saw this, where a lot of visitors were going to the website we looked at through organic search and arriving at the ‘branded’ pages. When looking at Google’s Customer Journey tool, we confirmed that these visitors using ‘branded search’ to arrive at a business website are generally at the ‘Decision Stage’ (in HubSpot parlance), most likely evaluating their options, looking for pricing, and ready to start a free trial or get on the phone.

Cool. Now let’s do something about it.

If possible, try to make the conventional a little more unconventional with the approach. For example, my agency’s latest site redesign tried to be a little more playful with a standard form CTA at the bottom of a page:

Not exactly earth shattering, but it effectively stands out from many similar options on other competitive websites (which people might be bouncing back-and-forth between at this stage in the game).

Even better though, is if you can give people a way to ‘test drive’ or sample what you might have at this stage to show them exactly what they’re going to get.

For example, United Material Handling allows potential prospects to actively search their inventory to receive both real-time quantity and pricing information.

That’s different and unique in their competitive, local B2B environment.

You can even validate (or disprove) your designs based on real user behavior and feedback. For example, simple, inexpensive options like CrazyEgg can immediately tell you if people are using this new brilliant option you dreamed up.

So far so good! The majority of clicks and page interactions are happening here on our interactive CTA.

Now compare that to the red button in the footer:

Yikes, it’s a ghost town. Time to try a different value proposition here and redesign!

Conclusion

A/B testing sounds awesome in theory.

Only trouble is, it’s often a time consuming, frustrating process that doesn’t always yield results.

A more predictable (albeit, boring) way to improve conversions is to continually monitor who’s coming to your website, why, and try to update it accordingly.

With a little bit of research, aligning the right message to the right visitor at the right time becomes exponentially easier.

Then you can hypothesize and test different value propositions or messaging based on real user behavior and feedback.

Focusing on sitewide conversion design, instead of obsessing over tiny elements on individual landing pages, gives you better odds at turning strangers into leads, and leads into satisfied customers.

Drill down, but don’t forget about the big picture either.

About the Author: Brad Smith is the founder of Codeless, a B2B content creation company. Frequent contributor to Kissmetrics, Unbounce, WordStream, AdEspresso, Search Engine Journal, Autopilot, and more.