A Quick Primer: Using A/B Split Testing to Improve Website Performance
A/B split testing is perhaps the most potent data gathering tool available for any form of marketing research – and it’s astoundingly easy to make use of online. Every single site performance and analytics suite on the market is equipped to maximize the use of split testing.
Unlike other forms of data collection, the information retrieved from an A/B split test is straightforward and typically lends itself to immediate improvements to your site. Let’s look together at what to test, how to test it, and how to interpret the data. We’ll also cover a few basic tips for optimizing your outcomes and a few pitfalls to avoid.
The Split-Testing Process
Before diving into what you should be testing, let’s go through a quick primer on what exactly split testing is and how these types of experiments are carried out.
What is Split Testing?
In a split test, in the most basic terms, you create a test variation of a page on your site. Then, you send half of your visitors to your original page version and the other half to the test variation you created. The page that performs better (according to whatever metric you choose) can be considered the winner.
Here’s a quick illustration of the process in practice:
Imagine that you’re a lawyer and that you use your website to generate new leads for your practice. You have an opt-in form on the homepage of your site, under the heading “Get a Free Consultation,” but you want to test whether or not this wording is as effective as it could be in helping you to capture new leads.
So you keep your original homepage (Variation A) and you create a test version of your home page that, instead, uses the headline “Texas’s Leading Bankruptcy Attorney” (Variation B). Using a split testing program (which we’ll talk about next), you deploy both versions, splitting your traffic equally between the two.
Once the test is complete, you find that Variation A – your original page – experienced a 29% conversion rate, while your test variation received only a 13% conversion rate in terms of leads generated. As a result, you reject your test headline and begin a new test with another variation.
Setting Up a Test
Despite how complicated the test described above may seem, split testing is actually quite simple to set up – but you’ll need some sort of analytics solution in place beforehand. After all, there’s little point in diverting visitors to two versions of the same page if you’re gathering data from neither.
Google Analytics’s Content Experiments tool is the easiest way to get split testing immediately; though more robust solutions can be found in programs like Hubspot and Kissmetrics.
To set up a test (once Google Analytics has been installed on your site), you’ll choose an experiment objective:
Image Source: Google
Configure your experiment by identifying your original and test variation pages (which you’ll need to create in advance of launching your experiment):
Image Source: Google
Place the experiment code on your original page to redirect part of your traffic:
Image Source: Google
And, finally, launch your experiment:
Image Source: Google
As your test runs, your split testing program will gather data and display your results. Be sure to run your experiment until your program announces that you’ve found a statistically significant winner. Calling your experiment off too quickly can result in decisions that are based on a too-small sample size to accurately predict future performance.
Image Source: Google
Image Source: Google
What Should You Be Testing?
Split testing only gives you as much information as you seek from it; in other words, for A/B split testing to be worth your time, you need to be testing the right things.
There are a nearly endless number of variables that could ultimately impact your website’s conversion rates and other stats, but from those countless variables, we’ve identified a few that are likely to have the greatest impact on your site’s performance.
Your Copy
The actual written content of your site has a huge impact on your conversion rate, so it’s in your best interests to make sure you’re optimizing it properly.
This means testing broader areas of interest compared to highly specific ones. Broader areas of copy worth split testing, for example, include things such as comparing long-form vs. short-form landing pages or different voices and writing levels against one another.
You’ll also want to split test specific areas of your copy, such as your headers, the first sentences in paragraphs and the like.
Your Calls to Action (CTAs)
Different calls to action work very differently with different audiences, so make sure you split test thoroughly to identify the approach or approaches best suited to your visitors.
When split testing CTAs, don’t do too much at once; make sure you’re testing different actions against different actions and different phrasings against different phrasing. If you change your copy extensively and change the action you’re calling for, you won’t know which aspect of the change mattered more.
Your Graphic Design
Trying different graphics, different fonts, different text sizes, different colors, and different layouts should be a major aspect of your split testing experiments.
It’s important to note, however, that while this is only a single category on this list, it should constitute a variety of your tests. Don’t compare two wildly different pages. Instead, test largely similar pages with specific differences you want to test.
Your Navigation
Depending on what exactly you’re trying to do with your site, this can be a little tricky to split test for, simply due to mechanics of how split-testing works.
In some cases, split testing navigation will mean constructing the better part of two sites. Thus, while split testing navigation can be quite useful, it is perhaps best delayed until you’ve nailed down the other factors between your current site and its maximized state.
Measuring Your Results
Anytime you want to improve something based upon data, you need to first define what you consider to be an ‘improvement’. In what way do you want to improve the performance of your site?
- Do you want to convert a higher proportion of visitors?
- Do you want to nurture guests so they’ll be better predisposed towards your brand, or perhaps develop them into higher-value leads than they would naturally be?
- Do you simply want more eyes on your content?
Your goals will depend on your business model, your business’s stage of growth, and myriad other factors. Make sure you take the time to establish your highest priority before you begin split testing, lest you waste time and effort on irrelevant or counterproductive tests and improvements.
Notable metrics vary between businesses, goals, industries, etc. Fortunately, you’ll be able to analyze the same raw data from whatever angles you need to – just make sure you take the time to figure out the correct metrics.
Regardless of your goals, you’ll want to configure your analytics suite to present you with ‘big picture’ metrics, like conversions per marketing dollar; the data that tells you which split makes you more money.
Taking Your Testing to the Next Level
Once you’ve run a test or two, use the guidance below to “level up” the way you use split testing to improve your website’s performance.
Testing with Low Traffic
Under most circumstances, your capacity for split testing will be limited by your traffic.
We touched on this above with the issue of statistical significance, but let’s look at it with an extreme example:
If ten people click a link, you can’t split those ten people to ten different pages and get anything useful. You simply can’t know if your results will apply to the next 10,000 people who come to your site because their limited experiences aren’t likely to be representative of a larger population.
To make low-traffic testing work, be patient and run one test at a time. Focus on bigger tests that’ll produce results with larger significance – a bit of research into basic statistics may be worth your time here, but common sense can do the job if you maintain the necessary skepticism.
Ultimately, there’s not much need or advantage to splitting hairs when your traffic stays low. At traffic levels where split testing becomes unreliable, your primary goal should be to produce more traffic, after all.
Thinking Outside of the Box
While the topics mentioned earlier in this primer are the most consistent and reliable areas to split test, we’d be remiss to ignore the importance of keeping an open mind.
Often, some minor detail you’re overlooking plays a crucial role in site performance. By keeping a flexible mind, paying attention to what customers have to say, and experimenting continuously, you’ll be far more likely to find these aspects of your site ripe for split testing.
Understanding the ‘Why’ of Your Results
Don’t let split testing push you to diversify into unnecessary directions, and don’t let it convince you to cut off channels and approaches you shouldn’t.
Determine why you’re getting the results you are from split testing. You may discover that rather than your split tests revealing a ‘better’ way to arrange your site, they’ve revealed an ‘alternative’ – improving results with one subset of visitors while worsening them with another.
Always consider your bottom line. If a particular change improves conversions on that specific page, but it’s pushing lower-value leads through compared to the previous version, you may come out worse with the ‘better’ result. Consider your test results holistically to get the most out of A/B split testing.
A/B split testing can play a critical role in the development of your website, but it’s still just one tool in your analytics toolbox. Without the support of a broader understanding of your market, your visitors, and the data you’re handling, it’s easy to fall into reactionary patterns.
Keep your head, think about the further implications of your tests, and you’ll be far more likely to achieve the best possible results from your site.
Are you using split testing to improve your website’s performance? If so, share any other tips and tricks you’ve picked up by leaving us a comment below:
