When testing headlines, A/B split testing can have a considerable impact on conversion rates.
There is a belief among many people in the online industry that A/B split testing can be used only to measure a single difference between two versions of a web page.
That's not so. In truth, almost every test includes more than one change between versions.
Consider a headline test. One headline versus the other. Each of them 10 words long. Are you testing just one change? It's most unlikely. There are probably a number of words that differ between the two headlines, not just one.
A single A/B split test may not identify the word, or group of words, that actually delivered the greatest increase or fall in conversion rates.
But it doesn't matter, so long as the results are valid and useful.
The same goes for a test we ran recently for an online publisher. We ran an A/B split test between two pages in an eight-day micro-test.
- Page A had text in two columns of copy.
- Page B combined the text into a single column.
Was that a single change being tested? No. By changing the layout we had changed the flow of the copy, the design and width of the page, the length of the page and more.
Nevertheless, the results were useful.
Conversion of Page B (one column) was 88 percent better than that of Page A (two columns).
As you can see, what we learned from this simple, single A/B split test proved extremely valuable to our research partner.
In addition to allowing you to test two, three, four or more very different versions of a page, A/B Split Testing offers the additional benefit of being fast and reliable.
So while multivariable testing has its place, don't assume that you can't use A/B split testing just because you are making more than one change to a page.
Find out more about the methodology and benefits of A/B split testing in our Professional Certification in Online Testing course. The Marketing Experiments Journal publishes primary test results from work with our research partners once every two weeks. Subscription to the Journal is free and gives you full access to both our archives and teleconference calls. Subscribe here.