A sequential test offers valuable insights such as helping you decide whether more extensive testing is warranted.

Faced with the time and expense involved  in conducting A/B split tests or multivariate tests, many companies choose not to test their webpages at all. Why not start with a sequential test?

A sequential test is when you put up version A of a page for one week, then replace it with version B for the following week. The duration need not be a week. It could be a few hours or a day or two.

How long you leave each version up will depend on the volume of traffic your pages receive. You want both A and B up there long enough to get enough traffic for some reliable results.

The testing experts reading this will likely now cry, "Foul!" They will tell you that pages tested over different periods of time are subject to validity errors due to potential outside influences. This is among a class of validation threats collectively referred to as history effects.

This is true. A sequential test is more vulnerable to validity errors than an A/B or multivariate test.

However, a sequential test can still offer valuable insights. It can, for instance, help you decide whether more extensive testing is warranted. 

Here are the results of a sequential test conducted recently for a discount health care provider.

The short page copy performed 14 percent better than the long page copy.

In the past, we have generally found that long copy out-pulls short copy for many sites. But as the above results suggest, this does not always hold true for every business.

However -- and here comes the history effect -- there were other factors in the second testing period that may have caused the long copy to perform poorly by comparison. The company in question made some changes to their PPC campaign during the second testing period. This may have altered quality of traffic sent to their site.

In other words, this sequential test gave us an indication of the relative performance of these two pages. But we may now want to follow up with an A/B split test to verify the findings.

So, when should sequential testing be used? Any time that a more sophisticated test is impractical for your schedule or your budget. Just remember that, as with any test, you should minimize the sources of validation threats; and for sequential testing, that is the history effect. That means: keep it brief, monitor for external factors and limit any changes unrelated to the test. Find out more about the methodology and benefits of testing through our Professional Certification in Online Testing course.

The Marketing Experiments Journal publishes primary test results from work with our research partners once every two weeks. Subscription to the Journal is free and gives you full access to both our archives and teleconference calls. Subscribe here.