Why A/B Testing Isn't Enough

A/B testing is so last millennium. Search engine advertising provides a connection to the consuming public while they are in the driver's seat. The fluid nature of paid listings allows for constant refinement in how you make the directive listing connection.

Refinement in a search engine advertising program means changing and adapting click messaging elements in two short lines of text. Keywords are added and removed according to which product offerings or services line up with content and user intent. Positions are optimized, maximized and delivered in the most efficient manner possible.

The process refining the search experience for users also involves determining which text and graphic representations they prefer. The most common form of comparison is the A/B test in which one variable is compared to another. Another way to compare effectiveness includes the multivariate method but it is complicated and often perceived as prohibitively expensive.

Search and multivariate
The A/B test enables an advertiser to compare creatives, messages or landing pages. The test is relatively simple to administer and most often will provide a definitive answer as to what works better. Simply put, while it may prove what works better, an advertiser certainly won’t know what works best.

A multivariate test will enable comparison of multiple variables. In order to really provide an effective analysis of user preference, many considerations must be addressed. A true multi-variable analysis will allow a site owner to compare several page designs and messages simultaneously, ultimately lifting desired action activity beyond previously established benchmarks.

As you might imagine these results don't come easy or cheaply. You can't simply toss a bunch of variables in the air and hope for the best. A sound strategic implementation is essential and the multivariate initiative requires several commitments to be successful.

Yahoo!'s move forward
An industry has sprung up around multivariate testing due to the complicated nature of these initiatives and there are many providers that offer outsourced testing solutions. A little along the way couldn't hurt and last week, Yahoo announced strategic partnerships with two such providers: Offermatica and Optimost.

Yahoo evaluated nearly a dozen testing providers before deciding on two partners, although the company left the option open for future relationships. "Partners were selected on strength of offering and customer service criteria," says Dan Boberg, vice president of strategic alliances.

Since cost is almost always an issue and comparison shopping can be cumbersome, Yahoo negotiated best pricing, which usually accounts 25 percent discount on volume, though implementations will vary dramatically. Yahoo plans to be an advocate for clients in the set-up process and guarantees the best pricing on the market.

Successful conclusions
Yahoo has logged several case studies of successful tests. Among them, Motley Fool tested 13 variables, and found subscriptions rose more than 36 percent; Monster.com increased companion downloads more than 35 percent in less than two weeks.

Boberg says that most clients have seen a big enough pull for ongoing testing, making it part of the media budget going forward. So bear in mind that testing doesn’t have to be a one-time effort.

With the upside potential painfully clear, there are a few preparations to make before beginning the test. Costs vary greatly and you'll want to make sure the initiative pays for itself. Although you will want to execute tests quickly, you don't want to rush through selecting variables.

Approach ancillary benefits like targeting and customer data acquisition cautiously. Listen to those who have completed tests in the past while making decisions. Finally, know exactly what you want out of the initiative and define the boundaries and responsibilities of each party involved.

Winning combinations
We can use this wonderful tool called the internet to do amazing things for humans. Ironically, although technology created by people defines the web, it doesn't seem to interpret their wants and needs well. Multivariate testing enables content owners the ability to not only dig into likes and dislikes in messaging but also helps with targeting abilities.

Every step closer to accurately defining the need of the online interaction and how the consuming public may wish to interact with our interactive presence either online or elsewhere is another step toward creating the human-driven interactive utopia.