Learn how to test to find the best creative: It can increase performance from 10 to 25 percent.

How many times have you heard this when approving a creative direction: "I think that most people will respond better to this one. Let's go with it."

We, as the agency, do our research on the front-end of the creative process to determine who the target is -- what they think, feel and believe -- and, what they will respond to. The client has their experience with their product or service, and is therefore typically very knowledgeable and quite qualified to make the final assessment about the creative product. However, all this background typically only guides us in very general directions. 

Things that we don't know -- even with all that front-end research and experience -- include the significant details that will make one piece of creative perform 10 to 25 percent better than another. That is what testing is all about: squeezing an additional few percentage points' response improvement from each creative piece.

This is an age-old model that has been used in direct mail, catalogs, retail and other traditional businesses. It is no surprise that this method is now being applied to the Web, with its always-on, real-time fundamentals.

A/B testing

A/B testing is a classic model of testing against a control. Simply put, one basic element in a piece of creative is changed to enable an apples-to-apples test. Alternate versions of the same creative with that one element changed are produced and tested to determine which performs the best under normal circumstances. The most effective creative then becomes the control to insert back into the cycle.

You can perform this testing model on just about anything. Important things to bear in mind are to keep the tests simple and identify all of the influencing factors that may disrupt an accurate test.

Tools of the trade

Site analytics tools enable traffic data to be sliced and diced in virtually any way desired. Software such as WebSideStory’s HBX can track path information, conversion funnels, geographic information and campaign ROI. 

These specific metrics provide insight beyond the initial click-through and dig more deeply into users' paths, providing information on where prospects are located and what they do on the site. Previously, with more direct marketing-style tactics, traditional researchers might have believed that a campaign was successful because they received more individual responses from one campaign over another. However, today we can identify which campaign or homepage will result in different types of long-term activity. The software can identify users' visit frequency and time spent on site, which may be even more important than the number of responses or click-through, acquisition and conversion alone.

When coupled with site analytics, an ad server can provide an easy means to tweak creative to capitalize on which campaigns work best for which audiences. Ad serving tools enable different ads to be served based on demographics, locations, etc., as well as by varying frequency, allocations and sizes. 

There are different types of ad servers, many of which are home-grown, but a primary benefit is to make it easy to test one piece of creative against another.

Between site analytics and ad serving, we have a solid platform on which to test our hypotheses.

Hypothesis

Though oft-overlooked, this is an absolute requirement in testing. Forming a hypothesis involves predicting what we think is going to happen in order to have something to test against. A hypothesis will be the primary source for developing our control, as well as for finalizing our test factors.

To test creative it should first be broken into components. Those components are often as simple as a word, a color or an image. We can also test by location, time of day, referral (the site that someone clicked from to get to ours), demographic, offer, etc.

Sample size

It is important to identify how many users need to see the creative to develop a good test sample. The number of users that should test often depends on the environment you are testing in. Often, we can see trends very quickly, but many times it may require some further segmentation to determine clear winners.

One thing to note is that it may be important to challenge the findings depending on whether or not the test ran into other variables that would influence the outcome. In other words, in order to get an adequate sample, your creative might have to run in multiple geographic locations or during different dayparts. These factors should be considered as influencers and be culled from the pack to determine if they have truly altered the results.

Is creative testing for you?

This type of testing can be extremely time-consuming as well as a resource drain. The basic question to ask when determining whether or not to test is: What can we stand to gain from a 15 percent increase in conversions from our campaign? If the answer yields a significant ROI when factoring in the time and cost of implementation, then you should do it. 

Also note that 15 percent is not a fixed number. Some attempts at creative testing yield far greater results, as well as some have conversion rates that continue to grow year over year. However, 15 percent is a good, safe benchmark and goal for improvement.

For a campaign going out to 100,000 prospects, that’s nothing to balk at, especially if the respondents become customers. And given the investment already made in design, development, printing and other costs, it may be a small price to pay to test for such a big difference in return on investment.

While it may not be a good fit for every campaign, creative testing can and should be a major factor in others.

Reid Carr is president of Red Door Interactive, helping clients -- such as the San Diego Convention Center, SkinMedica, Benetrac and Sharp Systems of America -- to lay out strategies for their online presence and interaction activities. Before founding Red Door Interactive, Carr formed the interactive arm for the San Diego-based PR agency, McQuerterGroup. Prior to that, he was chief operating officer and accounts director at PBJ Digital, a bi-coastal Interactive development and incubator shop in Los Angeles; before then, Carr handled account management in both the Venice and Playa del Rey offices of TBWA/Chiat/Day. He has a BA from the University of Oregon's advertising program.