Online DM campaigns, such as direct mail lists and banner campaigns, may pose some nasty tricks along the way. But with every spooky challenge, there are valuable lessons to be gained.

Once upon a time, there was a direct marketing expert named Charlie who decided to ply his trade on the Internet.

“What better channel could there be?” he wondered. “I can get much faster feedback on results than I can through direct mail, and I can make changes to my program based on those results in ‘real time.’ This should be a real treat!”

Or was it a trick?

The first thing he tried was an email campaign.

“This should be just like renting direct mail lists,” Charlie thought.

Well, not exactly. The first test he ran was supposed to be simple. He chose two lists and decided to just test two messages on each list. He bought just enough inventory to provide statistically valid results. It was a simple 2x2 test grid - what could go wrong?

But once the program launched, there was almost no response.

“Don’t worry,” the vendor said. “Sometimes it takes people a few days to read their email.”

Sure enough, the results started to come in and they were quite good. Charlie decided to roll out the program, choosing the message and the list that worked best.

But when the rollout launched, again there was no response.

“Let me see what I can do,” the vendor said. “Maybe I can send it out to some more people.”

“But I thought I said to roll out to the whole list!” Charlie said.

As concerned as Charlie was over this “trick,” he was even more upset when he learned that the lists he bought might have violated a new spam law!

What can we learn from Charlie’s scary story? Since vendors hold the lists and manage the distribution of the message, they can manipulate the results by turning the “spigot” of mailings on and off.

They can also load up the initial test with the “hottest” addresses (the most recent or frequent responders) or they can “over-mail” a test to increase the likelihood of a rollout campaign. Most distressing, they can load up their lists with illegitimate email addresses or addresses for which they have no permission to mail.

The world of email list rental can be pretty spooky. Before venturing in, marketers should be sure to verify the source of the lists, ensure that mailings take place in exactly the quantities planned and that the buy is not loaded with “hot” names. Most of all, it’s essential to stay up to date on the latest spam laws to avoid legal action.

Surely there must be other “treats” for direct marketers, right?

Well, yes and no.

Consider the story of Linus, another direct marketing guru. Linus wanted to use banner advertising to test the effectiveness of his message. He came up with a list of variables worth testing including different site categories, sites within categories, offers and changing the creative colors and sizes. With all of these variables, he built himself a fairly robust test matrix allowing him to keep track of results across each variable.

Of course, one of the big “treats” of banner advertising is that, via third-party tracking, it’s easy to see performance in near real time. Within the first couple of days, Linus could already spot some trends. Now the challenge was, should he tweak the campaign based on this initial data or should he wait until he had more statistically valid results?

This is one of the “tricks” of banner advertising; optimization is so readily available that it’s easy to make a move prematurely. Some ads require multiple exposures before generating response, but once they get that response they may convert better than the quicker response ads.

Linus had another problem to contend with: His offer test consisted of a 10% discount and a 20% discount. Going in, he knew he’d need to see much better response from the 20% discount to pay for the increased offer cost. But he soon discovered he was getting almost no response on the 10% discount.

Another trick? You bet. Because he put both ads in full rotation, the same people were being exposed to the same offer. In traditional direct response programs, these offers would have been isolated and sent to different people, but online it’s very difficult to create isolated test cells.

The lesson? Work with a single vendor who can A/B split the creative. Or test offers that are different, but are of roughly equal value (such as free shipping versus a 10% discount).

After resisting the urge to micro manage his campaign, Linus later discovered that some of the vendors had significantly under-delivered the planned impressions while others had significantly over-delivered. This skewed the statistical significance of his test because the test cells were not all of equal size.

Further, some of the cells were too small to be considered valid. While different cell sizes wouldn’t make a difference in direct mail, this can pose a serious issue for a banner campaign because it affects the frequency with which people are exposed to the ads. As mentioned above, frequency can have an impact on response so a variance of frequency between cells can affect the results of the test.

After their respective experiences, Linus and Charlie felt like they were on the quest for the mythical great pumpkin. Is the concept of the Internet as a great direct marketing tool just a myth? Was it all just a trick? Or is there a moral to this story?

The moral is: Don’t let the tricks dissuade you from pursuing the treats. If you remain flexible and understand the differences versus traditional direct response, it is possible to gain valuable lessons and to drive positive results.

Michael Comins is the Corporate Vice President and Director of Media Services for NetPlus Marketing, responsible for managing the planning, implementation and optimization of online advertising and marketing campaigns of various sizes and scope. He has over 15 years of industry experience including stops at Digitas, HookMedia, Media First International, J. Walter Thompson, and Dailey & Associates. He is widely recognized as a leading media strategist and is respected for his aggressive-but-fair negotiating style.