7 steps to improve email testing

Last month, I talked about the importance of personalization and design. While these are critical topics, they both hinge on this week's article: the importance of testing.

Without proper testing, there is no way to tell what your customers are actually seeing or how they are responding to your message. This article will provide practical tips for implementing a successful testing process for your customer email.

Before you start to test anything, you must first establish a comprehensive plan that outlines what you want to test and what you will consider a success or a failure. When putting together this plan, make sure you consider all factors, whether they're driven internally or by customer behavior. For example, will a successful test be dependent on more opens, higher clickthroughs or increased purchases?

Without determining the key metrics for a successful test before performing it, you will be unable to determine which message performs better.

In order to properly test different subject lines or content, you need to verify that your customers are receiving your email and that it is rendering properly, because if they don't receive your email, nothing else matters. To do this, there are a number of valuable tools that you'll want to become familiar with, including a campaign preview tool, an anti-spam testing tool and a seedlisting tool to see if your messages make it to the inbox or junk folder.

These tools should be used as part of your regular campaign process, both before and during your send. However, to avoid confusion, make sure you fully understand how to read the reports before using them. 

The following tips will help you get started with your testing program.

Test to see how your message renders
Once your message is fully created and coded, the next thing you want to do is test it in a campaign preview tool. These tools allow you to see how your message will appear in the various email clients. If any issues arise during this test (image rendering, formatting, content blocks, etc.), you then should make the necessary changes and test it again. Without this form of testing, you run the risk of confusing your customer or missing an opportunity to communicate key pieces of information that can result in increased sales and customer satisfaction.

Test to see where your messages are placed
If your message doesn't make it to the inbox, you are missing out on potential revenue. Plus, having your email land in the junk folder is also damaging to your brand. If you don't test inbox placement during every campaign, you run the risk of doing the same thing over and over, resulting in multiple ineffective campaigns.

Test your content
Content is one of the most fundamental and overlooked aspects of testing. By making easy and small changes to your content, like changing one word in the subject line or highlighting a product feature, you can increase clicks and sales dramatically.

Ideally, A/B and multivariate testing capabilities will be embedded into your email platform. If not, you'll want to find a solution that natively supports these critical testing functions. When testing campaigns with different content, you'll want to start by changing one element at a time. This allows you to more easily see what factor is having a direct impact on your email response rate. An easy and effective way to do this is to pull two randomized samples from your entire list. Depending on the size of your list, there are two common ways to perform A/B split tests.

A/B testing
For smaller lists, it is common to split the entire list into two randomized groups and send one version of the email to each of the groups and measure which group had a better response rate for opens, clicks or revenue generated. Next, apply those findings to future mailings. For larger lists where statistical significance is not a concern, many marketers use a 10/10/80 split, but you could also select an exact number of recipients for each test mailing. This method allows you to determine the best performing email and then send it to the remainder of the list for maximum impact. 

Multivariate testing
There is also value in testing a carefully selected combination of factors so that you can determine if any of the test elements interact with each other positively or negatively. There are cases where two best performing elements from earlier tests demonstrate poor performance when combined. However, before you embark on a multivariate testing program, you should recognize that multivariate testing is much more complicated than A/B split testing and requires advanced planning, stringent adherence to the test plan and an extra level of sophistication to interpret the results.

It is recommended that you start your testing program with simple A/B tests. Once you have fortified a testing process and discovered some easy ways that can improve the performance of your mailings, you can graduate to advanced A/B testing, finally expanding into multivariate testing.

Learn from your results
Be sure to track and document your testing results and share information on what works with key stakeholders involved with the strategy, design and promotional aspects of your campaign. Having an established process for sharing result data will help ensure that you apply the information appropriately to maximize success and avoid similar mistakes in the future.

Test continually
Every company has its busy season; for retailers it's the time between October and January, for others it might be the back-to-school season. Either way, it is important to test as much as possible before your busy season.  That way, when the critical time arrives, you can take advantage of your most effective messaging based on what you've already learned about your customers and what works for them.

So remember, don't be afraid to test anything -- test everything. By understanding your customers better, you will send them more relevant messages, which will increase your ROI.

If you don't have a proper testing plan, or the technology to support it, use these simple tips to start building with one today. Good luck and good sending.

Spencer Kollas is director of delivery service for StrongMail Systems. Read full bio.

 

Comments

Emerson Hartley
Emerson Hartley August 29, 2007 at 10:39 AM

You may want to test your email creative for different target segments and synch up with different landing pages, rather than a one size fits all approach. I was at the eMetrics show in San Francisco and Memetrics said they can help do that.

Jason Gindele
Jason Gindele August 27, 2007 at 10:57 AM

There is a nice application at www.emailexam.com that tests lots of these things, including rendering, blacklists, inbox delivery, broken links, etc.