4 critical steps for reliable email testing

  • Previous
  • 1 of 2
  • View as single page

There is a simple way to explain the importance of a testing plan: It's the only way to make the leap from what you thought would work to what you know will work. The information that testing generates will help you understand what's best for your customers in real-life situations and gauge how they are interacting with your brand. This is the foundation every email marketer needs for success.

For example, if you have a sale on iWidgets, which subject line is going to appeal to the majority of people looking to upgrade? Be careful. You don't want the greatest number of people -- just the most qualified people. It's easy to get everyone to open -- just say "Free iWidgets!" But that's no good because when people open the email and see that iWidgets start at $29.95, you could have an angry mob on your hands.

You need to know what's effective and to be able to prove it. Here are four key steps to help you get there.

1. Plan ahead
First, compile a list of questions you need to answer. What elements go into a great subject line? Should I use images? What is the prime placement for my content? No doubt when you brainstorm around these questions, you will come up with great ideas, and some will really push the boundaries of your strategy.

In addition, don't waste time building a long-term testing plan that people will veto when it comes to executing it. For example, you may want to find out if your customers respond best to a percentage discount or a dollar value -- but will your legal department allow you to go to market with two such different propositions? You may want to test elements of the splash pages to which your emails link -- but do you have the resources required to develop multiple splash pages? If there are other people in the equation, get their buy-in early on.

Here is another key point: Only ask questions if you are prepared to act once you uncover the answers. You could ask customers if they prefer to hear from you every week or once a year. But if the latter comes back as the choice, are you really prepared to build a strategy around mailing once every 12 months? Probably not. So don't waste your resources.

Here is the most important factor: Test one variable at a time and make absolutely certain you can isolate that variable. (Of course, this doesn't apply in multivariate testing, which is designed to test multiple variables, but that's a more complicated topic for another article.)

Here's an example: Acme Industries wants to find out if offering 1,000 Acme Points for its new iWidget product will entice people to open the email. So the company decides to test two subject lines:

1,000 Acme Points with our new Super iWidget XS
The new Super iWidget XS -- just $29.95

In this scenario, it's impossible to isolate the reasons for either line's success because there are too many variations. We couldn't tell whether it was the 1,000 points that had the impact, or mentioning the price. The test is only reliable if the two variations are as similar as possible, like this:

1,000 Acme Points with our new Super iWidget XS
Our new Super iWidget XS

Here "1,000 Acme Points" is the isolated variable, so if the top subject line is the winner, we'll know why customers opened the offer. This is valuable information when it's time to release iWidget 2.0.

Next page >>