Help for Choosing ESPs

Earlier this month, JupiterResearch released its 4th annual Email Marketing Buyer's Guide, a detailed evaluation and ranking of 30 vendors. Jupiter evaluated each vendor in the categories of client references, product demo, financial stability, services expertise, application features such as usability and functionality, partnerships, and technology integration. The company then based overall value, suitability and breadth on the distinct needs of six types of marketers -- small business, service-oriented, low-volume promotional, low-volume newsletter, high-volume promotional and high-volume newsletter.

iMedia: Why does Jupiter do this report?

Jupiter: JupiterResearch clients look to us to help them make strategic buying decisions and email marketing vendor selection is one area that we get a lot of inquiries. The annual report itself is also a tremendous resource for the vendors themselves as it is the most comprehensive research on this market sector.

iMedia: Has the evaluation criteria changed over the years or stayed pretty much the same? How did you choose those criteria?

Jupiter: The evaluation criteria has expanded over the years and includes among other things an extensive vendor questionnaire and a demo scenario that requires the providers to assemble and send a mailing based on data and content assets that we provide them. We collected approximately 360 data points on each vendor. The criteria are largely based on the input of the email marketers themselves. We do a survey annually that collects vendor selection information from email marketers. This year we made the buyers guide persona based, creating a JupiterResearch evaluation constellation for six different email marketing personas, which was based on the different needs we have discovered in these surveys. 

iMedia: One of the key questions you ask in the report is how have ESPs' offerings matured during the past 12 months? Can you summarize those findings?

Jupiter: There were few revolutionary product developments as many ESPs still struggle to include the basic and mission-critical compliance features, such as testing and CAN-SPAM support, which were found in only 66 percent and 63 percent of the applications respectively. While some have made big strides in the area of application usability and adding interesting functionality such as more robust template support, there are still a good number of providers that have done little to improve their offerings over the past few years.

iMedia: What makes criteria for the six types of marketers so different? In other words, what are some key differentiators between what a small business marketer and a high-volume newsletter, for example, would need in a vendor?

Jupiter: Marketers have told us that they place different emphasis on things, for example small business newsletter marketers are more concerned about the providers' delivery tools and reputation where as high-volume business-to-consumer newsletter marketers put more emphasis on the vendors' strategic and tactical services. We are not suggesting that these things are not important to both types of marketers; it is just that the emphasis they place on these items is different. Another example is application features such as the ability for the solution to integrate into other systems and the way content and data is moved into the system. For smaller marketers that don’t have complex mailings, it is completely reasonable to move content elements one at a time into the application, but that approach would obviously be far less useful for say a retailer or travel provider that has thousands of content elements and dozens of customer segments. These are the types of differences that we highlight in the way we approach the providers in this space.

iMedia: What traits do the vendors you rank as leaders share? What makes them standout?

Jupiter: They all scored very well with their reference clients in terms of satisfaction and service responsiveness. Additionally, they all have rather usable applications, which in our opinion transcends the ease-of-use aspects of the application to specific functionality such as segmentation, dynamic content and the organization of the assets within the application. It is amazing how many vendors still lack basic foldering support to organize lists and content elements -- for example, only 40 percent have folder support that is consistent across the application.

iMedia: How should marketers use your report? What diligence beyond the report should they be doing to evaluate their best ESP partners?

Jupiter: Marketers should use this to jump-start their selection process. It should help companies reduce their list of prospective vendors down to the top three or five. There is a concept report that we released in conjunction with the buyers guide titled “How to Select an ESP” which includes our 10-point guide to vendor selection. Some of the items discussed in that research are to investigate the providers' data orientation in order to determine if it is list oriented or if it is a relational database, as well as the ease at which response data can be interrogated. Marketers should use what we have found as a map when they take the prospective vendors’ application for a test drive.

iMedia: You also ask in the report about what attributes marketers should question. Any insight you can provide on that?

Jupiter: There are many, but one area would be how to understand how the ESP calculates its metrics, such as email delivery. There are many different methodologies to calculating this metric, which makes a true comparative assessment of delivery rates across providers hard to ascertain. Another one is to look closely at custom reporting, even in what are characterized as “self-service” applications. Marketers need to look at reporting, specifically custom reporting and the ability for the marketer to easily generate those reports or rely on a fee-based approach where the vendor does this for the marketer.

iMedia: Any trends you see developing within the space?

Jupiter: Despite the consolidation that this market has seen, there are still many viable standalone ESPs. The report goes into great detail regarding the growth patterns of consolidated mixed media ESPs and those that are standalone. It is clear that there is still a market for standalone ESPs despite the land grab by many of the offline marketing services providers. We also see marketers placing much more emphasis on the ability to use website click stream data in their email marketing targeting. 

iMedia: Anything more to add?

Jupiter: As marketers increase their sophistication and gravitate to more targeted marketing tactics, they are demanding a more comprehensive set of tools. ESPs must increase their product development endeavors, as the top request for improvement from their reference clients is to increase the speed at which new product enhancements are developed and rolled out.

Additional information:
Get information on how to order the buyers guide here.

 Dawn Anfuso is editor of iMedia Connection.