A lot of marketers think that testing the ideas of a marketer is synonymous with testing the marketer himself.
Another way of saying this is that when insightful marketers come up against the concept of running tests on their marketing campaigns, they sometimes balk. The impression is out there that top marketers intuitively know what gets consumers jazzed about a product: From this perspective, testing isn’t necessary if a marketer is at the top of his or her game. And since it takes time, there is a strong temptation to assume that “our experience will get us close enough.”
When you plan marketing campaigns, you always come up with more ideas than you can use. Testing -- whether it's A/B or multivariate -- gives you the opportunity to put forth your three or four or ten best ideas rather than limiting yourself to a single great concept.
So relax, enjoy the challenge, and abandon yourself to coming up with brilliant marketing ideas. Then, as you begin to fold testing into your marketing mix in the coming months, follow these 10 steps to ensure it’s a success:
Step #1: Forget about running a test or two
Some tests yield immediate value by increasing sales or sign-ups. Other tests prove new ideas won’t work.
It's the ongoing accumulation of incremental results that has true value. As you incorporate testing into the execution of all marketing campaigns, it stops being a discrete activity and simply becomes part of the decision-making process.
In fact, leading companies will run two or more tests per month this year, and our research shows that number could be as high as 10 per month next year and beyond.
Step #2: Go for the big money first
As your campaigns and sites become optimized, the big returns may be harder to find (more like panning for gold than drilling for oil). For now, drill where the oil is.
Almost every site contains areas that yield a million dollars or more in improvements if tested and optimized successfully. So why start with a test that results in less than $100,000 in value?
Generally, this means starting with the shopping cart or subscription form. Estimate the likely outcome and put a conservative number next to your prediction. Then test cart elements (such as navigation) that you estimate will yield the biggest returns.
Step #3: Understand how long you need to run a test
In a perfect situation, where the tested elements are very different and there’s a high volume of conversions on the tested pages, statistical confidence can be reached in a few days or less.
But if you don’t have as strong a variation in customer behavior as expected, then it could take weeks to get conclusive results.
In a couple of tests we have run lately, the obvious winner halfway into the test later put on the brakes, turning out to be the loser once statistical confidence was reached.
Step #4: Test what matters
If you wanted to test a cookie recipe, would you test where you stored the chocolate (pantry vs. cabinet), or would you test the amount of chocolate?
Where you store the chocolate may be important -- but in reality, the likelihood that it would affect the taste of the cookies is slim.
Similarly, headline, copy, price point, and page layout are more likely to impact your conversions than the color, size or even placement of a single element on the page.
However, we're often surprised at how many seemingly important things don't move the needle -- another argument for frequent testing. Use smart selection of the elements to test, and creative alternatives to those elements, and you'll soon see results.
Step #5: Keep testing
No single test delivers a final answer. In nearly every case, one test suggests another.
On a client’s checkout page, we recently tested the display of shipping and returns information and details of the security policy. Some of us believed that any link off the cart would decrease conversions, but the opposite was shown to be true: Visitors who saw the clear reference to the shipping and security information converted at a higher rate and had a higher average order.
Were all the elements -- shipping, returns and security -- improving conversions? One element more than another? Should we reference shipping and returns information earlier in the shopping process?
We designed a second and third test to drill down and find the truth.
Step #6: Test the whole session, not just a page
Only rarely do entire conversions happen on a single page. More often, a prospect sees an advertisement, email, or search listing, visits your site, clicks around and, if you're successful, goes through the sign-up or checkout process. This interaction takes five to 10 pages or more.
So how much can a single element on a single page affect his likelihood of buying?
Think in terms of the whole session. In its most basic form, optimization of a session would mean taking your offer from your ad (a “limited time only” approach, for example) and carrying it through to the landing page, the product detail page, and the shopping cart.
Step #7: Don’t hurt natural search results
As paid search advertising (SEM) rates rise, remember to protect and improve your natural search (SEO) rankings. Running an A/B test by creating one or more new pages, dividing traffic, and directing different traffic to different pages risks harming your natural search listings.
Step #8: Watch your tests carefully
Sometimes, one or more of your tested recipes will significantly under-perform the control within a day or two.
While we support running a test all the way to statistical confidence (if you're most interested in creating clean, defensible test results, for example), you may often decide that test purity is less important than maximizing revenue and profit.
You'll want the ability to track results real-time so you can make split-second changes.
Step #9: Involve creative early on
Testing tools do not tell us what to test, whom to test, or what alternatives to try. These questions are still answered by the branding, marketing and design experts.
As soon as people start to ask, “Let’s see how this will test,” rather than, “Why test?” the creative aspect of testing -- coming up with variables to test and discovering what really works and what doesn't -- becomes clear. People stop feeling "tested" and start feeling validated because they now know that their marketing ideas really do work!
Step #10: There are few general answers
In the hundreds of tests Offermatica has run, we’ve only found a few general answers that work across the board.
The best offer, page or navigation depends entirely on the particular company, product, time of year and more.
Your best recipe for success can usually be found quickly and inexpensively. When a variable or two changes, it's time to re-test.
As you follow these steps, remember one thing: Testing cannot do the work for you. Marketers must continue to be creative and intuitive in developing their campaigns. Only then can testing do its job effectively by maximizing your efforts.
On the other hand, if you don't test, you're not taking advantage of the tools marketers have available to increase the bottom line scientifically. It's my belief that, without testing, you're not really marketing at all. You're just guessing.
Jamie Roche is a founder and president of Offermatica. Roche brings to Offermatica the experience of leading a visionary technology company from the dawn of the commercial Internet, through the bubble burst and out again. Offermatica, formerly Fort Point Partners, Inc. is an eight-year-old software company that provides hosted testing and optimization services to some of the largest retailers in the industry including Restoration Hardware and Joann.com.
Prior to Fort Point Partners, Roche ran Webfactory, a provider of Internet products and services to Yahoo, Netscape and other leading Internet companies at their formation. Roche also worked for KPMG Peat Marwick and SiliconGraphics. He is a graduate of Yale University.
