From my seat, I’ve also seen more and more marketers embrace A/B and multivariate testing as a must-have in their ecommerce tool kit (Yay!). With DIY tools like Google Website Optimizer, almost anyone who has time, a little patience and bit of knowledge, can start to test various customer experiences on their website. Website Magazine even released a comprehensive list of tools and vendors who provide such services.
In a recent Wall Street Journal article, Forrester Research analyst Sucharita Mulpuru said such website "optimization" is growing as Web retailers try and emulate the likes of Amazon.com Inc., which does "a/b" testing to compare which website layout works better. Earlier this year, Amazon tested a new home page that is less cluttered than its old one, with fewer buttons, more white space and a bigger search box; the page also emphasized digital goods over physical ones.
Yep, Amazon is still doing it and so should everyone else. But not everyone is Amazon. So unless you have an internal team of testing experts at your disposal, the DIY version of testing might actually hurt you in the long run.
The reason: strategy is just as important as technology.
Marketers who overlook the process of determining what tests are most effective do so at their peril. By underestimating the impact of apparently subtle changes, a significant amount of benefit is lost. Multivariate testing on an ad-hoc basis simply does not generate the conversion increases that can be attained by knowing what to test, where and when based on a wealth of expertise and past experience. We must think not just about test results, but business and site goals.
To be effective, a multivariate testing campaign must test the right hypotheses and in the right order. Knowing what, how and when to test—and in what order—is key to achieving results and answering questions and fast. A Maxymiser survey revealed 91% of marketers under-estimated the importance of the position of price and call to action buttons on this checkout page. Yet, identifying the subtle changes which will have a large impact on critical pages in the funnel is vital to achieving good results.
I’d suggest that much of the value of engaging external expertise in planning and executing tests arises from being able to take a strategic view of the testing process, identifying the most effective places to test and then testing the changes most likely to make a difference. And having someone internally to champion the process, rather than just blindly figure it out will ensure you get the maximum benefit and ROI out of a testing program.
Best of luck optimizing in 2012!