ellipsis flag icon-blogicon-check icon-comments icon-email icon-error icon-facebook icon-follow-comment icon-googleicon-hamburger icon-imedia-blog icon-imediaicon-instagramicon-left-arrow icon-linked-in icon-linked icon-linkedin icon-multi-page-view icon-person icon-print icon-right-arrow icon-save icon-searchicon-share-arrow icon-single-page-view icon-tag icon-twitter icon-unfollow icon-upload icon-valid icon-video-play icon-views icon-website icon-youtubelogo-imedia-white logo-imedia logo-mediaWhite review-star thumbs_down thumbs_up

Why A/B Testing Isn't Enough

Kevin M. Ryan
Why A/B Testing Isn't Enough Kevin M. Ryan

A/B testing is so last millennium. Search engine advertising provides a connection to the consuming public while they are in the driver's seat. The fluid nature of paid listings allows for constant refinement in how you make the directive listing connection.

Refinement in a search engine advertising program means changing and adapting click messaging elements in two short lines of text. Keywords are added and removed according to which product offerings or services line up with content and user intent. Positions are optimized, maximized and delivered in the most efficient manner possible.

The process refining the search experience for users also involves determining which text and graphic representations they prefer. The most common form of comparison is the A/B test in which one variable is compared to another. Another way to compare effectiveness includes the multivariate method but it is complicated and often perceived as prohibitively expensive.

Search and multivariate
The A/B test enables an advertiser to compare creatives, messages or landing pages. The test is relatively simple to administer and most often will provide a definitive answer as to what works better. Simply put, while it may prove what works better, an advertiser certainly won’t know what works best.

A multivariate test will enable comparison of multiple variables. In order to really provide an effective analysis of user preference, many considerations must be addressed. A true multi-variable analysis will allow a site owner to compare several page designs and messages simultaneously, ultimately lifting desired action activity beyond previously established benchmarks.

As you might imagine these results don't come easy or cheaply. You can't simply toss a bunch of variables in the air and hope for the best. A sound strategic implementation is essential and the multivariate initiative requires several commitments to be successful.

Yahoo!'s move forward
An industry has sprung up around multivariate testing due to the complicated nature of these initiatives and there are many providers that offer outsourced testing solutions. A little along the way couldn't hurt and last week, Yahoo announced strategic partnerships with two such providers: Offermatica and Optimost.

Yahoo evaluated nearly a dozen testing providers before deciding on two partners, although the company left the option open for future relationships. "Partners were selected on strength of offering and customer service criteria," says Dan Boberg, vice president of strategic alliances.

Since cost is almost always an issue and comparison shopping can be cumbersome, Yahoo negotiated best pricing, which usually accounts 25 percent discount on volume, though implementations will vary dramatically. Yahoo plans to be an advocate for clients in the set-up process and guarantees the best pricing on the market.

Successful conclusions
Yahoo has logged several case studies of successful tests. Among them, Motley Fool tested 13 variables, and found subscriptions rose more than 36 percent; Monster.com increased companion downloads more than 35 percent in less than two weeks.

Boberg says that most clients have seen a big enough pull for ongoing testing, making it part of the media budget going forward. So bear in mind that testing doesn’t have to be a one-time effort.

With the upside potential painfully clear, there are a few preparations to make before beginning the test. Costs vary greatly and you'll want to make sure the initiative pays for itself. Although you will want to execute tests quickly, you don't want to rush through selecting variables.

Approach ancillary benefits like targeting and customer data acquisition cautiously. Listen to those who have completed tests in the past while making decisions. Finally, know exactly what you want out of the initiative and define the boundaries and responsibilities of each party involved.

Winning combinations
We can use this wonderful tool called the internet to do amazing things for humans. Ironically, although technology created by people defines the web, it doesn't seem to interpret their wants and needs well. Multivariate testing enables content owners the ability to not only dig into likes and dislikes in messaging but also helps with targeting abilities.

Every step closer to accurately defining the need of the online interaction and how the consuming public may wish to interact with our interactive presence either online or elsewhere is another step toward creating the human-driven interactive utopia.

Protecting the consumer relationship

Luckily, many people consider ads as just part of the internet landscape. Even in a survey of its own users, over 70 percent of people who use Eyeo's Adblock Plus said they would allow some unobtrusive ads to support free websites. When ads are targeted, relevant, well-executed, and unobtrusive, consumers often view them as informative and providing a service. Unfortunately, bad ads are too common, and it's this phenomena that is driving the mainstream adoption of ad blockers.

For these consumers, it's up to us -- publishers and advertisers -- to provide them with the experience they want, which means not sacrificing experience for monetization. As an industry, we're already taking some proactive measures, such as redesigning the user experience, but we need to also tackle bad ads. Luckily, companies such as GeoEdge, ClarityAd, The Media Trust, and RiskIQ are all developing solutions to help publishers protect themselves against bad ads.

In addition, it's our responsibility to educate these consumers on the connection between the content they love and the ads that pay for that content. Numerous publishers have started to do this, but it does require a light touch, as demonstrated by Destructoid's 2013 efforts. Read the frank feedback -- both positive and negative -- readers gave to its efforts. From it, you'll gather that many people will be ok with ads, so long as the publisher respects the user experience.

But what about the stalwarts?

That does, however, leave a large population of people who do not want ads, some for good reason.

One of these reasons is privacy. Consumers deserve to be informed about how their data is being used and be given a choice. Extensions like Crumble, which enable consumer to protect their privacy while also still enabling ads to be served, go some way to doing this, as do push notifications on websites that alert visitors on how cookies are being used. If we can regain consumers' trust and empower them, then we've given them one fewer reason to need to ad-block.

There are some people though, who are constitutionally opposed to ads. So what can we do about that? One idea is to flat out deny them content, which I think few publishers would be willing to do. Another is to use a pay-wall, which has met with limited success. This is not surprising, given that a recent survey  in the U.K. found that that only 2 percent of people would pay a lump sum for an ad-free internet.

A viable alternative, however, could be borrowed from in-app games: micropayments, where ad-blocking visitors make a small payment, say 25 cents, for a non-ad serving article. A small group of thinkers, including Walter Isaacson, believe this may be the way forward. The recent launch and VC funding of various micropayment-for-content companies, such as CoinTent, may remove much of the friction involved in micropayments, for example by providing one-click payment.

The pipe dream

The best solution, though, would be to remove underlying cause of the issue. By this, I don't mean somehow circumventing ad blockers. I've already laid out the reasons why I think this is at best a short-term tactic and at worst a fool's errand.

Instead, what I'm referring to is a changing the underlying assumption behind the ad blocker model, which can be boiled down to "guilty by default unless you pay us." At the moment, most ad blockers prevent all ads (when I downloaded it, it was enabled for all sites) unless either:

  • The consumer actively disables the software for specific sites

  • You are a small business and go through the rigmarole of registering with them, or

  • You pay them, like Google, Amazon, and others have done. (According to a report by the Financial Times, a fee equivalent to 30 percent of the additional ad revenues that would be made from being unblocked.)

The first option fails the "what's in it for the consumer?" test, as most consumers probably will not take the extra effort to enable ads if they can see the content without it. This phenomenon is similar to the Tragedy of the Commons. The latter two options, especially the pay-to-whitelist option, fails ad-blocking software's public statements about finding a balance between enabling revenue from non-intrusive ads and consumer choice, especially for smaller businesses.

So what other alternatives are there? One model could be changing the assumption to "innocent until proven guilty" with the consumers needing to elect to have ads disabled on certain sites they find annoying. This would justly spare the innocent. On the flip side, there would need to be a remediation process to enable sites that have fixed the underlying issue. This is similar to what is used in the email industry, a role played by non-profit organizations such as Spamhaus.

Another, more interesting alternative is an ad blocker that combines the idea of micropayments. One such example of this is Fairblocker, which enables users to decide how much an ad-free internet is worth to them a month and then splits the payment between the online publishers that the user visits. While there are some holes in the business model, publishers could do much by promoting the use of this type of technology to consumers who are vehemently against ads.

So what's in the future?

Like anything, the future impact of ad blocking will be a product of a number of the factors mentioned in this piece. However, I think that, despite the doom and gloom, there are various viable alternatives publishers can take to increase their chances. Smart publishers will test each one and see what works for them. Whatever they choose, however, it all starts with publishers safeguarding the customer experience while also working as an industry to provide better alternatives to consumers.

Keith Fagan is senior vice present and general manager of data solutions at sovrn Holdings, Inc.

On Twitter? Follow iMedia at @iMediaTweet.

"One hand preventing punch attack of another" image via Shutterstock.  


to leave comments.