We hear all the time that click fraud is one of most pervasive problems facing online advertising. Reports consistently place the percentage of click fraud in the search space at around 15 percent, while Google's and Yahoo!'s recent settlements with advertisers unhappy about fraudulent clicks focused additional industry attention on the issue. And the that it was looking into standardizing the definition of a click in order to better address the problem.
But maybe something is getting lost in all this controversy. What, exactly, is click fraud, anyway? And how significant of an issue is it? Maybe the answers to these questions seem obvious. Bear with me for a second, though-- things might not be as simple as they first seem.
Click fraud: What is it?
As Tom Cuthbert, CEO of the search analytics and auditing firm Click Forensics, puts it, a simple description of click fraud could be "any click that an advertiser shouldn't be paying for." Unfortunately, this only goes so far in outlining the problem-- we're still left to determine which clicks should be paid for and which shouldn't. More on that in a moment…
Most experts separate fraudulent clicks into two distinct types: automated and human driven. In the first category, there are those clicks that occur as a result of search engine crawlers or link verification programs. There are also automated click programs (bots)-- which artificially boost site traffic in order to increase advertiser payouts.
There are also multiple types of human-driven click fraud. First, some third party websites will artificially boost their ad payments by clicking on ads hosted on their own websites (in addition to using automated click programs). Second, advertisers have reportedly clicked on competitors' ads in order to drive down competing advertising budgets. Most ominously, there are rumors of "click farms" in India or within the former Soviet Union where workers are paid to click on ads. But while many commentators -- including myself -- have characterized click farms as a real threat, the only evidence for their existence seems to be a May 2004 Times of India article (if anyone has further substantiation, please let me know).
What are we to make of this? In general, there's little data on the relative scale of each type of click fraud. Some auditing firms play up the threat of human clicks on third party sites. Other commentators have emphasized the role of spyware and bot-nets, and still others have argued that the problems are of equal size. And studies on the overall percentage of click fraud largely don't break the problem down by type.
Click fraud: Defining "legitimate"
The IAB should be applauded for looking into the definition of a legitimate click. And I'm glad to hear that they're including search engines, advertisers and auditing firms in the discussion. But I think it will quickly become clear that these groups have very different ideas regarding what makes a click legitimate.
The auditing firm Click Forensics, for example, defines a fraudulent click in reference to the clicker's intent. If the clicker is an interested consumer who clicked on the ad in order to learn more about the advertiser's product, it's a legitimate click. Under this interpretation, just about any other category of click is considered illegitimate.
Conversely, the search engine's perspective has more to do with certain characteristics of a click and less with the clicker's intent. The search engines look to see if a click is automated, or if it comes from an IP address associated with the website an ad is hosted on. What this means is that when defining click fraud, advertisers cast a wider net than many search engines.
There may be reason to question the efficacy of the advertiser and audit firm's intent-heavy definition. For example, as part of their settlement with Lane's Gifts and Collectibles, Google was required to provide the court with an independent evaluation of their click fraud fighting mechanisms. The resulting report, from Professor Alexander Tuzhilin of New York University, cast some doubt on the idea of using clicker intent to define click fraud. "Unfortunately," Professor Tuzhilin writes, "in several cases it is hard or even impossible to determine the true intent of a click using any technological means." In other words, intent is in the eye of the clicker… or something like that.
Tuzhilin's report also brings out an interesting tidbit about click fraud: The majority of automated fraud is caught by Google (and, one surmises, other tier-one search engines). So the definition of click fraud used by the search engines may be quite effective at weeding out a great deal of automated fraud-- which may, after all, constitute a significant portion of the total click fraud out there.
But isn't click fraud an issue?
All of which is to suggest that the threat of click fraud may be somewhat overstated. With this in mind, we may want to take a second look at the reports that consistently play up click fraud's predominance.
Click Forensics, for example, released a study in July that placed the total amount of click fraud at 14.1 percent. What wasn't initially obvious, though, was that this included fraud that had already been caught by the search engines. In other words, the report didn't show the amount of click fraud advertisers pay for-- it merely highlighted the number of clicks that Click Forensics caught as illegitimate. Whether or not advertisers actually paid for these fraudulent clicks wasn't specified.
In fact, says, John Rodkin, CEO of the advertising and optimization firm ClickShift, advertisers pay a lot less than the Click Forensics numbers imply. If advertisers were to compare the total number of clicks on their ads to the amount they were billed for, they would see that the search engines aren't billing for a lot of fraud already. This, Rodkin argues, is the "fundamental misunderstanding of click fraud."
Moreover, Google recently announced that it will soon allow advertisers to see the percentage of fraudulent clicks it catches and doesn't bill them for. Of course, it's unlikely that Google catches every fraudulent click-- and the actual number of fraudulent clicks paid for by advertisers is probably anyone's guess at this point. But there's certainly a percentage that advertisers AREN'T billed for. So the evidence we do have suggests that the widespread advertiser reaction found by the research and analytics firm Outsell -- with 27 percent of advertisers decreasing their search spend in light of click fraud -- may be a little overblown.
Less (and more) of an issue
Eric Goldman, assistant professor at Santa Clara School of Law and a frequent commentator on issues of online marketing and law, agrees that advertisers sometimes overstate the threat of click fraud. "Ultimately," he told me, "this is a basic contract issue." So as long as advertisers and search engines agree on a definition of a click, then "search engines bill advertisers using this agreed-upon definition, and advertisers are getting exactly what they bargained for."
In looking for this definition, it's worth noting that some unwanted clicks are unquestionably fraudulent -- think automated bot clicks -- and advertisers clearly shouldn't have to pay for these. Unfortunately, the discussions around click fraud seem to have gotten somewhat out of hand, with the more extreme positions getting the most attention. Search engines argue that they don't need to go further in their fraud detection efforts. And some auditing firms have taken the opposite view. Google, says Click Forensics' Cuthbert, may be making a reasonable effort. But "reasonable," he says, just "isn’t enough." To explain his position, Cuthbert directed me to a website -- www.reasonableisnotenough.com -- that lays out Click Forensics' argument.
However, it's not exactly clear to me WHY a reasonable effort isn't enough in this context. For most business transactions, a reasonable effort is perfectly acceptable. Of course, when it comes to ensuring safety at a nuclear reactor or protecting children from predators, then we need to go beyond standards of reasonable effort. So although we all like to talk about holding our industry to a higher standard, is this really what we mean?
As industry groups begin to work out what constitutes a legitimate click, it's worth remembering: Not every click that doesn't lead to a sale is fraud. As more data becomes available, the search engines are likely to feel pressure to step up their level of the click fraud protection. But I think advertisers may find that they aren't paying for as many fraudulent clicks as they believe. While remaining an important problem, click fraud may turn out to be less of the industry-threatening blight it's sometimes made out to be.
Author's Note: I'd be remiss if I didn't mention the recent BusinessWeek cover story on click fraud. While I think the article is well written, it doesn't provide a tremendous amount of insight into the scale of the problem. The perpetrators outlined in the cover story aren't deriving a tremendous amount of revenue from click fraud. It would indeed take a "swarm" of fraudsters at "$25 to several thousand dollars a month apiece" to total half of a billion dollars in click fraud.
Having said that, perception is everything. And I'm certainly not suggesting that this is something that should be swept under the rug. One of the things that is particularly striking about the Business Week article is that it highlights this sad fact. For all of the billions of dollars generated by our industry, we as an industry don't appear to have a recognized spokesperson to articulate our vision-- or any vision for that matter.