In my last column I raised the question of how accurate Google AdWords' click-count is. This is the count of clicks which determines how much Google bills your AdWords account. With over $5.5 billion billed by AdWords in 2005, the accuracy of that billing system is extremely important. I cited a case in which there were dramatic differences between what AdWords claimed to have sent to a site, and what both my own and Google's web analytics systems counted. This article will explore Google's perception and handling of AdWord click-count disputes.
Google's AdWord click-count is based on log analysis. When someone (or something) clicks on a Google Ad, this is recorded in a log file on the Google servers. The logs are analyzed to determine the click-count which will be used as the basis for billing. In Google's view, discrepancies between AdWords' click-count and other web analytics systems don't call the accuracy of AdWords into question because AdWords is counting, while web analytics systems are merely estimating numbers. According to Richard Holden, Google's director of product management, "We don't look at AdWords as an analytics system. AdWords is the front-end to a system which measures clicks directly. Anything that we're measuring as a click is 100 percent accurate. We look at Google Analytics and other analytics software as doing estimation in the sense that they're estimating when clicks occur and don't have a direct view into the system. We don't look at AdWords as doing any estimation whatsoever."
Google does distinguish between "valid clicks" and "invalid clicks." In Google's terminology, a valid click is one you should pay for, and an invalid click is one you don't have to pay for. According to Holden, the only real issue with AdWord numbers is the validity of clicks and Google's attempts to filter out invalid ones.
Holden stated there are two types of invalid clicks-- "non-nefarious clicks, such as my grandmother double-clicking because she thinks that's the thing to do; or nefarious clicks, such as someone clicking on an ad because they're trying to generate costs for an advertiser."
According to Holden there are two types of queries over AdWord numbers. A customer can question the validity of some of the clicks they are being billed for, or they can question AdWord's click-count because their analytics system is reporting different numbers. In both cases, the case is assigned to an individual member of the AdWords Response Team. If someone has an analytics system which is producing different numbers, the Response Team's first step should be, according to Holden, to work through the different technical reasons why this can happen in an effort to educate the customer. If the customer is still unhappy at the end of that process, the Response Team case worker can ask someone in the Invalid Clicks Team to examine the data and verify accuracy.
The only people at Google who can actually check the data behind AdWord's billing are the members of the Invalid Clicks Team. While communication with customers is mainly handled by members of the AdWords Response Team, they cannot actually verify the data themselves; they must escalate the matter to the Invalid Clicks Team. Richard Holden told me that "this team does proactively look at accounts that we get complaints about. They can look at the logs; they can look at the data very closely." When a Response Team member tells you they've verified the data themselves, Holden states what they mean is that they have managed a process in which the Invalid Clicks Team has verified the data. According to Holden, Response Team members are not anxious to advertise to the world that the Invalid Clicks Team exists.
Holden told me the reason AdWords Response Team members cannot look at the data is privacy-- "We take the privacy of both the advertiser and the consumer very seriously. The Invalid Clicks Team has been screened and is cordoned off, so there is a limited set of people who have access to the data."
Next: Dainow talks with more Google staff, as well as a legal expert
During the course of this investigation I've spoken to members of Google's AdWords Response Team, Invalid Clicks Team, and to Google Director of Product Management Richard Holden. What comes across most clearly is an absolute rock-solid certainty that Google AdWords is recording clicks with absolute perfect accuracy. It never misses a click, it never double-counts a click. Ever. It never mis-allocates a click to the wrong advertiser. Every AdWords click is logged with absolute precision. The initial log analysis which counts those clicks is absolutely perfect. Given the number of servers and data centers Google operates on, this also requires that either the logs are merged together, or the counts are, and that this merging is absolutely flawless. The data has to be stored in one or more databases, which means it is being written, stored, queried and retrieved without a single glitch. I would imagine Google has systems in place to ensure data integrity, so these have to work perfectly as well. There is a great deal of effort put into checking whether all the clicks should be paid for, but the assumption is always that the initial log analysis is 100 percent perfect. As a result, there is simply no process inside Google to check whether this is the case.
As a mere advertiser, I am not in a position to judge how accurate AdWord's click-count is. This is not a new problem, nor one which is unique to Google. Accuracy of the information which determines billing for ads has been an issue since the first newspapers took their first ads. The accepted solution is to obtain an independent audit by a suitable organization-- an audit bureau. Once advertising commenced on the web, the audit bureaus moved online. Organizations such as Audit Bureau Circulation Electronic (ABCE) and the Internet Audit Bureau (IAB) exist in almost every country of the world. These bodies assess the accuracy of the systems used to count the numbers which underpin billing for online advertising. Most, but not all, follow standards laid down by the Joint Industry Committee for Web Standards.
Google does not have independent audits of its systems. Holden told me he does not believe this is necessary because, as far as he is aware, none of their major competitors are independently audited. Furthermore he does not believe that appropriate standards exist for such auditing.
"There are no IAB standards for what is a click today, but we are part of the working group that's working on what is a click," Holden told me. In this Holden is referring to IAB USA which does not support JICWEBS standards (most other national IAB's do). JICWEBS standards are more complete than those of IAB USA, and include a definition of a click as "the activation of a hypertext link by a valid user." Meanwhile IAB USA has been working to define a click for over two years.
So we have a situation in which Google is absolutely certain that the initial logging and analysis of AdWords clicks (as yet undefined) is 100 percent accurate all the time. The only basis upon which numbers may be investigated is to determine if the clicks which have been billed should have been billed. Finally, as Google's staff will freely tell you, it is a contractual condition of using AdWords that you accept Google's numbers.
Obviously most of Google's advertisers are happy most of the time; otherwise they would not be spending the sums they do. However, my own experience, and that of others I've spoken to, is that complaining rarely brings happiness. When you do complain Google's customer handling skills are nowhere near as perfect as their technology. According to Holden, the initial response to a query should be working through reasons for discrepancies. In my experience this is rarely the case, and I've spoken to enough people to know my experience is not unique. More frequently the initial response is little more than an assurance that the numbers are accurate, and a blunt reminder that it is a condition of using AdWords that you accept Google's count. It usually requires some pushing to get as far as being given a FAQ indicating why your own numbers could be wrong, and real dedication to get things escalated to the Invalid Clicks Team. Most of the people I have spoken to felt Google was arrogant, and that dealing with Google was frustrating.
It may be that part of the problem is lack of communication. You can't phone Google unless you are spending extremely large amounts of money, so you're limited to email. This is not what most people are used to. In any other area of business, an expenditure of (say) $100,000 a year would rate you at least a telephone contact, if not face-to-face meetings, but not with Google.
In addition, there is a lack of transparency to Google's billing. Google declines to explain how its technology works. According to Richard Holden-- "We believe we have unique intellectual property in how we track and ferret out these problems versus our competitors. This is one reason why we don't talk very specifically about how we actually do it, nor do we talk about specific numbers. The more we publish, the more we give the bad actors the ability to act against us. We do understand this is a double-edged sword because it means less people understand that there is not a wide-scale problem."
What this means is that you can't get a detailed breakdown of how your bill was calculated, nor will you ever have direct access to the data from which that bill was compiled, so you can't check for yourself.
I asked an international lawyer specializing in media and the internet to look into the legal aspects of the way Google handles AdWords disputes. Rupert Grey is the senior media partner at Farrer & Co, one of the UK's oldest and most prestigious legal firms (founded in 1790). Clients include The Times and members of British Royal Family. In his opinion, the U.S. and European contracts are probably legal-- just, but there is real confusion around what you're paying for and how you'll be charged.
"The contracts don't state what you're buying," Grey said, "and it's impossible to determine the basis for your charges. Even lawyers explain what you're being billed for."
It's questionable just how accurate AdWords is. Google believes the core technology is consistently 100 percent accurate. If you don't accept this, they see their task is to show you the error in your ways, not check the accuracy. This, plus other aspects of Google's behavior, leaves many people feeling frustrated. However, with Google earning over $1 million per hour from AdWords, I don't think we'll be seeing much change in the near future.
In my next column we'll examine the technical reasons why the Google AdWords' click-count can legitimately vary from your own analysis, and what you can do about it.
Brandt Dainow is an independent web analytics consultant. Read full bio.