How Accurate is Google AdWords?

According to Google's income statement to investors (on the Google website), Google earned over $6 billion from advertising during 2005 ($6,065,003,000 to be precise), and has earned over $4.6 billion in the first two quarters of this year. My estimate is that over 95 percent of this comes from AdWords; other sources claim 98 percent. This means Google's AdWords counting technology was responsible for at least $5.7 billion last year.

Given that such a vast amount of our money goes to Google on the say-so of a single web analytics system, it's extremely important to know how accurate that technology is.

It is well established that it is impossible for any web analytics technology to be 100 percent accurate. Most systems are accurate to within 95 percent. There are many reasons for this, which I've covered in earlier articles (see "Things That Throw Your Stats").

I have asked Google how accurate they consider the system to be. So far they have declined to provide any answer, so they haven't actually claimed 100 percent accuracy. Let's give Google the benefit of the doubt and consider the possibility that AdWords' click count is 99 percent accurate. This would make it the most accurate web analytics technology on earth by a long stretch. It would also mean it had mis-billed over $60 million last year. If it was 95 percent accurate, then it mis-billed $285 million last year.

My personal experience is that the accuracy of the billing system in Google AdWords varies. I run PPC campaigns for clients, and I always check Google's numbers with what our own software, InSite, says. Since we always run a unique tracking code on every Google ad, this is fairly easy. I have disputed numbers with Google several times over the last few years.

I'm not talking about click fraud here. Click fraud disputes do not involve a disagreement about the volume of traffic Google has relayed to your site. Click-fraud involves a disagreement about the validity of that traffic. Both sides agree on the numbers; they just disagree on whether all of it should be paid for.

The situations I'm concerned with are where Google AdWords claim to have sent more visitors to my site than I can verify for myself.

It's unreasonable to expect the numbers to match exactly. No two web analytics systems will ever agree perfectly. However, my experience of other systems suggests the disparity should not be too great. Overture/Yahoo PPC numbers consistently match mine to within two percent. Google AdWords usually match mine to within three percent. I think two percent is about as close as is possible, and three, or even four, percent is acceptable.

However, I do get concerned with the really big disparities. These are rare, but given the amount of money moving through AdWords, even rare disparities add up to a great deal of money. I have personally seen Google AdWords count 25 percent more traffic than I could verify on several occasions. This summer I had a situation in which the Google AdWords were claiming numbers ten times what I was seeing.

Here's what happened:

I recently ran a small test campaign for a new business. In the first two weeks Google AdWords claimed (and billed for) 96 clicks, but we only saw 11. When I queried this Google's AdWord Response Team said they checked and the numbers were correct. They didn't tell me how they'd checked the numbers, but they did point out I had a history of disputing their counts. They said this indicated my software was probably faulty. They invited me to use Google Analytics (once known as Urchin), which was, in their words, "one of the most powerful web analytics products on the market."

I was not convinced my software was faulty since it has always matched Yahoo/Overture, and usually matches Google AdWords, but if Google wanted to use their own system to check AdWords numbers I was willing to give it a try. I made sure to add all the tracking options Google recommended to ensure the Google Analytics count was as accurate as possible.

Having done everything Google asked of me -- by the book -- I waited a week for the numbers to cook. At the end of that time, Google Analytics told me I had 14 AdWord visits while the AdWords billing system itself claimed 59. My own software, InSite, also said 14 visits, the same count as Google Analytics.

On other metrics, such as traffic from native listings, or other sites, InSite matched Google Analytics to within three percent. I think this validates the accuracy of both InSite and Google Analytics. So the system that Google told me to use was telling me that Google AdWords were over-counting. I now had two different products telling me that Google AdWords was over-charging, and by a significant margin.

This article is not intended to be a rant against Google. I don't want to condemn them just because their AdWords system has made a mistake. I'm not even going to criticize them for insisting I use their software to check. They know nothing about mine, so why should they trust it? What is really important is how Google handles situations when a customer believes they have been over-charged. Given the volume of traffic AdWords handles, it would be incredible if the AdWords click-counting system never made a single mistake. However, given the amount of money at stake, it is very important Google handles situations like this in an appropriate manner.

My next step was to report my findings to Google's AdWord Response Team.

Their answer was straight-forward: "For privacy reasons I do not have access to your Analytics data. I've reviewed your account and found that you accrued charges for legitimate clicks made between July 28, 2006 and August 23, 2006 while your campaign was active. Therefore, we are unable to offer a refund for these charges."

This raises a number of questions: How did Google verify my charges were accurate? What is a "legitimate click"? How does Google explain the discrepancy between the two systems? Given that I'm still unhappy, what happens now?

My next article will explore Google's response, and then we'll look at the legal and business issues. In the interim, I'd like to hear from anyone else who has been in a similar situation. I'd like to get a handle on how big a problem this is. I've asked Google how often disputes occur, and the value of the disputed transactions, but they haven't provided an answer as of yet.

If you've felt Google AdWords has over-charged you, let me know. If you've disputed this with them, how have they responded? Do you feel Google had handled things appropriately, or did they leave you feeling frustrated? Email me with your story.

Brandt Dainow is an independent web analytics consultant. Read full bio.

 

Comments

Donna Tang
Donna Tang October 9, 2009 at 2:26 PM

I started to use Google Analytics several months ago, and started to use Google Adwords last month. The google analytics is not accurate, it does not match the visits track of my system, # of visits of a certain page at Google Analytics is even much lower than budurl tracking.

I got two leads (they signed up on a free trial page) after I launched Google Adwords. But Google Adwords show three clicks from CA, NE, LO, but the addresses our leads show that they are from Oklahoma and North Carolina

Anthony Borelli
Anthony Borelli September 3, 2009 at 1:43 PM

I am a long time search marketer and - generally - a big fan of Google and AdWords. Google has helped me make millions, so believe me... I don't have any axe to grind, but when it comes to suggesting possible issues with the AdWords algorithm, they are a brick wall.

You have zero chance of them offering a plausible explanation of these discrepancies. They will point out that AdWords tracks "clicks", while Analytics tracks "visitors" and no matter how out of sync those figures become (AdWords regularly reports 10% more traffic than Analytics does to me) they will simply claim that 10% of visitors are clicking on the same ad twice.

At the end of the day, though, it's all just the cost of doing business with - otherwise - the best tool available for reaching the most targeted audience possible.

Anthony Borelli
Author of Affiliate Millions

Gene Jordening
Gene Jordening January 26, 2008 at 1:38 PM

I am almost 76 yrs. old and am a retired Msgt. from the USAF. My last 7yrs. with the USAF was in JAG. (Judge Advocate Group) as a paralegal. I just read your article about Google and how they may, apparantely, overcharge in PPC. I also served in the Military Police and occasion as an investigator with the Military CID and also served as an investigator with the California Alcohol Beverage Control. From all this I have learned not to trust people or what they say they will do. As a result of my experience I have learned that I must check out what they say they will do against what they actually do. I am totally new to the PPC advertisement method of addvertising. I do not know how to confirm that X amount of clicks were made. From what I read of your article on "How Accurate Is Google Adwords" it makes me suspicious of their adwords and whether I can trust them or not. Being a novice at this form of advertiseing should I go to Yahoo before Google or maybe AOL or some other ISP? If you have any comment on my comment please email me.

Gene Jordening
Msgt. USAF (Ret)
Owner
Painter's Helper Co.
62 Oak Grove Cv.
Byhalia, MS 38611