The disturbing inaccuracy behind Google Analytics

At the time Google purchased the software Urchin, it treated (and counted) bounces as 0-duration visits. When calculating the average duration, this meant that it came up with a number that was under the real one. How inaccurate this was depended on the percentage of a site's arrivals who bounced.

In July 2007, Google changed the calculation of average duration so that it did not include bounces anymore. This was the correct thing to do and meant that its figures for average duration were now accurate. A month later, Google put it back to the old (wrong) way of calculation. In other words, Google intentionally rolled Google Analytics back so that it produced an incorrect average duration. Why? Brett Crosby, senior manager at Google Analytics, explained in a Google blog that it was because people complained the change meant the new (accurate) numbers were out of line with the old (inaccurate) ones. (That blog has since been removed by Google, but you can find it copied on many sites.) In other words, these people considered consistency more important than accuracy, and Google obliged them.

It's been that way ever since -- Google is intentionally and knowingly providing inaccurate numbers because a few people preferred neatness to truth.

Areas of error
Treating bounces as visits doesn't just affect the accuracy of average duration -- it affects any metric based on the number of visits. 
  • Visit count
    When you see a figure in your Google Analytic reports for Total Visits, ask yourself: What do you think that represents? If you see Total Visits as the number of people who entered your site, who reacted to the sales pitch, who engaged with your content, who potentially could have bought products, then you are wrong. It is the number of people who arrived at the front door of the site, nothing more.

  • Conversion rate
    The Conversion Rate tells me how successful my site is at selling. It is legitimate to calculate Conversion Rate including bounces, but my personal experience is that it is misleading to do so. I use Conversion Rate to improve my site's sales pitch. People who bounce were never exposed to it, so including them in the calculation means I cannot possibly know whether my sales pitch is working or not.

  • Exit rate
    Google Analytics tells me how many people exited the site from any given page. Like Conversion Rate, this is useful for assessing the sales performance of the page. However, getting someone to enter the site and getting them to stay in it once they have entered are two very different tasks. They have different factors and processes involved, and they have to be measured and improved separately. You can't assess the ability of a page to hold someone in the site and the ability of that page to engage a new arrival in the same number. Including bounces in the Exit Rate makes the Exit Rate metric useless.

  • AdWords
    It is important to bear in mind that this error does not affect the assessment of key metrics for AdWords traffic. You pay for people to come to the site, whether they bounce or not, so cost-per-visitor and ROI for AdWords is not affected.

<< Previous page | Next page >>

 

Comments

The Imtiaz
The Imtiaz November 18, 2009 at 12:13 AM

Google Analytics is one of the best analytics tool with so many report dimensions. I agree that there is a slight discrepancy in reports. No other tool in the market gives these many tracking dimensions of a visitor. That too free of cost.

I disagree with your point below

A "visit" only happens when someone reads more than one page on a site. If someone comes to your site, looks at the first page, then leaves, this is not a "visit" -- it is a "bounce."

Google Analytics perfectly tracks one page visits. I have many one page websites. GA tracks every visit perfectly.

I have used many web analytics tools in past, but Google Analytics stands at its best.

The Imtiaz
Internet Strategist

The Imtiaz
The Imtiaz November 18, 2009 at 12:12 AM

Google Analytics is one of the best analytics tool with so many report dimensions. I agree that there is a slight discrepancy in reports. No other tool in the market gives these many tracking dimensions of a visitor. That too free of cost.

I disagree with your point below


Google Analytics perfectly tracks one page visits. I have many one page websites. GA tracks every visit perfectly.

I have used many web analytics tools in past, but Google Analytics stands at its best.

Regards,

The Imtiaz
Internet Strategist

Joao Correia
Joao Correia November 16, 2009 at 10:12 AM

Nothing in the world is 100% correct !
Web Analytics its not about the numbers its about the trends.
If you stick with the same web analytics you will be fine.

Regards
Joao Correia

Bernadette Homor
Bernadette Homor January 13, 2009 at 11:36 AM

I like your article, makes a lot of sense

Page alizer
Page alizer November 20, 2008 at 12:32 AM

You are absolutely right. This is why we created http://www.pagealizer.com . It is easier to optimize your page design if you know how long a visitor visited your page before bouncing. After all a average bounce after 5 seconds is different than an average bounce after a 1min visit - each bounce pattern should be addressed differently by site design and marketing. This is crucial in landing page design where in a lot of times there are no "second page views" and therefore common web analytics cant calculate visit length. This is where Pagealizer comes in. After adding Pagealizer tracking code to your site the code pings our servers as long as the visitor is viewing the page. This way you are getting real visit length in places a lot of web analytics services fail. check us out :)

Brandt Dainow
Brandt Dainow November 19, 2008 at 4:20 PM

REPLY TO JODI McDERMOT
Comments are not the place for debates, but Jodi has accussed me of lying in public, so I am forced to reply.

I was appointed WAA standards liason to the UK IAB and ABCE in 2006. Note that IAB is not a single international organisation, but a conglomerate of national ones. I met with Richard Foan and Andy Flint of ABCE in June 2006. Here is a copy of my report to the WAA committee:

I met with ABCE last week. They have confirmed their original agreement that WAA and ABCE co-operate on standards. The mechanism we have agreed is necessarily loose because ABCE is controlled by its member organisations. In particular, it is part of their brief to keep things simple for people who are outside the web analytics arena, and to create a limited set of standards which are easy to understand. This means they cannot agree to automatically adopt all WAA standards. Instead, when we publish our standards documents we will forward a copy to them, and they will then determine which ones they will adopt.

They will also forward to us their own standards document. This is currently around 40 pages, but much of the information within is stuff they want to keep confidential and restricted to their members (who have to pay for this document). They will remove this material and forward the rest to us for us to approve as we see fit. WAA is not obliged to approve any of their standards any more than they are obligated to approve ours. What we have agreed is that we will continually supply each other with lists of standards as we create them. We have also agreed we will publisd PR on our sites stating our position with regard to each other's standards so that we do not create the impression of a conflict.

ABCE have also undertaken to discuss this process at the JICWEBS level and drive the process of getting JICWEBS to develop a formal position with regard to WAA standards. ABCE are the secretary for JICWEBS.

JICWEBS is The Joint Industry Committee for Web Standards, and feeds standards to 36 countries for the purposes of online auditing via IFABC (International Federation of Audit Bureaux of Circulations). Standards are restricted to census-only data. JICIMS handles standards for panel research. Other members of JICWEBS are DMG - the IPA Digital Media Group, IAB - Interactive Advertising Bureau, IPA - Institute of Practitioners in Advertising, ISBA - Incorporated Society of British Advertisers, NPA - Newspaper Publishers' Association, NS - Newspaper Society, AOP - Association of Online Publishers.

Countries following JICWEB standards are: USA, Canada, Mexico, South Africa, China, India, Japan, Korea, Malaysia, Singapore, Sri Lanka, Austria, Belgium, Blugaria, Czech Republic, Finland, France, Germany, Hungary, Ireland, Italy, Netherlands, Norway, Poland, Portugal, Romania, Russia, Spain, Sweden, Switzerland, Turkey, UK, Australia, New Zealand, Argentina, and Brazil.

WAA has begun a similar conversation with IAB. Richard Foan, chair of ABCE, recommended WAA discuss standards with ISBA and IPA, and offered to make appropriate introductions. I think we should consider talking to EACA (European Association of Communications Agencies) as well.

Jodi McDermott
Jodi McDermott November 19, 2008 at 3:40 PM

Brandt,

I noticed on your bio that you state that you are the Web Analytics Association (WAA) liaison to the Internet Advertising Bureau. Can you please provide proof that this is true? I am asking as I am the first (and only) liaison to from the WAA to the IAB. In fact I helped to put that relationship together and personally announced that relationship at the E-metrics Summit in Toronto in March of this year.

I would appreciate a public reply as to your relationship with the WAA and the level of your involvement in the Standards committee.

Regards,

Jodi McDermott
WAA Liaison to the IAB
Co-chair of the Social Media Standards Subcommittee

Bob Mitchell
Bob Mitchell November 19, 2008 at 7:48 AM

'All WA standards' ?!
The first standard that comes to mind is from the WAA (of which you're a founder) which says :

TERM: Visits/Sessions
Type: Count
Universe: Aggregate, Segmented
[quote]
Definition/Calculation:
A visit is an interaction, by an individual, with a website consisting of one or more
requests for an analyst-definable unit of content (i.e. "page view”). If an individual
has not taken another action (typically additional page views) on the site within a
specified time period, the visit session will terminate.
[/quote]

So, single-view visits are still visits, right?

Now, unless I'm misreading something here (which is entirely possible), doesn't that contradict what you're claiming?

Note that I don't dispute that it's important to know what your tool is doing - that's always been vital.

Michael Martinez
Michael Martinez November 18, 2008 at 5:23 PM

Brandt Dainow: "According to all the web analytics standards, a "visit" only happens when someone reads more than one page on a site. If someone comes to your site, looks at the first page, then leaves, this is not a 'visit' -- it is a 'bounce.'"

Michae: Every Web site experiences 100% bounce because sooner or later, people leave the site. To argue that anyone who spends 3 minutes reading the front page of a blog and then leaves is a "bounce" is a pretty naive measurement of traffic.

There are certainly plenty of problems with Google Analytics, its under-reporting of traffic being the worst, but complaining about how they calculate bounce rates is grasping at straws.

If the objective of the Web site is to get people to visit one page and one page only (and there are MANY such sites), then there is no point in using any metrics package that doesn't track the traffic accurately.

Accurate traffic is not determined by nameless, uncited standards, but rather by the purpose and goals of the Web site.

Latham Arneson
Latham Arneson November 18, 2008 at 3:09 PM

I think to use the words "wrong" and "incorrect" is not being fair. Wrong or right is not really the case here, it's all a matter of how - as in, how the numbers are calculated.

You bring up an important point that it's important to know how you're analytics system is measuring the data it's presenting to you, but just because systems do it differently doesn't make it wrong or right.

To me, I actually like the fact GA reports a bounce as a visit. I want to know, at a very basic level, how many people were linked (however it happened) to my site. Once they're there, then I can analyze what they're doing and trying to get all of them to "convert". Getting rid of these users by not counting them would be just as "inaccurate" if you didn't understand how the system calculated the number.

We could debate the merits of whether it's worth counting bounced visits as "visitors", but my main point is that I strongly disagree with the notion that there is a wrong or right here and it's more a matter of how you look at it and your perspective - and most importantly, understanding how the numbers are calculated.

Bennet Bayer
Bennet Bayer November 18, 2008 at 8:32 AM

My goodness, you are just becoming aware of this now? Google does one thing well...put money in its own pocket.

I have blogged on the detail of market analytics and other sources such as your log but there are many others more cleaver than I doing this.

Sujit Nair
Sujit Nair November 18, 2008 at 8:16 AM

Nice detailed article !