ellipsis flag icon-blogicon-check icon-comments icon-email icon-error icon-facebook icon-follow-comment icon-googleicon-hamburger icon-imedia-blog icon-imediaicon-instagramicon-left-arrow icon-linked-in icon-linked icon-linkedin icon-multi-page-view icon-person icon-print icon-right-arrow icon-save icon-searchicon-share-arrow icon-single-page-view icon-tag icon-twitter icon-unfollow icon-upload icon-valid icon-video-play icon-views icon-website icon-youtubelogo-imedia-white logo-imedia logo-mediaWhite review-star thumbs_down thumbs_up

The disturbing inaccuracy behind Google Analytics

The disturbing inaccuracy behind Google Analytics Brandt Dainow
VIEW SINGLE PAGE

Google Analytics is probably the most popular web analytics tool in use today. It is certainly the best tool you can get for the price -- it's free and worth every penny. Unfortunately, like every web analytics tool, it is not 100 percent accurate. There are reasons why it is impossible to build perfectly accurate web analytics software, which I have covered previously in "Things that throw your stats," so we should not blame Google for this. However, Google Analytics is different from other products in that it has been intentionally designed by Google to be inaccurate over and above the normal inaccuracies that are inevitable. These inaccuracies are so glaring that most people are getting a very false picture of what is happening in their sites. This article will explain where these inaccuracies lie and provide methods to correct for them.

Visits and bounces
Firstly, we must remember that Google did not create Google Analytics.  The company bought a product called Urchin and rebranded it. After about a year, it then released an upgraded version. Urchin had contained a very big mistake in its calculations, which Google fixed in the upgrade. The problem lay with counting visits. According to all the web analytics standards, a "visit" only happens when someone reads more than one page on a site. If someone comes to your site, looks at the first page, then leaves, this is not a "visit" -- it is a "bounce." The best way to think of this is in terms of a shop. If someone looks at the store window but moves on, they have bounced. It is only when they enter the store (or site) that you have a visit.

This is a very important definition to grasp because it fundamentally affects how you see the performance of the site. The reason we need to distinguish between bounces and visits is that we can't tell how long someone spent looking at a web page. We can only tell what time they accessed it. We calculate the time spent looking at a page by comparing what time they accessed it with what time they accessed the next page.

For example, if someone accesses my homepage at 2 p.m., then accesses the next page at 2:15 p.m., we assume they spent 15 minutes reading the homepage. If someone only accesses one page -- in other words, if they bounce -- we have no way of knowing how long they spent looking at that one page. So you can only calculate visit duration when someone reads two or more pages.

A key metric when analyzing the performance of a site is the average duration -- how long people are spending in the site. You need to know your overall average duration for the site, and you often want to know the average duration for visits from certain ads or for specific products. It's a very good way of assessing engagement. Average duration is calculated by totaling up all the individual visit durations and dividing this by the total number of visits.

Next page >>
At the time Google purchased the software Urchin, it treated (and counted) bounces as 0-duration visits. When calculating the average duration, this meant that it came up with a number that was under the real one. How inaccurate this was depended on the percentage of a site's arrivals who bounced.

In July 2007, Google changed the calculation of average duration so that it did not include bounces anymore. This was the correct thing to do and meant that its figures for average duration were now accurate. A month later, Google put it back to the old (wrong) way of calculation. In other words, Google intentionally rolled Google Analytics back so that it produced an incorrect average duration. Why? Brett Crosby, senior manager at Google Analytics, explained in a Google blog that it was because people complained the change meant the new (accurate) numbers were out of line with the old (inaccurate) ones. (That blog has since been removed by Google, but you can find it copied on many sites.) In other words, these people considered consistency more important than accuracy, and Google obliged them.

It's been that way ever since -- Google is intentionally and knowingly providing inaccurate numbers because a few people preferred neatness to truth.

Areas of error
Treating bounces as visits doesn't just affect the accuracy of average duration -- it affects any metric based on the number of visits. 


  • Visit count
    When you see a figure in your Google Analytic reports for Total Visits, ask yourself: What do you think that represents? If you see Total Visits as the number of people who entered your site, who reacted to the sales pitch, who engaged with your content, who potentially could have bought products, then you are wrong. It is the number of people who arrived at the front door of the site, nothing more.


  • Conversion rate
    The Conversion Rate tells me how successful my site is at selling. It is legitimate to calculate Conversion Rate including bounces, but my personal experience is that it is misleading to do so. I use Conversion Rate to improve my site's sales pitch. People who bounce were never exposed to it, so including them in the calculation means I cannot possibly know whether my sales pitch is working or not.


  • Exit rate
    Google Analytics tells me how many people exited the site from any given page. Like Conversion Rate, this is useful for assessing the sales performance of the page. However, getting someone to enter the site and getting them to stay in it once they have entered are two very different tasks. They have different factors and processes involved, and they have to be measured and improved separately. You can't assess the ability of a page to hold someone in the site and the ability of that page to engage a new arrival in the same number. Including bounces in the Exit Rate makes the Exit Rate metric useless.


  • AdWords
    It is important to bear in mind that this error does not affect the assessment of key metrics for AdWords traffic. You pay for people to come to the site, whether they bounce or not, so cost-per-visitor and ROI for AdWords is not affected.

Correcting Google Analytics
If I have inaccurate metrics that are being used by someone to assess performance, I like to aim for minimal disruption when I correct them. I therefore leave the existing metrics in place and add a few (accurate) ones. I have found the best way to correct for this problem is to add a metric I call "Retained Visits," leaving the existing Total Visits in place. I can then re-calculate my metrics using Retained Visits.

I get my number of Retained Visits by removing the bounces. For example, if Total Visits was 1,000 and the Bounce Rate was 25 percent, I had 250 bounces, so the number of Retained Visits was 750.

Mathematically, we express this as:


RV = TV – (TV * BR)


where:
RV is Retained Visits
TV is Total Visits
BR is Bounce Rate

Correcting average duration
In order to get the genuine Average Duration, I need to rewind the calculation Google made about half way. Google Analytics calculated Average Duration by adding up all the individual durations and then dividing that by the number of visits (including bounces). I rewind this by multiplying the Average Duration by the Total Visits. That gives me the number Google had before it factored in bounces, which we can call Total Duration. (Remember that the bounces, with durations of zero, contributed nothing to this number.) I then redo the calculation of average using Retained Visits, thus:


TAD = (AD * TV) / RV


where:
TAD is True Average Duration
AD is Average Duration (as reported by Google Analytics)
TV is Total Visits
RV is Retained Visits


It is surprising, even terrifying, how far apart the old and new numbers are, and the difference in the picture this provides. Here is a set of figures from a site that averages around 30 percent bounce rate:



These show that rather than an average engagement of around three minutes, the reality is closer to five -- people are spending almost twice as long on the site as Google Analytics says they are.



You can use Retained Visits to recalculate your Conversion Rate in a similar fashion. While I keep my calculations of AdWords ROI untouched, I also add another one based on Retained Visits. I have found that if you break down the ROI on each advertising campaign (or ad) by Retained Visits, you can often find extremely valuable revenue streams concealed inside a bland total.

Conclusion
It is critical to know how any web metrics package calculates its numbers. You cannot assume, no matter how big the company, that the numbers will be correct. A small variation doesn't make much difference, but you could be getting a completely false view of what is happening in your site. You may need to correct those numbers yourself. I recommend the use of Retained Visitors as a good way of correcting for errors in Google Analytics.

The other conclusion we can draw from this is how limited the understanding of web analytics is among practitioners, and what a low priority Google gives Google Analytics. Can you imagine an accountant insisting on using inaccurate numbers in their bookkeeping simply because that's what they had been doing before? How long do you think that business would survive? Can you imagine Google agreeing to downgrade the accuracy of its search results to meet user demand? On second thought, that's exactly what they agreed to do in China, isn't it?



Brandt Dainow is an independent web analytics and marketing consultant working in the U.K. and Ireland.

Brandt is an independent web analyst, researcher and academic.  As a web analyst, he specialises in building bespoke (or customised) web analytic reporting systems.  This can range from building a customised report format to creating an...

View full biography

Comments

to leave comments.

Commenter: The Imtiaz

2009, November 18

Google Analytics is one of the best analytics tool with so many report dimensions. I agree that there is a slight discrepancy in reports. No other tool in the market gives these many tracking dimensions of a visitor. That too free of cost.

I disagree with your point below

A "visit" only happens when someone reads more than one page on a site. If someone comes to your site, looks at the first page, then leaves, this is not a "visit" -- it is a "bounce."

Google Analytics perfectly tracks one page visits. I have many one page websites. GA tracks every visit perfectly.

I have used many web analytics tools in past, but Google Analytics stands at its best.

The Imtiaz
Internet Strategist

Commenter: The Imtiaz

2009, November 18

Google Analytics is one of the best analytics tool with so many report dimensions. I agree that there is a slight discrepancy in reports. No other tool in the market gives these many tracking dimensions of a visitor. That too free of cost.

I disagree with your point below


Google Analytics perfectly tracks one page visits. I have many one page websites. GA tracks every visit perfectly.

I have used many web analytics tools in past, but Google Analytics stands at its best.

Regards,

The Imtiaz
Internet Strategist

Commenter: Joao Correia

2009, November 16

Nothing in the world is 100% correct !
Web Analytics its not about the numbers its about the trends.
If you stick with the same web analytics you will be fine.

Regards
Joao Correia

Commenter: Bernadette Homor

2009, January 13

I like your article, makes a lot of sense

Commenter: Page alizer

2008, November 20

You are absolutely right. This is why we created http://www.pagealizer.com . It is easier to optimize your page design if you know how long a visitor visited your page before bouncing. After all a average bounce after 5 seconds is different than an average bounce after a 1min visit - each bounce pattern should be addressed differently by site design and marketing. This is crucial in landing page design where in a lot of times there are no "second page views" and therefore common web analytics cant calculate visit length. This is where Pagealizer comes in. After adding Pagealizer tracking code to your site the code pings our servers as long as the visitor is viewing the page. This way you are getting real visit length in places a lot of web analytics services fail. check us out :)

Commenter: Brandt Dainow

2008, November 19

REPLY TO JODI McDERMOT
Comments are not the place for debates, but Jodi has accussed me of lying in public, so I am forced to reply.

I was appointed WAA standards liason to the UK IAB and ABCE in 2006. Note that IAB is not a single international organisation, but a conglomerate of national ones. I met with Richard Foan and Andy Flint of ABCE in June 2006. Here is a copy of my report to the WAA committee:

I met with ABCE last week. They have confirmed their original agreement that WAA and ABCE co-operate on standards. The mechanism we have agreed is necessarily loose because ABCE is controlled by its member organisations. In particular, it is part of their brief to keep things simple for people who are outside the web analytics arena, and to create a limited set of standards which are easy to understand. This means they cannot agree to automatically adopt all WAA standards. Instead, when we publish our standards documents we will forward a copy to them, and they will then determine which ones they will adopt.

They will also forward to us their own standards document. This is currently around 40 pages, but much of the information within is stuff they want to keep confidential and restricted to their members (who have to pay for this document). They will remove this material and forward the rest to us for us to approve as we see fit. WAA is not obliged to approve any of their standards any more than they are obligated to approve ours. What we have agreed is that we will continually supply each other with lists of standards as we create them. We have also agreed we will publisd PR on our sites stating our position with regard to each other's standards so that we do not create the impression of a conflict.

ABCE have also undertaken to discuss this process at the JICWEBS level and drive the process of getting JICWEBS to develop a formal position with regard to WAA standards. ABCE are the secretary for JICWEBS.

JICWEBS is The Joint Industry Committee for Web Standards, and feeds standards to 36 countries for the purposes of online auditing via IFABC (International Federation of Audit Bureaux of Circulations). Standards are restricted to census-only data. JICIMS handles standards for panel research. Other members of JICWEBS are DMG - the IPA Digital Media Group, IAB - Interactive Advertising Bureau, IPA - Institute of Practitioners in Advertising, ISBA - Incorporated Society of British Advertisers, NPA - Newspaper Publishers' Association, NS - Newspaper Society, AOP - Association of Online Publishers.

Countries following JICWEB standards are: USA, Canada, Mexico, South Africa, China, India, Japan, Korea, Malaysia, Singapore, Sri Lanka, Austria, Belgium, Blugaria, Czech Republic, Finland, France, Germany, Hungary, Ireland, Italy, Netherlands, Norway, Poland, Portugal, Romania, Russia, Spain, Sweden, Switzerland, Turkey, UK, Australia, New Zealand, Argentina, and Brazil.

WAA has begun a similar conversation with IAB. Richard Foan, chair of ABCE, recommended WAA discuss standards with ISBA and IPA, and offered to make appropriate introductions. I think we should consider talking to EACA (European Association of Communications Agencies) as well.

Commenter: Jodi McDermott

2008, November 19

Brandt,

I noticed on your bio that you state that you are the Web Analytics Association (WAA) liaison to the Internet Advertising Bureau. Can you please provide proof that this is true? I am asking as I am the first (and only) liaison to from the WAA to the IAB. In fact I helped to put that relationship together and personally announced that relationship at the E-metrics Summit in Toronto in March of this year.

I would appreciate a public reply as to your relationship with the WAA and the level of your involvement in the Standards committee.

Regards,

Jodi McDermott
WAA Liaison to the IAB
Co-chair of the Social Media Standards Subcommittee

Commenter: Bob Mitchell

2008, November 19

'All WA standards' ?!
The first standard that comes to mind is from the WAA (of which you're a founder) which says :

TERM: Visits/Sessions
Type: Count
Universe: Aggregate, Segmented
[quote]
Definition/Calculation:
A visit is an interaction, by an individual, with a website consisting of one or more
requests for an analyst-definable unit of content (i.e. "page view”). If an individual
has not taken another action (typically additional page views) on the site within a
specified time period, the visit session will terminate.
[/quote]

So, single-view visits are still visits, right?

Now, unless I'm misreading something here (which is entirely possible), doesn't that contradict what you're claiming?

Note that I don't dispute that it's important to know what your tool is doing - that's always been vital.

Commenter: Michael Martinez

2008, November 18

Brandt Dainow: "According to all the web analytics standards, a "visit" only happens when someone reads more than one page on a site. If someone comes to your site, looks at the first page, then leaves, this is not a 'visit' -- it is a 'bounce.'"

Michae: Every Web site experiences 100% bounce because sooner or later, people leave the site. To argue that anyone who spends 3 minutes reading the front page of a blog and then leaves is a "bounce" is a pretty naive measurement of traffic.

There are certainly plenty of problems with Google Analytics, its under-reporting of traffic being the worst, but complaining about how they calculate bounce rates is grasping at straws.

If the objective of the Web site is to get people to visit one page and one page only (and there are MANY such sites), then there is no point in using any metrics package that doesn't track the traffic accurately.

Accurate traffic is not determined by nameless, uncited standards, but rather by the purpose and goals of the Web site.

Commenter: Latham Arneson

2008, November 18

I think to use the words "wrong" and "incorrect" is not being fair. Wrong or right is not really the case here, it's all a matter of how - as in, how the numbers are calculated.

You bring up an important point that it's important to know how you're analytics system is measuring the data it's presenting to you, but just because systems do it differently doesn't make it wrong or right.

To me, I actually like the fact GA reports a bounce as a visit. I want to know, at a very basic level, how many people were linked (however it happened) to my site. Once they're there, then I can analyze what they're doing and trying to get all of them to "convert". Getting rid of these users by not counting them would be just as "inaccurate" if you didn't understand how the system calculated the number.

We could debate the merits of whether it's worth counting bounced visits as "visitors", but my main point is that I strongly disagree with the notion that there is a wrong or right here and it's more a matter of how you look at it and your perspective - and most importantly, understanding how the numbers are calculated.

Commenter: Bennet Bayer

2008, November 18

My goodness, you are just becoming aware of this now? Google does one thing well...put money in its own pocket.

I have blogged on the detail of market analytics and other sources such as your log but there are many others more cleaver than I doing this.

Commenter: Sujit Nair

2008, November 18

Nice detailed article !