ThinkMetrics' CEO discusses the importance of establishing international analytic standards and ensuring vendors support and implement them.
There are very few standards in the field of web analytics. The field is relatively new; it was only a few years ago we were trying to decide what to call it ("e-metrics" was the favorite until people realized it was too close to "emetics"). This is why most online activity is still not measured at all, and why there are few standards. At this time there is little agreement on what should be measured. Where there is agreement, there are often differences in how that metric should be calculated. In extreme cases, the same system may vary in different places, according to the preferences of the individual programmer who wrote that section of code. This means some systems can generate different numbers for the same thing, depending on how that number is accessed by the user. In other cases, the same metric is given different names by different systems, leading to confusion over terminology.
In practice, the degree of confusion is bearable because there are not many software vendors in the marketplace. However, if you find yourself in a dispute over numbers, you'll often find this is because they were calculated in different ways. It's always advisable to begin any such dispute by comparing methods of data gathering and calculation. If you do have such a conversation, you need to anticipate that some issues may occur for which we do not have answers yet. One which keeps coming up for me at the moment is "how much does it matter where the tracking tag is placed?" I know placing JavaScript tracking tags in the <HEAD> is less accurate than in the <BODY>. I'm pretty sure the degree of variation depends on browser and operating system, but I couldn't put numbers to it. There is even less certainty about the amount of difference placing the tag at the top or bottom of the <BODY> makes, but many people suspect it does matter. Needless to say, there is no standard about how this should be handled, and vendors are unlikely to provide even product-specific guidance-- most of them aren't aware this is an issue.
Agreed international standards are essential to ensure we all measure the same things, and do so in the same way. At the same time, it's important to ensure vendors support these standards, and implement them in their software.
There are a number of bodies involved with the development of standards for web metrics; the Audit Bureau of Circulation Electronic or ABCE (www.abce.org.uk), Joint Industry Committee for Web Standards or JICWEBS (www.jicwebs.org), and the International Web Analytics Association or WAA (www.webanalyticsassociation.org).
ABCE and JICWEBS
Initial development of web metrics came out of online advertising in the 1990s. Since online advertising needed to prove its numbers in order to get paid, it was the first industry to see a need for metrics. Hence the first organization to form and develop standards was ABCE.
ABCE is based in London. It is formed from a coalition of large websites, agencies and a few of the larger web metrics vendors. In the late 1990s it developed a set of standards focused around the measurement of ad delivery. For the last few years ABCE's primary role has been using those metrics for auditing. ABCE will also certify metrics systems as meeting their standards. This means they will trust the figures the software provides when they audit. This doesn't mean they will refuse to audit your site if you use a different product; it just means you'll get a discount on the audit fees if you use a product they have certified.
It is important to understand that ABCE is not interested in providing a comprehensive set of metrics for measurement of all online activity. They are "owned" by their members who determine their agenda. Their members are in the business of online advertising, so the only metrics they need are those related to ad delivery. The majority of the definitions created by ABCE are of little use for most site analysis and include email distribution, interactive TV, PDA activity and so forth. However, some of the fundamental web metrics definitions, such as visit, page impression and unique visitor come from ABCE.
JICWEBS is the real heavy hitter in this field. ABCE provides the definitions for JICWEBS. JICWEBS standards are fed into the International Federation of Audit Bureau of Circulations, or IFABC. IFABC standards are supported in 34 countries, including the United States, Canada and most of Europe.
Not all ABCE standards are supported by JICWEBS at present. The web metrics defined at JICWEBS are Page Impression, Unique User, Visit, Visit Duration, Search, Stream, Stream Duration, Ad Impression and Ad Click. Notice there is no definition here for behavior over time (such as Repeat Visitors), anything related to browsers (such as User Agent) or anything related to links (such as Referrer). If you want to calculate frequency capping over longer time periods, or traffic between sites, you're not going to be dealing with anything which has an agreed standard.
ABCE and JICWEBS had effectively ceased development of standards by 2002. They had enough for their purposes, and extending them any further would have placed pressure on their members. Sources inside ABCE told me at the time members had no desire to create metrics that could compromise the income of advertising outlets or force analytics vendors to create new features in their products. Remember, the aim of ABCE and JICWEBS is to support auditing-- to enable you to compare two potential outlets on a common basis; not to create a detailed, or even accurate, picture of online activity. It doesn't matter if the numbers are a little inaccurate, so long as the inaccuracy is the same for everyone.
WAA
The International Web Analytics Association was formed in 2005. Part of the reason for formation was to provide standards for web metrics. Most web analytics professionals recognized the limitations in ABCE standards, but unless you lived in London, were prepared to pay over $5,000 a year to join ABCE, and were ready to fight the established status quo, there wasn't much you could do about it.
The WAA Standards Committee contains a variety of members representing the complete spectrum of those who work with web metrics, ranging from senior analysts at Yahoo!, to private consultants and ad people. It started by adopting most of the relevant JICWEBS standards. Since then, the committee has been quietly developing new standards, meeting once or twice a month in an online conference to argue and debate, then finally agree. It could be said they have been a little too quiet, in that they have neglected to publish the standards they have agreed upon. In late 2005 two members, Jason Burby and Guy Creese, published an excellent discussion paper containing a proposed list of KPI's on the WAA site. The standards committee has since been working on ratifying or tuning some, but not all, of those metrics. The initial set of standards they aimed to ratify has been completed, but as yet they have not been published.
The standards committee is now working on the next set of terms. These are Page Views per Visit, Page Exit Ratio, Call to Action Conversion, Average Order Value, Average Items per Order, Visits per Visitor, Frequency, Recency, % of Content and Cost (per campaign, visit, transaction, et cetera). It is to be hoped they will soon publish the list of standards they have already ratified in a definitive document.
The future
The creation of WAA prompted ABCE to start developing standards again, and for a while it looked like ABCE and WAA would start creating conflicting standards. However, both organizations have recently agreed to work together to prevent this happening. At this stage the agreement is nothing more than a statement of principle. It remains for these organizations to develop formal methods for regular liaison, and to define mechanisms for joint ratification.
Bearing in mind that there are very few published standards, it is not surprising that vendors of analytics software have each been doing things their own way. WAA is just now starting to think about creating methods of certification for analysts, and I would expect this to extend to software at some stage in the next few years.
In the longer term we can expect a full and comprehensive set of standards to emerge, though this will take several years. If you are working with web analytics at this time, bear in mind you are working in a field where most people can legitimately "do their own thing." If you find someone disagrees with you about key numbers, start by establishing a common language. If JICWEBS and WAA can't give you the metrics you need-- agree on some standards of your own.
Brandt Dainow is CEO of Think Metrics, creator of the InSite Web reporting system. Read full bio.
Advertisement