The NextStage CRO explains how to define your Key Performance Indicators when you're tracking site traffic.

I recently finished three weeks of research for a client that I believe will interest many of you, hence I'll share it here. The research dealt with website traffic and the validity of different tracking technologies. I'll give you the answer now so you can decide ahead of time if you want to read the rest of this column: The most valid and reliable tracking technology is the one you commit to using and which gives you KPIs (Key Performance Indicators) that you can work with.

Accuracy versus Validity

Let's start by recognizing there's a difference between a tracking method's accuracy and a tracking method's validity. Let me give you some working definitions from the WordWeb tool:

• Accuracy: The quality of nearness to the truth or the true value
• Validity: The quality of being logically valid

Dictionary definitions are nice but sometimes they don't have a place in the real world. For example, tracking technologies tend to fall into two camps according to Brian Clifton of Omega Digital Media: client side and server side. Client side refers to data gathered from the client's browser; server side refers to data gathered in the website server's logs. Which one is more accurate?

If a tree falls in a forest...

Most people know the freshman philosophical question, "if a tree falls in a forest and there's nobody there to hear it, does the tree make a sound?" What most people don't know is that there's a fallacy in the question. The tree doesn't make the sound when it falls; it's the act of falling which will or will not be heard. A similar website traffic tracking question is, "if a page is requested from a server but that page is never viewed by a person in a browser, should you expect to get a conversion?" It doesn't matter how many pages are requested, it only matters if you, the website owner, are getting the results (be they conversions or something else) that you want.

Or does it? This goes into accuracy versus validity. What are your KPIs based on? That, to me, is the more important question. Right next to it is, "which KPIs are you going to stick to over time?" I agree with Brian Clifton's statement about client side and server side tracking: "it is important to note that both techniques, when considered in isolation, have their limitations."

Moving from one tracking technology to another can often have surprising results. You thought you were doing well, and now you're not. You thought you were doing poorly and discover you're the head of your class. Let me share with you what accuracy and validity are to me.

Accuracy and Validity are to "makes sense" and "logically defendable" as...

Accuracy and validity have to share two elements: they both have to make sense within their paradigm, and they both have to be logically defendable. In both cases, my motto is the same as President Reagan's when negotiating with the Soviets, "Trust, and verify."

Let's start with "logically defendable."

Are your KPIs traceable to a consistent set of numbers? Do your KPIs show that your website traffic varies # 50 percent per month? Okay, no problem: just make sure that you can verify that somehow. It doesn't matter if you're using client side or server side tracking; all that matters is that the 50 percent monthly variation can be verified by some numbers you're willing to trust. This proves that your numbers are logically defendable.

Now what about "makes sense within their paradigm?"

Do other sites in your market experience the same fluctuations? Yes? Okay, your numbers make sense within their paradigm. No? Time to start checking around; there's a problem somewhere.

From the above you may get the idea that "accuracy" and "validity" have more to do with the market you're in than they have to do with the hard numbers they generate. Well, yes and no. Once again, it goes back to the KPIs you're going to work with.

Are you going to commit yourself to hard numbers? Good, go for it. Are you going to go with market statistics? Good, go for it.

Are you going to go for both, distill their essence and make business decisions based on all the information available? Excellent!

Time and effort

Distilling from all possible information sources will take time and effort. There's no getting away from that. Of course, the up side is that you'll learn more about your business and how to do your business on the web. You'll also quickly figure out which data sets are giving you information that you can turn into reliable, measurable results.

Even better, you'll develop a gauge that will indicate which information should get a look and which isn't worth your time. Do remember, though, that the true function of such a gauge isn't to ignore an information source. The function of this gauge is to alert you that something -- even when it's from a source usually ignored -- is worth your attention.

Some rules for making sense and being logically defendable

Regardless of your KPI strategies, here are some quick rules to keep in mind when deciding which numbers to focus your time and energy on:

1. Is the data being gathered self-consistent over time?
2. Is the data being gathered consistent in your market?
3. Is the data being gathered similar to what others in your market are experiencing who are using the same data-gathering methods you're using?
4. Can variations in any of the above be logically explained by things like geography, seasonality, disruptive and introduced technology...?

The answers to the above should both make sense and be logically defendable. They should also be self-accurate and self-validating.

Self-accurate and self-validating? Yes.

Think of it this way: "t-shirts and shorts are excellent outerwear." Everyone in Toronto reading this column when it's first published (Feb 2006) who is willing to spend the day outside in just a t-shirt and shorts, say "Aye!"

Your KPIs have to be accurate and valid within the paradigm that generates them. T-shirts and shorts are excellent outerwear when the outside air temperature is somewhere near or above body temperature (my opinion, of course). Otherwise, give me pants and shirts with full length sleeves.

As I said in the opening to this column, the most valid and reliable tracking technology is the one you commit to using and that gives you KPIs you can work with. Anything else is...well...

Joseph Carrabis has been everything from butcher to truck driver to Senior Knowledge Architect to Chief Research Scientist. His 22 books and 225 articles have ranged among cultural anthropology, mathematics, information mechanics, language acquisition, neurolinguistics, psychodynamics and psychosocial modeling, and other eclectic topics. His knowledge and data designs have been used by Caltech, Citibank, DOD, IBM, NASA, Owens-Corning and Smith-Barney among others. Carrabis is CRO and Founder of NextStage Evolution and NextStage Global, and founder of KnowledgeNH and NH Business Development Network. He is also the inventor and developer of Evolution Technology. You can download sections of Carrabis' next book, "Reading Virtual Minds," at www.hungrypeasant.com.