Can you identify the single biggest threat to your site's metrics? The answer might surprise you.
I am sure you are aware of the vast amount of junk email flooding the web these days. No matter how good your junk email filter systems are, I'm sure a certain amount gets through. If you have really tough protection, you probably find the occasional genuine email gets caught in the net and junked as well. We all know that the people who generate this stuff are the lowest form of pond life, often with criminal backgrounds or connections.
What I have been discovering lately is that these people are also subverting our web analytics systems. In some cases their activities are distorting web stats so badly as to completely invalidate some metrics. Worse still, there is absolutely nothing we can do about it.
Do you send junk email?
Many people are sending out junk email without realizing it. Web server administrators are fighting a desperate battle with junk emailers, a battle I think they're losing. The issue is not about keeping junk email from getting to you. The issue is about stopping junk emailers from using your server to send their junk.
When junk email first appeared it came from the sender's own computers. The defense was therefore to block all email coming from those machines. Most junk email defense systems contain lists of blocked IP addresses and domain names, called "filter lists," which are updated frequently.
The obvious way around this is to keep getting new computers from which to send your junk email. However, this is not so easy. Junk email works on a thin margin, and constantly finding new computers is costly both in time and labor. In addition, most hosting companies are not all that keen to provide services for junk emailers. Hosting companies learned some years ago that if they don't prevent junk emailers from using their servers, their entire network ends up on the filter lists, and none of their customers can send out email. ISPs have a tendency to close access to junk emailers for the same reason.
Since all junk emailers are amoral at best, and often just plain criminal, their solution was simple: take over someone else's web server and use that to send out the junk email. This has been the focus of much activity for the last few years. The junk emailers use software to automatically probe large numbers of web servers for vulnerabilities at rapid speed.
If they find one that they can subvert, then their systems will start pouring junk email through it as fast as possible until they get caught. If they don't get caught and stopped quickly they sell this information to other junk emailers so those people can use the same machine.
There is a constant war now underway on the web. It involves junk emailers trying to subvert web servers and network administrators trying to stop them. I have been involved in defending my own servers and those of my clients from a number of attacks. I consider this a monumental waste of my time, but it has been educational.
From what I can see the junk emailers are winning. Web servers are complex machines, and the range of potential vulnerabilities is huge. In addition, you need to be a real expert to lock down a server with any real degree of security, and such experts are rare and extremely expensive. The skills required are well beyond the scope of even a senior technician in the average hosting company. This is such a complex and fast-moving area it is unreasonable to expect your typical .php or .net developer to have even the vaguest comprehension of how to build really secure web sites.
As a result, the general body of knowledge in the online developer community does not give more than cursory consideration to what constitutes best practice when it comes to web security.
Many companies never realize they are sending junk email until they find they can't send legitimate email out or that their servers are clogged with unsent junk emails.