If your analytics systems are any good, they should tell you things you didn’t know before. Sometimes what they tell you will be bad news. Many of the things you analyze will be the results of decisions you made. This means that sooner or later you’re going to get bad news about a decision you made. In other words, you made a mistake.
This article is a plea to take it on the chin.
Most of my work is analyzing other people’s websites and telling them how to improve. This work isn’t all about enhancing and extending what they’re doing; often improvement means removing problem components. Sometimes people get attached to these and don’t like to hear what I’m saying.
For example, I was working with a client who had a travel site. The home page hadn’t changed for years, was very text heavy, and didn’t look very good. It was exactly the sort of thing we were all building in the late 1990s. He called in a designer and spent serious money getting the page replaced with a good looking new one, nice graphics, very clean layout, crystal-clear navigation, but not much text. After a week I had to tell him that the bounce rate for the new page was 50 percent, compared with only 10 percent for the old page. In other words, the new page was a step back, not forward. It looked better, but because it didn’t have as much information as the old one, people were not bothering to enter the site.
The conversation went something like this:
Me: “The bounce rate for the new page is five times what it was for the old one.”
Him: “But it looks much better than the old one.”
Me: “I agree, it looks great. But the bounce rate is 50 percent, instead of the old 10 percent.”
Him: “But this design cost me a fortune.”
Me: “I know, and it looks great, but it’s also more expensive to own, it’s costing you sales, a lot of sales.”
Him: “But I made a big deal of introducing this to all the sales staff, I can’t go back now.”
Me: “Well, maybe we can fix it. What if we made some modifications to the new page?”
Him: “I can’t do that; we’ve spent the entire budget getting this far.”
Me: “Look, a 10 percent bounce rate on the old page is really good; you don’t want to lose that.”
Him: “Yes, but the new page looks much better than the old one.”
The new page is still there, and it’s still costing them sales. But it sure looks great.
This conversation contains three of the key obstacles people have to accepting what the stats are telling them: emotional attachment to a feature, pride (or politics) which prevents them from reversing a position, and poor budgetary analysis. Let’s deal with these one at a time.
"The new design looks so much better than the old."
The instant you hear or think that, ask yourself: “Am I an art gallery or a business?”
It doesn’t matter what your site looks like.
It makes no difference at all how good it looks. Design is only a tool to make money. Colors, layout, artwork and any other visual feature you can think of are only important to the degree that they enhance or reduce your conversion rate. If a lousy design with horrible colors gets you more sales -- do it.
Remember: Most people will not look at your pages as closely as you will. And they don’t care anyway.
It doesn’t matter whether you like the design or not.
The site’s not for you. It’s for your customers. It’s their money you want, not yours. You’ve already got your money (well, maybe your spouse has it, or your kids). Believe it or not, most people will not pay as much attention to what your site looks like as you.
The only emotional reaction you care about is the one your customers have, not yours. And the only thing your customers care about is: can you give them what they want for a good price?
"But I’ve invested too much money in this design/site/feature"
Gee, life’s tough isn’t it? You spend lots of money on a new design, or component, or site, and those stupid customers perversely refuse to buy. Refusing to back down, or adapt, or invest in more changes, doesn’t minimize your losses -- it increases them.
If you’ve spent money on something that is resulting in a lower conversion rate than before, it’s going to cost you more money. When you budget for construction, always try to budget cost of ownership. In my experience, cost of ownership determines profitability or loss much more often than cost of construction.
If you deploy a new feature and it halves your conversion rate, the cost of ownership is double what it was. Continuing to leave it there may or may not recoup the cost of development, but removing it will definitely do so faster.
No matter how bad the old situation was, if the new one is worse, returning to the old one is an improvement.
"Pride/Politics won’t let me ..."
I was once told that no IT project over $2 million ever fails.
Imagine that: The biggest determinant of success is the size of the budget.
Or could it be that no one can stand up before their boss and admit project failure for fear of the consequences? Or no boss can stand before his staff and admit a mistake? Or that we just can’t admit it to ourselves?
Maybe the problem is the way we see things. No one intentionally sets out to fail. We all launch new initiatives expecting success. If things don’t work out the way we expected, we have an opportunity to learn something new. No one learns from their successes, we only learn from our mistakes -- if we let ourselves.
It’s usually the case that a detailed analysis of what isn’t working will tell you more than looking at what is working.
There are ways of minimizing potential negative impact. A/B testing is the ideal, and you should do it if you can. But A/B testing takes time and money, and many changes are not worth the investment. However, if you stop to think about it, changing one thing for something else is a form of A/B testing. You’re simply presenting the two options over time instead of at the same time.
Reducing the budget can help. The less expensive a development, the easier it is to throw it away. Try to allocate something for fine-tuning. Don’t assume that the first attempt will be perfect.
The most important step is your attitude. Scientists spend their lives conducting experiments. Most of them fail most of the time, but since science is based on objective assessment of facts, scientists don’t get upset. They use structured methodologies to analyze the results of the experiment and learn from them. If you’re using web analytics to understand your website, you’re asking for a mathematical assessment. That’s science, and it requires an objective assessment of what the numbers are telling you.
Brandt Dainow is CEO of Think Metrics, creator of the InSite Web reporting system. Read full bio.