SearchTHIS: Can You Forecast Search?

There’s a long list of things that drive me crazy. Topping the inventory are sentences and phrases that include the word “can’t.” For instance, when I hear things like, “Mr. Ryan we can’t upgrade you for this flight,” or “Kevin, I can’t seem to find your reservation.” Perhaps my personal favorite is, “Kevin we can’t give you a raise this year, but we’d like you to take on more responsibility.”

The most famous “can’t” I hear in pay for placement search is, “We can’t assign a budget to SEM.” That statement is usually followed by the explanation, “because it is impossible to predict search behavior.” Impossible you say? I say, "no estoy comprendo."

For most marketers today, finding the money for SEM isn’t the problem. Site owners know search is important, natural optimization costs can be aptly categorized as predictable, but assigning a final budget figure to paid search becomes a bit difficult. The word “final” might be the problem since search predictions aren’t an exact science. Maybe “flexible” might be a better word but “flexible” doesn’t have to be “painful,” or “impossible.”


At the core of every budget forecast is a list of keywords. At best, choosing and developing keyword lists is an ongoing process. A smart place to start is in site side analytics. Chances are you already know which keywords users enter to find your site with referrer information analysis. That list of keywords is your launching point and from there you can begin to develop your first entry points.

Keyword suggestion tools are the next step in the process. Many pay for placement providers have their own such as Overture and Google, which both offer keyword suggestion tools but you don’t have to stop there.

Probably the most well-known keyword selection tool beyond the provider-based realm is wordtracker and for search beyond the American borders you will find help for international search terms at dWoz.

As you move through this process, remember to include what I call stupid keywords. The most overlooked or misunderstood area of keyword selection is keywords or phrases that show little or no search activity. These phrases include variations of brand or product names, misspellings (though many search engines will suggest alternative words, you shouldn’t rely on them) which may not even appear in keyword suggestion tools. These include “Dodgeball,” Dodge [space] ball,” “Harley Davison,” and “Trao 600.” Even though many of these terms will show little or no search activity, they can and will receive motivated traffic. You know what they say about stupid people and their money -- and I should know.

Once you have keyword sets, you should begin to assign weights to these phrases and place them into categories (final weights will be determined by actual post-click performance.) Next, assign specific landing pages to each keyword and keyword grouping.

During the preliminary weighting process you should be determining the importance of these keywords to help with the initial budgeting process. For example, generic terms might receive a lower weight (higher traffic) or budget assignment than brand or product (lower traffic) specific terms since conversion or desired action rates will be much higher for terms associated with brand or product.

Not so easy

Toolsets for predicting click traffic into keyword sets have gotten a lot better since the GoTo.Com Duck first crossed our path’s back in 1997 but for the most part, search marketers are still guessing. Keyword suggestion tools are good indicators of static search activity, assuming demand in your keyword category is static, but you know what they say about ass-you-me-ing anything.

When developing the initial outlines for keyword cost projections it is important to keep in mind user behavior or the user buying funnel. More generic search terms that often have multiple meanings will show lower click-through rates while brand or product specific keywords will show higher click activity. Use caution with generic, multiple meanings as future post-click performance may indicate that you don’t need them at all.

For example, an auto manufacturer can predict a 2 percent click through rate (CTR) for the phrase “SUV” while the brand specific “Hummer” will show a much higher CTR in the 8 to 12 percent range for that brand. The only remaining “X” factor is the stupid keyword set, but simply assigning a cost of 1 percent of brand term cost to this grouping to account for terms like “Humer” is adequate for budgeting.

Seasonality is a key factor to bear in mind when assessing traffic projections and keyword costs. Search providers and SEM firms will step up to help determine demand activity for keyword sets within your buying cycles so don’t ignore them. Simply viewing last month’s search activity for terms just won’t do it.

Historically painful

Does budgeting in this manner work? Sure it does, though there will always be a margin of error. The process of initial budgeting for keywords is designed simply to provide you with an overall outline for costs and this method is extremely effective for outlining expenditures for initial post-click activity performance testing.

Since performance will define final budget allocations and search providers vary in how listing representation is defined, don’t make the mistake of simply assigning a budget projection based on the search activity and click cost for a single position with one provider. For example, estimating a click cost of $.14 on Overture for a term searched 1,000 times a month and assigning keyword projections will show an incredibly high margin of error since this figure will not account for the Google AdWords user determining listing representation criteria.

In the examples below I have simplified seasonality into high and low activity months. Figures account for the combined estimates from several pay for placement search providers.




Of course you can have thousands of keywords and hundreds of keyword groupings. Smart search marketing firms (and sometimes providers) will help you determine anticipated click activity for budgeting systems and account positioning as well. A comprehensive format for budgeting will also include components for contextual placements and assignments for keyword expenditures to either cost of sales or marketing.

Slightly less than impossible

And so my friends, we move into the great unknown. The last “can’t” in pay for placement search is the totally unpredictable. Tales of search marketing lore abound relating to product launches sans search along with accounts of perpetual woe as public relations fiascoes occur with accidental breast viewing during family entertainment television shows or clients who get indicted for any one of a hundred celebrity induced crimes that might impact brand endorsements.

The commonly accepted practice for the disintermediation of news results with pay for placement search damage control content in the case of public relations fiascoes or, in the instance of product launches (when they are included at all) is simply assigning an emergency or reserve budget figure to search placements.

You can’t predict the unknown, right? Not without a Ouija board and shiny crystal ball, but it is possible to analyze past activity for similar, uh, problems and opportunities and make intelligent inferences as to possible costs.

Continuing on the SUV theme, this week the world of pick-em up truck and SUV owners on a quest to own the biggest and best truck to compensate for other shortcomings may have delighted in the news that still another completely unnecessary vehicle was being introduced.

For about twice the price of a Hummer H2 one can now purchase an all-wheel drive pick up twice the size of an H2 built on a tractor trailer frame. The big truck company International introduced its model CXT this week with a bang, but how could the company possibly launch a search initiative for this beast?

Past search spike activity for similar launches in this category will undoubtedly provide some intelligence on not only search activity but click costs as well. Actual search data on keywords for other products can show activity from product launch all the way to the comparative stabilization of search activity due to awareness maturity. This discipline of category profiling can be applied to almost any product launch or PR nightmare.

Historical research for past events -- be they fame induced or in the more common example of product launch -- can yield smart guesses, not exact figures. Instances of search unknown can effectively be placed into categories using past search activity data available from search providers just as you would for any other launch and, although it is not perfect, the information gathered is still better than the “duh, we think it cost this much,” answer most search marketers get when seeking budgeting information.

The little search engine budget that can

Pay for placement search budgeting will never be an exact science and there will always be unexplainable spikes in search activity. UFO sightings and scantily clad concert singers aside, applied science and intelligent research will provide a sound launching platform for search budgeting. “Can’t” simply shouldn’t be part of your search engine pay for placement search vocabulary and whining about search’s unpredictable nature just won’t solve the budgeting problem. In the immortal words of Marge Simpson, “Don’t be a crybaby, be a do-baby.”

iMedia columnist Kevin Ryan’s current and former client roster reads like a “who’s who” in big brands; Rolex Watch, USA, State Farm Insurance, Farmers Insurance, Minolta Corporation, Samsung Electronics America, Toyota Motor Sales, USA, Panasonic Services, and the Hilton Hotels brands, to name a few. Kevin believes in sound guidance, creative thought, accountable actions and collaborative execution as applied to search, or any form of marketing. His principled approach and staunch commitment to the industry have made him one of the most sought after personalities in online marketing. Kevin volunteers his time with the Interactive Advertising Bureau, Search Engine Marketing Professional Organization, and several regional non-profit organizations. Meet Ryan at the Kelsey conference and Ad:Tech, NY.