ellipsis flag icon-blogicon-check icon-comments icon-email icon-error icon-facebook icon-follow-comment icon-googleicon-hamburger icon-imedia-blog icon-imediaicon-instagramicon-left-arrow icon-linked-in icon-linked icon-linkedin icon-multi-page-view icon-person icon-print icon-right-arrow icon-save icon-searchicon-share-arrow icon-single-page-view icon-tag icon-twitter icon-unfollow icon-upload icon-valid icon-video-play icon-views icon-website icon-youtubelogo-imedia-white logo-imedia logo-mediaWhite review-star thumbs_down thumbs_up

The Chronology of Search


I envy the traditional media people. I really do. I sit in meetings and feel my eyes glaze over amid bantering about the latest reach and frequency data for publication X while waiting for my turn to speak. Of course the online portion is last and I think to myself, “Oh, how far we have come.” Bronx cheer.

I envy them because it seems nothing ever changes in the traditional media world; for all intents and purposes it is the same schpiel I was making exactly ten years ago when I left that world behind forever. Oh how simple my life would be and how much time I would have on my hands had I not followed the white rabbit and taken the blue pill. Alas, I dialed into The Matrix of search marketing and it is a daily trial to stay abreast of its evolution.

In search marketing, if you don’t modify your strategy weekly to account for changes in the industry, you may be doing yourself an injustice. However, maintaining both an awareness of the changes and the impact to advertisers is like trying to do a crossword puzzle while skydiving. Like any good online marketing journalist, I couldn’t have completed this puzzle without the help of the greatest editor in the business and the hard working people eMarketer standing by to open my parachute.

Here’s a breakdown of search events in the recent past, grouped into three categories: M&A activity, (self explanatory) Advertiser Impact, (those events which hit advertisers the hardest) Press Grabs (also, self defining and my favorite). As you might expect, a few things have happened since eMarketer’s report was published in July, so you’ll find the eMarketer data in blue and my assembled data in black.

Mergers and Acquisitions

The status quo for search marketing since paid search officially became the hottest thing since Starbucks has been consolidation in the industry. Using a Lion King Circle of Life analogy, I find the best way to describe the state of search marketing is this: The lions are eating up the gazelle, and the hyenas continue to step in and munch on the remains.

Topping the charts for smart moves in this arena are Yahoo!’s purchase of Overture, Google’s bid for Sprinks, and ValueClick’s acquisition of fledgling search provider Search123 along with affiliate powerhouse Commission Junction.

January 2003: Yahoo! plunked down $235 million to buy Inktomi—a search-technology provider and a Google competitor—to allow the portal to create and license its own search service. “Many analysts saw [the purchase] as a first step for Yahoo! to sever ties with Google in response to the competitive challenge it feels,” wrote Internet Advertising Report.

February 2003: Overture purchased AltaVista in a $140 million cash-and-stock transaction. By gaining AltaVista’s algorithmic search technology and paid-inclusion product, Overture can supplement its paid listings with a complete search package, and therefore better compete with Google.

April 2003: Overture completed its purchase of the Web search unit of Fast Search & Transfer (FAST)—a Norwegian developer of enterprise search and real-time filtering technologies—for $70 million in cash. The transaction also includes a performance-based cash incentive payment of up to an additional $30 million over three years.

April 2003: Google acquired Applied Semantics, a producer of software applications for semantic text processing and online advertising; this is expected to help Google strengthen its search and advertising programs, notably its fast-growing content-targeted advertising offering. In addition, the Internet Advertising Report writes, “The deal is significant considering Applied is a top 10 affiliate (based on revenue) and technology partner of Google’s paid search advertising competitor Overture.”

June 2003: ValueClick acquired Search123.com, a privately held pay-per-click search engine company, for an aggregate purchase price of approximately $5 million.

July 2003: Yahoo! announced purchase intent for Overture services setting off mass speculation relating to the future of Overture’s listing syndication relationship. The $1.6 billion deal led Yahoo’s ownership of the three major search properties Inktomi, Alta Vista and FAST.

September 2003: In a move to beef up ecommerce capability, search provider FindWhat.Com announced a plan to acquire Miva—provider of e-commerce solutions.

October 2003: Google announced a plan to acquire PRIMEDIA’s Sprinks unit and contacted investment banks about an initial public offering (IPO). The (IPO) could value the company in the range of $15 billion and $25 billion, according to separate reports in the Financial Times and Wall Street Journal. Google’s deal with PRIMEDIA alone not only gave it the Sprinks pay-per-click advertising network, it also included a four-year distribution- and revenue-sharing agreement for the targeted media company’s About.com.

October 2003: ValueClick, Inc. signed a definitive agreement to acquire Commission Junction, a privately held affiliate marketing technology and services company, a move that will double the size of its affiliate marketing division.

October 2003: In an interesting move to make the Stanford University science labs the complete search engine gene pool, Google acquired Kaltix, a ninety-day-old user-criteria based search engine. Evidently, Google wants to help make search a little more personalized.

Advertiser Impact

The hard-hitting brutal truth of much of the search-marketing world is that smart marketers need to make immediate changes in search initiatives. In the early part of 2002 and into 2003, Google established itself as a dominant force picking up partnerships from competing search listing providers. Search marketing graduated from its infancy with major measurement services introducing offerings to help quantify the madness, and key search providers lost partnerships -- once again making life for advertisers a state of daily confusion.

April 2002: Google used its paid-listings program, AdWords, to woo away former Overture partners EarthLink and Ask Jeeves.

May 2002: Google’s AdWords again swiped business from Overture -- this time a very big fish: AOL.

March 2003: In the best idea since Tony Montana bulletproofing his Porsche, search-marketing pundit Kevin Ryan published his first search-marketing column in the iMedia Newsletter. The industry reacts with cautious optimism. Search providers officially elevate to “orange” alert status.

April 2003: Google added Amazon.com to its stable, signing an agreement to provide paid links and Web search on the immense ecommerce site.

April 2003: comScore Networks launched qSearch, a system that tracks and reports consumers' distinct search queries across 25 major search engines and portals.

April 2003: Yahoo! released a more search friendly site design marking its rebirth as a portal. In this new interface, search boxes follow users throughout their search experience.

May 2003: Jupiter Research announced a tool to help advertisers determine how search can fit into marketing budgets.

June 2003: Google announced its latest drive toward making contextual search a four letter word with the publisher-driven AdSense program. Big pubs Switchboard.Com and Weather soon jump into the program.

June 2003: Nielsen//NetRatings launched AdRelevance 4.0, which includes among other things Nielson’s entrance into the search measurement frappe.

June 2003: Ebay announced a plan to offer keyword-based bidding for advertisers. Banner ads will be initialized in search results and bidding starts at ten cents.

July 2003: Google and Weather.com, Website of The Weather Channel®, announced a multi-year agreement that will make Google's search technology, and targeted content and search advertisements available on weather.com.

August 2003: The last bastion of under monetized search space was discovered by Overture services as their local search product is released for testing on Overture owned Alta Vista.

August 2003: Doubleclick jumped into the sack with iProspect and TrafficLeader for algorithmic search services and Paid Inclusion respectively. This move undoubtedly alienates every other search-marketing firm on the planet.

August 2003: Overture owned Keylime Software announced the release of LimeCommerce, a search marketing analytics app, proving you can own the traffic and tout its value at the same time.

September 2003: Google began testing its local search product as a Google labs experiment. At the same time, rumors abounded about Google being in talks with CitySearch for a local content partnership.

October 2003: Terra Lycos and Google Inc., have signed a multi-year agreement making contextually targeted advertisements through the Google AdSenseTM program available on selected sites throughout the Terra Lycos Network. Google will provide relevant contextually targeted ads to pages of Terra Lycos' U.S. properties including Angelfire, HotWired, Lycos.com, Matchmaker, Quote, Raging Bull, Terra.com, Tripod, Webmonkey, WhoWhere and Wired. Through AdSenseTM, Lycos users will see advertisements targeted to the unique content of pages throughout the Terra Lycos Network.

October 2003: Overture Services will provide paid search listing to Microsoft's MSN network in the United States and United Kingdom through June 2005.

October 2003: AOL expanded its relationship with Google in both editorial and paid search. This move is made to match search capabilities with Yahoo! and MSN. The deal is said to be a multi-year agreement and no expiration date was provided.

October 2003: MSN expelled Looksmart’s paid inclusion listings from search results. In the same time frame Looksmart introduced a “sponsored listing product.

Press Grabs

At least one immutable truth exists in search: Advertisers are bombarded with press releases, each hoping to ignite the universe of online marketing. In order to bring you this information in an easy to digest package, I have perused every press release since early 2002. The information contained in many of those press releases falls into the category of nice to know, but not earth shattering.

Incidentally, this activity most often sends me into a daydream where the Yellow Rose and I leave big corporate forever. We would start our own firm; I would give myself a cool title like “Chief Strategic Innovator” and start writing some really hot press releases.

April 2002: LookSmart purchased the WiseNut search engine for $9.25 million in stock.

January 2003: As of the start of the year, Overture depends on its partnerships with MSN and Yahoo! for two-thirds of its revenue.

February 2003: Overture reported net income of $9.5 million for Q4 2002, down from $20.8 million for the same quarter in 2001. However, for the entire year of 2002, Overture reported net income of $78.4 million, up from 2001’s net income of $20.2 million. (The company achieved its first quarter of net income profitability in Q3 2001.)

April 2003: Yahoo! reported Q1 2003 revenues of $282.9 million, of which $190 million came from marketing services, including paid search. That was up 38% from the same period in 2002.

April 2003: Overture announced that its 2003 earnings would be much lower than expected, with net income for the year projected at $22 million to $26 million, or between 35 and 42 cents per share, less than the 63 cents per share earnings forecast by Wall Street analysts. Overture blamed a variety of factors, including increased spending from its recent purchases of AltaVista and FAST, lower revenue from its U.S. paid listings, and higher-than-anticipated traffic-acquisition costs (the amount of money Overture gives to partners such as Yahoo! and MSN who carry its paid listings and make up two thirds of its revenue).

April 2003: Ask Jeeves reported revenue of $25.2 million for Q1 2003, a 57% gain from a year earlier. And net income mushroomed to $7.7 million, compared to a loss of $10.4 million in Q1 2002.

May 2003: LookSmart announced that its earnings for the year—before interest, taxes, depreciation and amortization—would hit only $13 million, down considerably from the $22 million to $25 million it had forecast. The market leader for paid inclusion said it still expects revenue in the $140 million to $150 million range. Even as it announced reduced revenues, LookSmart also said it earned a $1.1 million profit in Q1 2003, or 1 cent per share (lower, though, than the 4 cents per share market analysts expected).

May 2003: Ask Jeeves—operator of the Ask.com and Teoma search sites (the algorithmic search engine it bought in September 2001)—announced it would sell its enterprise search division and use proceeds from an upcoming $100 million debt offering for “general corporate purposes which might include potential acquisitions and investments.” The enterprise unit saw revenues decline in Q1 2003 to $3.6 million from $5.3 million in the corresponding 2002 quarter.

June 2003: AOL’s MapQuest picked up search listings for Google’s AdWordsTM program in another move to test the local search model.

July 2003: MSN and eBay partnered up for search results for some search results from MSN's homepage, Internet Explorer search, and shopping and auctions searches.

August 2003: Ask Jeeves dropped the butler and announced an effort to make search a bit more human with “Smart Answers" tools. People can now use everyday language to search. (What language were they using before?)

August 2003: Yahoo continues its move to be reborn as search utility, making search tools visible to searchers throughout the site. Yahoo is quoted as saying they want searchers too stick with them from inspiration through purchase.

August 2003: The first all-search-all-the-time industry association was formed. SEMPO, or Search Engine Marketing Professional Organization, is said to be working on standards for all forms of search marketing.

October 2003: Ecommerce superstore Amazon.com raised the curtains on a new "Search Inside the Book" feature that lets shoppers preview full text of titles. The feature, which has the approval of book publishers, puts some 33 million pages of searchable text at the disposal of Amazon.com shoppers.

Hypothesis Prognostics

Of course, I wanted to bring you the world to date with the possible exception of late-breaking changes, which cancel out earlier changes that happened only last week (gasping for air) so I couldn’t get them to my editor in time. Still the most frequent question I get from advertisers is how much more consolidation can the industry handle? Sure, there are still a few holdout candidates out there, (like FindWhat) who might be consumed by bigger providers. But, with Google about to go public, it is anybody’s game.

Ad:Tech is kicking off in New York this week and the event has always been a favorite for big announcements, so stay to tuned to iMedia as the search marketing world continues to take shape. Yours truly will be there and you can catch me on a search-marketing panel Wednesday morning.

About the author: iMedia search columnist Kevin Ryan’s current and former client roster reads like a “who’s who” in big brands; Rolex Watch, USA, State Farm Insurance, Farmers Insurance, Minolta Corporation, Samsung Electronics America, Toyota Motor Sales, USA, Panasonic Services, and the Hilton Hotels brands, to name a few. He is currently Director Market Development of IPG’s Wahlstrom Interactive where he provides guidance in directional online marketing to Wahlstrom’s prestigious list of clients and sister agency brands. Also, Kevin really, really likes his job at Wahlstrom and was only kidding about that “leave big corporate” and “Chief Strategic Innovator” title comment.

These definitions should be provided in a clear and unambiguous way that leaves absolutely no room for interpretation. Most internet standards come from the IETF or from W3C. In broad terms, IETF handles the hardware-related stuff, such as TCP/IP and HTTP, while W3C handles the "soft" stuff such as HTML and XML. Where possible, both provide their definitions in a special format called Extended Backus-Naur Format, or EBNF. EBNF is also used to define the syntax and operation of programming languages. EBNF is so precise you can create EBNF processing software that is able to run any programming language for which it has the EBNF definition.

EBNF achieves this precision by avoiding words and using symbolic notation instead. For example, the definition for a vowel would look like this in EBNF:

Vowel := A | a | E | e | I | i | O | o | U | u

As you may have guessed, "|" is used as "or." Notice the precision. There is no possibility of ambiguity or misunderstanding -- a computer could understand this. This is the level of precision you need in order to build software that can implement your rules.

By contrast, the WAA's definition of a page is "an analyst definable unit of content." In other words, a page is whatever you say it is. That's not just a vague definition, it literally is no definition at all. What this says is "whatever you want." I cannot implement that in web analytics software; I cannot check to see if someone is compliant with the standard because (by definition) everyone is; I can't even compare my data with someone else's because I don't know if they are measuring the same thing as me.

EBNF is not an academic ideal. HTTP, HTML, XML and many other standards are written in EBNF. In other words, the web runs on EBNF. For example, here's the EBNF definition for specifying the version of HTTP in a message:

HTTP-Version := "HTTP" "/" 1*DIGIT "." 1*DIGIT

Some people have argued it is not possible to achieve this level of precision in web analytics. That is, of course, rubbish -- it has already been done, and done many times.

Every piece of web analytics software operates to the level of precision offered by EBNF -- it has to. Web Trends, Omniture and Google Analytics can't decide on the fly what constitutes a page. These are just programs. They work mathematically, with mathematical precision. The people who wrote them had to put 100 percent precise definitions into their system. It's simply not possible to write vague computer code. Computers can't decide for themselves. Even when they appear to, all they can do is select from a set of pre-determined, precisely-defined alternatives. 

Not only have programmers created precise web analytics definitions, other standards bodies have as well. The first web analytics standards were created in the 1990s by the Joint Industry Committee for Web Standards (JICWEBS). For example, WAA says a unique visitor is "an inferred person." By contrast, JICWEBS defines a unique visitor as "a unique and valid identifier. Sites may use IP+User-Agent and/or Cookie." This is how it is done in practice by software. For example, Google says "a visitor is defined using a unique numeric identifier stored in the Google Analytics tracking cookies." Google couldn't implement the WAA definition if it wanted to because the concept of an "inferred person" is meaningless in computing terms.

There was a push by some people to have the WAA standards committee work to this level of precision in the early days. However, the committee decided that it didn't want anything in the standard that would be "too complicated." In other words, it was more concerned with being popular than doing the job it was appointed for.

I think the WAA underestimated our intelligence.

The standard doesn't even meet the requirements the WAA standards committee set. When the committee started work on the standard, it decided to accept the existing standards from bodies like JICWEBS. It recognized, or said it did, that having competing standards would move the industry backwards, not forwards. I guess you could have a debate about whether the WAA followed through on this or not.  While the WAA's definitions are certainly at a variance from the JICWEBS standards, you could argue that they don't compete because they aren't clear enough to be an alternative -- vague waffling can't be seen as an alternative to a precise definition.

However, it is not necessary to be this precise in order to be a standard. A standard can become a standard if people actually use it as such. For example, the IETF states that something is a standard if it "has multiple, independent, and interoperable implementations." In other words, if there are multiple systems deployed that use a definition in the same way.

This is certainly the case for the JICWEBS standards -- they are used universally for online auditing. Furthermore, every online audit is "interoperable." This means the readership numbers you get from one online publication have been calculated in exactly the same way as the readership numbers from a competing publication. When you compare them, you are comparing like with like.

The problem for the WAA is that absolutely nobody is using its standard, and nobody can -- it's too vague. Companies like Google and Indextools have publicly stated where they comply with the WAA standards, but none are 100 percent compliant. Even if they were, it wouldn't mean much. For example, if a page view can be whatever you like, to be compliant all you need to do is report page views, no matter how you calculated them. This would not make WAA-compliant systems interoperable. They could be calculating totally different things under the same name.

We could do better. WAA standards are not vague because it's impossible to define things, or measure them, but because no one can be bothered to raise an intellectual sweat and try. Creating precise, robust, useable standards has been done for much more complicated things than visitor behavior on websites. My guess is that some definitions would take things to new levels such that no software was 100 percent compliant. This has been the problem with the JICWEBS standards. The groups that define their standards are dominated by the web analytics software manufacturers, and they have said they won't create a standard that isn't already implemented in their systems.

Both JICWEBS and WAA are wrong in this regard. You don't look at the state of things, write it down and call it a standard. Can you imagine if we created laws like that? "Well, people get angry and kill each other, so we'll make that legal."

The WAA should be setting the agenda, not following the crowd. The task of the WAA standards committee should be to determine how web analytics metrics should be calculated in order to achieve the highest degree of precision possible. The WAA should be laying out the roadmap for the way things should be. It then falls to the vendors to bring their software into line. It may be that some definitions that came out of such a process would be impossible to implement with today's technology. So what? That's what R&D is all about. If we only aimed for what we already knew, we'd still be sitting in caves picking fleas off each other. The entire web is the result of people creating standards first that were later picked up by vendors.

Tim Berners-Lee didn't invent HTML by copying what IBM and Microsoft were doing. He designed the system first and then found people to encode it in software. Google didn't copy what Yahoo was doing -- first it invented the algorithms for ranking on paper, then it worked out how to implement them in software. I'll bet Google had no idea how it was going to implement some of its algorithms when it wrote them. The company still has things it wants to do but no idea how to. That's what keeps Google moving forward.

What the WAA has done is a retrograde step -- the WAA standard has less precision and utility than the JICWEBS standards, so it moves us backward not forward. However, WAA is a major force in the world of web analytics and online marketing. What it says matters. In this light, the work of the WAA standards committee is a disaster for the web analytics community. It will take years to undo the damage and create proper precise standards that can be implemented in software. The WAA "standard" is not a standard, it's just second-rate muttering.

Brandt Dainow is an independent web analytics and marketing consultant working in the U.K. and Ireland.

iMedia: You've taken a contrarian and very pessimistic approach to next year's online ad picture.

Denton: I've been a skeptic for a good long while -- it's not like this is a sudden discovery.

iMedia: You recently pulled the plug on some sites and before that, you eliminated writer bonuses for page views. Are these the kind of hard-nosed decisions more publishers should be making?

Denton: We've been culling sites for nearly three years now; we've (thankfully) been conservative for a long while. It's rather late for publishers to be rationalizing now; they should have been evaluating their properties even during the good times.

iMedia: Many blog sites seem to be expanding, with the Huffington Post pulling in a new round of financing. Any publishers you think do a good job of managing properties?

Denton: AOL, actually, has focused most of its attention on Engadget and a few other properties -- and abandoned the no-hope titles it bought from Jason Calacanis. As a reader, I'm glad that sites such as the Huffington Post and Politico are taking up the cause of political reporting -- but we pulled out of that category because it's toxic to advertisers. What's good? Our video game site, Kotaku, seems to be doing particularly well with both audience and advertisers right now.

iMedia: Games seem like the perfect topic: need for reviews, need for tips and locus for advertisers, right? Plus, there are lots and lots of game and hardware companies.

Denton: Yes, audience and advertiser interests overlap relatively rarely. Video games, entertainment and consumer electronics are among the doubly blessed. And that's why we put most of our energy against those categories.

iMedia: When Google is asked how it will monetize new services, it always says, "We'll figure that out later." And hundreds of other internet companies have followed suit. Do you think we'll see a shift away from the internet wisdom of "We'll monetize it somehow?"

Denton: The only outfit I know that still takes that attitude is Barry Diller's IAC, which is pouring an extraordinary amount of money into Tina Brown's Daily Beast. Lucky her!

iMedia: Well, when you're a diva ... While the online ad industry is patting itself on the back, saying they'll get more ad dollars from traditional media next year, you're predicting a 30 to 40 percent spending decrease in all media. You based that on Mary Meeker's regression analysis of ad spending vs. GDP and then plugged in what happened in Japan, Sweden and Indonesia in the 1990s. Can you also give us the non-math-whiz version?

Denton: Everybody accepts that advertising is highly sensitive to changes in overall consumer spending and economic output. But the advertising analysts haven't caught up with the economists, and the economists haven't anticipated next year's plunge. The economy will perform much worse than people think and, because advertising is so volatile, it's reasonable to assume it will perform much, much worse. But you should know that I'm a congenital pessimist. I was predicting doom three years ago. Someday I might be right.


iMedia: You've complained that publishers have forced marketers into a straightjacket of standard ad units too small for brands to breathe. And you introduced two non-standard units on your sites: the marquee, which is 1,000x250 pixels at the top of each page, and the panorama, 800x250 within posts. It sounds like you're saying the bigger, the better.

Denton: We aspire to the production values that Conde Nast is known for in the magazine world. Why shouldn't online ads be as gloriously glossy as a spread in Vanity Fair? (Oh, and the bigger, better ads provide enough space for video, too.)

iMedia: But ad units were standardized to make it easier for creative agencies to produce them, media agencies to place them and publishers' reps to see them. Do you think those concerns were invalid? Or are they just a bunch of whiners?

Denton: All I know is that clients are all over our custom offerings, and I haven't seen those clickthrough rates for the last 10 years.

iMedia: Do you think we'll see any big shifts or changes in the online marketing industry in the coming year, revenue aside?

Denton: The best thing that's happening is that all the distractions -- the moronic widget makers and other fads, the me-too knock-off sites, the bogus viral programs -- are being eliminated. Media buyers have time again, and they seem to be devoting it to the properties that will still be around at the end of next year.

iMedia: What's your advice to online advertisers who want to make the most of opportunities with their surviving properties?

Denton: I have just one bit of advice. After looking at the Nielsen numbers for a network, do a simple test: Have you or your friends actually heard of any of these properties? Way too much money is still going to networks such as Glam, which amalgamates random blogs and disengages passthrough visitors. It's embarrassing that the advertising industry hasn't yet figured that out.

iMedia: Now, give us some hope. Any bright spots for next year?

Denton: This economic downturn is an extinction-level event. It will be bad for the dinosaurs -- but great for any mammals that survive.

Susan Kuchinskas is a freelance writer who has written for Adweek, Business 2.0, M-Business and internetnews.com.

Kevin Ryan founded the strategic consulting firm Motivity Marketing in April 2007. Ryan is known throughout the world as an interactive marketing thought leader, particularly in the search marketing arena. Today's Motivity is a group of...

View full biography


to leave comments.