How to know if your DSP is really working

  • 2 of 2
  • next
  • Sign in for single page

Quadrant 1: Non-performance
The impressions here are of low quality, meaning they aren't very good at generating the actions the buyer cares about. Moreover, these impressions have low relative value, meaning they are "expensive" (i.e., the market value is high) relative to what they are worth to the buyer. They may only cost $0.50 (not expensive in an absolute sense), but they are too expensive relative to the awful performance they generate. Unfortunately, this is where the vast majority of RTB impressions are found -- roughly 50-70 percent. Nearly all of these impressions won't be effective for the buyer's campaign, and the cost of winning them exceeds what they are worth to the buyer.

But perhaps the worst news is that this is also where most DSPs spend their clients' dollars. DSPs that don't have the algorithmic capability to find the high-quality and high-value impressions typically end up guessing at who their clients' audiences should be with pre-defined "audience buys," often incurring incremental data costs to do so. While a segment here or there may pop, the result on average is usually poor. In the world of SEM, search bidders tend to know in advance what keywords work, how well, and also what to bid. But in display, it's usually not obvious what impressions and user characteristics make a campaign work. You need an algorithm to determine which impressions and users to buy and at what price. You simply can't "audience guess" your way through 15 billion daily impressions, especially not when you have large budgets to spend. If you do, you'll wallow in this pit of non-performance.

Quadrant 2: Cost-driven performance
As with non-performance, the impression quality here is low. But the relative value is actually high, meaning that even though these impressions don't perform very well, they are dirt cheap enough to squeak by and meet the buyer's goals. It's a bit like fast food -- you're getting a pretty lousy burger, but it only costs $1, so it works for you. But just as that isn't a great long-term strategy for your health, it's also not a great long-term strategy for you campaign's health, or for the ecosystem at large, for that matter.

From the advertiser's standpoint, you may be technically hitting your goal, but you're doing it in a cheap, low-quality environment. These impressions are most likely on long-tail sites, below the fold, deep in the frequency curve, or all of the above. Not the best use of the buyer's dollar. From the publisher's standpoint, the eCPMs are low, and while that may be OK for some, it's not going to encourage the quality publishers to show up in force.

Unfortunately, this is also where a lot of impressions live in the RTB world -- about 20-50 percent. And sadly, as the ad exchanges and SSPs know, a lot of DSPs service their clients by simply "spraying and praying" low bids. They are bidding low and winning low-quality impressions that are cheap enough to technically meet the buyer's goal, but actually failing to maximize total value, which is what brings us to the next quadrants.

Quadrant 3: Quality-driven performance
Now we're dealing with high-quality impressions that deliver high-action rates as defined by the buyer: strong brand lift, strong engagement and click-through, high response and conversion, or high ROI. However, the relative value here is low, meaning you are going to pay full price for that quality. And that's fine. A high-quality impression worth $6.50 to the buyer, who wins it for $6, likely results in a happy buyer and a happy seller.

For a given campaign, less than 10 percent of RTB impressions are typically in this quadrant. The good news is that's billions of daily impressions. The bad news is that if your DSP isn't capable of modeling these impressions out and quantitatively defining their characteristics, you'll only buy them about 10 percent of the time (or less actually, if you're just spraying low bids that have a lower chance of winning these quality impressions), and that means you'll also perform 10 times worse.

Quadrant 4: Value-driven performance
This is where RTB realizes its full potential. Here, buyers enjoy the best of both worlds: not only is the impression quality is high, resulting in high effectiveness, but so is the relative value, resulting in high efficiency as well. These impressions are gold for the buyer, but like gold, they are hard to come by. For any given campaign, only a small percent of impressions will be in this quadrant.

What makes them even more interesting is that they are also gold for the publisher. Just because their relative value is high doesn't necessarily mean they are cheap. In fact, cheap impressions often don't make it into this quadrant because many cheap impressions happen to not have very high impression quality (some do, but most do not). It simply means they cost less than what they are worth to the buyer. A typical scenario here would be where the impression quality is so high that it has a buyer value of $12, and a Market Value of $4. So the buyer is getting high performance at a great relative price, and the seller is realizing high absolute eCPMs. Great, right?

But imagine if the buyer and seller could then take this to next level. Imagine the buyer saying, "Hey, your inventory is working great for me in RTB -- I'd be willing to pay even more than a $4 CPM to get more of it," while the seller says, "I've only been putting a trickle of that inventory into RTB because I was worried about channel conflict, price erosion, and advertiser quality. But for an $8 CPM, you can have all of it." Deal.

So, having explored each of the four quadrants, how should a buyer spend their monthly display budget to find the best 0.01 percent of the RTB universe? By picking off the impressions and users tucked into very top-right corner of the graph (labeled "maximum performance"), representing the highest-quality and highest-value impressions for that campaign. As budgets increase and more impressions are needed, larger scale buyers should expand out from that corner, carving out larger portions of that upper-right section of the graph to meet their budget needs (as the concentric circles in the graph suggest).

Ironically, most DSPs actually spend their clients' budgets in the lower left portion of the graph (non-performance) because (a) that's where most of the impressions are, and (b) they lack the ability to quantify and model things like impression quality, buyer value, or market value. Instead, they are making manual, time-consuming, but inaccurate guesses at what to buy, or simply just bidding low and praying. That underscores the fact that none of this smart buying is possible without the underlying algorithms to determine these quantities, and to do so fast, accurately, and often in a data-rich and highly dynamic environment. That's the area where most DSPs fall down.

How do you know if your DSP is doing this? A good way to tell is to ask for log-level data on the bid price for each impression they deliver for your campaign. If all their bids are simply falling within some limited range (e.g., $1.25 plus or minus $0.25), then you know you've got an algorithm-less DSP. Alternatively, one can ask: "If I were to give you an impression and all its associated characteristics, could you tell me the predicted user action rate (and hence the impression quality), as well as the correct buyer value, market value, and relative value?" (Your DSP may not use these terms, but the underlying principles should be the same).  If they say yes, then ask for this data on all the impressions for your campaign.

If they can't, then you have to wonder how they are spending the budget. Is it on the right 0.01 percent? If they are just guessing at audiences or bidding low, the odds of that are about the same as finding a four-leaf clover on the first try. Good luck!

Ari Buchalter oversees MediaMath's product team, designing and delivering best-in-class solutions to the market via the TerminalOne platform, as well as MediaMath's business operations group focused on delivering exceptional results to clients.

On Twitter? Follow iMedia Connection at @iMediaTweet.

 

Comments

Troy Lerner
Troy Lerner October 18, 2011 at 7:36 PM

Thick read, but quite insightful! ...off to investigate the log files now!