How to know if your DSP is really working

  • Previous
  • 1 of 2
  • View as single page

Depending on who you ask, there are anywhere from 10-20 billion display impressions available globally each day for demand-side platforms (DSPs) and their clients to purchase on real-time bidding (RTB) sources like ad exchanges and supply side platforms (SSPs). On top of that, some DSPs have now enabled buying of other types of media, including premium display, social, video, and mobile, bringing the total addressable ad inventory closer to something like 30 billion daily impressions that can be serviced through the execution and decisioning layer of a DSP.

Let's put that in the context of a single advertiser. For easy math, let's say a display campaign spends $100,000 per month, or a little over $3,000 per day. At an eCPM of say $2.00, that's a little over 1.5 million impressions per day, or roughly 0.01 percent of the total available supply. Even if you tweak the assumptions above to be 10 times higher or lower, the bottom line is that a typical campaign is buying an incredibly small fraction of what's out there. But with such a small fraction to play with, how do you ensure that you're getting only the very best-performing impressions and audiences for your budget, and not simply an average cross-section of the total, or worse, a below-average one?

You can start with retargeting, which we all know works, but has limited scale. But what happens after you've remarketed the consumers who are already engaged with your brand and products? How do you bring new customers into the conversation? How do you get new prospects to convert? In short, how do you decide which 0.01 percent of the unwashed mass of impressions is the right 0.01 percent for your campaign?

This article introduces four important concepts and folds them together into a framework designed to pinpoint that 0.01 percent. As any data modeler knows, being able to single out the top decile, or in this case the top hundredth of a percentile, generates massively disproportionate returns. It's no different here.

Impression Quality is a measure of how effectively an impression delivers the action desired by a particular buyer, be it a positive brand response, a click, a purchase, or even ROI. When I say "impression" I don't just mean the blank space on a website waiting for an ad to show up. I mean that blank space, combined with the creative that is served into it, combined with the media in which it sits, combined with the user who is going to see it. The sum of all those things -- media + creative + user -- is an impression. Mathematically, the impression quality is an algorithmic prediction of how likely the user is to take the action desired by that buyer on that impression, relative to all other impressions.

For example, if a buyer's goal is to drive purchases, and a particular impression is predicted to have a response rate of 10 percent per 1,000 impressions, versus a campaign average of 2 percent, that impression has an impression quality of 5.0. The impression quality prediction is based on dozens if not hundreds of variables describing the creative (offer, message, image, etc.), the media (site, content category, ad size, ad placement, time of day, day of week, etc.), and the user (via any number of cookie-based and non-cookie-based user variables).

Impression quality is incredibly important because it separates the wheat from the chaff -- the quality impressions that drive high user action rates, from the low-quality impressions that don't. And since it depends not just on the media, but on the user who is seeing the ad and on the creative served, the very same impression could have high quality to buyer "A" (because that user is buyer A's ideal target consumer and that publisher has relevant content to buyer A) and low quality to buyer "B" (because that user is not buyer B's target, nor is that publisher relevant to buyer B).

The same impression could even have different quality to a single buyer, depending on which creative the buyer chooses to serve (e.g., because the user is more likely to take action on creative 1 versus creative 2).

High impression quality often comes at a high price, but not always. Imagine you place a certain ad on the homepage of a certain website and show it to a certain user at a certain time. That ad will have a certain click-through rate or response rate, or whatever "action rate" the buyer cares about, regardless of what the publisher happened to charge for it. Whether it was a $20 CPM, $2 CPM, or a free impression that was part of a "make good," the intrinsic quality and performance of that impression is actually the same. Price doesn't cause impression quality, but it does correlate because some of the media characteristics that drive high impression quality happen to be the same ones that publishers charge higher prices for. That said, much if not most of the information that goes into calculating impression quality is only known by the buyer. Just as beauty is in the eye of the beholder, impression quality is specific to the buyer, which is why it's critical for buyers to have the algorithmic capability to calculate it, for billions of impressions a day, in real-time.

Buyer Value is the dollar value that a specific impression is worth to a specific buyer, and it's a function of both the impression quality and the buyer's goals. In essence, it translates the impression quality into a CPM. Mathematically, buyer value is simply equal to the predicted user action rate multiplied by the buyer's goal value.

For example, if an impression has a predicted 2 percent response rate per 1,000 impressions, and the buyer's goal is a $75 CPA, that impression has a buyer value of $1.50 CPM. A different buyer, running a different creative, will have a different predicted action rate and a different goal value, resulting in perhaps a $0.10 CPM buyer value, while a third buyer with their creative, predicted action rate, and goal value may have a $7.75 CPM buyer value -- all for the same impression. That's why publishers actually have very little insight into buyer value -- like impression quality, it's primarily a function of factors that are specific to each advertiser.  And since it's related to impression quality, a sophisticated algorithm is required to accurately determine buyer value.

Market Value is the predicted price at which an impression is likely to clear in an RTB auction. Unlike Buyer Value, it isn't dependent on any single buyer, but rather on the interaction of all participants in the RTB marketplace. Mathematically, it's a prediction of the price at which an impression will clear, with a certain probability. For example, for a certain impression, on a certain site, a $5.00 bid will win 94 percent of the time, a $3.00 bid will win 68 percent of the time, a $1.00 bid will win 40 percent of the time, etc. Of course, it's not just a function of the site, but of many other variables like ad size, position on the page, time of day, and dozens of other variables. In contrast to Buyer Value, publishers do know the Market Value, because it quantifies the price at which their impressions clear, and at which they get paid. Buyers can easily know it too, down to the impression level, if they leverage an algorithm that can accurately predict it based on the myriad variables that characterize every impression.

And finally, Relative Value quantifies the gap between what a single buyer thinks an impression is worth (which governs the bid price), and what the rest of the market thinks it's worth (which governs the clearing price). Mathematically, it's simply a ratio of the buyer value to the market value. In other words, the larger the relative value, the more that particular buyer values the impression over the market, and hence the greater the value to the buyer. It's important to note that the relative value of an impression is not necessarily a reflection of price; it's the relative comparison of the buyer value to the market value that matters. A relative value of 3.0 could mean a buyer value of a $0.30 CPM versus a market value of a $0.10 CPM, or a buyer value of a $12 CPM versus a market value of a $4 CPM. As we'll see in a moment, the latter is much more interesting.

So what does this all mean? Let's take a look at the graph. It's a simplified 2x2 representation of impression quality versus relative value. Think of it as effectiveness (impression quality) versus efficiency (relative value).

The shading indicates how the universe of available RTB impressions populates the graph. Darker areas contain more available impressions in the RTB landscape, while lighter areas contain fewer impressions. Let's take a look at each quadrant of the graph:



Troy Lerner
Troy Lerner October 18, 2011 at 7:36 PM

Thick read, but quite insightful! to investigate the log files now!