Ad effectiveness studies have been used for more than a decade to measure the branding impact of online campaigns. Results from these studies, as well as aggregate industry knowledge, have helped advertisers design better media plans for future campaigns. Enter real-time data, which allows us to learn what's working in a specific campaign, and make changes while it's still in market. These changes help advertisers save money by reducing wasted, ineffective impressions and ensuring that the right message is delivered to the right consumers. But, there are rules that should govern the correct way of optimizing online branding campaigns and at the end of the day; optimization is part data and part instinct. The numbers alone shouldn't be the exclusive guide to your decisions. Common sense and experience are often just as important and should guide you through the following considerations for campaign optimization:
Define campaign success criteria and key performance indicators
How will you know that your campaign is effective or not effective? What should you be looking for? Is any incremental increase in the metrics going to be sufficient? These are all important questions to answer prior to making any changes to your campaign mix. When assessing performance of sites or creative formats, benchmarks are inherently built into the campaign because you can compare each one against each other. However, shifting impressions around within a failing campaign is a waste of time. Campaign-level normative benchmarks are useful to assess whether your campaign, as a whole, is among the top or bottom performing campaigns in the brand's industry. Normative databases can be used for benchmarking as well as planning what magnitude of change to expect. It's important to use not only normative data, but also your own experience with the brand, to create hypotheses of what results you expect from the campaign. This will give you something specific to test against. Often, testing a more quantitative goal, such as "increase purchase intent by X points," results in more insightful findings than just "increase purchase intent."
Assess what you have the power to change
Sites' insertion orders usually include an impression level commitment, so a change as drastic as pulling a site from the campaign usually isn't possible. However, agencies can usually move impressions across sections or placements within a site. Agencies with longer-term site contracts can also sometimes shift impressions from one campaign to a future campaign if the site appears to be a poor fit for the campaign in question.
If there are multiple creative executions in the campaign, agencies can easily shift impressions between iterations, especially within the same size unit. If a campaign is really performing poorly, though, often a change to the creative design is needed. Ideally, creatives should be pre-tested to ensure that the best executions go into field to begin with, minimizing subsequent changes. There is a lot of emphasis in our industry on mid-campaign adjustments because online creative can be redesigned inexpensively and relatively quickly. The reality, however, is that this seldom happens and results in a lot of money being wasted on ineffective media. The best practice is to put pre-tested creatives into field, and include the creative agency in your optimization plan so that they can be "on call" to make tweaks to the design if the results suggest it's needed.
When setting up an ad effectiveness study for your campaign, remember to communicate to your research partner what types of changes you're considering so that they can design an appropriate sampling plan, survey, and results dashboard against the objective.
The ability to optimize should never replace thoughtful media planning prior to the campaign. Throwing everything in and waiting to see what works will result in a lot of wasted media spend. The best campaigns will usually be those with the best initial media plan, and optimization will involve simply fine-tuning the mix. Complete overhauls should be limited to rare disasters, and most campaigns should require limited changes.
Don't make hasty decisions
It's tempting to start making changes just a few days into a campaign in order to benefit early from optimization and reduce waste. But, branding effects don't set in overnight. Frequency is one of the most important factors in campaign performance. A comparison of message association impact on CPG campaigns shows incremental increases in impact from the first exposure to more than 10 exposures.
Source: Dynamic Logic MarketNorms®, data from last 3 years through Q3/2011, N=765 CPG campaigns
Lauren Hadley, associate director of integrated insights at Starcom says, "Let the campaign build how it was planned to build. Don't optimize prematurely." Optimal frequency levels vary by industry, brand tenure, campaign objective, and even creative format. For example, the results in the chart above suggest that changes shouldn't be made before at least four exposures have been delivered to a majority of a CPG campaign's audience. Analyze historical normative data to home in on the optimal frequency level to expect for your brand's campaign.
Know when to say "when"
Don't over optimize. Observe, change, observe. Stop. Observe, change, observe. Stop. We recommend two sets of optimization periods per three-month campaign.
Use trusted data
Be cautious about the quality of the data that you're using. Don't be tempted by cheap studies that sacrifice the quality of the data that's delivered. Shorter surveys are best for optimization because they garner higher response rates and, therefore, earlier results. But a survey that's too short and only asks one or two questions can leave you in the dark about who the audience is, and whether the results are reliable. In ad effectiveness studies, it's critical that the control and exposed groups are recruited from the same sites and have similar audience profiles. Demographic and category usage questions, and some amount of weighting, are usually needed to further match the two groups. Look for a research vendor that weights the data in real-time so that you can view valid data at any point during the study.
Train media planners on how to use survey data
Unlike clickthrough data and other sources that are collected at the impression or user level, interpreting survey-based data requires expertise. Individual data points might represent mere anomalies, so conclusions should be drawn based on larger samples and repeated trends. If a particular creative unit on a specific site is performing well, look to see whether other units of that size, or other units on the same site are performing similarly. Also look at whom that unit is reaching. Is it reaching the desired target audience, or is it being delivered to many people who are not relevant to the brand?
Sample size is also critical. We suggest a minimum of 50 respondents per cell before making any decisions. With a smaller sample size, the results are very unstable and each incremental respondent can shift the data quite drastically.
Consider the relative cost per placement
When evaluating the relative performance of each creative, site or placement, consider the relative price paid for each. While video units may often perform better than standard display units, for example, the higher cost may not justify the incremental branding impact. According to Joe Rose at MediaVest, "Cost-per-increase is a better measure of success than branding metric increases on their own, and is more in line with how we interpret performance for direct response campaigns." When designing a branding effectiveness study, talk to your research partner about incorporating prices to calculate cost-per-increase metrics.
Michelle Eule is SVP of digital solutions at Dynamic Logic, Millward Brown's digital practice.
On Twitter? Follow iMedia Connection at @iMediaTweet.
"Small cat is hunting on green grass" image via Shutterstock.