Achieving great performance results from an online advertising campaign is an important goal for many marketers, but it's a short-lived victory when one is not able to repeat that success. Taking the time to set up proper testing beforehand, however, can make all the difference in repeating successful campaign results again and again.
In this article I will review how taking the time to create a control group and configure the campaign for multivariate testing enabled one advertiser to replicate performance across their advertising campaigns to achieve exceptional ongoing results. This example will demonstrate how applying these testing practices can not only lead to outstanding results, but also provide insight into what specifically made the marketing campaign so successful.
Before launching into this example, it is useful to define what is meant by a control group and multivariate testing. In this article, a control group is simply defined as a segment of people who will not be exposed to the marketing variables you'll be testing. Multivariate testing is defined as a process by which multiple variables for an online campaign can be isolated and tested against each other to determine which defined variable performs best. Applying these two types of testing best practices will insure you make the best optimization decisions for the performance of your advertising campaigns over time.
The following case study demonstrates how one advertiser utilized a control group and multivariate testing to achieve better results, and then used the information and insights gleaned from the results to further optimize and repeat this performance across other campaigns.
Step 1: Determine which campaign variables to test
In this example, the advertiser chose to test the impact of time lapse between visits, the promotional offer, language, and creative size, and then determine which combination yielded the best results.
Time lapse: The advertiser wanted to retarget display advertisements to users who previously visited its site but did not complete a registration form. The advertiser predicted that the amount of time since the last visit to the site was an important variable. The objective was to serve a specific banner based on the specific variable, in this case the amount of time that passed since that visitor had left the site. Visitors who had been to the site fewer than seven days ago were shown a different banner than those who had not visited the site for at least eight days.
Promotional offer: The advertiser also wanted to test two different levels of promotional discounts in each banner. It chose to test a 15-percent-off banner targeted to users who had been to the site within a week's time against a second 20-percent-off banner targeted to users who had not visited the site in more than a week.
Language: The preferred language of each visitor was another important variable the advertiser wanted to test, so in addition to the "time lapse" and "promotional offer" variables, it also wanted to target users by preferred language. It decided to show re-targeted banners in Spanish to those who had visited the Spanish language version of their site. All other visitors were served banners in English.
Creative size: Lastly, the advertiser chose to test two different creative banner sizes to determine if the size of the banner would impact the results. All banners were created in two sizes: 300x250 and 728x90.
With so many variables involved in the advertiser's campaign, it needed a way to attribute and measure the success or lack of success for each variable. In order to reach the campaign goals, the advertiser chose to utilize multivariate testing configured in a controlled environment.
Step 2: Create a control group
Once the advertiser had a baseline for which variables it would test, the next step was to create a control group for the campaign. A randomized 10 percent control test group was configured to be served generic banners on the same media sites as the test creative. Any conversions generated from the control group represented users who saw or clicked on a non-related banner and happened by chance to also convert on the advertiser's site. This control group provided an excellent baseline for confidently analyzing the overall campaign.
Step 3: Develop creative and set up campaign placements
The campaign was then structured with nine total placements to test all combinations of variables: creative size, language, and offer type. By properly setting campaigns and placements up front, the advertiser was confident in the test configuration and ready to launch the campaign.
Step 4: Analyze the results
After running the campaign over a three-month period, the advertiser achieved exceptional results. Because the advertiser effectively deployed in a controlled testing environment, it was able to determine which variables generated the most conversions and sales, and was therefore able to optimize future campaigns accordingly.
Some key findings from the test campaign included:
- Click-through rates for the entire campaign averaged four times higher than the control group
- The best performing set of variables (Spanish 728x90 creative with offer No. 2) had a click rate that was 11 times higher than the control group.
- Average dollar spent on service per click jumped from $5.02 for other campaigns running during the same time frame to an average of $530 per click with this highly-retargeted campaign.
Step 5: Optimize campaigns based on results for repeat performance
Without a doubt, the needle on the client's business was moved in the right direction by this campaign. Due to their foresight and discipline in setting up a control group and making sure they had a large enough sample size to begin with, the results from this campaign confidently speak for themselves, and are verifiable. Since each combination for each variable was carefully tested (banner size, offer type, and language) in separate placements, the advertiser can be confident in which combination of variables working together are most likely to produce the best results in future campaigns.
In addition to providing many answers to better inform the future of advertising campaigns, the results from this campaign invariably raise more questions. For example, since Spanish banners performed better than English, should the advertiser test Chinese or French creative next? The answer is absolutely -- along with a control group and multivariate testing.
Rodney Webster is director of product management at Mediaplex.
On Twitter? Follow iMedia at @iMediaTweet.