Do You A/B Test Your Creative?

Web site developers consider A/B testing table stakes for building good sites. And back in the day, advertising copy was focus grouped either formally or informally. But the technology for A/B testing is not used often enough before brand buys to help advertisers discover which ad will “pull” best, perhaps because in the past split testing was only used for direct response, and not for brand.

But what if we A/B tested brand ads as well?  We test our high impact formats primarily for CTRs, completions (video ads) and  viewability, and the  different creative perform differently on the same format on the same sites.

Here’s how to fun an A/B test on your ad creative:

1) Decide what you want to test. You can test copy, color, layout — virtually any element, however small or large, in an ad. Web developers have often found that clicks improve with button color, and that visibility improves with page layout. Of course you won’t get accurate results if you test too many variables, which is why A/B testing is now conducted with software like Optimizely, which allows you to select one variable at a time so you know your results come from the single variable. Traditionally, ads have been tested with different headlines, or perhaps different photos, but more is necessary. Whatever you do, don’t try to test two completely different layouts, since you’ll never know which different element caused the success rate to increase.

2) Decide also what the goal of the test is and set a metric for it. Is it brand lift? Is it engagement?  Typically brand lift is measured after the buy, and only determines success post facto. By that time you’ve spent your budget, and what if the ad didn’t pull? Fortunately, it is now possible to measure time spent with an ad. Especially for video, completion rates are an indicator. However, it makes a difference whether a video is part of a forced view, as in pre-roll, or is skippable, as in our own inArticle format. If the user is forced to watch then you won’t see low completion rates even for poor performing creative.

3) Grab your A and your B. The A is usually called the control, and the treatment is the B. One use of A/B testing in advertising is for “brand lift” was Obama’s second campaign, where this form of testing took a candidate in whom many people were already disappointed and still brought that candidate over the finish line. That campaign used A/B testing on every single variable associated with the campaign from the creative to the targeting to the selection of volunteers.

4) Run your test. I can imagine agencies saying they just don’t have the time or the resources to test too many variables, or that the client won’t pay for it. But trust us, clients will pay for results, and a little measurement given to the creative up front could be a differentiator for an agency. And for the client, it’s a potential big increase in ROI. Or just ask us and we will do it for you.