By now, everyone knows the routine. You look at a product on your favorite website. And lo and behold, that product follows you around the web on all the sites you like to visit — like a bad headache.
But does it really work? It must since retailers use it, right? Of course, the technology partners who provide this service give it glowing reviews. And then there is the other side, skeptics and cynics like me who are annoyed and feel like our privacy is being violated by this intrusion into our lives. But what does the data tell us?
I had the opportunity to dig into this issue at length and uncovered some interest facts. But first some background.
Test / Control is the most common measurement technique used for this marketing strategy. You set aside one group, the control group, and send them generic messaging like public service announcements. You find another group, the test group, that matches the characteristics of the control group. This group receives the marketing banner. You run the campaign for a period of time and then compare the conversions for both groups. The conversion delta between the test group and the control group is lift assuming the groups remain ‘pure’.
There is inherent bias in testing strategies used for digital banner advertising.
No matter the mitigation strategies, digital banner advertising tends to bias towards best customers. It skews towards messaging the more loyal customers who perhaps need less encouragement to convert. I and my colleagues never could totally get our arms around why this is true. After looking at the methodology and the process, things seemed sound. But the data consistently told me that while bias could be reduced, it was still inherent in the program.
To establish comparable populations, there must be knowledge sharing between the technology partner and the retailer. The technology partner knows things about the customers that the retailer doesn’t. And vice versa. Creating equal test / control populations and separating that from the general population is critical to measure the program and that requires as much understanding of the customers as possible.
Contamination in the test / control populations
One distinct challenge of evaluating marketing like this is the very nature of digital. Between people with multiple devices (I’m an old-timer but I’ve got 3 devices myself), cookie management, etc., it’s very difficult to keep the populations pure. You may start with pure test and control groups but over time with different devices and when cookies are cleared, members assigned to one population may end up in the other population. And analyzing data when the populations are contaminated is a real challenge.
People Rarely Click-Through Banner Ads
From what I can see in the data I looked at, people rarely click-through a banner ad. That makes analytics even that more difficult since it is dependent on indirect relationships in the data between the impression and the conversion. In most digital marketing analytics, it’s typical that you would place a URL parameter on the marketing link identifying the marketing program and then capture that in your digital analytics. That’s pretty common with emails and other forms of digital marketing, but it doesn’t really work with banner. Banner is as much about reminding the potential customer about the product and your brand, as it is about getting the click. They may see your banner, but then type in your site to actually buy, or they might be more likely to click the next email or walk into a store. So you have to come up with different methods to tie an impression to a conversion. And then the impacts to offline sales are even more indirect. Stitching the data together is like solving the Rubiks Cube.
There is a lot of data to sift through
The data collected in this process is overwhelming. Without some type of big data strategy, it can be virtually impossible to even deal with the volumes of data.
In order to measure the Return on Ad Spend (ROAS), great care has to be taken to properly setup and manage the test / control populations to make sure that they remain as equivalent as possible. Minimizing bias and contamination is essential. That’s a lot of hard work but the good news is that it can be done. If you have ‘pure’ groups, lift (increased sales in the group tested with the banners) can then be calculated. Once you have lift, ROAS follows. However if the populations become contaminated or biased, more creativity and subjectivity is introduced.
So without further ado, the answer you’ve been waiting for:
Is banner advertising effective? Answer: Depends.
If digital banner advertising is done incorrectly, it would be better to donate those monies to your favorite charity. At least you could get a tax write-off.
But digital banner advertising can be a strong marketing channel and can be measured if done correctly. It’s difficult and takes a lot of hard work but it can be done.
In our years of experience across several organizations, we have seen things we thought should be a slam dunk fall flat, and things that looked unlikely turn out to be a home run, and this includes the type of advice you might find in a blog like this.
The variables of our audience, industry, brand, and product have a larger effect on your banner ads than any list of tips or tricks.
Therefore, we truly believe the only quality advice we feel justified in offering is to follow the above steps to 1) create a test / control for each type of banner ad 2) build the environment and analytics you need to be sure your results are meaningful 3) use carefully crafted metrics to measure results, and then 4) double down on the ads that work, and refactor those that don’t.
Contact us if you need a partner to help organize your data, validate your performance analysis, and take advantage of the results. For WordPress users, consider the Marketing Performance Plugin, our free plugin that creates a clear, 360 degree picture of your marketing efforts with the true value of banner ads by channel, campaign, and ad.