By David Lockwood, Co-Founder & Insight Director of The Tapestry Agency
Measuring channel performance can be tricky. In the digital age, there are multiple touchpoints involved in making a sale, and these are interconnected in complicated ways. Social channels such as TikTok, Facebook, Instagram, and Pinterest are mixed with channels such as online search, affiliate marketing, and referrals. On top of this, there’s the offline channels such as direct mail, press ads, inserts, TV, radio, and of course PR, stores and wholesale, to mention just a few.
Recent developments have presented further challenges for multi-channel retailers. The marketing landscape has been rapidly changing due to COVID-19, the deprecation of third-party cookies, and the advent of ATT (App Tracking Transparency) in iOS and iPadOS. These have made targeting more difficult, which in turn has made channel measurement more difficult.
In our latest Retail Leaders Forum, David Lockwood shared Tapestry’s guidelines for building a fractional attribution model for more accurate channel measurement.
1. Understand the data you already have
First of all, Tapestry recommends going back to basics and taking time to understand your current sources of data. These usually include website visitors (typically Google Analytics), promotional tracking codes, and mailing match-backs. There will also be data you can’t link to individual customers or orders, such as self-reported social media data. Make sure you understand the intricacies of how that data is collected. For example, what is your conversion window on Facebook Ads? This may historically have been a seven-day view-through/click-through, but Tapestry believes that a one-day view-through/click-through may yield more accurate measurement, since it aligns better with recent changes made due to Apple’s ATT.
It’s also important to find out how your current sales are internally reported. Often, retailers will record one sale against one channel, then rank channels according to how many sales they generate. Naturally, you want to build a more sophisticated system over time, but it helps to appreciate how your current method works as a reference point.
2. Know your tools (and their limitations)
Your model will involve a number of data streams and a large number of variables. You’ll need to work out whether you have the data analysis skills in-house to understand all of the outputs and manipulate the data. You’ll also need the right software for the job. This isn’t something you can do in Microsoft Excel or Access; the application needs a relational database to handle the modelling.
3. Build a measurement matrix
Once you’ve got a handle on the data and your software tools, you can use them to begin investigating your channels. Start by assuming each channel accounts for 100% of sales, then overlay these figures for comparison to create a matrix of measurements. This will reveal overlaps and disparities between the degree of attribution each channel has. Be prepared for the matrix to get quite large, since the number of metrics will need to be multiplied by the number of channels.
Typically, you need three or four supporting data points to be confident about a channel’s effect. For example, if a similar proportion of sales coming from Google’s paid search reported by Google Analytics also uses a promotional code, which is only visible in that paid search, the attribution is more likely to be reliable. If there’s a disparity, for example if the same proportion of sales comes from paid search as from match-backs, then you can examine that crossover point more closely to understand how the channels relate to each other.
4. Perform incrementality testing to estimate lift
To understand fractional attribution, you need to isolate each channel’s effect. One of the easiest ways is to set up a series of incrementality tests using ‘holdout cells’ to find out what happens when you remove certain marketing activity. Essentially, a campaign audience is divided into two test groups: one which is identifiably exposed to the channel and one which isn’t (the holdout). If you keep other factors constant, the difference in sales between groups will reveal the incremental lift provided by the holdout activity.
But interconnections will create knock-on effects. For example, you could suspend direct mail from one group, which would remove match-back demand, but certain proportions of demand may also be lost from affiliates, search, and social media over the same period. These effects will give you ideas and clues as to how those channels interrelate. Apply those two test groups to your measurement matrix to see how you’ve lost demand, where it’s been lost from, and by how much. With this information, you can start to build a fractional attribution model.
5. Use increasingly sophisticated testing to improve your model
Incrementality testing can be applied to almost any medium or variable, e.g. uplifts in different geolocations, or ad inserts on particular TV networks. Then you can run the results through your measurement matrix, improving your model at each step. The most important thing here is not to stop, but to continue with this process. For example, just because you knew what your incremental paid search performance was two years ago, don’t assume it’s the same now, because it won’t be. Your audience will have changed, your marketing is likely to have changed, and, as you add and expand marketing channels, the incrementality of each of those channels and how they interrelate to each other begins to change. Your model won’t be perfectly accurate from the outset, but what’s important is that you get to understand your attribution a little better each time.
You can watch David Lockwood, Tapestry Co-Founder and Insight Director talking through these steps in this video: Attribution modelling to help multi-channel retailers understand marketing channel performance.
