This is the finale of a 3 part blog post series, transcribing talks I gave back in 2017/18 on attribution. The first blog post includes the deck from the talks as well as covering how the talk originated, the business problem it is focused on and a review of different attribution models using a football (soccer) analogy. The second blog post goes on to discuss in detail three key flaws that are inherent in all attribution models.
That sets up this final blog post, as there is no point pointing out problems if you can’t offer a solution…
The Alternative to Attribution (slides 44 – 56)
So, after discovering that all attribution modelling and tools are flawed, where does that leave us? Always focus on the business question (the real question, not necessarily the question you were asked).
How can I best spend my marketing budget to make the most money?
Just to note quickly that I don’t recommend throwing away all attribution reports, models and tools. They are still useful, just not as the single source of truth for your marketing strategy. They can provide useful insights into customer behaviour within channels and across channels and should be used accordingly. Of course, with that use case in mind (less value than managing your spend across all channels), evaluate the cost of any attribution tools you are paying for.
The Alternative Approach (slide 47)
What should you be doing then to get the best ROI on your marketing spend if not using attribution models?
Everyone will have seen a diagram like the one below, showing different stages of the customer consideration process, visualised as a funnel. Marketing campaigns may be viewed and clicked by people at any stage but they should be designed for and targeting only one stage. Not only should you know what stage of the cycle each campaign is targeting, you should have thought through how people at that stage who view the campaign should react. A reaction is an action and an action can be measured.
tl;dr – evaluate each campaign against a purpose/action for that campaign which can be directly measured.
Implicit in this approach is the need to stop trying to identify all touchpoints that led to conversion. For reasons given in the previous blog post, this is simply not possible.
An action needs to be identified for each campaign that represents success. This then needs to be equated to a metric and a monetary value assigned to this metric. This is where the data scientists can come into the picture, to determine the likelihood of someone going through to convert in the future after taking a pre-specified action (if that action was not the final conversion). Once you have that likelihood, a value can be calculated and assigned.
Using the value assigned to a metric (that is being measured) and associating it to a campaign (ignoring any other possible touchpoints), you can calculate an ROI. With an ROI, you can start making data informed strategic & tactical decisions for your marketing campaigns and marketing mix.
Identifying Success Actions (slides 48 – 50)
I glossed over the challenge of defining what success looks like for each marketing campaign pretty quickly. How do you do that step of thinking through how people will react to the marketing campaign?
This is where I do a very non-analyst type thing and tell you to forget about data. Forget the tools, cheap or expensive. The answer is not machine learning or artificial intelligence. You need to think about people. And how people behave. Where and when would someone be exposed to your marketing campaign? What should they do next (given their stage of the customer consideration process)? Does that action make sense? How does it take them to the next stage of the process? Can we track it – somehow/creatively?
As an example, let’s say someone in the UK has been exposed to a display ad for transferring money to Hungary with WorldRemit. They click on the ad (great first action) and land on a page providing information on sending information to Hungary.
What do they do next?
Do they scroll down the page they are on to read all about the process for sending money? Do you think they will click around the website researching WorldRemit? Will they go straight to sign up for an account?
Someone near the top of the customer consideration funnel won’t necessarily send money or even create an account immediately. They stumbled across WorldRemit via a Display ad (it was not a conscious search) and might only send money on payday or when the exchange rate hits a certain level. No guarentees then that they will be on the same device when signing up or sending the money either, if they eventually do so.
But my belief is that anyone who is interested in sending money from the UK to Hungary through WorldRemit, whether to do it immediately or at some point in the future, wants to know one key thing – they want to know what it will cost them. They need to know what the fees are and whether a good exchange rate is offered. This information is discovered by performing a calculation. The action of performing a calculation can be tracked and measured. It can then be used to evaluate campaign performance on a last click basis.
Now, this process may sound very difficult to near impossible though. Assigning values to different actions and using to evaluate different campaigns. Is this really possible? Clearly yes, as the same approach is used in fantasy leagues in every sport around the world…
I admit, it is much easier to buy an expensive tool/black box that does all the work for you. The approach I am recommending is much more difficult and will take longer. Depending on the alternative tool that is discarded, it might be cheaper or more expensive. Critically though, it will give you a true understanding of the impact and value of each & every marketing campaign.
Applying this Approach (slides 51 – 54)
Once you have a metric and a value for that metric and are calculating the value generated by a campaign (and therefore it’s ROI), how do you know you have the maths right? Don’t just trust your numbers, prove they right. Run tests on your marketing campaigns, just as you (should) do on your website pages/functionality. And with tests, you have more scope to evaluate against those final conversions, not just the interim actions, ensuring the rest of your maths is correct.
Again, this is not an easy or quick process. You can’t just show half your marketing campaigns to 50% of the population and something different/nothing to the other 50% (for most channels). But at the end of it, you will know the true impact of different marketing campaigns across your marketing channels.
If you need to get creative in running marketing tests, options include:
- Running different campaigns in different geographical regions (or isolate a campaign to a single region)
- If running a display campaign, replace the creative with an alternative that is not your company (be nice, use a charity)
- Pick one half of pairs of products that exhibit similar trends & behaviour, using the selected products in your campaigns
The purpose in each example here is to have a baseline that you can measure the uplift in performance against, for that geographical region, display campaign or set of products.
For the display campaign example, you need a fake campaign to use as the baseline. Some people were always going to buy from you, whether or not a display ad was placed on a page they were viewing. For these people, it does not matter if the display ad was for your brand or for a random charity so using the charity ad as your baseline sets it at the right level.
Taking this approach a step further, causal models can be used to evaluate campaigns, particularly when tracking actions triggered by the campaign is not possible. The obvious example here is offline campaigns, where it is unknown if visitors view the ad. But if you know the campaigns that are active, the level of activity and the total revenue being generated, you can calculate/estimate the impact on revenue from each campaign.
At this point, we are far beyond trying to identify every touchpoint of a customer. Instead we ignore the chaos and just focus on how much money is being made from the money being spent on your marketing.
The Desired Marketing Optimisation Tool (slides 55 – 56)
Putting all of this together, imagine the following set of functionality for my dream marketing optimisation tool (I don’t believe it currently exists).
The tool ingests all the data from historical marketing campaigns. This is not recording all touchpoints and conversions but instead how each campaign has performed against success metrics relevant to that campaign’s purpose. Users of the tool can adjust variables based on factors they know are changing e.g. the creative for a new ad is superior to the creative for the previous ad, therefore performance is expected to improve. Details of campaigns for the next X date range are also entered.
Given this set of inputs, the tool forecasts the incremental sales for each campaign based on the spend and all other factors inputted for that campaign. Note the word incremental used there, that is to align with the knowledge that some sales would happen anyway. And the output from this tool would not be just revenue, but also profitability and ideally, customer lifetime value.
When users of the tool review this output, they can adjust the marketing spend or other factors for a campaign to instantly see an update in the forecast performance based on the adjustments made. As time goes on, the tool would contain more historical data points and forecasts would become more accurate. Especially as actual performance is compared against the forecasts and used to refine the models.
That tool sounds great but is it right for you? Expensive attribution tools are more appealing to many companies, the salespeople use a lot of buzzwords and the tools look very impressive. This alternative means no shiny tool and instead a lot of hard thinking with no quick results.
But put in the time & work and you will have control over all your marketing. You will have a proper understanding of what factors actually impact marketing performance and which do not. This approach, this tool if it existed, would allow you to adjust your future marketing spend and activity in a way that truly maximises ROI.
What Next?
Thank you for attending my talk, I am now available for Q&A. What are your thoughts, do you agree with me or do you still believe in attribution?
Naturally, if this all sounds really interesting, do get in touch. I am always happy to talk if you want to understand more but I would be even happier if you wanted my help to implement a new way to get the best results from your marketing.
Reading and rereading this content, I have the feeling that the jump from theory to practice may feel very large. So keep an eye out for a follow-up that goes into the detail of the actions I would take if challenged to implement my theories.
