This post was written for Escape Velocity Blog and first appeared there.
Sen. Lindsey Graham: “How much influence did these ads have? …are there tools you can use to evaluate what impact these ads might or might not have had on each of your platforms with respect to the opinions… that Americans were forming…?”
Colin Stretch (Facebook’s General Counsel): “…we do have tools to help advertisers measure their return on investment… to help them understand different campaigns… For campaigns like we saw… they were intended to drive followership of the pages – getting people to like the page. There the return on investment is clear in how many people liked the page.”
Sen. Lindsey Graham: “So would you consider the return on roubles invested, on the money they invested, did it make sense? Were they under or over expectations?”
Colin Stretch: “I can’t say what the expectations were; I do think it’s clear they were able to drive a significant following”
This conversation happened right at the beginning of the Facebook, Google and Twitter testimony to the congressional committee hearing Russia’s election interference in last year’s U.S. presidential election. It demonstrates some of the confusion prevalent in measuring digital and social media activities… What should the campaign intention be? Are we measuring campaigns to manage them, or are we measuring in order to understand a campaigns’ actual impact? How do we define impact, and what tools do we use to measure this impact? What should the expectation of the ROI be from this medium? Judging from the answers given at the hearing, not many people, even within Facebook, understand how to quantify the ‘impact’ of activities conducted online in an offline environment.
What is the impact?
When Social Media took off in India, the first thing that brands asked was, “What should we do on this medium?” Advice came aplenty. Now that some of them have been on it for a few years and poured in more than a little money, they’ve started asking, “What is the impact of all my efforts on it?” The conversations that happen then are not different from the one quoted at the beginning of this post.
There are fundamentally three questions that need to be answered when assessing the impact of any marketing activity:
- Has the person of my choice – Target Group (TG) – viewed the message?
- Having viewed the message, does the TG think of my brand differently?
- Did the TG change behaviour / take an action that benefited (purchase of) my brand?
The muddling up of the objectives
Prior to any mass media activity, businesses and / or marketers are extremely diligent in defining TG, brand attributes, key message take-out, desired action (marketing objective) etc. Once these are clearly defined, the answers to the three questions above are obtained through measurement of the relevant parameters, and the success of the activity is determined.
But, when it comes to digital and social media, critical points such as objective, key message take-out etc. are not clearly defined, which makes measurement of impact difficult and accountability of the digital agency even tougher to enforce. At best, some effort goes into defining and answering the first question – reaching the right TG. Beyond this, thanks to strategies that social media evangelists and agencies propagate, ‘engagement’ becomes an end in itself. After all, isn’t social media all about, “listening in to conversations that customers are having and then participating and connecting with them by initiating and having more meaningful conversations?”
Then there are some more ‘activities’ thrown in as ‘objectives’ and ‘goals’ by the networks themselves – building presence, creating awareness, driving discovery, acquiring customers, customer service/experience etc. With a plethora of such ‘objectives’ floating around, it’s not surprising that businesses and marketers are confused and sometimes forget that these are the means to the end and not the end itself.
While setting campaign goals, it’s critical to focus on the ultimate objective – to induce a behaviour change in the consumer that leads to cash registers ringing – and tie-in all activities and metrics to it. Without establishing the link, any money you spend online is akin to gambling on little more than faith.
Drowning in data and measurements
Everything that the brand does during a campaign and how the consumer interacts with it is tracked and logged. Multiply the number of channels in play with the number of ad formats with the number of user actions possible and you get a sense of the amount of data to parse. For example, a brand could use a combination of websites and social networks, to show a mix of rich media, text and video ads using one or more ad servers that require a user to click and reach a brand page and eventually buy something.
Then there is the bigger problem of attribution and trying to measuring online-to-offline activity, which is increasingly complicated by the usage and movement of the users between devices, measurement in platform silos, non-linearity of user actions and use of tracking protection etc. For example a user might have viewed a brand video on a mobile app, remembered it and then searched for the brand on his desktop browser wherein the text ad was shown, clicked on it, landed on the website, read about the product and finally made an offline purchase a few weeks later.
With the amount of data captured in the example above, it’s easy to get tangled in answering activity level questions – trying to piece together what happened, where, why / why not etc., and lose sight of the more fundamental questions that we mentioned towards the beginning of this post. Marketers today have access to more data than ever before, yet there is still a lot of confusion about which data to track and focus on for assessing the impact on the brand.
An approach
While there isn’t a one-size-fits-all solution to measuring impact, here’s one approach – always think about these three fundamental questions. Start with understanding your online audience first. Then your marketing goals, the outcomes that you want to achieve. Identify data sources you need to help answer those outcome questions – both online and offline. Find ways to integrate these sources to get a more holistic understanding of offline actions basis online views. If real time decisions are needed, use estimates and proxies to tie media metrics to business outcomes. And of course, set up the right tests to validate or reject the hypotheses later.
For example, you might want to answer the question, ‘has the message been viewed by the right TG?’ Then, it would be critical to segment and zero in on the right audience prior to the activity, understand if the media buy reflected that audience, if the ad was served to this right audience and if this audience actually viewed the ad. The extended demographics of the audience, viewability and attention metrics of the ads are a few measures that help you answer this question.
If you are trying to answer, ‘has the TG changed his/her behaviour and purchased a brand in-store,’ determine what the best proxy to measure this is. For example, if the purchase funnel shows that awareness is the biggest driver, then evaluate ‘lift’ in ad recall and brand awareness scores in research studies apart from actual buying itself. If decisions need to be made real time, then use proxies of media metrics (such as mobile ad views, website visits, video completion rate etc.) that tie-in with the business outcome of purchase.
I’ve often heard digital agencies say that more measuring means limiting creative thinking, that the power of the medium is the agility that it offers, and that one can operate on an iterative and ‘learn as you go mechanism’. With the incredible growth of online advertising, it is high time businesses and marketers had access to a robust methodology to measure how brand perception shifts and sales are driven by activities conducted online.
In the next post, we’ll look at some ways that large firms measure impact and how we can improve them.
