Measuring the effectiveness of a brand campaign remains elusive even online. Digital marketers often look at engagement metrics such as likes, follows and shares. These are one-time boosts to the brand. The situation changes when you focus on people (how many advocates you have) rather than content (how many likes a post got).
Advocates connect with and identify with your brand. They actively share, promote and defend your brand, often creating their own content on social media. Developing these relationships with customers and prospects delivers ongoing predictable value. You can count on them re-tweeting your content and creating their own.
So, how do you define and measure success?
You can identify advocates and measure success of marketing campaigns using natural language processing. Idibon analyzed the #likeagirl campaign, which Always ran during and after this year’s Super Bowl.
Categorizing messages by their content enabled us to understand engagement with the brand and social issues. In the case of the #likeagirl campaign; we’re not just measuring an ad campaign, we’re also looking at discussions of equality and empowerment. We can do this sort of analysis with any kind of social media or other text data but tweets are especially handy and offer a better demographic representation than most other social networks so that’s what we’ll focus on here.
Calculating the social media ROI of a brand campaign
The average price of a 30-second ad for this year’s Super Bowl was $4.5 million–Always’ ad was 60-seconds and followed the half-time show–obviously there are additional costs for video production, but let’s just call $9 million the cost of the ad. There were 114.4 million viewers overall (118.5 million tuned in for Katy Perry’s half-time show right before the Always ad). That translates to a willingness to spend about $0.08 per viewer.
Some of these viewers took to social media to share and extend the brand message. But they did this in different ways–not all viewers have the same value. Some, like the 2,519 most influential advocates discussed below, help extend the campaign much further than Always themselves.
Compare that to Anheuser-Busch who invested in seven 30-second spots, valued about approximately $31.5 million (though they are likely to have gotten a package deal). That’s a rate of $0.28 per viewer. Not everyone watching the Super Bowl is a potential beer-buyer but proportionally, it probably is the case that more Super Bowl viewers are candidates for beer than for Always’ products, so it does make sense that they would invest more. But did they get equal benefits?
Always’ and Anheuser-Busch commercials were both positively received, but Budweiser was only mentioned during the game about 234,000 times on Twitter. During the game, there were about twice as many #likeagirl tweets, even though Always spent less than 30% of what Anheuser-Busch did.
Understanding the subtleties of engagement
Knowing that someone has engaged with your brand or cause is a great start. Understanding how they have engaged offers you insight and enables brand managers to take action.
In Always’ campaign there were six types of content creators: @Always, brand advocates, ad advocates, cause advocates, antagonists, and defenders. The total data set takes all of the “#likeagirl” and “#likeaboy” tweets from a couple days before the Super Bowl to a month later, a total of 648,102 tweets. Let’s focus first on the most influential users.
Table 1: The most influential content creators
|Type of content creator||Description||Number of people||Number of influential messages||Re-shares|
|@Always||Corporate account||1||70||52,393 (13%)|
|Cause advocates||Actively friendly to @Always (or their parent company, P&G) and explicitly mention Always while being positive about them and/or the campaign||256||1,401||89,424 (22%)|
|Ad advocates||Do not mention Always or P&G but do mention the advertising campaign in positive terms||537||562||56,039 (14%)|
|Brand advocates||Praise the cause, though they aren’t mentioning Always or the ad—a lot of people co-opt “#likeagirl” but in a good way, like sports teams promoting women in sports or particular events.||1,192||283||42,396 (10%)|
|Antagonists||Post sexist messages that demean women||368||439||61,354 (15%)|
|Defenders||People who take the sexist comments head on||534||634||106,702 (26%)|
|* Influential messages are defined as ones that had 9+ re-shares|
While we want to track the engagement of advocates across campaigns, we also need to understand detractors. In the case of #likeagirl, critics created sexist posts that demean women, e.g. “make me a sandwich #likeagirl”.
If antagonists are the negative response to a campaign, defenders are the advocates that counter them. These are the people who take the sexist comments head on. They offer examples of messages that might be detected as “negative” but which are actually positive about the campaign. Brand managers may or may not want to engage with defenders but they are a crucial part of the advocate ecosystem, especially in cause marketing.
Influential messages, influential advocates
Not all posts are created equal—some messages circulate more than others. Most tweets don’t actually get retweeted or if they do, only get a single retweet. The overall statistics suggest that, in this data, if you have 9 or more retweets, there’s something special about you as an author and/or about your message. So we’ll define a prominent message as something that gets nine or more shares.
There’s even more content that is re-shared that starts with someone else. There are twice as many posts by brand and ad advocates as there are posts by @Always (even including re-shares by their followers). And three times as many if you include posts by cause advocates. Ultimately Always’ marketing people want advocates to promote the brand in their own words and defend it in their stead. And that’s what happened.
Re-shares of @Always
@Always came up with some very good posts. There are only 186 #likeagirl tweets posted by @Always but these are hugely successful. Twenty-nine of @Always’ original content got more than 100 retweets. This is the most popular one, which got thousands of retweets:
Nearly a third of @Always messages that got re-shared were @Always retweeting someone else or throwing their weight behind someone else’s message. Always’ reply to Ivanka Trump was retweeted over 100 times.
In general, @Always’ original content was more successful than the content they chose to re-tweet.
Let’s think about re-shares more broadly. Re-shares amplify the brand message but they are a one-time boost. Most people only have a single engagement. Most people (94%) retweet @Always content once. There are 4% who retweet twice.
Finding and nurturing advocates can deliver an ongoing boost to the brand. Two important measures of campaign success are how many original content creators are positive about the brand and how far their reach goes.
Controversy and solidarity
For #likeagirl, people focused on the positive message of empowerment and on their own personal reactions. The sexist discourse within #likeagirl was mostly ignored. While there are plenty of people tweeting nasty things with #likeaboy, they are swamped by the number of people coming the defense of the #likeagirl campaign (and gender equality more generally).
If we lumped everyone who used the #likeagirl hashtag together, we wouldn’t be able to keep track of the difference between people who speak about #likeagirl as an ad versus those who talk about it as a cause. Off-the-shelf sentiment analysis tools are also likely to confuse Antagonists and Defenders since both groups are using negative terms.
Machine learning models are flexible and accurate only if they have relevant training data. The reason most sentiment analysis tools have low accuracy is that they are trained on some set of data that isn’t what you’re applying it to. Idibon optimizes “the human-in-the-loop” so that it’s possible to efficiently get training data to create models specifically for the data you care about, structuring it along the lines that you care about.
In the table below, we look at “typical” users—not the highly influential people, but the enormous number whose content had only zero, one or two re-shares. This gives us a lay of the land, the buzz beneath the buzz.
Table 2: Percentages for “normal” messages (0, 1, or 2 re-shares)
|Type of content creator||#likeagirl messages||#likeaboy messages||Messages with both hashtags|
Antagonists who were trying to get #likeaboy to trend (become a top hashtag on Twitter) got their wish partly because they chose to be offensive in a way that was hard to resist wanting to squash. Antagonists posted the highest density of sexist content to messages that they tagged with the #likeaboy hashtag. Defenders overwhelmed them with 3-4 times the amount of content.
Always may not want to get involved in negativity. Understanding the different types of advocates—in this case Defenders who are actively defending the brand value of empowerment—lets brands shape campaigns and responses in new ways.
Who was that masked crusader?
Being able to identify the people who support your cause or brand is valuable in and of itself. (Send them a gift basket! Re-tweet them!) In our next blog post, we’ll discuss the affinities of advocates in a way that gets beyond static demographic categories and into what advocates say they care about in their outside lives. This offers an even better way to get to know people engaged with your brands.