First things first: Dating advice
When you collect the right data and get it labeled, you can find really interesting things. For example, record over a thousand 4-minute speed dates and have the participants rate each interaction for how flirtatious they were and how flirtatious their partner was. Key finding: You thinking that someone is flirting doesn’t correlate to them saying they are flirting. It correlates with YOU saying that YOU are flirting.
Getting to meaning
The world is full of concrete things like bananas and bathrobes. But people do a lot more than just refer to objects in the world when they are speaking and writing and speed dating. They express emotions, they draw in their audiences or push them away, they share opinions about things and ideas in the world.
The most recent NLP Meetup was a tutorial about this by Stanford computational linguist (and Idibon advisor), Dan Jurafsky. You should go look at the slides, but it covered three useful areas: 1) how to think about affective stances, 2) which resources are out there ready for use, and 3) how to use computational linguistics methods to develop new resources, with examples from restaurant reviews and speed dating. Dan’s got a book on the language of food you should go check out—our post on it includes a video of him discussing those findings and methods in detail.
Emotion detection
There are lots of signals that convey emotion—speech rate, facial expressions, pause length, darting eyes. Like Dan’s talk, I’ll focus on words.
The first set of resources Dan presented were lists of words and phrases along the simple dimensions of positive/negative/neutral. There are fewer resources that do that by emotion.
Psychologists often talk about there being six basic emotions—Pixar even made them into the characters for Inside Out. But in practice, psychologists have had a hard time keeping the list that small. In a review of 21 different theories of basic emotions a few years ago, I counted 51 different emotions posited as basic. The average number of basic emotions posited across the 21 studies is 9. Here are the most common:
- Anger/rage/hostility (18)
- Fear/fright/terror (17)
- Joy/happiness/elation/enjoyment (14)
- Sadness/sorrow/distress/dejection (14)
- Disgust (12)
- Shame (9)
- Love/tender emotion (8)
- Anxiety/worry (7)
- Surprise (7)
- Guilt (6)
There are definitely cases where emotions are more actionable than plain positive/negative/neutral sentiment. But even if there are certain fundamental emotions, they won’t be useful if they are unrepresented in the contexts you care about. For example, a customer care center can make use of emotion detection, but the emotional universe of customer support is really more of disappointment vs. resignation vs. cold anger vs. hot fury vs. relief. Understanding if someone is actually full of joy by the end of a call is useful, but it doesn’t help in routing in the way that understanding level-of-irritation can.
This connects nicely with a theme in Dan’s talk—building statistical models that are based on the categories you care about. (If you want to know about Idibon’s take on this, check out our blog post on adaptive learning.)
Priors: The most important method in the talk
We know that conservatives and progressives talk about issues differently. But a lot of statistical methods for distinguishing Group A from Group B result in uninformative lists, either dominated by frequent words (the and to tell you a little but not much) or very rare words (how informative is it if we say argle-bargle is a conservative word just because Antonin Scalia used it once?).
Computational linguistics comes down to a lot of counting. A “prior” is something that lets you incorporate background information. Check out Dan’s slides 62 and 63, but don’t let the equations frighten you. Mark Liberman’s post on Obama’s language may also help.
Like Dan and Mark, one of my favorite papers on priors is Monroe, Colaresi and Quinn (2008). It’s useful because it walks through so many (inferior) methods before presenting priors and what they get you. For example, they show that in reproductive rights debates in Congress, Democratic representatives disproportionately use women, woman, right, decision, her, doctor, while Republicans use words like babies, kill, procedure, abort, and mother. These very different framings make sense and are interpretable. Other results aren’t nearly as clean.
For more background on priors, you might want to check out the current draft of the update for Jurafsky & Martin’s Speech and Language Processing, which is one of the most widely used textbooks for NLP. Check out the chapter on classification and the one on building affective lexicons.
A few other tricks
Dan mentioned a few techniques that are simple and surprisingly effective. For example, building a system that really understands negation is very difficult. But you get a long way by just detecting linguistic negation (not, n’t, never) and then doing something that amounts to flipping any positive words that follow, up to a comma or period. This method will get you the right classification for something like It’s not a hilarious or thrilling film, but it is majestic. You detect that not is a negative word and therefore treat hilarious and thrilling as negative, too. Otherwise you’d think this mostly negative sentence was mostly positive.
Another clever technique comes from Peter Turney, who wanted to get the semantic orientation of phrases in reviews of cars, banks, movies, and travel destinations: what phrases point to a thumbs up versus a thumbs down?
Knowing that things like “preposition + article” (on the, at a) don’t do much affective work, he came up with 5 part-of-speech patterns that basically gave him meaningful adjectives or adverbs with a bit of context (see Dan’s slide 41).
Every heuristic has its blindspots and certainly phrases like mic drop, swipe right, and on fleek are opinion-bearing phrases. Because of their parts-of-speech, these would be excluded from Turney’s calculations. But the Turney algorithm does find things like very handy and lesser evil. And as Dan said, “The recovery of virtual monopoly as negative is pretty cool.”
Getting at what matters
How many retweets or Facebook likes will Super Bowl campaigns get? How many minutes was a movie, how much did an appetizer cost, how tall was your date?
These are all, arguably, objective facts that you can measure. But there is another way of thinking about them: they are entirely superficial aspects—and therefore not even really about the ad/movie/restaurant/date at all.
By contrast, how did sexism get combatted in last year’s Always #likeagirl campaign? A feeling of wonder in a movie, the awfulness of service, the engagement from your date: these are subjective matters but they can be assessed and reveal deeper insights than the surface facts that are easiest to count. Natural Language Processing is still about counting, but it opens the possibility to count what counts.
Corpus linguistics 