Billboard Magazine has published an analysis of Top 100 charting song lyrics since 1958, when the modern chart began. The result? Songs about love top the list (65.8%), then songs “about” dancing–as in, “The Hustle” (7.1%), self-empowerment songs are next (6.9%), followed by anti-love songs (6.6%), a category labeled “surf city” which includes Toto’s “Africa” and “Wild Wild West” (4.1%), war (2.1%), funny novelty songs (1.4%) and then an “other” category that is 60 songs large.
Since I review about two dozen papers a year that employ the same methodology, and come to the same conclusions as Gary Trust, I thought this was an opportune moment to help future paper authors learn my preferences and biases. I trust other culture/quant scholars–or, social scientists in general–would have the same sort of concerns, so I think about this as a kind of community intervention with me as the voice. I hope this will change my life. Here we go:
1. Present us with a puzzle. Just reporting counts of things in categories doesn’t make for interesting reading. Why should I care about the lyrical content of the Top 100? If, at the end of your analysis you write something like, “When it comes to smash songs, for half a century and counting, simply, we’re obsessed with obsession”, you’ve got several massive problems.
- First, this is boring and an extraordinarily trite observation about American culture. And even if I spin this as a critique of the pop charts, you’re still not telling me anything that hasn’t been said a million times before. Americans love cheezy pop music that focuses on love and dancing, and almost never addresses stratification or politics. Wah, wa. It just makes me think you’re kind of an idiot for expecting differently.
- Second, you’re not able to draw this conclusion (“we’re obsessed with obsession”) from your data. In the preparation to write this sentence, your mind twisted the generic concept “love” into the pathological condition of “obsession.” Your love might be unhealthy, but you’ve provided no evidence to support the claim that the love described in these songs has an “obsessive” character.
- Third, you’re overgeneralizing. “We” don’t all listen to pop music, and even if “we” did, the pop charts are a totally inappropriate index of our state of mind.
2. Describe your method. In this case, we are told a great deal of information (for a short article) about the count of songs, and time range of the data, and their relative numbers within these categories, but I have no freaking idea how the input (song lyrics) produced the output (8 categories, and an “other”). Relatedly:
- How did you decide what the categories would be? Did you count all the nouns in the songs first, then identify synonymous concepts and keep reducing complexity in this fashion until you arrived at a total of 6 groups? Did you look for some of these themes on purpose, and others emerged as you noticed patterns in the data? Did you do this by hand? Use a data analysis tool of some kind? What about just “word count” in Word? Did you have anyone else look at the data, and see if their categories matched yours?
- How did you sort songs into categories? If a song primarily focused on romance, but it took place at the beach (e.g., the Beach Boys’ “Surfer Girl”), did you put this into the “surf city” category or the “love” category? Was the decision a function of the emphasis in the song (some internal count of word mentions?)? Was it more impressionistic? Did the sound of the song influence your decisions, so that love songs that had more beats per minute were eligible for both the “love” and “dancing” categories?
3. Make sure your categories are non-redundant and have face validity. In this case, I simply can’t make any sense out of the “Surf City” category (which doesn’t actually seem to be about surfing at all, but rather songs that mention places?). Love and anti-love seem like the same conceptual terrain to me, particularly because most love songs I know express a little skepticism about whether love is actually some kind of trick or drug that’s going to wear off, leaving the victim alone and in pain. “Dancing” is a total cop-out, and seems heavily weighted toward a particular moment in the pop charts when inventing dances to accompany songs was part of the larger branding process within the music industry. That you’ve collapsed those songs (e.g., “The Mashed Potato,” “Land of a 1000 Dances”) in with dance songs (e.g., the Madonna catalogue) is more ammunition for my complaint about your methods.
I’m sure this is better-than-average reporting for the Billboard folks, and it is one of the few times I’ve seen them publish something that purports to be social science (he writes: “(And, I could finally put my psychology minor to use).”). But in the effort to appeal to an audience, this article both fails to meet the lowest criteria for social science research and, perhaps worse, it fails to be information.
One final note: Don’t ever write a paper that I might review that focuses on the following puzzles:
- “Why doesn’t the audience for X music realize the artists aren’t good musicians / aren’t authentic / aren’t political enough? It is clear as day, to me.”
- “Why isn’t pop music more political, like I’d like it to be?”
- “How come rap artists are so preoccupied with money and sex?”
- “Is rap sexist?”
These are almost irretrievably stupid questions, roundly criticized by every legitimate scholar (and most non-scholar, music experts), and So. Played. Out.