Finding Hidden Gems in Music Research
by Joe Knapp, MusicMaster Founder
What if I told you that we’ve been missing some important data for decades when doing music research? What I’m about to tell you may come as a surprise. In fact, when I first spoke with someone about this many years ago, they thought I was insane, and they told me so in no uncertain terms.
There are several methods used to test music and nearly all of them involve polling an audience sample, playing hooks for them, and crunching the results. We try to find a panel that is filled with habitual listeners to our format, either on our station or our direct competition. We first find out if the person hearing the hook is familiar with the song. If not, we note that and move on to the next song in the test. If they know the song, we try to determine how they feel about it. Typically, this is done by grouping the responses into five categories, Like A Lot, Like Somewhat, So-So, Dislike Some, and Dislike a Lot. Sound familiar?
No pun intended, familiar is a big key to the problem. So is testing only your target audience. Why don’t we care how a panelist feels about a song they don’t know? Why aren’t we trying to find new listeners who are outside our target audience?
Continuing the test when a panelist is unfamiliar with a song doesn’t take much more time and certainly doesn’t cost us more money, but it could provide a wealth of information.
Just as an experiment, I decided to commission just such a research project with the good folks at Advantage Music Research. The results were a bit eye-opening. I will share some of them with you today. We had to tinker a bit with the polling algorithm, and we also had to modify the resultant metadata a bit, but that wasn’t difficult at all for the Cornerstone experts to manage.
I did pre-qualify the panelists with a five-hook sample of some cream of the crop oldies that are straight up the middle bona fide hits. We aren’t THAT crazy!
Testing demographics that are outside the comfort zone of your format could bring new demos to your station’s audience. For this test, I deliberately went way outside the usual sixties oldies demo to include 24-29, 30-39, and 40-49. There was an equal split between panelists who identified as male or female, but I believe that distinction is far less relevant than before. The same goes for race, with this test having about a two to one bias toward white panelists. Because streaming is not limited to a local market, the results were also broken down by geographic location, Northeast, Midwest, South, and West, all within the United States. My format has a large percentage of listeners from outside the country, but tossing the net that far would be cost-prohibitive. As it turns out, and this may surprise you, there were very little differences based on scores across the gender, race, or location spectrum, with just a few notable and minor exceptions.
Before I dive into the details, let me also suggest that we may be making yet another mistake when conducting music research. We tend to sample only top songs in our formats, current hits, and recent hits, basically songs that we already know are very strong. We’re basically looking for the weakest of the best so we can toss them out or start rotating them more slowly. That’s fine, of course, because playing fewer songs nearly always results in higher cume. But what if we found some potential monsters in those songs the panelists did not know? If we don’t test them too, we’ll never know.
For this test, I used my MusicMaster Oldies format that streams on Live365, TuneIn, and iHeart. Try “Alexa, Play MusicMaster Oldies” to sample it. This was done for selfish reasons, but I also found it difficult to choose a format that would be relevant to most of you who aren’t playing songs from a thousand years ago. Seriously, everyone seems to know a lot of songs from the sixties, mainly because a lot of them have been recycled in hit movies and TV shows. (Check out TuneFind.com) Others have been sampled by hip-hop and rap artists. (Check out SecondHandSongs.com) A few have been covered in singing competitions, like The Voice and American Idol, and ended getting decent download action over the following days. You often hear sixties oldies in bars, restaurants, carnivals, fairs, and, well, all over the place. It’s generic vanilla music, some of which was very well made and now stands the test of time. Need I cite the continued popularity of The Beatles?
I can no longer trust my “gut” to figure out which sixties oldies are hot, and which are not. The audience I’m trying to attract is not in my demo. I’m the P1 who grew up in the sixties. My panelists, by design, are not.
I did not test my Power category, nor did I test any Beatle or Elvis records. I know which of those I must play. I also tested very few songs from my Superstar artists. Instead, I tested my Secondary category, focusing on a mix of types and demographic appeals. I’m much more willing to tighten my rotations by getting rid of those weaker songs.
What I got was two tests in one. From the folks who were familiar with a song, I got the usual test scores that will help me move some stiffs out of the library. But, from the unfamiliar songs, I found some gems that I may need to rotate faster than I’d previously thought.
Which do you want first, the good news or the bad news?
Let’s start with the bad news.
I found some real clunkers that are being dismissed from my library faster than Tyreek Hill racing toward the end zone for a touchdown.
Don’t feel bad if you don’t know some of these songs. If you must, you can always sample them on Pandora, Tidal, Apple Music, or your music service of choice, if they even have them!
“The Shifting Whispering Sands” by Billy Vaughn was the worst song tested, by far. The song is from 1955 and was a #5 hit in September on both Billboard and Cashbox. It also went to #20 in England in January 1956 and all the way to #2 in Australia in May 1956. It’s dated, and it wasn’t a #1 hit anywhere, so don’t even ask me why I was playing it, even as a Secondary. Oh, and I do care about how songs charted in other countries because, well you know, streaming is global. This song was rated Dislike or Dislike a Lot by 55% of the panelists who were not familiar with it. It was also disliked, either somewhat or intensely, by just over half of the respondents who did claim to know it. That’s awful! Easy one to jettison, and it’s not a shocking revelation.
“Mr. Sandman” by the Chordettes was hated by 27% of those who didn’t know it, and only 5% of those who knew it. That’s more in line with our assumptions about unfamiliarity. Interestingly, neither group had any Dislike Some responses. The song was liked a lot or some by over 90% of those who knew it, and 55% of those who didn’t know it. Maybe they’d subconsciously heard it on a Family Guy episode. I’ll keep playing the song however, but probably not as often. Here’s a surprise for you. This song tested virtually equally across all age groups, genders, race, and geolocation. In fact, most of the songs did the same.
Others that did poorly among people who’d never heard it included “Surfin’ Bird” by the Trashmen, “Coconut” by Nilsson, “Kiss an Angel Good Morning” by Charley Pride, “Whatever Will Be Will Be” by Doris Day, and “Dungaree Doll” by Eddie Fisher.
Let’s flip things around now. Well over half the people who did not know the song “Lollipop” by Ronald and Ruby liked it a lot. Another 22% liked it some. I did not see this one coming. That song is from 1958 and the Chordettes had the bigger hit with it. Ronald and Ruby did it a few months earlier and did not do nearly as well in airplay or sales. The Chordettes do sound more dated now, and Ronald and Ruby were an R&B duo, R&B oldies do tend to have more staying power. Interestingly, Ronald and Ruby’s version did not make it on the R&B charts, but the Chordettes version did very well there. But the Chordettes version was included in the test, and it performed almost the same as Ronald and Ruby. In fact, it had almost no negative responses, with zero negatives among those who’d never heard it before.
Speaking of R&B, which is one of my favorite things to talk about, “Sweet Soul Music” by Arthur Conley was liked or loved by a whopping 78% of those who did not know the song. It tested positively with 87% of those who did know it. “I Got You (I Feel Good)” by James Brown was the third most loved by those who had never heard it before, clocking in at a healthy 87%. It did even better with those who know it at 94%, but that’s no surprise. What did surprise me was a ringer I threw into the mix that some may not even consider playing on an “oldies” station. “Changes” by David Bowie had positive responses from 67% of the panelists who did not know it, and 86% of those who did. His music did get a lot of play on TV after his recent death, and his vocals may resonate with the gay community more than others from back then. The song was even used in an episode of NCIS in October 2021 called “Face the Strange.”
The correlation of like scores between familiar and unfamiliar songs was almost insignificant, with only a handful of exceptions. “Killing Me Softly” by Roberta Flack was, by far, the biggest example of that. Those unfamiliar with that song did not like it much at all, even though it scored well with those who knew it. Others where the scores deviated in this regard were “It’s A Man’s Man’s World” by James Brown, “What’s New Pussycat?” by Tom Jones, “He’s Got the Whole World in His Hand” by Laurie London, and “Coconut” by (Harry) Nilsson. There were a few songs that were liked more by those who did not know them, for example, “Susan” by The Buckinghams is liked a bit more by those who’ve never heard it.
Now let’s look again at the disliked songs. “Yummy Yummy Yummy” by Ohio Express had the biggest difference when it came to those familiar with the song vs those who did not know it. Forty percent of those who’d never heard it before hated it, compared to 13% of those who knew it.
Which songs tested the best?
Top ten favorites among those familiar with the song:
- “I Got You (I Feel) Good” by James Brown
- “When a Man Loves a Woman” by Percy Sledge
- “Lollipop” by The Chordettes
- “Lollipop” by Ronald and Ruby
- “Monster Mash” by Bobby ‘Boris’ Pickett (really!)
- “Mr. Sandman” by The Chordettes
- “In The Summertime” by Mungo Jerry
- “Splish Splash” by Bobby Darin
- “Up Around the Bend” by Creedence Clearwater Revival
- “Banana Boat (Day-O)” by Harry Belafonte
Top ten favorites among those unfamiliar with the song:
- “I Got You (I Feel Good) by James Brown
- “Summer In the City” by the Lovin’ Spoonful
- “Lollipop” by Ronald and Ruby
- “Sweet Soul Music” by Arthur Conley
- “Living For the City” by Stevie Wonder
- “Jenny Take a Ride!” by Mitch Ryder and the Detroit Wheels
- “Green Onions” by Booker T and the MG’s
- “Splish Splash” by Bobby Darin
- “Changes” by David Bowie
- “Funky Street” by Arthur Conley
Least liked songs by those familiar with them:
- “The Shifting Whispering Sands” by Billy Vaughan and his Orchestra
- “Glendora” by Perry Como
- “Je T’aime Moi Non Plus” by Serge Gainsbourg and Jane Birkin
- “Wooden Heart” by Joe Dowell
- “Tower of Strength” by Gene McDaniels
- “Sailor (Your Home Is the Sea)” by Lolita
- “Stay Awhile” by The Bells
- “Dungaree Doll” by Eddie Fisher
- “Black Denim Trousers and Motorcycle Boots” by Cheers
- “Tie Me Kangaroo Down Sport” by Rolf Harris
Least liked by those who were unfamiliar:
- “The Shifting Whispering Sands” by Billy Vaughn and his Orchestra
- “Glendora” by Perry Como
- “Kiss An Angel Good Morning” by Charley Pride
- “Footsteps” by Steve Lawrence
- “Wooden Heart” by Joe Dowell
- “You Were Made for Me” by Freddie and the Dreamers
- “Tie Me Kangaroo Down Sport” by Rolf Harris
- “Yummy Yummy Yummy” by Ohio Express
- “Sailor (Your Home Is the Sea)” by Lolita
- “Tower Of Strength” by Gene McDaniels
The fact that you see a lot of the same songs on these lists highlights the fact that familiarity is nearly irrelevant. New music starts out being unfamiliar. Some have even tested new songs that nobody knows yet to see which have hit potential. If you’re programming a gold-based format, why not see if any songs your audience doesn’t know have hit potential? I would not add a song if those who are familiar with it don’t like it, regardless of how those who don’t know it feel. But I would factor in the feelings of the unfamiliar when determining the overall strength of a song in my active library.
My conclusion is that you should rely on the scores of your core audience as reflected in the Familiar responses. However, if the Unfamiliar respondents dislike the song intensely, you should consider playing it less often. Conversely, if the Unfamiliar respondents love the song, you might consider giving it more weight when it comes to deciding how to rotate the song in the future.
The “weighting” that’s built into the analysis software identified the ten best songs regardless of familiarity, but it didn’t quite do as well with those that were disliked. It did, however, identify that Billy Vaughn tune as the biggest clunker in the test.
I copy and paste like and dislike scores into custom fields in MusicMaster, then use those fields in conjunction with Optimum Goal Scheduling to favor songs that have higher like scores, or lower dislike scores, when scheduling equally best testing songs. Optimum Goal Scheduling is a form of artificial intelligence because MusicMaster studies your database and makes decisions based on what it learns, without you having to set up any parameters or make any adjustments as your library evolves over time. If you’d like to do the same, get in touch with your support agent.
If you’re interested in hearing more results from this test, feel free to reach out to me. I need to thank Deena Hollander at Advantage Music Research for helping me put this unusual test together. If you want to try something similar, she knows how to make it work.