This is Part 2 of a multi-part blog posting analyzing the way that Reverbnation, the popular artist/dj/rapper/band ranking website, ranks artists.
As we concluded in our last post, while there is no perfect band/artist rating system, it seemed to us that Reverbnation was genuinely trying to put together a fair, objective rankings system.
Since posting that blog, however, some aspect of the Reverbnation rankings approach bothered us, but we couldn’t quite figure out what. Then it hit us. But we’re not going to relay our conclusion – yet. Instead, we wanted to see if we could demonstrate what we think the flaw is (with hopes it can be addressed). That being said, we respect Reverbnation, which – and this is no small statement – we genuinely see as a key player in the future evolution of music worldwide.
What’s the best way to “rank” (and therefore promote) talented artists?
We decided to put together a group of people who are hard-core fans of Electronica – a “focus group,” in marketing language. Our idea was to play tracks by the Electronic Music artists ranked in the top 50 or top 100 on the Reverbnation Charts, but without telling the participants which artist made which song, or the current ranking of each artist, so as not to bias them towards choosing an artist just because that artist happened to be ranked higher in the charts (i.e., to prevent the ‘popularity bias’).
Then we decided we’d also test another theory. Instead of playing music by the Top 100 Electronica Artists, we would instead play 3 tracks by each of the Top 50 artists and mix that with tracks produced by artists ranked from 1001 to 1050 – again, without telling our focus group which artist made what track, etc.
Why do this? We figured there was obviously a connection between an artist’s/producer’s ranking on the Reverbnation Electronica Charts and quality of that artist’s music (certainly more so than what is reflected on more mainstream charts, like Billboard). However, we thought the current rankings might unevenly reward artists who regularly perform in public and/or were better at the administrative aspects of the music business, benefiting them over artists who also produced great work, but were unpolished (e.g., they didn’t know how to master their tracks) and/or simply did not perform publicly and, therefore, had a hard time accumulating fans – perhaps because they were studio producers rather than performers. Just because an artist was out in public promoting itself, we wondered, should that necessarily elevate them in the rankings over an introverted producer? Consider this: exactly 2 years ago, Lady Gaga was ranked #728 in the Electronica category on Reverbnation! She now purportedly has over 10 millions fans on Facebook. Did she just suddenly improve?
The answer, of course, is that Lady Gaga was just as good in October 2009 as she is currently – but 10 million people didn’t know about her – and this got reflected in her Reverbnation ranking. Aside from the obvious marketing benefits, there are, of course, advantages to performing publicly. Having been DJs ourselves, we are aware of the pressure of keeping a crowd moving for hours. Fun? Yes. Easy? No way, and we respect those who do it. But we should also remember that there are advantages to another approach: cloistering oneself, not that the two approaches are mutually exclusive.
Secluding oneself for a time in order to record music has the advantage of giving the artist time to reflect inwardly and perfect their craft. Frankfurt Germany’s electronic genius, Oliver Lieb (a/k/a/ Spicelab, LSG, etc.) is, we hear, one such example. Many of us remember our high school days being surprised to learn that the quiet girl no one knew graduated at the top of the class. It also gives an artist space to develop their own unique sound. There are times when it’s good to hear other artists. The trick is, preserving what it is that makes you, well, you, and not parroting other musicians. It’s our philosophy that what makes an artist great is his/her ability to express their uniqueness through their music. Unlike the conventional thinking of big business, people aren’t looking for a product/service – song, anything, produced for them. Instead, the power of a creation is when it represents an authentic expression of its creator – each of whom is unique. THAT is what we all want to connect with (or, in business parlance, purchase) because, to be candid, we’re all connected. Literally.
And so, after some initial failures, we managed to assemble a focus group. Yes, we actually did.
The Monster Loop Electronic Music “Focus Group”
What were we after in our group? First, because the Electronica category includes many different styles (e.g., Dubstep, Psytrance, House, Progressive, Ambient, etc.), we wanted people who liked many styles of Electronica. Second, we wanted men & women. Third, we wanted people who represented different age groups, but who all liked & actively listened to Electronica. And fourth, we wanted people who were into Electronic Music more than they were into the Electronic Music scene.
The Focus Group – it actually worked
We ended up finding 5 people, each of whom we paid a modest sum, to listen to music. And it wasn’t that hard. Little did they know at the time, they were about to hear A LOT of music. To respect their privacy, we will not reveal their identities. They did, however, give us permission to relay the following:
Participant #1 (“Katja”) is a 24 year old female from Germany studying mathematics at a Chicago-area university. She listens to Electronic Music around 5 hours a week and is “heavy” into the European Electronica scene.
Participant #2 (“Mike”) is a 45 year old man from Chicago who manages a record store. By the end of the weekend, he acquired a nickname – “Rain Man,” (he was cool with this) due to his ability to recall obscure facts about almost any techno artist from the past 30 years. He claims he has over 5,000 vinyl records, most of which are Electronica, though he pointed out that term “is of recent lineage.”
Participant #3 (“Jose”) is a 28 year old man from Hammond, Indiana. He’s a graphic designer in Chicago and said he likes every style of Electronica. His favorite style is Dubstep.
Participant #4 (“Jason”) is a 29 year old unemployed man from Chicago who likes ambient, downbeat, and experimental techno. He thinks Aphex Twin (Richard James) is “the best ever.”
Participant #5 (“Seth”) is a 40 year old man who has been listening to electronic music since 1988. His favorite styles are “Acid House, Progressive, and Psytrance” and “almost any style if it has a good beat.” Like ‘Rain Man,’ he had an impressive knowledge of the genre.
Weaknesses of our Group
The youngest participant in our focus group is 24 years old. There is obviously a huge contingent of fans under age 24 – and we’d love to get their viewpoint. This particular focus group, however, had alcohol. Enough said. Most of the participants are from Chicago; there is only one non-American (a German) and 4 of the 5 participants are men. A more subjective “weakness” (strength?) of the group is that there appeared to be a higher than average IQ level – just a guess and we’re not sure of the impact. Finally, 5 is hardly enough people to draw too many conclusions. From our own experience, however, we have continually been surprised to find that people who have listened to and commented on our music have generally liked the same tracks.
What the Electronic Music Focus Group listened to & liked
Results will be discussed in the following two blogs. For now, we wanted to relay what we had our little group do (bwaa haa ha). After meeting for dinner on Friday evening to make sure no one was Jack the Ripper, each person agreed to show up the following morning at my home outside Chicago (note: Monster Loop is listed as Atlanta on the charts because William lives there and I used to. Note also the photo on the left which was taken while setting up). They were then asked to listen to samples of 260 Electronic Music tracks. Yes, 260. And, yes, it took all day. Probably poor planning on our part. Beer was eventually brought in, and food. People took breaks, etc.
After hearing a track, each person was asked to score it between 1 and “10,” 10 being best. We did not provide any guidance other than that 10 was whatever they liked best, however they defined “best,” whether or not it had the slickest production, etc.
And they did. They actually did. Many funny things happened that day. That is the subject of my next post.
G’night!
This is Part 1 of a new 
multi-part discussion analyzing the way that Reverbnation, the popular artist/dj/rapper/band ranking website, ranks artists.
Background for those unfamiliar with Reverbnation
Reverbnation is perhaps the leading online music-marketing platform. Used by over 1.7 million DJs/artists/bands, managers, and record labels, to increase their presence (and sales) on the internet, Reverbnation provides valuable marketing tools to music professionals (e.g., promotion, fan relationship & measurement, digital distribution, marketing, and concert booking ). A significant part of Reverbnation.com, though, is the sites rankings/charts, which are organized by musical category (e.g., country, rock, pop, rap, heavy metal, electronica, etc.) These can be viewed locally (a 25-mile radius), nationally, and globally.
The higher an artist is ranked, the more opportunities generally come the artists’ way because (1) Reverbnation is now working more closely with music industry professionals and insiders and (2) Reverbnation’s charts are becoming an efficient way for record company professionals to sort through the large group of emerging artists (a subject of a future blog entry). The rankings are viewed by many as a “screen” of sorts. It is, therefore, in an artists’ interest to rise in the Reverbnation rankings. The following is an image of “Infected Mushroom,” from L.A., currently #1 in the US, #4 Globally, in the Electronica category.
So, how are the rankings decided?
According to its website, the Reverbnation charts are based on Band Equity Score, or “BES.” BES is designed to measure popularity based on 4 factors: Reach, Influence, Access, and Recency. Reverbnation points out, however, that BES ”encompasses hundreds of things on the Reverbnation site,” including statistics not made available to the artists. Examples given are promoter plays, percentage of emails opened by your fans, and shared widgets.
How do these factors play out?
Those familiar with Reverbnation are already aware that each artist has its own webpage, similar in many ways to a Facebook page. On this page, artists have the option of sharing with their fans various information including the number of the artists (1) fans, (2) song plays (songs may be streamed for free on each artists’ page), (3) visits, and (4) widget hits (widgets are mini mp3 players fans can place on their own websites - the point being, widgets are a way for people to play an artists’ music from a different location).
But which factor is most important? And what are the relative weight of these, and other, factors? Reverbnation is secretive about the mathematical formula used to determine BET, which in turn determines an artists ranking. Let’s see what we can deduce about BET.
The value of factors can be deceptive
The following illustrates why many artists are confused by the method used to rank artists. At the moment, the top 10 Electronica artists on the Global Charts are as follows:
- Inna (Romania)
- Paul van Dyk (Germany)
- ATB (Germany)
- Infected Mushroom (US – L.A.)
- Bassnectar (US – San Francisco)
- Datsik (Canada)
- Ibrahimcelik (Turkey)
- Pretty Lights (US – Denver)
- Excision (Canada)
- Alderec King (Spain)
The artists, #4 “Infected Mushroom ” and #8 “Pretty Lights” chose not to make their fan data public, so for purposes of this example, we’re using #11 Frankie O. Solovely and #12 Umek, to have 10 artists.
How important are the number of fans? #1 Inna clearly leads this category (has by far the largest number of fans). So far that seems a key metric. However, #12 “Umek” has over 350,000 fans, while #10 “Alderec King” has only around 60,000 – almost 300,000 less than Umek, yet is ranked higher. Is this because Alderec pulled in a large number of fans in the past 2 weeks? Or is the number of fans not a key factor? We’ll come back to this.
How important are visits to an artists Reverbnation page? ”Excision,” rated #9 on the Global Charts, leads this category with around 215,000 visits, but Inna has less than half as many visits, yet Inna is #1. Again, this conceivably could be due to recency, or perhaps artist visits is not a key factor in one’s ranking.
How important are song plays? In terms of the number of times music on an artists’ Reverbnation page has been played (streamed), the clear leaders in our example are “Ibrahimcelik” and “Datsik.” Both have over 2 million plays; in contrast, Inna has only around 150,000 plays - a sizable difference – yet Inna is #1. Does this suggest song plays are given little weight or can this be explained away if, for example, song plays for Ibrahimcelik and Datsik occurred several months back? Is it possible Reverbation is concerned that heavily weighting song plays leads to rankings distorted by artists continuously playing their own music and/or entering into pacts with other artists to play each other’s music non-stop and therefore artificially increase their BES?
And there’s another angle here. These statistics may suggest that the average fan of Ibrahimcelik (112,019 fans), plays 19 of his tracks on average, while an Inna fan (3.6 million fans) does not, on average, play an Inna song even once (0.5 plays per fan - total of 167,497 plays). Huh? Something seems odd here. Are Inna’s fans that much less into playing her music than Ibrahimcelik’s fans? Does Inna just have a more organized fan-collection system? If Ibrahimcelik’s fans are playing each of his songs that many more times, should Reverbnation assist in promoting a lesser known artist (at least lesser known to us, but maybe that’s not saying much). Assuming his fans are streaming Ibrahimcelik’s music that often, should he only be #7 while Inna is #1, that is, what constitutes a genuine “fan”? It appears Reverbnation partly attempts to address this by purportedly giving more “credit” to fans that are registered on Reverbnation.
How important are widget hits/impressions? The leader here is Bassnectar, with over 300,000. Paul Van Dyk has 14,000, but Van Dyk is ranked #2 and Bassnectar is ranked #5. So widgets are important, but the rankings don’t appear to be driven by this factor.
Post more music? Another interesting factor is whether an artist is penalized/rewarded for posting more music. “Frankie O. Solovely” (#11) has over 125 tracks posted, while “Alderec King” (#10) has just 2. Solovely has around 2,500 total song plays, while King has 8,182. In other words, a track composed by King averages around 4,000 plays per track, while a track by Solovely averages 21 plays per track – quite a difference - yet the two artists are neck-in-neck in the ratings. And Bassnectar (ahead of both at #5) averages around 1,000 plays per track, so plays-per-track does not appear to be a key factor. Should it be? And should an artist be rewarded for posting more material? What incentive do artists already selling their music have to post loads of their music on Reverbnation to be streamed for free?
We ran some back-of-the-envelope calculations and it appears there is a penalty for having more than 3 songs, increasing in severity up to around 10-12 tracks, and then capping (i.e., once you hit 12 tracks, there’s no additional penalty for posting more than 12 tracks). We are not entirely sure of our conclusion here, but based on these 10 artists, the data supports this preliminary conclusion. In other words, it appears that an artists number of listens/widget hits, etc. is divided by the number of tracks (though using a more complex formula – at least it appears this is the case).
While it appears confusing, a factor does appear to stand out
In Reverbnation’s Questions & Answers section, it addresses this confusion by reminding artists of two key components: (1) recency and (2) there are dozens – perhaps over a hundred – factors they consider.
So one can never truly decipher Reverbnation’s formula, right? That’s probably wrong, and the answer is pretty basic.
The ranking of the 10 Electronica artists shows a very strong correlation between one’s BES and the number of fans. For math nerds reading this, we calculate a correlation of .8303. Based on the example featured in this blog – and we admit 10 artists is a VERY small data set and not enough to draw broad conclusions - the number of one’s fans, for example, is 2-3 times more important than the number of fan visits to one’s reverbnation page.
There’s a logical reason for this (we suppose). Put yourself in Reverbnation’s shoes. Reverb wants to attract top talent – among other reasons, doing so will further increase its credibility in the music industry. 
Assume Lady Gaga was considering Reverbnation (she may already be on there, we have not checked). If the key criteria were the number of visits or plays, it would seem that Lady Gaga would have a large disadvantage because visits = visits on Reverbnation and plays = plays on Reverbnation music players. Lady Gaga purportedly has over 10 million fans on her Facebook page. Reverbnation allows artists to carry fans over. This way Lady Gaga could, if she chose to register on Reverbnation, instantly be ranked #1 (at least, I assume that would be enough to carry the day!). Reverbnation surely is aware that a hot artist would shy away from having to prove themselves among a large group of unknown but very talented, emerging artists.
A problem with this is, it is difficult to authenticate a fan list. A band can simply provide a huge excel file with fan email addresses and get credit for this list. To reduce the risk of distortion, Reverbnation “quarantines” such contacts for 3 days, in which time the new fan has a chance to accept or reject their status as a fan for Reverbnation purposes. If the person does nothing, they are counted.
Soooo, what’s the bottom line? We’re not sure (yet), sorry. We’re still analyzing it with the help of an autistic friend. Our view is – and we’re not going out on a limb here – there is no perfect way to rank artists, and Reverbnation appears to be making a genuine attempt to come up with a fair system. We assume they keep it secret so that artists won’t discover and take advantage of loop holes in the scoring. The idea is to come up with a way to discover talented artists – even if the artist doesn’t happen to have friends at a large record label. Time will tell.
24
Oct
Yes, yes we know – it’s been a ridiculous amount of time since we’ve posted. Shame on us! In fact, it’s been so long, we couldn’t even remember our login information. After a few hours digging around, however, we managed to scrounge it up in our search through a back catalogue of a thousand emails – mostly useless spam.
Why were we gone so long? What artists will we discuss next? All of these answers - and more - coming soon. We’re dusting off the cobwebs, so to speak, and will post in the next 24 hours or our name isn’t Ignatius T. Kettlewoop. Well, actually it isn’t, but yes we do plan to post.
In the meantime, for your listening enjoying and general amusement, we recommend you check out Swamp Princess & T-Diggity, a 6 and 8 year old who love electronic music and are currently ranked #26 on the global charts (#18 U.S.) ”Children’s” category. We, Monster Loop, perform the music so perhaps we have a slight conflict of interest!
