Tags Clouds and Diversity Rating

RSS
Dela

16 nov 2006, 07:52

I added the tag cloud option long time ago to Last.fm Extra Stats, but recently while I was playing with it and browsing through some groups specialized in people with broad tastes in music,I decided to design a formula for being able to measure the level of diversity in someone's taste. I realized that the tag clouds can be used for that.

In this journal I will explain how the tag clouds and the diversity rating are generated by the Last.fm Extra Stats.

The tags cloud is generated searching the tags ratings for each artist in your chart and multipling it for the number of tracks listened. for example if someone has the following 3 top artist and wants to make his or her tag cloud them (obviously It's better to use 50 artists)

from first to last 420
Carcass 230
t.A.T.u. 150


Using the data from the webservices (I'm only going to use the top 3 tags for this example), can be obtained a rating for each tag.
http://ws.audioscrobbler.com/1.0/artist/from+first+to+last/toptags.xml
http://ws.audioscrobbler.com/1.0/artist/Carcass/toptags.xml
http://ws.audioscrobbler.com/1.0/artist/t.A.T.u./toptags.xml

death metal = 100*230 = 23 000
grindcore = 24*230 = 5 520
Melodic Death Metal = 5*230 = 1 150
screamo = 100*430 =43 000
emo = 63*430 = 27 090
hardcore = 21*430 = 9 030
pop = 100*150 = 15 000
russian = 40*150 = 6 000
female vocalists = 19*150 = 2850

organizing them in a chart, the tag cloud is obtained.

screamo 43 000
emo 27 090
death metal 23 000
pop 15 000
hardcore 9 030
russian 6 000
grindcore 5 520
female vocalists 2 850
Melodic Death Metal 1 150

For the diversity rating I realized that the problem is similar to one that I have already solved. I applied the AEP formula but instead of using it in the top 50 tags, I used it in the top20 tags.

here are some examples, I think it works reasonable well.

first a profile with the following artists:

The Streets, Björk, Lily Allen, Feist, Amy Winehouse, Basement Jaxx, Peaches, Cassie, Juliette & The Licks, Mariah Carey, Jill Scott, Thomas Newman, Sia, Sade, John Legend, Zero 7, Prince, The Smashing Pumpkins, Arctic Monkeys, Kings of Convenience, Red Hot Chili Peppers, Regina Spektor, Missy Elliott, Common, Air, Eva Cassidy, Death Cab for Cutie, Boards of Canada, Damien Rice, Audio Bullys, Robin Thicke, Massive Attack, Esthero, The Beatles, T.I., Nelly Furtado, Sufjan Stevens, Portishead, Rihanna, Madonna, Radiohead, Panic! at the Disco, Lisa Ekdahl, Jamiroquai, Imogen Heap, John Mayer, The Kooks, Maria Mena, The Pipettes, Ani DiFranco,


the tag cloud



the diversity rating = 2



now a profile with the following artists:

Skillet, Three Days Grace, Ill Niño, Breaking Benjamin, A Perfect Circle, Senses Fail, Evans Blue, System of a Down, My Chemical Romance, Staind, Linkin Park, P.O.D., 30 Seconds to Mars, Evanescence, 10 Years, Papa Roach, Tool, Rise Against, Nine Inch Nails, Art of Dying, Lacuna Coil, Incubus, Seether, Sugarcult, Anti-Flag, Ko?n, Cold, Deftones, Thousand Foot Krutch, Silverstein, Green Day, Default, Weezer, Marilyn Manson, Mudvayne, Edgewater, Switchfoot, Bowling for Soup, 3 Doors Down, Billy Talent, Yellowcard, Hinder, Strike Anywhere, Hurt, Smile Empty Soul, Sum 41, Disturbed, Everything At Once, Run Kid Run, Crossfade,


the tag cloud



the diversity rating = -2


But the procedure is not perfect, it has some issues:

1.The same genre can have associed 2 or more different tags, for example : post rock, postrock and post-rock. this problem can produce a higher diversity rating , this can be partially solved using the list the join tags option of last.fm extra stats.

2.There are some tags that are not genres, for example French, this problem may produce a higher diversity rating too,and can be partially solved using the ignore tags option of last.fm extra stats.

3. Some musical genres are not well represented in tags, the tags aren't fully reliable. this inherent from the tag system and we can't do anything to solve it.


but something is better than nothing :)


you can find your diversity rating and your tag cloud using Last.fm Extra Stats

Kommentarer

  • iamdanielj

    Man this software looks cool.....downloading now. Thank you.

    16 nov 2006, 12:20
  • tobymacfan

    Wow that is real cool how it takes all the artists and puts all the genres with them. I will have to check out that Last FM Extras some time soon.

    16 nov 2006, 19:58
  • ___natalie

    hey, that's really cool. [=

    22 nov 2006, 21:08
  • mlansman

    Is the Diversity rating between -25 and 5 like the AEP?

    18 feb 2007, 22:11
  • C26000

    yes, because it's the same formula but applied to the top 20 tags and using the values obtained from the tag cloud instead of the tracks played that are used in the aep. Obviously is harder to listen 20 different genres than 50 different artists same amount of times, that's why the normal values are around 0. and a very high value is 3.

    18 feb 2007, 23:06
  • Lizarking

    Great utility!!! thx

    10 apr 2007, 20:52
  • gifool

    I prefer this method to the 'similar artists' method, thanks for the great app. [quote]The same genre can have associed 2 or more different tags, for example : post rock, postrock and post-rock. this problem can produce a higher diversity rating , this can be partially solved using the list the join tags option of last.fm extra stats.[/quote] [quote]Obviously is harder to listen 20 different genres than 50 different artists same amount of times, that's why the normal values are around 0. and a very high value is 3.[/quote] I can't find these options... gifool Diversity rating is 2.03 =) I get a lower rating in the similar artists thing because I use Last.fmn similar artists to discover new bands, and that is reflected in my charts, but sometimes the similar artists aren't actualy similar.

    21 maj 2007, 11:23
  • C26000

    In the 3.0 version these options are located in the lateral tag cloud panel, I have already moved them to the general preferences page in the 3.1.

    21 maj 2007, 17:50
  • milehi

    The tool is great, I love it. With regards to diversity and tag clouds, I see a problem in that some tags are relatively broad. For instance, Indie. Indie applies to RJD2, Built to Spill, Calexico, and Hood. At least in my mind these bands are 4 distinct styles of music (underground hip hop, Indie Rock, Alt-country, and post-rock).

    26 jun 2007, 23:06
  • GrandTerminus

    So maybe I am missing it somewhere....but is there a clear concise definition of Diversity Rating? Anywhere? What are the max values attainable. Is it 1 to 10 or -10 to 10 etc etc etc. Where 10 means Uber diverse and 0 is average and -10 is just daft?

    1 mar 2008, 23:50
  • C26000

    the diversity rating is just the aep formula applied to the genres chart, so it's between -20 and 5 just like the aep, but obviously is harder to listen to many genres than many bands so a high diversity rating is around 2

    2 mar 2008, 00:26
  • The_10th_Plague

    I have a mathmatical question: when I set all parameters (number of tags, number of artists) to default, I get an diversity rating of 2.7. As you see this number has 1 decimal place (one digit after the dot). When I change the parameters the rating naturally changes, but as a matter of fact all those ratings have 2 decimal places (two digits after the dot; i.e x.xx). It seems a bit inconsistent to have 1 decimal in one calculation, and 2 decimals in the other calculation. May this be due to a fact of the last digit being a 0? (as in: x.x0)

    1 aug 2008, 20:09
  • C26000

    yeah maybe it's because that, I don't remember adding any rule to round the rating when using different parameters

    20 aug 2008, 02:47
  • josiesomethin

    Great tool! Love the charts etc.

    4 jan 2009, 21:40
  • t_hisa

    3.12 :3

    29 mar 2010, 21:28
  • Pogo-The-Clown

    0.62

    24 aug 2010, 12:40
  • MarquisLek

    2.62 cool :D

    7 apr 2011, 04:23
Se alla 17 kommentarer
Lämna en kommentar. Logga in på Last.fm eller registrera dig (det är gratis).