How do you organize your music files?

Seltzer · 04-14-2010, 11:05 PM

Quote:

Originally Posted by tore

How do you organize mixed artists compilations like certain soundtracks or albums like "golden oldies"?

For each track in a mixed artist compilation or VA soundtrack I set the artist as whoever the artist is and album artist as VA. For the compilations, I check the Part of a Compilation tag and set the genre to Compilation. For soundtracks, I simply set the genre to Soundtrack. This seems to be iPod friendly.

As for how I physically store compilations/soundtracks on my file system, I do it as below. Most of my soundtracks are in Soundtrack Albums but if I have heaps of soundtracks by one artist (i.e. Ennio Morricone) then I have a folder for them in Soundtrack Artists.

Compilations & Soundtracks
|--Compilations
|--Soundtrack Albums
|--Soundtrack Artists

For the MB compilations, I set the album art as the MB logo.

Quote:

Originally Posted by noise

that doesn't sound pedantic at all. in fact, it sounds absurdly inefficient. you'd be much better off using tags for things like genre and year, then using a strong library-based media player like foobar for organization.

also, embedding artwork is redundant, and it only serves to increase the size of your collection. i assume you do it for the sake of your ipod - again, if you use foobar, then you can embed artwork on the fly while syncing.

or you can go balls out and flash your ipod with rockbox firmware. not only will it let you use folder.jpg for album art, but you can use your own directory structure for organization, and can do away with all 'syncing' apps. just copy files in disc mode.

I organise according to tagging (genre tags + Foobar) as well as folder structure. Personally, I prefer the first approach but I also maintain a folder structure for the benefit of other people downloading music from me (since they obviously can't make use of genre tags when downloading music off our flat server). And yeah, for the structure by tags approach to work, it is necessary to use a strong library-based media player like Foobar as most others including WMP are laughably crippled by large collections.

I embed artwork in every song for the sake of my iPod but also in case people want to download individual songs from me. If each track has its own album art embedded then it doesn't matter if it is taken out of context. I ensure that my album art is < 400k (and usually 20-50k) so that the storage requirement is negligible (unless I'm dealing with grindcore albums haha).

I've been meaning to look into flashing my iPod but I think I'll wait until the warranty expires. I've figured out most of the vagaries of iTunes/iPods so it won't be too much of a headache until then. One of my main gripes with iTunes was its unanchored shift-select but they seem to have fixed that.

Quote:

Originally Posted by tore

I also include album type in filename (f.ex "1972 - Close to the Edge [Studio album]") and all that together made the transition to foobar and retagging albums a breeze, for example by automatically filling %album type% with the information from that directory. Not sure what the benefit is right now, but if somehow my tags get screwed, it'll be of use.

I forgot to mention that I do that too - I annotate album names with (Compilation), (EP), (Single), (Bootleg), (Live Album) etc...

noise · 04-14-2010, 11:22 PM

ipod flashing is very safe. all the original firmware remains intact you can boot into it at any time. as far as i understand, the only thing actually flashed is a sort of boot loader that directs things towards the custom firmware. and of course it's all completely reversible

Guybrush · 04-15-2010, 02:12 AM

Quote:

Originally Posted by Seltzer

For each track in a mixed artist compilation or VA soundtrack I set the artist as whoever the artist is and album artist as VA. For the compilations, I check the Part of a Compilation tag and set the genre to Compilation. For soundtracks, I simply set the genre to Soundtrack. This seems to be iPod friendly.

As for how I physically store compilations/soundtracks on my file system, I do it as below. Most of my soundtracks are in Soundtrack Albums but if I have heaps of soundtracks by one artist (i.e. Ennio Morricone) then I have a folder for them in Soundtrack Artists.

Compilations & Soundtracks
|--Compilations
|--Soundtrack Albums
|--Soundtrack Artists

For the MB compilations, I set the album art as the MB logo.

Sounds good, but I'm not so sure about using the genre tag for information like soundtrack and compilations! If you use foobar, why not just store that information in a new tag (f.ex "album type"?).

I didn't like sorting by genres back in the days, but now I've got a python script part written by myself which I can run from inside foobar that fetches genre and style tags from the discogs database for releases. It's fast too! It's nice because discogs have a reasonably strict set of allowed genres and styles and any album can have more than one in each field which is fine since foobar has no problems with multiple values.

Style Guide - The Unofficial Discogs Wiki
Example album on discogs, Over-nite sensation (note genre and style information)

If I didn't use this, I'd probably have over a thousand different genres still on my computer ..

noise · 04-15-2010, 03:26 AM

tore, are you willing to share that script? it sounds incredibly useful!

at one point i tried using various plugins to grab genre/style info from AllMusic, but never got very good results...

i'm not sure how much i would like using someone else's categorizations, but i can always back up tags and restore them if things don't work out

Guybrush · 04-15-2010, 08:03 AM

Quote:

Originally Posted by noise

tore, are you willing to share that script? it sounds incredibly useful!

at one point i tried using various plugins to grab genre/style info from AllMusic, but never got very good results...

i'm not sure how much i would like using someone else's categorizations, but i can always back up tags and restore them if things don't work out

Certainly

It's a bit of a workaround - it's actually based on a plugin which allows custom python scripts to fetch lyrics .. But since they are custom scripts, it's not hard to rewrite them so that they fetch other things. My own script is a merge of two other scripts, one to fetch discogs genres and one to fetch styles .. First, I just put those two together. Then there was a problem with queries, you're only allowed to access discogs 5000 times per 24 hours that way, so you could only tag 5000 files. I came up with a workaround for that problem by using text files as temporary storage for information. Basically, when the script starts working on a file, it writes that song's album title into a text file and fetches discogs info. When it moves on to the next file, it compares the album title to the information written to the text file by the previous song. If it's the same, it just tags it with the same info. If the album is different, it fetches new info and rewrites the text files. It also uses a text file for temporary storage of the information that goes into the genre and style tags, but that's the jist of it. Instead of tagging 5000 songs, you can tag 5000 albums ..

It doesn't work for all files, but at least for more than 3/4ths of my own collection. Aside from artists and albums having wrong titles, a possible problem is when there are more bands with the same name. Discogs register them as Band (1), Band (2) and so on (f.ex Arsenal (1), Arsenal (2) etc.) and since we don't normally have numbers in parantheses after band names, those may not get tagged.

Anyways, to make it work, there are a few things you're gonna need:

foo_utils

And these two, both found at the same location :

The latter of those two contains some files and folders (ex. python25.dll) that all need to be located in your base foobar folder. The files and folders should look something like this :

Code:

C:\Program Files\foobar2000>dir p*.dll /s/b & dir *grab* /s/b & dir pygrabber\* /s/b & ver
C:\Program Files\foobar2000\python25.dll
C:\Program Files\foobar2000\lyrics_grabber_provider.cfg
C:\Program Files\foobar2000\pygrabber
C:\Program Files\foobar2000\components\foo_grabber_python.dll
C:\Program Files\foobar2000\components\foo_lyricsgrabber.dll
C:\Program Files\foobar2000\pygrabber\libs
C:\Program Files\foobar2000\pygrabber\newscripts
C:\Program Files\foobar2000\pygrabber\scripts
C:\Program Files\foobar2000\pygrabber\system
C:\Program Files\foobar2000\pygrabber\newscripts\newscripts.rar
C:\Program Files\foobar2000\pygrabber\scripts\AZLyrics.py
C:\Program Files\foobar2000\pygrabber\scripts\DarkLyrics.py
C:\Program Files\foobar2000\pygrabber\scripts\Discogs_GetGenre.py
C:\Program Files\foobar2000\pygrabber\scripts\Discogs_GetStyle.py
C:\Program Files\foobar2000\pygrabber\scripts\LastFm_Bio.py
C:\Program Files\foobar2000\pygrabber\scripts\LastFm_TopTag.py
C:\Program Files\foobar2000\pygrabber\scripts\LastFm_TrackTags.py
C:\Program Files\foobar2000\pygrabber\scripts\LeosLyrics.py
C:\Program Files\foobar2000\pygrabber\scripts\LyrDB.py
C:\Program Files\foobar2000\pygrabber\scripts\Lyricist(LRC).py
C:\Program Files\foobar2000\pygrabber\scripts\TTPlayer(LRC).py
C:\Program Files\foobar2000\pygrabber\system\autoexec.py
C:\Program Files\foobar2000\pygrabber\system\BeautifulSoup.py
C:\Program Files\foobar2000\pygrabber\system\BeautifulSoup.pyc
C:\Program Files\foobar2000\pygrabber\system\Html2Text.py
C:\Program Files\foobar2000\pygrabber\system\Html2Text.pyc
C:\Program Files\foobar2000\pygrabber\system\LevenshteinDistance.py
C:\Program Files\foobar2000\pygrabber\system\LevenshteinDistance.pyc
C:\Program Files\foobar2000\pygrabber\system\Lucky.py
C:\Program Files\foobar2000\pygrabber\system\Lucky.pyc
C:\Program Files\foobar2000\pygrabber\system\pyexpat.pyd
C:\Program Files\foobar2000\pygrabber\system\Python25.zip
C:\Program Files\foobar2000\pygrabber\system\unicodedata.pyd
C:\Program Files\foobar2000\pygrabber\system\_socket.pyd

Sorry if I'm making it look tricky. If you have foo_utils, then all you really should have to do is unpack those two archives to your foobar folder, but I managed to misplace these files on my first try, so I figured I'd just make it clear.

Your foobar folder will now contain a folder called "/pygrabber/scripts". This is where you should put the script I've written. The simplest way to do it I guess is just paste the script here for you to copy and paste. Then I can show you the changes you should make.

So, copy the following script and save it in a text file in the scripts folder. Rename it to something like "tores_discogs_script.py" and then you may want to edit it and have a closer look at the red parts :

Code:

import urllib, urllib2, gzip, cStringIO
import xml.etree.ElementTree
from xml.dom import minidom
from encodings import utf_8
from grabber import LyricProviderBase

class Discogs_GetGenre(LyricProviderBase):
    def GetName(self):
        return 'Discogs GenStyles'
    
    def GetVersion(self):
        return '0.1'

    def GetURL(self):
        return 'http://www.discogs.com'

    def Query(self, handles, status, abort):
        result = []
        api_key = '783001745d'

        for handle in handles:
            status.Advance()
            
            if abort.Aborting():
                return result
            
            artist = handle.Format("[%artist%]")
            album = handle.Format("[%album%]")
            album_file = open("C:\Python26\!album.txt", "r")
            test = album_file.read()
            album_file.close()
            
            try:
                if album == test:
                    text_file = open("C:\Python26\!tag3.txt", "r")
                    writeout = text_file.read()
                    text_file.close()
                    result.append(writeout)
                else:
                    URL_s = 'http://www.discogs.com/search?type=all&q=' + artist.lower().replace(' ','+') + '+' + album.lower().replace(' ','+') + '&f=xml&api_key=' + api_key
                    request = urllib2.Request(URL_s)
                    request.add_header('Accept-Encoding', 'gzip')
                    response = urllib2.urlopen(request)
                    data = response.read()
                    unzipped_data = gzip.GzipFile(fileobj = cStringIO.StringIO(data)).read()
                    res = minidom.parseString(unzipped_data)
                    uri_1 = res.getElementsByTagName("uri")[0]
                    rel_id = uri_1.childNodes[0].data.encode('utf-8').rpartition('/')[2]

                    URL_r = 'http://www.discogs.com/release/' + rel_id + '?f=xml&api_key=' + api_key
                    request = urllib2.Request(URL_r)
                    request.add_header('Accept-Encoding', 'gzip')
                    response = urllib2.urlopen(request)
                    data = response.read()
                    unzipped_data = gzip.GzipFile(fileobj = cStringIO.StringIO(data)).read()
                    xml.etree.ElementTree.fromstring(unzipped_data)

                    def getGenres(tree):
                        genres = []
                        release = tree.find('release')
                        genreList = release.find('genres')
                        if genreList:
                            for i in genreList:
                                genres.append(i.text)
                        return genres
                
                    def getStyles(tree):
                        styles = []
                        release = tree.find('release')
                        styleList = release.find('styles')
                        if styleList:
                            for i in styleList:
                                styles.append(i.text)
                        return styles

                    lyric=getGenres(xml.etree.ElementTree.fromstring(unzipped_data))
                    lyric.append('//')
                    lyric.append(getStyles(xml.etree.ElementTree.fromstring(unzipped_data)))
                    lyric=str(lyric).strip('[').strip(']').replace(',', ';').replace('\'','').replace('[','').replace('Folk; World; & Country','Folk, World, & Country')
                    result.append(lyric)
                    album_file = open("C:\Python26\!album.txt", "w")
                    album_file.write(album)
                    album_file.close()
                    text_file = open("C:\Python26\!tag3.txt", "w")
                    text_file.write(lyric)
                    text_file.close()              
            except Exception, e:
                traceback.print_exc(file=sys.stdout)
                result.append('')
                album_file = open("C:\Python26\!album.txt", "w")
                album_file.write(album)
                album_file.close()
                text_file = open("C:\Python26\!tag3.txt", "w")
                text_file.write('')
                text_file.close()    
                continue
        
        return result

if __name__ == "__main__":
    LyricProviderInstance = Discogs_GetGenre()

Continued ->

noise · 04-15-2010, 08:09 AM

awesome! i'll definitely play with this over the weekend. i like your text file workaround, it will let me run through my whole collection in one go. i expect i'll have to do a lot manually, and i have quite a few releases that are not even listed on discogs, but such is life

many thanks for taking the time to put together those instructions, i really appreciate it

Guybrush · 04-15-2010, 08:14 AM

<- continued from last post

The red bits you might wanna change are the folders and filenames of the text files for temporary storage. You may wanna keep these text files out of the program files folders - at least Win 7 doesn't give my script the rights to create or write to text files in there. You may also wanna create the text files before you run the script. As I'm not actually a python programmer, I'm unsure if the script will create them, but it uses them if they at all exist.

After that, you need to start up foobar. Right click any file and go to "Legacy commands (unsorted) -> Configuration". There's an option for provider. Choose Python and then "option -> setting". A new screen opens. Click refresh in the bottom left, then OK to close it. In the configuration screen, also write the name of the tag the information should be written to, f.ex "genstyles".

Now, all you have to do is mark the albums you wanna tag, then right click and choose : "Legacy commands (unsorted) -> Python -> Discogs Genstyles".

All the information will be written to the tag you specified like this : "Genre; //; Style"

To split them into genre and styles tags, right click the selection, choose properties. In the bottom left corner, click tools and choose "automatically fill values". In the new window, change "source" to "other" and then write %genstyles% or whatever tag you used to save the information to .. Then in the pattern field, you write :

%genre%; //; %style%

And press okay. That's it!

EDIT : Included a picture as well!

The reason this ain't all beautiful is because when I started fiddling with this, I'd never looked at a python script before and I didn't make this with the intention of publishing it anywhere, but it should work. If it doesn't, just ask and I'm sure I can help.

edit :

Quote:

Originally Posted by noise

awesome! i'll definitely play with this over the weekend. i like your text file workaround, it will let me run through my whole collection in one go. i expect i'll have to do a lot manually, and i have quite a few releases that are not even listed on discogs, but such is life

many thanks for taking the time to put together those instructions, i really appreciate it

You're welcome! I suspect I made it look harder than it is, but since you have foobar experience already, it should be fairly easy

noise · 04-15-2010, 08:23 AM

oops sorry, didn't mean to interrupt your post

i've used that script plugin before so it won't be tough to sort out.

thanks again. looking forward to this!

Guybrush · 04-15-2010, 08:25 AM

Quote:

Originally Posted by noise

oops sorry, didn't mean to interrupt your post

i've used that script plugin before so it won't be tough to sort out.

thanks again. looking forward to this!

Let me know how it works out

Seltzer · 04-15-2010, 11:03 AM

Quote:

Originally Posted by noise

ipod flashing is very safe. all the original firmware remains intact you can boot into it at any time. as far as i understand, the only thing actually flashed is a sort of boot loader that directs things towards the custom firmware. and of course it's all completely reversible

I'm reading about Rockbox now and the manual makes it sound like nothing is flashed at all as the Rockbox bootloader & firmware are stored on disk rather than in flash memory. The Rockbox bootloader apparently replaces the iPod firmware on a boot partition (and presumably the original iPod firmware is copied elsewhere) so the boot sequence would be iPod bootloader (flash) --> Rockbox bootloader (disk) --> Rockbox firmware (disk), and the original iPod firmware could still be loaded from another location if need be.

This not only makes it even safer than I would have imagined but would also allow me to erase any evidence of using Rockbox should I need to make a warranty claim on my iPod. Up until now, I'd avoided looking into Rockbox because I assumed it manipulated flash memory and would thus make it harder to discreetly revert but that doesn't seem to be the case and so it's easily reversible as you say.

Well I guess I'll install Rockbox when I have the time - the project sounds quite mature now so I'm sure it's relatively free from threading, performance and power consumption issues. The coder/hacker in me really wants to play around with this.

Quote:

Originally Posted by tore

Sounds good, but I'm not so sure about using the genre tag for information like soundtrack and compilations! If you use foobar, why not just store that information in a new tag (f.ex "album type"?).

I didn't like sorting by genres back in the days, but now I've got a python script part written by myself which I can run from inside foobar that fetches genre and style tags from the discogs database for releases. It's fast too! It's nice because discogs have a reasonably strict set of allowed genres and styles and any album can have more than one in each field which is fine since foobar has no problems with multiple values.

Style Guide - The Unofficial Discogs Wiki
Example album on discogs, Over-nite sensation (note genre and style information)

If I didn't use this, I'd probably have over a thousand different genres still on my computer ..

Yeah it is a bit of a hacky solution to set the genre tag as Compilation or Soundtrack but on the other hand, it is iPod friendly (iPods don't natively support custom ID3 tags afaik).

I didn't mind applying the genre tags to my entire collection because by that point I'd already had each artist in a genre folder so writing that structure to a tag was a simple batch operation using Tag & Rename. This does mean however that all of my artists each belong only to one genre but iPods don't allow songs to have multiple genres anyway... if a song has a genre tag which is a comma/semicolon delimited list of genres then iTunes/iPods simply see the song as belonging to a custom genre defined by that string rather than belonging to each of the genres. A common hacky solution is to use delimited genre tags and create smart playlists which include music on the basis of some rule (i.e. genre tag contains 'Jazz'). I can't see iTunes properly supporting multiple genres any time soon (people have been after it for ages) and even when it does, iPods would still require firmware updates to support this.

Nice work on the Python script; I might use it one day when I get around to using the smart playlist hack or if I decide to flash my iPod.

04-14-2010, 11:22 PM	#52 (permalink)
noise thirsty ears Join Date: Sep 2009 Location: Boulder Posts: 742	ipod flashing is very safe. all the original firmware remains intact you can boot into it at any time. as far as i understand, the only thing actually flashed is a sort of boot loader that directs things towards the custom firmware. and of course it's all completely reversible __________________ my flac collection

04-15-2010, 03:26 AM	#54 (permalink)
noise thirsty ears Join Date: Sep 2009 Location: Boulder Posts: 742	tore, are you willing to share that script? it sounds incredibly useful! at one point i tried using various plugins to grab genre/style info from AllMusic, but never got very good results... i'm not sure how much i would like using someone else's categorizations, but i can always back up tags and restore them if things don't work out __________________ my flac collection

04-15-2010, 08:09 AM	#56 (permalink)
noise thirsty ears Join Date: Sep 2009 Location: Boulder Posts: 742	awesome! i'll definitely play with this over the weekend. i like your text file workaround, it will let me run through my whole collection in one go. i expect i'll have to do a lot manually, and i have quite a few releases that are not even listed on discogs, but such is life many thanks for taking the time to put together those instructions, i really appreciate it __________________ my flac collection

04-15-2010, 08:23 AM	#58 (permalink)
noise thirsty ears Join Date: Sep 2009 Location: Boulder Posts: 742	oops sorry, didn't mean to interrupt your post i've used that script plugin before so it won't be tough to sort out. thanks again. looking forward to this! __________________ my flac collection