an easier way to move an iTunes library

When I first got my MacBook Pro, I was a little flummoxed. How was my 85GB iTunes library going to work with my 80GB hard drive? So I got a 250GB USB drive, connected it to the XP machine, and siphoned off the whole library. I used a grep tool to find and replace all instances of “C:\Documents and Settings\Dan\My Music\iTunes Music” with “MYNEWUSBDRIVE::iTunes”, and when I got everything back together, it all worked nice and peachy.

I decided, though, that USB was too slow (I was using a hub), and then determined that I just didn’t like being tethered to an external drive whenever iTunes was open. I went through the same process (including grepping and finding and replacing) and restored my library to the hard drive of the XP machine. (This has advantages: the XP machine is on all day and connected, so it can grab podcasts. Any downloaded music pretty much comes through the XP machine anyway, and the power on the USB ports seems more reliable. If you’ve even closed the lid on a MacBook while an iPod was syncing or charging - heck, if you’ve ever gone to the Apple store and seen the majority of the iPods trying to shake themselves out of a “Do not disconnect” loop, you know what I mean.)

Of course, now the library is bigger, I depend and rely on my Mac more and more, they sell movies in the store, and there’s just no way that a MacBook drive could ever keep up with it. But, all about the convenience as I am, I figured the XP machine was just becoming the main fileserver of a home network - I’d pull MP3s wirelessly off the network like the cool kids do.

That meant sharing my MP3 folder across the network, but that’s something I already do. It also meant grepping through that stupid file again, right?

Turns out not so much. If you take an iTunes library file (it’s the one called “iTunes Library”, or, on a PC, “iTunes Library.itl”), and you move it to a new machine, it’ll complain that it can’t find any files. But if you go into “iTunes music folder location” and give it the name of the folder where it can find those MP3s, it’ll suddenly fly straight, apparently running that find and replace automatically. It said “updating library” for about 20 minutes over the wireless connection, but it worked, and less than 15 minutes later I was cursing myself - first, for waiting so long, and second, for thinking that I could keep an active connection leeching 320kpbs MP3s over a wireless connection that’s also trying to keep the phone line and everyone else’s internet afloat.

You might say that iTunes has a perfectly good “share playlists over the network” feature, but it doesn’t let me edit, modify, or even rate songs in the library. (It doesn’t bump the play counts - but that’s just a pet peeve.) You can look up people who’ve tried to throw their libraries on networks, or even Amazon’s S3, but I think the iTunes part of the digital hub works best when everything is tied down, or when the wireless machines are truly acting as satellites and not pretending they’re in constant contact with the base.

So the whole thing bombed out, but since I hadn’t actually changed any files, I just closed down iTunes on the Mac here and started it back up on the XP machine. As I said, it’s probably better that way.

smart playlists considered harmful

iTunes performance on my PC has just been dragging lately. Seriously. Whether it’s adding files, syncing stuff up, or just fixing title case, I feel like I can have iTunes open or everything else.

“This is probably because my library’s so big,” I concluded. I’ve been reading up on splitting libraries, moving libraries to USB drives, storing your library on Amazon’s file servers, and a bunch of other plans and schemes, but no real guidelines on how big is too big. (Then again, if they sell an 80GB iPod, they have to expect that people will have libraries that big, and not because they’ve bought Pirates of the Caribbean 120 times.)

Further research inspired an uncomfortable realization: Smart Playlists might maybe wreak havoc on system performance. And why shouldn’t they? If you have a Recently Played or Most Frequently Played playlist, then those are updated every time a song is played. If you have a Top Rated playlist, it’s updated every time a song is rated. And if you have a “Brit’s Greatest Hits” playlist, with Artist Name = “Britney Spears” and My Rating greater than three stars, then every renamed file checks with that playlist to see if the new metadata qualifies it for inclusion.

(Implementation note: I think you could update playlists only on viewing, or on iPod syncs, but playlist sharing and other implementation trivia might get in the way of that. I don’t know how this is stored, much less how it’s optimized, so I could be talking out of my butt on this specific point.)

I may or may not have a larger library than the average iTunes user (it looks like there are certainly power users well beyond my 100GB), but I would be willing to bet I’m in the top 1% of Smart Playlist junkies. I have created SPs for:

  • Artist discographies
  • Year-end Best ofs
  • BPM filtering (yeah, right, for all the DJing I do)
  • Diagnostics (songs with obviously bad years, no album title, etc.)
  • Favorites lists
  • Genre-based mood lists
  • Artist “groups”, to keep musical families or record labels together
  • Odd queries (which four-star songs haven’t I heard in three months? what have I imported but not listened to yet? what have I heard a million times but not rated?)
  • Year-based reminiscing (rock albums from 1992 to 1995)
  • Filetype filtering, because sometimes other programs really want MP3s and not anything else

My love for this feature probably borders on abuse, and the stark fact is that I have a lot of very large smart playlists that I only very rarely use, or could recreate almost immediately if it came to that. I had no idea I was suffocating it with my love. A lot of these are playlists that could be converted, after they query the database once, to regular playlists, and I wouldn’t miss much.

So that theory makes sense. Unfortunately, I can’t find a really thorough discussion of the problem beyond 2005ish, and iTunes has gone through many updates and enhancements since then. Someone described smart playlists as tiny little applications themselves. iTunes operates at an acceptable level with 100 smart playlists of varying sizes and complexities, but if you’re starting to become performance-oriented, they might be dragging your system down.

Regardless, I wiped most of the dynamic playlists from my iTunes (converting some to regular playlists), and it seems like it’s a little peppier today. Again, it is completely possible that there’s a release note with iTunes 5 or 6 or 7 that says “fixed the problems with smart playlists - create as many as you want now”, but it seems reasonable to think that smart playlists might be best used in moderation.

(I made an exception for the three gigantic playlists that fill my iPod: 20GB of recently added files, 20GB of top rated files, and 20GB of frequently played files. I absolutely cannot put my iPod into manual file-draggy mode. Never.)

real quick, what a GTD app should do

Wow, the internet is lousy with halfway-there GTD apps. It’s disappointing, because as much as you can do this whole thing with paper and pencil, it lends itself supremely to the digital world.

First things first: chop off the parts of GTD that won’t work better in the computer world. Ubiquitous capture just can’t happen online, in a desktop app, because you aren’t always there. (If you are close to always there, then you are close to ubiquitous capture, which isn’t ubiquitous at all.) So throw that out. Higher-level reviews are really about life planning and not losing what you considered life-focusing or important at any day or time, so I guess some sort of master list of that stuff could go in there, but you’d really want it journaled, wouldn’t you?

It leaves a to-do list. Really, really amazing to-do lists are trying to pass themselves off as “GTD solutions”, and I’m not buying it. The key to the GTD to-do list is that it has to be multi-dimensional.

Basically, all the tasks have to have a matrix of optional attributes, including:

  • project
  • context
  • topic (high-level project)
  • time estimated
  • deadline
  • urgency
  • importance

Some of these dimensions are actually multi-dimensional themselves, like a project tied to two life-goals or a task that can be completed in any of a few contexts. The kids today love organizing that by “tags”, because it’s a freeform way to describe your stuff, but the open-ended tagging systems I’ve used (in stikkit, rememberthemilk, and others) are simply too open-ended. It’s great to be able to describe your tasks in these ways, but you can’t stop there. I haven’t seen in any system a way to pull the tasks that are lacking a context, for example, or the ones that don’t have time estimates, or ones for which the deadline is in the next 7 days, or whatever.

If you think about it, this is the exact same kind of filtering that iTunes does. I think the optimal solution is something along those lines: enter a ton of tasks, projects and ideas (as simply as possible), and then slice and dice through those based on what attributes they need defined (to finish up the data entry) or already have. iTunes playlists can tell you what jazz tracks, four stars or higher, haven’t been listened to in the past four weeks. Address Book can tell you which contacts you have in Ohio with no home phone number defined. A GTD to-do list should offer just as simple filtering.

The best thing about this approach is that it’s not too difficult to set up (if the data entry is simple enough), maintain, or tweak. iTunes newcomers don’t have their custom playlists tweaked out into infinity, but those who do can share their playlist definitions with others. It’s simple and extensible.

The important part of this is getting out of the way of data entry, because “task definition, assign context, tie to project, set importance, enter deadline, estimate time” doesn’t work for everyone and/or every task.

Honestly, I think the problem is pretty simple, and is just being danced around / poorly understood by the people who are doing this work. I like ThinkingRock best so far, because it actually meets you halfway between “anything goes” data entry (because this kind of work really is creative) and a supportive structure. It doesn’t go tag-crazy (and it could, and it will, I’m sure), it doesn’t accommodate time estimates (which I find useful, just because I pick off 15 minute things more often than I plan half-day things), but it reflects what I understand about GTD better than any other solution (downloaded or home-brewed).

switching back (for iTunes)

It was one of the few reasons I got interested in Macs in the first place: sure, iTunes is awesome on XP, but if I got a Mac, then I’d have a bunch of other like-minded applications that could work with iTunes.

And then it was one of the only reasons I was apprehensive about the MacBook Pro: I have about 50% MORE in iTunes gigabytes than the MacBook has hard drive space. And I’ll need some of that space for, y’know, the OS.

But I got the MacBook Pro, added a 160GB portable hard drive, and I was off and running. (Actually, I plugged it into a USB 1.1 hub, and wondered why the whole thing was so damn slow, but eventually figured out my problem.)

iTunes on Mac is nice because:

  • It’s pretty zippy. My XP copy gets caught up on little laggy issues every once in a while, but I’m not sure if that’s memory or what.
  • While it’s not quite what they promise you with “the lifestyle” on Macs, being able to pull actual iTunes playlists into random apps (like Apple’s presentation software, iDVD, Dashboard widgets, or even third party slide show software) is pretty close to magic.
  • There are some killer AppleScripts that do really nice things. (I believe I’ve mentioned that one runs through your selection, cleaning up improper title case. That’s just one example.)
  • pzizz talks directly to it, as does Handbrake, which meant DVD rips happen with about two clicks. Scary fast.

iTunes on Mac is pretty much exactly the same for:

  • Working with iPods. I was really expecting to see some magic here, but it’s just as fast on XP, and just as easy.
  • Large playlists. I was thinking that my thousands of songs might get sliced through a little more quickly, but no. Even Cover Flow is about as laggy on my Mac as on my PC.
  • Purchasing / burning / etc. There are no additional barriers to buying, downloading, or burning anything on either platform. They’re basically identical.

iTunes on XP is better for:

  • MusicBrainz. Jeez, there’s a Mac equivalent called iEatBrainz, and it’s really pushing the limits of “equivalent”. I had no end of problems with that software. MusicBrainz is a remarkably complete solution (and usually has correct title case, to boot)
  • VolumeLogic. I paid for this plugin a long time ago: it adjusts the sound from iTunes and equalizes the volume. Easy enough, right? Well, the Intel Macs confound the developers of the app, so they haven’t released a version I can use on my MacBook. And it’s all a moot point now, because apparently 7.0 broke their implementation, and it doesn’t work for PCs now, either. (If you’re on 6, get Volume Logic. Seriously.)

So, in a fair fight, I think iTunes on Macs is a better app / experience than iTunes on PCs. But there’s one thing to consider: how light and portable the MacBook Pro is.

I don’t think twice about bringing my Mac anywhere: even if I’m pretty sure I’m not going to use it, it packs up so small that I don’t mind carrying it around. However, with the external hard drive connected, you have to remember to eject that before you go. Even if the Mac is sleeping, you have to wake it up, eject the disk (closing iTunes first, of course, unless you want all those hateful little exclamation points to tell you your music is missing), and then disconnect it. Then, when you’re on the road, you can’t use iTunes (all or most of your music is back on the detached HD, and even if you partition your music, you get those evil exclamation points). You have to come back and reconnect the drive before iTunes is an option again.

So I started noticing all this psychic inertia to not move the Mac, because I like iTunes to be running, or feeling like I was constantly connecting and reconnecting USB cables. That’s something I thought I was leaving behind. For all the advantages iTunes offers in a Mac environment, the problems of the laptop-two-disks scenario can’t overcome the ease-of-use offered by a desktop machine - PC or Mac - and the 24-7 docking / charging you’re probably pretty used to.

But: there no reason for this. Seriously. I spent a chunk of yesterday moving my iTunes installation back to my XP desktop. I mean, I really like using iTunes on the Mac, but I have too much music to really use iTunes on a MacBook. (Of course, now they come with enough room to store my whole library, and the next ones will probably have twice again as much room, but that’s the “buying from Apple” experience for you - as soon as you get it home, they’ve innovated yours into shame and ridicule.)

It’s freeing. There’s only one cable connected to my MacBook now: the power. When I want to take it somewhere, I’ll only have to shut it and go, not dismount the external drive (which I can use for backups now, I guess). When I want to dock an iPod, I won’t have to worry about if my Mac is where it’s supposed to be.

And the killer feature is that, thanks to what iTunes does over the network (playlist sharing), I can enjoy what’s on the PC’s iTunes with the Mac. (There are limitations: I can’t rate songs, my listens don’t bump the listen count, and I can’t edit song titles to correct their improper title case, but those are all my neuroses, and you shouldn’t have to worry about them.)

I thought I might detail “moving an iTunes library from one PC to another”, but I’ve already gone on at Scorcese-length about this one dumb thing, so I guess I’ll spare you for now.

time to lock everything down

everything's OKAdam snuck into the computer room this morning (while I slept) and purchased “Everything’s OK”, Al Green’s 2005 album, from iTunes.

Also, I am serious. I just bought a mouse for the Mac: I think this helped him (the touchpad kind of confused him). I have my password turned off for iTunes (I just turned it off a few months ago), and he managed to click “buy now”, and when asked “are you sure?”, likely had no problem with finding his favorite button, “Yes”.

As I explained to him, he owes me $10. But Al Green belongs to him now. It’s supposed to be good. I’ll let you know.

Note: I went on a tear of 1-976 joke-line calling when I was 8 or 9. On one hand, I understood what I was doing, but on the other hand, there’s no way I could, you know? So I have sympathy for Adam.