Tag Archive for 'music'

How to Not Suck at Transcoding Music

Sometimes I get really interested in something and can’t sleep and end up spending all night working on it. This was one of those nights.

I wrote flacsync a while ago in order to make an MP3 with suitable tags for every FLAC I own. I do this so that I can actually play my favorite music on my iPhone and also so that I can keep a lightweight copy of my music database on my laptop, which has far less space than my desktop.

The idea was to keep a database containing an MD5 hash for every FLAC I have and whenever I run the script, check that hash to see if the FLAC has changed and needs to be re-transcoded. If so, transcode it and store the new hash to the database.

I made some remarkably stupid initial design choices. I knew that I wanted to thread it in order to maximize throughput, but I had some ridiculous bottlenecks. For example, for some reason, I thought it would be a good idea to use a SQLite database to store the hashes and then have a complicated DBWorker thread that would interface with all the processing threads. Although I got this original design working, it was slow. It took maybe 30 minutes to run through all my FLACs.

I later redesigned the script to just load a Python dictionary containing all the hashes from a file into memory. Then, I could update the table freely and wouldn’t even really need to worry about locking since only one thread acted on a track.

But this was still slow for a few reasons:

  • I wasn’t ordering the list of files intelligently at all. It would make sense to try to process the most recently changed files first, wouldn’t it?
  • Python’s [cci]threading[/cci] module isn’t actually capable of performing tasks on multiple processors. It can still only perform on at most one core.
  • Running [cci]md5sum[/cci] on an entire FLAC is slow and therefore dumb.

So, I redesigned the whole script to use the [cci]multiprocessing[/cci] module’s [cci]Pool[/cci] abstraction where you can apply a function onto a list with a pool of workers and then gather the results. Now, each worker returns either an indication that the file didn’t change or a new hash for the file. The system tallies up all the new hashes at the end, updates the database, saves it to disk, and exits. Oh and when it first starts up and finds all the FLACs in my music directory, it sorts them so that the most recently changed files are first.

Moreover, I was lazy in that I was just calling the [cci]md5sum[/cci] program on the entire FLAC, so I used Python’s [cci]hashlib[/cci] module to only take the MD5 of the first 4096 bytes of the FLAC. This is pretty okay because the header information is almost always entirely contained there.

The result is that I can fly through my entire music library in like a second (okay some of that is coming from disk cache — I haven’t tried it on a cold boot yet). Transcoding on my computer (piping [cci]flac[/cci] to [cci]lame[/cci] and then copying tags over) takes about 15 seconds on average.

So 30 minutes just to check a already-sync’d database to a few seconds. Pretty good speedup.

I guess now I should go to work.

Glowscape – An Experiment with Glowstick Dye

It’s been like forever since I’ve blogged. Sadness. Here’s something cool though. 🙂

A while back, I bought an absurd amount of glowsticks for a few bucks on Woot.com. I was wondering what I could do with some of them.

One idea I had was to make some kind of art thing or whatever. After a little preliminary investigation, I discovered that the glass vials in the glowsticks were full of the fluorescent dye that can also be activated by ultraviolet light. So I decided to test it out.

Basically, I just taped a UV LED to the bottom of the glass vial that normally goes in the glowstick. It actually looks not bad. I set up a quick demo with some UV LEDs. Hmm… what could I do?

Maybe a VU meter for sound? Yeah, that might work. 🙂

So, I grabbed 16 red, orange, yellow, and green glowsticks and headed over to MITERS to get to work.

The first step was to sand down the LEDs so they were flat. First they got the bandsaw treatment, followed by some more gentle loving with fine-grained sandpaper.

Hell yeah. All right, time to cut the glowsticks open and extract the glass vials.

So the other stuff left in the plastic part of the glowsticks is high concentration hydrogen peroxide. I decided to save it in a bottle. The concentration is so high that it’s actually very sludgy.

Next step was to sand down the end of the glass vials to make them as flat as possible. Then, I used duct tape to hold them against the UV LEDs.

All right. So then, I ended up letting the project sit for several weeks because I got super busy at the start of the semester. I also had to layout a PCB for the design I wanted. Essentially, my plan was to make something compatible with ACRIS but also provide a standalone mode so that it can independently analyze music.

But, I didn’t want to pay people to produce a board. I wanted to do my own etching. Previously, doing this was extremely difficult because MITERS didn’t have the right tools to do it.

To refresh your mind, etching goes like this. First, print out the PCB layout onto photo paper with a laser printer. Then, use a heat source to transfer that laser printer toner onto sanded copper-clad board. Next, throw that board in PCB etchant and wait for the copper you don’t want to disappear.

But it’s never that simple. The real problem is the transfer. My awesome girlfriend Jordan used a clothes iron to perform the transfer. This method is incredibly unreliable. It’s really hard to get right and every time I’ve done it, I’ve always lost huge portions of the transfer. Jordan made several boards and the later ones she built looked absolutely spectacular — she figured out the exact amount of ironing time and pressure. I’ve tried to do this too, but I’ve never been able to get it right.

But I got to bypass all that because MITERS recently bought a laminator. Laminators provide an even amount of heat across the board with relatively little pressure, so they transfer the toner perfectly every time. Check this out:

The other awesome thing is that MITERS found some new paper that’s really perfect. Usually, it is recommended that you use glossy photo paper. But, a lot of photo paper is very plastic-y. So, when you iron it, the plastic actually forms a boundary so that when you try to wash off the paper after the transfer, it never gets soaked. This new paper isn’t photo paper — it’s just regular glossy printer paper. And it works perfectly because it washes off really easily.

Another thing is that I made the traces really thick because I didn’t want to take any chances. You can’t really do very complicated boards this way. In the future, I’m going to

Next step was to throw it in the etchant. MITERS reuses etchant a lot, so in combination with an agitator, the etching time was about 30 minutes total. It’s important to constantly check it because you don’t want to etch longer than you have to — otherwise you may etch underneath the toner.


Okay. Time for drilling. This one is really hard. It’s very difficult to drill small holes through fiberglass — you can easily blow through a pack of bits if you don’t know what you’re doing. MITERS has this cute little drillpress that I forgot to take a picture of. But it does a great job of drilling stably so that the bit doesn’t break.

Okay, time for board assembly.

Ugly jumpers are ugly. 🙁

My first idea was to align the tops of all the tops of the glowstick vials.

Then I didn’t like the lead spacing at the bottom, so I put it back.

On the first power-up, one of the UV LEDs blew out. Not sure why — I think the quality control on these LEDs is kind of bad.

IDK, it’s not as bright as I would have liked, but I’m trying to figure out what I can do about that. Here’s a video of it just doing a simple chaser effect:

Schematics and board layouts are available on the project page.

Music Visualization Framework Started!

Check this out:

I’m using PyAudio to capture data from my sound card. Right now, I’m just using audioop’s RMS feature to make a loudness meter.

New Glow-ey Project

I was looking at some empty glass Coke bottles the other day and started wondering how I could make something beautiful out of them. Since I can’t seem to shut up for 2 minutes about high-power LEDs and music visualization, I started envisioning some kind of music visualizer based on glowing bottles.

Not sure what the best material to use is yet. My thought is that UV glow paint might work well, but it can be kind of expensive.

I bought a ton of glowsticks off of Woot a while ago, so I’ve been wondering how I can use them. I decided to crack a few open to test things out.

The first one I opened was blue. I first cracked the glass vial inside it, let the reaction finish, and then held it up to a UV light. Then, I added some water to see how much I could dilute it.

I noticed that the material wasn’t dissolving too well. Also, I couldn’t tell where the fluorescent dye was located. A glowstick basically has two parts: the first is a reservoir of hydrogen peroxide; the second contains dye and some other stuff. When glass vial is broken, the peroxide mixes and causes a chemical reaction to occur. However, the dye itself can also respond to a UV light and that’s what I’m trying to take advantage of.

I decided to open up two red glowsticks to see… Turns out, for these glowsticks, the dye is located in the vial (most times, the peroxide is in the vial).

So, I wrapped the very tip with a paper towel and used a pair of diagonal colors to chip off the very top and pour it into a Coke bottle. Awesome picture ahead:

Here’s what two vials worth of glowstick dye look like in the bottle.

I tried adding water. Looked okay at first:

But it separated out! Damn… it must be oil-based.

So I carefully got the bubble out of the bottle of water (tipped it upside down, waited for the bubble to fall down to the bottom, let it out). Then, I refilled the bottle with a bit of olive oil to see if it would mix with the oil. Turns out, it does!

(the stuff at the bottom is just excess water… ignore it.)

Unfortunately, you can’t use olive oil forever — it’ll just go rancid. So, you need to use mineral oil, which isn’t particularly cheap. Maybe it would better to use these glass vials of dye in their own project. I could easily make some kind of visualizer out of them. Perhaps I’ll save the Coke bottles for a future project.

(Sorry about the crappy photos — I was using my phone).

flacsync: Automatically Sync FLACs to MP3s

In an effort to make organizing my music collection suck less, I wrote a script to automate the process of converting my FLAC files to MP3s for when I want to listen to them with my MP3 player.

Previously, I had no way to do this in an automated way. My music collection changes rapidly, so I needed a way to convert FLACs that haven’t yet been converted (or have changed since the last conversion) to MP3s. The first step of this process is to decode the FLAC and then pipe that to an MP3 encoder.

Next, I had to extract as much tag information as I could out of the FLAC and convert it into ID3 tags. Finally, if there are image files in the directory containing the FLACs, the script automatically embeds those images into the MP3 file (and tries to determine what type of image they are).

It also has a configurable number of worker threads, so you can process the files in parallel. It keeps a database of hashed files in ~/.flacsync/db, so when you re-run it, it will only retranscode new or changed FLAC files.

You can find it here. It uses some Unix commands like find. It requires Python 2, Mutagen, FLAC, and LAME.