AAC-HEv2

Wow, that looks like a lot of technological psychobabble. In reality, it isn't, it's quite nice.

You know how all that music you have in iTunes gets ripped when you import from a CD to this funky format called m4a? Well, that's just a fancy name for an audio format called AAC (Advanced Audio Coding). M4A is an abreviation of MP4-Audio, and MP4 is a container format like AVI. And for those of you who didn't know, AVI's contain XVID video codecs or MPEG-2 video and who knows what audio format, could be MP3, MP2, Vorbis, etc... That's why sometimes you'll download an AVI and Windows Media Player will complain about not being able to play it. The AVI is fine, you just don't have the filter to play whatever video or audio format is in it.

Enough on that. So all this music you rip from CDs in iTunes are by default in the M4A format using AAC audio. There are multiple types of AAC. The two most common are AAC-LC (Low Complexity) and AAC-HE (High Energy, comes in v1 and v2). iTunes uses AAC-LC, and the underlying Quicktime does the same. That's what you hear when you listen to your music, audio encoded with AAC-LC.

AAC-HE is AAC-LC with some extra data thrown in that helps the audio quality. What does that mean? You can have an AAC-HE audio file and you can still play it in Quicktime, it'll just play the AAC-LC components. And it'll probably sound like crap.

Explaining why it sounds like crap requires me to go into some explanation about digital file formats. Lossy formats are encoded to a certain bitrate. In general, higher bitrate = higher quality. That's because the encoder can produce a bigger file, retaining more of the original audio data, meaning less compression. You might've seen it downloading music using Limewire. There's a column that says Bitrate, and it has numbers like 64, 128, 196, 256, and some others. 128 is standard. The default encoding quality in iTunes is 128kbps. If you've ever listened to 64kbps or 96kbps encoding, you'll most likely be able to tell a difference and agree: it sounds like crap.

So what does that mean about the LC component of an AAC-HE audio file? Yup. Low bitrate. Commonly encoded at 64kbps. Yes, it can go up to 96 or 128, and probably 160 (or down to 32 or 16), but that would defeat the purpose. But an AAC-HE audio file, when listened to in a player that supports the HE part (foobar2000, Nero's player, WMP with the right filters) it'll sound great. It's the HE part that makes that difference.

The HE part came about from new discoveries in digital music. Audio taken from a CD is sampled at 44100Hz (samples per second). That means that a number representing the audio is taken 44100 times per second. Due to the Nyquist theorem, that means the highest frequencies that can be held in it will reach 22050. That's convenient - the ear can only hear about that high anyways. And thats an ear in perfect condition - with ear damage over time that lowers to 18000. Thats one way MP3 and AAC compresses audio - gets rid of the highest frequencies. That gets rid of a good 18ish percent of all the audio data, and that goes a long way to compressing the audio.

A 64kbps audio file will push those limits and will seriously reduce the frequency, getting rid of most stuff above the half way mark - 11025. Half of the audio data. Surprisingly, even with all those frequencies gone, the audio can still be recognized. It'll even sound halfway decent! Think about it - the adult voice ranges between a frequency of 85 and 255 Hz (lower half for men, higher half for women). Children's voices don't get that much higher. So all the lyrics get preserved. And most instruments get preserved as well. The reason it won't sound as good is because a little bit overlaps into the higher frequencies, and that little bit makes a difference.

But it's only a little bit, so for LC encoding, they just get rid of that. However, it can be reasonably well reconstructed. That's the principle that AAC-HE is built off. Low frequencies will predict the higher frequencies, so they can be reconstructed from the audio data. (This concept is called Spectral Band Replication.) The small amount of extra data that HE saves is enough to make the reconstruction more precise. It's about 1 (yes, one) kbps. Maybe two. This extra data, when used to help recreate the higher frequencies, makes the LC audio sound as if it had twice the frequency. An M4A format encoded at 64kbps sounds like it was encoded at 128kbps even though it wasn't.

Wow. That's amazing. So why doesn't iTunes support that?

It takes a lot of processing time to restore data through Spectral Band Replication. Processing time the iPod doesn't have. If the iPod couldn't play back the audio real time, it would sound choppy, and listeners would get pissed. Not good.

Seems like a dead end. But as the standard gets better, and programmers keep optimizing the code, realtime playback on the iPod could be a real thing, and when that comes push for support for iTunes to include it. And if/when it does, don't let the 64kbps sounds terrible mentality fool you. It'll sound just fine.

For those of you who want to try out AAC-HE files, an alternative audio player called foobar2000 supports them. Anything that is based off a remotely recent version of FAAD (an open sourced AAC decoder) will playback AAC-HE. Don't try playing them in iTunes. All that'll play will be the 64kbps terrible sounding LC audio.