Why extra headroom is needed for MP3s

It's a nasty gloomy, rainy Sunday here in the Pacific Northwest, a good day for reading a book. Unfortunately, I have no new books to read, so I went back to some old favorites and was reading up on MP3 compression. Hey, I'm a hopeless geek, what can I say?

I decided to do some experiments to test a well-known truism, that you need to leave extra headroom when mastering for MP3s. It's true, by the way.

Here's a piece of an audio file in which Ozone has done its job of limiting output (in this case, to -1db, my longtime standard). Despite being a fairly highly-compressed section (about -9db RMS), no samples exceed -1db (the horizontal white line represents -1db):



Now here's a section of the same file that's been encoded to a high-quality MP3:



Note that a few peaks now exceed the previous limit. Fortunately, they're not overs because my conservative -1db limit left enough headroom to absorb these new peaks.

This phenomenon gets worse - much worse - with lower-quality MP3s. Below is the same song, this time encoded to 128Kb/s, CBR. Not the lowest quality, but representative of what Soundclick provides with a free account.



Note that many peaks now exceed the old limit, and in fact some of them are actual overs. The peaks have somehow grown by more than 1db.

You may be wondering why this occurs. It has to do with the fact that compression schemes such as MP3 rely on narrow bandpass filters.

You may recall that steep filters can cause ringing that raises peak levels. It's one of the reasons we usually try to avoid steep filters in mixing and mastering - but you can't avoid them if you're encoding to MP3.

There is a good article titled "Distortion to the People", written by Thomas Lund from TC Electronic and available as a pdf download. It discusses the need for headroom, not just with MP3 but other compressed formats and CDs. It recommends 5db headroom for MP3. Theoretically, you can get up to 6db increases when an MP3 is decoded.

Furthermore, uncompressed CDs are not immune from surprise peak values. Intersample peaks can be as much as 3db above the highest "peak" value when converted to analog.

Bottom line is that if you're mastering to -0.1db, or even -0.5db, stop that. Nobody is going to notice if your CD is limited to -2db or -3db, but the quality of your MP3s might noticeably improve if you do.

Thanks for posting this.
Wayne

Yeah, I noticed that when converting to mp4/m4a files as well using Sound Forge or iTunes. A drum track limited to -2db grew to having peaks at full scale!

Not only that, but the mp4/m4a conversion also sometimes seems to crop about 20 -30 ms off the front of the file too and I'm having to add a bit of silence at the beginning before conversion to compensate. It may not seem a lot for a stand-alone file but if you are dropping compressed clips into a DAW its best to check that the transients of the audio line up where they should.

Cliff

So, does a highly compressed track have more exceeding samples on average than a more dynamic one? Does the mp3 encoder kinda work worse on hot tracks, or is it just based on probability? A more compressed track will have a higher percentage of samples that are very close to the max headroom allowance than a less compressed one, so a less compressed one will have less overs as there were less samples within a dangerous range to become overs in the first place? Does that make sense? Either way, interesting stuff. If you got any recommended technical reading on this I would love to know.

Cheers

So, does a highly compressed track have more exceeding samples on average than a more dynamic one? Does the mp3 encoder kinda work worse on hot tracks, or is it just based on probability? A more compressed track will have a higher percentage of samples that are very close to the max headroom allowance than a less compressed one, so a less compressed one will have less overs as there were less samples within a dangerous range to become overs in the first place? Does that make sense? Either way, interesting stuff. If you got any recommended technical reading on this I would love to know.

Actually, highly-compressed files are not necessarily more likely to exhibit this problem, since it's about how many peaks cross the line. A lightly-compressed file could still have the same number of peaks that touch the limit, even though the average RMS is lower.

It's more about the chosen MP3 conversion quality. If, for example, you upload songs to a free Soundclick or MySpace account, they're going to convert it to 128KB/s CBR, not a particularly high-grade format. It will have worse problems than, say, a 256Kb/s file.

So if you're distributing MP3s at 320Kb/s CBR, you'll be OK as long as you maintain at least 1 or 2db headroom. (If you have a choice, CBR may be better than VBR, which is prone to sudden bursts of high-frequency content as the bitrate changes. VBR is better when file size is the overriding requirement.)

Here are a couple of relevant links:

Pleasurize Music Foundation: MP3 headroom
Pleasurize Music Foundation: intersample clipping
Distortion to the People

Principles of Digital Audio by Ken Pohlmann is a fat, dense textbook and if you have the patience to wade through it, contains a lot of good information, including a chapter on perceptual coding.

I just sent them a .wav file 16/44 peaks at -1.2dBFS RMS at -14dBFS.

Once you send up your baby it's completely out of your hands.

Each vendor they supply too has a different way to do it but presumably it's all automated and quite heartless.

Tunecore began with accepting .mp3, then they moved to FLAC, now they request that you upload 16/44 .wav so that's what I did.

My wife is building me a shopping cart and I'll host and offer 320kbs .mp3 files direct from my site in the near future... because all the sites Tunecore will help yu with will only do lo-res. Apparently if someone in house at ITunes thinks its worthwhile they MAY offer a high res for you... but it's totally at their whim.

File all this under "you just have to try" because I'm not so much expecting anything to happen as I am acknowledging that nothing can happen if you do not make it possible. So I'm finally taking the first step.

best regards,
mike

I guess the only way you'll know is to download their version and check it out. I wonder if you have any recourse if you don't like the result based on technical criteria.

GREAT info bitflipper... I've been working so hard lately to get my stuff mixed better, and I'll add this to the pile! Grazie!

If you edit an existing mpg and recompress it, and again, and again - you get the same results as when editing a jpg in Photoshop and saving it as jpg every time. After so many rounds there will be holes appearing in your file. It's awful.

That's why I never work on compressed files unless it's the only way to get the data. You want to work with either lossless compression, or failing that, none at all.

Isn't it sort of like editing GOP video?

Bitflipper -

Do you use the free DR plug in from the pleasurize site ?

It looks like another interesting visual tool to help guide one to place where there would at lease be some consistency to the 'loudness' of one's efforts.

That MP3 tutorial you pasted may explain why my 'rough MP3s' sometimes sound better than my 'final MP3s'. (my rough MP3s I do not limit that much)

JR

Do you use the free DR plug in from the pleasurize site ?

Yes, I've been experimenting with both the offline version and the plugin.

At this time I am unable to say whether it's actually helped me much, though.

I am uncomfortable that its creator has not, AFAIK, publicly described its inner workings. I'm therefore not sure what it's really measuring, and not knowing that, I am not sure what useful information it's conveying.

But it's fun to play with.

ORIGINAL: bitflipper

You mention the compression of the Soundclick site - when songs are uploaded there, they've already been compressed?

What happens is they are re-encoded to meet the site's requirements. As you might imagine, encoding the same file twice does not enhance its fidelity!

This applies to free Soundclick accounts only. Paid "premium" accounts just store the files you upload, without modification. But free accounts are restricted to 128kb/s CBR.

Anything else you send them, even 128kb/s VBR, will be automatically re-encoded and their quality reduced. Experimentation has shown that sending them a higher-quality file results in less degradation. But sending high-quality MP3s means bigger files, so you run into the catch-22 of the 10MB size limitation. You therefore want to send them the highest quality file that comes in under 10MB in size.

LAME has a boatload of options. Here is a list.

As for quality options, generally the higher the bit rate the better. 64kb/s is OK for speech. 128kb/s is about the minimal acceptable quality for music. 192kb/s is a good compromise between quality and file size. 256kb/s or 320kb/s is near-CD quality.

You can specify either constant or variable bit rates. With the latter, the encoder uses higher bit rates for sections that have a lot of high-frequency content and lower bit rates for sections that do not. The result is better resolution without increasing file size too much.

Use the -V n option to allow LAME to use variable bit rate (VBR), where n is a number between 0 and 9 that indicates the quality level. -V 0 gives the best quality, -V 9 gives the smallest files.

If you have a file size limitation (as with a free Soundclick account) you'll have to experiment to find which option you can get away with for any particular song. Start with -V 0 and see how big the file is. If it's too big, work your way down from -V 1 to -V 9 until you get the file size down where you need it to be.

Soundclick has both Lo-Fi and Hi-Fi Options for either free or paid accounts.

The file at Soundclick is re-encoded to Lo-Fi, if you take note when uploading be it free account or paid once the file is uploaded another window opens stating: 'now convertining to Lo-Fi' -- *ugh* I can live with 128kbs but I wish there was an option to not include Lo-Fi. Anyway... people soon realise it's best not to play the lo-fi version.

Generally! 128kbs is OK (not great but OK) anything less is not so great at all.

I set my main output levels to -1 and have never had any issues.

For years my mixes have always been -1 (outputs) including for CD release, however.... on some sites if their MP3 playback don't sound as good I simply reduce output level by another 1 or 2 but so far I have not had to do that because I have not had any issues other than corruption on upload (simply re-upload the same file and it's OK, if that option is not immediately available then contact support, they will remove it so you can try again).

Another thing is... increased peaks in that wavform you demonstrated is nothing to get overly worried about (if there's no horrible distortion or overly noticeable artifacts occuring) then live with it... that is just the nature of 'conversion' to MP3 format and we all know that isn't the best format but that's what a lot of listeners are use to (MP3) and some bands offer as a 'free' song (single) download (this is a good marketing stradegy I am seriously consider doing after reading up on it and since piracy can't be beaten) I understand why MP3 downloads is all too common it's because the band or record label wants to generate more interest for fans to purchase their CD (Thank you for downloading our song (MP3 format), we also have a CD available that is of the highest quality - click here to purchase) Having said that... do you get what I'm trying to say Bitflipper? bands give MP3 away for free so what if the sound is less than spactacular compared to CD. It's the song that matters most to fans and if they want more and if they want a better quality product give them the option to BUY the CD. Please read the following article: Click Here

So bit-- has this revelation induced you to consider rethinking your ceiling, to maybe -2 db?