Why Handbrake compressing a video from a DVD can become 300MB, 1GB, or 2GB when the original should be only 600MB?

Question

Handbrake can convert a DVD clip which is about 30 minutes to a 300MB .mkv file. For experiment, I tried also 1GB file size and 2GB, and they both came out as 1GB or 2GB.

But I estimate that the original video on the DVD should be only about 600MB, so how can it make the file "bigger than it was before"? (the mkv, mp4 format should allow better compression that the DVD's mpeg2).

I also remember I tried experimenting with divx or xvid before, and there was a limit to file size: for example, I can specify a 1.5 hour video to have 3GB or to have 8Mbit/s, but the content simply didn't have so much "bits" for it to store (because it can highly compress the video), so the final file could not exceed 1.8GB or something like that, no matter how high the bit rate or desired file size I gave it. But with Handbrake, it seems like I can create a 1GB or 2GB file with such a short clip. Why is that?

Solution 1

It has to do with upsampling/downsampling.

Imagine having to write this software: You have to read a compressed file that could have been compressed in a variety of formats, and then recompress it in one of another variety of formats.

Wait!! Why can't you go directly to another format??

Well, how would you? The compression is lossy, and in order to understand how to compress it, you first have to decompress it, to see what it's saying!

Furthermore, you don't want your algorithms to explode; i.e. if you're converting between 10 formats, you don't want to have 10 * 9 = 90 algorithms for compression and 90 for decompression to convert from every format to every other format, but you'd rather have 10 algorithms to go to a decompressed format, and 10 more to go down to another compressed format.

So you have to decompress it first. And after that, it's your choice how to recompress -- and no matter what you choose, you'll lose a lot more than if you hadn't done anything at all.

i.e. This is a bit naive of a calculation, but if something preserves 50% of the original, and you decompress it and recompress it, you got 50% of that -- i.e. 25%. This doesn't capture some of the other aspects, but it does explain why recompression (aka transcoding) is rarely recommended; if you need it, try to recompress the original instead, if possible.

Hope this makes sense.

Solution 2

Media compression isn't like zip and similar compression algorithms, which opportunistically exploit exact repetition. At its simplest, it's more like finding a representation of the signal which can have detail removed (or added) with minimal visible change to the decoded signal. That being the case, it's very easy to "compress" a small file into a larger file. You derive that transformed representation, then pad it out with extra neutral data.

That's pretty much what you get when you demand a conversion from a low-bitrate video file to a high-bitrate video file.

Strictly speaking, you probably are gaining something by having 2GB rather than 1GB, even for a video that started at 600MB - though it's probably not something that you want at any price.

The original video was compressed with MPEG2 (for DVD) rather than using MKV. Different compression systems have different "artifacts". Each conversion of the file can at best preserve the available detail. With the change in compression scheme, it is inevitable that at least something is lost. With a larger file, there's less loss - even with files much larger than the original. That means you're less likely to lose all the unwanted artifacts of the previous compression scheme.

Another way to look at it...

The set of videos that can be perfectly represented using any compression scheme is very limited, in comparison with uncompressed video. Reduce the file size by one byte, and in principle you have divided the number of possible videos that can be represented by 256.

The set of videos that can be perfectly represented in MPEG2 is very different to the set of videos that can be perfectly represented in MKV. Even if you had perfect codecs, you're unlikely to perfectly preserve your video converting from one compression method to another - even converting to a much higher bitrate.

Comments

[Matthew Fioravante]

If you want specialized conversion algorithms with 10 formats (uncompressed included), that's 100 conversion algorithms needed. There is no separate compression and decompression, only combined conversion algorithms. But you do need e.g. conversion from high-bitrate MPEG2 to low-bitrate MPEG2. +1, though, for the N*M rather than N+M argument.

[Matthew Fioravante]

There are some tools around that special-case some (fairly trivial) conversions, though - for example MPEG to VOB conversions that don't decompress-then-recompress the video data, but simply repackage the existing compressed data in the new container file format. It's a nice trick for saving time and avoiding further losses.

[afrazier]

-1: The free and open source x264 encoder is a class-leading encoder, even when compared to commercial encoders.

[afrazier]

There's also the distinct possibility that it might be (and probably is) mathematically impossible to do a lossless transcode from from codec to another (e.g. MPEG-2 to H.264), and anyone interested in doing it would have to have an expert level understanding of both codecs. That's an extremely rare trait, and it'd be a labor of love. Most people would pour their efforts into a single codec, not two (or more).

[afrazier]

Even doing a compressed domain transcode with the same codec (e.g. MPEG-2 to MPEG-2) generally results in worse quality than a full re-encode does. See here.

[surfasb]

Yet different H264 products produce different levels of quality. It's like saying Fedora is the same as Slacker. . .