The Video Compression Cheat Sheet

If you’ve ever produced a digital video of any kind, you’ll know the feeling of satisfaction you get from finally completing a great-looking film. And most likely, you’ll also know the feeling of intimidation, frustration and/or seething rage when first confronted with the huge selection of video render settings.

I hear you: ‘All I want is to get my video out of the editing program looking as good as it does inside the program so why do I need to know if it’s lossy or lossless and what’s a H.264 and why can’t I click a button and it just goes?’

Search for answers on online forums and the prevailing response will inevitably be along the lines of No single setting works best in all situations. And while this is indeed true, it’s about as helpful as a Slushie machine on the Titanic. Conversely, an in-depth description of every codec available in today’s technology would only confuse matters. That’s why today we’re taking a brief look at some of the more popular codecs and formats used in compressing digital video, and the recommended uses for each.

h.264

One of the most popular video compression codecs at this point in time is H.264, a lossy codec which exports to an .mp4 or .mov file container. Where H.264 shines is its ability to significantly cut down the final file size of video files with comparatively minimal quality loss. Because of this, H.264 has become a go-to format for final exports and file sharing.

Settings for this file type include an adjustable bit-rate which (depending on the visual complexity of your video) will determine fine quality and export times. While generally offering a solid compromise between file size and visual quality, the lossy nature of H.264 isn’t ideal for exporting working files between programs. Each time a video file is lossy-compressed it degrades the visual performance. Transferring video between post-production software in this way causes compound quality loss, leaving us with a far inferior image quality than what we originally started with.

H.264’s lossy compression also tends to cause a ‘banding’ effect on gradients and soft lighting effects if the bit-rate and BPC (bits per channel) settings aren’t high enough. Furthermore, the colour and lighting of cel-shaded 2D animation will respond differently to H.264’s compression than live-action video, so it’s important to play around with the export settings to find what works best for your output needs.

animation codec

The first thing one might think upon seeing the animation codec in the quicktime export settings list is ‘Ah! Surely this is the perfect setting for animation and nothing else!’ Now, hold up there Assumptions McGee. If you’ve worked with digital video before you’ll know by now that things are rarely that straightforward.

Being a lossless format the animation codec is most advantageous in mid-workflow, where video files need to be transferred between different post-production softwares with as little quality degradation as possible (say, exporting an animated scene from TV Paint and importing into After Effects to add VFX). Saving a file with the animation codec keeps both visuals and audio in the same condition they left in, ready for further editing/effects in the next program.

Using the animation codec for sharing/playback of your finished product is an entirely different story. Due to the lossless compression, file sizes for this format are always monstrously huge (in the realm of several gigabytes for even a two-minute video file). Additionally, general playback is prone to being slow and jittery and uploading to video sharing sites in this format is extremely inefficient. For these reasons, the animation codec is not built for video sharing but for enabling the transfer of video content during the production pipeline.

prores

ProRes is a lossy format which currently comes in six different flavours, all of which support the current major frame sizes including SD, HD, 2K and 4K. What separates all the ProRes variations is bit-rate, ranging from ProRes 4444 (minimal compression, larger file size) to ProRes 422 Proxy (heavy compression, smaller file size).

In terms of both file size and visual degradation, ProRes sits somewhere in the middle between lossless compression and highly compressed lossy files. File sizes will generally be much smaller than uncompressed or lossless codecs, but with a visual quality much closer to lossless than highly compressed formats like H.264. Similarly to the Animation codec, ProRes is best used for intermediate compression of working files rather than the end-product viewing.

image sequences

Image sequences come in a number of different file formats (jpeg, png, targa, tiff), and can be lossy or lossless depending on the export settings. As the name would suggest, this compression method exports your animation as a series of image files rather than a single video file.

‘Ah!’ I hear you say, ‘Surely a series of images would be inferior to a video!’ Now just a second there, Judgey McJudy. While initially this might sound useless, there are a number of unique reasons to use image sequences as an intermediate file export type.

Foremost, it’s much safer to export large lossless files as image sequences than as single video files (like with the animation codec). Why? Because large, high-quality video files take a long time to render. Short films routinely take hours or days, while movie-length files can take months to render across multiple computers. If you’re exporting to a single video file and an error or even something as simple as a power outage interrupts the rendering process, you’re left to re-render the entire video file all over again. But encounter an error in the middle of rendering an image sequence? No worries. Because each frame is a separate image file you can simply re-render those few corrupted frames, or onward from the point of interruption.

On the flipside, by very nature of their format image sequences are entirely unsuitable for final compression. Firstly, you’ll be hard-pressed to find any video-sharing website or standard media player that accepts image sequences as a video format. And secondly, image sequences contain no sound file. Sound must be added during post-processing (as is often the case with film).
For these reasons, image sequences are ideal as an intermediate export format for moving working files from production to post-production but not designed for final export.

conclusion

‘Ah!’ I hear you say again, ‘Surely there are many more file formats and codecs to discuss which may be suitable for compressing my video.’ Indeed there are, Captain McObvious. There are a range of other compression formats, some which offer similar functions to those listed above, others which operate for a specific pixel ratio (such as DV Cam) or purpose (such as MPEG-2, commonly known as the ‘DVD format’). The best way to learn about the effects of each is to have a play around with the settings and do some test renders, examining the visual differences of each one.

In any case, the main things to consider when choosing your export settings and compression method are 1) your intended use for the exported video (finished product or moving between programs) and 2) your desired specifications, if any (quality level, dimensions, aspect ratio). These two criteria will ultimately inform your decision of which export format to go with.