Video Codecs

Video codecs work behind the scenes to stream video over the Internet. However, the choice of codec can affect things like resolution and video quality.

Read on to learn more about video codecs, including how they work, the different types available, and how to choose the right one.

What is a Video Codec?

A video codec compresses and decompresses media files like video and audio. They're designed to reduce file sizes and make it easier to store and distribute online videos for viewing. The term "codec" combines "encoder" and "decoder."

Codecs are important for the following reasons:

• They reduce file sizes: Uncompressed 4K video files can be several terabytes. A codec can compress a raw video format into a more manageable size.
• They enable efficient transfers: Transmitting uncompressed files uses a lot of bandwidth while using a codec to compress and transmit files uses less bandwidth.
• They reduce storage costs: By reducing file sizes, codecs can deliver significant savings in data usage and storage costs.
• They improve streaming quality: Some codecs can deliver high-quality streaming at lower bitrates, offering better viewing experiences to end users.

Different types of codecs use different compression techniques with varying levels of quality. This allows content creators and content distributors to choose a codec that suits their needs. We'll cover each of these in a later section, but for now, let's look at how codecs work.

How Do Codecs Work?

Video codecs facilitate the compression and decompression aspects of video streaming.

Here's how these processes work.

Video Encoding

A single video file contains a lot of data — image data for the video frames, audio data for the sound, metadata like the title, and other elements like subtitles. Storing and distributing an uncompressed video as it is would require a considerable amount of disk space and bandwidth.

Video encoding involves compressing the size of raw digital video files and turning them into a more efficient format for distribution. Practically every video streaming platform you can think of — YouTube, Netflix, Hulu — uses an encoder to deliver content to its users.

There are two widely used compression techniques:

• Intraframe: Intraframe compression, also known as spatial compression, compresses each frame in a video individually and looks for any redundancies to reduce data. For example, a blue sky has nearly identical pixel data, so a block of a uniform color can represent those areas to cut down on file size. This compression technique is effective at reducing file sizes while maintaining high image quality.
• Interframe: Interframe compression, also known as temporal compression, uses a more complex technique to reduce file sizes. Instead of compressing each frame individually, it only encodes the differences in subsequent frames. This technique delivers more compression than intraframe compression.

Here's a graphic that illustrates these two compression techniques:

Modern video codecs use a combination of these two compression techniques to reduce file sizes and maintain video quality. 

Video Decoding

Once videos are compressed and transmitted, they have to be decompressed or converted back into their original form to be viewed. The goal of the decoding process is to reproduce the video as close to the original as possible. However, video quality playback can vary depending on the codec used. 

Types of Video Compression

There are two types of video compression: lossy and lossless. Both reduce video file sizes, but they do so in different ways.

Lossy compression

Lossy compression algorithms reduce file sizes by removing certain types of data, especially those less noticeable to the human eye. This method is typically used when the file in question can "afford" to lose some data and when saving on storage space is a priority.

The downside of lossy compression is a loss in video playback quality. However, the trade-off is smaller file sizes and faster transmission rates.

Lossless compression

Lossless compression algorithms reduce file sizes by eliminating redundant data. They can be restored to their original format after being decompressed. This type of video compression means you won't save as much space, but it's ideal for archival purposes.

Here's an image that shows the difference between lossy and lossless compression:

The type of video compression you use will largely depend on your use case. If you need to free up disk space and don't mind lower-quality playback, then opt for lossy compression. If you want to preserve the quality of the original video and don't want to lose any data, then choosing lossless is your best option.

Different Types of Video Codecs

There are numerous types of video codecs to choose from in the below list. Depending on which one you choose, you can expect differences in quality, size, and performance.

Here's a look at popular video codecs, along with the pros and cons of each.

H.264/AVC (Advanced Video Coding)

H.264 or MPEG-4 AVC is one of the most widely used encoding formats, as it enables high-quality streaming at low bitrates — the amount of bits that can be transferred over a period of time. Streaming platforms like Netflix, Hulu, YouTube, and Vimeo use the AVC codec to stream content to their users.

Pros:

• Compatible with a range of devices and online platforms.
• Offers a good balance between streaming quality and compression.
• Supports both lossy and lossless codecs.

Cons:

• Has licensing fees, which can be high depending on how many subscribers a platform has ($0.20 per subscriber for 100,001 to 5,000,000 subscribers).
• Uses more processing power.

H.265/HEVC (High Efficiency Video Coding)

H.265 is the official successor to H.264. It's capable of compressing videos at around half the bitrate of H.264 while maintaining similar video quality, making it ideal for high-resolution streaming. It's also designed to support resolutions up to 8K with a high frame rate.

Pros:

• Offers significant savings in bandwidth and storage.
• Supports videos up to 8K at 300 frames per second (FPS).
• Better image quality over H.264.

Cons:

• Not as widely supported.
• Has high licensing costs.

H.266/VVC (Versatile Video Coding)

H.266 is a new codec developed by the Joint Video Experts Team (JVET). It was released in 2020 as the successor to H.265 and aims to deliver more efficient compression. The previous standard H.265 would require about 10 GB of data to transmit a 90-minute Ultra High Definition (UHD) video, but H.266 can transmit the same video using only 5 GB of data while maintaining the same quality.

Pros:

• Designed to support a range of video qualities and formats.
• Supports future video technologies, like 360-degree video and High Dynamic Range (HDR).
• More efficient for high-resolution video streams.

Cons:

• Requires significant computing resources.
• Has complex licensing fees.

VP8

VP8 is an open-source codec that was developed by On2 Technologies and later acquired by Google. It's designed to provide high-quality compression for video conferencing and streaming applications. It's the default codec used in Web Real-Time Communications (WebRTC) — a protocol that enables audio, video, and text communication between browsers and devices.

Pros:

• Free to use without any licensing fees.
• Provides good streaming quality at low bitrates.
• Supported by many browsers and platforms.

Cons:

• Not as efficient as newer video codecs like VP9.
• Encoding is more CPU-intensive than other codecs.

VP9

VP9 is an open-source video codec developed by Google. The VP9 codec is designed to offer improved compression and super video quality at lower bitrates than VP8.

Pros:

• Supports 4K and 8K resolutions.
• Ideal for mobile streaming.
• Free to use.

Cons:

• Not as widely supported as VP8.
• Requires more power to decode.

AV1

AV1 is an open-source video codec from Alliance of Open Media (AOMedia). It's designed to deliver quality video streaming over the Internet. Netflix is one of the key contributors to this technology and is already rolling it out to its members' TVs.

Pros:

• Open-source and royalty-free video codec.
• Optimized for video streaming over the Internet.
• Supports future technologies, like virtual reality and 8K.

Cons:

• Not as widely adopted as other codecs.
• Requires more processing power.

How to Choose the Right Codec

There are several factors to consider when choosing a codec, from compression efficiency to playback compatibility and licensing terms.

Here's a closer look at these considerations in more detail.

Compression Efficiency

Compression efficiency refers to how well a codec can compress a video without losing critical data. Some codecs have low compression efficiency, which can preserve the original quality of the content but result in larger file sizes. Others have high compression efficiency but at the expense of lower-quality playback. Consider the trade-off between streaming quality and file size when choosing a codec.

Playback Compatibility

Another important consideration is playback compatibility.

Even the most efficient codec won't help if it's incompatible with your users' devices. For example, VP9 is a free codec that offers excellent streaming quality at lower bit rates compared to other codecs. However, it may cause playback issues on certain Apple devices.

To avoid such issues, you'll want to choose a codec that's compatible with a variety of devices. Opting for H.264 is a safe bet, as it's one of the most widely used codecs.

Licensing Terms

Some codecs have licensing fees, while others are open-source. Codecs like H.266 only make sense from a financial standpoint if the potential benefits outweigh the costs. Otherwise, using a royalty-free codec like AV1 may suit your needs better. It's free to use and it's supported on popular platforms, such as Netflix, YouTube, and Amazon.

Best Practices For Using Video Codecs

Whether for streaming or storage purposes, follow these best practices when using a codec.

Adjust the Bitrate and Resolution

Bitrate and resolution affect a video's quality and size. High bitrates deliver better video quality but use more bandwidth and storage. Low bitrates deliver lower video quality but use less bandwidth and storage.

Most codecs allow you to adjust the bitrate and resolution, enabling you to optimize your videos based on your target audience's bandwidth and devices. However, finding the right settings is often a balancing act.

One option is to use adaptive bitrate streaming (ABS) — a method that dynamically adjusts the resolution and bitrate of a video. It involves encoding multiple versions of a video and selecting the "best" one regardless of a user's device or connection.

Test Different Codecs

Different codecs yield different results in terms of video quality and file size. Test different codecs to find the best balance based on your specific requirements. Be sure to also test your video on different devices and browsers to ensure compatibility. 

Stay Updated

New codecs and updates to existing ones are released regularly. Stay informed with the newest developments to ensure you're using the latest video compression technology.

Frequently Asked Questions