Video Codecs and Encoding: What You Should Know

Different devices require different codecs in order to stream content. Video encoding is the process of compressing a video file so that it can be delivered across the internet. This compression is done through a codec, which is a two-part tool that allows distributors to condense a video file. 

There are many different codecs available, and each has its own strengths and weaknesses. The best video codec for streaming will depend on the user's specific needs. However, some of the most popular codecs include H.264, HEVC, and VP9.

What is video encoding?

Encoders use video and audio codecs to compress raw video into a more manageable size for delivery. Video encoding is vital when live streaming content, helping to ensure quick delivery and playback.

Many software options are available for video encoding, including Vmix, Wirecast, and OBS Studio. To compress the raw video into a workable size, encoders will use a video and an audio code. This essentially applies an algorithm to your video to shrink the bulky video for better delivery. 

What is a codec?

Different content distributors use different codecs to compress video into a streamable size. This allows for more efficient delivery and storage of video content.

Codecs are computer programs that encode and decode video data. They use algorithms to compress and decompress the video, making it possible to store and transmit large amounts of data efficiently. When streaming video, codecs employ lossy compression, which discards unnecessary data in order to reduce the file size.

Codecs are required to compress and decompress the data when streaming audio and video. H.264, also called AVC (Advanced Video Coding), is a popular video codec, while AAC (Advanced Audio Coding) is a standard audio codec.

H.264 gets its other name, AVC, from its full title: Advanced Video Coding. With so many acronyms in use, it can be challenging to keep them all straight. Streaming codec expert Jan Ozer explains it:

"H.264/AVC and H.265/HEVC have two names because each codec was standardised by both MPEG and the International Telecommunication Union (ITU). Versatile Video Coding, or VVC, is also H.266 for the same reason."

What is a video container format?

Many different codecs and containers are used for streaming audio and video content. Different playback platforms often support different codecs and containers, which is why it is essential to use multi-format encoding when you intend to stream to a variety of different devices.

For example, a .mov file and a .wmv file might contain the same data and codecs. However, the .mov file would be used for playback on Macbook's QuickTime player, whereas a .wmv file will predominantly be used for playback on a PC's window media player. 

Video codecs vs. containers: what's the Difference?

Video codecs are used to compress and decompress video files. This is done through lossy compression, in which any data that isn't required is removed. This allows for smaller file sizes and smoother playback.

A video container format is a type of file that stores video codecs, audio codecs, and metadata. This information is used by programs to playback the video or audio file. Container files typically have a .mp4 or .mkv extension.

Best video codecs for streaming

Different devices require different codecs to play video content from the internet. By having a variety of codecs, better encoding quality and efficiency can be achieved while also supporting playback on older devices.

Here is what Netflix, one of the largest distributors of video streaming around, have to say:

"Netflix says it utilises a deep toolbox of codecs, which can be called upon to stream compatible formats to display devices. Although Netflix continually adds new and improved codecs, it has never abandoned one — it continues to support the VC1 codec it started within the first Netflix streaming device, a 10-year-old LG Blu-ray player."

How can you use this information?

For anyone with a home media server like Plex or Emby, using or encoding video files with the correct codec and container can save on storage space and make content available to stream to all home devices. Using H.264 with AAC codecs in an MP4 or MKV container will allow any device to be able to playback content. The advantage goes to an H.265/HEVC - AAC encodes in an mkv container if your devices all support it. This will support higher compression with less quality loss and subtitles packaged into the container.