What Is Video Encoding?

Video encoding is the conversion of raw video data into a new, compressed format. Raw video files are uncompressed or minimally compressed, and so they are not optimal for storage, transfer, streaming, or other distribution. Typically, a raw video file is very large in size due to its high-quality definition. Video encoding seeks to retain as much of the visual quality of a raw video file as possible while shrinking the file size, by various compression techniques, into standardized formats.

Video encoding is central to all online video streaming, playback, and broadcasting, as it allows high-quality files to transmit rapidly due to their smaller file sizes. Encoding a video also allows for adaptation into the correct file format and thus delivery method. Here are a number of reasons why video encoding is important.

Bandwidth optimization

A raw video file can be very large, with high-definition videos exceeding hundreds of gigabytes to terabytes in file size for a single hour of playback. If an organization were to transmit a file of that size, it would consume a large portion of bandwidth, creating a strain on all other services sharing the network connection.

Video encoding solves this issue by significantly reducing a video file’s total size. When it comes time to transmit the video, the smaller size enables smoother transmission, reducing buffering or dropped frames. Advanced encoding techniques, such as variable bitrate (VBR) encoding, can further reduce the required bandwidth usage by strategically allocating bits.

Storage efficiency

Raw files cause problems when you need to store them. High-definition video footage, when uncompressed or minimally compressed, can require substantial amounts of on-premises or cloud storage, creating high costs for organizations. After compressing a video with video encoding, its smaller file size makes it much easier and less costly to store.

For digital services that rely on delivering videos to end users (like video streaming platforms like YouTube or Netflix), storage efficiency in their content delivery networks significantly lowers operational costs.

Device compatibility

A video file might need to play on various end devices, ranging from smartphones and tablets to desktop computers and TVs. Each of these devices has a different total capability for processing, as well as distinct supported video file formats. Encoding videos ensures they are compatible with these devices, transcoding raw files into highly standardized formats and codecs that most devices will support.

Quality control

Although many of the benefits of video encoding stem from the size changes, the process also gives businesses granular control over the video’s quality parameters. Frame rate, color depth, compression ratio, and resolution are all elements that you can configure when encoding a video. For quality control and ensuring the final product has a high degree of visual fidelity to the raw version, these granular options are useful.

Video encoding characteristics determine a video file’s compatibility, quality, and playback efficiency.

Video codec selection

Different video codecs, such as H.264 (Advanced Video Coding), H.265 (High Efficiency Video Coding), and VP9, all offer different levels of efficiency, device support, and processing demand. A video encoder compresses video and audio data from raw video files into a usable digital format, based on the selected codec.

Container formats

Container formats define how encoded streams are packaged into the end product video file. The vast majority of common video formats will store compressed metadata, subtitles, audio, and video data together. The selected container, such as MP4 or MKV, dictates which systems can play the digital video files.

Compression ratios

Compression ratios are how much the encoder reduces the size of the video. Higher ratios use stronger video compression to minimize file size, but can impact quality in doing so. Different video encoding formats will attempt to balance this trade-off by using algorithms to increase visual fidelity between the final product and the original.

Quality metrics

Quality metrics, like PSNR (Peak Signal to Noise Ratio), SSIM (Structural Similarity Index Measure), and VMAF (Video Multi-Assessment Method Fusion), all evaluate whether the final encoded video matches the source material. They aim to quantify the quality of the encoding process, helping content engineers to create better final versions in the future.

There are three main types of video encoding, each offering a different advantage in terms of visual quality, source data handling, and storage size.

Lossless encoding

Lossless encoding is a video encoding strategy that offers the highest possible fidelity to the source data. Every single bit of the original video data is identical in the output, making this the chosen strategy for professional video production companies where visual details are extremely important. Scientific imaging also uses lossless encoding to keep the final images accurate and precise.

Lossless encoding uses strategies such as predictive coding, entropy coding, and reversible compression algorithms to compress data without degrading quality. The main downside to a lossless strategy is that its final compression ratios are more limited.

Lossy encoding

Lossy encoding discards less important visual data to reduce the file size. While the final output is visually similar to the source data, the discarded elements mean that it won’t be as high-quality. Typically, what lossy encoding lacks in precision it makes up for in compression, significantly decreasing file size.

Lossy encoding uses strategies such as transform coding, quantization, and inter-frame compression to reduce redundant data and reduce precision in areas that are less visually important. Many consumer and streaming video codecs (H.264/AVC, AV1, and H.265/HEVC) stem from lossy encoding.

The rise of optimization techniques such as content-adaptive quantization and psychovisual rate-distortion has helped make lossy encoding more feasible by minimizing visual degradation using this strategy.

Adaptive bitrate encoding

Adaptive bitrate (ABR) encoding creates a range of different video resolutions and bitrates that the user can select. Based on real-time factors such as how much bandwidth a device has to consume or buffer health, the user can select a higher or lower output on the ABR ladder.

By producing multiple bitrate outputs (240p at 300 kbps, 480p at 1 Mbps, 1080p at 10 Mbps), users can adapt the video quality while watching to better match their connection in that moment. Many services offer dynamic adaptation, where the service monitors total throughput and automatically switches to a more optimal version of a video during playback.

Here are the main stages that a video goes through during the encoding process.

Pre-processing

The pre-processing stage prepares the original video data for compression. It cleans up the raw video data by reducing noise, scaling the resolution, adjusting the frame rate, and stabilizing the image. Pre-processing prepares a consistent input for the following stages.

Compression

The compression stage is where the majority of the encoding process occurs, including frame analysis, motion estimation, spatial and temporal compression, and entropy coding. These are covered in the following section. The final product of this stage is a compact bitstream that reflects the chosen video codec settings.

Post-processing

The post-processing phase aims to further refine the visual quality to meet any other technical restraints a video may have. For example, post-processing could include adding watermarks or subtitles, as well as adding metadata to the file.

Packaging

Packaging organizes the encoded output into a digital format that is easy to deliver and play on devices. The compressed video stream is placed into standardized video formats, such as MP4, which structures audio, video, subtitles, and metadata in a single container.

Video encoding is a process that analyzes video frames, reduces redundancy, compresses temporal and spatial information, and then packages the video into a compact bitstream.

Firstly, an encoder performs frame analysis. Frame analysis identifies I-frames (intra-coded), P-frames (predicted), and B-frames (bi-directionally predicted), and then divides them into blocks. I-frames store a complete frame, whereas P-frames store the difference from the previous frame, and B-frames store differences between the previous and next frames. By evaluating scene changes, texture complexity, and motion intensity, the encoder can decide how significantly it can compress each region while retaining visual quality.

Next, the encoder uses motion estimation, comparing blocks in the current frame to blocks in reference frames to see how things have moved. By contrasting the two, it can calculate the motion vectors of a video, predicting most of the frame’s content with high accuracy. Knowing the vector allows the encoder to avoid re-encoding full blocks of frames, needing only to transmit the residual data from the prediction error.

Spatial compression then compresses the prediction error of each block. For example, the encoder can transform this residual data from pixel space into frequency space using a Discrete Cosine Transform. DCT reduces a block of pixels into frequency coefficients, which can then be quantized, reducing any components that contribute less to visual detail. Afterwards, spatial compression reduces any redundancy within a block, shrinking data size.

After spatial compression, encoders use temporal compression to minimize the total data stored across frames even more. Temporal compression encodes only the differences between predicted and actual content across frames, using the motion vectors to avoid re-encoding unchanged regions.

Finally, after these two forms of compression, an encoder uses lossless entropy coding to translate all of that information into a compact, codified bitstream. Entropy coding methods include Context-adaptive variable-length coding (CAVLC) for H.264. The motion information, block segmentation decisions, and quantized coefficients are all packaged. The encoder uses binary codes to compress the file to its maximum, without losing any of the information.

The final stage produces an encoded output that is ready for efficient transmission or storage.

Video encoding capabilities are the modifications that allow you to tailor the final video file you produce to meet more specific requirements.

Format conversion and encoding format

Format conversion lets you re-encode a video from one video codec to another, which allows the video to play on different devices. For example, you could convert a legacy source into a more efficient codec such as H.264.

Bitrate optimization

Bitrate optimization changes adjust the total amount of data allocated per second of video. These optimizations are made to balance video quality and bandwidth consumption. Encoders can use a few strategies here, including constant bitrate (CBR), variable bitrate (VBR), and content-adaptive bitrate to maintain quality while reducing wasted bits.

Resolution scaling and file size

Resolution scaling is a strategy that allows you to resize video frames to lower or higher resolutions. By downscaling resolution, you can decrease file size, reducing bandwidth consumption. Alternatively, upscaling resolution prepares a video for a higher-definition playback setting.

Frame rate adjustment

Adjusting the frame rate changes the number of frames per second (FPS) to match display requirements. Most of the time, you adjust the frame rate to reduce bandwidth or improve the smoothness of the video. Alternatively, some devices or regional standards might need a certain FPS.

Here are some of the key considerations when performing video encoding.

Source quality requirements

The quality of the source video directly influences the final output. If you feed a video encoder a high-quality video, it has more detail to preserve. This can maintain a high video quality even after using strong video compression or preparing it for multiple video encoding formats.

Target delivery platform

Different target systems have different needs when it comes to video formats, compression levels, and audio and video codecs. Streaming platforms, for example, use H.264 and H.265 most frequently, which can influence your selection when you encode video for these systems.

Bandwidth constraints

When planning distribution, you must consider the viewer’s available bandwidth. If bandwidth is limited, you can opt for adaptive bitrate streaming to adjust quality levels automatically.

Processing resources

Different codecs and settings influence how fast the encoder runs. Powerful editing programs or specialized video encoding software can handle more advanced compression, improving your ability to downsize video files.

Latency tolerance

Many live video use cases, such as in live streaming, online gaming, or video conferencing, require low-latency video. Here, you might opt for simplified compression tools to provide the speed needed for your specific use case.

AWS offers a range of solutions to help with your video coding, streaming, and broadcasting, all with adjustable parameters to fit your needs. Explore the AWS Elemental suite:

AWS Elemental MediaConvert helps you process video files to prepare on-demand content for distribution or archiving. AWS Elemental MediaConvert uses cloud-based transcoding for you to produce high-quality video.

AWS Elemental MediaLive allows you to encode live video for broadcast and streaming to any device. With AWS Elemental MediaLive, you can produce broadcast-quality live streams with high availability.

Get started with video encoding on AWS by creating a free account today.