Introduction to the x264 Loop Filter Module

H264 Loop Filtering

H.264 loop filtering is a key technique used in video coding to minimize block effects and improve video quality. It is mainly performed on the encoding side and must be performed in the decoding part as well, and is an integral part of the video coding scheme. Loop filtering consists of various cases as follows:

1. Four cases of loop filtering: In the Baseline configuration, loop filtering may include the following four scenarios :

   • Bs = 4: Either side on the macroblock boundary is an intraframe prediction.
   • Bs = 3: Either side on the macroblock boundary is an intraframe prediction, or an intraframe prediction exists on the block boundary.
   • Bs = 2: Both sides are inter-frame predicted and the macroblock on either side is coded.
   • Bs = 1: Both sides are inter-frame predicted, no macroblocks are encoded and different reference pictures are used, or the MV component on either side is not less than 4.
   • Bs = 0: Otherwise, no filtering is required.

2. filtering process: The filtering process consists of horizontal and vertical macroblock filtering. The four boundaries of each macroblock are divided into four equal parts, each corresponding to a boundary of minimum block size, and filtering is performed as a unit. The length of each boundary is 16 pixels .

3. Causes of the block effect: The block effect is mainly caused by two factors :

   • Block-based intra- and inter-frame prediction residuals are integer transformed and quantized, and the discrete nature of the quantization operation leads to errors when recovering the transformed coefficients by inverse quantization, creating visual discontinuities especially at image boundaries.
   • During motion compensation, even the most matched blocks may have minor inconsistencies due to interpolation of sample points from different locations in different frames, resulting in discontinuities in the boundary data of the replicated blocks.

4. Purpose of loop filtering: The purpose of loop filtering is to eliminate the block effect introduced due to the coding process and to improve the smoothness and visual quality of the video by smoothing the image boundaries .

5. Distinguishing between real and fake borders: Before performing loop filtering, it is necessary to distinguish between true and false boundaries. True boundaries are features of the image itself, while false boundaries are caused by quantization and motion compensation during the coding process.The H.264 standard defines two variables, α and β, to decide whether the boundaries need to be loop filtered or not.

6. filtering operation: H.264 uses different strengths of filters depending on the boundary strength Bs. A weaker filter is used for Bs = 1, 2, 3, while a stronger or weaker filter can be selected for Bs = 4, depending on whether there is a lot of detail information.

7. Adaptive filter strengthThe filtering strength can be adjusted according to the quantization parameter QP: the larger the QP, the larger the values of α and β, which means the larger the quantization error and the more obvious the block effect, and therefore the threshold value should be taken as a larger value to increase the filtering effect.

8. Filter complexity: In practical implementations, the computational complexity of the deblocking filter is very high and may take up a significant percentage of the decoding process .

By these measures, H.264 loop filtering can effectively reduce the block effect of the encoded video and improve the video quality.

x264 Loop Filtering

Relevant coding parameters

• b_deblocking_filter
• i_deblocking_filter_alphac0
• i_deblocking_filter_beta

Encoding Parameter Setting

1. Default:

param->b_deblocking_filter = 1;
param->i_deblocking_filter_alphac0 = 0;
param->i_deblocking_filter_beta = 0;

• 1
• 2
• 3

2. preset= ultrafast

param->b_deblocking_filter = 0;

• 1

3. tune= film

param->i_deblocking_filter_alphac0 = -1;
param->i_deblocking_filter_beta = -1;

• 1
• 2

4. tune=animation

param->i_deblocking_filter_alphac0 = 1;
param->i_deblocking_filter_beta = 1;

• 1
• 2

5. tune = grain

param->i_deblocking_filter_alphac0 = -2;
param->i_deblocking_filter_beta = -2;

• 1
• 2

6. tune = stillimage

param->i_deblocking_filter_alphac0 = -3;
param->i_deblocking_filter_beta = -3;

• 1
• 2

7. tune = fastdecode

param->b_deblocking_filter = 0;

• 1

8. tune= touhou

param->i_deblocking_filter_alphac0 = -1;
param->i_deblocking_filter_beta = -1;

• 1
• 2

Loop de-blocking related setups

1. existvalidate_parametersfunction will limit the encoded parameters to a reasonable range:

h->param.i_deblocking_filter_alphac0 = x264_clip3( h->param.i_deblocking_filter_alphac0, -6, 6 );
h->param.i_deblocking_filter_beta    = x264_clip3( h->param.i_deblocking_filter_beta, -6, 6 );

• 1
• 2

2. existslice_header_initThe function will convert the encoding parameters into the x264_slice_header_t structure:

    int deblock_thresh = i_qp + 2 * X264_MIN(param->i_deblocking_filter_alphac0, param->i_deblocking_filter_beta);
    /* If effective qp <= 15, deblocking would have no effect anyway */
    if( param->b_deblocking_filter && (h->mb.b_variable_qp || 15 < deblock_thresh ) )
        sh->i_disable_deblocking_filter_idc = param->b_sliced_threads ? 2 : 0;
    else
        sh->i_disable_deblocking_filter_idc = 1;
    sh->i_alpha_c0_offset = param->i_deblocking_filter_alphac0 * 2;
    sh->i_beta_offset = param->i_deblocking_filter_beta * 2;

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8

3. existslice_header_writefunction writes the relevant information into the stream header

    if( sh->pps->b_deblocking_filter_control )
    {
        bs_write_ue( s, sh->i_disable_deblocking_filter_idc );
        if( sh->i_disable_deblocking_filter_idc != 1 )
        {
            bs_write_se( s, sh->i_alpha_c0_offset >> 1 );
            bs_write_se( s, sh->i_beta_offset >> 1 );
        }
    }

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9

4. About b_deblocking_filter_control
   • existx264_pps_initThe value 1 is assigned in the function:pps->b_deblocking_filter_control = 1; [What is the significance of this variable being assigned directly to 1 and not changing in between? 🤔]
   • existx264_pps_writefunction writes it to the stream:bs_write1( s, pps->b_deblocking_filter_control );

loop filteringfunction (math.)relationship diagram

Introduction to Loop Filtering Core Functions

1. fdec_filter_row·: Responsible for filtering a line of macroblocks in the x264 encoder. This function accomplishes the following three main steps:
   • Loop filtering (de-blocking filtering): by calling thex264_frame_deblock_row() function implementation for smoothing macroblock boundaries and reducing visual discontinuities.
   • Half-pixel interpolation: by calling thex264_frame_filter() function implementation for generating reference frames with half-pixel precision to improve coding efficiency.
   • Calculation of video quality SSIM and PSNR: The PSNR is created by calling thex264_pixel_ssd_wxh() function is implemented, and SSIM is computed using thex264_pixel_ssim_wxh() function implementation .
2. x264_macroblock_deblock_strength·: The function serves to calculate the strength of the deblocking filter. In the H.264/AVC video coding standard, the deblocking filter is used to minimize discontinuities between macroblocks, known as the block effect. This effect is usually introduced during the coding process due to the independent encoding and quantization of blocks.x264_macroblock_deblock_strength The function determines the strength parameters of the filter by analyzing the characteristics of the macroblocks, which are subsequently used in the deblocking filtering process.
3. x264_macroblock_deblock Function: When RDO is turned on, it is called when calculating the rd cost in each macroblock analysis.