**3. Method**

In this section, we document our local dimming method elaborately. We first introduce the holistic structure of the method. Then, the proposed backlight extraction method and the compensation method are described, respectively.

The diagram of the proposed method is shown in Figure 3. The whole architecture consists of two modules: an Adjustable Backlight Extraction (ABE) module and a pixel compensation module. Furthermore, the ABE module consists of base backlights extraction and optimal backlight selection.

**Figure 3.** The diagram of the proposed method. It is better to look at it in color: orange block, adjustable backlight extraction module; white block, backlight smoothing module; and pink block, pixel compensation module.

To avoid color distortion, most of the existing local dimming algorithms are performed on the luminance rather than the chroma component. Following this, we separate the luminance information by converting the color space from *RGB* into *YCbCr* [19] before all of the succeeding operations. The conversion formula is

$$\begin{cases} \begin{bmatrix} \mathbf{Y} & \mathbf{C}\mathbf{b} & \mathbf{C}\mathbf{r} \end{bmatrix}^{T} = \mathbf{M}\_{\mathbf{s}\mathbf{R}\mathbf{G}\mathbf{B}} \times \begin{bmatrix} \mathbf{R} & \mathbf{G} & \mathbf{B} \end{bmatrix}^{T} + \begin{bmatrix} 16 & 128 & 128 \end{bmatrix}^{T} \\\\ \mathbf{M}\_{\mathbf{s}\mathbf{R}\mathbf{G}\mathbf{B}} = \begin{bmatrix} 0.257 & 0.564 & 0.098 \\ -0.148 & -0.291 & 0.439 \\ 0.439 & -0.368 & -0.071 \end{bmatrix} \end{cases} \tag{1}$$

According to the Retinex theory [20], which is widely used in image processing [21], an image is composed of reflectance and illuminance. The former presents the detail information and the latter determines the dynamic range. This is represented as:

$$\mathbf{S}(\mathbf{x}, y) = \mathbf{R}\_{\mathbf{c}}(\mathbf{x}, y) \times \mathbf{I}(\mathbf{x}, y) \tag{2}$$

where (*x*, *y*) is the coordinate of the pixel in the image, **S** is the image perceived by human eyes, **Rc** is the reflectance, and **I** is the illuminance. In our method, **Y** component in *YCbCr* color space is considered as **S**. **I** is obtained by:

$$\mathbf{I}(\mathbf{x}, y) = \mathbf{F}(\mathbf{x}, y) \otimes \mathbf{Y}(\mathbf{x}, y) \tag{3}$$

where **F** is the Weighted Least Squares (WLS) filter [22], which is known as an edge-preserving filter; ⊗ is the convolution operation; and **Y** is the **Y** component in *YCbCr* color space.

Image edge is the most concentrated part of image information such as the change of gray level and the mutation of texture structure, which contains rich details. Therefore, the edge information must be decomposed and kept to improve image quality in image processing. An alternative filter is bilateral filtering, which has been used in many previous works as a base-detail decomposition technique. However, WLS filter is chosen in this paper because of its better performance, especially for increased blur level compared with bilateral filtering. WLS filter is well suited for progressive coarsening of images and for multi-scale detail extraction. For an input image *g*, an image *u* is expected to be as close to *g* as possible and be smoother except for some places where the gradient of the edge of *g* changes greatly. Formally, the solution to minimize the objective function in Equation (4) is the result of filtering *u*.

$$\sum\_{p} \left( \left( u\_p - g\_p \right)^2 + \lambda \left( a\_{x, p(\mathfrak{g})} \left( \frac{\partial u}{\partial \mathbf{x}} \right)^2 + a\_{y, p(\mathfrak{g})} \left( \frac{\partial u}{\partial y} \right)^2 \right) \right) \tag{4}$$

where the subscript *p* represents the coordinate of the pixel. The first term *up* − *gp* <sup>2</sup> is used to measure the similarity between *g* and *u*. The second term is a regular term, and *λ* is a weight coefficient of the regular term. The larger *λ* is, the smoother the image will be. The image *g* is smoothed by minimizing the partial derivative of *u*, and the weights of smoothing terms are *ax*,*p*(*g*) and *ay*,*p*(*g*), respectively. The definitions of *ax*,*p*(*g*) and *ay*,*p*(*g*) are shown in Equation (5).

$$\begin{cases} a\_{x,p(\mathcal{J})} = \left( \left| \frac{\partial l}{\partial x} \left( p \right) \right|^a + \epsilon \right)^{-1} \\\\ a\_{\mathcal{Y},p(\mathcal{J})} = \left( \left| \frac{\partial l}{\partial \mathcal{Y}} \left( p \right) \right|^a + \epsilon \right)^{-1} \end{cases} \tag{5}$$

where *l* is the logarithmic transformation of *g*, the exponential parameter *α* is used to determine the gradient sensitivity, and  is the offset for avoiding invalid division when *<sup>∂</sup><sup>l</sup> <sup>∂</sup><sup>x</sup>* (*p*) or *<sup>∂</sup><sup>l</sup> <sup>∂</sup><sup>y</sup>* (*p*) is zero. From the above equations, *ax*,*p*(*g*) and *ay*,*p*(*g*) decrease when the gradient of *l* increases, by which the edge information is kept and unnecessary details is smoothed.

Backlight extraction and pixel compensation are processed on the illuminance **I**. Then, the logarithm of **Rc** is defined in Equation (6).

$$\mathbf{r}(\mathbf{x}, y) = \log \left( \mathbf{R}\_{\mathbf{f}} \left( \mathbf{x}, y \right) \right) \\ = \log \left( \mathbf{Y} \left( \mathbf{x}, y \right) \right) - \log \left( \mathbf{I} \left( \mathbf{x}, y \right) \right) \tag{6}$$

where **r** is the logarithmic result of **Rc**, and it is used in IBHE in Section 3.2.
