*3.1. Adjustable Backlight Extraction Module*

To respect the image content and avoid drawbacks of a single algorithm, we propose a three-step backlight extraction method to determine an optimal backlight, making displayed images perceived vividly. We document the first step in Section 3.1.1 and the second and the third steps in Section 3.1.2.

#### 3.1.1. Base Backlights Extraction

The first step is to extract base backlights. As mentioned above, each single existing backlight extraction method is not enough to adapt to images with diverse characteristics and contents. However, their respective strengths are complementary and compatible. Each of them can be a base backlight from which we absorb advantages. Assuming that *N* is the number of base backlights, the base backlights extraction in Figure 3 is defined as follows.

$$\mathbf{BL}^t = f^t(I) \qquad t = 1, 2, \cdots, N \tag{7}$$

where *f <sup>t</sup>* is the *tth* base backlight algorithm and **BL***<sup>t</sup>* is the *tth* base backlight.

### 3.1.2. Optimal Backlight Selection

Optimal backlight constraint conditions are constructed by all of the base backlights extracted above. One of the base backlights is selected as the target backlight for its ability in reducing power consumption and improving the contrast ratio.

The second step of our method is to adjust the selected target backlight. Specifically, we adjust it based on the segmentation method in [23] and the obtained constraint conditions.

Based on our self-developed LCD-LED dual modulation display [10], the optimal backlight is selected from adjusted backlights by objective evaluation as well as subjective evaluation in the third step.

• Backlight constraint conditions

We change specific values of the target backlight to obtain several adjusted backlights as alternations of the optimal backlight. The change needs to be within the effective range of backlight to prevent the deterioration of image quality. For an image block, we go through all base backlights for its corresponding maximum and minimum values. The maximum and minimum matrices are denoted as **Pmax** and **Pmin**. This process is defined as:

$$\begin{cases} \mathbf{P\_{max}} = \max\left(\mathbf{B}\mathbf{L}^t(m, n)\right) \\ \mathbf{P\_{min}} = \min\left(\mathbf{B}\mathbf{L}^t(m, n)\right) \end{cases} \tag{8}$$

where (*m*, *n*) is the coordinate of each backlight value in the backlight image.

Considering that limited backlight extraction algorithms are used to construct backlight constraint conditions, **Pmax** is increased by 10% with an upper boundary 255 and **Pmin** is decreased by 10% with a lower boundary 0. The adjusted **Pmax** and **Pmin** are represented as **PAmax** and **PAmin**, and they form the constraint conditions for obtaining the optimal backlight.

• Backlight adjustment and optimal backlight selection

Just Noticeable Difference (JND) [24] reflects the sensitivity of human vision. As shown in Figure 4, under different background luminance, JND is different.

**Figure 4.** JND curve.

In image quality evaluation, if the luminance of details is too close to the neighboring pixels, that is, the difference is less than JND, then the details of this image are not well-displayed. In real

display, the background luminance may be changed due to light diffusion between different backlights, leading to the change of JND. Therefore, the image quality in real display may be degraded. To this end, we adjust the target backlight to change the background luminance and select the backlight that presents the details effectively based on display quality as the optimal backlight.

Assume that target backlight is the *kth* base backlight denoted as **BL0**.

$$\mathbf{BL\_0} = f^k(I) \qquad k \in \{1, 2, \dots, \cdot, N\} \tag{9}$$

where *f <sup>k</sup>* is the *kth* base backlight extraction method. Improving the dynamic range is important in local dimming. Hence, for **BL0**, we strengthen the luminance in bright area and weaken it in dark area to improve its dynamic range. Specifically, the bright and the dark area are selected by mean and variance of the backlight image. The process is expressed as Equations (10) and (11).

$$\begin{cases} M = \left(\sum\_{m=1}^{W} \sum\_{n=1}^{H} \mathbf{BL\_{\bullet}}(m, n)\right) \div (W \times H) \\\\ V = \left(\sum\_{m=1}^{W} \sum\_{n=1}^{H} \left(\mathbf{BL\_{\bullet}}(m, n) - M\right)^{2}\right) \div (W \times H) \end{cases} \tag{10}$$
 
$$\begin{cases} P\_{1} = M - V \\ P\_{2} = M + V \end{cases} \tag{11}$$

where *W*, *H*, *M*, and *V* are the width, height, mean, and variance of **BL0**, respectively. *P*<sup>1</sup> and *P*<sup>2</sup> are the breakpoints to partition areas of different luminance. The pixel with luminance less than *P*<sup>1</sup> is considered as dark area and the pixel with luminance larger than *P*<sup>2</sup> is considered as bright area,

Since the backlight value ranges in 0–255, we use the exponent of 2 as the adjustment step to adjust the target backlight **BL0**. The process is expressed as Equation (12).

$$\begin{cases} \mathbf{BL}\_{i}\left(m,n\right) = \begin{cases} \mathbf{BL}\_{\mathbf{0}}\left(m,n\right) - 2^{i} & \mathbf{BL}\_{\mathbf{0}}\left(m,n\right) < P\_{1} \\ \mathbf{BL}\_{\mathbf{0}}\left(m,n\right) + 2^{i} & \mathbf{BL}\_{\mathbf{0}}\left(m,n\right) > P\_{2} \end{cases} \\\ \mathbf{BL}\_{i}\left(m,n\right) = \begin{cases} \min\left(\mathbf{PA\_{\mathbf{max}}}\left(m,n\right), \mathbf{BL}\_{i}\left(m,n\right)\right) & \mathbf{BL}\_{i}\left(m,n\right) > \mathbf{PA\_{max}}\left(m,n\right) \\ \max\left(\mathbf{PA\_{\mathbf{min}}}\left(m,n\right), \mathbf{BL}\_{i}\left(m,n\right)\right) & \mathbf{BL}\_{i}\left(m,n\right) < \mathbf{PA\_{min}}\left(m,n\right) \end{cases} \end{cases} \tag{12}$$

where *i* = 1, 2, ··· ,8, **BL***<sup>i</sup>* means the *ith* adjusted backlight based on **BL0**. **PAmax** and **PAmin** are used to prevent the adjusted backlight from making poor display quality.

Contrast Ratio (CR) and Dynamic Range (DR) are two objective indicators of display quality. Both reflect the change of brightness ranging from dark to bright, and the higher are CR and DR, the wider is the brightness range. To select the optimal backlight, the display luminance is first measured under all adjusted backlights. Then, CR and DR of each measured luminance are calculated. The adjusted backlights with the top three performances based on the results of CR and DR are selected. Finally, the optimal backlight is selected from the three adjusted backlights by display quality subjectively. The process is shown in Figure 5.

The subjective evaluation is set as follows. The three selected adjusted backlights are demonstrated on the display prototype. Observers are asked to vote for the optimal backlight based on visual perception over details, contrast, and brightness. The backlight with the largest number of votes is determined to be the optimal backlight. Given that subjective feeling is susceptible to factors such as gender, age, occupation, and surroundings, the selection was done by 16 observers who are non-experts in image and video processing field. Their ages range from 22 to 30, with eight males and eight females. All of them have normal visual ability, that is, none of them have eye problems such as color blindness, color weakness, shortsighted, etc.

**Figure 5.** Process of selecting the optimal backlight: (**a**) objective evaluation using Contrast Ratio (CR) and Dynamic Range (DR); and (**b**) subjective evaluation via voting based on display quality. *Ai* (i = 1, 2 ··· 8) means adjusted backlight; *Sj* (j = 1, 2, 3) means backlights with top three objective indicators.
