Double-Exposure Algorithm: A Powerful Approach to Address the Accuracy Issues of Fractional Vegetation Extraction under Shadow Conditions

Featured Application

Accuracy verification of large-scale remote sensing fractional vegetation cover products; precision agriculture; ecological research

Abstract

When recording the vegetation distribution with a camera, shadows can form due to factors like camera angle and direct sunlight. These shadows result in the loss of pixel information and texture details, significantly reducing the accuracy of fractional vegetation coverage (FVC) extraction. To address this issue, this study proposes an efficient double-exposure algorithm. The method reconstructs the pixel information in shadow areas by fusing normal-exposure and overexposed images. This approach overcomes the limitations of the camera’s dynamic range in capturing pixel information in shadowed regions. The study evaluates images with five levels of overexposure combined with five vegetation extraction indices. The aim is to determine the best-performing double-exposure combination under shadow conditions and the most suitable vegetation index. Experimental results reveal that the R² value between the best vegetation index and the FVC calculated from the fused double-exposure images and the ground truth FVC increases from 0.750 to 0.969. The root mean square error (RMSE) reduces from 0.146 to 0.046, and the intersection over union (IOU) increases from 0.856 to 0.943. These results demonstrate the excellent vegetation extraction capability of the double-exposure algorithm under shadow conditions, offering a straightforward and effective solution to low accuracy of FVC in shadowed areas.

1. Introduction

Vegetation is a crucial part of natural ecosystems and an indispensable natural resource [1]. The root system of vegetation is closely related to the soil, and vegetation branches, stems, leaves, etc., are closely related to soil erosion [2]. Photosynthesis, the conversion from inorganic into organic matter, is a crucial process in maintaining the Earth’s carbon–oxygen balance [3]. Transpiration amplifies the circulation of water, which is a vital element of life in ecosystems [4]. Therefore, changes in the vegetation type and quality can notably affect the cycle and profile of the whole ecosystem. The fractional vegetation cover (FVC), a critical parameter characterizing vegetation features, is important for the study of ecology, hydrology, and meteorology and in other aspects. It is often used as fundamental data for research on regional or globalization issues [5,6]. Consequently, FVC investigations are important.

The FVC is usually defined as the vertical projection of a vegetation canopy area to the ground and can be expressed as a fraction or percentage of the reference area [7]. Within the context of remote sensing, the FVC can be defined as the percentage of the green vegetation area to the total observed area [8]. The FVC is an important phenotypic factor in agriculture, forestry, and ecology and an important indicator for the study of changes in each circle of the entire ecosphere and their interactions. Moreover, the FVC is a key biophysical parameter for capturing horizontal exchange at the surface-atmosphere boundary using soil, vegetation, and atmospheric transfer models [9,10,11].

Considering that the FVC is a vital indicator, it is important to measure it at different scales for different applications [12]. For example, the remote sensing technique has provided an important data source for large-scale FVC studies for decades due to loaded positions and greater ease of efficient, large-scale monitoring [13,14]. The first commonly used method is the vegetation index-based method. It aims to extract the FVC with the help of various vegetation indices, such as the normalized difference vegetation index (NDVI), perpendicular vegetation index (PVI), soil adjusted vegetation index (SAVI), modified soil adjusted vegetation index (MSAVI), and transformed soil adjusted vegetation index (TSAVI) [15,16]. Through the different vegetation and non-vegetation characteristics determined by these vegetation indices, thresholds obtained from various algorithms can be used to distinguish the vegetation cover from the background. Two aspects must be investigated in this method: one is the determination of a suitable vegetation index for different application scenarios, and the other is the establishment of a suitable algorithm to obtain a fixed or self-adaptive threshold for vegetation extraction. Using the vegetation index-based method, Bac et al. produced a map of the extent of mangroves in the Philippines using Google Earth MVI [17]. Another commonly used technique is the regression method. This method involves first establishing a mathematical regression model based on several data sources and the FVC over a relatively small range of sample points and then generalizing the results to the entire study area [18]. Currently, with the rapid development of computer technology, machine learning [19,20,21,22] and deep learning [23,24,25] methods are emerging.

Large-scale remote sensing FVC measurements require more accurate ground data at smaller scales as validation data, which can be obtained from traditional small-scale FVC measurements [26]. The validation dataset strives to meet the following objectives: (1) the instrument should be economical, practical, and easy to operate; (2) the instrument should capture valid, accurate, and objective ground observations; (3) the measurement duration should be short; and (4) the influence of human factors should be minimized [27,28]. The most traditional on-site FVC measurement method is the visual interpretation method, which can be divided into the direct visual estimation method [29] and the photo visual estimation method [30]. However, the visual estimation method is limited by subjectivity, an extreme dependence on observer experience, and a tendency to produce large errors. Therefore, various sampling methods, such as the point measurement method [31], sample strip method [1], shade method [32], and square viewpoint frame method [33], have been proposed for use in this area. The point measurement method entails the use of long, sharp needles stuck vertically in the sample plot, and the FVC is determined using the ratio of the number of needles passing through vegetation to the total number of needles. The shade method (ruler measurement) utilizes the relationship between a given scale and the dark shadows produced by vegetation to measure the FVC. However, the square viewpoint frame method considers the gaps in the small holes of the frame to obtain a sample within the study area. The quadrat method reduces the workload when increasing the observation accuracy due to the application of statistical principles. Relevant measurement instruments have also been developed to obtain FVC data via spatial quantitative meters and moving light meters. With the rapid development of large-scale integrated circuits, the photography technique is no longer an expensive, complex, or time-consuming option, as in film era, digital photography and digital image processing techniques have grown quickly [34]. Digital cameras provide instant shooting results, greatly improve intuition, and reduce the learning costs of workflows; methods for extracting FVC products from digital images have been proposed and modified owing to their efficiency and convenience [35]. Melville et al. used a portable photographic instrument to measure the FVC in the field [36]. Liu et al. also proposed an FVC extraction algorithm based on the LAB color space [20].

However, photography methods in actual shooting face certain challenges. Shadows often create extreme illumination contrasts, causing substantial luminance differences within a single image scene. A study revealed that the reflectance of shaded leaves is lower than that of sunlit leaves at all wavelengths [37]. The presence of shadows could disrupt the accurate recording of pixel information. When the field scene is not covered by a hood or similar structure, shadows are inevitable [38]. In many cases, shadows constitute a high-incidence area where commission and omission errors occur during FVC extraction. As a consequence, the FVC extraction capability in the shadow region must be addressed to improve the segmentation performance. To address the issue of low FVC accuracy based on photography methods under shadow conditions, several studies have proposed solutions from different perspectives. There are two primary approaches: first, specific techniques can be employed during data acquisition to compensate for the limitations due to shadows impeding the accurate recording of vegetation information [39,40]; second, algorithms have been designed for the vegetation extraction process to minimize the impact of shadows. In numerous studies, the characteristics of high-dynamic range (HDR) imaging have been utilized to effectively expand the dynamic range of photos. This technique allows for better capture of vegetation information and is often combined with thresholding methods or deep learning techniques to calculate FVC [41,42,43,44,45]. Yang et al. proposed acquiring polarization images in the shooting process to complement and enhance pixel information that cannot be effectively captured under shadows [46]. From a postprocessing perspective, Song et al. introduced the SHAR-LABFVC shadow-resistant algorithm, which alters the color space distribution of captured images to mitigate the adverse effect of shadows on the calculation of FVC [47].

Although various methods have been proposed to address the issue of low FVC accuracy under shadow conditions, there are still areas that need improvement. First, the computational efficiency of these methods still needs to be enhanced. Methods using deep learning and machine learning techniques are extremely time-consuming. Moreover, the accuracy of these methods is closely tied to the quality of data samples, which means that they are significantly influenced by human factors. Second, there is a lack of solutions for FVC calculation under complex shadow conditions. Previous algorithms such as SHAR-LABFVC, methods based on HDR images, and algorithms incorporating polarized images essentially aim to supplement pixel information in shadowed areas to improve FVC accuracy. These studies all indicate that reconstructing shadow pixel information is an effective and feasible solution. However, they have not addressed complex shadow situations more specifically. For instance, the SHAR-LABFVC algorithm enhances shadow pixels based on the overall light distribution of the entire image.

To address the issue of shadows affecting the accurate recording of pixel information and subsequently reducing the FVC accuracy, a simple and effective solution is proposed in this study. The approach is based on a pixel-level classification strategy using a double-exposure algorithm with the ability to process images with different shadow levels. The double-exposure fusion data are derived from a normal-exposure image and a specific overexposed image. By increasing the exposure compensation during shooting, shadow regions are enhanced to show more vegetation details. Moreover, sunlit vegetation may be enhanced by overexposure. Therefore, a fusion operation is performed between the shadow pixels of the normal-exposure image and the corresponding pixels of the overexposed image. This process yields a fused double-exposure image containing a strengthened shadow region and a normal sunlit region. In the research process, multiple records of overexposed images at different exposure levels were captured. Various vegetation indices were assessed to identify the most suitable candidates for the double-exposure method and the optimal overexposed images under shadow conditions. This article is divided into four sections (materials and methods, experimental results and analysis, discussion, and conclusion) elaborating on the double-exposure method.

2. Materials and Methods

2.1. Data Acquisition

The camera used in this study is a Pentax K-S2 digital camera (Ricoh, Tokyo, Japan). It is equipped with a CMOS sensor measuring 23.5 × 15.6 mm² and provides a resolution of 20.12 million effective pixels. The exposure compensation function of this camera allows adjustment within the −5 ev to +5 ev range. During data collection, we positioned the camera vertically approximately 1 m above the ground to capture vegetation images. Throughout the shooting process, the ISO and focal length were fixed at 200 and 18 mm, respectively. Camera exposure compensation relies on automatic calculation and adjustment of exposure time and aperture to achieve the desired image exposure levels. To capture images with varying exposure levels, we adjusted the exposure compensation parameter values during the shooting process. The exposure compensation parameter ranged from 0 to +5 ev, with images captured at each gear, resulting in multiple sets of data containing normal exposure and five levels of overexposure. A schematic of the data capture process is shown in Figure 1. Data were captured in March 2023, yielding a total of 14 sets of experimental data. The image size is 5472 pixels ×3648 pixels, and the captured vegetation type is green herbaceous plants.

Figure 2 shows one of the experimental datasets, consisting of exposure data at various exposure compensation levels: 0 ev, +1 ev, +2 ev, +3 ev, +4 ev, and +5 ev. Notably, overexposure not only increases the image brightness but also results in varying pixel restoration capabilities for shadow areas at different overexposure levels. Compared to the normal-exposure image, the textural information of vegetation and non-vegetation areas within the red box is more distinct and accurate. However, at the same time, positive exposure compensation exerts a detrimental effect on pixels in nonshadow areas. The sharp increase in brightness causes varying degrees of damage to these pixels, resulting in blurred texture information, indistinct vegetation boundaries, and color distortion. In addition, to facilitate better comparison and analysis of the experimental results, data augmentation is provided in this study. The images of the 14 sets of data were cropped to 512 pixels × 512 pixels using a stride of 512 pixels. The cropped results were then selected and filtered, leading to a final collection of 414 images of shaded green vegetation.

2.2. Methods

The experimental flowchart of this study is shown in Figure 3, which primarily encompasses three sections: data preprocessing, vegetation extraction, and method evaluation. Regarding the vegetation images captured under normal and overexposure conditions, the data preprocessing module utilizes a custom fusion algorithm to generate fused images. This approach significantly extends the dynamic range of the fused images and enriches the pixel information in shadow areas. As a result, vegetation and other features in the captured area can be more accurately represented in digital images. Subsequently, based on the values of five vegetation indices and the Otsu thresholding method, image-based vegetation classification is achieved. The FVC can be obtained from the classification results. The strategy of designing five vegetation extraction methods aims to find the best-fitting vegetation extraction approach for the double-exposure method, thereby maximizing the performance of the double-exposure fusion data product. In method evaluation, the actual impact of the double-exposure method on the FVC accuracy under shadow conditions is assessed. Factors considered include the intensity of exposure compensation, the vegetation extraction method, and the proportion of shadows. The optimal double-exposure combination suitable for different degrees of shadow issues was evaluated.

2.2.1. Data Preprocessing

To obtain the intensity value of each pixel, the image must be converted from the RGB color space to the HIS color space. The detailed conversion method can be expressed as follows:

$$\mathrm{i}={\displaystyle \frac{\mathrm{R}+\mathrm{G}+\mathrm{B}}{3}}$$

(1)

$$\mathsf{\theta }={\cos }^{-1}\left\{{\displaystyle \frac{\frac{1}{2}\left[\left(\mathrm{R}-\mathrm{G}\right)+\left(\mathrm{R}-\mathrm{B}\right)\right]}{{\left[{\left(\mathrm{R}-\mathrm{G}\right)}^2+\left(\mathrm{R}-\mathrm{B}\right)\left(\mathrm{G}-\mathrm{B}\right)\right]}^{\frac{1}{2}}}}\right\}$$

(2)

$$\mathrm{h}=\left\{\begin{array}{c} \mathsf{\theta } \mathrm{B}\le \mathrm{G} \\ 360-\mathsf{\theta } \mathrm{B}>\mathrm{G}\end{array}\right.$$

(3)

$$\mathrm{s}=1-{\displaystyle \frac{\left.\mathrm{mi}\mathrm{n}(\left\{\mathrm{R},\mathrm{G},\mathrm{B}\right\}\right) }{\mathrm{i}}}$$

(4)

where $\mathrm{i}$, $\mathrm{h}$, and $\mathrm{s}$ represent the intensity, hue, and saturation, respectively, and $\mathrm{R}$, $\mathrm{G}$, and $\mathrm{B}$ denote the red, green, and blue component values of each pixel, respectively.

Based on the intensity component values of each pixel, a threshold of 0.2 is used to distinguish between shadow and nonshadow regions, and the distribution range of shadows can be calculated. For shadow area pixels, a fusion algorithm is employed to combine the corresponding pixel values obtained under normal and overexposure conditions. The fusion algorithm determines the weight of the overexposed pixel values in the fusion process based on the intensity values. The strategy of adaptively adjusting fusion weights can better handle images with varying degrees of shadow. This results in reconstructed fusion pixel values with recovered information, thereby generating a shadow-free fused image.

$${\mathrm{D}\mathrm{N}}_{\mathrm{d}}\left(\mathrm{x},\mathrm{y}\right)={\mathrm{D}\mathrm{N}}_{\mathrm{n}}\left(\mathrm{x},\mathrm{y}\right)+{\mathrm{D}\mathrm{N}}_{\mathrm{o}}\left(\mathrm{x},\mathrm{y}\right)\times {\displaystyle \frac{\left(0.2-\mathrm{i}\right)}{0.2}}$$

(5)

The DN values of the shadow pixels at the spatial position $\left(\mathrm{x},\mathrm{y}\right)$ in the double-exposure, normal-exposure, and overexposure images are denoted as ${\mathrm{D}\mathrm{N}}_{\mathrm{d}}\left(\mathrm{x},\mathrm{y}\right)$, ${\mathrm{D}\mathrm{N}}_{\mathrm{n}}(\mathrm{x},\mathrm{y})$, and ${\mathrm{D}\mathrm{N}}_{\mathrm{o}}(\mathrm{x},\mathrm{y})$, respectively. In addition, $\mathrm{i}$ denotes the intensity value of the shadow pixels. The fusion effect of the algorithm is shown in Figure 4. The pixel categories in the red box of the fused image also become more distinct because the introduction of pixel information under overexposed conditions allows for the recovery of pixel information in shadow areas. As a result, the boundary information between vegetation and non-vegetation areas is also clearer. However, excessive fusion of overexposed information can affect the accuracy of pixels in weak shadow areas. These pixels become whitish, making it difficult to accurately describe the color information of ground objects. In contrast, incorporating too little overexposure information could lead to failure to fully reconstruct and recover pixel information in intense shadow areas. Therefore, the most suitable fusion approach should be determined based on the actual shadow conditions.

2.2.2. Vegetation Extraction

This study chose to employ a threshold classification method based on vegetation indices, achieving fully automated pixel-level vegetation extraction from vegetation images. This method is an unsupervised approach that can be used directly without relying on manual sample labeling. It features high computational efficiency, fast processing speed, and strong robustness, allowing it to adapt well to different types of vegetation images. Moreover, it does not require additional training data, making it particularly advantageous for rapid FVC calculation in practical scenarios. In addition to this method, machine learning methods such as random forests, support vector machines, and deep learning have also been applied to calculate FVC. However, all of these methods require labeled data to construct datasets for model training, and they need a substantial number of samples to support the development of the final model for vegetation extraction in images. This process consumes a considerable amount of time and computational resources. Consequently, this strategy was ultimately selected to complete the vegetation extraction task.

Four vegetation indices based on the RGB color space and the hue component (HUE) of the HIS color space were adopted as the basis for vegetation extraction. The Otsu thresholding method was used to determine the optimal threshold segmentation point, thereby separating the vegetation and non-vegetation parts of the image data. The four vegetation indices used are the excess green index ($\mathrm{E}\mathrm{X}\mathrm{G}$), color index of vegetation extraction ($\mathrm{C}\mathrm{I}\mathrm{V}\mathrm{E}$), excess red index ($\mathrm{E}\mathrm{X}\mathrm{R}$), and excess green minus excess red index ($\mathrm{E}\mathrm{X}\mathrm{G}\mathrm{R}$). These indices can be calculated by linearly combining the red component value ($\mathrm{R}$), green component value ($\mathrm{G}$), and blue component value ($\mathrm{B}$) of each pixel in the image as follows:

$$\mathrm{E}\mathrm{X}\mathrm{G}=2\mathrm{G}-\mathrm{R}-\mathrm{B}$$

(6)

$$\mathrm{E}\mathrm{X}\mathrm{R}=1.4\mathrm{R}-\mathrm{G}$$

(7)

$$\mathrm{E}\mathrm{X}\mathrm{G}\mathrm{R}=\mathrm{E}\mathrm{X}\mathrm{G}-\mathrm{E}\mathrm{X}\mathrm{R}$$

(8)

$$\mathrm{C}\mathrm{I}\mathrm{V}\mathrm{E}=0.441\mathrm{R}-0.811\mathrm{G}+0.385\mathrm{B}+18.78745$$

(9)

Otsu thresholding is a widely used adaptive threshold segmentation algorithm designed for image binary classification problems. The algorithm aims to find a threshold value, denoted as $\mathrm{t}$, that maximizes the between-class variance ${\mathsf{\sigma }}^2$. This approach maximizes the difference between the foreground and background, achieving optimal classification results. Ultimately, it generates a binary image that reflects the specific distribution of the two classes. The between-class variance can be calculated as follows:

$${\mathsf{\sigma }}^2\left(\mathrm{t}\right)={\mathrm{P}}_{\mathrm{a}}\left(\mathrm{t}\right){\mathrm{P}}_{\mathrm{b}}\left(\mathrm{t}\right){\left({\mathsf{\mu }}_{\mathrm{a}}\left(\mathrm{t}\right)-{\mathsf{\mu }}_{\mathrm{b}}\left(\mathrm{t}\right)\right)}^2$$

(10)

where ${\mathrm{P}}_{\mathrm{a}}$ and ${\mathrm{P}}_{\mathrm{b}}$ denote the proportions of the class a and b distributions, respectively, at a threshold of $\mathrm{t}$. ${\mathsf{\mu }}_{\mathrm{a}}$ and ${\mathsf{\mu }}_{\mathrm{b}}$ are the pixel means corresponding to classes a and b, respectively.

2.2.3. Accuracy Evaluation

In this study, experimentally obtained vegetation classification binary images and FVC data were used to comprehensively evaluate and analyze the impact of the double-exposure algorithm on the accuracy of vegetation extraction from shadow images. This evaluation was conducted from both semantic segmentation and statistical analysis perspectives. The ground truth labels of the vegetation images in the experimental data were obtained through manual visual interpretation and annotation. This process allowed for the calculation of the true FVC for each image, providing a reliable baseline for our analysis. The confusion matrix consists of four parts: TP, FN, FP, and TN [48]. TP represents the portion where both the actual and predicted values are vegetation. FN represents the portion where the actual value is vegetation but is predicted as non-vegetation. FP represents the portion where the actual value is non-vegetation but is predicted as vegetation. TN represents the portion where both the actual and predicted values are non-vegetation. Based on the confusion matrix, the above indices can be calculated as follows:

$$\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(11)

$$\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}={\displaystyle \frac{\mathrm{T}\mathrm{P}+\mathrm{T}\mathrm{N}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}+\mathrm{F}\mathrm{N}+\mathrm{T}\mathrm{N}}}$$

(12)

$$\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}}}$$

(13)

$$\mathrm{a}={\displaystyle \frac{\left(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}\right)\ast \left(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{P}\right)+\left(\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}\right)\ast \left(\mathrm{F}\mathrm{N}+\mathrm{T}\mathrm{N}\right)}{{\left(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}+\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}\right)}^2}}$$

(14)

$$\mathrm{K}\mathrm{a}\mathrm{p}\mathrm{p}\mathrm{a}={\displaystyle \frac{\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}-\mathrm{a}}{1-\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}}}$$

(15)

$$\mathrm{I}\mathrm{O}\mathrm{U}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}+\mathrm{F}\mathrm{P}}}$$

(16)

$$\mathrm{m}\mathrm{I}\mathrm{O}\mathrm{U}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{2\left(\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}+\mathrm{F}\mathrm{P}\right)}}+{\displaystyle \frac{\mathrm{T}\mathrm{N}}{2\left(\mathrm{F}\mathrm{P}+\mathrm{T}\mathrm{N}+\mathrm{F}\mathrm{N}\right)}}$$

(17)

$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}$ is the proportion of vegetation instances correctly predicted as vegetation among all actual vegetation instances. $\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}$ denotes the proportion of correct predictions overall. $\mathrm{P}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}$ is defined as the proportion of correctly predicted vegetation instances. In this study, the intersection over union ($\mathrm{I}\mathrm{O}\mathrm{U}$) of the vegetation class denotes the ratio of the intersection to the union of the predicted and ground truth values. Both the $\mathrm{k}\mathrm{a}\mathrm{p}\mathrm{p}\mathrm{a}$ coefficient and the mean intersection over union ($\mathrm{m}\mathrm{I}\mathrm{O}\mathrm{U}$) are used to evaluate the overall classification performance of vegetation and non-vegetation classes. The closer the values of these six metrics are to 1, the higher the image classification performance and accuracy. To evaluate the accuracy of FVC prediction, a statistical analysis approach was adopted in this study. A univariate linear regression model was constructed to analyze the relationship between the experimental FVC values and the ground truth values obtained through manual visual interpretation. The regression equation, the R², and the root mean square error (RMSE) of the model were obtained. These metrics were utilized to assess the accuracy of shadow images FVC using the double-exposure algorithm.

3. Results

3.1. Performance of the Different Extraction Methods under the Double-Exposure Algorithm

In this study, linear regression analysis was conducted of the FVC results obtained from 414 sets of normal-exposure images with shadows and five types of double-exposure fusion images using five vegetation extraction indices. The results are shown in Figure 5, Figure 6, Figure 7, Figure 8 and Figure 9. The diagonal line in the figures is the reference line indicating equal predicted and true FVC values. Each index responds differently to the different double-exposure images. Among them, the $\mathrm{C}\mathrm{I}\mathrm{V}\mathrm{E}$ and $\mathrm{E}\mathrm{X}\mathrm{G}$ indices performed the best on the nor+over2 image. The HUE achieved the highest accuracy in extracting the FVC from the nor+over3 image. The $\mathrm{E}\mathrm{X}\mathrm{R}$ and $\mathrm{E}\mathrm{X}\mathrm{G}\mathrm{R}$ indices achieved the best vegetation extraction results on the nor+over4 and nor+over5 images, respectively. The fusion of normal-exposure images with images compensated at the +1 ev gear, resulting in the nor+over1 image, was insufficient for effectively solving the problem of low accuracy of FVC under shadow conditions. This occurs because the lower exposure compensation level fails to completely avoid the presence of shadow pixels, making it challenging to capture sufficient pixel information under shadows. As a result, the obtained double-exposure image pixels are still damaged by shadows. Therefore, to fully leverage the benefits of the double-exposure algorithm, it is crucial to choose appropriate exposure compensation and utilize vegetation indices based on actual shadow conditions. This approach can significantly improve the FVC accuracy from shadow images and mitigate the adverse effect of shadows on vegetation extraction. In Figure 5a, Figure 6a, and Figure 7a, the scatter points derived from the $\mathrm{C}\mathrm{I}\mathrm{V}\mathrm{E}$, $\mathrm{E}\mathrm{X}\mathrm{G}$, and HUE indices under normal exposure conditions are mostly clustered around or below the diagonal line. This indicates that for some experimental datasets under normal exposure, the predicted FVC is lower than the true FVC. In other words, some shaded vegetation in the images was not correctly identified. As shown in Figure 8a and Figure 9a, the scatter points for the $\mathrm{E}\mathrm{X}\mathrm{R}$ and $\mathrm{E}\mathrm{X}\mathrm{G}\mathrm{R}$ indices under normal exposure conditions are more scattered and disorganized. These two methods indicate cases where the extracted FVC is higher than the true FVC, as well as instances where it is lower. These findings suggest that these indices not only fail to detect vegetation in shadow areas but also cause misclassification of non-vegetation areas as vegetation areas. Among the five indices, using double-exposure images generally improved the FVC accuracy over using normal-exposure images as a reference. However, while the double-exposure method significantly reduced vegetation misdetection or misclassification, only the HUE index achieved the best performance in terms of all three metrics: R², bias, and RMSE. The predicted FVC exhibited a strong correlation with the true FVC, demonstrating reliability and stability. Particularly in the nor+over3 image, the R², bias, and RMSE values were increased from 0.750, −0.018, and 0.146 in the normal-exposure image to 0.969, 0.006, and 0.046, respectively, indicating excellent vegetation extraction capability.

To provide a comprehensive and objective evaluation of the experimental results for the different double-exposure images, the results were also analyzed from a semantic segmentation perspective. This analysis employed evaluation metrics such as the $\mathrm{m}\mathrm{I}\mathrm{O}\mathrm{U}$ index and $\mathrm{k}\mathrm{a}\mathrm{p}\mathrm{p}\mathrm{a}$ coefficient.

Table 1. Accuracy statistics of vegetation classification for five double-exposure fusion combinations of the double-exposure algorithm and normal-exposure images using the CIVE. The higher the accuracy indicator value, the higher the vegetation extraction accuracy.

	nor	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5
mIOU	0.822	0.885	0.896	0.884	0.868	0.837
IOU	0.842	0.905	0.916	0.904	0.889	0.858
Accuracy	0.899	0.942	0.949	0.941	0.930	0.908
Precision	0.988	0.988	0.987	0.984	0.982	0.980
Recall	0.853	0.917	0.928	0.919	0.905	0.874
Kappa	0.790	0.868	0.882	0.866	0.847	0.809

,

Table 2. Accuracy statistics of vegetation classification for five double-exposure fusion combinations of the double-exposure algorithm and normal-exposure images using the EXG.

	nor	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5
mIOU	0.806	0.878	0.887	0.870	0.846	0.815
IOU	0.825	0.899	0.909	0.891	0.868	0.835
Accuracy	0.887	0.938	0.944	0.932	0.914	0.893
Precision	0.985	0.988	0.985	0.982	0.979	0.977
Recall	0.837	0.910	0.922	0.908	0.886	0.854
Kappa	0.770	0.859	0.870	0.849	0.820	0.782

,

Table 3. Accuracy statistics of vegetation classification for five double-exposure fusion combinations of the double-exposure algorithm and normal-exposure images using the HUE index.

	nor	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5
mIOU	0.871	0.907	0.924	0.930	0.927	0.927
IOU	0.856	0.910	0.933	0.943	0.940	0.940
Accuracy	0.935	0.961	0.970	0.973	0.972	0.972
Precision	0.878	0.939	0.967	0.978	0.976	0.977
Recall	0.874	0.929	0.952	0.963	0.960	0.959
Kappa	0.831	0.887	0.914	0.924	0.920	0.918

,

Table 4. Accuracy statistics of vegetation classification for five double-exposure fusion combinations of the double-exposure algorithm and normal-exposure images using the EXR.

	nor	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5
mIOU	0.649	0.716	0.752	0.786	0.802	0.793
IOU	0.708	0.756	0.784	0.815	0.825	0.815
Accuracy	0.788	0.833	0.859	0.885	0.898	0.893
Precision	0.797	0.848	0.874	0.899	0.904	0.899
Recall	0.870	0.880	0.890	0.903	0.910	0.905
Kappa	0.533	0.636	0.690	0.738	0.760	0.752

and

Table 5. Accuracy statistics of vegetation classification for five double-exposure fusion combinations of the double-exposure algorithm using the EXGR.

	nor	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5
mIOU	0.815	0.872	0.886	0.892	0.895	0.888
IOU	0.837	0.894	0.906	0.910	0.912	0.906
Accuracy	0.894	0.937	0.945	0.947	0.949	0.947
Precision	0.945	0.964	0.968	0.969	0.972	0.974
Recall	0.887	0.927	0.936	0.938	0.938	0.930
Kappa	0.780	0.852	0.869	0.875	0.879	0.870

provide the specific performance of the five indices given the normal- and double-exposure data. A comparison between the double- and normal-exposure data indicated that the use of double-exposure data significantly improved the FVC accuracy among the five indices. For example, the notable increase in $\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}$ values for the $\mathrm{E}\mathrm{X}\mathrm{G}$ and $\mathrm{C}\mathrm{I}\mathrm{V}\mathrm{E}$ indices indicates a significant reduction in the number of missed detections of vegetation. The significant increase in $\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}$ for the $\mathrm{E}\mathrm{X}\mathrm{R}$ index suggests a lower probability of misclassifying non-vegetation as vegetation. Additionally, the substantial improvement in the $\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}$ and $\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}$ values for the HUE index also demonstrates the effectiveness in resolving double-exposure data.

When further comparing the performance of different indices given double-exposure data, the most suitable double-exposure combination results for the $\mathrm{C}\mathrm{I}\mathrm{V}\mathrm{E}$, $\mathrm{E}\mathrm{X}\mathrm{G}$, HUE, and $\mathrm{E}\mathrm{X}\mathrm{R}$ indices strongly conform with the conclusions obtained from linear analysis methods. However, for the EXGR index, the most suitable double-exposure image changes from nor+over5 to nor+over4. Combining Figure 7 and Table 3, it can be concluded that when using double-exposure data for vegetation extraction, the HUE index provides a significant improvement in accuracy relative to normal-exposure data. This indicates that the HUE index is well-suited for analyzing double-exposure data. Additionally, the FVC accuracy under the HUE index is substantially higher than it is under the other four indices. Among all five indices evaluated, the HUE index is the optimal choice for calculating FVC when using double-exposure data.

Figure 10 shows a set of experimental data with a high proportion of shadows, in which shadows blur pixel information and have a greater impact on the accuracy of FVC. After merging overexposed images, the distribution of vegetation and non-vegetation becomes more clearly visible. Moreover, double-exposure images with different degrees of overexposure show varying results, as demonstrated in the first column. The change in pixel areas within the red circle is particularly noticeable. Consequently, the vegetation extraction results obtained from the shadow-affected normal-exposure image and the shadow-removed double-exposure fusion image using the five vegetation indices substantially differ as a result. When extracting vegetation from the normal-exposure data, the results across the various vegetation indices indicate that a high proportion of shadow nearly renders the provided vegetation extraction function ineffective. Among these, the CIVE and EXG perform better compared to other indices. However, they extract only a small portion of vegetation and misclassify some non-vegetation as vegetation, resulting in an increased occurrence of missed vegetation detection overall. With the introduction of double-exposure fusion data, the vegetation extraction results become more reliable. However, there are still areas where vegetation cannot be accurately extracted, and some non-vegetation areas are misidentified as vegetation areas. In the case of the HUE index, the loss of pixel information caused by shadows significantly impacts the effectiveness of vegetation extraction. However, in the nor+over2 images and higher-gear double-exposure images, the hue index demonstrates a stronger ability to extract vegetation compared to other indices, with lower probabilities of vegetation misclassification and omission. This indicates that the removal of shadows can greatly improve the HUE index’s classification performance. Therefore, as the exposure compensation level increases along the positive direction, pixel information in the shadow regions of the double-exposure fusion image is increasingly restored, allowing accurate vegetation extraction. In contrast to those of the previous three indices, the extraction results of the $\mathrm{E}\mathrm{X}\mathrm{G}$ and $\mathrm{E}\mathrm{X}\mathrm{G}\mathrm{R}$ show different results when applied to normal-exposure images with a high proportion of shadows. These two indices indicate a high frequency of vegetation misclassification in the presence of a high proportion of shadows. These two indices reveal that a significant number of non-vegetation pixels are erroneously classified as vegetation pixels, resulting in a low FVC accuracy. Although the use of double-exposure data can reduce misclassification occurrence, the overall effectiveness of vegetation extraction remains unsatisfactory, especially for results based on the $\mathrm{E}\mathrm{X}\mathrm{R}$. Overall, in this set of experimental data, the hue index demonstrates good adaptability to double-exposure images, exhibiting superior vegetation extraction capabilities.

To comprehensively analyze and evaluate the performance of the double-exposure algorithm, this study also compared it with the classic algorithm, SHAR-LABFVC. Both algorithms are based on the principle of image enhancement through intensity values. Both algorithms prove that image enhancement based on the intensity value is direct and effective. We conducted vegetation extraction on 14 sets of experimental data, with results shown in

Table 6. Comparison of FVC extraction results between the double-exposure algorithm and the SHARLAB-FVC algorithm for 14 sets of experimental data. In the table, nor+over1, nor+over2, nor+over3, nor+over4, and nor+over5 represent the results under five double-exposure combination conditions. The double-exposure algorithm is based on the HUE index.

Data ID	FVC Ground Truth	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5	SHAR-LABFVC
1	0.380	0.362	0.364	0.364	0.363	0.363	0.309
2	0.427	0.427	0.427	0.428	0.429	0.429	0.402
3	0.439	0.436	0.437	0.437	0.437	0.437	0.417
4	0.426	0.432	0.431	0.430	0.430	0.430	0.408
5	0.429	0.432	0.432	0.432	0.431	0.431	0.425
6	0.210	0.213	0.213	0.213	0.213	0.213	0.209
7	0.224	0.193	0.201	0.201	0.201	0.200	0.163
8	0.685	0.688	0.691	0.692	0.691	0.691	0.660
9	0.653	0.653	0.651	0.651	0.651	0.651	0.640
10	0.609	0.616	0.621	0.624	0.624	0.623	0.566
11	0.295	0.296	0.296	0.296	0.296	0.296	0.261
12	0.505	0.497	0.499	0.498	0.498	0.499	0.473
13	0.563	0.557	0.558	0.557	0.557	0.557	0.542
14	0.623	0.609	0.609	0.607	0.607	0.606	0.613

. When compared to the FVC ground truth, the double-exposure algorithm demonstrated a significant advantage in accuracy over the SHAR-LABFVC algorithm. It offers a more effective solution to the problem of vegetation extraction in shadowed areas. This advantage stems from the fundamental difference in approach between the two algorithms. The SHAR-LABFVC algorithm directly employs statistical methods to optimize the intensity value of all pixels; therefore, its enhancement approach is not as refined. The double-exposure algorithm specifically divides the image into shadow and non-shadow regions based on the intensity value, and image enhancement is completed for shadow areas by combining overexposed images on a pixel-by-pixel basis. This targeted approach enables more accurate reconstruction of shadowed image information and, consequently, yields more precise vegetation extraction results.

3.2. Performance of the Different Double-Exposure Combinations under Varying Shadow Proportions

The vegetation extraction capabilities of the five double-exposure images vary depending on the shadow proportion in the vegetation images. The HUE index performs the best in extracting vegetation from shadow images. Therefore, the focus is placed on determining the most effective double-exposure image under the different shadow levels based on the FVC results using the HUE index. This approach aims to improve vegetation extraction effectiveness under shadow conditions. Based on the proportion of shadow pixels in the normal-exposure image, the experimental data are categorized into three classes: weak shadows (0–40% shadow pixels), moderate shadows (40–70% shadow pixels), and strong shadows (70–100% shadow pixels). According to

Table 7. Comparative analysis of vegetation extraction performance under weak shadow conditions (0–40% shadow pixels) using the HUE index-based double-exposure algorithm for five double-exposure fusion combinations. The higher the accuracy indicator value, the higher the vegetation extraction accuracy.

		nor	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5
0–40% shadow pixels	mIOU	0.932	0.940	0.941	0.940	0.938	0.938
	IOU	0.936	0.954	0.954	0.953	0.953	0.953
	Accuracy	0.973	0.979	0.979	0.979	0.978	0.978
	Precision	0.960	0.981	0.982	0.982	0.981	0.982
	Recall	0.950	0.971	0.971	0.971	0.970	0.970
	Kappa	0.920	0.936	0.937	0.936	0.934	0.934

,

Table 8. Comparative analysis of vegetation extraction performance under moderate shadow conditions (40–70% shadow pixels) using the HUE index-based double-exposure algorithm for five double-exposure fusion combinations.

		nor	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5
40–70% shadow pixels	mIOU	0.811	0.892	0.909	0.918	0.913	0.914
	IOU	0.793	0.901	0.923	0.937	0.931	0.931
	Accuracy	0.883	0.953	0.961	0.966	0.966	0.966
	Precision	0.810	0.927	0.958	0.973	0.966	0.967
	Recall	0.811	0.922	0.942	0.963	0.953	0.952
	Kappa	0.756	0.865	0.893	0.908	0.899	0.900

and

Table 9. Comparative analysis of vegetation extraction performance under strong shadow conditions (70–100% shadow pixels) using the HUE index-based double-exposure algorithm for five double-exposure fusion combinations.

		nor	nor+over1	nor+over2	nor+over3	nor+over4	nor+over5
70–100% shadow pixels	mIOU	0.755	0.823	0.891	0.914	0.911	0.906
	IOU	0.684	0.785	0.881	0.918	0.915	0.910
	Accuracy	0.878	0.917	0.952	0.965	0.963	0.960
	Precision	0.707	0.819	0.932	0.973	0.974	0.974
	Recall	0.713	0.806	0.906	0.940	0.937	0.932
	Kappa	0.645	0.758	0.866	0.903	0.899	0.893

, in the case of weak shadow data, the nor+over2 double-exposure image demonstrates the best extraction performance. Compared to normal-exposure data, there are slight improvements in all the evaluation metrics, but the changes are not significant. For the data with moderate shadow levels, the nor+over3 double-exposure image proves optimal. It demonstrates a 10.7% increase in the $\mathrm{m}\mathrm{I}\mathrm{O}\mathrm{U}$ metric, an 8.3% increase in accuracy, and more than a 14% increase in the other metrics. Similarly, for data with strong shadows, the nor+over3 double exposure image is the best choice, with all the metrics surpassing 0.9. Moreover, compared to those for the normal-exposure data, $\mathrm{m}\mathrm{I}\mathrm{O}\mathrm{U}$ increases from 0.755 to 0.914, and the $\mathrm{k}\mathrm{a}\mathrm{p}\mathrm{p}\mathrm{a}$ coefficient increases from 0.645 to 0.903. These improvements greatly enhance the reliability of the vegetation extraction results and ensure high vegetation extraction accuracy under shadow conditions.

4. Discussion

4.1. Advantages of Applying the Double-Exposure Algorithm

The double-exposure algorithm proposed in this paper addresses the issue of low FVC accuracy in the case of shadows. By merging normal-exposure and overexposed images, this approach efficiently captures the pixel information in shadow areas to restore their true characteristics, facilitating accurate vegetation identification. Extracting shadow-affected vegetation essentially involves capturing real-life scenarios under an extreme lighting contrast. For this purpose, scholars worldwide have proposed several approaches. The main solutions are as follows: (1) The probability statistics method can be used. One approach to extracting vegetation from digital images entails using the segmentation point between vegetation and non-vegetation in the a* component of the LAB color space [49]. The channels in this color space exhibit a low correlation and have been utilized in studies focused on mitigating the impact of vegetation shadows [50,51]. Song et al. [47] suggested preprocessing the brightness histogram of the digital image in the HIS color space, followed by utilizing the probability distribution of the a* component for FVC calculation. This method’s performance evaluation relies solely on the overall accuracy of FVC calculation from the image. It does not require or consider assessing the classification correctness of each pixel. Therefore, the FVC accuracy using this method should be further investigated and verified. (2) HDR imaging techniques can be used. Suh et al. employed the HDR imaging technique to identify shadow areas in images and mitigated their impact on vegetation extraction [38]. Wang et al. proposed using HDR images to capture the vegetation distribution and obtain images with a wider dynamic range, thereby reducing the shadow occurrence frequency [41]. This technology can improve shadow issues in images, but HDR imaging cannot completely resolve shadow problems [38]. (3) Polarization information can be combined. Yang et al. and Cui et al. proposed the idea of capturing polarization information simultaneously while recording vegetation images to supplement the information of shadow areas in normal-exposure images, thus aiding in high-precision FVC [46,52]. However, the acquisition of polarization information relies on the lighting conditions during photography. When the light intensity is too low, the amount of polarized light reduces, making it difficult to capture and record images, which hinders the recovery of shadow pixel information. In this study, the following factors are considered when selecting a vegetation extraction strategy under shadow conditions. First, several vegetation indices based on the RGB color space, such as CIVE, EXG, and EXGR, perform poorly under shadow conditions [53]. However, vegetation index-based methods can achieve a high classification efficiency and are easy to implement [54,55,56]. Second, the process of machine learning classification is complex. In existing research, deep learning methods with excellent image classification performance are used for classification [23,57]. However, this method is cumbersome and requires time-consuming sample annotation and complex training processes, making it labor-intensive. After weighing the advantages and disadvantages, we ultimately adopt the vegetation index method to achieve efficient and accurate vegetation extraction, even under shadow conditions. After addressing the issue of shadows, the vegetation index method provides excellent vegetation extraction performance, with vegetation identification levels comparable to those of deep learning methods.

This study is based on multidimensional experimental tests to determine a suitable vegetation extraction index for the double-exposure algorithm. By considering shadow intensity differences, the best double-exposure combination under different shadow conditions was identified. The design of this algorithm considers both the strengths and weaknesses of existing related research. The double-exposure data obtained by this algorithm can be adapted to the vegetation index method. Ultimately, the design objective of a simple, direct, targeted, and efficient anti-shadow algorithm based on vegetation extraction was achieved in this study. The double-exposure algorithm is a vegetation extraction method that can mitigate shadow interference. This approach provides a new and effective strategy for precise and rapid FVC calculation, high-precision monitoring of ecological environments, and ground validation of remote sensing products.

4.2. Limitations and Future Perspectives

Due to various limitations, this study focused only on sampling and analyzing herbaceous plants under shadow conditions. Research on the extraction of other types of green vegetation, such as coniferous plants, under shadow conditions is lacking. Different types of plants exhibit significant differences in texture and color features. Therefore, further evaluation and analysis of the inclusion of shadow data for various vegetation types should be conducted. Additionally, we will consider the effects of more complex lighting scenarios and seasonal changes on shadow vegetation extraction. Furthermore, while the double-exposure algorithm was primarily evaluated and analyzed on ground truth data, shadow-related issues during vegetation extraction also occur in unmanned aerial vehicle (UAV) data. Therefore, there is a need to assess the double-exposure algorithm on UAV remote sensing imagery. This approach will contribute to improving the accuracy of large-scale FVC and expanding the application scenarios of the double-exposure algorithm.

5. Conclusions

This study proposed a method to address the issue of shadows in images, which can conceal true object information and fail to accurately reflect the actual vegetation distribution. By fusing normal-exposure and overexposed images, we yielded double-exposure fusion images that mitigate the negative impact of shadows. Comparing the experimental results of five vegetation indices using five double-exposure fusion combinations clearly revealed that the HUE index yields the highest accuracy in calculating the FVC from the double-exposure fusion data. The correlation coefficient between the FVC and ground truth reached 0.969, with bias and RMSE values of 0.006 and 0.046, respectively. Moreover, $\mathrm{m}\mathrm{I}\mathrm{O}\mathrm{U}$, $\mathrm{I}\mathrm{O}\mathrm{U}$, and the other six evaluation metrics exceeded 0.92, indicating the excellent adaptability of the HUE index to double-exposure fusion data. Based on the classification of the experimental data according to the degree of shadow coverage, the following conclusions can be drawn: (1) When shadow coverage in the image is less than 40%, the use of the HUE index for FVC calculation is less affected by shadows. Satisfactory results can be obtained for normal-exposure images. (2) When shadow coverage exceeds 40%, shadows exert a significant impact on vegetation extraction, leading to a notable reduction in the FVC accuracy. (3) In images with shadow coverage above 40%, utilizing nor+over3 double-exposure fusion data with the HUE index can increase the FVC accuracy to above 0.9. In conclusion, this study demonstrates the effectiveness of the double-exposure algorithm in improving FVC accuracy in shadowed areas. It provides a new, simple, efficient solution for precise vegetation extraction under shadow conditions. The application of this method can offer more reliable data support for ground-truth validation of satellite FVC products and ecological environment monitoring. In future research, it would be valuable to test the double-exposure algorithm in more complex vegetation types and lighting environments. Additionally, methods for automatically determining optimal exposure parameters are worthy of further exploration. This method can improve the accuracy of FVC calculation and provide more accurate FVC information. The codes and data are available at https://github.com/Jiaaaaa88/double-exposure-algorithm (accessed on 17 August 2024).