Refinement of Cropland Data Layer with Effective Confidence Layer Interval and Image Filtering

Abstract

Various systems have been developed to process agricultural land data for better management of crop production. One such system is Cropland Data Layer (CDL), produced by the National Agricultural Statistics Service of the United States Department of Agriculture (USDA). The CDL has been widely used for training deep learning (DL) segmentation models. However, it contains various errors, such as salt-and-pepper noise, and must be refined before being used in DL training. In this study, we used two approaches to refine the CDL for DL segmentation of major crops from a time series of Sentinel-2 monthly composite images. Firstly, different confidence intervals of the confidence layer were used to refine the CDL. Secondly, several image filters were employed to improve data quality. The refined CDLs were then used as the ground-truth in DL segmentation training and evaluation. The results demonstrate that the CDL with +45% and +55% confidence intervals produced the best results, improving the accuracy of DL segmentation by approximately 1% compared to non-refined data. Additionally, filtering the CDL using the majority and expand–shrink filters yielded the best performance, enhancing the evaluation metrics by about 1.5%. The findings suggest that pre-filtering the CDL and selecting an effective confidence interval can significantly improve DL segmentation performance, contributing to more accurate and reliable agricultural monitoring.

1. Introduction

The agricultural sector has historically met food production demands; however, concerns about its ability to sustain this role for an expanding global population persist in food security discussions [1]. While significant strides have been made in reducing hunger globally, substantial challenges remain [2,3]. Achieving effective situational awareness in food production necessitates a comprehensive understanding of the earth as an integrated system, a capability that is uniquely provided by satellite technology [4]. Satellites offer the ability to map farmland areas and crop types, estimate planted areas and yields and provide early warnings for droughts and floods. Consequently, numerous monitoring systems have been developed in recent years to utilize these satellite data. Among these, Cropland Data Layer (CDL), published annually by the United States Department of Agriculture (USDA), stands out by offering 30 m resolution crop type maps derived from extensive labeled training samples [5,6,7]. A comprehensive overview of various global agricultural monitoring systems is available in the work of Fritz, et al. [8].

The CDL is widely regarded as the primary source for ground-truth data in machine learning model training owing to its advanced system architecture and adherence to geospatial web services standards within a publicly accessible web environment [9]. Numerous studies affirm the CDL’s credibility as a reliable ground-truth for crop mapping [10,11,12,13]. Currently, the CDL represents the most comprehensive national crop type map available for free use and download, offering exceptional overall accuracy for major crops, particularly corn and soybeans, with accuracy rates exceeding 95% [5]. However, several studies have highlighted defects within the CDL, primarily due to its reliance on remote sensing datasets, which raises concerns regarding the quality of CDL products [9,14,15,16,17,18]. To address these concerns, the USDA National Agricultural Statistics Service (NASS) has provided an accuracy rating since 2008 and introduced the confidence layer in 2017 to enhance data reliability [5].

The use of various CDL confidence layer percentages has been explored in some studies to utilize the CDL as ground-truth data [13,19]. For instance, Hao et al. [13] employed a 95% confidence interval, rejecting all CDLs below this threshold. However, there is no clear justification for selecting the 95% threshold, and similar studies lack a rationale for their chosen confidence intervals. Additionally, a review of the CDL reveals significant errors, some resembling salt-and-pepper noise, even within the 100% confidence interval. These inaccuracies negatively impact deep learning (DL) data outcomes, necessitating the removal of erroneous CDLs through refining techniques. While some studies [9,14] have attempted to address this issue, fully accurate results have yet to be achieved, indicating a need for further research and the development of alternative approaches.

To refine the CDL and improve DL training outcomes, this research had two primary objectives. The first objective was to establish effective confidence intervals for refining the CDL to enhance DL segmentation accuracy. Various confidence interval values were applied to the CDL of major crops in the study area, and the resulting data were used as the ground-truth in a DL model. The effectiveness of these intervals was analyzed using multi-criteria decision analysis (MCDA) to determine the best approach for future applications. Additionally, the generalizability of these confidence intervals was tested in a different study area, confirming the robustness of our method across different geographical regions. The second objective was to evaluate the effectiveness of different image filtering techniques in mitigating errors within the CDL. By applying these filters, we assessed their contribution to improving the quality of the CDL and subsequently enhancing DL model accuracy for mapping croplands using Sentinel-2 imagery. This research also involved comparing our methods with the refined CDL (R-CDL) from Lin et al. [9], further demonstrating the efficiency and practicality of our approach.

2. Materials and Methods

2.1. Study Area

In this study, we focused on a specific region within the Mississippi Delta, highlighted in Figure 1, selected for its agricultural diversity and distinct seasonal phenological stages [20]. In 2021, soybeans were predominantly cultivated in this area, covering 31.7% of the landscape, along with significant portions of corn, cotton and rice, illustrating its agricultural variety. This diverse region, covering approximately 575,908 square kilometers and featuring various land cover types and surface features, provides an ideal environment for evaluating CDL refinements and the effectiveness of image filtering techniques in improving DL segmentation accuracy for precise agricultural monitoring.

To evaluate the generalizability of our method and conclusions, we conducted additional testing in a different geographical region named T14TNK, identified by its UTM Zone 14 within the Military Grid Reference System (MGRS). This area, covering approximately 100 km by 100 km, is located between Nebraska and Kansas, defined by the coordinates 97°42′ W–99°0′ W and 39°40′ N–40°38.5′ N, and includes parts of the Republican River. The major crops in this region are corn and soybeans, making it an ideal candidate for validating our methodology across different agricultural landscapes. By applying the same confidence interval and image filtering techniques, we aimed to assess the consistency and reliability of our approach. The results from this additional region further support the applicability of our method to diverse agricultural settings.

2.2. Materials

2.2.1. Sentinel-2 Imagery

To utilize satellite images, we selected multispectral images from the European Sentinel-2 sensors. These images are widely used in large-scale vegetation extraction, both in research and various application systems, due to their comprehensive optical spectrum for crop identification and acceptable spatial resolution compared to the CDL [21,22]. The red (B4), green (B3), blue (B2) and near-infrared (B8) bands have a spatial resolution of 10 m. Additionally, the three red-edge bands (B5, B6 and B7), two short-wave infrared bands (B11, B12) and one narrow NIR band (B8a) offer spatial resolutions that can exceed 20 m, adding extra band attributes for classification [21,22,23,24]. Bands 1, 9 and 10 correspond to coastal aerosols, water vapor and cirrus clouds, respectively [22]. These bands have a 60 m resolution, which makes accurate recording of the reflectance of minor surface objects challenging. Therefore, after excluding bands 1, 9 and 10, we selected a total of 10 bands for our investigation.

Utilizing Google Earth Engine (GEE) [21,23], we processed monthly minimum composite datasets from May to September, comprising 872 tiles in Mississippi Delta and 29 tiles in T14TNK. This period aligns with the peak growing season in the study area, as indicated by the Normalized Difference Vegetation Index (NDVI) [25] in Figure 2. To provide a comprehensive view of vegetation dynamics, NDVI curves for the entire year of 2021 were generated, contrasting growing and non-growing periods. The NDVI for each major crop was calculated using Sentinel-2 satellite imagery, with cloud masking applied to remove cloudy pixels. The data were composited into 15-day intervals over 2021, and NDVI was computed. Stratified random sampling selected 10,000 sample points for each major crop based on the CDL with 100% confidence interval, extracting NDVI values to provide a detailed temporal profile for each crop.

2.2.2. Cropland Data Layer (CDL)

Training and validation ground-truth data were derived from the CDL. The USDA—NASS produces the CDL, which offers an annual crop type for the United States at a 30 m resolution with more than 100 crop classes [6]. Since 2006, the state-level CDL classification has been conducted using the supervised RuleQuest See5.0 [26] decision tree classification approach [27,28]. Each US state’s metadata include producer accuracy (PA), user accuracy (UA) and Kappa score, and the CDL is evaluated using a subset of ground-truth data. According to Boryan et al. [27], the primary crops in the CDL have excellent classification accuracy ranging from 85% to 95% [5]. The chart in Figure 3 displays the distribution of various land cover types in Mississippi Delta, highlighting major crops such as soybeans, corn, cotton and rice. Additionally, we included the “other” class to account for non-crop areas and different crop types from major crops.

To compare the effectiveness of our method, we utilized refined CDL data based on the methodology outlined in Lin et al.’s study [9]. This study aimed to identify and resolve inaccuracies in CDL crop classifications by employing a decision tree method to detect and refine questionable pixels using spatial and temporal crop information. The refined CDL (R-CDL) data were validated against high-resolution satellite images and USDA’s official acreage estimates, ensuring their robustness. By incorporating this refined dataset, we were able to benchmark our proposed methods against a well-validated standard, providing a comprehensive evaluation of our approach in relation to existing techniques.

2.2.3. CDL Confidence Layer

The confidence layer is an additional layer provided alongside the CDL, which was evaluated in this study. This layer geographically indicates the expected confidence associated with each output pixel based on the classification rules applied. It offers a visual representation of the distribution and magnitude of classification confidence or error, which is highly beneficial for users. The confidence layer represents classification accuracies as a confidence interval, where a rating of 100 signifies very high confidence, and a rating of 0 signifies very low confidence. Essentially, the confidence value measures how well a pixel fits into the decision tree ruleset, rather than the absolute accuracy of a specific pixel [5].

By utilizing the confidence layer, users can better assess the reliability of the CDL for different applications. In this study, we employed various confidence interval thresholds to refine the CDL, which was then used as the ground-truth for training DL models. The impact of these confidence intervals on the quality of training data and DL performance was systematically evaluated to identify the most effective threshold for accurate crop segmentation from Sentinel-2 imagery.

2.3. Research Approaches

This section delineates the research methodology, with a flowchart in Figure 4 summarizing the essential steps, inputs and outputs. The inputs include monthly minimum composites of Sentinel-2 time series during the growing season, while the outputs encompass various DL segmentation evaluation metrics and maps of major crops under different conditions of confidence layer data and CDL refinements. To enhance the quality of the CDL for use as the ground-truth in the DL, two approaches are proposed: utilizing different confidence intervals of the CDL confidence layer and applying various image filters to mitigate noise and errors. Initially, CDL crop data are refined using specified confidence intervals and overlaid with Sentinel-2 imagery to create a comprehensive database. The DL model is then trained on these datasets, with accuracy metrics compared through MCDA. In the second approach, different image filters are applied to the major crops of the CDL. These refined datasets are subsequently used to train the DL model, and the results are compared to identify the most effective refinement technique.

2.3.1. CDL Refinement Using the Confidence Layer

Given the focus on major crops, the initial step involves extracting crop data with varying confidence levels by querying the CDL under different confidence level values. For example, a +75% confidence layer includes crop data with confidence intervals of 75% and higher (e.g., 95%) while excluding data with confidence values below 75%. This method retains higher quality data and discards lower quality data. However, removing low-quality data can sometimes weaken the dataset for DL applications, whereas using all available data may negatively impact the DL by incorporating erroneous data. Numerous studies have explored varying confidence intervals within the confidence layer [13,19], while others have opted to utilize the entire dataset [20]. A critical aim of this research is to determine the most effective confidence interval for refining the CDL as ground-truth data in DL cropland segmentation. To address this, an experiment was designed using crop layer data with confidence intervals from 0% to 100%.

2.3.2. CDL Refinement Using Image Filters

Despite the CDL having defined confidence levels, substantial amounts of erroneous data persist, even at the 100% confidence level [9]. Visual inspection of data in respect of different crops reveals defects such as salt-and-pepper noise and errors [14,17]. These issues are apparent across various confidence intervals. Additionally, the CDL erroneously classifies field borders and roads as croplands, contributing to significant noise and inaccuracies. To address the pervasive issues of noise and errors within the CDL, the next step involves refining the CDL using an image filter to establish a more accurate ground-truth for DL applications. The aim of this process is to eliminate noise, incorrect border areas between fields and other unknown defects. Initially, each major crop from the CDL was separated, and various image filters were applied individually, including the following:

• Aggregation [29]: This filter smooths the data by combining small, isolated patches of classified pixels into larger, contiguous regions, thereby reducing salt-and-pepper noise.
• Boundary cleaning [30]: This filter simplifies the raster by smoothing zone boundaries using expansion (dilation) and shrinking (erosion) techniques. This process involves evaluating each input cell based on its orthogonal and diagonal neighbors to achieve the desired smoothing effect.
• Expand [31]: The expand filter increases the size of designated zones within a raster by a certain number of cells. These designated zones are treated as the foreground, while the rest are considered the background, allowing the foreground to grow into the surrounding background areas.
• Shrink [32]: The shrink filter designates specific zone values as the foreground, enabling their expansion into surrounding background zones. This process effectively reduces noise and minor misclassifications, resulting in a more accurate and conservative delineation of crop extents.
• Majority [33]: This filter reclassifies each pixel based on the majority class of its neighbors, helping to correct misclassified pixels by considering local context.
• Expand–shrink: This technique involves initially shrinking the classified regions in an image to remove noise and small misclassified areas and then expanding the remaining areas back to their original size. This process helps in retaining the core areas of the classified regions while minimizing the impact of noise and erroneous small regions.

After filtering, the refined major crop datasets were merged into a single comprehensive layer, similar to the original CDL. This process generated six distinct filtered datasets, each treated with different image filters. These improved data serve as a more reliable input for DL models, thereby enhancing the precision of cropland segmentation. These filtering techniques were selected for their ability to mitigate specific types of errors and noise present in the CDL.

2.4. Data Pre-Processing

A robust pre-processing workflow was applied to each monthly Sentinel-2 dataset using the GEE platform to optimize data for crop segmentation analysis. The initial step involved a cloud-and-shadow masking technique utilizing the satellite’s Scene Classification Layer (SCL) [34] and cloud probability data. This step effectively excluded atmospheric obstructions, enhancing image clarity. Following this, the workflow generated minimum composites for each month, selecting the pixel with the lowest reflectance value from all clear observations to minimize cloud interference. Despite a potential increase in shadow presence, this method prioritizes reducing cloud cover due to its significant impact on spectral data accuracy. Simultaneously, the major crops in the CDL were selected to focus on corn, cotton, rice and soybeans. This involved selectively masking the CDL to retain only these crops and clipping it to match the spatial extent of the Sentinel-2 composite. The 256 × 256 pixel grid sampling generated 977 samples in the study area and 211 in the test area, filtering out less informative samples to optimize data richness. The refined samples were randomly divided into training and validation groups in an 80–20% split and organized into batches of 64 for efficient DL model training.

2.5. Model Architecture and Training

For the crop segmentation task using Sentinel-2 imagery, the Pytorch-based DeepLabV3Plus [35] model with a ResNet34 encoder was employed. This model was selected due to its proven efficacy in semantic segmentation across various applications and ResNet34’s robust feature extraction capabilities [36]. The DeepLabV3Plus architecture, with its standard decoder and batch normalization, was adapted to suit the unique requirements of this dataset. The inputs consisted of 5 monthly minimum composites of Sentinel-2 imagery from May to September, each comprising 10 bands, resulting in a 50-layer input. This configuration captures the phenological changes in major crops. The model output was designed to classify five distinct classes corresponding to major crops (corn, cotton, rice, soybeans and other), optimizing it for detailed segmentation in the study area. The Adam optimizer [37] was used with a learning rate of 0.01. During training, cross-entropy loss was employed, with early stopping implemented after 50 epochs of no improvement to prevent overfitting.

2.6. Accuracy Metrics and Comparative Analysis

Overall accuracy (OA), user’s accuracy (UA), producer’s accuracy (PA) and F1-score (F1) [38,39] accuracy metrics were employed to evaluate the approaches.

$$\mathrm{O}\mathrm{A}={\displaystyle \frac{TP+TN}{\begin{array}{c}\mathrm{Totali}\mathrm{Instances}\\ \end{array}}}$$

(1)

$$\mathrm{U}\mathrm{A}={\displaystyle \frac{TP}{\begin{array}{c}TP+FP\\ \end{array}}}$$

(2)

$$\mathrm{P}\mathrm{A}={\displaystyle \frac{TP}{\begin{array}{c}TP+FN\\ \end{array}}}$$

(3)

$$\mathrm{F}1=2\times {\displaystyle \frac{\mathrm{U}\mathrm{A}\times \mathrm{P}\mathrm{A}}{\begin{array}{c}\mathrm{U}\mathrm{A}+\mathrm{P}\mathrm{A}\\ \end{array}}}$$

(4)

where TP represents the number of true positives; FP represents the number of false positives; FN represents the number of false negatives; and TN represents the number of true negatives.

Also, a confusion matrix (error matrix) [38,40] is used to evaluate the performance of our DL models in classifying different crop types. A confusion matrix is a table that outlines the performance of the model by comparing the actual classes to the predicted classes. Each cell in the matrix represents the proportion of instances (expressed as a percentage) that were correctly or incorrectly classified by the model. Values range from 0 to 100, indicating the percentage of instances that fall into each category. For instance, a value of 95 in the matrix indicates that 95% of the instances of a particular class were correctly predicted, whereas a value of 0.005 might indicate a small proportion of misclassifications. These values provide a detailed insight into the model’s accuracy, UA, PA and overall performance in distinguishing between different crop types.

In addition, MCDA in this context involved evaluating multiple criteria (accuracy metrics) across various confidence interval alternatives in the DL metric results. Let A represent the set of confidence interval alternatives and C represent the set of criteria such as UA, PA and F1 [38,39]. Each criterion $c\in C$ was normalized to bring different accuracy metrics onto a comparable scale. The normalization process for a criterion c for alternative a was calculated as

$$n_{ac}={\displaystyle \frac{x_{ac}}{\sqrt{{\sum }_{a\in A}{x}_{ac}^2}}}$$

(5)

where $x_{ac}$ is the original value of criterion c for alternative a, and $n_{ac}$ is the normalized value. Weights $w_c$ were assigned to each criterion c, reflecting its importance in the decision-making process:

$$w_c={\displaystyle \frac{1}{\left|C\right|}}$$

(6)

The weighted normalized value for each criterion of each alternative was calculated as:

$$v_{ac}=n_{ac}·w_{c_{}}$$

(7)

Identification of the ideal best (A+) and ideal worst (A−) was as follows:

$$A_c^{+}={\mathrm{m}\mathrm{a}\mathrm{x}}_{a\in A}{v}_{ac}\mathrm{a}\mathrm{n}\mathrm{d}{A}_c^{-}={\mathrm{m}\mathrm{i}\mathrm{n}}_{a\in A}{v}_{\begin{array}{c}ac\\ \end{array}}$$

(8)

Calculation of the distance to ideal best (D+) and ideal worst (D−) was as follows:

$$D_a^{+}=\sqrt{{\sum }_{c\in C}{(v_{ac}-A_c^{+})}^2}\mathrm{a}\mathrm{n}\mathrm{d}{D}_a^{-}=\sqrt{{\sum }_{c\in C}{(v_{ac}-A_c^{-})}^2}$$

(9)

The relative closeness of each alternative to the ideal solution was calculated as

$${RC}_a={\displaystyle \frac{D_a^{-}}{D_a^{+}+D_{\begin{array}{c}a\\ \end{array}}^{-}}}$$

(10)

Alternatives were then ranked based on their relative closeness to the ideal solution, with a higher ${RC}_a$ indicating a closer proximity to the ideal, and thus, a higher preference. MCDA systematically analyzes DL model performance across different confidence intervals and image filters, integrating metrics like OA, UA, PA and F1.

For the generalizability analysis, significance analysis experiments were conducted to quantify the similarity and statistical significance of the results between the Mississippi Delta and T14TNK areas. The Pearson correlation coefficient [40,41] and paired t-tests [39] were employed to compare the performance of different refining methods between these two regions. The Pearson correlation coefficient measures the strength and direction of the linear relationship between two variables. The paired t-test is used to determine whether the means of two paired samples differ significantly. The p-value associated with the T-statistic indicates whether the difference between the pairs is statistically significant. To ensure the comparison focuses on trends rather than absolute values, data normalization is performed.

$$\mathrm{T}\text{-}\mathrm{s}\mathrm{t}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{s}\mathrm{t}\mathrm{i}\mathrm{c}={\displaystyle \frac{\overline{d}}{\raisebox{1ex}{$s_d$}\!\left/ \!\raisebox{-1ex}{$\sqrt{n}$}\right.}}$$

(11)

In the T-statistic, $\overline{d}$ is the mean of the differences between paired observations; $s_d$ is the standard deviation of the differences; and n is the number of pairs.

$$\mathrm{P}\mathrm{e}\mathrm{a}\mathrm{r}\mathrm{s}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{o}\mathrm{r}\mathrm{r}\mathrm{e}\mathrm{l}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{o}\mathrm{n}\mathrm{c}\mathrm{o}\mathrm{e}\mathrm{f}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}={\displaystyle \frac{\sum \left(x_i-\overline{x}\right)\left(y_i-\overline{y}\right)}{\sqrt{\sum {\left(x_i-\overline{x}\right)}^2\sum {\left(y_i-\overline{y}\right)}^2}}}$$

(12)

where $x_i$ and $y_i$ are the individual sample points, and $\overline{x}$ and $\overline{y}$ are the means of the sample sets x and y, respectively. A Pearson correlation coefficient close to 1 indicates a strong positive correlation, while a value close to −1 indicates a strong negative correlation.

3. Results

To implement the proposed approaches, the research was divided into three phases. The first phase involved evaluating the approaches using the CDL with varying confidence levels. In the second phase, different image filters were applied to the CDL, and the DL segmentation results were subsequently evaluated. In the third phase, the methodologies were tested on an additional test area, T14TNK, to compare and validate the effectiveness of the approaches in a different geographical and crop context.

3.1. Impact of CDL Confidence Intervals on Segmentation

The confidence interval percentages applied to the major crop of the CDL result in a progressive reduction in the number of crop pixels within the image. As the confidence interval increases toward 100%, the number of pixels representing crops correspondingly decreases. Figure 5 quantifies this reduction, displaying the area (in hectares) occupied by different crops at each confidence interval. Notably, in some scenarios, the pixel count can be reduced by up to two times. It is crucial to understand the impact of this reduction on machine learning applications, as it directly affects the accuracy and reliability of DL segmentation outcomes. Figure 6 illustrates the distribution of major crop pixels at various confidence intervals, showing how increased confidence thresholds progressively refine the CDL.

Table 1. Performance metrics of the DL model using CDL datasets at different confidence interval percentages. The metrics include overall accuracy (OA), user’s accuracy (UA), producer’s accuracy (PA) and F1-score (F1) and MCDA relative closeness. “All CDL” refers to using the entire CDL without any confidence interval filtering, serving as a baseline for comparison.

Confidence Interval	Overall				Corn			Cotton			Rice			Soybeans			Other
	OA	UA	PA	F1	UA	PA	F1	UA	PA	F1	UA	PA	F1	UA	PA	F1	UA	PA	F1	MCDA
All CDL	89.4	89.5	89.7	89.6	88.7	88.1	88.4	88.8	91.9	90.3	93.3	91.4	92.3	85.6	86.0	85.8	91.3	91.1	91.2	85.8
+5%	89.5	89.2	89.9	89.5	89.6	86.8	88.2	87.0	93.2	90.0	91.6	92.8	92.2	87.1	84.6	85.8	90.7	92.0	91.4	84.8
+15%	89.7	90.3	89.7	90.0	89.7	87.7	88.7	90.1	91.9	91.0	93.0	92.3	92.6	89.1	83.0	85.9	89.5	93.6	91.5	86.4
+25%	89.9	90.2	90.0	90.1	89.7	88.0	88.8	89.7	92.2	91.0	93.0	92.7	92.8	88.9	83.5	86.1	89.9	93.4	91.6	87.5
+35%	90.0	90.5	89.8	90.2	90.8	87.7	89.2	90.2	91.8	91.0	94.0	91.5	92.7	86.5	85.9	86.2	91.1	92.3	91.7	88.0
+45%	90.5	90.7	90.7	90.7	90.1	89.7	89.9	90.8	91.6	91.2	93.0	93.8	93.4	88.3	85.6	86.9	91.4	92.8	92.1	92.3
+55%	90.6	90.0	91.4	90.7	90.7	89.3	90.0	89.0	93.2	91.0	91.0	95.7	93.3	86.8	87.7	87.2	92.6	91.2	91.9	90.8
+65%	90.6	89.8	91.2	90.4	88.6	91.0	89.8	88.6	92.6	90.6	90.6	95.2	92.8	89.4	84.6	87.0	91.5	92.3	91.9	90.1
+75%	90.3	89.0	90.6	89.8	86.5	92.4	89.4	87.1	91.8	89.3	92.2	91.5	91.9	87.2	85.9	86.5	92.1	91.4	91.7	86.2
+85%	88.6	86.2	87.7	86.9	84.5	88.2	86.3	84.1	85.1	84.5	87.4	92.7	89.9	84.4	81.8	83.1	90.9	90.7	90.8	68.1
+95%	84.7	77.0	79.6	78.1	73.6	81.6	77.4	71.5	75.1	73.2	77.1	87.3	81.9	73.5	64.5	68.7	89.1	89.4	89.2	8.9
+100%	84.9	76.3	79.5	77.8	72.7	81.8	77.0	70.4	75.1	72.6	77.3	82.6	79.8	71.3	69.2	70.2	89.9	88.9	89.4	8.4

displays the outcomes (OA, UA, PA and F1) of the DL implementation using different CDL datasets at varying confidence interval percentages. Additionally, the MCDA relative closeness values are provided to indicate the effectiveness of each confidence interval. The best overall performance can be observed at confidence intervals of +45%, +55% and +65%, with OAs of 90.5%–90.6%, indicating that these intervals effectively balance the inclusion of high-quality data while excluding erroneous data. Conversely, the lowest performances are seen at +85%–100% confidence intervals, with OAs less than 88.6%, indicating that excluding too much data adversely impacts the model’s performance.

For crop-specific performance, corn and soybeans exhibited the highest F1 at the 55% confidence interval, while cotton and rice achieved the highest F1 at the 45% confidence interval. The MCDA values corroborate these results, showing the highest relative closeness at the 45% and 55% intervals. This reinforces the conclusion that mid-range confidence intervals provide the effective balance by retaining high-quality data and excluding noise. These findings suggest that the 45% and 55% confidence intervals for the CDL strike the best balance, significantly enhancing the DL model’s accuracy and reliability for agricultural mapping applications.

The results of the DL segmentation, trained using data with various confidence intervals and the entire CDL dataset, are illustrated in Figure 7. In the segmentation results utilizing the entire dataset without the confidence layer, noise is effectively mitigated, and the original fields are clearly delineated. However, the method encounters challenges in accurately identifying roads and gaps between agricultural fields. In several instances, the segmentation erroneously classified non-cropland areas as cropland, which represents a significant limitation. Conversely, the results for the +85% and higher confidence intervals, while successfully removing noise, also erroneously exclude numerous agricultural fields.

Conversely, the segmentation results using the +35% to +75% confidence intervals are highly satisfactory. These datasets accurately depict the fields in their entirety, correctly identify gaps between the fields and eliminate non-agricultural areas. There are slight variations between the +45% and +55% confidence intervals; in some cases, the +45% interval outperforms, while in others, the +55% interval yields better results. This suggests that a confidence level within this range could provide the most effective segmentation performance, balancing accuracy and reliability.

The confusion matrices for different confidence intervals of the CDL in Figure 8 reveal various impacts on the performance of DL models in segmenting major crops. The All CDL matrix shows balanced performance across crops but contains some misclassifications, particularly in distinguishing corn and soybeans. As the confidence interval increases to +45% and +55%, there is a slight improvement in the identification of cotton and rice, evidenced by higher UA values in these categories. The +55% confidence level shows a noticeable enhancement in OA, especially for corn and soybeans, reducing false positives and false negatives significantly. The +65% interval continues this trend, with improved accuracy for cotton and rice. However, at the +95% and 100% confidence levels, there is a marked decline in performance across all crops, with increased misclassification rates, particularly for soybeans, indicating that overly stringent confidence levels may exclude valuable data, thereby impairing model accuracy.

3.2. Evaluating Filtered CDL

To address the discrepancies in CDL quality, various image filters were applied to enhance the accuracy of the DL models. The aim of these filters is to reduce noise and correct errors within the CDL dataset, ensuring a more reliable ground-truth for DL training.

Table 2. Performance metrics of DL models using filtered CDL with various image filters. “R-CDL” refers to the CDL refined using the method described in Lin et al.’s study [9]. “No Filter” is equivalent to “All CDL” in Table 1.

	Overall				Corn			Cotton			Rice			Soybeans			Other
Filters	OA	UA	PA	F1	UA	PA	F1	UA	PA	F1	UA	PA	F1	UA	PA	F1	UA	PA	F1	MCDA
Aggregate	90.1	89.5	89.0	89.2	87.3	88.8	88.1	88.7	88.9	88.8	91.8	91.1	91.4	89.2	83.5	86.3	90.6	92.8	91.7	52.9
Boundary Clean	90.3	91.6	90.0	90.7	92.9	85.2	88.9	92.6	91.9	92.2	94.5	91.9	93.2	86.3	88.8	87.5	91.4	92.1	91.8	66.0
Expand	86.7	87.9	85.4	86.5	88.4	78.5	83.1	89.4	85.0	87.2	90.4	87.1	88.7	85.6	81.8	83.7	85.4	94.7	89.8	14.7
Expand–Shrink	91.7	91.5	91.7	91.6	91.7	90.3	91.0	90.1	93.6	91.8	93.2	93.9	93.5	90.7	87.3	89.0	92.0	93.5	92.8	84.0
Majority	91.4	91.8	91.8	91.8	90.2	92.0	91.1	92.5	93.2	92.8	94.0	94.2	94.1	90.9	86.0	88.4	91.4	93.8	92.6	83.8
Shrink	91.4	89.4	89.4	89.3	89.5	85.3	87.4	86.8	89.0	87.9	90.0	93.0	91.4	87.4	85.9	86.6	93.2	93.8	93.5	51.3
R-CDL [9]	90.7	90.8	91.3	91.0	90.0	90.3	90.2	90.0	94.5	92.2	93.5	92.5	93.0	88.8	86.1	87.4	91.4	92.8	92.1	75.9
No Filter	89.4	89.5	89.7	89.6	88.7	88.1	88.4	88.8	91.9	90.3	93.3	91.4	92.3	85.6	86.0	85.8	91.3	91.1	91.2	58.1

presents the outcomes for each filter in terms of OA, UA, PA and F1 for the major crops. The “No Filter” and “R-CDL [9]” results in Table 2 are included for comparison. It should be noted that “No Filter” corresponds to “All CDL” in Table 1.

Applying different image filters to the CDL significantly improved DL model performance over the No Filter baseline. The majority filter and expand–shrink filter each enhanced the OA and F1 by approximately 2%, with similar improvements in user and producer accuracy across all crops. Both filters demonstrated superior effectiveness in refining the CDL for DL applications. The boundary clean filter provided notable enhancements, particularly in OA and F1, showing an approximate 1% improvement over the No Filter results. Although the shrink filter also delivered improvements, it was slightly less effective than the majority and expand–shrink filters, with a modest increase in the OA and F1. The results for R-CDL from Lin et al.’s study [9] show that it achieves competitive performance, indicating its effectiveness in improving CDL quality. Conversely, the expand filter resulted in the lowest performance, highlighting the importance of choosing appropriate filters to avoid amplifying noise. The application of these filters, especially the majority and expand–shrink filters, proved crucial in enhancing the quality of the CDL, thereby improving the DL model’s accuracy in segmenting agricultural fields.

The visual comparison of the segmentation results using different image filters on the CDL, as shown in Figure 9, reveals several insights. The No Filter column depicts the raw CDL, displaying significant noise and misclassification, especially along the boundaries of crop fields and roads. The aggregate filter, while effective at smoothing out some noise, still maintains a distinct separation between crop fields and adjacent features, suggesting good performance in boundary areas. Conversely, the shrink filter, although it reduces noise, tends to exaggerate the separation between crop fields and roads, sometimes over-segmenting these areas. The expand filter performs the poorest, as it amplifies noise and leads to over-segmentation, failing to accurately delineate crop fields from non-crop areas. The boundary clean filter, intended to enhance edge detection, shows weaknesses in accurately defining the boundaries of fields, often merging them with surrounding areas, which reduces its effectiveness in precise segmentation.

Overall, the majority and expand–shrink filters stand out. The majority filter shows the highest accuracy in maintaining the integrity of crop fields and effectively reducing noise, making it the best performer overall. The expand–shrink filter also performs well, particularly in maintaining clear boundaries between crop fields and roads, indicating a robust capability in handling complex boundary definitions. These filters significantly improve the segmentation results compared to the No Filter approach, with noticeable enhancements in the clarity and accuracy of the segmented crop fields. This visual analysis aligns with the numerical results, confirming that appropriate filtering techniques can substantially enhance the quality of the CDL for DL applications.

In Figure 10, the confusion matrices for different image filters applied to the CDL reveal distinct variations in the performance of DL models across major crops. The majority filter shows robust performance, particularly in identifying rice and soybeans, with higher UA and PA values compared to the other filters. This filter effectively reduces false positives and false negatives, making it highly reliable for segmenting various crop types. The expand–shrink filter also performs well, particularly in accurately identifying corn and cotton, though its performance slightly lags behind the majority filter.

Conversely, the expand filter exhibits the weakest performance, as evidenced by its higher rates of misclassification, particularly for corn and rice. The boundary clean filter shows moderate effectiveness, with relatively good accuracy for soybeans but less consistency for corn. The aggregate filter performs reasonably well but struggles with distinguishing between crops and non-crop areas, particularly in boundary regions. The majority and expand–shrink filters stand out as the most effective in improving DL segmentation accuracy for the major crops, highlighting their value in enhancing the quality of the CDL for agricultural applications.

3.3. CDL Refinement in T14TNK Area

To ensure the robustness and generalizability of our proposed methodologies in a different geographical and agricultural context, we conducted an additional experiment in the T14TNK test area. The performance metrics of the DL models, trained with the CDL at different confidence intervals, are summarized in

Table 3. Performance metrics of DL models trained with the CDL at different confidence intervals for the T14TNK area.

Confidence Interval	Overall				Corn			Soybeans			Other
	OA	UA	PA	F1	UA	PA	F1	UA	PA	F1	UA	PA	F1	MCDA
All CDL	92.7	91.9	92.2	92.1	92.4	90.3	91.4	88.7	91.8	90.2	94.5	94.6	94.6	92.0
+5%	92.7	92.0	92.1	92.0	91.6	90.9	91.2	90.1	90.6	90.3	94.5	94.7	94.6	92.2
+15%	92.7	92.0	92.2	92.1	93.2	89.7	91.4	88.8	91.8	90.3	94.1	95.1	94.6	91.2
+25%	92.7	92.1	92.1	92.1	92.8	90.0	91.4	89.4	91.3	90.3	94.1	95.0	94.5	92.3
+35%	92.8	92.0	92.3	92.2	91.9	91.1	91.5	89.3	91.5	90.4	94.8	94.4	94.6	93.6
+45%	93.1	92.3	92.6	92.5	92.2	91.5	91.8	89.9	91.7	90.8	94.9	94.6	94.8	96.1
+55%	92.9	92.1	92.5	92.3	91.8	92.1	92.0	89.9	91.2	90.6	94.7	94.0	94.3	94.2
+65%	92.6	91.8	91.9	91.9	91.9	91.9	91.9	89.8	90.3	90.0	93.8	93.6	93.7	91.0
+75%	91.8	90.4	91.6	91.0	91.0	91.4	91.2	86.4	91.2	88.7	93.9	92.2	93.1	82.1
+85%	90.4	88.8	89.1	89.0	88.2	89.9	89.1	85.7	85.7	85.7	92.5	91.8	92.2	63.6
+95%	87.3	82.9	83.4	83.1	80.7	81.7	81.2	77.0	77.8	77.4	91.1	90.6	90.8	6.0
+100%	87.5	82.7	84.1	83.4	78.8	84.8	81.7	77.4	77.5	77.5	91.9	90.0	90.9	9.4

. Notably, the results indicate that the 45% and 55% confidence intervals yield the highest accuracy, suggesting that these intervals effectively balance data quality and noise reduction. This result aligns with our findings in the Mississippi Delta study area, confirming the effectiveness of these confidence intervals across different regions. Conversely, higher confidence intervals, such as 95% and 100%, exhibit the worst results, likely due to excessive exclusion of valid data, thus supporting the importance of selecting appropriate confidence levels to enhance model performance.

The performance metrics of DL models using different image filters on the CDL for the T14TNK area are summarized in

Table 4. Performance metrics of DL models using various image filters on the CDL for the T14TNK area. The R-CDL [9] results are included for comparison, demonstrating the effectiveness of different filtering techniques in enhancing CDL quality.

	Overall				Corn			Soybeans			Other
Filters	OA	UA	PA	F1	UA	PA	F1	UA	PA	F1	UA	PA	F1	MCDA
Aggregate	92.2	90.7	91.5	91.1	90.5	91.6	91.1	87.2	90.0	88.6	94.2	93.0	93.6	70.3
Boundary Clean	92.5	92.4	91.5	91.9	92.2	96.2	94.1	94.0	88.5	91.2	91.1	89.7	90.4	74.8
Expand	85.6	86.0	85.0	84.6	80.8	98.6	88.8	97.5	73.1	83.5	79.6	83.4	81.5	45.6
Expand–Shrink	93.1	91.8	93.3	92.5	95.7	92.7	94.2	91.3	93.8	92.5	88.2	93.6	90.8	77.7
Majority	93.5	92.7	93.2	93.0	95.4	94.2	94.8	91.8	93.2	92.5	91.1	92.2	91.6	80.7
Shrink	85.6	79.8	91.2	83.7	98.7	79.5	88.1	72.9	97.2	83.3	67.8	96.9	79.8	41.8
R-CDL [9]	92.7	92.2	92.1	92.1	92.1	90.9	91.5	90.5	90.9	90.7	93.9	94.4	94.1	75.5
No Filter	92.7	91.9	92.2	92.1	92.4	90.3	91.4	88.7	91.8	90.2	94.5	94.6	94.6	74.5

. The results demonstrate that the majority and expand–shrink filters significantly enhance model accuracy, with the OA reaching 93.5% and 93.1%, respectively. The majority filter showed the highest effectiveness, achieving the best results across most metrics. The expand–shrink filter also performed well, indicating that it is a reliable method for improving CDL data quality. In contrast, the expand filter yielded the lowest performance metrics, highlighting the importance of selecting appropriate filtering techniques. The results of the R-CDL from Lin et al.’s study [9] also show competitive accuracy, supporting the robustness of our proposed approaches. These findings align with our previous results in the Mississippi Delta, confirming the generalizability and effectiveness of our methodology across different regions.

4. Discussion

4.1. CDL Refinement Using Confidence Layer

The superior performance observed with the +45% and +55% confidence intervals, as indicated by the highest F1 and MCDA relative closeness, suggests that this level effectively balances the inclusion of high-quality data while minimizing noise. This interval likely provides a robust training set that enhances the model’s ability to generalize accurately across different validation samples. In contrast, the decline in performance at the +85%–100% confidence intervals can be attributed to the excessive exclusion of data, which results in a training set that is too limited to capture the full variability in the croplands, thereby impairing the model’s performance.

The MCDA relative closeness plays a crucial role in explaining these results by providing a comprehensive evaluation that integrates multiple criteria, such as OA, UA, PA and F1. This holistic approach allows for a more nuanced assessment of the model’s performance across different confidence intervals. The high MCDA scores for the +45% and +55% intervals underscore their overall effectiveness, balancing various performance metrics more successfully than other intervals. The insights gained from MCDA highlight the importance of selecting an effective confidence interval that not only maximizes individual accuracy metrics but also achieves a harmonious balance across multiple evaluation criteria, leading to a more reliable and robust model for cropland mapping. This emphasizes the necessity for a strategic approach in choosing confidence intervals to enhance the quality and effectiveness of DL segmentation models in agricultural applications.

4.2. CDL Refinement Using Image Filters

The results of the DL segmentation using various image filters on the CDL underscore the importance of data pre-processing in improving model accuracy. The majority and expand–shrink filters emerged as the top performers due to their ability to effectively reduce noise and preserve the integrity of crop boundaries. The success of the majority filter can be attributed to its ability to smooth out irregularities and enhance the homogeneity of crop areas, thereby improving the model’s ability to accurately classify different crop types. This filter’s effectiveness in maintaining clear distinctions between crops and non-crop areas, particularly along field boundaries, is crucial for accurate segmentation. Similarly, the expand–shrink filter’s ability to manage the complexities of field boundaries and reduce noise without over-segmenting highlights its robustness in handling diverse agricultural landscapes.

However, the poorer performance of the expand filter highlights the risks of amplifying noise during the pre-processing stage. This filter’s tendency to over-segment and misclassify non-crop areas as crops underscores the need for careful selection of filtering techniques. The boundary clean filter’s moderate performance points to its partial success in edge detection, which, while beneficial, may not be sufficient alone for accurate segmentation. The results indicate that a nuanced approach to data filtering, which balances noise reduction and boundary preservation, is essential for optimizing DL model performance in crop segmentation tasks.

4.3. Comparative Analysis of CDL Refinement Methods

Among the various methods tested, DL segmentation trained with data processed by the majority filter yielded the best results. The findings indicate that approaches utilizing filtered data outperform those using different confidence intervals, with the second-best results achieved using the expand–shrink filter. Conversely, the approaches using data with +95% and 100% confidence intervals produced the worst results, despite their frequent use in other studies. This suggests that high confidence intervals may exclude too much data, leading to poor model performance. Similarly, the DL model trained with expanded data showed significant weaknesses in correct segmentation, with results comparable to the worst-case confidence interval scenarios.

The substantial difference in accuracy between the filter-based approaches and the confidence-interval-based approaches, often around 1%, highlights the importance of pre-processing the CDL with appropriate filters before training DL models. This finding suggests that the application of noise reduction filters on the CDL should be a preliminary step to improve DL performance. Furthermore, combining confidence intervals and filter approaches could potentially enhance CDL quality even further. However, this combined approach requires careful evaluation, as the simultaneous use of filters and confidence intervals might have unforeseen negative effects on DL results.

In comparing the results of our best-performing approach with the R-CDL from Lin et al.’s study [9], our findings demonstrate notable improvements. The R-CDL, which involves sophisticated steps to refine the CDL, was outperformed by the expand–shrink and majority filter approaches, which scored around 1% higher in various accuracy metrics. While our methods using the +45% and +55% confidence intervals resulted in slightly lower accuracies, they still demonstrated significant efficacy. This comparison, as illustrated in the accompanying chart in Figure 11, underscores that our simpler filter-based approaches can achieve superior or comparable accuracy metrics without the complexity of the R-CDL method, thus validating the robustness and effectiveness of our methodologies.

4.4. Generalizability Assessment

To assess the generalizability of our method across different study areas, we conducted a detailed comparative analysis using the F1-score. We used the F1-score as the primary accuracy metric for comparison because it balances precision and recall, providing a comprehensive measure of model performance, especially in the context of imbalanced classes. The study areas included the Mississippi Delta and the T14TNK region. We first normalized the F1-scores to ensure that the trends were the main focus rather than the absolute values. The Pearson correlation coefficient and paired t-tests were employed to quantify the similarity and statistical significance of the results between these areas. This approach allowed us to understand whether the trends observed in one region could be replicated in another, thus validating the robustness of our method.

The comparison yielded promising results. For all the approaches, the Pearson correlation coefficient was 0.82, indicating a strong positive correlation between the two study areas. The paired t-test resulted in a T-statistic of 3.14 × 10⁻¹⁴ and a p-value of 1, suggesting no significant difference between the means of results in the two regions. When focusing on the confidence-interval-based methods, the correlation coefficient was an impressive 0.99, with a T-statistic of 3.68 × 10⁻¹⁵ and a p-value of 1, reinforcing the high similarity and consistency of the results. For the filter-based methods, the correlation coefficient was 0.76, and the paired t-test yielded a T-statistic of 1.04 × 10⁻¹⁴ and a p-value of 1, indicating a robust but slightly less strong correlation compared to the confidence interval methods. This is likely due to the poor performance of the shrink filter in the T14TNK area, which could be attributed to the major crops in that region being corn and soybeans. The shrink filter may have had a detrimental effect on accurately segmenting these crops.

A p-value of almost 1 indicates no statistically significant difference between the results, suggesting that the methods’ performance in the Mississippi Delta and T14TNK regions is statistically indistinguishable. This reinforces the robustness and generalizability of the methods, showing consistent performance across different regions. Although the trend is similar, the results for T14TNK are generally 1–2% better, except for the shrink and expand filters. This improvement might be due to the DL model having fewer classes to differentiate, allowing it to perform better in T14TNK. Additionally, the major crops in T14TNK, corn and soybeans, may be easier to classify accurately in this region due to their distinct spectral signatures. The chart in Figure 12 visually represents these trends, confirming that our approach is highly generalizable across different geographical regions. To thoroughly analyze the generalizability of our approach, it is essential to test the methods in multiple study areas, which should be undertaken in future research.

4.5. Implementation Prospects and Limitations

By integrating the refined CDL with Sentinel-2 imagery, this approach provides a robust framework for creating more accurate and reliable cropland maps. These advancements could play a critical role in agricultural monitoring, enabling better resource management and decision-making processes. The potential to combine these methods with advanced pre-processing techniques further amplifies their utility, potentially leading to more precise segmentation models that can benefit a wide range of agricultural applications.

However, the study has several limitations that must be addressed in future research to fully validate and optimize these methods. A significant constraint is the reliance on the CDL as the sole ground truth, which limits the ability to independently verify the results. Access to additional ground-truth data from different sources would enable a more comprehensive validation of the refined CDL and the effectiveness of the applied filters. Additionally, testing the methods in multiple study areas is essential to ensure that our findings are robust and applicable across different geographical regions. Furthermore, while the current filtering methods have shown efficacy, the development and integration of more advanced filtering techniques could further enhance data quality and segmentation accuracy. Future research should focus on exploring and validating these advanced methods, as well as establishing collaborations to obtain diverse ground-truth datasets, to overcome these limitations and improve the robustness of DL-based cropland segmentation models.

5. Conclusions

Numerous studies have leveraged the CDL and satellite imagery for the automation of cropland classification and segmentation using machine learning models. Despite its utility, the CDL is marred by errors such as salt-and-pepper noise, which adversely affect the performance of DL models in cropland segmentation. Consequently, refining the CDL to mitigate these inaccuracies is crucial. In this research, we investigated two primary approaches for refining the CDL: applying varying confidence intervals to the CDL and employing different image filters. For the time series data, five monthly minimum composites of Sentinel-2 imagery from the Mississippi Delta were used as satellite data for training and testing. To further validate our methods, we tested the approach in a new study area (T14TNK).

The analysis indicates that the CDL with confidence intervals ranging from +45% to +55% yielded the best results in terms of OA, enhancing the accuracy by approximately 1% compared to using the unfiltered CDL. The best results were observed with the CDL processed using the majority and expand–shrink filters, which improved the evaluation metrics by about 1.5%. The application of MCDA further corroborated these findings by providing a comprehensive evaluation across multiple criteria, underscoring the overall efficacy of the majority filter.

The results from T14TNK confirmed the generalizability of our method, showing similar trends in accuracy improvements. The performance metrics in T14TNK were generally around 1%–2% higher compared to the Mississippi Delta, likely due to fewer crop classes in T14TNK, which may have allowed the DL model to perform better. The Pearson correlation coefficient and paired t-tests were employed to quantify the similarity and statistical significance of the results between these areas. The Pearson correlation coefficient for confidence interval methods was 0.99, indicating a very strong correlation, while the paired t-tests yielded a T-statistic close to 0 and a p-value of 1 after normalization, suggesting no significant difference between the performance in the two regions. This reinforces the robustness and generalizability of our methods across different geographical regions.

Based on the findings of this research, it is recommended to filter the CDL with image filters such as majority or expand–shrink before employing it for DL segmentation training to identify crops. Additionally, selecting data with confidence intervals of +45% prior to applying these filters can further enhance the quality and reliability of the training data. Future research should explore the synergistic effects of combining filtering and confidence interval techniques and investigate the potential of integrating advanced CDL pre-processing methods to augment the performance of DL models in agricultural applications. This holistic approach could lead to more precise and dependable segmentation models, thereby benefiting various agricultural mapping and monitoring tasks.