Winter Wheat Mapping Method Based on Pseudo-Labels and U-Net Model for Training Sample Shortage

Abstract

In recent years, the semantic segmentation model has been widely applied in fields such as the extraction of crops due to its advantages such as strong discrimination ability, high accuracy, etc. Currently, there is no standard set of ground true label data for major crops in China, and the visual interpretation process is usually time-consuming and laborious. The sample size also makes it difficult to support the model to learn enough ground features, resulting in poor generalisation ability of the model, which in turn makes the model difficult to apply in fine extraction tasks of large-area crops. In this study, a method to establish a pseudo-label sample set based on the random forest algorithm to train a semantic segmentation model (U-Net) was proposed to perform winter wheat extraction. With the help of the GEE platform, Winter Wheat Canopy Index (WCI) indicators were employed in this method to initially extract winter wheat, and training samples (i.e., pseudo labels) were built for the semantic segmentation model through the iterative process of “generating random sample points—random forest model training—winter wheat extraction”; on this basis, the U-net model was trained with multi-time series remote sensing images; finally, the U-Net model was employed to obtain the spatial distribution map of winter wheat in Henan Province in 2022. The results illustrated that: (1) Pseudo-label data were constructed using the random forest model in typical regions, achieving an overall accuracy of 97.53% under validation with manual samples, proving that its accuracy meets the requirements for U-Net model training. (2) Utilizing the U-Net model, U-Net++ model, and random forest model constructed based on pseudo-label data for 2022, winter wheat mapping was conducted in Henan Province. The extraction accuracy of the three models is in the order of U-Net model > U-Net++ model > random forest model. (3) Using the U-Net model to predict the winter wheat planting areas in Henan Province in 2019, although the extraction accuracy decreased compared to 2022, it still exceeded that of the random forest model. Additionally, the U-Net++ model did not achieve higher classification accuracy. (4) Experimental results demonstrate that deep learning models constructed based on pseudo-labels exhibit higher classification accuracy. Compared to traditional machine learning models like random forest, they have higher spatiotemporal adaptability and robustness, further validating the scientific and practical feasibility of pseudo-labels and their generation strategies, which are expected to provide a feasible technical pathway for intelligent extraction of winter wheat spatial distribution information in the future.

1. Introduction

Since winter wheat is one of three major crops in China, timely and accurate acquisition of its spatial distribution would help relevant departments adjust agricultural planning structures, disaster monitoring, and field production management, which is the basis and premise for winter wheat yield estimation. Furthermore, it would also be of great significance for China to formulate food production policies and regulate crop planting structures, thereby achieving national food safety and promoting agricultural production and development [1,2,3]. However, traditional methods to acquire spatial distribution of winter wheat including level-by-level reports and sampling surveys had shortcomings such as low timeliness, low accuracy, and high investment costs, and were also easily affected by human subjective factors [4].

Remote sensing technology has advantages such as strong timeliness, a large amount of information, wide coverage, and low cost, which could provide effective methods and means for acquiring winter wheat spatial distribution data. In recent years, Sentinel series satellite remote sensing images have been widely used in research on ground object information extraction due to their high spatial and temporal resolution, rich spectral features, and wide bandwidth [5]. Rujoju-Mare et al. employed the spectral characteristics of Sentinel-2A images to distinguish land cover types in areas with higher and lower land heterogeneity based on the maximum likelihood method and the support vector machine, and both methods achieved high classification accuracy [6]; Aryal et al. established a normalized vegetation index (NDVI) based on Sentinel-2A image data, introduced a supervised classification method to classify urban vegetation in government areas across Victoria, and proved the effectiveness of this method [7]; Ali et al. introduced both Sentinel-2A and Landsat-8 image data respectively to classify land use and land cover, and the nearest neighbour algorithm, KD-Tree, the random forest, and other classification methods were introduced for comparison, with the results showing that Sentinel-2A images achieved the highest accuracy in the classification results of each algorithm [8]. The studies listed above have achieved high classification or extraction accuracy. However, most of the machine learning algorithms, such as the random forest, would have a shallow structure. Due to the limited network structure, these algorithms cannot effectively express complex surface types, which also makes it difficult to perform fine classification of surface types. When the number and diversity of animal samples increase, it is difficult for shallow learning methods to achieve good results, and machine learning methods usually have poor portability and are not suitable for massive data processing [9].

With the continuous development of deep learning algorithms [10,11], Convolutional Neural Networks (CNN) have performed outstandingly in fields such as image classification, semantic segmentation, and target detection [12,13,14]. Convolutional Neural Networks can adaptively learn shallow features such as image colour and texture, and also learn deep semantic features, and can handle image segmentation in complex scenes [9]. Long et al. proposed a Fully Convolutional Neural Network (FCN) based on CNN, in which FCN employs transposed convolution to replace the fully connected layer of CNN with an up-sampling layer, thus improving the segmentation efficiency, giving the output image the same dimensions as the input [15]. After that, most of the subsequent semantic segmentation methods were improved based on FCN [16], such as SegNet [17], RefineNet [18], U-Net [19], Deep Lab V3+ [20], etc. Among them, the U-Net model has been widely used in semantic segmentation tasks because of its smaller model, fewer parameters, and faster processing speed. Wei et al. [9] and Wei et al. [21] introduced Sentinel-1 data and the U-Net network model to extract rice planting areas in the Arkansas River Basin in the United States and Northeast China, respectively, and both achieved high extraction accuracy, proving the feasibility of spatiotemporal migration based on the U-Net model; Du et al. introduced the U-Net model to extract rice planting areas in Arkansas, and employed the random forest model extraction results as a control group for comparison, with the results showing that the U-Net achieved the optimal segmentation accuracy [22].

However, unlike traditional supervised classification algorithms, the training samples of the semantic segmentation model consist of a large number of annotated image sets, which require all pixels in the image to be annotated to build the samples required for semantic segmentation model training [23,24,25]. Currently, most research only constructs training samples through visual interpretation, which is time-consuming and laborious, lacks timeliness, and limits the application of semantic segmentation models in large-scale crop mapping [26,27,28]. Addressing the issue of scarce training samples, Lee first proposed the pseudo-label technique in 2013 [29]. This is an innovative semi-supervised learning method aimed at optimising and enhancing the performance of supervised classification models using unlabelled sample data. In his research, Lee directly used the predictions of the task model as pseudo-labels and input them into the next round of training. He pointed out that this operation is equivalent to adding an entropy regularisation term to the loss function, which can force the overlap of predictions from different classes to be lower. In recent years, some scholars have begun to use pseudo-labels in large-scale crop extraction tasks and have achieved good performance. For example, Zhu et al. combined phenology methods with deep learning models, utilised phenological feature indicators for preliminary rice extraction, constructed training samples for LSTM models, and completed rice mapping [30]. Wei et al. successfully extracted the spatial distribution information of rice in Jiangsu Province based on a pseudo-label established by K-means and random forest algorithms, providing a new option for the application of semantic segmentation models in areas with a shortage of training samples [31].

Thus, in order to fully explore the solution to the shortage of training samples for semantic segmentation models and efficiently establish the high-quality training labels required for semantic segmentation models, this study employed a combination of machine learning and deep learning to achieve precise extraction of winter wheat in Henan Province. Firstly, WCI phenology indicators were established in typical areas, their segmentation thresholds were extracted using the Otsu algorithm, and preliminary extraction of winter wheat was completed. Secondly, points were randomly selected after improvement of this basis, and random sample points were used to train the random forest model, with random forest model extraction results being continuously optimized and improved through the iterative “random point selection-model training” process, selecting two overlapping extraction results with a rate of 99% to be used as training samples (i.e., pseudo labels) for the U-Net model. Lastly, based on the U-Net model, the fine spatial distribution of winter wheat in Henan Province in 2022 was obtained. To verify the scientific validity of the experimental method, this study selected the U-Net++ model to extract the spatial distribution of winter wheat in Henan Province through the same steps. Additionally, this study used the U-Net model to predict the spatial distribution information of winter wheat in Henan Province in 2019, and used a random forest model for direct extraction of winter wheat in Henan Province to reveal the limitations of the random forest algorithm in feature extraction representation and model transfer, as well as the feature extraction ability and advantages of deep learning models in agricultural crop type mapping applications.

2. Materials and Methods

2.1. Study Area

Henan Province (31°23′–36°22′N, 110°21′–116°39′E) is located in the hinterland of the Central Plains of China, which is one of the main winter wheat producing areas in China. The terrain of Henan Province is higher in the west and lower in the east. The central and eastern parts are mostly plains, and the western part is mostly mountainous and hilly. The climate is subtropical and temperate monsoon, with hot and rainy summers and cold and dry winters. The annual frost-free period in Henan Province is 201–285 days, and the average temperature in the province is about 15.8 °C. Rainfall is mainly concentrated from June to August, and the average precipitation in the province is about 594.3 mm. The annual average sunshine hours are about 2300–2600 h, which is suitable for planting many crops. For the establishment of the U-Net model and the preparation of pseudo-label data sets, a Study Area A of 1.5° × 1.5° in central Henan Province was selected. This study area had a moderate distance from all other parts of Henan Province, fewer differences in phenological characteristics, and a high proportion of winter wheat planting areas, exhibiting significant typicality. Details are shown in Figure 1.

2.2. Data Source

Sentinel-2 is a Sentinel series satellite developed by ESA, and is a high-resolution multi-spectral imaging satellite consisting of two satellites, 2A and 2B. Sentinel-2 carries a multi-spectral imager (MSI) comprising 13 spectral bands, and the return period is 5 days with spatial resolutions of 10 m, 20 m, and 60 m [32]. In this study, a Level-2 surface reflection product (MSI) of Sentinel-2 data was introduced with a spatial resolution of 10 m, provided by Google Earth Engine [33]. Published studies have shown that visible light and short-wave infrared bands play an important role in vegetation classification; therefore, the red (B4), green (B3), blue (B2), and near-infrared (B8) bands were selected in the image as research data in this study [34,35]. Because the product had undergone pre-processing such as radiometric calibration, atmospheric correction, terrain correction, and geometric correction, it belongs to the category of surface reflectance products. Therefore, this study only needed to perform cloud removal, stitching, band merging, and other operations on the image. Sentinel-2 images with cloud cover below 20% were screened in the study area and cloud pixels were removed using the QA60 band. This cloud removal method performs bitwise operations directly on the QA60 quality band to filter pixel values. At the same time, it masks pixels such as clouds, cirrus clouds, rain, and snow to achieve cloud removal effect [36,37].

The phenological period of winter wheat in Study Area A is similar to that of the entire Henan Province [38]. It is generally sown in later autumn and early winter and harvested the following summer. Different ground object sample points in Study Area A were selected to analyse the time series change characteristics of the Normalised Difference Vegetation Index (NDVI), as shown in Figure 2. The analysis showed that from the sowing period (October) to the heading period (April), the chlorophyll content of the winter wheat vegetation canopy increased and the vegetation index increased as well. However, after the maturity period, the vegetation index decreased rapidly, which was significantly different from the spectral characteristics of other land types. According to the pattern of the winter wheat canopy spectral changes, this study selected Sentinel-2A images from October 2022, April 2023, and May–June 2023 within Study Area A and Henan Province to construct pseudo-labels and conduct winter wheat mapping. Meanwhile, this study also selected Sentinel-2A images from October 2018, April 2019, and May–June 2019 in Henan Province to form a multi-temporal image dataset, which was used to validate the temporal suitability of the U-Net model. The specific image data information is shown in

Table 1. Sentinel-2 image data acquisition information within Study Area A and Henan Province.

Time Series	Image Information	Study Area A (2022)	Henan Province (2022)	Study Area A (2019)	Henan Province (2019)
1	Date	11 October 2022–21 October 2022	11 October 2022–21 October 2022	1 October 2018–21 October 2018	1 October 2018–21 October 2018
	Number	15	81	22	134
2	Date	26 April 2022–16 May 2022	26 April 2022–16 May 2022	16 April 2019–16 May 2019	16 April 2019–16 May 2019
	Number	13	86	17	86
3	Date	1 June 2023–21 June 2023	1 June 2023–21 June 2023	1 June 2019–21 June 2019	1 June 2019–21 June 2019
	Number	24	105	17	91

. On the other hand, optical images would be affected by could cover and other factors, making it difficult to obtain images in the same time interval over a wide range. However, the differences in winter wheat plant conditions during the same period in adjacent years were small, so images at the same time period in adjacent years were selected as data supplements. The Sentinel-2A data in the GEE platform had completed pre-processing such as radiometric calibration, terrain correction, and atmospheric correction; therefore, operations such as cloud removal and fusion on the images were performed in this study. At the same time, the Winter Wheat Canopy Index (WCI) index was established using image data in Study Area A for preliminary extraction of winter wheat. The specific formula is shown as below [39], in which

$$NDVI=\frac{B8-B4}{B8+B4}$$

(1)

$$WCI=\left({NDVI}_2-{NDVI}_1\right)\times ({NDVI}_2-{NDVI}_3)$$

(2)

where, NDVI₁, NDVI₂, and NDVI₃, respectively, represent the NDVI at the first, second, and third time series.

Besides, based on the 2022 Google Earth high-resolution images and Sentinel-2 multi-temporal images, this study employed visual interpretation methods to select 1500 and 1300 sample points, respectively, within Study Area A and Henan Province for accuracy validation of pseudo-label data and model extraction results. Simultaneously, using high-resolution images from 2019, the sample set for Henan Province in 2022 was refined and supplemented, resulting in a sample dataset for Henan Province in 2019 containing 1200 sample points, as detailed in

Table 2. Sample collection table.

Type of Sample Points	Study Area A (2022)	Henan Province (2022)	Henan Province (2019)
Winter Wheat	600	443	400
Non-Winter Wheat	900	857	800
Total number	1500	1300	1200

.

2.3. Dataset Production

In order to enable the U-Net model to fully obtain the correlation between each piece of time series image data during the training process, the bands of multi-time series images in chronological order were merged in this study to form multi-channel input. However, due to the large size of remote sensing image data and the limited computing capacity of calculators, it was difficult for the U-Net model to directly perform pixel-level classification of the entire image; a 512 × 512 sliding window was introduced in this study to crop the image, and the sliding step was set to one-third of the window size. Finally, 4479 images and labels with a size of 512 × 512 × c were obtained, where c represents the number of channels in multi temporal images, as shown in Figure 3. After that, the produced sample set was divided into a training set, verification set, and test set according to the ratios of 70%, 20%, and 10%. The training set was used to train the U-Net model, the verification set was used to fine-tune the parameters in the model, and the test set was used to evaluate the accuracy of the extraction results of the model.

3. Method

3.1. Technical Research Path

This study was performed based on the GEE platform Sentinel-2A image data, and machine learning and deep learning algorithms were introduced to extract the spatial distribution of winter wheat in Henan Province in 2022. The technical path is illustrated in Figure 4, consisting of two main components:

(1) Construction of pseudo-label data based on WCI and random forest models. Firstly, using the WCI index, threshold segmentation was conducted within the Study Area A via the GEE platform to obtain preliminary winter wheat extraction results. Subsequently, the segmentation results were optimised and improved, random samples were generated, and with the integration of multi-temporal NDVI dataset, the random forest model was trained. Utilising the trained model, winter wheat extraction was performed within Study Area A. This process of “generating training samples—training random forest model—winter wheat extraction” was iteratively executed to continuously optimise the extraction results of the random forest model, thereby obtaining pseudo-label data.

(2) Winter wheat mapping based on the U-Net model. Utilising the 2022 multi-temporal image dataset and pseudo-label data, the U-Net model was constructed, and spatial distribution information of winter wheat in Henan Province for both 2022 and 2019 was extracted and evaluated for accuracy.

3.2. Winter Wheat Pseudo-Label Production Based on Random Forest

3.2.1. Dataset Establishment

Based on the GEE platform, Sentinel-2A images with cloud cover of less than 20% in Study Area A in October 2022, March 2023, and April 2023 were obtained, and the NDVI images in four time periods were calculated and generated, while the multi-time series NDVI images were obtained by merging the bands in chronological order, including four bands representing four series.

3.2.2. Initial Winter Wheat Extraction Based on WCI

In this study, the WCI index was employed for threshold segmentation to obtain the preliminary extraction results of winter wheat in the Study Area A. The threshold size could directly determine the accuracy of the segmentation results, and the manual setting method would be easily affected by subjective factors and were prone to errors: therefore, the Otsu algorithm was used to determine the threshold of each segmentation index. The Otsu algorithm [40] has been recognised as one of the best algorithms for threshold selection in the field of image segmentation. It has been widely used in the field of digital image processing due to its advantages of simple calculation and being independent of changes in image brightness and contrast. The WCI segmentation threshold in Study Area A was extracted using the Otsu algorithm, and was 0.96. Since land types with a WCI value around the segmentation threshold might be more likely to be misclassified, the WCI segmentation threshold should be adjusted to eliminate potential misclassified land types. Through experimental comparative analysis, pixels with WCI greater than 1.06 were regarded as winter wheat, while pixels with WCI less than 0.89 were regarded as non-winter wheat. Finally, the respective results of winter wheat land type and non-winter wheat land type were obtained.

3.2.3. Generating Random Samples

Crop planting boundary pixels are prone to misclassification, and the threshold method extraction results are usually interfered with by pixel outliers and image noise. Therefore, in order to eliminate the negative effects listed above as much as possible and improve the representativeness of random sample points, it was necessary to screen the winter wheat patches and non-winter wheat patches in the preliminary extraction results, and then randomly select points. Based on the vector boundary of each patch, an inward 10 m buffer was generated and excluded from the patch results, followed by calculating the area of each patch, eliminating patches with an area less than 5000 m², and establishing random sample points in the remaining retained patches. The number of sample points generated in each patch was set to “patch area/5000”, and finally 800 winter wheat sample points and 1600 non-winter wheat sample points were randomly selected from them for subsequent random forest model training, in which 70% were chosen for training and 30% were chosen for verification [21].

3.2.4. Pseudo-Label Production

The multi-time series feature data set produced in Section 3.2.1 and the random sample points generated in Section 3.2.3 were introduced as model input. The random forest model was established and trained through the built-in function “ee.Classifier.smileRandomForest” of GEE. Then, the winter wheat planting area in Study Area A was extracted using the random forest model as a new current sample, and the overlap rate between the current and the previous sample (the ratio of the number of pixels with the same classification result to the total number of pixels) was calculated. If the overlap rate was lower than the set threshold, points would be randomly selected again based on the current sample, and the steps of random forest model training and extraction would be repeated until the overlap rate could exceed the set threshold. After the experimental comparison, the overlap rate threshold was set to 99%. Finally, the last-time extraction results were employed as training samples for the deep learning model, which would be pseudo-labels.

3.3. Establishment and Training of the U-Net Model

3.3.1. Establishment of the U-Net Model

The network structure of this study was adapted from the classic U-Net network model. It is a U-shaped symmetric network without a fully connected layer, and the input and output are both images, as shown in Figure 5. The left-hand side of the network is the encoder. As a typical Convolutional Neural Network structure, it consists of four sets of identical coding blocks, with each set of coding blocks containing two 3 × 3 convolution kernel layers and a maximum pooling layer. The convolution kernel layer can extract image features, and the maximum pooling layer can filter out unimportant high-frequency information, reduce feature dimensions, and increase the receptive field. Repeated convolution and pooling operations can fully extract the deep features of the image. Each group of coding blocks doubles the feature map channels and reduces the scale by half. The right-hand side of the network is the decoder, which consists of four groups of identical decoding blocks, with each group of decoding blocks containing two 3 × 3 convolution kernel layers and a deconvolution layer. Each group of decoding blocks can reduce the feature map channels by half and double the scale. In the meantime, there is a skip connection between the coding block and the decoding block, and the feature map of each group of coding blocks will be saved and fused with the output of the corresponding scale decoding block as the input of the next deconvolution layer. Skip connection can reduce the information loss caused by pooling operations in the encoding block, provide feature maps of different resolutions for the decoding block, and help the decoder better restore target details and image accuracy. In the final output layer, two convolutional layers will be used to map the 64-dimensional feature map into a 2-dimensional output map. The Softmax classifier is used to calculate the probability that each pixel in the feature map belongs to each targe land type, and the probability is mapped to obtain the ground object classification map [19,41].

3.3.2. Data Pre-Processing

The pixel value of remote sensing image data is usually between 0–4095. When the data distribution range is large, the training process of the semantic segmentation model to find the optimal solution will be very slow, and the result will not even converge. Therefore, the data should be normalised to control the data distribution within a certain range. Data normalisation can convert data of different orders of magnitude into the same order of magnitude, helping to improve the model’s convergence speed and classification accuracy. The min–max normalisation method was employed in this study to normalise the four research bands of Sentinel-2A remote sensing images. The calculation formula is as follows:

$$x^{\prime }=\frac{x-x_{min}}{x_{min}-x_{max}}$$

(3)

where x is the value of a certain pixel; x_min is the minimum value of sample data; and x_max is the maximum value of the sample data.

3.3.3. Model Training

The process of training model is as shown in Algorithm 1. Multi-temporal images and pseudo-label are used as model inputs. And this study sets the learning rate to 0.0001 and the training batch to 50 rounds. During the training process, the performance of the model is evaluated by calculating the loss function, and the model parameters are updated and optimized through forward propagation. Finally, output the trained model.

Algorithm 1. Winter Wheat Extraction Model Implementation Steps

1 Input: Image data/Pseudo-label data; 2 Initialisation: Random initialisation of training parameters, parameter settings, learning rate 0.0001, training rounds 50 epochs; 3 Processing: 4 for j = 0 to N do (N is the number of iteration calculations) 5   calculate the loss L_loss 6  if (L_loss ≤ threshold) then 7    outputs 8   else extract features again based on the parameters updated by forward propagation to calculate L_loss; 9 Output: Trained model

3.3.4. Model Prediction

The model prediction is that after the model training is completed, the final model will be used to calculate the probability that each pixel in the image to be classified belongs to each category, and then the “argmax” function will be used to find the category label to which the maximum probability belongs. During the model prediction process, in order to avoid memory overflow, the image to be classified needs to be cropped into image blocks with the same size as the training image (512 × 512 × 12) for prediction one by one and then spliced into the entire image. In addition, a sliding window method was used in this study to obtain image blocks with certain overlapping areas, and the classification results of the middle area of each image block were retained, the results with inaccurate edges were discarded, followed by splicing them. Such an action could avoid obvious splicing traces and improve the performance of image prediction.

3.4. Accuracy Evaluation Indicators

In this study, two types of accuracy evaluation were applied to evaluate the extraction results of winter wheat. The first method was a remote sensing classification and recognition result verification method based on the confusion matrix, in which manually collected sample point data were compared with the classification results to calculate the classification accuracy. The main evaluation indicators were Overall Accuracy (OA), User’s Accuracy (UA), Producer’s Accuracy (PA), F1-score, and the Kappa Coefficient. Since this study regarded a two-classification problem, the confusion matrix can be expressed in

Table 3. Binary classification confusion matrix.

	Estimate Value = 1	Estimate Value = 0
True value = 1	TP	FN
True value = 0	FP	TN

.

In Table 3, TP represents true positives, which is the number of pixels with both the true value and the predicted value being 1; FP represents the false positive, which is the number of pixels with the true value being 0 but the predicted value being 1; FN represents false negatives, which is the number of pixels with a true value of 1 and a predicted value of 0; TN represents true negatives, which is the number of pixels with both a true value and a predicted value of 0. The formulas of various accuracy indicators are shown as follows:

Overall Accuracy is a commonly used accuracy evaluation index that represents the probability of correct classification of each pixel. The calculation formula is shown as follows:

$$OA=\frac{TP+TN}{TP+FN+FP+TN}$$

(4)

User’s Accuracy represents the proportion of the number of pixels that was correctly classified to the total number of pixels, which was classified as target land types. The calculation formula is shown as follows:

$$UA=\frac{TP}{TP+FP}$$

(5)

Producer’s Accuracy represents the proportion of the number of correctly classified pixels to the total number of pixels that are actually the target land types. The calculation formula is shown as follows:

$$PA=\frac{TP}{TP+FN}$$

(6)

The F1-score is a comprehensive evaluation indicator of the User’s and Producer’s Accuracy. The calculation formula is shown as follows:

$$F1=2\times \frac{UA\times PA}{UA+PA}$$

(7)

The Kappa Coefficient represents the ratio of error reduction between classification and completely random classification. The calculation formula is shown as follows:

$$\begin{gathered} po=OA=\frac{TP+TN}{TP+FN+FP+TN} \\ pe=\frac{\left(TP+FP\right)\left(TP+FN\right)+\left(FN+TN\right)\left(FP+TN\right)}{{\left(TP+FP+TN+FN\right)}^2} \\ Kappa=\frac{po-pe}{1-pe} \end{gathered}$$

(8)

The second accuracy evaluation method was based on the statistical data of the Henan Province Statistical Yearbook (http://www.henan.gov.cn/zwgk/zfxxgk/fdzdgknr/tjxx/tjnj/, accessed on 1 January 2023), using Absolute Errors (AE), Absolute Percentage Errors (APE), Correlation Coefficient (R²), and other indicators for accuracy verification. The calculation formulas of each indicator are shown as follows:

$$\begin{gathered} AE=|AA-PA| \\ APE=\left(\right|AA-PA\left|\right)/AA\times 100\mathrm{\%} \\ {R}^2=\frac{{\sum }_{i=1}^n{(x_i-\overline{x})}^2{(y_i-\overline{y})}^2}{{\sum }_{i=1}^n{(x_i-\overline{x})}^2{\sum }_{i=1}^n{(y_i-\overline{y})}^2} \end{gathered}$$

(9)

in which n represents the number of administrative units, AA represents the statistical value, and PA represents the predicted value.

4. Results and Discussion

4.1. Pseudo-Label Production and Accuracy Verification Based on the Random Forest Model

The experimental results showed that the process of “generating random sample points—model training—winter wheat extraction” cycled a total of four times, and the corresponding overlap rate for each time is shown in

Table 4. Random forest model extraction result overlap rate.

Training Times	First	Second	Third	Fourth
Overlap Rate	96.10%	98.53%	98.86%	99.25%

. As the number of training times of the random forest model increased, the overlap rate between the new round of extraction results and the previous round of extraction results increased steadily, eventually reaching 99.25%. Among them, the overlap rate between the first extraction result and the WCI segmentation result reached 96.1%, which reflected that the WCI index could effectively capture the spectral variation characteristics of winter wheat and is highly sensitive to winter wheat.

In order to further verify the accuracy of pseudo-labels, manually-selected sample points were employed in this study, as shown in

Table 5. Evaluation of pseudo-label accuracy based on artificial samples.

Indicators	OA	Kappa Coefficient	F1-Score		UA		PA
			Winter Wheat	Non-Winter Wheat	Winter Wheat	Non-Winter Wheat	Winter Wheat	Non-Winter Wheat
Value	97.53%	0.95	0.97	0.98	95.77%	98.76%	98.17%	97.11%

. As illustrated by the results, the overall accuracy of the pseudo-label reached 97.53%, the Kappa Coefficient was 0.95, and the F1-score of winter wheat and non-winter wheat land types reached 0.97 and 0.98, respectively. From the perspective of manual-selected sample point verification accuracy, the pseudo-label held higher accuracy, which provided more reliable sample data for subsequent training of deep learning models.

On the other hand, the high-resolution satellite remote sensing image in May 2022 was introduced as a reference, which aimed to verify the scientific nature of the pseudo-label production method and whether the pseudo-label would have sufficient accuracy by comparing it with the WCI segmentation results. The details are shown in Figure 6. Although the WCI could effectively obtain winter wheat planting areas, there were still some misclassifications shown in Figure 6a–c. However, such misclassifications were successfully eliminated in the pseudo-label data, which benefitted from the “sample point establishment—model training” process. Some potential errors were effectively corrected by the classification ability of the random forest algorithm. Besides, the field ridges in the pseudo-labels were also further differentiated.

4.2. Extraction Result Analysis

To fully verify whether the pseudo-label data constructed based on the above methods can meet the requirements of training deep learning models and the scientificity of conducting winter wheat mapping based on pseudo-label data, this study employed the U-Net model, U-Net++ model, and random forest model to extract winter wheat in Henan Province in 2022, and the extraction results are shown in Figure 7. Simultaneously, to verify whether the deep learning models constructed based on pseudo-labels have transferability, we input the multi-temporal image dataset of Henan Province in 2019 into the U-Net model of 2022 for prediction, and the extraction results are shown in Figure 7d. From Figure 7a–c, it can be observed that the spatial distribution characteristics of winter wheat in Nanyang City and the southern part of Luoyang City in the extraction results of the random forest model are significantly different from the other two deep learning models. By comparing the original images, we found that the western part of Nanyang City and the southern part of Luoyang City are mainly mountainous terrain, covered by extensive forest vegetation, but the random forest model misclassifies large areas of vegetation as winter wheat during the transfer prediction process.

To further compare the extraction results of winter wheat in Henan Province in 2022 among the three models, this study selected the original images from April to May 2022 in Henan Province as reference data. In areas where there were significant differences between the extraction results of the U-Net model and the random forest model, six rectangular regions with a spatial distribution size of 0.1° × 0.1° (a–f) were selected for enlarged comparison of the extraction results of the three models. The approximate locations of the rectangular regions are shown in Figure 8, and the comparison details are shown in Figure 9.

As can be seen from Figure 9a–c, the U-Net model demonstrates the highest extraction accuracy, followed by the U-Net++ model, while the random forest model has the lowest accuracy. Figure 9a–c represent relatively dense winter wheat planting areas, where the U-Net and U-Net++ models extract more complete and contiguous winter wheat plots. In contrast, the random forest model’s extraction results show significant omission of non-winter wheat areas. In regions where winter wheat planting areas are disturbed by bare land, evergreen vegetation, and other crop types Figure 9e,f, the U-Net and U-Net++ models still maintain high classification accuracy. However, compared to identifying large contiguous fields, these models are more prone to misclassification in scattered plots and boundary areas between different land use types. This is because the U-Net and U-Net++ models would learn the outline features and boundary relationships of the target features during the training process. If the target feature (winter wheat) was scattered and planted, the boundary outline features would become insignificant and reflected in the image, while spectral features would be more complex, making the model more prone to errors in the classification and recognition process. Although the random forest model could show high accuracy in the winter wheat extraction task in a small area, due to the poor robustness and transferability of the machine learning model, it was easy for “misclassification” and “missing classification” to occur if the model was transferred to a non-model training area for prediction, making it difficult to apply in large areas.

4.3. Accuracy Verification of Extraction Results

The 2022 extraction results of the U-Net model, U-Net++ model, and random forest model were validated for accuracy using statistical data and manual sample data from Henan Province in 2022. The 2019 prediction results of the U-Net model were validated for accuracy using statistical data and manual sample data from Henan Province in 2019.

4.3.1. Accuracy Evaluation of Extraction Results Based on Statistical Data

Experimental data indicated that the winter wheat planting areas in Henan Province in 2022, based on the U-Net model, U-Net++ model, and random forest model, were 5915.03 K hectares, 5275.55 K hectares, and 4852.01 K hectares, respectively. Using the statistical data (5682.46 K hectares) as a reference, the Absolute Errors (AE) were 232.57 K hectares, 406.91 K hectares, and 830.45 K hectares, respectively, with Absolute Percentage Errors (APE) of 4.09%, 7.16%, and 14.61%. The winter wheat planting area in Henan Province in 2019, predicted using the U-Net model, was 5348.16 K hectares, with an absolute error of 365.44 K hectares compared to the statistical data (5713.60 K hectares), resulting in an absolute error percentage of 6.40%.

To further evaluate the 2022 extraction results of the three models and the 2019 prediction results based on the U-Net model, Absolute Error (AE), Absolute Percentage Error, and Correlation Coefficient (R²) were employed to carry out accuracy evaluation at the municipal scale. The AE and APE of 18 cities in Henan Province were divided into gradients, and the number of cities in each interval was counted as shown in Figure 10.

From the 2022 accuracy evaluation results (Figure 10a–e), it can be observed that the extraction accuracy of the three models is in the order of U-Net model > U-Net++ model > RF. In terms of the AE indicator, most cities in the U-Net model extraction results have an AE between 0 and 50 K hectares, with the highest number of cities in the 0–10 K hectares range. In contrast, in the U-Net++ model and random forest model extraction results, the highest number of cities have an AE between 20 and 50 K hectares, followed by 0–10 K hectare and 50–100 K hectare ranges. For the APE indicator, most cities in the U-Net model extraction results have an APE below 10%, but two cities have an APE exceeding 50%. In the U-Net++ model and random forest model extraction results, most cities have an APE below 30%, but three cities have an APE exceeding 50%. An analysis of cities with large extraction errors in the U-Net and U-Net++ models shows that the cities with an AE exceeding 100 K hectares in the U-Net model are Zhoukou City (statistical data: 734.84 K hectares) and Luoyang City (statistical data: 231.46 K hectares). The cities with an APE exceeding 50% are Luoyang City and Sanmenxia City (statistical data: 75.11 K hectares). Similarly, in the U-Net++ model extraction results, Luoyang City has an AE exceeding 100 K hectares, and the cities with an APE exceeding 50% are Luoyang City and Sanmenxia City. These cities also have significant errors in the random forest model extraction results. By analysing the experimental data, this study believes that the higher errors in these three cities’ deep learning model extraction results are primarily due to their locations in the western and eastern parts of Henan Province, while the pseudo-label construction area is in the central part of the province. Therefore, the phenological cycles or spectral characteristics of winter wheat in these three cities differ from those in the model construction area. Additionally, the pseudo-label data, constructed based on the random forest model, are not entirely accurate. There are inherent errors due to factors like spectral differences that the random forest model cannot overcome. Although Lee pointed out that pseudo-labels with incorrect labels can be treated as an entropy regularisation term in the loss function, allowing deep learning models to maintain normal training, the models cannot entirely avoid being affected by these errors, resulting in some misclassification and ultimately failing to correctly identify all winter wheat land types. This also further explains why the cities with the largest extraction errors in both the deep learning models and the random forest model are the same. Therefore, in future research, further analysis on how to improve the quality of pseudo-labels is needed.

Moreover, correlation analysis with statistical data shows that the U-Net model has the highest Correlation Coefficient (R²) at 0.9741, followed by the U-Net++ model at 0.9689, and the random forest model at only 0.8835, indicating that the U-Net model achieved the highest classification accuracy.

Using the U-Net model, which has higher extraction accuracy, the spatial distribution of winter wheat in Henan Province in 2019 was predicted. The accuracy evaluation results are shown in Figure 10a,e,f. Due to factors such as differences in the phenological period of winter wheat between different years and the varying times of image acquisition, the accuracy of the 2019 winter wheat mapping results has decreased. However, in terms of the AE and APE indicators, the AE and APE for most cities are still below 50 K hectares and 30%, respectively. Additionally, the Correlation Coefficient (R²) between the model’s prediction results and the statistical data is 0.9313, indicating that the U-Net model constructed based on pseudo-labels can still maintain high extraction accuracy in other years.

4.3.2. Accuracy Verification Based on Manually-Selected Sample Points

Manual sample data were used to validate the extraction results of the three models for 2022 and the prediction results of the U-Net model for 2019. The OA, F1-score, Kappa coefficient, UA, and PA were calculated, as shown in

Table 6. Accuracy evaluation results based on confusion matrix. U-Net 2019 refers to the accuracy evaluation of the 2019 extraction results based on the U-Net model.

Model	OA	Kappa Coefficient	F1-Score		UA		PA
			Winter Wheat	Non-Winter Wheat	Winter Wheat	Non-Winter Wheat	Winter Wheat	Non-Winter Wheat
U-Net	93.08%	0.85	0.91	0.95	84.41%	98.73%	97.74%	90.67%
U-Net++	92.64%	0.84	0.89	0.94	87.23%	95.67%	91.85%	93.05%
RF	86.08%	0.70	0.81	0.89	76.95%	91.52%	84.42%	86.93%
U-Net 2019	89.15%	0.77	0.85	0.91	81.76%	93.76%	89.08%	89.19%

. The 2022 accuracy evaluation results indicate that the extraction accuracy of the three models is in the order of U-Net > U-Net++ > RF. In the U-Net++ model’s extraction results, the UA for winter wheat and the PA for non-winter wheat are higher than those in the U-Net model, suggesting that the U-Net++ model made more errors in classifying winter wheat areas as non-winter wheat areas. This result corresponds with the winter wheat planting area extracted by the two models, where the U-Net model overestimates and the U-Net++ model underestimates the winter wheat planting area. In contrast, the random forest model has the lowest extraction accuracy. In the 2019 accuracy evaluation results, the overall accuracy of the U-Net model reached 89.15%, indicating that the U-Net model maintains higher extraction accuracy than the random forest model in other years as well.

The accuracy evaluation results above indicate: (1) Deep learning models constructed based on pseudo-labels exhibit higher classification accuracy in winter wheat extraction tasks, whereas the random forest model is more susceptible to factors such as phenological crop differences and spectral differences when transferring to non-training areas for prediction, leading to a higher likelihood of misclassification. Compared to traditional machine learning algorithms like random forest, deep learning models demonstrate higher spatiotemporal adaptability and robustness. Under the premise of consistent image acquisition dates, they can achieve winter wheat extraction in different years, indirectly confirming that the pseudo-label data generated in this study meet the training accuracy requirements of deep learning models. (2) The U-Net++ model did not achieve higher extraction accuracy. The extraction accuracy of deep learning models is influenced by various factors such as model structure, image quality, sample accuracy, and number of training iterations. This study replaced manually interpreted samples with pseudo-label samples for model construction, which inherently introduces a certain level of error. Thus, sample accuracy becomes the predominant factor affecting model extraction performance. Despite introducing more complex skip connections on top of the U-Net model, the U-Net++ model cannot demonstrate higher extraction performance due to the limitation of pseudo-label accuracy. (3) The pseudo-label construction method for winter wheat proposed in this study is scientifically feasible. The pseudo-labels meet the accuracy requirements for training deep learning models and are expected to provide a feasible technical pathway for intelligent extraction of winter wheat spatial distribution information in the future.

4.4. Limitation & Expectation

In this study, a method to establish pseudo-labels was proposed, aiming to solve the problem of lacking training samples for deep learning models during the mapping task of large-area crops. On this basis, the establishment and spatial distribution information extraction of winter wheat with the U-Net model was accomplished, achieving well-performed results. However, some limitations existed during the research, which should be overcome in the future.

Firstly, as for the initial extraction method of winter wheat, only the WCI indicators were employed. Although such indicators could effectively represent changes in spectral features of the winter wheat canopy, there were still some unavoidable errors due to the differences in growing seasons, disturbance of clouds, and anomalies of pixel values; secondly, the pseudo-label data were obtained from the random forest model, in which there were potential errors that could not be overcome during the extraction of the model; these errors were further passed to deep learning models and were enhanced. As a result, this point was testified by cities with the most significant errors in both extraction results of the random forest model and the U-Net model being the same.

In short, other supplemental data and indicators should be employed in subsequent research to optimise the extraction results of winter wheat; in the meantime, pixels with higher confidence levels should also be selected for the establishment of pseudo-labels in order to eliminate significant parts, improving the accuracy of pseudo-labels.

5. Conclusions

In this study, Sentinel-2A multi-time series image data were employed with the help of the GEE cloud platform; WCI phenological indicators and the Otsu algorithm were used to complete the preliminary extraction of winter wheat; random sample points were produced; the random forest algorithm and random sample points were used to establish pseudo-label data for U-Net model training in typical areas; and finally, the winter wheat planting areas in Henan Province in 2022 were extracted based on the trained U-Net model. The extraction results were compared with statistical yearbooks and manually-selected sample points to verify the feasibility and reliability of the research method framework in this study. The main conclusions are as follows: (1) Pseudo-label data were established through the iterative process of “random sample point generation—random forest model training—winter wheat extraction”. As the number of iterations increased, the overlap rate of the random forest model extraction results increased steadily until 99.25%; (2) The winter wheat in Henan Province in 2022 was extracted using the U-Net model, U-Net++ model, and random forest model constructed based on pseudo-label data for 2022. Additionally, the U-Net model was used to predict the winter wheat planting areas in Henan Province in 2019. The accuracy evaluation results indicate that the extraction accuracy of the three models in 2022 is in the order of U-Net model > U-Net++ model > random forest model, and although the extraction accuracy of the U-Net model for 2019 prediction results has decreased, it remains higher than that of the random forest model. Moreover, due to the limitation of pseudo-label accuracy, the U-Net++ model did not achieve higher extraction accuracy; (3) Experimental results demonstrate that deep learning models constructed based on pseudo-labels exhibit higher extraction accuracy. Compared to traditional machine learning models like random forest, they have higher spatiotemporal adaptability and robustness, further validating the scientific and practical feasibility of pseudo-labels and their generation strategies. This approach is expected to provide a feasible technical pathway for intelligent extraction of winter wheat spatial distribution information in the future.