Early-Season Crop Mapping by PRISMA Images Using Machine/Deep Learning Approaches: Italy and Iran Test Cases

Abstract

Despite its high importance for crop yield prediction and monitoring, early-season crop mapping is severely hampered by the absence of timely ground truth. To cope with this issue, this study aims at evaluating the capability of PRISMA hyperspectral satellite images compared with Sentinel-2 multispectral imagery to produce early- and in-season crop maps using consolidated machine and deep learning algorithms. Results show that the accuracy of crop type classification using Sentinel-2 images is meaningfully poor compared with PRISMA (14% in overall accuracy (OA)). The 1D-CNN algorithm, with 89%, 91%, and 92% OA for winter, summer, and perennial cultivations, respectively, shows for the PRISMA images the highest accuracy in the in-season crop mapping and the fastest algorithm that achieves acceptable accuracy (OA 80%) for the winter, summer, and perennial cultivations early-season mapping using PRISMA images. Moreover, the 1D-CNN algorithm shows a limited reduction (6%) in performance, appearing to be the best algorithm for crop mapping within operational use in cross-farm applications. Machine/deep learning classification algorithms applied on the test fields cross-scene demonstrate that PRISMA hyperspectral time series images can provide good results for early- and in-season crop mapping.

1. Introduction

Early-season crop mapping is valuable for estimating the area under cultivation and for yield prediction [1], which is vital information for marketing and decision making involved in food security. Moreover, accurate and detailed in terms of number of crop species, crop mapping is important for agricultural management, economic development planning, and agroecosystem conservation. Precise crop maps could be used for analyzing the implementation and effects of agri-environmental policies [2] and planning of optimal and sustainable agronomic management based on crop rotations [3]. National and international institutions require an early estimation of the planted area for food security issues [4]. Parallelly, there is increasing interest from crop insurance companies in precise early crop mapping and weather-based crop health indicators to be included in their area-yield crop insurance schemes, especially in the early season when the vegetation cover is low and fields do not show a dense homogeneous canopy.

Time series-based analysis of multispectral images, e.g., Landsat and Sentinel, are the most common data sources that are used for crop mapping [5,6,7]. The time series method is more accurate when data covering the whole growing season are used [8]. Interference from clouds and aerosols is the main drawback of optical remote sensing data, making time series analysis more challenging [9].

Concerning hyperspectral remote sensing, UAV hyperspectral cameras [10,11,12,13] and airborne hyperspectral data [14,15] have been extensively used for crop mapping. Spaceborne hyperspectral images have been used for crop mapping [16,17,18]. However, moving from proximal/airborne to spaceborne sensors, new challenging issues, related to low signal-to-noise ratio (SNR), spectral mixing issues related to the 30 m/pixel supported by the present missions, and atmosphere attenuations, must be considered. The two hyperspectral technology demonstration missions, i.e., the NASA Hyperion onboard the Earth Observing-1 (EO-1) spacecraft [19] and the Compact High Resolution Imaging Spectrometer (CHRIS) on ESA’s Proba-1 microsatellite [20] did not show suitable temporal resolution and did not receive much attention from geoscientists and agronomists for crop-type mapping. Conversely, significant improvements in this context can be obtained nowadays, following the operation of the Italian Space Agency (ASI) PRISMA (Hyperspectral Precursor of the Application Mission) [21] hyperspectral satellite mission since 2019, the German EnMAP (Environmental Mapping and Analysis Program) since 2022 [22], the JPL-NASA EMIT (Earth Surface Mineral Dust Source Investigation) [23] since 2022, and the German DLR Earth Sensing Imaging Spectrometer (DESIS) [24], which provide hyperspectral images to the community worldwide. Furthermore, potential added value would be explored by forthcoming hyperspectral missions such as the ESA CHIME [25], ASI-PRISMA Second Generation (https://space.leonardo.com/en/news-and-stories-detail/-/detail/prisma-accordo-progetto-seconda-generazione, accessed on 5 May 2024), and NASA SBG [26]. Despite improvements in the suitable swath width and revisit time, the ongoing hyperspectral satellites are still not fully suitable for early crop mapping and monitoring [27] unless applied in a tandem configuration on specific selected test sites. However, it is foreseen that the higher temporal resolution of the hyperspectral imagery will be able to provide, in the following years, denser time series suitable to the development of new high value products covering the crop growing season [28]. One of the main challenges posed by spaceborne hyperspectral imagery is the availability of relevant information in high-dimensional data containing highly correlated spectral information. As a branch of artificial intelligence, machine learning (ML) refers to algorithms that can explore and derive meaningful information from data and use them within a self-learning approach to construct algorithms/models for accurate classification or prediction. ML has gradually gained popularity due to its accuracy and reliability. ML algorithms, e.g., artificial neural networks (ANNs), support vector machines (SVMs), and random forest (RF) have been widely applied in the last decade in crop type identification [16,17,29,30,31,32].

Furthermore, the use of deep learning (DL) algorithms has allowed to explore several levels of distributed representations from the input training dataset. Among them, convolutional neural network (CNN) is the most common DL method applied to crop mapping [33]. This is because CNNs reduce the need for building handcrafted feature extractors, by demonstrating an exceptional ability to learn complex representations directly from image bands. Larger and more diverse training data lead to improved functionality of CNNs. A long running time, especially when working with big data, has been a challenging issue that is expected to be solved by improved hardware and software components of machine vision systems [34,35]. In comparison with CNNs, the architectures with more layers and parameters, like Visual Geometry Group (VGG-16), AlexNet, and Res-Net, need more training data to fit the model parameters [36].

Classical ML algorithms process each pixel vector based on spectral features without considering spatial contextual information [37]. Keeping the spatial information could help crop type mapping because neighboring pixels have higher probabilities of belonging to the same crop class [38]. Hence, 2D-CNN can calculate spatial features in both directions, but it does not have spectral features. 2D-CNN has been used for crop mapping [39] to identify the phenology in Sentinel-2 images [40] and leaf age from a single image [41]. 3D-CNN, even if at a higher computational cost, is able to exploit high-dimensional spectral features [42]. The main issue in 3D methods is the increased dimensionality of samples that may affect the classification accuracy and efficiency [43]. Spectral/spatial pixel-wise hyperspectral image classification can be achieved by integrating spatial features into spectral information. Therefore, different feature extraction methods have been used to overcome this issue [18,44,45,46].

Early-season crop mapping using supervised methods could be severely hampered by the absence of contemporary Earth observation (EO) data and ground truth acquisitions. Ideally, ground truth should be obtained from surveys that collect targeted first-hand information, but it is often impeded by the substantial cost of time and labor [47]. Most classifiers work well through training using similar ground truth data, while the spatial transferability of these methods is still a challenging issue. In addition, environmental conditions (rainfall, temperature, etc.) could change over the years, severely impacting crop growing stages. To overcome this limitation, several early-season mapping methodologies, e.g., sample migration [48] and temporal encoding [49], have been developed to effectively classify crop types, even in situations where there is limited ground truth data available for that specific growing season. These approaches leverage historical data to generate crop labels and prior information for early-season crop identification. Moreover, a transfer learning (TL) scheme based on DL approaches helps to gain crop mapping results in transfer sites.

This study presents the application of the 1D-CNN and 3D-CNN deep learning algorithms and an ensemble of learning techniques like RF, SVM, K-nearest neighbor (KNN), and multiclass naive bayes (MNB) algorithms to a first available time series of new hyperspectral imagery belonging to four farm test cases. At this scope, the PRISMA mission provides an ideal benchmark for demonstrating the advancement offered by hyperspectral time series for the application in the agriculture context for crop mapping. This brings us to clarify the objectives of our study: (1) to develop a cross-farm ML/DL training and accuracy assessment when there are no in situ data available for training and (2) to test fine early-season crop mapping capabilities of hyperspectral time series data in comparison with multispectral imagery with a fine temporal resolution. The novelty of this research lies in (1) the supply of contemporaneous ground truth prepared by analyzing the PRISMA and Sentinel-2 optical images for training and validation; (2) demonstrating by ML and DL algorithms the importance of the use of hyperspectral satellite time series data for the accurate early- and in-season crop mapping. Section 2 describes the study areas used for the field data collection and analysis, and the methodology for achieving this study’s objectives. Section 3 and Section 4 report and discuss the results.

2. Materials and Methods

2.1. Study Areas

Three agricultural test sites, Maccarese, Grosseto, and Jolanda di Savoia, in Italy (Figure 1a), and a site in Iran (Figure 1b) were selected as case studies for crop mapping.

The Jolanda di Savoia farm is in northeast Italy (latitude 44°52′59″N, longitude 11°58′48″E), with an altitude of about 1 m.a.s.l. and mean annual precipitation of 691 mm. This region has a warm and temperate climate with an annual mean temperature of 13.6 °C [50]. The soils mostly show clayey and silty textures. The area belongs to the Bonifiche Ferraresi S.p.A., the largest farm in Italy, encompassing about 3850 ha (Figure 1c). The sowing time of winter cereals varies between October and December, while for summer crops it varies between April and June

The Maccarese S.p.A. farm (latitude 41°52′18″N, longitude 12°14′05″E), with an altitude of 8 m.a.s.l. and a long-term average annual precipitation of 812.9 mm, is in central Italy near Rome (Figure 1d). The farming area has soil texture ranging from sandy to clay loam [50]. The area has a typical coastal Mediterranean climate (hot summer Mediterranean climate, Csa in Köppen classification), with an average minimum temperature in the winter of 5.0 °C and an average maximum temperature in summer of 27.4 °C. The rainiest seasons are autumn and winter, and the sowing time for winter cereals varies between October and December, while for summer crops it varies between April and June, according to farm management needs and precipitation patterns [50].

The Grosseto test area (Figure 1e) is in central Italy (latitude 42°50′19″N, longitude 11°02′18″E, altitude 10 m.a.s.l.) and has a long-term average annual precipitation of 655 mm. The Grosseto area has a Mediterranean climate with very mild wet winters and very hot dry summers.

The Mirza Koochak-Khan (MKK) farming and industrial lands (Figure 1f) are in the Khuzestan province in southwest Iran (30°55′15″N; 48°15′35″E) with a mean altitude of 5 m.a.s.l., and a long-term average annual precipitation of 266 mm. The MKK soil is mostly characterized by silty clay loam and clay loam textures.

2.2. Overview of the Implemented Crop Mapping Procedure

The block diagram in Figure 2 shows the flowchart designed to map crop types using PRISMA and Sentinel-2 images. Data gathering, including ground truth data collection and satellite image acquisition, is described in Section 2.3. The pre-processing and processing of data is described in Section 2.4.1 and Section 2.4.2, respectively. The classification algorithms and the TR/ACC (TRaining and ACCuracy assessment) procedure are described in Section 2.5 and Section 2.6.

2.3. Data Collection

2.3.1. Ground Reference Data

The ground truth data, including the type of crop grown in the farms, were provided directly by field campaigns, while the 2020 crop map of the Grosseto site was extracted from the last published research by Spiller et al. [16]. MKK site crop maps for 2021–2023 were provided by Mirza Koochak-Khan farming and industrial company. The common species of this site among the other sites were wheat and sunflower. Some field photos of field campaigns in the Maccarese and Jolanda di Savoia farms of wheat and maize at different phenological stages from leaf development to ripening are shown in Figure 3.

The crop calendar of the investigated crops in the three sites located in Italy (Maccarese, Jolanda, and Grosseto) and in the MKK site is shown in

Table 1. Crop calendar of different crops in Maccarese, Grosseto, Jolanda di Savoia, and MKK sites. The planting, growth, and harvesting stages are shown in blue, green, and red colors, respectively.

Site	Species	January	February	March	April	May	June	July	August	September	October	November	December
Maccarese, Jolanda and Grosseto	Wheat
	Herbage
	Barley
	Pea
	Triticale
	Fava bean
	Cardoon
	Maize
	Rice
	Tomato
	Soybean
	Sunflower
	Sorghum
	Apple
	Olive
	Almond
	Pear
	Cardoon
	Alfalfa
MKK	Sunflower
	Wheat

to briefly provide an outlook of the development stage of each crop in correspondence with the PRISMA and Sentinel-2 acquisitions.

In Italy, durum (Triticum durum Desf.) and winter wheat (Triticum aestivum L.), herbage, barley (Hordeum vulgare L.), pea (Pisum sativum L.), triticale (x Triticosecale Wittmack), and fava bean (Vicia faba var. equina L.) are sown in October/November and grow during winter and spring with harvest between May and July depending on the species. These species were considered here as winter crops. Maize (Zea mays L.), rice (Oryza sativa L.), tomato (Solanum lycopersicum L.), soybean (Glycine max L.), sunflower (Helianthus annus L.), and sorghum (Sorghum bicolor Moench) are planted from March to June and harvested from August to October depending on the species. These species were considered as summer crops. Apple, almond, and pear are perennial trees actively vegetating from March until November or December, while olive is an evergreen species. Cardoon (Cynara cardunculus L. var. Altilis) is a perennial crop, starting to grow in October, and is harvested in August. Alfalfa (Medicago sativa L.) starts to grow in March, and it is harvested approximately every 45 days (4 times per each growing season). These species are considered perennial crops.

Table 2. Availability of ground truth crop data of MKK, Grosseto, Maccarese, and Jolanda di Savoia sites for 2021–2023 winter, summer, and perennial cultivations. Group A is shown in green color, Group B is shown in red color. Species not being cultivated annually in the other group are shown by diagonal borders (×).

Cult.	Species		Sites
			Grosseto	MKK			Maccarese			Jolanda
			2020	2021	2022	2023	2021	2022	2023	2021	2022	2023

* D—durum, C—common.

reports the list of the ground truth crop data for Maccarese, Grosseto, Jolanda di Savoia, and MKK sites for 2020–2023 winter, summer, and perennial cultivations that are available along the time frame covered by PRISMA and Sentinel-2 acquisitions. Different test sites and different crops have been used within two training and validation scenarios: (i) same-farm TR/ACC and (ii) cross-farms TR/ACC. For the cross-farm TR/ACC, the dataset was divided into two groups: Group A (MKK, Grosseto, and Maccarese farms, shown with green color bar) and Group B (Jolanda di Savoia farm, shown with red color bar). For the cross-farm TR/ACC scenario, in order to cover all relevant species, the 2021 peas, sorghum, pear, and apple species growing in Jolanda di Savoia (Group A) and the 2022 triticale, olive, almond, and cardoon species growing on the farms in Maccarese (Group B), not being cultivated annually in the other group, were assumed to be cultivated in the other group. These data are shown by diagonal borders (×) in Table 2.

2.3.2. Satellite Imagery

PRISMA is a satellite hyperspectral push-broom sensor with a GSD of 30 m, a swath of 30 km, and a spectral resolution better than 12 nm in the spectral range from 400 nm to 2500 nm. Sentinel-2 is an operational wide-swath (290 km), high-resolution (10 m), and multispectral (13 bands) imaging satellite constellation with a 5-day revisit frequency. Based on the crop calendar (Table 1) and crop ground truth occurrence maps (Table 2), the dataset used for this study was divided into three cultivations consisting of winter, summer, and perennial crops. This study applied PRISMA hyperspectral images for all the sites, and 12 images from Jolanda di Savoia, 11 images from Maccarese, and 15 images from MKK were available. Moreover, the closest (±6 days to each PRISMA acquisition) Sentinel-2 cloudy free images (11 bands resampled to the 30 m spatial resolution) were used for reference and for classification comparison. A summary of all the imagery used is reported in

Table 3. Field data and PRISMA and Sentinel-2 images used for crop type mapping of MKK, Grosseto, Maccarese, and Jolanda di Savoia sites for 2020–2023. The PRISMA images used for Group A and Group B were shown in green and red color, respectively. The images highlighted in bold were assumed as if they were acquired in the other Group.

Site	Year	N. of Fields	Area (ha)	Sentinel-2	PRISMA Image Date (Month/Day)
Maccarese	2021	92	1600	6	04/01, 05/17, 06/27, 09/04
	2022	165	1600	6	01/16, 04/12, 05/29, 06/15, 07/14
	2023	70	700	2	02/02, 03/21
Jolanda	2021	176	3700	6	04/24, 06/04, 06/21
	2022	210	4100	8	04/30, 05/12, 07/03, 08/01
	2023	105	2863	4	03/04, 04/07, 05/24, 07/03, 08/07
Grosseto	2020	6	20	1	07/31
MKK-Iran	2021	24	120	5	01/23, 03/11, 04/09, 05/14, 05/19, 06/23, 07/22
	2022	30	310	7	01/29, 02/27, 03/28, 05/08, 05/25, 07/05, 07/22
	2023	26	130	3	02/28, 03/17, 04/09

, where the PRISMA images used for Group A and Group B are shown in green and red color, respectively. For the cross-farm TR/ACC scenario, to cover all the relevant species cultivated exclusively in Group A or Group B, the images highlighted in bold were assumed as if they were acquired in the other group, as similarly indicated with × in Table 2.

2.4. Pre-Processing and Processing EO Data

2.4.1. Pre-Processing

PRISMA L2D images were co-registered, with an Automated and Robust Open-Source Image Co-Registration Software (AROSICS v.1.11.0) algorithm using the Sentinel-2 image acquired at the closest date, to assure the co-registration (of about 0.5pixel of RMS), and then smoothed by a Savitzky–Golay filter (frame size of 7, 3rd degree polynomial). A total of 55 bands, including overlapping bands (band numbers 63–68), atmospheric absorption bands, and bands with SNR below 100, were excluded from processing. The image dimensions (composed by 173 bands after band removal) were reduced by the principal component analysis (PCA) approach and then PCs were normalized by computing the standard deviation of the image. An optimum of 15 and 5 PCs were selected for the PRISMA and Sentinel-2 image dimension reduction, respectively. To reduce the effects of the spectral mixing of the plants cultivated by farmers around fields, a negative 30 m buffer (1 pixel) at the edge of the fields was considered.

2.4.2. Machine Learning Classification Algorithms

Different machine and deep learning algorithms for crop type mapping using PRISMA and Sentinel-2 data were considered. To avoid errors that could happen between species with different cultivation seasons, we decided to develop separate classifiers for winter, summer, and perennial crop seasons. Each method is described in the following paragraphs.

Multiclass Naive Bayes (MNB): The MNB classifier is based on Bayes theorem and belongs to the family of generative learning algorithms, which means that it models the distribution of inputs for a given class or category. This approach assumes that the features of the input data are conditionally independent given the class, allowing the algorithm to make predictions quickly and accurately [51].

K-Nearest Neighbor (KNN): The KNN algorithm is an instance-based learning method that classifies elements based on the closest k training samples in the resource space. KNN is a non-parametric MLA that makes no assumptions regarding data distribution. This is important when classifying processes of land use change, for which there is little or no prior knowledge of data distribution. In KNN, the pixel whose class is unknown is assigned to the same class as its spectrally closest neighbors [52].

Random Forest (RF): The RF technique [53] is an ensemble classifier that employs many decision trees and combines their results to forecast an output. There are two main steps in building each decision tree in an RF: (1) creation of a new training set by random selection of subsamples through bootstrapping sampling from a specific number of samples in the original dataset, and (2) random selection of features. These two random processes help to avoid overfitting [54], because RF averages all these multiple decision trees, trained on different parts of the same training set, and assigns each pixel to the class label that garnered the most “votes”. The number of trees is obtained using out-of-bag data analysis, and the number of features is obtained using the square root of the features. The RF algorithm, in particular, can process high-dimensional data, works well with mislabeled data, and has high accuracy for large-scale studies [55].

Support Vector Machine (SVM): SVM was originally proposed as a linear binary classifier. Subsequently, by introducing suitable kernels, the algorithm can perform nonlinear classification. Previous studies show that the radial basis function (RBF) kernel works properly for classification [56]. The RBF kernel was used in this research, where C (cost) and gamma parameters were optimized. The optimal parameters were determined using the grid search method considering its shorter run time for finding the best combination of C and gamma parameters compared with other methods. C is the penalty coefficient that determines the allowed level of error or misclassification. Therefore, a larger C always achieves a better result in training, but the generated model has a risk of overfitting, which decreases the classification’s generalization. The smaller the C, the higher the number of unclassified samples [57].

Hyperparameters tuning: The performance of the ML algorithm depends on hyperparameter adjustment. The grid search method was used in sequence to tune the hyperparameters of ML algorithms. In the grid search method, the domain of hyperparameters is discretized to a grid. Then, the performance of all possible combinations is assessed using statistical metrics (here, cross-validation (CV)). The set of hyperparameters that can maximize the average value in CV is selected as an optimal one for training the model.

Table 4. Hyperparameters tuning process for MNB, KNN, SVM, and RF algorithms.

MNB	Parameter	Distribution		Kernel type
	Range	Gaussian, kernel		Box, Epanechnikov, Gaussian, Triangle
	Tuned	Kernel		Gaussian
KNN	Parameter	Distance weights	n_neighbors	Distance metric
	Range	Equal, Inverse, Squared inverse	1–21	Euclidean, Minkowski, Spearman, Hamming, Jaccard
	Tuned	Equal	7	Euclidean
RF	Parameter	NumLearningCycles	Method	MaxNumSplits	MinLeafSize
	Range	10–500	Bag, LSBoost	1–50	1–50
	Tuned	100	LSBoost	15	10
SVM	Parameter	Gamma	C	Kernel type
	Range	0.001–100	0.001–100	Gaussian, Linear, Quadratic, Cubic, RBF
	Tuned	10	1.27	RBF

presents the tuned values for the hyperparameters of the ML algorithms. For the other parameters of ML algorithms, the default settings were used. All the ML classification methods and hyperparameter optimization of the present work were implemented in the Classification Learner App in MATLAB 2023b.

2.4.3. Deep Learning Classification Algorithms

A convolutional neural network (CNN) usually consists of a sequence of convolution layers and pooling layers, followed by a fully connected layer. The main advantage of CNN to regular neural networks is that the input of CNN models is an image, which allows specific features to be encoded into the model. Those features then help to reduce the number of network parameters [58]. The advantages of the CNN algorithm are automatic feature learning, multilayer feature learning, high accuracy, and high generalizability [59,60]. The CNN method is popular for working with high-dimensional features, performing hierarchical learning operations, and enabling automatic feature extraction. Two kinds of CNNs (1D and 3D) have been used for classification [61].

One-Dimensional Convolutional Neural Networks (1D-CNN): 1D-CNN uses 1D filters to record the input signal spectral pattern. The implementation of 1D-CNN combines pooling layers, fully-connected layers, and dropout. Convolutional filter widths of 3 and 5 were used. The convolutional layers have 32, 64, and 128 filters, respectively, and the channel number increases when going deeper. Pooling layers were fixed as max-pooling, with a window size of 2. The probability of dropping neurons was set to 10%. This model had one fully connected layer at the output end. The second last layer collected information from previous layers as a flat array, and the size was determined by considering the dimension of the input to the layer. The learning rate was set to 0.001 and the batch size was set to 256. To keep only the positive values, rectified linear unit (ReLU) was used (Figure 4).

Three-Dimensional Convolutional Neural Networks (3D-CNN): Other than spectral data, 3D-CNN takes advantage of the spatial information (which is not present in other methods) [60], which leads to increasing the computational cost. 3D-CNN executes successive sliding convolution operations in the input feature cube’s plane and depth dimensions. Convolutional filter widths of 7, 5, and 3 were used, based on the smallest field of this study’s sites. The convolutional layers had 32, 64, and 128 filters, respectively. For 3D-CNN, the network was trained by an initial learning rate of 0.001 for 100 epochs, a 256-batch size, momentum 0.9, learning rate factor 0.01, and Adam optimization. Different spatial window size was used to find an optimum window size. For each pixel, a 4D patch with size m × n × d × l, where m × n refers to spatial window, d refers to spectral window, and l refers to label, were computed and used as input of CNN (Figure 4).

2.5. Classification Scenarios

Two scenarios were proposed for training and validation.

Same-farm TR/ACC: TRaining and ACCuracy assessment (TR/ACC) occurred in the same field. In this scenario, the pixels were randomly divided into TR (70%) and ACC sets (30%).

Cross-farm TR/ACC: To consider the spatial/temporal transferability of the trained model, TR/ACC data were selected from different farms. For sorghum, apple, and pear (Jolanda) and olive and almond (Maccarese) crops, which are not present across all farms, cross-year data from same-farm were used (Table 2).

2.6. Accuracy Assessment Scenarios

The same TR/ACC pixels were used for all the applied methods to avoid the effects of different partitioning of the dataset on the performance of methods. User accuracy (UA) and producer accuracy (PA), overall accuracy (OA), and Kappa coefficient were determined from the confusion matrix with respect to the ground validation data. The accuracy validation metrics are defined as follows:

$$\mathrm{O}\mathrm{v}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{l}\mathrm{l}\mathrm{ }\mathrm{a}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}\mathrm{ }(\mathrm{O}\mathrm{A})=\frac{{\sum }_{n=1}^q{\mathrm{n}}_{ii}}{n}\times 100$$

(1)

$$\mathrm{K}\mathrm{a}\mathrm{p}\mathrm{p}\mathrm{a}\mathrm{ }\mathrm{c}\mathrm{o}\mathrm{e}\mathrm{f}\mathrm{f}\mathrm{i}\mathrm{c}\mathrm{i}\mathrm{e}\mathrm{n}\mathrm{t}=\frac{\left[n.{\sum }_{n=1}^q{\mathrm{n}}_{ii}-{\sum }_{n=1}^q{(\mathrm{n}}_{.i}\times n_{.i})\right]}{\left[n^2-{\sum }_{n=1}^q{(n}_{i.}\times n_{.i})\right]}$$

(2)

$$\mathrm{U}\mathrm{s}\mathrm{e}\mathrm{r}\mathrm{ }\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{y}\mathrm{ }(\mathrm{U}\mathrm{A})=\frac{{\mathrm{n}}_{ii}}{n_{i.}}\times 100$$

(3)

$$\mathrm{P}\mathrm{r}\mathrm{o}\mathrm{d}\mathrm{u}\mathrm{c}\mathrm{e}\mathrm{r}\mathrm{ }\mathrm{A}\mathrm{c}\mathrm{c}\mathrm{u}\mathrm{r}\mathrm{a}\mathrm{c}\mathrm{ }(\mathrm{P}\mathrm{A})=\frac{{\mathrm{n}}_{ii}}{n_{.i}}\times 100$$

(4)

where n_ii, n._i, and n_i. are the values of the diagonal entry, the i-th row sum, and the i-th column sum in the confusion matrix, respectively; n is the total number of validation samples; and q is the number of rows of the confusion matrix.

3. Results

3.1. Spectral and Temporal Behavior of Species

Gaining knowledge on the spectral and temporal behavior of species in advance could lead to the design of a better procedure for classification. Figure 5 shows the simulated phenology of the dominant crop species of the Maccarese and Jolanda di Savoia farms (Table 1) derived from the Sentinel-2 normalized difference vegetation index (NDVI) time series data of 2023. As shown in Figure 5a, winter and durum wheat, barley, and triticale have high similarity in phenology patterns. Fava beans and peas start growing later than wheat and barley and are harvested at around the same time. Cardoon starts growing in autumn and has totally different phenology, thus it is expected to be discriminated easily from other species in early winter (Figure 5a). Maize, sorghum, rice, soybean, sunflower, and tomato species start to green-up from May but are harvested from September to October (Figure 5b). Peach, apple, and almond orchards start to green-up from January until October in this study’s areas. Olive is an evergreen plant, and, even in winter, the NDVI is higher than 0.25. Simulation of alfalfa using the NDVI of Sentinel-2 was not successful, because alfalfa is cut 4–5 times between May and September, which makes the phenology pattern very different from other species. For this reason, instead of simulated phenology, the NDVI time series derived from Sentinel-2 is presented (Figure 5c).

In Figure 6, the PRISMA-derived spectral signature of the species (acquired in 2023) is shown. The similarity between the species’ spectra, calculated using the cosine similarity method (cosine similarity function of MATLAB), is shown as a heat map in Figure 6e. The spectral signatures of winter and durum wheat, and barley acquired on 1 April 2021 at the Maccarese farm show high similarity ‘>95%’) (Figure 6a), while the rest of the winter species (herbage and fava bean acquired on 1 April 2021 at the Maccarese farm and pea acquired on 24 April 2021 at the Jolanda di Savoia farm) show less similarity (Figure 6b). For summer crops (maize, sorghum, rice, soybean, sunflower, and tomato acquired on 3 July 2022 at the Jolanda di Savoia farm), there is less similarity between species’ spectral signatures, except between sorghum and maize. Rice shows a lower value compared with other summer species in the short-wave infrared (SWIR), which could be related to the presence of water in rice fields (Figure 6c). For perennial crops (olive, almond, and alfalfa acquired on 17 May 2021 at the Maccarese farm and pear and apple acquired on 4 June 2021 at the Jolanda di Savoia farm), pear shows less absorption in the red (630–680 nm) and higher reflectance in the 1300–2400 nm spectral range with respect to other perennial species (Figure 6d).

Figure 7a shows the spectra of a specific pixel of the wheat fields from April and May’s PRISMA images. In April, the wheat is in the heading stage and has strong chlorophyll absorption and high reflectance in the 800–1200 nm spectral range. In May, wheat is in the growth stage between flowering and ripening, exhibiting weak red absorption (less chlorophyll) and strong absorption in the 2100 nm spectral range (i.e., high lignin content). Based on the achieved primary result, the after-April and August images were eliminated from the winter and summer cultivation TR/ACC, respectively. Figure 7b shows the three different fields of alfalfa species at the Jolanda di Savoia farm in the acquired image on 3 July 2022. These fields are in different growing stages because of different harvesting dates and accordingly have different spectral reflectance.

3.2. Earliest Identifiable Time of Different Crops

Considering the earliest time that it is possible to apply crop type mapping, the classification was performed by the first available image in the season, and then all of the next available images were added one by one to the TR/ACC of the algorithms.

Figure 8 presents the OA of 1D-CNN algorithm applied to the time series of PRISMA images for each crop from 30 January (DOY 30) to 7 August (DOY 219). For winter (Figure 8a), cultivation DOY 81 is the first time the OA of all species reached 80% or higher. Wheat and triticale have lower OA until the end of the season, while the fava bean gains the best OA after DOY 102.

For summer (Figure 8b) cultivation, DOY 172 is the first time the OA of all species reached 80% or higher. Rice OA is reduced after DOY 190. Sorghum and soybean could be identified 20 days earlier than other species. For soybeans, this result depends on the fact that it is seeded before other species. For perennial (Figure 8c) cultivation, DOY 91 is the first time the OA of all species reached 80% or higher. Almond and cardoon could be identified 20 days earlier than other species

3.3. Classification Accuracy

The OA (for all cultivations) of classification using different machine and deep learning methods for PRISMA and Sentinel-2 images are shown as a bar graph in Figure 9. Figure 9 depicts that 1D-CNN and 3D-CNN methods were the most performing classifiers for all the seasons and sensors.

For same-farm TR/ACC of PRISMA, 1D-CNN and 3D-CNN provided overall accuracy higher than 95%, while for cross-farm TR/ACC, the OA was higher than 88%. 1D-CNN provided a 1–4% better result compared with 3D-CNN. Following these, CNN, RF (76% < OA < 79%), and SVM (73% < OA < 77%) algorithms showed the highest accuracy for cross-field validation. The accuracy of KNN (67% < OA< 73%) was slightly lower than RF and SVM, while the result of the MNB (OA < 53%) method was poor.

For Sentinel-2 image same-farm TR/ACC, 1D-CNN and 3D-CNN derived an overall accuracy higher than 88%, while for cross-farm TR/ACC, the OA was higher than 79%. 1D-CNN provided better results in winter and summer crops, while 3D-CNN provided better results for perennial crops. The RF (62% < OA< 66%) and SVM (60% < OA < 64%) algorithms had the highest accuracy for cross-farm TR/ACC after CNN methods. The accuracy of KNN (58% < OA < 64%) was a bit lower than RF and SVM, while the result of MNB (43% < OA < 47%) was poor.

The per-species classification PA and UA derived from PRISMA and Sentinel-2 images by the MNB, KNN, SVM, RF, and 1D-CNN and 3D-CNN classification algorithms are reported in Figure 10. Winter and durum wheat are merged in order to be counted as the same species to have a higher accuracy, especially in PRISMA images. For PRISMA image classification, among the other species, alfalfa showed the lowest UA using all the methods. The rest of the species did not show a stable trend in all methods. For PRISMA, alfalfa had the lowest UA, and maize showed the highest UA when in the cross-farm TR/ACC scenario. For Sentinel-2, alfalfa showed (similarly to PRISMA) the lowest UA, while winter wheat was characterized by the highest UA when TR/ACC in cross-farm (Figure 10).

Figure 11e,g show the classification maps for the year 2022 obtained using the 3D-CNN for the Jolanda di Savoia and MKK farms, respectively. Figure 11f shows the map of the Maccarese farm produced by the 1D-CNN method for the year 2021. Figure 11h shows the 1D-CNN method of the Grosseto farm for 2020. To avoid assigning two different labels to the same pixel, the fields used for both summer and winter cultivation (a few fields) were eliminated. The winter, summer, and perennial crop maps were merged to produce a comprehensive crop map of each field for one year. On all sites, it can be observed that most of the errors of omission and commission occurred in the pixels located at the margins of the fields. Classification errors in field edge pixels were higher than in central pixels, which was related to spectral mixing with neighboring fields with no similar cultivation. It was also related to the presence of trees/brushes surrounding the cultivated fields and water channels. A negative 30 m buffer (1 pixel) at the edges of fields resulted in 1% and 3% improvement in the OA of classification by 3D-CNN and SVM, respectively. This could have an impact on the usability of PRISMA in fragmented agricultural areas in the case that pansharpening techniques are not applied [62,63,64].

4. Discussion

4.1. Effects of Reflectance Temporal Variation and Field Heterogeneity on Classifier Performances

In this study, we split the cultivar into three seasonal categories (winter, summer, and perennial). Based on the plant phenology provided by the Sentinel-2 NDVI time series (Figure 5), it is evident that the same-season crops show limited differences in the growing phenology. As an example, the higher accuracy in the detection of cardoon and olive orchards (Figure 8c) occurred in the early season images (related to the earlier growing stage) (Figure 5c), while for peas, the later phenological growing stage led to a one-month delay in comparison with other winter-season species (Figure 5a).

Among the species, alfalfa showed the lowest accuracy as a direct factor related to the multiple harvesting patterns (Figure 5c) and the difficulties in obtaining PRISMA images suitable to catch its growing behavior. This is also depicted in Figure 6c, where alfalfa shows low spectral similarity with the winter crops while showing high similarity with respect to the summer crops growing in accordance with the harvesting cycle. As mentioned in the study [65], temporal resolution is more important than spectral resolution for alfalfa (the high capability of Sentinel-2 time series data for alfalfa mapping is also highlighted in the literature).

The spectral signatures of the analyzed crops do not show clearly differences between species (Figure 6e). These high spectral similarities lead to increasing the classification omission and commission errors between barley, wheat, and triticale species. The reason is related both to the similarity between the growing calendar (Figure 5) and their spectral behavior (Figure 6).

This result agrees with the work of Wilson et al., 2015 [65], that taking advantage of the growing season maximizes the accuracy in the discrimination of soybeans, canola, wheat, oats, and barley using hyperspectral reflectance. Furthermore, in accordance with Buchhart et al. [66], the crop temporal spectral signatures change according to the growing season, as pointed out by Figure 7. This is connected to the change in the plant’s biophysical/biochemical characteristics (e.g., canopy structure, chlorophyll, and water content [67]), in accordance with the different growth stages [68]. Methods like feature selection applied to spectral data [69] could be useful to minimize the influence of growth stages on crop identification accuracy, but this still needs to be further analyzed in accordance with the study of Sun et al. [70] and Graeff and Claupein [71], which observed that reflectance temporal changes pattern should be considered within the algorithm configuration. This issue can be better faced taking advantage of the increasing availability of hyperspectral satellites offered by EnMAP, EMIT, and DESIS, and upcoming missions like CHIME and PRISMA-SG. At this scope, the conventional CNNs or trending sequence models, like recurrent neural networks (RNNs) and transformers, can hardly address (i.e., simultaneously and efficiently) the spectral/temporal variability issues. For future work, the authors’ suggestion is to tackle this variability issue by implementing state space models like SpectralMamba [72].

4.2. Effects of Field Heterogeneity on Classifier Performances

The spectral differences that can occur within the same field could be related to the different soil properties in term of texture and organic carbon (soil fertility) that could affect the growing rate and then the biophysical and biochemical properties of plants [73]. These changes in soil properties have a clear impact on the spectral signature that could lead to an incorrect classification. This has been observed clearly at the Jolanda di Savoia site, where texture within a field could vary significantly where paleo riverbeds determine sharp variations in the soil fertility [74] as they are related to sharp passages from clay to silty soils (Figure 12c). Figure 12a shows the 2022 crop map identifying the wheat, while Figure 12b shows the 3D-CNN classification results, leading to an erroneous classification of the sparse area classified as barley instead of wheat with a few sparse pixels of pea that, as shown in Figure 6e, are characterized by a high similarity impacting the covariance used by the classification algorithms. Figure 12c shows the PRISMA true-color RGB of an image acquired on 30 April 2022 from the Jolanda di Savoia farm. Two different areas, depicted in Figure 12a, with red and yellow boxes, correspond to “sparse” and “dense” wheat selected to compare their reflectance, which is shown in Figure 12d. The spectral difference related to sparse and dense vegetation appears to be important along all the spectra, a difference large enough to confound the algorithm in the crop species detection.

4.3. Early-Stage Crop Mapping Using ML and DL Algorithms

Regarding the performances of the different algorithms tested, for all cultivations, (Figure 9) both CNN (1D and 3D) methods showed to be the best performing classifiers, achieving OAs higher than 88% and 79% for PRISMA and Sentinel-2, respectively. In all the cases we considered, 1D-CNN produced higher OA than 3D-CNN when applied to the PRISMA imagery. Similarly, for the Sentinel-2 time series, with the exception of the cross-farm TR/ACC scenarios applied to the perennial cultivation, the 1D-CNN provided higher OA than the 3D-CNN (Figure 9).

Following 1D-CNN and 3D-CNN, the RF (OA higher than 76% and 62% for PRISMA and Sentinel-2, respectively) and SVM (OA higher than 73% and 60% for PRISMA and Sentinel-2, respectively) algorithms gained the next level of accuracy. The accuracy of KNN (OA higher than 67% and 58% for PRISMA and Sentinel-2, respectively) was lower than RF and SVM, while the result of the MNB (OA higher than 46% and 43% for PRISMA and Sentinel-2, respectively) method performed not well, showing lower accuracy (Figure 9).

If we analyze the classification performance along the available image time series, assuming an OA of 80% as an acceptable threshold for a suitable crop mapping, then, at DOY 81, all winter species can be recognized by PRISMA images (Figure 8a). The triticale showed the maximum OA earlier than wheat and barley in accordance with the different winter development among crops (Figure 8a). Wheat and triticale had lower OA until the end of the season (Figure 8a). The spectral similarity between barley triticale and winter wheat was high (Figure 6e), which led to confusion for the classifiers. The fava beans gained the best OA after DOY 102, as it was related to later growth start with respect to the other species (Figure 7a).

For summer cultivation, DOY 172 was the first time for which all species could be separated by PRISMA images (Figure 8b). Rice OA reached the maximum earlier and then started to reduce after DOY 190. Sorghum and soybeans could be identified 20 days earlier than other species, while maize and sorghum showed the better identification after DOY 185 (Figure 5b). For perennial cultivation, DOY 91 was the first time for which they could be separated by PRISMA images (Figure 6c) and, among them, almonds could be identified 20 days earlier than other species. Cardoon showed the highest user accuracy until DOY 91 (OA up to 92%). This was also related to the fact that cardoons started to grow before other species (Figure 5b), which led to having less similarity in reflectance during the first growing stages.

We were able to estimate the cultivated area for maize, tomato, and cardoon two months ahead of harvest. For soybeans, wheat, barley, triticale, and peas, the estimation could be carried out about three months before the harvest, while, for rice, sunflower, and sorghum, we could estimate their cultivated areas about four months before the harvest. This information is very useful for a range of applications, including monitoring for compliance with agricultural policies such as the EU Common Agricultural Policy (CAP), in terms of surfaces declared by farmers for crops. Additionally, since the proper identification of cultivated areas is a fundamental requirement for yield forecast analysis at the regional scale, this information is extremely valuable for entities dealing with both food security and food commodity trading, as well as insurance. This supporting information allows to detect at the earliest the critical regions that could suffer food crises, allowing local governments and international humanitarian organizations to plan remediation activities [75]. Moreover, it is crucial to have current information about the planting locations and potential supplies of cereals (maize, wheat, and rice) and legumes (soybeans, peas) months before harvest in order to predict more accurately yields and expected food prices [76].

4.4. The Effects of Pixel/Field Size on the 3D-CNN

As shown in Figure 9, the 1D-CNN and 3D-CNN show better performance in comparison with other classical ML algorithms and the 1D-CNN shows a slight superiority to 3D-CNN. The superiority of 1D to 3D-CNN was not expected, because 3D-CNN takes advantage of the spatial information besides spectral data, while the 1D-CNN method is only based on spectral data and considers each pixel as independent. To better understand the effect of field size on the performance of these two methods, the fields were classified based on their width into three categories (small when <150 m, moderate in the range 150–250 m, and large when >250 m). The Kappa coefficient of 1D-CNN and 3D-CNN methods for these categories are shown in Figure 13. The Kappa coefficients for large field classifications using 3D-CNN and PRISMA and Sentinel-2 images were 0.839 and 0.74, respectively. For small fields, the Kappa coefficients were reduced to 0.75 and 0.66, respectively. The Kappa coefficient of the 1D-CNN method did not show a significant change between different categories for PRISMA- and Sentinel-2 image-derived maps.

The limited results of 3D-CNN with respect to the 1D-CNN were hence strictly related to the small field size and, since most of the Maccarese and Jolanda di Savoia fields had widths lower than 200 m (Figure 1d), this made it difficult for the 3D-CNN algorithm to perform well. It is, therefore, expected to overcome this limited performance of the 3D-CNN in the case of future pansharpening of the hyperspectral at 30 m to 5 m resolution offered by the co-registered PRISMA PAN or by new missions like PRISMA-2 (having a higher spatial resolution mode).

4.5. Cross-Farm TR/ACC for Unavailable Data Situation

Comparison of the OA results between the same-farm and the cross-farm TR/ACC scenarios (Figure 9) showed significant differences in the classifiers’ performances. Regarding PRISMA data (reported in panels (a) and (b)), apart from a general decrease in accuracy, it should be noted that the CNN algorithms presented a limited reduction of about 6% in OA, while for the SVM, RF, and MNB algorithms, the OA was reduced by 17%, 15%, and 22%, respectively. The stability of CNNs in the cross-farm TR/ACC scenario seems quite promising and deserves consideration in future studies. The results (Figure 9) show that the classifiers’ performances in terms of accuracy were lower with respect to the cross-farm TR/ACC compared with the same-farm scenario. With respect to the same-farm scenario, just a limited reduction of 6% OA was observed by using the CNN algorithms, while OA was significantly reduced by 17%, 15%, and 22% for SVM, RF, and MNB, respectively. The reason behind the stability of CNNs in comparing the ML algorithms in the cross-farm TR/ACC scenario needs additional consideration.

The transfer learning scheme based on deep learning approaches helps to gain crop mapping results in transfer sites, even though abundant historical labels of crop type are required. This study states an innovative data set offered by the combination of hyperspectral imagery and known related ground truth. As a preliminary exploitation of the hyperspectral time series potential, we decided to use trivial cross-farm approaches to simply mimic more complex learning approaches testing different TL types and theories (e.g., fine-tuning, feature extraction, few-shot learning, domain adaptation).

Moreover, this study highlights some points that are to be considered for further study. Dimension reduction/feature extraction being performed at present with a trivial PCA could be enhanced with more sophisticated feature extraction methods, better preserving the spectral information contents, or balanced with techniques like stacked auto-encoder (SAE). Furthermore, the self-attention mechanism could be used to focus on the most informative features. Future work should explore the capability of developed foundation models (e.g., SAM, GPT, DINO) in excluding the feature extraction step within the present ML workflow.

5. Conclusions

In this study, we exploited the first available PRISMA time series collected by the hyperspectral satellite over the more densely surveyed agricultural test sites located in Europe and Iran. PRISMA time series were processed to tune and test ML and DL classification approaches for large-scale crop mapping in agriculture, with particular attention paid to early crop mapping. The comparison of the performance of six classification methods—MNB, KNN, RF, SVM, 1D-CNN, and 3D-CNN—on PRISMA and Sentinel-2 time series confirmed the powerfulness of the PRISMA times series when combined with the 1D-CNN algorithm in the production of an accurate crop mapping and in the definition of a precise early crop map product. All common crops from three agricultural sites in Italy and from a region with strong diverse agronomy in South Iran were in fact accurately detected. 1D-CNN showed the highest accuracy in the crop type identification, followed by 3D-CNN, RF, SVM, and KNN as the next accurate methods (89%, 91%, and 92% for winter, summer, and perennial cultivations, respectively). Classification accuracy remained comparable whether a cross-farm or single-farm training scenario was utilized (reduction of 6% in AO). The product accuracy obtained by using the Sentinel-2 time series was substantially lower compared with PRISMA (14%, 15%, 18%, 19%, and 23% for 1D-CNN, 3D-CNN, SVM, RF, and MNB, respectively). The results demonstrate that, when using the PRISMA time series and the 1D-CNN algorithm, early-season crop mapping in the northern Mediterranean agricultural sites is feasible at the appropriate time, which aids with decision making regarding food security and marketing.

These results show the way to use the hyperspectral time series, even though still timely discontinuous, with agricultural applications on a global scale. Moreover, it is expected that future algorithm development, especially in context of the feature selection in the combined temporal and spectral domains, will boost classification performance.