Machine Learning Classification of Fused Sentinel-1 and Sentinel-2 Image Data towards Mapping Fruit Plantations in Highly Heterogenous Landscapes

Abstract

Mapping smallholder fruit plantations using optical data is challenging due to morphological landscape heterogeneity and crop types having overlapping spectral signatures. Furthermore, cloud covers limit the use of optical sensing, especially in subtropical climates where they are persistent. This research assessed the effectiveness of Sentinel-1 (S1) and Sentinel-2 (S2) data for mapping fruit trees and co-existing land-use types by using support vector machine (SVM) and random forest (RF) classifiers independently. These classifiers were also applied to fused data from the two sensors. Feature ranks were extracted using the RF mean decrease accuracy (MDA) and forward variable selection (FVS) to identify optimal spectral windows to classify fruit trees. Based on RF MDA and FVS, the SVM classifier resulted in relatively high classification accuracy with overall accuracy (OA) = 0.91.6% and kappa coefficient = 0.91% when applied to the fused satellite data. Application of SVM to S1, S2, S2 selected variables and S1S2 fusion independently produced OA = 27.64, Kappa coefficient = 0.13%; OA= 87%, Kappa coefficient = 86.89%; OA = 69.33, Kappa coefficient = 69. %; OA = 87.01%, Kappa coefficient = 87%, respectively. Results also indicated that the optimal spectral bands for fruit trees mapping are green (B3) and SWIR_2 (B10) for S2, whereas for S1, the vertical-horizontal (VH) polarization band. Including the textural metrics from the VV channel improved crop discrimination and co-existing land use cover types. The fusion approach proved robust and well suited for accurate smallholder fruit plantation mapping.

1. Introduction

Timely and precise spatial information on crop types is crucial for managing and monitoring agricultural activities, especially in areas with increasing populations and rapidly growing food security demands [1]. Accurate location monitoring of agricultural land at a crop types level plays an essential role in precision agriculture and achieving a better understanding of farm management practices, forecasting crop yields, and facilitating an enhanced response to food insecurity [2,3]. Promoting good agricultural practices and achieving sustainable agricultural systems depend on detailed spatial information of the current cropping systems on small-field parcels, especially for South Africa [4]. The fruit-tree industry is a pillar that provides livelihoods, creates employment opportunities, and reduces poverty and food insecurity by generating income for local agricultural-based livelihoods while contributing to sub-tropical rural economic development [5,6]. Production of fruit-tree crops is increasing worldwide, and export sales have become more competitive [7]. Thus, there is a need to develop effective management to improve fruit-tree crop production and advance the competitive standing of small-scale farmers. One of the essential requirements for precise crop management is providing decision support tools that offer up-to-date spatial information on the tree crops systems and dynamics to maximize production and returns [8,9].

Historically, crop types were conventionally identified and mapped using field-based methods such as ground surveys and agricultural statistics as they work well at small local scales [10,11,12]. Although this method provides accurate spatial data about crop type distribution, many studies have shown the method is costly, time-consuming and labour intensive and is not feasible for large-scale crop mapping [13]. As such, the agricultural industry has undergone a digital revolution characterized by technologies such as artificial intelligence, remote sensing and geographical information systems specifically designed to advance the agricultural sector [14,15]. These technologies are promising tools for innovating agricultural operations, fostering the adoption of precision agriculture measures, and enhancing production [16,17].

Remote sensing has shown potential in overcoming the above mentioned limitations as it provides accurate crops information, as well as timely and cost-effective farming opportunities needed for site-specific crop management [18]. It is a tool used in conjunction with machine learning algorithms, big data analytics and artificial intelligence to extract useful information on fruit species regardless of the varieties [19]. The unprecedented availability of high-resolution spectral, spatial and temporal resolution images has advanced the use of remote sensing in crop mapping over various scales and geographic locations [20].

The application of different satellite technologies for decision making and precision agriculture in horticulture has been shown in various studies [12,21,22]. Among the various satellite types, MODIS data has been widely tested in crop mapping at regional and global scales due to its global coverage [23]. However, the efficacy of using MODIS data in crop types mapping is limited to commercial farming [24]. Its solutions for precision farming practices are challenging to envisage and implement due to MODIS spatial resolution resulting in mixed pixels and deteriorated classification accuracy when applied to heterogeneous terrains in which smallholder farmers operate [25].

Remote sensing data with intermediate spatial resolution < 30 m (Landsat 8) and high spatial resolution < 10 m, such as, Quickbird, Worldview, SPOT, and Sentinel-2 data, are used to overcome the problem of spectral mixing and improve crop classification in agricultural systems characterized by small fields [26]. The applicability of high spatial resolution data has been tested in producing accurate site-specific geospatial information for precision agriculture due to their resolutions being smaller than smallholder fields [27,28,29]. The regional and localized applications of high-resolution data include cropland mapping [30], identification of irrigated crops [31], and tree crops condition mapping [32]. However, the applicability of the above datasets for designing precise sustainable farming solutions in smallholder farming systems in sub-Saharan Africa is still limited due to various dimensions such as lack of modern hardware and network infrastructure, software, technological and technical skill gaps and lack of access to funding [25].

Optical data are weather-dependent and rely on the sun’s energy, and are mainly affected by clouds, thus, hampering their applicability in crop mapping in subtropical regions such as South Africa [33]. The synthetic aperture radar (SAR) data from the Copernicus program has unlimited availability and is a powerful tool that can be used to overcome the impact of optical data unavailability, as radar systems lack illumination and can penetrate cloud covers [34,35]. The SAR sensors can differentiate the crops geometric and structural features due to their ability to capture textural information consisting of pixel intensities characterized by spatial variations that can detect subtle structural plant traits [20]. The radar systems are sensitive to geometric and dielectric properties of the crops enabling the systems to transmit emissions and backscatter from the objects, rendering them viable for vegetation mapping [35]. However, the lack of spectral features and sensitivity to the surface parameters (e.g., vegetation, soil moisture, soil roughness) limit their applicability in crop mapping [36]. Conversely, the leaf structure and water content influence the reflected energy from optical sensors, rendering them effective for crop mapping [37].

In recent studies, the hypothesis of data fusion in increasing crop mapping accuracy by capturing vegetation structural information representations from SAR and optical has been supported [38,39]. Data fusion compensates for the limitations of single-source sensors by combining different sensor spectral, spatial and temporal characteristics offering broader information content of the target object [38,40]. The growing research on crop types mapping reflects the diverse nature of agricultural landscape locally, regionally and globally and its site-specific challenges such as spectral similarities and intra-class variability, crop diversity, environmental and weather conditions, and farming system [38,40,41,42]. Different approaches and earth observation data have been tested to improve crops classification. The combination of Sentinel-1 and Sentinel-2 is well investigated in various countries such as Belgium, Australia, India, and China [3,43,44]. These studies confirm that SAR data improve crop types classification accuracies and overcome the limitations of cloud cover. This is due to SAR data characteristics and signals sensitivity to plant canopies’ physical and structural properties, which complements optical information from multispectral sensors such as Sentinel-2 [44,45]. However, the potential for mapping smallholder farming systems, particularly in African countries, has not been thoroughly tested and remains challenging due to the high spatial and temporal dynamics of agricultural landscapes in Africa. For example, in rural areas of South Africa, the farmland depicts a small scale but very diverse mix of crops such as maize, fruit trees and vegetables. Fields are relatively small and often separated by woody vegetation or plantation; the growing conditions and mix crops phenology and canopies physical structure vary enormously [46,47]. The optical sensors’ observations show high spatial and temporal crop spectral variability, and therefore, they may not be adequate to map crop types in small farms accurately [26,34].

Accurate remote sensing depends not only on the data characteristic and the site specifications but also on the classification methods. The selection of classifiers in image classification depends on the size of the crop field, landscape complexities, management strategies, and the diversity of the crops [48]. Machine learning-based classifiers such as support vector machines (SVM), random forest (RF), decision trees, and k_nearest neighbors (KNN) have been used to create thematic maps from satellite images for different fruit-tree crops with reasonable accuracies [48,49,50,51,52].

Levubu is one of the sub-tropical farming areas that makes Limpopo Province a leading fruits export in South Africa [5]. Smallholder farming systems (defined here as farms under 2 ha) play a significant role in global food production and are the backbone of South Africa’s economy [4]. These agricultural systems create employment opportunities that generate income and ensure food security for the local agricultural-based livelihoods [5]. The Levubu farms are predominantly planted with fruit tree crops such as macadamia nut, avocado, banana, guava, mango, pine trees. The co-existing land-use types include water bodies, built-up areas, and woody vegetation. Hence, mapping these fruit trees with higher accuracy is necessary for precise agricultural management and improving food production. Despite being the most prominent fruit exporter in Limpopo Province, the management of these fruit tree crops relies on ground-based surveys that are time-consuming and often subjected to human error [26].

The European Space Agency (ESA) provides SAR and multispectral data with a high spatial resolution, giving ample opportunities for crop phenotyping in smallholder agriculture in sub-tropical regions [53]. Sentinel-1 is suitable for smallholder crop mapping due to different polarization channels (VH and VV), which interact with crops differently, providing increased data content to support decision-making in a cost-effective manner [54]. The potential for integrating the Sentinels products has been tested for mapping crop growth conditions, leaf area index, and crop types patterns in various landscapes [13,43,55]. However, the Sentinels products’ integrated use has not been tested in mapping smallholder fruit plantations in South Africa despite their free availability, consisting of suitable spatial and spectral resolutions that offer unlimited capabilities for mapping at the landscape level. There is limited research in South Africa that has integrated the SAR and optical data from Sentinel satellite sensors to map smallholder fruit-tree crops due to complex image processing and complex sensor interactions with target parameters [33]. Therefore, the study aims to address this problem by investigating the integrated use of Sentinel 1 and Sentinel-2 data in fruit-tree crops mapping using machine learning classifiers. The objectives of the research are:

• To evaluate the effectiveness of RF and SVM in discriminating tree crops from each sensor separately;
• To examine if the application of RF and SVM classifiers on fused data can improve the crop mapping accuracy at the landscape level;
• Assess the use of selected important variables in improving the mapping accuracy of fruit tree crops at the landscape level.

2. Materials and Methods

2.1. Method Overview

The proposed methodology shown in Figure 1 consists of the following steps: image pre-processing, converting Sentinel-1 (S1) image intensities (VV and VH) into decibels (dB), resampling six bands of Sentinel-2 data (i.e., rededge_1(B5), rededge_2(B6), rededge_3(B7), narrow NIR(B8A), SWIR_1(B11) and SWIR(B12)) were resampled from 20 m to 10 m spatial resolution using the Nearest Neighbor algorithm. After that, the resampled bands were combined with the 10 m resolution bands (i.e., Blue: B2, Red: B3, Green: B4, NIR: B8), and the resulting image composite consisted of 10 spectral bands. The image classification process involved the creation of image composite using S1 and S2, variable selection on the fused image (S1S2). The ground truth data were used to extract the image reflectance using four imageries (S1, S2, S1S2, S1S2 variable selection) and classify them using random forest (RF) and support vector machine (SVM). The accuracy assessment was carried out on the classified maps to evaluate the image that produced the best fruit trees and co-existing land-uses map.

2.2. The Study Area

The study site (41 hectares) covers the Levubu sub-tropical fruit-tree crops farming area in Vhembe District Municipality, Limpopo, South Africa (Figure 2). The study area is geographically located at latitudes 23°5′31.25″ S to 23°6′45.03″ S and longitudes 30°13′40.88″ E to 30°21′26.03″ E. The subtropical-farming area covers Tshakuma village, Mulangampuma, Levubu, Mukhuro, Valdezia and Ratondo. Levubu has a population of 207,000, in which white people account for 10.9% [56].

The area’s temperature is warm and wet from December to February, ranging from 16 to 40 °C, and the cool, dry season from May to August, ranging from 12 to 22 °C. The environmental, ecological, and biophysical environments are optimal for growing subtropical tree crops.

The Levubu area has two major rivers, namely the Luvuvhu (the main river) and the Lotonyanda River [5]. The Soutpanberg influences the annual rainfall distribution reaching 2000 mm at Entabeni, the middle part of the Soutpanberg and 340 mm in the western part peaking up during January and February. The precipitation season of the area hinders the acquisition of good quality optical data, which presents an ideal region to examine the potential for integrating SAR and optical data.

2.3. Remote Sensing Data Acquisition and Pre-Processing

The optical Sentinel-2A (S2) Level-1C and Interferometric Wide (IW) level-1 GRD (ground range detected) SAR Sentinel-1A (S1) were used in this study. The data were freely downloaded from the European Space Agency Portal, the Sentinel Scientific Hub (https://scihub.copernicus.eu/ (accessed on 16 January 2021). Both S1 and S2 images were acquired on 28 December 2019, which coincided with the data collection. The sentinel mission aims to provide data continuity to MSI from SPOT mission and continued supply of observations to be used in services such as land-cover mapping, food security, and forest monitoring, to name a few [57]. Sentinel-2 (S2) is a Multispectral Instrument (MSI) launched on 23 June 2015, with a wide- swath of 290 km providing data at 10 m to 60 m spatial resolution in 13 spectral bands ranging from visible to shortwave infrared regions with a revisit time of 5 days [58]. S2 contains three spectral bands strategically positioned in the red-edge position, crucial for improving crop condition mapping and retrieval of chlorophyll content [59].

S2 image was atmospherically corrected using a level 2A Prototype processor (SEN2COR) version 2.5.5, as stipulated on the Sentinel-2 processor for users [60]. All bands were resampled to 10 m resolution, and the corrected spectral bands were used in the analysis. The spectral configurations of the used sensors are presented in

Table 1. Spectral configurations for Sentinel-1 and sentinel-2 data.

Sentinel 2 MSI					Sentinel-1 SAR	Wavelength
Band	Name	Centre	Width	Resolution(m)	Name	Centre
2	Blue	490	65	10	VH polarization	C-band
3	Green	560	35	10	VV polarization	C-band
4	Red	665	30	10
5	Red edge	705	15	20
6	Red edge	740	15	20
7	Red edge	783	20	20
8	NIR	842	115	10
8a	Narrow NIR	865	20	20
11	SWIR	1610	30	20
12	SWIR	2190	180	20

.

The Sentinel-1A (S1) is a synthetic aperture radar (SAR) satellite, launched on 3 April 2014. The data comprises two polar-orbiting instruments performing radar imaging through the C-band at 5.405 GHz operating in four imaging channels [61]. The sensor has a wide swath of 400 km, different spectral resolutions of down to 5 m and concise temporal resolution enabling rapid data collection. Sentinel-1 is in dual-polarization mode, and its data layers include the vertical transmit and horizontal polarized received (VH) and vertical transmit and vertical polarized received (VV) operating at 1 dB (3Σ) radiometric accuracy, sensitive to vegetation dynamics and crucial for crop mapping [62].

Sentinel-1 was atmospherically corrected using Sentinel-1 Kit Module in ESA SNAP and a python script configured to read digital elevation model (DEM) and the radiative transfer lookup tables during image processing using Anaconda (python 2.7) [60]. The image corrections include geometric correction, which is an essential step in quantitative analysis. In addition, we conducted radiometric calibration and speckle filtering to remove the sensor distortions and noise. The image speckles were reduced using a 5 × 5 m adaptive Lee filter commonly used for SAR data and have been proven to produce the best filtering results, especially in areas with high heterogeneity [37]. The image was terrain corrected using a Range Doppler STRM digital elevation model (DEM) 1Sec HGT (auto download) as an input and a pixel spacing of 10.0 m × 10.0 m with a line spacing of 10.0 m.

The analysis and interpretation of the Sentinel-1 SAR GRD product were enhanced by converting the processed image into Beta sigma naught (σ⁰), and the VH and VV units were converted into decibels (dB) based on Equation (1).

X = 10 log₁₀ x

(1)

where x illustrates the pixel value.

2.4. Data Fusion

Integrating Sentinel-1 and Sentinel-2 is used to improve the textual, structural and spatial resolution for better crop type discrimination [40]. In this study, data fusion was performed using the Sentinel toolbox from SeNTinel Application Platform (SNAP) software at a pixel level using band combination. The images were first co-registered before the synergy. Sentinel-1 data were used as a master product since its geolocation accuracy was high after range doppler terrain correction.

The nearest neighbor model was used to resample and produce a fused image with 10 m pixel resolution using the Universal Transverse Mercator (UTM) at zone 36 South [63,64]. Sentinel-1 was used as master and Sentinel-2 as a slave, and the fused image was segmented using the collocation layer to aggregate the pixels with similar values.

2.5. Field Data Collection

The training data were collected on 28, and 29 December 2019, as well as on 2 and 3 January 2020. The common fruit-tree crops were identified through extensive field surveys, interviewing people and consulting Google Earth Pro. Where there was accessibility, a roadside sampling method was used to capture points [65]. A handheld Global Position System (GPS), Garmin eTrex 20X, with a 5 m positional accuracy, was used to capture the coordinates systems of the samples at the center of the farms. At each point, metadata were captured, and a note was made on mixed cropping systems to ensure that these fields were avoided during digitizing. Using a 10 m distance to account for Sentinel-2 spatial resolution. A total number of 304 (n = 304) ground-truth points were captured from the dominant land cover classes; these include: avocado (n = 49), banana (n = 53), built-up (n = 7), guava (n = 12), macadamia nut (n = 95), mango (n = 18), pine tree (n = 22), water bodies (n = 4) and woody vegetation (n = 10). These points (n = 304) were then used to guide digitizing additional points where the field is big enough using informative observation on ArcGIS and Google Earth [66].

2.6. Assessing Spatial Autocorrelation

The ground truth data were evaluated for spatial autocorrelation using which produced a Moran’s Index of 1.004 with a Z-zcore of −24.76 indicating a dispersed pattern [67]. Furthermore, the points were spatially separated using “Spatially Rarerely Occurrence Data for Species Distribution Models (SDM) (reduce spatial autocorrelation)” function available in SDMPro Toolbox in ArcGIS Pro version 2.4. A Moran’s Index of 0.09 with a Z-zcore of 0.68 was obtained, which indicates a strong random pattern among the points [68].

The retained data were partitioned into 70% (387) train data and a 30% (176) validation set using SDM Random Select Points function to prevent biasness in the classification model [69]. Furthermore, purposeful sampling was employed to digitize additional samples for the underrepresented classes (i.e., bare soil, build-up, guava). The summary statistics of the most common fruit-tree crops and other land-use types identified in the field with their average reflectance, field photos, and the reference data are presented in Figure 3.

3. Image Classification

3.1. Machine learning Classification Algorithms

Two supervised machine learning algorithms, random forest (RF) and support vector machines (SVM), were evaluated in classifying the tree crop types using four dataset’s configurations at a pixel-based level. The models were tuned and were trained using the “caret package” in R studio, statistical software and programming language version 4.0 [70,71].

3.1.1. The Random Forest Algorithm (RF)

Random Forest (RF) is an ensemble machine learning algorithm that uses bagging to improve predictions by reducing the variance introduced by Breiman [72]. In this study, the RF model was optimized using two primary parameters, namely mtry and ntree, the former denotes the number of predictors tested at each tree node, and the latter illustrates the number of decision trees ran at each iteration, respectively [73]. The classification accuracy was improved by optimizing the number of grown trees (ntree) and variables at each tree split (mtry) based on the OOB error rate using a 10-fold cross-validation method and a grid search [74]. The optimal trees were searched at 500 intervals, using ntree parameters ranging from 500 to 10,000 [73]. On the other hand, the optimal mtry was searched using the square root of the total number of the explanatory variables as it has been proven to provide satisfactory results [48,75].

The RF model measures the importance of each explanatory variable applied in the classification using the OOB samples (unbiased estimates) and permutes the variables to calculate the mean decrease accuracy (MDA) and mean decrease Gini (MDG) [76].

The RF also builds trees using randomly replaced bootstrapped samples comprising two-thirds of the training dataset, decreasing the variance in classification errors [77,78]. The one-third of the training data excluded during the bootstrapped sample is referred to here as the out-of-bag (OOB) error rate. The OOB is used to internally evaluate the performance of the RF classification during the cross-validation process [79]. The model with the lowest OOB error is considered the best data fit for the RF model [4]. The equation used to define the OOB error in a classification framework is as follows [77].

errOOB = {1/n Card {i ∈ {1, …, n} | y_i ≠yˆ_i}

(2)

where yˆ_i illustrates the predicted values aggregated by the number of trees t where (x_i, y_i) belongs to the OOB sample associated with it.

The RF algorithm has been commonly used to create crop types maps, as it has been shown to produce better accuracies [52]. The classifier is robust and suitable for heterogeneous landscapes and can handle the problem of noise and correlations that are intrinsically inherited in remote sensing data [77]. The crop types were classified using datasets generated from Sentinel-1 and Sentinel-2.

Assessing the Importance of Variables

The application of noisy bands has been reported to significantly impact classification conducted at the pixel level [80]. Selecting a set of optimal variables is essential to properly characterize objects and improve classification accuracy [81]. In this study, the RF algorithm’s out-of-bag (OOB) strategy was used to measure and rank the importance of variables from the training data using myImagePlorR in R studio from the data fusion model [72]. The RF model quantifies the variable importance (VI) using a mean decrease in accuracy (MDA) index, which ranks the ranks the importance of independent variables in predicting the dependent variable using a random permutation-based score [72,82]. The variables with lower MDA values indicates less association with the dependent variable, while the variables with higher MDA values are considered the most significant independent variables in classification [78,83,84,85].

The variable importance (VI) produced by each model differed significantly between the four dataset configurations. The spikes show the significance of each band per class in the band importance graphs. Standard bands for different datasets were identified for mapping the same crop types, but band importance inconsistencies amongst classes were observed [86].The selected optimal bands were used to form the model used to create the final crop maps.

Forward Variable Selection

A forward variable selection (FVS) is used to select relevant features that will yield lower misclassification error rates and use them to build multiple RF models [87]. FVS is based on a RF mean decrease accuracy (MDA) metric that ranks the variable importance of the remote sensing dataset [72,88,89]. Statistically, the objective of variable selection is to assist diagnostics, interpretation, as well as shorten the data processing time [77]. The data fusion comprised 12 spectral bands; hence, the FVS method was used to select a set of optimal variables to improve the overall accuracy. Six variables were selected and applied to the fused model using the training data to eliminate model overfitting, and used thereafter to classify crops using Random Forest (RF) [90,91].

3.1.2. The Support Vector Machines Algorithm (SVM)

The support vector machine (SVM) is a non-parametric supervised learning algorithm based on spectrally weighted kernel functions [92,93]. The method uses hyperplane to separate classes, which minimizes misclassifications [81].

The SVM consists of four types of kernels different types of kernels. However, only four kernels are commonly used to classify remote sensing data, which are linear, radial basis function (RBF), sigmoid, and polynomial kernels [94]. In this study, the SVM classifier was trained using the radial basis function (RBF) kernel in the Equation below:

$$\mathit{K}\mathbf{(}\mathit{u}\mathbf{,}\mathit{v})=\mathbf{e}\mathbf{x}{\mathbf{p}}^{-\gamma \times |u-v{|}^2}$$

(3)

where γ denotes the kernel parameter in the RBF kernel.

In the SVM model, two parameters are used to tune the RBF kernel and select the best parameters, which are “C” cost and “gamma” (γ) [61]. The gamma and cost were optimized using 10-fold cross-validation to select the optimal C and γ parameters. The C parameter allows a trade-off between model complexity and training error. The optimal hyperplanes with sensitive spectral information are extracted and used to increase discrimination and classification accuracy [95]. Using a small C value increases the training errors while the large C value overfit the model [96]. The gamma (γ) is defined as a free parameter of the RBF; when the γ value is large, it indicates a small variance, which leads to a high-bias model [96].

The SVM has proven robust on imbalanced samples and is computationally fast [97,98]. The RBF is commonly used in remote sensing existing literature and has been proven to produce high overall accuracy compared to the other kernels, which depend on image resolution [76]. The RF based selected variables were again applied on SVM S1S2 variable selection model and used to classify crops.

3.2. Accuracy Assessment and Model Validation

Accuracy assessment was used to check how well the locations of the mapped crops correspond to the actual earth surface locations [99]. The method tests whether the sampling design and data analysis methods best fit the data [99]. Further, it validates the classified map based on location-specific data using the 30% test dataset. Previous studies have cited the confusion matrix as a widely used method for assessing the accuracy of thematic maps classified using the reference data [100]. The confusion matrixes were constructed, and the OOB error was used to identify the actual and misclassified class [87]. Further, different confusion matrix metrics such as the overall accuracy (OA), user accuracy (UA), producer accuracy (PA) and kappa coefficient (kappa) were computed to aid the explanations through Equations (4) and (5) [99,101].

$$\left.\begin{array}{l}OA=\frac{TP+TN}{TP+TN+FN+FP}\\ Precision=\frac{TP}{TP+FP}\\ Recall=\frac{TP}{TP+FN}\end{array}\right\}$$

(4)

$$K-index=1-\frac{1-OA}{1-Pe}$$

(5)

where TP, TN, are the samples predicted as true positive and true negatives while FP, FN is samples predicted as a false positive and false negative. At the same time, P_e indicates the probability of chance agreement.

4. Results

4.1. The Variable Importance

The variables selected as the most optimal for the data fusion model are shown in Figure 4a. In order of merit, these are the green (B3:560 nm), red (B4:665), rededge_1 (B5:705 nm), SWIR_1 (B11:1610 nm), SWIR_2 (B12:2190 nm) and the VH polarization. The significance of the SAR and optical bands in classifying classes was evaluated using the RF’s variable importance and mean decrease accuracy (MDA) consisting of spikes showing the most optimal bands as presented in Figure 4b. The contribution of each band is highly noticeable when data dimensionality is reduced.

The S2 Green band (B3) located in the visible region played a major role in differentiating between built-up areas, pine tree, guava and mango. The red band (B4) located at 650–680 wavelength played a significant role in distinguishing woody vegetation, guava and avocado, followed by the green band, which detected the pine trees, built-up areas, guava and mango, better. The SWIR_2 (B12:2190) was imperative in discriminating woody vegetation, pine tree, bare soil, and water bodies. Similarly, the built-up areas, pine trees and bare soil were better identified using the RedEdge_1 (B5:665). The SWIR_1 (B11:1610) played a significant role in distinguishing water bodies, pine trees, guava, avocado, and woody vegetation. For S1, the results highlight a vast contribution of VH polarization to discriminating the banana class.

4.2. The Variable Selection Using the Forward Feature Selection (FFS) Method

All fused Sentinel-1 and Sentinel-2 bands (n = 12) produced an OOB error of 14.29%. A forward feature selection (FFS) was used to select a set of optimal features combination that could lead to high mapping accuracy while mitigating high dimension and model over-fitting (Meyer et al., 2018). The FFS was carried out based on mean decrease accuracy (MDA) calculated during the training of the RF model. The MDA measure filters the explanatory variables based on their prediction strength (Kuhn and Johnson, 2013). The results indicate that the lowest OOB error is achievable with six variables, as the error rate increases from seven variables upwards (as shown in Figure 5).

4.3. Mapping Outputs

The patterns of RF and SVM in mapping tree crop species and other existing land use types based on Sentinel-1 SAR data. Figure 6A,B) shows that the woody vegetation is the dominant land use in the radar scenes. The other land use types were not detected accurately, making it difficult to interpret the spatial distribution patterns.

The optical data classification shows a heterogeneous area of bananas mixed with guava. Most of the classes are incorrectly classified by optical scene compared to the radar scenes. The RF map presented in Figure 6C,D display the spatial distribution of classes showing misclassifications between avocado, bare soil and built-up areas. As a result, the bananas were misclassified as guavas and mangoes. The highest spectral confusions were reported between guava, banana, avocado, built-up regions, mango, and macadamia nut trees. Conversely, in Figure 6D and

Table 2. Confusion matrices for the models that produced the models that produced the highest overall accuracy using RF and SVM algorithms. The mapped classes are avocado (AV), banana (BN), bare soil (BS), guava (GV), macadamia nut (MN), mango (MG), pine tree (PT), built-up (BU), water body (WB), and woody vegetation (WV).

	(A) RF-S1
Class	AV	BN	BS	GV	MN	MG	PT	ST	WB	WV	Total
AV	4	2	1	2	2	2	1	2	0	2	18
BN	2	5	3	2	2	3	0	1	0	1	19
BS	1	1	4	2	1	1	2	1	1	1	15
GV	2	1	3	4	1	2	1	0	0	0	14
MN	4	8	3	4	4	2	2	2	1	1	31
MG	2	1	0	2	2	4	1	1	0	0	13
PT	1	0	0	2	1	2	8	1	0	0	15
BU	2	2	1	0	0	2	1	4	0	1	13
WB	1	0	0	0	0	0	0	2	12	0	15
WV	2	2	1	0	0	1	1	1	2	13	23
Total	21	22	16	18	13	19	17	15	16	19	176
OA: 29.84%
kappa: 0.23%
	(B) SVM = S2
Class	AV	BN	BS	GV	MN	MG	PT	ST	WB	WV	Total
AV	16	0	0	0	1	1	0	0	0	0	18
BN	0	16	0	0	1	1	0	0	0	1	19
BS	0	0	14	1	0	0	0	0	0	0	15
GV	1	1	0	11	0	1	0	0	0	0	14
MN	0	1	1	0	28	1	0	0	0	0	31
MG	0	0	0	1	1	11	0	0	0	0	13
PT	0	1	0	0	1	0	13	0	0	0	15
BU	2	0	1	1	0	0	0	10	0	0	13
WB	1	0	0	0	1	0	0	0	13	0	15
WV	0	1	0	0	0	0	0	0	0	22	23
Total	20	20	16	14	33	15	13	10	13	23	176
OA: 87%
kappa: 86.89%
	(C) RF-S2 SELECTED
Class	AV	BN	BS	GV	MN	MG	PT	ST	WB	WV	Total
AV	15	0	0	1	1	1	0	0	0	0	18
BN	0	16	0	0	1	1	0	0	0	1	19
BS	0	0	12	2	1	1	0	0	0	0	15
GV	0	1	0	7	5	1	0	0	0	0	14
MN	0	2	2	4	18	5	0	0	0	0	31
MG	0	1	0	0	4	7	1	0	0	0	13
PT	0	1	0	0	2	1	9	1	0	0	15
BU	2	0	0	0	0	1	1	9	0	0	13
WB	0	0	0	0	0	0	0	3	12	0	15
WV	0	1	0	1	0	4	0	0	0	17	23
Total	17	22	14	15	32	22	11	13	12	18	176
OA = 69.33%
Kappa = 69%
	(D) SVM-S1S2
Class	AV	BN	BS	GV	MN	MG	PT	ST	WB	WV	Total
AV	16	0	0	0	1	1	0	0	0	0	18
BN	0	17	0	0	0	1	0	0	0	1	19
BS	0	0	13	0	1	1	0	0	0	0	15
GV	1	1	0	11	0	1	0	0	0	0	14
MN	0	0	1	1	28	1	0	0	0	0	31
MG	0	0	0	2	1	10	0	0	0	0	13
PT	0	1	0	0	1	1	12	0	0	0	15
BU	1	0	0	0	0	0	0	12	0	0	13
WB	1	0	0	0	1	0	0	0	13	0	15
WV	0	1	0	0	0	0	0	0	0	22	23
Total	19	20	14	14	33	16	12	12	13	23	176
OA = 87.01%
Kappa = 0.87
	(E) SVM-S1S2 SELECTED
Class	AV	BN	BS	GV	MN	MG	PT	ST	WB	WV	Total
AV	17	0	0	0	0	1	0	0	0	0	18
BN	0	17	0	0	0	1	0	0	0	1	19
BS	0	0	13	0	1	1	0	0	0	0	15
GV	1	1	0	11	0	1	0	0	0	0	14
MN	0	0	0	1	29	1	0	0	0	0	31
MG	0	0	0	0	1	12	0	0	0	0	13
PT	0	1	0	0	1	0	13	0	0	0	15
BU	1	0	0	0	0	0	0	12	0	0	13
WB	0	0	0	0	0	0	0	0	15	0	15
WV	0	1	0	0	0	0	0	0	0	22	23
Total	19	20	13	12	32	17	13	12	15	23	176
OA = 91.63%
Kappa = 0.91%

B, the SVM map displayed less spectral confusion between mangoes and banana, macadamia nuts and mangoes.

As depicted in Figure 7E,F, Table 1 and Table 2C, the RF maps shows the spectral confusion between avocado and mangoes. The avocado crop overlaps with banana, mango and macadamia nut, as shown in Table 1. Some mangoes were classified as macadamia nut, while some macadamia nut trees were classified as guavas. The SVM classifier recorded misclassifications between these classes (Figure 7F).

The addition of radar data to the optical data reduced the image speckles. The SVM map displayed spectral confusion for most of the classes except for avocado, mangoes, and bananas (Table 2D). The RF map shows less spectral, and a shift of spectral confusion towards the avocado and the guava, the mangoes and the banana class were noticed, Figure 6G and Figure 7B. As depicted in Table 1, the mango crop overlaps with avocado and macadamia classes. Hence, most of the areas of avocadoes were classified as mangoes, while a few macadamia nut trees were misclassified as guava in Figure 6H.

Accurate class separation was achieved based on selected six optical variables using 1 SAR and 1 Optical scene. There is a reduction in class confusion in a macadamia nut and avocado areas (Table 2), while a noticeable amount of mango pixels was still misclassified (Figure 6G), as well as some avocados being misclassified as guavas. The best classification model was based on the SVM algorithm (Figure 6H). The areal coverage from the map with the highest classification accuracy shows the woody vegetation as the largest class covering 36.3% of the south, northeast and past southeast of the study area. The Guava crop is the second largest, covering 16.80% and dominates in the eastern part of the study area compared to a water body and built-up, covering 0.8% and 2.6%, respectively.

4.4. Classification Accuracy

In this study, the classification accuracy was assessed using testing data. Five different models comprising different variables, namely S1 (n = 2) (VV, VH); S2 (n = 10) (B3: Blue, B3: green, B4: red, B5: red edge_1, B6: red edge_2, B7: red edge_3, B8: NIR, B8a: Narrow NIR, B11: SWIR_1, B12: SWIR_2); S2 selected variables (n = 6)(B2: blue, B3:green, B4: red, B5: red edge_1, B11:SWIR_1S1S2, SWIR_2:B12); S1S2 (n = 12) (B2: blue, B3: green, B4: red, B5: red edge_1, B6: red edge_2, B7: red edge_3, B8: NIR, B8a: Narrow NIR, B11: SWIR_1, B12: SWIR_2, VV, VH), and S1S2 selected variables (n = 06) (B3: green, B4: red, B5: red edge_1, B11: SWIR_1, B12: SWIR_2, VH), were used and classified using RF and SVM. The S1S2 selected variable model using the SVM classifier obtained the highest classification accuracy compared to the RF classifier for the other models (S1, S2, S2 selected variables and S1S2), as shown in Figure 8. The S1 produced an overall classification accuracy of 28.04% and 27.64%, using the RF and SVM model. The S2 model had an overall classification accuracy of 81.96% and 87% for RF and SVM, while the S2 selected variables produced an overall accuracy of 70,04% and 69.33% using the RF and SVM, respectively. A steady accuracy was observed from the data fusion approach, wherein an overall classification of 81.93% and 87.01% was achieved. The combination of selected optimal variables produced an increased accuracy of 86.50 and 91.6% for RF and SVM, respectively, amounting to a 4% improvement over S2 data. A bar plot showing the highest classification accuracies from both models is presented in Figure 5.

4.5. Class Accuracies

The classification accuracies of the four datasets at a class-wise level were evaluated using the test dataset. The user’s accuracy (UA) and producer’s accuracy (PA) in

Table 3. The performance of RF and SVM over four datasets (S1, S2, S2 selected variables, S1S2, S1S2 selected variables importance (VI) in terms of user’s and producer’s accuracies in percentages (%). The mapped classes are avocado (AV), banana (BN), bare soil (BS), guava (GV), macadamia nut (MN), mango (MG), pine tree (PT), built-up (BU), water body (WB), and woody vegetation (WV).

	(a) RF User’s Accuracy (%)						(b) RF Producer’s accuracy (%)
Class	S1	S2	S2 VI	S1S2	S1S2 VI	S1	S2	S2 VI	S1S2	S1S2 VI
AV	18.57	87.32	44.16	71.08	93.75	19.12	77.50	49.28	73.75	75.00
BN	40.00	75.71	62.5	78.75	88.06	28.57	76.81	67.90	91.30	85.51
BS	18.18	84.48	82.35	89.09	90.74	17.39	89.09	91.30	89.09	89.06
GV	19.05	80.00	60.87	86.59	79.79	24.00	81.93	66.67	84.52	90.36
MN	15.28	75.58	50.00	74.67	85.91	16.92	79.27	45.83	68.29	86.84
MG	23.73	74.19	80.00	65.96	66.67	29.17	47.92	61.54	64.58	75.56
PT	27.69	92.06	50.00	96.72	93.75	35.29	95.08	84.31	96.72	98.36
BU	34.88	79.22	82.14	93.33	90.63	27.78	89.71	82.14	82.35	86.57
WB	93.88	98.11	96.00	91.53	96.30	86.79	95.63	96.00	100.00	96.30
WV	100.00	99.06	100.00	100.00	100.00	17.65	93.15	100.00	87.67	94.52
		(a) SVM User’s accuracy (%)					(b) SVM Producer’s accuracy (%)
Class	S1	S2	S2 VI	S1S2	S1S2 VI	S1	S2	S2 VI	S1S2	S1S2 VI
AV	17.74	38.71	44.87	83.75	94.67	32.35	39.71	50.72	83.75	94.67
BN	38.96	67.65	64.71	85.14	88.73	38.96	59.74	67.90	91.30	88.33
BS	23.08	84.91	87.76	94.64	94.55	13.04	96.53	93.48	93.36	90.00
GV	23.89	50.67	84.31	93.6	87.78	50.00	76.00	76.79	91.67	96.3
MN	14.58	52.63	55.10	72.94	94.59	11.48	45.45	64.29	75.61	95.18
MG	0.00	57.58	52.00	80.00	85.11	0.00	39.58	54.17	66.67	92.11
PT	20.83	85.11	74.07	95.08	93.00	29.41	78.43	76.79	95.08	88.89
BU	29.09	74.24	91.11	93.85	95.00	15.69	90.74	80.39	89.71	98.36
WB	34.61	100.00	94.12	94.74	97.36	34.91	97.26	88.89	94.52	85.07
WV	0.00	100.00	100.00	100.00	100.00	0.00	79.33	74.07	100.00	98.63

show the overall accuracies obtained by each model using RF and SVM. The producer’s and user’s accuracies give crucial insight into how each model contributed to detecting classes.

The produced UA’s and PA’s accuracies ranged from 17.74 to 100%. The sentinel-1 SAR data discriminated the water bodies and woody vegetation with an overall accuracy of 80% and above, while the other crops were below 40% for both classifiers. Optical data (S2) increased the UA’s and PA’s accuracies to above 70% for all classes using the RF. Conversely, the UA’s and PA’s accuracy scores were below 60% for avocado, guava, pine tree and mango using the SVM classifier. However, a decrease in UA’s and PA’s scores was recorded for the mango and pine tree classes. The model S2 selected variables recorded an increase of 5.81% in the UA for the mango class. The UA values for the built-up and woody vegetation classes increased by 2.98% and 0.40%. The PA values of the bare soil, macadamia nut, mango and water body increased by 2.21%, 12.41%,1 7.85% and 4%, respectively.

Using the selected optical variables from data fusion, the mango class recorded a 20–30% accuracy increase compared to the other models. Though the SVM proved superior to the RF model, the individual class accuracies for avocado, guava, mango, and macadamia nut are comparable. The RF improved the producer and user accuracy margins of areas identified as guava, pine tree, built-up, water bodies and woody vegetation by over 80%, including the mango class, which was challenging to detect in other models.

5. Discussion

The study evaluated the utility of using RF and SVM models using Sentinel-1 (SAR) and Sentinel-2 Optical, their synergy to map fruit-tree crop species and co-existing land-use types in a heterogeneous smallholder landscape. The study further tested the utility of feature selection by investigating the relevance of the Sentine-1 and Sentinel-2 bands by calculating the variable importance using a RF algorithm to eliminate data dimensionality and increase classification accuracy [89]. The free availability of SAR and multispectral data ESA Copernicus with suitable spectral, temporal and spatial resolution similar to SPOT-5 provides promising opportunities for mapping smallholder agriculture at the landscape level [102].

The results showed that Sentinel-1 data could not be used alone, as it could not differentiate between the studied fruit-tree crops and other existing land uses due to a limited number of channels [103]. As expected, the classification based on the SAR image produced an overall accuracy of 28.04% using RF and 27.64% using the SVM, respectively. Similar to findings by s, the VH polarization was optimal in distinguishing the fruit-tree crops due to its sensitivity to the structure and geometrical features of crops. The application of Sentinel-1 data alone resulted in false cropping patterns using both classifiers (RF and SVM). The findings concur with previous studies that mapping tree crops using single date images produced unsatisfactory results due to similarities in physiological and morphological characteristics that commonly exist in fruit-tree species [104,105]. It is even more challenging to obtain precise information when mapping crops at the landscape level [81]. The reason why Sentinel-1 SAR data report confused cropping patterns on all land cover classes, except the banana crop, is yet to be studied.

The Sentinel-2 data performed well in discriminating crops since they consist of spectral information, which is crucial in crop discrimination [106]. In contrast, the NIR has shown to be less critical to all models in discriminating tree crops, as [107] reported. The NIR band was noisy and less beneficial in crop discrimination [80]. The Individual class accuracy results showed some difficulties Sentinel optical data in discriminating banana, guava, and mango within built-up areas due to spectral overlapping, suggesting a high level of crop dependence on R.S. spatial resolution to be classified accurately. Hence, the success of crop mapping relies on the careful selection of remote sensing data with appropriate spectral and spatial characteristics to improve classification accuracy [104].

The use of 12 spectral bands yielded less accuracy than the selected six optimal bands due to noise contamination [48]. Similar to the existing literature on mapping fruit-tree parameters, the highest classification accuracy of 91.60% was achieved from the combination of selected variables from SAR and optical data, much better than results derived from high-resolution SAR and optical images [34,106,108]. Through the use of the SVM model, the classification accuracy improved by 4% showcasing the importance of using SAR-optical fusion in areas where landscape heterogeneity and climatic conditions challenge remote sensing data acquisition [105]. Data fusion is a promising approach in remote sensing that provides smallholder farmers with spatially explicit information while offering practical opportunities to achieve precision agriculture at the landscape level.

As reported in previous studies [109,110], including Sentinel-1 backscatter, improved crop separability and the overall classification accuracy due to its sensitivity to biomass and tree canopies. The class accuracy was improved for crops and other land-use types when radar data were integrated with optical data. The effect of variable selection for enhancing the class accuracy was high with regard to mango crops, where a 20–30% increase was recorded, which agrees with findings by [111]. The RF improved the producer and user accuracy margins of areas identified as guava, pine tree, built-up, water bodies and woody vegetation by over 80%, including the mango class, which was challenging to detect in other models. Heterogeneous classes such as avocado, guava, mango, and macadamia nut were distinguished correctly from the other crops using the variable selection. Although the SVM proved superior to the RF model in terms of overall accuracy, the implemented RF model using the selected variables was robust and informative.

The class-specific variable selection showed that the green band (B3:560 nm) was the most influential spectral band in discriminating pine trees, mango, and guava. The Sentinel-1 VH band was effective in detecting the banana crop, which corresponds with previous studies [112]. Sentinel-2 green band (B3: 560 nm) was crucial for mapping pine trees, mango, and guava. The red band was essential for detecting avocado and woody vegetation, while the red edge_1 (B5: 705 nm) distinguished avocado and bare soil. SWIR bands were critical in discriminating built-up areas, banana, mango, guava, pine tree, water bodies and woody vegetation. The model detected all crops well, including the small patches of mango and bananas hidden within built-up areas. Similar to [109], the data fusion model was a better discriminator for mango patches within built-up regions that could not be detected using S1 and S2 alone. The visible bands (green (B3:560 nm) and red (B4: 665 nm), SWIR_2 (B12: 2.190 nm), SWIR_1 (B11: 1.610 nm) and the rededge_1 (B5: 705 nm) were the most sensitive to chlorophyll content and leaf structure and contributed highly in distinguishing crops [113]. Using selected pivotal bands from the data fusion produced less spectral confusion and was suitable for mapping fruit-tree crops in a heterogeneous environment. The criticality of the SWIR bands in fruit-tree crop mapping was highlighted [40]. The maps depicting tree crops and co-existing land-use types show that the integrated approach discriminates subtle areas, which was not the case in optical and SAR models alone [114]. The selection of optimal variables proved to be a well-suited mapping approach for the study area, which agrees with the findings of [111,115].

The comparison of Sentinel-2 and data fusion showed good agreement in this study. As [61] reported, including the SAR data, particularly the VH polarization, increased the detection of smaller crop aggregations while the optical data overestimated the crop distribution in the study area. The Sentinel-2 method missed some small patches of mango and guava crops (Figure 6C,D). The data fusion model accurately captured the cropping dynamics in heterogeneous smallholder agriculture. The visual assessment of the final maps generated using the selected optimal variables showed enhanced interpretability [40] on SVM products while some noise in homogenous maps produced by the RF algorithm, similar to [73] findings. The data distinguished crops better due to the swath width and high spatial resolution, which account for crop patterns in small field sizes for heterogeneous areas. The noise depicted on maps produced using each dataset separately was reduced by the fused model. The radar data are sensitive to vegetation structure and texture, and it contributed much to discriminating banana, mango, and guava found within build-up areas, which was difficult to determine using single date images.

The contribution of SAR and optical data in crop mapping of smallholder agriculture in an African setting was ascertained. Further, the potential of S1S2 spectral discriminating power was analyzed. Although the results were not significantly increased, the model stability on fused data increased, demonstrating data fusion’s potential to enhance smallholder crop mapping. The combination of backscatter from Sentinel-1 (SAR) at C-band and spectral features from Sentinel-2 (optical) have proven to be a better approach in discovering complex distribution patterns of crops, including classes that are difficult to map using a single image. Integrating S1S2 provided spectral and textural information to detect different tree crops and co-existing land-use types better in a heterogeneous landscape [108]. To ascertain this, we further investigated the effect of variable selection on S2 data. The model produced an overall accuracy of 70.04% and 69.33% using RF and SVM, respectively. Although the levels of class accuracies were low on the S1S2 model compared to the S2 model, the results of this study proved that incorporating the SAR data enhanced the discrimination of the crop types. A clear trade-off was observed between S2 and data fusion, where the highest classification results were obtained using data fusion information. Due to the Sentinel-2 low geometric resolution, the crops from small farm parcels that belong to the same family were not accurately classified [104].

The approach provides opportunities to map sub-tropical smallholder agriculture with mixed cropping systems for field management, precision agriculture, decision-making by agronomists and government. However, the method was time-consuming and required advanced remote sensing and GIS skills and a high processing computer to accomplish the task. The study was limited to a single date image from each Sentinel platform to simplify the mapping process. The launch of Sentinel-1B with five days’ temporal resolution allows the application of SAR data in mapping heterogeneous landscapes. Although Sentinel-1 failed to detect crop types, it is still helpful to integrate its data with optical data as it offers opportunities to enhance the distribution patterns in heterogeneous landscapes.

Therefore, the study recommends testing this model on multi-time series data to improve crop detection using crop temporal behaviors from different phenological stages. Due to various cropping calendars, it is advisable to use multi-time series images to assess different vegetative stages and identify the optimum window relevant for accurate crop mapping for precision agriculture (PA) [80]. Further, the study recommends selecting a few optimal polarimetric images to reduce high data dimensionality and increase classification accuracy. Discriminating all crops with high accuracy, including minor crops, is essential. Hence, the study suggests an image fusing approach based on time-series data to identify temporal windows crucial for accurate crop mapping.

6. Conclusions

Overall, the use of single-source data (S1) proved unsuitable for mapping crops due to the lack of spectral characteristics. Although the optical data (S2) performed better than S1, it could not discriminate crops with spectral similarities (i.e., mango, macadamia, pine tree). Combining SAR (S1) and optical (S2) data into machine learning classifiers can be used to map fruit-tree crops in a sub-tropical heterogeneous landscape experiencing frequent cloud coverage. The results show that the SVM classifier provides the best classification results for fruit-tree crops and co-existing land use types by utilizing six key variables, of which five were S2 bands (i.e., green (B3), red (B5), rededge_1 (B5), SWIR_1 (B11), SWIR_2 (B12)) and one S1 channel (the VH polarization). The multi-source approach provides timely information for evidence-based decision making.