Detection of Planting Systems in Olive Groves Based on Open-Source, High-Resolution Images and Convolutional Neural Networks

Abstract

This paper aims to evaluate whether an automatic analysis with deep learning convolutional neural networks techniques offer the ability to efficiently identify olive groves with different intensification patterns by using very high-resolution aerial orthophotographs. First, a sub-image crop classification was carried out. To standardize the size and increase the number of samples of the data training (DT), the crop images were divided into mini-crops (sub-images) using segmentation techniques, which used a different threshold and stride size to consider the mini-crop as suitable for the analysis. The four scenarios evaluated discriminated the sub-images efficiently (accuracies higher than 0.8), obtaining the largest sub-images (H = 120, W = 120) for the highest average accuracy (0.957). The super-intensive olive plantings were the easiest to classify for most of the sub-image sizes. Nevertheless, although traditional olive groves were discriminated accurately, too, the most difficult task was to distinguish between the intensive plantings and the traditional ones. A second phase of the proposed system was to predict the crop at farm-level based on the most frequent class detected in the sub-images of each crop. The results obtained at farm level were slightly lower than at the sub-images level, reaching the highest accuracy (0.826) with an intermediate size image (H = 80, W = 80). Thus, the convolutional neural networks proposed made it possible to automate the classification and discriminate accurately among traditional, intensive, and super-intensive planting systems.

1. Introduction

Olea europaea L. (olive tree) can be considered one of the most important crops in the Mediterranean Basin, providing both edible fruit and storable oil. Worldwide, the Mediterranean Basin represents 93% of the olive tree area harvested [1] and 99% of the production [2], of which Spain is the largest olive producer in the world with 30.4% of the production, followed by Italy and Morocco with the 11.2% and 9.8%, respectively [3].

Over time, traditional olive groves have been an important economic, environmental, and social component of the Mediterranean landscape. Nevertheless, the current economic market has meant that many of the traditional olive groves are not considered sustainable, and more intensive systems are emerging in the valleys, such as drip irrigation and the intensification of patterns [4]. The management of this crop is changing from traditional low-density rainfed olive groves to medium- or high-density groves, mostly associated with irrigation, that are promoting the substitution of extensive crops such as wheat, barley, sunflowers, or cotton in some regions. This strong modification of olive growing systems can be clearly observed in Andalusia, one of the main olive-growing regions in the world with 46.7% of the olive-growing area in Spain [5]. From 2015 to 2019, high- and super high-density olive groves (more than 400 olive trees ha⁻¹) steadily increased in area by 48.5%, from 54,140 ha in 2015 to 80,386 ha in 2019 [5,6].

Since olive groves produce an important range of ecosystem services such as healthy food, carbon sequestration, biodiversity, employment in rural areas, etc., the changes in the management of the olive groves are causing variations at the economic, environmental, and social levels. Government institutions are greatly concerned about the new challenges and their future impacts, and they promulgate legislation on different scales to enhance the sustainability of farming systems. For example, the European Union (EU) has shown great interest in the conservation of the traditional olive groves and their benefits through strategies such as the European Landscape Convention [7]. In the same direction, the European Commission and the Member States just adopted a new common agricultural policy (CAP), 2023-27, which seeks to ensure a sustainable future for European farmers, highlighting the importance of the environment and the conservation of the biodiversity of agricultural landscapes [8]. Of course, when necessary, government institutions also encourage policies that can promote more productive systems through farming intensification techniques, such as high tree densities or irrigation. In this sense, and although higher olive tree densities can show important improvements [9], such as an increase in carbon sequestration, the new intensive managements also show an important disadvantage since they would increase irrigation needs, which is a serious problem due to the low availability of water in arid and semi-arid areas [10]. In this sense, studies such as [11,12] stress the need for more data at the farm level to increase sustainable practices. Therefore, government institutions need techniques that allow them to accurately control the spatial distribution of crops and their management.

Since traditional methods based on sample and ground visits to a small percentage of fields are considered imprecise, expensive, and time-consuming, it is necessary to develop new techniques that increase the controlled area and reduce costs. Imagen analysis with remote sensing (RS) techniques can significantly improve the deficiencies of ground visits by obtaining accurate maps of large areas. A significant number of RS sensors are utilized, alone or in combination, to estimate and map different tree parameters in agroforestry environments to assist in making appropriate management decisions in a non-destructive manner [13]: RGB (Red-Green-Blue) [14,15], multispectral [16,17,18,19], hyperspectral [20,21,22,23], thermal [24,25,26,27], or even LiDAR (Light Detection and Ranging) and RADAR (Radio Detection and Ranging) sensors [28,29,30,31]. Specifically, in olive grove studies, different combinations of RS platforms and technologies have been widely used. Nevertheless, the main limitation of tree-level studies is the need to analyze images with very high spatial resolution.

Although images obtained from satellites have the advantage of covering wide areas quickly, delineation studies of olive tree crowns with these images alone are scarcer given the lower availability of satellite images with very high spatial resolution [32,33,34]. In some cases, it is necessary to implement complementary techniques, such as pan-sharpening fusion techniques, to allow obtaining images with higher spatial resolution than the originals [35,36]. On the other hand, images obtained from cameras on board Unmanned Aerial Vehicles (UAV) or manned flights can provide high-resolution data (cm resolution); this type of data is used more frequently in the analysis of olive tree canopies than satellite images [37,38,39]. Nevertheless, although this very high spatial resolution imagery overcomes the spatial limitation, this technology often requires high costs due to the equipment and field campaigns [40]. Moreover, since UAVs are mainly battery-powered, they have limited flight time and are not suitable for large-scale surveys [41]. The use of open data imagery with appropriate spatial resolution acquired from freely accessible platforms can provide a suitable alternative to reduce costs. Some countries have programs for acquiring and updating digital orthophotographs for cartography and general geographic knowledge of their territory, such as the digital aerial orthophotography of the National Plan for Aerial Orthophotography (PNOA) [42].

With the continuous increase of RS data (images and derived information), traditional classification methods based on spectral distance-angles or probabilities are not the most appropriate since they do not take advantage of all the information efficiently [43]. The rapid development of new technologies such as machine learning (ML) or deep learning (DL) techniques in the field of RS are showing more accurate classifications and target detections [44,45,46]. To facilitate the automation of classification processes, deep learning (DL) can be a good approach. DL techniques are artificial neural networks in which multiple layers of processing are able to automatically learn informative representations of input data and extract progressively higher-level features. Due to its great potential, the use of a DL technique in image classification presents a new challenge. RS images are more complex than the scene images used in conventional DL developments. For example, the high spatial resolution of RS images may involve various types of objects in the same scene, and the high spectral resolution (especially with hyperspectral images, which contain hundreds of bands) can involve a large amount of data. This characteristic may demand a large number of neurons in a DL network [47]. Although it is increasingly common to see studies that use DL techniques in RS, they are still scarce, and knowing how to utilize this information still requires further research.

Despite the importance of new methodologies to automatize the detection of planting systems, no studies directly related to this necessity were found. The studies that were found determine the density of plants [48,49,50,51], but they do not directly predict the plantation system of the study area. To achieve this, the analysis must take into account both the density of the plants and the spatial relationship among them (position and distance of each plant with respect to the others), or even the size of the plant canopy. In this regard, the main contribution of this study is to evaluate the potential of DL techniques to accurately distinguish among traditional, intensive, and super-intensive olive groves. For this purpose, this paper seeks to develop and validate a novel DL methodology based on sub-image (mini-crops in this paper) classification of the fraction canopy cover (FCC) of olive groves using convolutional neural networks. As a result, the proposed system can automatically discriminate between olive grove planting systems at farm level based on open data sources. Furthermore, the effect of mini-crop size variations to optimize the time-cost/accuracy ratio has been evaluated. By sequentially analyzing every sub-image from the orthophotography, the system results in a useful tool for not only planting system discrimination, but also as a segmentation analysis method for studying the distribution of the olives trees across a crop.

The article outline is as follows. In Section 2, the materials and procedures used to generate the dataset of high-resolution images of olives groves are described, and the convolutional neural network architecture used for the target classification problem in this article was performed. Section 3 contains the results that were obtained and their discussion. Finally, Section 4 comprises the conclusions that can be drawn from this work.

2. Materials and Methods

2.1. Study Area and Image Acquisition

The analysis was conducted on 1187 olive groves distributed throughout Andalusia, a representative agricultural region of the typical continental Mediterranean climate (Figure 1). This climate is characterized by short mild winters and long, dry, and hot summers. The olive plots were sampled in three Andalusian provinces with more hectares of olive trees: Jaen, Cordova, and Seville [5]. Jaen is the province with the largest olive-growing area in Spain, where the traditional olive groves are predominant. An important mixture of olive grove managements can be observed in Cordova, the second largest olive-growing area in Spain. In recent years, Seville has increased its olive-growing area, especially with more intensive management systems.

Several olive grove areas were evaluated; olive groves with three different planting systems were identified and analyzed: traditional, intensive, and super-intensive olive groves (Figure 2). The traditional olive groves are the dominant system in Spain, with stocking density of up 400 trees ha⁻¹. The average density usually ranges between 80–120 olive trees ha⁻¹, with square plantation frames where 10–12 m is the most frequent separation between trees. Densities up to 1000–1500 olive trees ha⁻¹ are considered intensive management systems, although densities not higher than 600 are usually observed. The most usual plantation frames are 7 m × 7 m, 8 m × 4 m, and 7 m × 5 m, always leaving corridors with a minimum width of 6 m to favor the mechanization of olive groves. Finally, the super-intensive olive grove management shows densities of up to 2500–3000 olive trees ha⁻¹, with an inter-row spacing of about 1.2–1.5 m and an intra-row spacing of 3.2–4.0 m. On the other hand, as can be observed in Figure 2, the crown diameter also varies according to the planting system: between 8, 5.5, and 3.5 m for super-intensive, intensive, and traditional systems, respectively. All the olive plots analyzed were farmer-managed groves, in which farmers made decisions individually. Thus, different sizes and morphologies of olive crowns and types of soil tillage were found.

The set of olive groves analyzed came from very high-resolution aerial orthophotographs of the PNOA [42]. The studied area was covered by the most recent PNOA photogrammetric flights, dated between June and August 2019. These orthophotographs provided three multispectral bands (blue, B: 450–520 nm; green, G: 520–600 nm; and red, R: 630–690 nm) with a spatial resolution of 0.5 m. The radiometric resolution was 12 bits.

The identification and location of olive grove plots were obtained from the Agricultural Plot Geographic Information System (SIGPAC) [52], which enables the geographic identification of plots declared by farmers and livestock farmers under any subsidy regime relating to the area which is cultivated or used for livestock.

The automatic image download and FCC extraction were obtained through the modules Automatic Image Acquisition and Identification of Elements of Interest developed in the methodology for the automatic inventory of olive groves at plot and polygon level [49].

2.2. Procedure

The first step was the identification of the images with the different planting systems, which were then downloaded and processed to obtain the FCC to create the PNOA dataset. Subsequently, the PNOA dataset was divided into mini-crop images to train, test, and validate the classifier through a segmentation method (data augmentation). Finally, with the information generated, two classifiers based on convolutional neural networks were developed, the first approach oriented in sub-image classification (mini-crop) and the second one at farm level based on the most frequent class detected in the sub-images of each crop.

2.2.1. PNOA Dataset Generation

The process of downloading, identifying, and extracting the FCC of the different olive groves contained three steps. (i) The olive groves with different planting systems were identified through an observer in SIGPAC [52]. (ii) After being identified, they were automatically downloaded from the PNOA through the module “Automatic Image Acquisition” developed in the study [53] which made use of the Web Map Service (WMS) [54] provided by the IGN [55]. (iii) Finally, the FCC of the olive groves was extracted using the method “Identification of elements of interest” developed and validated in the study [53]. Figure 3 shows the images downloaded for a crop.

This module generated three images for each crop (Crop.tiff, Crop_FCC.tiff, and Crop_mask.tiff). Crop.tiff was an RGB image with the information of only the olive grove from PNOA and the area selected by the observer. Crop_FCC.tiff was a binary image with olive grove FCC, and Crop_mask.tiff was a binary image with the delimitations of the olive crop. All images were geo-referenced and covered about 2.5 ha of crop. As a result, the PNOA dataset of processed images for 1187 crops was obtained.

2.2.2. Mini-Crop Set Generation

The PNOA images were divided into a validation set of 236 crops and a training set of 951 crops before the sub-image segmentation. These training images were subjected to the sub-image extraction operation and also divided into a training, test, and validation set (see Figure 4).

The sub-image extraction process divided the crop images into mini-crops to standardize the size and increase the number of samples (data augmentation). To perform this process, segmentation techniques were employed which used the mini-crop size (Hm, Wm) and the stride distance (s) to divide the image into sub-images. A mini-crop was only considered interesting and, therefore, selected for classification (either for training or validation) if the mean value of its corresponding sub-mask image was over a threshold defined as $C_{TH}$ (see Figure 5). As previously discussed, the separation between validation and training sets was performed before sub-image segmentation. Therefore, there was no problem with data duplication because we separated the validation from the testing values without any risk of data duplication.

Every crop was divided into 15 random mini-crops and designated to the training, test, or validation set. The number of mini-crops for every class was (4984, 4276, 5005) for super-intensive, intensive, and traditional classes, respectively. The division of the data for the training, test, and validation sets was (0.8, 0.1, 0.1). These values were chosen to maximize the data for training in this particular subtask. As the significant validation values corresponded to the complete crop classification task, the higher the training data in this subtask, the better the model learned. The test set was used for comparing the performance between trainings, and the validation served as a merit figure for the final performance of the sub-image classifier. Figure 6 shows the distribution of the dataset of mini-crops for each class.

Four mini-crop sets were generated for a set of values of (Hm, Wm, s) (50, 50, 5; 80, 80, 8; 100, 100, 10; 120, 120, 12). Figure 7 shows examples of different mini-crops of 80 × 80 px. The differences between classes can be easily observed. While the super-intensive crops tend to be formed by lines of cultivation, the intensive and traditional crops have sparse olives. It was imposed for every mini-crop that it must contain a $C_{TH}$ percentage of cultivable area to avoid unnecessary, uninformative images; namely, the value selected was 0.1. This value was set heuristically according to the resolution of the crops with scarce FCC zones. We have observed that values lower than 0.1 will overfill the network with too many void images. This thresholding also alleviates training times since the need of an additional ‘void’ label disappears.

2.2.3. Planting System Classifier Based on Convolutional Neural Network (CNN)

Two classifiers have been developed and evaluated, one at the sub-image level and the other at the olive grove farm-level. In both cases, the CNN was composed of 4 consecutive convolutional operations with (32,64,128,256) filters each (see Figure 8). Every filter had a kernel size operation of (3 × 3). Once the input image was processed with this initial feature extractor, the last convolutional layer was flattened and processed by a dense linear network of 3 layers of (3,512,1024) neurons each. Every layer performed a batch normalization operation for a better generalization. The maximum pooling operation after every convolutional module allowed the reduction of the dimension and alleviated the computation cost.

This architecture has been designed following the classical LeNet architecture [56], which is an efficient and compact network sufficient for this classification task. Larger architectures, like ResNet [57], albeit their good results in multi-class classification tasks, are inefficient in terms of computational speed and resources for this 3-class problem. In this sense, the aim of this simple design was to allow a fast inference without GPUs for researchers and authorities or a retraining with other data.

Table 1. Hyperparameters for the training phase of the model.

Optimizer	Adam
Adam Weight Decay	1 × 10−3
Training Epochs	50
Batch size	64
Learning Rate	1 Cycle LR Schedule with LRmax = 1 × 10−2

shows the hyperparameters for the training phase of the model. The learning rate was chosen dynamically using a 1Cycle Learning Rate Scheduler [58]. This led to a robust convergence with higher speeds.

The classifier at sub-imagen level only made use of the CNN proposed, which predicted the planting system from the mini-crop validated in the segmentation method. On the other hand, the architecture of the proposed farm-level classifier is shown in Figure 9. This process of classifiers required more modules. (i) The input image was processed by the sliding windows operation that extracts the mini-crops. (ii) Valid mini-crops were selected depending on the percentage of orchard area in the image. (iii) Those valid mini-crops were processed by the convolutional classifier. (iv) The most frequent class was the selected one. The classifier also output the segmented image for a better explanation of the classification task.

The evaluation metrics to compute the performance of the methods were the following:

• Accuracy: The accuracy computed the fraction of correct predictions among all the predictions done for the evaluation set.
• Precision (macro): The precision was the ratio Tp/(Tp + Fp) where Tp was the number of true positives and Fp was the number of false positives. For this multiclass classification problem, the precision was computed as the average precision for every single label (macroscopic precision).
• ROC AUC 1 vs. 1 (macro): The area under Receiver Operating Characteristic Curve (ROC AUC) indicated the area under the trade-off curve between the true positive rate and the false positive rate. When 1 vs. 1, the AUC was computed for all possible pairwise combinations of classes available and averaged (macro).
• ROC AUC 1 vs. Others (macro): The ROC AUC 1 vs. Others (macro) was similar to the 1 vs. 1 AUC, but it computed the area of each class against the rest.

3. Results and Discussion

The main objective of this study was to develop and validate a DL methodology to discriminate between different planting systems in olive groves. In this sense, a model based on CNN has been proposed to identify super-intensive, intensive, or traditional olive groves in sub-images of the crop and in the farm. The results for the validation set for the sub-imagen and farm level are shown in

Table 2. Sub-image grove classification results for the validation set.

Sub-Image Size (H,W)	Stride Size (s)	Accuracy	Precision (Macro)	AOC 1 vs. 1	AOC 1 vs. R
(50, 50)	5	0.887	0.890	0.974	0.974
(80, 80)	8	0.945	0.944	0.990	0.990
(100, 100)	10	0.930	0.931	0.986	0.986
(120, 120)	12	0.957	0.957	0.994	0.994

and

Table 3. Classification metrics for the complete grove validation set.

Sub-Image Size (H,W)	Stride Size (s)	Accuracy	Precision (Macro)	AOC 1 vs. 1	AOC 1 vs. R
(50, 50)	5	0.800	0.807	0.848	0.845
(80, 80)	8	0.826	0.832	0.874	0.876
(100, 100)	10	0.819	0.813	0.868	0.970
(120, 120)	12	0.809	0.813	0.867	0.867

, respectively. The highest result for a sub-image (accuracy of 0.957) was achieved for the largest sub-images (H = 120, W = 120) while the classification at farm level achieved the highest result (accuracy of 0.826) for intermediate values of sub-image size, specifically (H = 80, W = 80).

The accuracy at farm level was slightly lower than the results at sub-image level since some farms have areas with several planting systems. Figure 10 shows an example of a common misclassification for a grove. The most typical misclassification scenario occurred when a labeled intensive grove had characteristics from both intensive and traditional agriculture styles. In spite of the binary classification, the algorithm provided the segmentation matrix for better explanation.

In this context, sequential analysis at the mini-crop level has proven useful as a segmentation tool, allowing for improved system training through data augmentation. It also provides the possibility of identifying mixed plantation systems. This is proposed as a future line of research.

The study achieved an accuracy of 0.957 % for sub-images and 0.826% at farm level. The data obtained provides accurate and valuable information to current cartographic and agronomic information systems such as SIGPAC [52], making it possible to create periodic mosaics of the agricultural olive landscape and to incorporate additional information at the farm level, a need detected in the studies [11,12].

This automatic and periodic characterization of the olive landscape allows the study of the evolution of planting systems, which are tending towards intensification as shown in some studies [12,59,60]. This trend implies changes in soil management, irrigation, and ground cover vegetation [12], which has agronomic, economic, and environmental consequences [10]. Systematizing the monitoring of olive tree densities with detailed and updated information makes it possible to identify factors that influence the productive yield and sustainability of the olive grove.

One factor to be considered in soil management that changes based on intensification is the water requirement, which will differ according to the geographical location of the crop. The proposed system in this study makes it possible to calculate water demand based on the changes that occur in olive planting systems, with the possibility of improving decision-making.

Another point to highlight from the work is the ability to detect the super-intensive plantings. Such planting has been the easiest class to identify; this can be observed in Figure 11, which shows the confusion matrix for every resolution under validation. As can be seen, the most difficult task is to distinguish between the intensive class (I) and the traditional one (T). The super-intensive class (SI) is easily classified for most of the sub-image sizes. This may be since the super-intensive plantings are structured in rows to harvest the olives with fully mechanized, improved pruning, and improved pest control treatments, which are more complex or impossible in intensive and traditional plantation systems [9].

Although the main objective of this study was to detect planting systems for olive groves, identifying the super-intensive plantations with greater certainty is a step forward because, currently, the most complete published dataset shows a continuous increase in oil yield per hectare up to 14 years after planting [61]. Monitoring these planting systems will allow a more detailed study of their evolution.

Regarding the use of DL techniques for high-resolution image classification, it has been shown that they present promising results, and this study has contributed to making a step forward in the utilization of DL techniques on RS techniques [47].

Therefore, the methodology proposed and validated in the study enables the monitoring of the impact of olive grove intensification, as there are divided opinions on its sustainability. There are research and techniques that promote these cultivation models through deficit irrigation with the aim of increasing mechanization, improving profitability, and increasing carbon sequestration [9], and there are others who defend traditional systems, as they are typical agroforestry systems, cultural heritage, and contributing factors to soil and resource conservation [59,60].

Our proposal has achieved satisfactory results as it is based on very high resolution and freely available aerial orthophotography images from the PNOA [62]. On the other hand, the use of this data source is limited by the temporal resolution since it is updated every three years. To overcome this limitation, future studies should extrapolate the methodology to satellite images of lower spatial resolution (10 m × 10 m) but higher temporal resolution (5 days).

4. Conclusions

This study has made it possible to evaluate the potential of modern data analysis techniques, more specifically deep learning, to discriminate between traditional, intensive, and super-intensive olive groves. As previously mentioned, Olea europaea L. (olive trees) can be considered one of the most important crops in the Mediterranean basin and Andalusia, which is one of the main olive-growing regions in the world with 46.7% of the olive-growing area of Spain. The use of traditional methods based on samples and ground visits to a small percentage of fields are imprecise, expensive, and time-consuming, and they should be replaced by modern techniques, such as the one presented in this paper.

The automation of this process through the development of surface characterization algorithms by means of the interpretation of satellite images is a technique widely used in other fields and which has been validated in this case for the agri-food sector, offering a decision-support system that allows this process to be optimized, making it more efficient and sustainable. Furthermore, although images obtained from satellites have the advantage of covering wide areas quickly, they have the disadvantage of a lower availability of satellite images with very high spatial resolution. An added value of this work has been the acquisition of high-resolution digital orthophotographs from freely accessible platforms, which allows the optimization of results and reduction of costs.

For this purpose, a DL methodology based on the mini-crops classification of fraction canopy cover (FCC) of olive groves using convolutional neural networks to discriminate between olive grove planting systems at farm level based on automatic analysis techniques of open data sources has been developed and validated. The effect of mini-crop size variations to optimize the time-cost/accuracy ratio had to be verified. The goodness of the results obtained, with an accuracy of over 82%, allows us to ensure that it is a useful tool for not only planting system discrimination, but also as a segmentation analysis method for studying the distribution of the olive trees across a crop. Future lines of work should be based on the development of new algorithms that allow us to perform the classification with other types of images obtained from satellites in order to decrease the temporal resolution of the methodology.