Detection and Analysis of Dubas-Infested Date Palm Trees Using Deep Learning, Remote Sensing, and GIS Techniques in Wadi Bani Kharus

Abstract

Many insects attack date palm trees but date palm trees in the Sultanate are particularly under threat due to the spread of pests and the Dubas bug (Db). Date palm productivity in Oman has been reduced by 28% due to Db infestation. The manual field detection of these pests requires huge efforts and costs, making field surveys time consuming and difficult. In this context, remote sensing integrated with deep learning techniques can help in the early detection of Db infestation. A total of 240 date palms with corrected geospatial locations and coordinates and their health status were systematically recorded throughout the 66-square-kilometer study area. We used advanced remote sensing tools and deep learning techniques to detect individual palm trees and their health levels in terms of Db infestation. Very-high-resolution (50 cm) satellite images rendered in visible and NIR bands were used as datasets to delineate and identify individual tree positions and determine their health condition. Our proposed method resulted in an overall accuracy of 87% for the detection of date palm trees and 85% for the detection of health levels of the plants. The overall detection accuracy of high and low infestation levels was observed with high precision at 95% and 93%, respectively. Hence, we can conclude with confidence that our technique performed well by accurately detecting individual date palm trees and determining their level of Db infestation. The approach used in this study can also provide farmers with useful knowledge regarding the Db risk and damage control for better management of Db. Moreover, the model used in this study may also lay the foundations for other models to detect infested plants and trees other than date palms.

1. Introduction

The arable lands in the Sultanate cover an area of 2.12 million hectares, equal to 7% of the country’s total area, which covers 30.2 million hectares [1]. The total agricultural area in Oman has increased from 0.20% [2,3] to 1.5% of the total land area. Date palm (Phoenix dactylifera L.) is a unisexual fruit tree that is native to the hot arid areas of the world, grown in the Middle East and North Africa [4]. Date palm is the most common tree in Oman, with a production rate of 260,000 million tons of dates per annum, making Oman the eighth largest date-producing country in the world [5]. People consumed approximately one half of the total dates whereas the remaining dates were used in industry [6]. Date palm fruits are considered an ideal food that is endowed with a wide range of vital nutrients with many potential health benefits.

Many insects attack date palm trees but date palm trees in the Sultanate are particularly under threat due to the spread of pests and the Dubas bug (Db) (Ommatissus lybicus de Bergevin). The latter is considered one of the most dangerous insect pests (for the life, development, and growth) for palm trees in Oman, the Gulf region, and throughout the globe [7,8,9,10]. Db sucks the sap fluid by damaging the leaves of the date palm, prompting direct destruction [11]. Db has a life cycle of approximately 4 months, producing two generations in a single year. The first generation is born from February to May while the latter is from August to November. Al-Khatri [11] stated that this pest could be controlled by spraying only. However, the detection of this pest and spray management depends upon the field survey conducted by the field staff of the Ministry of Agriculture and Fisheries in Oman [12].

The manual field detection of these pests requires huge efforts and costs. These limitations make field surveys time consuming and difficult [12,13,14,15]. The detection of this creature in the early stage of infestation is limited due to its severity. Tactics such as using insect pheromone traps are used in data palm management to attract these difficult creatures [14,16,17]. Applications of the pheromone traps in the field predominantly capture females [18,19]. Various pesticides were used on trees as a preventative or curative measure [20]. Db stands as one of the most invasive pest species in palms after its global spread, especially in the Middle East [14]. The aforementioned studies showed that the detection of Db does not provide active and consistent information about the affected trees. Moreover, Db infestation detection at an early stage is the main issue in Db control and management. The manual and visual detection of the Db is simply overlooked and needs to be detected remotely.

On top of the aforementioned studies, remote sensing can help obtain and analyze data without physical interaction with the object [21], as it is cheaper than most conventional methods. Measurements of surface reflectance or radiation can be detected by mounting a sensor on a satellite or a drone [22]. Remote sensing techniques have substantially improved vegetation cover mapping by providing complete high-resolution satellite images [23]. For example, remote sensing data were used to detect Hessian fly infestation in winter wheat [24]. Many vegetation indices can be monitored to investigate the health of the plant and it is believed that Db detection could also be achieved by using different combinations of bands and spectral signatures [12,25]. One study [12] estimated 32 different vegetation indices and used the classification technique to detect Db-infested trees in Samail, Oman, using WorldView-3 (spatial resolution of 2 m). The results from their study enumerated the changes in reflectance for different intensities of Db infestation.

Using the wide range of satellite images, deep learning techniques also play a role in obtaining information for different applications regarding the earth’s surface. In recent years, deep learning (as a subfield of machine learning) has attracted substantial attention. Fan [26] used unmanned aerial vehicle (UAV) images (35 mm) to detect tobacco plants using a deep neural network algorithm with an overall accuracy of 93%. Various pests in coconut trees were detected using the NVIDIA Tegra system on chip by integrating UAV images [27]. Convolutional Neural Networks (CNNs) using high-resolution UAV images with a spatial resolution of 2–5 m were used to map different tree species [28]. Their study found that CNNs were powerful tools in mapping vegetation patterns using high-resolution satellite image data. In this context, very-high-resolution (VHR) images with spatial resolutions >1 m are constantly acquired over the earth’s surface by VHR satellites. Utilizing CNNs, VHR aerial images (10 cm) were used to detect pine wood nematodes [29]. Vegetation regions infested with numerous pests and diseases were also discovered using VHR-UAV (0.5 m) images [30]. Duarte [31] used UAVs to detect and quantify eucalyptus longhorned borer damage to eucalyptus trees. Red palm weevils (RPWs) in date palm trees were detected in the semi-arid climate region of Spain using a temporal analysis of UAV images [32]. In addition, WorldView-3 satellite images with a spatial resolution of 0.31 m were used to recognize and monitor RPWs in Saudi Arabia [25]. VHR satellite images were used to detect date palm distribution through CNNs [33]. Health monitoring and preservation of date palm trees was carried using VHR-UAV images to map individual trees [34]. Another study [35] used VHR satellite images for pest detection in mixed hardwood forests. An analysis of diseases on permanent crops was also carried using VHR satellite images [36]. A deep CNN model incorporated with VHR images was developed to recognize damage due to pest infestation [37]. The advent of UAV and VHR satellite images can open the door for new and diverse applications.

One of the first studies on Db infestations was carried out in [38], which examined the influence of environmental variables. Their study correlated date palm concentration and environmental variables (slope, elevation, soil type, water type, etc.). Models were created based on correlations between the infestation rate and environmental factors to forecast the Db infestation rate. Their study resulted in a positive significant relationship between Db infestation and high elevation areas (251–1250 m). Al-Kindi [8] illustrated the Db infestation rate and changes in infestation level in Northern Oman using hot spot analysis of annual data from 2006 to 2015. A spatiotemporal analysis was carried out in ArcGIS 10.3 using the data from the Ministry of Agriculture and Fisheries. A field survey was carried out in two ways: First, the infestation level was examined by cutting and counting the number of bugs per leaf. Second, the honeydew method was used to observe the level of infestation. The study resulted in nine northern governorates being found with significant hotspot infestations but most of the hotspots were on the mountain plains.

WorldView satellite images were used in [9] to count the number of date palm trees using the local maxima technique. Three different scale windows were verified: 3, 5, and 7 m (6 × 6, 10 × 10, and 14 × 14 pixels, respectively). Their study adopted NDVI to segregate the vegetation from non-vegetative portions of the images. Their study showed that the 7-m window size (14 × 14 pixels) was the most suitable for detecting date palm trees with an error value of 11.8%. A one-to-one graph between mean infestation values and tree density resulted in a positive correlation, which showed that the infestation rate increased with an increase in the date palm densities.

Thirty-two vegetation indices were used to assess the response of reflectance on different infestation levels [12]. Maximum likelihood classification was also carried out on the satellite images and correlated using an in-field survey. The study showed that reflectance values decreased in near-infrared and red bands with the increase in infestation levels. The study found that 19 out of the 32 vegetation indices were significantly correlated with the Db infestation levels.

Previous studies have shown the great importance of remote sensing techniques incorporated with CNNs in the detection and mapping of vegetation. To the best of our knowledge, there are few studies on the detection of infested date palm trees in the hot and hyper-arid region of Oman using deep learning models and CNNs. Moreover, the use of very-high-spatial-resolution images is still very limited in this region. Hence, this study can act as a platform to further explore remote sensing and CNN applications in the field of agriculture management and environmental monitoring. Moreover, the model used in this study may also lay the foundation for other models to detect infested plants and trees other than date palms.

The main objective of this study was to detect date palms infested with the Dubas (Db) insect in Wadi Bani Kharus using very-high-resolution satellite images and deep learning techniques. An accuracy assessment was also carried out by collecting information about the spatial distribution of infested and healthy trees on the image acquisition day.

2. Materials and Methods

2.1. Study Area

Wadi Bani Kharus is located in the Wilayat of Al-Awabi, Al-Batinah South Governorate at coordinates of 23.2615° N and 57.5164° E (Figure 1a). This study focused on a location in this Wadi, which is known as one of areas with the most Dubas-infested date palm trees, with an approximate area of 66 square kilometers (Figure 1b). The climate of the study area is hot and hyper-arid with temperatures that reach 45 °C, especially in summer (July) with an annual rainfall of 200 mm [39].

2.2. Satellite Imagery

The dataset used in this study was a very-high-resolution (50 cm) Pleiades-1B image (

Table 1. Pleiades-1B band descriptions.

Bands	Wavelength (nm)	Spectral Region
1	430–550	Blue
2	490–610	Green
3	600–720	Red
4	750–950	Near-Infrared

) rendered in blue, green, red, and NIR bands. The satellite images were obtained on 17 October 2020. This particular day was selected due to the absence of cloud cover and time stamp of the second generation of Db in the year.

2.3. Study Approach

The study approach incorporates multispectral images as an input and results in the identification of an individual tree’s position and health condition. The layout of the study methodology is presented in Figure 2.

2.3.1. Date Palm Tree Detection

The tree detection and evaluation of its health were modeled as a 2D confidence map estimation [40]. The first step in creating the model was the creation of training samples that were further used as a platform to train the model. The training samples were created through a schema which was created through labels in a plant geodatabase and the spectral signatures of the date palm trees [41]. The training samples were used to train seven deep learning libraries in ArcGIS Pro. These deep learning libraries include TensorFlow, Keras, Scikit-image, Pillow, Fastai, PyTorch, and LibTIFF. They are briefly explained in the following sections.

TensorFlow Library

TensorFlow (T_f) is structured on data flow graphs [42] with two basic computation units: nodes and edges. Raschka [43] defined nodes as representing mathematical expressions and edges as a multidirectional arrays (Figure 3). T_f contains support for deep learning as a built-in platform to help in implementation and provide an easy-to-use environment [44]. T_f supports multiple applications and studies across many areas.

Keras Library

Keras (Ks) is a high-level deep API that acts as a tool and was written in Python. Ks is an easy-to-learn but high-level, compact Python library for deep learning. Ks allows the user to focus on the main concepts of deep learning [45]. Simply put, it directs the developer to create layers without the complexities of tensor details and mathematical information [46]. In Ks, the user can create a sub-sequential model and add layers while each layer can have convolution, activation, and batch normalization. Ks is a key tool in providing essential abstraction and building blocks for developing and enabling machine learning with a high iteration velocity. Ks acts as an interface for the tenser flow library to help implement commonly used neural network building block layers (e.g., objectives activation, functions, and optimizers). A basic flow diagram for Ks is shown in Figure 4.

Scikit-Image

In Python programming, Scikit-image (Sk) is a well-documented API that was developed by a team of international associates [47]. Sk (Figure 5) is considered an image library that applies different mathematical algorithms for research, industry, and educational applications.

Pillow

Pillow (Pl) (Figure 6) is an updated Python image library that was used to load and edit the images [48]. It is also used to perform many complex and sophisticated image-handling functions. Pl library was installed and used in this study as it acted as a platform for image support in other Python libraries.

Fastai

Fastai (Fs) is also a deep learning library that can provide state-of-the-art results and predictions swiftly and effortlessly using deep learning techniques [49]. The basic layout of Fs is shown in Figure 7. Moreover, Fs can also provide low-level components that can be used as truth data for the new component. The Fs library consists of a computer vision library that is GPU-augmented, with the possibility of further extension in pure Python. It also contains a two-way callback system that can collect any part of a model or data and alter it at any training stage.

PyTorch

PyTorch (Pyt) is defined as a Python library that can perform dynamic tensor computations with automated differentiation and GPU acceleration [50]. Pyt is a scientific computing python-based package that can be used as a replacement for NumPy, which helps in using the power of GPUs. It is also used as a research platform to offer more flexibility and speed. Pyt consists of multiple APIs that are used for integration purposes and can generate multiple dynamic graphs. Figure 8 shows the basic layout of the Pyt setup that calls data, trains the model, and then implements it in the production stage. The implementation of this library was conducted flowing [51].

LibTIFF

LibTIFF (Lt) is software support for the reading, processing, and storing of the Tag Image File Format (TIFF). This software contains a library to access the TIFF files for processing, a small collection tool for TIFF file manipulation, and a documentation file for the library and tools [52].

2.3.2. Spectral Analysis of the Trained Model

The Python Command prompt was used to create a new environment named “Deep Learning”. The Python environment and the training process used the exported image file. Epochs were used to determine the right combination of date palm pixels in the images [53].

Once the model was trained, the model was applied to detect the date palm trees in the acquired images. Two spectral signatures, namely, the Visible Atmospherically Resistant Index (VARI) and modified soil-adjusted vegetation index (MSAVI2) were used to reduce atmospheric effects and assess vegetation health. VARI has also been used to estimate the leaf area index and vegetation fraction using reflectance values from visible wavelengths [54,55]. On the other hand, MSAVI2 calculates the moderate and healthy vegetation fraction using reflectance values from the visible (red band) and near-infrared (NIR) wavelength.

$$\mathrm{V}\mathrm{A}\mathrm{R}\mathrm{I}=\frac{{\mathrm{R}}_g-{\mathrm{R}}_r}{{\mathrm{R}}_g+{\mathrm{R}}_r-\mathrm{R}({\mathrm{R}}_g-{\mathrm{R}}_b)}$$

(1)

$$\mathrm{MSAVI}2=\frac{\left(2{\mathrm{R}}_{nir}+1-\surd (2{\mathrm{R}}_{nir}+1)^2-8({\mathrm{R}}_{nir}-{\mathrm{R}}_r)\right)}{2}$$

(2)

where R_r, R_g, R_b, and ${\mathrm{R}}_{nir}$ are reflectance values for the red, green, blue, and NIR bands [56].

2.3.3. Health Statistics

The last step in infested tree detection was to estimate the health statistics of the date palm trees. For this purpose, the health statistics of each date palm tree were calculated based on the cell values of the raster images [54,56]. The accuracy assessment was conducted in two steps. First, ground truth data were compared with the detected data after considering the four health levels of date palm trees: highly infested (H), moderately infested (M), less infested (L), and not infested (N). In the second step, the ground truth data were compared with the detected data after considering two health levels of date palm trees by combining highly and moderately infested data into one category (H) and combining less and not infested data into one category (N). The second method was considered after finding that the above-mentioned categories were determined by observation which leaves a possibility of subjective decisions.

2.3.4. Performance Assessment

To assess the performance of the evolving deep learning model on all infestation levels, different metrics were calculated including classification/detection accuracy (A_o), precision (P), and recall (Rc) values to quantify the capability of the model [57] in addition to the Dice similarity coefficient (F1_score) which was added with the intent to balance the relevance of P and Rc equally [58]:

$$A_o=\frac{TP+TN}{n}$$

(3)

$$P=\left(\frac{TP}{TP+FP}\right)$$

(4)

$$Rc=\left(\frac{TP}{TP+FN}\right)$$

(5)

$$F1\_score=2\times \left(\frac{P\times Rc}{P+Rc}\right)$$

(6)

where $TP$ is the total number of true positives, $TN$ is the total number of true negatives, n is the total number of samples, FP is the incorrectly identified infested trees, and FN is the missed infested trees.

The standard deviation (SD) method was applied to estimate the amount of variation in actual infestation level from the in situ survey values. SD was measured using Equation (7).

$$SD=\sqrt{\frac{\sum {\left(x-x_m\right)}^2}{n}}$$

(7)

where x is the model value, while $x_m$ is the mean value estimated by the model and n is the total number of observation values used for the SD estimation.

3. Results

3.1. Extraction of Spectral Signatures

The spectral signatures of the date palm trees were collected from the study area using the schema tool. Figure 9 shows the potential of this study carried out in the study area and the importance of the study area. The red color shows the spatial distribution of samples collected from the date palm trees for spectral signatures. Figure 9 shows the extracted signatures of individual date palm trees in the study area. These extracted signatures of individual date palm trees were then used as training samples to train the deep-learning models.

3.2. Date Palm Tree Detection

Figure 10 shows that the trained model successfully differentiated the date palms from other vegetation as well as other land uses. Some similar studies found it difficult to detect uneven date palm trees using different window sizes [8] but our trained model stands unique as it successfully detected the date palm trees that were cultivated in both regular and irregular patterns.

3.3. Detection of Infested Trees

This study produced classified images (Figure 11) based on the level of D_b infestation on the date palm trees in the study area. The health levels of individual palm trees were classified into four categories: highly infested date palm trees (H), which means an immediate inspection was needed for these trees due to severe infestation; moderately infested date palm trees (M), which means the infestation symptoms were very obvious on the plant; less infested date palm trees (L), which means the symptoms of infestation were starting to be observed on the plant; and not infested date palm trees (N), which means no Db infestation was observed on the plant.

Figure 11 shows the spatial distribution of the Db infestation level of date palm trees. The results show that the model was successful in the detection of date palm trees and the infestation levels. The results showed clear differences between different infestation levels, as Figure 11a shows an abundance of H and M categories of infestation in the study area. The same trend can be seen in Figure 11b,f. Figure 11a,f clearly shows that the model clearly differentiated between date palm trees, other vegetation, and land use type. Moreover, Figure 11a,b,f detected more date palm trees compared to Figure 11c,d,g.

3.4. Accuracy Assessment

A total of 240 date palms with correct geospatial coordinates and their health status were systematically recorded throughout the 66-square-kilometer study area. These ground truth data covered more than 8% of the total date palms in the study area. The date palm tree coordinates (latitude and longitude) were verified with the imaginary technique, resulting in an 87% match.

Both methods resulted in an overall accuracy of 85%. The accuracy of the first step ranged from 63 to 94%, whereas the accuracy of the latter ranged from 83 to 92% (Figure 12 and Figure 13).

In this study, we calculated evaluation matrices based on the P, Rc, and F1_score using TP, FP, and FN. The F1_score was designed to balance both the relevance of P and Rc, thus defined as the harmonic mean of both measures.

Table 2. Evaluation results for the different Db infestation levels in the study area.

Health	FN	FP	TP	Precision (P)	Recall (Rc)	F1_score	SD
H	11	8	162	0.95	0.94	0.94	0.87
M	11	8	26	0.76	0.70	0.73	1.18
L	11	3	38	0.93	0.78	0.84	0.58
N	1	2	6	0.75	0.86	0.80	1.1

lists the results of the evaluation for the different Db infestation levels in the study area.

The P and Rc curves in Figure 14 show that the trained model is more precise in the H and L levels. However, the overall accuracy of M and N infestation levels was greater than 20% that of the H and L infestation levels. One of the reasons for these results might be the level of infestation as the ground truth data recorded approximately 75% of the study area being infested with the H category and only 1% of the date palm trees were recorded with no infestation levels. The overall deviation of infestation levels from their ground truth position was less than 1 m at the H and N levels. The high standard deviation (SD) values in the M and L levels of infestation occurred as the model was less precise in the detection of these infestation levels. Nevertheless, the SD values for all four infestation levels did not exceed 1.5 m.

4. Discussion

The trained model detected the date palm trees in the study area after being trained using the deep learning model libraries. The trained model was able to distinguish the date palms from other plants and various land uses, as seen in Figure 11, which was difficult to achieve in a previous study [8].

The Visible Atmospherically Resistant Index (VARI) and modified soil-adjusted vegetation index (MSAVI2) were used in this study to detect vegetation health as these models involve the near-infrared spectrum that has shown its effectiveness in detecting date palm trees infestation [12]. Individual date palm tree health levels were divided into four categories: highly (H), moderately (M), and less infested (L), and no infestation (N). The results indicated that the trained model was successful in identifying date palm trees and the levels of infestation, and it was able to distinguish between various infestation levels. The coordinates of each date palm tree (latitude and longitude) were initially confirmed using the imaginary approach, yielding an 87% match. These coordinates were collected in a field campaign by a group from the Ministry of Agriculture and Fisheries.

A recent study [59] for Db infestation claimed a strong relationship between infestation level and date palm density which also coincides with our study. The area with a higher density of date palm trees showed a higher Db infestation level (Figure 11a,f). Moreover, the area with a lower density of date palm trees (Figure 11d,g) was less infested with Db.

The accuracy assessment was conducted in two ways. First, (Figure 13) ground truth data were compared with the detected data after considering the four health levels of the date palms. After exploring that, these categories were established through assessment, which provides room for a judgment to be made based on subjective criteria, which was taken into consideration in the second technique. The combined accuracy of the two approaches was 85%. The first method’s accuracy varied from 63 to 94%, whereas the second method’s accuracy ranged from 83 to 92%. Our study exhibits results for the detection of date palm trees that are comparable with those in [9] which used WorldView-3 satellite images. Moreover, they opted for NDVI to segregate the vegetation from non-vegetative portions of the images while we used MSAVI2 and VARI to assess vegetation health.

The evaluation matrices showed that the trained model was more accurate at the H and L levels because of the P and Rc curves shown in Figure 14. However, the M and N infestation levels were not less accurate overall than 20% of the H and L infestation levels. The amount of infestation may be one of the causes of these results. According to ground truth data, approximately 75% of the research area was included in the H category, while just 1% of the date palm trees had no infestations.

The same higher infestation results were found in [59] which showed that the infestation was higher near the freshwater wadis (in the same area as the current study: Wadi Bani Khurus). The results from our study are also consistent with recent studies [38,60] that found higher Db infestation levels in the current study area. These studies also declared the current study area as a hotspot for Db infestation which is consistent to our analysis that found just 1% of the overall area was not found to be infested with Db.

5. Conclusions

Many insects attack date palm trees but date palm trees in the Sultanate are particularly under threat due to the spread of pests and the Dubas bug (Db). A total of 240 date palms with correct geospatial coordinates and their health status were systematically recorded throughout the 66-square-kilometer study area. A novel approach was used in this study, integrating advanced remote sensing tools with deep learning models to detect individual palm trees and their health level in terms of Db infestation. Our proposed method resulted in an overall accuracy of 87% for the detection of date palm trees and 85% for the detection of the health levels of the plants. The trained model was more accurate at the H and L levels, according to evaluation matrices. Moreover, the overall accuracy at the M and N infestation levels was greater than 20% of that of the H and L infestation levels. Nevertheless, the SD values for all four infestation levels did not exceed 1.5 m.

We present subjective experimental findings to demonstrate the effectiveness of very-high-resolution satellite images with deep learning models for detection of date palm tree and their health levels, concluding with confidence that the current study was successful in correctly identifying individual date palm trees and determining their levels of Db infestation. The integration of remote sensing techniques with deep learning models can help in detecting and monitoring the spread and health of the infested trees, reducing insecticide use. The approach used in this study can also provide farmers with useful knowledge regarding the Db risk and damage. More studies are needed to better understand the detection and dispersion of Db infestation to detect it early before any damage is visible.