Detection of Cherry Tree Crown Based on Improved LA-dpv3+ Algorithm

Abstract

Accurate recognition of the canopy is a prerequisite for precision orchard yield estimation. This paper proposed an enhanced LA-dpv3+ approach for the recognition of cherry canopies based on UAV image data, with a focus on enhancing feature representation through the implementation of an attention mechanism. The attention mechanism module was introduced to the encoder stage of the DeepLabV3+ architecture, which improved the network’s detection accuracy and robustness. Specifically, we developed a diagonal discrete cosine transform feature strategy within the attention convolution module to extract finer details of canopy information from multiple frequency components. The proposed model was constructed based on a lightweight DeepLabv3+ network architecture that incorporates a MobileNetv2 backbone, effectively reducing computational costs. The results demonstrate that our proposed method achieved a balance between computational cost and the quality of results when compared to competing approaches. Our model’s accuracy exceeded 89% while maintaining a modest model size of only 46.8 MB. The overall performance indicated that with the help of a neural network, segmentation failures were notably reduced, particularly in high-density weed conditions, resulting in significant increases in accuracy (ACC), F1-score, and intersection over union (IOU), which were increased by 5.44, 3.39, and 8.62%, respectively. The method proposed in this paper may be applied to future image-based applications and contribute to automated orchard management.

1. Introduction

Agricultural remote sensing is the process of optimizing crop production using satellites, human-powered aircraft, and unmanned aerial vehicles (UAVs) [1,2]. Advances in UAV technology have allowed farmers to obtain high-resolution images at far lower costs than traditional methods. Thus, UAV images have become an important data source for agricultural remote sensing [3,4,5,6]. Furthermore, innovative approaches to UAV image processing have become a consistent pursuit in this domain [7,8,9]. In particular, studies that have focused on individual tree identification and extraction using UAV images have suggested the platform’s potential for the detection and delineation of tree canopies. Subsequently, it is expected that the morphological parameters of trees can be accurately identified on a singular basis [10,11,12].

Accurate tree identification is a prerequisite for canopy extraction studies. Feature-based machine-learning methods are widely used for tree identification owing to their simplicity and universality in terms of canopy textures [13,14,15], grayscales [16,17,18], and spectra [19,20]. Such image features typically do not appear individually, but are combined together in statistical analyses to achieve various types of advanced tasks [21,22,23]. Interestingly, complex and highly detailed images allow manual feature engineering to resolve object detection problems, but doing so is costly and difficult, especially when the object and backgrounds share similar features [24]. Compared with manual feature engineering, deep learning (DL) networks enable the automatic detection of the characteristics of single trees, making their identification an ideal candidate process for low-altitude and high-spatial-resolution UAV image processing.

Many advanced DL techniques have been introduced for UAV tree extraction owing to their powerful feature extraction capabilities. According to the output results, these DL models can be divided into three categories: object detection, semantic segmentation, and instance segmentation. Object detection-based tree canopy detection methods adopt rectangular box-labeling of the tree canopy boundary. Santos assessed the performance of three classic detection algorithms, including the “faster” region-based convolutional neural network (RCNN), YOLOv3, and RetinaNet, for urban environments [25]. Their results showed that target detection algorithms based on a convolutional neural network (CNN) accurately locate single target trees in a canopy. Zheng proposed a framework based on faster RCNN to delineate a bounding box around the individual canopies of single oil palm trees, successfully providing information about their positions and locations [26]. Although this information is valuable, some key details regarding the morphology of the tree canopy (e.g., individual shapes and contours) were not provided.

Instance segmentation applies different labels to separate instances of objects belonging to the same class. The latest research reported that these technologies accurately delineate the boundaries of distinct trees while providing detailed location and spatial information [27,28,29]. However, this comes at the price of high computational and per-pixel image annotation costs. Most instance segmentation algorithms focus on improving model performance while ignoring the balance between computational costs and accuracy. In contrast, our approach in this paper emphasizes the use of semantic segmentation. Semantic segmentation algorithms excel at capturing object boundaries compared to object detection and offer computational efficiency compared to instance segmentation [30].

Semantic segmentation has ushered in a new era of benefits via the introduction of fully convolutional networks (FCNs) [31]. Building upon this, U-Net adopts an encoder–decoder structure to stitch shallow and deep features to accurately recover the details of reduced images during image size recovery [32]. Kattenborn achieved semantic segmentations with the U-Net architecture applied to shrubs, herbs, and trees with 84% accuracy [33]. However, the segmentation of large-scale objects suffers because the detailed location/spatial information related to object boundaries is missing owing to pooling or striding operations, despite the U-shaped encoder–decoder architecture storing the richest information [34].

Models such as DeepLab later extended these approaches by introducing atrous convolutions, conditional random fields (i.e., DeepLabv1), residual networks, and Atrous Spatial Pyramid Pooling (ASPP). The current DeepLabv3+ surpasses the previous versions in the DeepLab family, and experimental results using the PASCAL VOC-2012 dataset have demonstrated that DeepLabv3+ outperforms SegNet [35], the pyramid scene parsing network (PSP) [36], and FCN [37,38]. DeepLabv3+ has, therefore, been widely used for image segmentation tasks in various complex scenes, such as along roads [39], water systems [40], and farmland extraction [41]. Morales reported that the DeepLabv3+ model outperforms four U-Net architectures in identifying semantically segmented mauritia flexuosa palm trees from UAV images [42]. Ferreira used DeepLabv3+ alongside morphological post-processing to detect palm tree canopies and compared the results to the standard FCN, showing an improvement in canopy detection of 34.7% [43].

Although DeepLabv3+ shows excellent performance, the application strategies that can be applied to different scenarios require constant reassessment. For example, although it improves accuracy and speed by adopting the improved Xception backbone, these improvements come at the price of complex architectures and persistently high computational costs, which make it difficult to run on farm terminals with limited computing power. Hence, Torres evaluated the performance of Xception and MobileNetv2 as interchangeable backbone models of DeepLabv3+ for Brazilian Cumbaru tree segmentation [30]. Quantitative analyses showed that the MobileNetv2-based model effectively delineated canopy types with lower computational costs than when using the Xception backbone. Encoding lightweight backbone networks allows for practical algorithmic convergence with lower costs, which is quite amenable to agricultural uses. However, DeepLabv3+ and its lightweight backbone networks do not perform well with noncompact cherry tree extractions. Normally, fruit tree pruning, which comprises a task different to forest-land classification, encourages branches to grow toward an outer spherical threshold so that both air and light penetrate the inner foliage to reach the center of the tree as much as possible. This human-induced growth style exacerbates DeepLabv3+’s inability to completely estimate the relationship between the local features of large-scale targets owing to their non-customized expansion ratios, resulting in the appearance of holes in large-scale target segmentation despite the high overall accuracy.

To remedy these shortcomings, the current study aims to develop a low-cost lightweight tree canopy extraction model, provide a semantic segmentation strategy for non-compact fruit trees with gaps between branches, and assess the DeepLabv3+ backbone variants in the performance of tree segmentation.

2. Study Site and Materials

2.1. Study Site and Image Acquisition

Beijing Zhongnong Futong Horticulture Co., Ltd., is an agricultural high-tech service enterprise, leveraging expertise and technologies from leading scientific research institutions, including China Agricultural University, Chinese Academy of Agricultural Sciences, Beijing Academy of Agriculture and Forestry, and Beijing Agricultural College. The study area took place in Zhongnong Futong’s commercial cherry orchard (Figure 1a–c). The orchard spans over 200 acres, featuring cherry tree heights ranging from 2 to 5 m. The cherry trees were meticulously spaced at intervals of 4 m with 5 m paths between rows. The canopy had grown irregularly, the branches and leaves were scattered with many gaps, and some had noticeable holes (Figure 1d) and black-hole shadows (Figure 1e). Weeds of different densities were randomly distributed on both sides of each row, and their physiological characteristics (e.g., color and texture) were similar to those of a natural tree canopy (Figure 1f). Images of the study areas were obtained using a commercial-grade DJI Phantom 3 UAV quadcopter (DJI Technology Co., Ltd., Shenzhen, China) equipped with a Sony EXMOR 1/2.3 in complementary metal–oxide–semiconductor digital camera (Sony Corporation, Tokyo, Japan), outputting 3000 × 4000-pixel red–green–blue three-channel images. Observations were taken in dry, windless weather (<1.5 m/s) at multiple distances from the trees (20–60 m) to capture variations in scale.

A typical UAV flight plan includes a procedure to obtain video batch training datasets and high-quality images by hovering at multiple distances from the trees. In our study, 1037 images were taken from the test area. For more details about the experimental setup, please see Cheng et al. (2020) [24].

2.2. Dataset Construction

Semantic segmentation requires pixel-level labeled datasets. In this study, each image pixel was manually assigned to a tree or background category using free DL LabelMe software (version 5.0.1, developed by the Massachusetts Institute of Technology, Cambridge, MA, USA) [44]. Each cherry tree crown was delineated manually by an expert with experience in agricultural image processing, and the data were used as a benchmark for model training and evaluation (Figure 2). To improve the generalizability of the model, the original cherry tree images were selected under different weather and altitude conditions and further expanded using image augmentation methods that included random rotations of 45°, 90°, 135°, and 180° and random flipping in mirror, horizontal, and vertical modes, resulting in an augmented dataset of 2168 images, which were divided into training, validation, and testing sets at a ratio of 8:1:1 by random selection. The test set consisted of images near the ground (20–30 m), at low altitudes (50–60 m), and with low- and high-density weed coverage. The statistics of these datasets are shown in

Table 1. Detailed dataset.

Dataset	Training Set	Validation Set	Test Set		Total Number
			Near Ground	Low-Altitude
Number of images	1730	215	109	109	2163

.

3. Methodology

The proposed method can be summarized in two steps. The first distinguishes the tree canopy regions from the background using the LA-dpv3+ network. The model’s DeepLabv3+ framework was modified by replacing Xception with MobileNetv2 as the backbone for feature extraction, which effectively reduces the memory required for network parameters and operations. To strengthen tree information extractability, an effective discrete cosine transform (DCT) module was designed to enhance the convolutional block attention module (CBAM). A simplified connection to the main module is shown in Figure 2. The second stage, following background removal, discriminates individual tree detections in the face of continuous tree canopies (see Section 3.2).

3.1. LA-dpv3+ Network

This section briefly introduces the basic DeepLabv3+ architecture, MobileNetv2 network, and enhanced CBAM modules, which comprise blocks of the proposed approach. It then discusses the proposed segmentation framework.

3.1.1. Fundamentals of DeepLabv3+ Architecture

DeepLabv3+ was proposed by Google and was built upon its predecessor, DeepLabv3 [35]. It is mainly implemented by encoder and decoder modules. The encoder stage is dedicated to extracting image features and consists of the X_ception-65 backbone and the ASPP mechanism. Specifically, X_ception-65 outputs low-level features for the decoder module and high-level semantics for the ASPP layer. In DeepLabv3+, X_ception-65 removes all max-pooling layers of the aligned Xception backbone and replaces the depth-wise separable convolution with striding. It also adds batch normalization and rectified linear unit (ReLU) operations. The ASPP convolution’s custom dilation rate was used to capture the global context of each image. It consists of a 1 × 1 and three 3 × 3 atrous layers at rates of 6, 12, and 18, and it incorporates an image-pooling layer. In the decoder stage, the features extracted by the ASPP module were bilinearly upsampled by a factor of four, followed by concatenation with the corresponding low-level features from the encoder stage. Before concatenation, low-level features were first convolved with a 1 × 1 filter to reduce the number of channels (Chen et al., 2018). Finally, the three 3 × 3 convolutions were applied to refine the fused features, followed by 4× bilinear upsampling.

Figure 3 shows that X_ception-65 contained 65 convolutional layers, including 11 in the ingress stream, 48 in the intermediate stream, and 6 in the egress stream. Although X_ception-65 is a powerful feature extractor, its computational complexity is high, resulting in a heavier-than-warranted DeepLab3+ encoder module. To reduce computational redundancy, a lightweight network is needed that operates using limited calculation resources so that agricultural platforms can be supported.

3.1.2. MobileNetv2 Backbone

MobileNetv2 [45] is a network conceived for mobile and embedded devices with fewer parameters and fewer computational complexities than larger models. Therefore, it is suitable for real-time applications on embedded terminals. Additionally, research has reported that, although DeepLabv3+ uses the new generation MobileNetv3 to receive slight advantages in computation, its accuracy is lower than that of DeepLabv3+ using MobileNetv2 [46]. Therefore, MobileNetv2 was chosen in this study as the main feature extraction network. It builds upon the core MobileNetv1 [47] benefits of extensive depth-wise separable convolutions used as efficient building blocks, which significantly reduces the number of parameters. In particular, MobileNetv2 introduced the inverted residual block, in which residual connections exist between bottleneck layers. The bottleneck structure was first developed as part of the ResNet network, and adding a bottleneck structure to MobileNetv2 avoids information loss from tensors with a small number of channels caused by the ReLU operation. Because the last layer of the point-by-point convolution uses the linear activation function, it also has a “linear bottleneck” structure. This inverse residual structure is based on the standard ResNet, which entails first reducing the dimensionality, followed by increasing it, whereas the inverse residual increases the dimension first using the point-by-point convolution and reduces the dimension of the result of the depth-wise convolution. Compared with the standard design, the inverted structure is significantly more memory-efficient. The MobileNetv2 encoder reduces the computational complexity of the model, but it also weakly captures and fuses semantic information. The solution to this problem is described in the next subsection.

3.1.3. Enhanced Attention Module

Holes and other irregular canopy shapes are commonly observed in cherry orchard scenarios, and they call for contextual and detailed semantic information so that neural network models can pay attention to these features. CBAM (a convolutional block attention module) is a sophisticated neural network module designed to enhance the performance of convolutional neural networks (CNNs) by prioritizing relevant spatial and channel-wise features for capturing intricate image features [48]. Demonstrating its effectiveness across various computer vision tasks, including image classification and object detection, CBAM has proven to be a powerful tool in diverse research areas [49,50,51]. Consequently, this paper utilizes CBAM to refine the features extracted from the lightweight model. Therefore, this paper leverages CBAM to refine the features extracted from the lightweight model.

The CBAM was developed from human visual attention methods, which pay more attention to key features and ignore those that contribute less. It consists of channel and spatial attention blocks, and the channel attention mechanism identifies the most meaningful features located by the spatial attention mechanism. The CBAM uses global average pooling (GAP) to simply average the features, which causes sufficient semantic information to be preserved in feature maps, even when combined with global max-pooling. Qin et al. showed that GAP is a special case of DCT, and using GAP in the channel attention mechanism means that only the lowest-frequency information is retained, whereas it discards important information from other frequencies [52]. Inspired by Qin et al., our frequency layer is incorporated into the channel attention mechanism of CBAM to enhance the information extractability of tree crowns (Figure 4).

The DCT is a mathematical operation capable of transforming an image from the spatial domain to the frequency domain. Particularly, a two-dimensional (2D) version of the DCT is widely used for feature extraction, and in various image processing applications. For an image feature tensor in networks $x^{2\mathrm{d}}\in R{ }^{H\times W}$, the 2D DCT can be formulated as [53]

$$T{P}_{{}_{h,w}}^{{}^{2d}}={\displaystyle \sum _{i=0}^{h-1}{\displaystyle \sum _{i=0}^{w-1}{x}_{i,j}^{2d}\cos \left(\frac{\pi h}{H}\left(i+\frac{1}{2}\right)\right)}}\cos \left(\frac{\pi w}{W}\left(j+\frac{1}{2}\right)\right), s.t. 0\le h\le H-1, 0\le w\le W-1$$

(1)

where $T{P}_{{}_{h,w}}^{{}^{2d}}\in R{ }^{H\times W}$ is the frequency spectrum of the 2D DCT at index (h, w), where h and w are the height and width, respectively, of $x^{2\mathrm{d}}$. Supposing h and ω are both equal to zero, we obtain [53]

$$\begin{array}{l}T{P}_{{}_{0,0}}^{{}^{2d}}={\displaystyle \sum _{i=0}^{h-1}{\displaystyle \sum _{i=0}^{w-1}{x}_{i,j}^{2d}\cos \left(\frac{0}{H}\left(i+\frac{1}{2}\right)\right)}}\cos \left(\frac{0}{W}\left(j+\frac{1}{2}\right)\right)\\ \\ =GAP(x_{i,j}^{2d})HW\end{array}$$

(2)

This indicates that the GAP is a feature decomposition case with its lowest frequency in the frequency domain. It is clearly preferable to capture adequate information by exploiting more frequency components from feature maps instead of only those of the lowest frequencies.

Inspired by DCT theory, the frequency layer is further incorporated into the channel attention mechanism of CBAM to strengthen the information extractability of tree crowns. In terms of the spatial attention module, we embraced the well-established iterative process outlined by Sanghyun Woo et al. [47]. The overall structure of the enhanced CBAM is shown in Figure 4. Typically, the feature map is equally split into several groups along the channel dimension, and each part is assigned a corresponding 2D DCT frequency component. In this study, a diagonal DCT feature strategy was used for efficiency. Specifically, five DTC frequency components from low to high in the horizontal and vertical directions were selected. According to Equations (1), these DCT frequency components can be written as

$$\begin{array}{l}\{\begin{array}{l}TP\left(0,0\right)={\displaystyle \sum _{i=0}^{h-1}{\displaystyle \sum _{i=0}^{w-1}{x}_{i,j}^{2d}\cos \left(\frac{0}{H}\left(i+\frac{1}{2}\right)\right)}}\cos \left(\frac{0}{W}\left(j+\frac{1}{2}\right)\right), \\ \\ TP\left(H-1,0\right)={\displaystyle \sum _{i=0}^{h-1}{\displaystyle \sum _{i=0}^{w-1}{x}_{i,j}^{2d}\cos \left(\frac{\pi \times (H-1)}{H}\left(i+\frac{1}{2}\right)\right)}}, \\ \\ TP\left(0,W-1\right)={\displaystyle \sum _{i=0}^{h-1}{\displaystyle \sum _{i=0}^{w-1}{x}_{i,j}^{2d}\cos \left(\frac{\pi \times (W-1)}{W}\left(j+\frac{1}{2}\right)\right)}}, \\ \\ TP\left(\frac{H-1}{2},\frac{W-1}{2}\right)={\displaystyle \sum _{i=0}^{h-1}{\displaystyle \sum _{i=0}^{w-1}{x}_{i,j}^{2d}\cos \left(\frac{\pi \times \left(H-1\right)}{2H}\left(i+\frac{1}{2}\right)\right)}}\cos \left(\frac{\pi \times \left(W-1\right)}{2W}\left(j+\frac{1}{2}\right)\right),\\ \\ TP\left(H-1,W-1\right)={\displaystyle \sum _{i=0}^{h-1}{\displaystyle \sum _{i=0}^{w-1}{x}_{i,j}^{2d}\cos \left(\frac{\pi \times \left(H-1\right)}{H}\left(i+\frac{1}{2}\right)\right)}}\cos \left(\frac{\pi \times \left(W-1\right)}{W}\left(j+\frac{1}{2}\right)\right), s.t. 0\le k\le C, \end{array} \\ \end{array}$$

(3)

where $TP\left(h,w\right)$ denotes the 2D DCT results at indices (0,0), (H-1,0), (0, W-1), ($\frac{H-1}{2}$,$\frac{W-1}{2}$), and (H-1, W-1), respectively. If given an entire feature map, $X\in R{ }^{C\times H\times W}$, C is the number of channels, each of which is processed by the same frequency component and concatenated. Hence, a total of five multi-frequency feature vectors are obtained:

$$\{\begin{array}{l}{F}_1=\mathrm{cat}\left[(T{P}_1\left(0,0\right)),(T{P}_2\left(0,0\right)),\dots ,T{P}_C\left(0,0\right))\right]\\ \\ F_2=\mathrm{cat}\left[(T{P}_1\left(H-1,0\right)),(T{P}_2\left(H-1,0\right)),\dots ,T{P}_C\left(H-1,0\right))\right]\\ \\ F_3=\mathrm{cat}\left[(T{P}_1\left(0,W-1\right)),(T{P}_2\left(0,W-1\right)),\dots ,T{P}_C\left(0,W-1\right))\right]\\ \\ F_4=\mathrm{cat}\left[(T{P}_1\left(\frac{H-1}{2},\frac{W-1}{2}\right)),(T{P}_2\left(\frac{H-1}{2},\frac{W-1}{2}\right)),\dots ,T{P}_C\left(\frac{H-1}{2},\frac{W-1}{2}\right))\right]\\ \\ F_5=\mathrm{cat}\left[(T{P}_1\left(H-1,W-1\right)),(T{P}_2\left(H-1,W-1\right)),\dots ,T{P}_C\left(H-1,W-1\right))\right]\\ \end{array}$$

(4)

where F₁–F₅ is the output of the DTC attention mechanism. Subsequently, these output feature vectors are forwarded to the multilayer perceptron in parallel with the aggregated max-pooling vectors to produce channel attention vectors. The whole channel attention can thus be written as

$$Mc(F)=sigmoid\left(MLP\left(Maxpool\left(F\right)\right)+{\displaystyle \sum _{n=0}^{5}MLP\left(F_n\right)}\right)$$

(5)

3.1.4. Architecture of LA-dpv3

The proposed framework, LA-dpv3+, is based on DeepLabv3+, MobileNetv2, and enhanced CBAM modules (Figure 5). The LA-dpv3+ framework was modified in the DeepLabv3+ framework by using the lightweight encoder MobileNetv2 and retaining parts, such as the Atrous Spatial Pyramid Pooling and encoder–decoder modules. Because a tree canopy can be distinguished from the background by features of color, texture, or shape, which are generally extracted by a shallow neural network, an enhanced CBAM was added after the quarter layers of MobileNetv2 as part of the decoder module. The refined features were then compressed by a 1 × 1 convolution to be concatenated with multiscale features from the ASPP module.

3.2. Individual Tree Detection

The proposed tree canopy extraction algorithm relies on an individual tree detection module, as described by Cheng et al. (2020) [24]. It was developed to identify trees from mingled canopies. This detection method was tested against different lighting and overlapping conditions. For example, a canopy can have heavily intermingled or barely overlapping branches with different numbers of interweaving trees. The method involves several steps in which the background is removed by applying the Excess Green Index (EXG) and Otsu’s method, the target regions are extracted using the length threshold of the circumscribed rectangle of the connected area, a Gaussian mixture model (GMM) is built by fitting a projection histogram curve in the horizontal direction of the extracted interlacing regions, and image segmentation is performed by estimating the GMM parameters.

3.3. Experimental Setup and Evaluation Indices

3.3.1. Experimental Setup

The experiments utilized the Python and PyTorch deep learning framework for training and testing the performance of the LA-dpv3+ method. Additionally, the specific configuration details of the experiments are shown in

Table 2. Experimental software and hardware configuration.

Item	Detail
CPU	Intel® Core (TM) i7-6700 k
RAM	16 GB
Operating system	Ubuntu 16.04 LTS
CUDA	CUDA10.0 and CUDNN7.5
Python	Python 3.7 and PyTorch 1.4.0

. The input image size for our model was consistently set at 512 × 512 pixels. All experiments were trained using the Adam optimizer with a learning rate of 0.007, the momentum factor was set to 0.9, the batch size was set to 24, and 200 epochs were used. As we had two possible outcomes, tree or background, binary cross-entropy was used as the loss function.

3.3.2. Evaluation Indices

The proposed method was evaluated in terms of segmentation accuracy and implementation costs. Four evaluation indicators (i.e., model size, parameter quantity, training time, and test time) were used to evaluate the running cost, and the indicators of accuracy (ACC), F1-score, and intersection over union (IOU) were used to measure segmentation accuracy. ACC is the percentage of correctly extracted canopy pixels (positives) from the background (negatives). The F1-score is the harmonic mean of precision and recall, indicating the balance between false positives and false negatives (background). The IOU reflects the consistency of the segmented canopy area with the ground-truth area. These metrics are defined by the following equations:

$$ACC=\frac{TP+TN}{TP+FP+TN+FN}\times 100\%$$

(6)

$$F1=\frac{2TP}{2TP+FP+FN}\times 100\%$$

(7)

$$IOU=\frac{\left|S\cap S_i\right|}{\left|S\cup S_i\right|}\times 100\%$$

(8)

where a true positive (TP) is the number of canopy pixels correctly predicted as being produced by the segmentation algorithm. A false positive (FP) represents the number of background pixels classified as the canopy. A true negative (NT) is the number of background pixels correctly predicted by the segmentation algorithm. Lastly, a false negative (FN) is the number of canopy pixels counted as the background. S and Si are the segmentation maps and ground truth images, respectively.

4. Experimental Results

First, the improved strategy based on DeepLabv3+ was verified, including the performance of the DeepLabv3+ variant and the efficacy of the enhanced CBAM module. Second, the overall performance of the proposed method under weed-covered conditions was tested, which had already been determined to be the most challenging extraction case [24].

4.1. Comparison of Three Backbones

Three DeepLabv3+ variants were trained in this study: ResNet (the backbone of DeepLabV3), Xception (the backbone of DeepLabv3+), and MobileNetv2. The results are shown in

Table 3. Implementation cost and segmentation accuracy for the DeepLabv3+ variant, Resnet, Xception, and MobileNetv2.

Backbone	Implementation Cost				Segmentation Accuracy
	Size (MB)	Number of Parameters	Total Training Time (Hours)	Inference Time (Seconds)	ACC (%)	F1-Score (%)	IOU (%)
Resnet101	475.3	59,339,426	12.31	0.51	94.67	95.31	89.87
Xception71	438.6	54,700,434	11.59	0.57	94.01	95.18	88.68
MobileNetV2	46.8	5,811,170	5.29	0.35	93.43	94.70	87.65

. All networks were quite successful in their segmentation accuracy, achieving over 87% in ACC, F1-score, and IOU separately. Among them, ResNet provided the best ACC, F1-score, and IOU values of 94.67, 95.32, and 89.87%, respectively. The segmentation accuracy of MobileNetv2 was slightly lower than that of ResNet by 1.24, 0.61, and 2.22%, respectively. However, in terms of running costs, MobileNetv2 effectively controlled the functional parameters and memory usage of the model. The size of MobileNetv2 is only 46.8 MB, which is about one tenth of ResNet and one ninth of Xception. The numbers of parameters used for ResNet and Xception are ~9.4 and 10.2× that of MobileNetv2, respectively. Regarding time costs, ResNet took 9 h, 31 min to process; Xception took 8 h, 59 min; and MobileNetv2 only took 3 h, 29 min for training, saving ~6 and 5 h, respectively.

As shown in Table 3, the running cost of the model was proportional to its accuracy. Compared with MobileNetv2, ResNet and Xception achieved small accuracy gains at the cost of larger sizes, more parameters, and greater training and inference times. Although the segmentation accuracy of MobileNetv2 was slightly lower than that of ResNet and Xception, the model’s ACC, F1-score, and IOU values were greater than 87.65%. From the perspective of the trade-off between segmentation accuracy and implementing efficiency, applying MobileNetv2 as the backbone is the optimal choice.

4.2. Effectiveness of Attention Module Embedding

To evaluate the effectiveness of the attention mechanism for key information extraction, an intermediate feature map was visualized, as shown in Figure 6. Figure 6a shows the input feature image of the DTC-channel attention module. Each square in the figure represents a channel of the convolution operation, which indicates the current learning results (i.e., a class of low-level image features). Visually explicit or indistinct appearances in the squares reflect critical or minor feature contributions, respectively. Figure 6b shows the output of the DTC-channel attention module. The values in the squares in Figure 6b correspond to the weights of each channel in Figure 6a. The weight value of the explicit channel was larger than that of the indistinct channel, which indicates that the key features were focused through our channel attention module. The output feature maps of the spatial attention module are shown in Figure 6c. The outline of the canopy is highlighted, indicating that the spatial attention mechanism accurately and effectively locates the features. Additionally, the main body of the canopy is surrounded by a complete borderline, implying that the spatial attention mechanism achieves a high level of correlation between similar features at different distances.

To further evaluate the feasibility of the proposed attention mechanism, tests were performed on images taken at different flight heights. Figure 7 illustrates an example of the segmentation results. The first row shows the results of an image captured at approximately 20 m, and the second row represents 50 m. Green rectangles are set on the image for easy visual inspection. Comparing the extraction results of the algorithm before and after adding the enhanced CBAM, it is obvious that the proposed attention module eliminates the hollow inside part (hole) of the tree canopy, which is particularly significant to the performance of MobileNetv2, followed by Xception and ResNet. Comparing images taken at different heights, it can be seen that, with the increase in the target scale, the holes inside the canopy are more obvious, and complete canopy extraction becomes more difficult. Limited by the model scale, MobileNetv2 turns out to be inferior to Xception and Resnet in the segmentation of large-scale objects; however, the addition of the attention mechanism makes up for this. A quantitative description is provided below.

To quantitatively verify the effectiveness of the improved CBAM in the DeepLabv3+ framework, comparative experiments were conducted on different combinations of backbones and convolutional attention mechanisms. Comparing Table 3 and

Table 4. Comparison of attention mechanism and different backbone network combinations.

Model	Computational Cost Assessment Index				Segmentation Evaluation Index
	Size (MB)	Number of Parameters	Total Training Time (h)	Inference Time (s)	ACC (%)	F1-Score (%)	IOU (%)
Resnet101+ Enhanced CBAM	484.9	60,528,388	12.57	0.55	94.75	95.78	90.01
Xception71+ Enhanced CBAM	441.0	54,998,004	11.21	0.58	94.34	95.68	89.26
MobileNetV3+ Enhanced CBAM	46.9	5,821,780	5.32	0.36	94.54	95.97	89.64

, it can be seen that, no matter the backbone, adding the attention mechanism has a positive impact on the performance of DeepLabv3+ at a small cost increase. Notably, the combination of MobileNetv2 and the enhanced CBAM improved the ACC by 1.11%, F1-score by 1.9%, and IOU by 1.99%; memory added only 0.1 MB of resource consumption. Compared with the DeepLabv3+ using only Xception, the ACC, F1-score, and IOU improved by 0.53, 0.97, and 0.96%, respectively, whereas the overall memory requirement was reduced by 397.7 MB.

4.3. Overall Performance of the Proposed Algorithm

The orchard image samples were taken in a variety of weather and background conditions, and the primary focus area was weed-cover conditions. The proposed algorithm was compared to Cheng’s [24] method and the classic Mask-RCNN [54] instance segmentation algorithm. Figure 8 provides an example of the experimental results. Typical test images included various weed-cover conditions, as shown in Figure 8a, where trees in the left and middle columns of the image overlapped high-density weeds, whereas trees in the right column overlapped low-density weeds. Figure 8b shows the individual canopies produced by Mask-RCNN. Figure 8c shows the segmentation results of the algorithm proposed by Cheng et al. (2020), and Figure 8d presents the results of the proposed method. The extracted tree canopy is marked with red rectangles (Figure 8b,d), or highlighted patches (Figure 8c).

Figure 8b shows the excellent performance of Mask-RCNN in distinguishing between weeds and tree canopies. However, many image details were missed, resulting in overly smooth tree canopy borders. Additionally, the mask prediction of the target shape was inconsistent. Examining Figure 8c and the results of the preprocessing method that applied the EXG index and Otsu, the previous algorithm failed to discriminate the canopy from the background with heavy overlapping of weeds. Additionally, because lighting conditions have huge effects on the color index-based method, some shaded areas of tree canopies were misclassified as background. However, with the proposed method, the incorrect segmentations were corrected.

Table 5. Average results for 200 images using three different algorithms.

Method	ACC (%)	F1-Score (%)	IOU (%)
Mask-RCNN	88.51	88.72	77.38
Algorithm (Cheng et al. (2020) [24])	86.49	87.62	82.15
Proposed Algorithm	91.93	91.01	90.41

shows the segmentation results of the 200 test images using three different segmentation methods. The average ACC, F1-score, and IOU values of Mask-RCNN were 88.51, 88.72, and 77.38%, respectively. The overall performance of the proposed method was higher than that of Mask-RCNN, with the average ACC, F1-score, and IOU values increasing by 3.42, 2.29, and 13.03%, respectively. When preprocessing by color index, the ACC, F1-score, and IOU values were 88.4, 87.62, and 82.15%, respectively. Adding the neural network increased these scores to 91.9, 90.53, and 90.41%, respectively.

5. Discussion and Conclusions

This paper applied the LA-dpv3+ neural network for fruit tree canopy and back-ground segmentation to improve the previously effective state-of-the-art method of canopy extraction with weeds and open spaces in the background. The proposed model was built upon the DeepLabv3+ network architecture and integrated the lightweight Mo-bileNetv2 network along with an enhanced CBAM that includes a frequency layer.

Three variants of DeepLabv3+ were tested with ResNet, Xception, and MobileNetv2 backbones. The results align with those specified in Torres et al. [30], demonstrating that the MobileNetv2 variant consistently outperformed its Xception counterpart. The performance was further improved by combining MobileNetv2 with the enhanced CBAM. The model accuracy exceeded 89%, while the model size was only 46.8 MB, showcasing a favorable trade-off between segmentation accuracy and implementation efficiency. In terms of overall canopy extraction performance, the segmentation evaluation metrics, including ACC, F1-score, and IOU, increased by 5.44, 3.39, and 8.62%, respectively, compared to the approach by Cheng et al. [24]. This result is particularly promising because deploying high-end hardware in the field is often impractical due to its exorbitant costs. In contrast, lightweight models designed for embedded terminals offer a more cost-effective, compact, and readily deployable solution.

However, the LA-dpv3+ model is sensitive to light changes, which is minimized when capturing images around noon or on cloudy days. During early morning or evening hours, there is lengthening of the shadows between tree canopies due to the changing position of the sun, which can lead to false detections. Further training and validation would also be required to apply this method to similar fruit trees and to use it in different orchards. This potential adaptation would enable users to capture videos in the field using smartphones and leverage cloud resources for near-real-time detection. Future research efforts should prioritize addressing these challenges with the ultimate goal of applying the proposed model to embedded terminals for the intelligent image-based management of cherry orchards.