**1. Introduction**

In recent years, the new orchard intelligent machinery has shown great advantages in improving agricultural production efficiency and solving the labor shortage problem. First, such machinery has the ability to avoid direct contact between people and their working environments [1]. For example, there are some toxic or high-temperature scenarios which are not conducive to the human body in some operations. Moreover, the repetitive and monotonous nature of some phases of the orchard fruit production process, such as fruit picking, can be tiring and lead to missed operations or accidents. How to achieve autonomous navigation is one of the hot research topics in the field of intelligent machinery for orchards. With its advantages of wide range of detection information and comprehensive information acquisition, visual navigation has become the most widely used robotic navigation method throughout the world. The key aspect of visual navigation is its accurate and reliable extraction of the navigation baseline through image processing technology [2–4].

For the autonomous navigation problem, research ideas are focused on two aspects: road- or sky-based navigation line generation and crop detection–based fitting of navigation lines. Road- or sky-based navigation methods are highly robust to plant species, shape,

**Citation:** Zhou, J.; Geng, S.; Qiu, Q.; Shao, Y.; Zhang, M. A Deep-Learning Extraction Method for Orchard Visual Navigation Lines. *Agriculture* **2022**, *12*, 1650. https://doi.org/ 10.3390/agriculture12101650

Academic Editor: Michele Pisante

Received: 7 September 2022 Accepted: 6 October 2022 Published: 9 October 2022

**Publisher's Note:** MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Copyright:** © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/).

and height and therefore constitute a hot research topic for scholars throughout the world. Crop detection–based navigation methods require accurate identification of crop trunks and are highly robust to complex road environments, thus requiring high adaptability.

Using the features shown in orchard images, He et al. proposed a horizontal projection method to recognize the main trunk area dynamically [5]. The color-difference R-B and twodimensional Otsu algorithm were employed to segment the trunk from the background. A morphological method was adopted to eliminate noises from tiny branches and fading fallen leaves. Similarly, an optimal path extraction method proposed by Li also adopted color-model and segmentation methods [6]. The least-square and Hough transform methods are the most generally used line fitting methods. Based on the least-square method, both studies fit the reference lines of the fruit trees on both sides. The experimental results showed that the path generation method can provide a theoretical basis and technical support for the walking of a kiwi fruit–picking robot [6].

To achieve a better result, Ali et al. proposed a classification-based tree detection algorithm [7]. Color and texture cues were combined to yield better performance than individual cues could accomplish. Lyu et al. applied the Naive Bayesian classification (Artificial Neural Networks (ANN) and K-nearest neighbor (KNN) in [7]) to detect the boundary between trunk and ground and proposed a method to determine the centerline of orchard rows [8]. The advantage of the Bayesian classification is that it requires a small number of samples and a simple training process. In addition, it can effectively reduce impact from branches, soil, weeds, or tree shadows on the ground. In orchard navigation tests, the steering angle deviations generated by the proposed algorithm were much smaller than those generated from manual decisions. This showed that the orchard navigation method is more stable than a method that determines the centerline extraction manually.

Thus far, most researchers have developed algorithms that take advantage of the ground structures of orchards. These studies use the segmented sky from the tree canopy background and the centroid features of the segmented object as the process variables to guide the unmanned ground vehicle moving in the tree rows [1]. Experiments have shown that these approaches have the potential to guide utility vehicles.

Light detection and ranging (LiDAR) technology is also widely used in orchard navigation. Zhou et al. proposed a method for calculating the center point of the trunk with LiDAR sensory data [9]. LiDARs were used to scan the trunks on both sides of the fruit tree row. Point clusters with approximately circular arc shapes were formed. The central coordinate position and the radius of the trunk could be determined through geometric derivation. As the robot moved, its position and posture were corrected in real time by comparing the detected coordinates of the center point of the trunk with those obtained previously. Blok et al. paid more attention to the robot's self-positioning [3]. This research validated the applicability of two probabilistic localization algorithms that used a single 2D LiDAR scanner for in-row robot navigation in orchards. The first localization algorithm was a particle filter (PF) with a laser beam model, and the second was a Kalman filter (KF) with a line detection algorithm. Experiments were designed to test the navigation accuracy and robustness of the two methods, and the results showed that PF with a laser beam model was preferred over a line-based KF for in-row navigation.

Shalal et al. combined LiDAR and cameras in their research [10,11]. The LiDAR was used to detect edge points to determine the width of trunks and of non-trunk objects. The color and parallel edges of the trunks and non-trunk objects were verified by camera images.

Traditional image processing methods are easily affected by sunlight, canopy occlusion, and weeds. With the development of artificial intelligence, Zhang et al. tried to apply deep learning image processing in orchard management [12]. A multi-class object detection algorithm was proposed on the basis of a region convolutional neural network (R-CNN) model to detect branches, trunks, and apples in the orchard environment. VGG16 and VGG19 (the highest MAP of 82.4%) both achieved higher detection accuracy than Alexnet for the skeleton fitting of branches and trunks [13–15]; this study provided a foundation and possibility for developing a fully automated shake-and-catch apple harvesting system.

According to the above analysis of orchard autonomous navigation research results, the limitations of current orchard navigation are reflected in the following three points: <sup>1</sup> In orchards with large tree canopies, it is more difficult to extract the vanishing point, and the application of generating navigation lines based on roads or skies will be limited. <sup>2</sup> The use of traditional image processing methods based on tree trunk detection to fit the navigation path is susceptible to light intensity, shadows, and other factors. <sup>3</sup> Using radar data to improve the midpoint of fruit tree trunks provides a method for fruit tree row extraction, and image sensors have the advantage of low cost.

To address the limitations of the existing methods, we provide a DL\_LS method that uses a deep learning model to extract the trunks of fruit trees near the ground and calculate the fruit tree reference points, fit the fruit tree row lines through the reference points, and calculate the centerlines through the row lines on both sides. In our method, we employ the YOLO V3 network to detect trunks of fruit trees in contact with the ground area, which can be basically independent of light intensity, shade, and disturbances. Furthermore, we use the detected trunk bounding box to determine the key points or reference points of the tree row, which are the middle points of the bottom lines of the bounding boxes, and then extract the tree row lines by the least-square method in order to improve the accuracy of the tree row line extraction.

Our method consists of four steps: detection of the fruit tree trunks using the deep learning method, determination of the fruit tree reference points, fitting of the fruit tree reference row lines, and generation of the orchard centerlines. The deep convolution neural network, which replaces the traditional feature extraction methods, can automatically detect the target after training with enough sampled learning data. The algorithm of the fruit tree row line fitting is put forward using a least-square algorithm, which can effectively extract the orchard machinery walking route.
