Embedding a Real-Time Strawberry Detection Model into a Pesticide-Spraying Mobile Robot for Greenhouse Operation

Abstract

The real-time detection of fruits and plants is a crucial aspect of digital agriculture, enhancing farming efficiency and productivity. This study addresses the challenge of embedding a real-time strawberry detection system in a small mobile robot operating within a greenhouse environment. The embedded system is based on the YOLO architecture running in a single GPU card, with the Open Neural Network Exchange (ONNX) representation being employed to accelerate the detection process. The experiments conducted in this study demonstrate that the proposed model achieves a mean average precision (mAP) of over 97%, processing eight frames per second for 512 × 512 pixel images. These results affirm the utility of the proposed approach in detecting strawberry plants in order to optimize the spraying process and avoid inflicting any harm on the plants. The goal of this research is to highlight the potential of integrating advanced detection algorithms into small-scale robotics, providing a viable solution for enhancing precision agriculture practices.

1. Introduction

Agriculture continues to play a crucial role in economies relying on agricultural activities [1], particularly in countries like Morocco, where it constitutes approximately 10.33% of the Gross Domestic Product (GDP) as of 2022 [2]. This sector is a significant source of employment in Morocco, contributing to nearly 33.3% of jobs and accounting for over 23% of exports [3]. Fresh strawberries are securing their position among the top 10 most exported fruits and vegetables from this country: the revenue earned through strawberry production consistently ranges from USD 40 to USD 70 million almost every year and is still increasing by an average of 3% annually [4]. However, the production of such soft fruits faces a great challenge because of pest infestation and diseases: the costs of managing these can contribute to nearly half of the production costs [5].

Recently, innovative techniques have been developed and used to improve agricultural practices, giving birth to the term “precision agriculture” [6]. The introduction of new tools and technologies, such as sophisticated irrigation systems or modern agricultural machinery, has significantly increased productivity and the ability to feed a growing population. However, plant detection in large areas remains a challenging task for farmers, particularly when dealing with small plants such as strawberry plants. This issue has prompted numerous studies aimed at finding solutions, including research on plant segmentation [7], fruit counting [8], and other related approaches [9].

In red fruit industries, such as the tomato industry [10,11], the integration of robotics with machine learning algorithms enables a more precise, efficient, and sustainable approach, assisting farmers in making informed decisions to enhance yields, optimize the use of agricultural inputs, and reduce their environmental impact [12]. These human-like, perceptive, high-capability machines have shown great potential, especially when dealing with detection tasks. Malik et al. [10] introduced an advanced yet easy-to-embed algorithm for detecting ripe tomatoes, utilizing enhancements in the HSV (Hue, Saturation, Value) color space and watershed segmentation. This approach was able to reach an accuracy detection of up to 81.6% and was designed to be used for all red fruits. However, these methods, despite exhibiting acceptable performance for certain data acquisition conditions, do not generalize effectively. Hence, recent studies have concentrated on exploring universal feature extraction methods to address the constraints faced by traditional image detection algorithms.

To the best of our knowledge, Sa et al. [13] pioneered the application of deep learning networks for fruit detection. These techniques are known for their capability to learn complex features autonomously and can be used in the development of an end-to-end non-linear model with high precision for detection and other applications. An example of such a model was designed by Lamb et al. [14], who used a convolutional neural network for strawberry fruit detection. Their network was based on three sets of convolutional layers (classification, detection, and prediction) capable of being embedded in Raspberry Pi 3B. The detection speed achieved by this model was 1.63 frames per second, with an average precision (AP) of 84.2%. Another example was introduced by Yu et al. [15], who employed the widely recognized Mask R-CNN framework [16] with a ResNet 50 backbone [17] combined with a feature pyramid network (FPN) [18] for feature extraction to attain an AP of 95.78% for strawberry fruits in a non-structured environment. This method covers overlapping and hidden fruits under varying illumination intensities. Moreover, the processing time for images of 640 × 480 pixels was just 0.13 s when using a high-performance computer. However, these multi-stage detection methods demand substantial resources for region proposal selection, thereby posing limitations on the detection speed. Consequently, they are not suitable for real-time field detection applications.

Delving further into the realm of real-time detection, Redmon et al. introduced the YOLO (You Only Look Once) architecture [19]. Unlike traditional methods that predict a proposal and subsequently refine it, YOLO directly predicts the final detection by dividing the image into grids and performing a prediction for each grid cell. This makes the YOLO network the most representative network with real-time capabilities [20]. Currently, there are 10 versions that have improved upon the performance of the original YOLO. YOLOv3 was the first YOLO model to be widely adopted in real-time applications. It demonstrates remarkable results in detecting the smallest objects, thanks to its deep architecture, which incorporates three size prediction layers. Prico et al. [21] proposed a real-time weed detection system for green onion crops using YOLOv3. The system was able to reach an AP of 93.81% and an F1 score of 0.94 on a five-minute UAV video with a resolution of 864 × 688 using a high-performance computer. In another study, Liu et al. [22] used a circular bounding box with the YOLOv3 model. This resulted in a mean AP of 0.96 when using an image of 3648 × 2056 pixels at a speed detection rate of 0.054 s.

Regarding strawberry-related applications, He et al. [23] applied the YOLOv4 network for strawberry maturity detection and localization for robotic harvesting in RGB images. Their approach employed a two-stage deep learning methodology utilizing YOLOv4 and YOLOv4-tiny models to identify mature strawberries and estimate their centers from both RGB and depth images. They reached an AP of 91.73% with a processing speed of 55.19 ms per image for maturity detection and a mean AP of 86.45% with a 4.16 ms processing time per image for strawberry localization. Xie et al. [24] proposed a method for the rapid and accurate identification of strawberry fruit in greenhouses using an improved YOLOv5s approach. Their method, called YOLO-SR (YOLO–Strawberry Ripeness), replaced the C3 module with a lightweight multi-path aggregation network based on Channel Shuffle, enhancing the feature extraction capability. This method achieved a mAP of 94.3% and an F1 score of 93.7%. However, running these algorithms in a practical setting using small robots is very difficult because the YOLO network needs a strong GPU (Graphics Processing Unit) with over 4 Gigabytes of memory, which most electronic boards cannot handle [20].

As a solution, researchers have developed a lightweight version of the YOLO network, introducing the term YOLO-tiny. In their study, Zhang et al. [25] designed and implemented a lightweight deep neural network (RTSD-Net) for real-time strawberry detection on embedded devices, such as Jetson Nano. By reducing the number of convolutional layers and the cross-stage partial network of a YOLOv4-tiny architecture, and by using the TensorRT method to speed up the model, they achieved a mean AP of 82.44% on an aerial strawberry dataset, and the processing speed was 25.2 frames per second (FPS). These results were achieved using a dataset comprising only three types of objects, with a uniform image background, which may not be representative of all types of scenarios. Additionally, the network architecture of the YOLO-tiny model has fewer layers and parameters, allowing for quicker performance but increasing the risk of information loss in complex environments [26]. To the authors’ best knowledge, there have been no previous studies on the use of YOLO models for strawberry plant detection in complex environments for pesticide spraying application. Consequently, this paper explores the possibility of using the YOLO model for detecting strawberry plants in such types of applications.

The pesticide-spraying process is a complex task that requires instant decision-making by the robot during a survey [27], in contrast to applications such as yield prediction [28] or disease detection [29], which can be performed separately using a high-performance computer. For that, limited choices on the suitable detection model are available. This study highlighted the strengths and weaknesses of YOLO versions that are already being implemented in robotic platforms for instant decision-making, offering a data-driven basis for selecting the most suitable model for pesticide spraying robots.

For embedded systems with limited computational resources, YOLOv3 is the most suitable YOLO version in terms of memory and computational requirements compared with the later versions. Some studies, such as [11,20,22,30], used embedded YOLO and affirmed a good balance between the speed and accuracy of YOLOv3 for real-time applications, even if newer versions existed. Based on that, this study aims to develop a real-time plant detection algorithm that can be implemented in an agricultural pesticide-spraying robot using an NVIDIA Jetson TX2 GPU. The developed method, using YOLOv3, was specifically applied to strawberry plants within a greenhouse environment. To assess the effectiveness of our implementation, we used the Open Neural Network Exchange representation (ONNX). In order to show the performance of our proposal, a set of real-world experiments was carried out in a didactical strawberry greenhouse, which revealed that the proposed model can reach a mAP of over 97% with a processing time of 15 ms per frame.

This paper is organized as follows: Section 2 presents a brief discussion of the robot and YOLOv3 model that is used in our application. The main results are detailed in Section 3, and a section on the main conclusions ends this paper.

2. Materials and Methods

2.1. AgriEco Robot

The mobile robot “AgriEco Robot” was developed in our laboratory to be used in a didactical strawberry greenhouse [31]. Its primary objective is to control the pesticide-spraying process, encompassing aspects such as quantity regulation and ensuring precise localized spraying. It consists of a robotic chassis 550 × 650 mm in size made of steel, equipped with four-wheel drive. The spraying system is composed of a 10 L bank of polythene, a submerged pump, and two motorized arms. A navigation system, using Hokuyo URG-04LX-UG01 2D Lidar, from HOKUYO AUTOMATIC CO LTD in Osaka, Japan [32], is installed in the robot’s front section. The robot has a vision system using 2 types of cameras, a ZED camera stereo and omnidirectional camera. See Figure 1.

The robot utilizes the open-source framework Robot Operating System (ROS) [33], which combines CPU and GPU electronic components to execute its diverse range of tasks. For simple tasks like controlling the arms, the pumps, and the wheels, the robot uses Raspberry Pi 4. For deep learning processing tasks, it uses a built-in NVIDIA Jetson TX2 cart from NVIDIA company in the USA [34], equipped with a Quad-core 2.0 Ghz 64-bit ARMv8 A57, a dual-core 2.0 Ghz ARMv8 Denver, a 256 CUDA core 1.3 MHz Nvidia Pascal, and 8 GB memory. To power the robot, a 36 V/30 Ah lithium-ion battery is used. These configurations make it well-suited as an embedded system for this study application.

2.2. Deep Learning Model

While YOLO models themselves are well-established, this work innovatively integrates these models into an already existing pesticide-spraying robot designed for the complex greenhouse environment. This integration is non-trivial and requires addressing unique challenges related to real-time object detection, decision-making, and precise actuation in dynamic greenhouse settings. These challenges include the robot’s autonomous navigation system, the plants’ orchard environment, and pesticide actuation control. Additionally, several other challenges must be considered because of their direct impact on the proposed application, such as the lack of a sufficient training dataset, the choice of the loss function to improve performance, and the implementation using a single GPU NVIDIA Jetson TX2 board. These constraints limited our choices to versions that are stable, easily embedded, and exhibit excellent performance in instant decision-making.

Nowadays, there are 10 versions of YOLO, and the newest versions show a fast yet accurate performance, making them highly suitable for real-time applications [35]. However, earlier versions of YOLO still offer distinct advantages, such as simplicity and resource efficiency, that make them better candidates in applications with limited computational resources such as edge devices [36]. To select the most suitable YOLO version for this study, a comparative analysis was performed by comparing the most-used version in the embedded system (3, 4-tiny, 5-small, 5-medium) using NVIDIA Jetson TX2. This comparison took into consideration the model’s performance ins term of the number of parameters, processing speed, and mAP on the COCO dataset [37].

Table 1. Comparison between most used YOLO version in embedded system with NVIDIA Jetson TX2.

YOLO Version	Parameters	Processing Speed (FPS)	mAP (COCO)
YOLOv3 [38]	61.5 million	~10–15 FPS	57.9%
YOLOv4-tiny [39]	6 million	~25–30 FPS	36.9%
YOLOv5-Small [40]	7.2 million	~20–25 FPS	39.4%
YOLOv5-medium [40]	21.2 million	~15–20 FPS	45.4%

summarizes the overall performance.

The values in Table 1 suggest that YOLOv5-medium provides the best balance of performance between the processing speed and mAP. In contrast, YOLOv3 demonstrates the best mAP compared with the other versions, but it has the lowest frames per second (FPS), which is expected given its high number of parameters. Based on this analysis, YOLOv5-medium can be considered optimal for applications where moderate accuracy and moderate speed are acceptable, while YOLOv3 is more suitable for applications that require high accuracy with an accepted slower processing speed, which is the case in this study.

Pesticide spraying is a stationary and critical process, where the robot navigates slowly between the strawberry rows to maintain stability and ensure a straight navigation line. Also, localized spraying is crucial to avoid harming the plants. This requires the robot to come to a full stop before adjusting the spraying nozzle to avoid the maximum surface of the plant. Based on that, YOLOv3 was chosen in this study since the spraying process is critical and demands high precision, which requires the highest mAP, unlike the processing speed, which is less critical because of the robot’s slow movement.

The YOLOv3 [38] architecture, an automatic, robust, real-time, deep learning model, was chosen to perform real-time detection of the strawberry crop in a greenhouse. YOLO stands for (You Only Look Once), and it is a family of models used for real-time object detection. As suggested by its name, YOLO refers to a single network that localizes and identifies objects in a single pass, making it rather efficient. It has been applied in many agricultural applications, e.g., [41,42,43]. The third version YOLOv3, which is recommended for small objects [38], combines 2 key contributions from 2 different architectures as follows: the skip connections from the ResNet architecture [17] and 3 prediction stages from Feature Pyramid Networks (FPNs) [18], resulting in the architecture shown in Figure 2.

Compared with vanilla feedforward neural networks, the ResNet architecture uses skip connections [45] between the layers. Consequently, in the backpropagation, the weight updates are based on eliminating $F\left(x\right)$ in order to obtain $y=x$. This makes the ResNet architecture state-of-the-art in many deep learning models for segmentation tasks [46], image denoising [47], and object detection.

The YOLOv3 algorithm uses a multi-scale feature extraction network named DarkNet53, which originally had 53 layers pretrained on the ImageNet dataset. For the detection task, an additional 53 layers are stacked on top of it, giving a fully convolutional underlying architecture of 106 layers for YOLOv3 (Figure 3). Also, it consists of three prediction stages, each followed by an up-sampling layer to increase the size of the feature maps. This enables the model to detect objects of various sizes effectively [48]. Initially, predictions are made on smaller feature maps to detect large-scale objects. Subsequently, predictions are made on medium-sized feature maps to identify medium-sized objects, and finally, predictions on the largest feature maps focus on detecting small objects. By combining these predictions into a single output, the model generates an image with bounding boxes delineating different objects.

One other characteristic of the YOLOv3 model is the employment of intersection over union (IOU) [49]. This metric is frequently used in object detection benchmarks. YOLOv3 uses this metric by determining the error between the predicted bounding box and the ground truth. These bounding boxes are determined by five predictions. The center of the bounding box coordinates is labeled $x$ and $y$, the dimensions of the bounding box are labeled $h$ and $w$, and finally, the class probability is determined. The probability value is equal to 1 if a grid cell has a portion of the ground truth bounding box, and 0 if not. Figure 4 illustrates the detection process of the YOLO models. The first step involves taking the image grids (Figure 4a) and predicting the class probabilities map, distinguishing the object from the background (Figure 4b). Subsequently, the model predicts the bounding boxes (Figure 4c), which collectively form the predicted bounding box surrounding the object. In this instance, the object refers to the plants (Figure 4d).

3. Proposed Method

The main idea is to train a model based on the YOLOv3 architecture for strawberry real-time detection. The model is then implemented in a robotic platform via the NVIDIA Jetson TX2 card using ONNX representation to accelerate detection.

3.1. Data Acquisition

The dataset used to train the proposed model was locally captured from a strawberry greenhouse using a mobile phone camera with a resolution of 3472 × 3472 pixels, capturing various plant sizes in different angles and lighting conditions. The greenhouse is considered small with an area of 36 m² containing four rows with five plants each Figure 5. The images were captured from an overhead perspective to simulate the robot’s visual system. Furthermore, the following distinct times of day were considered: morning, noon, and afternoon.

Because of the limited availability of the environment, a total of 150 images were captured, focusing on the intricate orchard environment of strawberry plants within the greenhouse, as shown in Figure 6. Each image presents a complex backdrop, featuring various objects that pose challenges to training a model effectively. The diverse assortment of objects within the background adds layers of complexity to the task. Moreover, the small size of the dataset used makes the training process for object detection more complex and harder to achieve. To address this issue, a data augmentation step was employed to improve the training data by applying a series of transformations such as rotation, translation, and color enhancement [50].

3.2. Data Augmentation

To train a YOLO model for object detection, a large dataset is required as it improves the model’s precision [51]. In this study case, only 150 images were taken because of the small environment in which the experiment was performed. Furthermore, to achieve accurate detection, this study accounted for various types of interference that could arise during plant detection, as shown in Figure 6; however, this complicated background can cause an overfitting to the model [50]. To address this issue, the data augmentation method was used to expand the training dataset. Methods such as adjusting color brightness, rotating images, and vertical–horizontal translations have the potential to enhance the performance of detection [52,53]. In this study, by using a Python script, transformations such as image rotation, color adjustment, and spatial translations were considered to expand the data.

The dataset was augmented to reach a total of 556 images, where 400 images were considered as the training dataset and 156 images as the validation dataset. The annotations used to train and test the network were generated through manual labeling using [54], which involved marking the plants with rectangular bounding boxes in all the images of the training and testing datasets.

3.3. Data Pre-Processing

The image size was modified from the original size to 512 × 512 pixels, which represents the maximum feasible image size to train the model using the computer hardware utilized in this study. To create annotations, we use MakeSense [54], a free-to-use online tool, to label the areas of each plant by drawing a bounding box around it and then exporting it as an xml file in PASCAL-VOC format. A total of 556 labels were exported, including information about the image shape along with the object type followed by the plant bounding box coordinates.

3.4. Running Environment

The main hardware platform used to train this model was a desktop computer equipped with an Intel i7 8th Gen processor, 16 GB of DDR4 2666 MHz memory, and Nvidia GeForce GTX 1070 with 8 GB VRAM. We used Python 3.8.8, TensorFlow 2.3, the CUDA 10.1 parallel computing framework with the CUDNN 7.6 deep neural network acceleration library, and OpenCV computer vision 4.0.0., as detailed in

Table 2. Machine used for training.

CPU	Intel® i7 8th Gen
GPU	NVidia GTX 1070
Frequency	3.2 GHz
RAM	16 GB
Storage	500 Gb SSD
Operating system	Windows 10
Programming platform	Python 3.8

.

3.5. Model Training

As discussed in the previous section, our model is based on the YOLOv3 architecture. A total of over 556 images were randomly distributed to the training dataset, which received 400 images, and 156 images were distributed to the validation dataset. A batch size of four and fixed momentum value of 0.9, as well as a learning rate of 0.0001, were considered. The training process was repeated 10 times independently, where the trainable weights were initialized. To track the best model for each execution, an early stopping method was applied when the loss on the validation set was not improved for five consecutive epochs. The validation loss results over the 10-time training process is summarized in

Table 3. The validation loss results over 10 independent executions.

Training process	1	2	3	4	5	6	7	8	9	10
Best validation loss	8.75	8.43	8.69	8.40	8.33	8.00	7.75	8.25	8.10	7.94
Epoch	13	17	10	12	8	13	7	15	11	9

. These results indicate that, on average, it took 10 to 11 epochs for the model to reach stability. Each process took over 12 h to complete 800 iterations across the 10 epochs, totaling 8000 iterations (

Table 4. Model training hyperparameters.

Iterations	8000
Batch size	4
Learning rate	0.0001
Epochs	10
Confidence threshold	0.5

).

To overcome the limitation of the training dataset and obtain more accurate outcomes, a transfer learning approach was employed to train the YOLOv3 model. It offers the benefit of enabling a network trained on a limited ground-truth dataset to achieve significant accuracy in detection [20]. Trained weights from ImageNet for YOLOv3 darknet [33] were transferred to the model.

During training, the model produced an output in every iteration and validation loss was minimized. The evaluation metrics used in this study are mean average precision (mAP), recall, and F1-score, defined by Equations (1), (2), and (3), respectively. The model with the highest precision among the 10 executions was chosen as the optimal model.

$$mAP={\displaystyle \frac{1}{N}}{\sum }_{i=1}^{N}A{P}_i=AP$$

(1)

where $N$ is the number of classes and ${AP}_i$ is the average precision for class $i$. In this study case, $N=1,$ which implies $mAP=AP$.

$$\mathrm{R}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}={\displaystyle \frac{\mathrm{T}\mathrm{P}}{\mathrm{T}\mathrm{P}+\mathrm{F}\mathrm{N}}}$$

(2)

$$\mathrm{F}1=2\times {\displaystyle \frac{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}\times \mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}{\mathrm{p}\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{i}\mathrm{s}\mathrm{i}\mathrm{o}\mathrm{n}+\mathrm{r}\mathrm{e}\mathrm{c}\mathrm{a}\mathrm{l}\mathrm{l}}}$$

(3)

The loss function used in this study is given in (4), where loss₁, loss₂, and loss₃ are the bounding box regression loss function, Smooth L1 loss, and classification loss, respectively.

$$loss={loss}_1+{loss}_2+{loss}_3$$

(4)

For loss₁, we utilize intersection over union (IoU) based on the provided annotations. These annotations include $(x_1^g,y_1^g) \mathrm{a}\mathrm{n}\mathrm{d} (x_2^g,y_2^g)$ for representing the top-left and bottom-right coordinates of the ground truth bounding box, respectively. In the prediction, we have $x_1^p, y_1^p \mathrm{a}\mathrm{n}\mathrm{d} (x_2^p,y_2^p)$, and Equation (5) is used to calculate the IOU as follows:

$${loss}_1=IoU={\displaystyle \frac{{Area}_{intersection}}{{Area}_{union}}}$$

(5)

where

$${Area}_{intersection}=\mathit{max}\left(0,\mathit{min}\left(x_2^p,x_2^g\right)-\mathit{max}\left(x_1^p,x_1^g\right)\right)\times \mathit{max}\left(0,\mathit{min}\left(y_2^p,y_2^g\right)-\mathit{max}\left(y_1^p,y_1^g\right)\right),$$

and

$${Area}_{union}={Area}_{predicted}+{Area}_{ground-truth}-{Area}_{intersection}$$

Smooth L1 loss is given in (6), where x represents the difference between the predicted and the target bounding box coordinates. The reason behind the employment of this loss function pertains to [55], where it is affirmed that Smooth L1 loss penalizes large errors more gently compared with traditional MSE loss, which makes it suitable for object detection tasks.

$${loss}_2=\mathrm{S}\mathrm{m}\mathrm{o}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{L}1\left(x\right)=\left\{\begin{array}{c}0.5\ast x^2 if \left|x\right|<1\\ \left|x\right|-0.5 otherwise\end{array}\right.$$

(6)

This study represents a binary classification problem, so the binary cross-entropy loss function shown in (7) is used.

$${{loss}_3=L}_{binary\_crossentropy}=-(y_{true}\log \left(y_{pred}\right)+\left(1-y_{true}\right)\log \left(1-y_{pred}\right))$$

(7)

where

$y_{true}$ is the true class label (0 or 1).

$y_{pred}$ is the predicted probability of the positive class (class 1).

4. Results and Discussion

The performance of the best model extracted from the 10 different executions on the validation set are listed in

Table 5. YOLOv3 and YOLOv3-tiny validation metric summary on the strawberry dataset.

Model	Precision	Recall	F1 Score
YOLOv3	0.73	0.95	0.83
YOLOv3-tiny	0.60	0.71	0.65

, indicating a precision of 0.73 and a recall of 0.95, leading to an F1-score of 0.83 for the YOLOv3 model. YOLOv3-tiny showed minor results, with a precision of 0.60 and a recall of 0.71, leading to an F1-score of 0.65. These results are expected because of the small training dataset along with the reduced number of layers in the tiny version. An F1-score of 0.83 indicates that YOLOv3 has a good balance between the precision and recall for this dataset, showing its efficiency in identifying true positives while minimizing false positives.

Based on that, real-world experiments were conducted using the strawberry plants in the greenhouse to verify the model’s generalization performance. First, we performed an in-lab test using the tools shown in Figure 7, including the NVIDIA Jetson TX2 card, a web camera with a resolution of $640\times 480$, a monitor for visualization, a keyboard, and a mouse. To assess the effectiveness of our implementation, the ONNX representation was used. To assess the accuracy, suitability, and consistency of the proposed model, three alternative contemporary scenarios were trained and tested on the same datasets. These scenarios included traditional YOLOv3, YOLOv3-tiny, and YOLOv3-tiny with ONNX. The experiment was performed using two types of data, i.e., video and live video. We encountered certain challenges with the robot, as it is still in the development stage. So, we conducted a simulation utilizing the tools illustrated in Figure 7, positioning the camera at an optimal altitude to cover two rows of the plantation.

Table 6. Model speed and accuracy with and without ONNX representation.

		Model
		YOLOv3		YOLOv3 with ONNX		YOLOv3-Tiny		YOLOv3-Tiny with ONNX
Data	Resolution	mAP	FPS	mAP	FPS	mAP	FPS	mAP	FPS
Video	1920 × 1080	93.5%	3	96.3%	6	89.5%	5	90.7%	8
Live video	640 × 480	90.4%	4	94%	8	86.1%	7	88.3%	10

summarizes numerical evidence on the performance of the model compared to the other scenarios. The model achieves an accuracy of more than 97% with a processing time of over 15 milliseconds per frame.

The results shown in Table 6 demonstrate real-time plant detection with mAP above 96%, which reveals the effectiveness and utility of the proposed model. Moreover, it showcases superior performance compared with the other three scenarios for both types of data. However, when considering processing speed, YOLOv3-tiny exhibits potential for achieving faster speeds because of its optimal balance between detection speed and accuracy, particularly when utilizing the ONNX representation. Nevertheless, in applications such as robotic pesticide spraying, detection accuracy holds greater significance, as the robot must ensure precise spraying while avoiding substantial areas of the plants. Additionally, YOLOv3-tiny versions are characterized by their compact networks and fewer prediction stages, which may risk information loss on objects of various sizes, given the varying sizes of plants. This issue is particularly significant in our case, where we have a limited dataset to train the model. Furthermore, when assessing robot navigation speed, a difference of 2 FPS may not be as critical as an approximate 6% increase in accuracy. It is important to acknowledge that these results were obtained by running the models using the NVIDIA Jetson TX2 board and not a computer. Figure 8 shows some online real-time detection in the greenhouse environment covering different day illumination conditions in the morning and afternoon.

It is also noticeable that the mAP is affected by the type of data used, possibly because of variations in acquisition resolution. As illustrated in Table 6, testing the model on higher-resolution videos yields superior results across all scenarios. This improvement could also be attributed to the stability of the acquisition process; for instance, the tested video exhibits less vibration compared with live video feeds. Furthermore, the quality of the training data may contribute to these outcomes. However, greater flexibility and diversity in factors such as angle, lighting conditions, focal length, sensitivity, and exposure duration during image capture result in a larger volume of collected data. Moreover, employing a variety of data augmentation techniques enhances the effectiveness of model training, leading to higher detection accuracy. In summary, the results of this study indicate that YOLOv3 with ONNX representation notably enhances detection accuracy. Additionally, it significantly boosts the image processing speed, which makes it suitable for implementation in electronic boards, such as the one used in this study.

5. Conclusions

This paper presented a deep learning model for strawberry plant real-time detection embedded in a pesticide-spraying robotic platform for greenhouse applications. Comparative studies on YOLO versions for embedded systems were conducted, leading to the selection of YOLOv3 as the suitable model for this application. To train the model, a dataset was locally captured covering different illumination conditions in the greenhouse environment. To perform the detection and ensure a better and more effective implementation, the ONNX representation was needed. The model shows high accuracies, above 97%, which affirm its utility in plant detection tasks. However, it is important to acknowledge that the model has certain limitations, as the accuracies may not consistently exceed 97%. We attribute this to potential issues related to the lack of training data and the quality of annotations. As an extension of this study, a tracking system based on a machine learning model will be added so the robot can track the plants accurately, thereby facilitating tasks such as plant counting and other applications.