1. Introduction
The agricultural sector is of high importance to the worldwide economy as the rising global population creates a constantly increasing demand for food [
1]. Plant diseases present a serious threat to the global agriculture industry, causing significant production losses and deterioration of the quality of the final product quality [
2]. The emergence of diseases that affect plants has been more likely during the last decades due to the rapid increase in international produce exchanges and the general climate change (like global warming) [
3,
4,
5,
6,
7]. The transmission of most emerging diseases is mainly due to biological invasions by plant pathogens, such as viruses, fungi, and bacteria [
8,
9]. Grapevine (
Vitis vinifera subsp.
Sativa L.) is a crop that is especially threatened by this kind of disease transmission throughout Europe [
10].
Grapevine is an economically important plant that is cultivated mainly in the Northern hemisphere for grapes, either for consumption or for fermentation to produce wine. Its cultivation is hindered by a number of diseases worldwide, most important Powdery Mildew (induced by
Uncinulanecator (Schw.) Burr.), Downy Mildew (induced by
Plasmoparaviticola), Esca (induced by
Phaeomoniella clamidospora and
Phaeoacremonium aleophilum), Black rot (induced by
Guignardia bidwellii), and Botrytis (induced by
Botrytis Cinerea) [
11]. These diseases can affect the plant leaves or the trunk, or they can directly affect the grapes. They have the potential to cause significant damage, resulting in huge yield losses or severely inferior product quality [
12].
Traditional methods to determine the pathogen infecting the grapevine rely on visual inspection by trained experts and, for some pathogens, further evaluation by laboratory techniques may be necessary. The aforementioned techniques are destructive, time consuming, and, in the case of the visual inspection, may not always be accurate [
13]. Moreover, infections are commonly addressed with the use of chemical compounds for preventive reasons or curable cases, or with destruction of crops to avoid further spread. This method, however, results in significant losses in crop yield, decrease in farmer’s income, and ecological contamination [
14,
15,
16], without ensuring fewer losses [
17].
In recent years, phytopathology and crop protection have been enhanced with the use of technologies such as robotics, sensing technologies, and artificial intelligence. These novel agro-technology methods belong to the field of precision agriculture (PA) and have helped minimise agricultural waste and maximise productivity [
18,
19,
20,
21]. To detect grapevine diseases, several studies have employed spectral sensing techniques, such as remote sensing that uses a multispectral camera on unmanned aerial vehicles (UAVs) [
22,
23], proximal sensing using thermal imaging [
24,
25] or hyperspectral sensing [
26,
27,
28]. Although the use of the aforementioned equipment gives accurate results even at a pre-symptomatic level, they must be used by experienced scientific personnel, and they are also expensive.
For this reason, and with the rise of deep learning (DL), the use of simpler input signals, such as RGB images, has drawn scientific attention to disease detection [
29,
30]. Deep learning is a subcategory of machine learning and, as the name suggests, it is a method that employs a neural network with a large number of layers [
31]. Thanks to convolutional neural networks (CNNs), DL has gained popularity in the field of computer vision, especially for classification problems [
32,
33,
34], and has become a popular approach for plant disease identification [
35,
36,
37,
38]. Specifically for the detection of grapevine diseases, DL has been used to detect downy mildew and spider mite; this has been completed using RGB photos in Gutiérrez et al. (2021) [
39] or esca in Alessandrini et al. (2021) [
40].
Transfer learning (TL) is a DL approach that combats problems of limited data by training a pre-trained model that has proven its efficiency in classification problem using pre-existing datasets, such as those from online libraries [
41]. In TL, the weights of those pre-trained models are kept and subsequently applied and partially updated to the new data [
41]. Then, the knowledge of the original DL model is transferred to a new but similar classification problem, even though it does not have the same feature space or distribution [
42]. The TL technique has been effectively used in the agricultural domain for disease detection in grapevines [
43], tomatoes [
44], wheat [
45], and apple leaf [
46]. After training, the models can be saved in a format that is readable by edge devices that can connect to a web services platform with the required sensors (cameras, complete with other sensors etc.) and be deployed for online validation, essentially giving farmers a way to precisely monitor the health status of their crops. This interconnection between the sensors in the field and the farmer’s computer or cell phone is called the Internet of Things (IoT) and has been successfully implemented in agriculture for smart irrigation [
47,
48], soil management [
49,
50], and pesticide management [
51,
52], among others [
53]. IoT is, in other words, the internet-based interconnection of embedded computing devices, which allows them to send and receive data. The problem of crop’s health status monitoring is usually tackled by employing computer vision techniques with IoT [
54], as in, for example, [
55,
56,
57].
The this research aims to develop an integrated IoT system through validating the performance of an edge device, like the Raspberry Pi (RPi), based on DL algorithms, for the online detection of five grapevines diseases, based on symptoms in the leaves, in field conditions. The validation of the integrated system was performed in the Chatzivariti winery vineyard, where only Esca was identified. The system’s results are geotagged using an onboard GPS sensor, then are reported through Amazon Web Services (AWS) cloud platform on a web-based service in order to provide an online disease alert.
4. Discussion
The objective of this research is to demonstrate an automated approach for disease detection in agriculture. The research focused on utilising pre-existing datasets from online libraries for the original training of DL models and apply a transfer learning approach for disease detection on real-life conditions in a winery estate. This approach was proven to be successful, with accuracy of more than 0.9 in both training and validation, which is in accordance with Ouhami et al. (2020) and Hasan et al. (2019), who have used a TL approach in their study for disease detection in tomato leaves [
74,
75]. They have also used pre-trained DL models (VGG16 and Inception v3 respectively), but these are not suitable for online deployment in edge devices given the performance requirements of the RPi. Aravind et al. (2020) have used smartphone images from four crops and succeeded with a validation accuracy of 0.9 on the online validation, using the VGG16 model [
76].
Image segmentation with Mobile-UNet successfully preserved the regions of interest, i.e., the plant leaves, while omitting most of the background noise. As it was mentioned, U-Net is a widely used model for semantic segmentation due to its unique architecture, but it requires higher computational power than the RPi can give. Indeed, modified versions of U-Net have been used for the segmentation of leaves diseases by Zhang and Zhang (2023) [
77] and Liu et al. (2022) [
78]. Their results are evaluated offline, however, in contrast with what has been proposed in this study. As of the time of the writing of this paper, the performance of the Mobile-UNet has not been evaluated in the agricultural sector. The use of a different segmentation method, such as the depth dimension for faster inference of the models, should be considered for a future study.
The DL models that have been employed in this study were the EfficientNet B0 and the MobileNet V2. Their high performance in DL tasks and their high efficiency in terms of computational power, due to their lightweight architecture, made them suitable choices for online deployment.
Table 6 shows that the obtained accuracies of 0.94 and 0.92 for MobileNet V2 and EfficientNet B0 respectively showcase the strong predictive capabilities of these models when applied to real-world data. These results are almost in accordance with what Bir et al. (2020) have found in their study on tomato leaves, where the two models have shown comparable results as well, but EfficientNet B0 outperformed MobileNet V2, by a very small amount (0.4%) [
79]. Regarding the validation performance of the DL models, it was in general slightly lower than in the training. This should not come as unexpected as the models were initially trained and validated on the same dataset (Eden Library) and the new data from the field may cause the models to not accurately detect all the features the same way they did with the training set.
Model evaluation in real-world conditions was achieved by importing the models with an RPi device that uses a camera module to capture RGB images from the video stream. Those images are processed by the DL models in the RPi and outputs the result, which, in turn, is transmitted through fastAPI in a web service interface, as is also shown by Sharma et al. (2022), who have developed a web-based service that takes tomato leaf images as an input and performs disease detection [
80]. In parallel, the RPi can be integrated in a robotic platform using ROS and, through ROS nodes, can receive a different input from an RGB camera topic, mounted on a UGV, and transmit the decision output to the developed web service along with various other messages from other sensors in the robotic platform, depending on the configuration. In this work, the ROS node was not tested online.
Certain limitations apply for this use case. While there are more powerful sensors, such as hyperspectral cameras [
81], that capture more information from the electromagnetic spectrum than RGB, these are usually commercial, licensed products. The same considerations apply for the software used to create the models. This makes the use of open-source software a necessity. For this reason, this research used python as a programming language for the algorithms and a Linux distribution as the RPi’s operating system to deploy the models.
While this research focused on grapevine plants and, specifically, diseases affecting their leaves, it is important to note that this approach should be primarily validated in the other disease classes on which the models have been trained, not only in Esca, and can also be extended to detect diseases in other types of crops. Moreover, research can be focused on different types of disease, affecting fruits of the plants or, in the case of grapevines, their grapes. Botrytis cinerea is a pathogen affecting more than 200 crops worldwide. In the case of grapevines, visual symptoms are observable on the grapes. Another possible approach is the detection of pests and insects that can allow for an even earlier warning about crop safety before the disease causes visible symptoms.
5. Conclusions
The present study focuses on an IoT approach to disease detection in grapevine plants in field conditions by implementing deep learning (DL) algorithms, trained under the transfer learning (TL) technique and using commercially available data sets.
Firstly, this study demonstrates the effective use of TL in adapting pretrained DL models to the context of grapevine disease detection. This approach strikes a balance between model efficiency and computational requirements, enabling the deployment of lightweight models on edge devices like the Raspberry Pi while maintaining high accuracy.
Additionally, the Mobile-UNet was successfully implemented for real-time segmentation of plants from their background, with a high intersection over union (IoU) accuracy and fast inference times, which play a crucial role in the generation of the final model’s decision.
Among the DL models considered, MobileNet has been identified as the optimal algorithm for disease detection in this study. It has showcased superior performance while necessitating lower power and memory resources. As a result, MobileNet enables real-time disease detection with the designated frame rate of the RGB camera, contributing to efficient and timely results.
This study goes beyond traditional disease detection by integrating the DL models into an IoT framework that can be further connected to robotic platforms, using ROS, for automated, real-time monitoring and response systems in agriculture.
Emphasizing the use of open-source software and commercially available datasets, this research addresses the practical constraints of agricultural technology deployment, making it accessible and replicable.
In conclusion, this research not only highlights the feasibility of employing IoT devices with modest power and memory requirements for the real-time detection of grapevine diseases under field conditions, it also introduces novel methodologies and integrations that have the potential to significantly advance disease monitoring and management in the grapevine industry and beyond.