Plant Disease Identification Using Machine Learning Algorithms on Single-Board Computers in IoT Environments
Round 1
Reviewer 1 Report
Comments and Suggestions for AuthorsThe manuscript explores the application of Machine Learning (ML) algorithms, specifically a custom Convolutional Neural Network (CNN), for identifying plant diseases. The evaluation is conducted on various Single Board Computers (SBCs) in IoT environments. This review assesses the manuscript's methodology, coverage, and contributions. The topic is highly relevant, addressing the application of ML in agriculture and IoT environments. The focus on Single Board Computers, including mainstream choices like Raspberry Pi and specialized units like Google Coral Dev TPU Board, adds practical value. The manuscript employs a rigorous experimental methodology, training a custom CNN on Google Colab and evaluating it on different SBCs. The inclusion of various evaluation metrics, including hardware-related factors (CPU utilization, RAM usage, etc.), enhances the completeness of the study. The comparative analysis of different SBCs, each with distinct characteristics (Raspberry Pi, NVIDIA GPU, Google Coral Dev TPU), provides insights into the suitability of these devices for ML applications in plant disease identification.
Please find some comments for minor revision before acceptance:
While the manuscript mentions various evaluation metrics, providing detailed results, especially in tabular or graphical form, would enhance the reader's understanding of the performance of the custom CNN on different SBCs. Besides, it is suggested to introduce more relevant work on the deployment of ML models on SBCs, such as: https://doi.org/10.1007/s00170-022-10335-8.
Include a more comprehensive discussion of the findings, particularly insights gained from the evaluation metrics. Discuss how each SBC performs in terms of knowledge extraction time, power consumption, etc., and highlight any notable observations. Discuss any external factors that might influence the experimental results, such as variations in the training dataset or potential biases introduced during the training process.
Acknowledge and discuss any practical challenges faced during the experimentation, such as limitations in computational resources or challenges specific to deploying ML models on SBCs.
In conclusion, the manuscript offers a valuable exploration of ML applications in agriculture and IoT. The rigorous methodology, device comparison, and consideration of practical factors contribute to its significance. Addressing the suggested improvements, especially providing detailed results and a more in-depth discussion of findings, would enhance the manuscript's impact and readability. Overall, it is a commendable study with practical implications for the intersection of ML, agriculture, and IoT.
Comments on the Quality of English LanguageIn summary, the overall quality of English language is good.
Author Response
Overall comment: All changes in the revised manuscript are highlighted in green for the reviewers’ convenience.
Reviewer #1
Comment 1.1. The manuscript explores the application of Machine Learning (ML) algorithms, specifically a custom Convolutional Neural Network (CNN), for identifying plant diseases. The evaluation is conducted on various Single Board Computers (SBCs) in IoT environments. This review assesses the manuscript's methodology, coverage, and contributions. The topic is highly relevant, addressing the application of ML in agriculture and IoT environments. The focus on Single Board Computers, including mainstream choices like Raspberry Pi and specialized units like Google Coral Dev TPU Board, adds practical value. The manuscript employs a rigorous experimental methodology, training a custom CNN on Google Colab and evaluating it on different SBCs. The inclusion of various evaluation metrics, including hardware-related factors (CPU utilization, RAM usage, etc.), enhances the completeness of the study. The comparative analysis of different SBCs, each with distinct characteristics (Raspberry Pi, NVIDIA GPU, Google Coral Dev TPU), provides insights into the suitability of these devices for ML applications in plant disease identification.
Response 1.1. The authors would like to thank the Reviewer for their comment.
Comment 1.2. While the manuscript mentions various evaluation metrics, providing detailed results, especially in tabular or graphical form, would enhance the reader's understanding of the performance of the custom CNN on different SBCs. Besides, it is suggested to introduce more relevant work on the deployment of ML models on SBCs, such as: https://doi.org/10.1007/s00170-022-10335-8.
Response 1.2. To address this comment and to enhance the reader's understanding of the performance of the custom CNN on different SBCs extended additional results have been presented in Section 5 (part highlighted in green in section 5.1, including Table 1 and Figures 3, 8, 10, 12, 42 and 43 that present overall results in the form of bar charts).
For the proposed paper (https://doi.org/10.1007/s00170-022-10335-8), it has been referred to and elaborated upon at the end of sub-section 2.2.
Comment 1.3. Include a more comprehensive discussion of the findings, particularly insights gained from the evaluation metrics. Discuss how each SBC performs in terms of knowledge extraction time, power consumption, etc., and highlight any notable observations. Discuss any external factors that might influence the experimental results, such as variations in the training dataset or potential biases introduced during the training process.
Response 1.3. To address this comment, paragraph 2 in sub-section 5.1 has been added, as well as the newly introduced subsection 5.3 (both highlighted in green), which elaborate further on the experimental evaluation findings in the directions suggested by the reviewer.
Comment 1.4. Acknowledge and discuss any practical challenges faced during the experimentation, such as limitations in computational resources or challenges specific to deploying ML models on SBCs.
Response 1.4. To address this comment, subsection 5.2 has been introduced, elaborating on the limitations and challenges in the current work.
Comment_1.5. In conclusion, the manuscript offers a valuable exploration of ML applications in agriculture and IoT. The rigorous methodology, device comparison, and consideration of practical factors contribute to its significance. Addressing the suggested improvements, especially providing detailed results and a more in-depth discussion of findings, would enhance the manuscript's impact and readability. Overall, it is a commendable study with practical implications for the intersection of ML, agriculture, and IoT.
Response_1.5. To address this comment, paragraphs 1, 2 and 4 have been added in subsection 5.3 that elaborate on the overall experimental evaluation findings providing a more in-depth discussion of the obtained findings.
Reviewer 2 Report
Comments and Suggestions for AuthorsIn this manuscript, the authors present valuable insights into the application of ML in agriculture regarding the health of plants, especially in image classification.
Thank you for the opportunity to read this manuscript, and please allow me to make a few suggestions
I find the list of references to be a little short. As a point of reference for this study, I would advise the authors to include additional studies in the field from other authors.
What constitutes acceptable accuracy in the context of this study? The authors mention the accuracy at “about 90% “ but why this accuracy is chosen as a benchmark is missing.
Are there any alternative methods or models that might handle the issue of identical images (perhaps better?) ?
I suggest the authors to discuss a little bit more on the significance of the variations in processing times across the devices used, and perhaps on the alternative hardware configurations or optimizations for the slower devices.
Are there any practical consequences of high temperatures and varying power consumption for prolonged device use for real-world applications ?
I would also suggest the authors to extend a little bit the Conclusions section to include something about the generalizability of the proposed approach to other datasets or agricultural scenarios. Furthermore, I would suggest also in the Conclusions perhaps a more critical reflection on the limitations and potential biases of the study.
Author Response
Overall comment: All changes in the revised manuscript are highlighted in green for the reviewers’ convenience.
Comment 2.1. In this manuscript, the authors present valuable insights into the application of ML in agriculture regarding the health of plants, especially in image classification.
Response 2.1. The authors would like to thank the Reviewer for their comment.
Comment 2.2. Thank you for the opportunity to read this manuscript, and please allow me to make a few suggestions
Response 2.2. The authors would like to thank the Reviewer for their comment.
Comment 2.3. I find the list of references to be a little short. As a point of reference for this study, I would advise the authors to include additional studies in the field from other authors.
Response 2.3. To address this comment, the authors have significantly extended Section 2 presenting a review of the related literature. In this respect, the section is split in two subsections, the first one presenting the ML models used in the agricultural domain and the second one elaborating on executing CNNs in single board computers. Overall, more than one and a half page has been added in Section 2 and 4 new related references.
Comment 2.4. What constitutes acceptable accuracy in the context of this study? The authors mention the accuracy at “about 90% “but why this accuracy is chosen as a benchmark is missing.
Response 2.4. To clarify this matter, highlighted by the reviewer, the 4th paragraph of subsection 5.1 has been added that elaborates on the acceptable accuracy levels in related state of the art approaches.
Comment 2.5. Are there any alternative methods or models that might handle the issue of identical images (perhaps better?)?
Response 2.5. To address this comment, the following text has been introduced in 7th paragraph of subsection 5.1:
“Concerning the issue of identical images and the challenges this introduces, since the output of individual characteristics is difficult, an efficient solution can be the collection and usage of higher populations of images per class, which can allow for the efficient filtering of (near) identical images. The research presented in this paper uses 33 classes, which is significantly high for the scale of the experiments carried out and the respective hardware used. Nevertheless, as the population of images per class was not that high, the problem of almost identical images has occasionally been detected. For example, the Tomato Early Blight class included very similar images that led to accuracy level of 79%, which is significantly lower than the accepted threshold of 90% in the current work.”
Comment 2.6. I suggest the authors to discuss a little bit more on the significance of the variations in processing times across the devices used, and perhaps on the alternative hardware configurations or optimizations for the slower devices.
Response 2.6. To address this comment, the last two paragraphs of subsection 5.1 have been introduced that elaborates on the significance of the variations in processing times across the devices used, and perhaps on the alternative hardware configurations or optimizations for the slower devices, as well as Figures 42 and 43 and Table 1.
Comment 2.7. Are there any practical consequences of high temperatures and varying power consumption for prolonged device use for real-world applications?
Response 2.7. To address this comment, the authors have introduced the 3rd paragraph in subsection 5.3 that elaborates on the practical consequences of high temperatures and of the varying power consumption in case of prolonged device use for real-world applications.
Comment 2.8. I would also suggest the authors to extend a little bit the Conclusions section to include something about the generalizability of the proposed approach to other datasets or agricultural scenarios. Furthermore, I would suggest also in the Conclusions perhaps a more critical reflection on the limitations and potential biases of the study.
Response 2.8. To address this comment, the authors have introduced the following text in section 6:
“Finally, as concluded in the Related Work section, ML models targeting the agriculture domain or different fields that are executed on SBC with low power solutions, demonstrate several limitations. This is mainly due to the fact that CPU/TPU/GPU are much less powerful than Desktop PCs, especially when Cloud computing resources are used. This has significant consequences regarding the completion time of the inference task as it one can easily observe in the related graphs in Section 5. Another critical factor is the fact that low power hardware is escorted by limited RAM, as currently the RAM in SBCs ranges from 1GB to 4GB, thus introducing another constraint in the field targeted by this paper. In the hardware solutions used, when there is the proposed approach needs to operate in the presence of exclusive battery power supply, it is recommended to supplement the battery energy source with a small solar panel or a small wind generator, especially for the NVIDIA Jetson Nano which is the most energy hungry of the 4 SBCs studied. It should be mentioned that cooling solutions are recommended when the targeted hardware devices are used, especially during summer periods, which of course imposes an additional power consumption overhead corresponding to the operation of cooling fans for example. As it is clearly demonstrated by the graphs of Section 5, when the batch size is increased, the completion time is decreased, but more resources are required leading to the devices being stressed.”
