Apple Grading Method Design and Implementation for Automatic Grader Based on Improved YOLOv5
Round 1
Reviewer 1 Report
The manuscript can be evaluated after addressing following comments/problems and 2nd review:
1. How the dataset is captured? It is not clear? The dataset images contain set of apples?
2. The grading process is not clear. How a single apple of different quality in a row of apples is separated?
3. The bounding box labels are not clear in the figures. How a specific apple to separate is located and then communicated to the final grading system?
4. The details of this embedded system is not clear. How different components are connected?
5. How the trained model is used in the machine and how it is processing images? The processor/micro-processor/etc. details are not given? The details are only given for training/developing models.
6. The results for different loss and activation functions should be provided and discussed.
7. The model training details should be well provided so that the readers can replicate this method.
8. The complete grader (embedded system/machine) is developed or just used? The authors should provide clear details and accordingly, the title, abstract, intro and other sections should be modified.
9. What are the number of images for different quality classes?
10. Include images of different quality apples
11. The manuscript title can be improved
Author Response
Many thanks for the valuable comments on the manuscript which are much appreciated. We have revised the paper according to these constructive comments. The detailed response to each comment is given below. The revised parts are all highlighted in red in revision paper. We wish our sincere efforts would satisfy the requirements.
- How the dataset is captured? It is not clear? The dataset images contain set of apples?
Response and modification: Thanks for your comments. The apple data set for this article was sourced from apple markets and Fengxian orchards. Most of the apple-grade1 and grade-2 was purchased from apple markets. Apple-grade 3 was mainly picked from orchards. Additional information on the source of the data set has been provided in this paper. (Please see page 5, lines148-151,155-158 for details in revision)
- The grading process is not clear. How a single apple of different quality in a row of apples is separated?
Response and modification: Thank you for your comments. The apple grading process is described additionally in section 3.2 of this paper in the system solution validation. After the apples have been transferred through t Feeding and material handling lifting mechanism, they enter the tumbling and inspection system one by one. When different grades in the same row of apples are identified (e.g. two grade 1 apples, two grade 2 apples). The automatic apple grading software will automatically output information about the position and grade of the apples, then graded actuators below receives the apples in order and automatically drops the first two apples at the primary channel and the last two apples at the secondary channel. (Please see page 15, lines 461-465 for details in revision)
- The bounding box labels are not clear in the figures. How a specific apple to separate is located and then communicated to the final grading system?
Response and modification: Thank you for your comments. The clear original figure has been reinserted into the article, and the experimental results have been reworked. (Please see page 12, lines 373-390 for details in revision). The automatic apple grading software designed in this paper uses Python-snap7 with a control system PLC to complete the information transfer (Ref. 35). Each picture taken will output information on the grade and location of the apples within the picture, which will be received by the grading actuator in order and dropped at the corresponding grade channel in turn. (Please see page 15, lines 461-465 for details)
- The details of this embedded system is not clear. How different components are connected?
Response and modification: Thank you for your comments. The hardware of the automatic apple grading control system includes a CCD industrial camera, IPC-610L industrial computer, PLC-1212 controller, AC contactor, inverter and AC motor. The CCD industrial camera uses a GigE interface for data transmission and acquisition with IPC-610L industrial computer. The industrial computer and PLC-1212 controller use the snap7 library to transmit information via a network cable. PLC controls AC motor to drive grading actuator through AC contactor. (Please see page 15, line 449-454 for details)
- How the trained model is used in the machine and how it is processing images? The processor/micro-processor/etc. details are not given? The details are only given for training/developing models.
Response and modification: Thank you for your comments. In this paper, the trained model is made into auto-grading software and installed on an IPC. After the auto-grading software has processed the apple grade information during operation, the information is transmitted via the snap7 library to TIA Portal V15.1(update a reference [37]), which performs the grading action on the PLC control system. Details such as IPC computers and processing images have been added to the system solution. (Please see page 15, lines 454-457,461-465 for details in revision)
- The results for different loss and activation functions should be provided and discussed.
Response and modification: Thank you for your comments. This paper adds comparative experiments on loss and activation functions, updates a reference [31], and analyzes the experimental results, making the comparative results more obvious. (Please see page 10，lines 337-357 for details in revision)
- The model training details should be well provided so that the readers can replicate this method.
Response and modification: Thank you for your comments. This article provides a more detailed description of the dataset training process and specific details, making the article easier to understand. (Please see page 10, lines 317-320 for detail in revision.)
- The complete grader (embedded system/machine) is developed or just used? The authors should provide clear details and accordingly, the title, abstract, intro and other sections should be modified.
Response and modification: Thank you for your comments. The automatic apple grader in this paper is in the self-development and testing stage. Experimental tests have shown that the developed automatic apple grader has high efficiency and grading accuracy. The conclusion section of this paper has been revised to give the reader a better understanding of what has been accomplished with the automatic grader. The other sections, such as the title, abstract and intro, have also been modified.
- What are the number of images for different quality classes?
Response and modification: Thank you for your comments. The dataset for this paper has been expanded to 6000 sheets, with more primary and secondary fruit accounting for 40% and 40% of the total respectively, and a smaller sample of tertiary fruit accounting for 20%. A more detailed description of the dataset is provided. (Please see page 5，lines180-187 for details in revision）
- Include images of different quality apples
Response and modification: Thank you for your comments. An update has been made to the experimental results image, which contains apples of different qualities. (Please see page 12, Figure 14 for more details)
- The manuscript title can be improved
Response and modification: Thanks for your comments. We have amended the title for “Apple Grading Method Design and Implementation for Automatic Grader Based on Improved YOLOv5”. (Please see title)
Reviewer 2 Report
The topic treated by the authors is interesting and the presented results are encouraging, However, there are some difficulties to understand the proposal and originality of the work. In addition, I would like to make following comments:
1. The literature review needs to be improved to describe completely the proposal of each work and, after that, to point out the gab of research and the novelty of their proposal.
For example, lines 41-54, the authors should discuss the limitations of the related work
2. The authors did not consider that in some cases the apple may has a defect or deformity at bottom and could be graded as grade 1 based on the features for the apple from top view. However, it should be grade 3. The authors should develop a mechanism that follow the apple from four direction while its moving in the conveyor.
3. The authors need to clearly discuss the motivation of using DIoU and SELayer to fill the gap of research and improving the fruit grading results.
4. In Line 170, what is the size of the original dataset?
5. In text of the manuscript, it is necessary to describe all symbols and acronyms before used such as
- You should define A,B,C in equation 4
- "SE module " in line 255, "SPP", "SELayer" in line 349, "SSD" in line 360, "mAP" in line 360
6. The authors need to make very clear the proposed method. For example, the process for determining the mAP, how to calculate, the authors did not present the detail process obtained.
7. The authors need to describe the numbers on apples in figure 6-b.
8. In line 176, the authors mentioned that YOLOV5 is a new algorithm, however it has been used in several research. I suggest removing the word "new"
9. In lines 306-309, the authors should be consistent in using terms such as PR curve or P-R curve
10. In Figure 13, The authors need to improve the accuracy of this figure. Labels are not visible even with zooming. As well, the authors mentioned that they used Yolov5s small in fig 13-b and the don't mention the improved Yolov5 is small as well or what in fig 13-c?
11. The authors should review numbering the tables and figures in context correctly, there are some typos as follows:
- In Line 366, replace Table 2 with Table 3
- In line 414, replace Table 3 with Table 4.
12. In line 350, the authors used term Im-YOLOv5, I suggest authors to use this term in all discussions for the improved YOLOV5. As well in Table 3 column1, last row it would be better if use Im-YOLOV5s.
13. In Table 4, column 5 and row 4 what is the value 82??
Author Response
Many thanks for the valuable comments on the manuscript which are much appreciated. We have revised the paper according to these constructive comments. The detailed response to each comment is given below. The revised parts are all highlighted in blue in revision paper. We wish our sincere efforts would satisfy the requirements.
- The literature review needs to be improved to describe completely the proposal of each work and, after that, to point out the gab of research and the novelty of their proposal.
For example, lines 41-54, the authors should discuss the limitations of the related work
Response and modification: Thank you for your comments. The literature review section of this paper has been revised to provide a more complete description of the current state of research in each work and to highlight the objectives of the paper's work for better understanding by the reader. (Please see page 2, lines 54-56, 71-73, 84-89 for detail in revision）
- The authors did not consider that in some cases the apple may has a defect or deformity at bottom and could be graded as grade 1 based on the features for the apple from top view. However, it should be grade 3. The authors should develop a mechanism that follow the apple from four direction while its moving in the conveyor.
Response and modification: Thank you for your comment. We have carefully studied your suggestions. The visual inspection system in this paper ensures that the full surface of the apple is obtained by taking three images at different moments in time. The grading algorithm will be optimised in the future to avoid misjudgements.
- The authors need to clearly discuss the motivation of using DIoU and SELayer to fill the gap of research and improving the fruit grading results.
Response and modification: Thank you for your comments. The motivation and advantages of using DIoU and SELayer have been added to this article in detail. (Please see page 8, lines 245-248 and page 9, lines 267-269 for details of the changes)
- In Line 170, what is the size of the original dataset?
Response and modification: Thank you for your comments. The images taken by the industrial cameras are of a uniform size of 1280*1024. This article gives more detail on the data set taken. (Please see page 6, lines 184 for details in revision)
- In text of the manuscript, it is necessary to describe all symbols and acronyms before used such as
- You should define A,B,C in equation 4
- "SE module " in line 255, "SPP", "SELayer" in line 349, "SSD" in line 360, "mAP" in line 360
Response and modification: Thank you for your comment. A, B,C in equation 4 were defined. The abbreviation of the relevant symbols has been indicated in the article to avoid ambiguity. (Please see page 8, lines 241-244, page 9, lines 275, page 10, lines 323-324, and page 13, lines 400-401, 412 for details of the changes)
- The authors need to make very clear the proposed method. For example, the process for determining the mAP, how to calculate, the authors did not present the detail process obtained.
Response and modification: Thank you for your comment. The formulae for calculating the relevant evaluation indicators have been added to the article. (Please see page 10, lines 322-334 for details of the changes)
- The authors need to describe the numbers on apples in figure 6-b.
Response and modification: Thank you for your comments. The numbers in Figure 6 are to determine the orientation of the camera shot. To avoid ambiguity, the image of the captured apple has been reinserted into Figure 6.
- In line 176, the authors mentioned that YOLOV5 is a new algorithm, however it has been used in several research. I suggest removing the word "new"。
Response and modification: Thank you for your comment. The relevant sentence has been amended. (See page 6, line 192 for details)
- In lines 306-309, the authors should be consistent in using terms such as PR curve or P-R curve
Response and modification: Thank you for your comment. We have checked the full text to ensure that the terminology has been aligned. (Please see page 11, line 361-362 for details)
- In Figure 13, The authors need to improve the accuracy of this figure. Labels are not visible even with zooming. As well, the authors mentioned that they used Yolov5s small in fig 13-b and the don't mention the improved Yolov5 is small as well or what in fig 13-c?
Response and modification: Thank you for your comments. The clear original figure has been reinserted into the article, and the experimental results have been re-stated to make them easier for the reader to understand. (Please see page 12, lines 373-390 for details).
- The authors should review numbering the tables and figures in context correctly, there are some typos as follows:
- In Line 366, replace Table 2 with Table 3
- In line 414, replace Table 3 with Table 4.
Response and modification: Thank you for your comment. The incorrect markings in the text have been reworked and the full text has been checked to ensure there are no similar errors. (See lines 419, 476 for more information)
- In line 350, the authors used term Im-YOLOv5, I suggest authors to use this term in all discussions for the improved YOLOV5. As well in Table 3 column1, last row it would be better if use Im-YOLOV5s.
Response and modification: Thank you for your comments. The terminology involved has been changed in this article, please see the revised version for details.
- In Table 4, column 5 and row 4 what is the value 82??
Response and modification: Thank you for your comment. The grading completion time is a range and is subject to statistical error. The exact figures have been revised. (Please see page 16, Table 4 for details)
Reviewer 3 Report
The paper introduces the design of the apple grading machine and the implementation of the algorithm, which has strong application value, but some issues deserve further clarification and discussion.
(1) In the abstract, the expression "apple industry" is not very appropriate.
(2) The Apple grading criteria is based "Red Fuji GB/T", while the apples used in this dataset are the "Fengxian apples" and the "Yantai apples", do these types of apples have common ripening and grading characteristics?
(3) From the apple pictures acquired by the device in the paper, it is obvious that the white balance is abnormal. Are the camera and light source calibrated with white balance? Does the inaccurate white balance affect the recognition results?
(4) The grading criteria include "Diameter ≥ 70mm", "Area not exceeding 4cm²" , "Fruit shape", etc. Do these metrics need to be labeled and trained in the dataset? What is the standard of apple shape?
(5) How to prove that the use of "Mish" activation function is Improvement?Are there any relevant experiments to support this assertion?
(6)The results of the apple grading are not shown very visually. It is recommended to add some visual images to prove the validity of the grading results.
Author Response
Many thanks for the valuable comments on the manuscript which are much appreciated. We have revised the paper according to these constructive comments. The detailed response to each comment is given below. The revised parts are all highlighted in green in revision paper. We wish our sincere efforts would satisfy the requirements.
- In the abstract, the expression "apple industry" is not very appropriate.
Response and modification: Thanks for your comments. We have looked at your suggestions and have revised some of the 'abstracts' so that the language of the study has been improved. (Please see page 1, lines 26 for details of the changes)
- The Apple grading criteria is based "Red Fuji GB/T", while the apples used in this dataset are the "Fengxian apples" and the "Yantai apples", do these types of apples have common ripening and grading characteristics?
Response and modification: Thank you for your comments. The "Fengxian" and "Yantai" apples used are all representative varieties of the Red Fuji apple. It's just that the origin is different. They have good fruit shape, high yield, and colour, and both have similar quality characteristics. This article gives more detail on the sources of the Apple dataset. (Please see page 5, lines148-151,155-158 for details in revision)
- From the apple pictures acquired by the device in the paper, it is obvious that the white balance is abnormal. Are the camera and light source calibrated with white balance? Does the inaccurate white balance affect the recognition results?
Response and modification: Thank you for your comments. The image capture equipment in this article can be pre-set with white balance parameters or allow for automatic adjustments when acquiring images, resulting in higher quality apple images being acquired. Inaccurate white balance can affect apple grading accuracy. The apple images in this paper were taken without the white balance parameters set at the time of capture and have been re-set with the white balance parameters set to 3.00. The resulting images have been modified in the text. (Please see page 5, Figure 6(b) for more details)
- The grading criteria include "Diameter ≥ 70mm", "Area not exceeding 4cm²", "Fruit shape", etc. Do these metrics need to be labeled and trained in the dataset? What is the standard of apple shape?
Response and modification: Thank you for your comment. The grading metrics broken area and size do not need to be labelled and trained in the dataset. As the height of the industrial camera is a fixed value, the longest side of the rectangular box calibrated in the dataset is used as the criterion for fruit diameter; the ratio of the long side to the short side of the rectangular box is used as the criterion for fruit shape; apples with poor ripeness and defects are not carefully classified and are judged to be grade-3 apples, the relevant training details have been updated in the text. (Please see page 6, lines 176-187 in revision for details)
- How to prove that the use of "Mish" activation function is Improvement? Are there any relevant experiments to support this assertion?
Response and modification: Thank you for your comments. This paper adds comparative experiments on loss and activation functions, updates a reference [31], and analyzes the experimental results, making the comparative results more obvious. (Please see page 11, lines 337-357 for detail in revision)
- The results of the apple grading are not shown very visually. It is recommended to add some visual images to prove the validity of the grading results.
Response and modification: Thank you for your comments. As the grading fruit cup of grading actuator (see figure 5) is not perfect. At same time, considering limited space for this article, the visual images results cannot be given. The statistical data of apple grading results are given in Table 4.
Round 2
Reviewer 1 Report
The comments are addressed in a well manner. This paper can be accepted in present form.
Reviewer 3 Report
The authors responded to my concerns and made the necessary changes in the paper.
