Characterization and Detection Classification of Moldy Corn Kernels Based on X-CT and Deep Learning

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Interesting article. The charts need to be corrected, some of them are illegible. It is necessary to check whether all unit records comply with MDPI requirements.

Author Response

Thank you for your valuable feedback on this paper. I have greatly benefited from your comments and have carefully revised each one. The specific details are as follows:

1.Interesting article.The charts need to be corrected, some of them are illegible. It is necessary to check whether all unit records comply with MDPI requirements.

  I have made changes and substitutions to the relevant unclear charts. Please review them. Thank you again for your valuable comments.

Reviewer 2 Report

Comments and Suggestions for Authors

The proposed study is a relatively detailed image analysis technology for Moldy Corn classification based on X-CT and Deep Learning, and is very interesting.

However, some improvements are required for subscribers.

1. Table 1 can be deleted. It would be nice to have additional technology for the SKYSCAN1275 X-ray tomography scanner.

2. Figures 5-8 need to be enlarged for subscribers. Additionally, the graph corresponding to the experimental results requires further explanation.

What are the incubation time units?

3. Figures 5, 6, 7 Expressing it as a Max-min error bar rather than standard deviation is more helpful in objectively evaluating the actual performance results.    And you can discuss error bar levels.

4. You can mention the economic value (processing time reduction, etc.) that can be obtained in the future through the proposed technology.

Author Response

Thank you for your valuable feedback on this paper. I have greatly benefited from your comments and have carefully revised each one. The specific details are as follows:

1. Table 1 can be deleted. It would be nice to have additional technology for the SKYSCAN1275 X-ray tomography scanner.

    I have now removed Diagram 1 of the equipment prepared for the mold experiment. Please refer to it. Thank you again for your valuable comments.            

Lines 129-130

2. Figures 5-8 need to be enlarged for subscribers. Additionally, the graph corresponding to the experimental results requires further explanation.What are the incubation time units?

    Figure 5-8 in the original paper has been modified and expanded. Due to structural adjustments in the article, the original Figure 5-8 now corresponds to Figure 7-9 in the revised paper. The unit of measurement for corn culture time is in days, and corn samples from four culture stages (0 days, 5 days, 10 days, and 15 days) were selected for analysis. Thank you again for your comments.       

3. Figures 5, 6, 7 Expressing it as a Max-min error bar rather than standard deviation is more helpful in objectively evaluating the actual performance results. And you can discuss error bar levels.

    Considering the significance of the graph, these three sets of charts are designed to illustrate the nonlinear trend of embryo, endosperm, and grain volume during the cultivation of corn grain mold. The revised charts clearly depict the changes, rendering the standard deviation chart unnecessary. The original Figure 5-8 now corresponds to Figure 7-9 in the revised paper.    Thank you again for your comments.     

4. You can mention the economic value (processing time reduction, etc.) that can be obtained in the future through the proposed technology.

    I have made a corresponding change to your comment in the conclusion of the article. Thank you again for your comments.               

 Lines400--403

Reviewer 3 Report

Comments and Suggestions for Authors

In the manuscript ‘Characterization and Detection Classification of Moldy Corn 2 Kernels Based on X-CT and Deep Learning’ Zhang et al. present the use of X-ray computed tomography (CT) to non-destructively analyze the internal structural changes in corn kernels as they become mouldy over time. The 3D visualization and models of the corn kernel tissues are constructed to show the progression of mould infection starting in the embryo. Irregular damage to the kernel structure is observed. The volumetric analysis shows the size of the kernel, embryo, endosperm, etc initially increases and then decreases as mould develops. A deep learning model (ResNet50) is trained to detect mould in 2D CT image slices with 93% accuracy. This enables mould severity classification.

 

The manuscript is overall interesting and well-organised. However, I believe that some points should be addressed. Please consider the following comments:

1.       What motivated the study of mouldy corn kernels with CT and deep learning? What applications or problems does this aim to address?

2.       How do the CT and modelling methods compare to other ways of analyzing internal grain structure like MRI? What are the limitations?

3.       On a similar note, I suggest the authors to add some reference on terahertz spectroscopy as it has recently opened a new route to a non-destructive way to image samples; e.g., Refs.[1-3]

4.       Were any surprises or new insights obtained from the 3D visualizations of mould progression?

5.       What causes the non-linear volume changes observed? How does mould metabolism impact the kernel tissues?

6.       How well does the deep learning model generalize to new corn samples? What are the key challenges for robust mould detection?

7.       Could a similar methodology be applied to detect defects or contaminants in other agricultural products?

References

[1] Stantchev, R.I., Mansfield, J.C., Edginton, R.S. et al. Subwavelength hyperspectral THz studies of articular cartilage. Sci Rep 8, 6924 (2018). https://doi.org/10.1038/s41598-018-25057-9

[2] Cecconi V, Kumar V, Pasquazi A et al. Nonlinear field-control of terahertz waves in random media for spatiotemporal focusing [version 3; peer review: 2 approved]. Open Res Europe 2023, 2:32 (https://doi.org/10.12688/openreseurope.14508.3)

[3] Luana Olivieri, et.al., ACS Photonics 2023 10 (6), 1726-1734, DOI: 10.1021/acsphotonics.2c01727

Comments on the Quality of English Language

I can understand the English. Minor typos sometimes. 

Author Response

Thank you for your valuable feedback on this paper. I have greatly benefited from your comments and have carefully revised each one. The specific details are as follows:

1. What motivated the study of mouldy corn kernels with CT and deep learning? What applications or problems does this aim to address?

      In response to your question, my answer is as follows: CT can accurately detect the internal structure of non-metallic materials. It can non-destructively identify the internal structure characteristics of corn grains and visualize the internal structure model in three dimensions. This allows for a more intuitive observation of the structural changes of corn kernel mold at each stage, making it a viable method for detecting corn kernel mildew. The two-dimensional grayscale image of a corn kernel obtained after CT scanning shows that when the kernel is mildewed, the chemical composition of its structure changes. This results in a reduced absorption of X-rays, leading to a decrease in gray value and a corresponding darkening of the two-dimensional slice image. Since mildew in corn kernels originates from the inside, a combination of CT scanning and deep learning is specifically employed to promptly and accurately detect mildew in corn kernels stored in grain silos.

2. How do the CT and modelling methods compare to other ways of analyzing internal grain structure like MRI? What are the limitations?

         In response to your question, my answer is as follows: CT: fast scanning, fast imaging, low body position requirements, can see the tissue structure of microscopic objects, disadvantages: radiation, lack of clarity of soft tissue imaging in medicine, and MRI: no radiation, high definition of soft tissue scanning, disadvantages: long scanning time, high price.

3. On a similar note, I suggest the authors to add some reference on terahertz spectroscopy as it has recently opened a new route to a non-destructive way to image samples; e.g., Refs.[1-3]

      On the question you raised, I am very helpful and have quoted articles related to terahertz technology.

4. Were any surprises or new insights obtained from the 3D visualizations of mould progression?

        In response to your question, my answer is as follows: In this study, the corn kernels cultured by mold were scanned by CT according to the culture time of 0 days, 5 days, 10 days and 15 days (identified as 0d, 5d, 10d and 15d in Figure 4), and the three-dimensional model of the kernel, embryo, endosperm and cracks of each corn kernel was reconstructed, as shown in Figure 4 in the article.

5.What causes the non-linear volume changes observed? How does mould metabolism impact the kernel tissues?

    In response to your question, my answer is as follows: First of all, the conditions required for corn kernel mold growth are suitable temperature and moisture. Corn kernel mold absorbs a significant amount of water during its hygroscopic stage. As the corn kernel absorbs water, its volume increases. During the metabolic stage of mold growth, it absorbs moisture and free oxygen atoms from the corn, consumes various chemical components of corn kernels, produces CO2 and metabolic waste, and causes a decrease in the volume of each tissue of the corn kernels.

6. How well does the deep learning model generalize to new corn samples? What are the key challenges for robust mould detection?

        In response to your question, my answer is as follows: the new corn sample CT scan image is used as the test set, and the ResNet50 model with enhanced deep learning is employed for testing, achieving high accuracy (93% test accuracy) and a high level of generalization. The main challenge to the robustness of mold detection is that when collecting the slice image of the corn kernel, the direction of the collected image and the presence of large cracks on the corn kernel itself will affect the accuracy of identification.                        

  Lines 316--319

7. Could a similar methodology be applied to detect defects or contaminants in other agricultural products?

       In response to your question, my answer is as follows: This method is not only limited to the detection of mildew in corn but can also be used for detecting weevils in corn kernels. Additionally, it can be applied to detecting defects in crops such as wheat and soybeans.

Reviewer 4 Report

Comments and Suggestions for Authors

The paper focuses on the impact of mold on corn kernels, particularly focusing on the detrimental effects of aflatoxin and gibberellin produced by moldy corn, which pose significant health risks if ingested. The research utilizes X-ray tomography technology alongside image processing and model reconstruction algorithms to non-destructively analyze and visualize the structural changes in mold-infected corn kernels over time, including the embryo, pores, cracks, endosperm, and seed coat. The study's findings highlight the pattern of mold infection spreading from the embryo and the irregular damage it causes to the corn kernel's tissue structure. Furthermore, it introduces an enhanced ResNet50 deep learning model.

1. Introduction:

The introduction provides a general overview of the problems associated with mold in corn but could benefit from more specific details regarding the prevalence, economic impact, and geographical relevance of these issues. Quantitative data or statistics could strengthen the case for the study's significance.

While the introduction mentions various traditional and innovative detection methods, it lacks a clear explanation of why these traditional methods are inadequate beyond general terms.

The introduction references several studies that have utilized X-ray tomography and deep learning in agricultural contexts. However, it does not critically discuss how these prior works relate to the current study's objectives. A brief critique or comparison could highlight the novelty or gap this study aims to address.

2. Materials and Methods:

More detailed explanations regarding the choice of specific conditions (e.g., moisture level to 20%, room temperature at 25°C) would enhance understanding.

The description of image data acquisition and processing steps is somewhat vague. Expanding on why specific preprocessing steps like disabling the smoothing function were chosen and how they affect the quality or interpretability of the resulting images could clarify the methodology.

The paper mentions the use of an interactive threshold segmentation algorithm but does not elaborate on its implementation. A more detailed discussion on the algorithm's suitability for the indistinct boundary contours of corn kernel tissues would strengthen this section.

While the process of model reconstruction is briefly described, additional details on the software or algorithms used for rendering and assigning distinct colors to each tissue would be helpful.

The process of creating the dataset for the ResNet50 model is described, but there is a lack of detail regarding how images were selected or classified as having mildew features or not.

The description of the fine-tuning process for the ResNet50 model is too brief.

While the paper mentions the use of an expanded image dataset augmented with additional data, it lacks details on the augmentation techniques and how they were applied.

3. Results

While volumetric parameters are quantitatively analyzed, the interpretation of these results lacks depth.

The ResNet50 model training results are briefly discussed, with high training accuracy and low classification loss highlighted. However, elaborating on the challenges encountered during training, such as overfitting or data imbalance, and how they were addressed, would offer a more comprehensive view of the model development process. Talk more about the hyperparameters and any transfer learning.

A comparative analysis of the ResNet50 model's performance against existing mold detection methods or other deep learning models is needed.

Model interpretability discussion is needed.

4. Discussion

Too brief.

Author Response

Thank you for your valuable feedback on this paper. I have greatly benefited from your comments and have carefully revised each one. The specific details are as follows:

1. Introduction:

(1) The introduction provides a general overview of the problems associated with mold in corn but could benefit from more specific details regarding the prevalence, economic impact, and geographical relevance of these issues. Quantitative data or statistics could strengthen the case for the study's significance.

    In response to your question, I have revised the first paragraph of the introduction as follows: Every year, China experiences a grain loss of about 70 billion catties, with farmers' grain storage accounting for 8% of this loss. Mildew is identified as the primary cause of this loss.

Lines 47--49

(2) While the introduction mentions various traditional and innovative detection methods, it lacks a clear explanation of why these traditional methods are inadequate beyond general terms.

    In response to your question, I have revised the second paragraph of the introduction as follows: In order to intuitively monitor the changes in the morphology of the embryo, endosperm, and grain at each stage of corn kernel development, and to timely detect early moldy corn kernels, a non-destructive testing method that preserves the morphology of corn kernels is needed.

Lines 65--68

(3) The introduction references several studies that have utilized X-ray tomography and deep learning in agricultural contexts. However, it does not critically discuss how these prior works relate to the current study's objectives. A brief critique or comparison could highlight the novelty or gap this study aims to address.

    In response to your question, I have revised the third paragraph of the introduction to read as follows: Although traditional image processing methods and model reconstruction can be used to observe changes in the structural characteristics of wheat grains, the processing process is cumbersome, and the defects of wheat cannot be quickly identified after CT scanning.

Lines 93--96

2. Materials and Methods:

(4) More detailed explanations regarding the choice of specific conditions (e.g., moisture level to 20%, room temperature at 25°C) would enhance understanding.

    Regarding the question you raised, I modified it in section “2.1 Preparation of Materials”.

Lines 121--129

(5) The description of image data acquisition and processing steps is somewhat vague. Expanding on why specific preprocessing steps like disabling the smoothing function were chosen and how they affect the quality or interpretability of the resulting images could clarify the methodology.

    Regarding this question you raised, I about this question you raised, I wrote in “2.4.1. Dataset creation Modify “and2.4.2” Image Preprocessing Modifications”.

Lines 189--194      Lines 204--217

(6) The paper mentions the use of an interactive threshold segmentation algorithm but does not elaborate on its implementation. A more detailed discussion on the algorithm's suitability for the indistinct boundary contours of corn kernel tissues would strengthen this section.

    In response to your question, I have modified Figure 1 of this paper. In this figure, (d) and (e) represent the effect images of corn kernels obtained using the interactive threshold segmentation algorithm.

Lines 163--165

(7) While the process of model reconstruction is briefly described, additional details on the software or algorithms used for rendering and assigning distinct colors to each tissue would be helpful.

    In response to your question, I have revised Figure 1 of this article to include the following description of the rendering: As shown in Figure 1(f), the red part is the corn embryo, the blue part is the endosperm, and the black part is the pore cracks and background.       

Lines163--170

(8) The process of creating the dataset for the ResNet50 model is described, but there is a lack of detail regarding how images were selected or classified as having mildew features or not.

    Regarding the question you raised, I addressed it in section 2.4.1. “Dataset creation “and add a secondary description in Figure 2.

  Lines 189--194    Lines 199-200

(9) The description of the fine-tuning process for the ResNet50 model is too brief.

    Regarding the issue you raised, I made the following additions in 2.4.3 ResNet50: Optimization was performed using an optimizer with stochastic gradient descent with momentum (SGDM) with an initial learning rate of 0.01. The learning rate was reduced by a factor of 0.2 every five cycles, with a minimum batch size of 128. A maximum of 30 theses were trained and validated using 1 iteration per 50 each.

 Lines 224--228

(10) While the paper mentions the use of an expanded image dataset augmented with additional data, it lacks details on the augmentation techniques and how they were applied.

    In response to your question, I elaborated on the application details of image enhancement in “2.4.2 Image Preprocessing”,attaching Figure 3 above.3. Results.

Lines 204--219

(11) While volumetric parameters are quantitatively analyzed, the interpretation of these results lacks depth.

    In response to your question, I have now elaborated on the results of the quantitative analysis of the volume parameters of the “3.2 Quantitative Analysis of Moldy Maize Kernels ”and revealed the reasons for this result.    

Lines 287--297

(12) The ResNet50 model training results are briefly discussed, with high training accuracy and low classification loss highlighted. However, elaborating on the challenges encountered during training, such as overfitting or data imbalance, and how they were addressed, would offer a more comprehensive view of the model development process. Talk more about the hyperparameters and any transfer learning.

    My answer to this question is as follows: The challenge in training deep learning models lies in the collection of slice images of corn kernels. Two main factors that can affect the accuracy of recognition are: 1) the orientation of the image, and 2) the presence of large cracks in the corn kernel itself. In order to prevent overfitting or data imbalance, the dataset was enriched through normalization and data augmentation using the original corn kernel dataset. In this paper, an optimizer utilizing Stochastic Gradient Descent with Momentum (SGDM) was employed for optimization. The optimizer had an initial learning rate of 0.01, with a 0.2-fold reduction in the learning rate every five cycles, a minimum batch size of 128, a maximum of 30 epochs for training, 1 validation per 50 iterations, and fine-tuning of ResNet50. The transfer learning method involves using the ImageNet public dataset to initialize the parameters for identifying corn mildew. By iteratively training on the corn kernel mildew dataset, a high accuracy rate can be achieved within 200 iterations.

(13) A comparative analysis of the ResNet50 model's performance against existing mold detection methods or other deep learning models is needed.

  In response to your question, I have now added “3.4 Summary of this chapter” to the article to explore it.

Lines 335--345

(14) Model interpretability discussion is needed.

    In response to your question, my answer is as follows: the interpretability of deep learning models is a new topic in the field of artificial intelligence. The main interpretability methods can be summarized and analyzed from four aspects: self-explanatory model, specific model interpretation, agnostic model interpretation, and causal interpretability. In this experiment, the deep learning model was primarily utilized for the rapid detection of corn kernel mildew. The optimizer employed was stochastic gradient descent with momentum (SGDM) for optimization. The initial learning rate was set at 0.01, and the learning rate was decreased by 0.2 times every five cycles. The minimum batch size was 128, the maximum number of epochs trained was 30, and the model was fine-tuned by validation once every 50 iterations. Using the public dataset ImageNet, the parameters trained by the public dataset were utilized as the initial parameters for corn mildew identification. The dataset of corn kernel mildew was then employed for iterative training, resulting in a high accuracy rate that could be achieved in 200 iterations. After repeated tests, it has been verified that the model's accuracy in detecting corn grain mold is as high as 93%. This level of accuracy is sufficient to support the prediction of corn mildew during the corn storage process.

4 Discussion

(15) Too brief.

    In response to your question, I have now rewritten the entire paragraph in the Discussion section, so please check it out.

Round 2

Reviewer 4 Report

Comments and Suggestions for Authors The manuscript has been sufficiently improved to warrant publication in Applied Sciences