Identifying Queenlessness in Honeybee Hives from Audio Signals Using Machine Learning

Round 1

Reviewer 1 Report

This manuscript presents a method of acoustic signal classification using Mel Frequency Cepstral Coefficients combined with machine learning techniques, which have been widely used in the literature during the last decades.

In addition, the implementation of previous algorithms can be easily obtained from

https://github.com/

 

Therefore, this work should not be published in its current condition due to its poor contribution.

 

Author Response

Thank you for your comments. We have now substantially revised the manuscript, emphasising its novel aspects and contributions to the field. We have added further references and tried to make all already existing sections clearer.

Reviewer 2 Report

See attached document.

Comments for author File: Comments.pdf

Author Response

Thank you for your comments. We have tried to address them all as best we can.

We have added a paragraph at the end of the introduction, noting the novel aspects of this work and emphasising its contributions to the field.

We have also added a "Previous Related Work" section, describing previous efforts to address this and closely related problems, and noting how there have been few previous attempts to apply Machine Learning to them.

We have, we believe, addressed/corrected all the typos and other errors you noted.

Regarding your point number 7, we have added a comment regarding this in the "Conclusions and Future Work" section. This would not be possible based on the datasets we currently have, but might be feasible if sufficient long-term data were available, so that patterns in the sounds produced by the bees over a period running up to the death of a queen could be investigated and compared with those from "healthy" hives under otherwise similar conditions.

 

Reviewer 3 Report

The use of sound framing and windowing should be justified in a better way. It is not clear enough why there is a 50% overlap in consecutive frames.  

There are no details about the architectures of the CNN and LSTM models proposed in this research.

Author Response

Thank you for your comments

We have added further details, and rationale, for the framing and windowing process, as requested.

We have added some further details about the layer structure of the CNN used, and of the number of units used in the LSTM, along with information about the optimisers used.

We have tried to check for spelling, typographic and grammar errors, and we believe we have now corrected all of those.

Reviewer 4 Report

Introduction is good in terms of justifying the fact that sound can be used to know whether a hive is in decline or if bees are about to swarm, but it does not in any way address the need for machine learning (ML), what type of ML will be the focus of the paper nor does the Introduction give any idea of what is to follow in the paper. MFCC needs to be introduced among other candidate features and then justified. There are many features for sound analysis. Also, many deep learns have been used in sound analysis. Why are the authors using LSTM? Briefly, describe what other work using LSTM has been performed in sound classification. The use of ML techniques is not justified and confidence that the authors have a handle on the subject of sound classification generally is unconvincing. It is good that they compare performance with MLP and LR. Also, say something about the differences in sound characteristics between healthy and unhealthy hives as already reported in the literature on bees. The Introduction needs a complete revision. 

Clearly state innovations and contributions in the introduction and abstract. Is this the first study classifying the sound of beehives into the classes QP and QA? Be clear.

How many total samples? This data set is very small. How did you avoid overfitting with CNN?

Appropriate performance metrics were used.

Descriptions of the learners is adequate. 

ln 104: Though it may seem obvious, state why recovering high frequencies are needed: in fact, what is the needed frequency bands for the classification process? This information needs to be presented in the introduction.

Section B needs a graph of the whole process/methodology along with a brief description.

Table 1. needs much more explaination. I don't understand 3/8/12. Also go ahead and provide the meaning of QP and QA after stating what C means even though explained in the text. Repeat it for clarity.

 

Figure 4 is unclear, too blurry.

Make sure all variables are defined in the math.

Figure 6: why is your input a letter?

Add limitations to the study in the conclusion (like the small data set).

Author Response

Thank you for your comments.

We have added an extra paragraph at the end of the introduction, highlighting the novelty and main contributions of our work. Although there have been many applications of Machine Learning to various other sound analysis problems, there have been very few such studies applying ML to the problem of a bee colony losing its queen.

We have added a new "Previous Related Work" section, describing previous approaches to this and closely related problems relating to monitoring the health and well-being of honeybees. Most of these have relied on visual inspection of spectrograms (or similar methods requiring human intervention or judgement), so incorporating Machine Learning successfully should enable the processing to be largely automated.

LSTMs are particularly well-suited to the analysis of sequential data, of which time-varying sound signals are an example.

We have tried to make the number of sound samples used clearer, and briefly discuss the issue of potential overfitting.

We have added some comments about the need for inclusion of higher frequency bands.

We have added a diagram (Figure 3) giving an overview of the whole process.

3/8/12 (and similar) in Table 1 referred to the dates on which the audio samples were recorded. We have changed these to a hopefully clearer date format, and defined that (C) means control group.

We have re-drawn the old Figure 4 (now Figure 5), as requested, and hopefully it is now much clearer.

We believe that all the mathematical symbols and functions used have now been defined.

We have added a note about the input used for illustration in (old) Figure 6 (now Figure 7) to explain that the handwritten digit "2" is used for illustration. We felt that using an image of a sound spectrogram might be too "abstract" for some readers to relate to.

We have added some comments about the limitations of the study to the "Discussion" and "Conclusions and Future Work" sections.

Round 2

Reviewer 2 Report

The quality of the proposed work has been improved according to my comments and recommendations. This work can be considered for publication.

Author Response

Thank you for your new set of comments. We have tried to proof read our latest version of the manuscript very carefully, hopefully clarifying any ambiguous points and correcting any typographic, spelling or grammar errors. We have added a couple of diagrams at the request of one of the other reviewers and tried to clarify the captions of some others. Hopefully you will find the latest version further improved.

Reviewer 4 Report

* Please highlight changes in the manuscript. Did something happen? I do not see anything highlighted.

LSTMs are particularly well-suited to the analysis of sequential data, of which time-varying sound signals are an example.

* Yes, but other methods have been used with sound as well.

3/8/12 (and similar) in Table 1 referred to the dates on which the audio samples were recorded. We have changed these to a hopefully clearer date format, and defined that (C) means control group.

* Sure, you stated what they meant, but they needed to be stated on the figure for clarity; no sense in making the reader search for what they mean. Glad you did this.

We felt that using an image of a sound spectrogram might be too "abstract" for some readers to relate to.

* No, it's not too abstract. It is done all the time in ML and sound and in this very journal. Many know how to read them. **Please change it to a sample spectrogram. In fact, providing samples of QP and QA would help researchers see differences. I suggest the addition.**

 

 

Author Response

Thank you for this latest set of comments.

* Please highlight changes in the manuscript. Did something happen? I do not see anything highlighted.

I thought I had used the "Comment" feature in "Track Changes" in Microsoft Word to do that, but my attempt to do so may not have succeeded. My apologies. Hopefully that will be correct in the new version.

LSTMs are particularly well-suited to the analysis of sequential data, of which time-varying sound signals are an example.

* Yes, but other methods have been used with sound as well.

I am aware of methods such as Linear Predictive Coding being used (e.g.) in the analysis of human speech. However, I have never used such methods myself and am not aware of them ever being used in the context of bee acoustics. This might be an appropriate method for use in a further study. Do you feel that a review of such approaches is necessary here ?

In the paper, we have used other methods - Logistic Regression, Multi-Layer Perceptron and Convolutional Neural Networks - in addition to the LSTM. Should we include attempts using other methods as well ?

3/8/12 (and similar) in Table 1 referred to the dates on which the audio samples were recorded. We have changed these to a hopefully clearer date format, and defined that (C) means control group.

* Sure, you stated what they meant, but they needed to be stated on the figure for clarity; no sense in making the reader search for what they mean. Glad you did this.

Hopefully this is clearer now.

We felt that using an image of a sound spectrogram might be too "abstract" for some readers to relate to.

* No, it's not too abstract. It is done all the time in ML and sound and in this very journal. Many know how to read them. **Please change it to a sample spectrogram. In fact, providing samples of QP and QA would help researchers see differences. I suggest the addition.

We have revised Figure 7 (the example CNN architecture) to have a spectrogram as its input. We have also added an extra figure, Figure 11, giving two examples of each of spectrograms from Queen Present and Queen Absent hives, and trying to draw the reader's attention to the most salient visible differences between them.