The Importance of Context Awareness in Acoustics-Based Automated Beehive Monitoring
Abstract
:1. Introduction
2. Materials and Methods
2.1. Data Acquisition
2.2. Acoustic Feature Extraction
2.2.1. Root Mean Square Power
2.2.2. Spectrogram
2.2.3. Mel-Frequency Cepstral Coefficients
- Apply a pre-emphasis filter to enhance high frequencies.
- Compute the STFT of the pre-emphasized signal and its power spectrum.
- Apply the mel filterbank, composed of triangular filters simulating cochlear processing.
- Apply the logarithm operation.
- Compute the discrete cosine transform (DCT) to extract the mel frequency cepstral coefficients.
2.2.4. Discrete Wavelet Transform
2.2.5. Modulation Spectrogram
2.3. Beehive Strength Prediction Model
2.4. Experiment Setup and Figures-of-Merit
3. Results and Discussion
3.1. Urban Sound Effects
3.2. Speech Artifacts
3.3. Acoustic Effects of Heavy Rain
3.4. Prediction Model Performance
4. Recommendations
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
References
- FAO; Apimondia; CAAS; IZSLT. Good Beekeeping Practices for Sustainable Apiculture; FAO: Rome, Italy, 2021.
- Hadjur, H.; Ammar, D.; Lefèvre, L. Toward an intelligent and efficient beehive: A survey of precision beekeeping systems and services. Comput. Electron. Agric. 2022, 192, 106604. [Google Scholar] [CrossRef]
- Ruvinga, S.; Hunter, G.J.; Duran, O.; Nebel, J.C. Use of LSTM Networks to Identify “Queenlessness” in Honeybee Hives from Audio Signals. In Proceedings of the 2021 17th International Conference on Intelligent Environments (IE), Dubai, United Arab Emirates, 21–24 June 2021; pp. 1–4. [Google Scholar]
- Dubois, S.; Choveton-Caillat, J.; Kane, W.; Gilbert, T.; Nfaoui, M.; El Boudali, M.; Rezzouki, M.; Ferré, G. Bee Detection For Fruit Cultivation. In Proceedings of the 2021 IEEE International Symposium on Circuits and Systems (ISCAS), Daegu, Republic of Korea, 22–28 May 2021; pp. 1–5. [Google Scholar]
- Peng, R.; Ardekani, I.; Sharifzadeh, H. An Acoustic Signal Processing System for Identification of Queen-less Beehives. In Proceedings of the 2020 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC), Auckland, New Zealand, 7–10 December 2020; pp. 57–63. [Google Scholar]
- Henry, E.; Adamchuk, V.; Stanhope, T.; Buddle, C.; Rindlaub, N. Precision apiculture: Development of a wireless sensor network for honeybee hives. Comput. Electron. Agric. 2019, 156, 138–144. [Google Scholar] [CrossRef] [Green Version]
- Cecchi, S.; Spinsante, S.; Terenzi, A.; Orcioni, S. A Smart Sensor-Based Measurement System for Advanced Bee Hive Monitoring. Sensors 2020, 20, 2726. [Google Scholar] [CrossRef] [PubMed]
- Braga, A.R.; Gomes, D.G.; Rogers, R.; Hassler, E.E.; Freitas, B.M.; Cazier, J.A. A method for mining combined data from in-hive sensors, weather and apiary inspections to forecast the health status of honey bee colonies. Comput. Electron. Agric. 2020, 169, 105161. [Google Scholar] [CrossRef]
- Meikle, W.G.; Rector, B.G.; Mercadier, G.; Holst, N. Within-day variation in continuous hive weight data as a measure of honey bee colony activity. Apidologie 2008, 39, 694–707. [Google Scholar] [CrossRef] [Green Version]
- Stalidzans, E.; Berzonis, A. Temperature changes above the upper hive body reveal the annual development periods of honey bee colonies. Comput. Electron. Agric. 2013, 90, 1–6. [Google Scholar] [CrossRef]
- Edwards-Murphy, F.; Magno, M.; Whelan, P.M.; O’Halloran, J.; Popovici, E.M. b+ WSN: Smart beehive with preliminary decision tree analysis for agriculture and honey bee health monitoring. Comput. Electron. Agric. 2016, 124, 211–219. [Google Scholar] [CrossRef]
- Hambleton, J. The quantitative and qualitative effect of weather upon colony weight changes. J. Econ. Entomol. 1925, 18, 447–448. [Google Scholar] [CrossRef]
- Cetin, U. The effects of temperature changes to bee losts. Uludag Bee J. 2004, 4, 171–174. [Google Scholar]
- Seeley, T.; Heinrich, B. Regulation of Temperature in the Nests of Social Insects; Wiley: Hoboken, NJ, USA, 1981. [Google Scholar]
- Seeley, T.D. Honeybee ecology. In Honeybee Ecology; Princeton University Press: Princeton, NJ, USA, 2014. [Google Scholar]
- Zacepins, A.; Kviesis, A.; Stalidzans, E.; Liepniece, M.; Meitalovs, J. Remote detection of the swarming of honey bee colonies by single-point temperature monitoring. Biosyst. Eng. 2016, 148, 76–80. [Google Scholar] [CrossRef]
- Abou-Shaara, H.; Owayss, A.; Ibrahim, Y.; Basuny, N. A review of impacts of temperature and relative humidity on various activities of honey bees. Insectes Sociaux 2017, 64, 455–463. [Google Scholar] [CrossRef]
- Human, H.; Nicolson, S.W.; Dietemann, V. Do honeybees, Apis mellifera scutellata, regulate humidity in their nest? Naturwissenschaften 2006, 93, 397–401. [Google Scholar] [CrossRef] [PubMed]
- Michelsen, A.; Kirchner, W.H.; Lindauer, M. Sound and vibrational signals in the dance language of the honeybee, Apis mellifera. Behav. Ecol. Sociobiol. 1986, 18, 207–212. [Google Scholar] [CrossRef]
- Hunt, J.; Richard, F.J. Intracolony vibroacoustic communication in social insects. Insectes Sociaux 2013, 60, 403–417. [Google Scholar] [CrossRef]
- Bromenshenk, J.J.; Henderson, C.B.; Seccomb, R.A.; Rice, S.D.; Etter, R.T. Honey bee acoustic recording and analysis system for monitoring hive health. US Patent 7,549,907, 23 June 2009. [Google Scholar]
- Zlatkova, A.; Kokolanski, Z.; Tashkovski, D. Honeybees swarming detection approach by sound signal processing. In Proceedings of the 2020 XXIX International Scientific Conference Electronics (ET), Sozopol, Bulgaria, 16–18 September 2020; pp. 1–3. [Google Scholar]
- Žgank, A. Acoustic monitoring and classification of bee swarm activity using MFCC feature extraction and HMM acoustic modeling. In Proceedings of the 2018 ELEKTRO, Moscow, Russia, 16–19 April 2018; pp. 1–4. [Google Scholar]
- Abdollahi, M.; Giovenazzo, P.; Falk, T.H. Automated Beehive Acoustics Monitoring: A Comprehensive Review of the Literature and Recommendations for Future Work. Appl. Sci. 2022, 12, 3920. [Google Scholar] [CrossRef]
- Heise, D.; Miller-Struttmann, N.; Galen, C.; Schul, J. Acoustic detection of bees in the field using CASA with focal templates. In Proceedings of the 2017 IEEE Sensors Applications Symposium (SAS), Glassboro, NJ, USA, 13–15 March 2017; pp. 1–5. [Google Scholar]
- Kim, J.; Oh, J.; Heo, T.Y. Acoustic Scene Classification and Visualization of Beehive Sounds Using Machine Learning Algorithms and Grad-CAM. Math. Probl. Eng. 2021, 2021, 5594498. [Google Scholar] [CrossRef]
- Nolasco, I.; Benetos, E. To bee or not to bee: Investigating machine learning approaches for beehive sound recognition. arXiv 2018, arXiv:1811.06016. [Google Scholar]
- Zhang, T.; Zmyslony, S.; Nozdrenkov, S.; Smith, M.; Hopkins, B. Semi-Supervised Audio Representation Learning for Modeling Beehive Strengths. arXiv 2021, arXiv:2105.10536. [Google Scholar]
- Terenzi, A.; Cecchi, S.; Orcioni, S.; Piazza, F. Features extraction applied to the analysis of the sounds emitted by honey bees in a beehive. In Proceedings of the 2019 11th International Symposium on Image and Signal Processing and Analysis (ISPA), Dubrovnik, Croatia, 23–25 September 2018; pp. 03–08. [Google Scholar]
- Nolasco, I.; Terenzi, A.; Cecchi, S.; Orcioni, S.; Bear, H.L.; Benetos, E. Audio-based identification of beehive states. In Proceedings of the ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 12–17 May 2019; pp. 8256–8260. [Google Scholar]
- Terenzi, A.; Ortolani, N.; Nolasco, I.; Benetos, E.; Cecchi, S. Comparison of Feature Extraction Methods for Sound-based Classification of Honey Bee Activity. IEEE/ACM Trans. Audio Speech Lang. Process. 2021, 30, 112–122. [Google Scholar] [CrossRef]
- Krzywoszyja, G.; Rybski, R.; Andrzejewski, G. Bee swarm detection based on comparison of estimated distributions samples of sound. IEEE Trans. Instrum. Meas. 2018, 68, 3776–3784. [Google Scholar] [CrossRef]
- Anand, N.; Raj, V.B.; Ullas, M.; Srivastava, A. Swarm Detection and Beehive Monitoring System using Auditory and Microclimatic Analysis. In Proceedings of the 2018 3rd International Conference on Circuits, Control, Communication and Computing (I4C), Bangalore, India, 3–5 October 2018; pp. 1–4. [Google Scholar]
- Zlatkova, A.; Gerazov, B.; Tashkovski, D.; Kokolanski, Z. Analysis of parameters in algorithms for signal processing for swarming of honeybees. In Proceedings of the 2020 28th Telecommunications Forum (TELFOR), Belgrade, Serbia, 24–25 November 2020; pp. 1–4. [Google Scholar]
- Qandour, A.; Ahmad, I.; Habibi, D.; Leppard, M. Remote Beehive Monitoring Using Acoustic Signals. Acoust. Aust. 2014, 42, 205. [Google Scholar]
- Sharif, M.Z.; Wario, F.; Di, N.; Xue, R.; Liu, F. Soundscape Indices: New Features for Classifying Beehive Audio Samples. Sociobiology 2020, 67, 566–571. [Google Scholar] [CrossRef]
- Zhao, Y.; Deng, G.; Zhang, L.; Di, N.; Jiang, X.; Li, Z. Based investigate of beehive sound to detect air pollutants by machine learning. Ecol. Inform. 2021, 61, 101246. [Google Scholar] [CrossRef]
- Pérez, N.; Jesús, F.; Pérez, C.; Niell, S.; Draper, A.; Obrusnik, N.; Zinemanas, P.; Spina, Y.M.; Letelier, L.C.; Monzón, P. Continuous monitoring of beehives’ sound for environmental pollution control. Ecol. Eng. 2016, 90, 326–330. [Google Scholar] [CrossRef]
- Hunter, G.; Howard, D.; Gauvreau, S.; Duran, O.; Busquets, R. Processing of multi-modal environmental signals recorded from a “smart” beehive. Proc. Inst. Acoust. 2019, 41, 339–348. [Google Scholar]
- Cecchi, S.; Terenzi, A.; Orcioni, S.; Riolo, P.; Ruschioni, S.; Isidoro, N. A preliminary study of sounds emitted by honey bees in a beehive. In Proceedings of the Audio Engineering Society Convention 144, Milan, Italy, 23–26 May 2018. [Google Scholar]
- Zacepins, A.; Kviesis, A.; Ahrendt, P.; Richter, U.; Tekin, S.; Durgun, M. Beekeeping in the future—Smart apiary management. In Proceedings of the 2016 17th International Carpathian Control Conference (ICCC), High Tatras, Slovakia, 29 May–1 June 2016; pp. 808–812. [Google Scholar]
- Imoize, A.L.; Odeyemi, S.D.; Adebisi, J.A. Development of a Low-Cost Wireless Bee-Hive Temperature and Sound Monitoring System. Indones. J. Electr. Eng. Inform. (IJEEI) 2020, 8, 476–485. [Google Scholar]
- Robles-Guerrero, A.; Saucedo-Anaya, T.; González-Ramírez, E.; De la Rosa-Vargas, J.I. Analysis of a multiclass classification problem by lasso logistic regression and singular value decomposition to identify sound patterns in queenless bee colonies. Comput. Electron. Agric. 2019, 159, 69–74. [Google Scholar] [CrossRef]
- Robles-Guerrero, A.; Saucedo-Anaya, T.; González-Ramérez, E.; Galván-Tejada, C.E. Frequency Analysis of Honey Bee Buzz for Automatic Recognition of Health Status: A Preliminary Study. Res. Comput. Sci. 2017, 142, 89–98. [Google Scholar] [CrossRef]
- Seeley, T.D.; Tautz, J. Worker piping in honey bee swarms and its role in preparing for liftoff. J. Comp. Physiol. A 2001, 187, 667–676. [Google Scholar] [CrossRef]
- Simpson, J.; Cherry, S.M. Queen confinement, queen piping and swarming in Apis mellifera colonies. Anim. Behav. 1969, 17, 271–278. [Google Scholar] [CrossRef]
- van der Zee, R.; Pisa, L.; Andonov, S.; Brodschneider, R.; Charriere, J.D.; Chlebo, R.; Coffey, M.F.; Crailsheim, K.; Dahle, B.; Gajda, A.; et al. Managed honey bee colony losses in Canada, China, Europe, Israel and Turkey, for the winters of 2008–9 and 2009–10. J. Apic. Res. 2012, 51, 100–114. [Google Scholar] [CrossRef] [Green Version]
- Jacques, A.; Laurent, M.; Consortium, E.; Ribière-Chabert, M.; Saussac, M.; Bougeard, S.; Budge, G.E.; Hendrikx, P.; Chauzat, M.P. A pan-European epidemiological study reveals honey bee colony survival depends on beekeeper education and disease control. PLoS ONE 2017, 12, e0172591. [Google Scholar] [CrossRef] [Green Version]
- Kulhanek, K.; Steinhauer, N.; Rennich, K.; Caron, D.M.; Sagili, R.R.; Pettis, J.S.; Ellis, J.D.; Wilson, M.E.; Wilkes, J.T.; Tarpy, D.R.; et al. A national survey of managed honey bee 2015–2016 annual colony losses in the USA. J. Apic. Res. 2017, 56, 328–340. [Google Scholar] [CrossRef] [Green Version]
- Brodschneider, R.; Gray, A.; Adjlane, N.; Ballis, A.; Brusbardis, V.; Charrière, J.D.; Chlebo, R.; Coffey, M.F.; Dahle, B.; de Graaf, D.C.; et al. Multi-country loss rates of honey bee colonies during winter 2016/2017 from the COLOSS survey. J. Apic. Res. 2018, 57, 452–457. [Google Scholar] [CrossRef] [Green Version]
- Gray, A.; Brodschneider, R.; Adjlane, N.; Ballis, A.; Brusbardis, V.; Charrière, J.D.; Chlebo, R.; Coffey, M.F.; Cornelissen, B.; Amaro da Costa, C.; et al. Loss rates of honey bee colonies during winter 2017/18 in 36 countries participating in the COLOSS survey, including effects of forage sources. J. Apic. Res. 2019, 58, 479–485. [Google Scholar] [CrossRef] [Green Version]
- Gray, A.; Adjlane, N.; Arab, A.; Ballis, A.; Brusbardis, V.; Charrière, J.D.; Chlebo, R.; Coffey, M.F.; Cornelissen, B.; Amaro da Costa, C.; et al. Honey bee colony winter loss rates for 35 countries participating in the COLOSS survey for winter 2018–2019, and the effects of a new queen on the risk of colony winter loss. J. Apic. Res. 2020, 59, 744–751. [Google Scholar] [CrossRef]
- Porrini, C.; Mutinelli, F.; Bortolotti, L.; Granato, A.; Laurenson, L.; Roberts, K.; Gallina, A.; Silvester, N.; Medrzycki, P.; Renzi, T.; et al. The status of honey bee health in Italy: Results from the nationwide bee monitoring network. PLoS ONE 2016, 11, e0155411. [Google Scholar] [CrossRef] [Green Version]
- Stanimirović, Z.; Glavinić, U.; Ristanić, M.; Aleksić, N.; Jovanović, N.; Vejnović, B.; Stevanović, J. Looking for the causes of and solutions to the issue of honey bee colony losses. Acta Vet. 2019, 69, 1–31. [Google Scholar] [CrossRef] [Green Version]
- Davis, S.; Mermelstein, P. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Trans. Acoust. Speech Signal Process. 1980, 28, 357–366. [Google Scholar] [CrossRef] [Green Version]
- Barchiesi, D.; Giannoulis, D.; Stowell, D.; Plumbley, M.D. Acoustic scene classification: Classifying environments from the sounds they produce. IEEE Signal Process. Mag. 2015, 32, 16–34. [Google Scholar] [CrossRef]
- Gaballah, A.; Tiwari, A.; Narayanan, S.; Falk, T.H. Context-aware speech stress detection in hospital workers using Bi-LSTM classifiers. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Toronto, ON, Canada, 6–12 June 2021; pp. 8348–8352. [Google Scholar]
- Stange, E. Optimizing urban beekeeping. In Achieving Sustainable urban Agriculture; Burleigh Dodds Science Publishing: Cambridge, UK, 2020; pp. 331–352. [Google Scholar]
- Nectar. Available online: https://www.nectar.buzz/ (accessed on 15 November 2022).
- Chabert, S.; Requier, F.; Chadoeuf, J.; Guilbaud, L.; Morison, N.; Vaissiere, B.E. Rapid measurement of the adult worker population size in honey bees. Ecol. Indic. 2021, 122, 107313. [Google Scholar] [CrossRef]
- Mallat, S.G. Multiresolution approximations and wavelet orthonormal bases of L2(R). Trans. Am. Math. Soc. 1989, 315, 69–87. [Google Scholar]
- Daubechies, I. Ten Lectures on Wavelets; SIAM: Philadelphia, PA, USA, 1992. [Google Scholar]
- Falk, T.H.; Chan, W.Y. Modulation spectral features for robust far-field speaker identification. IEEE Trans. Audio Speech Lang. Process. 2009, 18, 90–100. [Google Scholar] [CrossRef]
- Avila, A.R.; Monteiro, J.; O’Shaughneussy, D.; Falk, T.H. Speech emotion recognition on mobile devices based on modulation spectral feature pooling and deep neural networks. In Proceedings of the 2017 IEEE International Symposium on Signal Processing and Information Technology (ISSPIT), Bilbao, Spain, 18–20 December 2017; pp. 360–365. [Google Scholar]
Layer | Description |
---|---|
Convolution | 128 filters, kernel size = , strides = |
Convolution | 128 filters, kernel size = , strides = |
Batch normalization | momentum = , gamma = , epsilon = |
Max-pooling | pool size = , strides= |
Convolution | 64 filters, kernel size = , strides = |
Batch normalization | momentum = , gamma = , epsilon = |
Convolution | 64 filters, kernel size = , strides = |
Batch normalization | momentum = , gamma = , epsilon = |
Max-pooling | pool size = , strides = |
Dropout | 0.25 rate |
Dense | 128 units |
Batch normalization | momentum = , gamma = , epsilon = |
Dropout | 0.5 rate |
Dense | 1 unit |
Random-Split | ||||||||
---|---|---|---|---|---|---|---|---|
Features | Clean | Urban sound | Speech artifacts | Heavy rain | ||||
MAE | RMSE | MAE | RMSE | MAE | RMSE | MAE | RMSE | |
Spectrogram | 1.03 | 1.93 | 1.46 | 2.13 | 1.23 | 2.43 | 2.88 | 4.53 |
MFCCs | 0.86 | 2.34 | 1.17 | 2.72 | 1.86 | 3.93 | 1.00 | 2.52 |
DWT | 1.42 | 4.43 | 4.84 | 6.52 | 1.92 | 6.12 | 6.47 | 9.01 |
Hive-Independent | ||||||||
Spectrogram | 4.28 | 5.17 | 6.37 | 6.97 | 7.98 | 8.02 | 7.20 | 7.39 |
MFCCs | 4.91 | 5.73 | 6.93 | 7.86 | 8.77 | 9.23 | 6.95 | 7.75 |
DWT | 4.94 | 5.33 | 6.62 | 8.39 | 9.07 | 9.42 | 8.91 | 9.10 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Abdollahi, M.; Henry, E.; Giovenazzo, P.; Falk, T.H. The Importance of Context Awareness in Acoustics-Based Automated Beehive Monitoring. Appl. Sci. 2023, 13, 195. https://doi.org/10.3390/app13010195
Abdollahi M, Henry E, Giovenazzo P, Falk TH. The Importance of Context Awareness in Acoustics-Based Automated Beehive Monitoring. Applied Sciences. 2023; 13(1):195. https://doi.org/10.3390/app13010195
Chicago/Turabian StyleAbdollahi, Mahsa, Evan Henry, Pierre Giovenazzo, and Tiago H. Falk. 2023. "The Importance of Context Awareness in Acoustics-Based Automated Beehive Monitoring" Applied Sciences 13, no. 1: 195. https://doi.org/10.3390/app13010195