Beehive Smart Detector Device for the Detection of Critical Conditions That Utilize Edge Device Computations and Deep Learning Inferences
Abstract
:1. Introduction
2. Related Work
3. Materials and Methods
3.1. BeeSD High-Level Architecture
- BeeSD end node device:
- This is the end node IoT device placed inside the beehive (see Figure 2(1), BeeSD node). The BeeSD IoT device can operate in two modes: training and detection. In training mode, the device periodically transmits 20 to 45-minute raw sound measurements and hourly temperature and humidity measurements. Training data transmissions are performed via (a) MQTT publish over the MQTT broker (see Figure 2(4)) and then are stored to the BeeSD file server (see Figure 2(5)) for later processing and model training. Usually, during training mode, the sound recordings are performed at night to avoid both daily noises and stress conditions caused to the beehive by human interventions or animals during the day. Moreover, to increase the system security and data integrity, each BeeSD IoT device is equipped with a VPN key in order to connect to the cloud services and transmit training data, authenticated via the VPN service (see Figure 2(3)). The BeeSD end node IoT device design and prototype are illustrated in Figure 3.In detection mode, the BeeSD device transmits sensory measurements (see Figure 4a), Mel spectrogram images (see Figure 4b), major FFT parameters and attribute values (called beegram values; see Figure 4c), which correspond to the significant frequency bands presented in Figure 1, mean aggregated beegram values (see Figure 4d), which correspond to beehive growth levels, queen tone, thermal stress, and swarming indications, detection inferences for queen loss and swarming acquired by the CNN image detection models, and CCD conditions detection calculated by the fuzzy-stranded-NN model (see Figure 2(f), detection model inference). These values are transmitted over HTTP telemetry posts to the ThingsBoard Application Server (AS) [78] (see Figure 2(6)). Suppose a model update parameter is set at the AS. In that case, the BeeSD IoT device may also download an updated version of a specific pre-trained model used for either detecting swarming and queen loss or critical conditions (see Figure 2(e), HTTP GET model request).
- BeeSD gateway:
- This is the BeeSD system gateway (see Figure 2(2)), placed in the center of the beekeeping area, and it includes a Wi-Fi-to-3G/LTE access point gateway device with its corresponding 12 V–100 Ah battery, 20 W photovoltaic panel, and voltage regulator to provide autonomous uninterruptible operation. This gateway connects all BeeSD IoT nodes to the BeeSD system’s cloud services.
- Data logging and visualization services:
- The BeeSD system also performs measurement logging and visualization in detection mode and measurement logging in training mode. The ThingsBoard AS performs detection mode logging and visualization tasks (see Figure 4). Measurements are stored in its Cassandra database, while IoT device management and dashboard operations are stored in its PostgreSQL database (see Figure 2(5)). Therefore, ThingsBoard’s core functionality is for storing data measurements sent by the BeeSD IoT nodes via HTTP POSTs, offering pre-trained model updates to the IoT nodes, and dashboard visualizations via HTTP GET requests to the beekeepers. The training mode data logging is performed using different MQTT channel topics (MQTT publish) over an MQTT broker (see Figure 2(4)). These trainable data are then stored in trainable data files (images, sensory measurements) in the BeeSD file server (see Figure 2(5)).
- AI component:
- The BeeSD system AI component (see Figure 2(7)) is a dockerized container including all necessary libraries and training tools for performing deep learning model training. The container has access to the BeeSD File server via HTTP requests (see Figure 2(b)); to obtain all necessary data needed for model training, it can perform data filtering, fuzzy annotation, CNN model training, and fuzzy-stranded-NN model training. The training process is performed on demand and manually by the system’s supervisor (see Figure 2(c)). The supervisor is also responsible for uploading pre-trained models as per BeeSD device server parameters to the ThingsBoard AS using HTTP POSTS (see Figure 2(d)).
- Mobile phone Application:
- The BeeSD system mobile phone application is an end-user application capable of visualizing ThingsBoard HTTP dashboards. It is a web progressive Android application that the beekeeper uses to connect to the ThingsBoard AS and monitor his device (per device) dashboards (see Figure 2(g) and dashboard illustrations in Figure 4). That is, each device is identified as a ‘hive-id’–‘node-id’ tuple. Then, for each device, a specific dashboard is manually generated (using ThingsBoard plugins and IoT device telemetry measurements) (see Figure 4) and assigned to each beekeeper. This way, the beekeeper can monitor the measurements of his/her devices, beegram attribute values, beegram correlation functions values, spectrogram, listen to audio recordings, and monitor the DL notifications and alerts for swarming events, queen loss, and critical abiotic conditions.
3.2. Bee Smart Detection IoT Device
3.3. BeeSD DL Detection Process
- Class 0:
- Queen loss class. It corresponds to the queen loss class, trained using audio recordings of forced queen removal events performed in selected beehives in the author’s beekeeping station over the years 2023 and 2024. This annotated dataset contains 14GB of audio recordings (approximately 100 h of queen loss audio recordings performed in four beehives corresponding to 5882 JPEG spectrogram images).
- Class 1:
- Weak bee class. This class corresponds to the annotated class recordings of weak bees (under five frames inside the beehive) that are in the phase of growing or under stress since they have newly born queens. This dataset has been recorded in three young beehives during spring and two during late autumn of 2022–2023. It contains 23.3 GB of audio recordings (approximately 167 recorded hours, corresponding to 10,020 min JPEG spectrogram images).
- Class 2:
- Normal bee class. It corresponds to the annotated class recordings of normal beehives (strong beehives of at least ten frames per beehive). This dataset was acquired during the spring and summer seasons of 2023–2024. It contains 17.3 GB of audio recordings (approximately 126 recorded hours, corresponding to 7560 min JPEG spectrogram images).
- Class 3:
- This class is the early swarming detection class. It was trained using audio recordings of beehives approximately ten days before the first swarming event. This dataset was acquired during forced swarming events out of four 10-frame beehives in the spring and summer seasons of 2022–2023. It contains 12 GB of audio recordings (approximately 86 recorder hours, corresponding to 5160 min JPEG spectrogram images).
- Class 4:
- This is the late swarming detection class. It was trained using audio recordings of beehives five days before the swarming event. This dataset has been acquired during forced swarming events out of 10-frame beehives in the spring and summer seasons of 2022–2023, out of seven beehives. It contains 16 GB of audio recordings, approximately 115 recorded hours, corresponding to 6900 min JPEG spectrogram images).
- Class 0:
- It is the abiotic cold condition class. This class includes lid temperatures below 0 °C and mean probe temperatures below 10 °C. This critical condition case requires immediate intervention (beehive relocation to a warmer place, use of passive insulation, use of active thermal actuating systems [56]) to avoid CCD winter bee stress and freezing events of imminent colony death. For this class, humidity measurements have no significant influence.
- Class 1:
- It is the disease condition incubation class. It includes humidity measurements above 80% RH for mean temperature values below 20 °C () and humidity measurements above 90% for mean temperatures above 20 °C (). In this class, appropriate interventions should occur (check for varroa mite and nozema fungus concentrations in the frames, for wingless bees, and for internal water leak concentrations in the brood box). These class conditions are also triggered by multiple days of continuous rainfall.
- Class 2:
- It is the normal condition class. This class includes probe temperature differences, specifically lid–probe temperature differences no more than 1–6 °C, where the maximum temperature probe value is between 30 and 37 °C. In this class, humidity values are between 30 and 80% of RH. Typical normal temperature probe values are around 34–36 °C and humidity levels are between 40 and 65% of RH values.
- Class 3:
- It is the abiotic hot condition class. It includes relative humidity values below 75% RH, lid temperature values above 42 °C, and temperature differences less than 3–4 °C between probes (mean temperature value) and lid (DHT22). This critical condition usually occurs after a swarming or heat stress event in a weak bee population. It requires beehive relocation, water installments near the beehive, specialized interventions of larvae frame addition taken from other beehives, or even queen removal (replacement). Especially if mean temperature probe values are above 37 °C, it is a critical indication of a catastrophic collapse (inability to thermoregulate the hive).
- Class 4:
- It is the outlier/erroneous measurement class. It includes all measurements not fitting into one of the classes above. Generally, this class includes erroneous measurements due to broken or misplaced probes, measurements of an abandoned (empty) beehive, or measurements that do not feel like beehive colony measurements.
4. Experimental Scenario
Evaluation Metrics
5. Experimental Results and Discussion
6. Conclusions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
Abbreviations
AI | Artificial Intelligence |
AS | Application Server |
CCD | Colony Colapse Disorder events |
CNN | Convolutional Neural Network |
CPU | Central Processing Unit |
DL | Deep Learning |
DSS | Decision Support System |
FFT | Fast Fourier Transformation |
k-NN | k-Nearest Neighbor |
LSTM | Long short-term memory |
Mels | Melody scale of logarithmic frequency transformation |
ML | Machine Learning |
MFCCs | Mel Frequency Cepstral Coefficients |
LTE | Telecommunication Systems Long-Term Evolution (4G) |
NN | Neural Network |
RAM | Random Access Memory |
RTC | Real-Time Clock |
PV | Photovoltaic cell |
SPI | Synchronous Peripheral Interface |
SVM | Support Vector Machine |
STFT | Short Time Fourier Transformation |
References
- Danieli, P.P.; Addeo, N.F.; Lazzari, F.; Manganello, F.; Bovera, F. Precision Beekeeping Systems: State of the Art, Pros and Cons, and Their Application as Tools for Advancing the Beekeeping Sector. Animals 2024, 14, 70. [Google Scholar] [CrossRef]
- Zaman, A.; Dorin, A. A framework for better sensor-based beehive health monitoring. Comput. Electron. Agric. 2023, 210, 107906. [Google Scholar] [CrossRef]
- Astuti, P.K.; Hegedűs, B.; Oleksa, A.; Bagi, Z.; Kusza, S. Buzzing with Intelligence: Current Issues in Apiculture and the Role of Artificial Intelligence (AI) to Tackle It. Insects 2024, 15, 418. [Google Scholar] [CrossRef]
- Hadjur, H.; Ammar, D.; Lefèvre, L. Toward an intelligent and efficient beehive: A survey of precision beekeeping systems and services. Comput. Electron. Agric. 2022, 192, 106604. [Google Scholar] [CrossRef]
- Peladarinos, N.; Piromalis, D.; Cheimaras, V.; Tserepas, E.; Munteanu, R.A.; Papageorgas, P. Enhancing Smart Agriculture by Implementing Digital Twins: A Comprehensive Review. Sensors 2023, 23, 7128. [Google Scholar] [CrossRef] [PubMed]
- Kiromitis, D.I.; Bellos, C.V.; Stefanou, K.A.; Stergios, G.S.; Andrikos, I.; Katsantas, T.; Kontogiannis, S. Performance Evaluation of Classification Algorithms to Detect Bee Swarming Events Using Sound. Signals 2022, 3, 807–822. [Google Scholar] [CrossRef]
- Mrozek, D.; Gorny, R.; Wachowicz, A.; Malysiak-Mrozek, B. Edge-Based Detection of Varroosis in Beehives with IoT Devices with Embedded and TPU-Accelerated Machine Learning. Appl. Sci. 2021, 11, 1078. [Google Scholar] [CrossRef]
- Voudiotis, G.; Moraiti, A.; Kontogiannis, S. Deep Learning Beehive Monitoring System for Early Detection of the Varroa Mite. Signals 2022, 3, 506–523. [Google Scholar] [CrossRef]
- Kiromitis, D.I.; Bellos, C.V.; Stefanou, K.A.; Stergios, G.S.; Katsantas, T.; Kontogiannis, S. Bee Sound Detector: An Easy-to-Install, Low-Power, Low-Cost Beehive Conditions Monitoring System. Electronics 2022, 11, 3152. [Google Scholar] [CrossRef]
- Zhao, H.; Li, G.; Guo, D.; Li, H.; Liu, Q.; Xu, B.; Guo, X. Response mechanisms to heat stress in bees. Apidologie 2021, 52, 388–399. [Google Scholar] [CrossRef]
- Zhao, H.; Mashilingi, S.K.; Liu, Y.; An, J. Factors Influencing the Reproductive Ability of Male Bees: Current Knowledge and Further Directions. Insects 2021, 12, 529. [Google Scholar] [CrossRef]
- Heller, J. The Beekeeper’s Bible: Bees, Honey, Recipes and Other Home Uses; Collins: Glasgow, UK, 2010. [Google Scholar]
- Gregorc, A.; Sampson, B. Diagnosis of Varroa Mite (Varroa destructor) and Sustainable Control in Honey Bee (Apis mellifera) Colonies—A Review. Diversity 2019, 11, 243. [Google Scholar] [CrossRef]
- Schuler, V.; Liu, Y.C.; Gisder, S.; Horchler, L.; Groth, D.; Genersch, E. Significant, but not biologically relevant: Nosema ceranae infections and winter losses of honey bee colonies. Commun. Biol. 2023, 6, 229. [Google Scholar] [CrossRef] [PubMed]
- Kang, Y.; Wu, T.; Han, B.; Yang, S.; Wang, X.; Wang, Q.; Gao, J.; Dai, P. Interaction of acetamiprid, Varroa destructor, and Nosema ceranae in honey bees. J. Hazard. Mater. 2024, 471, 134380. [Google Scholar] [CrossRef]
- Insolia, L.; Molinari, R.; Rogers, S.R.; Williams, G.R.; Chiaromonte, F.; Calovi, M. Honey bee colony loss linked to parasites, pesticides and extreme weather across the United States. Sci. Rep. 2022, 12, 20787. [Google Scholar] [CrossRef]
- Evans, J.D.; Chen, Y.J. Colony Collapse Disorder and Honey Bee Health. In Honey Bee Medicine for the Veterinary Practitioner; John Wiley & Sons, Ltd.: Hoboken, NJ, USA, 2021; pp. 229–234. [Google Scholar] [CrossRef]
- El-Seedi, H.R.; Ahmed, H.R.; El-Wahed, A.A.A.; Saeed, A.; Algethami, A.F.; Attia, N.F.; Guo, Z.; Musharraf, S.G.; Khatib, A.; Alsharif, S.M.; et al. Bee Stressors from an Immunological Perspective and Strategies to Improve Bee Health. Vet. Sci. 2022, 9, 199. [Google Scholar] [CrossRef] [PubMed]
- vanEngelsdorp, D.; Evans, J.D.; Saegerman, C.; Mullin, C.; Haubruge, E.; Nguyen, B.K.; Frazier, M.; Frazier, J.; Cox-Foster, D.; Chen, Y.; et al. Colony Collapse Disorder: A Descriptive Study. PLoS ONE 2009, 4, e0006481. [Google Scholar] [CrossRef]
- Flores, J.M.; Gamiz, V.; Jimenez-Marin, A.; Flores-Cortes, A.; Gil-Lebrero, S.; Garrido, J.J.; Hernando, M.D. Impact of Varroa destructor and associated pathologies on the colony collapse disorder affecting honey bees. Res. Vet. Sci. 2021, 135, 85–95. [Google Scholar] [CrossRef]
- Kviesis, A.; Komasilovs, V.; Komasilova, O.; Zacepins, A. Application of fuzzy logic for honey bee colony state detection based on temperature data. Biosyst. Eng. 2020, 193, 90–100. [Google Scholar] [CrossRef]
- Bellos, C.V.; Fyraridis, A.; Stergios, G.S.; Stefanou, K.A.; Kontogiannis, S. A Quality and disease control system for beekeeping. In Proceedings of the 2021 6th South-East Europe Design Automation, Computer Engineering, Computer Networks and Social Media Conference (SEEDA-CECNSM), Preveza, Greece, 24–26 September 2021; pp. 1–4. [Google Scholar] [CrossRef]
- Zacepins, A.; Brusbardis, V.; Meitalovs, J.; Stalidzans, E. Challenges in the development of Precision Beekeeping. Biosyst. Eng. 2015, 130, 60–71. [Google Scholar] [CrossRef]
- Zacepins, A.; Kviesis, A.; Ahrendt, P.; Richter, U.; Tekin, S.; Durgun, M. Beekeeping in the future—Smart apiary management. In Proceedings of the 2016 17th International Carpathian Control Conference (ICCC), High Tatras, Slovakia, 29 May–1 June 2016; pp. 808–812. [Google Scholar] [CrossRef]
- DeGrandi-Hoffman, G.; Curry, R. A mathematical model of Varroa mite (Varroa destructor Anderson and Trueman) and honeybee (Apis mellifera L.) population dynamics. Int. J. Acarol. 2004, 30, 259–274. [Google Scholar] [CrossRef]
- Voudiotis, G.; Kontogiannis, S.; Pikridas, C. Proposed Smart Monitoring System for the Detection of Bee Swarming. Inventions 2021, 6, 87. [Google Scholar] [CrossRef]
- Bilik, S.; Kratochvila, L.; Ligocki, A.; Bostik, O.; Zemcik, T.; Hybl, M.; Horak, K.; Zalud, L. Visual Diagnosis of the Varroa Destructor Parasitic Mite in Honeybees Using Object Detector Techniques. Sensors 2021, 21, 2764. [Google Scholar] [CrossRef]
- Camayo, A.I.C.; Munoz, M.A.C.; Corrales, J.C. ApIsoT: An IoT Function Aggregation Mechanism for Detecting Varroa Infestation in Apis mellifera Species. Agriculture 2024, 14, 846. [Google Scholar] [CrossRef]
- Botias, C.; Martin-Hernandez, R.; Garrido-Bailon, E.; Gonzalez Porto, A.; Martinez-Salvador, A.; De la Rua, P.; Meana, A.; Higes, M. The growing prevalence of Nosema ceranae in honey bees in Spain, an emerging problem for the last decade. Res. Vet. Sci. 2011, 93, 150–155. [Google Scholar] [CrossRef]
- Erler, S.; Lommatzsch, S.; Lattorff, H.M.G. Comparative analysis of detection limits and specificity of molecular diagnostic markers for three pathogens (Microsporidia, Nosema spp.) in the key pollinators Apis mellifera and Bombus terrestris. Parasitol. Res. 2012, 110, 1403–1410. [Google Scholar] [CrossRef]
- Mazur, E.D.; Gajda, A.M. Nosemosis in Honeybees: A Review Guide on Biology and Diagnostic Methods. Appl. Sci. 2022, 12, 5890. [Google Scholar] [CrossRef]
- Yuan, Y.; Liu, G.; Chai, Y.; Yuan, R. Electrochemical proximity assay-coupled highly nonenzymatic amplifying strategy for total protein of Nosema bombycis detection. Sens. Actuators B Chem. 2017, 246, 402–407. [Google Scholar] [CrossRef]
- Snow, W.J.; Koydemir, H.C.; Kerim Karinca, D.; Liang, K.; Tseng, D.; Ozcan, A. Rapid imaging, detection, and quantification of Nosema ceranae spores in honey bees using mobile phone-based fluorescence microscopy. Lab Chip 2019, 19, 789–797. [Google Scholar] [CrossRef]
- Dghim, S.; Travieso-González, C.M.; Burget, R. Analysis of the Nosema Cells Identification for Microscopic Images. Sensors 2021, 21, 3068. [Google Scholar] [CrossRef]
- Sharif, M.Z.; Di, N.; Yu, B. Honeybee (Apis spp.) (Hymenoptera: Apidae) Colony Monitoring Using Acoustic Signals from the Beehive: An Assessment by Global Experts and Our Feedback. Agriculture 2023, 13, 769. [Google Scholar] [CrossRef]
- Nolasco, I.; Terenzi, A.; Cecchi, S.; Orcioni, S.; Bear, H.L.; Benetos, E. Audio-based Identification of Beehive States. In Proceedings of the ICASSP 2019—2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Brighton, UK, 12–17 May 2019; pp. 8256–8260. [Google Scholar] [CrossRef]
- Cejrowski, T.; Szymanski, J.; Mora, H.; Gil, D. Detection of the Bee Queen Presence Using Sound Analysis. In Intelligent Information and Database Systems: 10th Asian Conference, ACIIDS 2018, Dong Hoi City, Vietnam, 19–21 March 2018, Proceedings, Part II 10; Springer International Publishing: Cham, Switzerland, 2018; pp. 297–306. [Google Scholar] [CrossRef]
- Barbisan, L.; Turvani, G.; Riente, F. A Machine Learning Approach for Queen Bee Detection Through Remote Audio Sensing to Safeguard Honeybee Colonies. IEEE Trans. Agric. Food Electron. 2024, 1–8. [Google Scholar] [CrossRef]
- Ruvinga, S.; Hunter, G.; Duran, O.; Nebel, J.C. Identifying Queenlessness in Honeybee Hives from Audio Signals Using Machine Learning. Electronics 2023, 12, 1627. [Google Scholar] [CrossRef]
- Soares, B.; Luz, J.; Macêdo, V.; Silva, R.; Araújo, F.; Magalhães, D. MFCC-based descriptor for bee queen presence detection. Expert Syst. Appl. 2022, 201, 117104. [Google Scholar] [CrossRef]
- Aydin, S.; Aydin, M. Design and implementation of a smart beehive and its monitoring system using microservices in the context of IoT and open data. Comput. Electron. Agric. 2022, 196, 106897. [Google Scholar] [CrossRef]
- Ntawuzumunsi, E.; Kumaran, S.; Sibomana, L. Self-Powered Smart Beehive Monitoring and Control System (SBMaCS). Sensors 2021, 21, 3522. [Google Scholar] [CrossRef] [PubMed]
- Edwards-Murphy, F.; Magno, M.; Whelan, P.; O’Halloran, J.; Popovici, E. B+WSN: Smart beehive with preliminary decision tree analysis for agriculture and honey bee health monitoring. Comput. Electron. Agric. 2016, 124, 211–219. [Google Scholar] [CrossRef]
- Zhang, T.; Zmyslony, S.; Nozdrenkov, S.; Smith, M.; Hopkins, B. Semi-Supervised Audio Representation Learning for Modeling Beehive Strengths. arXiv 2021, arXiv:2105.10536. [Google Scholar]
- Arribas, F.A.; Hortelano, M.R. An Internet of Living Things based device for a better understanding of the state of the honey bee population in the hive during the winter months. Comput. Electron. Agric. 2023, 212, 108026. [Google Scholar] [CrossRef]
- Terenzi, A.; Cecchi, S.; Spinsante, S.; Orcioni, S.; Piazza, F. Real-time System Implementation for Bee Hives Weight Measurement. In Proceedings of the 2019 IEEE International Workshop on Metrology for Agriculture and Forestry (MetroAgriFor), Portici, Italy, 24–26 October 2019; pp. 231–236. [Google Scholar] [CrossRef]
- Meikle, W.G.; Holst, N.; Colin, T.; Weiss, M.; Carroll, M.J.; McFrederick, Q.S.; Barron, A.B. Using within-day hive weight changes to measure environmental effects on honey bee colonies. PLoS ONE 2018, 13, e0197589. [Google Scholar] [CrossRef]
- Anwar, O.; Keating, A.; Cardell-Oliver, R.; Datta, A.; Putrino, G. WE-Bee: Weight Estimator for Beehives Using Deep Learning: 36th AAAI Conference on Artificial Intelligence. In Proceedings of the AAAI Conference on Artificial Intelligence 2022, Virtually, 22 February–1 March 2022. [Google Scholar]
- Winston, M.L. Intra-colony demography and reproductive rate of the Africanized honeybee in South America. Behav. Ecol. Sociobiol. 1979, 4, 279–292. [Google Scholar] [CrossRef]
- Seeley, T.D.; Tautz, J. Worker piping in honey bee swarms and its role in preparing for liftoff. J. Comp. Physiol. A 2001, 187, 667–676. [Google Scholar] [CrossRef]
- Terenzi, A.; Cecchi, S.; Spinsante, S. On the Importance of the Sound Emitted by Honey Bee Hives. Vet. Sci. 2020, 7, 168. [Google Scholar] [CrossRef]
- Mesaros, A.; Heittola, T.; Ellis, D. Datasets and Evaluation. In Computational Analysis of Sound Scenes and Events; Virtanen, T., Plumbley, M.D., Ellis, D., Eds.; Springer International Publishing: Cham, Switzerland, 2018; pp. 147–179. [Google Scholar] [CrossRef]
- Terenzi, A.; Ortolani, N.; Nolasco, I.; Benetos, E.; Cecchi, S. Comparison of Feature Extraction Methods for Sound-Based Classification of Honey Bee Activity. IEEE/ACM Trans. Audio Speech Lang. Process. 2022, 30, 112–122. [Google Scholar] [CrossRef]
- Libal, U.; Biernacki, P. MFCC Selection by LASSO for Honey Bee Classification. Appl. Sci. 2024, 14, 913. [Google Scholar] [CrossRef]
- Hong, W.; Xu, B.; Chi, X.; Cui, X.; Yan, Y.; Li, T. Long-Term and Extensive Monitoring for Bee Colonies Based on Internet of Things. IEEE Internet Things J. 2020, 7, 7148–7155. [Google Scholar] [CrossRef]
- Kontogiannis, S. An Internet of Things-Based Low-Power Integrated Beekeeping Safety and Conditions Monitoring System. Inventions 2019, 4, 52. [Google Scholar] [CrossRef]
- Hamza, A.S.; Tashakkori, R.; Underwood, B.; O’Brien, W.; Campell, C. BeeLive: The IoT platform of Beemon monitoring and alerting system for beehives. Smart Agric. Technol. 2023, 6, 100331. [Google Scholar] [CrossRef]
- Esch, H. The sounds produced by swarming honey bees. Z. Vgl. Physiol. 1967, 56, 408–411. [Google Scholar] [CrossRef]
- Wenner, A.M. Sound Communication in Honeybees. Sci. Am. 1964, 210, 116–125. [Google Scholar] [CrossRef]
- Michelsen, A.; Kirchner, W.H.; Lindauer, M. Sound and vibrational signals in the dance language of the honeybee, Apis mellifera. Behav. Ecol. Sociobiol. 1986, 18, 207–212. [Google Scholar] [CrossRef]
- Dornhaus, A.; Chittka, L. Why do honey bees dance? Behav. Ecol. Sociobiol. 2004, 55, 395–401. [Google Scholar] [CrossRef]
- Ishay, J.S.; Sadeh, D. The sounds of honey bees and social wasps are always composed of a uniform frequency. J. Acoust. Soc. Am. 1982, 72, 671–675. [Google Scholar] [CrossRef]
- Simpson, J. The mechanism of honey-bee queen piping. Z. Vgl. Physiol. 1964, 48, 277–282. [Google Scholar] [CrossRef]
- Ferrari, S.; Silva, M.; Guarino, M.; Berckmans, D. Monitoring of swarming sounds in bee hives for early detection of the swarming period. Comput. Electron. Agric. 2008, 64, 72–77. [Google Scholar] [CrossRef]
- Simonyan, K.; Zisserman, A. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv 2015, arXiv:1409.1556. [Google Scholar]
- He, K.; Zhang, X.; Ren, S.; Sun, J. Deep Residual Learning for Image Recognition. arXiv 2015, arXiv:1512.03385. [Google Scholar]
- Szegedy, C.; Vanhoucke, V.; Ioffe, S.; Shlens, J.; Wojna, Z. Rethinking the Inception Architecture for Computer Vision. arXiv 2015, arXiv:1512.00567. [Google Scholar]
- Szegedy, C.; Ioffe, S.; Vanhoucke, V.; Alemi, A. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. arXiv 2016, arXiv:1602.07261. [Google Scholar] [CrossRef]
- Anwar, A. Difference between alexnet, vggnet, resnet and inception. Medium-Towards Data Sci. 2019. Available online: https://towardsdatascience.com/the-w3h-of-alexnet-vggnet-resnet-and-inception-7baaaecccc96 (accessed on 11 February 2021).
- Muhammad, U.; Wang, W.; Chattha, S.P.; Ali, S. Pre-trained VGGNet Architecture for Remote-Sensing Image Scene Classification. In Proceeding of the 2018 24th International Conference on Pattern Recognition (ICPR), Beijing, China, 20–24 August 2018; Volume 1, pp. 1622–1627. [Google Scholar] [CrossRef]
- Bagaskara, A.; Suryanegara, M. Evaluation of VGG-16 and VGG-19 Deep Learning Architecture for Classifying Dementia People. In Proceeding of the 2021 4th International Conference of Computer and Informatics Engineering (IC2IE), Depok, Indonesia, 14–15 September 2021; Volume 1, pp. 1–4. [Google Scholar] [CrossRef]
- Hindarto, D. Comparison of detection with transfer learning architecture RESTNET18, RESTNET50, RESTNET101 on corn leaf disease. J. Teknol. Inf. Univ. Lambung Mangkurat (JTIULM) 2023, 8, 41–48. [Google Scholar] [CrossRef]
- Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015; Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F., Eds.; Springer International Publishing: Cham, Switzerland, 2015; pp. 234–241. [Google Scholar] [CrossRef]
- McNeely-White, D.; Beveridge, J.R.; Draper, B.A. Inception and ResNet features are (almost) equivalent. Cogn. Syst. Res. 2020, 59, 312–318. [Google Scholar] [CrossRef]
- Zagoruyko, S.; Komodakis, N. Wide Residual Networks. arXiv 2017, arXiv:1605.07146. [Google Scholar]
- Kontogiannis, S.; Koundouras, S.; Pikridas, C. Proposed Fuzzy-Stranded-Neural Network Model That Utilizes IoT Plant-Level Sensory Monitoring and Distributed Services for the Early Detection of Downy Mildew in Viticulture. Computers 2024, 13, 63. [Google Scholar] [CrossRef]
- Kontogiannis, S.; Gkamas, T.; Pikridas, C. Deep Learning Stranded Neural Network Model for the Detection of Sensory Triggered Events. Algorithms 2023, 16, 202. [Google Scholar] [CrossRef]
- ThingsBoard. ThingsBoard Open-Source IoT Platform. 2019. Available online: https://thingsboard.io/ (accessed on 15 October 2020).
- McFee, B.; Raffel, C.; Liang, D.; Ellis, D.P.W.; McVicar, M.; Battenberg, E.; Nieto, O. Librosa: Audio and Music Signal Analysis in Python. In Proceedings of the 14th Python in Science Conference, Austin, TX, USA, 6–12 July 2015; Huff, K., Bergstra, J., Eds.; Brian McFee, Center for Data Science, New York University: New York, NY, USA, 2015; pp. 18–24. [Google Scholar] [CrossRef]
- Scikit-Learn: Machine Learning in Python. 2014. Available online: https://github.com/scikit-learn/scikit-learn (accessed on 22 March 2016).
- PyTorch–An Open Source Machine Learning Framework That Accelerates the Path from Research Prototyping to Production Deployment. 2018. Available online: https://www.pytorch.org (accessed on 22 November 2020).
- GoogleTF. Tensorflow 2.0: A Machine Learning System for Deep Neural Networks. 2020. Available online: https://tensorflow.org (accessed on 22 March 2020).
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. arXiv 2017, arXiv:1412.6980. [Google Scholar]
Beegram Measures | Correlation Functions Calculated from the Beegram Vector Elements | Critical Event Values |
---|---|---|
Beehive hunger measure (Bh) | , | |
Beehive growth measure (Bg) | ||
Queen tone measure (Qt) | , | |
Beehive thermal stress (Th) | ||
Swarming measure (Sw) |
Fuzzy Sets | Humidity Dry | Humidity Normal | Humidity Wet |
---|---|---|---|
Temperature cold | minus (class-0) | class-0 | plus (class-0) |
Temperature disease | class-2 | class-1 | plus (class-1) |
Temperature normal | class-2 | plus (class-2) | minus (class-1) |
Temperature hot | plus (class-3) | class-3 | minus (class-2) |
Model | Input Size | No. Trainable Params (Mil) | Params Size (MB) | Total Size (MB) | Cloud Inference Speed (s) |
---|---|---|---|---|---|
Stranded-NN-96 (1d-1h-4mp) | (96,) | 0.013 | 0.054 | <1 MB | 0.015 |
Stranded-NN-144 (1d-1h-6mp) | (144,) | 0.046 | 0.221 | <1 MB | 0.019 |
ResNet-18 | (224, 224) | 11.691 | 44.60 | 106.25 | 0.024 |
Inception_v3 | (224, 224) | 19.534 | 74.52 | 150.72 | 0.057 |
ResNet-50 | (224, 224) | 21.280 | 81.18 | 177.65 | 0.040 |
Half-UNet | (224, 224) | 19.636 | 74.91 | 441.86 | 0.084 |
VGG-16 | (224, 224) | 134.288 | 512.27 | 834.53 | 0.117 |
VGG-19 | (224, 224) | 139.600 | 532.53 | 884.53 | 0.139 |
WideResNet-40 | (224, 224) | 34.13 | 35.12 | 1939.70 | 0.338 |
Model | Accuracy|Conv. 20 Epochs | Accuracy|Conv. 50 Epochs | Accuracy|Conv. 100 Epochs |
---|---|---|---|
VGG-16 | 58.52|1.22 | 72.02|0.73 | 83.04|0.12 |
VGG-19 | 88.90|2.14 | 92.32|0.02 | 92.75|0.011 |
Inception_v3 | 53.62|1.48 | 57.52|0.23 | 63.53|0.295 |
Half-UNet | 96.06|0.33 | 96.84|−0.18 | 96.31|0.14 |
ResNet-18 | 95.89|0.47 | 95.71|−0.034 | 93.88|−0.11 |
ResNet-50 | 97.79|0.16 | 98.79|0.0016 | 99.26|0.006 |
WideResNet-40 | 96.03|0.52 | 97.52|0.08 | 98.41|0.044 |
Stranded-NN-96 (1d-1h-4mp) | 78.5|1.56 | 88.7|0.92 | 93.6|0.02 |
Stranded-NN-144 (1d-1h-6mp) | 84.6|1.78 | 94.6|0.45 | 95.2|0.01 |
Model | Data Transform Time (s) | Model Load Time (s) | Inference Time (s) |
---|---|---|---|
Inception_v3 | 99.932 | 4.6 | 50.214 |
Half-UNet | 99.932 | 5.6 | 234.778 |
ResNet-18 | 99.932 | 1.59 | 23.616 |
ResNet-50 | 99.932 | 8.2 | 67.153 |
WideResNet-40 | 99.932 | 48.2 | allocation issues at least 1.2 GB of RAM required |
VGG-16 | 99.932 | 63.1 | 285.7 |
VGG-19 | 99.932 | 97.5 | 421.5 |
Stranded-NN-96 | - | 0.8 | 6.2 |
Stranded-NN-144 | - | 0.8 | 8.6 |
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2024 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Share and Cite
Kontogiannis, S. Beehive Smart Detector Device for the Detection of Critical Conditions That Utilize Edge Device Computations and Deep Learning Inferences. Sensors 2024, 24, 5444. https://doi.org/10.3390/s24165444
Kontogiannis S. Beehive Smart Detector Device for the Detection of Critical Conditions That Utilize Edge Device Computations and Deep Learning Inferences. Sensors. 2024; 24(16):5444. https://doi.org/10.3390/s24165444
Chicago/Turabian StyleKontogiannis, Sotirios. 2024. "Beehive Smart Detector Device for the Detection of Critical Conditions That Utilize Edge Device Computations and Deep Learning Inferences" Sensors 24, no. 16: 5444. https://doi.org/10.3390/s24165444