Next Article in Journal
The Spatio-Temporal Patterns and Influencing Factors of Different New Agricultural Business Entities in China—Based on POI Data from 2012 to 2021
Next Article in Special Issue
Division of Cow Production Groups Based on SOLOv2 and Improved CNN-LSTM
Previous Article in Journal
Development and Experimental Validation of an Agricultural Robotic Platform with High Traction and Low Compaction
Previous Article in Special Issue
Soybean-MVS: Annotated Three-Dimensional Model Dataset of Whole Growth Period Soybeans for 3D Plant Organ Segmentation
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Identifying an Image-Processing Method for Detection of Bee Mite in Honey Bee Based on Keypoint Analysis

1
Department of Interdisciplinary Program in Smart Agriculture, Kangwon National University, Chuncheon 24341, Republic of Korea
2
Agriculture and Life Sciences Research Institute, Kangwon National University, Chuncheon 24341, Republic of Korea
3
Apiculture Division, National Institute of Agricultural Science, 310 Nongsaengmyeng-ro, Deokjin-gu, Jeonju 54875, Republic of Korea
4
Department of Mechatronics Engineering, Korea Polytechnics, 56 Munemi-ro 448 beon-gil, Bupyeong-gu, Incheon 21417, Republic of Korea
5
Department of Biosystems Engineering, Kangwon National University, Chuncheon 24341, Republic of Korea
*
Author to whom correspondence should be addressed.
Agriculture 2023, 13(8), 1511; https://doi.org/10.3390/agriculture13081511
Submission received: 11 June 2023 / Revised: 14 July 2023 / Accepted: 20 July 2023 / Published: 28 July 2023

Abstract

:
Economic and ecosystem issues associated with beekeeping may stem from bee mites rather than other bee diseases. The honey mites that stick to bees are small and possess a reddish-brown color, rendering it difficult to distinguish them with the naked eye. Objective and rapid technologies to detect bee mites are required. Image processing considerably improves detection performance. Therefore, this study proposes an image-processing method that can increase the detection performance of bee mites. A keypoint detection algorithm was implemented to identify keypoint location and frequencies in images of bees and bee mites. These parameters were analyzed to determine the rational measurement distance and image-processing. The change in the number of keypoints was analyzed by applying five-color model conversion, histogram normalization, and two-histogram equalization. The performance of the keypoints was verified by matching images with infested bees and mites. Among 30 given cases of image processing, the method applying normalization and equalization in the RGB color model image produced consistent quality data and was the most valid keypoint. Optimal image processing worked effectively in the measured 300 mm data in the range 300–1100 mm. The results of this study show that diverse image-processing techniques help to enhance the quality of bee mite detection significantly. This approach can be used in conjunction with an object detection deep-learning algorithm to monitor bee mites and diseases.

1. Introduction

Honeybee is a pollinating insect that maintains the ecosystem. Honeybees possess the ability to produce honey, wax, and royal jelly for beekeeping. However, beekeeping is experiencing a dual crisis of earning-shock and colony collapse disorder due to climate change, pests, and disease [1,2].
Among pests, Varroa destructor is the most severe, and may lead to several economic disadvantages compared to other diseases [3]. Bee mites can parasitize larvae and bees, and this may further result in growth decline, wing deformity, abdominal reduction, and death [4]. Methods for detecting bee mites include sugar testing, brood testing, and floor testing, but they have limitations in providing objective, quantitative indicators. Bee mite management is one of the main tasks of beekeeping managers, and research exists to prevent and control it [5,6].
Managing pest inspection in the beehive state is required by beekeeping farmers. Typically, checking by humans involves observation with the naked eye and nonobjective knowledge. Bee mites have a small size of 1.1 mm × 1.6 mm and are reddish brown. Their color is similar to that of the bee pattern. Hence, their identification is difficult, and a distinct deviation may be present, depending on the skill of the beekeepers. This has led to the need for rapid and objective detection methods.
Considerable visual information can be distinguished during beekeeping. A few examples include honey, bees, queen bees, bee larvae, diseases, and pests. Visual data have been extensively used in computer science analyses. Each class is classified into an image using deep learning. Object detection algorithms are fast and non-destructive approaches to detect bee mites in beecomb images.
Computer vision is the field of computing that uses image data. Computer vision systems have been widely used in machinery, medicine, and agriculture. In precision agriculture, a weak classifier model has been developed using object detection [7]. In a recent study, a banana disease detection model was built using a neural network and transfer learning [8].
Several efforts have been made to achieve more precise beekeeping using computer vision systems. Ngo et al. developed a monitoring system that possessed the ability to count the number of bees at the entrance [9]. Bjerge et al. constructed a measurement system at the entrance to a hive and attempted to monitor bee mites using near-infrared and deep learning [10].
Artificial intelligence used for object detection learns from object keypoints, which are regarded the most important values during image matching, detection, and tracking. Increasing the number of enhanced keypoints helps improve the detection performance.
The keypoint detection algorithm is primarily affected by the measurement environment, even for the same object. The inference performance was changed using a detector. Thus, a keypoint detector must be selected based on its speed and accuracy [11]. Image matching was based on the keypoints of each object. It can connect to similar keypoints. The matching quality is affected by keypoint frequency and location.
This study aims to develop an image processing method in order to improve the quality of bee mite detection. A beecomb measurement system must be developed for image acquisition. A keypoint detector was used to estimate the keypoints. The frequencies and locations of the keypoints were analyzed using a rational image processing method. The image processing methods implemented are color model conversion, histogram normalization, and equalization. The combination of image processing generated 30 analysis cases.

2. Materials and Methods

2.1. Materials and Location

Eight beehives were used for image data measurements at the apiary of the National Institute of Agricultural Sciences in Jeollabuk-do, Republic of Korea, and at a beekeeping farm in Gangwon-do. The honey bee is a Western honey bee (Apis mellifera) that is adaptable to the environment and yields high productivity. In another study, two common species of bee mites, Varroa jacobsoni and Varroa destructor, were measured, which possess dimensions of 1.0630 × 1.5068 mm and 1.1673 × 1.7089 mm, respectively [12]. The observed bee mite had a size of 1.2 × 1.7 mm. Therefore, it was assumed to be Varroa destructor (Figure 1).

2.2. RGB Image Acquisition System and Measurement Method

An image acquisition system must be established to define an optimal image-processing method to detect bees and bee mites. Image data were acquired for bees and bee mites in beecombs in a manner similar to that for human inspection.
The image acquisition system was built using a camera, a laptop, and a beecomb supporter (Figure 1). The supporter can directly control this angle. A CMOS-type BlackflySGigE camera (FLIR, Wilsonville, OR, USA), with a resolution of 2048 × 1536 pixels, was used in this study.
The measurement software was developed in Python 3.7. An acquisition area was set for the entire beecomb, which was the same as that visually inspected by humans. The numbers of shots required varied depending on the distance from the camera to the beecomb. The images were measured at five shooting distances at 200-mm intervals from 300 mm to 1100 mm, and the number of measurements per distance is shown in Table 1. The number of image measurements was set to measure one side of the beecomb.
Image acquisition of the beecomb with bees and bee mites was performed in apiary. After adjusting the distance between the camera and the beecomb, the angle and position of the camera and supporter were set. The camera and support angle were fixed at 15 degrees in order to prevent light saturation. The aperture and exposure time were manually adjusted in response to changes in environmental factors, such as changing sunlight and weather.

2.3. Bee and Bee Mite Image Dataset

An image was selected and a region of interest was extracted from the measured RGB image data (Figure 2). The selected image contained bee mites, and the region of interest was infested with bees (parasitized by Varroa destructor). A total 65 images were extracted at five measurement distances, with 13 images at each level, and 65 images were used in the analysis (Figure 3).

2.4. Optimal Image Processing

To determine the optimal image processing method for detecting bee mites, an analysis based on keypoints and image matching must be performed after image processing. Various image processing methods, such as color model conversion, histogram normalization, and equalization, have been applied to improve the matching rate of bee mites in beekeeping images. There were a total of 30 image processing combinations, and these were applied to the image of the extracted infected bees (i.e., bees parasitized by a bee mite). The image processing methods are shown in Section 2.4.1 and Section 2.4.2.

2.4.1. Color Model Conversion

To identify the characteristics of the infected bees that did not appear in the color model of the existing image, five color model conversions were performed. The color models RGB, HSV, Lab, YCrCb, and Gray were representative color models classified according to the mixing method, brightness component, and color-difference component.
According to previous research, RGB is more suitable for neural network learning compared to the one-dimensional value according to the H values of RGB and HSV [13]. A color model refers to 3D array data for expressing colors, and each dimension has a component value for implementing the color. The three dimensions of the RGB model represent the components red, green, and blue, and the HSV model represents the components hue, saturation, and brightness. The YCrCb model consists of brightness and color difference information (Cr and Cb), and the Lab model consists of brightness, red-green, and yellow-blue components. The HSV, YCrCb, and Lab had a common component that represented brightness. The gray color model represents one-dimensional array data. Therefore, gray represents only the intensity of a pixel.
The cvtColor function of OpenCV was used for the color model conversion. The color model conversion equations are as follows (OpenCV, 2022): color model conversion was performed based on a floating-point number with a value between 0 and 1, substituted from the RGB model data. After the model-change formula was applied, it was redefined as 8-bit data, with values ranging from 0 to 255. In the case of HSV, YCrCb, and Gray, they were converted at once in a specific way corresponding to the coefficients defined, as in Equations (1), (3) and (4). The Lab case was converted to the color model XYZ, as shown in Equation (2):
V   =   max R ,   G ,   B   S   =   V min R , G , B V 0 if   V       0 otherwise
  H   =   { 60 G B   /   V min R , G , B 120   +   60 B R / V min R , G , B 240 + 60 R B / V min R , G , B 0 } if   V     =   R if     V   =   G if     V   =   B         if     R = G = B
X Y Z   =   0.412453       0.357580       0.180423 0.212671       0.715160       0.072169 0.019334       0.119193       0.950227 R G B
X   =   X X n   ,   where   X n   =   0.950456
Z   =   Z Z n   ,   where   Z n   =   1.088754  
L   =   116     Y 1 3   16 903.3     Y   for   Y   > 0.008856     for   Y 0.008856
a   =   500 f X   f Y   +   128
b   =   200 f X     f Z   +   128
f t   =   { t 1 / 3 7.787 t + 16 / 116 }   for       t   >   0.008856   for       t   0.008856
Y   =   0.299 R   +   0.587 G   +   0.114 B
Cr   =   R Y 0.713   + 128
Cb   =   B Y 0.564   +   128
Gray   =   0.299 R   +   0.587 G   +   0.114 B
R   =   Pixel of intensity R channel
G   =   Pixel of intensity G channel
B   =   Pixel of intensity B channel

2.4.2. Histogram Normalization and Equalization

An image acquisition experiment was performed outdoors according to the same condition as that of a visual inspection by a beekeeper. In outdoor image acquisition experiments, the intensity of sunlight changed depending on factors such as measurement time, clouds, and weather. Sunlight variations caused deviations in the measurement data. In other words, measurement errors, such as sunlight, deviation of appropriate exposure time, and aperture value, may occur. Histogram calibration may reduce further deviations due to changes in light intensity. Histogram normalization and equalization was one of the methods used to calibrate the intensity of each component. Both histogram correction methods could normalize data and enhance contours and contrast.
The minimum–max normalization was calculated using Equation (5). In the Equalization method, there were various algorithms, such as Global Histogram Equalization (GHE), Local Histogram Equalization (LHE), and Dynamic Histogram Equalization (DHE) [14]. Histogram equalization was performed using the cumulative distribution in Equation (6). Global Histogram Equalization and Contrast-Limited Adaptive Histogram Equalization (CLAHE), which are calculated by dividing the image into a grid, were selected for the equalization method.
In this study, normalization (not applied, applied) and equalization (not applied, GHE, and CLAHE) were applied to color-converted images for a detailed comparison of the effects of histogram correction. The color image consisted of three channels. The normalized channels differed for each color model. In the HSV, YCrCb, and Lab color models, normalization was applied to brightness components, and the RGB and the Gray color models were applied to all components:
I normalization = I original Min original 255 Max original Min original
  • Inormalization: Normalized image
  • Ioriginal: Original image
  • Maxoriginal: Maximum pixel value of original image
  • Minoriginal: Minimum pixel value of original image
  H v = round cdf v cdf min M N   cdf min L 1
  • H′(v): Equalized Histogram
  • v: Value of pixel
  • round(v): Rounds Function
  • cdf(v): Histogram cumulative function
  • cdfmin: Minimum cumulative value, usually 1
  • M ∗ N: Resolusion of image,(M: Width, N: Height)
  • L: Range of pixel value, 256

2.5. Keypoint Detection Algorithm of Bees and Bee Pests

A keypoint is the point at which an object can be distinguished locally. This is used as a matching point for object matching, detection, and tracking. In addition, as an essential factor, the keypoint must be derived in order to recognize an object or structure using a computer. Therefore, as recognition points for objects such as honeybees and bee mites, the frequency and the location accuracy of keypoints can be used to evaluate the quality of the images to which image processing was applied.
The keypoint detection algorithm should be selected according to the data characteristics, and both its speed and its accuracy may vary depending on the analysis hardware [11]. There were research to identify bee pollen with RGB image and the vector of locally aggregated descriptors encoded by the Scale-Invariant Feature Transform (SIFT) keypoint detection algorithm [15]. Oriented FAST and Rotated BRIEF (ORB) are keypoint detection algorithms based on the Features from Accelerated Segment Test (FAST) that is applied to real-time systems and the Binary Robust Independent Elementary Features (BRIEF) with rotation invariance [16]. The keypoints of each patch were detected using FAST, and efficient points were calculated among the detected keypoints based on the BRIEF descriptor. Among four keypoint detection algorithms (BRISK, SIFT, SURF, and ORB), the ORB algorithm showed the best performance in terms of evaluation of feature point frequency, calculation efficiency, matching efficiency, and detection speed [17]. As the distortion of an image varies depending on the type of camera or lens, distortion correction is necessary. A comparison of the detection and matching performance of SIFT, SURF, and ORB for distortion based on data with 30% salt-and-pepper noise compared to the original showed that the ORB algorithm was the best [18]. Thus, the ORB algorithm was applied to data with minimal image-warping distortion.

2.6. Performance of Keypoint

This study aimed to investigate the optimal measurement distance and image-processing method for honeybee and bee mite recognition. The keypoint detection performance was based on frequency and location accuracy analysis for each image processing step.

2.6.1. Analysis of Keypoint Location and Frequency

The keypoints detected through the ORB are composed of an object that stores keypoint information and an object that stores descriptor information. The stored keypoint information was stored, and it had the following values: pt, size, angle, response, octave, and class_id. (here, pt denotes the location of a feature point). Therefore, by contrasting the values of pt, the regions of bees, and bee mites, it is possible to determine the frequency of keypoints that would actually be used for object matching.
The location information of the bee mites in the images is labeled in a boxed JSON format. Bee mite region information can also be used to extract the mite area within an image.
The pt component of the keypoint was obtained from both the original and each processed image. The pt components of the extracted keypoints were compared with the coordinates of the bee mites, and the number of valid keypoints for bee mite identification for each image processing step was calculated.

2.6.2. Image Matching Algorithm

Image matching algorithms could match detected keypoints in two images. Matching performance was affected by the quality of the keypoints in the image. Image matching was performed to verify the performance of the detected bee and honeybee keypoints and to compare the changes according to the image processing method.
The BFMatcher function in OpenCV was used for image matching. The BFMatcher is an algorithm that uses a brute-force match to compute all matchable keypoints in order to produce good results. The matching parameters for the brute-force operations were NORM_HAMMING, which uses the Hamming distance, and CrossCheck, which determines whether the matching results in both directions are the same. The image matching result is represented as Dmatch with four components: queryIdx, trainIdx, imgIdx, and distance. QueryIdx and trainIdx were the indices of the keypoints that are detected in the images used for matching. The imgIdx is the component that is used when matching multiple images, and the distance is the matching value between the keypoint vectors. A small matching distance indicates a high similarity. The components of the matching results are sorted in order of decreasing distance to select the top-matching objects with high matching similarity.
In this study, image matching was implemented based on the original image, and an image with histogram normalization and equalization. The top ten matching objects were selected based on the distance component to compare the matching performance. The selected matching objects are checked for anomalous matches. An abnormal match is observed when different points on an object are matched.
The overall process of applying image processing, keypoint detection, and image matching to 150 conditions under five measurement distance conditions and 30 image processing combinations is shown in the flowchart in Figure 4.

3. Results and Discussion

3.1. Color Model Conversion and Histogram Analysis of Bee and Bee Mite Image

To identify the optimal color model for detecting honeybee and bee mite keypoints, RGB images were converted into four color models (Gray, HSV, Lab, and YCrCb). The image and histogram analyses of the original RGB image and the converted color models are shown in Figure 5. The average intensity of the 13 images for each color model was used for histogram analysis.
The measured image data exhibited values in the range 0–255. The RGB color model analysis showed that values 0–2 were not present in the R and G channels, and values 0–1 were not present in the B channel. In addition, the values 197–255 for G and those for 181–255 for B were not present. In the HSV color model, values 179–255 in the H channel, 224–255 in the S channel, and 0–3 in the V channel were not present.
In the LAB, the distribution was skewed toward values between 120 and 135 in channels A and B, with no values between 0 and 1 and 200 and 255 in channel L; 0 and 105 and 159 and 255 in channel A; and 0 and 78 and 166 and 255 in channel B. YCrCb had a distribution in which the frequencies of the Cr and Cb channels were clustered around values between 120 and 135, with 0–2 and 192–255 for the Y channel; 0–98 and 155–255 for the Cr channel; and 0–99 and 173–255 for the Cb channel. The single-channel color model, gray, had no values between 0 and 2 and 192 and 255.
The distribution of the color values tended to be skewed toward some specific values rather than the full range, and some values were empty. Therefore, normalization was required to ensure that the color component values were evenly distributed over the range of 0–255.

3.2. Histogram Normalization and Equalization for Bee and Bee Mite Image

Honeybee and bee mite images were subjected to histogram normalization and equalization. After image processing, each beekeeping image was converted into 30 images, including the original image. As shown in Figure 6, when histogram normalization was applied, the pixel values were distributed in the range 0–255 and the contrast was improved.
The normalization algorithm required the maximum and minimum values of the data (Equation (1)). If the measured data possessed values in the range 0–255, the normalization algorithm might not work correctly. Figure 7 shows the normal and abnormal operations of histogram normalization. If the values at either end of the distribution are 0 or 255, the normalization algorithm will not work properly, and contrast improvement cannot be expected.
As shown in Figure 8, histogram equalization resulted in a relative improvement in the contrast compared to the original. The equalization of the entire basis and CLAHE yielded different results. The CLAHE method divides an image into grids and equalizes each grid. This enhances the unique color of bees and cells. However, global equalization is applied according to the entire image, which further improves the overall brightness.

3.3. Detection of Keypoints in Bee and Bee Mite Image

The ORB was applied to the beekeeping images in 150 different cases to detect the keypoints. The results showed that most keypoints were detected at a shooting distance of 300 mm. The number of keypoints tended to decrease as the shooting distance increased (Figure 9). These results suggest that resolution-dependent measurement distances should be considered when recognizing bees using images.
The numbers of keypoints detected were compared using a color model. The average number of detected keypoints in RGB and Gray were 152 and 155, and the mean error rate and standard deviation were 1.03% and 1.39%, respectively. Similar performance results were obtained, but the Gray model required a color model conversion from the original data.
In the case of the YCrCb and Lab color models, up to four features were detected in the images that were measured at a distance of 300 mm, with average detection frequency of one and two. Therefore, it is not ideal to use the color models Lab and YCrCb to analyze bees and bee mites.
The HSV color model detected the most keypoints at all measurement distances. However, for HSV, the detected keypoints were often not located in the bee or bee mite zones (Figure 10b). An increase in non-object keypoints may result in a decrease in the matching rate of the bee mites. The keypoints for object recognition must be used accurately as matching points, otherwise inaccurate recognition may occur. Based on the comparison of the keypoint detection of five different color models (RGB, HSV, Lab, YCrCb, and Gray), we determined that the RGB color model was suitable for beekeeping monitoring.
The average keypoint detection performance increased by 44%, from 278 to 398, using the normalization algorithm (Table 2). Among the GHE and CLAHE methods used for equalization, a higher number of keypoints was detected using GHE. However, the GHE-detected keypoints were not specific to bees and bee mites (Figure 10). Therefore, CLAHE is more suitable than GHE as an image-processing method for bee-monitoring data.
Based on the original data, an average of two keypoints was detected at a shooting distance of 700 mm, an average of one keypoint at 900 mm, and zero keypoints at 1100 mm. Compared with the 300-mm image data, the number of keypoints detected at a distance of 500 mm was reduced by 69%, from 67 to 21. For images with measurement distances of 700, 900, and 1100 mm, the detection performance decreased by 97%, 99%, and 100%, respectively, compared to the 300-mm image. In particular, the images measured at a distance of 1100 mm with only one or two keypoints were detected even after image processing. Images measured at distances greater than 1100 mm were not available for analysis.
The keypoint detection performances were compared by applying 30 image-processing methods. Histogram normalization and equalization can help improve the image contrast and subsequently increase the number of keypoints. However, normalization may or may not be applicable depending on the histogram distribution, and, thus, equalization should also be applied. Consequently, the optimal image processing conditions were the application of histogram normalization and histogram equalization (CLAHE) to the RGB color model. The RGB color model can be effective for analysis because it can represent the reddish brown of bee mite more effectively than other color models.

3.4. Validation and Histogram Analysis Based on Optimal Image Processing

To verify the performance of the optimal image-processing conditions, the frequency of the keypoints was analyzed. Performance was validated using a bee image that was captured at a distance of 300 mm. When normalization was applied, the distribution of pixels was split between 0 and 255 and the number of keypoints increased by 399% (Figure 11b). Equalization improved the number of keypoints by 269% over the normalized data by spreading out the distributions concentrated on a few values (Figure 11c). By integrating an RGB color model, normalization, and equalization (CLAHE) to bee and bee mite images, the quality was enhanced considerably. The processed images contained more keypoints. This image-processing method improved the recognition rate of honeybees and mites. Image processing methods that affect an image locally were more effective than methods that affect the entire image. Image processing to homogenize an image to distinguish objects sharpened the image. Objects in images with increased sharpness have more points that are distinct (darker or lighter) from their surroundings, and the frequency of feature points may increase.

3.5. Analyzing Frequency of Keypoints in Bee Mite

The average values of the numbers of keypoints in the original and processed images are shown in Figure 12. When image processing was applied to the data that were measured at a distance of 300 mm, the number of keypoints was the highest, with an average of 31 in the bee mite area. This was approximately 340% higher than the frequency before image processing was applied. Through optimal image processing, 500 mm, 700 mm, and 900 mm data showed an increase of 380%, 1733%, and 2400% in keypoints, respectively, for bee mites. The data measured at a distance of 1100 mm did not detect any keypoints in the mite area. Among beekeeping objects, such as bees, queens, and workers, the bee mite belongs to the small scale. Therefore, the increase in keypoints of bee mites was noteworthy. This may be a clue to solving the problem of simultaneous recognition of small and large objects.

3.6. Bee Mite Image Matching Results––Comparison of Top Ten Matching Objects

We checked whether the optimal image processing method could improve bee mite detection performance. The image matching with the coordinates of the bee mite was used for verification.
Image matching was used for the bee mite region. If the matching point was not correct, it was judged as an abnormal match (Figure 13).
The original data measured at a distance of 300 mm could not generate 10 matching objects, in accordance with the image. Abnormal matches were 3.7 (50%) based on an average of 7.4 matching objects. Given image processing, the top ten matching objects were generated from all images. For the processed images, an average of 3.7 (38%) abnormal matches were obtained in the top ten matching objects. Thus, through optimal image processing, the matching performance could be improved by 12% based on images with a measurement distance of 300 mm (Figure 14).
For the data measured at a distance of 500 mm, cases were present that produced fewer than 10 matching objects for each image. On average, 1.5 (64%) of the abnormal matchings were found in the 2.3 matching objects. Even with image processing, an image with a distance of 500 mm generated fewer than ten matching objects. Image processing increased the average number of matching objects to 4.9 but resulted in abnormal matching of 4.6 (94.4%). In other words, for a beecomb RGB image in a case where the measurement distance is longer than 500 mm, the bee mite-matching performance may be degraded.
For the images that were measured at distances greater than 700 mm, the object matching algorithm did not work regardless of the image processing. Given a camera with a resolution of 2048 × 1536, bee mite RGB data measured at a distance greater than 700 mm could not be used for image matching.

4. Conclusions

Bee mites cause more economic damage than other honeybee pests and diseases. Bee mites are small and reddish-brown in color, making it difficult to distinguish them from bees when attached to them. This has generated the need for technology that can objectively and quickly test for Varroa mite outbreaks. Image-based analytics, such as object detection, possess the potential to recognize bee mites. However, their small size and protective color may be a problem for computer vision systems.
Therefore, in this study, we applied image processing, keypoint detection, and image matching algorithms to images of bees and bee mites to improve the matching rate of bee mites. The frequency and location of the keypoints were analyzed and the quality of the matched objects was evaluated accordingly.
The analysis results for 30 combinations of image processing methods, including color model conversion, histogram normalization, histogram equalization, and five measurement distances, are as follows: applying normalization and equalization (CLAHE) based on the RGB color model to bee and bee mite images resulted in better keypoint detection by reinforcing the image quality. The effectiveness of the optimal image processing method was observed through the data that were measured at 300 mm of the 300–1100 mm measurement distance, with improved keypoint detection and matching performance. Regardless of image processing, it was difficult to match images to bee mites at the measurement distance of 700 mm or more. At measurement distances of 500 mm, image matching of bee mite images was possible, but with a high mismatch rate.
The improved matching quality can lead to improved detection performance of deep learning-based algorithms. The optimal image processing method and measurement distance for identifying bee mites can be used to simultaneously detect beekeeping objects with different sizes and shapes. The results of this study can be used as basic supporting data for recognizing bee mites, which are small objects, and bees, which are relatively large objects. In future research, we would like to apply this image processing condition to deep learning-based object detection to develop a model for identifying bee mites and beekeeping objects, such as bee, larva, cell, egg.

Author Contributions

Conceptualization, H.G.L., S.-b.K. and C.M.; methodology, H.G.L., M.-J.K. and C.M.; software, H.G.L. and H.L.; validation, H.G.L., S.-b.K. and S.L.; formal analysis, H.G.L.; investigation, M.-J.K.; resources, S.-b.K. and S.L.; data curation, H.G.L. and M.-J.K.; writing—original draft preparation, H.G.L.; writing—review and editing, C.M.; visualization, J.Y.S.; supervision, C.M.; project administration, C.M.; funding acquisition, C.M. All authors have read and agreed to the published version of the manuscript.

Funding

This study was supported by the Rural Development Administration as “Cooperative Research Program for Agriculture Science and Technology Development [Project Nos. PJ01582601, RS-2023-00232224]”.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors have no conflicting financial or other interest.

References

  1. Ellis, J. The honey bee crisis. Outlooks Pest Manag. 2012, 23, 35–40. [Google Scholar] [CrossRef]
  2. Eliash, N.; Mikheyev, A. Varroa mite evolution: A neglected aspect of worldwide bee collapses? Curr. Opin. Insect Sci. 2020, 39, 21–26. [Google Scholar] [CrossRef] [PubMed]
  3. Boecking, O.; Genersch, E. Varroosis—The ongoing crisis in bee keeping. J. Verbr. Lebensmittelsicherh. 2008, 3, 221–228. [Google Scholar] [CrossRef]
  4. Sammataro, D.; Gerson, U.; Needham, G. PARASITIC MITES OF HONEY BEES: Life history, implications, and impact. Annu. Rev. Entomol. 2000, 45, 519–548. [Google Scholar] [CrossRef] [PubMed]
  5. Roth, M.A.; Wilson, J.M.; Tignor, K.R.; Gross, A.D. Biology and management of Varroa destructor (Mesostigmata: Varroidae) in Apis mellifera (Hymenoptera: Apidae) colonies. J. Integr. Pest Manag. 2020, 11, 1. [Google Scholar] [CrossRef]
  6. Jack, C.J.; Ellis, J.D. Integrated pest management control of Varroa destructor (Acari: Varroidae), the most damaging pest of (Apis mellifera L. (Hymenoptera: Apidae)) colonies. J. Insect Sci. 2021, 21, 6. [Google Scholar] [CrossRef]
  7. Salazar-Gomez, A.; Darbyshire, M.; Gao, J.; Sklar, E.I.; Parsons, S. Towards Practical Object Detection for Weed Spraying in Precision Agriculture. 2021. Available online: http://arxiv.org/abs/2109.11048 (accessed on 22 September 2021).
  8. Selvaraj, M.G.; Vergara, A.; Ruiz, H.; Safari, N.; Elayabalan, S.; Ocimati, W.; Blomme, G. AI-powered banana diseases and pest detection. Plant Methods 2019, 15, 1–11. [Google Scholar] [CrossRef] [Green Version]
  9. Ngo, T.N.; Wu, K.C.; Yang, E.C.; Lin, T.T. A real-time imaging system for multiple honey bee tracking and activity monitoring. Comput. Electron. Agric. 2019, 163, 104841. [Google Scholar] [CrossRef]
  10. Bjerge, K.; Frigaard, C.E.; Mikkelsen, P.H.; Nielsen, T.H.; Misbih, M.; Kryger, P. A computer vision system to monitor the infestation level of Varroa destructor in a honeybee colony. Comput. Electron. Agric. 2019, 164, 104898. [Google Scholar] [CrossRef]
  11. Liu, C.; Xu, J.; Wang, F. A review of keypoints’ detection and feature description in image registration. In Scientific Programming; Hindawi Publishing Limited: London, UK, 2021; Volume 2021, pp. 1–25. [Google Scholar] [CrossRef]
  12. Anderson, D.L.; Trueman, J.W.H. Varroa jacobsoni (Acari: Varroidae) is more than one species. Exp. Appl. Acarol. 2000, 24, 165–189. [Google Scholar] [CrossRef] [PubMed]
  13. Dembski, J.; Szymański, J. Bees detection on images: Study of different color models for neural networks. In Lecture Notes in Computer Science; Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics; Speinger: Berlin, Germany, 2019; Volume 11319, pp. 295–308. [Google Scholar] [CrossRef]
  14. Abdullah-Al-Wadud, M.; Hasanul Kabir, M.; Ali Akber Dewan, M.; Chae, O. A Dynamic Histogram Equalization for Image Contrast Enhancement. IEEE Trans. Consum. Electron. 2007, 53, 593–600. [Google Scholar] [CrossRef]
  15. Stojnić, V.; Risojević, V.; Pilipović, R. Detection of pollen bearing honey bees in hive entrance images. In Proceedings of the 17th International Symposium on INFOTEH-JAHORINA, INFOTEH 2018, Jahorina, Bosnia and Herzegovina, 21–23 March 2018; Volume 2018, pp. 1–4. [Google Scholar] [CrossRef]
  16. Rublee, E.; Rabaud, V.; Konolige, K.; Bradski, G. ORB: An efficient alternative to SIFT or SURF. In Proceedings of the IEEE International Conference on Computer Vision, Barcelona, Spain, 6–13 November 2011; pp. 2564–2571. [Google Scholar] [CrossRef]
  17. Tareen, S.A.K.; Saleem, Z. A Comparative Analysis of SIFT, SURF, KAZE, AKAZE, ORB, and BRISK; IEEE Publications: Sukkur, Pakistan, 2018. [Google Scholar] [CrossRef]
  18. Karami, E.; Prasad, S.; Shehata, M. Image Matching Using SIFT, SURF, BRIEF and ORB: Performance Comparison for Distorted Images. arXiv 2017, arXiv:1710.02726. [Google Scholar]
Figure 1. (a): image acquisition systems and (b): image of bees and bee mites (red circles: bee mites).
Figure 1. (a): image acquisition systems and (b): image of bees and bee mites (red circles: bee mites).
Agriculture 13 01511 g001
Figure 2. Region of Interest (RoI) cropping: (a) original image with the green box meant the extracted area and (b) cropped image after the extraction.
Figure 2. Region of Interest (RoI) cropping: (a) original image with the green box meant the extracted area and (b) cropped image after the extraction.
Agriculture 13 01511 g002
Figure 3. Infested bee with parasitic mite according to measurement distance.
Figure 3. Infested bee with parasitic mite according to measurement distance.
Agriculture 13 01511 g003
Figure 4. Flowchart of optimal image processing for feature point detection according to color and image correction.
Figure 4. Flowchart of optimal image processing for feature point detection according to color and image correction.
Agriculture 13 01511 g004
Figure 5. Average histogram of each channel by color models: (a) RGB, (b) HSV, (c) Lab, (d) YCrCb, and (e) Gray.
Figure 5. Average histogram of each channel by color models: (a) RGB, (b) HSV, (c) Lab, (d) YCrCb, and (e) Gray.
Agriculture 13 01511 g005
Figure 6. Histogram equalization processing image: (a) original, (b) GHE, (c) CLAHE, (d) normalized image, (e) normalized GHE, and (f) normalized CLAHE.
Figure 6. Histogram equalization processing image: (a) original, (b) GHE, (c) CLAHE, (d) normalized image, (e) normalized GHE, and (f) normalized CLAHE.
Agriculture 13 01511 g006
Figure 7. (a) Normal operation and (b) abnormal operation of histogram normalization processing. When values existed between 0 and 255, as in (b), histogram normalization did not work properly.
Figure 7. (a) Normal operation and (b) abnormal operation of histogram normalization processing. When values existed between 0 and 255, as in (b), histogram normalization did not work properly.
Agriculture 13 01511 g007
Figure 8. Histogram equalization processing image: (a) original, (b) CLAHE, and (c) GHE. The red boxes in each image represented bees infected with bee mite.
Figure 8. Histogram equalization processing image: (a) original, (b) CLAHE, and (c) GHE. The red boxes in each image represented bees infected with bee mite.
Agriculture 13 01511 g008
Figure 9. Number of average keypoints of bees for each imaging distance in original and each color model image.
Figure 9. Number of average keypoints of bees for each imaging distance in original and each color model image.
Agriculture 13 01511 g009
Figure 10. Keypoint detection images for original images of four color models (RGB (a1), HSV (b1), Lab (c1), Gray (d1)), GHE processed image (a2d2), and CLAHE processed image (a3d3).
Figure 10. Keypoint detection images for original images of four color models (RGB (a1), HSV (b1), Lab (c1), Gray (d1)), GHE processed image (a2d2), and CLAHE processed image (a3d3).
Agriculture 13 01511 g010
Figure 11. Histogram distribution change original image (a), after histogram normalization (b), and normalization with CLAHE (c). Red, green, and blue lines represent the red, green, and blue conponent of the RGB channels, respectively.
Figure 11. Histogram distribution change original image (a), after histogram normalization (b), and normalization with CLAHE (c). Red, green, and blue lines represent the red, green, and blue conponent of the RGB channels, respectively.
Agriculture 13 01511 g011
Figure 12. Number of average keypoints of bee mites for each image measurement distance in original image and image processed with histogram normalization and histogram equalization (CLAHE). The numbers above the bars were the average of keypoints.
Figure 12. Number of average keypoints of bee mites for each image measurement distance in original image and image processed with histogram normalization and histogram equalization (CLAHE). The numbers above the bars were the average of keypoints.
Agriculture 13 01511 g012
Figure 13. Normal and abnormal matching points in image matching between bee (left) and bee mite (right): (a) source image and (b) matching result. Each line connects the matched keypoints. The lines provide a visual representation of the normal and abnormal match between the bee mite on the bee and the bee mite image.
Figure 13. Normal and abnormal matching points in image matching between bee (left) and bee mite (right): (a) source image and (b) matching result. Each line connects the matched keypoints. The lines provide a visual representation of the normal and abnormal match between the bee mite on the bee and the bee mite image.
Agriculture 13 01511 g013
Figure 14. Normal matching result of bee mite of image data measured at distances of 300 and 500 mm.
Figure 14. Normal matching result of bee mite of image data measured at distances of 300 and 500 mm.
Agriculture 13 01511 g014
Table 1. Number of image measurements according to imaging distance for measuring the entire beecomb area.
Table 1. Number of image measurements according to imaging distance for measuring the entire beecomb area.
Imaging Distance300 mm500 mm700 mm900 mm1100 mm
Number of image measurement (ea)96421
Table 2. Average number of keypoints of the RGB image according to imaging distance and image processing.
Table 2. Average number of keypoints of the RGB image according to imaging distance and image processing.
300 mm500 mm700 mm900 mm1100 mm
Original6721210
Normalization2765030121
GHE4569350311
CLAHE50210534241
Normalization and GHE4569350311
Normalization and CLAHE75812966372
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Share and Cite

MDPI and ACS Style

Lee, H.G.; Kim, M.-J.; Kim, S.-b.; Lee, S.; Lee, H.; Sin, J.Y.; Mo, C. Identifying an Image-Processing Method for Detection of Bee Mite in Honey Bee Based on Keypoint Analysis. Agriculture 2023, 13, 1511. https://doi.org/10.3390/agriculture13081511

AMA Style

Lee HG, Kim M-J, Kim S-b, Lee S, Lee H, Sin JY, Mo C. Identifying an Image-Processing Method for Detection of Bee Mite in Honey Bee Based on Keypoint Analysis. Agriculture. 2023; 13(8):1511. https://doi.org/10.3390/agriculture13081511

Chicago/Turabian Style

Lee, Hong Gu, Min-Jee Kim, Su-bae Kim, Sujin Lee, Hoyoung Lee, Jeong Yong Sin, and Changyeun Mo. 2023. "Identifying an Image-Processing Method for Detection of Bee Mite in Honey Bee Based on Keypoint Analysis" Agriculture 13, no. 8: 1511. https://doi.org/10.3390/agriculture13081511

APA Style

Lee, H. G., Kim, M. -J., Kim, S. -b., Lee, S., Lee, H., Sin, J. Y., & Mo, C. (2023). Identifying an Image-Processing Method for Detection of Bee Mite in Honey Bee Based on Keypoint Analysis. Agriculture, 13(8), 1511. https://doi.org/10.3390/agriculture13081511

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop