Towards a Taxonomy Machine: A Training Set of 5.6 Million Arthropod Images

Abstract

The taxonomic identification of organisms from images is an active research area within the machine learning community. Current algorithms are very effective for object recognition and discrimination, but they require extensive training datasets to generate reliable assignments. This study releases 5.6 million images with representatives from 10 arthropod classes and 26 insect orders. All images were taken using a Keyence VHX-7000 Digital Microscope system with an automatic stage to permit high-resolution (4K) microphotography. Providing phenotypic data for 324,000 species derived from 48 countries, this release represents, by far, the largest dataset of standardized arthropod images. As such, this dataset is well suited for testing the efficacy of machine learning algorithms for identifying specimens into higher taxonomic categories.

1. Summary

The identification of organisms is a fundamental part of recognizing and describing biodiversity. The development of automated methods, which can identify specimens without involving taxonomists, is critical given the taxonomic impediment [1,2]. One effective solution involves the adoption of identification systems based on the analysis of sequence variation in short, standardized DNA regions [3]. In addition, digitization initiatives at major natural history collections [4] and imaging linked to large DNA barcoding projects [5,6] are providing the basis for image-based identification systems driven by machine learning algorithms.

Images can be used to build classification systems capable of identifying species [7,8,9]. Various methods and datasets have been proposed to advance image-based identifications for arthropods [10,11]. Most past work has considered arthropod classification from (1) the context of integrated pest management [12,13,14], (2) as part of crowd-sourced citizen science efforts such as iNaturalist [15,16], or (3) interpreting data acquired by camera traps [17,18].

Machine learning algorithms, especially convolutional artificial neural networks and their variants, have emerged as the most effective method for object recognition and detection [10,19,20]. However, datasets of sufficient size for properly training these models are scarce [20,21,22,23], as creating them is labour-intensive and expensive. On the other hand, they are urgently needed to avoid performance issues and limitations for vision models resulting from insufficient training [24]. For instance, most available biodiversity image datasets have poor representation of arthropod species even though these represent a large fraction of eukaryotic life. In recent years, substantial efforts have been invested into the digitization of arthropod natural history collections, which have imaged both individual specimens [25,26,27] and entire drawers of specimens [6,28,29]. This work provides training material for classification algorithms, but the images are structurally uniform. Specimens are usually mounted and therefore captured in a single standard orientation (mostly dorsal view), limiting their application in less regimented contexts [24]. In addition, there are biases in selecting insects, which translate into the availability of specimens for digitization or images gathered by community science [30,31].

In this study, we present a dataset of 5.6 million arthropod images gathered during a large-scale DNA barcoding project. As specimens were not placed in a standard orientation before photography, any machine learning-based object detection must interpret specimens in varying orientations to classify them taxonomically, an approach that requires massive training sets.

2. Data Generation

The present dataset was generated as part of an ongoing effort to build a global DNA barcode reference library for terrestrial arthropods [32]. The workflow involves placing each small (<5 mm) specimen into a well of a round-bottom 96-well microplate before DNA extraction. The specimens are individually photographed at 4K resolution in plate format using a Keyence VHX-7000 Digital Microscope system (Keyence, Osaka, Japan) with a fully integrated head and an automatic stage (Figure 1). The setup uses an adjustable Keyence illumination adapter to ensure uniform light conditions and a scanning stage equipped with a custom-engineered mount that holds each plate (Figure 1, Supplementary File S1). This system can capture 95 images within 15 min by controlling stage movements in X-Y coordinates. It also has the capability to automatically control the height of the stage with a precision of 0.1 μm. By moving the lens throughout the different focal planes of each specimen, the VHX system captures every pixel that comes into focus at each level and combines them into a single, fully focused image. For all the images, the system was set to a brightness of 27.8 ms, and colours were set to R1.7 G1.0 B2.19.

The resulting images are packaged with a Python script (Supplementary File S2) and uploaded onto the Barcode of Life Data Systems V4 (BOLD) [33] where they are automatically associated with individual specimen records. BOLD is a cloud-based data storage and analysis platform that supports the assembly and use of DNA barcode data. Users can generate and populate the specimen records. BOLD enables its users to upload and store many collaterals such as specimen-associated taxonomic labels, images, and DNA sequences [33]. A backup copy of each image is subsequently transferred to a third-party cloud service using an automated script (Supplementary File S3).

3. Data Description

The present dataset includes 5,675,731 images, mostly of terrestrial arthropods. Associated metadata include information on provenance and taxonomy (Supplementary File S4). The dataset includes 1.13 million images from a previous release [20] together with 4.54M images generated from 2019 to 2022. The size of each image is 2880 × 2160 pixels, creating an average file size of 17.9 MB for a TIFF file and 1.88 MB for a JPEG file. Figure 2 shows an array of 95 images taken with the Keyence system (the empty space at the lower right corresponds to an empty control well in the microplate).

3.1. Geographic Coverage

The dataset contains images for specimens from 1698 sites in 48 countries (Figure 3A). Most specimens were derived from Costa Rica (62%), followed by South Africa (6%), United States (5%), and Thailand (3%) (Figure 3B, Supplementary File S4). Most specimens were collected using Malaise traps, but about 1500 were captured using plankton and dip nets [34].

3.2. Taxonomic Coverage

Most of the images show individuals from the class Insecta (98%) (Figure 4A). Figure 4B shows the distribution of insect orders within the dataset. A very high proportion of the complete set of specimens has taxonomic assignments at the ordinal (99.4%) and family (85.9%) levels, but only 20.3% possess a generic assignment. Just 7.6% of the specimens (430,036) have a Linnean species designation, but 90.6% (N = 5,143,970) possess a Barcode Index Number (BIN) assignment. The BIN system [35] is a key feature of BOLD. It employs an algorithm that combines single linkage and Markov clustering to group DNA barcode sequences into Operational Taxonomic Units that represent good species proxies [35]. A total of 324,427 such BINs are represented in this dataset.

4. Conclusions

This dataset of 5.6 million images representing over 320,000 species of arthropods, mostly insects, is the largest of its kind. It is also unique in the geographic distribution of the photographed samples. As such, it represents a dataset that should aid the development of machine learning algorithms for object recognition and classification despite a considerable class imbalance (69% Diptera). Data from a previous release were successfully used for several classification tasks across arthropod orders [20]. In addition, a larger subset of this dataset has been combined with DNA sequence information to develop and benchmark multimodal classification experiments [36]. These images can also be used to estimate biomass [37,38] or even abundance when entire communities are digitized [39,40]. Each image in the dataset is also available as an element of the collateral data for a barcode record on BOLD [33] providing support for taxonomic assignment and to enable direct visual comparisons between individuals.

Generating high quality images at the described rate can be difficult with any automated system given the manifold differences in shape and size of specimens [6]. Some large and some very small individuals might be outside the standard stacking depth, which can result in an out-of-focus image. Furthermore, mistakes during the placement of individuals onto the microplates can lead to accidental imaging of an empty well or, rarely, the placement of more than one individual into one well.

By using three Keyence VHX-7000 systems for 50 h per week, the Centre for Biodiversity Genomics generates three million images per year. The deployment of Keyence systems at a few core facilities could readily generate ten million images per year, allowing rapid growth in the training sets required to hone AI-enabled identification systems, including routines that automatically flag instances of lower quality as described above.