Machine Learning-Based Species Classification Methods Using DART-TOF-MS Data for Five Coniferous Wood Species

Abstract

Various problems worldwide are caused by illegal production and distribution of timber, such as deception about timber species and origin and illegal logging. Numerous studies on wood tracking are being conducted around the world to demonstrate the legitimacy of timber. Tree species identification is the most basic element of wood tracking research because the quality of wood varies greatly from species to species and is consistent with the botanical origin of commercially distributed wood. Although many recent studies have combined machine learning-based classification methods with various analytical methods to identify tree species, it is unclear which classification model is most effective. The purpose of this work is to examine and compare the performance of three supervised machine learning classification models, support vector machine (SVM), random forest (RF), and artificial neural network (ANN), in identifying five conifer species and propose an optimal model. Using direct analysis in real-time ionization combined with time-of-flight mass spectrometry (DART-TOF-MS), metabolic fingerprints of 250 individual specimens representing five species were collected three times. When the machine learning models were applied to classify the wood species, ANN outperformed SVM and RF. All three models showed 100% prediction accuracy for genus classification. For species classification, the ANN model had the highest prediction accuracy of 98.22%. The RF model had an accuracy of 94.22%, and the SVM had the lowest accuracy of 92.89%. These findings demonstrate the practicality of authenticating wood species by combining DART-TOF-MS with machine learning, and they indicate that ANN is the best model for wood species identification.

1. Introduction

Various problems around the world are caused by illegally produced and distributed timber, such as deception about the species and origin of timber and illegal logging. Illegal manufacture of timber is a major cause of climate change and poverty by means of deforestation, as well as economic damage [1]. As logwood goes through several stages of processing after logging and is distributed as a variety of wood products, tracking the legality of raw wood materials is difficult. Nonetheless, studies on wood tracking are being carried out around the world to prove the legitimacy of timber for sale [2,3,4]. Timber tracking research uses a hierarchical approach appropriate to the production and distribution of timber, such as identifying the species and determining its origin and harvesting site. Tree species identification is the most basic element of timber tracking because the quality of wood varies dramatically depending on the species and corresponds to botanical identification information about commercially distributed wood [5].

The following methods are generally used for species identification: (1) morphological methods, such as anatomy and dendrochronology [6]; (2) genetic methods such as DNA barcoding, DNA fingerprinting, and phylogeography [7]; (3) chemical methods such as stable isotopes, near-infrared (NIR) spectroscopy, and direct analysis in real time–mass spectrometry (DART-MS) [4]. Morphological studies (analyses of macroscopic and microscopic anatomical features) are traditionally and most frequently used for identification; however, only trained experts can perform such analyses, and even they can struggle to identify wood at the species level. Therefore, extensive and automated image analysis methods using machine learning and neural computation are being studied [8]. DNA analysis has been successful in species identification [9], but it is time-consuming and expensive. Furthermore, analyzing DNA sequences from wood samples for genetic differentiation is challenging because isolating DNA from dried and processed wood is difficult [7]. The use of stable isotopes has proved successful for species and regions [2]; however, that technique also has challenges [4]. NIR spectroscopy has proved successful in distinguishing between species and in determining the origin of a few species [10,11,12].

Studies using DART-MS have been conducted to classify white and red oak [13], to distinguish between eucalyptus and Convention on International Trade in Endangered Species of Wild Flora and Fauna (CITES)-listed Aruacariacea [14], and to distinguish between wild and cultivated agarwood [15]. Zhang et al. [16,17] discriminated between Pterocarpus tinctorius Welw. and endangered Pterocarpus santalinus L.f. by combining DART with Fourier transform ion cyclotron resonance mass spectrometry and using a multivariate analysis in Soft Independent Modeling by Class Analogy (SIMCA) statistical software. DART-MS can ionize molecules in ambient conditions with minimal sample preparation, offering the advantages of high throughput, fast analysis speed, and non-destructive testing [13,18,19]. Those advantages are critical to time-constrained enforcement policies, so DART-MS is becoming increasingly popular [20]. Moreover, it is less expensive than genetic and stable isotope analyses. Recent studies have combined DART-MS analyses with machine learning [14,16,17,21,22,23,24,25].

Machine learning is a computer algorithm for development of artificial intelligence that can automatically recognize and learn relevant information from input data to find relationships and patterns in the data and solve problems [26,27,28]. Machine learning methods are generally characterized into two major groups based on type of learning: supervised and unsupervised. The supervised learning method uses labeled data to train an algorithm to compute output variables [29]. Unsupervised learning methods, on the other hand, use unlabeled data in training an algorithm to identify hidden patterns [30]. The strength of applying machine learning to wood science and engineering is its ability to analyze any type of data, such as images [31], anatomy data [32], infrared spectra [12], MS spectra [33], and U(H)PLC chromatograms [34]. Furthermore, as more data accumulate and can be used to train an algorithm, its accuracy and predictive power improve. The big data-based machine learning industry is growing and finding applications in various parts of the wood industry for genotype discrimination [35,36], species identification [8,12,23,37], and wood moisture content prediction [38,39].

Multivariate analysis and machine learning methods are mainly used to classify species using chemical fingerprint data [40], but there has been no research to assess which method is best for determining the biological origin of wood through DART-MS analysis for wood classification. In this paper, in order to reveal which classification method is best for DART-MS data, origin determination was tested using principal component analysis (PCA), partial least squared discriminant analysis (PLS-DA), orthogonal partial least squared discriminant analysis (OPLS-DA), support vector machine (SVM), random forest (RF), and artificial neural network (ANN) methods. This study aimed to prevent or solve problems in the wood distribution market caused by the use of wood of the wrong origin by presenting an effective identification method. In addition, the practicality of DART-TOF-MS combined with machine learning is demonstrated to authenticate wood species and indicate which model is the best for identification. These results may be proposed for future work to enable other researchers to complete research in this field.

2. Materials and Methods

2.1. Wood Materials

Wood samples of five coniferous species, Chamaecyparis obtusa (Co) Siebold & Zucc. Endl., Larix kaempferi (Lk) (Lamb.) Carr., Pinus densiflora (Pd) Sieb. et Zucc., Pinus koraiensis (Pk) Sieb. et Zucc., and Pinus thunbergii (Pt) Parl. were collected from different areas of South Korea by the National Institute of Forest Science in the Korean Forest Service. Fifty samples of each species were tested (detailed information is contained in Supplementary Materials).

All samples were collected in 2021, and they were either increment cores with a diameter of 5 mm and a maximum length of 15 cm or water-depth direction cores with a diameter of 5 mm and an average length of 7.5 cm. The wood samples were stored at room temperature with silica gel until analysis.

2.2. DART-TOF-MS Conditions

With no further sample preparation, all wood samples were analyzed three times using a DART simplified voltage and pressure ion source (IonSense, Saugus, MA, USA) equipped with a JEOL JMS-T100TD (AccuTOF-TLC) mass spectrometer (JEOL Ltd., Tokyo, Japan) in positive ion mode. Each sample was placed on a lab-made wood sample tray module (Figure 1) in a stream of heated helium gas produced by the DART ion source for four seconds. The mass spectra of the heartwood were acquired using the DART ion source parameters and mass spectrometer with settings of helium gas flow rate, 3 L/min; gas heater temperature, 350 °C; desolvating temperature, 250 °C; orifice 1 temperature, 80 °C; orifice 1 voltage, 10 V; orifice 2 voltage, 5 V; ion guide peak voltage, 600 V; ion guide bias voltage, 25 V; focus voltage, −145 V; condenser lens voltage, 6.0 V; quadrupole lens voltage, 15.0 V; right/left lens voltage, −2.0 V; top/bottom lens voltage, 0.1 V; pusher bias voltage, −0.55 V; reflectron voltage, 950.0 V; ring lens voltage, 5 V; and detector voltage 2000 V. Spectra were obtained over the mass range of m/z 100 to 500 with a sampling interval of 0.5 ns and recording interval of 0.5 s. A polyethylene glycol 600 (PEG 600) standard was measured before and after each sample analysis for mass calibration.

2.3. Data Preprocessing

The data preprocessing flow chart is provided in Figure 1. The raw data were calibrated using the PEG 600 standard and exported as centroid mass spectra by Mass Center Main software (JEOL), which has 41,938 m/z features for each sample.

In most instances, DART-TOF-MS data are applied to algorithms in the form of a matrix or frame, with columns and rows corresponding to variables and samples, respectively. Furthermore, a process called binning is required to segment variables with the same specific m/z value based on mass tolerance. In this study, because the total ion chromatogram peaks had positively skewed distributions, the m/z value expressed to five decimal places was rounded down to one decimal place, and all peak intensities with the same integer value of m/z were added to produce 4000 variants per sample.

Data processing and subsequent classification were performed using Python in the PyCharm integrated development environment. Target data were processed through one-hot encoding, as shown in Figure 1, and seventy percent of the obtained data (n = 525) were randomly set as the training data, with the rest (n = 225) used as test data, to analyze the accuracy of the developed models.

Before machine learning classification, the data were preprocessed as appropriate for each model. The input data for SVM required a Z-score normalization process [41] because SVM is sensitive to features with a large scale and has difficulty with accurate predictions with a large number of features in the dataset [42]. As RF is a tree-based model [43] and ANN was developed to perform batch normalization after each hidden layer, the input data for RF and ANN were used without normalization.

2.4. Modeling for Classification

In this study, SVM, RF, and ANN were used to classify samples of five coniferous woods at the species level (Figure 2). The SVM and RF models were developed using the Scikit-learn package, and the ANN model was developed using the Keras library.

Cortes and Vapnik [44] introduced SVM to simultaneously minimize classification error and maximize the margin between two classes. A multiclass SVM model uses a Gaussian-based radial basis function as the kernel function to map the given data to a high-dimensional feature space according to a one-to-one scheme and thereby refine the decision boundary [44,45]. Two parameters, cost (C) and gamma, have to be set by the user. C determines how many data samples are allowed to be classified into faulty categories, and gamma determines the curvature of the decision boundary, i.e., the distance over which the data samples exert influence [46]. If the values of C and gamma are set too low, the possibility of outliers is increased and the influence distance of each datapoint becomes broader, producing a high probability of underfitting. On the other hand, if the values of C and gamma are set too high, the probability of outliers is low, and the influence distance of each datapoint narrows, resulting in a high probability of overfitting. We set the C value as 1.0 and gamma as auto (1/n_features), which means the gamma value of each sample decreases as the number of samples increases.

RF is a combination of tree predictors in which each tree depends on the value of an independently sampled random vector that has the same distribution for all trees in the forest [47]. This algorithm forms multiple decision trees, passes the data through each tree at the same time, votes on the classification results for each tree, and selects the result with the most votes as the final classification result. Although overfitting can result from RF, generating a large number of trees prevents such overfitting from having a significant effect on the overall prediction [47]. The hyper-parameters for RF are as follows:

• “n_estimators” means the number of trees in the forest;
• “max_depth” means the maximum depth of the trees;
• “min_samples_split” means the minimum number of samples required to split an internal node;
• “min_samples_leaf” means the minimum number of samples required to be at a leaf node;
• “max_features” means the number of features to consider when looking for the best split;
• “max_leaf_nodes” means the number of groups to be classified;
• “max_samples” means the number of samples to draw from the total data to train each base estimator.
• We optimized n_estimators as 1000 and max_depth as 10 and allowed the rest of the parameters to be set to default values by Scikit-learn (min_samples_split, 2; min_samples_leaf, 1; max_features, auto; max_leaf_nodes, none; max_samples, none).

In an ANN, artificial neurons form a network through synaptic combinations and solve problems by learning to change the strength of their synaptic bindings [48]. In this study, we developed a multilayer ANN model. Instead of directly using the given data to form a classifier, we added a structure called a hidden layer while training the model with transformation operations optimized for the specified data [49]. The update rules for the parameters of the neuron are derived by calculating weights and biases that minimize the cost function defined by the labels contained in the training dataset. However, using hidden layers creates a problem in that the cost function cannot be defined because there are no labels. To solve that problem, a backpropagation algorithm was developed to train a multilayer ANN [50]. Therefore, we developed a multilayer ANN model with three hidden layers, considered as a deep neural network [51,52,53], using the Leaky ReLU activation function and Adam optimization method with a sparse categorical cross entropy loss function with a learning rate of 0.0001.

3. Results

3.1. Figures, Tables, and Schemes

Representative mass spectra for the heartwood of each coniferous species are shown in Figure 3. Each spectrum was obtained by overlapping 150 spectra per species with equal weighting. Among the major peaks, m/z 205.19 was observed in Co, Lk, and Pd; m/z 207.25 and 303.23 were observed in all species except Co; and m/z 338.34 and 355.37 were detected in all species except Pd. It was not possible to distinguish the species based on their major peaks.

3.2. Multivariate Analysis

PCA is an unsupervised analytical method used to find differences between groups of samples, understand group associations, and validate the relative roles of compounds in group separation [54,55]. As a supervised method, PLS-DA reduces dimensions by considering the variance of Y [56,57]. Therefore, PLS-DA requires extraction of a new variable that maximizes the covariance between the linear combination of the independent variable X and the dependent variable Y. This variable is not reflected in the PCA, and has the advantage of building an efficient model with a small number of extracted variables [56,57]. OPLS-DA can maximize discrimination between groups by addition of orthogonal components from PLS using class information [58]. We performed PCA, PLS-DA, and OPLS-DA using SIMCA-P software ver. 14.1 (Umetrics, Umea, Sweden). In the PCA and PLS-DA results, no significant distinctions were found between species in the PCA scatter plot and only Co was clustered to distinguish it from other species (Figure S1). The OPLS-DA score plot (Figure 4) shows that C. obtusa was distinguished from the other species, and L. kaempferi and P. koraiensis were adequately sorted in the OPLS-DA score 3D plot. However, P. densiflora and P. thunbergii were not clearly distinguished in either the OPLS-DA score plot or the OPLS-DA 3D score plot. The OPLS-DA analysis provided the following statistical parameters: R²X = 0.797, R²Y = 0.841, and Q² = 0.813. To test the validity of the OPLS-DA model and prevent overfitting, permutation tests were performed using 100 random permutations and the same number of components. Those results show that the model is valid because the extrapolated intercept value (Q2) of −0.157 indicates that the model has no overfitting [59].

3.3. Classification Model Performance

When classifications were performed with the same random seed, the accuracy increased in the order of SVM, RF, and ANN. In all models, the genus-level classification accuracy was 100% in our sample group, meaning the accuracy of each model in the genus Pinus depends on species-level classification. The SVM model produced identification errors of about 7% in the test set, including classifying P. densiflora as P. thunbergii, P. koraiensis as P. densiflora, and P. thunbergii as L. kaempferi or P. densiflora, with a total prediction accuracy rate of 92.89%. The RF model’s prediction accuracy rate of 94.22% was slightly higher than that of the SVM, and its errors involved predicting P. densiflora as P. thunbergii and P. thunbergii as P. densiflora. The ANN model produced the same errors as the RF, but the frequency was significantly lower, with an overall prediction accuracy rate of 98.22%. To validate the results, we added a new test set with the same size as that drawn from samples that were not used for developing the classification models. The accuracy results for the training (70%), test (30%), and new test sets for each model are provided in

Table 1. Classification accuracies for training, test, and new test set data for each machine learning model.

Model	Train Set (70%)	Test Set (30%)						New Test Set
		Total	Co	Lk	Pd	Pk	Pt
Support Vector Machine (SVM)	98.48%	92.89%	52/52	39/39	33/36	46/49	39/49	90.22%
			100%	100%	91.67%	93.88%	79.59%
Random Forest (RF)	99.81%	94.22%	52/52	39/39	32/36	49/49	40/49	93.33%
			100%	100%	88.89%	100%	81.63%
Artificial Neural Network (ANN)	100%	98.22%	52/52	39/39	33/36	49/49	48/49	98.67%
			100%	100%	91.67%	100%	97.96%

. A confusion matrix for the three models is presented in Figure 5.

3.4. Evaluation of Model Performance

The purpose of model evaluation is to prevent overfitting and identify the optimal model. Accuracy or error rate is one of the most commonly used metrics in practice, and many researchers use it to evaluate the generalization ability of a classifier [60]. Accuracy is a measure of a trained classifier in terms of overall correctness, as the total number of instances in which the trained classifier presented a correct prediction [61]. The benefits of accuracy as a study metric are its ease of computation with low complexity; suitability for multiclass and multilabel problems; user friendly scoring; and ease of understanding. However, the simplicity of the accuracy metric could lead to suboptimal solutions, especially when dealing with imbalanced class distribution [60,61,62].

Therefore, to evaluate the performance of the classification models, in addition to accuracy, the following metrics were estimated: precision, recall, F1-score, receiver operating characteristic (ROC) curve, and area under the ROC curve (AUC) from the confusion matrix (

Table 2. Evaluation metrics of the average values of five species for each machine learning model.

Model	Accuracy	Precision	Recall	F1-Score	AUC
Support Vector Machine (SVM)	0.929	0.927	0.930	0.929	0.904
Random Forest (RF)	0.942	0.938	0.941	0.939	0.964
Artificial Neural Network (ANN)	0.982	0.982	0.979	0.981	0.984

) [61,63,64]. The ROC curves for the models are presented in Figure 6. All metrics have a value between 0 and 1, and the closer is the result to 1, the higher the accuracy of classification [61].

4. Discussion

Identification of tree species is important when the wood is distributed and used as a raw material because wood quality and price depend on species [65]. Therefore, various studies have been conducted to distinguish the exact origins of similar woods [6,7,8,9,10,11,12,13,14,15,16,17,37].

For the inter-genus classification, all three machine learning models showed 100% prediction accuracy in the sample group collected. In the inter-species classification, the ANN model had the highest classification prediction accuracy, 98.22%. The RF model had 94.22% accuracy, and the SVM had the lowest accuracy, 92.89%. Classification at the species level is difficult because the samples are genetically similar and have similar metabolomic properties. In particular, for our samples, Pd and Pt are genetically closer to each other than to Pk in the phylogenetic tree [66].

In a study in South Africa, Omer et al. [67] tested SVM and ANN models for pixel-based classification of six tree species from mixed native coastal forests. The methods produced similar classification results; the overall accuracy of SVM and ANN was 77% and 75%, respectively, using a randomly selected 70% (583) of the dataset for training and the remaining 30% (244) of the dataset for evaluation. In the same study area, Cho et al. [68] used an SVM algorithm to compare pixel-based and object-based approaches for classifying three major three species. The pixel-based and object-based methods achieved overall accuracy scores of 85% and 89%, respectively. In the above papers, it was confirmed that SVM is more suitable for pixel-based classification, indicating the need to evaluate a suitable machine learning algorithm according to data type. Karlson et al. [69] utilized an RF model for WorldView-2 imagery object-based classification of five major tree species in Burkina Faso, West Africa, and achieved the most accurate results of the overall accuracy of 83.4% based on the multiseasonal dataset which is followed by 78.4% based on the dry season dataset and 68.1% based on the wet season dataset.

The advantage of RF is that it incorporates a feature importance property into the model itself, so variable importance can be determined without any special process. Deklerck et al. [22] improved overfitting and bias caused by random discriminant ion selection by performing kernel discriminant analysis on data obtained from RF and using the result to classify afromosia and provide the minimum number of ions needed to achieve the highest accuracy. In our study, the most important variables were m/z values of 271.1, 270.1, and 241.1, which were not confirmed as the major m/z values in the spectra (Figure 3).

In this study, we scaled down the m/z value to one decimal place for the convenience of data handling for the machine learning, which is similar to that of low-resolution MS data, and it means that low-resolution mass data can be used with machine learning classification methods. We selected an ANN with three hidden layers as a deep learning method [51,52,53] because it is suitable enough for the application of MS spectrum data in a vector form. In the case of supervised learning as in this study, multilayer perceptron has been effectively utilized as a feedforward ANN that involves some hidden layers separating the input and output layers that correspond to the network computational engine [52]. Furthermore, ANN has the potential to be developed as a database tool that can store spectra of different wood species and classify unknown samples accordingly. As related research continues and more information about various wood species accumulates, more information will be available to teach the ANN, which will improve its accuracy in wood species prediction. Since DART-MS is one of the ambient mass spectrometer methods and the measurement is performed effectively in high throughput [19], the metabolomics data of DART-MS spectra can be effectively applied to machine learning using a method such as SVM, RF, and ANN.

5. Conclusions

Identification of marker compounds after multivariate analysis is normally used for metabolite classification analyses [70]. Complex metabolites, such as those in plants, however, often cannot be clearly classified by multivariate analysis. In the field of plant metabolite research, there has been a trend in recent decades to study the overall aspects of metabolites using various data analysis techniques rather than the results of specific marker compounds [71,72,73].

In this study, we examined the performance of SVM, RF, and ANN algorithms and the potential of DART-TOF-MS data to classify wood species. Five conifers representing three genera and five species were analyzed by DART-TOF-MS and classified using three machine learning models. We demonstrated that, when sufficient data are available, SVM, RF, and ANN all show excellent classification results with DART-TOF-MS data that contain many variables. Evaluation of the performance of each model showed a value of 0.9 or higher in all metrics, indicating excellent performance. Among them, ANN showed the highest accuracy. These findings substantiate the practicality of authenticating wood species using DART-TOF-MS coupled with machine learning and indicate that ANN is the best model type for wood species identification. These results provide a foundation for additional research in tree species identification.