Aggregation of Multimodal ICE-MS Data into Joint Classifier Increases Quality of Brain Cancer Tissue Classification

Abstract

Mass spectrometry fingerprinting combined with multidimensional data analysis has been proposed in surgery to determine if a biopsy sample is a tumor. In the specific case of brain tumors, it is complicated to obtain control samples, leading to model overfitting due to unbalanced sample cohorts. Usually, classifiers are trained using a single measurement regime, most notably single ion polarity, but mass range and spectral resolution could also be varied. It is known that lipid groups differ significantly in their ability to produce positive or negative ions; hence, using only one polarity significantly restricts the chemical space available for sample discrimination purposes. In this work, we have developed an approach employing mass spectrometry data obtained by eight different regimes of measurement simultaneously. Regime-specific classifiers are trained, then a mixture of experts techniques based on voting or mean probability is used to aggregate predictions of all trained classifiers and assign a class to the whole sample. The aggregated classifiers have shown a much better performance than any of the single-regime classifiers and help significantly reduce the effect of an unbalanced dataset without any augmentation.

1. Introduction

Mass spectrometry is widely used to analyze biological samples, especially since the introduction of soft ionization methods: matrix-assisted laser desorption and ionization (MALDI) and electrospray ionization (ESI). Aside from highly sensitive targeted analysis of particular molecular species [1,2], mass spectrometry is capable of the rapid analysis of complex mixtures. This capability has a variety of applications, such as proteomics [3,4] and lipidomics [5,6]. Combined with advances in ambient ionization mass spectrometry [7,8,9,10,11,12], as well as methods of multidimensional data processing and machine learning [13,14,15,16,17], mass spectrometry enables “fingerprinting” approaches. A sample is analyzed under reproducible conditions, then the measured mass spectra are regarded as multidimensional data (intensity values corresponding to the m/z values) irrespective of the molecular species responsible for the signal. Then, depending on the aims of the research, a classifier or regression model is built using these spectra. Further samples can then be assigned a specific class (e.g., clinical diagnosis [11,12,18,19]) by a classification model or characteristic value (e.g., tumor cell percentage [20,21]) by a regression model using only their mass spectra, which can be obtained relatively quickly and do not require any subjective interpretation.

Most of the time, classifiers are built using mass spectrometric data from a single polarity, resolution, and m/z range. These specific parameters depend on the variety of molecules corresponding to the underlying biological concept. However, such a limitation is not necessary as multiple classes or subclasses of biomolecules could be affected by the investigated pathological condition. In the case of cancer lipidomics, it is crucial that lipid groups differ significantly in their ability to produce positive or negative ions, so using one polarity only significantly restricts the chemical space used for sample discrimination. The investigation of tumor tissue samples using inline cartridge extraction mass spectrometry (ICE-MS) demonstrates [22] that negative-ion mode spectra are more variable than their positive-ion mode counterparts. If the spectra are measured in the narrow m/z range (e.g., m/z 500–1000), the peaks can be attributed to lipids [23], while in the wider range (m/z 100–2000) other molecules contribute to the signal, such as a neural tissue marker NAA (m/z 174) [24]. The motivation for this work was to develop a classification approach that uses MS data from different regimes to provide a more robust and reliable classifier.

In the theory of machine learning, weak learners are expected to provide classifiers that are only slightly better than a random guess, in contrast to strong learners, which are expected to provide highly accurate predictions. Ensemble learning methods merge several weak learner algorithm implementations to create a strong learner [25]. This is the idea behind bootstrap aggregating, random forests, and boosting algorithms such as the XGboost [26] algorithm, which was used in this work to build individual (single regime) classifiers. These classifiers, trained to discriminate between mass spectra of glioblastoma and non-tumorous tissue samples, achieve more than 95% accuracy on the same data that was used to teach the classifier; however, when classifying an “unfamiliar” data set, the accuracy drops to 80%. This situation is called overfitting and is typical for highly multidimensional data such as mass spectra.

In this work, we apply an additional aggregation step in that we combine the results of ensemble learners built on the data for individual regimes and show that this approach is able to overcome overfitting in a way similar to how ensemble learning overcomes the low accuracy of weak classifiers. We use the XGboost algorithm to model the relationship between mass-spectrometry data and the diagnosis assigned to the spectra. We implement two ways to aggregate the XGBoost scan-level predictions into a sample-level prediction and to aggregate predictions made from different mass-spectrometry regimes into a final sample prediction.

2. Materials and Methods

The samples of glioblastoma and non-tumorous pathologies were obtained from the N.N. Burdenko National Scientific and Practical Center for Neurosurgery (NSPCN) and analyzed under an approved N.N. Burdenko NSPCN Institutional Review Board protocol in accordance with the Helsinki Declaration as revised in 2013. A signed informed consent explicitly noting that all removed tissues can be used for further research was obtained from all patients. Brain tumor tissues were resected during elective surgery and non-tumor pathological tissues were resected in the course of surgery for drug-resistant epilepsy. In total, 55 biopsy samples from 41 glioblastoma patients and 8 samples from 8 non-tumor patients were obtained for training and testing subsets. An additional 26 biopsy samples with known tumor cell percentage (ranging from 0% to 100%) from 11 glioblastoma patients were taken as a validation subset.

The molecular profiles of tissue sample fragments (up to 3 fragments per sample) are acquired using ICE-MS [12] with an LTQ XL Orbitrap mass spectrometer (Thermo Fisher Scientific, San Jose, CA, USA) in eight different regimes of measurement (all combinations of positive (+) or negative (−) ion mode, high (H) or low (L) resolution, m/z range 100–2000 (w) or 500–1000 (n)) to cover as many molecular features as possible [22]. Briefly, the sequence of regimes (30 s per regime) is Lw−, Hw−, Hn−, Ln−, Lw+, Hw+, Hn+, Ln+, then repeat from the start. This sequence is defined by an instrumental method made in XCalibur software and the instrumental method is kept consistent throughout the experiments. To produce a stable spray and ionization, the following parameters were used: solvent flow 3.0 μL/min, spray voltage 2.9 kV for positive ions and 3.5 kV for negative ions, and capillary temperature 220 °C. The solvent consisted of 3:3:3:1 (vol.) Methanol:Isopropanol:Acetonitrile:Water, with 0.1% (vol.) Acetic Acid. HPLC-grade solvent and acid were obtained from Merck (Merck KGaA, Darmstadt, Germany).

The data are transformed from the proprietary RAW format into the CDF file format, where a spectrum is represented as a set of scans, where each scan is a table with two columns consisting of m/z and the corresponding intensity values. Each scan also has a total ion current attribute and a time attribute, that is the time elapsed since the start of the measurement. In total, 165 fragments of tumor samples and 23 fragments of non-tumor samples were measured.

All simulations were performed in a standard machine-learning cross-validation setup. An algorithm for producing classifier models was applied for each of the eight MS measurement regimes as follows. The spectra were transformed into a feature matrix with individual scans as rows in accordance with the guideline described in [21]. Firstly, the intensities of the spectra were calibrated with regard to the total ion current. Next, m/z aligning and peak detection was performed, followed by the filtering out of peaks presented in fewer than 10 scans per diagnosis. The resulting list of mass spectrometry peaks was transformed into the matrix of peak intensities. Then, all the rows corresponding to the samples with known tumor cell percentage data were set aside and used as the validation set. Columns with little variability (sd < 0.2) and columns with high mutual correlation (rho > 0.8) were excluded from the matrix. The resulting matrix rows were divided into training and testing sets in a 60/40 ratio, ensuring that scans from the same tissue fragment are always assigned to the same set. From the training set matrix, which contained approx. 5000–7000 rows (depending on measurement regime), 600 rows were chosen at random and were used for the optimization of the XGboost algorithm metaparameters. An optimal set of metaparameters were then used to build a classifier using the training set, and the classifier performance was evaluated using a testing subset. To assign a class to the spectrum, predictions for individual scans were aggregated either using voting or averaging the class probability (these are referred to as aggregation methods below).

The 16 tables obtained (8 measurement regimes and 2 aggregation methods) were aggregated into one, using the same method that was used for the individual scans: either voting or a mean probability (meanP) calculation. These aggregated results characterize clinical samples, with all spectra from a sample contributing to the final prediction.

The mass spectra data were preprocessed using the R environment version 4.0.4 with the R packages MALDIquant [27], and model analysis was performed with the KNIME Analytics Platform ver. 4.5.2 [28]. The classification algorithm was realized as the KNIME workflow and is available at the KNIME Community Hub via the link https://kni.me/w/dWtqs1_6S2XVP6EG (accessed on 26 December 2022).

3. Results

The data were split into three sets based on the tissue sample data: the validation set contains all samples, for which tumor cell percentage was evaluated manually by a pathologist [21], and the rest of the data were split into training and testing subsets in a 60/40 ratio. The results obtained for the testing subset are shown in

Table 1. The classifiers validated on the testing subset. Accuracy—the ratio of correctly identified samples to all samples; sensitivity—the ratio of correctly identified tumor samples to all tumor samples; specificity—the ratio of correctly identified nontumor pathology samples to all non-tumor pathology samples.

Resolution	Mode	m/z Range	Aggregation	Sensitivity	Specificity	F-Measure	Accuracy
High	Neg	Wide	vote	1.0	0.833	0.989	0.980
High	Neg	Wide	meanP	1.0	0.556	0.971	0.947
High	Neg	Narrow	vote	1.0	0.833	0.989	0.980
High	Neg	Narrow	meanP	1.0	0.611	0.975	0.954
High	Pos	Wide	vote	0.978	0.889	0.981	0.967
High	Pos	Wide	meanP	1.0	0.333	0.957	0.921
High	Pos	Narrow	vote	0.978	0.833	0.978	0.961
High	Pos	Narrow	meanP	0.993	0.278	0.95	0.908
Low	Neg	Wide	meanP	0.943	0.913	0.960	0.937
Low	Neg	Wide	vote	0.968	0.797	0.959	0.934
Low	Neg	Narrow	vote	0.973	0.754	0.959	0.932
Low	Neg	Narrow	meanP	0.997	0.377	0.931	0.881
Low	Pos	Wide	vote	0.996	0.879	0.984	0.974
Low	Pos	Wide	meanP	0.992	0.862	0.980	0.967
Low	Pos	Narrow	meanP	1.0	0.673	0.966	0.942
Low	Pos	Narrow	vote	0.979	0.769	0.965	0.942

. The voting-based classifiers achieve higher accuracy for all regimes, with the sole exception of positive ion mode, narrow m/z range, low-resolution regime. The probability-based classifiers show a similar sensitivity to the voting-based classifiers; however, the specificity is significantly lower, showing a lesser susceptibility to overfitting for voting-based classifiers. A more balanced dataset could resolve the overfitting issue (as the number of tumor tissue samples was five times that of non-tumor control samples). Ethics and epidemiology limit the number of available tissue samples, especially control samples, which leads to a disproportion between tumor and non-tumor tissue numbers in all studies dedicated to the analysis of ex vivo human brain pathological tissues.

The results obtained using the same algorithm for the validation set are shown in

Table 2. The classifiers validated on the validation subset. Accuracy—the ratio of correctly identified samples to all samples; sensitivity—the ratio of correctly identified tumor samples to all tumor samples; specificity—the ratio of correctly identified non-tumor pathology samples to all nontumor pathology samples.

Resolution	Mode	m/z Range	Aggregation	Sensitivity	Specificity	F-Measure	Accuracy
High	Neg	Wide	vote	1.0	0.25	0.914	0.85
High	Neg	Wide	meanP	1.0	0.0	0.889	0.8
High	Neg	Narrow	meanP	1.0	0.0	0.889	0.8
High	Neg	Narrow	vote	1.0	0.0	0.889	0.8
High	Pos	Wide	meanP	1.0	0.0	0.889	0.8
High	Pos	Wide	vote	0.813	0.75	0.867	0.8
High	Pos	Narrow	vote	1.0	0.5	0.941	0.9
High	Pos	Narrow	meanP	1.0	0.0	0.889	0.8
Low	Neg	Wide	vote	0.992	0.263	0.943	0.897
Low	Neg	Wide	meanP	0.865	0.579	0.897	0.828
Low	Neg	Narrow	vote	0.981	0.316	0.934	0.882
Low	Neg	Narrow	meanP	1.0	0.111	0.931	0.873
Low	Pos	Wide	meanP	1.0	0.438	0.958	0.924
Low	Pos	Wide	vote	1.0	0.353	0.949	0.908
Low	Pos	Narrow	meanP	1.0	0.0	0.929	0.867
Low	Pos	Narrow	vote	0.981	0.063	0.923	0.858

. The accuracy and overall quality are significantly lower for the validation set. The overfitting problem is significantly more profound in the case of the validation set. For more than half of all regimes, the probability-based classifiers show specificity as low as zero, meaning no non-tumor sample was assigned the correct class. The vote-based classifiers show better but still poor specificity.

The aggregation of prediction for eight regimes resulted in better performance than classifiers built using any single regime: out of 26 clinical samples from the validation set, only one sample was classified incorrectly (the same one for the voting-based and the mean probability-based algorithms). Notably, this sample contained 44% tumor cells and 56% non-tumor cells and therefore present the most challenging case for the classifiers in general, as the molecular signatures of both tumor and non-tumor cells are present in such samples in equal proportions. We have chosen the subset of samples with known tumor cell percentage (TCP) as a validation set to be able to understand the influence of the sample heterogeneity on the performance of the classifier. The TCP quantitatively evaluated by histology was not accessed by the classifier during training in any way. That is why it would be incorrect to say that the arbitrary threshold was set in such a way that the samples with TCP higher than 50% were considered a tumor, but instead, we had developed a classifier that is able to separate samples with TCP above 50% from samples with TCP below 50% Strictly speaking, the threshold TCP value should be chosen to be consistent with clinical requirements and considerations that will arise when the proposed method is applied to more specific problems, but to achieve this ability to train classifiers for a predetermined threshold value, we need a much larger set of histologically evaluated samples with a TCP estimate. For the complete set of samples, 100% sensitivity and 97.5% specificity were obtained, corresponding to 1 wrong classification in 214 samples for both voting- and probability-based aggregated classifiers.

4. Discussion

The most common data format in statistical analysis and machine learning is a feature matrix. In the case of mass spectra, the columns of such matrices correspond to m/z values and the rows correspond to the scans. Separate columns can be used to add metadata, such as spectra identifiers, information about the tissue and the patient, etc. A column that contains clinical diagnosis can be used to build a classifier [12], and a column containing an estimate of TCP can be used to train a regression model that predicts this percentage [21]. Notably, the diagnosis should be associated with a clinical biopsy sample rather than a patient because the particular sample can sometimes be taken from tumor margins and contain non-tumor as well as tumor cells. From the molecular point of view, such samples may be more similar to non-tumorous brain tissues.

When building a feature matrix, the intensities are often averaged over all scans of a spectrum or even a tissue fragment (clinical sample). This lessens the noise and smoothens the signal. However, after such averaging, the feature matrix becomes too “wide”: the number of columns is greater than the number of rows, sometimes by order(s) of magnitude. Some methods of statistical analysis are not applicable to such cases. For example, in discriminant analysis, this is called Small Sample Size (SSS) problem.

Sampling techniques, such as bootstrapping, are used to overcome the SSS problem. Bootstrapping is a procedure that produces a multitude of data sets from a single input data set using sampling with replacement. Each resulting data set is used to build a separate classifier. The outputs from these classifiers are then aggregated through averaging or voting, producing the final result.

In this work, the matrix still consists of individual scans, so it is kept “long,” which lessens the impact of a small number of biopsy samples. Variations in peak intensity in the scans have two causes: changes in the extraction process or random differences in the ionization and ion cloud manipulation inside the mass spectrometer. The former can provide information about the cell composition of the tissue and the sample heterogeneity; the latter will still exist in scans of a perfectly homogenous sample, which can be made useful: having a large number of scans with random noise, the resulting algorithm would be less prone to overfitting, which is the main problem of machine learning. This means that analyzing new data that were not present at the training stage will not affect the accuracy of prediction significantly. In this work, this proved to be the case. Due to the random component in the peak intensity, retaining the individual scans in the feature matrix can be considered an “instrumental bootstrapping.”

Usually, machine learning is performed using a single dataset. However, the direct aggregation of the multimodal data into one dataset is not helpful regarding the overfitting problem, as a weak learner will remain a still overfitted weak one. In this work, eight datasets obtained using different MS measurement regimes (polarity, resolution, and m/z range) were used. Since no specific method for the aggregation of such multimodal data has been proposed yet, a joint meta-classifier was produced from all eight separate datasets, using the mixture of experts approach. Two options for obtaining the final result, namely voting and probability averaging, were investigated.

In voting, each classifier made a prediction, and the class that was predicted by the majority of classifiers for each scan (for single-regime classifiers) or regime became the final prediction. In probability averaging, the probability of belonging to each class was calculated using XGboost for each scan or regime separately, then these probabilities were averaged, and the data were assigned a class using probability thresholds.

For a validation set, we used data that were previously annotated by a professional pathologist to build a regression model for tumor cell percentage prediction [21]. This way, additional information on tumor cell percentages are available for tumor samples. This can be important because there were tumor samples with as little as 5% of actual tumor cells. This quantitative value enables the investigation of algorithm robustness concerning the heterogeneity of clinical samples. This approach has been useful because it provided a likely explanation for the only sample for which the class was incorrectly assigned by both meta-classifiers (voting-based and probability-based), as it was highly heterogenous, as we have shown above.

It is worth noting that since eight regimes are measured instead of one, the measurement itself takes significantly more time. Under the current ICE-MS experimental protocol, the cartridge preparation takes no more than 2 min, and one regime is measured in 30 s. Some additional (device- and regime-specific) time needs to be added to account for switching between regimes, so for this work, eight regimes are measured in 4.8 min. Taking all the steps together, measuring eight regimes takes slightly less than 7 min, while one regime can be measured in 2.5 min. Performing a technical repeat immediately after measuring the eight regimes shows that the sample is not significantly exhausted during this time [22]. Despite this increase in time, improved classification performance is important for the development of the method, especially if it is aimed at clinical practice.

The obvious next step should be estimating the contribution of each measurement regime to the prediction of the aggregated classifier so that an informed decision could be made about which regimes should be measured. Preliminary analysis shows that positive polarity alone gives slightly better results than negative because of a lower number of false positives. This task, however, is not as simple as it seems. Simple sequential discard of one regime from classification would only work in the case of a linear contribution of regimes. Since various classes of lipids and metabolites could be detected in different polarities, it is obvious that the aggregation of positive and negative modes data will improve the classification results; however, the effect of spectra resolution, as well as the mass range, is not clear. Analysis of Table 1 shows that the wider mass range provides better specificity in all polarity and resolution values, and high-resolution data gives better accuracy than low-resolution data. But in some combinations the contribution of polarities could be the opposite—positive mode provides better results than negative mode with voting in low-resolution data but worse in high-resolution. The most reliable way to estimate such contributions is a calculation of each regime’s Shapley value, but this procedure is quite time-consuming, and taking the overfitting problem into account, it should be repeated with several train/test data splits. Such calculations are an ongoing project of the group, but they are still far from complete.

5. Conclusions

Tumor versus non-tumorous pathology classifiers using a single mass spectrometric measurement regime provide suboptimal performance. We have shown that the performance can be significantly improved when aggregating predictions from several such classifiers that use different measurement regimes with different polarity, m/z range, and resolution. Additional information on the tumor cell percentage of the samples is beneficial for interpreting predictions obtained from such classifiers but is extremely cumbersome. It is shown that in contrast to vote-based classification, probability-based classifiers result in lower accuracy due to overfitting that originated from the lower availability of non-tumor pathological human brain tissue samples. However, the mixture of experts approach provides the same classification efficiency for both aggregation methods. Thus, the collection and analysis of multimode data simplify the implementation of machine learning methods to unbalanced sample cohorts typical of clinical applications. We are estimating the contribution of each regime to the final classifier accuracy for the future development of a reliable tumor tissue classifier.