FFICL-Net: A Fusing Symmetric Feature-Importance Ranking Contrastive-Learning Network for Multivariate Time-Series Classification

Abstract

Supervised contrastive learning has emerged as a novel method to help solve the problem of multivariate time-series classification. By utilizing labeled data, it maximally learns the feature-representation differences between various categories. However, existing supervised contrastive-learning approaches lack model interpretability, making it difficult to analyze the importance ranking among features. Experimentally, different preprocessing of the data often results in variations in feature-importance ranking. Therefore, we propose FFICL-Net, which combines LSTM, to analyze the importance of sequence variables, and ITransformer, to treat each variable as a token, learning the self-attention relationships between variables and their relationship to the final feature representation. This approach contrasts the feature importance derived from the two modules, making two feature-importance ranking results more similar and forming a kind of symmetry, allowing the resultant feature representation to fuse the characteristics of both models, leading to more stable and accurate feature-importance ranking results and aiding in improving classification accuracy. We conducted comparative experiments on all 30 public UEA datasets and achieved the best results on most of these datasets compared to the current top models. The average accuracy reached 72.8%, which is an improvement of 0.7% over the best-performing model.

1. Introduction

Multivariate time-series data [1] refers to a collection of data points across multiple related variables over a set period of time at specified intervals. This type of data is crucial and widely used in various fields, including economics [2], climate [3], medicine [4], and social sciences [5]. Through multivariate time series, research tasks such as classification, forecasting [6], and imputation [7] can be performed on existing time-series data.

Contrastive learning [8] is often used in time-series classification tasks to address the issue of scarce labeled data. With contrastive learning, effective feature representations can be learned through self-supervised or semi-supervised learning under limited labeled conditions. For multivariate time series with complete classification labels, contrastive learning can also enhance the performance of supervised learning by generating similar representations for time-series data of the same category, while differentiating the feature representations between different categories.

However, current supervised contrastive learning [9] often lacks model interpretability to yield variable importance results for multivariate variables due to its black-box property and the sole focus on extracting important feature representations. In recent years, with the development of machine-learning and deep-learning technologies, an increasing number of researchers have begun to focus on the interpretability of computational models. Attention mechanisms are widely applied in the field of interpretability to highlight important features in the data. The classic Transformer [10] structure uses self-attention to focus on relationships between different time steps, while the ITransformer [11] structure, in contrast, focuses on the relationships between variables. However, this inversion initially ignores the sequential computation of each time step in the time series when extracting and embedding features, which is especially critical for multivariate time-series data.

LSTM (Long Short-Term Memory) [12] is effective at capturing long-term dependencies and avoiding problems of gradient vanishing or exploding. However, traditional LSTM models lack interpretability regarding the influence of each multivariate variable during sequential computation and are unable to determine feature importance.

To address the limitations of ITransformer and LSTM in time-series feature extraction, as well as their lack of interpretability, which makes it challenging to obtain stable and accurate feature-importance rankings, we designed FFICL-Net, a feature-importance contrastive-learning classification network. To achieve more stable and accurate feature-importance results that aid supervised contrastive learning in feature-representation learning, we expanded the traditional supervised contrastive-learning approach. We designed an interpretable LSTM model to learn the feature importance of each variable at every time step in the time series and an interpretable ITransformer model to learn the feature importance among variable embeddings. The feature-importance outcomes from both modules are compared, aiming to reduce their differences and incorporate this as a loss function. Additionally, the fused feature importance is used to weight the feature representations, assisting in the classification of multivariate time series.

Specifically, our work contributes in four main areas:

• To address the issue of unstable feature-importance results obtained from interpretable models, we introduced contrastive-learning methods to predict time-series data. Feature fusion is performed across different temporal dimensions of the time series, which is beneficial for both the prediction and interpretation of the time series.
• We design two methods incorporating attention mechanisms, adding attention layers to the ITransformer and LSTM, respectively, to capture the feature importance of variables in time-series data.
• We propose a new loss function that combines supervised contrastive-learning loss, feature-importance contrast loss, and classifier loss.
• Experimentally, we test our methods on thirty public UEA (University of East Anglia) datasets, achieving results that surpass current state-of-the-art time-series classification models.

2. Related Works

2.1. Multivariate Time-Series Classification

In recent years, the problem of time-series classification has seen extensive application research in fields such as medicine, finance, and engineering. Multivariate time series, where each sequence has multiple dimensions—for instance, ECG (Electrocardiogram) [13] and EEG (Electroencephalogram) [14] data—have garnered significant attention from researchers. In 2018, the UEA archive [15] published 30 multivariate time-series classification datasets, providing a robust benchmark for comparing the strengths and weaknesses of proposed algorithms.

Like many deep-learning challenges, multivariate time-series classification issues can often be addressed through feature extraction using convolutional neural networks. Chia-Yu Hsu [16] and others proposed an MTS (Multivariate Time-Series)-CNN (Convolutional Neural Network) model for multivariate time-series classification, which extracts temporal features along sub-sequences of a sliding window using CNNs, followed by a fully connected layer to classify faulty semiconductor components. CNNs do not intuitively utilize the temporal step features of time series, whereas LSTM [17] structures, through gating units, effectively transmit and preserve information over long time series. Zhao [18] and colleagues introduced LSTM-MFCN (Multi-scale Fully Convolutional Network), which combines LSTM and CNN modules to tackle multivariate time-series classification, constructing a multi-scale receptive field with MFCN [19] to extract spatial features over varying ranges, while the LSTM network captures temporal information, significantly improving the model’s classification performance by integrating spatio-temporal features.

Furthermore, the Transformer [20] model addresses long-term dependencies in time series through self-attention mechanisms that process an entire sequence for embedding. Liu [21] and others expanded the gated Transformer network by merging two Transformer blocks, combining time channel and time step feature information for classification. In contrast, ITransformer [11] does not change the Transformer’s network architecture but switches the roles of self-attention mechanisms and feed-forward networks. ITransformer considers different variables separately, encoding each variable as an independent token and using self-attention to model inter-variable correlations while using feed-forward networks to model the temporal correlations of variables, achieving better sequential temporal representations.

However, current research, such as LSTM and ITransformer, lacks interpretability and cannot extract certain features of time series.

2.2. Interpretable Time-Series Classification

Unlike the general field of computer vision, time-series classification can employ CAM (Class Activation Mapping) [22], using attention maps to analyze important regions within images. Inês Neves [23] researched an Explainable Artificial Intelligence (XAI) scheme, introducing an advanced conceptual framework for explainable time series, including PFI (Permutation Feature Importance) [24], LIME (Local Interpretable Model-agnostic Explanations) [25], and SHAP (SHapley Additive exPlanations) [26]. More specifically, Kevin Fauvel [27] proposed XCM (eXplainable Convolutional neural network for Multivariate time-series classification), a new compact convolutional neural network that directly identifies and extracts observation variables and time steps crucial for classification outcomes from the input data. Similarly, Tsung-Yu Hsieh [28] adopted a CAM-like attention mechanism network to specifically analyze the contributions of input variables to classification, using convolution-based feature extraction layers and attention mechanism layers to concurrently assess the importance of variables across the entire time series and at different time steps for classification results. However, the results obtained through existing interpretable models are often unstable.

2.3. Contrastive Learning for Time-Series Classification

The goal of contrast learning [8] is to encode similar data from the same category similarly, while ensuring that encodings of different category data differ as much as possible. Contrast learning introduces additional information to effectively extract features. Supervised contrast learning [9] goes further by specifying that the external information to be extracted is related to different image classifications, differing from general self-supervised contrast learning, pairing each sample with its data-augmented counterpart as positive pairs, and each sample with other samples as negative pairs. Supervised contrast learning uses existing label information to better reduce the distance between feature representations of the same category in feature space and increase the distance between different categories, which is greatly beneficial for classification issues.

Moreover, in time-series classification contrast learning, differing from general data augmentation, Luo [29] proposed InfoTs, a new information-aware contrast-learning method that adaptively selects the optimal augmentations for learning time-series representations. Zhang [30] introduced SMDE, a unique time-series contrastive-learning framework, diverging from conventional techniques that predominantly concentrate on instance-level contrast. SMDE leverages signal decomposition and ensemble methodologies, enhancing focus on localized mode alterations within time-series signals.

Therefore, we combined interpretable time-series classification and a supervised contrastive-learning framework to address the problems of existing time-series models lacking interpretability and existing interpretable models producing unstable results.

3. Method

In this section, we propose a supervised contrastive-learning model that integrates feature importance (FFICL-Net); the overall framework is illustrated in Figure 1. By combining the interpretable LSTM and interpretable ITransformer, we obtain more accurate feature-importance ranking. Furthermore, employing an end-to-end supervised contrastive-learning framework enables the model to extract more precise feature representations to aid classification. Implementation and functionality of the interpretable LSTM module, interpretable ITransformer module, and improved supervised contrastive-learning framework will be detailed in the following subsections.

3.1. Problem Formulation

Define a multivariate time series as $X=(X_1,X_2,\dots ,X_T)$, where T denotes the length of the time series, starting from time step 1 and ending at time step T. Each $X_t=[x_t^1,\dots ,x_t^p]$, where $x_t$ belongs to ${\mathbb{R}}^p$ and p represents the dimension of the multivariate variables, indicating that there are p variables that collectively constitute the data at time step t. For each X, there is a corresponding label y, which represents the true label of this time series. The task is to learn the features and inherent information of the time series from a set of known input multivariate time series and labels, where $\mathcal{X}=\{(X^1,y_1),\dots ,(X^N,y_N)\}$, N represents the number of samples, and predict the labels y for other multivariate time series.

While obtaining the classification results for each time series, we also aim to determine the influence of each multivariate variable on the classification outcome, that is, the feature-importance results $I=[a_1,\dots ,a_p]$, where I belongs to ${\mathbb{R}}^p$ and $a_p$ represents the attention score of the p-th variable.

$$\begin{array}{c}\hfill \sum _{p=1}^P{a}_p=1\end{array}$$

(1)

3.2. Interpretable LSTM

The interpretable LSTM module processes the original time-series data, capturing hidden states at each time step. To enhance the interpretability of the standard LSTM framework, we adopt a strategy where the hidden states corresponding to each variable are separated and then encoded individually, producing distinct feature representations for each variable. These representations are subsequently evaluated through an attention mechanism, which assigns a weight to each variable based on its relevance to the classification task. The architecture of the model is illustrated in Figure 2a.

To modify the original LSTM network structure, we define ${\tilde{h}}_t={[h_t^1,\dots ,h_t^P]}^T$, where ${\tilde{h}}_t\in {\mathbb{R}}^{P\times d},h_t^p\in {\mathbb{R}}^d$, ${\tilde{h}}_t$ as the hidden state matrix at time step t, where $h_t^p$ represents the state of the p-th variable at time t, defined as a custom-length vector of size $1\times d$. Consequently, the entire ${\tilde{h}}_t$ layer has a size of $D=P\times d$, where P is the number of variables and d is the dimension of each variable’s state vector.

Subsequently, to convert the input vector $x_t$ at time t into a vector of size $P\times d$, we define the transformation matrix parameter $U_j=$ ${[U_i^1,\dots ,U_i^P]}^T,\mathrm{where}{U}_i\in {\mathbb{R}}^{P\times d\times d},U_i^p\in {\mathbb{R}}^{d\times d}$ with dimension d. Additionally, to preserve the LSTM network’s capability of recursively extracting information from the time series, we define another transformation matrix parameter, $W_j=[W_j^1,\dots ,W_j^P],\mathrm{where}{W}_j\in {\mathbb{R}}^{P\times d\times d}\mathrm{and}{W}_j^p\in {\mathbb{R}}^{d\times d}$. Using $U_j$ and $W_j$, we can perform the necessary computations.

$$\begin{array}{c}\hfill {\tilde{j}}_t=tanh\left(W_j⊛{\tilde{h}}_{t-1}+U_j⊛x_t+b_j\right)\end{array}$$

(2)

Here, ${\tilde{j}}_t\in {\mathbb{R}}^{P\times d}$; each $j_t^p$ represents the hidden state of the variable p at that moment. The tensor dot operation ⊛ is defined as the product of two tensors along the P axis.

Further, by replacing ${\tilde{j}}_t$ as the hidden state update in the standard LSTM, we can recursively update the input gate $i_t$, forget gate $f_t$, output gate $o_t$, and the memory cells $c_t$ using the following equations:

$$\begin{array}{cc}\hfill \left[\begin{array}{c}{i}_t\\ f_t\\ o_t\end{array}\right]& =\sigma \left(W\left[x_t\oplus \mathrm{vec}\left({\tilde{h}}_{t-1}\right)\right]+b\right)\hfill \end{array}$$

(3)

$$\begin{array}{cc}\hfill c_t& =f_t\odot c_{t-1}+i_t\odot \mathrm{vec}\left({\tilde{j}}_t\right)\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill {\tilde{h}}_t& =\mathrm{mat}\left(o_t\odot tanh\left(c_t\right)\right)\hfill \end{array}$$

(5)

In the above formulas, $\mathrm{vec}(·)$ refers to the vectorization operation, which concatenates the columns of a matrix into a single vector. The concatenation operation is denoted by ⊕, and element-wise multiplication is denoted by ⊙. The operator $\mathrm{mat}(·)$ reshapes a vector from ${\mathbb{R}}^D$ into a ${\mathbb{R}}^{P\times d}$ matrix.

After recursively obtaining the final step $\widetilde{h_T}$, we can compute the feature importance I using the formula:

$$\begin{array}{c}\hfill I_A=\sum _{p=1}^P{a}_p\ast \widetilde{h_T^i}\end{array}$$

(6)

3.3. Interpretable ITransformer

For the ITransformer, unlike the Transformer, which encodes data points at the same time step into a unified temporal token using self-attention mechanisms to model temporal correlations between different moments, the ITransformer encodes different variables separately. This approach addresses the issue in Transformers where inter-variable correlations are often neglected. The tokens produced by the ITransformer, combined with self-attention mechanisms, can reveal the correlations between different variables.

However, concerning the issue of feature importance, the self-attention mechanism in the ITransformer only considers the relationships between different variables and does not take into account the correlation between the weights of the variables and the final feature representation. We have improved the original ITransformer structure by adding an additional layer of attention mechanism to extract the feature importance of variables and integrate it with the existing feature vectors. This enhancement allows the model to focus more on important variables, resulting in better feature representations. The model structure is illustrated in Figure 2b.

For the input $X_T$, it first goes through an Embedding process, as shown in the equation:

$$\begin{array}{c}\hfill M=\mathrm{Embedding}\left(X_T\right)\end{array}$$

(7)

This reduces the original input vector with time-series length T to D dimensions. That is, $\mathrm{Embedding}:{\mathbb{R}}^T\to {\mathbb{R}}^D$, $M=\{m_1,\dots ,m_P\}$ belongs to ${\mathbb{R}}^{P\times D}$. Here, each $m_p$ represents the intermediate feature extracted for the p-th variable, with a time step length smaller than the original length. Through attention, the importance $I_B$ of the time series after embedding can be obtained.

The attention layer employs two linear layers to calculate the attention weights, and the formulas are given by:

$$\begin{array}{c}\hfill s_t={\sigma }_2\left({\sigma }_1\left(W_1{m}_t+B_1\right)W_2+B_2\right)\end{array}$$

(8)

$$\begin{array}{c}\hfill I_B=\mathrm{softmax}\left(s_t\right)\end{array}$$

(9)

where $W_1$, $W_2$, $B_1$, and $B_2$ are parameters that the model learns, and ${\alpha }_1$ and ${\alpha }_2$ are the nonlinear activation functions used, here chosen to be tanh.

3.4. Improved Supervised Contrastive Learning

To extract better feature information, a common approach is to use supervised contrastive learning. Supervised contrastive learning is an extension of contrastive learning, which is generally unsupervised. For classification models, supervised contrastive learning leverages the additional label information to maximize the differences between categories. Furthermore, to address the issue of unstable important feature indicators produced by the interpretable LSTM, we decided to refine the supervised contrastive-learning approach by comparing the feature-importance results obtained from the interpretable LSTM and the interpretable ITransformer discussed in the first two sections. The feature-importance results obtained from the two explainable models should be similar; therefore, the differences in their comparisons should be minimized. By computing feature importance twice, we aim to ensure that the feature-importance results obtained on the dataset do not vary significantly with different training sets.

After obtaining accurate feature-importance rankings through the refined supervised contrastive learning, we employed an end-to-end model to integrate the learned feature representations with the feature-importance ranking results to obtain the final classification feature Z. Z is then used as the input vector for both the supervised contrastive learning and the classification model.

$$\begin{array}{c}\hfill Z=H\ast I_B^T\end{array}$$

(10)

Next, the representation $\mathbf{Z}$ is fed into a classifier g, which produces the estimated sample label $\widehat{y}=g\left(\mathbf{Z}\right)$. With the ground truth $y\in \mathcal{Y}$, we measure the classification loss.

3.5. Loss Function

The proposed end-to-end model is trained by minimizing a hybrid loss function L, which combines feature-importance contrastive loss, supervised contrastive loss, and classification loss with weighted terms.

Feature importance contrastive loss: Feature-importance contrastive loss specifically seeks to reduce the differences in feature importance derived from both the interpretable LSTM and interpretable ITransformer models, making two feature-importance ranking results more similar and forming a kind of symmetry. By minimizing this loss, the model learns to align the feature-importance representations between two temporal patterns to ensuring greater stability of the model.

$$\begin{array}{c}\hfill L1={\displaystyle \frac{1}{N}}\sum _{n=1}^N(I_B^n-I_A^n)\end{array}$$

(11)

Supervised contrastive loss: The supervised contrastive loss $L2$ encourages the model to increase the similarity between representations of samples from the same class while reducing the similarity between representations of samples from different classes, thereby aiding in learning meaningful feature representations. Unlike unsupervised loss, the true labels of the samples are directly used during the model’s training process, which helps the model capture the distinguishing features between different classes. The formula for this loss is as follows:

$$\begin{array}{c}\hfill L2=-log{\displaystyle \frac{{\sum }_{y=y^{+}}exp(\mathrm{sim}(z,z^{+})/\tau )}{{\sum }_{y=y^{-}}exp(\mathrm{sim}(z,z^{-})/\tau )}}\end{array}$$

(12)

$Z^{+}$ is the representation derived from samples that belong to the same class as Z, whereas $Z^{-}$ is derived from samples with different labels.

Classification loss: We use a loss function widely employed for multi-class classification problems, the softmax cross-entropy loss function. The specific formula is as follows:

$$\begin{array}{c}\hfill L3=-{\displaystyle \frac{1}{N}}\sum _{i=1}^N\sum _{c=1}^C\{y_i=c\}log{\widehat{y}}_c\end{array}$$

(13)

where C represents the different classes and N denotes the number of samples.

Hybird loss: The hybrid loss combines the three types of losses mentioned above and weights their sum. The specific formula is as follows:

$$\begin{array}{c}\hfill L=L1+L2+L3\end{array}$$

(14)

The hybrid loss enables the model to effectively balance the contributions from feature-importance contrastive, supervised contrastive, and classification losses, thereby achieving the best possible classification performance for the given time-series classification task.

4. Results

We conducted comparative experiments on thirty public UEA datasets, comparing with State-Of-The-Art (SOTA) models such as ITransformer and MICOS. We tested the differences in classification effectiveness between FFICL-Net and Transformer-like networks, and mixed supervised contrastive-learning models. For the issue of interpretability, we specifically calculated the feature-importance ranking on the FaceDetection dataset from among the thirty datasets and conducted a series of validation experiments.

4.1. Datasets

UEA datasets: The UEA dataset is currently an important open-source dataset resource in the field of time-series mining. Each dataset sample includes a class label and contains time-series data such as ECG/EEG brain signals, human activity recognition, and motion classification. The detailed information about the thirty UEA datasets we used is shown in

Table 1. The basic information and feature dimensions of all 30 UEA datasets.

Dataset	Traincases	Testcases	Dimensions	Length	Classes
ArticularyWordRecognition	275	300	9	144	25
AtrialFibrillation	15	15	2	640	3
BasicMotions	40	40	6	100	4
CharacterTrajectories	1422	1436	3	182	20
Cricket	108	72	6	1197	12
DuckDuckGeese	60	40	1345	270	5
EigenWorms	128	131	6	17,984	5
Epilepsy	137	138	3	206	4
EthanolConcentration	261	263	3	1751	2
ERing	30	30	4	65	6
FaceDetection	5890	3524	144	62	2
FingerMovements	316	100	28	50	2
HandMovementDirection	320	147	10	400	4
Handwriting	150	850	3	152	26
Heartbeat	204	205	61	405	2
JapaneseVowels	270	370	12	29	9
Libras	180	180	2	45	15
LSST	2459	2466	6	36	14
InsectWingbeat	30,000	20,000	200	78	10
MotorImagery	278	100	64	3000	2
NATOPS	180	180	24	51	6
PenDigits	7494	3498	2	8	10
PEMS-SF	267	173	963	144	7
Phoneme	3315	3353	11	217	39
RacketSports	151	152	6	30	4
SelfRegulationSCP1	268	293	6	896	2
SelfRegulationSCP2	200	280	7	1152	2
SpokenArabicDigits	6599	2199	13	93	10
StandWalkJump	12	15	4	2500	3
UWaveGestureLibrary	120	320	3	315	8

.

FaceDetection: FaceDetection [15] includes MEG (Magnetoencephalography) recordings and category labels, where class 0 represents Scramble and class 1 represents Face. The data come from ten subjects (subject01 to subject10) with test data from six additional subjects (subject11 to subject16). Each subject has approximately 580–590 trials. Each trial includes 1.5 seconds of MEG recording and the corresponding category label.

4.2. Dataset Format and Preprocessing

As some datasets in the UEA dataset we used were not normalized, we have normalized these datasets with Z-score normalization.

$$\begin{array}{c}\hfill x^{\prime }={\displaystyle \frac{x-\mu }{\sigma }}\end{array}$$

(15)

Meanwhile, to ensure the security of the dataset during transtmission, we utilized a new method [31] for the encryption of medical datasets.

4.3. Training Details

For our end-to-end model, we have a Tesla P100 GPU device for 100 epochs, with a batch size of 16 and a learning rate of 1 $\times {10}^{-3}$. The loss function is as described in Section 3, using the adam optimizer.

4.4. Results

To validate the effectiveness and generalizability of our experiment, we conducted tests on thirty public UEA datasets, comparing them with classical methods (DTW (Dynamic Time Warping) [32], MLP (Multilayer Perceptron) [33] and LSTM [17], Transformer-based methods (FED [34] and Crossformer [35]), and recent publicated state-of-the-art methods (ITransformer [11], TimesNet [36], and SMDE [30]). As shown in

Table 2. Model accuracy comparison on 30 public UEA datasets, compared with the results of classical methods, Transformer-based methods, and recent publicated state-of-the-art methods on 30 UEA datasets.

Datasets/Models	Classical Methods		RNN	Transformers				Mixed Supervised		Ours
	DTW-KNN	MLP	MLSTM-FCN	FED	CrossF	ITransformer	TimesNet	TS-TCC	SMDE
	(1994)	(2014)	(2019)	(2022)	(2022)	(2023)	(2023)	(2022)	(2024)
ArticularyWordRecognition	93.0	97.3	97.3	97.7	98.0	98.0	96.2	97.3	96.3	98.0
AtrialFibrillation	20.0	46.7	26.7	53.3	46.7	40.0	33.3	30.0	20.0	40.0
BasicMotions	100.0	85.0	95.0	92.5	90.0	92.5	100.0	100.0	97.5	100.0
CharacterTrajectories	96.1	98.8	98.5	99.1	98.2	99.4	99.2	99.1	99.2	99.7
Cricket	97.2	91.7	91.7	84.7	84.7	98.6	87.5	93.8	100.0	100.0
DuckDuckGeese	44.0	42.0	67.5	57.5	60.0	55.0	62.5	42.0	60.0	66.0
EigenWorms	63.4	52.7	50.4	61.8	55.0	83.2	84.0	39.3	84.0	87.8
Epilepsy	97.8	60.1	76.1	65.9	73.2	73.2	78.1	95.7	97.1	74.6
ERing	91.9	82.6	94.1	92.9	84.4	93.3	91.4	94.4	90.0	93.7
EthanolConcentration	28.9	33.5	37.3	28.9	35.0	31.2	25.1	27.2	28.9	32.3
FaceDetection	54.9	67.4	54.5	68.9	66.2	66.0	67.8	54.9	54.9	70.3
FingerMovements	53.0	64.0	58.0	62.0	64.0	60.0	59.0	47.0	53.0	62.0
HandMovementDirection	24.3	58.1	36.5	58.1	58.1	40.0	50.0	40.0	37.8	50.0
Handwriting	27.2	22.5	28.6	26.0	26.2	28.0	23.1	39.2	41.9	32.0
Heartbeat	69.3	73.2	66.3	76.6	76.6	73.7	75.1	69.5	74.6	78.5
InsectWingbeat	N/A	10.0	16.7	10.0	27.6	11.0	10.0	47.9	55.8	21.8
JapaneseVowels	88.1	97.8	97.6	98.7	98.9	98.4	96.5	90.8	97.0	98.1
Libras	81.1	73.3	85.6	81.1	76.1	87.0	77.8	78.3	85.0	87.8
LSST	52.6	35.8	37.3	67.8	42.8	57.5	59.2	39.1	61.9	62.7
MotorImagery	46.0	61.0	51.0	61.0	61.0	59.0	51.0	51.0	62.0	59.0
NATOPS	85.0	93.9	88.9	96.7	88.3	93.9	81.8	80.0	91.7	86.7
PEMS-SF	79.8	82.1	69.9	88.4	82.1	80.9	83.2	63.5	80.9	89.6
PenDigits	94.6	93.0	97.8	97.3	93.7	97.6	98.2	96.7	98.2	98.1
PhonemeSpectra	13.3	7.1	11.0	11.7	7.6	10.1	18.2	16.5	21.9	11.0
RacketSports	84.9	79.0	80.3	84.2	81.6	81.6	82.6	82.5	84.2	83.6
SelfRegulationSCP1	75.6	88.4	87.4	89.8	91.5	88.7	91.8	89.0	89.4	92.2
SelfRegulationSCP2	48.3	51.7	47.2	54.4	53.3	54.4	53.3	53.3	57.8	58.3
SpokenArabicDigits	90.6	96.7	99.0	99.7	96.4	98.3	98.4	99.8	97.8	99.1
StandWalkJump	40.0	60.0	6.7	60.0	53.3	53.3	53.3	45.0	53.3	60.0
UWaveGestureLibrary	84.7	81.9	89.1	80.0	81.6	87.5	85.3	80.6	91.9	91.3
Average Accuracy	64.2	68.4	64.8	70.2	68.4	69.7	69.1	66.1	72.1	72.8
Best accuracy count	3	3	2	5	3	1	1	3	6	12
Average Rank	7.33	6.7	6.47	4.57	5.72	5.1	5.43	6.5	4.5	2.95

, our model achieved the best results in 12 out of 30 datasets, with the highest average accuracy as well. Particularly in the FaceDetection dataset, a large dataset, our model achieved a 1.5% improvement, demonstrating that the proposed contrastive-learning framework can learn more complex features. Figure 3 shows the critical difference plot of all the above methods.

In the FaceDetection dataset, we also used 5-fold cross-validation on five different metrics to measure the model’s classification results: AUC, Accuracy, Recall, Specificity, and F1 score. The experimental results were averaged and are shown in

Table 3. Comparison of different models on FaceDetection dataset. This table shows the results of the five evaluation indicators AUC, Accuracy, Recall, Specificity, and F1-Score.

Metrics	Methods	Result
AUC	SMDE	0.67
	ITransformer	0.71
	FFICLNet	0.74
Accuracy	SMDE	0.651
	ITransformer	0.685
	FFICLNet	0.703
Recall	SMDE	0.654
	ITransformer	0.68
	FFICLNet	0.642
Specificity	SMDE	0.648
	ITransformer	0.665
	FFICLNet	0.741
F1-score	SMDE	0.652
	ITransformer	0.675
	FFICLNet	0.676

, indicating that our model, compared to the baseline, achieved the best results in all five evaluation metrics—AUC, Accuracy, Recall, Specificity, and F1 score.

The ROC curve comparison of the three models is shown in Figure 4:

4.5. Interpretability Analysis

We validated feature-importance ranking on the FaceDetection dataset using 5-fold cross-validation. The importance ranking results obtained from a single experimental model were used as the standard. We selected a subset of features, tested every ten dimensions on classical sequence models and generalization tests, validating these on the other four validation sets in the 5-fold cross-validation. The average accuracy was calculated, and all the results in this section are the averages of multiple tests.

Feature importance ranking results: To verify that our method can obtain stable feature-importance ranking results, we selected multiple epochs from 5-fold cross-validation and extracted their feature-importance ranking results. A total of five attention scores were obtained, and we calculated the cosine similarity between each pair, then compared the minimum cosine similarity among the five groups to analyze the differences in feature-importance ranking results.

As can be seen from

Table 4. The minimum cosine similarity changes with the number of training epochs.

Epoch	Epoch 1	Epoch 25	Epoch 50	Epoch 75	Epoch 100
Minimum Cosine Similarity	0.5484	0.6921	0.8247	0.8761	0.9212

, with the increase in training rounds, the minimum cosine similarity also gradually increases, indicating that the differences among the five attention scores become smaller and smaller. Through the improved supervised contrastive-learning framework, combined with the attention scores of interpretable LSTM and interpretable ITransformer, we can obtain stable feature-importance ranking results. The subsequent subsections will analyze the correctness of the obtained feature-importance ranking results.

Model self-testing on selected dimensions: Similarly, we utilize the feature-importance ranking results $I_A$ obtained from the FaceDetection training dataset. We select the top 25 dimensions (1–25), middle 25 dimensions (50–75), and the least important 25 dimensions (119–144). As shown in Figure 5a, the linear accuracy of FFICL decreases in descending order of dimensions, validating the correctness of our model’s importance ranking. The less important dimensions provide less assistance to classification.

Temporal recursive models on selected dimensions: We tested every ten dimensions on the classical sequence models RNN (Recurrent Neural Network) [37], LSTM [17], and BiLSTM [38] using the by-product feature-importance ranking results $I_A$ obtained during training on Face Detection. As shown in Figure 5b, initially, the accuracy remained high because features with higher importance were more correlated with the ground truth. However, as the importance decreased continuously, unimportant features could not assist these sequence models in correctly classifying, leading to a sharp decline in accuracy.

Out-of-distribution generalization: Apart from validation on in-domain data, we also tested the model on another dataset, SelfRegulationSCP1. We used the model parameters trained on FaceDetection as a pre-trained model to test the importance ranking obtained from the training of SelfRegulationSCP1. SelfRegulationSCP1 comprises six features, and for each iteration one feature is removed, and classification is performed using the remaining features. As shown in Figure 5c, removing more important feature indicators results in a descending order of accuracy in model classification.

4.6. Ablation Study

In order to compare and contrast the effects of the learning framework and the explainable attention mechanism, we conducted the following ablation experiments. The specific experimental results are shown in

Table 5. An ablation experiment conducted on the classification models of the unused FFI (Fusion Feature Importacne) module and unused CL (Contrastive Learning) module, respectively (“w/o” means none). The AUC, Accuracy, Recall, Specificity, and F1-Score results obtained from the experiments are shown in this table.

Metrics	Methods	Result
AUC	FFICLNet w/o FFI	0.69
	FFICLNet w/o CL	0.72
	FFICLNet	0.74
Accuracy	FFICLNet w/o FFI	0.66
	FFICLNet w/o CL	0.672
	FFICLNet	0.703
Recall	FFICLNet w/o FFI	0.589
	FFICLNet w/o CL	0.68
	FFICLNet	0.642
Specificity	FFICLNet w/o FFI	0.727
	FFICLNet w/o CL	0.665
	FFICLNet	0.741
F1-score	FFICLNet w/o FFI	0.631
	FFICLNet w/o CL	0.675
	FFICLNet	0.676

.

It can be seen that, if only interpretable LSTM and only interpretable ITransformer are used, without a contrastive-learning structure, the accuracy obtained is 67.2%. Similarly, if only the original ITransformer structure is compared and learned without feature-importance score, the accuracy obtained is 66%. Thus, we can see the role of the two modules.

It is worth noting that the accuracy obtained using only ITransformer in Table 3 is 68.5% higher than 67.2% in Table 5, where both interpretable ITransformer and interpretable LSTM are used for feature fusion. This is because, when interpretable ITransformer and interpretable LSTM undergo simple feature fusion, they produce different feature-importance ranking results, and their attention scores affect the final classification results. This is also one of the reasons for adding the CL module, to ensure that the two feature-importance ranking results are more similar.

5. Discussion

The novel interpretable supervised contrastive-learning framework proposed in this study can be applied to various time series to rank the importance of each feature within the time series. This holds significant implications for time-series data in fields such as medicine and finance. For instance, the importance ranking results can be further analyzed to assess the impact of medical indicators on patient prognosis or to identify the factors that most significantly influence market prices in financial data. Additionally, the improved supervised contrastive-learning framework presented in this paper offers a potential approach for contrastive learning to extract better feature representations.

However, this study also has certain limitations. First, the method for testing the importance of indicators is not sufficiently robust, lacking a quantitative way to prove the effectiveness of the feature-importance results. The model also performs poorly on datasets with fewer features. Secondly, the model was only tested on the public UEA datasets, which do not provide detailed information about the dimensionality of features. Thus, the feature-importance rankings obtained cannot provide meaningful explanations as in datasets from the finance or climate sectors, where some prior knowledge of feature importance is available from other studies to aid or validate explainable models.All of the aforementioned areas are worthy of further research.

6. Conclusions

The interpretability of deep-learning models for time-series classification is a significant area of research. Previous explainable models often failed to provide a stable importance ranking, with results potentially varying significantly based on the choice of training set random seeds. In this study, we introduced FFICL-Net for classifying multivariate time series. We conducted tests on thirty public UEA datasets to validate the model’s generalizability and the stability of feature-importance rankings obtained during training.

The introduced improvements in the contrastive-learning framework and attention mechanisms effectively enhanced the accuracy of the final outcome classification for patients. Combining the feature-importance rankings from the interpretable LSTM with those from the improved attention mechanism of the interpretable ITransformer helped to effectively extract features. The average accuracy achieved on the thirty datasets was 0.7% higher than previous State-Of-The-Art (SOTA) models. It can be said that the proposed integrated model outperforms existing research and provides valuable insights for subsequent studies.

In summary, our proposed model that integrates feature-importance ranking and contrastive learning is the first to combine explainable modules and improve the existing contrastive learning framework, using deep-learning methods to study the problem of multivariate time-series classification. This approach is highly accurate, and the feature-importance rankings it generates are of considerable reference value.