Feature Selection and Classification

After characterizing the EEG data, either through real-valued or phase-based effective connectivity measures, we set up a subject-dependent classification system for the MI and WM databases.

For the MI data, we do the following: Since the MI database has training and testing datasets, we divide our classification system into a training-validation stage and a testing stage. For the training-validation stage, we first specify a cross-validation scheme of 10 iterations. For each iteration, 70% of the trials of the training dataset are randomly assigned to a training set and the remaining 30% to a validation set. Then, we use CKA (see Section 2.4) over the connectivity features obtained from the training set to generate a relevance vector ∈ [0, 1] *<sup>P</sup>*, where *P* equals the number of features in **Φ**. *P* varies according to the connectivity measure used to characterize the data. Then, we use to rank **Φ**. Next, we select a varying percentage of the ranked features, from 5% to 100% in 5% steps, and input them to the classification algorithm. The features associated with the highest values of are input first, and as the percentage of features increases those associated with lower values of are progressively included. In this work, we use a support vector classifier (SVC) with an RBF kernel [52]. All classification parameters, including the percentage of discriminant features, are tuned at this stage through a grid search. We select the parameters according to the classification accuracy, aiming to improve the system's performance. Then, for the testing stage, we train an SVC using the connectivity features from all trials in the training dataset as well as the parameters found in the previous stage. Lastly, we quantify the performance of the trained system in terms of the classification accuracy, obtained after predicting the MI task class labels of the testing dataset from its connectivity features.

The classification system we set up for the WM data closely resembles the one previously detailed for the MI data, with three changes. First, the WM database consists of one set of data for each subject, instead of two, so there is only a training-validation stage. Second, given the reduced number of trials available for each memory load level, each of the 10 iterations of the cross-validation scheme follows an 80–20% split for the training and validation sets (instead of a 70–30% split). Third, since the results provided by CKA are not stable for the low number of trials available from each subject (27.7 trials per class, on average), we opted to add an auxiliary cross-validation step, with the same characteristics as the one described above, and use it to estimate a single relevance vector **¯**, obtained as the average of the relevance vectors of each data split. Then, we use **¯** to perform feature selection in every iteration of the main cross-validation scheme.
